8.05.2 Questionnaires
The most frequently used methodology to assess key constructs in health psychology is questionnaire assessment. Typically, questionnaires consist of closed-ended questions, which are answered using a Likert scale. Thereby, they aim to provide insight into a person's psychological state (e.g., level of anxiety), health behavior (e.g., frequency of exercising), or underlying mechanisms (e.g., attention vigilance). The use of questionnaires is highly popular due to its ease of administration (both using paper and pencil or online), which can be achieved without the presence of an assessor and at a very low cost. Online assessment has further increased its popularity. Persons can fill out questionnaires online at home, and summary scores can be immediately provided showing how well/poor a person is doing in a certain health domain. Online assessment has furthermore been facilitated by the availability of easy-to-use open source survey apps (e.g., LimeSurvey at https://www.limesurvey.org/, Redcap at https://www.project-redcap.org/software/, Qualtrics at https://www.qualtrics.com/). These survey apps allow researchers to administer questionnaires to a large number of people without much effort. Yet, despite the many advantages, assessing health outcomes via questionnaires is not without risks. The use of questionnaires has limitations, and answers can be systematically distorted by response bias, an individual's tendency to respond inaccurately or incorrectly to a question.
The best known response bias is the social desirability bias, or the tendency to answer questions in a manner that will be viewed favorably by others. Explicitly asking individuals to answer questions may result in the over-reporting of “good” behaviors, beliefs, or attitudes, and the under-reporting of “bad” behaviors, beliefs, or attitudes. To overcome the presence of a social desirability bias, psychologists have developed implicit measures that do not require reflection and introspection and reduce the ability of participants to control their answers. These measures are assumed to be less sensitive to social-desirability and positive self-presentation and, hence, to have better validity. Examples are the implicit association test (IAT), affective priming tasks, the go/no-go association test, and the implicit relational assessment procedure (see Gawronski and De Houwer, 2014). This is an active area of research, also in health psychology (Sheeran et al., 2016). However, as yet, in most situations self-report measures outperform implicit measures in terms of reliability and validity (Meissner et al., 2019). Another strategy to address social desirability bias can be achieved by overcoming the conditions that facilitate social desirability and positive self-representation. Indeed, social desirability bias occurs when individuals avoid judgment by others and feelings of shame. Therefore, having an empathic and non-judgmental approach, which also assures confidentiality and trust, may be key factors to reduce social desirability bias. Likely, this is easier to achieve in an applied setting with health-care providers than in a national health and/or illness survey performed by a private company. All in all, social-desirability bias reminds us that self-report is always an act of communication in a relational context. This is no different for self-report questionnaires.
A second type of response bias relates to one's tendency to agree with questionnaire items, without considering the specific content of the question (Messick, 1967). This acquiescence bias may be more present in some cultural subgroups (Rammstedt et al., 2017). It is most prominent when respondents are asked to confirm a statement or if the question is answered with opposite answer options (e.g., “agree/disagree”; Kuru and Pasek, 2016). Several techniques have been proposed to overcome acquiescence biases, such as the use of balanced scales (Cloud and Vaughan, 1970), item-specific questions (Höhne et al., 2018), and statistical correctives (Kuru and Pasek, 2016). Although these techniques may reduce the bias, they may also complicate assessment (Kuru and Pasek, 2016). Furthermore, they can reduce user-friendliness, or bring along new problems (e.g., decreased content validity when using reverse-scored items).
Third, most often questionnaire items ask people to reflect over a long time window (e.g., over the last 2 weeks, how much pain did you experience; over the past 3 months, how much has pain interfered with your life activities?) or do not specify a time window at all, allowing memory processes to play a role. This is particularly true when people report on experiences that fluctuate highly over time and contexts (e.g., emotions, bodily symptoms). Recall bias has been well-investigated in the context of pain, whereby it has been suggested that recall of pain is disproportionately affected by the most recent and the highest pain levels within the recall period (i.e., peak-end effect; Kahneman et al., 1993). In addition, research suggests that people who recall pain tend to overestimate their symptom severity (Broderick et al., 2008) and indicate that the association between retrospective data and daily recall is only modest (Stone et al., 2005). Recall bias is, however, not unique to pain. Topp et al. (2019) investigated recall bias in the measurement of health-related quality of life and found that recall bias (report of past 4 weeks) was considerable on the individual patient level and could impact upon decision-making in clinical practice. Generally, the length of the recall period is inversely related to the accuracy of recall (Broderick et al., 2010; Stull et al., 2009). Yet, shorter recall periods can lead to the under-reporting of symptoms in some conditions (Norquist et al., 2011). As such, it has been suggested that the recall period of questionnaire items should be well-considered and take into account respondent burden and their ability to easily and accurately recall the information, the attributes of the construct of interest (e.g., variability over time), and the needs of the administering clinicians/researchers (Batterham et al., 2017a,b).
Finally, questionnaire items are frequently interpretable in multiple ways and unclear to respondents. This increases the risk of obtaining different interpretations for the same question by different people. The latter issue demands careful attention while developing questionnaire items to assess health constructs (see also Content Validity section) The presence of an administrator while completing the questionnaire might help, as it allows to explain unclear items to people. Yet, explaining the meaning of every question is cumbersome and is only possible for a limited number of items. The presence of an examiner itself may also impact upon the answers of a person on a questionnaire, as a responder may not feel comfortable selecting extreme or unconventional choices.