Methodology

The following section provides a brief overview of the methods used to develop and implement the 2020 Wellcome Global Monitor. For a complete discussion of the research methodology, see the Methodology report. [PDF 140KB]

The 2020 Wellcome Global Monitor questionnaire was developed using a careful research and design process that included:

• interviews with leading researchers and senior Wellcome stakeholders
• cognitive testing in ten countries to ensure questions could be understood across countries and by various demographic groups
• pilot tests in 10 countries

The questionnaire was then translated into the major conversational languages of each country and checked by an independent third party for quality assurance.

The COVID-19 pandemic meant that significant changes were required to Gallup’s methods of using face-to-face and phone surveys for global data collection, resulting in all 2020 Wellcome Global Monitor interviews being conducted entirely via telephone in only 113 countries rather than in over 140, as in 2018 (in 34 countries by phone and in 110 face to face). 

The samples from each country are nationally representative of the resident population aged 15 and older with access to a phone (either landline or mobile); however, the inability to conduct in-person interviews reduced population coverage in many low-income countries (see the ‘Research limitations’ box).

Gallup used available demographic information from each country to calculate a set of weights for each respondent, helping ensure that the overall sample reflected subgroups in a population. Gallup made weighting adjustments based on gender, age and (where reliable data were available) education or socioeconomic status. In many countries where interviewing was conducted via telephone for the first time, additional demographic factors such as employment status, urban compared with rural residence and region were included to help account for the inability to reach people without access to a phone.

Gallup then calculated a margin of sampling error and study design effects for each country to account for the influence of data weighting. For more information on design effects and sampling error and to see a list of the relevant figures, please see the ‘Country dataset details’ box in the Methodology report.

The Gallup World Poll has been run by Gallup survey specialists and statisticians in collaboration with local in-country partners for over 15 years. This structure ensures that the same survey approach and methodology are applied consistently in all countries, with training provided by Gallup Regional Directors to ensure that local partners apply best-practice survey methodology.

Even so, certain methodological choices sometimes impact country-level results. Gallup has found that applying a consistent methodology across all countries reduces this impact, and testing shows that the survey data and results are robust and reliable. Nevertheless, it is important to highlight some methodological choices that could impact the national representativeness of the sample and the interpretation of the data. (More detail can be found in the Methodology report).

The main limitations of the Wellcome Global Monitor are:

• The survey asked people about anxiety and depression, defined as being so extreme that they impacted their ability to function as they normally would for two weeks or more – a definition drawn from the World Health Organization (WHO). However, no more detail was offered about what was meant by anxiety or depression, which may have implications for the interpretation of the results.
• This survey is not an epidemiological study, and the focus is not on mental health disorders but on people’s self-reported feelings of anxiety or depression as defined in the survey. Additionally, some people may have felt uncomfortable saying they had personally experienced anxiety or depression and therefore did not give an answer.
• Decisions relating to the exclusion of certain regions in some countries due to factors such as political unrest, conflict or remoteness.
• Decisions relating to which languages were used to field the survey in countries with tens, hundreds or even thousands of languages. To mitigate this effect, Gallup generally surveys in languages that are spoken by 5% or more of the population in each country. This decision could also be associated with specific social norms and interpretations of certain words among some groups and may sometimes be relevant for the translation of the terms ‘anxiety’ and ‘depression’.
• The decision to survey people aged 15 years or older and the exclusion of institutionalised segments of the population.
• It is also important to note that lack of random assignment and independent assessment of outcomes means we cannot draw conclusions about the efficacy of different approaches for managing anxiety or depression.

In 2020, COVID-19 and the associated transition from face-to-face to phone surveys for approximately 82 countries complicated data collection. As a result, the 2020 Wellcome Global Monitor included fewer countries than previous waves (113 rather than 144 countries). The exclusive reliance on phone interviews may also have skewed responses in some lower middle-income countries towards urban and higher income residents.

Although for most of the countries in which face-to-face interviews were previously used but in which telephone interviews had to be used in this study, the coverage error is not expected to be greater than 10%, Gallup estimates that the size of the coverage error – i.e., the percentage of a target population who could not be reached – may be higher than 20% in a small number of countries such as Ethiopia, Venezuela and Zambia.

To help adjust for these coverage deficits, Gallup used an expanded set of demographic variables when calculating weights for the data collected in these countries. Please see the Methodology report for further details.

Readers should be aware of these limitations when drawing conclusions, particularly with regard to cross-national comparisons.