We have all been through it at some point in our academic lives. Palpable tension, and just a sprinkle [or five] of panic when the exam invigilator barks “15 minutes left!” at the class. You thought you had more time; you thought you had all the answers. So you change strategy. You pick ‘A’ for all remaining multiple choice questions — hoping there’s bound to be a correct one — and frantically write a sentence or two for the essay questions. The lecturer will certainly wonder whether the crisp penmanship at the beginning of the paper belongs to the scribbler at the end.
How we’re refining research methods for higher quality data
This is certainly not the experience we’d want respondents to have where surveys are concerned. Much of social science research relies on self-reported data, collected through surveys, whose outcomes could possibly influence policy. Therefore, respondents should feel at ease to share their true thoughts and experiences, which are critical to retrieving high quality data for analysis. This is the aim of the Research Methods agenda at Busara; to ensure that quality is at the core of data collection in the Global South.
To do so, we will try to answer questions like:
- What is an ideal length for a self-administered survey? The goal is that the respondents handwriting should not drastically shift from neat to barely legible.
- Are the respondents actually paying attention to our surveys? It is no good for us if the respondent “selected ‘A’ throughout”, simply to get to the end.
- Is the information shared reflective of what is true to the respondent, not what they think the researcher wants to hear? In social science research, we would call that the social desirability or experimenter demand bias.
Beyond survey design, we can also think of other structural biases that could come in the way of collecting good quality and representative data. For instance this The Mobile Gender Gap Report 2021 states that “Female mobile users feel less able than male users to learn a new activity on a phone by themselves”. We need to then think more carefully of the implications of using novel and digital methods of data collection. The report also states that women in low- and middle-income countries are 15 per cent less likely to use the internet than men. With the proliferation of online remote surveys, are we perpetuating a gap in who we can reach?
More formally, between 2022 and 2025, we hope that through our research on methods we can comprehensively answer these core questions:
- How can we contextualize, measure and improve access, response, attention, comprehension and depth of responses in surveys, experiments and qualitative research for populations in the Global South?
- How can we quantify data quality in both qualitative and quantitative research?
- How do the answers to these questions vary across gender, racial, national and economic groups? How can we build inclusion across different modes of research, for those populations with the least social power, and make certain these modes do not perpetuate exclusion?
- How do the answers to these questions vary across common research methods, including in-person and remote data collection?
- How do the answers differ for the most sensitive and difficult to approach research topics?
- What protocols and practices should we adopt to maintain high data quality across methods and groups?
Busara operates both remote and in-person data collection, the former through methods including phone surveys, SMS, online experiments and self-administered surveys on KITE, Busara’s own tool for collecting data. We will test different methods of measuring and improving data quality, running two experimental studies per quarter to continually refine and improve across our many objectives. Initially participants will be drawn from the Busara Lab Pool, which is primarily composed of a low-income population in Nairobi. With time, the aim is to expand to other segments of the population across the countries that Busara serves (including Ethiopia, India, Nigeria, , Tanzania, and Uganda).
And that is the Busara methods research agenda, bringing certainty in our findings. If you have made it this far in the the blog, we’re glad to still have your attention. Did you notice the repeated ‘the’ in the last sentence? If you did, then you are really paying attention. We hope you’ll continue to do so, over the next three years. For more information on the methods research agenda, please refer to the Blue Paper here.
