Imagine waking up one morning with a burning desire to bring about social change in the developing world (for some of us, all over again). How many ways can you think of doing this? Would you want to be a part of designing policies, starting an NGO, funding one, or building a team that works on the last mile in this long channel of stakeholders? There are many possible pathways and much to do because social change is challenging.
When it happens naturally, social change usually occurs over long timeframes. Rarely is there a single identifiable moment of change that can help divide and classify shifting social dynamics. It is also challenging to measure social change at any given point in time as it is so dynamic and complex.
Yet, in applied social development our job is to do exactly that — effect, or at least find, evidence-based ways to support social change. But social change is a complex untameable beast, Its tread is untraceable, except perhaps in retrospect (with some healthy doses of hindsight bias.
From the low-hanging fruit to long-term social change
The complexities begin right from the start. Take an example: let’s say the problem you want to tackle lies at the heart of the financial inclusion of women in India. The following is the first result you read when you look for statistics around the topic. “The World Bank shows, 76.6% of women in India were ‘financially included’, i.e., were registered within the formal banking channels.” How would you define making an impact in this context? Does it mean reaching close to 100% of women to be financially included? Does it mean deepening the inclusion of the 77% of women by translating their inclusion into increased financial knowledge or activity? Who decides where the priority lies? And most importantly, where do these women lie in the long list of stakeholders? What does successful financial inclusion look like for them? We need measurements, but we might need to broaden our lens to find contextually appropriate ways to define success.
Defining metrics
Most social development endeavors are connected and influenced by stakeholders ranging from the funder to the grassroots organization that implements a program. In between gather a host of specialized organizations like subject matter experts, government institutions, or academic researchers. In such a complex mix of stakeholders, it can be difficult to gain a clear consensus on metrics of change and, with that, of success.
Further, broad-scoped institutions like funders have more say in defining metrics as they have to pull off the balancing act between governance interests, impact on end-beneficiaries as well as managing present and future relationships with key stakeholders. In a way, funders often naturally have to have a narrower lens of what success is to keep their business going. In this process of finding the right balance among a range of stakeholders, there lie varied metrics that can help define what success means to each one on the entire spectrum.
At Busara, as in general applied behavioral science across the world, project metrics have varied from experiment-based measures to qualitatively-generated insights that describe and contextualize a problem. Often research objectives need to be narrowed to simplistic metrics that are verifiable under the timelines of short-term projects to draw out relevant conclusions.
Clearly, success will depend on how these stakeholders define the finish line as well as the waypoints. This creates an interesting dilemma. As change in the development sector deals with deeply entrenched social fabric and is rarely one-shot, development professionals are tasked with reducing these complex mechanisms into measurable metrics. Applied behavioral researchers usually come up with solutions rooted in experimental economics and behavioral science, for example, experiments to test hypotheses, qualitative deep dives to understand barriers and enablers, and even design workshops to create inclusive solutions. All of these, especially when iterated (as outlined by Ashok in the beScIGHTS Magazine) are successful to a certain extent in solving a bite-sized abstraction of a problem. But when applied behavioral researchers then put on the wide-angle lens to zoom out and think about long-term social change and end-benefactors, they do not automatically arrive at the bigger picture of what the steps of success towards that broader change are.
Yet, this does not devalue applied behavioral science as a toolbox. The global trend towards using more behavioral insights is helping to fill the right gaps in research methodologies to strengthen social development initiatives (even if the research that draws on data collected in the Global South is extremely limited) Henrich, J. (2020). Researchers need to be aware of the pros and cons of their chosen methodological pathways that help deconstruct overall research goals into bite-sized targets (and with that, likely bite-sized notions of success).
Read more of this essay in BeSciGHTS magazine — an explorative look at 3 years of applied behavioral science in India. Download it here.
