(Photo by iStock/mohd izzuan)
Any organization trying to improve people’s lives has to answer two questions, one about experience and the other about outcomes. Are we serving people well? And are their lives actually getting better? The organizations that create the most lasting social change treat them as one practice rather than a choice between two.
It’s easy to lose sight of the difference between measurement sophistication and organizational seriousness about that measure. They are not the same thing. A sophisticated study that no one acts on is not serious. A simple one that changes a decision is.
With the rise of customer “listening” in social change organizations, including wider use of mainstream metrics like customer satisfaction scores and Net Promoter Score®, a hierarchy has quietly formed between measuring outcomes and listening. Because the hierarchy ranks methods by sophistication—or privileges “objectivity” over “subjectivity”—it mistakes that for seriousness. Thus, while randomized trials and quasi-experimental designs sit at the top, customer listening is seen as a “soft” option, at best; at worst, it is viewed as impact washing.
Such a hierarchy misses what listening actually does. Listening creates an orientation. This orientation, along with the organizational response it facilitates, is highly valuable in itself, and both serve as essential inputs for organizations and companies looking to create social change. A study’s worth depends on more than its design: the best study in the world is meaningless if it is not feasible, fundable, and acted upon, if the results cannot be compared to other similar studies, or if it is immediately filed away and never shared. A listening orientation helps make measuring worth doing in the first place.
What Is ‘Objective’ Data?
So much of what we treat as objective and authoritative data in this field, strictly speaking, is built on a subjective foundation. After all, most widely accepted, research-based global metrics—including, for example, the World Bank's numbers on national poverty levels—are created by talking to people and getting their answers to a standard set of questions. If we rule out subjective data, we cannot ask a farmer about her planting methods and what information sources she uses, we cannot ask a head of household how she spent her additional income, we cannot ask someone about their sense of agency or confidence in managing their financial life. All of these measures that the sector relies on every day are subjective in exactly the same sense.
The more useful questions are not whether data is subjective, but:
1. Whether respondents have reason to misreport,
2. Whether the survey instrument was designed to minimize response bias, and
3. Whether people’s subjective take on their own lives is inherently valuable, even in cases where perception differs from reality. (We believe it is).
Because subjective data is the lived experience of the people we are trying to serve, it should be taken at face value unless there is a specific reason to doubt it. And if you have gone to the trouble of asking, you owe it to people to act on what they tell you.
A Happy Customer Might Not Be Better Off
This is not to say, of course, that subjective measures of satisfaction are the same as social impact. A satisfied customer might be happy but may not be meaningfully better off. The two are likely correlated—if nothing else, poor satisfaction is unlikely to coexist with high social impact—but it would be incorrect to claim social impact based on satisfaction.
Indeed, there are many cases where a satisfied customer is clearly not better off —casinos and online betting being the obvious examples. More relevant to the social sector are examples like financial products that customers like but that doesn’t improve their financial resilience, or training programs farmers value for the relationship they build but that doesn’t change what they do on their farms.
The productive distinction is not subjective versus objective, but between customer experience and customer outcomes. Both are measurable, both matter, and conflating them tends to leave organizations with an incomplete picture of whether their work is achieving what they set out to do. But listening well underlies the process of making that measurement, as well as acting on it.
Operators Who Do This Well
For organizations focused on generating social impact for the people they serve, customer experience measures tend to function as a necessary first step rather than a final answer. A product that breaks, one with terrible customer service, or one that customers simply don’t want to use is very unlikely to become a meaningful part of their lives and therefore less likely to generate a positive impact.
However, the best operators go further than avoiding those failure modes, toward a deep customer-centric orientation, one of eagerly, actively, nearly constantly listening, to understand the experience of the product itself, the needs customers have, and how well those needs are actually being met. That quality of listening is already doing more than satisfaction measurement does. It is a genuine attempt to understand what is working in people’s lives and what is not.
A consumer brand can reasonably define success as customers liking the product and buying it again. But socially minded organizations must start there and keep going, towards the question of whether the product is genuinely improving lives. That requires going beyond experience to outcomes, as well as a willingness to keep asking even when the experience data looks good.
What we are arguing for is rejecting a hierarchy between customer experiences and social outcomes. We need both at once. Customer listening around satisfaction and rigorous outcomes measurement. They reinforce one another rather than compete, as they are both grounded in the same orientation: sustained engagement with the people being served, and treating data as something to act on in doing it better. Both require humility about whether what holds for one group of customers using one product will hold for a different group, or at a larger scale. What differs is the depth of the question.
What This Looks Like in Practice
Root Capital finances agricultural businesses, cooperatives, processors and exporters that source from and support smallholder farmers across Latin America, Africa and Asia. While interventions are made at the level of the agricultural small- and medium-sized enterprise (agri-SME), the desired impact is at the farmer level. The measurement challenge is layered, both understanding the experience and outcomes of the businesses Root Capital finances and the experience and outcomes of the farmers connected to those businesses.
That layered challenge prompted the development, in partnership with 60 Decibels, of a Climate Resilience Assessment built around three measures: perceived resilience, realized resilience, and resources and enablers:
- Perceived resilience is what farmers and business leaders report about their own stability and capacity to absorb shocks.
- Realized resilience is what measurably happened to businesses and households when shocks occurred.
- Resources and enablers track access to the specific capabilities that drive business and household resilience over time.
The assessment takes farmers’ own account of their resilience seriously in its own right, and sets it alongside what measurably happened to their households. The two answer different questions, and a full picture needs both.
The study these measures fed into was deliberately rigorous: a global, multi-year, quasi-experimental evaluation spanning 30 agri-SMEs and over 2,000 farmers across six countries, using a fixed-effects difference-in-differences model. It compared businesses and their farmer networks that received Root Capital’s climate loans against businesses that were pre-qualified but did not ultimately receive them, the comparison that lets the analysis isolate the effect of the loans themselves.
Perceived and realized resilience are the relationship to watch: When the two align, there is evidence that a program is working in ways customers can feel. When they diverge, there is a signal worth investigating. A system that cannot detect that divergence is answering a narrower question than it appears to be.
In this study, they aligned. Agri-SMEs that accessed Root Capital’s climate loans showed a 23 percent improvement in their Climate Resilience Index scores relative to the comparison group, driven primarily by the capacities and enablers component, with impact cascading to the farmer level through the realized resilience component. The full findings are available here.
Root Capital has also seen the other side of this directly. Since 2019 it has run a Digital Extension Advisory Service in Rwanda, where repeated assessments found it improved coffee farmers’ knowledge and adoption of good agricultural practices. When the program was extended to macadamia farmers in Kenya in 2024, Root Capital was interested in whether the program’s success would transfer to a new context where the aggregator-to-farmer relationship was quite different. Perception surveys suggested farmers valued the service and would recommend it. An embedded randomized controlled trial, as well as a difference-in-differences analysis alongside it, found no significant improvement in their knowledge of the practices the program was designed to convey. The two kinds of measurement disagreed; this disagreement was the finding.
Two caveats are worth noting in this case. Only 6 percent of the 6,000+ registered farmers recalled receiving materials, and only 9 percent answered the final survey, so the satisfaction signal may reflect the most engaged users rather than the whole group.
The findings informed a decision to discontinue the program in that context, part of a trend toward increasing the granularity with which Root Capital targets services to different client segments and value chains, in support of differentiated impact objectives by segment.
The CRA study didn’t so much confirm a prior as it changed decisions. Informed by the findings, Root Capital is implementing a series of strategic and programmatic shifts: evolving its loan products and more tightly coupling them with Advisory Services to make them more accessible to agri-SMEs, and launching Root Ventures, a risk-buffered venture debt facility designed to address the growing climate finance gap for smallholder agriculture by scaling climate loans.
These are structural decisions that require confidence in the underlying evidence, and that confidence came from a measurement system designed to understand not just how customers experienced the product, but how it was changing their lives.
For the Sector
The rise of customer listening in social change organizations is a welcome development. Organizations that treat the people they serve as customers with valuable input tend to design products that work better and to hear more quickly when they don’t. But this shift only reaches its full potential when there is clarity about what different measures can and cannot tell us.
Satisfaction and NPS provide important information about customer experience, and no organization should ignore these insights. But in our experience, they work best as the start of a measurement conversation rather than the end of one. The organizations we have seen create the most sustained social benefit tend to stay curious beyond the satisfaction score, asking not just whether customers valued the product but whether it is genuinely improving their lives and why, and using this information as input into a constant iteration-and-improvement cycle. That requires rigorously gathered data about outcomes as well as experience, and a willingness to share findings even when they are not what was hoped for.
For organizations that draw on philanthropy or subsidy, this is not an academic distinction. Outcome evidence is what justifies the capital in the first place, but a satisfied customer is a necessary condition for impact, not a sufficient one. How organizations handle the gap between those two things is, in our experience, one of the more consequential measurement questions in the sector.
Read more stories by Sasha Dichter & Kate Labeille.