Measuring Social Value

(Illustration by Luke Best) 

Over the last few decades, many people have attempted to measure what is sometimes called social, public, or civic value—that is, the value that nongovernmental organizations (NGOs), social enterprises, social ventures, and social programs create.¹ The demand for these metrics has come from all sectors: Foundations want to direct their grants to the most effective programs; public officials, policymakers, and government budget offices have to account for their spending decisions; investors want hard data analogous to measures of profit; and nonprofits need to demonstrate their impact to funders, partners, and beneficiaries. Metrics to meet these needs have proliferated over the last 40 years, resulting in hundreds of competing methods for calculating social value.²

Despite the enthusiasm for metrics, few people actually use them to guide decisions. In the nonprofit sector, good managers are very rigorous about tracking costs and income. But few use sophisticated metrics to help allocate resources. Meanwhile, in the public sector, political judgment counts more than cost-benefit assessments. In the rare cases when decision makers do use metrics of social value, it’s far from clear that they should.

I’ve dealt with social value metrics in a variety of roles: as director of policy and strategy under United Kingdom Prime Minister Tony Blair; as director of the Young Foundation, an NGO that has created dozens of ventures, some for-profit, some social enterprises, and some public; and as an advisor to many other governments. In these positions, I’ve seen not only why social value metrics are ignored, but also how to make them more useful.

One recent project that proved particularly informative was a collaboration between the United Kingdom’s National Health Service (NHS) and the Young Foundation. The NHS commissioned the Young Foundation to develop a practical tool for assessing service innovations and guiding investment decisions. The NHS is a vast organization with a budget of around $150 billion, a workforce of some 1.2 million employees, and contracts with more than 30,000 social enterprises. It needed a set of tools that would be both robust and flexible, and that could be used for decision making as well as evaluation.

We started by scanning existing social value metrics, such as the ones described in the table “10 Ways to Measure Social Value” on page 41. We found hundreds of competing tools, of which foundations and NGOs generally use one set, governments another, and academics yet another. In addition to discovering this segmentation, our survey suggested two more reasons why so few metrics guide real decisions. First, most metrics assume that value is objective, and therefore discoverable through analysis. Yet as most modern economists now agree, value is not an objective fact. Instead, value emerges from the interaction of supply and demand, and ultimately reflects what people or organizations are willing to pay. Because so few of the tools reflect this, they are inevitably misaligned with an organization’s strategic and operational priorities.

The second reason that current measures of social value fail to influence decision makers is that they conflate three very different roles: accounting to external stakeholders, managing internal operations, and assessing societal impact. In the business sector, decision makers use different tools for each of these tasks. An airplane manufacturer, for instance, would use one set of metrics, mandated by laws and regulations, to explain to external stakeholders how it spends its money. The company would then use a second set of metrics to allocate resources in the building of airplanes. (It is a brave manager who would let investors see these internal accounts.) The company would then use entirely different kinds of measures to explain how its activities affect larger economic indicators such as gross domestic product.

Yet in the social and public sectors, some proponents of new social value measures claim that their metric can play all three roles. Not surprisingly, and despite courageous efforts, these attempts to do three things at once have resulted in the failure to do any one of them well.

Here, I describe a better way to think about social value: the product of the dynamic interaction between supply and demand in the evolution of markets for social value. I then show how decision makers in the nonprofit and public sectors can use these insights to measure what can be measured without pretending to measure what can’t be. Finally I recommend better ways to make social value metrics. My main advice is that nonprofits and foundations should resist the current trend of developing assessment tools entirely separately from public policy and academic social science, and instead should collaborate across sectors.

Elusive Quarry

The failure of the social and public sectors to measure the value they create does not stem from a paucity of intelligence or good intention. Rather, it reflects four unavoidable complexities that bedevil the measurement of social value. First among these is the lack of hard-and-fast laws and regularities in the social field. Many people would love the social field to be more like natural science, so that they could definitely predict the effects of, say, a $10 million investment in a crime prevention program.

But unlike molecules, which follow the rules of physics rather obediently, human beings have minds of their own, and are subject to many social, psychological, and environmental forces. Several decades of involvement in evidence-based policymaking has shown me that although evidence should inform all action, very few domains allow precise predictions about what causes will lead to what effects. The social sciences (including business) simply do not have laws in the way that physics has. Even seemingly solid economic principles, such as the rule that demand falls when prices rise, have many exceptions.

A second reason that measuring social value is hard is that, in many of the most important fields of social action—such as crime prevention, childcare, and schooling—people do not agree about what the desired outcome should be. In other words, the public argues not only about social value, but also about social values. For example, many people want to imprison criminals to punish them, even when incarceration costs more and confers fewer benefits than do alternatives to prison. Psychologists call this willingness to sacrifice a lot to penalize others altruistic punishment.

Because people’s ethics, morals, and priorities vary, social value assessments that look only at costs and benefits are bound not to influence many members of the public and the politicians who represent them. Philosophers (from John Dewey to Luc Boltanski) have long recognized that societies are made up of competing and conflicting systems of valuation and justification. But measurers of social value have often tried to deny this.

Even without these problems, many social value metrics are inherently unreliable. Measurements of social return on investment (SROI), for example, often quite arbitrarily estimate costs and paybacks, which dramatically affects the final calculated value. SROI calculations can help in broad-stroke predictions, but they can’t help with finer-grained decisions.

Revealed preference and stated preference methods are also notoriously unreliable. Although they try to provide precise numbers, they are not very rigorous about the means of deriving these numbers. As a result, these methods confuse rigor with precision—a point that REDF and others in the SROI field increasingly recognize.

A final reason that measuring social value is difficult is the problem of time—estimating how much good an action will bring about many years in the future, relative to how much it will cost to implement it now. In predictions of commercial returns on investment (ROI), businesspeople use discount rates to account for the assumption that a given amount of money will be worth less in the future than it is in the present. With a 5 percent discount rate, for example, $100 of today’s money will be worth only $35.85 in 30 years, and only $7.69 in 50 years. Many current measures of social value, such as SROI, likewise use commercial discount rates—perhaps because of a mistaken belief that treating social discount rates as equal to commercial ones will make social value metrics seem more rigorous.

But it’s not clear why social organizations and governments should use commercial discount rates, especially as these rates radically devalue the future. Indeed, we should hope that the people in these organizations give greater weight to the interests of future generations than do commercial markets. A closer analysis of discount rates suggests that they do.³ In health, many countries apply a very low or zero discount rate, on the grounds that younger generations should not be disadvantaged relative to older ones. Governments ignore discount rates when investing in education and defense technologies. And in climate change policy, a furious debate has raged about what discount rates to apply—again in part a moral argument about how to weigh the needs of future generations against the needs of current ones. These examples reflect my broader point: Social value is not an objective fact. Instead, it emerges from the interaction of supply and demand, and therefore may change across time, people, places, and situations.

Constructing Value

Borrowing practices from business and economics has led to many mistakes in the measurement of social value. Yet these fields still offer some important lessons for the field of social innovation. For much of human history, philosophers and economists believed that value was an objective fact. Aristotle thought that there was a “just price” for everything, for instance. And Karl Marx thought that value came from labor.

But more recently, most economists have accepted that the only meaningful concept of value is that it arises from the interaction of demand and supply in markets. In other words, something is valuable only if someone is willing to pay for it. This blunt approach upsets many people because it implies that there may be no economic value in a beautiful sunset, an endangered species, or a wonderful work of art. But this definition of value is useful because it forces economists to observe real behavior, rather than trying to uncover hidden realities.

The time is ripe for the social field to take an equally simple starting point, and to view social value as arising from the interplay of what I call effective demand and effective supply. Effective demand means that someone is willing to pay for a service or an outcome. That “someone” may be a public agency, a foundation, or individual citizens. Effective supply means that the service or outcome works, is affordable, and is implementable. I use the qualifier “effective” because social problems will always invite simple supply and simple demand. But to measure social value, the supplies and demands must be effective in the senses described above.

Markets, conversations, and negotiations then link, on the one hand, people and organizations with needs and resources, with, on the other hand, people and organizations with solutions and services. Social value metrics are useful if they give shape to these markets, conversations, and negotiations.

In some fields, the links between supply and demand are mature. For example, many voters are willing to pay taxes for police forces and primary schools, and many governments are able to supply these services. Likewise, many donors are willing to fund health care for children in developing countries, and many local charities and churches are able to deliver this care. In these domains, analyzing social value is not difficult, because the links between what funders want and what providers know they can offer is clear.

But for other social issues, the links between supply and demand are missing. In some cases, effective demand may be lacking because funders, politicians, or private citizens do not view a need as pressing enough to warrant their resources. For example, some states are unwilling to fund sex education or drug treatment. In other cases, effective demand may be present—for instance, many governments are willing to pay to reduce obesity—but the supply of cost-effective interventions is limited. In these situations any descriptions of social value are bound to be more tentative and exploratory.

In still other cases, both sides of the supply-demand equation may be murky or complex. Many social policy makers, for instance, understand that more holistic solutions often yield better results. But holistic approaches necessarily have to deal with purchasers—that is, demand—that are split across many different public agencies and NGOs, each with its own view of what really counts as valuable. The supply side may also be fragmented: Helping homeless people, for example, may depend on the contribution of many different agencies to provide therapy, alcohol treatment, job training, and housing. In these fields, too, social value can become clearer only through iterative processes that bring together supply and demand in deliberation and discussion. Even the most brilliant researcher cannot measure or even describe social value if she is not immersed in these discussions.

Healthy Numbers

All of these points have become particularly clear in our work for the NHS, which is involved in everything from routine checkups, to surgeries, to behavior-change interventions, to community programs. The advantage of a single, integrated health service like the NHS is that it has to be explicit about its demands, that is, what it needs and what it is willing to pay for. The NHS’s effective supply side is also reasonably easy to define, and it includes doctors, nurses, managers, social enterprises, private providers, and members of the public.

To help the NHS make better decisions and allocate its resources more effectively, we at the Young Foundation created a tool that makes explicit the social value of various alternatives. Earlier on I described the three very different roles metrics can play—external accountability, internal decision making, and assessment of broader social impact. The tool we developed focuses squarely on the second of these goals. It attempts to capture the value that accrues to the individual from being healthy, rather than sick; to caregivers; to the wider community (for example, from the control of infectious diseases); and to the taxpayer.

The tool we created is not a simple computer program or calculator. Instead, it is a framework for thinking about value. Like many of the tools used to assess social value, this one requires a series of judgments. The judgments fall into four main categories: 1) strategic fit (how well the proposed innovation meets the needs of the health service); 2) potential health outcomes (including likely impact on quality-adjusted life years and patient satisfaction); 3) cost savings and economic effects; and 4) risks associated with implementation.

When faced with a proposal, users of the tool apply a 0 to 5 scale to rate the proposal on items in each of these categories. Proposals range from a promising idea from a group of doctors or nurses, to an idea that has already been piloted on a small scale or a venture that is ready for scaling up. Users can also provide commentary along with their ratings.

In some cases, decision makers can draw on strong data—for example, evidence from a randomized controlled trial. In other cases, they must rely on less certain numbers. To capture this variability, the tool also includes measures of the reliability of the evidence on which judgments are based. The visual presentation of the results then makes judgments and their reliability very clear.

Once mastered, the NHS tool is quick to use and transparent. Multiple users can interrogate the judgments, and in due course compare them with what actually happened. It is also publicly available. Ten regional innovation funds (worth around $350 million in total) are using it as a basis for decisions and encouraging applicants to use the tool to assess themselves. Decision makers can also use the tool to review each other’s work, to ensure consistency in their decisions, and to communicate with other public agencies.

The net result of the NHS tool is a picture of social value that is explicit about what’s valued and what isn’t; that doesn’t try to combine everything into a single number; that is transparent and interrogatable; and that is simple enough to help decision makers having to cope with a large volume of examples. It also avoids the flaw of trying to impose a single discount rate onto diverse measurements.

Making Markets

I describe this tool because it is one approach to operationalizing social value that balances coherence, consistency, and simplicity with the flexibility needed to cope with messy and complex phenomena. Developing it was helped by the fact that the NHS is an organization with clear supplies and demands. But for most NGOs, supply and demand are fuzzier, and each field brings with it a different set of concerns.

For example, primary, secondary, and tertiary educational institutions create value for students and the wider society. They rely on a strong research base to decide which types of education deliver which returns to whom. Vocational education, in contrast, presents a different set of considerations. Certain kinds of skill may be of value to only one sector, or to a small set of employers. A program offering intensive support to a chaotic drug user will have a still more complicated set of values, including value for the individual (both financial and health-related), value for the community (for example, from lower crime), as well as value for a wide range of public agencies (from hospitals whose emergency services will be used less, to police, prisons, and welfare agencies).

Seen through this lens, the job of funders is not to alight on one particular method for measuring value. Although common frameworks for thinking about value are useful, funders must adapt these frameworks to the organization and field under consideration.

Indeed, the greatest contribution that funders can make is often not to measure value, but to forge the links between supply and demand that will later generate value. For example, they can invest in effective supply by supporting promising projects and collecting evidence of what works. They can invest in effective demand by persuading governments to use their much greater resources to pay for new services. And they can use their convening power to connect purchasers and providers and then encourage them to talk.

Foundations can also help less powerful players have a voice in the market. Many groups, such as homeless people, migrant workers, and people with mental illnesses, have clear needs but lack the resources and political power to translate their needs into demand. Foundations can turn this latent demand into effective demand. For instance, several European foundations that support undocumented migrants have developed the demand side of this emerging social market by encouraging larger NGOs and public authorities to allocate resources (for example, for housing and health care) to it. On the supply side, these foundations have funded promising projects that are more effective at meeting the needs of this group.

Some foundations are likewise developing the market for addressing elder abuse. On the demand side, they have funded research on the extent of the problem and influenced commissioners to allocate resources and attention to it. On the supply side, they have supported innovative programs to prevent or mitigate abuse.

In both cases, governments’ resources vastly outweigh those of foundations and NGOs. This is almost always the case. Just about anyone wanting to make a big social impact has to engage with the worlds of politics and public provision.

Good Accounts

The field of social innovation can learn some lessons from business and economics. But it should not be naive. As the collapses of Enron and Lehman Brothers revealed, even such seemingly objective metrics as profit are not the facts they appear to be in economics textbooks. And in business, accounts are just that: accounts. They are ways of explaining what is being done, with an eye toward the often conflicting interests of investors, managers, regulators, and consumers. They involve judgments as well as facts.

Anyone who wants to finance social goods and anyone who wants to provide them should use metrics to clarify how inputs can contribute to outcomes, as well as to clarify choices and trade-offs. But they should abandon metrics that obscure these choices or that pretend to offer a spurious objectivity. And they should use metrics only in proportionate ways. It’s not sensible for a small NGO to invest scarce resources in apparently elaborate estimates of social value—not least because these estimates are bound to crumble under serious scrutiny.

Meanwhile, larger NGOs that do need measures of social value should clearly distinguish between those that are primarily about external accountability, those that help internal management, and those that support assessments of broader patterns of social impact. If an organization is using the same method for all three, its findings are almost certainly flawed.

People involved in funding social value, whether at the stage of promising innovations or of large-scale practice, likewise need sharper common frameworks. Greater use of these shared frameworks would be more valuable than proliferation of ever more assessment tools. Building on these frameworks, what matters is the quality of the discussion and negotiation, and the depth of the learning several years later, when participants reflect on what worked and what didn’t.

Read more stories by Geoff Mulgan.