(Illustration by Patrick Fennessy)
Sir John Sinclair’s Statistical Account of Scotland, published between 1791 and 1799, marked the first use of the word statistics in the English language. It also launched the first national measurement system for “ascertaining the quantum of happiness enjoyed by its inhabitants, and the means of its future improvement.”
Few today would disagree that better measurement of societal outcomes can drive a more enlightened approach to statecraft. Governments are increasingly reorientating policy around well-being outcomes rather than indicators of economic growth. Public officials and social investors are more frequently conditioning funding on outcomes achieved rather than services delivered. The United Nations’ Sustainable Development Goals (SDGs)—17 commitments for a more peaceful, sustainable, and prosperous planet that UN member nations accepted in 2015—also use outcomes to advance and coordinate the worldwide response.
Despite such growth and intensive research, outcomes-based approaches to governance and social reform are poorly understood, and empirical evidence for their effectiveness remains disappointing. Where did Sinclair’s vision for social accounting go wrong? Two significant experiments with outcomes-based approaches, emerging more than 200 years after Sinclair’s innovation, may hold the answer.
Outcomes as Performance Management
The first experiment began with the onset of neoliberalism and its expression in public sector reform in the late 1980s and 1990s. The new public management (NPM), coined by government policy scholar Christopher Hood in a 1991 paper, sought to improve social services and programs by introducing market forces and private sector management techniques. Where direct privatization and marketization was infeasible or politically unpopular, such as with the United Kingdom’s National Health Service, NPM advocates applied market incentives by tying payments or sanctions to the achievement of quantified performance targets.
Early NPM reforms typically adopted service-level output measures (e.g., arrests made, patients examined, school class sizes) but increasingly came to focus on social outcomes (e.g., crime rates, mortality rates, educational attainment). Through appropriate outcome-based measurement and evaluation methods, policy makers could objectively determine the social impact of policies, services, and social programs, and hold accountable those involved in their contribution. More contemporary approaches, such as social impact bonds (outcome-based contracts involving private capital), impact investing (social investment predicated on demonstrable impact), and outcomes funds (financial pools for outcome-based commissioning), pave the way for a supercharged version of NPM, in which the state becomes a commissioner of outcomes rather than a provider of services.
Outcomes promise to set social interventions on an objective basis—winnowing initiatives that work from those that don’t and refining contracts to pay for outcomes “achieved” rather than services delivered. Advocates of outcome-based reform present this approach as a gift to the public, promising greater accountability and value for its money. But their agenda ultimately serves the interests of the powerful—politicians, business leaders, and wealthy philanthropists. By moving from service provider to outcome commissioner, public officials avoid the logistical headache of service delivery and the resultant blame for when things go wrong.
The prominent models that have developed from this first experiment—results-based accountability, payment-by-results contracting, pay-for-success, collective impact, social impact bonds, outcomes funds, and impact investing—underplay the complexities involved by relying on overly simplistic performance management mechanisms that have proved costly to implement and prone to backfiring. Instead of anticipated benefits, we find mounting evidence of a range of perverse effects.
Outcomes approaches encourage gaming of performance measures at the cost of genuine improvement; promote short-termism and risk-avoidance rather than innovation; and ultimately lead to the destruction of morale, collegiality, and adaptability in public institutions. Outcomes-based reform advocates might once have plausibly explained away these disappointments as aberrations or misapplications, but by now they can only be read as signs of a failed model.
Outcomes as Collaborative Governance
But interested practitioners have other options. In fact, a second experiment is already well underway that instead uses outcomes to drive long-term collaboration toward shared goals. National governments such as Scotland, Wales, New Zealand, and Canada have been applying this approach to reconfigure their purpose and functions around societal well-being. Cross-sector civic partnerships like the Western Australia Alliance to End Homelessness use outcomes frameworks to help coordinate large-scale campaigns for social justice. The SDGs and the international well-being movement operate across nation-states, using outcomes to energize collaborative stewardship of grand societal challenges.
These initiatives all proceed without hierarchical control or direct results-based accountabilities and carry no clear blueprints for action or implementation plans. They instead use outcomes as organizing instruments to rally together broad coalitions toward ambitious shared goals. They also apply measurement differently, using it to structure a long-term collaborative endeavor rather than to control performance. Architects of outcomes frameworks focus on performance attraction, not performance management; they seek to articulate a shared vision that is compelling and accessible enough to motivate cooperation and foster collaborative innovation among multiple parties over the long time periods necessary to bring about social change.
Compared with the NPM experiment, this alternative offers a number of benefits. First, no central authority is required. Rather than rewards and sanctions, outcomes frameworks rely on participants (businesses, governments, nonprofits) to buy in and adopt new goals and indicators. The construction of outcomes frameworks needs to be participatory and inclusive to ensure that they express a collective mission in which other parties actively want to join.
Second, performance systems are not paralyzed by imperfect data. Without the need to attach direct accountabilities for results, outcomes-focused collaborations can pick a direction and build the plane as they fly. As data deficiencies are encountered or priorities change, measurement systems can be adjusted—imperfect data still serves to rally partners around shared causes and can still inform decision-making.
Third, outcomes frameworks need not invalidate or squeeze out local priorities. Outcomes are broad enough to contain different approaches and interpretations, and they can be adapted to local contexts. Rather than overriding local priorities, outcomes become a basis for approaching these priorities systematically.
To be sure, this alternative approach offers no panacea. Critics would argue that outcomes approaches are toothless when decoupled from strong results-linked incentives. But participants in outcomes-focused collaborations often want stronger models of accountability: Peer pressure, shame, reputational damage, and forms of soft power play a critical role in motivating participation and ensuring that missions are shaped by shared values. Instead of abandoning accountability, this model extends it upward and outward to a broader range of actors: Advocacy groups, civil society coalitions, and the media can all apply shared values and public accountability to promote change.
Relatedly, critics claim that outcomes can prompt lip service without inspiring genuine change, remaining benign artifacts or becoming corporate wallpaper. Companies and governments have been accused of “SDG washing”—adopting the SDGs in name without taking meaningful action. Outcomes and related performance indicators must influence what organizations do and alter how they think: Goals must shape budgeting and strategic plans, and performance indicators must help them orient their activities and guide their contribution to a shared mission. New Zealand’s well-being budget, which prioritizes broad indicators of human welfare over GDP and other, narrower economic indicators, links government budgeting to priorities for well-being, such as physical and mental health, meaningful employment, and equity for ethnic minorities. The government collects data that helps it anticipate threats to well-being and form a strategic response.
Critics may also worry that undertaking a more inclusive and participatory approach to outcomes frameworks risks capture by special interest groups. But this perspective ignores the innately political nature of setting performance indicators. Incorporating diverse perspectives in the construction and implementation of outcomes frameworks is essential to forming genuinely shared missions and navigating interdependencies in the pursuit of complex goals. The inclusion of diverse and even divergent perspectives leads not to capture but to an overdue rebalancing of power relationships. We see this, for example, in the way the SDGs broadened consultation with Global South countries vis-à-vis the Millennium Development Goals, thereby moving beyond technical fixes to focus on more fundamental structural issues.
As the outcomes agenda gathers pace internationally, it approaches a fork in the road. Do we continue to view outcomes as a technocratic apparatus for service commodification, or do we instead see them as democratic organizing instruments of a purpose-driven model for collaborative governance? Outcomes can be a mechanism to control others’ behavior toward a unilateral goal, or they can promote an inclusive and dialogic engagement with multiple valid interpretations of goals. They can drive further hollowing out of the state or become a reassertion of its leadership role in tackling social problems at their roots. They can mark a retreat from the complexity of governance or prompt meaningful engagement with it.
Those with interest in the latter model can access an alternative arsenal for an outcomes-based approach that involves codesigning outcome frameworks to reflect shared values, constructing platforms for collaboration rather than blueprints for implementation, and mediating accountability through softer controls such as peer pressure and public opinion. Only in this way can outcomes measurement, as Sir John Sinclair hoped, “hold the means of future improvement.”
Read more stories by Max French.