In “The Payoff of Pay-for-Success,” V. Kasturi Rangan and Lisa A. Chase raise important questions about the long-term trajectory of the Pay-For-Success (PFS) movement in the United States. Will only the most sophisticated organizations with the ability to collect and analyze data on the impacts of their programs be able to be funded under this new approach, they ask? If so, what will happen to organizations that lack this capacity but nevertheless serve important populations in need? Rangan and Chase are also concerned that the range of interventions for which PFS may be successful as a funding vehicle may be narrow given the challenges involved in demonstrating that a particular intervention leads to cost savings in other programs.

We have a related question about the scalability of PFS: To what extent will the PFS field be held back by the significant deficit of knowledge about which interventions really work? In our view, the expanded use of PFS financing is contingent on having a robust ‘pipeline’ of interventions with a strong level of evidence, and having a commitment to expand organizations’ capacity to conduct strong experimental or quasi-experimental impact evaluations. Some good work is already underway in both areas, but more is needed to build this pipeline and capacity.

Through its Top Tier Initiative, the Coalition for Evidence-Based Policy, a nonpartisan nonprofit organization focused on promoting effective social policy research, has identified only 10 social programs that have top-tier evidence of their effectiveness and another 12 that have evidence that is near (but not at) the top tier.¹ This is obviously a very small number of proven interventions with which to work. Several other promising interventions are currently being tested with high-quality evaluations through the federally funded tiered evidence programs (such as Investing in Innovation and the Social Innovation Fund). While the tiered evidence programs are making important progress in building the evidence base in different fields, given the large number of social interventions underway in the United States and around the world, we need to continue to focus on building the pipeline of interventions with strong evidence. As a field, we have underinvested in building knowledge about which social interventions are most effective. Is it any wonder then that the early PFS transactions have clustered around a small handful of programmatic strategies?

Before turning to our prescription for addressing this challenge, we must confess to being deeply moved by the constructive energy unleashed by the PFS movement. All across the United States nonprofit organizations and local and state governments are focusing serious attention on measuring the performance of social programs and documenting real-world impacts. This is, in and of itself, a real achievement! PFS has also provided a welcomed opportunity to begin to bridge programmatic silos. With government funding split across numerous subject-specific agencies, it has been difficult to fund an intervention in one field (such as providing supportive housing for homeless people with severe mental illness) that generates benefits or savings in other fields (such as reduced use of prisons and hospitals and fewer days in homeless shelters). PFS provides a vehicle for facilitating these types of cross-silo initiatives, which should lead to increased efficiency and better outcomes for vulnerable households.

To take full advantage of the potential of PFS to scale up promising interventions and generate broad-reaching impacts, however, we will need to dramatically expand the base of rigorous evidence about which programs are both effective in producing socially desired outcomes and likely to result in substantial savings to other government programs. This will require a substantial commitment on the part of funders (especially the philanthropic sector), policy-makers, nonprofit organizations, and third-party evaluators. Not every program that is rigorously evaluated will prove to be effective and efficient, but many will, generating a much larger base of programs for policymakers and investors to consider in developing new PFS projects. As an important side benefit, programs that fail to demonstrate a substantial impact will get important feedback about the need to adjust their approach to increase their effectiveness.

A major investment in building evidence about what works would help to create a robust pipeline for future PFS transactions utilizing interventions with a strong level of empirical evidence about their effectiveness in achieving both their social goals and savings in other government programs. This investment would also help to broaden the range of organizations and interventions able to be funded through PFS, addressing several of the key concerns raised by Rangan and Chase.

Read the rest of the responses.