How Impact Investors Actually Measure Impact

Those of us who study impact investing all can agree that measuring an investment’s social effect is important, that measurement can help organizations make better decisions and communicate their value, and that financial returns should be balanced with social returns. But most of these points of agreement remain theoretical: Few resources discuss the specific practices and methodologies that investors actually use to measure social impact. This is what we set out to do in our study, Measuring Impact in Impact Investing. 

For this independent research project at the Harvard Business school, we interviewed more than 20 leading impact investors and practitioners in related fields, from organizations such as Acumen, Bridges Ventures, and Root Capital. Through these interviews, we learned that investors use impact measurements for different objectives in different parts of the investment cycle, and that methods for measuring impact vary based on the objective.  

These objectives fall into four main phases. Investors first work on estimating impact, conducting due diligence to assess the potential social return before committing to an investment. Then comes planning impact, choosing the metrics and data collection methods that the investor will use to monitor a program’s effects. Once the program is underway, investors and investees focus on monitoring impact, measuring and analyzing impact throughout the life of the investment to track the intervention’s effects. And finally, sometimes investors turn to evaluating impact, measuring an investment’s social consequences after the program concludes to assess portfolio performance and next steps for the investor, including re-investment. 

Investors have different measurement objectives in different phases of the investment cycle.

It was no surprise that impact investors have developed a range of different measurement methods to accomplish these objectives. What did surprise us, however, was that we didn’t find any previous analysis of how these methods relate to each other, or how to classify them in a way that could help investors choose which method would be best suited to achieving a particular objective. To that end, we took a stab at grouping these methods into four main categories: 

• Expected return methods weigh the anticipated benefits of an investment against its costs; social return on investment (SROI), in particular, provides a framework to calculate an investment’s present social value of impact compared to the value of inputs. For example, the Robin Hood Foundation’s benefit-cost ratio (BCR) estimates the poverty-fighting benefits of a program compared to the costs to the foundation in order to determine which grants would yield high impact. Robin Hood computes its BCR on an ongoing basis, and during the re-investment or re-granting process it may increase investment in programs with high BCRs, though the foundation does not make decisions on the basis of these calculations alone. Among the organizations we spoke to, grants-based organizations made the greatest use of expected return methods, but some impact investors also used it. 
  
• Theory of change methods outline the intended process for achieving social impact, often using a logic model, a tool that maps the linkages between input, activities, output, outcomes, and ultimately impact. When estimating impact, Acumen uses a logic model to identify assumptions in an intervention’s theory of change that may need further review (for example, would x output really translate into y outcome?). Logic models also help assess impact risk, the factors that could jeopardize the expected social impact of an intervention. For each of their investments, the Acumen team outlines what they think the biggest impact risks are and then comes up with risk mitigation strategies to monitor and manage any potential challenges. LGT Venture Philanthropy also uses a logic model, though in their case to identify specific metrics for input, activities, output, outcomes, and impact. 
• Mission alignment methods measure the execution of strategy against the project’s mission and end goals over time, using rubrics such as scorecards to monitor and manage key performance metrics on operational performance, organizational effectiveness, finances, and social value. Meaningful analysis often compares current key performance indicators to a historical baseline, to an original forecast, or to those of industry peers. Bridges Ventures developed its Impact Scorecard for such purposes. 
• Experimental and quasi-experimental methods are after-the-fact evaluations that use randomized control trials or other counterfactual approaches to determine the impact of an intervention compared to the situation if the intervention had not taken place. Where possible, Bridges Ventures draws on such data from previous studies when assessing a new potential investment’s impact risk. Various social impact bonds have also employed quasi-experimental and experimental methods to evaluate a program’s impact, which determines the financial return on investment.

Of course, each measurement method carries advantages and disadvantages (which we discuss in detail in our paper), but the main point is that they do not all accomplish the same objectives. As seen below, we analyzed our case studies to determine which measurement method leading impact investors use to accomplish each objective:

Investors use different measurement methods in each phase of the investment cycle.

Based on these observations, we propose an integrated model for investors looking to improve their impact measurement practices. We understand that impact measurement can be cost- and time-intensive, and that both impact investors and investees vary in their levels of organizational maturity and resources. We therefore recommend different sets of impact measurement methods based on the maturity of the investor and the investee. 

Our research on existing practices also revealed best practices for executing these measurement efforts: 

• Consider how to make a survey process valuable to respondents. This may help convince investee organizations of the value of impact measurement, and may lessen survey fatigue among beneficiaries. One investor that takes this approach, Root Capital, positions itself as a value-added partner that observes and measures impact to help farmers and enterprises increase their value, rather than as an impartial outsider measuring impact for its own purposes alone. 
• Design incentive structures, such as a social impact carry, a method that rewards portfolio managers based on the measured social impact of investments under their management. Core Innovation Capital, for example, ties a general partner’s financial compensation to an impact score, providing a clear incentive for general partners to manage portfolios in a way that yields strong social as well as financial results. 
• Embed impact measurement in the broader investment process. Instead of assigning the work of impact measurement to a dedicated resource outside of the core investment system, impact investors should consider integrating their impact measurement work closely with their investment and portfolio management work. At LGT Venture Philanthropy, investment managers are responsible for impact measurement, including building the theory of change, conducting site visits, and working with the funded projects to collect impact data. 

An integrated model of impact measurement methods.

While impact investing continues to gather momentum, inadequate and unstructured measurement approaches could prevent it from realizing its full potential. If a certain level of rigor in impact measurement is not established across the industry, the label “impact investing” may risk becoming diluted and used merely as a marketing tool for commercial investors. Our recommendations can help investors stay focused and ensure that impact investing continues to deliver real impact.

Read more stories by Ivy So & Alina S. Capanyola.