In selecting a portfolio of investments, deciding whether to attend business school, or choosing the deductible for their car insurance, people regularly compare the costs, risks, and potential benefits of their choices. The concept of cost-benefit analysis has even begun to permeate government regulations: Under Executive Order 12866, which President Bill Clinton signed in 1993, federal agencies must “assess all costs and benefits of available regulatory alternatives, including the alternative of not regulating.”
For the most part, philanthropy lags behind the practices of investors, individual decision makers, and governments. Philanthropists grant billions of dollars a year without assessing whether their chosen strategies are likely to solve the problems that motivate their giving, and without attempting to assess the effectiveness of the organizations they fund.
With the hope of stimulating a more outcome-oriented approach to philanthropy, this article examines efforts to take a quantitative approach to three different kinds of philanthropic activities: (1) assessing ongoing social programs; (2) predicting the return on mission investments; and (3) planning long-term program strategies.
Our overarching concept is one of expected return. We use this slightly modified version of the conventional formula:1
In this equation, the outcome is the impact a grant or other philanthropic investment would have if everything went according to plan and if the funder were solely responsible for the outcome. Because philanthropic investments are by no means certain to achieve their intended outcomes, the numerator corrects for the probability of the outcome. The numerator also accounts for the fact that philanthropists seldom achieve outcomes entirely by themselves. Instead, they contribute to only a portion of the outcome—their philanthropic contribution.
Having estimated the benefit of an investment in the numerator, the formula then accounts for the cost of a grant in the denominator. The cost of a grant is the amount of the grant plus administrative costs. (In the case of a mission investment that expects a financial as well as a social return, as in the Acumen Fund example below, the costs are reduced by the amount of the financial return.)
The expected return is the predicted benefit per dollar invested. Typically, a philanthropist might use this metric to make comparisons within a single domain—say, between alternative initiatives to reduce HIV/AIDS. At least one foundation, Robin Hood, uses expected return very ambitiously to make comparisons across domains—to compare its workforce, education, and health investments, for example. Robin Hood uses the analysis to answer the question “How do we compare the poverty-fighting impact of apples (charter schools) with the poverty-fighting impact of oranges (job training for home health aides)?”2 In any case, the funder’s goal is to deploy its philanthropic dollars most effectively.
What to measure?
Although financial investors may differ in their risk preferences and desired time frames, they share a common metric— dollars—for assessing the returns on their investments. By contrast, because philanthropists pursue a broad array of goals, they lack a common currency. Any analysis of expected return begins by defining clear goals and indicators of success in achieving them. Differences in indicators may signal different goals.
Consider, for example, the measures used by Robin Hood and the venture philanthropy firm REDF in assessing their workforce development programs. Although REDF has since broadened its view, it originally measured the value of moving trainees into productive jobs only in terms of the benefits to government through the trainees’ increased tax contributions and their reduced use of publicly funded benefits and services.3 By using these measures, REDF could demonstrate the public value of its social investments, with the hope that governments would participate in funding its programs. REDF also believed that these numbers constituted more credible and consistent measures of value than the benefits to individual clients.
In contrast, Robin Hood measures impact in terms of the personal benefits that accrue to its grantee organizations’ poor clients and their families. For example, in a job training program for ex-offenders, Robin Hood estimates the impact of job placements on trainees’ recidivism and on their future earnings. In supporting early childhood programs, it estimates the impact of reading readiness on high school graduation rates and, therefore, on the children’s projected earnings as adults.
Although Robin Hood does not make information about particular grantees publicly available, it demonstrates its approach through a fictional example: Bob’s Jobs, a workforce training program for women that is based on Robin Hood’s actual programs.
In this example, 150 women enrolled in Bob’s Jobs and 72 completed the training. Of these 72 newly minted construction workers, 41 held on to their jobs for only three months. The remaining 31 still had jobs at the end of one year.
How much does Bob’s Jobs benefit trainees? Robin Hood compares the salary of each participant before she entered the program and after she graduated. The 41 women with short-term employment enjoyed an average salary increase of about $2,900, or $120,000 in total. The average annual salary increase for the 31 women who held jobs for at least a year was approximately $12,000. To compute the value of the program for these 31 women, Robin Hood makes the following assumptions:
- They will continue to be employed for 30 years.
- Their annual salaries will increase by 1.5 percent above inflation.
- They have an average of 1.8 children each; given research findings on the effects of parents’ employment, each family will realize an intergenerational income boost of $56,000.
- The discount rate—that is, the number used to calculate how much an amount of future money is worth today—is 3.5 percent.
On the basis of these assumptions, Robin Hood estimates that the net present value of the benefits to the 31 long-term workers is $9.1 million. Adding the onetime salary boost for the 41 short-term workers, the total benefit of the program is $9.2 million. Therefore, the program’s outcome in the expected return equation is $9.2 million.
No social program achieves its goals 100 percent. For example, participants drop out of Bob’s Jobs or do not gain long-term employment. In this case, as with most programs with known results over time, the risk of failure is already accounted for: The net benefits are actually the product of the intended outcome times the risk of individual participants’ not benefiting from the program. Therefore, Robin Hood does not further discount the value of the benefits based on the probability of success; it assigns a value of 1.0 to the probability of outcome in the expected return equation.
Robin Hood does, however, discount its own role in generating these benefits, taking account of the fact that other donors, both private and public, support Bob’s Jobs. The foundation assigns a value of 0.5 to philanthropic contribution in the equation, calling this the “Robin Hood factor.” Multiplying the benefits of $9.2 million by 0.5 yields $4.6 million as the total benefits attributable to Robin Hood.
What about the costs? The grant to Bob’s Jobs was $200,000. Robin Hood does not include the administrative costs of making the grant, both because they are not large and because they do not differ significantly from those of the other poverty-fighting programs to which it will compare Bob’s Jobs.
Dividing the numerator of $4.6 million by the denominator of $200,000 results in an expected return of $23 per $1 invested:
That is, for each dollar that Robin Hood spends on Bob’s Jobs, trainees and their families gain $23—a pretty good return on investment.
Robin Hood has made similar calculations for its other poverty programs, including a program to help people manage budgets, bank accounts, and loans (expected return = $1.90) and a clinic that deals with asthma, hepatitis, and cancer (expected return = $12).
Although Robin Hood considers expected returns in its grantmaking, it does not rely exclusively on them. The foundation recognizes that the metrics are imprecise, with each expected return calculation depending on complex and uncertain empirical assumptions. Thus Robin Hood continues to test its metrics against the informed intuitions of program officers and experts in the field. At the same time, the foundation presses its staff to justify their intuitions against the numbers generated by analysis.
The Robin Hood Foundation can base its calculations of expected returns on a substantial sample of participants in an ongoing program. But some organizations make one-of-a-kind mission investments in start-up enterprises with little or no track record. One such organization is Acumen Fund, a venture philanthropy firm that provides loans and equity capital to social enterprises in developing countries. Acumen Fund is particularly concerned with reaching the poorest people in those countries—those at the “bottom of the pyramid” (BOP).
To assess a potential investment, Acumen Fund asks whether it is likely to outperform the best alternative charitable option (BACO). BACOs may be real or hypothetical. Although comparisons to BACOs have a large margin of error, they inform Acumen Fund’s due diligence process and help ensure that its investments are likely to deliver more social output per dollar than conventional grantmaking alternatives.
Acumen Fund’s loan to A to Z Textile Mills in Tanzania is illustrative.4 With a $325,000 loan, Acumen Fund sought to help a local firm produce long-lasting insecticide-treated bed nets (LLITNs) that would protect people against the mosquitoes that transmit malaria. LLITNs are effective for five years, whereas conventional insecticide-treated bed nets (ITNs) are effective for only two and a half years. The BACO for this investment in A to Z was a hypothetical $325,000 grant to an international nongovernmental organization (NGO) that would use the money to distribute ITNs.
Acumen Fund’s analysis has three steps: (1) calculate the expected return of the BACO; (2) calculate the expected return of the investment in A to Z; and (3) compare the expected returns of the two options.
1. Expected Return of the BACO (i.e., a grant to an NGO).
Acumen Fund first estimated the outcome of making a $325,000 grant to an NGO that would distribute ITNs. An NGO could purchase and distribute conventional ITNs at a cost of $3.50 per bed net, for a total of 92,857 nets. As a grant-driven approach can have more control over the end-consumer profile than can a market-based approach, Acumen Fund assumes that 100 percent of these nets would be given to BOP people. Each net would protect an average of two people, for a total of 185,714 people living at the bottom of the economic pyramid.
Acumen Fund chose not to assign a dollar value to this outcome because valuations of health and life are complex and controversial.5 Rather, it calculated the benefits in terms of people years of malaria protection, multiplying the number of people served by the number of years they could use the nets—an average of two and a half years. Acumen Fund estimated that the BACO grant could produce 464,285 people years of protection from malaria.
Although Acumen Fund acknowledged that the probability of either the BACO or the investment’s success was certainly less than 100 percent, for the sake of simplicity it did not account for risk in its calculations of expected return. Hence it assigned 1.0 as the probability of achieving the outcome in both cases. Because Acumen Fund would fully fund the grant, its philanthropic contribution for the BACO was 100 percent—also a value of 1.0 in the equation.
Acumen Fund next calculated the cost of the BACO grant. Unlike Robin Hood, the organization includes the costs of administering the BACO grant—in this case, $32,500—bringing the cost of the
$325,000 grant to $357,500. Because grants do not earn returns, Acumen Fund accounts for the sunk costs of the grant and administration in the denominator of the expected return formula. Solving the equation, Acumen Fund figured that the expected return on the BACO would be:
2. Expected Return of Investing in A to Z Textile Mills.
Acumen Fund next estimated the outcome of investing in A to Z Textile Mill. Fully funded, A to Z Textile could produce 2 million LLITNs. Because the LLITNs would be sold on the open market, they would have a diversity of purchasers, of whom half were estimated to be among Tanzania’s BOP population. As with the BACO, Acumen Fund multiplied the number of nets (2 million), by the portion of nets going to BOP people (0.5), by the average number of people using each net (two people), by the number of years they would use the nets (five years). By this calculation, A to Z textile factory could provide 10 million BOP person years of malaria protection.
As was the case with the BACO, Acumen Fund did not formally factor in the risk of not achieving the outcome, and so the value for the probability of outcome in this calculation is 1.0. Whereas Acumen Fund fully funded the BACO, however, here its investment would be 20 percent of the total capital invested in A to Z for this project. Thus the value for its philanthropic contribution was 0.2. Using these numbers, Acumen Fund estimated that its investment in the A to Z Textile Mill would result in a total benefit of 2 million BOP person years of protection.
Acumen Fund then calculated the cost of its investment in A to Z Textile Mill. Making this loan would cost an estimated $65,000 on top of the $325,000 cost of the loan itself, for a total of $390,000. But unlike the BACO grant, which delivers no financial returns to Acumen Fund, the loan to A to Z Textile should theoretically yield both the return of the principal and (a below-market rate) 6 percent interest compounded annually over the course of the three-year loan. But Acumen Fund made the conservative estimate that A to Z would repay the principal but not pay the interest. Subtracting the projected financial return (i.e., the $325,000 principal) from the total cost of making the loan (i.e., $390,000), Acumen Fund calculated a cost of only $65,000.
Acumen Fund thus estimated the expected return on its investment in A to Z Textile Mill:
3. Comparing investments.
As summarized in the figure below, the numbers indicate that investing in A to Z Textile is some 24 times more cost-eff ective than donating to the BACO. Moreover, if A to Z Textile pays the 6 percent annual interest as well as repaying the principal, the loan/BACO ratio increases substantially. If A to Z Textile defaults entirely but nonetheless manufactures the bed nets, the ratio falls to four—still a better deal than the BACO.
In addition to guiding particular grants and mission investments, expected return analysis can inform overarching philanthropic strategies. In collaboration with the Boulder, Colo.-based Redstone Strategy Group, the William and Flora Hewlett Foundation used expected return analyses to develop a new program in global development, which has the goal of improving the well-being of people living on less than $2 per day. In pursuit of this goal, program staff considered a variety of strategies in developing countries.
Using expected return analysis to guide choices among investments requires a common metric. To this end, the Hewlett Foundation developed an index that incorporates income, health, and education. To simplify the following discussion, however, we focus only on income in describing how the foundation estimated the effects of promoting government transparency and accountability in Nigeria.6
Despite Nigeria’s $50 billion in annual oil revenues and $1 billion in annual aid, 92 percent of its population—more than 120 million people—lives in abject poverty. Many experts believe that government corruption contributes much to this situation. Research indicates that decreasing corruption improves government effectiveness, increases gross national income per capita, and improves the wellbeing of the very poorest citizens. The evidence suggests that activities such as expenditure tracking, budget monitoring, and citizen report cards on public services help reduce corruption.
The Hewlett Foundation wanted to know what approximately $30 million in grants for such activities, disbursed over eight years, could achieve toward increasing the welfare of the very poor by reducing corruption in Nigeria. To calculate the investment’s potential benefits, the foundation relied on analyses by development experts and the foundation’s on-the-ground experience with similar work in Mexico, and took into account the particular circumstances of Nigeria. These analyses suggested that investments by the foundation and others could double the incomes of about 8 million Nigerians currently living on less than $2 per day.
The Hewlett Foundation next calculated the dollar amount of this increase in income. Data suggest that the average income of a person in Nigeria living on less than $2 per day is $644 per year, so doubling that would result in an additional $644 in income per year. The foundation assumed (conservatively) that the person would earn the increased amount for 10 years, yielding a total increase of $6,440 per person for a decade.
The Hewlett Foundation then calculated the likelihood of success. Virtually all philanthropic interventions face strategic, organizational, and external risks. To calculate the risks, the foundation consulted experts at the Center for Global Development, the Brookings Institution, and Oxfam. It also took into account its experience with similar grantmaking in Mexico. Although staff members were confident about the robustness of the proposed strategies and the strength of potential grantees in Nigeria, they understood that their theory of change required many moving parts to work together. Taking into account all the risks, they gave the program a 25 percent probability of success.
Next, the Hewlett Foundation calculated its contribution—an estimate of the portion of the outcome for which it would be responsible. This estimate reflected both the amount of dollars invested and the influence of those dollars. Since the program relied on donations by other foundations as well as non-philanthropic investments, the Hewlett Foundation’s contribution would be less than five percent of the total effort. Nevertheless, because of the foundation’s leadership in this field, its involvement was likely to be catalytic. Therefore the team estimated that the foundation’s contribution would account for 10 percent of any observed changes.
The Hewlett Foundation then estimated the total benefit of its investment in transparency and accountability work in Nigeria. Multiplying the 8 million target population by the 25 percent probability of success and by the 10 percent of the effect attributable to the foundation’s contribution, the staff concluded that the foundation’s investment was likely to double the incomes of about 200,000 people now living on less than $2 per day.
Finally, the Hewlett Foundation calculated the total cost of its investment. Costs associated with the benefits under consideration included grants to NGOs for budget and revenue monitoring and tracking expenditures, as well as the costs of training government officials to implement freedom of information laws. The foundation also included the administrative costs involved in making, monitoring, and evaluating grants. These expenses totaled $30 million.
If the foundation’s only outcome measure were income, the expected return on its transparency and accountability grants in Nigeria would be:
As mentioned above, the Hewlett Foundation had outcome measures besides income. Using these measures, the foundation created a composite index to compare investments in various strategies. On this index, transparency and accountability achieved a high score, as did impact evaluation, improving agricultural markets for the rural poor, and improving the quality of education. The foundation decided to pursue all of these strategies and is using expected return analysis to determine in which African countries it could be most effective.
THE EXPECTED RETURN OF EXPECTED RETURN ANALYSIS
What can one learn from the three examples of expected return analysis considered above?
The Robin Hood Foundation had the easiest task in gathering and analyzing data. Workforce development organizations like Bob’s Jobs have actual records of performance with enough participants to allow good statistical estimates of their benefits. Although Robin Hood’s estimates come with significant margins of error, as long as the organization consistently compares the benefits of Bob’s Jobs with those of other workforce development programs, it can fairly approximate the programs’ relative expected returns. The margins of error are considerably greater when Robin Hood seeks to compare workforce development programs with programs seeking to alleviate poverty within different populations and through different means.
Acumen Fund’s analysis of A to Z Textile is grounded in considerable accumulated knowledge about the efficacy of different types of antimalarial bed nets. Also, for all the variation among different products, the causal chain from manufacture to distribution and end use of products, as well as the risks of breaks in the chain, fit within a marketing paradigm with centuries of experience. Nonetheless, Acumen Fund is making a one-shot investment in an early-stage project that is fraught with uncertainties. Its margins of error are much greater than Robin Hood’s.
Estimating the expected return of the Hewlett Foundation’s strategy for improving transparency and accountability in Nigeria has all of the uncertainties involved in A to Z Textile—and then some. In contrast to the manufacture and distribution of a product, the foundation’s transparency and accountability work involves a complex network of causes and hoped-for effects with deep uncertainties at almost every node. Estimating the value of this work in a single country is a speculative enterprise. And comparing its value in different countries and the relative benefits of alternative strategies to benefit Africa’s poorest citizens is far more so.
Does this mean that expected return analysis was useful for Robin Hood, marginal for Acumen Fund, and not worthwhile for the Hewlett Foundation? That depends on the expected return of doing these expected return analyses. After all, the concept of expected return implies that a cost is justified to the extent (but only to the extent) that it contributes to net benefit. Every dollar of a foundation’s personnel and other administrative costs reduces the funds available for making grants. In this respect, the costs of undertaking an expected return analysis are no different from the costs of strategic planning, conducting due diligence, and monitoring and evaluating grants.
Even if the margins of error of an analysis are huge—as they are with the Hewlett Foundation’s strategic planning for transparency and accountability—the estimated expected return is one’s best guess of the impact of a strategy or grant: There’s no a priori reason to believe that the estimate either over- or understates the actual outcome. Equally important, as Brian Trelstad, Acumen Fund’s chief investment officer notes, the value of the analysis “is not in the number … that is spit out at the end, but in forcing the team to think through the marginal analysis of whether or not we really are generating significantly more social impact for our philanthropic dollar than prevailing approaches.”7
In considering how much to invest in expected return explorations, it is also worth considering their potential to inform the field as a whole. Of course, this requires that funders candidly share their experiences with others. The organizations described in this article have set a good example in this respect. REDF, a pioneer in using expected return analysis, has led the way with a candid assessment of the challenges of its first efforts.8
We have spoken of the costs of an expected return approach to philanthropy. But can it do affirmative harm? Bruce Sievers, formerly executive director of the Walter & Elise Haas Fund and now a visiting scholar at Stanford University, has been a vocal critic of this approach. His essential criticism is that an emphasis on “[m]easurable outcomes… may distort an organization’s program or actually cause more important, intangible aims to be overlooked.”9 The concern would be exemplified by a foundation that supported the performing arts and evaluated grantees only in terms of their audience size or box office receipts to the exclusion of the quality of their performances. Sievers goes on to note:
[T]he environmental movement, the rise of the conservative agenda in American political life, and the movement toward equality for the gay and lesbian communities, all aided by significant philanthropic support, have transformed American life in ways that lie beyond any calculations of “return on investment.” … Commitment of philanthropic resources to these issues was not merely a matter of analyzing increments of inputs and output; it was a moral engagement with wooly, unpredictable issues that called for deeply transformational action.
Along similar lines, President Clinton’s executive order governing federal regulations provides, “Costs and benefits shall be understood to include both quantifiable measures (to the fullest extent that these can be usefully estimated) and qualitative measures of costs and benefits that are difficult to quantify, but nevertheless essential to consider.”
We agree entirely with these points. It would be a tragic error for a philanthropist to distort his or her mission in order to seek only measurable outcomes. But the fact that a technique can produce bad results if taken to ridiculous extremes is no reason to forgo its benefits, and we have seen no signs of the single-minded pursuit of numerical results in philanthropy. On the contrary, the field has generally been indifferent, if not resistant, to assessments of its impact.
In the end, expected return analysis is not a substitute for intuition, but rather a structure for testing one’s intuitions about what strategies are likely to work. An expected return approach encourages philanthropists to be realistic about what they can achieve with their resources. It encourages candor about risks and the possibilities of failure—a necessary prerequisite to considering strategies to mitigate them. If ambitious philanthropy is akin to sailing in uncharted waters, expected return analysis is no GPS. But it’s a lot better than dead reckoning.
This article is adapted with the permission of Paul Brest and Hal Harvey from Chapter 10 of Money Well Spent: A Strategic Plan for Smart Philanthropy (Bloomberg Press, 2008). In several instances, the article uses more recent data than are contained in the book. We are grateful for the assistance of Cynthia Gair and Carla Javits (REDF), Michael Weinstein (Robin Hood Foundation), Brian Trelstad and Katie Hill (Acumen Fund), and Ivan Barkhorn (Redstone Strategy Group).
Paul Brest is president of The William and Flora Hewlett Foundation. Before joining the Hewlett Foundation, he was a professor at Stanford Law School, serving as dean from 1987 to 1999. He is coauthor of a forthcoming book on problem solving, decision making, and professional judgment.
Hal Harvey is founder and president of ClimateWorks Foundation. He previously directed the Hewlett Foundation’s Environment Program. He has held positions at several different nonprofit foundations, including the Mertz Gilmore Foundation, the Heinz Endowments, and the Ploughshares Fund.
Kelvin Low is an airline network analyst at Singapore Airlines. A 2007 graduate of Duke University, he remains passionate about social innovation, venture philanthropy, and collaborations between nonprofit and for-profit enterprises.
1 In the world of finance, costs are subtracted in the numerator, in addition to appearing in the denominator. We follow the (somewhat less conservative) practice of most calculations in the domain of policy and philanthropy and put costs only in the denominator.
2 Michael M. Weinstein with the assistance of Cynthia Esposito Lamy, Measuring Success: How The Robin Hood Foundation Estimates the Impact of Grants, New York: Robin Hood Foundation, 2008.
3 See Carla I. Javits, REDF’s Current Approach to SROI, May 2008. http://www.redf.org/download/sroi/REDFs-Current-Approach-to-SROI.pdf
4 This example is drawn from Acumen Fund Concept Paper: The Best Available Charitable Option (BACO) , available on the organization’s Web site.
5 Some governments and NGOs such as Robin Hood use quality-adjusted life years (QALYs) for such estimations. (One QALY is equal to one year of perfect health, and a QALY of less than one is equal to one year lived in pain or disability).
6 The example is taken from Redstone Strategy Group, Making Every Dollar Count: How Expected Return Can Transform Philanthropy, April 10, 2008.
7 E-mail correspondence with Paul Brest, January 5, 2008.
8 Cynthia Gair, A Report from the Good Ship SROI. www.redf.org/download/sroi/goodshipsroi2.doc
9 Bruce Sievers, “Philanthropy’s Blindspots,” in Just Money: A Critique of Contemporary American Philanthropy, ed. by H. Peter Karoff (Boston: TPI Editions, 2004): 132.