The fundamental tenets of strategic philanthropy are that funders and their grantees should have clear goals, strategies based on sound theories of change, and robust methods for assessing progress toward their goals. Although these ideas are gaining traction, some prominent philanthropic thinkers continue to express reservations about how they may affect the balance of power between funders and the organizations they support.
For example, former Ford Foundation president Susan Berresford expresses concerns about “funder-led strategic planning that imposes wearying and unnecessary demands on applicants and grantees,” and wistfully asks, “Has the role of the quiet, patient, and responsive funder become less appealing?” 1 She quotes the Indian social entrepreneur Sheela Patel’s complaint about funders’ imposition of logic models and their demand “that in a period of two years, we can implement perfect strategies and produce complete solutions.”
Similarly, Sean Stannard-Stockton, the founder and CEO of Tactical Philanthropy Advisors and philanthropic blogger, argues that the idea of a theory of change makes sense in a “static landscape, where you can learn more and more about what works and what doesn’t and finally craft the perfect theory,” but “fails in a dynamic landscape, such as social change, where what you learned on your last trip might not apply this time.” He asserts that funders should focus on building great organizations rather than on honing theories of change.2
Social change is inevitably complex and dynamic, and funders should be patient and forbear from micromanaging their grantees. But a funder has a legitimate interest in knowing whether an organization is on the path to success and, at some point, whether it is actually achieving impact. Indeed, it is the funder’s confidence in an organization’s theory of change, as well as in its leadership and management, that justifies patience in asking for proof of impact.
I believe that the issues raised by Berresford, Patel, Stannard-Stockton, and others are best addressed through a developmental approachto the assessment of theories of change and the actual impact of social interventions. A developmental approach is premised on the following observations about theories of change.
- Both funders and grantee organizations ultimately care about actual impact and not theories, but impact is often difficult to measure. Therefore, they must sometimes rely on beliefs about the soundness of a theory of change that underlies an organization’s strategy as a predictor of its future impact
- Theories of change are drawn from the fields (such as education and health) in which both funders and their grantees operate.
- The theories of change available to a field are not static but rather evolve as a field matures, proving themselves to be valid (or not).
- Funders’ and grantees’ intuitions about theories of change often play an important role at the very early stages of an organization’s or field’s development. But over time, a theory of change must be subjected to empirical validation and, eventually, to demonstration of actual impact.
Before going into more detail on the developmental approach to theories of change, it is important to better understand the views of those who argue that funders should not concern themselves with theories of change. (See “Defining Terms” below or on page 49 of the PDF for further explanation of theory of change and related terms.)
Radical Skeptics and Agnostics
Critics of strategic philanthropy can be grouped into two points of view, which I call radical skepticism and theory-of-change agnosticism.
William Schambra and Bill Somerville from conservative and progressive positions, respectively are radical skeptics not only about theories of change but also about the possibilities of evaluating impact. Schambra, formerly the director of programs at the Lynde and Harry Bradley Foundation, now directs the Hudson Institute’s Bradley Center for Philanthropy and Civic Renewal. Somerville, formerly the executive director of the Peninsula Community Foundation, now runs the Philanthropic Ventures Foundation.
Schambra and Somerville argue that funders should focus their grantmaking on community-based organizations, and that funders should trust an organization’s leaders without requiring them to articulate theories of change or imposing formal evaluation processes. Schambra’s position is rooted in a conservative skepticism about the possibilities of broad social change; Somerville’s position comes from a distrust of professional foundation staff and a strong belief in his own intuitive ability to pick good organizations.
These skeptics are implicitly analogizing grantees to idiots savants— individuals who are able to do complex calculations, such as finding the square root of any number, in their heads without knowing, let alone being able to explain, how they do it. Somerville’s writings imply that even if his grantees could explain their strategies, they are so busy doing good that it would be wicked to make them take the time to do so.
If there are social entrepreneurs who can demonstrate their impact without a strategy, or who can pursue a successful strategy without any articulable theory of change, all the more power to them. But this strikes me as highly unlikely. Achieving social change is much harder than doing square roots. And knowing whether one is actually achieving impact adds a further complexity. When the savant tells you that the square root of 7 is 2.645, you can check his result on a calculator. It’s a lot harder to verify whether a community youth organization is having a positive impact on the lives of disadvantaged kids. Therefore, both the organization and its funders must often rely at least in the short or medium term on the plausibility of its theory of change as a proxy for impact.
Indeed, most evaluations of social interventions require understanding the theories that mediate between inputs, activities, and outcomes.3 Theory-based evaluation requires understanding the steps embedded in a logic model of how an organization’s activities lead to impact. Even when the outcomes of social interventions can be evaluated through rigorous studies, the evaluators must know what variable to examine. Identifying the proper variables and metrics requires that one articulate the program’s goals and its underlying theory of change.4
The second group of critics, exemplified by Stannard-Stockton, is what I call theory-of-change agnostics. They believe that it is difficult to create a meaningful theory of change because social problems are complex and ever changing. Rather than spending time and money trying to craft or assess theories of change, agnostics think it is more productive for funders and grantees to focus instead on building great organizations.
Jeff Berndt, a partner in the venture philanthropy fund New Profit, is also an agnostic. He writes: “We see our role as a funder as identifying the best solutions to social problems, then providing these solutions with the financial and strategic support needed to grow their social impact. At New Profit we’ve decided not to become subject matter experts. We don’t subscribe to any one point of view on education reform, health-care efficiencies, or workforce development strategies. Instead, we believe entrepreneurs hold the insights and are best suited to design and grow their innovations.” 5
Stannard-Stockton and Berndt believe that a funder should be concerned with the organization rather than the theory of change underlying its work. But the two are not mutually exclusive. If the organization is the horse and the theory of change the carriage, then as Frank Sinatra famously sang, you can’t have one without the other. Both are necessary to get passengers whether disadvantaged youth living in the United States or people living on less than $2 a day in India to their intended destination.
Kristi Kimball, a program officer in The William and Flora Hewlett Foundation’s Education Program, approaches the issue from a perspective different from Stannard-Stockton and Berndt’s, but she ends up with a quite similar funding strategy. She argues that a funder should adopt a widely pluralistic approach in order to promote innovation. For example, there are numerous promising ways to improve K-12 student outcomes, including supporting good charter schools, using formative assessment to continuously improve instruction, improving school district management, and fostering competition and innovation through school choice and vouchers. Kimball asserts that a funder that privileges any one of these approaches stifles innovation. Indeed, by virtue of a funder’s reputation in a field, the very act of making a grant may confer legitimacy on one approach to the exclusion of others.
Kimball’s view may be right to a degree, but the combination of a funder’s limited financial and human resources constrains how many diverse approaches it can pursue. Working to improve the policy and management of public school systems involves a different set of actors and institutional dynamics from improving and replicating successful charter schools. Even a foundation that believes in the promise of both approaches may choose to focus its resources on one. As long as funders in aggregate are supporting a variety of promising approaches and sharing their knowledge about what works, any one funder’s decision to specialize internally doesn’t stifle innovation. Indeed, innovation is stifled when funders don’t provide sufficient support to give a promising idea a fair chance to be tested.
Stages of Development
There is much to be said for funders being pluralistic in supporting theories of change in the early stages of tackling a social issue. But as a field develops, weak theories of change will be abandoned in favor of ones that are proven to work. A developmental approach reflects the reality that one’s confidence in a particular theory of change and indeed, one’s ability to measure impact, often depend on an organization’s or an entire field’s stage of development.
The Edna McConnell Clark Foundation (EMCF), a venture philanthropy organization that builds youth development and other service delivery programs, provides a good example of a funder that takes a developmental approach. EMCF characterizes its grantees’ capacity in three developmental stages and provides the funding and technical assistance to help organizations move from one stage to the next.6
In the first stage, apparent effectiveness, an organization has a strategy or program based on a plausible theory of change, anecdotal stories of success, a positive reputation in the community, and perhaps some data about who participates in its program. Often, however, the organization does not collect systematic data about the populations it serves.
Over the next several years, the organization is expected to reach demonstrated effectiveness, in which data collection is rigorous, youths’ achievement can be compared to that of similar youths not in the program, and the theory of change is verified. In addition to relying on external evaluation, this is a period for the organization to begin to develop internal capacity for evaluation.
At the highest stage of proven effectiveness, an organization has a well-documented theory of change and has undertaken statistically rigorous evaluations, such as randomized controlled trials, that clearly demonstrate impact. For EMCF, an organization’s evaluation system is an important part of its infrastructure, and the foundation includes the cost of the evaluation in its grants.
Though not an EMCF grantee, the KIPP (Knowledge Is Power Program) charter school provides a good example of the three stages of development.
The process of social innovation is a messy mixture of theory, practice, evaluation, revision, and replication. A social entrepreneur begins with extant practices and knowledge in the field, accepts some, rejects others, and experiments the social equivalent of rapid prototyping in product design. She rejects what doesn’t work and keeps what seems to work, improving the impact for her own organization and diffusing knowledge for the benefit of others.
In the early stages, these elements are highly fluid and knowledge is often tacit. Funders have little to go on but their own informed intuitions about the quality of an organization’s leadership and the plausibility of the theory of change that underlies its strategy.
KIPP began in the mid-1990s when two Teach for America graduates started inner-city public school programs in Houston and the South Bronx. As KIPP evolved, its theory of change came to include a core set of practices: high expectations, parental choice, long school days, autonomous school leaders, and a relentless focus on results. These practices were not fully developed in the early years, nor were their results rigorously evaluated. But anecdotal evidence of the schools’ effectiveness began to accrete enough to justify Donald and Doris Fisher’s substantial philanthropic commitment in 2000 to establish a number of KIPP schools in the San Francisco Bay Area. (The Fishers founded the Gap.)
Although an early-stage funder might hope to let a hundred flowers bloom, nurturing the seeds can be expensive. The Fishers’ due diligence process, like EMCF’s, included an inquiry into KIPP’s theory of change and was based on its apparent effectiveness. The Fishers focused their substantial but finite resources on one organization with a particular theory of change rather than spreading them broadly.
As a field develops, tacit knowledge becomes explicit, and theories of change are subject to assessment. An essential aspect of a foundation program officer’s task is to learn from grantees, experts, programs’ clients, and others which approaches seem to be working and which are not. Of course, there’s always a danger that a program officer will become enamored of one approach to the exclusion of promising alternatives. But what some might characterize as a funder “imposing” its theory of change on grantees may just be the result of field-wide learning.
The greater danger is that organizations and funders rely on their intuitions for too long, postponing inquiry into whether a theory of change is actually working. The common human tendency to avoid seeking evidence that challenges our beliefs is exacerbated when funders and organizations that depend on their support have made substantial investments in strategies based on particular theories of change.
Early indications that KIPP was making a real difference were corroborated by a 2005 evaluation of its Bay Area schools, using a cross-sectional approach: KIPP students’ achievements were compared to similar students in public schools. Notwithstanding its methodological limitations, the study indicated that KIPP had a large positive effect on student outcomes.7 This provided a justification both for continuing support and for a more rigorous evaluation.
At some point, practices become so widespread within a field that it makes sense to evaluate not just one organization’s effectiveness but also the theory of change that underlies a general strategy or intervention. Although it is important to prevent premature convergence on one solution to a complex social problem, it is also essential to learn what works and to reject what doesn’t.
For example, there may have been a time when reasonable people could believe that either an abstinence-only program or one that included comprehensive sex education was more effective in preventing teen pregnancy. In the absence of data, people’s intuitions perhaps influenced by ideology played a legitimate role in deciding which program to back. As the field matured, however, the time came to put these intuitions to an empirical test. In 1997 Congress funded a large-scale study that strongly indicated that abstinence-only did not work.8
There are increasing bodies of evidence about programs that do and do not work in other fields as well. For example, the What Works Clearinghouse summarizes the results of evaluations of interventions in education. It is worth noting that the Social Innovation Fund, created by the Serve America Act of 2009, is seeking to collaborate with philanthropists and nonprofit organizations to bring proven programs to scale.9
By 2008, KIPP had matured to a point where it made sense to evaluate its theory of change through a randomized controlled trial. The study, conducted by Mathematica Policy Research and scheduled for publication in 2014, is likely to have field-wide benefits well beyond the particular schools involved.10 If the intervention has a big effect and is generalizable to other schools with similar student populations, then one need not repeat the evaluation for every school. Rather, one can take the much less costly approach of observing whether a school is following the KIPP principles and, of course, assessing its pupils’ progress. This also permits refining an established strategy by providing a baseline against which variations can be measured.
A Learning Process
A well-tested theory of change doesn’t reflect any single funder’s or organization’s vision. Like most knowledge, an empirically based theory of change, or good evidence that a theory of change does not work, is a public good that justifies discarding, replicating, or building on innovations.
Foundations can contribute greatly to the public good by disseminating what they have learned about successful or unsuccessful strategies and theories. There are, however, several barriers to doing this. First, most foundations do not systematically capture and organize knowledge of this sort even for their own use. Second, there are not many effective channels for disseminating what is known.11 And third, the quip that “victory has a thousand fathers, but defeat is an orphan” applies no less to foundations than to other institutions. There are signs that each of these barriers is being reduced, but, like most social change, it is slow.
As a postscript, let me return briefly to Berndt’s disclaimer of subject matter expertise by his venture philanthropy fund and Kimball’s argument that a foundation should pursue myriad educational theories. Whatever their professed modesty, savvy philanthropists will select organizations whose strategies and theories of change seem plausible and reject those that don’t. Evaluating an organization’s strategy and its underlying theory of change can never substitute for assessing its organizational strength and the quality of its leadership. But neither can an organization’s strength and leadership substitute for a sound strategy based on an empirically valid theory of change and this calls for a degree of subject matter expertise. Indeed, Berndt confirms as much when he precedes the disclaimer of expertise by saying “we see our role as … identifying the best solutions.”
Tackling Adaptive Problems
So far, I have focused on programs that provide services, such as schools, youth programs, and teen pregnancy prevention clinics. Service providers such as these must be tactically adaptive, able to respond to the challenges thrown at them from one day to the next. But the core problems they address are not subject to rapid changes: For a charter school or an after-school program, next year’s cohort of disadvantaged inner-city kids is pretty much like this year’s. This is what permits refining the program and its underlying theory of change over a number of years.
In contrast, the operating environments for organizations that employ advocacy and political mobilization to transform social, economic, and political systems may change dramatically and unexpectedly. For example, a change of national or state administrations may greatly affect the strategies of organizations concerned with issues ranging from climate change to civil rights to foreign aid. These organizations’ missions and time horizons require that they be not only tactically adaptive but strategically adaptive, able to continually engage in high-level problem solving.
In “Leading Boldly” (Stanford Social Innovation Review, winter 2004), Ronald Heifetz, John Kania, and Mark Kramer note that most complex social problems “are not … well defined, the answers are not known in advance, and many different stakeholders are involved, each with their own perspectives.” The authors call these “adaptive problems.”
Problems of these sorts are paradigmatic of Stannard-Stockton’s “dynamic landscapes.” Funders must give organizations tackling such problems considerable autonomy in experimenting with different theories of change and must be patient about demanding measurable impact. As Berresford writes, “If funders are funding people trying to change deep-seated attitudes or individual and institutional behavior, then they need to engage in a dynamic, long-term, and open-ended consultative process with them. Such processes have been a key to success in funders’ and grantees’ struggle against apartheid and other forms of discrimination, and in building entirely new kinds of organizations empowering the poor here and around the globe.”
A good example of an organization tackling this type of problem is California Forward, which is dedicated to reforming governance and fiscal policy in California.12 California Forward’s mission requires strong theories of change of two different sorts: first, a theory of how a particular reform say, redistricting by a nonpartisan commission will improve (and not worsen) governance; and second, a theory underlying an advocacy strategy to achieve the reform for example, whose decisions must be influenced and how? Although there is general consensus among the organization’s leadership and its funders about its general aims, California Forward needs broad discretion not only in the types of tools it uses education, policy research and advocacy, and community organizing to achieve those aims, but also in deciding which reforms to pursue at what time, depending on its perception of their ripeness.
No sharp boundary divides tactical and strategic adaptiveness, and both concepts contain their own continua. For example, good research institutions whether universities or think tanks have bureaucratic structures, within which individual researchers have great freedom to follow their own lights. Even within a single field, nonprofits run the gamut of adaptiveness. For example, some charter management organizations impose significant constraints on their member schools’ culture and curriculum, whereas others encourage innovation.
External changes in the policy or economic environment can press any organization to become adaptive, but not all organizations need to be strategically adaptive all of the time. Constant attention to adaptation can divert attention and resources from effective program implementation. Many organizations add social value mainly by replicating well-tested strategies.
Like the continuum of adaptiveness, the allocation of authority between a funder and grantee is not binary. The extent to which funders accord grantee organizations the freedom to be strategically adaptive depends on a number of factors, keys among which are the quality of their leadership, their role in the field (innovation vs. replication of others’ strategies), and the state of the field itself. At some times, a field must be highly dynamic in seeking new solutions; at other times, it appropriately consolidates and replicates the achievement of an earlier time. Adaptive problems usually cannot be solved by a single entity. Therefore, in addition to providing support to particular organizations, funders may advance the development of nascent fields by bringing together multiple organizations and stakeholders.
Listening to the Field
Doubtless, funders occasionally come up with idiosyncratic theories of change that they impose on grantees. But the very nature of the development of knowledge generally makes this unlikely. Rather than listening, a funder would have to intentionally close its ears not to be affected by what’s going on in the field. Indeed, the history of innovation is one of parallel discoveries and inventions for example, Gottfried Leibniz and Isaac Newton coming up with calculus at the same time, and Alexander Graham Bell and Elisha Gray simultaneously inventing the telephone. Ideas for social change are no less part of the zeitgeist than those in science and engineering. 13
Berresford notes that the relationship between a funder and its grantees especially but not only adaptive organizations ought to be “a dynamic, long-term, and open-ended consultative process.” Nothing in the nature of strategic philanthropy militates against this. On the contrary, a study by the Center for Effective Philanthropy found that the most strategic foundations “do not … impose their goals and strategies on a field without concern for grantees’ or other practitioners’ expertise.” Rather, “they interact with stakeholders and communities grantees, experts, and others to create feedback loops to develop and refine their strategy.” 14
The tensions discussed in this essay are not resolved either through the sheer exercise of power or through a laissez-faire attitude, but through a flexible agreement that works to the benefit of the shared goals of the funder, its grantees, and their ultimate beneficiaries.
1 Susan Berresford, “The 1 Art of Grantmaking,” Stanford Social Innovation Review, spring 2009.
2 http://www.huffingtonpost.com/paul-brest/whats-the-evidence-for-ev_b_156141.html (guest blog).
3 Carol Weiss and Johanna Birckmayer, “Theory-Based Evaluation in Practice: What Do We Learn?” Evaluation Review, 24, 2000: 407; Carol Weiss, “How Can Theory- Based Evaluation Make Greater Headway?” Evaluation Review, 21, 1997: 518.
4 Peter Rossi, Mark Lipsey, and Howard Freeman, Evaluation: A Systematic Approach Sixth Edition, Thousand Oaks, Calif.: SAGE Publications, 1999.
6 Summarized by Paul Brest and Hal Harvey, Money Well Spent: A Strategic Plan for Smart Philanthropy, New York: Bloomberg Press, 2008.
7 Bay Area KIPP Schools, A Study of Early Implementation: First Year Report 2004–05, Menlo Park, Calif.: SRI International Inc., 2006. (The Hewlett Foundation contributed $320,000 to cover evaluation costs.)
8 Christopher Trenholm, Barbara Devaney, Ken Fortson, et al., Impacts of Four Title V, Section 510 Abstinence Education Programs, Princeton, N.J.: Mathematica Policy Research Inc., April 2007.
9 http://www.americaforward.org/2009/05/details-on-the-social-innovation-fundfrom- the-serve-america-act.
10 The William and Flora Hewlett Foundation is supporting a portion of this project.
11 IssueLab and PolicyArchive are two organizations taking steps in this direction.
12 The foundations are the California Endowment, The William and Flora Hewlett Foundation, The David and Lucile Packard Foundation, The James Irvine Foundation, and the Evelyn and Walter Haas, Jr. Fund.
13 Malcolm Gladwell, “Who Says Big Ideas Are Rare?” The New Yorker, May 7, 2009; Stephen Stigler, “Stigler’s Law of Eponymy,” Transactions of the New York Academy of Sciences, Series II, 39, 1980: 147-157.
14 Kevin Bolduc, Ellie Buteau, Greg Laughlin, et al., Beyond the Rhetoric: Foundation Strategy, Cambridge, Mass.: Center for Effective Philanthropy, October 2007.
Paul Brest is president of The William and Flora Hewlett Foundation. Before joining the Hewlett Foundation, he was a professor at Stanford Law School, serving as dean from 1987 to 1999. He is coauthor of the book Money Well Spent: A Strategic Plan for Smart Philanthropy. He is also coauthor of “Calculated Impact” (Stanford Social Innovation Review, winter 2009) and author of “Smart Money: Strategic General Operating Support” (Stanford Social Innovation Review, winter 2003).