Evaluation’s Next Generation

The practice of evaluation, which came into its own with the Great Society initiative of the 1960s, has become more diverse and pluralistic in response to the increasing complexity of the social sector. It has expanded its use beyond accountability to include learning and beyond compliance to include capacity building. Most traditional evaluations, however, continue to fall short of their potential to catalyze real change in organizations and the social sector.

A recent survey on nonprofits and their relationship with data found that only 50 percent of nonprofits are tracking any kind of outcome data. Foundations are no different. A benchmarking study of evaluations in foundations found that only 38 percent of foundations reported, “Management regularly models the use of evaluation in its own decision making.” The authors of the study go on to suggest that this appears to be less about the willingness of foundations to use evaluation, and more about the poor fit between the decisions they need to make, the evaluation questions they ask, and the data that nonprofits produce. In other words, evaluation isn’t giving people what they need.

At FSG, we have spent the last several months digging deeper into the question of how evaluation can continue to remain relevant, timely, and useful in the social sector. This has become more salient in the light of three major trends:

• New philanthropic innovations: Many new and untested innovations are coming to the fore as philanthropists experiment with new ways to solve chronic social problems.
• Different rules of interaction: The environment has become more fluid and less predictable, and the number of solutions involving multiple actors has vastly grown.
• Proliferation of digital infrastructure: Widespread adoption of technology and social media is enabling enhanced communication while generating massive amounts of data.

As part of our Next Generation Evaluation initiative, we reviewed literature, interviewed 15 thought leaders in the field, and studied organizations that are doing pioneering work around new ideas and approaches. Our research led us to identify six characteristics that unite the social sector’s most promising efforts to evolve and expand its practice of evaluation, summarized below. It is important to emphasize that we see these as complementary to, rather than outright replacements of, traditional evaluation approaches and methods.

 

We also identified three approaches that best embody the six characteristics of Next Generation Evaluation. These are:

• Developmental evaluation, which offers more real-time, learning-oriented feedback and insights, as opposed to adhering to a fixed evaluation plan.
• Shared measurement, where a group of organizations come together to co-determine outcomes and indicators, and to learn from each other.
• Big data—examining the massive amounts of data that technology and social media are currently generating, and how we can glean insights from it.

We have captured the synthesis of our research in this learning brief. The conversation will continue at the Next Generation Evaluation Conference that we are co-hosting with the Stanford Social Innovation Review at Stanford University on November 14th. Over the next few months, we will continue to build out additional ideas and examples of Next Generation Evaluation, and solicit feedback and insights from the field. We hope that you will join us in this journey.

Read more stories by Srik Gopalakrishnan.