Getting Results Data Off the Dusty Shelf

For charities, the why of impact measurement is generally clear—most funders ask charities they support to provide some evidence of impact. Perhaps unsurprisingly, new research from New Philanthropy Capital (NPC) reveals that 88 percent of the 114 funders we surveyed believe that impact measurement makes charities more effective, with one respondent claiming, “It’s part of a culture of learning and reflecting.”

But how funders should approach impact measurement is a slightly different story: It involves the demands they place on grantees, as well as efforts to measure their own impact—both tricky enough tasks on their own.

We know that funders (and charities) are investing more and more in impact measurement, with 82 percent of respondents planning to increase their measurement activity over the next 3 years. But does this evidence of take up translate into a culture of learning and reflection among funders?

We found that although many funders collect a huge amount of information about what interventions work, this data often doesn’t go anywhere—only a third of funders use evidence of impact from grantees to influence others. Funders often doubt the quality of the evidence provided by grantees and are therefore cautious about passing it on. Of course, it follows that if capacity around impact measurement improves among grantees, then funders would more likely use and share that data.

But the challenges of impact measurement also depend on the type of funder. Generalist funders are responsive to needs that charities identify; they don’t set their own goals. This flexibility affects their ability to use impact measurement— by supporting a wide range of charities and interventions, it becomes harder to compare grantees, and aggregation is almost impossible. Funders with more targeted programs can make more comparisons between grantees, because they are funding similar types of work. These funders can extract and share lessons from their grantees.

Funder type also affects internal impact measurement. Responsive funders find it hard to aggregate data in a meaningful way to describe their impact on a particular social issue—it’s like comparing apples and oranges. However, those that have targeted funding programs can do this—especially if they are willing to request that their grantees use a shared system to measure the same outcome in the same way. Funders working toward a well-defined goal—eradicating a disease, for instance—find it easier (although it’s never easy) to measure their impact on a social issue.

Of course, the charity sector thrives in part because there are different types of funders—generalist and focused, responsive and proactive. We therefore recommend that funders’ adapt their approach to evidence depending on the type of funding. Responsive funders should focus on their approach, asking questions such as, “Am I actually a flexible, needs-driven funder?” And they should share this information, where possible, with other funders to better understand where the need is coming from. For targeted funders helping many organizations work toward a similar goal, impact measurement should concentrate on building the evidence base for interventions in that sector. Goal-orientated funders should seek to build on this to provide really robust evidence around what works—ensuring that they share this—and dig deeper into whether they are making progress on the social issues they are trying to address.

To get to a stage where funders can better use impact measurement, we clearly need more investment in the capacity of grantees. We also need to help funders with the task of impact measurement, and provide guidance and tools for action and learning—the Inspiring Impact program, run by NPC with an ACF-strand for funders, is already trying to respond to this demand. And we need better platforms for sharing information, so instead of sitting on a dusty shelf, funders can put data they collect to better use.

Read the full report here.

Read more stories by Angela Kail.