Just Do It

When it comes to measuring impact, perfect is often the enemy of good. Nonprofits and governments often aim too high and struggle to get started as a result.

Most organizations understand that by measuring impact they can identify problems, improve their programs, and attract funders, but many organizations still are not doing it.

In our work with nonprofits and governments around the world, we’ve heard leaders articulate a number of barriers to impact measurement. Three of the most common ones are: 

• It’s hard to get reliable data on our impact.
• We won’t know if our program caused any changes.
• We don’t have the resources to measure our impact.

The most important first step in overcoming these barriers is to put the ideal of perfection aside and just get started using existing tools.

Getting accurate data on impact

Program managers face two challenges: choosing a number to indicate their impact and getting accurate data on program participants. If your program aims to reduce crime or improve educational outcomes, for example, you may feel that arrest rates or standardized test scores do not really capture the impact your program has on the people you serve. And while it may be easiest to get the data by asking participants what their school grades are or whether they have been arrested, government databases may be a more reliable source.

There is often a trade-off between the ease of getting information and its reliability. Choose a data source that is reliable enough to enable you to make decisions. For recidivism, this information will likely have to come from the government, and you will need to sign an agreement to get the data from the relevant agency. If you are assessing drug use, you can get the information from participants using a validated assessment tool, such as the Global Appraisal of Individual Needs (GAIN).

Knowing if your program is making a difference

If 60 percent of your program’s participants score well on maternal health indicators, how do you know whether that is a positive result? Even if you find out that your participants score higher on reading tests after completing your program than before, can your program take credit for that? How would they have done without your program? Having a randomly selected control group is ideal but expensive, and it requires that the program randomly deny services to some beneficiaries. You need to have a baseline of performance to compare against, even if it is imperfect. Your ideal comparison will consist of people who look like your participants and who are going through the same system at the same time, but who are not getting your service. This might be people who didn’t get into the program because of capacity. If you can’t get that, you sometimes can look at data for the same population before the program began or data on people from a similar neighborhood. Where do you get this data?

• Published reports from government agencies (you may need to ask the agency to isolate people from your jurisdiction or your age group)
• Data from researchers who have studied the population
• A special data request to the relevant agency

The money to measure impact

If you wait until you have enough money to hire an outside evaluator, you may never get started. Don’t wait: Start measuring impact yourself, without high-tech data systems and expensive experts (although you should ask an expert to review your plans before starting). When the New York Police Department started holding itself accountable for crime rates, precinct commanders wrote down their crime stats on pieces of paper and carried them to police headquarters; from there, the process grew into a well-regarded system that holds commanders accountable for major crimes in their precincts, “CompStat.”

Do what you can—and get the cheapest help you can. One organization we worked with asked its clients to complete a survey using free online survey software before they left the office. You can also get data and/or analytic support from government agencies, a pro bono lawyer, or graduate students. Foundation funders may be willing to make a small investment because it will help increase the impact of their grants.

If you are a nonprofit or government leader who has found impact measurement daunting, you are not alone. Aiming for the perfect impact measurement system has frozen many organizations in their tracks. Our advice to you is: Just do it! Pick one or two impact measures, develop a credible baseline, and start collecting and analyzing the data as best you can. Once you start using the measures, you can modify, add to, and systematize them. By starting small and building from there, you can reduce fear, build skill, and better deploy your resources so you can achieve your mission in the most efficient way possible. Good luck!

Read more stories by Liana Downey & Megan Golden.