Beyond ‘X Number Served’

(Illustration by iStock/Alisa Zahoruiko)

Metrics matter, but they should always be plural. Focus on the speedometer, ignore the gas gauge, and you’re sure to stop short of your destination. But while the plague of metric monomania can occasionally be an issue in business, it’s an even bigger problem within the social sector. After all, market discipline forces business leaders to weigh tradeoffs between costs and sales, or between product quality and service level speed. Multiple metrics help executives get the balance right, even as they scale.

By contrast, nonprofits too often receive (well-intended) guidance from stakeholders like funders and board members to disproportionately zero in on a single goal: serving the maximum number of beneficiaries. That’s a perfectly understandable impulse, of course. But it confuses scale with just one impact dimension, reach. “We have to recognize that a higher number does not necessarily indicate transformation,” says Lisha McCormick, CEO of Last Mile Health, which supports countries in building strong community health systems. “Higher reach alone does not equate to impact.”

This is a problem because excessively defining and valuing programs by the number of people they serve can give rise to unintended consequences. Nonprofit leaders can find themselves discussing how to serve more people through “lighter touch” models or debating ambiguous metrics like “reached” or “touched” to expand participant numbers (while fighting uneasiness about the potential adverse implications for program quality).

This is a slippery slope.

The right way to expand a nonprofit’s impact is to build programs on three pillars: breadth, depth, and durability. Scaling means advancing all three of these dimensions simultaneously. And while the specific metrics an organization uses will vary, some version of each of those elements must be measured and advanced together to stay on track.

Let’s start with the example I know best, Generation, a global employment nonprofit network that recruits, trains, and places adult learners into careers that would otherwise be inaccessible to them. At Generation, we define “breadth” as the number of learners who graduate from our programs—the most basic unit of impact—but we also use depth to capture the employment and income outcomes of those graduates within three months of program completion. Finally, we also track durability, or how well these outcomes persist over time: We follow the financial and personal well-being of our alumni for up to five years after they graduate by measuring their ability to cover daily financial needs while saving for the future and taking care of their mental and physical health.

Each of these pillars is equally important, and our goal is to move all three forward together. The challenge, however, is that increasing breadth can often come at the cost of depth and/or durability. For example, at Generation, we could choose to train tens of thousands more people annually without pre-confirming job vacancies for them to fill, but our employment and income outcomes would fall precipitously. Any nonprofit that provides multi-step services over time, and aims to improve quality of life outcomes, faces similar trade-offs. By contrast, nonprofits that focus on delivering pure point-in-time service (e.g., vaccinations, home construction, food banks) may have more freedom to anchor on breadth.

Another social sector leader that has come to believe in the power of multiple metrics is the Community Health Impact Coalition (CHIC), a network of health delivery organizations in 40+ countries working to ensure community health workers get fair pay, supplies, and ongoing support. Convinced that “momentum with stakeholders is created by shared progress,” CHIC’s executive director, Dr. Madeleine Ballard, has collaborated with her member organizations to align on a minimum viable common dashboard. Their dashboard has three types of metrics, echoing aspects of Generation’s: coverage, quality, and speed. Coverage shows the percentage of a community that can access primary health services. Quality tracks whether those community health workers follow proper treatment and referral protocols when presented with symptoms (e.g. a fever). Speed assesses how quickly patients receive care after symptoms first appear.

Initially, funders in the community health space cared most about tracking the absolute number of patients “reached.” Replacing this perspective with the concept of coverage, says Ballard, added the vital notion of “a denominator,” since success in community healthcare is driven by the share of a population served; “our goal is for coverage to be 100 percent.” Speed was the hardest metric on which to achieve alignment with stakeholders. To reinforce its importance, CHIC began sharing data on how rapidly diseases like malaria progress (most children who die from malaria are killed within 48 hours of symptom onset).

Getting metrics right is a dynamic process that evolves over time. When South African nonprofit Harambee began its work on youth employment ten years ago, it too focused on job placement and volume of graduates. Later it turned to tracking income because so many graduates were going into freelance gig economy roles. Lately, Harambee has taken a systems focus and begun measuring whether the South African labor market is becoming more inclusive, for example, by dropping rigid degree requirements for job roles and creating alternative education-to-employment pathways. “Some of the most important stuff is hard to measure, and we have to be willing to sit with that complexity,” says Sharmi Surianarain, Harambee’s chief impact officer.

Multiple measures inevitably make educating stakeholders harder, especially when tensions arise between metrics. Consider community health. Workers in remote, rural areas environments with bad roads and minimal public transport still aspire for 100 percent coverage. But it can be challenging to score as highly on speed as colleagues working in a crowded city. “Avoid the simplicity of glomming onto a single number,” advises CHIC’s Ballard. Instead, “we spend time explaining to stakeholders that we cannot flatten the nuance between service” in rural versus urban areas.

Last Mile Health faces the same challenge in a similar way. When stakeholders suggest that a sharper focus on cities could pump up coverage numbers, Lisha McCormick responds with the story of how the Ebola virus came to urban areas precisely because, after first appearing in rural areas, it was allowed to grow undetected and unchecked. Balance is everything.

Striking the right balance begins with how we, nonprofit leaders, communicate both internally and externally. While it may be snappier to say things like “XX thousand served annually,” calling out just one pillar can be both insufficient and misleading. To ensure that stakeholders are inspired by their true societal impact, nonprofit leaders must consistently—in everything from team meetings to board meetings, public speeches to annual reports—define, measure, and underscore metrics that capture the full range of breadth, depth, and durability.

As part of this process, thinking clearly and precisely about how best to measure each pillar is critical. As CHIC found by insisting on a “denominator,” breadth is a dimension often better captured by a ratio than by a numerical tally. At Generation, in addition to graduate volume, we also measure our breadth by calculating the percent of annual vacancies for a target profession in a target city that are filled by our graduates. It’s far more meaningful to state that we have filled six percent of all annual entry-level java developer and software engineering roles in Sao Paolo (which is true, by the way) than to declare we placed 475 graduates. As we homed in on this metric, we have come to believe two things: First, a market share of 10 percent is a maximum goal for Generation to achieve within any profession/city combination; and second, hitting this target is the best proof that we have successfully shifted employer behavior towards hiring diverse profiles.

What gets measured, as the saying goes, gets managed. But if the real goal is growth, then remember this: Growth is merely the outcome of all the inputs that drive it. By investing the time and effort to identify and deploy the right set of multiple metrics, social sector leaders will increase alignment with key stakeholders, from employees to board members to funders, about the type of growth in impact that really matters, and how best to scale it.

Read more stories by Mona Mourshed.