What Gets Measured Gets Done

The debate about the practice and value of strategic planning and evaluation in the philanthropic sector is robust. Critics ask whether strategic philanthropy that attempts to predict outcomes is effective for solving complex social problems, and whether such approaches limit flexibility and stifle innovation. Last summer, John Kania, Mark Kramer, and Patty Russell argued that while we can apply strategic philanthropy to both simple and complex problems, using a more emergent approach that allows for regular modification of strategy might work better for complex problems. On the other hand, supporters feel that when crafted well, a predictive strategy that is tied to specific outcomes allows for evolution; Phil Buchanan argued precisely this point last year when debunking myths related to foundation strategy. 

One important piece to add to this point is the critical step of quantifying one’s “denominator”—the full number of people who need a service or are affected by a problem—to understand a problem and make a meaningful difference. Stephen Isaacs and Paul Jellinek have written about this and explain that once foundations understand the denominator, they can develop targeted measures of progress to use in determining whether a program strategy is working or needs refinement. Sometimes referred to as program indicators, metrics, outcome measures, expected outcomes, or goals, targeted measures of progress outline very specific outcomes that relate back to strategies and goals. In our own work at the New York State Health Foundation (NYSHealth), we believe that a strategic-but-flexible approach to grantmaking leads to the best outcomes; we also adhere to the old adage that what gets measured gets done. We have seen that well-designed program strategies are framed by the magnitude of the problem, and that program staff and grantees need flexibility in how to apply that strategy over time.

Our work to improve the health for people with diabetes provides a good example, and illustrates how a predictive strategy tied to an ambitious targeted measure of progress enabled us to make both difficult and innovative funding decisions that helped us reach our goal. 

When we began in 2007, nearly 1.8 million New Yorkers—almost 1 in 10—were estimated to have diabetes. This number of people influenced how we framed our strategy and how we selected a targeted measure of progress. We decided that one central aspect of our strategy would be to help large numbers of primary care providers deliver the best care to patients with diabetes and help patients achieve good health outcomes. After discussions with primary care provider groups and insurers, a strategy emerged of accelerating participation in national recognition programs for diabetes care. Housed at the National Committee for Quality Assurance (NCQA) and Bridges to Excellence (BTE), these programs indicate that clinicians are achieving good outcomes for patients, such as controlled blood sugar, blood pressure, and cholesterol levels.

Using Rogers Diffusion of Innovation theory, which explains how ideas or products spread through a specific population or social system over time, we set a goal to reach 20 percent of New York’s primary-care physicians (the target audience for behavior change) to spur a better standard of care across the entire state. The final targeted measure of progress was: to increase the number of New York State’s primary care physicians achieving recognition for good patient outcomes from 149 to 3,000 by the end of 2013. These providers cared for approximately 600,000 New Yorkers with diabetes.

Initially, the foundation awarded grants to four statewide associations of primary care physicians. The idea was for them to call their members’ attention to the issue of diabetes and provide technical assistance to help them achieve diabetes recognition. This grantmaking approach fell short: By 2011, only 1,000 primary care providers had achieved recognition; after our association partners reached the early adopters, several of the associations struggled to engage their members in the program. If we were going to reach 3,000 physicians by the end of 2013, we knew we needed to make radical programming decisions—but we didn’t lose sight of our original target. 

The solution was an innovative pay-for-performance grants program, which targeted both small and larger physician organizations. Called Meeting the Mark: Achieving Excellence in Diabetes Care, this open request for proposals supported primary-care groups interested in attaining diabetes recognition. Grantees were eligible for up to a $2,500 incentive payment per physician who became recognized. If a health center employed 100 primary care providers, they would be eligible for a grant of $250,000; if a small group practice of 10 physicians was interested in participating, they would be eligible for a grant of $25,000. The foundation waited to pay out nearly 80 percent of the award until the grantees actually achieved their proposed goals. We were not prescriptive in how the grantees should spend funding, what strategy they would use, or what types of partnerships they could form as part of this initiative. All that mattered was the outcome, and we believed it was important to put some of the risk on the grantees so that they had strong incentives to carry out their plans. 

While we maintained the technical assistance component, the new approach resulted in a large uptick in providers achieving recognition for excellent diabetes care. Adhering to our strategy of accelerating participation in national diabetes recognition programs ultimately paid off: By the close of 2013, 3,005 primary care providers were recognized for delivering optimal diabetes care. 

The key to all of this was that we attached the payment (the grant award) to the outcomes. We have seen few foundations pursue pay-for-performance approaches. Moreover, foundations too often are so beholden to the process and milestones within the process that they can lose sight of the ultimate goal. The fixation on process or on a specific program may limit foundations’ and grantees’ opportunities to leverage their talent, creativity, and external resources to experiment with options to achieve a particular goal. 

Without a clear, specific, measurable goal in mind—which we stated publicly to hold ourselves accountable—it seems unlikely that NYSHealth would have achieved the success it did in our work to improve the health of people with diabetes. By identifying and staying relentlessly focused on our desired outcome, we had the freedom to change course rather than cling to an initial strategy that wasn’t paying off. 

What gets measured gets done, and it gets done better when foundations focus on the “what” and maintain some flexibility when it comes to the “how.” 

Read more stories by Kelly A. Hunt & Jacqueline Martinez Garcel.