Design Thinking

Getting Beyond Hype: Four Questions to Predict Real Impact

A simple way to think about investment and design.

Through a kind of magic I don’t fully understand, some emerging poverty solutions—products, services, technologies—become shiny new objects that trigger something akin to a feeding frenzy. Celebration at CGI and SOCAP and TED, breathless articles in Wired and Fast Company, endorsements from celebrities, awards of all kinds—all that stuff is great if the thing has real potential for impact. Too often, though, the appetizer becomes the entrée and hyperbolic celebration displaces systematic evaluation.

That’s a shame, because we really do need to sort good ideas from bad, and it ought to happen before we stoke the fires of publicity. At Mulago, we’ve spent a lot of time trying to figure out if start-ups with a new “thing”—a product, service, or technology—are likely to create real impact in the lives of the people we’re trying to serve. Over time, we’ve evolved a set of four questions that help us make better predictions. Because we’ve been thinking a lot about cookstoves and indoor pollution lately, I’ll use improved cookstoves to illustrate how those questions work.

1. Is it needed?

Given the nature of the thing, is there a profound social impact to be had? Every thing ought to have a mission (preferably in one that’s eight words or less). If you can’t figure out a clear mission that matters, all you may have is a solution in search of problem—and some things ought to be killed while they’re young.

We might reasonably ascribe various missions to better stoves, including: improve health, prevent deforestation, save time and money, and decrease carbon emissions. Since indoor air pollution is a major killer and bad stoves are the major contributor, we think that the mission to “Prevent Respiratory Disease in Poor Families” is a compelling one and rises above all the others (especially since the evidence connecting the other potential missions is considerably more sketchy). So the answer to question one is yes, absolutely, we need better stoves.

2. Does it work?

If the thing is deployed as designed, under the conditions for which it was designed, will it have real impact? In the case of stoves, this is mostly about emissions: Does a given stove reduce pollutants enough to drive a significant improvement in health? Recent research indicates that most improved cookstoves don’t. It turns out that you have to reduce more than 85-90 percent of pollutants before you see any real reduction in respiratory illness (Burnett et al, 2014). A stove that emits 50 percent less pollution doesn’t accomplish much. To date, the only stoves that make the cut are the expensive “forced-draft” models that use a built-in fan to stoke a much more efficient burn. The answer for most stoves, however admirable the effort may be, is no. Really.

3. Will people use it as designed?

Behavior, behavior, behavior. Despite all that user-centered-design, iterative-rapid-prototyping stuff, people use and abuse things in ways that leave even the best designers scratching their heads. Too many—way too many—products have made their way into wide distribution without evidence that people use them as intended. Successful things, though, usually have these qualities in common: They fit well with local customs and culture, are easy to use right and hard to use wrong, and need little maintenance (and when they do, they’re fixable and you can get parts).

Everyone has heard about stoves that people use improperly—or not at all—but field researchers have handed the stove industry another fun surprise: Even when people do use that wonderful new stove, they often use the old one too (Ruiz-Mercado et al, 2011). It’s become known as “stove-stacking,” and you see it when you visit homes—there’s the new stove, fired up and roaring, and there’s the old three-stone fire still belching smoke in the corner. Oh no! If you want to fulfill the mission—prevent respiratory disease in poor families—not only do you have to provide an expensive stove, you might have to provide two! It kind of sucks, but that may be what it takes to get to yes.

4. Will it get to those who need it most (a lot of them)?

Nothing matters if the thing doesn’t reach the people it is intended to serve. If you don’t have a reality-based idea about distribution, you should hold off on the design process until you do. Three things matter above all else:

  1. Price. If it’s too expensive, they won’t buy it. Knowing the customer’s price point is your first order of business, and you need to design the thing to hit that number.
  2. Distribution channel. Poor people live in a world of market failure, and there are a finite number of ways to get things to them—the hardest is to set up your own network.
  3. Sales. Somebody has to make the transaction happen—who are they and how are they going to do it?

And, of course, if the thing costs more than about 20 bucks, you’d better think about financing, too…

This is where stoves have always struggled: The affordable ones are inadequate, and the good ones are unaffordable. Even with the (unreliable) assistance of carbon credit subsidies, stoves that can fulfill our health mission are still too expensive. Emerging technologies will almost certainly lower the price of forced-draft stoves, but for the time being, they’re mostly out of reach for those who need them most. Financing can help, but poor people often have more urgent things to do with available credit.

One organization working on the stove problem that gets to “yes” to all four questions is Inyenyeri, a Rwandan company that leases ultra-low emission, forced-draft cookstoves to households at a nominal cost (about $7 per year), and sells customers the fuel pellets to burn in them. The pellets cost less than charcoal, and people who can’t afford to buy them can trade biomass for pellets. Inyenyeri leases families two or even three stoves to account for stacking. It’s still early days and there are a lot of moving parts, but it’s headed in the right direction.

The four questions are unforgiving; getting a no on any one of them means you won’t pass go on your way to impact. That doesn’t mean that a no answer should lead you to abandon the whole effort, but it should refocus your efforts on getting to yes. Stick with R&D, and avoid the big splash until you’re pretty sure you’re there—it’s the least we owe the people we serve.

Read more stories by Kevin Starr.

Tracker Pixel for Entry


  • Cathy Lada's avatar

    BY Cathy Lada

    ON September 3, 2014 09:17 AM

    Kevin - excellent points, all. I agree wholeheartedly that beyond proof of concept, all of these great social innovations we’re hearing about need to pass some rigorous reality checks. I think the four you’ve enumerated above capture the essence of what I’m seeing in scholarly literature, and, more often (and thanks to people like you) in popular literature. The importance of understanding - deeply - local culture is often de-valued, and I’m so pleased to see it mentioned here. The only thing I’d add are 2 items to your 4 questions are in #4 - first, manufacturing. I’m not sure if you’re assuming the product (stoves in this case) are made abroad and brought in, or being manufactured locally (which adds jobs, etc. but adds training, CAPEX, and labor concerns, among others). When you factor in manufacturing, there are also supplier (esp. availability and quality of components) concerns. The second thing I’d add to #4 is maintenance and repair. How will replacement parts be fabricated and/or made available? Who is qualified to repair the item?

  • Debbie Hall's avatar

    BY Debbie Hall

    ON September 5, 2014 10:32 AM

    Excellent story, Kevin.  I’ve seen numerous examples of ‘improved cookstoves’ and have always wondered if they are improved enough!

    You article brought to mind a product I noticed on a trip this summer to Kenya.  At the home of a struggling farmer, a “LifeStraw” hung on the wall of the hut. I inquired about it and learned it was handed out for free by a nonprofit working in the community.  I asked them mother if she uses it and the answer was “yes, when I have time”.  But most of the time she’s in a hurry to use the water and doesn’t take the time to filter it first. The product is facing issues two points you raised. Point #1, does she perceive the need for the product? While she understands clean water is important, it appears the need is not a high enough priority to always purify the water. And point #3, can she use it as designed? She feels the rate at which water comes through the filter is not fast enough for her.

  • BY David White

    ON September 5, 2014 11:45 AM

    very well stated….makes me consider our own work.  Thank you.

  • BY Steve Hardgrave

    ON September 10, 2014 11:13 PM

    Here, here! Spot on Kevin. So much of what will make a difference is the unglamorous hard work, persistence and tiny adjustments that don’t make headlines. We definitely need the pendulum to swing from talk to action.

  • BY Brook Manville

    ON September 17, 2014 08:24 AM

    Another question to ask is whether change will be based on working with a network or in fact a community…the difference matters….

  • Number 3 is so important when predicting real impact. Many people’s actual behavior is very different from what they say they want out of a product/service. Understanding your end user, pivoting, and providing true value is critical… This follows a very lean startup approach. Great article!

  • BY Samer Abdelnour

    ON October 13, 2014 04:53 AM

    Excellent post! I too suggest that Number 3 is a ‘black box’ so to speak, one that requires much unpacking. Simply stated, if stoves are used ‘as designed’ this does not equate the fulfilling of a social mission, nor does it predict impact. As you and others rightfully note, cookstove designs are often very technically sound, however,  they in no way predict user behaviour.

    Related to this is the current debate in the world of improved cookstoves relating to the tests used to legitimate stove designs (for example, the standardized in-lab water boiling test). Some important questions to consider: What do these tests actually measure/predict? Are in-lab tests capable of predicting behaviour or even health/environmental outcomes? How do in-lab tests compare with those that examine stove use in the ‘field’ as they are used in everyday life?

    More emphasis needs to be placed on how stoves are legitimated, and on monitoring and evaluation. This demands a refocusing away from the technical, marketing, and supply chain aspects of the cookstoves industry, to longitudinal measures of user behaviour and health/environmental outcomes. Predicting impact requires measuring impact over time; this will also help to ground the (sometimes incredible) claims of the problems stoves are purported as capable of resolving (such as sexual violence in conflict zones).

  • BY willem kuipers

    ON February 2, 2016 03:00 AM


    very spot on and our tests in tanzania show exactly your points.  we are progressing in the same way to help our neighbouring villages

  • BY Reid Harvey

    ON February 17, 2016 08:42 AM

    About “Getting Beyond Hype: Four Questions to Predict Real Impact,” where do we answer the four questions, and submit this??  We have an insulating pottery rocket stove, which meets all four requirements.

Leave a Comment


Please enter the word you see in the image below:


SSIR reserves the right to remove comments it deems offensive or inappropriate.