The nonprofit organization Worldreader launched in 2010 with a simple and clear mission: to bring digital books to disadvantaged children and their families. In just five years, the organization has expanded its operations to 54 developing countries. Today it offers 15,000 books in 43 languages, and it has reached more than two million readers.
Worldreader is not just on the leading edge of international education and technology. Its growth also reflects a new wave of nonprofit organizations that employ a rapid experimentation method called “lean.” First developed for use in the for-profit business world, the lean method focuses on swiftly turning new ideas for products or services into iterative experiments. Lean practitioners build simple prototypes called “minimum viable products” (MVPs), move quickly to get feedback on these MVPs from constituents, and then develop iterations of their MVPs on the basis of that feedback.
(Illustration by Luc Melanson)
The founders of Worldreader embraced a culture of lean experimentation from day one. Instead of spending significant amounts of time and money launching a full-fledged platform, they developed the smallest-scale version of the platform that they could muster (their MVP) and tested it in the field. In that experiment, which began in March 2010, they introduced Amazon Kindle e-readers to 16 sixth-graders in Ayenyah, Ghana. The Worldreader team hypothesized that the kids would embrace the e-readers, that they would read more, and that their literacy rates would increase.
Focusing on a single school may seem terribly inefficient. But this high-touch MVP approach (sometimes called a “concierge MVP” in lean parlance) enabled Worldreader to find holes in its platform and to troubleshoot problems before investing more time and resources in the initiative. The Worldreader team saw, for example, that the screens on the Kindles kept breaking because kids were sitting on the devices during recess. “We taught students how to care for e-readers,” says David Risher, cofounder and president of Worldreader. “And we took the broken screens to the Kindle factory and asked [people at Amazon] to make the next generation of Kindles more durable—and they did.”
The most important test of Worldreader’s rapid experimentation approach came in late 2011, when the organization faced a critical challenge: The proliferation of basic-feature mobile phones—what we now, in the era of smartphones, call “dumb phones”—across the developing world created an opening for Worldreader to expand far beyond its incremental, Kindle-based growth model. How could it take advantage of that opportunity?
Some nonprofits, confronted with an opportunity of this kind, might shift into a strategic planning mode. They would start by conducting a series of internal debates about what the “right” strategy is, and then they would focus on developing work plans, board presentations, and funding proposals. Worldreader followed a different course. Instead of launching a grand planning and development process, the organization set up a small experiment to test a critical hypothesis—the “riskiest hypothesis,” as lean practitioners call it—of the proposed mobile strategy: Would children in the countries targeted by Worldreader actually read books on a basic phone? To answer this question, the Worldreader team partnered with an app developer that had already created a basic-phone reading app. The MVP version of Worldreader Mobile consisted of nothing more than lists of books and a simple text reader. It had no book covers, descriptions, ratings, comments, or bookmarks. But it had just enough functionality to allow the team to test that hypothesis.
Almost immediately, as it turned out, thousands of users downloaded the app and began using it. Only at that point—after the experiment had verified the hypothesis regarding user demand—did Worldreader enter a formal contract with its app developer and begin to make improvements to the product. Today, more than 185,000 users read books on the Worldreader mobile platform every month. Continuous rapid experimentation, along with a focus on building solutions that work for children and their families, has made that kind of growth possible.
The Art of Lean
In our work, we regularly interact with leaders of social purpose organizations. When they first hear about lean, they often say, “Of course I run a lean organization. I run a nonprofit.” But there is a world of difference between being lean and being frugal. Most nonprofit leaders believe that they have to be frugal: They pay low salaries, rely on donated goods, and work in cheaply furnished offices. Being lean goes far beyond cost-cutting, however. The lean process enables organizations to speed up and focus experimentation in order to reduce wasted effort. Many organizations spend a great deal of time and resources on building solutions that don’t end up achieving their intended impact. Lean accelerates the process of weeding out ineffective ideas and helps quickly validate ideas that show real promise.
Today, the dominant mode of operation in the nonprofit sector puts a premium on strategic planning. It emphasizes processes that generate multi-year plans that cover—often in elaborate detail—a variety of tactics, roles, and outcomes. The old adage “Plan your work and work your plan” captures the spirit of this approach. Planning is important, of course. But by its nature, it discourages experimentation and risk-taking. The emergence of strategic philanthropy has reinforced this emphasis on planning. Under that model, funders encourage nonprofits to propose specific tactics for every desired outcome and to adhere to those tactics over multiple years. The strategic philanthropy model works well for problems with clear, proven solutions, but often it doesn’t work for problems that require new approaches.
The lean model reinvents the traditional strategic planning process. In effect, it offers a new adage to follow: “Plan your tests and test your plans.” Lean practitioners don’t enumerate the precise tactics that they will use because they don’t know in advance which ones will be successful. Instead, they run many small tests and adjust their efforts after discovering what works (and what doesn’t work). Done well, lean helps organizations innovate more efficiently, build new services that meet the needs of their constituents, and develop disruptive solutions to seemingly intractable problems. Lean can be particularly effective as a means of testing and validating revenue models that have the potential to create sustainable, long-term funding streams.
The adoption of rapid experimentation has been slower in the social sector than in the business sector (for reasons that we will explore below). Yet there are a growing number of nonprofits that use lean to support innovation in education, health care, international development, and other fields within the social sector.
The Origins of Lean
To understand the lean method—and its applicability to the nonprofit sector—it helps to understand its origins in the for-profit sector. Part of a broad revolution in the business world, lean belongs to a set of innovation and process improvement methods that also includes Six Sigma, which managers at Motorola developed to enable error reduction; Agile, a flexible and iterative approach to software development; and Human-Centered Design, a solution-building process created by leaders at the design firm IDEO.
Lean has two distinct strains: “lean production” (also known as “lean manufacturing”), a structured method first developed by Toyota more than 25 years ago that applies to complex processes like manufacturing, logistics, and health services; and “lean startup,” a set of principles and practices developed in Silicon Valley over the past decade that help entrepreneurs and intrapreneurs launch new products and services.1 Think of lean production as a way to maximize the efficiency and impact of a good idea, and think of lean startup as a way to figure out whether an idea is worth pursuing in the first place. Although the two strains developed separately and have distinct processes, they share a commitment to identifying clear hypotheses, conducting rapid experiments, and developing new product or service models in response to experimental data.
Over the past decade, several developments—increasing global competition, accelerated technological change, the emergence of big data—have forced nearly every major company to adopt data-driven, rapid experimentation methods in most aspects of their operations. Today, when you buy a pair of stretch pants at H&M or download a new iPhone app or make a purchase from Amazon or click a link on Facebook, you are generating data for a series of experiments that will inform how companies make their next strategic decision. Companies that have incorporated rapid experimentation into their operations range from large corporations like General Electric, Target, 3M, and Xerox to high-growth start-ups like Dropbox, Etsy, and Upworthy.
The Elements of Lean
The lean process, as it applies to the business world, has several core components. We have adapted those components to form a model that suits the way that organizations operate in the social sector. Here we will list the components in the order that they might occur in a typical lean experimentation project. But keep in mind that lean is more circular than it is linear, and the sequencing of components in any given experiment will vary. (See “The Lean Experimentation Process” below.)
Ideation and analysis | With your target constituents in mind (or, better yet, with your constituents in the same room), generate ideas for programs and solutions that you think might solve their problems or help them achieve their aspirations. These ideas are what we call “value hypotheses.” As you develop such ideas, analyze similar programs and solutions that already exist, and figure out how your approach might improve on those offerings. (In the business world, that process is called “competitive differentiation”).
Constituent discovery | Get out of your office and listen to the people you hope to serve. Through surveys and one-on-one conversations, find out what your constituents truly need and want. Put your value hypotheses in front of constituents, and observe how they respond to those ideas. (In the business world, this process is called “customer discovery.”) Done well, constituent discovery will bring to light ideas that you hadn’t considered, and those ideas in turn should lead you back to the ideation phase. Ideation and constituent discovery should complement each other in a rapid feedback loop.
Building | Determine the one or two “riskiest hypotheses” that apply to your idea. A risky hypothesis, in this context, is an assumption that is critical to the success of your idea—an assumption that may, however, prove to be invalid. In the lean process, you should focus your attention on the riskiest hypotheses. To test those hypotheses, develop an MVP (that is, a basic prototype of your idea). Also create a rough financial model for your idea that covers cost estimates and potential revenue sources. In many cases, your MVP will be a small-scale version of your program or service. (One common lean tactic is to customize and test pre-built products. This approach is widespread in the technology world, where there has been a proliferation of ready-to-use tools for developing apps, social platforms, and the like.2) Another option is to build a “paper MVP”—a lean tool that dramatically reduces the cost of testing demand for a program. A paper MVP can take the form of a simple flyer about a not-yet-built program, for example, or a basic online sign-up page for a prospective service.
Testing | Design a plan to validate (or invalidate) your riskiest hypotheses. Then roll out your MVP to a group of constituents and collect data on how they react to it. Be sure to test the MVP in a way that will provide data on metrics that pertain to those hypotheses. Avoid focusing on vanity metrics that might give you feel-good results but don’t actually help you validate or invalidate an idea.
Responding to data | Analyze the results of your test. Did your MVP appeal to fewer people than you had hoped it would? Did it encounter unforeseen logistical challenges? Did you charge a price for it that ended up being too high?
If your data show that you have a flop on your hands, hit the reset button and begin the experimentation process again before investing more resources in your idea. In the lean startup field, that’s called a “pivot.”
If your data show promise, use feedback from the test to build a better iteration of your idea. Then test that version of the idea, and continue iterating and testing the idea until you have verified that it will deliver its intended value. We call this process the “build-test-respond” cycle. (It’s a variation on the “build-measure-learn” cycle used in the lean startup model.)
Scaling up | Once you have an idea that works, use the data that you have gathered during the constituent discovery and testing phases to get buy-in—from your board, your staff, and your funders—for implementing the idea more widely. As you scale up, continue to run experiments on ways to increase efficiency and to create additional value for your constituents.
The Practice of Lean
The Coalition for Humane Immigrant Rights of Los Angeles (CHIRLA) provides a textbook case of how an organization can use lean to identify promising service models. CHIRLA serves people who confront barriers related to language, discrimination, undocumented status, poverty, and limited access to technology. In 2014, the organization was seeking to develop new services that would meet the needs of its community, significantly increase its membership, and provide financial sustainability. (In particular, it sought to create services that would generate at least as much income as they cost to provide). Instead of devoting large amounts of time and money to implementing one or two ideas—ideas that may or may not have worked—CHIRLA leaders launched a lean experimentation process. Over the course of just a few months, they were able to test the viability of more than a dozen potential services.
As part of an ideation and analysis phase, CHIRLA leaders drew on their deep experience with serving constituents to gain a sense of what those constituents might want or need. Using that insight, they developed a list of more than two dozen offerings that they thought had the potential to provide significant value in a financially sustainable way. The list included financial services (such as prepaid debit cards for unbanked immigrants), legal services, English classes, prescription discount cards, low-cost international phone cards, and health insurance products. The CHIRLA team then did market research to learn about similar services that other organizations were already offering.
A building phase came next. The CHIRLA team chose 14 of the proposed services and developed paper MVPs for them. Instead of building a full working version of any of those offerings, the team developed flyers that described each potential service. For most services, the riskiest hypothesis hinged on a simple question: Would people actually sign up for them—and would they pay a price that would make them sustainable? The flyers made the services tangible and allowed the CHIRLA team to begin assessing how much demand there might be for each offering.
In the following phase of its work, the CHIRLA team engaged in both constituent discovery and testing. The team developed a survey that combined general questions with MVPspecific questions that focused on determining the viability of their ideas for new services. Rosamaria Segura, membership coordinator at CHIRLA, led the constituent discovery process. She delved into the lives of local immigrants to understand their needs and aspirations. In each constituent interview, she also tested the riskiest hypotheses for six to eight service ideas. “Inviting our constituents to help us discover what services they really needed was a game-changer,” Segura says. “As the data came in, we responded to the feedback, reconfigured the surveys, and quickly got a sense of whether our ideas were worth pursuing and where our blind spots were.”
Ultimately, Segura completed more than 100 constituent discovery interviews. She and her colleagues now had data on which services people would or would not sign up for. The work of responding to the data began almost immediately. CHIRLA leaders concluded that 10 of the proposed services either didn’t have sufficient demand or would require significant iteration before further testing could take place. Four of the proposed services, meanwhile, had strong demand and merited further exploration. In addition, a review of the interview data led the CHIRLA team to explore several new service ideas.
One of those ideas involved offering classes to help people in the CHIRLA community pass the written driver’s-license exam in California. The state had recently passed a bill that would allow undocumented immigrants to apply for driver’s licenses. According to state records, however, 70 percent of those who take the exam in a language other than English fail in their first attempt. CHIRLA leaders, noting that many undocumented immigrants would fall into that category, saw a new need that their organization could fill. They developed a plan to offer classes on passing the exam and moved quickly to test the viability of that idea.
Initially, CHIRLA staff members thought that they might need multiple sessions to prepare immigrants for the exam. But instead of building a curriculum around that hypothesis, they designed a simple three-hour course—an MVP, in other words—and ran trial classes for 60 constituents. After that single three-hour session, nearly 90 percent of participants passed a mock version of the driver’s-license exam. Clearly, a multiple-session course wouldn’t be necessary. Through the MVP test, CHIRLA also learned that demand was high for the classes and that people would pay to gain access to them. On the basis of those findings, CHIRLA invested resources in curriculum development, a train-the-trainer program, and marketing materials for the new offering.
Lean experimentation enabled CHIRLA to identify a program model that had three crucial features: high constituent demand, demonstrated impact, and financial sustainability. Today, the organization continues to improve the driver’s-license exam class, and it plans to scale up the model in order to serve thousands of immigrants.
The Varieties of Lean
Organizations throughout the nonprofit sector have begun to apply the lean method to their operations. Although lean can help organizations to test and improve a wide array of programs and processes, it is particularly effective as a way to optimize certain core activities.
Demand testing of new ideas | Lean can help an organization determine whether anyone will take advantage of a given program or service. Using lean, nonprofits can test assumptions about the pain points, needs, and aspirations of their constituents.
GuideStar, an organization that gathers and shares information about nonprofits, recently created a user advisory panel that includes about 250 members. The purpose of the panel is to provide rapid, actionable feedback that will help the organization decide which innovations are worth exploring. In its first two months of working with the panel, GuideStar called on users to help test four MVPs, along with 10 ideas that were at the concept stage. Among the products tested were a mobile app, a Charity Check widget, and a product to help organizations prepare their US tax forms. After examining feedback from the user panel, GuideStar is moving forward on the best ideas and is altering or scrapping the others.3
Short-term outcome testing | Through lean, an organization can rapidly test strategies for achieving clearly defined short-term outcomes—outcomes that relate to school attendance, reading rates, job placement, health improvement behaviors, and the like. A-B testing, in which an organization tests alternative approaches on randomized samples of constituents, is a critical tool of lean outcome testing.
At a school in the Los Angeles Unified School District, investigators conducted a rapid experiment to test the impact of parental involvement on student performance. In an experiment that involved A-B testing, the investigators arranged to send some parents of high-school students text and email messages to notify them that their kids had missed an assignment. As it turns out, the students whose parents received the messages experienced performance improvements that were much larger than the gains shown by students whose parents didn’t receive such messages.4
Process efficiency improvements | Lean production, the process improvement strain of lean, can help an organization improve the flow of a process by identifying and eliminating waste. In that way, lean can streamline a program and increase its impact.
The American Red Cross used practices from the Toyota Production System (TPS)—a precursor to lean production—to improve in-the-field training for its disaster volunteers. Through that effort, the organization was able to reduce the time required to register and train volunteers from 3 hours and 45 minutes to just 30 minutes.5 Similarly, the Food Bank for New York City used TPS-based practices to test alternative approaches to serving, seating, and line management. As a result, the organization cut the average wait time for its patrons from 1 hour and 30 minutes to 18 minutes.6
Revenue growth | Given its roots in the business world, the lean method is particularly well suited to testing new revenue-generating strategies. Lean, for example, can help an organization evaluate its plans for fundraising optimization, membership growth, social ventures, and program fee changes.
Environment America is a federation of state-based advocacy groups. Before the start of each major campaign, a small team of canvassers from the organization tests a variety of pitch messages. The purpose of those messages is both to recruit supporters and to generate income for the organization. Following that initial test, members of the team analyze metrics that include the percentage of people who listen to a pitch, the percentage of people who make a contribution, and the average contribution amount. Using those data, they determine which pitch is most effective, develop materials to support that pitch, and then train hundreds of staff members to use it.7
Citizen organizing | Lean enables advocacy groups to experiment with various campaigns and campaign tactics. Through lean, such groups can rapidly test which media channels and which messages actually move people to take action.
SumOfUs, a corporate watchdog group that organizes citizens through online petitions, has more than five million members. Each week, the organization conducts micro-experiments to evaluate dozens of email-based corporate accountability campaigns. It ends up shelving more than 80 percent of those campaigns because the experiments reveal a lack of member interest in them. Then it focuses its resources on the campaigns that its members clearly care about.8
The Challenge of Lean
In 2013, Steve Blank wrote an article in Harvard Business Review titled “Why the Lean Start-Up Changes Everything.”9 It was a provocative title, but it was accurate enough: Rapid experimentation methods have permeated the business world. So why haven’t they spread as widely within the social sector? Does lean not apply as directly to social problems as it does to commercial situations?
There are, to be sure, limitations to applying lean in the social sector. It cannot replace longitudinal research. No form of rapid experimentation, for instance, can test whether an intervention aimed at kids in preschool will affect high school graduation rates. Nonprofit leaders also find it difficult to measure social impact using the kind of cold, hard numbers that lean favors. It’s easy to measure revenue. It’s much harder to measure (say) the effect that a given strategy might have had on changing people’s minds about a social issue. Lean, moreover, can be disruptive to existing programs and disorienting for staff members who are comfortable with established approaches to pursuing social impact. Perhaps most important, lean works best as a tool for testing and improving discrete programs and processes. It cannot serve as a master strategy, and it cannot answer fundamental questions about the theory of change that governs an organization’s overarching approach.
In the business sector, companies have adopted rapid experimentation methods partly in response to increased global competition and accelerated technology change. For-profit companies that don’t quickly adapt to the new environment will ultimately collapse. Organizations in the social sector are generally less vulnerable to such disruptive forces. But as these forces spread across the sector, more and more organizations are likely to adopt the lean method.
For rapid experimentation to become widespread in the social sector, funders will need to embrace new approaches to supporting innovation. Traditional funding processes for nonprofits discourage rapid experimentation by reinforcing risk aversion and an adherence to top-down planning. The rules for submitting grant proposals often require nonprofits to spell out every strategy, tactic, and outcome in a detailed timeline. In addition, many funders have adopted cumbersome grant amendment processes that inhibit efforts to test new approaches.
A few pioneering foundations are trying to change this situation by explicitly funding experimentation.10 Contests and prizes like those funded by the Gates, Knight, and MacArthur foundations have opened up space for experimentation. The use of prizes, however, works only within very limited parameters and sometimes causes more harm than good.11 Fellowships, meanwhile, provide people who have an entrepreneurial mindset with an opportunity to develop and test new ideas. But that kind of individual support is rarely enough to catalyze a culture of rapid experimentation throughout an organization. Funders, therefore, should make sponsorship of lean experimentation a larger part of their ordinary grantmaking process.
The Power of Lean
All too often, the process by which nonprofit organizations develop and launch new products and programs can stretch for months or even years. The lean process, in contrast, enables teams to build and test a new approach in a matter of weeks or even days. If that approach is not effective, teams can pivot away quickly. If the approach needs improvement, they can undertake new iterations rapidly. And if the approach shows promise, they can cite data to prove its effectiveness so that funders can invest in it with confidence.
Various tools are now available that will help nonprofit leaders to engage in rapid experimentation.12 But at its core, the lean process is simple. In 2014, at a Lean for Social Good Summit in San Francisco, one of us (Steve Ma) saw just how quickly that process can unfold. Dominique Aubry, who is now president of Lean Leadership Inc., spoke about the lean process for an hour. She then broke participants into teams and had them develop solutions to specific problems. Next, after they had spent two hours refining their ideas, Aubry told them to leave the conference facility, hit the streets, and interview relevant constituents about their proposed solutions. In just one day, participants went through ideation, constituent discovery, building, and testing—followed by iterating, testing again, and iterating again.
If you’re ready to make the leap into lean, start by testing it out. You don’t need to hire consultants who are experts in lean. (We are consultants, so trust us: You don’t need consultants.) You don’t need to hold a board vote about implementing lean. And you don’t need special grant funding or funder buy-in to run lean experiments. You do need buy-in from your team to embrace rapid experimentation, and you need to be willing to look at the data that you gather and to change your approach accordingly. Once you’re ready, get out of your office and talk to your constituents. Identify your value hypotheses, build MVPs fast, and test them in the field. Then respond to the results—and iterate.
Support SSIR’s coverage of cross-sector solutions to global challenges.
Help us further the reach of innovative ideas. Donate today.
Read more stories by Steve Ma & Peter Murray.