As the previous articles have made clear, innovation is essential to developing the breakthrough ideas and practicable solutions that contribute to social progress. The process of innovation is very difficult, however: full of challenges and characterized by paradoxes. It is understandable, therefore, that people look for checklists, normatives, and practices they can adopt and follow—or shortcuts and workarounds that will enable them to avoid getting involved with innovation altogether. Experienced leaders, however, know that innovation is necessary to further social progress, and successful innovators know that the challenges and paradoxes inherent in the endeavor cannot be avoided.

One way to smooth the path of innovation is to be alert to the most common challenges that arise. Interestingly, some of the most onerous barriers to innovation—especially in a global, cross-organizational context—have less to do with the skills of the actors involved than with distinct paradoxes that are embedded in the process. As with any paradox, these contain conundrums and sometimes fly in the face of conventional wisdom. At the Rockefeller Foundation, we have identified three paradoxes in our work with innovators around the world.

1. How to pursue innovation without falling prey to “cultification.”
2. How to collaborate without being derailed by compromise.
3. How to scale up breakthrough inventions within the established conventions of organizations.

These paradoxes can be managed, but they are stubborn, and they can lead to a state of innovation dissonance—a palpable tension between the regularity of the status quo and the uncertainty that comes with change. The dissonance shows up in many ways. People find themselves unsure about how to behave in certain unaccustomed situations. They may have to shoulder new responsibilities and therefore make uncharacteristic missteps. Or they may feel concern, even anxiety, about the nature of new relationships.

The presence of these paradoxes, however, should not make us shy away from the critical need to innovate and collaborate, because the benefits to social progress are inarguable. What’s more, handling the paradoxes often leads to institutional and individual growth.

The Paradox of Cultification

The many proponents of innovation have done an effective job of making the case for innovation and also of defining associated issues and bringing to light practices and methods. This focus is laudable, but ironically it has also produced, through its very success, a kind of cult around innovation, its methods, and its most successful practitioners. As a result, innovation has become the default mode for people in almost any situation where some change or improvement might be desirable. Innovation is now so fervently favored that it almost cannot be questioned.

Experienced leaders know that innovation is necessary to further social progress, and successful innovators know that the challenges and paradoxes inherent in the endeavor cannot be avoided.

We all know, however, that a large percentage of our time and our organization’s energy is necessarily spent on activities that don’t require innovation. We also know that scaling up an innovation depends on the operation of relatively routine tasks and processes, many of which are in place and already have been proved effective.

An example of this paradox is the experience of the mHealth Alliance, cofounded by the Rockefeller Foundation. The mission of the alliance is to improve health by championing the use of mobile technologies—most typically cell phones—to support a wide variety of health-care-related activities, from the collection of patient information to the integration of systems and platforms.

There is so much potential in the mHealth Alliance that there has been an explosion of new projects and pilot programs. The proliferation of programs has reached such a level that Patricia Mechael, executive director, says they are struggling with what she calls “pilotitis.”

Why is this a problem? Because organizations expend so much of their energy in the conceptualizing and testing phases that execution—financing, manufacturing, scaling up, marketing, and managing—gets less attention. As a result, a high percentage of initiatives do not progress beyond the pilot stage.

This is precisely what happened in the mobile apps industry in Uganda, where pilotitis became such a problem that even the few projects that did come to fruition failed to catalyze systemic change. Finally, in early 2012, the Ugandan minister of health declared a moratorium on all electronic health care pilots until other critical issues—such as coordination, interoperability, ownership, and institutional structures—could be resolved.

Some organizations in the mobile health industry have avoided falling under the spell of the innovation cult. Switchboard, for example, is deliberately focusing on execution issues rather than the invention of yet another mobile app. The nonprofit has partnered with existing mobile operators to network health care workers in Liberia and Ghana. Switchboard can now scale up and replicate its success in new areas, such as Tanzania, where it is developing what may be the largest network of health workers in the developing world.

The lesson from mHealth, Switchboard, and others we have studied is that in organizations where innovation has achieved cult status, execution takes a back seat to invention. To succeed in the face of this paradox, we have found that there are two paths to follow. First, organizations can link pilot approval phases to the solving of associated executional demands. By so doing, they will heighten the status of non-invention activities and reduce the number of shooting-star pilots. Second, leaders can choose to focus their organization’s efforts solely on execution and let others do the invention. They can then assert their well-functioning operational capabilities as an essential asset to the broader process of innovation.

The Paradox of Collaborative Compromise

Organizations almost always pursue innovation when they need a solution to a complex, rather than a simple, problem. The search typically involves multiple players with different experiences and approaches, multiple commitments to different groups affected by the problem, and unacknowledged and intertwining problems.

In the face of such complexity, organizations often look to their leaders to set priorities and make judgments about how resources should be allocated. Ideally, a collaborative approach—in which the diverse resources, disparate views, and separate goals are integrated—can yield an innovative solution that is greater than the sum of its parts. Often, however, the collaboration becomes a competition for resources and a protracted negotiation over priorities. This is particularly true when senior leaders turn their attention away from the collaboration and hand it over to deputies after the excitement of launch is over. Factions may form and positions may harden. The result is rarely a solution, but rather a compromise, and often at the lowest common denominator.

One organization that has had to work through this paradox is Global Pulse, a UN initiative and Rockefeller Foundation grantee, that seeks to encourage UN agencies and member governments to make greater use of Big Data. The initiative required the UN’s bureaucratic wheels to turn in a new way, because real political and technical constraints had to be overcome. Not only can it be a technical nightmare to share real-time data that exist in different forms and locations, it can cause political problems. UN agencies work through member states, and if data shared by a UN agency have not gone through the proper national government channels and are somehow misused, it can cause problems for the UN agency.

Early on, Global Pulse recognized that the main challenge they faced was not skepticism about the potential of big data, but rather concern about the risks involved in collaboration. Who will decide what? How will resources be allocated? How will sectors and governments be prioritized? How will we protect our IP, our reputation, and our strategy? Who will come out “ahead”?

So the leaders spent a good deal of time getting early buy-in from the participants. Once there was sufficient buy-in, a core data and research team was formed. The members were decision-makers—called secondments—from UN and government agencies with domain experts in fields ranging from transnational crime to early childhood education, as well as volunteers from partners in the private sector and academia, including statisticians and technical experts in big data analytics.

The role of the secondments was to help the technical experts understand the on-the- ground issues; the experts were there to help the secondments master the concepts of big data. Together, their purpose was to integrate the multiple views, goals, and approaches into superior, workable solutions. “The idea was to create a space conducive to open and active debate,” says Robert Kirkpatrick, director of Global Pulse. “We maintain minimal hierarchy on the team so that good ideas can flow free.”

Global Pulse created a series of proof-of-concept projects to demonstrate the opportunities presented by big data. Each project involved interdisciplinary teams, typically including a secondment, a partner expert, a data scientist, a culture and language expert from the relevant country, and a project manager who could “translate” between and among the players.

One question they explored was whether there were real-time digital data sources that could serve as a proxy for actual food prices. If so, that capability could help decision-makers gain insights into food price inflation, day by day rather than month by month. In consultation with colleagues at the World Food Programme, the project team formulated preliminary research hypotheses and posed them to its partners. Then, together with Price Stats, a company that daily tracks the prices of five million products advertised online, they completed the project.

This and other proof-of-concept projects demonstrated what might be possible through the innovative use of big data. Global Pulse’s leaders spent several months presenting the projects to colleagues in the UN. Soon Global Pulse was being invited to give presentations to individual units within UN agencies. These presentations led to a much richer understanding of how big data could be applied to specific lines of work. As a result, colleagues throughout the UN now seek to co-develop projects with Global Pulse.

The lesson from Global Pulse and other initiatives we have studied is that collaboration can be derailed by individual, disciplinary, and organizational concerns—all of which can be valid. Leaders who choose not to make executive decisions may do so in a genuine belief in the power of collaboration, but they may not fully understand the real difficulties it can create when a committee-created innovation comes to be translated into on-the-ground execution. No wonder collaborations often turn into elaborate rituals of bartering and protectionism.

Proof-of-concept programs like those at Global Pulse can quickly build trust, create knowledge, build collaboration skills, and avoid compromised solutions. One needs the right combination of people to make these programs work. These are usually people who are skilled translators and are willing to engage in battle over substantive issues and still respect one another’s goals.

The Paradox of Invention Within Convention

A third paradox of innovation involves the disconnect between the process of invention—developing the core, original breakthrough—and the effort required to scale it up and integrate it into a larger, conventional system. The skills of the inventor are rarely those of the integrator.

This is a particular problem in large organizations that have optimized themselves around a founding innovation. They know they must continue to innovate, but the proven methods of innovation go against the conventions of how they currently operate. Their organization is not constructed of small, flexible entities with porous borders through which people, ideas, and resources can easily flow. So they often pursue innovation by forming separate innovation teams, such as ad hoc units, skunk works, one-off projects, or partnerships with outsiders.

Even when these innovation efforts are successful, the organization may find them challenging to manage. The organization wants to encourage and support these initiatives but it also wants to protect its organizational assets, further its own departmental interests, and not neglect its current operations. The tensions intensify when the parent organization wants to bring its nascent innovation back into the fold and scale it up—without mangling the invention and without any disintegration of the methods and structures that have made it successful.

Just as the actors at Global Pulse worried that collaboration across entities could threaten their situations, actors within a large organization have similar concerns when the space probe tries to dock with the mother ship. How will this solution affect our current ones? How can we be sure this grain of exotic sand will become a practical pearl and not just an irritant to a system that already works well?

Root Capital faced this paradox and has figured out an effective way to pursue innovation outside its main organization and, when the invention is ready to scale up, to bring it inside and take advantage of the parent’s superior resources and processes.

Root Capital is a nonprofit social investment fund that lends capital, delivers financial training, and strengthens market connections for small and growing rural businesses in Africa and Latin America. It created Root Lab as a way to be both freer and less ad-hoc about innovation. The initiative is driven by an R&D team that is based in a physical laboratory, and whose members also include innovation officers in each of the field offices. The lab has a dedicated budget, but it does not operate with the same risk-reward expectations as other units in the company. Nor is the lab required to follow the same processes.

The field officers are focused on finding new opportunities, piloting innovation loans, determining what went right and what went wrong, and then culling and systematizing the learning. They work closely with Root Capital’s core loan officers, interacting on problems that emerge and taking in the essential and nuanced perspective that only a core loan officer could have. Field officers then take these ideas and experiences back to the lab, where they build out the potential innovation.

This partnership between the lab and the African field offices led to a startling discovery: three-quarters of African crops are grown for domestic use. This finding contradicted the long-held notion that the best way to raise rural incomes was to grow high-value, organically grown, fair-trade crops for export. Root Capital, which had concentrated its loan activities on supporting export endeavors, adjusted course and began piloting innovation loans to community farmers. After much iteration, Root Capital moved this activity into its core operation and has built it into an $8 million business.

The lesson from Root Capital is that the process of invention, even when pursued through an entity separate from the main organization, should not operate in secret. In the quest for the next innovation, an organization need neither marginalize its innovation capability nor place it on a pedestal. Regular interaction between the innovation group and the implementation group yields the best innovations. Equally, integration of an invention should not take place in one fell swoop—as in a massive implementation or transformation program—but incrementally, so that field learning can flow back into inventive thinking.

Innovation Dissonance

While engaging in the process of innovation, we inevitably run up against one or more of these three paradoxes. They create tensions between actors and disciplines, and between intentions and executional issues. But the tension is a sure sign that innovation is happening, that people are working through their differences, finding common ground, and sparking new combinations and directions that would never have appeared otherwise. It is, therefore, a productive tension that we call “innovation dissonance.”

We believe that innovation occurs when different points of view and different elements are reframed, reimagined, or recombined in new ways. To manage this coming-together of disparate elements and crossing-over of multiple boundaries requires an understanding of the paradoxes that put pressure on collaboration and an ability to identify and relieve them enough for innovation to thrive.

We have seen that people and organizations around the world are finding their own path to innovation—by being innovation enthusiasts without kowtowing to every practice of the cult, by integrating disparities without neutralizing their distinctive contribution, by building extended teams that know how to integrate invention outposts into the larger landscape of the organization, and by recognizing that the dissonance involved is usually short-lived and that social benefit can last for lifetimes.

As people at the Rockefeller Foundation have been learning for 100 years, innovation isn’t easy, but it may be that wrestling with these innovation paradoxes creates much of the energy that drives the creation of new products, processes, and services that can fundamentally improve the lives of poor or vulnerable people.