Act on Facts, Not Faith

The 19th-century French physician Pierre-Charles-Alexandre Louis put a lot of leeches out of business. For centuries before his research, doctors believed that removing a few pints of a person’s blood would help cure all sorts of ailments. In the 1830s, doubting bloodletting’s alleged effects, Louis undertook one of the first clinical trials. He compared the fates of 41 pneumonia victims who had undergone early and aggressive bloodletting to the fates of 36 pneumonia victims who had not. The body count was clear: 44 percent of the bled patients subsequently died, compared to only 25 percent of the patients who did not get the bleeding treatment.¹ Louis’ discovery helped convince physicians to abandon bloodletting and earned him the title “Father of Epidemiology.”

Louis’ study is a touchstone of the modern evidence-based medicine movement, which trains physicians to conduct, evaluate, and act according to research. Despite the wealth of research on what makes organizations and people more effective, there isn’t yet an evidence-based movement in management. Instead, managers frequently base their business decisions on hope, fear, dearly held ideologies, what others are doing, and what they have done in the past – in short, on lots of things other than evidence. As a result, many managers inadvertently harm their organizations and stakeholders in much the same way that bloodletting doctors inadvertently harmed their patients.

The time has come for an evidence-based management movement. Like evidence-based medicine, evidence-based management can help managers figure out what works and what doesn’t, identify the dangerous half-truths that constitute so much of what passes for wisdom, and reject the total nonsense that too often passes for sound advice. Although much of our research is in the business sector, evidence-based management is just as applicable in the nonprofit and government sectors. Managers who adopt the approach we suggest will find it easier to sort out what advice to follow and – more importantly – what advice to ignore.²

Impediments to Evidence Abound

Several barriers block the best facts from rising to the top levels of an organization. Although the evidence for what works in management is ample, the signal drowns in an ocean of noise. Business writers, gurus, and consultants routinely tout their “breakthrough ideas” in more than 100 magazines and newspapers devoted to business issues³ and in over 30,000 books on business topics.⁴ To confuse things even more, many of the ideas in this massive snarl conflict with each other. Consider a few book titles: “In Search of Excellence” and “The Myth of Excellence”; “Built to Last” and “Corporate Failure by Design”; “Thinking Inside of the Box” and “Out of the Box.”

“If you don’t like the theory du jour, just wait a few minutes, and it will change,” remarks Daniel Ben-Horin, founder and president of CompuMentor, a San Francisco-based nonprofit that provides technology assistance to other nonprofits. “Especially during the dot-com boom, every week some well-meaning board member would walk in, throw a book on the table, and say, ‘If you don’t read this book in the next 15 minutes, you’re going down.’”

Even when writers and researchers do not consider their findings to be earth-shattering breakthroughs, the press often bills them as such. From 2001 through 2005 the Harvard Business Review published a list of “Breakthrough Ideas for Today’s Business Agenda” in its February issue. Three of our ideas have been selected for this list. One of these, Sutton’s “The Weird Rules of Creativity,” really wasn’t a breakthrough because these rules were derived from past, sometimes quite old, research. And Pfeffer’s 2005 “breakthrough idea” was evidence-based management, which might be unsettling to some people, but we are the first to admit it isn’t new.

The cacophony in the marketplace of ideas is not the only barrier to evidence-based management; so is the pecking order within organizations. When done right, evidence- based management disrupts established power dynamics, replacing formal authority, reputation, and intuition with data. A student of ours who worked at Netscape told us that James Barksdale, a former CEO of that company, once remarked at a company meeting something to this effect: “If the decision is going to be made by the facts, anyone’s facts, as long as they are relevant, are equal. If the decision is going to be made on the basis of people’s opinions, then mine [he was the CEO at the time] count for a lot more.” Facts and evidence are great levelers of hierarchy, and decision makers sometimes do not want to give up their place at the top of the heap.

Demand the Facts

To make decisions based on evidence, managers must of course get the evidence in the first place. In some cases, managers have to learn how to do their own research. In others, they can consult existing evidence and then evaluate and apply it according to sound standards. (See sidebar, p. 43.) In either case, managers must instill cultures where everyone in the organization is equally committed both to getting and to using the best facts.

DaVita Inc., an El Segundo, Calif.-based operator of kidney dialysis centers, is one such organization. When Kent Thiry joined DaVita as CEO in 1999, the company was close to bankruptcy. Today, DaVita is a thriving $2 billion company whose stock has jumped more than tenfold since Thiry joined. There are many reasons for the turnaround, but an important ingredient was Thiry’s commitment to evidence-based management. He taught facility administrators to base their management decisions on data, not on preconceived ideas or on what seemed to have worked elsewhere. To help administrators make decisions they now receive eight-page monthly reports that compare their facilities to others on such measures as employee retention, patient retention, treatments per day, labor hours per treatment, and controllable expenses.

An important feature of these monthly reports is the data that aren’t yet included. When the company decides that a particular measure is important, but is not yet able to collect it, it includes the measure on the report with the notation “not available.” This motivates employees to figure out ways to gather the vital yet missing information.

DaVita’s emphasis on evidence-based decision making, as well as its culture of speaking the truth about how things are going, are crucial to its success. Even now when Thiry meets with frontline employees and tells them that DaVita has the best quality of treatment in the industry, he demonstrates that fact with specific, quantitative comparisons. “No brag, just facts” is his motto.

Large nonprofit organizations and many government agencies have the capabilities to collect a wide assortment of quantitative data on their operations, but many small and midsized nonprofits lack sufficient resources to do so. But that doesn’t mean they can’t do anything. For many, collecting qualitative data in the field is an economical way to uncover the truth. Michelle Plane, CEO of St. Laurence Community Services in Melbourne, Australia, relates how she checks in with her 450 employees to learn what is really going on. “About three times a year, we do what I’ve come to call ‘road shows.’ We take all five general managers and a junior manager to all 20 of our sites.”

“In my own experience as a general manager, I learned that people really want to see the leadership team,” said Plane. “I also learned that to run an organization well, you have to understand the issues people are encountering at the coal face.”

Plane notes that her road shows not only give her useful data about whom to promote and how to grow, but also plant the seeds for more honest feedback from her staff. “When CEOs walk around and get to know their staff, employees are more relaxed and actually tell you what is going on rather than what you want to hear.”

Beware of Your Preconceived Ideas

Hard-won evidence does no good when it is ignored, yet all too often that is exactly what happens when people’s rigid adherence to ideology, theory, or preconceived ideas blinds them to the facts. This happens in part because people see what they believe, and because theories can become self-fulfilling: When we act as our pet theories suggest we should, we can produce the very behavior we expect in ourselves and those around us.

One example of theory getting in the way of hard facts is the theory that people are self-interested actors, and so the best way to motivate people to work harder is to pay them more money. Economists use the self-interest motive not only to explain the power of financial incentives, but also to explain why people fall in love, get married (including why they may or may not prefer polygamy), and have children. Yet these economists ignore evidence that being selfish is not a hardwired human trait, but rather is learned and varies widely across people, groups, and countries.⁵ Moreover, many studies of human and organizational performance suggest that people are not all that moved by money. For example, a Watson Wyatt 2003-2004 survey of 1,700 high-performing employees found that these top performers ranked maintaining a positive reputation as their first motivation, but ranked expecting a significant financial reward ninth out of the 10 items.⁶

The recent clamoring for performance-based pay for schoolteachers has similarly proceeded with little regard for the evidence. Merit pay for teachers is an idea that is almost 100 years old, and that has been subjected to much research. In 1918, “48 percent of U.S. school districts sampled in one study used compensation systems that they called merit pay.”⁷ The evidence shows that merit pay plans seldom last longer than five years and consistently fail to improve student performance.⁸

Merit pay also backfires because it sends the message that all that matters is students’ test scores. Economists Brian Jacob and Steven Levitt have shown that the greater the incentive pay for enhancing students’ scores, the more teachers and students cheat.⁹ Anthony Bryk, a prestigious educational researcher, tells us that the same problems emerged when merit pay systems were implemented in the 1980s. Bryk observes, “It is like policymakers suffer from amnesia.”¹⁰

Rethink Your Assumptions

Even without gathering data, there is something managers can do to assess management ideas: Rethink the assumptions that underlie them. We have found that a thoughtful consideration of the assumptions that underpin interventions is often sufficient to reproduce the insights gained from piles of empirical research.

Going back to the example of merit pay for teachers, let’s examine two assumptions inherent in virtually all teacher pay-for-performance plans: 1) Teachers are significantly motivated by financial incentives, and so merit pay will induce greater and more effective effort; and 2) Teacher motivation is a, perhaps the, determinant of student learning and achievement.

Do these assumptions seem plausible? Can you imagine a person saying, “I am motivated by a lot by money, and so I think teaching first-graders is the career for me”? And how important do you think teacher motivation is to student achievement, compared to teacher skill, parental involvement, the community where the children live, the quality of the facilities and resources, school culture, and parental education and income – none of which are affected by pay incentives?

Nonprofit organizations are likewise under pressure to offer financial incentives – a concern made all the more poignant by nonprofits’ relative lack of resources. Yet by most accounts, winning the hearts and minds of nonprofit employees has little to do with lining their wallets.¹¹ Most people who work at nonprofits are motivated by meaning, not money. As Ben-Horin recalls: “When the dot-com bubble busted, we had this enormous pool of private-side talent available to us because of all the layoffs. They had been making a lot of money and doing interesting things, but in the end they felt they were selling toothpaste on the Web. At CompuMentor, they were pleased to find a place to do work that was important to them. Many wound up feeling blessed to be laid off.”

Avoid Casual Benchmarking

One assumption that commonly gets managers in trouble is that what works for other organizations will work for their own. There is nothing wrong with learning from the experience of others – it is a lot cheaper and easier to learn from their mistakes, setbacks, and successes than to treat every management challenge as your own personal terra incognita. That is why benchmarking, using others’ performance and experience to set standards for your own company, makes a lot of sense.

The problem with benchmarking comes when managers use it casually, paying too little attention to what works, why it works, and whether it will work elsewhere. Consider the case of United Airlines, which in 1994 decided to compete with Southwest Airlines in the intra-California marketplace by trying to imitate Southwest. United dressed its flight attendants in casual clothes, stocked its fleet with Boeing 737s, increased the frequency of flights, and reduced the time that planes spent on the ground. It even gave the venture a different name, “Shuttle by United.”

What United failed to realize is that Southwest’s success stems from its culture and management philosophy and the priority it places on its employees (Southwest did not lay off one person following the 9/11 meltdown of the aviation industry), not on its flight attendants’ wardrobes, its planes, or its flight schedules. After United launched its service, Southwest wound up with a higher market share in California than it had before.¹² The Shuttle is now shuttered.

Nonprofits also feel the rub of ill-fitting benchmarks. “There are many people in the nonprofit art world who think that everything needs to be run like a business,” notes Steven Bridgeland, managing director of the Columbus (Ohio) Children’s Theatre. “The problem is you end up with arts organizations that don’t have the heart that drives the people to continue to be in or support them.”

“Last year we did a production of ‘Wiley and the Hairy Man,’ which is a scary play about a boy overcoming his very real fears about the Bogeyman,” said Bridgeland. “No one had heard of it, and we knew that we were going to lose money. But it was one of the best pieces of theater we have ever put on – it was very visual, very in-their-face. The kids really felt like they were Wiley and perhaps worked through their own fears. They related to the play as art, and not just as entertainment. And so we gave a great, life-changing experience to our audiences. If we had run this strictly like a business, we would have never done that play.”

Know When the Past Has Passed

Just as borrowing from other people’s experiences can lead managers astray, so too can the practice of borrowing the wrong practices from their own past.

Plane recalls how one general manager misapplied his learning from the corporate world to the government sector. She was working at a local government organization (a council) where a new GM, recently hired from one of the biggest telecommunications companies in Australia, instituted a monthly briefing format based on the one at his former corporate job. For these briefings, the CEO and GMs would first meet for 10 minutes, during which time the CEO spoke on no more than four critical points. To convey urgency, all attendees had to remain standing. The GMs then briefed their teams in the same stand-up, 10-minute style, and so on down the line so that within 24 hours, all of the staff had heard the same message.

“It was a monumental flop,” said Plane. “The employees were laughing because it seemed so ‘Mission Impossible,’ what with all the standing up and the seriousness. And there were road crews and night shifts who were inevitably left out of the briefings. The new GM didn’t understand the culture of the organization, and totally lost credibility with the staff.”

Bridgeland experienced firsthand how some of the techniques he learned while working in the corporate sector don’t work in the world of children’s theater. To relieve interpersonal tensions within the theater’s touring troupe, Bridgeland attempted to teach its young actors the communication strategies he learned while working at Microsoft. The strategies require people to identify their own and others’ goals in an interaction, to figure out an acceptable compromise, when necessary, and then to modify their behavior accordingly. When Bridgeland explained these techniques to the group, he saw a lot of head nods, “but I haven’t seen any changes in behavior,” he says. He speculates that either the actors’ clashes do not arise from a communication problem, or that the rational problem-solving techniques that worked so well for software engineers just aren’t that interesting for the theater’s artistic personalities.

Adopt an Attitude of Wisdom

On the first day of class, the deans of many medical schools greet their first-year medical students with this sobering fact: “Half of what we know is wrong. The problem is, we don’t know which half.” After all, even leeches have been resurrected from the scrap heap of medical science, and are currently enjoying a second run as the stewards of severed appendages.

Doctors nevertheless proceed with their doctoring. And managers, similarly deluged with a confusing tangle of verity and falsehood, must likewise proceed with their managing. While the points we’ve offered here can help them do a better job of evaluating and applying business knowledge, something else, something broader, is more important than any single guideline for reaping the benefits of evidence-based management: wisdom.

Wisdom means acting with knowledge while doubting what you know. It entails striking a balance between arrogance (assuming you know more than you do) and insecurity (believing that you know too little to act). It requires asking for help and asking questions, as well as giving help and answering questions. With an attitude of wisdom, managers can do things now, but still keep learning along the way.¹³ (See sidebar on p. 46 for types of wise employees)

The leader of the modern evidence-based medicine movement, Dr. David Sackett of McMaster University, personifies the sort of wisdom needed to spark constant improvement in medicine and management alike.¹⁴ He worries openly that people will treat his ideas as gospel rather than as an initial effort that must be constantly revised and challenged. He despises when people call him an expert on evidence-based medicine because “so-called experts can’t help but be biased toward their own published views.” Sackett has gone to extreme lengths to convey that he is not a lone genius, that he was just part of a team at McMaster University that developed modern evidence-based medicine, and so doesn’t deserve to be singled out as the guru.

Unfortunately, there are few David Sacketts in the management idea marketplace. Modesty is in short supply and absolutes abound – in recommendations as to what to do, in conclusions about what affects individual and organizational performance, and in beliefs about what is true and what is false. Little wonder, then, that management lore is soaked in snake oil and riddled with dangerous half-truths, causing all sorts of problems for those who are seduced by their appeal. That is why creating a culture that fosters an evidence-based approach to management and problem solving is one of leaders’ most crucial tasks, regardless of whether they are in charge of a business, a nonprofit, or a government agency.

1 Patterson, K. “What Doctors Don’t Know (Almost Everything), New York Times Magazine, May 5, 2002.

5 Miller, D.T. “The Norm of Self-Interest,” American Psychologist 54 (1999): 1053-1060.

7 Murnane R.J. & Cohen, D.K. “Merit Pay and the Evaluation Problem: Why Most Merit Pay Plans Fail and a Few Survive,” Harvard Educational Review 56 (1983): 2.

11 See SSIR’s Spring 2005 Upfront, “Helping Do-Gooders Do Their Best,” which relays the findings reported in Besley, T.J. & Ghatak, M., “Competition and Incentives With Motivated Agents,” American Economic Review (September 2004).

12 McCartney, S. and McCarthy, M.J. “Southwest Flies Circles Around United’s Shuttle,” The Wall Street Journal (Feb. 20, 1996): B16.

13 See Meacham, J.A. “The Loss of Wisdom,” Wisdom: Its Nature, Origins, and Development (Cambridge, England: Cambridge University Press, 1990): 181-211; and Meacham, J.A. “Wisdom and the Context of Knowledge: Knowing That One Doesn’t Know,” On the Development of Developmental Psychology (Basel, Switzerland: Krager, 1983): 111-134; and Sternberg, R.J. “Implicit Theories of Intelligence, Creativity, and Wisdom,” Journal of Personality and Social Psychology 49 (1985): 607-627.

14 Bauer, G. “A Reluctant Policy Wonk: Dr. David Sackett, a Pioneer in Evidence-Based Medicine, Has No Time to Play King, but the World Seems Intent on Crowning Him,” Medical Post (Aug. 22, 2002): 43.

JEFFREY PFEFFER is the Thomas D. Dee II Professor of Organizational Psychology at the Stanford Graduate School of Business, where he has taught since 1979. He may be reached at [email protected].

ROBERT I. SUTTON is a professor of management science and engineering at Stanford’s School of Engineering, where he is also co-director of the Center for Work, Technology, and Organization. He may be reached at [email protected].

Pfeffer and Sutton are co-authors of “The Knowing-Doing Gap: How Smart Companies Turn Knowledge Into Action” (Harvard Business School Press, 1999) and of “Hard Facts, Dangerous Half-Truths, and Total Nonsense: Profiting From Evidence-Based Management” (Harvard Business School Press, 2006), upon which this article is based.

Read more stories by Jeffrey Pfeffer & Robert I. Sutton.