As China’s economy boomed, classrooms in rural provinces emptied. One in four teenagers were dropping out of junior high to work in factories to help their impoverished families. Giving cash to students as an incentive to stay in school, as many nonprofits have done in other countries, seemed like an obvious solution.
But for Scott Rozelle, a good idea isn’t good enough—it has to be backed by empirical evidence. So Rozelle, founder and co-director of the Rural Education Action Project (REAP) at Stanford University, designed an experiment: 300 of the poorest seventh-graders in an area of Shaanxi province in northern China were identified. Half of them were chosen at random to receive $75 if they stayed in school for another semester during the 2009-10 school year. Half were not; they served as the control group.
The results were better than Rozelle imagined. The dropout rate for the students who received money was 60 percent less than for those who didn’t.
The Chinese government was impressed, too. Thanks to long-established connections in the country, a policy brief written by REAP and its partners made its way to China’s State Council, the equivalent in the United States to the Cabinet. Soon afterward, China’s top education official directed $10 million to be spent on cash incentives for students.
For Rozelle, taking the time to do such experiments is the difference between helping 150 kids and affecting millions. “Our entire raison d’être is to change policy to help the poor,” says Rozelle, an agricultural economist. “When we’re implementing projects we’re always thinking, how would the government do it?”
Poverty in China is such an immense challenge, Rozelle says, that the government has to be involved to make substantial change. The canyon between rich and poor, urban and rural is wider in China than anywhere else.
China has the second-largest economy and the second-largest number of billionaires (after the United States)—and the second-largest number of people in poverty (after India). In China’s cities, 70 percent of children go to college. In the countryside, most children suffer from severe malnutrition, intestinal worms, and anemia, and fewer than 5 percent will continue their education past high school.
Against this inequity, REAP leverages research. Since its founding in 2007, REAP has conducted about 30 sets of experiments that have produced policy briefs read by top government officials. From there, the government has issued directives and money to improve nutrition among children and make education more affordable.
In 2008, China bestowed on Rozelle the Friendship Award, the highest honor available to a non-Chinese, for significantly contributing to the country’s economic and social advancement. REAP’s $2 million annual budget is funded by a wide range of donors, from foundations such as the Ford Foundation to corporations such as Nokia to individuals who had no previous interest in China.
Even as REAP enjoys such support, the work of putting the policy into practice remains a challenge—as it does for any social endeavor. Managing relationships and changing human behavior is much harder than crunching data.
Before REAP, China lacked rigorous, systematic research on
its rural areas. Data collection consisted of a person surveying a few schools picked because of connections with the principals.
In one province, REAP showed that most children were anemic and that teaching parents about nutrition wasn’t making a difference. A few more research trials later, local officials finally conceded that vitamins are the best solution. But they still hesitated to distribute vitamins that a company donated; they worried about liability.
Rozelle shakes his head over the bureaucratic absurdity, but he smiles with optimism. Convincing the officials to consider vitamins was a major step forward, and the rest will follow, eventually. “Policy change,” he says, “is a long haul.”
Rigor Plus Collaboration
Although REAP is only five years old, its roots reach back more than two decades. That’s how long Rozelle and Linxiu Zhang have been working together on research about rural China. Zhang, of the Chinese Academy of Sciences in Beijing, a national research institution, is REAP’s co-director.
Rozelle, who grew up in a middle-class Los Angeles suburb, became interested in learning Chinese on the recommendation of his father, whose military career included a stint in Shanghai. Zhang, the daughter of an illiterate farmer, was admitted to college in 1978, the first year that China allowed entrance strictly through test scores rather than family connections.
Although their backgrounds may be different, Rozelle and Zhang rely on each other’s contributions. The philosophy of shared learning became a part of REAP. “We know better about our country, but partners like Scott know better about the research methods,” Zhang says. “It’s mutually beneficial.”
That collaborative spirit has been crucial to REAP’s credibility, says Prashant Loyalka, a professor at Peking University who is part of an institute that advises the government about education policy. Loyalka says that some education nonprofits from overseas will send teachers and resources to China, but operate only within the confines of their program. REAP, however, has expanded its network and encouraged others to be invested in its outcomes.
At Northwest University in Xi’an, Loyalka says that REAP increased the capacity of researchers there to design and lead the work, rather than simply participate. During meetings, even local university students who serve as data collectors, usually the lowliest cog in a project, were asked for their input.
“I don’t think REAP sees itself as from the outside,” says Loyalka, who received his doctorate in education from Stanford. “I think REAP sees itself as being part of China and wanting to help China grow.”
Before REAP, China lacked rigorous, systematic research on its rural areas. Data collection often consisted of a person surveying a few schools picked because of connections with the principals. REAP’s methods involve random sampling, control trials, and repeated visits over the course of a school year. This type of research, sometimes called “poverty labs,” is similar to how drug companies test new products. Although randomized sampling had been increasingly used in other developing countries, it had not been introduced in China until REAP.
With the research, REAP is often able to push ahead of others trying to influence government policy. “There’s a natural proclivity to rely on evidence,” Rozelle says. “Lots of people say that they have the best sliced bread, but if you can come and give them evidence-based research, what we find is that they’ll always consider it. They don’t always act on it, but they’ll consider it.”
The research methods are controversial, however, among some NGOs and academics. Critics have asked why some people are given access to a resource, such as scholarships, while those in the control group are denied. Rozelle says researchers follow the medical edict of “do no harm.” “No one leaves worse off,” he says, adding that in a study involving scholarships, REAP distributed money to more students after research was completed. Although the subjects might not be worse off, the organization conducting the study may feel under attack. Many nonprofits aren’t eager to find out that a much touted program isn’t effective.
Economist Yaojiang Shi says that by saving face, the organizations are limiting their potential. As director of the Northwest Socioeconomic Development Research Center in Xi’an, Shi works with REAP as well as nonprofits that don’t use research methods. Some nonprofits have invested 10 times more money than REAP, Shi says, but they can’t identify what part of their efforts is working. Without that, he said, it’s hard to change broader policy.
For REAP, data are the real measure of progress, not anecdotes. “Storytelling is fine, but you need to have basic facts,” Zhang says. “What do you want to achieve? Do you want to know whether your efforts are worthwhile?”
Sometimes, the research results are surprising and revoke REAP’s strongest assumptions. Two years ago, REAP hosted an intensive training for English teachers in migrant schools. The three-week program was featured on REAP’s website, with photos of beaming teachers.
“The students and teachers and volunteers all agreed the goal of improving the English proficiency of the migrant teachers was achieved,” the press release trumpeted. Zhang was convinced that the teachers would go back to their villages and raise student performance. A semester later, REAP’s research showed that the training had no impact.
“It’s a failure, but it’s a good experience,” Zhang says. “We learned a lot.” REAP is now following up training with periodic text messages of reinforced lessons. If that doesn’t work, Zhang says, they’re still another step closer to the solution that will improve the lives of millions of children.