(Illustration by Ben Wiseman)
The research up to this point on charter schools has focused primarily on test scores,” says Will Dobbie, an assistant professor of economics and public affairs at Princeton University. “That focus misses something important, because test scores might not translate into life outcomes.” Indeed, critics of charter schools often charge that these institutions simply train students to perform well on standardized tests. Partly in response to that argument, Dobbie joined with Roland G. Fryer Jr., the Henry Lee Professor of Economics at Harvard University, to study whether attending a charter school confers other, more enduring benefits.
Dobbie and Fryer set out to evaluate a range of both academic and nonacademic outcomes among students who enrolled in Promise Academy, a K-12 charter school operated by the Harlem Children’s Zone, a nonprofit organization in New York City. In 2012, the researchers tracked down people who had entered a lottery to enter one of the Promise Academy’s middle schools six years earlier. Next they compared outcomes for lottery entrants who gained admission to the school with outcomes for students who entered the lottery but didn’t win it.
Parag Pathak, a professor of economics at the Massachusetts Institute of Technology, contends that studying lottery entrants is a “gold standard” for evaluating student outcomes. “The beauty of a lottery is that there should be no difference [in family background, for example] between those who were and were not selected,” says Pathak, who has studied charter school effectiveness in Boston and other US cities.
In Dobbie and Fryer’s study, respondents took the Woodcock-Johnson tests in math and reading—a commonly used vehicle for measuring general learning achievement. Those who had attended Promise Academy scored higher on the tests than those who hadn’t enrolled at the school. (Dobbie notes that the charter school did not specifically prepare students for these tests.)
The researchers also surveyed respondents about their level of educational attainment. On this front, the effects of attending the Promise school proved to be relatively modest. “Kids who didn’t attend the charter school eventually caught up in years of schooling,” says Dobbie. Compared with those students, though, Promise Academy students were more likely to graduate from high school in the standard four-year time frame, and more likely to enroll in college right after graduation.
In addition, Dobbie and Fryer surveyed respondents on whether they had experienced outcomes that reflect “risky behaviors” (to use the researchers’ term)—such as becoming pregnant (in the case of females) or becoming incarcerated (in the case of males). The results suggest that charter school attendance can yield benefits in this area: Females who won the lottery and then attended the Promise school were 10.1 percent less likely to become pregnant during their teen years than those who didn’t win the lottery. Male lottery winners were 4.4 percent less likely to become incarcerated than their non-winning counterparts.
“The main innovation of [Dobbie and Fryer’s] paper,” says Pathak, is that it counters the common view that “schools can do only so much to influence achievement, [because] disadvantaged children have to overcome their family background.” But Pathak notes that the study—despite its “compelling result”—has an important limitation. “What does a single-school study tell us about more general policy questions for charter schools?” he asks. “That question is still up in the air.”
To broaden their inquiry, Dobbie and Fryer are now conducting a study that will focus on people who attended a group of charter schools in Texas. In that project, they will examine state-level data sets to track the labor force outcomes—including earnings over time—of those former students. Still, Dobbie suggests, the Promise Academy study holds value in its own right: “We show, for the first time, that there are significant non-test outcomes that are important [indicators] of how well someone’s life is going.”
Will Dobbie and Roland G. Fryer Jr., “The Medium-Term Impacts of High-Achieving Charter Schools,” Journal of Political Economy, 123, 2015.
Read more stories by Kristine Wong.