Education researchers have analyzed the effects of individual teachers on students’ grades and standardized-test scores. But a large body of data suggests that noncognitive skills, such as resilience and self-discipline, are far more important for determining the longterm success of students. Is there a way of tracking the ability of teachers to instill these skills in their pupils, and if so, what would this mean for educational assessment?
(Illustration by Adam McCauley)
A new study by Clement Kirabo Jackson, a professor of human development and social policy at Northwestern University, examines student outcomes that are not related to academic performance, to see how teachers can affect their students’ lives in other ways. His research shows that teachers’ influence on test scores is only weakly correlated with their impact on other student behaviors and choices. What’s more, teachers who demonstrate success in improving students’ noncognitive abilities and boosting their chances to graduate are not just the teachers whose students achieve high test scores.
“Relative to using only test-score measures, using teacher effects on both test-score and noncognitive proxy measures more than doubles the variance of predictable teacher impacts on longer-run outcomes,” Jackson writes in his working paper for the National Bureau of Economic Research.
Today, we measure school quality primarily by how students perform on standardized tests. But test scores aren’t always a useful predictor of how students will do in life, Jackson says. If teachers and schools are affecting skills other than those measured on the tests, and we’re not seeing those effects, then we could be evaluating teachers incorrectly and missing out on important teacher-retention goals.
“We do a poor job of finding effective teachers and schools using only test scores, and we would do a much better job if we measured soft skills,” says Jackson.
Jackson, a labor economist, brings an econometrics approach to education data. His study considers all public school ninth graders in North Carolina from 2005 through 2012, using education-department data on their test scores and grades as well as other factors. His model looks at cognitive and noncognitive skills, and measures the latter by quantifiable data such as absences, suspensions, course grades, and on-time grade progression. From such data, he constructs a “behavior index” that indicates how students are performing outside of their academic progress.
The analysis shows that an individual teacher who raises a student’s performance on the behavior index—what Jackson calls noncognitive value-added—by one standard deviation increases a student’s chance of high school graduation by 1.47 percentage points. If the teacher raises the student’s test scores by the same amount, the student’s graduation chances go up by just 0.12 percentage points, he found. Jackson identified the same effect on a range of outcomes for students, including being promoted to the next grade, whether a student takes the SAT, senior-year GPA, and plans to attend a four-year college.
Jackson chose ninth graders to study because the first year of high school is when many students drop out or decide they soon will. “If you can just get a bunch of those kids to stay, you change the trajectory of their education forever,” he says. And since the students get different teachers in the next grades, it enables researchers to assess the value-added effect of ninth-grade teachers.
The paper offers compelling evidence that teachers have an effect on outcomes other than test scores that are important to society, says Steven Rivkin, head of the University of Illinois at Chicago’s economics department.
“It’s really important to recognize that teachers can influence a lot, not just how much math someone knows, but can help to shape a student’s social and emotional skills in a positive or a negative way,” Rivkin says.
Although Jackson’s findings are intriguing, turning them into a way to measure teacher quality directly would be difficult, he and Rivkin agree. “It’s not clear that it would provide incentives to educators to do the things that would be helpful to long-term outcomes,” Rivkin says. At the end of the paper, Jackson offers some policy suggestions, including identifying teaching practices that improve student behaviors and using incentives to encourage such practices.
The findings also don’t allow educators to look backward and identify which characteristics set apart teachers whose students later go on to have good outcomes, with a goal of training teachers to act in a conducive way. More research is needed, Jackson says.
Clement Kirabo Jackson, “What Do Test Scores Miss? The Importance of Teacher Effects on Non-Test-Score Outcomes,” NBER working paper, April 15, 2017.
Read more stories by Chana R. Schoenberger.