March 30, 2012

Teaching Beyond the Test

A good teacher’s influence can affect student test scores — and long-term success in life.

There’s no doubt that teachers make a difference in students’ lives. But new research shows that lessons from great educators may live on in more than just memories; in fact, they can directly affect where students live, how much money they make, and even when they have children.

School districts across the country have scrambled to find the most effective ways to evaluate teachers in recent years, especially in the wake of 2001’s No Child Left Behind Act. The federal legislation rates schools and teachers based on the percentage of students meeting a particular level on standardized tests, at which they are deemed proficient. Critics have consistently pointed out the flaws in one-size-fits-all standards, says Professor Jonah Rockoff, who explains that the academic and family backgrounds of students gives high-income suburban districts an edge over low-income urban schools.

“No Child Left Behind essentially has one goal for all students, no matter where they started,” Rockoff says. “It tells schools serving advantaged kids that they are doing well and schools serving disadvantaged kids that they’re doing a poor job.”

In search of fairer evaluation measures, Rockoff worked with Raj Chetty and John Friedman of Harvard to explore a different approach, called value-added, and to test whether teachers’ impacts on student test scores are actually a good measure of teacher quality. In general, value-added refers to the amount by which the value of something is increased at each stage of its production, excluding initial costs. When using this approach in teaching, how much value a teacher adds each year — essentially, the effectiveness of a teacher — is measured by growth in his or her students’ test scores. In order to distinguish true teacher impacts, however, Rockoff points out that one must account for student backgrounds. “Value-added aims to set reasonable benchmarks for how much each child’s achievement should grow from one year to the next,” he explains. “Instead of setting the same standard for all students, you should compare the performance of teachers to other teachers with classrooms of similar students. How well did your students do compared to other students who look like them?”

To compare apples to apples, the researchers controlled for outside factors like incoming achievement levels and family background in the largest study ever conducted on value-added: they tracked 2.5 million students and 18 million test scores for math and English for 20 years, from 1989 to 2009. The information came from two databases: the first, test scores and classroom and teacher assignments in grades 3 through 8 from a large US urban school district; the second, data from US tax records that provides information on student outcomes such as earnings, college attendance, teenage births, and parent characteristics like household income, retirement savings, and mother’s age at child’s birth. The researchers matched nearly 90 percent of the school district data to the tax data, allowing them to track a large group of individuals from elementary school to early adulthood.

The results were overwhelming: students assigned to high value-added teachers — those whose students consistently achieved higher test score growth — are more likely to attend college, attend higher-ranked colleges, earn higher salaries, live in higher income neighborhoods, and save more for retirement, and less likely to have children as teenagers — regardless of their background or parental income. Replacing a teacher whose value-added is in the bottom 5 percent with even an average teacher would increase the present value of students’ lifetime incomes by more than $250,000 per classroom, on average, in the researchers’ sample. Each classroom was made up of approximately 28 students.

“We believe that value-added measures uncover the real impact of teachers on student test scores,” Rockoff says. “High valued-added teachers improve students’ long-term outcomes, which means that a higher test score doesn’t just reflect mindless teaching to the test. At least some of it reflects real improvement in student achievement growth that has long-lasting benefits.”

Rockoff adds that the research puts definitive numbers to the long-held belief that teachers matter in students’ lives. “We invest in public education with taxpayer dollars and private dollars, and as a society we need to know what benefits we’re getting from those investments,” he explains. “Just having a rough sense that education is important doesn’t give policymakers a way to gauge whether to increase or decrease funding for certain programs, or how much to spend on a new program to improve the quality of education for students.”

But putting value-added measures into practice isn’t so simple. For one, there are fears that teachers and districts would — and that some already do — focus only on testing, at the expense of other lessons. To avoid bad behavior and potential cheating by teachers to pass students, Rockoff says districts should avoid applying a value-added approach in a formulaic way. Instead, principals should take more into account than student test scores: they should also observe a teacher in the classroom; solicit peer, parent, and student feedback; review portfolios of student work; and analyze teacher lesson plans.

“We see this type of review in most professional occupations, especially those as complex as teaching,” Rockoff says. “Like other types of managers, principals should make decisions based on lots of information and data pulled together.”

However, with education funding and teacher evaluation determined by state legislatures, political realities could make implementing these more comprehensive measures difficult. If individual districts were given enough autonomy to implement value-added measures that produce good results, Rockoff says, schools would have more honest insight into which educators are helping — and hurting — students most.

“When teachers struggle, their students feel lasting damage and grow far less than their peers in other classrooms,” Rockoff says. “But teachers who greatly improve the achievement of their students have long-term impacts on those kids’ livelihoods that are worth several times a teacher's annual salary. We all knew teachers mattered, but now we also know just how much.”

Jonah Rockoff is the Sidney Taurel Associate Professor of Business in the Finance and Economics Division at Columbia Business School.

Read the Research

Raj Chetty, John N. Friedman, Jonah Rockoff

"The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood"

Download PDF 
View abstract/citation