Another limitation of value-added assessment is that it applies only to those teachers for whom yearly test scores are available, possibly a minority of a school’s staff. New teachers, those with less than three to five years of experience, cannot be evaluated, because there is not enough long-term test score data. Teachers of history, social studies, the arts, science, technology, physical education, and foreign languages cannot be evaluated, because their subjects are not regularly tested. Only teachers of reading and mathematics in elementary school and middle schools can be evaluated, and they can be evaluated only if scores are available from the previous year and only if they have been teaching for at least three years.
The idea of using value-added test scores to fire teachers or to award tenure posed another problem. In 2008, economist Dan Goldhaber and his assistant Michael Hansen asked whether teacher effectiveness, measured in this way, is stable over the years. Goldhaber said that if performance were extremely stable over time, then it would be a good idea to use students’ scores for high-stakes decisions. But he discovered that teacher effects were unstable over time. Comparing thousands of North Carolina teachers, pre-tenure and post-tenure, Goldhaber found that 11 percent of teachers who were in the lowest quintile in their early years of teaching reading were in the highest quintile after they received tenure. Only 44 percent of reading teachers and 42 percent of math teachers in the top quintile before tenure were still in the top quintile in the post-tenure period. This meant that if an administrator assigned low-performing students to a top-quintile teacher, there was a good possibility that the teacher would not be a top-quintile teacher the next year. Some elementary teachers were more effective at teaching reading than math; others were more effective at teaching math than reading. An administrator who denied tenure to all teachers who were in the bottom quintile in either reading or math would terminate almost a third of the teachers. Moreover, almost a third of them would have eventually landed in the top two quintiles in reading, and about a quarter in the top two quintiles in math, had they not been terminated. Goldhaber also found that consistency of job performance and productivity became greater as teachers became more experienced. 20 Other studies of the stability of teacher effects reached the same conclusion: Most teachers who ranked in the top quintile one year were not the “best” teachers the next year, and most teachers who ranked in the lowest quintile one year got better results the next year.21
In other words, being an effective teacher is not necessarily a permanent, unchanging quality. Some teachers are outstanding year after year, when judged by increases in their students’ test scores. Others are effective one year, but not the next, by the same measure. Some become better teachers over time. Some do not. Apparently, the test scores of their students reflected something other than what the teachers did, such as the students’ ability and motivation, or the characteristics of a class or conditions in the school. Maybe the teacher had a great interaction with the class one year but not the next. Or maybe the students assigned to the teacher were not similar from year to year.
Economists Brian A. Jacob, Lars Lefgren, and David Sims identified another major problem with the claim that three or four or five years of great teachers in a row would wipe out the achievement gap. They said that learning gains do not all persist over time. Students forget, gains fade. After a year, only about 20 percent and at best only one-third of any gain due to teacher quality persists. After two years, unless there is continual reinforcement of learning, only one-eighth of the gain persists. The authors concluded, “Our results indicate that contemporary teacher value-added measures may overstate the ability of teachers, even exceptional ones, to influence the ultimate level of student knowledge since they conflate variation in short-term and long-term knowledge. Given that a school’s objective is to increase the latter, the importance of teacher value-added measures as currently estimated may be substantially less than the teacher value-added literature indicates.”22
If these studies are right, then the gains students make each year do not remain intact and accumulate. You can’t just add up the gains of one year and multiply by three, four, or five. Teachers are very important, but students don’t remember and retain everything they learn.
Another economist, Jesse Rothstein, tested three methods of calculating value added by teachers and determined that none of them was reliable. He surveyed data for 99,000 North Carolina fifth graders and asked what effect their current teachers had on their scores in third and fourth grades. This was a falsification test, a deliberate attempt to ask a question whose answer should have been “none,” since a fifth-grade teacher cannot influence students’ test scores in third and fourth grades. Yet he found that all three ways of calculating value added produced large effects on fourth-grade test scores, meaning that they were flawed. The reason for these results, he speculated, was that many students are not assigned to classes randomly, but according to their prior achievement. He reported that students’ gains tended to decay, and that only a third of the teachers who were in the top quintile based on two years of data were in the top quintile based on a single year of data.23 In sum, value-added modeling is rife with technical problems.
While economists traded arguments, prominent journalists joined the fray. Malcolm Gladwell, the best-selling author and writer for the New Yorker, wisely noted that teachers are not “solely responsible for how much is learned in a classroom, and not everything of value that a teacher imparts to his or her students can be captured on a standardized test.” Nonetheless, he was impressed by the likelihood that three or four years of testing data would show which teachers are very good and which are very bad. The biggest problem in school reform, he argued, is “finding people with the potential to be great teachers.” He cited the economists who had written that it doesn’t matter if a teacher has certification, a master’s degree, or high test scores. In light of all this, Gladwell concluded that “teaching should be open to anyone with a pulse and a college degree—and teachers should be judged after they have started their jobs, not before.”24
Nicholas D. Kristof, a columnist for the New York Times, described education as “Our Greatest National Shame.” He said that the national school system was broken, but he saw a bright spot on the horizon. First, he wrote, “good teachers matter more than anything; they are astonishingly important. It turns out that having a great teacher is far more important than being in a small class, or going to a good school with a mediocre teacher. A Los Angeles study suggested that four consecutive years of having a teacher from the top 25 percent of the pool would erase the black-white testing gap.” The Los Angeles study to which he referred was the analysis by Gordon, Kane, and Staiger. Impressed by their study, Kristof held that entry credentials and qualifications don’t matter, that it doesn’t matter if prospective teachers had certification or a graduate degree, went to a better college, or had higher SAT scores. Predictably, he proposed “scrapping certification, measuring better through testing which teachers are effective, and then paying them significantly more—with special bonuses to those who teach in ‘bad’ schools.”25
The theory seemed reasonable, but still and all, it was only a theory. The fact was that the theory had never been demonstrated anywhere. No school or school district or state anywhere in the nation had ever proved the theory correct. Nowhere was there a real-life demonstration in which a district had identified the top quintile of teachers, assigned low-performing students to their classes, and improved the test scores of low-performing students so dramatically in three, four, or five years that the black-white test score gap closed. Nor had any scholar adduced evidence that top-performing nations had opened the teaching profession to any college graduate who wanted to teach, without regard to their credentials or experience or qualifications.
One beneficiary of the ongoing debate about the importance of great teachers was Teach for America (TFA). This program began to attract national attention at the same time that growing numbers of economi
sts questioned the value of traditional teacher-training requirements for entry into teaching. In what became an oft-told tale, Princeton student Wendy Kopp wrote a senior thesis in which she proposed the idea of a teacher corps composed of top-flight graduates from the nation’s elite colleges and universities to teach for two years in schools enrolling low-income students. After her graduation, Kopp deployed her superb organizational skills to raise millions of dollars from corporations and philanthropists to create Teach for America, with the mission of closing the achievement gap. Its first class of 500 teachers went into classrooms in six low-income communities in 1990. By 2002, the TFA corps consisted of 2,500 teachers working at eighteen sites. Each year the number of applicants increased, and by 2009, there were nearly 7,500 corps members working in thirty-four urban and rural settings.26 Some of the TFA graduates later founded their own charter schools, such as KIPP. Some remained in teaching. The most prominent TFA alumna was Michelle Rhee, the chancellor of the Washington, D.C., school system. Most left teaching after two years, but certainly the time they spent as urban teachers was an important life experience for an influential group of well-educated young men and women.
TFA is akin to the Peace Corps in its appeal to youthful idealism. But does TFA improve the quality of education in the poor urban and rural districts where its members have taught? Linda Darling-Hammond criticized TFA for sending inexperienced young people to teach the nation’s most vulnerable children; Darling-Hammond had been the executive director of the National Commission on Teaching & America’s Future, whose call for better-trained, more professional teachers was in direct opposition to the economists’ critique of entry requirements and paper credentials. TFA seemed to be proof of the economists’ views. TFA corps members received only a brief training period in the summer before their jobs commenced, yet were supposedly more successful than teachers with paper credentials and experience.
However, the evidence was far from conclusive. Researchers who examined the effects of TFA came to contradictory conclusions. A study in Arizona in 2002 held that TFA teachers had a negative impact on their students as compared to certified teachers.27 A national evaluation in 2004 concluded that students with TFA teachers did as well in reading as those taught by a control group, and significantly better in mathematics. The study had a small sample size of forty-one TFA teachers and a control group of fifty-seven teachers. The gains for TFA teachers were significant but small: Their students’ math scores rose from the 14th percentile to only the 17th percentile.28
Darling-Hammond led a study of 4,400 teachers and 132,000 students in Houston and concluded that certified teachers consistently produced significantly higher achievement than uncertified teachers, and that uncertified TFA teachers had a negative or non-significant effect on student achievement. TFA teachers who stayed long enough to gain certification performed as well as other certified teachers.29 By contrast, a study of high school teachers in North Carolina concluded that TFA teachers were more effective than traditional teachers. It held that “the TFA effect . . . exceeds the impact of additional years of experience,” especially in mathematics and science. However, another study of high school students in North Carolina determined that traditionally prepared secondary teachers were more successful than beginning teachers, including TFA corps members, who lacked teacher training.30
Thomas J. Kane, Jonah E. Rockoff, and Douglas O. Staiger studied different cohorts of New York City teachers (certified, uncertified, TFA, and Teaching Fellows) and concluded that certification doesn’t make a difference in student test scores, but experience does, especially for newer teachers. They wrote that New York City’s teachers “are no different from other teachers around the country. Teachers make long strides in their first three years, with very little experience-related improvement after that.”31
Most studies find that new teachers are less effective than experienced teachers and that the first two years of teaching are the least successful. Most TFA teachers in urban districts leave after their two-year commitment ends, and 80 percent or more are gone after their third or fourth year.32 Thus, many TFA teachers leave the field just at the point when teachers become most effective.
So is TFA the answer to the nation’s need for large numbers of effective teachers? Can TFA meet its goal and close the achievement gap?
TFA is a worthy philanthropic effort to recruit bright young people to teach in beleaguered districts. The organization brings highly educated young people into the teaching profession, if only for a few years. If some choose to remain in teaching, that is a net plus. Those who leave will have a deeper understanding of the needs of the schools as they enter other walks of life, and that too is a net plus for them and for the schools.
But it is simply an illusion to see TFA as the answer to the nation’s need for more and better teachers: The number of TFA teachers is insufficient to make a large difference in the teaching profession, and most will be gone after two or three years. TFA sends fewer than 10,000 new teachers each year into a profession with nearly 4 million members.
Similarly, it is an illusion to imagine that TFA will close the achievement gap. While some studies report significant gains for their students, none of the reported gains is large enough to close the gap that now exists between students of different racial groups. And since TFA members leave teaching at a higher rate than other new teachers, whatever gains they achieve cannot be sustained. A constant churn of new teachers into urban schools does not help the schools achieve the stability or the experienced teaching staff they need.
We should applaud the idealism of young people from our finest universities who want to devote two or three years to teaching. Nevertheless, the overwhelming majority of new teachers will not be drawn from elite institutions but will continue to come from state universities, where the quality of their education will determine their ability to improve the education of the next generation. The nation needs a steady infusion of well-educated teachers who will make a commitment to teaching as a profession. Every state university and teacher-preparation program should ensure that their graduates have a strong foundation in the liberal arts and sciences and are deeply grounded in the subjects they plan to teach.
To make teaching attractive to well-educated young people who are just starting their careers, teaching should offer good salaries and good working conditions. When a man or woman becomes a teacher, he or she should immediately have the support of mentors and colleagues. Simply knowing a lot about history or mathematics or reading theory is no guarantee that one can teach it well. On the other hand, too many teachers are immersed in pedagogy but are poorly educated in any subject matter. Teachers need both. They need to be well educated in whatever they plan to teach, preferably in more than one subject since assignments change. New teachers need to know about classroom management, interacting with children, helping children with special needs, communicating with parents, and working with colleagues. New teachers would also benefit if the schools in which they taught had a coherent curriculum, so they knew what they were expected to teach. And teaching would be enhanced if schools of education stopped insisting on pedagogical conformity and recognized that there are many ways to be a successful teacher.
When it is time for a principal to decide whether a teacher should get tenure, it should be treated as a weighty responsibility, not a routine matter. Given the availability of data about test score gains that every state now collects, it is inevitable that this information will become part of the tenure decision. Principals should know which teachers were very effective in teaching their students to read or do math, which teachers were effective on average, and which were extremely ineffective. Undoubtedly this information will become part of the principal’s decision and should be used in conjunction with observations and peer evaluations. Presumably the principal will not grant tenure to a teacher whose students consistently failed to learn anything over a three- or four-year period.
Because the principal must decide w
hich teachers will receive tenure, it is crucial that principals have prior experience as teachers and understand what good teaching is and how to recognize it. They will be called upon to evaluate and help struggling teachers, which they cannot do unless they have experience in the classroom. In recent years, a number of programs have recruited and trained new faces to serve as principals, some of them young teachers with only a few years of classroom experience or noneducators who have never been teachers. People with so little personal knowledge of good instruction—what it looks like, how to do it, and how to help those who want to do it—are likely to rely exclusively on data because they have so little understanding of teaching. This is akin to putting a lawyer in charge of evaluating doctors, or a corporate executive in charge of evaluating airline pilots. The numbers count for something, but on-site evaluation by an experienced, knowledgeable professional should count even more.
The Death and Life of the Great American School System Page 24
