More Than a Score

by Jesse Hagopian

  JH: That’s wonderful. So instead of a student at the end of a semester sitting down with a pencil, paper, test, and eliminating wrong-answer choices, they have to do a literary analysis. And what does that look like in the classroom?

  PT: We have teacher-designed rubrics, and I think if you look at the rubrics, you’ll understand what kind of standards we’re trying to hold the students to. They are writing papers all semester long. It’s not like all of a sudden they do a PBAT at the end of the semester. In every English class they’re taking, they’re doing maybe three literary analyses or taking in-class essay exams. There are lots of assessments going on during the year. I might say to a student, “This paper is a strong analysis and you came up with some really good ideas. I think you need to revise it , and it has a possibility of being a PBAT paper.” Whereas I might say to another student, “We need to work on a lot of revision. You still need to get your writing to a better level, and maybe next semester you’ll be able to do a PBAT paper.”

  Then the way the PBAT paper works is not only I but another teacher has to approve it using the rubric and saying this paper has reached at least a competent level on the rubric. And then the student conducts a presentation for someone other than the classroom teacher. Two other people evaluate a student’s work. We’re in the middle of Manhattan, so we’ve always been able to get a lot of people very interested in meeting with a student and talking to them about their work. They get the papers in advance. Or sometimes the student reads a completely new book and an external evaluator sits down with them and evaluates. We have lawyers and writers and journalists and other teachers participating. We’re located very near Rockefeller University, which has graduate-level science and research, so for our science classes we’ve always had people come in and question the students on their work. And what’s interesting is you can have a kid whose paper is at the competent level, it’s not beyond that. It’s not outstanding but shows a level competency. And then you listen to her defend her paper, and you can see that this is a kid who knows so much more than she’s been able to express through her writing. Orally some students can sometimes express much more about what they’ve learned than might be evident in their paper. And the external evaluators are very impressed by what they see a student knows.

  At Urban, we also have a studio art PBAT and an art criticism PBAT—each school in the consortium adds on supplementary PBATs. And we have artists come in and critique the students’ work. I mean, I can’t imagine being sixteen or seventeen and having had an adult come in and spend so much time looking at my work. It must be so flattering for the student to have that kind of critique and someone to take your work so seriously. That must be an incredible experience when you’re young. So we have a number of people who are all too happy to sit down and have a really interesting discussion and ask the students questions.

  JH: That’s wonderful. So you draw on the community around you to come in and bring in their expertise.

  PT: Yes and we’ve learned that the oral examination can be very much a learning experience, not just an assessment experience, and that’s what we really want. We hope we can reach that level with every kid, where a kid at the end can say about our assessments, “That was really interesting.”

  JH: That seems like a big difference to the standardized test that is wasted pedagogical time. So the assessment is inspiring rather than mind-numbing and anxiety-producing.

  PT: Yes.

  JH: That’s great. I want to ask you a follow-up question on this because I was reading a book by Daniel Koretz called Measuring Up.

  PT: Sure.

  JH: And he has some critiques of norm-referenced standardized tests and some important things we need to look at in terms of pitfalls that the tests can create. But he also has critiques of performance assessments and I’m wondering how you would respond to his charges that performance assessments are difficult to score reliably. And he also charges that they’re hard to compare the scores year to year and that this makes them invalid.

  PT: On our Web site [www.performanceassessment.org], we publish exemplar papers. And those were papers that went through quite a number of levels of moderation within the consortium and then were sent out to educators who had nothing to do with the consortium. So those are models of competent, good, and outstanding papers. And we always refer people to look at those. Plus we do the moderation study every year. We’re developing a new Web site. We’re going to post more of those papers so that you can get an idea of what “competence” in science looks like, what “outstanding” in math looks like, for example.

  One of the psychometricians we worked with a lot was Dr. Robert Stake, and he used to say to me, “So what if a paper gets judged outstanding by one person and just competent by another person? Really? How much does it matter?” The important thing for him—and I think for most psychometricians—was to determine if the work was passing or not passing.

  But more than that, we want to find out what are the outcomes for our students in their lives. For years when we were doing this, all anybody cared about was test results. And from the very beginning we were talking about what happens to the kids when they graduate? Can they go on to college? Can they succeed? Can they sustain themselves in college? For years we said, you know, you need the big picture here. And that’s kind of been our—what do they call it?—“predictive reliability.” That’s really what we’ve always thought was the important part.

  I’m not saying that every student should be in college. Unfortunately, there aren’t many vocational options available at this point. But you know, our goal has always been, we want to present every student with the option of college. We want to know that we’ve gotten every kid to the level where they can survive in college. And then looking at how many are doing that. Our very first study of outcomes from our program was done by Dr. Martha Foote, who looked at transcripts of the students in their second year of college to see if they were maintaining their GPA, and had they even returned for a second year—there is often a big drop-off between freshman and sophomore year.

  JH: Right.

  PT: And we did very well. Then we did the more recent data report, with information from the National Student Clearinghouse, and it shows our students are surviving into eighteen months, into the second year of college. So for us, that’s much more important than getting the statistical reliability on a standardized test.

  JH: So, real-world application. From what I’ve seen on the data report, the consortium schools have higher graduation rates than the comparable New York Public Schools, and higher college attendance rates. And then what you pointed out, the students are actually staying in college—probably because they have the critical thinking skills that are more valued at that level. That’s a really good reframing of how to look at the outcomes of schools rather than just the test score.

  PT: See, it all depends on what question you’re asking.

  JH: Right, and I think it’s important to point out in terms of reliability of scoring, that after our schooling is over, we stop filling in bubbles and the work we do is judged subjectively by human beings all the time.

  We’re at a really important crossroads moment in public education right now. I wonder how you would advise [secretary of education] Arne Duncan, and what you hope for the future of public education in the United States.

  PT: One of the problems with the resistance movement is there’s not enough being offered about what could be instead of standardized testing. So I just wish that there were more discussions about how education itself can benefit from a change in this kind of attitude of standardization, and there’s not enough discussion about the amount of money that is going into wasteful practices. Can you imagine the billions that have been spent on the worst kinds of testing and activities and all the groups that have gotten rich from all of this? The amount of money that’s being spent on so-called gurus and specialists and people who talk to each other but are so distant from the classroom and from the kids.
It’s just incredible. It’s a topsy-turvy world.

  But resistance is growing. The NEA [National Education Association] just asked Arne Duncan to resign. You know federal policies, like evaluating teachers using kids’ standardized test scores, are the source of a lot of problems. It’s important to note that most private schools, particularly the elite ones, are not subjected to an excessive standardized testing regime. I would say: what’s good enough for the elite of the country is good enough for every public school child, too.

  Afterword

  There is an educational crisis upon us, but it isn’t the one that the politicians, businessmen, pundits, and philanthropists would have you believe. Yes, there are significant and persistent problems with public education in this country. On the whole, working-class students and students of color and their communities have not been served well by our system of public education. Indeed, my language here is not strong enough. Historically speaking, public schools have done damage to working-class kids and kids of color. Historically, public education has been used as a tool for forcing cultural assimilation, first of poor European immigrants and then for all children in public schools. For decades we’ve had systems of tracking and funneling poor kids into low-level classes in preparation for low-skill jobs. By every available indicator (graduation rates, dropout rates, discipline rates, college entrance rates, even test scores) low-income black and brown kids are not getting access to equitable educational experiences and resources. The United States has one of the most stratified and inequitable school funding structures in the world, one that makes sure poor neighborhoods and communities also receive fewer resources for public education. Bilingual education is illegal in some states, and students of color are disproportionately losing recess, social studies, the sciences, and art, all in favor of increasing test scores.

  This kind of disparity and educational inequality has been the result of one of the fundamentally defining functions of public schooling in the United States: to effectively reproduce social, economic, and cultural relations that exist “outside” of school (I put “outside” in quotations marks here because we cannot really separate what is outside of school from what is inside schools). No matter how we look at it, this function of schooling has been an empirical fact for at least a hundred years, and standardized testing has played a critical role in legitimizing the reproduction of this ongoing inequality in education. Going back to their origins in IQ testing, with strong connections to the eugenics movement, these tests have been continuously promoted by psychologists, psychometricians (the formal term for test-making professionals and experts), politicians, business leaders, and even the general public as providing efficient and objective measures of the learning and intelligence of human populations.

  Based on that presumed objectivity, students have been categorized and placed into different levels with different expectations for achievement, higher education, and future vocation. The advent of formal high-stakes testing (attaching stakes to standardized tests) in recent decades has meant that these tests are now being used to do the same kinds of categorization of teachers, administrators, schools, districts, states, and countries. And all along this kind of categorization has been used to categorize and stigmatize entire populations—immigrants, the working class, and people of color, among others.

  Given the historical and contemporary realities of educational inequality in the United States, it is no wonder that communities of color have a historical distrust of public education. It is also no wonder that the current crop of corporate education reformers have seized on public education’s penchant for reproducing inequality as an opening for advancing their own solutions for fixing education in their preferred image: as a business model that sees humans as products, teaching and learning as data points, and education as a marketplace ripe for profiteering—all in the name of increasing educational equality, no less.

  The chapters of More Than a Score are important in this political and historical moment. They highlight the hypocrisy of a corporate education reform movement—one backed by both major political parties, both major teachers unions, business leaders, and philanthropists—that claims to promote equality through policies built around high-stakes standardized testing. As the essays here illustrate, high-stakes testing is doing the exact opposite: It is ruining curriculum and pedagogy, hurting children and communities of color, and creating a substandard education for the masses of people, all while alienating parents, teachers, students, and administrators from public education in the process. Put differently, high-stakes testing is alienating the public from public education and filling the space with corporate interests instead of human ones.

  More Than a Score is important here for another reason as well. It highlights one of the central contradictions posed by public education. In spite of its critical role in re-creating social, economic, and cultural inequality, public schools have also always carried within them a radically democratic impulse. In the rawest sense, this impulse is carried in the fact that learning always contains within it a potential for helping us understand things in new and different ways and thus spurring us to action. In just the same way that education can be used to suppress different kinds of teaching and maintain hegemony, education also can be used to help people better understand the conditions of their existence and take individual and collective action to change those very same conditions. Further, public education in this country is one of the last remaining public institutions: public schools are open to everyone and can be a meeting ground for all of our communities (unfortunately this is less true given the sharp increase in school segregation). In this sense, public education is a part of the “commons” for all of us, and they thus hold the possibility for community dialogue and collective action based on the needs of our children and society.

  So More Than a Score is an important book because the chapters included here illustrate the radically democratic impulse of public education in action. Corporate education reformers are ruining public education and the parents, students, teachers, administrators, and activists included here are fighting back by saying no! and doing so on grounds that are principled, ethical, humanistic, and educationally sound. The authors and activists included here also point us to the real crisis in educational inequality: massive disinvestment in state and social services and massive socioeconomic inequality in our nation.

  Test scores correlate most strongly with family wealth and education. We have known this for decades, and politicians, pundits, and, I’m sorry to say, some educational researchers, have either purposefully ignored this fact or obfuscated it behind the idea that we cannot even consider equalizing these things. Lack of adequate housing, food insecurity, inadequate access to health care—all things connected directly to poverty—are the real culprits here. But the neoliberal corporate education reformers do not want us to look for solutions in massive social and economic change. They do not want to admit that public education is part of a larger web of social services that are impacted by poverty. When poverty rises, for instance, we will see the effects in educational achievement in any way it is measured. Instead they play games with high-stakes test scores, blaming teachers for everything through the sham of VAM (value-added models), and they talk about “grit” and how any individual can succeed if she or he just works hard enough (of course never admitting that the privileged have to work less hard, thereby increasing their chances of success to begin with). They keep chanting the corporate reformer mantra of “the market will provide,” closing their eyes to the increasing inequality associated with their schemes, and all the while making obscene profits. As neoliberalism and the logics of private industry continue their creep into mainstream politics, More Than a Score reminds us of the importance of defending public education from these corporate raiders and rebuilding our schools in radically democratic, humanistic, and equitable ways.

  Acknowledgments

  The primary belief of the testocracy is in the supremacy of indi
vidual achievement at the expense of collaboration. Consider this book a repudiation of that crackpot theory: the development of the ideas within these pages about why and how to resist high-stakes testing has been a profoundly social and collaborative project. I happened to be swept up in events that gave me the opportunity to help create this book; however, it was the collective protest of countless students, parents, teachers, administrators, writers, and activists that made this book possible. I am grateful to all the wonderful authors and activists who told their stories of resistance to high-stakes testing in this book. Special thanks are owed to Aaron Dixon, Alexia Garcia, Brian Jones, Karen Lewis, Marilena Marchetti, Monty Neill, Adam Sanchez, Lee Sustar, Dan Troccoli, and Dave Zirin, whose counsel has been invaluable to building this movement. A special appreciation is due to the members of the Social Equality Educators, who have devoted their lives to education and assessment justice.

  I want to express my gratitude to the Bulldogs—the parents, students, and educators at Garfield High School. My colleagues at Garfield refused to administer the MAP test and in so doing changed the world. I am in awe of your bravery, talent, and commitment to the highest level of pedagogy. All of you were crucial to this collective effort, but I would be remiss if I didn’t acknowledge those of you that I have worked so closely with in organizing the MAP test boycott and ongoing efforts to defend and transform public education: Mallory Clarke, Kris McBride, Kit McCormick, Jessica Griffin, Rachel Eells, Heather Robison, Adam Gish, and Jerry Neufeld-Kaiser. Many thanks as well to the Garfield Black Student Union and faculty co-advisor, Kristina Clark.

  The folks at Haymarket Books are a treasure to many social movement activists, especially those involved in the fight to stop high-stakes testing. Anthony Arnove’s and Julie Fain’s vision, consultation, and belief in this book made it possible. I couldn’t have been more fortunate than to have as my editor Dao X. Tran, an activist in the education justice movement and a contributing author—she was my teacher about what it takes to actually publish a book and provided invaluable feedback and revisions. Rachel Cohen’s great talent and generosity with her time helped achieve the cover art and layout of the book, a beautiful frame to hang our stories on. Rory Fanning, Jim Plank, and Jason Farbman all believe deeply in this project and their many skills have facilitated its production. Thanks to Robin Horne and Meredith Reese for transcribing some of the interviews in this collection.