Super Crunchers

by Ian Ayres

Class (in unison): “The Pet Goat.”

  Yes. “The Pet Goat.” Fingers under the first word of the story. Get ready to read the story the fast way. GET READY!

  The class begins reading the story in unison. As they read, the teacher taps her ruler against the board, beating out a steady rhythm. The students read one word per beat.

  Class (to the beat): A girl got a pet goat.

  Go on.

  Class (to the beat): She liked to go running with her pet goat.

  Go on.

  Class (to the beat): She played with her…

  Try it again. Get ready, from the beginning of that sentence. GET READY!

  Class (to the beat): She played with her goat in her house.

  Go on.

  Class (to the beat): The goat ate cans and he ate canes.

  Go on.

  Class (to the beat): One day her dad said that goat must go.

  What’s behind the word “said”?

  Class (in unison): Comma.

  And what does that comma mean?

  Class: Slow down.

  Let’s read that sentence again. Get ready!

  Class (to the beat): One day her dad said (pause) that goat must go.

  Go on.

  Class (to the beat): He eats too many things.

  Go on.

  Class (to the beat): The girl said that if you let the goat stay with us I will see that he stops eating all those things.

  Nice and loud, crisp voices. Let’s go.

  Class (to the beat): Her dad said that he will try it.

  Go on.

  Class (to the beat): But one day a car robber came to the girl’s house.

  Go on.

  Class (to the beat): He saw a big red car in the house and said I will steal that car.

  Go on.

  Class (to the beat): He ran to the car and started to open the door.

  Go on.

  Class (to the beat): The girl and the goat were playing in the backyard.

  Go on.

  Class (to the beat): They did not see the car robber. More to come.

  More to come? This is a real cliff-hanger. Will the goat stop the car robber? Will the dad get fed up and kick the goat out?

  Millions of us have actually seen Ms. Daniel’s class. However, in the videotape, our attention was centered not on the teacher or the students but on a special guest who was visiting that day. The special guest, who was sitting quietly by Ms. Daniel’s side, was President George W. Bush.

  The videotape of the class was a central scene in Michael Moore’s Fahrenheit 9/11. Just as Ms. Daniel was asking her students to “open your book to lesson sixty,” Andrew Card, the president’s chief of staff, came over and whispered into Bush’s ear, “A second plane hit the second tower. America is under attack.”

  Moore’s purpose was to criticize Bush for not paying more attention to what was happening outside the classroom. Yet what was happening inside Ms. Daniel’s classroom concerns one of the fiercest battles raging about how best to teach schoolchildren. Bush brought the press to visit this class because Ms. Daniel was using a controversial, but highly effective, teaching method called “Direct Instruction” (DI).

  The fight over whether to use DI, like the fight over evidence-based medicine, is at heart a struggle about whether to defer to the results of Super Crunching. Are we willing to follow a treatment that we don’t like, but which has been shown in statistical testing to be effective?

  Direct Instruction forces teachers to follow a script. The entire lesson—the instructions (“Put your finger under the first word.”), the questions (“What does that comma mean?”), and the prompts (“Go on.”)—is written out in the teacher’s instruction manual. The idea is to force the teacher to present information in easily digestible, bitesize concepts, and to make sure that the information is actually digested.

  Each student is called upon to give up to ten responses each minute. How can a single teacher pull this off? The trick is to keep a quick pace and to have the students answer in unison. The script asks the students to “get ready” to give their answers and then after a signal from the teacher, the class responds simultaneously. Every student is literally on call for just about every question.

  Direct Instruction also requires fairly small groups of five to ten students of similar skill levels. Small group sizes make it harder for students to fake that they’re answering and it lets the teacher from time to time ask individual students to respond, if the teacher is concerned that someone is falling behind.

  The high-speed call and response of a DI class is both a challenging and draining experience. As a law professor, it sounds to me like the Socratic method run amok. Most grade schoolers can only handle a couple of hours a day of being constantly on call.

  The DI approach is the brainchild of Siegfried “Zig” Engelmann, who started studying how best to teach reading at the University of Illinois in the 1960s. He has written over 1,000 short books in the “Pet Goat” genre. Engelmann, now in his seventies, is a disarming and refreshingly blunt academic who for decades has been waging war against the great minds of education.

  Followers of the Swiss developmental psychologist Jean Piaget have championed child-centered approaches to education that tailor the curriculum to the desires and interests of individual students. Followers of the MIT linguist and polymath Noam Chomsky have promoted a whole-language approach to language acquisition. Instead of breaking reading into finite bits of information in order to teach kids specific phonic skills, the whole-language approach embraces a holistic immersion in listening to and eventually reading entire sentences.

  Engelmann flatly rejects both the child-centered and whole-language approaches. He isn’t nearly as famous as Chomsky or Piaget, but he has a secret weapon—data. Super Crunching doesn’t say on a line-by-line basis what should be included in Zig’s scripts, but Super Crunching on the back end tells him what approaches actually help students learn. Engelmann rails against educational policies that are the product of top-down philosophizing instead of a bottom-up attention to what works. “Decision makers don’t choose a plan because they know it works,” he says. “They choose a plan because it’s consistent with their vision of what they think kids should do.” Most educators, he says, seem to have “a greater investment in romantic notions about children” than they do “in the gritty detail of actual practice or the fact that some things work well.”

  Engelmann is a thorough pragmatist. He started out in his twenties as an advertising exec who tried to figure out how often you had to repeat an ad to get the message to stick. He kept asking the “Does it work?” question when he turned his attention to education.

  The evidence that DI works goes all the way back to 1967. Lyndon Johnson, as part of his War on Poverty, wanted to “follow through” on the vanishing gains seen from Head Start. Concerned that “poor children tend to do poorly in school,” the Office of Education and the Office of Economic Opportunity sought to determine what types of education models could best break this cycle of failure. The result was Project Follow Through, an ambitious effort that studied 79,000 children in 180 low-income communities for twenty years at a price tag of more than $600 million. It is a lot easier to Super Crunch when you have this kind of sustained support behind you. At the time, it was the largest education study ever done. Project Follow Through looked at the impact of seventeen different teaching methods, ranging from models like DI, where lesson plans are carefully scripted, to unstructured models where students themselves direct their learning by selecting what and how they will study. Some models, like DI, emphasized acquisition of basic skills like vocabulary and arithmetic, others emphasized higher-order thinking and problem-solving skills, and still others emphasized positive attitudes toward learning and self-esteem. Project Follow Through’s designers wanted to know which model performed the best, not only in developing skills in its area of emphasis, but also across the board.

  Direct Instruction won hands down. Education write
r Richard Nadler summed it up this way: “When the testing was over, students in DI classrooms had placed first in reading, first in math, first in spelling, and first in language. No other model came close.” And DI’s dominance wasn’t just in basic skill acquisition. DI students could also more easily answer questions that required higher-order thinking. For example, DI students performed better on tests evaluating their ability to determine the meaning of an unfamiliar word from the surrounding context. DI students were also able to identify the most appropriate pieces to fill in gaps left in mathematical and visual patterns. DI even did better in promoting students’ self-esteem than several child-centered approaches. This is particularly striking because a central purpose of child-centered teaching is to promote self-esteem by engaging children and making them the authors of their own education.

  More recent studies by both the American Federation of Teachers and the American Institutes for Research reviewed data on two dozen “whole school” reforms and found once again that the Direct Instruction model had the strongest empirical support. In 1998, the American Federation of Teachers included DI among six “promising programs to increase student achievement.” The study concluded that when DI is properly implemented, the “results are stunning,” with DI students outperforming control students along every academic measure. In 2006, the American Institutes for Research rated DI as one of the top two out of more than twenty comprehensive school reform programs. DI again outperformed traditional education programs in both reading and math.

  “Traditionalists die over this,” Engelmann said. “But in terms of data we whump the daylights out of them.”

  But wait—it gets even better. Direct Instruction is particularly effective at helping kids who are reading below grade level. Economically disadvantaged students and minorities thrive under DI instruction. And maybe most importantly, DI is scalable. Its success isn’t contingent on the personality of some über-teacher. DI classes are entirely scripted. You don’t need to be a genius to be an effective DI teacher. DI can be implemented in dozens upon dozens of classrooms with just ordinary teachers. You just need to be able to follow the script.

  If you have a school where third graders year after year are reading at a first-grade level, they are seriously at risk of being left behind. DI gives them a realistic shot of getting back to grade. If the school adopts DI from day one of kindergarten, the kids are much less likely to fall behind in the first place.

  Imagine that. Engelmann has a validated and imminently replicable program that can help at-risk students. You’d think schools would be beating a path to his door.

  What Am I, a Potted Plant?

  Direct Instruction has faced severe opposition from educators on the ground. They criticize the script as turning teachers into robots, and for striving to make education “teacher proof.”

  Can you blame them for resisting? Would you want to have to follow a script most of your working day, repeating ad nauseam stale words of encouragement and correction? Most teachers are taught that they should be creative. It is a stock movie genre to show teachers getting through to kids with unusual and idiosyncratic techniques (think To Sir with Love, Stand and Deliver, Music of the Heart, Mr. Holland’s Opus). No one’s going to make a motivational drama about Direct Instruction.

  Engelmann admits that teacher resistance is a problem. “Teachers initially think this is horrible,” he said. “They think it is confining. It’s counter to everything they’ve ever been taught. But within a couple of months, they realize that they are able to teach kids things that they’ve tried to teach before and never been able to teach.”

  Direct Instruction caused a minor schism when it was introduced into Arundel Elementary in 1996. Arundel Elementary is perched upon a hill in Baltimore’s struggling Cherry Hill neighborhood. It is surrounded by housing projects and apartment complexes. Ninety-five percent of its students are poor enough to qualify for federally subsidized lunches. When Arundel adopted DI, several teachers were so frustrated with the script that they transferred to other schools. The teachers who stayed, though, have come to embrace the system. Matthew Carpenter teaches DI seven hours a day. “I like the structure,” he said. “I think it’s good for this group of kids.”

  Most readers of this book probably couldn’t abide the idea of having to follow a script hour after hour. Still, there is a joy is seeing your students learn. And a public school teacher confided in me that some of her colleagues liked it for a very mundane reason: “Zero prep,” she said. That’s right, instead of having to plan your own class lesson day after day, DI instructors can walk into class, open the book, and read, “Good morning, class…”

  Engelmann’s website is clear, if somewhat diplomatic, in emphasizing that teacher discretion is reduced by the Direct Instruction method. “The popular valuing of teacher creativity and autonomy as high priorities must give way to a willingness to follow certain carefully prescribed instructional practices,” reads the DI website. Engelmann puts the matter more bluntly: “We don’t give a damn what the teacher thinks, what the teacher feels,” he said. “On the teachers’ own time they can hate it. We don’t care, as long as they do it.”

  The Empire Strikes Back

  Engelmann also faces resistance from the academic establishment. The education community is largely unified in their opposition to Direct Instruction. Ignoring the data, they argue that DI doesn’t teach high-order thinking, thwarts creativity, and is not consistent with developmental practices.

  Opponents argue that DI’s strict methodology does not promote learning so much as prompting students to robotically repeat stock answers to scripted questions. They contend that while students learn to memorize responses to questions they expect, students are not prepared to apply this base knowledge to new situations. DI’s critics also express concern that its structured approach, with tedious drills and repetition, stifles both student and teacher creativity. They argue that the method treats students as automatons leaving little room for individual thinking. These criticisms, however, ignore the possibility—supported by evidence from standardized tests—that DI equips students who have acquired a stronger set of basic skills with a greater capacity to build and develop creativity. Teachers interviewed after implementation of DI in Broward County, Florida, said the “approach actually allowed more creativity, because a framework was in place within which to innovate,” and added that classroom innovation and experimentation were a lot easier once DI had helped students acquire the necessary skills.

  Lastly, critics try to discredit DI by arguing that DI causes antisocial behavior. At public meetings, whenever the possibility of switching to DI is mentioned, someone is sure to bring up a Michigan study claiming that students who are taught with DI are more likely to be arrested in their adolescent years. Here’s evidence, they say, that DI is dangerous. The problem is that this randomized study was based on the experience of just sixty-eight students. And the students in the DI and the control groups were not similar.

  In the end, the Michigan study is just window dressing. The education establishment is wedded to its pet theories regardless of what the evidence says. Education theorist and developer of the Success for All teaching model Robert Slavin puts it this way: “Research or no research, many schools would say that’s just not a program that fits with their philosophy.” For many in the education establishment, philosophy trumps results.

  The Bush administration, however, begs to differ. The 2001 “No Child Left Behind” law (NCLB) mandates that only “scientifically based” educational programs are eligible for federal funding. The NCLB statute uses the term “scientifically based research” more than one hundred times. To qualify as “scientifically based,” research must “draw on observation or experiment” and “involve rigorous data analyses that are adequate to test the stated hypotheses.” This is the kind of stuff that would make any Super Cruncher salivate. Finally a fair fight, where the education model that teaches the best will prevail.

&nbs
p; Bush’s education advisors have been taking the mandate quite seriously. The Department of Education has taken the lead, spending more than $5 billion in funding randomized testing and funding evidence-based literature reviews to assess the state of knowledge of “what works.” As Fahrenheit 9/11 shows, Bush is personally flogging the effectiveness of Direct Instruction.

  On the ground, however, the requirement that states adopt scientifically based methods has not worked a sea change on the education environment. State education boards currently tend to require textbooks and materials to be a mishmash of elements that individually are supposed to be scientifically based. A “balanced literacy” approach, which mixes elements of phonetic awareness as well as holistic experiences in reading and writing, is now in the ascendancy. California requires that primary reading materials contain a mixture of broad features.

  Ironically, NCLB’s requirement of “scientifically based” methods has become the catalyst for excluding Direct Instruction from many states’ approved lists because it does not contain some holistic elements. There are no good studies indicating that “balanced learning” materials perform as well as Direct Instruction, but that doesn’t keep states from disqualifying DI as even an option for local school adoption. At the moment, Direct Instruction, the oldest and most validated program, has captured only a little more than 1 percent of the grade-school market. Will this share rise as the empirical commands of NCLB are more fully realized? In the immortal words of “The Pet Goat,” “more to come.”

  The Status Squeeze

  The story of Engelmann’s struggle with the educational establishment raises once again the core themes of this book. We see the struggle of intuition, personal experience, and philosophical inclination waging war against the brute force of numbers. Engelmann for decades has staked out the leading edge of the Super Cruncher’s camp. “Intuition is perhaps your worst enemy,” Engelmann said, “if you want to be smart in the instructional arena. You have to look at the kid’s performance.”