More Than Good Intentions

by Dean Karlan

  But the most striking difference between the standard program and the two behaviorally motivated variations was in tertiary institution enrollment. Here the impact of the basic program was statistically indistinguishable from zero, while the variations increased enrollment substantially. Starting from an enrollment rate of 21 percent in the control group, both variations were big improvements: The first increased tertiary enrollment by almost half, and the second more than tripled it!

  We don’t want to get lost in the numbers, but it is important to see just how powerful the details—like timing—can be. Like many other behavioral nudges we have seen so far, Bogotá’s variations on the basic conditional cash transfer program have me excited. They are elegant and clever. More important, they are attractive to policymakers and practitioners, who understand—and would rather avoid—the challenges inherent in completely overhauling programs or designing new ones from scratch.

  Subtle improvements like these, which leverage the importance of timing in household decision making, can have tremendous consequences.

  The Surprise Knockout: Deworming

  When all is said and done, there is one school attendance program that stands head and shoulders above the rest; and, to be honest, it snuck up on researchers unexpectedly.

  Michael Kremer was at work again in Kenya, this time with Edward “Ted” Miguel of UC Berkeley, on the unseemly problem of worms—hookworm, roundworm, whipworm, and schistosomiasis. Many of us know these characters primarily as the villains of travelers’ stories, where they usually amount to an annoyance; but they are a far more tragic reality of everyday life for billions, especially in developing countries. They infect one in four people worldwide.

  Heavy worm infections can bring on symptoms like severe abdominal pain, anemia, and protein malnutrition, which might put a person out of commission; but the vast majority of cases are milder. Ironically, this is a big part of the problem. Worms can cause a general and persistent malaise—lethargy, slight queasiness—that people get used to. Many live with it all the time.

  Biologically, the worms are parasites that live and breed in human and animal feces. They are typically contracted when people come into contact with fecally contaminated water or soil. The particulars of transmission vary slightly from worm to worm, but they’re all devastatingly easy to catch. Depending on the species, ingesting motes of contaminated dirt by eating with unwashed hands, playing in fresh water, or even walking barefoot through puddles near where infected people or animals have defecated—any of these commonplace behaviors could result in infection. It doesn’t take much imagination to see why worms are a scourge of children in developing countries.

  Fortunately, a highly effective treatment exists for these parasites—a single deworming pill that eradicates roughly 99 percent of worms currently in the body and provides protection for about four months. Even better, the total cost of manufacturing, transporting, and administering the treatment to at-risk children is about twenty cents per pill.

  From a public health perspective, providing such an inexpensive cure to anybody who wants it is practically a no-brainer, purely on the basis of benefits to the individual, but there is actually a stronger case for giving deworming pills away. The spread of worms is a chain reaction: Worms spread through contaminated soil and water; soil and water become contaminated through the presence of contaminated feces; feces are contaminated only if the people producing them have worms. So when more people in a community are infected, they create a hazardous environment for everyone else. Conversely, when fewer individuals are infected, the rest of the community is safer too.

  A case like this, where the general public benefits when an individual gets treatment, practically cries out for an intervention. We should do whatever we can to get people dewormed—not just for their own sake, but for everybody’s. This sound reasoning was one of the factors that led Kremer and Miguel to get involved in evaluating a program that provided deworming pills for free to students in primary schools in western Kenya in 1998.

  They partnered again with ICS (the same organization that gave away school uniforms) and devised a simple program. ICS officers would first meet with students’ parents at the school to describe deworming and secure their consent. Then they would return and administer the pills to all the students whose parents had agreed. Most parents (about 80 percent) signed their children up.

  Kremer and Miguel designed a study to test the impact of the program on both health and education outcomes. ICS had identified seventy-five primary schools to work with, and the researchers divided them into three groups. Twenty-five schools would get the program in 1998, twenty-five schools in 1999, and the remainder in 2001. Like the Progresa evaluation in Mexico, the phase-in design allowed ICS to provide treatment to everybody they wanted (albeit over time), and also generate rigorous evidence about their program.

  Given the proven effectiveness of deworming medication, the researchers fully expected to see noticeable gains in students’ health. They were not disappointed. The program cut the total number of worm infections in half—not just for those who took the pills, but for all the students in the schools where they were offered. It was the story of cascading community benefits, just as they had hoped: Disrupting the cycle of infection made even those who weren’t taking the pills better off. There were simply fewer worms around to infect people.

  But another result emerged that took them by surprise, at least in its magnitude: Students started coming to class more. A lot more. The absence rate in program schools fell by about a quarter. Much to their—and the students’—satisfaction, ICS had hit on a tremendously powerful way to get kids into classrooms.

  Dollar for dollar, it’s no contest. The other attendance programs did work, but compared with deworming they cost an arm and a leg. Crunching the numbers, an additional year of school enrollment from Progresa comes out to about $1,000 a head. Generating an extra year of school attendance with the uniform-giveaway program costs roughly $100 per student. An additional year of attendance from deworming costs $3.50. Yes, you read that right.

  Sure enough, the remarkable results of Miguel and Kremer’s initial study made their way around the development world. Soon there was interest in school-based deworming far beyond the Kenyan bush. Miguel and Kremer were confident in the research they had done, but they weren’t ready to recommend school-based deworming always and everywhere. They recognized that a single evaluation could only tell them so much.

  At the end of the day, they had a sizable piece of hard evidence to support a simple theory: Where school absenteeism and worm infection rates are high, school-based deworming can be a powerful attendance driver. As with any scientific theory, the only way to add credence to theirs was to put it to the test again.

  They didn’t have to wait long. In 2001, just as the Kenyan study was wrapping up, Miguel, along with Gustavo Bobonis of the University of Toronto and Charu Puri-Sharma of India’s Niramaya Health Foundation, designed an RCT to evaluate a deworming program for preschool students in Delhi, India. Here they were up against more than intestinal worms, which afflicted about one in three students. The other problem was anemia, another bane of children in developing countries that can be reliably treated for pennies (in this case, with iron supplements) but rarely is. A staggering 69 percent of preschool students in their study suffered from the disease.

  The program ran much like the Kenyan one: Program officers sought permission from students’ parents, then administered deworming, iron, and vitamin A pills three times per year at schools. Sure enough, absence rates fell by about 20 percent—about the same amount they had fallen in Kenya.

  Replicating the initial Kenyan result greatly strengthened the case for scaling up school-based deworming around the world. With a sensible theory—that school-based deworming can work in settings where worm infections are common—and mounting evidence to support it, advocates were soon beating the drums and calling from the hilltops. Their arguments were further bolst
ered by research by Hoyt Bleakley at the University of Chicago on historical data from the American South, where the Rockefeller Foundation’s efforts to eradicate hookworm in 1910 led to higher incomes in the long run. Evidence still rolling in from Kenya corroborates this story. Follow-up surveys with participants from Kremer and Miguel’s original deworming study found that, a decade later, students who had been assigned to the early treatment groups (and had thus received two or three additional years of school-based deworming treatment) were working 13 percent more hours and earning 20 to 29 percent more income than their late-treatment counterparts. Those are big, long-lasting gains from a few twenty-cent pills.

  Happily, all this good news has grabbed people’s attention. School-based deworming has been one of the great recent success stories of evidence-based decision making in development, with upward of twenty million students in 26 countries dewormed in 2009 alone.

  Anthony Again

  When we left Anthony he was walking through the rain down the hill toward the trotro station, hurrying to make it there in time for the last van heading north out of Accra. He had no free uniform, no merit scholarship, and no conditional cash transfer bonus waiting for him; but he did have aspirations and a potential benefactor, and these were better than nothing. Some weeks later, after more discussion (this time, thankfully, by phone), Jake agreed to pay the fees for two applications. One went to the four-year liberal arts university, and the other to the two-year training college for teachers. Anthony was on tenterhooks waiting for his admissions letters.

  In mid-June he called to say he had been accepted to the training college, and a few weeks later came the news that he “might have a chance” at the university. He sounded excited. When Jake asked what “might have a chance” meant, he explained that some applicants are accepted outright, others rejected, and still others offered “a chance” to matriculate—meaning they can bribe an admissions officer to get a spot. Apparently it would have been gauche for the officer to name a specific price, but Anthony figured a couple hundred dollars would do the trick.

  Now the picture was becoming clear, and it looked lousy. Jake told Anthony he was willing to cover tuition, but not a bribe. But Anthony was adamant that it wasn’t really a bribe per se. This was how things worked. Still, Jake was disgusted just thinking about it—a big, smiling man closing his meaty hand around a stack of bills in some sweaty back office while Anthony stood nervously, also smiling, his eyes darting so as not to fix on the money. Besides, where would it end? Anthony conceded that students who slip in through the back door are sometimes called upon for more palm greasing later.

  So Anthony settled for his second choice, the teacher training college. The good news was that simply being enrolled there gave him the opportunity to begin working right away, as a part-time teacher at a private elementary school. He found a job in a village not far from the college and began working that summer. He caught up with Jake some months later when he called to ask for a loan to cover his rent. He had been keeping a single room in a boardinghouse near the school.

  Jake was confused. “Anthony, why can’t you pay it yourself? Haven’t you been earning money from teaching?”

  “Yes, Mister Jacob, yes. I have been earning money from teaching. But it is just that I have not been getting it.”

  “Not getting what?”

  “The money.”

  “I don’t understand. Have you been paid?”

  “Yes. No. That is, the school proprietor, the one who is owing us money. He said he wanted to pay us, the teachers, but he is not having anything.”

  “Oh. How can he do that? How can he ask you to work if he doesn’t have money to pay you?”

  “Yes, that is our challenge. As for the payment, he said he cannot give us what he himself does not have.”

  “Well, when was the last time you received a paycheck?”

  “I am still waiting on that one.”

  Anthony had been working four months and hadn’t seen a dime. He and the other teachers had a plan, though, and their plan made sense. If the proprietor couldn’t pay, they wouldn’t work. It was that simple. The only loose end, it seemed, was the students.

  Step Two: Getting Teachers into Classrooms

  As was said earlier, though we cannot claim to know the whole recipe for education, we are sure of at least two ingredients: students and teachers. So far in this chapter we have visited a number of innovative programs that helped to fill up classrooms. But there is a question on the lips of all those kids—in Anthony’s school, for instance—who look up from their desks at an empty blackboard. Where is the teacher?

  If you listen closely, you might hear this question being asked in Hindi. India has about a quarter-billion school-age children, many of whom suffer from teacher absenteeism on a regular basis. A series of unscheduled visits to rural schools across the country found that a quarter of teachers were missing, and that fully half of those who were in their classrooms were not teaching! That probably helps to explain some dismal facts about the state of learning in the country: A nationwide 2005 survey found that 65 percent of public school students in grades 2 through 5 couldn’t read a simple paragraph, and 50 percent couldn’t do basic arithmetic.

  Those are bad numbers and they reflect grim realities for those children who do make it to school on time. Why should the teacher be an adversary, and not an ally, in the fight for education? Of course, teachers aren’t supposed to skip school, but the fault is not entirely theirs. Some of the blame belongs to the principals and administrators who either fail to check that their classrooms are staffed, or, worse, who tolerate teacher absenteeism. Which is not to say that their jobs are easy either: Even with the right rules in place, monitoring teacher attendance in small rural schools is tedious and time-consuming.

  A Picture Is Worth a Thousand Rupees

  Seva Mandir, an Indian NGO, knows a thing or two about these problems. It runs about 150 small schools in the remote, hilly country outside Udaipur, a beautiful and ancient city in Rajasthan, a state in western India. The schools are one-room affairs in tribal villages with a single teacher each. Seva Mandir’s response to the problem of teacher absenteeism was to innovate.

  Working with Esther Duflo and Rema Hanna, an economist at Harvard University, they hit on a potential solution in a combination of monitoring and incentives. Since checking attendance directly was too cumbersome, they devised a clever way for teachers to do it themselves, using disposable cameras that cost a couple dollars each. At the beginning and end of each school day, a student was chosen to photograph the teacher and the rest of the class together. The cameras marked each photograph with a tamperproof time and date stamp. This way Seva Mandir administrators at the head office could verify weeks’ worth of teacher attendance all at once by reviewing a roll of film.

  Having cameras serve as their eyes in the field solved the monitoring problem, but they still needed to give teachers a reason not to get caught playing hooky. The program needed teeth. So Seva Mandir decided to tie teacher salaries to their attendance records. Under the old regime teachers were paid a thousand rupees (about $23) per month, provided they showed up at least twenty days, and warned that they could be dismissed for skipping. In practice, though, firings were very rare, even when it was clear they were well-deserved. The new plan was to pay a flat five hundred rupees ($11.50) per month for teaching ten or fewer days, plus an additional fifty rupees ($1.15) for each day over ten. The cameras were exactly the tool they needed to enforce the new incentive structure.

  They thought they were onto something, but they would not be content with a hunch. As an organization, Seva Mandir is as serious about evaluation as it is about innovation. Its management firmly believes that the best way to help the poor is by drawing resources toward programs that have been proven effective, and repairing or abandoning programs that haven’t. Duflo and Hanna coordinated an RCT and randomly assigned half of more than a hundred Seva Mandir schools to switch to the new system. The res
t were monitored as a control group.

  It didn’t take any subtle analysis to see what was happening. The combination of cameras and incentives caused teachers to show up more—a lot more. Absences were halved, from 42 percent in comparison schools to 21 percent in the ones using the new system. Although student attendance rates didn’t change in response to the program, the increase in teaching days alone meant that students were getting almost a third more instruction than under the old regime. Further, a series of unannounced field visits to the schools confirmed that teachers were actually teaching during these additional days, and not just showing up.

  Everything came together at test time, when students at the camera schools performed markedly better than their counterparts under the old system. Encouraged by the hard evidence from the evaluation, Seva Mandir made the program standard policy for all its schools. The gains in teacher attendance persisted, and children continue to reap the benefits today.

  When You Need More Than Attendance

  In Mumbai, the issue was not that teachers were missing school; it was that schools were missing teachers. There were just more students than could be effectively taught.

  Pratham, an Indian NGO, took a commonsense approach to the problem. If we don’t have enough teachers, they figured, let’s get more. In partnership with the government schools, they developed a program that pulled the lowest-achieving students out of class for two hours each school day to work on basic competencies with an instructor hired and trained by Pratham. The instructors were called balsakhis—Hindi for “the child’s friend.”

  Esther Duflo, with fellow MIT economist Abhijit Banerjee, Shawn Cole of the Harvard Business School, and Leigh Linden of Columbia University, set up a study to find out whether, and how, the balsakhi program affected students’ learning. They tested by monitoring test scores over two years for about 350 schools, roughly half of which had been randomly assigned to receive the program. As expected, the struggling students in balsakhi schools who were pulled out for remedial instruction did better. Even the optimists were surprised by the size of the improvement, though receiving instruction from a balsakhi produced just as big an increase in test scores as half a year of regular schooling.