by Andrew Leigh
Another relatively cheap campaign technique is the ‘robocall’. These are carried out by a computer, which automatically dials telephone numbers and then plays a message to the person or answering machine that picks up. Across randomised trials covering more than a million people, Gerber and Green suggest that robocalls perhaps get one more voter for every 1000 people who hear the message.16 But as with partisan direct mail, the effect is so small that that the true impact may be zero.
A more personal approach is for real people to make the calls. This has traditionally been done by groups of pizza-munching volunteers gathered together in an office. But some campaigns have also experimented with using commercial phone banks. Across a series of randomised trials, both volunteers and paid workers have successfully increased turnout, but the impacts vary a lot. Chatty calls seem to be more effective than delivering a short, formal message.17 Phoning closer to election day seems to have a bigger impact. And volunteers seem to outperform commercial callers. On average, Gerber and Green estimate it takes thirty-eight completed phone calls to get an additional person to the polls.18
Knocking on the door of a stranger and asking for their vote is hard work. Over the years, I’ve done plenty of doorknocking for others’ campaigns and my own. On different occasions, I’ve got frozen, sunburnt, drenched by rain and swooped by magpies. Most people who answer the door are perfectly pleasant, but I’ve accumulated enough insults to make a bushranger blush.
Doorknocking is certainly tougher to do than sending letters or making phone calls. But is it more effective? In an attempt to increase voter turnout rates among minority citizens, the California Votes Initiative ran over a hundred experiments, with tens of thousands of people being contacted.19 In two-thirds of those experiments, citizens who had been contacted were more likely to vote. Similar results have been observed in large-scale experiments involving Democratic and Republican candidates, as well as those involving nonpartisan groups in Britain, and partisan groups in France.20 On average, Gerber and Green estimate, one additional person votes for every fourteen householders who speak with a face-to-face canvasser.21 For a candidate, these results suggest that knocking on a person’s door is nearly three times as effective as calling them on the telephone, and probably 700 times more effective than sending them a letter. Given this, political scientists have lately been puzzling as to why campaigns spend only about one-twentieth of their budgets on personal campaigning.22
Finally, there’s the effect of campaigning online. A truism of campaign reporting is that every election in the twenty-first century has been described as ‘the first truly online election’. And while it is true that citizens are steadily getting more information from email, the web and social media, it doesn’t follow that campaigns run through these platforms are likely to get people to vote a particular way (or at all).
In the case of email, campaigns by non partisan groups to get people to the polls have generally been ineffective. For example, a non-profit group that aimed to increase turnout among US college students sent over 200,000 emails, with no statistically significant increase in voter turnout.23 Similarly, partisan emails sent to New York City Democrats had no impact on whether recipients voted.24 The only way an email will get you to the ballot box, randomised trials suggest, is if it comes from one of your friends, or from the electoral commission itself.25 A similar pattern comes from text messages, which seem to work only when voters have opted in to receive them, or when they are sent by the officials in charge of the election.26
Much the same is true of online advertisements. In recent years, Facebook has crowed that partisan ads on their platform could shift as many as one in five voters, Google has claimed that its ads made a significant difference in Senate races, and a bevy of journalists have written about the power of internet advertising to change the shape of an election.27 Most recently, the Republican National Committee’s Gary Coby boasted that the Trump campaign tested 40,000 to 50,000 variants of their Facebook advertisements every day of the 2016 presidential election campaign.28
Facebook facilitates easy A/B testing, but the outcomes that the platform measures are views and clicks, not votes. This matters because in experiments involving both Democratic and Republican candidates, Facebook ads raised candidates’ name recognition, but didn’t make people any more likely to vote for them.29 And in experiments involving more than 300,000 voters, Facebook ads placed by Rock the Vote – a popular nonpartisan group – had no impact on behaviour.30
As with emails, online advertisements seem to change behaviour when your friends are involved. On the day of the 2010 mid-term congressional elections, Facebook ran a massive experiment on 61 million of its US users.31 People were shown one of three messages: an ‘I voted’ button with pictures of their friends who had clicked it (social pressure message), an ‘I voted’ button with statistics on how many people had pressed it (informational message), or no message at all (control). Matching the data with voting records, the researchers found that for every 250 people who got the social pressure message, one additional person voted. By contrast, those who got the informational message were no more likely to vote than the control group. People don’t vote because millions are voting – they vote because their closest friends are voting.
In general, the results from randomised experiments on political campaigning are a reminder of how hard it is to change whether or how people vote. But a savvy randomista could still swing a close election. In recent decades, there have been dozens of state, local and national elections where the winning margin was less than one vote in 1000. They include the 2000 presidential election, which saw George W. Bush beat Al Gore by 537 votes in the critical state of Florida. In politics, even the smallest edge can mean the difference between victory and defeat.
*
Another set of randomised trials have focused on how to fundraise. Working with charities and political parties, researchers have run a series of experiments looking at which strategies work best when asking for donations. In one experiment, researchers in North Carolina sent charity workers out asking for money to support an environmental centre at the local university.32 The workers randomly varied whether they simply asked for donations, or whether they told householders that donors would also have the chance to win a prize. With a prize on offer, nearly twice as many people participated, and about 50 per cent more money was raised.
The researchers also noted another difference between household behaviour: when the charity worker was an attractive woman, men were more likely to donate. (As Aristotle once noted, ‘Beauty is a better introduction than any letter.’) While you could argue that this ranks as the least surprising finding ever to emerge from a randomised experiment, it is interesting that the size of the prize effect and the beauty effect are about the same.33
While some charity workers can brighten the day, an intriguing experiment suggests that many are unwanted. Prior to a fundraising drive for a Chicago children’s hospital, certain households were randomly sent flyers which listed the precise time that they would be doorknocked.34 Knowing who was knocking reduced the share of people opening the door by up to one-quarter. In some cases, the flyer included a ‘do not disturb’ checkbox, which cut down giving by nearly one-third. Again, the experiment provides an insight that theory couldn’t deliver. Some scholars have argued that giving is primarily motivated by the desire to enjoy a warm inner glow. If this were true, householders should have been more likely to open the door when they knew the person on the other side would be asking for money. The fact that people hide from charity collectors suggests that many donors are simply yielding to social pressure.35
Working with the Salvation Army’s ‘Red Kettle’ Christmas drive, economists tested the lengths to which we will go to avoid guilt.36 The researchers chose a Boston supermarket with two main entrances, and randomly placed a bell-ringer on one of them. On average, the Salvation Army worker collected 33 cents a minute. Next came the ask. Salvation Army workers began say
ing ‘please give’ to each shopper. This had two effects. Average donations shot up to 55 cents a minute, and the number of people shooting out the other door rose by one-third. Now, the team placed a Salvation Army bell-ringer on the alternative exit. Faced with two bell-ringers politely asking for donations, and no guilt-free way of escaping the supermarket, Bostonians gave at the rate of 99 cents a minute. By maximising guilt while minimising the opportunity to avoid it, the Salvation Army tripled donations. Red Kettle donations were one part generosity, two parts guilt-avoidance.
Other randomised trials have helped charities refine the way they fundraise. When people are told that a ‘lead donor’ has already committed money, they are more likely to donate.37 In situations as diverse as a fundraising drive for a German opera company and a capital campaign for a Florida environmental policy centre, people who receive letters that mention a lead donor gave between 50 per cent and 100 per cent more. Such seed money seems to provide a signal that the charity is worthy of support.
Another effective strategy is to offer matching donations – for example, telling people that if they donate $1 then a matching donor will give an extra $1 (or an extra $2 or $3). In a randomised mail experiment conducted with a US civil rights charity, people turned out to give one-fifth more when they were told that it would be matched.38 However, increasing the match rate – for example from $1 to $3 – had no impact on donors’ generosity. This is a puzzling result for economists, since a higher match rate effectively reduces the ‘price’ of charitable giving. For example, a $1 match means that it costs 50 cents to get $1 to a charity, while a $3 match means that it only costs 25 cents to get $1 to a charity. The randomised results are also at odds with what many fundraising experts had argued. For example, a handbook written by ‘one of America’s most successful and respected fundraisers’ assured readers that when compared to a 1:1 match, ‘a richer challenge (2:1) greatly adds to the match’s attractiveness’.39 The theory sounds sensible, but it took a randomised evaluation to find out that it was bunkum.
An unexpectedly successful strategy turned out to be the ‘once and done’ campaign, in which aid charity Smile Train promised donors ‘Make one gift now and we’ll never ask for another donation again’.40 The strategy ran contrary to standard fundraising wisdom, which suggested that charities should focus on building a relationship with their strongest supporters. It turned out that donors liked the option of opting out of future mailings, but most did not tick the box to take themselves off the mailing list. ‘Once and done’ increased total fundraising by nearly 50 per cent.
Other randomised charity experiments have confirmed popular theories. An international aid organisation found that request letters which included a modest gift (four postcards drawn by Bangladeshi street children) nearly doubled their donations.41 A university business school discovered that offering alumni an exclusive invitation to a popular economics talk more than doubled their donations.42 A New Zealand art gallery concluded that donations were 50 per cent higher on days when their gift box was seeded with a smattering of banknotes than when it started off empty.43
Then there’s the question of whether charities can raise more money by suggesting a particular amount. If you’ve ever seen a street performer hold out their hat and say, ‘Notes only, please!’ you’ll have a sense of why it might raise or lower the total take. Charity suggestion experiments include radio phone-a-thons which suggest different amount to callers, appeals that name a single figure, and letters which suggest three possible amounts. The conclusion: picking the optimal suggested amount is hard. Depending on the dollar amount chosen, studies have found instances in which suggested figures have both raised and lowered total donations.44
As charities have become more scientific about their fundraising, the appetite has grown for high-quality studies on why, how and to whom we give. In 2012 the John Templeton Foundation donated nearly us$5 million to the University of Chicago to establish the ‘Science of Philanthropy Initiative’. As director John List describes it, ‘Each time we set up these partnerships, we form an experimental relationship where we teach the partner how to think about experimentation: how to think about randomisation of mail solicitations, how to think about randomisation within a phone-a-thon, how to think about how to randomise different techniques for large donors.’45
In the political realm, one study tested whether people were more likely to donate money in order to stop something bad or to enable something good. A randomised experiment with a pro-choice lobby group found that people were more likely to give money when they perceived a threat.46 Barack Obama’s 2012 presidential campaign reached a similar conclusion. Randomised testing of email subject headers found that a fundraising appeal titled ‘Do this for Michelle’ raised about $700,000, while ‘I will be outspent’ raised $2.6 million.47 Given that politics is a zero-sum contest, it’s likely that many of the insights on political fundraising aren’t yet public. But there is some sharing of ideas among ideological bedfellows. For example, Dan Wagner, who led Obama’s 2012 data science team, went on to found Civis Analytics, which offers analysis to progressives, including Justin Trudeau’s successful 2015 campaign for the Canadian prime ministership.
*
On 2 February 2001, a public meeting was held in the West African village of Tissierou by supporters of presidential candidate Sacca Lafia.48 Villagers were informed that Lafia was the first candidate from that region since 1960. ‘If elected,’ they were told, ‘he will help promote the interests of the Borgou-Alibori region, by building new schools, hospitals and roads and more importantly, hiring more Bariba people in the public administration.’ It was a direct appeal to the self-interest of the local community.
The next day, the same campaign team visited the nearby village of Alafiarou. Again, they held a meeting on behalf of Lafia – but this time the message was different. They told the crowd: ‘If elected, he will engage in a nationwide reform of the education and health-care systems, with emphasis on building new schools and new hospitals and running vaccination campaigns. In conjunction with other opposition leaders, we will fight corruption and promote peace between all ethnic groups and all the regions of Benin.’ It was a high-minded appeal to good public policies aimed at building a better country.
Across Benin, presidential campaign teams were giving similar speeches. The leading contenders for president had agreed to work with social scientists to devise two kinds of speeches for their candidates: pork-barrelling or nation-building. For decades, political scientists had tried to understand when each kind of campaigning was effective. Now, these theories were being put to the test.
After the election, voters in the different villages were surveyed, and a curious result emerged: pork-barrelling speeches tended to boost support among male voters, while nation-building speeches appealed more to female voters. A unique experiment had dug up a finding that had barely been discussed by previous studies: Benin men tended to think local, while women were more likely to think national.
A growing number of experiments are being run with – or on – politicians. Following the Benin experiment on electoral speeches, researchers in Sierra Leone tested the impact of election debates. Working in randomly selected districts, they organised candidate debates.49 In Sierra Leone, elections had generally consisted of ‘giving out bags of rice and T-shirts’. Debates were a novelty. Sometimes candidates who had never spoken in public before did well. In other cases, candidates bombed. As debate organiser Saa Badabla put it, ‘If someone performed badly, people would say: “In Parliament they’re going to be talking about laws. How can it be that this person would go?”’50
In constituencies where debates were held, candidates worked harder. Citizens learnt more about the issues. Exposure to a debate made voters 9 percentage points more likely to support the candidate whose top priority issue aligned with theirs. After the election, researchers also saw a change in behaviour among elected members, with those in debate districts holding twice as man
y public meetings during their first year in office. Particularly in areas wracked by ethnic conflict, researchers now think that election debates may be a vital tool in encouraging a culture of disagreement over important issues without resort to violence.
Are US politicians racist? In one study, researchers sent thousands of state legislators a simple email asking about registering to vote.51 To test for racial bias, the researchers randomly varied whether the emails came from Jake Mueller, a typically white name, or DeShawn Jackson, a strongly African-American name. The experiment found that the black name received 5 per cent fewer replies overall.52 White politicians were more likely to respond to white voters, while black legislators were more likely to respond to black constituents. Another randomised study in South Africa produced a similar result.53
Over recent years, US elections have become eye-wateringly expensive. As a rule of thumb, successful candidates need to spend over US$1 million to win a House seat, US$10 million to win a Senate seat, and nearly US$1 billion to win the presidency.54 As one campaign finance expert wryly notes, such sums are not raised at bake sales.55 And yet experts have differed on the extent to which donors get better treatment from politicians. To test this, one lobbying organisation sought to line up meetings with 191 members of the US Congress.56 They were working on behalf of a group of people who had all donated money to their local congressperson. But in some of the cases, the meeting was requested for a ‘constituent’, and in others for a ‘donor’. Those who self-identified as donors were three to four times more likely to get a meeting.
Other studies have looked at whether giving politicians additional information can change how they vote. In a New Mexico experiment, some state politicians were sent information about their constituents’ attitudes.57 In a New Hampshire experiment, some politicians were sent a lobbying email.58 In both studies there was an impact on the floor of the legislature, suggesting that some representatives can be swayed by just one poll or a single lobbying activity. The ability of small interventions to shape results was backed up by a randomised experiment which sent out warning letters to US state politicians about the consequences of making misleading statements. Legislators who got the letter were less likely to receive a negative fact-checking rating.59
