Rebel Ideas- the Power of Diverse Thinking

by Matthew Syed

  Peaceful protestors took to the streets shouting: ‘Can’t pay, won’t pay!’ Some marches were infiltrated by militants, and rioting ensued. A march through London with up to a quarter of a million people led to the smashing of shop windows, cars set ablaze and stores looted. In total, there were 339 arrests and more than 100 injuries. For a few heady days, there was a fear that the violence might prove contagious.

  Anthony King and Ivor Crewe, two experts on British politics, have written:

  More than two decades later the whole episode still evokes wonder and astonishment . . . Every dire prediction made about the poll tax was sooner or later fulfilled. Its perpetrators walked into clearly visible traps with their eyes open, but they evidently saw nothing. They blundered on, impervious to warnings. In the end, their failure was abject and total.2

  How did it happen? According to King and Crewe, the Poll Tax debacle is part of a deeper pattern that extends across post-war British political history. They argue that for all their superficial differences, ‘a substantial proportion’ of all the biggest blunders, by governments of all political complexions, shares the same root cause: a lack of diversity. In particular, they focus on the lack of social diversity in political elites. In the case of the Poll Tax, they note that Nicholas Ridley, the secretary for the environment responsible for its implementation, was the son of Viscount Ridley, and grew up in the magnificent Blagdon Hall in Northumberland. His mother was a daughter of the architect Sir Edwin Lutyens and a niece of the painter Neville Lytton. The other environment secretaries during the life of the Poll Tax were Patrick Jenkin (Clifton and Cambridge), Kenneth Baker (St Paul’s and Oxford), Chris Patten (St Benedict’s and Oxford), all of whom went to fee-paying schools and then Oxbridge. Ridley, for his part, went to Eton and Oxford.

  As for the review group, this was led by William Waldegrave, whose father was Geoffrey Noel Waldegrave, 12th Earl Waldegrave, KG, GCVO, TD, known as Viscount Chewton, Lord Warden of the Stannaries. His mother was Mary Hermione Grenfell, the daughter of Hilda Lyttleton and Arthur Morton Grenfell, from a line of imperial businessmen. He grew up in Chewton House, one of the largest mansions in the county of Somerset.

  In his memoirs, A Different Kind of Weather, Waldegrave is admirably honest about just how detached he was from the lives of most people. ‘I never played with a local child,’ he wrote. ‘When we mentioned our neighbours, we meant the Jolliffes at Ammerdown, eight miles away, or the Asquiths at Mells, the Duckworths at Orchard Leigh, the Hobhouses at Hadspen, or the Bishop at his palace at Wells.’

  The Waldegraves holidayed at Loch Moidart with others of their class – the Lindsays, the Heathcoat-Amorys, the Orloff-Davidoffs and the Seymours – where a famous concert pianist would play the grand piano in the drawing room. Their other holiday destination was Champéry, where they took a horse-drawn sleigh to their chalet, the Chalet des Frênes. The young Waldegrave had a cook and a governess. He and his brother shot pheasants on their estate. When a black man was seen near the house, his mother’s first thought was that he was a terrorist, and she grabbed a garden strimmer to confront him. She then realised that the ‘terrorist’ had gathered up young William who had crashed his bicycle out of sight, and was trying to help him.

  According to King and Crewe, Waldegrave – who also attended Eton – may have had a background very different from the general population, but it was similar to other members of the review team. Not all were as privileged as Waldegrave himself, but they each came from unusually wealthy backgrounds. ‘No member represented any other section of British society’, they note. This was a group not just squeezed into a single quadrant of the problem space, then, but into the tiniest of pixels. They were smart, but they were also homogenous. They were not clones in a genetic sense, but in a demographic sense. And when it comes to politics, where diversity of experience is so critical to informing policy choices, this would prove catastrophic.

  And yet here is the irony: the review group loved working together. It was the most wonderful experience. King and Crewe quote insiders talking about a ‘remarkable esprit de corps’. They were agreeing, mirroring, parroting, corroborating, confirming, reflecting. They were basking in the warm glow of homophily. This social harmony deluded them into thinking they were homing in on a wise policy. In fact, it showed the opposite. They were entrenching each other’s blind spots.

  As they played bridge together, and even went to parties together, they couldn’t hear the alarm bells that would have been deafening to anyone familiar with diversity science. And is it any wonder that such a clone-like group struggled to foresee the practical problems of collection, the difficulties of implementation, and the fact that families would struggle to pay? Is it any wonder that they failed to recognise the pressure this would put on local government and, ultimately, the social fabric itself?

  For elderly people, the cost could be particularly devastating: ‘A pensioner couple in inner London could find themselves paying 22% of their net income in Poll Tax, whereas a better off couple in the suburbs pay only 1%.’ Yet when confronted with the tragic situation facing elderly couples who didn’t have the cash to pay, Nicholas Ridley struggled to grasp the problem. He replied (apparently seriously): ‘Well, they could always sell a picture.’

  A few years earlier, Patrick Jenkin, Ridley’s predecessor, made a similarly revealing comment during the energy crisis of the 1970s. In a TV interview, he encouraged the public to save electricity by ‘brushing their teeth in the dark’. It later emerged that Jenkin himself used an electric toothbrush, and his north London home was photographed with lights on in every room.

  The problem with the Poll Tax wasn’t, according to King and Crewe, with any individual politician or official. Many were devoted public servants and would go on to have distinguished careers. They were also impressive thinkers. The psephologist Sir David Butler quoted an insider as saying that they were ‘the brightest selection of people ever gathered’ to consider local government reforms. King and Crewe also point out that privilege should never be a bar to high office, and that many from wealthy backgrounds, inherited or otherwise, have contributed much to the common good.

  But this cuts to the essence of the problem: when smart people from a singular background are placed into a decision-making group, they are liable to become collectively blind. As King and Crewe put it: ‘Everyone projects onto others his or her lifestyles, preferences and attitudes. Some do it all the time; most of us do it some of the time. [Officials in] Whitehall and Westminster unthinkingly project onto others, values, attitudes and whole ways of life that are not remotely like their own.’FN1 3

  This isn’t just about Conservative politicians, of course. King and Crewe cite many examples involving Labour governments. One instance was in a speech in July 2000 by Tony Blair, which called for new powers for the police to deal with antisocial behaviour. ‘A thug might think twice about kicking in your gate, throwing traffic cones around your street or hurling abuse into the night sky if he thought he might be picked up by the police, taken to a cashpoint and asked to pay an on-the-spot fine of, for example, £100,’ he said. The response to this speech was swift, not least among human rights campaigners, worried about an extension of police powers. But what few campaigners or, indeed, journalists picked up on was a more prosaic problem, which meant the policy was flawed on its own terms. Why? Because a large proportion of thugs would not have a valid debit card, nor anything like as much as £100 in their account. As King and Crewe put it: ‘the prime minister was assuming that other people lived lives much like his own. His assumption was unfounded.’FN2

  III

  Homophily is pervasive. Our social networks are full of people with similar experiences, views and beliefs. Even when groups start out with diversity, this can be squeezed out by a process of social osmosis as people converge upon the dominant assumptions, a phenomenon known as ‘assimilation’. The author Shane Snow has shared a telling quote from a senior executive at a major bank:


  She told me, shaking her head, how painful it was to see the company hire all these great college kids – all sorts of backgrounds; all sorts of ideas brimming in their heads – only to watch them gradually remoulded to ‘fit’ the culture of the organisation. They came with unique insights and voices. She heard those voices fade, unless it was to echo the company’s ‘accepted’ way of thinking.4

  The clustering of people in small parts of the problem space, then, is a predictable consequence of human psychology. Groups have an inbuilt tendency to become clone-like. In this sense, the CIA and the Poll Tax review team are not outliers, they are symptoms. Indeed, look at many cabinets, law firms, army leadership teams, senior civil servants and even executives at some tech companies. To say that many of these groups are homogenous is not to criticise any individual, it is to note that when smart individuals have overlapping frames of reference, they become collectively myopic.

  Wise groups express a different dynamic. They are not clone-like. They do not parrot the same views. Instead, they are more like groups of rebels. They do not disagree for the sake of it, but bring insights from different regions of the problem space. Such groups contain people with perspectives that challenge, augment, diverge and cross-pollinate. This represents the hallmark of collective intelligence: how wholes become more than the sum of their parts.

  In the diagram below, the individuals are no smarter than those in the team of Figure 2 in the opening section of this chapter. And yet they possess vastly higher levels of collective intelligence. They have coverage. And they reveal why, when it comes to complex problems, it is important to work with people who think differently.

  The first step for any group seeking to tackle a tough challenge, then, is not to learn more about the problem itself. It is not to probe deeper into its various dimensions. Rather, it is to take a step back and ask: where are the gaps in our collective understanding? Are we beset by conceptual blinkers? Has homophily pulled us into one tight corner of the problem space?

  Unless this deeper question is confronted, organisations run the risk of a pervasive glitch in group deliberation: examining a problem, looking ever-deeper, while doing little more than reinforcing their blind spots. We need to address cognitive diversity before tackling our toughest challenges. It is only then that team deliberation can lead not to mirroring, but to enlightenment.

  *

  Karlskoga is a beautiful town in northern Sweden, which sits on the northern edge of Lake Möckeln. It is replete with woodland and fine buildings. I travelled there often when I lived in Sweden in my late teens, and found the place captivating.

  Anybody who has spent time in Sweden will know that one of the most important local government policies is snow-clearing. On average, Stockholm, the capital, receives around 170 days of precipitation, with much of it occurring in autumn and winter. I remember spending many mornings helping my room-mates to shovel snow from the driveway. For decades, the snow-clearing operation in Karlskoga followed what seemed like a logical approach. It started with the major traffic arteries and ended with pedestrian walkways and bicycle paths. The (mostly male) officials in the town council wanted to make the daily commute as efficient as possible. They were looking out for the interests of their electorate.

  But the council came to an unusual but perceptive realisation. They were too homogenous. Remember that in policymaking, which affects huge numbers of people, demographic differences help to inform deliberations. In her fine book Invisible Women Caroline Criado Perez highlights that when more women were brought into decision-making positions, collective intelligence started to undergo a remarkable shift.

  A fresh analysis revealed that the sexes, on average, travel differently, something that had not previously struck the officials. Men typically take the car to work while women are more likely to take public transport or walk. In France, for example, 66 per cent of public transport passengers are women, while in Philadelphia and Chicago it is 64 per cent and 62 per cent.

  Men and women also have different travel patterns. Men tend to have a twice-daily commute in and out of town in their cars. Women, who do 75 per cent of the world’s unpaid care work, tend to ‘drop children off at school before going to work; take an elderly relative to the doctor and do the grocery shopping on the way home’, Perez writes. This is sometimes called trip-chaining. This disparity is found across Europe and is particularly pronounced in households with young children.

  As new perspectives were aired, other statistics, previously overlooked, loomed into view. This is important because smart judgements hinge not just on how we interpret data, but the data we look for in the first place. Statistics from northern Sweden showed that hospital admissions for injuries are dominated by pedestrians, who are hurt three times more often than motorists in slippery or icy conditions. These exert a cost in healthcare and lost productivity. According to one estimate, this accounted for 36 million kronor in Skåne County alone over the course of a single winter. This is about twice the cost of winter road maintenance.

  With the conceptual blinkers off, Karlskoga took the decision to change decades of policy, prioritising pedestrians and public transport users on the grounds that ‘driving a car through three inches of snow is easier than pushing a buggy (or a wheelchair, or a bike) through three inches of snow’. This was not just better for women, but for the community – and the balance sheet. ‘Prioritizing pedestrians on the snow-clearing schedule makes economic sense,’ Perez writes.

  Perhaps the key thing to note is that the original schedule was not written out of malice towards women. It wasn’t consciously prioritising drivers above those pushing buggies. No, the problem was perspective blindness. As Perez puts it: ‘It came about as a result of . . . a gap in perspective. The men . . . who originally devised the schedule knew how they travelled and they designed around their needs. They didn’t deliberately set out to exclude women. They just didn’t think about them.’

  IV

  Perhaps the most vivid way to highlight the difference between teams of rebels and teams of clones is through the science of prediction. This may sound like an arcane topic, but predictions are baked into everyday life. Any time an organisation decides to do X rather than Y, they are implicitly predicting that X will be better. Prediction is central to pretty much every decision we make, whether at work, or in everyday life.FN3

  Perhaps the most brilliant study of prediction in recent times was led by Jack Soll, a psychologist at Duke University. He and his colleagues analysed 28,000 forecasts by professional economists. Their first finding was not at all surprising. Some economists are better performers than others. Indeed, the top forecaster was around 5 per cent more accurate than an average forecaster.

  But then Soll added a twist. Instead of looking at individual predictions, he took the average prediction of the top six economists. To stretch language a little, these forecasters were being placed into a team. The average of their predictions is what you might call their collective judgement. Soll then checked if this prediction was more accurate than the top-ranked economist.

  Now, in a simple task, the answer must be ‘no’. In a running race, the average time of six runners has to be slower than the time of the fastest runner. This is what Justice Scalia had in mind when he argued for a trade-off between diversity and excellence. But this analysis flips when we move from simple to complex problems. Indeed, when Soll compared the collective judgement of six economists with the judgement of the top economist, it was not less accurate, it was more accurate. And not just a little more accurate but 15 per cent more accurate. This is a staggering difference – so large, in fact, that it shocked the researchers.

  This may sound puzzling, but it underscores what we have already learned in this book. Think back to the experiment in Chapter 1 when Japanese and American people look at underwater scenes. You’ll remember that they tended to see different things. Why? Because Americans and Japanese, on average, have different frames of reference. This is why combining t
hese different perspectives creates a more comprehensive picture.

  It turns out that economic forecasters have frames of reference, too. These are sometimes called models. A model is a way of making sense of the world: a perspective, a point of view, often expressed as a set of equations. No economic model is complete, however. Each model contains blind spots. The economy is complex (unlike, say, the orbit of Jupiter, which can be precisely predicted). The rate of industrial production, for example, hinges on the decisions of thousands of businesspeople, operating tens of thousands of factories and firms, and influenced by millions of variables. No model can account for all this complexity. No economist is omniscient.

  But this implies that if we bring different models together, we create a more complete picture. No economist has the whole truth, but a group of diverse economists gets closer to the truth. Often, much closer. With prediction tasks, this is known as ‘the wisdom of crowds’. There are now dozens of examples of this aspect of diversity science. When, for example, the researcher Scott Page asked his students to estimate the length in miles of the London Underground by writing their guesses on slips of paper, the collective prediction was 249 miles. The true value is 250 miles.

  Group wisdom emerges whenever information is dispersed among different minds. Think of the students guessing the length of the London Underground: one may have visited London, another may have familiarity with the subway in New York, and so on. When people make estimates (to paraphrase Philip Tetlock), they are translating whatever information they have into a number. Each guess adds to the pool of information.

  Of course, each person is also contributing mistakes, myths and blind spots. This creates a pool of error almost as big as the pool of information. But, the information is, by definition, pointing towards the correct answer. The errors, on the other hand, emerge from different sources, and point in different directions. Some estimates are too high, others are too low, tending to cancel each other out. As Tetlock puts it: ‘With valid information piling up and errors nullifying themselves, the net result is an astonishingly accurate estimate.’ James Surowiecki, who has written a fine book on group decision-making, puts it this way: ‘Each person’s guess, you might say, has two components: information and error. Subtract the error, and you’re left with the information.’5