The Crowd and the Cosmos: Adventures in the Zooniverse

by Lintott, Chris

  tried in these pages to convey not only my excitement about the

  results of our projects, but that of the people who get caught up

  in them. Participation in Zooniverse projects gets science done,

  but it’s more than that. A study led by Karen Masters, Galaxy

  Zoo’s project scientist (and now the elected spokesperson of the

  entire Sloan Digital Sky Survey), which asked questions of volun-

  teers while they were classifying, showed that people who par-

  ticipate in such projects learn things.

  They learn things, in fact, that they couldn’t have learned from

  the projects themselves. In other words, taking part in, say,

  Galaxy Zoo, inspires people to go out and seek out more infor-

  mation about the Universe. The projects act as an engine of

  motivation, creating a cohort of people who are actively seeking

  information they never knew they wanted. Think of Planet

  Hunters finding the details of transiting planets, or Old Weather

  participants digging into naval history, or any of the other byways

  and distractions we’ve inspired. Gamifying the experience—hid-

  ing the science behind a thin veneer of play, making it feel less

  like real science and more like any other game on your phone—

  might make projects more efficient, but it would kill this most

  important side effect of participation stone dead.

  Three PaThs 241

  You can see this in the studies researchers have done of the

  effect of even the simple gamification—the ranks on board the

  ship—that we added to the Old Weather project. Volunteers

  they interviewed said that the game worked as designed, in that

  it made them more likely to do more work, but there was

  another, more disturbing result. Instead of describing partici-

  pating as fun and interesting, they suddenly used language that

  made it seem like work. One volunteer, anonymous in the

  study, said that though they made it to captain they found it

  stressful trying to stay ahead. So our options in reality two,

  where we need to resort to either hiding the task within a game

  or to using the kind of manipulation of reward that makes people

  feel like they’re burdened with a task, are between projects

  which don’t change people’s thoughts about whether they can

  participate in science and those that feel like you’ve taken on a

  second job. In this reality, science is still reserved for the clever few who capture the attention and time of others through

  design, using the resulting effort for their own purposes.

  Participants are motivated not by curiosity, but by competi-

  tion. It may be effective, but it seems a long way from the best

  that we could manage.

  I want us to live in a third reality. This one is going to take some

  work, I think. It’s a universe in which we don’t need to rely on

  advances in machine learning to get the best out of the wealth of

  scientific data that we now have access to. It’s one where human

  intuition and pattern recognition are still needed to get the most

  from data, even when machines are good at classification them-

  selves. I feel pretty confident that this is indeed part of the reality we live in; though the recent advances in machine learning have

  been breathtaking, I think that our science is weird enough and

  our requirements exact enough that there will be a human

  element to it for a long while yet.

  242 Three PaThs

  However, as I’ve repeatedly stressed, our ability to collect

  information about the Universe continues to astound. We

  shouldn’t expect the pace at which new data flows in to decrease,

  nor should we expect it to become less open.* Because I want to

  keep communities of volunteers consciously participating in sci-

  ence, with no hiding in games, this means accepting that we

  won’t be able to rely on citizen scientists to do all the work.

  We’ve already opened the door to a solution. The supernova

  project showed that when humans and machines classify in con-

  cert, the outcome can be better than either working alone. I

  reckon that machines, as they improve while the surveys grow,

  will take on more of the burden, leaving the volunteers to review

  the unusual, the unexpected, and the interesting. The work

  comes in deciding how to divide the effort in such a way that

  allows the most interesting objects to be found; this probably

  means wading through a lot of confusing images or, for some

  projects, a lot of junk with little inherent interest. We need to

  understand how participants in Zooniverse projects want to

  work alongside the robot colleagues that my clever machine-

  learning colleagues are building for them.

  This isn’t a problem unique to us. In clicking your way around

  the material that Facebook chooses to show when you log in,

  you are in some sense collaborating with its algorithms. You are

  providing information about what the site should do next, which

  it responds to by showing you things it thinks you want to see.

  * I’m somewhat dismayed that the LSST project has now taken money from international collaborators, including those of us in the UK, to help fund its operating costs in exchange for privileged access to data. I hope the leadership will see sense and, despite the need for cash to keep the lights on and the servers humming, find a way to go back to what was once imagined as the most open of projects, with data freely shared and available to everyone. The sky the telescope will scan, image, and monitor, after all, belongs to no one, and there is certainly plenty of science to go round.

  Three PaThs 243

  More precisely, it will show you some combination of what it

  thinks you want to see, content that is most likely to expand the

  time you choose to spend on Facebook and content such as

  adverts that is profitable for the platform.

  I hope that makes your skin crawl just a little. I think we’re just

  beginning to understand how our attention is being manipulated

  on the internet, and to work out how to talk about it. I think that

  setting up a project like Galaxy Zoo, but with machine-learning

  classifiers actively working alongside human ones, is a fascinat-

  ing problem which allows us to think about what we want. Even

  simple examples pose dilemmas. One worry is that if we allow a

  neural network to take the images it is most certain about away

  from classifiers then our poor humans might lose the brightest

  and most interesting images faster than the faint blobs, which are

  harder to classify. If we assume that people are, for all that they

  tell us they’re in it for the science, partly interested in the brightest and most interesting galaxies, even if subconsciously the

  dopamine hit of suddenly seeing something spectacular keeps

  them classifying, then in allowing the machine to remove pre-

  cisely these galaxies from circulation we have built a project

  which gets progressively less appealing over time.

  Yet we have a problem. I can’t just put the bright galaxies back

  in the pot without compromising on the promise, implicit in any

  citizen science project and explicit on the Zooniverse, that any

  work done by someone will actually be used. I
experienced an

  early warning that this was going to be difficult when we shut

  down our original supernova project. As I said earlier, that deci-

  sion was triggered when the researchers involved switched to an

  automatic classifier, and told me that they would no longer use

  the results of of citizen scientists’ efforts.

  On the face of it, an easy decision. The classifications were not

  being used, so we no longer had a project worth participating in.

  244 Three PaThs

  Yet the participants, many of them dedicated people who would

  come back whenever new data was released, searching for super-

  novae time and time again, were not happy. I ended up calling a

  few of them to understand their views, and they were pretty

  unanimous in the fact they wanted their work to be used, for sure.

  No one I spoke to would participate in the project if we told them

  that we would just throw the results away. But they didn’t under-

  stand why the researchers would switch to an automated system.

  The researchers, I think, saw the automated system as just eas-

  ier to understand. Given that a modern convolutional neural net-

  work can be essentially a black box, as inscrutable at least as the

  average person, I’m not sure that this is justified, but I see why

  they might think like that. We didn’t think clearly enough about

  what this would feel like to the volunteers. One day they were

  contributing classifications that made a real difference scientific-

  ally. The next day they weren’t, even though from their perspec-

  tive nothing had changed; it’s not as if they suddenly got worse.

  What had seemed to be a collaborative project was suddenly

  looking rather one-sided.

  I think what we got wrong here was the lack of control we gave

  the volunteers, suddenly wrenching their project away from

  them, and I think that’s the key to how to cope with the com-

  plexities of this third reality, when we combine human and

  machine classifications. If we give people control over what they

  see, they can make their own decisions about how they want

  things to run. I really like the idea of a project that says, ‘We know if we give you more beautiful galaxies (or spectacular penguins,

  or interesting texts) you’re likely to stay around longer. These

  classifications won’t count, but do you want to see these images

  anyway? If so, how often?’

  That seems honest and interesting, and I hope will lead to a sys-

  tem that can cope with the majority of the data heading our way.

  Three PaThs 245

  If we don’t do something like this, I worry that we’ll miss out on the most unexpected of finds. On the contrary, I think we’re likely—if

  we get it right—to be overwhelmed with interesting things.

  Imagine a typical night, a few years from now when LSST is

  operating. As the Sun sets over the Chilean Andes, the dome

  containing the telescope opens up to allow it to cool in the cold

  night air. As the sky darkens, the enormous beast of a machine

  inside starts to methodically work its way across the sky, never

  pausing in one place for very long but often flicking back to

  where it has already been, to keep an eye out for asteroids and

  other rapidly moving or changing phenomena.

  As the telescope and its camera work away, the images it takes

  are flowing digitally away from the mountaintop observatory

  and out into the world. They will soon end up at the US National

  Supercomputing Center in Illinois, where code will compare

  each one to previous images of the same field, checking the

  brightness of millions of objects. In any given image, millions of

  times a night, some object or other will be found to have changed

  in brightness, or to have apparently appeared from nowhere—or

  vanished completely.

  LSST deals with this by issuing an alert, a public declaration of

  something happening in the sky. Its massive database, eventually

  laden with the fruits of ten years of surveying, will provide details of the history of each source. And then it’s up to the rest of us.

  Software ‘brokers’ will try to filter this unprecedented torrent of

  data, sending the cosmologists pristine type 1a supernovae and

  planetary scientists a steadily growing list of candidate asteroids

  and Kuiper Belt Objects. One of these brokers will be listening

  too, but it will have a different job, directing objects to the screens of volunteers around the world.

  Alerts will ping on a thousand mobile phones; something has

  happened in the sky, and we need your help. By the time the Sun

  246 Three PaThs

  rises in Chile, tens of thousands or maybe hundreds of thousands

  of images will have been inspected by a crowd consisting of both

  the astronomically passionate and the mildly curious. Sunrise in

  Chile means that it’s nearly night in Australia, and we’ll need to

  have identified the most interesting things by the time telescopes

  there are opening for the evening.

  Maybe the centre of a nearby galaxy has brightened because

  something is falling into its black hole. Maybe a slow-moving

  object looks like a promising contender to be the latest member

  of the swarm of bodies out around Pluto. Maybe we caught a

  planet in transit in front of a star, or just the star itself behaving badly. Whatever the case, contributions from people like you

  will help determine what happens next. As the Earth spins, tele-

  scopes in Hawai’i and the Canary Islands and in South Africa join

  in; for the most energetic events, information from space-based

  satellites will be added to the mix.

  For each of these events, triggering the worldwide network of

  observatories to stare at the right place is merely the start.

  Understanding what they are telling us will take a lot of time, and

  will overwhelm professional astronomers like myself. As data

  becomes more open, we’ll see networks of citizen astronomers

  spring up to discuss and debate their favourite objects. Some of

  the participants are undoubtedly already experts in the field;

  some will bring skills that are of great use, and others just a will-

  ingness to learn. They will talk to and collaborate with the

  increasing number of scientists who have discovered just how

  powerful working in this way really is.

  Between us, in this best of all possible worlds, we will have

  built a new way of exploring the Universe: something that takes

  the best features of Galaxy Zoo and Planet Hunters and all the

  other projects from the last decade and turns them into some-

  thing even more inclusive, more powerful, and above all else

  Three PaThs 247

  much more fun, as volunteers control not only the discovery

  but the investigation of the things that are uncovered. I hope

  that if we look back in a decade’s time at twenty years of citizen

  science through these projects, there will be a completely new

  crop of strange anomalies and curious objects to talk about.

  I would be very, very surprised to find myself in any reality other

  than this one.

  It does, however, need you. You and your ver
y human talent

  for pattern recognition and for being distracted from a task. You,

  with your curiosity and interest and willingness to spend just a

  few moments in your day doing something in contemplation of

  the Universe. You, with just a little time to join with millions of

  others so that collectively we can all achieve amazing things.

  Making the best of our capacity as a species to explore the

  Universe, and to understand the world around us, I believe,

  depends on finding a way that everyone on the planet can partici-

  pate as an active observer and interpreter of the data that’s now

  available. If we really can get everyone to join in—even if only for

  a few minutes—with this great endeavour, who knows what we

  might find, sitting out there and just waiting to be discovered.

  REFERENCES

  I haven’t tried to give a complete bibliography of works about citizen science, or even about the Zooniverse. An updated and nearly complete list of publica-tions produced by Zooniverse projects is maintained at Zooniverse.org/publications, and the ‘Meta’ category there is a good starting point for those looking for academic studies of what we’ve been up to.

  Preface

  Christiansen, Jessie L. et al., 2018, The K2–138 System: A Near-Resonant Chain of Five Sub-Neptune Planets Discovered by Citizen Scientists, Astronomical Journal, 155, 2, https://arxiv.org/abs/1801.03874.

  Lakdawalla, Emily, 2009, An ‘Amateur’ Discovers Moons in Old Voyager

  Images, planetary.org, 5 August, http://www.planetary.org/blogs/emily-

  lakdawalla/2009/2035.html.

  chaPter 2

  Bailey, Jeremy et al., 1998, Circular Polarization in Star-Formation Regions: Implications for Biomolecular Homochirality, Science, 281, 5377, 672.

  Finkbeiner, Anne, 2010, A Grand and Bold Thing, Free Press. Provides a history of the Sloan Digital Sky Survey project.

  Ivezić, Željko et al., 2018, LSST: From Science Drivers to Reference Design and Anticipated Data Products, https://arxiv.org/abs/0805.2366.

  Reid, David and Chris Lintott, 1996, Astronomy at Torquay Boys’ Grammar School, Journal of the British Astronomical Association, 5, 265, http://articles.

  adsabs.harvard.edu/full/1996JBAA..106..265R.

  chaPter 3

  Land, Kate et al., 2008, Galaxy Zoo: The Large-Scale Spin Statistics of Spiral Galaxies in the Sloan Digital Sky Survey—Clockwise and Anticlockwise

  Galaxies, Monthly Notices of the Royal Astronomical Society, 388, 1686.