From this point on, I assume that the basic message of Prey is true. I assume that the growth of biological knowledge during the century now beginning will bring grave dangers to human society and to the ecology of our planet. The rest of this review is concerned with the question of what we should do to mitigate the dangers. What is the appropriate response to dangers that are hypothetical and poorly understood? In this matter, as in other situations where public health hazards and environmental risks must be assessed and regulated, there are two strongly opposed points of view. One point of view is based on the “precautionary principle.” The precautionary principle says that when there is any risk of a major disaster, no action should be permitted that increases the risk. If, as often happens, an action promises to bring substantial benefits together with some risk of a major disaster, no balancing of benefits against risks is to be allowed. Any action carrying a risk of major disaster must be prohibited, regardless of the costs of prohibition.
The opposing point of view holds that risks are unavoidable, that no possible course of action or inaction will eliminate risks, and that a prudent course of action must be based on a balancing of risks against benefits and costs. In particular, when any prohibition of dangerous science and technology is contemplated, one of the costs that must be considered is the cost to human freedom. I call the first point of view precautionary and the second point of view libertarian. In April 2000, Bill Joy, co-founder and chief scientist at Sun Microsystems, a large and successful computer company, published an article in Wired magazine with the title “Why the Future Doesn’t Need Us,” and the subtitle “Our most powerful 21st-century technologies—robotics, genetic engineering, and nanotech—are threatening to make humans an endangered species.” It was a big surprise to see one of the leaders of high-tech industry arguing passionately for a slowing down of technology that might become dangerous. Bill Joy became a spokesman for the precautionary view.
Nine months later, in January 2001, the annual meeting of the World Economic Forum was held in Davos, Switzerland. Most of the people at the forum are captains of industry, presidents of foundations, or government officials. But in 2001 they decided to invite some scientists and writers and artists to add some intellectual sparkle to the meeting. Bill and I were both invited and asked to debate the question: Is our technology out of control? Bill was taking an extreme position on the precautionary side, and I was asked to take an extreme position on the libertarian side, to make the debate interesting. In what follows I shall summarize our debate.2 To be sure that I am not misrepresenting Bill, I quote here only from his published writings.
My first quote is from Bill’s article in Wired:
The 21st-century technologies—genetics, nanotechnology, and robotics (GNR)—are so powerful that they can spawn whole new classes of accidents and abuses. Most dangerously, for the first time, these accidents and abuses are widely within the reach of individuals or small groups. They will not require large facilities or rare raw materials. Knowledge alone will enable the use of them.
Thus we have the possibility not just of weapons of mass destruction but of knowledge-enabled mass destruction (KMD), this destructiveness hugely amplified by the power of self-replication.
I think it is no exaggeration to say we are on the cusp of the further perfection of extreme evil, an evil whose possibility spreads well beyond that which weapons of mass destruction bequeathed to the nation-states, on to a surprising and terrible empowerment of extreme individuals.
This was written a year and a half before the events of September 2001. I don’t know whether Bill at that time had Osama bin Laden in mind. He certainly had in mind the possibility of a Unabomber taking his revenge on society with genetically engineered microbes rather than with chemical explosives.
Second quote. Here Bill is quoting Eric Drexler, the chief prophet of nanotechnology. Drexler set up the Foresight Institute to promote the benign uses of nanotechnology and to warn against the dangerous uses. Here is Drexler:
Tough omnivorous [synthetic] “bacteria” could out-compete real bacteria: They could spread like blowing pollen, replicate swiftly, and reduce the biosphere to dust in a matter of days. Dangerous replicators could easily be too tough, small, and rapidly spreading to stop—at least if we make no preparation. We have trouble enough controlling viruses and fruit flies.…
We cannot afford certain kinds of accidents with replicating assemblers.
The idea of nanotechnology is to build machines on a tiny scale that are as capable as living cells, but made of different materials so that they are more rugged and more versatile. One kind of nanomachine is the assembler, which is a tiny factory that can manufacture other machines, including replicas of itself. Drexler understood from the beginning that a replicating assembler would be a tool of immense power for good or for evil. Fortunately or unfortunately, nanotechnology has moved more slowly than Drexler expected. Nothing remotely resembling an assembler has yet emerged. The most useful products of nanotechnology so far are computer chips. They have no capacity for replicating either themselves or anything else.
My last quote from Bill comes from an article he published in The Washington Post, summing up the dangers that he foresees and recommending a program of action to avoid them:
We who are involved in advancing the new technologies must devote our best efforts to heading off disaster. I offer here a list of first steps suggested by our history with weapons of mass destruction:
(1) Have scientists and technologists (and corporate leaders as well) take a vow, along the lines of the Hippocratic Oath, to avoid work on potential and actual weapons of mass destruction.…
(2) Create an international body to publicly examine the dangers and ethical issues of new technology.…
(3) Use stricter notions of liability, forcing companies to take responsibility for consequences through a private-sector mechanism—insurance.…
(4) Internationalize control of knowledge and technologies that have great potential but are judged too dangerous to be made commercially available.…
(5) Relinquish pursuit of that knowledge and development of those technologies so dangerous that we judge it better that they never be available. I too believe in the pursuit of knowledge and development of technologies; yet, we already have seen cases, such as biological weapons, where relinquishment is the obvious wise choice.
Next comes my response to Bill. I agreed that the dangers he described are real, but I disagreed with some details of his argument, and I disagreed strongly with his remedies. I began by speaking about the history of biological weapons and gene-splicing experiments, and the successes and failures of efforts to regulate them. Bill ignores the long history of effective action by the international biological community to regulate and prohibit dangerous technologies. Gene-splicing experiments began in many countries when the technique of sticking pieces of DNA together was discovered in 1975. Two leading biologists, Maxine Singer and Paul Berg, issued a call for a moratorium on all such experiments until the dangers could be carefully assessed. There were obvious dangers to public health, for example if genes for deadly toxins could be inserted into bacteria that are normally endemic in human populations. Biologists all over the world quickly agreed to the moratorium, and experiments were halted everywhere for ten months. During the ten months, two international conferences were held to work out the guidelines for permissible and forbidden experiments. The guidelines established rules of physical and biological containment for permitted experiments involving various degrees of risk. The most dangerous experiments were forbidden outright. These guidelines were adopted voluntarily by the biologists and have been observed ever since, with changes made from time to time in response to new discoveries. As a result, no serious health hazards have arisen from the experiments in twenty-five years. This is a shining example of responsible citizenship, showing that it is possible for scientists to protect the public from injury while preserving the freedom of science.
The history of biological weapons is a more complicated story. The United States, Britain, and the Soviet Union all had large programs to develop and stockpile biological weapons during and after the Second World War. But these were low-key efforts compared with the programs to develop nuclear weapons. Unlike the well-known physicists who pushed the nuclear bomb programs ahead with great enthusiasm, the biologists never pushed hard for biological weapons. The great majority of biologists had nothing to do with weapons. The few biologists who were involved with the weapons program were mostly opposed to it.
The strongest of the opponents in the United States was Matthew Meselson, who had the good luck to be a neighbor and friend of Henry Kissinger in 1968 when Nixon became president. Kissinger became national security adviser to President Nixon. Meselson seized the opportunity to convince Kissinger, and Kissinger convinced Nixon, that the American biological weapons program was far more dangerous to the United States than to any possible enemy. On the one hand, it was difficult to imagine any circumstances in which the United States would wish to use these weapons, and on the other hand, it was easy to imagine circumstances in which some of the weapons could fall into the hands of terrorists.
So Nixon in 1969 boldly declared that the United States was dismantling the entire program and destroying the stockpile of weapons. This was a unilateral move, not requiring any international agreement or ratification by the American Senate. The development of weapons was duly stopped and the weapons were destroyed. Britain quickly followed suit. In 1972, as a result of Nixon’s initiative, an international convention was signed by the US, the UK, and the USSR, imposing a permanent prohibition of biological weapons on all three countries. Many other countries subsequently signed the convention.
As we now know, the Soviet Union violated the Biological Weapons Convention of 1972 on an extensive scale, continuing to develop new weapons and to accumulate stockpiles until its collapse in 1991. After the collapse, Russia declared its adherence to the convention and announced that the Soviet program had now finally been stopped. But many of the old Soviet research and production centers remain hidden behind walls of secrecy, and Russia has never provided the world with convincing evidence that the program is not continuing. It is quite possible that stockpiles of biological weapons continue to exist in Russia and in other countries. Nevertheless, the 1972 convention remains legally in force and most countries have signed it. Even if the convention is unverifiable and even if it is violated, we are far better off with it than without it. Without the convention, we would not have any legal ground for complaint or for preventive action whenever a biological weapons program anywhere in the world is discovered. With the convention, the danger of biological weapons is not eliminated but it is significantly reduced. Again, biologists in general and Meselson in particular deserve credit for making this happen in the real world of national politics and international rivalries.
The last part of my reply to Bill Joy concerns remedies for the dangers that we all agree exist. Bill says, “Internationalize control of knowledge” and “Relinquish pursuit of that knowledge … so dangerous that we judge it better that [it] never be available.” Bill is advocating censorship of scientific inquiry, either by international or national authorities. I am opposed to this kind of censorship. It is often said that the risks of modern biotechnology are historically unparalleled because the consequences of letting a new living creature loose in the world may be irreversible. I think we can find a good historical parallel where a government was trying to guard against dangers that were equally irreversible.
Three hundred and fifty-nine years ago, the poet John Milton wrote a speech with the title Areopagitica, addressed to the Parliament of England. He was arguing for the liberty of unlicensed printing. I am suggesting that there is an analogy between the seventeenth-century fear of moral contagion by soul-corrupting books and the twenty-first-century fear of physical contagion by pathogenic microbes. In both cases, the fear was neither groundless nor unreasonable. In 1644, when Milton was writing, England was engaged in a long and bloody civil war, and the Thirty Years’ War, which devastated Germany, had four years still to run. These seventeenth-century wars were religious wars, in which differences of doctrine played a great part. In that century, books not only corrupted souls but also mangled bodies. The risks of letting books go free into the world were rightly regarded by the English Parliament as potentially lethal as well as irreversible. Milton argued that the risks must nevertheless be accepted. I believe his message may still have value for our own times, if the word “books” is replaced by the word “experiments.” Here is Milton:
I deny not, but that it is of greatest concernment in the Church and Commonwealth, to have a vigilant eye how books demean themselves as well as men; and thereafter to confine, imprison, and do sharpest justice on them as malefactors.… I know they are as lively, and as vigorously productive, as those fabulous dragon’s teeth; and being sown up and down, may chance to spring up armed men.
The important word in Milton’s statement is “thereafter.” Books should not be convicted and imprisoned until after they have done some damage. What Milton declared unacceptable was prior censorship, prohibiting books from ever seeing the light of day. Next, Milton comes to the heart of the matter, the difficulty of regulating “things, uncertainly and yet equally working to good and to evil”:
Suppose we could expel sin by this means; look how much we thus expel of sin, so much we expel of virtue: for the matter of them both is the same; remove that, and ye remove them both alike.
This justifies the high providence of God, who, though he commands us temperance, justice, continence, yet pours out before us even to a profuseness all desirable things, and gives us minds that can wander beyond all limit and satiety. Why should we then affect a rigor contrary to the manner of God and of nature, by abridging or scanting those means, which books freely permitted are, both to the trial of virtue, and the exercise of truth. It would be better done to learn that the law must needs be frivolous which goes to restrain things, uncertainly and yet equally working to good and to evil.
My last quotation expresses Milton’s patriotic pride in the intellectual vitality of seventeenth-century England, a pride that twenty-first-century Americans have good reason to share:
Lords and Commons of England, consider what Nation it is whereof ye are, and whereof ye are the governors: a Nation not slow and dull, but of a quick, ingenious, and piercing spirit, acute to invent, subtle and sinewy to discourse, not beneath the reach of any point the highest that human capacity can soar to.… Nor is it for nothing that the grave and frugal Transylvanian sends out yearly from as far as the mountainous borders of Russia, and beyond the Hercynian wilderness, not their youth, but their staid men, to learn our language and our theologic arts.
Perhaps, after all, as we struggle to deal with the enduring problems of reconciling individual freedom with public safety, the wisdom of a great poet who died more than three hundred years ago may still be helpful.
That was the end of the debate. No vote was taken to determine who won. The purpose of the debate was not to win but to educate. Bill Joy and I remain friends.
1. HarperCollins, 2002.
2. This description of my debate with Bill Joy is taken from a lecture that I gave at the University of Virginia in 2004. The lecture will be published in a forthcoming book, A Many-Colored Glass: Reflections on the Place of Life in the Universe (University of Virginia Press, 2006).
5
WHAT A WORLD!
IT IS REFRESHING to read a book full of facts about our planet and the life that has transformed it, written by an author who does not allow facts to be obscured or overshadowed by politics. Vaclav Smil is well aware of the political disputes that are now raging about the effects of human activities on climate and biodiversity, but in The Earth’s Biosphere: Evolution, Dynamics, and Change1 he does not give them more attention than they deserve. He emphasizes the enormous gaps in our knowledge, the spar
seness of our observations, and the superficiality of our theories. He calls attention to the many aspects of planetary evolution that are poorly understood, and that must be better understood before we can reach an accurate diagnosis of the present condition of our planet. When we are trying to take care of a planet, just as when we are taking care of a human patient, diseases must be diagnosed before they can be cured.
The book has two themes, a major and a minor one. The major theme is the description of the biosphere. The biosphere is the interacting web of plants and rocks, fungi and soils, animals and oceans, microbes and air that constitutes the habitat of life on our planet. To understand the biosphere, it is essential to see it from both sides, from below as a multitude of details and from above as a single integrated system. This book gives a comprehensive account of biological details and a summary of the global cycles of matter and energy that tie the system together. Every detail and every cycle is documented with references to the technical literature. There are forty pages of bibliography, containing more than a thousand references, ranging from John Ray’s 1686 History of Plants to the 2001 report of the Intergovernmental Programme on Climatic Change. The bibliography will make this book a useful work of reference for students and teachers. The text is also intended to be read by ordinary citizens who are not students or teachers but have a serious interest in environmental problems.
The minor theme of the book is the life and work of Vladimir Vernadsky. Vernadsky did not invent the word “biosphere,” but he was the first to make it a central concept unifying the study of the earth with the study of life. In Russia he is honored as one of the leading figures of twentieth-century science, while in the West his name is hardly known. Vaclav Smil, who is himself a bridge between East and West, educated in Prague and living in Canada, uses this book as an opportunity to bring Vernadsky to life and to make the West aware of his ideas. Every chapter begins with a quotation from Vernadsky’s book The Biosphere, which summarized his thinking and was written for a wide audience. The first chapter, with the title “Evolution of the Idea,” begins with Vernadsky saying, “A new character is imparted to the planet by this powerful cosmic force. The radiations that pour upon the Earth cause the biosphere to take on properties unknown to lifeless planetary surfaces, and thus transform the face of the Earth.” The last chapter, with the title “Civilization and the Biosphere,” begins with the quotation “Man, alone, violates the established order.”
The Scientist as Rebel Page 6
