The Code Book: The Science of Secrecy from Ancient Egypt to Quantum Cryptography

by Simon Singh

  In due course a wider variety of encrypted notes appeared in the newspapers. Cryptographers began to insert blocks of ciphertext merely to challenge their colleagues. On other occasions, encrypted notes were used to criticize public figures or organizations. The Times once unwittingly carried the following encrypted notice: “The Times is the Jeffreys of the press.” The newspaper was being likened to the notorious seventeenth-century Judge Jeffreys, implying that it was a ruthless, bullying publication which acted as a mouthpiece for the government.

  Another example of the public’s familiarity with cryptography was the widespread use of pinprick encryption. The ancient Greek historian Aeneas the Tactician suggested conveying a secret message by pricking tiny holes under particular letters in an apparently innocuous page of text, just as there are dots under some letters in this paragraph. Those letters would spell out a secret message, easily read by the intended receiver. However, if an intermediary stared at the page, they would probably be oblivious to the barely perceptible pinpricks, and would probably be unaware of the secret message. Two thousand years later, British letter writers used exactly the same method, not to achieve secrecy but to avoid paying excessive postage costs. Before the overhaul of the postage system in the mid-1800s, sending a letter cost about a shilling for every hundred miles, beyond the means of most people. However, newspapers could be posted free of charge, and this provided a loophole for thrifty Victorians. Instead of writing and sending letters, people began to use pinpricks to spell out a message on the front page of a newspaper. They could then send the newspaper through the post without having to pay a penny.

  The public’s growing fascination with cryptographic techniques meant that codes and ciphers soon found their way into nineteenth-century literature. In Jules Verne’s Journey to the Center of the Earth, the decipherment of a parchment filled with runic characters prompts the first step on the epic journey. The characters are part of a substitution cipher which generates a Latin script, which in turn makes sense only when the letters are reversed: “Descend the crater of the volcano of Sneffels when the shadow of Scartaris comes to caress it before the calends of July, audacious voyager, and you will reach the center of the Earth.” In 1885, Verne also used a cipher as a pivotal element in his novel Mathias Sandorff. In Britain, one of the finest writers of cryptographic fiction was Sir Arthur Conan Doyle. Not surprisingly, Sherlock Holmes was an expert in cryptography and, as he explained to Dr. Watson, was “the author of a trifling monograph upon the subject in which I analyze one hundred and sixty separate ciphers.” The most famous of Holmes’s decipherments is told in The Adventure of the Dancing Men, which involves a cipher consisting of stick-men, each pose representing a distinct letter.

  On the other side of the Atlantic, Edgar Allan Poe was also developing an interest in cryptanalysis. Writing for Philadelphia’s Alexander Weekly Messenger, he issued a challenge to readers, claiming that he could decipher any monoalphabetic substitution cipher. Hundreds of readers sent in their ciphertexts, and he successfully deciphered them all. Although this required nothing more than frequency analysis, Poe’s readers were astonished by his achievements. One adoring fan proclaimed him “the most profound and skillful cryptographer who ever lived.”

  In 1843, keen to exploit the interest he had generated, Poe wrote a short story about ciphers, which is widely acknowledged by professional cryptographers to be the finest piece of fictional literature on the subject. “The Gold Bug” tells the story of William Legrand, who discovers an unusual beetle, the gold bug, and collects it using a scrap of paper lying nearby. That evening he sketches the gold bug upon the same piece of paper, and then holds his drawing up to the light of the fire to check its accuracy. However, his sketch is obliterated by an invisible ink, which has been developed by the heat of the flames. Legrand examines the characters that have emerged and becomes convinced that he has in his hands the encrypted directions for finding Captain Kidd’s treasure. The remainder of the story is a classic demonstration of frequency analysis, resulting in the decipherment of Captain Kidd’s clues and the discovery of his buried treasure.

  Figure 19 A section of the ciphertext from The Adventure of the Dancing Men, a Sherlock Holmes adventure by Sir Arthur Conan Doyle.

  Although “The Gold Bug” is pure fiction, there is a true nineteenth-century story containing many of the same elements. The case of the Beale ciphers involves Wild West escapades, a cowboy who amassed a vast fortune, a buried treasure worth $20 million and a mysterious set of encrypted papers describing its whereabouts. Much of what we know about this story, including the encrypted papers, is contained in a pamphlet published in 1885. Although only 23 pages long, the pamphlet has baffled generations of cryptanalysts and captivated hundreds of treasure hunters.

  The story begins at the Washington Hotel in Lynchburg, Virginia, sixty-five years before the publication of the pamphlet. According to the pamphlet, the hotel and its owner, Robert Morriss, were held in high regard: “His kind disposition, strict probity, excellent management, and well ordered household, soon rendered him famous as a host, and his reputation extended even to other States. His was the house par excellence of the town, and no fashionable assemblages met at any other.” In January 1820 a stranger by the name of Thomas J. Beale rode into Lynchburg and checked into the Washington Hotel. “In person, he was about six feet in height,” recalled Morriss, “with jet black eyes and hair of the same color, worn longer than was the style at the time. His form was symmetrical, and gave evidence of unusual strength and activity; but his distinguishing feature was a dark and swarthy complexion, as if much exposure to the sun and weather had thoroughly tanned and discolored him; this, however, did not detract from his appearance, and I thought him the handsomest man I had ever seen.” Although Beale spent the rest of the winter with Morriss and was “extremely popular with every one, particularly the ladies,” he never spoke about his background, his family or the purpose of his visit. Then, at the end of March, he left as suddenly as he had arrived.

  Figure 20 The title page of The Beale Papers, the pamphlet that contains all that we know about the mystery of the Beale treasure. (photo credit 2.3)

  Two years later, in January 1822, Beale returned to the Washington Hotel, “darker and swarthier than ever.” Once again, he spent the rest of the winter in Lynchburg and disappeared in the spring, but not before he entrusted Morriss with a locked iron box, which he said contained “papers of value and importance.” Morriss placed the box in a safe, and thought nothing more about it and its contents until he received a letter from Beale, dated May 9, 1822, and sent from St. Louis. After a few pleasantries and a paragraph about an intended trip to the plains “to hunt the buffalo and encounter the savage grizzlies,” Beale’s letter revealed the significance of the box:

  It contains papers vitally affecting the fortunes of myself and many others engaged in business with me, and in the event of my death, its loss might be irreparable. You will, therefore, see the necessity of guarding it with vigilance and care to prevent so great a catastrophe. Should none of us ever return you will please preserve carefully the box for the period of ten years from the date of this letter, and if I, or no one with authority from me, during that time demands its restoration, you will open it, which can be done by removing the lock. You will find, in addition to the papers addressed to you, other papers which will be unintelligible without the aid of a key to assist you. Such a key I have left in the hand of a friend in this place, sealed and addressed to yourself, and endorsed not to be delivered until June 1832. By means of this you will understand fully all you will be required to do.

  Morriss dutifully continued to guard the box, waiting for Beale to collect it, but the swarthy man of mystery never returned to Lynchburg. He disappeared without explanation, never to be seen again. Ten years later, Morriss could have followed the letter’s instructions and opened the box, but he seems to have been reluctant to break the lock. Beale’s letter had mentioned that a note would be sent
to Morriss in June 1832, and this was supposed to explain how to decipher the contents of the box. However, the note never arrived, and perhaps Morriss felt that there was no point opening the box if he could not decipher what was inside it. Eventually, in 1845, Morriss’s curiosity got the better of him and he cracked open the lock. The box contained three sheets of enciphered characters, and a note written by Beale in plain English.

  The intriguing note revealed the truth about Beale, the box, and the ciphers. It explained that in April 1817, almost three years before his first meeting with Morriss, Beale and 29 others had embarked on a journey across America. After traveling through the rich hunting grounds of the Western plains, they arrived in Santa Fe, and spent the winter in the “little Mexican town.” In March they headed north and began tracking an “immense herd of buffaloes,” picking off as many as possible along the way. Then, according to Beale, they struck lucky:

  One day, while following them, the party encamped in a small ravine, some 250 or 300 miles north of Santa Fe, and, with their horses tethered, were preparing their evening meal, when one of the men discovered in a cleft of the rocks something that had the appearance of gold. Upon showing it to the others it was pronounced to be gold, and much excitement was the natural consequence.

  The letter went on to explain that Beale and his men, with help from the local tribe, mined the site for the next eighteen months, by which time they had accumulated a large quantity of gold, as well as some silver which was found nearby. In due course they agreed that their newfound wealth should be moved to a secure place, and decided to take it back home to Virginia, where they would hide it in a secret location. In 1820, Beale traveled to Lynchburg with the gold and silver, found a suitable location, and buried it. It was on this occasion that he first lodged at the Washington Hotel and made the acquaintance of Morriss. When Beale left at the end of the winter, he rejoined his men who had continued to work the mine during his absence.

  After another eighteen months Beale revisited Lynchburg with even more to add to his stash. This time there was an additional reason for his trip:

  Before leaving my companions on the plains it was suggested that, in case of an accident to ourselves, the treasure so concealed would be lost to their relatives, without some provision against such a contingency. I was, therefore, instructed to select some perfectly reliable person, if such could be found, who should, in the event of this proving acceptable to the party, be confided in to carry out their wishes in regard to their respective shares.

  Beale believed that Morriss was a man of integrity, which is why he trusted him with the box containing the three enciphered sheets, the so-called Beale ciphers. Each enciphered sheet contained an array of numbers (reprinted here as Figures 21, 22 and 23), and deciphering the numbers would reveal all the relevant details; the first sheet described the treasure’s location, the second outlined the contents of the treasure, and the third listed the relatives of the men who should receive a share of the treasure. When Morriss read all of this, it was some 23 years after he had last seen Thomas Beale. Working on the assumption that Beale and his men were dead, Morriss felt obliged to find the gold and share it among their relatives. However, without the promised key he was forced to decipher the ciphers from scratch, a task that troubled his mind for the next twenty years, and which ended in failure.

  In 1862, at the age of eighty-four, Morriss knew that he was coming to the end of his life, and that he had to share the secret of the Beale ciphers, otherwise any hope of carrying out Beale’s wishes would die with him. Morriss confided in a friend, but unfortunately the identity of this person remains a mystery. All we know about Morriss’s friend is that it was he who wrote the pamphlet in 1885, so hereafter I will refer to him simply as the author. The author explained the reasons for his anonymity within the pamphlet:

  I anticipate for these papers a large circulation, and, to avoid the multitude of letters with which I should be assailed from all sections of the Union, propounding all sorts of questions, and requiring answers which, if attended to, would absorb my entire time, and only change the character of my work, I have decided upon withdrawing my name from the publication, after assuring all interested that I have given all that I know of the matter, and that I cannot add one word to the statements herein contained.

  To protect his identity, the author asked James B. Ward, a respected member of the local community and the county’s road surveyor, to act as his agent and publisher.

  Everything we know about the strange tale of the Beale ciphers is published in the pamphlet, and so it is thanks to the author that we have the ciphers and Morriss’s account of the story. In addition to this, the author is also responsible for successfully deciphering the second Beale cipher. Like the first and third ciphers, the second cipher consists of a page of numbers, and the author assumed that each number represented a letter. However, the range of numbers far exceeds the number of letters in the alphabet, so the author realized that he was dealing with a cipher that uses several numbers to represent the same letter. One cipher that fulfils this criterion is the so-called book cipher, in which a book, or any other piece of text, is itself the key.

  Figure 21 The first Beale cipher.

  Figure 22 The second Beale cipher.

  Figure 23 The third Beale cipher.

  First, the cryptographer sequentially numbers every word in the keytext. Thereafter, each number acts as a substitute for the initial letter of its associated word. 1For 2example, 3if 4the 5sender 6and 7receiver 8agreed 9that 10this 11sentence 12were 13to 14be 15the 16keytext, 17then 18every 19word 20would 21be 22numerically 23labeled, 24each 25number 26providing 27the 28basis 29for 30encryption. Next, a list would be drawn up matching each number to the initial letter of its associated word:

  1 = f

  2 = e

  3 = i

  4 = t

  5 = s

  6 = a

  7 = r

  8 = a

  9 = t

  10 = t

  11 = s

  12 = w

  13 = t

  14 = b

  15 = t

  16 = k

  17 = t

  18 = e

  19 = w

  20 = w

  21 = b

  22 = n

  23 = l

  24 = e

  25 = n

  26 = p

  27 = t

  28 = b

  29 = f

  30 = e

  A message can now be encrypted by substituting letters in the plaintext for numbers according to the list. In this list, the plaintext letter f would be substituted with 1, and the plaintext letter e could be substituted with either 2, 18, 24 or 30. Because our keytext is such a short sentence, we do not have numbers that could replace rare letters such as x and z, but we do have enough substitutes to encipher the word beale, which could be 14-2-8-23-18. If the intended receiver has a copy of the keytext, then deciphering the encrypted message is trivial. However, if a third party intercepts only the ciphertext, then cryptanalysis depends on somehow identifying the keytext. The author of the pamphlet wrote, “With this idea, a test was made of every book I could procure, by numbering its letters and comparing the numbers with those of the manuscript; all to no purpose, however, until the Declaration of Independence afforded the clue to one of the papers, and revived all my hopes.”

  The Declaration of Independence turned out to be the keytext for the second Beale cipher, and by numbering the words in the Declaration it is possible to unravel it. Figure 24 shows the start of the Declaration of Independence, with every tenth word numbered to help the reader see how the decipherment works. Figure 22 shows the ciphertext-the first number is 115, and the 115th word in the Declaration is “instituted,” so the first number represents i. The second number in the ciphertext is 73, and the 73rd word in the Declaration is “hold,” so the second number represents h. Here is the whole decipherment, as printed in the pamphlet:

  I have deposited in the county
of Bedford, about four miles from Buford’s, in an excavation or vault, six feet below the surface of the ground, the following articles, belonging jointly to the parties whose names are given in number “3,” herewith:

  The first deposit consisted of one thousand and fourteen pounds of gold, and three thousand eight hundred and twelve pounds of silver, deposited November, 1819. The second was made December, 1821, and consisted of nineteen hundred and seven pounds of gold, and twelve hundred and eighty-eight pounds of silver; also jewels, obtained in St. Louis in exchange for silver to save transportation, and valued at $13,000.

  The above is securely packed in iron pots, with iron covers. The vault is roughly lined with stone, and the vessels rest on solid stone, and are covered with others. Paper number “1” describes the exact locality of the vault, so that no difficulty will be had in finding it.

  It is worth noting that there are some errors in the ciphertext. For example, the decipherment includes the words “four miles,” which relies on the 95th word of the Declaration of Independence beginning with the letter u. However, the 95th word is “inalienable.” This could be the result of Beale’s sloppy encryption, or it could be that Beale had a copy of the Declaration in which the 95th word was “unalienable,” which does appear in some versions dating from the early nineteenth century. Either way, the successful decipherment clearly indicated the value of the treasure-at least $20 million at today’s bullion prices.

  Not surprisingly, once the author knew the value of the treasure, he spent increasing amounts of time analyzing the other two cipher sheets, particularly the first Beale cipher, which describes the treasure’s location. Despite strenuous efforts he failed, and the ciphers brought him nothing but sorrow: