Code Warriors
Page 9
The work had to be suspended a few days later when the British Post Office authorities called up to report that radio listeners in the area were complaining that they could not pick up the BBC and wanted to know if “any unusual electrical machinery” was being operated: apparently the distributors and related electronics of the HMFS were generating considerable local radio interference. Worried that more nosy inquiries might follow, and satisfied that the secret tests had been a complete success, GC&CS shut down the operation and sent some of the equipment to its intercept station at Knockholt in Kent, which had been collecting German teleprinter traffic during the war; at the end of July, one of the German nine-channel receivers and two of the two-channel receivers were on their way to Washington. The German prisoners stayed in England for four months of further interrogation.
Other TICOM teams meanwhile were producing some other extraordinary finds. U.S. Army divers working to recover the body of a drowned American soldier were dragging the Schliersee, a small lake in Bavaria, when they snagged a large waterproof box. A team of Army engineers was brought in for a methodical search of the deep lake, and over the next several weeks they found twenty-eight more boxes. Inside were four tons of documents, which proved to be the complete archives of the German high command’s cipher bureau. Two C-47s flew the entire haul to the RAF’s airfield at Biggin Hill, southeast of London, where trucks waited to rush the material to Bletchley Park.19
Earlier, another TICOM party, Team 3, stumbled on a German Foreign Office cryptologic unit in a castle in Naumburg, near the city of Leipzig. Warning the staff that they would be shot if any of their files were destroyed, the head of the TICOM team returned a few days later with the rest of the unit to pack up and remove the documents, three hundred thousand pages in all. The trove, filling 170 steel file cases, was spirited out under the noses of Russian forces moving in to occupy the area.
Among the Naumburg material were copies of several codebooks that—only much later—would Arlington Hall realize had been recovered from a burning pile left when the Russians hastily evacuated their consulate in northern Finland during fighting in 1941, and were subsequently passed by the Finns to their German quasi-allies. One was a codebook the NKGB called Kod Pobeda, which had been used for its one-time-pad messages from 1939 until November 1943, when the Jade codebook came into use. It was only in the mid-1950s, when the American codebreakers launched a second round of work on the Russian one-time-pad messages, that they discovered they already had in their possession the actual codebook for this earlier period, whose messages up until then had largely defied solution.20
—
One of the German prisoners rounded up by TICOM Team 1 who stood out as particularly intelligent and helpful was Unteroffizier Erich Karrenberg. He had grown up in Russia, the son of a German manufacturer, and spoke the language fluently; before the war he had been a concert pianist and lecturer in the history of art and music at Berlin University. He obviously had a mathematical mind as well, and from the start the GC&CS experts who spoke with him saw that he was thoroughly conversant in cryptanalytic terminology.
During his time in England, Karrenberg was extensively interviewed and at the request of his interrogators also wrote several detailed reports—“homework,” they called it—describing Russian radio call-sign and contact procedures, the organization of the Red Army’s radio networks, and the encipherment systems used on the Russian teleprinter traffic. Karrenberg stated that some of the traffic was sent in the clear or with very minimal concealment, such as scrambling some of the multiplexed channels, which he said he was regularly able to break if he had two thousand letters of depth to work from. There were other messages that consisted of three-, four-, or five-letter groups, evidently enciphered codes, some of which the Germans had solved but some they had concluded were enciphered with a one-time pad, and thus unbreakable.21
But there was also a considerable amount of text on the two-channel circuits that apparently was being enciphered by an online device akin to the SZ40, which used a bank of cipher wheels to generate a stream of continually changing key that was automatically added to the Baudot code of each character as it was transmitted to produce the outgoing cipher text. The Germans called this device the “Bandwurm.” The wheels advanced automatically in step with the outgoing or incoming text, so once the machines on each end were set up with their wheels in matching positions, the entire encryption and decryption process was invisible and seamless.22
But Karrenberg reported that the Russian operators were often extremely careless in the way they handled the setup of their messages; they would frequently give away the starting-position indicator of a message during their preliminary unenciphered “chat” as they established a connection, directly revealing instances when two or more messages had been sent with the same starting positions—a huge shortcut to finding depths. Another gaping insecurity was the inclusion of standard preambles and addresses within the enciphered portion of the messages, which meant that there were highly predictable, stereotyped openings that were the same from one message to another, making it fairly easy to locate depths in the traffic by looking for repeated strings of cipher text indicating that the same text had been enciphered with the exact same key. Another rich source of depths came from the fact that the Russians’ transmitters and receivers would often get out of sync in the middle of a transmission, requiring messages to be resent, resulting in large amounts of duplicated plaintext being enciphered.23
The Germans had not tried to reconstruct the machine’s actual wheel patterns and thereby directly predict the key streams it generated, as the cryptanalysts at Bletchley had done to routinely read the German Fish traffic. But even without cracking the key-generation algorithm of the Bandwurm, the German codebreakers, using little more than inspired guesswork, were often able to decode paired Russian messages that they knew from the operator’s chat or other clues were in depth. (The method exploited the fact that subtracting two such messages in depth from each other eliminated the key altogether, leaving only the difference of the two underlying plaintexts; see appendix B.) Bandwurm traffic read by the Germans included reports from Army Front headquarters to GHQ Moscow on their position and situation; orders concerning postings, transfers, and promotions of officers; signals intelligence and POW interrogation reports; and other similar information of considerable intelligence importance. Some messages of the Soviet air force and the armed units of the NKVD security forces also passed on the two-channel Bandwurm networks.24
The British referred to the Russian Baudot scrambler at first as the “Russian Fish” and then, “pending official designation” of a cover name, “Caviar,” which was hardly an improvement as far as security went, but the name stuck, at least informally, for much of the next year.25 GC&CS and Arlington Hall and Op-20-G began at once a concerted effort to recover long stretches of key, as had been done at the start of the attack on the German Fish. From that it might be possible to deduce the wheel patterns and thereby reconstruct the device’s key-generation algorithm. Then, finally, with powerful enough computational machinery, every possible sequence of key could be slid against a received message text and tested for statistical evidence of a likely match. With such a complete system, any message—not just the small percentage that happened to be in depth—could be read consistently.
But, as the Op-20-G War Diary noted the following spring, the work was going slowly: “The greatest need is more long key, which is only available from reading depths.” Both the Army and the British decided to temporarily put the project on hold; the Navy was left with only one or two officers, borrowed from other sections, to carry the work alone through the spring and summer; and then in August 1946, Howard Campaigne reported that Op-20-G had temporarily abandoned the project as well due to “lack of personnel.”26
The main problem, besides a shortage of manpower, was a lack of sufficient intercepted traffic to work from. The HMFS demultiplexers had saved six to nine months’ work that woul
d otherwise have been required to design and build equally advanced devices. But the number of devices that had been found at Rosenheim and brought to Britain and the United States was hardly enough to provide complete coverage of the Soviets’ teleprinter networks, which numbered ten to fifteen in Europe and three to four in the Far East; at best they would be able to provide “spot checks” on each of the networks. It took the better part of a year for the cryptanalysts’ pleas for the means to fully cover the teleprinter traffic to be heard. The Army and Navy finally agreed to budget $200,000 to manufacture additional demultiplexers, and a small assembly line was set up in the basement underneath the cafeteria at Arlington Hall to produce them, but it was not until the very end of 1946 that the sets began arriving at the Army’s and Navy’s intercept stations.27
Although nearly all the details remained classified by NSA even seven decades later, scattered hints in the Op-20-G War Diary and elsewhere strongly suggest that Bandwurm, alias Caviar, was the same system subsequently code-named Longfellow by the British and Americans.28 When work resumed in early 1947, the British were able to report a “rapid advance” on Longfellow that February. Some twenty “analogs” of the teleprinter encipherment attachment were built (code-named Tan, they employed relays in place of the rotating wheels and were used both for generating key and for decipherment). But it was still not possible to break all of the traffic; doing that hinged on the construction of a planned “superbombe” that would be able to statistically match each message with its encipherment key, allowing continuous reading of the Longfellow messages. (The computer was named Hiawatha, an obvious allusion to Longfellow’s famous poem.) The superbombe project faced repeated delays, however. The device was an ambitious gamble: its calculating and logic circuits were to be built around forty thousand vacuum tubes, more than twice the number contained in any of the pioneering digital computers under development in the United States and Britain at that time, and its projected cost was $1 million. Hiawatha was finally approved in late 1947—just a few months before the Soviets abruptly stopped using the Longfellow system, causing the entire project to collapse.
That ignominious end infuriated Campaigne, who observed in disgust that if the Longfellow traffic had been fully collected from the time it first appeared in 1943, the system would probably have been solved in 1945. “And if we had supported this by the analytical machinery recently planned, we could have broken out most of the available traffic. The entire story is one of ‘too little too late.’ This system was in use for five years, yet we were not ready to read it in quantity until it disappeared.”29
—
The huge reduction in staff resulting from postwar demobilization was the most obvious problem causing headaches for the men in charge of the signals intelligence units, though it was actually just one manifestation of a much larger problem they faced in transforming hothouse wartime organizations that had run on improvisation, inspiration, and adrenaline into permanent peacetime establishments that could settle down efficiently to the quiet, steady task ahead, during years or perhaps even decades of peace.
The Army and Navy signals intelligence organizations had both drawn up plans for a permanent workforce of about half their wartime levels: five thousand at what was now called the Army Security Agency (ASA), and twenty-five hundred at Op-20-G, with about half of those totals stationed at their respective Washington headquarters.30 That looked fine on paper, but said nothing about whom they were actually going to find to fill those jobs once the shooting stopped. Inevitably, the wartime excitement that had drawn in an astonishing number of unusually talented men and women and imbued the work with such a sense of urgency, importance, and necessity was gone. At GC&CS and Arlington Hall in particular, formal lines of authority had never counted for much during the war; getting the job done was what mattered, and in large part because no one planned to make a career of the work, no one was very career-minded about office politics or promotion or pay or protecting their bureaucratic turf. Cecil Phillips remembered wartime Arlington Hall as a true “meritocracy” where a sergeant, who in a considerable number of cases might have a degree from MIT or Harvard or some other top school, and a lieutenant might work side by side as equals on the same problem and no one thought much about it.
At Bletchley Park the culture of informality and individual initiative was even more manifestly a part of the wartime ethos. “No one ever really regarded anyone as their boss,” recalled Stuart Milner-Barry, a top-rated member of the British chess team who became head of Hut 6, the army and air force Enigma section. Milner-Barry said he had never once had to actually order anyone to do their work.31 But by the same token few of the top-level mathematicians and scholars who had put aside their careers to help win the war, and whose intellectual firepower and unmilitary individualism and collegiality had contributed to Bletchley’s unique atmosphere, had much interest in being part of a government or military bureaucracy in peacetime. Max Newman and Jack Good, both first-rate mathematicians who had conceived the attack on the German Tunny teleprinter codes and been instrumental in the design of Colossus, left, as did the formidable Alan Turing; all three joined a project at the University of Manchester to develop one of the world’s first stored-program, general-purpose electronic computers, the Mark 1. Theirs was just part of a mass exodus that sapped the postwar vitality of GC&CS. William P. Bundy, who had been operations officer of the 6813th Signal Security Detachment, the first American contingent sent to work at Hut 6, returned to England for a vacation in the summer of 1947 and, as he reported in a newsletter that the group’s veterans published for a few years, “spent three weeks in England…saw almost all of the old Hut 6 people….The trip left me very depressed…the wartime spark had gone, and no new urge had taken its place.”32
At ASA, peace brought a flood of pettifogging orders, policy directives, and procedural instructions, accompanied by a succession of martinet junior officers who rotated in and out and often knew nothing about cryptanalysis but were sticklers for organization, military protocol, and the chain of command. Lengthy interoffice memoranda circulated dissecting the merits of developing a personnel handbook, or analyzing whether a proposed change in policy that would allow civilian employees of Arlington Hall to be admitted to the post movie theater was consistent with Paragraph 10, AR 210-389 of the Army Regulations. “Low pay and too many military bosses” would be a recurring complaint from ASA’s civilian workforce over the next few years, along with a sense that no matter how much experience they had or how qualified they were, the top positions in each division would always go to a less qualified Army officer.33
The Navy’s troubles were exacerbated by personnel policies that prevented most of Op-20-G’s members of the Naval Reserve—which included virtually all of the civilian experts it had recruited during the war—from being considered for permanent peacetime positions. Wenger noted that Op-20-G had an unusually high proportion of older men (19 percent of them had completed their PhD studies at the time they became reserve officers), which made them ineligible to apply for a transfer to the regular Navy because of age limits intended to ensure that new officers could devote a full career to the Navy. Wenger proposed creating five hundred new civil service positions to retain Op-20-G’s most experienced men, and Louis Tordella and Howard Campaigne were two of the unit’s wartime officers who had the experience of going out the door of Nebraska Avenue one day, returning the next day in civilian clothes, sitting down at exactly the same desk, and resuming exactly the same duties. But the bureaucratic machinery was slow to turn, and only three hundred civil service slots had been approved by the summer of 1946.34
At Arlington Hall the civilian workforce was down to twenty-two hundred by the summer of 1946, and as at GC&CS, many of the brilliant linguistic, mathematical, and legal minds who had contributed to the triumphs of the war were gone, Richard Hallock, Ferdinand Coudert, Genevieve Feinstein, and Alfred McCormack among them. William Friedman and his original three assistants, Frank Rowlett, Abe
Sinkov, and Solly Kullback, had all decided to stay on, however, and they would prove a stabilizing force in maintaining the technical professionalism of the agency through its difficult times to come. They had been able to persuade a number of their best protégés to stay on as well—among them Cecil Phillips; Leo Rosen, an MIT electrical engineering major who had contributed a crucial idea to the reconstruction of the Japanese Purple machine; and Samuel Snyder, who had devised the first method for using IBM card sorters and tabulators to search for repeated groups in enciphered codes in 1937.35
But Rowlett, in a long paper he wrote toward the end of the war, pointed out that holding on to the currently trained group of cryptanalysts was only the most immediate challenge; if ASA was going to be set on a permanent footing it would have to face up to the reality that recruiting was a never-ending requirement. Finding people who wanted to make a career of cryptanalysis was an entirely different proposition from what had gone on during the war. One thing the war had shown was that screening tests were of little use in identifying those who had an aptitude for the work, and that only lengthy on-the-job training and experience would tell. There was after all no such thing as a pool of professional cryptologic experts in civilian life that the services could draw on: “No cryptanalytic proving ground exists outside of the field itself,” Rowlett wrote.