Although the total volume of Sigint available to the superpowers during the Cold War was much greater than during the Second World War, Soviet high-grade cipher systems – based on the theoretically unbreakable one-time pad – were far less vulnerable than those of Nazi Germany and Imperial Japan. In the immediate aftermath of the Second World War, the failure to produce a Soviet Ultra led to what a later top-secret US official inquiry described as ‘a sense of frustration and anti-climax’. Both American and British cryptanalysts, however, had some successes against Soviet traffic. Though most of these are still classified, one of them, the Venona operation, was declassified in the mid-1990s. Venona was the final codename given to approximately 3,000 Soviet intelligence and other classified telegrams intercepted during the period 1940 to 1948, which – as the result of errors in Soviet cipher production – used the same one-time pads more than once and thus became vulnerable to cryptanalytic attack. Most were decrypted, in whole or in part, by American code-breakers with some assistance from the British in the late 1940s and early 1950s.
Though the decrypts provided important information on Soviet espionage in regions of the world as far apart as Scandinavia and Australia, the most numerous and important revelations concerned intelligence operations in the United States. Venona revealed that over 200 Americans were working as Soviet agents during, and sometimes after, the Second World War, and that the leadership of the American Communist Party was hand-in-glove with the KGB. Every section of the wartime administration of Franklin D. Roosevelt, from the foreign intelligence agency, the Office of Strategic Services (OSS) to the atomic programme, had been penetrated by Soviet intelligence. Had the terminally ill Roosevelt died in 1944, the penetration would have been worse still. In 1944, Roosevelt would have been succeeded not by Truman but by his then vice-president, Henry Wallace. In preparation for this eventuality Wallace had already selected some of his future administration. The men he had chosen to be his Secretary of State and Secretary of the Treasury, Larry Duggan and Harry Dexter White, were both Soviet agents, codenamed respectively Frank and Jurist. Venona also identified agent Homer as the British diplomat Donald Maclean, one of the ‘Magnificent Five’ young Cambridge graduates recruited by Soviet intelligence in the mid-1930s. The unmasking of Maclean led eventually to the identification of the other four and to the belated discovery that the ‘Fifth Man’, John Cairncross, had worked as a Soviet agent inside Bletchley Park at the turning point of the war on the Eastern Front in 1942–3.
The legacy of Ultra influenced the handling of Venona in two ways. First, it was treated with at least the same level of extreme secrecy. Basking in the reflected glory of Ultra, Anglo–American cryptanalysts found it much easier than before the war to persuade those with access to their intelligence to keep secret both their past successes and current operations. Secondly, as with Ultra during the Second World War, collaboration on Venona during the early Cold War worked better within the transatlantic Special Relationship than within the US intelligence community. GCHQ liaison officers at the postwar US military Sigint service, the Army Security Agency (ASA), were informed of the initial Venona breakthrough even before the FBI. By 1948, according to a recent CIA/NSA study, ‘there was complete and profitable US–UK cooperation’ on the project. As well as collaborating closely with GCHQ, the chief American cryptanalyst working on the Venona project, Meredith Gardner, had regular meetings in the later 1940s with the representatives in Washington of MI6 and MI5, Peter Dwyer and Geoffrey Patterson, both of whom provided information on individuals referred to in the Soviet telegrams to assist the process of decryption. GCHQ also regularly briefed MI6 and MI5 on Venona.
But while ASA, and later the Armed Forces Security Agency (AFSA), kept the British intelligence agencies informed of progress on Venona, they concealed the entire project from the Central Intelligence Agency (CIA), founded in 1947. Ironically, therefore, as MI6 liaison officer in Washington, Peter Dwyer had to be careful not to mention the decrypts in his meetings with CIA officers. The initial decision to keep the CIA in the dark seems to have been taken by the ASA at the urging of J. Edgar Hoover, the autocratic, long-serving director of the Federal Bureau of Investigation (FBI), which was responsible for internal security. Hoover regarded the CIA as a dangerous upstart, which prevented him achieving his ambition of extending Bureau operations into the field of foreign intelligence, and also wrongly suspected it of being, like its wartime predecessor the OSS, penetrated by Soviet intelligence. General Omar Nelson Bradley, who became Chairman of the Joint Chiefs of Staff in 1949, probably at Hoover’s prompting, also opposed sharing Venona with the CIA. Bradley regarded the new Agency as insecure and as an unwelcome threat to the service intelligence agencies; he also resented the fact that, unlike the OSS, it was not placed under the authority of the Joint Chiefs. Most remarkably of all, Truman also appears not to have been informed of Venona – probably for fear that he would mention it to the Director of Central Intelligence (DCI, head of the CIA) at his weekly meetings with him. As a result the President remained confused about the nature and reality of Soviet intelligence penetration of the United States. Significantly, in over 1,200 pages of presidential memoirs, Truman never mentioned the Rosenbergs, Alger Hiss or anyone else publicly accused of being a Soviet agent during his presidency. The CIA was not briefed on Venona until Eisenhower (who had probably been briefed at the outset) was elected to succeed Truman in November 1952.
Even though the relevant British files have yet to be declassified, there is no reason to doubt that the British Prime Minister, Clement Attlee, and at least a few of his senior ministers were briefed on Venona. One remarkable consequence of the British–American Sigint alliance forged during the Second World War was that until November 1952 the British Government and intelligence community were better informed on both the achievements of American cryptanalysts and Soviet espionage in the United States than the President and the DCI. So was Joseph Stalin. In 1950, the AFSA was shocked to discover that one of its employees, William Weisband, had been a Soviet agent ever since he joined the wartime army Sigint agency in 1942. Thus it was that the Venona secret was communicated to Moscow almost six years before it reached either the President or the CIA. Although Weisband was arrested in 1950, Philby, who succeeded Dwyer as MI6 liaison officer in Washington in 1949 and was fully briefed on Venona, continued to inform Moscow until his recall in 1951.
The attempt to keep the Ultra secret continued for over a quarter of a century after the Second World War. The British Chiefs of Staff agreed on 31 July 1945 that ‘It is imperative that the fact that such intelligence was available should NEVER be disclosed.’ This demand for perpetual secrecy was based chiefly on the conviction that revelation of the past Sigint successes would alert foreign powers to the possibility that their current ciphers were being broken by GCHQ and cause them to introduce cipher systems which would be difficult, if not impossible, to crack. GCHQ adduced one further, shorter-term reason for keeping the Ultra secret. If the Germans knew that their codes had been broken and that this had hastened their defeat, they might come to believe, as after the First World War, that they had not been ‘well and fairly defeated’ and succumb once again to a variety of the ‘stab in the back’ myth which had been exploited by Adolf Hitler.
GCHQ feared from the outset, however, that it might prove impossible to preserve the Ultra secret: ‘[T]he comparing of the German and British documents is bound to arouse suspicion in [historians’] minds that we succeeded in reading the enemy ciphers.’ It now seems astonishing that for over a quarter of a century the great majority of historians suspected no such thing.
With the gift of hindsight, some of the clues now seem remarkably obvious. The fact that American cryptanalysts had broken the main Japanese diplomatic cipher in 1940 was extensively publicized during the Congressional inquiry into Pearl Harbor at the end of the war. It was also common knowledge that British cryptanalysts had broken German codes during the First World War; indeed, one well-publicize
d German decrypt – the Zimmermann telegram – had hastened American entry into the war. But, until the revelation of Ultra in 1973, almost no historian even discussed the possibility that German ciphers had been extensively broken during the Second World War as well as the First. The minority of academic historians who had served at Bletchley Park, or had been ‘indoctrinated’ into Ultra while writing official histories, were thus in the remarkable position of knowing that colleagues in their university departments who wrote about the Second World War had misunderstood an important aspect of the war, and of being forbidden by the Official Secrets Act to discuss this with them.
To a remarkable degree, the lack of interest in Sigint by historians and specialists in international relations has survived even the revelation of the Ultra secret. Although no historian of the Second World War nowadays fails to make some mention of Ultra, few stop to consider the influence of Sigint on the rest of the twentieth century. Even after the disclosure of Ultra’s role in British and American wartime operations, it took another fifteen years before any historian raised the rather obvious question of whether there was a Russian Ultra on the Eastern Front as well. The great majority of histories of the Cold War do not refer to Sigint at all. Although most studies of US Cold War foreign policy mention the CIA, there is rarely any reference to the NSA – despite the public acknowledgement by George Bush (the first) that Sigint was a ‘prime factor’ in his foreign policy. The small circle of those in the know in Washington used to joke that NSA stood for ‘No Such Agency’. The NSA, however, has a bigger budget than the CIA, employs far more people, and generates far more intelligence.
The abrupt disappearance of Sigint from the historical landscape immediately after VJ Day has produced a series of curious anomalies, even in some of the leading studies of policy-makers and postwar international relations. Thus, for example, Sir Martin Gilbert’s magisterial multi-volume official biography of Churchill acknowledges his passion for Sigint as war leader but fails to discuss his continuing interest in it as peacetime Prime Minister from 1951 to 1955. Similarly, although Eisenhower’s wartime enthusiasm for Ultra is well known, none of his biographers, so far as I am aware, mentions the enormous resources he devoted to Sigint as President of the United States from 1953 to 1961. There is even less about Sigint in biographies of Stalin. Indeed, it is difficult to think of any history of the Soviet Union which devotes as much as a sentence to the enormous volume of Sigint generated by the KGB and GRU. The US embassy in Moscow, however, appears to have been successfully bugged during the Second World War and for almost twenty years afterwards, and for a significant part of the Cold War, thanks to the achievements of Soviet cryptanalysts, France, Italy and some other NATO countries seem, without realizing it, to have been conducting open diplomacy in their dealings with the Soviet Union.
Although the neglect of Sigint by many leading historians is due partly to the overclassification of intelligence archives, it derives at root from what educational psychologists call ‘cognitive dissonance’ – the difficulty all of us have in grasping new concepts which disturb our existing view of the world. For many twentieth-century historians, political scientists and international relations specialists, secret intelligence was just such a concept. At the beginning of the twenty-first century, however, the traditional academic disregard for intelligence is in serious, if not yet terminal, decline. A new generation of scholars has begun to emerge, less disoriented than their predecessors by the role of intelligence and its use (or abuse) by policy-makers. A vast research agenda awaits them.
In the meantime, the story of Bletchley Park and what has begun to emerge about the history of cryptanalysis in the remainder of the twentieth century confronts all historians of the Cold War and international relations specialists with a major challenge, which most have avoided so far: either to seek to take account of the role of Sigint since the Second World War or to explain why they do not consider it necessary to do so.
APPENDIX I
THE VERY SIMPLE CIPHER WHICH ‘SNOW’, THE FIRST DOUBLE CROSS AGENT, WAS GIVEN BY HIS GERMAN CONTROLLERS
The code is based on the word CONGRATULATIONS, and the grid is set out as follows:
KEY blank space
1st blank = 5
2nd blank = 5 + 6 = 11
3rd blank = 11 + 7 = 18
4th blank = 18 + 8 = 26
5th blank = 26 + 9 = 35
6th blank = 35 + 10 = 45
7th blank = 45 + 11 = 56
8th blank = 56 + 12 = 68
9th blank = 68 + 13 = 81
The grid is 15 squares long (to fit the 15 letters of CONGRATULATIONS) and 12 squares deep, this last figure being chosen for convenience.
The blank spaces are decided as follows:
The first blank space is placed, quite arbitrarily, at the 5th space, reading from left to right and starting at the left-hand top corner.
The next is 6 spaces after, i.e. at the 11th space, the next is 7 spaces after that, i.e. at the 18th space, and so on in arithmetical progression until the 9th blank, placed at the 81st space, is reached. It will be found that this is in the row of the grid, i.e. halfway down.
The grid is now turned upside down and the same procedure followed as above, so that the blanks are symmetrical.
The message to be coded is now written in the grid, ignoring the blanks, that is to say, no letter must be written in a blank space. The blanks are then filled in with any letter which can be chosen quite arbitrarily.
At the top of the grid, the word CONGRATULATIONS is filled in, and underneath this each letter is numbered in order of the alphabet. Thus, A is 1, the second A is 2, C is 3, G is 4, and so on, until each column of the grid is numbered.
The procedure for sending out the message is as follows:
A line should be drawn at the end of the message when written out in the grid to separate it from the unused portion of the grid.
Each column (downwards) of letters is then written out, the order depending on the date on which the message is being sent. Thus if the date is the 8th of the month, the column headed 8 is written out, followed by the column headed 9, then 10 and so on until the string of letters is ended with the column headed 7. Should the date be more than 15, e.g. the 20th of the month, subtract 15 from the date and start on the difference, i.e. 5. This long string of letters is then counted off into blocks of 5 letters each, and in this form the message is transmitted.
Before the actual message is sent, the time, date and number of letters in the message is given. These are coded as follows:
The word CONGRATULATIONS is written out and those letters which recur, i.e. A, T, O and N are struck out, and the remaining letters numbered thus:
I and S are numbered 0 (nought) and can be used indiscriminately to represent this number.
The time of transmitting is then coded on this scale. E.g. if the time is 10.30 p.m., i.e. 2230 hours, this would be coded as OONI (or OONS); or 1245 hours would be coded as COGR.
The date on which the transmission is taking place is similarly coded, the day and month only being sent.
The total number of letters (i.e. including the ‘bogus’ letters used to fill in the blanks) is then sent using the same code.
N.B. If the message being transmitted is in answer to a message just received, and the time and date are therefore very much the same, it is possible that they may be omitted.
(Reproduced with thanks to the Public Record Office. PRO KV 2/453)
APPENDIX II
WEHRMACHT ENIGMA INDICATING SYSTEMS, EXCEPT THE KRIEGSMARINE’S KENNGRUPPENBUCH SYSTEM
Ralph Erskine
Heer and Luftwaffe indicating system from 1 May 1940 onwards (except on Yellow): singly enciphered message keys.
To encipher a message, the operator first set up his machine according to the daily key (rotor order, Stecker and ring settings). He then –
a) chose a random group of three letters for the Grundstellung (the setting of the rotors when starting to encipher or
decipher the message key) (e.g. WEP), wrote it down, and set his rotors to those positions;
b) chose three letters (e.g. RNL) as a message key, and typed them once only (RNL), producing perhaps HFI, which he again wrote down;
c) turned the rotors to RNL, and entered the plain-text of the message, writing down each letter as it lit up on the lampboard (generally a two-man operation, with one man writing as the other typed the text).
The transmitted message included –
d) the sender’s and receiver’s call signs;
e) a five-letter group consisting of two null letters (which were dropped later), followed by a three-letter discriminant, which showed which Enigma cipher was being employed. This allowed the recipient to look up and use the correct key-list (e.g. that for Red);
f) the unenciphered Grundstellung, followed by the three letters of the enciphered message key;
g) the enciphered text, in five-letter groups.
A typical Heer or Luftwaffe Enigma message is shown in Figure AII.1, with its various components marked. To decipher the text, the receiving operator set up his machine according to the daily key (rotor order, Stecker and ring settings).
He then –
h) set his rotors to the Grundstellung (WEP);
i) typed HFI – which lit up the lamps for RNL;
j) turned his rotors to RNL, and entered the cipher text, writing down the resulting letters as they lit up (again, often a two-man operation).
Heer and Luftwaffe indicating system from 15 September 1938 to 30 April 1940: doubly enciphered message keys.
The Bletchley Park Codebreakers Page 47
