Watts and Pickles both had their own indexes of compounds they had encountered and successfully solved, and I began to build one of my own – which I still possess, having illegally retained it at the end of the war. A short selection of the English equivalents will give some idea of the kind of material we had to translate. None of these terms was in any dictionary available to us: prevention of espionage; yield (of a mechanical process); tail-plane; cordon sanitaire; superimpose; (electric) earth; warming up (of an aircraft engine); (electrical) waveguide; (in optics) pupil-diameter; silver paper; angle of incidence; synthetic oil; magnetron; gun-turret; quadrilateral; reducing solution; stall (of aircraft); cavity resonator; Order of the Rising Sun; lens tube; suicide squad; combustibility; screw cap; coal slag; directional; catapult; sensitive nose fuse; temporary disbursing officer; laminar flow; cloud height; oils and fats; Trans-Siberian railway.
The suikan story is worth telling, though it was not mine. Pickles spent a long time wrestling with a report describing night-vision for fighter-pilots, but he could make nothing of a word which occurred repeatedly, spelled suikan. We were familiar enough with sensuikan (‘submarine’), but clearly this was made up of very different characters. SUI offered few possibilities, and almost always it meant ‘water’ or ‘liquid’ as in the word for submarine. But KAN was one of our worst horrors, with something like thirty characters to choose from. Only after scouring local libraries was he able to find a book on anatomy, which gave an account of the microscopic structure of the retina of the eye, where he met for the first time the phrase ‘rods and cones’, regularly used to describe it. Now one possible value for SUI was ‘cone’, and among the many possible characters for KAN was one which meant ‘rod’. Once identified it was easy to write the correct characters and thus to understand the passage.
It was a great honour to have been chosen to work, even as a humble assistant, with these men, who performed an invaluable service by providing translations of such recondite material. I never heard of them again after our organization was dissolved at the end of the war, so perhaps this tribute, posthumous as it is likely to be, is an appropriate note on which to end this memoir.
8
AN UNDERVALUED EFFORT: HOW THE BRITISH BROKE JAPAN’S CODES
MICHAEL SMITH
Introduction
The obsessive secrecy of GCHQ about the historic achievements of Britain’s codebreakers, and the contrasting willingness of the US authorities to release their own material, has distorted the history of codebreaking in the first half of the twentieth century. Nowhere has this been more evident than in the British contribution to the war in the Far East. While a few books like Alan Stripp’s Codebreaker in the Far East and the collaborative Codebreakers have shed light on the work carried out by the British on Japanese codes and ciphers, they have merely scraped the surface. It was not until the late 1990s that GCHQ finally began to release files on codebreaking operations in the Far East, revealing that the British had been just as successful in this field as the Americans. The work of remarkable men like John Tiltman and Hugh Foss has had to be reassessed in the light of recent releases. Tiltman in particular emerges as a man who truly earned his wartime title as Chief Cryptographer, making the first inroads into the super-enciphered codes adopted by the Japanese in the late 1930s, and breaking the most famous, the main naval code JN-25, within weeks of its appearance. This chapter highlights the British contribution to the codebreaking war in the Far East. It also examines the sometimes fractious relationship between the British codebreakers and the US Navy, an area in which there may be yet more revelations to come.
MS
Japan was one of the most important targets for GC&CS during the interwar years; indeed for much of that period it was second only to Bolshevik Russia. The Japanese had emerged from the First World War as the third largest naval power behind Britain and America and were determined to expand their influence in the Far East, particularly in China. But despite their arrival as a super-power the Japanese codes and ciphers were relatively unsophisticated, said Alastair Denniston, the operational head of GC&CS. The codebreakers’ Japanese expert, Ernest Hobart-Hampden, a former senior official at the British embassy in Tokyo and co-editor of the leading English-Japanese dictionary, ‘soon acquired an uncanny skill in never missing the important’, Denniston recalled. ‘Throughout the period down to 1931, no big conference was held in Washington, London or Geneva in which he did not contribute all the views of the Japanese government and of their too verbose representatives.’
The first attacks on the codes and ciphers used by the Japanese armed forces were directed against the Imperial Japanese Navy. During the early 1920s, William ‘Nobby’ Clarke, one of the former members of the Admiralty’s Room 40 codebreaking section, persuaded a number of Royal Navy officers to spend their spare time listening in to messages between Japanese ships and their naval bases. There were a number of inherent problems, one of which was that the Japanese Morse code consisted of many more signs than its English equivalent.
The advent of the telegraph had brought problems for the Japanese, whose written language was based on pictorial characters or ideographs, called kanji, and around seventy phonetic symbols called kana. The sound of words containing kanji can be represented using kana. But Japanese has a large number of different words which, while having distinctive written forms, sound the same. So a system of transliteration known as romaji developed which allowed the kana syllables to be spelled out in Roman letters. The Japanese created their own Morse code, which contained all the kana syllables plus the romaji letters and was totally different from the standard international system.
Paymaster Lieutenant-Commander Harry Shaw, a Royal Navy officer who had just completed a Japanese interpretership at the British embassy in Tokyo was sent to GC&CS to assist Hobart-Hampden. But more Japanese experts were needed and the Royal Navy decided to try to poach a Royal Australian Navy officer who had achieved amazing results on his own interpretership course. Paymaster Lieutenant Eric Nave was now serving on board HMAS Sydney, the RAN’s flagship, where he had set up his own operation to intercept Japanese messages.
The RAN agreed to lend Nave to the British and he was posted to HMS Hawkins, the flagship of the Commander-in-Chief, China Squadron, to act as a Japanese interpreter. He arrived at the British naval base in Shanghai in July 1925 and eventually received his instructions that he was to intercept and decipher Japanese radio messages. ‘The extent to which this method of obtaining intelligence can be utilized in war largely depends on a plentiful supply of naval cipher messages in peace time,’ the messages from the sea lords said. ‘Up to the present, only a small number of naval cipher messages have been received and a great many more are required. The use of a ship as a combined intercepting and deciphering centre appears to offer the best solution.’
Nave was given a trained wireless operator, Petty Officer Gordon Flintham, to intercept the messages. With the assistance of Japanese officers, who believed their language alone made the messages impenetrable, and a Japanese operator who helpfully ran through the complete Japanese Morse code in a practice message. Nave began to intercept Japanese naval messages. They soon made good headway, breaking into the tasogare, the basic naval reporting code used by the Japanese to announce the sailings of individual ships. All the information Nave managed to produce, together with any messages he was unable to break, was to be sent back by bag to the Admiralty in London, which passed it straight on to the codebreakers. Any further results were passed back to Nave and by the end of 1928 the Japanese Navy’s Main Operational Code could be read without problems.
The early 1930s saw major improvements in the systems of codes and ciphers used by the Japanese. But the increase in traffic resulting from the Japanese occupation of China, and an influx of more Japanese experts, allowed the British codebreakers to keep on top of the problem. There was also a marked increase in Japanese espionage against the British, particularly targeted at the naval base in Singapo
re and led by the Japanese naval and military attachés in the embassies around the world. The Japanese military attaché code was broken in 1933 by John Tiltman. ‘There was a small basic code chart of, I think, 240 units which meant that a large part of the plain-text had to be spelled out in syllables,’ he said. ‘I don’t remember the details of the system except that the code-chart had to be reconstructed and forty different sets of lines and column coordinates recovered.’
In order to disguise the espionage, the Japanese naval attachés began using a cipher machine, but this was broken in September 1934 by Oliver Strachey, a veteran of the Army’s First World War codebreaking operations, and Hugh Foss, who had learned Japanese while his father was working as a missionary in Japan. The improvised nature of the British operations at the time is perhaps best demonstrated by the Heath Robinson nature of the first attempts to replicate the Japanese machine. ‘The first trial was made in the office using a brown foolscap file cover with a collar stud retrieved from a returning laundry parcel, a piece of string and slots cut in the cover for the letters,’ Nave recalled. ‘This worked, so we asked the Signal School at Portsmouth to help and received some expertly finished models in Bakelite.’ The Japanese diplomatic codes and ciphers were translated by Hobart-Hampden, Harold Parlett and N. K. Roscoe – all of whom were former British consular officers in Japan, as well as J. K. Marsden, a former military attaché in Japan, and Captain Malcolm Kennedy, who had been seconded to the Japanese Army and had also been Reuters correspondent in Tokyo.
By now it was clear that the telegrams carrying the most sensitive information were being enciphered using a machine. The Japanese referred to this as the angoo-ki taipu a, the Type A cipher machine. It was very similar to the Japanese naval attaché machine broken by Foss and Strachey, although it used romaji letters rather than kana syllables. It consisted of two typewriters, one to input the plain-text, the other to type out the enciphered message, a standard telephone exchange plugboard, and the encipherment mechanism. Pressing one of the keys on the input typewriter sent an electrical impulse through the machine producing the enciphered letter on the output typewriter. Basic cryptographic analysis of the messages enciphered on the machine, almost certainly carried out by Foss and Strachey, showed that the keys changed every ten days and that it was extremely vulnerable to attack. By November 1934, they had found a way in. Recovering the messages was likely to prove time-consuming, so the Metropolitan Police signals expert Harold Kenworthy was asked to produce a machine that would allow the codebreakers easy access to the Japanese diplomatic cipher. The ‘J Machine’, as it was known, was working by August 1935.
Shortly after the breaking of the Type A machine, in November 1934, Britain decided to set up its own espionage operation based in Hong Kong. The Far East Combined Bureau (FECB) was to collect intelligence from every possible source, including Sigint, and an intercept site was set up on Stonecutters Island, four miles across the harbour from the FECB offices, where there was a small team of codebreakers led by Harry Shaw. They focused on the three main Japanese Navy codes and ciphers: the Japanese Naval General Cipher, the Flag Officer Code used by naval staff officers based in China, and the tasogare.
There were also military officers and operators attached to the FECB and plenty of military messages arising from the continued occupation of China and frontier clashes with the Russians. Many of these had to be sent back to London where they were deciphered by Tiltman. At the end of 1937, the Japanese again improved their codes and ciphers, going over to a super-enciphered code system. The message was encoded using a four-figure group codebook and then enciphered using an additive book containing 10,000 randomly selected four-figure groups from which a series of groups was selected and then added digit by digit to the encoded message. This was carried out using the Fibonacci system in which no figures are carried over – five plus seven therefore becomes two rather than twelve. The messages were also bisected, cut into two with the second half sent first to disguise the stereotyped preambles and make it more difficult for the codebreakers to read the message. This bisected super-enciphered code was the main type of system used by the Japanese Army and Navy throughout the war. But by the late summer of 1938, Tiltman had managed to break the Japanese military system.
Concern that Japan might further improve its code and cipher systems led in 1938 to the creation of a separate section to attack Japanese commercial systems in order to help keep track of supply convoys. The concerns soon proved justified. Within the space of six months, the main Japanese naval system changed twice. The second system, introduced on 1 June 1939, was similar to the military super-enciphered system broken by Tiltman except that it was based on five-figure groups and used a much larger codebook. It was known to the British as the Japanese Navy General Operational Code but was to become much better known by its American designation – JN-25. Remarkably, Tiltman made the first break into the system within the space of weeks.
Shortly before the outbreak of war with Germany, the FECB moved to Singapore for fear of Japanese attack, leaving a small team of codebreakers and intercept operators in Hong Kong. On the same day, GC&CS moved en masse to Bletchley Park. As Bletchley Park concentrated on the battle to break the German Enigma ciphers, the Japanese military and naval sections were stripped of their staff to reinforce the FECB, taking with them the JN-25 groups recovered by Tiltman. The FECB codebreaking section had around forty people working solely on JN-25 and by May 1940 they had enough of the code groups recovered to read simple messages. The largest Japanese section at GC&CS was now the diplomatic section housed in the neighbouring Elmer’s School. But the amount of useful intelligence they could produce had been limited by the introduction on all the main links between Tokyo and its embassies abroad of a new Japanese cipher machine, the angoo-ki taipu b, the Type B machine. Like the Type A machine, it was electro-mechanical and had two typewriter keyboards. However, the encipherment systems on the Type B ran through a series of telephone stepping switches comparable to the action of the rotors used on previous cipher machines such as the Type A or the German Enigma machine. With the British machine specialists putting the main thrust of their efforts into the Enigma ciphers, Bletchley Park made no apparent effort to break the Type B machine. Fortunately, it was being attacked on the other side of the Atlantic by both the US Army and US Navy, It was broken by a team of US Army codebreakers led, not as is commonly supposed by the veteran US cryptanalyst William Friedman, but by Frank Rowlett, a bespectacled former mathematics teacher.
The breakthrough was to provide one of the first bartering chips in the signals intelligence alliance between Britain and America that exists to this day. A team of four US codebreakers, two from the army and two from the navy, travelled to Britain in January 1941, almost a year before America joined the war. One of the top-secret packages they brought with them to exchange for the British expertise in breaking the German Enigma messages was a Purple machine, designed to decipher the telegrams enciphered using the Type B machine. Two days after the American delegation arrived at Bletchley Park, Admiral John Godfrey, the British Director of Naval Intelligence, authorized a full exchange of Japanese signals intelligence between the Far East Combined Bureau and a US Navy codebreaking and intercept site, on the island of Corregidor in the Philippines. The Americans provided a ‘pinch’ – a stolen version of the Japanese merchant shipping code, a naval personnel code, a new diplomatic cipher called ‘Hagi’, and a ‘nearly empty’ JN-25 codebook. The main British contribution was recoveries of the latest JN-25 super-enciphered code, which were greater in number than those available to the US station.
The Japanese had changed the JN-25 system on 1 December 1940, just two months earlier, introducing a new codebook which the Allies would designate JN-25B. But they had made a serious mistake in not changing the additives at the same time. As a result, the codebreakers had been able to break into the JN-25B immediately, recalled Neil Barham, one of the FECB JN-25 experts: ‘The Japanese introduced a new cod
ebook but unfortunately for them, retained in use the current reciphering table and indicator system. These had already been solved in some positions and new codegroups were discovered immediately. But for this mistake on the part of the Japanese the form of the book might have taken a matter of months to discover.’
Harry Shaw gave the Americans a JN-25B codebook with 500 of the groups already recovered, plus 4,000 cipher additive groups and 290 indicator additive groups, all of which were from the old system in use before the codebook changed. Although the additive system had now changed as well, these groups covered the two-month window into the new system created by the Japanese mistake, allowing the codebreakers to use them to recover more groups. These in turn allowed sustained recovery of the new additive and further reconstruction of the JN-25B code-book. But the JN-25 exchange was very much a two-way street. Although the British were in the lead at this stage, Corregidor was better placed to intercept Japanese Navy messages than Singapore, which could only pick up the Combined Fleet, based in Japanese home waters, at night. The two sites exchanged signals every three days, giving lists of the first three groups of every JN-25 message they had intercepted, and then sent on hard copies of the messages the other site had missed on the regular Pan-American Airlines flight between Manila and Singapore.
The increased co-operation allowed both stations to surge ahead with their recoveries. The Americans had developed a highly mechanized system, using punch-card tabulating machines to sort the code groups, and they soon began to catch up with their British counterparts. By April 1941, the combined effort had recovered 30 per cent of the new additive system. There was also collaboration on Japanese military systems between the British codebreakers and the US Army’s ‘Station 6’ intercept site at Fort McKinley, near Manila. Two US Army codebreakers were sent to Singapore, where Peter Marr-Johnson, the chief British Army cryptanalyst, handed them partial solutions of two Japanese Army codes, and Lt. Geoffrey Stevens, the FECB’s other military codebreaker, was posted to Washington to liaise with the US Army codebreakers.
The Bletchley Park Codebreakers Page 14
