The Secret War

by Max Hastings

  German intelligence continued wildly to overestimate Anglo-American strength, as it had been doing for at least two years. In October 1944, FHW estimated that fourteen divisions of the British Army were still poised at home, awaiting commitment to the European campaign. In truth there were none; existing British formations were being cannibalised to maintain the shrinking strengths of Montgomery’s units. But the Germans were haunted by a belief in Churchill’s hidden reserve, which caused them to fear an amphibious landing behind their flank, perhaps in Heligoland Bight – evidence of ‘Fortitude’s’ lingering influence. Finally, of course, the Germans were crippled by battlefield weakness. During the Ardennes battle, Albert Praun’s eavesdroppers broke into the US Military Police net. They knew that the white helmets were stationed at important road junctions – ideal targets for Luftwaffe assault. Yet the airmen shrugged that not a plane could be spared for such missions. The Germans lacked means to act, even when they discovered things.

  In the last months of the struggle against Hitler, Ultra delivered an unprecedented volume of decrypts to Allied commanders – 25,000 were dispatched from Bletchley to British and American Special Liaison Units between January 1944 and May 1945, many of these during the final stages of the campaign. Copious strength and casualty reports, documents recording fuel states and tank serviceability were intercepted, but few made much impact on the battlefield when German capacity to take the initiative was spent. The challenge for Eisenhower’s armies was merely to drive small forces of weary but stubborn enemy soldiers out of fixed positions. Moreover, in the mood of hubris prevailing as victory loomed, Allied commanders lost interest in studying the enemy’s motions, and in attempting to deceive him about their own. ‘Deception during these closing months seems to have been an unsatisfactory and largely unsuccessful affair,’ wrote the British official historian, Michael Howard. ‘Few of the commanders of the huge new forces committed to battle fully understood what their deception staffs could be expected to do … Most important of all, Allied strategy itself was so opportunistic … that no serious cover plans could be made … The Allies were so strong that they effectively dispensed with strategy altogether and simply attacked all along the line, much as they had done in the closing months of 1918.’

  Bletchley suffered one last alarm call: in the course of 1944 the Luftwaffe, whose traffic had been the easiest to break since 1940, introduced a new rewirable reflector on its Enigma – UmkehrwalzeD, christened ‘Uncle Dick’ by Bletchley – which threatened to render the bombes incapable of reading its wheel settings until the changed wirings were identified and applied. The flurry of alarm this development provoked among the codebreakers caused the Americans to embark on a crash effort to produce technology to overcome it, but the war ended before it became necessary. The Luftwaffe innovation emphasised that the Germans might at any stage have made modest changes to the Enigma and its manner of use which would have rendered its signals impervious to Allied penetration. It was a miracle that they had not done so.

  The final significant intelligence issue of the war in the West was a chimera: Eisenhower’s headquarters were disturbed for weeks by the possibility that the Nazis would stage a last-ditch stand in an ‘Alpine redoubt’. It was a measure of the Allies’ profound respect for the fighting power of Hitler’s legions that, even as their survivors bled to death amid the ashes of the Third Reich, they could still inspire fear in their conquerors.

  19

  Black Widows, Few White Knights

  1 FIGHTING JAPAN

  The war in Asia and the Pacific embraced four vast theatres, wherein the only common factor was the participation of the Japanese. In the central Pacific, Nimitz’s patch, during the year following Joe Rochefort’s triumph at Midway, codebreaking played only a marginal role, because the Japanese navy’s higher ciphers mostly defied penetration in real time. While Ultra in Europe became from 1943 onwards plentiful, if never comprehensive, progress with breaking Japan’s codes was slower and more erratic, partly because some manually-encrypted enemy army traffic proved less vulnerable than Enigma. It has been suggested that Arlington Hall made a strategic mistake by focusing too many resources on reading Purple traffic, which had been broken and offered little intelligence directly relevant to the battlefield, and insufficient skilled manpower on the huge problem of Japanese military communications.

  Sigint contributed to some 1942–43 naval battles, but achieved maturity only in 1944–45, and even then never influenced a single action as dramatically as it had done at Midway. During the Solomons and New Hebrides campaigns, for instance, coast-watchers played a more important role than Hypo at Pearl Harbor: the heroic Australian Paul Mason, a plantation manager on Bougainville, provided wireless warning of incoming Japanese air attacks during the long struggle for Guadalcanal. The Allied naval disaster at Savo in August 1942 reflected continuing American difficulties with the enemy’s JN-25 cipher. Better information became available later that month for the Battle of the East Solomons, but all through the savage, costly naval actions the rest of that year and into 1943, Nimitz’s squadrons often groped for their enemies. No major US strategic decision in the Pacific during 1943 was significantly influenced by Ultra.

  Bletchley Park had a small Japanese section, a Cinderella whose staff were frequently frustrated to discover that they had spent hours or days breaking signals already read in Washington – GC&CS’s communications and intercept facilities were much inferior to those of the US Navy. BP’s branches outside Delhi, and at the Colombo headquarters of South-East Asia Command, addressed themselves almost entirely to studying Japanese traffic in British operational areas. In March 1943, Arlington Hall made the first break into a higher Japanese army system – the so-called Water Transport Code, which soon yielded fifty to a hundred decrypts a day. The British in Delhi achieved their own entry to it at about the same time, but thereafter – not without considerable hand-wringing – they relinquished the lead on Japanese material to the Americans. John Hurt, one of SIS’s veteran Japanese linguists at Arlington Hall, asserted after the war that even as late as 1944 and early 1945 codebreaking was ‘performed rather inefficiently’. It is a fundamental reality of the Pacific and Asian war that the Allies never enjoyed anything like the strength of sigint coverage they achieved in Europe.

  From 1942 onwards the US Navy’s cryptographic operations expanded on a similar scale to those of the US Army, and likewise shifted out of Washington – to Mount Vernon Academy in Virginia. An eleven-month crash Japanese language course was established at Boulder, Colorado, which by 1945 had processed a thousand students. Much codebreaking activity took place in the US rather than abroad, because Mount Vernon, like Arlington Hall, possessed batteries of machines – in addition to bombes, it had two hundred IBM tabulators by 1945, up from sixteen in December 1941 – that were unavailable in such quantity in overseas theatres. Increasing numbers of women WAVES were recruited to operate the machines, after solemn inaugural briefings in the Navy Chapel, warning them that if they spoke of their work outside the Annex, they were liable to be shot. The egregious Captain Joseph Redman continued as director of naval communications, with a few months’ break in 1942, while his brother John filled an influential role in the office of the chief of naval operations; both contributed unhelpfully to the Allied war effort.

  Nimitz ran his own intelligence operation at Pearl Harbor as an almost independent fiefdom, much expanded from Rochefort’s original Hypo, renamed FRUPAC – Fleet Radio Unit Pacific – supposedly working with Commander Rudolph Fabian’s Cast station at Melbourne, though collaboration was never Fabian’s forte. In April 1943 FRUPAC’s staff were transferred from the Dungeon to a new building, sunnier and healthier, with an air-conditioned machine room, close to the rim of the Makalapa Crater, and also to Nimitz’s headquarters. The labours of the codebreakers and translators remained as relentless as ever, entrusted exclusively to service personnel, albeit often civilians in uniform. Shortages of qualified staff bedevilled all Allied
sigint activities, and Jasper Holmes felt that the Pearl operation suffered from the C-in-C’s ban on women. Nimitz considered their presence a breach of naval custom and discipline, even ashore, yet when women belatedly joined FRUPAC in the last weeks of the war they made a significant contribution.

  The challenge posed by the successive variations of the Japanese navy’s main code, JN-25, was enormous. From August 1942 onwards, the additive books for most of the ciphers contained 100,000 entries, changed every sixty days. In all there were ten codebooks and at least seventy-seven ciphers. Until the cryptanalysts had seen a significant volume of messages at the beginning of each new period, showing the most used additives, it was impossible to make much of a start on reading traffic. It is less surprising that it took so long to achieve ongoing penetration of Japanese communications than that this happened at all. Some early fruits were seen in January 1944, when almost all of Nimitz’s subordinate commanders expected and advocated progressive assaults on the outer islands of the Marshall group. Instead, the admiral decreed that his forces would drive headlong for the key atoll of Kwajalein, because he knew from Ultra that the Japanese were weakening its garrison to strengthen the outer islands. The subsequent 30 January invasion was a brilliant success. Thereafter, Nimitz was generally aware of his enemies’ deployments, except when wireless silence was imposed on their warship sailings.

  More Japanese language specialists slowly became available – eighty-four were deployed in the field with the Okinawa invasion force in April 1945, and FRUPAC produced 127 tons of intelligence material about every known topographical feature and defensive position on the island, for distribution afloat and ashore. No quantity of such information, however, could spare the Americans from desperate fighting in the 1944–45 campaigns. The US Army and Marine Corps found themselves battering at Japanese defensive positions, the locations and subtleties of which they were ignorant, because these were neither visible to reconnaissance aircraft and photographic interpreters, nor revealed by enemy signal traffic.

  The most important achievement of Ultra in the Pacific in 1943–44 was to empower Nimitz’s submarine flotillas to launch the most devastating assault in maritime history against Japan’s overseas commerce, lifeblood of its home industries. The ‘Maru’ cipher, by which the merchant service communicated, was broken in 1943, and ever more warship traffic was read. A direct telephone link from FRUPAC to the submarine operational headquarters enabled the codebreakers instantly to forward intelligence on Japanese convoy movements – and to receive news of consequent sinkings, which helped the intelligence staff to feel in touch with the outcomes of their travails. FRUPAC made possible such a signal as that dispatched at 8 a.m. on 9 June 1943 to the submarines Trigger and Salmon patrolling the Japanese Inland Sea:

  ANOTHER HOT ULTRA COMSUBPAC SERIAL 27 LARGEST AND NEWEST NIP CARRIER WITH TWO DESTROYERS DEPARTS YOKOSUKA AT 5 HOURS GMT 10 JUNE AND CRUISES AT 22 KNOTS ON COURSE 155 DEGREES UNTIL REACHING 33.55 NORTH 140 EAST WHERE THEY REDUCE SPEED TO 18 KNOTS AND CHANGE COURSE TO 230 DEGREES X SALMON AND TRIGGER INTERCEPT IF POSSIBLE AND WATCHING FOR EACH OTHER. WE HAVE ADDITIONAL DOPE ON THIS CARRIER FOR THE BOYS NEAR TRUK WHICH WE HOPE WE WONT NEED SO LET US KNOW IF YOU GET HIM.

  The British would have considered such a signal a reckless breach of Ultra security, because it was sent to low-level personnel in an operational area, but the Americans got away with this one, and many more like it. Commander John Cromwell refused the chance of escape from his own doomed boat Sculpin, to join his crew in Japanese captivity, because as he said laconically, ‘I can’t go with you. I know too much.’ About Ultra was what he meant, of course. Trigger indeed attacked on the night of 10 June at point-blank range, badly damaging the Japanese carrier Hiyo, but torpedo failures, the blight of the US Navy in 1943, prevented its sinking. Only when this deficiency was belatedly made good did it become possible for Nimitz’s submarines to strangle enemy supply lines, as well as sink many warships. In 1942 the Japanese lost a million tons of merchant shipping to all causes. By 1945 ten times that tonnage was gone. Between January and April 1944, US submarines sent to the bottom 179 ships totalling 799,000 tons, and by the end of August a further 219 vessels. When the Japanese sought to dispatch two army divisions from Shanghai to New Guinea in April, the so-called ‘Bamboo 1’ convoy carrying the troops was almost wiped out at sea, and eventually abandoned the attempt to reinforce New Guinea. Even before the USAAF launched its intensive sea-mining campaign in the last months of the war, the Japanese merchant fleet had been largely destroyed by submarine attack.

  It was often difficult to judge how, or whether at all, to exploit sensitive information. In April 1944, FRUPAC learned that the trawler Tajina Maru, a vessel deliberately chosen by the enemy for its insignificance, was carrying the Japanese navy’s new codes to Wake Island. Two US submarines were dispatched to capture Tajina Maru – but instead blew it to pieces. Jasper Holmes and his colleagues fumed, regretting that an intelligence officer had not been put aboard one of the submarines to supervise the operation and ensure the trawler’s seizure intact. An American intelligence officer, Commander Kenneth Knowles, said after the war, ‘The British were more clever in use [of Ultra], we more daring.’ Jasper Holmes wrote: ‘Intelligence, like money, may be secure when it is unused and locked up in a safe, but it yields no dividends until it is invested.’ It was this conviction that led the Americans to override British security scruples and launch a ferocious assault on U-boat refuelling rendezvous in the summer of 1943, which inflicted dramatic losses on Dönitz’s fleet. Likewise in the Pacific, one of the more dramatic US Navy coups was prompted by a May 1944 decrypt forwarded from FRUPAC, allocating new patrol positions to all Japanese submarines off the Solomons. Armed with this information, a destroyer escort group led by the USS England sank six enemy vessels in twelve days. Kenneth Knowles was gracious enough to add to his post-war remarks about British caution in exploiting Ultra: ‘But they had more to lose.’ Which was true.

  Nimitz was a wise man, MacArthur was not. This helps to explain why Ultra exercised only marginal influence on America’s South-West Pacific campaign against Japan. The general rejected the War Department’s intelligence system and instead established his own ‘Central Bureau’, first in Melbourne, later at Hollandia and Leyte. MacArthur banned all OSS personnel from his theatre, though he backed guerrilla activity in the Philippines, provoking predictably brutal Japanese reprisals. His intelligence chief Maj. Gen. Charles Willoughby, whose pomposity caused him to be known to subordinates as ‘Sir Charles’, was no cleverer than the supremo’s other courtiers, and held sigint in some contempt. Instead of the Special Liaison Units the US Army adopted in the European theatre for handling and protecting Ultra, MacArthur instead allowed his staff to handle decrypts with a casualness that would have appalled Allied officers anywhere else in the world. Documents, personnel and even office furniture were openly addressed to ‘The Ultra Section’, and officers freely discussed codebreaking.

  Sigint influenced three important events in the South-West Pacific campaign: the 1943 Kokoda Trail battles in Papua-New Guinea; the March 1944 decision to leapfrog six hundred miles forward to Hollandia; and the July defeat of the Japanese assault down the Driniumor river, which cost the enemy 9,000 dead. In each of these actions Ultra – in 1944 assisted by 9th Australian Division’s capture of a pile of the Japanese 20th Division’s buried codebooks in New Guinea – flagged the enemy’s intentions and vulnerabilities, though MacArthur, in the same fashion as Montgomery, afterwards attributed the Allied victories to his own clairvoyance. His USAAF officers adopted a more enlightened view. Maj. Gen. George Kenney, who commanded 5th Air Force, was an exceptionally able airman who used sigint to good effect, especially for attacking Japanese reinforcement convoys, and most notably in the March 1943 Battle of the Bismarck Sea. Familiar inter-service feuding caused the US Navy to refuse to provide Ultra material to Gen. Claire Chennault, commanding the USAAF’s 14th Air Force in China. The British eventually forwarded
such material to him via SEAC headquarters in Colombo, which improved air targeting of Japanese shipping in the last phase of the war.

  The Burma campaign was of marginal relevance to the defeat of Japan, but mattered immensely to the self-esteem of the British, and especially to the struggle to regain possession of their South-East Asian empire. Britain’s generals in the theatre complained loudly and often about the weakness of battlefield intelligence, both humint and sigint. Beyond the many Anglo–American tensions and disputes, the officers of SOE and MI6 were barely on speaking terms with each other, and intelligence-gathering was poor. In the summer of 1943 Lt. Col. Gerald Wilkinson, MI6’s liaison officer on MacArthur’s staff, wrote in his diary: ‘Far East intelligence from [Britain’s secret service] has now dwindled to a trickle from a few Chinese coolies.’ Admiral Lord Louis Mountbatten, the Supreme Commander, urged MI6 to abandon its fumbling efforts to secure intelligence about the Japanese, and instead concentrate on studying the various regional nationalist movements, which would obviously play a critical role in determining post-war outcomes.

  Latterly SOE had 1,250 personnel in Asia and MI6 175, but few of them commanded much confidence. The local MI6 chief, a soldier named Lt. Col. Leo Steveni, was a typical Broadway placeman, who became a laughing stock at SEAC meetings and ran his operations out of Delhi because he knew that Mountbatten in Colombo had no time for him. Steveni was finally sacked in July 1944, but his replacement was no improvement – a 16th Lancers officer named Brigadier ‘Bogey’ Bowden-Smith, who took the job after a chance meeting in Boodle’s Club, where he mentioned that he was out of a job because he was thought too old to command troops in the field. Mountbatten’s intelligence coordinator, a Royal Navy captain, deplored the chaos that stemmed from lack of unified control: ‘Two characteristics are always present in personnel of Clandestine Services. The first is jealousy and the second is what I would describe as scoop-mindedness.’