The Bletchley Park Codebreakers

by Michael Smith

  More (and improved) Robinsons were ordered and began to arrive in late 1943. More significantly Newman consulted with Flowers about something much more powerful. Flowers had the idea of a machine in which the wheel patterns and how they affected each other could be set up on valves. The cipher text (as with Robinson) was on perforated tape that rotated on a bedstead (but much faster). The scores and wheel-positions were typed out. In their design they went for great flexibility in the combinations that could be counted, which proved to be important. This machine, known as Colossus, was installed in February 1944. After a successful run on Robinson a new tape had to be mounted, and another often had to be made, before the next run. On Colossus you only had to change the plugging; it was liberating.

  The Testery and the Newmanry

  The Newmanry and the Testery had at first distinct jobs: the Testery read depths and recovered wheel-patterns; the Newmanry set transmissions on these patterns; the Testery handled the deciphering. We realized, however, in November 1943 that the Testery could set the psi- and μ-patterns on any transmission from which the chi-pattems had been correctly subtracted. We gave them the de-chi, D = P + ψ'. The Testery analysts could use the skill that enabled them to read depths to split D into its component parts, P and ψ'. It was therefore decided that the Newmanry should content itself with setting the chi-wheels and delivering the de-chis to the Testery. We would also give a character-count of ΔD, and the duty officer responsible for accepting the settings as correct would add his initials. The Testery could (and did) assign priorities, based on the look of the count and the initials that went with it. The decision meant, of course, that more transmissions could be deciphered; Robinson had formed the bottleneck, not the Testery.

  About that time, the Newmanry moved from its original intimate hut into lavish accommodation in Block F near to the Testery. It housed the first Colossus, two improved Robinsons, and other improved Colossi as they arrived. The two sections formed a joint Registry, and a non-Morse section of Sixta (which was responsible for traffic analysis) was created. Its job included liaison with our suppliers, Knockholt, and with our customers, the intelligence sections. Knockholt intercepted more transmissions than Bletchley Park could decipher; Sixta, knowing both the intelligence and the cryptanalytic priorities, could get Knockholt to send us the transmission that could be used for wheel-breaking and the decipherable transmission that the customers most wanted to read.

  New Wheel-breaking Methods

  By the start of 1944 the days of great achievements and inventions had passed. There was a lot we could and did do to improve our methods of exploitation; but for the recovery of wheel-patterns we had so far depended on depths. At about this time, however, some links began re-using autoclave limitations, and for such links depths could not be read. We needed to break the patterns from single transmissions. A method, known as rectangling, derived from one of Tutte’s hand experiments, came to our aid.

  It used the Δχ1 + Δχ2 feature that we had been using to set transmissions. If ΔZ1 + ΔZ2 were written out on a width of 41×31, the excess of dots over crosses (positive or negative) in each column gave evidence about whether Δχ1 + Δχ2 at those positions were equal or not. These excesses were written into a rectangle of 31 rows and 41 columns diagonally downwards, reappearing in the obvious way when they came to the bottom or the right-hand end; each row, therefore, referred to a single χ2 position and each column to a χ1 position. Any assumed pattern for Δχ2, consisting of 1s for dots, –1s for crosses and 0s for ‘don’t know’ could be applied to each column to assemble a total excess for that bit of Δχ1. From that a fragmentary putative pattern could be deduced for Δχ1. This could, in the same way, give an excess for each bit of Δχ2 and so on. This process, known as crude convergence, continued until believable patterns for Δχ1, and Δχ2 were reached. A bad start was likely to lead to a bad end, and devices were found for getting a good start with a few dots and crosses and a lot of ‘don’t knows’.

  Jack Good provided significance tests. One, on the total excesses arising from the final patterns, determined whether it would be worth proceeding to the next stage. Another (too lengthy until a way was found of calculating it on Colossus) picked out the transmissions whose rectangles were likely to work.

  Colossus could be used to provide the original 41×31 excesses that were written into the rectangle. Wrens did convergences and became very good at it. Once a convincing pair of patterns was produced, Colossus could be used to get partial patterns for the other chi-wheels, and eventually to give complete patterns. At first this involved Wrens or (much worse) cryptanalysts going behind the machine and fiddling with the gadgets that set up the wheel-patterns. The engineers hated anything like that and soon provided us with panels at the front of some of the Colossi on which we could do the job without risk to the machines.

  The method worked well enough and we sent reliable de-chis to the Testery; but we wondered at first whether they could expand their method of ψ-setting to the much harder job of ψ-breaking. We need not have wondered. In February 1944, they had their first such success. The method was to fit a longish crib, say thirty characters long, that gave a plausible stretch of ψ'; to contract this to a stretch of ψ, perhaps twenty long; and to cycle the twenty-long stretches of ψ1, ψ2, … through one cycle each. The shortest cycle-length is forty-three and the longest fifty-nine, so that there would be four consecutive complete ψ characters.

  The analyst could guess whereabouts they would appear, probably extended, and place them by recognizing a fragment of plain-text. From then on it was a question of extending the plaintext and building up the complete psi-patterns.

  This first success was rightly hailed as a triumph, but it was repeated confidently on almost all occasions on which we sent a correct de-chi (and even sometimes when it wasn’t entirely correct). In one very favourable case, the ψs were recovered in 35 minutes.

  So by early 1944, we could break the patterns and decipher transmissions on any link/month that had one long enough transmission or a readable depth. But soon we had an addition to our armoury. Sixta discovered that some messages on perforated tape were sent on more than one link. For this to be fruitful three things had to go right: a pair of such transmissions had to be spotted; one of them had to be deciphered; and the plain-text had to be matched against the correct stretch in the other and the resulting key broken. Sixta had useful clues to spotting a pair. Each message in a transmission carried a serial number, which would, of course, be enciphered; when a transmission had been deciphered by its recipient, he often sent, as a receipt, the last two digits of the serial numbers of each message in the transmission. When Sixta found among the receipts of different links the same pair of digits at similar times there was a chance that they included the same message. Since links generally passed more than a hundred messages each day, there were plenty of coincidences that had not arisen from retransmission. Sixta became familiar with pairs of links liable to be sending the same message, and they could often make use of priority signals and other indications.

  They would submit to the Testery crib section daily predictions of retransmissions. This crib section read all deciphered messages and could often pick out a message that was likely to have been retransmitted on another link. They would pass such ideas to Sixta and could include the enciphered serial number, from which Sixta could often pick out a likely pair. Most of the successful pairings were spotted in this way.

  The plain-text to be used was provided by the Testery and sent with the supposed match to the Newmanry crib section. Robinson was the ideal machine to subtract the plain-text at all likely offsets from stretches of the cipher text, and count for some suitable property of key. Many such properties were proposed and used. When a convincing match was found, the resulting stretch of key was broken by standard methods.

  All this called for close liaison with Knockholt, painstaking checking both of plain and cipher text, and ingenuity and mathematics to devise the most efficie
nt tests. The Newmanry crib section worked on 250 cases out of nearly 900 suggested by Sixta. Of these 72 were successful.

  Exploitation

  Throughout the whole Tunny operation, research and experience led to continual improvement. Experience was responsible for one improvement. The Testery deciphered the transmissions that had been set, and they often found it went wrong before the end. They soon discovered that the cipher tape sent by Knockholt sometimes left out or inserted a character or two, generally because of poor radio reception. We called that a ‘slide’. A slide, of course, would weaken any statistical effect being used to set patterns. We therefore asked for subsequent Colossi to have the facility to ‘span’, to restrict its count to any stretch of the cipher tape that we could specify by giving the positions at which to start and to end. A good count could then be repeated on (say) the first two-thirds and again on the last two-thirds of the tape. Often one was strong and the other weak, suggesting a slide; subsequent spans could home in on the approximate position of the slide and later runs could be limited to the largest span available that seemed to be slide-free.

  In the Testery, they sharpened their methods and came to know more and more about the cribs useful for the various links. They commissioned and received a machine called Dragon. This was used on a de-chi; a likely crib was ‘dragged’ against it; it was subtracted at a range of plausible positions to give a stretch of putative ψ'. Dragon removed any repeated characters to give putative unextended ψ, and checked against the known psi-patterns, only recording a possible hit when it could fit all five.

  They also refined Turingery by a process (I think Peter Hilton invented it) called Devil Exorcism. In Turingery, characters of Δψ' that look likely to be / are assumed to be so, giving consequent Δχ bits that get cycled throughout. When the assumption is wrong it can proliferate wrong bits of Δχ. Devil exorcism was a technique that limited the damage done by such wrong assumptions.

  Both sections kept research books for anyone to record bright ideas. The Newmanry produced over forty of these, containing suggestions of new routines, calculations of significance tests, bits of mathematics that seemed relevant, reports of plain-text statistics and much else. In the Newmanry, tea parties were started; anyone could call one, ideas were bandied about, you came if you could and you brought your own tea. Tea parties didn’t decide things but they led to action on all fronts. Newman ran a comfortably democratic and friendly section. I was not the only one to persuade one of the Wrens to marry him. Mathematicians had occasional weeks off for research or secondment. I remember with particular pleasure a secondment to the Testery where Hilton guided me through a successful Turingery.

  Both sections grew steadily. The Testery recruited from the military, the Newmanry from the Wrens, the universities and from Dollis Hill, which also supplied us with one new Colossus each month, as well as most of the other machines. We depended totally on our engineers, who installed incoming machines and maintained the battery already installed. It was only because of them that we could keep on increasing our output.

  The Wrens were housed in the splendour they deserved in Woburn Abbey. We eventually had over 250 of them in the Newmanry, and their contribution (not just socially) was outstanding. They controlled the flow of tapes and ordered the runs to be made, with occasional advice from the analyst who was duty officer. They operated the machines, including the Colossi, with aplomb. Some of them would be in charge of Colossi on their own, moving expertly from run to run. For wheel-breaking, there was always an analyst in charge, but he would be helpless without a Wren to wind the tapes onto the bedstead and plug up the runs.

  Several analysts were seconded to us from the US Army and one from the US Navy; we also had highly professional advice at our tea parties from a US liaison officer. Unquestionably the Americans had generously sent us some of their best men. Four of them later held, as civilians, highly important posts in the postwar National Security Agency. They also sent us, near the end of the war, a photographic team. They had developed efficient ways of comparing two streams at all offsets by sliding photographic strips over each other and measuring the amount of light that shone through – instantaneous counting. It had the advantage of speed over Robinson, but came too late to make the impact the equipment deserved.

  The Last Months

  The Germans saluted the Normandy landings by changing wheelpatterns not monthly but daily. That meant that we had to break the patterns every day for all the links of intelligence interest. Fortunately by then our techniques of converging rectangles and completing the χ-breaking on Colossus were well established; we just had more of it to do. The Newmanry had by then spread into Block H where the wheel-breaking and the crib section were sited. It housed a large roomful of Wrens in Block H, converging rectangles, and the Colossi that had been adapted for wheel-breaking. The Testery had more dechis from which to break the psi- and μ-patterns; and, as the Germans progressively abandoned the autoclave limitations, more depths to disentangle. The job of setting known psi- and μ-patterns on de-chis was taken back into the Newmanry; Colossus was very well adapted for the job and we then no longer had to make de-chi tapes. That meant that, once a day’s patterns had been broken, all the setting was done in the Newmanry (as it had been in September 1943) and the settings were sent to the Testery for them to decipher the transmission.

  This period lacked incident except for the fire that we had in one of the Block F rooms. The fire was put out, and on that day, as it so happened, we solved more transmissions than ever before. All-round success persisted; technical advances and refinements continued. We were still getting better at it when the German war came to an end. Soon after that we were happy to be sent a mobile German Tunny machine; it was probably of more detailed interest to the engineers and telecommunication experts than to the rest of us.

  After the war some from both sections joined GCHQ. One Newmanry analyst was rumoured to have set up a bogus university that issued impressive degree documents to anyone willing to buy. Most of the mathematicians ended up teaching at real universities. Two Testery analysts were bold enough to start new organizations: Roy Jenkins later initiated and led the Social Democratic Party; and, most remarkably, Peter Benson started Amnesty, which later became the highly influential Amnesty International. I never lose an opportunity to claim them as wartime colleagues. Whatever we did after the war – ATS, Wrens, engineers, analysts – we all knew that we had been part of a very successful joint venture, and that it had significantly helped the Allied armed forces to win the German war.

  19

  COLOSSUS AND THE DAWNING OF THE COMPUTER AGE

  B. JACK COPELAND

  Introduction

  Historians describe intelligence as ‘the missing dimension’. Without an understanding of the intelligence those in power were receiving and its influence on the decisions they made, any history will be incomplete. It is now easy to see how important the breaking of the Axis codes and ciphers at Bletchley Park was in influencing the Allies’ conduct during the Second World War. But while details of many of the Cold War intelligence operations have made their way into the public domain, very little is still known about codebreaking successes during that period. We do not as yet know quite how important Sigint was in the decisions taken by Western politicians during the critical moments of the Cold War such as the Cuban missile crisis, the Soviet invasions of Hungary and Czechoslovakia, and the Solidarity-led industrial unrest in Poland. Sigint remains the ‘missing dimension’ to any history of the second half of the twentieth century.

  But it is not the only ‘missing dimension’. The second half of the twentieth century was dominated by the computer age. The tentative first steps taken during the war led initially to widespread use of computers within industry, later to the advent of the personal computer and finally to the construction of the Internet. Computers now dominate virtually every aspect of our lives but until only recently there was a ‘missing dimension’ to the history of the computer a
ge, and one in which, like the intelligence resulting from the breaking of the Axis codes and ciphers, Bletchley Park played a groundbreaking role.

  There were many remarkable people at Bletchley Park. But in early 1943, two extraordinarily gifted men came together to create Colossus, the world’s first electronic digital computer. Alan Turing did not, as is sometimes assumed, have any role in the construction of Colossus. A telephone engineer called Tommy Flowers built the computer, working on specifications laid down by Max Newman, a Cambridge mathematician. Told that it might affect the course of the war, Flowers had Colossus up and running by the end of 1943. The full details of that story remained secret until only very recently. Even now, Colossus remains something of a ‘missing dimension’ in computing history. Books claiming that the American ENIAC computer was the first electronic digital computer continue to be published. This chapter, by the distinguished computer historian Professor Jack Copeland, tells the true story of the birth of the modern computer.

  MS

  Colossus was the world’s first large-scale electronic digital computer. In the hands of the Bletchley Park codebreakers, it gave the Allies access to the most secret German radio communications, including messages from Hitler to his front-line generals. It was built during 1943 by Thomas (Tommy) Flowers and his team of engineers and wiremen, a ‘band of brothers’ who worked in utmost secrecy and at terrific speed. The construction of the machine took them ten months, working day and night, pushing themselves until (as Flowers said) their ‘eyes dropped out’. In January 1944, the racks of electronic components were transferred from Flowers’ workshops at the Post Office Research Station at Dollis Hill, north London, to Bletchley Park, where Colossus was assembled by Flowers’ engineers. Despite its complexity, and the fact that no such machine had previously been attempted, the computer was in working order almost straight away – testimony to the quality of Flowers’ design and the accuracy of the engineering work carried out at Dollis Hill. The name ‘Colossus’ was certainly apt: the machine was the size of a small room and weighed approximately a ton. By February 1944 Colossus was in use by the codebreakers of the Newmanry, a Bletchley section named after its head and founder, Cambridge mathematician Maxwell (Max) Newman.