The Electronic Computers, Part 2: Colossus

In 1938 the head of the British Secret Intelligence Service quietly bought up a sixty acre estate fifty miles from London. Located at the junction of railways running up from London to parts north and from Oxford in the west to Cambridge in the east, it was an ideal site for an organization that needed to be out of view, yet within easy reach of the most important centers of British knowledge and power. The estate, known as Bletchley Park, became the center of Britain’s code-breaking effort during World War II. It is perhaps the only place in the world that is famous for cryptography.

Tunny

In the summer of 1941, work was already well underway at Bletchley on cracking the famous Enigma cipher machine, used by both the German Army and Navy. If you have seen a movie about British code-breaking, it was about Enigma, but we will have little to say about it here. That’s because, shortly after the invasion of the Soviet Union, Bletchley picked up on a new kind of encrypted traffic.

It did not take long for the cryptanalysts to discern the general nature of the machine used to send this traffic, which they dubbed “Tunny.”

Unlike Engima, which required hand transcription of messages, Tunny was attached directly to a teletypewriter. The teletypewriter converted each character typed by the operator into a stream of “dots and crosses” (equivalent to Morse dots and dashes) in the standard Baudot code, with five symbols per character. This was the unencrypted message text. The Tunny machine simultaneously used twelve wheels to generate its own parallel stream of dots and crosses: the key. It then “added” the key to the message, producing the encrypted cipher text which went out over the air.  This addition was done with modulo-2 arithmetic (i.e. wrapping 2 back around to 0), reading the dot as 0 and the cross as 1:

0 + 0 = 0  
0 + 1 = 1
1 + 1 = 0

Another Tunny on the receiver’s side, with the same settings, generated the same key and added it to the cipher text to extract the original message, which was printed onto paper tape at the receiver’s teletypewriter.¹

The work of understanding Tunny was made much easier by the fact that in the early months of its use, senders transmitted the wheel settings to be used before sending the message. Later the Germans issued codebooks with pre-defined wheel settings  – the sender would send only the code, which the recipient could use to look up the proper wheel setting in the book. Eventually they began changing the codebooks daily, forcing Bletchley to start all over with cracking the wheel settings for the codes each morning.

Intriguingly, the cryptanalysts also discerned the function of Tunny, based on the location of the sending and receiving stations. It connected the nerve centers of the German High Command to army and army group commanders at the various European war fronts, from occupied France to the Russian steppes. It was a seductive problem: to break into Tunny promised direct access to the intentions and capabilities of the enemy at the highest level.

Then, through a combination of German operator error, cleverness, and sheer bloody-minded determination, a young mathematician named William Tutte went well beyond these basic inferences about Tunny. Without having ever seen the machine itself, he determined its complete internal structure. He deduced the possible positions of each wheel (a different prime number for each), and exactly how the arrangement of wheels generated the key. Armed with this information, Bletchley built replica Tunnys that could be used to decode a message – once they worked out the proper wheel setting.

Heath Robinson

By the end of 1942, Tutte had continued his attack on Tunny, by devising a strategy for doing just that. It was based on the concept of a delta: the modulo-2 sum between one signal (dot or cross, 0 or 1) in a message stream and the next. He realized that due to a stutter in the motion of the wheels of the Tunny, there was a correlation between the delta of the cipher text and the delta of the key text²: they would tend to change together. So if you could compare cipher text against the key text generated at various wheel settings, you could compute the delta of each and count the number of matches. A match rate significantly higher than 50% would indicate a potential candidate for the real key of the message. It was a nice idea in theory, but was impossible to carry out in practice, requiring 2,400 passes through each message to test the various possible settings.

Tutte brought his problem to another mathematician, Max Newman, who oversaw a section at Bletchley, known simply as the Newmanry. Newman was at first glance an improbable figure to lead a sensitive British intelligence organization, given that his father was German-born. However he probably seemed a rather unlikely spy for Hitler, given that his family was also Jewish. So alarmed was he at the progress of Hitler’s domination of Europe that he evacuated his family to safety in New York shortly after the collapse of France in 1940, and for a time considered moving to Princeton himself.

It just so happened that Newman had an idea for tackling the calculations required by Tutte’s method – by building a machine. Using machines for cryptanalysis was nothing new to Bletchley. That was, after all, how Enigma was being cracked. But Newman had a particular, electronic device in mind for the Tunny machine. He had taught at Cambridge before the war (Alan Turing had been a student of his), and knew about the electronic counters Charles Eryl Wynn-Williams had built for counting particles at the Cavendish. Here was Newman’s idea: If one could synchronize two loops of tape, spinning at high speed – one with the key and one with the enciphered message – and read each element into a processing unit that computed the deltas, an electronic counter could accumulate the results. By reading off the final count at the end of each run, one could decide if the key was a promising one or not.

It so happened that a group of engineers with suitable expertise was readily available. Among them, Wynn-Williams himself. Turing had recruited Wynn-Williams from the radar lab in Malvern, to help build a new rotor for the Enigma decoding machine that would use electronics to register the rotations. Assisting him with that and another related Enigma project were three engineers from the Post Office Research Station at Dollis Hill: William Chandler, Sidney Broadhurst, and Tommy Flowers. (Recall that the British Post Office was a high-tech operation, responsible for telegraphy and telephony, as well as paper mail). Both projects had gone bust and now the men were at loose ends. Newman scooped them up. He assigned Flowers to lead the team building the “combining unit”, which would compute the deltas and pass the result on to the counter, which was the responsibility of Wynn-Williams.

Along with the engineers to build them, Newman tasked the Women’s Royal Naval Service, or Wrens, to operate his message processing machines. While the government trusted only men with high-level, executive positions, women performed much of the operational work at Bletchley, from message transcription to setting up decoding runs. They naturally transitioned from performing clerical work themselves to the tending of the machines that automated that work. The Wrens facetiously dubbed their charge “Heath Robinson”, the British equivalent to Rube Goldberg.

Indeed, the Heath Robinson, though sound in concept, suffered from serious practical problems. Most notably, the need to keep the two tapes – cipher text and key text – in perfect sync. Any stretching or slippage in either tape would ruin an entire run. In order to minimize the risk of error, the machine processed no more than 2,000 characters per second, though the belts could have been run faster. Flowers, though he reluctantly went along with the Heath Robinson project, believed there was a better way: a machine built almost entirely from electronic components.

Colossus

Thomas Flowers had served as an engineer in the Post Office research division since 1930, where he was initially set the problem of investigating mis-dialed and failed connections in the new automatic exchanges. This led him to ruminate on how to build an altogether better telephone system, and by 1935 he had become a missionary for replacing the electro-mechanical components of that system, such as relays, with electronics. This cause would define his entire career thereafter.

Most engineers dismissed electronic components as too balky and unreliable for use on a large scale, but Flowers showed that if kept running continuously and at well below their rated power, tubes actually had remarkable longevity. He proved out his ideas by replacing all the terminals for establishing connection tones in a 1,000 line switch with tubes; three or four thousand in total. This installation went into live service in 1939. In the same period, he also experimented with replacing the relay-based registers for storing telephone numbers with electronic switches.

Flowers believed that the Heath Robinson that he had been recruited to help build was seriously flawed, and that he could do much better by using many more tubes and fewer mechanical parts. In February 1943, he brought his alternative design to Newman. In an imaginative leap, Flowers disposed with the key text tape altogether, completely eliminating the synchronization problem. Instead his machine would generate the key text on the fly. It would simulate a Tunny electronically, iterating through wheel settings and trying each against the cipher text and recording possible matches. He estimated that this approach would entail the use of about 1,500 tubes. 

Newman, and the rest of the Bletchley leadership, cast a skeptical eye on this proposal. Like most of Flowers’ contemporaries, they doubted whether electronics could be made to work on such a scale. They further doubted whether, even were it made to work, such a machine could be built in time to be useful to the war effort.

Flowers’ boss at Dollis Hill nonetheless authorized him to assemble a team to build his electronic monster – though Flowers may have given a misleading impression of just how much backing he had from Bletchley.³ In addition to Flowers, Sidney Broadhurst and William Chandler played a major part in the design work, and the effort as a whole required fully fifty people, half of Dollis Hill’s resources. The team drew on precedents from telephone machinery: counters, branching logic, equipment for routing and translating signals, and “routiners” for putting equipment through a series of pre-programmed tests. Broadhurst was a master of these electro-mechanical circuits, while Flowers and Chandler were the electronics experts, who understood how to translate concepts from the world of relays into the world of valves. By early 1944, the team had delivered a working model to Bletchley.⁴ The giant machine acquired the code name Colossus, and quickly proved that it could outshine the Heath Robinson, reliably processing 5,000 characters per second.

Newman and the rest of the command chain at Bletchley were not slow to realize their earlier mistake in dismissing Flowers’ ambitions. In February 1944, they requested twelve more Colossoi to be in operation by June 1 – the intended date for the invasion of France, though of course Flowers knew nothing of that. Flowers flatly declared this to be impossible, but with heroic efforts his team managed to deliver a second machine by May 31, with a new team member, Allan Coombs, making many of the design improvements.

This revised design, known as the Mark II, expanded on the success of the first. In addition to the tape feed, it consisted of 2,400 tubes, 12 rotary switches, 800 relays and an electric typewriter.

It was configurable and flexible enough to perform a variety of tasks within its milieu. After installation, each team of Wrens customized their Colossus to suit the particular problems they needed to solve. A plugboard, modeled on a telephone operator’s patch panel, established the settings for the electronic rings which simulated the wheels of the Tunny machine. More broadly, a series of switches allowed operators to set up any number of different function units to compute over the two data streams: the external tape and the internal signals generated by the rings. By combining a variety of different logic gates, Colossus could compute arbitrary Boolean functions on that data, i.e. functions that output a 0 or 1. Every output of 1 incremented Colossus’ counter. A separate control unit made branching decisions based on the state of the counter, e.g. stop and print the output if the counter is greater than 1000.

Let us not imagine, however, that Colossus was a programmable, general-purpose computer in the modern sense. It could logically combine two data streams – one on tape, one generated from ring counters – and count the number of 1s encountered, and that was all. Much of the “programming” of Colossus was actually carried out on paper, with operators executing decision trees prepared by analysts; e.g. “if the output was less than X, set up configuration B and do Y, otherwise do Z”.⁵

Nonetheless, for the task it was asked to do, Colossus was quite capable. Unlike the Atanasoff-Berry Computer, Colossus was extraordinarily fast – able to process 25,000 characters per second, each of which might require several Boolean operations. (The Mark II  quintupled the speed of the Mark I by simultaneously reading and processing five different sections of tape.) It avoided coupling the entire system to slow electro-mechanical input and output devices by using photoelectric cells (derived from anti-aircraft proximity fuses) to read the input tapes and using a register to buffer output to the teletypewriter. The leader of a team that rebuilt Colossus in the 1990s showed that, in its wheelhouse, it would still easily outperform a 1995-era Pentium processor.⁶

This powerful text-processing engine became the centerpiece of the Tunny code-breaking effort.⁷ Another ten Mark II’s were built before the end of the war, the panels churned out one per month by workers at the Post Office factory in Birmingham who had no idea what they were building, then assembled at Bletchley. One exasperated official at the Ministry of Supply, on receiving yet another requests for thousands of specialized valves, wondered whether the Post Office folks were “shooting them at the Jerries.”⁸ Not until well into the 1950s would another electronic computer be produced in this fashion: as an industrial product, rather than a one-off research project. At Flowers’ instruction, in order to preserve the valves, each Colossus remained on night and day, until the end of the war. Glowing quietly in the dark, warming the damp British winter, they waited patiently for instructions, until they day came when they were no longer needed.

A Curtain of Silence

An understandable enthusiasm for the intriguing drama of Bletchley has sometimes led to a wanton exaggeration of its military achievements. To imply, as does the movie The Imitation Game, that British civilization would have been extinguished if not for Alan Turing, is a monstrous absurdity. Colossus, specifically, seems to have had no real effect on the course of the the struggle for Europe. Its most widely touted achievement was to prove that the deception plan around the 1944 Normandy landing had worked. The Tunny traffic revealed that the allies had succeeded in convincing Hitler and his High Command that the true blow would land farther east, at the Pas de Calais. Reassuring information, but reducing the cortisol levels of allied commanders probably did not help to win the war.

The technological achievement represented by Colossus itself, on the other hand, is incontrovertible. But the world would not soon learn of it. Churchill ordered all the Colossoi in existence at the end of the war to be dismantled, taking the secret of their construction with them to the scrapyard.  Two machines somehow survived this death sentence, remaining in use within the British intelligence apparatus until about 1960.⁹ Still the British government did not lift the curtain of silence around the activities at Bletchley. Not until the 1970s would its existence become public knowledge.

The decision to suppress all discussion of the activities at Bletchley Park indefinitely was on the part of the British government a mild excess of caution. But it was a personal tragedy for Flowers. Denied all the honors and prestige that would be due to the inventor of the Colossus, he suffered frustration and disappointment, stymied repeatedly in his ongoing effort to replace relays with electronics in the British telephone system. Had he been able to pull the historic achievement of Colossus from his back pocket, he might well have had the influence necessary to carry his vision forward. By the time his achievements became known in full, Flowers had long since retired, and could no longer influence anything. 

The few, scattered enthusiasts for electronic computing might have suffered a similar setback from the secrecy around Colossus, lacking any evidence to prove its viability to the skeptics. Electro-mechanical computing might have continued to dominate for some time. But there was, in fact, another project that would pave the way for electronic computing’s rise to dominance. Though also the result of a secret military effort, it was not kept hidden after the war, but instead revealed to the world with great fanfare, as ENIAC.

Further Reading

Jack Copeland, ed. Colossus: The Secrets of Bletchley Park’s Codebreaking Computers (2006)

Thomas H. Flowers, “The Design of Colossus,” Annals of the History of Computing, July 1983

Andrew Hodges, Alan Turing: The Enigma (1983)