Enigma: Why the fight to break Nazi encryption still matters

It was night when three British sailors and a 16-year-old canteen assistant boarded a sinking U-boat off the coast of Egypt. A spotlight shone on them from the HMS Petard, the Royal Navy destroyer that had hunted down the German submarine and now slowly circled the vessel. The U-boat's commander lay dead below the hatch as water poured in from a crack in the hull.

The four men began searching the ship, but not for survivors. They were looking for codebooks.

These red-covered guides were vital to breaking a diabolical code that made Nazi radio messages unintelligible. The Germans had been using a typewriter-like machine to encrypt their communications. They called it Enigma and were sure the code was unbreakable.

The British were determined to prove them wrong.

Wading past bodies through slowly rising water, First Lieutenant Anthony Fasson, Able Seamen Colin Grazier and Kenneth Lacroix, and young Tommy Brown found the captain's quarters and began searching drawers and breaking into cabinets. They found two codebooks written in red, water-soluble ink: the Short Weather Cipher, used to condense weather reports into a seven-letter message, and the Short Signal Book, used to report convoy sightings, along with other documents.

While Grazier and Fasson continued to search below, Brown carried the books up the ladder of the sub's conning tower to awaiting boat. They were racing against time as seawater poured into the submarine.

On his third trip up the ladder, Brown called for his shipmates to come up, too -- but it was too late. U-559 sank before Fasson and Grazier could escape that night in October 1942. As Hugh Sebag-Montefiore recounts in "Enigma: The Battle for the Code," their bravery helped changed the course of World War II.

The U-boat codes created by Enigma were especially hard to break, and the Allies found themselves locked out for weeks or months at a time. But several months after they recovered the codebooks from U-559 -- on March 19, 1943 -- cryptographers stationed in Britain's Bletchley Park broke through into U-boats' Enigma-coded messages and were never fully locked out again.

From then on, their efforts only improved. By September of that year, the Allies were reading encrypted U-boat messages within 24 hours of intercepting them. The breakthrough allowed the Allies to decrypt detailed field messages on German defenses in Normandy, the site of the impending D-Day invasion.  And the machines themselves advanced the world's technology  -- pushing forward ideas about computer programming and memory.

"I'd call it the key to computing," says Ralph Simpson, a retired computer expert and amateur Enigma historian.

The years since have given us a cat-and-mouse game between codebreakers and cryptographers, with each side trying to outwit the other. Those battles are still raging. But they're no longer confined to blackboards and spinning rotors on crude computers. They move at the speed of electrons flowing through your computer's processor.

Today's computer-enabled encryption -- technology that scrambles what unauthorized viewers see  --  is so complex that computers can't break it unless it's been used incorrectly. It's so powerful that the US government and others have tried to legally require tech companies to unlock their own encryption, as was the case with Apple and the government last year over a terrorist's locked iPhone .

And today's encryption is so useful that dissidents, spies and terrorists rely on it to protect their conversations.

The innovation won't stop. Future advances in quantum computing might be able to crack even perfectly implemented encryption. That's led mathematicians to pre-emptively try to make encryption even stronger.

It's a cycle without end in sight.

Encryption everywhere

Before the internet wove its way into our lives, encryption was pretty much something businesses and governments used to protect sensitive data, like financial documents and Social Security records.

"Mostly it was banks, diplomatic services and the military who used cryptography throughout history," says Bill Burr, a retired cryptographer from the US National Institute of Standards and Technology.

The internet increased the use of encryption, as business and governments sent information over networks that hackers and spies could easily intercept. But few regular people went out of their way to use encryption as part of daily life. Maybe your paranoid friend would encrypt his email, forcing you to use extra software to read it.

That changed after disclosures by former NSA contractor Edward Snowden , who in the summer of 2013 revealed the existence of government mass surveillance programs designed to collect reams of information from everything -- our emails, calls and texts. Though we were told the programs weren't designed to target Americans, the disclosures forced us to ask how much information we want to put on the internet -- and potentially expose.

The tech industry has tried to address the problem by offering us another option: encrypting as much of our lives as we can.

What's made this possible was the Engima, and the men, women, mathematicians, computer scientists and linguists who ultimately beat it.

This is their story.

A mystery inside an Enigma

Watch this: This is how the Nazi Enigma machine works

04:44

The Enigma has a surprisingly understated design for being such a deadly tool. It could easily be mistaken for a typewriter with a few extra parts, housed in a plain wooden box.

Lifting the lid of an Enigma, a German operator saw what might on first glance seem like two typewriters squished together. One set of keys, closest to the operator, was the actual keyboard to be typed on.

Above it was a second set of keys, laid out just like the keyboard. But when you type on the real keyboard, these letters light up. Type an "a" on the normal keyboard, for example, and "x" lights up above.

So if you start typing a word, each letter lights up in code.

This was Enigma's genius. The German operators didn't need to understand the complex math or electronics that scrambled what they typed on the keyboard. All they knew was that typing "H-E-L-L-O" would light up as "X-T-Y-A-E," for example. And that's the message they sent around.

This jumbling of letters changed each day at midnight, when Nazi commanders would send new settings that Enigma operators would use to turn dials and change the plugs on a board below the keys, all designed to match the day's code. Without the code, the message couldn't be unscrambled.

Enigma was so sophisticated it amounted to what's now called a 76-bit encryption key. One example of how complex it was: typing the same letters together, like "H-H" (for Heil Hitler") could result in two different letters, like "L-N."

That type of complexity made the machines impossible to break by hand, Simpson says.

How impossible? If you gave 100,000 operators each their own Enigma machine, and they spent 24 hours a day, 7 days a week testing a new setting every second, "it would take twice the age of the universe to break the code," Simpson says.

Obviously, codebreaking by hand wasn't going to cut it.

"Because we now have machine encryption for the first time, it took a machine to break it," Simpson says.

Equally fascinating is that Nazi military leaders knew, in theory, that someone could develop a machine-assisted way to speed up their code cracking. But they didn't believe their enemies would put in the time and resources needed.

They were wrong.

See Alan Turing's lost notes, found in the walls of Bletchley Park 70 years later

+11 More

See all photos

A series of tubes

Of course, the UK was very motivated to break the Enigma. German U-boats were sinking hundreds of British ships, costing thousands of lives and choking the country off from vital supplies being shipped from the United States and Canada. What's more, the country was desperate for any advantage in the early days of the war, filled with German bombing campaigns and fears of a land invasion.

So resources, manpower and the lives of sailors like Fasson and Glazier were poured into cracking the Enigma codes. The first result of these efforts was the Bombe.

Custom-designed by British mathematicians like Alan Turing , Bombes were about the size of three vending machines stacked side by side, with a series of spinning rotators connected in the back by a 26-way cable. They were based on the Polish "Bomba" codebreaking machine, which the Poles were forced to abandon in 1939, after their country was invaded by Germany.

Housed at a secretive intelligence program on the grounds of manor house Bletchley Park, less than 50 miles outside of London, and other nearby installations, the Bombes were run by teams of Navy women.

Each of the Bombe's rotators had letters on it and, as they spun, the machine tested possible solutions to a given Enigma code much faster than a human could.

Researchers like Turing and his team were able to make the Bombes more efficient by using "pinched" codebooks from U-boats and other clues, eliminating thousands of possible solutions.

"If we understand the book, we then know what the submarines are likely to say," says David Kenyon, a research historian at the Bletchley Park Trust.

Breaking into the U-boat's "Shark" code in 1943 set in motion a series of dominoes that ultimately led to the Nazi defeat. Intercepted U-boat messages made the Allies better at sinking the vessels, which contributed to the German Navy's decision to pull its U-boats out of the Atlantic later that year, Kenyon says. That respite allowed the Allies to prepare for D-Day in 1944 and to end the war in 1945.

While codebreaking alone didn't win World War II, it was one of the most powerful weapons invented for that purpose.

"There was no point in the Second World War where the outcome was a foregone conclusion," Kenyon says. There's no telling what might have happened "if you took away any of the factors that were working in the Allies' favor."

Photo Tour of Bletchley Park

+32 More

See all photos

Making computers real

The work done on the Bombes and other codebreaking machines didn't just aid in the fight against the Nazis. They proved theories about computer programming and data storage, the lifeblood of today's modern computers.  

One of these breakthroughs came when the Joseph Desch of the US Navy found a way to speed up the Bombe. The machines could only run so fast, because operators read the results of the codebreaking analysis right off of the wheels themselves. Go any faster and the wheels would spin right past the correct answer.

Desch's solution was a primitive form of digital memory. When the Bombe came upon the correct answer, electrical relays would detect and record it. That let the US Bombes spin more than 17 times faster than the British Bombes.

Then there was Colossus. This machine -- designed not to break Enigma, but rather the more sophisticated Lorenz codes used by the German High Command -- advanced vacuum tube tech that later came to power the world's first true computers, like the ENIAC and Mark-1, and then the first generation of IBM mainframes.

To create a codebreaking machine powerful enough to crack Lorenz, British engineer Tommy Flowers found a way to run more than 2,000 vacuum tubes at once. While it had been theorized this approach could power a programmable computer, Flowers was the first to make it happen.

Flowers himself didn't get a chance to push this technology to its next logical conclusion. But Turing and other Bletchley alums worked at the University of Manchester after the war, creating the Ferranti Mark 1 -- a programmable computer run with vacuum tubes.

That the work at Bletchley showed up later in the first general-purpose computers doesn't surprise Burr. The codebreakers were able to fully understand the workings of Enigma and the Lorenz code create machines to break them at a time when the principles of computing only existed in theory.

"It's hard for me to imagine people smart enough to do that," says Burr, who's an expert in cryptography.

A war to start the crypto-wars

In terms of global politics, encryption was pretty straightforward during World War II. One nation tapped its linguists and mathematicians -- and relied on the heroism of men who boarded sinking U-boats -- to crack the encryption tech of an enemy force.

The world's gotten a lot more complicated since then.

Just as in World War II, law enforcement and spy agencies today try to read the communications of criminals, terrorists and spies. But now that almost everyone uses encryption, a government's ability to break it doesn't just worry our country's enemies -- it concerns us, too.

And despite the advances in computing and encryption since Bletchley Park, we haven't come close to agreeing on when it's okay to break encryption.

Case in point: the 2016 conflict between Apple and the US Federal Bureau of Investigation. The FBI wanted Apple's help breaking into the iPhone of a suspected terrorist, but Apple argued that this could put everyone who uses an iPhone at risk.

Burr, who saw the inside of public controversies over the government breaking encryption during his time at the National Institute of Standards and Technology, says there's no clear path forward.

"There's just a big dilemma there," he says. Creating ways to break encryption "will weaken the actual strength of your security against bad guys of ability. And you have to count among those the state actors and pretty sophisticated and organized criminals."

In their laser-focused effort to crack Nazi encryption, codebreakers like Turing and soldiers like Fasson and Grazier were unlikely to have imagined a world like this. But here it is: the catch-22 of computerized encryption. And it's not going away anytime soon.

Special Reports: CNET's in-depth features in one place.

Tech Enabled: CNET chronicles tech's role in providing new kinds of accessibility.