The Secret State

Introduction

ABOUT THIS BOOK

This is a book that has its roots in Military Intelligence Blunders, first written in 1999. It sold well and a second, updated edition came out in 2004 after the 9/11 attack on the World Trade Center and the invasion of Iraq. That too became a best-seller and I was both surprised and heartened to see that the book has since been translated into six languages, including Turkish and Japanese, and has become recommended reading by intelligence agencies, as well as a text book for a number of university courses on intelligence.

Since 2004 much has changed however and in this new book, which is much wider in scope than its predecessors, I have tried to capture some of these changes and assess their impact to give the reader a much broader view of the whole subject of intelligence. Hence its title: On Intelligence, However, unlike Clausewitz’s monumental 1832, On War, this work on intelligence is intended to avoid the pitfalls of the now dated German meisterwerk.

For a start it is meant to be readable. With a book that is an important issue. It would be interesting to discover just how many people (let alone academic military historians) have actually read all eight volumes and all the complexities of von Clausewitz in the original German. I suspect, rather like the Communist–Socialist bible, Das Kapital, very few folk have actually read and absorbed the master’s deep thoughts, let alone understood the linguistic subtleties and nuances contained therein.

On Intelligence is therefore not intended as a ruminative academic stroll through historic events to reach some broad reflective conclusions, before getting completely lost in the thickets of philosophy and Hegelian dialectic. Rather it is intended as an up-to-date analysis of intelligence in the recent past, and how its impact has affected great events. Wherever possible, case studies and hard examples are analysed in some detail, looking for answers rather than the unfinished metaphysical reflections that so characterise On War. There are some tough lessons to be learned from intelligence failures especially today, and from intelligence successes, too.

But On War and On Intelligence do both share a common aim. In von Clausewitz’s own words from long, long ago: ‘To write a book that will not be forgotten after two or three years.’

Most of us have read press accounts and books about the events that unfold on these pages. But very few of us have seen the events from the inside. The inside implies knowledge: and knowledge means power.

By ‘inside’, I do not mean the views of politicians or the other self-satisfied classes like senior civil servants or even those very grand journalists who often write memoirs on the lines of, ‘Well, as the president said to me . . .’

The real ‘insiders’ knowledge’ is always the intelligence that was available at the time. It was that secret intelligence that shaped events and made the people who took the decisions into heroes or villains. This book tries to lift the veil on what really happened behind the scenes in the intelligence world during some of the most well-known military events that have shaped our lives. It tries to show why decisions were made, for good or ill, by a number of famous and not so famous characters, based on the intelligence and the secrets they had to work with at the time. This book tends to highlight intelligence mistakes and blunders (carefully concealed from the taxpayer who paid for them) for the simple reason that they are more interesting than the far more numerous successes of intelligence.

The book also identifies numerous deceptions, lies and cover-ups. Not all of these were committed to deceive the enemy. Turning over stones invariably lets a harsh light onto some creatures of the shadows. There are many intelligence officers and government officials, in all regimes, who prefer to keep their role and decisions secret, and thrive on secrecy in order to protect their careers and way of life. It guarantees their income, their status and their pensions. Secrecy is everything to them and far too often it is not for any motives of ‘national security.’ In this desire to cover things up, they are too frequently encouraged – and joined – by their political masters.

In this they are only outclassed by their bosses, those very senior civil servants and ‘securocrats’ of every country’s administration, who equally thrive and prosper in the sure and certain knowledge that access to secret intelligence has given them the ultimate benefit of the mistress or harem: power without responsibility. But, like their political masters, good intelligence officers should always have the courage of their convictions, and be ready to show moral courage by taking responsibility for their advice. Intelligence is, after all, about predicting the most likely future, not just regaling concerned decision makers with hot new facts. That is called news. CNN, the BBC, Fox, Sky News and now the immediacy of social media do the rolling 24/7 news agenda far better than any intelligence agency, as every professional modern intelligence officer knows. However, when careers are at stake that is a hard precept to follow.

If I had to offer a motto for any intelligence officer I would unhesitatingly quote the 1950s American Friends (Quaker) phrase, ‘Speak Truth unto Power.’ That phrase has since been long associated with the British Civil Service. To be able to give honest and objective advice to ministers has always been seen as something that all impartial civil servants and intelligence officers should be able to do. Sadly, in their complicity over the scandal of Tony Blair and Alastair Campbell’s use of a notoriously misleading (if not downright mendacious) ‘sexed-up’ intelligence dossier to convince Parliament and the British people to enter the 2003 Iraq War, Whitehall’s not so faceless intelligence bureaucrats let their calling – and their countrymen – down badly.

The various case studies in the book are intended to provide an accessible and readable narrative of the events they describe, accompanied by some professional intelligence insights into how those events came about and unfolded. They tread a delicate path between the laboriously footnoted and exhaustive detail of the academic PhD tome, and the trivial, flippant (and often inaccurate) ‘popular history’ of cheap journalism. Where possible, quotations are clearly indicated. For those who would like to delve into the stories in more detail, a short reading list is provided so that they can be better informed about events and individuals. History is, after all, ‘a never ending argument’.

In putting together this book I have been aided across the years by the Director and staff of the Royal United Services Institute in Whitehall. Various members of the Study Group on Intelligence at the RUSI have made invaluable suggestions over the years on a variety of topics. In particular Professor Christopher Andrew and the late (and much missed) Professor M. R. D. Foot have been influential, as have their colleagues, Professors Richard Aldrich and Gary Sheffield. The Intelligence Corps Archives, now at Chicksands, have dotted many I’s and crossed many T’s on key reminiscences. For details of Vietnam I owe a considerable debt to my many American friends and colleagues over the years, especially the late Colonel John Moon and Colonel John Robbins of the United States Army for their perceptive comments on my drafts of American events, and also for their previously unpublished memories of Tet. The late Val Heller of the Defense Intelligence Agency contributed a number of reveal-ing dry – and wry – comments about the internecine inter-agency turf battles common to the alphabet soup that constitutes the US intelligence agencies.

Many others in the intelligence world of several nationalities have helped me, both on and off the record. I am particularly grateful to Andreas Campomar, late of Constable & Robinson (now Little, Brown) for recognising the need for a new, more all-embracing book on intelligence, and also to Andy Hayward, a doyen of the London publishing scene, for suggesting the idea in the first place. For a sharp professional scrutiny of the manuscript and the avoidance of any ‘fouls’ I can only express deep gratitude to my editor, Josh Ireland, who pains-takingly checked the text and made numerous helpful suggestions to improve the finished product. No author likes (in the words of Robert Louis Stevenson) to ‘murder his darlings’. Josh has been a skilful executioner of cherished – but redundant – text.

Any errors and omissions however are mine own, as are the opinions expressed. They represent the broad overview of over forty years working with, and thinking about, intelligence in general and military intelligence in particular. Having said that, this is most certainly not a theoretical text about international affairs. On Intelligence instead gives the reader a broad overview of a vast subject, ranging from biblical spies to modern satellites and cyberwar. Every one of the topics covered rates a complete book of its own; and in many cases, has already filled whole library shelves. The book’s chapters aim to provide the reader with accessible, well-informed insights into a variety of issues, with many secrets and technicalities. Because, above all, On Intelligence is a book that is meant to be read and enjoyed by the general reader and the intelligence professional alike.

I genuinely hope both gain something from my labours.

John Hughes-Wilson, Cyprus

PART 1

SETTING THE SCENE

CHAPTER ONE

A Little History

‘If you do not know others and do not know yourself, you will be in danger in every single battle.’

Sun Tzu, The Art of War

From the dawn of time, intelligence has mattered. The knowledge of where your enemy is, what forces he possesses, and what he plans to do, should help even the most dim-witted and stubborn politician or general.

Every great commander in history has relied on good intelligence as a ‘force multiplier’, or as a guide as to when and where he should strike. Those unfortunates without good intelligence have, in their turn, usually gone down to defeat, death and disgrace.

Some call intelligence the ‘Second Oldest Profession’ – experts say, however, that it is the first, because knowledge of where the enemy is and the need for self-preservation and survival rates above all other urges. Ask any bird hunting for worms in a cat-infested garden . . .

What is intelligence? There are many different meanings of the word, but we all admire it and we all recognise the real thing when we see it.

The Bible has some of the first evidence of military intelligence: in particular, its reference to Moses’ famous spying expedition into present-day Israel. On the run from Egypt in the inhospitable burning wastes of the Sinai desert, the wandering tribes needed somewhere to settle with abundant water, fertile soil and green trees. The Hebrews knew exactly where to turn for assistance in their hour of need: as the Old Testament says,

And the Lord spoke unto Moses, saying, ‘Send thou men that they may search the land of Canaan’ . . . And Moses sent them to spy out the land of Canaan and said unto them . . . ‘And see the land, what it is; and the people that dwell therein, whether they be strong or weak, few or many. And what the land is that they dwell in, whether in tents or in strongholds . . .’

In modern parlance, Moses had issued his ‘critical intelligence requirements’.

The Bible is full of such tales of military prowess, battles, deception and, above all, intelligence gathering. This is hardly surprising, as many modern scholars believe that the Hebrew’s chosen god, ‘Jehovah’, was the Israelites’ original god of war from their pantheon. Later, Moses’ successor, Joshua, sends two spies to report on the walled city of Jericho and its defences. According to the Bible story, the two spies are hidden by a friendly prostitute called Rahab; the first, but by no means the last, evidence of close collaboration between the two ‘oldest professions’. Betrayed by an informer, the Hebrew agents flee the city with Rahab’s help, for which service her house and life are spared when the well-briefed Jewish army later storms and sacks Jericho. It is a classic example of intelligence in action.

Another Old Testament example is the story of Delilah, the earliest recorded example of a ‘honey trap’, or what the KGB called a ‘swallow’. The Philistines decide that she is the ideal tool with which to entrap the Israelites’ strongman, Samson, and Delilah lures the Jewish champion into her boudoir. Samson blurts out the secret of his strength (his hair) before, like so many men before and since, falling into the deep sleep of the ‘just after’. Shorn of his strength by a gentle barber and blinded by not so gentle Philistines, Samson languishes in captivity before finally pulling down the columns – and the roof – of his prison on his tormentors once his hair has grown back. It is a cautionary tale of the power of women and the weakness of men confronted with a suitably seductive and available source of temptation.

There is other, more concrete, evidence of ancient intelligence in action. On the wall of the great temple at Karnak in Egypt is the graphic story of Tutmoses III’s triumphant campaign against the Syrian uprising of 1488 BC. The warrior Pharaoh’s speedy reaction to the rebellion was made possible by his secret agents in Megiddo. These undercover spies noted Khadesh’s growing army in the north and promptly rode south to warn the Egyptian outpost fort at Tjuru (near present-day Port Said) of the gathering storm, months before the rebels were ready to move against their new young Pharaoh.

According to the Greek chronicler Herodotus, when Histiaeus the Milesian was sent to the court of King Darius, the Persians took him for a possible spy, briefed to report on any weakness in the Persian Empire. He was placed under house arrest and all his communications back home blocked. However, Histiaeus thought that he had identified the weak spot in the Persians’ grip on their sprawling Empire; but how to tell his fellow Greeks and evade the Persians’ total shutdown on messengers or letters?

Histiaeus’ solution was simple. He cut the hair off a servant’s head, tattooed the crucial message onto the shorn scalp, and let the hair grow back. The servant was then despatched home. Darius’s men let him pass, little knowing that his head bore the crucial words, ‘Histiaeus bids thee to incite the Ionians to rise up and revolt against Darius.’ The ruse worked. The simple servant returned home bearing the vital intelligence on, not in, his head.

Legend has it that the travel-stained servant was brought in to Histiaeus’s son-in-law Aristagoras carrying the bizarre request, ‘Thy father-in-law Histiaeus bids thee to shave my head . . .’ The Greeks took note and Aristagoras duly incited Darius’s Ionian vassals to rise against their unpopular overlord. If nothing else, the story stresses the need for very clear and very brief secret communications. There is, after all, only so much space on the human scalp.

The first known writer on intelligence in the ancient world appears to have been Sun Tzu, a serious soldier and government bureaucrat who lived and fought around the Yellow River province of Wu around 500 BC – well before Rome’s pre-eminence. Like many professional soldiers before and since, Sun Tzu realised that there was far more to war than just battles. Sometime towards the end of his career he wrote a classic textbook called, The Art of War (Ping Fa), which placed a heavy emphasis on intelligence.

Realising that any half-decent and well-led enemy will rarely have the good grace or courtesy to conform to his own wishes, (and indeed, may even have some tiresome ideas of their own) Sun Tzu thought deeply about intelligence, and made it a priority for soldiers and statesmen alike.

Sun Tzu classifies his ‘collection agents’ into five main groups:

1.   Local inhabitants.

2.   Government officials in the enemy camp who would betray their government in order to stay in their jobs.

3.   Enemy spies who could be ‘turned’ and doubled to play back disinformation.

4.   Expendable agents who can be sacrificed to feed false information to the enemy.

5.   Spies who can be relied on to penetrate the enemy, survive and report accurately from inside the enemy camp.

Sun Tzu realised that the relatively small resources spent on garnering intelligence in peacetime were in fact investments: insurance policies against having to spend far, far more if – or when – war broke out. To Sun Tzu, good intelligence in peacetime was as much a part of national defence as an army on the march in war: and a damned sight cheaper too . . .

His philosophy can best be seen by his clear understanding of intelligence, or ‘foreknowledge’:

Now the reason that the enlightened prince and wise general conquer their enemy is foreknowledge.

What is called foreknowledge cannot be divined from spirits, gods, comparison with past events nor from calculations. It must be obtained from men who know the enemy’s true situation . . .

Sun Tzu’s judgements on intelligence were by no means isolated examples of the wisdom of the Orient. A century later in India, Kautilya, the nom de plume of the Indian statesman and philosopher Chanakya, chief adviser and Prime Minister to Emperor Chandragupta, first ruler of the Mauryan Empire, was explaining how collecting intelligence was the best way to secure the state and to disrupt an enemy. Kautilya, often called the ‘Indian Machiavelli’, had uncompromising views on how to be a successful commander and also on the vital importance of good intelligence. In the fourth century BC he set them down in his Arthashastra, or testament.

We have plentiful other evidence of the importance of intelligence in antiquity. Mithridates, the youthful King of Pontus, is alleged to have spent no fewer than seven years of his adolescent exile wandering Asia Minor collecting intelligence in the bazaars and markets dressed as a camel boy or a merchant, before finally taking up his crown at the age of twenty-one. Then, at the head of a small but tactically competent army in 88 BC, he launched a devastating assault on Asia Minor. His commanders seemed to know every pass, every road in advance. More dangerously, they knew the identities of the disaffected, the traitors and the would-be turncoats who would rally to their side everywhere they went. City after city yielded to him. Within a few short years, Mithridates ruled much of Asia Minor; a resounding victory based on good intelligence.

Rome relied on spies from its earliest days. According to Livy, as early as 300 BC the Consul Quintus Fabius Maximus had sent his brother undercover, disguised as an Etruscan farm worker to infiltrate the ranks of their Umbrian rivals and spy behind the enemy lines.

By the time Hannibal and his elephants had burst into Italy following their surprise march from Spain a century later, Rome found itself on the receiving end of someone else’s very sophisticated intelligence effort. Hannibal used agents widely to keep himself informed about Carthage’s arch-enemy, even infiltrating his spies deep into Rome itself. Well might Cato have thundered angrily from his seat in the Senate at the close of every speech, ‘Carthage must be destroyed!’ On the evidence, at least some of his distinguished Roman audience were probably paid agents of Carthage.

By the time Scipio Africanus took over the Roman effort against Hannibal in 210 BC he had learned that he must control his own intelligence effort. He even ordered some of his centurions to disguise themselves as ‘slaves’ to accompany a peace delegation into the enemy camp. As mere slaves they could wander around unchecked, while secretly reconnoitring the enemy positions, strengths and numbers.

Unfortunately, one of Hannibal’s Numidians recognised one of the undercover soldiers, Lucius Statorius, as a centurion in the legions. Scipio denied the claim, promptly ordered the accused ‘slave’ to be publicly flogged as a demonstration of his servile status. The unfortunate Statorius endured his painful ordeal, doubtless consoling himself by reflecting on the republican virtues of fortitude and stoicism demanded in the service of Rome. His cover held. The dubious Numidians were convinced. No self-respecting Roman centurion would ever tolerate or endure such a public humiliation. Sometimes the price for securing intelligence can be a painful one.

By the time of Julius Caesar in the first century BC we can clearly see the first recognisably modern system of intelligence in action, although Roman intelligence organisations of the day were privately run, usually by the rich: senators, merchants and politicians. For example, Crassus, Caesar’s senior partner and money lender, set up an intelligence network covering the whole of the Republic, designed to warn him of every development, whether a threat to Rome or, more importantly for the richest man in the city, any shift in the market forces that governed the Republic’s far-flung trade.

Caesar developed this into the first real ‘national’ intelligence system. As a successful soldier he realised the importance of timely, accurate information on his enemies and the need for fast, secure communications to keep his own plans secret. His Gallic Wars contains numerous references to intelligence collection, the most notable being the reconnaissance of Britain by his agent Gaius Volusenus in 55 BC, just before Caesar’s landing in Kent. However the organisation remained essentially Caesar’s, not Rome’s. Intelligence, even national intelligence, was still an individual venture, run by a successful general.

With Caesar’s own murder in 44 BC, the network passed on to Octavius, later Caesar Augustus. As the final beneficiary of the bloody civil war, Augustus was taking no chances. He rapidly drew all the existing military and diplomatic intelligence into his own hands and established an empire-wide network of communications, the cursus publicus, which was to become the core of Rome’s imperial secret service. With its thousands of waystations along the network of main roads it allowed people, taxes – and intelligence – to move swiftly. As Suetonius explained:

To enable what was going on in each of the provinces to be reported and known more speedily and promptly, he [Augustus] at first stationed young men at short intervals along the military roads, and afterwards post-chaises. The latter has seemed the more convenient arrangement, since the same men who bring the dispatches from any place can, if occasion demands, be questioned as well.

Using this network, Rome could now build on Crassus’s and Caesar’s private foundations to establish an imperial police and intelligence service designed to give early warning of any threats to the imperial throne and of corruption and unrest in the Empire. Trajan and his successors further refined this Roman secret service. They created a network of spies and informers that covered the civilised world to support their need for intelligence on imperial security and commerce, and to backup the legions guarding the long frontiers. These undercover secret agents were controlled by official intelligence officers: the speculatores, drawn from the intelligence officers of the legions and from the military procurement system, the frumentarii, whose ostensible role was to obtain grain for the army at the best prices.

The frumentarii were aided by their more sinister counterparts the peregrini, who were based in a special barracks on the Caelian Hill. As their name implies the peregrini acted as a roving secret military police and enforcement unit for the emperor’s orders anywhere in the Empire. The peregrini quickly acquired the happy reputation common to all such organisations: they were heartily detested by their fellow citizens.

The role of the peregrini as a kind of early Gestapo in the great cities started in the first century AD, with the Christians as one of their special quarries. Any political movement that advocated the equality of both slaves and freemen and the existence of a heavenly kingdom clearly posed a serious challenge to the Roman social order, and as such was considered deeply subversive of domestic security. The frumentarii were tasked to hunt down these dangerous revolutionaries.

Like most secret police and security services however, the frumentarii seem to have lacked imagination. One early Christian, Dionysius, evaded them by the devious expedient of hiding in his own house. This particularly cunning ploy confused the frustrated secret policemen as they scoured Rome for four days, fruitlessly searching for the tricky and dangerous dissident. Dionysius was eventually smuggled to safety by the Christian underground, thus proving the truth that a repressive regime will, as often as not, inspire an equally determined and resourceful resistance movement with its own intelligence networks and spies.

Despite this secret service reporting to the imperial palace, Rome’s intelligence service does not seem to have been especially efficient. Many Roman emperors fell to the assassins’ deadly stroke rather than peacefully in their beds. In the words of the distinguished historian of Rome’s grubbier past, Professor Rose Mary Sheldon, ‘Ironically, for all their reputation as empire builders, the Romans were never as good at watching their enemies as they were at watching each other.’

With the collapse of the Western Empire in AD 476, the surviving Eastern Empire remained to carry on the genius that was Rome. Byzantium’s administrative institutions, dynastic complications and fortunes for the next thousand years depended to a large extent on the competence or otherwise of Constantinople’s spy network; and the evidence is that Byzantium’s secret intelligence service was very busy indeed.

Byzantium’s long struggles were caused, in large part, by the constant threat from Islam and the heirs of the Prophet. Rampaging Islam absorbed a great deal of Constantinople’s effort and was covered, over many centuries, by a comprehensive network of informants, agents and spies in the Mohammedan camp. The best sources were the major trading routes and flow of commerce that flourished across the Near East and followed the great caravan routes that linked Byzantium to Baghdad.

Baghdad, in its turn, showed equal zeal in keeping an eye on its own internal pre-occupations. The Abbasid Caliph, Harun-al-Rashid, who ruled between 786–809 AD and has been immortalised in the One Thousand and One Nights, was said to have been prone to disguising himself and wandering the streets and souks of Baghdad to collect his own intelligence on what the people really thought about their ruler. ‘Uneasy lies the head that wears the crown.’ But using secret police to keep a close eye on the views of the faithful has remained a feature of Islamic states over the centuries, and still remains a priority for authoritarian Islamic leaders today, as the citizens of Cairo, Damascus, Baghdad, Teheran and even Istanbul know only too well.

Byzantium’s other great intelligence preoccupation highlights intelligence’s handmaiden: security. Constantinople’s principal counter-espionage problem was to prevent the secret formula of their ultimate terror weapon, ‘Greek Fire’, from falling into the hands of their Islamic foes.

Greek Fire appears to have been a viscous liquid mixture of naphtha, liquid bitumen, turpentine resin from pine trees, sulphur and quicklime. It could be dropped onto men or ships in clay pots, flung from catapults and, suitably diluted, even be projected from siphons or tubes like a flame-thrower. It burned with a tenacious, all-consuming flame and only sand, urine or vinegar could extinguish it swiftly; water merely helped the fire to spread. A small pot of Greek Fire hurled like a hand grenade into a cluster of Moorish warriors would ignite their flowing cotton surcoats and stick tenaciously to armour and skin, turning men into flailing human torches, burning in their own fat. It was an effective and terrifying ‘secret weapon’ – no wonder the Muslim armies wanted its recipe.

The Islamic agents of the Caliphate were as anxious to discover the secrets of Byzantium’s terror weapon as the defenders of Christendom’s Eastern Empire were to keep its secrets. For several hundred years, its formula was a Moorish intelligence priority. Astonishingly, and despite the systematic torture of thousands of captives over the years, and the lure of treachery and gold, the Byzantines appear to have kept their jealously guarded secret safe until about 1100.

The other major intelligence collectors of the Middle East at the time were the fanatical followers of Hassan-i Sabbah, known as the ‘Assassins’. They took their name from hashish, not because they were regular users of the drug (hashashin means hashish eaters), but because, in the opinion of the moderate Muslim world of the time, anyone who behaved quite as wildly as the Assassins must have been drugged up to the eyeballs

At his mountain stronghold at Alamut [in the Persian mountains north-west of Teheran] . . . the old man kept a number of young men from twelve to twenty, each with a taste to be a warrior, and told them stories of Paradise, as had the Prophet before him . . . When the old man wanted a lord killed, he would say to a youth, ‘Go thou and slay so-and-so and if you return my angels will carry you straight to Paradise. Even if ye die, my angels will convey you thence, to the realm beyond Earthly joys . . .

Marco Polo

Such a sect relied almost entirely on good intelligence and inside knowledge to focus its murderous activities. By the twelfth century, the ‘Old Man of the Mountains’ allegedly had a spy, informer or paid agent ‘in every lord’s tent’.

Confronted by this far-flung network, attempts to stamp the Assassins out proved fruitless. Their excellent intelligence service forewarned them of impending attacks and allowed them to stop any threat well in advance. When, for example, Sindjar, Sultan of the Seljuks, decided to move against the order in the tenth century, Hassan-i Sabbah ordered his agents to ask the Sultan to desist – or else. Sultan Sinjar ignored the approach and the Assassins resorted to direct action. One morning, the closely-guarded Sultan awoke to find an Assassin’s dagger thrust into the floorboards alongside his bed. As the shocked potentate was coming to terms with the implications of this rude awakening, a slave entered carrying a message. It read,

Were I not well inclined toward Sultan Sindjar, the hand that planted my knife in the floor would have plunged it into the sultan’s bosom. Let him know that I, from my mountain far off, guide the hands of those who surround him. Peace be with you . . .

But a new threat was looming, one that relied on intelligence to grease the slipway of its victorious march through Asia. Fresh from his victories in the east, by the late 1230s the Great Khan’s army was on the march west, looking for new conquests in Europe.

By 1241, when the Mongol Horde was poised to launch its assault on Eastern Europe from the Donets Basin, Yelii Chucai was the principal intelligence coordinator for the Mongol Empire. For over a decade, his agents had penetrated virtually every European court and major city and discovered not only the Europeans’ capabilities and intentions but also their fears. To the Khan’s intelligence service, Europe was an open book. Chucai, along with Subutai, the great Mongol general, appears to have been running a remarkably integrated – and modern – intelligence operation, building up a detailed picture of their next conquest and running advance spoiling operations against potential adversaries in Europe.

The papal legate Fra Carpini was sent as an envoy to the Mongols. From his chronicle we can piece together the basics of Mongols’ intelligence system. First, sometimes years ahead, came the ‘merchant spies’, always travelling in pairs and reporting back, diligently hunting for every scrap of basic information on the potential victim: topography, climate, roads, bridges, cities, the political and demographic make-up of the region and the loyalty of its tribes.

Inevitably, military forces, weapons and fighting ability figured large in the Mongols’ plans. It is reasonable to conjecture that such low-key but pervasive penetration of the enemy was reported back to the Mongols’ commanders (and, ultimately, to the Great Khan himself) by means of the commercial codes used by merchants travelling along the caravan routes of Asia.

Genghis Khan’s code of law, the Yassa, was a complex synthesis of tribal laws and customs. Of the twenty-two articles that we know of, two in particular reinforce the Khan’s pre-occupation with intelligence:

. . . send out spies and bring in captured informers who must be questioned and made to give information that we can check off against the spies’ reports . . .

and, as a second order prescribes,

Spies and false witnesses are to be condemned to death . . .

For the Mongols, intelligence was a priority – by law.

Genghis Khan and Sun Tzu’s thoughts and observations have stood the test of time. They remain proof that intelligence has always been crucial: not just for survival, but for battlefield success and political power.

The Catholic Church’s intelligence service rested on four principal pillars: the power of the confessional; a monopoly on literacy and learning; good communications; and the Inquisition. The terrors of hell and the powers of the priests enforced obedience and, more importantly, through the all-important confessional, they encouraged timely information on man’s temporal intentions that could be used for the benefit of Holy Mother Church.

The sacrament of confession and penance rested on the simple theory that sin – except for mortal sin – could be forgiven after it had been confided to an ordained priest. This also happened to be a useful tool, both for collecting information and for social control. The confessional heard everything and forgave much. This marvellous intelligence apparatus pervaded the Church from top to bottom. To hold back was to risk damnation and so, either through faith or sheer superstition, for hundreds of years, millions of people, from peasants to princes, poured out their innermost secrets to Catholic priests across Europe. The confessional was quite simply a fabulous information gathering system.

Priests were supposed to keep the secrets of the confessional, but the system had an inbuilt flaw: the priests themselves had to confess. Thus guilty knowledge would be passed up the Church’s hierarchy until it reached Rome itself. It was not supposed to happen, but it would be naive to pretend that it did not. Moreover, it flies in the face of all our experience of human nature and the way hierarchical organisations operate, to pretend that confessors did not curry favour by warning their bosses of impending trouble or heretical ideas.

For example, in about 1550, the Archbishop of Milan gave clear orders to his clergy that they were to reveal the names of any heretics or suspected enemies of the Church obtained through confession in order to collect intelligence on Rome’s Protestant foes. It would fly in the face of reason to suppose that it was the only example over hundreds of years. The main beneficiary was the Inquisition.

By the early thirteenth century, the need to stamp out opposition to the authority of Rome and uncover ‘heresy’ led to the formation of the Inquisition by Pope Gregory IX in 1231. The teachings of Christ himself were even quoted as the basis of the Inquisition’s most feared practice, burning at the stake: ‘If anyone abide not in me he shall be cast forth as a branch and wither: and they shall gather him up, cast him into the fire, and he burneth.’

The first task of the Inquisition was to collect intelligence on the faithless. For this, the Church was well equipped with its new armies of friars: chief among them being the Dominicans, whose satirical nickname Domini Canes (Hounds of the Lord) sums up their task. By the year 1250, the secret service of the Church was no longer playing a passive collection role. Its ‘hounds’ were actively engaged sniffing out valuable intelligence.

The most notorious and most successful achievements of the Inquisition were in Spain, where the Inquisition’s intelligence process was surprisingly modern. Spies, agents and informers built up a comprehensive ‘database’ on likely suspects long before the Inquisition proper made their entrance. The Inquisition’s advance guard of friars would then arrive and preach a list of heresies. They would then offer an ‘Edict of Grace’ – effectively an inducement to denounce others. If the faithful came forward and confessed all, then their sins would be pardoned – for a price. This was used to defray the Inquisition’s costs.

Hardly surprisingly, a blizzard of self-seeking denunciations and confessions usually assailed the Inquisition when they rolled into town. The Inquisition’s police state was a chilling portent of the excesses of twentieth-century secret police, and often more efficient.

Further to the east, scheming Venice survived – and thrived – on a mercantile society deeply rooted in international trade. International trade meant money; and making money demands good information and intelligence. From 1250 onwards, Venice set up probably the finest intelligence system in the world, for its day. Although it was primarily commercial, Venice’s prosperity and national security were indivisible from her market intelligence. The markets in their turn needed every scrap of political and military intelligence in order to operate for the greatest benefit of the oligarchy and citizens of Venice.

Although Venetian secrecy has hidden many details across the years, we have a unique mirror image of Venetian intelligence methods and an insight into just how they worked from the detailed records of one small city-state. Dubrovnik, or Ragusa, was an independent city state only 250 miles to the south, on the other side of the Adriatic. It prospered in the shadow of Venice during La Serenissima’s century of greatness and modelled many of its institutions on its large and dangerous neighbour, albeit on a much smaller scale.

Ragusa’s security policy relied on a unique doctrine based on intelligence alone. From 1301, the Ragusan governing council formally decreed that,

for the fortification and security of the City . . . to choose good and competent men to explore where they consider best, both inside and outside the Republic, all information and inform the Prince as necessary for the good and prosperity of the state.

One of the prime early targets was Venice itself. By 1348, we see the Ragusan Senate ordering the Council to assign ‘5 men of knowledge and wisdom to enquire and check on intelligence from Italy and Serbia’. Ragusa’s main agents were, like Venice’s, ‘consuls’ or ambassadors sent far afield with specific orders to report everything they saw or heard. Between 1250 and 1590, the number of deployed intelligence service agents abroad on the City payroll rose from three to over forty, all reporting back to Ragusa’s senate. Intelligence was the Ragusans’ key to survival.

Like Venice, Ragusa had to keep a close eye on the Ottomans, and often behaved as a vassal state. We know that her ambassadors to Istanbul/Constantinople rotated every two years and had written orders instructing them: ‘. . . if something happens you consider important, we order you to inform us and not to spare horse couriers, for we shall pay them. Be cautious in sending them . . .’

Ragusa’s intelligence collection priorities went much wider than mere conventional spying and diplomatic reporting however. Like Venice, as early as the sixteenth century, the Ragusan Senate realised the value of the primitive news-sheets then beginning to circulate. Within the next hundred years Ragusa was formally buying, and using as an ‘open source’ of intelligence, the Gazetto di Toscan, the Gazette de Leyde from the Netherlands and inevitably, Venice’s own very first news-sheet, the Notizie del Mondo. Like some latter-day Special Branch, Ragusa subscribed to and supported her adversaries’ newspapers.

With the coming of the printing press, economic intelligence became more open and systematic. Venetian agents were collecting market intelligence at the great European trade fairs from the mid-1400s onwards.

By 1540 a recognisable commercial intelligence system based on the flow of trade and the markets of Europe was firmly established, certainly in Antwerp, where Venice’s agent there, van Bombergen, had a clear brief not only to buy and sell items for his Venetian masters (he was on commission) but was also tasked to report formally on key items of market – and all other – intelligence. Such an intelligence network was far too important to be left to mere merchants in such troubled times. As the political upheaval of the Reformation exploded through Europe, the spy masters and intelligence collectors of the various courts and governments began to realise that, in this emerging network of money, market and commercial intelligence, there was an intelligence system that would work for their national survival too.

The English courtier and bureaucrat Francis Walsingham was a dedicated Protestant in the service of Queen Elizabeth. In his early twenties, during the reign of the Catholic Queen Mary, he had been forced to flee abroad, where he had learned the arts of intrigue as practised by the Venetians – the skilful use of diplomacy and secret intelligence to manipulate politics, trade and international influence. Sir Francis Walsingham had actually witnessed the massacre of the French Huguenots in Paris on St Bartholomew’s Day, 1572. He genuinely feared that if the Catholics ever took power again in England his own neck would be on the line.

Once the new Queen’s reign made it safe to return to London, Walsingham’s quiet, discreet efficiency made him a natural choice as the principal agent runner for Robert Cecil (later Lord Burghley) Elizabeth’s principal Secretary of State.

A vicious undercover war was raging between the English secret service and the Jesuits, a fanatical new order of Catholic ‘Soldiers of Christ’, swearing direct allegiance to the Pope and the spearhead of Rome’s Counter-Reformation. If the Pope could get Queen Elizabeth killed, then a Catholic regime could be re-installed in England. England desperately needed intelligence on the Vatican’s plots. Walsingham put together a shadowy and highly effective network of agents.

In 1583 Sir Edward Stafford was appointed English Ambassador to Paris. He lacked any background in diplomacy and was soon in trouble, short of money and floundering at the sophisticated French Court. The secret compromisers of the day moved in to snare the unwary envoy. By 1585, de Mendoza, the Spanish Ambassador in Paris triumphantly informed Madrid in a secret letter that he could bribe the British Ambassador into showing him ‘Secret Dispatches from London’, adding, ‘this ambassador is much pressed for money’. A delighted Madrid promptly authorised a bribe of2,000 golden crowns.

But Walsingham already knew all about the Ambassador’s treachery. He had been intercepting and reading the communications between Paris and Madrid and had a trusted informant on the inside, reporting on England’s hapless Ambassador. Moreover, he had also identified Stafford as being in contact with Catholic agents of Mary, Queen of Scots and was convinced that the Ambassador was a courier and go-between for Catholic plotters in England and France.

Walsingham could have used his knowledge to send the treacherous Stafford to the block – or worse, ‘hanging, drawing and quartering’, the usual sentence for treason in Tudor England. But Walsingham was a professional intelligence officer and agent runner. He decided instead to play the treacherous Stafford back to his unseen Spanish masters. Walsingham used the unsuspecting diplomat to unwittingly pass on disinformation and false intelligence back into Spain.

By 1586 Stafford’s value as an ‘unconscious’ double agent was further reinforced when he became aware of the Spanish plans for an impending invasion of England a good two years before the Armada sailed. Walsingham was informed immediately.

Walsingham’s intelligence network was well placed to frustrate Spanish intentions. Using his agents in northern Italy he got the Spanish government’s invoices and bills for loans ‘contested’ when they were presented for payment to the banks at Genoa (who were lending the capital to bankroll the Spanish project to invade England). Delay followed delay and, by autumn 1587, Walsingham could confirm that the Armada was held up for at least another year as his secret hand choked off the vital supply of Italian gold that Spain needed to fund their ‘Great Enterprise Against England’.

When finally the Armada did sail in the summer of 1588, two years late, surprise was lost from the moment the great expedition hauled anchor. Walsingham had deployed fishing boats off the Spanish ports and a network of harbour watchers to report the exact size, composition, course and speed of the approaching threat the minute it appeared. Harassed by the English fleet up-Channel, storm and shipwreck completed what the Navy Royal and Walsingham had already begun. Of the 130 ships and 15,000 men who set out, only thirty-five ships and 6,000 exhausted survivors finally limped home to Spain in September after their 3,000-mile voyage. It was Walsingham’s greatest military intelligence triumph.

Walsingham was a master of false-flag operations. Once the Catholic Mary Stuart fled Scotland in 1568 for exile in England, she posed a permanent threat to Elizabeth and the Protestant regime. Although Elizabeth kept her rival under house arrest, the Queen of Scots inevitably became the focus of the Catholic cause, and attracted plots and plotters. After twenty years of intrigues, Walsingham decided to trap her once and for all. His tool was a double agent tasked to run a dangerous sting operation.

A convicted crook called Gifford approached Anthony Babington, the leader of yet another plot to free the Scottish Queen, claiming to be a trusted courier of Catholic messages. Babington was fooled and began a correspondence with Mary. Walsingham read it all and baited a trap suggesting a plot to ‘rescue’ Mary and kill Elizabeth. Unwisely the Queen of Scots wrote back, agreeing to the attempt; in doing so she signed her own death warrant. Babington and his accomplices were arrested and executed; Mary Queen of Scots went to the block herself in Fotheringay Castle on 8 February 1587 – victim of another of Walsingham’s well-planned intelligence operations.

Christopher Marlowe was a protégé of Sir Roger Manwood, who had close links with Walsingham. It was almost inevitable that one of Manwood’s clever scholars from the King’s School in Canterbury would drift into the world of intelligence. Once Marlowe had gone up to Cambridge in 1580 he supposedly became a Catholic convert and was sent to a seminary at Rheims. However, his real mission appears to have been to smell out plots against Elizabeth and to identify Catholic agents under training on the continent. His job of identifying enemy agents done, Marlowe fled back to England and became a famous playwright.

Marlowe’s mysterious death in 1593 points to some hidden hand at work. On 18 May 1593, he had been arrested and hauled before a special court of privy councillors charged with the correction of slanders, heresies, libels and riots. They released him on bail, but on 30 May, he spent the day in the house of a Mrs Bull at Deptford with three known government spies, all paid agents of the Walsingham intelligence web: Ingram Frizer, Nicholas Skeers and Robert Poley. A struggle took place and Marlowe was stabbed in the head. Frizer was tried and acquitted after pleading self-defence. The affair looks suspiciously like a cleverly executed ‘wet job’ – the killing of a dangerous and garrulous double agent who knows far too much and must be silenced at all costs. Spying has always been dangerous work.

Armand Jean du Plessis, better known as Cardinal Richelieu of France, was heavily involved in intelligence and intrigue. His greatest triumph was using a Capuchin monk called de Tremblay (the original ‘éminence grise’) as an agent of influence to manipulate the Austrian Habsburgs during their long war with the Protestants in Germany. Richelieu’s agent network spread lies, false intelligence and used active measures to influence events all over Europe to keep France’s enemies fighting each other and weaken them.

His ‘cabinet noir’ of letter openers and code breakers alerted him to Buckingham’s abortive expedition to la Rochelle to help the rebellious Huguenots in 1627 long before it sailed to certain failure. The Duke of Buckingham’s