The spy novel was one of the great genre successes of the 1930s, its plots crammed with secret blueprints, psychotic enforcers, brilliant disguises and sudden betrayal. Fed on a diet of such stuff, MI5 and the Abwehr panted to get at each other. No imagination was complete without visions of a foolishly lust-stricken enemy officer being filled with Brandy Alexanders and having his ear nibbled, before he blurts out the technical specifications for a new magnetic rocket gun.

The actual experience turned out sadly different. The two dozen or so secret agents infiltrated into Britain early in the war by the Nazis, loaded up with radios and micro-dots, may have dreamed of careers filled with tension, glamour and crème de Cacao-based drinks, but it was not to be. As ever, it was the little things that gave them away – such as, in some cases, not being able to speak English. Very rapidly they were all imprisoned or executed having achieved nothing, all their training rendered completely irrelevant. The whole idea of spying in its purest sense (in other words, individuals scampering around rather than radio interception and the rest) just does not work very well. The second world war is bedevilled by spies, at great risk, trying to discover "secrets", but these "secrets" are so poorly defined as to be meaningless. A reasonably maintained wooden fence or a policeman on the beat, let alone a concealed safe in a locked office in a secure building (and which building, and which office?) would have been more than enough to see off some parachuted Nazi unfortunate who has been wasting months memorising Cockney slang, cricket scores and conversational gambits about the British monarchy.

Ben Macintyre's new book attempts to make the case for the importance of the British response to these German agents – the realisation that some of them could be "turned" to feed back to their Abwehr handlers misleading "secrets", which would spread chaos across the Reich. Run by a small, fiercely dedicated band in St James's (including Anthony Blunt, who passed everything on to Stalin), this became an ever more elaborate operation. The reader is introduced to a bizarre group of oddballs – some genuinely heroic, some venal, some quite boring – who become entangled in these schemes. Unlike some of Macintyre's earlier books, this one is cursed by its lack of focus, the well-known nature of much of the material and the sprawling cast, but he does everything he can to make it all entertaining.

There are, of course, a number of incidental pleasures. The information fed back to the Germans conforms to pre-war ideas of what spies ought to be like. So one double agent ("Agent Bronx") really does claim to be learning about troop movements from drunken intimates in casinos, others fabricate rumours of secret weapons (electrically powered canoes), and so on. That the Germans swallow this seems scarcely credible, but they do. The double agents and their controllers become ever more reckless, even inventing an absurd troupe of dedicated Welsh antisemites who stop at nothing to infiltrate top secret bases. There is an edge of hysteria to all this, and its success depended on a state of affairs on the whole invisible to the British at the time – namely that the Abwehr was riddled with incompetence, corruption and anti-Hitler sentiment and was sneered at and disregarded by Himmler and his associates. If it had been the band of forensically minded sadists assumed to be fitted as standard throughout the Third Reich, then the Welsh Nazi material would have been spotted a mile away. Luckily for the fantasists of St James's, the key figures – the Abwehr handlers in Lisbon, Madrid and Paris – were pathetic inadequates, fiddling expenses, toying with their secretaries and priding themselves on having successfully placed such – on the face of it – suspiciously successful agents right under Churchill's nose.

The cliché about spying is that it is like a fairground hall of mirrors, but reading Double Cross, such an attraction does not really seem worth spending good money to visit. Indeed, it is impossible not to feel that a helpful prelude to the second world war might have been a time-and-resource-saving Anglo-German agreement not to bother with spying. There is one magnificent moment in Double Cross. It involves a cunning plan in 1943 to trick the Germans into believing that a cross-Channel invasion was happening, so as to force them to divert forces from other fronts. To achieve this, the air was thick with deceptive radio traffic for the Germans to listen in to, obliquely yet insistently suggesting the coup to come. The double agents poured out nonsense about troop movements and torpedo-boats massing in Dover. The dummy spearhead of a handful of ships was then sent steaming towards France – but as it turned out the Germans had completely failed to be alerted by all this British trickery and did not react in any way at all. As Nazi coastal defences remained silent, and as there was no real plan to attack, the dummy spearhead had to turn sheepishly back to harbour.

The futility of all this is, in Macintyre's view, exonerated by the role of the double agents in the approach to D-Day. In what is meant to be a classic case of British ingenuity they duck and weave, creating an elaborate mass of misleading data for Hitler about armies in Scotland preparing to attack Norway, troops in the south-west planning to attack the Bay of Biscay, with yet more poised to fall on Calais. This last group was crucial, as Calais was, with its short sea-crossing, such an obviously good place to attack and the more German forces were stationed there, the fewer there would be able to cover the real landing sites in Normandy. So Agent Garbo, Agent Bronx and the rest laboured away with their coded telegrams and invisible ink, imagining troop manoeuvres and conveniently overheard military indiscretions. All this so confused the Germans that the double agents kept the D-Day secret safe.

But even the most cursory reader of Macintyre's account has to be chilled by the stupidity of it all. A counter-argument could be made that the agents themselves were the greatest threat to D-Day. With total air, naval and code-breaking superiority by the summer of 1944, the allies had effectively sealed Britain off. Left to their own devices, the Germans had no idea what was going on and were obliged to spread ever thinner forces across dozens of possible invasion locations from Norway to the Pyrenees.

The only substantial German source therefore was a handful of weirdos in St James's, prone to tiresome cricket analogies and themselves harbouring a Soviet agent. The Germans came extremely close to realising that the agents were fakes – the Abwehr had been shut down earlier in the year, its functions taken over by the ferocious Reich Main Security Office and its disloyal and corrupt officials arrested, tortured and killed. The greatest danger to D-Day (aside from bad weather) now became the loopy telegrams and invisible ink letters sent as part of "Operation Fortitude". If even one of the agents were to be blown (and one was teetering on the verge of giving herself away out of irritation that the British had accidentally killed her dog), then it would become clear that the one place they were not talking about was the real landing site, ie Normandy. It is a tribute to Macintyre's skill that the reader is haggard with anxiety as D-Day approaches – as, at any moment, one of these people could betray the great Anglo-American crusade. But I am not sure that this is the effect he wishes to achieve.

The outcome of the second world war was decided by many millions of people first making and then handling industrial equipment to kill one another on an inconceivably terrible scale. By the summer of 1944 the allies had used high technology to destroy the German air force and navy and to infiltrate German communications. The Atlantic wall – on which Rommel had spent so much time, resource and slave labour – was flattened in a morning and the allies began the grim process of liberating France. Right at the outermost fringes of the war a handful of people on both sides engaged in poorly supervised and deluded fantasies about spying, which were mostly pointless but sometimes had horrible results for the sometimes brave individuals involved. Double Cross is a good example of its genre, but it is unclear whether it is a genre that should thrive.

Simon Winder's The Man Who Saved Britain: A Personal Journey into the Disturbing World of James Bond is published by Picador.