This is a golden age for spy biographies. Almost every month there is another fascinating portrait of an agent who fought or supported one of the great totalitarian philosophies of the 20th century. I think it is fair to say, though, that no matter how many of those stories get told, none will be as absolutely belief-beggaring as that of Richard Sorge.

Born in Baku, of Russian-German parentage, he grew up in Wilhelmine Germany, and served in the first world war. Injured on the eastern front, he grew progressively radicalised by the misery all around him, and became a communist. His is a story with uncanny parallels to Kim Philby, except he took the struggle to the streets for a while, even working as a miner in the Ruhr in an attempt to take the party’s message to the workers, before discovering his natural vocation as a spy.

Owen Matthews has tremendous fun with Sorge’s life, which is so packed with incident as to be barely credible. He was “an idealistic communist and a cynical liar”, Matthews writes. “A pedant, a drunk, and a womaniser. He was addicted to risk, a braggart, often wildly indisciplined ... He was a raging intellectual snob whose natural milieu was the casinos, whorehouses and dancehalls of pre-war Shanghai and Tokyo.”

Sorge did try to live in Moscow for a while, and married a Soviet woman called Katya, who spends most of this tale lamenting his absence. He had ambitions to be an academic, but his desire for excitement kept getting the better of him, which is how he ended up being dispatched to Shanghai, then a freewheeling city controlled by the colonial powers, packed with disreputable foreigners, and thus a centre for espionage. He created a spy ring, which produced valuable intelligence about the Chinese civil war and other topics, before being sent to the place that would make his name: Japan.

Tokyo was a very different prospect to Shanghai, with few foreigners and a suspicious government, so spying would require a high degree of mastery. Sorge, however, set to work with a will. He put together a network that penetrated into the very heart of the Japanese administration and was able to produce crucial information on the country’s intentions, resources and abilities. He dived into the German community, sleeping with its women, drinking with its men, and establishing a reputation as the best-informed foreign observer of Japanese politics.

He was, simultaneously, a member of the Soviet Communist party and of Germany’s Nazi party; special adviser to the German ambassador, and lover of his wife; outrageously indiscreet barfly in Tokyo’s louchest nightspots, and supplier of the most startling secrets to the Soviet Union’s spy agency.

These secrets were of existential significance. Sorge repeatedly warned Moscow of German plans to invade in 1941. Then, when that attack had come, he was able to accurately inform his masters that Tokyo’s priority lay to the south, that it would not attack Siberia and force the Red Army to fight on two fronts. This allowed Joseph Stalin to shift soldiers from the far east to shore up the desperate defence of Moscow, and thus to save the Soviet Union. Sorge is one of those rarest of spies, one whose information genuinely changed the course of world history.

Perhaps the most extraordinary thing about his tale, however, is the use that the Soviet Union made of most of the information provided by him: it ignored it. Although Stalin did act upon the knowledge that Japan was no threat to his eastern provinces, he took no action to respond to Sorge’s most important information, that Adolf Hitler was preparing to invade.

Sorge was part of a generation of committed communists who took lunatic risks to help Stalin understand the threat from the Axis powers. They were rewarded by being belittled, ridiculed and smeared. “You can send your ‘source’ from the headquarters of German aviation to his fucking mother,” Stalin scrawled on an intelligence report containing warnings from an agent in Berlin.

Sorge was eventually captured by the Japanese, and his network rolled up, to the embarrassed disbelief of his German dupes (including an unpleasant man known as the “butcher of Warsaw”, who had been sent to investigate Sorge and ended up as his drinking buddy). Moscow completely failed to respond to all of this. There was no attempt to rescue him or to bring him back to the USSR. His posthumous fame dates from the 1960s, when Soviet leaders saw a Franco-German film about him, realised how amazing he had been, and awarded him the country’s highest honour in a belated and inadequate recognition of his sacrifice.

The final chapter of this magnificently written book dramatically highlights the contrast between the lofty professed ideals of the Soviet Union and its squalid reality, along with the sad fates of those people unwise enough to trust the communist state with their lives. Stalin didn’t deserve Sorge, and these poor women deserved far better than Sorge too. An Impeccable Spy is packed with humour and insight and all served up with a rare lightness of touch. Ben Macintyre and John le Carré fans alike will find themselves very much at home.

• An Impeccable Spy by Owen Matthews is published by Bloomsbury (£25). To order a copy go to guardianbookshop.com or call 0330 333 6846. Free UK p&p over £15, online orders only. Phone orders min p&p of £1.99