The obvious, though almost entirely forgotten, truth about the deaths of the SIS cryptanalyst Gareth Williams and the British businessman Neil Heywood is that they were loved by their families and friends and this grief is no less than any of us would suffer.

But this barely registers when the public scents a real-life thriller – the appetite for intrigue and mystery quickly trumps any respect for the bereaved as we marvel at the artfulness of fate. Either the news is becoming more like thrillers or thriller writers are people of exceptional prescience who have somehow fracked the drama out of modern times with new precision.

In the Williams and Heywood deaths, it's really striking how they comply with the laws of mystery writing and, without changing a single detail, these exact circumstances could easily have been written into serviceable fiction.

In the Heywood murder, we have a police chief who takes refuge in the American consulate after alleging the murder; Heywood's main patron, an ambitious Maoist governor, destined for the top; and his wife, who is alleged to have had an affair with Heywood and who is also known to have imported hot air balloons from Somerset, as a means to conceal the illegal transfer of cash from China.

Like the Williams case, where the brilliant young victim was found locked into a red holdall, it is all too bizarre, much richer and more extravagant than most mystery writers have the capacity to imagine. Yet conventions are respected. In both stories, there is an unexplained death of a man who is engaged on secret, or at least highly secretive, work and who expires out of context – the old Harrovian Heywood was in China and Williams was killed in an anonymous flat in London, away from his home patch of Cheltenham. We are struck by their isolation and the lonely terror of their final moments, though to dwell on this is far from entertaining.

In both cases, families, colleagues and the authorities were slow to realise what had happened and it took time to establish that Williams and Heywood had in fact been killed. There were different degrees of cover-up in both stories and these suggest hinterlands of corruption and intrigue in Chinese and British authorities. We have no idea why these two apparently blameless individuals were murdered, exactly how they met their deaths or what was – or still is – at stake. In fiction, the whole thing would be wrapped up in under 500 pages by a heroically flawed, and therefore lovable, protagonist, who solves the murder and, by the by, exposes the conspiracy.

Here, the real world and fiction part company, because in neither case is there likely to be a completely satisfactory solution. Gareth Williams and Neil Heywood will be memorialised in headlines about the manner of their deaths (Spy in the Bag, Death in a Hotel Room), not by loving epitaphs. Life is rarely as neat or as just as fiction.

Yet there's good evidence that we desire a merger, or overlap, of reality and the fictive world – a demand for narrative in the news and a requirement that the works of the imagination should be based, as the movies say, on actual events. We want life to be melodramatic, because, despite impressions to the contrary and the hysteria of the war on terror, most human existence in the west today is peaceful and humdrum and bears out the theory in Steven Pinker's book, The Better Angels of Our Nature, that violence in history is in steady decline.

However, modern societies are beautifully set up to meet the plot demands of a thriller. We communicate with each other incessantly; we are at all times watched and photographed and traceable; and we move around a lot and are often far from home. (Incidentally, when I hear the words "a British businessman", my Pavlovian response is to expect trouble, as in any of the following verbs – murdered, jailed, extradited or disappeared.)

Reality has obliged us with high-concept melodrama, inconveniencing many a writer of hard-boiled dialogue and diabolic scenarios. "You couldn't make it up" is the phrase that springs to our lips as we contemplate jetliners piling into skyscrapers, US Navy Seals snuffing out Osama bin Laden – with the president and secretary of state on the other end of the live feed from Seal head cams – and the chief of the IMF ruling himself out of today's French presidential election by molesting a maid in a New York hotel that was miraculously wired for compromise.

You couldn't make it up, nor would you. As a writer of espionage fiction, I am not sure I would have dared to imagine Anna Chapman and the herds of Russian spies, grazing in America's suburbs and communicating with each other's laptops by coded wireless transmissions, while – and here is the bizarre and unimaginable part – conveying very little useful information to their spymasters in Moscow Centre. I would not have made up MI6's phony rock with its secret compartment, lying in a Moscow park, because it seems exactly like something a thriller writer would make up.

That story may provide a hint of what is going on. Not long after 9/11, I was on a book tour in Canada with a female novelist, who made a persuasive case that thriller writers had corrupted the imagination of mankind by enabling evil men such as Bin Laden to act out their extravagant fantasies. The constant drip of conspiracy and demonic plotting had changed us for the worse, she argued, rather like porn distorts sexual behaviour.

The first proper thriller writer was Erskine Childers, who in 1903 published Riddle of the Sands, a novel slightly less thrilling than Childers's own life, which was ended by a firing squad in 1922. But not before he shook hands with its members and told them: "Take a step or two forward, lads – it'll be easier that way." It was as though Childers had jumped out of one of his own novels. There was a confusion of author and character; his fiction was somehow decreeing how his life should end.

Ever since then, writers of intrigue have also been participants in the great game (Maugham, Buchan, Fleming, Greene, le Carré) and have benefited hugely from their experience.

Those who do not have that time in the trenches take care to stand close to power and politics and it would be odd if, over the course of a century of this relationship, some kind of exchange did not occur.