You would think that the disappearance of the gravest threat in the history of humanity would bring a sigh of relief among commentators on world affairs. Contrary to numerous expert predictions, there was no invasion of western Europe by Soviet tanks, no escalation of a crisis in Cuba or Berlin or the Middle East into a nuclear holocaust. The cities of the world were not vaporised; the atmosphere was not poisoned by radioactive fallout or choked with debris that blacked out the sun and sent Homo sapiens the way of the dinosaurs.

Not only that, but (again, contrary to expert predictions) a reunified Germany did not turn into a Fourth Reich, democracy did not go the way of monarchy, and the great powers did not fall into a third world war but rather a "long peace", which keeps getting longer. Surely the experts have been acknowledging the improvements in the world's fortunes from a few decades ago.

But no – the pundits are glummer than ever. In 1989 John Gray foresaw "a return to the classical terrain of history, a terrain of great power rivalries … and irredentist claims and wars". A New York Times editor wrote in 2007 that this return had already taken place: "It did not take long [after 1989] for the gyre to wobble back on to its dependably blood-soaked course, pushed along by fresh gusts of ideological violence and absolutism."

The pessimism is bipartisan: in 2007 the conservative writer Norman Podhoretz published a book called World War IV (on "the long struggle against Islamofascism"), while the liberal columnist Frank Rich wrote that the world was "a more dangerous place than ever". If Rich is correct then the world was more dangerous in 2007 than it was during the two world wars, the Berlin crises of 1949 and 1961, the Cuban missile crisis, and all the wars in the Middle East. That's pretty dangerous.

Why the gloom? Partly it's the result of market forces in the punditry business, which favour the Cassandras over the Pollyannas. But mainly, I think, it comes from the innumeracy of our journalistic and intellectual culture. If we don't keep an eye on the numbers, the programming policy "If it bleeds, it leads" will feed the cognitive short cut "The more memorable, the more frequent", and we will end up with what has been called a false sense of insecurity.

The pessimism has been inspired by "new wars" involving guerrillas and paramilitaries that plague the developing world, symbolised by images of Kalashnikov-toting teenagers. It has been stoked by the widely repeated (and completely bogus) meme that at the beginning of the 20th century 90% of war deaths were suffered by soldiers and less than 10% by civilians, but by the end of the century these proportions had been reversed. It has fed on the claim that the world learned nothing from the Holocaust, and that genocides are as common as ever. And of course it has been redoubled by the threat of terrorism, which has been said to pose an "existential threat" to western countries, having the capacity to "do away with our way of life" or to end "civilisation itself".

Each of these scourges continues to take a toll in human lives. But it's only recently that political scientists have tried to measure how big a toll it is, and they have reached a surprising conclusion: all these kinds of killing are in decline. Battle deaths per 100,000 of the world population have fallen from 300 during the height of the second world war to the teens in the postwar years, single digits during the cold war, and less than one in the 21st century.

The deliberate killing of civilians has shown a similar bumpy yet downward trajectory. And other than in the war zones of Iraq and Afghanistan, deaths from terrorism in the past decade were far lower than they were in the 1970s and 1980s, with their hijackings and bombings by countless revolutionary fronts, leagues, brigades and factions. A mental model in which the world has a constant allotment of violence – so that every ceasefire is reincarnated somewhere else as a new war, and every interlude of peace is just a time-out in which martial tensions build up and seek release – is factually mistaken.

It's not easy to see the bright side in the world today, where the remnants of war continue to cause tremendous misery. The effort to quantify the misery can seem heartless, especially when it undermines claims that are serving as effective propaganda for raising money and attention. But there is a moral imperative in getting the facts right, and not just because truth is better than error.

The discovery that fewer people are dying in wars all over the world can thwart cynicism among compassion-fatigued newsreaders who might otherwise think that poor countries are irredeemable hellholes. And a better understanding of what drove the numbers down can steer us towards doing things that make people better off, rather than congratulating ourselves on how morally sophisticated we are.

• To join the discussion, and explore The Better Angels of our Nature in depth over the next two weeks, click here