It’s that time of year: autumn, party conferences, and the season of mellow recall. Soon-to-be published political memoirs will be stacked up over the bookshops thicker than flights over Heathrow. Efforts from Ken Clarke, Alan Johnson and Chris Mullin, and the almost annual volume of Alastair Campbell’s diaries, are all pending. Memoirs from Nick Clegg and Ed Balls came out earlier this month. And now we learn – not with any surprise – that David Cameron is already at work on his.

Politicians are compulsive memoirists: by definition, they have the self-regard, the sense of history and the need to construct their own version of the recent past that makes not writing an account of their time in power a spectacular feat of self-denial. Yet hardly any of them do it well, or even readably. Between a title that is often beyond parody – Westminster, Wales and Water by Nicholas Crickhowell will be hard to beat – and the final words of thanks, there lies acre on acre of grey, self-serving prose.

Ever since 2010, when he became Britain’s 75th prime minister, David Cameron has been musing on his place in history, discussing events on a monthly basis with his old friend, the Conservative peer and Times journalist Danny Finkelstein. How could he not, living in the house in Downing Street occupied by every prime minister since Robert Walpole? How could he reverse a trend begun by Clement Attlee and followed by every prime minister since, to explain and justify all the controversies of their time, and occasionally enlighten their readers?

The Cameron memoirs will earn a £1m advance. But there are some rules he needs to follow if they are to be readable too. The conversations with Finkelstein – there are said to be 53 hours of digital recordings – will be a good basis for a thoughtful, coherent account of the great events of the Cameron years. Conversations rather than diaries make it more likely they avoid the first and greatest peril of the memoirist, which is too much detail. That speech at the CBI really, really doesn’t need remembering. On the other hand, the daily diary has an immediacy that the monthly review will lose.

There are details, and then there are details. The lothario Tory MP Alan Clark wrote his diaries like a novel, with an eye for significant detail, an acute ear for dialogue and a good analytical brain that somehow contextualised the most offensive observation. They are bitchy, astute and racy in a way that would be truly shocking from the uxorious Cameron.

From my experience writing a biography of Barbara Castle, what remained of the unpublished diary she kept on a daily basis before and after she became too preoccupied with her job as a cabinet minister is a vivid, angry marathon through the previous 24 hours. It is shot through with moments of rage, some obsessively bonkers conspiracy theories and the occasional heart-rending moment of self-loathing.

Towards the end, out of office after a brutal dismissal by Jim Callaghan, she mused enviously on the glow of power that lit up the woman who now looked likely to win the place in history as first female prime minister that Castle herself had coveted. Although her diaries are still a good read for keen students of the Wilson years (OK, not so many of those), little of this raw humanity survived the clean up the millions of words underwent before making it into print. The memoir she published later, Fighting all the Way, was downright dreary.

If Cameron wants to be read, he has to be prepared to offer more than the smooth exterior that has defined his public image as prime minister. When he first became an MP, he wrote a smart fortnightly blog for the Guardian. It’s a bit cocky for a former prime minister, but he could do worse than remind himself of how he wrote before he could take it for granted people would trouble to look.

He needs to be bold, and not too fastidious, about the privileged information that six years as an international figure has given him. The “Queen purred” moment after the Scottish independence vote, indicating how much she had worried about the result and hinting at her gratitude to her prime minister, suggests he knows exactly how to be indiscreet.

There are encounters – with Angela Merkel and Barack Obama – that one longs to know the truth about. If, without being transparently dishonest, he can dispel the impression that often lingered of a talented man working from the Tony Blair playbook, he will have done much for his reputation.

Chris Mullin, the former MP whose three volumes of diaries are all in print, one of them in its 14th edition (not a boast many political diarists can make), has some advice for Cameron. “Obviously he wants to get his version on the record,” he says, quoting Churchill on the advantages of writing one’s own history. “But he also needs to explain what motivates him, and why, and do it credibly. If he does that, then I think he’ll be worth reading. He’s an articulate and thoughtful man. I think he could do it.”

Mullin has another book, a memoir, Hinterland, out shortly. The difference between a memoir and diaries, he says, is a matter of time. “A memoir is reflective. Diaries that work are immediate, and honest, not based on hindsight and they need to be a little self-deprecating.”

The hot trend in political memoir is to abandon the chronological narrative in favour of a thematic account. Mullin’s Hinterland has chapters on episodes in his life (“Good Morning, Vietnam” about his journalistic career in the far east, “Loony MP backs bomb gang” about his campaign to free the Birmingham Six), and Ed Balls looks at his career through the prism of political dilemma.

Cameron’s life in politics is even briefer than Ed Balls’. He is only 49: the youngest former prime minister ever with, presumably, a whole other life ahead of him. That’s one drawback. Unlike other readable memoirists – John Major, for example – his early life lacks almost all hint of a compelling struggle or trauma. Mullin had been a journalist, campaigner and political activist for years before becoming an MP. Other politicians still read today – Roy Jenkins or Denis Healey, say – fought in the war; Cameron’s entire political career occupies little more time than the years Jenkins and Healey served before becoming ministers.

This accelerated living is challenging stuff for the traditional memoir. Viewed from the outside, it seems almost frictionless. The essay-crisis prime minister, notorious for living in the moment, has to find some rough edges to haul his readers in. It’s said that many of the conversations with Finkelstein revolved around Europe and the referendum. Hubris and nemesis. Maybe nothing more is needed.