“You should write a book about writing this book,” Saul Bellow told his biographer James Atlas during one of the many meetings they had during Atlas’s research. Decades later – 18 years after the biography and 13 since Bellow’s death – he has done just that, recalling the ups and downs of their relationship, exploring his “agonised” ambivalence about the job he did, and reflecting on the dilemmas of the life writing trade, especially when the person you’re writing about is alive. “There will be tears before bedtime,” Atlas was warned about his Bellow project, and there were. Hence the motivation for this book – part mea culpa, part self-exoneration, part memoir.

Atlas dates his fascination with biography back to the two years he spent as a postgraduate in Oxford, where James Joyce’s biographer Richard Ellmann acted as his supervisor. Back in the US, he touted for reviewing work, got a lucky break or two, and – still in his early 20s – was commissioned to write a biography of the poet Delmore Schwartz, a cult figure whose reputation was then in decline. Atlas describes the transgressive thrill of reading Schwartz’s diaries – and the challenge of interviewing those who’d known him. Most were now old men and found talking about their vanished youth a painful business: “My presence couldn’t have been more unsettling if I’d worn a hooded cloak and carried a scythe.” But others seemed almost oblivious to him, happily revealing long-kept secrets.

The resulting biography owed some of its success to Atlas’s close identification with Schwartz – “his unrealisable expectations, his piercing loneliness, his book hunger, his literary ambition, his dread of failure, his sense of the sadness of life”. Such empathy can easily disappear in the course of research but it’s important to have it at the outset. When Atlas agreed to write a life of Edmund Wilson, he soon realised his mistake. Juicy anecdotes aside, why devote years to an antisemitic brute who boasted of how many women he had slept with and was dismissed as “a cold fishy leprous person” by one of his four wives? Atlas returned the advance.

Bellow had even more wives – five – as well as many lovers, but Atlas found nothing surprising about that. He could see how attractive Bellow was, physically and intellectually. And, as with Schwartz, he felt a strong sense of affinity – both he and Bellow had been raised in Chicago, albeit in very different neighbourhoods. He imagined the two of them becoming as companionable as Boswell and Johnson. Occasionally he chafed at the inequality: why did Bellow show no interest in him? And the way Bellow exploited others – whether in life or as fictional material – gave him pause. But for the most part he remained in thrall and willingly cast himself in the role of acolyte, hanging on to the great man’s every word and cajoling him to co-operate. It was a tricky business. Bellow was wary and quick to take offence. For all the time they spent together, Atlas never felt comfortable enough to call him Saul.

Biographers, Bellow once said, are “the shadow of the tombstone in the garden”, a bon mot Atlas has adapted (and watered down) for the title of this book. Interspersed with his accounts of pursuing his two quarries are tributes to biographers ancient and modern. Artfully arranged to interrupt and add suspense to the main narrative – how are things with Bellow going to pan out? – they also allow for some terrific stories and quotes, including John Aubrey on the capacity of a biographer “to raise the dead, so that the retrieving of these forgotten Things from Oblivion in some sort resembles the Art of a Conjuror”. The tricks required can be dirty as well as magical. Reductiveness is inevitable; subjectivity too; and you can never get it right. “If a man has not supped with his subject, he cannot know him well enough to write his biography,” Dr Johnson said. Maybe so. But with Bellow, Atlas found that knowing him was “a hindrance to understanding him”.

When his biography finally came out, several reviewers were struck by its ungenerous tone. Guiltily combing through the pages, Atlas marked with yellow Post-its all the sentences that he wished he had removed: the Twelve Errors, he called them, before reducing the number to six. Conversations with two of Bellow’s three sons – still smarting at their father’s neglect of them – reassured him he’d been fairer and less vengeful than critics alleged. Unimpressed by the argument that the people to whom Bellow gave pleasure, through fiction, far outnumbered those he hurt in life, Atlas speculates that Bellow needed pseudo-sons, such as James Wood and Martin Amis, whose love was unconditional. There’s something in this, though it’s trite to suggest that Amis was drawn to Bellow because his own father, Kingsley, was unappreciative and “defective”.

Atlas may have written only two biographies but he has thought long and hard about the pitfalls of the genre: he likens himself to “a car mechanic, repair manual in hand, peering under the hood of a steaming engine”. His book motors smoothly along: it is well written, wide-ranging, packed with gossip and has some of the best footnotes you’ll ever come across. But it’s hard to miss the suffering underneath – the story of a life made sadder by the writing of a Life.

• The Shadow in the Garden: A Biographer’s Tale by James Atlas (Corsair, £30). To order a copy for £25.50, go to guardianbookshop.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min. p&p of £1.99.