Kate Summerscale was working on the obituaries desk at the Daily Telegraph when she came across the life of Marion Barbara Carstairs. Joe Carstairs, as she was known, was an eccentric heiress who earned the sobriquet "the fastest woman on water" for motorboat racing, before withdrawing from public life to become ruler of the Bahamian island of Whale Cay. She had more than 100 girlfriends, but was to remain devoted until her death to an outsized doll, whom she christened Lord Tod Wadley. So unusual and intriguing was this story, Summerscale found that she "wanted to fill in the delicate gaps that characterise obituaries, name the oddities suggested but left respectfully – or wryly – untouched". The result was her first book The Queen of Whale Cay (pronounced "key"), published in 1997, when she was in her early 30s.
Ten years later, The Suspicions of Mr Whicher or The Murder at Road Hill House, a minutely researched account of a real Victorian country house murder, written with the psychological acuity and suspense of a detective novel, won the Samuel Johnson prize for non-fiction in 2008 and was a Richard and Judy book club choice. Along with red-ink blood spots, the paperback cover is splattered with superlatives from John le Carré, Sarah Waters and Ian Rankin; Craig Brown described it as "nothing less than a masterpiece". It is the sort of hybrid, genre-busting bestseller that every publisher and agent dreams of signing up, and many journalists and historians dream of writing. With just two books, Summerscale pulled off the publishing feat of becoming a brand – "in the tradition of Mr Whicher" now appears on the back of other titles.
Her new book, Mrs Robinson's Disgrace: The Private Diary of a Victorian Lady, returns to the same era. Building on the success of Mr Whicher, which featured "the real-life Sherlock Holmes", the book is being billed by her publisher, Bloomsbury, as the story of a "real-life Madame Bovary": it's about a scandalous love affair between the wealthy Isabella Robinson and a dashing younger doctor, Edward Lane, and the highly public divorce that ensued. "The seed was planted" while Summerscale was researching Mr Whicher and came across the Robinson case in a book about Victorian crime and sensation literature, but felt "it was an endnote or an aside too far" in the Road Hill House murder. "For a long time I felt unsure as to whether there would be enough to say. It was such an obscure case; it was a big sensation for a few days, but the people involved are not well known, and there were no obvious repercussions."
But she was hooked enough to look up the original newspaper articles and found that there was a file on the divorce in the national archives. The discovery at the heart of the book, and the case itself, is Isabella Robinson's diary, into which, in the breathless, confessional style of popular novels of the period, Isabella poured her infatuation with her lover and her contempt for her boorish, avaricious husband, Henry Robinson. The diary was used against Isabella, "almost as if it were a character in the case", to protect the respectably married Edward by interpreting her words as the delusional fantasies of a middle-aged nymphomaniac; the staggering contortions performed by the legal profession and male players to uphold the double standards of the age give the book its wider resonance. "In their efforts to save Edward while condemning Isabella, the lawyers had come up with a sex act in which she had been able to indulge without the participation of any man" – the writing of a personal diary.
Summerscale describes herself as "a journalist playing historian, and then trying to convert what I've found into something that approximates a novel". As a child – her father was a diplomat and her childhood was spent abroad, in Japan and then Chile – she thought "writing novels would be the best thing ever to do". After attending Bedales boarding school, she studied English at Oxford, and became "more realistic about writing", going to Stanford in California to study a communications MA. Back in London, she began work as a subeditor on the Independent in its early glory days, when colleagues included Sebastian Faulks, literary editor at the time, and Allison Pearson.
But it was the obits job on the Telegraph that really shaped her as a writer: "It was a bit like being a biographer in brief – every day you'd write somebody's biography, writing in this very assured and often cheeky way about a completely different generation. The anonymity of the form gave you real freedom." It also gave her an appetite for foraging in the newspaper archives: "I loved getting the files down, forgotten fragments of people's lives – and they are fragments, because it is just the moments that they emerged into public view. It was fantastic fun to piece it all together and make stories out of it."
Written at the height of the golden age of biography, when career-long doorstoppers about eminent (often Victorian) figures were making biographers themselves names to be reckoned with in literary circles, The Queen of Whale Cay was a bold departure. Her publishers "made a virtue of its brevity and difference and oddity" by publishing it in a small format, quite unusual then. A year later, Amanda Foreman was to publish Georgiana, Duchess of Devonshire and tip the balance decisively in favour of breezy lives of less obvious subjects; this loosening of the biographical stays, combined with a vogue for "faction", paved the way for hybrid hits such as Mr Whicher.
After a brief stint as editor of the Independent on Sunday's review section, Summerscale returned to the Telegraph as literary editor in 1999. Landing her dream job "in a funny way spurred me on to leave", she says. "I had been doing it for six years, quite a long time in newspapers, and I didn't have any further ambitions for anything else in journalism." Shortly after the birth of their son, she separated from her long-term partner, a woodcarver and teacher, and while being "the main breadwinner" made the decision to resign scarier, the prospect of working at home was also a further incentive: "It was a bit of a now-or-never moment."
It was "one of those happy things – a gamble that paid off", she says, pouring tea at the large wooden table in the kitchen of the pretty West Hampstead flat where she lives with her son, now 10. What does writing a hit like Mr Whicher really mean? "It has been incredible", she says. "It is life-changing in the sense that it means I can relax and believe that I can make a living as a writer. But it is not life-changing in the sense that you suddenly have a huge amount of spare capital. No." She says it has been "chastening" to realise how hard it is for writers to get by, let alone get rich. The real luxury "is that you are doing something you love, you are creating. It is an enviable way to live your life."
She wanted to start work on a new book fairly quickly, to cement the idea that writing is what she does now, and that she wouldn't be a one-book wonder. "Whereas with Mr Whicher I felt I was working away almost in secret, it would just be what it would be, now there is a sense of expectation. But it was so much more validating than it was scary. It gave me confidence."
While Mr Whicher was acclaimed as a triumph of research and storytelling, there was a polite sniff or two from historians of the period that this was by no means an unexamined case. When she first came across it, she thought: "This is the most amazing story, how come we don't all know about it?" But then she discovered that a number of books had indeed been written about it already. Although she was initially discouraged – "I thought, I'm hardly going to re-solve the case. What would be the purpose?" – she couldn't shake off her fascination with the story, and with the detective Whicher in particular. Then she had "a moment of revelation", which was to give it a personality. "Nobody had written an account of the case in which he was a leading character. In real-life cases you hardly ever even see the detective. And in this case, you are not only aware of him as a player, but [that] his life was destroyed by it. He was my key. He was the thing that allowed me to walk in on the case through a different door."
The Robinson divorce case is much less well known or documented. Like that of Whicher himself, her work involves a lot of "playing detective, ingenious guesswork and trying things out". For instance, she couldn't find any mention of the Robinsons in the Scottish census of 1851, even though she knew they lived in Edinburgh at this time. In the court records she noticed that one of the witnesses was a nursemaid with the family, so she looked her up and found they had been misrecorded as Robertson. When she finally tracked them down, she had the added satisfaction of being able to stay in their house, now turned into a B&B. "Good stories tend to generate material and you have to have the faith that it will be out there somewhere," which means following a paper trail without knowing where it is leading. "It makes it exciting. I don't think I would take on a subject that I knew how the story ended. I want it to feel like an adventure while I'm doing it because that is where the energy comes from."
Does she ever come up against dead ends? Or worry that it is never all going to come together? "God yes. With all my books there's come a point where I've said to friends or my agent 'this just isn't going to work'." Focusing on small stories rather than great swathes of history gives her "the nerve" to tackle periods in which she isn't an expert. "It is all piece by piece and eventually I hope it does build up to some kind of portrait."
The crucial document, Isabella's diary, is missing; Summerscale thinks it was probably destroyed by the odious Henry, who discovered it while his wife was delirious with diphtheria. But thanks to the punctiliousness of the lawyers and the prurience of the press, who printed all the racy bits, Summerscale had access to the innermost thoughts of her subject. As with the discovery, fairly late into the research for The Queen of Whale Cay, of tape recordings made by Joe Carstairs in the 1970s, the voice of Isabella breathes life into the dusty historical documents and so into the book.
While Summerscale might rely on the resources of the amateur genealogist to structure her plots, she uses the sensibilities and techniques of the novelist to bring them to life. It was her love of Victorian literature that first attracted her to the period. But the books are not just informed by literature, they are shaped by it: Mr Whicher has the pace of a whodunnit and Mrs Robinson the tension of the courtroom drama. "Sometimes borrowing from novelistic genres helps in terms of thinking about pace and suspense and character, and setting – how much you need to know," she says. "All that is exactly the same, I should imagine, for a novelist or for somebody writing the sort of books I am."
Mr Whicher is steeped in references to contemporary novels such as Bleak House, The Moonstone and Lady Audley's Secret, in which the figure of the literary detective was emerging as a character in his own right. While there is a lineage from Wilkie Collins's Sergeant Cuff to Sherlock Holmes through to Miss Marple and Poirot, Summerscale sees Whicher as a predecessor to latter-day maverick sleuths in the more rough-and-ready mould of Rebus, Frost or Wallander.
Mrs Robinson can likewise be read as a "literary amalgam": in the writing of her diary, Isabella was producing a sensation serial of her own, while its discovery has striking echoes with scenes in The Tenant of Wildfell Hall and The Woman in White. And the disappointments and disillusion of life after marriage that the story lays bare are features of the Victorian marriage plot as it was being rewritten by women novelists such as George Eliot.
Of course, the 19th-century adulteress to whom Isabella has the closest affinity is the similarly bored and narcissistic Emma Bovary; Flaubert would have been writing his novel, published in 1856, during exactly the same period that Isabella was writing her diary. Both Isabella and Emma are anti-heroines, punished for their sexual transgressions, but unashamed of their desires or their hunger to experience more than was available to them as women of their class and time. "I was fascinated by the fact that Isabella had this moral code about doing the right thing by her children and by her lover and the friends who had been good to her, while not subscribing at all to the standard moral codes about being God-fearing and being faithful to her husband."
Leading biographers, such as Claire Tomalin, have led us to expect a degree of novelistic licence in modern life-writing; but how did Summerscale decide to include descriptions about the weather or a certain gesture of a character – details of which she couldn't possibly be certain? "I wrestle with things like that," she concedes, "with what's on the side of being generous to the reader because you are giving texture and life and so on and what's trickery and not-trueness. I don't want things to be made up because it undermines everything that's true. I've got to try and give it its integrity."
But equally, she is happy to leave gaps unexplained or to make clear when something is only supposition. "I don't think it does any damage, in fact it can enhance reading about something true, if there are occasional reminders of what we can't know and what's mysterious: it will just be for you as the reader to imagine or decide," she explains. "But you can't have too many of those, of that feeling, or the story disappears."
More tricky still, especially in a book that rests on determining the true psychological state of its subject, is intuiting what Isabella was thinking or feeling. The first draft, she says, "was more detached and more clinical about what we can and can't know", but gradually, as she got closer to the material and understood the context more deeply, she felt more sure of Isabella's internal life. However, she is always careful to distinguish interpretation from fact "because it does feel like some sort of violation if you are writing about real people, to appropriate them and say what they felt when you don't know".
As Mr Whicher proved, once she has disinterred these long-forgotten bodies, they assume an afterlife of their own. A postscript to the paperback reveals that readers sent photographs of Jonathan Whicher, the first images of him she had ever seen, and of Road Hill House, which seemed to suggest further mysteries. Summerscale hopes a picture of Isabella, whom she is sure would have been photographed, comes to light. There were pictures of Henry and other illustrations that she might have included in the book, but with the absence of Isabella herself she decided to leave appearances to the reader's imagination, just like in a novel.
Does she think she might write a novel one day? "I've never, so far, been able to write a novel from scratch. Somehow the research, the historical investigation and curiosity enables me to get over some barrier to writing. It provides me with the material, it fills up the blank page. Maybe if I come across a story that needs to be told fictionally rather than factually then that is what will happen. But I imagine that it will be as a result of coming across something out in the world."
Looking at the three titles together, she says, "if they succeed or fail, it is at the level of whether they work as narratives, not whether they are important, not whether they reveal something significant – all that is extra. It is just about the story."
Making sites popular is about more than Twitter and Facebook. Burgess Powers makes social: personal