'Every journalist who is not too stupid or full of himself to notice what is going on knows that what he does is morally indefensible," wrote Janet Malcolm, in the famous opening lines of her 1990 study The Journalist and the Murderer. "He is a kind of confidence man, preying on people's vanity, ignorance, or loneliness, gaining their trust and betraying them without remorse." The lines are arresting, and while not entirely true – not least because of a kind of glibness – they do get at something true about the often uncomfortable transaction that is the telling of someone else's life story. "Like the credulous widow who wakes up one day to find the charming young man and all her savings gone," she continues, "so the consenting subject of a piece of non-fiction learns – when the article or book appears – his hard lesson."

And so, often, does the journalist. On Wednesday Penguin published, in the US, The Mockingbird Next Door, a memoir by journalist Marja Mills of her friendship with the author of To Kill a Mockingbird, Harper Lee (it will appear in Britain at the end of this month).

Malcolm was writing about a case in which journalist Joe McGinniss befriended a murderer and convinced him to talk by claiming, among other things, that he believed in the man's innocence when actually he was convinced of his guilt (incidentally, he later ran into trouble again with a biography of Sarah Palin). Mills is nothing like as unscrupulous or faithless: The Mockingbird Next Door is on the whole a warm portrait of a place (Monroeville, Alabama) and two people (Nelle Harper Lee and her sister, Alice) at a golden moment late in their lives. Although it covers all the bases – childhood, young womanhood, family, relationships, character strengths and flaws – it is not prurient, or given to hunting down answers where they are not offered. "Did she date?" Mills asks each sister at one point, curious about the fact that neither ever married. "A little," came the answer, from each. "And that was that." There is much space given to process – how and when she approached them, and why – and mostly the proceedings feel honourable.

And yet. There is little surprise, to the reader, that news the memoir was being written was greeted in April 2011 with a categorical statement from Harper Lee: "Contrary to recent news reports, I have not willingly participated in any book written or to be written by Marja Mills. Neither have I authorised such a book. Any claims otherwise are false." Mills counters that the letter was written after  Harper Lee suffered a serious stroke, and has released a May 2011 letter from Alice: "Imagine my shock when I began to read and get clear about the statement … Poor Nelle Harper can't see and can't hear and will sign anything put before her by anyone in whom she has confidence. Now she has no memory of the incident." This week Harper met the actual publication of the book with a clear reiteration of her position, and an elaboration: "Miss Mills befriended my elderly sister, Alice. It did not take long to discover Marja's true mission: another book about Harper Lee. I was hurt, angry and saddened, but not surprised. I immediately cut off all contact with Miss Mills, leaving town whenever she headed this way. I understand that Miss Mills has a statement signed by my elderly sister claiming I co-operated with this book. My sister would have been 100 years old at the time."

It should be pointed out, in the interests of fairness, that this latter suggestion of incapacity is somewhat belied by the fact that Alice Lee was still practising law at 100, prompting various gobsmacked newspaper profiles at the time. Nevertheless, Harper Lee's feelings seem clear enough, do they not? "I think the letter from Alice to me speaks to the conditions under which the 2011 letter was written," replies Mills, who spoke during her book tour of the US, "so I question that Nelle really wrote the letter that was released in her name this week."

Harper Lee stopped giving interviews in 1964, and is well known, along with a handful of other writers – JD Salinger, Thomas Pynchon, Emily Dickinson, Cormac McCarthy – as a stalwart of guarding her own privacy. This, of course, gives the book, however quiet its tone, a massive frisson. It also plants a seed of doubt – it contains much detail of how it all started with a commission for a newspaper feature, how Mills expected nothing, how Alice invited her into the house then Nelle, intrigued, dropped round to her hotel to say hello – and cannot answer the silent question in the reader's mind: can it really be the case that someone who once said that their answer to any request for an interview was "hell, no" has suddenly let down their defences so completely?

Then again, people are complicated, and they change; they are reachable at different points in their lives, in different ways. And there is the question of friendship. After the newspaper article was published, the sisters seem to have reached out to Mills, who moved from Chicago to Alabama and lived next door to them for 14 months. At the height of her involvement with the Lees, Mills regularly went driving around Alabama with Harper, as well as accompanying her to McDonald's for coffee, to the laundromat, to feed ducks, and even to exercise classes. She spent precious time at their house: "They were reading peacefully, companionably, as they did so many evenings. Routine for them. Magical for me."

She is not the only chronicler of an artist's life to have arrived at her story through assiduous friendship. In 2013, for instance, Geordie Greig published his account of Lucian Freud, another categorical life-writing refusenik who once sent an East End gangster round to a putative biographer's house to make his point. Greig literally moved in – to the basement of the building in which Freud worked. (McGinniss moved next door to Palin.) He dropped round with copies of the Evening Standard when he was editing it; he brought his children to breakfast with the artist. He manipulated Freud into being photographed by asking him to be in a frame that was meant, ostensibly, to focus on Freud's oldest friend, Frank Auerbach. One reviewer noted a "forced-entry feel to much of the book, [which] adds to its sense of a private life, once guarded with menaces, now exposed".

In Mills' book, friendship allows not only access, but a structural dodge: she has written a memoir – about the process of research and discovery, about her life and affliction with lupus (about which the sisters were extremely kind; another slight unease, because, as Alice Munro once wrote, "a sick person … emotionally, holds all the cards"), and the progress of their friendship. It is not a biography – although, in fact, it does a great deal of careful biographical work. Of course, says Michael Holroyd, author of biographies of Lytton Strachey, Augustus John and Bernard Shaw, "had [Mills] written a memoir without such an illustrious neighbour, it would not have been published. [But for a memoir] she doesn't technically need permission."

Friendship – especially, perhaps, the informed, imbalanced friendship of fandom – blurs the lines in all sorts of ways. It invites confidences, flatters a certain kind of understandable vanity. "People like to confide," as Louis Menand once wrote in the New Yorker. "They want their stories told, and they somehow persuade themselves that in the right, sympathetic hands … readers will appreciate the challenge, the complexity, the sheer human variousness of what it is like to be them."

With the Lees there was also, Mills argues, the issue of age. Their stories had not been told, and they – in particular Alice, who had made a study of her family and the history of the area – wanted to tell them. She notes how close friends worried about how "the tangle of myths and half-truths [has] flourished amid Nelle's decades-long silence … 'When she and Alice go,'" said one friend, "'people are going to start "remembering" things as they didn't happen, or outright making things up, and they won't be here to set the record straight. So keep taking notes, girl.'" Mills says now that "Nelle (Harper) Lee was often my guide … Once she and her sister Alice gave me their blessing to write a book, Nelle would often crook her index finger at me at an especially interesting moment in conversation and say, 'Now you be sure to put that in your book.'"

It is interesting how clearly this – and, for that matter decades-long silence – speaks of the need for control. For Lee is a writer, even if she never published again after To Kill a Mockingbird; and writers, says Holroyd, never want to cede control of the narrative. Especially to lesser writers. There are exceptions to the latter, of course: Ian Hamilton on Salinger. He famously made the fatal error of sending his manuscript to Salinger for approval; Salinger sued for copyright because of the liberal use of quotation. Forced to rewrite entirely, Hamilton eventually produced, says Holroyd, a brilliant and more original work, In Search of JD Salinger. There was also Lytton Strachey on Eminent Victorians, and Virginia Woolf on Roger Fry – but Mills, while conscientious and clear, is no Strachey or Hamilton or Woolf.

Mills is a reporter, formerly of the Chicago Tribune. At the height of her research, she was interviewing Alice Lee, on tape, for hours every Sunday afternoon. So no hint here of another perennial challenge for the biographer, the constraining relative: a Dmitri Nabokov, a Valerie Eliot, or an Elsie Kipling, who not only demanded two-thirds of the proceeds from Lord Birkenhead's biography of her father, but also its copyright. When it was finally written, in 1948, she banned its publication entirely.

However, asked directly, Mills suggests that while Harper Lee seems to have directed operations to a certain extent ("Nelle and I would discuss which comments I wanted to use, and which experiences with her I wanted to relate. Often, her directive was to use my own judgment. To her credit, much of what she wanted off the record was to spare the feelings of a relative or a friend"), she was given no formal interviews. "She did not wish to be interviewed, and I respected that."

Perhaps this is why Lee now insists she did not give permission or authorisation – which is both understandable, and if true, rather naive. Surely she must have known, from the moment she knocked on the door of Mills' hotel room to see who this person was who had made such an impression on her sister, that everything would be noticed and recorded. That first meeting was strictly, absolutely off the record, and true to her word, Mills did not put it in the resulting newspaper piece. But she obviously took detailed notes, because it appears in the book with warp and weft, and with direct quotation. It is both an immediate scene, and a slightly troubling one. Fascinating, too, in perhaps unintended ways, and further illustration of why novel after novel – AS Byatt's Possession, Robert Harris's The Ghost, Jonathan Raban's Surveillance (about a reclusive writer tracked down by a journalist) uses the trope of the relationship between writer and subject.

Birkenhead's biography of Kipling was published after Elsie's death in 1976, when, as Ion Trewin, himself the biographer of Alan Clark, notes, "it was highly praised". Greig waited until after Freud had died. Was this something Mills ever considered? "No. They both knew at the time that they were coming to the end of their lives, and they wanted to make sure especially that their family stories weren't lost. It was a joke among them and our friends in Monroeville that when Alice passes on, it will be like a library burning, so great was her knowledge of their family, their region and their history. I believe they co-operated with me with posterity in mind." Asked if she spelled out what exactly she was doing, Mills replies, "they knew this was a memoir of my time spent with them".

But doesn't publishing now, when Alice is 102, and Harper 88, and both in assisted-living facilities, risk hurting people in old age who, as Holroyd puts it, "have not asked to be hurt"? Their claims and counterclaims about the other's frailty in their lawyers' statements raises that worry. Did she have any ethical qualms? "I do not. I was a witness to the joy, love and respect between the two of them as the three of us rode the back roads of south Alabama, and I saw how much they enjoyed sharing their stories with each other and with me." Though not, from Harper Lee's point of view it seems, with the rest of the world.