I met Tony Benn at the end of 1985, and worked for him until his death in March 2014. When I started – as a temporary typist – I knew very little about him. He drank gallons of tea. He had renounced a peerage. He had opposed the Falklands war. He had defended the miners and the Wapping printers. In those days Tony resembled a restless caged animal, pent up in his basement office. The man who nearly became prime minister had gone from hero to almost zero; he had undergone a sustained mauling by press and broadcasting, which took its toll on his health and his personal life.

He had kept diaries on and off since he was a boy, and it was the publication of the diaries, the first of nine volumes in 1987, that gave him a new lease of life and turned his office into a hive of activity, with young “interns” from school or university (including Ed Miliband) queueing up to work for him.

Edward (as we knew him) was very sweet – modest, clever and quick. I took him, and the other interns, under my wing as I set them to work indexing Tony’s extensive archive of paper, audio and video records. My role morphed into editing the handwritten and transcribed diaries: Tony marking up one copy and me a second, then resolving our differences. Only once or twice in the ensuing years did I flounce out of the office.

Tony’s public persona when I first knew him was, perhaps not surprisingly given the history of the Labour party in the 1980s, rather puritanical and humourless; but privately he was very mischievous, and often childlike in his sense of fun. As a boy he was not allowed pets, and when I heard this, I gave him a clockwork ginger kitten, which would walk a few steps, then sit down and miaow pitifully before taking a few more steps. He loved this toy and would entertain passengers on trains or staff at the House of Commons by letting the kitten loose, to gales of laughter. The combination of political bogeyman and innocent joker was irresistible. In 1990, he decided it was time that Emily Wilding Davison was celebrated in the Commons, and he and fellow Labour leftwinger Jeremy Corbyn went into the broom cupboard in which she had hidden during the campaign for women’s votes, and screwed a brass plaque to the inside of the door, much to the disapproval of Black Rod and the Serjeant at Arms.

Tony wrote in his diary when he was 70: “I’d like to throw a light on the future, the next century … but I’m a bit nervous that I would not have the intellectual capacity to produce anything good. That’s why I tend to stay in the old grooves.” In compiling an anthology of his best speeches, interviews and articles, I rediscovered the power and originality of his earlier work. He wrote for the Guardian during the early 1960s (when he was out of the Commons, fighting to disinherit his peerage), and produced some excoriating journalism, including the condemnation of “the shoddy acts of state that pass for a policy” in South Africa; and a description of the hanging of Russell Pascoe in Horfield prison in December 1963 while “the centre of Bristol was ablaze with twinkling lights and full of shoppers carrying their parcels”. It would have convinced any doubters of the wickedness of capital punishment.

In fact, despite his fear of being “stuck in the old grooves” the last 10 years of Tony’s life produced a fresh bout of creative thinking; he argued the case for the legalisation of drugs; for the House of Commons to become an English parliament with the House of Lords acting as the UK federal chamber; and for an end to drugs testing on animals. He was an ardent vegetarian. He was also the world’s greatest junk-food fan. Burgers and milkshakes were his staple diet until he gave up meat, and then he turned to pizza and ice-cream for the rest of his life.

His children and his carers would stock the fridge with tempting gourmet meals to no avail.

By the time I worked for Tony, he was the Labour MP for Chesterfield – a mining town that had offered him a political rebirth in 1984, and had been devastated by the miners’ strike of 1984-5. I have dwelt on his idiosyncrasies and mischievousness because that is what I most liked and remember about Tony, but when he sat every Friday in his constituency office at the Labour Club in Chesterfield I witnessed at first hand a professional at his most brilliant. For a few weeks I filmed on a hand-held video camera the cases that he – as many other conscientious MPs – had to deal with in the course of a surgery. Tony would occupy the upper room of the club, while a queue of constituents would wait in the hall outside. Among them on one particular Friday, there was a young man with a plastic bag full of documents of Kafkaesque complexity; a woman, accompanied by an older man (her father? uncle? husband? boyfriend? – it wasn’t clear) with a bogus story of being robbed and destitute; a retired couple who wanted to move because their life was being ruined by youngsters who trashed their front garden. Tony would cut to the heart of a long and often convoluted tale, like a hot knife through butter, to deal with the constituent’s problem – or help him or her to realise that they were the problem. I never saw him intimidated, bored or angry during these Friday sessions. He retired in 2001, free from constituency and parliamentary considerations, leaving a priceless social archive.

In 2010 I was contacted by a young film director, Skip Kite. Like Tony, he was an avid pipe smoker and that was perhaps why I was persuaded (where other filmmakers had failed) to arrange a meeting between them a few days later. Skip brought with him to Tony’s house a book of John Betjeman poems, which he read while they puffed their pipes and Tony nodded off, having had a stroke from which he was still recovering.

The three years that it took for Skip to make the resulting film – Tony Benn: Will and Testament – were dominated by Tony’s ill health, but the project revitalised him. When he was later taken to the Chelsea and Westminster hospital after another collapse, I received a stern email from his consultant, Michael Pelly. “The patient has disappeared from the ward with his friend. He has missed a physiotherapy appointment.” Skip had helped Tony “escape” to a coffee shop next door, where they were found at a table on the pavement, smoking and surrounded by students. He was in his element.

Ruth Winstone is the editor of The Best of Benn: Letters, Diaries, Speeches and Other Writings published by Hutchinson at £20 RRP (hardback). Click here to order a copy for £17. The film Tony Benn: Will and Testament is out now on DVD.