A secret back channel, which allowed the government to communicate with the IRA, existed for 20 years until it was revealed in 1993:

It is very hard for democratic governments to admit to talking to terrorist groups while those groups are still killing innocent people. But on the basis of my experience I think it is always right to talk to your enemy. Luckily for this process, the British government's back channel to the Provisional IRA had been in existence whenever required from 1973 onwards.

When Stormont collapsed [in 1972] a diverse collection of home civil servants, diplomats and spooks was sent out to try and make sense of the place. One of these was a remarkable Secret Intelligence Service (SIS) officer, Frank Steele, a former colonial officer and travelling companion to the explorer Wilfred Thesiger.

Steele made it his job to get out into Catholic ghettos like the Falls Road in Belfast and the Bogside in Derry and to make contacts at all levels. Eventually he was able to get in touch with Provisional IRA leaders and suggest they come to London to see Willie Whitelaw [the first Northern Ireland secretary] in 1972. Steele was the first British official to meet the IRA.

When Steele left Northern Ireland in May 1973, his contacts were inherited by his successor, another SIS officer, Michael Oatley, a subtle and independent-minded intelligence operative who within a few months of his arrival managed to develop connections with and around the IRA leadership to see whether it might be encouraged in the direction of political activity. Cautious probing had produced three different potential lines of contact to the IRA.

The most promising, and the one he pursued, involved a Derry businessman called Brendan Duddy.

Duddy was very much an Irish republican, but he was a pacifist and a firm believer in dialogue. Referred to as "the contact", Duddy worked selflessly and at great risk to himself over many years to bring about a peaceful settlement in Northern Ireland. In 1991 it was on his initiative that a meeting between Martin McGuinness and Michael Oatley took place which reactivated the "link" and helped lead eventually to the peace process.

Oatley's initial contacts with the IRA through Duddy were cloaked in secrecy. Not even Merlyn Rees, the Northern Ireland Secretary, knew about them. Oatley informed Frank Cooper, the senior British official in Northern Ireland, of the existence of "the contact" and he in turn consulted John Hunt, the cabinet secretary, and Harold Wilson.

From a British government point of view there were no negotiations. Throughout the series of meetings which now occurred at ever more frequent intervals at a safe house in Derry, to which the republican participants were smuggled from across the border under the eyes of the British army, Oatley maintained that the purpose of the dialogue was not to negotiate, but to advise the IRA of what action the British government and security forces might take if there were a cessation of violence.

The talks continued for weeks, with the hawks on the IRA's army council sending ever tougher demands. One was that their leaders be allowed to carry handguns when they returned to their areas after being "on the run" in the south. The elaborate British response was to point out that, with the ending of violence, the army would withdraw from republican areas, senior IRA men might helpfully be designated as official points of contact for crisis resolution, and these contacts might then be given special passes which would make it very unlikely they would be searched if they stayed in their own areas.

They reached consensus on an indefinite ceasefire, but needed a cover story for why violence was stopping. It was provided by the parallel talks going on in Feakle in Co Clare with a group of clergymen in December 1974, and for nearly 20 years both sides kept secret what had actually taken place in Derry. The ceasefire itself stuttered on for some months and then collapsed in response to renewed loyalist violence. Oatley had, by then, left Northern Ireland but he deliberately continued his relationship with Duddy.

Oatley felt that the reliable and secret channel of communication was too valuable to waste, and without any authority to do so he agreed with Duddy that the new IRA leadership, with Martin McGuinness as a central figure, should be told that while there was nothing now to talk about the channel remained available against the day when there might be.

It was, metaphorically, contained in a locked box, the key to which was in the possession of Brendan Duddy and behind which, wherever he happened to be in the world, was Oatley, known to the IRA as 'Mountainclimber'. This arrangement gave Duddy continuing access. He worked diligently to maintain the relationship, meeting members of the leadership and providing political discussion and strategy papers in his bid to promote peace.

IRA leadership

For the next 10 years [after the first IRA hunger strike in 1980 when the channel was activated] Oatley remained in contact with Duddy, and Duddy continued his relationship with the IRA leadership. At last, early in 1991 he told Oatley that the IRA might now be ready to discuss a political way forward. He arranged for Oatley to meet Martin McGuinness in Derry just 10 days before Oatley's retirement.

Oatley told McGuinness that while the British army might never stop the IRA altogether, it would always be able to contain it. Unless a political track were found, the cycle of senseless violence would repeat itself all over again. If the leadership were prepared to move in the direction of politics the British government might be willing to reopen the link. McGuinness indicated they were ready for such a dialogue and asked what the channel would be. Oatley explained he was retiring but said he would talk to the government about who might replace him.

On returning to London, Oatley went to see John Chilcott, the new permanent secretary of the NIO [Northern Ireland Office], and told him he had opened a channel to Martin McGuinness who was ready to discuss a political way forward.

MI5 insisted that the operation had to be mounted by one of their staff and found a retired senior SIS officer who had recently been re-employed by the security service. This officer introduced himself to Duddy (who named him "Fred") and took over running the link.

'The conflict is over'

The link served a useful purpose, not least in focusing the British government's mind on what they wanted, but it really took off when, in February 1993, "Fred" brought back a message purporting to come from Martin McGuinness. The message contained the following: "The conflict is over but we need your advice on how to bring it to an end. We wish to have an unannounced ceasefire in order to hold a dialogue leading to peace. We cannot announce such a move as it will lead to confusion for the volunteers because the press will interpret it as surrender. We cannot meet the secretary of state's public renunciation of violence, but it would be given privately as long as we were sure that we were not being tricked."

Ever since it became public, there has been much controversy over the provenance of this message. Whoever wrote the message, it inevitably created a stir within the British government. John Major convened a special cabinet committee to discuss how to respond to it and, on March 19, the government sent a substantive reply saying that, if the IRA proceeded to a ceasefire, the government would be "bold and imaginative" in response.

The Warrington bombs the next day, which killed two children, nearly derailed the process, but the government held its nerve and there was a carefully orchestrated exchange of messages with the IRA. There was never a long enough gap in IRA violence to allow a sensible face-to-face discussion to happen, but unbeknown to officials at the government end of the link, Duddy insisted on setting up a meeting in April for "Fred" and the head of MI5 in Northern Ireland to talk to McGuinness and Gerry Kelly, a leading republican who had been imprisoned for the Old Bailey bomb of 1973. "Fred" was pressured into turning up for the meeting but the Northern Ireland head of MI5 did not appear.

On November 2, there was a final message purporting to be from Martin McGuinness offering a "total end to violence". The IRA has subsequently denied ever sending such a message but the government responded enthusiastically, offering the guarantee of exploratory dialogue after a permanent ceasefire was called. The link was shortly to go down, however.

As 1993 drew to an end, its existence was leaked. The real significance of the link in the last chapter of its 20-year history lay in the impact which McGuinness' supposed desire for an unofficial ceasefire had on Major and the cabinet, who, with the exception of the sceptical Ken Clarke, were convinced there was at last the possibility of making progress towards peace.

· Great Hatred, Little Room - Making Peace in Northern Ireland by Jonathan Powell is publshed by The Bodley Head on March 20 at £20. To order a copy for £18 with free UK P&P go to theguardian.com/bookshop or call 0870 836 0875