For someone who had supposedly just been demoted, Chris Smith looked pretty cheerful when he was appointed secretary of state for national heritage in 1997. His last post in opposition had been shadow health secretary, but the arts world had lobbied Tony Blair for Smith's appointment, having been impressed with his shadowing of heritage in the mid 90s. As the prime minister put it to him, "you're a victim of your own success". So a smiling Smith stepped outside 10 Downing Street, made a bullish statement about the importance of the creative industries, scrapped the dreadful heritage name and launched the ministry of culture, media and sport.

There was no doubt he was the best candidate. He had a PhD in romantic poetry, his love of hill walking fitted with the tourism part of the job, his support for Arsenal covered sport, and as a regular gallery, concert, theatre and cinema goer, he had a genuine love and appreciation of the arts. And just as importantly, he was a seasoned politician. His initial claim to prominence might have come in 1984, when he became the first openly out gay MP at Westminster, but since then he had held a series of significant shadow frontbench posts covering the treasury, environment and social security, as well as heritage and health.

However, soon after his appointment things began to turn sour, with theatrical knights taking it in turn to heap abuse on him. "We hoped that this government would make us feel treasured. They haven't," intoned Sir Peter Hall, solemnly. Equally unsettling were the persistent rumours that Peter Mandelson was stalking his job.

"The first two years were undoubtedly difficult," Smith now acknowledges, "mainly because we kept to Tory spending plans. There was a period when I could hardly go to an awards ceremony without being berated from the stage by some panjandrum complaining that government was not putting enough money in the arts."

But it wasn't just a lack of cash that took the gloss off Smith and the then voguish concept of Cool Britannia (which Smith goes out of his way to distance himself from, pointing out that it was his Tory predecessor, Virginia Bottomley, who jumped on that particular bandwagon, using the words five times in press releases during her last year in power). Disproportionately time-consuming were the fall-out over fat cats running the lottery, the ongoing Millennium Dome saga, funding crises at Covent Garden, and disputes over the new Wembley. The absolute low point of Smith's tenure came in 1998 when he published a book of his speeches, Creative Britain. Perhaps it was the Damien Hirst book jacket, or the mood of the times, but Smith and the book were slaughtered by reviewers. "An insult to its readers", said one; "unpublishable maudlin trash", said another.

But three years on, and the world has changed again. While many of his colleagues in government struggle to maintain their reputations, Smith is apparently riding high. He is not only one of the few ministers to have been at the same post for the entire life of this government, but he has also synchronised a series of policy triumphs with the electoral timetable. At the moment when it matters most, Chris Smith has all his ducks in a line.

Melvyn Bragg - who, it should be said, is also a Labour peer - recently itemised some of Smith's achievements in a newspaper article: a seven-year secure financial base for the BBC; free museum access for all, which has already increased visitor numbers by 20% for children and 40% for pensioners; hugely increased funding, which has meant extra money for regional theatre; partnerships between arts professionals and schools in deprived areas; and a pledge that all children will have access to musical tuition. In the pipeline are ambitious proposals for the major artistic institutions, as well as for individual artists. "The polite, rather fastidious arts lover has delivered," wrote Bragg. "Now that the smoke has cleared, Chris Smith can be seen as the unlikely bespectacled hero in a western, the guy who won the shoot-out."

Seeing Smith arrive at his Trafalgar Square office, he unwittingly plays out a perfect tableau to illustrate his job. In one hand he holds a ministerial red box, in the other a dinner jacket. "I'm going to the National Gallery trustees dinner this evening," he explains, "but I suppose these two things do sort of sum up the job."

In the same way, the art that he has chosen from the government collection for his office also provides a potted biography. A portrait by Scottish artist Craigie Aitchison connects with his Edinburgh upbringing; a large etching of Lake Windermere by Norman Ackroyd chimes with his love of romantic poetry and hill walking; and some architectural shapes by Langlands and Bell, who exhibited in the definitive BritArt show, Sensation, provide a reminder that he is not afraid of the cutting edge. There is also a bright blue Howard Hodgkin lithograph, an Epstein bust of the conductor Klemperer to reflect his interest in music, and the politician's common touch is evidenced by a red, white and blue papier-mache raven made by the children of a school he visited.

Smith, who is 50 later this year, is a slighter figure than he appears on television and confidently projects a sense of understated straightforwardness. He plainly relishes discussing the ins and outs of social policy, as perhaps befits someone with a family tradition of public service. He is the second son of Colin Smith, a Whitehall civil servant, and Gladys, a maths teacher. The family lived in Watford until Smith was 10, and then moved to Edinburgh when his father was transferred to the Scottish Office and his mother became head of maths at a girls' school. The couple, now both retired, still live in Scotland and Smith's elder brother, Roger, teaches physics at a school there. Friends say the atmosphere in the home was, "fairly serious and obviously academically inclined". It was a Christian household, and Smith has been an active member of the Church of Scotland since his teenage years. There were also politics, although his parent's brand wasn't to the young Chris's taste. "I have to admit that they are both staunch Liberal Democrats," he winces.

In Edinburgh he attended George Watson's College, which was then a direct grant school and is now independent. Smith supported Labour at elections in the 60s and his friend, Douglas Graham, now a solicitor in Inverness, says Smith's interest in party politics didn't mark him out at all. "But he wasn't a dominant or a loud character. He was obviously highly intelligent and very determined, but he was always slightly reserved."

Smith says that, while there was a lot of reading at home, it was through school that he was really exposed to the arts. "A teacher helped out at the Edinburgh Festival, so we got tickets to concerts and other events. I remember seeing Ian McKellen in both Richard II and Edward II sometime in the late 60s. It bowled me over." He did play the piano - "I got to grade six" - but his main leisure activity was walking and hill climbing. He introduced the late John Smith to the sport, and he is the only MP who has climbed all the Munros, the 277 Scottish hills over 3,000ft.

He started climbing on a school trip when he was 12, and has been hooked ever since. "It is not only great exercise and a challenge. There is no better antidote to being in the smoke-filled corridors than to be standing on a mountain with half the country laid out before you. It really does put everything else into perspective." It is not only because of departmental responsibilities that he has been given the job of speaking up for the tourist industry in the face of the foot and mouth epidemic. Few other members of the cabinet have anything like his empathy for rural Britain.

Douglas Graham was present when he completed the last Munro, and says it is a tribute to Smith that friends came from all over Britain to be with him for the occasion. "It was a typically drizzly sort of day, but we had some champagne and a dram at the top and there was a lot of sincere friendship in evidence. There were people from school and university and politics. It was obvious that high office hadn't changed him as a man."

Smith's passion for politics and climbing were forged at the same time. He says that, growing up in the 60s, he was aware it was, "an incredible time of hope and belief that you could change the world". He had been impressed by the optimism surrounding Harold Wilson coming into office in 1964, and in America by the Kennedy and then Johnson administrations, the civil rights protests and anti-Vietnam war movements. "And there was my Christian faith. I don't think anyone can sit down and read the Sermon on the Mount without ending up a socialist. All that bundle of emotions about hope and change and democracy led me into the Labour party, and to a considerable extent has sustained me in the Labour party ever since."

The issues that still most animate him are those of basic equality of opportunity. He proudly praises his former university supervisor, who forged new links between Scottish comprehensives and Oxford colleges, and so enabled their brightest pupils to get to Oxford for the first time. In his current job, he has been a fervent supporter of increased access to cultural institutions.

"The aim of ensuring that the maximum amount of people have access to the best things in life has been at the heart of everything we have tried to do," he says. "It is the lodestone upon which everything else is based. I become very angry at some people who have argued that there are too many people coming into our museums. They regret that what they call the stillness of the old museums has gone. That is an exclusive argument about keeping the best things for the privileged few. It is perfectly possible for someone to have a high-art experience, even if they don't have a high-art background."

Jude Kelly, artistic director of the Yorkshire playhouse, sees an essentially democratic strand running throughout his policies. "He knows that if you are really looking for democracy, then that should apply to health and education, and also to culture. Until regions are served with first-class work, then you don't have a democracy, and he is genuinely committed to this."

As a student reading English at Cambridge in the late 60s, Smith was elected president of the union and chairman of the Fabian society. He took part in his share of demos, but recalls that much of the protest at Cambridge was remarkably genteel. "There was once a sit-in which lasted three or four days and was enormous fun. But when the vanguard of the invading students first entered the building, they actually stopped and held up the students behind them while they rolled up the William Morris carpet so it wouldn't get damaged."

A contemporary was Rabbi Julia Neuberger, who remembers him as, "very clever, even among a lot of very clever people. You could talk to Chris about things you wouldn't talk to other people about. We really would have conversations about Wordsworth." She echoes the view that he is largely unchanged. "I still sometimes want to play the Jewish mother with him and tell him to get more sleep," she laughs, "which is exactly the same as it was when we first met."

Smith says that at Cambridge, he asked himself if he wanted a life in politics. The answer was, "yes, probably", but he felt he had to do other things first. After gaining a double first, he embarked on a PhD about ideas of solitude in 18th-century poetry, with particular emphasis on Wordsworth and Coleridge. He then spent a year at Harvard as a Kennedy scholar.

"I was mainly working on my thesis, but I also hugely enjoyed the experience of being in America. I admired the American way of seeing how something could be done rather than ways that it couldn't. I was there in 75/76, which was post-Watergate and coming up to the bicentennial and presidential elections." He worked on Arizona democrat Mo Udall's failed presidential campaign and, on returning to Britain, thought about joining the civil service. "But I had the perspicacity to know that I would not want to spend my life serving a Tory government. So I worked in housing instead."

He was taken on as a graduate trainee by the government quango, the Housing Corporation. Nine months later he left to work for the Shaftesbury housing association, a Christian-based organisation that provides sheltered homes for the elderly and disabled. Smith says that even now, when he passes a building he was partly responsible for, he has a small surge of pride.

Mick Sweeney, now director of a housing association, was a colleague of Smith and a fellow branch official of their union. "He didn't put himself about like the stereotype of a politician on the make," recalls Sweeney. "He took his work very seriously, and he believed in providing low-cost housing to people. I think it's quite refreshing that someone like Chris has got the position that he has." So unassuming was Smith that when, some years later, another housing colleague attended a conference at which the Labour social security spokesman was speaking, he was amazed that, "it was that Chris Smith who came on stage. It just never occurred to me that it could be the same person."

In fact, Smith had begun climbing the political greasy pole when he was elected as an Islington councillor in 1978. The following year he stood in the general election in the unwinnable Tory seat of Epsom and Ewell, and afterwards jokingly confided that his biggest concern while canvassing was to avoid falling in the suburban swimming pools.

At Islington he became Labour whip and chair of the housing committee. It was an extremely difficult time, with a split Labour group, over half of whom eventually defected to the SDP. Acrimonious meetings would often carry on through the night as the differing factions cajoled reluctant councillors to vote their way, and it is a measure of Smith's toughness that he survived politically, let alone prospered. His cabinet colleague, Mo Mowlam, says he's always been strong but, "he's tough in a way that doesn't seem tough. Remember that he got through the dome as the minister responsible. He's not a table-thumping toughie, but he doesn't give in."

Islington council during this period was the spiritual home of what the tabloids liked to call the loony left. The red flag literally flew over the town hall and the widespread perception that councils such as Islington were more interested in political posturing than providing services reflected badly on Labour nationally. How does he look back on those years? "Well, I'm proud of many of the actual achievements, such as estate renovation programmes, rehabilitation of street houses, and some children's services," he says. "But I'm not proud of the murky politics. There was a lot of unpleasantness, a lot of sheer difficulty in getting decisions made."

He says that, as a chair of housing and a housing professional, he became ever more conscious that it was decisions taken by central government that dictated what he was able to do. "While I had taken socialism on board intellectually, it was while I was a councillor in Islington that I saw people living in poor housing and in poverty and became convinced that government exists to work for people like these. I really wanted to get closer to where the decision-making was in order to make a difference." His chance came at the 1983 general election, when he was selected to stand against the sitting Islington MP, George Cunningham, who had defected from Labour to the SDP. It was such a close contest that on his way to the count, Smith's agent advised him to prepare two speeches. After a recount, he eventually won by 363 votes.

It was 18 months later that he came out as gay, but contingency plans were in place for the election. "It was a reasonably well known fact that I was gay, as I'd made no particular secret of it. If it had emerged during the campaign, I would have said yes." As an MP he wanted to say something publicly as a way of putting the issue to rest. When, in November 1984, the Conservative-controlled council in Rugby dropped a policy banning discrimination on the grounds of sexuality, Smith was asked to address a protest meeting. A few minutes before he spoke, he decided that this was the moment. His opening line was: "My name is Chris Smith. I'm the Labour MP for Islington South and Finsbury, and I'm gay."

While his statement was reported in the press, it didn't spark a tabloid frenzy, and it was four months before the Sun even mentioned it. However, in the next general election campaign in 1987, Smith canvassers had a script as to what to say if anyone raised it. "The first point," he says, "was, 'isn't it a refreshing change to have a politician who is honest'. The second line was, 'he's a good MP who works hard for everyone'. And the third one, if people really persisted, was, 'you're not telling me you're prejudiced, are you?' No canvasser ever had to get to line number three, and I won with an increased majority."

Colin Richardson, the editor of Gay Times, says Smith's coming out was obviously important. "But even more important was when he was re-elected. There was a fear that even though he had what should have been a safe seat, it would rebound. But it was the opposite."

Smith has said he didn't really realise he was gay until his early 20s, and he didn't tell his parents until shortly before the Rugby statement. "They lived in Scotland and I was in London. We never really had the chance to have that rather difficult conversation." He says that they were initially concerned, "but subsequently they have been wonderful. They always ask about Dorian when I speak to them on the phone."

Smith's partner is Dorian Jabri, a consultant advising companies on ethical investment. Jabri was communications director for the teacher training agency when he first met Smith in 1988 at a lobby of MPs about section 28. Jabri then asked Smith to speak at a conference, and a few months later they were living together. There was a minor press flurry shortly after the 1997 election, when Jabri was invited to a Buckingham Palace function as Smith's partner, but mostly their social life is fairly low-key. Strikingly, considering Smith's job, they still regularly attend cultural events where they actually pay for themselves. Their most recent private evening out was to see Fiona Shaw's Medea a few weeks ago.

Smith says: "After the 1984 statement, I made a deliberate point of being absolutely prepared to plunge into issues that affected the lesbian and gay community; be it raids on bookshops, police entrapment or the start of section 28. But at the same time I made it clear I wasn't a one-issue politician. I also wanted to talk about the economy and housing and pensions and poverty."

Richardson says that he has struck a good balance. "He's maintained a high profile as a gay politician, been supportive of gay rights organisations, and has turned up to most of the gay rights votes that his schedule will permit. When you hear MPs say they won't be a one-issue politician, you worry they are not going to be a gay MP at all. But he has shown that you can be true to your principles and still be an extremely effective politician. He's not diminished by it; in fact, I think it has added to his stature."

Smith became an MP at Labour's lowest point, but says he only once thought he would spend his whole career in opposition - when Labour failed to win Basildon in 1992. That confirmed John Major's unlikely victory. "I knew that the oppositionalist mentality of the early 80s had not served anyone well," says Smith. He characterises himself as soft left at the time, and didn't vote for Tony Benn in the divisive deputy leadership contest of 1981. He says the incident that finally persuaded him Labour had to change was at his constituency re-selection meeting in 1985. A caucus of Militant supporters disrupted the meeting by demanding that Smith oppose the party inquiry into the Militant-dominated Liverpool local party. He took them on in a 20-minute public slanging match. "I got a lot of support from ordinary members and we didn't see [the Militants] again."

Smith says he was proud to then be part of the changes to the party begun by Neil Kinnock, and he still has a high regard for him as, "the person more than any other who saved the Labour party", and particularly for the dignified way he has conducted himself since resigning in 1992. But the leader he was closest to was John Smith, and on his desk is a picture he took of Smith on a Scottish mountainside five weeks before he died. "I still can't fully get over the sense of loss," he says. "It is very hard in both personal and political terms. What he had achieved was to put across to the electorate that passionate belief in social justice with a sense of economic competence. He had convinced people that it was both possible and necessary for the two to sit together in any sensible government, and it took root in people's minds as to what the Labour party stood for."

After John Smith's sudden death, he quickly decided to support Blair, who he not only saw as someone who could win for Labour but as someone who had a radical streak. Chris Smith also supported the whole New Labour project, although he stresses that, "the one thing I was very keen to ensure we didn't do was to lose the absolutely fundamental belief in social justice". Much was read into his invitation to Cherie Blair's 40th birthday party, in terms of him officially joining the Blairite coterie.

As an opposition frontbencher in the 90s, it was said that while he was competent, the Tories never thought he posed a real threat. However, Stephen Dorrell, whom he twice shadowed at heritage and health, describes Smith as, "always perfectly honourable. If he wasn't willing to do something, he said so, and if he was willing to do something, then he did it."

Perhaps part of the reason for any ineffectualness in opposition was that he was so frequently moved between briefs. At social security, he was the first of a string of people Blair asked to "think the unthinkable" in restructuring the government's vast expenditure. Mutterings from Gordon Brown's camp imply that all he did was work out schemes that would have cost billions, and Smith was moved on. He now acknowledges that he did have a policy disagreement with Brown on child benefit. Smith felt that withdrawing it from children over 16 might be a disincentive to education, but says now that he is completely in agreement with the amended policy that emerged in government, and that he and Brown work well together.

"It is a remarkably cohesive cabinet," he says. "There are very few disagreements on policy matters, and we all get on pretty well on a personal level." He is closest, politically and socially, to Mo Mowlam, who describes him as, "a decent human being who works incredibly hard and believes in what we are trying to do. He wants to help people, and is a principled guy who will find a way to achieve what he wants."

In particular, he has found a way round the major political anomaly of his post. Previous culture ministers have been hamstrung by the arm's-length principle of funding, by which the minister gets the cash out of the treasury but other organisations, such as the arts council, decide what it is spent on. Such has been Smith's influence on policy-making that, although the independence of funding has been maintained, he is very much seen by the big hitters in the arts world as the man who can get things done.

The philanthropic publisher and financier, Christopher Ondaatje, is impressed. Having worked with Smith after he donated £2.75m towards the new wing of the National Portrait Gallery, Ondaatje recently gave £2m to the Labour Party. He particularly praises the model of national institutions, the private sector and government working together. "Something seems to have happened in the arts in this country that is right. It hasn't happened over night, but Chris Smith has lent his support and enthusiasm to a number of institutions that seem to have lead a cultural revolution in England. I am really very proud to have been able to work with him and to have done something worthwhile."

Smith operates on the politician's seemingly obligatory four or five hours' sleep a night, filling his days with meetings and his evenings with functions. He worries about the lack of time to refresh and re-think his politics. "It's one of the problems of being in government; not seeing the wood for the trees. You are constantly dealing with individual issues and problems, and you try to run the tactics and you forget the strategy. You have to make time to sit back and look at the big themes you are trying to put in place."

Julia Neuberger says she thinks there is still a really big book in him. "I thought he would write the big Wordsworth book, but I am not sure what it will be now. But after he retires I am sure he will write something important, maybe on cultural policy."

The notion of Smith having more time to spend with his Wordsworth and his walking has been a regular feature of political gossip, and more recently there have been rumours that his whole department is to be scrapped. But he insists the department's future is secure. "I've been told by the very highest authority that it is not going to happen," he states meaningfully, adding that while, of course, it is the PM's decision, he also hopes to have plenty more wear out of both his red box and his dinner jacket.

"We need to follow through. There are the creative partnerships between cultural organisations and schools. It will allow children to get the chance to work with arts professionals. Far too few children have these opportunities, which could be of major benefit to them. Then there's Culture on Line, by which people can interact with the best of cultural activity and resources. This will be the equivalent of the Open University in the 60s, or Channel 4 in the 80s. More trust in the excellent institutions, with longer grants and less interference, and a whole new agenda for school sport. There' s certainly more work to be done."

Life at a glance

Christopher Robert Smith

Born: 24 July 1951

Education: Cassiobury Primary School, Watford; George Watson's College, Edinburgh; Pembroke College, Cambridge 69-75, Harvard University 75-76

Family: partner Dorian Jabri

Career: Development Secretary, Shaftesbury Housing Association 77-80; Development Co-ordinator Society for Co-operative Dwellings 80-83.

Political career: Islington councillor, 78-83; MP Islington South and Finsbury, since 83

Posts held: Opposition spokesman on treasury and economic affairs 87-92, on environmental protection 92-94, on national heritage 94-95, on social security 95-96, on health 96-97; secretary of state for culture, media and sport since 97

Other posts: Executive of NCCL 86-88; executive committee of National Trust 95-97; Governor Sadler's Wells Theatre 87-97

Books: Creative Britain 98