The day I was punched in the face, I was so appropriately dressed I might almost have seen it coming. I was wearing glasses and holding a pile of library books. The first punch knocked my glasses to the ground, leaving me with a black eye. I dropped the books to protect my face just as a second attacker jumped on my back. The second punch split my lip and filled my mouth with blood. My assailants were both 13 years old, though the one hanging on to my back looked younger. A small, dark-haired child, he was barely 5ft 2in. The puncher was a ghostly pale redhead, also small for his age. The attack came on the third-floor walkway of the council block where I live in London. Afterwards, the redhead threw my library books over the balcony rail and swaggered a little as he warned me I would be next.

In a sense, I did know the attack was coming. Two nights earlier, my wife and I had called the police out twice as stones and kicks were aimed at our door by a group of boys that included this pair, the youngest and most obstreperous among them. The trouble had begun almost exactly a year earlier, in the summer holidays of 2007. Boys from neighbouring blocks began hanging out in our stairwell, staring sullenly as we struggled to pass, and littering the stairwell with cans, Popsicle wrappers and the debris of spliffs. The kids spat on the floor between drags and used the corners of the landings to pee. The block had recently been renovated - our share of the bill came close to £15,000 - and when I saw fresh punch marks in the thin steel walls of the lift, I knew to the penny how much this was costing me.

For most of that year, Leila - my wife - worked abroad. With no one but myself to worry about, I decided to throw these kids out whenever I found them inside the block. On the whole, they would leave when asked, though always with demands for respect. But a few responded with abuse, and by the time Leila returned the following spring, abuse had turned to threats. I reported all of this to the local Safer Neighbourhood Team, a police unit made up of part-time special constables drawn from the area. They supplied diary forms and asked if we could keep records to show a "course of harassment". As a result, a couple of boys received ABCs, or antisocial behaviour contracts that, we were told, were a step towards the more robust asbo: antisocial behaviour order. It seemed everything we did was a step that would lead elsewhere, and do so endlessly. I think it was then I realised the cycle could end only when one of the boys committed a more serious offence. When the attack came, my library books and my spectacles made it seem all the more dramatic: made me appear more harmless and unworldly; them more vicious.

The police arrived in 20 minutes. I was asked if I wanted to go to hospital, but refused. Leila was out and I did not want to get stuck in A&E. Unfortunately, the policeman did not tell me that the Crown Prosecution Service was unlikely to take the case to court without a doctor's report. His partner said nothing, simply stood with a finger on his radio earpiece as though alert to a more interesting case. I tried to stoke their enthusiasm by describing the worst of the recent events: the stone-throwing, the abuse, the relentless taunts of "snake" and "snake's wife". I told them that if the boys were not charged, they would feel they had a green light to do whatever they wanted. The police agreed, but soon left, leaving me deflated. My shirt was streaked with blood, so I took it off and sat on the stairs, bare-chested, face turned to the skylight above.

Our little two-storey flat is spectacularly sunny: it channels and traps all the available sunlight. The block is situated in a well-to-do corner of west London, giving us an unlikely, upmarket address. We bought it 10 years ago from a council tenant who had exercised her right to buy. A third of the flats are now in private hands, though the rest are buy-to-lets, leaving us the sole owner-occupiers among a mix of council tenants and transient professionals. I loved the place, but as I sat, half-naked, feeling the stinging in my lip and the swelling of my eye, I was as despondent as I had ever been. After a year of abuse, I had begun to dread turning the corner on to the estate in case I saw the kids in my stairwell.

At that moment, Leila called to ask what was in the fridge. We discussed possible supper ideas before I told her what had happened. At once, she insisted I call the police again, both the local station and the Safer Neighbourhood Team. She galvanised me and, within an hour, I was in an interview room with two new policemen who apologised that the original pair had finished their shift. As they took a detailed statement, they expressed surprise that it had not been taken on the scene. They also explained why it would have been better to have a doctor's report, and then rustled up an old Polaroid camera to take pictures under the gloomy strip lighting. One of them told me he had once been a police driver, and I sensed he missed this work. He became loquacious when he learned I had written crime novels and TV dramas such as Waking The Dead. He explained that they could arrest the boys, but said they would sit in the cells only until a youth worker and solicitor were found the next day. He knew both of them. The smaller one had an ABC and I recounted my first memory of him: he was smoking a spliff on the stairs and when I told him to leave, he began screaming "batty man" at me in a startling rage. The ex-driver described the redhead as "the Clearasil kid" and suggested I mention the boy's copious acne in my statement. He made the same complaint that I had heard over and over from the regular police, that it was no longer like the old days, and they could not clip the kids' ears. Then he asked if I was ready to hear the worst: "He will make a complaint against you. He'll claim that you assaulted him."

I was not there to see the arrest. The next morning, Leila and I flew to Greece. I had been hired to teach creative writing and, for the first few days, I wore sunglasses so no one's first impression of me would be a black eye. Of course, after telling one person the story, everyone soon knew and a man came forward to tell me he was a GP. I finally got the necessary doctor's report as well as good, clear photographs taken in natural light. By this time, bruises had emerged on my thighs, the marks of the scrabbling by the smaller boy as he clung to my back. Another participant turned out to be a senior police officer and she also warned that the boys would make a counter-allegation. She recalled an assault on one of her officers and complained bitterly that the Crown Prosecution Service had initially refused to prosecute because there were no independent witnesses. I kept hearing the same complaint: the CPS worked to targets, they were interested only in wins, they preferred CCTV evidence. There was no CCTV on my estate and no witnesses to the assault. I reflected on other problems, too, especially the broken-down quality of the local police station: the computer without internet; the Polaroid that looked as though it had been dredged from lost property.

Staying in Greece, a thousand miles away, we had no idea what was happening. I had a crime number that supposedly allowed me to track the case, but my emails went unanswered and attempts to telephone the station led to call waiting. Eventually, an email returned with the news that my case had been marked confidential, with access restricted to the investigating officers. This seemed extraordinary and when I wrote back, I was assured that it was - but they had no explanation. Soon, I was fantasising that I had become the chief villain and would be arrested.

The problem was, Leila was returning to the flat alone. I was heading on to a conference in Serbia. If the case was going to court, I knew the kids would try to frighten Leila. If it had been abandoned, they would be even worse.

The 24-hour train journey to Belgrade meant we were completely out of contact. When I finally spoke to Leila, she tried to reassure me, but in subsequent calls she admitted the kids had been using dogs to frighten her, gloating over what had happened to her husband. I returned home a few days later, long after dark, and the redhead passed me on his bicycle. He immediately returned to the estate and picked up the smaller boy on his handlebars. As they trailed me to my door, they made weird, birdlike noises and faked hyena-like laughs. At the door to my block, a group of eight boys joined in the jeers. The lobby smelled of marijuana, spit was strewn over the floor. Nothing had changed, everything was worse.

Yet we had reached a turning point. In the few days Leila had spent alone, a female sergeant had learned our story and decided this could not continue. She assigned the case to an energetic constable who looked barely older than the boys. Unable to access the restricted case files, he took my statement again - by hand because the computer had conked out. He never received the Polaroids, so was overjoyed to learn that I now had a doctor's report and my own photographs. Unlike other policemen, he seemed to enjoy his argy-bargy with the CPS as he built up a body of evidence to take to court.

The boys were charged with common assault and harassment, of me alone. Immediately, I encountered a whole new world. Two separate but related agencies telephoned: Witness Support, an independent service that operates within the police, and Victim Support, a charity funded largely by the Home Office. Witness Support had the job of keeping me informed of events. It had been their email that informed me the files had been restricted and they now supplied the reason: the redhead's mother was a civilian police worker at a different station, and she had persuaded her inspector to restrict the files. She may only have wanted to avoid gossip, but I felt sick as I realised how close her actions had come to sabotaging the case against her son, since no one, bar the original investigating team who were now reassigned, could view the files.

Victim Support offered counselling and help in filling out compensation forms. I rejected both offers, but agreed to a tour of the courtrooms to help steady my nerves. Alongside these new supporters, I also gained new forms of protection: the boys' bail conditions made it an offence to approach me, or anyone I knew, or approach my residence, or cause anyone else to approach me or my residence. I was urged to dial 999 whenever I saw them, and was told that a night in the cells often worked wonders. The boys made it easy. They ignored the bail conditions and over a single week I had them arrested three times and became the talk of the estate. Overnight, no one under the age of 20 would even approach the block. As a new era of calm dawned, I received handshakes and congratulations from my neighbours, and resentful glares from friends and relatives of the two boys. A policewoman told me the redhead's mother was distraught that her 13-year-old had spent a night alone in a police cell.

There was, still, the counter-allegation of assault. The redhead had made it as soon as he was arrested, but I was repeatedly told that it was dead - the new energetic constable gave me his personal assurance. Then, suddenly, and with apologies, I was asked to attend a formal interview. I wondered if I needed a solicitor? The police thought not, but I made a call to a local law firm and learned that their services would be paid for by legal aid. A solicitor met me at the door to the police station and we talked for 10 minutes before a constable led us inside, past racks of riot gear and banks of CCTV, to an interview room where old-fashioned cassette tapes were unwrapped and legal positions explained. I knew - perhaps only from novels and television, perhaps only from my own scripts - that it is wise to say almost nothing. I was super-garrulous. At first, it was nerves, but as the questions continued, it was pure confidence. The redhead's account was not only untrue, it was incoherent. As the police sergeant turned off the tape machine, he told me it was over - no charges would be made. On the way out, my solicitor told me that if it had gone to court, I would have made a superb witness.

How good a witness? I had to prove myself the next month in the youth court at Hammersmith. I had taken a tour the previous week with a woman from Victim Support who told me the most important advice was not to let the lawyers annoy me: it was their job to be unpleasant.

Youth courts aim to be less formal than adult courts. The defendants sit rather than stand. The boys were sitting with their lawyers, their mothers behind them. Three magistrates sat at a long desk, beside a legal expert surrounded by books. Aside from the boys and me, everyone was a woman. Remembering the advice of Victim Support, I tried not to become argumentative with the lawyers, with limited success. At one point I had to describe the small landings between floors in the stairwell, and introduced the term "interstitial", only to worry I had made it up. The redhead's version of events had changed again, but the outline of a defence began to emerge: the boys had been boisterous, even cheeky, over many months and I had responded by threats and abuse, as well as with calls to the police. The situation degenerated as I sought to criminalise the boys. As I responded to their lawyers' questions, the redhead sneered at me out of sight of his mother yet in plain view of the magistrates. It was so dim of him that, for the first time in months, I remembered that he was a child. Did his mother see him as vulnerable? Even as a victim? Did she, perhaps, feel a little victimised herself, following him from police station to court, missing a day of work each time?

I was on the stand for almost an hour, and when I left I was told I would learn the verdict the next day - the end of a long journey through the youth justice system, for both me and the boys. From the start I had wondered if it is still possible to tell off children who are misbehaving. In February last year, David Cameron launched the Tories' childhood review by saying it was the social responsibility of all adults to discipline other people's children. But the public mood seemed very different: in a YouGov poll conducted by Barnardo's, 54% of adults said that children were beginning to behave like "feral" animals, while 49% said that children were increasingly a danger to each other and to adults. For their part, the NSPCC suggested that adults should confront children only if, for instance, a child were in danger.

So what of my case? Maria Galan, a counsellor at Victim Support, tells me the charity does not recommend specific courses of action, but presents possibilities. So would she have confronted the kids? "No." What about the manager of my estate? "No. The problem is, you become a target."

Government figures from 2008 show antisocial youth crimes are dropping, and while these statistics have been robustly criticised by the former head of the Youth Justice Board as "smoke and mirrors", reports from Barnardo's in 2008 and the Teacher Support Network in 2007 agree that attacks on adults are rare. So how did we become so intimidated by our own kids? As I turned the matter over in my mind, I kept coming back to the issue of respect. Every time I had asked the children to get out of the stairwell, they had complained that I was disrespecting them. If respect meant so much to the kids, could it be part of the problem?

In 2005, Tony Blair announced that the series of changes his government had made to the youth justice system amounted to a "Respect Agenda". With that single word, "respect", the new system legitimised the child's eye view of the world: it is all about respect. So why was it chosen? In the 60s, Aretha Franklin turned Otis Redding's song Respect into a plangent demand - "Give it to me, give it to me, give it to me." Since the civil rights era, the demand for respect has become intimately entwined with the notion of human rights. When the boys on my estate turned from abuse to direct threats, my attempts to deal with them led to questions about children's rights. For instance, the estate caretaker warned me against filming their threats, in the belief that anyone who photographs kids "can get done for child abuse". In contrast, a policeman recommended I film the boys on the grounds that "they film us all the time, whenever we stop and speak to them". The kids themselves were aggressive in asserting that they were picked on, victimised, "dissed".

The kind of respect associated with human rights is the respect due to you as a fellow human being, rather than for any specific achievements. The children on my estate felt unregarded, powerless. Yet they did have one achievement: they had staked out their territory. A youth worker at the local youth club suggests that antisocial behaviour often has to do with protecting such turf: "Young people don't go out of their comfort zones," he says. "They are territorial. If they have to go to another area, then they will move in groups." If the children needed to hold on to territory to assert a sense of identity, then I was disrespecting them by trying to deal with their persistent trespassing. This is the darker side of their demand for respect, heard in rap music, the demand of the underdog who is prepared to bite back.

At the centre of a child's eye view of territory - the Endz, as current youth slang has it - there is invariably a stairwell. The autumn of 2006 saw the murders of two adults - Stevens Nyembo- Ya-Muteba, 40, and Alan Toogood, 50 - in the stairwells of council blocks by youths who objected to being told off for their antisocial behaviour. Through 2006, a gang called Murder Zone hijacked the lobby of a tower block in Brixton to deal drugs, while in Hackney in 2007 a gang known as the Kingzhold Boys dragged a 14-year-old girl around the stairwells of their estate and repeatedly raped her. The boys who attempted to take over my block are not Murder Zone or the Kingzhold crew: they are not gangsters. Yet they chose to act as though they were. A 2008 report from the UK Children's Commissioners declared, "The incessant portrayal of children as 'thugs' and 'yobs' not only reinforces the fears of the public, but also influences policy and legislation." This may be true, but it is also true that behaving as though one is a thug or yob has become cool for some children.

A day later than expected, I received a call from Witness Support. The boys had been found guilty on both charges. They would be sentenced in a fortnight. At once, Leila and I received a new visitor: a restorative justice worker from the Youth Offending Team. She asked about the psychological effects of the year-long harassment and our views on an appropriate punishment. We tried to describe the slow poisoning of our home lives and asked that the parents be made responsible for their sons' behaviour. She told us how a parenting order worked, and we realised how serious this could be. If their children continued offending, the parents risked losing their homes and gaining criminal records.

The sentences came: the boys received night-time curfews for two months and parenting orders for a year. Though we wished the curfews had been longer, we were happy and expressed a desire to meet the parents. Our restorative justice worker told us this might be a problem. The mother of the smaller boy was willing, but the redhead's mother was blaming me for victimising her son. We learned that the boy had vehemently resisted signing an ABC, claiming he had done nothing.

At last a message came that his mother would agree to a meeting in the new year - then it was off. On Christmas Eve, Witness Support called to say an appeal had been lodged against the redhead's sentence. The man on the phone doubted it would make it to court, but I resigned myself to fresh rounds within the youth justice system. It was only later that I realised the boy's curfew would be over long before any possible hearing. Only the parenting order would remain. So why appeal? Was the mother expressing unquenchable faith in her son? Was it shame that she would be seen as a bad mother? Perhaps the reason was simply that she did not want her job and her home to depend upon the actions of a 13-year-old. I doubt I will ever know, though I well understand how hard it is to live when your peace of mind depends on the caprice of a child.