The first time Baxter Dury performed on stage was at his famous father’s wake. While various stars worked their way through the Ian Dury songbook, Baxter, who had recently launched his musical career at the age of 29, was the obvious choice to reprise My Old Man, Ian’s tribute to his own father, Bill Dury. A working-class east Londoner, bus driver and chauffeur, Bill hadn’t figured large in the life of his son, who was raised by his mother and her family, members of what Baxter calls “the bohemian intelligentsia”.

It’s tempting to suggest that one absent father led to another. Certainly, Ian had an elastic sense of parental responsibility, leaving his marriage to Betty Rathmell soon after the birth of Baxter and his older sister Jemima, then showing up erratically while contributing “a pittance” to their upkeep. His ambition to become a lead singer was all that counted, pursued first with Kilburn and the High Roads, and later with the Blockheads, once 1977’s New Boots and Panties!! had secured his breakthrough at age 35 (six-year-old Baxter is there on the album’s classic cover). With a severe physical disability, inflicted by polio when he was seven, Ian was an unlikely rock star and had become an even more unlikely national treasure by the time of his death in 2000.

By turns charming, funny, abusive and threatening, Ian was a notoriously tricky character, the threat often being provided by his minders: “Dad admired anyone able-bodied and potentially violent,” recalls Baxter in this entertaining memoir centred on his teenage years in west London.

Prior to New Boots, his was a poor, provincial (though arty) childhood in Buckinghamshire. His father’s success brought a thrilling immersion in showbiz – Baxter recalls that on a UK tour, “when my clothes became too smelly I was bought a new football kit in whatever town we were in” – though “little responsibility in overseeing our welfare… Jemima and I were shoved around between an assortment of managers, minders, girlfriends and roadies”.

A more dramatic change came later, when Betty moved to Chiswick, just down the road from Ian’s riverside flat in Hammersmith, and Baxter was enrolled at Chiswick school, an institution he quickly came to loathe and from which he truanted compulsively. Efforts to integrate came to nothing. A concerned phone call from the deputy head to Dury brought an abrupt response: “Why don’t you fuck off you snotty little maggot?” he shouted. There’s nothing like having your parents’ backing.

Baxter is clear-eyed about his father’s strengths and failings, of which he saw more once he had effectively dropped out of school altogether and was lodging in his father’s flat. For Baxter and his friends, Ian became a “a pot-soaked Fagin”, a source of dope, street wisdom and cool jazz records – after all, “the one thing he loved was attention”. The boundaries between parent and child, as between day and night, were hopelessly scrambled. “Mum listened and supported. Dad broke your confidence and replaced it with his own.”

Things took a weirder and more alarming turn with the arrival of a new minder, a 6ft 7in “malodorous giant” who had previously worked for Led Zeppelin, among others. His name was Peter Rush but he was commonly known by his nickname, the Sulphate Strangler, a soubriquet earned by drug consumption (and dealing) and a party trick involving picking up people by their throat. Ian moved him into his flat, meaning Baxter was evicted from the spare room to the front room’s “decaying Victorian day bed” that his father fancifully called “the chaise longue”. The Strangler became a bizarre in loco parentis whenever Ian was away, acting in a series of second-rate European films.

Baxter’s mother had not abandoned hope that he might yet receive an education, and Baxter in turn “wanted people around that weren’t always fucked... and I wanted a normal breakfast”. He was enrolled in a “crammer college, designed to help the rich and unruly complete their education”. This, too, would not work out, though it did change Baxter’s prejudice against posh people, whom he recognised as kindred spirits from equally chaotic backgrounds, but with money, cars and more upmarket drug tastes.

Whatever he tried, chaos followed Baxter into his 20s like a hungry dog, be it another school (expelled for drug dealing), starting a night club (instantly bankrupt), working in a West End watch shop (burned down), moving into a flat (squalor ensued) or a six-month spell in Barcelona (went broke). He fared best when accompanying his father on another European film shoot, where he found himself promoted to an assistant director simply by being on set. Tragedy was not far behind, as his mother became terminally ill, her death coinciding with his father settling down and siring another child. By then, Baxter was working in TV and film.

A career as a musician almost inevitably followed, and for two decades Baxter Dury has established himself as a nuanced songwriter over half a dozen albums. The most recent, last year’s The Night Chancers, is his most accomplished, a series of sharp character vignettes delivered with a growl. The folks of the title track even show up in Chaise Longue as part of the Strangler’s retinue, “people sold on the idea that eroticised plastic clothes and taking speed somehow defined them as something”.

His memoir is chaotically organised, shuttling back and forth in time, but written with linguistic gusto. His father would be made up.

• Chaise Longue by Baxter Dury is published by Corsair (£16.99). To support the Guardian and Observer order your copy at guardianbookshop.com. Delivery charges may apply