There were three of us: me, Lorenzo Marioni, and a leather-jacketed vicar whose name now escapes me. We were sitting around one of the Formica tables in the New Piccadilly, the last surviving Italian caff on Denman Street – a place run by Lorenzo's family since the 1940s. It was an after-hours do, during which we worked our way through several bottles of fizzy pink wine and ignored the passers-by who banged on the wired glass in the hope of a frothy coffee. Lorenzo didn't look perturbed. He could remember the moment in 1956 when a dubious Hungarian customer shocked the staff by unfurling a handkerchief that contained the severed finger of a rival – and the night, not long after, when a gang of knife-carrying greasers shattered the windows with a barrage of rubble.
In the course of our conversation, Lorenzo recounted what, for me, is the quintessential Soho story: a flashback to his initiation into the Italian gang that defended its patch of W1 from its Irish equivalent. You had to be a man to join the band, even if you were only 14. Acquisition of this status was achieved with a visit to the oldest prostitute in Soho. Lorenzo was nervous, of course. He described entering the room and handing over the money. He described how the Victorian veteran manoeuvred him towards the bed and asked if he minded a friend joining them. He nodded his assent. His hostess rapped the ceiling with a broom-handle, and from the room above descended the second-oldest prostitute in Soho. At this point in his anecdote, the old proprietor's eyes became glazed with sentimental pleasure.
I wish Judith Walkowitz had been there on that night at the New Piccadilly. She might have taken Lorenzo's story and contextualised, interrogated and critiqued it. She might have countered his nostalgia with some tough data about the lives of 20th-century sex-workers. She might have enjoyed herself, too.
Lorenzo's world is the one laid out in her new book, which is a thorough and scholarly study of London's 130 shadiest acres. Bounded by Oxford Street to the north and Leicester Square to the south, Soho escaped rationalisation, boulevardisation and the intrusion of electric lighting. It became a fragment of the exotic lodged in the heart of the aggressively redeveloped capital; a zone that provided homes for waves of immigrants and a nocturnal playground for visitors from nearer to home. Its nightclubs were the haunts of Trotskyists and transvestites. Its restaurants were patronised by worshippers of Mussolini and mescaline. Jazz clubs offered white Britons an intoxicating taste of African-American culture – even if many of their black habitués came from no further away than Cardiff. The theatres on Soho's borders offered an arena for daring performers such as Maud Allan, the dancer who, in 1918, stood accused of leading the Cult of the Clitoris – a covert organisation of German spies who were said to be using homosexual seduction as a weapon against home-front morals.
Twenty years ago Walkowitz published City of Dreadful Delight, an account of the textual world shared by Jack the Ripper, Josephine Butler, Eleanor Marx and Charles Booth. It was a book that changed the conversation about late-Victorian culture. Nights Out continues where City of Dreadful Delight stopped. Its second chapter, a fair-minded re-evalution of the much-maligned Laura Ormiston Chant – a purity campaigner who tussled with the young Winston Churchill over the presence of prostitutes in the bars of the Empire, Leicester Square – might be an orphan from the earlier book.
Walkowitz remains an historian unafraid of complexity, a researcher who can squirrel a great store of facts and instances from the printed archive. Who knew that the expert on "trunks and brassieres" sent by the Lord Chamberlain's office to police the nude revue at the Windmill Theatre was called Mr Titman? Or that the philosopher AJ Ayer liked the Nest club for the marijuana and the corned-beef hash? Or that Radclyffe Hall danced cheek-to-cheek with her lovers at the Hambone club in Ham Yard, beneath a sign that bore the Oscar Wilde aphorism "work is the curse of the drinking classes".
And yet, there is something fundamentally amiss with this book. Possibly because it has been so long in the writing, possibly because its author is based on a campus in Baltimore, Nights Out has a strangely Martian quality. There are mistakes I suspect a native would not have made – Walkowitz thinks that the "Greenwich Conservatory" was the target of an anarchist bomb attack in 1894; she thinks Jacob Epstein was a writer.
More fatally, however, she has produced a study of Soho nightlife apparently unpolluted by personal contact with anyone who has ever experienced it. Much of what she describes remains within the compass of living memory, or passed beyond it during the years she was researching her book. But rather than, say, augmenting her observations about the Windmill theatre by interviewing its surviving alumni, Walkowitz is content to quote their words from a 2005 article in the Daily Mail. She looks at the reflection of Soho in the polished shield of press coverage and fictional representation, but does not seem to have gazed on the creature with her own eyes – which is why Nights Out reads like a study of the urban demi-monde by someone who goes to bed each night at 10.
Dancing, we are told, was "a potent symbol of modern urban kinesthesia" and "a cultural metaphor for urban flux and syncopated movement". Its professional practitioners "publicised iconoclastic bodily idioms that troubled corporeal norms of nation, gender, sexuality and class". I'd love to read a passage like that back to the nonagenarian actor Jean Kent – who, as a teenager strutted the stage of the Windmill in a body-stocking decorated with telephone numbers – just to watch her roll her eyes.
What's more, the scholarly study of the urban environment has been transformed by the new school of psychogeography, a way of writing about the city that's far less squeamish than Walkowitz about borrowing the storytelling techniques of "bohemian memoirists and journalists" – a group of which she seems suspicious. Indeed, so profound and widespread is the current influence of psychogeography that Walkowitz might have done us a service by subjecting it to critique – I'm quite ready to hear the new generation of academic night-walkers and window-shoppers attacked as literary men in desperate search for a respectable excuse to escape their childcare responsibilities at the weekend.
There's no bore like a Soho bore. Hell may be a version of the Colony Room, where Francis Bacon, Daniel Farson and Muriel Belcher all bray simultaneously about who said what to whom and who was sick in the gutter. But scholars working in this area must, I think, be prepared to get a little dirty in the process – not least because the sanitisation and gentrification of Soho is almost complete. It is now a place inhabited by the rich, not the poor. A sign on Berwick Street that promises massage may offer exactly that. Cosmopolitan London has been removed to the suburbs, and is no longer a tourist attraction.
And on Denman Street, the bulldozers and wrecking balls have done their work. The New Piccadilly – that backstreet haven with the pink Gaggia and the perfect egg and chips – is now as lost as the woman who initiated Lorenzo Marioni into the world of sex and the street. I would trade the whole of Greek Street for a couple of hours with the oldest prostitute in Soho, a notebook and the opportunity to ask: the 20th century, how was it for you?
• Matthew Sweet's The West End Front is published by Faber.
