At the start of the 2010s, rates of teenage mental illness took a sharp upward turn, and they have been rising ever since. Among US college students, diagnoses of depression and anxiety more than doubled between 2010 and 2018. More worrying still, in the decade to 2020 the number of emergency room visits for self-harm rose by 188% among teenage girls in the US and 48% among boys. The suicide rate for younger adolescents also increased, by 167% among girls and 91% among boys. A similar trend has been observed in the UK and many other western countries. The American social psychologist Jonathan Haidt believes this mental health crisis has been driven by the mass adoption of smartphones, along with the advent of social media and addictive online gaming. He calls it “the Great Rewiring of Childhood”.

Children are spending ever less time socialising in person and ever more time glued to their screens, with girls most likely to be sucked into the self-esteem crushing vortex of social media, and boys more likely to become hooked on gaming and porn. Childhood is no longer “play-based”, it’s “phone-based”. Haidt believes that parents have become overprotective in the offline world, delaying the age at which children are deemed safe to play unsupervised or run errands alone, but do too little to protect children from online dangers. We have allowed the young too much freedom to roam the internet, where they are at risk of being bullied and harassed or encountering harmful content, from graphic violence to sites that glorify suicide and self-harm.

Haidt is a professor at New York University and frequently collaborates with the American psychologist Jean Twenge, who was one of the first to attribute rising rates of mental illness among gen Z (those born in the mid to late 1990s) to smartphones. Sceptics of this research sometimes argue that young people simply have more things to feel anxious and depressed about, between climate change, rising inequality, global conflict and political perma-crisis. But Haidt makes his case persuasively. Earlier generations have also grown up in the shadow of war and global instability, he points out, and collective crises don’t typically produce individual psychological ones, perhaps because they often engender a sense of greater social solidarity and purpose. Instead, the evidence linking mental illness to smartphones and social media use is mounting.

The British millennium cohort study, which followed 19,000 children born in 2000-02, found that, among girls especially, rates of depression rose in tandem with hours spent on social media. Girls who spent more than five hours a day on social media were three times more likely to become depressed than those who didn’t use it at all. This study alone isn’t enough to prove that social media causes depression (it’s possible that depressed people spend more time online) – but there’s more. Facebook was initially offered only to students at a small number of universities, so one study compared the mental health of students at institutions with Facebook with those who didn’t yet have social media – and found that Facebook increased poor mental health on campus. Five other studies have demonstrated a link between the arrival of high-speed internet and rising rates of mental illness.

So why might “phone-based” childhoods have this effect? Smartphones pull us away from our immediate surroundings and the people closest to us, rendering us, as the sociologist Sherry Turkle puts it, “forever elsewhere”. Teens are not only the most compulsive smartphone users – one 2022 Pew Media report found that 46% of them are online “almost constantly” – but they are also the most vulnerable, partly because adolescence is a period of rapid social and emotional development. Smartphones are “experience blockers”, Haidt writes: consider how many enriching activities were displaced when young people began spending hours a day online, chasing likes, following vapid influencers, substituting the richness of real-life friendship with shallow online communication. Social media encourages constant social comparison, and it can be unforgiving and cruel. These observations might sound old-fashioned, but they are also true. What middle-aged adult doesn’t feel relief to have grown up before smartphones? Adolescence was hard enough without the threat of online humiliation, the possibility of quantifying, through engagement and follower numbers, exactly how much of a loser you are.

One avenue Haidt doesn’t explore, which feels like an omission, is that his critics might be partly right about teenagers feeling anxious and depressed in response to global events – or at least to coverage of them. Could the internet’s 24-hour news cycle, its emotional fever-pitch and the sharing of graphic frontline footage, be contributing to a permanent sense of threat? It has certainly distorted our perspective on current affairs, amplifying people’s sense of personal danger. As the Oxford climate scientist Hannah Ritchie observed in her recent book, Not the End of the World, death rates from natural disasters have fallen tenfold in the past century, but almost everyone thinks they have risen. It’s also clear that today’s defining crises, such as the pandemic and climate change, won’t necessarily deepen social solidarity in an era of filter bubbles and “alternative facts”.

Haidt’s theory that overprotective parents are contributing to the mental health crisis is much less substantiated than his research on phones. He argues that children are “antifragile”: like saplings that need to be buffeted by winds in order to grow properly, they need to experience setbacks to develop resilience. Mollycoddled kids become defensive and insecure, Haidt writes, starting to view ideas as dangerous and demanding safety from beliefs they find challenging. This is an argument he advanced in his 2018 book, The Coddling of the American Mind, co-written with Greg Lukianoff. In the years since, it has become painfully apparent that the groups most likely to treat ideas as dangerous are the ultra-conservatives who organise book bans – and most of these rightwing activists are old enough to have enjoyed free-range childhoods themselves. I actually agree with Haidt that children ought to be given greater freedom to play unsupervised, but he overstates his case.

The Anxious Generation is nonetheless an urgent and essential read, and it ought to become a foundational text for the growing movement to keep smartphones out of schools, and young children off social media. As well as calling for school phone bans, Haidt argues that governments should legally assert that tech companies have a duty of care to young people, the age of internet adulthood should be raised to 16, and companies forced to institute proper age verification – all eminently sensible and long overdue interventions.

I felt a gnawing anxiety as I read the book, thinking not only of my three young children, who I’d like to keep away from the badlands of social media for as long as possible, but also of the uncounted hours I have spent on my phone, mindlessly scrolling. “There’s a God-shaped hole in every human heart,” Haidt writes, paraphrasing the French philosopher Blaise Pascal. “If it doesn’t get filled with something noble and elevated, modern society will quickly pump it full of garbage.” Maybe we ought to start thinking more about all the things we didn’t look at, all the people we didn’t speak to, all the thoughts we didn’t allow ourselves to finish, because we were glued to our stupid smartphones.

• The Anxious Generation: How the Great Rewiring of Childhood Is Causing an Epidemic of Mental Illness is published by Allen Lane (£25). To support the Guardian and the Observer buy a copy at guardianbookshop.com. Delivery charges may apply.