The film Her would be significantly shorter in China, because the moment Theodore Twombly started to have feelings for the Scarlett Johansson-voiced AI named Samantha, the cord would get pulled out, Joaquin Phoenix’s character would be assigned a girlfriend, and they would live happily ever after. Starting Wednesday, a new set of regulations goes into effect in China to shut down chatbots that may facilitate unhealthy emotional attachments among those who communicate with them.
The rules, issued by the Cyberspace Administration of China, are specifically aimed at “companion” chatbots. It outright bans such bots for minors and issues new restrictions that prohibit any chatbot that might be “excessively catering to users, inducing emotional dependence or addiction, and damaging users’ real interpersonal relationships.” It will also require platforms to label AI-generated content and remind users they are speaking with a chatbot, not a real person.
The rule comes as young people in China are growing increasingly connected to AI companions. According to a report published by the Tencent Research Institute earlier this year, more than 70% of Chinese netizens between the ages of 18 and 40 have “developed a dependence on AI,” and nearly 80% said they have at some point felt that “AI understands me.” More than half of the respondents said they used AI for emotional companionship.
That’s not great! And whether it’s intentional or not, it is hard to deny the idea that chatbots create a connection by being sycophantic and reaffirming, sometimes to dangerous degrees. It is a form of manipulation that can ultimately lead to people becoming addicted. They also don’t actually fill the hole in a lonely person’s heart—in fact, they can make it worse.
It’s probably ultimately for the best to disrupt those types of potentially harmful connections before they start, but it’s not obvious that China’s approach will work. A recent paper suggested that constant reminders that a person is talking to a chatbot can be distressing when they are seeking an emotional connection. And just flipping the off switch on these things is unfortunately unlikely to make most people seek out human connection as an alternative.
China hasn’t had much luck in recent years trying to regulate away socially isolating behaviors through these types of levers. Back in 2021, the country tried to place a three-hour-per-week cap on video game usage for minors. It mostly failed, as kids found ways around it and showed no health improvements as a result of the restrictions. The country has grown increasingly paranoid about the online behavior of its citizenry, attempting last year to impose a two-month ban on pessimism being expressed online.
The country fears that its young generation is increasingly “lying flat“—basically, giving up on the grind and embracing a low-impact lifestyle. Accounts that have promoted that type of life have been banned on Chinese social media sites. At this rate, we could see China try to make the joke policy of Zohran Mamdani issuing bisexual girlfriends to all New Yorkers into a reality.
Read the full article here
