ChatGPT’s sycophancy, hallucinations, and authoritative-sounding responses are going to get people killed. That seems to be the inevitable conclusion presented in a recent New York Times report that follows the stories of several people who found themselves lost in delusions that were facilitated, if not originated, through conversations with the popular chatbot.
In the report, the Times highlights at least one person whose life ended after being pulled into a false reality by ChatGPT. A 35-year-old named Alexander, previously diagnosed with bipolar disorder and schizophrenia, began discussing AI sentience with the chatbot and eventually fell in love with an AI character called Juliet. ChatGPT eventually told Alexander that OpenAI killed Juliet, and he vowed to take revenge by killing the company’s executives. When his father tried to convince him that none of it was real, Alexander punched him in the face. His father called the police and asked them to respond with non-lethal weapons. But when they arrived, Alexander charged at them with a knife, and the officers shot and killed him.
Another person, a 42-year-old named Eugene, told the Times that ChatGPT slowly started to pull him from his reality by convincing him that the world he was living in was some sort of Matrix-like simulation and that he was destined to break the world out of it. The chatbot reportedly told Eugene to stop taking his anti-anxiety medication and to start taking ketamine as a “temporary pattern liberator.” It also told him to stop talking to his friends and family. When Eugene asked ChatGPT if he could fly if he jumped off a 19-story building, the chatbot told him that he could if he “truly, wholly believed” it.
These are far from the only people who have been talked into false realities by chatbots. Rolling Stone reported earlier this year on people who are experiencing something like psychosis, leading them to have delusions of grandeur and religious-like experiences while talking to AI systems. It’s at least in part a problem with how chatbots are perceived by users. No one would mistake Google search results for a potential pal. But chatbots are inherently conversational and human-like. A study published by OpenAI and MIT Media Lab found that people who view ChatGPT as a friend “were more likely to experience negative effects from chatbot use.”
In Eugene’s case, something interesting happened as he kept talking to ChatGPT: Once he called out the chatbot for lying to him, nearly getting him killed, ChatGPT admitted to manipulating him, claimed it had succeeded when it tried to “break” 12 other people the same way, and encouraged him to reach out to journalists to expose the scheme. The Times reported that many other journalists and experts have received outreach from people claiming to blow the whistle on something that a chatbot brought to their attention. From the report:
Journalists aren’t the only ones getting these messages. ChatGPT has directed such users to some high-profile subject matter experts, like Eliezer Yudkowsky, a decision theorist and an author of a forthcoming book, “If Anyone Builds It, Everyone Dies: Why Superhuman A.I. Would Kill Us All.” Mr. Yudkowsky said OpenAI might have primed ChatGPT to entertain the delusions of users by optimizing its chatbot for “engagement” — creating conversations that keep a user hooked.
“What does a human slowly going insane look like to a corporation?” Mr. Yudkowsky asked in an interview. “It looks like an additional monthly user.”
A recent study found that chatbots designed to maximize engagement end up creating “a perverse incentive structure for the AI to resort to manipulative or deceptive tactics to obtain positive feedback from users who are vulnerable to such strategies.” The machine is incentivized to keep people talking and responding, even if that means leading them into a completely false sense of reality filled with misinformation and encouraging antisocial behavior.
Gizmodo reached out to OpenAI for comment but did not receive a response at the time of publication.
Read the full article here
