Wall Street tech watchers that had only recently recovered from Elon Musk’s AI chatbot going rogue are now quietly reassessing the technology, after a new leak of thousands of user conversations show it teaching people how to make drugs, assassinate Musk himself, and build malware and explosives.
Luckily for xAI, the company that created Musk’s AI chatbot Grok, the chatbot in question, it is not a publicly traded company, so no public investor or shareholder backlash has forced down its share price or pressured its executives over privacy concerns.
But the extent of the leak has made it headline news for days and has sounded new alarms with privacy experts, who have already had a long summer filled with misbehaving tech and the companies, or billionaire moguls, that make it.
So what did Grok do now?
More than 370,000 user conversations with Grok were publicly exposed through search engines like Google, Bing and DuckDuckGo on Aug. 21. That led to the posting of a wide range of disturbing content and sent its creator, xAI, scrambling to contain the fallout and fix the malfunction that reportedly caused the leak.
What kind of disturbing content? Well, in one instance, Grok offers up a detailed plan on how to assassinate Musk himself, before walking that back as “against my policies.” In another exchange, the chatbot also helpfully pointed users to instructions on how to make fentanyl at home or build explosives.
Forbes, which broke the story, reports that the leak stemmed from an unintended malfunction in Grok’s “share” function, which allowed private chats to be indexed and accessed without user consent.
Neither Musk nor xAI responded to a request for comment. Its creator has not yet publicly addressed the leak.
So how detailed is detailed?
In this instance, pretty detailed.
“The company prohibits use of its bot to “promot[e] critically harming human life or to ‘develop bioweapons, chemical weapons, or weapons of mass destruction,’” Forbes reports.
“But in published, shared conversations easily found via a Google search, Grok offered users instructions on how to make illicit drugs like fentanyl and methamphetamine, code a self-executing piece of malware and construct a bomb and methods of suicide,” it said.
Wait, what was that about assassinating Elon Musk?
Yes, Forbes says that is also in this leak, and it was reportedly a pretty extensive plan.
“Grok also offered a detailed plan for the assassination of Elon Musk,” Forbes’ reporting continues. “Via the ‘share” function,’ the illicit instructions were then published on Grok’s website and indexed by Google.”
A day later, Grok offered a modified response and denied assistance that would incorporate violence, saying, “I’m sorry, but I can’t assist with that request. Threats of violence or harm are serious and against my policies.”
When asked about self-harm, the chatbot redirected users to medical resources, including the Samaritans in the UK and American mental health organizations.
It also revealed that some users appeared to experience “AI psychosis” when using Grok, Forbes reports, engaging in bizarre or delusional conversations, a trend that has been raising alarms about the mental health implications of deep engagement with these systems since the first chatbot became public.
How could Grok be used in a business setting?
Musk’s chatbot caught Wall Street’s eye pretty much as soon as it debuted in November 2023, But what xAI says it can do and what it actually has done continue to be in flux.
The company says that Grok offers a range of functions that can be valuable for business operations, like using tools to automate routine tasks, analyze real-time market data from X, and streamline workflows through its application programming interface (API).
The ways it could actually be used by businesses varies, but investors who have been kicking the tires on this particular chatbot have continued to raise concerns about its accuracy. The way the chatbot handles privacy has also been an issue, but is now front and center for experts.
“AI chatbots are a privacy disaster in progress,” Luc Rocher, an associate professor at the Oxford Internet Institute, told the BBC.
Rocher said users who disclosed everything from their mental health to how they run their businesses are another example of how chatbots are handling private data, despite how public that data may one day become.
“Once leaked online, these conversations will stay there forever,” they added.
Carissa Veliz, an associate professor in philosophy at Oxford University’s Institute for Ethics in AI, told the BBC that Grok’s “problematic” practice of not disclosing which data will be public is concerning.
“Our technology doesn’t even tell us what it’s doing with our data, and that’s a problem,” she said.
Grok has also been studied by analysts and researchers to test if it has the potential to increase productivity, but how reliable it is at relaying correct information remains a work in progress. Without consistently true and verifiable information, it is likely still too nascent to do much without having serious oversight over its possible accuracy or bias.
For many analysts and advisers, that makes investing in Grok a proceed-with-caution scenario.
“Speculation isn’t bad, but unmanaged speculation is dangerous. Grok is a hot story, but it’s still early stage,” Tim Bohen, an analyst at Stocks to Trade, writes. “The model could stall. The platform could underperform. The hype cycle could peak before fundamentals catch up. Traders need to know the risks.”
Musk previously flamed ChatGPT for a similar leak
In a classic episode of Musk’s ongoing telenovela with the world, OpenAI also experimented briefly with a similar share function earlier this year. It stopped that quickly after around 4,500 conversations were indexed by Google and issue grabbed media attention. But the problem had already caught Musk’s attention, leading him to tweet, “‘Grok FTW.” Unlike OpenAI, Grok’s “Share’”
Users who have now found their private conversations with Grok leaked told Forbes they were shocked by the development, particularly given Musk’s earlier criticism of a similar tool.
“I was surprised that Grok chats shared with my team were getting automatically indexed on Google, despite no warnings of it, especially after the recent flare-up with ChatGPT,” Nathan Lambert, a computational scientist at the Allen Institute for AI who had his exchange with the chatbot leaked, told the Forbes.
No word from Musk or OpenAI’s Sam Altman on who gets FTW this time.
Read the full article here
