Humans have been keeping time as far back as 3500 B.C. ChatGPT is still figuring it out.
Last week, OpenAI CEO Sam Altman appeared on the show Mostly Human to talk about the future of AI, his company, and humanity as a whole. The interview was relatively standard fare—the host, Laurie Segall, got Altman on the record on the demise of Sora, OpenAI sliding in following the Pentagon’s spat with Anthropic, etc. But at one point, she asked Altman to react to a viral video posted by TikTok user @huskistaken in which he asks ChatGPT’s voice model to time him running a mile. The chatbot very obviously makes up a time instead of actually keeping track.
Husk is a pro at showing the limitations of AI models, and getting ChatGPT to not only pretend to keep time for him but then also insist that it actually did so when it obviously hasn’t is a solid bit of prodding at the machine. Altman laughs when shown the video, though it looks like it’s in the way that people laugh when they’re actually angry and trying to play it off. When Segall asks him if he needs to show the video to his product team, Altman pretty tersely responds, “No, no, that’s a known issue.”
Then, unprompted, Altman offers up a kind of shocking timeline for the groundbreaking feature of counting: “Maybe another year before something like that works well.” Per Altman, ChatGPT’s voice model doesn’t have the capability of starting a timer or keeping track of time. “But we will add the intelligence into the voice models,” he said.
AI models, kinda notoriously, do not handle time well. People have asked ChatGPT’s text model to track how long a conversation has been going, and it also seemingly just makes up a duration in response. Most models also struggle to tell what time it is on a clock when presented with an image of one, and image generation models have always had a hard time creating clocks that display a specific hour or minute. Something about numbers and the concept of time just really throws these systems for a loop.
Husk happened to see the video in which Altman responded to him, and decided to add one more layer to the reaction chain by showing Altman’s take to ChatGPT—specifically, that the AI model does not actually possess this capability, no matter how much it insists it does. After getting ChatGPT to confirm that it claims to have a timer (it calls keeping time “just a basic part of what I can do”), he plays the clip of Altman saying that the model cannot, in fact, time anything.
Upon being presented with Altman, CEO of the company that makes ChatGPT, saying that it cannot keep time, ChatGPT insists, “What he’s saying is that some voice models might not have all the capabilities, but I do.” Pressed on this, the model said, “I definitely have a time capability.” Finally, when asked to time Huck running a mile, which he claims is complete almost immediately, ChatGPT clocks the run at 7 minutes and 42 seconds.
So, the countdown clock is ticking; Altman has 1 year to get this sorted out.
Read the full article here
