AI grows stronger, but its errors are getting worse
A new generation of “reasoning” AIs from firms like OpenAI is generating inaccurate information more frequently—and even their creators can't fully explain why.
AI Bots Are Getting Smarter—But Also More Wrong: The Cursor Case Highlights a Growing Concern
Just last month, users of Cursor—a rapidly growing software tool for developers—were caught off guard when they received a surprising message from the platform’s AI-powered support bot. The bot claimed that users could no longer access Cursor from more than one device, citing a new policy. The backlash was immediate. Some users canceled their subscriptions, while others took to forums to voice their frustration—only to learn that the policy didn’t exist.
“There is no such policy. You're absolutely allowed to use Cursor on multiple devices,” clarified Cursor CEO Michael Truell in a Reddit post, explaining that the erroneous message was generated by their AI-driven support system.
This incident shines a light on a growing problem: as AI becomes more advanced, its mistakes are becoming more frequent and harder to predict.
The Paradox of Progress: Smarter AIs, More Mistakes
Over two years since ChatGPT first launched, AI has become deeply embedded in everything from creative writing to software development and customer support. Yet as adoption grows, confidence in its factual accuracy is waning.
Strikingly, the newest AI models from companies like OpenAI, Google, and China’s DeepSeek—built to showcase stronger logical reasoning—are now making factual errors more often, not less. Although they excel in tasks like solving math problems or coding, they frequently get basic facts wrong. And even the developers behind these models admit they don’t fully understand why.
The root of the issue lies in how these systems work. AI models are trained on vast amounts of internet data and generate responses by predicting likely word sequences, not by checking facts. This leads to a troubling phenomenon known as "hallucinations"—confident, believable-sounding answers that are flat-out wrong.
In fact, recent evaluations show some advanced models have hallucination rates as high as 79%.
Why Hallucinations Happen—and Why They're Hard to Eliminate
Unlike traditional software, which runs on rigid rules and logic, AI language models rely on probabilistic guesses based on patterns in data. That means errors—sometimes bizarre or completely made-up—are an inherent risk.
“Despite our best efforts, they will always hallucinate,” said Amr Awadallah, CEO of AI startup Vectara and a former Google executive. “That will never go away.”
While some AI missteps are harmless—such as misstating a city name or misquoting trivia—they can have serious consequences in high-stakes settings like legal services, medical advice, or business strategy.
Pratik Verma, CEO of Okahu, a company working on AI reliability, noted: “You end up spending a lot of time just trying to figure out what’s real and what’s not. If you don’t fix that, you lose the entire value AI is supposed to offer—saving time.”
What's Your Reaction?
