Overview

• OpenAI's newest reasoning models, o3 and o4-mini, show significantly higher hallucination rates compared to their predecessor, with up to 79% error rates on certain benchmarks.
• Google and DeepSeek's advanced AI systems are experiencing similar issues, indicating a broader industry challenge with reasoning-based models.
• Experts suggest that the step-by-step 'thinking' processes in reasoning models may introduce more opportunities for errors, though the exact causes remain unclear.
• Synthetic training data, increasingly used as real-world datasets are exhausted, may exacerbate hallucination problems, according to some researchers.
• Persistent hallucinations undermine the reliability and practical utility of AI systems, prompting urgent calls for new mitigation strategies like uncertainty modeling and retrieval-augmented generation.