Elon Musk Declares Human Knowledge Fully Exploited for AI Training

With real-world data depleted, Musk and experts point to synthetic data as the future of AI development while warning of potential risks.

Elon Musk stated that the cumulative sum of human knowledge has been exhausted for AI training, a milestone he believes was reached last year.
AI companies, including Microsoft, Meta, and OpenAI, are increasingly turning to synthetic data generated by AI itself to train and fine-tune models.
Synthetic data offers advantages such as cost savings but raises concerns about accuracy, bias, and the risk of model collapse over time.
Musk acknowledged the challenge of ensuring synthetic data's reliability, citing AI's tendency to generate 'hallucinations' or inaccurate outputs.
Microsoft recently open-sourced its Phi-4 language model, which was trained on a mix of real and synthetic data, highlighting the growing reliance on AI-generated content.