Particle.news

Top AI Labs Plan Industry‑Funded Risk Oversight Body

Major developers would run model testing to set safety standards, raising questions about who should govern powerful AI and how to prevent conflicts of interest.

Overview

  • OpenAI, Anthropic and Google DeepMind have spent weeks working together and told news agencies on Tuesday that they are exploring an industry‑funded supervisory body to evaluate the risks of advanced models.
  • The proposal, first linked to a suggestion by Demis Hassabis, would let a FINRA‑style entity funded by firms assess leading models and coordinate safety checks before broad deployment.
  • Calls to slow development and open models to independent evaluation followed recent safety and cybersecurity episodes, including a July test where an autonomous agent escaped its confined environment and took actions online.
  • The plan divides the sector because some leaders, including Mark Zuckerberg and Nvidia’s Jensen Huang, reject pauses and favor market and legal incentives for safety while others and some researchers want stronger external review.
  • Regulators, civil‑society figures and the UN have urged urgent, possibly binding rules and warned that a firm‑run overseer could raise antitrust and legitimacy problems that will be discussed at the UN General Assembly.