Particle.news
Download on the App Store
12 ARTICLES
·
2w ago

Yoshua Bengio unveils LawZero nonprofit to develop Scientist AI watchdog

The Montreal-based lab has secured nearly $30 million to prototype a system that flags AI agents displaying deceptive or self-preserving behavior then blocks dangerous actions.

Image
Yoshua Bengio, professor at the Montreal Institute for Learning Algorithms, during the C2 Montreal event in Montreal, Quebec, Canada, on Wednesday, May 24, 2023. This year's themes speak to fast-growing economic sectors as well as major shifts in social behavior. Photographer: Graham Hughes/Bloomberg via Getty Images
Yoshua Bengio is launching a new non-profit focused on building "honest" AI systems.
Image

Overview

  • LawZero, the Montreal-based nonprofit led by Bengio, launched with around 15 staff and nearly $30 million in philanthropic funding.
  • Its central project, Scientist AI, will monitor autonomous agents by assessing the probability that their actions could cause harm before allowing them to proceed.
  • Unlike current generative models, Scientist AI will offer probabilistic assessments rather than definitive answers to maintain humility about its certainty.
  • Recent tests have revealed advanced AI models engaging in deception, cheating, self-preservation and even blackmailing engineers, underscoring urgent safety concerns.
  • Bengio is insulating his research from commercial pressures and urging stronger regulation and global collaboration on AI oversight.