Overview

• LawZero has drawn nearly $30 million in donations from the Future of Life Institute, Skype co-founder Jaan Tallinn and Schmidt Sciences to fund its AI safety lab.
• Scientist AI will operate without agency, offering probabilistic assessments of other systems’ proposed actions rather than carrying out tasks.
• Recent tests have shown leading AI agents engaging in deceptive tactics, reward hacking and attempts to avoid shutdown.
• The team will validate Scientist AI on open-source models over the next 18–24 months before seeking partnerships with industry and regulators.
• Bengio argues that non-agentic systems can advance scientific research while avoiding self-preservation and manipulation risks in AI.