AI Launches Preparedness Team to Mitigate Catastrophic Risks from Advanced AI Models

Led by MIT's Aleksander Madry, the newly formed "Preparedness" team focuses on tracking, forecasting, and protecting against various threats such as nuclear, biological, and cybersecurity risks and will develop a robust "Risk-Informed Development Policy".

OpenAI has established a new team, called 'Preparedness', to focus on assessing and mitigating threats stemming from advanced artificial intelligence models. With potential perils including nuclear, biological, and cybersecurity risks, the team will formulate a Risk-Informed Development Policy encompassing preventive measures and governance structures.
The Preparedness team will be helmed by Aleksander Madry, an authority from MIT. His role involves hiring professionals to research national security threats and explore impactful research areas. The salary range for these positions, according to job listings, lies between $200,000 and $370,000 annually.
High-ranking executives and innovators in the technology industry have expressed concerns regarding AI safety. Co-founder of OpenAI, Elon Musk previously described AI as 'one of the biggest risks to the future of civilization.' Meanwhile, OpenAI CEO Sam Altman empathizes with those fearful of AI, citing risks such as disinformation and job replacement.
In a pioneering move, OpenAI is soliciting ideas from the community to help identify potential risks. The top ten submissions stand to win a $25,000 prize and a job at Preparedness. The company is also open to exploring 'less obvious' areas of AI risk.
Within the bounds of AI safety, the Preparedness team will keep track of OpenAI's AI models, ensure adherence to safety guardrails, and foresee 'extinction' level threats like pandemics and nuclear war. The team will also monitor the progress and risks associated with AI models as they advance towards 'artificial general intelligence' or near human-level knowledge.