OpenAI's long-term safety team disbands
Add Axios as your preferred source to
see more of our stories on Google.
/2024/05/17/1715973160111.gif?w=3840)
Illustration: Brendan Lynch/Axios
OpenAI no longer has a separate "superalignment" team tasked with ensuring that artificial general intelligence (AGI) doesn't turn on humankind.
Why it matters: The non-profit firm — founded to protect the world from the gravest threats AI could pose — is looking more and more like an impatient Silicon Valley startup cranking out new products at warp speed.
- The company said the work the superalignment team performed would now be more deeply integrated with OpenAI's research teams — a change that began "weeks ago."
Driving the news: OpenAI co-founder Ilya Sutskever, co-leader of the superalignment team from its start last year, left the company earlier this week.
- So did the team's other leader, Jan Leike, who sharply criticized the company on his way out.
What they're saying: "Over the past years, safety culture and processes have taken a backseat to shiny products," Leike said Friday in a thread on X.
- Leike criticized the ChatGPT maker for failing to invest enough computing resources in "figuring out how to steer and control AI systems much smarter than us."
The other side: OpenAI CEO Sam Altman posted that Leike was "right we have a lot more to do; we are committed to doing it. i'll have a longer post in the next couple of days."
- On Saturday morning OpenAI co-founder and president Greg Brockman wrote a longer response to Leike on X claiming that the company was willing to delay releases for safety concerns.
- "Figuring out how to make a new technology safe for the first time isn't easy," Brockman wrote in the post that was signed "Sam and Greg."
- "We believe both in delivering on the tremendous upside and working to mitigate the serious risks," he added.
The intrigue: Other AI leaders, including Google and Meta, have also had reorgs that spread or scattered the members and work of safety and responsibility teams.
- The companies have all argued the benefit of having this work take place throughout the company rather than in a silo.
- Critics say that a dedicated team is the best way to make sure safety work gets the resources it needs and has a seat at the decision-making table.
Catch up quick: In the AI industry, "alignment" refers to efforts to make sure that AI programs share the same goals as the people who use them.
- "Superalignment" typically refers to making sure that the most advanced AGI projects that OpenAI and others in the industry hope to develop do not turn on their creators.
The bottom line: Companies and leaders who want to slow down AI development because they foresee existential dangers are in retreat across the AI industry — and those who believe in flooring the pedal and cleaning up any messes later are in charge.
Editor's note: This story was updated to include an new response from Greg Brockman and Sam Altman.
