Sam Altman warns of risks from self-learning AI

OpenAI has slowed work on an advanced AI model after its behavior during training raised safety concerns, CEO Sam Altman said on the Sources podcast.

Altman pointed to rapid advances in reinforcement learning, which are allowing models to develop new capabilities faster than researchers can build reliable ways to control them. OpenAI encountered unexpected behavior while testing one of its frontier systems and ultimately chose to scale back training, using some of the computing capacity instead to work on new safety measures.

For Altman, the episode reflects a broader change in where the biggest AI risks may emerge. The industry has spent years worrying about what users could do with powerful models once they were released, but increasingly capable systems are creating new problems earlier in the development process. Keeping AI behavior aligned with developers’ intentions, alongside improving cybersecurity, will be among the most pressing issues for AI companies over the next year, he said.

OpenAI already uses its Preparedness Framework to assess whether increasingly capable models pose unacceptable risks before they are deployed or scaled further. The internal system tracks capabilities in areas including cybersecurity and autonomy and can halt a release when a model crosses predefined safety thresholds.