Sam Altman reveals ‘two ways AI progress could go very badly’ days after backing ‘slowdown’ call

Sam Altman's call to “pace” the development of artificial intelligence does not mean stopping AI progress, the entrepreneur has said. Instead, the OpenAI CEO wants companies building advanced AI models to slow down enough for safety and monitoring systems to keep up.

Sam Altman, chief executive officer of OpenAI Inc, said AI poses two major risks (Bloomberg)He also flagged two major risks of AI -- humans losing control of AI and too much power ending up with one person, company or country.

This came days after Altman, Elon Musk and Anthropic CEO Dario Amodei called for AI companies to “pace the frontier” of increasingly powerful models.

“When we talk about ‘pacing’, we do not mean ‘stopping’. Progress has been rapid and will continue to be,” Altman wrote on X. He added that AI development should be slower than it otherwise could be because safety measures such as evaluations, monitoring and “safety cases” took time and resources.

“Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring,” Altman said.

Altman said the debate over AI safety involved two major risks. The first is the possibility that humans could lose control of increasingly powerful AI systems.

“This is unacceptable; we are unapologetically on Team Humanity, and AI must always serve people,” he said. He argued that safety and alignment techniques must stay ahead of the growth in AI capabilities.

The second risk, he said, was excessive concentration of power. Altman warned that an extremely powerful AI system controlled by one person, company or country could allow that entity to impose its worldview on others.

Avoiding both risks would require a “narrow middle path” between moving too quickly and allowing too much power to accumulate.

Altman said the idea was to make sure AI capabilities did not move faster than humans’ ability to understand and control the risks associated with them.

He said OpenAI now prepared explicit safety cases before some “frontier reinforcement-learning runs” that are expected to significantly increase a model’s capabilities. The company would continue the safety work it already conducted before releasing models.

Altman also backed the use of independent auditors to assess AI systems and said companies should develop shared standards for areas such as misalignment, monitoring and safety.

He said earlier safety frameworks, including responsible scaling policies and preparedness frameworks, were designed mainly around the deployment of completed models. The industry now needed tools that also addressed safety during the development of increasingly capable models, Altman said.

Altman said OpenAI supports a federal framework that would establish consistent safety requirements for frontier AI companies in the US.

He said companies should not wait for legislation before improving safety practices. “We welcome a federal framework that sets consistent safety requirements for frontier AI,” Altman said, adding that the industry could begin providing greater confidence about AI safety on its own.

He said the US government would also need to help with international coordination because AI development was not limited to one country.

ALSO READ | Could AI kill humans? Anthropic CEO Dario Amodei responds

Altman’s comments followed a detailed call from Anthropic CEO Dario Amodei to slow the development of frontier AI models. In his essay We Must Pace the Frontier, Amodei said that AI companies needed to give safety systems more time to catch up with rapidly improving capabilities.

He proposed independent evaluators with employee-level access to AI companies, industry-wide coordination and international cooperation.

Billionaire Elon Musk also backed the call. “Dario is right,” the xAI owner wrote on X.

Altman then said he agreed with Amodei that the industry needed to “pace the frontier” and indicated that OpenAI would also support independent safety evaluators.

Microsoft CEO Satya Nadella also backed the push for safer and more broadly distributed AI development. Nadella said any pursuit of superintelligence must ensure that AI remains “helping humanity and under human control”.

OpenAI, Anthropic and xAI are competing to develop increasingly powerful AI systems.

Follow the latest breaking news, major developments and agenda-setting stories from India and around the world with the newsdesk at Hindustan Times. Operating round the clock, the desk brings together experienced editors, reporters and correspondents to deliver fast, accurate and contextual reporting across subjects that influence public policy, governance, business, society and international affairs. The HT News Desk covers politics, elections, government policies, the economy, business and markets, science and technology, the environment, law and order, infrastructure, education, climate issues and geopolitics, while closely tracking developments across states, institutions and global capitals. The team also leads coverage of major breaking news events, policy announcements, court proceedings, natural disasters, public emergencies and significant international developments. Reports published by the newsdesk are based on information gathered from reporters on the ground, official statements, government agencies, court records, regulatory filings, recognised institutions and other authoritative sources. Stories undergo editorial scrutiny and verification processes to ensure accuracy, fairness and relevance, and are updated as events evolve and additional information becomes available. Whether covering a key political decision in New Delhi, an economic policy shift affecting millions, a landmark court ruling or a major global event, the HT News Desk aims to provide readers with reliable, fact-based journalism that delivers not only the latest developments but also the context and analysis needed to understand their wider implications.Read More