Commentary: Some Anthropic engineers think AI might end us. Why race ahead? - CNA

Anthropic was founded to be more safety-conscious than OpenAI, yet it clings to that mission statement while building ever more powerful agents, says Parmy Olson for Bloomberg Opinion.

Anthropic logo is seen in this illustration taken May 20, 2024. REUTERS/Dado Ruvic/Illustration

LONDON: It’s becoming harder for Anthropic to justify some of the most unfalsifiable circular logic to ever come out of Silicon Valley: the pursuit of artificial superintelligence, which by its own admission could kill a lot of humans.

Anthropic researcher Jacob Coxon recently resigned, saying that his previous employer and its rival OpenAI were gambling with our lives by building systems that could improve themselves. One of the company’s safety directors then set off a mass frenzy online by agreeing.

“Yes, we really do earnestly believe AI could kill all humans,” Alignment Science lead Evan Hubinger wrote in a post, adding that he personally thought there was a more than 10 per cent chance of that happening within a decade. The original tweet by Coxon has been viewed nearly 150 million times.

It’s been known for years that Anthropic’s bosses believe AI could annihilate human civilisation, but Coxon’s and Hubinger’s posts come at an inflection point for the technology. Tech firms say their models have hacked into other systems and deceived their human overseers, and admitted that to make them smarter they must become more difficult to monitor. US citizens, meanwhile, are becoming ever more sceptical about the benefits of AI and data centres.

CNA Games Guess Word Crack the word, one row at a time Buzzword Create words using the given letters Mini Sudoku Tiny puzzle, mighty brain teaser Mini Crossword Small grid, big challenge Word Search Spot as many words as you can Show More Show Less The leaders of OpenAI and chipmaker Nvidia say the software has finally become smarter than humans.