AI staff 'genuinely frightened' for humanity's future, ex-Anthropic researcher tells BBC

Anthropic CEO Dario Amodei has publicly urged a significant slowdown in the development of artificial intelligence, citing growing concerns over the rapid advancement of "superintelligent" systems and their potential risks to humanity. Amodei warned that if left unchecked, AI could pose a threat such as autonomously taking over the internet within six to twelve months. His call for increased prudence comes as experts and former employees express alarm over the industry's fast pace.

This plea for caution is amplified by the recent resignation of Jacob Coxon, a former AI researcher at Anthropic and OpenAI, who accused both companies of "gambling with our lives" in their pursuit of self-improving AI models. Coxon's fears are echoed by other current and former AI employees who believe that the technology's rapid progress is outpacing the ability to implement adequate safety measures, potentially leading to catastrophic outcomes, including human extinction.

Amodei's proposal includes a three-part plan, with Anthropic unilaterally committing to providing third-party evaluators with "employee-like access" to their systems to verify safety. This initiative seeks to allow safety measures to catch up with technological capabilities and has garnered support from other prominent figures in the AI industry, including OpenAI CEO Sam Altman and xAI owner Elon Musk.