Jacob Coxon, a pretraining researcher who previously spent three years at OpenAI, announced his resignation from Anthropic and said he was leaving the AI industry. He argued that leading labs are racing toward self-improving superintelligence while accepting risks they believe are serious, because stopping could leave the lead to a less cautious competitor. He also said decisions about such capabilities should not be made by private companies alone. Evan Hubinger, Anthropic’s alignment science lead, publicly agreed that Coxon’s criticism was valid. Hubinger said he personally assigns a probability above 10% to AI killing all humans within the next decade, while stressing that this is a subjective estimate, not a company forecast or measurement. He added that Anthropic is trying, but lacks a solution to superintelligence alignment and is not clearly on track to find one. Anthropic has not issued a corporate response.

TNW | Anthropic

An Anthropic researcher quit saying AI labs are gambling with our lives

Jacob Coxon resigned from Anthropic saying AI labs are gambling with our lives. Alignment science lead Evan Hubinger replied that he is correct, an...