On September 9th, Jacob Coxon, a researcher at Anthropic, made public his decision to step away from the artificial intelligence (AI) sector, driven by concerns that AI laboratories and their rivals are in a headlong rush to create systems that may prove uncontrollable. Coxon, an expert in training new AI models with vast datasets, expressed his reluctance to be part of the broader industry's push towards developing AI systems capable of self-improvement. He fears that such systems could become uncontrollable and pose a significant threat to humanity.

The 27-year-old British mathematician noted that many of his peers in the industry characterize the trend towards self-improving AI models with phrases like "a make-or-break juncture" and "the final showdown." Coxon is of the opinion that, given the current rate of progress, the situation could escalate beyond control by the end of next year. He further commented that in the heat of competition, companies frequently make compromises on safety measures.