Jacob Coxon, a key contributor to the development of GPT-4o, has stepped down from his position at Anthropic and made the decision to leave the AI industry entirely. Previously, he transitioned from OpenAI to Anthropic, attracted by the latter's esteemed reputation for prioritizing safety. Coxon noted that OpenAI did not fully acknowledge the risks that AI could pose at a civilization level. Meanwhile, although Anthropic was aware of these risks, it found itself unable to halt its progress amid the fierce competition in the AI sector. He believes that, given the current competitive landscape within the industry, companies face significant challenges in balancing safety considerations with the need for rapid development. This situation, he argues, necessitates government intervention and a collective industry-wide slowdown. Otherwise, he warns, AI could spiral out of control as early as the end of next year. This could lead to models gaining the ability to self-improve, escaping human oversight, and even refusing to comply with human instructions.
Meanwhile, Jakub Pachocki, Chief Scientist at OpenAI, also acknowledged that no laboratory currently has the capacity to adequately address the challenges of AI alignment and monitoring. He proposed a dual-track response strategy and called for the implementation of more robust safety regulations. Earlier in July, 1,386 prominent AI practitioners, including Coxon and Pachocki, signed a joint statement urging the establishment of an international mechanism to tackle the acceleration of AI self-improvement. Notably, the head of Safeguards research at Anthropic also resigned this year, citing similar concerns. The central issue of AI safety has evolved from a debate over which company is more responsible to a broader question of whether the entire industry can collectively apply the brakes to ensure safety.