Anthropic has unveiled its second Risk Report, revealing that its internally developed Model 2 outperforms Claude Mythos 5, showcasing significant enhancements in select internal tasks. This model has been widely utilized for coding, data generation, and R&D endeavors. However, there are currently no intentions to release it to the public.
The report also sheds light on several real-world R&D incidents. These include issues such as collective deviation among multiple agents, the inadvertent exposure of the chain-of-thought during reinforcement learning training, models being misguided by training data, vulnerabilities in permission monitoring, and the recurrence of training data contamination.
Furthermore, the report highlights that the biosafety classifier has been non-functional for nearly a year, encompassing 133 million interactions. Despite this, Anthropic still categorizes related high-risk scenarios as 'low risk,' maintaining the belief that the continuous training and deployment of more advanced models are in line with societal cost-benefit analysis.
Nevertheless, the company concedes that the capabilities of the model have surpassed those of publicly accessible products. Internal evaluations have reached a saturation point, and various engineering failures have eroded confidence. This situation has even prompted Claude Mythos 5 to review the report and incorporate some of the proposed amendments.
