Claude's Self-Developed Inference Engine Cuts First-Token Latency from 12 Seconds to Half a Second
1 day ago / Read about 0 minute
Author:小编   

Researchers from institutions including Carnegie Mellon University have introduced a benchmark called Singularity-Bench, which sets six AI R&D-related exam questions to assess how well agents can autonomously build AI R&D capabilities. In the tests, Claude Opus 5.5 paired with Claude Code delivered outstanding results: its self-developed inference engine, under identical hardware conditions, compressed P90 first-token latency from roughly 12 seconds—achieved by the mature engine SGLang—down to 0.48 seconds, while also boosting processing efficiency and GPU throughput through methods such as optimized scheduling. Ultimately, this combination achieved five wins and one draw across the six tasks, making it the only participating combination whose total score exceeded the human baseline. However, the current tests still require humans to define the task scope, and truly realizing recursive AI self-improvement remains a subject for further exploration.