The ByteDance Seed team has pinpointed that the DeepSeek-V4 series models demonstrate notable performance inconsistencies, manifesting in a 4-Token cycle pattern when handling input data. During 128K long-context retrieval evaluations, the accuracy disparity for information located at various positions within the input can surge up to 40.2 percentage points. This irregularity is attributed to the segmented KV Cache compression technique employed in DeepSeek-V4, which inadvertently introduces systematic variations in the model's ability to retrieve information across different segments within the compression window—a phenomenon termed as "phase sensitivity." Essentially, it signifies a cyclical retrieval vulnerability that may arise from the design of segmented compression. Although subsequent fine-tuning and architectural refinements can mitigate the performance gap to some extent, periodic fluctuations remain evident. The team corroborated their findings through meticulously controlled experiments and suggested that when assessing models utilizing segmented KV Cache compression, it is crucial to evaluate information retrieval across diverse compression segments, rather than solely depending on the overall average accuracy as a metric.
