During the inference phase of large models, the KV Cache acts as the 'memory' responsible for storing the intermediate computational results of preceding contexts. As the volume of conversations and requests grows, the KV Cache consumes a substantial portion of GPU memory. This leads to reduced concurrency, increased redundant computations, and longer user wait times. To tackle this challenge, a 'memory offloading' strategy can be implemented, delegating memory management tasks to the CPU. In this strategy, the KV Cache is stored hierarchically across various media, including GPU memory, CPU memory, and SSDs, depending on its activity level. By utilizing Intel QAT's specialized hardware for cache compression, we can free up general-purpose CPU core resources. Additionally, storage formats and scheduling strategies are optimized through techniques such as KV Shrink. This method not only reduces the cache size and enhances cache reuse efficiency but also keeps overhead in check, thereby minimizing redundant GPU computations. Consequently, GPUs can concentrate more on generating new tokens, enabling operators to achieve an optimal balance between response time, throughput, and overall cost. Furthermore, technologies like KV Fuse, KV Cascade, and KV Infinity are employed to resolve issues such as cache fusion, content filtering, and long-task cache expansion, respectively.