NVIDIA Achieves Cache Reuse Across Models, DeepSeek and Kimi Optimize KV Cache

In the past two years, optimization of large model inference has focused on KV Cache, which stores intermediate computation results left by the model after reading context. The longer the content, the greater the consumption of memory and bandwidth. DeepSeek-V2 reduces KV Cache by 93.3% using MLA technology, while Kimi Linear achieves up to a 75% reduction. Efforts are being made to lower the costs of long contexts. However, existing solutions have limitations: caches can typically only be reused within the same model, requiring recalculation of tens of thousands or even hundreds of thousands of tokens when switching models, making repeated calculations unavoidable due to frequent model routing. NVIDIA's latest paper proposes a solution, discovering a clear linear relationship in cache between different-sized models that match in family and KV structure. By calibrating with 500 segments of text, each containing 1024 tokens, the KV Cache from one model can be transferred for use in another model. When switching from Qwen3-14B to 32B, recalculating a 32K context takes about 7 seconds, while transferring the cache only takes about 0.28 seconds, making it approximately 25 times faster. This means smaller models can first process long contexts to generate KV Cache, and when more capability is needed, the cache can be transferred to a larger model for generation. The mature application of cross-model KV Cache will save more computing power in model routing, allowing simple tasks to be handled by smaller models while complex problems can call on larger models, avoiding the need to read the context from scratch each time.