学习从键值缓存中驱逐

The increasing size of Large Language Models (LLMs) poses challenges for efficient inference, primarily due to the memory requirements of the autoregressive Key-Value (KV) cache. Existing eviction or compression methods reduce costs but rely on heuristics such as recency or past attention scores, which serve as indirect proxies for a token's future utility and introduce computational overhead. This research reframes KV cache eviction as a reinforcement learning (RL) problem, focusing on learning to rank tokens based on their predicted usefulness for future decoding. To achieve this, the KV Policy (KVP) framework is introduced, consisting of lightweight per-head RL agents trained on pre-computed generation traces using only key and value vectors. Each agent learns a specialized eviction policy.