Figure 1: RestoreKV narrows the gap to full-cache behavior under aggressive compression at negligible one-time cost. Applied to KVzip+, RestoreKV reaches 86.4 RULER accuracy at 16× compression on the KVPress Benchmark while adding negligible compression-time overhead.
Table 1: Generalization across KV cache eviction methods. RestoreKV improves five different base methods on Qwen3-4B under matched KV budgets, with larger gains typically observed at tighter budgets. Colored subscripts indicate absolute changes from the corresponding baseline.
Method
RULER-4K
QASPER
QuALITY
LongHealth
r=0.2
r=0.1
r=0.05
r=0.2
r=0.1
r=0.05
r=0.2
r=0.1
r=0.05
r=0.2
r=0.1
r=0.05
KVzip
91.4
80.1
38.2
38.2
33.9
18.0
66.0
58.5
46.3
71.5
70.0
61.0
KVzip + Ours
93.5 +2.1
88.8 +8.7
73.2 +35.0
39.0 +0.8
37.7 +3.8
28.7 +10.7
67.0 +1.0
63.3 +4.8
52.4 +6.1
75.8 +4.3
73.8 +3.8
67.5 +6.5
KVzip+
91.3
84.8
51.6
38.1
32.4
23.3
66.5
60.0
51.9
73.2
67.8
63.2
KVzip+ + Ours
93.3 +2.0
89.7 +4.9
70.7 +19.1
40.9 +2.8
38.5 +6.1
31.6 +8.3
66.8 +0.3
62.9 +2.9
57.9 +6.0
74.2 +1.0
73.0 +5.2
71.5 +8.3
ContrastKV
91.6
80.7
38.0
40.1
31.9
17.8
66.2
62.2
50.7
74.2
69.0
49.0
ContrastKV + Ours
92.3 +0.7
84.3 +3.6
40.2 +2.2
40.9 +0.8
36.0 +4.1
21.6 +3.8
67.6 +1.4
63.3 +1.1
51.9 +1.2
75.2 +1.0
70.5 +1.5
54.0 +5.0
SnapKV
33.8
20.6
12.7
22.4
19.0
14.3
53.9
53.0
42.5
41.0
36.8
36.2
SnapKV + Ours
37.7 +3.9
26.3 +5.7
14.3 +1.6
29.9 +7.5
24.0 +5.0
20.7 +6.4
59.4 +5.5
51.8 −1.2
49.9 +7.4
51.8 +10.8
41.8 +5.0
38.5 +2.3
H2O
8.0
3.5
3.2
22.0
13.8
14.4
59.2
50.4
41.5
60.2
48.8
37.0
H2O + Ours
17.3 +9.3
11.6 +8.1
5.7 +2.5
30.4 +8.4
24.7 +10.9
21.3 +6.9
60.9 +1.7
54.9 +4.5
47.7 +6.2
64.8 +4.6
59.0 +10.2
46.8 +9.8
Figure 2: Overview of RestoreKV. After context prefill, a single LoRA-adapted restore pass processes n restore tokens (n=8 by default) with causal access to the full KV cache, generating nLH context-conditioned KV pairs. These restore states are combined with the retained context states under the same total KV budget. LoRA is active only during restore-cache generation; subsequent query processing and decoding use the original frozen backbone. Training (offline): The restore-token embeddings and LoRA adapters (0.4% of the 4B backbone) are optimized by self-distillation from the frozen full-cache teacher, Inference: The trained restore tokens and LoRA adapters generate the restore cache once for each new context.
Table 2: Component and LoRA-target ablations on RULER-4K (Qwen3-4B). The first row is KVzip without a restore cache; the last row is the full RestoreKV configuration. “Fixed” replaces all eight learned restore-token embeddings with the embedding of the line-break token \n.
KV budget ratio r
Restore emb.
LoRA target
#Params
0.2
0.1
0.05
–
–
–
91.4
80.1
38.2
Learned
–
0.02M (0.0005%)
92.7
82.6
42.1
Learned
q/k/v
4.0M (0.1%)
93.4
88.2
72.4
Learned
q/k/v/o
5.9M (0.15%)
93.5
88.4
72.0
Fixed
q/k/v/o+MLP
16.5M (0.4%)
93.5
88.0
71.9
Learned
q/k/v/o+MLP
16.5M (0.4%)
93.5
88.8
73.2
Figure 3: Performance across four model backbones and four benchmarks. RestoreKV and RestoreKV+ are applied to KVzip and KVzip+, respectively, under the same KV budget. Both methods reduce compression-induced performance loss, with larger gains under tighter cache budgets.
Table 3: Effect of full-context conditioning on RULER-4K with Qwen3-4B. Generating the restore cache from the full cache before eviction adds 8.8 points at r=0.05 over generating it from the already evicted cache.
KV budget ratio r
Method
0.2
0.1
0.05
KVzip (n=0)
91.4
80.1
38.2
Generated from evicted cache
93.7
87.6
64.4
Generated from full KV cache
93.5
88.8
73.2
Figure 4: Comparison with a Synthesized-Cache Baseline on Qwen3-4B. RestoreKV+ consistently improves token-eviction baselines and remains competitive with AM-fast, a strong synthesized-cache method, at ∼13× lower compression time on 4K-token contexts.
Table 4: Ablation on the number of restore tokens (RULER-4K, Qwen3-4B). All variants share the same training recipe and total KV budget.
KV budget ratio r
Method
0.2
0.1
0.05
n=0 (KVzip)
91.4
80.1
38.2
n=1
93.0
85.3
65.3
n=2
93.4
87.6
70.4
n=4
93.4
87.0
68.5
n=8 (default)
93.5
88.8
73.2
n=16
93.3
87.8
69.1
Figure 5: Layer-wise context-attention recovery on RULER-4K (Qwen3-4B, r=0.05). KVzip substantially suppresses attention from the final query token to non-sink context KV pairs. RestoreKV recovers part of the lost attention (shaded), increasing the average context-attention mass from 0.56% to 1.30% and reducing KL divergence from the full-cache model by 16% (→0.167).
Table A: Ablation on the training ratio range (RULER-4K, Qwen3-4B). RestoreKV is trained by sampling the retention ratio from 𝒰(rmin,rmax). The first row reports the KVzip baseline without restoration.
Training ratio range
KV budget ratio r
rmin
rmax
0.4
0.2
0.1
0.05
-
-
93.5
91.4
80.1
38.2
0.025
0.25
94.3
93.5
88.8
73.2
0.025
0.50
94.1
93.1
86.1
63.5
0.050
0.25
94.3
93.2
86.5
65.6
0.050
0.50
94.0
92.8
84.5
51.5
Figure 6: Efficiency analysis (Llama-3.1-8B-Instruct, 32K context). (a) Restore-cache generation adds only ∼0.04 s (0.5%) and 84 MB (0.4%) on top of any base scorer. (b) Attention latency and KV memory are nearly identical to the base method.
Table B: Effect of the query-position offset (RULER-4K, Qwen3-4B). Offset-only KVzip applies the same +8 query offset as RestoreKV without adding restore states. Its performance remains close to standard KVzip, suggesting that the positional offset alone is unlikely to account for RestoreKV’s improvement.
Variant
Restore states
Query offset
KV budget ratio r
0.2
0.1
0.05
KVzip
No
0
91.4
80.1
38.2
Offset-only KVzip
No
+8
90.5
80.8
36.9
RestoreKV
Yes
+8
93.5
88.8
73.2
Figure A: RestoreKV composes with Fast KVzip, a learning-based scorer (RULER-4K, Qwen3-8B). At 16×, Fast KVzip drops to 46.7, RestoreKV restores it to 79.3, above plain KVzip (59.0).
Table C: Effect of training–inference evictor mismatch (RULER-4K, Qwen3-4B). The inference evictor is fixed to KVzip, while RestoreKV is trained using the evictor in the first column.
Training evictor
Inference evictor
KV budget ratio r
0.20
0.10
0.05
-
KVzip
91.4
80.1
38.2
SnapKV
KVzip
93.5
81.6
42.0
KVzip
KVzip
93.5
88.8
73.2
Figure B: SCBench results on long-context tasks (Qwen3-4B).
Table D: LongBench results over 16 tasks under aggressive KV compression. Categories are single-document QA (S-QA), multi-document QA (M-QA), summarization (Summ), few-shot learning (Fewshot), synthetic tasks (Synth), and code tasks (Code). Rel. denotes the average score relative to the full KV cache.
Method
S-QA
M-QA
Summ
Fewshot
Synth
Code
Avg
Rel.
Full KV cache
Llama-3.1-8B-Instruct
44.6
47.6
29.2
54.0
55.1
48.1
46.4
100.0%
KV budget ratio = 0.125
KVzip
42.7
43.9
27.7
61.9
44.0
47.7
44.7
96.3%
KVzip + Ours
43.6
43.0
28.5
63.9
45.9
49.3
45.7
98.4%
KV budget ratio = 0.0625
KVzip
33.3
32.4
24.0
49.1
20.6
41.6
33.5
72.1%
KVzip + Ours
38.2
35.4
27.0
59.3
22.8
43.2
37.7
81.2%
Table E: RestoreKV training data statistics (context length in tokens).
Source
Samples
Ctx (mean)
Ctx range
LongAlpaca
2,488
7,306
4,961–15,237
PG-19
2,260
3,071
3,071–3,072
Tulu-3 FLAN
1,500
898
512–9,283
Total
6,248
4,236
512–15,237
Table F: Experimental environment. Hardware and software configurations used for all experiments.
Hardware
GPU
NVIDIA RTX PRO 6000 Blackwell (96 GB)
CPU
Intel Xeon Gold 6530
Memory
503 GiB
Software
OS
Ubuntu 22.04.5 LTS
Framework
PyTorch 2.8.0 (CUDA 12.8)
Table G: Sensitivity to Training Seeds. RULER-4K accuracy on Qwen3-4B, reported as mean±std over three training seeds.
KV budget ratio
0.40
0.20
0.10
0.05
KVzip
93.46
91.38
80.08
38.23
RestoreKV
94.25±0.04
93.37±0.11
88.21±0.51
72.81±0.57
Table H: Evaluation metrics used for each benchmark.
Benchmark
Evaluation metric
RULER-4K
Official string-matching score
QASPER
Maximum token-F1 over reference annotations
QuALITY
Answer-letter accuracy
LongHealth
Answer-letter accuracy
LongBench
Official per-task metric (F1, ROUGE-L, or accuracy)
Query-agnostic KV cache eviction compresses a context once and reuses the resulting cache for arbitrary future queries, but performance can collapse under tight budgets. Existing methods primarily improve which original KV pairs are retained. We introduce RestoreKV, which complements this selection-based formulation with learned restoration under the same total KV budget. Our key insight is that, although the information lost through eviction is context-specific, the mechanism for generating its
