Table 1: Selected structural budgets. CC and ρ are percentages; ΔInv-F1 is measured relative to Ωfull after full-data training.
Model
KL/L
KG/G
CCL
CCG
ρ
ΔInv-F1 (%)
Ql
20/36
3/8
97.7
47.3
79.2
−0.2
Qs
20/28
7/8
99.8
99.3
37.5
−0.7
(b) Attention from subsequent conversation tokens onto resources, normalized and aligned by relative position.
Table 2: Resource-wise attention ablation.
Method
Inv-F1
ID-F1
Halluc.
TTFT (ms)
Dense
82.4
96.0
0.0
26.319
Block
82.2
95.8
0.1
–
Ωfull
82.3
96.0
0.2
7.200
Figure 2: ReCache constructs reusable KV blocks for each Ri and progressively reduces their subsequent cache retention and access. (a) Resource-wise attention removes inter-resource attention and resets positions from zero within each Ri. (b) Structural pruning exposes resource KV states (using Mresource) only through selected layer (Li)–KV-head-group (Gi) routes Ω⋆ (highlighted in blue). The remaining routes use Mcontext. (c) Semantic pruning retains resource names, argument (arg) names and descriptions (desc.), and the final suffix token. The final mask combines these decisions, ensuring that Y attends exclusively to retained semantic fields within assigned routes during decoding.
Table 3: Results on 𝒯IND. Efficiency is reported relative to Dense. SMP denotes Semantic Pruning.
Method
Attn. ×
Mem. ↓
Inv-F1
ID-F1
Halluc.
Dense
–
–
82.4
96.0
0.0
Ωfull
1.001×
0.47%
82.3
96.0
0.2
Structural pruning
Ω20,G
1.016×
44.71%
82.5
95.7
0.3
Ωfull + SPEED
1.013×
44.71%
79.3
92.9
2.8
Ω20,3
1.314×
79.27%
82.1
95.6
0.2
Ωfull + SA20,3
1.302×
79.27%
79.1
93.7
1.0
Semantic pruning
Ωfull + Gist
1.029×
99.22%
39.2
46.9
51.4
Ωfull + Beacon
1.020×
75.42%
78.6
92.5
1.4
Ωfull + SMP
1.021×
63.57%
81.6
95.2
0.2
ReCache
1.423×
92.43%
80.3
94.9
0.2
Figure 3: Structural contribution coverage in Ql and Qs. Cumulative leave-one-in contributions of layers (top) and KV head groups (bottom); vertical lines mark the selected budgets.
Figure 4: Efficiency on varying resource lengths Dℛ. Requests are grouped by average resource length into Small (0≤Dℛ<1K), Medium (1K≤Dℛ<5K), Large (5K≤Dℛ<10K), and XL (Dℛ≥10K). SMP denotes semantic pruning.
Table 5: Dataset composition and split statistics. Source rows report eligible examples before final selection; the Toucan aggregate is not double-counted in the summary row. Distinct names counts source-level unique declared tools or skills, Subset dup. counts records beyond the first occurrence of an identical candidate configuration, and Recurring names counts resources appearing in multiple records. The summary row corresponds to 18.8% duplicated candidate records and 77.3% recurring resource names. IND cand. and OOD cand. denote eligible in-distribution and out-of-distribution candidates, and dashes mark unavailable statistics.
Group
Source
Train pool
Distinct names
Subset dup.
Recurring names
IND cand.
OOD cand.
Tool
ToolACE
8,400
14,566
178
7,144
200
195
APIGEN
10,000
2,806
734
2,805
500
146
Toucan (all generators)
13,071
4,355
6,901
4,355
600
493
- Kimi-K2
3,333
1,103
1,772
1,103
200
145
- GPT-OSS-120B
4,184
1,412
2,162
1,412
200
130
- Qwen3-32B
5,554
1,840
2,967
1,840
200
218
ToolRet
1,150
1,593
235
431
100
150
ToolMind
20,000
14,504
1,853
14,497
500
190
WildToolBench
256
–
–
–
–
–
Skill
SkillRouter
87
–
–
–
–
–
Sources with frequency statistics
52,621
37,824
9,901
29,232
1,900
1,174
Candidate pool
52,964
–
–
–
1,900
1,174
Final dataset
49,424
–
–
–
1,000
1,000
(b) TPOT
Table 6: Structural budget ablation for Qs across training data regimes with comparable budgets.
Config
Inv-F1
Resource ID
Halluc.
ID-P
ID-R
ID-F1
Ωfull
72.1
92.0
90.8
90.9
0.4
ΩL,5
49.6
74.5
73.5
73.4
4.8
ΩL,6
53.4
79.6
77.9
78.3
3.2
ΩL,7
57.3
83.4
81.8
82.1
2.9
Ω15,G
69.3
89.2
88.3
88.3
1.1
Ω20,G
70.5
90.6
89.5
89.6
1.1
(c) Attn.
Table 7: Structural budget ablation trained on 𝒟.
Config
Inv-F1
Resource ID
Halluc.
ID-P
ID-R
ID-F1
𝑸𝒔
Ωfull
80.7
94.9
94.3
94.4
0.1
Ω20,5
78.8
94.1
93.5
93.6
0.5
Ω20,7
80.0
94.7
94.0
94.1
0.3
𝑸𝒍
Ωfull
82.3
96.4
95.8
96.0
0.2
Ω20,2
81.9
95.3
94.7
94.7
0.3
Ω20,3
82.1
96.1
95.5
95.6
0.2
(d) Mem.
Table 8: Semantic pruning ablation trained on 𝒟subset. Arg stands for argument, Desc. stands for description. Ω20,3 denotes the routing configuration with KL=20,KG=3.
Config
Inv-F1
Resource ID
Halluc.
ID-P
ID-R
ID-F1
Ω20,3
76.1
93.1
92.8
92.5
0.9
Schema representation ablation
Suffix-only Schema
14.8
22.8
23.0
22.9
76.0
+ Resource Name
45.2
89.9
88.8
88.9
1.8
+ Arg Name
67.7
92.1
91.1
91.2
1.3
+ Resource Desc.
67.9
92.2
91.4
91.4
1.1
+ Arg Desc.
72.8
92.8
91.9
92.0
1.0
+ Both Desc.
73.4
93.0
92.2
92.2
1.2
Table 9: Individual contribution weights and cumulative contribution coverage of structural units. Within each panel, units are ranked independently by decreasing wi, and CC at rank k is the cumulative positive contribution weight through rank k. Bold entries fall within the empirical budgets KL=20,KG=3 for Ql and KL=20,KG=7 for Qs.
Agentic language models repeatedly encode tool and skill schemas that recur across requests in different combinations and orders, preventing standard prefix caching from reusing their key--value (KV) states. We introduce \textbf{ReCache}, a framework for independently caching resource representations while reducing their inference-time computational and memory overhead. Resource-wise attention removes cross-resource interactions and assigns resource-local positions, producing composition-invariant KV blocks. ReCache then restricts resource visibility to contribution-selected layer--KV-head-group routes and retains only invocation-critical fields through structural and semantic pruning. We evaluate ReCache on a benchmark assembled from seven public tool- and skill-use datasets, including resource-disjoint tests. Resource-wise attention matches dense invocation performance (82.3\% versus 82.4\% Inv-F1) while providing a 3.655$\times$ time-to-first-token speedup. The complete framework reduces allocated KV-tensor memory by 92.43\% and accelerates attention by 1.423$\times$. These results show that separating reusable schema encoding from selective resource access substantially reduces agentic inference costs with limited effectiveness loss. The code is available at https://github.com/EIT-NLP/ReCache.
