AI news and explainers at 7 AM weekdays, plus a Sunday weekly at 8Get it in your inbox›
3DZip: Spatial-Aware Feature Diversity-Guided Token Compression for 3D Question Answering
arXiv:2608.011852026-08-01
A 3D question-answering AI keeps 94.7% of its accuracy after cutting its visual tokens from about 1,400 down to just 128
3D vision-language models build tokens by projecting multi-view camera images into 3D space, but this creates thousands of tokens per scene and heavy compute cost. 3DZip compresses these tokens in three steps: grouping nearby tokens into voxels, picking feature-diverse representative tokens with a Determinantal Point Process (DPP), and merging remaining tokens into nearby representatives. Across three 3D question-answering benchmarks it consistently beat existing compression methods, keeping 94.7% of original accuracy with only 128 tokens and running 1.92x faster.
METAL LAB explanatory visual
3DZip's Three-Stage Token Compression Pipeline
Evidence statusMeasured results reported
Input: multi-view 3D tokensThousands of geometry-aware tokens per scene, formed by projecting multi-view 2D image features into 3D world coordinates using depth and camera pose
Stage 1: coarse voxelizationNearby tokens caused by the same surface being seen from multiple viewpoints are averaged within fixed-size 3D grid cells (voxels), reducing point-level duplication
Stage 2: feature-diversity anchor selection (DPP)From the voxelized tokens, a Determinantal Point Process picks feature-diverse anchor tokens that represent distinct objects, reducing repeated representation of the same object
Stage 3: spatially-constrained mergingRemaining non-anchor tokens are merged only into spatially nearby anchors, adding complementary context while preserving geometric consistency
Output: compressed token setA reduced token set (as few as 128, 64, or 32 tokens) is passed to the language model for 3D question answering
An explanatory diagram made by METAL LAB, not a figure supplied by the paper's authors.
What they did
3D vision-language models (3D VLMs) lift 2D visual features from multiple camera views into a shared 3D coordinate system using depth and camera pose, creating thousands of geometry-aware tokens per scene, which the paper identifies as a major compute and memory bottleneck.
The authors show experimentally that spatial grouping alone cannot fix 'object-level redundancy', where the same object produces many tokens from different viewpoints: choosing tokens by spatial diversity (XYZ-DPP) covered only 51% of ground-truth objects, while choosing by feature diversity (Feature-DPP) raised coverage to 70%.
Based on this, 3DZip runs three stages: Stage 1 groups nearby tokens into voxels to remove point-level duplication; Stage 2 uses a Determinantal Point Process (DPP) to pick feature-diverse anchor tokens that represent distinct objects; Stage 3 merges the remaining non-anchor tokens into spatially nearby anchors under a distance constraint to preserve geometric structure.
Using LLaVA-3D as the backbone, 3DZip was compared against 2D-oriented methods (FastV, SparseVLM, VisionZip, VisPruner) and the 3D-aware method DTC, and it achieved the best accuracy at every tested token budget (128, 64, and 32 tokens) on SQA3D, OpenEQA, and ScanQA.
On a single RTX 4090, compared with the uncompressed LLaVA-3D baseline (1,410 tokens, 342 ms/sample), 3DZip with 128 tokens ran in 178 ms/sample (1.92x faster), cut FLOPs by 90.2% and KV-cache size by 86.0%, while accuracy dropped only 2.5 points (55.7 to 53.2 EM).
Figure 1: (a) Per-object token allocation under different selection strategies. Object coverage is defined as the fraction of GT object instances that contain at least one selected token, computed using GT object masks and aggregated over 10 scenes from SQA3D. Spatial sampling (XYZ-DPP) concentrates tokens on large objects (51% coverage), whereas Feature-DPP distributes tokens more evenly across objects, increasing coverage to 70%. (b) 3DZip overview. 3DZip compresses dense 3D tokens to only 128 while retaining 94.7% accuracy and achieving 1.92× faster inference.
Table 1: Comparison of token compression methods. The 3D-Aware column indicates whether spatial geometry is explicitly considered during token selection.
Method
3D-Aware
ScanQA
SQA3D
OpenEQA
Rel.
All 1410 Tokens
LLaVA-3D (ICCV’25)
✓
26.5
55.7
60.3
100.0%
Retain 128 Tokens ↓(9.1%)
FastV (ECCV’24)
✗
21.9
50.9
56.0
88.9%
SparseVLM (ICML’25)
✗
21.8
50.3
53.9
87.3%
VisionZip (CVPR’25)
✗
22.2
51.5
56.2
89.8%
VisPruner (ICCV’25)
✗
22.3
52.0
55.8
90.0%
Voxelization
✓
23.6
51.5
54.6
90.7%
DTC (CVPR’25)
✓
22.1
51.3
54.8
88.8%
3DZip (Ours)
✓
24.2
53.2
58.6
94.7%
Retain 64 Tokens ↓(4.5%)
FastV (ECCV’24)
✗
21.1
49.5
53.8
85.9%
SparseVLM (ICML’25)
✗
20.9
49.8
53.6
85.7%
VisionZip (CVPR’25)
✗
20.0
49.1
53.4
84.1%
VisPruner (ICCV’25)
✗
21.9
50.1
53.9
87.3%
Voxelization
✓
21.9
49.8
54.6
87.5%
DTC (CVPR’25)
✓
21.3
50.2
54.3
86.8%
3DZip (Ours)
✓
23.3
52.8
56.7
92.3%
Retain 32 Tokens ↓(2.3%)
FastV (ECCV’24)
✗
19.9
47.7
52.5
82.6%
SparseVLM (ICML’25)
✗
20.4
48.7
52.2
83.7%
VisionZip (CVPR’25)
✗
19.6
47.0
52.0
81.5%
VisPruner (ICCV’25)
✗
20.9
49.1
53.5
85.2%
Voxelization
✓
20.7
47.9
53.4
84.2%
DTC (CVPR’25)
✓
20.6
48.8
52.6
84.2%
3DZip (Ours)
✓
21.9
51.1
55.7
88.9%
Figure 2: Overview of geometry-aware 3D token construction and the compression objective. Multi-view RGB-D inputs are projected into world coordinates to form geometry-aware 3D tokens 𝒱 via 3D positional embedding. Due to the large token cardinality N=M×N2D, an efficient token compression strategy is crucial.
Table 2: Category-wise OpenEQA results. 3DZip achieves the highest average at all token budgets, with the largest gains in attribute recognition and object recognition.
OpenEQA
Method
attribute recognition
functional reasoning
object localization
object recognition
object state recognition
spatial understanding
world knowledge
Avg.
All 1410 Tokens
LLaVA-3D (ICCV’25)
64.0
62.7
51.5
55.9
73.7
54.2
59.1
60.3
Retain 128 Tokens ↓(9.1%)
FastV(ECCV’24)
57.8
58.9
46.8
47.7
75.6
45.3
59.4
56.0
SparseVLM(ICML’25)
54.0
54.1
48.3
45.9
74.1
44.5
55.5
53.9
VisionZip(CVPR’25)
58.4
57.8
48.4
51.0
69.4
47.2
60.9
56.2
Voxelization
56.3
58.5
47.2
47.2
68.6
49.1
54.9
54.6
DTC(CVPR’25)
56.4
57.7
48.5
46.2
71.5
47.2
55.3
54.8
3DZip (Ours)
64.2
60.0
50.0
53.2
72.5
49.5
60.1
58.6
Retain 64 Tokens ↓(4.5%)
FastV(ECCV’24)
51.2
58.2
47.1
43.2
76.3
43.2
56.7
53.8
SparseVLM(ICML’25)
50.6
57.8
46.5
45.3
74.3
45.2
55.5
53.6
VisionZip(CVPR’25)
51.7
57.4
48.6
44.7
69.4
45.6
56.3
53.4
Voxelization
50.8
61.4
46.2
47.9
71.5
48.5
55.7
54.6
DTC(CVPR’25)
55.5
58.2
45.5
43.2
74.1
47.8
55.0
54.3
3DZip (Ours)
60.9
59.7
48.8
51.2
69.6
48.1
58.0
56.7
Retain 32 Tokens ↓(2.3%)
FastV(ECCV’24)
50.7
58.6
43.9
41.2
73.6
42.6
56.6
52.5
SparseVLM(ICML’25)
50.8
57.4
45.6
40.6
73.3
42.9
54.2
52.2
VisionZip(CVPR’25)
51.0
57.0
45.2
42.6
68.1
44.4
55.6
52.0
Voxelization
50.4
60.3
46.9
43.8
68.8
46.1
57.7
53.4
DTC(CVPR’25)
51.3
56.8
45.1
42.9
72.2
44.2
55.6
52.6
3DZip (Ours)
56.3
59.7
47.4
50.5
70.7
47.7
57.4
55.7
Figure 3: Overview of the proposed three-stage token compression pipeline. Given geometry-aware 3D tokens 𝒱, we first apply coarse voxelization to obtain a spatially reduced set 𝒱vox. We then perform diversity-aware anchor selection via DPP on 𝒱vox to identify semantically representative anchors 𝒜. Finally, spatially-constrained token merging aggregates nearby non-anchor tokens into anchors to produce the compressed set 𝒱′.
Table 3: Component-wise ablation results on SQA3D. (a) feature vs. spatial distance in anchor selection, (b) the effect of varying voxel sizes (δ) in coarse voxelization, and (c) the role of the merging step and spatial constraints.
Method
EM
All 1410 Tokens
LLaVA-3D
55.7
Retain 64 Tokens
Voxelization
49.8
DTC (CVPR’25)
50.2
Spatial distance
50.1
Feature distance
52.8
Retain 32 Tokens
Voxelization
47.9
DTC (CVPR’25)
48.8
Spatial distance
49.0
Feature distance
51.1
Figure 4: Qualitative comparison of retained 3D tokens on OpenEQA. VisPruner tends to allocate many tokens to floor regions, while DTC may miss some key objects in the scene. In contrast, 3DZip retains tokens on multiple semantically important objects and further enriches them through spatially-constrained merging.
(a)
Method
EM
All 1410 Tokens
LLaVA-3D
55.7
Retain 64 Tokens
Voxelization
49.8
DTC (CVPR’25)
50.2
Spatial distance
50.1
Feature distance
52.8
Retain 32 Tokens
Voxelization
47.9
DTC (CVPR’25)
48.8
Spatial distance
49.0
Feature distance
51.1
Figure 5: Additional qualitative examples on SQA3D. This figure illustrates the retained 3D tokens produced by 3DZip across different indoor scenes.
(b)
Method
EM
All 1410 Tokens
LLaVA-3D
55.7
Retain 64 Tokens
w/o Coarse voxelize
51.8
δ=0.1m
52.6
𝜹=0.2m
52.8
δ=0.3m
52.1
Retain 32 Tokens
w/o Coarse voxelize
50.2
δ=0.1m
50.9
𝜹=0.2m
51.1
δ=0.3m
50.6
Figure 6: Additional qualitative examples on OpenEQA. This figure illustrates the retained 3D tokens produced by 3DZip across different indoor scenes.
(c)
Method
EM
All 1410 Tokens
LLaVA-3D
55.7
Retain 64 Tokens
DTC (CVPR’25)
50.2
w/o Merge
52.5
w/o Spatial-Const.
52.3
w/ All
52.8
Retain 32 Tokens
DTC (CVPR’25)
48.8
w/o Merge
50.5
w/o Spatial-Const.
50.8
w/ All
51.1
Figure 7: Additional qualitative examples on ScanQA. This figure illustrates the retained 3D tokens produced by 3DZip across different indoor scenes.
Table 4: Efficiency and accuracy comparison on a single RTX 4090 using SQA3D.
Method
Retain Tokens
FLOPs (T)
Latency (ms/sample)
Cache Size (MB)
EM
LLaVA-3D
1410
9.18
342
722
55.7
FastV (ECCV’24)
128
1.41
191
139
50.9
DTC (CVPR’25)
128
0.90
196
101
51.3
3DZip (Ours)
128
0.90
178
101
53.2
Figure 8: Object-level token allocation across selection strategies on the full SQA3D test scenes. Per-object token counts for 1,729 foreground object instances across 67 ScanNet scenes from the SQA3D test split. The voxel-only distribution (gray) reveals a pronounced long-tail pattern, where a small subset of objects receives a disproportionately large number of tokens. Spatial sampling (XYZ-DPP) reduces the extreme concentration but still allocates tokens to a limited subset of objects, resulting in 47% object coverage. In contrast, Feature-DPP promotes diversity in feature space and distributes tokens more evenly across object instances, improving object coverage to 64%.
Table 5: Generalization across projection-based 3D VLM models. We evaluate 3DZip on two recent projection-based 3D VLMs, Video-3D-LLM and SR-3D. Under identical token budgets, 3DZip consistently outperforms all baselines across both architectures, demonstrating that the proposed compression strategy generalizes beyond the LLaVA-3D model used in the main experiments.
Method
ScanQA
SQA3D
OpenEQA
All 3920 Tokens
Video-3D-LLM (CVPR’25)
29.9
58.4
59.4
Retain 128 Tokens ↓(3.3%)
VisPruner (ICCV’25)
23.6
51.2
53.5
Voxelization
24.6
52.2
52.9
DTC (CVPR’25)
23.7
51.9
53.5
3DZip (Ours)
24.7
53.1
54.6
Retain 64 Tokens ↓(1.6%)
VisPruner (ICCV’25)
22.3
49.2
51.4
Voxelization
22.7
50.0
51.9
DTC (CVPR’25)
22.5
49.2
52.0
3DZip (Ours)
23.8
52.2
54.6
Retain 32 Tokens ↓(0.8%)
VisPruner (ICCV’25)
21.0
48.4
50.2
Voxelization
20.9
48.3
50.3
DTC (CVPR’25)
20.8
49.0
50.0
3DZip (Ours)
23.3
51.8
53.0
(a) Video-3D-LLM [52]
Method
ScanQA
SQA3D
OpenEQA
All 3920 Tokens
Video-3D-LLM (CVPR’25)
29.9
58.4
59.4
Retain 128 Tokens ↓(3.3%)
VisPruner (ICCV’25)
23.6
51.2
53.5
Voxelization
24.6
52.2
52.9
DTC (CVPR’25)
23.7
51.9
53.5
3DZip (Ours)
24.7
53.1
54.6
Retain 64 Tokens ↓(1.6%)
VisPruner (ICCV’25)
22.3
49.2
51.4
Voxelization
22.7
50.0
51.9
DTC (CVPR’25)
22.5
49.2
52.0
3DZip (Ours)
23.8
52.2
54.6
Retain 32 Tokens ↓(0.8%)
VisPruner (ICCV’25)
21.0
48.4
50.2
Voxelization
20.9
48.3
50.3
DTC (CVPR’25)
20.8
49.0
50.0
3DZip (Ours)
23.3
51.8
53.0
(b) SR-3D [9]
Method
ScanQA
SQA3D
OpenEQA
All 2420 Tokens
SR-3D (ICLR’26)
29.6
61.0
61.7
Retain 128 Tokens ↓(5.3%)
VisPruner (ICCV’25)
23.0
51.5
54.3
Voxelization
22.8
51.1
52.2
DTC (CVPR’25)
23.5
51.4
53.5
3DZip (Ours)
24.2
53.2
54.5
Retain 64 Tokens ↓(2.6%)
VisPruner (ICCV’25)
21.9
50.3
52.0
Voxelization
21.1
48.8
49.6
DTC (CVPR’25)
21.1
49.6
52.0
3DZip (Ours)
23.1
51.9
52.6
Retain 32 Tokens ↓(1.3%)
VisPruner (ICCV’25)
20.1
49.1
51.5
Voxelization
19.9
47.9
50.3
DTC (CVPR’25)
19.7
48.1
51.4
3DZip (Ours)
21.5
49.6
51.7
Table 6: 3D dense captioning results on the Scan2Cap benchmark. We compare 3DZip with existing token compression methods under different token budgets.
Method
CIDEr
BLEU-4
METEOR
ROUGE-L
All 1410 Tokens (Baseline)
LLaVA-3D (ICCV’25)
65.8
11.2
15.5
37.0
Retain 128 Tokens ↓(9.1%)
VisPruner (ICCV’25)
43.9
9.5
13.9
33.9
DTC (CVPR’25)
39.9
9.0
13.4
33.7
3DZip (Ours)
49.2
9.6
14.3
34.4
Retain 64 Tokens ↓(4.5%)
VisPruner (ICCV’25)
38.9
9.0
13.6
33.2
DTC (CVPR’25)
36.0
8.8
13.3
33.1
3DZip (Ours)
45.7
9.3
14.0
33.8
Table 7: Additional captioning-oriented metrics on ScanQA. We report BLEU-4, METEOR, ROUGE-L, and CIDEr scores under different token budgets. 3DZip consistently outperforms existing token compression methods across all metrics.
Method
CIDEr
BLEU-4
METEOR
ROUGE-L
EM
All 1410 Tokens (Baseline)
LLaVA-3D (ICCV’25)
85.5
11.2
17.1
43.6
26.5
Retain 128 Tokens ↓(9.1%)
FastV(ECCV’24)
71.8
8.9
14.6
37.4
21.9
SparseVLM(ICML’25)
69.2
9.7
14.0
36.3
21.8
VisionZip(CVPR’25)
73.1
10.9
14.7
37.7
22.2
VisPruner (ICCV’25)
73.1
10.3
14.8
37.7
22.3
Voxelization
75.8
10.5
15.3
39.0
23.6
DTC (CVPR’25)
72.6
10.4
14.8
37.8
22.1
3DZip (Ours)
78.2
11.4
15.6
40.3
24.2
Retain 64 Tokens ↓(4.5%)
FastV(ECCV’24)
68.9
8.0
14.0
36.0
21.1
SparseVLM(ICML’25)
67.4
8.5
13.8
35.5
20.9
VisionZip(CVPR’25)
66.0
8.3
13.7
34.9
20.0
VisPruner (ICCV’25)
71.6
9.0
14.5
37.1
21.9
Voxelization
70.9
10.0
14.6
37.1
21.9
DTC (CVPR’25)
69.5
8.7
14.3
36.6
21.3
3DZip (Ours)
76.1
10.7
15.3
39.0
23.3
Retain 32 Tokens ↓(2.3%)
FastV(ECCV’24)
65.5
8.1
13.6
34.6
19.9
SparseVLM(ICML’25)
65.5
7.8
13.4
34.8
20.4
VisionZip(CVPR’25)
64.9
7.7
13.7
34.7
19.6
VisPruner (ICCV’25)
67.4
8.0
14.0
35.9
20.9
Voxelization
66.9
8.6
13.8
35.4
20.7
DTC (CVPR’25)
67.7
9.0
14.1
35.8
20.6
3DZip (Ours)
70.5
9.7
14.5
37.0
21.9
Table 8: Point-level redundancy under voxel aggregation. PLR (Eq. (21)) is the mean number of tokens surviving per physical point; 1.0 denotes no duplication. PLR drops monotonically with voxel size δ while object purity degrades, and EM peaks at δ=0.2 m.
δ
PLR ↓
Obj. purity ↑
Tok./vox.
EM@64 ↑
w/o
4.74
N/A
N/A
51.8
0.10 m
3.09
0.975
3.1
52.6
0.20 m
2.50
0.915
9.5
52.8
0.30 m
2.25
0.847
19.4
52.1
Table 9: Component-wise ablation on SQA3D. (a) Comparison of diversity-promoting strategies for anchor selection. (b) Effect of different feature-space distance metrics. (c) Effect of the spatial constraint threshold τg in token merging.
Method
EM
All 1410 Tokens
LLaVA-3D
55.7
Retain 64 Tokens
Voxelization
49.8
DTC (CVPR’25)
50.2
FPS-based
52.2
MMDP-based
52.5
DPP-based
52.8
Retain 32 Tokens
Voxelization
47.9
DTC (CVPR’25)
48.8
FPS-based
51.0
MMDP-based
51.0
DPP-based
51.1
(a)
Method
EM
All 1410 Tokens
LLaVA-3D
55.7
Retain 64 Tokens
Voxelization
49.8
DTC (CVPR’25)
50.2
FPS-based
52.2
MMDP-based
52.5
DPP-based
52.8
Retain 32 Tokens
Voxelization
47.9
DTC (CVPR’25)
48.8
FPS-based
51.0
MMDP-based
51.0
DPP-based
51.1
(b)
Method
EM
All 1410 Tokens
LLaVA-3D
55.7
Retain 64 Tokens
Voxelization
49.8
DTC (CVPR’25)
50.2
ℓ1
52.5
ℓ2
52.4
Cosine
52.8
Retain 32 Tokens
Voxelization
47.9
DTC (CVPR’25)
48.8
ℓ1
50.9
ℓ2
50.6
Cosine
51.1
(c)
Method
EM
All 1410 Tokens
LLaVA-3D
55.7
Retain 64 Tokens
(w/o Merge)
52.5
τg=1
52.5
τg=3
52.7
τg=5
52.8
τg=7
52.6
(w/o Spatial-Const.)
52.3
Retain 32 Tokens
(w/o Merge)
50.5
τg=1
50.6
τg=3
50.9
τg=5
51.1
τg=7
51.1
(w/o Spatial-Const.)
50.8
Table 10: Comparison with 2D Diversity-based method. 3DZip consistently outperforms DivPrune, demonstrating that spatial awareness is a necessary complement to feature diversity in 3D token compression.
Method
3D-Aware
ScanQA
SQA3D
OpenEQA
Rel.
All 1410 Tokens
LLaVA-3D (ICCV’25)
✓
26.5
55.7
60.3
100.0%
Retain 128 Tokens ↓(9.1%)
DivPrune (CVPR’25)
✗
22.2
51.6
57.3
90.5%
3DZip (Ours)
✓
24.2
53.2
58.6
94.7%
Retain 64 Tokens ↓(4.5%)
DivPrune (CVPR’25)
✗
20.6
50.6
56.2
87.3%
3DZip (Ours)
✓
23.3
52.8
56.7
92.3%
Retain 32 Tokens ↓(2.3%)
DivPrune (CVPR’25)
✗
19.6
49.3
54.9
84.5%
3DZip (Ours)
✓
21.9
51.1
55.7
88.9%
Table 11: Robustness of Stage 2 to different Stage-1 aggregation strategies (SQA3D). Under an identical 64-token budget, adding DPP-based anchor selection (Stage 2) consistently improves EM across all Stage-1 strategies. All variants exclude Stage 3; “+ DPP” denotes adding Stage 2 on top of the corresponding Stage-1 aggregation strategies.
Method
EM
ConceptFusion [21]
50.3
ConceptFusion + DPP
52.7 +2.4
OpenFusion++ [22]
49.8
OpenFusion++ + DPP
52.3 +2.5
Voxel-mean
49.8
Voxel-mean + DPP
52.5 +2.7
Table 12: Fine-grained performance analysis on ScanQA (EM@64 tokens). Subsets: Fine detail (shape/contour/material/color), Local struct. (part-/local structure), Small obj. (bottom 10% by size). Small obj. purity: fraction of a voxel’s tokens from one small object.
Method
EM@64 ↑
Small obj. purity ↑
All
Fine.
Local.
Small obj.
3DZip (Full)
23.3
30.5
22.4
19.5
0.426
w/o Voxel (S2+S3)
22.8
31.7
22.4
21.6
N/A
Voxel-only (S1)
21.9
29.9
20.1
16.9
0.070
Table 13: Scene-size dependence on SQA3D. At a 32-token budget, small scenes favor smaller δ and tighter τg, large scenes the opposite; the fixed setting (δ=0.2m, τg=5) is the best overall trade-off.
Param.
Small
Overall
Large
Retain 32 Tokens (↓2.3%)
δ=0.1m
54.1
50.9
43.8
δ=0.2m
53.6
51.1
44.6
δ=0.3m
53.5
50.6
43.8
τg=1
54.1
50.6
43.8
τg=3
53.9
50.9
44.9
τg=5
53.6
51.1
44.6
τg=7
53.6
51.1
45.9
Table 14: Latency comparison of token selection methods at K=128. We report the selection algorithm time, total inference latency, and the proportion of the selection overhead relative to the total time.
Method
Retain Tokens
Algorithm Latency (ms)
Total Latency (ms)
Overhead (%)
DTC
128
36
196
18.4
DPP (Ours)
128
18
178
10.1
Findings
At a 64-token budget on SQA3D, 3DZip reached 52.8 EM, ahead of DTC (50.2) and VisionZip (49.1), and it stayed at 51.1 EM even at 32 tokens, outperforming both 2D-based and existing 3D-aware baselines.
3DZip kept the highest relative performance (Rel.) of all compared methods, retaining 94.7% and 92.3% of uncompressed accuracy at 128 and 64 tokens respectively.
On the OpenEQA category breakdown, attribute recognition reached 64.2 at 128 tokens (close to the uncompressed 64.0) and beat the second-best method by 5.8 points; object recognition stayed at 50.5 even at 32 tokens, the largest margin (+6.7) among methods tested.
Switching the DPP kernel from spatial (XYZ) distance to feature distance raised SQA3D EM from 50.1 to 52.8 at 64 tokens, and raised object coverage from 51% to 70% in a 10-scene SQA3D analysis.
On an RTX 4090, latency dropped from 342 ms/sample (uncompressed LLaVA-3D) to 178 ms/sample with 3DZip (a 48.0% reduction), and 3DZip was 9.2% faster than DTC (196 ms) under the same 128-token budget.
Where it can be used
Could be used as a preprocessing step to reduce compute load when an indoor robot or embodied AI needs to answer questions about a 3D scene.
Can be applied on top of an already-trained 3D VLM (e.g., LLaVA-3D) without additional training, mainly to speed up inference.
The authors also tested applying the same approach to other projection-based 3D VLM architectures such as Video-3D-LLM and SR-3D.
Limits and open work
The authors' own analysis shows that Stage 1's coarse voxelization can merge small objects' tokens with nearby objects or background, losing fine-grained detail.
The voxel size (δ) and merge distance threshold (τg) are fixed across all scenes rather than adapted to scene size or object density, which the paper notes may be suboptimal in different environments.
OpenEQA evaluation was limited to the ScanNet subset only, because the original camera trajectory data for the HM3D subset is no longer publicly available.
For questions about small objects or fine shape/material/color detail, a variant without voxelization actually performed better than full 3DZip, marking this as an open failure case.
The paper shows scene-size-adapted hyperparameters could improve results further, but leaves automatic adaptation to scene complexity as future work.
Why it matters
For applications like embodied robots or indoor AI assistants that need to reason about 3D scenes in real time, large numbers of visual tokens make GPU memory and latency a bottleneck, and this work shows a way to cut token counts by over 90% without giving up much accuracy. Because the method is training-free and drops into existing projection-based 3D VLMs, it is a practical reference point for lowering deployment cost of such systems.
Terms in this paper
3D VLM (3D vision-language model) · A model that projects images and depth into 3D coordinates so it can answer text questions about a 3D scene
token · A unit of information the model processes; here, a patch of image content tagged with a 3D location
Determinantal Point Process (DPP) · A probabilistic method for picking a subset of items that are mutually diverse rather than similar
voxel · A single cell in a 3D grid used to divide up space into fixed-size cubes
EM (Exact Match) · An accuracy metric measuring how often the model's answer exactly matches the ground-truth answer
Original abstract (English)
Recent 3D vision-language models (3D VLMs) construct geometry aware tokens by projecting 2D visual features into world coordinates, enabling spatial reasoning for tasks such as 3D question answering. However, this design generates thousands of tokens per scene, resulting in substantial computational and memory overhead. While token compression has been extensively studied in 2D VLMs, existing approaches rely on semantic relevance or attention-based selection that overlook the structured spatial
