AI news and explainers at 7 AM weekdays, plus a Sunday weekly at 8Get it in your inbox›
AtlasVLA: Persistent World-Ego State Modeling for Vision-Language-Action Models
arXiv:2608.067292026-08-06
Giving a robot with only a wrist camera a memory of 'where things were' and 'what it already did'
Robot arms that see through a single wrist-mounted camera lose track of objects once they leave the frame and lose track of task progress during multi-step jobs. AtlasVLA fixes this by continuously fusing momentary 2D views into a persistent 3D map and by separately tracking what the robot has already accomplished. Across LIBERO, RLBench simulations and real-world trials, it beat baselines that use multiple cameras despite relying on only one.
METAL LAB explanatory visual
AtlasVLA's dual-memory architecture
Evidence statusMeasured results reported
Wrist camera inputOnly a single 2D frame arrives at each moment; depth estimation and camera pose convert it into 3D coordinates.
Persistent World State MemoryContinuously fuses 2D observations into a 3D voxel map so objects out of frame are still remembered.
Ego-Working State MemoryLearnable queries compress the robot's task progress and intent into a compact running summary.
Dual retrievalPulls the currently relevant world context and task-progress context from the two memories.
DiT action generationUses both retrieved contexts as conditioning to generate the robot's next 7-dimensional action (pose plus gripper state).
An explanatory diagram made by METAL LAB, not a figure supplied by the paper's authors.
What they did
Current Vision-Language-Action (VLA) models react only to the immediate camera frame, so with a wrist-only camera they forget object locations once objects leave view and forget task progress during long multi-step jobs.
AtlasVLA lifts each 2D wrist-camera frame into 3D space using depth estimation, continuously fusing it into a voxel-hashed (small 3D grid cell based) 4D 'Persistent World State Memory,' while a separate 'Ego-Working State Memory' condenses what tasks have already been completed.
Both memories are retrieved and fed as conditioning into a diffusion transformer (DiT, a network that generates precise outputs by gradually removing noise), which produces the robot's next actions.
Using only a wrist camera, AtlasVLA reached 97.6% average success on LIBERO and 70.8% on RLBench, outperforming baselines that use multiple camera views.
On the demanding long-horizon LIBERO-Long benchmark it improved success rate by 9.4 percentage points, and in real-world long-horizon tasks by 17.5 percentage points, both compared to prior models such as π0.
Figure 1: The dual bottlenecks of current reactive VLAs versus the advantage of AtlasVLA. (A) Partial observation: Wrist-only VLAs suffer from limited FoV and partial observation during manipulation. (B) Task-progress forgetting: Reactive baselines lack temporal context, forgetting completed task progress and state. (C) Our solution: Our AtlasVLA integrates a 4D Persistent World State for spatial awareness and an Ego-Working State for task tracking, ensuring robust execution.
Table 1: Quantitative comparison on the LIBERO benchmark. Success rates (%) are averaged over 50 trials per task across five suites, each containing 10 tasks. For methods without LIBERO-90 results, we report the average over the first four suites.
Method
Cameras
Spatial
Object
Goal
Long
90
Average
OpenVLA (15)
3rd
84.7
88.4
79.2
53.7
73.5
75.9
π0 (1)
3rd
90.8
91.8
89.6
80.2
–
88.1
4D-VLA (42)
3rd
93.8
92.8
95.6
86.5
–
92.2
CogACT (19)
3rd
97.2
98.0
90.2
88.8
92.1
93.2
MemoryVLA (35)
3rd
98.4
98.4
96.4
93.4
95.6
96.5
π0 (1)
3rd + wrist
96.8
98.8
95.8
85.2
–
94.2
OpenVLA-OFT
3rd + wrist
97.6
98.4
97.9
94.5
–
97.1
GE-ACT (21)
3rd + wrist
98.2
97.6
95.8
94.4
–
96.5
CogACT (19)
wrist
96.4
95.8
88.6
86.2
87.4
90.9
π0 (1)
wrist
94.4
96.6
90.8
80.8
–
90.7
MemoryVLA (35)
wrist
96.2
99.2
96.4
87.6
90.7
94.0
AtlasVLA (Ours)
wrist
99.4
99.8
98.2
94.6
95.8
97.6
Figure 2: Overall Architecture of AtlasVLA. Relying solely on wrist-camera images Itw, AtlasVLA extracts visual tokens Xtw and lifts them into 3D via depth Dtw and extrinsics 𝐓tex to form an instantaneous world state memory mt. This memory dynamically updates the persistent world state memory ℳt−1 via spatial neighborhood fusion and a temporal sliding window. Concurrently, an ego-working memory condenses context into ego-working tokens via learnable queries Qego, updating an ego-working memory bank through memory consolidation. Finally, world state and ego-working retrievals extract contexts Ctworld and Ctego from the dual memory banks to condition the world-ego-guided action DiT, yielding robust actions At.
Table 2: Quantitative comparison on the RLBench benchmark. Success rates (%) are reported on validation over 20 episodes.
Method
Cameras
Sweep to Dustpan
Phone on Base
Umbrella Out
Frame off Hanger
Wine at Rack
Water Plants
Avg.
OpenVLA (15)
3rd
50.0
20.0
35.0
15.0
10.0
10.0
23.3
CogACT (19)
3rd
50.0
50.0
55.0
45.0
30.0
25.0
42.5
FiS-VLA (5)
3rd
55.0
50.0
50.0
70.0
55.0
20.0
50.0
MemoryVLA (35)
3rd
50.0
60.0
75.0
60.0
80.0
55.0
63.3
π0 (1)
3rd + wrist
30.0
30.0
30.0
70.0
10.0
30.0
33.3
GE-ACT (21)
3rd + wrist
10.0
15.0
40.0
35.0
40.0
45.0
30.8
CogACT (19)
wrist
40.0
35.0
50.0
35.0
20.0
25.0
34.2
MemoryVLA (35)
wrist
40.0
55.0
65.0
60.0
60.0
50.0
55.0
AtlasVLA (Ours)
wrist
70.0
70.0
80.0
65.0
75.0
65.0
70.8
Figure 3: Qualitative results of AtlasVLA on real-world long-horizon tasks.
Table 3: Quantitative results on the real-world general tasks. We report average success rates (%) over 50 trials for each task.
Method
Cameras
General Tasks
Pepper on Plate
Pepper in Box
Stack Cubes
Carrot on Plate
Cube in Drawer
Can in Drawer
Avg.
π0 (1)
3rd + wrist
68.0
60.0
62.0
74.0
66.0
70.0
66.7
MemoryVLA (35)
3rd
72.0
64.0
66.0
78.0
70.0
74.0
70.7
MemoryVLA (35)
wrist
62.0
56.0
58.0
70.0
62.0
66.0
62.3
AtlasVLA (Ours)
wrist
78.0
72.0
76.0
84.0
82.0
80.0
78.7
Figure 4: Our real-world robotic platform.
Table 4: Results on the real-world long-horizon tasks. We report average success rates (%) over 50 trials for each task.
Method
Long-horizon Tasks
Change Cubes
Stack Cubes Order
Clean Desk
Pick Place Order
Avg.
π0
54
50
52
52
52.0
MemoryVLA
62
58
60
62
60.5
AtlasVLA (Ours)
74
66
68
70
69.5
Figure 5: Qualitative results of AtlasVLA on real-world long-horizon tasks.
Table 5: Ablation study on LIBERO and real-world tasks.
No.
Methods
LIBERO
Real-world Long
1)
w/o World State Memory
93.5
54.0
2)
w/o Ego-Working Memory
95.0
56.5
3)
AtlasVLA
97.6
69.5
4)
w/o World State Update
94.6
58.0
5)
w World State Update
97.6
69.5
6)
w/o Spatial PE
96.4
67.5
7)
w/o Temporal PE
96.8
65.0
8)
Spatio-Temporal PE
97.6
69.5
9)
w/o World State Conditioning
95.2
61.5
10)
w World State Conditioning
97.6
69.5
Figure 6: Qualitative results of AtlasVLA on real-world general tasks.
Table 6: Training and model hyperparameters for AtlasVLA.
Hyperparameter
Value
Global batch size
256 (32 × 8)
Learning rate
2×10−5
Action chunk size
16
Inference denoising steps
10
CFG scale (classifier-free guidance)
1.5
Condition dropout
0.1
Capacity of world state memory
2048
The voxel size of the world state
0.025m
Number of ego-working memory retrieval layer
2
Number of ego-guided world retrieval layer
4
Number of ego-working tokens
4
Figure 7: Qualitative results of AtlasVLA on LIBERO Benchmark.
Table 7: Quantitative analysis of runtime efficiency and memory footprint. The inference latency and action generation throughput are measured to evaluate the feasibility of real-time robotic deployment.
Methods
Latency (s)
Throughput (Hz)
GPU Memory (GB)
Success rate (%)
MemoryVLA (35)
0.146
109.5
16.7 GB
62.3
AtlasVLA (Ours)
0.158
101.3
18.1 GB
78.7
Figure 8: More Qualitative results of AtlasVLA on long LIBERO-10 Benchmark.
Table 8: Ablation study on the memory length of the ego-working state across LIBERO and real-world tasks.
No.
Memory length
LIBERO
Real-world Long
1)
8
97.3
66.4
2)
16 (Ours)
97.6
69.5
3)
32
97.2
69.8
Table 9: Ablation study on the voxel size of the world state across LIBERO and real-world tasks.
No.
Voxel size
LIBERO
Real-world Long
1)
0.01
96.3
65.7
2)
0.025 (Ours)
97.6
69.5
3)
0.05
97.2
64.0
4)
0.1
95.9
58.5
Findings
On LIBERO, AtlasVLA reached 97.6% average success using only a wrist camera, 3.4 percentage points higher than π0 which uses multiple views.
On the hardest LIBERO-Long suite it reached 94.6% success, 7.0 percentage points above MemoryVLA.
On RLBench it reached 70.8% average success, 15.8 percentage points above MemoryVLA under the same wrist-only setting.
In real-world general tasks it reached 78.7% average success and in real-world long-horizon tasks 69.5%, up to 17.5 percentage points above π0.
Inference latency rose only marginally to 0.158s (from MemoryVLA's 0.146s), sustaining 101.3Hz throughput suitable for real-time control, while GPU memory usage was 18.1GB, only 1.4GB more than MemoryVLA.
Where it can be used
Industrial or service robot arms where mounting multiple cameras is impractical, enabling multi-step tasks like sorting, assembly, or ordered object placement with a single wrist camera
Automation in cluttered workspaces (drawers, cabinets) where target objects frequently leave the camera's view
Task-planning modules for household or warehouse robots that must remember long action sequences
Limits and open work
Evaluations are limited to a Franka robot arm and specific settings (tabletop, cabinets), so generalization to other robot embodiments or more complex outdoor environments is unverified.
Hyperparameters such as memory length (16) and voxel size (0.025m) were tuned as optimal for these benchmarks and may need re-tuning for different task scales.
Performance depends on the accuracy of the depth estimation model (Depth Anything v3), and behavior with reflective or transparent objects that are hard to estimate depth for was not reported.
Real-time latency and memory usage were measured only on a specific GPU (A100) setup; performance on lower-spec hardware is not confirmed.
Why it matters
Many real robots can't afford or physically fit multiple cameras, so a method that remembers out-of-view spots and task history from a single wrist camera could make complex, multi-step automation practical in constrained setups. This points toward more reliable robots for logistics, household, or industrial tasks where sensor placement is limited.
Terms in this paper
Vision-Language-Action (VLA) model · An AI model that turns camera images and language instructions into robot arm control commands
voxel hashing · A way of dividing 3D space into small grid cells (voxels) and efficiently storing/updating only the ones that matter
diffusion transformer (DiT) · A neural network that starts from random noise and gradually refines it into a precise output, here robot actions
sliding window · A memory management approach that discards very old information while keeping a recent span of data
wrist camera · A camera mounted on the robot arm's end-effector that moves along with the arm
Original abstract (English)
While Vision-Language-Action (VLA) models have advanced embodied AI, their fundamentally reactive paradigm severely limits performance in partially observable and long-horizon tasks. When restricted to a single wrist-mounted camera, they inevitably suffer from perception forgetting as objects exit the field of view, and temporal task-progress forgetting} during multi-step execution. To overcome these bottlenecks, we propose AtlasVLA, a novel framework that transitions from direct reactive manipulation to proactive reasoning through a persistent world-ego state. AtlasVLA features a dual-memory architecture: a 4D Persistent World State Memory that lifts transient 2D observations into a globally updated, voxel-hashed spatial state to resolve visual blind spots, and an Ego-Working State Memory that tracks historical ego state and task progress. By conditioning a diffusion transformer (DiT) on this joint World-Ego state, AtlasVLA enables robust spatial reasoning. Extensive evaluations across LIBERO, RLBench, and real-world benchmarks demonstrate that AtlasVLA achieves state-of-the-art performance using solely a wrist camera. Remarkably, it decisively outperforms multi-view baselines, yielding absolute success rate improvements of 9.4% on LIBERO-Long and 17.5% in real-world long-horizon tasks.
