Summary
- NVIDIA introduced the concept of a World Action Model (WAM), built on video world models rather than vision-language models (VLMs)
- The open model Cosmos 3, a Mixture-of-Transformers architecture trained on hundreds of millions of image, video, and action samples, was presented as the foundation for WAM
- WAM is said to learn physical dynamics first, strengthening zero-shot transfer performance to new tasks, robots, and environments
NVIDIA has proposed the World Action Model (WAM) as a new foundation for robot policies. Existing vision-language-action (VLA) models work by adding an action module to a pretrained vision-language model (VLM); while strong at describing scenes, they were noted to be weak at modeling dynamics — that is, predicting how a scene will change.
WAM replaces this language backbone with a video world model, pretraining on physical changes such as how a cup deforms when grasped or how a towel folds. NVIDIA's research paper, "World Action Models are Zero-shot Policies," explains that jointly predicting video and action gives the model a capability that VLA struggles to achieve: the ability to learn from diverse data.
Cosmos 3, an open model, was presented as the foundation for this approach. Cosmos 3 uses a Mixture-of-Transformers architecture and was trained on a large-scale multimodal dataset containing hundreds of millions of image, video, and action samples. NVIDIA stated that Cosmos 3 supports deployment across multiple tiers, from serving on high-performance workstations to real-time on-device inference using Jetson hardware, and significantly reduces the amount of task-specific data needed to adapt to different robot forms.
Comments