Generalist AI Teaches Robots New Tasks From a 3-Second Demo

GEN-1.5 is a multimodal model that takes in video, sensor data, language, and proprioception (the sense of where your own body is and how much force it's applying) all at once. It holds 30 seconds of memory and outputs action trajectories at 100Hz. The key piece is something Generalist AI calls "physical prompting": you drag and drop sensor data and an actual motion trajectory into that 30-second window, the rest of the window fills with live observations, and the robot performs the task on the spot. No gradient updates that change the weights, no separate fine-tuning, no task-specific programming.

Generalist AI says this ability wasn't something they deliberately engineered. There was no architectural change meant to encourage in-context learning, no meta-learning loop, no auxiliary objective pushing the model toward improvisation. Instead, the company says the ability emerged naturally after more than eight months of continued pretraining on physical interaction data collected from homes, warehouses, and factories. They drew a comparison to how one-shot prompting emerged in OpenAI's GPT-3 purely from pretraining scale, without anyone specifically designing for it.

What the numbers show

Across 10 distinct manipulation tasks, success rates broke down by training method as follows:

Training method	Data used	Success rate
One-shot in-context prompting	1 demonstration, 3–12 seconds	59% (±10%)
Fine-tuning (10 gradient updates)	5 minutes per task, ~50 demonstrations	83% (±9%)
Minimal adaptation (1 gradient update)	1 minute of data	66.5% (held-out task)

What stands out here is the compute involved. Adapting a robot policy to a new task usually takes tens of thousands of gradient steps, but GEN-1.5 saw a major performance jump from just 10. And of those 10 steps, less than 0.15% of the weight values actually changed. That suggests the model isn't building new representations from scratch — it's rearranging knowledge it already has. Generalist AI describes this as "test-time learning with extremely little data."

Learning in simulation, working in reality

Three transfer capabilities GEN-1.5 demonstrated are worth noting. First, when two demonstrations recorded separately from different positions were loaded into the memory window together, the robot generated and stitched together its own repositioning, regrasping, and error-recovery motions — none of which appeared in either original demonstration. Second, even though the pretraining data contained no rendered video or simulated physics at all, demonstrations recorded purely in simulation worked directly on the real robot. That means some tasks may no longer require physically collecting demonstrations at all. Third, the team confirmed human-to-robot imitation: when a person demonstrated a task with their own hands in front of the robot's camera, the robot reproduced it with its own hands.

More interesting generalization showed up after fine-tuning. After training on five minutes of sweeping blocks into a bowl, the robot picked up a banana instead of a brush to sweep with, and in a different setup, it chose an entirely different contact method — scooping the blocks up with a dustpan. It also cleared away paper covering the bowl on its own, and even though it had only been shown the task with one hand, it alternated between both hands during execution.

Still a research release, available only through partnerships

GEN-1.5 isn't something you can try out today. There are no public weights, no API, no pricing, no self-serve access. Generalist AI is currently running the model only within its own robot fleet and data engine, and outside access requires a direct partnership. The company itself acknowledges that the tasks tested so far are simple, short-horizon actions. Still, it says this is the first time, to the team's knowledge, that one-shot learning of physical skills has emerged at this scale.

Editor's take

Most of the numbers we've seen so far in the robot foundation model race have been bragging rights about how much data a model was trained on. When Dyna Robotics released DYNA-2 on August 11, it led with the fact that it pretrained on a million hours of human video — roughly 170 years' worth. Google DeepMind's Gemini Robotics 2 competed on task-specific success rates, like 76.3% for shelf-picking and 36% for light-bulb insertion. GEN-1.5 is playing a different game. Instead of touting total data volume, it's making "how much can be learned from a single demonstration" the headline metric. It's essentially importing the GPT-3 moment from language models — handling new tasks from a handful of examples with no fine-tuning — straight into robotics.

Practically speaking, the value of this approach comes down to adaptation cost. Adapting a robot to a new task used to require tens of thousands of gradient steps and a large volume of demonstration data. GEN-1.5 got from 59% to 83% with just 10 steps. That translates into a real cut in the time and labor needed to teach a robot a new task on a factory floor or in a warehouse — a genuine reduction in the barrier to deploying robots. That said, numbers like 59% and 83% still mean frequent failure. It's too early to put this straight into logistics or manufacturing; for now, it's best read as a research signal.

The thing to watch over the coming months is how quickly other companies catch up on this "physical prompting" approach. If it's true that this kind of ability emerges naturally just from scaling up pretraining, then other robotics startups with similar data pipelines are likely to report the same phenomenon before long.