Figure 1: Overview of RynnValue. RynnValue is a language-conditioned value model trained on over 7,000 hours of heterogeneous embodied data, comprising roughly 3M instruction-conditioned trajectory clips across diverse embodiments, viewpoints, and task domains. Given a language instruction and a sequence of sampled observations, the model builds an interleaved multimodal sequence of repeated absolute-value and relative-value queries, which RynnBrain encodes in a single forward pass. Two distributional heads then predict the absolute temporal distance to task completion and the signed relative temporal displacement between observations, while the language branch produces video analysis and language-conditioned verification. The resulting temporal values serve as a unified interface for progress estimation, failure detection, and reward specification in robotic reinforcement learning.
Table 1: Composition of the heterogeneous data mixture before subtask expansion. The corpus contains 1.67M original episodes and is further converted into over 3M instruction-conditioned trajectory segments after subtask segmentation and cutoff relabeling.
Data Source
# Original Episodes
# Segmentations
# Instructions
Segmentation Source
AgiBot 3
167,535
1,166,042
3,741
coarse task
EgoDex 10
338,234
338,234
2,038
full trajectory
Galaxea Open-World 13
16,979
95,671
11,070
coarse task
InternData-A1 27
320,905
320,905
348
full trajectory
Open X-Embodiment 23
693,037
693,037
180,090
full trajectory
RDT 17
6,109
6,109
272
per-file coarse task
RoboCOIN 30
67,420
410,877
2,124
coarse task
RoboMIND 29
32,138
32,138
184
full trajectory
RoboTwin 20
27,414
27,414
23,527
full trajectory
Soft-FOLD 35
1,542
1,542
1
per-file coarse task
Total
1,671,313
3,091,969
223,395
–
Figure 2: RynnValue training pipeline and value-isolation attention. (a) Training strategy. Random temporal sampling and temporal-order shuffling suppress shortcuts tied to sampling intervals and sequence position, while instruction-mismatch augmentation strengthens language–visual grounding. RynnValue jointly learns absolute temporal distance, relative temporal displacement, and natural-language supervision. (b) Value-isolation attention. Within each value-query group, repeated queries attend to one another and to the language–visual context, while remaining isolated from other value-query groups. Colored cells denote visible attention connections.
Table 2: Per-dataset trajectory-ranking results on the RBM-EVAL-OOD test suite, measured by Kendall’s τa (↑). Bold values indicate the best overall results. † denotes the best result among methods trained without explicit trajectory-level preference supervision, i.e., progress/value-only methods. Baseline results are taken from Robometer 16.
Method
USC Franka
USC Koch
USC Trossen
USC xArm
MIT Franka
UTD SO101
Average
GVL 19
0.250
−0.008
0.292
0.056
0.306
0.300
0.199
VLAC-2B 34
0.292
0.167
−0.111
0.167
−0.017
−0.033
0.077
VLAC-8B 34
0.271
0.064
−0.417
0.139
0.072
0.167
0.049
RoboDopamine 26
0.167
0.175
0.000
0.014
0.220
0.067
0.107
Dopamine-GRM-2.0-8B-Preview 26
0.479
0.442
0.333
0.431
0.431
0.700
0.453
RoboReward-4B 15
0.625
0.332
0.333
0.528
0.494
0.700
0.502
RoboReward-8B 15
0.625
0.264
0.389
0.347
0.396
0.767
0.465
Robometer (RoboReward data) 16
0.583
0.533
0.646
0.403
0.479
0.667
0.552
ReWiND 33
−0.125
0.336
0.028
−0.167
0.080
−0.067
0.014
Robometer (RBM-1M) 16
0.646
0.471
0.653
0.694
0.601
0.867
0.655
Robometer (Progress only) 16
0.083
0.231
0.333
0.389
0.183
0.533
0.292
RynnValue-4B
0.542
0.488
0.917
0.667†
0.473
0.933†
0.670
RynnValue-8B
0.667†
0.544†
1.000†
0.500
0.503†
0.833
0.675†
Figure 3: Instruction-trajectory confusion matrices. Each cell shows the predicted reward when an instruction (rows) is paired with a trajectory (columns); a well-grounded model concentrates mass on the diagonal. Values below each matrix report the normalized diagonal margin. All models are re-evaluated under a unified protocol from their publicly released weights.
Table 3: Ablation study on RBM-EVAL-OOD. We report Kendall’s τa across six out-of-distribution robot datasets. Shuffle denotes temporal-order shuffling, Isolation denotes value-isolation attention, Language denotes the auxiliary natural-language supervision, Random denotes random temporal sampling, and Relative denotes the relative modeling component.
Design Components
Kendall’s τa
Variant
Shuffle
Isolation
Language
Random
Relative
USC Franka
USC Koch
USC Trossen
USC xArm
MIT Franka
UTD SO101
Average
w/o Shuffle
✗
✓
✓
✓
✓
0.583
0.090
0.055
0.222
-0.017
0.200
0.189
w/o Isolation
✓
✗
✓
✓
✓
0.583
0.428
0.694
0.389
0.400
0.400
0.482
w/o Language
✓
✓
✗
✓
✓
0.250
0.491
0.819
0.361
0.501
0.800
0.537
Uniform Sampling
✓
✓
✓
✗
✓
0.375
0.400
0.305
0.250
0.310
0.633
0.379
w/o Relative
✓
✓
✓
✓
✗
0.667
0.587
0.639
0.639
0.464
0.767
0.627
Full Model (8B)
✓
✓
✓
✓
✓
0.667
0.544
1.000
0.500
0.503
0.833
0.675
Figure 4: Scaling episode volume vs. task diversity. Mean absolute temporal-distance error on a held-out validation set of unseen tasks. We independently scale two aspects of the training set: (orange) episode count with the full task set fixed, and (blue) task count with per-task episode counts fixed. Both curves converge to the same full-scale training set at 100%. Task diversity yields a consistently steeper error reduction than episode volume, which saturates early.
Table 4: Real-world reinforcement-learning results. We report success rates and the average number of action chunks over successful episodes. Average denotes the unweighted mean success rate across four tasks.
Algorithm
Baseline
Bread Basket Placement
Steak Serving with a Spatula
Box-in-Drawer Placement
Bimanual Box Transfer
Average Success
Success ↑
Avg. Steps ↓
Success ↑
Avg. Steps ↓
Success ↑
Avg. Steps ↓
Success ↑
Avg. Steps ↓
Online RL
RynnValue
45.0%
25.9±8.2
75.0%
18.6±13.1
70.0%
27.0±8.1
100.0%
22.8±4.7
72.5%
Robometer
35.0%
22.7±5.5
45.0%
15.2±2.7
65.0%
27.7±5.8
65.0%
25.6±7.4
52.5%
Sparse
40.0%
56.0±31.7
45.0%
18.4±4.9
40.0%
27.4±6.6
70.0%
23.5±2.7
48.8%
Offline RL
RynnValue
100.0%
16.8±3.1
90.0%
14.9±4.0
90.0%
14.9±4.0
50.0%
33.6±10.5
82.5%
Robometer
80.0%
18.9±2.7
80.0%
19.4±7.2
50.0%
27.3±6.3
45.0%
28.7±9.3
63.8%
Sparse
70.0%
26.1±9.1
20.0%
30.2±3.3
0.0%
–
0.0%
–
22.5%
SFT
–
70.0%
24.8±8.0
25.0%
18.6±6.2
0.0%
–
0.0%
–
23.8%
Figure 5: Temporal-value curve comparison on a real-world trajectory. Higher values indicate closer proximity to task completion. The highlighted interval marks a period of task regression where the robot moves away from a productive state; RynnValue responds with a sharp potential drop, whereas Robometer remains relatively flat.
Table 5: Representative annotation issues addressed by data curation. Non-English examples are described rather than reproduced verbatim to avoid introducing non-Roman fonts into the manuscript.
Issue
Representative annotation
Source
Non-English task annotation
Chinese-language manipulation instruction
Galaxea
Placeholder or truncated label
P, shirts, undefined
InternA1
Data-quality metadata
no robot motion, skip frame
OXE
Pure-motion instruction
move to the table, approach the cabinet
Galaxea
Figure 6: Representative demonstrations for real-world evaluation. Each row shows a sequence of observations from one manipulation task: Each row shows a sequence of observations from one manipulation task. These tasks cover object grasping, spatial manipulation, and articulated-object interaction.
Table 6: Summary of source-specific data curation. OXE and InternA1 are counted in episodes, while Galaxea and RoboCOIN are counted in annotated subtask segments. The total therefore denotes an aggregate number of trajectory units rather than a homogeneous episode count.
Source
Raw units
Curated units
Unit retention
Raw instructions
Curated instructions
Instruction retention
OXE (episode)
961,253
693,037
72.10%
180,290
180,090
99.89%
InternA1 (episode)
320,910
320,905
99.99%
350
348
99.43%
Galaxea (segment)
97,287
78,692
80.89%
12,685
10,909
86.00%
RoboCOIN (segment)
343,516
343,516
100.00%
1,642
1,642
100.00%
Total
1,722,966
1,436,150
83.35%
194,967
192,989
98.99%
Table 7: Real-world task instructions and dataset statistics. Success rate is computed over all collected trajectories for each task.
Task
Language instruction
Trajectories
Success rate
Success
Failure
Total
Bread Basket Placement
“Put the two pieces of bread in the basket.”
99
4
103
96.1%
Steak Serving with a Spatula
“Move the steak from the pan to the plate.”
98
4
102
96.1%
Box-in-Drawer Placement
“Put the box in the drawer and close it.”
101
3
104
97.1%
Bimanual Box Transfer
“Move the box from the right side to the left side.”
100
1
101
99.0%
Total
–
398
12
410
97.1%
Table 8: Camera streams used by the policy and RL encoders. Entries list the exact streams provided to each pathway.
Pathway
Single-arm tasks
Bimanual Box Transfer
VLA
left_side, left_wrist
left_side, left_wrist, right_wrist
IQL critic
left_side, left_wrist
left_side, left_wrist, right_wrist
DSRL actor–critic
left_side, left_wrist
left_side, right_side, left_wrist, right_wrist
Table 9: Offline IQL and SFT hyperparameters. All IQL reward variants share the same mixed-expertise dataset, sparse task reward, policy initialization, and optimization configuration. SFT uses the same policy optimizer and learning-rate schedule and is also trained for 10,000 steps per task.
Hyperparameter
Value
Base policy
π0.5 with flow matching; action dimension 32
Action horizon H
16
Batch size
64
Policy optimizer
AdamW with β1=0.9, β2=0.95, ϵ=10−8, weight decay 10−10, and gradient-norm clipping at 1.0
Learning-rate schedule
Cosine decay with 2,000 linear warm-up steps
Peak / final policy learning rate
3×10−5 / 3×10−6
Policy EMA decay
0.99
Training steps per task
10,000
Critic / value optimizer
Adam with a learning rate of 3×10−4
Offline discount γoff
0.99
Target update rate ρoff
0.005
Expectile parameter τe
0.8
Advantage temperature β
10.0
Maximum advantage weight wmax
100
Number of Q-functions KIQL
2; minimum aggregation
Critic and value encoder
ResNet-18 with GroupNorm and spatial softmax; 50-dimensional bottleneck
Critic and value hidden dimensions
(256,256)
Number of critic cameras
2 for single-arm tasks and 3 for the bimanual task
Critic input resolution
224×224 using the VLA preprocessing pipeline
Policy warm-up Nwarm
200 optimization steps with w(o,a)=1
Sparse task reward
−1 before task completion and 0 upon task completion
Potential-based shaping reward
rhshape=γoffΦh+1−Φh
Shaping coefficient κ
0.1 for RynnValue and 1.0 for Robometer; fixed across tasks
Sparse-reward baseline
κ=0
Image augmentation
Random cropping applied to both current and next observations; no color jitter
Table 10: Online DSRL hyperparameters. SAC operates in the latent space of the frozen VLA, which decodes latent variables into executable action chunks. All reward variants share the same optimization configuration and differ only in the potential source and shaping coefficient.
Hyperparameter
Value
Base policy
Frozen SFT checkpoint for Bread Basket Placement and Steak Serving with a Spatula; frozen Robometer offline-RL checkpoint for Box-in-Drawer Placement and Bimanual Box Transfer
Latent action space
z∈[−1,1]H×dz, with H=16 and dz=32
RL algorithm
SAC with automatic entropy tuning and initial temperature α0=1.0
Actor optimizer
Adam with a learning rate of 1×10−4
Critic optimizer
Adam with a learning rate of 3×10−4
Temperature optimizer
Adam with a learning rate of 3×10−4
Gradient clipping
None
Target entropy ℋ¯
−dim(z)
Online SAC discount γon
0.999
Target update rate ρon
0.005
Number of Q-functions KSAC
10; mean aggregation
Actor and critic hidden dimensions
(128,128,128)
Image encoder
Four-layer CNN with 32 channels per layer, strides (2,1,1,1), VALID padding, GroupNorm, spatial softmax, and a 50-dimensional bottleneck
SAC input resolution
64×64
Batch size
256
Update-to-data ratio
100
Training length
6,000 training steps
Online rollout trajectories
60 per task
Replay-buffer capacity
max(training steps/UTD,104)=104
Update frequency
After each episode
Exploration warm-up
Nnoise=2 episodes with Gaussian noise standard deviation σ=0.1
Minimum replay size Nstart
200 transitions
Maximum episode length
600 environment steps
Policy-decision frequency
Low-level control at 10 Hz, with one policy decision every q=10 environment steps
Number of DSRL cameras
2 for single-arm tasks and 4 for the bimanual task
Sparse task reward
−1 before task completion and 0 upon task completion
Shaping coefficient κ
0.1 for RynnValue, 1.0 for Robometer, and 0 for the sparse-reward baseline; fixed across tasks
General-purpose reward models are increasingly the bottleneck for scaling robot learning, yet the recipe for learning value-related capabilities from large-scale heterogeneous corpora remains underexplored. Existing approaches tie supervision to task-internal anchors such as preferences or normalized progress, none of which transfer cleanly across embodiments and data sources. We introduce RynnValue, an open-source value foundation model for robotic manipulation that replaces these anchors with temporal distance, the directed cost-to-go from an observation to the language-specified goal. Because temporal-distance labels can be derived directly from timestamps, RynnValue scales to over 7,000 hours and roughly 3M instruction-conditioned clips without preference or progress annotations. To make temporal-value learning reliable at scale, we combine random temporal sampling, temporal-order shuffling, and value-isolation attention, suppressing shortcuts that would leave predictions insensitive to failures and regressions. Trained without preference labels, RynnValue attains an average Kendall's tau_a of 0.675 on RBM-EVAL-OOD, surpassing the fully preference-supervised state of the art (0.655) and more than doubling a progress-only counterpart (0.292), while generalizing zero-shot to unseen tasks, embodiments, and viewpoints. Converted into dense rewards via potential-based shaping, it raises real-world policy success from 52.5% to 72.5% online and from 63.8% to 82.5% offline. These results establish temporal distance as a scalable supervision target and practical reward interface for generalist robot policies.
