RynnValue: Scaling Robotic Value Foundation Models with Temporal Distance
Abstract
RynnValue is a scalable open-source value foundation model for robot manipulation that uses temporal distance instead of preferences or progress to learn generalizable value predictions and improve real-world policy success.
General-purpose reward models are increasingly the bottleneck for scaling robot learning, yet the recipe for learning value-related capabilities from large-scale heterogeneous corpora remains underexplored. Existing approaches tie supervision to task-internal anchors such as preferences or normalized progress, none of which transfer cleanly across embodiments and data sources. We introduce RynnValue, an open-source value foundation model for robotic manipulation that replaces these anchors with temporal distance, the directed cost-to-go from an observation to the language-specified goal. Because temporal-distance labels can be derived directly from timestamps, RynnValue scales to over 7,000 hours and roughly 3M instruction-conditioned clips without preference or progress annotations. To make temporal-value learning reliable at scale, we combine random temporal sampling, temporal-order shuffling, and value-isolation attention, suppressing shortcuts that would leave predictions insensitive to failures and regressions. Trained without preference labels, RynnValue attains an average Kendall's tau_a of 0.675 on RBM-EVAL-OOD, surpassing the fully preference-supervised state of the art (0.655) and more than doubling a progress-only counterpart (0.292), while generalizing zero-shot to unseen tasks, embodiments, and viewpoints. Converted into dense rewards via potential-based shaping, it raises real-world policy success from 52.5% to 72.5% online and from 63.8% to 82.5% offline. These results establish temporal distance as a scalable supervision target and practical reward interface for generalist robot policies.
Community
Model weights and code are available at
Github: https://github.com/alibaba-damo-academy/RynnValue
HuggingFace: https://huggingface.co/collections/Alibaba-DAMO-Academy/rynnvalue
Modelscope: https://www.modelscope.cn/collections/DAMO_Academy/RynnValue
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- World Value Models for Robotic Manipulation (2026)
- RARM: Confidence-Gated Progress Reward Modeling for RL in Manipulation (2026)
- Beyond Monotonic Progress: Retry-Supervised Value Learning for Robot Imitation (2026)
- WARP-RM: A Warp-Augmented Relative Progress Reward Model for Data Curation (2026)
- WorldScape Policy 2.0: Empowering Steerable World Action Modeling with Reasoning-Augmented Memory (2026)
- STEAM: Self-Supervised Temporal Ensemble Advantage Modeling for Real-World Robot Learning (2026)
- DenseReward: Dense Reward Learning via Failure Synthesis for Robotic Manipulation (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2608.09853 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 2
Alibaba-DAMO-Academy/RynnValue-8B
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper