World Representation
Is there a universally optimal world representation for robot policy learning, or is representation effectiveness fundamentally coupled with the integration paradigm?
World Modeling × VLA
What Matters for World Modeling in Vision-Language-Action Models?
A systematic, controlled empirical investigation of world modeling in VLA policies across diverse world representations, four representative integration paradigms, distinct policy backbones, and diverse task conditions.
4integration paradigms
6world representations
2policy backbones
6evaluation settings
01 — Overview
Beyond reactive
observation-to-action mapping.
Standard VLAs mainly predict actions from the current observation. World models enable policies to anticipate future world dynamics.
Existing approaches vary simultaneously in their world representations, integration paradigms, and policy backbones, making it difficult to determine what truly matters for effective world model integration.
Integration Paradigm
What are the performance trade-offs among different integration paradigms, and is coupled or decoupled world modeling more effective?
Backbone Generalization
Do world modeling integration paradigms remain effective across policy backbones, spanning models without robot pretraining and pretrained foundation VLAs?
Comprehensive Performance
How do different world modeling integration paradigms perform under diverse task conditions?
02 — Unified framework
Four integration
paradigms.
The following foresight strategies share this unified baseline, differing in how they integrate world modeling into action generation.
01Coupled
WM-Align
Implicit Future Alignment
Supervises the VLM's visual representations to capture predictive world dynamics. At inference, the auxiliary predictor is discarded, leaving policy deployment identical to the baseline.
02Coupled
WM-Query
Explicit Future Query
Uses learnable future query tokens to aggregate task-relevant multimodal context into compact predictive representations that explicitly augment the action expert's context.
03Decoupled
WM-MoT
MoT Future Expert
Uses a dedicated Future Expert as a parallel MoT stream. The Action Expert interacts with both the VLM and Future Expert across all intermediate layers.
04Decoupled
WM-Ext
External World Predictor
Decouples world modeling into a standalone module and explicitly conditions action generation on the final predicted world representation.
Shared objective
ℒ = ℒaction + λworld ℒworld
Four future observations at offsets {4, 8, 12, 16} provide identical temporal supervision across all strategies.
03 — Experiments
No single paradigm is
universally optimal.
The decoupled MoT Future Expert offers competitive overall performance, particularly in OOD robustness.
Finding 1
World representation rankings reverse across integration paradigms.
No single world representation universally dominates across architectures; instead, their relative rankings reverse across paradigms.
Finding 2: 2D semantic representations offer robust, cross-paradigm gains. DINOv3 and SigLIP consistently improve both ID and OOD averages across all four paradigms.
World modeling improves overall performance and visual robustness.
WM-MoT provides the largest overall boost and improves across all seven categories.
World modeling yields substantial gains on long-horizon tasks.
Improvements progressively increase with horizon length under Clean2Clean.
Decoupled predictive designs achieve stronger gains in dynamic manipulation.
WM-Ext and WM-MoT reach 15.1% and 15.5% success, respectively.
World modeling consistently improves real-world manipulation performance.
Average success rises from 45.0% to 56.3–73.8% across the four paradigms.
Core results
The key numbers, in full.
Success rates (%). Green and red deltas indicate changes from the corresponding action-only baseline.
Table 01
LIBERO & LIBERO-Plus
Performance across world modeling paradigms.
| Method | LIBERO | LIBERO-Plus | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Camera | Robot | Language | Light | Background | Noise | Layout | Avg. | ||
| OpenVLA-OFT | 97.1 | 56.4 | 31.9 | 79.5 | 88.7 | 93.3 | 75.8 | 74.2 | 69.6 |
| Fast-WAM | 97.6 | 18.8 | 45.7 | 70.1 | 83.2 | 45.7 | 29.8 | 62.7 | 49.1 |
| 4D-WAM | 98.6 | 45.2 | 64.3 | 90.6 | 94.3 | 57.7 | 69.1 | 79.2 | 71.0 |
| Baseline | 96.7 | 48.2 | 51.4 | 91.4 | 96.0 | 92.9 | 65.3 | 71.3 | 71.8 |
| WM-Align | 98.6 +1.9 | 58.0 +9.8 | 49.7 −1.7 | 89.5 −1.9 | 94.6 −1.4 | 93.5 +0.6 | 71.8 +6.5 | 73.9 +2.6 | 74.1 +2.3 |
| WM-Query | 97.6 +0.9 | 54.4 +6.2 | 50.6 −0.8 | 88.5 −2.9 | 95.1 −0.9 | 96.3 +3.4 | 69.5 +4.2 | 69.5 −1.8 | 72.9 +1.1 |
| WM-MoT | 97.9 +1.2 | 59.7 +11.5 | 53.5 +2.1 | 91.6 +0.2 | 97.6 +1.6 | 95.9 +3.0 | 72.3 +7.0 | 75.2 +3.9 | 76.2 +4.4 |
| WM-Ext | 98.2 +1.5 | 55.7 +7.5 | 46.8 −4.6 | 91.2 −0.2 | 96.8 +0.8 | 95.5 +2.6 | 74.2 +8.9 | 75.9 +4.6 | 74.7 +2.9 |
Table 02
RoboTwin 2.0
Performance under Clean2Clean and Clean2Random evaluation.
| Method | Clean2Clean | Clean2Random | ||||||
|---|---|---|---|---|---|---|---|---|
| Short | Medium | Long | Avg. | Short | Medium | Long | Avg. | |
| starVLA | 49.7 | 51.9 | 28.8 | 46.5 | 4.0 | 3.2 | 0.6 | 3.2 |
| AHA-WAM | 60.3 | 76.7 | 59.1 | 64.3 | 2.6 | 5.8 | 1.2 | 3.2 |
| GalaxeaVLA | 63.0 | 71.6 | 48.9 | 62.7 | 11.5 | 19.6 | 6.7 | 12.7 |
| Baseline | 53.4 | 65.0 | 30.7 | 52.3 | 13.4 | 15.4 | 6.6 | 12.7 |
| WM-Align | 62.0 +8.6 | 72.4 +7.4 | 48.8 +18.1 | 62.3 +10.0 | 15.3 +1.9 | 17.9 +2.5 | 8.9 +2.3 | 14.8 +2.1 |
| WM-Query | 61.1 +7.7 | 77.8 +12.8 | 57.0 +26.3 | 64.7 +12.4 | 11.9 −1.5 | 15.8 +0.4 | 10.3 +3.7 | 12.6 −0.1 |
| WM-MoT | 60.3 +6.9 | 74.1 +9.1 | 52.9 +22.2 | 62.6 +10.3 | 12.6 −0.8 | 20.1 +4.7 | 9.8 +3.2 | 14.0 +1.3 |
| WM-Ext | 58.9 +5.5 | 73.8 +8.8 | 47.2 +16.5 | 60.7 +8.4 | 10.9 −2.5 | 16.4 +1.0 | 7.9 +1.3 | 11.8 −0.9 |
Table 03
DOMINO Level 1
Performance on dynamic manipulation.
| Method | SR | MS |
|---|---|---|
| InternVLA-M1 | 5.4 | 27.6 |
| Isaac-GR00T | 6.1 | 28.6 |
| OpenVLA-OFT | 9.1 | 24.1 |
| π0.5 | 9.6 | 26.2 |
| Baseline | 10.6 | 29.1 |
| WM-Align | 11.9 +1.3 | 30.1 +1.0 |
| WM-Query | 13.5 +2.9 | 32.0 +2.9 |
| WM-MoT | 15.5 +4.9 | 32.9 +3.8 |
| WM-Ext | 15.1 +4.5 | 33.0 +3.9 |
Table 04
π0.5 on RoboTwin 2.0
Performance under Clean2Clean and Clean2Random evaluation.
| Method | Clean2Clean | Clean2Random | ||||||
|---|---|---|---|---|---|---|---|---|
| Short | Medium | Long | Avg. | Short | Medium | Long | Avg. | |
| π0.5 | 67.9 | 72.5 | 68.2 | 69.1 | 45.6 | 53.3 | 39.0 | 46.4 |
| + WM-Align | 64.1 −3.8 | 66.8 −5.7 | 57.6 −10.6 | 63.6 −5.5 | 39.9 −5.7 | 47.2 −6.1 | 35.9 −3.1 | 41.0 −5.4 |
| + WM-Query | 67.8 −0.1 | 72.4 −0.1 | 66.0 −2.2 | 68.7 −0.4 | 45.4 −0.2 | 50.7 −2.6 | 42.3 +3.3 | 46.2 −0.2 |
| + WM-MoT | 69.6 +1.7 | 76.8 +4.3 | 68.6 +0.4 | 71.3 +2.2 | 49.6 +4.0 | 51.5 −1.8 | 47.2 +8.2 | 49.7 +3.3 |
| + WM-Ext | 71.5 +3.6 | 74.9 +2.4 | 67.9 −0.3 | 71.7 +2.6 | 48.1 +2.5 | 52.4 −0.9 | 44.1 +5.1 | 48.5 +2.1 |
Finding 5
Decoupled predictive designs achieve stronger gains in dynamic manipulation.
Dedicated predictive modules can more flexibly capture continuous object motion than shared token-wise alignment or bottleneck queries.
Finding 6: Decoupled paradigms retain stronger performance under data scarcity. WM-Ext achieves the highest in-domain sample efficiency, while WM-MoT provides the strongest out-of-distribution robustness.
96.5%WM-Ext · 50% data · LIBERO
62.4%WM-MoT · 25% data · LIBERO-Plus
Finding 7
Decoupled designs show better compatibility with the pretrained VLA.
Decoupled architectures transfer more favorably to the pretrained backbone. In contrast, WM-Align imposes auxiliary predictive objectives directly onto the shared visual representations, which can distort the pretrained feature space and disrupt the manipulation priors established during pretraining.
Coupled
Decoupled
Table 05
Real-world manipulation success rates (%)
Physical Franka Research 3; 40 trials per method and task.
| Method | Static tasks | Dynamic tasks | Overall | ||||
|---|---|---|---|---|---|---|---|
| Box cover | Corn rack | Avg. | Pick tomato | Tomato plate | Avg. | ||
| Baseline | 62.5 | 60.0 | 61.3 | 37.5 | 20.0 | 28.8 | 45.0 |
| WM-Align | 77.5 | 85.0 | 81.3 | 47.5 | 25.0 | 36.3 | 58.8 |
| WM-Query | 67.5 | 65.0 | 66.3 | 57.5 | 35.0 | 46.3 | 56.3 |
| WM-MoT | 87.5 | 77.5 | 82.5 | 75.0 | 55.0 | 65.0 | 73.8 |
| WM-Ext | 90.0 | 67.5 | 78.8 | 70.0 | 42.5 | 56.3 | 67.5 |
04 — Conclusion
Design principles for integrating world modeling.
-
01
World Representation
2D semantic representations provide reliable gains across paradigms, whereas pixel and 3D representations are more sensitive to integration architectures.
-
02
Integration Paradigm
Coupled designs offer compact representations for static manipulation, while decoupled architectures provide superior robustness under distribution shifts and dynamic manipulation.
-
03
Backbone Generalization
Decoupled integration shows better compatibility when adapting pretrained foundation VLAs.
05 — Qualitative Real-Robot Rollouts
Representative rollouts
for four tasks.
Representative physical-robot rollouts for the action-only baseline and four world modeling paradigms.
Each task is shown in the fixed order: Baseline, WM-Align, WM-Query, WM-MoT, and WM-Ext.