AdaWorldPolicy: World-Model-Driven Diffusion Policy with Online Adaptive Learning for Robotic Manipulation

1. Quick overview of the paper

World Model Diffusion Policy Flow Matching Test-Time Adaptation Force Feedback LoRA

Figure 1. AdaWorldPolicy's closed loop: Mode I generates actions and executes them; Mode II predicts the future based on the same observation and action; the error between the real future and imagined observation drives LoRA online update.

core contribution

• Unified multimodal diffusion framework.The world model, action model, and force predictor are all implemented as Flow Matching DiT and exchange information via MMSA.
• AdaOL was adapted online during testing.The model self-supervisedly updates a small number of LoRA parameters based on the gap between real feedback and imagined future to reduce visual and physical domain shift.
• Access force-torque feedback.Force Predictor predicts future force readings for handling dynamic force shift in contact-rich tasks.
• Multiple benchmark verification.Covers LIBERO-10, Variant PushT, CALVIN and 4 real robot tasks, and adds sampling step, MMSA, future frame visualization and implementation details in the appendix.

2. Motivation and related work

2.1 Why ordinary VLA is not enough

The paper points out that although the VLA model can combine language, vision and action, it usually relies on a large number of human demonstrations and has limited generalization in unseen or dynamically changing contact-rich scenes. The fundamental reason is that most of them are reactive mappings trained offline: the current observation comes in and the action is directly output, lacking a mechanism to explicitly predict the physical consequences and use real feedback to correct itself.

2.2 Why should we put the world model into a closed loop?

There are existing world models that are often used as "digital twin" or offline validators in robots; there are also WorldVLA, UVA, etc. that unify action generation and world prediction. However, the author believes that most of these methods are still offline training strategies and cannot adapt quickly when faced with visual and dynamic changes during deployment. The core motivation of AdaWorldPolicy is that the prediction error of the world model itself is a self-supervised signal, which can continuously correct the action model, world model and force predictor during testing.

3. Detailed explanation of method

3.1 Problem setting

The multi-modal historical observation $o=\{x_{\text{static}}, x_{\text{gripper}}, f\}$ is input at each moment, including static camera sequence, gripper camera sequence and force-torque readings. The three share the context length $T_c$. The model outputs the future action sequence $a=a_{t: t+T_a-1}$. The goal is to use its own interactive data $\{(o_t, a_t, o_{t+1})\}_{t=0}^{T}$ to update the parameters $\theta_t$ to $\theta_{t+1}$ in a test environment without adding new manual annotations or demonstrations.

Figure 2. Network architecture: World Model is based on Cosmos-Predict2 2B; Force Predictor and Action Model are 0.4B Flow DiT; the three exchange features through Multi-modal Self-Attention, and are supported by LoRA for online updates.

3.2 Three modules

3.3 Dual mode: same network, two roles

Figure 3. Mode I generates action sequences; Mode II conditions executed actions and predicts future observations. This switchable mode is the basis of AdaOL.

3.4 Multi-perspective expansion of World Model

The paper extends Cosmos-Predict2 from single-view video prediction to multi-view input: each camera view first obtains tokens through Cosmos VAE, and then splices along the temporal dimension; different views use independent RoPE to maintain the spatial and temporal structure across views. The input token is also equipped with a binary mask, where 1 represents the known condition and 0 represents the prediction target; after final denoising, the condition part will be replaced by the original input to ensure that the condition observation is not contaminated by the model generation results.

3.5 Multi-modal Self-Attention

MMSA is the bridge between the three modules. It is not a simple concat, nor is it a one-way cross-attention, but allows world / force / action to generate QKV respectively, and then performs self-attention after splicing the token dimension. In this way, the three can query each other's information while retaining their own dedicated representation space.

3.6 AdaOL online adaptation loop

AdaOL runs a closed loop after each execution: generate an action, execute it, receive real feedback, imagine future results of the same action, calculate the error, update a small number of parameters with LoRA. The author emphasizes that only low-rank matrices are updated, the proportion is less than 0.1%, and the online update overhead is controllable.

4. Mathematical forms and training objectives

4.1 Action Generation loss

4.2 Future Imagination loss

4.3 Joint objective

5. Experiments and results

5.1 Experimental setup

Figure 4. Three simulation benchmarks: Variant PushT, LIBERO, and CALVIN.

5.2 LIBERO-10

LIBERO-10 results prove that the AWP architecture itself is already strong without online adaptation: static-only super UVA, full multimodal super MODE and OpenVLA-OFT.

5.3 Variant PushT: OOD robustness

AdaOL is best seen here: AWP is already stronger than most baselines, but online learning continues to bring improvements on all OOD variants, especially random lighting from 0.71 to 0.77, and random color from 0.61 to 0.66.

5.4 CALVIN ABC→D

The increments for online learning on CALVIN are smaller but consistent: average length 3.51 to 3.54, with length 5 remaining at 48.0. The author explains this as TTA being able to fine-tune policies that have generalized well.

5.5 Real robot tasks

Figure 5. Real robot setup: INOVO robotic arm, static/gripper camera and force sensor; tasks include sweeping beans, putting eggs, pouring water, and wiping whiteboards; tablecloth, distractor, object, lighting, etc. domain shift is shown on the right.

Figure 6. Real-world results: AWP under in-domain has outperformed DP-Force and UVA for most tasks; AWP(ol) under domain shift has a stable improvement compared to AWP. In the figure, Pour goes from 80% to 90% under Object Change, which is consistent with the example in the text.

The appendix complements the real-world protocol: 150 expert demonstrations collected in-domain per task; 30 trials per model configuration per task/distribution, up to 1500 execution steps. AdaOL testing is divided into two phases: Trials 1-15 are continuously updated online, and Trials 16-30 freeze the updated model to evaluate the stability of the adapted policy.

5.6 Ablation

5.7 Appendix supplement: sampling steps, imagined future and super parameters

Appendix Figure. Imagined future versus real observation in simulation: PushT, CALVIN, and LIBERO10.

Appendix Figure. Imagined future in real scenes; the authors point out that artifacts and blur will appear in scenes with complex backgrounds and small objects, but the structural consistency is still sufficient for policy.

6. Repeat audit

6.1 Public resource status

6.2 Key hyperparameters

Real-world use of sparse prediction: imagined frames are only 4, while the action horizon is 32, covering the action time span with sparse future frames to reduce video generation latency.

6.3 Data processing and force data

• In the real task, 150 expert demonstrations were collected in each in-domain environment, using PS5 controller teleoperation, and mapping the force sensor readings to handle vibrations to provide feedback to the operator.
• Force readings do not have stable upper and lower bounds like pixels or relative poses; the appendix explains the use of quantile-based normalization, that is, scaling by 1st and 99th percentile to mitigate outlier/spike effects.
• DP baseline has also been enhanced to DP-Force: concat 6D force data and image observation features, and then fuse them through Transformer cross-attention to ensure fair comparison of real experiments.

6.4 Recurring gaps

• The complete code, model weights and data download path are not provided; the reproducibility framework can only be described based on the paper.
• The values in the real-world graph come from histograms, and the main text does not have a trial-by-trial table; the appendix only gives the protocol and success criteria, but does not have a complete real rollout table.
• The specific fine-tuning configuration of Cosmos-Predict2 depends on the original implementation; the paper states "largely follow training recipe", but does not list all scheduler, augmentation, and LoRA target module details.
• The online update measured about 5% slowdown on RTX 5880, but did not give details on the absolute latency and batch/sequence padding of each step.

7. Analysis, Limitations and Boundaries

7.1 The most valuable part of this paper

The most valuable point is to turn the world model error into an online learning signal, instead of just using the world model as a rollout visualizer or offline evaluator. The PushT OOD table, real-world domain shift diagram and ablation in the main article all point to the same Affiliations: when visual or physical conditions change, AWP(ol) is continuously modified through real feedback, which is more stable than a fixed offline policy.

7.2 Why the results hold up

The evidence chain of the paper covers two levels: "offline capability" and "online adaptation": LIBERO/CALVIN shows that the base AWP architecture itself is strong; PushT OOD and real domain shift show the increment brought by AdaOL; ablation shows that removing world supervision, force predictor or MMSA all decrease significantly. The sampling-step and imagined-future visualizations in the appendix further illustrate that the futures generated by the world model, while not photo-perfect, are structurally usable as supervision.

7.3 Limitations and future directions described by the author

• OOD generalization of world models is still not perfect.The appendix clearly states that Cosmos-Predict2 will experience future frame quality degradation under significant domain shifts, continuously changing lighting, and complex real-world scenes.
• AdaOL hyperparameters are fixed.The author does not adjust AdaOL hyperparameters for each task or environment; adaptive or meta-learning can be used to automatically adjust them in the future.
• Long-range planning failures remain to be addressed.The conclusion states that long-horizon planning failures will be further addressed in the future and extended to larger network architectures.

7.4 Applicable boundaries

AdaWorldPolicy is suitable for scenarios where there is continuous observation feedback, a small amount of update overhead can be tolerated during testing, and the world prediction error is related to the success or failure of the task. For scenarios that require strict safety constraints, cannot update parameters online, or have weak correlation between visual future prediction and real control objectives, the paper does not provide sufficient verification.