FlowWAM

TL;DR

World Action Models repurpose pretrained video generators for control, yet a modality gap persists: action signals must conform to the generator's visual priors while preserving the dense cross-frame motion that control requires. FlowWAM closes this gap with optical flow, a unified, video-native action representation that shares the RGB format, encodes spatially grounded per-pixel displacement, and is extractable from action-unlabeled video. A single shared dual-stream diffusion model generates flow for action prediction, conditions on flow for world modeling, and pretrains on large-scale unlabeled video.

RoboTwin · Clean

92.94%

Manipulation success rate

RoboTwin · Random

92.14%

Robust under random scenes

WorldArena · EWMScore

63.71

State-of-the-art overall

Trajectory Accuracy

+18.4%

Relative improvement

One representation. Three superpowers.

FlowWAM bridges executable robot actions and pixel-space video priors via dense flow videos — enabling action-unlabeled pretraining, policy inference, and action-conditioned world modeling within the same framework.

Why Optical Flow?

A motion signal already video-native.

01 · Uniformity

Same format as RGB

HSV color-wheel encoding maps per-pixel displacement into RGB images, closing the modality gap between control signals and pretrained video priors.

02 · Reliability

Dense pixel motion

A per-pixel motion field tightly steers video generation to follow the requested trajectory — far beyond sparse action tokens or static masks.

03 · Scalability

No action labels needed

Flow is extractable directly from raw egocentric videos, so FlowWAM scales by absorbing motion priors from large unlabeled corpora.

Method

A dual-stream diffusion transformer

Step 1

Flow RGB Encoding

Convert per-pixel optical flow into an RGB image via invertible HSV color-wheel encoding. Format-identical to scene frames — no separate action tokenizer needed.

Step 2

Shared DiT Backbone

RGB and flow latents share a frozen VAE encoder and all transformer blocks. Stream-specific only at patch embedding & output head. Joint self-attention over concatenated tokens.

Step 3

Action Expert

A transformer that cross-attends to flow & RGB tokens, conditions on proprioception, and predicts N-step action chunks under flow-matching — with stochastic latent conditioning to bridge train/inference.

Two Operating Modes

Generate flow. Or condition on it.

Both modes share the same dual-stream DiT. They differ only in whether the flow stream is unknown or observed.

Mode A

Policy Mode

flow is generated

Both streams initialized from Gaussian noise and jointly denoised. The model synthesizes a future RGB rollout and its corresponding flow video — then the action expert decodes executable actions from the predicted motion plan.

Generate future flow latents

Decode into N-step action chunk

Execute on robot

Mode B

World-Model Mode

flow is provided

Flow latents are set to the clean VAE encoding of a desired motion trajectory and held fixed. Only RGB is denoised. The model renders a future video that follows the specified motion — perfect for planning and policy evaluation.

Provide target flow trajectory

Generate RGB future from noise

Use for evaluation & planning

Experiments

Beating VLA and WAM baselines.

Manipulation Policy on RoboTwin 2.0

Average success rate over 50 bimanual tasks — Clean setting (fixed layout/lighting) and Random setting (randomized object pose, distractors, lighting, background).

Method	Type	Clean	Random
π_0.5	VLA	42.98%	43.84%
X-VLA	VLA	72.88%	72.84%
Motus	WAM	88.66%	87.02%
GigaWorld-Policy	WAM	86.36%	85.04%
X-WAM	WAM	89.76%	90.68%
Fast-WAM	WAM	91.88%	91.78%
FlowWAM (w/o pretrain)	WAM	82.40%	80.80%
★ FlowWAM (w/ pretrain)	WAM	92.94%	92.14%

Action-Conditioned World Modeling on WorldArena

6 headline metrics + overall EWMScore over 121-frame rollouts at 24 fps. Bold = best.

Method	Cond.	JEPA	Subj.	Bg.	Traj.	Depth	Sem.	EWMScore
CogVideoX	Text	94.84	80.97	88.38	34.79	91.09	89.70	58.79
Veo 3.1	Text	57.97	75.82	91.67	11.36	74.27	83.79	57.77
Cosmos-Predict 2.5	Action	90.65	67.24	72.72	27.49	90.35	89.86	54.29
IRASim	Action	93.72	72.82	78.69	35.92	93.50	89.43	56.15
Ctrl-World	Action	92.77	83.56	90.30	48.20	93.25	88.68	59.98
GigaWorld-1	Img-Act	96.77	80.97	86.43	54.27	98.44	89.42	62.34
★ FlowWAM (Ours)	Flow	97.14	82.46	89.97	64.26	98.97	89.93	63.71

Trajectory Accuracy

64.26

+18.4% over runner-up

Depth Accuracy

98.97

Best in class

JEPA Similarity

97.14

Best in class

Qualitative Results

Watch flow in action.

Policy Mode — Predicted Flow Trajectories

World-Model Mode — Flow-Conditioned Generation

Real-World — Rollouts

Citation

Cite FlowWAM

@misc{flowwam,
      title={FlowWAM: Optical Flow as a Unified Action Representation for World Action Models},
      author={Yixiang Chen and Peiyan Li and Yuan Xu and Qisen Ma and Jiabing Yang and Kai Wang and Jianhua Yang and Dong An and He Guan and Gaoteng Liu and Jianlou Si and Jun Huang and Jing Liu and Nianfeng Liu and Yan Huang and Liang Wang},
      year={2026},
      eprint={2607.13017},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2607.13017},
}