WorldPlay2: Extending Real-Time Interactive World Models in Control and Horizon
| Name | Last commit message | Last commit date |
|---|---|---|
| configs | ||
| reactor | ||
| worldplay2 | ||
| generate.py 8.2 KB | ||
| LICENSE.txt 21 B | ||
| README.md 9.1 KB | ||
| requirements.txt 200 B | ||
| run_ar.sh 3.7 KB | ||
| run_bi.sh 3.7 KB | ||
| run_few_step.sh 3.9 KB |
๐ฎ WorldPlay2: Extending Real-Time Interactive World Models in Control and Horizon
๐ฅ News
- [2026.10.7] ๐ WorldPlay2 is now on Reactor! Play in your browser, or deploy it locally with the Reactor integration and demo frontend in this repository. Thanks to Rising0321, Orion-Zheng, and notrealzapa for porting WorldPlay2 to Reactor and optimizing the inference infrastructure.
- [2026.10.6] ๐ค We have released the WorldPlay2 model weights and inference code!
๐ฅ Video
https://github.com/user-attachments/assets/3264efa7-ef6a-4c7b-a043-d819264a93bd
๐ ๏ธ Installation
Requires Python 3.10+ and a CUDA-capable NVIDIA GPU.
Create and activate a Conda environment, then install the dependencies:
conda create -n worldplay2 python=3.10 -y
conda activate worldplay2
python -m pip install -r requirements.txt
Install flash_attn:
pip install flash-attn --no-build-isolation
Optional SageAttention
Install SageAttention in your CUDA environment:
pip install sageattention==2.2.0 --no-build-isolation
All three launchers support ATTENTION_BACKEND (default: flash). For direct
CLI usage, pass --attention_backend sage.
Optional FP8
Install the additional dependencies on supported hardware:
pip install transformer_engine[pytorch]
After setting the model and input paths in run_few_step.sh or exporting them
as environment variables, enable FP8:
ENABLE_FP8_FFN=true ENABLE_FP8_QKV=true USE_DIT_FSDP=false bash run_few_step.sh
๐งฑ Model Download
Base Model
Download Wan2.2 I2V-A14B
and set CKPT_DIR to its local directory.
WorldPlay2 Models
Download the WorldPlay2 checkpoints for your inference mode. Set
LOW_NOISE_CKPT and HIGH_NOISE_CKPT to the corresponding checkpoint files.
| Model | Model Type | Default steps | Download Links |
|---|---|---|---|
| WorldPlay2-Fast | Autoregressive, 4-step (few_step) |
4 | ๐ค Hugging Face |
| WorldPlay2-BI | Bidirectional (bi) |
40 | ๐ค Hugging Face |
| WorldPlay2-AR | Autoregressive (ar) |
40 | ๐ค Hugging Face |
| WorldPlay2-TAE | Causal VAE | --- | (coming soon) |
Download the base model and WorldPlay2 few-step model using the Hugging Face CLI.
pip install "huggingface_hub[cli]>=0.34,<1.0"
# Base Wan2.2 model
hf download Wan-AI/Wan2.2-I2V-A14B --local-dir ./checkpoints/Wan2.2-I2V-A14B
# WorldPlay2 models
hf download aejion/WorldPlay2-Fast --local-dir ./checkpoints/WorldPlay2-Fast
hf download aejion/WorldPlay2-BI --local-dir ./checkpoints/WorldPlay2-BI
hf download aejion/WorldPlay2-AR --local-dir ./checkpoints/WorldPlay2-AR
Set CKPT_DIR to ./checkpoints/Wan2.2-I2V-A14B, and set LOW_NOISE_CKPT
and HIGH_NOISE_CKPT to the checkpoint files in ./checkpoints/WorldPlay2-Fast.
๐ฎ Quick Start
Set the paths to the WorldPlay2 few-step checkpoints and run from the repository root:
# CKPT_DIR: path to the original Wan2.2 I2V-A14B base model directory.
# LOW_NOISE_CKPT: path to the WorldPlay2 few-step low-noise checkpoint.
# HIGH_NOISE_CKPT: path to the WorldPlay2 few-step high-noise checkpoint.
CKPT_DIR=/path/to/model \
LOW_NOISE_CKPT=/path/to/model/low_noise_model.safetensors \
HIGH_NOISE_CKPT=/path/to/model/high_noise_model.safetensors \
INPUT_JSON=/path/to/input.json \
OUTPUT_PATH=./outputs/few_step \
bash run_few_step.sh
Use run_ar.sh or run_bi.sh for the other modes, with their corresponding
checkpoints. Launchers default to 8 GPUs with sequence parallelism and FSDP;
set NPROC_PER_NODE to match your GPU count. For one GPU, also set
USE_DIT_FSDP=false USE_T5_FSDP=false.
All input items are processed, and videos are saved as <output_name>.mp4 in
OUTPUT_PATH.
โก Quick Start with Reactor Runtime
The reactor/ integration serves WorldPlay2 as an interactive model with a browser frontend. The default configuration uses 4 NVIDIA B200 GPUs, Docker and the NVIDIA Container Toolkit. Follow the Reactor setup guide to install the CLI and prepare the weights under reactor/weights/base and reactor/weights/fast.
From the repository root, enter reactor/, then build and start the model on four available GPUs:
cd reactor
reactor build
reactor run --gpus '"device=0,1,2,3"' --port 8080
Once http://localhost:8080/health reports "state":"available", open another terminal at the repository root and start the frontend (Node.js 20+ and pnpm required):
cd reactor/demo
pnpm install --frozen-lockfile
pnpm dev
Open http://localhost:3000, select the Local endpoint (http://localhost:8080), and click Connect. Choose an example or add your own image and prompt to start playing, then use WASD to move and the arrow keys to look around. Release the keys or on-screen buttons to stop the corresponding action. For a remote GPU machine, see the connection instructions, including WebRTC requirements for SSH access.
๐ Input Format
Provide a JSON array. Relative image paths are resolved from the JSON file's directory.
[
{
"image_path": "input.jpg",
"perspective": "tps",
"action": "w-32,s-32",
"prompt_event": "prompt1-32,prompt2-32",
"prompt1": "The scene in the video is: a quiet forest trail. The character is: a hiker wearing a blue jacket. The event is: dark clouds gather and rain begins to fall.",
"prompt2": "The scene in the video is: a quiet forest trail. The character is: a hiker wearing a blue jacket. The event is: the rain stops and sunlight breaks through the clouds.",
"output_name": "sample_000"
}
]
- Perspective:
fpsortps. - Actions: comma-separated
action-durationcommands. Movement actions arew,s,a,d,wa,wd,sa,sd; rotation actions areleft,right,up,down; special action isspace. Combine tokens with+, for exampleright+space-32. - Durations: measured in latent frames; the first latent is stationary.
Action and prompt durations must have the same total. The total and prompt
boundaries must align with
chunk_length(4 in the few-step and AR launchers, 32 in BI).
Prompt Format
Construct each prompt field (prompt1, prompt2, etc.) using the template for
your perspective, replacing the placeholders with your descriptions.
FPS (fps): omit the The character is: ... sentence.
The scene in the video is: {scene description}. The event is: {event description}.
TPS (tps): include the character description, as in the JSON example above.
The scene in the video is: {scene description}. The character is: {character description}. The event is: {event description}.
The scene description is required for both perspectives; the character
description is only required for TPS. The event is optional for both: omit
the entire The event is: ... sentence when it is not needed. For example,
an FPS prompt without an event is:
The scene in the video is: a quiet forest trail.
The prompt_event scheduling field is still required, even when prompts omit
the event description.
๐ Configuration
Paths and launcher settings can be overridden through environment variables:
| Variable | Purpose |
|---|---|
NPROC_PER_NODE |
Number of GPUs |
ATTENTION_BACKEND |
Visual self-attention: flash (default) or sage |
SAMPLE_STEPS |
Sampling steps; few-step inference requires 4 |
GUIDE_SCALE |
Guidance scale for BI/AR |
YAW_ROTATION_SPEED_DEG |
Left/right rotation per latent |
PITCH_ROTATION_SPEED_DEG |
Up/down rotation per latent |
๐ Acknowledgements
We would like to thank HY-World 1.5, HY-World 2.0, and HunyuanWorld-Mirror for their great work.