SolarRenderer: Few-Step Generative Reference Rendering
SolarRenderer on Reactor
Render a recorded structure video with SolarRenderer, served through Reactor Runtime. Select a structure video, optionally supply an aligned appearance image, describe the scene, and receive the rendered video in a small browser demo.

Validation status: this four-B200 integration is a development preview. The browser has completed ten generation blocks (158 frames) with and without an appearance reference, including a forced TURN/TCP media path. Observed steady blocks take about 1.8–2.1 seconds for 17 frames; preparing a new video can take over a minute. This is below the native 24 FPS timeline. Docker image execution remains unvalidated. FA4 and static-prefix caching change floating-point results; the accelerated version is not bitwise identical to native inference.
What the integration adds
- A Reactor service with explicit input selection, Start, Pause and Resume.
- Streaming output as completed video segments become available.
- A browser client using the official Reactor JavaScript SDK.
- Four-GPU sequence parallelism, FA4, modulation/text/static-prefix caching and parallel VAE decoding.
- Clickable example inputs and a side-by-side structure/render view.
The service uses the released causal four-step model. Each video contains 158 frames and ends after ten generation blocks. Its native timeline is 24 FPS (about 6.58 seconds). Runtime adapts playback to generation time. The measured generation rate is substantially below the native 24 FPS timeline.
What you need
- A license and deployment location permitted by the model's weight terms.
- A Linux GPU machine with Docker and NVIDIA Container Toolkit. The recipe was tested on four B200 GPUs on one host, with shared weights/scratch storage.
- The prepared MiniMax-H3 base and the two causal SolarRenderer LoRA files.
- Node.js 22.12+ and npm for the demo.
1. Install the Reactor CLI
Install the CLI using the Reactor documentation.
Use a current CLI that supports reactor run --shm-size; older releases forward
unknown flags to Runtime. Check reactor version and reactor run --help.
Run the following commands from this directory.
2. Get the weights
Read the weight license
before downloading. Follow the authors' setup and causal inference instructions.
Use their scripts/prepare_h3.py to prepare the base and localize its component index.
Arrange the files under one directory:
weights/
MiniMax-H3/
modular_model_index.json
transformer_ref/
text_encoder/
processor/
tokenizer/
vae/
...
solarrenderer_causal_control_lora.safetensors
solarrenderer_causal_4step_lora.safetensors
Keep the prepared base at the same absolute path when mounting it into the container:
the localized component index refers to that path. The CLI's filesystem --weights
mount does this. The service does not automatically download or substitute checkpoints.
3. Start the model
reactor build
reactor run --gpus device=0,1,2,3 --shm-size 8g --port 8080 --weights ./weights
Select four available GPUs on a shared machine. In another terminal, check
curl http://localhost:8080/health. The service validates asset paths at startup;
the first Start new video performs encoding and loads the renderer. The original
text encoder and renderer use the GPU in sequence, so starting a new video has a
substantial preparation phase. Do not interpret HTTP readiness as first-frame readiness.
The manifest builds from YAML; no Dockerfile is needed. Rebuild after changing the
adapter. Ctrl+C stops the service started by this command.
4. Open the client
cd demo
npm ci
npm run dev -- --host 127.0.0.1
Vite reads the example inputs directly from this repository's
examples/sem-highway_synthwave/ directory and includes them in production builds.
No sample path configuration is needed.
Open http://localhost:5173. The default Runtime address is http://localhost:8080;
set VITE_RUNTIME_URL before starting the demo to use another local endpoint.
Click Connect, choose Synthwave Highway to fill the official video, image
and prompt, then click Start new video. Structure only uses the same scene
without an appearance image and adjusts the prompt accordingly. You can also upload
your own inputs. File or example selection alone
does not run the model. No example or reference image is selected automatically.
The authors provide a matched structure video, image and prompt. Use that complete set for the first comparison. Structure videos must cover at least 158 frames after native 24 FPS resampling; longer videos use the opening segment. Landscape input produces 1344×768 output and portrait input 768×1344. The original preprocessing center-fits the optional image to the selected canvas.
Remote machines and SSH
Forward the HTTP and demo ports if running both on a remote machine:
ssh -L 8080:127.0.0.1:8080 -L 5173:127.0.0.1:5173 user@gpu-host
HTTP forwarding alone does not carry WebRTC media. Configure a supported TURN-over-TCP relay and forward its TCP port as well, or use a network route that carries WebRTC. Docker host networking alone does not make SSH forward UDP. Keep relay credentials in deployment configuration; never put them in this repository. SSH/TURN media connectivity has been exercised in a relay-only local browser. Validate your own SSH tunnel and relay configuration before sharing the service.
Controls
| Control | Effect |
|---|---|
| Structure video | Select the recorded motion and layout for the next video. |
| Appearance reference | Optionally constrain the opening appearance. |
| Rendering prompt / Seed | Captured when Start is pressed. |
| Start new video | Start a fresh finite video, replacing any current render. |
| Pause / Resume | Stop or continue at a generation boundary; buffered video may still play. |
| Disconnect | Close the demo's connection through the SDK. |
Commands take effect after the current generation block. The demo allows up to 180 seconds for their replies; a Pause during generation can take about 35 seconds, and initial preparation can take longer. This changes the client wait limit, not the model's generation speed.
Prompt edits apply to the next video. This first integration preserves the original fixed text conditioning throughout each render; it does not advertise live prompt changes. There are no driving keys or game-engine controls: motion is supplied by the uploaded video. The decoder waits for the necessary temporal overlap before showing its first frames.
Validation and code map
Run CPU application tests with PYTHONPATH=. python -m pytest tests/ -q and export
the client contract with python -m reactor_runtime.schema --path . --out schema.json.
reactor validate checks the manifest; npm run build in demo/ checks the frontend.
solarrenderer.py owns client commands and output, solarrenderer_types.py defines
the public state, and solarrenderer_model.py owns the Runtime-independent step contract.
The backend reuses the pinned native encoder and LoRA loader. Sampling and decoder
helpers retain per-video state between calls. The build uses upstream revision
a2371eaecff9a8ab378b5ec36199c5863d138074 and Reactor Runtime 3.8.3.
The CPU comparison exercises the original causal loop with small substitute computation: four seed/reference combinations each run ten consecutive blocks with identical latent values, random-generator state and final KV contents. Decoder-window tests cover five lengths. The unaccelerated real-weight comparison uses the official example with a shared native-encoded input and seed 0: all ten blocks have zero latent and decoded-RGB difference before compression. This does not apply to the accelerated four-GPU configuration: FA4 and static-prefix reuse produce numerical and local appearance differences. Four denoising evaluations, the native history window and horizon remain unchanged. Both input modes completed 158-frame browser runs. The current file-based native encoder also re-reads validated uploads; moving preprocessing fully into the application requires its own native-input equivalence check.
Credits and licence
Thanks to Rising0321 for the Reactor integration. SolarRenderer and SolarWM remain the work of their original authors. The sampling and decoder helpers derive from SolarRenderer's Apache-2.0 code; retain its attribution and NOTICE. Weights remain subject to the MiniMax H3 Community License, including territorial restrictions. Please contact [email protected] with integration feedback or collaboration ideas.