SolarRenderer: Few-Step Generative Reference Rendering
Python
docs: measured speedups of the optional kernels for steps 3-5
henry-y19d1239daf66
diff --git a/docs/training.md b/docs/training.mdindex a5c1ad7..60e5cca 100644--- a/docs/training.md+++ b/docs/training.md@@ -59,7 +59,8 @@ kept under `TORCH_EXTENSIONS_DIR`, else the temporary directory) against the CUTcontainer; elsewhere point `SOLAR_CUTLASS_INCLUDE` at a CUTLASS `include/` directory. `SOLAR_ATTN_CPU_CACHE=24` keepsevery block's attention output for the backward pass instead of recomputing it, 24 blocks of them in pinned host memory(about 25 GB per GPU); raise it if GPU memory runs short. `SOLAR_SP_PACK_B1=1` packs the sequence-parallel exchange-directly by destination. These switches were measured and checked on this step only.+directly by destination. This step is the only one checked in full; the other steps below were timed in short runs and+compared with the original step by step.**3. Post-train the four-step LoRA with structure guidance** (800 steps at global batch 4). Convert the public DMAD LoRA first,then point `train.dmd.init_lora` at it and `train.dmd.control` at the control run's checkpoint directory from step 2 (the@@ -76,6 +77,10 @@ The structure guidance is `train.dmd.structure_guidance` (1 = plain distribution1.5; take the live adapter at the last step. To render with your own runs, export each run's LoRA with`python scripts/export_lora.py --checkpoint <checkpoint dir> --out <file>.safetensors`.+The same switches except `SOLAR_ATTN_CPU_CACHE` (the three model copies leave too little GPU memory for it) make this+step about 1.75x faster (254.5 s to 145.4 s per step on 8 H200s); the losses stayed within about 1% of the original+step by step, the student loss within 4.5%.+## Causal versionThe causal LoRAs follow the same two stages on top of the bidirectional ones. As above, the released causal LoRAs@@ -89,6 +94,9 @@ control run's checkpoint directory from step 2.torchrun --standalone --nproc-per-node 8 -m solarwm train --config configs/causal_control_lora.yaml```+With the same switches (without `SOLAR_ATTN_CPU_CACHE`) a step takes 55.4 s instead of 60.5 s (1.09x): most of this+step's attention is block-causal and keeps the original kernels.+**5. Post-train the causal four-step LoRA with structure guidance** (1,600 steps at global batch 1). The causal controlLoRA from step 4 (`train.dmd.student_init`) is frozen in the student's base; the student is a separate four-step LoRAthat starts as the bidirectional one (`train.dmd.accel_init`) and is distilled on its own block-by-block streams, with@@ -98,5 +106,7 @@ the bidirectional control LoRA from step 2 (`train.dmd.control`) as the teacher:torchrun --standalone --nproc-per-node 8 -m solarwm train --config configs/causal_structure_guided_4step.yaml```+With the same switches (without `SOLAR_ATTN_CPU_CACHE`) a step takes 53.7 s instead of 74.6 s (1.39x).+Export both runs with `scripts/export_lora.py` and render with `scripts/render.py --causal`.