SUPRA-Tex: 3D Generation with Super-Aligned Texture
Image-to-Material Translation frontend
SUPRA-Tex begins with foreground isolation, canonical-camera estimation, and
material prediction. The default qwen-image backend uses the SUPRA-Tex
Material Adapter based on Qwen-Image-2.1; firered selects the bundled factorized
modality-axis joint-attention model. Both produce the aligned albedo and MRA
maps used by surface-observation encoding.
Follow the fresh-checkout environment commands and CUDA extension instructions
in Installation. They install the pinned Python dependencies
and bundled MoGe-2/utils3d packages; the frontend extra alone is insufficient
for the native Qwen runtime.
The Qwen backend uses its native DiffSynth runtime and torch attention. The
FireRed backend selects FA3 for its material Transformer on compatible Hopper
setups and native Torch attention otherwise; its text encoder selects FA2 or
SDPA. The default Qwen backend supports --low-vram CPU offload. See
GPU compatibility for device settings and validation
status, and Weights for model artifacts.
After the model preparation step, run the frontend using the prepared local paths. For an RGB input, prepare RMBG-2 as well:
python scripts/prepare_models.py --with-rmbg
python -m supratex.frontend.pipeline --input input.png --output work \
--rmbg-model ckpts/ready/external/rmbg2 \
--moge-model ckpts/ready/external/moge2 \
--material-base ckpts/ready/external/qwen-base \
--material-model ckpts/ready/project/material
These paths assume the default preparation directory and no local model
overrides. The standalone frontend does not read models.json; when using a
different preparation directory or local overrides, supply the corresponding
paths recorded there. Without explicit material paths, this CLI falls back to
ckpts/qwen-base and ckpts/qwen-material under the source root.
The bundled Qwen runtime is selected automatically and uses packages installed in the active
Python environment. --qwen-runtime can select another compatible local runtime;
--qwen-deps optionally adds an existing dependency overlay. Base and model sources
also accept Hugging Face namespace/repo IDs; --cache-dir and
--local-files-only apply to resolution, and --revision selects the material
checkpoint revision. Runtime and dependency paths must be local directories.
For the same options on infer.py, use --material-revision instead of
--revision (which selects the completion model there).
Use a fresh output directory. The handoff contains condition_rgba.png,
condition_rgb.png, camera.json, image_depth.npy, albedo.png, and mra.png.
image_depth.npy stores float32 MoGe-2 camera-z depth on the condition pixel grid,
with NaN for invalid pixels. Its dimensions must match condition_rgba.png;
completion resizes it to the 1024-square observation grid with nearest-neighbor
sampling, preserving invalid pixels. Single-view completion uses it to exclude
source pixels whose depth disagrees with the generated geometry near depth edges.
The gate is skipped when depth is absent or a valid fit cannot be obtained;
malformed depth files are rejected.
The depth pre-filter is enabled by default for single-view infer.py and
python -m supratex.frontend.pipeline runs. Pass --no-filter (alias
--no-image-depth) to either command to disable it by omitting the depth file.
infer.py --dual-view and frontend --record-material-provenance omit the file
because dual-view completion does not support the pre-filter and rejects inputs
containing image_depth.npy. The standalone camera command exports depth only
when given --depth-output, as shown below.
The infer.py work directory's run.json records image_depth_filter.requested,
enabled, and disabled_reason (user_disabled or dual_view_unsupported).
These fields describe whether the depth input is enabled for the run; a later
degenerate fit can still skip the gate. An enabled run has a null disabled reason.
For an existing non-opaque foreground RGBA, omit --with-rmbg during preparation
and replace --rmbg-model with --use-input-alpha in the frontend command.
Foreground preprocessing writes a straight-color
condition_rgba.png and a matching black-premultiplied condition_rgb.png
after reducing the longest input dimension to at most 1024 pixels and applying
a square foreground crop with 10% padding. Qwen receives the straight
RGBA frame, resized once to its 1024-square model canvas; FireRed receives
the RGB condition. Do not premultiply Qwen's RGB channels before attaching alpha.
Foreground extraction uses input alpha or RMBG-2. Depth-based rejection occurs later during 3D observation construction: alpha erosion and first-hit depth discontinuity filtering discard unreliable observation pixels. It does not erode the saved foreground image using MoGe depth. See Texturing.
Supply geometry separately in the canonical frame, or set
TORCH_HOME="$PWD/ckpts/ready/torch" and add
--geometry-package deployment/runtime/geometry --geometry-model ckpts/ready/geometry
to generate the final mesh with Pixal3D's fixed mesh scale (add
--geometry-scale-search for bounded scale calibration). This uses the NAF
cache created by model preparation. See the
geometry frontend for source requirements and the matching
mesh/camera handoff.
Qwen-Image material backend
With the default preparation layout, the model and runtime paths are:
ckpts/ready/
external/qwen-base/ # complete Qwen-Image-2.1 base
project/material/
adapter_model.safetensors
adapter_config.json
deployment/runtime/
qwen/ # bundled DiffSynth runtime
Keep the bundled runtime tree, including diffsynth/ and
examples/qwen_image_21/pbr/{infer,inference_utils,inference_offload}.py.
The adapter requires its native JSON config and complete base model, including
processor, VAE, text encoder, and Transformer. The safe file includes learned
modality embeddings as well as LoRA and uses the SUPRA-Tex native loader.
Legacy checkpoint-0100000.pt plus its matching .complete.json also remain
supported. Neither adapter format replaces the external base model.
Inference fixes the native prompts, 1024 resolution, seed 42, sigma 1,
one endpoint-prediction step, and torch attention. The outputs are albedo and
metallic/roughness only; there is no normal or AO prediction. Native RGBA
outputs remain under material/native/. Canonical albedo.png and mra.png
are RGB: MRA red is metallic, green is roughness, and blue is the input
coverage. Native MR blue is a constant 255 placeholder, which the adapter
replaces with input coverage for completion. This coverage is not a prediction
of material transparency. Metallic and roughness use linear 0–255 encoding;
albedo is sRGB.
FireRed material backend
Select FireRed explicitly with its own base and material bundle:
python -m supratex.frontend.pipeline --input input.png --output work \
--rmbg-model /models/rmbg2 --moge-model /models/moge2 \
--material-backend firered --material-base /models/firered-base \
--material-model /models/firered-material
The legacy --firered-base option also supplies the FireRed base path.
For this backend only, replace --material-model with all three explicit
artifact arguments:
--material-weights full_transformer.safetensors \
--material-spec joint_contract.json \
--material-checkpoint-manifest checkpoint_complete.json
Individual foreground, camera, and FireRed stages remain available:
python -m supratex.frontend.preprocess --input input.png --rmbg-model /models/rmbg2 \
--output-dir work
python -m supratex.frontend.camera --image work/condition_rgb.png --model /models/moge2 \
--output work/camera.json --depth-output work/image_depth.npy
python -m supratex.frontend.firered --base-model /models/firered-base \
--condition work/condition_rgb.png --weights full_transformer.safetensors \
--spec joint_contract.json --checkpoint-manifest checkpoint_complete.json --output work/material
The FireRed adapter invokes the selected package's infer.py. The bundled
package accepts a JSONL record containing the condition image and instructions;
the adapter creates this record automatically. Additional fields retained for
checkpoint compatibility are not used as material targets during inference.
After decoding, the
adapter fixes the blue channel of <key>_mra.png to 255 (normalized opacity 1)
everywhere, preserving metallic in red, roughness in green, and the albedo pixels.
The checkpoint's native material-alpha prompt is retained; its predicted blue
channel is replaced before the maps reach the 3D material encoder. Foreground
coverage continues to come from the separate condition mask.
The adapter copies its <key>_albedo.png and <key>_mra.png outputs to the canonical
albedo.png and mra.png primary material handoff names. Compatibility
copies named firered_albedo.png and firered_mra.png are also written.
The bundled FireRed package is source code only. It retains the factorized
ModalityAxisJointAttention modules and the exact modality_joint_blocks.*
parameter names used by the checkpoint. The base model, factorized transformer
weights, joint contract, and completion manifest must be supplied explicitly;
no model weight is included in this repository. The frontend requires a
checkpoint that produces 1024-by-1024 RGB maps. A compatible external package
may be selected with --firered-package in the complete frontend, or --package
in the standalone FireRed command; its own entrypoint contract then applies.
Camera handoff
The camera receipt has camera_angle_x, distance, and mesh_scale. These
are the canonical fields consumed by the surface-observation path: camera at
(0, 0, distance * mesh_scale), looking at the origin with +Y up. MoGe-2
intrinsics are normalized and its horizontal focal length determines both FOV
and canonical distance. The optional image_width, image_height, and model
receipt fields are provenance only.