Official Python inference and LoRA trainer package for the LTX-2 audio–video generative model.
Changelog
1.5.0 - 2026-10-18
Added
ImageConditioningInputsupports explicit latent-replacement and keyframe-guide bindings. On video CLIs, pass--image PATH FRAME_IDX STRENGTH replace|guide [CRF]; omit the binding keyword for the prior automatic binding (frame 0 replaces, later frames guide). Replacement frame indices are expressed in pixel space and must be0or a causal latent boundary (1 + 8 * k).TI2VidTwoStagesPipelinesupports keyframe interpolation throughImageConditioningInput.guide()bindings, or--image … guideon the CLI. Setrefine_audio=Trueor pass--refine-audioto preserve the former dedicated pipeline's stage-2 audio refinement; the default preserves stage-1 audio.ImageConditioner.load_conditionings(images, color_space=None)resolves each image conditioning's CRF (likeresolve_crf), decodes each distinct still once (SDR, or EXR withcolor_space) and returnsLoadedImageConditionings: the decoded still plus the input'sbindingandstrength.MGPUController(cpu_threads_per_rank=8)caps each rank's intra-op CPU threads (torch.set_num_threads) so the ranks' small host tensor ops do not stall on many-core hosts. PassNoneto keep torch's default.LTX_MGPU_DECODE_RECEIVER_TIMEOUT_SECONDSsets how long a multi-GPU video decode waits for a rank that stopped streaming before it gives up (default 60).
Changed
- Multi-GPU pipelines encode prompts faster with the default Accelerate-sharded Gemma: the other ranks now wait on the host until the source rank's Gemma has finished on every GPU, instead of spinning in an NCCL receive on a GPU Gemma is still using.
BroadcastCoordinatormoved fromltx_pipelines.multigpu._broadcasttoltx_core.multigpu.broadcast, andBroadcastGemmaWrapper(and soAccelerateGemmaWrapper) takes an optionalcoordinator.BroadcastCoordinator.wait_for_signal_changenow consumes exactly one signal per call (a worker that polls late no longer skips a signal) and takes an optionaltimeout. ImageConditioningInputnow takes anImageReplaceorImageGuidebindinginstead of aframe_idx. Replace existingImageConditioningInput(path, frame_idx, strength, crf)calls withImageConditioningInput.at(path, frame_idx, strength, crf)to preserve the previous frame-0 replacement and later-frame guidance behavior; use.replace(path, frame_idx, strength, crf)or.guide(path, frame_idx, strength, crf)to select a binding explicitly. Both bindings use pixel-frame indices.image_conditionings()andimage_conditionings_for_chunk()takeLoadedImageConditionings (fromImageConditioner.load_conditionings) instead ofImageConditioningInputs, and no longer takecolor_space.- Performance improvements, in collaboration with doubleAI (https://doubleai.com): faster transformer passes (especially under multi-GPU sequence parallelism and with perturbation guidance) and LoRA switches, denoising samplers without per-step CPU synchronization, a DFR spatial epilogue (
--spatial-upscalings 2) on fewer tiles (at most 1024 px per side, overlapping by 64 px, instead of a 2x2 first step and 4x4 at a 320 px overlap) that multiplies its long video projections in fp8 on GPUs with fp8 GEMMs (compute capability 8.9+), carry keyframes rebuilt on the rank that decodes them, a multi-GPU video decode that streams to MP4 encoding as it decodes, faster diffusion VAE decoding with lower reserved GPU memory, a faster-compiling--diffvae-optimization blackwell_dslkernel, lower GPU memory during MP4 encoding, faster image conditioning, and faster stage starts with a warm model registry. Output is not bit-identical to previous releases, mainly because of the DFR spatial epilogue's new tiles and fp8 GEMMs; diffusion VAE decodes also differ by rounding, and under multi-GPU sequence parallelism audio cross-attention no longer attends to the video sequence's padding tokens.
Removed
ImageConditioningInput.frame_idx,.assert_in_clip(),.map_frame_index()and.to_item(); use the same members on theLoadedImageConditioningreturned byImageConditioner.load_conditionings, orinput.binding.frame_idx.KeyframeInterpolationPipelineand theltx_pipelines.keyframe_interpolationCLI; useTI2VidTwoStagesPipelinewithImageConditioningInput.guide()andrefine_audio=True, or pass--image … guidewith--refine-audio.
Fixed
- DiffVAE 1-tile decode no longer produces gray frame tails from frame 128 on long 1080p clips under torch < 2.13. CUDA RMSNorm wrapped flattened offsets at 2^32 elements; last-dim norms are now T-chunked.
- Compiling a transformer that uses blockwise FP8/FP6 quantization with
seq_dim_dynamicno longer pins the sequence length. Previously each new length recompiled the blocks, and a batch size divisible by 4 failed with aConstraintViolationError. - A warm multi-GPU DFR job (a second or later request in one process) could fail with "Cannot fit a DiffVAE decode tile". Cold and warm runs and every rank now plan the same decode tiles, and ranks with unequal GPUs no longer hang.
--diffvae-optimization blackwell_dslkeyframe decodes plan smaller tiles. - Diffusion VAE decode with
--diffvae-optimization blackwell_dslcould hang indefinitely on datacenter Blackwell: the fused stage-5 attention kernel re-armed its per-tile barriers while some warps were still waiting on them.
1.4.2 - 2026-10-01
Added
ltx_pipelines.alpha_gen, a one-stage IC-LoRA pipeline on the full model with video CFG 1 and STG off.
1.4.1 - 2026-09-30
Changed
- HDR video encoding streams decoded chunks directly into EXR and HLG outputs instead of retaining the full clip in memory.
1.4.0 - 2026-09-29
Added
--chunkedon Distilled, TI2Vid two-stage, A2Vid, Dub-It, and IC-LoRA enables long-video generation with the default chunk layout (97-frame windows and 25-frame carry) without setting--chunk-pixel-framesor--chunk-carry-frames;--chunk-blend-framescontrols decoded video seam crossfade length.ltx-kernelsproductionsm_100awheels now load precompiled LTX-2.5 diffusion-VAE CuTe DSL kernels, eliminating first-decode JIT for standard plain and keyframe decoding; custom kernel shapes still compile on first use.ICLoraPipelineacceptsstages, a sequence ofICLoraStageConfig(resolution, IC-LoRA, sigmas, optional transformer tiling). Default remains half-res with IC-LoRA, then a full-res refinement without it. Consecutive stages must stay at the same resolution or upsample by exactly 2x.spatial_tiled_ic_lora_stages()builds that two-stage recipe with fixed-size spatial tiling so large canvases stay on a trained window; both stages keep the IC-LoRA and reference.TiledDiffusionModelandFixedSizeSpatialTilinginltx_pipelines.utils.tiled_diffusionrun sequential transformer tiles with trapezoidal blending. DFR's spatial epilogue uses the same wrapper.--stage-2-ic-loraonltx_pipelines.ic_lorakeeps the IC-LoRA and reference on stage 2.--tileenables transformer tiling (default window 1024x1536; override with--tile-height/--tile-width). The CLI is always the two-stage recipe; omit--tileat 1080-class sizes to keep today's untiled path. A GPU OOM from that CLI is re-raised with--tile/--tile-height/--tile-widthand--offload cpu.decode_with_keyframesonTI2VidOneStagePipeline,TI2VidTwoStagesPipeline,TI2VidTwoStagesHQPipeline, andDistilledPipeline, plus thedistilled_mgpu,ti2vid_two_stages_mgpu, andti2vid_two_stages_hq_mgpurunners (CLI:--decode-with-keyframes). When set together with generated keyframe slots, the pipeline decodes through the keyframe-aware DiffVAE path using those slots as anchors. Two-stage pipelines re-attach seeded slots at stage 2 so the planes sit at final resolution before decode.AudioConditionByLatentIndexpins an audio latent at a given latent index, matchingVideoConditionByLatentIndexfor video.- Added
ltx_pipelines.chunks, a building block for long-video generation: plan a clip as temporal windows, run denoise → upsample → decode one window at a time, and stitch continuity across seams so VRAM stays bounded by window size rather than full clip length. - Distilled, TI2Vid two-stage, A2Vid, Dub-It, and IC-LoRA can generate long clips in overlapping windows instead of holding the full timeline in memory. Pass
--chunk-pixel-framesand optionally--chunk-carry-frameson the CLI (orchunk_config/stream_chunks()in Python) to tune window size and overlap; omit both flags for the existing one-shot path. - Chunked long-video generation plans generated keyframes per window: an integer is a per-chunk slot budget with automatic stitch anchors; an explicit index list is merged with those anchors. Reference conditioning on continuation chunks appends only the suffix after the carried overlap while still encoding the full window for causal VAE alignment.
- DFR diffusion work is importable from
ltx_pipelines.dfr_stages(denoise_stage1,denoise_stage2,run_one_temporal_round,run_spatial_epilogue) and helpers fromltx_pipelines.dfr_helpers, so an adapter can run the same path from a constructedDiffusionStagewithout wrappingDFRPipelineor instantiating Gemma. --exr-colorspaceonltx_pipelines.hdr_ic_lora(acescg/acescct/srgb_linear; defaultacescg). Only the EXR sidecar encoding changes; the HLG master stays BT.2020/HLG.--transformer-pathonltx_pipelines.hdr_ic_lora, which runs the pipeline from a split checkpoint pack instead of a distilled monolith. A split transformer carries no VAE weights, so--video-vae-pathhas to accompany it.EXRColorSpace/VideoInputinltx_pipelinesand an sRGB EOTF inltx_core.hdr, which convert an SDR source into ACEScct.TokenTimestepsinltx_core, which holds either dense per-token timesteps or a compact distinct-value table plus per-token indices.TokenTimesteps.per_token()expands either representation.DenoiseMaskinltx_core.types, which holds a per-token denoise mask and the table of its distinct values. Build the mask on the host;DenoiseMask.compact(device)tabulates its values and moves mask and table to the device before denoising.
Fixed
FlashAttention4accepts currentflash-attn-4releases that return the output tensor directly while remaining compatible with older tuple-returning releases.
Changed
- The
ltx_kernels_inductorcompile backend batches the fp8-cast policy's per-forward weight and bias upcasts into a few grouped kernels instead of one per tensor. Outputs are bit-identical. - The
ltx_kernels_inductorandltx_kernels_eagercompile backends run unmasked attention on whichever of cuDNN and flash attention is faster for each shape, which moves short-query attention (audio queries over video keys) off cuDNN's non-split fallback. The first call per attention shape times both kernels, so warmup takes slightly longer. This also applies when anSDPA_*attention backend is pinned. - Chunk planning owns generated-keyframe placement and appends slots after pipeline conditionings. Generated slots remain transformer conditioning across every stage;
decode_with_keyframesnow controls only whether the final slots anchor video decode. ChunkConfig.overlap_blend_framesdefaults to the full carry overlap (crossfade over all carried frames). Pass0for a hard cut at chunk seams.ChunkConfig.overlap_blend_frameslinearly crossfades a configurable suffix of each decoded overlap instead of cutting directly between windows, buffering only that suffix during streaming.- Under sequence parallelism, the
ltx_kernels_inductorcompile backend has the q/k RMS-norm + RoPE kernel and the v projection write each head straight into the all2all exchange buffer of the rank that owns it, instead of producing a local tensor for a separate exchange kernel. Outputs are bit-identical. - DFR spatial epilogues run their first denoising step on a 2x2 tile grid before retiling any remaining steps to 4x4. A one-step schedule stays on the 2x2 grid.
create_initial_audio_latentnow takes a video latent and requiredvideo_scale_factors. Joint first stages usecreate_initial_av_latents, which also requiresvideo_scale_factors.create_initial_video_latentrequiresscale_factors(no default). Pass the same video VAE factors used to size the video latent. Audio-only callers useAudioLatentShape.from_duration.- Distilled and DFR sample with Euler ancestral (
eta=1.0,s_noise=1.0) on LTX-2.5+ checkpoints -- distilled on both stages, DFR on stage 1, stage 2 and the spatial epilogue -- so generated output for those checkpoints differs from the previous release. Older generations sample with deterministic Euler. DFR's temporal densify tiles are unchanged ateta=0.5. PipelineOutputis again a four-field named tuple (video,audio,num_frames,tiling_config). Pipelines choose keyframe-aware decode internally; the returnedvideoiterator reflects that choice.- Conv, diffusion, and distributed
decode_video()accept extra keyword arguments and ignore ones they do not use. - Denoisers, denoising loops and
DiffusionStage.__call__returnVideoAudioinstead of a plain(video, audio)tuple. It still unpacks as two values, and also exposes.videoand.audio. Omit the absent side (VideoAudio(video=...)); constructing with bothNoneraises. DiffusionStage.__call__is now a latent entry point. Passmodalities=VideoAudio[ModalitySpec](either side may be omitted, but not both). EachModalitySpeccarries the latent to denoise, and that latent is the stage's only source of shape, so the stage never sizes a latent itself. Seed a pipeline's first stage withcreate_initial_video_latent/create_initial_audio_latent/create_initial_av_latentsinltx_pipelines.utils.helpers, and hand every later stage the previous stage's output, upscaled or not. Audio-only callers no longer need to pass placeholder video dimensions.encode_videoandencode_sdr_h264accept streaming video frame iterators and can mux audio supplied as it becomes available, so long chunked runs can write a file without assembling the full clip first.- One-shot
__call__on Distilled, TI2Vid, A2Vid, and Dub-It still returnsPipelineOutput; internally it uses the same chunked path with a single window. Reportednum_framesis the stitched length on the causal grid. ltx_pipelines.hdr_ic_lorais a single-stage ACEScct SDR-to-HDR pipeline. It writes a 10-bit BT.2020/HLG master plus an EXR sequence whose colour space is--exr-colorspace(default ACEScg;acescctwrites VAE log codes), where it previously ran two stages and returned linear float from a LogC3 inverse decode for the caller to tonemap and save.--input-colorspaceselects the input transform, so the source may be an SDR MP4/MOV (srgb_gamma,srgb) or a directory of EXR frames (srgb,acescg,acescct); EXR directories also need--frame-rate, which a container source must not set.- HDR EXR output directories are named for their colour space:
<output_stem>_acescct_exr,<output_stem>_acescg_exror<output_stem>_srgb_linear_exr. Every HDR run previously wrote<output_stem>_exr, so code that globs the old path will not find the frames. --distilled-checkpoint-pathonltx_pipelines.hdr_ic_lorais no longer required, since--transformer-pathcan supply the transformer instead. One of the two is still needed.- DFR seam keyframes on
ltx_pipelines.hdr_ic_loraare on by default.--with-keyframesis replaced by--no-keyframes. A 2.4 transformer fails the capability check unless--no-keyframesis passed; a 2.5 distilled monolith still needs--video-vae-pathto a keyframe-trained VAE or the keyframe path is a silent no-op. ltx-corerequirescolour-science, which supplies the primaries, YUV matrix and HLG transfer definitions that were previously hand-written constants.- The transformer projects AdaLN modulation once per distinct timestep value and reads it per token through
Modality.timesteps.indiceswhen a pipeline supplies a compact timestep table. Outputs are not bit-identical to previous releases. The saving is proportional to the token count while attention grows with its square, so its share of the forward shrinks as resolution rises: about 2-3% on one GPU at 16k tokens (544x960x241), and less per rank under sequence parallelism. Modality.timestepstakes aTokenTimestepsinstead of a tensor; wrap per-token timesteps asTokenTimesteps(timesteps).LatentState.denoise_masktakes aDenoiseMaskinstead of a tensor; wrap the mask asDenoiseMask(mask)withmaskon the host, and callcompact(device)on it before a denoising loop reads the state.
Removed
stepperonDiffusionStage.__call__,denoise_chunks, the DFR stage functions, and the denoising loops. Each loop constructs its own step. Passetaands_noisetoeuler_ancestral_denoising_loopandeuler_cfg_pp_denoising_loop.EulerAncestralDiffusionStepandEulerCfgPpDiffusionStepno longer take those arguments; pass them tostep.TiledModelWrapperinltx_pipelines.dfr_helpers.ops; useTiledDiffusionModelfromltx_pipelines.utils.tiled_diffusion.QuantizationPolicy.model_configurator; pass the configurator directly to the model builder orDiffusionStage.from_checkpoint(model_configurator=...).All2All.set_rank_tokens; All2All head exchange and AllGather now derive uniform token counts from their input tensors.PipelineOutput.keyframesandPipelineOutput.video_latent; usedecode_with_keyframes=True(CLI:--decode-with-keyframes) on pipelines that support it.width,height,framesandaudio_fpsonDiffusionStage.__call__; size the latent yourself and pass it asModalitySpec(latent=...). Audio sized off a playback rate that differs from the video RoPE time base, whichaudio_fpscovered in 1.3.0, is now done by building the audio latent at that rate withcreate_initial_audio_latent(..., fps=...).- Separate
videoandaudiokwargs onDiffusionStage.__call__; passmodalities=VideoAudio(...). ModalitySpec.initial_latent; useModalitySpec.latent, which is required.ImageConditioningInputfromltx_pipelines.utils.args; import it fromltx_pipelines.utils.types.--output-dironltx_pipelines.hdr_ic_lora; use--output-path, which names the HLG master and places the EXR directory beside it.--spatial-upsampler-path,--num-frames,--spatial-tileand--skip-mp4onltx_pipelines.hdr_ic_lora. The pipeline is single-stage and does not upsample, output length follows the source clip (whose frame count must be 8k+1), decode tiling is resolved automatically, and the HLG master is always written beside the EXR frames.
Fixed
- Generated audio could outlast the decoded video when
num_frameswas not on the video VAE temporal grid (for example 87 frames at 24 fps decoded 81 picture frames against a longer soundtrack). Initial audio latents are now sized from the snapped video canvas. - Chunked vocoded audio retains a 40 ms overlap and equal-power crossfades it at generation-window seams, avoiding clicks while keeping the causal decoder shortfall at stream startup.
- Multi-GPU sequence-parallel runs compiled with
torch.compilecould produce a corrupted, repeating texture instead of the prompted scene (seen with theltx_kernels_inductorbackend). The compiler could keep its own tensors in the All2All exchange buffer, which the next exchange overwrote.torch.ops.ltx_kernels.send_recv_headsandgather_headsnow take that buffer as an argument they write in place and return nothing;All2All.send_recv_headsandAll2All.gather_headsare unchanged. - Pinned equal-size transformer windows reject an overlap above half the window and round the tile count down, so three tiles never cover one cell. This is the layout used by IC-LoRA
--tile. The requested overlap is a target and the realized overlap may be smaller. Two tiles on a canvas only slightly larger than the window can still overlap by more than half, because there is no third tile.
1.3.0 - 2026-08-25
Added
- DFR can finish 4K in a tiled spatial epilogue instead of denoising the full canvas in stage 2.
--spatial-upscalings 2keeps stage 2 at half resolution and upscales in that epilogue, which is the path for better-quality 4K (use3840x2176;3840x2160is not on the size grid). Default--spatial-upscalings 1is unchanged. - Added
ltx_pipelines.dfr_mgpu, a multi-GPU DFR runner with the same flags asltx_pipelines.dfr_pipeline. - Added keyframe-aware diffusion-VAE decoding, which sharpens detail at the frames a generator anchored on. Pass keyframe latents into
VideoDecoder.decode_video(..., keyframes=). Needs a keyframe-trained video VAE; older checkpoints decode as before. --compile max_video_tokens=N max_audio_tokens=N(and the matchingCompilationConfigfields) cap CUDA-graph input memory whencapture=true, so several captured shapes share one buffer sized for the largest instead of paying for each separately.DiffusionStage.with_model_wrapper()runs a callable on each built transformer before denoising, and still composes with block streaming and quantization.VideoDecoder.decode_single_frames()decodes independent one-frame latents without neighbouring frames bleeding into each other.DecodeKeyframes.for_frame_span()and.crop_spatial()crop keyframe planes to a decode window, keeping the nearest anchors outside the window so a tiled decode matches a whole one.DiffusionStage.__call__accepts optionalaudio_fps=when audio should be sized from a different frame rate than the video RoPE time base.
Changed
DFRPipelineandltx_pipelines.dfr_mgputake a distilled checkpoint (--distilled-checkpoint-path) instead of a full checkpoint plus a distilled LoRA. Detailing IC-LoRA and optional user LoRAs are unchanged.DFR temporal rounds use
--temporal-upscalingsinstead of--temporal-upsample-rounds.DFR snaps transformer conditioning fps to 60 whenever playback fps is above 30 (previously only values above 60 were capped), so 48 and 50 fps stay in the trained range. Playback fps is unchanged.
--diffvae-optimization combined_compileno longer requires thenattenextra. Without it the mode falls back to Triton, or to PyTorch when Triton is unavailable.Automatic decode tiling sizes tiles for keyframe-aware decoding as well as plain decoding, so a keyframe decode is less likely to run out of memory on a clip a plain decode fits.
Video pipelines now return a named
PipelineOutputinstead of a 3- or 4-tuple. Extra fields arekeyframesandvideo_latent. Existing unpacking must switch toresult.video,result.audio, and so on.VideoDecoder.decode_video()takeskeyframes=on every implementation. The diffusion decoder uses them; conv logs a warning and decodes without them; distributed splits them across ranks. Callers that never pass keyframes are unaffected; decoder implementations must accept the keyword.NVFP4 quantization and CuTe DSL DiffVAE kernels (
blackwell_dsl) now support Jetson Thor. Build NVFP4 with the arch-specific110atarget.The DFR spatial epilogue now generates the keyframes it ships instead of shipping upscales. The planes it carries in are Lanczos x2 stretches of the previous stage, so anchoring the VAE decode on them blurred the very frames a keyframe is meant to sharpen; they are now conditioning only, and the epilogue attaches a slot a latent frame to the side of each (a slot cannot occupy the frame the plane holds) and ships those full-resolution results.
PipelineOutput.keyframes.pixel_frame_indicestherefore move 8 frames from the carried positions. A candidate that would land inside a temporal window's pinned prefix is skipped, so a few carried positions ship no plane at all.A DFR temporal tile (and epilogue window) begins on a keyframe plane rather than on a mid-canvas latent cell. A tile is denoised as its own clip, whose first cell the model reads as a single pixel frame under the causal convention, so starting it on an 8-frame cell put content and shape in disagreement and ran RoPE time 7 frames ahead of the canvas. Cell 0 is now the plane at the last keyframe before the seam, the cells between it and the seam are pinned to the previous tile's output, and the tile resumes after the seam -- the last pinned cell ends exactly on it, so the seam keyframe is absorbed and needs no conditioning of its own. The prefix length follows the keyframe grid rather than a configured overlap, so the tile split takes no overlap at all and kept runs are disjoint.
DFR temporal tiles and the spatial epilogue now hand over through a pinned prefix instead of synchronising on a shared keyframe. A non-first tile's lead-in cells are conditioned on the previous tile's finished output at strength 1, so they are that output at every step rather than a re-denoised approximation of it. Keyframe conditionings inside a pinned prefix are dropped, because the pinned cells hold the current stage's content while the plane holds an earlier stage's. Anchors from the resume point onward are unchanged, so the tile that owns a stretch still generates it against its keyframes.
The DFR spatial epilogue runs one denoising pass per temporal window, sequentially, instead of one pass over the whole canvas with every temporal tile re-run inside each Euler step. Spatial tiling and its per-step blending are unchanged. Temporal windows were already seam-cut, rectangular and never blended into each other, so this costs the same and is what gives each window a finished predecessor to pin its lead-in to.
Removed
DiffusionVideoDecoder.decode_video_with_keyframes; usedecode_video(..., keyframes=).DFROutput; usePipelineOutput.--checkpoint-pathand--distilled-loraonltx_pipelines.dfr_pipelineandltx_pipelines.dfr_mgpu; pass--distilled-checkpoint-pathinstead.--num-generated-keyframeson DFR; slot positions come from DFR's segment grid.--temporal-upsample-roundson DFR; use--temporal-upscalings.
Fixed
- Automatic decode tiling no longer OOMs in colour conversion on clips that looked like they had spare VRAM. The budget now reserves encoder memory for each chunk, so long videos decode in temporal pieces instead of as one oversize frame dump.
- A pipelines-only install (
ltx-core/ltx-pipelines, no trainer) no longer fails to load Gemma 4 for missing torchvision. uv syncon macOS no longer fails looking for CUDA-only torch wheels. Macs use PyPI (MPS on Apple Silicon); Linux and Windows still use the CUDA 13.2 indexes.--quantization fp8-castwith a LoRA no longer crashes on pre-Ada GPUs such as RTX 3060 or A100.- Blockwise FP8/FP6 quantization no longer fails on very long sequences or token counts that are not a multiple of 4.
- Multi-GPU DFR no longer crashes at tiling resolve, during carry-keyframe decode, or by dropping context at a temporal tile boundary.
- DFR image-to-video stills land on the correct frame after temporal upsampling (opening-frame
frame_idx=0was already correct). AUTO_TILINGis recognised after it crosses a process boundary, so multi-GPU pipelines resolve automatic tiling instead of rejecting the sentinel.
1.2.0 - 2026-08-11
Support for LTX 2.5
Added
- Added support for newer LTX checkpoints, including Gemma 4 text encoders, checkpoint-driven architecture selection, compatibility checks between checkpoints and Gemma roots, and LTX 2.5 training workflows.
- Added diffusion-based video VAE decoding with single- and multi-GPU support, optional NATTEN acceleration, and
chunked_eager,chunked_compile,combined_compile, and datacenter-Blackwell DSL optimization modes. - Added caption-based automatic duration prediction for distilled, text-to-audio, and text/image-to-video pipelines. Use
--auto-duration MIN_SECONDS MAX_SECONDS, or omit--num-frameswith a compatible checkpoint. - Added checkpoint-aware
--vae-checkpoint-pathoverrides and--diffvae-optimizationpipeline options. - Added optional dedicated prompt-enhancement Gemma models through
--prompt-enhancer-gemma-root, plus--enhance-static-cachefor reusable enhancement KV caches. - Added self-managed CUDA graph capture for compiled transformers with
capture=true, alongside controls for dynamic sequence dimensions and perturbed-block recompilation. - Added Euler ancestral diffusion sampling.
- Added checkpoint-aware size-, count-, and automatic-tiling APIs that support non-default VAE compression factors.
- Added reusable model-shell caching, configurable independently from checkpoint-weight caching, through
ModelRegistry. - Added disposable model support so cached module structure and non-persistent buffers can survive weight offloading.
- Added fused CuTe DSL diffusion-VAE kernels for datacenter Blackwell GPUs.
- Added model-version-aware image-conditioning compression defaults and expanded checkpoint metadata utilities.
- Added NVFP4 quantization, which cuts transformer memory use and speeds up inference on Blackwell GPUs.
--quantization nvfp4-castquantizes a BF16 checkpoint while loading it, and--quantization nvfp4-prequantloads an already-quantized NVFP4 checkpoint. Requires a Blackwell GPU and theltx-kernelspackage. - Added
DFRPipeline, which raises output quality and holds detail together noticeably better on fast motion. It generates interior keyframes, adds a full-resolution detailing pass, and can run up to two temporal upsampling rounds for smoother movement. Adds--detailing-lora,--temporal-upsampler-path,--temporal-upsample-rounds, and--num-generated-keyframes. - Added split-checkpoint loading, so you can download only the components a pipeline needs and mix them freely, such as a quantized transformer with a BF16 VAE. Each component has its own flag:
--transformer-path,--text-encoder-path,--video-vae-path,--audio-vae-path, and--duration-head-path. Single-file checkpoints keep working through--checkpoint-pathand--distilled-checkpoint-path. - Added HDR conditioning and output, so generated video can go into a colour-grading or VFX pipeline without a lossy intermediate.
--hdrselects the colour space for EXR conditioning and HDR encoding, which writes scene-linear EXR frames alongside a BT.2020/HLG video, and--video-conditioningaccepts a conditioning video as either SDR or EXR frames.
Changed
- Upgraded
ltx-coreto Transformers 5.8 or newer and added CUDA 13.2-compatible PyTorch, cuDNN, TorchCodec, NATTEN, and kernel-build dependency handling. - Gemma loading now derives model structure from each local Hugging Face configuration, supports Gemma 3 and Gemma 4, and validates that the text encoder matches the LTX checkpoint.
- Gemma tokenization now consistently inserts a leading BOS token while avoiding duplicate BOS tokens for Gemma 3.
- Pipeline image-condition CRF defaults now come from checkpoint metadata: newer checkpoints use their trained value, while explicit CRF values remain unchanged.
- Pipeline constructors now accept a separate VAE checkpoint, diffusion-VAE optimization policy, prompt-enhancement model, and enhancement-cache setting.
- Direct construction of the former convolutional
VideoDecodernow usesConvVideoDecoder;VideoDecoderis now the common protocol implemented by convolutional and diffusion decoders. - Trainer preprocessing, conditioning, validation, and latent decoding now derive spatial and temporal compression factors from checkpoint metadata instead of assuming 32x32x8.
- Trainer validation now uses separate video and audio CFG/STG controls, modality guidance, guidance rescaling, and checkpoint-aware frozen-modality handling.
- Trainer validation defaults now use 960x544x89 output, 24 fps, 30 inference steps, STG block 28, and a substantially expanded negative prompt.
- Legacy trainer validation guidance settings remain readable and are migrated automatically to their per-modality replacements.
- Model configurators now receive complete checkpoint metadata through
from_metadata, enabling architecture and version-dependent construction. - LoRA fusion and model reuse now preserve clean cached weights, avoid unnecessary tensor cloning, and support retained CPU weights.
- Transformer compilation modes that use CUDA graphs now require GPU-resident weights and fail early when used with incompatible offloading.
Fixed
- Fixed multi-GPU video decoding applying pixel normalization twice, using a diffusion decoder's single-step forward path, and incorrectly rejecting temporal tiling configurations that do not actually split a worker's tile.
- Fixed multi-GPU prompt enhancement when encoding and enhancement use different Gemma roots, while retaining shared residency when they use the same root.
- Fixed Gemma 4 text encoding without a leading BOS token.
- Fixed trainer validation guidance for frozen audio or video, audio-only and video-only generation, cross-modal isolation, and modality-specific STG.
- Fixed trainer masks, spatial crops, reference-video alignment, and prefix/suffix validation for checkpoints with non-default VAE scale factors.
- Fixed LoRA fusion corrupting registry-cached weights and blockwise quantization failures when companion scale tensors remained on CPU.
- Fixed diffusion-VAE tiled decoding and multi-GPU blending to use checkpoint-specific geometry with lower peak host memory.
- Fixed 8-bit Gemma loading to resolve standard tokenizer assets and use architecture-agnostic Hugging Face model loading.
- Fixed CUDA builds using mismatched system toolkits or cuDNN sublibraries.
- Fixed diffusion-VAE decode tiling on Apple Silicon. The decode memory budget was probed only on CUDA, so on other backends it reported zero bytes available and no decode tile could ever fit, at any resolution. Automatic tiling now sizes itself from the Metal working set on MPS.
Removed
- Removed
StateDictRegistry; useModelRegistry. - Removed
SpatialTilingConfigandTemporalTilingConfig; useDimensionSizeConfigwithTileSizeConfig. - Removed the old
TilingConfigconstructor fieldsspatial_configandtemporal_config; use the new per-axis size/count tiling configuration. - Removed
ltx_core.model.video_vae.tiling; import tiling APIs fromltx_core.tilingorltx_core.model.video_vae. - Removed
GemmaTextEncoder; useLTXGemmaTextEncoder. - Removed
LTXVGemmaTokenizer; useLTXGemmaTokenizer. - Removed
GEMMA_LLM_KEY_OPSandGEMMA_MODEL_OPS; useget_gemma_ops()for the selected Gemma root. - Removed the hard-coded
Gemma3RopeScaling,Gemma3TextConfig,Gemma3VisionConfig,Gemma3ConfigData, andGEMMA3_CONFIG_FOR_LTXdefinitions; Gemma configuration is now loaded from the model root throughgemma_model_config(). - Removed
ModelConfigurator.from_config(); implement and callfrom_metadata(). - Removed
DiffusionStage.model_context()andDiffusionStage.run(); call the stage directly so it manages transformer construction and disposal. - Removed
ltx_pipelines.utils.allocator_trim_strategy.AllocatorTrimStrategy; import it fromltx_core.allocator_trim_strategy. - Removed
ltx_trainer.training_strategies.VIDEO_SCALE_FACTORS; use checkpoint-derived scale factors orltx_core.types.VIDEO_SCALE_FACTORSwhen the legacy default is explicitly required. - Removed trainer validation fields
guidance_scale,stg_scale, andstg_modefrom the current schema; usevideo_cfg_scale/audio_cfg_scaleandvideo_stg_scale/audio_stg_scale. Legacy configuration files are migrated automatically. - Removed the generic
gemma_i2v_system_prompt.txtandgemma_t2v_system_prompt.txtnames; use thegemma3_*orgemma4_*prompt files matching the encoder family. - Removed
LipDubPipelineand theltx_pipelines.lipdubmodule; useDubItPipelinefromltx_pipelines.dubit.