Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
fix: keep the original source clip in a native LTX Extend render instead of the model's re-decode (#11171)
Adam Eivybf95282373b8
diff --git a/server/lib/README.md b/server/lib/README.mdindex 93d4e1e33c..b9c783a6ab 100644--- a/server/lib/README.md+++ b/server/lib/README.md@@ -241,7 +241,7 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub| `maintenanceSequence.js` | The maintenance ladder shared by both runners: `MAINTENANCE_TASK_ORDER` / `MAINTENANCE_ORDER_GUIDANCE`, the interleaved `MAINTENANCE_SEQUENCE_TYPES` (a perpetual `claim-issue` drain between every pair of audits), `maintenanceStepParams` (audits file issues, documentation does the work), and `buildMaintenanceSteps` (fix mode runs audits directly with one final drain; issue filing can omit claim jobs) which emits quota-burn-shaped run-once steps so `quotaBurnInvoke.js` can run a step from the Schedule tab's manual run or a Quota Burn family plan alike. Pure. || `meetingUrl.js` | Selects the one cacheable join URL (`selectMeetingUrl`) out of a raw Google Calendar event — first http(s) `video` entry point, else `hangoutLink` — and owns the shared `MEETING_URL_MAX` bound. Three-state: `undefined` = the producer never described conferencing, `null` = described with none usable. || `minimaxH3Memory.js` | The declared weight-placement table for the three MiniMax H3 entries (#5420) — H3 is the one video model family whose components fit nowhere unassisted, so "does this box have enough" is a render gate, not a UI fact. `MINIMAX_H3_MEMORY_PROFILES` maps entry id → `{ shippedRepo, shippedRevision, profiles }`, each profile carrying an honest `minMemoryGb` host floor and (CUDA only) a `minVramGb` device floor, ordered best-first. Every capacity number is HOISTED from what already existed — the CUDA tiers are `resolve_offload_profile()`'s own thresholds in `scripts/generate_minimax_h3_cuda.py`, the host floors are the entries' `memoryGb` — the sole new number being `MINIMAX_H3_HOST_RESERVE_GB`, a policy reserve held back for the OS. `applyMiniMaxH3MemoryProfiles(list)` is the load-time backfill (twin of migration 317) and guards BOTH `repo` and `revision`, like the speed-profile decorator. `selectMiniMaxH3MemoryProfile({ model, totalMemoryGb })` picks the best profile the HOST can hold (VRAM is the runner's call — the server has no synchronous device view); `miniMaxH3MemoryDeclineReason()` RETURNS the fail-closed reason so the render path can 400 it and a status surface can show it, and returns `null` on an UNMEASURED host — "not measured" is a deferral to the runner, never the same as zero. `validateMiniMaxH3MemoryProfileTable` / `sanitizeMiniMaxH3MemoryProfiles` warn + strip a hand-edited table (NaN floor, duplicate/reserved id, mis-ordered tiers) at load. |-| `videoContinuity.js` | How chunk N+1 of a chained video render is conditioned on chunk N. `resolveContextFrames(requested)` normalizes the tail-window size (absent → `DEFAULT_CONTEXT_FRAMES` = 22 ≈ 1s @ 24fps; an explicit `0` is preserved as "last frame only", NOT collapsed into the default) and clamps to `MIN_CONTEXT_FRAMES`..`MAX_CONTEXT_FRAMES`. `resolveContinuityStrategy({model, contextFrames})` picks `'window'` (LTX-2 `extend_from_video` conditioned on the prior chunk's last N frames — motion, not just a pose) or `'frame'` (extract the last frame, run i2v), degrading to `'frame'` on any runtime outside `CONTEXT_WINDOW_RUNTIMES` rather than rejecting. `extendLatentFrames` / `extendedPixelFrames` convert across the VAE's `LATENT_FRAME_STRIDE` (8 pixel frames per latent), `contextPrefixFrames({totalFrames, extendLatents})` measures how much of an extend render is echoed context to trim back off before stitching (0 = leave it alone), and `tailWindowStartFrame` gives the cut index for the window itself. Pure — importable from `prepareParams.js` without dragging in `local.js`. Mirrored for the picker in `client/src/lib/videoGenParams.js`, pinned by `videoContinuity.parity.test.js`. |+| `videoContinuity.js` | How chunk N+1 of a chained video render is conditioned on chunk N. `resolveContextFrames(requested)` normalizes the tail-window size (absent → `DEFAULT_CONTEXT_FRAMES` = 22 ≈ 1s @ 24fps; an explicit `0` is preserved as "last frame only", NOT collapsed into the default) and clamps to `MIN_CONTEXT_FRAMES`..`MAX_CONTEXT_FRAMES`. `resolveContinuityStrategy({model, contextFrames})` picks `'window'` (LTX-2 `extend_from_video` conditioned on the prior chunk's last N frames — motion, not just a pose) or `'frame'` (extract the last frame, run i2v), degrading to `'frame'` on any runtime outside `CONTEXT_WINDOW_RUNTIMES` rather than rejecting. `extendLatentFrames` / `extendedPixelFrames` convert across the VAE's `LATENT_FRAME_STRIDE` (8 pixel frames per latent), `contextPrefixFrames({totalFrames, extendLatents})` measures how much of an extend render is echoed context to trim back off before stitching (0 = leave it alone), `tailWindowStartFrame` gives the cut index for the window itself, and `planExtendSplice({sourceFrames, outputFrames, extendLatents})` decides where a plain native Extend joins the ORIGINAL source to the render's new frames (`null` = layout doesn't verify, keep the model output). Pure — importable from `prepareParams.js` without dragging in `local.js`. Mirrored for the picker in `client/src/lib/videoGenParams.js`, pinned by `videoContinuity.parity.test.js`. || `videoPromptLinter.js` | Deterministic lint pass for continuous-video clip prompts (part of #6217, independent of `scriptVideoCompiler.js` #6225 by interface — lints already-built prompt strings plus caller-supplied framing/reference metadata rather than a compiler-shaped clip). `lintClipPrompt(clip, {bible, maxLength})` checks a single clip: a `cutType: 'continue'` clip must open with `"Hard cut to <framing>:"` and use a framing distinct from `previousFraming`; every `references` entry (`{kind: 'cast'|'locations', id}`) must resolve to a bible descriptor present VERBATIM in the prompt (never paraphrased); the prompt must not contain the banned cross-clip referents (same/still/again/continues/as before), negatives (no/without/never), or UI-overlay terms (text/caption/overlay), matched on word boundaries so substrings like "against" or "context" don't false-positive; and the prompt must be `<= MAX_CLIP_PROMPT_LENGTH` (800) characters. Returns `{pass, reasons[]}`, never throws. `lintClips(clips, {bible, maxLength})` lints an ordered chain, deriving each clip's `previousFraming` from the preceding array element's `framing` when not set explicitly. || `videoDisclosure.js` | Video Gen provenance/licensing + backend policy-scope facts (#3674). `VIDEO_MODEL_DISCLOSURES` maps each shipped video model id to `{ shippedRepo, disclosure }` (model card URL, weights license, runtime license, pinned-snapshot download size in decimal GB, review date) — every value checked against a primary upstream source, and any fact that could not be established is OMITTED so the UI renders "Unknown" instead of guessing. `applyVideoDisclosures(list)` is the load-time backfill (twin of migration 237) with the same preservation guards: an existing `disclosure` key wins, a custom id is skipped, and a `repo` pointed at a fork keeps Unknown. `VIDEO_BACKEND_DISCLOSURES` / `videoBackendDisclosure(id)` state where inference runs (`execution: local or hosted`) and whose policy applies — execution facts only, never a restrictiveness ranking. `APACHE_2` / `GEMMA_TERMS` are the shared license descriptors `videoTextEncoders.js` reuses, so a license-text correction reaches both tables from one edit. || `videoDraftDecoders.js` | Preview-fidelity ("draft") video decode (#5423). `VIDEO_DRAFT_DECODERS` declares each entry id → a separately downloaded decoder asset, pin-guarded on repo AND revision; `applyVideoDraftDecoders(list)` is the load-time backfill and `validateDraftDecoderTable` / `sanitizeDraftDecoders` warn + strip a hand-edited row (missing pin, a multi-shard or path-traversing file list, a runtime whose builder emits no draft flags) so a full decode can never report itself as a draft one. `DRAFT_DECODE_FULL` (`'full'`) is a deliberate NO-OP — `isFullDecode(id)` makes absence and that sentinel the same request, so a full-decode render builds byte-identical spawn args. `draftDecodeDeclineReason({ model, decodeId, models, runtimeRevision, assetCached })` RETURNS (never throws) the reason a draft decode does not apply — the model is a delivery target in the finish graph (`isDeliveryVideoModel`), it declares no decoder, the installed runner checkout is not the revision the asset was verified against, or the weights are not downloaded — because a knob that only makes a render cheaper must degrade rather than 400 a submitted job; `resolveVideoDraftDecoder()` returns the concrete asset or `null`. `publicVideoDraftDecodeOptions(model)` is the picker payload (empty for a model with no decoder, so the client renders no control) and `downloadableVideoDraftDecoders(list)` the download targets. The table ships NO entry, and as of 2026-08-30 that is a decision rather than a gap ([ADR](../../docs/decisions/2026-08-30-h3-draft-decoder-asset.md)): the shim substitutes one file into upstream’s checkpoint root under a STRICT key match, so a candidate must be a complete unquantized full-`VideoVAE` checkpoint — which every genuinely light decoder (TAE, quantized repack, decoder-only head) is not. |@@ -431,7 +431,7 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub| `httpDrain.js` | `createHttpDrain(servers)` tracks every ordinary request the HTTP listeners accept; `begin(windowMs)` is the synchronous shutdown step that stops intake on all of them (`server.close()` plus a 503 `Connection: close` gate for requests arriving on still-open keep-alive sockets), turns off keep-alive on in-flight responses, drops open SSE streams, and resolves `{ remaining }` once accepted requests finish or the window ends (#8323). || `httpCompression.js` | `httpCompression` negotiates Brotli/gzip for API and static responses while preserving streaming, ranges, existing encodings, and `no-transform`. || `execGit.js` | `execGit(args, cwd, options)` and `execGitSafe(args, cwd, options)` shared process boundary for Git operations, worktrees, and `services/forgeAuth.js` credential resolution. The safe variant converts rejections into `{ exitCode: 1, stdout: '', stderr }`; `services/git.js` re-exports both for compatibility. `cwd` is REQUIRED — it rejects on a missing/blank one rather than letting `spawn` fall back to `process.cwd()` and run git against PortOS's own checkout (#4554). |-| `ffmpeg.js` | Shared ffmpeg helpers (videoGen + videoTimeline). `probeVideoGeometry(videoPath)` measures a finished clip's `{ width, height, fps, numFrames, durationSec }` (null when any axis is unreadable) for hosted renders that record no frame count. `generateThumbnail(videoPath, jobId, { atSec } = {})` accepts a clamped poster seek while preserving the single-clip default. Includes `probeFrameCount(videoPath)` (metadata `nb_frames`, falling back to a real `-count_frames` pass — expensive, so call it once per file) and `trimVideoFromFrame(videoPath, outPath, {startFrame, fps})` — a frame-EXACT head cut via the `trim` filter (an `-ss` seek can drift a frame, which reads as a stutter at a stitch seam), keeping audio in sync with `atrim` when the clip has a soundtrack and taking `-an` when it doesn't. Re-encodes by necessity, so a later concat must re-encode too; `outPath` may equal the input (temp-file + rename install). Used by the chained-render context window — see `videoContinuity.js`. `buildTrimConcatArgs({inputs, outPath, width, height, fps, withAudio})` builds the argv for the other half of that job: a concat that drops leading frames from some of its inputs inside a `filter_complex` graph, so the cuts ride along with the timeline encode instead of costing one pre-encode per clip (pass `withAudio` only when EVERY input has an audio stream — check with `hasAudioStream`). `H264_ENCODE_ARGS` / `AAC_ENCODE_ARGS` are the shared encode profile: clips produced by different paths here get concatenated together, so a mismatch shows up as one segment graded differently from its neighbours — spread these rather than re-typing the flags. Every re-encode here also pins BT.709: `BT709_CONTAINER_ARGS` (the `colr` atom, always emitted) plus `bt709TagFilter()` → the `BT709_TAG_FILTER` `setparams=…` string, or `null` on an ffmpeg without that filter (`supportsSetparamsFilter()` probes `-filters` once per process; `null` = not probed, and a probe that couldn't run stays uncached). Both halves are required — from ffmpeg 8 the encoder reads color properties off the FRAMES, silently overriding the container flags, so a flags-only output decodes washed-out. `buildTrimConcatArgs` takes the filter as its `colorTagFilter` option rather than probing, to stay pure. `installEncodedVideo(tmpPath, targetPath, label)` is the shared in-place install: move the original aside to a `.bak` on Windows (where `fs.rename` refuses an existing destination), rename the new file over it, and restore the backup — plus unlink the temp — on any failure, returning `{ok:true,outPath}` / `{ok:false,reason}` so the worst case is "skipped", never "the video is gone". Use it for EVERY replace-a-video-with-its-re-encode path; `pipeline/audioMux.js` is the out-of-module caller. `edgeFadeFilter(durationSec)` is the `-af` fade-in/fade-out fragment a mid-song clip (a social cut) appends to its trim. Loudness mastering (#10249): `MASTER_LOUDNESS` (-14 LUFS / -1.5 dBTP / LRA 11), `planLoudnessMaster` (measure then build the linear apply filter; throws on a silent source), `reportMasteredLoudness` (post-encode before/after numbers). |+| `ffmpeg.js` | Shared ffmpeg helpers (videoGen + videoTimeline). `probeVideoGeometry(videoPath)` measures a finished clip's `{ width, height, fps, numFrames, durationSec }` (null when any axis is unreadable) for hosted renders that record no frame count. `generateThumbnail(videoPath, jobId, { atSec } = {})` accepts a clamped poster seek while preserving the single-clip default. Includes `probeFrameCount(videoPath)` (metadata `nb_frames`, falling back to a real `-count_frames` pass — expensive, so call it once per file) and `trimVideoFromFrame(videoPath, outPath, {startFrame, fps})` — a frame-EXACT head cut via the `trim` filter (an `-ss` seek can drift a frame, which reads as a stutter at a stitch seam), keeping audio in sync with `atrim` when the clip has a soundtrack and taking `-an` when it doesn't. Re-encodes by necessity, so a later concat must re-encode too; `outPath` may equal the input (temp-file + rename install). Used by the chained-render context window — see `videoContinuity.js`. `buildTrimConcatArgs({inputs, outPath, width, height, fps, withAudio})` builds the argv for the other half of that job: a concat that drops leading frames from some of its inputs inside a `filter_complex` graph, so the cuts ride along with the timeline encode instead of costing one pre-encode per clip (pass `withAudio` only when EVERY input has an audio stream — check with `hasAudioStream`). `buildExtendSpliceArgs({sourcePath, generatedPath, outPath, keepFrames, newStartFrame, width, height, fps, audio})` is the plain-Extend counterpart: source frames `[0, keep)` + render frames from `newStartFrame`, one encode, audio crossfaded (~40 ms linear) at the seam — driven by `services/videoGen/extendSplice.js`. `H264_ENCODE_ARGS` / `AAC_ENCODE_ARGS` are the shared encode profile: clips produced by different paths here get concatenated together, so a mismatch shows up as one segment graded differently from its neighbours — spread these rather than re-typing the flags. Every re-encode here also pins BT.709: `BT709_CONTAINER_ARGS` (the `colr` atom, always emitted) plus `bt709TagFilter()` → the `BT709_TAG_FILTER` `setparams=…` string, or `null` on an ffmpeg without that filter (`supportsSetparamsFilter()` probes `-filters` once per process; `null` = not probed, and a probe that couldn't run stays uncached). Both halves are required — from ffmpeg 8 the encoder reads color properties off the FRAMES, silently overriding the container flags, so a flags-only output decodes washed-out. `buildTrimConcatArgs` takes the filter as its `colorTagFilter` option rather than probing, to stay pure. `installEncodedVideo(tmpPath, targetPath, label)` is the shared in-place install: move the original aside to a `.bak` on Windows (where `fs.rename` refuses an existing destination), rename the new file over it, and restore the backup — plus unlink the temp — on any failure, returning `{ok:true,outPath}` / `{ok:false,reason}` so the worst case is "skipped", never "the video is gone". Use it for EVERY replace-a-video-with-its-re-encode path; `pipeline/audioMux.js` is the out-of-module caller. `edgeFadeFilter(durationSec)` is the `-af` fade-in/fade-out fragment a mid-song clip (a social cut) appends to its trim. Loudness mastering (#10249): `MASTER_LOUDNESS` (-14 LUFS / -1.5 dBTP / LRA 11), `planLoudnessMaster` (measure then build the linear apply filter; throws on a silent source), `reportMasteredLoudness` (post-encode before/after numbers). || `forgeAccessErrors.js` | `isForgeNoAccessError(err)` / `isForgeNoAccessMessage(text)` / `logForgeNoAccessOnce(scope, repoSpec, detail)` / `forgeNoAccessRemedy(repoSpec)` — tells a PERMANENT `gh` no-access 404 (`Could not resolve to a Repository`, `HTTP 404`) apart from a transient blip. `execGh` stamps `ghNoAccess` on the rejection and keeps it OUT of the consecutive-failure backoff (which caps at 15 min and then re-fires forever), and the polling jobs (`branchReconcile`, `prWatcher`) narrate it exactly once per scope+repo per process with a remedy line instead of once per tick. `__resetForgeNoAccessLog()` is the test seam. || `forkHead.js` | `normalizeForkHead(value)` / `forkRemoteName(ownerLogin)` / `resolveTaskForkHead(metadata)` — the validated `{remoteUrl, ownerLogin}` address of a cross-repository PR's head branch, which has no `origin/<branch>` to attach to. Fail-closed: anything that isn't a usable pair (or that starts with `-`, which git would read as an option) is `null`, so every caller falls back to origin-only behavior instead of guessing. `forkRemoteName` is the deterministic `fork-<owner>` remote name that makes a re-attach reuse the remote it already added. Shared by `appPullRequests.js` (produces), `agentWorktreeCleanup.js`/`prRemediationFollowUp.js` (persist in task metadata), `agentWorkspacePrep.js` (threads), and `worktreeManager.js` (consumes — it has no forge client of its own). `resolveTaskForkHead` is the single reader of the generic `forkHead` task-metadata key, the fork-side counterpart to `taskTargetBranch.js`. || `frameQuality.js` | Scores decoded video frames so a chained render can CHOOSE its continuation anchor instead of taking whatever an end-seek lands on. `scoreFrame({width,height,data},{recency})` combines three terms over a raw single-channel buffer: log-compressed `gradientVariance` (focus), distance of `meanLuma` from mid-grey (exposure penalty, which rejects a fade-to-black or a blown highlight), and a small linear recency bonus that keeps the anchor near the cut when candidates are comparably clean. Usability and ranking are deliberately separate: `usable` gates on raw gradient variance against `MIN_SIGNAL_VARIANCE` alone, so only a mathematically degenerate frame is rejected and a legitimately dark low-key tail still qualifies — the same contract `imageFrameStats.js` keeps. Gating on the composite score instead would reject every night scene on the exposure term. `TAIL_WINDOW_SECONDS` is capped at the reach of the single `-sseof -1.0` seek this replaces: focus is systematically highest at the OLDEST candidate, so a wider window would widen the backward jump at each chain seam rather than tightening it. `pickBestFrame(paths)` decodes each candidate through `sharp` and returns the winner (or `null` when nothing decodes or every candidate is degenerate, so the caller degrades rather than failing a render); its `index` is the position in the array AS PASSED, because the caller turns that into the anchor's time offset. `MAX_CANDIDATES` is derived from the window, never set independently — the ffmpeg-side `-frames:v` cap truncates from the NEWEST end, exactly the frames recency prefers. `TAIL_WINDOW_SECONDS` / `CANDIDATE_FPS` / `MAX_CANDIDATES` define the window the caller decodes from. Consumer: `services/videoGen/local.js#extractLastFrame`. |diff --git a/server/lib/ffmpeg.js b/server/lib/ffmpeg.jsindex 49d2ac4b42..05104a2068 100644--- a/server/lib/ffmpeg.js+++ b/server/lib/ffmpeg.js@@ -784,6 +784,75 @@ export const buildTrimConcatArgs = ({ inputs, outPath, width, height, fps, withA];};+// Build the argv that joins the ORIGINAL source of a native extend render to the+// model's new frames: input 0 is the source clip (kept for frames+// `[0, keepFrames)`), input 1 is the render (kept from `newStartFrame` on).+// Lets a plain Extend deliver the source as shot instead of the model's VAE+// decode of it; `lib/videoContinuity.js#planExtendSplice` supplies the indices.+//+// Video is one single encode of source + new — never a per-round compounding of+// the model's own decode. Geometry and rate normalize to the RENDER's, which is+// what the extension was generated at. Audio follows the same split: the source+// span from the source's track, the new span from the generated one, joined by a+// short LINEAR crossfade (`crossfadeSec`) so the seam doesn't click. The source+// leg runs `crossfadeSec` past the seam so the overlap is source-vs-generated at+// the SAME timeline position, which keeps the output exactly as long as the render.+//+// `audio` is 'splice' (both inputs have a track), 'generated' (only the render+// has one — the source region has nothing to keep, so the render's track stands)+// or 'none'. Pure, like buildTrimConcatArgs: the caller probes and passes+// `colorTagFilter` from `bt709TagFilter()`.+export const buildExtendSpliceArgs = ({+ sourcePath, generatedPath, outPath, keepFrames, newStartFrame, width, height, fps,+ audio = 'none', crossfadeSec = 0.04, colorTagFilter = null,+}) => {+ const keep = Math.floor(Number(keepFrames));+ const start = Math.floor(Number(newStartFrame));+ const rate = Number(fps);+ const canonW = Number(width);+ const canonH = Number(height);+ if (!sourcePath || !generatedPath || !outPath) return null;+ if (!Number.isFinite(keep) || keep <= 0 || !Number.isFinite(start) || start <= 0) return null;+ if (!Number.isFinite(rate) || rate <= 0) return null;+ if (!Number.isFinite(canonW) || canonW <= 0 || !Number.isFinite(canonH) || canonH <= 0) return null;++ const normalize = `scale=${canonW}:${canonH}:force_original_aspect_ratio=decrease,pad=${canonW}:${canonH}:(ow-iw)/2:(oh-ih)/2,setsar=1,fps=${rate}`;+ const tag = typeof colorTagFilter === 'string' && colorTagFilter ? colorTagFilter : null;+ const filters = [+ `[0:v]trim=end_frame=${keep},setpts=PTS-STARTPTS,${normalize}[v0]`,+ `[1:v]trim=start_frame=${start},setpts=PTS-STARTPTS,${normalize}[v1]`,+ `[v0][v1]concat=n=2:v=1:a=0${tag ? '[cv]' : '[outv]'}`,+ ...(tag ? [`[cv]${tag}[outv]`] : []),+ ];++ const canonAudio = 'aresample=48000,aformat=sample_fmts=fltp:channel_layouts=stereo';+ const seamSec = keep / rate;+ const fade = Math.max(0.001, Number(crossfadeSec) || 0);+ if (audio === 'splice') {+ filters.push(+ // apad guards a source track that ends before the seam + overlap, which+ // acrossfade would otherwise reject as shorter than the fade.+ `[0:a]atrim=end=${(seamSec + fade).toFixed(6)},asetpts=PTS-STARTPTS,${canonAudio},apad=whole_dur=${(seamSec + fade).toFixed(6)}[a0]`,+ `[1:a]atrim=start=${seamSec.toFixed(6)},asetpts=PTS-STARTPTS,${canonAudio}[a1]`,+ `[a0][a1]acrossfade=d=${fade.toFixed(6)}:c1=tri:c2=tri[outa]`,+ );+ } else if (audio === 'generated') {+ filters.push(`[1:a]asetpts=PTS-STARTPTS,${canonAudio}[outa]`);+ }++ return [+ '-i', sourcePath, '-i', generatedPath,+ '-filter_complex', filters.join(';'),+ '-map', '[outv]',+ ...(audio === 'none' ? [] : ['-map', '[outa]']),+ ...H264_ENCODE_ARGS,+ ...BT709_CONTAINER_ARGS,+ ...(audio === 'none' ? ['-an'] : AAC_ENCODE_ARGS),+ '-movflags', '+faststart',+ '-y', outPath,+ ];+};+// 2× Lanczos upscale of an MP4 in place. Doubles width and height while// preserving the exact aspect ratio and the audio track. Used as a quick// post-render export option for LTX renders that come out at sub-720pdiff --git a/server/lib/videoContinuity.js b/server/lib/videoContinuity.jsindex f9d67df801..7c56768a6d 100644--- a/server/lib/videoContinuity.js+++ b/server/lib/videoContinuity.js@@ -166,3 +166,38 @@ export const tailWindowStartFrame = ({ totalFrames, frames }) => {if (!Number.isFinite(total) || !Number.isFinite(want)) return 0;return Math.max(0, Math.floor(total) - Math.floor(want));};++/**+ * Where to join the ORIGINAL source clip to the model's new frames in a plain+ * (non-chained) native extend render, so the delivered file keeps the source+ * region as shot instead of the model's VAE decode of it.+ *+ * `extend_from_video` returns `source + extension`; the source half has been+ * encoded and decoded again (and its audio re-synthesised), and extending that+ * result again repeats it over the whole accumulated clip. Conditioning is left+ * alone — only delivery changes: frames `[0, keepFrames)` come from the source+ * file and frames `[newStartFrame, outputFrames)` from the render.+ *+ * LTX only conditions on the first `1 + 8k` source frames, so a source with a+ * ragged tail has `droppedSourceFrames` (< one latent) the model never saw. The+ * extension continues from the last frame the model DID see, so those frames+ * are dropped from the join exactly as they already are in the model's output.+ *+ * Returns `null` — the caller keeps the model output — whenever the measured+ * numbers don't match that layout: an unprobeable clip, an echo longer than the+ * source, or a source whose tail the model skipped by more than a latent (it+ * truncated the context, so which frames it kept is not something to guess at).+ */+export const planExtendSplice = ({ sourceFrames, outputFrames, extendLatents }) => {+ const source = Math.floor(Number(sourceFrames));+ const total = Number(outputFrames);+ const keepFrames = contextPrefixFrames({ totalFrames: total, extendLatents });+ if (!Number.isFinite(source) || source <= 0 || keepFrames <= 0) return null;+ if (keepFrames > source || source - keepFrames >= LATENT_FRAME_STRIDE) return null;+ return {+ keepFrames,+ newStartFrame: keepFrames,+ newFrames: total - keepFrames,+ droppedSourceFrames: source - keepFrames,+ };+};diff --git a/server/lib/videoContinuity.test.js b/server/lib/videoContinuity.test.jsindex 016a79c067..9eb5e0121b 100644--- a/server/lib/videoContinuity.test.js+++ b/server/lib/videoContinuity.test.js@@ -2,7 +2,7 @@ import { describe, it, expect } from 'vitest';import {DEFAULT_CONTEXT_FRAMES, MIN_CONTEXT_FRAMES, MAX_CONTEXT_FRAMES, LATENT_FRAME_STRIDE,resolveContextFrames, resolveContinuityStrategy, supportsContextWindow,- extendLatentFrames, extendedPixelFrames, contextPrefixFrames, tailWindowStartFrame,+ extendLatentFrames, extendedPixelFrames, contextPrefixFrames, tailWindowStartFrame, planExtendSplice,} from './videoContinuity.js';describe('resolveContextFrames', () => {@@ -128,3 +128,29 @@ describe('tailWindowStartFrame', () => {expect(tailWindowStartFrame({ totalFrames: 97, frames: NaN })).toBe(0);});});++describe('planExtendSplice', () => {+ it('keeps every source frame the model saw and takes the rest from the render', () => {+ // 121-frame source (1 + 8*15) extended by 12 latents → 121 + 96 = 217.+ expect(planExtendSplice({ sourceFrames: 121, outputFrames: 217, extendLatents: 12 }))+ .toEqual({ keepFrames: 121, newStartFrame: 121, newFrames: 96, droppedSourceFrames: 0 });+ });++ it('drops the ragged tail LTX never conditioned on', () => {+ // 125 source frames → the model saw 1 + 8*15 = 121 of them.+ expect(planExtendSplice({ sourceFrames: 125, outputFrames: 121 + 24, extendLatents: 3 }))+ .toEqual({ keepFrames: 121, newStartFrame: 121, newFrames: 24, droppedSourceFrames: 4 });+ });++ it('declines (null = keep the model output) when the layout is not the one it assumes', () => {+ // Echo longer than the source: the render holds frames the source lacks.+ expect(planExtendSplice({ sourceFrames: 20, outputFrames: 25 + 24, extendLatents: 3 })).toBeNull();+ // The model skipped more than a latent of the source tail: it truncated the+ // context, so which frames it kept is unknown.+ expect(planExtendSplice({ sourceFrames: 200, outputFrames: 121 + 24, extendLatents: 3 })).toBeNull();+ // Unmeasurable clips and a nonsensical extension never read as a plan.+ expect(planExtendSplice({ sourceFrames: null, outputFrames: 145, extendLatents: 3 })).toBeNull();+ expect(planExtendSplice({ sourceFrames: 121, outputFrames: null, extendLatents: 3 })).toBeNull();+ expect(planExtendSplice({ sourceFrames: 121, outputFrames: 145, extendLatents: 0 })).toBeNull();+ });+});diff --git a/server/services/videoGen/extendSplice.js b/server/services/videoGen/extendSplice.jsnew file mode 100644index 0000000000..88f5cee47d--- /dev/null+++ b/server/services/videoGen/extendSplice.js@@ -0,0 +1,79 @@+/**+ * Video Gen — keep the ORIGINAL source in a native Extend render's output.+ *+ * `ExtendPipeline.extend_from_video` returns `source + extension`, decoded in+ * one pass: the source half has been through the VAE again and its audio+ * through the audio VAE/vocoder. Extending an already-extended clip repeats+ * that over the whole accumulated clip, so picture and sound drift each round.+ * Conditioning is untouched; this changes DELIVERY — the source span comes+ * from the source file, the new span from the render, joined at the seam+ * (layout math: `planExtendSplice` in lib/videoContinuity.js).+ *+ * Best-effort by design: any failure (no ffmpeg, an unprobeable clip, a layout+ * that doesn't match, an encode that doesn't verify) leaves the model's own+ * output in place and reports why. The render is never failed over this.+ */++import { existsSync } from 'fs';+import { unlinkGuarded } from '../../lib/fileUtils.js';+import {+ findFfmpeg, runFfmpegProcess, probeFrameCount, probeVideoStreamInfo,+ hasAudioStream, bt709TagFilter, buildExtendSpliceArgs, installEncodedVideo,+} from '../../lib/ffmpeg.js';+import { planExtendSplice } from '../../lib/videoContinuity.js';++/** Linear crossfade at the source/generated audio seam (~one 25 Hz audio token). */+export const SEAM_CROSSFADE_SEC = 0.04;++/**+ * Replace the source region of `outputPath` with the original `sourcePath`.+ *+ * Resolves `{ applied: true, keptFrames, droppedSourceFrames }` once the+ * spliced file is verified and installed over `outputPath`, or+ * `{ applied: false, reason }` with the render left exactly as the model wrote it.+ */+export async function spliceExtendSource({ sourcePath, outputPath, extendLatents }) {+ const tmpPath = `${outputPath}.splice.mp4`;+ const skip = (reason) => {+ console.warn(`⚠️ Extend source splice skipped, keeping the model output: ${reason}`);+ return { applied: false, reason };+ };+ if (!sourcePath || !existsSync(sourcePath)) return skip('source file missing');+ if (!outputPath || !existsSync(outputPath)) return skip('rendered file missing');+ const ffmpeg = await findFfmpeg();+ if (!ffmpeg) return skip('ffmpeg not found');++ const [sourceFrames, outputFrames, outputInfo] = await Promise.all([+ probeFrameCount(sourcePath), probeFrameCount(outputPath), probeVideoStreamInfo(outputPath),+ ]);+ const plan = planExtendSplice({ sourceFrames, outputFrames, extendLatents });+ if (!plan) return skip(`frame layout does not match (source ${sourceFrames ?? '?'}, output ${outputFrames ?? '?'}, +${extendLatents} latents)`);+ if (!outputInfo.width || !outputInfo.height || !outputInfo.fps) return skip('render geometry unreadable');++ const [sourceHasAudio, outputHasAudio] = await Promise.all([hasAudioStream(sourcePath), hasAudioStream(outputPath)]);+ const audio = !outputHasAudio ? 'none' : sourceHasAudio ? 'splice' : 'generated';+ const args = buildExtendSpliceArgs({+ sourcePath, generatedPath: outputPath, outPath: tmpPath,+ keepFrames: plan.keepFrames, newStartFrame: plan.newStartFrame,+ width: outputInfo.width, height: outputInfo.height, fps: outputInfo.fps,+ audio, crossfadeSec: SEAM_CROSSFADE_SEC, colorTagFilter: await bt709TagFilter(),+ });+ if (!args) return skip('could not build the splice graph');++ const encoded = await runFfmpegProcess({ bin: ffmpeg, args });+ if (!encoded.ok) {+ await unlinkGuarded(tmpPath).catch(() => {});+ return skip(encoded.reason);+ }+ // Verify before replacing anything: the join must be exactly the render's+ // length (kept source frames + new frames), or the seam is off by a frame.+ const splicedFrames = await probeFrameCount(tmpPath);+ if (splicedFrames !== outputFrames) {+ await unlinkGuarded(tmpPath).catch(() => {});+ return skip(`spliced clip has ${splicedFrames ?? '?'} frames, expected ${outputFrames}`);+ }+ const installed = await installEncodedVideo(tmpPath, outputPath, 'extend-spliced');+ if (!installed.ok) return skip(installed.reason);+ console.log(`🧵 Extend kept the original source: ${plan.keepFrames} source + ${plan.newFrames} new frames${audio === 'splice' ? ' (audio crossfaded at the seam)' : ''}`);+ return { applied: true, keptFrames: plan.keepFrames, droppedSourceFrames: plan.droppedSourceFrames };+}diff --git a/server/services/videoGen/extendSplice.test.js b/server/services/videoGen/extendSplice.test.jsnew file mode 100644index 0000000000..be57c33402--- /dev/null+++ b/server/services/videoGen/extendSplice.test.js@@ -0,0 +1,91 @@+/**+ * Native Extend keeps the ORIGINAL source in the delivered file (#11171).+ *+ * Runs real ffmpeg on tiny synthetic clips: a black "source" and a white+ * "render" whose opening frames stand in for the model's re-decode of the+ * source. The splice must put source pixels before the seam and render pixels+ * after it, at the render's exact length, and must leave the render untouched+ * whenever the layout doesn't verify.+ */+import { describe, it, expect, beforeAll, afterAll } from 'vitest';+import { mkdtempSync, rmSync, existsSync, readFileSync } from 'fs';+import { tmpdir } from 'os';+import { join } from 'path';+import { execFile } from 'child_process';+import { promisify } from 'util';+import { findFfmpeg, runFfmpegProcess, probeFrameCount, hasAudioStream } from '../../lib/ffmpeg.js';+import { spliceExtendSource } from './extendSplice.js';++const execFileAsync = promisify(execFile);++let tmpDir;+let ffmpeg;++const makeClip = (name, { color, frames, tone }) => runFfmpegProcess({+ bin: ffmpeg,+ args: [+ '-f', 'lavfi', '-i', `color=c=${color}:size=64x48:rate=24`,+ '-f', 'lavfi', '-i', `sine=frequency=${tone}:sample_rate=48000`,+ '-frames:v', String(frames), '-shortest',+ '-c:v', 'libx264', '-pix_fmt', 'yuv420p', '-c:a', 'aac', '-y', join(tmpDir, name),+ ],+ stderrTailBytes: 0,+}).then((r) => (r.ok ? join(tmpDir, name) : null));++// Mean luma of frame `n`, 0–255.+const frameLuma = async (path, n) => {+ const { stdout } = await execFileAsync(ffmpeg, [+ '-v', 'error', '-i', path, '-vf', `select=eq(n\\,${n})`, '-frames:v', '1',+ '-f', 'rawvideo', '-pix_fmt', 'gray', '-',+ ], { encoding: 'buffer', maxBuffer: 1 << 20 });+ return stdout.reduce((sum, v) => sum + v, 0) / stdout.length;+};++beforeAll(async () => {+ tmpDir = mkdtempSync(join(tmpdir(), 'portos-extend-splice-'));+ ffmpeg = await findFfmpeg();+});+afterAll(() => rmSync(tmpDir, { recursive: true, force: true }));++describe('spliceExtendSource', () => {+ it('delivers source frames before the seam and the render after it, at the render length', async () => {+ if (!ffmpeg) return;+ // 17 source frames (1 + 8*2) extended by 2 latents → a 33-frame render.+ const source = await makeClip('source.mp4', { color: 'black', frames: 17, tone: 440 });+ const output = await makeClip('render.mp4', { color: 'white', frames: 33, tone: 880 });+ if (!source || !output) return;++ const result = await spliceExtendSource({ sourcePath: source, outputPath: output, extendLatents: 2 });++ expect(result).toMatchObject({ applied: true, keptFrames: 17, droppedSourceFrames: 0 });+ expect(await probeFrameCount(output)).toBe(33);+ expect(await frameLuma(output, 0)).toBeLessThan(60); // source region: original black+ expect(await frameLuma(output, 16)).toBeLessThan(60);+ expect(await frameLuma(output, 17)).toBeGreaterThan(190); // first new frame: the render+ expect(await frameLuma(output, 32)).toBeGreaterThan(190);+ expect(await hasAudioStream(output)).toBe(true);+ expect(existsSync(`${output}.splice.mp4`)).toBe(false);+ });++ it('keeps the render byte-for-byte when the frame layout does not match', async () => {+ if (!ffmpeg) return;+ // 40 source frames against a render that only echoes 17: the model truncated+ // its context, so there is no seam to trust.+ const source = await makeClip('long-source.mp4', { color: 'black', frames: 40, tone: 440 });+ const output = await makeClip('short-render.mp4', { color: 'white', frames: 33, tone: 880 });+ if (!source || !output) return;+ const before = readFileSync(output);++ const result = await spliceExtendSource({ sourcePath: source, outputPath: output, extendLatents: 2 });++ expect(result.applied).toBe(false);+ expect(readFileSync(output).equals(before)).toBe(true);+ });++ it('declines without touching anything when the source file is gone', async () => {+ const result = await spliceExtendSource({+ sourcePath: join(tmpDir, 'missing.mp4'), outputPath: join(tmpDir, 'render.mp4'), extendLatents: 2,+ });+ expect(result).toEqual({ applied: false, reason: 'source file missing' });+ });+});diff --git a/server/services/videoGen/generateVideo.js b/server/services/videoGen/generateVideo.jsindex f75534e511..4c0705b84a 100644--- a/server/services/videoGen/generateVideo.js+++ b/server/services/videoGen/generateVideo.js@@ -77,6 +77,7 @@ import {resolveVideoDraftDecoder,} from '../../lib/videoDraftDecoders.js';import { isDefaultVideoStreamingMode } from '../../lib/videoStreamingMode.js';+import { extendLatentFrames, supportsContextWindow } from '../../lib/videoContinuity.js';// Re-export the extracted runtime + history surface so existing deep imports// (`from '../videoGen/local.js'`) keep resolving every symbol they used to.export * from './runtimes.js';@@ -735,6 +736,12 @@ export async function generateVideo({ pythonPath, prompt, negativePrompt = '', m...(await getSettings())?.videoGen,...(displaySleep != null ? { displaySleep } : {}),},+ // A plain native Extend delivers the original source, not the model's+ // re-decode of it. Chain chunks (`hidden`) are skipped: the chain trims the+ // echoed context itself, and a window hop's "source" is a temp tail clip.+ extendSplice: mode === 'extend' && extendFromVideoPath && !hidden && !batch && supportsContextWindow(model)+ ? { sourcePath: extendFromVideoPath, extendLatents: extendLatentFrames(parsedNumFrames) }+ : null,});return { jobId, generationId: jobId, filename, mode: 'local', model: modelId };diff --git a/server/services/videoGen/local.test.js b/server/services/videoGen/local.test.jsindex 988bec7e88..c38bd322c7 100644--- a/server/services/videoGen/local.test.js+++ b/server/services/videoGen/local.test.js@@ -300,6 +300,12 @@ vi.mock('../../lib/ffmpeg.js', async () => ({// hfChildEnv() carries the resolved token over the inherited child env; mocking here// avoids touching the real settings layer (which would await an unmocked// `getSettings()` chain and hang the spawn-mock-driven tests).+// The splice itself (real ffmpeg) is covered by extendSplice.test.js; here it is+// a spy so the suite can pin WHICH renders ask for it.+vi.mock('./extendSplice.js', () => ({+ spliceExtendSource: vi.fn(async () => ({ applied: false, reason: 'mocked' })),+}));+vi.mock('../hfToken.js', () => ({hfChildEnv: vi.fn(async () => ({})),getHfToken: vi.fn(async () => null),@@ -6483,3 +6489,42 @@ describe('generateVideo — post-completion native teardown abort', () => {expect(failed.mock.calls[0][0].results).toHaveLength(1);});});++describe('generateVideo — native Extend keeps the original source (#11171)', () => {+ const extendRender = async (jobId, request) => {+ const { spliceExtendSource } = await import('./extendSplice.js');+ vi.mocked(spliceExtendSource).mockClear();+ await renderToTerminal({+ jobId,+ pythonPath: '/usr/bin/python3',+ modelId: 'ltx2_unified',+ prompt: 'a quiet street at dusk',+ width: 512, height: 512, numFrames: 25, fps: 24,+ ...request,+ });+ return vi.mocked(spliceExtendSource);+ };++ it('splices a plain extend of a source clip, sized to the extension the helper was asked for', async () => {+ const splice = await extendRender('splice-plain-extend', { mode: 'extend', extendFromVideoPath: '/mock/source.mp4' });+ // numFrames 25 → 3 latents, the same count --extend-frames carries.+ expect(splice).toHaveBeenCalledExactlyOnceWith(expect.objectContaining({+ sourcePath: '/mock/source.mp4', extendLatents: 3, outputPath: expect.stringContaining('splice-plain-extend'),+ }));+ });++ it('leaves a chain chunk alone — the chain trims its own echoed context', async () => {+ const splice = await extendRender('splice-chain-chunk', { mode: 'extend', extendFromVideoPath: '/mock/tail-window.mp4', hidden: true });+ expect(splice).not.toHaveBeenCalled();+ });++ it('leaves a runtime whose extend is last-frame i2v alone — its output holds no source', async () => {+ const splice = await extendRender('splice-mlx-video', { modelId: 'ltx23_unified', mode: 'extend', extendFromVideoPath: '/mock/source.mp4' });+ expect(splice).not.toHaveBeenCalled();+ });++ it('does not splice a render with no source clip', async () => {+ const splice = await extendRender('splice-text', { mode: 'text' });+ expect(splice).not.toHaveBeenCalled();+ });+});diff --git a/server/services/videoGen/spawnWatch.js b/server/services/videoGen/spawnWatch.jsindex bb8f475d7a..e2f00dc4c5 100644--- a/server/services/videoGen/spawnWatch.js+++ b/server/services/videoGen/spawnWatch.js@@ -40,6 +40,7 @@ import {import { isDisplaySleepEnabled, sleepDisplayForVideo, wakeDisplayForVideo } from './displayPower.js';import { loadHistory, mutateVideoHistory } from './history.js';import { estimateRenderMs } from './eta.js';+import { spliceExtendSource } from './extendSplice.js';import { videoJobState } from './jobState.js';const MODULE_NOT_FOUND_RE = /ModuleNotFoundError: No module named ['"]([^'"]+)['"]/;@@ -72,6 +73,9 @@ export async function spawnAndWatchVideo({numFrames: parsedNumFrames,steps: actualSteps,videoGenSettings,+ // `{ sourcePath, extendLatents }` for a plain native Extend render, whose+ // delivered file should keep the original source (see ./extendSplice.js).+ extendSplice = null,}) {const retainOwnership = () => {// Chained renders use child job IDs; their ambient owner is the outer job.@@ -534,6 +538,10 @@ export async function spawnAndWatchVideo({broadcastSse(job, { type: 'complete', result });videoGenEvents.emit('completed', { generationId: jobId, ...result });} else {+ // Best-effort: a failed splice leaves the model's own output in place.+ if (extendSplice && (await spliceExtendSource({ ...extendSplice, outputPath })).applied) {+ meta.extendSourceOriginal = true;+ }await finalizeGeneratedVideo({ job, jobId, outputPath, filename, meta, actualSeed, mutateHistory: mutateVideoHistory, publication: publications.get(outputPath) });}}diff --git a/server/services/videoGen/spawnWatch.test.js b/server/services/videoGen/spawnWatch.test.jsindex d0379370a9..42036453c6 100644--- a/server/services/videoGen/spawnWatch.test.js+++ b/server/services/videoGen/spawnWatch.test.js@@ -11,6 +11,7 @@ const mocks = vi.hoisted(() => ({closeWatcher: vi.fn(),rm: vi.fn(),optimize: vi.fn(),+ splice: vi.fn(),thumbnail: vi.fn(),history: [],}));@@ -46,6 +47,7 @@ vi.mock('./runtimes.js', () => ({invalidateByovReadyCache: vi.fn(),pickDeathFingerprint: async () => null,}));+vi.mock('./extendSplice.js', () => ({ spliceExtendSource: mocks.splice }));vi.mock('./displayPower.js', () => ({isDisplaySleepEnabled: () => false,sleepDisplayForVideo: vi.fn(),@@ -237,4 +239,35 @@ describe('spawnAndWatchVideo claim lifecycle', () => {expect(completed).not.toHaveBeenCalled();expect(mocks.history).toEqual([]);});++ describe('native Extend source splice', () => {+ const extendSplice = { sourcePath: '/mock/videos/source.mp4', extendLatents: 3 };++ it('splices the original source into the render and records that on the history row', async () => {+ mocks.splice.mockResolvedValue({ applied: true });+ await spawnAndWatchVideo({ ...params, extendSplice });+ await closeChild(0);++ expect(mocks.splice).toHaveBeenCalledExactlyOnceWith({ ...extendSplice, outputPath: params.outputPath });+ expect(mocks.history).toEqual([expect.objectContaining({ id: params.jobId, extendSourceOriginal: true })]);+ });++ it('publishes the model output unflagged when the splice declines', async () => {+ mocks.splice.mockResolvedValue({ applied: false, reason: 'frame layout does not match' });+ await spawnAndWatchVideo({ ...params, extendSplice });+ await closeChild(0);++ expect(completed).toHaveBeenCalledTimes(1);+ expect(mocks.history).toHaveLength(1);+ expect(mocks.history[0]).not.toHaveProperty('extendSourceOriginal');+ });++ it('never touches a render that did not ask for it', async () => {+ await spawnAndWatchVideo(params);+ await closeChild(0);++ expect(mocks.splice).not.toHaveBeenCalled();+ expect(mocks.history[0]).not.toHaveProperty('extendSourceOriginal');+ });+ });});