Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
fix: make timeline preview play the source spans the export renders (#10866)
Adam Eivy13e5510dd732
diff --git a/client/src/components/media/VideoTimelineLanes.jsx b/client/src/components/media/VideoTimelineLanes.jsxindex 4e839db79e..8aec02874f 100644--- a/client/src/components/media/VideoTimelineLanes.jsx+++ b/client/src/components/media/VideoTimelineLanes.jsx@@ -303,7 +303,7 @@ export const AudioRow = memo(function AudioRow({ track, onAdd }) {* unambiguous, so this is where a removed bed gets paused; a detached but still* playing `<audio>` keeps producing sound in some browsers.*/-export function BedAudio({ trackKey, src, registry }) {+export function BedAudio({ trackKey, assetId, src, registry, onDuration }) {const elRef = useRef(null);useEffect(() => {const el = elRef.current;@@ -313,7 +313,31 @@ export function BedAudio({ trackKey, src, registry }) {el.pause();registry.delete(trackKey);};- }, [trackKey, registry]);+ }, [trackKey, registry, src]);++ // Report the file's real length so the preview can resolve this placement the+ // way the export's probe does (see `resolvePlaybackLanes`). Only a finite,+ // positive duration counts — a stream reports Infinity and a failed load+ // never fires `loadedmetadata`, and both must leave the authored span alone.+ // The listener belongs to this effect run: a changed asset or an unmount+ // disarms it, so a late load can't attribute its length to another file.+ useEffect(() => {+ const el = elRef.current;+ if (!el || !onDuration) return undefined;+ let current = true;+ const report = () => {+ if (current && Number.isFinite(el.duration) && el.duration > 0) onDuration(assetId, el.duration);+ };+ if (el.readyState >= 1) report();+ el.addEventListener('loadedmetadata', report);+ return () => {+ current = false;+ el.removeEventListener('loadedmetadata', report);+ };+ }, [assetId, src, onDuration]);+if (!src) return null;- return <audio ref={elRef} src={src} preload="auto" className="hidden" />;+ // Keyed by `src` so a different file always mounts a fresh element: its+ // readyState/duration can never be read back as the previous file's.+ return <audio key={src} ref={elRef} src={src} preload="auto" className="hidden" />;}diff --git a/client/src/lib/README.md b/client/src/lib/README.mdindex ecdcb27b5a..f7f79b96af 100644--- a/client/src/lib/README.md+++ b/client/src/lib/README.md@@ -294,7 +294,7 @@ grep -i "what you want to do" client/src/lib/README.md| `universeGraphLayout.js` | 2D positioning for the Universe Builder graph canvas: `seedPositions` (deterministic phyllotaxis), `stepLayout`/`settleLayout` (force sim with per-kind collision radius), `applyAnchors` (radial rings by kind, timeline lanes by first appearance), `computeFit`, `toScreen`/`toWorld`, `hitTest`, `radiusFor`, `GRAPH_LAYOUTS`. Distinct from `graphSimulation.js`, which settles the Brain/CoS graphs in 3D with no anchors, collision or screen fit. || `universeGraphPoster.js` | Print-ready infographic posters drawn from an `indexGraph` result: `POSTER_LAYOUTS` (cast roster, character dossier, place atlas, timeline strip, relationship web), `POSTER_SIZES`, `POSTER_THEMES`, `posterDimensions`, and `renderPoster(canvas, opts)` — the same call drives the on-screen preview (scale 1) and the 2× PNG download, so they can't diverge. || `universeBuilderCounts.js` | Pure prompt-count helpers for the Universe Builder render buttons (#2374), mirroring the server's compile/skip rules so inline counts match what `/render` will enqueue: `totalVariationCount`, `canonEntryHasContent`, `countCanonWithContent`, `renderPromptCount`, `scopedPromptCount`. The server `compilePrompts`/`synthesizeCanonPrompt` copy is authoritative — keep in sync. |-| `videoTimelineModel.js` | Pure model helpers for the layered video timeline, mirroring `server/services/videoTimeline/segments.js` so the browser preview and the ffmpeg export make the same cut/geometry/mix decisions: `segmentDuration`/`timelineDuration`/`findSegmentAt` (project-time ↔ segment mapping), `fadeMultiplier` (linear, matching ffmpeg's default `tri` curve), `overlayOpacityAt`, `audioTrackStateAt`, `segmentVolumeAt` (clipVolume × segment volume × fade — the product the export writes into each segment audio chain, so the preview auditions the mix that renders), `assetUrl(assetKind, assetFile)`, the edit clamps `clampTrim` (keeps ≥ 1 frame, matching the server's CLIP_TOO_SHORT floor) and `fitFadePatch` (shrinks an over-long fade pair proportionally instead of letting the PATCH 400), the index summary `projectSummary(project, thumbnailFor)` (block count / duration / card thumbnail — reads the `segments` lane, NEVER the derived `clips` mirror, which omits stills), and the save helpers `stripKey`/`withKeys`/`timelinePatch`. Change a rule here and in the server module in the same commit. Fade fitting re-exports the shared `server/lib/videoTimelineFades.js` leaf. |+| `videoTimelineModel.js` | Pure model helpers for the layered video timeline, mirroring `server/services/videoTimeline/segments.js` so the browser preview and the ffmpeg export make the same cut/geometry/mix decisions: `segmentDuration`/`timelineDuration`/`findSegmentAt` (project-time ↔ segment mapping), `fadeMultiplier` (linear, matching ffmpeg's default `tri` curve), `overlayOpacityAt`, `audioTrackStateAt`, `segmentVolumeAt` (clipVolume × segment volume × fade — the product the export writes into each segment audio chain, so the preview auditions the mix that renders), `assetUrl(assetKind, assetFile)`, the edit clamps `clampTrim` (keeps ≥ 1 frame, matching the server's CLIP_TOO_SHORT floor) and `fitFadePatch` (shrinks an over-long fade pair proportionally instead of letting the PATCH 400), the index summary `projectSummary(project, thumbnailFor)` (block count / duration / card thumbnail — reads the `segments` lane, NEVER the derived `clips` mirror, which omits stills), and the save helpers `stripKey`/`withKeys`/`timelinePatch`. Change a rule here and in the server module in the same commit. Fade fitting re-exports the shared `server/lib/videoTimelineFades.js` leaf, and the source-span rules come from `server/lib/videoTimelineSourceSpans.js`: `resolvePlaybackLanes(lanes, { sourceDurationFor, bedDurationFor })` turns the authored lanes into what the export renders (clip trims clamped to the history clip length, beds to their measured file length, fades refit; an unknown length keeps the authored span), `projectSummary` takes an optional `sourceDurationFor` so the index total agrees. || `wrSceneCursor.js` | Resolve which script scene the editor caret sits in (`sceneAtCursor`, `sceneAnchorIndex`) — inverse of WorkEditor's jump-to-scene text search; drives the live render preview's "scene at cursor" target. || `writingGuide.js` | Canonical Writers Room reference data + craft principles rendered by the Guide page (`/writers-room/guide`): `WRITING_LENGTH_TARGETS` (microfiction→novel word/char bands), `BOOK_LENGTH_ESTIMATES` (page-based), `WRITING_PRINCIPLES`, `PLANNED_ANALYSES` (e.g. the emotional-roadmap evaluator), and `classifyByWordCount(n)` for labelling a draft's length. Future word-count gauges / length checks read from here so targets don't drift from the docs. || `universeMarkdownFilename.js` | `slugifyUniverseName` / `universeMarkdownFilename` — client-side safe filename helpers for Universe Markdown world-bible downloads, kept in step with the server attachment name. |diff --git a/client/src/lib/videoTimelineModel.js b/client/src/lib/videoTimelineModel.jsindex eddcb4bc7b..93069ba181 100644--- a/client/src/lib/videoTimelineModel.js+++ b/client/src/lib/videoTimelineModel.js@@ -6,7 +6,11 @@* point of the lane model is that what the editor shows is what renders. When* a rule changes on one side (fade curve, overlay placement, asset kinds),* change it on the other in the same commit. Fade fitting is shared through- * server/lib/videoTimelineFades.js rather than mirrored.+ * server/lib/videoTimelineFades.js rather than mirrored, and so are the+ * source-span rules (server/lib/videoTimelineSourceSpans.js): the export clamps+ * a clip trim to the history clip's real length and a bed to its file's real+ * length, so the preview plays `resolvePlaybackLanes()` — authored lanes run+ * through those same rules — never the authored values directly.** No React, no I/O — safe to import from the page, its child components, and* tests alike.@@ -15,7 +19,15 @@import { clamp } from '../utils/formatters';import { fitFades } from '../../../server/lib/videoTimelineFades.js';+import {+ historySourceDuration,+ resolveClipSpan,+ resolveStillFades,+ resolveAudioSpan,+} from '../../../server/lib/videoTimelineSourceSpans.js';+export { fitFades } from '../../../server/lib/videoTimelineFades.js';+export { historySourceDuration };export const IMAGE_ASSET_KINDS = ['images', 'video-thumbnails'];export const AUDIO_ASSET_KINDS = ['audio', 'music'];@@ -44,6 +56,55 @@ export const segmentDuration = (segment) => {export const timelineDuration = (segments) => (segments || []).reduce((sum, s) => sum + segmentDuration(s), 0);+const sameFades = (entry, fades) => (entry.fadeInSec || 0) === fades.fadeInSec && (entry.fadeOutSec || 0) === fades.fadeOutSec;++/**+ * The segment as the export will play it: a clip's trim clamped to the history+ * clip's real length (`sourceDurationSec`, null when unknown) and its fades+ * refit; a still's fades refit against its hold. Returns the SAME object when+ * nothing had to move, so memoized blocks keep their identity.+ */+export const resolvePlaybackSegment = (segment, sourceDurationSec) => {+ if (!segment) return segment;+ if (segment.type === 'still') {+ const fades = resolveStillFades(segment);+ return sameFades(segment, fades) ? segment : { ...segment, ...fades };+ }+ const { inSec, outSec, fadeInSec, fadeOutSec } = resolveClipSpan(segment, sourceDurationSec);+ if (outSec === segment.outSec && inSec === segment.inSec && sameFades(segment, { fadeInSec, fadeOutSec })) return segment;+ return { ...segment, inSec, outSec, fadeInSec, fadeOutSec };+};++/** An audio bed as the export will play it, given its file's real length (null when unknown). */+export const resolvePlaybackTrack = (track, fileDurationSec) => {+ const span = resolveAudioSpan(track, fileDurationSec);+ const same = span.offsetSec === (track.offsetSec || 0)+ && span.durationSec === (track.durationSec || 0)+ && sameFades(track, span);+ return same ? track : { ...track, ...span };+};++/** Identity of a bed's audio file — what a loaded-duration report is keyed by. */+export const bedAssetId = (track) => `${track.assetKind}/${track.assetFile}`;++/**+ * Authored lanes → the lanes the preview plays. `sourceDurationFor(clipId)` and+ * `bedDurationFor(track)` return a known length in seconds or null; an unknown+ * length leaves the authored span alone. Entries keep their `_key`s and lane+ * positions, so selection and the dnd identity carry straight over. The authored+ * lanes remain what the editor edits and saves — this is a view of them.+ */+export const resolvePlaybackLanes = ({ segments, audio }, { sourceDurationFor, bedDurationFor }) => ({+ segments: (segments || []).map((segment) => resolvePlaybackSegment(+ segment,+ segment.type === 'still' ? null : sourceDurationFor(segment.clipId),+ )),+ audio: {+ ...audio,+ tracks: (audio?.tracks || []).map((track) => resolvePlaybackTrack(track, bedDurationFor(track))),+ },+});+/*** Map project-time `t` to the (segmentIndex, withinSegmentSec) pair the* preview needs. The strict `t < acc + dur` comparison falls through on exact@@ -231,14 +292,21 @@ export const segmentVolumeAt = (segment, clipVolume, within) => {* carries only clip segments, so a project built from stills would report zero* blocks and zero seconds while playing back perfectly in the editor. A v1* project that predates the lane falls back to `clips`, which for it IS the- * lane. `thumbnailFor(clipId)` resolves a video clip's thumbnail filename.+ * lane. `thumbnailFor(clipId)` resolves a video clip's thumbnail filename;+ * the optional `sourceDurationFor(clipId)` (seconds or null) makes the total+ * source-aware, matching what the editor previews and the export renders.*/-export const projectSummary = (project, thumbnailFor) => {+export const projectSummary = (project, thumbnailFor, sourceDurationFor = null) => {const segments = Array.isArray(project?.segments) ? project.segments : (project?.clips || []);let totalSec = 0;let firstThumb = null;for (const segment of segments) {- totalSec += segmentDuration(segment);+ // With history durations available, count what the export will render+ // rather than the trim as it was authored.+ const playable = sourceDurationFor && segment.type !== 'still'+ ? resolvePlaybackSegment(segment, sourceDurationFor(segment.clipId))+ : segment;+ totalSec += segmentDuration(playable);if (firstThumb) continue;firstThumb = segment.type === 'still'? assetUrl(segment.assetKind, segment.assetFile)diff --git a/client/src/lib/videoTimelineModel.test.js b/client/src/lib/videoTimelineModel.test.jsindex 09ba82a2c1..4b80a0e342 100644--- a/client/src/lib/videoTimelineModel.test.js+++ b/client/src/lib/videoTimelineModel.test.js@@ -19,6 +19,7 @@ import {segmentVolumeAt,projectSummary,canvasAspectRatio,+ resolvePlaybackLanes,} from './videoTimelineModel';const clip = (inSec, outSec) => ({ type: 'clip', clipId: 'c', inSec, outSec });@@ -247,6 +248,19 @@ describe('projectSummary — the index must count the lane, not the mirror', ()expect(summary).toMatchObject({ blockCount: 2, totalSec: 5 });});+ it('counts the source-clamped trim when history durations are supplied, like the export', () => {+ const project = {+ segments: [+ { type: 'clip', clipId: 'c1', inSec: 0, outSec: 8 },+ { type: 'still', assetKind: 'images', assetFile: 'plate.png', durationSec: 3 },+ ],+ };+ expect(projectSummary(project, thumbFor, () => 1)).toMatchObject({ totalSec: 4 });+ // An unknown source length keeps the authored trim.+ expect(projectSummary(project, thumbFor, () => null)).toMatchObject({ totalSec: 11 });+ expect(projectSummary(project, thumbFor)).toMatchObject({ totalSec: 11 });+ });+it('does not report a stills-only project as empty', () => {const summary = projectSummary({segments: [{ type: 'still', assetKind: 'images', assetFile: 'plate.png', durationSec: 4 }],@@ -344,3 +358,44 @@ describe('shared timeline fade contract', () => {expect(previewFitFades).toBe(sharedFitFades);});});++describe('resolvePlaybackLanes — the preview plays what the export renders', () => {+ const lanes = {+ segments: [+ { _key: 's0', type: 'clip', clipId: 'c1', inSec: 0, outSec: 8, fadeInSec: 3, fadeOutSec: 3, volume: 1 },+ { _key: 's1', type: 'clip', clipId: 'c2', inSec: 0, outSec: 4, fadeInSec: 0, fadeOutSec: 0, volume: 1 },+ ],+ audio: {+ clipVolume: 1,+ tracks: [{ _key: 'b0', assetKind: 'music', assetFile: 'bed.mp3', startSec: 0, offsetSec: 0, durationSec: 10, volume: 1, fadeInSec: 4, fadeOutSec: 4 }],+ },+ };+ const resolve = (sources = {}, beds = {}) => resolvePlaybackLanes(lanes, {+ sourceDurationFor: (id) => sources[id] ?? null,+ bedDurationFor: (track) => beds[track.assetFile] ?? null,+ });++ it('matches the export for a clip shorter than its trim (resolveTimeline: 1s, fades 0.5s/0.5s)', () => {+ const { segments } = resolve({ c1: 1 });+ expect(timelineDuration(segments)).toBe(5);+ // The second cut starts at 1s, not 8s.+ expect(findSegmentAt(segments, 1.2)).toMatchObject({ index: 1, startAtProj: 1 });+ expect(segments[0]).toMatchObject({ outSec: 1, fadeInSec: 0.5, fadeOutSec: 0.5, _key: 's0' });+ expect(segmentVolumeAt(segments[0], 1, 0.5)).toBe(1);+ });++ it('matches the export for a short bed file (resolveTimeline: 2s, fades 1s/1s)', () => {+ const [track] = resolve({}, { 'bed.mp3': 2 }).audio.tracks;+ expect(track).toMatchObject({ offsetSec: 0, durationSec: 2, fadeInSec: 1, fadeOutSec: 1, _key: 'b0' });+ expect(audioTrackStateAt(track, 0.5).volume).toBeCloseTo(0.5);+ expect(audioTrackStateAt(track, 2).active).toBe(false);+ });++ it('keeps the authored spans, by identity, when no source length is known', () => {+ const resolved = resolve();+ expect(resolved.segments[1]).toBe(lanes.segments[1]);+ // Fades already fit the authored span, so nothing moves.+ expect(resolved.segments[0]).toMatchObject({ outSec: 8, fadeInSec: 3, fadeOutSec: 3 });+ expect(resolved.audio.tracks[0]).toBe(lanes.audio.tracks[0]);+ });+});diff --git a/client/src/pages/VideoTimeline.jsx b/client/src/pages/VideoTimeline.jsxindex 6f31a9534e..e65480315d 100644--- a/client/src/pages/VideoTimeline.jsx+++ b/client/src/pages/VideoTimeline.jsx@@ -5,7 +5,7 @@ import PageSkeleton from '../components/ui/PageSkeleton';import toast from '../components/ui/Toast';import * as api from '../services/api';import { formatDurationSec } from '../utils/formatters';-import { projectSummary } from '../lib/videoTimelineModel';+import { projectSummary, historySourceDuration } from '../lib/videoTimelineModel';export default function VideoTimeline() {const navigate = useNavigate();@@ -54,7 +54,8 @@ export default function VideoTimeline() {const projectStats = useMemo(() => {const historyMap = new Map(history.map((h) => [h.id, h]));const thumbnailFor = (clipId) => historyMap.get(clipId)?.thumbnail;- return new Map(projects.map((p) => [p.id, projectSummary(p, thumbnailFor)]));+ const sourceDurationFor = (clipId) => historySourceDuration(historyMap.get(clipId));+ return new Map(projects.map((p) => [p.id, projectSummary(p, thumbnailFor, sourceDurationFor)]));}, [projects, history]);return (diff --git a/client/src/pages/VideoTimelineEditor.jsx b/client/src/pages/VideoTimelineEditor.jsxindex bfb9fdeaa6..8f34d7ae98 100644--- a/client/src/pages/VideoTimelineEditor.jsx+++ b/client/src/pages/VideoTimelineEditor.jsx@@ -39,6 +39,9 @@ import {overlayOpacityAt,audioTrackStateAt,segmentVolumeAt,+ resolvePlaybackLanes,+ historySourceDuration,+ bedAssetId,clampTrim,fitFadePatch,timelinePatch,@@ -118,6 +121,11 @@ function TimelineProjectEditor({ projectId }) {const [musicTracks, setMusicTracks] = useState([]);// Which catalogues have actually been fetched — see knownAbsent below.const [loaded, setLoaded] = useState({ clips: false, images: false, music: false });+ // Real lengths the browser measured for the beds' audio files, keyed by asset+ // identity (`music/<file>`). The preview resolves each placement against its+ // file's length exactly as the export's probe does; an asset with no entry+ // (metadata not loaded yet, or the load failed) keeps its authored span.+ const [bedDurations, setBedDurations] = useState({});const [loading, setLoading] = useState(true);const [error, setError] = useState(null);// The three lanes live in ONE state object so a save always ships a@@ -282,7 +290,19 @@ function TimelineProjectEditor({ projectId }) {const isOverlayMissing = useCallback((ov) => knownAbsent(ov.assetKind, ov.assetFile), [knownAbsent]);const isBedMissing = useCallback((tr) => knownAbsent(tr.assetKind, tr.assetFile), [knownAbsent]);- const total = useMemo(() => timelineDuration(segments), [segments]);+ // What the export will actually render: the authored lanes resolved against+ // the history clips' real lengths and the beds' measured file lengths, by the+ // same rules `resolveTimeline` applies server-side. Playback, block lengths,+ // fades and overlay timing all read THESE; the inspector and the save path+ // keep reading the authored `lanes`, so nothing probe-derived is persisted.+ const playback = useMemo(() => resolvePlaybackLanes(lanes, {+ sourceDurationFor: (clipId) => historySourceDuration(historyMap.get(clipId)),+ bedDurationFor: (track) => bedDurations[bedAssetId(track)] ?? null,+ }), [lanes, historyMap, bedDurations]);+ const playSegments = playback.segments;+ const playTracks = playback.audio.tracks;++ const total = useMemo(() => timelineDuration(playSegments), [playSegments]);const canvasAspect = useMemo(() => canvasAspectRatio(segments, (clipId) => historyMap.get(clipId)),[segments, historyMap],@@ -518,10 +538,10 @@ function TimelineProjectEditor({ projectId }) {}, [playing, total]);const { index: activeIndex, within: activeWithin } = useMemo(- () => (segments.length > 0 ? findSegmentAt(segments, t) : { index: -1, within: 0 }),- [segments, t],+ () => (playSegments.length > 0 ? findSegmentAt(playSegments, t) : { index: -1, within: 0 }),+ [playSegments, t],);- const activeSegment = activeIndex >= 0 ? segments[activeIndex] : null;+ const activeSegment = activeIndex >= 0 ? playSegments[activeIndex] : null;// `within` advances every frame; the media-sync effects read it from a ref so// they re-run on a segment CHANGE rather than once per animation frame.const withinRef = useRef(0);@@ -594,7 +614,7 @@ function TimelineProjectEditor({ projectId }) {// Drive the bed <audio> elements from the same playhead the export mixes// against, so what the user hears while scrubbing is what amix will produce.useEffect(() => {- for (const track of audio.tracks) {+ for (const track of playTracks) {const el = bedRefs.current.get(track._key);if (!el) continue;const state = audioTrackStateAt(track, t);@@ -612,7 +632,7 @@ function TimelineProjectEditor({ projectId }) {if (playing && el.paused) el.play().catch(() => {});if (!playing && !el.paused) el.pause();}- }, [audio.tracks, t, playing, muted]);+ }, [playTracks, t, playing, muted]);// Stop every bed when the editor unmounts — a detached <audio> that was// playing keeps producing sound in some browsers.@@ -740,6 +760,24 @@ function TimelineProjectEditor({ projectId }) {overlay: ov,})), [overlays]);+ // BedAudio reports a file's measured length here. Identical reports bail out+ // so a re-fired `loadedmetadata` doesn't re-derive the playback lanes.+ const reportBedDuration = useCallback((assetId, seconds) => {+ setBedDurations((prev) => (prev[assetId] === seconds ? prev : { ...prev, [assetId]: seconds }));+ }, []);+ // Forget a file's length once no bed uses it: a removed bed's measurement must+ // not resolve a later placement before that placement has loaded its own.+ useEffect(() => {+ const used = new Set(audio.tracks.map(bedAssetId));+ setBedDurations((prev) => {+ const stale = Object.keys(prev).filter((id) => !used.has(id));+ if (stale.length === 0) return prev;+ const next = { ...prev };+ for (const id of stale) delete next[id];+ return next;+ });+ }, [audio.tracks]);+const selectSegment = useCallback((key) => setSelection({ lane: 'segment', key }), []);const removeSegment = useCallback((key) => removeFromLane('segment', key), [removeFromLane]);const selectOverlay = useCallback((key) => setSelection({ lane: 'overlay', key }), []);@@ -984,8 +1022,10 @@ function TimelineProjectEditor({ projectId }) {<BedAudiokey={track._key}trackKey={track._key}+ assetId={bedAssetId(track)}src={assetUrl(track.assetKind, track.assetFile)}registry={bedRefs.current}+ onDuration={reportBedDuration}/>))}@@ -1058,7 +1098,7 @@ function TimelineProjectEditor({ projectId }) {><SortableContext items={segmentKeys} strategy={horizontalListSortingStrategy}><div className="flex gap-1 items-stretch min-w-min py-1">- {segments.map((segment) => (+ {playSegments.map((segment) => (<TimelineBlockkey={segment._key}clip={segment}@@ -1092,7 +1132,7 @@ function TimelineProjectEditor({ projectId }) {<FloatingLanetitle="Audio"- entries={audio.tracks}+ entries={playTracks}emptyHint="Add a bed from the Audio tab"tone="bg-port-success/15 border-port-success/40"labelOf={bedLabel}diff --git a/client/src/pages/VideoTimelineEditor.test.jsx b/client/src/pages/VideoTimelineEditor.test.jsxindex a90a21b1ca..d8fa56a807 100644--- a/client/src/pages/VideoTimelineEditor.test.jsx+++ b/client/src/pages/VideoTimelineEditor.test.jsx@@ -472,3 +472,111 @@ describe('lane caps', () => {await waitFor(() => expect(screen.getByRole('button', { name: 'Remove plate.png from timeline' })).toBeInTheDocument());});});++// The saved lanes hold what the author ASKED for; the export resolves them+// against the real sources (a shortened history clip, a short music file). The+// preview must play that resolved timeline, or its cuts, fades and overlays run+// on a different clock than the render (#10866).+describe('source-aware playback', () => {+ const seekTo = (seconds) => fireEvent.change(screen.getByRole('slider', { name: 'Playhead position' }), { target: { value: String(seconds) } });++ it('advances past a clip whose source is shorter than its authored trim, matching the export', async () => {+ // Authored 0-8s with 3s+3s fades, but the history entry is 1s long: the+ // export renders 1s with the fades refit to 0.5s+0.5s.+ api.history = [{ ...api.history[0], numFrames: 24, fps: 24 }];+ api.project = project({+ segments: [{ ...clipSegment, outSec: 8, fadeInSec: 3, fadeOutSec: 3 }, stillSegment],+ overlays: [{ ...overlayEntry, startSec: 0, durationSec: 10, fadeOutSec: 2 }],+ });+ await renderEditor();++ // Total is 1s clip + 3s still, not 8 + 3.+ expect(screen.getByRole('slider', { name: 'Playhead position' })).toHaveAttribute('max', '4');++ // Half a second in, the refit fade is already at full level: no scrim.+ seekTo(0.5);+ expect(screen.queryByTestId('fade-scrim')).not.toBeInTheDocument();++ // The next cut lands at 1s, so the still is up at 1.5s.+ const previewStill = () => document.querySelector('img.object-contain[src="/data/images/plate.png"]');+ expect(previewStill()).toBeNull();+ seekTo(1.5);+ expect(previewStill()).not.toBeNull();++ // The overlay's 10s window is clamped to the 4s timeline, so its fade-out+ // starts at 2s and is half spent at 3s.+ seekTo(3);+ expect(Number(screen.getByTestId('overlay-preview').style.opacity)).toBeCloseTo(0.5);+ });++ it('keeps the authored spans when the history clip has no frame metadata', async () => {+ api.history = [{ ...api.history[0], numFrames: undefined }];+ api.project = project({ segments: [{ ...clipSegment, outSec: 8, fadeInSec: 3 }] });+ await renderEditor();+ expect(screen.getByRole('slider', { name: 'Playhead position' })).toHaveAttribute('max', '8');+ });++ describe('bed audio', () => {+ const shortBed = { ...bedEntry, durationSec: 10, fadeInSec: 4, fadeOutSec: 4 };+ // The lane block is the one carrying an inline width; a selected bed's+ // inspector repeats the filename.+ const bedBlock = () => screen.getAllByText('bed.mp3').map((node) => node.parentElement).find((node) => node.style.width);+ const reportBedLength = (el, seconds) => {+ Object.defineProperty(el, 'duration', { configurable: true, value: seconds });+ act(() => { el.dispatchEvent(new Event('loadedmetadata')); });+ };++ it('plays a short file on the fade envelope the export builds once its length loads', async () => {+ api.project = project({ segments: [clipSegment], audio: { clipVolume: 1, tracks: [shortBed] } });+ const { container } = renderEditorRoute();+ await awaitPageLoaded('Loading timeline project');+ const el = container.querySelector('audio');++ // Until the length is known the authored placement stands.+ expect(bedBlock().style.width).toBe('600px');++ reportBedLength(el, 2);+ expect(bedBlock().style.width).toBe('120px');++ // 2s file, 4s+4s authored fades refit to 1s+1s: gain is 0.5 at 0.5s.+ seekTo(0.5);+ await waitFor(() => expect(el.volume).toBeCloseTo(0.5));++ // Past the end of the file the bed is inactive.+ seekTo(3);+ await waitFor(() => expect(el.volume).toBe(0));+ });++ it('ignores a length that is not finite and never persists a measured one', async () => {+ api.project = project({ segments: [clipSegment], audio: { clipVolume: 1, tracks: [shortBed] } });+ const { container, unmount } = renderEditorRoute();+ await awaitPageLoaded('Loading timeline project');+ const el = container.querySelector('audio');++ reportBedLength(el, Infinity);+ expect(bedBlock().style.width).toBe('600px');++ reportBedLength(el, 2);+ unmount();+ // Nothing was edited, so nothing is saved; the measured span is a view.+ expect(api.updateTimelineProject).not.toHaveBeenCalled();+ });++ it('forgets a removed bed so a later placement of the file starts from its own span', async () => {+ api.project = project({ segments: [clipSegment], audio: { clipVolume: 1, tracks: [shortBed] } });+ const { container } = renderEditorRoute();+ await awaitPageLoaded('Loading timeline project');+ reportBedLength(container.querySelector('audio'), 2);+ expect(bedBlock().style.width).toBe('120px');++ fireEvent.click(screen.getByRole('button', { name: 'Remove bed.mp3 from timeline' }));+ expect(container.querySelector('audio')).toBeNull();++ fireEvent.click(screen.getByRole('tab', { name: /Audio/ }));+ fireEvent.click(screen.getByRole('button', { name: 'Add' }));+ // The fresh placement is the 4s default (the timeline is 4s) until ITS+ // element reports a length — the earlier measurement must not leak in.+ expect(bedBlock().style.width).toBe('240px');+ });+ });+});diff --git a/server/lib/README.md b/server/lib/README.mdindex f8d09ea38b..50b5744dc2 100644--- a/server/lib/README.md+++ b/server/lib/README.md@@ -787,6 +787,7 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub| `providerTypes.js` | Pure shared `isCliProvider`, `isTuiProvider`, `isApiProvider`, `isProcessProvider`, and type-gated `isClaudeHarnessProvider` predicates. || `videoTimelineFades.js` | Browser-safe `fitFades()` proportionally fits a fade pair to its visible duration. Shared by timeline normalization/export and the editor preview/trim controls; legacy import paths re-export it. |+| `videoTimelineSourceSpans.js` | Browser-safe source-span resolution for the video timeline: `resolveClipSpan` (trim clamped to the history clip length, fades refit), `resolveStillFades`, `resolveAudioSpan` (bed offset/length/fades fit to the real file length; an unknown length keeps the authored slice), `historySourceDuration`, `MIN_MEDIA_SEC`. One implementation for the ffmpeg export (`resolveTimeline`) and the editor preview, so both play the same timeline. || `voiceEcho.js` | Pure voice-echo detection helpers — tokenization, trigram building, thresholds shared by server and client echo gates. || `voiceEngines.js` | Shared TTS engine IDs, display metadata, supported-engine set, persisted configuration keys, and retired TTS config migration. || `voiceModelAssets.js` | Completion contract for downloaded voice model assets (Whisper `.bin`, Piper `.onnx` + `.onnx.json`). `inspectVoiceAsset(kind, path)` is the read-only readiness check (`verified` / `unverified` / `incomplete` / `missing` — stat plus one small JSON read, no hashing or network); `verifyVoiceAssetHashes` and `writeVoiceAssetReceipt` are the script-side halves that re-hash and record the `.portos-complete.json` receipt after promotion. `isManagedVoiceAsset` separates setup-script-owned files from user-supplied ones. Shared by `voice/bootstrap.js`, `voice/health.js` and `scripts/voice-asset.js`. |diff --git a/server/lib/index.js b/server/lib/index.jsindex d366bc35de..540f9e243e 100644--- a/server/lib/index.js+++ b/server/lib/index.js@@ -755,6 +755,7 @@ export * from './creativeDirectorVideoCompiler.js';export * from './providerTypes.js';export * from './notificationTypes.js';export * from './videoTimelineFades.js';+export * from './videoTimelineSourceSpans.js';// `tokenize` collides with `bm25.js`'s own tokenizer — namespaced like `runners`.export * as voiceEcho from './voiceEcho.js';export * from './voiceEngines.js';diff --git a/server/lib/videoTimelineSourceSpans.js b/server/lib/videoTimelineSourceSpans.jsnew file mode 100644index 0000000000..fe94e65cc5--- /dev/null+++ b/server/lib/videoTimelineSourceSpans.js@@ -0,0 +1,78 @@+import { fitFades } from './videoTimelineFades.js';++/**+ * Browser-safe source-span resolution for the layered video timeline.+ *+ * A saved lane stores the span the author ASKED for; the sources can be shorter+ * (a history entry whose clip was re-cut, a music file shorter than the+ * guessed placement). The ffmpeg export resolves those spans against the real+ * sources, and the editor preview must resolve them with the SAME rules or it+ * plays a different timeline than the one that renders — a later cut lands+ * seconds off and a fade envelope ramps on the wrong clock. Both sides call+ * these functions (`resolveTimeline` in services/videoTimeline/local.js and the+ * client model in client/src/lib/videoTimelineModel.js); no I/O, no React.+ *+ * "Unknown" is never "zero": a source whose length could not be established+ * (no frame metadata, no ffprobe, a metadata load that failed) keeps the+ * authored span rather than collapsing the lane entry to nothing.+ */++// Floor for a probe-clamped audio slice — atrim with start === end produces an+// empty stream that amix rejects.+export const MIN_MEDIA_SEC = 0.05;++// Seconds, or null when the span cannot be established. Rejects anything that+// is not a positive finite number so Infinity (a streamed media element) and 0+// (a probe that found no duration) read as "unknown", not "available: forever"+// or "available: nothing".+const knownSeconds = (value) => (Number.isFinite(value) && value > 0 ? value : null);++/** Length of a video-history entry's clip in seconds, or null when unknown. */+export function historySourceDuration(entry) {+ return entry?.numFrames && entry?.fps ? knownSeconds(entry.numFrames / entry.fps) : null;+}++/**+ * The trim a clip segment actually plays: `outSec` clamped to the source length+ * (when known) and the fade pair refit against what is left, so a fade authored+ * for the stored trim cannot outlast a shortened clip (ffmpeg renders a fade+ * with a negative start time as an all-black segment). `duration` is never+ * negative — an `inSec` past the end of the source resolves to 0 and the caller+ * decides whether that is an error (the export rejects it; the preview skips it).+ */+export function resolveClipSpan(segment, sourceDurationSec) {+ const source = knownSeconds(sourceDurationSec);+ const inSec = Math.max(0, segment.inSec || 0);+ const outSec = source != null ? Math.min(segment.outSec || 0, source) : (segment.outSec || 0);+ const duration = Math.max(0, outSec - inSec);+ return { inSec, outSec, duration, ...fitFades(segment.fadeInSec, segment.fadeOutSec, duration) };+}++/** A still's fade pair refit against its own hold. */+export function resolveStillFades(segment) {+ return fitFades(segment.fadeInSec, segment.fadeOutSec, Math.max(0, segment.durationSec || 0));+}++/**+ * The slice of an audio file a bed actually plays, given the file's real length.+ * A probe reporting LESS than the requested slice is authoritative: the offset+ * is pulled back inside the file, the length shrinks to what remains, and the+ * fades are refit to the shortened length. An unknown length (`null`, a failed+ * probe, a media element that never reported) returns the authored slice+ * untouched — fades included, since nothing shortened them.+ */+export function resolveAudioSpan(track, fileDurationSec) {+ const offsetSec = track.offsetSec || 0;+ const durationSec = track.durationSec || 0;+ const available = knownSeconds(fileDurationSec);+ if (available == null) {+ return { offsetSec, durationSec, fadeInSec: track.fadeInSec || 0, fadeOutSec: track.fadeOutSec || 0 };+ }+ const fittedOffset = Math.min(offsetSec, Math.max(0, available - MIN_MEDIA_SEC));+ const fittedDuration = Math.min(durationSec, Math.max(MIN_MEDIA_SEC, available - fittedOffset));+ return {+ offsetSec: fittedOffset,+ durationSec: fittedDuration,+ ...fitFades(track.fadeInSec, track.fadeOutSec, fittedDuration),+ };+}diff --git a/server/services/videoTimeline/local.js b/server/services/videoTimeline/local.jsindex a5cd1b771b..01c6ea4a22 100644--- a/server/services/videoTimeline/local.js+++ b/server/services/videoTimeline/local.js@@ -47,6 +47,7 @@ import {fitFades,} from './segments.js';import { AUDIO_NORM, buildAudioBedMix, fmtSec } from './audioBedMix.js';+import { historySourceDuration, resolveClipSpan, resolveStillFades, resolveAudioSpan } from '../../lib/videoTimelineSourceSpans.js';const PROJECTS_FILE = join(PATHS.data, 'video-projects.json');@@ -55,9 +56,6 @@ const PROJECTS_FILE = join(PATHS.data, 'video-projects.json');const PROBE_CONCURRENCY = 8;// A stills-only project has no clip to take geometry from.const DEFAULT_CANVAS = { width: 1280, height: 720, fps: 24 };-// Floor for a probe-clamped audio slice — atrim with start === end produces-// an empty stream that amix rejects.-const MIN_MEDIA_SEC = 0.05;// Per-project render mutex map. Keyed by projectId so two different projects// can render in parallel; same project re-render returns 409 with the@@ -300,7 +298,7 @@ export async function resolveTimeline(rawProject) {if (!assetPath) { missingAssets.push(`${seg.assetKind}/${seg.assetFile}`); continue; }prepared.push({kind: 'still', i, seg, assetPath, duration: seg.durationSec,- fades: fitFades(seg.fadeInSec, seg.fadeOutSec, seg.durationSec),+ fades: resolveStillFades(seg),});continue;}@@ -308,20 +306,18 @@ export async function resolveTimeline(rawProject) {if (!entry) { missing.push(seg.clipId); continue; }const videoPath = safeUnder(PATHS.videos, entry.filename);if (!videoPath || !existsSync(videoPath)) { missing.push(seg.clipId); continue; }- const sourceDuration = entry.numFrames && entry.fps ? entry.numFrames / entry.fps : null;- const inSec = Math.max(0, seg.inSec);- const outSec = sourceDuration != null ? Math.min(seg.outSec, sourceDuration) : seg.outSec;- if (outSec - inSec < 1 / Math.max(1, entry.fps || 24)) {+ // Shared with the editor preview (lib/videoTimelineSourceSpans.js): the+ // trim is clamped to the history entry's real length and the fades refit+ // against what is left, so a fade authored for the stored trim cannot+ // outlast a clip that has since been shortened (a negative fade start+ // renders as an all-black segment).+ const { inSec, outSec, duration, fadeInSec, fadeOutSec } = resolveClipSpan(seg, historySourceDuration(entry));+ if (duration < 1 / Math.max(1, entry.fps || 24)) {throw new ServerError(`Clip ${i} trim too short — must be ≥ 1 frame`, {status: 400, code: 'CLIP_TOO_SHORT', context: { index: i, clipId: seg.clipId },});}- // A fade authored against the stored trim can outlast a clip that the- // history entry has since shortened; rescale rather than emitting a- // negative fade start (which renders as an all-black segment).- const duration = outSec - inSec;- const fades = fitFades(seg.fadeInSec, seg.fadeOutSec, duration);- prepared.push({ kind: 'clip', i, seg, entry, videoPath, inSec, outSec, duration, fades });+ prepared.push({ kind: 'clip', i, seg, entry, videoPath, inSec, outSec, duration, fades: { fadeInSec, fadeOutSec } });}const overlays = [];@@ -358,15 +354,9 @@ export async function resolveTimeline(rawProject) {for (const tr of audioTracks) {const probed = durationByPath.get(tr.assetPath);- // A probe reporting LESS than the requested slice is authoritative.- // `null` (no ffprobe on PATH, unreadable container) must NOT collapse into- // "0 seconds available" — leave the stored slice alone in that case.- if (probed == null) continue;- tr.offsetSec = Math.min(tr.offsetSec, Math.max(0, probed - MIN_MEDIA_SEC));- tr.durationSec = Math.min(tr.durationSec, Math.max(MIN_MEDIA_SEC, probed - tr.offsetSec));- const fitted = fitFades(tr.fadeInSec, tr.fadeOutSec, tr.durationSec);- tr.fadeInSec = fitted.fadeInSec;- tr.fadeOutSec = fitted.fadeOutSec;+ // An unknown probe (`null`: no ffprobe on PATH, unreadable container) keeps+ // the stored slice — it must not collapse into "0 seconds available".+ Object.assign(tr, resolveAudioSpan(tr, probed));}const resolvedSegments = prepared.map((p) => (p.kind === 'still'diff --git a/server/services/videoTimeline/local.resolve.test.js b/server/services/videoTimeline/local.resolve.test.jsindex 8983f67f6a..4851d568af 100644--- a/server/services/videoTimeline/local.resolve.test.js+++ b/server/services/videoTimeline/local.resolve.test.js@@ -160,6 +160,9 @@ describe('resolveTimeline — fades that a later change made too long', () => {expect(segments[0].duration).toBe(1);expect(segments[0].fadeInSec + segments[0].fadeOutSec).toBeCloseTo(1);+ // The editor preview resolves through the same helper and pins these exact+ // numbers in client/src/lib/videoTimelineModel.test.js.+ expect([segments[0].fadeInSec, segments[0].fadeOutSec]).toEqual([0.5, 0.5]);});it('refits a still fade against its own hold', async () => {@@ -205,6 +208,12 @@ describe('resolveTimeline — bed probing', () => {const { audioTracks } = await resolveTimeline(bedProject({ fadeInSec: 5, fadeOutSec: 5 }));expect(audioTracks[0].fadeInSec + audioTracks[0].fadeOutSec).toBeCloseTo(2);});++ it('resolves a short file to the span the editor preview pins (2s, fades 1s/1s)', async () => {+ h.probedDuration = 2;+ const { audioTracks } = await resolveTimeline(bedProject({ durationSec: 10, fadeInSec: 4, fadeOutSec: 4 }));+ expect(audioTracks[0]).toMatchObject({ offsetSec: 0, durationSec: 2, fadeInSec: 1, fadeOutSec: 1 });+ });});describe('resolveTimeline — audio presence', () => {