Open source CapCut alternative.
TypeScript
fix(site): the picture rides its ring across a change of address
David Le377b09a1b10d
diff --git a/docs/guides/cut/README.md b/docs/guides/cut/README.mdindex ac9cbf3a..a7de6b41 100644--- a/docs/guides/cut/README.md+++ b/docs/guides/cut/README.md@@ -54,7 +54,7 @@ A play whose clip has decoded nothing at all has nothing to play, and carrying tSome machines cannot decode the footage as fast as it plays, and no reader can invent frames that the decoder has not produced. Left alone that play never recovers: the sound goes on, the picture falls further behind with every second, and a minute in it is half a minute stale. So a preview that has stayed behind its sound for a few seconds decodes smaller from then on, and the picture catches up at a lower resolution. It is a trade taken once per play and never taken back — every open decoder is abandoned when the size changes and each source pays a fresh keyframe seek, so undoing it would cost more than the pixels are worth. Two things are excluded from it. The opening seconds of a play, where every machine trails while nothing is decoded yet and files are being opened, and a reader whose walk is waiting on bytes: decoding smaller does nothing for a picture that is late because the link has not delivered it, and the preview would end up softer for the rest of the session having gained nothing. The floor is 360, below which there is no picture worth keeping in sync.-Decoders live in a bounded pool, and leaving the pool is suspension before it is closure: an idle source gives up its decoder but keeps the parsed file and the canvases its frames land on, so the next visit wakes it for the cost of one keyframe seek. The canvases are the part that must not churn — a montage of short clips replayed for a few minutes, allocating a fresh set on every reopen, floods the GPU process until frames stop arriving at all. They also must not pile up, so the warm shelf of stood-down sources is capped by the memory it holds rather than by a count of them: a small stage keeps a montage's whole cast, a large one keeps a few, and what does not fit closes for real.+Decoders live in a bounded pool, and leaving the pool is suspension before it is closure: an idle source gives up its decoder but keeps the parsed file and the canvases its frames land on, so the next visit wakes it for the cost of one keyframe seek. The canvases are the part that must not churn — a montage of short clips replayed for a few minutes, allocating a fresh set on every reopen, floods the GPU process until frames stop arriving at all. They also must not pile up, so the warm shelf of stood-down sources is capped by the memory it holds rather than by a count of them: a small stage keeps a montage's whole cast, a large one keeps a few, and what does not fit closes for real. A file that moves to a new address mid-play — an import landing in project storage, a signed link re-minted ahead of expiry — keeps its source: the frames already decoded carry the picture while a fresh reader opens on the new address behind them, and the walk goes on from wherever the playhead stands. Closing there instead would put a cold open in front of the very next frame, and the picture would freeze for exactly as long as the open takes.Two things follow from that. A cut is a read from a different ring rather than a handover between players, so crossing a join costs nothing and needs no clock to change hands; and the picture answers a moved playhead inside a frame, because the answer was already in memory. Sound is scheduled on one audio context and its clock is what playback runs on — the picture follows the sound rather than the two chasing each other. When a browser holds that context suspended, as it does until the page has been interacted with, wall time carries the cut and the audio clock is picked up once it agrees — and when the clock jumps instead of agreeing, a context resumed late or an output device switched mid-play, every scheduled window restarts where the clock now stands rather than playing out the moment it was placed for. A context that stops running mid-play is asked back every frame, because the picture would otherwise carry on against wall time over a cut gone silent.diff --git a/site/src/cut/lib/frameSource.test.ts b/site/src/cut/lib/frameSource.test.tsindex d40d1a30..3528fbfd 100644--- a/site/src/cut/lib/frameSource.test.ts+++ b/site/src/cut/lib/frameSource.test.ts@@ -1,7 +1,93 @@-import { describe, expect, test } from "bun:test";-import { FrameRing, FrameSourcePool, mappingKey, walkClaim, type Timed } from "./frameSource";+import { describe, expect, mock, test } from "bun:test";+import type { Timed } from "./frameSource";import type { MediaAsset } from "./types";+// ── the files a source reads, stood in for ──────────────────────────────────+// Only the reads are replaced; the module mock spreads the real module, so+// every other export is carried through as it is (and other test files in the+// run see the real behavior for everything but these).++interface FakeInput {+ url: string;+ disposed: boolean;+ dispose(): void;+}+const inputs: FakeInput[] = [];+/** Addresses whose open is held until the test releases or refuses it. */+const gates = new Map<string, { promise: Promise<void>; release(): void; refuse(): void }>();+const gate = (url: string) => {+ let release!: () => void;+ let refuse!: () => void;+ const promise = new Promise<void>((res, rej) => {+ release = res;+ refuse = () => rej(new Error("no track"));+ });+ const g = { promise, release, refuse };+ gates.set(url, g);+ return g;+};++const openMedia = ((src: string | Blob) => {+ const input: FakeInput = {+ url: String(src),+ disposed: false,+ dispose() {+ this.disposed = true;+ },+ };+ inputs.push(input);+ return input;+}) as never;++interface FakeTrack {+ url: string;+ codec: string;+ getDurationFromMetadata(): Promise<number>;+}+const videoTrackOf = (async (input: FakeInput) => {+ const g = gates.get(input.url);+ if (g) await g.promise;+ const track: FakeTrack = { url: input.url, codec: "avc1", getDurationFromMetadata: async () => 10 };+ return track;+}) as never;++/** Every walk a fake sink was asked for, with the address it reads. */+const sinkWalks: { url: string; from: number }[] = [];+const frame = (track: FakeTrack, timestamp: number) => ({+ canvas: { width: 640, height: 360 },+ timestamp,+ duration: 1 / 30,+});+const frameSink = ((track: FakeTrack) => ({+ async *canvases(from: number) {+ sinkWalks.push({ url: track.url, from });+ for (let i = 0; ; i++) {+ const timestamp = from + i / 30;+ if (timestamp > 10) return;+ yield frame(track, timestamp);+ }+ },+ async getCanvas(t: number) {+ return frame(track, t);+ },+ async *canvasesAtTimestamps(asks: number[]) {+ for (const t of asks) yield frame(track, t);+ },+})) as never;+const keyframeTimeAt = (async (_track: unknown, t: number) => Math.max(0, Math.floor(t))) as never;++const media = await import("./mediaRead");+mock.module("./mediaRead", () => ({ ...media, openMedia, videoTrackOf, frameSink, keyframeTimeAt }));++const { ClipFrameSource, FrameRing, FrameSourcePool, mappingKey, walkClaim } = await import(+ "./frameSource"+);++/** Let the opens, walks and drains the last call set going run out. */+const settle = async (rounds = 400) => {+ for (let i = 0; i < rounds; i++) await Promise.resolve();+};+/** A decoded frame at 30fps, as the sink would hand it over. */const f = (timestamp: number, duration = 1 / 30): Timed => ({ timestamp, duration });@@ -229,17 +315,18 @@ describe("FrameSourcePool", () => {expect(pool.size).toBe(2);});- test("a source follows its asset's URL", () => {- // A signed URL re-mints, or a shot re-renders onto its asset id. The- // mapping still names the same pictures, but they now live somewhere else,- // so the old source is closed and a fresh one reads the new URL.+ test("a source follows its asset's URL, in place", () => {+ // A signed URL re-mints, or an import lands in project storage. The+ // mapping still names the same pictures at a new address, so the source+ // moves there itself — everything it has decoded keeps answering while+ // the new address opens behind it.const pool = new FrameSourcePool(4);pool.beginFrame();const before = pool.get("k", asset("a"), 480);expect(pool.get("k", asset("a"), 480)).toBe(before);const moved = { ...asset("a"), url: "https://example.test/a.mp4?e=2&s=next" };const after = pool.get("k", moved, 480);- expect(after).not.toBe(before);+ expect(after).toBe(before);expect(after.url).toBe(moved.url);expect(pool.size).toBe(1);});@@ -330,3 +417,69 @@ describe("FrameSourcePool", () => {expect(pool.get("a", asset("a"), 480)).toBe(a);});});++describe("ClipFrameSource retarget", () => {+ const reset = () => {+ inputs.length = 0;+ sinkWalks.length = 0;+ gates.clear();+ };++ test("the picture rides its ring across a change of address", async () => {+ // The failure this is here for: a library clip lands in project storage+ // mid-play and its asset moves to the stored URL, and closing the source+ // there put a cold open — a container read over the link, an empty ring —+ // in front of the very next frame. The frames already decoded carry the+ // picture while the new address opens behind them.+ reset();+ const before = asset("ride");+ const src = new ClipFrameSource(before, 360);+ src.want(0.1, true);+ await settle();+ expect(src.frameAt(0.1)).not.toBeNull();+ const movedUrl = "https://example.test/ride-landed.mp4";+ const g = gate(movedUrl);+ src.retarget({ ...before, url: movedUrl });+ expect(src.url).toBe(movedUrl);+ await settle();+ // The new address has not opened yet: everything decoded keeps answering,+ // and the old stack keeps walking frames in.+ expect(src.frameAt(0.1)).not.toBeNull();+ src.want(0.3, true);+ await settle();+ expect(src.frameAt(0.3)).not.toBeNull();+ expect(inputs[0].disposed).toBe(false);+ g.release();+ await settle();+ // The stack swapped: the old file let go, the frames kept.+ expect(inputs[0].disposed).toBe(true);+ expect(src.frameAt(0.3)).not.toBeNull();+ src.want(0.6, true);+ await settle();+ // The walk carrying on reads the new address.+ expect(sinkWalks[sinkWalks.length - 1].url).toBe(movedUrl);+ expect(src.frameAt(0.6)).not.toBeNull();+ src.close();+ });++ test("a new address that will not open leaves the picture standing", async () => {+ reset();+ const before = asset("stay");+ const src = new ClipFrameSource(before, 360);+ src.want(0.1, true);+ await settle();+ expect(src.frameAt(0.1)).not.toBeNull();+ const movedUrl = "https://example.test/stay-gone.mp4";+ const g = gate(movedUrl);+ src.retarget({ ...before, url: movedUrl });+ await settle();+ g.refuse();+ await settle();+ // The failed open is spent on the probe alone: the frames stay, the old+ // file stays open, and the usual retries go after the new address.+ expect(src.frameAt(0.1)).not.toBeNull();+ expect(inputs[0].disposed).toBe(false);+ expect(inputs[inputs.length - 1].disposed).toBe(true);+ src.close();+ });+});diff --git a/site/src/cut/lib/frameSource.ts b/site/src/cut/lib/frameSource.tsindex ab1e6569..eea7f2c3 100644--- a/site/src/cut/lib/frameSource.ts+++ b/site/src/cut/lib/frameSource.ts@@ -425,6 +425,10 @@ export class ClipFrameSource {/** A still's single frame; stills never stream. */private still: SourceFrame | null = null;private opening: Promise<void> | null = null;+ /** Bumped each time the source is sent to a new address. An open that comes+ * back to find the count moved built its stack for an address the source+ * has since left. */+ private aim = 0;private closed = false;/** Set when the file turns out to hold no picture this browser can read. */private unreadable = false;@@ -565,7 +569,7 @@ export class ClipFrameSource {}constructor(- private readonly asset: MediaAsset,+ private asset: MediaAsset,private readonly height: number,/** Called whenever a frame lands. A paused editor draws the nearest frame* it has and stops; without a nudge, the exact frame would decode into a@@ -781,6 +785,81 @@ export class ClipFrameSource {this.sink = null;}+ /**+ * Move this source onto the asset's current address, keeping every frame it+ * holds.+ *+ * An asset's URL moves while its clips are mid-play: an import lands in+ * project storage and leaves the URL it arrived on, a signed link re-mints+ * ahead of expiry. The same pictures now live at the new address, so the+ * ring keeps answering while a fresh stack — file, track, sink — opens on+ * that address behind it, exactly the way a voice plays out its scheduled+ * sound across the same move. The old stack serves reads until the new one+ * is ready: the import queue holds the source bytes for this window, and a+ * re-minted link is still inside its signing window. Once the stack swaps,+ * the next ask starts a walk from wherever the reader stands, on canvases+ * the ring's held frames no longer live on.+ */+ retarget(asset: MediaAsset): void {+ this.asset = asset;+ if (this.closed || asset.type === "image") return;+ const aim = ++this.aim;+ // Nothing open and nothing opening: the next ask opens the new address.+ if (!this.input && !this.opening) return;+ void (async () => {+ // An open in flight built its stack for the old address; it settles+ // first, so the swap below replaces a stack that has stopped moving.+ try {+ await this.opening;+ } catch {}+ if (this.closed || this.aim !== aim) return;+ const input = openMedia(this.asset.url);+ try {+ const track = await videoTrackOf(input);+ if (this.closed || this.aim !== aim) return input.dispose();+ if (!track) {+ input.dispose();+ this.fail();+ return;+ }+ const sink = frameSink(track, { height: this.height }, { poolSize: POOL, lowLatency: true });+ // The new stack installs first, so a walk starting this instant is+ // already on it; the walk still running was on the old one and ends+ // now. Its frames stay: the old sink has stopped cycling its+ // canvases, so they hold still, and they are what answers until the+ // first walk on the new stack lands. The old file is let go once the+ // old walk's drain has let go of it.+ const oldInput = this.input;+ this.input = input;+ this.track = track;+ this.sink = sink;+ this.opening = Promise.resolve();+ this.backSink = null;+ this.back = null;+ this.stopStream();+ await this.drainRun;+ oldInput?.dispose();+ if (this.closed || this.aim !== aim) return;+ // Walk bookkeeping belongs to the stack that is gone; the new one+ // starts clean, quick retries and all.+ this.streamDone = false;+ this.shortEndAt = -1;+ this.failStreak = 0;+ this.unreadable = false;+ this.attempts = 0;+ // A paused reader is only asked again on a repaint; the nudge is what+ // sends its ask to the new stack.+ this.onFrame();+ } catch {+ input.dispose();+ // The old stack stays installed and keeps serving its ring; the+ // failure books the usual retries, and the retry's open reads the+ // current address.+ if (this.aim === aim && !this.closed) this.fail();+ }+ })();+ }+/*** An open failed. That is usually a moment — a network blip, a signed URL a* few seconds past its window — so the source tells the link keeper (which@@ -823,6 +902,7 @@ export class ClipFrameSource {input.dispose();return;}+ this.input?.dispose();this.input = input;this.track = track;// Not awaited: it reads metadata the sink is about to read anyway, and@@ -1338,13 +1418,11 @@ export class FrameSourcePool {const id = `${key}|${height}`;let src = this.sources.get(id);// The mapping names which pictures; the URL is where they are read from,- // and it moves — a signed link re-mints, a shot re-renders onto its asset.- // A source is only current while it reads the store's current URL.- if (src && src.url !== asset.url) {- src.close();- this.sources.delete(id);- src = undefined;- }+ // and it moves — an import lands in project storage, a signed link+ // re-mints. The source follows the move in place, holding every frame it+ // has decoded, so a clip mid-play rides across the swap on its ring while+ // the new address opens behind it.+ if (src && src.url !== asset.url) src.retarget(asset);if (!src) {poolLog(`pool open ${id}`);src = new ClipFrameSource(asset, height, this.onFrame);