Skip to content

Open source CapCut alternative.

TypeScript

fix(site): measure every shot's camera before calling its pulses flicker [rebuild]

David Lee3b779cf3d47
diff --git a/site/src/cut/server/ai/catalog.ts b/site/src/cut/server/ai/catalog.ts
index d67d3a57..7325e79a 100644
--- a/site/src/cut/server/ai/catalog.ts
+++ b/site/src/cut/server/ai/catalog.ts
@@ -354,7 +354,7 @@ Someone points at a video — a file they imported, a link you brought in, a cli
**Clipping a long source.** A talk, an interview or a stream is mostly material that only makes sense in place. find_highlights ranks its stretches from the transcript, so an hour is read for its handful of moments without watching all of it; watch_video the spans it returns before committing, since the ranking never saw the picture. add_clip \`spans\` lays a pick down — several stretches of one source as one run of clips, the material between them left out — and a two-span result is one clip whose halves were never next to each other.
**Watch the whole thing first.** Every decision below comes off the record, so the record has to be complete: watch from 0 and keep going until \`unwatchedSeconds\` is 0, writing each pass into note_source before the next one. Anything over a couple of minutes gets a coarse pass end to end first (\`interval_seconds\` 15-30) to learn its shape, then a dense pass (0.5) over the stretches that carry text or cut fast. Read every card word for word — its text, its face, where it sits in the frame, when it comes and goes — because a turn later the sheets are gone and the notes are all you have. Reading type is what \`detail\` is for: the dense pass over a stretch carrying words runs at \`detail\` read, and a face or treatment you are about to name runs at original, where the frame arrives at the source's own resolution. When the cards ARE the narration — a caption track of what the speaker is saying — the transcript already holds every word, and the close passes are for how they are set: face, weight, size, colour, placement, how many words ride a card, how it enters and leaves. Read a few of those at original and take the rest from the transcript.
-**Measure it.** A contact sheet tells you what happens; the numbers come from the source's pixels. A reference recorded off a screen — a phone filming an editing app, a letterboxed film — has its picture inside a bigger frame: find_picture finds that box and keeps it on the source (detect_cuts does on its first read), every measuring tool and compare_to_source then reads inside it, and their x, y, w, h are fractions of it, so they land on the cut's frame as they are; set_aspect to its \`aspect\`. detect_cuts reads every frame: each cut on its frame, the \`shots\` between them, and the \`changes\` inside a shot — a word landing, a word swapping fill for outline, a shape popping in. Run it over the whole source before the read passes, which then land a frame in every shot and change, and time each title from them. Its times are to the millisecond, and every tool lands such a time on the frame it names: write them back as read. sample_color reads the hex of any region at any second. track_element gives an element's box frame by frame and the ease of each move; with match ink it gives a title's ink box. track_camera gives a shot's camera move frame by frame — zoom, shift, turn — and says when the picture under it is one frozen frame.
+**Measure it.** A contact sheet tells you what happens; the numbers come from the source's pixels. A reference recorded off a screen — a phone filming an editing app, a letterboxed film — has its picture inside a bigger frame: find_picture finds that box and keeps it on the source (detect_cuts does on its first read), every measuring tool and compare_to_source then reads inside it, and their x, y, w, h are fractions of it, so they land on the cut's frame as they are; set_aspect to its \`aspect\`. detect_cuts reads every frame: each cut on its frame, the \`shots\` between them, and the \`changes\` inside a shot — a word landing, a word swapping fill for outline, a shape popping in. Run it over the whole source before the read passes, which then land a frame in every shot and change, and time each title from them. Its times are to the millisecond, and every tool lands such a time on the frame it names: write them back as read. sample_color reads the hex of any region at any second. track_element gives an element's box frame by frame and the ease of each move; with match ink it gives a title's ink box. track_camera gives a shot's camera move frame by frame — zoom, shift, turn — and says when the picture under it is one frozen frame. Run it over every shot before you describe how it moves: a beat zoom on a frozen frame, a shake or a slow push reads on a contact sheet as a held shot with exposure flicker, and detect_cuts lists its pulses as whole-frame \`changes\` — a run of those inside one shot is a camera move until track_camera says otherwise.
**Lay down the sound, then block out the cuts.** Cuts timed to a reference are timed to its sound: detach_audio with the reference's \`assetId\` puts its whole track on the soundtrack and none of its picture. add_clip with \`blocks\` lays the skeleton from \`shots\` in one call — one block per shot on track 0, each its shot's \`from\` and \`to\` as read so every cut lands on its frame, flash cuts included, \`color\` what sample_color reads behind it and \`label\` naming what the person drops there. The label rides the timeline chip; the frame shows the colour alone. Footage dropped on a block, or replace_item, takes its place and its length, so everything built on top keeps its timing. When the ask is the reference's own picture playing, add_clip \`spans\` lays its shots down instead, one span per shot at those same times.