Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.
TypeScript agentic-aiaiai-agentsai-videocompilerdslffmpeggenerative-aillmmarkup-languagemonorepoplugin-systemprogramming-languagetext-to-videotypescriptvideovideo-automationvideo-clonevideo-editingvideo-generation
fix(hypit): one generation carries a Segment's speech and its picture
rpone6ed43d19551e
diff --git a/.agents/skills/hypit/references/playbooks/craft/frame-coverage.md b/.agents/skills/hypit/references/playbooks/craft/frame-coverage.mdindex 5f4caf80..a97aadbf 100644--- a/.agents/skills/hypit/references/playbooks/craft/frame-coverage.md+++ b/.agents/skills/hypit/references/playbooks/craft/frame-coverage.md@@ -5,8 +5,8 @@ the same shape, and the difference is where this whole class of defect lives.Nothing is ever absent. An instant no Track covers is not blank — it is filled by whatever liesbeneath, and the viewer sees that with exactly the weight of everything that was chosen. Which wrong-thing appears is decided by what happened to be underneath: the Film's background, a bed, the shot-before. So one defect wears several faces — a black gap, a flash of the wrong picture, a panel that+thing appears is decided by what happened to be underneath: the Film's background, a layer placed+behind, the shot before. So one defect wears several faces — a black gap, a flash of the wrong picture, a panel thatblinks out and returns — and reading them as separate problems is how they get fixed one at a time,for ever.@@ -17,11 +17,12 @@ an interval inherits its edges whenever its length comes from something other th- a window derived from speech, which starts and stops with words and therefore excludes the silencearound and between them;-- a generated take, which ends where its material ends rather than where the shot should — and then- draws nothing for the rest of its window, because `playback` defaults to `once-start`. This is the- one edge on the list a machine can find before a Build: `reconstruction_check` reads each Recipe- and refuses the default on timed generated material. `generated-dependencies.md` says what to set- instead;+- an insert, whose material ends where the generator stopped rather than where its window does — and+ then draws nothing for the rest of it, because `playback` defaults to `once-start`, so the base+ returns mid-phrase. This is the one edge on the list a machine can find before a Build:+ `reconstruction_check` reads each Recipe and refuses the default on generated material.+ `generated-dependencies.md` says what to set instead. The Segment's own picture has no such edge —+ the take that speaks it is the take that draws it, so the two lengths are one number;- a blend, whose frames are edges of partial coverage — a one-frame fade is one frame on which thelayer beneath is half visible, and naming the Recipe for a cut does not make it one;- a schedule inside a component, which draws only where its own Program says to and stops between.@@ -30,28 +31,50 @@ The list is not the point and will never be complete. The question is, and it isevery time: **for each edge, did I choose this instant, and what becomes visible on the other side ofit?** Anything derived answers "no" to the first half.-## A layer underneath is a safety net, not a repair+## Fix the interval; a layer underneath is not a repair-Putting something beneath a hole changes which wrong picture is shown; it does not close the hole.-That is worth doing — a held still under a voiceover beats black — but it makes the defect quieter-rather than absent, and quieter is harder to find. Fix the interval. Keep the bed for the frames no-interval should have to claim.+Putting something beneath a hole changes which wrong picture is shown. It does not close the hole,+and it makes the defect quieter rather than absent, which is harder to find. The interval is what has+to change: the window that ends early is lengthened, the material that runs out is replaced with+material that lasts, the stretch nothing claims is claimed by whatever the reference shows there.-This matters most right after a fix: closing a black gap by bedding a layer under it converts a-symptom everyone can see into one that reads as an edit, and the reconstruction looks finished while-the same edges are still wrong.+**A stretch is moving footage unless the shot's own observation says its background is static.** That+is the default, and it is not a judgement to make from the Source or to measure off the reference:+the third visual question already asks whether the picture moves, and a held photograph or card that+only appears and disappears is still however long it is on screen. Read the answer for that shot. No+answer means moving.-The placeholders the comparison loop renders under an element are not beds and cannot become one.-They exist in that render and nowhere else — the Source never names them, and the checks that decide+Nothing that came out of an ffmpeg filter can stand in for that answer. A filter reads the whole+composited frame, so a reference whose background is frozen under a moving overlay measures as+moving, and one measured as moving proves nothing about the background — which is the only thing the+question is about.++So a still spanning the program is never right: it would need every shot's observation to say the+background is static. Neither is one held under a voiceover because the take beneath it was too+short — that stretch has a length, and the take is what has to reach it.++The picture that goes wrong this way reads as an edit rather than as a fault, and the reconstruction+looks finished while the same edges are still wrong. Nothing downstream catches it either: a still+has no timeline, so it takes no `playback` and the pre-Build check does not apply to it, and Gate 4+measures black, which a still removes.++The placeholders the comparison loop renders under an element never reach the Source. They exist in+that render and nowhere else — the Source never names them, and the checks that decidecoverage read the Source's Recipes and bindings rather than any picture. A mock cannot quiet a holeit is structurally unable to reach.## Measure it; the eye is the wrong instrument-An instant is a thirtieth of a second. Watching finds a scene that is wrong and slides straight past-a frame that is, and the shorter the hole the more it reads as intentional cutting. Both measurements-are in `production-gates.md` Gate 4: one for holes over nothing, one for holes over something. Run-both, because passing the first is exactly what bedding a layer buys.+Two of these are settled before a Build, from the Source alone, and neither needs a threshold.+Whether every word carries a picture is a yes or no per word: `reconstruction_check` reads which+full-frame elements bind which Selections and names the words nothing claims. Whether the material+lasts its window is the `playback` reading above.++What is left for the delivery is what only the delivery shows. An instant is a thirtieth of a second:+watching finds a scene that is wrong and slides straight past a frame that is, and the shorter the+hole the more it reads as intentional cutting. Both measurements are in `production-gates.md` Gate 4,+one for holes over nothing and one for holes over something. Run both — a hole with something under+it passes the first.Anything the measurements report is an inherited edge. Take it back to the interval that produced itrather than to the layer that revealed it.diff --git a/.agents/skills/hypit/references/playbooks/craft/generated-dependencies.md b/.agents/skills/hypit/references/playbooks/craft/generated-dependencies.mdindex d15554a9..a9fa0af7 100644--- a/.agents/skills/hypit/references/playbooks/craft/generated-dependencies.md+++ b/.agents/skills/hypit/references/playbooks/craft/generated-dependencies.md@@ -26,9 +26,12 @@ only about the video: what it should say, who is in it, what it is for.Images therefore generate in rounds: the establishing image is accepted first, and the views derivedfrom it are generated after. `production-gates.md` Gate 1 is one stage per round, not one stage.-## A split shot's parts all start from the same image+## A split picture's parts all start from the same image-A continuous shot that exceeds the generator's duration ceiling is split into as many takes as it+This is about a generation that carries no speech — an insert, a cutaway, a stretch of B-roll. A take+that speaks its Segment is one generation by construction, and the Segment is cut to fit it.++A continuous picture that exceeds the generator's duration ceiling is split into as many takes as itneeds. **Every part references the same first-frame image** — the one accepted for that shot — and nopart references another part.@@ -53,19 +56,41 @@ re-establish the room across that seam is how continuity is lost for nothing.Consecutive Segments carrying one unbroken voiceover are one Segment with Selections inside it. Thepictures over that stretch are placed by those Selections — `frame-coverage.md` governs what that-costs and what has to cover the silence between them — and the speech stays one Take.+costs and what has to cover the silence between them — and the speech stays one Take, up to the+length one generation produces. Past that it is two Segments; the rule below says so and+`../../reconstruction/final-sources.md` says where the seam goes.Name Segments for what they hold. Names are handles: the Script's meaning lives in the words, and nopart of the pipeline reads a Segment name back for anything a reader would see.-## A voice is generated once+## A voice is generated once, and the take that draws a Segment is the take that speaks it-- One accepted voice sample per person, however it was chosen.-- That one sample feeds both the speech generated for every line they say and the voice reference of- every take they appear in.+- One accepted voice sample per person, from `mimo:VoiceDesign`. That is what the voice generator is+ for: a sample, once.+- **The sample is a reference on every take, and the take generates the speech.** Write+ `generate-audio="true"` and hand it the sample as `<seedance:Reference audio={…}/>`. The+ `whisperx:SemanticTake` for that Segment aligns that take's own media, and the full-frame picture+ over that Segment comes from the same generation.- Do not select a voice separately per take. Two selections of "the same warm mid-range voice" are twovoices, and a viewer hears it immediately.+One generation carries both, and that is what makes the Segment's length and its picture's length the+same number. Generating the speech separately makes them two numbers arrived at independently — the+Segment as long as whatever a voice generator produced, the picture as long as whatever a video+generator was asked for — and two numbers arrived at independently do not agree. Where they disagree,+the picture ends first and nothing is under it.++### A Segment fits one generation++`whisperx:SemanticTake` selects "the single authored Segment performed by this Take", so a Segment has+one Take and a Take has one Segment. The speech coming from that Take means the Segment can be no+longer than one generation. Read the selected model's declared duration range from the model.++Cut the Script so every Segment's speech fits. A passage longer than that is two Segments, and+`../../reconstruction/final-sources.md` decides where the seam falls — on a sentence, not mid-clause.++This is why the split above is about pictures rather than speech.+## A take starts from an accepted imageNever generate video from a prompt alone. Without a first frame nothing fixes identity, framing,@@ -99,14 +124,29 @@ form and the measurements. `during={story.selection.X}` spans that Selection's fso the pause between two Selections, the breath before a Segment's first word, and the words that fallbetween two occurrences are in none of them.-Decide, for each thing you place, which of two kinds it is:+Decide, for each thing you place, which of three kinds it is:-- **Continuously present** — a sheet the reference holds up, a bed under a voiceover, a badge that- stays. Take its window from a Segment, or from the first occurrence's start to the last one's end.+- **Continuously present** — a sheet the reference holds up, a panel that stays behind a whole+ passage, a badge that stays. Take its window from a Segment, or from the first occurrence's start to the last one's end.Never from a Selection whose occurrences have gaps, however well the occurrences line up with thewords: they do not touch.-- **Genuinely coming and going** — an insert that appears for one phrase and leaves. A Selection is- exactly right, and the gap is the point.+- **Handing over to the next one** — one insert replaced by another with nothing meant to show+ between them. Close the first on the second's first word, or open the second on the first's last:+ the Script's markers carry which side of a word a boundary lands on.++ | Marker | Where the boundary lands |+ |---|---|+ | `@id` | Open, right-absorbing — at the next word's start |+ | `~@id` | Open, left-absorbing — at the previous word's end |+ | `@/id` | Close, left-absorbing — at the previous word's end |+ | `@/id~` | Close, right-absorbing — at the next word's start |++ A plain `@a … @/a @b … @/b` closes on one word's end and opens on another word's start, and the+ pause between those two words is in neither Selection. Written `@/a~` or `~@b` the two windows meet,+ and what is under them stays under them. `docs/quickstart/script.md` is authoritative for the+ markers.+- **Genuinely coming and going** — an insert that appears for one phrase and leaves. A plain Selection+ is exactly right, and the gap is the point.The same question decides a component you write yourself. A Program scheduled from occurrences drawsonly inside them unless you give it a span of its own, so a page built from one occurrence per row@@ -116,42 +156,28 @@ a component both persists and changes, its schedule carries two different thingslong it is on screen, one window per item for when that item arrives — and conflating them is whatproduces the blink.-## An audio Take brings no picture, so its Segment is covered or it plays black--An audio-only Take — media normalized with `video="none"`, aligned by `whisperx:SemanticTake`, and-assembled by `speech:Take source={…}` — creates program time and speech and contributes no visual at-all. For as-long as it runs the picture is whatever the peer Tracks put there, and wherever they put nothing the-Film's own background shows through. A voiceover Segment is therefore an obligation: every frame of-it belongs to some Item, and the frames nobody claimed are black in the delivery.--Two different holes open, and both look identical on screen:--- **A stretch inside no Selection.** Mark the ranges the B-roll covers and one sentence between two- of them belongs to neither, so nothing draws it. The Selections have to *tile* the Segment — each- one picking up where the last left off — rather than merely landing in the right places. A line- that introduces what comes next usually belongs to the Selection it introduces.-- **A take shorter than the window it fills.** An Item whose window outlasts its own material runs- out partway, and what happens for the remainder is decided by the `playback` key on its appearance- Recipe. The default is `once-start`, which draws the material once and then draws nothing at all —- no sampling segment is emitted for those frames, so they fall through to whatever is beneath, and- beneath a full-frame base that is the Film background. `hold-start` pins the last frame for the- rest of the window, `loop-start` repeats, `stretch` retimes to fit.-- Read the model's duration ceiling too: Seedance `mini` stops at 15 seconds, so a longer stretch- needs more than one Item rather than one Item asked for a length the model refuses.--Set `playback` on every timed picture whose window comes from speech, and treat the material's length-as unknown. A generation is ordered in whole seconds, the estimate that sizes the order predicts the-words rather than measuring them, and the window is decided by the audio that was actually produced —-three separate numbers that will not agree. Lengthening the material moves the mismatch without-removing it; `playback` decides what the window shows once the material is spent, which is the part-that stays true whichever way the three numbers fall.--`reconstruction_check` reads this before a Build and refuses a Recipe that leaves `playback` at its+## An insert runs out where its material ends, and the base shows through++An insert is a second generation over a Segment the speaking take already fills, so its length and+its window are two numbers arrived at separately. Where the material is the shorter of the two, the+`playback` key on its appearance Recipe decides what the rest of the window shows. The default is+`once-start`: the material draws once and then nothing is drawn, so those frames fall through to+whatever is beneath. `hold-start` pins the last frame for the rest of the window, `loop-start`+repeats, `stretch` retimes to fit.++What appears is the base — the insert leaves early and the shot beneath returns mid-phrase. Read it+as an insert with the wrong length rather than as an edit, and give the Recipe a `playback` that says+what should happen there. `reconstruction_check` reads the Recipe before a Build and refuses thedefault on generated material.-A bed makes a blend visible, so check the Items' entry and exit while you are here: `enter` and `exit`+Read the selected model's duration ceiling too, from the model: a longer insert is more than one Item+rather than one Item asked for a length the model refuses.++The other way an insert leaves a stretch it should have held is the boundary above: a plain Selection+close lands on a word's end, so the pause before the next one belongs to neither. That one is fixed+in the Script rather than the Recipe.++A layer beneath makes a blend visible, so check the Items' entry and exit while you are here: `enter` and `exit`default to `none`, and a Recipe named for a cut that fades for a frame is one of the inherited edges`frame-coverage.md` describes.@@ -162,6 +188,9 @@ default to `none`, and a Recipe named for a cut that fades for a frame is one ofAn authored take stays inside the selected model's declared duration range — read the range from themodel, since it differs between them and a literal in a document goes stale.+This too is about a generation that carries no speech. A Segment always has its own speaking take,+however short its words are.+An observed stretch shorter than that floor is never authored as its own take. Decide by what theobservation says about the picture:diff --git a/.agents/skills/hypit/references/playbooks/craft/persona-and-audio.md b/.agents/skills/hypit/references/playbooks/craft/persona-and-audio.mdindex 6a06a9c6..0ecb54fd 100644--- a/.agents/skills/hypit/references/playbooks/craft/persona-and-audio.md+++ b/.agents/skills/hypit/references/playbooks/craft/persona-and-audio.md@@ -22,31 +22,31 @@ Connect only the references a component actually needs.- Use `gpt:Image` with ordered `gpt:Reference` children when a new character/scene image must preservesupplied identity or product facts. Pass every generated image through `production-gates.md`.-- Use `mimo:Preset` for a selected built-in voice, `mimo:VoiceDesign` for an English voice description,- or `mimo:VoiceClone` with one explicit `sample` Artifact.-- Feed exact Script speech text through each MiMo component's `speech={story.segment.NAME.speech}`- edge. Voice instructions affect delivery; they do not replace or rewrite Script words.+- Use `mimo:Preset` for a selected built-in voice or `mimo:VoiceDesign` for an English voice+ description. Either one produces the sample, once per person.+- That sample is a reference on every take the person appears in, and the take speaks the Segment.+ `generated-dependencies.md` holds the rule and what it costs: one generation carries the speech and+ the picture, so the Segment's length and its picture's length are one number.- For Seedance speaking formats, render the matching Kit and connect image/audio references to`seedance:ReferenceVideo` in the exact order defined by `seedance-directing.md`.- Do not add a face or voice reference to a shot that does not need that identity. Extra referencesincrease ambiguity and can contaminate the intended subject.-## Plan voice before picture duration+## The take's own length is the Segment's length-- When TTS is the sole narration source, choose the voice and generate or measure the real speech- before locking visual cut lengths.-- Use `estimate:Speech` only for deterministic pre-generation planning. Final caption and semantic+- The speaking take generates the speech, so a Segment is exactly as long as the take that speaks it.+ There is no second duration to reconcile.+- Use `estimate:Speech` only to size a generation before it is ordered. Final caption and semantictiming comes from each accepted normalized Take through `whisperx:SemanticTake`.-- One continuous voiceover may cover several independently selected visual shots. Do not split TTS- merely because the picture cuts.-- For dialogue generated inside speaking Seedance takes, preserve Script Segment order when assembling- `speech:Take` children.+- One continuous voiceover may cover several independently selected visual inserts. Do not cut the+ Script merely because the picture cuts — the inserts are Selections inside the Segment.+- Preserve Script Segment order when assembling `speech:Take` children.## Choose one path for speech audio-- Normalize authoritative speech-bearing generated or supplied audio, create its- `whisperx:SemanticTake`, then put that value in `speech:Track` as a `speech:Take`. An audio-only- Semantic Take contributes timing and speech audio without inventing a visual clip.+- Normalize the speaking take, create its `whisperx:SemanticTake`, then put that value in+ `speech:Track` as a `speech:Take`. The take carries the speech and the picture together, so that one+ value is both the Segment's timing and the picture over it.- Do not place the same narration on both Speech Track and an Audio Track. That duplicates speech inthe final mix and creates competing time authority.diff --git a/.agents/skills/hypit/references/playbooks/craft/production-gates.md b/.agents/skills/hypit/references/playbooks/craft/production-gates.mdindex 6e035c56..d2f1bdbf 100644--- a/.agents/skills/hypit/references/playbooks/craft/production-gates.md+++ b/.agents/skills/hypit/references/playbooks/craft/production-gates.md@@ -106,8 +106,9 @@ visual QA.## Gate 3: measure and author Tracks-- Normalize every accepted speech-bearing video or audio Take, create its Segment-local- `whisperx:SemanticTake`, then assemble those Takes with `speech:Track`.+- Normalize every accepted speaking take, create its Segment-local `whisperx:SemanticTake`, then+ assemble those Takes with `speech:Track`. That take carries the Segment's picture as well as its+ speech, so one accepted generation settles both.- Recompute the affected SemanticTake whenever its speech audio changes. Every Track timed against it — Caption,Media, Typography, Ranking, Deck, Comment, Screen, Audio — is measured against audio that no longerexists otherwise, and the drift is invisible in a still frame. A Style-only change does not prove a@@ -129,11 +130,14 @@ visual QA.- Inspect the durable Build and retrieve the final Artifact with `get`.- Watch and listen to the complete delivery, including the first and last second. Verify dialogue,captions, overlays, transitions, audio density, claims, and CTA as one program.-- **Measure the delivery's own speech against the Script.** An agent reads pictures and cannot hear,- so "listen and verify dialogue" above resolves to nothing on its own, and a take whose spoken words- came from somewhere other than the Script passes `check`, `plan`, `preview_check` and both- measurements below without a mark. Transcribe what was delivered and read it against the words the- Script holds:+- **Measure the delivery's own speech against the Script. This is the gate the whole program's words+ rest on.** The take that speaks a Segment generates that speech from its prompt, so what is said is+ decided by a video generator rather than handed to a synthesizer as text — it can drop a clause,+ reorder one, or say something adjacent. Nothing upstream reads it back: an agent reads pictures and+ cannot hear, so "listen and verify dialogue" above resolves to nothing on its own, and a take whose+ spoken words came from somewhere other than the Script passes `check`, `plan`, `preview_check` and+ both measurements below without a mark. Transcribe what was delivered and read it against the words+ the Script holds:```bashhypit-reference-video-tools prepare_reference --video-path output/final.mp4 --observer agentdiff --git a/.agents/skills/hypit/references/playbooks/index.md b/.agents/skills/hypit/references/playbooks/index.mdindex 0a6192c8..b474987c 100644--- a/.agents/skills/hypit/references/playbooks/index.md+++ b/.agents/skills/hypit/references/playbooks/index.md@@ -66,7 +66,7 @@ file that some jobs never open.- `craft/graphic-compositions.md` — what counts as a base picture, full-screen graphic compositions,and where missing material comes from.- `craft/frame-coverage.md` — what is on screen at every instant, and the edges nobody chose.-- `craft/generated-dependencies.md` — what one generation owes another: the location, the split shot,+- `craft/generated-dependencies.md` — what one generation owes another: the location, the split picture,the voice, the first frame, and the take too short to generate.- `craft/image-prompt-style.md` — English reference-image prompts and camera geometry.- `craft/seedance-directing.md` — Kit selection, Recipe axes, references, and motion direction.diff --git a/.agents/skills/hypit/references/quickstart.md b/.agents/skills/hypit/references/quickstart.mdindex f89e10f6..59ff376e 100644--- a/.agents/skills/hypit/references/quickstart.md+++ b/.agents/skills/hypit/references/quickstart.md@@ -23,8 +23,8 @@ Use explicit imports, exact fonts, Canvas/Frames, the SemanticTrack, Targets, anauthority. There is no implicit cache; reuse is explicit with `build-record` + `satisfy`.`speech:Take source={...}` assembles one aligned `whisperx:SemanticTake` into the Track, in program-order. A take whose media normalizes with `video="none"` creates program time and speech audio while-peer Media Tracks provide the visuals.+order. The take that speaks a Segment is the take that draws it, so that one value carries the+program time, the speech and the picture together; peer Media Tracks put inserts over it.For a program with no spoken words, keep the `whisperx:SemanticTake` and `speech:Track` declarations —the SemanticTrack is the frame domain every `start`/`end` window resolves into — satisfydiff --git a/.agents/skills/hypit/references/reconstruction/final-sources.md b/.agents/skills/hypit/references/reconstruction/final-sources.mdindex af91be15..63939e9a 100644--- a/.agents/skills/hypit/references/reconstruction/final-sources.md+++ b/.agents/skills/hypit/references/reconstruction/final-sources.md@@ -27,6 +27,11 @@ consecutive Segments carrying one unbroken voiceover are one Segment with Select`../playbooks/craft/generated-dependencies.md` says why — a Segment is a generation boundary, andsplitting a stretch the reference delivers unbroken invents a seam it does not have.+One thing does force a seam. The take that speaks a Segment generates that speech, so a Segment can+be no longer than one generation. Where an unbroken passage runs past what the model will produce in+one take, it becomes two Segments — and then the seam is yours to place rather than the ceiling's.+Put it where the speaker would draw breath: at a sentence end, never mid-clause.+## A take's duration is estimated, not read off the referenceEvery take is generated, and the generated speech plays at its own pace, not the reference's. Sodiff --git a/.agents/skills/hypit/references/reconstruction/index.md b/.agents/skills/hypit/references/reconstruction/index.mdindex 3d1d754e..e7b0ab36 100644--- a/.agents/skills/hypit/references/reconstruction/index.md+++ b/.agents/skills/hypit/references/reconstruction/index.md@@ -154,7 +154,7 @@ Before acting on the evidence, read these files completely in order:- `../playbooks/craft/graphic-compositions.md` — what may be a base picture, when a full screen isone authored composition, and where a missing picture comes from.- `../playbooks/craft/generated-dependencies.md` — what one generation owes another: the location,- the split shot, the voice, the first frame, and the stretch too short to be a take.+ the split picture, the voice, the first frame, and the stretch too short to be a take.- `../playbooks/craft/frame-coverage.md` — what is on screen at every instant, and the edgesnobody chose.- `../playbooks/craft/visual-continuity.md` — recurring anchors as explicit artifacts, onediff --git a/.agents/skills/hypit/references/reconstruction/observers.md b/.agents/skills/hypit/references/reconstruction/observers.mdindex b06abcca..62903ac6 100644--- a/.agents/skills/hypit/references/reconstruction/observers.md+++ b/.agents/skills/hypit/references/reconstruction/observers.md@@ -147,16 +147,19 @@ has for free:Answer the tasks in order yourself when the harness has no subagents. It is slower, and it is thefallback rather than the shape to aim for.-The comparison loop arrives here in batches. One element is compared against every shot the reference-shows it in before any repair, so a single element produces several comparisons at once, and each is-self-contained in the same way an observation task is. Dispatch one subagent per comparison and run-them together. Where there are no subagents, answer them one after another rather than folding them-into a single look: what the round is for is the set of differences across shots, and a shot answered-in the light of the previous shot's answer stops being independent evidence of anything.--A clip comparison reaches this observer as two frame tiles — the reference shot's own, and one built-from the render against that shot's duration. Read them as a pair of grids sampling the same stretch-at the same rate.+The comparison loop arrives here in batches. One element is compared against every stretch it is drawn+over before any repair, and `reconstruction-loop.md` has the renders all made before the first+comparison is sent, so they arrive together. Each is self-contained in the same way an observation+task is: no comparison's question depends on another's answer.++Dispatch one subagent per comparison and run them together. Where there are no subagents, answer them+one after another rather than folding them into a single look — what the round is for is the set of+differences across stretches, and one answered in the light of the previous answer stops being+independent evidence of anything.++A clip comparison reaches this observer as two frame tiles — the reference's own cut, and one built+from the render against the same duration. Read them as a pair of grids sampling the same stretch at+the same rate.## Comparing without subagentsdiff --git a/.agents/skills/hypit/references/reconstruction/reconstruction-loop.md b/.agents/skills/hypit/references/reconstruction/reconstruction-loop.mdindex 1c2b891a..da56a11f 100644--- a/.agents/skills/hypit/references/reconstruction/reconstruction-loop.md+++ b/.agents/skills/hypit/references/reconstruction/reconstruction-loop.md@@ -125,6 +125,15 @@ coverage round, and it comes before any repair — collect the whole differencerepair against it. Doing it the other way makes the comparison after a repair look like a thirdattempt at a stretch that was never examined.+**Render every stretch first, then send them all at once.** No comparison's question depends on+another's answer, so nothing is gained by waiting: rendering one and comparing it before rendering the+next makes the round as long as the sum of its parts. Render the whole set, then issue the+comparisons together — concurrent requests on the `gemini` observer, one subagent each on the `agent`+observer, and one after another only where the harness has no subagents.++That is a change in how long the round takes, not in what it costs. The ceilings below count repairs,+and a round is one look however many stretches it covers.+`--image` is available for one case only: the reference's own `visual:` observation states in wordsthat the element is completely still. The judgement comes from the reference's observation, neverfrom looking at your own render and concluding it does not move — deciding that from the@@ -282,8 +291,8 @@ waits for in `../playbooks/craft/production-gates.md`:measured at Gate 3, against the real SemanticTrack;- **whether the picture is continuous** — the frames nobody authored that `frame-coverage.md` isabout, at Gate 4, which measures them on the delivery. One inherited edge is settled before then:- `reconstruction_check` reads each Recipe's `playback` and refuses a timed picture configured to- stop when its material runs out, which is the edge a generated take produces every time. The rest+ `reconstruction_check` reads each Recipe's `playback` and refuses an insert configured to stop+ when its material runs out. The restof them — a window that starts late, a blend whose frames are half-covered, a schedule inside acomponent that stops between activations — are measured on the delivery;- **whether the delivery says the Script's words** — measured at Gate 4 by transcribing the finisheddiff --git a/.agents/skills/hypit/references/reconstruction/workflow.md b/.agents/skills/hypit/references/reconstruction/workflow.mdindex d64bcbca..b15c385a 100644--- a/.agents/skills/hypit/references/reconstruction/workflow.md+++ b/.agents/skills/hypit/references/reconstruction/workflow.md@@ -41,10 +41,10 @@ list_svml_packages→ run preview_check (final-sources.md) and repair until the graph traces —this has no attempt ceiling; a target Studio cannot trace is not done.Waiting on unrun Providers is a pass, not a failure-→ read reconstruction-loop.md, then for each authored element: render it with- render_element, which reads the sources and mocks the layers a Build has- not made, and compare it against every shot the reference shows it in before- repairing anything+→ read reconstruction-loop.md, then for each authored element: render every+ stretch it is drawn over with render_element, which reads the sources and+ mocks the layers a Build has not made, then send all those comparisons at+ once, before repairing anything→ repair against the differences that round returned, within the ceilings→ run reconstruction_check (index.md) and keep going until it passes; it namesevery drawing element that has never been compared, and every timeddiff --git a/packages/reference-video-tools/src/authoring.ts b/packages/reference-video-tools/src/authoring.tsindex 5a7c14f6..8b250666 100644--- a/packages/reference-video-tools/src/authoring.ts+++ b/packages/reference-video-tools/src/authoring.ts@@ -131,7 +131,7 @@ export function invokedFrom(): string {}/** The Script body, which `parseScript` takes on its own. */-function scriptBody(svml: string): { readonly text: string; readonly offset: number } {+export function scriptBody(svml: string): { readonly text: string; readonly offset: number } {const open = /<script\b[^>]*>/u.exec(svml);if (open === null) throw new Error("the Source declares no <script>");const start = open.index + open[0].length;diff --git a/packages/reference-video-tools/src/checks.ts b/packages/reference-video-tools/src/checks.tsindex d3121e7c..e440775f 100644--- a/packages/reference-video-tools/src/checks.ts+++ b/packages/reference-video-tools/src/checks.ts@@ -4,6 +4,7 @@ import { dirname, join, resolve } from "node:path";import { markupSurfaceHostFacetAbi } from "@hypit/markup";import type { RegisteredSurface } from "@hypit/markup";import { loadNodePackageSelection } from "@hypit/package-loader-node";+import { parseScript } from "@hypit/script";import { openStudioArchive } from "@hypit/studio/src/archive.js";import { loadStudioDomain } from "@hypit/studio/src/domain.js";import { loadStudioRun } from "@hypit/studio/src/run.js";@@ -11,7 +12,7 @@ import { readStudioSession } from "@hypit/studio/src/session.js";import type { StudioSession } from "@hypit/studio/src/session.js";import { inspectStudioRun } from "@hypit/studio/src/studio-preflight.js";-import { invokedFrom } from "./authoring.js";+import { invokedFrom, scriptBody } from "./authoring.js";import { assert } from "./media.js";export type PreviewCheckInput = {@@ -150,10 +151,25 @@ type PlaybackReport = {readonly playback: string;};+/** One run of consecutive Script words that nothing filling the frame is drawn over. */+type CoverageGap = {+ readonly segment: string;+ /** The words themselves, so a reader can find the stretch in the Script. */+ readonly words: string;+ readonly word_count: number;+};++/** How many words the Script has, and which runs of them carry no full-frame picture. */+type CoverageReport = {+ readonly words: number;+ readonly gaps: readonly CoverageGap[];+};+/*** Report what the route can still settle after the Source is written and before a Build runs: which- * reconstructed elements have never been compared against the reference, and which timed pictures are- * configured to stop before their window ends.+ * reconstructed elements have never been compared against the reference, which words of the Script+ * nothing draws a full-frame picture over, and which timed pictures are configured to stop before+ * their window ends.** `preview_check` proves the graph is wired. It proves nothing about whether what the graph draws* resembles the reference video, and a Source can pass every structural check while a component@@ -183,6 +199,12 @@ type PlaybackReport = {* comparison is a comparison. What it says is which elements have had their behaviour over their* window looked at and which have only had their layout looked at.*+ * Frame coverage is decided from the Source alone and is required: a word is either drawn over by+ * something bound to the whole picture or it is not. What covers is read from an element's own+ * opening tag — a full-bleed `frame=`, the `canvas=`, or a speech Track's full-frame `visual-frame=`+ * over the Segments whose Takes carry a picture — so a Track never inherits its children's `during=`+ * bindings.+ ** A project that places no drawing element passes with nothing to require.*/export async function reconstructionCheck(@@ -211,6 +233,7 @@ export async function reconstructionCheck(summary: ["the Source imports no package; nothing to compare."],elements: [],playback: [],+ uncovered: [],};}@@ -241,6 +264,16 @@ export async function reconstructionCheck(}}+ // Which Normalize ids carry a picture. A take normalized with `video="none"` is a voice: it has no+ // window to fill, and nothing it feeds puts anything on the Canvas.+ const moving = new Set<string>();+ for (const match of svml.matchAll(/<pipeline:Normalize\b([^>]*?)\/?>/gsu)) {+ const attributes = match[1] ?? "";+ const id = /\bid="([^"]+)"/u.exec(attributes)?.[1];+ const video = /\bvideo="([^"]+)"/u.exec(attributes)?.[1];+ if (id !== undefined && video !== undefined && video !== "none") moving.add(id);+ }+// ---- Timed pictures that stop before their window ends ----//// A generated take is asked for a whole number of seconds, and the window it has to fill comes from@@ -271,16 +304,6 @@ export async function reconstructionCheck(sheets.set(alias, recipes);}- // Which Normalize ids carry a picture. A take normalized with `video="none"` is a voice and has no- // window to fill.- const moving = new Set<string>();- for (const match of svml.matchAll(/<pipeline:Normalize\b([^>]*?)\/?>/gsu)) {- const attributes = match[1] ?? "";- const id = /\bid="([^"]+)"/u.exec(attributes)?.[1];- const video = /\bvideo="([^"]+)"/u.exec(attributes)?.[1];- if (id !== undefined && video !== undefined && video !== "none") moving.add(id);- }-// Which aliases belong to a package that draws. Plenty of elements consume a normalized media// without placing it — `whisperx:SemanticTake` reads one to align speech against it and puts no// picture on the Canvas, so it has no window to fill and no occupancy to choose. Requiring the@@ -313,6 +336,100 @@ export async function reconstructionCheck(return running;};+ // ---- Words nothing draws a full-frame picture over ----+ //+ // A delivery is a picture at every frame; a Source is a list of placements. A word no full-frame+ // element is drawn over is an instant the Source hands to whatever lies beneath it, and beneath the+ // last placement is the Film background, which is black. That is `frame-coverage.md`'s rule in the+ // one form the Source settles on its own, and it needs no threshold: it is a yes or no per word.+ //+ // What covers is read from each element's own opening tag and never from what is written inside it.+ // A Track holding forty Lines with `during=` bindings draws nothing itself; crediting it with its+ // children's bindings would report full coverage for a Source that places text over nothing.+ const coverageReport = (): CoverageReport => {+ const body = scriptBody(svml);+ const parsed = parseScript(svmlPath, body.text, body.offset);++ // The Frames that fill the Canvas, and the Canvas itself. A Frame with any other extent is an+ // insert: it draws over its own part of the picture and leaves the rest as it was.+ const fullFrames = new Set<string>();+ for (const match of svml.matchAll(/<space:Frame\b([^>]*?)\/?>/gsu)) {+ const attributes = match[1] ?? "";+ const id = /\bid="([^"]+)"/u.exec(attributes)?.[1];+ const edge = (name: string): string | undefined => new RegExp(`\\b${name}="([^"]+)"`, "u").exec(attributes)?.[1];+ if (id !== undefined && edge("left") === "0%" && edge("top") === "0%" && edge("right") === "100%" && edge("bottom") === "100%") {+ fullFrames.add(id);+ }+ }+ const canvases = new Set([...svml.matchAll(/<space:Canvas\b[^>]*?\bid="([^"]+)"/gu)].map((match) => match[1] ?? ""));++ const covered = new Array<boolean>(parsed.tokens.length).fill(false);+ const coverSegment = (id: string): void => {+ const segment = parsed.segments.find((item) => item.id === id);+ if (segment === undefined) return;+ for (let index = segment.tokenStart; index < segment.tokenEndExclusive; index += 1) covered[index] = true;+ };+ const coverSelection = (id: string): void => {+ const selection = parsed.selections.find((item) => item.id === id);+ if (selection === undefined) return;+ for (const occurrence of selection.occurrences) {+ for (let index = occurrence.open.boundary.tokenIndex; index < occurrence.close.boundary.tokenIndex; index += 1) covered[index] = true;+ }+ };++ // Every element bound to the whole picture, and the words its own `during=` claims. `program` is+ // the whole Script; a Segment or a Selection is the words it marks.+ for (const match of svml.matchAll(/<[a-z][a-z0-9-]*:[A-Za-z][A-Za-z0-9]*\b([^>]*?)\/?>/gsu)) {+ const attributes = match[1] ?? "";+ const frame = /\bframe=\{([A-Za-z0-9_-]+)\}/u.exec(attributes)?.[1];+ const canvas = /\bcanvas=\{([A-Za-z0-9_-]+)\}/u.exec(attributes)?.[1];+ if (!(frame !== undefined && fullFrames.has(frame)) && !(canvas !== undefined && canvases.has(canvas))) continue;+ if (/\bduring="program"/u.test(attributes)) covered.fill(true);+ const bound = /\bduring=\{story\.(segment|selection)\.([A-Za-z0-9_-]+)\}/u.exec(attributes);+ if (bound?.[1] === "segment") coverSegment(bound[2] ?? "");+ if (bound?.[1] === "selection") coverSelection(bound[2] ?? "");+ }++ // A speech Track draws each Take it assembles into its own `visual-frame`, so a full-frame one+ // covers the Segments whose Takes carry a picture. A Take fed by a Normalize with `video="none"`+ // is a voice and puts nothing there, so its Segment is spoken over whatever is already on screen.+ const takes = new Map<string, { readonly segment: string; readonly picture: boolean }>();+ for (const match of svml.matchAll(/<whisperx:SemanticTake\b([^>]*?)\/?>/gsu)) {+ const attributes = match[1] ?? "";+ const id = /\bid="([^"]+)"/u.exec(attributes)?.[1];+ const segment = /\bsegment=\{story\.segment\.([A-Za-z0-9_-]+)\}/u.exec(attributes)?.[1];+ const media = /\bmedia=\{([A-Za-z0-9_-]+)\.media\}/u.exec(attributes)?.[1];+ if (id === undefined || segment === undefined) continue;+ takes.set(id, { segment, picture: media !== undefined && moving.has(media) });+ }+ for (const track of svml.matchAll(/<speech:Track\b([^>]*?)>(.*?)<\/speech:Track>/gsu)) {+ const frame = /\bvisual-frame=\{([A-Za-z0-9_-]+)\}/u.exec(track[1] ?? "")?.[1];+ if (frame === undefined || !fullFrames.has(frame)) continue;+ for (const take of (track[2] ?? "").matchAll(/<speech:Take\b[^>]*?\bsource=\{([A-Za-z0-9_-]+)\.take\}/gu)) {+ const named = takes.get(take[1] ?? "");+ if (named?.picture === true) coverSegment(named.segment);+ }+ }++ // The uncovered words as runs, cut at Segment boundaries so each run is named by the Segment a+ // reader would look in to find it.+ const gaps: CoverageGap[] = [];+ for (const segment of parsed.segments) {+ let run: number[] = [];+ const close = (): void => {+ if (run.length === 0) return;+ gaps.push({ segment: segment.id, words: run.map((index) => parsed.tokens[index]!.text).join(" "), word_count: run.length });+ run = [];+ };+ for (let index = segment.tokenStart; index < segment.tokenEndExclusive; index += 1) {+ if (covered[index] === true) close();+ else run.push(index);+ }+ close();+ }+ return { words: parsed.tokens.length, gaps };+ };+const running = await playbackReport();if (drawn.length === 0 && running.length === 0) {@@ -322,9 +439,13 @@ export async function reconstructionCheck(summary: ["no drawing element is placed in the Source; nothing to require."],elements: [],playback: [],+ uncovered: [],};}+ const coverage = coverageReport();+ const uncoveredWords = coverage.gaps.reduce((sum, gap) => sum + gap.word_count, 0);+// The Source does not name the reference it reconstructs, so a single prepared reference is used// when there is exactly one and named explicitly when there is more than one.const referenceRoot = join(roots.workspaceRoot, ".hypit", "reference-video-tools");@@ -424,6 +545,9 @@ export async function reconstructionCheck(if (running.length > 0) {summary.push(`${running.length} window${running.length === 1 ? "" : "s"} will empty before ${running.length === 1 ? "it ends" : "they end"}.`);}+ summary.push(uncoveredWords === 0+ ? `every word of the Script is drawn over by something that fills the frame (${coverage.words}).`+ : `${uncoveredWords} of ${coverage.words} words are drawn over by nothing that fills the frame.`);const compared = elements.filter((element) => element.comparisons > 0).length;if (compared > 0) {summary.push(untimed.length === 0@@ -434,7 +558,7 @@ export async function reconstructionCheck(return {run: runPath,reference_id: reference,- passed: never.length === 0 && running.length === 0,+ passed: never.length === 0 && running.length === 0 && coverage.gaps.length === 0,summary,elements,...(never.length === 0 ? {} : {@@ -481,6 +605,15 @@ export async function reconstructionCheck(+ `--reference-id ${reference} --segment <the Segment the shot covers> --out <rendered clip>.mp4`),},}),+ uncovered: coverage.gaps,+ ...(coverage.gaps.length === 0 ? {} : {+ uncovered_next: "Each stretch listed here shows whatever the placements around it left on screen, and "+ + "beneath the last of them is the Film background, which is black. Read "+ + "playbooks/craft/frame-coverage.md, then claim each stretch with whatever the reference shows "+ + "there: an element bound to the full Frame or to the Canvas, drawn over those words. Placing "+ + "something beneath the stretch instead changes which wrong picture appears and leaves the stretch "+ + "unclaimed.",+ }),playback: running,...(running.length === 0 ? {} : {playback_next: "A generated take is ordered in whole seconds and its window is measured from speech the "diff --git a/packages/reference-video-tools/src/cli.ts b/packages/reference-video-tools/src/cli.tsindex d5fb383d..7563a5df 100644--- a/packages/reference-video-tools/src/cli.ts+++ b/packages/reference-video-tools/src/cli.ts@@ -28,9 +28,10 @@ function usage(): string {"Studio's unit of work is the Run, and it reads the .svml back out of it.","","reconstruction_check reports what the route can settle before a Build: which drawing elements have",- "never been compared against the reference, and which timed pictures are configured to stop before",- "their window ends. It asks for participation rather than convergence, so an element compared once",- "passes. With one prepared reference it uses that one; with several it requires --reference-id.",+ "never been compared against the reference, which words of the Script nothing draws a full-frame",+ "picture over, and which timed pictures are configured to stop before their window ends. Comparison",+ "asks for participation rather than convergence, so an element compared once passes. With one",+ "prepared reference it uses that one; with several it requires --reference-id.","Each element also carries the basis its comparisons were made on — reference, estimate, mixed, or","not recorded — which says whether anything that changes with elapsed time inside its window has","been looked at, or only its layout.",