Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.
TypeScript agentic-aiaiai-agentsai-videocompilerdslffmpeggenerative-aillmmarkup-languagemonorepoplugin-systemprogramming-languagetext-to-videotypescriptvideovideo-automationvideo-clonevideo-editingvideo-generation
feat: the check names the round, and the program is drawn once for all of it (#87)
rponeawa156ea2807380
diff --git a/.agents/skills/hypit/references/element-review.md b/.agents/skills/hypit/references/element-review.mdindex b70e23e1..c565199a 100644--- a/.agents/skills/hypit/references/element-review.md+++ b/.agents/skills/hypit/references/element-review.md@@ -25,9 +25,26 @@ the target that does not trace, the chain that is missing — so its repairs arenames what is visibly wrong with the picture, in words, and a repair aims at something the roundnamed. Neither gate expects you to guess; each one tells you what is wrong, and you repair that.-## The round covers one element at a time, across every stretch it is drawn over+## The round is named by the check, not chosen by you-Loop over one authored element at a time — the new component, one caption system, one inserted card.+`authoring_check` and `reconstruction_check` read the Source and return a `plan`: every stretch worth+looking at, as an element and a half-open range of the Script's own words, with the reason it earned a+place. Render that list and read that list. Nothing here asks you to decide which stretches matter.++Three rules produce it, and all three are read off the Source. **A distinct declaration is a distinct+picture**, so an element is looked at once per way it is declared, at the stretch where that way first+appears — which is also the floor, since every placed element declares something. **Content that can+break the layout earns its own look**, which is why a caption Style is read over its longest Cue rather+than its shortest. **A window length earns one only when something scales with it** — an enter costs+its frames at one edge whatever the window is, but a typewriter, a loop or a `stretch` playback runs+for as long as the window does.++A range rather than a name, because the answer is not always a name: a Cue ends at a speaker change, so+it is a run of words the Script never marked. `--tokens from:to` is how every command takes one, and+`--segment`/`--selection` remain for the ranges that do have names.++The gate then holds the round to that list. An element looked at once used to pass; now every entry the+plan names has to have been answered.### Render what the Source configures, not what the catalogue shows@@ -75,27 +92,16 @@ saves the mock coming back as a finding in the one round there is.A mock lives only in this render. It is never written into the Source, and no gate reads it: the`playback` check reads the Source's Recipes, so a mock cannot be mistaken for coverage.-### The derived Run holds one Segment, not the whole Source--The mocks are substituted into a Source that has already been cut to the window. `render_element`-keeps the Segment the window names, drops the rest of the Script, and then drops every element that-named a Segment, Selection or Moment that went with it — and every element naming one of those, to a-fixed point, plus any container the cut left with no children. So an element bound to a Selection in-another Segment is not in this render, and its absence says nothing about the Source.--One name survives the cut rather than taking its element with it: a Moment in `until=` that is marked-*after* the kept Segment. An element given the span it occupies and a Moment to close it is on screen-for the whole of a stretch that ends before that word, so the cut removes the close and keeps the-element. That is what makes an element bounded by a late Moment renderable over the earlier Segments-it is drawn over, which is every stretch this round asks for it in.--The cut Source is on disk at `<project>/.hypit/compare/sliced.svml`, byte-for-byte from the original-except for what was removed, and the derived Run sits beside it. Open it whenever something you-expected to see is not in the picture: an element that is not in the fragment was cut by the window,-and one that is in the fragment and still not on screen is the package failing to draw it. Those two-repair in opposite directions, and the fragment is what tells them apart.-`packages/reference-video-tools/src/slice.ts` records the same thing as `SliceResult.dropped` — each-dropped id and the name it could no longer reach — and explains what decides survival.+### The derived Run carries the mocks, and nothing else changes++The mocks are substituted into a Run that names the project's own Source. The whole program is drawn+and the window is cut out of the frames afterwards, so every element the Source places is present and+drawn wherever the Source puts it — what is on screen over the words being looked at is what the+delivery will put there.++That derived Run is on disk under `<project>/.hypit/compare/`, beside the frames it produced. Open it+whenever something you expected to see is not in the picture: an element that is in the Source and+still not on screen is the package failing to draw it.### A caption Style is read once, where it first appearsdiff --git a/.agents/skills/hypit/references/original-authoring/route.md b/.agents/skills/hypit/references/original-authoring/route.mdindex 5b08c4da..3685e6a7 100644--- a/.agents/skills/hypit/references/original-authoring/route.md+++ b/.agents/skills/hypit/references/original-authoring/route.md@@ -89,10 +89,17 @@ reused in the Run Source. `../runtime.md` — how to author a Profile.tracing. `../preview.md` proves the Run traces before a Provider is reached. Neither is bounded by anyattempt ceiling: a graph that does not trace is work that is not done.-### 9. Render every stretch of every element+### 9. Ask the check what to render, then render it-**Read now:** `../element-review.md` — the round unit, what a render has to include, and how many-stretches to draw. `../preview.md` holds `render_element` itself.+```bash+hypit-reference-video-tools authoring_check projects/<name>/build.svrun+```++Its `plan` is the list: each entry an element and a word range, with the reason. Feed it to+`render_element --batch` — the program is drawn once and every entry is cut out of those frames.++**Read now:** `../element-review.md` — what the three rules are and why the list is what it is.+`../preview.md` holds `render_element` itself.Render the whole set before reading any of it. Studio is also free and is for the author to look at;it is not the picture this round reads.diff --git a/.agents/skills/hypit/references/preview.md b/.agents/skills/hypit/references/preview.mdindex 7aa1f918..2eb966c4 100644--- a/.agents/skills/hypit/references/preview.md+++ b/.agents/skills/hypit/references/preview.md@@ -80,12 +80,17 @@ command appends the output suffix itself when it looks for the track, so `--elemmatches nothing: it refuses with `the Source places no element named captions.track` and lists thebare ids that are placed.-`--segment` and `--selection` take a Script name, never a timestamp. Without either, the whole program-is drawn. An `--out` ending `.mp4`, `.mov` or `.webm` writes the stretch as a clip; any other extension-writes one still from the middle of it.--A whole round is one call: `--batch <renders.json>`, an array of `{element, segment|selection, out}`-inheriting the Run. They run together, paced by the machine.+`--segment` and `--selection` take a Script name, never a timestamp. `--tokens from:to` takes a+half-open range of the Script's own words, which is what a stretch nobody named is written as — a+caption Cue ends at a speaker change, so it has a range and no id. Without any of the three, the whole+program is drawn. An `--out` ending `.mp4`, `.mov` or `.webm` writes the stretch as a clip; any other+extension writes one still from the middle of it.++A whole round is one call: `--batch <renders.json>`, an array of+`{element, segment|selection|tokens, out}` inheriting the Run. **The program is drawn once for the+whole round** and each entry is cut out of those frames, so asking for eight windows costs one render+and eight cuts. The picture does not depend on `--element` — it is everything the Source places over+those words — so two elements over one stretch share the render as well.Each Segment's length comes from the Source's own `estimate:Speech`, which is the only clock a programhas before its speech is synthesized. The result's `timing` says what sized each Segment, and the samediff --git a/.agents/skills/hypit/references/reconstruction/route.md b/.agents/skills/hypit/references/reconstruction/route.mdindex 91f0a66d..67eb76d2 100644--- a/.agents/skills/hypit/references/reconstruction/route.md+++ b/.agents/skills/hypit/references/reconstruction/route.md@@ -252,10 +252,14 @@ first written. Neither this gate nor step 16 is bounded by the round's attempt c**Read now:** `../element-review.md`, then `comparison-round.md`, as soon as the sources place anelement — that is the first moment the round has something to render.-### 19. Render every stretch of every authored element+### 19. Ask the check what to render, then render it++`reconstruction_check` returns a `plan`: each entry an element and a word range, with the reason it+earned a look. That is the round — it is not yours to work out from the Source.`../preview.md` holds `render_element`; `comparison-round.md` says to pass `--reference-id` every time-so the stand-in runs on the reference's clock. Render the whole set before comparing any of it.+so the stand-in runs on the reference's clock. The program is drawn once and every entry is cut out of+those frames, so render the whole list in one `--batch` call.### 20. Send every comparison for the element at oncediff --git a/packages/reference-video-tools/src/authoring.ts b/packages/reference-video-tools/src/authoring.tsindex f144566e..0dc55964 100644--- a/packages/reference-video-tools/src/authoring.ts+++ b/packages/reference-video-tools/src/authoring.ts@@ -3,8 +3,6 @@ import { existsSync } from "node:fs";import { spawnSync } from "node:child_process";import { writePlaceholder } from "./placeholder.js";-import { sliceSource } from "./slice.js";-import type { SliceResult } from "./slice.js";import { createHash } from "node:crypto";import { createRequire } from "node:module";import { cpus } from "node:os";@@ -80,12 +78,28 @@ export type StandInTakes = {readonly selections: ReadonlyMap<string, AuthoringWindow>;readonly frameCount: number;readonly timing: readonly StandInTiming[];+ /**+ * Where every Script word sits in the program, in order.+ *+ * A window named by a Segment or a Selection is looked up above; one named by index is read+ * straight off this, which is the same table those two are built from.+ */+ readonly frameOfToken: readonly { readonly frame: number; readonly end: number }[];};/** Which Segment the render is looking at, named directly or through a Selection's words. */export type StandInFocus = {readonly segment?: string;readonly selection?: string;+ /**+ * A half-open range of the Script's own speech tokens.+ *+ * The finest thing worth looking at is not always a thing the Script named. A caption Cue is a run+ * of words ended by a speaker change or an authored break, and the planner picks it out by index —+ * so the index is what both sides take. `segment` and `selection` remain as the convenient way to+ * write down a range that does have a name.+ */+ readonly tokens?: readonly [number, number];};/**@@ -277,24 +291,6 @@ const SILENT_AUDIO: BlobRef = { kind: "blob", digest: `sha256:${"0".repeat(64)}`* @param reference the reference's per-word times, from `referenceWords`* @returns one `{ segmentId, take }` per Segment in Script order, and what timed each of them*/-/** The Segment a Selection is marked in, so a window named by Selection can still name the cut. */-function segmentOfSelection(svml: string, selection: string | undefined): string | undefined {- if (selection === undefined) return undefined;- const open = /<script\b[^>]*>/u.exec(svml);- if (open === null) return undefined;- const start = open.index + open[0].length;- const end = svml.indexOf("</script>", start);- if (end < 0) return undefined;- const body = svml.slice(start, end);- const at = body.indexOf(`@${selection}`);- if (at < 0) return undefined;- let found: string | undefined;- for (const match of body.matchAll(/<([a-z][a-z0-9-]*)\b[^>]*>/gu)) {- if (match.index > at) break;- if (body.indexOf(`</${match[1]!}>`, match.index) > at) found = match[1]!;- }- return found;-}/*** The Author SVML a Run declares, both as it is written and as a path.@@ -495,15 +491,22 @@ export async function spokenRange(// Which Script words the range covers. A Selection is marked once, so its word range is the// tokens between the anchors its open and close markers name.+ //+ // A range given by index skips the lookup entirely: it is already the answer the other two forms+ // are resolved into, and it is the only form that can name a stretch the Script never marked.let from: number | undefined;let to: number | undefined;- if (focus.selection !== undefined) {+ if (focus.tokens !== undefined) {+ [from, to] = focus.tokens;+ assert(from >= 0 && to > from && to <= parsed.tokens.length,+ `token range [${from}, ${to}) is outside the Script's ${parsed.tokens.length} words`);+ } else if (focus.selection !== undefined) {const selection = parsed.selections.find((item) => item.id === focus.selection);assert(selection !== undefined, `the Script marks no Selection ${focus.selection}`);from = selection.open.boundary.tokenIndex;to = selection.close.boundary.tokenIndex;} else {- assert(focus.segment !== undefined, "name a Segment or a Selection to read a word range from");+ assert(focus.segment !== undefined, "name a Segment, a Selection or a token range to read a word range from");const segment = parsed.segments.find((item) => item.id === focus.segment);assert(segment !== undefined, `the Script has no Segment ${focus.segment}`);from = segment.tokenStart;@@ -582,20 +585,6 @@ export async function standInTakes(const sheets = await recipeSheets(svml, svmlPath);const { policies, shared, policyCount } = policiesBySegment(svml, sheets);- // Which Segment the render is actually looking at. Every other Segment still has to exist — a- // Speech Track assembles one Take per Segment and an unsatisfied one refuses the whole projection —- // but nothing needs it at its estimated length. Held to one frame per word it stays legal, keeps- // its anchors, and stops the renderer drawing a minute of program to show six seconds of it.- let focused: string | undefined;- if (focus.segment !== undefined) focused = focus.segment;- else if (focus.selection !== undefined) {- const selection = parsed.selections.find((item) => item.id === focus.selection);- const token = selection?.open.boundary.tokenIndex;- focused = token === undefined- ? undefined- : parsed.segments.find((segment) => token >= segment.tokenStart && token < segment.tokenEndExclusive)?.id;- }-// Which Script word the reference speaks when. The Script was transcribed from that video, so the// words are the same words in the same order, and a stand-in timed from them runs at the pace the// reconstruction is being compared against.@@ -648,9 +637,10 @@ export async function standInTakes(weights = tokens.map((token) => Math.max(1, countSpeechEstimateUnits(token.text, language)));}- const frameCount = focused !== undefined && segment.id !== focused- ? Math.max(1, tokens.length)- : Math.max(tokens.length, Math.round(seconds * frameRate));+ // Every Segment at the length its clock gives it. The program is drawn once and a window is cut+ // out of it by frame, so a Segment nobody is looking at costs nothing by being its true length —+ // and the frame numbers then mean what they say instead of indexing a compressed timeline.+ const frameCount = Math.max(tokens.length, Math.round(seconds * frameRate));timing.push({ segment: segment.id, basis, seconds: round(seconds), frames: frameCount, tokens: tokens.length, matched });// Where each word ends. On the reference clock a word holds the screen until the next one starts@@ -719,7 +709,7 @@ export async function standInTakes(if (first === undefined || last === undefined) continue;if (first.frame < last.end) selections.set(selection.id, { startFrame: first.frame, endFrameExclusive: last.end });}- return { takes, selections, frameCount: frameCursor, timing };+ return { takes, selections, frameCount: frameCursor, timing, frameOfToken };}function blobRef(bytes: Uint8Array, mediaType: string): BlobRef {@@ -745,6 +735,74 @@ function silentWav(sampleFrames: number): Buffer {return buffer;}+/**+ * One browser render per Run and clock, within this process.+ *+ * A round asks for several windows of one program, and each of them would otherwise draw the whole+ * program again to throw most of it away. Keyed by the same hash the working directory is, so two+ * entries that share a directory share the render that fills it. Between processes this holds+ * nothing, which is correct: a new process is the one case where the Source may have changed.+ */+const programRenders = new Map<string, Promise<void>>();+async function drawProgramOnce(key: string, draw: () => Promise<void>): Promise<void> {+ const running = programRenders.get(key);+ if (running !== undefined) return await running;+ const started = draw();+ programRenders.set(key, started);+ try { await started; } catch (error) { programRenders.delete(key); throw error; }+}++/**+ * How big a mock for one declared generation should be.+ *+ * The shape matters and the pixel count does not. A mock is a flat fill, so nothing reads its+ * resolution — but a `fit: cover` crops a 4:5 source into a 9:16 Frame differently from a 9:16 one,+ * and a mock of the wrong shape makes the reader report a framing difference the real render will+ * not have, or miss one it will.+ *+ * The generation declares its own `aspect-ratio`, so that is what is read. Where it says `adaptive`+ * — seedance's "inherit from the input" — the shape comes from whatever it was handed, so the edge+ * is followed one hop to the upstream generation. Beyond one hop the returns fall away and a+ * malformed or circular graph starts to matter, so the Canvas is the answer from there.+ *+ * `resolution` is deliberately not read. It is a vendor token (`720p` naming a short side, `4K` a+ * long one) with no pixel mapping anywhere in this repository, and it decides only how large a+ * rectangle of flat colour would be.+ */+function mockExtent(svml: string, id: string, canvas: { readonly width: number; readonly height: number }): {+ readonly width: number;+ readonly height: number;+} {+ const declaring = (target: string): string | undefined =>+ new RegExp(`<[a-z][a-z0-9-]*:[A-Za-z][A-Za-z0-9]*\\b[^>]*?\\bid="${target}"[^>]*?/?>`, "su").exec(svml)?.[0];++ const ratioOf = (target: string): string | undefined => {+ const tag = declaring(target);+ if (tag === undefined) return undefined;+ const declared = /\baspect-ratio="([^"]+)"/u.exec(tag)?.[1];+ if (declared !== undefined && declared !== "adaptive") return declared;+ // One hop upstream. Two shapes need it: `adaptive`, which takes the shape of whatever it was+ // handed, and a `pipeline:Normalize`, which is what a mock is actually keyed by — the Source+ // names the normalized media, and the generation that declared the shape is behind its `source`.+ const upstream = /\b(?:source|image|first-frame|reference-image)=\{([A-Za-z0-9_-]+)[.}]/u.exec(tag)?.[1];+ if (upstream === undefined) return undefined;+ const inherited = /\baspect-ratio="([^"]+)"/u.exec(declaring(upstream) ?? "")?.[1];+ return inherited === "adaptive" ? undefined : inherited;+ };++ const ratio = ratioOf(id);+ const parts = /^(\d+):(\d+)$/u.exec(ratio ?? "");+ if (parts === null) return { width: canvas.width, height: canvas.height };+ const wide = Number(parts[1]);+ const tall = Number(parts[2]);+ if (!(wide > 0 && tall > 0)) return { width: canvas.width, height: canvas.height };+ // Scaled off the Canvas's short side, so a mock is never larger than the picture it sits in.+ const short = Math.min(canvas.width, canvas.height);+ return wide >= tall+ ? { width: Math.round(short * wide / tall), height: short }+ : { width: short, height: Math.round(short * tall / wide) };+}+export type RenderElementInput = {/*** A round of renders, run together. Each entry names its own element, stretch and output and@@ -758,6 +816,8 @@ export type RenderElementInput = {readonly out?: string;readonly segment?: string;readonly selection?: string;+ /** A half-open token range, for a stretch the Script never named. See `StandInFocus`. */+ readonly tokens?: readonly [number, number];/*** A prepared reference to time the stand-in from. Its transcript holds the seconds each word was* spoken at, and the Script was transcribed from that video, so the stand-in runs at the pace the@@ -890,6 +950,7 @@ export async function renderElement(input: RenderElementInput): Promise<Record<s// difference between a few seconds and a few minutes.const segmentArgument = input.segment;const selectionArgument = input.selection;+ const tokensArgument = input.tokens;const focus = {...(segmentArgument === undefined ? {} : { segment: segmentArgument }),...(selectionArgument === undefined ? {} : { selection: selectionArgument }),@@ -901,45 +962,28 @@ export async function renderElement(input: RenderElementInput): Promise<Record<s// deleted the first one's sliced Source and mocks out from under it. The route asks for every// element to be rendered before any is compared, which is a set of renders with nothing to say to// each other, and they can now run together.+ //+ // The key is the Run and the clock, and deliberately not the element or the window: the picture+ // does not depend on either. `render_element` draws the whole program and the window is cut out of+ // the frames afterwards, so every render of one Run under one clock wants the identical directory,+ // the identical mocks and the identical frames.const renderKey = createHash("sha256")- .update(`${resolve(out)}\u0000${element}\u0000${focus.segment ?? focus.selection ?? ""}`)+ .update(`${resolve(runPath)}\u0000${input.reference_id ?? ""}`).digest("hex").slice(0, 12);const compareRoot = join(projectRoot, ".hypit", "compare", renderKey);- await rm(compareRoot, { recursive: true, force: true });+ // Made, not remade. The frames inside are reused by every window cut out of this program, and the+ // mocks beside them are written from ids that do not move, so emptying it would only throw away the+ // render the next entry of the round is about to ask for.await mkdir(compareRoot, { recursive: true });- // With a window named, the Source is cut down to it before anything else runs. Every Segment- // outside it would otherwise still be drawn — held to one frame per word, which is the shortest a- // Take can legally be, and still nearly half the frames. The cut is byte-for-byte from the- // original, so the fragment cannot say anything the Source does not; `slice.ts` explains what- // decides which elements come with it.- let sourcePath = svmlPath;- let sliced: SliceResult | undefined;- const focusedSegment = focus.segment ?? segmentOfSelection(svml, focus.selection);- if (focusedSegment !== undefined) {- sliced = sliceSource(svml, focusedSegment);- sourcePath = join(compareRoot, "sliced.svml");- // The fragment sits three directories below the Source it came from, so its own relative paths- // have to reach back the same distance the derived Run's carried files do.- //- // `src` belongs in this list beside the import attributes. It is how the author's own supplied- // material is named — a presenter still, a voice sample — and left alone it resolved against the- // fragment's directory, where there is no `assets/`. Every Source carrying a `media:Image` with a- // relative path failed to render at all, on either route, the moment a window was named.- const repointed = sliced.text.replace(/(\s(?:source|from|src)=")\.\//gu, "$1../../../");- await writeFile(sourcePath, repointed, "utf8");- // Everything downstream reads the Source: which Takes to stand in for, which media to mock, which- // outputs to satisfy. Left on the original it would declare mocks for elements the cut removed,- // and a `<satisfy>` naming an output that no longer exists is refused.- svml = repointed;- }+ const sourcePath = svmlPath;// The reference's per-word times, when one was named. Read after the cut so a transcript that is// missing is reported before a minute of rendering rather than after it.const reference = input.reference_id?.trim();const spoken = reference === undefined || reference.length === 0 ? [] : await referenceWords(reference);- const { takes, selections, frameCount: programFrames, timing } = await standInTakes(sourcePath, frameRate, focus, spoken, svmlPath);+ const { takes, selections, frameCount: programFrames, timing, frameOfToken } = await standInTakes(sourcePath, frameRate, focus, spoken, svmlPath);const framesBySegment = new Map(takes.map((item) => [item.segmentId, item.take.segment.endFrameExclusive]));// A Selection's window in frames, summed over the stand-in tokens it covers. This is the same word// span the Source binds to, carried into frames by the same clock that sized the Segment.@@ -1022,7 +1066,7 @@ export async function renderElement(input: RenderElementInput): Promise<Record<sawait Promise.all(mocks.filter(([, { picture }]) => picture).map(async ([id, { frames }]) => await writePlaceholder({- out: join(compareRoot, `${id}.mp4`), width: canvas.width, height: canvas.height,+ out: join(compareRoot, `${id}.mp4`), ...mockExtent(svml, id, canvas),video: true, seconds: Math.max(1, Math.ceil(frames / frameRate)), color: "mid",})));for (const [id, { frames, picture }] of mocks) {@@ -1038,7 +1082,7 @@ export async function renderElement(input: RenderElementInput): Promise<Record<slet visual: SynchronizedMedia["visual"];if (picture) {const bytes = await readFile(file);- visual = { artifact: blobRef(bytes, "video/mp4"), width: canvas.width, height: canvas.height };+ visual = { artifact: blobRef(bytes, "video/mp4"), ...mockExtent(svml, id, canvas) };declarations.push(` <file id="mock-${id}" type="@hypit/artifact@1#BlobArtifact" from="./${id}.mp4" media-type="video/mp4"/>`);}const media: SynchronizedMedia = {@@ -1056,7 +1100,7 @@ export async function renderElement(input: RenderElementInput): Promise<Record<sconst imageIds = [...new Set([...svml.matchAll(/\{([a-z0-9-]+)\.image\}/gu)].map((match) => match[1] ?? ""))].filter((id) => !alreadySatisfied.has(`${id}.image`));await Promise.all(imageIds.map(async (id) => await writePlaceholder({- out: join(compareRoot, `${id}.png`), width: canvas.width, height: canvas.height, color: "mid",+ out: join(compareRoot, `${id}.png`), ...mockExtent(svml, id, canvas), color: "mid",})));for (const id of imageIds) {declarations.push(` <file id="mock-${id}" type="@hypit/artifact@1#BlobArtifact" from="./${id}.png" media-type="image/png"/>`);@@ -1070,12 +1114,13 @@ export async function renderElement(input: RenderElementInput): Promise<Record<s`<svrun version="1">`,// The cut, when there was one: it sits beside this Run, and its own imports were repointed to// reach the project from here.- sliced === undefined- ? ` <author source="../../${author.replace(/^\.\//u, "")}"/>`- : ` <author source="./sliced.svml"/>`,+ // Three levels up: the derived Run sits at `.hypit/compare/<key>/`, and the Source it names is at+ // the project root. This branch used to run only when no window was asked for, which is the one+ // case nothing exercised, so it was short by one the whole time.+ ` <author source="../../../${author.replace(/^\.\//u, "")}"/>`,// Whatever the Run brought, repointed: the derived Run sits two directories deeper than the one// that declared these paths.- ...carried.map((line) => ` ${line.replace(/from="\.\//gu, 'from="../../')}`),+ ...carried.map((line) => ` ${line.replace(/from="\.\//gu, 'from="../../../')}`),// Targeting the element's own output rather than the Film prunes the closure to what this one// Track needs. Asking for the whole delivery would pull in every generation the Source declares// and report each as unresolved, which is true and useless: none of them is what is being looked at.@@ -1136,7 +1181,14 @@ export async function renderElement(input: RenderElementInput): Promise<Record<s// The window to render, named in words. A shot of the reference is found by the words spoken over it// and those words are a Segment or a Selection here, so no reference timestamp is ever read across.let window: AuthoringWindow = { startFrame: 0, endFrameExclusive: programFrames };- if (selectionArgument !== undefined) {+ if (tokensArgument !== undefined) {+ // A range given by index is the general form: a Segment and a Selection are both resolved into+ // one, and a Cue is a range that has no other way to be named.+ const [from, to] = tokensArgument;+ assert(from >= 0 && to > from && to <= frameOfToken.length,+ `token range [${from}, ${to}) is outside the Script's ${frameOfToken.length} words`);+ window = { startFrame: frameOfToken[from]!.frame, endFrameExclusive: frameOfToken[to - 1]!.end };+ } else if (selectionArgument !== undefined) {const marked = selections.get(selectionArgument);assert(marked !== undefined, `the Script marks no Selection ${selectionArgument}`);window = marked;@@ -1167,22 +1219,29 @@ export async function renderElement(input: RenderElementInput): Promise<Record<sconst clip = /\.(mp4|mov|webm)$/iu.test(outPath);const frames = join(compareRoot, "frames");- await rm(frames, { recursive: true, force: true });- await mkdir(frames, { recursive: true });- // The runtime ships with the tree, not with the project. Resolving it against the package root- // finds nothing whenever those two differ, which is every project that installs packages of its own.- const hyperframesCli = createRequire(join(repositoryRoot(), "packages/provider-hyperframes-local/package.json"))- .resolve("hyperframes/bin/hyperframes.mjs");- const drawn = spawnSync(process.execPath, [- hyperframesCli, "render", stage,- "--format", "png-sequence", "--output", frames, "--fps", String(frameRate),- "--workers", String(Math.max(1, cpus().length - 2)),- // Halves the time and lands on the same pixels — verified frame for frame against a run without- // it, PSNR reporting no error at all. The flag is marked experimental upstream; that is the- // reason to check the pixels, which is done, rather than the reason to draw twice as long.- "--experimental-fast-capture", "--no-best-effort", "--quiet",- ], { encoding: "utf8", windowsHide: true, timeout: 600_000 });- assert(drawn.status === 0, `the HyperFrames runtime refused: ${(drawn.stderr ?? "").trim().slice(-2000)}`);+ // Drawn once per Run and clock, however many windows are asked for.+ //+ // The picture does not depend on the element or the window — the whole program is drawn and the+ // window is cut out of these frames — so a round of eight looks over one Source is one browser+ // render and eight ffmpeg cuts. Rendering per entry would draw the same program eight times.+ await drawProgramOnce(renderKey, async () => {+ await rm(frames, { recursive: true, force: true });+ await mkdir(frames, { recursive: true });+ // The runtime ships with the tree, not with the project. Resolving it against the package root+ // finds nothing whenever those two differ, which is every project that installs packages of its own.+ const hyperframesCli = createRequire(join(repositoryRoot(), "packages/provider-hyperframes-local/package.json"))+ .resolve("hyperframes/bin/hyperframes.mjs");+ const drawn = spawnSync(process.execPath, [+ hyperframesCli, "render", stage,+ "--format", "png-sequence", "--output", frames, "--fps", String(frameRate),+ "--workers", String(Math.max(1, cpus().length - 2)),+ // Halves the time and lands on the same pixels — verified frame for frame against a run without+ // it, PSNR reporting no error at all. The flag is marked experimental upstream; that is the+ // reason to check the pixels, which is done, rather than the reason to draw twice as long.+ "--experimental-fast-capture", "--no-best-effort", "--quiet",+ ], { encoding: "utf8", windowsHide: true, timeout: 600_000 });+ assert(drawn.status === 0, `the HyperFrames runtime refused: ${(drawn.stderr ?? "").trim().slice(-2000)}`);+ });const written = (await readdir(frames)).filter((name) => name.endsWith(".png")).sort();assert(written.length > 0, "the HyperFrames runtime wrote no frames");diff --git a/packages/reference-video-tools/src/checks.ts b/packages/reference-video-tools/src/checks.tsindex ae0d5c3e..bd8ad0ba 100644--- a/packages/reference-video-tools/src/checks.ts+++ b/packages/reference-video-tools/src/checks.ts@@ -16,6 +16,7 @@ import { inspectStudioRun } from "@hypit/studio/src/studio-preflight.js";import { aliasPattern, invokedFrom, nearestPackageRoot, referenceRoot, scriptBody } from "./authoring.js";import { assert, round } from "./media.js";+import { reviewPlan } from "./plan.js";export type PreviewCheckInput = {readonly run: string;@@ -576,6 +577,7 @@ export async function authoringCheck(return each.flat();});const drawingTags = new Set<string>();+ const timeVaryingKeys = new Set<string>();for (const pack of loaded) {for (const facet of pack.contribution.hostFacets ?? []) {if (facet.abi !== markupSurfaceHostFacetAbi) continue;@@ -583,6 +585,17 @@ export async function authoringCheck(const draws = surface.outputs.some((output) =>output.module.name === "@hypit/composition" && (output.name === "VisualTrack" || output.name === "AudioTrack"));if (draws) drawingTags.add(`${pack.specifier}#${surface.tag}`);+ // Which Recipe properties the package itself calls time-varying.+ //+ // `caption-fine` sorts every property it declares into where / how / when, and `when` is+ // exactly the question one of the planning rules asks: does this run for as long as the window+ // does? Reading the package's own answer keeps that rule correct for properties added after+ // this file was written, which a list maintained here would not be.+ for (const attribute of surface.vocabulary?.attributes ?? []) {+ for (const property of attribute.recipe ?? []) {+ if (property.group === "when") timeVaryingKeys.add(property.name);+ }+ }}}@@ -820,6 +833,22 @@ export async function authoringCheck(};}+ // What a round has to look at, decided from the Source rather than left to whoever runs it.+ //+ // The Recipe bodies come along because one of the three rules asks whether an element's appearance+ // runs for as long as its window does, and that is written in the Recipe rather than on the tag.+ const recipeBodies = new Map<string, string>();+ for (const sheet of svml.matchAll(/<import\s+as="[^"]+"\s+source="([^"]+\.svs)"/gu)) {+ const text = await readFile(resolve(dirname(svmlPath), sheet[1] ?? ""), "utf8").catch(() => undefined);+ if (text === undefined) continue;+ for (const recipe of text.matchAll(/([A-Za-z0-9_.-]+)\s*\{([^}]*)\}/gu)) recipeBodies.set(recipe[1] ?? "", recipe[2] ?? "");+ }+ const plan = reviewPlan({ svml, svmlPath, scriptBody: scriptBody(svml), drawn, recipes: recipeBodies, timeVaryingKeys });+ // The same parse the plan was built from, so a look recorded against a Segment can be resolved to+ // the words that Segment marks and compared with a plan entry on one axis.+ const plannedBody = scriptBody(svml);+ const plannedScript = parseScript(svmlPath, plannedBody.text, plannedBody.offset);+const coverage = coverageReport();const uncoveredWords = coverage.gaps.reduce((sum, gap) => sum + gap.word_count, 0);@@ -847,7 +876,7 @@ export async function authoringCheck(type LoggedComparison = {readonly element?: string;readonly shot_id?: string;- readonly range?: { readonly segment?: string; readonly selection?: string };+ readonly range?: { readonly segment?: string; readonly selection?: string; readonly tokens?: readonly [number, number] };readonly status: string;readonly id?: string;readonly image_path?: string;@@ -921,6 +950,33 @@ export async function authoringCheck(return recorded.size === 1 ? [...recorded][0]! : "mixed";};+ // Which of the plan's entries have been answered.+ //+ // A logged look and a planned one are compared as word ranges, because that is the one form both+ // can always be put in: a look recorded against a Segment resolves to the words that Segment marks,+ // and a look recorded against a Cue was never anything else. Comparing the names instead would let+ // a Cue in the middle of a Segment match the Segment, and the coverage this gate claims would be+ // wider than the coverage it has.+ const asTokens = (entry: LoggedComparison): string | undefined => {+ if (entry.range?.tokens !== undefined) return `${entry.range.tokens[0]}-${entry.range.tokens[1]}`;+ const segment = entry.range?.segment === undefined+ ? undefined+ : plannedScript.segments.find((item) => item.id === entry.range!.segment);+ if (segment !== undefined) return `${segment.tokenStart}-${segment.tokenEndExclusive}`;+ const selection = entry.range?.selection === undefined+ ? undefined+ : plannedScript.selections.find((item) => item.id === entry.range!.selection);+ if (selection !== undefined) return `${selection.open.boundary.tokenIndex}-${selection.close.boundary.tokenIndex}`;+ // A shot names a cut in the reference's picture rather than a range of the Script's words, so it+ // answers no plan entry. It still counts as having looked, which `rounds` already records.+ return undefined;+ };+ const answered = new Set(log.flatMap((entry) => {+ const tokens = asTokens(entry);+ return entry.element === undefined || tokens === undefined ? [] : [`${entry.element} ${tokens}`];+ }));+ const owed = plan.filter((entry) => !answered.has(`${entry.element} ${entry.tokens[0]}-${entry.tokens[1]}`));+const never = drawn.filter((element) => (rounds.get(element.id) ?? []).length === 0);const unlabelled = log.filter((entry) => entry.element === undefined).length;// A misspelled --element is otherwise silent: the comparison happens, the log grows, and the element@@ -956,6 +1012,10 @@ export async function authoringCheck(summary.push(`every drawing element has been ${looked} (${elements.length}), and every timed picture fills its window.`);}if (never.length > 0) summary.push(`${never.length} of ${elements.length} elements have never been ${looked}.`);+ if (owed.length > 0) {+ summary.push(`${owed.length} of ${plan.length} stretch${plan.length === 1 ? "" : "es"} the Source asks for `+ + `${owed.length === 1 ? "has" : "have"} not been ${looked}.`);+ }if (running.length > 0) {summary.push(`${running.length} window${running.length === 1 ? "" : "s"} will empty before ${running.length === 1 ? "it ends" : "they end"}.`);}@@ -996,14 +1056,30 @@ export async function authoringCheck(return {run: runPath,...(reference === undefined ? {} : { reference_id: reference }),+ plan,// Where this command actually read and resolved from. Said here, no document has to describe it// from the outside and go stale when it moves.roots: mode === "reconstruction"? { reference: preparedRoot, packages: packageRoot }: { reviews: logPath, packages: packageRoot },- passed: never.length === 0 && running.length === 0 && coverage.gaps.length === 0 && unresolved.length === 0,+ passed: owed.length === 0 && never.length === 0 && running.length === 0 && coverage.gaps.length === 0 && unresolved.length === 0,summary,elements,+ ...(owed.length === 0 ? {} : {+ owed: {+ entries: owed,+ note: `${owed.length} stretch${owed.length === 1 ? "" : "es"} the Source asks for ${owed.length === 1 ? "has" : "have"} `+ + "not been looked at. Each one is an element over a range of the Script's own words, and the reason it is "+ + "here is beside it — a declaration nobody has seen, the longest cue a caption Style has to hold, or a "+ + "window long enough that an animation inside it behaves differently.",+ commands: owed.map((entry) => mode === "reconstruction"+ ? `hypit-reference-video-tools compare_reconstruction --reference-id ${reference} --run ${runPath} `+ + `--tokens ${entry.tokens[0]}:${entry.tokens[1]} --video <rendered clip>.mp4 --element ${entry.element}`+ : `hypit-reference-video-tools review_element --run ${runPath} `+ + `--tokens ${entry.tokens[0]}:${entry.tokens[1]} --video <rendered clip>.mp4 --element ${entry.element} `+ + `--intent-file <what ${entry.element} was asked to be>`),+ },+ }),...(never.length === 0 ? {} : {never_compared: {ids: never.map((element) => element.id),diff --git a/packages/reference-video-tools/src/cli.ts b/packages/reference-video-tools/src/cli.tsindex 369f2c2d..af1ff952 100644--- a/packages/reference-video-tools/src/cli.ts+++ b/packages/reference-video-tools/src/cli.ts@@ -4,6 +4,22 @@ import { readFile } from "node:fs/promises";import { createReferenceVideoTools } from "./tools.js";import type { CompareReconstructionInput, ObserveReferenceInput, RenderElementInput, ReviewElementInput } from "./tools.js";+/**+ * A word range written on the command line as `from:to`, half-open.+ *+ * The planner emits ranges because the finest thing worth looking at is not always a thing the+ * Script named. `--segment` and `--selection` remain for the ones that are.+ */+function tokenRange(value: string | undefined): readonly [number, number] | undefined {+ if (value === undefined) return undefined;+ const parts = /^(\d+):(\d+)$/u.exec(value.trim());+ if (parts === null) throw new Error(`--tokens takes a half-open word range written from:to, like 118:131; got ${value}`);+ const from = Number(parts[1]);+ const to = Number(parts[2]);+ if (to <= from) throw new Error(`--tokens ${value} ends at or before it starts`);+ return [from, to];+}+type Flags = ReadonlyMap<string, string | readonly string[] | boolean>;function usage(): string {@@ -18,14 +34,14 @@ function usage(): string {" hypit-reference-video-tools inspect_svml_vocabulary --package <name> [--package <name> ...] [--tag <tag> ...] [--without-previews]"," hypit-reference-video-tools inspect_visual_contract [--shape visual-track|text-flow|text-typography|text-paint|text-document|path-command] [--producers-of <package> ...]"," hypit-reference-video-tools paths",- " hypit-reference-video-tools compare_reconstruction --reference-id <id> --run <build.svrun> --segment <id>|--selection <id> --video <path>|--image <path> [--question <scope>] [--element <id>]",+ " hypit-reference-video-tools compare_reconstruction --reference-id <id> --run <build.svrun> --segment <id>|--selection <id>|--tokens <from:to> --video <path>|--image <path> [--question <scope>] [--element <id>]"," hypit-reference-video-tools compare_reconstruction --reference-id <id> --shot-id <id> --video <path>|--image <path> [--question <scope>] [--element <id>]"," hypit-reference-video-tools compare_reconstruction --reference-id <id> --batch <comparisons.json>",- " hypit-reference-video-tools review_element --run <build.svrun> --element <id> --segment <id>|--selection <id> --video <path>|--image <path> --intent-file <path> [--question <scope>]",+ " hypit-reference-video-tools review_element --run <build.svrun> --element <id> --segment <id>|--selection <id>|--tokens <from:to> --video <path>|--image <path> --intent-file <path> [--question <scope>]"," hypit-reference-video-tools review_element --run <build.svrun> --batch <reviews.json>"," hypit-reference-video-tools record_review --run <build.svrun> --review-id <id> --text <text>|--text-file <path>"," hypit-reference-video-tools make_placeholder --out <path> --width <w> --height <h> [--color light|mid|dark|white|black|#RRGGBB] [--video] [--seconds <s>]",- " hypit-reference-video-tools render_element <build.svrun> --element <id> --out <path.png|path.mp4> [--segment <id>] [--selection <id>] [--reference-id <id>]",+ " hypit-reference-video-tools render_element <build.svrun> --element <id> --out <path.png|path.mp4> [--segment <id>] [--selection <id>] [--tokens <from:to>] [--reference-id <id>]"," hypit-reference-video-tools render_element <build.svrun> --batch <renders.json> [--reference-id <id>]"," hypit-reference-video-tools render_previews <package-dir> [...]"," hypit-reference-video-tools preview_check <build.svrun> [<hypit.runtime.json>]",@@ -302,6 +318,7 @@ async function main(): Promise<void> {element: required(flags, "element"),...(one(flags, "segment") === undefined ? {} : { segment: one(flags, "segment") }),...(one(flags, "selection") === undefined ? {} : { selection: one(flags, "selection") }),+ ...(tokenRange(one(flags, "tokens")) === undefined ? {} : { tokens: tokenRange(one(flags, "tokens")) }),...(one(flags, "image") === undefined ? {} : { image_path: one(flags, "image") }),...(one(flags, "video") === undefined ? {} : { video_path: one(flags, "video") }),...(one(flags, "intent-file") === undefined ? {} : { intent_file: one(flags, "intent-file") }),@@ -336,12 +353,14 @@ async function main(): Promise<void> {const video = one(flags, "video");const segment = one(flags, "segment");const selection = one(flags, "selection");- const range = segment !== undefined || selection !== undefined;+ const tokens = tokenRange(one(flags, "tokens"));+ const range = segment !== undefined || selection !== undefined || tokens !== undefined;const input = supplied ?? {reference_id: required(flags, "reference-id"),...(range? {run: required(flags, "run"),+ ...(tokens === undefined ? {} : { tokens }),...(segment === undefined ? {} : { segment }),...(selection === undefined ? {} : { selection }),}@@ -398,9 +417,10 @@ async function main(): Promise<void> {out: required(flags, "out"),...(one(flags, "segment") === undefined ? {} : { segment: one(flags, "segment") }),...(one(flags, "selection") === undefined ? {} : { selection: one(flags, "selection") }),+ ...(tokenRange(one(flags, "tokens")) === undefined ? {} : { tokens: tokenRange(one(flags, "tokens")) }),...(one(flags, "reference-id") === undefined ? {} : { reference_id: one(flags, "reference-id") }),};- result = await tools.render_element(input as { run: string; element: string; out: string; segment?: string; selection?: string; reference_id?: string });+ result = await tools.render_element(input as RenderElementInput);} else if (command === "render_previews") {if (supplied === undefined && operands.length === 0) throw new Error(`a <package-dir> is required\n\n${usage()}`);const input = supplied ?? { package_dirs: operands };diff --git a/packages/reference-video-tools/src/plan.ts b/packages/reference-video-tools/src/plan.tsnew file mode 100644index 00000000..036606a6--- /dev/null+++ b/packages/reference-video-tools/src/plan.ts@@ -0,0 +1,499 @@+/**+ * Which stretches of a program are worth looking at, decided from the Source alone.+ *+ * An element drawn over five Segments does not need five looks. It needs one per way it can fail,+ * and two stretches fail the same way when they are the same declaration over the same kind of+ * content for the same length of time. Deciding that by reading the Source is what makes a round+ * reproducible and what lets the gate ask for coverage rather than for participation.+ *+ * Three rules produce the list, and every one of them reads text:+ *+ * 1. **A distinct declaration is a distinct picture.** Group an element's placements by what they+ * declare and look at each group once, where it first appears. This is also the floor: every+ * placed element has at least one declaration, so every element gets at least one look — which is+ * exactly what the gate required before this file existed.+ * 2. **Content that can break the layout is worth its own look.** Within a group, the stretch+ * holding the most is a different question from the stretch holding the least: a caption system+ * read over its shortest line looks correct, because one line has nothing to collide with.+ * 3. **A window length is worth its own look only when something scales with it.** An enter and an+ * exit happen at the edges for a fixed number of frames; a typewriter, a loop or a `stretch`+ * playback runs for as long as the window does.+ *+ * The unit is a token range rather than a Segment name, because the answer is not always a name. A+ * caption Cue is a run of words the Script never marked, and it is exactly what rule 2 wants to look+ * at. Both sides of a comparison already work in words — `frameOfToken` here, `spokenRange` on the+ * reference — so the range is what they both take, and a name comes along only to be read.+ */+import { parseScript } from "@hypit/script";+import type { ParsedNarrative } from "@hypit/script";++/** One thing to look at: an element, over a range of the Script's words, and why it earned a look. */+export type PlanEntry = {+ readonly element: string;+ /** Half-open over the Script's speech tokens, the same index space `frameOfToken` is in. */+ readonly tokens: readonly [number, number];+ /** What that range is, for a reader. Never parsed back — the range is the identity. */+ readonly named: string;+ readonly why: readonly string[];+};++/** A drawing element as the gate enumerates it, before its placements are read. */+export type PlannedElement = {+ readonly id: string;+ readonly tag: string;+ readonly alias: string;+ readonly specifier: string;+};++/**+ * Attributes that say *when* rather than *what*, so they never join a declaration key.+ *+ * Two placements differing only in these are the same design over different words, which is one+ * look and two stretches — the case rule 1 exists to collapse.+ */+const WINDOW_ATTRIBUTES = new Set([+ "during", "at", "for", "start", "end", "selection", "segment", "moment", "until",+ "until-boundary", "terminal", "semantic", "document", "narrative", "program-ref",+]);++/**+ * Attributes naming material a Build has not made, so they never join a declaration key either.+ *+ * Every one of these is a flat placeholder in the render this plan is for. Two Items differing only+ * in which unbuilt picture they name draw the identical mock, so looking at both spends a subagent+ * to see the same grey rectangle twice.+ */+const MOCKED_ATTRIBUTES = new Set(["image", "video", "media", "source", "icon", "audio", "source-audio"]);++/** Never a declaration: the id is what the key is being computed *for*. */+const IGNORED_ATTRIBUTES = new Set(["id"]);++/**+ * Recipe keys whose effect lasts as long as the window does.+ *+ * An `enter` costs its `enter-frames` at one edge whatever the window is; these do not. A window+ * twice as long runs twice as much typewriter, twice as many loop cycles, and stretches its material+ * to twice the duration — so two windows of very different lengths are two questions.+ *+ * `caption-fine` publishes the same judgement as machine-readable metadata (`group: "when"`), and+ * where that is present it is read instead of this list. This is the fallback for the packages that+ * do not yet classify their own properties.+ */+const WHOLE_WINDOW_KEYS = [+ "sustain", "playback", "playback-future", "playback-past", "trim-start", "trim-end",+ "loop", "loop-target", "loop-period-frames", "loop-intensity",+ "atom-reveal", "karaoke", "karaoke-transition", "active-underline",+ "active-box", "active-box-continuity",+];++type Placement = {+ readonly element: string;+ readonly tag: string;+ readonly key: string;+ readonly stretch: Stretch;+ /** Everything the placement declares, kept so rule 3 can read the Recipes it names. */+ readonly attributes: ReadonlyMap<string, string>;+};++type Stretch = {+ readonly tokens: readonly [number, number];+ readonly named: string;+};++/** Every attribute on an opening tag, as written. */+function attributesOf(text: string): ReadonlyMap<string, string> {+ const found = new Map<string, string>();+ for (const match of text.matchAll(/\b([a-z][a-z0-9-]*)="([^"]*)"/gu)) found.set(match[1] ?? "", match[2] ?? "");+ // A reference is written `attr={id}` rather than quoted, and it is the form every Recipe and Frame+ // arrives in, so a key built from quoted attributes alone would miss the whole declaration.+ for (const match of text.matchAll(/\b([a-z][a-z0-9-]*)=\{([^}]*)\}/gu)) found.set(match[1] ?? "", `{${match[2] ?? ""}}`);+ return found;+}++/**+ * What this placement declares, as one comparable string.+ *+ * Everything on the tag except the three excluded sets, sorted so attribute order — which this+ * markup does not make significant — cannot split one group into two. The tag name leads, because+ * the same attributes on a different element are a different picture.+ *+ * Taking everything rather than a list of known Recipe attributes is deliberate. `screen-overlay`+ * declares its entire appearance as tag literals and names no Recipe at all, and a package added+ * tomorrow will be read correctly without this file learning its vocabulary. The cost of the wide+ * net is a split group, which is one extra look; the cost of a narrow one is a merged group, which+ * is a picture nobody sees.+ */+function declarationKey(tag: string, attributes: ReadonlyMap<string, string>): string {+ const parts: string[] = [];+ for (const [name, value] of attributes) {+ if (WINDOW_ATTRIBUTES.has(name) || MOCKED_ATTRIBUTES.has(name) || IGNORED_ATTRIBUTES.has(name)) continue;+ parts.push(`${name}=${value}`);+ }+ return `${tag}|${parts.sort().join("|")}`;+}++/** The whole Script, for an element bound to the program rather than to a marked range. */+function wholeProgram(parsed: ParsedNarrative): Stretch {+ return { tokens: [0, parsed.tokens.length], named: "the whole program" };+}++/**+ * Where a placement's `during=` or `at=` puts it, in words.+ *+ * Returns undefined when the binding names something the Script does not mark, which is a Source+ * defect `preview_check` reports rather than something to guess at here.+ */+function stretchOf(attributes: ReadonlyMap<string, string>, parsed: ParsedNarrative): Stretch | undefined {+ const during = attributes.get("during");+ if (during === "program") return wholeProgram(parsed);++ const named = (value: string | undefined): { readonly kind: string; readonly id: string } | undefined => {+ const match = /^\{story\.(segment|selection|moment)\.([A-Za-z0-9_-]+)\}$/u.exec(value ?? "");+ return match === undefined || match === null ? undefined : { kind: match[1]!, id: match[2]! };+ };++ const bound = named(during) ?? named(attributes.get("at"));+ if (bound === undefined) return undefined;++ if (bound.kind === "segment") {+ const segment = parsed.segments.find((item) => item.id === bound.id);+ return segment === undefined+ ? undefined+ : { tokens: [segment.tokenStart, segment.tokenEndExclusive], named: `segment ${bound.id}` };+ }+ if (bound.kind === "selection") {+ const selection = parsed.selections.find((item) => item.id === bound.id);+ return selection === undefined+ ? undefined+ : {+ tokens: [selection.open.boundary.tokenIndex, selection.close.boundary.tokenIndex],+ named: `selection ${bound.id}`,+ };+ }+ // A Moment is an instant, not a range. What it marks is a change inside whichever Segment holds+ // it, so that Segment is the stretch a look would be rendered over.+ const moment = parsed.moments.find((item) => item.id === bound.id);+ if (moment === undefined) return undefined;+ const index = moment.boundary.tokenIndex;+ const holding = parsed.segments.find((item) => index >= item.tokenStart && index < item.tokenEndExclusive);+ return holding === undefined+ ? undefined+ : { tokens: [holding.tokenStart, holding.tokenEndExclusive], named: `segment ${holding.id} (at moment ${bound.id})` };+}++/**+ * The body of one element, so its children can be read.+ *+ * These Tracks do not nest inside themselves, so the first matching close tag is this element's.+ * Returns an empty string for a self-closing element, which has no children to find.+ */+function bodyOf(svml: string, alias: string, tag: string, id: string): string {+ const opening = new RegExp(`<${alias}:${tag}\\b[^>]*?\\bid="${id}"[^>]*?(/?)>`, "su").exec(svml);+ if (opening === null || opening[1] === "/") return "";+ const after = svml.slice(opening.index + opening[0].length);+ const close = after.indexOf(`</${alias}:${tag}>`);+ return close === -1 ? after : after.slice(0, close);+}++/** The opening tag of one element, by id. */+function openingOf(svml: string, alias: string, tag: string, id: string): string {+ return new RegExp(`<${alias}:${tag}\\b[^>]*?\\bid="${id}"[^>]*?/?>`, "su").exec(svml)?.[0] ?? "";+}++/**+ * The Styles a caption Program hands out, and the stretch each one covers.+ *+ * A caption Track is one tag with no window on it — it draws wherever the Script has words — so+ * reading its placements the way every other element is read finds exactly one, and reports one+ * design. That is wrong whenever the Program overrides a Style: `<caption:Use style=… selection=…>`+ * is how one stretch is drawn differently from the rest, and it is written inside the Program rather+ * than on the Track, so nothing on the Track says it happened.+ *+ * Each override is its own declaration over its own words. The default is a declaration too, over+ * whatever the overrides did not claim, which is where the Track's single placement belongs.+ */+function captionStyles(svml: string, element: PlannedElement, parsed: ParsedNarrative): readonly Placement[] {+ const opening = openingOf(svml, element.alias, element.tag, element.id);+ const program = /\bprogram=\{([A-Za-z0-9_-]+)\}/u.exec(opening)?.[1];+ if (program === undefined) return [];+ const declaring = new RegExp(`<[a-z][a-z0-9-]*:Program\\b[^>]*?\\bid="${program}"[^>]*?(/?)>`, "su").exec(svml);+ if (declaring === null) return [];++ const body = declaring[1] === "/" ? "" : (() => {+ const after = svml.slice(declaring.index + declaring[0].length);+ const close = after.search(/<\/[a-z][a-z0-9-]*:Program>/u);+ return close === -1 ? after : after.slice(0, close);+ })();++ const tag = `${element.alias}:${element.tag}`;+ const found: Placement[] = [];+ for (const use of body.matchAll(/<[a-z][a-z0-9-]*:Use\b([^>]*?)\/?>/gsu)) {+ const attributes = attributesOf(use[0] ?? "");+ // A Use names its stretch with `selection=`, not the `during=` every drawn element uses, because+ // it is assigning a Style to a run of words rather than placing a picture over them.+ const marked = /^\{story\.selection\.([A-Za-z0-9_-]+)\}$/u.exec(attributes.get("selection") ?? "")?.[1];+ const selection = marked === undefined ? undefined : parsed.selections.find((item) => item.id === marked);+ const stretch = selection === undefined+ ? stretchOf(attributes, parsed)+ : {+ tokens: [selection.open.boundary.tokenIndex, selection.close.boundary.tokenIndex] as readonly [number, number],+ named: `selection ${marked}`,+ };+ const style = attributes.get("style");+ if (stretch === undefined || style === undefined) continue;+ found.push({+ element: element.id, tag: `${tag} (caption:Use)`,+ key: `${tag}|style=${style}`, stretch, attributes,+ });+ }+ return found;+}++/**+ * Every placement one drawing element makes.+ *+ * A Track holding forty Items draws nothing itself: the Items carry the windows and the Recipes, and+ * they are what a look is actually of. So the element's own tag is a placement only when it binds a+ * window itself, and each child that binds one is a placement too.+ */+function placementsOf(svml: string, element: PlannedElement, parsed: ParsedNarrative): readonly Placement[] {+ const found: Placement[] = [];+ const opening = openingOf(svml, element.alias, element.tag, element.id);+ const own = attributesOf(opening);+ const ownStretch = stretchOf(own, parsed);+ if (ownStretch !== undefined) {+ found.push({+ element: element.id, tag: `${element.alias}:${element.tag}`,+ key: declarationKey(`${element.alias}:${element.tag}`, own), stretch: ownStretch, attributes: own,+ });+ }++ for (const child of bodyOf(svml, element.alias, element.tag, element.id)+ .matchAll(/<([a-z][a-z0-9-]*:[A-Za-z][A-Za-z0-9]*)\b([^>]*?)\/?>/gsu)) {+ const tag = child[1] ?? "";+ const attributes = attributesOf(child[0] ?? "");+ const stretch = stretchOf(attributes, parsed);+ if (stretch === undefined) continue;+ // The child's own declaration is what it draws, but it sits inside the Track's, so the Track's+ // is part of the key: the same Item under two Tracks with different Canvases is two pictures.+ found.push({+ element: element.id,+ tag,+ key: `${declarationKey(`${element.alias}:${element.tag}`, own)}>>${declarationKey(tag, attributes)}`,+ stretch,+ attributes,+ });+ }++ // An element that binds no window anywhere is on screen for the whole program — a persistent+ // badge, a Film-wide treatment — and the whole program is the stretch it is looked at over.+ if (found.length === 0) {+ found.push({+ element: element.id,+ tag: `${element.alias}:${element.tag}`,+ key: declarationKey(`${element.alias}:${element.tag}`, own),+ stretch: wholeProgram(parsed),+ attributes: own,+ });+ }+ return found;+}++/**+ * Rule 2, for captions: where the longest Cue is.+ *+ * A Cue is the unit a caption's layout can break on, and it is not a Segment. Four things end one,+ * and only one of them is the `||` the author wrote: the other three are a Segment ending, a speaker+ * changing, and a Style changing. Partitioning on the authored breaks alone reads a two-speaker+ * script as a handful of enormous Cues, and then reports the longest line as the whole exchange.+ *+ * The partition is computed in token space rather than by calling `temporalizeCaptionDocument`,+ * which needs a SemanticTrack this check never builds. The four facts are the same ones it groups on+ * and all four are Script-side, so the boundaries land in the same places; what is not computed here+ * is the frame each Cue occupies, which nothing on this path asks for.+ */+function longestCue(parsed: ParsedNarrative, styleBoundaries: readonly number[]): Stretch | undefined {+ const total = parsed.tokens.length;+ if (total === 0) return undefined;+ const boundaries = new Set<number>([0, total]);+ for (const segment of parsed.segments) boundaries.add(segment.tokenStart);+ for (const turn of parsed.turns) boundaries.add(turn.tokenStart);+ for (const authored of parsed.captionProjection.breaks) boundaries.add(authored.tokenIndex);+ for (const boundary of styleBoundaries) boundaries.add(boundary);++ const edges = [...boundaries].filter((at) => at >= 0 && at <= total).sort((one, other) => one - other);+ let best: { readonly from: number; readonly to: number; readonly characters: number } | undefined;+ for (let index = 0; index + 1 < edges.length; index += 1) {+ const from = edges[index]!;+ const to = edges[index + 1]!;+ if (to <= from) continue;+ const characters = parsed.tokens.slice(from, to).reduce((sum, token) => sum + token.text.length, 0);+ if (best === undefined || characters > best.characters) best = { from, to, characters };+ }+ if (best === undefined) return undefined;+ const holding = parsed.segments.find((item) => best!.from >= item.tokenStart && best!.from < item.tokenEndExclusive);+ return {+ tokens: [best.from, best.to],+ named: holding === undefined ? "the longest cue" : `the longest cue, in segment ${holding.id}`,+ };+}++/**+ * Where a `caption:Program` changes Style, in token space.+ *+ * `<caption:Use selection=…>` is how one stretch is drawn differently from the rest, and it is the+ * only place a caption's declaration varies — the Track's own tag is identical over the whole+ * program. Both ends of the named Selection are Cue boundaries, because a Style change ends a Cue.+ */+function captionStyleBoundaries(svml: string, parsed: ParsedNarrative): readonly number[] {+ const found: number[] = [];+ for (const use of svml.matchAll(/<[a-z][a-z0-9-]*:(?:Use|Mute)\b([^>]*?)\/?>/gsu)) {+ const selection = /\bselection=\{story\.selection\.([A-Za-z0-9_-]+)\}/u.exec(use[1] ?? "")?.[1];+ if (selection === undefined) continue;+ const marked = parsed.selections.find((item) => item.id === selection);+ if (marked === undefined) continue;+ found.push(marked.open.boundary.tokenIndex, marked.close.boundary.tokenIndex);+ }+ return found;+}++/**+ * Rule 2, for anything holding a list: which stretch holds the most of it.+ *+ * Items are written into the Source as children, each bound to its own Moment or Selection, so the+ * count per stretch is a count of children — no rendering and no measurement. The stretch holding+ * the most is where the layout has the most chance to run out of room.+ */+function fullestStretch(placements: readonly Placement[]): Stretch | undefined {+ const counts = new Map<string, { readonly stretch: Stretch; count: number }>();+ for (const placement of placements) {+ const key = `${placement.stretch.tokens[0]}-${placement.stretch.tokens[1]}`;+ const held = counts.get(key);+ if (held === undefined) counts.set(key, { stretch: placement.stretch, count: 1 });+ else held.count += 1;+ }+ const ranked = [...counts.values()].sort((one, other) => other.count - one.count);+ const most = ranked[0];+ // One each is not a list. Nominating a stretch here would just re-name what rule 1 already has.+ return most === undefined || most.count < 2 ? undefined : most.stretch;+}++/**+ * Rule 3: does anything about this element's appearance run for as long as its window does?+ *+ * `caption-fine` classifies its own properties, so where a Recipe body carries a key that package+ * marked `when`, that is the authority. Everything else is matched against the list above, which is+ * the same judgement made by hand for the packages that do not publish one yet.+ */+function scalesWithWindow(bodies: readonly string[], published: ReadonlySet<string>): boolean {+ const keys = [...new Set([...published, ...WHOLE_WINDOW_KEYS])];+ return bodies.some((body) => keys.some((key) => new RegExp(`\\b${key}\\s*:`, "u").test(body)));+}++/** The Recipe bodies one placement's declaration names, so rule 3 can read their keys. */+function recipeBodiesOf(attributes: ReadonlyMap<string, string>, recipes: ReadonlyMap<string, string>): readonly string[] {+ const bodies: string[] = [];+ for (const [name, value] of attributes) {+ if (WINDOW_ATTRIBUTES.has(name) || MOCKED_ATTRIBUTES.has(name)) continue;+ const named = /^\{([A-Za-z0-9_.-]+)\}$/u.exec(value)?.[1];+ if (named === undefined) continue;+ const body = recipes.get(named.replace(/^recipes\./u, ""));+ if (body !== undefined) bodies.push(body);+ }+ return bodies;+}++/** Rule 1: one look per distinct declaration, at the stretch where that declaration first appears. */+function firstOfEachDeclaration(placements: readonly Placement[]): readonly PlanEntry[] {+ const seen = new Set<string>();+ const entries: PlanEntry[] = [];+ for (const placement of placements) {+ if (seen.has(placement.key)) continue;+ seen.add(placement.key);+ entries.push({+ element: placement.element,+ tokens: placement.stretch.tokens,+ named: placement.stretch.named,+ why: [`first use of this declaration (${placement.tag})`],+ });+ }+ return entries;+}++/**+ * Merge nominations that landed on the same thing.+ *+ * Three rules can each name one stretch — the first occurrence of a Style may also hold its longest+ * line and its longest window — and that is one look with three reasons, not three looks.+ */+function merge(entries: readonly PlanEntry[]): readonly PlanEntry[] {+ const byKey = new Map<string, PlanEntry>();+ for (const entry of entries) {+ const key = `${entry.element} ${entry.tokens[0]}-${entry.tokens[1]}`;+ const held = byKey.get(key);+ if (held === undefined) { byKey.set(key, entry); continue; }+ byKey.set(key, { ...held, why: [...held.why, ...entry.why.filter((reason) => !held.why.includes(reason))] });+ }+ return [...byKey.values()];+}++/** What a round has to look at, for one Source. */+export function reviewPlan(input: {+ readonly svml: string;+ readonly svmlPath: string;+ readonly scriptBody: { readonly text: string; readonly offset: number };+ readonly drawn: readonly PlannedElement[];+ /** Recipe bodies by their sheet-local name, for rule 3. Absent means rule 3 nominates nothing. */+ readonly recipes?: ReadonlyMap<string, string>;+ /**+ * Property names the packages themselves marked time-varying, read from their own vocabulary.+ *+ * A package that classifies its properties is the authority on them, and it stays right as it adds+ * more. The list in this file is the fallback for the packages that publish no classification.+ */+ readonly timeVaryingKeys?: ReadonlySet<string>;+}): readonly PlanEntry[] {+ const parsed = parseScript(input.svmlPath, input.scriptBody.text, input.scriptBody.offset);+ const recipes = input.recipes ?? new Map<string, string>();+ const published = input.timeVaryingKeys ?? new Set<string>();+ const styleBoundaries = captionStyleBoundaries(input.svml, parsed);+ const entries: PlanEntry[] = [];++ for (const element of input.drawn) {+ // A caption Track's designs are declared inside its Program, not on the Track, so they are read+ // separately and joined to whatever the ordinary placement scan found.+ const placements = [+ ...placementsOf(input.svml, element, parsed),+ ...(element.specifier.includes("caption") ? captionStyles(input.svml, element, parsed) : []),+ ];+ entries.push(...firstOfEachDeclaration(placements));++ // Rule 2. A caption's layout breaks on a Cue, and a list's breaks on how many items one stretch+ // holds; every other element draws one thing and has no content extreme to find.+ if (element.specifier.includes("caption")) {+ const cue = longestCue(parsed, styleBoundaries);+ if (cue !== undefined) entries.push({ element: element.id, tokens: cue.tokens, named: cue.named, why: ["longest cue"] });+ }+ const fullest = fullestStretch(placements);+ if (fullest !== undefined) {+ entries.push({ element: element.id, tokens: fullest.tokens, named: fullest.named, why: ["most items in one stretch"] });+ }++ // Rule 3. Only where something runs for as long as the window does, and only then are the+ // longest and the shortest two different questions rather than the same picture twice.+ const animates = placements.some((placement) => scalesWithWindow(recipeBodiesOf(placement.attributes, recipes), published));+ if (animates && placements.length > 1) {+ const byLength = [...placements].sort((one, other) =>+ (one.stretch.tokens[1] - one.stretch.tokens[0]) - (other.stretch.tokens[1] - other.stretch.tokens[0]));+ const shortest = byLength[0]!;+ const longest = byLength.at(-1)!;+ if (longest.stretch.tokens[1] - longest.stretch.tokens[0] !== shortest.stretch.tokens[1] - shortest.stretch.tokens[0]) {+ entries.push({ element: element.id, tokens: longest.stretch.tokens, named: longest.stretch.named, why: ["longest window, and this element animates across it"] });+ entries.push({ element: element.id, tokens: shortest.stretch.tokens, named: shortest.stretch.named, why: ["shortest window, and this element animates across it"] });+ }+ }+ }+ return merge(entries);+}++export { WHOLE_WINDOW_KEYS };diff --git a/packages/reference-video-tools/src/slice.ts b/packages/reference-video-tools/src/slice.tsdeleted file mode 100644index 68d50e92..00000000--- a/packages/reference-video-tools/src/slice.ts+++ /dev/null@@ -1,291 +0,0 @@-/**- * Cut an Author Source down to the one stretch a comparison is looking at, mechanically.- *- * Rendering an element means rendering a program, and a program is as long as its Script. Comparing- * six seconds of a sixty-second video meant drawing all sixty and cropping, which is most of the- * cost of a comparison round.- *- * The cut is a **transformation of the original text, not a rewrite**: every element that survives is- * copied out of the Source byte for byte, at the range the parse gave it. Nothing is authored here,- * so the fragment cannot describe something the Source does not.- *- * What decides survival is reference integrity, not distance from the element being looked at.- * Following inputs backwards from the target would keep what it depends on and drop everything else —- * and the Tracks that can collide with it on screen are exactly the ones it has no edge to. A caption- * sliced that way hangs over nothing, which is the shape of the defect this whole path exists to- * catch. So instead: keep the Segment, drop the rest of the Script, and then drop only what can no- * longer resolve — an element naming a Segment, Selection or Moment that is gone, and anything- * naming an element dropped for that reason, to a fixed point. A peer bound inside the kept Segment- * survives by construction, because its binding still resolves.- */--export type SliceResult = {- /** The fragment, verbatim from the original except for what was removed. */- readonly text: string;- /** Element ids left out, and the name each one could no longer reach. */- readonly dropped: readonly { readonly id: string; readonly unresolved: string }[];- readonly keptSegment: string;-};--type Element = {- readonly id: string | undefined;- readonly start: number;- readonly end: number;- readonly depth: number;- readonly text: string;-};--/**- * Every element in the document with its byte range, including children. A tag is either- * self-closing or closed by name; comments and the Script's own prose are skipped, since a `<` inside- * spoken words is not a tag.- */-function elements(source: string, skip: readonly (readonly [number, number])[]): Element[] {- const found: Element[] = [];- const open: { name: string; start: number; depth: number }[] = [];- const inside = (index: number): boolean => skip.some(([from, to]) => index >= from && index < to);- const pattern = /<(\/?)([a-z][a-z0-9-]*(?::[A-Za-z][A-Za-z0-9]*)?)\b([^>]*?)(\/?)>/gsu;- for (const match of source.matchAll(pattern)) {- const index = match.index;- if (inside(index)) continue;- const whole = match[0];- const closing = match[1]!;- const name = match[2]!;- const attributes = match[3]!;- const selfClosing = match[4]!;- if (closing === "/") {- const last = open.pop();- if (last !== undefined && last.name === name) {- found.push({- id: undefined, start: last.start, end: index + whole.length, depth: last.depth,- text: source.slice(last.start, index + whole.length),- });- }- continue;- }- if (selfClosing === "/") {- found.push({- id: /\bid="([^"]+)"/u.exec(attributes)?.[1], start: index, end: index + whole.length,- depth: open.length, text: whole,- });- continue;- }- open.push({ name, start: index, depth: open.length });- }- // A closed element's id has to be read back off its own opening tag.- return found.map((element) => element.id === undefined- ? { ...element, id: /^<[^>]*?\bid="([^"]+)"/su.exec(element.text)?.[1] }- : element);-}--/** The Script names an element reaches for, and the element ids it reaches for. */-function references(text: string): { readonly story: Set<string>; readonly ids: Set<string> } {- const story = new Set<string>();- const ids = new Set<string>();- // A Script name is reached through more than one space — `story.selection.x` in word space and- // `story.caption.selection.x` in caption space name the same Selection. Matching only the first- // leaves an element holding a dead name in caption space, and the cut looks clean until the graph- // refuses it.- for (const match of text.matchAll(/\{story\.(?:[a-z]+\.)*?(segment|selection|moment)\.([A-Za-z0-9_-]+)/gu)) {- story.add(`${match[1]!}.${match[2]!}`);- }- for (const match of text.matchAll(/\{([A-Za-z][A-Za-z0-9_-]*)[.}]/gu)) {- const id = match[1]!;- if (id !== "story") ids.add(id);- }- return { story, ids };-}--/**- * @param source the Author SVML- * @param segment the Segment to keep, named in the Script- */-export function sliceSource(source: string, segment: string): SliceResult {- const open = /<script\b[^>]*>/u.exec(source);- if (open === null) throw new Error("the Source declares no <script>");- const bodyStart = open.index + open[0].length;- const bodyEnd = source.indexOf("</script>", bodyStart);- if (bodyEnd < 0) throw new Error("the Source's <script> is not closed");-- // The Segment to keep, taken out of the Script whole. A Segment is one element in the Script body,- // so this is a range, and what it contains — its Selections, its Moments — comes with it.- const body = source.slice(bodyStart, bodyEnd);- const kept = new RegExp(`<${segment}\\b[^>]*>.*?</${segment}>`, "su").exec(body);- if (kept === null) throw new Error(`the Script has no <${segment}> Segment`);-- /**- * A Selection may open in one Segment and close in another — the Script says so and the parser- * records each end's own Segment. Cutting the Script to one Segment left the surviving end of such a- * Selection with no partner, and the fragment was refused outright with SCRIPT_SELECTION_UNCLOSED:- * not the element dropped, the whole render.- *- * The end that is missing is supplied at the fragment's own edge. What that expresses is exactly- * what the Selection covers inside this stretch — from its mark to the end of the Segment, or from- * the start of the Segment to its mark — which is what a render over this stretch has to show.- */- // The end of a name has to be asserted, not merely looked past: a lookahead alone backtracks, and- // `@title-out!` was read as an open called `title-ou` with a `t` after it.- const opened = new Set([...kept[0].matchAll(/(?<![\w/])@([a-z][a-z0-9-]*)(?![a-z0-9!-])/gu)].map((match) => match[1]!));- const closed = new Set([...kept[0].matchAll(/@\/([a-z][a-z0-9-]*)~?/gu)].map((match) => match[1]!));- const marked = new Set([...kept[0].matchAll(/@([a-z][a-z0-9-]*)!/gu)].map((match) => match[1]!));- let fragment = kept[0];- for (const id of opened) {- if (closed.has(id) || marked.has(id)) continue;- const end = fragment.lastIndexOf("</");- fragment = `${fragment.slice(0, end)} @/${id}\n ${fragment.slice(end)}`;- closed.add(id);- }- for (const id of closed) {- if (opened.has(id)) continue;- // After the Segment's own opening tag and the speaker tag that follows it, which is where the- // Segment's first spoken word begins.- const speaker = /^<[^>]*>\s*<[^>]*>/su.exec(fragment);- const at = speaker === null ? fragment.indexOf(">") + 1 : speaker[0].length;- fragment = `${fragment.slice(0, at)} @${id}${fragment.slice(at)}`;- opened.add(id);- }-- // Which Script names survive: the Segment itself, and every Selection and Moment marked inside it.- const alive = new Set<string>([`segment.${segment}`]);- for (const match of fragment.matchAll(/@\/?([a-z][a-z0-9-]*)\b/gu)) {- alive.add(`selection.${match[1]!}`);- alive.add(`moment.${match[1]!}`);- }-- // Which Script names are marked after the kept Segment ends.- //- // A name outside the cut is not always a name the element can do without. An element given the span- // it occupies and a Moment to cut it — `during="program" until={story.moment.done}` — is on screen- // for the whole of any stretch that ends before that word. Dropping it made the one element a- // reference is about unrenderable over every Segment but the last, while the round asks for it to be- // compared over each of them. Marked before the cut it really is gone, and marked inside it survives- // already, so this is the one case: a close that has not arrived yet, which the fragment reaches by- // no longer closing.- const later = new Set<string>();- for (const match of body.slice(kept.index + kept[0].length).matchAll(/@\/?([a-z][a-z0-9-]*)\b/gu)) {- later.add(`selection.${match[1]!}`);- later.add(`moment.${match[1]!}`);- }- /** The `until=` attribute naming a name the cut left behind, when dropping it is enough. */- const closesLater = (tag: string, name: string): string | undefined => {- if (!later.has(name)) return undefined;- const written = new RegExp(`\\suntil=\\{story\\.(?:[a-z]+\\.)*?${name.replace(".", "\\.")}\\}`, "u").exec(tag);- return written?.[0];- };-- const skip: (readonly [number, number])[] = [[bodyStart, bodyEnd]];- for (const comment of source.matchAll(/<!--.*?-->/gsu)) skip.push([comment.index, comment.index + comment[0].length]);- const all = elements(source, skip);-- // Drop to a fixed point: an element that names something gone is gone, and so is one that names it.- //- // What an element names is on its own opening tag. Reading its whole text instead makes a container- // inherit its children's references, so a Speech Track dies of one dead Take and takes the Film- // with it. Children are elements too and answer for themselves — which is also why the set is keyed- // by position: a `<speech:Take>` has no id, and still has to be droppable on its own.- const opening = (element: Element): string => /^<[^>]*>/su.exec(element.text)?.[0] ?? element.text;- const dropped = new Map<number, string>();- /** Opening tags rewritten by the cut, keyed by where the element starts. */- const reopened = new Map<number, string>();- const byId = new Map(all.flatMap((element) => element.id === undefined ? [] : [[element.id, element] as const]));- const deadIds = new Set<string>();- for (;;) {- let changed = false;- for (const element of all) {- if (dropped.has(element.start)) continue;- const tag = reopened.get(element.start) ?? opening(element);- const { story, ids } = references(tag);- const missingStory = [...story].find((name) => !alive.has(name));- if (missingStory !== undefined) {- // A close that has not arrived in this stretch is removed rather than fatal: the element keeps- // the span it was given and simply does not end inside the fragment.- const close = closesLater(tag, missingStory);- if (close !== undefined) {- reopened.set(element.start, tag.replace(close, ""));- changed = true;- continue;- }- dropped.set(element.start, `story.${missingStory}`);- if (element.id !== undefined) deadIds.add(element.id);- changed = true;- continue;- }- const missingId = [...ids].find((id) => deadIds.has(id) && byId.has(id));- if (missingId !== undefined) {- dropped.set(element.start, missingId);- if (element.id !== undefined) deadIds.add(element.id);- changed = true;- }- }- // A container emptied by the cut goes with its children. A Track written with Items is refused- // when it has none, and the refusal names the Track rather than the Segment that took its Items- // away, so leaving it in turns one clean cut into a graph error a reader has to trace back.- for (const element of all) {- if (dropped.has(element.start)) continue;- const children = all.filter((other) =>- other !== element && other.start > element.start && other.end <= element.end);- if (children.length === 0) continue;- if (children.some((child) => !dropped.has(child.start))) continue;- dropped.set(element.start, "every child was cut");- if (element.id !== undefined) deadIds.add(element.id);- changed = true;- }- if (!changed) break;- }-- // Rebuild: the Script body becomes the one Segment, and every dropped element's range is removed.- // Only the outermost dropped element is cut, since cutting a child of a cut parent removes text- // twice and leaves the document short of a closing tag.- const cuts = all- .filter((element) => dropped.has(element.start))- .filter((element, _, list) => !list.some((other) =>- other !== element && other.start <= element.start && other.end >= element.end- && !(other.start === element.start && other.end === element.end)))- .sort((left, right) => left.start - right.start);-- // A rewritten opening tag replaces its own bytes, in the same pass and in the same order as a cut,- // so the two never disagree about where the cursor is. A tag inside a range being cut is not- // reached: the cut takes it with everything else.- const edits = [- ...cuts.map((element) => ({ start: element.start, end: element.end, text: undefined as string | undefined })),- ...[...reopened].flatMap(([start, tag]) => {- const element = all.find((candidate) => candidate.start === start);- if (element === undefined || dropped.has(start)) return [];- const length = (/^<[^>]*>/su.exec(element.text)?.[0] ?? element.text).length;- return [{ start, end: start + length, text: tag }];- }),- ].sort((left, right) => left.start - right.start);-- let text = "";- let cursor = 0;- for (const edit of edits) {- if (edit.start < cursor) continue;- text += source.slice(cursor, edit.start);- if (edit.text !== undefined) {- text += edit.text;- cursor = edit.end;- continue;- }- cursor = edit.end;- // Take the rest of the line with it, so a removal never leaves a blank indented line behind.- while (source[cursor] === " " || source[cursor] === "\t") cursor += 1;- if (source[cursor] === "\n") cursor += 1;- }- text += source.slice(cursor);-- // The Script body is replaced last, in the rebuilt text, since every range above indexes the- // original. Its content is unique enough to find again.- const rebuiltOpen = /<script\b[^>]*>/u.exec(text);- if (rebuiltOpen === null) throw new Error("the Script was removed by the cut");- const rebuiltStart = rebuiltOpen.index + rebuiltOpen[0].length;- const rebuiltEnd = text.indexOf("</script>", rebuiltStart);- text = `${text.slice(0, rebuiltStart)}\n ${fragment}\n ${text.slice(rebuiltEnd)}`;-- return {- text,- dropped: all.flatMap((element) => dropped.has(element.start)- ? [{ id: element.id ?? opening(element).slice(0, 40), unresolved: dropped.get(element.start)! }]- : []),- keptSegment: segment,- };-}diff --git a/packages/reference-video-tools/src/tools.ts b/packages/reference-video-tools/src/tools.tsindex 364a8bc1..56673881 100644--- a/packages/reference-video-tools/src/tools.ts+++ b/packages/reference-video-tools/src/tools.ts@@ -81,7 +81,9 @@ export type CompareReconstructionInput = {readonly shot_id?: string;readonly segment?: string;readonly selection?: string;- /** The Run whose Author SVML the words are read from. Required with `segment` or `selection`. */+ /** A half-open token range, for a stretch the Script never named. See `StandInFocus`. */+ readonly tokens?: readonly [number, number];+ /** The Run whose Author SVML the words are read from. Required with a word range. */readonly run?: string;/** A rendered still. Exactly one of `image_path` and `video_path` is given. */readonly image_path?: string;@@ -130,6 +132,8 @@ export type ReviewElementInput = {/** The window the render covers, named in words. Exactly one is given. */readonly segment?: string;readonly selection?: string;+ /** A half-open token range, for a stretch the Script never named. See `StandInFocus`. */+ readonly tokens?: readonly [number, number];/** The render. Exactly one of `image_path` and `video_path` is given. */readonly image_path?: string;readonly video_path?: string;@@ -234,6 +238,8 @@ function stateRoot(reference: string): string {export type ComparedRange = {readonly segment?: string;readonly selection?: string;+ /** The word range, when the caller gave one directly rather than naming a marked stretch. */+ readonly tokens?: readonly [number, number];readonly words: string;readonly words_start_seconds: number;readonly words_end_seconds: number;@@ -293,7 +299,7 @@ export type ReviewRecord = {* reference speaks those words and how the cut was snapped to its shot boundaries, and none of that* exists here. The gate reads the name and nothing else from either log.*/- readonly range?: { readonly segment?: string; readonly selection?: string };+ readonly range?: { readonly segment?: string; readonly selection?: string; readonly tokens?: readonly [number, number] };readonly image_path: string;readonly image_digest: string;readonly status: Observation["status"];@@ -389,12 +395,19 @@ async function answeredAlready(return undefined;}-/** What a look was made over, named the way it was asked for. */+/**+ * What a look was made over, in the form it was asked for.+ *+ * A word range is written as its own form because the finest thing worth looking at is not always a+ * thing the Script named — a caption Cue is a run of words with no id — and a gate comparing a round+ * against a plan has to be able to tell one of those from another.+ */function comparedStretch(record: {readonly shot_id?: string;- readonly range?: { readonly segment?: string; readonly selection?: string };+ readonly range?: { readonly segment?: string; readonly selection?: string; readonly tokens?: readonly [number, number] };}): string {return record.shot_id+ ?? (record.range?.tokens === undefined ? undefined : `tokens:${record.range.tokens[0]}-${record.range.tokens[1]}`)?? (record.range?.segment === undefined ? undefined : `segment:${record.range.segment}`)?? (record.range?.selection === undefined ? undefined : `selection:${record.range.selection}`)?? "";@@ -849,6 +862,7 @@ async function cutWordRange(record: {...(focus.segment === undefined ? {} : { segment: focus.segment }),...(focus.selection === undefined ? {} : { selection: focus.selection }),+ ...(focus.tokens === undefined ? {} : { tokens: focus.tokens }),words: `${range.first.text} … ${range.last.text}`,words_start_seconds: round(range.startSeconds),words_end_seconds: round(range.endSeconds),@@ -1506,8 +1520,8 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceV}const state = await loadState(input.reference_id);const root = stateRoot(state.reference_id);- const named = [input.shot_id, input.segment, input.selection].filter((value) => value !== undefined);- assert(named.length === 1, "name exactly one of shot_id, segment and selection");+ const named = [input.shot_id, input.segment, input.selection, input.tokens].filter((value) => value !== undefined);+ assert(named.length === 1, "name exactly one of shot_id, segment, selection and tokens");const supplied = [input.image_path, input.video_path].filter((value) => value !== undefined);assert(supplied.length === 1, "supply exactly one of image_path and video_path");const clip = input.video_path !== undefined;@@ -1540,11 +1554,12 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceVstretchSeconds = shot.duration_seconds;} else {assert(input.run !== undefined,- "run is required with segment or selection: the words are read from the Run's Author SVML");+ "run is required with a word range: the words are read from the Run's Author SVML");const { path: svmlPath } = await authorSource(resolve(invokedFrom(), input.run));const focus: StandInFocus = {...(input.segment === undefined ? {} : { segment: input.segment }),...(input.selection === undefined ? {} : { selection: input.selection }),+ ...(input.tokens === undefined ? {} : { tokens: input.tokens }),};const range = await spokenRange(svmlPath, focus, await referenceWords(state.reference_id));cut = await cutWordRange(state, root, focus, range, renderedPath, clip, slot);@@ -1783,8 +1798,8 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceVconst element = input.element?.trim() ?? "";assert(element.length > 0, "--element is required; a review with no element is credited to nothing");- const named = [input.segment, input.selection].filter((value) => value !== undefined);- assert(named.length === 1, "name exactly one of --segment or --selection: the window the render covers");+ const named = [input.segment, input.selection, input.tokens].filter((value) => value !== undefined);+ assert(named.length === 1, "name exactly one of --segment, --selection or --tokens: the window the render covers");const supplied = [input.image_path, input.video_path].filter((value) => value !== undefined);assert(supplied.length === 1, "supply exactly one of --image or --video");@@ -1818,7 +1833,12 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceV}const digest = await fileDigest(renderedPath);- const stretch = comparedStretch({ range: { ...(input.segment === undefined ? {} : { segment: input.segment }), ...(input.selection === undefined ? {} : { selection: input.selection }) } });+ const range = {+ ...(input.segment === undefined ? {} : { segment: input.segment }),+ ...(input.selection === undefined ? {} : { selection: input.selection }),+ ...(input.tokens === undefined ? {} : { tokens: input.tokens }),+ };+ const stretch = comparedStretch({ range });// Nothing about the picture or the question changed since this was last answered, so nothing new// can be said about it. Here this is what bounds the looking: a review costs no vendor money, so// without it a round could re-ask the same picture until it liked the answer.@@ -1836,7 +1856,7 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceVconst unit = asClip ? "grid of frames sampled evenly across the stretch, laid out in reading order: left to right, then top to bottom" : "still picture";const record: ReviewRecord = {at: startedAt, id: reviewId, element,- range: { ...(input.segment === undefined ? {} : { segment: input.segment }), ...(input.selection === undefined ? {} : { selection: input.selection }) },+ range,image_path: renderedPath, image_digest: digest, status: "pending",...(scope.length === 0 ? {} : { scope }), clip: asClip,...(standIn === undefined ? {} : { stand_in: standIn }),diff --git a/packages/reference-video-tools/test/plan.test.ts b/packages/reference-video-tools/test/plan.test.tsnew file mode 100644index 00000000..0a9fabd4--- /dev/null+++ b/packages/reference-video-tools/test/plan.test.ts@@ -0,0 +1,84 @@+import assert from "node:assert/strict";+import test from "node:test";++import { scriptBody } from "../src/authoring.js";+import { reviewPlan } from "../src/plan.js";++const SVML = `<?svml using="@hypit/markup@1"?>+<svml version="1">+ <import as="story" from="@hypit/script@1"/>+ <import as="media-track" from="@hypit/media-track@1"/>+ <script id="story">+ <one><HOST>alpha bravo @claim charlie delta @/claim echo.</one>+ <two><HOST>foxtrot @proof golf hotel @/proof india.</two>+ </script>+ <media-track:Track id="broll" semantic={speech.semantic} canvas={vertical}>+ <media-track:Item id="one" media={a.media} during={story.selection.claim}+ frame={card-a} appearance={recipes.media.card}/>+ <media-track:Item id="two" media={b.media} during={story.selection.proof}+ frame={card-b} appearance={recipes.media.wide}/>+ </media-track:Track>+</svml>+`;++const BROLL = [{ id: "broll", tag: "Track", alias: "media-track", specifier: "@hypit/media-track" }];++function planFor(svml: string) {+ return reviewPlan({ svml, svmlPath: "main.svml", scriptBody: scriptBody(svml), drawn: BROLL });+}++// Under-covering is the silent direction: the gate passes, the round runs, and the second design is+// simply never drawn. The two exclusion lists this key is built from will grow as packages are added,+// and widening one by an attribute that actually decides what a picture looks like merges two groups+// into one. That edit has no visible symptom, so it is the one worth pinning.+test("two placements that declare different pictures are two looks, and two that declare the same are one", () => {+ const distinct = planFor(SVML);+ assert.equal(distinct.length, 2, "a different appearance and Frame is a different picture");+ assert.deepEqual(distinct.map((entry) => entry.named), ["selection claim", "selection proof"]);++ // Only the declaration differs between these two runs. The windows, the ids and the media stay put.+ const same = planFor(SVML+ .replace("appearance={recipes.media.wide}", "appearance={recipes.media.card}")+ .replace("frame={card-b}", "frame={card-a}"));+ assert.equal(same.length, 1, "the same declaration over different words is one design, read once");++ // The material each Item names is mocked in the render this plan is for, so two Items differing+ // only in which unbuilt picture they point at draw the identical placeholder.+ const mocked = planFor(SVML+ .replace("appearance={recipes.media.wide}", "appearance={recipes.media.card}")+ .replace("frame={card-b}", "frame={card-a}")+ .replace("media={b.media}", "media={c.media}"));+ assert.equal(mocked.length, 1, "an unbuilt picture is a grey rectangle whichever generation names it");+});++const CAPTIONS = `<?svml using="@hypit/markup@1"?>+<svml version="1">+ <import as="story" from="@hypit/script@1"/>+ <import as="caption" from="@hypit/caption@1"/>+ <import as="caption-fine" from="@hypit/caption-fine@1"/>+ <script id="story">+ <one><HOST>alpha bravo charlie delta echo.</one>+ <two><HOST>foxtrot @punch golf hotel @/punch india.</two>+ </script>+ <caption-fine:Style id="plain" recipe={recipes.caption.plain}/>+ <caption-fine:Style id="loud" recipe={recipes.caption.loud}/>+ <caption:Program id="program" document={story.caption} narrative={story} default={plain}>+ <caption:Use style={loud} selection={story.selection.punch}/>+ </caption:Program>+ <caption-fine:Track id="captions" document={story.caption} semantic={speech.semantic} program={program}/>+</svml>+`;++// A caption Track is one tag with no window on it, so reading its placements the ordinary way finds+// one design however many the Program hands out. An override is written inside the Program, and the+// stretch it covers is a genuinely different picture that nothing on the Track mentions — so leaving+// it out is a design nobody looks at while the gate reports full coverage.+test("a caption Style override is its own look, over the stretch it covers", () => {+ const plan = reviewPlan({+ svml: CAPTIONS, svmlPath: "main.svml", scriptBody: scriptBody(CAPTIONS),+ drawn: [{ id: "captions", tag: "Track", alias: "caption-fine", specifier: "@hypit/caption-fine" }],+ });+ const overridden = plan.find((entry) => entry.named === "selection punch");+ assert.ok(overridden, `the overridden stretch is missing; got ${JSON.stringify(plan.map((e) => e.named))}`);+ assert.ok(plan.length >= 2, "the default and the override are two designs, not one");+});