Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

feat: the check names the round, and the program is drawn once for all of it (#87)

rponeawa156ea2807380
diff --git a/.agents/skills/hypit/references/element-review.md b/.agents/skills/hypit/references/element-review.md
index b70e23e1..c565199a 100644
--- a/.agents/skills/hypit/references/element-review.md
+++ b/.agents/skills/hypit/references/element-review.md
@@ -25,9 +25,26 @@ the target that does not trace, the chain that is missing — so its repairs are
names what is visibly wrong with the picture, in words, and a repair aims at something the round
named. Neither gate expects you to guess; each one tells you what is wrong, and you repair that.
-## The round covers one element at a time, across every stretch it is drawn over
+## The round is named by the check, not chosen by you
-Loop over one authored element at a time — the new component, one caption system, one inserted card.
+`authoring_check` and `reconstruction_check` read the Source and return a `plan`: every stretch worth
+looking at, as an element and a half-open range of the Script's own words, with the reason it earned a
+place. Render that list and read that list. Nothing here asks you to decide which stretches matter.
+
+Three rules produce it, and all three are read off the Source. **A distinct declaration is a distinct
+picture**, so an element is looked at once per way it is declared, at the stretch where that way first
+appears — which is also the floor, since every placed element declares something. **Content that can
+break the layout earns its own look**, which is why a caption Style is read over its longest Cue rather
+than its shortest. **A window length earns one only when something scales with it** — an enter costs
+its frames at one edge whatever the window is, but a typewriter, a loop or a `stretch` playback runs
+for as long as the window does.
+
+A range rather than a name, because the answer is not always a name: a Cue ends at a speaker change, so
+it is a run of words the Script never marked. `--tokens from:to` is how every command takes one, and
+`--segment`/`--selection` remain for the ranges that do have names.
+
+The gate then holds the round to that list. An element looked at once used to pass; now every entry the
+plan names has to have been answered.
### Render what the Source configures, not what the catalogue shows
@@ -75,27 +92,16 @@ saves the mock coming back as a finding in the one round there is.
A mock lives only in this render. It is never written into the Source, and no gate reads it: the
`playback` check reads the Source's Recipes, so a mock cannot be mistaken for coverage.
-### The derived Run holds one Segment, not the whole Source
-
-The mocks are substituted into a Source that has already been cut to the window. `render_element`
-keeps the Segment the window names, drops the rest of the Script, and then drops every element that
-named a Segment, Selection or Moment that went with it — and every element naming one of those, to a
-fixed point, plus any container the cut left with no children. So an element bound to a Selection in
-another Segment is not in this render, and its absence says nothing about the Source.
-
-One name survives the cut rather than taking its element with it: a Moment in `until=` that is marked
-*after* the kept Segment. An element given the span it occupies and a Moment to close it is on screen
-for the whole of a stretch that ends before that word, so the cut removes the close and keeps the
-element. That is what makes an element bounded by a late Moment renderable over the earlier Segments
-it is drawn over, which is every stretch this round asks for it in.
-
-The cut Source is on disk at `<project>/.hypit/compare/sliced.svml`, byte-for-byte from the original
-except for what was removed, and the derived Run sits beside it. Open it whenever something you
-expected to see is not in the picture: an element that is not in the fragment was cut by the window,
-and one that is in the fragment and still not on screen is the package failing to draw it. Those two
-repair in opposite directions, and the fragment is what tells them apart.
-`packages/reference-video-tools/src/slice.ts` records the same thing as `SliceResult.dropped` — each
-dropped id and the name it could no longer reach — and explains what decides survival.
+### The derived Run carries the mocks, and nothing else changes
+
+The mocks are substituted into a Run that names the project's own Source. The whole program is drawn
+and the window is cut out of the frames afterwards, so every element the Source places is present and
+drawn wherever the Source puts it — what is on screen over the words being looked at is what the
+delivery will put there.
+
+That derived Run is on disk under `<project>/.hypit/compare/`, beside the frames it produced. Open it
+whenever something you expected to see is not in the picture: an element that is in the Source and
+still not on screen is the package failing to draw it.
### A caption Style is read once, where it first appears
diff --git a/.agents/skills/hypit/references/original-authoring/route.md b/.agents/skills/hypit/references/original-authoring/route.md
index 5b08c4da..3685e6a7 100644
--- a/.agents/skills/hypit/references/original-authoring/route.md
+++ b/.agents/skills/hypit/references/original-authoring/route.md
@@ -89,10 +89,17 @@ reused in the Run Source. `../runtime.md` — how to author a Profile.
tracing. `../preview.md` proves the Run traces before a Provider is reached. Neither is bounded by any
attempt ceiling: a graph that does not trace is work that is not done.
-### 9. Render every stretch of every element
+### 9. Ask the check what to render, then render it
-**Read now:** `../element-review.md` — the round unit, what a render has to include, and how many
-stretches to draw. `../preview.md` holds `render_element` itself.
+```bash
+hypit-reference-video-tools authoring_check projects/<name>/build.svrun
+```
+
+Its `plan` is the list: each entry an element and a word range, with the reason. Feed it to
+`render_element --batch` — the program is drawn once and every entry is cut out of those frames.
+
+**Read now:** `../element-review.md` — what the three rules are and why the list is what it is.
+`../preview.md` holds `render_element` itself.
Render the whole set before reading any of it. Studio is also free and is for the author to look at;
it is not the picture this round reads.
diff --git a/.agents/skills/hypit/references/preview.md b/.agents/skills/hypit/references/preview.md
index 7aa1f918..2eb966c4 100644
--- a/.agents/skills/hypit/references/preview.md
+++ b/.agents/skills/hypit/references/preview.md
@@ -80,12 +80,17 @@ command appends the output suffix itself when it looks for the track, so `--elem
matches nothing: it refuses with `the Source places no element named captions.track` and lists the
bare ids that are placed.
-`--segment` and `--selection` take a Script name, never a timestamp. Without either, the whole program
-is drawn. An `--out` ending `.mp4`, `.mov` or `.webm` writes the stretch as a clip; any other extension
-writes one still from the middle of it.
-
-A whole round is one call: `--batch <renders.json>`, an array of `{element, segment|selection, out}`
-inheriting the Run. They run together, paced by the machine.
+`--segment` and `--selection` take a Script name, never a timestamp. `--tokens from:to` takes a
+half-open range of the Script's own words, which is what a stretch nobody named is written as — a
+caption Cue ends at a speaker change, so it has a range and no id. Without any of the three, the whole
+program is drawn. An `--out` ending `.mp4`, `.mov` or `.webm` writes the stretch as a clip; any other
+extension writes one still from the middle of it.
+
+A whole round is one call: `--batch <renders.json>`, an array of
+`{element, segment|selection|tokens, out}` inheriting the Run. **The program is drawn once for the
+whole round** and each entry is cut out of those frames, so asking for eight windows costs one render
+and eight cuts. The picture does not depend on `--element` — it is everything the Source places over
+those words — so two elements over one stretch share the render as well.
Each Segment's length comes from the Source's own `estimate:Speech`, which is the only clock a program
has before its speech is synthesized. The result's `timing` says what sized each Segment, and the same
diff --git a/.agents/skills/hypit/references/reconstruction/route.md b/.agents/skills/hypit/references/reconstruction/route.md
index 91f0a66d..67eb76d2 100644
--- a/.agents/skills/hypit/references/reconstruction/route.md
+++ b/.agents/skills/hypit/references/reconstruction/route.md
@@ -252,10 +252,14 @@ first written. Neither this gate nor step 16 is bounded by the round's attempt c
**Read now:** `../element-review.md`, then `comparison-round.md`, as soon as the sources place an
element — that is the first moment the round has something to render.
-### 19. Render every stretch of every authored element
+### 19. Ask the check what to render, then render it
+
+`reconstruction_check` returns a `plan`: each entry an element and a word range, with the reason it
+earned a look. That is the round — it is not yours to work out from the Source.
`../preview.md` holds `render_element`; `comparison-round.md` says to pass `--reference-id` every time
-so the stand-in runs on the reference's clock. Render the whole set before comparing any of it.
+so the stand-in runs on the reference's clock. The program is drawn once and every entry is cut out of
+those frames, so render the whole list in one `--batch` call.
### 20. Send every comparison for the element at once
diff --git a/packages/reference-video-tools/src/authoring.ts b/packages/reference-video-tools/src/authoring.ts
index f144566e..0dc55964 100644
--- a/packages/reference-video-tools/src/authoring.ts
+++ b/packages/reference-video-tools/src/authoring.ts
@@ -3,8 +3,6 @@ import { existsSync } from "node:fs";
import { spawnSync } from "node:child_process";
import { writePlaceholder } from "./placeholder.js";
-import { sliceSource } from "./slice.js";
-import type { SliceResult } from "./slice.js";
import { createHash } from "node:crypto";
import { createRequire } from "node:module";
import { cpus } from "node:os";
@@ -80,12 +78,28 @@ export type StandInTakes = {
readonly selections: ReadonlyMap<string, AuthoringWindow>;
readonly frameCount: number;
readonly timing: readonly StandInTiming[];
+ /**
+ * Where every Script word sits in the program, in order.
+ *
+ * A window named by a Segment or a Selection is looked up above; one named by index is read
+ * straight off this, which is the same table those two are built from.
+ */
+ readonly frameOfToken: readonly { readonly frame: number; readonly end: number }[];
};
/** Which Segment the render is looking at, named directly or through a Selection's words. */
export type StandInFocus = {
readonly segment?: string;
readonly selection?: string;
+ /**
+ * A half-open range of the Script's own speech tokens.
+ *
+ * The finest thing worth looking at is not always a thing the Script named. A caption Cue is a run
+ * of words ended by a speaker change or an authored break, and the planner picks it out by index —
+ * so the index is what both sides take. `segment` and `selection` remain as the convenient way to
+ * write down a range that does have a name.
+ */
+ readonly tokens?: readonly [number, number];
};
/**
@@ -277,24 +291,6 @@ const SILENT_AUDIO: BlobRef = { kind: "blob", digest: `sha256:${"0".repeat(64)}`
* @param reference the reference's per-word times, from `referenceWords`
* @returns one `{ segmentId, take }` per Segment in Script order, and what timed each of them
*/
-/** The Segment a Selection is marked in, so a window named by Selection can still name the cut. */
-function segmentOfSelection(svml: string, selection: string | undefined): string | undefined {
- if (selection === undefined) return undefined;
- const open = /<script\b[^>]*>/u.exec(svml);
- if (open === null) return undefined;
- const start = open.index + open[0].length;
- const end = svml.indexOf("</script>", start);
- if (end < 0) return undefined;
- const body = svml.slice(start, end);
- const at = body.indexOf(`@${selection}`);
- if (at < 0) return undefined;
- let found: string | undefined;
- for (const match of body.matchAll(/<([a-z][a-z0-9-]*)\b[^>]*>/gu)) {
- if (match.index > at) break;
- if (body.indexOf(`</${match[1]!}>`, match.index) > at) found = match[1]!;
- }
- return found;
-}
/**
* The Author SVML a Run declares, both as it is written and as a path.
@@ -495,15 +491,22 @@ export async function spokenRange(
// Which Script words the range covers. A Selection is marked once, so its word range is the
// tokens between the anchors its open and close markers name.
+ //
+ // A range given by index skips the lookup entirely: it is already the answer the other two forms
+ // are resolved into, and it is the only form that can name a stretch the Script never marked.
let from: number | undefined;
let to: number | undefined;
- if (focus.selection !== undefined) {
+ if (focus.tokens !== undefined) {
+ [from, to] = focus.tokens;
+ assert(from >= 0 && to > from && to <= parsed.tokens.length,
+ `token range [${from}, ${to}) is outside the Script's ${parsed.tokens.length} words`);
+ } else if (focus.selection !== undefined) {
const selection = parsed.selections.find((item) => item.id === focus.selection);
assert(selection !== undefined, `the Script marks no Selection ${focus.selection}`);
from = selection.open.boundary.tokenIndex;
to = selection.close.boundary.tokenIndex;
} else {
- assert(focus.segment !== undefined, "name a Segment or a Selection to read a word range from");
+ assert(focus.segment !== undefined, "name a Segment, a Selection or a token range to read a word range from");
const segment = parsed.segments.find((item) => item.id === focus.segment);
assert(segment !== undefined, `the Script has no Segment ${focus.segment}`);
from = segment.tokenStart;
@@ -582,20 +585,6 @@ export async function standInTakes(
const sheets = await recipeSheets(svml, svmlPath);
const { policies, shared, policyCount } = policiesBySegment(svml, sheets);
- // Which Segment the render is actually looking at. Every other Segment still has to exist — a
- // Speech Track assembles one Take per Segment and an unsatisfied one refuses the whole projection —
- // but nothing needs it at its estimated length. Held to one frame per word it stays legal, keeps
- // its anchors, and stops the renderer drawing a minute of program to show six seconds of it.
- let focused: string | undefined;
- if (focus.segment !== undefined) focused = focus.segment;
- else if (focus.selection !== undefined) {
- const selection = parsed.selections.find((item) => item.id === focus.selection);
- const token = selection?.open.boundary.tokenIndex;
- focused = token === undefined
- ? undefined
- : parsed.segments.find((segment) => token >= segment.tokenStart && token < segment.tokenEndExclusive)?.id;
- }
-
// Which Script word the reference speaks when. The Script was transcribed from that video, so the
// words are the same words in the same order, and a stand-in timed from them runs at the pace the
// reconstruction is being compared against.
@@ -648,9 +637,10 @@ export async function standInTakes(
weights = tokens.map((token) => Math.max(1, countSpeechEstimateUnits(token.text, language)));
}
- const frameCount = focused !== undefined && segment.id !== focused
- ? Math.max(1, tokens.length)
- : Math.max(tokens.length, Math.round(seconds * frameRate));
+ // Every Segment at the length its clock gives it. The program is drawn once and a window is cut
+ // out of it by frame, so a Segment nobody is looking at costs nothing by being its true length —
+ // and the frame numbers then mean what they say instead of indexing a compressed timeline.
+ const frameCount = Math.max(tokens.length, Math.round(seconds * frameRate));
timing.push({ segment: segment.id, basis, seconds: round(seconds), frames: frameCount, tokens: tokens.length, matched });
// Where each word ends. On the reference clock a word holds the screen until the next one starts
@@ -719,7 +709,7 @@ export async function standInTakes(
if (first === undefined || last === undefined) continue;
if (first.frame < last.end) selections.set(selection.id, { startFrame: first.frame, endFrameExclusive: last.end });
}
- return { takes, selections, frameCount: frameCursor, timing };
+ return { takes, selections, frameCount: frameCursor, timing, frameOfToken };
}
function blobRef(bytes: Uint8Array, mediaType: string): BlobRef {
@@ -745,6 +735,74 @@ function silentWav(sampleFrames: number): Buffer {
return buffer;
}
+/**
+ * One browser render per Run and clock, within this process.
+ *
+ * A round asks for several windows of one program, and each of them would otherwise draw the whole
+ * program again to throw most of it away. Keyed by the same hash the working directory is, so two
+ * entries that share a directory share the render that fills it. Between processes this holds
+ * nothing, which is correct: a new process is the one case where the Source may have changed.
+ */
+const programRenders = new Map<string, Promise<void>>();
+async function drawProgramOnce(key: string, draw: () => Promise<void>): Promise<void> {
+ const running = programRenders.get(key);
+ if (running !== undefined) return await running;
+ const started = draw();
+ programRenders.set(key, started);
+ try { await started; } catch (error) { programRenders.delete(key); throw error; }
+}
+
+/**
+ * How big a mock for one declared generation should be.
+ *
+ * The shape matters and the pixel count does not. A mock is a flat fill, so nothing reads its
+ * resolution — but a `fit: cover` crops a 4:5 source into a 9:16 Frame differently from a 9:16 one,
+ * and a mock of the wrong shape makes the reader report a framing difference the real render will
+ * not have, or miss one it will.
+ *
+ * The generation declares its own `aspect-ratio`, so that is what is read. Where it says `adaptive`
+ * — seedance's "inherit from the input" — the shape comes from whatever it was handed, so the edge
+ * is followed one hop to the upstream generation. Beyond one hop the returns fall away and a
+ * malformed or circular graph starts to matter, so the Canvas is the answer from there.
+ *
+ * `resolution` is deliberately not read. It is a vendor token (`720p` naming a short side, `4K` a
+ * long one) with no pixel mapping anywhere in this repository, and it decides only how large a
+ * rectangle of flat colour would be.
+ */
+function mockExtent(svml: string, id: string, canvas: { readonly width: number; readonly height: number }): {
+ readonly width: number;
+ readonly height: number;
+} {
+ const declaring = (target: string): string | undefined =>
+ new RegExp(`<[a-z][a-z0-9-]*:[A-Za-z][A-Za-z0-9]*\\b[^>]*?\\bid="${target}"[^>]*?/?>`, "su").exec(svml)?.[0];
+
+ const ratioOf = (target: string): string | undefined => {
+ const tag = declaring(target);
+ if (tag === undefined) return undefined;
+ const declared = /\baspect-ratio="([^"]+)"/u.exec(tag)?.[1];
+ if (declared !== undefined && declared !== "adaptive") return declared;
+ // One hop upstream. Two shapes need it: `adaptive`, which takes the shape of whatever it was
+ // handed, and a `pipeline:Normalize`, which is what a mock is actually keyed by — the Source
+ // names the normalized media, and the generation that declared the shape is behind its `source`.
+ const upstream = /\b(?:source|image|first-frame|reference-image)=\{([A-Za-z0-9_-]+)[.}]/u.exec(tag)?.[1];
+ if (upstream === undefined) return undefined;
+ const inherited = /\baspect-ratio="([^"]+)"/u.exec(declaring(upstream) ?? "")?.[1];
+ return inherited === "adaptive" ? undefined : inherited;
+ };
+
+ const ratio = ratioOf(id);
+ const parts = /^(\d+):(\d+)$/u.exec(ratio ?? "");
+ if (parts === null) return { width: canvas.width, height: canvas.height };
+ const wide = Number(parts[1]);
+ const tall = Number(parts[2]);
+ if (!(wide > 0 && tall > 0)) return { width: canvas.width, height: canvas.height };
+ // Scaled off the Canvas's short side, so a mock is never larger than the picture it sits in.
+ const short = Math.min(canvas.width, canvas.height);
+ return wide >= tall
+ ? { width: Math.round(short * wide / tall), height: short }
+ : { width: short, height: Math.round(short * tall / wide) };
+}
+
export type RenderElementInput = {
/**
* A round of renders, run together. Each entry names its own element, stretch and output and
@@ -758,6 +816,8 @@ export type RenderElementInput = {
readonly out?: string;
readonly segment?: string;
readonly selection?: string;
+ /** A half-open token range, for a stretch the Script never named. See `StandInFocus`. */
+ readonly tokens?: readonly [number, number];
/**
* A prepared reference to time the stand-in from. Its transcript holds the seconds each word was
* spoken at, and the Script was transcribed from that video, so the stand-in runs at the pace the
@@ -890,6 +950,7 @@ export async function renderElement(input: RenderElementInput): Promise<Record<s
// difference between a few seconds and a few minutes.
const segmentArgument = input.segment;
const selectionArgument = input.selection;
+ const tokensArgument = input.tokens;
const focus = {
...(segmentArgument === undefined ? {} : { segment: segmentArgument }),
...(selectionArgument === undefined ? {} : { selection: selectionArgument }),
@@ -901,45 +962,28 @@ export async function renderElement(input: RenderElementInput): Promise<Record<s
// deleted the first one's sliced Source and mocks out from under it. The route asks for every
// element to be rendered before any is compared, which is a set of renders with nothing to say to
// each other, and they can now run together.
+ //
+ // The key is the Run and the clock, and deliberately not the element or the window: the picture
+ // does not depend on either. `render_element` draws the whole program and the window is cut out of
+ // the frames afterwards, so every render of one Run under one clock wants the identical directory,
+ // the identical mocks and the identical frames.
const renderKey = createHash("sha256")
- .update(`${resolve(out)}\u0000${element}\u0000${focus.segment ?? focus.selection ?? ""}`)
+ .update(`${resolve(runPath)}\u0000${input.reference_id ?? ""}`)
.digest("hex").slice(0, 12);
const compareRoot = join(projectRoot, ".hypit", "compare", renderKey);
- await rm(compareRoot, { recursive: true, force: true });
+ // Made, not remade. The frames inside are reused by every window cut out of this program, and the
+ // mocks beside them are written from ids that do not move, so emptying it would only throw away the
+ // render the next entry of the round is about to ask for.
await mkdir(compareRoot, { recursive: true });
- // With a window named, the Source is cut down to it before anything else runs. Every Segment
- // outside it would otherwise still be drawn — held to one frame per word, which is the shortest a
- // Take can legally be, and still nearly half the frames. The cut is byte-for-byte from the
- // original, so the fragment cannot say anything the Source does not; `slice.ts` explains what
- // decides which elements come with it.
- let sourcePath = svmlPath;
- let sliced: SliceResult | undefined;
- const focusedSegment = focus.segment ?? segmentOfSelection(svml, focus.selection);
- if (focusedSegment !== undefined) {
- sliced = sliceSource(svml, focusedSegment);
- sourcePath = join(compareRoot, "sliced.svml");
- // The fragment sits three directories below the Source it came from, so its own relative paths
- // have to reach back the same distance the derived Run's carried files do.
- //
- // `src` belongs in this list beside the import attributes. It is how the author's own supplied
- // material is named — a presenter still, a voice sample — and left alone it resolved against the
- // fragment's directory, where there is no `assets/`. Every Source carrying a `media:Image` with a
- // relative path failed to render at all, on either route, the moment a window was named.
- const repointed = sliced.text.replace(/(\s(?:source|from|src)=")\.\//gu, "$1../../../");
- await writeFile(sourcePath, repointed, "utf8");
- // Everything downstream reads the Source: which Takes to stand in for, which media to mock, which
- // outputs to satisfy. Left on the original it would declare mocks for elements the cut removed,
- // and a `<satisfy>` naming an output that no longer exists is refused.
- svml = repointed;
- }
+ const sourcePath = svmlPath;
// The reference's per-word times, when one was named. Read after the cut so a transcript that is
// missing is reported before a minute of rendering rather than after it.
const reference = input.reference_id?.trim();
const spoken = reference === undefined || reference.length === 0 ? [] : await referenceWords(reference);
- const { takes, selections, frameCount: programFrames, timing } = await standInTakes(sourcePath, frameRate, focus, spoken, svmlPath);
+ const { takes, selections, frameCount: programFrames, timing, frameOfToken } = await standInTakes(sourcePath, frameRate, focus, spoken, svmlPath);
const framesBySegment = new Map(takes.map((item) => [item.segmentId, item.take.segment.endFrameExclusive]));
// A Selection's window in frames, summed over the stand-in tokens it covers. This is the same word
// span the Source binds to, carried into frames by the same clock that sized the Segment.
@@ -1022,7 +1066,7 @@ export async function renderElement(input: RenderElementInput): Promise<Record<s
await Promise.all(mocks
.filter(([, { picture }]) => picture)
.map(async ([id, { frames }]) => await writePlaceholder({
- out: join(compareRoot, `${id}.mp4`), width: canvas.width, height: canvas.height,
+ out: join(compareRoot, `${id}.mp4`), ...mockExtent(svml, id, canvas),
video: true, seconds: Math.max(1, Math.ceil(frames / frameRate)), color: "mid",
})));
for (const [id, { frames, picture }] of mocks) {
@@ -1038,7 +1082,7 @@ export async function renderElement(input: RenderElementInput): Promise<Record<s
let visual: SynchronizedMedia["visual"];
if (picture) {
const bytes = await readFile(file);
- visual = { artifact: blobRef(bytes, "video/mp4"), width: canvas.width, height: canvas.height };
+ visual = { artifact: blobRef(bytes, "video/mp4"), ...mockExtent(svml, id, canvas) };
declarations.push(` <file id="mock-${id}" type="@hypit/artifact@1#BlobArtifact" from="./${id}.mp4" media-type="video/mp4"/>`);
}
const media: SynchronizedMedia = {
@@ -1056,7 +1100,7 @@ export async function renderElement(input: RenderElementInput): Promise<Record<s
const imageIds = [...new Set([...svml.matchAll(/\{([a-z0-9-]+)\.image\}/gu)].map((match) => match[1] ?? ""))]
.filter((id) => !alreadySatisfied.has(`${id}.image`));
await Promise.all(imageIds.map(async (id) => await writePlaceholder({
- out: join(compareRoot, `${id}.png`), width: canvas.width, height: canvas.height, color: "mid",
+ out: join(compareRoot, `${id}.png`), ...mockExtent(svml, id, canvas), color: "mid",
})));
for (const id of imageIds) {
declarations.push(` <file id="mock-${id}" type="@hypit/artifact@1#BlobArtifact" from="./${id}.png" media-type="image/png"/>`);
@@ -1070,12 +1114,13 @@ export async function renderElement(input: RenderElementInput): Promise<Record<s
`<svrun version="1">`,
// The cut, when there was one: it sits beside this Run, and its own imports were repointed to
// reach the project from here.
- sliced === undefined
- ? ` <author source="../../${author.replace(/^\.\//u, "")}"/>`
- : ` <author source="./sliced.svml"/>`,
+ // Three levels up: the derived Run sits at `.hypit/compare/<key>/`, and the Source it names is at
+ // the project root. This branch used to run only when no window was asked for, which is the one
+ // case nothing exercised, so it was short by one the whole time.
+ ` <author source="../../../${author.replace(/^\.\//u, "")}"/>`,
// Whatever the Run brought, repointed: the derived Run sits two directories deeper than the one
// that declared these paths.
- ...carried.map((line) => ` ${line.replace(/from="\.\//gu, 'from="../../')}`),
+ ...carried.map((line) => ` ${line.replace(/from="\.\//gu, 'from="../../../')}`),
// Targeting the element's own output rather than the Film prunes the closure to what this one
// Track needs. Asking for the whole delivery would pull in every generation the Source declares
// and report each as unresolved, which is true and useless: none of them is what is being looked at.
@@ -1136,7 +1181,14 @@ export async function renderElement(input: RenderElementInput): Promise<Record<s
// The window to render, named in words. A shot of the reference is found by the words spoken over it
// and those words are a Segment or a Selection here, so no reference timestamp is ever read across.
let window: AuthoringWindow = { startFrame: 0, endFrameExclusive: programFrames };
- if (selectionArgument !== undefined) {
+ if (tokensArgument !== undefined) {
+ // A range given by index is the general form: a Segment and a Selection are both resolved into
+ // one, and a Cue is a range that has no other way to be named.
+ const [from, to] = tokensArgument;
+ assert(from >= 0 && to > from && to <= frameOfToken.length,
+ `token range [${from}, ${to}) is outside the Script's ${frameOfToken.length} words`);
+ window = { startFrame: frameOfToken[from]!.frame, endFrameExclusive: frameOfToken[to - 1]!.end };
+ } else if (selectionArgument !== undefined) {
const marked = selections.get(selectionArgument);
assert(marked !== undefined, `the Script marks no Selection ${selectionArgument}`);
window = marked;
@@ -1167,22 +1219,29 @@ export async function renderElement(input: RenderElementInput): Promise<Record<s
const clip = /\.(mp4|mov|webm)$/iu.test(outPath);
const frames = join(compareRoot, "frames");
- await rm(frames, { recursive: true, force: true });
- await mkdir(frames, { recursive: true });
- // The runtime ships with the tree, not with the project. Resolving it against the package root
- // finds nothing whenever those two differ, which is every project that installs packages of its own.
- const hyperframesCli = createRequire(join(repositoryRoot(), "packages/provider-hyperframes-local/package.json"))
- .resolve("hyperframes/bin/hyperframes.mjs");
- const drawn = spawnSync(process.execPath, [
- hyperframesCli, "render", stage,
- "--format", "png-sequence", "--output", frames, "--fps", String(frameRate),
- "--workers", String(Math.max(1, cpus().length - 2)),
- // Halves the time and lands on the same pixels — verified frame for frame against a run without
- // it, PSNR reporting no error at all. The flag is marked experimental upstream; that is the
- // reason to check the pixels, which is done, rather than the reason to draw twice as long.
- "--experimental-fast-capture", "--no-best-effort", "--quiet",
- ], { encoding: "utf8", windowsHide: true, timeout: 600_000 });
- assert(drawn.status === 0, `the HyperFrames runtime refused: ${(drawn.stderr ?? "").trim().slice(-2000)}`);
+ // Drawn once per Run and clock, however many windows are asked for.
+ //
+ // The picture does not depend on the element or the window — the whole program is drawn and the
+ // window is cut out of these frames — so a round of eight looks over one Source is one browser
+ // render and eight ffmpeg cuts. Rendering per entry would draw the same program eight times.
+ await drawProgramOnce(renderKey, async () => {
+ await rm(frames, { recursive: true, force: true });
+ await mkdir(frames, { recursive: true });
+ // The runtime ships with the tree, not with the project. Resolving it against the package root
+ // finds nothing whenever those two differ, which is every project that installs packages of its own.
+ const hyperframesCli = createRequire(join(repositoryRoot(), "packages/provider-hyperframes-local/package.json"))
+ .resolve("hyperframes/bin/hyperframes.mjs");
+ const drawn = spawnSync(process.execPath, [
+ hyperframesCli, "render", stage,
+ "--format", "png-sequence", "--output", frames, "--fps", String(frameRate),
+ "--workers", String(Math.max(1, cpus().length - 2)),
+ // Halves the time and lands on the same pixels — verified frame for frame against a run without
+ // it, PSNR reporting no error at all. The flag is marked experimental upstream; that is the
+ // reason to check the pixels, which is done, rather than the reason to draw twice as long.
+ "--experimental-fast-capture", "--no-best-effort", "--quiet",
+ ], { encoding: "utf8", windowsHide: true, timeout: 600_000 });
+ assert(drawn.status === 0, `the HyperFrames runtime refused: ${(drawn.stderr ?? "").trim().slice(-2000)}`);
+ });
const written = (await readdir(frames)).filter((name) => name.endsWith(".png")).sort();
assert(written.length > 0, "the HyperFrames runtime wrote no frames");
diff --git a/packages/reference-video-tools/src/checks.ts b/packages/reference-video-tools/src/checks.ts
index ae0d5c3e..bd8ad0ba 100644
--- a/packages/reference-video-tools/src/checks.ts
+++ b/packages/reference-video-tools/src/checks.ts
@@ -16,6 +16,7 @@ import { inspectStudioRun } from "@hypit/studio/src/studio-preflight.js";
import { aliasPattern, invokedFrom, nearestPackageRoot, referenceRoot, scriptBody } from "./authoring.js";
import { assert, round } from "./media.js";
+import { reviewPlan } from "./plan.js";
export type PreviewCheckInput = {
readonly run: string;
@@ -576,6 +577,7 @@ export async function authoringCheck(
return each.flat();
});
const drawingTags = new Set<string>();
+ const timeVaryingKeys = new Set<string>();
for (const pack of loaded) {
for (const facet of pack.contribution.hostFacets ?? []) {
if (facet.abi !== markupSurfaceHostFacetAbi) continue;
@@ -583,6 +585,17 @@ export async function authoringCheck(
const draws = surface.outputs.some((output) =>
output.module.name === "@hypit/composition" && (output.name === "VisualTrack" || output.name === "AudioTrack"));
if (draws) drawingTags.add(`${pack.specifier}#${surface.tag}`);
+ // Which Recipe properties the package itself calls time-varying.
+ //
+ // `caption-fine` sorts every property it declares into where / how / when, and `when` is
+ // exactly the question one of the planning rules asks: does this run for as long as the window
+ // does? Reading the package's own answer keeps that rule correct for properties added after
+ // this file was written, which a list maintained here would not be.
+ for (const attribute of surface.vocabulary?.attributes ?? []) {
+ for (const property of attribute.recipe ?? []) {
+ if (property.group === "when") timeVaryingKeys.add(property.name);
+ }
+ }
}
}
@@ -820,6 +833,22 @@ export async function authoringCheck(
};
}
+ // What a round has to look at, decided from the Source rather than left to whoever runs it.
+ //
+ // The Recipe bodies come along because one of the three rules asks whether an element's appearance
+ // runs for as long as its window does, and that is written in the Recipe rather than on the tag.
+ const recipeBodies = new Map<string, string>();
+ for (const sheet of svml.matchAll(/<import\s+as="[^"]+"\s+source="([^"]+\.svs)"/gu)) {
+ const text = await readFile(resolve(dirname(svmlPath), sheet[1] ?? ""), "utf8").catch(() => undefined);
+ if (text === undefined) continue;
+ for (const recipe of text.matchAll(/([A-Za-z0-9_.-]+)\s*\{([^}]*)\}/gu)) recipeBodies.set(recipe[1] ?? "", recipe[2] ?? "");
+ }
+ const plan = reviewPlan({ svml, svmlPath, scriptBody: scriptBody(svml), drawn, recipes: recipeBodies, timeVaryingKeys });
+ // The same parse the plan was built from, so a look recorded against a Segment can be resolved to
+ // the words that Segment marks and compared with a plan entry on one axis.
+ const plannedBody = scriptBody(svml);
+ const plannedScript = parseScript(svmlPath, plannedBody.text, plannedBody.offset);
+
const coverage = coverageReport();
const uncoveredWords = coverage.gaps.reduce((sum, gap) => sum + gap.word_count, 0);
@@ -847,7 +876,7 @@ export async function authoringCheck(
type LoggedComparison = {
readonly element?: string;
readonly shot_id?: string;
- readonly range?: { readonly segment?: string; readonly selection?: string };
+ readonly range?: { readonly segment?: string; readonly selection?: string; readonly tokens?: readonly [number, number] };
readonly status: string;
readonly id?: string;
readonly image_path?: string;
@@ -921,6 +950,33 @@ export async function authoringCheck(
return recorded.size === 1 ? [...recorded][0]! : "mixed";
};
+ // Which of the plan's entries have been answered.
+ //
+ // A logged look and a planned one are compared as word ranges, because that is the one form both
+ // can always be put in: a look recorded against a Segment resolves to the words that Segment marks,
+ // and a look recorded against a Cue was never anything else. Comparing the names instead would let
+ // a Cue in the middle of a Segment match the Segment, and the coverage this gate claims would be
+ // wider than the coverage it has.
+ const asTokens = (entry: LoggedComparison): string | undefined => {
+ if (entry.range?.tokens !== undefined) return `${entry.range.tokens[0]}-${entry.range.tokens[1]}`;
+ const segment = entry.range?.segment === undefined
+ ? undefined
+ : plannedScript.segments.find((item) => item.id === entry.range!.segment);
+ if (segment !== undefined) return `${segment.tokenStart}-${segment.tokenEndExclusive}`;
+ const selection = entry.range?.selection === undefined
+ ? undefined
+ : plannedScript.selections.find((item) => item.id === entry.range!.selection);
+ if (selection !== undefined) return `${selection.open.boundary.tokenIndex}-${selection.close.boundary.tokenIndex}`;
+ // A shot names a cut in the reference's picture rather than a range of the Script's words, so it
+ // answers no plan entry. It still counts as having looked, which `rounds` already records.
+ return undefined;
+ };
+ const answered = new Set(log.flatMap((entry) => {
+ const tokens = asTokens(entry);
+ return entry.element === undefined || tokens === undefined ? [] : [`${entry.element} ${tokens}`];
+ }));
+ const owed = plan.filter((entry) => !answered.has(`${entry.element} ${entry.tokens[0]}-${entry.tokens[1]}`));
+
const never = drawn.filter((element) => (rounds.get(element.id) ?? []).length === 0);
const unlabelled = log.filter((entry) => entry.element === undefined).length;
// A misspelled --element is otherwise silent: the comparison happens, the log grows, and the element
@@ -956,6 +1012,10 @@ export async function authoringCheck(
summary.push(`every drawing element has been ${looked} (${elements.length}), and every timed picture fills its window.`);
}
if (never.length > 0) summary.push(`${never.length} of ${elements.length} elements have never been ${looked}.`);
+ if (owed.length > 0) {
+ summary.push(`${owed.length} of ${plan.length} stretch${plan.length === 1 ? "" : "es"} the Source asks for `
+ + `${owed.length === 1 ? "has" : "have"} not been ${looked}.`);
+ }
if (running.length > 0) {
summary.push(`${running.length} window${running.length === 1 ? "" : "s"} will empty before ${running.length === 1 ? "it ends" : "they end"}.`);
}
@@ -996,14 +1056,30 @@ export async function authoringCheck(
return {
run: runPath,
...(reference === undefined ? {} : { reference_id: reference }),
+ plan,
// Where this command actually read and resolved from. Said here, no document has to describe it
// from the outside and go stale when it moves.
roots: mode === "reconstruction"
? { reference: preparedRoot, packages: packageRoot }
: { reviews: logPath, packages: packageRoot },
- passed: never.length === 0 && running.length === 0 && coverage.gaps.length === 0 && unresolved.length === 0,
+ passed: owed.length === 0 && never.length === 0 && running.length === 0 && coverage.gaps.length === 0 && unresolved.length === 0,
summary,
elements,
+ ...(owed.length === 0 ? {} : {
+ owed: {
+ entries: owed,
+ note: `${owed.length} stretch${owed.length === 1 ? "" : "es"} the Source asks for ${owed.length === 1 ? "has" : "have"} `
+ + "not been looked at. Each one is an element over a range of the Script's own words, and the reason it is "
+ + "here is beside it — a declaration nobody has seen, the longest cue a caption Style has to hold, or a "
+ + "window long enough that an animation inside it behaves differently.",
+ commands: owed.map((entry) => mode === "reconstruction"
+ ? `hypit-reference-video-tools compare_reconstruction --reference-id ${reference} --run ${runPath} `
+ + `--tokens ${entry.tokens[0]}:${entry.tokens[1]} --video <rendered clip>.mp4 --element ${entry.element}`
+ : `hypit-reference-video-tools review_element --run ${runPath} `
+ + `--tokens ${entry.tokens[0]}:${entry.tokens[1]} --video <rendered clip>.mp4 --element ${entry.element} `
+ + `--intent-file <what ${entry.element} was asked to be>`),
+ },
+ }),
...(never.length === 0 ? {} : {
never_compared: {
ids: never.map((element) => element.id),
diff --git a/packages/reference-video-tools/src/cli.ts b/packages/reference-video-tools/src/cli.ts
index 369f2c2d..af1ff952 100644
--- a/packages/reference-video-tools/src/cli.ts
+++ b/packages/reference-video-tools/src/cli.ts
@@ -4,6 +4,22 @@ import { readFile } from "node:fs/promises";
import { createReferenceVideoTools } from "./tools.js";
import type { CompareReconstructionInput, ObserveReferenceInput, RenderElementInput, ReviewElementInput } from "./tools.js";
+/**
+ * A word range written on the command line as `from:to`, half-open.
+ *
+ * The planner emits ranges because the finest thing worth looking at is not always a thing the
+ * Script named. `--segment` and `--selection` remain for the ones that are.
+ */
+function tokenRange(value: string | undefined): readonly [number, number] | undefined {
+ if (value === undefined) return undefined;
+ const parts = /^(\d+):(\d+)$/u.exec(value.trim());
+ if (parts === null) throw new Error(`--tokens takes a half-open word range written from:to, like 118:131; got ${value}`);
+ const from = Number(parts[1]);
+ const to = Number(parts[2]);
+ if (to <= from) throw new Error(`--tokens ${value} ends at or before it starts`);
+ return [from, to];
+}
+
type Flags = ReadonlyMap<string, string | readonly string[] | boolean>;
function usage(): string {
@@ -18,14 +34,14 @@ function usage(): string {
" hypit-reference-video-tools inspect_svml_vocabulary --package <name> [--package <name> ...] [--tag <tag> ...] [--without-previews]",
" hypit-reference-video-tools inspect_visual_contract [--shape visual-track|text-flow|text-typography|text-paint|text-document|path-command] [--producers-of <package> ...]",
" hypit-reference-video-tools paths",
- " hypit-reference-video-tools compare_reconstruction --reference-id <id> --run <build.svrun> --segment <id>|--selection <id> --video <path>|--image <path> [--question <scope>] [--element <id>]",
+ " hypit-reference-video-tools compare_reconstruction --reference-id <id> --run <build.svrun> --segment <id>|--selection <id>|--tokens <from:to> --video <path>|--image <path> [--question <scope>] [--element <id>]",
" hypit-reference-video-tools compare_reconstruction --reference-id <id> --shot-id <id> --video <path>|--image <path> [--question <scope>] [--element <id>]",
" hypit-reference-video-tools compare_reconstruction --reference-id <id> --batch <comparisons.json>",
- " hypit-reference-video-tools review_element --run <build.svrun> --element <id> --segment <id>|--selection <id> --video <path>|--image <path> --intent-file <path> [--question <scope>]",
+ " hypit-reference-video-tools review_element --run <build.svrun> --element <id> --segment <id>|--selection <id>|--tokens <from:to> --video <path>|--image <path> --intent-file <path> [--question <scope>]",
" hypit-reference-video-tools review_element --run <build.svrun> --batch <reviews.json>",
" hypit-reference-video-tools record_review --run <build.svrun> --review-id <id> --text <text>|--text-file <path>",
" hypit-reference-video-tools make_placeholder --out <path> --width <w> --height <h> [--color light|mid|dark|white|black|#RRGGBB] [--video] [--seconds <s>]",
- " hypit-reference-video-tools render_element <build.svrun> --element <id> --out <path.png|path.mp4> [--segment <id>] [--selection <id>] [--reference-id <id>]",
+ " hypit-reference-video-tools render_element <build.svrun> --element <id> --out <path.png|path.mp4> [--segment <id>] [--selection <id>] [--tokens <from:to>] [--reference-id <id>]",
" hypit-reference-video-tools render_element <build.svrun> --batch <renders.json> [--reference-id <id>]",
" hypit-reference-video-tools render_previews <package-dir> [...]",
" hypit-reference-video-tools preview_check <build.svrun> [<hypit.runtime.json>]",
@@ -302,6 +318,7 @@ async function main(): Promise<void> {
element: required(flags, "element"),
...(one(flags, "segment") === undefined ? {} : { segment: one(flags, "segment") }),
...(one(flags, "selection") === undefined ? {} : { selection: one(flags, "selection") }),
+ ...(tokenRange(one(flags, "tokens")) === undefined ? {} : { tokens: tokenRange(one(flags, "tokens")) }),
...(one(flags, "image") === undefined ? {} : { image_path: one(flags, "image") }),
...(one(flags, "video") === undefined ? {} : { video_path: one(flags, "video") }),
...(one(flags, "intent-file") === undefined ? {} : { intent_file: one(flags, "intent-file") }),
@@ -336,12 +353,14 @@ async function main(): Promise<void> {
const video = one(flags, "video");
const segment = one(flags, "segment");
const selection = one(flags, "selection");
- const range = segment !== undefined || selection !== undefined;
+ const tokens = tokenRange(one(flags, "tokens"));
+ const range = segment !== undefined || selection !== undefined || tokens !== undefined;
const input = supplied ?? {
reference_id: required(flags, "reference-id"),
...(range
? {
run: required(flags, "run"),
+ ...(tokens === undefined ? {} : { tokens }),
...(segment === undefined ? {} : { segment }),
...(selection === undefined ? {} : { selection }),
}
@@ -398,9 +417,10 @@ async function main(): Promise<void> {
out: required(flags, "out"),
...(one(flags, "segment") === undefined ? {} : { segment: one(flags, "segment") }),
...(one(flags, "selection") === undefined ? {} : { selection: one(flags, "selection") }),
+ ...(tokenRange(one(flags, "tokens")) === undefined ? {} : { tokens: tokenRange(one(flags, "tokens")) }),
...(one(flags, "reference-id") === undefined ? {} : { reference_id: one(flags, "reference-id") }),
};
- result = await tools.render_element(input as { run: string; element: string; out: string; segment?: string; selection?: string; reference_id?: string });
+ result = await tools.render_element(input as RenderElementInput);
} else if (command === "render_previews") {
if (supplied === undefined && operands.length === 0) throw new Error(`a <package-dir> is required\n\n${usage()}`);
const input = supplied ?? { package_dirs: operands };
diff --git a/packages/reference-video-tools/src/plan.ts b/packages/reference-video-tools/src/plan.ts
new file mode 100644
index 00000000..036606a6
--- /dev/null
+++ b/packages/reference-video-tools/src/plan.ts
@@ -0,0 +1,499 @@
+/**
+ * Which stretches of a program are worth looking at, decided from the Source alone.
+ *
+ * An element drawn over five Segments does not need five looks. It needs one per way it can fail,
+ * and two stretches fail the same way when they are the same declaration over the same kind of
+ * content for the same length of time. Deciding that by reading the Source is what makes a round
+ * reproducible and what lets the gate ask for coverage rather than for participation.
+ *
+ * Three rules produce the list, and every one of them reads text:
+ *
+ * 1. **A distinct declaration is a distinct picture.** Group an element's placements by what they
+ * declare and look at each group once, where it first appears. This is also the floor: every
+ * placed element has at least one declaration, so every element gets at least one look — which is
+ * exactly what the gate required before this file existed.
+ * 2. **Content that can break the layout is worth its own look.** Within a group, the stretch
+ * holding the most is a different question from the stretch holding the least: a caption system
+ * read over its shortest line looks correct, because one line has nothing to collide with.
+ * 3. **A window length is worth its own look only when something scales with it.** An enter and an
+ * exit happen at the edges for a fixed number of frames; a typewriter, a loop or a `stretch`
+ * playback runs for as long as the window does.
+ *
+ * The unit is a token range rather than a Segment name, because the answer is not always a name. A
+ * caption Cue is a run of words the Script never marked, and it is exactly what rule 2 wants to look
+ * at. Both sides of a comparison already work in words — `frameOfToken` here, `spokenRange` on the
+ * reference — so the range is what they both take, and a name comes along only to be read.
+ */
+import { parseScript } from "@hypit/script";
+import type { ParsedNarrative } from "@hypit/script";
+
+/** One thing to look at: an element, over a range of the Script's words, and why it earned a look. */
+export type PlanEntry = {
+ readonly element: string;
+ /** Half-open over the Script's speech tokens, the same index space `frameOfToken` is in. */
+ readonly tokens: readonly [number, number];
+ /** What that range is, for a reader. Never parsed back — the range is the identity. */
+ readonly named: string;
+ readonly why: readonly string[];
+};
+
+/** A drawing element as the gate enumerates it, before its placements are read. */
+export type PlannedElement = {
+ readonly id: string;
+ readonly tag: string;
+ readonly alias: string;
+ readonly specifier: string;
+};
+
+/**
+ * Attributes that say *when* rather than *what*, so they never join a declaration key.
+ *
+ * Two placements differing only in these are the same design over different words, which is one
+ * look and two stretches — the case rule 1 exists to collapse.
+ */
+const WINDOW_ATTRIBUTES = new Set([
+ "during", "at", "for", "start", "end", "selection", "segment", "moment", "until",
+ "until-boundary", "terminal", "semantic", "document", "narrative", "program-ref",
+]);
+
+/**
+ * Attributes naming material a Build has not made, so they never join a declaration key either.
+ *
+ * Every one of these is a flat placeholder in the render this plan is for. Two Items differing only
+ * in which unbuilt picture they name draw the identical mock, so looking at both spends a subagent
+ * to see the same grey rectangle twice.
+ */
+const MOCKED_ATTRIBUTES = new Set(["image", "video", "media", "source", "icon", "audio", "source-audio"]);
+
+/** Never a declaration: the id is what the key is being computed *for*. */
+const IGNORED_ATTRIBUTES = new Set(["id"]);
+
+/**
+ * Recipe keys whose effect lasts as long as the window does.
+ *
+ * An `enter` costs its `enter-frames` at one edge whatever the window is; these do not. A window
+ * twice as long runs twice as much typewriter, twice as many loop cycles, and stretches its material
+ * to twice the duration — so two windows of very different lengths are two questions.
+ *
+ * `caption-fine` publishes the same judgement as machine-readable metadata (`group: "when"`), and
+ * where that is present it is read instead of this list. This is the fallback for the packages that
+ * do not yet classify their own properties.
+ */
+const WHOLE_WINDOW_KEYS = [
+ "sustain", "playback", "playback-future", "playback-past", "trim-start", "trim-end",
+ "loop", "loop-target", "loop-period-frames", "loop-intensity",
+ "atom-reveal", "karaoke", "karaoke-transition", "active-underline",
+ "active-box", "active-box-continuity",
+];
+
+type Placement = {
+ readonly element: string;
+ readonly tag: string;
+ readonly key: string;
+ readonly stretch: Stretch;
+ /** Everything the placement declares, kept so rule 3 can read the Recipes it names. */
+ readonly attributes: ReadonlyMap<string, string>;
+};
+
+type Stretch = {
+ readonly tokens: readonly [number, number];
+ readonly named: string;
+};
+
+/** Every attribute on an opening tag, as written. */
+function attributesOf(text: string): ReadonlyMap<string, string> {
+ const found = new Map<string, string>();
+ for (const match of text.matchAll(/\b([a-z][a-z0-9-]*)="([^"]*)"/gu)) found.set(match[1] ?? "", match[2] ?? "");
+ // A reference is written `attr={id}` rather than quoted, and it is the form every Recipe and Frame
+ // arrives in, so a key built from quoted attributes alone would miss the whole declaration.
+ for (const match of text.matchAll(/\b([a-z][a-z0-9-]*)=\{([^}]*)\}/gu)) found.set(match[1] ?? "", `{${match[2] ?? ""}}`);
+ return found;
+}
+
+/**
+ * What this placement declares, as one comparable string.
+ *
+ * Everything on the tag except the three excluded sets, sorted so attribute order — which this
+ * markup does not make significant — cannot split one group into two. The tag name leads, because
+ * the same attributes on a different element are a different picture.
+ *
+ * Taking everything rather than a list of known Recipe attributes is deliberate. `screen-overlay`
+ * declares its entire appearance as tag literals and names no Recipe at all, and a package added
+ * tomorrow will be read correctly without this file learning its vocabulary. The cost of the wide
+ * net is a split group, which is one extra look; the cost of a narrow one is a merged group, which
+ * is a picture nobody sees.
+ */
+function declarationKey(tag: string, attributes: ReadonlyMap<string, string>): string {
+ const parts: string[] = [];
+ for (const [name, value] of attributes) {
+ if (WINDOW_ATTRIBUTES.has(name) || MOCKED_ATTRIBUTES.has(name) || IGNORED_ATTRIBUTES.has(name)) continue;
+ parts.push(`${name}=${value}`);
+ }
+ return `${tag}|${parts.sort().join("|")}`;
+}
+
+/** The whole Script, for an element bound to the program rather than to a marked range. */
+function wholeProgram(parsed: ParsedNarrative): Stretch {
+ return { tokens: [0, parsed.tokens.length], named: "the whole program" };
+}
+
+/**
+ * Where a placement's `during=` or `at=` puts it, in words.
+ *
+ * Returns undefined when the binding names something the Script does not mark, which is a Source
+ * defect `preview_check` reports rather than something to guess at here.
+ */
+function stretchOf(attributes: ReadonlyMap<string, string>, parsed: ParsedNarrative): Stretch | undefined {
+ const during = attributes.get("during");
+ if (during === "program") return wholeProgram(parsed);
+
+ const named = (value: string | undefined): { readonly kind: string; readonly id: string } | undefined => {
+ const match = /^\{story\.(segment|selection|moment)\.([A-Za-z0-9_-]+)\}$/u.exec(value ?? "");
+ return match === undefined || match === null ? undefined : { kind: match[1]!, id: match[2]! };
+ };
+
+ const bound = named(during) ?? named(attributes.get("at"));
+ if (bound === undefined) return undefined;
+
+ if (bound.kind === "segment") {
+ const segment = parsed.segments.find((item) => item.id === bound.id);
+ return segment === undefined
+ ? undefined
+ : { tokens: [segment.tokenStart, segment.tokenEndExclusive], named: `segment ${bound.id}` };
+ }
+ if (bound.kind === "selection") {
+ const selection = parsed.selections.find((item) => item.id === bound.id);
+ return selection === undefined
+ ? undefined
+ : {
+ tokens: [selection.open.boundary.tokenIndex, selection.close.boundary.tokenIndex],
+ named: `selection ${bound.id}`,
+ };
+ }
+ // A Moment is an instant, not a range. What it marks is a change inside whichever Segment holds
+ // it, so that Segment is the stretch a look would be rendered over.
+ const moment = parsed.moments.find((item) => item.id === bound.id);
+ if (moment === undefined) return undefined;
+ const index = moment.boundary.tokenIndex;
+ const holding = parsed.segments.find((item) => index >= item.tokenStart && index < item.tokenEndExclusive);
+ return holding === undefined
+ ? undefined
+ : { tokens: [holding.tokenStart, holding.tokenEndExclusive], named: `segment ${holding.id} (at moment ${bound.id})` };
+}
+
+/**
+ * The body of one element, so its children can be read.
+ *
+ * These Tracks do not nest inside themselves, so the first matching close tag is this element's.
+ * Returns an empty string for a self-closing element, which has no children to find.
+ */
+function bodyOf(svml: string, alias: string, tag: string, id: string): string {
+ const opening = new RegExp(`<${alias}:${tag}\\b[^>]*?\\bid="${id}"[^>]*?(/?)>`, "su").exec(svml);
+ if (opening === null || opening[1] === "/") return "";
+ const after = svml.slice(opening.index + opening[0].length);
+ const close = after.indexOf(`</${alias}:${tag}>`);
+ return close === -1 ? after : after.slice(0, close);
+}
+
+/** The opening tag of one element, by id. */
+function openingOf(svml: string, alias: string, tag: string, id: string): string {
+ return new RegExp(`<${alias}:${tag}\\b[^>]*?\\bid="${id}"[^>]*?/?>`, "su").exec(svml)?.[0] ?? "";
+}
+
+/**
+ * The Styles a caption Program hands out, and the stretch each one covers.
+ *
+ * A caption Track is one tag with no window on it — it draws wherever the Script has words — so
+ * reading its placements the way every other element is read finds exactly one, and reports one
+ * design. That is wrong whenever the Program overrides a Style: `<caption:Use style=… selection=…>`
+ * is how one stretch is drawn differently from the rest, and it is written inside the Program rather
+ * than on the Track, so nothing on the Track says it happened.
+ *
+ * Each override is its own declaration over its own words. The default is a declaration too, over
+ * whatever the overrides did not claim, which is where the Track's single placement belongs.
+ */
+function captionStyles(svml: string, element: PlannedElement, parsed: ParsedNarrative): readonly Placement[] {
+ const opening = openingOf(svml, element.alias, element.tag, element.id);
+ const program = /\bprogram=\{([A-Za-z0-9_-]+)\}/u.exec(opening)?.[1];
+ if (program === undefined) return [];
+ const declaring = new RegExp(`<[a-z][a-z0-9-]*:Program\\b[^>]*?\\bid="${program}"[^>]*?(/?)>`, "su").exec(svml);
+ if (declaring === null) return [];
+
+ const body = declaring[1] === "/" ? "" : (() => {
+ const after = svml.slice(declaring.index + declaring[0].length);
+ const close = after.search(/<\/[a-z][a-z0-9-]*:Program>/u);
+ return close === -1 ? after : after.slice(0, close);
+ })();
+
+ const tag = `${element.alias}:${element.tag}`;
+ const found: Placement[] = [];
+ for (const use of body.matchAll(/<[a-z][a-z0-9-]*:Use\b([^>]*?)\/?>/gsu)) {
+ const attributes = attributesOf(use[0] ?? "");
+ // A Use names its stretch with `selection=`, not the `during=` every drawn element uses, because
+ // it is assigning a Style to a run of words rather than placing a picture over them.
+ const marked = /^\{story\.selection\.([A-Za-z0-9_-]+)\}$/u.exec(attributes.get("selection") ?? "")?.[1];
+ const selection = marked === undefined ? undefined : parsed.selections.find((item) => item.id === marked);
+ const stretch = selection === undefined
+ ? stretchOf(attributes, parsed)
+ : {
+ tokens: [selection.open.boundary.tokenIndex, selection.close.boundary.tokenIndex] as readonly [number, number],
+ named: `selection ${marked}`,
+ };
+ const style = attributes.get("style");
+ if (stretch === undefined || style === undefined) continue;
+ found.push({
+ element: element.id, tag: `${tag} (caption:Use)`,
+ key: `${tag}|style=${style}`, stretch, attributes,
+ });
+ }
+ return found;
+}
+
+/**
+ * Every placement one drawing element makes.
+ *
+ * A Track holding forty Items draws nothing itself: the Items carry the windows and the Recipes, and
+ * they are what a look is actually of. So the element's own tag is a placement only when it binds a
+ * window itself, and each child that binds one is a placement too.
+ */
+function placementsOf(svml: string, element: PlannedElement, parsed: ParsedNarrative): readonly Placement[] {
+ const found: Placement[] = [];
+ const opening = openingOf(svml, element.alias, element.tag, element.id);
+ const own = attributesOf(opening);
+ const ownStretch = stretchOf(own, parsed);
+ if (ownStretch !== undefined) {
+ found.push({
+ element: element.id, tag: `${element.alias}:${element.tag}`,
+ key: declarationKey(`${element.alias}:${element.tag}`, own), stretch: ownStretch, attributes: own,
+ });
+ }
+
+ for (const child of bodyOf(svml, element.alias, element.tag, element.id)
+ .matchAll(/<([a-z][a-z0-9-]*:[A-Za-z][A-Za-z0-9]*)\b([^>]*?)\/?>/gsu)) {
+ const tag = child[1] ?? "";
+ const attributes = attributesOf(child[0] ?? "");
+ const stretch = stretchOf(attributes, parsed);
+ if (stretch === undefined) continue;
+ // The child's own declaration is what it draws, but it sits inside the Track's, so the Track's
+ // is part of the key: the same Item under two Tracks with different Canvases is two pictures.
+ found.push({
+ element: element.id,
+ tag,
+ key: `${declarationKey(`${element.alias}:${element.tag}`, own)}>>${declarationKey(tag, attributes)}`,
+ stretch,
+ attributes,
+ });
+ }
+
+ // An element that binds no window anywhere is on screen for the whole program — a persistent
+ // badge, a Film-wide treatment — and the whole program is the stretch it is looked at over.
+ if (found.length === 0) {
+ found.push({
+ element: element.id,
+ tag: `${element.alias}:${element.tag}`,
+ key: declarationKey(`${element.alias}:${element.tag}`, own),
+ stretch: wholeProgram(parsed),
+ attributes: own,
+ });
+ }
+ return found;
+}
+
+/**
+ * Rule 2, for captions: where the longest Cue is.
+ *
+ * A Cue is the unit a caption's layout can break on, and it is not a Segment. Four things end one,
+ * and only one of them is the `||` the author wrote: the other three are a Segment ending, a speaker
+ * changing, and a Style changing. Partitioning on the authored breaks alone reads a two-speaker
+ * script as a handful of enormous Cues, and then reports the longest line as the whole exchange.
+ *
+ * The partition is computed in token space rather than by calling `temporalizeCaptionDocument`,
+ * which needs a SemanticTrack this check never builds. The four facts are the same ones it groups on
+ * and all four are Script-side, so the boundaries land in the same places; what is not computed here
+ * is the frame each Cue occupies, which nothing on this path asks for.
+ */
+function longestCue(parsed: ParsedNarrative, styleBoundaries: readonly number[]): Stretch | undefined {
+ const total = parsed.tokens.length;
+ if (total === 0) return undefined;
+ const boundaries = new Set<number>([0, total]);
+ for (const segment of parsed.segments) boundaries.add(segment.tokenStart);
+ for (const turn of parsed.turns) boundaries.add(turn.tokenStart);
+ for (const authored of parsed.captionProjection.breaks) boundaries.add(authored.tokenIndex);
+ for (const boundary of styleBoundaries) boundaries.add(boundary);
+
+ const edges = [...boundaries].filter((at) => at >= 0 && at <= total).sort((one, other) => one - other);
+ let best: { readonly from: number; readonly to: number; readonly characters: number } | undefined;
+ for (let index = 0; index + 1 < edges.length; index += 1) {
+ const from = edges[index]!;
+ const to = edges[index + 1]!;
+ if (to <= from) continue;
+ const characters = parsed.tokens.slice(from, to).reduce((sum, token) => sum + token.text.length, 0);
+ if (best === undefined || characters > best.characters) best = { from, to, characters };
+ }
+ if (best === undefined) return undefined;
+ const holding = parsed.segments.find((item) => best!.from >= item.tokenStart && best!.from < item.tokenEndExclusive);
+ return {
+ tokens: [best.from, best.to],
+ named: holding === undefined ? "the longest cue" : `the longest cue, in segment ${holding.id}`,
+ };
+}
+
+/**
+ * Where a `caption:Program` changes Style, in token space.
+ *
+ * `<caption:Use selection=…>` is how one stretch is drawn differently from the rest, and it is the
+ * only place a caption's declaration varies — the Track's own tag is identical over the whole
+ * program. Both ends of the named Selection are Cue boundaries, because a Style change ends a Cue.
+ */
+function captionStyleBoundaries(svml: string, parsed: ParsedNarrative): readonly number[] {
+ const found: number[] = [];
+ for (const use of svml.matchAll(/<[a-z][a-z0-9-]*:(?:Use|Mute)\b([^>]*?)\/?>/gsu)) {
+ const selection = /\bselection=\{story\.selection\.([A-Za-z0-9_-]+)\}/u.exec(use[1] ?? "")?.[1];
+ if (selection === undefined) continue;
+ const marked = parsed.selections.find((item) => item.id === selection);
+ if (marked === undefined) continue;
+ found.push(marked.open.boundary.tokenIndex, marked.close.boundary.tokenIndex);
+ }
+ return found;
+}
+
+/**
+ * Rule 2, for anything holding a list: which stretch holds the most of it.
+ *
+ * Items are written into the Source as children, each bound to its own Moment or Selection, so the
+ * count per stretch is a count of children — no rendering and no measurement. The stretch holding
+ * the most is where the layout has the most chance to run out of room.
+ */
+function fullestStretch(placements: readonly Placement[]): Stretch | undefined {
+ const counts = new Map<string, { readonly stretch: Stretch; count: number }>();
+ for (const placement of placements) {
+ const key = `${placement.stretch.tokens[0]}-${placement.stretch.tokens[1]}`;
+ const held = counts.get(key);
+ if (held === undefined) counts.set(key, { stretch: placement.stretch, count: 1 });
+ else held.count += 1;
+ }
+ const ranked = [...counts.values()].sort((one, other) => other.count - one.count);
+ const most = ranked[0];
+ // One each is not a list. Nominating a stretch here would just re-name what rule 1 already has.
+ return most === undefined || most.count < 2 ? undefined : most.stretch;
+}
+
+/**
+ * Rule 3: does anything about this element's appearance run for as long as its window does?
+ *
+ * `caption-fine` classifies its own properties, so where a Recipe body carries a key that package
+ * marked `when`, that is the authority. Everything else is matched against the list above, which is
+ * the same judgement made by hand for the packages that do not publish one yet.
+ */
+function scalesWithWindow(bodies: readonly string[], published: ReadonlySet<string>): boolean {
+ const keys = [...new Set([...published, ...WHOLE_WINDOW_KEYS])];
+ return bodies.some((body) => keys.some((key) => new RegExp(`\\b${key}\\s*:`, "u").test(body)));
+}
+
+/** The Recipe bodies one placement's declaration names, so rule 3 can read their keys. */
+function recipeBodiesOf(attributes: ReadonlyMap<string, string>, recipes: ReadonlyMap<string, string>): readonly string[] {
+ const bodies: string[] = [];
+ for (const [name, value] of attributes) {
+ if (WINDOW_ATTRIBUTES.has(name) || MOCKED_ATTRIBUTES.has(name)) continue;
+ const named = /^\{([A-Za-z0-9_.-]+)\}$/u.exec(value)?.[1];
+ if (named === undefined) continue;
+ const body = recipes.get(named.replace(/^recipes\./u, ""));
+ if (body !== undefined) bodies.push(body);
+ }
+ return bodies;
+}
+
+/** Rule 1: one look per distinct declaration, at the stretch where that declaration first appears. */
+function firstOfEachDeclaration(placements: readonly Placement[]): readonly PlanEntry[] {
+ const seen = new Set<string>();
+ const entries: PlanEntry[] = [];
+ for (const placement of placements) {
+ if (seen.has(placement.key)) continue;
+ seen.add(placement.key);
+ entries.push({
+ element: placement.element,
+ tokens: placement.stretch.tokens,
+ named: placement.stretch.named,
+ why: [`first use of this declaration (${placement.tag})`],
+ });
+ }
+ return entries;
+}
+
+/**
+ * Merge nominations that landed on the same thing.
+ *
+ * Three rules can each name one stretch — the first occurrence of a Style may also hold its longest
+ * line and its longest window — and that is one look with three reasons, not three looks.
+ */
+function merge(entries: readonly PlanEntry[]): readonly PlanEntry[] {
+ const byKey = new Map<string, PlanEntry>();
+ for (const entry of entries) {
+ const key = `${entry.element}${entry.tokens[0]}-${entry.tokens[1]}`;
+ const held = byKey.get(key);
+ if (held === undefined) { byKey.set(key, entry); continue; }
+ byKey.set(key, { ...held, why: [...held.why, ...entry.why.filter((reason) => !held.why.includes(reason))] });
+ }
+ return [...byKey.values()];
+}
+
+/** What a round has to look at, for one Source. */
+export function reviewPlan(input: {
+ readonly svml: string;
+ readonly svmlPath: string;
+ readonly scriptBody: { readonly text: string; readonly offset: number };
+ readonly drawn: readonly PlannedElement[];
+ /** Recipe bodies by their sheet-local name, for rule 3. Absent means rule 3 nominates nothing. */
+ readonly recipes?: ReadonlyMap<string, string>;
+ /**
+ * Property names the packages themselves marked time-varying, read from their own vocabulary.
+ *
+ * A package that classifies its properties is the authority on them, and it stays right as it adds
+ * more. The list in this file is the fallback for the packages that publish no classification.
+ */
+ readonly timeVaryingKeys?: ReadonlySet<string>;
+}): readonly PlanEntry[] {
+ const parsed = parseScript(input.svmlPath, input.scriptBody.text, input.scriptBody.offset);
+ const recipes = input.recipes ?? new Map<string, string>();
+ const published = input.timeVaryingKeys ?? new Set<string>();
+ const styleBoundaries = captionStyleBoundaries(input.svml, parsed);
+ const entries: PlanEntry[] = [];
+
+ for (const element of input.drawn) {
+ // A caption Track's designs are declared inside its Program, not on the Track, so they are read
+ // separately and joined to whatever the ordinary placement scan found.
+ const placements = [
+ ...placementsOf(input.svml, element, parsed),
+ ...(element.specifier.includes("caption") ? captionStyles(input.svml, element, parsed) : []),
+ ];
+ entries.push(...firstOfEachDeclaration(placements));
+
+ // Rule 2. A caption's layout breaks on a Cue, and a list's breaks on how many items one stretch
+ // holds; every other element draws one thing and has no content extreme to find.
+ if (element.specifier.includes("caption")) {
+ const cue = longestCue(parsed, styleBoundaries);
+ if (cue !== undefined) entries.push({ element: element.id, tokens: cue.tokens, named: cue.named, why: ["longest cue"] });
+ }
+ const fullest = fullestStretch(placements);
+ if (fullest !== undefined) {
+ entries.push({ element: element.id, tokens: fullest.tokens, named: fullest.named, why: ["most items in one stretch"] });
+ }
+
+ // Rule 3. Only where something runs for as long as the window does, and only then are the
+ // longest and the shortest two different questions rather than the same picture twice.
+ const animates = placements.some((placement) => scalesWithWindow(recipeBodiesOf(placement.attributes, recipes), published));
+ if (animates && placements.length > 1) {
+ const byLength = [...placements].sort((one, other) =>
+ (one.stretch.tokens[1] - one.stretch.tokens[0]) - (other.stretch.tokens[1] - other.stretch.tokens[0]));
+ const shortest = byLength[0]!;
+ const longest = byLength.at(-1)!;
+ if (longest.stretch.tokens[1] - longest.stretch.tokens[0] !== shortest.stretch.tokens[1] - shortest.stretch.tokens[0]) {
+ entries.push({ element: element.id, tokens: longest.stretch.tokens, named: longest.stretch.named, why: ["longest window, and this element animates across it"] });
+ entries.push({ element: element.id, tokens: shortest.stretch.tokens, named: shortest.stretch.named, why: ["shortest window, and this element animates across it"] });
+ }
+ }
+ }
+ return merge(entries);
+}
+
+export { WHOLE_WINDOW_KEYS };
diff --git a/packages/reference-video-tools/src/slice.ts b/packages/reference-video-tools/src/slice.ts
deleted file mode 100644
index 68d50e92..00000000
--- a/packages/reference-video-tools/src/slice.ts
+++ /dev/null
@@ -1,291 +0,0 @@
-/**
- * Cut an Author Source down to the one stretch a comparison is looking at, mechanically.
- *
- * Rendering an element means rendering a program, and a program is as long as its Script. Comparing
- * six seconds of a sixty-second video meant drawing all sixty and cropping, which is most of the
- * cost of a comparison round.
- *
- * The cut is a **transformation of the original text, not a rewrite**: every element that survives is
- * copied out of the Source byte for byte, at the range the parse gave it. Nothing is authored here,
- * so the fragment cannot describe something the Source does not.
- *
- * What decides survival is reference integrity, not distance from the element being looked at.
- * Following inputs backwards from the target would keep what it depends on and drop everything else —
- * and the Tracks that can collide with it on screen are exactly the ones it has no edge to. A caption
- * sliced that way hangs over nothing, which is the shape of the defect this whole path exists to
- * catch. So instead: keep the Segment, drop the rest of the Script, and then drop only what can no
- * longer resolve — an element naming a Segment, Selection or Moment that is gone, and anything
- * naming an element dropped for that reason, to a fixed point. A peer bound inside the kept Segment
- * survives by construction, because its binding still resolves.
- */
-
-export type SliceResult = {
- /** The fragment, verbatim from the original except for what was removed. */
- readonly text: string;
- /** Element ids left out, and the name each one could no longer reach. */
- readonly dropped: readonly { readonly id: string; readonly unresolved: string }[];
- readonly keptSegment: string;
-};
-
-type Element = {
- readonly id: string | undefined;
- readonly start: number;
- readonly end: number;
- readonly depth: number;
- readonly text: string;
-};
-
-/**
- * Every element in the document with its byte range, including children. A tag is either
- * self-closing or closed by name; comments and the Script's own prose are skipped, since a `<` inside
- * spoken words is not a tag.
- */
-function elements(source: string, skip: readonly (readonly [number, number])[]): Element[] {
- const found: Element[] = [];
- const open: { name: string; start: number; depth: number }[] = [];
- const inside = (index: number): boolean => skip.some(([from, to]) => index >= from && index < to);
- const pattern = /<(\/?)([a-z][a-z0-9-]*(?::[A-Za-z][A-Za-z0-9]*)?)\b([^>]*?)(\/?)>/gsu;
- for (const match of source.matchAll(pattern)) {
- const index = match.index;
- if (inside(index)) continue;
- const whole = match[0];
- const closing = match[1]!;
- const name = match[2]!;
- const attributes = match[3]!;
- const selfClosing = match[4]!;
- if (closing === "/") {
- const last = open.pop();
- if (last !== undefined && last.name === name) {
- found.push({
- id: undefined, start: last.start, end: index + whole.length, depth: last.depth,
- text: source.slice(last.start, index + whole.length),
- });
- }
- continue;
- }
- if (selfClosing === "/") {
- found.push({
- id: /\bid="([^"]+)"/u.exec(attributes)?.[1], start: index, end: index + whole.length,
- depth: open.length, text: whole,
- });
- continue;
- }
- open.push({ name, start: index, depth: open.length });
- }
- // A closed element's id has to be read back off its own opening tag.
- return found.map((element) => element.id === undefined
- ? { ...element, id: /^<[^>]*?\bid="([^"]+)"/su.exec(element.text)?.[1] }
- : element);
-}
-
-/** The Script names an element reaches for, and the element ids it reaches for. */
-function references(text: string): { readonly story: Set<string>; readonly ids: Set<string> } {
- const story = new Set<string>();
- const ids = new Set<string>();
- // A Script name is reached through more than one space — `story.selection.x` in word space and
- // `story.caption.selection.x` in caption space name the same Selection. Matching only the first
- // leaves an element holding a dead name in caption space, and the cut looks clean until the graph
- // refuses it.
- for (const match of text.matchAll(/\{story\.(?:[a-z]+\.)*?(segment|selection|moment)\.([A-Za-z0-9_-]+)/gu)) {
- story.add(`${match[1]!}.${match[2]!}`);
- }
- for (const match of text.matchAll(/\{([A-Za-z][A-Za-z0-9_-]*)[.}]/gu)) {
- const id = match[1]!;
- if (id !== "story") ids.add(id);
- }
- return { story, ids };
-}
-
-/**
- * @param source the Author SVML
- * @param segment the Segment to keep, named in the Script
- */
-export function sliceSource(source: string, segment: string): SliceResult {
- const open = /<script\b[^>]*>/u.exec(source);
- if (open === null) throw new Error("the Source declares no <script>");
- const bodyStart = open.index + open[0].length;
- const bodyEnd = source.indexOf("</script>", bodyStart);
- if (bodyEnd < 0) throw new Error("the Source's <script> is not closed");
-
- // The Segment to keep, taken out of the Script whole. A Segment is one element in the Script body,
- // so this is a range, and what it contains — its Selections, its Moments — comes with it.
- const body = source.slice(bodyStart, bodyEnd);
- const kept = new RegExp(`<${segment}\\b[^>]*>.*?</${segment}>`, "su").exec(body);
- if (kept === null) throw new Error(`the Script has no <${segment}> Segment`);
-
- /**
- * A Selection may open in one Segment and close in another — the Script says so and the parser
- * records each end's own Segment. Cutting the Script to one Segment left the surviving end of such a
- * Selection with no partner, and the fragment was refused outright with SCRIPT_SELECTION_UNCLOSED:
- * not the element dropped, the whole render.
- *
- * The end that is missing is supplied at the fragment's own edge. What that expresses is exactly
- * what the Selection covers inside this stretch — from its mark to the end of the Segment, or from
- * the start of the Segment to its mark — which is what a render over this stretch has to show.
- */
- // The end of a name has to be asserted, not merely looked past: a lookahead alone backtracks, and
- // `@title-out!` was read as an open called `title-ou` with a `t` after it.
- const opened = new Set([...kept[0].matchAll(/(?<![\w/])@([a-z][a-z0-9-]*)(?![a-z0-9!-])/gu)].map((match) => match[1]!));
- const closed = new Set([...kept[0].matchAll(/@\/([a-z][a-z0-9-]*)~?/gu)].map((match) => match[1]!));
- const marked = new Set([...kept[0].matchAll(/@([a-z][a-z0-9-]*)!/gu)].map((match) => match[1]!));
- let fragment = kept[0];
- for (const id of opened) {
- if (closed.has(id) || marked.has(id)) continue;
- const end = fragment.lastIndexOf("</");
- fragment = `${fragment.slice(0, end)} @/${id}\n ${fragment.slice(end)}`;
- closed.add(id);
- }
- for (const id of closed) {
- if (opened.has(id)) continue;
- // After the Segment's own opening tag and the speaker tag that follows it, which is where the
- // Segment's first spoken word begins.
- const speaker = /^<[^>]*>\s*<[^>]*>/su.exec(fragment);
- const at = speaker === null ? fragment.indexOf(">") + 1 : speaker[0].length;
- fragment = `${fragment.slice(0, at)} @${id}${fragment.slice(at)}`;
- opened.add(id);
- }
-
- // Which Script names survive: the Segment itself, and every Selection and Moment marked inside it.
- const alive = new Set<string>([`segment.${segment}`]);
- for (const match of fragment.matchAll(/@\/?([a-z][a-z0-9-]*)\b/gu)) {
- alive.add(`selection.${match[1]!}`);
- alive.add(`moment.${match[1]!}`);
- }
-
- // Which Script names are marked after the kept Segment ends.
- //
- // A name outside the cut is not always a name the element can do without. An element given the span
- // it occupies and a Moment to cut it — `during="program" until={story.moment.done}` — is on screen
- // for the whole of any stretch that ends before that word. Dropping it made the one element a
- // reference is about unrenderable over every Segment but the last, while the round asks for it to be
- // compared over each of them. Marked before the cut it really is gone, and marked inside it survives
- // already, so this is the one case: a close that has not arrived yet, which the fragment reaches by
- // no longer closing.
- const later = new Set<string>();
- for (const match of body.slice(kept.index + kept[0].length).matchAll(/@\/?([a-z][a-z0-9-]*)\b/gu)) {
- later.add(`selection.${match[1]!}`);
- later.add(`moment.${match[1]!}`);
- }
- /** The `until=` attribute naming a name the cut left behind, when dropping it is enough. */
- const closesLater = (tag: string, name: string): string | undefined => {
- if (!later.has(name)) return undefined;
- const written = new RegExp(`\\suntil=\\{story\\.(?:[a-z]+\\.)*?${name.replace(".", "\\.")}\\}`, "u").exec(tag);
- return written?.[0];
- };
-
- const skip: (readonly [number, number])[] = [[bodyStart, bodyEnd]];
- for (const comment of source.matchAll(/<!--.*?-->/gsu)) skip.push([comment.index, comment.index + comment[0].length]);
- const all = elements(source, skip);
-
- // Drop to a fixed point: an element that names something gone is gone, and so is one that names it.
- //
- // What an element names is on its own opening tag. Reading its whole text instead makes a container
- // inherit its children's references, so a Speech Track dies of one dead Take and takes the Film
- // with it. Children are elements too and answer for themselves — which is also why the set is keyed
- // by position: a `<speech:Take>` has no id, and still has to be droppable on its own.
- const opening = (element: Element): string => /^<[^>]*>/su.exec(element.text)?.[0] ?? element.text;
- const dropped = new Map<number, string>();
- /** Opening tags rewritten by the cut, keyed by where the element starts. */
- const reopened = new Map<number, string>();
- const byId = new Map(all.flatMap((element) => element.id === undefined ? [] : [[element.id, element] as const]));
- const deadIds = new Set<string>();
- for (;;) {
- let changed = false;
- for (const element of all) {
- if (dropped.has(element.start)) continue;
- const tag = reopened.get(element.start) ?? opening(element);
- const { story, ids } = references(tag);
- const missingStory = [...story].find((name) => !alive.has(name));
- if (missingStory !== undefined) {
- // A close that has not arrived in this stretch is removed rather than fatal: the element keeps
- // the span it was given and simply does not end inside the fragment.
- const close = closesLater(tag, missingStory);
- if (close !== undefined) {
- reopened.set(element.start, tag.replace(close, ""));
- changed = true;
- continue;
- }
- dropped.set(element.start, `story.${missingStory}`);
- if (element.id !== undefined) deadIds.add(element.id);
- changed = true;
- continue;
- }
- const missingId = [...ids].find((id) => deadIds.has(id) && byId.has(id));
- if (missingId !== undefined) {
- dropped.set(element.start, missingId);
- if (element.id !== undefined) deadIds.add(element.id);
- changed = true;
- }
- }
- // A container emptied by the cut goes with its children. A Track written with Items is refused
- // when it has none, and the refusal names the Track rather than the Segment that took its Items
- // away, so leaving it in turns one clean cut into a graph error a reader has to trace back.
- for (const element of all) {
- if (dropped.has(element.start)) continue;
- const children = all.filter((other) =>
- other !== element && other.start > element.start && other.end <= element.end);
- if (children.length === 0) continue;
- if (children.some((child) => !dropped.has(child.start))) continue;
- dropped.set(element.start, "every child was cut");
- if (element.id !== undefined) deadIds.add(element.id);
- changed = true;
- }
- if (!changed) break;
- }
-
- // Rebuild: the Script body becomes the one Segment, and every dropped element's range is removed.
- // Only the outermost dropped element is cut, since cutting a child of a cut parent removes text
- // twice and leaves the document short of a closing tag.
- const cuts = all
- .filter((element) => dropped.has(element.start))
- .filter((element, _, list) => !list.some((other) =>
- other !== element && other.start <= element.start && other.end >= element.end
- && !(other.start === element.start && other.end === element.end)))
- .sort((left, right) => left.start - right.start);
-
- // A rewritten opening tag replaces its own bytes, in the same pass and in the same order as a cut,
- // so the two never disagree about where the cursor is. A tag inside a range being cut is not
- // reached: the cut takes it with everything else.
- const edits = [
- ...cuts.map((element) => ({ start: element.start, end: element.end, text: undefined as string | undefined })),
- ...[...reopened].flatMap(([start, tag]) => {
- const element = all.find((candidate) => candidate.start === start);
- if (element === undefined || dropped.has(start)) return [];
- const length = (/^<[^>]*>/su.exec(element.text)?.[0] ?? element.text).length;
- return [{ start, end: start + length, text: tag }];
- }),
- ].sort((left, right) => left.start - right.start);
-
- let text = "";
- let cursor = 0;
- for (const edit of edits) {
- if (edit.start < cursor) continue;
- text += source.slice(cursor, edit.start);
- if (edit.text !== undefined) {
- text += edit.text;
- cursor = edit.end;
- continue;
- }
- cursor = edit.end;
- // Take the rest of the line with it, so a removal never leaves a blank indented line behind.
- while (source[cursor] === " " || source[cursor] === "\t") cursor += 1;
- if (source[cursor] === "\n") cursor += 1;
- }
- text += source.slice(cursor);
-
- // The Script body is replaced last, in the rebuilt text, since every range above indexes the
- // original. Its content is unique enough to find again.
- const rebuiltOpen = /<script\b[^>]*>/u.exec(text);
- if (rebuiltOpen === null) throw new Error("the Script was removed by the cut");
- const rebuiltStart = rebuiltOpen.index + rebuiltOpen[0].length;
- const rebuiltEnd = text.indexOf("</script>", rebuiltStart);
- text = `${text.slice(0, rebuiltStart)}\n ${fragment}\n ${text.slice(rebuiltEnd)}`;
-
- return {
- text,
- dropped: all.flatMap((element) => dropped.has(element.start)
- ? [{ id: element.id ?? opening(element).slice(0, 40), unresolved: dropped.get(element.start)! }]
- : []),
- keptSegment: segment,
- };
-}
diff --git a/packages/reference-video-tools/src/tools.ts b/packages/reference-video-tools/src/tools.ts
index 364a8bc1..56673881 100644
--- a/packages/reference-video-tools/src/tools.ts
+++ b/packages/reference-video-tools/src/tools.ts
@@ -81,7 +81,9 @@ export type CompareReconstructionInput = {
readonly shot_id?: string;
readonly segment?: string;
readonly selection?: string;
- /** The Run whose Author SVML the words are read from. Required with `segment` or `selection`. */
+ /** A half-open token range, for a stretch the Script never named. See `StandInFocus`. */
+ readonly tokens?: readonly [number, number];
+ /** The Run whose Author SVML the words are read from. Required with a word range. */
readonly run?: string;
/** A rendered still. Exactly one of `image_path` and `video_path` is given. */
readonly image_path?: string;
@@ -130,6 +132,8 @@ export type ReviewElementInput = {
/** The window the render covers, named in words. Exactly one is given. */
readonly segment?: string;
readonly selection?: string;
+ /** A half-open token range, for a stretch the Script never named. See `StandInFocus`. */
+ readonly tokens?: readonly [number, number];
/** The render. Exactly one of `image_path` and `video_path` is given. */
readonly image_path?: string;
readonly video_path?: string;
@@ -234,6 +238,8 @@ function stateRoot(reference: string): string {
export type ComparedRange = {
readonly segment?: string;
readonly selection?: string;
+ /** The word range, when the caller gave one directly rather than naming a marked stretch. */
+ readonly tokens?: readonly [number, number];
readonly words: string;
readonly words_start_seconds: number;
readonly words_end_seconds: number;
@@ -293,7 +299,7 @@ export type ReviewRecord = {
* reference speaks those words and how the cut was snapped to its shot boundaries, and none of that
* exists here. The gate reads the name and nothing else from either log.
*/
- readonly range?: { readonly segment?: string; readonly selection?: string };
+ readonly range?: { readonly segment?: string; readonly selection?: string; readonly tokens?: readonly [number, number] };
readonly image_path: string;
readonly image_digest: string;
readonly status: Observation["status"];
@@ -389,12 +395,19 @@ async function answeredAlready(
return undefined;
}
-/** What a look was made over, named the way it was asked for. */
+/**
+ * What a look was made over, in the form it was asked for.
+ *
+ * A word range is written as its own form because the finest thing worth looking at is not always a
+ * thing the Script named — a caption Cue is a run of words with no id — and a gate comparing a round
+ * against a plan has to be able to tell one of those from another.
+ */
function comparedStretch(record: {
readonly shot_id?: string;
- readonly range?: { readonly segment?: string; readonly selection?: string };
+ readonly range?: { readonly segment?: string; readonly selection?: string; readonly tokens?: readonly [number, number] };
}): string {
return record.shot_id
+ ?? (record.range?.tokens === undefined ? undefined : `tokens:${record.range.tokens[0]}-${record.range.tokens[1]}`)
?? (record.range?.segment === undefined ? undefined : `segment:${record.range.segment}`)
?? (record.range?.selection === undefined ? undefined : `selection:${record.range.selection}`)
?? "";
@@ -849,6 +862,7 @@ async function cutWordRange(
record: {
...(focus.segment === undefined ? {} : { segment: focus.segment }),
...(focus.selection === undefined ? {} : { selection: focus.selection }),
+ ...(focus.tokens === undefined ? {} : { tokens: focus.tokens }),
words: `${range.first.text} … ${range.last.text}`,
words_start_seconds: round(range.startSeconds),
words_end_seconds: round(range.endSeconds),
@@ -1506,8 +1520,8 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceV
}
const state = await loadState(input.reference_id);
const root = stateRoot(state.reference_id);
- const named = [input.shot_id, input.segment, input.selection].filter((value) => value !== undefined);
- assert(named.length === 1, "name exactly one of shot_id, segment and selection");
+ const named = [input.shot_id, input.segment, input.selection, input.tokens].filter((value) => value !== undefined);
+ assert(named.length === 1, "name exactly one of shot_id, segment, selection and tokens");
const supplied = [input.image_path, input.video_path].filter((value) => value !== undefined);
assert(supplied.length === 1, "supply exactly one of image_path and video_path");
const clip = input.video_path !== undefined;
@@ -1540,11 +1554,12 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceV
stretchSeconds = shot.duration_seconds;
} else {
assert(input.run !== undefined,
- "run is required with segment or selection: the words are read from the Run's Author SVML");
+ "run is required with a word range: the words are read from the Run's Author SVML");
const { path: svmlPath } = await authorSource(resolve(invokedFrom(), input.run));
const focus: StandInFocus = {
...(input.segment === undefined ? {} : { segment: input.segment }),
...(input.selection === undefined ? {} : { selection: input.selection }),
+ ...(input.tokens === undefined ? {} : { tokens: input.tokens }),
};
const range = await spokenRange(svmlPath, focus, await referenceWords(state.reference_id));
cut = await cutWordRange(state, root, focus, range, renderedPath, clip, slot);
@@ -1783,8 +1798,8 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceV
const element = input.element?.trim() ?? "";
assert(element.length > 0, "--element is required; a review with no element is credited to nothing");
- const named = [input.segment, input.selection].filter((value) => value !== undefined);
- assert(named.length === 1, "name exactly one of --segment or --selection: the window the render covers");
+ const named = [input.segment, input.selection, input.tokens].filter((value) => value !== undefined);
+ assert(named.length === 1, "name exactly one of --segment, --selection or --tokens: the window the render covers");
const supplied = [input.image_path, input.video_path].filter((value) => value !== undefined);
assert(supplied.length === 1, "supply exactly one of --image or --video");
@@ -1818,7 +1833,12 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceV
}
const digest = await fileDigest(renderedPath);
- const stretch = comparedStretch({ range: { ...(input.segment === undefined ? {} : { segment: input.segment }), ...(input.selection === undefined ? {} : { selection: input.selection }) } });
+ const range = {
+ ...(input.segment === undefined ? {} : { segment: input.segment }),
+ ...(input.selection === undefined ? {} : { selection: input.selection }),
+ ...(input.tokens === undefined ? {} : { tokens: input.tokens }),
+ };
+ const stretch = comparedStretch({ range });
// Nothing about the picture or the question changed since this was last answered, so nothing new
// can be said about it. Here this is what bounds the looking: a review costs no vendor money, so
// without it a round could re-ask the same picture until it liked the answer.
@@ -1836,7 +1856,7 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceV
const unit = asClip ? "grid of frames sampled evenly across the stretch, laid out in reading order: left to right, then top to bottom" : "still picture";
const record: ReviewRecord = {
at: startedAt, id: reviewId, element,
- range: { ...(input.segment === undefined ? {} : { segment: input.segment }), ...(input.selection === undefined ? {} : { selection: input.selection }) },
+ range,
image_path: renderedPath, image_digest: digest, status: "pending",
...(scope.length === 0 ? {} : { scope }), clip: asClip,
...(standIn === undefined ? {} : { stand_in: standIn }),
diff --git a/packages/reference-video-tools/test/plan.test.ts b/packages/reference-video-tools/test/plan.test.ts
new file mode 100644
index 00000000..0a9fabd4
--- /dev/null
+++ b/packages/reference-video-tools/test/plan.test.ts
@@ -0,0 +1,84 @@
+import assert from "node:assert/strict";
+import test from "node:test";
+
+import { scriptBody } from "../src/authoring.js";
+import { reviewPlan } from "../src/plan.js";
+
+const SVML = `<?svml using="@hypit/markup@1"?>
+<svml version="1">
+ <import as="story" from="@hypit/script@1"/>
+ <import as="media-track" from="@hypit/media-track@1"/>
+ <script id="story">
+ <one><HOST>alpha bravo @claim charlie delta @/claim echo.</one>
+ <two><HOST>foxtrot @proof golf hotel @/proof india.</two>
+ </script>
+ <media-track:Track id="broll" semantic={speech.semantic} canvas={vertical}>
+ <media-track:Item id="one" media={a.media} during={story.selection.claim}
+ frame={card-a} appearance={recipes.media.card}/>
+ <media-track:Item id="two" media={b.media} during={story.selection.proof}
+ frame={card-b} appearance={recipes.media.wide}/>
+ </media-track:Track>
+</svml>
+`;
+
+const BROLL = [{ id: "broll", tag: "Track", alias: "media-track", specifier: "@hypit/media-track" }];
+
+function planFor(svml: string) {
+ return reviewPlan({ svml, svmlPath: "main.svml", scriptBody: scriptBody(svml), drawn: BROLL });
+}
+
+// Under-covering is the silent direction: the gate passes, the round runs, and the second design is
+// simply never drawn. The two exclusion lists this key is built from will grow as packages are added,
+// and widening one by an attribute that actually decides what a picture looks like merges two groups
+// into one. That edit has no visible symptom, so it is the one worth pinning.
+test("two placements that declare different pictures are two looks, and two that declare the same are one", () => {
+ const distinct = planFor(SVML);
+ assert.equal(distinct.length, 2, "a different appearance and Frame is a different picture");
+ assert.deepEqual(distinct.map((entry) => entry.named), ["selection claim", "selection proof"]);
+
+ // Only the declaration differs between these two runs. The windows, the ids and the media stay put.
+ const same = planFor(SVML
+ .replace("appearance={recipes.media.wide}", "appearance={recipes.media.card}")
+ .replace("frame={card-b}", "frame={card-a}"));
+ assert.equal(same.length, 1, "the same declaration over different words is one design, read once");
+
+ // The material each Item names is mocked in the render this plan is for, so two Items differing
+ // only in which unbuilt picture they point at draw the identical placeholder.
+ const mocked = planFor(SVML
+ .replace("appearance={recipes.media.wide}", "appearance={recipes.media.card}")
+ .replace("frame={card-b}", "frame={card-a}")
+ .replace("media={b.media}", "media={c.media}"));
+ assert.equal(mocked.length, 1, "an unbuilt picture is a grey rectangle whichever generation names it");
+});
+
+const CAPTIONS = `<?svml using="@hypit/markup@1"?>
+<svml version="1">
+ <import as="story" from="@hypit/script@1"/>
+ <import as="caption" from="@hypit/caption@1"/>
+ <import as="caption-fine" from="@hypit/caption-fine@1"/>
+ <script id="story">
+ <one><HOST>alpha bravo charlie delta echo.</one>
+ <two><HOST>foxtrot @punch golf hotel @/punch india.</two>
+ </script>
+ <caption-fine:Style id="plain" recipe={recipes.caption.plain}/>
+ <caption-fine:Style id="loud" recipe={recipes.caption.loud}/>
+ <caption:Program id="program" document={story.caption} narrative={story} default={plain}>
+ <caption:Use style={loud} selection={story.selection.punch}/>
+ </caption:Program>
+ <caption-fine:Track id="captions" document={story.caption} semantic={speech.semantic} program={program}/>
+</svml>
+`;
+
+// A caption Track is one tag with no window on it, so reading its placements the ordinary way finds
+// one design however many the Program hands out. An override is written inside the Program, and the
+// stretch it covers is a genuinely different picture that nothing on the Track mentions — so leaving
+// it out is a design nobody looks at while the gate reports full coverage.
+test("a caption Style override is its own look, over the stretch it covers", () => {
+ const plan = reviewPlan({
+ svml: CAPTIONS, svmlPath: "main.svml", scriptBody: scriptBody(CAPTIONS),
+ drawn: [{ id: "captions", tag: "Track", alias: "caption-fine", specifier: "@hypit/caption-fine" }],
+ });
+ const overridden = plan.find((entry) => entry.named === "selection punch");
+ assert.ok(overridden, `the overridden stretch is missing; got ${JSON.stringify(plan.map((e) => e.named))}`);
+ assert.ok(plan.length >= 2, "the default and the override are two designs, not one");
+});