Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

The round's three commands hand each other their files, not the agent (#95)

rponeawae239875ecfcc
diff --git a/.agents/skills/hypit/references/authoring.md b/.agents/skills/hypit/references/authoring.md
index 3758cbdd..1a3b772e 100644
--- a/.agents/skills/hypit/references/authoring.md
+++ b/.agents/skills/hypit/references/authoring.md
@@ -11,16 +11,16 @@ Never invent a
component, attribute, child, port, Recipe property or literal value. Do not infer one package's
syntax from a neighboring package.
-Install dependencies after package selections change. Validate with the existing commands:
+Install dependencies after package selections change. Validate with one command:
```bash
-hypit check path/to/main.svml
-hypit check path/to/recipes.svs
-hypit check path/to/build.svrun
+hypit-reference-video-tools preview_check path/to/project/build.svrun
```
-Check all of them, not only the Author Source: a Recipe the Author Source references and a Target the
-Run Source demands are just as able to be wrong. Do not create a check wrapper.
+It checks every Source the Run reaches before it proves the graph traces — the Run, the Author it
+names, and the Recipe sheets and kits the Author imports — then goes on to the wiring itself. One
+`hypit check` of the Run covers that same closure, but running it separately would only report the
+same failure twice.
**A check that passes is not a graph that traces.** `hypit check` proves a Source is legal — its
syntax, its references, its graph structure — and proves nothing about whether the tracks it declares
diff --git a/.agents/skills/hypit/references/environment.md b/.agents/skills/hypit/references/environment.md
index 2bd2d454..9dc2174b 100644
--- a/.agents/skills/hypit/references/environment.md
+++ b/.agents/skills/hypit/references/environment.md
@@ -87,15 +87,10 @@ Reconstructing a video given as a link rather than as a file needs `uv` too: `yt
`services/yt-dlp` and run from there, so the version is the repository's rather than the machine's.
`prepare_reference` names what is missing when a link is passed without it.
-When the Runtime Profile selects the local WhisperX Endpoint, verify that the service answered its
-health probe before `prepare_reference` or a Build, since a first start can spend several minutes
-loading the model:
-
-```powershell
-Invoke-WebRequest -UseBasicParsing http://127.0.0.1:8765/health
-hypit programs status
-hypit doctor
-```
+When the Runtime Profile selects the local WhisperX Endpoint, `prepare_reference` probes the service's
+health itself before transcribing, because a first start can spend several minutes loading the model.
+A service that does not answer is reported in the command's result with a pointer to the repair rather
+than silently producing an empty transcript.
`host-setup.md` holds the host toolchain installations and repairs, and is read when a command
reports that a service or a binary is unavailable.
diff --git a/.agents/skills/hypit/references/local-author-package.md b/.agents/skills/hypit/references/local-author-package.md
index 34da02e7..11f8a9ee 100644
--- a/.agents/skills/hypit/references/local-author-package.md
+++ b/.agents/skills/hypit/references/local-author-package.md
@@ -237,18 +237,11 @@ Repair in this order:
5. implementation behavior;
6. source usage.
-Use the installed checks:
-
-```bash
-hypit check path/to/main.svml
-hypit check path/to/recipes.svs
-hypit check path/to/build.svrun
-```
-
A package that cannot be *wired* is not done. `hypit check` proves the Source is legal, and nothing
more; it will not tell you that the track a package produces cannot be traced to a Film at all. Run
-the preview check and repair until it passes — a graph failure is not a difference to weigh, it is
-work that is not finished, and it is not bounded by the round's attempt ceiling:
+the preview check — it checks every Source the Run reaches, then proves the graph traces — and repair
+until it passes. A graph failure is not a difference to weigh, it is work that is not finished, and it
+is not bounded by the round's attempt ceiling:
```bash
hypit-reference-video-tools preview_check path/to/build.svrun
diff --git a/.agents/skills/hypit/references/preview.md b/.agents/skills/hypit/references/preview.md
index 2eb966c4..4aa9cb50 100644
--- a/.agents/skills/hypit/references/preview.md
+++ b/.agents/skills/hypit/references/preview.md
@@ -86,11 +86,12 @@ caption Cue ends at a speaker change, so it has a range and no id. Without any o
program is drawn. An `--out` ending `.mp4`, `.mov` or `.webm` writes the stretch as a clip; any other
extension writes one still from the middle of it.
-A whole round is one call: `--batch <renders.json>`, an array of
-`{element, segment|selection|tokens, out}` inheriting the Run. **The program is drawn once for the
-whole round** and each entry is cut out of those frames, so asking for eight windows costs one render
-and eight cuts. The picture does not depend on `--element` — it is everything the Source places over
-those words — so two elements over one stretch share the render as well.
+A whole round is one call: `--batch <renders.json>`, an array of `{element, segment|selection|tokens,
+out}` inheriting the Run, and the `renders` array `reconstruction_check` returns is exactly that.
+**The program is drawn once for the whole round** and each entry is cut out of those frames, so asking
+for eight windows costs one render and eight cuts. The picture does not depend on `--element` — it is
+everything the Source places over those words — so two elements over one stretch share the render as
+well.
Each Segment's length comes from the Source's own `estimate:Speech`, which is the only clock a program
has before its speech is synthesized. The result's `timing` says what sized each Segment, and the same
diff --git a/.agents/skills/hypit/references/reconstruction/comparison-round.md b/.agents/skills/hypit/references/reconstruction/comparison-round.md
index cf0c5898..787b62be 100644
--- a/.agents/skills/hypit/references/reconstruction/comparison-round.md
+++ b/.agents/skills/hypit/references/reconstruction/comparison-round.md
@@ -33,7 +33,8 @@ lengths, and only this clock puts the render on the reference's. It is also what
appearance is a function of elapsed time — a progressive reveal, a typewriter, staggered rows — be
compared at the speed it will be seen at.
-A `--batch <renders.json>` round inherits the reference along with the Run.
+A `--batch` round inherits the reference along with the Run, and `reconstruction_check`'s output is
+the batch file.
The window is still named in words on both sides. A word range and a shot are different divisions of
the same video — one Segment routinely runs across five shots — and what a render and the reference
@@ -43,7 +44,8 @@ One geometry has to agree before any of it means anything. The reference's own p
stored under `video` in its `state.json`, and they size nothing here — but their **aspect** has to
match the Canvas's. When it does not, every comparison puts two differently-shaped pictures side by
side and the observer reports proportion differences that belong to the Canvas rather than to the
-element. Fix the Canvas before comparing.
+element. `reconstruction_check` refuses with a `canvas_aspect` entry and a false `passed` until the
+Canvas matches the reference's shape.
## Compare the whole stretch, against every stretch the element is drawn over
@@ -77,27 +79,17 @@ stretch per distinct Style, at its first occurrence.
Where the reference shows a caption behaving differently somewhere, that is a stretch worth its own
comparison; where it shows the same design drawn over different words, it is not.
-On the `gemini` observer the whole round is one call. Write the comparisons to a JSON file — the same
-objects the flags produce, minus `reference_id` — and hand the file over:
-
-```json
-[
- {"run": "projects/<name>/build.svrun", "segment": "pro",
- "video_path": "projects/<name>/renders/board-pro.mp4", "element": "board"},
- {"run": "projects/<name>/build.svrun", "selection": "wait",
- "video_path": "projects/<name>/renders/marks-wait.mp4", "element": "marks"},
- {"run": "projects/<name>/build.svrun", "tokens": [48, 52],
- "video_path": "projects/<name>/renders/captions-48-52.mp4", "element": "captions"}
-]
-```
-
-`reconstruction_check`'s `plan` names its windows as `tokens`, so that is the form most of a round
-arrives in. It is two whole numbers, `[from, to)`, not the `"48:52"` the flag takes — a string is
-refused rather than read as its first two characters. Each entry carries its own `run`; only
-`--reference-id` is inherited from the flag.
+On the `gemini` observer the whole round is one call. The comparisons arrive from the check, not from
+a file written by hand: `reconstruction_check`'s output carries a `comparisons` array — one entry per
+stretch still owed, each already holding `run`, `tokens`, `video_path` and `element` — and that
+output is the batch file. `plan` names its windows as `tokens`, which is the form the batch takes: two
+whole numbers, `[from, to)`, not the `"48:52"` the flag takes (a string is refused rather than read as
+its first two characters). Each entry carries its own `run`; only `--reference-id` is inherited from
+the flag.
```
-hypit-reference-video-tools compare_reconstruction --reference-id <id> --batch comparisons.json
+hypit-reference-video-tools reconstruction_check projects/<name>/build.svrun --reference-id <id> > round.json
+hypit-reference-video-tools compare_reconstruction --reference-id <id> --batch round.json
```
It paces the requests itself, keeps each comparison's derived cuts apart, and returns them under
@@ -160,7 +152,7 @@ That spends the expensive half of the route — the renders and the observer —
rather than on learning anything about the reference.
So **a repair is not verified here**, and the honest place to say so is the report. Passing
-`pnpm check` and `hypit check` is the precondition for the round, not a result of it. The one thing
+`preview_check` is the precondition for the round, not a result of it. The one thing
that is not bounded by the round is settling disagreeing evidence: a narrow
`observe_reference --question` does that at any point, which is what `evidence.md` is for.
diff --git a/.agents/skills/hypit/references/reconstruction/route.md b/.agents/skills/hypit/references/reconstruction/route.md
index f22306e6..8bba2541 100644
--- a/.agents/skills/hypit/references/reconstruction/route.md
+++ b/.agents/skills/hypit/references/reconstruction/route.md
@@ -235,12 +235,11 @@ deliverable even though this route runs nothing: without it the first thing the
Do not author one source fragment per shot.
-### 16. Check every source
+### 16. Check every source, then prove the graph traces
-`../authoring.md` holds the check set — every command, not only the Author Source — and why no wrapper
-is written for it.
-
-### 17. Prove the graph traces
+One command does both: it checks every Source the Run reaches — the Run, the Author it names, the
+Recipe sheets and kits the Author imports — and then proves the graph traces. A Source that does not
+check is refused before tracing, with the file and position.
```bash
hypit-reference-video-tools preview_check /path/to/project/build.svrun
@@ -248,7 +247,7 @@ hypit-reference-video-tools preview_check /path/to/project/build.svrun
**Read now:** `../preview.md`, which says which refusal is the pass. A Source that declares its
generations rather than performing them is the state every reconstruction is in when its sources are
-first written. Neither this gate nor step 16 is bounded by the round's attempt ceilings.
+first written. This gate is not bounded by the round's attempt ceilings.
---
@@ -262,16 +261,28 @@ element — that is the first moment the round has something to render.
### 19. Ask the check what to render, then render it
`reconstruction_check` returns a `plan`: each entry an element and a word range, with the reason it
-earned a look. That is the round — it is not yours to work out from the Source.
+earned a look, and a picture path it will be rendered to. That is the round — it is not yours to work
+out from the Source.
+
+Its output is already a `render_element --batch` file: `renders` is in it. So:
+
+```bash
+hypit-reference-video-tools reconstruction_check projects/<name>/build.svrun --reference-id <id> > round.json
+hypit-reference-video-tools render_element projects/<name>/build.svrun --batch round.json --reference-id <id>
+```
+
+`comparison-round.md` says to pass `--reference-id` every time so the stand-in runs on the
+reference's clock. The program is drawn once and every entry is cut out of those frames, so render
+the whole list in one `--batch` call.
-`../preview.md` holds `render_element`; `comparison-round.md` says to pass `--reference-id` every time
-so the stand-in runs on the reference's clock. The program is drawn once and every entry is cut out of
-those frames, so render the whole list in one `--batch` call.
+### 20. Send every comparison at once
-### 20. Send every comparison for the element at once
+The same file is a `compare_reconstruction --batch` file too — `comparisons` is in it, and each entry
+carries `--element` already, so step 22 can credit it. Nothing is written between the two commands.
-`comparison-round.md` holds `compare_reconstruction` and its batch form. **Pass `--element <id>` every
-time**, or step 22 credits the comparison to nothing.
+```bash
+hypit-reference-video-tools compare_reconstruction --reference-id <id> --batch round.json
+```
### 21. Repair against the differences that round returned
diff --git a/packages/reference-video-tools/src/checks.ts b/packages/reference-video-tools/src/checks.ts
index dd114d8c..162e9b06 100644
--- a/packages/reference-video-tools/src/checks.ts
+++ b/packages/reference-video-tools/src/checks.ts
@@ -1,4 +1,5 @@
import { readFile, readdir } from "node:fs/promises";
+import { spawnSync } from "node:child_process";
import { dirname, join, resolve, sep } from "node:path";
import { markupSurfaceHostFacetAbi } from "@hypit/markup";
@@ -15,8 +16,9 @@ import type { StudioSession } from "@hypit/studio/src/session.js";
import { inspectStudioRun } from "@hypit/studio/src/studio-preflight.js";
import { aliasPattern, invokedFrom, nearestPackageRoot, referenceRoot, scriptBody } from "./authoring.js";
-import { assert, round } from "./media.js";
+import { assert, readJson, round } from "./media.js";
import { reviewPlan } from "./plan.js";
+import type { ReferenceState } from "./types.js";
export type PreviewCheckInput = {
readonly run: string;
@@ -25,6 +27,43 @@ export type PreviewCheckInput = {
const AWAITING = "the Studio projection closure requires unresolved capabilities:";
+/**
+ * Every Source the Run reaches, checked before anything is traced.
+ *
+ * One `hypit check` of the Run covers the closure: it compiles the Author the Run names, the Author's
+ * `<import>`ed Recipe sheets and kits along with it, and it typechecks the project's own author
+ * packages. A break in a kit is reported as `kits/street-interview-v1.svs:5993`, with the path the
+ * file has from the project — so one check reads more than a list of them would and says where in the
+ * same breath.
+ */
+async function checkedSources(runPath: string): Promise<{ readonly ok: boolean; readonly output: string }> {
+ // The CLI as it was launched, when this process was launched by it, and the Distribution's own
+ // otherwise — `hypit-reference-video-tools` does not set the launcher variable, and the Distribution
+ // root is where `bin/` sits in a checkout and in an install alike.
+ const launcher = process.env.HYPIT_CLI_LAUNCHER?.trim();
+ const distributionRoot = videoCliDistribution.packageRoot;
+ if ((launcher === undefined || launcher.length === 0) && distributionRoot === undefined) {
+ throw new Error("active Hypit Distribution has no package root");
+ }
+ const hypit = launcher !== undefined && launcher.length > 0
+ ? launcher
+ : join(distributionRoot!, "bin", "hypit.mjs");
+
+ // The project is the package root, the way `hypit check` finds it when it is run from there. A
+ // project inside a larger tree resolves none of its own `packages/local-*` without this.
+ const checked = spawnSync(process.execPath, [hypit, "check", runPath, "--package-root", dirname(runPath)], {
+ encoding: "utf8", windowsHide: true, timeout: 600_000,
+ });
+ // Node writes its own deprecation warnings to the same stream the refusal arrives on, and this
+ // output is read as the reason a check failed. Two lines about `module.register()` above the file
+ // and position are two lines between the reader and the thing they came for.
+ const output = `${checked.stdout ?? ""}${checked.stderr ?? ""}`
+ .split("\n")
+ .filter((line) => !/^\(node:\d+\)/u.test(line) && !line.startsWith("(Use `node --trace-"))
+ .join("\n").trim();
+ return { ok: checked.status === 0, output };
+}
+
/**
* Preview-check a reconstruction before it is delivered.
*
@@ -52,6 +91,12 @@ export async function previewCheck(
): Promise<Record<string, unknown>> {
const cwd = invokedFrom();
const runPath = resolve(cwd, input.run);
+
+ // The sources first, then whether the graph traces. A Source that does not check has nothing useful
+ // to say about tracing, and its refusal names the file and the position it is at; the same mistake
+ // met while tracing arrives as whatever the compiler happened to fail on afterwards.
+ const sources = await checkedSources(runPath);
+ if (!sources.ok) return { run: runPath, sound: false, sources, refused: sources.output };
const runtimePath = input.runtime === undefined ? undefined : resolve(cwd, input.runtime);
const packageRoot = nearestPackageRoot(dirname(runPath)) ?? roots.packageRoot;
const workspaceRoot = dirname(runPath);
@@ -96,7 +141,7 @@ export async function previewCheck(
await archive?.close();
}
- if (refusal !== undefined) return { run: runPath, sound: false, refused: refusal };
+ if (refusal !== undefined) return { run: runPath, sound: false, sources, refused: refusal };
if (awaiting !== undefined) {
// The graph traced all the way to a Film and a semantic spine; what is left is
@@ -106,7 +151,8 @@ export async function previewCheck(
return {
run: runPath,
sound: true,
- summary: `the graph is sound, waiting on ${awaiting.length} ${noun}.`,
+ sources,
+ summary: `every source checks; the graph is sound, waiting on ${awaiting.length} ${noun}.`,
awaiting,
};
}
@@ -120,9 +166,15 @@ export async function previewCheck(
}
if (problems.length === 0) {
- return { run: runPath, sound: true, summary: `every Track resolved (${tracks.length}).`, tracks: tracks.length };
+ return {
+ run: runPath,
+ sound: true,
+ sources,
+ summary: `every source checks; every Track resolved (${tracks.length}).`,
+ tracks: tracks.length,
+ };
}
- return { run: runPath, sound: false, problems };
+ return { run: runPath, sound: false, sources, problems };
}
export type ReconstructionCheckInput = {
@@ -838,7 +890,20 @@ export async function authoringCheck(
if (text === undefined) continue;
for (const recipe of text.matchAll(/([A-Za-z0-9_.-]+)\s*\{([^}]*)\}/gu)) recipeBodies.set(recipe[1] ?? "", recipe[2] ?? "");
}
- const plan = reviewPlan({ svml, svmlPath, scriptBody: scriptBody(svml), drawn, recipes: recipeBodies });
+ // Where each stretch's picture goes, named here rather than by whoever reads this.
+ //
+ // A round is three commands — this one, `render_element --batch`, `compare_reconstruction --batch` —
+ // and the middle one writes exactly what the last one reads. A path invented at the call site has to
+ // survive being written down twice and stay identical; a path named here survives by never being
+ // written down at all. `renders/` beside the Run is where a project already keeps them.
+ //
+ // Named for the range and not for `named`: the range is what identifies an entry, so two entries of
+ // one element always differ in it, while `named` is a sentence written to be read.
+ const plan = reviewPlan({ svml, svmlPath, scriptBody: scriptBody(svml), drawn, recipes: recipeBodies })
+ .map((entry) => ({
+ ...entry,
+ out: join(dirname(runPath), "renders", `${entry.element}-${entry.tokens[0]}-${entry.tokens[1]}.mp4`),
+ }));
// The same parse the plan was built from, so a look recorded against a Segment can be resolved to
// the words that Segment marks and compared with a plan entry on one axis.
const plannedBody = scriptBody(svml);
@@ -868,6 +933,36 @@ export async function authoringCheck(
assert(prepared.includes(reference), `reference ${reference} is not prepared under ${preparedRoot}`);
}
+ // The reference's shape against the Canvas's.
+ //
+ // The reference's own pixel dimensions size nothing on this route — a render is drawn at the
+ // Canvas's size — but their proportion has to agree. A Canvas of another shape puts two
+ // differently-proportioned pictures in front of the observer, and the differences it reports are
+ // the ones the Canvas made rather than the ones the element made. It is the precondition that
+ // fails quietly: every comparison in the round succeeds, and every one of them is about the wrong
+ // thing, which is why it is a refusal here rather than a note.
+ const aspect: { readonly canvas: string; readonly canvas_size: string; readonly reference_size: string }[] = [];
+ if (mode === "reconstruction" && reference !== undefined) {
+ const video = (await readJson<ReferenceState>(join(preparedRoot, reference, "state.json")))?.video;
+ if (video !== undefined && video.width > 0 && video.height > 0) {
+ const wanted = video.width / video.height;
+ for (const [id, box] of frameGeometry(svml).canvases) {
+ const width = box.right - box.left;
+ const height = box.bottom - box.top;
+ if (!(width > 0 && height > 0)) continue;
+ // One percent, so a Canvas that is the reference's shape at another size agrees: 1440x2560 and
+ // 1080x1920 are both 9:16, and a reference whose own dimensions are odd is within it. Nothing
+ // wider — 1080x1920 against 1080x1440 is the case this exists to catch.
+ if (Math.abs(width / height - wanted) <= wanted * 0.01) continue;
+ aspect.push({
+ canvas: id,
+ canvas_size: `${width}x${height}`,
+ reference_size: `${video.width}x${video.height}`,
+ });
+ }
+ }
+ }
+
type LoggedComparison = {
readonly element?: string;
readonly shot_id?: string;
@@ -1052,14 +1147,42 @@ export async function authoringCheck(
run: runPath,
...(reference === undefined ? {} : { reference_id: reference }),
plan,
+ // The round as the two commands that run it already take it. `render_element --batch` reads
+ // `renders` out of the file it is given and `compare_reconstruction --batch` reads `comparisons`,
+ // so this command's own output is a batch file for both and the round is three commands with
+ // nothing written by hand in between. Both are the stretches still owed rather than the whole
+ // plan: what has already been looked at does not want rendering again.
+ ...(owed.length === 0 ? {} : {
+ renders: owed.map((entry) => ({ element: entry.element, tokens: entry.tokens, out: entry.out })),
+ }),
+ // Only where there is a reference to compare against. The description route's second command is
+ // `review_element`, whose entries carry an `intent_file` saying what the element was asked to be,
+ // and that is written rather than derived.
+ ...(owed.length === 0 || mode !== "reconstruction" ? {} : {
+ comparisons: owed.map((entry) => ({
+ run: runPath, tokens: entry.tokens, video_path: entry.out, element: entry.element,
+ })),
+ }),
// Where this command actually read and resolved from. Said here, no document has to describe it
// from the outside and go stale when it moves.
roots: mode === "reconstruction"
? { reference: preparedRoot, packages: packageRoot }
: { reviews: logPath, packages: packageRoot },
- passed: owed.length === 0 && never.length === 0 && running.length === 0 && coverage.gaps.length === 0 && unresolved.length === 0,
+ passed: owed.length === 0 && never.length === 0 && running.length === 0 && coverage.gaps.length === 0
+ && unresolved.length === 0 && aspect.length === 0,
summary,
elements,
+ ...(aspect.length === 0 ? {} : {
+ canvas_aspect: {
+ entries: aspect,
+ note: "The reference and the Canvas are different shapes. Every comparison in a round puts the "
+ + "reference clip beside a render drawn at the Canvas's proportions, so the observer reads the "
+ + "difference between the two shapes and reports it as the element's — a caption that is the "
+ + "right size looks wrong, and a repair made against that reading moves it away from the "
+ + "reference. Change the Canvas to the reference's shape before comparing anything. The "
+ + "reference's own pixel dimensions are not the requirement; its proportion is.",
+ },
+ }),
...(owed.length === 0 ? {} : {
owed: {
entries: owed,
@@ -1069,9 +1192,9 @@ export async function authoringCheck(
+ "window long enough that an animation inside it behaves differently.",
commands: owed.map((entry) => mode === "reconstruction"
? `hypit-reference-video-tools compare_reconstruction --reference-id ${reference} --run ${runPath} `
- + `--tokens ${entry.tokens[0]}:${entry.tokens[1]} --video <rendered clip>.mp4 --element ${entry.element}`
+ + `--tokens ${entry.tokens[0]}:${entry.tokens[1]} --video ${entry.out} --element ${entry.element}`
: `hypit-reference-video-tools review_element --run ${runPath} `
- + `--tokens ${entry.tokens[0]}:${entry.tokens[1]} --video <rendered clip>.mp4 --element ${entry.element} `
+ + `--tokens ${entry.tokens[0]}:${entry.tokens[1]} --video ${entry.out} --element ${entry.element} `
+ `--intent-file <what ${entry.element} was asked to be>`),
},
}),
@@ -1088,13 +1211,16 @@ export async function authoringCheck(
+ "render the element as the Source configures it, mock the layers a Build has not made, "
+ "and have it read against what this element was asked to be — one review per Segment or "
+ "Selection it is drawn over.",
- commands: never.map((element) => mode === "reconstruction"
- ? `hypit-reference-video-tools compare_reconstruction --reference-id ${reference} --run ${runPath} `
- + `--segment <each Segment ${element.id} is drawn over> `
- + `--video <rendered clip>.mp4 --element ${element.id}`
- : `hypit-reference-video-tools review_element --run ${runPath} `
- + `--segment <each Segment ${element.id} is drawn over> `
- + `--video <rendered clip>.mp4 --element ${element.id} --intent-file <what ${element.id} was asked to be>`),
+ // One command per stretch the plan gives this element, rather than one per element naming the
+ // Segments it is drawn over. The plan already decided which stretches this element earns and
+ // named a picture for each, so both of the things a reader used to fill in are here.
+ commands: never.flatMap((element) => plan.filter((entry) => entry.element === element.id)
+ .map((entry) => mode === "reconstruction"
+ ? `hypit-reference-video-tools compare_reconstruction --reference-id ${reference} --run ${runPath} `
+ + `--tokens ${entry.tokens[0]}:${entry.tokens[1]} --video ${entry.out} --element ${element.id}`
+ : `hypit-reference-video-tools review_element --run ${runPath} `
+ + `--tokens ${entry.tokens[0]}:${entry.tokens[1]} --video ${entry.out} --element ${element.id} `
+ + `--intent-file <what ${element.id} was asked to be>`)),
},
}),
...(awaiting.length === 0 ? {} : {
@@ -1159,9 +1285,10 @@ export async function authoringCheck(
+ "it carries, so each window runs for as long as the reference spends on it. What is left after "
+ "that is alignment against the speech the Build synthesizes, which is settled once that "
+ "speech exists.",
- commands: untimed.map((element) =>
- `hypit-reference-video-tools render_element ${runPath} --element ${element.id} `
- + `--reference-id ${reference} --segment <the Segment the shot covers> --out <rendered clip>.mp4`),
+ commands: untimed.flatMap((element) => plan.filter((entry) => entry.element === element.id)
+ .map((entry) =>
+ `hypit-reference-video-tools render_element ${runPath} --element ${element.id} `
+ + `--reference-id ${reference} --tokens ${entry.tokens[0]}:${entry.tokens[1]} --out ${entry.out}`)),
},
}),
uncovered: coverage.gaps,
diff --git a/packages/reference-video-tools/src/cli.ts b/packages/reference-video-tools/src/cli.ts
index 40012d5a..1c87e6e0 100644
--- a/packages/reference-video-tools/src/cli.ts
+++ b/packages/reference-video-tools/src/cli.ts
@@ -31,6 +31,7 @@ function usage(): string {
" hypit-reference-video-tools observe_reference --reference-id <id> --shot-id <id> [--shot-id <id> ...] --question <text>",
" hypit-reference-video-tools observe_reference --reference-id <id> --batch <questions.json>",
" hypit-reference-video-tools record_observation --reference-id <id> --key <key> --text <text>|--text-file <path>",
+ " hypit-reference-video-tools record_observation --reference-id <id> --batch <answers.json>",
" hypit-reference-video-tools inspect_svml_vocabulary --package <name> [--package <name> ...] [--tag <tag> ...] [--without-previews]",
" hypit-reference-video-tools inspect_visual_contract [--shape visual-track|text-flow|text-typography|text-paint|text-document|path-command] [--producers-of <package> ...]",
" hypit-reference-video-tools paths",
@@ -142,6 +143,12 @@ function usage(): string {
"what makes it count toward coverage; until then reconstruction_check lists it under",
"`awaiting_answer`.",
"",
+ "--batch records a whole sweep's answers at once. The file holds a JSON array — or an object with an",
+ "`answers` array — of `{key, text}`, or `{key, text_file}` where the answer was written to a file:",
+ "`[{\"key\": \"visual:shot-003\", \"text_file\": \"answers/shot-003.md\"}, …]`. Paths are read against the",
+ "working directory. They are applied in the order they are written, and one that fails takes only",
+ "itself down and arrives under `failures`; the rest are under `records`.",
+ "",
"review_element is the counterpart on the route with no reference: it reads one rendered element",
"against what that element was asked to be, rather than against a video to copy. --intent-file",
"carries the author's own words and is required; --element is required too, so a review credits an",
@@ -299,15 +306,33 @@ async function main(): Promise<void> {
};
result = await tools.observe_reference(input as { reference_id: string; shot_ids?: readonly string[]; question?: string; reobserve?: boolean });
} else if (command === "record_observation") {
- // An observation is paragraphs of prose. --text keeps a short one on the command line; --text-file
- // is how a long one arrives without the shell deciding where it ends.
- const textFile = one(flags, "text-file");
- const input = supplied ?? {
- reference_id: required(flags, "reference-id"),
- key: required(flags, "key"),
- text: textFile === undefined ? required(flags, "text") : await readFile(textFile, "utf8"),
- };
- result = await tools.record_observation(input as { reference_id: string; key: string; text: string });
+ // A whole sweep's answers at once. Each entry carries its own text, either inline or as a file the
+ // answer was written to, which is read here the way --text-file is.
+ const answerFile = one(flags, "batch");
+ if (answerFile !== undefined) {
+ const parsed: unknown = JSON.parse(await readFile(answerFile, "utf8"));
+ const list = Array.isArray(parsed) ? parsed : (parsed as { answers?: unknown }).answers;
+ if (!Array.isArray(list)) throw new Error(`${answerFile} must hold a JSON array of answers, or an object with an "answers" array`);
+ const answers = await Promise.all((list as { key?: unknown; text?: unknown; text_file?: unknown }[]).map(async (entry) => {
+ if (typeof entry.key !== "string") throw new Error(`every answer needs a "key"; received ${JSON.stringify(entry)}`);
+ const file = entry.text_file;
+ // Against the working directory, the way every other batch entry's paths are.
+ if (typeof file === "string") return { key: entry.key, text: await readFile(file, "utf8") };
+ if (typeof entry.text !== "string") throw new Error(`answer ${entry.key} needs a "text" or a "text_file"`);
+ return { key: entry.key, text: entry.text };
+ }));
+ result = await tools.record_observation({ reference_id: required(flags, "reference-id"), answers });
+ } else {
+ // An observation is paragraphs of prose. --text keeps a short one on the command line; --text-file
+ // is how a long one arrives without the shell deciding where it ends.
+ const textFile = one(flags, "text-file");
+ const input = supplied ?? {
+ reference_id: required(flags, "reference-id"),
+ key: required(flags, "key"),
+ text: textFile === undefined ? required(flags, "text") : await readFile(textFile, "utf8"),
+ };
+ result = await tools.record_observation(input as { reference_id: string; key: string; text: string });
+ }
} else if (command === "review_element") {
const reviewFile = one(flags, "batch");
if (reviewFile !== undefined) {
diff --git a/packages/reference-video-tools/src/tools.ts b/packages/reference-video-tools/src/tools.ts
index a4dc5a42..d283ad5c 100644
--- a/packages/reference-video-tools/src/tools.ts
+++ b/packages/reference-video-tools/src/tools.ts
@@ -39,7 +39,7 @@ import {
shotTile,
writeJson,
} from "./media.js";
-import { prepareTranscript } from "./transcript.js";
+import { prepareTranscript, whisperxHealth } from "./transcript.js";
import type { Observation, ObservationTaskRequest, Observer, PrepareResult, ReferenceState, Shot, Transcript } from "./types.js";
export type PrepareReferenceInput = {
@@ -61,8 +61,16 @@ export type ObserveReferenceInput = {
};
export type RecordObservationInput = {
readonly reference_id: string;
- readonly key: string;
- readonly text: string;
+ readonly key?: string;
+ readonly text?: string;
+ /**
+ * A round of answers in one call.
+ *
+ * The `agent` observer answers everything by hand, and a reference of twenty shots owes an answer
+ * for each of three questions about it, plus the boundaries, plus the four about the reference
+ * itself. Handed out one call at a time that is seventy-odd invocations to close one sweep.
+ */
+ readonly answers?: readonly { readonly key: string; readonly text: string }[];
};
export type InspectVocabularyInput = {
readonly package_names: readonly string[];
@@ -1136,6 +1144,13 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceV
}
// WhisperX is local and slow and shares nothing with the observation requests, so it runs beside
// them rather than ahead of them.
+ //
+ // The service is the one precondition this command can check itself. The route used to probe it
+ // by hand first; it reports rather than refuses — a machine without the service still prepares
+ // everything except the transcript, which stays recoverable — and points at the repair. A
+ // reference whose transcript is already complete and unchanged needs no probe.
+ const needsTranscript = !(state.transcript?.status === "complete" && !redoTranscript);
+ const whisperx = needsTranscript ? await whisperxHealth() : undefined;
const transcribing = state.transcript?.status === "complete" && !redoTranscript
? Promise.resolve(state.transcript)
: prepareTranscript(reference, videoPath, root, info.hasAudio, "en", redoTranscript);
@@ -1178,6 +1193,9 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceV
observer,
...(fetched === undefined ? {} : { source_url: fetched.url, downloaded: !fetched.cached }),
pending_observations: pending,
+ ...(whisperx?.ok === false
+ ? { whisperx: `${whisperx.reason}. Read .agents/skills/hypit/references/host-setup.md for the failure branches.` }
+ : {}),
};
},
@@ -1737,8 +1755,30 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceV
// than through a return value. The four whole-reference observations live in the state and the
// rest in the observation cache, which is where each one was already read from.
async record_observation(input): Promise<Record<string, unknown>> {
+ if (input.answers !== undefined) {
+ const round = input.answers;
+ assert(round.length > 0, "answers is empty");
+ // Read once, before the round, so a reference that records its own answers refuses the call
+ // rather than every entry in it.
+ const reading = await loadState(input.reference_id);
+ assert(reading.observer === "agent", `reference ${input.reference_id} is read by the ${reading.observer ?? "gemini"} observer, which records its own answers`);
+ // One at a time. Every answer is a read-modify-write of one file, and `serially` below already
+ // holds them apart, so a round could run at once and be correct — it would only be a queue at
+ // a lock, for writes that take no time. Running them in the order they were written keeps the
+ // failures in that order too.
+ const { done, failures } = await runBatch(round, { concurrency: 1, gapMs: 0 }, async (one) =>
+ await tools.record_observation({ reference_id: input.reference_id, key: one.key, text: one.text }));
+ return {
+ recorded: done.length,
+ failed: failures.length,
+ records: done,
+ ...(failures.length === 0 ? {} : { failures }),
+ };
+ }
const state = await loadState(input.reference_id);
assert(state.observer === "agent", `reference ${input.reference_id} is read by the ${state.observer ?? "gemini"} observer, which records its own answers`);
+ assert(input.key !== undefined, "key is required");
+ assert(input.text !== undefined, "text is required");
const key = input.key.trim();
assert(key.length > 0, "key is required");
const text = input.text.trim();
diff --git a/packages/reference-video-tools/src/transcript.ts b/packages/reference-video-tools/src/transcript.ts
index 3792761e..b9418575 100644
--- a/packages/reference-video-tools/src/transcript.ts
+++ b/packages/reference-video-tools/src/transcript.ts
@@ -12,6 +12,30 @@ import type { WhisperXLanguage } from "@hypit/whisperx";
import { assert, command, readBytes, round, writeJson } from "./media.js";
import type { Transcript, TranscriptFile, TranscriptPassage, TranscriptWord } from "./types.js";
+/**
+ * Whether the local WhisperX service answers.
+ *
+ * The route used to probe the service by hand, as three commands, before `prepare_reference`. The
+ * service answers `/health`, which is the same call the Provider makes before transcribing — made
+ * early enough to be reported before the minutes a first start spends loading the model. It reports
+ * rather than refuses: a machine without the service still prepares everything except the transcript,
+ * which `prepareTranscript` leaves recoverable.
+ */
+export async function whisperxHealth(baseUrl = "http://127.0.0.1:8765"): Promise<{ readonly ok: boolean; readonly reason: string }> {
+ const normalized = baseUrl.replace(/\/+$/u, "");
+ try {
+ const response = await fetch(`${normalized}/health`, { signal: AbortSignal.timeout(5_000) });
+ if (!response.ok) return { ok: false, reason: `WhisperX answered ${response.status}` };
+ const body = await response.json().catch(() => undefined) as { readonly ok?: unknown } | null | undefined;
+ if (body === null || typeof body !== "object" || body.ok !== true) {
+ return { ok: false, reason: "WhisperX is up but not ready; a first start spends a few minutes loading the model" };
+ }
+ return { ok: true, reason: "" };
+ } catch {
+ return { ok: false, reason: `WhisperX is not reachable at ${normalized}` };
+ }
+}
+
// WhisperX measures in samples of the canonical evidence rate and refuses audio of any other shape,
// so this is both what the audio is extracted at and what its word times are divided by.
const EVIDENCE_SAMPLE_RATE = 16_000;