Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

perf(reference-video-tools): draw one preview render per round

rponeawa763b13ac78a8
diff --git a/packages/reference-video-tools/package.json b/packages/reference-video-tools/package.json
index aece8a60..1563550a 100644
--- a/packages/reference-video-tools/package.json
+++ b/packages/reference-video-tools/package.json
@@ -26,9 +26,11 @@
"@hypit/package-loader-node": "workspace:*",
"@hypit/program-space": "workspace:*",
"@hypit/protocol": "workspace:*",
+ "@hypit/render-hyperframes": "workspace:*",
"@hypit/run-markup": "workspace:*",
"@hypit/runtime": "workspace:*",
"@hypit/provider-media-local": "workspace:*",
+ "@hypit/provider-hyperframes-local": "workspace:*",
"@hypit/provider-hypihub": "workspace:*",
"@hypit/provider-vertex": "workspace:*",
"@hypit/script": "workspace:*",
diff --git a/packages/reference-video-tools/src/authoring.ts b/packages/reference-video-tools/src/authoring.ts
index 9561cc9a..2b7658fd 100644
--- a/packages/reference-video-tools/src/authoring.ts
+++ b/packages/reference-video-tools/src/authoring.ts
@@ -3,8 +3,6 @@ import { existsSync } from "node:fs";
import { spawnSync } from "node:child_process";
import { createHash } from "node:crypto";
-import { createRequire } from "node:module";
-import { cpus } from "node:os";
import { basename, dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
@@ -23,8 +21,11 @@ import { preview } from "@hypit/studio/src/programme.js";
import { loadStudioRun } from "@hypit/studio/src/run.js";
import { inspectStudioRun } from "@hypit/studio/src/studio-preflight.js";
import { realizePreviewMock } from "@hypit/preview-mock";
-import { EndpointRegistry } from "@hypit/driver-node";
+import { EndpointRegistry, MemoryArtifactStore } from "@hypit/driver-node";
+import { mediaTypes } from "@hypit/media";
import { createLocalMediaProvider } from "@hypit/provider-media-local";
+import { createLocalHyperframesProvider } from "@hypit/provider-hyperframes-local";
+import { renderHyperframesCapabilities } from "@hypit/render-hyperframes";
import { assert, ensureDir } from "./media.js";
import type { TranscriptFile } from "./types.js";
@@ -667,6 +668,32 @@ export async function realizeAuthoringPreview(input: {
}
}
+const realizedPreviews = new Map<string, Promise<RealizedAuthoringPreview>>();
+
+/**
+ * Realize one Run once within this process, and let every later caller await that one realization.
+ *
+ * A round is one call per window, and each one realized the same Source again: the same compile, the
+ * same Producers, the same mock media encoded over the bytes already sitting in the store. Only the
+ * draw was held to one run, so eight windows over one program paid for eight realizations of it.
+ *
+ * Between processes this holds nothing, for the reason `once` holds nothing: the realization root is
+ * named after the Author SVML and the Run alone, so an edited Recipe or project-local package lands
+ * in the same directory, and a new process is where that edit has to be picked up.
+ */
+async function realizedOnce(input: {
+ readonly run: string;
+ readonly runtime?: string;
+ readonly package_root?: string;
+}): Promise<RealizedAuthoringPreview> {
+ const key = JSON.stringify([resolve(invokedFrom(), input.run), input.runtime ?? "", input.package_root ?? ""]);
+ const held = realizedPreviews.get(key);
+ if (held !== undefined) return await held;
+ const started = realizeAuthoringPreview(input);
+ realizedPreviews.set(key, started);
+ try { return await started; } catch (error) { realizedPreviews.delete(key); throw error; }
+}
+
/** Materialize a realized composition and its served artifacts without opening a browser. */
export async function stageAuthoringPreview(realized: Pick<RealizedAuthoringPreview, "built" | "previewMock">): Promise<{
readonly stage: string;
@@ -724,7 +751,7 @@ export async function renderElement(input: RenderElementInput): Promise<Record<s
const segmentArgument = input.segment;
const selectionArgument = input.selection;
const tokensArgument = input.tokens === undefined ? undefined : tokenWindow(input.tokens, "tokens");
- const realized = await realizeAuthoringPreview({
+ const realized = await realizedOnce({
run,
...(input.package_root === undefined ? {} : { package_root: input.package_root }),
});
@@ -778,58 +805,66 @@ export async function renderElement(input: RenderElementInput): Promise<Record<s
}
const clip = /\.(mp4|mov|webm)$/iu.test(outPath);
- const stage = join(compareRoot, "stage");
- const frames = join(compareRoot, "frames");
- // Staged and drawn once per Run and clock, however many windows are asked for.
+ const program = join(compareRoot, "program.mp4");
+ // The whole program, drawn once per Run and clock, however many windows are asked for.
//
- // The picture does not depend on the element or the window — the whole program is drawn and the
- // window is cut out of these frames — so a round of eight looks over one Source is one browser
- // render and eight ffmpeg cuts. Rendering per entry would draw the same program eight times.
+ // The picture does not depend on the element or the window — every window is cut out of this one
+ // file — so a round of eight looks over one Source is one render and eight ffmpeg cuts. Rendering
+ // per entry would draw the same program eight times.
//
- // The staged document is written inside the same run for the same reason the derived Run is: every
- // entry compiles the identical document to the identical path, `writeFile` truncates before it
- // writes, and the runtime is reading that path. An entry staging `index.html` while the runtime
- // loads it hands the runtime a file with no timeline in it, reported as
- // `Composition has zero duration`.
+ // Drawn through the Provider a Build renders with, rather than by driving the HyperFrames CLI from
+ // here. The staging layout, the browser flags and the output check belong to the render path, and a
+ // preview that spawns its own copy of them is a second renderer to keep in step with the first: it
+ // was already staging the document by a different file naming and drawing with flags the delivery
+ // never takes. What preview still owns is the Run it draws — the mocked, estimate-timed one — not
+ // how that Run is drawn.
await once(`draw:${renderKey}`, async () => {
- // Compile and materialize through the same no-media staging helper used by layout_check.
- await stageAuthoringPreview({ built, previewMock });
-
- await rm(frames, { recursive: true, force: true });
- await mkdir(frames, { recursive: true });
- // The runtime ships with the tree, not with the project. Resolving it against the package root
- // finds nothing whenever those two differ, which is every project that installs packages of its own.
- const hyperframesCli = createRequire(join(repositoryRoot(), "packages/provider-hyperframes-local/package.json"))
- .resolve("hyperframes/bin/hyperframes.mjs");
- const drawn = spawnSync(process.execPath, [
- hyperframesCli, "render", stage,
- "--format", "png-sequence", "--output", frames, "--fps", String(frameRate),
- "--workers", String(Math.max(1, cpus().length - 2)),
- // Halves the time and lands on the same pixels — verified frame for frame against a run without
- // it, PSNR reporting no error at all. The flag is marked experimental upstream; that is the
- // reason to check the pixels, which is done, rather than the reason to draw twice as long.
- "--experimental-fast-capture", "--no-best-effort", "--quiet",
- ], { encoding: "utf8", windowsHide: true, timeout: 600_000 });
- assert(drawn.status === 0, `the HyperFrames runtime refused: ${(drawn.stderr ?? "").trim().slice(-2000)}`);
+ const document = compileHyperframesDocument(built.composition, built.space as ProgramSpace);
+ // The endpoint reads what the document names out of a store, so the served bytes are put in one.
+ // They are the same bytes under the same digests, which is what makes the lookup find them.
+ const artifacts = new MemoryArtifactStore();
+ for (const file of built.served.values()) await artifacts.put(file.bytes, file.mediaType);
+ const endpoints = new EndpointRegistry();
+ // `auto` leaves the GPU decision to the runtime, which is what this render had before it went
+ // through the Provider. The delivery default asks for hardware, and a preview is drawn on
+ // whatever machine is authoring rather than on a render host chosen to have one.
+ await createLocalHyperframesProvider({ browserGpu: "auto" }).install(endpoints);
+ const need = {
+ id: "preview-visual",
+ capability: renderHyperframesCapabilities.renderVisual,
+ returns: mediaTypes.renderedVisual,
+ constraints: { document } as never,
+ result: "preview-visual",
+ };
+ const registration = endpoints.resolve(need);
+ assert(registration.status === "resolved" && registration.registration.kind === "immediate",
+ "the local HyperFrames render endpoint did not resolve");
+ const drawn = await registration.registration.handler({
+ command: { kind: "fulfill-need", id: need.id, need }, need, artifacts, credentials: {},
+ });
+ assert(drawn.value.kind === "inline", "the HyperFrames endpoint returned no inline RenderedVisual");
+ const rendered = drawn.value.value as { readonly artifact?: { readonly digest?: `sha256:${string}` } };
+ const digest = rendered.artifact?.digest;
+ assert(digest !== undefined, "the HyperFrames endpoint returned a RenderedVisual with no Artifact");
+ const bytes = await artifacts.get(digest);
+ assert(bytes !== undefined, `the HyperFrames endpoint kept no bytes for ${digest}`);
+ await writeFile(program, bytes);
});
- const written = (await readdir(frames)).filter((name) => name.endsWith(".png")).sort();
- assert(written.length > 0, "the HyperFrames runtime wrote no frames");
- const first = Math.min(window.startFrame, written.length - 1);
- const last = Math.min(window.endFrameExclusive, written.length);
- // The runtime draws with an alpha channel and leaves unpainted area transparent. What a viewer is
- // under is the Film's own clear colour, so it is composited in here rather than left to whatever
- // opens the file: a reference clip is opaque, and an observer handed a transparent counterpart reads
- // the difference as design when it came from the encoding.
- const clear = built.canvas?.clearColor ?? "#000000";
- const background = `color=c=${clear.replace("#", "0x")}:s=${canvas.width}x${canvas.height}:r=${frameRate}`;
+ const first = Math.min(window.startFrame, programFrames - 1);
+ const last = Math.min(window.endFrameExclusive, programFrames);
+ // The Film's own clear colour is already under the picture: the document paints it on the root and
+ // on the body, and an mp4 carries no alpha for anything to show through. Nothing is composited in
+ // here, which is what the alpha of a PNG sequence used to need.
+ const at = clip ? first : Math.min(first + Math.floor((last - first) / 2), programFrames - 1);
const count = Math.max(1, clip ? last - first : 1);
const encoded = spawnSync("ffmpeg", [
"-hide_banner", "-loglevel", "error", "-y",
- "-f", "lavfi", "-i", background,
- "-framerate", String(frameRate), "-start_number", String(clip ? first : Math.min(first + Math.floor((last - first) / 2), written.length - 1)),
- "-i", join(frames, "frame_%06d.png"),
- "-filter_complex", "[0][1]overlay=shortest=1[v]", "-map", "[v]",
+ "-i", program,
+ // Sought on the output rather than the input, so the cut opens on the frame the window names
+ // instead of the keyframe before it. The windows are frame ranges and the comparison trims both
+ // sides by frames, so a cut that slid to a keyframe would put the two sides out of step.
+ "-ss", String(at / frameRate),
"-frames:v", String(count),
...(clip ? ["-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "20"] : []),
outPath,
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index c2e63cbc..13530cf8 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -2449,6 +2449,9 @@ importers:
'@hypit/protocol':
specifier: workspace:*
version: link:../protocol
+ '@hypit/provider-hyperframes-local':
+ specifier: workspace:*
+ version: link:../provider-hyperframes-local
'@hypit/provider-hypihub':
specifier: workspace:*
version: link:../provider-hypihub
@@ -2458,6 +2461,9 @@ importers:
'@hypit/provider-vertex':
specifier: workspace:*
version: link:../provider-vertex
+ '@hypit/render-hyperframes':
+ specifier: workspace:*
+ version: link:../render-hyperframes
'@hypit/run-markup':
specifier: workspace:*
version: link:../run-markup
diff --git a/skills/hypit/references/element-review.md b/skills/hypit/references/element-review.md
index 307c9db2..a35b6e3b 100644
--- a/skills/hypit/references/element-review.md
+++ b/skills/hypit/references/element-review.md
@@ -125,7 +125,7 @@ A mock lives only in this render. It is never written into the Source, and no ga
### The derived Run carries the mocks, and nothing else changes
The mocks are substituted into a Run that names the project's own Source. The whole program is drawn
-and the window is cut out of the frames afterwards, so every element the Source places is present and
+and the window is cut out of it afterwards, so every element the Source places is present and
drawn wherever the Source puts it — what is on screen over the words being looked at is what the
delivery will put there.
diff --git a/skills/hypit/references/original-authoring/route.md b/skills/hypit/references/original-authoring/route.md
index 268458aa..52d63eff 100644
--- a/skills/hypit/references/original-authoring/route.md
+++ b/skills/hypit/references/original-authoring/route.md
@@ -217,7 +217,7 @@ hypit-reference-video-tools authoring_check projects/<name>/build.svrun
```
Its `plan` is the list: each entry an element and a word range, with the reason. Feed it to
-`render_element --batch` — the program is drawn once and every entry is cut out of those frames.
+`render_element --batch` — the program is drawn once and every entry is cut out of that one render.
Keep `.hypit/layout-check.json` with the renders. The reader still decides whether visible text or
marks fit the intent and whether any measured offset, overlap or bleed is deliberate.
diff --git a/skills/hypit/references/preview.md b/skills/hypit/references/preview.md
index cb59f42d..a7918d83 100644
--- a/skills/hypit/references/preview.md
+++ b/skills/hypit/references/preview.md
@@ -102,12 +102,11 @@ extension writes one still from the middle of it.
A whole round is one call: `--batch <renders.json>`, an array of `{element, segment|selection|tokens,
out}` inheriting the Run, and the `renders` array `reconstruction_check` returns is exactly that.
-The program is realized once into a shared preview frame cache for the whole round, then the entries
-are processed serially and cut from those frames. Asking for eight windows therefore costs one full
-draw and eight cuts without concurrent workers racing over the shared `preview.svrun`. The picture
-does not depend on `--element` — it is
-everything the Source places over those words — so two elements over one stretch share the render as
-well.
+The program is realized and drawn once for the whole round, into one mp4 of the whole mocked
+program, then the entries are processed serially and cut from it. Asking for eight windows therefore
+costs one full draw and eight cuts without concurrent workers racing over the shared `preview.svrun`.
+The picture does not depend on `--element` — it is everything the Source places over those words — so
+two elements over one stretch share the render as well.
Each Segment's length comes from the Source's own `estimate:Speech`; `--reference-id` selects reference
evidence and comparison windows only. The sidecar records `"timing_basis": "estimate"`.
diff --git a/skills/hypit/references/reconstruction/route.md b/skills/hypit/references/reconstruction/route.md
index 8691ff74..9b3a323f 100644
--- a/skills/hypit/references/reconstruction/route.md
+++ b/skills/hypit/references/reconstruction/route.md
@@ -390,7 +390,7 @@ hypit-reference-video-tools render_element projects/<name>/build.svrun --batch r
```
`comparison-round.md` uses preview-mock estimate timing; `--reference-id` selects comparison evidence, not mock timing.
-The program is realized once into a shared preview frame cache and entries are cut serially, so render
+The program is realized and drawn once into one mp4 and entries are cut from it serially, so render
the whole list in one `--batch` call; the batch does not run concurrent preview staging workers.
Keep `.hypit/layout-check.json` beside the round. Its realized stable-state measurements remain