Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

Optimize package previews with stable single-frame capture

rpone9c2439addbb1
diff --git a/.agents/skills/hypit/references/local-author-package.md b/.agents/skills/hypit/references/local-author-package.md
index 94276c9b..207ad72e 100644
--- a/.agents/skills/hypit/references/local-author-package.md
+++ b/.agents/skills/hypit/references/local-author-package.md
@@ -218,6 +218,11 @@ nothing.
is most of its life.
- A component that cannot render on its own cannot produce the preview image its Surface owes. Treat
a missing preview as evidence of this mistake rather than a step to skip.
+- Produce that image with `hypit-reference-video-tools render_previews <package-dir>`. It keeps the
+ package-owned preview SVML, SVS and SVRun on the standard preview-mock/Producer path, then seeks the
+ middle frame of the target Present's longest stable interval in the fixed local browser. It does
+ not render a complete PNG sequence; when no interval is stable for two frames, it uses the middle
+ frame of the longest Present.
## Freeze the Types before writing in parallel
diff --git a/docs/guide/author-packages.md b/docs/guide/author-packages.md
index 7dbd0ae4..e49b2cef 100644
--- a/docs/guide/author-packages.md
+++ b/docs/guide/author-packages.md
@@ -179,11 +179,13 @@ package-owned `preview/preview.svml`, `preview/recipes.svs` and `preview/build.s
one representative frame under `preview/`. The command resolves the package from its explicit
workspace/package root and the active Distribution fallback; it does not require a
`packages/<slug>/node_modules` self-link or a manual `npm link`. It compiles the preview Run, lets the
-native preview-mock path satisfy undeclared media, renders the complete deterministic composition
-through the local HyperFrames Provider, and writes the promised image named by the Surface Manifest.
+native preview-mock path satisfy undeclared media, and runs the real Producers. It then finds the
+target Present's longest stable interval and seeks the middle frame in a fixed local browser before
+writing the promised image named by the Surface Manifest. It does not encode a complete PNG sequence,
+and it never changes HyperFrames itself.
-Choose a frame mid-behaviour rather than at rest. A preview of an element that has not entered yet,
-or has already settled into a static end state, shows the least useful thing about it.
+If no stable interval has at least two frames, the middle frame of the longest Present is used. This
+keeps the picture representative while avoiding a full render solely to obtain one catalogue image.
## 6. Write the activation descriptor
diff --git a/docs/zh/guide/author-packages.md b/docs/zh/guide/author-packages.md
index 4e7f4b79..2a5d7740 100644
--- a/docs/zh/guide/author-packages.md
+++ b/docs/zh/guide/author-packages.md
@@ -145,12 +145,10 @@ package-owned preview Source,不需要阅读仓库的视觉测试源码。步
1. 用你自己包的 render 函数,在一个封好的 ProgramSpace 上构造出 Track 值;
2. `sealComposition({ id, canvas, tracks })`,再用 `@hypit/hyperframes` 的 `compileHyperframesDocument(composition, space)`;
3. `materializeHyperframesHtml(document, resolve)` 写进一个临时目录,其中 `resolve` 把声明的每个 Artifact(字体、图片)映射到本地文件;
-4. 用经由 `@hypit/provider-hyperframes-local` 解析出的、版本被钉住的 HyperFrames CLI 对该目录执行 `render`,输出 PNG 序列;
-5. 留下能代表该组件状态的一帧,提交到 `preview/` 下。
+4. 通过标准的 SVRun → preview-mock → Producer 路径得到真实 Composition;
+5. 找到目标 Present 最长的稳定区间,在固定版本的本地浏览器中 seek 到区间中间帧并截图,提交到 `preview/` 下。
-另一条路是写一个窄的 `.svrun` Target,交给本地 HyperFrames Provider 渲染,再用 `ffmpeg` 取一帧。那是一次 Build,需要 Runtime Profile,也更重。
-
-选帧要选行为进行中的那一刻,而不是静止的时刻。一个还没入场、或者已经落定成静态终态的预览图,展示的恰好是这个元素最没用的一面。
+不会为了这一张图生成完整 PNG 序列,也不修改 HyperFrames。若没有至少两帧的稳定区间,就使用该包最长 Present 的中间帧。
## 6. 编写 activation 描述符
diff --git a/packages/reference-video-tools/src/authoring.ts b/packages/reference-video-tools/src/authoring.ts
index 6115fa32..92c002db 100644
--- a/packages/reference-video-tools/src/authoring.ts
+++ b/packages/reference-video-tools/src/authoring.ts
@@ -8,6 +8,8 @@ import { cpus } from "node:os";
import { basename, dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
+import puppeteer from "puppeteer-core";
+
import { compileHyperframesDocument, materializeHyperframesHtml } from "@hypit/hyperframes";
import type { ProgramSpace } from "@hypit/program-space";
import type { CompiledGraph } from "@hypit/protocol";
@@ -854,6 +856,64 @@ export async function renderElement(input: RenderElementInput): Promise<Record<s
};
}
+/**
+ * Capture one package preview frame through the already-realized SVRun composition.
+ *
+ * This deliberately does not use HyperFrames' encoder: the normal SVRun → preview-mock → Producer
+ * path has already produced the staged Composition, and a fixed local browser can seek that same
+ * document directly. Package previews therefore cost one browser frame rather than a complete PNG
+ * sequence while retaining the exact runtime semantics.
+ */
+async function capturePreviewFrame(
+ realized: Awaited<ReturnType<typeof realizeAuthoringPreview>>,
+ frame: number,
+ outPath: string,
+): Promise<void> {
+ const { stage, mediaTypes } = await stageAuthoringPreview(realized);
+ const { serve, browserPath } = await import("./layout.js");
+ const local = await serve(stage, mediaTypes);
+ const browser = await puppeteer.launch({ executablePath: browserPath(), headless: true, args: ["--no-sandbox", "--disable-dev-shm-usage"] });
+ const page = await browser.newPage();
+ try {
+ await page.setViewport({ width: realized.built.canvas.width, height: realized.built.canvas.height, deviceScaleFactor: 1 });
+ await page.goto(local.url, { waitUntil: "load", timeout: 60_000 });
+ await page.evaluate(() => {
+ for (const root of Array.from(document.querySelectorAll("[data-composition-id][data-width][data-height]"))) {
+ const width = Number(root.getAttribute("data-width"));
+ const height = Number(root.getAttribute("data-height"));
+ if (root instanceof HTMLElement && Number.isFinite(width) && Number.isFinite(height)) {
+ root.style.width = `${width}px`;
+ root.style.height = `${height}px`;
+ }
+ }
+ });
+ await page.evaluate(async () => { await document.fonts.ready; });
+ const fps = realized.built.frameRate.numerator / realized.built.frameRate.denominator;
+ await page.evaluate(async ({ frame: at, fps: rate }) => {
+ const runtime = window as unknown as { __hf?: { seek?: (seconds: number) => unknown }; __player?: { renderSeek?: (seconds: number) => unknown } };
+ const seconds = at / rate;
+ const seek = runtime.__player?.renderSeek ?? runtime.__hf?.seek;
+ if (typeof seek === "function") await seek.call(runtime.__player ?? runtime.__hf, seconds);
+ else window.dispatchEvent(new CustomEvent("hf-seek", { detail: { time: seconds } }));
+ for (const clip of Array.from(document.querySelectorAll(".hypit-visual-present[data-start][data-duration]"))) {
+ const start = Number(clip.getAttribute("data-start"));
+ const duration = Number(clip.getAttribute("data-duration"));
+ (clip as HTMLElement).style.display = seconds >= start && seconds < start + duration ? "" : "none";
+ }
+ await new Promise<void>((accept) => requestAnimationFrame(() => requestAnimationFrame(() => accept())));
+ }, { frame, fps });
+ const clip = await page.$eval("[data-composition-id]", (element) => {
+ const rect = element.getBoundingClientRect();
+ return { x: Math.max(0, rect.left), y: Math.max(0, rect.top), width: rect.width, height: rect.height };
+ });
+ await page.screenshot({ path: outPath, type: "png", clip });
+ } finally {
+ await page.close().catch(() => undefined);
+ await browser.close().catch(() => undefined);
+ await local.close().catch(() => undefined);
+ }
+}
+
export type RenderPreviewsInput = {
readonly package_dirs: readonly string[];
};
@@ -907,6 +967,7 @@ export async function renderPreviews(input: RenderPreviewsInput): Promise<Record
.filter((name) => /\.(png|jpe?g|svg)$/iu.test(name));
const promisedPictures = promised.length > 0 ? promised : existing;
assert(promisedPictures.length > 0, `${directory} promises no preview picture; nothing to draw`);
+ let realized: Awaited<ReturnType<typeof realizeAuthoringPreview>> | undefined;
for (const picture of promisedPictures) {
if (/\.svg$/iu.test(picture)) {
@@ -923,10 +984,24 @@ export async function renderPreviews(input: RenderPreviewsInput): Promise<Record
// Resolve the package from its workspace root rather than requiring a self-link under
// packages/<slug>/node_modules. The preview Run still lives inside the package directory;
// only dependency discovery needs the parent workspace.
- // The resolver root is the project/workspace containing `packages/`, not that directory
- // itself. Keeping this explicit also lets the canonical fixture render without a
- // package-manager self-link under the package it is previewing.
- await renderElement({ run, element, out, package_root: dirname(dirname(packageDir)) });
+ realized ??= await realizeAuthoringPreview({ run, package_root: dirname(dirname(packageDir)) });
+ const placed = realized.built.tracks.find((track) => String(track.outputRef ?? "").endsWith(`::output::${element}.track`)
+ || String(track.outputRef ?? "").includes(`::output::${element}.`));
+ assert(placed !== undefined, `${directory} preview places no element named ${element}`);
+ const trackId = (placed.value as { readonly id?: unknown } | null)?.id;
+ assert(typeof trackId === "string", `${directory} preview element ${element} has no visual Track id`);
+ const { stableFrameForTrack } = await import("./layout.js");
+ const compositionTrack = realized.built.composition.tracks.find((track) => track.kind === "visual" && track.id === trackId);
+ assert(compositionTrack !== undefined && compositionTrack.kind === "visual" && compositionTrack.presents.length > 0,
+ `${directory} preview element ${element} has no visual Present`);
+ const fallback = [...compositionTrack.presents].sort((left, right) => {
+ const leftLength = left.span.endFrameExclusive - left.span.startFrame;
+ const rightLength = right.span.endFrameExclusive - right.span.startFrame;
+ return rightLength - leftLength || left.span.startFrame - right.span.startFrame || left.id.localeCompare(right.id);
+ })[0]!;
+ const frame = stableFrameForTrack(realized.built.composition, trackId)
+ ?? fallback.span.startFrame + Math.floor((fallback.span.endFrameExclusive - fallback.span.startFrame - 1) / 2);
+ await capturePreviewFrame(realized, frame, out);
} catch (error) {
throw new Error(`${directory} could not draw ${picture}: ${error instanceof Error ? error.message : String(error)}`);
}
diff --git a/packages/reference-video-tools/src/layout.ts b/packages/reference-video-tools/src/layout.ts
index 71a5107f..9a3224f4 100644
--- a/packages/reference-video-tools/src/layout.ts
+++ b/packages/reference-video-tools/src/layout.ts
@@ -115,7 +115,7 @@ function contentType(path: string): string {
return "application/octet-stream";
}
-async function serve(directory: string, mediaTypes: ReadonlyMap<string, string>): Promise<{ readonly url: string; close(): Promise<void> }> {
+export async function serve(directory: string, mediaTypes: ReadonlyMap<string, string>): Promise<{ readonly url: string; close(): Promise<void> }> {
const root = resolve(directory);
const server = createServer(async (request, response) => {
const pathname = decodeURIComponent(new URL(request.url ?? "/", "http://127.0.0.1").pathname);
@@ -139,7 +139,7 @@ async function serve(directory: string, mediaTypes: ReadonlyMap<string, string>)
};
}
-function browserPath(): string {
+export function browserPath(): string {
const cli = createRequire(join(repositoryRoot(), "packages/provider-hyperframes-local/package.json"))
.resolve("hyperframes/bin/hyperframes.mjs");
const result = spawnSync(process.execPath, [cli, "browser", "path"], { encoding: "utf8", windowsHide: true, timeout: 60_000 });
@@ -149,14 +149,16 @@ function browserPath(): string {
return path;
}
-type StableSample = {
+export type StableSample = {
readonly frame: number;
readonly subjects: readonly { readonly track: string; readonly present: string }[];
readonly overlap_subjects: readonly { readonly track: string; readonly present: string }[];
};
-function framePlan(composition: Awaited<ReturnType<typeof realizeAuthoringPreview>>["built"]["composition"]): readonly StableSample[] {
- const intervals: { readonly track: string; readonly present: string; readonly startFrame: number; readonly endFrameExclusive: number; readonly frame: number }[] = [];
+type StableInterval = { readonly track: string; readonly present: string; readonly startFrame: number; readonly endFrameExclusive: number; readonly frame: number };
+
+function stableIntervals(composition: Awaited<ReturnType<typeof realizeAuthoringPreview>>["built"]["composition"]): readonly StableInterval[] {
+ const intervals: StableInterval[] = [];
for (const track of composition.tracks) {
if (track.kind !== "visual") continue;
for (const present of track.presents) {
@@ -214,6 +216,22 @@ function framePlan(composition: Awaited<ReturnType<typeof realizeAuthoringPrevie
});
}
}
+ return intervals;
+}
+
+/** The middle frame of one Track's longest stable Present interval, with earliest interval winning ties. */
+export function stableFrameForTrack(
+ composition: Awaited<ReturnType<typeof realizeAuthoringPreview>>["built"]["composition"],
+ trackId: string,
+): number | undefined {
+ return stableIntervals(composition)
+ .filter((interval) => interval.track === trackId)
+ .sort((left, right) => (right.endFrameExclusive - right.startFrame) - (left.endFrameExclusive - left.startFrame)
+ || left.startFrame - right.startFrame || left.present.localeCompare(right.present))[0]?.frame;
+}
+
+export function framePlan(composition: Awaited<ReturnType<typeof realizeAuthoringPreview>>["built"]["composition"]): readonly StableSample[] {
+ const intervals = stableIntervals(composition);
const byFrame = new Map<number, { readonly frame: number; readonly subjects: { track: string; present: string }[] }>();
for (const interval of intervals) {
const subject = { track: interval.track, present: interval.present };