Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.
TypeScript agentic-aiaiai-agentsai-videocompilerdslffmpeggenerative-aillmmarkup-languagemonorepoplugin-systemprogramming-languagetext-to-videotypescriptvideovideo-automationvideo-clonevideo-editingvideo-generation
Optimize package previews with stable single-frame capture
rpone9c2439addbb1
diff --git a/.agents/skills/hypit/references/local-author-package.md b/.agents/skills/hypit/references/local-author-package.mdindex 94276c9b..207ad72e 100644--- a/.agents/skills/hypit/references/local-author-package.md+++ b/.agents/skills/hypit/references/local-author-package.md@@ -218,6 +218,11 @@ nothing.is most of its life.- A component that cannot render on its own cannot produce the preview image its Surface owes. Treata missing preview as evidence of this mistake rather than a step to skip.+- Produce that image with `hypit-reference-video-tools render_previews <package-dir>`. It keeps the+ package-owned preview SVML, SVS and SVRun on the standard preview-mock/Producer path, then seeks the+ middle frame of the target Present's longest stable interval in the fixed local browser. It does+ not render a complete PNG sequence; when no interval is stable for two frames, it uses the middle+ frame of the longest Present.## Freeze the Types before writing in paralleldiff --git a/docs/guide/author-packages.md b/docs/guide/author-packages.mdindex 7dbd0ae4..e49b2cef 100644--- a/docs/guide/author-packages.md+++ b/docs/guide/author-packages.md@@ -179,11 +179,13 @@ package-owned `preview/preview.svml`, `preview/recipes.svs` and `preview/build.sone representative frame under `preview/`. The command resolves the package from its explicitworkspace/package root and the active Distribution fallback; it does not require a`packages/<slug>/node_modules` self-link or a manual `npm link`. It compiles the preview Run, lets the-native preview-mock path satisfy undeclared media, renders the complete deterministic composition-through the local HyperFrames Provider, and writes the promised image named by the Surface Manifest.+native preview-mock path satisfy undeclared media, and runs the real Producers. It then finds the+target Present's longest stable interval and seeks the middle frame in a fixed local browser before+writing the promised image named by the Surface Manifest. It does not encode a complete PNG sequence,+and it never changes HyperFrames itself.-Choose a frame mid-behaviour rather than at rest. A preview of an element that has not entered yet,-or has already settled into a static end state, shows the least useful thing about it.+If no stable interval has at least two frames, the middle frame of the longest Present is used. This+keeps the picture representative while avoiding a full render solely to obtain one catalogue image.## 6. Write the activation descriptordiff --git a/docs/zh/guide/author-packages.md b/docs/zh/guide/author-packages.mdindex 4e7f4b79..2a5d7740 100644--- a/docs/zh/guide/author-packages.md+++ b/docs/zh/guide/author-packages.md@@ -145,12 +145,10 @@ package-owned preview Source,不需要阅读仓库的视觉测试源码。步1. 用你自己包的 render 函数,在一个封好的 ProgramSpace 上构造出 Track 值;2. `sealComposition({ id, canvas, tracks })`,再用 `@hypit/hyperframes` 的 `compileHyperframesDocument(composition, space)`;3. `materializeHyperframesHtml(document, resolve)` 写进一个临时目录,其中 `resolve` 把声明的每个 Artifact(字体、图片)映射到本地文件;-4. 用经由 `@hypit/provider-hyperframes-local` 解析出的、版本被钉住的 HyperFrames CLI 对该目录执行 `render`,输出 PNG 序列;-5. 留下能代表该组件状态的一帧,提交到 `preview/` 下。+4. 通过标准的 SVRun → preview-mock → Producer 路径得到真实 Composition;+5. 找到目标 Present 最长的稳定区间,在固定版本的本地浏览器中 seek 到区间中间帧并截图,提交到 `preview/` 下。-另一条路是写一个窄的 `.svrun` Target,交给本地 HyperFrames Provider 渲染,再用 `ffmpeg` 取一帧。那是一次 Build,需要 Runtime Profile,也更重。--选帧要选行为进行中的那一刻,而不是静止的时刻。一个还没入场、或者已经落定成静态终态的预览图,展示的恰好是这个元素最没用的一面。+不会为了这一张图生成完整 PNG 序列,也不修改 HyperFrames。若没有至少两帧的稳定区间,就使用该包最长 Present 的中间帧。## 6. 编写 activation 描述符diff --git a/packages/reference-video-tools/src/authoring.ts b/packages/reference-video-tools/src/authoring.tsindex 6115fa32..92c002db 100644--- a/packages/reference-video-tools/src/authoring.ts+++ b/packages/reference-video-tools/src/authoring.ts@@ -8,6 +8,8 @@ import { cpus } from "node:os";import { basename, dirname, join, resolve } from "node:path";import { fileURLToPath } from "node:url";+import puppeteer from "puppeteer-core";+import { compileHyperframesDocument, materializeHyperframesHtml } from "@hypit/hyperframes";import type { ProgramSpace } from "@hypit/program-space";import type { CompiledGraph } from "@hypit/protocol";@@ -854,6 +856,64 @@ export async function renderElement(input: RenderElementInput): Promise<Record<s};}+/**+ * Capture one package preview frame through the already-realized SVRun composition.+ *+ * This deliberately does not use HyperFrames' encoder: the normal SVRun → preview-mock → Producer+ * path has already produced the staged Composition, and a fixed local browser can seek that same+ * document directly. Package previews therefore cost one browser frame rather than a complete PNG+ * sequence while retaining the exact runtime semantics.+ */+async function capturePreviewFrame(+ realized: Awaited<ReturnType<typeof realizeAuthoringPreview>>,+ frame: number,+ outPath: string,+): Promise<void> {+ const { stage, mediaTypes } = await stageAuthoringPreview(realized);+ const { serve, browserPath } = await import("./layout.js");+ const local = await serve(stage, mediaTypes);+ const browser = await puppeteer.launch({ executablePath: browserPath(), headless: true, args: ["--no-sandbox", "--disable-dev-shm-usage"] });+ const page = await browser.newPage();+ try {+ await page.setViewport({ width: realized.built.canvas.width, height: realized.built.canvas.height, deviceScaleFactor: 1 });+ await page.goto(local.url, { waitUntil: "load", timeout: 60_000 });+ await page.evaluate(() => {+ for (const root of Array.from(document.querySelectorAll("[data-composition-id][data-width][data-height]"))) {+ const width = Number(root.getAttribute("data-width"));+ const height = Number(root.getAttribute("data-height"));+ if (root instanceof HTMLElement && Number.isFinite(width) && Number.isFinite(height)) {+ root.style.width = `${width}px`;+ root.style.height = `${height}px`;+ }+ }+ });+ await page.evaluate(async () => { await document.fonts.ready; });+ const fps = realized.built.frameRate.numerator / realized.built.frameRate.denominator;+ await page.evaluate(async ({ frame: at, fps: rate }) => {+ const runtime = window as unknown as { __hf?: { seek?: (seconds: number) => unknown }; __player?: { renderSeek?: (seconds: number) => unknown } };+ const seconds = at / rate;+ const seek = runtime.__player?.renderSeek ?? runtime.__hf?.seek;+ if (typeof seek === "function") await seek.call(runtime.__player ?? runtime.__hf, seconds);+ else window.dispatchEvent(new CustomEvent("hf-seek", { detail: { time: seconds } }));+ for (const clip of Array.from(document.querySelectorAll(".hypit-visual-present[data-start][data-duration]"))) {+ const start = Number(clip.getAttribute("data-start"));+ const duration = Number(clip.getAttribute("data-duration"));+ (clip as HTMLElement).style.display = seconds >= start && seconds < start + duration ? "" : "none";+ }+ await new Promise<void>((accept) => requestAnimationFrame(() => requestAnimationFrame(() => accept())));+ }, { frame, fps });+ const clip = await page.$eval("[data-composition-id]", (element) => {+ const rect = element.getBoundingClientRect();+ return { x: Math.max(0, rect.left), y: Math.max(0, rect.top), width: rect.width, height: rect.height };+ });+ await page.screenshot({ path: outPath, type: "png", clip });+ } finally {+ await page.close().catch(() => undefined);+ await browser.close().catch(() => undefined);+ await local.close().catch(() => undefined);+ }+}+export type RenderPreviewsInput = {readonly package_dirs: readonly string[];};@@ -907,6 +967,7 @@ export async function renderPreviews(input: RenderPreviewsInput): Promise<Record.filter((name) => /\.(png|jpe?g|svg)$/iu.test(name));const promisedPictures = promised.length > 0 ? promised : existing;assert(promisedPictures.length > 0, `${directory} promises no preview picture; nothing to draw`);+ let realized: Awaited<ReturnType<typeof realizeAuthoringPreview>> | undefined;for (const picture of promisedPictures) {if (/\.svg$/iu.test(picture)) {@@ -923,10 +984,24 @@ export async function renderPreviews(input: RenderPreviewsInput): Promise<Record// Resolve the package from its workspace root rather than requiring a self-link under// packages/<slug>/node_modules. The preview Run still lives inside the package directory;// only dependency discovery needs the parent workspace.- // The resolver root is the project/workspace containing `packages/`, not that directory- // itself. Keeping this explicit also lets the canonical fixture render without a- // package-manager self-link under the package it is previewing.- await renderElement({ run, element, out, package_root: dirname(dirname(packageDir)) });+ realized ??= await realizeAuthoringPreview({ run, package_root: dirname(dirname(packageDir)) });+ const placed = realized.built.tracks.find((track) => String(track.outputRef ?? "").endsWith(`::output::${element}.track`)+ || String(track.outputRef ?? "").includes(`::output::${element}.`));+ assert(placed !== undefined, `${directory} preview places no element named ${element}`);+ const trackId = (placed.value as { readonly id?: unknown } | null)?.id;+ assert(typeof trackId === "string", `${directory} preview element ${element} has no visual Track id`);+ const { stableFrameForTrack } = await import("./layout.js");+ const compositionTrack = realized.built.composition.tracks.find((track) => track.kind === "visual" && track.id === trackId);+ assert(compositionTrack !== undefined && compositionTrack.kind === "visual" && compositionTrack.presents.length > 0,+ `${directory} preview element ${element} has no visual Present`);+ const fallback = [...compositionTrack.presents].sort((left, right) => {+ const leftLength = left.span.endFrameExclusive - left.span.startFrame;+ const rightLength = right.span.endFrameExclusive - right.span.startFrame;+ return rightLength - leftLength || left.span.startFrame - right.span.startFrame || left.id.localeCompare(right.id);+ })[0]!;+ const frame = stableFrameForTrack(realized.built.composition, trackId)+ ?? fallback.span.startFrame + Math.floor((fallback.span.endFrameExclusive - fallback.span.startFrame - 1) / 2);+ await capturePreviewFrame(realized, frame, out);} catch (error) {throw new Error(`${directory} could not draw ${picture}: ${error instanceof Error ? error.message : String(error)}`);}diff --git a/packages/reference-video-tools/src/layout.ts b/packages/reference-video-tools/src/layout.tsindex 71a5107f..9a3224f4 100644--- a/packages/reference-video-tools/src/layout.ts+++ b/packages/reference-video-tools/src/layout.ts@@ -115,7 +115,7 @@ function contentType(path: string): string {return "application/octet-stream";}-async function serve(directory: string, mediaTypes: ReadonlyMap<string, string>): Promise<{ readonly url: string; close(): Promise<void> }> {+export async function serve(directory: string, mediaTypes: ReadonlyMap<string, string>): Promise<{ readonly url: string; close(): Promise<void> }> {const root = resolve(directory);const server = createServer(async (request, response) => {const pathname = decodeURIComponent(new URL(request.url ?? "/", "http://127.0.0.1").pathname);@@ -139,7 +139,7 @@ async function serve(directory: string, mediaTypes: ReadonlyMap<string, string>)};}-function browserPath(): string {+export function browserPath(): string {const cli = createRequire(join(repositoryRoot(), "packages/provider-hyperframes-local/package.json")).resolve("hyperframes/bin/hyperframes.mjs");const result = spawnSync(process.execPath, [cli, "browser", "path"], { encoding: "utf8", windowsHide: true, timeout: 60_000 });@@ -149,14 +149,16 @@ function browserPath(): string {return path;}-type StableSample = {+export type StableSample = {readonly frame: number;readonly subjects: readonly { readonly track: string; readonly present: string }[];readonly overlap_subjects: readonly { readonly track: string; readonly present: string }[];};-function framePlan(composition: Awaited<ReturnType<typeof realizeAuthoringPreview>>["built"]["composition"]): readonly StableSample[] {- const intervals: { readonly track: string; readonly present: string; readonly startFrame: number; readonly endFrameExclusive: number; readonly frame: number }[] = [];+type StableInterval = { readonly track: string; readonly present: string; readonly startFrame: number; readonly endFrameExclusive: number; readonly frame: number };++function stableIntervals(composition: Awaited<ReturnType<typeof realizeAuthoringPreview>>["built"]["composition"]): readonly StableInterval[] {+ const intervals: StableInterval[] = [];for (const track of composition.tracks) {if (track.kind !== "visual") continue;for (const present of track.presents) {@@ -214,6 +216,22 @@ function framePlan(composition: Awaited<ReturnType<typeof realizeAuthoringPrevie});}}+ return intervals;+}++/** The middle frame of one Track's longest stable Present interval, with earliest interval winning ties. */+export function stableFrameForTrack(+ composition: Awaited<ReturnType<typeof realizeAuthoringPreview>>["built"]["composition"],+ trackId: string,+): number | undefined {+ return stableIntervals(composition)+ .filter((interval) => interval.track === trackId)+ .sort((left, right) => (right.endFrameExclusive - right.startFrame) - (left.endFrameExclusive - left.startFrame)+ || left.startFrame - right.startFrame || left.present.localeCompare(right.present))[0]?.frame;+}++export function framePlan(composition: Awaited<ReturnType<typeof realizeAuthoringPreview>>["built"]["composition"]): readonly StableSample[] {+ const intervals = stableIntervals(composition);const byFrame = new Map<number, { readonly frame: number; readonly subjects: { track: string; present: string }[] }>();for (const interval of intervals) {const subject = { track: interval.track, present: interval.present };