Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.
TypeScript agentic-aiaiai-agentsai-videocompilerdslffmpeggenerative-aillmmarkup-languagemonorepoplugin-systemprogramming-languagetext-to-videotypescriptvideovideo-automationvideo-clonevideo-editingvideo-generation
fix(hyperframes): preserve element animation and resource staging
kashorin08ba946aa820
diff --git a/packages/hyperframes/README.md b/packages/hyperframes/README.mdindex 65f55c08..f2209cf9 100644--- a/packages/hyperframes/README.md+++ b/packages/hyperframes/README.md@@ -12,6 +12,8 @@ This package understands only `VisualTrack`, `ProgramSpace` and canvas geometry.does not know Caption, Speech, B-roll, Seedance or any other author-domain component. It flattensevery Track's Presents, orders them by their own absolute stacking keys and emits frame-bound localkeyframes. An authoring Track never becomes an isolated render stacking surface.+Terminal text and SVG mask sources retain the same Present-relative frame animations as other+elements, including direct seeks into the middle of a Present.It deliberately ignores `AudioTrack`. HyperFrames produces a silent visual fact; the media pipelinecompiles and renders program audio separately, then an explicit mux Provider joins the two. Changing@@ -23,6 +25,12 @@ a Provider can verify and name staged bytes without guessing from the hash. A lorender Runtime calls `materializeHyperframesHtml()` with its own Artifact URL resolver beforehanding the HTML to HyperFrames. That environment-specific materialization is not a new compiledRecord and does not change the compiled document.+Staging assigns short local filenames; opaque Resource IDs are never interpreted as filesystem+paths or filenames, so their length and punctuation do not restrict the host filesystem.+`stageHyperframesProject` streams each asset into its staged file. Its `validateSurface(surface, path,+signal)` callback borrows that completed file during inspection; it receives neither a whole-file+byte copy nor ownership of the file. The staging caller keeps the directory alive until all work+has settled, including cancellation.The document exposes its render domain directly rather than asking an Endpoint to scrape HTML:exact rational `frameRate`, integer `frameCount`, and canvas dimensions are explicit document@@ -53,8 +61,12 @@ places a direct typed child. Every child is placed once, including sampled videoThis keeps actual video available to the renderer's exact source-frame preparation.CSS is scoped to the generated program root with `@scope`; `:scope` styles that root. The HTML can-contain arbitrary local structure, SVG, internal stacking, masks and backdrop filters. The optional-`setup` string is a JavaScript function body with `root` and `data` arguments. It returns a synchronous+contain arbitrary local structure, SVG, internal stacking, masks and backdrop filters.+CSS scope does not isolate HTML/SVG IDs. Use classes or data attributes for local queries; when an+SVG needs an ID for `url(#...)` or `href`, derive it from the unique `root.id` in `setup` and set both+the definition and its references there. Reusable instances must not repeat hard-coded SVG IDs.++The optional `setup` string is a JavaScript function body with `root` and `data` arguments. It returns a synchronous`render(localFrame)` function, evaluated on initial load and active `hf-seek` events. Outside thePresent's lifetime, the dispatcher settles the boundary pose once instead of repeatedly updatingan invisible program. Active seeks always redraw, including the same frame after resources become@@ -63,6 +75,8 @@ objects in `setup`; express animation state as a function of frame and inputs sobegin at any frame. Async work belongs tomaterial preparation before rendering; browser resources belong in the program's declared artifacts.Use `hyperframesResourceUri(artifact.resource)` in resource-bearing markup or CSS.+Returning a Promise from `render` reports an authoring error; frame capture never waits for an+unbounded asynchronous drawing task or races it.Ordinary child sampling follows the Present clock. Reframing the parent leaves source playbackunchanged. `projectTimelineMedia` from `@hypit/hypit/timeline` gives a component selected prepareddiff --git a/packages/hyperframes/src/browser-program.ts b/packages/hyperframes/src/browser-program.tsindex f4265f6a..e360dff8 100644--- a/packages/hyperframes/src/browser-program.ts+++ b/packages/hyperframes/src/browser-program.ts@@ -76,7 +76,12 @@ export function browserProgramScript(entries: readonly {// including repeated frame 0 after images/fonts finish loading.if (region !== 'active' && entry.region === region) continue;try {- entry.render(Math.max(0, Math.min(entry.durationFrames, local)));+ const result = entry.render(Math.max(0, Math.min(entry.durationFrames, local)));+ if (result != null && typeof result.then === 'function') {+ // Observe a later rejection, but never let asynchronous drawing race frame capture.+ Promise.resolve(result).catch(fail);+ throw new Error('Browser program render(localFrame) must be synchronous; prepare asynchronous resources before rendering.');+ }entry.region = region;}catch (error) { fail(error); }diff --git a/packages/hyperframes/src/document.ts b/packages/hyperframes/src/document.tsindex 9ece71dc..41ede3e5 100644--- a/packages/hyperframes/src/document.ts+++ b/packages/hyperframes/src/document.ts@@ -229,23 +229,22 @@ function exactFontStyle(element: VisualElement, stableId: StableDomId): string[]];}-function renderElement(- element: VisualElement,- children: ReadonlyMap<string, readonly VisualElement[]>,- context: {- readonly trackId: string;- readonly presentId: string;- readonly presentStart: string;- readonly presentDuration: string;- readonly presentDurationFrames: number;- readonly presentStartFrame: number;- readonly programNumerator: number;- readonly programDenominator: number;- readonly stackIndex: number;- readonly emittedFilterIds: Set<string>;- readonly stableId: StableDomId;- },-): string {+type ElementContext = {+ readonly trackId: string;+ readonly presentId: string;+ readonly presentStart: string;+ readonly presentDuration: string;+ readonly presentDurationFrames: number;+ readonly presentStartFrame: number;+ readonly programNumerator: number;+ readonly programDenominator: number;+ readonly stackIndex: number;+ readonly emittedFilterIds: Set<string>;+ readonly stableId: StableDomId;+};++/** Every emitted element, including SVG mask sources, keeps the same authored frame clock. */+function elementPresentation(element: VisualElement, context: ElementContext) {const id = context.stableId([context.trackId, context.presentId, element.id]);const animationName = element.animation === undefined? undefined@@ -277,6 +276,16 @@ function renderElement(? "": ` data-hypit-frame-animation data-hypit-animation-start-frame="${context.presentStartFrame}" data-hypit-animation-duration-frames="${animationDurationFrames}" data-hypit-animation-sample-frames="${context.presentDurationFrames}" data-hypit-animation-properties="${animationProperties.join(",")}"`;const common = `${commonAttributes}${animationAttributes} style="${escapeHtml(inlineStyle)}"`;+ return { inlineStyle, commonAttributes: `${commonAttributes}${animationAttributes}`, common };+}++function renderElement(+ element: VisualElement,+ children: ReadonlyMap<string, readonly VisualElement[]>,+ context: ElementContext,+): string {+ const { inlineStyle, commonAttributes, common } = elementPresentation(element, context);+ const id = context.stableId([context.trackId, context.presentId, element.id]);if (element.kind === "mask") {const direct = children.get(element.id) ?? [];const maskRoot = direct.find((child) => child.id === element.maskElement);@@ -295,6 +304,7 @@ function renderElement(throw new Error(`Local mask ${element.id} mask source must be one terminal owned element.`);}const maskSource = (() => {+ const sourcePresentation = elementPresentation(maskRoot, context);if (maskRoot.kind === "text") {const alignment = maskRoot.style.find((declaration) => declaration.name === "text-align")?.value;const anchor = alignment === "right" || alignment === "end" ? "end" : alignment === "left" || alignment === "start" ? "start" : "middle";@@ -310,17 +320,17 @@ function renderElement(? blockAlignment === "flex-start" ? "0" : blockAlignment === "flex-end" ? "100%" : "50%": String(blockAlignment === "flex-start" ? paddingTop : blockAlignment === "flex-end" ? maskHeight - paddingBottom : (paddingTop + maskHeight - paddingBottom) / 2);const baseline = blockAlignment === "flex-start" ? "text-before-edge" : blockAlignment === "flex-end" ? "text-after-edge" : "central";- const textStyle = [css(maskRoot.style), ...exactFontStyle(maskRoot, context.stableId), "fill:currentColor"].filter(Boolean).join(";");- return `<text${attributes(maskRoot.attributes)} x="${x}" y="${y}" text-anchor="${anchor}" dominant-baseline="${baseline}" style="${escapeHtml(textStyle)}">${escapeHtml(maskRoot.text)}</text>`;+ const textStyle = `${sourcePresentation.inlineStyle};fill:currentColor`;+ return `<text ${sourcePresentation.commonAttributes} x="${x}" y="${y}" text-anchor="${anchor}" dominant-baseline="${baseline}" style="${escapeHtml(textStyle)}">${escapeHtml(maskRoot.text)}</text>`;}if (maskRoot.kind === "text-flow" || maskRoot.kind === "path-text") {return `<foreignObject x="0" y="0" width="100%" height="100%"><div xmlns="http://www.w3.org/1999/xhtml" style="position:relative;width:100%;height:100%">${renderOwned(maskRoot)}</div></foreignObject>`;}if (maskRoot.kind === "image") {- return `<image x="0" y="0" width="100%" height="100%" preserveAspectRatio="none" href="${escapeHtml(hyperframesResourceUri(maskRoot.artifact.resource))}" style="${escapeHtml(css(maskRoot.style))}"/>`;+ return `<image ${sourcePresentation.common} x="0" y="0" width="100%" height="100%" preserveAspectRatio="none" href="${escapeHtml(hyperframesResourceUri(maskRoot.artifact.resource))}"/>`;}if (maskRoot.kind === "surface" && maskRoot.surface.timing.kind === "still") {- return `<image data-hypit-surface-resource="${maskRoot.surface.artifact.resource}" x="0" y="0" width="100%" height="100%" preserveAspectRatio="none" href="${escapeHtml(hyperframesResourceUri(maskRoot.surface.artifact.resource))}" style="${escapeHtml(css(maskRoot.style))}"/>`;+ return `<image ${sourcePresentation.common} data-hypit-surface-resource="${maskRoot.surface.artifact.resource}" x="0" y="0" width="100%" height="100%" preserveAspectRatio="none" href="${escapeHtml(hyperframesResourceUri(maskRoot.surface.artifact.resource))}"/>`;}throw new Error(`Local mask ${element.id} requires a terminal owned text, image or still Surface mask source.`);})();diff --git a/packages/hyperframes/src/project.ts b/packages/hyperframes/src/project.tsindex d273dccd..7bab3e98 100644--- a/packages/hyperframes/src/project.ts+++ b/packages/hyperframes/src/project.ts@@ -12,9 +12,10 @@ export type HyperframesArtifactReader = (artifact: BlobRef,signal?: AbortSignal,) => Promise<Uint8Array | AsyncIterable<Uint8Array>>;+/** Inspect the completed staged file; its lifetime belongs to the staging caller. */export type HyperframesSurfaceValidator = (surface: CompositableSurfaceRef,- bytes: Uint8Array,+ path: string,signal?: AbortSignal,) => Promise<void>;@@ -39,9 +40,7 @@ function extension(mediaType: string): string {/*** Lay one document out as a directory HyperFrames can render.*- * The same layout serves a local render and a distributed one: the local- * Provider serves it to the capture engine, and the AWS Provider tars it into a site. If- * the two laid it out differently, one Need would name two different projects.+ * A local or hosted Provider can serve the same staged document to its capture engine.* Resource names are local to this staged project and carry no content claim.*/export async function stageHyperframesProject(options: {@@ -63,25 +62,25 @@ export async function stageHyperframesProject(options: {await mkdir(artifactDirectory, { recursive: true });const paths = new Map<string, string>();const surfaces = new Map(document.surfaces.map((surface) => [surface.artifact.resource, surface]));- await Promise.allSettled(document.artifacts.map(async (artifact) => {+ await Promise.allSettled(document.artifacts.map(async (artifact, index) => {try {signal.throwIfAborted();- const name = `${artifact.resource}${extension(artifact.mediaType)}`;+ // Resource identities are portable protocol values, not filesystem names.+ const name = `asset-${index}${extension(artifact.mediaType)}`;const opened = await read(artifact, signal);signal.throwIfAborted();const chunks = opened instanceof Uint8Array? (async function* () { yield opened; })(): opened;const surface = surfaces.get(artifact.resource);- const retained: Uint8Array[] = [];let size = 0;- const target = await open(join(artifactDirectory, name), "w");+ const path = join(artifactDirectory, name);+ const target = await open(path, "w");try {for await (const chunk of chunks) {signal.throwIfAborted();- await target.write(chunk);+ await target.writeFile(chunk);size += chunk.byteLength;- if (surface !== undefined) retained.push(Uint8Array.from(chunk));}} finally {await target.close();@@ -89,17 +88,9 @@ export async function stageHyperframesProject(options: {if (size !== artifact.size) {throw new Error(`HyperFrames Artifact ${artifact.resource} size differs`);}- const bytes = surface === undefined ? undefined : (() => {- const value = new Uint8Array(size);- let offset = 0;- for (const chunk of retained) {- value.set(chunk, offset);- offset += chunk.byteLength;- }- return value;- })();if (surface !== undefined) {- await options.validateSurface!(structuredClone(surface), bytes!.slice(), signal);+ // The validator borrows the completed staged file for this call only.+ await options.validateSurface!(structuredClone(surface), path, signal);}paths.set(artifact.resource, `./artifacts/${name}`);} catch (error) {diff --git a/packages/hyperframes/test/document.test.ts b/packages/hyperframes/test/document.test.tsindex 5c1271aa..bd840735 100644--- a/packages/hyperframes/test/document.test.ts+++ b/packages/hyperframes/test/document.test.ts@@ -583,3 +583,15 @@ test("inactive programs settle once, while re-entry and repeated active seeks stseek({ detail: { time: 0.5 } });assert.deepEqual(root.frames, [0, 0, 0, 30, 15, 0]);});++test("an asynchronous browser program reports its unsupported frame behavior instead of racing capture", async () => {+ const { runInNewContext } = await import("node:vm");+ const { browserProgramScript } = await import("../src/browser-program.js");+ const window = { addEventListener() {}, __hypitBrowserProgramError: undefined as string | undefined };+ runInNewContext(browserProgramScript([{ id: "scene", startFrame: 0, durationFrames: 30,+ program: { html: "", setup: 'return async frame => { await Promise.resolve(); throw new Error("late drawing failure"); };' },+ }], 30, 1), { window, document: { getElementById: () => ({}) } });+ assert.match(window.__hypitBrowserProgramError!, /must be synchronous/u);+ await new Promise(resolve => setImmediate(resolve));+ assert.match(window.__hypitBrowserProgramError!, /late drawing failure/u);+});diff --git a/packages/hyperframes/test/project.test.ts b/packages/hyperframes/test/project.test.tsindex 03f17d20..6752fc6e 100644--- a/packages/hyperframes/test/project.test.ts+++ b/packages/hyperframes/test/project.test.ts@@ -1,5 +1,5 @@import assert from "node:assert/strict";-import { mkdtemp, rm } from "node:fs/promises";+import { mkdtemp, readFile, readdir, rm } from "node:fs/promises";import { tmpdir } from "node:os";import { join } from "node:path";import { setTimeout as delay } from "node:timers/promises";@@ -9,6 +9,41 @@ import { stageHyperframesProject } from "../src/project.js";import { sealComposition, sealVisualTrack } from "@hypit/composition";import { sealProgramSpace } from "@hypit/program-space";+test("typed Surface inspection borrows the completed streamed file used by capture", async () => {+ const surface = { artifact: { kind: "blob" as const, resource: "res_streamed-surface" as const,+ size: 6, mediaType: "image/png" }, width: 1, height: 1, colorSpace: "srgb" as const,+ alphaMode: "straight" as const, timing: { kind: "still" as const } };+ const space = sealProgramSpace({ id: "space", durationSec: 1,+ frameRate: { numerator: 30, denominator: 1 } });+ const track = sealVisualTrack({ id: "surface", visualIr: "hypit.visual-ir@1", programSpaceId: space.id,+ presents: [{ id: "surface", span: { startFrame: 0, endFrameExclusive: 30 }, stacking: { order: 0, tieBreak: "surface" },+ elements: [{ id: "surface", kind: "surface", surface, order: 0, style: [] }] }] });+ const document = compileHyperframesDocument(sealComposition({ id: "main",+ canvas: { width: 64, height: 64, clearColor: "#000000" }, tracks: [track] }), space);+ const directory = await mkdtemp(join(tmpdir(), "hypit-stage-surface-"));+ let inspected = false;+ let streamFinished = false;+ try {+ await stageHyperframesProject({ document, directory, read: async () => (async function* () {+ // Readers may reuse their chunk buffer after each write.+ const chunk = new Uint8Array([1, 2, 3]);+ yield chunk;+ chunk.set([4, 5, 6]);+ yield chunk;+ streamFinished = true;+ })(), validateSurface: async (value, path) => {+ assert.ok(streamFinished);+ assert.deepEqual(value, surface);+ assert.equal(path, join(directory, "artifacts", "asset-0.png"));+ assert.deepEqual([...await readFile(path)], [1, 2, 3, 4, 5, 6]);+ inspected = true;+ } });+ assert.ok(inspected);+ assert.deepEqual(await readdir(join(directory, "artifacts")), ["asset-0.png"]);+ assert.match(await readFile(join(directory, "index.html"), "utf8"), /\.\/artifacts\/asset-0\.png/u);+ } finally { await rm(directory, { recursive: true, force: true }); }+});+test("staging cancels and joins sibling reads before exposing a failure", async () => {const space = sealProgramSpace({ id: "space", durationSec: 1,frameRate: { numerator: 30, denominator: 1 } });@@ -32,3 +67,27 @@ test("staging cancels and joins sibling reads before exposing a failure", asyncassert.equal(active, 0);} finally { await rm(directory, { recursive: true, force: true }); }});++test("staging gives opaque Resource identities portable local names and writes complete streams", async () => {+ const resource = `res_${"x".repeat(247)}:img` as const;+ const space = sealProgramSpace({ id: "space", durationSec: 1, frameRate: { numerator: 30, denominator: 1 } });+ const track = sealVisualTrack({ id: "image", visualIr: "hypit.visual-ir@1", programSpaceId: space.id,+ presents: [{ id: "image", span: { startFrame: 0, endFrameExclusive: 30 }, stacking: { order: 0, tieBreak: "image" },+ elements: [{ id: "image", kind: "image", order: 0, style: [],+ artifact: { kind: "blob", resource, size: 5, mediaType: "image/png" } }] }] });+ const document = compileHyperframesDocument(sealComposition({ id: "main", canvas: { width: 64, height: 64, clearColor: "#000000" }, tracks: [track] }), space);+ const directory = await mkdtemp(join(tmpdir(), "hypit-stage-identity-"));+ try {+ await stageHyperframesProject({ document, directory, read: async artifact => {+ assert.equal(artifact.resource, resource);+ return (async function* () { yield new Uint8Array([1, 2]); yield new Uint8Array([3, 4, 5]); })();+ } });+ const files = await readdir(join(directory, "artifacts"));+ assert.equal(files.length, 1);+ assert.match(files[0]!, /^[a-z0-9-]+\.png$/u);+ assert.deepEqual([...await readFile(join(directory, "artifacts", files[0]!))], [1, 2, 3, 4, 5]);+ const html = await readFile(join(directory, "index.html"), "utf8");+ assert.ok(html.includes(`./artifacts/${files[0]}`));+ assert.ok(document.html.includes(resource), "materialization does not rewrite the portable document");+ } finally { await rm(directory, { recursive: true, force: true }); }+});