Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

fix(hyperframes): preserve element animation and resource staging

kashorin08ba946aa820
diff --git a/packages/hyperframes/README.md b/packages/hyperframes/README.md
index 65f55c08..f2209cf9 100644
--- a/packages/hyperframes/README.md
+++ b/packages/hyperframes/README.md
@@ -12,6 +12,8 @@ This package understands only `VisualTrack`, `ProgramSpace` and canvas geometry.
does not know Caption, Speech, B-roll, Seedance or any other author-domain component. It flattens
every Track's Presents, orders them by their own absolute stacking keys and emits frame-bound local
keyframes. An authoring Track never becomes an isolated render stacking surface.
+Terminal text and SVG mask sources retain the same Present-relative frame animations as other
+elements, including direct seeks into the middle of a Present.
It deliberately ignores `AudioTrack`. HyperFrames produces a silent visual fact; the media pipeline
compiles and renders program audio separately, then an explicit mux Provider joins the two. Changing
@@ -23,6 +25,12 @@ a Provider can verify and name staged bytes without guessing from the hash. A lo
render Runtime calls `materializeHyperframesHtml()` with its own Artifact URL resolver before
handing the HTML to HyperFrames. That environment-specific materialization is not a new compiled
Record and does not change the compiled document.
+Staging assigns short local filenames; opaque Resource IDs are never interpreted as filesystem
+paths or filenames, so their length and punctuation do not restrict the host filesystem.
+`stageHyperframesProject` streams each asset into its staged file. Its `validateSurface(surface, path,
+signal)` callback borrows that completed file during inspection; it receives neither a whole-file
+byte copy nor ownership of the file. The staging caller keeps the directory alive until all work
+has settled, including cancellation.
The document exposes its render domain directly rather than asking an Endpoint to scrape HTML:
exact rational `frameRate`, integer `frameCount`, and canvas dimensions are explicit document
@@ -53,8 +61,12 @@ places a direct typed child. Every child is placed once, including sampled video
This keeps actual video available to the renderer's exact source-frame preparation.
CSS is scoped to the generated program root with `@scope`; `:scope` styles that root. The HTML can
-contain arbitrary local structure, SVG, internal stacking, masks and backdrop filters. The optional
-`setup` string is a JavaScript function body with `root` and `data` arguments. It returns a synchronous
+contain arbitrary local structure, SVG, internal stacking, masks and backdrop filters.
+CSS scope does not isolate HTML/SVG IDs. Use classes or data attributes for local queries; when an
+SVG needs an ID for `url(#...)` or `href`, derive it from the unique `root.id` in `setup` and set both
+the definition and its references there. Reusable instances must not repeat hard-coded SVG IDs.
+
+The optional `setup` string is a JavaScript function body with `root` and `data` arguments. It returns a synchronous
`render(localFrame)` function, evaluated on initial load and active `hf-seek` events. Outside the
Present's lifetime, the dispatcher settles the boundary pose once instead of repeatedly updating
an invisible program. Active seeks always redraw, including the same frame after resources become
@@ -63,6 +75,8 @@ objects in `setup`; express animation state as a function of frame and inputs so
begin at any frame. Async work belongs to
material preparation before rendering; browser resources belong in the program's declared artifacts.
Use `hyperframesResourceUri(artifact.resource)` in resource-bearing markup or CSS.
+Returning a Promise from `render` reports an authoring error; frame capture never waits for an
+unbounded asynchronous drawing task or races it.
Ordinary child sampling follows the Present clock. Reframing the parent leaves source playback
unchanged. `projectTimelineMedia` from `@hypit/hypit/timeline` gives a component selected prepared
diff --git a/packages/hyperframes/src/browser-program.ts b/packages/hyperframes/src/browser-program.ts
index f4265f6a..e360dff8 100644
--- a/packages/hyperframes/src/browser-program.ts
+++ b/packages/hyperframes/src/browser-program.ts
@@ -76,7 +76,12 @@ export function browserProgramScript(entries: readonly {
// including repeated frame 0 after images/fonts finish loading.
if (region !== 'active' && entry.region === region) continue;
try {
- entry.render(Math.max(0, Math.min(entry.durationFrames, local)));
+ const result = entry.render(Math.max(0, Math.min(entry.durationFrames, local)));
+ if (result != null && typeof result.then === 'function') {
+ // Observe a later rejection, but never let asynchronous drawing race frame capture.
+ Promise.resolve(result).catch(fail);
+ throw new Error('Browser program render(localFrame) must be synchronous; prepare asynchronous resources before rendering.');
+ }
entry.region = region;
}
catch (error) { fail(error); }
diff --git a/packages/hyperframes/src/document.ts b/packages/hyperframes/src/document.ts
index 9ece71dc..41ede3e5 100644
--- a/packages/hyperframes/src/document.ts
+++ b/packages/hyperframes/src/document.ts
@@ -229,23 +229,22 @@ function exactFontStyle(element: VisualElement, stableId: StableDomId): string[]
];
}
-function renderElement(
- element: VisualElement,
- children: ReadonlyMap<string, readonly VisualElement[]>,
- context: {
- readonly trackId: string;
- readonly presentId: string;
- readonly presentStart: string;
- readonly presentDuration: string;
- readonly presentDurationFrames: number;
- readonly presentStartFrame: number;
- readonly programNumerator: number;
- readonly programDenominator: number;
- readonly stackIndex: number;
- readonly emittedFilterIds: Set<string>;
- readonly stableId: StableDomId;
- },
-): string {
+type ElementContext = {
+ readonly trackId: string;
+ readonly presentId: string;
+ readonly presentStart: string;
+ readonly presentDuration: string;
+ readonly presentDurationFrames: number;
+ readonly presentStartFrame: number;
+ readonly programNumerator: number;
+ readonly programDenominator: number;
+ readonly stackIndex: number;
+ readonly emittedFilterIds: Set<string>;
+ readonly stableId: StableDomId;
+};
+
+/** Every emitted element, including SVG mask sources, keeps the same authored frame clock. */
+function elementPresentation(element: VisualElement, context: ElementContext) {
const id = context.stableId([context.trackId, context.presentId, element.id]);
const animationName = element.animation === undefined
? undefined
@@ -277,6 +276,16 @@ function renderElement(
? ""
: ` data-hypit-frame-animation data-hypit-animation-start-frame="${context.presentStartFrame}" data-hypit-animation-duration-frames="${animationDurationFrames}" data-hypit-animation-sample-frames="${context.presentDurationFrames}" data-hypit-animation-properties="${animationProperties.join(",")}"`;
const common = `${commonAttributes}${animationAttributes} style="${escapeHtml(inlineStyle)}"`;
+ return { inlineStyle, commonAttributes: `${commonAttributes}${animationAttributes}`, common };
+}
+
+function renderElement(
+ element: VisualElement,
+ children: ReadonlyMap<string, readonly VisualElement[]>,
+ context: ElementContext,
+): string {
+ const { inlineStyle, commonAttributes, common } = elementPresentation(element, context);
+ const id = context.stableId([context.trackId, context.presentId, element.id]);
if (element.kind === "mask") {
const direct = children.get(element.id) ?? [];
const maskRoot = direct.find((child) => child.id === element.maskElement);
@@ -295,6 +304,7 @@ function renderElement(
throw new Error(`Local mask ${element.id} mask source must be one terminal owned element.`);
}
const maskSource = (() => {
+ const sourcePresentation = elementPresentation(maskRoot, context);
if (maskRoot.kind === "text") {
const alignment = maskRoot.style.find((declaration) => declaration.name === "text-align")?.value;
const anchor = alignment === "right" || alignment === "end" ? "end" : alignment === "left" || alignment === "start" ? "start" : "middle";
@@ -310,17 +320,17 @@ function renderElement(
? blockAlignment === "flex-start" ? "0" : blockAlignment === "flex-end" ? "100%" : "50%"
: String(blockAlignment === "flex-start" ? paddingTop : blockAlignment === "flex-end" ? maskHeight - paddingBottom : (paddingTop + maskHeight - paddingBottom) / 2);
const baseline = blockAlignment === "flex-start" ? "text-before-edge" : blockAlignment === "flex-end" ? "text-after-edge" : "central";
- const textStyle = [css(maskRoot.style), ...exactFontStyle(maskRoot, context.stableId), "fill:currentColor"].filter(Boolean).join(";");
- return `<text${attributes(maskRoot.attributes)} x="${x}" y="${y}" text-anchor="${anchor}" dominant-baseline="${baseline}" style="${escapeHtml(textStyle)}">${escapeHtml(maskRoot.text)}</text>`;
+ const textStyle = `${sourcePresentation.inlineStyle};fill:currentColor`;
+ return `<text ${sourcePresentation.commonAttributes} x="${x}" y="${y}" text-anchor="${anchor}" dominant-baseline="${baseline}" style="${escapeHtml(textStyle)}">${escapeHtml(maskRoot.text)}</text>`;
}
if (maskRoot.kind === "text-flow" || maskRoot.kind === "path-text") {
return `<foreignObject x="0" y="0" width="100%" height="100%"><div xmlns="http://www.w3.org/1999/xhtml" style="position:relative;width:100%;height:100%">${renderOwned(maskRoot)}</div></foreignObject>`;
}
if (maskRoot.kind === "image") {
- return `<image x="0" y="0" width="100%" height="100%" preserveAspectRatio="none" href="${escapeHtml(hyperframesResourceUri(maskRoot.artifact.resource))}" style="${escapeHtml(css(maskRoot.style))}"/>`;
+ return `<image ${sourcePresentation.common} x="0" y="0" width="100%" height="100%" preserveAspectRatio="none" href="${escapeHtml(hyperframesResourceUri(maskRoot.artifact.resource))}"/>`;
}
if (maskRoot.kind === "surface" && maskRoot.surface.timing.kind === "still") {
- return `<image data-hypit-surface-resource="${maskRoot.surface.artifact.resource}" x="0" y="0" width="100%" height="100%" preserveAspectRatio="none" href="${escapeHtml(hyperframesResourceUri(maskRoot.surface.artifact.resource))}" style="${escapeHtml(css(maskRoot.style))}"/>`;
+ return `<image ${sourcePresentation.common} data-hypit-surface-resource="${maskRoot.surface.artifact.resource}" x="0" y="0" width="100%" height="100%" preserveAspectRatio="none" href="${escapeHtml(hyperframesResourceUri(maskRoot.surface.artifact.resource))}"/>`;
}
throw new Error(`Local mask ${element.id} requires a terminal owned text, image or still Surface mask source.`);
})();
diff --git a/packages/hyperframes/src/project.ts b/packages/hyperframes/src/project.ts
index d273dccd..7bab3e98 100644
--- a/packages/hyperframes/src/project.ts
+++ b/packages/hyperframes/src/project.ts
@@ -12,9 +12,10 @@ export type HyperframesArtifactReader = (
artifact: BlobRef,
signal?: AbortSignal,
) => Promise<Uint8Array | AsyncIterable<Uint8Array>>;
+/** Inspect the completed staged file; its lifetime belongs to the staging caller. */
export type HyperframesSurfaceValidator = (
surface: CompositableSurfaceRef,
- bytes: Uint8Array,
+ path: string,
signal?: AbortSignal,
) => Promise<void>;
@@ -39,9 +40,7 @@ function extension(mediaType: string): string {
/**
* Lay one document out as a directory HyperFrames can render.
*
- * The same layout serves a local render and a distributed one: the local
- * Provider serves it to the capture engine, and the AWS Provider tars it into a site. If
- * the two laid it out differently, one Need would name two different projects.
+ * A local or hosted Provider can serve the same staged document to its capture engine.
* Resource names are local to this staged project and carry no content claim.
*/
export async function stageHyperframesProject(options: {
@@ -63,25 +62,25 @@ export async function stageHyperframesProject(options: {
await mkdir(artifactDirectory, { recursive: true });
const paths = new Map<string, string>();
const surfaces = new Map(document.surfaces.map((surface) => [surface.artifact.resource, surface]));
- await Promise.allSettled(document.artifacts.map(async (artifact) => {
+ await Promise.allSettled(document.artifacts.map(async (artifact, index) => {
try {
signal.throwIfAborted();
- const name = `${artifact.resource}${extension(artifact.mediaType)}`;
+ // Resource identities are portable protocol values, not filesystem names.
+ const name = `asset-${index}${extension(artifact.mediaType)}`;
const opened = await read(artifact, signal);
signal.throwIfAborted();
const chunks = opened instanceof Uint8Array
? (async function* () { yield opened; })()
: opened;
const surface = surfaces.get(artifact.resource);
- const retained: Uint8Array[] = [];
let size = 0;
- const target = await open(join(artifactDirectory, name), "w");
+ const path = join(artifactDirectory, name);
+ const target = await open(path, "w");
try {
for await (const chunk of chunks) {
signal.throwIfAborted();
- await target.write(chunk);
+ await target.writeFile(chunk);
size += chunk.byteLength;
- if (surface !== undefined) retained.push(Uint8Array.from(chunk));
}
} finally {
await target.close();
@@ -89,17 +88,9 @@ export async function stageHyperframesProject(options: {
if (size !== artifact.size) {
throw new Error(`HyperFrames Artifact ${artifact.resource} size differs`);
}
- const bytes = surface === undefined ? undefined : (() => {
- const value = new Uint8Array(size);
- let offset = 0;
- for (const chunk of retained) {
- value.set(chunk, offset);
- offset += chunk.byteLength;
- }
- return value;
- })();
if (surface !== undefined) {
- await options.validateSurface!(structuredClone(surface), bytes!.slice(), signal);
+ // The validator borrows the completed staged file for this call only.
+ await options.validateSurface!(structuredClone(surface), path, signal);
}
paths.set(artifact.resource, `./artifacts/${name}`);
} catch (error) {
diff --git a/packages/hyperframes/test/document.test.ts b/packages/hyperframes/test/document.test.ts
index 5c1271aa..bd840735 100644
--- a/packages/hyperframes/test/document.test.ts
+++ b/packages/hyperframes/test/document.test.ts
@@ -583,3 +583,15 @@ test("inactive programs settle once, while re-entry and repeated active seeks st
seek({ detail: { time: 0.5 } });
assert.deepEqual(root.frames, [0, 0, 0, 30, 15, 0]);
});
+
+test("an asynchronous browser program reports its unsupported frame behavior instead of racing capture", async () => {
+ const { runInNewContext } = await import("node:vm");
+ const { browserProgramScript } = await import("../src/browser-program.js");
+ const window = { addEventListener() {}, __hypitBrowserProgramError: undefined as string | undefined };
+ runInNewContext(browserProgramScript([{ id: "scene", startFrame: 0, durationFrames: 30,
+ program: { html: "", setup: 'return async frame => { await Promise.resolve(); throw new Error("late drawing failure"); };' },
+ }], 30, 1), { window, document: { getElementById: () => ({}) } });
+ assert.match(window.__hypitBrowserProgramError!, /must be synchronous/u);
+ await new Promise(resolve => setImmediate(resolve));
+ assert.match(window.__hypitBrowserProgramError!, /late drawing failure/u);
+});
diff --git a/packages/hyperframes/test/project.test.ts b/packages/hyperframes/test/project.test.ts
index 03f17d20..6752fc6e 100644
--- a/packages/hyperframes/test/project.test.ts
+++ b/packages/hyperframes/test/project.test.ts
@@ -1,5 +1,5 @@
import assert from "node:assert/strict";
-import { mkdtemp, rm } from "node:fs/promises";
+import { mkdtemp, readFile, readdir, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { setTimeout as delay } from "node:timers/promises";
@@ -9,6 +9,41 @@ import { stageHyperframesProject } from "../src/project.js";
import { sealComposition, sealVisualTrack } from "@hypit/composition";
import { sealProgramSpace } from "@hypit/program-space";
+test("typed Surface inspection borrows the completed streamed file used by capture", async () => {
+ const surface = { artifact: { kind: "blob" as const, resource: "res_streamed-surface" as const,
+ size: 6, mediaType: "image/png" }, width: 1, height: 1, colorSpace: "srgb" as const,
+ alphaMode: "straight" as const, timing: { kind: "still" as const } };
+ const space = sealProgramSpace({ id: "space", durationSec: 1,
+ frameRate: { numerator: 30, denominator: 1 } });
+ const track = sealVisualTrack({ id: "surface", visualIr: "hypit.visual-ir@1", programSpaceId: space.id,
+ presents: [{ id: "surface", span: { startFrame: 0, endFrameExclusive: 30 }, stacking: { order: 0, tieBreak: "surface" },
+ elements: [{ id: "surface", kind: "surface", surface, order: 0, style: [] }] }] });
+ const document = compileHyperframesDocument(sealComposition({ id: "main",
+ canvas: { width: 64, height: 64, clearColor: "#000000" }, tracks: [track] }), space);
+ const directory = await mkdtemp(join(tmpdir(), "hypit-stage-surface-"));
+ let inspected = false;
+ let streamFinished = false;
+ try {
+ await stageHyperframesProject({ document, directory, read: async () => (async function* () {
+ // Readers may reuse their chunk buffer after each write.
+ const chunk = new Uint8Array([1, 2, 3]);
+ yield chunk;
+ chunk.set([4, 5, 6]);
+ yield chunk;
+ streamFinished = true;
+ })(), validateSurface: async (value, path) => {
+ assert.ok(streamFinished);
+ assert.deepEqual(value, surface);
+ assert.equal(path, join(directory, "artifacts", "asset-0.png"));
+ assert.deepEqual([...await readFile(path)], [1, 2, 3, 4, 5, 6]);
+ inspected = true;
+ } });
+ assert.ok(inspected);
+ assert.deepEqual(await readdir(join(directory, "artifacts")), ["asset-0.png"]);
+ assert.match(await readFile(join(directory, "index.html"), "utf8"), /\.\/artifacts\/asset-0\.png/u);
+ } finally { await rm(directory, { recursive: true, force: true }); }
+});
+
test("staging cancels and joins sibling reads before exposing a failure", async () => {
const space = sealProgramSpace({ id: "space", durationSec: 1,
frameRate: { numerator: 30, denominator: 1 } });
@@ -32,3 +67,27 @@ test("staging cancels and joins sibling reads before exposing a failure", async
assert.equal(active, 0);
} finally { await rm(directory, { recursive: true, force: true }); }
});
+
+test("staging gives opaque Resource identities portable local names and writes complete streams", async () => {
+ const resource = `res_${"x".repeat(247)}:img` as const;
+ const space = sealProgramSpace({ id: "space", durationSec: 1, frameRate: { numerator: 30, denominator: 1 } });
+ const track = sealVisualTrack({ id: "image", visualIr: "hypit.visual-ir@1", programSpaceId: space.id,
+ presents: [{ id: "image", span: { startFrame: 0, endFrameExclusive: 30 }, stacking: { order: 0, tieBreak: "image" },
+ elements: [{ id: "image", kind: "image", order: 0, style: [],
+ artifact: { kind: "blob", resource, size: 5, mediaType: "image/png" } }] }] });
+ const document = compileHyperframesDocument(sealComposition({ id: "main", canvas: { width: 64, height: 64, clearColor: "#000000" }, tracks: [track] }), space);
+ const directory = await mkdtemp(join(tmpdir(), "hypit-stage-identity-"));
+ try {
+ await stageHyperframesProject({ document, directory, read: async artifact => {
+ assert.equal(artifact.resource, resource);
+ return (async function* () { yield new Uint8Array([1, 2]); yield new Uint8Array([3, 4, 5]); })();
+ } });
+ const files = await readdir(join(directory, "artifacts"));
+ assert.equal(files.length, 1);
+ assert.match(files[0]!, /^[a-z0-9-]+\.png$/u);
+ assert.deepEqual([...await readFile(join(directory, "artifacts", files[0]!))], [1, 2, 3, 4, 5]);
+ const html = await readFile(join(directory, "index.html"), "utf8");
+ assert.ok(html.includes(`./artifacts/${files[0]}`));
+ assert.ok(document.html.includes(resource), "materialization does not rewrite the portable document");
+ } finally { await rm(directory, { recursive: true, force: true }); }
+});