Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

feat: the shapes a component may emit are a command, and so is where a command read from (#77)

rponeawa18736a04834c
diff --git a/.agents/skills/hypit/references/environment.md b/.agents/skills/hypit/references/environment.md
index 46e975cb..f906c840 100644
--- a/.agents/skills/hypit/references/environment.md
+++ b/.agents/skills/hypit/references/environment.md
@@ -99,7 +99,10 @@ Run this skill-owned probe from the installed skill directory when diagnosing a
node scripts/check-environment.mjs
```
-`hypit` and Node are required. `ffmpeg` and `ffprobe` are required by local media and preview paths.
+`hypit` and Node are required. `ffmpeg` and `ffprobe` are required by local media and preview paths; a
+host package manager puts them on `PATH` on macOS and Linux, and the section below unpacks them by
+hand on Windows. Credentials load from a project's `.env` in the same shell that runs the commands:
+`set -a && . ./.env && set +a`.
`uv` is required only when the selected Runtime Profile uses a managed Python program such as
WhisperX or OpenCV. Install `uv` with the host package manager, then let `uv` install the pinned
Python interpreter.
diff --git a/.agents/skills/hypit/references/local-author-package.md b/.agents/skills/hypit/references/local-author-package.md
index d45d49eb..5f6eca4f 100644
--- a/.agents/skills/hypit/references/local-author-package.md
+++ b/.agents/skills/hypit/references/local-author-package.md
@@ -44,8 +44,20 @@ Then read:
next one, and that is what a new package is built out of. Author every file yourself against them:
the Manifest, the Types, the Surface and decoder, the Producers, the Fragment.
-Open a package's source when the docs genuinely do not settle a question — a role whose shape the
-guide leaves implicit, an export whose signature you need exactly. Use `hypit paths --json` to locate
+**What a Producer that draws may return is a command, not a package to read.**
+
+```
+hypit-reference-video-tools inspect_visual_contract
+```
+
+It answers the questions a component is written against — which element kinds exist, which style
+names are admitted on them, which of those take an enum, how few keyframes an animation carries, and
+the four rules the seal enforces about parents, `order` and interpolation. Every line is generated
+from the Composition schema, so it says what will be accepted rather than what one package happened
+to do. `inspect_svml_vocabulary` answers the other half: what a Source may write.
+
+Open a package's source only when neither command settles it — a role whose shape the guide leaves
+implicit, an export whose signature you need exactly. Use `hypit paths --json` to locate
the installed Distribution and read **the one role you are stuck on**, not the package end to end.
Find it by what it exports, since the filenames differ: `ranking` calls two of them `schedule.ts` and
`render.ts`, `media-track` calls them `program.ts` and `lower.ts`, `comment-sticker` calls them
diff --git a/.agents/skills/hypit/references/reconstruction/workflow.md b/.agents/skills/hypit/references/reconstruction/workflow.md
index 40d41b79..4c507e82 100644
--- a/.agents/skills/hypit/references/reconstruction/workflow.md
+++ b/.agents/skills/hypit/references/reconstruction/workflow.md
@@ -25,7 +25,8 @@ the repository root.
A reference is found from the Distribution rather than from where you are standing, so these commands
may be run from anywhere. What does depend on where you are is which packages resolve: name the
-project with `--package-root` when the command is not run from inside it.
+project with `--package-root` when the command is not run from inside it. `paths` reports both roots
+and every prepared reference, which is what to run when a command reports something it cannot see.
Defaults are sufficient for normal use. Each also accepts `--input <json>`. Follow this sequence:
diff --git a/packages/reference-video-tools/package.json b/packages/reference-video-tools/package.json
index 47b08d4c..f73b60e5 100644
--- a/packages/reference-video-tools/package.json
+++ b/packages/reference-video-tools/package.json
@@ -13,6 +13,7 @@
},
"dependencies": {
"@google/genai": "1.52.0",
+ "@hypit/composition": "workspace:*",
"@hypit/driver-node": "workspace:*",
"@hypit/estimate": "workspace:*",
"@hypit/hyperframes": "workspace:*",
diff --git a/packages/reference-video-tools/src/cli.ts b/packages/reference-video-tools/src/cli.ts
index 5f1b989c..66746614 100644
--- a/packages/reference-video-tools/src/cli.ts
+++ b/packages/reference-video-tools/src/cli.ts
@@ -16,6 +16,8 @@ function usage(): string {
" hypit-reference-video-tools observe_reference --reference-id <id> --batch <questions.json>",
" hypit-reference-video-tools record_observation --reference-id <id> --key <key> --text <text>|--text-file <path>",
" hypit-reference-video-tools inspect_svml_vocabulary --package <name> [--package <name> ...] [--tag <tag> ...] [--without-previews]",
+ " hypit-reference-video-tools inspect_visual_contract [--shape visual-track|text-flow|text-typography|text-paint|text-document|path-command]",
+ " hypit-reference-video-tools paths",
" hypit-reference-video-tools compare_reconstruction --reference-id <id> --run <build.svrun> --segment <id>|--selection <id> --video <path>|--image <path> [--question <scope>] [--element <id>]",
" hypit-reference-video-tools compare_reconstruction --reference-id <id> --shot-id <id> --video <path>|--image <path> [--question <scope>] [--element <id>]",
" hypit-reference-video-tools compare_reconstruction --reference-id <id> --batch <comparisons.json>",
@@ -128,6 +130,14 @@ function usage(): string {
"returns each observation as a task carrying its prompt and one tiled picture per shot, which the",
"calling agent answers with record_observation.",
"",
+ "inspect_visual_contract answers what a Producer that draws may return: the element kinds, the style",
+ "names admitted on them, which take an enum, and how few keyframes an animation carries. Every line is",
+ "generated from the Composition schema, so it says what the seal will accept rather than what one",
+ "package happened to do. inspect_svml_vocabulary answers the other half — what a Source may write.",
+ "",
+ "paths reports where this command reads reference state and resolves packages from, and which",
+ "references are prepared. Use it when a check reports something it cannot see.",
+ "",
"--package-root <dir> is where the packages a Source imports are resolved from, and it is accepted by",
"every command. It defaults to the working directory. A project that declares a vocabulary gap and",
"fills it publishes those packages under its own scope and installs them against the project root, so",
@@ -295,6 +305,10 @@ async function main(): Promise<void> {
...(one(flags, "element") === undefined ? {} : { element: one(flags, "element") }),
};
result = await tools.compare_reconstruction(input as { reference_id: string; shot_id?: string; segment?: string; selection?: string; run?: string; image_path?: string; video_path?: string; question?: string; element?: string });
+ } else if (command === "inspect_visual_contract") {
+ result = await tools.inspect_visual_contract({ ...(one(flags, "shape") === undefined ? {} : { shape: one(flags, "shape")! }) });
+ } else if (command === "paths") {
+ result = await tools.paths();
} else if (command === "inspect_svml_vocabulary") {
const packages = many(flags, "package");
const input = supplied ?? {
diff --git a/packages/reference-video-tools/src/contract.ts b/packages/reference-video-tools/src/contract.ts
new file mode 100644
index 00000000..177e2fa2
--- /dev/null
+++ b/packages/reference-video-tools/src/contract.ts
@@ -0,0 +1,61 @@
+import type { ValueSchema } from "@hypit/protocol";
+
+/**
+ * Read a declared shape back as prose an author can act on.
+ *
+ * A package that draws returns a Visual Track from its Producer, and every field on it — which
+ * element kinds exist, which style names are admitted, which of them take an enum, how few keyframes
+ * an animation may have — is declared in `composition`'s own schema. Nothing read that schema, so the
+ * only way to learn the shape was to open another package and copy it, which is how a component gets
+ * written against one example's habits rather than against the contract.
+ *
+ * This renders the schema instead. It is generated from the declaration, so it cannot say something
+ * the code does not.
+ */
+export function describeSchema(schema: ValueSchema, indent = ""): string[] {
+ const step = `${indent} `;
+ switch (schema.kind) {
+ case "object": {
+ const lines: string[] = [];
+ for (const [name, field] of Object.entries(schema.fields)) {
+ const inner = describeSchema(field.schema, step);
+ const optional = field.optional === true ? " (optional)" : "";
+ lines.push(`${indent}${name}${optional}: ${inner[0]!.trimStart()}`, ...inner.slice(1));
+ }
+ if (schema.allowUnknown === true) lines.push(`${indent}… further fields are admitted`);
+ return lines;
+ }
+ case "oneOf": {
+ // A union of objects each pinned by one literal is how the schema spells "these kinds"; naming
+ // the discriminating values is more use than printing every variant in full.
+ const pinned = schema.variants.map((variant) => {
+ if (variant.kind !== "object") return undefined;
+ const literal = Object.entries(variant.fields)
+ .find(([, field]) => field.schema.kind === "literal");
+ return literal === undefined ? undefined : String((literal[1].schema as { value: unknown }).value);
+ });
+ if (pinned.every((value) => value !== undefined)) return [`${indent}one of ${pinned.join(", ")}`];
+ return [`${indent}one of ${schema.variants.length} shapes`];
+ }
+ case "array": {
+ const bound = schema.minItems === undefined ? "" : ` (at least ${schema.minItems})`;
+ const inner = describeSchema(schema.items, step);
+ return [`${indent}a list${bound} of:`, ...inner];
+ }
+ case "string": {
+ if (schema.enum !== undefined) return [`${indent}one of ${schema.enum.join(", ")}`];
+ const bound = schema.minLength === undefined ? "text" : `text, at least ${schema.minLength} characters`;
+ return [`${indent}${bound}`];
+ }
+ case "number": {
+ const parts = [schema.integer === true ? "a whole number" : "a number"];
+ if (schema.minimum !== undefined) parts.push(`at least ${schema.minimum}`);
+ if (schema.maximum !== undefined) parts.push(`at most ${schema.maximum}`);
+ return [`${indent}${parts.join(", ")}`];
+ }
+ case "literal": return [`${indent}exactly ${JSON.stringify(schema.value)}`];
+ case "boolean": return [`${indent}true or false`];
+ case "null": return [`${indent}null`];
+ default: return [`${indent}${(schema as { kind: string }).kind}`];
+ }
+}
diff --git a/packages/reference-video-tools/src/tools.ts b/packages/reference-video-tools/src/tools.ts
index 56886445..4bb9953f 100644
--- a/packages/reference-video-tools/src/tools.ts
+++ b/packages/reference-video-tools/src/tools.ts
@@ -10,6 +10,13 @@ import { loadNodePackageSelection } from "@hypit/package-loader-node";
import { markupSurfaceHostFacetAbi } from "@hypit/markup";
import type { RegisteredSurface, SurfaceVocabulary } from "@hypit/markup";
import { exactModelHostAbi } from "@hypit/model-kit";
+import type { ValueSchema } from "@hypit/protocol";
+import {
+ visualPathCommandSchema, visualTextDocumentSchema, visualTextFlowSchema,
+ visualTextPaintSchema, visualTextTypographySchema, visualTrackSchema,
+} from "@hypit/composition";
+
+import { describeSchema } from "./contract.js";
import { videoCliDistribution } from "@hypit/video-cli";
import { authorSource, invokedFrom, referenceRoot, referenceWords, renderElement, renderPreviews, spokenRange, standInSidecarPath } from "./authoring.js";
@@ -119,6 +126,8 @@ export type ReferenceVideoTools = {
prepare_reference(input: PrepareReferenceInput): Promise<PrepareResult>;
observe_reference(input: ObserveReferenceInput): Promise<Record<string, unknown>>;
inspect_svml_vocabulary(input: InspectVocabularyInput): Promise<Record<string, unknown>>;
+ inspect_visual_contract(input: { readonly shape?: string }): Promise<Record<string, unknown>>;
+ paths(): Promise<Record<string, unknown>>;
compare_reconstruction(input: CompareReconstructionInput): Promise<Record<string, unknown>>;
record_observation(input: RecordObservationInput): Promise<Record<string, unknown>>;
make_placeholder(input: MakePlaceholderInput): Promise<Record<string, unknown>>;
@@ -1213,6 +1222,56 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceV
return { packages: input.package_names, surfaces };
},
+ /**
+ * What a Producer that draws is allowed to return.
+ *
+ * `inspect_svml_vocabulary` answers what a Source may write; this answers what the package behind
+ * it may emit — the element kinds, the style names admitted on them, which of those take an enum,
+ * and how few keyframes an animation may carry. Every line is generated from `composition`'s own
+ * declared schema, so it cannot drift from what the seal will accept. Without it the only way to
+ * learn the shape was to open a package that already draws and copy its habits.
+ */
+ async inspect_visual_contract(input): Promise<Record<string, unknown>> {
+ const shapes: Record<string, ValueSchema> = {
+ "visual-track": visualTrackSchema,
+ "text-flow": visualTextFlowSchema,
+ "text-typography": visualTextTypographySchema,
+ "text-paint": visualTextPaintSchema,
+ "text-document": visualTextDocumentSchema,
+ "path-command": visualPathCommandSchema,
+ };
+ const asked = input.shape?.trim();
+ assert(asked === undefined || asked.length === 0 || Object.hasOwn(shapes, asked),
+ `shape must be one of ${Object.keys(shapes).join(", ")}`);
+ const chosen = asked === undefined || asked.length === 0 ? Object.keys(shapes) : [asked];
+ return {
+ shapes: chosen.map((name) => ({ shape: name, describes: describeSchema(shapes[name]!).join("\n") })),
+ // Each of these is refused somewhere, or follows from how the emitted CSS is written. None is
+ // a convention: a rule stated here that the code does not hold would be worse than silence.
+ rules: [
+ "A Present holds exactly one element with no `parent`; every other element names one, and it "
+ + "must be a box or a mask. (composition/src/track.ts)",
+ "`order` is unique across the whole Present, not among siblings. (composition/src/track.ts)",
+ "A child's position is measured from its parent's box, not from the Canvas, so a layout "
+ + "computed in Canvas pixels subtracts the parent's origin.",
+ "An animation carries at least two keyframes, and every keyframe of one animation declares "
+ + "the same properties: one that appears in some and not others is interpolated from the "
+ + "element's own value on the frames it is missing from.",
+ ],
+ };
+ },
+
+ /** Where this command reads and resolves from, so nothing has to describe it from outside. */
+ async paths(): Promise<Record<string, unknown>> {
+ return {
+ reference_state: referenceRoot(),
+ package_root: packageRoot,
+ working_directory: invokedFrom(),
+ references: (await readdir(referenceRoot(), { withFileTypes: true }).catch(() => []))
+ .filter((entry) => entry.isDirectory()).map((entry) => entry.name),
+ };
+ },
+
async compare_reconstruction(input): Promise<Record<string, unknown>> {
// Every render is finished before any comparison starts, so the comparisons are independent of
// each other and there is nothing to gain by running them one at a time. Handing the whole list
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index fb98c34a..45097825 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -1970,6 +1970,9 @@ importers:
'@google/genai':
specifier: 1.52.0
version: 1.52.0
+ '@hypit/composition':
+ specifier: workspace:*
+ version: link:../composition
'@hypit/driver-node':
specifier: workspace:*
version: link:../driver-node