Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.
TypeScript agentic-aiaiai-agentsai-videocompilerdslffmpeggenerative-aillmmarkup-languagemonorepoplugin-systemprogramming-languagetext-to-videotypescriptvideovideo-automationvideo-clonevideo-editingvideo-generation
Three defects a reconstruction round ran into (#91)
rponeawa3115fb0ef86e
diff --git a/.agents/skills/hypit/references/playbooks/craft/persona-and-audio.md b/.agents/skills/hypit/references/playbooks/craft/persona-and-audio.mdindex 8d43130b..6a1de536 100644--- a/.agents/skills/hypit/references/playbooks/craft/persona-and-audio.md+++ b/.agents/skills/hypit/references/playbooks/craft/persona-and-audio.md@@ -68,7 +68,7 @@ as `SynchronizedMedia`, then place it with `audio:Track`:- Keep primary speech, music, ambience, and effects as independently inspectable contributions. AddSpeech Track's audio projection and every selected Audio Track to `film:Film`.-- Raw Media Track video is visual-only unless `audio="include"` is explicitly authored.+- A Media Track carries picture alone unless the source was normalized with `audio="default"`.- Dialogue wins the mix. Use explicit gain and fades; do not assume automatic ducking, loudnessnormalization, or mastering.diff --git a/.agents/skills/hypit/references/playbooks/craft/pip-overlay.md b/.agents/skills/hypit/references/playbooks/craft/pip-overlay.mdindex 7ffaceb2..d14f5f84 100644--- a/.agents/skills/hypit/references/playbooks/craft/pip-overlay.md+++ b/.agents/skills/hypit/references/playbooks/craft/pip-overlay.md@@ -5,8 +5,8 @@ secondary feed, or supporting image stays visible.## Choose the source form-- Use a normal rectangular `media-track:Item video={...}` for a live reaction, presenter feed, screen- recording, or secondary camera.+- Use a normal rectangular `media-track:Item media={...}` for a live reaction, presenter feed, screen+ recording, or secondary camera, normalizing the footage through `pipeline:Normalize` first.- Use `remove:Background` when a still portrait must become a transparent cutout. Place the resultingimage with `image={cutout.image}` plus an explicit `space:Extent`.- Use `compose:Image` with ordered `compose:Layer` children when the PIP should be frozen into one@@ -46,8 +46,8 @@ secondary feed, or supporting image stays visible.- A reaction PIP may remain visually alive through breathing, eye movement, and expression whilesilent. Do not imply that it speaks unless its real audio is explicitly authored.-- For a video Item, add `audio="include"` only when that source should contribute sound; otherwise keep- it visual-only and put the intended audio on a separate Track.+- Normalize a video Item's source with `audio="default"` only when it should contribute sound;+ otherwise pass `audio="none"` and put the intended audio on a separate Track.- Only the active Script speaker moves their mouth in a dialogue format. A listener PIP reacts withoutinvented words.- Preserve the same identity, wardrobe, crop, scale, and placement when a PIP persists across edits.diff --git a/docs/quickstart/styles.md b/docs/quickstart/styles.mdindex 35fe9cab..b5f06cf4 100644--- a/docs/quickstart/styles.md+++ b/docs/quickstart/styles.md@@ -195,7 +195,7 @@ Position remains an explicit graph edge:```svml<space:Frame id="product-frame" within={vertical}left="8%" top="20%" right="92%" bottom="68%"/>-<media-track:Item video={product-motion.video}+<media-track:Item media={product-media.media}during={story.selection.demo} frame={product-frame}appearance={recipes.media.product} motion={recipes.motion.product}/>```diff --git a/docs/quickstart/tracks.md b/docs/quickstart/tracks.mdindex bedd77a3..ca3847bb 100644--- a/docs/quickstart/tracks.md+++ b/docs/quickstart/tracks.md@@ -178,8 +178,11 @@ Placement is an explicit Spatial Frame edge; appearance and motion remain reusab<space:Frame id="product-frame" within={vertical}left="8%" top="20%" right="92%" bottom="68%"/>+<pipeline:Normalize id="product-media" source={product-motion.video}+ video="primary-moving" audio="none" span-authority="video" frame-rate="30"/>+<media-track:Track id="product-broll" semantic={speech.semantic} canvas={vertical}>- <media-track:Item video={product-motion.video} frame={product-frame}+ <media-track:Item media={product-media.media} frame={product-frame}during={story.selection.product-demo}appearance={recipes.media.product}motion={recipes.motion.product}/>@@ -199,16 +202,14 @@ Every Item, Member or sample Layer declares exactly one visual input form:| Input | Value | Meaning ||---|---|---|| `image={...}` + `extent={...}` | Blob + authored pixel extent | A durationless still image |-| `video={...}` | Generated/raw video Blob | Inspect, select and normalize to this Track's `space` automatically || `media={...}` | `SynchronizedMedia` | Connect an explicitly prepared timed source directly || `surface={...}` | `CompositableSurfaceRef` | Connect an alpha-aware still or timed surface directly |-Raw `video=` is visual-only by default. Add `audio="include"` when its own audio should be-normalized and emitted by the Track; `audio-gain` remains available for that selected source.-These forms are explicit so a generic Blob is never guessed to be an image or video. The convenient-surface syntax does not bypass the graph: `video=` expands to ordinary bind-request, inspect,-select and normalize Operations. An explicit `<pipeline:Normalize>` remains available for shared-or specially selected media, whose output then connects through `media=`.+A generated or imported video Blob reaches a Track through `<pipeline:Normalize>`, which inspects it,+selects its streams and puts them on one frame domain; its `.media` output is what `media=` connects+to. `audio="none"` carries the picture alone, and `audio="default"` carries the source's own sound,+which `audio-gain` then scales. These forms are explicit so a generic Blob is never guessed to be an+image or a video.**Outputs:** `{product-broll.visual}` and, only when explicitly authored, `{product-broll.audio}`.@@ -549,8 +550,10 @@ All four track families together in one source file:left="10%" top="20%" right="90%" bottom="70%"/><!-- Media: one ordinary Item used editorially as B-roll -->+<pipeline:Normalize id="card-media" source={motion.video}+ video="primary-moving" audio="none" span-authority="video" frame-rate="30"/><media-track:Track id="cards" semantic={speech.semantic} canvas={vertical}>- <media-track:Item video={motion.video} frame={card-frame}+ <media-track:Item media={card-media.media} frame={card-frame}during={story.selection.demo} appearance={recipes.media.card} motion={recipes.motion.card}/></media-track:Track>diff --git a/docs/zh/quickstart/styles.md b/docs/zh/quickstart/styles.mdindex ee5f5d50..ff3de8a6 100644--- a/docs/zh/quickstart/styles.md+++ b/docs/zh/quickstart/styles.md@@ -190,7 +190,7 @@ motion.product {```svml<space:Frame id="product-frame" within={vertical}left="8%" top="20%" right="92%" bottom="68%"/>-<media-track:Item video={product-motion.video}+<media-track:Item media={product-media.media}during={story.selection.demo} frame={product-frame}appearance={recipes.media.product} motion={recipes.motion.product}/>```diff --git a/docs/zh/quickstart/tracks.md b/docs/zh/quickstart/tracks.mdindex f9b919e3..c0743165 100644--- a/docs/zh/quickstart/tracks.md+++ b/docs/zh/quickstart/tracks.md@@ -132,8 +132,11 @@ B-roll 是通用 Media Track 的一种剪辑用途,不是独立 Track 家族<space:Frame id="product-frame" within={vertical}left="8%" top="20%" right="92%" bottom="68%"/>+<pipeline:Normalize id="product-media" source={product-motion.video}+ video="primary-moving" audio="none" span-authority="video" frame-rate="30"/>+<media-track:Track id="product-broll" semantic={speech.semantic} canvas={vertical}>- <media-track:Item video={product-motion.video} frame={product-frame}+ <media-track:Item media={product-media.media} frame={product-frame}during={story.selection.product-demo}appearance={recipes.media.product}motion={recipes.motion.product}/>@@ -147,13 +150,10 @@ B-roll 是通用 Media Track 的一种剪辑用途,不是独立 Track 家族| 输入 | 值 | 含义 ||---|---|---|| `image={...}` + `extent={...}` | Blob + 作者声明的像素尺寸 | 没有自带时长的静态图 |-| `video={...}` | 生成/原始视频 Blob | 自动检查、选流并按本 Track 的 `space` 规范化 || `media={...}` | `SynchronizedMedia` | 直接连接显式准备好的含时素材 || `surface={...}` | `CompositableSurfaceRef` | 直接连接带透明度语义的静态或含时 Surface |-原始 `video=` 默认只取画面;需要它自己的声音时添加 `audio="include"`,并可继续用-`audio-gain` 调节所选源音频。输入名必须显式,是为了绝不靠猜测把一个通用 Blob 当成图片或视频。简洁语法没有绕过图:`video=` 会展开为普通的绑定请求、检查、选流、规范化-Operation。需要共享或特殊选流时仍可显式写 `<pipeline:Normalize>`,再把结果用 `media=` 接入。+生成或导入的视频 Blob 经 `<pipeline:Normalize>` 进入 Track:它检查该 Blob、选出其中的流并放到同一个帧域上,输出的 `.media` 即 `media=` 所连接的值。`audio="none"` 只取画面,`audio="default"` 取源自带的声音,再由 `audio-gain` 调节。输入名必须显式,是为了绝不靠猜测把一个通用 Blob 当成图片或视频。**输出:**`{product-broll.visual}`;只有作者显式选择了源音频或 SFX 时才会出现`{product-broll.audio}`。@@ -449,8 +449,10 @@ Track 接受 `id`、`canvas` 与 `semantic`。时长写作 `12f`、`250ms` 或 `left="10%" top="20%" right="90%" bottom="70%"/><!-- Media:Selection 期间显示一个普通 Item -->+<pipeline:Normalize id="card-media" source={motion.video}+ video="primary-moving" audio="none" span-authority="video" frame-rate="30"/><media-track:Track id="cards" semantic={speech.semantic} canvas={vertical}>- <media-track:Item video={motion.video} frame={card-frame}+ <media-track:Item media={card-media.media} frame={card-frame}during={story.selection.demo} appearance={recipes.media.card} motion={recipes.motion.card}/></media-track:Track>diff --git a/packages/media-track/src/manifest.ts b/packages/media-track/src/manifest.tsindex a5c31514..6263ed03 100644--- a/packages/media-track/src/manifest.ts+++ b/packages/media-track/src/manifest.ts@@ -391,16 +391,12 @@ export const mediaTrackMarkupSurfaces = [{summary: "Clips the Item to an authored Path." },{ name: "image", kind: "reference", required: false, accepts: [artifactTypes.blob],summary: "Shows a durationless still image as the Item's direct source." },- { name: "video", kind: "reference", required: false, accepts: [artifactTypes.blob],- summary: "Shows a raw video the Track inspects, selects and normalizes against its `space`." },{ name: "media", kind: "reference", required: false, accepts: [mediaTypes.synchronized],summary: "Shows an explicitly prepared timed source." },{ name: "surface", kind: "reference", required: false, accepts: [mediaTypes.compositableSurface],summary: "Shows an alpha-aware still or timed Surface." },{ name: "extent", kind: "reference", required: false, accepts: [spatialTypes.extent],summary: "Gives the still image its authored pixel Extent. The `fit` scales this Extent into the Frame, so what it decides is the shape: its width-to-height ratio has to be the picture's, and the pixel numbers themselves only have to hold that ratio. A generated picture's own size is the generator's to choose and is not knowable when the Source is written, so write the ratio you asked that generator for." },- { name: "audio", kind: "literal", required: false, values: ["include", "omit"],- summary: "Decides whether a raw `video` source contributes its own audio; defaults to `omit`." },{ name: "source-audio", kind: "literal", required: false,summary: "Names the layer whose source audio this Item emits." },{ name: "audio-gain", kind: "literal", required: false,@@ -437,15 +433,18 @@ export const mediaTrackMarkupSurfaces = [{{ name: "audio", type: compositionTypes.audioTrack,summary: "The rendered sound, published only when a source audio selection or a Sound is authored." },],- example: `<media-track:Track id="cutaways" semantic={speech.semantic} canvas={vertical}>- <media-track:Item id="bags" video={cutaway-bags.video} during={story.selection.bags}+ example: `<pipeline:Normalize id="cutaway-bags" source={bags-take.video}+ video="primary-moving" audio="none" span-authority="video" clock={clock}/>++<media-track:Track id="cutaways" semantic={speech.semantic} canvas={vertical}>+ <media-track:Item id="bags" media={cutaway-bags.media} during={story.selection.bags}frame={full} appearance={recipes.media.cutaway} motion={recipes.motion.cut}/></media-track:Track>`,notes: ["A Track requires at least one Item or Sequence and accepts no text content.","An Item states exactly one window form: `during`, `at` with `for`, `until` with `for`, or `start` with `end`; `selection`, `segment` and `moment` bind a start/end window and cannot be written together.","A point expression is `program.start`, `program.end`, `selection.start`, `selection.end`, `segment.start`, `segment.end` or `moment.cue`, each optionally offset by `+` or `-` and a duration, or a bare duration read as an absolute position.",- "A unit that names a direct source names exactly one of `image`, `video`, `media` or `surface`; `extent` is required with `image` and refused otherwise, and `audio` is only valid with `video`.",+ "A unit that names a direct source names exactly one of `image`, `media` or `surface`; `extent` is required with `image` and refused otherwise.","`audio-gain` is refused without selected source audio.","`clip` is refused on an Item or Sequence whose Recipe already states a clip.","An `appearance` or `motion` Recipe is refused when it carries a property outside its own set.",@@ -463,11 +462,9 @@ export const mediaTrackMarkupSurfaces = [{"| `boundary` | literal (start, end) | with a Selection | Which edge of the activating Selection the Member starts on; refused for a Moment |","| `appearance` | reference (@hypit/svs@1#Recipe) | no | This Member's own Recipe in place of the Sequence's |","| `image` | reference (@hypit/artifact@1#Blob) | one source form | A durationless still image, which also requires `extent` |",- "| `video` | reference (@hypit/artifact@1#Blob) | one source form | A raw video the Track inspects, selects and normalizes against its `space` |","| `media` | reference (@hypit/media@1#SynchronizedMedia) | one source form | An explicitly prepared timed source |","| `surface` | reference (@hypit/media@1#CompositableSurfaceRef) | one source form | An alpha-aware still or timed Surface |","| `extent` | reference (@hypit/spatial@1#IntrinsicExtent) | with `image` | The authored pixel extent of the still image |",- "| `audio` | literal (include, omit) | no | Whether a raw `video` source contributes its own audio; defaults to `omit` |","| `source-audio` | literal | no | Names the layer whose source audio this Member emits |","| `audio-gain` | literal | with selected source audio | Linear gain applied to the selected source audio; defaults to `1` |",].join("\n"),@@ -511,11 +508,9 @@ export const mediaTrackMarkupSurfaces = [{"|---|---|---|---|","| `id` | identifier | no | Names this layer, which `source-audio` selects by; the Track derives one from the unit and the layer position otherwise |","| `image` | reference (@hypit/artifact@1#Blob) | one source form | A durationless still image, which also requires `extent` |",- "| `video` | reference (@hypit/artifact@1#Blob) | one source form | A raw video the Track inspects, selects and normalizes against its `space` |","| `media` | reference (@hypit/media@1#SynchronizedMedia) | one source form | An explicitly prepared timed source |","| `surface` | reference (@hypit/media@1#CompositableSurfaceRef) | one source form | An alpha-aware still or timed Surface |","| `extent` | reference (@hypit/spatial@1#IntrinsicExtent) | with `image` | The authored pixel extent of the still image |",- "| `audio` | literal (include, omit) | no | Whether a raw `video` source contributes its own audio; defaults to `omit` |","| `appearance` | reference (@hypit/svs@1#Recipe) | no | This layer's own Recipe in place of the unit's |",].join("\n"),"A Layer `appearance` Recipe carries the fit properties and the sample properties and nothing else, so a Layer states its own Recipe rather than inheriting the unit's, which requires `stack-order`.",diff --git a/packages/reference-video-tools/src/authoring.ts b/packages/reference-video-tools/src/authoring.tsindex 584a6315..37e15387 100644--- a/packages/reference-video-tools/src/authoring.ts+++ b/packages/reference-video-tools/src/authoring.ts@@ -102,6 +102,22 @@ export type StandInFocus = {readonly tokens?: readonly [number, number];};+/**+ * A token window, read from whatever a caller supplied.+ *+ * The type says two numbers, and nothing but this checked that. A batch entry is JSON a caller wrote+ * by hand, so `"79:86"` arrives as a string, and destructuring one yields its first two characters:+ * `from` is `"7"` and `to` is `"9"`. Every guard downstream then compares strings — `"9" > "7"` holds,+ * and so does `"9" <= 135` once it coerces — so the window passes as valid and words 7 to 9 render+ * under the name of words 79 to 86. The call reports success, and the difference only shows up as a+ * comparison that makes no sense against a picture nobody asked for.+ */+export function tokenWindow(value: unknown, subject: string): readonly [number, number] {+ assert(Array.isArray(value) && value.length === 2 && value.every((item) => Number.isInteger(item)),+ `${subject} must be two whole numbers, as [from, to) — received ${JSON.stringify(value)}`);+ return [value[0] as number, value[1] as number];+}+/*** The Hypit tree this package is installed into.*@@ -499,7 +515,7 @@ export async function spokenRange(let from: number | undefined;let to: number | undefined;if (focus.tokens !== undefined) {- [from, to] = focus.tokens;+ [from, to] = tokenWindow(focus.tokens, "the token window");assert(from >= 0 && to > from && to <= parsed.tokens.length,`token range [${from}, ${to}) is outside the Script's ${parsed.tokens.length} words`);} else if (focus.selection !== undefined) {diff --git a/packages/reference-video-tools/src/checks.ts b/packages/reference-video-tools/src/checks.tsindex c2a1ae7a..dd114d8c 100644--- a/packages/reference-video-tools/src/checks.ts+++ b/packages/reference-video-tools/src/checks.ts@@ -522,6 +522,7 @@ export async function authoringCheck(// instead meant a Run named by a path resolved its packages somewhere else entirely, and the// project's own elements read as packages that do not exist.const packageRoot = nearestPackageRoot(dirname(runPath)) ?? roots.packageRoot;+ const distributionPackageRoot = videoCliDistribution.packageRoot;const runSource = await readFile(runPath, "utf8").catch(() => undefined);assert(runSource !== undefined, `cannot read ${runPath}`);@@ -569,9 +570,14 @@ export async function authoringCheck(// drawing nothing and the check passed on an empty answer. Fall back to loading them one at a time so// the ones that resolve still count, and keep the names of the ones that did not.const unresolved: string[] = [];- const loaded = await loadNodePackageSelection(specifiers, packageRoot).catch(async () => {+ // A project installs its own packages against itself, and `@hypit/*` comes from the Distribution+ // this command was launched from. Loading against the project root alone leaves the second kind+ // with nowhere to resolve from, so a project whose Source imports the standard vocabulary reports+ // every one of those imports as a package that does not exist.+ const loading = { ...(distributionPackageRoot === undefined ? {} : { fallbackRoots: [distributionPackageRoot] }) };+ const loaded = await loadNodePackageSelection(specifiers, packageRoot, loading).catch(async () => {const each = await Promise.all(specifiers.map(async (specifier) =>- await loadNodePackageSelection([specifier], packageRoot).catch(() => {+ await loadNodePackageSelection([specifier], packageRoot, loading).catch(() => {unresolved.push(specifier);return [];})));diff --git a/packages/reference-video-tools/src/tools.ts b/packages/reference-video-tools/src/tools.tsindex d8511a88..a4dc5a42 100644--- a/packages/reference-video-tools/src/tools.ts+++ b/packages/reference-video-tools/src/tools.ts@@ -18,7 +18,7 @@ import {import { describeSchema } from "./contract.js";import { videoCliDistribution } from "@hypit/video-cli";-import { authorSource, invokedFrom, referenceRoot, referenceWords, renderElement, renderPreviews, spokenRange, standInSidecarPath } from "./authoring.js";+import { authorSource, invokedFrom, referenceRoot, referenceWords, renderElement, renderPreviews, spokenRange, standInSidecarPath, tokenWindow } from "./authoring.js";import type { RenderElementInput, RenderPreviewsInput, SpokenRange, StandInFocus, StandInSidecar } from "./authoring.js";import { writePlaceholder } from "./placeholder.js";import { downloadReferenceVideo, isReferenceUrl } from "@hypit/yt-dlp";@@ -1528,6 +1528,7 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceVconst root = stateRoot(state.reference_id);const named = [input.shot_id, input.segment, input.selection, input.tokens].filter((value) => value !== undefined);assert(named.length === 1, "name exactly one of shot_id, segment, selection and tokens");+ if (input.tokens !== undefined) tokenWindow(input.tokens, "tokens");const supplied = [input.image_path, input.video_path].filter((value) => value !== undefined);assert(supplied.length === 1, "supply exactly one of image_path and video_path");const clip = input.video_path !== undefined;@@ -1806,6 +1807,7 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceVassert(element.length > 0, "--element is required; a review with no element is credited to nothing");const named = [input.segment, input.selection, input.tokens].filter((value) => value !== undefined);assert(named.length === 1, "name exactly one of --segment, --selection or --tokens: the window the render covers");+ if (input.tokens !== undefined) tokenWindow(input.tokens, "tokens");const supplied = [input.image_path, input.video_path].filter((value) => value !== undefined);assert(supplied.length === 1, "supply exactly one of --image or --video");