Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

Three defects a reconstruction round ran into (#91)

rponeawa3115fb0ef86e
diff --git a/.agents/skills/hypit/references/playbooks/craft/persona-and-audio.md b/.agents/skills/hypit/references/playbooks/craft/persona-and-audio.md
index 8d43130b..6a1de536 100644
--- a/.agents/skills/hypit/references/playbooks/craft/persona-and-audio.md
+++ b/.agents/skills/hypit/references/playbooks/craft/persona-and-audio.md
@@ -68,7 +68,7 @@ as `SynchronizedMedia`, then place it with `audio:Track`:
- Keep primary speech, music, ambience, and effects as independently inspectable contributions. Add
Speech Track's audio projection and every selected Audio Track to `film:Film`.
-- Raw Media Track video is visual-only unless `audio="include"` is explicitly authored.
+- A Media Track carries picture alone unless the source was normalized with `audio="default"`.
- Dialogue wins the mix. Use explicit gain and fades; do not assume automatic ducking, loudness
normalization, or mastering.
diff --git a/.agents/skills/hypit/references/playbooks/craft/pip-overlay.md b/.agents/skills/hypit/references/playbooks/craft/pip-overlay.md
index 7ffaceb2..d14f5f84 100644
--- a/.agents/skills/hypit/references/playbooks/craft/pip-overlay.md
+++ b/.agents/skills/hypit/references/playbooks/craft/pip-overlay.md
@@ -5,8 +5,8 @@ secondary feed, or supporting image stays visible.
## Choose the source form
-- Use a normal rectangular `media-track:Item video={...}` for a live reaction, presenter feed, screen
- recording, or secondary camera.
+- Use a normal rectangular `media-track:Item media={...}` for a live reaction, presenter feed, screen
+ recording, or secondary camera, normalizing the footage through `pipeline:Normalize` first.
- Use `remove:Background` when a still portrait must become a transparent cutout. Place the resulting
image with `image={cutout.image}` plus an explicit `space:Extent`.
- Use `compose:Image` with ordered `compose:Layer` children when the PIP should be frozen into one
@@ -46,8 +46,8 @@ secondary feed, or supporting image stays visible.
- A reaction PIP may remain visually alive through breathing, eye movement, and expression while
silent. Do not imply that it speaks unless its real audio is explicitly authored.
-- For a video Item, add `audio="include"` only when that source should contribute sound; otherwise keep
- it visual-only and put the intended audio on a separate Track.
+- Normalize a video Item's source with `audio="default"` only when it should contribute sound;
+ otherwise pass `audio="none"` and put the intended audio on a separate Track.
- Only the active Script speaker moves their mouth in a dialogue format. A listener PIP reacts without
invented words.
- Preserve the same identity, wardrobe, crop, scale, and placement when a PIP persists across edits.
diff --git a/docs/quickstart/styles.md b/docs/quickstart/styles.md
index 35fe9cab..b5f06cf4 100644
--- a/docs/quickstart/styles.md
+++ b/docs/quickstart/styles.md
@@ -195,7 +195,7 @@ Position remains an explicit graph edge:
```svml
<space:Frame id="product-frame" within={vertical}
left="8%" top="20%" right="92%" bottom="68%"/>
-<media-track:Item video={product-motion.video}
+<media-track:Item media={product-media.media}
during={story.selection.demo} frame={product-frame}
appearance={recipes.media.product} motion={recipes.motion.product}/>
```
diff --git a/docs/quickstart/tracks.md b/docs/quickstart/tracks.md
index bedd77a3..ca3847bb 100644
--- a/docs/quickstart/tracks.md
+++ b/docs/quickstart/tracks.md
@@ -178,8 +178,11 @@ Placement is an explicit Spatial Frame edge; appearance and motion remain reusab
<space:Frame id="product-frame" within={vertical}
left="8%" top="20%" right="92%" bottom="68%"/>
+<pipeline:Normalize id="product-media" source={product-motion.video}
+ video="primary-moving" audio="none" span-authority="video" frame-rate="30"/>
+
<media-track:Track id="product-broll" semantic={speech.semantic} canvas={vertical}>
- <media-track:Item video={product-motion.video} frame={product-frame}
+ <media-track:Item media={product-media.media} frame={product-frame}
during={story.selection.product-demo}
appearance={recipes.media.product}
motion={recipes.motion.product}/>
@@ -199,16 +202,14 @@ Every Item, Member or sample Layer declares exactly one visual input form:
| Input | Value | Meaning |
|---|---|---|
| `image={...}` + `extent={...}` | Blob + authored pixel extent | A durationless still image |
-| `video={...}` | Generated/raw video Blob | Inspect, select and normalize to this Track's `space` automatically |
| `media={...}` | `SynchronizedMedia` | Connect an explicitly prepared timed source directly |
| `surface={...}` | `CompositableSurfaceRef` | Connect an alpha-aware still or timed surface directly |
-Raw `video=` is visual-only by default. Add `audio="include"` when its own audio should be
-normalized and emitted by the Track; `audio-gain` remains available for that selected source.
-These forms are explicit so a generic Blob is never guessed to be an image or video. The convenient
-surface syntax does not bypass the graph: `video=` expands to ordinary bind-request, inspect,
-select and normalize Operations. An explicit `<pipeline:Normalize>` remains available for shared
-or specially selected media, whose output then connects through `media=`.
+A generated or imported video Blob reaches a Track through `<pipeline:Normalize>`, which inspects it,
+selects its streams and puts them on one frame domain; its `.media` output is what `media=` connects
+to. `audio="none"` carries the picture alone, and `audio="default"` carries the source's own sound,
+which `audio-gain` then scales. These forms are explicit so a generic Blob is never guessed to be an
+image or a video.
**Outputs:** `{product-broll.visual}` and, only when explicitly authored, `{product-broll.audio}`.
@@ -549,8 +550,10 @@ All four track families together in one source file:
left="10%" top="20%" right="90%" bottom="70%"/>
<!-- Media: one ordinary Item used editorially as B-roll -->
+<pipeline:Normalize id="card-media" source={motion.video}
+ video="primary-moving" audio="none" span-authority="video" frame-rate="30"/>
<media-track:Track id="cards" semantic={speech.semantic} canvas={vertical}>
- <media-track:Item video={motion.video} frame={card-frame}
+ <media-track:Item media={card-media.media} frame={card-frame}
during={story.selection.demo} appearance={recipes.media.card} motion={recipes.motion.card}/>
</media-track:Track>
diff --git a/docs/zh/quickstart/styles.md b/docs/zh/quickstart/styles.md
index ee5f5d50..ff3de8a6 100644
--- a/docs/zh/quickstart/styles.md
+++ b/docs/zh/quickstart/styles.md
@@ -190,7 +190,7 @@ motion.product {
```svml
<space:Frame id="product-frame" within={vertical}
left="8%" top="20%" right="92%" bottom="68%"/>
-<media-track:Item video={product-motion.video}
+<media-track:Item media={product-media.media}
during={story.selection.demo} frame={product-frame}
appearance={recipes.media.product} motion={recipes.motion.product}/>
```
diff --git a/docs/zh/quickstart/tracks.md b/docs/zh/quickstart/tracks.md
index f9b919e3..c0743165 100644
--- a/docs/zh/quickstart/tracks.md
+++ b/docs/zh/quickstart/tracks.md
@@ -132,8 +132,11 @@ B-roll 是通用 Media Track 的一种剪辑用途,不是独立 Track 家族
<space:Frame id="product-frame" within={vertical}
left="8%" top="20%" right="92%" bottom="68%"/>
+<pipeline:Normalize id="product-media" source={product-motion.video}
+ video="primary-moving" audio="none" span-authority="video" frame-rate="30"/>
+
<media-track:Track id="product-broll" semantic={speech.semantic} canvas={vertical}>
- <media-track:Item video={product-motion.video} frame={product-frame}
+ <media-track:Item media={product-media.media} frame={product-frame}
during={story.selection.product-demo}
appearance={recipes.media.product}
motion={recipes.motion.product}/>
@@ -147,13 +150,10 @@ B-roll 是通用 Media Track 的一种剪辑用途,不是独立 Track 家族
| 输入 | 值 | 含义 |
|---|---|---|
| `image={...}` + `extent={...}` | Blob + 作者声明的像素尺寸 | 没有自带时长的静态图 |
-| `video={...}` | 生成/原始视频 Blob | 自动检查、选流并按本 Track 的 `space` 规范化 |
| `media={...}` | `SynchronizedMedia` | 直接连接显式准备好的含时素材 |
| `surface={...}` | `CompositableSurfaceRef` | 直接连接带透明度语义的静态或含时 Surface |
-原始 `video=` 默认只取画面;需要它自己的声音时添加 `audio="include"`,并可继续用
-`audio-gain` 调节所选源音频。输入名必须显式,是为了绝不靠猜测把一个通用 Blob 当成图片或视频。简洁语法没有绕过图:`video=` 会展开为普通的绑定请求、检查、选流、规范化
-Operation。需要共享或特殊选流时仍可显式写 `<pipeline:Normalize>`,再把结果用 `media=` 接入。
+生成或导入的视频 Blob 经 `<pipeline:Normalize>` 进入 Track:它检查该 Blob、选出其中的流并放到同一个帧域上,输出的 `.media` 即 `media=` 所连接的值。`audio="none"` 只取画面,`audio="default"` 取源自带的声音,再由 `audio-gain` 调节。输入名必须显式,是为了绝不靠猜测把一个通用 Blob 当成图片或视频。
**输出:**`{product-broll.visual}`;只有作者显式选择了源音频或 SFX 时才会出现
`{product-broll.audio}`。
@@ -449,8 +449,10 @@ Track 接受 `id`、`canvas` 与 `semantic`。时长写作 `12f`、`250ms` 或 `
left="10%" top="20%" right="90%" bottom="70%"/>
<!-- Media:Selection 期间显示一个普通 Item -->
+<pipeline:Normalize id="card-media" source={motion.video}
+ video="primary-moving" audio="none" span-authority="video" frame-rate="30"/>
<media-track:Track id="cards" semantic={speech.semantic} canvas={vertical}>
- <media-track:Item video={motion.video} frame={card-frame}
+ <media-track:Item media={card-media.media} frame={card-frame}
during={story.selection.demo} appearance={recipes.media.card} motion={recipes.motion.card}/>
</media-track:Track>
diff --git a/packages/media-track/src/manifest.ts b/packages/media-track/src/manifest.ts
index a5c31514..6263ed03 100644
--- a/packages/media-track/src/manifest.ts
+++ b/packages/media-track/src/manifest.ts
@@ -391,16 +391,12 @@ export const mediaTrackMarkupSurfaces = [{
summary: "Clips the Item to an authored Path." },
{ name: "image", kind: "reference", required: false, accepts: [artifactTypes.blob],
summary: "Shows a durationless still image as the Item's direct source." },
- { name: "video", kind: "reference", required: false, accepts: [artifactTypes.blob],
- summary: "Shows a raw video the Track inspects, selects and normalizes against its `space`." },
{ name: "media", kind: "reference", required: false, accepts: [mediaTypes.synchronized],
summary: "Shows an explicitly prepared timed source." },
{ name: "surface", kind: "reference", required: false, accepts: [mediaTypes.compositableSurface],
summary: "Shows an alpha-aware still or timed Surface." },
{ name: "extent", kind: "reference", required: false, accepts: [spatialTypes.extent],
summary: "Gives the still image its authored pixel Extent. The `fit` scales this Extent into the Frame, so what it decides is the shape: its width-to-height ratio has to be the picture's, and the pixel numbers themselves only have to hold that ratio. A generated picture's own size is the generator's to choose and is not knowable when the Source is written, so write the ratio you asked that generator for." },
- { name: "audio", kind: "literal", required: false, values: ["include", "omit"],
- summary: "Decides whether a raw `video` source contributes its own audio; defaults to `omit`." },
{ name: "source-audio", kind: "literal", required: false,
summary: "Names the layer whose source audio this Item emits." },
{ name: "audio-gain", kind: "literal", required: false,
@@ -437,15 +433,18 @@ export const mediaTrackMarkupSurfaces = [{
{ name: "audio", type: compositionTypes.audioTrack,
summary: "The rendered sound, published only when a source audio selection or a Sound is authored." },
],
- example: `<media-track:Track id="cutaways" semantic={speech.semantic} canvas={vertical}>
- <media-track:Item id="bags" video={cutaway-bags.video} during={story.selection.bags}
+ example: `<pipeline:Normalize id="cutaway-bags" source={bags-take.video}
+ video="primary-moving" audio="none" span-authority="video" clock={clock}/>
+
+<media-track:Track id="cutaways" semantic={speech.semantic} canvas={vertical}>
+ <media-track:Item id="bags" media={cutaway-bags.media} during={story.selection.bags}
frame={full} appearance={recipes.media.cutaway} motion={recipes.motion.cut}/>
</media-track:Track>`,
notes: [
"A Track requires at least one Item or Sequence and accepts no text content.",
"An Item states exactly one window form: `during`, `at` with `for`, `until` with `for`, or `start` with `end`; `selection`, `segment` and `moment` bind a start/end window and cannot be written together.",
"A point expression is `program.start`, `program.end`, `selection.start`, `selection.end`, `segment.start`, `segment.end` or `moment.cue`, each optionally offset by `+` or `-` and a duration, or a bare duration read as an absolute position.",
- "A unit that names a direct source names exactly one of `image`, `video`, `media` or `surface`; `extent` is required with `image` and refused otherwise, and `audio` is only valid with `video`.",
+ "A unit that names a direct source names exactly one of `image`, `media` or `surface`; `extent` is required with `image` and refused otherwise.",
"`audio-gain` is refused without selected source audio.",
"`clip` is refused on an Item or Sequence whose Recipe already states a clip.",
"An `appearance` or `motion` Recipe is refused when it carries a property outside its own set.",
@@ -463,11 +462,9 @@ export const mediaTrackMarkupSurfaces = [{
"| `boundary` | literal (start, end) | with a Selection | Which edge of the activating Selection the Member starts on; refused for a Moment |",
"| `appearance` | reference (@hypit/svs@1#Recipe) | no | This Member's own Recipe in place of the Sequence's |",
"| `image` | reference (@hypit/artifact@1#Blob) | one source form | A durationless still image, which also requires `extent` |",
- "| `video` | reference (@hypit/artifact@1#Blob) | one source form | A raw video the Track inspects, selects and normalizes against its `space` |",
"| `media` | reference (@hypit/media@1#SynchronizedMedia) | one source form | An explicitly prepared timed source |",
"| `surface` | reference (@hypit/media@1#CompositableSurfaceRef) | one source form | An alpha-aware still or timed Surface |",
"| `extent` | reference (@hypit/spatial@1#IntrinsicExtent) | with `image` | The authored pixel extent of the still image |",
- "| `audio` | literal (include, omit) | no | Whether a raw `video` source contributes its own audio; defaults to `omit` |",
"| `source-audio` | literal | no | Names the layer whose source audio this Member emits |",
"| `audio-gain` | literal | with selected source audio | Linear gain applied to the selected source audio; defaults to `1` |",
].join("\n"),
@@ -511,11 +508,9 @@ export const mediaTrackMarkupSurfaces = [{
"|---|---|---|---|",
"| `id` | identifier | no | Names this layer, which `source-audio` selects by; the Track derives one from the unit and the layer position otherwise |",
"| `image` | reference (@hypit/artifact@1#Blob) | one source form | A durationless still image, which also requires `extent` |",
- "| `video` | reference (@hypit/artifact@1#Blob) | one source form | A raw video the Track inspects, selects and normalizes against its `space` |",
"| `media` | reference (@hypit/media@1#SynchronizedMedia) | one source form | An explicitly prepared timed source |",
"| `surface` | reference (@hypit/media@1#CompositableSurfaceRef) | one source form | An alpha-aware still or timed Surface |",
"| `extent` | reference (@hypit/spatial@1#IntrinsicExtent) | with `image` | The authored pixel extent of the still image |",
- "| `audio` | literal (include, omit) | no | Whether a raw `video` source contributes its own audio; defaults to `omit` |",
"| `appearance` | reference (@hypit/svs@1#Recipe) | no | This layer's own Recipe in place of the unit's |",
].join("\n"),
"A Layer `appearance` Recipe carries the fit properties and the sample properties and nothing else, so a Layer states its own Recipe rather than inheriting the unit's, which requires `stack-order`.",
diff --git a/packages/reference-video-tools/src/authoring.ts b/packages/reference-video-tools/src/authoring.ts
index 584a6315..37e15387 100644
--- a/packages/reference-video-tools/src/authoring.ts
+++ b/packages/reference-video-tools/src/authoring.ts
@@ -102,6 +102,22 @@ export type StandInFocus = {
readonly tokens?: readonly [number, number];
};
+/**
+ * A token window, read from whatever a caller supplied.
+ *
+ * The type says two numbers, and nothing but this checked that. A batch entry is JSON a caller wrote
+ * by hand, so `"79:86"` arrives as a string, and destructuring one yields its first two characters:
+ * `from` is `"7"` and `to` is `"9"`. Every guard downstream then compares strings — `"9" > "7"` holds,
+ * and so does `"9" <= 135` once it coerces — so the window passes as valid and words 7 to 9 render
+ * under the name of words 79 to 86. The call reports success, and the difference only shows up as a
+ * comparison that makes no sense against a picture nobody asked for.
+ */
+export function tokenWindow(value: unknown, subject: string): readonly [number, number] {
+ assert(Array.isArray(value) && value.length === 2 && value.every((item) => Number.isInteger(item)),
+ `${subject} must be two whole numbers, as [from, to) — received ${JSON.stringify(value)}`);
+ return [value[0] as number, value[1] as number];
+}
+
/**
* The Hypit tree this package is installed into.
*
@@ -499,7 +515,7 @@ export async function spokenRange(
let from: number | undefined;
let to: number | undefined;
if (focus.tokens !== undefined) {
- [from, to] = focus.tokens;
+ [from, to] = tokenWindow(focus.tokens, "the token window");
assert(from >= 0 && to > from && to <= parsed.tokens.length,
`token range [${from}, ${to}) is outside the Script's ${parsed.tokens.length} words`);
} else if (focus.selection !== undefined) {
diff --git a/packages/reference-video-tools/src/checks.ts b/packages/reference-video-tools/src/checks.ts
index c2a1ae7a..dd114d8c 100644
--- a/packages/reference-video-tools/src/checks.ts
+++ b/packages/reference-video-tools/src/checks.ts
@@ -522,6 +522,7 @@ export async function authoringCheck(
// instead meant a Run named by a path resolved its packages somewhere else entirely, and the
// project's own elements read as packages that do not exist.
const packageRoot = nearestPackageRoot(dirname(runPath)) ?? roots.packageRoot;
+ const distributionPackageRoot = videoCliDistribution.packageRoot;
const runSource = await readFile(runPath, "utf8").catch(() => undefined);
assert(runSource !== undefined, `cannot read ${runPath}`);
@@ -569,9 +570,14 @@ export async function authoringCheck(
// drawing nothing and the check passed on an empty answer. Fall back to loading them one at a time so
// the ones that resolve still count, and keep the names of the ones that did not.
const unresolved: string[] = [];
- const loaded = await loadNodePackageSelection(specifiers, packageRoot).catch(async () => {
+ // A project installs its own packages against itself, and `@hypit/*` comes from the Distribution
+ // this command was launched from. Loading against the project root alone leaves the second kind
+ // with nowhere to resolve from, so a project whose Source imports the standard vocabulary reports
+ // every one of those imports as a package that does not exist.
+ const loading = { ...(distributionPackageRoot === undefined ? {} : { fallbackRoots: [distributionPackageRoot] }) };
+ const loaded = await loadNodePackageSelection(specifiers, packageRoot, loading).catch(async () => {
const each = await Promise.all(specifiers.map(async (specifier) =>
- await loadNodePackageSelection([specifier], packageRoot).catch(() => {
+ await loadNodePackageSelection([specifier], packageRoot, loading).catch(() => {
unresolved.push(specifier);
return [];
})));
diff --git a/packages/reference-video-tools/src/tools.ts b/packages/reference-video-tools/src/tools.ts
index d8511a88..a4dc5a42 100644
--- a/packages/reference-video-tools/src/tools.ts
+++ b/packages/reference-video-tools/src/tools.ts
@@ -18,7 +18,7 @@ import {
import { describeSchema } from "./contract.js";
import { videoCliDistribution } from "@hypit/video-cli";
-import { authorSource, invokedFrom, referenceRoot, referenceWords, renderElement, renderPreviews, spokenRange, standInSidecarPath } from "./authoring.js";
+import { authorSource, invokedFrom, referenceRoot, referenceWords, renderElement, renderPreviews, spokenRange, standInSidecarPath, tokenWindow } from "./authoring.js";
import type { RenderElementInput, RenderPreviewsInput, SpokenRange, StandInFocus, StandInSidecar } from "./authoring.js";
import { writePlaceholder } from "./placeholder.js";
import { downloadReferenceVideo, isReferenceUrl } from "@hypit/yt-dlp";
@@ -1528,6 +1528,7 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceV
const root = stateRoot(state.reference_id);
const named = [input.shot_id, input.segment, input.selection, input.tokens].filter((value) => value !== undefined);
assert(named.length === 1, "name exactly one of shot_id, segment, selection and tokens");
+ if (input.tokens !== undefined) tokenWindow(input.tokens, "tokens");
const supplied = [input.image_path, input.video_path].filter((value) => value !== undefined);
assert(supplied.length === 1, "supply exactly one of image_path and video_path");
const clip = input.video_path !== undefined;
@@ -1806,6 +1807,7 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceV
assert(element.length > 0, "--element is required; a review with no element is credited to nothing");
const named = [input.segment, input.selection, input.tokens].filter((value) => value !== undefined);
assert(named.length === 1, "name exactly one of --segment, --selection or --tokens: the window the render covers");
+ if (input.tokens !== undefined) tokenWindow(input.tokens, "tokens");
const supplied = [input.image_path, input.video_path].filter((value) => value !== undefined);
assert(supplied.length === 1, "supply exactly one of --image or --video");