Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.
TypeScript agentic-aiaiai-agentsai-videocompilerdslffmpeggenerative-aillmmarkup-languagemonorepoplugin-systemprogramming-languagetext-to-videotypescriptvideovideo-automationvideo-clonevideo-editingvideo-generation
feat: document the playground, and pick a stylesheet with a file dialog
rpone51778fee06f1
diff --git a/docs/.vitepress/config.ts b/docs/.vitepress/config.tsindex fd5aea4a..b43477ab 100644--- a/docs/.vitepress/config.ts+++ b/docs/.vitepress/config.ts@@ -155,6 +155,7 @@ const enTheme = {{ text: "Adding an Author Package", link: "/guide/author-packages" },{ text: "Adding a Provider", link: "/guide/providers" },{ text: "Runtime Profile", link: "/guide/runtime-profile" },+ { text: "Component Playground", link: "/guide/playground" },{ text: "Local Services", link: "/guide/services" },{ text: "Testing", link: "/guide/testing" },{ text: "Conventions", link: "/guide/conventions" },@@ -197,6 +198,7 @@ const zhTheme = {{ text: "添加 Author 包", link: "/zh/guide/author-packages" },{ text: "添加 Provider", link: "/zh/guide/providers" },{ text: "Runtime Profile", link: "/zh/guide/runtime-profile" },+ { text: "组件 Playground", link: "/zh/guide/playground" },{ text: "本地服务", link: "/zh/guide/services" },{ text: "测试", link: "/zh/guide/testing" },{ text: "代码规范", link: "/zh/guide/conventions" },diff --git a/docs/guide/develop.md b/docs/guide/develop.mdindex 0b61ed25..6f9a7596 100644--- a/docs/guide/develop.md+++ b/docs/guide/develop.md@@ -56,6 +56,7 @@ narratage/| [Adding an author package](./author-packages.md) | Step-by-step: new component, Surface, activation || [Adding a Provider](./providers.md) | Step-by-step: new Endpoint adapter || [Runtime Profile](./runtime-profile.md) | JSON and TypeScript configuration, diagnostics, Build archive |+| [Component playground](./playground.md) | Previewing one visual component without a Build || [Local services](./services.md) | WhisperX and OpenCV setup || [Testing](./testing.md) | Test runner, patterns, examples, boundary tests || [Conventions](./conventions.md) | Naming, module boundaries, wire data, TypeScript config |diff --git a/docs/guide/playground.md b/docs/guide/playground.mdnew file mode 100644index 00000000..7d6d8524--- /dev/null+++ b/docs/guide/playground.md@@ -0,0 +1,106 @@+---+title: Component Playground+description: Previewing a visual component without running a Build.+---++# Component Playground++Seeing what a caption or an overlay looks like normally costs a whole Build:+Author Source, Run Source, a Runtime Profile, generation, alignment, render. The+playground renders one component on its own, in a browser, in milliseconds.++```bash+pnpm playground+```++Then open `http://localhost:5178`. Nothing else is needed — no Runtime Profile,+no credentials, no local services.++## What you see++| Control | What it does |+|---|---|+| Component list | Every module that can put pixels on a frame |+| W / H / fps / sec | The canvas and the frame domain |+| Canvas from a stylesheet | Adopt a `.svs` file's frame size |+| Input forms | One per input the component takes |+| Scrubber | Frame-exact; the counter shows `frame / last` |++The preview is the real thing. Each component's own renderer runs —+`renderCaptionTrack`, `renderTextTrack`, `compileBrollProduct`,+`projectSpeechVisual` — and `compileHyperframesDocument` produces the same+document a Build would render.++## It lists nothing++The playground has no list of components. It reads the manifests and offers+every module that declares a Producer whose output type is `VisualTrack`. Add a+module that emits one and it appears; stop emitting one and it disappears.++Everything else is read the same way:++| Question | Answered by |+|---|---|+| What a component needs | that Producer's `inputs` |+| What shape each input has | the declaring module's `TypeDeclaration.schema` |+| What to start from | that type's `TypeDeclaration.default` |+| What a control looks like | `ValueSchema.format` |++So a caption's `mode` is a dropdown because its schema declares an `enum`, and+its `display` is a text area because the field is `format: "multiline"`. Neither+is configured here.++## Formats++`format` is how a module says what a value means when its type alone is+ambiguous — a colour and a font family are both strings.++| Format | Control |+|---|---|+| `color` | Swatch beside a text box. The text is authoritative, because `#RRGGBBAA` carries alpha a swatch cannot. |+| `unit-fraction` | Slider plus a number box |+| `multiline` | Text area |+| `digest` | File picker |+| `duration` | Number box, in seconds |++`color` and `digest` have an exact shape, so `validateStoredValue` enforces+them. The others describe without narrowing.++## Media++A `format: "digest"` field takes a file. The playground hashes it with SHA-256+and registers a real content-addressed identity, because a compiled document+cross-checks every Artifact reference and would reject a stub.++Modules that draw supplied footage — B-roll, speaker footage — therefore have no+default: a module has no bytes to point at, so it says nothing rather than+inventing a digest. Until you choose a file those components draw nothing, which+is a state to pass through and not an error.++## Stylesheets++Choosing a `.svs` file reads **only its Film Recipe**, and only for the frame+size. Point it at `examples/talking-film-golden/studio.svs` and the canvas+becomes 1080×1920 at 30 with that sheet's background.++Nothing else in the sheet is read. What a component looks like comes from the+module that owns it, not from a stylesheet the playground happened to open.++## When something is wrong++Nothing in the playground validates. The modules' own `seal` and `assert`+functions are the judge, because they are what a real Build runs. A rejected+value shows the compiler's own message under the canvas and leaves the last good+frame on screen, so you can read the error and the picture together.++## Known divergences from a real render++- Fonts lower to a plain CSS `font-family` string. The production+ `FontArtifactRef` path with hashed `@font-face` families is not exercised.+- Audio tracks are not played.+- A `ProgramSpace` input is filled from the canvas controls rather than offered+ in a form, so there is only one place to change the frame domain.++See [`tools/playground/README.md`](https://github.com/cashdiffusion/svml/blob/main/tools/playground/README.md)+for how the timeline shim works and what a module must declare to be+previewable.diff --git a/docs/zh/guide/develop.md b/docs/zh/guide/develop.mdindex b69e52c5..6500c465 100644--- a/docs/zh/guide/develop.md+++ b/docs/zh/guide/develop.md@@ -56,6 +56,7 @@ narratage/| [添加 Author 包](./author-packages.md) | 分步说明:新增组件、Surface、activation || [添加 Provider](./providers.md) | 分步说明:新增 Endpoint 适配器 || [Runtime Profile](./runtime-profile.md) | JSON 与 TypeScript 配置、诊断、Build 归档 |+| [组件 Playground](./playground.md) | 不跑 Build 预览单个视觉组件 || [本地服务](./services.md) | WhisperX 与 OpenCV 的搭建 || [测试](./testing.md) | 测试运行器、写法、示例、boundary tests || [代码规范](./conventions.md) | 命名、模块边界、wire 数据、TypeScript 配置 |diff --git a/docs/zh/guide/playground.md b/docs/zh/guide/playground.mdnew file mode 100644index 00000000..48f040ef--- /dev/null+++ b/docs/zh/guide/playground.md@@ -0,0 +1,90 @@+---+title: 组件 Playground+description: 不跑 Build 也能预览视觉组件。+---++# 组件 Playground++想看一条字幕或一个文字层长什么样,通常要付出一整次 Build 的代价:Author Source、Run Source、+Runtime Profile、生成、对齐、渲染。Playground 在浏览器里单独渲染一个组件,耗时以毫秒计。++```bash+pnpm playground+```++然后打开 `http://localhost:5178`。不需要别的——没有 Runtime Profile,没有凭据,也不需要本地服务。++## 界面++| 控件 | 作用 |+|---|---|+| 组件下拉 | 所有能在画面上产生像素的模块 |+| W / H / fps / sec | 画布与帧域 |+| Canvas from a stylesheet | 采用某个 `.svs` 的画幅 |+| 输入表单 | 组件每个输入各一份 |+| 刻度条 | 帧精确,计数显示为 `当前帧 / 末帧` |++预览的是真实结果。跑的是各组件自己的渲染器——`renderCaptionTrack`、`renderTextTrack`、+`compileBrollProduct`、`projectSpeechVisual`——再由 `compileHyperframesDocument` 产出与+Build 相同的文档。++## 它不维护任何清单++Playground 里没有组件列表。它读取各 manifest,把所有声明了「输出 `VisualTrack` 的 Producer」的模块+列出来。新增这样的模块,它自己会出现;不再输出,它自己会消失。++其余信息同样是读出来的:++| 问题 | 来源 |+|---|---|+| 组件需要什么 | 该 Producer 的 `inputs` |+| 每个输入什么形状 | 声明方的 `TypeDeclaration.schema` |+| 从什么值开始 | 该类型的 `TypeDeclaration.default` |+| 控件长什么样 | `ValueSchema.format` |++所以字幕的 `mode` 是下拉框,因为它的 schema 声明了 `enum`;`display` 是多行文本框,因为该字段是+`format: "multiline"`。这两者都不在 Playground 里配置。++## format++当类型本身不足以表达含义时,模块用 `format` 说明——颜色和字体名同样都是字符串。++| format | 控件 |+|---|---|+| `color` | 色板配文本框。以文本框为准,因为 `#RRGGBBAA` 带的 alpha 色板表达不了 |+| `unit-fraction` | 滑块加数字框 |+| `multiline` | 多行文本框 |+| `digest` | 文件选择 |+| `duration` | 数字框,单位秒 |++`color` 和 `digest` 有确切形状,由 `validateStoredValue` 校验。其余只描述,不收窄。++## 媒体++`format: "digest"` 的字段接受一个文件。Playground 用 SHA-256 计算并注册真实的内容寻址标识——+编译产出的文档会交叉校验每一个 Artifact 引用,占位值会被拒绝。++因此绘制外部素材的模块(B-roll、说话人画面)**没有默认值**:模块手里没有那些字节,与其编造一个+digest,不如什么都不说。在你选择文件之前这些组件不画任何东西,这是一个可以经过的状态,不是错误。++## 样式表++选择 `.svs` 文件**只读其中的 Film Recipe**,且只用于画幅。指向+`examples/talking-film-golden/studio.svs`,画布就会变成 1080×1920@30 并采用该表的背景色。++表里其余内容一概不读。组件长什么样由拥有它的模块决定,而不是由 Playground 恰好打开的某张样式表决定。++## 出错时++Playground 自身不做任何校验。裁判是各模块自己的 `seal` 与 `assert` 函数——它们正是真实 Build 会跑的+东西。被拒绝的值会把编译器原本的报错显示在画布下方,并保留上一帧正确画面,让你能同时看到错误和图像。++## 与真实渲染的已知偏差++- 字体降级为普通 CSS `font-family` 字符串,不会走生产环境中带哈希 `@font-face` 的+ `FontArtifactRef` 路径。+- 不播放音频轨。+- `ProgramSpace` 输入由画布控件填充而非出现在表单里,使帧域只有一处可改。++时间线 shim 的原理、以及一个模块需要声明什么才能被预览,见+[`tools/playground/README.md`](https://github.com/cashdiffusion/svml/blob/main/tools/playground/README.md)。diff --git a/tools/playground/README.md b/tools/playground/README.mdindex ca83b58a..135dce8e 100644--- a/tools/playground/README.md+++ b/tools/playground/README.md@@ -7,7 +7,10 @@ pnpm playground```Pick a producer, fill in its inputs, and it renders on a canvas of the size you-choose. Browse to a `.svs` stylesheet and its Film Recipe fills in the frame.+choose. Choose a `.svs` stylesheet and its Film Recipe fills in the frame.++[The guide page](../../docs/guide/playground.md) covers using it. This file+covers how it works.## It lists nothingdiff --git a/tools/playground/src/server/browse.ts b/tools/playground/src/server/browse.tsdeleted file mode 100644index 77bd2252..00000000--- a/tools/playground/src/server/browse.ts+++ /dev/null@@ -1,129 +0,0 @@-import { readdir, readFile, stat } from "node:fs/promises";-import type { ServerResponse } from "node:http";-import { extname, relative, resolve, sep } from "node:path";-import type { Plugin } from "vite";--/**- * A read-only window onto the repository for the browser.- *- * A browser cannot list a directory, and the File System Access API cannot- * follow `src: ./assets/Inter-SemiBold.woff2` out of a stylesheet without a- * second grant for its parent. Serving the listing from the dev server avoids- * both, and keeps every path decision on this side where it can be checked.- */--const SHEET = new Set([".svs"]);-const MEDIA = new Set([- ".png", ".jpg", ".jpeg", ".webp", ".gif", ".avif",- ".mp4", ".webm", ".mov", ".m4a", ".mp3", ".wav",- ".woff2", ".woff", ".otf", ".ttf",-]);--const MEDIA_TYPES: Readonly<Record<string, string>> = {- ".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg",- ".webp": "image/webp", ".gif": "image/gif", ".avif": "image/avif",- ".mp4": "video/mp4", ".webm": "video/webm", ".mov": "video/quicktime",- ".m4a": "audio/mp4", ".mp3": "audio/mpeg", ".wav": "audio/wav",- ".woff2": "font/woff2", ".woff": "font/woff",- ".otf": "font/otf", ".ttf": "font/ttf",-};--const SKIP = new Set(["node_modules", ".git", ".svml", "dist", ".DS_Store"]);--export type DirectoryEntry = {- readonly name: string;- readonly kind: "dir" | "file";-};--export type DirectoryListing = {- readonly dir: string;- readonly parent: string | undefined;- readonly entries: readonly DirectoryEntry[];-};--/**- * Resolves a request path inside the root, or returns undefined.- *- * Containment is checked after resolution, on the resolved path, so `..` and- * symlinked escapes are both refused rather than merely discouraged.- */-export function within(root: string, requested: string): string | undefined {- const target = resolve(root, requested);- if (target !== root && !target.startsWith(root + sep)) return undefined;- return target;-}--export async function listDirectory(root: string, dir: string): Promise<DirectoryListing> {- const target = within(root, dir);- if (target === undefined) throw new Error("Path is outside the repository.");- const entries = await readdir(target, { withFileTypes: true });- const visible = entries- .filter((entry) => !SKIP.has(entry.name) && !entry.name.startsWith("."))- .flatMap((entry): DirectoryEntry[] => {- if (entry.isDirectory()) return [{ name: entry.name, kind: "dir" }];- const extension = extname(entry.name).toLowerCase();- if (!SHEET.has(extension) && !MEDIA.has(extension)) return [];- return [{ name: entry.name, kind: "file" }];- });- visible.sort((left, right) =>- left.kind === right.kind- ? left.name.localeCompare(right.name)- : left.kind === "dir" ? -1 : 1);- const here = relative(root, target);- return {- dir: here,- parent: target === root ? undefined : relative(root, resolve(target, "..")),- entries: visible,- };-}--function fail(response: ServerResponse, status: number, message: string): void {- response.statusCode = status;- response.setHeader("content-type", "application/json");- response.end(JSON.stringify({ error: message }));-}--function json(response: ServerResponse, value: unknown): void {- response.setHeader("content-type", "application/json");- response.end(JSON.stringify(value));-}--export function playgroundBrowsePlugin(root: string): Plugin {- return {- name: "svml-playground-browse",- configureServer(server) {- server.middlewares.use("/__pg", (request, response, next) => {- void (async () => {- const url = new URL(request.url ?? "/", "http://localhost");- try {- if (url.pathname === "/list") {- json(response, await listDirectory(root, url.searchParams.get("dir") ?? "."));- return;- }- const file = url.searchParams.get("file");- if (file === null) { fail(response, 400, "file is required"); return; }- const target = within(root, file);- if (target === undefined) { fail(response, 403, "Path is outside the repository."); return; }- const info = await stat(target);- if (!info.isFile()) { fail(response, 404, "Not a file."); return; }-- if (url.pathname === "/read") {- response.setHeader("content-type", "text/plain; charset=utf-8");- response.end(await readFile(target, "utf8"));- return;- }- if (url.pathname === "/asset") {- const extension = extname(target).toLowerCase();- response.setHeader("content-type", MEDIA_TYPES[extension] ?? "application/octet-stream");- response.end(await readFile(target));- return;- }- next();- } catch (error) {- fail(response, 404, error instanceof Error ? error.message : String(error));- }- })();- });- },- };-}diff --git a/tools/playground/src/ui/browse.ts b/tools/playground/src/ui/browse.tsdeleted file mode 100644index a9d43a34..00000000--- a/tools/playground/src/ui/browse.ts+++ /dev/null@@ -1,112 +0,0 @@-export const BROWSE_CSS = `-.browse { display: flex; flex-direction: column; gap: 5px; }-.browse-path {- color: var(--muted); font-size: 11px; font-family: ui-monospace, monospace;- overflow: hidden; text-overflow: ellipsis; white-space: nowrap; direction: rtl; text-align: left;-}-.browse-list {- max-height: 190px; overflow-y: auto; border: 1px solid var(--line);- border-radius: 6px; background: #131316;-}-.browse-list button {- display: block; width: 100%; text-align: left; border: 0; background: none;- color: var(--text); padding: 4px 9px; font: inherit; font-size: 12px; cursor: pointer;-}-.browse-list button:hover { background: #1c1c21; }-.browse-list button.dir { color: var(--muted); }-.browse-list button.sheet { color: var(--accent); }-.browse-empty { padding: 8px 9px; color: var(--muted); font-size: 12px; }-`;--type Entry = { readonly name: string; readonly kind: "dir" | "file" };-type Listing = {- readonly dir: string;- readonly parent: string | undefined;- readonly entries: readonly Entry[];-};--async function list(dir: string): Promise<Listing> {- const response = await fetch(`/__pg/list?dir=${encodeURIComponent(dir)}`);- if (!response.ok) throw new Error(`Cannot read ${dir || "the repository root"}.`);- return await response.json() as Listing;-}--export async function readSheet(path: string): Promise<string> {- const response = await fetch(`/__pg/read?file=${encodeURIComponent(path)}`);- if (!response.ok) throw new Error(`Cannot read ${path}.`);- return await response.text();-}--export type Browser = {- readonly element: HTMLElement;- open(dir: string): void;-};--/**- * A directory list, starting at the repository root.- *- * Only directories and stylesheets are offered, because those are the only- * things this panel can act on. A folder with neither reads as empty, which is- * the truth about what the playground can do with it.- */-export function createBrowser(onSheet: (path: string) => void): Browser {- const element = document.createElement("div");- element.className = "browse";- const path = document.createElement("div");- path.className = "browse-path";- const listing = document.createElement("div");- listing.className = "browse-list";- element.append(path, listing);-- let current = ".";-- function open(dir: string): void {- current = dir;- void list(dir).then((result) => {- path.textContent = result.dir === "" ? "/" : `/${result.dir}`;- path.title = path.textContent;- listing.replaceChildren();-- if (result.parent !== undefined) {- const up = document.createElement("button");- up.type = "button";- up.className = "dir";- up.textContent = "../";- up.addEventListener("click", () => open(result.parent!));- listing.append(up);- }- for (const entry of result.entries) {- const item = document.createElement("button");- item.type = "button";- const child = result.dir === "" ? entry.name : `${result.dir}/${entry.name}`;- if (entry.kind === "dir") {- item.className = "dir";- item.textContent = `${entry.name}/`;- item.addEventListener("click", () => open(child));- } else if (entry.name.endsWith(".svs")) {- item.className = "sheet";- item.textContent = entry.name;- item.addEventListener("click", () => onSheet(child));- } else {- continue;- }- listing.append(item);- }- if (listing.childElementCount === 0) {- const empty = document.createElement("div");- empty.className = "browse-empty";- empty.textContent = "No folders or stylesheets here.";- listing.append(empty);- }- }).catch((error: unknown) => {- listing.replaceChildren();- const failed = document.createElement("div");- failed.className = "browse-empty";- failed.textContent = error instanceof Error ? error.message : String(error);- listing.append(failed);- });- }-- open(current);- return { element, open };-}diff --git a/tools/playground/src/ui/shell.ts b/tools/playground/src/ui/shell.tsindex f8a2757b..be44af3a 100644--- a/tools/playground/src/ui/shell.ts+++ b/tools/playground/src/ui/shell.ts@@ -12,7 +12,6 @@ import {sealProgramSpace,} from "../svml.js";import type { CanonicalValue, ProgramSpace, Track } from "../svml.js";-import { BROWSE_CSS, createBrowser, readSheet } from "./browse.js";import { FORM_CSS, blankValue, buildForm } from "./form.js";import { createStage } from "./stage.js";@@ -35,7 +34,12 @@ const CSS = `.module { color: var(--muted); font-size: 11px; font-family: ui-monospace, monospace; margin-top: 4px; }.sheet-note { color: var(--muted); font-size: 12px; margin-top: 7px; }.sheet-note b { color: var(--accent); font-weight: 500; }-${BROWSE_CSS}+.sheet-file { font-size: 11px; color: var(--muted); max-width: 100%; }+.sheet-file::file-selector-button {+ border: 1px solid var(--line); border-radius: 5px; background: #1c1c21;+ color: var(--text); font: inherit; font-size: 11px; padding: 3px 9px; margin-right: 7px; cursor: pointer;+}+.sheet-file::file-selector-button:hover { border-color: #3a3a42; }${FORM_CSS}`;@@ -163,18 +167,28 @@ export async function mountShell(root: HTMLElement): Promise<void> {}}- function loadSheet(path: string): void {- void readSheet(path).then((source) => {- const header = parseSourceHeader(path, source);+ const note = document.createElement("div");+ note.className = "sheet-note";++ /**+ * Reads a chosen stylesheet for its canvas.+ *+ * A Film Recipe is where a frame size comes from, so a sheet is offered as+ * a way to adopt one. Nothing else in it is read: what a component looks+ * like comes from the module that owns it.+ */+ function loadSheet(file: File): void {+ void file.text().then((source) => {+ const header = parseSourceHeader(file.name, source);if (!header.using.startsWith("@narratage/svs@")) {- throw new Error(`${path} is a ${header.using} source, not a stylesheet.`);+ throw new Error(`${file.name} is a ${header.using} source, not a stylesheet.`);}- const sheet = parseSvs(path, maskSourceHeader(source, header));+ const sheet = parseSvs(file.name, maskSourceHeader(source, header));const filmShape = [...filmRecipeKeys].sort().join(" ");const found = sheet.recipes.find((recipe) =>Object.keys(recipe.value.properties).sort().join(" ") === filmShape);if (found === undefined) {- note.textContent = "This stylesheet declares no Film Recipe, so the canvas is unchanged.";+ note.textContent = `${file.name} declares no Film Recipe, so the canvas is unchanged.`;return;}const read = filmCanvasFromRecipe(found.value.properties);@@ -195,9 +209,14 @@ export async function mountShell(root: HTMLElement): Promise<void> {});}- const browser = createBrowser(loadSheet);- const note = document.createElement("div");- note.className = "sheet-note";+ const sheetPicker = document.createElement("input");+ sheetPicker.type = "file";+ sheetPicker.accept = ".svs";+ sheetPicker.className = "sheet-file";+ sheetPicker.addEventListener("change", () => {+ const file = sheetPicker.files?.[0];+ if (file !== undefined) loadSheet(file);+ });function section(title: string): HTMLElement {const element = document.createElement("div");@@ -209,7 +228,7 @@ export async function mountShell(root: HTMLElement): Promise<void> {function renderRail(): void {const values = draft(active);moduleLine.textContent = active.moduleName;- body.replaceChildren(section("Canvas from a stylesheet"), browser.element, note);+ body.replaceChildren(section("Canvas from a stylesheet"), sheetPicker, note);for (const input of active.inputs) {// The shell owns the frame domain; offering it again would be a seconddiff --git a/tools/playground/vite.config.ts b/tools/playground/vite.config.tsindex 5954dc34..ad71126f 100644--- a/tools/playground/vite.config.ts+++ b/tools/playground/vite.config.ts@@ -4,8 +4,6 @@ import { fileURLToPath } from "node:url";import { defineConfig } from "vite";-import { playgroundBrowsePlugin } from "./src/server/browse.js";-const here = dirname(fileURLToPath(import.meta.url));const repoRoot = resolve(here, "../..");@@ -48,8 +46,9 @@ export default defineConfig({},server: {port: 5178,- // The playground reads sources and media from anywhere in the repository.+ // Workspace sources resolve through the aliases above, which point outside+ // this root. Stylesheets and media are chosen by the operator and read in+ // the browser, so the server serves no repository content of its own.fs: { allow: [repoRoot] },},- plugins: [playgroundBrowsePlugin(repoRoot)],});