Skip to content

Code-first video & image, rendered natively on WebGPU.

feat(components): M3 captions & terminal

okanasl67dd9b73d7e1
diff --git a/packages/components/SPEC.md b/packages/components/SPEC.md
index b7a1414a..9882a351 100644
--- a/packages/components/SPEC.md
+++ b/packages/components/SPEC.md
@@ -973,7 +973,7 @@ Each milestone ends with a changeset for `framefields` (minor), docs, and its ex
| **M0 · Foundations** ✅ | Package scaffold; host binding in framefields (`./components` export, root re-exports, tsdown entry, `bindComponent`); `core/*` modules (§5); test harness (fake host, GPU render test, determinism); tree-shake check; `components.json` collector. | Done: a test-only `Probe` component passes §9.1–9.5 (§12). |
| **M1 · Pointer & camera** ✅ | `MousePointer` (rebuilt to the existing changeset design plus the Wave 1 additions), `FocusZoom`, `AppFrame`. | Example `30_product_demo`: a screen recording in a browser frame, pointer with clicks and auto-zoom, click SFX placed from cues. |
| **M2 · Type** ✅ | `TextReveal`, `Counter`, `Annotate`. | All §7.4–7.6 acceptance tests green (§14). No example film for M2 or later milestones: examples stop at `30_product_demo`. |
-| **M3 · Speech & shell** | `Captions` (incl. transcript parsers and `behind`), `Terminal`. | Example `32_talking_head_social` (portrait, captions behind the speaker, audio pulse) and a terminal segment in a dev-tool launch cut. |
+| **M3 · Speech & shell** ✅ | `Captions` (incl. transcript parsers and `behind`), `Terminal`. | §7.3 / §7.8 acceptance tests green (§15); no example film. |
| **M4 · Wave 2a** | `CodeBlock`, `TitleCard`, `LowerThird`, `Meter`. | Full specs written first (§8 → §7 format). |
| **M5 · Wave 2b** | `ChatThread`, `Callout`, `Spotlight`, `Confetti`, `Toasts`. | As above. |
| **M6 · Re-score** | Rank Wave 3 against usage and requests. | Updated §8.2. |
@@ -1090,3 +1090,44 @@ Deviations from §7.4–7.6:
- **Annotate:** targets must be static (moving targets belong to `Callout`). `hand.wobble`
is 0–1 (× 5 px at theme scale), `pencil` has no grain yet, and `highlight` multiplies on
light themes and tints on dark ones instead of being inserted under the target.
+
+## 15. M3 delivery notes
+
+What landed: `Captions` (§7.3) with SRT / WebVTT / word-JSON parsers and the phrase
+builder, `Terminal` (§7.8), `core/typing.ts` (shared typing rhythm), `host.cutout`, unit tests
+and `m3.render.test.ts` (karaoke colour changes on `round(start × fps)` and not a frame
+earlier; captions and terminal frames identical in any render order; a finished terminal
+holds still).
+
+How it is built:
+
+- **Renderer indexing.** `textIndex` skips line breaks: after a `\n` every character sits
+ one index earlier than in the string, while `FontManager.measure` keeps string indices.
+ `rendererChars(text, values)` converts any per-character array before it reaches an
+ expression; TextReveal and Captions both use it.
+- **Captions:** one text node per phrase, lines broken by the phrase builder (measured
+ width, `maxChars`, `maxLines`, sentence ends, pauses ≥ `minGap`). Word colour, pops and
+ gradients are glyph animators with exact step / window expressions; `highlight` and `pill`
+ markers are keyed boxes that leave on the word's first frame and land a few frames in, and
+ the text on the marker flips colour as it lands. `outline` uses a layer-style stroke
+ outside the glyphs. Auto font size fits `maxChars` across the safe area, and lines are
+ checked by measured width. `behind` stacks `host.cutout(video)` (segmentation matte; `subject`
+ uses the larger model) over the phrases.
+- **Terminal:** `planTerminal` runs the script on a clock (typing, Enter beat, output pacing,
+ spinners, progress, clears) and the nodes are built from the plan. Lines are drawn only
+ while on screen (`startFrame` / `durationFrames`), so live text nodes stay within
+ `rows + 6`. Scrolling moves by whole lines; a scroll during a glide retargets it, and
+ `clear` snaps. Spinners and check marks are vector shapes, and the progress percentage is
+ a composed `Counter`; the window is a composed `AppFrame` (`frame: false` for the bare
+ screen).
+
+Deviations from §7.3 / §7.8:
+
+- **Captions:** `pulse` (voice-driven active-word scale) and `avoid` are deferred: glyph
+ signal selectors need a way to bind user signals into a text node. Phrase cues are
+ `{ frame, end, text }`. The default font is the theme's display face at weight 800 (body
+ weight for `clean`).
+- **Terminal:** the default prompt is `$ ` and spinners / checks are drawn, because common
+ mono fonts (the bundled JetBrains Mono included) lack `❯`, `✔` and braille. A done line's
+ leading `✔` / `✓` is replaced by the drawn check. The monospace check (warn when not
+ mono) is not implemented.
diff --git a/packages/components/src/captions/index.test.ts b/packages/components/src/captions/index.test.ts
new file mode 100644
index 00000000..81996936
--- /dev/null
+++ b/packages/components/src/captions/index.test.ts
@@ -0,0 +1,102 @@
+import { describe, expect, it } from "vitest";
+import { bind } from "../core/contract.js";
+import { evaluateGlyphExpression, type GlyphAnimator } from "../core/glyphs.js";
+import { fakeHost } from "../test/fake-host.js";
+import { captions } from "./index.js";
+
+const make = () => bind(fakeHost(), captions);
+type Node = {
+ id?: string;
+ children?: Node[];
+ animators?: GlyphAnimator[];
+ startFrame?: number;
+ x?: number;
+ width?: number;
+ styles?: unknown[];
+};
+const find = (n: unknown, id: string): Node | undefined => {
+ const node = n as Node;
+ if (node.id === id) return node;
+ for (const c of node.children ?? []) {
+ const hit = find(c, id);
+ if (hit) return hit;
+ }
+ return undefined;
+};
+const words = "Ship it faster than ever before today"
+ .split(" ")
+ .map((text, i) => ({
+ text,
+ start: 0.5 + i * 0.4,
+ end: 0.8 + i * 0.4,
+ }));
+
+describe("Captions", () => {
+ it("cues every word on round(start × fps), in the parent's timeline", () => {
+ const c = make()({ id: "c", words, at: 10 });
+ expect(c.cues.words.map((w) => w.frame)).toEqual(
+ words.map((w) => 10 + Math.round(w.start * 30)),
+ );
+ expect(c.cues.phrases[0]?.frame).toBe(10 + 15);
+ });
+
+ it("karaoke turns each word to ink exactly on its frame", () => {
+ const c = make()({ id: "c", words, style: "karaoke" });
+ const spoken = find(c, "c-phrase-0")?.animators?.find(
+ (a) => a.name === "spoken",
+ );
+ const expr = spoken?.selectors[0]?.expression ?? "";
+ const phraseStart = c.cues.phrases[0]?.frame ?? 0;
+ const local = (c.cues.words[2]?.frame ?? 0) - phraseStart;
+ expect(
+ evaluateGlyphExpression(expr, { frame: local - 1, wordIndex: 2 }),
+ ).toBe(1);
+ expect(evaluateGlyphExpression(expr, { frame: local, wordIndex: 2 })).toBe(
+ 0,
+ );
+ });
+
+ it("keeps every line inside the safe area at the auto-fitted size", () => {
+ const long =
+ "a fairly long caption sentence that keeps going on and on for several lines of text"
+ .split(" ")
+ .map((text, i) => ({ text, start: i * 0.2, end: i * 0.2 + 0.18 }));
+ const c = make()({ id: "c", words: long, width: 1080, height: 1920 });
+ const safe = 1080 * 0.84;
+ // Fake measure: 0.5 em glyphs, so the longest line × size / 2 must fit.
+ const size = (find(c, "c-phrase-0") as { fontSize?: number }).fontSize ?? 0;
+ const longest = Math.max(...phraseLines(c));
+ expect(longest * size * 0.5).toBeLessThanOrEqual(safe + 1e-6);
+ });
+
+ it("emphasises matching words and supports transcripts", () => {
+ const srt = "1\n00:00:00,000 --> 00:00:01,000\nmake it faster\n";
+ const c = make()({ id: "c", transcript: srt, emphasis: ["faster"] });
+ expect(c.cues.words.map((w) => w.emphasised)).toEqual([false, false, true]);
+ expect(() => make()({})).toThrow(/give `words` or a `transcript`/);
+ });
+
+ it("puts a cut-out of the speaker over the captions", () => {
+ const c = make()({ id: "c", words, behind: { video: "talk.mp4" } });
+ const kids = ((c as unknown as Node).children?.[0]?.children ??
+ []) as Node[];
+ expect(kids[kids.length - 1]?.id).toBe("c-behind");
+ });
+
+ it("outlines with a layer-style stroke, never over the glyphs", () => {
+ const c = make()({ id: "c", words, style: "outline" });
+ expect(find(c, "c-phrase-0")?.styles?.[0]).toMatchObject({
+ stroke: { position: "outside" },
+ });
+ });
+});
+
+/** Every laid-out line's length, across all phrases. */
+function phraseLines(c: unknown): number[] {
+ const out: number[] = [];
+ for (let p = 0; ; p++) {
+ const node = find(c, `c-phrase-${p}`) as { text?: string } | undefined;
+ if (!node) return out;
+ out.push(...(node.text ?? "").split("\n").map((l) => l.length));
+ }
+}
diff --git a/packages/components/src/captions/index.ts b/packages/components/src/captions/index.ts
new file mode 100644
index 00000000..a08c755e
--- /dev/null
+++ b/packages/components/src/captions/index.ts
@@ -0,0 +1,626 @@
+/**
+ * Captions: word-timed captions for talking-head and social video, in the
+ * styles people publish, with exact word highlighting. Each phrase is one
+ * text node; per-word colour, pops and markers are GPU glyph animators and
+ * keyed plates, timed to the word's own frame.
+ *
+ * Coordinates are the frame's: the root covers the video (default
+ * 1920 × 1080 × theme scale, or the given width / height) from 0, 0.
+ */
+import {
+ type KeyframeTuple,
+ LayerAnimation,
+ type LayoutNode,
+ type TextNode,
+} from "@framefields/compositions/program";
+import { z } from "zod";
+import { defineComponent } from "../core/contract.js";
+import type { Rect } from "../core/draw.js";
+import {
+ constantExpression,
+ type GlyphAnimator,
+ glyphAnimator,
+ rendererChars,
+ scheduleExpression,
+ stepExpression,
+ windowExpression,
+} from "../core/glyphs.js";
+import type { Animatable } from "../core/host.js";
+import { type MeasureOptions, rangeRects } from "../core/text.js";
+import { contrastRatio, themeFrames } from "../core/theme.js";
+import { parseTranscript, type Word } from "./parse.js";
+import { buildPhrases, type GroupOptions, type Phrase } from "./phrases.js";
+
+export type { Word } from "./parse.js";
+
+export const CAPTION_STYLES = [
+ "clean",
+ "karaoke",
+ "highlight",
+ "pop",
+ "bold",
+ "outline",
+ "pill",
+ "gradient",
+] as const;
+export type CaptionStyle = (typeof CAPTION_STYLES)[number];
+
+export interface CaptionsCues {
+ [name: string]: ReadonlyArray<{ frame: number }>;
+ /** Each word, from its first frame to its last. */
+ words: ReadonlyArray<{
+ frame: number;
+ end: number;
+ index: number;
+ text: string;
+ emphasised: boolean;
+ }>;
+ /** Each phrase on screen. */
+ phrases: ReadonlyArray<{ frame: number; end: number; text: string }>;
+}
+
+/** Per-style grouping and placement defaults. */
+const GROUP: Record<CaptionStyle, Partial<GroupOptions>> = {
+ clean: { maxChars: 36, maxLines: 2 },
+ karaoke: {},
+ highlight: {},
+ pop: { maxChars: 22 },
+ bold: { maxChars: 14, maxWords: 2, maxLines: 1 },
+ outline: { maxChars: 22 },
+ pill: { maxChars: 24, maxLines: 1 },
+ gradient: {},
+};
+
+const BASE_GROUP: GroupOptions = {
+ maxChars: 28,
+ maxWords: 12,
+ maxLines: 2,
+ breakOnPunctuation: true,
+ minGap: 0.6,
+};
+
+/** The area captions stay inside: clear of edges and, in portrait, platform overlays. */
+function safeArea(
+ W: number,
+ H: number,
+ area: "portrait" | "landscape" | Rect | undefined,
+): Rect {
+ if (area && typeof area === "object") return area;
+ const portrait = area ? area === "portrait" : H > W;
+ return portrait
+ ? { x: W * 0.08, y: H * 0.12, width: W * 0.84, height: H * 0.66 }
+ : { x: W * 0.1, y: H * 0.08, width: W * 0.8, height: H * 0.82 };
+}
+
+export const captions = /* @__PURE__ */ defineComponent({
+ name: "Captions",
+ props: () => ({
+ /** `{ text, start, end }` in seconds. Or `transcript`. */
+ words: z
+ .array(
+ z
+ .object({
+ text: z.string(),
+ start: z.number().min(0),
+ end: z.number().min(0),
+ })
+ .strict(),
+ )
+ .optional(),
+ /** SRT, WebVTT, or word-level JSON (`[{ word | text, start, end }]`). */
+ transcript: z.string().optional(),
+ style: z.enum(CAPTION_STYLES).default("karaoke"),
+ group: z
+ .object({
+ maxChars: z.number().int().positive().optional(),
+ maxWords: z.number().int().positive().optional(),
+ maxLines: z.number().int().positive().optional(),
+ breakOnPunctuation: z.boolean().optional(),
+ minGap: z.number().min(0).optional(),
+ })
+ .strict()
+ .default({}),
+ /** Words or patterns in the accent colour, or a test per word. */
+ emphasis: z
+ .union([
+ z.array(z.union([z.string(), z.instanceof(RegExp)])),
+ z.custom<(word: string, index: number) => boolean>(
+ (v) => typeof v === "function",
+ ),
+ ])
+ .default([]),
+ case: z.enum(["as-is", "upper", "lower"]).optional(),
+ placement: z
+ .object({
+ anchor: z.enum(["bottom", "center", "top"]).optional(),
+ safeArea: z
+ .union([
+ z.enum(["portrait", "landscape"]),
+ z.object({
+ x: z.number(),
+ y: z.number(),
+ width: z.number(),
+ height: z.number(),
+ }),
+ ])
+ .optional(),
+ /** Extra px from the anchor edge, toward the centre. */
+ offset: z.number().default(0),
+ })
+ .strict()
+ .default({ offset: 0 }),
+ /** A clip whose person / subject is cut out and drawn over the captions. */
+ behind: z
+ .object({
+ video: z.string(),
+ mode: z.enum(["person", "subject"]).default("person"),
+ })
+ .strict()
+ .optional(),
+ /** A backing plate behind each phrase (`clean`, `karaoke`). */
+ plate: z.boolean().default(false),
+ fontFamily: z.string().optional(),
+ /** Default: fits `group.maxChars` across the safe area. */
+ fontSize: z.number().positive().optional(),
+ fontWeight: z.union([z.number(), z.string()]).optional(),
+ fill: z.string().optional(),
+ accent: z.string().optional(),
+ }),
+ defaults: { enter: false, exit: false },
+ build(ctx, props) {
+ const { host, theme, fps } = ctx;
+ const W = ctx.requested.width ?? 1920 * theme.scale;
+ const H = ctx.requested.height ?? 1080 * theme.scale;
+ const style = props.style;
+
+ // ── Words ─────────────────────────────────────────────────────────
+ let words: Word[];
+ if (props.words) words = props.words.map((w) => ({ ...w }));
+ else if (props.transcript) words = parseTranscript(props.transcript);
+ else throw new Error("Captions: give `words` or a `transcript`.");
+ const casing = props.case ?? (style === "bold" ? "upper" : "as-is");
+ if (casing !== "as-is")
+ words = words.map((w) => ({
+ ...w,
+ text: casing === "upper" ? w.text.toUpperCase() : w.text.toLowerCase(),
+ }));
+ const emphasised = words.map((w, i) =>
+ typeof props.emphasis === "function"
+ ? props.emphasis(w.text, i)
+ : props.emphasis.some((p) =>
+ typeof p === "string"
+ ? w.text.toLowerCase().replace(/[^\p{L}\p{N}%]/gu, "") ===
+ p.toLowerCase()
+ : new RegExp(p.source, p.flags.replace("g", "")).test(w.text),
+ ),
+ );
+
+ // ── Layout ────────────────────────────────────────────────────────
+ const group: GroupOptions = {
+ ...BASE_GROUP,
+ ...GROUP[style],
+ ...stripUndefined(props.group),
+ };
+ const safe = safeArea(W, H, props.placement.safeArea);
+ const fontFamily = props.fontFamily ?? theme.font.display;
+ const weight = props.fontWeight ?? (style === "clean" ? undefined : 800);
+ const measureAt = (text: string, size: number, width?: number) =>
+ host.measure(text, {
+ fontFamily,
+ fontSize: size,
+ fontWeight: weight,
+ ...(width !== undefined && { width, align: "center" as const }),
+ } satisfies MeasureOptions);
+ // Auto size: a line of `maxChars` average glyphs spans the safe width.
+ const sample = "nonsense words".repeat(4).slice(0, group.maxChars);
+ const fs =
+ props.fontSize ??
+ Math.min(
+ (safe.width / Math.max(1, measureAt(sample, 100).width)) * 100 * 0.94,
+ safe.height / (group.maxLines * 1.6),
+ );
+ const fits = (line: string) => measureAt(line, fs).width <= safe.width;
+ const phrases = buildPhrases(words, group, fits);
+
+ const ink = props.fill ?? theme.color.ink;
+ const accent = props.accent ?? theme.color.accent;
+ const anchor =
+ props.placement.anchor ?? (style === "bold" ? "center" : "bottom");
+ const frameOf = (seconds: number) => Math.round(seconds * fps);
+ const fast = themeFrames(theme, "fast", fps);
+ const children: LayoutNode[] = [];
+ const parts: Record<string, Animatable<LayoutNode>> = {};
+ const wordCues: Array<CaptionsCues["words"][number]> = [];
+ const phraseCues: Array<CaptionsCues["phrases"][number]> = [];
+
+ phrases.forEach((phrase, p) => {
+ const layout = phraseLayout(phrase, words);
+ const m = measureAt(layout.text, fs, safe.width);
+ const options: MeasureOptions = {
+ fontFamily,
+ fontSize: fs,
+ fontWeight: weight,
+ width: safe.width,
+ align: "center",
+ };
+ const y =
+ anchor === "bottom"
+ ? safe.y + safe.height - m.height - props.placement.offset
+ : anchor === "top"
+ ? safe.y + props.placement.offset
+ : safe.y + (safe.height - m.height) / 2;
+ const start = frameOf(phrase.start);
+ const end = Math.max(start + 1, frameOf(phrase.end));
+ // Word windows, local to the phrase (its box starts at `start`).
+ const wStart = layout.words.map(
+ (w) => frameOf((words[w.index] as Word).start) - start,
+ );
+ const wEnd = layout.words.map((w, k) =>
+ k + 1 < layout.words.length
+ ? (wStart[k + 1] as number)
+ : Math.max(
+ (wStart[k] as number) + 1,
+ frameOf((words[w.index] as Word).end) - start,
+ ),
+ );
+ const rects = layout.words.map(
+ (w) => rangeRects(m, w.from, w.to, options)[0] as Rect,
+ );
+ const animators: GlyphAnimator[] = [];
+
+ const emphasisWeights = layout.words.map((w) =>
+ emphasised[w.index] ? 1 : 0,
+ );
+ if (emphasisWeights.some(Boolean))
+ animators.push(
+ glyphAnimator(
+ "emphasis",
+ constantExpression("word", emphasisWeights),
+ { fillColor: accent },
+ ),
+ );
+
+ const node: Partial<TextNode> = {
+ id: ctx.partId(`phrase-${p}`),
+ position: "absolute",
+ x: 0,
+ y: 0,
+ width: safe.width,
+ align: "center",
+ fontFamily,
+ fontSize: fs,
+ fill: ink,
+ ...(weight !== undefined && { fontWeight: weight }),
+ };
+ const plates: LayoutNode[] = [];
+ const box = (
+ id: string,
+ r: Rect,
+ pad: number,
+ background: string,
+ radius: number,
+ extra = {},
+ ) =>
+ host.box({
+ id: ctx.partId(`phrase-${p}-${id}`),
+ position: "absolute",
+ x: r.x - pad,
+ y: r.y - pad * 0.5,
+ width: r.width + pad * 2,
+ height: r.height + pad,
+ background,
+ borderRadius: radius,
+ ...extra,
+ });
+ const blockRect: Rect = {
+ x: Math.min(...rects.map((r) => r.x)),
+ y: 0,
+ width:
+ Math.max(...rects.map((r) => r.x + r.width)) -
+ Math.min(...rects.map((r) => r.x)),
+ height: m.height,
+ };
+
+ switch (style) {
+ case "clean":
+ break;
+ case "karaoke":
+ // Spoken words turn from muted to ink on their exact frame.
+ animators.push(
+ glyphAnimator("spoken", stepExpression("word", wStart), {
+ fillColor: theme.color.muted,
+ }),
+ );
+ break;
+ case "highlight":
+ case "pill": {
+ const pad = fs * (style === "pill" ? 0.22 : 0.16);
+ const glide = Math.max(2, Math.round(fast * 0.4));
+ if (style === "pill")
+ plates.push(
+ box(
+ "plate",
+ blockRect,
+ fs * 0.5,
+ theme.color.surface,
+ theme.radius.pill,
+ { opacity: 0.92 },
+ ),
+ );
+ const marker = box(
+ "marker",
+ rects[0] as Rect,
+ pad,
+ accent,
+ style === "pill" ? theme.radius.pill : theme.radius.md,
+ );
+ marker.animate(markerKeys(rects, wStart, pad, glide));
+ plates.push(marker);
+ // Text on the marker reads in the background colour when ink wouldn't.
+ if ((contrastRatio(ink, accent) ?? 21) < 3)
+ animators.push(
+ glyphAnimator(
+ "on-marker",
+ // Switch as the marker lands under the word.
+ windowExpression(
+ "word",
+ wStart.map((f) => f + Math.ceil(glide / 2)),
+ wEnd.map((f) => f + Math.ceil(glide / 2)),
+ ),
+ { fillColor: theme.color.bg },
+ ),
+ );
+ break;
+ }
+ case "pop":
+ animators.push(
+ glyphAnimator(
+ "pop",
+ scheduleExpression(
+ "word",
+ {
+ starts: wStart,
+ duration: Math.max(3, Math.round(fast * 0.9)),
+ ease: "back.out(2.2)",
+ },
+ "enter",
+ ),
+ { opacity: 0, scale: 0.72, y: fs * 0.12 },
+ ),
+ );
+ break;
+ case "bold":
+ case "outline":
+ animators.push(
+ glyphAnimator("active", windowExpression("word", wStart, wEnd), {
+ fillColor: accent,
+ }),
+ );
+ break;
+ case "gradient": {
+ // Active word: accent → accent2 across its letters.
+ const chars = layout.text.length;
+ const starts = new Array<number>(chars).fill(-1);
+ const ends = new Array<number>(chars).fill(-1);
+ const along = new Array<number>(chars).fill(0);
+ layout.words.forEach((w, k) => {
+ for (let c = w.from; c < w.to; c++) {
+ starts[c] = wStart[k] as number;
+ ends[c] = wEnd[k] as number;
+ along[c] =
+ w.to - w.from > 1 ? (c - w.from) / (w.to - w.from - 1) : 0;
+ }
+ });
+ animators.push(
+ glyphAnimator(
+ "active",
+ windowExpression(
+ "char",
+ rendererChars(layout.text, starts),
+ rendererChars(layout.text, ends),
+ ),
+ {
+ fillColor: accent,
+ },
+ ),
+ glyphAnimator(
+ "active-end",
+ windowExpression(
+ "char",
+ rendererChars(layout.text, starts),
+ rendererChars(layout.text, ends),
+ rendererChars(layout.text, along),
+ ),
+ { fillColor: theme.color.accent2 },
+ ),
+ );
+ break;
+ }
+ }
+ if (props.plate && (style === "clean" || style === "karaoke"))
+ plates.unshift(
+ box(
+ "plate",
+ blockRect,
+ fs * 0.35,
+ "rgba(0,0,0,0.55)",
+ theme.radius.md,
+ ),
+ );
+
+ if (animators.length > 0) node.animators = animators as never;
+ const text = host.text(layout.text, node);
+ // A heavy outline around the glyphs, never over them.
+ if (style === "outline")
+ host.style(text, {
+ stroke: { size: fs * 0.08, position: "outside", color: "#000000" },
+ });
+ const holder = host.box({
+ id: ctx.partId(`phrase-${p}-box`),
+ position: "absolute",
+ x: safe.x,
+ y,
+ width: safe.width,
+ height: m.height,
+ startFrame: start,
+ durationFrames: end - start,
+ children: [...plates, text],
+ });
+ holder.animate(
+ phraseMotion(style, p, end - start, fs, fast, { x: safe.x, y }),
+ );
+ parts[`phrase-${p}`] = holder;
+ children.push(holder);
+
+ phraseCues.push({
+ frame: start,
+ end,
+ text: layout.text.replace(/\n/g, " "),
+ });
+ layout.words.forEach((w, k) => {
+ wordCues.push({
+ frame: start + (wStart[k] as number),
+ end: start + (wEnd[k] as number),
+ index: w.index,
+ text: (words[w.index] as Word).text,
+ emphasised: emphasised[w.index] ?? false,
+ });
+ });
+ });
+
+ if (props.behind) {
+ const cut = host.cutout(props.behind.video, {
+ id: ctx.partId("behind"),
+ position: "absolute",
+ x: 0,
+ y: 0,
+ width: W,
+ height: H,
+ fit: "cover",
+ mode: props.behind.mode,
+ });
+ parts.behind = cut;
+ children.push(cut);
+ }
+
+ const last = phraseCues[phraseCues.length - 1]?.end ?? 1;
+ return {
+ children,
+ width: W,
+ height: H,
+ duration: last,
+ origin: [0, 0],
+ cues: { words: wordCues, phrases: phraseCues } as CaptionsCues,
+ parts,
+ };
+ },
+});
+
+/** A phrase's text and each word's character range in it. */
+function phraseLayout(phrase: Phrase, words: readonly Word[]) {
+ const out: Array<{ index: number; from: number; to: number }> = [];
+ let text = "";
+ phrase.lines.forEach((line, l) => {
+ if (l > 0) text += "\n";
+ line.forEach((index, k) => {
+ if (k > 0) text += " ";
+ const from = text.length;
+ text += (words[index] as Word).text;
+ out.push({ index, from, to: text.length });
+ });
+ });
+ return { text, words: out };
+}
+
+/** A marker that glides onto each word, arriving on the frame it is spoken. */
+function markerKeys(
+ rects: readonly Rect[],
+ starts: readonly number[],
+ pad: number,
+ glide: number,
+): LayerAnimation {
+ const keyed = (value: (r: Rect) => number) => {
+ const keys: KeyframeTuple[] = [[0, value(rects[0] as Rect)]];
+ rects.forEach((r, k) => {
+ const prev = rects[k - 1];
+ if (!prev) return;
+ const at = starts[k] as number;
+ const last = (keys[keys.length - 1] as KeyframeTuple)[0];
+ // Leave on the word's first frame and land a few frames in: the
+ // marker never runs ahead of the voice.
+ keys.push([Math.max(last + 1, at), value(prev)]);
+ keys.push([Math.max(last + 2, at + glide), value(r), "power3.out"]);
+ });
+ return keys;
+ };
+ return LayerAnimation.create()
+ .keys(
+ "x",
+ keyed((r) => r.x - pad),
+ )
+ .keys(
+ "y",
+ keyed((r) => r.y - pad * 0.5),
+ )
+ .keys(
+ "width",
+ keyed((r) => r.width + pad * 2),
+ )
+ .keys(
+ "height",
+ keyed((r) => r.height + pad),
+ )
+ .keys("opacity", [
+ [0, 0],
+ [starts[0] ?? 0, 1, "hold"],
+ ]);
+}
+
+/** How a phrase arrives and leaves. `bold` alternates sides; the rest fade up. */
+function phraseMotion(
+ style: CaptionStyle,
+ index: number,
+ length: number,
+ fs: number,
+ fast: number,
+ at: { x: number; y: number },
+): LayerAnimation {
+ const anim = LayerAnimation.create();
+ const inF = Math.min(
+ Math.max(2, Math.round(fast * 0.6)),
+ Math.floor(length / 2),
+ );
+ const out = Math.min(
+ Math.max(2, Math.round(fast * 0.5)),
+ Math.floor(length / 3),
+ );
+ anim.keys("opacity", [
+ [0, 0],
+ [inF, 1, "power2.out"],
+ [length - out, 1],
+ [length, 0, "power2.in"],
+ ]);
+ if (style === "bold") {
+ const side = index % 2 === 0 ? -1 : 1;
+ anim.keys("x", [
+ [0, at.x + side * fs * 0.6],
+ [inF + 2, at.x, "back.out(1.8)"],
+ ]);
+ anim.keys("scale", [
+ [0, 0.86],
+ [inF + 2, 1, "back.out(1.8)"],
+ ]);
+ } else if (style !== "pop") {
+ anim.keys("y", [
+ [0, at.y + fs * 0.14],
+ [inF, at.y, "power3.out"],
+ ]);
+ }
+ return anim;
+}
+
+function stripUndefined<T extends object>(o: T): Partial<T> {
+ return Object.fromEntries(
+ Object.entries(o).filter(([, v]) => v !== undefined),
+ ) as Partial<T>;
+}
diff --git a/packages/components/src/captions/meta.ts b/packages/components/src/captions/meta.ts
new file mode 100644
index 00000000..8082545e
--- /dev/null
+++ b/packages/components/src/captions/meta.ts
@@ -0,0 +1,19 @@
+import { defineMeta } from "../core/meta.js";
+
+export default defineMeta({
+ name: "Captions",
+ summary:
+ "Word-timed captions from words or a transcript (SRT, WebVTT, word JSON) in eight published styles, highlighting each word on its exact frame, optionally tucked behind the speaker.",
+ tags: [
+ "captions",
+ "subtitles",
+ "social",
+ "talking head",
+ "transcript",
+ "karaoke",
+ ],
+ parts: ["phrase-<n>", "behind"],
+ cues: ["words", "phrases"],
+ example: `Captions({ transcript: srt, style: "karaoke", emphasis: ["faster"] })`,
+ since: "M3",
+});
diff --git a/packages/components/src/captions/parse.test.ts b/packages/components/src/captions/parse.test.ts
new file mode 100644
index 00000000..abaaf61c
--- /dev/null
+++ b/packages/components/src/captions/parse.test.ts
@@ -0,0 +1,52 @@
+import { describe, expect, it } from "vitest";
+import { parseTimestamp, parseTranscript, splitCue } from "./parse.js";
+
+const srt = `1
+00:00:01,000 --> 00:00:02,000
+Ship it
+
+2
+00:00:02,500 --> 00:00:03,000
+<i>faster</i>
+`;
+const vtt = `WEBVTT
+
+00:01.000 --> 00:02.000
+Ship it
+
+00:02.500 --> 00:03.000
+faster
+`;
+const json = JSON.stringify([
+ { word: "Ship", start: 1, end: 1.4 },
+ { text: "it", start: 1.4, end: 2 },
+ { word: "faster", start: 2.5, end: 3 },
+]);
+
+describe("transcripts", () => {
+ it("reads timestamps in any common form", () => {
+ expect(parseTimestamp("01:02:03,456")).toBeCloseTo(3723.456, 6);
+ expect(parseTimestamp("02:03.5")).toBeCloseTo(123.5, 6);
+ expect(() => parseTimestamp("x")).toThrow(/bad timestamp/);
+ });
+
+ it("splits a cue's time by word length", () => {
+ const words = splitCue("a bbb", 0, 6);
+ expect(words.map((w) => w.text)).toEqual(["a", "bbb"]);
+ expect(words[0]?.end).toBeCloseTo(2, 6);
+ expect(words[1]?.end).toBeCloseTo(6, 6);
+ });
+
+ it("SRT, WebVTT and word JSON give the same words", () => {
+ const texts = (s: string) => parseTranscript(s).map((w) => w.text);
+ expect(texts(srt)).toEqual(["Ship", "it", "faster"]);
+ expect(texts(vtt)).toEqual(texts(srt));
+ expect(texts(json)).toEqual(texts(srt));
+ expect(parseTranscript(srt)[2]).toMatchObject({ start: 2.5, end: 3 });
+ });
+
+ it("names the problem in bad JSON", () => {
+ expect(() => parseTranscript('[{"word":"x"}]')).toThrow(/needs `word`/);
+ expect(() => parseTranscript("nothing here")).toThrow(/no words/);
+ });
+});
diff --git a/packages/components/src/captions/parse.ts b/packages/components/src/captions/parse.ts
new file mode 100644
index 00000000..29edb8b5
--- /dev/null
+++ b/packages/components/src/captions/parse.ts
@@ -0,0 +1,99 @@
+/**
+ * Transcripts to timed words. Word-level JSON keeps its times; cue-level
+ * formats (SRT, WebVTT) split each cue into words and share the cue's time
+ * between them by length, so longer words are on screen longer.
+ */
+
+/** A spoken word, times in seconds. */
+export interface Word {
+ text: string;
+ start: number;
+ end: number;
+}
+
+/** `01:02:03,456`, `02:03.456` or `3.456` → seconds. */
+export function parseTimestamp(value: string): number {
+ const parts = value.trim().replace(",", ".").split(":");
+ let seconds = 0;
+ for (const p of parts) {
+ const n = Number(p);
+ if (!Number.isFinite(n))
+ throw new Error(`Captions: bad timestamp "${value}".`);
+ seconds = seconds * 60 + n;
+ }
+ return seconds;
+}
+
+/** Spread a cue's [start, end) over its words, weighted by length. */
+export function splitCue(text: string, start: number, end: number): Word[] {
+ const tokens = text.split(/\s+/).filter(Boolean);
+ const weights = tokens.map((t) => t.length + 1);
+ const total = weights.reduce((a, b) => a + b, 0) || 1;
+ let t = start;
+ return tokens.map((token, i) => {
+ const span = ((end - start) * (weights[i] as number)) / total;
+ const word = { text: token, start: t, end: t + span };
+ t += span;
+ return word;
+ });
+}
+
+const ARROW = /^\s*([\d:.,]+)\s*-->\s*([\d:.,]+)/;
+
+/** SRT or WebVTT cues → words. Markup tags (`<i>`, `<c.x>`, `{\an8}`) are dropped. */
+function parseCues(source: string): Word[] {
+ const words: Word[] = [];
+ const blocks = source.replace(/\r/g, "").split(/\n\s*\n/);
+ for (const block of blocks) {
+ const lines = block.split("\n");
+ const at = lines.findIndex((l) => ARROW.test(l));
+ if (at < 0) continue;
+ const m = (lines[at] as string).match(ARROW) as RegExpMatchArray;
+ const text = lines
+ .slice(at + 1)
+ .join(" ")
+ .replace(/<[^>]+>|\{[^}]*\}/g, "")
+ .trim();
+ if (text)
+ words.push(
+ ...splitCue(
+ text,
+ parseTimestamp(m[1] as string),
+ parseTimestamp(m[2] as string),
+ ),
+ );
+ }
+ return words;
+}
+
+/** `[{ word | text, start, end }]` (seconds). */
+function parseWordJson(source: string): Word[] {
+ const data = JSON.parse(source) as unknown;
+ const list = Array.isArray(data)
+ ? data
+ : ((data as { words?: unknown[] })?.words ?? []);
+ return (list as Array<Record<string, unknown>>).map((w, i) => {
+ const text = (w.word ?? w.text) as unknown;
+ if (
+ typeof text !== "string" ||
+ typeof w.start !== "number" ||
+ typeof w.end !== "number"
+ )
+ throw new Error(
+ `Captions: word ${i} needs \`word\` (or \`text\`), \`start\` and \`end\`.`,
+ );
+ return { text: text.trim(), start: w.start, end: w.end };
+ });
+}
+
+/** Any supported transcript (word JSON, WebVTT or SRT), detected from its content. */
+export function parseTranscript(source: string): Word[] {
+ const trimmed = source.trim();
+ const words =
+ trimmed.startsWith("[") || trimmed.startsWith("{")
+ ? parseWordJson(trimmed)
+ : parseCues(trimmed.replace(/^WEBVTT[^\n]*\n/, ""));
+ if (words.length === 0)
+ throw new Error("Captions: the transcript has no words.");
+ return words.filter((w) => w.text.length > 0);
+}
diff --git a/packages/components/src/captions/phrases.test.ts b/packages/components/src/captions/phrases.test.ts
new file mode 100644
index 00000000..38da9ab1
--- /dev/null
+++ b/packages/components/src/captions/phrases.test.ts
@@ -0,0 +1,48 @@
+import { describe, expect, it } from "vitest";
+import type { Word } from "./parse.js";
+import { buildPhrases, phraseText } from "./phrases.js";
+
+const timed = (text: string, gapAfter: Record<number, number> = {}): Word[] => {
+ let t = 0;
+ return text.split(" ").map((w, i) => {
+ const word = { text: w, start: t, end: t + 0.3 };
+ t += 0.3 + (gapAfter[i] ?? 0.05);
+ return word;
+ });
+};
+const base = {
+ maxChars: 20,
+ maxWords: 12,
+ maxLines: 2,
+ breakOnPunctuation: true,
+ minGap: 0.6,
+};
+
+describe("phrases", () => {
+ it("fill lines up to maxChars without splitting words", () => {
+ const words = timed("one two three four five six seven eight");
+ const [p] = buildPhrases(words, base);
+ expect(phraseText(p as never, words)).toBe(
+ "one two three four\nfive six seven eight",
+ );
+ });
+
+ it("end at sentence ends and long pauses", () => {
+ const words = timed("Hello there. Next one comes after pause", { 4: 1 });
+ const texts = buildPhrases(words, base).map((p) => phraseText(p, words));
+ expect(texts).toEqual(["Hello there.", "Next one comes", "after pause"]);
+ });
+
+ it("respect measured width, not just characters", () => {
+ const words = timed("wide wide wide wide");
+ const [p] = buildPhrases(words, base, (line) => line.length <= 9);
+ expect(p?.lines.map((l) => l.length)).toEqual([2, 2]);
+ });
+
+ it("stay until the next phrase, never overlapping it", () => {
+ const words = timed("a. b", { 0: 0.1 });
+ const [first, second] = buildPhrases(words, base);
+ expect(first?.end).toBeLessThanOrEqual(second?.start ?? 0);
+ expect(second?.end).toBeCloseTo((words[1]?.end ?? 0) + 0.6, 6);
+ });
+});
diff --git a/packages/components/src/captions/phrases.ts b/packages/components/src/captions/phrases.ts
new file mode 100644
index 00000000..a177cd2c
--- /dev/null
+++ b/packages/components/src/captions/phrases.ts
@@ -0,0 +1,98 @@
+/**
+ * Words to on-screen phrases: what is shown together, broken into lines.
+ * A phrase never splits a word, prefers to end at punctuation or a pause,
+ * and every line fits the safe area as measured (not just by characters).
+ */
+import type { Word } from "./parse.js";
+
+export interface GroupOptions {
+ maxChars: number;
+ maxWords: number;
+ maxLines: number;
+ breakOnPunctuation: boolean;
+ /** A silence this long (seconds) always starts a new phrase. */
+ minGap: number;
+}
+
+export interface Phrase {
+ /** Indices into the words, per line. */
+ lines: number[][];
+ /** First and last word index (inclusive). */
+ first: number;
+ last: number;
+ /** Seconds. */
+ start: number;
+ end: number;
+}
+
+const STRONG = /[.!?…]["')\]]*$/;
+const SOFT = /[,;:—–]["')\]]*$/;
+
+/** Group words into phrases. `fits(line)` says whether a line of text fits the width. */
+export function buildPhrases(
+ words: readonly Word[],
+ o: GroupOptions,
+ fits: (line: string) => boolean = () => true,
+ /** Seconds a phrase lingers after its last word, if nothing follows sooner. */
+ linger = 0.6,
+): Phrase[] {
+ const phrases: Phrase[] = [];
+ let lines: number[][] = [];
+ const lineText = (line: number[]) =>
+ line.map((i) => words[i]?.text ?? "").join(" ");
+ const count = () => lines.reduce((a, l) => a + l.length, 0);
+ const chars = () => lines.map(lineText).join(" ").length;
+
+ const close = () => {
+ if (lines.length === 0) return;
+ const flat = lines.flat();
+ const first = flat[0] as number;
+ const last = flat[flat.length - 1] as number;
+ phrases.push({
+ lines,
+ first,
+ last,
+ start: (words[first] as Word).start,
+ end: (words[last] as Word).end,
+ });
+ lines = [];
+ };
+
+ words.forEach((word, i) => {
+ const prev = words[i - 1];
+ if (prev && lines.length > 0) {
+ const pause = word.start - prev.end >= o.minGap;
+ const strong = o.breakOnPunctuation && STRONG.test(prev.text);
+ const soft =
+ o.breakOnPunctuation &&
+ SOFT.test(prev.text) &&
+ chars() >= o.maxChars * 0.6;
+ if (pause || strong || soft || count() >= o.maxWords) close();
+ }
+ const line = lines[lines.length - 1];
+ if (line) {
+ const candidate = `${lineText(line)} ${word.text}`;
+ if (candidate.length <= o.maxChars && fits(candidate)) {
+ line.push(i);
+ return;
+ }
+ if (lines.length >= o.maxLines) close();
+ }
+ lines.push([i]);
+ });
+ close();
+
+ // On screen until the next phrase starts, lingering a little after the last word.
+ phrases.forEach((p, k) => {
+ const next = phrases[k + 1];
+ p.end = next ? Math.min(next.start, p.end + linger) : p.end + linger;
+ });
+ return phrases;
+}
+
+/** The text a phrase shows, lines joined with newlines. */
+export function phraseText(p: Phrase, words: readonly Word[]): string {
+ return p.lines
+ .map((l) => l.map((i) => words[i]?.text ?? "").join(" "))
+ .join("\n");
+}
diff --git a/packages/components/src/core/glyphs.ts b/packages/components/src/core/glyphs.ts
index a055297a..4e4dfabc 100644
--- a/packages/components/src/core/glyphs.ts
+++ b/packages/components/src/core/glyphs.ts
@@ -84,6 +84,39 @@ export function scheduleExpression(
].join("");
}
+/**
+ * 1 while each unit is inside its window [start, end) (a word being spoken),
+ * else 0: a hard edge on exact frames. Indexed per unit; `factors` scale it.
+ */
+export function windowExpression(
+ unit: GlyphUnit,
+ starts: readonly number[],
+ ends: readonly number[],
+ factors?: readonly number[],
+): string {
+ const idx = UNIT_INDEX[unit];
+ const f = factors ? `*(${list(factors)}[${idx}]??0)` : "";
+ return `((S,E)=>{const s=S[${idx}];return s!==undefined&&frame>=s&&frame<E[${idx}]?1${f}:0;})(${list(starts)},${list(ends)})`;
+}
+
+/** 1 until each unit's start frame, then 0 (an exact step, no ease). */
+export function stepExpression(
+ unit: GlyphUnit,
+ starts: readonly number[],
+): string {
+ const idx = UNIT_INDEX[unit];
+ return `((S)=>frame<(S[${idx}]??S[S.length-1])?1:0)(${list(starts)})`;
+}
+
+/**
+ * A per-character array (one entry per string index) in the renderer's
+ * order: its `textIndex` skips line breaks, so every character after a
+ * `\n` sits one index earlier than in the string.
+ */
+export function rendererChars<T>(text: string, values: readonly T[]): T[] {
+ return values.filter((_, i) => text[i] !== "\n");
+}
+
/** A fixed weight per unit (e.g. 1 for emphasised characters). */
export function constantExpression(
unit: GlyphUnit,
diff --git a/packages/components/src/core/host.ts b/packages/components/src/core/host.ts
index 8af5dcea..a09ab79e 100644
--- a/packages/components/src/core/host.ts
+++ b/packages/components/src/core/host.ts
@@ -87,4 +87,13 @@ export interface Host {
* background). The matte is laid out like a sibling of `node`; returns `node`.
*/
matte<T extends Animatable<LayoutNode>>(node: T, matte: LayoutNode): T;
+ /**
+ * A video showing only its person / subject (vision segmentation), the
+ * rest transparent: stacked over captions so text passes behind them.
+ */
+ cutout(source: string, options: CutoutOptions): Animatable<LayoutNode>;
+}
+
+export interface CutoutOptions extends MediaOptions {
+ mode?: "person" | "subject";
}
diff --git a/packages/components/src/core/typing.ts b/packages/components/src/core/typing.ts
new file mode 100644
index 00000000..2c9bdd47
--- /dev/null
+++ b/packages/components/src/core/typing.ts
@@ -0,0 +1,34 @@
+/**
+ * Human typing rhythm, shared by everything that types (text reveals,
+ * terminals): seeded jitter, a beat after spaces, a pause after punctuation.
+ */
+import type { Rng } from "./rng.js";
+
+/**
+ * Keystroke frames for typing at `cps` characters a second: seeded ±30 %
+ * jitter, a beat after spaces, a longer one after punctuation. Frame `k[i]`
+ * is when character `i` appears.
+ */
+export function typingFrames(
+ text: string,
+ cps: number,
+ fps: number,
+ r: Rng,
+ start = 0,
+): number[] {
+ const per = fps / cps;
+ const out: number[] = [];
+ let t = start;
+ for (let i = 0; i < text.length; i++) {
+ const prev = text[i - 1] ?? "";
+ let gap = i === 0 ? 0 : per * (1 + r.spread(0.3));
+ if (prev === " ") gap += per * 0.6;
+ if (/[.,!?;:]/.test(prev)) gap += per * 3;
+ t += gap;
+ out.push(Math.max(start, Math.round(t)));
+ }
+ // Two characters never land on one frame.
+ for (let i = 1; i < out.length; i++)
+ out[i] = Math.max(out[i] as number, (out[i - 1] as number) + 1);
+ return out;
+}
diff --git a/packages/components/src/index.ts b/packages/components/src/index.ts
index 0265072b..04af8b5d 100644
--- a/packages/components/src/index.ts
+++ b/packages/components/src/index.ts
@@ -16,6 +16,13 @@ export {
type AppFrameKind,
appFrame,
} from "./app-frame/index.js";
+export {
+ CAPTION_STYLES,
+ type CaptionStyle,
+ type CaptionsCues,
+ captions,
+ type Word,
+} from "./captions/index.js";
export {
type BakedKey,
type BakeOptions,
@@ -67,11 +74,14 @@ export {
type GlyphUnit,
glyphAnimator,
scheduleExpression,
+ stepExpression,
type UnitSchedule,
+ windowExpression,
} from "./core/glyphs.js";
export type {
Animatable,
AnimatableMethods,
+ CutoutOptions,
Host,
LayerStyleConfig,
MediaOptions,
@@ -142,6 +152,7 @@ export {
toSeconds,
within,
} from "./core/timing.js";
+export { typingFrames } from "./core/typing.js";
export {
type CounterCues,
type CounterFormat,
@@ -161,6 +172,11 @@ export {
mousePointer,
type Waypoint,
} from "./mouse-pointer/index.js";
+export {
+ type TerminalCues,
+ type TerminalStep,
+ terminal,
+} from "./terminal/index.js";
export {
EXIT_STYLES,
type ExitStyle,
diff --git a/packages/components/src/terminal/index.test.ts b/packages/components/src/terminal/index.test.ts
new file mode 100644
index 00000000..35a3b0e2
--- /dev/null
+++ b/packages/components/src/terminal/index.test.ts
@@ -0,0 +1,65 @@
+import { describe, expect, it } from "vitest";
+import { bind } from "../core/contract.js";
+import { fakeHost } from "../test/fake-host.js";
+import { terminal } from "./index.js";
+
+const make = () => bind(fakeHost(), terminal);
+type Node = { id?: string; children?: Node[]; x?: number };
+const find = (n: unknown, id: string): Node | undefined => {
+ const node = n as Node;
+ if (node.id === id) return node;
+ for (const c of node.children ?? []) {
+ const hit = find(c, id);
+ if (hit) return hit;
+ }
+ return undefined;
+};
+
+describe("Terminal", () => {
+ it("cues keystrokes, submits and the end in the parent's timeline", () => {
+ const t = make()({
+ id: "t",
+ at: 20,
+ script: [{ cmd: "ls" }, { output: "a" }],
+ });
+ expect(t.cues.keystrokes).toHaveLength(2);
+ expect(t.cues.keystrokes[0]?.frame).toBeGreaterThan(20);
+ expect(t.cues.done[0]?.frame).toBeGreaterThan(
+ t.cues.submits[0]?.frame ?? 0,
+ );
+ expect(t.span.end).toBe(Number.POSITIVE_INFINITY);
+ });
+
+ it("wraps in a terminal window by default, bare on request", () => {
+ expect(
+ find(make()({ id: "t", script: [{ cmd: "ls" }] }), "t-window"),
+ ).toBeDefined();
+ expect(
+ find(
+ make()({ id: "t", script: [{ cmd: "ls" }], frame: false }),
+ "t-window",
+ ),
+ ).toBeUndefined();
+ });
+
+ it("draws spinners, checks and progress (with a counter) as their own parts", () => {
+ const t = make()({
+ id: "t",
+ script: [
+ { spinner: "Installing", for: 20 },
+ { progress: "Encoding", for: 30 },
+ ],
+ });
+ expect(find(t, "t-spinner-0")).toBeDefined();
+ expect(find(t, "t-check-0")).toBeDefined();
+ expect(find(t, "t-progress-0-percent")).toBeDefined();
+ // The label sits two columns in, clear of the spinner.
+ expect(find(t, "t-line-0")?.x).toBeGreaterThan(0);
+ });
+
+ it("rejects unknown steps", () => {
+ expect(() => make()({ script: [{ run: "ls" } as never] })).toThrow(
+ /invalid props/,
+ );
+ });
+});
diff --git a/packages/components/src/terminal/index.ts b/packages/components/src/terminal/index.ts
new file mode 100644
index 00000000..06c13bc9
--- /dev/null
+++ b/packages/components/src/terminal/index.ts
@@ -0,0 +1,463 @@
+/**
+ * Terminal: a scripted shell session. Commands type with a human rhythm,
+ * output streams in, spinners resolve to check marks, progress bars fill,
+ * and the view scrolls by whole lines, all on cue. Wrapped in an `AppFrame`
+ * terminal window by default.
+ */
+import {
+ type KeyframeTuple,
+ LayerAnimation,
+ type LayoutNode,
+ type TextSpan,
+} from "@framefields/compositions/program";
+import { z } from "zod";
+import { appFrame } from "../app-frame/index.js";
+import { bind, defineComponent } from "../core/contract.js";
+import type { Animatable } from "../core/host.js";
+import { toPathData, type Vec2 } from "../core/path.js";
+import type { Theme } from "../core/theme.js";
+import { counter } from "../counter/index.js";
+import {
+ planTerminal,
+ SCROLL_GLIDE,
+ type Span,
+ type Step,
+ spansText,
+} from "./plan.js";
+
+export type { Step as TerminalStep } from "./plan.js";
+
+export interface TerminalCues {
+ [name: string]: ReadonlyArray<{ frame: number }>;
+ /** Every typed character: for key sounds. */
+ keystrokes: ReadonlyArray<{ frame: number; index: number; step: number }>;
+ /** Enter, per command. */
+ submits: ReadonlyArray<{ frame: number; step: number }>;
+ /** Each output step, first line to last. */
+ outputs: ReadonlyArray<{ frame: number; end: number; step: number }>;
+ /** The script has finished. */
+ done: ReadonlyArray<{ frame: number }>;
+}
+
+/** The 16-colour palette, from theme tokens where they exist. */
+function palette(theme: Theme): Record<string, string> {
+ const c = theme.color;
+ return {
+ black: "#1b1d23",
+ red: c.negative,
+ green: c.positive,
+ yellow: c.highlight,
+ blue: c.accent,
+ magenta: c.accent2,
+ cyan: "#4cc9e0",
+ white: c.ink,
+ gray: c.muted,
+ dim: c.muted,
+ accent: c.accent,
+ };
+}
+
+function stepSchema() {
+ const frames = z.number().int().positive();
+ return z.union([
+ z.object({ cmd: z.string(), prompt: z.string().optional() }).strict(),
+ z.object({ output: z.union([z.string(), z.array(z.string())]) }).strict(),
+ z
+ .object({ spinner: z.string(), for: frames, done: z.string().optional() })
+ .strict(),
+ z.object({ progress: z.string(), for: frames }).strict(),
+ z.object({ pause: frames }).strict(),
+ z.object({ clear: z.literal(true) }).strict(),
+ ]);
+}
+
+export const terminal = /* @__PURE__ */ defineComponent({
+ name: "Terminal",
+ props: () => ({
+ script: z.array(stepSchema()).min(1),
+ prompt: z.string().default("$ "),
+ /** Typing speed, characters a second (seeded ±30 % jitter). */
+ cps: z.number().positive().default(18),
+ /** Output lines a second, or `"instant"`. */
+ outputRate: z
+ .union([z.number().positive(), z.literal("instant")])
+ .default(40),
+ rows: z.number().int().positive().default(16),
+ /** Columns. Default: the widest line, between 40 and 100. */
+ cols: z.number().int().positive().optional(),
+ /** `{green}ok{/}`-style colours (black red green yellow blue magenta cyan white gray dim accent, bold). */
+ markup: z.boolean().default(true),
+ /** Wrap in an AppFrame terminal window (`false` for the bare screen). */
+ frame: z
+ .union([
+ z.boolean(),
+ z
+ .object({
+ title: z.string().optional(),
+ elevation: z.number().int().min(0).max(3).optional(),
+ })
+ .strict(),
+ ])
+ .default(true),
+ caret: z.enum(["block", "bar", "underline"]).default("block"),
+ fontFamily: z.string().optional(),
+ fontSize: z.number().positive().optional(),
+ }),
+ defaults: { enter: false, exit: false },
+ build(ctx, props) {
+ const { host, theme, fps } = ctx;
+ const s = theme.scale;
+ const fs = props.fontSize ?? 24 * s;
+ const fontFamily = props.fontFamily ?? theme.font.mono;
+ const colors = palette(theme);
+ const LINE_HEIGHT = 1.5;
+ const glyph = host.measure("M", {
+ fontFamily,
+ fontSize: fs,
+ lineHeight: LINE_HEIGHT,
+ });
+ const adv = glyph.width;
+ const lh = glyph.lineHeight;
+
+ const plan = planTerminal(props.script as Step[], {
+ prompt: props.prompt,
+ cps: props.cps,
+ outputRate: props.outputRate,
+ rows: props.rows,
+ fps,
+ markup: props.markup,
+ rng: ctx.rng("typing"),
+ });
+ const cols = props.cols ?? Math.min(100, Math.max(40, plan.widest + 2));
+ const pad = fs * 0.9;
+ const viewW = cols * adv + pad * 2;
+ const viewH = props.rows * lh + pad * 2;
+
+ const parts: Record<string, Animatable<LayoutNode>> = {};
+ const content: LayoutNode[] = [];
+ /** A node drawn from `from` until `until` (local frames). */
+ const live = (from: number, until: number) => ({
+ startFrame: from,
+ ...(Number.isFinite(until) && {
+ durationFrames: Math.max(1, until - from),
+ }),
+ });
+ const toSpans = (spans: readonly Span[]): TextSpan[] =>
+ spans.map((sp) => ({
+ text: sp.text,
+ fill: sp.color
+ ? (colors[sp.color] ?? theme.color.ink)
+ : theme.color.ink,
+ ...(sp.bold && { fontWeight: 700 }),
+ }));
+
+ // ── Lines ─────────────────────────────────────────────────────────
+ plan.lines.forEach((line, i) => {
+ const node = host.text(spansText(line.spans), {
+ id: ctx.partId(`line-${i}`),
+ position: "absolute",
+ x: (line.indent ?? 0) * adv,
+ y: line.row * lh,
+ fontFamily,
+ fontSize: fs,
+ lineHeight: LINE_HEIGHT,
+ fill: theme.color.ink,
+ spans: toSpans(line.spans),
+ ...live(line.appear, line.until),
+ });
+ if (line.typing) {
+ const total = spansText(line.spans).length;
+ // The text track is a reveal amount: ≤ 1 a fraction, > 1 a count.
+ const count = (c: number) => (c <= 1 ? c / total : c);
+ const shown = line.typing.prompt;
+ const keys: KeyframeTuple[] = [[0, count(shown)]];
+ line.typing.frames.forEach((f, k) => {
+ keys.push([f - line.appear, count(shown + k + 1), "hold"]);
+ });
+ node.animate(LayerAnimation.keys("text" as never, keys));
+ }
+ content.push(node);
+ });
+
+ // ── Spinners, checks, progress ────────────────────────────────────
+ const cell = (row: number, col: number) => ({ x: col * adv, y: row * lh });
+ plan.spinners.forEach((sp, i) => {
+ const r = fs * 0.32;
+ const c: Vec2 = [adv / 2, lh / 2];
+ const arc: Vec2[] = Array.from({ length: 25 }, (_, k) => {
+ const a = (k / 24) * Math.PI * 1.5;
+ return [c[0] + Math.cos(a) * r, c[1] + Math.sin(a) * r];
+ });
+ const spinner = host.shape("path", {
+ id: ctx.partId(`spinner-${i}`),
+ position: "absolute",
+ ...cell(sp.row, 0),
+ width: adv,
+ height: lh,
+ d: toPathData(arc),
+ fillType: "none",
+ strokeColor: colors.accent,
+ strokeWidth: fs * 0.1,
+ strokeLineCap: "round",
+ ...live(sp.from, sp.to),
+ });
+ spinner.animate(
+ LayerAnimation.keys("rotation", [
+ [0, 0],
+ [sp.to - sp.from, (360 * (sp.to - sp.from)) / (fps * 0.8), "none"],
+ ]),
+ );
+ content.push(spinner);
+ });
+ plan.checks.forEach((ck, i) => {
+ const line = plan.lines.find(
+ (l) => l.kind === "done" && l.row === ck.row && l.appear === ck.at,
+ );
+ const pts: Vec2[] = [
+ [adv * 0.12, lh * 0.52],
+ [adv * 0.4, lh * 0.72],
+ [adv * 0.9, lh * 0.28],
+ ];
+ const check = host.shape("path", {
+ id: ctx.partId(`check-${i}`),
+ position: "absolute",
+ ...cell(ck.row, 0),
+ width: adv,
+ height: lh,
+ d: toPathData(pts),
+ fillType: "none",
+ strokeColor: colors.green,
+ strokeWidth: fs * 0.12,
+ strokeLineCap: "round",
+ strokeLineJoin: "round",
+ trimEnd: 0,
+ ...live(ck.at, line?.until ?? Number.POSITIVE_INFINITY),
+ });
+ check.animate(
+ LayerAnimation.keys("trimEnd", [
+ [0, 0],
+ [Math.max(3, Math.round(fps * 0.2)), 1, "power2.out"],
+ ]),
+ );
+ content.push(check);
+ });
+ const makeCounter = bind(host, counter);
+ plan.progress.forEach((pg, i) => {
+ const line = plan.lines.find(
+ (l) => l.kind === "progress" && l.row === pg.row,
+ );
+ const barCols = 24;
+ const x0 = (pg.label.length + 1) * adv;
+ const barW = barCols * adv;
+ const barH = Math.max(4, lh * 0.32);
+ const y = (lh - barH) / 2;
+ const length = pg.to - pg.from;
+ const fill = host.box({
+ id: ctx.partId(`progress-${i}-fill`),
+ position: "absolute",
+ x: 0,
+ y: 0,
+ width: 0,
+ height: barH,
+ background: colors.accent,
+ borderRadius: barH / 2,
+ });
+ fill.animate(
+ LayerAnimation.keys("width", [
+ [0, 0],
+ [length, barW, "power1.inOut"],
+ ]),
+ );
+ const track = host.box({
+ id: ctx.partId(`progress-${i}-track`),
+ position: "absolute",
+ x: x0,
+ y,
+ width: barW,
+ height: barH,
+ background: theme.color.line,
+ borderRadius: barH / 2,
+ overflow: "hidden",
+ children: [fill],
+ });
+ const percent = makeCounter({
+ id: ctx.partId(`progress-${i}-percent`),
+ to: 100,
+ suffix: "%",
+ style: "count",
+ countDuration: length,
+ ease: "power1.inOut",
+ digitStagger: 0,
+ fontFamily,
+ fontSize: fs,
+ fill: theme.color.muted,
+ x: x0 + barW + adv,
+ y:
+ (lh - host.measure("0", { fontFamily, fontSize: fs }).lineHeight) / 2,
+ theme,
+ });
+ content.push(
+ host.box({
+ id: ctx.partId(`progress-${i}`),
+ position: "absolute",
+ ...cell(pg.row, 0),
+ width: viewW,
+ height: lh,
+ children: [track, percent],
+ ...live(pg.from, line?.until ?? Number.POSITIVE_INFINITY),
+ }),
+ );
+ });
+
+ // ── Caret ─────────────────────────────────────────────────────────
+ const caretW = props.caret === "bar" ? Math.max(2, fs * 0.08) : adv;
+ const caretH =
+ props.caret === "underline" ? Math.max(2, fs * 0.1) : fs * 1.15;
+ const caretDy = props.caret === "underline" ? lh * 0.82 : (lh - caretH) / 2;
+ const xs: KeyframeTuple[] = [];
+ const ys: KeyframeTuple[] = [];
+ const on: KeyframeTuple[] = [];
+ const half = Math.max(1, Math.round(fps * 0.5));
+ plan.caret.forEach(([f, at], k) => {
+ if (at) {
+ xs.push([f, at.col * adv, "hold"]);
+ ys.push([f, at.row * lh + caretDy, "hold"]);
+ }
+ on.push([f, at ? 1 : 0, "hold"]);
+ // Idle with the caret up: blink until the next change.
+ const next = plan.caret[k + 1]?.[0] ?? f + fps * 6;
+ if (at)
+ for (
+ let b = f + half * 2, shown = false;
+ b < next;
+ b += half, shown = !shown
+ )
+ on.push([b, shown ? 1 : 0, "hold"]);
+ });
+ const caret = host.box({
+ id: ctx.partId("caret"),
+ position: "absolute",
+ x: 0,
+ y: 0,
+ width: caretW,
+ height: caretH,
+ background: theme.color.ink,
+ opacity: 0,
+ });
+ caret.animate(
+ LayerAnimation.create()
+ .keys("x", ascending(xs))
+ .keys("y", ascending(ys))
+ .keys("opacity", ascending(on)),
+ );
+ content.push(caret);
+ parts.caret = caret;
+
+ // ── Screen and scroll ─────────────────────────────────────────────
+ // Whole lines at a time: glide on new output, snap on clear. A scroll
+ // that arrives mid-glide retargets the glide instead of stacking.
+ const scrollY = (top: number) => pad - top * lh;
+ const scrollKeys: KeyframeTuple[] = [
+ [0, scrollY(plan.scroll[0]?.[1] ?? 0)],
+ ];
+ for (const [f, top, jump] of plan.scroll.slice(1)) {
+ const last = scrollKeys[scrollKeys.length - 1] as KeyframeTuple;
+ if (jump)
+ scrollKeys.push([Math.max(f, last[0] + 1), scrollY(top), "hold"]);
+ else if (last[0] > f && last[2] !== "hold") {
+ scrollKeys[scrollKeys.length - 1] = [
+ f + SCROLL_GLIDE,
+ scrollY(top),
+ last[2] as string,
+ ];
+ } else {
+ scrollKeys.push([Math.max(f, last[0] + 1), last[1]]);
+ scrollKeys.push([
+ Math.max(f, last[0] + 1) + SCROLL_GLIDE,
+ scrollY(top),
+ "power2.out",
+ ]);
+ }
+ }
+ const lines = host.box({
+ id: ctx.partId("lines"),
+ position: "absolute",
+ x: pad,
+ y: pad,
+ width: cols * adv,
+ // Every row, not just a screenful: children outside a box are culled.
+ height: (Math.max(0, ...plan.lines.map((l) => l.row)) + 1) * lh,
+ children: content,
+ });
+ lines.animate(LayerAnimation.keys("y", ascending(scrollKeys)));
+ const screen = host.box({
+ id: ctx.partId("screen"),
+ position: "absolute",
+ x: 0,
+ y: 0,
+ width: viewW,
+ height: viewH,
+ overflow: "hidden",
+ ...(props.frame === false && {
+ background: theme.color.surface,
+ borderRadius: theme.radius.md,
+ }),
+ children: [lines],
+ });
+ parts.lines = lines;
+ parts.screen = screen;
+
+ let root: LayoutNode = screen;
+ let width = viewW;
+ let height = viewH;
+ if (props.frame !== false) {
+ const o = props.frame === true ? {} : props.frame;
+ const bar = 40 * s;
+ const frameNode = bind(
+ host,
+ appFrame,
+ )({
+ id: ctx.partId("window"),
+ kind: "terminal",
+ width: viewW,
+ height: viewH + bar,
+ content: [screen],
+ contentBackground: theme.color.surface,
+ enter: false,
+ theme,
+ ...(o.title !== undefined && { title: o.title }),
+ ...(o.elevation !== undefined && { elevation: o.elevation }),
+ });
+ parts.window = frameNode;
+ root = frameNode;
+ height = viewH + bar;
+ width = viewW;
+ }
+
+ return {
+ children: [root],
+ width,
+ height,
+ duration: plan.done + 1,
+ openEnded: true,
+ cues: {
+ keystrokes: plan.keystrokes,
+ submits: plan.submits,
+ outputs: plan.outputs,
+ done: [{ frame: plan.done }],
+ } as TerminalCues,
+ parts,
+ };
+ },
+});
+
+/** Keys must ascend: a later key at the same frame replaces the earlier one. */
+function ascending(keys: KeyframeTuple[]): KeyframeTuple[] {
+ const out: KeyframeTuple[] = [];
+ for (const k of keys) {
+ while (out.length > 0 && (out[out.length - 1] as KeyframeTuple)[0] >= k[0])
+ out.pop();
+ out.push(k);
+ }
+ return out.length > 0 ? out : [[0, 0]];
+}
diff --git a/packages/components/src/terminal/meta.ts b/packages/components/src/terminal/meta.ts
new file mode 100644
index 00000000..2435941d
--- /dev/null
+++ b/packages/components/src/terminal/meta.ts
@@ -0,0 +1,12 @@
+import { defineMeta } from "../core/meta.js";
+
+export default defineMeta({
+ name: "Terminal",
+ summary:
+ "A scripted shell session: commands typed with a human rhythm, streaming output, spinners that resolve to check marks, progress bars and scrolling, with a cue for every keystroke.",
+ tags: ["terminal", "shell", "cli", "developer", "demo", "typing"],
+ parts: ["window", "screen", "lines", "caret"],
+ cues: ["keystrokes", "submits", "outputs", "done"],
+ example: `Terminal({ script: [{ cmd: "pnpm add framefields" }, { spinner: "Installing", for: 40, done: "1 package added" }] })`,
+ since: "M3",
+});
diff --git a/packages/components/src/terminal/plan.test.ts b/packages/components/src/terminal/plan.test.ts
new file mode 100644
index 00000000..7c4d7b82
--- /dev/null
+++ b/packages/components/src/terminal/plan.test.ts
@@ -0,0 +1,79 @@
+import { describe, expect, it } from "vitest";
+import { rng } from "../core/rng.js";
+import { parseMarkup, planTerminal, type Step } from "./plan.js";
+
+const opts = {
+ prompt: "$ ",
+ cps: 18,
+ outputRate: 40 as const,
+ rows: 4,
+ fps: 30,
+ markup: true,
+ rng: rng(1),
+};
+const plan = (script: Step[], o = {}) =>
+ planTerminal(script, { ...opts, rng: rng(1), ...o });
+
+describe("terminal markup", () => {
+ it("maps colour tags to spans and leaves unknown tags alone", () => {
+ expect(parseMarkup("{green}ok{/} done {nope}")).toEqual([
+ { text: "ok", color: "green" },
+ { text: " done {nope}" },
+ ]);
+ expect(parseMarkup("{bold}{red}x{/}{/}")).toEqual([
+ { text: "x", bold: true, color: "red" },
+ ]);
+ expect(parseMarkup("{red}x", false)).toEqual([{ text: "{red}x" }]);
+ });
+});
+
+describe("terminal plan", () => {
+ it("types each command, then submits, then streams output", () => {
+ const p = plan([{ cmd: "ls" }, { output: ["a", "b"] }]);
+ expect(p.keystrokes).toHaveLength(2);
+ const submit = p.submits[0]?.frame ?? 0;
+ expect(submit).toBeGreaterThan(p.keystrokes[1]?.frame ?? 0);
+ expect(p.outputs[0]?.frame).toBeGreaterThan(submit);
+ expect(p.outputs[0]?.end).toBeGreaterThan(p.outputs[0]?.frame ?? 0);
+ });
+
+ it("is deterministic per seed", () => {
+ const a = plan([{ cmd: "pnpm install" }]).keystrokes;
+ expect(plan([{ cmd: "pnpm install" }]).keystrokes).toEqual(a);
+ expect(
+ planTerminal([{ cmd: "pnpm install" }], { ...opts, rng: rng(2) })
+ .keystrokes,
+ ).not.toEqual(a);
+ });
+
+ it("scrolls by whole lines once the screen is full, and retires scrolled lines", () => {
+ const p = plan([
+ { output: Array.from({ length: 10 }, (_, i) => `line ${i}`) },
+ ]);
+ const tops = p.scroll.map(([, top]) => top);
+ expect(tops.every((t) => Number.isInteger(t))).toBe(true);
+ expect(tops[tops.length - 1]).toBe(p.lines.length - opts.rows);
+ // Never more than a screenful plus the lines gliding out (spec budget: rows + 6).
+ for (let f = 0; f <= p.done + 10; f++) {
+ const live = p.lines.filter((l) => l.appear <= f && f < l.until).length;
+ expect(live).toBeLessThanOrEqual(opts.rows + 6);
+ }
+ });
+
+ it("resolves spinners to a check mark on the same row", () => {
+ const p = plan([{ spinner: "Installing", for: 30, done: "✔ installed" }]);
+ const [spin] = p.spinners;
+ const done = p.lines.find((l) => l.kind === "done");
+ expect(done?.row).toBe(spin?.row);
+ expect(done?.appear).toBe(spin?.to);
+ expect(done?.spans.map((s) => s.text).join("")).toBe("installed");
+ expect(p.checks[0]?.at).toBe(spin?.to);
+ });
+
+ it("clears the screen at once", () => {
+ const p = plan([{ output: "a" }, { clear: true }, { output: "b" }]);
+ const a = p.lines.find((l) => l.spans[0]?.text === "a");
+ const jump = p.scroll.find(([, , j]) => j);
+ expect(a?.until).toBe(jump?.[0]);
+ });
+});
diff --git a/packages/components/src/terminal/plan.ts b/packages/components/src/terminal/plan.ts
new file mode 100644
index 00000000..a340ca47
--- /dev/null
+++ b/packages/components/src/terminal/plan.ts
@@ -0,0 +1,288 @@
+/**
+ * Terminal planning, kept pure: run the script on a clock and record what
+ * appears where and when. Nodes are built from this plan; tests read it.
+ */
+import type { Rng } from "../core/rng.js";
+import { typingFrames } from "../core/typing.js";
+
+export type Step =
+ | { cmd: string; prompt?: string }
+ | { output: string | string[] }
+ | { spinner: string; for: number; done?: string }
+ | { progress: string; for: number }
+ | { pause: number }
+ | { clear: true };
+
+/** A run of text in one colour (a palette name) and weight. */
+export interface Span {
+ text: string;
+ color?: string;
+ bold?: boolean;
+}
+
+export interface PlannedLine {
+ row: number;
+ /** Frame the line appears. */
+ appear: number;
+ /** Frame it stops being drawn (scrolled out or cleared); Infinity if never. */
+ until: number;
+ spans: Span[];
+ step: number;
+ kind: "cmd" | "output" | "spinner" | "done" | "progress" | "prompt";
+ /** Columns the text starts in (room for a spinner or check mark). */
+ indent?: number;
+ /** For commands: characters already shown (the prompt) and each keystroke's frame. */
+ typing?: { prompt: number; frames: number[] };
+}
+
+export interface TerminalPlan {
+ lines: PlannedLine[];
+ /** Rotating spinners: row, from frame, to frame. */
+ spinners: Array<{ row: number; from: number; to: number; step: number }>;
+ /** Check marks drawn at `at` on `row`, before the text from column 2. */
+ checks: Array<{ row: number; at: number; step: number }>;
+ progress: Array<{
+ row: number;
+ from: number;
+ to: number;
+ label: string;
+ step: number;
+ }>;
+ /** First visible row from each frame on; `jump` (a clear) snaps instead of gliding. */
+ scroll: Array<[frame: number, top: number, jump?: boolean]>;
+ /** Where the caret is, from each frame on (null = hidden). */
+ caret: Array<[frame: number, at: { row: number; col: number } | null]>;
+ keystrokes: Array<{ frame: number; index: number; step: number }>;
+ submits: Array<{ frame: number; step: number }>;
+ outputs: Array<{ frame: number; end: number; step: number }>;
+ done: number;
+ /** Longest line, in characters. */
+ widest: number;
+}
+
+export interface PlanOptions {
+ prompt: string;
+ cps: number;
+ /** Lines a second, or every output line at once. */
+ outputRate: number | "instant";
+ rows: number;
+ fps: number;
+ markup: boolean;
+ rng: Rng;
+}
+
+const COLORS = new Set([
+ "black",
+ "red",
+ "green",
+ "yellow",
+ "blue",
+ "magenta",
+ "cyan",
+ "white",
+ "gray",
+ "dim",
+ "accent",
+]);
+
+/** `{green}ok{/} done` → spans. `{bold}` sets weight; unknown tags stay literal. */
+export function parseMarkup(source: string, markup = true): Span[] {
+ if (!markup) return [{ text: source }];
+ const spans: Span[] = [];
+ const stack: Array<{ color?: string; bold?: boolean }> = [{}];
+ let buffer = "";
+ const flush = () => {
+ if (!buffer) return;
+ const top = stack[stack.length - 1] ?? {};
+ spans.push({ text: buffer, ...top });
+ buffer = "";
+ };
+ const re = /\{(\/|[a-z]+)\}/g;
+ let last = 0;
+ for (const m of source.matchAll(re)) {
+ const tag = m[1] as string;
+ const index = m.index ?? 0;
+ buffer += source.slice(last, index);
+ last = index + m[0].length;
+ if (tag === "/") {
+ flush();
+ if (stack.length > 1) stack.pop();
+ } else if (tag === "bold" || COLORS.has(tag)) {
+ flush();
+ const top = stack[stack.length - 1] ?? {};
+ stack.push(
+ tag === "bold" ? { ...top, bold: true } : { ...top, color: tag },
+ );
+ } else buffer += m[0];
+ }
+ buffer += source.slice(last);
+ flush();
+ return spans;
+}
+
+export const spansText = (spans: readonly Span[]) =>
+ spans.map((s) => s.text).join("");
+
+/** Frames between Enter and the first line of output. */
+const SUBMIT_BEAT = 0.3;
+/** Frames the scroll takes to glide one or more lines. */
+export const SCROLL_GLIDE = 4;
+/** A check mark replaces these leading glyphs (fonts often lack them). */
+const DONE_MARK = /^\s*[✔✓√]\s*/;
+
+export function planTerminal(
+ script: readonly Step[],
+ o: PlanOptions,
+): TerminalPlan {
+ const plan: TerminalPlan = {
+ lines: [],
+ spinners: [],
+ checks: [],
+ progress: [],
+ scroll: [[0, 0]],
+ caret: [[0, null]],
+ keystrokes: [],
+ submits: [],
+ outputs: [],
+ done: 0,
+ widest: 0,
+ };
+ let t = 0;
+ let row = 0;
+ let top = 0;
+ let typed = 0;
+ const perLine = o.outputRate === "instant" ? 0 : o.fps / o.outputRate;
+
+ const show = (line: Omit<PlannedLine, "until" | "row">): PlannedLine => {
+ if (row - top >= o.rows) {
+ top = row - o.rows + 1;
+ plan.scroll.push([line.appear, top]);
+ }
+ const planned = { ...line, row, until: Number.POSITIVE_INFINITY };
+ plan.lines.push(planned);
+ plan.widest = Math.max(
+ plan.widest,
+ spansText(line.spans).length + (line.indent ?? 0),
+ );
+ row++;
+ return planned;
+ };
+ const caretAt = (frame: number, at: { row: number; col: number } | null) =>
+ plan.caret.push([frame, at]);
+
+ script.forEach((step, s) => {
+ if ("cmd" in step) {
+ const prompt = step.prompt ?? o.prompt;
+ const line = show({
+ appear: t,
+ spans: [{ text: prompt, color: "accent" }, { text: step.cmd }],
+ step: s,
+ kind: "cmd",
+ });
+ const frames = typingFrames(
+ step.cmd,
+ o.cps,
+ o.fps,
+ o.rng,
+ t + Math.round(o.fps * 0.25),
+ );
+ line.typing = { prompt: prompt.length, frames };
+ caretAt(t, { row: line.row, col: prompt.length });
+ frames.forEach((f, i) => {
+ plan.keystrokes.push({ frame: f, index: typed++, step: s });
+ caretAt(f, { row: line.row, col: prompt.length + i + 1 });
+ });
+ const submit =
+ (frames[frames.length - 1] ?? t) + Math.round(o.fps * SUBMIT_BEAT);
+ plan.submits.push({ frame: submit, step: s });
+ caretAt(submit, null);
+ t = submit + 1;
+ } else if ("output" in step) {
+ const lines = Array.isArray(step.output)
+ ? step.output
+ : step.output.split("\n");
+ const first = Math.round(t);
+ let at = t;
+ for (const text of lines) {
+ show({
+ appear: Math.round(at),
+ spans: parseMarkup(text, o.markup),
+ step: s,
+ kind: "output",
+ });
+ at += perLine;
+ }
+ plan.outputs.push({
+ frame: first,
+ end: Math.round(Math.max(t, at - perLine)),
+ step: s,
+ });
+ t = Math.round(at);
+ } else if ("spinner" in step) {
+ const from = Math.round(t);
+ const to = from + step.for;
+ const spinning = show({
+ appear: from,
+ spans: [{ text: step.spinner }],
+ indent: 2,
+ step: s,
+ kind: "spinner",
+ });
+ spinning.until = to;
+ plan.spinners.push({ row: spinning.row, from, to, step: s });
+ row--;
+ const doneText = (step.done ?? step.spinner).replace(DONE_MARK, "");
+ show({
+ appear: to,
+ spans: parseMarkup(doneText, o.markup),
+ indent: 2,
+ step: s,
+ kind: "done",
+ });
+ plan.checks.push({ row: spinning.row, at: to, step: s });
+ t = to;
+ } else if ("progress" in step) {
+ const from = Math.round(t);
+ const line = show({
+ appear: from,
+ spans: [{ text: step.progress }],
+ step: s,
+ kind: "progress",
+ });
+ plan.progress.push({
+ row: line.row,
+ from,
+ to: from + step.for,
+ label: step.progress,
+ step: s,
+ });
+ plan.widest = Math.max(plan.widest, step.progress.length + 32);
+ t = from + step.for;
+ } else if ("pause" in step) {
+ t += step.pause;
+ } else {
+ const at = Math.round(t);
+ for (const line of plan.lines) line.until = Math.min(line.until, at);
+ top = row;
+ plan.scroll.push([at, top, true]);
+ }
+ });
+
+ plan.done = Math.round(t);
+ // A fresh prompt waits at the end.
+ const last = show({
+ appear: plan.done,
+ spans: [{ text: o.prompt, color: "accent" }],
+ step: script.length,
+ kind: "prompt",
+ });
+ caretAt(plan.done, { row: last.row, col: o.prompt.length });
+
+ // Lines stop drawing once scrolled out (after the glide), so only a
+ // screenful of nodes is ever live.
+ for (const line of plan.lines) {
+ const gone = plan.scroll.find(([, top]) => top > line.row);
+ if (gone) line.until = Math.min(line.until, gone[0] + SCROLL_GLIDE);
+ }
+ return plan;
+}
diff --git a/packages/components/src/test/fake-host.ts b/packages/components/src/test/fake-host.ts
index d3b713a0..bf355c39 100644
--- a/packages/components/src/test/fake-host.ts
+++ b/packages/components/src/test/fake-host.ts
@@ -115,6 +115,14 @@ export function fakeHost(): Host & { created: LayoutNode[] } {
];
return node;
},
+ cutout: (source, o) =>
+ make({
+ kind: "media",
+ id: o.id ?? "cutout",
+ source,
+ cutout: o.mode ?? "person",
+ ...o,
+ }) as never,
matte: (node, matte) => {
(node as unknown as { matte?: unknown }).matte = { node: matte };
return node;
diff --git a/packages/components/src/text-reveal/index.ts b/packages/components/src/text-reveal/index.ts
index 58c2d6b3..e63e9b2f 100644
--- a/packages/components/src/text-reveal/index.ts
+++ b/packages/components/src/text-reveal/index.ts
@@ -22,6 +22,7 @@ import {
type GlyphProps,
type GlyphUnit,
glyphAnimator,
+ rendererChars,
scheduleExpression,
type UnitSchedule,
} from "../core/glyphs.js";
@@ -35,11 +36,11 @@ import {
} from "../core/text.js";
import { type Theme, themeFrames } from "../core/theme.js";
import { scaleFrames } from "../core/timing.js";
+import { typingFrames } from "../core/typing.js";
import {
burstStarts,
indexedStarts,
scrambleSet,
- typingFrames,
unitStarts,
unitsOf,
} from "./plan.js";
@@ -304,9 +305,13 @@ export const textReveal = /* @__PURE__ */ defineComponent({
const emphasised = emphasisWeights(text, props.emphasis);
if (emphasised) {
animators.push(
- glyphAnimator("emphasis", constantExpression("char", emphasised), {
- fillColor: props.accent ?? theme.color.accent,
- }),
+ glyphAnimator(
+ "emphasis",
+ constantExpression("char", rendererChars(text, emphasised)),
+ {
+ fillColor: props.accent ?? theme.color.accent,
+ },
+ ),
);
}
@@ -437,7 +442,10 @@ export const textReveal = /* @__PURE__ */ defineComponent({
const sched = schedule(starts);
const per = (key: string) => {
const q = ctx.rng(key);
- return Array.from({ length: text.length }, () => q.spread(1));
+ return rendererChars(
+ text,
+ Array.from({ length: text.length }, () => q.spread(1)),
+ );
};
const rx = per("scatter-x");
const ry = per("scatter-y");
@@ -597,7 +605,7 @@ export const textReveal = /* @__PURE__ */ defineComponent({
host.matte(textNode, w.matte);
parts.matte = w.matte;
// The front trails a little blur over the glyphs it is uncovering.
- const xs = glyphCenters(text, m, measureOptions);
+ const xs = rendererChars(text, glyphCenters(text, m, measureOptions));
const front = Array.from({ length: duration + 1 }, (_, f) =>
w.front(
w.hidden + (w.shown - w.hidden) * evaluateEase(ease, f / duration),
diff --git a/packages/components/src/text-reveal/plan.test.ts b/packages/components/src/text-reveal/plan.test.ts
index c860a7dd..d4b19321 100644
--- a/packages/components/src/text-reveal/plan.test.ts
+++ b/packages/components/src/text-reveal/plan.test.ts
@@ -1,13 +1,8 @@
import { describe, expect, it } from "vitest";
import { rng } from "../core/rng.js";
+import { typingFrames } from "../core/typing.js";
import { fakeMeasure } from "../test/fake-host.js";
-import {
- burstStarts,
- indexedStarts,
- scrambleSet,
- typingFrames,
- unitsOf,
-} from "./plan.js";
+import { burstStarts, indexedStarts, scrambleSet, unitsOf } from "./plan.js";
const m = (text: string, width?: number) =>
fakeMeasure(text, { fontFamily: "Inter", fontSize: 20, width });
diff --git a/packages/components/src/text-reveal/plan.ts b/packages/components/src/text-reveal/plan.ts
index 3ad07e7d..5a22f563 100644
--- a/packages/components/src/text-reveal/plan.ts
+++ b/packages/components/src/text-reveal/plan.ts
@@ -1,8 +1,8 @@
/**
* TextReveal timing, kept pure: which characters form each unit, when each
- * unit starts, the typing rhythm and the streaming bursts.
+ * unit starts, the streaming bursts (typing rhythm lives in `core/typing.ts`).
*/
-import type { GlyphUnit } from "../core/glyphs.js";
+import { type GlyphUnit, rendererChars } from "../core/glyphs.js";
import type { Rng } from "../core/rng.js";
import { type Metrics, wordRanges } from "../core/text.js";
import { type StaggerOrder, staggerRanks } from "../core/timing.js";
@@ -78,6 +78,7 @@ export function indexedStarts(
starts: readonly number[],
): number[] {
if (by !== "char") return [...starts];
+ // Built per string index, returned in the renderer's index order.
const out = new Array<number>(text.length).fill(0);
let next = starts[starts.length - 1] ?? 0;
for (let u = units.length - 1, i = text.length - 1; i >= 0; i--) {
@@ -88,36 +89,7 @@ export function indexedStarts(
}
out[i] = next;
}
- return out;
-}
-
-/**
- * Keystroke frames for typing at `cps` characters a second: seeded ±30 %
- * jitter, a beat after spaces, a longer one after punctuation. Frame `k[i]`
- * is when character `i` appears.
- */
-export function typingFrames(
- text: string,
- cps: number,
- fps: number,
- r: Rng,
- start = 0,
-): number[] {
- const per = fps / cps;
- const out: number[] = [];
- let t = start;
- for (let i = 0; i < text.length; i++) {
- const prev = text[i - 1] ?? "";
- let gap = i === 0 ? 0 : per * (1 + r.spread(0.3));
- if (prev === " ") gap += per * 0.6;
- if (/[.,!?;:]/.test(prev)) gap += per * 3;
- t += gap;
- out.push(Math.max(start, Math.round(t)));
- }
- // Two characters never land on one frame.
- for (let i = 1; i < out.length; i++)
- out[i] = Math.max(out[i] as number, (out[i - 1] as number) + 1);
- return out;
+ return rendererChars(text, out);
}
/**
diff --git a/packages/framefields/scripts/check-components-treeshake.mjs b/packages/framefields/scripts/check-components-treeshake.mjs
index 4fdb6459..1022e915 100644
--- a/packages/framefields/scripts/check-components-treeshake.mjs
+++ b/packages/framefields/scripts/check-components-treeshake.mjs
@@ -23,6 +23,8 @@ const REVEAL = "is not registered. Registered:";
const COUNTER = "Counter: unsupported format";
const ANNOTATE = "Moving targets arrive with Callout";
const GLYPHS = "L[k+1]-L[k]";
+const CAPTIONS = "give `words` or a `transcript`";
+const SHELL = "spinner-";
/** [names imported, markers that must not survive, max added bytes]. */
const cases = [
@@ -33,14 +35,41 @@ const cases = [
// Each component alone: none of the others.
[
["MousePointer"],
- [FRAME, CAMERA, REVEAL, COUNTER, ANNOTATE, GLYPHS],
+ [FRAME, CAMERA, REVEAL, COUNTER, ANNOTATE, GLYPHS, CAPTIONS, SHELL],
60_000,
],
- [["AppFrame"], [POINTER, CAMERA, REVEAL, COUNTER, ANNOTATE, GLYPHS], 45_000],
- [["FocusZoom"], [POINTER, FRAME, REVEAL, COUNTER, ANNOTATE, GLYPHS], 40_000],
- [["TextReveal"], [POINTER, FRAME, CAMERA, COUNTER, ANNOTATE], 55_000],
- [["Counter"], [POINTER, FRAME, CAMERA, REVEAL, ANNOTATE, GLYPHS], 45_000],
- [["Annotate"], [POINTER, FRAME, CAMERA, REVEAL, COUNTER, GLYPHS], 50_000],
+ [
+ ["AppFrame"],
+ [POINTER, CAMERA, REVEAL, COUNTER, ANNOTATE, GLYPHS, CAPTIONS, SHELL],
+ 45_000,
+ ],
+ [
+ ["FocusZoom"],
+ [POINTER, FRAME, REVEAL, COUNTER, ANNOTATE, GLYPHS, CAPTIONS, SHELL],
+ 40_000,
+ ],
+ [
+ ["TextReveal"],
+ [POINTER, FRAME, CAMERA, COUNTER, ANNOTATE, CAPTIONS, SHELL],
+ 55_000,
+ ],
+ [
+ ["Counter"],
+ [POINTER, FRAME, CAMERA, REVEAL, ANNOTATE, GLYPHS, CAPTIONS, SHELL],
+ 45_000,
+ ],
+ [
+ ["Annotate"],
+ [POINTER, FRAME, CAMERA, REVEAL, COUNTER, GLYPHS, CAPTIONS, SHELL],
+ 50_000,
+ ],
+ [
+ ["Captions"],
+ [POINTER, FRAME, CAMERA, REVEAL, COUNTER, ANNOTATE, SHELL],
+ 55_000,
+ ],
+ // Terminal composes AppFrame and Counter on purpose.
+ [["Terminal"], [POINTER, CAMERA, REVEAL, ANNOTATE, GLYPHS, CAPTIONS], 80_000],
];
async function bundle(contents) {
@@ -73,6 +102,8 @@ for (const marker of [
COUNTER,
ANNOTATE,
GLYPHS,
+ CAPTIONS,
+ SHELL,
]) {
if (base.includes(marker))
failures.push(`import { Layer } alone kept component code ("${marker}")`);
diff --git a/packages/framefields/src/components/host.ts b/packages/framefields/src/components/host.ts
index 42c11fb0..bce5628d 100644
--- a/packages/framefields/src/components/host.ts
+++ b/packages/framefields/src/components/host.ts
@@ -39,6 +39,21 @@ export const host: Host = {
(node as unknown as AnimatableNode).withEffect(layerStyle(config));
return node;
},
+ cutout: (source, { mode = "person", ...options }) =>
+ Layer.video(source, {
+ muted: true,
+ volume: 0,
+ fit: "cover",
+ ...options,
+ }).withVision({
+ mode: "matte",
+ enableSegmentation: true,
+ // The larger model separates a subject (not just people) more cleanly.
+ variant: mode === "subject" ? "m" : "s",
+ confidence: 0.25,
+ featherRadius: 0.04,
+ keyBackground: true,
+ }),
matte: (node, matte) => {
(node as unknown as AnimatableNode).withMatte(matte, "alpha");
return node;
diff --git a/packages/framefields/src/components/index.ts b/packages/framefields/src/components/index.ts
index 032da361..996d2b9b 100644
--- a/packages/framefields/src/components/index.ts
+++ b/packages/framefields/src/components/index.ts
@@ -16,9 +16,11 @@ import {
type ComponentDefinition,
type ComponentExtras,
type Cues,
+ captions,
counter,
focusZoom,
mousePointer,
+ terminal,
textReveal,
} from "@framefields/components";
import type { BoxNode } from "@framefields/compositions/program";
@@ -62,6 +64,12 @@ export const Counter = /* @__PURE__ */ bindComponent(counter);
/** Marks that draw on to point at something: underline, circle, arrow, highlight … hand-drawn or clean. */
export const Annotate = /* @__PURE__ */ bindComponent(annotate);
+/** Word-timed captions in published styles, each word on its exact frame, optionally behind the speaker. */
+export const Captions = /* @__PURE__ */ bindComponent(captions);
+
+/** A scripted shell session: typed commands, streaming output, spinners, progress and scrolling. */
+export const Terminal = /* @__PURE__ */ bindComponent(terminal);
+
export {
type Anchor,
type AnnotateCues,
@@ -86,6 +94,9 @@ export {
boilKeys,
boundsOf,
boxPath,
+ CAPTION_STYLES,
+ type CaptionStyle,
+ type CaptionsCues,
type ComponentDefaults,
type ComponentDefinition,
type ComponentExtras,
@@ -97,6 +108,7 @@ export {
type Cues,
type CuesOf,
type CursorStyle,
+ type CutoutOptions,
catmullRom,
chain,
constantExpression,
@@ -167,8 +179,11 @@ export {
squigglePath,
staggerOffsets,
staggerRanks,
+ stepExpression,
type Target,
type TargetContext,
+ type TerminalCues,
+ type TerminalStep,
type TextRevealApi,
type TextRevealCues,
type Theme,
@@ -180,10 +195,13 @@ export {
toFrames,
toPathData,
toSeconds,
+ typingFrames,
type UnitSchedule,
type Vec2,
type Waypoint,
+ type Word,
type WordRange,
+ windowExpression,
within,
wobblePoints,
wordRanges,
diff --git a/packages/framefields/src/components/m3.render.test.ts b/packages/framefields/src/components/m3.render.test.ts
new file mode 100644
index 00000000..3d0f1acb
--- /dev/null
+++ b/packages/framefields/src/components/m3.render.test.ts
@@ -0,0 +1,100 @@
+import { beforeAll, describe, expect, it } from "vitest";
+import { FontManager } from "../fonts/index.js";
+import { Composition } from "../index.js";
+import { computeMse, decodePngPixels } from "../test-utils/pixels.js";
+import { Captions, Terminal } from "./index.js";
+
+/**
+ * Render tests for the M3 components (spec §7.3, §7.8): word colour changes
+ * on its exact frame, and frames don't depend on render order.
+ */
+beforeAll(async () => {
+ await FontManager.register("assets/fonts/Inter.ttf");
+ await FontManager.register("assets/fonts/JetBrainsMono.ttf");
+});
+
+const W = 960;
+const H = 540;
+function comp(nodes: Parameters<Composition["add"]>[0]) {
+ const c = new Composition({
+ width: W,
+ height: H,
+ fps: 30,
+ durationMs: 8000,
+ backgroundColor: "#202833",
+ });
+ c.add(nodes);
+ return c;
+}
+const frame = async (c: Composition, n: number) =>
+ decodePngPixels(await c.renderFrame({ frame: n }));
+const words = "one two three"
+ .split(" ")
+ .map((text, i) => ({ text, start: 0.5 + i, end: 1.3 + i }));
+
+describe("Captions (render)", () => {
+ it("karaoke: a word changes colour on round(start × fps), not a frame before", async () => {
+ const c = comp(Captions({ words, style: "karaoke", width: W, height: H }));
+ const at = Math.round((words[1]?.start ?? 0) * 30); // word 2 starts at 1.5 s
+ expect(at).toBe(45);
+ const before = await frame(c, at - 2);
+ const justBefore = await frame(c, at - 1);
+ const on = await frame(c, at);
+ expect(computeMse(before.pixels, justBefore.pixels)).toBe(0);
+ expect(computeMse(justBefore.pixels, on.pixels)).toBeGreaterThan(0);
+ });
+
+ it("renders the same frame in any order", async () => {
+ const build = () =>
+ comp(Captions({ words, style: "highlight", width: W, height: H }));
+ const frames = [20, 47, 80];
+ const forward = await Promise.all(frames.map((f) => frame(build(), f)));
+ const c = build();
+ for (const [i, f] of [...frames].reverse().entries()) {
+ const img = await frame(c, f);
+ expect(
+ computeMse(
+ img.pixels,
+ (forward[frames.length - 1 - i] as typeof img).pixels,
+ ),
+ ).toBe(0);
+ }
+ });
+});
+
+describe("Terminal (render)", () => {
+ const script = [
+ { cmd: "pnpm add framefields" },
+ { spinner: "Resolving", for: 20, done: "1 package added" },
+ { output: Array.from({ length: 14 }, (_, i) => `line ${i}`) },
+ ];
+
+ it("renders the same frame in any order (typing, spinner, scroll)", async () => {
+ const build = () =>
+ comp(Terminal({ script, rows: 6, x: 40, y: 40, seed: 3 }));
+ const done =
+ Terminal({ script, rows: 6, seed: 3 }).cues.done[0]?.frame ?? 100;
+ const frames = [15, 60, done - 2, done + 5];
+ const forward = await Promise.all(frames.map((f) => frame(build(), f)));
+ const c = build();
+ for (const [i, f] of [...frames].reverse().entries()) {
+ const img = await frame(c, f);
+ expect(
+ computeMse(
+ img.pixels,
+ (forward[frames.length - 1 - i] as typeof img).pixels,
+ ),
+ ).toBe(0);
+ }
+ });
+
+ it("holds still once the script is done (bar the caret blink)", async () => {
+ const t = Terminal({ script, rows: 6, x: 40, y: 40, caret: "bar" });
+ const done = t.cues.done[0]?.frame ?? 0;
+ const c = comp(t);
+ // Two frames inside one solid caret phase after the scroll has settled.
+ const a = await frame(c, done + 6);
+ const b = await frame(c, done + 12);
+ expect(computeMse(a.pixels, b.pixels)).toBe(0);
+ });
+});
diff --git a/packages/framefields/src/index.ts b/packages/framefields/src/index.ts
index d19bb783..944465c7 100644
--- a/packages/framefields/src/index.ts
+++ b/packages/framefields/src/index.ts
@@ -3404,6 +3404,10 @@ export {
type AppFrameCues,
type AppFrameKind,
bindComponent,
+ CAPTION_STYLES,
+ type CaptionStyle,
+ Captions,
+ type CaptionsCues,
type ComponentNode,
Counter,
type CounterCues,
@@ -3418,6 +3422,9 @@ export {
type MousePointerApi,
type MousePointerCues,
type RevealStyle,
+ Terminal,
+ type TerminalCues,
+ type TerminalStep,
TextReveal,
type TextRevealApi,
type TextRevealCues,