Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

feat: decide a caption by its words, and declare what draws one

rpone33254cf7d951
diff --git a/.agents/skills/hypit/references/playbooks/craft/captions.md b/.agents/skills/hypit/references/playbooks/craft/captions.md
index cd6394b0..ef4166c4 100644
--- a/.agents/skills/hypit/references/playbooks/craft/captions.md
+++ b/.agents/skills/hypit/references/playbooks/craft/captions.md
@@ -3,6 +3,34 @@
Captions are one program-wide interpretation of the authoritative Script, measured against the real
speech program. Do not create separate caption systems for speakers, regions, or styles.
+## What is a caption is decided by the words, not by the styling
+
+**Text that arrives with speech and reads exactly what is being said is a caption.** That is the
+whole test, and it is checkable rather than a judgement: the words match the transcript verbatim, and
+they appear as those words are spoken. A reference video's word-level transcript is prepared for
+exactly this comparison.
+
+Nothing about how it is drawn changes the answer. A caption may be set in two typefaces, sit on a
+coloured slab, arrive a word at a time, be enormous, be decorative, or open the video — and it is
+still a caption. Handsome styling is the commonest reason this gets misfiled, and it is not a reason
+at all.
+
+So it is authored with the caption vocabulary, driven by the Script and the alignment map. Never
+reproduce it as `typo:` or `text:` with the words typed in by hand. That looks equivalent on the
+first frame and is not: hand-typed words no longer follow the Script, so re-recording the line leaves
+them stale, they are unreachable by anything that reasons about captions, and the identity that makes
+them checkable is gone.
+
+When the caption vocabulary cannot express the appearance, that is a vocabulary gap and it is
+declared as one. Write a new project-local caption package modelled on the installed caption
+packages — read them as the pattern for Style, Program, planning and alignment — and drive it from
+the same Script and map. Reaching for a typography Track because it already draws the shape is the
+mistake this section exists to prevent.
+
+Text that is *not* a caption keeps its own vocabulary: a title nobody says, a lower third, a label on
+a product, an editorial line that paraphrases rather than transcribes. The distinction is whether the
+words are the spoken words.
+
## Author one caption pipeline
Use one pipeline for the program:
diff --git a/.agents/skills/hypit/references/reference-video/vocabulary.md b/.agents/skills/hypit/references/reference-video/vocabulary.md
index baa9c151..5711efbb 100644
--- a/.agents/skills/hypit/references/reference-video/vocabulary.md
+++ b/.agents/skills/hypit/references/reference-video/vocabulary.md
@@ -26,6 +26,19 @@ Use this decision order:
changing the observed semantics.
3. Declare a real vocabulary gap only when neither option works.
+Step 2 is not a way around step 3. Composition may express an observation with several components; it
+may not move an element out of its role because some other tag happens to draw the same shape. What
+an element **is** decides which vocabulary owns it — words spoken aloud are captions, a full-screen
+designed field carrying content is one composition — and that answer does not change because the
+element is styled beautifully or because the owning vocabulary is missing one property. A package
+that owns the role but cannot express the observed appearance is a gap, exactly as much as no package
+at all: `../playbooks/craft/captions.md` works one such case through.
+
+Before accepting either step, name the observed appearance properties and say where each one lands in
+the declared vocabulary. A property with nowhere to land is the finding. Either it makes the gap, or
+it is written down as an accepted deviation with its reason before you continue — the one thing that
+may not happen is quietly authoring something else that resembles it.
+
Do not force a similar-looking tag into a role it does not own. Do not invent attributes or write a
nonexistent tag with the intention of implementing it later. A picture that
`../playbooks/craft/graphic-compositions.md` defines as one self-contained graphic composition may
diff --git a/packages/caption-fine/src/manifest.ts b/packages/caption-fine/src/manifest.ts
index 1a096e6d..7937f926 100644
--- a/packages/caption-fine/src/manifest.ts
+++ b/packages/caption-fine/src/manifest.ts
@@ -252,7 +252,7 @@ export const captionFineMarkupSurfaces = [
name: "track", tag: "Track", mode: "structured", outputs: [compositionTypes.visualTrack],
vocabulary: {
summary: "Joins a planned Caption against the measured SemanticMap and renders it as one ordinary peer VisualTrack.",
- appearance: "One block of caption text wrapped into lines inside a rounded Cue box, placed at a Recipe-chosen point on the Canvas and spanning a fraction of its width. Cues follow the speech one after another, each arriving and leaving with its own motion, and the Words of a Cue either stand there together from its first Frame or uncover as they are spoken. As the speech advances, the Word being spoken, or every Word up to it, is repainted in the active Paint, snapping at the Word boundary or sweeping across the glyphs, and it may take a rule beneath it, a rounded highlight capsule behind it and a brief pop at the moment it becomes the spoken one. That capsule either stands alone on each Atom or grows as one continuous run over everything already read, following the Words across line breaks.",
+ appearance: "One block of caption text wrapped into lines inside a rounded Cue box, placed at a Recipe-chosen point on the Canvas and spanning a fraction of its width. Cues follow the speech one after another, each arriving and leaving with its own motion, and the Words of a Cue either stand there together from its first Frame or uncover as they are spoken. As the speech advances, the Word being spoken, or every Word up to it, is repainted in the active Paint, snapping at the Word boundary or sweeping across the glyphs, and it may take a rule beneath it, a rounded highlight capsule behind it and a brief pop at the moment it becomes the spoken one. That capsule either stands alone on each Atom or grows as one continuous run over everything already read, following the Words across line breaks.\n\nEvery Word of every Cue is drawn in one typeface, the one its Style names: emphasis varies by Word through the active Paint, never by typeface, and lines exist only where the text wraps, so a line cannot be given a face, colour or weight of its own. A caption whose lines are set in different typefaces is outside what this Track can draw.",
preview: previewImage("Track.png"),
attributes: [
{ name: "id", kind: "identifier", required: true,
diff --git a/packages/markup/src/types.ts b/packages/markup/src/types.ts
index 866f37b3..6c6b99a6 100644
--- a/packages/markup/src/types.ts
+++ b/packages/markup/src/types.ts
@@ -143,6 +143,8 @@ export type SurfaceChildVocabulary = {
readonly cardinality: "one" | "optional" | "many";
readonly summary: string;
readonly attributes?: readonly SurfaceAttributeVocabulary[];
+ /** Elements written inside this child, declared to the same depth the Surface accepts them. */
+ readonly children?: readonly SurfaceChildVocabulary[];
readonly text?: string;
};
diff --git a/packages/typography-track/README.md b/packages/typography-track/README.md
index ba56e29b..411225f2 100644
--- a/packages/typography-track/README.md
+++ b/packages/typography-track/README.md
@@ -42,3 +42,10 @@ Content has two explicit author forms:
one plain document run; the Track still owns placement, timing, appearance and motion. Inline body
content owns a bounded rich `VisualTextDocument`. Dynamic rich text is intentionally not smuggled
through generic `Text`; it would require a separate explicit rich-document contract.
+
+`<typo:P>`, `<typo:Span>` and `<typo:Break>` are declared children of every item, so they are
+discoverable from the Surface vocabulary rather than from prose. A `style` on a `<typo:P>` or a
+`<typo:Span>` replaces the whole typography record, not only the paints: that paragraph or run is
+shaped with the referenced Style's own exact font at its own size, weight and slant, so one document
+can mix typefaces. The Style's area, point, path and stacking-order properties are ignored there;
+those stay with the item.
diff --git a/packages/typography-track/src/manifest.ts b/packages/typography-track/src/manifest.ts
index 9bcf745d..6c7017b7 100644
--- a/packages/typography-track/src/manifest.ts
+++ b/packages/typography-track/src/manifest.ts
@@ -163,6 +163,30 @@ const textMaskSpecSchema = object({
});
+/** The document an item owns, written the same way inside a Point, an Area or a Path. */
+const documentChildren = [
+ { tag: "P", cardinality: "many",
+ summary: "One paragraph of the item's document. It holds direct text, Span runs and Break line breaks in the order they are written, and can be drawn in a Style of its own.",
+ attributes: [
+ { name: "id", kind: "identifier", required: false, summary: "Names this paragraph inside the document; an omitted id is generated from the paragraph's position." },
+ { name: "style", kind: "reference", required: false, accepts: [typographyTrackTypes.style], summary: "Redraws the whole paragraph in another compiled Style, whose typography and Paint replace the item's: the paragraph is shaped with that Style's exact font and set at its size, weight and slant, and painted in its fills, outlines, glows, shadows, boxes and decorations. That Style's area, point, path and stacking-order properties are ignored here." },
+ ],
+ text: "Direct text between the nested elements is one run of the paragraph, drawn in the paragraph's own Style.",
+ children: [
+ { tag: "Span", cardinality: "many",
+ summary: "One run of the paragraph set apart from its neighbours, drawn in a Style, shaped under a language or laid out in a writing direction of its own.",
+ attributes: [
+ { name: "id", kind: "identifier", required: false, summary: "Names this run inside the paragraph; an omitted id is generated from the run's position." },
+ { name: "style", kind: "reference", required: false, accepts: [typographyTrackTypes.style], summary: "Redraws this run alone in another compiled Style, whose typography and Paint replace the paragraph's: the run is shaped with that Style's exact font and set at its size, weight and slant, and painted in its fills, outlines, glows, shadows, boxes and decorations. That Style's area, point, path and stacking-order properties are ignored here." },
+ { name: "language", kind: "literal", required: false, summary: "Names the language tag this run alone is shaped under." },
+ { name: "direction", kind: "literal", required: false, values: ["auto", "ltr", "rtl"], summary: "Decides the base writing direction of this run alone." },
+ ],
+ text: "Direct text is the run's whole content; a Span carries no nested elements and cannot be empty." },
+ { tag: "Break", cardinality: "many",
+ summary: "Breaks the line at this point and continues the same paragraph on the next one. It is written empty and takes no attributes." },
+ ] },
+] as const;
+
export const typographyTrackMarkupSurfaces = [
{ name: "style", tag: "Style", mode: "structured", outputs: [typographyTrackTypes.style],
vocabulary: {
@@ -444,6 +468,7 @@ export const typographyTrackMarkupSurfaces = [
{ name: "moment", kind: "reference", required: false, accepts: [narrativeTypes.moment], summary: "Binds the Moment that resolves `moment.cue` in a start/end window." },
{ name: "occurrences", kind: "literal", required: false, values: ["one", "each"], summary: "Decides whether a semantic source contributes one window or every occurrence; defaults to `one`." },
],
+ children: documentChildren,
text: "Direct text is the item's whole document, read as one paragraph." },
{ tag: "Area", cardinality: "many",
summary: "One text item flowed inside a SpatialFrame. It owns its own document, written as direct text or as P children.",
@@ -462,6 +487,7 @@ export const typographyTrackMarkupSurfaces = [
{ name: "moment", kind: "reference", required: false, accepts: [narrativeTypes.moment], summary: "Binds the Moment that resolves `moment.cue` in a start/end window." },
{ name: "occurrences", kind: "literal", required: false, values: ["one", "each"], summary: "Decides whether a semantic source contributes one window or every occurrence; defaults to `one`." },
],
+ children: documentChildren,
text: "Direct text is the item's whole document, read as one paragraph." },
{ tag: "Path", cardinality: "many",
summary: "One text item set along a SpatialPath. It owns its own document, written as direct text or as P children.",
@@ -480,6 +506,7 @@ export const typographyTrackMarkupSurfaces = [
{ name: "moment", kind: "reference", required: false, accepts: [narrativeTypes.moment], summary: "Binds the Moment that resolves `moment.cue` in a start/end window." },
{ name: "occurrences", kind: "literal", required: false, values: ["one", "each"], summary: "Decides whether a semantic source contributes one window or every occurrence; defaults to `one`." },
],
+ children: documentChildren,
text: "Direct text is the item's whole document, read as one paragraph." },
],
ports: [
@@ -492,14 +519,16 @@ export const typographyTrackMarkupSurfaces = [
<text:Area id="title" placement={title-frame} style={title-style} during="program">
EDIT MEANING, NOT TIMELINES
</text:Area>
+ <text:Area id="standfirst" placement={standfirst-frame} style={body-style} during="program">
+ <text:P>Edit meaning,<text:Break/>not <text:Span style={accent-serif}>timelines</text:Span>.</text:P>
+ </text:Area>
</text:Track>`,
notes: [
"A Track requires at least one `<Point>`, `<Area>` or `<Path>`, accepts no text content of its own, and refuses `map` when no item binds a Selection or a Moment.",
"An item states exactly one window form: `during`, `at` with `for`, or `start` with `end`; `selection` and `moment` bind a start/end window and cannot be written together.",
"A point expression is `program.start`, `program.end`, `selection.start`, `selection.end` or `moment.cue`, each optionally offset by `+` or `-` and a duration, or a bare duration read as an absolute position.",
- "An item written without `content` owns its own document: direct text becomes one paragraph, and `<P>` children carry rich runs instead.",
- "`<P>` accepts `id` and a `style` reference to a TextStyle, and contains text, `<Span>` and `<Break>` children; a document mixes neither `<P>` children with direct text nor direct text with nested elements.",
- "`<Span>` accepts `id`, a `style` reference, `language` and `direction`, contains text only, and cannot be empty; `<Break>` is written empty and takes no attributes.",
+ "An item written without `content` owns its own document: direct text becomes one paragraph, and `<P>` children carry rich runs instead; a document mixes neither `<P>` children with direct text nor direct text with nested elements.",
+ "Neither a `<P>` nor a `<Span>` may be empty, and a `style` on either must name a Style Record authored in this Source or imported from another, because its typography and Paint are copied into the document as the item is decoded.",
],
},
},
diff --git a/packages/typography-track/src/surface.ts b/packages/typography-track/src/surface.ts
index 4a9cd4e9..d1f62604 100644
--- a/packages/typography-track/src/surface.ts
+++ b/packages/typography-track/src/surface.ts
@@ -632,6 +632,8 @@ function paragraph(element: StructuredElement, index: number, resolve: (path: st
const name = localName(child.name);
if (name === "Break") {
allowed(child, []);
+ // A break carries nothing, and text written inside one was being dropped without a word.
+ empty(child);
inlines.push({ kind: "break", id: `break-${inlineIndex}` });
continue;
}