Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

refactor: separate caption planning from style rendering

kashorinf3d47a43c2b8
diff --git a/docs/quickstart.md b/docs/quickstart.md
index e473bef8..97e1e605 100644
--- a/docs/quickstart.md
+++ b/docs/quickstart.md
@@ -62,7 +62,7 @@ before spending money.
| [SVS Stylesheets](./quickstart/styles.md) | CSS-like Recipes for film, caption, B-roll, text, speech and fonts |
| [Media & Generation](./quickstart/generation.md) | media:Image, media:Audio, estimate:Speech, Seedance, speaker:Take |
| [Timing & Assembly](./quickstart/timing.md) | speech:Spine, whisperx:Alignment, ProgramSpace, SemanticMap |
-| [Caption, B-roll & Text](./quickstart/tracks.md) | caption:Style/Program/Planner/Track, broll:Track, text:Track |
+| [Caption, B-roll & Text](./quickstart/tracks.md) | caption-fine:Style/Track, caption:Program, Planner, B-roll and Text |
| [Film & Rendering](./quickstart/composition.md) | film:Film, render:Video, full pipeline walkthrough |
| [Run Source & Builds](./quickstart/run.md) | .svrun syntax, targets, reuse, runtime profile, build workflow |
diff --git a/docs/quickstart/composition.md b/docs/quickstart/composition.md
index 87ad65d7..adb07e57 100644
--- a/docs/quickstart/composition.md
+++ b/docs/quickstart/composition.md
@@ -49,7 +49,7 @@ Common Track sources:
|---|---|---|
| `{speech.visual}` | VisualTrack | `speech:Spine` — full-screen talking head |
| `{speech.audioTrack}` | AudioTrack | `speech:Spine` — synchronized audio |
-| `{captions.track}` | VisualTrack | `caption:Track` — timed captions |
+| `{captions.track}` | VisualTrack | a Caption Style-family Track — timed captions |
| `{cards.visual}` | VisualTrack | `broll:Track` — B-roll overlays |
| `{titles.track}` | VisualTrack | `text:Track` — text overlays |
@@ -114,6 +114,7 @@ The complete data flow from Script to rendered video. This example is based on
<import as="speech" from="@narratage/speech-spine@1"/>
<import as="whisperx" from="@narratage/whisperx@1"/>
<import as="caption" from="@narratage/caption@1"/>
+ <import as="caption-fine" from="@narratage/caption-fine@1"/>
<import as="caption-ai" from="@narratage/caption-gemini@1"/>
<import as="broll" from="@narratage/broll@1"/>
<import as="text" from="@narratage/text-track@1"/>
@@ -143,19 +144,12 @@ The complete data flow from Script to rendered video. This example is based on
<whisperx:Alignment id="timing" narrative={story} audio={speech.audio}/>
<!-- 4. Tracks: captions, B-roll, text -->
- <caption:Style id="base-caption" appearance={studio.caption.base}>
- <caption:Cues>Prefer short complete semantic phrases.</caption:Cues>
- <caption:Field id="important" type="boolean"
- min-per-cue="0" max-per-cue="2">
- Select zero, one, or two words whose emphasis best communicates
- this Cue.
- </caption:Field>
- </caption:Style>
- <caption:Program id="caption-program" narrative={story}
+ <caption-fine:Style id="base-caption" recipe={studio.caption.base}/>
+ <caption:Program id="caption-program" words={story.caption.words}
default={base-caption}/>
- <caption-ai:Planner id="cue-plan" narrative={story}
+ <caption-ai:Planner id="cue-plan" words={story.caption.words}
program={caption-program} model="gemini-2.5-flash"/>
- <caption:Track id="captions" narrative={story} map={timing.map}
+ <caption-fine:Track id="captions" narrative={story} words={story.caption.words} map={timing.map}
space={speech.space} plan={cue-plan.plan} program={caption-program}/>
<broll:Track id="cards" map={timing.map} space={speech.space}>
@@ -199,6 +193,9 @@ The complete data flow from Script to rendered video. This example is based on
enter: slide-up 4f; exit: fade 4f;
}
caption.base {
+ cue-min-words: 1; cue-max-words: 5;
+ important-min-per-cue: 0; important-max-per-cue: 2;
+ important-fill: #FFF16A; important-scale: 1.12;
stack-order: 70; x: 0.08; y: 0.76; width: 0.84;
font: Inter; weight: 600; size: 58; line-height: 1; align: center;
fill: #FFFFFF; background: #09090BCC; padding: 16 24; radius: 18;
diff --git a/docs/quickstart/script.md b/docs/quickstart/script.md
index b3f37c6b..be4d0a70 100644
--- a/docs/quickstart/script.md
+++ b/docs/quickstart/script.md
@@ -78,8 +78,8 @@ Role Cues produce different text projections:
| **caption** | `What time is it?`<br>`It's 8:30.` |
The dialogue projection includes Role Cue prefixes. Speech and caption projections strip them.
-Components like `seedance:Speech` use `{story.segment.dialogue.dialogue}` (with labels) while
-`caption:Track` uses the caption projection (without labels).
+Components like `seedance:Speech` use `{story.segment.dialogue.dialogue}` (with labels). Script also
+emits the complete ordered `{story.caption.words}` universe used by Caption Programs and Planners.
## Dual Text
@@ -159,9 +159,12 @@ Selections are not required to nest like XML tags. They can cross each other:
</demo>
```
-Selection markers are zero-width and never appear in any text projection. They compile into
-`SelectionSet` values containing `Range[]`. Script itself contains no seconds or frame numbers —
-timing comes from WhisperX alignment.
+Selection markers are zero-width and never appear in any text projection. Each public occurrence
+contains only its resolved start/end semantic anchor identities. Script itself contains no seconds
+or frame numbers — consumers project those anchors through an explicitly connected SemanticMap.
+
+For Caption only, Script additionally emits `{story.caption.selection.<id>}` as an ordered word
+subset. Caption therefore never reads parser token indexes from the general time Selection.
Other components reference Selections via `{story.selection.problem}` to bind visual content to
semantic moments in the narrative.
@@ -182,8 +185,9 @@ Moments are named time **points** (not ranges):
| `@id!` | Right-absorbing (point at the next word's start) |
| `~@id!` | Left-absorbing (point at the previous word's end) |
-Moments compile into `MomentSet` values containing `Point[]`. Selection and Moment share the same
-name namespace — the same id cannot be used for both.
+Each Moment occurrence contains only one resolved semantic anchor identity. A temporal consumer
+projects it through the same explicit SemanticMap. Selection and Moment share the same name
+namespace — the same id cannot be used for both.
Other components reference Moments via `{story.moment.ranking}`.
diff --git a/docs/quickstart/styles.md b/docs/quickstart/styles.md
index cdee23ae..05d69711 100644
--- a/docs/quickstart/styles.md
+++ b/docs/quickstart/styles.md
@@ -73,12 +73,18 @@ Referenced by `film:Film` via the `appearance` attribute:
<film:Film id="main" space={speech.space} appearance={studio.film.vertical}>
```
-## Caption
+## Caption Fine
-Caption visual appearance — position, typography, and container styling.
+The first official Caption Style family keeps planning and rendering parameters in one Recipe.
```svs
caption.dialogue {
+ cue-min-words: 2;
+ cue-max-words: 7;
+ important-min-per-cue: 0;
+ important-max-per-cue: 2;
+ important-fill: #FFD166;
+ important-scale: 1.08;
stack-order: 70;
x: 0.08;
y: 0.76;
@@ -97,6 +103,9 @@ caption.dialogue {
| Property | Description |
|---|---|
+| `cue-min-words`, `cue-max-words` | Common Cue word-count bounds |
+| `important-min-per-cue`, `important-max-per-cue` | Fine's per-Cue emphasis cardinality; max `0` disables the field |
+| `important-fill`, `important-scale` | Fine's rendering of words assigned the `important` field |
| `stack-order` | Z-stacking order among all Tracks (higher = on top) |
| `x`, `y` | Position as fraction of canvas (0–1) |
| `width` | Width as fraction of canvas |
@@ -110,10 +119,10 @@ caption.dialogue {
| `padding` | Container padding in pixels (single value or `vertical horizontal`) |
| `radius` | Container border radius in pixels |
-Referenced by `caption:Style` via the `appearance` attribute:
+Resolved by the Fine Style family:
```svml
-<caption:Style id="primary-caption" appearance={studio.caption.dialogue}>
+<caption-fine:Style id="primary-caption" recipe={studio.caption.dialogue}/>
```
### Per-role caption styles
@@ -122,6 +131,9 @@ Define multiple caption Recipes for different speakers:
```svs
caption.alice {
+ cue-min-words: 2; cue-max-words: 5;
+ important-min-per-cue: 0; important-max-per-cue: 2;
+ important-fill: #FFFFFF; important-scale: 1.08;
stack-order: 70;
x: 0.08; y: 0.76; width: 0.84;
font: Inter; weight: 600; size: 58;
@@ -131,6 +143,9 @@ caption.alice {
}
caption.bob {
+ cue-min-words: 2; cue-max-words: 5;
+ important-min-per-cue: 0; important-max-per-cue: 0;
+ important-fill: #FFFFFF; important-scale: 1;
stack-order: 70;
x: 0.08; y: 0.76; width: 0.84;
font: Inter; weight: 600; size: 58;
@@ -143,7 +158,10 @@ caption.bob {
Then assign them via `caption:Program`:
```svml
-<caption:Program id="caption-program" narrative={story} default={studio.caption.dialogue}>
+<caption-fine:Style id="default-caption" recipe={studio.caption.dialogue}/>
+<caption-fine:Style id="alice-caption" recipe={studio.caption.alice}/>
+<caption-fine:Style id="bob-caption" recipe={studio.caption.bob}/>
+<caption:Program id="caption-program" words={story.caption.words} default={default-caption}>
<caption:Use role="ALICE" style={alice-caption}/>
<caption:Use role="BOB" style={bob-caption}/>
</caption:Program>
@@ -367,6 +385,12 @@ A complete `studio.svs` file for a four-take talking-head project:
}
caption.primary {
+ cue-min-words: 2;
+ cue-max-words: 5;
+ important-min-per-cue: 0;
+ important-max-per-cue: 2;
+ important-fill: #FFF16A;
+ important-scale: 1.08;
stack-order: 70;
x: 0.08;
y: 0.74;
@@ -394,7 +418,7 @@ This file is imported once in the `.svml` source and its values are referenced t
<speaker:Take id="hook-take" ... recipe={studio.speaker.host} .../>
-<caption:Style id="primary-caption" appearance={studio.caption.primary} .../>
+<caption-fine:Style id="primary-caption" recipe={studio.caption.primary}/>
<film:Film id="main" space={speech.space} appearance={studio.film.vertical}>
```
diff --git a/docs/quickstart/timing.md b/docs/quickstart/timing.md
index a11dbc39..94968d7d 100644
--- a/docs/quickstart/timing.md
+++ b/docs/quickstart/timing.md
@@ -74,7 +74,7 @@ This produces the **SemanticMap** — the bridge between Script text and physica
| Output | Type | Used by |
|---|---|---|
-| `{timing.map}` | CompleteSemanticMap | `caption:Track`, `broll:Track`, `text:Track` — timed placement |
+| `{timing.map}` | CompleteSemanticMap | Caption Style-family Tracks, `broll:Track`, `text:Track` — timed placement |
The SemanticMap maps every authored Script anchor to a time point. It covers all `2M + 2N` identities
(where M = total speech tokens, N = number of Segments). This is how Selections and Moments declared
@@ -87,7 +87,7 @@ component that needs to know the total program duration and frame domain.
```svml
<film:Film id="main" space={speech.space} ...>
-<caption:Track id="captions" ... space={speech.space} .../>
+<caption-fine:Track id="captions" ... space={speech.space} .../>
<text:Track id="titles" space={speech.space}>
<render:Video id="final" composition={main.composition} space={speech.space}/>
```
@@ -112,7 +112,7 @@ Components that use the map take it via the `map` attribute:
```svml
<broll:Track id="cards" map={timing.map} ...>
-<caption:Track id="captions" ... map={timing.map} .../>
+<caption-fine:Track id="captions" ... map={timing.map} .../>
```
Each point in the map can be:
@@ -153,7 +153,7 @@ speaker:Take outputs ──► speech:Spine ──► whisperx:Alignment
.visual .map (SemanticMap)
.audio │
.audioTrack ▼
- .space ──────────► caption:Track
+ .space ──────────► caption-fine:Track
│ broll:Track
│ text:Track
▼ film:Film
diff --git a/docs/quickstart/tracks.md b/docs/quickstart/tracks.md
index a4f11cfa..bc936673 100644
--- a/docs/quickstart/tracks.md
+++ b/docs/quickstart/tracks.md
@@ -11,167 +11,90 @@ covers the three main visual Track types: captions, B-roll, and text overlays.
## Caption system
-Captions are built from four components that form a pipeline:
+Caption uses a small common language and a replaceable Style family:
```text
-caption:Style → caption:Program → caption-ai:Planner → caption:Track
+Script words → Style family → Caption Program → Planner → Style-family Track
```
```svml
<import as="caption" from="@narratage/caption@1"/>
+<import as="caption-fine" from="@narratage/caption-fine@1"/>
<import as="caption-ai" from="@narratage/caption-gemini@1"/>
```
-### caption:Style
+`@narratage/caption` owns only common Cue bounds, generic per-word fields, total Style assignment,
+Plan validation and the timing join. `@narratage/caption-fine` is one concrete Style family: it owns
+the `important` field, font/box parameters and visual rendering.
-Declares a complete visual appearance plus planning instructions for one style of caption.
+### caption-fine:Style
-```svml
-<caption:Style id="primary-caption" appearance={studio.caption.primary}
- mode="proportional-word">
- <caption:Cues>
- Split each Script Segment into short complete semantic phrases of two to
- seven words. Never cross a sentence or Segment boundary.
- </caption:Cues>
- <caption:Field id="important" type="boolean" min-per-cue="1" max-per-cue="2">
- Select one or two words whose emphasis best communicates this Cue.
- </caption:Field>
-</caption:Style>
-```
-
-| Attribute | Required | Description |
-|---|---|---|
-| `id` | yes | Unique identifier |
-| `appearance` | yes | SVS caption Recipe — position, font, colors, container |
-| `mode` | no | Caption timing mode, e.g. `proportional-word` |
+A Style is one indivisible pair of planning requirements and rendering parameters. Fine resolves
+both from one package-owned SVS Recipe:
-**Children:**
-
-- `<caption:Cues>` — natural-language instructions for the AI planner telling it how to split text
- into cues. The planner receives display text (left side of Dual Text) and these instructions, but
- never audio or timing data.
-- `<caption:Field>` — declares a typed per-atom annotation. The planner assigns these fields to
- individual words within each cue.
+```svs
+caption.primary {
+ cue-min-words: 2;
+ cue-max-words: 7;
+ important-min-per-cue: 1;
+ important-max-per-cue: 2;
+ important-fill: #FFD166;
+ important-scale: 1.08;
+ stack-order: 70;
+ x: 0.08; y: 0.76; width: 0.84;
+ font: Inter; weight: 600; size: 58; line-height: 0.96;
+ align: center; fill: #FFFFFF; background: #09090BCC;
+ padding: 16 24; radius: 18;
+}
+```
-| `<caption:Field>` attribute | Description |
-|---|---|
-| `id` | Field name (e.g. `important`) |
-| `type` | Field type: `boolean` |
-| `min-per-cue` | Minimum annotations per cue |
-| `max-per-cue` | Maximum annotations per cue |
+```svml
+<caption-fine:Style id="primary-caption" recipe={studio.caption.primary}/>
+```
-The element body of `<caption:Field>` is a natural-language instruction for the planner.
+Another Caption package may define completely different planning fields and visual parameters
+without changing the common package.
### caption:Program
-Assigns caption styles to the narrative. Declares a default style and optional per-role or
-per-Selection overrides.
+The Program starts from the complete ordered display-word universe emitted by Script. One explicit
+default Style covers every word; no `@whole` Selection or complement is required. Ordered `Use`
+rules replace the whole Style on a Role or explicit Caption word subset, with the last match winning.
```svml
-<caption:Program id="caption-program" narrative={story} default={primary-caption}>
+<caption:Program id="caption-program" words={story.caption.words}
+ default={primary-caption}>
<caption:Use role="ALICE" style={alice-caption}/>
<caption:Use role="BOB" style={bob-caption}/>
- <caption:Use on={story.selection.product-demo} style={dialogue-caption}/>
+ <caption:Use words={story.caption.selection.product-demo}
+ style={dialogue-caption}/>
</caption:Program>
```
-| Attribute | Required | Description |
-|---|---|---|
-| `id` | yes | Unique identifier |
-| `narrative` | yes | The Script component |
-| `default` | yes | Default `caption:Style` for all text |
-
-**Children:**
-
-`<caption:Use>` applies style overrides. Rules are applied in source order — last match wins.
-
-| `<caption:Use>` attribute | Description |
-|---|---|
-| `role` | Match by Role Cue label (e.g. `"ALICE"`) |
-| `on` | Match by Selection reference (e.g. `{story.selection.product-demo}`) |
-| `style` | The `caption:Style` to apply |
-
-Use `role=` to give different speakers different caption colors. Use `on=` to override style during
-specific Selections (e.g. a product demo section uses a different caption style).
+`role=` is convenient author syntax for a word subset, not a temporal condition. `words=` consumes
+the Caption-specific projection of a Script Selection; the public time Selection remains only a
+pair of semantic anchors. Partial ownership of an indivisible Dual Text display word is rejected.
### caption-ai:Planner
-AI-driven cue planning via Gemini. The planner receives the display text atoms, the Style
-instructions, and the Program assignments. It splits text into cues and assigns Field values.
-
```svml
-<caption-ai:Planner id="caption-plan" narrative={story}
+<caption-ai:Planner id="caption-plan" words={story.caption.words}
program={caption-program} model="gemini-2.5-flash"/>
```
-| Attribute | Required | Description |
-|---|---|---|
-| `id` | yes | Unique identifier |
-| `narrative` | yes | The Script component |
-| `program` | yes | The `caption:Program` |
-| `model` | yes | Gemini model: `gemini-2.5-flash` |
+The planner receives immutable display words and already-resolved Style runs. It may only cut each
+run into ordered Cues and attach declared fields to word ids. It cannot rewrite text, select Styles,
+see audio, or invent time. Its output is `{caption-plan.plan}`.
-The planner never receives audio, timing data, or the speech side of Dual Text. It works entirely
-from the caption (display) projection.
-
-**Output:** `{caption-plan.plan}` — the cue plan, passed to `caption:Track`.
-
-### caption:Track
-
-Joins the cue plan, SemanticMap, ProgramSpace, and Program to produce a timed VisualTrack.
-
-```svml
-<caption:Track id="captions" narrative={story} map={timing.map}
- space={speech.space} program={caption-program} plan={caption-plan.plan}/>
-```
-
-| Attribute | Required | Description |
-|---|---|---|
-| `id` | yes | Unique identifier |
-| `narrative` | yes | The Script component |
-| `map` | yes | SemanticMap from `whisperx:Alignment` |
-| `space` | yes | ProgramSpace from `speech:Spine` |
-| `program` | yes | The `caption:Program` |
-| `plan` | yes | Cue plan from `caption-ai:Planner` |
-
-**Output:** `{captions.track}` — a VisualTrack added to `film:Film`.
-
-### Caption combination example
-
-The full caption pipeline with per-role styles:
+### caption-fine:Track
```svml
-<caption:Style id="dialogue-caption" appearance={studio.caption.dialogue}>
- <caption:Cues>Use short complete semantic phrases, two to five words.</caption:Cues>
-</caption:Style>
-
-<caption:Style id="alice-caption" appearance={studio.caption.alice}>
- <caption:Cues>Use short complete semantic phrases, two to five words.</caption:Cues>
- <caption:Field id="important" type="boolean" min-per-cue="0" max-per-cue="2">
- Select at most two words whose emphasis best communicates this Cue.
- </caption:Field>
-</caption:Style>
-
-<caption:Style id="bob-caption" appearance={studio.caption.bob}>
- <caption:Cues>Use short complete semantic phrases, two to five words.</caption:Cues>
-</caption:Style>
-
-<caption:Program id="caption-program" narrative={story} default={dialogue-caption}>
- <caption:Use role="ALICE" style={alice-caption}/>
- <caption:Use role="BOB" style={bob-caption}/>
- <caption:Use on={story.selection.product-demo} style={dialogue-caption}/>
-</caption:Program>
-
-<caption-ai:Planner id="caption-plan" narrative={story}
- program={caption-program} model="gemini-2.5-flash"/>
-
-<caption:Track id="captions" narrative={story} map={timing.map}
+<caption-fine:Track id="captions" narrative={story} words={story.caption.words} map={timing.map}
space={speech.space} program={caption-program} plan={caption-plan.plan}/>
```
-ALICE gets green captions (`#73FBD3`), BOB gets gold (`#FFD166`), and during the product-demo
-Selection both switch to the neutral dialogue style. The `important` Field only applies to ALICE's
-style — her emphasized words get special treatment.
+The common Caption timing step joins the Plan to the independent SemanticMap. Fine then renders all
+default and override Styles into one ordinary peer `VisualTrack`: `{captions.track}`.
## B-roll
@@ -315,21 +238,17 @@ All three track types together in one source file:
```svml
<import as="caption" from="@narratage/caption@1"/>
+<import as="caption-fine" from="@narratage/caption-fine@1"/>
<import as="caption-ai" from="@narratage/caption-gemini@1"/>
<import as="broll" from="@narratage/broll@1"/>
<import as="text" from="@narratage/text-track@1"/>
<!-- Captions: primary style for all text -->
-<caption:Style id="base-caption" appearance={studio.caption.base}>
- <caption:Cues>Prefer short complete semantic phrases.</caption:Cues>
- <caption:Field id="important" type="boolean" min-per-cue="0" max-per-cue="2">
- Select zero, one, or two words whose emphasis best communicates this Cue.
- </caption:Field>
-</caption:Style>
-<caption:Program id="caption-program" narrative={story} default={base-caption}/>
-<caption-ai:Planner id="cue-plan" narrative={story}
+<caption-fine:Style id="base-caption" recipe={studio.caption.base}/>
+<caption:Program id="caption-program" words={story.caption.words} default={base-caption}/>
+<caption-ai:Planner id="cue-plan" words={story.caption.words}
program={caption-program} model="gemini-2.5-flash"/>
-<caption:Track id="captions" narrative={story} map={timing.map}
+<caption-fine:Track id="captions" narrative={story} words={story.caption.words} map={timing.map}
space={speech.space} plan={cue-plan.plan} program={caption-program}/>
<!-- B-roll: generated video during a Selection -->
diff --git a/docs/zh/quickstart.md b/docs/zh/quickstart.md
index adaf6451..75c441bd 100644
--- a/docs/zh/quickstart.md
+++ b/docs/zh/quickstart.md
@@ -59,7 +59,7 @@ pnpm narratage plan examples/talking-film-graph-check/build.svrun \
| [SVS 样式表](./quickstart/styles.md) | CSS 风格的 Recipe:film、caption、B-roll、text、speech、字体 |
| [媒体与生成](./quickstart/generation.md) | media:Image、media:Audio、estimate:Speech、Seedance、speaker:Take |
| [时序与装配](./quickstart/timing.md) | speech:Spine、whisperx:Alignment、ProgramSpace、SemanticMap |
-| [字幕、B-roll 与文字](./quickstart/tracks.md) | caption:Style/Program/Planner/Track、broll:Track、text:Track |
+| [字幕、B-roll 与文字](./quickstart/tracks.md) | caption-fine:Style/Track、caption:Program、Planner、B-roll 与 Text |
| [Film 与渲染](./quickstart/composition.md) | film:Film、render:Video、完整流水线演练 |
| [Run Source 与 Build](./quickstart/run.md) | .svrun 语法、targets、复用、runtime profile、build 工作流 |
diff --git a/docs/zh/quickstart/composition.md b/docs/zh/quickstart/composition.md
index 1c0e19f4..ec20de93 100644
--- a/docs/zh/quickstart/composition.md
+++ b/docs/zh/quickstart/composition.md
@@ -48,7 +48,7 @@ Composition。然后渲染器将该 Composition 编译为 MP4 视频。
|---|---|---|
| `{speech.visual}` | VisualTrack | `speech:Spine`——全屏说话人画面 |
| `{speech.audioTrack}` | AudioTrack | `speech:Spine`——同步音频 |
-| `{captions.track}` | VisualTrack | `caption:Track`——定时字幕 |
+| `{captions.track}` | VisualTrack | Caption 样式族 Track——定时字幕 |
| `{cards.visual}` | VisualTrack | `broll:Track`——B-roll 叠加层 |
| `{titles.track}` | VisualTrack | `text:Track`——文字叠加层 |
@@ -111,6 +111,7 @@ Track 是**扁平的**——没有嵌套或分组。Z 轴排序完全由每个 T
<import as="speech" from="@narratage/speech-spine@1"/>
<import as="whisperx" from="@narratage/whisperx@1"/>
<import as="caption" from="@narratage/caption@1"/>
+ <import as="caption-fine" from="@narratage/caption-fine@1"/>
<import as="caption-ai" from="@narratage/caption-gemini@1"/>
<import as="broll" from="@narratage/broll@1"/>
<import as="text" from="@narratage/text-track@1"/>
@@ -140,19 +141,12 @@ Track 是**扁平的**——没有嵌套或分组。Z 轴排序完全由每个 T
<whisperx:Alignment id="timing" narrative={story} audio={speech.audio}/>
<!-- 4. Tracks: captions, B-roll, text -->
- <caption:Style id="base-caption" appearance={studio.caption.base}>
- <caption:Cues>Prefer short complete semantic phrases.</caption:Cues>
- <caption:Field id="important" type="boolean"
- min-per-cue="0" max-per-cue="2">
- Select zero, one, or two words whose emphasis best communicates
- this Cue.
- </caption:Field>
- </caption:Style>
- <caption:Program id="caption-program" narrative={story}
+ <caption-fine:Style id="base-caption" recipe={studio.caption.base}/>
+ <caption:Program id="caption-program" words={story.caption.words}
default={base-caption}/>
- <caption-ai:Planner id="cue-plan" narrative={story}
+ <caption-ai:Planner id="cue-plan" words={story.caption.words}
program={caption-program} model="gemini-2.5-flash"/>
- <caption:Track id="captions" narrative={story} map={timing.map}
+ <caption-fine:Track id="captions" narrative={story} words={story.caption.words} map={timing.map}
space={speech.space} plan={cue-plan.plan} program={caption-program}/>
<broll:Track id="cards" map={timing.map} space={speech.space}>
@@ -196,6 +190,9 @@ Track 是**扁平的**——没有嵌套或分组。Z 轴排序完全由每个 T
enter: slide-up 4f; exit: fade 4f;
}
caption.base {
+ cue-min-words: 1; cue-max-words: 5;
+ important-min-per-cue: 0; important-max-per-cue: 2;
+ important-fill: #FFF16A; important-scale: 1.12;
stack-order: 70; x: 0.08; y: 0.76; width: 0.84;
font: Inter; weight: 600; size: 58; line-height: 1; align: center;
fill: #FFFFFF; background: #09090BCC; padding: 16 24; radius: 18;
diff --git a/docs/zh/quickstart/script.md b/docs/zh/quickstart/script.md
index 8dcc3416..d8250c5e 100644
--- a/docs/zh/quickstart/script.md
+++ b/docs/zh/quickstart/script.md
@@ -70,7 +70,7 @@ Role Cue 会产生不同的文本投影:
| **speech** | `What time is it?`<br>`It's 8:30.` |
| **caption** | `What time is it?`<br>`It's 8:30.` |
-dialogue 投影包含 Role Cue 前缀。speech 和 caption 投影会去除前缀。像 `seedance:Speech` 这样的组件使用 `{story.segment.dialogue.dialogue}`(带标签),而 `caption:Track` 使用 caption 投影(不带标签)。
+dialogue 投影包含 Role Cue 前缀,speech 和 caption 投影会去除前缀。像 `seedance:Speech` 这样的组件使用 `{story.segment.dialogue.dialogue}`(带标签)。Script 还会显式输出完整有序的 `{story.caption.words}` 词全集,供 Caption Program 和 Planner 使用。
## Dual Text
@@ -148,7 +148,9 @@ Selection 不要求像 XML 标签那样嵌套,它们可以互相交叉:
</demo>
```
-Selection 标记是零宽度的,不会出现在任何文本投影中。它们编译为包含 `Range[]` 的 `SelectionSet` 值。Script 本身不包含秒数或帧号——时间信息来自 WhisperX 对齐。
+Selection 标记是零宽度的,不会出现在任何文本投影中。每个公开 occurrence 只包含已经解析好的首尾语义锚点身份。Script 本身不包含秒数或帧号;消费者必须通过显式连接的 SemanticMap 投影这些锚点。
+
+Caption 还会得到 `{story.caption.selection.<id>}` 这一有序词子集,因此不会从通用时间 Selection 中读取 Parser token 下标。
其他组件通过 `{story.selection.problem}` 引用 Selection,将视觉内容绑定到叙事中的语义时刻。
@@ -168,7 +170,7 @@ Moment 是具名的时间**点**(不是范围):
| `@id!` | 右吸附(时间点位于下一个单词的起始处) |
| `~@id!` | 左吸附(时间点位于前一个单词的末尾) |
-Moment 编译为包含 `Point[]` 的 `MomentSet` 值。Selection 和 Moment 共享同一命名空间——同一个 id 不能同时用于两者。
+每个 Moment occurrence 只包含一个已经解析好的语义锚点身份;时间消费者通过同一份显式 SemanticMap 投影它。Selection 和 Moment 共享同一命名空间——同一个 id 不能同时用于两者。
其他组件通过 `{story.moment.ranking}` 引用 Moment。
diff --git a/docs/zh/quickstart/styles.md b/docs/zh/quickstart/styles.md
index fb1933f5..fee08037 100644
--- a/docs/zh/quickstart/styles.md
+++ b/docs/zh/quickstart/styles.md
@@ -69,12 +69,18 @@ film.vertical {
<film:Film id="main" space={speech.space} appearance={studio.film.vertical}>
```
-## 字幕
+## Caption Fine
-字幕视觉外观——位置、排版和容器样式。
+第一种官方 Caption 样式族把规划要求和渲染参数放在同一个 Recipe 中。
```svs
caption.dialogue {
+ cue-min-words: 2;
+ cue-max-words: 7;
+ important-min-per-cue: 0;
+ important-max-per-cue: 2;
+ important-fill: #FFD166;
+ important-scale: 1.08;
stack-order: 70;
x: 0.08;
y: 0.76;
@@ -93,6 +99,9 @@ caption.dialogue {
| 属性 | 描述 |
|---|---|
+| `cue-min-words`、`cue-max-words` | 通用 Cue 字数边界 |
+| `important-min-per-cue`、`important-max-per-cue` | Fine 每个 Cue 的强调数量;最大值为 `0` 时禁用该字段 |
+| `important-fill`、`important-scale` | Fine 对 `important` 词的渲染方式 |
| `stack-order` | 所有 Track 之间的 Z 轴层叠顺序(值越大越靠前) |
| `x`、`y` | 位置,以画布比例表示(0–1) |
| `width` | 宽度,以画布比例表示 |
@@ -106,10 +115,10 @@ caption.dialogue {
| `padding` | 容器内边距(像素)(单个值或 `垂直 水平`) |
| `radius` | 容器圆角半径(像素) |
-通过 `caption:Style` 的 `appearance` 属性引用:
+由 Fine 样式族解析:
```svml
-<caption:Style id="primary-caption" appearance={studio.caption.dialogue}>
+<caption-fine:Style id="primary-caption" recipe={studio.caption.dialogue}/>
```
### 按角色设置字幕样式
@@ -118,6 +127,9 @@ caption.dialogue {
```svs
caption.alice {
+ cue-min-words: 2; cue-max-words: 5;
+ important-min-per-cue: 0; important-max-per-cue: 2;
+ important-fill: #FFFFFF; important-scale: 1.08;
stack-order: 70;
x: 0.08; y: 0.76; width: 0.84;
font: Inter; weight: 600; size: 58;
@@ -127,6 +139,9 @@ caption.alice {
}
caption.bob {
+ cue-min-words: 2; cue-max-words: 5;
+ important-min-per-cue: 0; important-max-per-cue: 0;
+ important-fill: #FFFFFF; important-scale: 1;
stack-order: 70;
x: 0.08; y: 0.76; width: 0.84;
font: Inter; weight: 600; size: 58;
@@ -139,7 +154,10 @@ caption.bob {
然后通过 `caption:Program` 进行分配:
```svml
-<caption:Program id="caption-program" narrative={story} default={studio.caption.dialogue}>
+<caption-fine:Style id="default-caption" recipe={studio.caption.dialogue}/>
+<caption-fine:Style id="alice-caption" recipe={studio.caption.alice}/>
+<caption-fine:Style id="bob-caption" recipe={studio.caption.bob}/>
+<caption:Program id="caption-program" words={story.caption.words} default={default-caption}>
<caption:Use role="ALICE" style={alice-caption}/>
<caption:Use role="BOB" style={bob-caption}/>
</caption:Program>
@@ -361,6 +379,12 @@ caption.dialogue {
}
caption.primary {
+ cue-min-words: 2;
+ cue-max-words: 5;
+ important-min-per-cue: 0;
+ important-max-per-cue: 2;
+ important-fill: #FFF16A;
+ important-scale: 1.08;
stack-order: 70;
x: 0.08;
y: 0.74;
@@ -388,7 +412,7 @@ caption.dialogue {
<speaker:Take id="hook-take" ... recipe={studio.speaker.host} .../>
-<caption:Style id="primary-caption" appearance={studio.caption.primary} .../>
+<caption-fine:Style id="primary-caption" recipe={studio.caption.primary}/>
<film:Film id="main" space={speech.space} appearance={studio.film.vertical}>
```
diff --git a/docs/zh/quickstart/timing.md b/docs/zh/quickstart/timing.md
index 54a7aaa4..07ac37e6 100644
--- a/docs/zh/quickstart/timing.md
+++ b/docs/zh/quickstart/timing.md
@@ -69,7 +69,7 @@ Spine 产生四个输出,供下游组件使用:
| 输出 | 类型 | 使用方 |
|---|---|---|
-| `{timing.map}` | CompleteSemanticMap | `caption:Track`、`broll:Track`、`text:Track`——定时放置 |
+| `{timing.map}` | CompleteSemanticMap | Caption 样式族 Track、`broll:Track`、`text:Track`——定时放置 |
SemanticMap 将每个 Script 中标注的锚点映射到一个时间点。它覆盖所有 `2M + 2N` 个标识(其中 M = 语音词元总数,N = Segment 数量)。这就是 Script 中声明的 Selection 和 Moment 如何转化为下游视觉组件所需的实际时间范围和时间点。
@@ -79,7 +79,7 @@ ProgramSpace 不是一个需要声明的组件——它由 `speech:Spine` 产生
```svml
<film:Film id="main" space={speech.space} ...>
-<caption:Track id="captions" ... space={speech.space} .../>
+<caption-fine:Track id="captions" ... space={speech.space} .../>
<text:Track id="titles" space={speech.space}>
<render:Video id="final" composition={main.composition} space={speech.space}/>
```
@@ -100,7 +100,7 @@ SemanticMap 是连接 Script 文本与物理时间的类型化桥梁。当你在
```svml
<broll:Track id="cards" map={timing.map} ...>
-<caption:Track id="captions" ... map={timing.map} .../>
+<caption-fine:Track id="captions" ... map={timing.map} .../>
```
映射中的每个点可以是:
@@ -141,7 +141,7 @@ speaker:Take outputs ──► speech:Spine ──► whisperx:Alignment
.visual .map (SemanticMap)
.audio │
.audioTrack ▼
- .space ──────────► caption:Track
+ .space ──────────► caption-fine:Track
│ broll:Track
│ text:Track
▼ film:Film
diff --git a/docs/zh/quickstart/tracks.md b/docs/zh/quickstart/tracks.md
index 0e7977f0..1dde2aab 100644
--- a/docs/zh/quickstart/tracks.md
+++ b/docs/zh/quickstart/tracks.md
@@ -9,158 +9,72 @@ description: 视觉 Track 组件——字幕、B-roll 叠加层和文字叠加
## 字幕系统
-字幕由四个组件构成一条流水线:
+字幕由一套很小的公共语言与可替换的样式族组成:
```text
-caption:Style → caption:Program → caption-ai:Planner → caption:Track
+Script 词全集 → 样式族 → Caption Program → Planner → 样式族 Track
```
```svml
<import as="caption" from="@narratage/caption@1"/>
+<import as="caption-fine" from="@narratage/caption-fine@1"/>
<import as="caption-ai" from="@narratage/caption-gemini@1"/>
```
-### caption:Style
+公共 Caption 只负责 Cue 字数边界、通用逐词字段、完整样式分配、Plan 校验与时间
+拼接。Fine 是第一种具体样式族,负责 `important` 字段、字体/框参数和视觉渲染。
-声明一种字幕样式的完整视觉外观及规划指令。
+### caption-fine:Style
+
+一个 Style 是不可拆分的“规划要求 + 渲染参数”。Fine 从一个包自有的 SVS Recipe
+同时解析两者:
```svml
-<caption:Style id="primary-caption" appearance={studio.caption.primary}
- mode="proportional-word">
- <caption:Cues>
- Split each Script Segment into short complete semantic phrases of two to
- seven words. Never cross a sentence or Segment boundary.
- </caption:Cues>
- <caption:Field id="important" type="boolean" min-per-cue="1" max-per-cue="2">
- Select one or two words whose emphasis best communicates this Cue.
- </caption:Field>
-</caption:Style>
+<caption-fine:Style id="primary-caption" recipe={studio.caption.primary}/>
```
-| 属性 | 必填 | 描述 |
-|---|---|---|
-| `id` | 是 | 唯一标识符 |
-| `appearance` | 是 | SVS 字幕 Recipe——位置、字体、颜色、容器 |
-| `mode` | 否 | 字幕计时模式,例如 `proportional-word` |
-
-**子元素:**
-
-- `<caption:Cues>` —— 给 AI 规划器的自然语言指令,告诉它如何将文本拆分为提示单元。规划器接收显示文本(Dual Text 的左侧)和这些指令,但不会接收音频或时间数据。
-- `<caption:Field>` —— 声明一个有类型的逐词注解。规划器将这些字段分配给每个提示单元中的各个词。
-
-| `<caption:Field>` 属性 | 描述 |
-|---|---|
-| `id` | 字段名称(例如 `important`) |
-| `type` | 字段类型:`boolean` |
-| `min-per-cue` | 每个提示单元的最少注解数 |
-| `max-per-cue` | 每个提示单元的最多注解数 |
-
-`<caption:Field>` 的元素主体是给规划器的自然语言指令。
+Recipe 同时包含 `cue-min-words`、`cue-max-words`、Fine 自有的
+`important-*-per-cue`、强调外观和完整字体/框参数。其他字幕包可以定义完全不同
+的字段和渲染方式,无需修改公共 Caption。
### caption:Program
-将字幕样式分配给叙事。声明一个默认样式,并可选择按角色或按 Selection 覆盖。
+Program 消费 Script 显式输出的完整有序显示词全集。一个必填的默认 Style 自动覆盖
+所有词,不需要作者制造 `@whole` 或补集。`Use` 按源码顺序替换整个 Style,后命中
+者获胜。
```svml
-<caption:Program id="caption-program" narrative={story} default={primary-caption}>
+<caption:Program id="caption-program" words={story.caption.words}
+ default={primary-caption}>
<caption:Use role="ALICE" style={alice-caption}/>
<caption:Use role="BOB" style={bob-caption}/>
- <caption:Use on={story.selection.product-demo} style={dialogue-caption}/>
+ <caption:Use words={story.caption.selection.product-demo}
+ style={dialogue-caption}/>
</caption:Program>
```
-| 属性 | 必填 | 描述 |
-|---|---|---|
-| `id` | 是 | 唯一标识符 |
-| `narrative` | 是 | Script 组件 |
-| `default` | 是 | 所有文本的默认 `caption:Style` |
-
-**子元素:**
-
-`<caption:Use>` 用于应用样式覆盖。规则按源代码顺序应用——后匹配优先。
-
-| `<caption:Use>` 属性 | 描述 |
-|---|---|
-| `role` | 按 Role Cue 标签匹配(例如 `"ALICE"`) |
-| `on` | 按 Selection 引用匹配(例如 `{story.selection.product-demo}`) |
-| `style` | 要应用的 `caption:Style` |
-
-使用 `role=` 为不同说话者设置不同的字幕颜色。使用 `on=` 在特定 Selection 期间覆盖样式(例如产品演示部分使用不同的字幕样式)。
+`role=` 是词子集查询的作者语法,不是时间条件。`words=` 接收 Selection 的字幕专用
+词投影;通用 Selection 的公开值仍只有语义首尾锚点。
### caption-ai:Planner
-通过 Gemini 驱动的 AI 提示单元规划。规划器接收显示文本原子、Style 指令和 Program 分配,将文本拆分为提示单元并分配 Field 值。
-
```svml
-<caption-ai:Planner id="caption-plan" narrative={story}
+<caption-ai:Planner id="caption-plan" words={story.caption.words}
program={caption-program} model="gemini-2.5-flash"/>
```
-| 属性 | 必填 | 描述 |
-|---|---|---|
-| `id` | 是 | 唯一标识符 |
-| `narrative` | 是 | Script 组件 |
-| `program` | 是 | `caption:Program` |
-| `model` | 是 | Gemini 模型:`gemini-2.5-flash` |
-
-规划器不会接收音频、时间数据或 Dual Text 的语音侧。它完全基于字幕(显示)投影进行工作。
-
-**输出:**`{caption-plan.plan}` —— 提示单元计划,传递给 `caption:Track`。
+Planner 只接收不可改写的显示词与已解析好的 Style runs。它只能分 Cue,并给词 id
+附上样式声明的字段;看不到音频、时间或 Dual Text 右侧。
-### caption:Track
-
-将提示单元计划、SemanticMap、ProgramSpace 和 Program 组合在一起,生成一个带时间信息的 VisualTrack。
-
-```svml
-<caption:Track id="captions" narrative={story} map={timing.map}
- space={speech.space} program={caption-program} plan={caption-plan.plan}/>
-```
-
-| 属性 | 必填 | 描述 |
-|---|---|---|
-| `id` | 是 | 唯一标识符 |
-| `narrative` | 是 | Script 组件 |
-| `map` | 是 | 来自 `whisperx:Alignment` 的 SemanticMap |
-| `space` | 是 | 来自 `speech:Spine` 的 ProgramSpace |
-| `program` | 是 | `caption:Program` |
-| `plan` | 是 | 来自 `caption-ai:Planner` 的提示单元计划 |
-
-**输出:**`{captions.track}` —— 添加到 `film:Film` 的 VisualTrack。
-
-### 字幕组合示例
-
-包含按角色样式的完整字幕流水线:
+### caption-fine:Track
```svml
-<caption:Style id="dialogue-caption" appearance={studio.caption.dialogue}>
- <caption:Cues>Use short complete semantic phrases, two to five words.</caption:Cues>
-</caption:Style>
-
-<caption:Style id="alice-caption" appearance={studio.caption.alice}>
- <caption:Cues>Use short complete semantic phrases, two to five words.</caption:Cues>
- <caption:Field id="important" type="boolean" min-per-cue="0" max-per-cue="2">
- Select at most two words whose emphasis best communicates this Cue.
- </caption:Field>
-</caption:Style>
-
-<caption:Style id="bob-caption" appearance={studio.caption.bob}>
- <caption:Cues>Use short complete semantic phrases, two to five words.</caption:Cues>
-</caption:Style>
-
-<caption:Program id="caption-program" narrative={story} default={dialogue-caption}>
- <caption:Use role="ALICE" style={alice-caption}/>
- <caption:Use role="BOB" style={bob-caption}/>
- <caption:Use on={story.selection.product-demo} style={dialogue-caption}/>
-</caption:Program>
-
-<caption-ai:Planner id="caption-plan" narrative={story}
- program={caption-program} model="gemini-2.5-flash"/>
-
-<caption:Track id="captions" narrative={story} map={timing.map}
+<caption-fine:Track id="captions" narrative={story} words={story.caption.words} map={timing.map}
space={speech.space} program={caption-program} plan={caption-plan.plan}/>
```
-ALICE 使用绿色字幕(`#73FBD3`),BOB 使用金色字幕(`#FFD166`),在 product-demo Selection 期间两者都切换为中性对话样式。`important` Field 仅应用于 ALICE 的样式——她的强调词会获得特殊处理。
+公共 Caption 先把 Plan 与独立 SemanticMap 拼接,Fine 再把所有默认/覆盖样式渲染成
+一个普通的对等 `VisualTrack`:`{captions.track}`。
## B-roll
@@ -301,21 +215,17 @@ B-roll 搭配生成的 Seedance 视频,在 Script Selection 期间出现:
```svml
<import as="caption" from="@narratage/caption@1"/>
+<import as="caption-fine" from="@narratage/caption-fine@1"/>
<import as="caption-ai" from="@narratage/caption-gemini@1"/>
<import as="broll" from="@narratage/broll@1"/>
<import as="text" from="@narratage/text-track@1"/>
<!-- Captions: primary style for all text -->
-<caption:Style id="base-caption" appearance={studio.caption.base}>
- <caption:Cues>Prefer short complete semantic phrases.</caption:Cues>
- <caption:Field id="important" type="boolean" min-per-cue="0" max-per-cue="2">
- Select zero, one, or two words whose emphasis best communicates this Cue.
- </caption:Field>
-</caption:Style>
-<caption:Program id="caption-program" narrative={story} default={base-caption}/>
-<caption-ai:Planner id="cue-plan" narrative={story}
+<caption-fine:Style id="base-caption" recipe={studio.caption.base}/>
+<caption:Program id="caption-program" words={story.caption.words} default={base-caption}/>
+<caption-ai:Planner id="cue-plan" words={story.caption.words}
program={caption-program} model="gemini-2.5-flash"/>
-<caption:Track id="captions" narrative={story} map={timing.map}
+<caption-fine:Track id="captions" narrative={story} words={story.caption.words} map={timing.map}
space={speech.space} plan={cue-plan.plan} program={caption-program}/>
<!-- B-roll: generated video during a Selection -->
diff --git a/examples/talking-film-golden/README.md b/examples/talking-film-golden/README.md
index 9027ec32..fd5a009d 100644
--- a/examples/talking-film-golden/README.md
+++ b/examples/talking-film-golden/README.md
@@ -74,9 +74,9 @@ Candidates and Operations.
| `whisperx:Alignment` | `@narratage/whisperx` | 48k speech master -> explicit 16k evidence Need -> WhisperX -> provider-neutral `@narratage/speech-alignment` -> Map | Surface, Graph Fragment, deterministic components and local Provider/service implemented |
| `seedance:Video model="mini"` | `@narratage/seedance` | explicit Seedance Mini video Need plus primary-video projection | implemented |
| `broll:Track` | `@narratage/broll` | semantic windows + normalized media + recipe -> peer Visual/Audio Tracks | Surface and deterministic lowering implemented |
-| `caption:Style` / `caption:Program` | `@narratage/caption` | default total Style + ordered whole-Style replacement by Role or Selection | implemented |
-| `caption-ai:Planner` | `@narratage/caption-gemini` | immutable display atoms + per-run requirements -> cue cuts and per-atom fields | implemented; Google Vertex Endpoint implemented separately |
-| `caption:Track` | `@narratage/caption` | CaptionPlan + independent SemanticMap + complete Styles -> VisualTrack | implemented |
+| `caption-fine:Style` / `caption:Program` | Fine + common Caption | explicit default over all words + ordered whole-Style replacement by Role or word subset | implemented |
+| `caption-ai:Planner` | `@narratage/caption-gemini` | immutable display words + per-run requirements -> Cue cuts and per-word fields | implemented; Google Vertex Endpoint implemented separately |
+| `caption-fine:Track` | `@narratage/caption-fine` | CaptionPlan + independent SemanticMap + complete Fine Styles -> VisualTrack | implemented |
| `text:Track` | `@narratage/text-track` | package Spec + ProgramSpace -> VisualTrack | provider-free Surface/lowering implemented; exact-font use remains |
| `film:Film` | `@narratage/film` | finite TrackSet fold -> Composition | Graph Fragment and official Surface implemented |
| `render:Video` | `@narratage/render-hyperframes` | Composition -> silent HyperframesDocument render + explicit program audio + mux -> final video Artifact + Receipt | Surface, Fragment and all local execution Providers implemented |
diff --git a/examples/talking-film-golden/main.svml b/examples/talking-film-golden/main.svml
index 29027b6b..6b23a4d7 100644
--- a/examples/talking-film-golden/main.svml
+++ b/examples/talking-film-golden/main.svml
@@ -13,6 +13,7 @@
<import as="speech" from="@narratage/speech-spine@1"/>
<import as="whisperx" from="@narratage/whisperx@1"/>
<import as="caption" from="@narratage/caption@1"/>
+ <import as="caption-fine" from="@narratage/caption-fine@1"/>
<import as="caption-ai" from="@narratage/caption-gemini@1"/>
<import as="broll" from="@narratage/broll@1"/>
<import as="text" from="@narratage/text-track@1"/>
@@ -57,31 +58,22 @@
</seedance:Prompt>
<!-- Complete styles are declared once. Selection and use remain separate. -->
- <caption:Style id="dialogue-caption" appearance={studio.caption.dialogue}>
- <caption:Cues>Use short complete semantic phrases, usually between two and five words.</caption:Cues>
- </caption:Style>
-
- <caption:Style id="alice-caption" appearance={studio.caption.alice}>
- <caption:Cues>Use short complete semantic phrases, usually between two and five words.</caption:Cues>
- <caption:Field id="important" type="boolean" min-per-cue="0" max-per-cue="2">
- Select at most two words whose emphasis best communicates this Cue.
- </caption:Field>
- </caption:Style>
-
- <caption:Style id="bob-caption" appearance={studio.caption.bob}>
- <caption:Cues>Use short complete semantic phrases, usually between two and five words.</caption:Cues>
- </caption:Style>
-
- <caption:Program id="caption-program" narrative={script} default={dialogue-caption}>
+ <caption-fine:Style id="dialogue-caption" recipe={studio.caption.dialogue}/>
+
+ <caption-fine:Style id="alice-caption" recipe={studio.caption.alice}/>
+
+ <caption-fine:Style id="bob-caption" recipe={studio.caption.bob}/>
+
+ <caption:Program id="caption-program" words={script.caption.words} default={dialogue-caption}>
<caption:Use role="ALICE" style={alice-caption}/>
<caption:Use role="BOB" style={bob-caption}/>
<!-- Ordered whole-style replacement: no complement Selection is required. -->
- <caption:Use on={script.selection.product-demo} style={dialogue-caption}/>
+ <caption:Use words={script.caption.selection.product-demo} style={dialogue-caption}/>
</caption:Program>
<caption-ai:Planner
id="caption-plan"
- narrative={script}
+ words={script.caption.words}
program={caption-program}
model="gemini-2.5-flash"
/>
@@ -134,10 +126,12 @@
</broll:Track>
<!-- This imported package explicitly chooses Gemini cue planning; it cannot rewrite Script. -->
- <caption:Track
+ <caption-fine:Track
id="captions"
narrative={script}
+ words={script.caption.words}
map={timing.map}
+ space={speech.space}
program={caption-program}
plan={caption-plan.plan}
/>
diff --git a/examples/talking-film-golden/studio.svs b/examples/talking-film-golden/studio.svs
index 04acda76..81e4f24d 100644
--- a/examples/talking-film-golden/studio.svs
+++ b/examples/talking-film-golden/studio.svs
@@ -51,6 +51,12 @@
}
caption.dialogue {
+ cue-min-words: 2;
+ cue-max-words: 5;
+ important-min-per-cue: 0;
+ important-max-per-cue: 0;
+ important-fill: #FFFFFF;
+ important-scale: 1;
stack-order: 70;
x: 0.08;
y: 0.76;
@@ -67,6 +73,12 @@
}
caption.alice {
+ cue-min-words: 2;
+ cue-max-words: 5;
+ important-min-per-cue: 0;
+ important-max-per-cue: 2;
+ important-fill: #FFF16A;
+ important-scale: 1.12;
stack-order: 70;
x: 0.08;
y: 0.76;
@@ -83,6 +95,12 @@
}
caption.bob {
+ cue-min-words: 2;
+ cue-max-words: 5;
+ important-min-per-cue: 0;
+ important-max-per-cue: 0;
+ important-fill: #FFD166;
+ important-scale: 1;
stack-order: 70;
x: 0.08;
y: 0.76;
diff --git a/examples/talking-film-graph-check/README.md b/examples/talking-film-graph-check/README.md
index d4dbfc2d..0898b52b 100644
--- a/examples/talking-film-graph-check/README.md
+++ b/examples/talking-film-graph-check/README.md
@@ -14,6 +14,7 @@ pnpm narratage plan examples/talking-film-graph-check/build.svrun \
--package-lock examples/talking-film-graph-check/svml.packages.lock --root .
```
-The Gemini planner receives immutable left-side display atoms and resolved style runs only. Its generic
+The Gemini planner receives immutable ordered display words and resolved Style runs only. Its generic
`CaptionPlan` joins the independent `CompleteSemanticMap` in `@narratage/caption`; neither the planner
-nor the Vertex Provider has text-rewriting or timing authority.
+nor the Vertex Provider has text-rewriting or timing authority. `@narratage/caption-fine` alone owns
+the concrete `important` field, Recipe interpretation and visual renderer.
diff --git a/examples/talking-film-graph-check/main.svml b/examples/talking-film-graph-check/main.svml
index d0f13bc9..6d1569ef 100644
--- a/examples/talking-film-graph-check/main.svml
+++ b/examples/talking-film-graph-check/main.svml
@@ -7,6 +7,7 @@
<import as="whisperx" from="@narratage/whisperx@1"/>
<import as="broll" from="@narratage/broll@1"/>
<import as="caption" from="@narratage/caption@1"/>
+ <import as="caption-fine" from="@narratage/caption-fine@1"/>
<import as="caption-ai" from="@narratage/caption-gemini@1"/>
<import as="text" from="@narratage/text-track@1"/>
<import as="film" from="@narratage/film@1"/>
@@ -26,20 +27,15 @@
</speech:Spine>
<whisperx:Alignment id="timing" narrative={story} audio={speech.audio}/>
- <caption:Style id="base-caption" appearance={studio.caption.base}>
- <caption:Cues>Prefer short complete semantic phrases. Never cross clause punctuation.</caption:Cues>
- <caption:Field id="important" type="boolean"
- min-per-cue="0" max-per-cue="2">
- Select zero, one, or two words whose emphasis best communicates this Cue.
- </caption:Field>
- </caption:Style>
- <caption:Program id="caption-program" narrative={story} default={base-caption}/>
- <caption-ai:Planner id="cue-plan" narrative={story} program={caption-program} model="gemini-2.5-flash"/>
+ <caption-fine:Style id="base-caption" recipe={studio.caption.base}/>
+ <caption:Program id="caption-program" words={story.caption.words} default={base-caption}/>
+ <caption-ai:Planner id="cue-plan" words={story.caption.words} program={caption-program} model="gemini-2.5-flash"/>
<broll:Track id="cards" map={timing.map} space={speech.space}>
<broll:Item source={motion.video} during={story.selection.demo} appearance={studio.broll.card}/>
</broll:Track>
- <caption:Track id="captions" narrative={story} map={timing.map} space={speech.space}
+ <caption-fine:Track id="captions" narrative={story} words={story.caption.words}
+ map={timing.map} space={speech.space}
plan={cue-plan.plan} program={caption-program}/>
<text:Track id="titles" space={speech.space}>
<text:Item text="MEANING" during="full" appearance={studio.text.title}/>
diff --git a/examples/talking-film-graph-check/studio.svs b/examples/talking-film-graph-check/studio.svs
index 9d37b9a3..941df60f 100644
--- a/examples/talking-film-graph-check/studio.svs
+++ b/examples/talking-film-graph-check/studio.svs
@@ -7,6 +7,9 @@
background: #111116; radius: 20; enter: slide-up 4f; exit: fade 4f;
}
caption.base {
+ cue-min-words: 1; cue-max-words: 5;
+ important-min-per-cue: 0; important-max-per-cue: 2;
+ important-fill: #FFF16A; important-scale: 1.12;
stack-order: 70; x: 0.08; y: 0.76; width: 0.84; font: Inter; weight: 600;
size: 58; line-height: 1; align: center; fill: #FFFFFF;
background: #09090BCC; padding: 16 24; radius: 18;
diff --git a/examples/talking-film-graph-check/svml.packages.lock b/examples/talking-film-graph-check/svml.packages.lock
index 0460aac2..708f461b 100644
--- a/examples/talking-film-graph-check/svml.packages.lock
+++ b/examples/talking-film-graph-check/svml.packages.lock
@@ -11,15 +11,20 @@
"version": "0.0.0-dev",
"digest": "sha256:f058e3161f5ab570e5f180d548ef31e8c70945e8f37d7530caced1d9db31e368"
},
+ {
+ "name": "@narratage/caption-fine",
+ "version": "0.0.0-dev",
+ "digest": "sha256:0b803c8876f1838b25ff8d5a3511bd2f5510751940d4f2793db1eab4a9cc98cd"
+ },
{
"name": "@narratage/caption-gemini",
"version": "0.0.0-dev",
- "digest": "sha256:8a631e9345ea2c5901f3785058fde176a8280b9f20dd77e3333d651d56991797"
+ "digest": "sha256:2a8f75b53ffc67401aeb94f5d2b79afc4871ad2de11dab731ac65d74147b6aab"
},
{
"name": "@narratage/caption",
"version": "0.0.0-dev",
- "digest": "sha256:dc090e7d96c7a6c3a53c4aabd0874f1b743f02ecd7e05c81f50fef8cfa88ae6a"
+ "digest": "sha256:56883d4bed9bbf939d2c1267df7594b59902c240e18281e259f5eb0b5bca89e1"
},
{
"name": "@narratage/component-kit",
@@ -79,7 +84,7 @@
{
"name": "@narratage/narrative",
"version": "0.0.0-dev",
- "digest": "sha256:85e98e61021310c2dd6473189dcbcc9daa54039277169cbb5ed51ed70277a894"
+ "digest": "sha256:61e1d8b149008c83cabb05b532c5fae49d0aadf479a405dd9b7bf98ff3dd4a09"
},
{
"name": "@narratage/program-space",
@@ -109,7 +114,7 @@
{
"name": "@narratage/script",
"version": "0.0.0-dev",
- "digest": "sha256:5b741e43dd8966543d977e26a00e4e94973c57934e98037915f5fdface77518c"
+ "digest": "sha256:60860383d3df5645cc437e9a516b62db524a37d5dbe1acfa9a10008c4de5f709"
},
{
"name": "@narratage/seedance",
@@ -119,7 +124,7 @@
{
"name": "@narratage/semantic-map",
"version": "0.0.0-dev",
- "digest": "sha256:4293518a23ead1320f81d5d38e60bef96ea7f27149495bc890b9e4c275572864"
+ "digest": "sha256:425c1ea0d89039fc070de2a5de13b7d414d7a3a7d4c2e155445603fad3d60572"
},
{
"name": "@narratage/source",
@@ -159,12 +164,12 @@
{
"name": "@narratage/text-track",
"version": "0.0.0-dev",
- "digest": "sha256:171ba9562ae4827148895363a247728be740fb2782db5d90cbe5a023176898e5"
+ "digest": "sha256:03f9bdfb00df7dabbf196e48bdd56c8a581079fbdd2fb7c97ef0df0d4096fdc5"
},
{
"name": "@narratage/text",
"version": "0.0.0-dev",
- "digest": "sha256:99605f90eeca6d07428bd3e90741c3ddb36c75f8cd6a0e0958112d0994d13fd9"
+ "digest": "sha256:0558cd7cde9d4f0b8d6d880da23212f83d2328f1e531afc6eb7567e527125964"
},
{
"name": "@narratage/validation",
@@ -198,8 +203,8 @@
"name": "@narratage/broll",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:a30c9a993e7f24f39a78b80f8262022ca8d6250b4d454fc8de91c8922bf8d45a",
- "closureDigest": "sha256:afaf9e5edbab9a7ec16a50b043d06090ac52aeed7c4553ca3b0420a8f73c3e69"
+ "facetsDigest": "sha256:490a4d536c592a308a4441cdbf5e8884a84aca07da6bd380a1bf4893052c6601",
+ "closureDigest": "sha256:6fc6e1e3fb1de56e2cb2b4c1edeb88c971b1da1e2d30aa575cba5a1238416709"
},
{
"specifier": "@narratage/caption",
@@ -207,8 +212,17 @@
"name": "@narratage/caption",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:05d147f0231a47895852c6f1a42e049e8ecfa1487c6af2c091d39a0b151450bd",
- "closureDigest": "sha256:0bc1436bb393f3f640849a4b328a371e9a0c3703b5af6bb09c3737118a3b0102"
+ "facetsDigest": "sha256:2bde33270a3381610524b375782fe1bfd8cd73e75b9c93251e02b3d88b480ad0",
+ "closureDigest": "sha256:bce7573a505437d7e87900be19ba9b9e2470efd2b34b1504a56df285c38174e6"
+ },
+ {
+ "specifier": "@narratage/caption-fine",
+ "package": {
+ "name": "@narratage/caption-fine",
+ "version": "0.0.0-dev"
+ },
+ "facetsDigest": "sha256:24be8a66a292efc633e44781cee2ae391932e5b6a0ee917e2e14e02ef07f6afc",
+ "closureDigest": "sha256:cf273f4247731bd0dca0c693af35eae3ea5ee7c21357d8baac51442cc54e218c"
},
{
"specifier": "@narratage/caption-gemini",
@@ -216,8 +230,8 @@
"name": "@narratage/caption-gemini",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:e893c2186ed38bac9fe95840856ccfcbc8465e0ceca384d8fcb6830d7ab052b0",
- "closureDigest": "sha256:795108b2ab3574875fc53cab3d8fd74dbb1a2e68e4b3e1540a7209cbeaccb993"
+ "facetsDigest": "sha256:7a4037e49c13ceb17587d43b01b663c46c1a3af451d340fcac52d64b5aba4671",
+ "closureDigest": "sha256:40b5f28cb10b36aeed3871743fa43485a93b33315e0ac3ec817b5bae08188bb8"
},
{
"specifier": "@narratage/composition",
@@ -226,7 +240,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:ec680d429d88ec1c55531125a18c1620fea0c81600de55ba6d43c4fbf873e480",
- "closureDigest": "sha256:218aa1bb041c9701bb74cd0c5ac25df041b808d87b68f03c81f38eaf2e1641ae"
+ "closureDigest": "sha256:4a528d83d47df3588bd8223ed4f13b5497f54d8b9f1e50b3fba52ff0de7aacb9"
},
{
"specifier": "@narratage/film",
@@ -235,7 +249,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:61839fefb8cf723e9b55f4ac0932928b2dba5de4935ca03c0c4335246c6ab5d8",
- "closureDigest": "sha256:01563360044ec05fe1484609d3c37a634f467a31ac269211e2768ab6554f80eb"
+ "closureDigest": "sha256:fe2b5a6656a1fa1e7df66a13b5a6b35a7231c085ddab77508c2ceb686c6f48c3"
},
{
"specifier": "@narratage/generation",
@@ -253,7 +267,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:34bcacd52b6d396bf53f3b9709b43d54377a48c3b0332180079c3488a7fbe6ec",
- "closureDigest": "sha256:036c344ce50db44c6508d2cff8cf9fae7795618da062ce375bb0d916c2a8c3ca"
+ "closureDigest": "sha256:2473311ab583320ab5c6cb383dd1cb20f902b1891af2821a78fe9ed7e571b966"
},
{
"specifier": "@narratage/media",
@@ -262,7 +276,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:c883f03447734dc30a6c239c828083e6e9708258e457f845c2cfa5a2f95c0387",
- "closureDigest": "sha256:de7edf7b6436e4f35c782ffafd23f277419b015d32b024ba45ff30f8ef341fe8"
+ "closureDigest": "sha256:ec5993223ae3961246e318e1c036a73dfbc5c5918b4ae7aaebea242d6c9a8b9b"
},
{
"specifier": "@narratage/media-pipeline",
@@ -270,8 +284,8 @@
"name": "@narratage/media-pipeline",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:783aeed74e2586b3f4cff5350a27d06de9cb9472a237dabc0d5c68680cca3546",
- "closureDigest": "sha256:620abf19fcb09f9543773e0aa7d3e6639bc3828782bf7d24dfcc84929a9b700f"
+ "facetsDigest": "sha256:c93d4a7ced859956130610ebd14539663bbaed6152f3309eb7cb786cefd40458",
+ "closureDigest": "sha256:661de1ca6a2907fb93758800c63439de8fabce230238bfb78e0dbdcf1a5c17a3"
},
{
"specifier": "@narratage/narrative",
@@ -279,8 +293,8 @@
"name": "@narratage/narrative",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:849b7e49ef228b44d07ae57a106b18a24c7730c88783057bcdbad1e2e61153d3",
- "closureDigest": "sha256:e8030a009c4904f38373be51c372d5fcf0947aff81ed07317fce62b1930904cb"
+ "facetsDigest": "sha256:f4d042f2b821793beae3e33d1a35da83d45129e9e1bd33c54092a8abb75cccd7",
+ "closureDigest": "sha256:a324b56d3f1550ae1b94f3d178d1e184f7c04aa8fde796f8f10fa034d9778d9c"
},
{
"specifier": "@narratage/program-space",
@@ -297,8 +311,8 @@
"name": "@narratage/render-hyperframes",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:f1bf7447dfd5a468a97681a61684e00452588b1c4e10b6ee280be109055a7fc3",
- "closureDigest": "sha256:36d404f6f847aaf7606e924afd82d323eecd62d27573d3d6eaa0fb1a56e20c36"
+ "facetsDigest": "sha256:8a8726172d2396d54f8f99f483523827353ef1acb8fa3b050cd62ae16d9cd8ef",
+ "closureDigest": "sha256:03f37df3468dab6d4a9dfa0b727c36c040dbe251526cecdf45edf49e39a27804"
},
{
"specifier": "@narratage/run-text",
@@ -307,7 +321,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:fbcab7cea86223fe00ea04b51c2160bb534aafbb99067f5877e6825b0ce4eefc",
- "closureDigest": "sha256:16f3d7feeac702adf6962a0f6e651a42f25171ba899171f457452d99a57419f9"
+ "closureDigest": "sha256:a6d37eabad64b436b87c5edf4a4200412415b303be3c571fc1af6f6c3ad289db"
},
{
"specifier": "@narratage/script",
@@ -315,8 +329,8 @@
"name": "@narratage/script",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:2aea9956ef27554936214ce0738c0667da0cf8a860fafd63b4def5d22dab3ae8",
- "closureDigest": "sha256:ec9ffd07ba10bb8f906bbfb58351b33cf605cce7491e8ad308d7d580519230ff"
+ "facetsDigest": "sha256:c4500a649b05a9ba4b07bef7ee9cdc9d1d3ebaff76b9a678361131a7fca982f3",
+ "closureDigest": "sha256:ad9cf9128e425d6bff194b83a64e7ae719df5d04a7622f8dba6228232ca0a5b4"
},
{
"specifier": "@narratage/seedance",
@@ -324,8 +338,8 @@
"name": "@narratage/seedance",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:5d01251026ee31cc7ba33f40bb3d50d59a1dd5ccbc725b91fee3499c3988af67",
- "closureDigest": "sha256:e6ada0f380625c17f47b6638a25fa6ab6f86c80f3f310c5a74672cfb54d3c734"
+ "facetsDigest": "sha256:bcb5da66886c2e21be9193492db40e3585991cb3846ee3b014a1848b4a00986e",
+ "closureDigest": "sha256:e6d4aa8d6351bf7652760f94315dac192a484270cf5dbe33beb630da9f8086c3"
},
{
"specifier": "@narratage/semantic-map",
@@ -333,8 +347,8 @@
"name": "@narratage/semantic-map",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:4f041c5844a3eb9a70e8472d4cf4f534de960a77745819366dac0061ec600e76",
- "closureDigest": "sha256:d420abb0bf5c4b8815930b60a37040b646a939d7cc47f489df9100afb2670d16"
+ "facetsDigest": "sha256:40f349ffdc8a2f276c971f2e62a22d6e162d97ed47f608e33d5a2c6e51cd93a4",
+ "closureDigest": "sha256:5da5da001f31dfb0e09b96bf8fb3135ddc5e011f396b9ca4c69e46132e6f01c1"
},
{
"specifier": "@narratage/speech",
@@ -342,8 +356,8 @@
"name": "@narratage/speech",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:fbecf0c9334b34c91f49c4edd9f2592d7b49856ec2d8a875dd272f47ade29a18",
- "closureDigest": "sha256:1b016e5f394ee73a1740552fa370bb7b8d4554d87aefa8c5fa91b9e0a7568e42"
+ "facetsDigest": "sha256:606702bab5ea6bbab372be70e5d9d032975721c88754e64001929d20aa690bf7",
+ "closureDigest": "sha256:c07b24e334b2efc6af8b0c7f8a729a40f00fd73e51693e68118984963a9a8ca0"
},
{
"specifier": "@narratage/speech-alignment",
@@ -351,8 +365,8 @@
"name": "@narratage/speech-alignment",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:b3f48d736480bd0e1911a9195d9143ffb5c38a91bdcc55584845d6488cccc14f",
- "closureDigest": "sha256:1b3da03de14b740434bee79cc128d5e76f32816dedc1f7fcab1368a2eb8f26c5"
+ "facetsDigest": "sha256:ba23d1fe5fccd3c2210994a9d6e245bfed0b283e144e3040bbfcb621f440aefb",
+ "closureDigest": "sha256:502f3454cbdeb1a7769cbeb25b0ec792afc5180f73e40129ec745978362c6b18"
},
{
"specifier": "@narratage/speech-basis",
@@ -360,8 +374,8 @@
"name": "@narratage/speech-basis",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:b69ede6dba97a02452c0ceb824bb4373606a13a13ec24ae10901390774ac41e7",
- "closureDigest": "sha256:d40be58e90198d03ef08e52921e8178f9e4b2ee0dc23592c5f07ffa7bef576e9"
+ "facetsDigest": "sha256:c98ce13f17cc1a5ebb225d3335db8804a49f70965732deb46ad133941d58f02b",
+ "closureDigest": "sha256:57a24ce23026e4e4b1ca0d30ad58f55ce5840c252843653ebd5012105ef894d2"
},
{
"specifier": "@narratage/speech-evidence",
@@ -369,8 +383,8 @@
"name": "@narratage/speech-evidence",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:69110ba7b1740b0e609f6beef673fe4ef113ba5b0c803fb785ecf1dc9325f0a7",
- "closureDigest": "sha256:b0bc5b3bc1d303a3ea9964012acf37ee7cd5bb32840f81ed03eea109ed0f85c7"
+ "facetsDigest": "sha256:0de6289f6e0bc6b139e4c2fe063bde23877158f69be0935d506e83d2771a9af3",
+ "closureDigest": "sha256:805278f1a0cf261172314edf17896df0e836622d4a9bd7519d5b2d17fa42ce84"
},
{
"specifier": "@narratage/speech-spine",
@@ -378,8 +392,8 @@
"name": "@narratage/speech-spine",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:8c66979579d259706192dca1adec774a5eb6559e3e66a87a23d1939420f3e357",
- "closureDigest": "sha256:0e9f376a96c1b1491edee79dab9d1a5fcd8563f5a84eb5fd9d2f2228dee9dcd6"
+ "facetsDigest": "sha256:76bb403ab59040cd460c140b1f458a86305907a5c16076a75f745341a494092d",
+ "closureDigest": "sha256:592f7aed7f9d27366d579f2bc9d6d6a2ffde05b68e7e47dd08d8b27f306a3370"
},
{
"specifier": "@narratage/svs",
@@ -396,8 +410,8 @@
"name": "@narratage/text-track",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:a9d66c62cfd43269152d504d8500bfbe97a69939d5327afaa4e6bddd6c53e284",
- "closureDigest": "sha256:42ac8ae147f6df6e69709618e129fb00973f61dbe52e59bdf2809d05e608d140"
+ "facetsDigest": "sha256:e57b2aa7713a21b5eb200d7f537ec5ddf216421f80c35a8437d180c3998785bd",
+ "closureDigest": "sha256:68359a51ffe4a5aca13806a4652f9519f95ba5f7145f25d0440188e6c9b17312"
},
{
"specifier": "@narratage/visual-ir",
@@ -414,9 +428,9 @@
"name": "@narratage/whisperx",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:1899c422f35229f82da0a2b0ae341ff0234d6b174fc14ccaaf0ccc47624a58a9",
- "closureDigest": "sha256:c8f596b99ea0e634a7e1239f71af6f265dee8a3e78beccc670dc481396165fdd"
+ "facetsDigest": "sha256:d2c6ba803cd4fce5260bb40d5187bc71e0e5540d576d98a55689e36f01a40cde",
+ "closureDigest": "sha256:88a8d09bc98716c2ac5ab47a6fdbdd02e9422d9287aff2782aa9cdac1c870b97"
}
],
- "digest": "sha256:5618f97278cfc8e1290257d0b31a32f6922170475729c551049a2079639e44ee"
+ "digest": "sha256:860cfabb975aab6ae63e86deb43af34393006f022bec4d554151c79dceb14b07"
}
diff --git a/examples/talking-film-live/main.svml b/examples/talking-film-live/main.svml
index ab0bc9a5..67cb5526 100644
--- a/examples/talking-film-live/main.svml
+++ b/examples/talking-film-live/main.svml
@@ -8,6 +8,7 @@
<import as="speech" from="@narratage/speech-spine@1"/>
<import as="whisperx" from="@narratage/whisperx@1"/>
<import as="caption" from="@narratage/caption@1"/>
+ <import as="caption-fine" from="@narratage/caption-fine@1"/>
<import as="caption-ai" from="@narratage/caption-gemini@1"/>
<import as="text" from="@narratage/text-track@1"/>
<import as="film" from="@narratage/film@1"/>
@@ -88,24 +89,20 @@
<whisperx:Alignment id="timing" narrative={story} audio={speech.audio}/>
- <caption:Style id="primary-caption" appearance={studio.caption.primary}>
- <caption:Cues>Use short complete semantic phrases of two to five words. Never cross sentence boundaries.</caption:Cues>
- <caption:Field id="important" type="boolean" min-per-cue="0" max-per-cue="2">
- Select at most two words whose emphasis best communicates this Cue.
- </caption:Field>
- </caption:Style>
+ <caption-fine:Style id="primary-caption" recipe={studio.caption.primary}/>
- <caption:Program id="caption-program" narrative={story} default={primary-caption}/>
+ <caption:Program id="caption-program" words={story.caption.words} default={primary-caption}/>
<caption-ai:Planner
id="caption-plan"
- narrative={story}
+ words={story.caption.words}
program={caption-program}
model="gemini-2.5-flash"
/>
- <caption:Track
+ <caption-fine:Track
id="captions"
narrative={story}
+ words={story.caption.words}
map={timing.map}
space={speech.space}
program={caption-program}
diff --git a/examples/talking-film-live/studio.svs b/examples/talking-film-live/studio.svs
index f237b4c6..7911fd04 100644
--- a/examples/talking-film-live/studio.svs
+++ b/examples/talking-film-live/studio.svs
@@ -9,6 +9,12 @@
}
caption.primary {
+ cue-min-words: 2;
+ cue-max-words: 5;
+ important-min-per-cue: 0;
+ important-max-per-cue: 2;
+ important-fill: #FFF16A;
+ important-scale: 1.12;
stack-order: 70;
x: 0.08;
y: 0.74;
diff --git a/examples/talking-film-live/svml.packages.lock b/examples/talking-film-live/svml.packages.lock
index ccc00711..05b21ab2 100644
--- a/examples/talking-film-live/svml.packages.lock
+++ b/examples/talking-film-live/svml.packages.lock
@@ -6,15 +6,20 @@
"version": "0.0.0-dev",
"digest": "sha256:b16107fdad5256905782fe28142cd19e77673f190691e839fd12d4f1f536ea0f"
},
+ {
+ "name": "@narratage/caption-fine",
+ "version": "0.0.0-dev",
+ "digest": "sha256:0b803c8876f1838b25ff8d5a3511bd2f5510751940d4f2793db1eab4a9cc98cd"
+ },
{
"name": "@narratage/caption-gemini",
"version": "0.0.0-dev",
- "digest": "sha256:8a631e9345ea2c5901f3785058fde176a8280b9f20dd77e3333d651d56991797"
+ "digest": "sha256:2a8f75b53ffc67401aeb94f5d2b79afc4871ad2de11dab731ac65d74147b6aab"
},
{
"name": "@narratage/caption",
"version": "0.0.0-dev",
- "digest": "sha256:dc090e7d96c7a6c3a53c4aabd0874f1b743f02ecd7e05c81f50fef8cfa88ae6a"
+ "digest": "sha256:56883d4bed9bbf939d2c1267df7594b59902c240e18281e259f5eb0b5bca89e1"
},
{
"name": "@narratage/component-kit",
@@ -79,7 +84,7 @@
{
"name": "@narratage/narrative",
"version": "0.0.0-dev",
- "digest": "sha256:85e98e61021310c2dd6473189dcbcc9daa54039277169cbb5ed51ed70277a894"
+ "digest": "sha256:61e1d8b149008c83cabb05b532c5fae49d0aadf479a405dd9b7bf98ff3dd4a09"
},
{
"name": "@narratage/program-space",
@@ -109,7 +114,7 @@
{
"name": "@narratage/script",
"version": "0.0.0-dev",
- "digest": "sha256:5b741e43dd8966543d977e26a00e4e94973c57934e98037915f5fdface77518c"
+ "digest": "sha256:60860383d3df5645cc437e9a516b62db524a37d5dbe1acfa9a10008c4de5f709"
},
{
"name": "@narratage/seedance",
@@ -119,7 +124,7 @@
{
"name": "@narratage/semantic-map",
"version": "0.0.0-dev",
- "digest": "sha256:4293518a23ead1320f81d5d38e60bef96ea7f27149495bc890b9e4c275572864"
+ "digest": "sha256:425c1ea0d89039fc070de2a5de13b7d414d7a3a7d4c2e155445603fad3d60572"
},
{
"name": "@narratage/source",
@@ -159,12 +164,12 @@
{
"name": "@narratage/text-track",
"version": "0.0.0-dev",
- "digest": "sha256:171ba9562ae4827148895363a247728be740fb2782db5d90cbe5a023176898e5"
+ "digest": "sha256:03f9bdfb00df7dabbf196e48bdd56c8a581079fbdd2fb7c97ef0df0d4096fdc5"
},
{
"name": "@narratage/text",
"version": "0.0.0-dev",
- "digest": "sha256:99605f90eeca6d07428bd3e90741c3ddb36c75f8cd6a0e0958112d0994d13fd9"
+ "digest": "sha256:0558cd7cde9d4f0b8d6d880da23212f83d2328f1e531afc6eb7567e527125964"
},
{
"name": "@narratage/validation",
@@ -198,8 +203,17 @@
"name": "@narratage/caption",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:05d147f0231a47895852c6f1a42e049e8ecfa1487c6af2c091d39a0b151450bd",
- "closureDigest": "sha256:0bc1436bb393f3f640849a4b328a371e9a0c3703b5af6bb09c3737118a3b0102"
+ "facetsDigest": "sha256:2bde33270a3381610524b375782fe1bfd8cd73e75b9c93251e02b3d88b480ad0",
+ "closureDigest": "sha256:bce7573a505437d7e87900be19ba9b9e2470efd2b34b1504a56df285c38174e6"
+ },
+ {
+ "specifier": "@narratage/caption-fine",
+ "package": {
+ "name": "@narratage/caption-fine",
+ "version": "0.0.0-dev"
+ },
+ "facetsDigest": "sha256:24be8a66a292efc633e44781cee2ae391932e5b6a0ee917e2e14e02ef07f6afc",
+ "closureDigest": "sha256:cf273f4247731bd0dca0c693af35eae3ea5ee7c21357d8baac51442cc54e218c"
},
{
"specifier": "@narratage/caption-gemini",
@@ -207,8 +221,8 @@
"name": "@narratage/caption-gemini",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:e893c2186ed38bac9fe95840856ccfcbc8465e0ceca384d8fcb6830d7ab052b0",
- "closureDigest": "sha256:795108b2ab3574875fc53cab3d8fd74dbb1a2e68e4b3e1540a7209cbeaccb993"
+ "facetsDigest": "sha256:7a4037e49c13ceb17587d43b01b663c46c1a3af451d340fcac52d64b5aba4671",
+ "closureDigest": "sha256:40b5f28cb10b36aeed3871743fa43485a93b33315e0ac3ec817b5bae08188bb8"
},
{
"specifier": "@narratage/composition",
@@ -217,7 +231,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:ec680d429d88ec1c55531125a18c1620fea0c81600de55ba6d43c4fbf873e480",
- "closureDigest": "sha256:218aa1bb041c9701bb74cd0c5ac25df041b808d87b68f03c81f38eaf2e1641ae"
+ "closureDigest": "sha256:4a528d83d47df3588bd8223ed4f13b5497f54d8b9f1e50b3fba52ff0de7aacb9"
},
{
"specifier": "@narratage/estimate",
@@ -225,8 +239,8 @@
"name": "@narratage/estimate",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:0dc2391fa5ddaaf7e53136ba3ea13bab243db1e598cedce9f5296f0e0fc07dab",
- "closureDigest": "sha256:0228faf9bca5b950b753a49899f1e35df7049535aa66383da163dbb20ffe8bb7"
+ "facetsDigest": "sha256:eefe70d98cbf8e501d8102fc30d95778193b7985bf684a28b59a041b03ea10b6",
+ "closureDigest": "sha256:06ca52046a5fc0ebaecab78a060923a62f1d79e9505be02ec99df920cabb4c2a"
},
{
"specifier": "@narratage/film",
@@ -235,7 +249,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:61839fefb8cf723e9b55f4ac0932928b2dba5de4935ca03c0c4335246c6ab5d8",
- "closureDigest": "sha256:01563360044ec05fe1484609d3c37a634f467a31ac269211e2768ab6554f80eb"
+ "closureDigest": "sha256:fe2b5a6656a1fa1e7df66a13b5a6b35a7231c085ddab77508c2ceb686c6f48c3"
},
{
"specifier": "@narratage/generation",
@@ -253,7 +267,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:34bcacd52b6d396bf53f3b9709b43d54377a48c3b0332180079c3488a7fbe6ec",
- "closureDigest": "sha256:036c344ce50db44c6508d2cff8cf9fae7795618da062ce375bb0d916c2a8c3ca"
+ "closureDigest": "sha256:2473311ab583320ab5c6cb383dd1cb20f902b1891af2821a78fe9ed7e571b966"
},
{
"specifier": "@narratage/media",
@@ -262,7 +276,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:c883f03447734dc30a6c239c828083e6e9708258e457f845c2cfa5a2f95c0387",
- "closureDigest": "sha256:de7edf7b6436e4f35c782ffafd23f277419b015d32b024ba45ff30f8ef341fe8"
+ "closureDigest": "sha256:ec5993223ae3961246e318e1c036a73dfbc5c5918b4ae7aaebea242d6c9a8b9b"
},
{
"specifier": "@narratage/media-pipeline",
@@ -270,8 +284,8 @@
"name": "@narratage/media-pipeline",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:783aeed74e2586b3f4cff5350a27d06de9cb9472a237dabc0d5c68680cca3546",
- "closureDigest": "sha256:620abf19fcb09f9543773e0aa7d3e6639bc3828782bf7d24dfcc84929a9b700f"
+ "facetsDigest": "sha256:c93d4a7ced859956130610ebd14539663bbaed6152f3309eb7cb786cefd40458",
+ "closureDigest": "sha256:661de1ca6a2907fb93758800c63439de8fabce230238bfb78e0dbdcf1a5c17a3"
},
{
"specifier": "@narratage/narrative",
@@ -279,8 +293,8 @@
"name": "@narratage/narrative",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:849b7e49ef228b44d07ae57a106b18a24c7730c88783057bcdbad1e2e61153d3",
- "closureDigest": "sha256:e8030a009c4904f38373be51c372d5fcf0947aff81ed07317fce62b1930904cb"
+ "facetsDigest": "sha256:f4d042f2b821793beae3e33d1a35da83d45129e9e1bd33c54092a8abb75cccd7",
+ "closureDigest": "sha256:a324b56d3f1550ae1b94f3d178d1e184f7c04aa8fde796f8f10fa034d9778d9c"
},
{
"specifier": "@narratage/program-space",
@@ -297,8 +311,8 @@
"name": "@narratage/render-hyperframes",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:f1bf7447dfd5a468a97681a61684e00452588b1c4e10b6ee280be109055a7fc3",
- "closureDigest": "sha256:36d404f6f847aaf7606e924afd82d323eecd62d27573d3d6eaa0fb1a56e20c36"
+ "facetsDigest": "sha256:8a8726172d2396d54f8f99f483523827353ef1acb8fa3b050cd62ae16d9cd8ef",
+ "closureDigest": "sha256:03f37df3468dab6d4a9dfa0b727c36c040dbe251526cecdf45edf49e39a27804"
},
{
"specifier": "@narratage/run-text",
@@ -307,7 +321,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:fbcab7cea86223fe00ea04b51c2160bb534aafbb99067f5877e6825b0ce4eefc",
- "closureDigest": "sha256:16f3d7feeac702adf6962a0f6e651a42f25171ba899171f457452d99a57419f9"
+ "closureDigest": "sha256:a6d37eabad64b436b87c5edf4a4200412415b303be3c571fc1af6f6c3ad289db"
},
{
"specifier": "@narratage/script",
@@ -315,8 +329,8 @@
"name": "@narratage/script",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:2aea9956ef27554936214ce0738c0667da0cf8a860fafd63b4def5d22dab3ae8",
- "closureDigest": "sha256:ec9ffd07ba10bb8f906bbfb58351b33cf605cce7491e8ad308d7d580519230ff"
+ "facetsDigest": "sha256:c4500a649b05a9ba4b07bef7ee9cdc9d1d3ebaff76b9a678361131a7fca982f3",
+ "closureDigest": "sha256:ad9cf9128e425d6bff194b83a64e7ae719df5d04a7622f8dba6228232ca0a5b4"
},
{
"specifier": "@narratage/seedance",
@@ -324,8 +338,8 @@
"name": "@narratage/seedance",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:5d01251026ee31cc7ba33f40bb3d50d59a1dd5ccbc725b91fee3499c3988af67",
- "closureDigest": "sha256:e6ada0f380625c17f47b6638a25fa6ab6f86c80f3f310c5a74672cfb54d3c734"
+ "facetsDigest": "sha256:bcb5da66886c2e21be9193492db40e3585991cb3846ee3b014a1848b4a00986e",
+ "closureDigest": "sha256:e6d4aa8d6351bf7652760f94315dac192a484270cf5dbe33beb630da9f8086c3"
},
{
"specifier": "@narratage/semantic-map",
@@ -333,8 +347,8 @@
"name": "@narratage/semantic-map",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:4f041c5844a3eb9a70e8472d4cf4f534de960a77745819366dac0061ec600e76",
- "closureDigest": "sha256:d420abb0bf5c4b8815930b60a37040b646a939d7cc47f489df9100afb2670d16"
+ "facetsDigest": "sha256:40f349ffdc8a2f276c971f2e62a22d6e162d97ed47f608e33d5a2c6e51cd93a4",
+ "closureDigest": "sha256:5da5da001f31dfb0e09b96bf8fb3135ddc5e011f396b9ca4c69e46132e6f01c1"
},
{
"specifier": "@narratage/speech",
@@ -342,8 +356,8 @@
"name": "@narratage/speech",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:fbecf0c9334b34c91f49c4edd9f2592d7b49856ec2d8a875dd272f47ade29a18",
- "closureDigest": "sha256:1b016e5f394ee73a1740552fa370bb7b8d4554d87aefa8c5fa91b9e0a7568e42"
+ "facetsDigest": "sha256:606702bab5ea6bbab372be70e5d9d032975721c88754e64001929d20aa690bf7",
+ "closureDigest": "sha256:c07b24e334b2efc6af8b0c7f8a729a40f00fd73e51693e68118984963a9a8ca0"
},
{
"specifier": "@narratage/speech-alignment",
@@ -351,8 +365,8 @@
"name": "@narratage/speech-alignment",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:b3f48d736480bd0e1911a9195d9143ffb5c38a91bdcc55584845d6488cccc14f",
- "closureDigest": "sha256:1b3da03de14b740434bee79cc128d5e76f32816dedc1f7fcab1368a2eb8f26c5"
+ "facetsDigest": "sha256:ba23d1fe5fccd3c2210994a9d6e245bfed0b283e144e3040bbfcb621f440aefb",
+ "closureDigest": "sha256:502f3454cbdeb1a7769cbeb25b0ec792afc5180f73e40129ec745978362c6b18"
},
{
"specifier": "@narratage/speech-basis",
@@ -360,8 +374,8 @@
"name": "@narratage/speech-basis",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:b69ede6dba97a02452c0ceb824bb4373606a13a13ec24ae10901390774ac41e7",
- "closureDigest": "sha256:d40be58e90198d03ef08e52921e8178f9e4b2ee0dc23592c5f07ffa7bef576e9"
+ "facetsDigest": "sha256:c98ce13f17cc1a5ebb225d3335db8804a49f70965732deb46ad133941d58f02b",
+ "closureDigest": "sha256:57a24ce23026e4e4b1ca0d30ad58f55ce5840c252843653ebd5012105ef894d2"
},
{
"specifier": "@narratage/speech-evidence",
@@ -369,8 +383,8 @@
"name": "@narratage/speech-evidence",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:69110ba7b1740b0e609f6beef673fe4ef113ba5b0c803fb785ecf1dc9325f0a7",
- "closureDigest": "sha256:b0bc5b3bc1d303a3ea9964012acf37ee7cd5bb32840f81ed03eea109ed0f85c7"
+ "facetsDigest": "sha256:0de6289f6e0bc6b139e4c2fe063bde23877158f69be0935d506e83d2771a9af3",
+ "closureDigest": "sha256:805278f1a0cf261172314edf17896df0e836622d4a9bd7519d5b2d17fa42ce84"
},
{
"specifier": "@narratage/speech-spine",
@@ -378,8 +392,8 @@
"name": "@narratage/speech-spine",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:8c66979579d259706192dca1adec774a5eb6559e3e66a87a23d1939420f3e357",
- "closureDigest": "sha256:0e9f376a96c1b1491edee79dab9d1a5fcd8563f5a84eb5fd9d2f2228dee9dcd6"
+ "facetsDigest": "sha256:76bb403ab59040cd460c140b1f458a86305907a5c16076a75f745341a494092d",
+ "closureDigest": "sha256:592f7aed7f9d27366d579f2bc9d6d6a2ffde05b68e7e47dd08d8b27f306a3370"
},
{
"specifier": "@narratage/svs",
@@ -396,8 +410,8 @@
"name": "@narratage/text-track",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:a9d66c62cfd43269152d504d8500bfbe97a69939d5327afaa4e6bddd6c53e284",
- "closureDigest": "sha256:42ac8ae147f6df6e69709618e129fb00973f61dbe52e59bdf2809d05e608d140"
+ "facetsDigest": "sha256:e57b2aa7713a21b5eb200d7f537ec5ddf216421f80c35a8437d180c3998785bd",
+ "closureDigest": "sha256:68359a51ffe4a5aca13806a4652f9519f95ba5f7145f25d0440188e6c9b17312"
},
{
"specifier": "@narratage/visual-ir",
@@ -414,9 +428,9 @@
"name": "@narratage/whisperx",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:1899c422f35229f82da0a2b0ae341ff0234d6b174fc14ccaaf0ccc47624a58a9",
- "closureDigest": "sha256:c8f596b99ea0e634a7e1239f71af6f265dee8a3e78beccc670dc481396165fdd"
+ "facetsDigest": "sha256:d2c6ba803cd4fce5260bb40d5187bc71e0e5540d576d98a55689e36f01a40cde",
+ "closureDigest": "sha256:88a8d09bc98716c2ac5ab47a6fdbdd02e9422d9287aff2782aa9cdac1c870b97"
}
],
- "digest": "sha256:d56dc9e720df8517c2fe1046e45effe0c0dcf5e3de00d4fa7e7472e827afa11c"
+ "digest": "sha256:048b34e973b3e1d3685681f6bb86fbe3d69c3c4c13ead0204f73502be8345e07"
}
diff --git a/examples/talking-head-aroll/main.svml b/examples/talking-head-aroll/main.svml
index 895b1b35..9dd0c1d0 100644
--- a/examples/talking-head-aroll/main.svml
+++ b/examples/talking-head-aroll/main.svml
@@ -8,6 +8,7 @@
<import as="speech" from="@narratage/speech-spine@1"/>
<import as="whisperx" from="@narratage/whisperx@1"/>
<import as="caption" from="@narratage/caption@1"/>
+ <import as="caption-fine" from="@narratage/caption-fine@1"/>
<import as="caption-ai" from="@narratage/caption-gemini@1"/>
<import as="film" from="@narratage/film@1"/>
<import as="render" from="@narratage/render-hyperframes@1"/>
@@ -99,29 +100,22 @@
<!-- One measured evidence pass locates the complete authored Script. -->
<whisperx:Alignment id="timing" narrative={story} audio={speech.audio}/>
- <!-- Gemini sees display atoms and these requirements, never audio or timing. -->
- <caption:Style id="primary-caption" appearance={studio.caption.primary} mode="proportional-word">
- <caption:Cues>
- Split each Script Segment into short complete semantic phrases of two to seven words.
- Never cross a sentence or Segment boundary.
- </caption:Cues>
- <caption:Field id="important" type="boolean" min-per-cue="1" max-per-cue="2">
- Select one or two words whose emphasis best communicates this Cue.
- </caption:Field>
- </caption:Style>
-
- <caption:Program id="caption-program" narrative={story} default={primary-caption}/>
+ <!-- Gemini sees display words and these requirements, never audio or timing. -->
+ <caption-fine:Style id="primary-caption" recipe={studio.caption.primary}/>
+
+ <caption:Program id="caption-program" words={story.caption.words} default={primary-caption}/>
<caption-ai:Planner
id="caption-plan"
- narrative={story}
+ words={story.caption.words}
program={caption-program}
model="gemini-2.5-flash"
/>
<!-- Caption becomes an ordinary self-contained VisualTrack after joining the measured map. -->
- <caption:Track
+ <caption-fine:Track
id="captions"
narrative={story}
+ words={story.caption.words}
map={timing.map}
space={speech.space}
program={caption-program}
diff --git a/examples/talking-head-aroll/studio.svs b/examples/talking-head-aroll/studio.svs
index f0e0973a..8535f8f4 100644
--- a/examples/talking-head-aroll/studio.svs
+++ b/examples/talking-head-aroll/studio.svs
@@ -32,6 +32,12 @@
}
caption.primary {
+ cue-min-words: 2;
+ cue-max-words: 7;
+ important-min-per-cue: 1;
+ important-max-per-cue: 2;
+ important-fill: #FFF16A;
+ important-scale: 1.12;
stack-order: 70;
x: 0.08;
y: 0.74;
diff --git a/examples/talking-head-aroll/svml.packages.lock b/examples/talking-head-aroll/svml.packages.lock
index bd36d161..056080d1 100644
--- a/examples/talking-head-aroll/svml.packages.lock
+++ b/examples/talking-head-aroll/svml.packages.lock
@@ -6,15 +6,20 @@
"version": "0.0.0-dev",
"digest": "sha256:b16107fdad5256905782fe28142cd19e77673f190691e839fd12d4f1f536ea0f"
},
+ {
+ "name": "@narratage/caption-fine",
+ "version": "0.0.0-dev",
+ "digest": "sha256:0b803c8876f1838b25ff8d5a3511bd2f5510751940d4f2793db1eab4a9cc98cd"
+ },
{
"name": "@narratage/caption-gemini",
"version": "0.0.0-dev",
- "digest": "sha256:8a631e9345ea2c5901f3785058fde176a8280b9f20dd77e3333d651d56991797"
+ "digest": "sha256:2a8f75b53ffc67401aeb94f5d2b79afc4871ad2de11dab731ac65d74147b6aab"
},
{
"name": "@narratage/caption",
"version": "0.0.0-dev",
- "digest": "sha256:dc090e7d96c7a6c3a53c4aabd0874f1b743f02ecd7e05c81f50fef8cfa88ae6a"
+ "digest": "sha256:56883d4bed9bbf939d2c1267df7594b59902c240e18281e259f5eb0b5bca89e1"
},
{
"name": "@narratage/component-kit",
@@ -79,7 +84,7 @@
{
"name": "@narratage/narrative",
"version": "0.0.0-dev",
- "digest": "sha256:85e98e61021310c2dd6473189dcbcc9daa54039277169cbb5ed51ed70277a894"
+ "digest": "sha256:61e1d8b149008c83cabb05b532c5fae49d0aadf479a405dd9b7bf98ff3dd4a09"
},
{
"name": "@narratage/program-space",
@@ -114,7 +119,7 @@
{
"name": "@narratage/script",
"version": "0.0.0-dev",
- "digest": "sha256:5b741e43dd8966543d977e26a00e4e94973c57934e98037915f5fdface77518c"
+ "digest": "sha256:60860383d3df5645cc437e9a516b62db524a37d5dbe1acfa9a10008c4de5f709"
},
{
"name": "@narratage/seedance-speaker",
@@ -129,7 +134,7 @@
{
"name": "@narratage/semantic-map",
"version": "0.0.0-dev",
- "digest": "sha256:4293518a23ead1320f81d5d38e60bef96ea7f27149495bc890b9e4c275572864"
+ "digest": "sha256:425c1ea0d89039fc070de2a5de13b7d414d7a3a7d4c2e155445603fad3d60572"
},
{
"name": "@narratage/source",
@@ -169,7 +174,7 @@
{
"name": "@narratage/text",
"version": "0.0.0-dev",
- "digest": "sha256:99605f90eeca6d07428bd3e90741c3ddb36c75f8cd6a0e0958112d0994d13fd9"
+ "digest": "sha256:0558cd7cde9d4f0b8d6d880da23212f83d2328f1e531afc6eb7567e527125964"
},
{
"name": "@narratage/validation",
@@ -203,8 +208,17 @@
"name": "@narratage/caption",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:05d147f0231a47895852c6f1a42e049e8ecfa1487c6af2c091d39a0b151450bd",
- "closureDigest": "sha256:0bc1436bb393f3f640849a4b328a371e9a0c3703b5af6bb09c3737118a3b0102"
+ "facetsDigest": "sha256:2bde33270a3381610524b375782fe1bfd8cd73e75b9c93251e02b3d88b480ad0",
+ "closureDigest": "sha256:bce7573a505437d7e87900be19ba9b9e2470efd2b34b1504a56df285c38174e6"
+ },
+ {
+ "specifier": "@narratage/caption-fine",
+ "package": {
+ "name": "@narratage/caption-fine",
+ "version": "0.0.0-dev"
+ },
+ "facetsDigest": "sha256:24be8a66a292efc633e44781cee2ae391932e5b6a0ee917e2e14e02ef07f6afc",
+ "closureDigest": "sha256:cf273f4247731bd0dca0c693af35eae3ea5ee7c21357d8baac51442cc54e218c"
},
{
"specifier": "@narratage/caption-gemini",
@@ -212,8 +226,8 @@
"name": "@narratage/caption-gemini",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:e893c2186ed38bac9fe95840856ccfcbc8465e0ceca384d8fcb6830d7ab052b0",
- "closureDigest": "sha256:795108b2ab3574875fc53cab3d8fd74dbb1a2e68e4b3e1540a7209cbeaccb993"
+ "facetsDigest": "sha256:7a4037e49c13ceb17587d43b01b663c46c1a3af451d340fcac52d64b5aba4671",
+ "closureDigest": "sha256:40b5f28cb10b36aeed3871743fa43485a93b33315e0ac3ec817b5bae08188bb8"
},
{
"specifier": "@narratage/composition",
@@ -222,7 +236,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:ec680d429d88ec1c55531125a18c1620fea0c81600de55ba6d43c4fbf873e480",
- "closureDigest": "sha256:218aa1bb041c9701bb74cd0c5ac25df041b808d87b68f03c81f38eaf2e1641ae"
+ "closureDigest": "sha256:4a528d83d47df3588bd8223ed4f13b5497f54d8b9f1e50b3fba52ff0de7aacb9"
},
{
"specifier": "@narratage/estimate",
@@ -230,8 +244,8 @@
"name": "@narratage/estimate",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:0dc2391fa5ddaaf7e53136ba3ea13bab243db1e598cedce9f5296f0e0fc07dab",
- "closureDigest": "sha256:0228faf9bca5b950b753a49899f1e35df7049535aa66383da163dbb20ffe8bb7"
+ "facetsDigest": "sha256:eefe70d98cbf8e501d8102fc30d95778193b7985bf684a28b59a041b03ea10b6",
+ "closureDigest": "sha256:06ca52046a5fc0ebaecab78a060923a62f1d79e9505be02ec99df920cabb4c2a"
},
{
"specifier": "@narratage/film",
@@ -240,7 +254,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:61839fefb8cf723e9b55f4ac0932928b2dba5de4935ca03c0c4335246c6ab5d8",
- "closureDigest": "sha256:01563360044ec05fe1484609d3c37a634f467a31ac269211e2768ab6554f80eb"
+ "closureDigest": "sha256:fe2b5a6656a1fa1e7df66a13b5a6b35a7231c085ddab77508c2ceb686c6f48c3"
},
{
"specifier": "@narratage/generation",
@@ -258,7 +272,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:34bcacd52b6d396bf53f3b9709b43d54377a48c3b0332180079c3488a7fbe6ec",
- "closureDigest": "sha256:036c344ce50db44c6508d2cff8cf9fae7795618da062ce375bb0d916c2a8c3ca"
+ "closureDigest": "sha256:2473311ab583320ab5c6cb383dd1cb20f902b1891af2821a78fe9ed7e571b966"
},
{
"specifier": "@narratage/media",
@@ -267,7 +281,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:c883f03447734dc30a6c239c828083e6e9708258e457f845c2cfa5a2f95c0387",
- "closureDigest": "sha256:de7edf7b6436e4f35c782ffafd23f277419b015d32b024ba45ff30f8ef341fe8"
+ "closureDigest": "sha256:ec5993223ae3961246e318e1c036a73dfbc5c5918b4ae7aaebea242d6c9a8b9b"
},
{
"specifier": "@narratage/media-pipeline",
@@ -275,8 +289,8 @@
"name": "@narratage/media-pipeline",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:783aeed74e2586b3f4cff5350a27d06de9cb9472a237dabc0d5c68680cca3546",
- "closureDigest": "sha256:620abf19fcb09f9543773e0aa7d3e6639bc3828782bf7d24dfcc84929a9b700f"
+ "facetsDigest": "sha256:c93d4a7ced859956130610ebd14539663bbaed6152f3309eb7cb786cefd40458",
+ "closureDigest": "sha256:661de1ca6a2907fb93758800c63439de8fabce230238bfb78e0dbdcf1a5c17a3"
},
{
"specifier": "@narratage/narrative",
@@ -284,8 +298,8 @@
"name": "@narratage/narrative",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:849b7e49ef228b44d07ae57a106b18a24c7730c88783057bcdbad1e2e61153d3",
- "closureDigest": "sha256:e8030a009c4904f38373be51c372d5fcf0947aff81ed07317fce62b1930904cb"
+ "facetsDigest": "sha256:f4d042f2b821793beae3e33d1a35da83d45129e9e1bd33c54092a8abb75cccd7",
+ "closureDigest": "sha256:a324b56d3f1550ae1b94f3d178d1e184f7c04aa8fde796f8f10fa034d9778d9c"
},
{
"specifier": "@narratage/program-space",
@@ -311,8 +325,8 @@
"name": "@narratage/render-hyperframes",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:f1bf7447dfd5a468a97681a61684e00452588b1c4e10b6ee280be109055a7fc3",
- "closureDigest": "sha256:36d404f6f847aaf7606e924afd82d323eecd62d27573d3d6eaa0fb1a56e20c36"
+ "facetsDigest": "sha256:8a8726172d2396d54f8f99f483523827353ef1acb8fa3b050cd62ae16d9cd8ef",
+ "closureDigest": "sha256:03f37df3468dab6d4a9dfa0b727c36c040dbe251526cecdf45edf49e39a27804"
},
{
"specifier": "@narratage/run-text",
@@ -321,7 +335,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:fbcab7cea86223fe00ea04b51c2160bb534aafbb99067f5877e6825b0ce4eefc",
- "closureDigest": "sha256:16f3d7feeac702adf6962a0f6e651a42f25171ba899171f457452d99a57419f9"
+ "closureDigest": "sha256:a6d37eabad64b436b87c5edf4a4200412415b303be3c571fc1af6f6c3ad289db"
},
{
"specifier": "@narratage/script",
@@ -329,8 +343,8 @@
"name": "@narratage/script",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:2aea9956ef27554936214ce0738c0667da0cf8a860fafd63b4def5d22dab3ae8",
- "closureDigest": "sha256:ec9ffd07ba10bb8f906bbfb58351b33cf605cce7491e8ad308d7d580519230ff"
+ "facetsDigest": "sha256:c4500a649b05a9ba4b07bef7ee9cdc9d1d3ebaff76b9a678361131a7fca982f3",
+ "closureDigest": "sha256:ad9cf9128e425d6bff194b83a64e7ae719df5d04a7622f8dba6228232ca0a5b4"
},
{
"specifier": "@narratage/seedance",
@@ -338,8 +352,8 @@
"name": "@narratage/seedance",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:5d01251026ee31cc7ba33f40bb3d50d59a1dd5ccbc725b91fee3499c3988af67",
- "closureDigest": "sha256:e6ada0f380625c17f47b6638a25fa6ab6f86c80f3f310c5a74672cfb54d3c734"
+ "facetsDigest": "sha256:bcb5da66886c2e21be9193492db40e3585991cb3846ee3b014a1848b4a00986e",
+ "closureDigest": "sha256:e6d4aa8d6351bf7652760f94315dac192a484270cf5dbe33beb630da9f8086c3"
},
{
"specifier": "@narratage/seedance-speaker",
@@ -347,8 +361,8 @@
"name": "@narratage/seedance-speaker",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:2872bd09ced07727dc023a7d0c9319aca4d61d4a1a266679075489a0b593641a",
- "closureDigest": "sha256:d629674758d24d2d37364d7d09894bc82cca116dd2ad4485b9060305799ba034"
+ "facetsDigest": "sha256:a1d08a41b12a83684754d636d4a0290957bbc970b7d49643774fa56597ec5865",
+ "closureDigest": "sha256:90bd5bc73e49b63b93c8964042b4855c0d53af9d441bbb1eb4f10a1e20930041"
},
{
"specifier": "@narratage/semantic-map",
@@ -356,8 +370,8 @@
"name": "@narratage/semantic-map",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:4f041c5844a3eb9a70e8472d4cf4f534de960a77745819366dac0061ec600e76",
- "closureDigest": "sha256:d420abb0bf5c4b8815930b60a37040b646a939d7cc47f489df9100afb2670d16"
+ "facetsDigest": "sha256:40f349ffdc8a2f276c971f2e62a22d6e162d97ed47f608e33d5a2c6e51cd93a4",
+ "closureDigest": "sha256:5da5da001f31dfb0e09b96bf8fb3135ddc5e011f396b9ca4c69e46132e6f01c1"
},
{
"specifier": "@narratage/speech",
@@ -365,8 +379,8 @@
"name": "@narratage/speech",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:fbecf0c9334b34c91f49c4edd9f2592d7b49856ec2d8a875dd272f47ade29a18",
- "closureDigest": "sha256:1b016e5f394ee73a1740552fa370bb7b8d4554d87aefa8c5fa91b9e0a7568e42"
+ "facetsDigest": "sha256:606702bab5ea6bbab372be70e5d9d032975721c88754e64001929d20aa690bf7",
+ "closureDigest": "sha256:c07b24e334b2efc6af8b0c7f8a729a40f00fd73e51693e68118984963a9a8ca0"
},
{
"specifier": "@narratage/speech-alignment",
@@ -374,8 +388,8 @@
"name": "@narratage/speech-alignment",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:b3f48d736480bd0e1911a9195d9143ffb5c38a91bdcc55584845d6488cccc14f",
- "closureDigest": "sha256:1b3da03de14b740434bee79cc128d5e76f32816dedc1f7fcab1368a2eb8f26c5"
+ "facetsDigest": "sha256:ba23d1fe5fccd3c2210994a9d6e245bfed0b283e144e3040bbfcb621f440aefb",
+ "closureDigest": "sha256:502f3454cbdeb1a7769cbeb25b0ec792afc5180f73e40129ec745978362c6b18"
},
{
"specifier": "@narratage/speech-basis",
@@ -383,8 +397,8 @@
"name": "@narratage/speech-basis",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:b69ede6dba97a02452c0ceb824bb4373606a13a13ec24ae10901390774ac41e7",
- "closureDigest": "sha256:d40be58e90198d03ef08e52921e8178f9e4b2ee0dc23592c5f07ffa7bef576e9"
+ "facetsDigest": "sha256:c98ce13f17cc1a5ebb225d3335db8804a49f70965732deb46ad133941d58f02b",
+ "closureDigest": "sha256:57a24ce23026e4e4b1ca0d30ad58f55ce5840c252843653ebd5012105ef894d2"
},
{
"specifier": "@narratage/speech-evidence",
@@ -392,8 +406,8 @@
"name": "@narratage/speech-evidence",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:69110ba7b1740b0e609f6beef673fe4ef113ba5b0c803fb785ecf1dc9325f0a7",
- "closureDigest": "sha256:b0bc5b3bc1d303a3ea9964012acf37ee7cd5bb32840f81ed03eea109ed0f85c7"
+ "facetsDigest": "sha256:0de6289f6e0bc6b139e4c2fe063bde23877158f69be0935d506e83d2771a9af3",
+ "closureDigest": "sha256:805278f1a0cf261172314edf17896df0e836622d4a9bd7519d5b2d17fa42ce84"
},
{
"specifier": "@narratage/speech-spine",
@@ -401,8 +415,8 @@
"name": "@narratage/speech-spine",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:8c66979579d259706192dca1adec774a5eb6559e3e66a87a23d1939420f3e357",
- "closureDigest": "sha256:0e9f376a96c1b1491edee79dab9d1a5fcd8563f5a84eb5fd9d2f2228dee9dcd6"
+ "facetsDigest": "sha256:76bb403ab59040cd460c140b1f458a86305907a5c16076a75f745341a494092d",
+ "closureDigest": "sha256:592f7aed7f9d27366d579f2bc9d6d6a2ffde05b68e7e47dd08d8b27f306a3370"
},
{
"specifier": "@narratage/svs",
@@ -428,9 +442,9 @@
"name": "@narratage/whisperx",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:1899c422f35229f82da0a2b0ae341ff0234d6b174fc14ccaaf0ccc47624a58a9",
- "closureDigest": "sha256:c8f596b99ea0e634a7e1239f71af6f265dee8a3e78beccc670dc481396165fdd"
+ "facetsDigest": "sha256:d2c6ba803cd4fce5260bb40d5187bc71e0e5540d576d98a55689e36f01a40cde",
+ "closureDigest": "sha256:88a8d09bc98716c2ac5ab47a6fdbdd02e9422d9287aff2782aa9cdac1c870b97"
}
],
- "digest": "sha256:c2356e0130b1fb28274d49ec02dd2e996673787e0f8149c71e528d45ee62039f"
+ "digest": "sha256:d15e5202ee8594ed492642182f70499e1eaeb238045f3c1e7134ddd0460ad1ef"
}
diff --git a/package.json b/package.json
index 1ce7fa70..fe51d4ae 100644
--- a/package.json
+++ b/package.json
@@ -24,6 +24,7 @@
"@narratage/artifact": "workspace:*",
"@narratage/broll": "workspace:*",
"@narratage/caption": "workspace:*",
+ "@narratage/caption-fine": "workspace:*",
"@narratage/caption-gemini": "workspace:*",
"@narratage/composition": "workspace:*",
"@narratage/estimate": "workspace:*",
diff --git a/packages/caption-fine/README.md b/packages/caption-fine/README.md
new file mode 100644
index 00000000..790f839b
--- /dev/null
+++ b/packages/caption-fine/README.md
@@ -0,0 +1,20 @@
+# `@narratage/caption-fine`
+
+The first official fine-grained Caption Style family. It turns an SVS Recipe into one complete
+`CaptionStyle`: common Cue bounds, this family's `important` planning field, and the exact rendering
+parameters that understand that field.
+
+```xml
+<fine:Style id="primary" recipe={studio.caption.primary}/>
+
+<caption:Program id="captions" words={story.caption.words} default={primary}>
+ <caption:Use words={story.caption.selection.callout} style={callout}/>
+</caption:Program>
+
+<fine:Track id="captions-track" narrative={story} words={story.caption.words} map={timing.map}
+ program={captions} plan={caption-plan.plan} space={speech.space}/>
+```
+
+One `fine:Track` handles the default and all ordered overrides and emits one ordinary
+`VisualTrack`. The common `@narratage/caption` package does not know this package's fonts, boxes,
+colors or the meaning of `important`.
diff --git a/packages/caption-fine/package.json b/packages/caption-fine/package.json
new file mode 100644
index 00000000..920b1fef
--- /dev/null
+++ b/packages/caption-fine/package.json
@@ -0,0 +1,27 @@
+{
+ "name": "@narratage/caption-fine",
+ "version": "0.0.0-dev",
+ "private": true,
+ "type": "module",
+ "exports": {
+ ".": "./src/index.ts"
+ },
+ "svml": {
+ "activation": "./src/activation.ts"
+ },
+ "dependencies": {
+ "@narratage/caption": "workspace:*",
+ "@narratage/component-kit": "workspace:*",
+ "@narratage/composition": "workspace:*",
+ "@narratage/elaborator": "workspace:*",
+ "@narratage/narrative": "workspace:*",
+ "@narratage/program-space": "workspace:*",
+ "@narratage/protocol": "workspace:*",
+ "@narratage/semantic-map": "workspace:*",
+ "@narratage/svs": "workspace:*",
+ "@narratage/text": "workspace:*"
+ },
+ "devDependencies": {
+ "@narratage/script": "workspace:*"
+ }
+}
diff --git a/packages/caption-fine/src/activation.ts b/packages/caption-fine/src/activation.ts
new file mode 100644
index 00000000..a9b0e5ff
--- /dev/null
+++ b/packages/caption-fine/src/activation.ts
@@ -0,0 +1,39 @@
+import { createTextSurfaceHostFacet } from "@narratage/text";
+
+import {
+ captionFineComponent,
+ captionFineManifest,
+ captionFineModuleRef,
+ captionFineStyleSurfaceImplementationDigest,
+ captionFineTrackSurfaceImplementationDigest,
+ decodeFineCaptionStyleSurface,
+ decodeFineCaptionTrackSurface,
+} from "./index.js";
+
+export const svmlPackage = {
+ format: "svml.node-package@1" as const,
+ name: "@narratage/caption-fine",
+ modules: [{
+ manifest: captionFineManifest,
+ specifiers: ["@narratage/caption-fine", "@narratage/caption-fine@1"],
+ }],
+ components: [captionFineComponent],
+ hostFacets: [
+ createTextSurfaceHostFacet({
+ module: captionFineModuleRef,
+ surface: "style",
+ mode: "structured",
+ implementationDigest: captionFineStyleSurfaceImplementationDigest,
+ handler: decodeFineCaptionStyleSurface,
+ }),
+ createTextSurfaceHostFacet({
+ module: captionFineModuleRef,
+ surface: "track",
+ mode: "structured",
+ implementationDigest: captionFineTrackSurfaceImplementationDigest,
+ handler: decodeFineCaptionTrackSurface,
+ }),
+ ],
+};
+
+export default svmlPackage;
diff --git a/packages/caption-fine/src/component.ts b/packages/caption-fine/src/component.ts
new file mode 100644
index 00000000..ee6062b2
--- /dev/null
+++ b/packages/caption-fine/src/component.ts
@@ -0,0 +1,31 @@
+import type { CaptionProgram, TimedCaptionProjection } from "@narratage/caption";
+import type { ComponentPackage } from "@narratage/component-kit";
+import type { CaptionWordSequence } from "@narratage/narrative";
+import type { ProgramSpace } from "@narratage/program-space";
+import type { StoredValue } from "@narratage/protocol";
+import { canonicalize } from "@narratage/protocol";
+
+import { captionFineProducers } from "./manifest.js";
+import { renderFineCaption, renderFineCaptionImplementationDigest } from "./render.js";
+
+function inline<T>(value: StoredValue | undefined, subject: string): T {
+ if (value?.kind !== "inline") throw new Error(`${subject} must be inline`);
+ return value.value as T;
+}
+
+export const captionFineComponent = {
+ name: "@narratage/caption-fine",
+ producers: [{
+ producer: captionFineProducers.render,
+ implementationDigest: renderFineCaptionImplementationDigest,
+ handler: ({ inputs }) => ({
+ outputs: { track: { kind: "inline", value: canonicalize(renderFineCaption(
+ inline<TimedCaptionProjection>(inputs.caption?.value, "TimedCaptionProjection"),
+ inline<CaptionProgram>(inputs.program?.value, "CaptionProgram"),
+ inline<CaptionWordSequence>(inputs.words?.value, "CaptionWordSequence"),
+ inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),
+ )) } },
+ needs: {},
+ }),
+ }],
+} satisfies ComponentPackage;
diff --git a/packages/caption-fine/src/fragment.ts b/packages/caption-fine/src/fragment.ts
new file mode 100644
index 00000000..f60381cf
--- /dev/null
+++ b/packages/caption-fine/src/fragment.ts
@@ -0,0 +1,51 @@
+import { captionProducers, captionTypes } from "@narratage/caption";
+import { compositionTypes } from "@narratage/composition";
+import { sealGraphFragment } from "@narratage/elaborator";
+import { narrativeTypes } from "@narratage/narrative";
+import { programSpaceTypes } from "@narratage/program-space";
+import { semanticMapTypes } from "@narratage/semantic-map";
+
+import { captionFineProducers } from "./manifest.js";
+
+const input = (name: string) => ({ kind: "fragment-input" as const, name });
+const operation = (id: string) => ({ kind: "fragment-operation" as const, operation: id });
+
+/** One timing join plus one Style-family render operation, regardless of how many Styles are used. */
+export const fineCaptionTrackFragment = sealGraphFragment({
+ name: "@narratage/caption-fine/track@1",
+ inputs: [
+ { name: "narrative", type: narrativeTypes.narrative },
+ { name: "map", type: semanticMapTypes.complete },
+ { name: "words", type: narrativeTypes.captionWordSequence },
+ { name: "plan", type: captionTypes.plan },
+ { name: "program", type: captionTypes.program },
+ { name: "space", type: programSpaceTypes.programSpace },
+ ],
+ operations: [
+ {
+ id: "caption:temporalize-plan",
+ producer: captionProducers.temporalizePlan,
+ inputs: {
+ narrative: input("narrative"), map: input("map"), words: input("words"),
+ plan: input("plan"), program: input("program"),
+ },
+ result: { kind: "output", name: "caption" },
+ },
+ {
+ id: "caption-fine:render",
+ producer: captionFineProducers.render,
+ inputs: {
+ caption: operation("caption:temporalize-plan"), program: input("program"),
+ words: input("words"), space: input("space"),
+ },
+ result: { kind: "output", name: "track" },
+ },
+ ],
+ exports: [{
+ name: "track",
+ type: compositionTypes.visualTrack,
+ root: operation("caption-fine:render"),
+ semanticInputs: ["narrative", "map", "words", "plan", "program", "space"],
+ fidelity: "exact",
+ }],
+});
diff --git a/packages/caption-fine/src/index.ts b/packages/caption-fine/src/index.ts
new file mode 100644
index 00000000..431ef825
--- /dev/null
+++ b/packages/caption-fine/src/index.ts
@@ -0,0 +1,18 @@
+export { captionFineComponent } from "./component.js";
+export { fineCaptionTrackFragment } from "./fragment.js";
+export {
+ captionFineManifest,
+ captionFineModuleRef,
+ captionFineProducers,
+ captionFineStyleSurfaceImplementationDigest,
+ captionFineTrackSurfaceImplementationDigest,
+} from "./manifest.js";
+export { renderFineCaption, renderFineCaptionImplementationDigest } from "./render.js";
+export {
+ assertFineCaptionParameters,
+ FINE_CAPTION_FAMILY,
+ fineCaptionParameters,
+ fineCaptionStyle,
+} from "./style.js";
+export { decodeFineCaptionStyleSurface, decodeFineCaptionTrackSurface } from "./surface.js";
+export type * from "./types.js";
diff --git a/packages/caption-fine/src/manifest.ts b/packages/caption-fine/src/manifest.ts
new file mode 100644
index 00000000..ef37437d
--- /dev/null
+++ b/packages/caption-fine/src/manifest.ts
@@ -0,0 +1,56 @@
+import { captionManifest, captionModuleRef, captionTypes } from "@narratage/caption";
+import { compositionDependency, compositionTypes } from "@narratage/composition";
+import { programSpaceDependency, programSpaceTypes } from "@narratage/program-space";
+import { narrativeDependency, narrativeTypes } from "@narratage/narrative";
+import { semanticMapDependency } from "@narratage/semantic-map";
+import { digestOf } from "@narratage/protocol";
+import type { ModuleManifest, ProducerRef } from "@narratage/protocol";
+
+import { renderFineCaptionImplementationDigest } from "./render.js";
+
+export const captionFineModuleRef = { name: "@narratage/caption-fine", version: "0.0.0-dev" } as const;
+export const captionFineProducers = {
+ render: { module: captionFineModuleRef, name: "render-fine-caption" },
+} satisfies Record<string, ProducerRef>;
+export const captionFineStyleSurfaceImplementationDigest = digestOf("@narratage/caption-fine/style-surface@1");
+export const captionFineTrackSurfaceImplementationDigest = digestOf("@narratage/caption-fine/track-surface@1");
+
+export const captionFineManifest: ModuleManifest = {
+ format: "svml.module@1",
+ name: captionFineModuleRef.name,
+ version: captionFineModuleRef.version,
+ dependencies: [
+ { module: captionModuleRef, digest: digestOf(captionManifest) },
+ compositionDependency,
+ narrativeDependency,
+ programSpaceDependency,
+ semanticMapDependency,
+ ],
+ types: [],
+ capabilities: [],
+ surfaces: [
+ {
+ name: "style", tag: "Style", mode: "structured", outputs: [captionTypes.style],
+ implementation: { kind: "trusted-frontend-surface", locator: "@narratage/caption-fine/style-surface",
+ digest: captionFineStyleSurfaceImplementationDigest },
+ },
+ {
+ name: "track", tag: "Track", mode: "structured", outputs: [compositionTypes.visualTrack],
+ implementation: { kind: "trusted-frontend-surface", locator: "@narratage/caption-fine/track-surface",
+ digest: captionFineTrackSurfaceImplementationDigest },
+ },
+ ],
+ producers: [{
+ name: captionFineProducers.render.name,
+ inputs: [
+ { name: "caption", type: captionTypes.timedProjection },
+ { name: "program", type: captionTypes.program },
+ { name: "words", type: narrativeTypes.captionWordSequence },
+ { name: "space", type: programSpaceTypes.programSpace },
+ ],
+ outputs: [{ name: "track", type: compositionTypes.visualTrack }],
+ needs: [],
+ implementation: { kind: "registered", locator: "@narratage/caption-fine/render",
+ digest: renderFineCaptionImplementationDigest },
+ }],
+};
diff --git a/packages/caption-fine/src/render.ts b/packages/caption-fine/src/render.ts
new file mode 100644
index 00000000..37412ad2
--- /dev/null
+++ b/packages/caption-fine/src/render.ts
@@ -0,0 +1,159 @@
+import { assertCaptionProgramForWords, assertTimedCaptionProjection } from "@narratage/caption";
+import type {
+ CaptionFieldAssignment,
+ CaptionProgram,
+ TimedCaptionProjection,
+} from "@narratage/caption";
+import { assertVisualTrackIdentity, sealVisualTrack } from "@narratage/composition";
+import type { VisualElement, VisualStyleDeclaration, VisualTrack } from "@narratage/composition";
+import type { CaptionWordSequence } from "@narratage/narrative";
+import { assertProgramSpaceIdentity, programSpaceFrameCount } from "@narratage/program-space";
+import type { ProgramSpace } from "@narratage/program-space";
+import { digestOf } from "@narratage/protocol";
+
+import { assertFineCaptionParameters, FINE_CAPTION_FAMILY } from "./style.js";
+import type { FineCaptionParameters } from "./types.js";
+
+export const renderFineCaptionImplementationDigest = digestOf("@narratage/caption-fine/render@1");
+
+function frameAt(space: ProgramSpace, seconds: number): number {
+ return Math.round(seconds * space.frameRate.numerator / space.frameRate.denominator);
+}
+
+function captionPieces(
+ display: string,
+ wordIds: readonly string[],
+ wordText: ReadonlyMap<string, string>,
+): string[] {
+ let cursor = 0;
+ const starts = wordIds.map((wordId) => {
+ const text = wordText.get(wordId);
+ if (text === undefined) throw new Error(`Fine Caption Cue references unknown word ${wordId}`);
+ const start = display.indexOf(text, cursor);
+ if (start < cursor) throw new Error(`Fine Caption Cue cannot locate word ${wordId} in its authored display`);
+ cursor = start + text.length;
+ return start;
+ });
+ return starts.map((start, index) => display.slice(index === 0 ? 0 : start, starts[index + 1] ?? display.length));
+}
+
+function textStyle(parameters: FineCaptionParameters, important: boolean): VisualStyleDeclaration[] {
+ return [
+ { name: "color", value: important ? parameters.important.fill : parameters.typography.fill },
+ { name: "font-family", value: parameters.typography.fontFamily },
+ { name: "font-size", value: `${parameters.typography.fontSizePx}px` },
+ { name: "font-weight", value: parameters.typography.fontWeight },
+ { name: "line-height", value: parameters.typography.lineHeight },
+ { name: "white-space", value: "pre-wrap" },
+ ...(important ? [{ name: "transform", value: `scale(${parameters.important.scale})` }] : []),
+ ];
+}
+
+function isImportant(fields: readonly CaptionFieldAssignment[], wordId: string): boolean {
+ return fields.some((field) =>
+ field.declarationId === "important" && field.wordId === wordId && field.value === "true");
+}
+
+function cueElements(
+ display: string,
+ wordIds: readonly string[],
+ fields: readonly CaptionFieldAssignment[],
+ parameters: FineCaptionParameters,
+ wordText: ReadonlyMap<string, string>,
+): VisualElement[] {
+ const pieces = captionPieces(display, wordIds, wordText);
+ return [
+ {
+ id: "placement",
+ order: 0,
+ kind: "box",
+ style: [
+ { name: "align-items", value: "center" },
+ { name: "display", value: "flex" },
+ { name: "justify-content", value: parameters.typography.textAlign === "left" ? "flex-start"
+ : parameters.typography.textAlign === "right" ? "flex-end" : "center" },
+ { name: "left", value: `${parameters.placement.x * 100}%` },
+ { name: "position", value: "absolute" },
+ { name: "top", value: `${parameters.placement.y * 100}%` },
+ { name: "width", value: `${parameters.placement.width * 100}%` },
+ ],
+ },
+ {
+ id: "cue",
+ parent: "placement",
+ order: 1,
+ kind: "box",
+ style: [
+ { name: "background", value: parameters.box.background },
+ { name: "border-radius", value: `${parameters.box.radiusPx}px` },
+ { name: "display", value: "flex" },
+ { name: "flex-wrap", value: "wrap" },
+ { name: "justify-content", value: parameters.typography.textAlign === "left" ? "flex-start"
+ : parameters.typography.textAlign === "right" ? "flex-end" : "center" },
+ { name: "padding", value: `${parameters.box.paddingYPx}px ${parameters.box.paddingXPx}px` },
+ { name: "text-align", value: parameters.typography.textAlign },
+ ],
+ },
+ ...wordIds.map((wordId, index): VisualElement => {
+ const important = isImportant(fields, wordId);
+ return {
+ id: `word-${index + 1}`,
+ parent: "cue",
+ order: index + 2,
+ kind: "text",
+ text: pieces[index]!,
+ style: textStyle(parameters, important),
+ attributes: [
+ { name: "data-caption-word", value: wordId },
+ ...(important ? [{ name: "data-caption-field-important", value: "true" }] : []),
+ ],
+ };
+ }),
+ ];
+}
+
+export function renderFineCaption(
+ projection: TimedCaptionProjection,
+ program: CaptionProgram,
+ words: CaptionWordSequence,
+ space: ProgramSpace,
+): VisualTrack {
+ assertTimedCaptionProjection(projection);
+ assertCaptionProgramForWords(program, words);
+ assertProgramSpaceIdentity(space);
+ const styles = new Map(program.styles.map((style) => [style.id, style]));
+ const wordText = new Map(words.words.map((word) => [word.id, word.text]));
+ for (const style of styles.values()) {
+ if (style.rendering.family !== FINE_CAPTION_FAMILY) {
+ throw new Error(`Fine Caption cannot render Style family ${style.rendering.family}`);
+ }
+ assertFineCaptionParameters(style.rendering.parameters as unknown as FineCaptionParameters);
+ }
+ const totalFrames = programSpaceFrameCount(space);
+ const presents = projection.regions.flatMap((region) => {
+ if (region.styleId === undefined || region.wordIds === undefined || region.wordIds.length === 0) {
+ throw new Error(`Fine Caption region ${region.id} lacks resolved Style or word identities`);
+ }
+ const style = styles.get(region.styleId);
+ if (style === undefined) throw new Error(`Fine Caption region ${region.id} references unknown Style ${region.styleId}`);
+ const parameters = style.rendering.parameters as unknown as FineCaptionParameters;
+ const startFrame = Math.max(0, frameAt(space, region.startSec));
+ const measuredEnd = Math.min(totalFrames, frameAt(space, region.endSec));
+ const endFrameExclusive = Math.min(totalFrames, Math.max(startFrame + 1, measuredEnd));
+ if (!region.display || startFrame >= totalFrames || endFrameExclusive <= startFrame) return [];
+ return [{
+ id: region.id,
+ span: { startFrame, endFrameExclusive },
+ stacking: { order: parameters.stackingOrder, tieBreak: `${program.id}:${region.id}` },
+ elements: cueElements(region.display, region.wordIds, region.fields ?? [], parameters, wordText),
+ }];
+ });
+ const track = sealVisualTrack({
+ contract: "svml.visual-track@1",
+ visualIr: "svml.visual-ir@1",
+ id: program.id,
+ presents,
+ });
+ assertVisualTrackIdentity(track, space);
+ return track;
+}
diff --git a/packages/caption-fine/src/style.ts b/packages/caption-fine/src/style.ts
new file mode 100644
index 00000000..1618dfd6
--- /dev/null
+++ b/packages/caption-fine/src/style.ts
@@ -0,0 +1,139 @@
+import type { CaptionStyleIntent } from "@narratage/caption";
+import { sealCaptionStyle } from "@narratage/caption";
+import type { SvsRecipe } from "@narratage/svs";
+
+import type { FineCaptionParameters } from "./types.js";
+
+export const FINE_CAPTION_FAMILY = "@narratage/caption-fine@1";
+
+function number(value: SvsRecipe, name: string): number {
+ const property = value.properties[name];
+ if (typeof property !== "number" || !Number.isFinite(property)) {
+ throw new Error(`Fine Caption Recipe ${name} must be a number`);
+ }
+ return property;
+}
+
+function integer(value: SvsRecipe, name: string): number {
+ const property = number(value, name);
+ if (!Number.isSafeInteger(property)) throw new Error(`Fine Caption Recipe ${name} must be an integer`);
+ return property;
+}
+
+function string(value: SvsRecipe, name: string): string {
+ const property = value.properties[name];
+ if (typeof property !== "string" || !property.trim()) {
+ throw new Error(`Fine Caption Recipe ${name} must be a string`);
+ }
+ return property.trim();
+}
+
+function color(value: SvsRecipe, name: string): string {
+ const property = string(value, name);
+ if (!/^#[0-9a-f]{6}(?:[0-9a-f]{2})?$/iu.test(property)) {
+ throw new Error(`Fine Caption Recipe ${name} must be an RGB or RGBA hex color`);
+ }
+ return property;
+}
+
+function padding(value: string): { readonly x: number; readonly y: number } {
+ const parts = value.trim().split(/\s+/u).map(Number);
+ if ((parts.length !== 1 && parts.length !== 2) || parts.some((item) => !Number.isFinite(item) || item < 0)) {
+ throw new Error("Fine Caption Recipe padding must contain one or two non-negative pixel numbers");
+ }
+ return { y: parts[0]!, x: parts[1] ?? parts[0]! };
+}
+
+export function fineCaptionParameters(recipe: SvsRecipe): FineCaptionParameters {
+ const expected = [
+ "align", "background", "cue-max-words", "cue-min-words", "fill", "font",
+ "important-fill", "important-max-per-cue", "important-min-per-cue", "important-scale",
+ "line-height", "padding", "radius", "size", "stack-order", "weight", "width", "x", "y",
+ ];
+ if (Object.keys(recipe.properties).sort().join("\0") !== expected.sort().join("\0")) {
+ throw new Error(`Fine Caption Recipe requires exactly ${expected.join(", ")}`);
+ }
+ const align = string(recipe, "align");
+ if (align !== "left" && align !== "center" && align !== "right") {
+ throw new Error("Fine Caption Recipe align is invalid");
+ }
+ const pad = padding(string(recipe, "padding"));
+ const parameters: FineCaptionParameters = {
+ contract: "svml.caption-fine-parameters@1",
+ stackingOrder: integer(recipe, "stack-order"),
+ placement: { x: number(recipe, "x"), y: number(recipe, "y"), width: number(recipe, "width") },
+ typography: {
+ fontFamily: string(recipe, "font"),
+ fontSizePx: number(recipe, "size"),
+ fontWeight: integer(recipe, "weight"),
+ lineHeight: number(recipe, "line-height"),
+ textAlign: align,
+ fill: color(recipe, "fill"),
+ },
+ box: {
+ background: color(recipe, "background"),
+ paddingXPx: pad.x,
+ paddingYPx: pad.y,
+ radiusPx: number(recipe, "radius"),
+ },
+ important: { fill: color(recipe, "important-fill"), scale: number(recipe, "important-scale") },
+ };
+ assertFineCaptionParameters(parameters);
+ return parameters;
+}
+
+export function assertFineCaptionParameters(value: FineCaptionParameters): void {
+ if (value.contract !== "svml.caption-fine-parameters@1") throw new Error("Unsupported Fine Caption parameters");
+ if (!Number.isSafeInteger(value.stackingOrder) || value.stackingOrder < 0) {
+ throw new Error("Fine Caption stacking order is invalid");
+ }
+ const numeric = [
+ value.placement.x, value.placement.y, value.placement.width, value.typography.fontSizePx,
+ value.typography.fontWeight, value.typography.lineHeight, value.box.paddingXPx, value.box.paddingYPx,
+ value.box.radiusPx, value.important.scale,
+ ];
+ if (numeric.some((item) => !Number.isFinite(item) || item < 0)
+ || value.placement.x > 1 || value.placement.y > 1 || value.placement.width <= 0 || value.placement.width > 1
+ || value.typography.fontSizePx <= 0 || value.typography.lineHeight <= 0 || value.important.scale <= 0) {
+ throw new Error("Fine Caption parameters contain invalid numeric bounds");
+ }
+ if (!value.typography.fontFamily.trim()) throw new Error("Fine Caption font family is empty");
+ for (const colorValue of [value.typography.fill, value.box.background, value.important.fill]) {
+ if (!/^#[0-9a-f]{6}(?:[0-9a-f]{2})?$/iu.test(colorValue)) throw new Error("Fine Caption color is invalid");
+ }
+}
+
+export function fineCaptionStyle(id: string, recipe: SvsRecipe): CaptionStyleIntent {
+ const minimumWords = integer(recipe, "cue-min-words");
+ const maximumWords = integer(recipe, "cue-max-words");
+ const importantMinimum = integer(recipe, "important-min-per-cue");
+ const importantMaximum = integer(recipe, "important-max-per-cue");
+ if (minimumWords <= 0 || maximumWords < minimumWords) {
+ throw new Error("Fine Caption Recipe Cue word bounds are invalid");
+ }
+ if (importantMinimum < 0 || importantMaximum < importantMinimum || importantMaximum > maximumWords) {
+ throw new Error("Fine Caption Recipe important bounds are invalid");
+ }
+ return sealCaptionStyle({
+ contract: "svml.caption-style@1",
+ id,
+ planning: {
+ cue: {
+ minimumWords,
+ maximumWords,
+ instruction: `Split into complete semantic phrases of ${minimumWords} to ${maximumWords} display words. Never cross punctuation when avoidable.`,
+ },
+ fields: importantMaximum === 0 ? [] : [{
+ id: "important",
+ value: { kind: "boolean" },
+ instruction: "Select the words whose emphasis best communicates this Cue. The words need not be contiguous.",
+ minimumPerCue: importantMinimum,
+ maximumPerCue: importantMaximum,
+ }],
+ },
+ rendering: {
+ family: FINE_CAPTION_FAMILY,
+ parameters: fineCaptionParameters(recipe),
+ },
+ });
+}
diff --git a/packages/caption-fine/src/surface.ts b/packages/caption-fine/src/surface.ts
new file mode 100644
index 00000000..491adc85
--- /dev/null
+++ b/packages/caption-fine/src/surface.ts
@@ -0,0 +1,96 @@
+import { captionTypes } from "@narratage/caption";
+import { narrativeTypes } from "@narratage/narrative";
+import { programSpaceTypes } from "@narratage/program-space";
+import { semanticMapTypes } from "@narratage/semantic-map";
+import { svsRecipeType } from "@narratage/svs";
+import type { SvsRecipe } from "@narratage/svs";
+import type {
+ StructuredElement,
+ StructuredSurfaceHandler,
+ SurfaceResolvedReference,
+ TextAttributeValue,
+} from "@narratage/text";
+
+import { fineCaptionTrackFragment } from "./fragment.js";
+import { fineCaptionStyle } from "./style.js";
+
+function sameType(left: SurfaceResolvedReference["type"], right: SurfaceResolvedReference["type"]): boolean {
+ return left.module.name === right.module.name && left.module.version === right.module.version && left.name === right.name;
+}
+
+function attributes(element: StructuredElement, required: readonly string[]): void {
+ const actual = Object.keys(element.attributes);
+ if (required.some((name) => element.attributes[name] === undefined) || actual.some((name) => !required.includes(name))) {
+ throw new Error(`${element.name} requires exactly ${required.join(", ")}`);
+ }
+}
+
+function stringAttribute(element: StructuredElement, name: string): string {
+ const value = element.attributes[name];
+ if (typeof value !== "string" || !value.trim()) throw new Error(`${element.name}.${name} must be a non-empty string`);
+ return value.trim();
+}
+
+function reference(
+ element: StructuredElement,
+ name: string,
+ expected: SurfaceResolvedReference["type"],
+ resolveReference: (path: string) => SurfaceResolvedReference | undefined,
+): SurfaceResolvedReference {
+ const raw: TextAttributeValue | undefined = element.attributes[name];
+ if (typeof raw !== "object" || raw.kind !== "reference") {
+ throw new Error(`${element.name}.${name} must be a whole-value reference`);
+ }
+ const value = resolveReference(raw.path);
+ if (value === undefined) throw new Error(`${element.name}.${name} cannot resolve ${raw.path}`);
+ if (!sameType(value.type, expected)) throw new Error(`${element.name}.${name} has the wrong type`);
+ return value;
+}
+
+function inline<T>(referenceValue: SurfaceResolvedReference, subject: string): T {
+ if (referenceValue.record?.value.kind !== "inline") throw new Error(`${subject} must reference an authored inline Record`);
+ return referenceValue.record.value.value as unknown as T;
+}
+
+export const decodeFineCaptionStyleSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
+ attributes(element, ["id", "recipe"]);
+ if (element.children.some((child) => child.kind === "element" || child.value.trim())) {
+ throw new Error(`${element.name} does not accept children`);
+ }
+ const id = stringAttribute(element, "id");
+ const recipe = inline<SvsRecipe>(reference(element, "recipe", svsRecipeType, resolveReference), `${element.name}.recipe`);
+ const style = fineCaptionStyle(id, recipe);
+ return {
+ records: [{ id, type: captionTypes.style, value: { kind: "inline", value: style }, range: element.range }],
+ components: [],
+ fragments: [],
+ };
+};
+
+export const decodeFineCaptionTrackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
+ attributes(element, ["id", "narrative", "map", "words", "program", "plan", "space"]);
+ if (element.children.some((child) => child.kind === "element" || child.value.trim())) {
+ throw new Error(`${element.name} does not accept children`);
+ }
+ const id = stringAttribute(element, "id");
+ const narrative = reference(element, "narrative", narrativeTypes.narrative, resolveReference);
+ const map = reference(element, "map", semanticMapTypes.complete, resolveReference);
+ const words = reference(element, "words", narrativeTypes.captionWordSequence, resolveReference);
+ const program = reference(element, "program", captionTypes.program, resolveReference);
+ const plan = reference(element, "plan", captionTypes.plan, resolveReference);
+ const space = reference(element, "space", programSpaceTypes.programSpace, resolveReference);
+ return {
+ records: [],
+ components: [{
+ id,
+ fragment: fineCaptionTrackFragment.id,
+ inputs: {
+ narrative: narrative.ref, map: map.ref, words: words.ref,
+ program: program.ref, plan: plan.ref, space: space.ref,
+ },
+ outputs: { track: `${id}.track` },
+ range: element.range,
+ }],
+ fragments: [fineCaptionTrackFragment],
+ };
+};
diff --git a/packages/caption-fine/src/types.ts b/packages/caption-fine/src/types.ts
new file mode 100644
index 00000000..e52464d0
--- /dev/null
+++ b/packages/caption-fine/src/types.ts
@@ -0,0 +1,27 @@
+export type FineCaptionParameters = {
+ readonly contract: "svml.caption-fine-parameters@1";
+ readonly stackingOrder: number;
+ readonly placement: {
+ readonly x: number;
+ readonly y: number;
+ readonly width: number;
+ };
+ readonly typography: {
+ readonly fontFamily: string;
+ readonly fontSizePx: number;
+ readonly fontWeight: number;
+ readonly lineHeight: number;
+ readonly textAlign: "left" | "center" | "right";
+ readonly fill: string;
+ };
+ readonly box: {
+ readonly background: string;
+ readonly paddingXPx: number;
+ readonly paddingYPx: number;
+ readonly radiusPx: number;
+ };
+ readonly important: {
+ readonly fill: string;
+ readonly scale: number;
+ };
+};
diff --git a/packages/caption-fine/test/caption-fine.test.ts b/packages/caption-fine/test/caption-fine.test.ts
new file mode 100644
index 00000000..c0376ff1
--- /dev/null
+++ b/packages/caption-fine/test/caption-fine.test.ts
@@ -0,0 +1,76 @@
+import assert from "node:assert/strict";
+import test from "node:test";
+
+import { resolveCaptionProgram } from "@narratage/caption";
+import type { TimedCaptionProjection } from "@narratage/caption";
+import { fineCaptionStyle, renderFineCaption } from "@narratage/caption-fine";
+import { sealProgramSpace } from "@narratage/program-space";
+import { captionWordSequence, parseScript } from "@narratage/script";
+import type { SvsRecipe } from "@narratage/svs";
+
+const recipe: SvsRecipe = {
+ contract: "svml.svs-recipe@1",
+ path: "caption.primary",
+ properties: {
+ "cue-min-words": 1,
+ "cue-max-words": 5,
+ "important-min-per-cue": 0,
+ "important-max-per-cue": 2,
+ "important-fill": "#FFE044",
+ "important-scale": 1.15,
+ "stack-order": 70,
+ x: 0.08,
+ y: 0.76,
+ width: 0.84,
+ font: "Inter",
+ weight: 800,
+ size: 58,
+ "line-height": 1,
+ align: "center",
+ fill: "#FFFFFF",
+ background: "#09090BCC",
+ padding: "16 24",
+ radius: 18,
+ },
+};
+
+test("one Fine renderer handles the default Style, per-word fields and one peer VisualTrack", () => {
+ const narrative = parseScript("fine.svml", "<line>Meaning becomes visible.</line>");
+ const words = captionWordSequence(narrative, "story.caption.words");
+ const style = fineCaptionStyle("primary", recipe);
+ const program = resolveCaptionProgram(words, "captions", style, []);
+ const wordIds = words.words.map((word) => word.id);
+ const projection: TimedCaptionProjection = {
+ contract: "svml.timed-caption-projection@1",
+ text: "Meaning becomes visible.",
+ regions: [{
+ id: "cue:1",
+ runId: program.runs[0]!.id,
+ styleId: style.id,
+ display: "Meaning becomes visible.",
+ segmentId: "line",
+ kind: "identity",
+ sourceTokenIds: narrative.tokens.map((token) => token.id),
+ startSec: 0,
+ endSec: 2,
+ refinements: [],
+ wordIds,
+ fields: [{ declarationId: "important", wordId: wordIds[1]!, value: "true" }],
+ }],
+ };
+ const space = sealProgramSpace({
+ contract: "svml.program-space@1",
+ durationSec: 2,
+ frameRate: { numerator: 30, denominator: 1 },
+ });
+ const track = renderFineCaption(projection, program, words, space);
+ assert.equal(track.contract, "svml.visual-track@1");
+ assert.equal(track.presents.length, 1);
+ const wordElements = track.presents[0]!.elements.filter((element) => element.kind === "text");
+ assert.equal(wordElements.length, 3);
+ assert.deepEqual(wordElements.map((element) => element.kind === "text" ? element.text : undefined),
+ ["Meaning ", "becomes ", "visible."]);
+ assert.equal(wordElements[1]!.style.some((declaration) =>
+ declaration.name === "color" && declaration.value === "#FFE044"), true);
+ assert.equal(wordElements[0]!.style.some((declaration) => declaration.name === "transform"), false);
+});
diff --git a/packages/caption-gemini/README.md b/packages/caption-gemini/README.md
index 74c7cbee..33cfc01c 100644
--- a/packages/caption-gemini/README.md
+++ b/packages/caption-gemini/README.md
@@ -2,31 +2,29 @@
Exact Gemini Caption planning, without transcript correction, Style selection or timing authority.
-The package consumes an already-resolved `CaptionProgram`. Gemini sees only its immutable left-side
-display atoms and the planning requirements for each resolved run. It never receives WhisperX
-text, timestamps or the right side of Dual Text, and it cannot return replacement wording.
-
```xml
<caption-ai:Planner
id="caption-plan"
- narrative={story}
+ words={story.caption.words}
program={caption-program}
model="gemini-2.5-flash"
/>
```
-The output is `caption-plan.plan`. Its response has one entry per Program run. Within each run the
-model may only:
+Gemini receives the immutable ordered display words and the planning requirements of each resolved
+Program run. It never receives WhisperX text, timestamps or the pronunciation side of Dual Text,
+and it cannot return replacement wording.
+
+Within each run the response may only:
-- choose `after_atom_id` Cue boundaries that partition the run exactly once and in order;
-- attach declared field values to atoms inside that Cue.
+- choose `after_word_id` boundaries that partition the run exactly once and in order;
+- attach declared field values to `word_id` values inside that Cue.
-Fields are generic per-atom attributes, not hard-coded emphasis spans. A Style may declare no
-fields, one `important` flag, independent `best` and `medium` flags, enums, bounded numbers, or any
-other combination expressible by its declarations. Each atom may receive zero, one or multiple
-attributes. The validator enforces ids, value types and per-Cue cardinality before a `CaptionPlan`
-enters the graph.
+Fields are generic per-word attributes. A Style family may declare no fields, one `important`
+boolean, several independent booleans, enums or bounded numbers. A word may receive zero, one or
+multiple fields. The model package validates word identities, Cue coverage, Cue word bounds, field
+types and per-Cue cardinality before a `CaptionPlan` enters the graph.
-Model choice is author-visible. Network execution is separate: a Runtime explicitly binds the
-exact Gemini capability to `@narratage/provider-google-vertex` or another compatible endpoint. The local
-Scheduler owns concurrency; credentials and transport never enter `.svml` or BuildState.
+Model choice remains author-visible. Network execution is separate: a Runtime explicitly binds the
+exact Gemini capability to `@narratage/provider-google-vertex` or another compatible Endpoint.
+Credentials, queues and transport never enter `.svml` or BuildState.
diff --git a/packages/caption-gemini/src/component.ts b/packages/caption-gemini/src/component.ts
index b67e0399..294dd7d1 100644
--- a/packages/caption-gemini/src/component.ts
+++ b/packages/caption-gemini/src/component.ts
@@ -1,6 +1,6 @@
import type { ComponentPackage } from "@narratage/component-kit";
import type { CaptionProgram } from "@narratage/caption";
-import type { Narrative } from "@narratage/narrative";
+import type { CaptionWordSequence } from "@narratage/narrative";
import type { StoredValue } from "@narratage/protocol";
import { canonicalize } from "@narratage/protocol";
@@ -25,7 +25,7 @@ export const captionGeminiComponent = {
request: {
kind: "inline",
value: canonicalize(compileCaptionGeminiRequest(
- inline<Narrative>(inputs.narrative?.value, "Narrative"),
+ inline<CaptionWordSequence>(inputs.words?.value, "CaptionWordSequence"),
inline<CaptionProgram>(inputs.captionProgram?.value, "CaptionProgram"),
inline<CaptionGeminiProgram>(inputs.program?.value, "CaptionGeminiProgram"),
)),
diff --git a/packages/caption-gemini/src/fragment.ts b/packages/caption-gemini/src/fragment.ts
index 3f3348c1..9a713350 100644
--- a/packages/caption-gemini/src/fragment.ts
+++ b/packages/caption-gemini/src/fragment.ts
@@ -10,7 +10,7 @@ const operation = (id: string) => ({ kind: "fragment-operation" as const, operat
export const captionGeminiPlanningFragment = sealGraphFragment({
name: "@narratage/caption-gemini/planning@1",
inputs: [
- { name: "narrative", type: narrativeTypes.narrative },
+ { name: "words", type: narrativeTypes.captionWordSequence },
{ name: "captionProgram", type: captionTypes.program },
{ name: "program", type: captionGeminiTypes.program },
],
@@ -18,7 +18,7 @@ export const captionGeminiPlanningFragment = sealGraphFragment({
{
id: "caption-gemini:compile",
producer: captionGeminiProducers.compile,
- inputs: { narrative: input("narrative"), captionProgram: input("captionProgram"), program: input("program") },
+ inputs: { words: input("words"), captionProgram: input("captionProgram"), program: input("program") },
result: { kind: "output", name: "request" },
},
{
@@ -32,7 +32,7 @@ export const captionGeminiPlanningFragment = sealGraphFragment({
name: "plan",
type: captionTypes.plan,
root: operation("caption-gemini:plan"),
- semanticInputs: ["narrative", "captionProgram", "program"],
+ semanticInputs: ["words", "captionProgram", "program"],
fidelity: "exact",
}],
});
diff --git a/packages/caption-gemini/src/manifest.ts b/packages/caption-gemini/src/manifest.ts
index 6a75fd8f..500a6a02 100644
--- a/packages/caption-gemini/src/manifest.ts
+++ b/packages/caption-gemini/src/manifest.ts
@@ -50,7 +50,8 @@ const fieldDeclaration = object({
});
const planningRun = object({
id: { schema: string }, styleId: { schema: string },
- atomIds: { schema: { kind: "array", minItems: 1, items: string } },
+ wordIds: { schema: { kind: "array", minItems: 1, items: string } },
+ cueMinimumWords: { schema: nonNegativeInteger }, cueMaximumWords: { schema: nonNegativeInteger },
cueInstruction: { schema: string }, fields: { schema: { kind: "array", items: fieldDeclaration } },
});
export const captionGeminiProgramSchema: ValueSchema = object({
@@ -60,7 +61,7 @@ export const captionGeminiProgramSchema: ValueSchema = object({
export const captionGeminiRequestSchema: ValueSchema = object({
contract: { schema: { kind: "literal", value: "svml.caption-gemini-request@1" } },
model: { schema: model },
- atoms: { schema: { kind: "array", minItems: 1, items: object({ id: { schema: string }, text: { schema: string } }) } },
+ words: { schema: { kind: "array", minItems: 1, items: object({ id: { schema: string }, text: { schema: string } }) } },
runs: { schema: { kind: "array", minItems: 1, items: planningRun } },
systemInstruction: { schema: string }, prompt: { schema: string },
temperature: { schema: { kind: "literal", value: 0.2 } },
@@ -105,7 +106,7 @@ export const captionGeminiManifest: ModuleManifest = {
{
name: captionGeminiProducers.compile.name,
inputs: [
- { name: "narrative", type: narrativeTypes.narrative },
+ { name: "words", type: narrativeTypes.captionWordSequence },
{ name: "captionProgram", type: captionTypes.program },
{ name: "program", type: captionGeminiTypes.program },
],
diff --git a/packages/caption-gemini/src/plan.ts b/packages/caption-gemini/src/plan.ts
index a47c1163..a0c46749 100644
--- a/packages/caption-gemini/src/plan.ts
+++ b/packages/caption-gemini/src/plan.ts
@@ -42,32 +42,32 @@ function cueFromRaw(
raw: unknown,
runId: string,
cueIndex: number,
- runAtomIds: readonly string[],
+ runWordIds: readonly string[],
cursor: number,
declarations: ReadonlyMap<string, CaptionFieldDeclaration>,
): { readonly cue: CaptionPlannedCue; readonly nextCursor: number } {
const value = object(raw, `Caption cue ${runId}/${cueIndex + 1}`);
- exactKeys(value, ["after_atom_id", "fields"], `Caption cue ${runId}/${cueIndex + 1}`);
- assert(typeof value.after_atom_id === "string", `Caption cue ${runId}/${cueIndex + 1} endpoint is invalid`);
- const endpoint = runAtomIds.indexOf(value.after_atom_id, cursor);
+ exactKeys(value, ["after_word_id", "fields"], `Caption cue ${runId}/${cueIndex + 1}`);
+ assert(typeof value.after_word_id === "string", `Caption cue ${runId}/${cueIndex + 1} endpoint is invalid`);
+ const endpoint = runWordIds.indexOf(value.after_word_id, cursor);
assert(endpoint >= cursor, `Caption cue ${runId}/${cueIndex + 1} endpoint is outside its remaining run`);
- const atomIds = runAtomIds.slice(cursor, endpoint + 1);
+ const wordIds = runWordIds.slice(cursor, endpoint + 1);
assert(Array.isArray(value.fields), `Caption cue ${runId}/${cueIndex + 1} fields are invalid`);
const assignments = value.fields.map((rawField, fieldIndex) => {
const field = object(rawField, `Caption field ${runId}/${cueIndex + 1}/${fieldIndex + 1}`);
- exactKeys(field, ["declaration_id", "atom_id", "value"], "Caption field assignment");
+ exactKeys(field, ["declaration_id", "word_id", "value"], "Caption field assignment");
assert(typeof field.declaration_id === "string", "Caption field declaration id is invalid");
const declaration = declarations.get(field.declaration_id);
assert(declaration !== undefined, `Caption field ${field.declaration_id} was not declared for run ${runId}`);
- assert(typeof field.atom_id === "string" && atomIds.includes(field.atom_id),
- `Caption field ${declaration.id} atom lies outside its cue`);
+ assert(typeof field.word_id === "string" && wordIds.includes(field.word_id),
+ `Caption field ${declaration.id} word lies outside its cue`);
return {
declarationId: declaration.id,
- atomId: field.atom_id,
+ wordId: field.word_id,
value: fieldValue(declaration, field.value),
};
});
- const unique = new Set(assignments.map((field) => `${field.declarationId}\0${field.atomId}`));
+ const unique = new Set(assignments.map((field) => `${field.declarationId}\0${field.wordId}`));
assert(unique.size === assignments.length, `Caption cue ${runId}/${cueIndex + 1} repeats a field assignment`);
for (const declaration of declarations.values()) {
const count = assignments.filter((field) => field.declarationId === declaration.id).length;
@@ -75,7 +75,7 @@ function cueFromRaw(
`Caption field ${declaration.id} count ${count} violates ${declaration.minimumPerCue}..${declaration.maximumPerCue} per cue`);
}
return {
- cue: { id: `${runId}:cue:${cueIndex + 1}`, atomIds, fields: assignments },
+ cue: { id: `${runId}:cue:${cueIndex + 1}`, wordIds, fields: assignments },
nextCursor: endpoint + 1,
};
}
@@ -103,11 +103,14 @@ export function sealCaptionGeminiPlan(
const declarations = new Map(expectedRun.fields.map((field) => [field.id, field]));
let cursor = 0;
const cues = rawRun.cues.map((rawCue, cueIndex) => {
- const planned = cueFromRaw(rawCue, expectedRun.id, cueIndex, expectedRun.atomIds, cursor, declarations);
+ const planned = cueFromRaw(rawCue, expectedRun.id, cueIndex, expectedRun.wordIds, cursor, declarations);
cursor = planned.nextCursor;
+ assert(planned.cue.wordIds.length >= expectedRun.cueMinimumWords
+ && planned.cue.wordIds.length <= expectedRun.cueMaximumWords,
+ `Caption Cue ${planned.cue.id} violates ${expectedRun.cueMinimumWords}..${expectedRun.cueMaximumWords} words`);
return planned.cue;
});
- assert(cursor === expectedRun.atomIds.length, `Caption run ${expectedRun.id} Cues do not cover its final atom`);
+ assert(cursor === expectedRun.wordIds.length, `Caption run ${expectedRun.id} Cues do not cover its final word`);
return { id: expectedRun.id, styleId: expectedRun.styleId, cues };
});
return sealCaptionPlan({
diff --git a/packages/caption-gemini/src/request.ts b/packages/caption-gemini/src/request.ts
index 3fa10116..81ba6a07 100644
--- a/packages/caption-gemini/src/request.ts
+++ b/packages/caption-gemini/src/request.ts
@@ -1,6 +1,6 @@
-import { assertCaptionProgramForNarrative } from "@narratage/caption";
+import { assertCaptionProgramForWords } from "@narratage/caption";
import type { CaptionFieldDeclaration, CaptionProgram } from "@narratage/caption";
-import type { Narrative } from "@narratage/narrative";
+import type { CaptionWordSequence } from "@narratage/narrative";
import { verifyCaptionGeminiProgram } from "./program.js";
import type {
@@ -19,33 +19,35 @@ function fieldInstruction(field: CaptionFieldDeclaration): string {
: field.value.kind === "enum"
? `one of ${field.value.values.map((item) => JSON.stringify(item)).join(", ")}`
: "a finite base-10 number encoded as a string";
- return `${field.id}: ${field.minimumPerCue}..${field.maximumPerCue} atom assignments per cue; value=${value}. ${field.instruction}`;
+ return `${field.id}: ${field.minimumPerCue}..${field.maximumPerCue} word assignments per cue; value=${value}. ${field.instruction}`;
}
export function captionGeminiSystemInstruction(runs: readonly CaptionPlanningRun[]): string {
const styleRules = new Map<string, CaptionPlanningRun>();
runs.forEach((run) => styleRules.set(run.styleId, run));
return [
- "You are a caption cue and per-word attribute planner. Display atoms are immutable author truth.",
+ "You are a caption cue and per-word attribute planner. Display words are immutable author truth.",
"Return only response-schema JSON. Never return corrected text, replacement text, pronunciation text, timestamps, Markdown, or explanations.",
"Each run is a hard boundary. Plan every run independently; a cue may never cross a run boundary.",
- "Within each run, cues must preserve and exactly partition atom_ids. Return only each cue's final after_atom_id; the final cue must end at the run's final atom.",
- "Every field assignment targets exactly one atom inside its cue and uses a field declared by that run's style.",
+ "Within each run, cues must preserve and exactly partition word_ids. Return only each cue's final after_word_id; the final cue must end at the run's final word.",
+ "Every field assignment targets exactly one word inside its cue and uses a field declared by that run's style.",
...[...styleRules.values()].flatMap((run) => [
- `Style ${run.styleId} cue judgment: ${run.cueInstruction.trim()}`,
+ `Style ${run.styleId} cue judgment (${run.cueMinimumWords}..${run.cueMaximumWords} words): ${run.cueInstruction.trim()}`,
...run.fields.map((field) => `Style ${run.styleId} field ${fieldInstruction(field)}`),
]),
"Echo every run exactly once. Use an empty fields array when no declaration applies. Never invent ids.",
].join("\n");
}
-function promptContent(request: Pick<CaptionGeminiRequest, "atoms" | "runs">): string {
+function promptContent(request: Pick<CaptionGeminiRequest, "words" | "runs">): string {
return JSON.stringify({
- display_atoms: request.atoms,
+ display_words: request.words,
runs: request.runs.map((run) => ({
run_id: run.id,
style_id: run.styleId,
- atom_ids: run.atomIds,
+ word_ids: run.wordIds,
+ cue_minimum_words: run.cueMinimumWords,
+ cue_maximum_words: run.cueMaximumWords,
cue_instruction: run.cueInstruction,
field_declarations: run.fields,
})),
@@ -56,11 +58,13 @@ function requestContent(value: CaptionGeminiRequest) {
return {
contract: "svml.caption-gemini-request@1" as const,
model: value.model,
- atoms: value.atoms.map((atom) => ({ ...atom })),
+ words: value.words.map((word) => ({ ...word })),
runs: value.runs.map((run) => ({
id: run.id,
styleId: run.styleId,
- atomIds: [...run.atomIds],
+ wordIds: [...run.wordIds],
+ cueMinimumWords: run.cueMinimumWords,
+ cueMaximumWords: run.cueMaximumWords,
cueInstruction: run.cueInstruction.trim(),
fields: run.fields.map((field) => structuredClone(field)),
})),
@@ -71,11 +75,11 @@ function requestContent(value: CaptionGeminiRequest) {
}
export function compileCaptionGeminiRequest(
- narrative: Narrative,
+ words: CaptionWordSequence,
captionProgram: CaptionProgram,
program: CaptionGeminiProgram,
): CaptionGeminiRequest {
- assertCaptionProgramForNarrative(captionProgram, narrative);
+ assertCaptionProgramForWords(captionProgram, words);
verifyCaptionGeminiProgram(program);
const styles = new Map(captionProgram.styles.map((style) => [style.id, style]));
const runs = captionProgram.runs.map((run) => {
@@ -84,15 +88,17 @@ export function compileCaptionGeminiRequest(
return {
id: run.id,
styleId: run.styleId,
- atomIds: [...run.atomIds],
- cueInstruction: style.planning.cueInstruction,
+ wordIds: [...run.wordIds],
+ cueMinimumWords: style.planning.cue.minimumWords,
+ cueMaximumWords: style.planning.cue.maximumWords,
+ cueInstruction: style.planning.cue.instruction,
fields: style.planning.fields.map((field) => structuredClone(field)),
};
});
const base = {
contract: "svml.caption-gemini-request@1" as const,
model: program.model,
- atoms: captionProgram.atoms.map((atom) => ({ id: atom.id, text: atom.text })),
+ words: words.words.map((word) => ({ id: word.id, text: word.text })),
runs,
};
const systemInstruction = captionGeminiSystemInstruction(runs);
@@ -108,17 +114,20 @@ export function verifyCaptionGeminiRequest(value: unknown): asserts value is Cap
assert(request.contract === "svml.caption-gemini-request@1", "Caption Gemini request contract is invalid");
assert(request.model === "gemini-2.5-flash" || request.model === "gemini-3.1-pro-preview", "Caption Gemini request model is unsupported");
assert(request.temperature === 0.2, "Caption Gemini temperature is not the package-owned value");
- assert(Array.isArray(request.atoms) && request.atoms.length > 0, "Caption Gemini request atoms are empty");
- const atomIds = request.atoms.map((atom) => atom.id);
- assert(atomIds.every((id) => typeof id === "string" && id.length > 0) && new Set(atomIds).size === atomIds.length,
- "Caption Gemini atom ids are invalid or repeated");
- assert(request.atoms.every((atom) => typeof atom.text === "string" && atom.text.length > 0),
- "Caption Gemini display atom text is invalid");
- const atomSet = new Set(atomIds);
- const planned = request.runs.flatMap((run) => run.atomIds);
- assert(request.runs.length > 0 && planned.length === atomIds.length && new Set(planned).size === atomIds.length
- && planned.every((id) => atomSet.has(id)), "Caption Gemini runs are not an exact display-atom partition");
- assert(request.runs.every((run) => run.atomIds.length > 0 && run.cueInstruction.trim().length > 0),
+ assert(Array.isArray(request.words) && request.words.length > 0, "Caption Gemini request words are empty");
+ const wordIds = request.words.map((word) => word.id);
+ assert(wordIds.every((id) => typeof id === "string" && id.length > 0) && new Set(wordIds).size === wordIds.length,
+ "Caption Gemini word ids are invalid or repeated");
+ assert(request.words.every((word) => typeof word.text === "string" && word.text.length > 0),
+ "Caption Gemini display word text is invalid");
+ const wordSet = new Set(wordIds);
+ const planned = request.runs.flatMap((run) => run.wordIds);
+ assert(request.runs.length > 0 && planned.length === wordIds.length && new Set(planned).size === wordIds.length
+ && planned.every((id) => wordSet.has(id)), "Caption Gemini runs are not an exact display-word partition");
+ assert(request.runs.every((run) => run.wordIds.length > 0
+ && Number.isSafeInteger(run.cueMinimumWords) && run.cueMinimumWords > 0
+ && Number.isSafeInteger(run.cueMaximumWords) && run.cueMaximumWords >= run.cueMinimumWords
+ && run.cueInstruction.trim().length > 0),
"Caption Gemini request contains an empty run");
assert(request.systemInstruction === captionGeminiSystemInstruction(request.runs),
"Caption Gemini system instruction differs from the model package");
diff --git a/packages/caption-gemini/src/surface.ts b/packages/caption-gemini/src/surface.ts
index cf36e5a9..84c2691a 100644
--- a/packages/caption-gemini/src/surface.ts
+++ b/packages/caption-gemini/src/surface.ts
@@ -29,7 +29,7 @@ function reference(
}
export const decodeCaptionGeminiPlannerSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
- const required = ["id", "narrative", "program", "model"];
+ const required = ["id", "words", "program", "model"];
if (required.some((name) => element.attributes[name] === undefined)
|| Object.keys(element.attributes).some((name) => !required.includes(name))) {
throw new Error(`${element.name} requires ${required.join(", ")}`);
@@ -43,7 +43,7 @@ export const decodeCaptionGeminiPlannerSurface: StructuredSurfaceHandler = ({ el
if (model !== "gemini-2.5-flash" && model !== "gemini-3.1-pro-preview") {
throw new Error(`${element.name}.model is unsupported`);
}
- const narrative = reference(element, "narrative", narrativeTypes.narrative, resolveReference);
+ const words = reference(element, "words", narrativeTypes.captionWordSequence, resolveReference);
const captionProgram = reference(element, "program", captionTypes.program, resolveReference);
const optionsId = `${id}.gemini`;
const options = sealCaptionGeminiProgram({
@@ -56,7 +56,7 @@ export const decodeCaptionGeminiPlannerSurface: StructuredSurfaceHandler = ({ el
id,
fragment: captionGeminiPlanningFragment.id,
inputs: {
- narrative: narrative.ref,
+ words: words.ref,
captionProgram: captionProgram.ref,
program: { kind: "record", id: optionsId },
},
diff --git a/packages/caption-gemini/src/types.ts b/packages/caption-gemini/src/types.ts
index ff2e41b8..b6c8231d 100644
--- a/packages/caption-gemini/src/types.ts
+++ b/packages/caption-gemini/src/types.ts
@@ -26,7 +26,7 @@ export type CaptionGeminiProgram = {
readonly model: CaptionGeminiModel;
};
-export type CaptionPlanningAtom = {
+export type CaptionPlanningWord = {
readonly id: string;
readonly text: string;
};
@@ -34,7 +34,9 @@ export type CaptionPlanningAtom = {
export type CaptionPlanningRun = {
readonly id: string;
readonly styleId: string;
- readonly atomIds: readonly string[];
+ readonly wordIds: readonly string[];
+ readonly cueMinimumWords: number;
+ readonly cueMaximumWords: number;
readonly cueInstruction: string;
readonly fields: readonly CaptionFieldDeclaration[];
};
@@ -42,7 +44,7 @@ export type CaptionPlanningRun = {
export type CaptionGeminiRequest = {
readonly contract: "svml.caption-gemini-request@1";
readonly model: CaptionGeminiModel;
- readonly atoms: readonly CaptionPlanningAtom[];
+ readonly words: readonly CaptionPlanningWord[];
readonly runs: readonly CaptionPlanningRun[];
readonly systemInstruction: string;
readonly prompt: string;
@@ -53,10 +55,10 @@ export type RawCaptionGeminiResponse = {
readonly runs: readonly {
readonly run_id: string;
readonly cues: readonly {
- readonly after_atom_id: string;
+ readonly after_word_id: string;
readonly fields: readonly {
readonly declaration_id: string;
- readonly atom_id: string;
+ readonly word_id: string;
readonly value: string;
}[];
}[];
diff --git a/packages/caption-gemini/test/planning.test.ts b/packages/caption-gemini/test/planning.test.ts
index 8895ff9f..a26f664f 100644
--- a/packages/caption-gemini/test/planning.test.ts
+++ b/packages/caption-gemini/test/planning.test.ts
@@ -1,39 +1,30 @@
import assert from "node:assert/strict";
import test from "node:test";
-import {
- defaultCaptionTrackProgram,
- resolveCaptionProgram,
- sealCaptionProgram,
- sealCaptionStyle,
-} from "@narratage/caption";
-import type { CaptionStyleIntent } from "@narratage/caption";
+import { resolveCaptionProgram, sealCaptionStyle } from "@narratage/caption";
+import type { CaptionFieldDeclaration, CaptionStyleIntent } from "@narratage/caption";
import {
compileCaptionGeminiRequest,
sealCaptionGeminiPlan,
sealCaptionGeminiProgram,
} from "@narratage/caption-gemini";
import type { CaptionGeminiRequest, RawCaptionGeminiResponse } from "@narratage/caption-gemini";
-import { parseScript } from "@narratage/script";
+import { captionWordSequence, parseScript } from "@narratage/script";
-function style(id: string, fields: CaptionStyleIntent["planning"]["fields"] = []) {
- const base = defaultCaptionTrackProgram(id);
+function style(id: string, fields: readonly CaptionFieldDeclaration[] = []): CaptionStyleIntent {
return sealCaptionStyle({
contract: "svml.caption-style@1",
id,
planning: {
- cueInstruction: "Prefer short complete semantic phrases and never cross clause punctuation.",
+ cue: { minimumWords: 1, maximumWords: 7, instruction: "Prefer short complete semantic phrases." },
fields,
},
- presentation: { mode: "whole", stackingOrder: base.stacking.order, style: base.style },
+ rendering: { family: "test-caption@1", parameters: {} },
});
}
function options() {
- return sealCaptionGeminiProgram({
- contract: "svml.caption-gemini-program@1",
- model: "gemini-2.5-flash",
- });
+ return sealCaptionGeminiProgram({ contract: "svml.caption-gemini-program@1", model: "gemini-2.5-flash" });
}
function validResponse(request: CaptionGeminiRequest): RawCaptionGeminiResponse {
@@ -41,10 +32,10 @@ function validResponse(request: CaptionGeminiRequest): RawCaptionGeminiResponse
runs: request.runs.map((run) => ({
run_id: run.id,
cues: [{
- after_atom_id: run.atomIds.at(-1)!,
+ after_word_id: run.wordIds.at(-1)!,
fields: run.fields.length === 0 ? [] : [{
declaration_id: run.fields[0]!.id,
- atom_id: run.atomIds[0]!,
+ word_id: run.wordIds[0]!,
value: "true",
}],
}],
@@ -52,107 +43,74 @@ function validResponse(request: CaptionGeminiRequest): RawCaptionGeminiResponse
};
}
-test("Gemini sees only immutable display atoms while Role and roleless runs come from Caption Program", () => {
+test("Gemini sees only immutable left-side display words and resolved Style runs", () => {
const narrative = parseScript("roles.svml", `
<opening>Meet <SVML | semantic video markup language>.</opening>
<answer><ALICE>Meaning becomes the source.</answer>
`);
- const normal = style("normal");
+ const words = captionWordSequence(narrative, "story.caption.words");
const alice = style("alice", [{
- id: "important",
- value: { kind: "boolean" },
- instruction: "Select zero or one word whose emphasis best communicates this Cue.",
- minimumPerCue: 0,
- maximumPerCue: 1,
+ id: "important", value: { kind: "boolean" }, instruction: "Select up to one important word.",
+ minimumPerCue: 0, maximumPerCue: 1,
}]);
- const program = resolveCaptionProgram(narrative, "captions", normal, [{
+ const program = resolveCaptionProgram(words, "captions", style("normal"), [{
id: "alice-use",
- selector: { kind: "role", role: "ALICE" },
+ words: {
+ contract: "svml.caption-word-subset@1", id: "role:ALICE", sequenceId: words.id,
+ wordIds: words.words.filter((word) => word.role === "ALICE").map((word) => word.id),
+ },
style: alice,
}]);
- const request = compileCaptionGeminiRequest(narrative, program, options());
+ const request = compileCaptionGeminiRequest(words, program, options());
assert.deepEqual(request.runs.map((run) => run.styleId), ["normal", "alice"]);
- assert.equal(request.atoms.some((atom) => atom.text === "SVML"), true);
- assert.equal(request.atoms.some((atom) => atom.text === "semantic"), false);
+ assert.equal(request.words.some((word) => word.text === "SVML"), true);
+ assert.equal(request.words.some((word) => word.text === "semantic"), false);
assert.equal(request.prompt.includes("semantic video markup language"), false);
- assert.equal(request.prompt.includes("pronunciation"), false);
assert.equal("whisperx" in request, false);
- assert.equal("transcript" in request, false);
- const tampered = sealCaptionProgram({
- ...program,
- atoms: program.atoms.map((atom, index) => index === 0 ? { ...atom, text: "TAMPERED" } : atom),
- });
- assert.throws(() => compileCaptionGeminiRequest(narrative, tampered, options()),
- /display atoms differ from its Narrative/u);
+ const foreignWords = { ...words, id: "other.caption.words" };
+ assert.throws(() => compileCaptionGeminiRequest(foreignWords, program, options()), /does not partition/u);
});
-test("Gemini may only cut each resolved run and assign declared attributes to atoms inside its Cue", () => {
- const narrative = parseScript("plan.svml", "<line><ALICE>Meaning becomes the source.</line>");
- const emphasis = style("emphasis", [{
- id: "important",
- value: { kind: "boolean" },
- instruction: "Select exactly one important word.",
- minimumPerCue: 1,
- maximumPerCue: 1,
- }]);
- const program = resolveCaptionProgram(narrative, "captions", emphasis, []);
- const request = compileCaptionGeminiRequest(narrative, program, options());
+test("Gemini may only cut runs and assign declared fields to words inside each Cue", () => {
+ const narrative = parseScript("plan.svml", "<line>Meaning becomes the source.</line>");
+ const words = captionWordSequence(narrative, "story.caption.words");
+ const program = resolveCaptionProgram(words, "captions", style("emphasis", [{
+ id: "important", value: { kind: "boolean" }, instruction: "Select exactly one important word.",
+ minimumPerCue: 1, maximumPerCue: 1,
+ }]), []);
+ const request = compileCaptionGeminiRequest(words, program, options());
const plan = sealCaptionGeminiPlan(request, validResponse(request));
- assert.deepEqual(
- plan.runs.flatMap((run) => run.cues.flatMap((cue) => cue.atomIds)),
- request.runs.flatMap((run) => run.atomIds),
- );
- assert.equal(plan.runs[0]?.cues[0]?.fields[0]?.declarationId, "important");
+ assert.deepEqual(plan.runs.flatMap((run) => run.cues.flatMap((cue) => cue.wordIds)),
+ request.runs.flatMap((run) => run.wordIds));
- assert.throws(
- () => sealCaptionGeminiPlan(request, { ...validResponse(request), corrected_text: "tampered" }),
- /unknown or missing fields/u,
- );
const outside = structuredClone(validResponse(request));
- const field = outside.runs[0]!.cues[0]!.fields[0]! as { atom_id: string };
- field.atom_id = "foreign";
+ (outside.runs[0]!.cues[0]!.fields[0]! as { word_id: string }).word_id = "foreign";
assert.throws(() => sealCaptionGeminiPlan(request, outside), /outside its cue/u);
});
-test("one atom may carry multiple independent fields and fields need not form contiguous spans", () => {
- const narrative = parseScript("multi-field.svml", "<line>one two three four five.</line>");
- const multi = style("multi", [
- {
- id: "best",
- value: { kind: "boolean" },
- instruction: "Select up to two best words.",
- minimumPerCue: 0,
- maximumPerCue: 2,
- },
- {
- id: "medium",
- value: { kind: "boolean" },
- instruction: "Independently select up to two medium words.",
- minimumPerCue: 0,
- maximumPerCue: 2,
- },
- ]);
- const program = resolveCaptionProgram(narrative, "captions", multi, []);
- const request = compileCaptionGeminiRequest(narrative, program, options());
- const atoms = request.runs[0]!.atomIds;
- const plan = sealCaptionGeminiPlan(request, {
- runs: [{
- run_id: request.runs[0]!.id,
- cues: [{
- after_atom_id: atoms.at(-1)!,
- fields: [
- { declaration_id: "best", atom_id: atoms[0]!, value: "true" },
- { declaration_id: "best", atom_id: atoms[2]!, value: "true" },
- { declaration_id: "medium", atom_id: atoms[1]!, value: "true" },
- { declaration_id: "medium", atom_id: atoms[2]!, value: "true" },
- ],
- }],
+test("one word may carry multiple independent, non-contiguous field assignments", () => {
+ const narrative = parseScript("multi.svml", "<line>one two three four five.</line>");
+ const words = captionWordSequence(narrative, "story.caption.words");
+ const fields: CaptionFieldDeclaration[] = ["best", "medium"].map((id) => ({
+ id, value: { kind: "boolean" }, instruction: `Independently select ${id} words.`,
+ minimumPerCue: 0, maximumPerCue: 2,
+ }));
+ const program = resolveCaptionProgram(words, "captions", style("multi", fields), []);
+ const request = compileCaptionGeminiRequest(words, program, options());
+ const ids = request.runs[0]!.wordIds;
+ const plan = sealCaptionGeminiPlan(request, { runs: [{
+ run_id: request.runs[0]!.id,
+ cues: [{
+ after_word_id: ids.at(-1)!,
+ fields: [
+ { declaration_id: "best", word_id: ids[0]!, value: "true" },
+ { declaration_id: "best", word_id: ids[2]!, value: "true" },
+ { declaration_id: "medium", word_id: ids[1]!, value: "true" },
+ { declaration_id: "medium", word_id: ids[2]!, value: "true" },
+ ],
}],
- });
- const fields = plan.runs[0]!.cues[0]!.fields;
- assert.equal(fields.filter((field) => field.atomId === atoms[2]).length, 2);
- assert.deepEqual(fields.filter((field) => field.declarationId === "best").map((field) => field.atomId),
- [atoms[0], atoms[2]]);
+ }] });
+ assert.equal(plan.runs[0]!.cues[0]!.fields.filter((field) => field.wordId === ids[2]).length, 2);
});
diff --git a/packages/caption/README.md b/packages/caption/README.md
index 905306eb..69cec005 100644
--- a/packages/caption/README.md
+++ b/packages/caption/README.md
@@ -1,58 +1,44 @@
# `@narratage/caption`
-Provider-neutral Caption planning, timing projection and lowering to an ordinary peer
-`VisualTrack`.
+Style-family-neutral Caption contracts and timing.
-Caption has two independent inputs:
+The package owns four pieces of common language:
-- `Narrative.captionProjection` is the immutable display truth. Only the left side of Dual Text is
- visible here; `<SVML | semantic video markup language>` contributes the display atom `SVML`, not
- its pronunciation words.
-- `CaptionProgram` is the complete author intent for planning and presentation. It assigns exactly
- one full `CaptionStyle` to every visible display atom.
+- a complete ordered `CaptionWordSequence` supplied explicitly by the Script package;
+- one total `CaptionProgram`, with an explicit default Style and ordered whole-Style overrides;
+- a generic `CaptionPlan`, which partitions words into Cues and assigns declared fields;
+- the deterministic join from planned display words to an independent `CompleteSemanticMap`.
-A Style contains both its planning requirements and its complete visual appearance. The Program
-names one default Style and then applies ordered whole-Style replacements by Role or explicit
-Script Selection. Later matching applications win. Consequently roleless prose and every
-unmentioned word receive the default automatically, while changing one sentence needs only one
-Selection and one `Use`; the author never writes the complement.
+It does **not** own a concrete font, box model, field meaning, renderer, LLM or Provider. A Style
+family such as `@narratage/caption-fine` creates complete Styles and renders the resulting Program.
```xml
-<caption:Style id="plain" appearance={studio.caption.plain}>
- <caption:Cues>Use complete phrases of two to five words.</caption:Cues>
-</caption:Style>
-
-<caption:Style id="impact" appearance={studio.caption.impact}>
- <caption:Cues>Use complete phrases of two to seven words.</caption:Cues>
- <caption:Field id="important" type="boolean" min-per-cue="0" max-per-cue="2">
- Select at most two words whose emphasis best communicates this Cue.
- </caption:Field>
-</caption:Style>
-
-<caption:Program id="captions" narrative={story} default={plain}>
+<caption-fine:Style id="plain" recipe={studio.caption.plain}/>
+<caption-fine:Style id="impact" recipe={studio.caption.impact}/>
+
+<caption:Program id="captions" words={story.caption.words} default={plain}>
<caption:Use role="ALICE" style={impact}/>
- <caption:Use on={story.selection.special} style={plain}/>
+ <caption:Use words={story.caption.selection.special} style={plain}/>
</caption:Program>
```
-`Use` order is semantic. The second rule above replaces the entire Style for `special`; it does not
-merge arbitrary fields from two styles. A Role Cue is optional in Script, and Role state never
-crosses a Segment boundary.
+The default covers the whole word universe. Authors do not create a fake `@whole` Selection or its
+complement. Each `Use` replaces one complete Style on an explicit word subset; later matching rules
+win. `role=` is author-surface sugar for a subset of the same ordered word universe. It is not a
+second time-range language.
-The planner-neutral `CaptionPlan` has only two freedoms:
+The Program stores only sequence identity and word-id-to-Style runs. It does not copy the upstream
+word payload. Planner, timing and rendering packages receive `story.caption.words` through their own
+explicit graph inputs.
-1. partition each already-resolved Program run into ordered Cues;
-2. assign zero, one or more declared attributes to each display atom.
+One Style is an indivisible pair:
-It cannot rewrite text, change Style assignment or invent time. `temporalize-caption-plan` later
-joins those atom identities with the independent `CompleteSemanticMap`. Exact display/speech
-correspondence reuses measured timing. A display alias without word-level audio evidence receives
-an explicitly estimated local projection and never contaminates the global speech map.
+- common planning requirements: Cue word bounds and generic per-word field declarations;
+- package-owned rendering family and opaque resolved parameters.
-The final `render-caption-program` Producer lowers the timed result and each selected full Style to
-one self-contained `VisualTrack`. Caption is not a privileged Composition layer: Film consumes it
-exactly like Speech, B-roll or Text Track output.
+The planner has only two freedoms: partition each resolved run without reordering or rewriting its
+words, and assign zero or more declared fields to each word. It never sees audio or time. The common
+timing Producer later joins the validated Plan to Script's Caption projection and the measured
+semantic map.
-The package owns deterministic Types, validators and Producers only. It contains no LLM, Provider,
-credential, queue or Core authority. A planning package such as `@narratage/caption-gemini` may fulfill
-the narrow `CaptionPlan` contract; presentation and timing remain here.
+The package contains no LLM, Provider, credential, queue, visual renderer or Core authority.
diff --git a/packages/caption/package.json b/packages/caption/package.json
index fe938469..42026219 100644
--- a/packages/caption/package.json
+++ b/packages/caption/package.json
@@ -11,13 +11,10 @@
},
"dependencies": {
"@narratage/component-kit": "workspace:*",
- "@narratage/composition": "workspace:*",
"@narratage/elaborator": "workspace:*",
"@narratage/narrative": "workspace:*",
- "@narratage/program-space": "workspace:*",
"@narratage/protocol": "workspace:*",
"@narratage/semantic-map": "workspace:*",
- "@narratage/svs": "workspace:*",
"@narratage/text": "workspace:*"
},
"devDependencies": {
diff --git a/packages/caption/src/activation.ts b/packages/caption/src/activation.ts
index 0cc7ded3..c51ea4c4 100644
--- a/packages/caption/src/activation.ts
+++ b/packages/caption/src/activation.ts
@@ -1,9 +1,7 @@
import { createTextSurfaceHostFacet } from "@narratage/text";
import {
captionComponent, captionManifest, captionModuleRef,
- captionProgramSurfaceImplementationDigest, captionStyleSurfaceImplementationDigest,
- captionSurfaceImplementationDigest, decodeCaptionProgramSurface, decodeCaptionStyleSurface,
- decodeCaptionTrackSurface,
+ captionProgramSurfaceImplementationDigest, decodeCaptionProgramSurface,
} from "./index.js";
export const svmlPackage = {
@@ -12,9 +10,7 @@ export const svmlPackage = {
modules: [{ manifest: captionManifest, specifiers: ["@narratage/caption", "@narratage/caption@1"] }],
components: [captionComponent],
hostFacets: [
- createTextSurfaceHostFacet({ module: captionModuleRef, surface: "style", mode: "structured", implementationDigest: captionStyleSurfaceImplementationDigest, handler: decodeCaptionStyleSurface }),
createTextSurfaceHostFacet({ module: captionModuleRef, surface: "program", mode: "structured", implementationDigest: captionProgramSurfaceImplementationDigest, handler: decodeCaptionProgramSurface }),
- createTextSurfaceHostFacet({ module: captionModuleRef, surface: "track", mode: "structured", implementationDigest: captionSurfaceImplementationDigest, handler: decodeCaptionTrackSurface }),
],
};
export default svmlPackage;
diff --git a/packages/caption/src/component.ts b/packages/caption/src/component.ts
index 7789c04c..75a9bd06 100644
--- a/packages/caption/src/component.ts
+++ b/packages/caption/src/component.ts
@@ -1,8 +1,6 @@
import type { ComponentPackage } from "@narratage/component-kit";
-import type { Narrative } from "@narratage/narrative";
-import type { ProgramSpace } from "@narratage/program-space";
+import type { CaptionWordSequence, Narrative } from "@narratage/narrative";
import type { CompleteSemanticMap } from "@narratage/semantic-map";
-import type { VisualTrack } from "@narratage/composition";
import type { StoredValue } from "@narratage/protocol";
import { canonicalize } from "@narratage/protocol";
@@ -16,19 +14,11 @@ import {
import { assertCaptionPlan } from "./plan.js";
import { assertCaptionProgram, assertCaptionStyle } from "./style.js";
import { temporalizeCaption, temporalizeCaptionPlan } from "./temporalize.js";
-import {
- assertCaptionTrackProgram,
- assertTimedCaptionProjection,
- renderCaptionProgram,
- renderCaptionTrack,
- renderCaptionProgramImplementationDigest,
- renderCaptionTrackImplementationDigest,
-} from "./track.js";
+import { assertTimedCaptionProjection } from "./temporalize.js";
import type {
CaptionPlan,
CaptionProgram,
CaptionStyleIntent,
- CaptionTrackProgram,
TimedCaptionProjection,
} from "./types.js";
@@ -37,7 +27,7 @@ function inline<T>(value: StoredValue | undefined, subject: string): T {
return value.value as T;
}
-/** Deterministic timing and VisualTrack lowering only; cue generation and author styling stay outside. */
+/** Deterministic timing joins only; cue generation, Style meaning and visual lowering stay outside. */
export const captionComponent = {
name: "@narratage/caption",
producers: [
@@ -67,6 +57,7 @@ export const captionComponent = {
value: canonicalize(temporalizeCaptionPlan(
inline<Narrative>(inputs.narrative?.value, "Narrative"),
inline<CompleteSemanticMap>(inputs.map?.value, "CompleteSemanticMap"),
+ inline<CaptionWordSequence>(inputs.words?.value, "CaptionWordSequence"),
inline<CaptionProgram>(inputs.program?.value, "CaptionProgram"),
inline<CaptionPlan>(inputs.plan?.value, "CaptionPlan"),
)),
@@ -75,40 +66,6 @@ export const captionComponent = {
needs: {},
}),
},
- {
- producer: captionProducers.renderProgram,
- implementationDigest: renderCaptionProgramImplementationDigest,
- handler: ({ inputs }) => ({
- outputs: {
- track: {
- kind: "inline",
- value: canonicalize(renderCaptionProgram(
- inline<TimedCaptionProjection>(inputs.caption?.value, "TimedCaptionProjection"),
- inline<CaptionProgram>(inputs.program?.value, "CaptionProgram"),
- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),
- )),
- },
- },
- needs: {},
- }),
- },
- {
- producer: captionProducers.renderTrack,
- implementationDigest: renderCaptionTrackImplementationDigest,
- handler: ({ inputs }) => ({
- outputs: {
- track: {
- kind: "inline",
- value: canonicalize(renderCaptionTrack(
- inline<TimedCaptionProjection>(inputs.caption?.value, "TimedCaptionProjection"),
- inline<CaptionTrackProgram>(inputs.program?.value, "CaptionTrackProgram"),
- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),
- )),
- },
- },
- needs: {},
- }),
- },
],
validators: [
{
@@ -133,12 +90,5 @@ export const captionComponent = {
inline<TimedCaptionProjection>(value, "TimedCaptionProjection"),
),
},
- {
- type: captionTypes.trackProgram,
- implementationDigest: captionValidatorDigests.trackProgram,
- handler: ({ value }) => assertCaptionTrackProgram(
- inline<CaptionTrackProgram>(value, "CaptionTrackProgram"),
- ),
- },
],
} satisfies ComponentPackage;
diff --git a/packages/caption/src/display.ts b/packages/caption/src/display.ts
index eed0556f..871c0391 100644
--- a/packages/caption/src/display.ts
+++ b/packages/caption/src/display.ts
@@ -1,102 +1,89 @@
-import type { CaptionRegion, Narrative, NarrativeSelectionRef } from "@narratage/narrative";
+import type {
+ CaptionWord,
+ CaptionWordSequence,
+ CaptionWordSubset,
+ Narrative,
+} from "@narratage/narrative";
-import type { CaptionDisplayAtom } from "./types.js";
-
-const WORD =
- /[\p{Script=Han}\p{Script=Hiragana}\p{Script=Katakana}]\p{M}*|[\p{L}\p{M}\p{N}]+(?:['’.-][\p{L}\p{M}\p{N}]+)*/gu;
-
-function displayWords(value: string): Array<{ readonly text: string; readonly start: number; readonly end: number }> {
- const result: Array<{ text: string; start: number; end: number }> = [];
- WORD.lastIndex = 0;
- let match: RegExpExecArray | null;
- while ((match = WORD.exec(value))) {
- result.push({ text: match[0], start: match.index, end: match.index + match[0].length });
- }
- return result;
+function assert(condition: unknown, message: string): asserts condition {
+ if (!condition) throw new Error(message);
}
-function turnForRegion(narrative: Narrative, region: CaptionRegion): Narrative["turns"][number] {
- const turn = narrative.turns.find((candidate) =>
- candidate.segmentId === region.segmentId
- && candidate.tokenStart < region.endTokenExclusive
- && candidate.tokenEndExclusive > region.startToken);
- if (turn === undefined) throw new Error(`Caption region ${region.id} is not owned by a Narrative Turn`);
- return turn;
+export function assertCaptionWordSequence(value: CaptionWordSequence): void {
+ assert(value.contract === "svml.caption-word-sequence@1" && value.id.length > 0,
+ "CaptionWordSequence identity is invalid");
+ assert(value.words.length > 0, "CaptionWordSequence is empty");
+ const ids = new Set<string>();
+ value.words.forEach((word, index) => {
+ assert(word.id.length > 0 && !ids.has(word.id) && word.index === index,
+ "CaptionWord identity is invalid or repeated");
+ ids.add(word.id);
+ assert(word.regionId.length > 0 && word.segmentId.length > 0 && word.turnId.length > 0 && word.text.length > 0,
+ `CaptionWord ${word.id} context is invalid`);
+ assert(Number.isSafeInteger(word.displayStart) && Number.isSafeInteger(word.displayEnd)
+ && word.displayStart >= 0 && word.displayEnd > word.displayStart,
+ `CaptionWord ${word.id} display range is invalid`);
+ assert(Number.isSafeInteger(word.sourceTokenStart) && Number.isSafeInteger(word.sourceTokenEndExclusive)
+ && word.sourceTokenStart >= 0 && word.sourceTokenEndExclusive > word.sourceTokenStart,
+ `CaptionWord ${word.id} source range is invalid`);
+ assert(word.correspondence === "exact" || word.correspondence === "region-envelope",
+ `CaptionWord ${word.id} correspondence is invalid`);
+ });
}
-/** The only word universe visible to Caption planning. Speech-side alias text never enters it. */
-export function captionDisplayAtoms(narrative: Narrative): CaptionDisplayAtom[] {
- const atoms: CaptionDisplayAtom[] = [];
- for (const region of narrative.captionProjection.regions) {
- if (region.kind === "hidden") continue;
- const turn = turnForRegion(narrative, region);
- for (const word of displayWords(region.display)) {
- const exact = region.refinements.find((refinement) =>
- refinement.displayStart === word.start && refinement.displayEnd === word.end);
- atoms.push({
- id: `${region.id}:display:${atoms.length + 1}`,
- index: atoms.length,
- regionId: region.id,
- segmentId: region.segmentId,
- turnId: turn.id,
- ...(turn.role === undefined ? {} : { role: turn.role }),
- text: word.text,
- displayStart: word.start,
- displayEnd: word.end,
- sourceTokenStart: exact?.startToken ?? region.startToken,
- sourceTokenEndExclusive: exact?.endTokenExclusive ?? region.endTokenExclusive,
- correspondence: exact === undefined ? "region-envelope" : "exact",
- });
- }
+export function assertCaptionWordSubset(value: CaptionWordSubset, sequence: CaptionWordSequence): void {
+ assertCaptionWordSequence(sequence);
+ assert(value.contract === "svml.caption-word-subset@1" && value.id.length > 0,
+ "CaptionWordSubset identity is invalid");
+ assert(value.sequenceId === sequence.id, `CaptionWordSubset ${value.id} belongs to another word sequence`);
+ const positions = new Map(sequence.words.map((word) => [word.id, word.index]));
+ let previous = -1;
+ const seen = new Set<string>();
+ for (const id of value.wordIds) {
+ const position = positions.get(id);
+ assert(position !== undefined && position > previous && !seen.has(id),
+ `CaptionWordSubset ${value.id} is not an ordered subset of ${sequence.id}`);
+ previous = position;
+ seen.add(id);
}
- if (atoms.length === 0) throw new Error("Caption display projection contains no visible words");
- return atoms;
}
-function occurrenceRange(
- occurrence: NarrativeSelectionRef["occurrences"][number],
- tokenCount: number,
-): { readonly start: number; readonly endExclusive: number } {
- // Outward affinity moves the boundary in time, not in words: `~@x hello @/x~`
- // and `@x hello @/x` cover the same token, differing only in the surrounding
- // silence they absorb.
+/** Author-surface sugar for Role selection. The Program itself receives only the resolved subset. */
+export function captionWordsForRole(sequence: CaptionWordSequence, role: string): CaptionWordSubset {
+ assertCaptionWordSequence(sequence);
+ const normalized = role.trim();
+ assert(normalized.length > 0, "Caption Role is empty");
return {
- start: Math.max(0, occurrence.open.boundary.tokenIndex),
- endExclusive: Math.min(tokenCount, occurrence.close.boundary.tokenIndex),
+ contract: "svml.caption-word-subset@1",
+ id: `role:${normalized}`,
+ sequenceId: sequence.id,
+ wordIds: sequence.words.filter((word) => word.role === normalized).map((word) => word.id),
};
}
-export function displayAtomMatchesSelection(
- atom: CaptionDisplayAtom,
- selection: NarrativeSelectionRef,
- tokenCount: number,
-): boolean {
- return selection.occurrences.some((occurrence) => {
- const range = occurrenceRange(occurrence, tokenCount);
- return atom.sourceTokenStart < range.endExclusive && atom.sourceTokenEndExclusive > range.start;
- });
-}
-
-export function displayTextForAtoms(
+export function displayTextForWords(
narrative: Narrative,
- selected: readonly CaptionDisplayAtom[],
+ universe: readonly CaptionWord[],
+ selected: readonly CaptionWord[],
): string {
- const byRegion = new Map<string, CaptionDisplayAtom[]>();
- for (const atom of selected) {
- const values = byRegion.get(atom.regionId) ?? [];
- values.push(atom);
- byRegion.set(atom.regionId, values);
+ const byRegion = new Map<string, CaptionWord[]>();
+ for (const word of selected) {
+ const values = byRegion.get(word.regionId) ?? [];
+ values.push(word);
+ byRegion.set(word.regionId, values);
}
const parts: string[] = [];
for (const region of narrative.captionProjection.regions) {
- const atoms = byRegion.get(region.id);
- if (atoms === undefined || atoms.length === 0) continue;
- const ordered = [...atoms].sort((left, right) => left.displayStart - right.displayStart);
- const all = displayWords(region.display);
+ const words = byRegion.get(region.id);
+ if (words === undefined || words.length === 0) continue;
+ const ordered = [...words].sort((left, right) => left.displayStart - right.displayStart);
+ const all = universe
+ .filter((word) => word.regionId === region.id)
+ .sort((left, right) => left.displayStart - right.displayStart);
const first = ordered[0]!;
const last = ordered.at(-1)!;
- const after = all.find((word) => word.start > last.displayStart);
- parts.push(region.display.slice(first.displayStart, after?.start ?? region.display.length).trim());
+ const after = all.find((word) => word.displayStart > last.displayStart);
+ parts.push(region.display.slice(first.displayStart, after?.displayStart ?? region.display.length).trim());
}
return parts.join(" ").replace(/\s+/gu, " ").replace(/\s+([,.;:!?])/gu, "$1").trim();
}
diff --git a/packages/caption/src/fragment.ts b/packages/caption/src/fragment.ts
index bc9e7a0b..5ee08798 100644
--- a/packages/caption/src/fragment.ts
+++ b/packages/caption/src/fragment.ts
@@ -1,8 +1,5 @@
import { narrativeTypes } from "@narratage/narrative";
-import { programSpaceTypes } from "@narratage/program-space";
import { semanticMapTypes } from "@narratage/semantic-map";
-import { compositionTypes } from "@narratage/composition";
-import type { VisualTrack } from "@narratage/composition";
import { sealGraphFragment } from "@narratage/elaborator";
import { captionProducers, captionTypes } from "./manifest.js";
@@ -10,71 +7,6 @@ import { captionProducers, captionTypes } from "./manifest.js";
const input = (name: string) => ({ kind: "fragment-input" as const, name });
const operation = (id: string) => ({ kind: "fragment-operation" as const, operation: id });
-/** Provider-free Caption base: authored display text + measured map -> peer VisualTrack. */
-export const captionTrackSurfaceFragment = sealGraphFragment({
- name: "@narratage/caption/track-surface@1",
- inputs: [
- { name: "narrative", type: narrativeTypes.narrative },
- { name: "map", type: semanticMapTypes.complete },
- { name: "program", type: captionTypes.trackProgram },
- { name: "space", type: programSpaceTypes.programSpace },
- ],
- operations: [
- {
- id: "caption:temporalize",
- producer: captionProducers.temporalize,
- inputs: { narrative: input("narrative"), map: input("map") },
- result: { kind: "output", name: "caption" },
- },
- {
- id: "caption:render",
- producer: captionProducers.renderTrack,
- inputs: { caption: operation("caption:temporalize"), program: input("program"), space: input("space") },
- result: { kind: "output", name: "track" },
- },
- ],
- exports: [{
- name: "track",
- type: compositionTypes.visualTrack,
- root: operation("caption:render"),
- semanticInputs: ["narrative", "map", "program", "space"],
- fidelity: "exact",
- }],
-});
-
-/** Planner-neutral Cue/field facts join the independent measured map only at Caption lowering. */
-export const plannedCaptionTrackSurfaceFragment = sealGraphFragment({
- name: "@narratage/caption/planned-track-surface@1",
- inputs: [
- { name: "narrative", type: narrativeTypes.narrative },
- { name: "map", type: semanticMapTypes.complete },
- { name: "plan", type: captionTypes.plan },
- { name: "program", type: captionTypes.program },
- { name: "space", type: programSpaceTypes.programSpace },
- ],
- operations: [
- {
- id: "caption:temporalize-plan",
- producer: captionProducers.temporalizePlan,
- inputs: { narrative: input("narrative"), map: input("map"), program: input("program"), plan: input("plan") },
- result: { kind: "output", name: "caption" },
- },
- {
- id: "caption:render-plan",
- producer: captionProducers.renderProgram,
- inputs: { caption: operation("caption:temporalize-plan"), program: input("program"), space: input("space") },
- result: { kind: "output", name: "track" },
- },
- ],
- exports: [{
- name: "track",
- type: compositionTypes.visualTrack,
- root: operation("caption:render-plan"),
- semanticInputs: ["narrative", "map", "plan", "program", "space"],
- fidelity: "exact",
- }],
-});
-
export const captionTimingFragment = sealGraphFragment({
name: "@narratage/caption/timing@1",
inputs: [
@@ -96,25 +28,30 @@ export const captionTimingFragment = sealGraphFragment({
}],
});
-/** Official caption lowering; the exported result is an ordinary peer VisualTrack. */
-export const captionTrackFragment = sealGraphFragment({
- name: "@narratage/caption/track@1",
+/** Cue and field facts meet measured speech time here, before any Style renderer. */
+export const plannedCaptionTimingFragment = sealGraphFragment({
+ name: "@narratage/caption/planned-timing@1",
inputs: [
- { name: "caption", type: captionTypes.timedProjection },
- { name: "program", type: captionTypes.trackProgram },
- { name: "space", type: programSpaceTypes.programSpace },
+ { name: "narrative", type: narrativeTypes.narrative },
+ { name: "map", type: semanticMapTypes.complete },
+ { name: "words", type: narrativeTypes.captionWordSequence },
+ { name: "plan", type: captionTypes.plan },
+ { name: "program", type: captionTypes.program },
],
operations: [{
- id: "render-caption-track",
- producer: captionProducers.renderTrack,
- inputs: { caption: input("caption"), program: input("program"), space: input("space") },
- result: { kind: "output", name: "track" },
+ id: "temporalize-caption-plan",
+ producer: captionProducers.temporalizePlan,
+ inputs: {
+ narrative: input("narrative"), map: input("map"), words: input("words"),
+ plan: input("plan"), program: input("program"),
+ },
+ result: { kind: "output", name: "caption" },
}],
exports: [{
- name: "track",
- type: compositionTypes.visualTrack,
- root: operation("render-caption-track"),
- semanticInputs: ["caption", "program", "space"],
+ name: "caption",
+ type: captionTypes.timedProjection,
+ root: operation("temporalize-caption-plan"),
+ semanticInputs: ["narrative", "map", "words", "plan", "program"],
fidelity: "exact",
}],
});
diff --git a/packages/caption/src/index.ts b/packages/caption/src/index.ts
index 38cbdc7c..e2d8af5e 100644
--- a/packages/caption/src/index.ts
+++ b/packages/caption/src/index.ts
@@ -1,50 +1,36 @@
export { CaptionProjectionError } from "./error.js";
-export { captionDisplayAtoms, displayAtomMatchesSelection, displayTextForAtoms } from "./display.js";
+export {
+ assertCaptionWordSequence,
+ assertCaptionWordSubset,
+ captionWordsForRole,
+ displayTextForWords,
+} from "./display.js";
export { captionComponent } from "./component.js";
-export { captionTrackSurfaceFragment, plannedCaptionTrackSurfaceFragment } from "./fragment.js";
+export { captionTimingFragment, plannedCaptionTimingFragment } from "./fragment.js";
export {
captionImplementationDigest,
captionProgramSchema,
captionProgramSurfaceImplementationDigest,
captionPlanImplementationDigest,
captionPlanSchema,
- captionSurfaceImplementationDigest,
captionStyleSchema,
- captionStyleSurfaceImplementationDigest,
captionManifest,
captionModuleRef,
captionProducers,
- captionTrackProgramSchema,
captionTypes,
captionValidatorDigests,
timedCaptionProjectionSchema,
} from "./manifest.js";
-export {
- decodeCaptionProgramSurface,
- decodeCaptionStyleSurface,
- decodeCaptionTrackSurface,
-} from "./surface.js";
-export { planCaptionPresentation } from "./presentation.js";
+export { decodeCaptionProgramSurface } from "./surface.js";
export { assertCaptionPlan, assertCaptionPlanForProgram, sealCaptionPlan } from "./plan.js";
export {
assertCaptionProgram,
- assertCaptionProgramForNarrative,
+ assertCaptionProgramForWords,
assertCaptionStyle,
resolveCaptionProgram,
sealCaptionProgram,
sealCaptionStyle,
} from "./style.js";
-export type { CaptionStyleApplication, CaptionStyleSelector } from "./style.js";
-export { temporalizeCaption, temporalizeCaptionPlan } from "./temporalize.js";
-export {
- assertCaptionTrackProgram,
- assertTimedCaptionProjection,
- defaultCaptionTrackProgram,
- renderCaptionTrack,
- renderCaptionProgram,
- renderCaptionTrackImplementationDigest,
- renderCaptionProgramImplementationDigest,
- sealCaptionTrackProgram,
-} from "./track.js";
+export type { CaptionStyleApplication } from "./style.js";
+export { assertTimedCaptionProjection, temporalizeCaption, temporalizeCaptionPlan } from "./temporalize.js";
export type * from "./types.js";
-export { captionTimingFragment, captionTrackFragment } from "./fragment.js";
diff --git a/packages/caption/src/manifest.ts b/packages/caption/src/manifest.ts
index 995125f2..4e3ac56b 100644
--- a/packages/caption/src/manifest.ts
+++ b/packages/caption/src/manifest.ts
@@ -1,29 +1,18 @@
import { narrativeDependency, narrativeTypes } from "@narratage/narrative";
-import { programSpaceDependency, programSpaceTypes } from "@narratage/program-space";
import { semanticMapDependency, semanticMapTypes } from "@narratage/semantic-map";
-import { compositionDependency, compositionTypes } from "@narratage/composition";
-import type { Track } from "@narratage/composition";
import { digestOf } from "@narratage/protocol";
import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@narratage/protocol";
-import {
- renderCaptionProgramImplementationDigest,
- renderCaptionTrackImplementationDigest,
-} from "./track.js";
-
export const captionModuleRef = { name: "@narratage/caption", version: "0.0.0-dev" } as const;
export const captionProducers = {
temporalize: { module: captionModuleRef, name: "temporalize-caption" },
temporalizePlan: { module: captionModuleRef, name: "temporalize-caption-plan" },
- renderProgram: { module: captionModuleRef, name: "render-caption-program" },
- renderTrack: { module: captionModuleRef, name: "render-caption-track" },
} satisfies Record<string, ProducerRef>;
export const captionTypes = {
style: { module: captionModuleRef, name: "CaptionStyle" },
program: { module: captionModuleRef, name: "CaptionProgram" },
plan: { module: captionModuleRef, name: "CaptionPlan" },
timedProjection: { module: captionModuleRef, name: "TimedCaptionProjection" },
- trackProgram: { module: captionModuleRef, name: "CaptionTrackProgram" },
} satisfies Record<string, TypeRef>;
export const captionImplementationDigest = digestOf("@narratage/caption/temporalize@1");
export const captionPlanImplementationDigest = digestOf("@narratage/caption/temporalize-plan@1");
@@ -32,10 +21,7 @@ export const captionValidatorDigests = {
program: digestOf("@narratage/caption/validate-program@1"),
plan: digestOf("@narratage/caption/validate-plan@1"),
timedProjection: digestOf("@narratage/caption/validate-timed-projection@1"),
- trackProgram: digestOf("@narratage/caption/validate-track-program@1"),
} as const;
-export const captionSurfaceImplementationDigest = digestOf("@narratage/caption/track-surface@1");
-export const captionStyleSurfaceImplementationDigest = digestOf("@narratage/caption/style-surface@1");
export const captionProgramSurfaceImplementationDigest = digestOf("@narratage/caption/program-surface@1");
const string = { kind: "string", minLength: 1 } as const;
@@ -45,135 +31,80 @@ const object = (
fields: Readonly<Record<string, { readonly schema: ValueSchema; readonly optional?: boolean }>>,
): ValueSchema => ({ kind: "object", fields });
-const timedCaptionRefinement = object({
- id: { schema: string },
- display: { schema: string },
- displayStart: { schema: integer },
- displayEnd: { schema: integer },
- sourceTokenIds: { schema: { kind: "array", items: string } },
- startSec: { schema: number },
- endSec: { schema: number },
- relation: { schema: { kind: "literal", value: "exact" } },
-});
-const timedCaptionRegion = object({
- id: { schema: string },
- runId: { schema: string, optional: true },
- styleId: { schema: string, optional: true },
- display: { schema: { kind: "string" } },
- segmentId: { schema: string },
- kind: { schema: { kind: "string", enum: ["identity", "alias", "hidden"] } },
- sourceTokenIds: { schema: { kind: "array", items: string } },
- startSec: { schema: number },
- endSec: { schema: number },
- refinements: { schema: { kind: "array", items: timedCaptionRefinement } },
- fields: { schema: { kind: "array", items: object({
- declarationId: { schema: string },
- atomId: { schema: string },
- value: { schema: string },
- }) }, optional: true },
-});
const captionFieldAssignment = object({
- declarationId: { schema: string },
- atomId: { schema: string },
- value: { schema: string },
+ declarationId: { schema: string }, wordId: { schema: string }, value: { schema: string },
});
const captionPlannedCue = object({
id: { schema: string },
- atomIds: { schema: { kind: "array", minItems: 1, items: string } },
+ wordIds: { schema: { kind: "array", minItems: 1, items: string } },
fields: { schema: { kind: "array", items: captionFieldAssignment } },
});
const captionPlannedRun = object({
- id: { schema: string },
- styleId: { schema: string },
+ id: { schema: string }, styleId: { schema: string },
cues: { schema: { kind: "array", minItems: 1, items: captionPlannedCue } },
});
export const captionPlanSchema: ValueSchema = object({
contract: { schema: { kind: "literal", value: "svml.caption-plan@1" } },
runs: { schema: { kind: "array", minItems: 1, items: captionPlannedRun } },
});
+
+const timedCaptionRefinement = object({
+ id: { schema: string }, display: { schema: string }, displayStart: { schema: integer },
+ displayEnd: { schema: integer }, sourceTokenIds: { schema: { kind: "array", items: string } },
+ startSec: { schema: number }, endSec: { schema: number },
+ relation: { schema: { kind: "literal", value: "exact" } },
+});
+const timedCaptionRegion = object({
+ id: { schema: string }, runId: { schema: string, optional: true }, styleId: { schema: string, optional: true },
+ display: { schema: { kind: "string" } }, segmentId: { schema: string },
+ kind: { schema: { kind: "string", enum: ["identity", "alias", "hidden"] } },
+ sourceTokenIds: { schema: { kind: "array", items: string } }, startSec: { schema: number },
+ endSec: { schema: number }, refinements: { schema: { kind: "array", items: timedCaptionRefinement } },
+ wordIds: { schema: { kind: "array", items: string }, optional: true },
+ fields: { schema: { kind: "array", items: captionFieldAssignment }, optional: true },
+});
export const timedCaptionProjectionSchema: ValueSchema = object({
contract: { schema: { kind: "literal", value: "svml.timed-caption-projection@1" } },
- text: { schema: { kind: "string" } },
- regions: { schema: { kind: "array", items: timedCaptionRegion } },
-});
-
-const captionTrackStyleSchema = object({
- fontFamily: { schema: string },
- fontSizePx: { schema: number },
- fontWeight: { schema: number },
- color: { schema: string },
- backgroundColor: { schema: string, optional: true },
- paddingXPx: { schema: number },
- paddingYPx: { schema: number },
- borderRadiusPx: { schema: number },
- bottomPercent: { schema: number },
- maxWidthPercent: { schema: number },
- leftPercent: { schema: number, optional: true },
- topPercent: { schema: number, optional: true },
- widthPercent: { schema: number, optional: true },
- lineHeight: { schema: number, optional: true },
- textAlign: { schema: { kind: "string", enum: ["left", "center", "right"] } },
- });
-
-export const captionTrackProgramSchema: ValueSchema = object({
- contract: { schema: { kind: "literal", value: "svml.caption-track-program@1" } },
- id: { schema: string },
- mode: { schema: { kind: "string", enum: ["whole", "proportional-word", "character-flow"] } },
- stacking: { schema: object({ order: { schema: integer }, tieBreak: { schema: string } }) },
- style: { schema: captionTrackStyleSchema },
+ text: { schema: { kind: "string" } }, regions: { schema: { kind: "array", items: timedCaptionRegion } },
});
const captionFieldValueSchema: ValueSchema = {
kind: "oneOf",
variants: [
object({ kind: { schema: { kind: "literal", value: "boolean" } } }),
- object({
- kind: { schema: { kind: "literal", value: "enum" } },
- values: { schema: { kind: "array", minItems: 1, items: string } },
- }),
- object({
- kind: { schema: { kind: "literal", value: "number" } },
+ object({ kind: { schema: { kind: "literal", value: "enum" } },
+ values: { schema: { kind: "array", minItems: 1, items: string } } }),
+ object({ kind: { schema: { kind: "literal", value: "number" } },
minimum: { schema: { kind: "number" }, optional: true },
- maximum: { schema: { kind: "number" }, optional: true },
- }),
+ maximum: { schema: { kind: "number" }, optional: true } }),
],
};
const captionFieldDeclaration = object({
- id: { schema: string },
- value: { schema: captionFieldValueSchema },
- instruction: { schema: string },
- minimumPerCue: { schema: integer },
- maximumPerCue: { schema: integer },
+ id: { schema: string }, value: { schema: captionFieldValueSchema }, instruction: { schema: string },
+ minimumPerCue: { schema: integer }, maximumPerCue: { schema: integer },
});
export const captionStyleSchema: ValueSchema = object({
contract: { schema: { kind: "literal", value: "svml.caption-style@1" } },
id: { schema: string },
planning: { schema: object({
- cueInstruction: { schema: string },
+ cue: { schema: object({
+ minimumWords: { schema: integer }, maximumWords: { schema: integer }, instruction: { schema: string },
+ }) },
fields: { schema: { kind: "array", items: captionFieldDeclaration } },
}) },
- presentation: { schema: object({
- mode: { schema: { kind: "string", enum: ["whole", "proportional-word", "character-flow"] } },
- stackingOrder: { schema: integer },
- style: { schema: captionTrackStyleSchema },
+ rendering: { schema: object({
+ family: { schema: string },
+ parameters: { schema: { kind: "object", fields: {}, allowUnknown: true } },
}) },
});
-const captionDisplayAtom = object({
- id: { schema: string }, index: { schema: integer }, regionId: { schema: string }, segmentId: { schema: string },
- turnId: { schema: string }, role: { schema: string, optional: true }, text: { schema: string },
- displayStart: { schema: integer }, displayEnd: { schema: integer }, sourceTokenStart: { schema: integer },
- sourceTokenEndExclusive: { schema: integer },
- correspondence: { schema: { kind: "string", enum: ["exact", "region-envelope"] } },
-});
const captionProgramRun = object({
id: { schema: string }, styleId: { schema: string },
- atomIds: { schema: { kind: "array", minItems: 1, items: string } },
+ wordIds: { schema: { kind: "array", minItems: 1, items: string } },
});
export const captionProgramSchema: ValueSchema = object({
contract: { schema: { kind: "literal", value: "svml.caption-program@1" } },
- id: { schema: string }, defaultStyleId: { schema: string },
+ id: { schema: string }, wordSequenceId: { schema: string }, defaultStyleId: { schema: string },
styles: { schema: { kind: "array", minItems: 1, items: captionStyleSchema } },
- atoms: { schema: { kind: "array", minItems: 1, items: captionDisplayAtom } },
runs: { schema: { kind: "array", minItems: 1, items: captionProgramRun } },
});
@@ -181,136 +112,49 @@ export const captionManifest: ModuleManifest = {
format: "svml.module@1",
name: captionModuleRef.name,
version: captionModuleRef.version,
- dependencies: [
- narrativeDependency,
- programSpaceDependency,
- semanticMapDependency,
- compositionDependency,
- ],
+ dependencies: [narrativeDependency, semanticMapDependency],
types: [
- {
- name: captionTypes.style.name,
- schema: captionStyleSchema,
- validator: { abi: "svml.type-validator@1", implementation: {
- kind: "registered", locator: "@narratage/caption/validate-style", digest: captionValidatorDigests.style,
- } },
- },
- {
- name: captionTypes.program.name,
- schema: captionProgramSchema,
+ { name: captionTypes.style.name, schema: captionStyleSchema, validator: { abi: "svml.type-validator@1", implementation: {
+ kind: "registered", locator: "@narratage/caption/validate-style", digest: captionValidatorDigests.style,
+ } } },
+ { name: captionTypes.program.name, schema: captionProgramSchema, validator: { abi: "svml.type-validator@1", implementation: {
+ kind: "registered", locator: "@narratage/caption/validate-program", digest: captionValidatorDigests.program,
+ } } },
+ { name: captionTypes.plan.name, schema: captionPlanSchema, validator: { abi: "svml.type-validator@1", implementation: {
+ kind: "registered", locator: "@narratage/caption/validate-plan", digest: captionValidatorDigests.plan,
+ } } },
+ { name: captionTypes.timedProjection.name, schema: timedCaptionProjectionSchema,
validator: { abi: "svml.type-validator@1", implementation: {
- kind: "registered", locator: "@narratage/caption/validate-program", digest: captionValidatorDigests.program,
- } },
- },
- {
- name: captionTypes.plan.name,
- schema: captionPlanSchema,
- validator: {
- abi: "svml.type-validator@1",
- implementation: {
- kind: "registered",
- locator: "@narratage/caption/validate-plan",
- digest: captionValidatorDigests.plan,
- },
- },
- },
- {
- name: captionTypes.timedProjection.name,
- schema: timedCaptionProjectionSchema,
- validator: {
- abi: "svml.type-validator@1",
- implementation: {
- kind: "registered",
- locator: "@narratage/caption/validate-timed-projection",
- digest: captionValidatorDigests.timedProjection,
- },
- },
- },
- {
- name: captionTypes.trackProgram.name,
- schema: captionTrackProgramSchema,
- validator: {
- abi: "svml.type-validator@1",
- implementation: {
- kind: "registered",
- locator: "@narratage/caption/validate-track-program",
- digest: captionValidatorDigests.trackProgram,
- },
- },
- },
+ kind: "registered", locator: "@narratage/caption/validate-timed-projection",
+ digest: captionValidatorDigests.timedProjection,
+ } } },
],
capabilities: [],
- surfaces: [
- {
- name: "style", tag: "Style", mode: "structured", outputs: [captionTypes.style],
- implementation: { kind: "trusted-frontend-surface", locator: "@narratage/caption/style-surface", digest: captionStyleSurfaceImplementationDigest },
- },
- {
- name: "program", tag: "Program", mode: "structured", outputs: [captionTypes.program],
- implementation: { kind: "trusted-frontend-surface", locator: "@narratage/caption/program-surface", digest: captionProgramSurfaceImplementationDigest },
- },
- {
- name: "track", tag: "Track", mode: "structured",
- outputs: [captionTypes.trackProgram, compositionTypes.visualTrack],
- implementation: { kind: "trusted-frontend-surface", locator: "@narratage/caption/track-surface", digest: captionSurfaceImplementationDigest },
- },
- ],
+ surfaces: [{
+ name: "program", tag: "Program", mode: "structured", outputs: [captionTypes.program],
+ implementation: { kind: "trusted-frontend-surface", locator: "@narratage/caption/program-surface",
+ digest: captionProgramSurfaceImplementationDigest },
+ }],
producers: [
{
name: captionProducers.temporalize.name,
- inputs: [
- { name: "narrative", type: narrativeTypes.narrative },
- { name: "map", type: semanticMapTypes.complete },
- ],
- outputs: [{ name: "caption", type: captionTypes.timedProjection }],
- needs: [],
- implementation: {
- kind: "registered",
- locator: "@narratage/caption/temporalize",
- digest: captionImplementationDigest,
- },
+ inputs: [{ name: "narrative", type: narrativeTypes.narrative }, { name: "map", type: semanticMapTypes.complete }],
+ outputs: [{ name: "caption", type: captionTypes.timedProjection }], needs: [],
+ implementation: { kind: "registered", locator: "@narratage/caption/temporalize",
+ digest: captionImplementationDigest },
},
{
name: captionProducers.temporalizePlan.name,
inputs: [
{ name: "narrative", type: narrativeTypes.narrative },
{ name: "map", type: semanticMapTypes.complete },
+ { name: "words", type: narrativeTypes.captionWordSequence },
{ name: "program", type: captionTypes.program },
{ name: "plan", type: captionTypes.plan },
],
- outputs: [{ name: "caption", type: captionTypes.timedProjection }],
- needs: [],
- implementation: {
- kind: "registered",
- locator: "@narratage/caption/temporalize-plan",
- digest: captionPlanImplementationDigest,
- },
- },
- {
- name: captionProducers.renderProgram.name,
- inputs: [
- { name: "caption", type: captionTypes.timedProjection },
- { name: "program", type: captionTypes.program },
- { name: "space", type: programSpaceTypes.programSpace },
- ],
- outputs: [{ name: "track", type: compositionTypes.visualTrack }],
- needs: [],
- implementation: { kind: "registered", locator: "@narratage/caption/render-program", digest: renderCaptionProgramImplementationDigest },
- },
- {
- name: captionProducers.renderTrack.name,
- inputs: [
- { name: "caption", type: captionTypes.timedProjection },
- { name: "program", type: captionTypes.trackProgram },
- { name: "space", type: programSpaceTypes.programSpace },
- ],
- outputs: [{ name: "track", type: compositionTypes.visualTrack }],
- needs: [],
- implementation: {
- kind: "registered",
- locator: "@narratage/caption/render-track",
- digest: renderCaptionTrackImplementationDigest,
- },
+ outputs: [{ name: "caption", type: captionTypes.timedProjection }], needs: [],
+ implementation: { kind: "registered", locator: "@narratage/caption/temporalize-plan",
+ digest: captionPlanImplementationDigest },
},
],
};
diff --git a/packages/caption/src/plan.ts b/packages/caption/src/plan.ts
index 3c1257af..ce21d2e3 100644
--- a/packages/caption/src/plan.ts
+++ b/packages/caption/src/plan.ts
@@ -12,7 +12,7 @@ function content(value: CaptionPlan) {
styleId: run.styleId,
cues: run.cues.map((cue) => ({
id: cue.id,
- atomIds: [...cue.atomIds],
+ wordIds: [...cue.wordIds],
fields: cue.fields.map((field) => ({ ...field })),
})),
})),
@@ -32,7 +32,7 @@ export function assertCaptionPlan(value: unknown): asserts value is CaptionPlan
assert(plan.contract === "svml.caption-plan@1", "Unsupported CaptionPlan contract");
assert(Array.isArray(plan.runs) && plan.runs.length > 0, "CaptionPlan runs are empty");
const runIds = new Set<string>();
- const atomIds = new Set<string>();
+ const wordIds = new Set<string>();
const cueIds = new Set<string>();
for (const run of plan.runs) {
assert(run.id.length > 0 && !runIds.has(run.id) && run.styleId.length > 0,
@@ -40,18 +40,18 @@ export function assertCaptionPlan(value: unknown): asserts value is CaptionPlan
runIds.add(run.id);
assert(run.cues.length > 0, `CaptionPlan run ${run.id} has no Cues`);
for (const cue of run.cues) {
- assert(cue.id.length > 0 && !cueIds.has(cue.id) && cue.atomIds.length > 0,
+ assert(cue.id.length > 0 && !cueIds.has(cue.id) && cue.wordIds.length > 0,
`CaptionPlan run ${run.id} contains an empty or repeated Cue`);
cueIds.add(cue.id);
- assert(new Set(cue.atomIds).size === cue.atomIds.length, `CaptionPlan Cue ${cue.id} repeats a display atom`);
- for (const atomId of cue.atomIds) {
- assert(!atomIds.has(atomId), `CaptionPlan repeats display atom ${atomId}`);
- atomIds.add(atomId);
+ assert(new Set(cue.wordIds).size === cue.wordIds.length, `CaptionPlan Cue ${cue.id} repeats a display word`);
+ for (const wordId of cue.wordIds) {
+ assert(!wordIds.has(wordId), `CaptionPlan repeats display word ${wordId}`);
+ wordIds.add(wordId);
}
for (const field of cue.fields) {
- assert(field.declarationId.length > 0 && field.atomId.length > 0 && field.value.length > 0,
+ assert(field.declarationId.length > 0 && field.wordId.length > 0 && field.value.length > 0,
`CaptionPlan Cue ${cue.id} field is invalid`);
- assert(cue.atomIds.includes(field.atomId), `CaptionPlan Cue ${cue.id} field lies outside its Cue`);
+ assert(cue.wordIds.includes(field.wordId), `CaptionPlan Cue ${cue.id} field lies outside its Cue`);
}
}
}
@@ -82,18 +82,21 @@ export function assertCaptionPlanForProgram(plan: CaptionPlan, program: CaptionP
const actual = plan.runs[index]!;
assert(actual.id === expected.id && actual.styleId === expected.styleId,
`CaptionPlan run ${actual.id} differs from Program order or Style`);
- assert(actual.cues.flatMap((cue) => cue.atomIds).join("\0") === expected.atomIds.join("\0"),
- `CaptionPlan run ${actual.id} does not partition its Program atoms exactly in order`);
+ assert(actual.cues.flatMap((cue) => cue.wordIds).join("\0") === expected.wordIds.join("\0"),
+ `CaptionPlan run ${actual.id} does not partition its Program words exactly in order`);
const style = styles.get(expected.styleId);
assert(style !== undefined, `Caption Program Style ${expected.styleId} is absent`);
const declarations = new Map(style.planning.fields.map((field) => [field.id, field]));
for (const cue of actual.cues) {
+ assert(cue.wordIds.length >= style.planning.cue.minimumWords
+ && cue.wordIds.length <= style.planning.cue.maximumWords,
+ `Caption Cue ${cue.id} violates Style ${style.id} word-count bounds`);
const unique = new Set<string>();
for (const field of cue.fields) {
const declaration = declarations.get(field.declarationId);
assert(declaration !== undefined, `Caption field ${field.declarationId} is not declared by Style ${style.id}`);
- const key = `${field.declarationId}\0${field.atomId}`;
- assert(!unique.has(key), `Caption Cue ${cue.id} repeats field ${field.declarationId} on ${field.atomId}`);
+ const key = `${field.declarationId}\0${field.wordId}`;
+ assert(!unique.has(key), `Caption Cue ${cue.id} repeats field ${field.declarationId} on ${field.wordId}`);
unique.add(key);
assertFieldValue(declaration, field.value);
}
diff --git a/packages/caption/src/presentation.ts b/packages/caption/src/presentation.ts
deleted file mode 100644
index 63912e2c..00000000
--- a/packages/caption/src/presentation.ts
+++ /dev/null
@@ -1,137 +0,0 @@
-import type {
- CaptionPresentationMode,
- CaptionPresentationPlan,
- CaptionPresentationUnit,
- TimedCaptionRegion,
-} from "./types.js";
-
-const DISPLAY_WORD = /[\p{L}\p{M}\p{N}]+(?:['’.-][\p{L}\p{M}\p{N}]+)*/gu;
-
-type DisplayPart = {
- readonly display: string;
- readonly start: number;
- readonly end: number;
-};
-
-function displayWords(value: string): DisplayPart[] {
- const words: DisplayPart[] = [];
- DISPLAY_WORD.lastIndex = 0;
- let match: RegExpExecArray | null;
- while ((match = DISPLAY_WORD.exec(value))) {
- words.push({ display: match[0], start: match.index, end: match.index + match[0].length });
- }
- return words;
-}
-
-function wholeRegion(region: TimedCaptionRegion): CaptionPresentationUnit[] {
- if (!region.display) return [];
- return [{
- id: `${region.id}:whole`,
- regionId: region.id,
- ...(region.runId === undefined ? {} : { runId: region.runId }),
- ...(region.styleId === undefined ? {} : { styleId: region.styleId }),
- display: region.display,
- displayStart: 0,
- displayEnd: region.display.length,
- startSec: region.startSec,
- endSec: region.endSec,
- ...(region.fields === undefined ? {} : { fields: region.fields.map((field) => ({ ...field })) }),
- }];
-}
-
-function proportionalWords(region: TimedCaptionRegion): CaptionPresentationUnit[] {
- const words = displayWords(region.display);
- if (!words.length) return [];
- const exact = new Map(region.refinements.map((item) => [`${item.displayStart}:${item.displayEnd}`, item]));
- const units: Array<CaptionPresentationUnit | undefined> = words.map((word, index) => {
- const refinement = exact.get(`${word.start}:${word.end}`);
- return refinement ? {
- id: `${region.id}:word:${index + 1}`,
- regionId: region.id,
- ...(region.runId === undefined ? {} : { runId: region.runId }),
- ...(region.styleId === undefined ? {} : { styleId: region.styleId }),
- display: word.display,
- displayStart: word.start,
- displayEnd: word.end,
- startSec: refinement.startSec,
- endSec: refinement.endSec,
- } : undefined;
- });
-
- let cursor = 0;
- while (cursor < words.length) {
- if (units[cursor]) {
- cursor += 1;
- continue;
- }
- const runStart = cursor;
- while (cursor < words.length && !units[cursor]) cursor += 1;
- const runEnd = cursor;
- const left = runStart === 0 ? region.startSec : units[runStart - 1]!.endSec;
- const right = runEnd === words.length ? region.endSec : Math.max(left, units[runEnd]!.startSec);
- const weights = words.slice(runStart, runEnd).map((word) => Math.max(1, [...word.display].length));
- const total = weights.reduce((sum, weight) => sum + weight, 0);
- let used = 0;
- for (let index = runStart; index < runEnd; index += 1) {
- const weight = weights[index - runStart]!;
- const startSec = left + (right - left) * used / total;
- used += weight;
- const endSec = left + (right - left) * used / total;
- const word = words[index]!;
- units[index] = {
- id: `${region.id}:word:${index + 1}`,
- regionId: region.id,
- ...(region.runId === undefined ? {} : { runId: region.runId }),
- ...(region.styleId === undefined ? {} : { styleId: region.styleId }),
- display: word.display,
- displayStart: word.start,
- displayEnd: word.end,
- startSec,
- endSec,
- };
- }
- }
- return units as CaptionPresentationUnit[];
-}
-
-function characterFlow(region: TimedCaptionRegion): CaptionPresentationUnit[] {
- const characters = [...region.display];
- if (!characters.length) return [];
- const duration = region.endSec - region.startSec;
- const units: CaptionPresentationUnit[] = [];
- let sourceOffset = 0;
- for (const [index, character] of characters.entries()) {
- const start = sourceOffset;
- sourceOffset += character.length;
- if (!character.trim()) continue;
- units.push({
- id: `${region.id}:character:${index + 1}`,
- regionId: region.id,
- ...(region.runId === undefined ? {} : { runId: region.runId }),
- ...(region.styleId === undefined ? {} : { styleId: region.styleId }),
- display: character,
- displayStart: start,
- displayEnd: sourceOffset,
- startSec: region.startSec + duration * index / characters.length,
- endSec: region.startSec + duration * (index + 1) / characters.length,
- });
- }
- return units;
-}
-
-export function planCaptionPresentation(
- projection: { readonly regions: readonly TimedCaptionRegion[] },
- mode: CaptionPresentationMode = "whole",
-): CaptionPresentationPlan {
- const units = projection.regions.flatMap((region) =>
- mode === "whole"
- ? wholeRegion(region)
- : mode === "proportional-word"
- ? proportionalWords(region)
- : characterFlow(region));
- return {
- contract: "svml.caption-presentation-plan@1" as const,
- mode,
- units,
- };
-}
diff --git a/packages/caption/src/style.ts b/packages/caption/src/style.ts
index ef9a9233..c55d00cb 100644
--- a/packages/caption/src/style.ts
+++ b/packages/caption/src/style.ts
@@ -1,21 +1,16 @@
-import type { Narrative, NarrativeSelectionRef } from "@narratage/narrative";
+import type { CaptionWord, CaptionWordSequence, CaptionWordSubset } from "@narratage/narrative";
import { canonicalize, digestOf } from "@narratage/protocol";
-import { captionDisplayAtoms, displayAtomMatchesSelection } from "./display.js";
+import { assertCaptionWordSequence, assertCaptionWordSubset } from "./display.js";
import type {
- CaptionDisplayAtom,
CaptionFieldDeclaration,
CaptionProgram,
CaptionStyleIntent,
} from "./types.js";
-export type CaptionStyleSelector =
- | { readonly kind: "role"; readonly role: string }
- | { readonly kind: "selection"; readonly selection: NarrativeSelectionRef };
-
export type CaptionStyleApplication = {
readonly id: string;
- readonly selector: CaptionStyleSelector;
+ readonly words: CaptionWordSubset;
readonly style: CaptionStyleIntent;
};
@@ -43,15 +38,22 @@ function normalizedField(field: CaptionFieldDeclaration): CaptionFieldDeclaratio
};
}
-function styleContent(value: CaptionStyleIntent) {
+function styleContent(value: CaptionStyleIntent): CaptionStyleIntent {
return canonicalize({
contract: "svml.caption-style@1",
id: value.id,
planning: {
- cueInstruction: value.planning.cueInstruction.trim(),
+ cue: {
+ minimumWords: value.planning.cue.minimumWords,
+ maximumWords: value.planning.cue.maximumWords,
+ instruction: value.planning.cue.instruction.trim(),
+ },
fields: value.planning.fields.map(normalizedField),
},
- presentation: canonicalize(value.presentation),
+ rendering: {
+ family: value.rendering.family.trim(),
+ parameters: canonicalize(value.rendering.parameters),
+ },
}) as unknown as CaptionStyleIntent;
}
@@ -63,7 +65,12 @@ export function sealCaptionStyle(value: CaptionStyleIntent): CaptionStyleIntent
export function assertCaptionStyle(value: CaptionStyleIntent): void {
assert(value.contract === "svml.caption-style@1" && ID.test(value.id), "Caption Style identity is invalid");
- assert(value.planning.cueInstruction.length > 0, `Caption Style ${value.id} Cue instruction is empty`);
+ const cue = value.planning.cue;
+ assert(Number.isSafeInteger(cue.minimumWords) && cue.minimumWords > 0,
+ `Caption Style ${value.id} Cue minimum is invalid`);
+ assert(Number.isSafeInteger(cue.maximumWords) && cue.maximumWords >= cue.minimumWords,
+ `Caption Style ${value.id} Cue maximum is invalid`);
+ assert(cue.instruction.length > 0, `Caption Style ${value.id} Cue instruction is empty`);
const fields = new Set<string>();
for (const field of value.planning.fields) {
assert(ID.test(field.id) && !fields.has(field.id), `Caption Style ${value.id} field id is invalid or repeated`);
@@ -73,6 +80,8 @@ export function assertCaptionStyle(value: CaptionStyleIntent): void {
`Caption Style ${value.id} field minimum is invalid`);
assert(Number.isSafeInteger(field.maximumPerCue) && field.maximumPerCue >= field.minimumPerCue,
`Caption Style ${value.id} field maximum is invalid`);
+ assert(field.maximumPerCue <= cue.maximumWords,
+ `Caption Style ${value.id} field maximum exceeds its Cue word maximum`);
if (field.value.kind === "enum") {
assert(field.value.values.length > 0 && new Set(field.value.values).size === field.value.values.length,
`Caption Style ${value.id} field enum is invalid`);
@@ -83,44 +92,8 @@ export function assertCaptionStyle(value: CaptionStyleIntent): void {
`Caption Style ${value.id} field numeric maximum is invalid`);
}
}
- assert(Number.isSafeInteger(value.presentation.stackingOrder) && value.presentation.stackingOrder >= 0,
- `Caption Style ${value.id} stacking order is invalid`);
- assert(["whole", "proportional-word", "character-flow"].includes(value.presentation.mode),
- `Caption Style ${value.id} presentation mode is invalid`);
- const appearance = value.presentation.style;
- assert(appearance.fontFamily.trim().length > 0, `Caption Style ${value.id} font family is empty`);
- assert(appearance.textAlign === "left" || appearance.textAlign === "center" || appearance.textAlign === "right",
- `Caption Style ${value.id} text alignment is invalid`);
- const numeric = [
- appearance.fontSizePx,
- appearance.fontWeight,
- appearance.paddingXPx,
- appearance.paddingYPx,
- appearance.borderRadiusPx,
- appearance.bottomPercent,
- appearance.maxWidthPercent,
- appearance.leftPercent,
- appearance.topPercent,
- appearance.widthPercent,
- appearance.lineHeight,
- ].filter((item): item is number => item !== undefined);
- assert(numeric.every((item) => Number.isFinite(item) && item >= 0),
- `Caption Style ${value.id} appearance contains an invalid number`);
- assert(appearance.fontSizePx > 0 && appearance.maxWidthPercent > 0 && appearance.maxWidthPercent <= 100
- && appearance.bottomPercent <= 100,
- `Caption Style ${value.id} appearance bounds are invalid`);
- for (const color of [appearance.color, appearance.backgroundColor].filter((item): item is string => item !== undefined)) {
- assert(/^#[0-9a-f]{6}(?:[0-9a-f]{2})?$/iu.test(color), `Caption Style ${value.id} color ${color} is invalid`);
- }
-}
-
-function matches(
- atom: CaptionDisplayAtom,
- application: CaptionStyleApplication,
- narrative: Narrative,
-): boolean {
- if (application.selector.kind === "role") return atom.role === application.selector.role;
- return displayAtomMatchesSelection(atom, application.selector.selection, narrative.tokens.length);
+ assert(value.rendering.family.length > 0, `Caption Style ${value.id} rendering family is empty`);
+ canonicalize(value.rendering.parameters);
}
function programContent(value: CaptionProgram) {
@@ -135,55 +108,52 @@ export function sealCaptionProgram(value: CaptionProgram): CaptionProgram {
export function assertCaptionProgram(value: CaptionProgram): void {
assert(value.contract === "svml.caption-program@1" && ID.test(value.id), "Caption Program identity is invalid");
- assert(value.atoms.length > 0 && value.runs.length > 0 && value.styles.length > 0, "Caption Program is empty");
+ assert(value.wordSequenceId.length > 0 && value.runs.length > 0 && value.styles.length > 0,
+ "Caption Program is empty");
const styles = new Map(value.styles.map((style) => [style.id, style]));
assert(styles.size === value.styles.length && styles.has(value.defaultStyleId), "Caption Program styles are invalid");
value.styles.forEach(assertCaptionStyle);
- const atomIds = value.atoms.map((atom) => atom.id);
- assert(new Set(atomIds).size === atomIds.length, "Caption Program display atom ids are repeated");
- value.atoms.forEach((atom, index) => {
- assert(atom.id.length > 0 && atom.index === index && atom.regionId.length > 0 && atom.segmentId.length > 0
- && atom.turnId.length > 0 && atom.text.length > 0, "Caption Program display atom identity is invalid");
- assert(Number.isSafeInteger(atom.displayStart) && Number.isSafeInteger(atom.displayEnd)
- && atom.displayStart >= 0 && atom.displayEnd > atom.displayStart,
- `Caption Program display atom ${atom.id} display range is invalid`);
- assert(Number.isSafeInteger(atom.sourceTokenStart) && Number.isSafeInteger(atom.sourceTokenEndExclusive)
- && atom.sourceTokenStart >= 0 && atom.sourceTokenEndExclusive > atom.sourceTokenStart,
- `Caption Program display atom ${atom.id} source range is invalid`);
- assert(atom.correspondence === "exact" || atom.correspondence === "region-envelope",
- `Caption Program display atom ${atom.id} correspondence is invalid`);
- });
+ const families = new Set(value.styles.map((style) => style.rendering.family));
+ assert(families.size === 1, "One Caption Program must use one Style rendering family");
assert(new Set(value.runs.map((run) => run.id)).size === value.runs.length,
"Caption Program run ids are repeated");
- const planned = value.runs.flatMap((run) => run.atomIds);
- assert(planned.join("\0") === atomIds.join("\0"),
- "Caption Program runs must partition every display atom exactly once and in order");
- assert(value.runs.every((run) => styles.has(run.styleId) && run.atomIds.length > 0), "Caption Program run style is invalid");
+ const planned = value.runs.flatMap((run) => run.wordIds);
+ assert(new Set(planned).size === planned.length, "Caption Program runs repeat a display word");
+ assert(value.runs.every((run) => styles.has(run.styleId) && run.wordIds.length > 0),
+ "Caption Program run style is invalid");
+ for (const run of value.runs) {
+ const style = styles.get(run.styleId)!;
+ const fewestCues = Math.ceil(run.wordIds.length / style.planning.cue.maximumWords);
+ const mostCues = Math.floor(run.wordIds.length / style.planning.cue.minimumWords);
+ assert(fewestCues <= mostCues,
+ `Caption Program run ${run.id} cannot satisfy Style ${style.id} Cue word bounds`);
+ }
}
-/** Bind a Program's immutable display universe to the exact Narrative consumed by a Producer. */
-export function assertCaptionProgramForNarrative(value: CaptionProgram, narrative: Narrative): void {
+/** Bind a Program's immutable display universe to the exact Script-produced sequence. */
+export function assertCaptionProgramForWords(value: CaptionProgram, sequence: CaptionWordSequence): void {
assertCaptionProgram(value);
- assert(digestOf(value.atoms) === digestOf(captionDisplayAtoms(narrative)),
- "Caption Program display atoms differ from its Narrative");
+ assertCaptionWordSequence(sequence);
+ assert(value.wordSequenceId === sequence.id
+ && value.runs.flatMap((run) => run.wordIds).join("\0") === sequence.words.map((word) => word.id).join("\0"),
+ "Caption Program does not partition its CaptionWordSequence exactly once and in order");
}
-/** Resolve a base style plus ordered, whole-style replacements. Later matching applications win. */
+/** Resolve an explicit default Style plus ordered whole-Style replacements. Later applications win. */
export function resolveCaptionProgram(
- narrative: Narrative,
+ sequence: CaptionWordSequence,
id: string,
defaultStyle: CaptionStyleIntent,
applications: readonly CaptionStyleApplication[],
): CaptionProgram {
+ assertCaptionWordSequence(sequence);
assertCaptionStyle(defaultStyle);
applications.forEach((application) => {
assert(ID.test(application.id), "Caption Style Application identity is invalid");
assertCaptionStyle(application.style);
- if (application.selector.kind === "role") {
- assert(application.selector.role.trim().length > 0, `Caption application ${application.id} Role is empty`);
- }
+ assertCaptionWordSubset(application.words, sequence);
+ assert(application.words.wordIds.length > 0, `Caption application ${application.id} selects no visible display word`);
});
- const atoms = captionDisplayAtoms(narrative);
const styles = new Map<string, CaptionStyleIntent>([[defaultStyle.id, defaultStyle]]);
applications.forEach((application) => {
const previous = styles.get(application.style.id);
@@ -191,39 +161,40 @@ export function resolveCaptionProgram(
`Caption Style ${application.style.id} has conflicting definitions`);
styles.set(application.style.id, application.style);
});
- for (const application of applications) {
- assert(atoms.some((atom) => matches(atom, application, narrative)),
- `Caption application ${application.id} selects no visible display atom`);
- }
- const assignments = atoms.map((atom) => {
+ const selected = applications.map((application) => new Set(application.words.wordIds));
+ const assignments = sequence.words.map((word) => {
let styleId = defaultStyle.id;
- for (const application of applications) {
- if (matches(atom, application, narrative)) styleId = application.style.id;
- }
- return { atom, styleId };
+ applications.forEach((application, index) => {
+ if (selected[index]!.has(word.id)) styleId = application.style.id;
+ });
+ return { word, styleId };
});
- const runs: Array<{ id: string; styleId: string; atomIds: string[]; turnId: string; segmentId: string }> = [];
+ const runs: Array<{ id: string; styleId: string; wordIds: string[]; turnId: string; segmentId: string }> = [];
for (const assignment of assignments) {
const current = runs.at(-1);
if (current === undefined || current.styleId !== assignment.styleId
- || current.turnId !== assignment.atom.turnId || current.segmentId !== assignment.atom.segmentId) {
+ || current.turnId !== assignment.word.turnId || current.segmentId !== assignment.word.segmentId) {
runs.push({
id: `${id}:run:${runs.length + 1}`,
styleId: assignment.styleId,
- atomIds: [assignment.atom.id],
- turnId: assignment.atom.turnId,
- segmentId: assignment.atom.segmentId,
+ wordIds: [assignment.word.id],
+ turnId: assignment.word.turnId,
+ segmentId: assignment.word.segmentId,
});
} else {
- current.atomIds.push(assignment.atom.id);
+ current.wordIds.push(assignment.word.id);
}
}
- return sealCaptionProgram({
+ const program = sealCaptionProgram({
contract: "svml.caption-program@1",
id,
+ wordSequenceId: sequence.id,
defaultStyleId: defaultStyle.id,
styles: [...styles.values()],
- atoms,
runs: runs.map(({ turnId: _turn, segmentId: _segment, ...run }) => run),
});
+ assertCaptionProgramForWords(program, sequence);
+ return program;
}
+
+export type { CaptionWord };
diff --git a/packages/caption/src/surface.ts b/packages/caption/src/surface.ts
index 6c65d3c3..fa63d4e7 100644
--- a/packages/caption/src/surface.ts
+++ b/packages/caption/src/surface.ts
@@ -1,9 +1,5 @@
import { narrativeTypes } from "@narratage/narrative";
-import type { Narrative, NarrativeSelectionRef } from "@narratage/narrative";
-import { programSpaceTypes } from "@narratage/program-space";
-import { semanticMapTypes } from "@narratage/semantic-map";
-import { svsRecipeType } from "@narratage/svs";
-import type { SvsRecipe } from "@narratage/svs";
+import type { CaptionWordSequence, CaptionWordSubset } from "@narratage/narrative";
import type {
StructuredElement,
StructuredSurfaceHandler,
@@ -11,17 +7,11 @@ import type {
TextAttributeValue,
} from "@narratage/text";
-import { captionTrackSurfaceFragment, plannedCaptionTrackSurfaceFragment } from "./fragment.js";
import { captionTypes } from "./manifest.js";
-import { sealCaptionTrackProgram } from "./track.js";
-import { resolveCaptionProgram, sealCaptionStyle } from "./style.js";
+import { captionWordsForRole } from "./display.js";
+import { resolveCaptionProgram } from "./style.js";
import type { CaptionStyleApplication } from "./style.js";
-import type {
- CaptionFieldDeclaration,
- CaptionFieldValueSchema,
- CaptionPresentationMode,
- CaptionStyleIntent,
-} from "./types.js";
+import type { CaptionStyleIntent } from "./types.js";
function sameType(left: SurfaceResolvedReference["type"], right: SurfaceResolvedReference["type"]): boolean {
return left.module.name === right.module.name && left.module.version === right.module.version && left.name === right.name;
@@ -31,7 +21,9 @@ function attributes(element: StructuredElement, required: readonly string[], opt
const allowed = new Set([...required, ...optional]);
const actual = Object.keys(element.attributes);
if (required.some((name) => element.attributes[name] === undefined) || actual.some((name) => !allowed.has(name))) {
- throw new Error(`${element.name} requires ${required.join(", ")}${optional.length ? `; optional: ${optional.join(", ")}` : ""}`);
+ throw new Error(
+ `${element.name} requires ${required.join(", ")}${optional.length ? `; optional: ${optional.join(", ")}` : ""}`,
+ );
}
}
@@ -41,6 +33,13 @@ function stringAttribute(element: StructuredElement, name: string): string {
return value.trim();
}
+function optionalString(element: StructuredElement, name: string): string | undefined {
+ const value = element.attributes[name];
+ if (value === undefined) return undefined;
+ if (typeof value !== "string" || !value.trim()) throw new Error(`${element.name}.${name} must be a non-empty string`);
+ return value.trim();
+}
+
function reference(
element: StructuredElement,
name: string,
@@ -48,55 +47,15 @@ function reference(
resolveReference: (path: string) => SurfaceResolvedReference | undefined,
): SurfaceResolvedReference {
const raw: TextAttributeValue | undefined = element.attributes[name];
- if (typeof raw !== "object" || raw.kind !== "reference") throw new Error(`${element.name}.${name} must be a whole-value reference`);
+ if (typeof raw !== "object" || raw.kind !== "reference") {
+ throw new Error(`${element.name}.${name} must be a whole-value reference`);
+ }
const value = resolveReference(raw.path);
if (value === undefined) throw new Error(`${element.name}.${name} cannot resolve ${raw.path}`);
if (!sameType(value.type, expected)) throw new Error(`${element.name}.${name} has the wrong type`);
return value;
}
-function recipe(reference: SurfaceResolvedReference): SvsRecipe {
- if (!sameType(reference.type, svsRecipeType) || reference.record?.value.kind !== "inline") {
- throw new Error("Caption appearance must reference an authored SVS Recipe");
- }
- return reference.record.value.value as unknown as SvsRecipe;
-}
-
-function number(value: SvsRecipe, name: string): number {
- const property = value.properties[name];
- if (typeof property !== "number" || !Number.isFinite(property)) throw new Error(`Caption Recipe ${name} must be a number`);
- return property;
-}
-
-function integer(value: SvsRecipe, name: string): number {
- const property = number(value, name);
- if (!Number.isSafeInteger(property)) throw new Error(`Caption Recipe ${name} must be an integer`);
- return property;
-}
-
-function string(value: SvsRecipe, name: string): string {
- const property = value.properties[name];
- if (typeof property !== "string" || !property.trim()) throw new Error(`Caption Recipe ${name} must be a string`);
- return property.trim();
-}
-
-function padding(value: string): { readonly x: number; readonly y: number } {
- const parts = value.trim().split(/\s+/u).map(Number);
- if ((parts.length !== 1 && parts.length !== 2) || parts.some((item) => !Number.isFinite(item) || item < 0)) {
- throw new Error("Caption Recipe padding must contain one or two non-negative pixel numbers");
- }
- return { y: parts[0]!, x: parts[1] ?? parts[0]! };
-}
-
-function mode(element: StructuredElement): CaptionPresentationMode {
- const value = element.attributes.mode;
- if (value === undefined) return "whole";
- if (typeof value === "string" && ["whole", "proportional-word", "character-flow"].includes(value)) {
- return value as CaptionPresentationMode;
- }
- throw new Error(`${element.name}.mode is invalid`);
-}
-
function inline<T>(reference: SurfaceResolvedReference, subject: string): T {
if (reference.record?.value.kind !== "inline") throw new Error(`${subject} must reference an authored inline Record`);
return reference.record.value.value as unknown as T;
@@ -106,125 +65,21 @@ function localName(name: string): string {
return name.includes(":") ? name.slice(name.lastIndexOf(":") + 1) : name;
}
-function body(element: StructuredElement): string {
- if (element.children.some((child) => child.kind === "element")) throw new Error(`${element.name} accepts only natural-language text`);
- const value = element.children.map((child) => child.kind === "text" ? child.value : "").join("").trim();
- if (!value) throw new Error(`${element.name} instruction is empty`);
- return value.replace(/\s+/gu, " ");
-}
-
-function optionalString(element: StructuredElement, name: string): string | undefined {
- const value = element.attributes[name];
- if (value === undefined) return undefined;
- if (typeof value !== "string" || !value.trim()) throw new Error(`${element.name}.${name} must be a non-empty string`);
- return value.trim();
-}
-
-function nonNegativeInteger(element: StructuredElement, name: string): number {
- const value = Number(stringAttribute(element, name));
- if (!Number.isSafeInteger(value) || value < 0) throw new Error(`${element.name}.${name} must be a non-negative integer`);
- return value;
-}
-
-function fieldValue(element: StructuredElement): CaptionFieldValueSchema {
- const type = stringAttribute(element, "type");
- if (type === "boolean") return { kind: "boolean" };
- if (type === "enum") {
- const values = stringAttribute(element, "values").split(",").map((item) => item.trim()).filter(Boolean);
- if (!values.length) throw new Error(`${element.name}.values is empty`);
- return { kind: "enum", values };
- }
- if (type === "number") {
- const minimum = optionalString(element, "minimum");
- const maximum = optionalString(element, "maximum");
- const min = minimum === undefined ? undefined : Number(minimum);
- const max = maximum === undefined ? undefined : Number(maximum);
- if ((min !== undefined && !Number.isFinite(min)) || (max !== undefined && !Number.isFinite(max))) {
- throw new Error(`${element.name} number bounds must be finite`);
- }
- return { kind: "number", ...(min === undefined ? {} : { minimum: min }), ...(max === undefined ? {} : { maximum: max }) };
- }
- throw new Error(`${element.name}.type must be boolean, enum or number`);
-}
-
-function captionAppearance(element: StructuredElement, value: SvsRecipe) {
- const expected = ["align", "background", "fill", "font", "line-height", "padding", "radius", "size", "stack-order", "weight", "width", "x", "y"];
- if (Object.keys(value.properties).sort().join("\u0000") !== expected.sort().join("\u0000")) {
- throw new Error(`Caption Recipe requires exactly ${expected.join(", ")}`);
- }
- const align = string(value, "align");
- if (align !== "left" && align !== "center" && align !== "right") throw new Error("Caption Recipe align is invalid");
- const pad = padding(string(value, "padding"));
- return {
- mode: mode(element),
- stackingOrder: integer(value, "stack-order"),
- style: {
- fontFamily: string(value, "font"),
- fontSizePx: number(value, "size"),
- fontWeight: integer(value, "weight"),
- color: string(value, "fill"),
- backgroundColor: string(value, "background"),
- paddingXPx: pad.x,
- paddingYPx: pad.y,
- borderRadiusPx: number(value, "radius"),
- bottomPercent: 0,
- maxWidthPercent: number(value, "width") * 100,
- leftPercent: number(value, "x") * 100,
- topPercent: number(value, "y") * 100,
- widthPercent: number(value, "width") * 100,
- lineHeight: number(value, "line-height"),
- textAlign: align,
- },
- } as const;
-}
-
-export const decodeCaptionStyleSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
- attributes(element, ["id", "appearance"], ["mode"]);
- const id = stringAttribute(element, "id");
- const appearance = recipe(reference(element, "appearance", svsRecipeType, resolveReference));
- let cueInstruction: string | undefined;
- const fields: CaptionFieldDeclaration[] = [];
- for (const child of element.children) {
- if (child.kind === "text") {
- if (child.value.trim()) throw new Error(`${element.name} accepts only Cues and Field children`);
- continue;
- }
- if (localName(child.name) === "Cues") {
- attributes(child, []);
- if (cueInstruction !== undefined) throw new Error(`${element.name} repeats Cues`);
- cueInstruction = body(child);
- } else if (localName(child.name) === "Field") {
- attributes(child, ["id", "type", "min-per-cue", "max-per-cue"], ["values", "minimum", "maximum"]);
- fields.push({
- id: stringAttribute(child, "id"),
- value: fieldValue(child),
- instruction: body(child),
- minimumPerCue: nonNegativeInteger(child, "min-per-cue"),
- maximumPerCue: nonNegativeInteger(child, "max-per-cue"),
- });
- } else {
- throw new Error(`${element.name} accepts only Cues and Field children`);
- }
- }
- if (cueInstruction === undefined) throw new Error(`${element.name} requires one Cues instruction`);
- const style = sealCaptionStyle({
- contract: "svml.caption-style@1",
- id,
- planning: { cueInstruction, fields },
- presentation: captionAppearance(element, appearance),
- });
- return {
- records: [{ id, type: captionTypes.style, value: { kind: "inline", value: style }, range: element.range }],
- components: [],
- fragments: [],
- };
-};
-
+/**
+ * Resolve one complete word assignment at author-compilation time. Script has already projected
+ * Selection syntax to CaptionWordSubset data; this Surface lowers Role sugar from the same sequence.
+ */
export const decodeCaptionProgramSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
- attributes(element, ["id", "narrative", "default"]);
+ attributes(element, ["id", "words", "default"]);
const id = stringAttribute(element, "id");
- const narrativeRef = reference(element, "narrative", narrativeTypes.narrative, resolveReference);
- const defaultRef = reference(element, "default", captionTypes.style, resolveReference);
+ const sequence = inline<CaptionWordSequence>(
+ reference(element, "words", narrativeTypes.captionWordSequence, resolveReference),
+ `${element.name}.words`,
+ );
+ const defaultStyle = inline<CaptionStyleIntent>(
+ reference(element, "default", captionTypes.style, resolveReference),
+ `${element.name}.default`,
+ );
const applications: CaptionStyleApplication[] = [];
for (const child of element.children) {
if (child.kind === "text") {
@@ -232,95 +87,31 @@ export const decodeCaptionProgramSurface: StructuredSurfaceHandler = ({ element,
continue;
}
if (localName(child.name) !== "Use") throw new Error(`${element.name} accepts only Use children`);
- attributes(child, ["style"], ["role", "on"]);
+ attributes(child, ["style"], ["role", "words"]);
const role = optionalString(child, "role");
- const on = child.attributes.on;
- if ((role === undefined) === (on === undefined)) throw new Error(`${child.name} requires exactly one of role or on`);
- const styleRef = reference(child, "style", captionTypes.style, resolveReference);
- const selector = role !== undefined
- ? { kind: "role" as const, role }
- : {
- kind: "selection" as const,
- selection: inline<NarrativeSelectionRef>(
- reference(child, "on", narrativeTypes.selection, resolveReference),
- `${child.name}.on`,
- ),
- };
+ const wordsAttribute = child.attributes.words;
+ if ((role === undefined) === (wordsAttribute === undefined)) {
+ throw new Error(`${child.name} requires exactly one of role or words`);
+ }
+ const words = role === undefined
+ ? inline<CaptionWordSubset>(
+ reference(child, "words", narrativeTypes.captionWordSubset, resolveReference),
+ `${child.name}.words`,
+ )
+ : captionWordsForRole(sequence, role);
applications.push({
id: `${id}.use.${applications.length + 1}`,
- selector,
- style: inline<CaptionStyleIntent>(styleRef, `${child.name}.style`),
+ words,
+ style: inline<CaptionStyleIntent>(
+ reference(child, "style", captionTypes.style, resolveReference),
+ `${child.name}.style`,
+ ),
});
}
- const program = resolveCaptionProgram(
- inline<Narrative>(narrativeRef, `${element.name}.narrative`),
- id,
- inline<CaptionStyleIntent>(defaultRef, `${element.name}.default`),
- applications,
- );
+ const program = resolveCaptionProgram(sequence, id, defaultStyle, applications);
return {
records: [{ id, type: captionTypes.program, value: { kind: "inline", value: program }, range: element.range }],
components: [],
fragments: [],
};
};
-
-export const decodeCaptionTrackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
- if (element.attributes.program !== undefined) {
- attributes(element, ["id", "narrative", "map", "program", "plan", "space"]);
- if (element.children.some((child) => child.kind === "element" || child.value.trim())) {
- throw new Error(`${element.name} with a Caption Program does not accept children`);
- }
- const id = stringAttribute(element, "id");
- const narrative = reference(element, "narrative", narrativeTypes.narrative, resolveReference);
- const map = reference(element, "map", semanticMapTypes.complete, resolveReference);
- const program = reference(element, "program", captionTypes.program, resolveReference);
- const plan = reference(element, "plan", captionTypes.plan, resolveReference);
- const space = reference(element, "space", programSpaceTypes.programSpace, resolveReference);
- return {
- records: [],
- components: [{
- id,
- fragment: plannedCaptionTrackSurfaceFragment.id,
- inputs: { narrative: narrative.ref, map: map.ref, program: program.ref, plan: plan.ref, space: space.ref },
- outputs: { track: `${id}.track` },
- range: element.range,
- }],
- fragments: [plannedCaptionTrackSurfaceFragment],
- };
- }
- attributes(element, ["id", "narrative", "map", "space", "appearance"], ["mode"]);
- if (element.children.some((child) => child.kind === "element" || child.value.trim())) {
- throw new Error(`${element.name} provider-free base does not accept Role children`);
- }
- const id = stringAttribute(element, "id");
- const narrative = reference(element, "narrative", narrativeTypes.narrative, resolveReference);
- const map = reference(element, "map", semanticMapTypes.complete, resolveReference);
- const space = reference(element, "space", programSpaceTypes.programSpace, resolveReference);
- const appearance = recipe(reference(element, "appearance", svsRecipeType, resolveReference));
- const resolvedAppearance = captionAppearance(element, appearance);
- const program = sealCaptionTrackProgram({
- contract: "svml.caption-track-program@1",
- id,
- mode: resolvedAppearance.mode,
- stacking: { order: resolvedAppearance.stackingOrder, tieBreak: id },
- style: resolvedAppearance.style,
- });
- const programId = `${id}.program`;
- return {
- records: [{ id: programId, type: captionTypes.trackProgram, value: { kind: "inline", value: program }, range: element.range }],
- components: [{
- id,
- fragment: captionTrackSurfaceFragment.id,
- inputs: {
- narrative: narrative.ref,
- map: map.ref,
- space: space.ref,
- program: { kind: "record", id: programId },
- },
- outputs: { track: `${id}.track` },
- range: element.range,
- }],
- fragments: [captionTrackSurfaceFragment],
- };
-};
diff --git a/packages/caption/src/temporalize.ts b/packages/caption/src/temporalize.ts
index a1195e4d..8661b25d 100644
--- a/packages/caption/src/temporalize.ts
+++ b/packages/caption/src/temporalize.ts
@@ -1,4 +1,4 @@
-import type { Narrative } from "@narratage/narrative";
+import type { CaptionWord, CaptionWordSequence, Narrative } from "@narratage/narrative";
import { tokenSpanSeconds } from "@narratage/semantic-map";
import type { CompleteSemanticMap } from "@narratage/semantic-map";
@@ -6,14 +6,13 @@ import { CaptionProjectionError } from "./error.js";
import type {
CaptionPlan,
CaptionProgram,
- CaptionDisplayAtom,
TimedCaptionProjection,
TimedCaptionRefinement,
TimedCaptionRegion,
} from "./types.js";
import { assertCaptionPlanForProgram } from "./plan.js";
-import { displayTextForAtoms } from "./display.js";
-import { assertCaptionProgramForNarrative } from "./style.js";
+import { displayTextForWords } from "./display.js";
+import { assertCaptionProgramForWords } from "./style.js";
export function temporalizeCaption(
narrative: Narrative,
@@ -53,62 +52,75 @@ export function temporalizeCaption(
refinements,
};
});
- const payload = {
- contract: "svml.timed-caption-projection@1" as const,
+ return {
+ contract: "svml.timed-caption-projection@1",
text: narrative.captionProjection.text,
regions,
};
- return payload;
}
function cleanDisplay(value: string): string {
return value.replace(/\s+/gu, " ").replace(/\s+([,.;:!?])/gu, "$1").trim();
}
+export function assertTimedCaptionProjection(projection: TimedCaptionProjection): void {
+ if (projection.contract !== "svml.timed-caption-projection@1") {
+ throw new Error("Unsupported TimedCaptionProjection contract.");
+ }
+}
+
/** Join planner-neutral Cue/field facts with the one authoritative Script map. */
export function temporalizeCaptionPlan(
narrative: Narrative,
map: CompleteSemanticMap,
+ sequence: CaptionWordSequence,
program: CaptionProgram,
plan: CaptionPlan,
): TimedCaptionProjection {
- assertCaptionProgramForNarrative(program, narrative);
+ assertCaptionProgramForWords(program, sequence);
assertCaptionPlanForProgram(plan, program);
- const atomById = new Map(program.atoms.map((atom) => [atom.id, atom]));
+ const wordById = new Map(sequence.words.map((word) => [word.id, word]));
const regionById = new Map(narrative.captionProjection.regions.map((region) => [region.id, region]));
- const atomsByRegion = new Map<string, CaptionDisplayAtom[]>();
- for (const atom of program.atoms) {
- const values = atomsByRegion.get(atom.regionId) ?? [];
- values.push(atom);
- atomsByRegion.set(atom.regionId, values);
+ const wordsByRegion = new Map<string, CaptionWord[]>();
+ for (const word of sequence.words) {
+ const values = wordsByRegion.get(word.regionId) ?? [];
+ values.push(word);
+ wordsByRegion.set(word.regionId, values);
}
- const atomWindow = (atom: CaptionDisplayAtom) => {
+ const wordWindow = (word: CaptionWord) => {
const sourceIds = narrative.tokens
- .slice(atom.sourceTokenStart, atom.sourceTokenEndExclusive)
+ .slice(word.sourceTokenStart, word.sourceTokenEndExclusive)
.map((token) => token.id);
const window = tokenSpanSeconds(map, sourceIds);
if (window === undefined) {
- throw new CaptionProjectionError("CAPTION_SPEECH_COVERAGE", `Caption display atom ${atom.id} is absent from the complete speech map.`);
+ throw new CaptionProjectionError(
+ "CAPTION_SPEECH_COVERAGE",
+ `Caption display word ${word.id} is absent from the complete speech map.`,
+ );
}
- if (atom.correspondence === "exact") return window;
- const region = regionById.get(atom.regionId);
+ if (word.correspondence === "exact") return window;
+ const region = regionById.get(word.regionId);
if (region === undefined || region.display.length === 0) {
- throw new CaptionProjectionError("CAPTION_DISPLAY_REGION", `Caption display atom ${atom.id} has no owning display region.`);
+ throw new CaptionProjectionError(
+ "CAPTION_DISPLAY_REGION",
+ `Caption display word ${word.id} has no owning display region.`,
+ );
}
- const siblings = atomsByRegion.get(atom.regionId) ?? [];
- const next = siblings.find((candidate) => candidate.displayStart > atom.displayStart);
+ const siblings = wordsByRegion.get(word.regionId) ?? [];
+ const next = siblings.find((candidate) => candidate.displayStart > word.displayStart);
const duration = window.endSec - window.startSec;
return {
- startSec: window.startSec + duration * atom.displayStart / region.display.length,
+ startSec: window.startSec + duration * word.displayStart / region.display.length,
endSec: window.startSec + duration * (next?.displayStart ?? region.display.length) / region.display.length,
};
};
- const visible = new Set(program.atoms.map((atom) => atom.id));
- const plannedIds = plan.runs.flatMap((run) => run.cues.flatMap((cue) => cue.atomIds));
- if (plannedIds.length !== visible.size || new Set(plannedIds).size !== visible.size || plannedIds.some((id) => !visible.has(id))) {
+ const visible = new Set(sequence.words.map((word) => word.id));
+ const plannedIds = plan.runs.flatMap((run) => run.cues.flatMap((cue) => cue.wordIds));
+ if (plannedIds.length !== visible.size || new Set(plannedIds).size !== visible.size
+ || plannedIds.some((id) => !visible.has(id))) {
throw new CaptionProjectionError(
"CAPTION_PLAN_COVERAGE",
- "CaptionPlan Cues must be disjoint and cover every display atom exactly once.",
+ "CaptionPlan Cues must be disjoint and cover every display word exactly once.",
);
}
const programRuns = new Map(program.runs.map((run) => [run.id, run]));
@@ -118,57 +130,63 @@ export function temporalizeCaptionPlan(
if (expectedRun === undefined || expectedRun.styleId !== run.styleId) {
throw new CaptionProjectionError("CAPTION_PLAN_RUN", `Caption run ${run.id} differs from its Program.`);
}
- const runAtoms = run.cues.flatMap((cue) => cue.atomIds);
- if (runAtoms.join("\0") !== expectedRun.atomIds.join("\0")) {
- throw new CaptionProjectionError("CAPTION_PLAN_RUN", `Caption run ${run.id} changes its Program atom order.`);
+ const runWords = run.cues.flatMap((cue) => cue.wordIds);
+ if (runWords.join("\0") !== expectedRun.wordIds.join("\0")) {
+ throw new CaptionProjectionError("CAPTION_PLAN_RUN", `Caption run ${run.id} changes its Program word order.`);
}
for (const cue of run.cues) {
- const source = cue.atomIds.map((id) => atomById.get(id));
- if (source.some((atom) => atom === undefined)) {
- throw new CaptionProjectionError("CAPTION_PLAN_ATOM", `Caption Cue ${cue.id} references an unknown display atom.`);
+ const source = cue.wordIds.map((id) => wordById.get(id));
+ if (source.some((word) => word === undefined)) {
+ throw new CaptionProjectionError("CAPTION_PLAN_WORD", `Caption Cue ${cue.id} references an unknown display word.`);
}
- const atoms = source as CaptionDisplayAtom[];
- if (atoms.some((atom, index) => index > 0 && atom.index <= atoms[index - 1]!.index)) {
- throw new CaptionProjectionError("CAPTION_PLAN_ORDER", `Caption Cue ${cue.id} changes display atom order.`);
+ const words = source as CaptionWord[];
+ if (words.some((word, index) => index > 0 && word.index <= words[index - 1]!.index)) {
+ throw new CaptionProjectionError("CAPTION_PLAN_ORDER", `Caption Cue ${cue.id} changes display word order.`);
}
- if (new Set(atoms.map((atom) => atom.segmentId)).size !== 1) {
+ if (new Set(words.map((word) => word.segmentId)).size !== 1) {
throw new CaptionProjectionError("CAPTION_PLAN_SEGMENT", `Caption Cue ${cue.id} crosses a Script Segment.`);
}
- const sourceIndexes = [...new Set(atoms.flatMap((atom) => Array.from(
- { length: atom.sourceTokenEndExclusive - atom.sourceTokenStart },
- (_, offset) => atom.sourceTokenStart + offset,
+ const sourceIndexes = [...new Set(words.flatMap((word) => Array.from(
+ { length: word.sourceTokenEndExclusive - word.sourceTokenStart },
+ (_, offset) => word.sourceTokenStart + offset,
)))].sort((left, right) => left - right);
const sourceTokens = sourceIndexes.map((index) => narrative.tokens[index]);
if (sourceTokens.some((token) => token === undefined)
|| tokenSpanSeconds(map, sourceTokens.map((token) => token!.id)) === undefined) {
- throw new CaptionProjectionError("CAPTION_SPEECH_COVERAGE", `Caption Cue ${cue.id} is absent from the complete speech map.`);
+ throw new CaptionProjectionError(
+ "CAPTION_SPEECH_COVERAGE",
+ `Caption Cue ${cue.id} is absent from the complete speech map.`,
+ );
}
- const firstWindow = atomWindow(atoms[0]!);
- const lastWindow = atomWindow(atoms.at(-1)!);
+ const firstWindow = wordWindow(words[0]!);
+ const lastWindow = wordWindow(words.at(-1)!);
for (const field of cue.fields) {
- if (!cue.atomIds.includes(field.atomId)) {
- throw new CaptionProjectionError("CAPTION_PLAN_FIELD", `Caption field ${field.declarationId} lies outside Cue ${cue.id}.`);
+ if (!cue.wordIds.includes(field.wordId)) {
+ throw new CaptionProjectionError(
+ "CAPTION_PLAN_FIELD",
+ `Caption field ${field.declarationId} lies outside Cue ${cue.id}.`,
+ );
}
}
regions.push({
id: cue.id,
runId: run.id,
styleId: run.styleId,
- display: displayTextForAtoms(narrative, atoms),
- segmentId: atoms[0]!.segmentId,
- kind: atoms.some((atom) => atom.correspondence === "region-envelope") ? "alias" : "identity",
+ display: displayTextForWords(narrative, sequence.words, words),
+ segmentId: words[0]!.segmentId,
+ kind: words.some((word) => word.correspondence === "region-envelope") ? "alias" : "identity",
sourceTokenIds: sourceTokens.map((token) => token!.id),
startSec: firstWindow.startSec,
endSec: lastWindow.endSec,
refinements: [],
+ wordIds: [...cue.wordIds],
fields: cue.fields.map((field) => ({ ...field })),
});
}
}
- const payload = {
- contract: "svml.timed-caption-projection@1" as const,
+ return {
+ contract: "svml.timed-caption-projection@1",
text: cleanDisplay(regions.map((region) => region.display).join(" ")),
regions,
};
- return payload;
}
diff --git a/packages/caption/src/track.ts b/packages/caption/src/track.ts
deleted file mode 100644
index f74309ca..00000000
--- a/packages/caption/src/track.ts
+++ /dev/null
@@ -1,254 +0,0 @@
-import { assertProgramSpaceIdentity, programSpaceFrameCount } from "@narratage/program-space";
-import type { ProgramSpace } from "@narratage/program-space";
-import { assertVisualTrackIdentity, sealVisualTrack } from "@narratage/composition";
-import type { Track, VisualStyleDeclaration, VisualTrack } from "@narratage/composition";
-import { digestOf } from "@narratage/protocol";
-
-import { planCaptionPresentation } from "./presentation.js";
-import { assertCaptionProgram } from "./style.js";
-import type { CaptionProgram, CaptionTrackProgram, TimedCaptionProjection } from "./types.js";
-
-export const renderCaptionTrackImplementationDigest = digestOf("@narratage/caption/render-track@1");
-export const renderCaptionProgramImplementationDigest = digestOf("@narratage/caption/render-program@1");
-
-function normalizedProgram(value: CaptionTrackProgram): CaptionTrackProgram {
- return {
- contract: "svml.caption-track-program@1",
- id: value.id,
- mode: value.mode,
- stacking: { ...value.stacking },
- style: { ...value.style },
- };
-}
-
-export function sealCaptionTrackProgram(value: CaptionTrackProgram): CaptionTrackProgram {
- return normalizedProgram(value);
-}
-
-export function defaultCaptionTrackProgram(id = "captions"): CaptionTrackProgram {
- return sealCaptionTrackProgram({
- contract: "svml.caption-track-program@1",
- id,
- mode: "whole",
- stacking: { order: 100, tieBreak: id },
- style: {
- fontFamily: "Inter, sans-serif",
- fontSizePx: 72,
- fontWeight: 700,
- color: "#ffffff",
- backgroundColor: "#000000cc",
- paddingXPx: 24,
- paddingYPx: 12,
- borderRadiusPx: 16,
- bottomPercent: 10,
- maxWidthPercent: 88,
- textAlign: "center",
- },
- });
-}
-
-export function assertCaptionTrackProgram(program: CaptionTrackProgram): void {
- if (program.contract !== "svml.caption-track-program@1") throw new Error("Unsupported CaptionTrackProgram contract.");
- if (!program.id || !program.stacking.tieBreak || !program.style.fontFamily) {
- throw new Error("CaptionTrackProgram identity and font family must not be empty.");
- }
- if (!Number.isSafeInteger(program.stacking.order) || program.stacking.order < 0) {
- throw new Error("CaptionTrackProgram stacking order is invalid.");
- }
- const numeric: Readonly<Record<string, number>> = {
- fontSizePx: program.style.fontSizePx,
- fontWeight: program.style.fontWeight,
- paddingXPx: program.style.paddingXPx,
- paddingYPx: program.style.paddingYPx,
- borderRadiusPx: program.style.borderRadiusPx,
- bottomPercent: program.style.bottomPercent,
- maxWidthPercent: program.style.maxWidthPercent,
- ...(program.style.leftPercent === undefined ? {} : { leftPercent: program.style.leftPercent }),
- ...(program.style.topPercent === undefined ? {} : { topPercent: program.style.topPercent }),
- ...(program.style.widthPercent === undefined ? {} : { widthPercent: program.style.widthPercent }),
- ...(program.style.lineHeight === undefined ? {} : { lineHeight: program.style.lineHeight }),
- };
- for (const [name, value] of Object.entries(numeric)) {
- if (!Number.isFinite(value) || value < 0) throw new Error(`CaptionTrackProgram ${name} is invalid.`);
- }
- if (program.style.fontSizePx <= 0 || program.style.maxWidthPercent <= 0 || program.style.maxWidthPercent > 100) {
- throw new Error("CaptionTrackProgram text size or width is invalid.");
- }
- if (program.style.bottomPercent > 100) throw new Error("CaptionTrackProgram bottomPercent is invalid.");
- for (const color of [program.style.color, program.style.backgroundColor].filter((value): value is string => value !== undefined)) {
- if (!/^#[0-9a-f]{6}(?:[0-9a-f]{2})?$/iu.test(color)) throw new Error(`CaptionTrackProgram color ${color} is invalid.`);
- }
-}
-
-export function assertTimedCaptionProjection(projection: TimedCaptionProjection): void {
- if (projection.contract !== "svml.timed-caption-projection@1") {
- throw new Error("Unsupported TimedCaptionProjection contract.");
- }
-}
-
-function frameAt(programSpace: ProgramSpace, seconds: number): number {
- return Math.round(seconds
- * programSpace.frameRate.numerator
- / programSpace.frameRate.denominator);
-}
-
-function textStyle(style: CaptionTrackProgram["style"]): VisualStyleDeclaration[] {
- return [
- { name: "background", value: style.backgroundColor ?? "transparent" },
- { name: "border-radius", value: `${style.borderRadiusPx}px` },
- { name: "color", value: style.color },
- { name: "font-family", value: style.fontFamily },
- { name: "font-size", value: `${style.fontSizePx}px` },
- { name: "font-weight", value: style.fontWeight },
- ...(style.lineHeight === undefined ? [] : [{ name: "line-height", value: style.lineHeight }]),
- { name: "padding", value: `${style.paddingYPx}px ${style.paddingXPx}px` },
- { name: "text-align", value: style.textAlign },
- { name: "white-space", value: "pre-wrap" },
- ];
-}
-
-export function renderCaptionTrack(
- projection: TimedCaptionProjection,
- program: CaptionTrackProgram,
- programSpace: ProgramSpace,
-): VisualTrack {
- assertTimedCaptionProjection(projection);
- assertCaptionTrackProgram(program);
- assertProgramSpaceIdentity(programSpace);
- const totalFrames = programSpaceFrameCount(programSpace);
- const plan = planCaptionPresentation(projection, program.mode);
- const track = sealVisualTrack({
- contract: "svml.visual-track@1",
- visualIr: "svml.visual-ir@1",
- id: program.id,
- presents: plan.units.flatMap((unit) => {
- const startFrame = Math.max(0, frameAt(programSpace, unit.startSec));
- const measuredEnd = Math.min(totalFrames, frameAt(programSpace, unit.endSec));
- const endFrameExclusive = Math.min(totalFrames, Math.max(startFrame + 1, measuredEnd));
- if (!unit.display || startFrame >= totalFrames || endFrameExclusive <= startFrame) return [];
- return [{
- id: unit.id,
- span: { startFrame, endFrameExclusive },
- stacking: { order: program.stacking.order, tieBreak: `${program.stacking.tieBreak}:${unit.id}` },
- elements: [
- {
- id: "root",
- order: 0,
- kind: "box" as const,
- style: program.style.leftPercent === undefined || program.style.topPercent === undefined || program.style.widthPercent === undefined ? [
- { name: "align-items", value: "center" },
- { name: "bottom", value: `${program.style.bottomPercent}%` },
- { name: "display", value: "flex" },
- { name: "justify-content", value: "center" },
- { name: "left", value: "50%" },
- { name: "max-width", value: `${program.style.maxWidthPercent}%` },
- { name: "position", value: "absolute" },
- { name: "transform", value: "translateX(-50%)" },
- ] : [
- { name: "align-items", value: "center" },
- { name: "display", value: "flex" },
- { name: "justify-content", value: "center" },
- { name: "left", value: `${program.style.leftPercent}%` },
- { name: "position", value: "absolute" },
- { name: "top", value: `${program.style.topPercent}%` },
- { name: "width", value: `${program.style.widthPercent}%` },
- ],
- },
- {
- id: "text",
- parent: "root",
- order: 1,
- kind: "text" as const,
- text: unit.display,
- style: textStyle(program.style),
- attributes: [
- ...(unit.runId === undefined ? [] : [{ name: "data-caption-run", value: unit.runId }]),
- ...(unit.fields === undefined ? [] : [{ name: "data-caption-fields", value: JSON.stringify(unit.fields) }]),
- ],
- },
- ],
- }];
- }),
- });
- assertVisualTrackIdentity(track, programSpace);
- return track;
-}
-
-/** Render one peer Track whose Presents may each select a complete, resolved Caption Style. */
-export function renderCaptionProgram(
- projection: TimedCaptionProjection,
- program: CaptionProgram,
- programSpace: ProgramSpace,
-): VisualTrack {
- assertTimedCaptionProjection(projection);
- assertCaptionProgram(program);
- const styles = new Map(program.styles.map((style) => [style.id, style]));
- assertProgramSpaceIdentity(programSpace);
- const totalFrames = programSpaceFrameCount(programSpace);
- const units = projection.regions.flatMap((region) => {
- const styleId = region.styleId ?? program.defaultStyleId;
- const style = styles.get(styleId);
- if (style === undefined) throw new Error(`Caption region ${region.id} references unknown Style ${styleId}`);
- return planCaptionPresentation(
- { regions: [region] },
- style.presentation.mode,
- ).units.map((unit) => ({ unit, style }));
- });
- const track = sealVisualTrack({
- contract: "svml.visual-track@1",
- visualIr: "svml.visual-ir@1",
- id: program.id,
- presents: units.flatMap(({ unit, style }) => {
- const startFrame = Math.max(0, frameAt(programSpace, unit.startSec));
- const measuredEnd = Math.min(totalFrames, frameAt(programSpace, unit.endSec));
- const endFrameExclusive = Math.min(totalFrames, Math.max(startFrame + 1, measuredEnd));
- if (!unit.display || startFrame >= totalFrames || endFrameExclusive <= startFrame) return [];
- const appearance = style.presentation.style;
- return [{
- id: unit.id,
- span: { startFrame, endFrameExclusive },
- stacking: { order: style.presentation.stackingOrder, tieBreak: `${program.id}:${unit.id}` },
- elements: [
- {
- id: "root",
- order: 0,
- kind: "box" as const,
- style: appearance.leftPercent === undefined || appearance.topPercent === undefined || appearance.widthPercent === undefined ? [
- { name: "align-items", value: "center" },
- { name: "bottom", value: `${appearance.bottomPercent}%` },
- { name: "display", value: "flex" },
- { name: "justify-content", value: "center" },
- { name: "left", value: "50%" },
- { name: "max-width", value: `${appearance.maxWidthPercent}%` },
- { name: "position", value: "absolute" },
- { name: "transform", value: "translateX(-50%)" },
- ] : [
- { name: "align-items", value: "center" },
- { name: "display", value: "flex" },
- { name: "justify-content", value: "center" },
- { name: "left", value: `${appearance.leftPercent}%` },
- { name: "position", value: "absolute" },
- { name: "top", value: `${appearance.topPercent}%` },
- { name: "width", value: `${appearance.widthPercent}%` },
- ],
- },
- {
- id: "text",
- parent: "root",
- order: 1,
- kind: "text" as const,
- text: unit.display,
- style: textStyle(appearance),
- attributes: [
- { name: "data-caption-style", value: style.id },
- ...(unit.runId === undefined ? [] : [{ name: "data-caption-run", value: unit.runId }]),
- ...(unit.fields === undefined ? [] : [{ name: "data-caption-fields", value: JSON.stringify(unit.fields) }]),
- ],
- },
- ],
- }];
- }),
- });
- assertVisualTrackIdentity(track, programSpace);
- return track;
-}
diff --git a/packages/caption/src/types.ts b/packages/caption/src/types.ts
index 6d5852d9..b5eedfee 100644
--- a/packages/caption/src/types.ts
+++ b/packages/caption/src/types.ts
@@ -1,4 +1,6 @@
+import type { CanonicalValue } from "@narratage/protocol";
+
export type CaptionFieldValueSchema =
| { readonly kind: "boolean" }
| { readonly kind: "enum"; readonly values: readonly string[] }
@@ -13,34 +15,23 @@ export type CaptionFieldDeclaration = {
};
export type CaptionPlanningRequirements = {
- readonly cueInstruction: string;
+ readonly cue: {
+ readonly minimumWords: number;
+ readonly maximumWords: number;
+ readonly instruction: string;
+ };
readonly fields: readonly CaptionFieldDeclaration[];
};
-export type CaptionDisplayAtom = {
- readonly id: string;
- readonly index: number;
- readonly regionId: string;
- readonly segmentId: string;
- readonly turnId: string;
- readonly role?: string;
- readonly text: string;
- readonly displayStart: number;
- readonly displayEnd: number;
- readonly sourceTokenStart: number;
- readonly sourceTokenEndExclusive: number;
- readonly correspondence: "exact" | "region-envelope";
-};
-
export type CaptionFieldAssignment = {
readonly declarationId: string;
- readonly atomId: string;
+ readonly wordId: string;
readonly value: string;
};
export type CaptionPlannedCue = {
readonly id: string;
- readonly atomIds: readonly string[];
+ readonly wordIds: readonly string[];
readonly fields: readonly CaptionFieldAssignment[];
};
@@ -60,26 +51,27 @@ export type CaptionStyleIntent = {
readonly contract: "svml.caption-style@1";
readonly id: string;
readonly planning: CaptionPlanningRequirements;
- readonly presentation: {
- readonly mode: CaptionPresentationMode;
- readonly stackingOrder: number;
- readonly style: CaptionTrackProgram["style"];
+ readonly rendering: {
+ /** Package-owned Style family. It must understand every declared planning field. */
+ readonly family: string;
+ /** Complete Recipe-resolved parameters, opaque to the common Caption package. */
+ readonly parameters: CanonicalValue;
};
};
export type CaptionProgramRun = {
readonly id: string;
readonly styleId: string;
- readonly atomIds: readonly string[];
+ readonly wordIds: readonly string[];
};
/** Total, ordered style assignment over one Caption display universe. */
export type CaptionProgram = {
readonly contract: "svml.caption-program@1";
readonly id: string;
+ readonly wordSequenceId: string;
readonly defaultStyleId: string;
readonly styles: readonly CaptionStyleIntent[];
- readonly atoms: readonly CaptionDisplayAtom[];
readonly runs: readonly CaptionProgramRun[];
};
@@ -105,6 +97,7 @@ export type TimedCaptionRegion = {
readonly startSec: number;
readonly endSec: number;
readonly refinements: readonly TimedCaptionRefinement[];
+ readonly wordIds?: readonly string[];
readonly fields?: readonly CaptionFieldAssignment[];
};
@@ -113,52 +106,3 @@ export type TimedCaptionProjection = {
readonly text: string;
readonly regions: readonly TimedCaptionRegion[];
};
-
-export type CaptionPresentationMode = "whole" | "proportional-word" | "character-flow";
-
-export type CaptionPresentationUnit = {
- readonly id: string;
- readonly regionId: string;
- readonly runId?: string;
- readonly styleId?: string;
- readonly display: string;
- readonly displayStart: number;
- readonly displayEnd: number;
- readonly startSec: number;
- readonly endSec: number;
- readonly fields?: readonly CaptionFieldAssignment[];
-};
-
-export type CaptionPresentationPlan = {
- readonly contract: "svml.caption-presentation-plan@1";
- readonly mode: CaptionPresentationMode;
- readonly units: readonly CaptionPresentationUnit[];
-};
-
-/** Official caption package's author-facing lowering program, not a Core type. */
-export type CaptionTrackProgram = {
- readonly contract: "svml.caption-track-program@1";
- readonly id: string;
- readonly mode: CaptionPresentationMode;
- readonly stacking: {
- readonly order: number;
- readonly tieBreak: string;
- };
- readonly style: {
- readonly fontFamily: string;
- readonly fontSizePx: number;
- readonly fontWeight: number;
- readonly color: string;
- readonly backgroundColor?: string;
- readonly paddingXPx: number;
- readonly paddingYPx: number;
- readonly borderRadiusPx: number;
- readonly bottomPercent: number;
- readonly maxWidthPercent: number;
- readonly leftPercent?: number;
- readonly topPercent?: number;
- readonly widthPercent?: number;
- readonly lineHeight?: number;
- readonly textAlign: "left" | "center" | "right";
- };
-};
diff --git a/packages/caption/test/caption.test.ts b/packages/caption/test/caption.test.ts
index 26bfbf8d..eb45d632 100644
--- a/packages/caption/test/caption.test.ts
+++ b/packages/caption/test/caption.test.ts
@@ -1,33 +1,36 @@
+import assert from "node:assert/strict";
+import test from "node:test";
+
import {
- captionComponent,
- defaultCaptionTrackProgram,
- planCaptionPresentation,
- renderCaptionProgram,
- renderCaptionTrack,
+ assertCaptionPlanForProgram,
resolveCaptionProgram,
sealCaptionPlan,
sealCaptionStyle,
- sealCaptionTrackProgram,
temporalizeCaption,
temporalizeCaptionPlan,
} from "@narratage/caption";
-import type { Narrative, NarrativeSelectionRef } from "@narratage/narrative";
+import type { CaptionFieldDeclaration, CaptionStyleIntent } from "@narratage/caption";
+import type { Narrative } from "@narratage/narrative";
+import { digestOf } from "@narratage/protocol";
import { sealProgramSpace } from "@narratage/program-space";
import { sealSpeechBasis } from "@narratage/speech";
import type { SpeechAudioBasis } from "@narratage/speech";
+import { locateSpeechTiming } from "@narratage/speech-alignment";
import { sealAlignedTranscriptEvidence } from "@narratage/speech-evidence";
import type { AlignedTranscriptSegment } from "@narratage/speech-evidence";
-import type { VisualTrack } from "@narratage/composition";
-import assert from "node:assert/strict";
-import test from "node:test";
-
-import { digestOf } from "@narratage/protocol";
-import { narrativeSelectionValue, parseScript } from "@narratage/script";
-import { locateSpeechTiming } from "@narratage/speech-alignment";
+import {
+ captionSelectionWordSubset,
+ captionWordSequence,
+ parseScript,
+} from "@narratage/script";
function locate(narrative: Narrative, durationSec: number, segments: readonly AlignedTranscriptSegment[]) {
- const programSpace = captionSpace(durationSec);
- const audioDigest = digestOf(`caption:audio:${narrative.segments.map((segment) => segment.id).join("+")}`);
+ const space = sealProgramSpace({
+ contract: "svml.program-space@1",
+ durationSec,
+ frameRate: { numerator: 30, denominator: 1 },
+ });
+ const audio = { digest: digestOf("caption:test-audio"), size: 1, mediaType: "audio/wav", durationSec };
const basisSegments = narrative.segments.map((segment, index) => ({
segmentId: segment.id,
startSec: segments[index]!.startSec,
@@ -35,26 +38,17 @@ function locate(narrative: Narrative, durationSec: number, segments: readonly Al
}));
const basis = sealSpeechBasis({
contract: "svml.speech-basis@1",
- programSpace,
- audio: { digest: audioDigest, size: 1, mediaType: "audio/wav", durationSec },
+ programSpace: space,
+ audio,
visualTrack: { clips: basisSegments.map((segment) => ({
segmentId: segment.segmentId,
- artifact: {
- digest: digestOf(`caption:clip:${segment.segmentId}`),
- size: 1,
- mediaType: "video/mp4",
- durationSec: segment.endSec - segment.startSec,
- },
+ artifact: { digest: digestOf(segment.segmentId), size: 1, mediaType: "video/mp4", durationSec: segment.endSec - segment.startSec },
startSec: segment.startSec,
endSec: segment.endSec,
})) },
segments: basisSegments,
});
- const evidence = sealAlignedTranscriptEvidence({
- contract: "svml.aligned-transcript-evidence@1",
- durationSec,
- segments,
- });
+ const evidence = sealAlignedTranscriptEvidence({ contract: "svml.aligned-transcript-evidence@1", durationSec, segments });
const audioBasis: SpeechAudioBasis = {
contract: "svml.speech-audio-basis@1",
programSpace: basis.programSpace,
@@ -64,306 +58,122 @@ function locate(narrative: Narrative, durationSec: number, segments: readonly Al
return locateSpeechTiming(narrative, audioBasis, evidence);
}
-function captionSpace(durationSec: number) {
- return sealProgramSpace({
- contract: "svml.program-space@1",
- durationSec,
- frameRate: { numerator: 1_000, denominator: 1 },
+function style(id: string, fields: readonly CaptionFieldDeclaration[] = []): CaptionStyleIntent {
+ return sealCaptionStyle({
+ contract: "svml.caption-style@1",
+ id,
+ planning: {
+ cue: { minimumWords: 1, maximumWords: 5, instruction: "Use short complete semantic phrases." },
+ fields,
+ },
+ rendering: { family: "test-caption@1", parameters: {} },
});
}
-test("Caption-owned validators reject semantically invalid values", () => {
- const narrative = parseScript("validator.svml", "<line>Hello.</line>");
- const projection = temporalizeCaption(narrative, locate(narrative, 1, [{
- sourceSegmentId: "line",
- startSec: 0,
- endSec: 1,
- words: [{ text: "Hello", startSec: 0.1, endSec: 0.6 }],
- chars: [],
- }]));
- const program = defaultCaptionTrackProgram("validator-caption");
- const projectionValidator = captionComponent.validators?.find((facet) =>
- facet.type.name === "TimedCaptionProjection");
- const programValidator = captionComponent.validators?.find((facet) =>
- facet.type.name === "CaptionTrackProgram");
- assert(projectionValidator);
- assert(programValidator);
- assert.throws(
- () => projectionValidator.handler({
- type: projectionValidator.type,
- value: { kind: "inline", value: { ...projection, contract: "invalid" } },
- }),
- /Unsupported TimedCaptionProjection/u,
- );
- assert.throws(
- () => programValidator.handler({
- type: programValidator.type,
- value: { kind: "inline", value: { ...program, id: "" } },
- }),
- /identity and font family must not be empty/u,
- );
-});
-
-test("Caption temporalization preserves evidence envelopes and refines them per word", () => {
+test("Script projects an ordered word universe and explicit Selection word subset", () => {
const narrative = parseScript(
- "caption.svml",
- "<line><that was insane | what the fuck> <15% off | fifteen percent off></line>",
- );
- const map = locate(narrative, 2, [{
- sourceSegmentId: "line",
- startSec: 0,
- endSec: 2,
- words: [
- { text: "what", startSec: 0.1, endSec: 0.25 },
- { text: "the", startSec: 0.3, endSec: 0.42 },
- { text: "fuck", startSec: 0.48, endSec: 0.72 },
- { text: "fifteen", startSec: 0.9, endSec: 1.15 },
- { text: "percent", startSec: 1.2, endSec: 1.42 },
- { text: "off", startSec: 1.48, endSec: 1.65 },
- ],
- chars: [],
- }]);
- const originalMap = structuredClone(map);
- const timed = temporalizeCaption(narrative, map);
-
- assert.equal(timed.regions[0]?.display, "that was insane");
- assert.deepEqual(
- [timed.regions[0]?.startSec, timed.regions[0]?.endSec, timed.regions[0]?.refinements.length],
- [0.1, 0.72, 0],
- );
- assert.deepEqual(timed.regions[1]?.refinements.map((item) => item.display), ["off"]);
-
- const whole = planCaptionPresentation(timed, "whole");
- assert.deepEqual(
- [whole.units[0]?.display, whole.units[0]?.startSec, whole.units[0]?.endSec],
- ["that was insane", 0.1, 0.72],
+ "selection.svml",
+ "<line>Keep this @special one sentence different @/special and return.</line>",
);
+ const words = captionWordSequence(narrative, "story.caption.words");
+ const special = captionSelectionWordSubset(words, narrative.selections[0]!);
+ assert.deepEqual(words.words.map((word) => word.text), ["Keep", "this", "one", "sentence", "different", "and", "return"]);
+ assert.deepEqual(special.wordIds.map((id) => words.words.find((word) => word.id === id)!.text),
+ ["one", "sentence", "different"]);
- const words = planCaptionPresentation(timed, "proportional-word");
- assert.deepEqual(map, originalMap, "caption presentation must not modify the global speech map");
-});
-
-test("multiple Cues inside one display alias receive ordered local estimates, not duplicate envelopes", () => {
- const narrative = parseScript("alias-plan.svml", "<line><that was insane | what the fuck></line>");
- const map = locate(narrative, 1, [{
- sourceSegmentId: "line",
- startSec: 0,
- endSec: 1,
- words: [
- { text: "what", startSec: 0.1, endSec: 0.25 },
- { text: "the", startSec: 0.3, endSec: 0.42 },
- { text: "fuck", startSec: 0.48, endSec: 0.72 },
- ],
- chars: [],
+ const program = resolveCaptionProgram(words, "captions", style("normal"), [{
+ id: "special-use",
+ words: special,
+ style: style("special"),
}]);
- const base = defaultCaptionTrackProgram("alias-base");
- const style = sealCaptionStyle({
- contract: "svml.caption-style@1",
- id: "alias-style",
- planning: { cueInstruction: "One word per Cue for this test.", fields: [] },
- presentation: { mode: "whole", stackingOrder: 100, style: base.style },
- });
- const program = resolveCaptionProgram(narrative, "alias-caption", style, []);
- const run = program.runs[0]!;
- const plan = sealCaptionPlan({
- contract: "svml.caption-plan@1",
- runs: [{
- id: run.id,
- styleId: run.styleId,
- cues: run.atomIds.map((atomId, index) => ({ id: `alias-cue:${index + 1}`, atomIds: [atomId], fields: [] })),
- }],
- });
- const projection = temporalizeCaptionPlan(narrative, map, program, plan);
-
- assert.deepEqual(projection.regions.map((region) => region.display), ["that", "was", "insane"]);
- assert.equal(projection.regions[0]!.endSec <= projection.regions[1]!.startSec, true);
- assert.equal(projection.regions[1]!.endSec <= projection.regions[2]!.startSec, true);
- assert.deepEqual(
- [projection.regions[0]!.startSec, projection.regions[2]!.endSec],
- [0.1, 0.72],
- );
-});
-
-test("hidden speech owns time but emits no visible presentation unit", () => {
- const narrative = parseScript("hidden.svml", "<line>Hello < | um> world.</line>");
- const map = locate(narrative, 1, [{
- sourceSegmentId: "line",
- startSec: 0,
- endSec: 1,
- words: [
- { text: "Hello", startSec: 0.1, endSec: 0.25 },
- { text: "um", startSec: 0.3, endSec: 0.4 },
- { text: "world", startSec: 0.45, endSec: 0.7 },
- ],
- chars: [],
- }]);
- const timed = temporalizeCaption(narrative, map);
- const hidden = timed.regions.find((region) => region.kind === "hidden");
- assert.deepEqual([hidden?.startSec, hidden?.endSec], [0.3, 0.4]);
- assert.equal(planCaptionPresentation(timed).units.some((unit) => unit.regionId === hidden?.id), false);
+ assert.deepEqual(program.runs.map((run) => [run.styleId, run.wordIds.length]), [
+ ["normal", 2], ["special", 3], ["normal", 2],
+ ]);
+ assert.equal("words" in program, false, "Program must reference the word edge rather than copy its payload");
});
-test("official caption styling lowers to an ordinary self-contained VisualTrack", () => {
- const narrative = parseScript("track.svml", "<line>Hello world.</line>");
- const map = locate(narrative, 1, [{
- sourceSegmentId: "line",
- startSec: 0,
- endSec: 1,
- words: [
- { text: "Hello", startSec: 0.1, endSec: 0.35 },
- { text: "world", startSec: 0.4, endSec: 0.75 },
- ],
- chars: [],
+test("Role replacement is a word subset and the explicit default covers roleless prose", () => {
+ const narrative = parseScript("roles.svml", `
+ <intro>Roleless words stay readable.</intro>
+ <answer><ALICE>Only this sentence changes.</answer>
+ `);
+ const words = captionWordSequence(narrative, "story.caption.words");
+ const aliceIds = words.words.filter((word) => word.role === "ALICE").map((word) => word.id);
+ const program = resolveCaptionProgram(words, "captions", style("normal"), [{
+ id: "alice-use",
+ words: { contract: "svml.caption-word-subset@1", id: "role:ALICE", sequenceId: words.id, wordIds: aliceIds },
+ style: style("alice"),
}]);
- const projection = temporalizeCaption(narrative, map);
- const program = defaultCaptionTrackProgram("primary-caption");
- const track = renderCaptionTrack(projection, program, captionSpace(1));
-
- assert.equal(track.contract, "svml.visual-track@1");
- assert.equal(track.presents[0]?.elements.some((element) => element.kind === "text"), true);
- assert.equal(
- track.presents.flatMap((present) => present.elements).some((element) => "text" in element && element.text.includes("Hello")),
- true,
- );
+ assert.deepEqual(program.runs.map((run) => run.styleId), ["normal", "alice"]);
});
-test("two caption styles become two peer Tracks without mutating one another", () => {
- const narrative = parseScript("two-tracks.svml", "<line>Hello world.</line>");
- const projection = temporalizeCaption(narrative, locate(narrative, 1, [{
- sourceSegmentId: "line",
- startSec: 0,
- endSec: 1,
- words: [{ text: "Hello", startSec: 0.1, endSec: 0.35 }, { text: "world", startSec: 0.4, endSec: 0.75 }],
- chars: [],
- }]));
- const first = defaultCaptionTrackProgram("speaker-a");
- const second = sealCaptionTrackProgram({
- ...first,
- id: "speaker-b",
- stacking: { order: 101, tieBreak: "speaker-b" },
- style: { ...first.style, color: "#00ff00", bottomPercent: 20 },
- });
- const firstTrack = renderCaptionTrack(projection, first, captionSpace(1));
- const secondTrack = renderCaptionTrack(projection, second, captionSpace(1));
-
- assert.notDeepEqual(firstTrack, secondTrack);
- assert.deepEqual(
- firstTrack.presents.map((present) => present.span),
- secondTrack.presents.map((present) => present.span),
- );
- assert.equal(firstTrack.presents[0]?.stacking.order, 100);
- assert.equal(secondTrack.presents[0]?.stacking.order, 101);
-});
-
-test("a planner-neutral CaptionPlan joins SemanticMap only after Cue and field planning", () => {
- const narrative = parseScript("planned.svml", "<line><ALICE>Meaning becomes the source.</line>");
- const map = locate(narrative, 2, [{
- sourceSegmentId: "line",
- startSec: 0,
- endSec: 2,
- words: [
- { text: "Meaning", startSec: 0.1, endSec: 0.35 },
- { text: "becomes", startSec: 0.4, endSec: 0.7 },
- { text: "the", startSec: 0.8, endSec: 1.0 },
- { text: "source", startSec: 1.1, endSec: 1.5 },
- ],
- chars: [],
- }]);
- const base = defaultCaptionTrackProgram("planned-caption");
- const style = sealCaptionStyle({
- contract: "svml.caption-style@1",
- id: "alice",
- planning: {
- cueInstruction: "Use short semantic phrases.",
- fields: [{
- id: "important",
- value: { kind: "boolean" },
- instruction: "Select at most one important word.",
- minimumPerCue: 0,
- maximumPerCue: 1,
- }],
- },
- presentation: { mode: "whole", stackingOrder: 100, style: base.style },
- });
- const program = resolveCaptionProgram(narrative, "planned-caption", style, []);
- const ids = program.atoms.map((atom) => atom.id);
+test("Caption Plan validates Cue word bounds and independent per-word fields", () => {
+ const narrative = parseScript("plan.svml", "<line>one two three four.</line>");
+ const words = captionWordSequence(narrative, "story.caption.words");
+ const important: CaptionFieldDeclaration = {
+ id: "important",
+ value: { kind: "boolean" },
+ instruction: "Select up to two important words.",
+ minimumPerCue: 0,
+ maximumPerCue: 2,
+ };
+ const program = resolveCaptionProgram(words, "captions", style("fine", [important]), []);
+ const run = program.runs[0]!;
const plan = sealCaptionPlan({
contract: "svml.caption-plan@1",
- runs: [{
- id: program.runs[0]!.id,
- styleId: style.id,
- cues: [
- {
- id: "cue:1",
- atomIds: ids.slice(0, 2),
- fields: [{ declarationId: "important", atomId: ids[0]!, value: "true" }],
- },
- { id: "cue:2", atomIds: ids.slice(2), fields: [] },
+ runs: [{ id: run.id, styleId: run.styleId, cues: [{
+ id: "cue:1",
+ wordIds: run.wordIds,
+ fields: [
+ { declarationId: "important", wordId: run.wordIds[0]!, value: "true" },
+ { declarationId: "important", wordId: run.wordIds[2]!, value: "true" },
],
- }],
+ }] }],
});
- const projection = temporalizeCaptionPlan(narrative, map, program, plan);
- assert.deepEqual(projection.regions.map((region) => region.display), ["Meaning becomes", "the source."]);
- assert.deepEqual(projection.regions[0]?.fields?.map((field) => field.declarationId), ["important"]);
- const track = renderCaptionProgram(projection, program, captionSpace(2));
- const firstText = track.presents[0]?.elements.find((element) => element.kind === "text");
- assert.equal(firstText?.attributes?.some((attribute) =>
- attribute.name === "data-caption-style" && attribute.value === "alice"), true);
- assert.equal(firstText?.attributes?.some((attribute) =>
- attribute.name === "data-caption-fields" && attribute.value.includes("important")), true);
+ assert.doesNotThrow(() => assertCaptionPlanForProgram(plan, program));
});
-test("a default Style covers roleless text and ordered whole-style overrides use last match", () => {
- const narrative = parseScript("cascade.svml", `
- <intro>Roleless words stay readable.</intro>
- <answer><ALICE>Only this sentence changes.</answer>
- `);
- const base = defaultCaptionTrackProgram("base");
- const make = (id: string, color: string) => sealCaptionStyle({
+test("Program rejects a Style run that no legal Cue partition can satisfy", () => {
+ const narrative = parseScript("impossible.svml", "<line>one two three four five six.</line>");
+ const words = captionWordSequence(narrative, "story.caption.words");
+ const impossible = sealCaptionStyle({
contract: "svml.caption-style@1",
- id,
- planning: { cueInstruction: `Plan ${id}.`, fields: [] },
- presentation: { mode: "whole", stackingOrder: 100, style: { ...base.style, color } },
+ id: "impossible",
+ planning: { cue: { minimumWords: 4, maximumWords: 5, instruction: "Use four or five words." }, fields: [] },
+ rendering: { family: "test-caption@1", parameters: {} },
});
- const normal = make("normal", "#ffffff");
- const alice = make("alice", "#00ff00");
- const final = make("final", "#ff00ff");
- const program = resolveCaptionProgram(narrative, "captions", normal, [
- { id: "alice", selector: { kind: "role", role: "ALICE" }, style: alice },
- { id: "alice-later", selector: { kind: "role", role: "ALICE" }, style: final },
- ]);
- assert.deepEqual(program.runs.map((run) => run.styleId), ["normal", "final"]);
- assert.equal(program.styles.some((style) => style.id === "alice"), true, "declared overrides stay auditable");
- assert.throws(() => resolveCaptionProgram(narrative, "invalid", normal, [{
- id: "typo",
- selector: { kind: "role", role: "ALIEC" },
- style: alice,
- }]), /selects no visible display atom/u);
+ assert.throws(() => resolveCaptionProgram(words, "captions", impossible, []), /cannot satisfy/u);
});
-test("an explicit Selection replaces only its words without authoring the complement", () => {
- const narrative = parseScript(
- "selection-style.svml",
- "<line>Keep this @special one sentence different @/special and return.</line>",
- );
- const base = defaultCaptionTrackProgram("base");
- const make = (id: string, color: string) => sealCaptionStyle({
- contract: "svml.caption-style@1",
- id,
- planning: { cueInstruction: `Plan ${id}.`, fields: [] },
- presentation: { mode: "whole", stackingOrder: 100, style: { ...base.style, color } },
+test("planned display words join measured speech timing only after planning", () => {
+ const narrative = parseScript("timed.svml", "<line><that was insane | what the fuck></line>");
+ const words = captionWordSequence(narrative, "story.caption.words");
+ const program = resolveCaptionProgram(words, "captions", style("fine"), []);
+ const run = program.runs[0]!;
+ const plan = sealCaptionPlan({
+ contract: "svml.caption-plan@1",
+ runs: [{ id: run.id, styleId: run.styleId, cues: run.wordIds.map((wordId, index) => ({
+ id: `cue:${index + 1}`, wordIds: [wordId], fields: [],
+ })) }],
});
- const normal = make("normal", "#ffffff");
- const special = make("special", "#ff00ff");
- const selection = narrativeSelectionValue(narrative.selections[0]!) as unknown as NarrativeSelectionRef;
- const program = resolveCaptionProgram(narrative, "captions", normal, [{
- id: "special-use",
- selector: { kind: "selection", selection },
- style: special,
+ const map = locate(narrative, 1, [{
+ sourceSegmentId: "line", startSec: 0, endSec: 1,
+ words: [
+ { text: "what", startSec: 0.1, endSec: 0.25 },
+ { text: "the", startSec: 0.3, endSec: 0.42 },
+ { text: "fuck", startSec: 0.48, endSec: 0.72 },
+ ],
+ chars: [],
}]);
- assert.deepEqual(program.runs.map((run) => [run.styleId, run.atomIds.length]), [
- ["normal", 2],
- ["special", 3],
- ["normal", 2],
- ]);
+ const projection = temporalizeCaptionPlan(narrative, map, words, program, plan);
+ assert.deepEqual(projection.regions.map((region) => region.display), ["that", "was", "insane"]);
+ assert.equal(projection.regions[0]!.endSec <= projection.regions[1]!.startSec, true);
+ assert.deepEqual(projection.regions.map((region) => region.wordIds?.length), [1, 1, 1]);
+ assert.equal(temporalizeCaption(narrative, map).text, "that was insane");
+});
+
+test("a Selection cutting an indivisible Dual Text word fails before Caption planning", () => {
+ const narrative = parseScript("partial.svml", "<line><lmao | laughed my @part ass out @/part></line>");
+ const words = captionWordSequence(narrative, "story.caption.words");
+ assert.throws(() => captionSelectionWordSubset(words, narrative.selections[0]!), /owns only part/u);
});
diff --git a/packages/narrative/src/index.ts b/packages/narrative/src/index.ts
index 41459a8e..523c2e82 100644
--- a/packages/narrative/src/index.ts
+++ b/packages/narrative/src/index.ts
@@ -1,3 +1,3 @@
export { narrativeDependency, narrativeManifest, narrativeManifestDigest, narrativeModuleRef, narrativeTypes } from "./manifest.js";
-export { captionProjectionSchema, narrativeDialogueExcerptSchema, narrativeExcerptSchema, narrativeMomentSchema, narrativeSchema, narrativeSelectionSchema, narrativeSpeechExcerptSchema } from "./schema.js";
+export { captionProjectionSchema, captionWordSequenceSchema, captionWordSubsetSchema, narrativeDialogueExcerptSchema, narrativeExcerptSchema, narrativeMomentSchema, narrativeSchema, narrativeSelectionSchema, narrativeSpeechExcerptSchema } from "./schema.js";
export type * from "./types.js";
diff --git a/packages/narrative/src/manifest.ts b/packages/narrative/src/manifest.ts
index 85b5e395..e5200af9 100644
--- a/packages/narrative/src/manifest.ts
+++ b/packages/narrative/src/manifest.ts
@@ -2,6 +2,8 @@ import { digestOf } from "@narratage/protocol";
import type { ModuleManifest, TypeRef } from "@narratage/protocol";
import {
captionProjectionSchema,
+ captionWordSequenceSchema,
+ captionWordSubsetSchema,
narrativeDialogueExcerptSchema,
narrativeExcerptSchema,
narrativeSchema,
@@ -19,6 +21,8 @@ export const narrativeTypes = {
selection: { module: narrativeModuleRef, name: "NarrativeSelection" },
moment: { module: narrativeModuleRef, name: "NarrativeMoment" },
captionProjection: { module: narrativeModuleRef, name: "CaptionProjection" },
+ captionWordSequence: { module: narrativeModuleRef, name: "CaptionWordSequence" },
+ captionWordSubset: { module: narrativeModuleRef, name: "CaptionWordSubset" },
} satisfies Record<string, TypeRef>;
export const narrativeManifest: ModuleManifest = {
format: "svml.module@1", name: narrativeModuleRef.name, version: narrativeModuleRef.version,
@@ -31,6 +35,8 @@ export const narrativeManifest: ModuleManifest = {
{ name: narrativeTypes.selection.name, schema: narrativeSelectionSchema },
{ name: narrativeTypes.moment.name, schema: narrativeMomentSchema },
{ name: narrativeTypes.captionProjection.name, schema: captionProjectionSchema },
+ { name: narrativeTypes.captionWordSequence.name, schema: captionWordSequenceSchema },
+ { name: narrativeTypes.captionWordSubset.name, schema: captionWordSubsetSchema },
],
capabilities: [], surfaces: [], producers: [],
};
diff --git a/packages/narrative/src/schema.ts b/packages/narrative/src/schema.ts
index 0596bbcc..8d4d20d1 100644
--- a/packages/narrative/src/schema.ts
+++ b/packages/narrative/src/schema.ts
@@ -5,17 +5,6 @@ const integer = { kind: "number", integer: true, minimum: 0 } as const;
const object = (
fields: Readonly<Record<string, { readonly schema: ValueSchema; readonly optional?: boolean }>>,
): ValueSchema => ({ kind: "object", fields });
-const markerBoundary = object({
- tokenIndex: { schema: integer },
- structuralPosition: { schema: integer },
- segmentId: { schema: string, optional: true },
- anchorId: { schema: string },
-});
-const markerEdge = object({
- affinity: { schema: { kind: "string", enum: ["left", "right"] } },
- boundary: { schema: markerBoundary },
-});
-
export const narrativeSchema: ValueSchema = object({
contract: { schema: { kind: "literal", value: "svml.narrative@1" } },
segments: { schema: { kind: "array", minItems: 1, items: object({
@@ -33,13 +22,12 @@ export const narrativeSchema: ValueSchema = object({
}) } },
selections: { schema: { kind: "array", items: object({
id: { schema: string }, occurrences: { schema: { kind: "array", minItems: 1, items: object({
- occurrence: { schema: integer }, open: { schema: markerEdge }, close: { schema: markerEdge },
+ occurrence: { schema: integer }, startAnchorId: { schema: string }, endAnchorId: { schema: string },
}) } },
}) } },
moments: { schema: { kind: "array", items: object({
id: { schema: string }, occurrences: { schema: { kind: "array", minItems: 1, items: object({
- occurrence: { schema: integer }, affinity: { schema: { kind: "string", enum: ["left", "right"] } },
- boundary: { schema: markerBoundary },
+ occurrence: { schema: integer }, anchorId: { schema: string },
}) } },
}) } },
captionProjection: { schema: object({
@@ -100,16 +88,31 @@ export const captionProjectionSchema: ValueSchema = object({
}) } },
}) } },
});
+const captionWord = object({
+ id: { schema: string }, index: { schema: integer }, regionId: { schema: string }, segmentId: { schema: string },
+ turnId: { schema: string }, role: { schema: string, optional: true }, text: { schema: string },
+ displayStart: { schema: integer }, displayEnd: { schema: integer }, sourceTokenStart: { schema: integer },
+ sourceTokenEndExclusive: { schema: integer },
+ correspondence: { schema: { kind: "string", enum: ["exact", "region-envelope"] } },
+});
+export const captionWordSequenceSchema: ValueSchema = object({
+ contract: { schema: { kind: "literal", value: "svml.caption-word-sequence@1" } },
+ id: { schema: string }, words: { schema: { kind: "array", minItems: 1, items: captionWord } },
+});
+export const captionWordSubsetSchema: ValueSchema = object({
+ contract: { schema: { kind: "literal", value: "svml.caption-word-subset@1" } },
+ id: { schema: string }, sequenceId: { schema: string },
+ wordIds: { schema: { kind: "array", items: string } },
+});
export const narrativeMomentSchema: ValueSchema = object({
contract: { schema: { kind: "literal", value: "svml.narrative-moment@1" } }, id: { schema: string },
occurrences: { schema: { kind: "array", minItems: 1, items: object({
- occurrence: { schema: integer }, affinity: { schema: { kind: "string", enum: ["left", "right"] } },
- boundary: { schema: markerBoundary },
+ occurrence: { schema: integer }, anchorId: { schema: string },
}) } },
});
export const narrativeSelectionSchema: ValueSchema = object({
contract: { schema: { kind: "literal", value: "svml.narrative-selection@1" } }, id: { schema: string },
occurrences: { schema: { kind: "array", minItems: 1, items: object({
- occurrence: { schema: integer }, open: { schema: markerEdge }, close: { schema: markerEdge },
+ occurrence: { schema: integer }, startAnchorId: { schema: string }, endAnchorId: { schema: string },
}) } },
});
diff --git a/packages/narrative/src/types.ts b/packages/narrative/src/types.ts
index ef111492..2bd73d0a 100644
--- a/packages/narrative/src/types.ts
+++ b/packages/narrative/src/types.ts
@@ -1,13 +1,3 @@
-export type Affinity = "left" | "right";
-
-export type MarkerBoundary = {
- readonly tokenIndex: number;
- readonly structuralPosition: number;
- readonly segmentId?: string;
- /** The exact 2M+2N anchor this marker's affinity resolves to, fixed at parse time. */
- readonly anchorId: string;
-};
-
export type NarrativeSegment = {
readonly id: string;
readonly index: number;
@@ -38,8 +28,9 @@ export type NarrativeTurn = {
export type NarrativeSelectionOccurrence = {
readonly occurrence: number;
- readonly open: { readonly affinity: Affinity; readonly boundary: MarkerBoundary };
- readonly close: { readonly affinity: Affinity; readonly boundary: MarkerBoundary };
+ /** The exact semantic anchors chosen by the author Surface's affinity syntax. */
+ readonly startAnchorId: string;
+ readonly endAnchorId: string;
};
export type NarrativeSelection = {
@@ -54,8 +45,8 @@ export type NarrativeSelectionRef = NarrativeSelection & {
export type NarrativeMomentOccurrence = {
readonly occurrence: number;
- readonly affinity: Affinity;
- readonly boundary: MarkerBoundary;
+ /** The exact semantic anchor chosen by the author Surface's affinity syntax. */
+ readonly anchorId: string;
};
export type NarrativeMoment = {
@@ -97,6 +88,37 @@ export type CaptionProjection = {
/** Whole authored display projection. It contains no pronunciation replacement text. */
export type CaptionProjectionRef = CaptionProjection;
+/** One visible display word. Pronunciation-only text never enters this value. */
+export type CaptionWord = {
+ readonly id: string;
+ readonly index: number;
+ readonly regionId: string;
+ readonly segmentId: string;
+ readonly turnId: string;
+ readonly role?: string;
+ readonly text: string;
+ readonly displayStart: number;
+ readonly displayEnd: number;
+ readonly sourceTokenStart: number;
+ readonly sourceTokenEndExclusive: number;
+ readonly correspondence: "exact" | "region-envelope";
+};
+
+/** The complete ordered visible-word universe for one Script Caption projection. */
+export type CaptionWordSequence = {
+ readonly contract: "svml.caption-word-sequence@1";
+ readonly id: string;
+ readonly words: readonly CaptionWord[];
+};
+
+/** One ordered subset of an exact CaptionWordSequence, projected by Script structure. */
+export type CaptionWordSubset = {
+ readonly contract: "svml.caption-word-subset@1";
+ readonly id: string;
+ readonly sequenceId: string;
+ readonly wordIds: readonly string[];
+};
+
export type SemanticAnchor = {
readonly id: string;
readonly kind: "segment-start" | "token-start" | "token-end" | "segment-end";
diff --git a/packages/provider-google-vertex/src/provider.ts b/packages/provider-google-vertex/src/provider.ts
index 6e16f6f9..fb758b07 100644
--- a/packages/provider-google-vertex/src/provider.ts
+++ b/packages/provider-google-vertex/src/provider.ts
@@ -67,23 +67,23 @@ const captionResponseSchema: Schema = {
items: {
type: Type.OBJECT,
properties: {
- after_atom_id: { type: Type.STRING },
+ after_word_id: { type: Type.STRING },
fields: {
type: Type.ARRAY,
items: {
type: Type.OBJECT,
properties: {
declaration_id: { type: Type.STRING },
- atom_id: { type: Type.STRING },
+ word_id: { type: Type.STRING },
value: { type: Type.STRING },
},
- required: ["declaration_id", "atom_id", "value"],
- propertyOrdering: ["declaration_id", "atom_id", "value"],
+ required: ["declaration_id", "word_id", "value"],
+ propertyOrdering: ["declaration_id", "word_id", "value"],
},
},
},
- required: ["after_atom_id", "fields"],
- propertyOrdering: ["after_atom_id", "fields"],
+ required: ["after_word_id", "fields"],
+ propertyOrdering: ["after_word_id", "fields"],
},
},
},
diff --git a/packages/provider-google-vertex/test/provider.test.ts b/packages/provider-google-vertex/test/provider.test.ts
index 344f40ad..be874440 100644
--- a/packages/provider-google-vertex/test/provider.test.ts
+++ b/packages/provider-google-vertex/test/provider.test.ts
@@ -8,7 +8,6 @@ import {
} from "@narratage/caption-gemini";
import {
captionTypes,
- defaultCaptionTrackProgram,
resolveCaptionProgram,
sealCaptionStyle,
} from "@narratage/caption";
@@ -22,16 +21,16 @@ import {
googleVertexProviderImplementationDigest,
} from "@narratage/provider-google-vertex";
import type { GenerateCaptionContent } from "@narratage/provider-google-vertex";
-import { parseScript } from "@narratage/script";
+import { captionWordSequence, parseScript } from "@narratage/script";
function request(): CaptionGeminiRequest {
const narrative = parseScript("provider.svml", "<line><ALICE>Meaning becomes the source.</line>");
- const base = defaultCaptionTrackProgram("important");
+ const words = captionWordSequence(narrative, "story.caption.words");
const style = sealCaptionStyle({
contract: "svml.caption-style@1",
id: "important",
planning: {
- cueInstruction: "Prefer one short complete semantic phrase.",
+ cue: { minimumWords: 1, maximumWords: 7, instruction: "Prefer one short complete semantic phrase." },
fields: [{
id: "important",
value: { kind: "enum", values: ["important"] },
@@ -40,10 +39,10 @@ function request(): CaptionGeminiRequest {
maximumPerCue: 2,
}],
},
- presentation: { mode: "whole", stackingOrder: 100, style: base.style },
+ rendering: { family: "test-caption@1", parameters: {} },
});
- const captionProgram = resolveCaptionProgram(narrative, "captions", style, []);
- return compileCaptionGeminiRequest(narrative, captionProgram, sealCaptionGeminiProgram({
+ const captionProgram = resolveCaptionProgram(words, "captions", style, []);
+ return compileCaptionGeminiRequest(words, captionProgram, sealCaptionGeminiProgram({
contract: "svml.caption-gemini-program@1",
model: "gemini-2.5-flash",
}));
@@ -53,8 +52,8 @@ function response(requestValue: CaptionGeminiRequest): RawCaptionGeminiResponse
return { runs: requestValue.runs.map((run) => ({
run_id: run.id,
cues: [{
- after_atom_id: run.atomIds.at(-1)!,
- fields: [{ declaration_id: "important", atom_id: run.atomIds[0]!, value: "important" }],
+ after_word_id: run.wordIds.at(-1)!,
+ fields: [{ declaration_id: "important", word_id: run.wordIds[0]!, value: "important" }],
}],
})) };
}
diff --git a/packages/script/src/index.ts b/packages/script/src/index.ts
index e217acfc..222917db 100644
--- a/packages/script/src/index.ts
+++ b/packages/script/src/index.ts
@@ -2,6 +2,8 @@ export { ScriptSyntaxError } from "./error.js";
export { formatScript } from "./format.js";
export {
captionProjectionType,
+ captionWordSequenceType,
+ captionWordSubsetType,
narrativeDialogueExcerptType,
narrativeExcerptType,
narrativeMomentType,
@@ -15,6 +17,10 @@ export {
} from "./manifest.js";
export {
captionProjectionValue,
+ captionSelectionWordSubset,
+ captionSelectionWordSubsetValue,
+ captionWordSequence,
+ captionWordSequenceValue,
narrativeDialogueExcerptValue,
narrativeSegmentExcerptValue,
narrativeMomentValue,
diff --git a/packages/script/src/manifest.ts b/packages/script/src/manifest.ts
index 2c7ca3e5..706b9e71 100644
--- a/packages/script/src/manifest.ts
+++ b/packages/script/src/manifest.ts
@@ -10,6 +10,8 @@ export const narrativeSpeechExcerptType: TypeRef = narrativeTypes.speechExcerpt;
export const narrativeSelectionType: TypeRef = narrativeTypes.selection;
export const narrativeMomentType: TypeRef = narrativeTypes.moment;
export const captionProjectionType: TypeRef = narrativeTypes.captionProjection;
+export const captionWordSequenceType: TypeRef = narrativeTypes.captionWordSequence;
+export const captionWordSubsetType: TypeRef = narrativeTypes.captionWordSubset;
export { narrativeSchema };
export const scriptSurfaceImplementationDigest = digestOf("@narratage/script/surface@1");
@@ -34,6 +36,8 @@ export const scriptManifest: ModuleManifest = {
narrativeSelectionType,
narrativeMomentType,
captionProjectionType,
+ captionWordSequenceType,
+ captionWordSubsetType,
],
implementation: {
kind: "trusted-frontend-surface",
diff --git a/packages/script/src/narrative.ts b/packages/script/src/narrative.ts
index 368d19c3..360550eb 100644
--- a/packages/script/src/narrative.ts
+++ b/packages/script/src/narrative.ts
@@ -1,8 +1,101 @@
import { canonicalize } from "@narratage/core";
import type { CanonicalValue } from "@narratage/protocol";
+import type { CaptionRegion, CaptionWord, CaptionWordSequence, CaptionWordSubset } from "@narratage/narrative";
import type { ParsedNarrative } from "./types.js";
+const DISPLAY_WORD =
+ /[\p{Script=Han}\p{Script=Hiragana}\p{Script=Katakana}]\p{M}*|[\p{L}\p{M}\p{N}]+(?:['’.-][\p{L}\p{M}\p{N}]+)*/gu;
+
+function displayWords(value: string): Array<{ readonly text: string; readonly start: number; readonly end: number }> {
+ const result: Array<{ text: string; start: number; end: number }> = [];
+ DISPLAY_WORD.lastIndex = 0;
+ let match: RegExpExecArray | null;
+ while ((match = DISPLAY_WORD.exec(value))) {
+ result.push({ text: match[0], start: match.index, end: match.index + match[0].length });
+ }
+ return result;
+}
+
+function turnForRegion(parsed: ParsedNarrative, region: CaptionRegion): ParsedNarrative["turns"][number] {
+ const turn = parsed.turns.find((candidate) =>
+ candidate.segmentId === region.segmentId
+ && candidate.tokenStart < region.endTokenExclusive
+ && candidate.tokenEndExclusive > region.startToken);
+ if (turn === undefined) throw new Error(`Caption region ${region.id} is not owned by a Narrative Turn`);
+ return turn;
+}
+
+/** Script owns display-word projection because only it still knows authored source structure. */
+export function captionWordSequence(parsed: ParsedNarrative, id: string): CaptionWordSequence {
+ const words: CaptionWord[] = [];
+ for (const region of parsed.captionProjection.regions) {
+ if (region.kind === "hidden") continue;
+ const turn = turnForRegion(parsed, region);
+ for (const word of displayWords(region.display)) {
+ const exact = region.refinements.find((refinement) =>
+ refinement.displayStart === word.start && refinement.displayEnd === word.end);
+ words.push({
+ id: `${region.id}:word:${words.length + 1}`,
+ index: words.length,
+ regionId: region.id,
+ segmentId: region.segmentId,
+ turnId: turn.id,
+ ...(turn.role === undefined ? {} : { role: turn.role }),
+ text: word.text,
+ displayStart: word.start,
+ displayEnd: word.end,
+ sourceTokenStart: exact?.startToken ?? region.startToken,
+ sourceTokenEndExclusive: exact?.endTokenExclusive ?? region.endTokenExclusive,
+ correspondence: exact === undefined ? "region-envelope" : "exact",
+ });
+ }
+ }
+ if (words.length === 0) throw new Error("Caption display projection contains no visible words");
+ return { contract: "svml.caption-word-sequence@1", id, words };
+}
+
+/** Project one authored Selection into visible words before public values lose source positions. */
+export function captionSelectionWordSubset(
+ sequence: CaptionWordSequence,
+ selection: ParsedNarrative["selections"][number],
+): CaptionWordSubset {
+ const ranges = selection.occurrences.map((occurrence) => ({
+ start: occurrence.open.boundary.tokenIndex,
+ endExclusive: occurrence.close.boundary.tokenIndex,
+ }));
+ const wordIds: string[] = [];
+ for (const word of sequence.words) {
+ const intersects = ranges.some((range) =>
+ word.sourceTokenStart < range.endExclusive && word.sourceTokenEndExclusive > range.start);
+ const contained = ranges.some((range) =>
+ word.sourceTokenStart >= range.start && word.sourceTokenEndExclusive <= range.endExclusive);
+ if (intersects && !contained) {
+ throw new Error(
+ `Selection ${selection.id} owns only part of Caption word ${word.id}; split the Dual Text atom`,
+ );
+ }
+ if (contained) wordIds.push(word.id);
+ }
+ return {
+ contract: "svml.caption-word-subset@1",
+ id: selection.id,
+ sequenceId: sequence.id,
+ wordIds,
+ };
+}
+
+export function captionWordSequenceValue(parsed: ParsedNarrative, id: string): CanonicalValue {
+ return canonicalize(captionWordSequence(parsed, id));
+}
+
+export function captionSelectionWordSubsetValue(
+ sequence: CaptionWordSequence,
+ selection: ParsedNarrative["selections"][number],
+): CanonicalValue {
+ return canonicalize(captionSelectionWordSubset(sequence, selection));
+}
+
function cleanProjection(value: string): string {
return value
.replace(/\s+/gu, " ")
@@ -115,8 +208,8 @@ export function narrativeSelectionValue(selection: ParsedNarrative["selections"]
id: selection.id,
occurrences: selection.occurrences.map((occurrence) => ({
occurrence: occurrence.occurrence,
- open: { affinity: occurrence.open.affinity, boundary: semanticBoundary(occurrence.open.boundary) },
- close: { affinity: occurrence.close.affinity, boundary: semanticBoundary(occurrence.close.boundary) },
+ startAnchorId: occurrence.open.boundary.anchorId,
+ endAnchorId: occurrence.close.boundary.anchorId,
})),
} as const;
return canonicalize(content);
@@ -128,22 +221,12 @@ export function narrativeMomentValue(moment: ParsedNarrative["moments"][number])
id: moment.id,
occurrences: moment.occurrences.map((occurrence) => ({
occurrence: occurrence.occurrence,
- affinity: occurrence.affinity,
- boundary: semanticBoundary(occurrence.boundary),
+ anchorId: occurrence.boundary.anchorId,
})),
} as const;
return canonicalize(content);
}
-function semanticBoundary(boundary: {
- readonly tokenIndex: number;
- readonly structuralPosition: number;
- readonly segmentId?: string;
- readonly anchorId: string;
-}): CanonicalValue {
- return canonicalize(boundary);
-}
-
export function narrativeValue(parsed: ParsedNarrative): CanonicalValue {
return canonicalize({
contract: "svml.narrative@1",
@@ -176,22 +259,15 @@ export function narrativeValue(parsed: ParsedNarrative): CanonicalValue {
id: selection.id,
occurrences: selection.occurrences.map((occurrence) => ({
occurrence: occurrence.occurrence,
- open: {
- affinity: occurrence.open.affinity,
- boundary: semanticBoundary(occurrence.open.boundary),
- },
- close: {
- affinity: occurrence.close.affinity,
- boundary: semanticBoundary(occurrence.close.boundary),
- },
+ startAnchorId: occurrence.open.boundary.anchorId,
+ endAnchorId: occurrence.close.boundary.anchorId,
})),
})),
moments: parsed.moments.map((moment) => ({
id: moment.id,
occurrences: moment.occurrences.map((occurrence) => ({
occurrence: occurrence.occurrence,
- affinity: occurrence.affinity,
- boundary: semanticBoundary(occurrence.boundary),
+ anchorId: occurrence.boundary.anchorId,
})),
})),
captionProjection: {
diff --git a/packages/script/src/parser.ts b/packages/script/src/parser.ts
index bba8c509..e9a27727 100644
--- a/packages/script/src/parser.ts
+++ b/packages/script/src/parser.ts
@@ -45,11 +45,11 @@ const RESERVED_SEGMENT_IDS = new Set(["script"]);
/** A marker's structural position before its affinity picks one anchor. */
type RawMarkerBoundary = Omit<MarkerBoundary, "anchorId">;
type RawEdge<T> = Omit<T, "boundary"> & { readonly boundary: RawMarkerBoundary };
-type RawSelectionOccurrence = Omit<ParsedSelectionOccurrence, "open" | "close"> & {
+type RawSelectionOccurrence = Omit<ParsedSelectionOccurrence, "startAnchorId" | "endAnchorId" | "open" | "close"> & {
readonly open: RawEdge<ParsedSelectionOccurrence["open"]>;
readonly close: RawEdge<ParsedSelectionOccurrence["close"]>;
};
-type RawMomentOccurrence = Omit<ParsedMomentOccurrence, "boundary"> & { readonly boundary: RawMarkerBoundary };
+type RawMomentOccurrence = Omit<ParsedMomentOccurrence, "anchorId" | "boundary"> & { readonly boundary: RawMarkerBoundary };
function normalizeWord(value: string): string {
return value
@@ -733,15 +733,24 @@ export function parseScript(
turns,
selections: [...selections].sort(([left], [right]) => left.localeCompare(right)).map(([id, occurrences]) => ({
id,
- occurrences: occurrences.map((occurrence) => ({
- ...occurrence,
- open: anchored(occurrence.open),
- close: anchored(occurrence.close),
- })),
+ occurrences: occurrences.map((occurrence) => {
+ const open = anchored(occurrence.open);
+ const close = anchored(occurrence.close);
+ return {
+ ...occurrence,
+ startAnchorId: open.boundary.anchorId,
+ endAnchorId: close.boundary.anchorId,
+ open,
+ close,
+ };
+ }),
})),
moments: [...moments].sort(([left], [right]) => left.localeCompare(right)).map(([id, occurrences]) => ({
id,
- occurrences: occurrences.map(anchored),
+ occurrences: occurrences.map((occurrence) => {
+ const value = anchored(occurrence);
+ return { ...value, anchorId: value.boundary.anchorId };
+ }),
})),
captionProjection: {
contract: "svml.caption-projection@1",
diff --git a/packages/script/src/surface.ts b/packages/script/src/surface.ts
index 3b2f6cdc..f252db5d 100644
--- a/packages/script/src/surface.ts
+++ b/packages/script/src/surface.ts
@@ -3,6 +3,8 @@ import { canonicalize } from "@narratage/core";
import { ScriptSyntaxError } from "./error.js";
import {
captionProjectionValue,
+ captionSelectionWordSubsetValue,
+ captionWordSequence,
narrativeDialogueExcerptValue,
narrativeSegmentExcerptValue,
narrativeMomentValue,
@@ -13,6 +15,8 @@ import {
} from "./narrative.js";
import {
captionProjectionType,
+ captionWordSequenceType,
+ captionWordSubsetType,
narrativeDialogueExcerptType,
narrativeExcerptType,
narrativeMomentType,
@@ -77,6 +81,8 @@ export function decodeScriptSurface(input: ScriptSurfaceInput): ScriptSurfaceOut
input.source.slice(input.contentStart, close.start),
input.contentStart,
);
+ const wordsId = `${rawId}.caption.words`;
+ const words = captionWordSequence(parsed, wordsId);
return {
nextOffset: close.end,
records: [
@@ -112,6 +118,21 @@ export function decodeScriptSurface(input: ScriptSurfaceInput): ScriptSurfaceOut
value: { kind: "inline" as const, value: captionProjectionValue(parsed) },
range: { start: input.openingStart, end: close.end },
},
+ {
+ id: wordsId,
+ type: captionWordSequenceType,
+ value: { kind: "inline" as const, value: canonicalize(words) },
+ range: { start: input.openingStart, end: close.end },
+ },
+ ...parsed.selections.map((selection) => ({
+ id: `${rawId}.caption.selection.${selection.id}`,
+ type: captionWordSubsetType,
+ value: { kind: "inline" as const, value: captionSelectionWordSubsetValue(words, selection) },
+ range: {
+ start: selection.occurrences[0]!.open.range.start,
+ end: selection.occurrences.at(-1)!.close.range.end,
+ },
+ })),
...parsed.selections.map((selection) => ({
id: `${rawId}.selection.${selection.id}`,
type: narrativeSelectionType,
diff --git a/packages/script/src/types.ts b/packages/script/src/types.ts
index b0a91a6a..29d1bcc5 100644
--- a/packages/script/src/types.ts
+++ b/packages/script/src/types.ts
@@ -1,7 +1,17 @@
import type { CanonicalValue, SourceRange, StoredValue, TypeRef } from "@narratage/protocol";
-import type { CaptionProjection, CaptionRefinement, CaptionRegion, Narrative, NarrativeMoment, NarrativeMomentOccurrence, NarrativeSegment, NarrativeSelection, NarrativeSelectionOccurrence, NarrativeToken, NarrativeTurn } from "@narratage/narrative";
+import type { CaptionProjection, CaptionRefinement, CaptionRegion, Narrative, NarrativeMomentOccurrence, NarrativeSegment, NarrativeSelectionOccurrence, NarrativeToken, NarrativeTurn } from "@narratage/narrative";
-export type { Affinity, CaptionProjection, MarkerBoundary, Narrative, SemanticAnchor } from "@narratage/narrative";
+export type { CaptionProjection, Narrative, SemanticAnchor } from "@narratage/narrative";
+
+export type Affinity = "left" | "right";
+
+/** Script-only source structure. Public Selection/Moment values expose only resolved anchors. */
+export type MarkerBoundary = {
+ readonly tokenIndex: number;
+ readonly structuralPosition: number;
+ readonly segmentId?: string;
+ readonly anchorId: string;
+};
export type ParsedTextAtom = {
readonly kind: "text";
@@ -30,16 +40,22 @@ export type ParsedToken = NarrativeToken & { readonly range: SourceRange };
export type ParsedTurn = NarrativeTurn & { readonly range: SourceRange };
export type ParsedSelectionOccurrence = NarrativeSelectionOccurrence & {
- readonly open: NarrativeSelectionOccurrence["open"] & { readonly range: SourceRange };
- readonly close: NarrativeSelectionOccurrence["close"] & { readonly range: SourceRange };
+ readonly open: { readonly affinity: Affinity; readonly boundary: MarkerBoundary; readonly range: SourceRange };
+ readonly close: { readonly affinity: Affinity; readonly boundary: MarkerBoundary; readonly range: SourceRange };
};
-export type ParsedSelection = Omit<NarrativeSelection, "occurrences"> & {
+export type ParsedSelection = {
+ readonly id: string;
readonly occurrences: readonly ParsedSelectionOccurrence[];
};
-export type ParsedMomentOccurrence = NarrativeMomentOccurrence & { readonly range: SourceRange };
-export type ParsedMoment = Omit<NarrativeMoment, "occurrences"> & {
+export type ParsedMomentOccurrence = NarrativeMomentOccurrence & {
+ readonly affinity: Affinity;
+ readonly boundary: MarkerBoundary;
+ readonly range: SourceRange;
+};
+export type ParsedMoment = {
+ readonly id: string;
readonly occurrences: readonly ParsedMomentOccurrence[];
};
diff --git a/packages/script/test/script.test.ts b/packages/script/test/script.test.ts
index 7275259c..b81571b0 100644
--- a/packages/script/test/script.test.ts
+++ b/packages/script/test/script.test.ts
@@ -3,6 +3,7 @@ import { readFileSync } from "node:fs";
import test from "node:test";
import { digestOf } from "@narratage/core";
+import type { Narrative } from "@narratage/narrative";
import {
ScriptSyntaxError,
formatScript,
@@ -80,6 +81,11 @@ test("selections, moments and Dual Text preserve separate semantic projections",
assert.deepEqual(parsed.selections.map((selection) => selection.id), ["whole"]);
assert.deepEqual(parsed.moments.map((moment) => moment.id), ["beat"]);
assert.equal(parsed.captionProjection.regions.find((region) => region.kind === "alias")?.display, "laughed");
+ const publicNarrative = narrativeValue(parsed) as unknown as Narrative;
+ assert.deepEqual(Object.keys(publicNarrative.selections[0]!.occurrences[0]!).sort(),
+ ["endAnchorId", "occurrence", "startAnchorId"]);
+ assert.deepEqual(Object.keys(publicNarrative.moments[0]!.occurrences[0]!).sort(),
+ ["anchorId", "occurrence"]);
});
test("Caption Projection owns speech ranges without inventing alias word timing", () => {
diff --git a/packages/semantic-map/src/location.ts b/packages/semantic-map/src/location.ts
index 3f6314d0..c100bd9a 100644
--- a/packages/semantic-map/src/location.ts
+++ b/packages/semantic-map/src/location.ts
@@ -71,8 +71,8 @@ export function selectionFrameSpans(
assertProgramSpaceIdentity(programSpace);
const frames = anchorFrames(map);
return selection.occurrences.map((occurrence) => {
- const startFrame = frameFor(frames, occurrence.open.boundary.anchorId, `NarrativeSelection ${selection.id}`);
- const endFrameExclusive = frameFor(frames, occurrence.close.boundary.anchorId, `NarrativeSelection ${selection.id}`);
+ const startFrame = frameFor(frames, occurrence.startAnchorId, `NarrativeSelection ${selection.id}`);
+ const endFrameExclusive = frameFor(frames, occurrence.endAnchorId, `NarrativeSelection ${selection.id}`);
return { startFrame, endFrameExclusive };
});
}
@@ -109,5 +109,5 @@ export function momentFrames(
assertProgramSpaceIdentity(programSpace);
const frames = anchorFrames(map);
return moment.occurrences.map((occurrence) =>
- frameFor(frames, occurrence.boundary.anchorId, `NarrativeMoment ${moment.id}`));
+ frameFor(frames, occurrence.anchorId, `NarrativeMoment ${moment.id}`));
}
diff --git a/packages/semantic-map/test/location.test.ts b/packages/semantic-map/test/location.test.ts
index 64385da7..aada1467 100644
--- a/packages/semantic-map/test/location.test.ts
+++ b/packages/semantic-map/test/location.test.ts
@@ -45,16 +45,14 @@ const map = {
],
} as unknown as CompleteSemanticMap;
-const boundary = (anchorId: string) => ({ tokenIndex: 0, structuralPosition: 0, anchorId });
-
function selection(openAnchor: string, closeAnchor: string): NarrativeSelectionRef {
return {
contract: "svml.narrative-selection@1",
id: "x",
occurrences: [{
occurrence: 0,
- open: { affinity: "right", boundary: boundary(openAnchor) },
- close: { affinity: "left", boundary: boundary(closeAnchor) },
+ startAnchorId: openAnchor,
+ endAnchorId: closeAnchor,
}],
} as NarrativeSelectionRef;
}
@@ -93,8 +91,8 @@ test("a Moment locates one instant per occurrence", () => {
contract: "svml.narrative-moment@1",
id: "reveal",
occurrences: [
- { occurrence: 0, affinity: "left", boundary: boundary("segment:second:start") },
- { occurrence: 1, affinity: "right", boundary: boundary("segment:second:token:2:start") },
+ { occurrence: 0, anchorId: "segment:second:start" },
+ { occurrence: 1, anchorId: "segment:second:token:2:start" },
],
} as NarrativeMomentRef;
assert.deepEqual(momentFrames(map, moment, space), [5, 7]);
@@ -117,13 +115,13 @@ test("overlapping occurrences pass through untouched, in Script order", () => {
occurrences: [
{
occurrence: 0,
- open: { affinity: "right", boundary: boundary("segment:opening:token:1:start") },
- close: { affinity: "left", boundary: boundary("segment:opening:end") },
+ startAnchorId: "segment:opening:token:1:start",
+ endAnchorId: "segment:opening:end",
},
{
occurrence: 1,
- open: { affinity: "right", boundary: boundary("segment:second:token:1:end") },
- close: { affinity: "left", boundary: boundary("segment:second:end") },
+ startAnchorId: "segment:second:token:1:end",
+ endAnchorId: "segment:second:end",
},
],
} as NarrativeSelectionRef;
@@ -143,13 +141,13 @@ test("Script order is preserved even when it runs backwards in time", () => {
occurrences: [
{
occurrence: 0,
- open: { affinity: "right", boundary: boundary("segment:second:token:1:start") },
- close: { affinity: "left", boundary: boundary("segment:second:end") },
+ startAnchorId: "segment:second:token:1:start",
+ endAnchorId: "segment:second:end",
},
{
occurrence: 1,
- open: { affinity: "right", boundary: boundary("segment:opening:token:1:start") },
- close: { affinity: "left", boundary: boundary("segment:opening:end") },
+ startAnchorId: "segment:opening:token:1:start",
+ endAnchorId: "segment:opening:end",
},
],
} as NarrativeSelectionRef;
diff --git a/packages/text-track/test/text-track.test.ts b/packages/text-track/test/text-track.test.ts
index cd88376b..20c380d5 100644
--- a/packages/text-track/test/text-track.test.ts
+++ b/packages/text-track/test/text-track.test.ts
@@ -128,14 +128,8 @@ test("a selected Text Item is located only through explicit Selection and Semant
id: "callout",
occurrences: [{
occurrence: 1,
- open: {
- affinity: "right" as const,
- boundary: { tokenIndex: 1, structuralPosition: 1, segmentId: "opening", anchorId: "segment:opening:token:2:start" },
- },
- close: {
- affinity: "left" as const,
- boundary: { tokenIndex: 2, structuralPosition: 2, segmentId: "opening", anchorId: "segment:opening:token:2:end" },
- },
+ startAnchorId: "segment:opening:token:2:start",
+ endAnchorId: "segment:opening:token:2:end",
}],
};
const selection: NarrativeSelectionRef = selectionContent;
diff --git a/packages/text/test/text.test.ts b/packages/text/test/text.test.ts
index 878084b8..4acba223 100644
--- a/packages/text/test/text.test.ts
+++ b/packages/text/test/text.test.ts
@@ -52,7 +52,7 @@ test("Text learns <script> only from an imported Script Manifest", async () => {
scriptContext(),
);
- assert.equal(result.module.records.length, 5);
+ assert.equal(result.module.records.length, 6);
assert.equal(result.module.records[0]?.id, "story");
assert.equal(result.module.records[0]?.type.name, "Narrative");
assert.equal(result.module.records.some((record) =>
@@ -63,6 +63,8 @@ test("Text learns <script> only from an imported Script Manifest", async () => {
record.id === "story.segment.opening.speech" && record.type.name === "NarrativeSpeechExcerpt"), true);
assert.equal(result.module.records.some((record) =>
record.id === "story.caption" && record.type.name === "CaptionProjection"), true);
+ assert.equal(result.module.records.some((record) =>
+ record.id === "story.caption.words" && record.type.name === "CaptionWordSequence"), true);
assert.equal(result.sourceMaps.length, 1);
assert.doesNotThrow(() => link(scriptContext().closure, [result.module]));
});
diff --git a/packages/video-cli/test/cli.test.ts b/packages/video-cli/test/cli.test.ts
index 53334a40..3e35b6b7 100644
--- a/packages/video-cli/test/cli.test.ts
+++ b/packages/video-cli/test/cli.test.ts
@@ -90,6 +90,10 @@ test("official video CLI checks a real Script source through the Node compiler h
module: { name: "@narratage/narrative", version: "0.0.0-dev" },
name: "CaptionProjection",
}, kind: "record" },
+ { name: "story.caption.words", type: {
+ module: { name: "@narratage/narrative", version: "0.0.0-dev" },
+ name: "CaptionWordSequence",
+ }, kind: "record" },
{ name: "story.segment.opening", type: {
module: { name: "@narratage/narrative", version: "0.0.0-dev" },
name: "NarrativeExcerpt",
@@ -231,7 +235,7 @@ test("one checked-in fixture closes the complete provider-free video plan", asyn
"request-whisperx-alignment",
"request-caption-gemini-plan",
"compile-broll",
- "render-caption-program",
+ "render-fine-caption",
"render-text-track",
"compile-composition",
"request-visual-render",
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index 8aa83864..a2f72246 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -17,6 +17,9 @@ importers:
'@narratage/caption':
specifier: workspace:*
version: link:packages/caption
+ '@narratage/caption-fine':
+ specifier: workspace:*
+ version: link:packages/caption-fine
'@narratage/caption-gemini':
specifier: workspace:*
version: link:packages/caption-gemini
@@ -216,27 +219,18 @@ importers:
'@narratage/component-kit':
specifier: workspace:*
version: link:../component-kit
- '@narratage/composition':
- specifier: workspace:*
- version: link:../composition
'@narratage/elaborator':
specifier: workspace:*
version: link:../elaborator
'@narratage/narrative':
specifier: workspace:*
version: link:../narrative
- '@narratage/program-space':
- specifier: workspace:*
- version: link:../program-space
'@narratage/protocol':
specifier: workspace:*
version: link:../protocol
'@narratage/semantic-map':
specifier: workspace:*
version: link:../semantic-map
- '@narratage/svs':
- specifier: workspace:*
- version: link:../svs
'@narratage/text':
specifier: workspace:*
version: link:../text
@@ -254,6 +248,43 @@ importers:
specifier: workspace:*
version: link:../speech-evidence
+ packages/caption-fine:
+ dependencies:
+ '@narratage/caption':
+ specifier: workspace:*
+ version: link:../caption
+ '@narratage/component-kit':
+ specifier: workspace:*
+ version: link:../component-kit
+ '@narratage/composition':
+ specifier: workspace:*
+ version: link:../composition
+ '@narratage/elaborator':
+ specifier: workspace:*
+ version: link:../elaborator
+ '@narratage/narrative':
+ specifier: workspace:*
+ version: link:../narrative
+ '@narratage/program-space':
+ specifier: workspace:*
+ version: link:../program-space
+ '@narratage/protocol':
+ specifier: workspace:*
+ version: link:../protocol
+ '@narratage/semantic-map':
+ specifier: workspace:*
+ version: link:../semantic-map
+ '@narratage/svs':
+ specifier: workspace:*
+ version: link:../svs
+ '@narratage/text':
+ specifier: workspace:*
+ version: link:../text
+ devDependencies:
+ '@narratage/script':
+ specifier: workspace:*
+ version: link:../script
+
packages/caption-gemini:
dependencies:
'@narratage/caption':
diff --git a/spec/author-surface-binding.md b/spec/author-surface-binding.md
index 0614de6b..4be67467 100644
--- a/spec/author-surface-binding.md
+++ b/spec/author-surface-binding.md
@@ -57,7 +57,7 @@ rewrite an already compiled child SourceUnit or a previous declaration.
For this source:
```svml
-<caption:Track appearance={studio.caption.dialogue}/>
+<caption-fine:Style recipe={studio.caption.dialogue}/>
```
the responsibilities are:
diff --git a/spec/caption-program.md b/spec/caption-program.md
index 5800c3df..acf0b855 100644
--- a/spec/caption-program.md
+++ b/spec/caption-program.md
@@ -1,29 +1,29 @@
# Caption Program
-Status: implemented pre-release contract. Public names may still be polished before package
-publication; the semantic laws below are executable.
+Status: pre-release contract under implementation. Public names may still be polished before
+package publication; the semantic laws below are the target for executable tests.
## 1. Scope
Caption is an ordinary video component family. It owns the transformation from immutable authored
-display text to one self-contained `VisualTrack`. It has no authority over Script wording,
+display words to one self-contained `VisualTrack`. It has no authority over Script wording,
pronunciation, speech alignment, global time, Composition order, Provider selection or Runtime
reuse policy.
The graph has two independent branches:
```text
-Narrative.captionProjection + CaptionProgram ── planner ─────── CaptionPlan
-speech audio ─────────────────────────────────── evidence/map ─ CompleteSemanticMap
+CaptionWordSequence + CaptionProgram ── planner ────── CaptionPlan
+speech audio ────────────────────────────────── evidence/map ─ CompleteSemanticMap
-Narrative + Program + Plan + Map ── temporalize ── TimedCaptionProjection
-TimedCaptionProjection + Program ── render ─────── VisualTrack
+Narrative + Words + Program + Plan + Map ── temporalize ── TimedCaptionProjection
+TimedCaptionProjection + Words + Program ── Style renderer ── VisualTrack
```
The branches meet only at temporalization. Planning contains no timestamps, and alignment contains
no Caption Style or Cue judgment.
-## 2. Display truth
+## 2. Display truth and word subsets
Caption planning sees only the visible left side of Script. For Dual Text:
@@ -34,63 +34,106 @@ Caption planning sees only the visible left side of Script. For Dual Text:
Caption sees `SVML`; speech generation and alignment see `semantic video markup language`. The
right side must never enter a Caption planning request.
-The Caption package deterministically tokenizes every visible `CaptionProjection` region into an
-ordered `CaptionDisplayAtom` universe. Each atom has a stable id, display text, Segment/Turn
-ownership, optional Role and a source speech-token range. An atom's correspondence is either:
+Script deterministically projects every visible `CaptionProjection` region into one explicit,
+ordered `CaptionWordSequence`. This is the complete Caption word universe `W`; it is not a bag of
+strings. Each `CaptionWord` has a stable id and global index, display text and source offsets,
+Segment/Turn ownership, optional Role, and a source speech-token range. A word's correspondence is
+either:
- `exact`: its own source speech-token range is proven;
- `region-envelope`: only the enclosing display region's speech range is proven.
The package does not invent word timing for the second case at this stage.
+Script additionally emits ordered `CaptionWordSubset` values for explicit Selections. Role
+ownership is already explicit on each word, so the Caption author Surface can derive the equivalent
+Role subset directly from the sequence. A subset contains only ids from the exact
+`CaptionWordSequence` and preserves their global order. Selection-to-word projection is a
+Script/source-structure concern: Caption never reads token indices, source ranges or marker affinity
+from a general `NarrativeSelection`.
+
+This creates two deliberately different projections of one authored Selection:
+
+- the general `NarrativeSelection` contains only semantic start/end anchor identities and can be
+ located in time by any consumer;
+- the Caption-specific `CaptionWordSubset` names visible display words wholly owned by that
+ Selection and can be used before time exists.
+
+The two values are not interchangeable and neither is hidden metadata on the other. A Selection
+that cuts only part of an indivisible Dual Text display region cannot become a Caption word subset;
+the author must split the Dual Text atom.
+
## 3. Complete Style intent
-A `CaptionStyle` is a complete author declaration with two independent payloads:
+A `CaptionStyle` is a complete author declaration with two inseparable payloads:
-- `planning`: one Cue instruction and zero or more typed per-atom field declarations;
-- `presentation`: one complete visual appearance, presentation mode and absolute stacking order.
+- `planning`: universal Cue word-count bounds plus zero or more Style-owned typed per-word field
+ declarations and their instructions;
+- `rendering`: the owning Style family identity plus that family's complete, Recipe-resolved
+ parameters.
-Styles are declared separately from where they apply. A field declaration has a stable id, value
-schema, natural-language instruction and minimum/maximum assignment count per Cue. Boolean, enum
-and bounded numeric values are supported. One atom may receive zero, one or multiple independent
-fields; fields need not be contiguous.
+Styles are declared separately from where they apply. The common Caption contract owns Cue
+partitioning and field assignment shapes, but it does not own fonts, colors, three-box layout,
+animation modes or the meaning of `important`. Those belong to a concrete Style family such as
+`@narratage/caption-fine`.
+
+A field declaration has a stable id, value schema, natural-language instruction and
+minimum/maximum assignment count per Cue. Boolean, enum and bounded numeric values are supported.
+One word may receive zero, one or multiple independent fields; fields need not be contiguous.
A Style is a whole value. Applying another Style replaces it completely. The language does not
-implicitly merge arbitrary visual properties, Cue rules or fields from two Styles.
+implicitly merge arbitrary visual properties, Cue rules or fields from two Styles. Planning and
+rendering cannot be selected independently: replacing a Style replaces both. This prevents a
+planner from emitting a field that the selected renderer does not understand.
+
+SVS stores named Recipe data. A Style package parses the Recipe into its own complete parameters
+and emits the common `CaptionStyle` envelope. SVS does not understand Caption fields or rendering.
## 4. Total assignment and ordered replacement
-Every `CaptionProgram` must name one `default` Style. That Style initially owns every visible atom,
-including roleless Turns. The Program then applies zero or more ordered `Use` rules:
+Every `CaptionProgram` names one explicit `default` Style. That Style initially owns every word in
+`W`, including roleless Turns. The word universe itself is the whole set: authors must not write
+`@whole ... @/whole` merely to style all captions, and the compiler must not invent such an authored
+time Selection.
+
+`words={story.caption.words}` is a real graph input, not payload to copy through every downstream
+value. The Program stores the sequence identity and ordered word-id-to-Style runs only. Planner,
+temporalizer and Style renderer each declare their own explicit edge to the same word sequence when
+they need word text or source correspondence.
+
+The Program then applies zero or more ordered whole-Style replacements:
```xml
-<caption:Program id="captions" narrative={story} default={plain}>
+<caption:Program id="captions" words={story.caption.words} default={plain}>
<caption:Use role="ALICE" style={alice}/>
- <caption:Use on={story.selection.special} style={impact}/>
+ <caption:Use words={story.caption.selection.special} style={impact}/>
</caption:Program>
```
-A `Use` selector is either one exact Role label or one explicit Script Selection. It cannot contain
-both. Rules run in source order; for every atom, the last matching rule wins. A rule that selects no
-visible atom is an error. Authors never spell a complement merely to preserve the default.
+An author Surface may offer `role="ALICE"` as sugar, but it must resolve to the explicit Role word
+subset before the Program is constructed. It is not a temporal Selection and it introduces no
+fourth `when` language.
-Role is optional per Script Turn and resets at every Segment boundary. A roleless Turn therefore
-receives the default unless an explicit Selection replaces it.
+Rules run in source order; for every word, the last matching rule wins. A rule that selects no
+visible word is an error. Role is optional per Script Turn and resets at every Segment boundary. A
+roleless Turn therefore receives the default unless an explicit word subset replaces it. Authors
+never spell a complement merely to preserve the default.
-After assignment, the Program deterministically creates maximal contiguous runs. A run never
-crosses a Segment or Turn boundary and every atom in it has one Style. Runs are disjoint, ordered and
-partition the complete display universe exactly once.
+After assignment, one deterministic resolver creates maximal contiguous runs. A run never crosses
+a Segment or Turn boundary and every word in it has one Style. Runs are disjoint, ordered and
+partition the complete display universe exactly once. It does not create one graph node per Style.
## 5. Planner contract
A planner consumes resolved runs rather than interpreting Role, Selection or Style precedence. For
each run it may only:
-1. split the atom sequence into one or more ordered Cues;
-2. assign declared fields to atoms inside each Cue.
+1. split the word sequence into one or more ordered Cues, respecting the Style's common minimum and
+ maximum word count;
+2. assign declared fields to words inside each Cue.
The result must contain every Program run exactly once. Cues must preserve order and partition the
-run's atoms exactly once. Field ids, atom ids, value schemas and per-Cue cardinalities fail closed.
+run's words exactly once. Field ids, word ids, value schemas and per-Cue cardinalities fail closed.
There is no response field for text, pronunciation, time or Style.
`@narratage/caption-gemini` is one explicit implementation of this contract. It is not part of the
@@ -99,20 +142,21 @@ Gemini, but the author graph must explicitly choose them.
## 6. Timing projection
-`CompleteSemanticMap` remains the only global speech timing truth. Exact display atoms reuse their
-source token windows. For a `region-envelope` alias, a Caption renderer may allocate local windows
-inside that region's envelope. Those local boundaries remain downstream Caption facts and are never
-written back into the global map.
+`CompleteSemanticMap` remains the only global speech timing truth. Exact display words reuse their
+source token windows. For a `region-envelope` alias, a Caption temporalizer may allocate local
+windows inside that region's envelope. Those local boundaries remain downstream Caption facts and
+are never written back into the global map.
Multiple Cues inside one alias must receive ordered, non-overlapping local windows. They must not
each copy the same whole region envelope.
## 7. Rendering and composition
-Each timed Cue selects the full presentation payload of its resolved Style. The renderer may expose
-declared field values to its own visual lowering, but cannot sample or mutate another Track. The
-result is exactly one ordinary `VisualTrack` whose Presents carry their own frame spans and absolute
-stacking facts.
+Each timed Cue selects the full rendering payload of its resolved Style. One Style-family renderer
+consumes all runs of that family, including the default and every override, and returns exactly one
+ordinary `VisualTrack`. The renderer may interpret its own declared fields, but cannot sample or
+mutate another Track. In v1 every Style in one Program must name the same rendering family; mixing
+families requires separate Programs/Tracks until a real cross-family composition contract exists.
Film does not recognize Caption specially. It folds the returned Track alongside Speech, B-roll,
Text or any third-party Track satisfying the same public contract.
@@ -127,7 +171,8 @@ Caption Program v1 does not define:
- automatic cache reuse or prompt-based memoization;
- hidden Runtime fallback;
- a global Caption layer in Composition;
-- field-specific visual semantics in Core.
+- field-specific visual semantics in Core;
+- a condition language parallel to Selection, Moment or Caption word subsets.
Reuse, Existing Values and substitutes are selected by the external BuildRequest/realization
closure. They do not change Caption data or planner semantics.
diff --git a/spec/script-surface.md b/spec/script-surface.md
index b01407c6..16b7e653 100644
--- a/spec/script-surface.md
+++ b/spec/script-surface.md
@@ -156,10 +156,10 @@ B: It’s 8:30.
```
在 `speech` 和 `caption` 投影中,`<A>`、`<B>` 都被移除。Role Cue 不是
-speaker 数据模型,不建立人物实体,不选择音色,不成为可接线字段,也不对
-字幕隐式分组。外部 Program 可以显式写 `role="A"` 查询这些 spoken turn;
-Compiler 将结果降低为一个可非连通的派生 SelectionSet。仅有 `<A>` 本身不触发
-任何样式、人物、音色或素材行为。
+speaker 数据模型,不建立人物实体,不选择音色,也不对字幕隐式分组。外部
+Program 可以显式写 `role="A"` 查询这些 spoken turn。Caption Program 将这类
+查询直接降低为 `CaptionWordSubset`;它不是时间 Selection,也不伪造 Selection
+Record。仅有 `<A>` 本身不触发任何样式、人物、音色或素材行为。
为消除歧义:
@@ -223,16 +223,18 @@ Dual Text 只表达“实际说了什么”。IPA、SSML、重音、语速、语
## 4. Selection
-Selection 是区间声明,不是供下游自行拼接的两个公开锚点:
+Selection 是由作者显式闭合的语义区间声明:
```svml
@id ... @/id
~@id ... @/id~
```
-每个开始标记都必须有同 id 的结束标记。编译器输出一个完整
-`SelectionSet`,消费者只接收该集合中的闭合 range;不存在 start-only
-对象、消费者默认补尾或“信息传了一半”的状态。
+每个开始标记都必须有同 id 的结束标记。Script 的公开值为每个 occurrence
+输出已经解析好的 `startAnchorId` 与 `endAnchorId`;不存在 start-only 对象、
+消费者默认补尾或“信息传了一半”的状态。左右 affinity、token 下标和源码范围
+属于 Parser/Source Map,不泄漏到公开 Selection。需要物理时间的下游必须把这
+两个身份与显式连接的 `CompleteSemanticMap` 投影为区间。
“闭合”指源码的两个端点都被明确声明。物理运行时使用半开
`[startFrame, endFrameExclusive)` Frame Span。
@@ -273,7 +275,7 @@ before ~@x hello @/x~ after
### 4.2 闭合、交叉与非连通
-同一个 id 可以顺序出现多次,编译为一个非连通 SelectionSet:
+同一个 id 可以顺序出现多次,编译为一个含多个独立锚点对的非连通 Selection:
```svml
@beat one @/beat ... @beat three @/beat
@@ -376,7 +378,7 @@ gap B.start > A.end
若未来需要真正重排或并行 speech,应发布显式的非线性叙事模型;v1 不让普通
Range 在不同 Locator 下反向或消失。
-同一份 SelectionSet 在不同 fulfillment 产生的 `CompleteSemanticMap` 上可得到
+同一份 Selection 在不同 fulfillment 产生的 `CompleteSemanticMap` 上可得到
预览或成片区间。Map 使用相同 identity。Script 本身不含秒数、帧号或采样点。
Script Surface 不绑定帧率、采样率或渲染器。后端一旦选择物理时钟,必须只
量化一次并让所有消费者复用同一整数边界;后端时钟变化不改变本语言表面。
@@ -419,8 +421,8 @@ overlap 或 gap 落在不同点。
规范规则:
-- 同 id 的 `@id!` / `~@id!` 可以重复,按源码顺序编译成一个
- `MomentSet`;一次 occurrence 是单 Moment,多次只是同一集合含多个点。
+- 同 id 的 `@id!` / `~@id!` 可以重复,按源码顺序编译成一个 Moment;每个
+ occurrence 的公开值只有一个已解析的 `anchorId`。
- Selection 与 Moment 共享 temporal name namespace。同一个 id 不能同时
声明两种类型;`@x! ... @/x` 必须以类型冲突失败。
- 单独的 `@x` 永远是未闭合 Selection open,必须报错;解析器不得因为没有
@@ -432,14 +434,14 @@ overlap 或 gap 落在不同点。
Dual Text 的 speech 侧,但均不得切入 v1 speech token。
- Moment 只选择 Semantic Anchor Index 中已有的一个候选点,不新增 anchor
identity;因此 `2M + 2N` 的精确计数不因 Moment 数量改变。
-- MomentSet 与 SelectionSet 复用同一份 CompleteSemanticMap 和一次性
+- Moment 与 Selection 复用同一份 CompleteSemanticMap 和一次性
帧量化;区别只在最终载体是 `Point[]` 而不是 `Range[]`。
消费端保持两个互不相混的类型:
```text
-SelectionSet = Range[]
-MomentSet = Point[]
+NarrativeSelection = { startAnchorId, endAnchorId }[]
+NarrativeMoment = { anchorId }[]
```
推荐的显式使用关系是:
@@ -540,30 +542,33 @@ temporal marker、Role Cue 或结构标签内部。注释可出现在 atom 之
token range 和 source range;
- plain / Dual Text / Slot atom 及其源码范围;
- caption atom 到一个或多个 speech token 的显式映射;
-- 每个 SelectionSet 的一个或多个有序 occurrence;
-- 每个端点的左右 affinity 和端点源码位置。
-- 每个 MomentSet 的一个或多个有序 point、左右 affinity 和源码位置。
-
-外部 Program 的 `role="label"` selector 可以从上述 Turn 范围投影自己的选择集合;
-同一 label 的多次 Turn 是多个有序、互不相邻的命中。它是消费组件内部的显式查询,
-不是 Role Cue 的隐式字幕行为,也不建立 speaker entity。当前 Caption Program 直接把
-该查询投影到可见 Display Atom,不伪造一个作者写下的 Selection Record。
+- 每个 Selection 的一个或多个有序 occurrence,以及各 occurrence 已解析的
+ `startAnchorId` / `endAnchorId`;
+- 每个 Moment 的一个或多个有序 occurrence,以及各 occurrence 已解析的
+ `anchorId`;
+- Parser/Source Map 内部另行保留 affinity、token/source range,公开值不携带它们;
+- 完整有序的 `CaptionWordSequence`,以及每个显式 Selection 对应的
+ `CaptionWordSubset`。字幕无需也不得从公开 Selection 反推词下标。
+
+外部 Program 的 `role="label"` selector 可以从上述词全集投影自己的词子集;
+同一 label 的多次 Turn 是多个有序、互不相邻的命中。它是 Caption author surface
+内部的显式查询,不是 Role Cue 的隐式字幕行为,也不建立 speaker entity。
SelectionSet/MomentSet 是消费者边界,不是 Segment 的子对象。典型外部
关系是:
```text
Script ──compile──> Narrative IR ──project──> dialogue / speech / caption
- │
- └──resolve──> SelectionSet[] / MomentSet[]
+ ├──resolve──> NarrativeSelection[] / NarrativeMoment[]
+ └──project──> CaptionWordSequence / CaptionWordSubset[]
-SelectionSet A ──during──> consumer.port_1
-MomentSet P ──at──────> consumer.port_2
-SelectionSet C ──during──> consumer.item_3
+NarrativeSelection A ──during──> consumer.port_1
+NarrativeMoment P ──at──────> consumer.port_2
+CaptionWordSubset C ──words───> caption.Program
```
一个 Ranking、B-roll 或其他节点可以有多个动态端口,各端口接受不同
-SelectionSet/MomentSet;同一集合也可以被多个消费者复用。未来 Graph
+Selection/Moment;同一集合也可以被多个消费者复用。未来 Graph
语法可以把这些类型边写成 `during=` / `at=`,但它们必须引用已解析的名字
或端点投影,不能接受模糊自然语言 locator。
diff --git a/tsconfig.json b/tsconfig.json
index b0dbc6a2..a637e9a6 100644
--- a/tsconfig.json
+++ b/tsconfig.json
@@ -72,6 +72,9 @@
"@narratage/caption": [
"packages/caption/src/index.ts"
],
+ "@narratage/caption-fine": [
+ "packages/caption-fine/src/index.ts"
+ ],
"@narratage/speech-basis": [
"packages/speech-basis/src/index.ts"
],