Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

refactor: make caption planning atom-safe

kashorina3114f992b4c
diff --git a/docs/guide/author-packages.md b/docs/guide/author-packages.md
index d67c12e5..6676e322 100644
--- a/docs/guide/author-packages.md
+++ b/docs/guide/author-packages.md
@@ -80,7 +80,7 @@ export const decodeMyComponentSurface: TextSurfaceDecoder = (element, context) =
Look at existing Surface implementations for reference:
- `packages/seedance/src/surface.ts` — Prompt, Speech and Video Surfaces
-- `packages/caption/src/surface.ts` — Style, Program and Track Surfaces
+- `packages/caption/src/surface.ts` — the common Program Surface; concrete Style/Track Surfaces live in Style-family packages
- `packages/broll/src/surface.ts` — Track and Item Surfaces
## 5. Write the activation descriptor
@@ -157,7 +157,8 @@ authority.
|---|---|
| `packages/seedance/` | Model family with multiple Surfaces (Prompt, Speech, Video) |
| `packages/seedance-speaker/` | Higher-level binding that composes Script, Prompt Kit and Seedance |
-| `packages/caption/` | Style, Program and Track Surfaces with typed field declarations |
+| `packages/caption/` | common Program, Cue/field contracts and whole-Atom timing |
+| `packages/caption-fine/` | one field-free Style and Track Surface family |
| `packages/broll/` | Track with Item/transition behavior |
| `packages/text-track/` | Simple text overlay Track |
| `packages/film/` | Composition target that consumes peer Tracks |
diff --git a/docs/quickstart/composition.md b/docs/quickstart/composition.md
index adb07e57..c4a55756 100644
--- a/docs/quickstart/composition.md
+++ b/docs/quickstart/composition.md
@@ -145,11 +145,11 @@ The complete data flow from Script to rendered video. This example is based on
<!-- 4. Tracks: captions, B-roll, text -->
<caption-fine:Style id="base-caption" recipe={studio.caption.base}/>
- <caption:Program id="caption-program" words={story.caption.words}
+ <caption:Program id="caption-program" display={story.caption}
default={base-caption}/>
- <caption-ai:Planner id="cue-plan" words={story.caption.words}
+ <caption-ai:Planner id="cue-plan" display={story.caption}
program={caption-program} model="gemini-2.5-flash"/>
- <caption-fine:Track id="captions" narrative={story} words={story.caption.words} map={timing.map}
+ <caption-fine:Track id="captions" display={story.caption} correspondence={story.caption.correspondence} map={timing.map}
space={speech.space} plan={cue-plan.plan} program={caption-program}/>
<broll:Track id="cards" map={timing.map} space={speech.space}>
@@ -194,8 +194,6 @@ The complete data flow from Script to rendered video. This example is based on
}
caption.base {
cue-min-words: 1; cue-max-words: 5;
- important-min-per-cue: 0; important-max-per-cue: 2;
- important-fill: #FFF16A; important-scale: 1.12;
stack-order: 70; x: 0.08; y: 0.76; width: 0.84;
font: Inter; weight: 600; size: 58; line-height: 1; align: center;
fill: #FFFFFF; background: #09090BCC; padding: 16 24; radius: 18;
diff --git a/docs/quickstart/script.md b/docs/quickstart/script.md
index be4d0a70..8545bb16 100644
--- a/docs/quickstart/script.md
+++ b/docs/quickstart/script.md
@@ -79,7 +79,8 @@ Role Cues produce different text projections:
The dialogue projection includes Role Cue prefixes. Speech and caption projections strip them.
Components like `seedance:Speech` use `{story.segment.dialogue.dialogue}` (with labels). Script also
-emits the complete ordered `{story.caption.words}` universe used by Caption Programs and Planners.
+emits `{story.caption}` as the ordered display Atom/Word universe and
+`{story.caption.correspondence}` as the separate Atom-to-speech edge used only when timing joins.
## Dual Text
diff --git a/docs/quickstart/styles.md b/docs/quickstart/styles.md
index 05d69711..5ea30d86 100644
--- a/docs/quickstart/styles.md
+++ b/docs/quickstart/styles.md
@@ -81,10 +81,6 @@ The first official Caption Style family keeps planning and rendering parameters
caption.dialogue {
cue-min-words: 2;
cue-max-words: 7;
- important-min-per-cue: 0;
- important-max-per-cue: 2;
- important-fill: #FFD166;
- important-scale: 1.08;
stack-order: 70;
x: 0.08;
y: 0.76;
@@ -104,8 +100,6 @@ caption.dialogue {
| Property | Description |
|---|---|
| `cue-min-words`, `cue-max-words` | Common Cue word-count bounds |
-| `important-min-per-cue`, `important-max-per-cue` | Fine's per-Cue emphasis cardinality; max `0` disables the field |
-| `important-fill`, `important-scale` | Fine's rendering of words assigned the `important` field |
| `stack-order` | Z-stacking order among all Tracks (higher = on top) |
| `x`, `y` | Position as fraction of canvas (0–1) |
| `width` | Width as fraction of canvas |
@@ -132,8 +126,6 @@ Define multiple caption Recipes for different speakers:
```svs
caption.alice {
cue-min-words: 2; cue-max-words: 5;
- important-min-per-cue: 0; important-max-per-cue: 2;
- important-fill: #FFFFFF; important-scale: 1.08;
stack-order: 70;
x: 0.08; y: 0.76; width: 0.84;
font: Inter; weight: 600; size: 58;
@@ -144,8 +136,6 @@ caption.alice {
caption.bob {
cue-min-words: 2; cue-max-words: 5;
- important-min-per-cue: 0; important-max-per-cue: 0;
- important-fill: #FFFFFF; important-scale: 1;
stack-order: 70;
x: 0.08; y: 0.76; width: 0.84;
font: Inter; weight: 600; size: 58;
@@ -161,7 +151,7 @@ Then assign them via `caption:Program`:
<caption-fine:Style id="default-caption" recipe={studio.caption.dialogue}/>
<caption-fine:Style id="alice-caption" recipe={studio.caption.alice}/>
<caption-fine:Style id="bob-caption" recipe={studio.caption.bob}/>
-<caption:Program id="caption-program" words={story.caption.words} default={default-caption}>
+<caption:Program id="caption-program" display={story.caption} default={default-caption}>
<caption:Use role="ALICE" style={alice-caption}/>
<caption:Use role="BOB" style={bob-caption}/>
</caption:Program>
@@ -387,10 +377,6 @@ A complete `studio.svs` file for a four-take talking-head project:
caption.primary {
cue-min-words: 2;
cue-max-words: 5;
- important-min-per-cue: 0;
- important-max-per-cue: 2;
- important-fill: #FFF16A;
- important-scale: 1.08;
stack-order: 70;
x: 0.08;
y: 0.74;
diff --git a/docs/quickstart/tracks.md b/docs/quickstart/tracks.md
index bc936673..0ccdb749 100644
--- a/docs/quickstart/tracks.md
+++ b/docs/quickstart/tracks.md
@@ -14,7 +14,7 @@ covers the three main visual Track types: captions, B-roll, and text overlays.
Caption uses a small common language and a replaceable Style family:
```text
-Script words → Style family → Caption Program → Planner → Style-family Track
+Script Display → Caption Program → Planner + measured Atom timing → Style-family Track
```
```svml
@@ -23,9 +23,9 @@ Script words → Style family → Caption Program → Planner → Style-family T
<import as="caption-ai" from="@narratage/caption-gemini@1"/>
```
-`@narratage/caption` owns only common Cue bounds, generic per-word fields, total Style assignment,
-Plan validation and the timing join. `@narratage/caption-fine` is one concrete Style family: it owns
-the `important` field, font/box parameters and visual rendering.
+`@narratage/caption` owns only common Cue bounds, generic optional per-Word fields, total Style
+assignment, Plan validation and the timing join. `@narratage/caption-fine` is one field-free Style
+family: it owns its font/box parameters and uniform Cue rendering.
### caption-fine:Style
@@ -36,10 +36,6 @@ both from one package-owned SVS Recipe:
caption.primary {
cue-min-words: 2;
cue-max-words: 7;
- important-min-per-cue: 1;
- important-max-per-cue: 2;
- important-fill: #FFD166;
- important-scale: 1.08;
stack-order: 70;
x: 0.08; y: 0.76; width: 0.84;
font: Inter; weight: 600; size: 58; line-height: 0.96;
@@ -62,7 +58,7 @@ default Style covers every word; no `@whole` Selection or complement is required
rules replace the whole Style on a Role or explicit Caption word subset, with the last match winning.
```svml
-<caption:Program id="caption-program" words={story.caption.words}
+<caption:Program id="caption-program" display={story.caption}
default={primary-caption}>
<caption:Use role="ALICE" style={alice-caption}/>
<caption:Use role="BOB" style={bob-caption}/>
@@ -78,18 +74,19 @@ pair of semantic anchors. Partial ownership of an indivisible Dual Text display
### caption-ai:Planner
```svml
-<caption-ai:Planner id="caption-plan" words={story.caption.words}
+<caption-ai:Planner id="caption-plan" display={story.caption}
program={caption-program} model="gemini-2.5-flash"/>
```
-The planner receives immutable display words and already-resolved Style runs. It may only cut each
-run into ordered Cues and attach declared fields to word ids. It cannot rewrite text, select Styles,
-see audio, or invent time. Its output is `{caption-plan.plan}`.
+The planner receives immutable display Atoms/Words and already-resolved Style runs. It may only cut
+each run between whole Atoms and attach declared fields to Word ids. Fine declares no fields. The
+planner cannot rewrite text, select Styles, see audio or invent time. Its output is
+`{caption-plan.plan}`.
### caption-fine:Track
```svml
-<caption-fine:Track id="captions" narrative={story} words={story.caption.words} map={timing.map}
+<caption-fine:Track id="captions" display={story.caption} correspondence={story.caption.correspondence} map={timing.map}
space={speech.space} program={caption-program} plan={caption-plan.plan}/>
```
@@ -245,10 +242,10 @@ All three track types together in one source file:
<!-- Captions: primary style for all text -->
<caption-fine:Style id="base-caption" recipe={studio.caption.base}/>
-<caption:Program id="caption-program" words={story.caption.words} default={base-caption}/>
-<caption-ai:Planner id="cue-plan" words={story.caption.words}
+<caption:Program id="caption-program" display={story.caption} default={base-caption}/>
+<caption-ai:Planner id="cue-plan" display={story.caption}
program={caption-program} model="gemini-2.5-flash"/>
-<caption-fine:Track id="captions" narrative={story} words={story.caption.words} map={timing.map}
+<caption-fine:Track id="captions" display={story.caption} correspondence={story.caption.correspondence} map={timing.map}
space={speech.space} plan={cue-plan.plan} program={caption-program}/>
<!-- B-roll: generated video during a Selection -->
diff --git a/docs/zh/guide/author-packages.md b/docs/zh/guide/author-packages.md
index e28921c2..15dc3d35 100644
--- a/docs/zh/guide/author-packages.md
+++ b/docs/zh/guide/author-packages.md
@@ -76,7 +76,7 @@ export const decodeMyComponentSurface: TextSurfaceDecoder = (element, context) =
可以参考已有的 Surface 实现:
- `packages/seedance/src/surface.ts` —— Prompt、Speech 和 Video Surface
-- `packages/caption/src/surface.ts` —— Style、Program 和 Track Surface
+- `packages/caption/src/surface.ts` —— 公共 Program Surface;具体 Style/Track Surface 属于各样式族包
- `packages/broll/src/surface.ts` —— Track 和 Item Surface
## 5. 编写 activation 描述符
@@ -151,7 +151,8 @@ pnpm narratage lock-packages <lock-file> \
|---|---|
| `packages/seedance/` | 带多个 Surface(Prompt、Speech、Video)的模型族 |
| `packages/seedance-speaker/` | 组合 Script、Prompt Kit 和 Seedance 的更高层绑定 |
-| `packages/caption/` | 带类型化字段声明的 Style、Program 和 Track Surface |
+| `packages/caption/` | 公共 Program、Cue/字段合同和整 Atom 定时 |
+| `packages/caption-fine/` | 一种无字段的 Style 与 Track Surface 样式族 |
| `packages/broll/` | 带 Item/转场行为的 Track |
| `packages/text-track/` | 简单的文字叠加 Track |
| `packages/film/` | 消费同级 Track 的合成 target |
diff --git a/docs/zh/quickstart/composition.md b/docs/zh/quickstart/composition.md
index ec20de93..8fd1a057 100644
--- a/docs/zh/quickstart/composition.md
+++ b/docs/zh/quickstart/composition.md
@@ -142,11 +142,11 @@ Track 是**扁平的**——没有嵌套或分组。Z 轴排序完全由每个 T
<!-- 4. Tracks: captions, B-roll, text -->
<caption-fine:Style id="base-caption" recipe={studio.caption.base}/>
- <caption:Program id="caption-program" words={story.caption.words}
+ <caption:Program id="caption-program" display={story.caption}
default={base-caption}/>
- <caption-ai:Planner id="cue-plan" words={story.caption.words}
+ <caption-ai:Planner id="cue-plan" display={story.caption}
program={caption-program} model="gemini-2.5-flash"/>
- <caption-fine:Track id="captions" narrative={story} words={story.caption.words} map={timing.map}
+ <caption-fine:Track id="captions" display={story.caption} correspondence={story.caption.correspondence} map={timing.map}
space={speech.space} plan={cue-plan.plan} program={caption-program}/>
<broll:Track id="cards" map={timing.map} space={speech.space}>
@@ -191,8 +191,6 @@ Track 是**扁平的**——没有嵌套或分组。Z 轴排序完全由每个 T
}
caption.base {
cue-min-words: 1; cue-max-words: 5;
- important-min-per-cue: 0; important-max-per-cue: 2;
- important-fill: #FFF16A; important-scale: 1.12;
stack-order: 70; x: 0.08; y: 0.76; width: 0.84;
font: Inter; weight: 600; size: 58; line-height: 1; align: center;
fill: #FFFFFF; background: #09090BCC; padding: 16 24; radius: 18;
diff --git a/docs/zh/quickstart/script.md b/docs/zh/quickstart/script.md
index d8250c5e..48995c65 100644
--- a/docs/zh/quickstart/script.md
+++ b/docs/zh/quickstart/script.md
@@ -70,7 +70,7 @@ Role Cue 会产生不同的文本投影:
| **speech** | `What time is it?`<br>`It's 8:30.` |
| **caption** | `What time is it?`<br>`It's 8:30.` |
-dialogue 投影包含 Role Cue 前缀,speech 和 caption 投影会去除前缀。像 `seedance:Speech` 这样的组件使用 `{story.segment.dialogue.dialogue}`(带标签)。Script 还会显式输出完整有序的 `{story.caption.words}` 词全集,供 Caption Program 和 Planner 使用。
+dialogue 投影包含 Role Cue 前缀,speech 和 caption 投影会去除前缀。像 `seedance:Speech` 这样的组件使用 `{story.segment.dialogue.dialogue}`(带标签)。Script 还会显式输出 `{story.caption}` 作为有序的显示 Atom/Word 全集,并以 `{story.caption.correspondence}` 单独提供 Atom 到口播 token 的对应;只有定时汇合处需要后者。
## Dual Text
diff --git a/docs/zh/quickstart/styles.md b/docs/zh/quickstart/styles.md
index fee08037..232f4562 100644
--- a/docs/zh/quickstart/styles.md
+++ b/docs/zh/quickstart/styles.md
@@ -77,10 +77,6 @@ film.vertical {
caption.dialogue {
cue-min-words: 2;
cue-max-words: 7;
- important-min-per-cue: 0;
- important-max-per-cue: 2;
- important-fill: #FFD166;
- important-scale: 1.08;
stack-order: 70;
x: 0.08;
y: 0.76;
@@ -100,8 +96,6 @@ caption.dialogue {
| 属性 | 描述 |
|---|---|
| `cue-min-words`、`cue-max-words` | 通用 Cue 字数边界 |
-| `important-min-per-cue`、`important-max-per-cue` | Fine 每个 Cue 的强调数量;最大值为 `0` 时禁用该字段 |
-| `important-fill`、`important-scale` | Fine 对 `important` 词的渲染方式 |
| `stack-order` | 所有 Track 之间的 Z 轴层叠顺序(值越大越靠前) |
| `x`、`y` | 位置,以画布比例表示(0–1) |
| `width` | 宽度,以画布比例表示 |
@@ -128,8 +122,6 @@ caption.dialogue {
```svs
caption.alice {
cue-min-words: 2; cue-max-words: 5;
- important-min-per-cue: 0; important-max-per-cue: 2;
- important-fill: #FFFFFF; important-scale: 1.08;
stack-order: 70;
x: 0.08; y: 0.76; width: 0.84;
font: Inter; weight: 600; size: 58;
@@ -140,8 +132,6 @@ caption.alice {
caption.bob {
cue-min-words: 2; cue-max-words: 5;
- important-min-per-cue: 0; important-max-per-cue: 0;
- important-fill: #FFFFFF; important-scale: 1;
stack-order: 70;
x: 0.08; y: 0.76; width: 0.84;
font: Inter; weight: 600; size: 58;
@@ -157,7 +147,7 @@ caption.bob {
<caption-fine:Style id="default-caption" recipe={studio.caption.dialogue}/>
<caption-fine:Style id="alice-caption" recipe={studio.caption.alice}/>
<caption-fine:Style id="bob-caption" recipe={studio.caption.bob}/>
-<caption:Program id="caption-program" words={story.caption.words} default={default-caption}>
+<caption:Program id="caption-program" display={story.caption} default={default-caption}>
<caption:Use role="ALICE" style={alice-caption}/>
<caption:Use role="BOB" style={bob-caption}/>
</caption:Program>
@@ -381,10 +371,6 @@ caption.dialogue {
caption.primary {
cue-min-words: 2;
cue-max-words: 5;
- important-min-per-cue: 0;
- important-max-per-cue: 2;
- important-fill: #FFF16A;
- important-scale: 1.08;
stack-order: 70;
x: 0.08;
y: 0.74;
diff --git a/docs/zh/quickstart/tracks.md b/docs/zh/quickstart/tracks.md
index 1dde2aab..397eb78a 100644
--- a/docs/zh/quickstart/tracks.md
+++ b/docs/zh/quickstart/tracks.md
@@ -12,7 +12,7 @@ description: 视觉 Track 组件——字幕、B-roll 叠加层和文字叠加
字幕由一套很小的公共语言与可替换的样式族组成:
```text
-Script 词全集 → 样式族 → Caption Program → Planner → 样式族 Track
+Script 显示全集 → Caption Program → Planner + Atom 实测时间 → 样式族 Track
```
```svml
@@ -21,8 +21,8 @@ Script 词全集 → 样式族 → Caption Program → Planner → 样式族 Tra
<import as="caption-ai" from="@narratage/caption-gemini@1"/>
```
-公共 Caption 只负责 Cue 字数边界、通用逐词字段、完整样式分配、Plan 校验与时间
-拼接。Fine 是第一种具体样式族,负责 `important` 字段、字体/框参数和视觉渲染。
+公共 Caption 只负责 Cue 字数边界、可选的通用逐词字段、完整样式分配、Plan 校验与时间
+拼接。Fine 是第一种无字段样式族,负责自己的字体/框参数和统一 Cue 视觉渲染。
### caption-fine:Style
@@ -33,9 +33,8 @@ Script 词全集 → 样式族 → Caption Program → Planner → 样式族 Tra
<caption-fine:Style id="primary-caption" recipe={studio.caption.primary}/>
```
-Recipe 同时包含 `cue-min-words`、`cue-max-words`、Fine 自有的
-`important-*-per-cue`、强调外观和完整字体/框参数。其他字幕包可以定义完全不同
-的字段和渲染方式,无需修改公共 Caption。
+Recipe 同时包含 `cue-min-words`、`cue-max-words` 和完整字体/框参数。Fine 不声明任何
+逐词字段;其他字幕包可以定义完全不同的字段和渲染方式,无需修改公共 Caption。
### caption:Program
@@ -44,7 +43,7 @@ Program 消费 Script 显式输出的完整有序显示词全集。一个必填
者获胜。
```svml
-<caption:Program id="caption-program" words={story.caption.words}
+<caption:Program id="caption-program" display={story.caption}
default={primary-caption}>
<caption:Use role="ALICE" style={alice-caption}/>
<caption:Use role="BOB" style={bob-caption}/>
@@ -59,17 +58,18 @@ Program 消费 Script 显式输出的完整有序显示词全集。一个必填
### caption-ai:Planner
```svml
-<caption-ai:Planner id="caption-plan" words={story.caption.words}
+<caption-ai:Planner id="caption-plan" display={story.caption}
program={caption-program} model="gemini-2.5-flash"/>
```
-Planner 只接收不可改写的显示词与已解析好的 Style runs。它只能分 Cue,并给词 id
-附上样式声明的字段;看不到音频、时间或 Dual Text 右侧。
+Planner 只接收不可改写的显示 Atom/Word 与已解析好的 Style runs。它只能在完整 Atom
+之间分 Cue,并给 Word id 附上样式声明的字段;Fine 没有字段。它看不到音频、时间或
+Dual Text 右侧。
### caption-fine:Track
```svml
-<caption-fine:Track id="captions" narrative={story} words={story.caption.words} map={timing.map}
+<caption-fine:Track id="captions" display={story.caption} correspondence={story.caption.correspondence} map={timing.map}
space={speech.space} program={caption-program} plan={caption-plan.plan}/>
```
@@ -222,10 +222,10 @@ B-roll 搭配生成的 Seedance 视频,在 Script Selection 期间出现:
<!-- Captions: primary style for all text -->
<caption-fine:Style id="base-caption" recipe={studio.caption.base}/>
-<caption:Program id="caption-program" words={story.caption.words} default={base-caption}/>
-<caption-ai:Planner id="cue-plan" words={story.caption.words}
+<caption:Program id="caption-program" display={story.caption} default={base-caption}/>
+<caption-ai:Planner id="cue-plan" display={story.caption}
program={caption-program} model="gemini-2.5-flash"/>
-<caption-fine:Track id="captions" narrative={story} words={story.caption.words} map={timing.map}
+<caption-fine:Track id="captions" display={story.caption} correspondence={story.caption.correspondence} map={timing.map}
space={speech.space} plan={cue-plan.plan} program={caption-program}/>
<!-- B-roll: generated video during a Selection -->
diff --git a/examples/talking-film-golden/README.md b/examples/talking-film-golden/README.md
index fd5a009d..d84b260a 100644
--- a/examples/talking-film-golden/README.md
+++ b/examples/talking-film-golden/README.md
@@ -75,7 +75,7 @@ Candidates and Operations.
| `seedance:Video model="mini"` | `@narratage/seedance` | explicit Seedance Mini video Need plus primary-video projection | implemented |
| `broll:Track` | `@narratage/broll` | semantic windows + normalized media + recipe -> peer Visual/Audio Tracks | Surface and deterministic lowering implemented |
| `caption-fine:Style` / `caption:Program` | Fine + common Caption | explicit default over all words + ordered whole-Style replacement by Role or word subset | implemented |
-| `caption-ai:Planner` | `@narratage/caption-gemini` | immutable display words + per-run requirements -> Cue cuts and per-word fields | implemented; Google Vertex Endpoint implemented separately |
+| `caption-ai:Planner` | `@narratage/caption-gemini` | immutable display Atoms/Words + per-run requirements -> whole-Atom Cue cuts and optional per-Word fields | implemented; Google Vertex Endpoint implemented separately |
| `caption-fine:Track` | `@narratage/caption-fine` | CaptionPlan + independent SemanticMap + complete Fine Styles -> VisualTrack | implemented |
| `text:Track` | `@narratage/text-track` | package Spec + ProgramSpace -> VisualTrack | provider-free Surface/lowering implemented; exact-font use remains |
| `film:Film` | `@narratage/film` | finite TrackSet fold -> Composition | Graph Fragment and official Surface implemented |
diff --git a/examples/talking-film-golden/main.svml b/examples/talking-film-golden/main.svml
index 6b23a4d7..cb8679a6 100644
--- a/examples/talking-film-golden/main.svml
+++ b/examples/talking-film-golden/main.svml
@@ -64,7 +64,7 @@
<caption-fine:Style id="bob-caption" recipe={studio.caption.bob}/>
- <caption:Program id="caption-program" words={script.caption.words} default={dialogue-caption}>
+ <caption:Program id="caption-program" display={script.caption} default={dialogue-caption}>
<caption:Use role="ALICE" style={alice-caption}/>
<caption:Use role="BOB" style={bob-caption}/>
<!-- Ordered whole-style replacement: no complement Selection is required. -->
@@ -73,7 +73,7 @@
<caption-ai:Planner
id="caption-plan"
- words={script.caption.words}
+ display={script.caption}
program={caption-program}
model="gemini-2.5-flash"
/>
@@ -128,8 +128,8 @@
<!-- This imported package explicitly chooses Gemini cue planning; it cannot rewrite Script. -->
<caption-fine:Track
id="captions"
- narrative={script}
- words={script.caption.words}
+ display={script.caption}
+ correspondence={script.caption.correspondence}
map={timing.map}
space={speech.space}
program={caption-program}
diff --git a/examples/talking-film-golden/studio.svs b/examples/talking-film-golden/studio.svs
index 81e4f24d..403e35a1 100644
--- a/examples/talking-film-golden/studio.svs
+++ b/examples/talking-film-golden/studio.svs
@@ -53,10 +53,6 @@
caption.dialogue {
cue-min-words: 2;
cue-max-words: 5;
- important-min-per-cue: 0;
- important-max-per-cue: 0;
- important-fill: #FFFFFF;
- important-scale: 1;
stack-order: 70;
x: 0.08;
y: 0.76;
@@ -75,10 +71,6 @@
caption.alice {
cue-min-words: 2;
cue-max-words: 5;
- important-min-per-cue: 0;
- important-max-per-cue: 2;
- important-fill: #FFF16A;
- important-scale: 1.12;
stack-order: 70;
x: 0.08;
y: 0.76;
@@ -97,10 +89,6 @@
caption.bob {
cue-min-words: 2;
cue-max-words: 5;
- important-min-per-cue: 0;
- important-max-per-cue: 0;
- important-fill: #FFD166;
- important-scale: 1;
stack-order: 70;
x: 0.08;
y: 0.76;
diff --git a/examples/talking-film-graph-check/README.md b/examples/talking-film-graph-check/README.md
index 0898b52b..162e96e3 100644
--- a/examples/talking-film-graph-check/README.md
+++ b/examples/talking-film-graph-check/README.md
@@ -14,7 +14,7 @@ pnpm narratage plan examples/talking-film-graph-check/build.svrun \
--package-lock examples/talking-film-graph-check/svml.packages.lock --root .
```
-The Gemini planner receives immutable ordered display words and resolved Style runs only. Its generic
-`CaptionPlan` joins the independent `CompleteSemanticMap` in `@narratage/caption`; neither the planner
-nor the Vertex Provider has text-rewriting or timing authority. `@narratage/caption-fine` alone owns
-the concrete `important` field, Recipe interpretation and visual renderer.
+The Gemini planner receives readable indivisible display atoms and resolved Style runs only. Its
+generic `CaptionPlan` joins the independent `CompleteSemanticMap` in `@narratage/caption`; neither
+the planner nor the Vertex Provider has text-rewriting or timing authority.
+`@narratage/caption-fine` alone owns Recipe interpretation and the concrete visual renderer.
diff --git a/examples/talking-film-graph-check/main.svml b/examples/talking-film-graph-check/main.svml
index 6d1569ef..39da298c 100644
--- a/examples/talking-film-graph-check/main.svml
+++ b/examples/talking-film-graph-check/main.svml
@@ -28,13 +28,13 @@
<whisperx:Alignment id="timing" narrative={story} audio={speech.audio}/>
<caption-fine:Style id="base-caption" recipe={studio.caption.base}/>
- <caption:Program id="caption-program" words={story.caption.words} default={base-caption}/>
- <caption-ai:Planner id="cue-plan" words={story.caption.words} program={caption-program} model="gemini-2.5-flash"/>
+ <caption:Program id="caption-program" display={story.caption} default={base-caption}/>
+ <caption-ai:Planner id="cue-plan" display={story.caption} program={caption-program} model="gemini-2.5-flash"/>
<broll:Track id="cards" map={timing.map} space={speech.space}>
<broll:Item source={motion.video} during={story.selection.demo} appearance={studio.broll.card}/>
</broll:Track>
- <caption-fine:Track id="captions" narrative={story} words={story.caption.words}
+ <caption-fine:Track id="captions" display={story.caption} correspondence={story.caption.correspondence}
map={timing.map} space={speech.space}
plan={cue-plan.plan} program={caption-program}/>
<text:Track id="titles" space={speech.space}>
diff --git a/examples/talking-film-graph-check/studio.svs b/examples/talking-film-graph-check/studio.svs
index 941df60f..4e0ea301 100644
--- a/examples/talking-film-graph-check/studio.svs
+++ b/examples/talking-film-graph-check/studio.svs
@@ -8,8 +8,6 @@
}
caption.base {
cue-min-words: 1; cue-max-words: 5;
- important-min-per-cue: 0; important-max-per-cue: 2;
- important-fill: #FFF16A; important-scale: 1.12;
stack-order: 70; x: 0.08; y: 0.76; width: 0.84; font: Inter; weight: 600;
size: 58; line-height: 1; align: center; fill: #FFFFFF;
background: #09090BCC; padding: 16 24; radius: 18;
diff --git a/examples/talking-film-graph-check/svml.packages.lock b/examples/talking-film-graph-check/svml.packages.lock
index 708f461b..afb4ba4a 100644
--- a/examples/talking-film-graph-check/svml.packages.lock
+++ b/examples/talking-film-graph-check/svml.packages.lock
@@ -14,17 +14,17 @@
{
"name": "@narratage/caption-fine",
"version": "0.0.0-dev",
- "digest": "sha256:0b803c8876f1838b25ff8d5a3511bd2f5510751940d4f2793db1eab4a9cc98cd"
+ "digest": "sha256:bd101448eeae8bdd8a3f9e7db98d85a2900020bf7ee6a4c7317e6d3826f4073d"
},
{
"name": "@narratage/caption-gemini",
"version": "0.0.0-dev",
- "digest": "sha256:2a8f75b53ffc67401aeb94f5d2b79afc4871ad2de11dab731ac65d74147b6aab"
+ "digest": "sha256:418af81230bf3ac8a7e307484b6098f283575bbae0b18a94cb525c3b52ce26e8"
},
{
"name": "@narratage/caption",
"version": "0.0.0-dev",
- "digest": "sha256:56883d4bed9bbf939d2c1267df7594b59902c240e18281e259f5eb0b5bca89e1"
+ "digest": "sha256:3a569665a5bc60fde2f049786f76928e0729b4fd723c91a38d568014f53cdbb1"
},
{
"name": "@narratage/component-kit",
@@ -84,7 +84,7 @@
{
"name": "@narratage/narrative",
"version": "0.0.0-dev",
- "digest": "sha256:61e1d8b149008c83cabb05b532c5fae49d0aadf479a405dd9b7bf98ff3dd4a09"
+ "digest": "sha256:d68aca30ab820e2ef794fc81d80aadb1c150937311048a6646df8b2dab3b99b0"
},
{
"name": "@narratage/program-space",
@@ -114,7 +114,7 @@
{
"name": "@narratage/script",
"version": "0.0.0-dev",
- "digest": "sha256:60860383d3df5645cc437e9a516b62db524a37d5dbe1acfa9a10008c4de5f709"
+ "digest": "sha256:d0b7b906c6527d7e3dd6bd71e227b10b468b75f0d6a1cc1a4ed6f03a343c7956"
},
{
"name": "@narratage/seedance",
@@ -134,7 +134,7 @@
{
"name": "@narratage/speech-alignment",
"version": "0.0.0-dev",
- "digest": "sha256:277ffa2f5453636c61f6eef5713ad9fc4d3f71c2529456232f37df4ad6686b20"
+ "digest": "sha256:9f54aefc38b06344902d0a9f1bb0e76f03cd9e9e29d1bd8d79263979c73da6d2"
},
{
"name": "@narratage/speech-basis",
@@ -149,7 +149,7 @@
{
"name": "@narratage/speech-spine",
"version": "0.0.0-dev",
- "digest": "sha256:99ba2c670a48d35eb25e538c6099c2b31f3a20801771d81642c8e78efe13c8b4"
+ "digest": "sha256:d7f8c6b8c274a78b7c62c67e2dcf4b5010d4fb74f04690b17649df0183ec6efb"
},
{
"name": "@narratage/speech",
@@ -169,7 +169,7 @@
{
"name": "@narratage/text",
"version": "0.0.0-dev",
- "digest": "sha256:0558cd7cde9d4f0b8d6d880da23212f83d2328f1e531afc6eb7567e527125964"
+ "digest": "sha256:5dbe71a52a2b77f29a77ee7cff790081044cbecbbc410ca085769dd27be2566c"
},
{
"name": "@narratage/validation",
@@ -203,8 +203,8 @@
"name": "@narratage/broll",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:490a4d536c592a308a4441cdbf5e8884a84aca07da6bd380a1bf4893052c6601",
- "closureDigest": "sha256:6fc6e1e3fb1de56e2cb2b4c1edeb88c971b1da1e2d30aa575cba5a1238416709"
+ "facetsDigest": "sha256:22290ed789d8ca7425ad00f88c69f59db17d85ad3bd6c359528941a3b7f56498",
+ "closureDigest": "sha256:aca8a03f76c53ecb87efec9b94ddf2b6754ed35a8428cee005451ba116ce60c4"
},
{
"specifier": "@narratage/caption",
@@ -212,8 +212,8 @@
"name": "@narratage/caption",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:2bde33270a3381610524b375782fe1bfd8cd73e75b9c93251e02b3d88b480ad0",
- "closureDigest": "sha256:bce7573a505437d7e87900be19ba9b9e2470efd2b34b1504a56df285c38174e6"
+ "facetsDigest": "sha256:1b92b73928d937b1825a4f026a5864ddd0d4bd6af73ae7a4699c4dbd0ce3a8e0",
+ "closureDigest": "sha256:98690f96d5049bd1627db8b69774abc83dbe4593dd7fd40038ff6da43b6d091a"
},
{
"specifier": "@narratage/caption-fine",
@@ -221,8 +221,8 @@
"name": "@narratage/caption-fine",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:24be8a66a292efc633e44781cee2ae391932e5b6a0ee917e2e14e02ef07f6afc",
- "closureDigest": "sha256:cf273f4247731bd0dca0c693af35eae3ea5ee7c21357d8baac51442cc54e218c"
+ "facetsDigest": "sha256:acb8acf3319ff676999190b6440776fb8ce65949d7890f7ad5f1f1bcffcbaaee",
+ "closureDigest": "sha256:7ef6b0f639f236c21c2107148343719ac69da211b9ef1f28ccbfee283ae134f3"
},
{
"specifier": "@narratage/caption-gemini",
@@ -230,8 +230,8 @@
"name": "@narratage/caption-gemini",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:7a4037e49c13ceb17587d43b01b663c46c1a3af451d340fcac52d64b5aba4671",
- "closureDigest": "sha256:40b5f28cb10b36aeed3871743fa43485a93b33315e0ac3ec817b5bae08188bb8"
+ "facetsDigest": "sha256:bdb5bc2561b4f91f72db885edbcef4caaeb8665861009166d0965c2d97182666",
+ "closureDigest": "sha256:33f7f84ca4e31c17262831816af63d7336fd1d706e79e02372f30f807254adfc"
},
{
"specifier": "@narratage/composition",
@@ -240,7 +240,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:ec680d429d88ec1c55531125a18c1620fea0c81600de55ba6d43c4fbf873e480",
- "closureDigest": "sha256:4a528d83d47df3588bd8223ed4f13b5497f54d8b9f1e50b3fba52ff0de7aacb9"
+ "closureDigest": "sha256:0d3642864d00b894d390e68f4ef575baa278414935ea227202fb3b572ecf1efb"
},
{
"specifier": "@narratage/film",
@@ -249,7 +249,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:61839fefb8cf723e9b55f4ac0932928b2dba5de4935ca03c0c4335246c6ab5d8",
- "closureDigest": "sha256:fe2b5a6656a1fa1e7df66a13b5a6b35a7231c085ddab77508c2ceb686c6f48c3"
+ "closureDigest": "sha256:d5b125075c494cd26a4d428d4a6caed63ce911771ef28fcebf361602e85e6e73"
},
{
"specifier": "@narratage/generation",
@@ -267,7 +267,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:34bcacd52b6d396bf53f3b9709b43d54377a48c3b0332180079c3488a7fbe6ec",
- "closureDigest": "sha256:2473311ab583320ab5c6cb383dd1cb20f902b1891af2821a78fe9ed7e571b966"
+ "closureDigest": "sha256:278ff28b7121f641e2df2bee6a891b3f4365aa8c58a08ee87c03adcc81d6d58c"
},
{
"specifier": "@narratage/media",
@@ -276,7 +276,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:c883f03447734dc30a6c239c828083e6e9708258e457f845c2cfa5a2f95c0387",
- "closureDigest": "sha256:ec5993223ae3961246e318e1c036a73dfbc5c5918b4ae7aaebea242d6c9a8b9b"
+ "closureDigest": "sha256:0906fb66319369378e604c5afdf4a466d215a575c679cd9ff244d68c6f8986c1"
},
{
"specifier": "@narratage/media-pipeline",
@@ -284,8 +284,8 @@
"name": "@narratage/media-pipeline",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:c93d4a7ced859956130610ebd14539663bbaed6152f3309eb7cb786cefd40458",
- "closureDigest": "sha256:661de1ca6a2907fb93758800c63439de8fabce230238bfb78e0dbdcf1a5c17a3"
+ "facetsDigest": "sha256:1b66155f5949570bfb11b31b20645615188cd6b0ffd034b1386b20f81a9bed42",
+ "closureDigest": "sha256:e4c966ba99f2f0f8a08bb732c7ca431a46e7af4c6a5eeec204dff7c11194af92"
},
{
"specifier": "@narratage/narrative",
@@ -293,8 +293,8 @@
"name": "@narratage/narrative",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:f4d042f2b821793beae3e33d1a35da83d45129e9e1bd33c54092a8abb75cccd7",
- "closureDigest": "sha256:a324b56d3f1550ae1b94f3d178d1e184f7c04aa8fde796f8f10fa034d9778d9c"
+ "facetsDigest": "sha256:778a963cda86b3e5e44359e66b6ba0b47cde31952c6ec2e09d0a1220df71bed0",
+ "closureDigest": "sha256:f793162590dcc0637ce25aec9b232d974bd4872fb2f8c1914dfd30c38c94f895"
},
{
"specifier": "@narratage/program-space",
@@ -311,8 +311,8 @@
"name": "@narratage/render-hyperframes",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:8a8726172d2396d54f8f99f483523827353ef1acb8fa3b050cd62ae16d9cd8ef",
- "closureDigest": "sha256:03f37df3468dab6d4a9dfa0b727c36c040dbe251526cecdf45edf49e39a27804"
+ "facetsDigest": "sha256:146a78ac53e7d5bf6910c62534e729d7567043636068ac9026e6ee6002cbc947",
+ "closureDigest": "sha256:1c80bcca6dff9ceea096c5b0c0b15536ec5ba1f68f92e6ee2cb087f12c4a3eaa"
},
{
"specifier": "@narratage/run-text",
@@ -321,7 +321,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:fbcab7cea86223fe00ea04b51c2160bb534aafbb99067f5877e6825b0ce4eefc",
- "closureDigest": "sha256:a6d37eabad64b436b87c5edf4a4200412415b303be3c571fc1af6f6c3ad289db"
+ "closureDigest": "sha256:0658c375a7b94becfb9e3b25fa79971ecc09f109ac835e6972dac2f57f4c31fe"
},
{
"specifier": "@narratage/script",
@@ -329,8 +329,8 @@
"name": "@narratage/script",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:c4500a649b05a9ba4b07bef7ee9cdc9d1d3ebaff76b9a678361131a7fca982f3",
- "closureDigest": "sha256:ad9cf9128e425d6bff194b83a64e7ae719df5d04a7622f8dba6228232ca0a5b4"
+ "facetsDigest": "sha256:1407d1c367c4131e3419837a66d0b81ae7a13ff44413540f8f63ef9663f6792f",
+ "closureDigest": "sha256:95a11a979adffded4e7a34565f7aceedc5e059fef38a3af3da24b65ed9c2df6a"
},
{
"specifier": "@narratage/seedance",
@@ -338,8 +338,8 @@
"name": "@narratage/seedance",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:bcb5da66886c2e21be9193492db40e3585991cb3846ee3b014a1848b4a00986e",
- "closureDigest": "sha256:e6d4aa8d6351bf7652760f94315dac192a484270cf5dbe33beb630da9f8086c3"
+ "facetsDigest": "sha256:e33699a3f4b87279ca22fa2c3422012013a3b91368355cffd10a5a529b6a7ca4",
+ "closureDigest": "sha256:11a501219f1f0f2d0f8ded90f24892ff16f8230a3e89136b8ebc71d842ef0e94"
},
{
"specifier": "@narratage/semantic-map",
@@ -347,8 +347,8 @@
"name": "@narratage/semantic-map",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:40f349ffdc8a2f276c971f2e62a22d6e162d97ed47f608e33d5a2c6e51cd93a4",
- "closureDigest": "sha256:5da5da001f31dfb0e09b96bf8fb3135ddc5e011f396b9ca4c69e46132e6f01c1"
+ "facetsDigest": "sha256:ee0976d45ddcf1c3ad79e0e2b1dc3d3481035a11bc1752e437407aa1e2055f34",
+ "closureDigest": "sha256:099227a8ca88bc795efe84b06862fb914e6ed52c2778dbee521378b1a9ab232d"
},
{
"specifier": "@narratage/speech",
@@ -356,8 +356,8 @@
"name": "@narratage/speech",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:606702bab5ea6bbab372be70e5d9d032975721c88754e64001929d20aa690bf7",
- "closureDigest": "sha256:c07b24e334b2efc6af8b0c7f8a729a40f00fd73e51693e68118984963a9a8ca0"
+ "facetsDigest": "sha256:0c006d9930a89bf8eb30db761c533ce478fcfc2f6c3697ffb76e0684284f63c1",
+ "closureDigest": "sha256:3af835a772697839c4f2eff6da7f44636767ebc54071353a917ce9ad3a2be542"
},
{
"specifier": "@narratage/speech-alignment",
@@ -365,8 +365,8 @@
"name": "@narratage/speech-alignment",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:ba23d1fe5fccd3c2210994a9d6e245bfed0b283e144e3040bbfcb621f440aefb",
- "closureDigest": "sha256:502f3454cbdeb1a7769cbeb25b0ec792afc5180f73e40129ec745978362c6b18"
+ "facetsDigest": "sha256:75e2dc61bb3cc417c5be0a04712b41bf9769c1940ec1764378d1a117bfd004d8",
+ "closureDigest": "sha256:fb2641832716592148a15a5f6cee2c05e8aa26311ad35d84a58ce7affcb5e74c"
},
{
"specifier": "@narratage/speech-basis",
@@ -374,8 +374,8 @@
"name": "@narratage/speech-basis",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:c98ce13f17cc1a5ebb225d3335db8804a49f70965732deb46ad133941d58f02b",
- "closureDigest": "sha256:57a24ce23026e4e4b1ca0d30ad58f55ce5840c252843653ebd5012105ef894d2"
+ "facetsDigest": "sha256:cc802b2e713bd0e92e838eb1803bed926e07658972b18aa6da596a89017bffd0",
+ "closureDigest": "sha256:2da1a922a375087e76ada57d92cfa9510a6e119218ca12756c59202ccc46e137"
},
{
"specifier": "@narratage/speech-evidence",
@@ -383,8 +383,8 @@
"name": "@narratage/speech-evidence",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:0de6289f6e0bc6b139e4c2fe063bde23877158f69be0935d506e83d2771a9af3",
- "closureDigest": "sha256:805278f1a0cf261172314edf17896df0e836622d4a9bd7519d5b2d17fa42ce84"
+ "facetsDigest": "sha256:4814f163a676c156b5a436182a041c3ea7c3154459b48108897395d015d65098",
+ "closureDigest": "sha256:3c253b02645a0fde101f85b0997a116485227fed74dad05fe58c8d7bfe5f0a7a"
},
{
"specifier": "@narratage/speech-spine",
@@ -392,8 +392,8 @@
"name": "@narratage/speech-spine",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:76bb403ab59040cd460c140b1f458a86305907a5c16076a75f745341a494092d",
- "closureDigest": "sha256:592f7aed7f9d27366d579f2bc9d6d6a2ffde05b68e7e47dd08d8b27f306a3370"
+ "facetsDigest": "sha256:2d5b77c3e4fe6e1f2737134659877198999ba4e7b43f6ba6c5477db615632817",
+ "closureDigest": "sha256:902ed264e7876cef0b96bdd73d75301761984a4f772e25083280b22a7335f4cb"
},
{
"specifier": "@narratage/svs",
@@ -410,8 +410,8 @@
"name": "@narratage/text-track",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:e57b2aa7713a21b5eb200d7f537ec5ddf216421f80c35a8437d180c3998785bd",
- "closureDigest": "sha256:68359a51ffe4a5aca13806a4652f9519f95ba5f7145f25d0440188e6c9b17312"
+ "facetsDigest": "sha256:c86bf1dacd060148ce6b7d1151c5c9ccb9640c92aee9348fa3cac02f76e7491b",
+ "closureDigest": "sha256:a553084e5e5f46971364a9b763ca2d0a876133169bd4e5f312f6318c3119e7aa"
},
{
"specifier": "@narratage/visual-ir",
@@ -428,9 +428,9 @@
"name": "@narratage/whisperx",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:d2c6ba803cd4fce5260bb40d5187bc71e0e5540d576d98a55689e36f01a40cde",
- "closureDigest": "sha256:88a8d09bc98716c2ac5ab47a6fdbdd02e9422d9287aff2782aa9cdac1c870b97"
+ "facetsDigest": "sha256:c53108f29b771a1238fe124428927d200c0dadef6a25724f77ce62c8d80a4738",
+ "closureDigest": "sha256:c6634d4fdc7dfafb8b16e4219018957528f27e581c892a9344f894c820bde296"
}
],
- "digest": "sha256:860cfabb975aab6ae63e86deb43af34393006f022bec4d554151c79dceb14b07"
+ "digest": "sha256:b7da0d1b0c3e83d40b208922f6e4009c64343668b7fc02ec098897bc4f14cdf4"
}
diff --git a/examples/talking-film-live/main.svml b/examples/talking-film-live/main.svml
index 67cb5526..d935ea55 100644
--- a/examples/talking-film-live/main.svml
+++ b/examples/talking-film-live/main.svml
@@ -91,18 +91,18 @@
<caption-fine:Style id="primary-caption" recipe={studio.caption.primary}/>
- <caption:Program id="caption-program" words={story.caption.words} default={primary-caption}/>
+ <caption:Program id="caption-program" display={story.caption} default={primary-caption}/>
<caption-ai:Planner
id="caption-plan"
- words={story.caption.words}
+ display={story.caption}
program={caption-program}
model="gemini-2.5-flash"
/>
<caption-fine:Track
id="captions"
- narrative={story}
- words={story.caption.words}
+ display={story.caption}
+ correspondence={story.caption.correspondence}
map={timing.map}
space={speech.space}
program={caption-program}
diff --git a/examples/talking-film-live/studio.svs b/examples/talking-film-live/studio.svs
index 7911fd04..14d8a859 100644
--- a/examples/talking-film-live/studio.svs
+++ b/examples/talking-film-live/studio.svs
@@ -11,10 +11,6 @@
caption.primary {
cue-min-words: 2;
cue-max-words: 5;
- important-min-per-cue: 0;
- important-max-per-cue: 2;
- important-fill: #FFF16A;
- important-scale: 1.12;
stack-order: 70;
x: 0.08;
y: 0.74;
diff --git a/examples/talking-film-live/svml.packages.lock b/examples/talking-film-live/svml.packages.lock
index 05b21ab2..6e09e25e 100644
--- a/examples/talking-film-live/svml.packages.lock
+++ b/examples/talking-film-live/svml.packages.lock
@@ -9,17 +9,17 @@
{
"name": "@narratage/caption-fine",
"version": "0.0.0-dev",
- "digest": "sha256:0b803c8876f1838b25ff8d5a3511bd2f5510751940d4f2793db1eab4a9cc98cd"
+ "digest": "sha256:bd101448eeae8bdd8a3f9e7db98d85a2900020bf7ee6a4c7317e6d3826f4073d"
},
{
"name": "@narratage/caption-gemini",
"version": "0.0.0-dev",
- "digest": "sha256:2a8f75b53ffc67401aeb94f5d2b79afc4871ad2de11dab731ac65d74147b6aab"
+ "digest": "sha256:418af81230bf3ac8a7e307484b6098f283575bbae0b18a94cb525c3b52ce26e8"
},
{
"name": "@narratage/caption",
"version": "0.0.0-dev",
- "digest": "sha256:56883d4bed9bbf939d2c1267df7594b59902c240e18281e259f5eb0b5bca89e1"
+ "digest": "sha256:3a569665a5bc60fde2f049786f76928e0729b4fd723c91a38d568014f53cdbb1"
},
{
"name": "@narratage/component-kit",
@@ -84,7 +84,7 @@
{
"name": "@narratage/narrative",
"version": "0.0.0-dev",
- "digest": "sha256:61e1d8b149008c83cabb05b532c5fae49d0aadf479a405dd9b7bf98ff3dd4a09"
+ "digest": "sha256:d68aca30ab820e2ef794fc81d80aadb1c150937311048a6646df8b2dab3b99b0"
},
{
"name": "@narratage/program-space",
@@ -114,7 +114,7 @@
{
"name": "@narratage/script",
"version": "0.0.0-dev",
- "digest": "sha256:60860383d3df5645cc437e9a516b62db524a37d5dbe1acfa9a10008c4de5f709"
+ "digest": "sha256:d0b7b906c6527d7e3dd6bd71e227b10b468b75f0d6a1cc1a4ed6f03a343c7956"
},
{
"name": "@narratage/seedance",
@@ -134,7 +134,7 @@
{
"name": "@narratage/speech-alignment",
"version": "0.0.0-dev",
- "digest": "sha256:277ffa2f5453636c61f6eef5713ad9fc4d3f71c2529456232f37df4ad6686b20"
+ "digest": "sha256:9f54aefc38b06344902d0a9f1bb0e76f03cd9e9e29d1bd8d79263979c73da6d2"
},
{
"name": "@narratage/speech-basis",
@@ -149,7 +149,7 @@
{
"name": "@narratage/speech-spine",
"version": "0.0.0-dev",
- "digest": "sha256:99ba2c670a48d35eb25e538c6099c2b31f3a20801771d81642c8e78efe13c8b4"
+ "digest": "sha256:d7f8c6b8c274a78b7c62c67e2dcf4b5010d4fb74f04690b17649df0183ec6efb"
},
{
"name": "@narratage/speech",
@@ -169,7 +169,7 @@
{
"name": "@narratage/text",
"version": "0.0.0-dev",
- "digest": "sha256:0558cd7cde9d4f0b8d6d880da23212f83d2328f1e531afc6eb7567e527125964"
+ "digest": "sha256:5dbe71a52a2b77f29a77ee7cff790081044cbecbbc410ca085769dd27be2566c"
},
{
"name": "@narratage/validation",
@@ -203,8 +203,8 @@
"name": "@narratage/caption",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:2bde33270a3381610524b375782fe1bfd8cd73e75b9c93251e02b3d88b480ad0",
- "closureDigest": "sha256:bce7573a505437d7e87900be19ba9b9e2470efd2b34b1504a56df285c38174e6"
+ "facetsDigest": "sha256:1b92b73928d937b1825a4f026a5864ddd0d4bd6af73ae7a4699c4dbd0ce3a8e0",
+ "closureDigest": "sha256:98690f96d5049bd1627db8b69774abc83dbe4593dd7fd40038ff6da43b6d091a"
},
{
"specifier": "@narratage/caption-fine",
@@ -212,8 +212,8 @@
"name": "@narratage/caption-fine",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:24be8a66a292efc633e44781cee2ae391932e5b6a0ee917e2e14e02ef07f6afc",
- "closureDigest": "sha256:cf273f4247731bd0dca0c693af35eae3ea5ee7c21357d8baac51442cc54e218c"
+ "facetsDigest": "sha256:acb8acf3319ff676999190b6440776fb8ce65949d7890f7ad5f1f1bcffcbaaee",
+ "closureDigest": "sha256:7ef6b0f639f236c21c2107148343719ac69da211b9ef1f28ccbfee283ae134f3"
},
{
"specifier": "@narratage/caption-gemini",
@@ -221,8 +221,8 @@
"name": "@narratage/caption-gemini",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:7a4037e49c13ceb17587d43b01b663c46c1a3af451d340fcac52d64b5aba4671",
- "closureDigest": "sha256:40b5f28cb10b36aeed3871743fa43485a93b33315e0ac3ec817b5bae08188bb8"
+ "facetsDigest": "sha256:bdb5bc2561b4f91f72db885edbcef4caaeb8665861009166d0965c2d97182666",
+ "closureDigest": "sha256:33f7f84ca4e31c17262831816af63d7336fd1d706e79e02372f30f807254adfc"
},
{
"specifier": "@narratage/composition",
@@ -231,7 +231,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:ec680d429d88ec1c55531125a18c1620fea0c81600de55ba6d43c4fbf873e480",
- "closureDigest": "sha256:4a528d83d47df3588bd8223ed4f13b5497f54d8b9f1e50b3fba52ff0de7aacb9"
+ "closureDigest": "sha256:0d3642864d00b894d390e68f4ef575baa278414935ea227202fb3b572ecf1efb"
},
{
"specifier": "@narratage/estimate",
@@ -239,8 +239,8 @@
"name": "@narratage/estimate",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:eefe70d98cbf8e501d8102fc30d95778193b7985bf684a28b59a041b03ea10b6",
- "closureDigest": "sha256:06ca52046a5fc0ebaecab78a060923a62f1d79e9505be02ec99df920cabb4c2a"
+ "facetsDigest": "sha256:e12b59419cfadb67a2066ef31977f2f23aa5540dbadb20215599f1f7eab16824",
+ "closureDigest": "sha256:17dde773aa24a177093ced2060e14216f7b47e1732688965f6354ca5132d521e"
},
{
"specifier": "@narratage/film",
@@ -249,7 +249,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:61839fefb8cf723e9b55f4ac0932928b2dba5de4935ca03c0c4335246c6ab5d8",
- "closureDigest": "sha256:fe2b5a6656a1fa1e7df66a13b5a6b35a7231c085ddab77508c2ceb686c6f48c3"
+ "closureDigest": "sha256:d5b125075c494cd26a4d428d4a6caed63ce911771ef28fcebf361602e85e6e73"
},
{
"specifier": "@narratage/generation",
@@ -267,7 +267,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:34bcacd52b6d396bf53f3b9709b43d54377a48c3b0332180079c3488a7fbe6ec",
- "closureDigest": "sha256:2473311ab583320ab5c6cb383dd1cb20f902b1891af2821a78fe9ed7e571b966"
+ "closureDigest": "sha256:278ff28b7121f641e2df2bee6a891b3f4365aa8c58a08ee87c03adcc81d6d58c"
},
{
"specifier": "@narratage/media",
@@ -276,7 +276,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:c883f03447734dc30a6c239c828083e6e9708258e457f845c2cfa5a2f95c0387",
- "closureDigest": "sha256:ec5993223ae3961246e318e1c036a73dfbc5c5918b4ae7aaebea242d6c9a8b9b"
+ "closureDigest": "sha256:0906fb66319369378e604c5afdf4a466d215a575c679cd9ff244d68c6f8986c1"
},
{
"specifier": "@narratage/media-pipeline",
@@ -284,8 +284,8 @@
"name": "@narratage/media-pipeline",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:c93d4a7ced859956130610ebd14539663bbaed6152f3309eb7cb786cefd40458",
- "closureDigest": "sha256:661de1ca6a2907fb93758800c63439de8fabce230238bfb78e0dbdcf1a5c17a3"
+ "facetsDigest": "sha256:1b66155f5949570bfb11b31b20645615188cd6b0ffd034b1386b20f81a9bed42",
+ "closureDigest": "sha256:e4c966ba99f2f0f8a08bb732c7ca431a46e7af4c6a5eeec204dff7c11194af92"
},
{
"specifier": "@narratage/narrative",
@@ -293,8 +293,8 @@
"name": "@narratage/narrative",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:f4d042f2b821793beae3e33d1a35da83d45129e9e1bd33c54092a8abb75cccd7",
- "closureDigest": "sha256:a324b56d3f1550ae1b94f3d178d1e184f7c04aa8fde796f8f10fa034d9778d9c"
+ "facetsDigest": "sha256:778a963cda86b3e5e44359e66b6ba0b47cde31952c6ec2e09d0a1220df71bed0",
+ "closureDigest": "sha256:f793162590dcc0637ce25aec9b232d974bd4872fb2f8c1914dfd30c38c94f895"
},
{
"specifier": "@narratage/program-space",
@@ -311,8 +311,8 @@
"name": "@narratage/render-hyperframes",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:8a8726172d2396d54f8f99f483523827353ef1acb8fa3b050cd62ae16d9cd8ef",
- "closureDigest": "sha256:03f37df3468dab6d4a9dfa0b727c36c040dbe251526cecdf45edf49e39a27804"
+ "facetsDigest": "sha256:146a78ac53e7d5bf6910c62534e729d7567043636068ac9026e6ee6002cbc947",
+ "closureDigest": "sha256:1c80bcca6dff9ceea096c5b0c0b15536ec5ba1f68f92e6ee2cb087f12c4a3eaa"
},
{
"specifier": "@narratage/run-text",
@@ -321,7 +321,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:fbcab7cea86223fe00ea04b51c2160bb534aafbb99067f5877e6825b0ce4eefc",
- "closureDigest": "sha256:a6d37eabad64b436b87c5edf4a4200412415b303be3c571fc1af6f6c3ad289db"
+ "closureDigest": "sha256:0658c375a7b94becfb9e3b25fa79971ecc09f109ac835e6972dac2f57f4c31fe"
},
{
"specifier": "@narratage/script",
@@ -329,8 +329,8 @@
"name": "@narratage/script",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:c4500a649b05a9ba4b07bef7ee9cdc9d1d3ebaff76b9a678361131a7fca982f3",
- "closureDigest": "sha256:ad9cf9128e425d6bff194b83a64e7ae719df5d04a7622f8dba6228232ca0a5b4"
+ "facetsDigest": "sha256:1407d1c367c4131e3419837a66d0b81ae7a13ff44413540f8f63ef9663f6792f",
+ "closureDigest": "sha256:95a11a979adffded4e7a34565f7aceedc5e059fef38a3af3da24b65ed9c2df6a"
},
{
"specifier": "@narratage/seedance",
@@ -338,8 +338,8 @@
"name": "@narratage/seedance",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:bcb5da66886c2e21be9193492db40e3585991cb3846ee3b014a1848b4a00986e",
- "closureDigest": "sha256:e6d4aa8d6351bf7652760f94315dac192a484270cf5dbe33beb630da9f8086c3"
+ "facetsDigest": "sha256:e33699a3f4b87279ca22fa2c3422012013a3b91368355cffd10a5a529b6a7ca4",
+ "closureDigest": "sha256:11a501219f1f0f2d0f8ded90f24892ff16f8230a3e89136b8ebc71d842ef0e94"
},
{
"specifier": "@narratage/semantic-map",
@@ -347,8 +347,8 @@
"name": "@narratage/semantic-map",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:40f349ffdc8a2f276c971f2e62a22d6e162d97ed47f608e33d5a2c6e51cd93a4",
- "closureDigest": "sha256:5da5da001f31dfb0e09b96bf8fb3135ddc5e011f396b9ca4c69e46132e6f01c1"
+ "facetsDigest": "sha256:ee0976d45ddcf1c3ad79e0e2b1dc3d3481035a11bc1752e437407aa1e2055f34",
+ "closureDigest": "sha256:099227a8ca88bc795efe84b06862fb914e6ed52c2778dbee521378b1a9ab232d"
},
{
"specifier": "@narratage/speech",
@@ -356,8 +356,8 @@
"name": "@narratage/speech",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:606702bab5ea6bbab372be70e5d9d032975721c88754e64001929d20aa690bf7",
- "closureDigest": "sha256:c07b24e334b2efc6af8b0c7f8a729a40f00fd73e51693e68118984963a9a8ca0"
+ "facetsDigest": "sha256:0c006d9930a89bf8eb30db761c533ce478fcfc2f6c3697ffb76e0684284f63c1",
+ "closureDigest": "sha256:3af835a772697839c4f2eff6da7f44636767ebc54071353a917ce9ad3a2be542"
},
{
"specifier": "@narratage/speech-alignment",
@@ -365,8 +365,8 @@
"name": "@narratage/speech-alignment",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:ba23d1fe5fccd3c2210994a9d6e245bfed0b283e144e3040bbfcb621f440aefb",
- "closureDigest": "sha256:502f3454cbdeb1a7769cbeb25b0ec792afc5180f73e40129ec745978362c6b18"
+ "facetsDigest": "sha256:75e2dc61bb3cc417c5be0a04712b41bf9769c1940ec1764378d1a117bfd004d8",
+ "closureDigest": "sha256:fb2641832716592148a15a5f6cee2c05e8aa26311ad35d84a58ce7affcb5e74c"
},
{
"specifier": "@narratage/speech-basis",
@@ -374,8 +374,8 @@
"name": "@narratage/speech-basis",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:c98ce13f17cc1a5ebb225d3335db8804a49f70965732deb46ad133941d58f02b",
- "closureDigest": "sha256:57a24ce23026e4e4b1ca0d30ad58f55ce5840c252843653ebd5012105ef894d2"
+ "facetsDigest": "sha256:cc802b2e713bd0e92e838eb1803bed926e07658972b18aa6da596a89017bffd0",
+ "closureDigest": "sha256:2da1a922a375087e76ada57d92cfa9510a6e119218ca12756c59202ccc46e137"
},
{
"specifier": "@narratage/speech-evidence",
@@ -383,8 +383,8 @@
"name": "@narratage/speech-evidence",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:0de6289f6e0bc6b139e4c2fe063bde23877158f69be0935d506e83d2771a9af3",
- "closureDigest": "sha256:805278f1a0cf261172314edf17896df0e836622d4a9bd7519d5b2d17fa42ce84"
+ "facetsDigest": "sha256:4814f163a676c156b5a436182a041c3ea7c3154459b48108897395d015d65098",
+ "closureDigest": "sha256:3c253b02645a0fde101f85b0997a116485227fed74dad05fe58c8d7bfe5f0a7a"
},
{
"specifier": "@narratage/speech-spine",
@@ -392,8 +392,8 @@
"name": "@narratage/speech-spine",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:76bb403ab59040cd460c140b1f458a86305907a5c16076a75f745341a494092d",
- "closureDigest": "sha256:592f7aed7f9d27366d579f2bc9d6d6a2ffde05b68e7e47dd08d8b27f306a3370"
+ "facetsDigest": "sha256:2d5b77c3e4fe6e1f2737134659877198999ba4e7b43f6ba6c5477db615632817",
+ "closureDigest": "sha256:902ed264e7876cef0b96bdd73d75301761984a4f772e25083280b22a7335f4cb"
},
{
"specifier": "@narratage/svs",
@@ -410,8 +410,8 @@
"name": "@narratage/text-track",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:e57b2aa7713a21b5eb200d7f537ec5ddf216421f80c35a8437d180c3998785bd",
- "closureDigest": "sha256:68359a51ffe4a5aca13806a4652f9519f95ba5f7145f25d0440188e6c9b17312"
+ "facetsDigest": "sha256:c86bf1dacd060148ce6b7d1151c5c9ccb9640c92aee9348fa3cac02f76e7491b",
+ "closureDigest": "sha256:a553084e5e5f46971364a9b763ca2d0a876133169bd4e5f312f6318c3119e7aa"
},
{
"specifier": "@narratage/visual-ir",
@@ -428,9 +428,9 @@
"name": "@narratage/whisperx",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:d2c6ba803cd4fce5260bb40d5187bc71e0e5540d576d98a55689e36f01a40cde",
- "closureDigest": "sha256:88a8d09bc98716c2ac5ab47a6fdbdd02e9422d9287aff2782aa9cdac1c870b97"
+ "facetsDigest": "sha256:c53108f29b771a1238fe124428927d200c0dadef6a25724f77ce62c8d80a4738",
+ "closureDigest": "sha256:c6634d4fdc7dfafb8b16e4219018957528f27e581c892a9344f894c820bde296"
}
],
- "digest": "sha256:048b34e973b3e1d3685681f6bb86fbe3d69c3c4c13ead0204f73502be8345e07"
+ "digest": "sha256:ba1796603708eefed02adc16db787a018bd81379fbfee172cbe09f7dbb9e0510"
}
diff --git a/examples/talking-film-live/svml.runtime-packages.lock b/examples/talking-film-live/svml.runtime-packages.lock
index 0f848e7c..ed48c98e 100644
--- a/examples/talking-film-live/svml.runtime-packages.lock
+++ b/examples/talking-film-live/svml.runtime-packages.lock
@@ -34,12 +34,12 @@
{
"name": "@narratage/caption-gemini",
"version": "0.0.0-dev",
- "digest": "sha256:8a631e9345ea2c5901f3785058fde176a8280b9f20dd77e3333d651d56991797"
+ "digest": "sha256:418af81230bf3ac8a7e307484b6098f283575bbae0b18a94cb525c3b52ce26e8"
},
{
"name": "@narratage/caption",
"version": "0.0.0-dev",
- "digest": "sha256:dc090e7d96c7a6c3a53c4aabd0874f1b743f02ecd7e05c81f50fef8cfa88ae6a"
+ "digest": "sha256:3a569665a5bc60fde2f049786f76928e0729b4fd723c91a38d568014f53cdbb1"
},
{
"name": "@narratage/component-kit",
@@ -99,7 +99,7 @@
{
"name": "@narratage/narrative",
"version": "0.0.0-dev",
- "digest": "sha256:85e98e61021310c2dd6473189dcbcc9daa54039277169cbb5ed51ed70277a894"
+ "digest": "sha256:d68aca30ab820e2ef794fc81d80aadb1c150937311048a6646df8b2dab3b99b0"
},
{
"name": "@narratage/program-space",
@@ -114,7 +114,7 @@
{
"name": "@narratage/provider-google-vertex",
"version": "0.0.0-dev",
- "digest": "sha256:13e1058f0d4058cb6790a969a3ed54b778d8e818305eac1625bd939c4eb81c09"
+ "digest": "sha256:4ff2b97f210866712da258f09fc315ee296096c5fd29b23f471eb0e1becea733"
},
{
"name": "@narratage/provider-hyperframes-local",
@@ -159,7 +159,7 @@
{
"name": "@narratage/semantic-map",
"version": "0.0.0-dev",
- "digest": "sha256:4293518a23ead1320f81d5d38e60bef96ea7f27149495bc890b9e4c275572864"
+ "digest": "sha256:425c1ea0d89039fc070de2a5de13b7d414d7a3a7d4c2e155445603fad3d60572"
},
{
"name": "@narratage/source",
@@ -169,7 +169,7 @@
{
"name": "@narratage/speech-alignment",
"version": "0.0.0-dev",
- "digest": "sha256:277ffa2f5453636c61f6eef5713ad9fc4d3f71c2529456232f37df4ad6686b20"
+ "digest": "sha256:9f54aefc38b06344902d0a9f1bb0e76f03cd9e9e29d1bd8d79263979c73da6d2"
},
{
"name": "@narratage/speech-evidence",
@@ -181,15 +181,10 @@
"version": "0.0.0-dev",
"digest": "sha256:1296f5241ab87621cbed2019ffd09e51a2ecd79a49848715ade628a9d722d8c1"
},
- {
- "name": "@narratage/svs",
- "version": "0.0.0-dev",
- "digest": "sha256:8973ec01d5c9390da4e5a3f74c7e5af0a4bb49e15796e98e748b75ce62363d9d"
- },
{
"name": "@narratage/text",
"version": "0.0.0-dev",
- "digest": "sha256:99605f90eeca6d07428bd3e90741c3ddb36c75f8cd6a0e0958112d0994d13fd9"
+ "digest": "sha256:5dbe71a52a2b77f29a77ee7cff790081044cbecbbc410ca085769dd27be2566c"
},
{
"name": "@narratage/validation",
@@ -855,7 +850,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:b2636492ab0b0b500d674899ab2ff14fc61c7f2b22ef44eb8f816f141b21ec4d",
- "closureDigest": "sha256:308ab3eba0e8ab28b601890f48b342645e16f2b6809efc8ddd27ef4a9568ee31"
+ "closureDigest": "sha256:a9da145f8ca7d78926bb835ac5e641d177632164cd5466050bab83fc18ba4603"
},
{
"specifier": "@narratage/provider-hyperframes-local",
@@ -864,7 +859,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:b14b721bdc0c9d5d9ced1c345107c92a3a5878f3383dea2ecd3e310442c2a40c",
- "closureDigest": "sha256:cd38be1844a865ae960decea036fb307d90e23a26f6b2b3654dd9add0d8a81f9"
+ "closureDigest": "sha256:2ee2076c51c8e5e3570287da0b6abbf055d664c099a05b7993130d12c2ccf667"
},
{
"specifier": "@narratage/provider-kie",
@@ -882,7 +877,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:701abcc923bf8a711c20b6e308d8cc8be27bf560f249b8c5a801e1dbd9cb3979",
- "closureDigest": "sha256:14ffbed83708b1a4dbf62c2087f42813c728d483f71969a0959b1587c3793051"
+ "closureDigest": "sha256:9c5ef78ea75970ea2d295a7dd9fcc60c890f190e46f2d3ca918a8a61077d0fef"
},
{
"specifier": "@narratage/provider-whisperx-local",
@@ -891,8 +886,8 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:3686b84c42cc2b1bcaccf57cbdcda25a15c7d9d3778fb92d25437f9880ed8f5d",
- "closureDigest": "sha256:df22e2ec9d843674c4fdbcb6d6d243f63cf426033e4586106917508a7023a508"
+ "closureDigest": "sha256:76b93481c5df50da73ebead3050abd139b350d0bedf8b802a08dc540f61ccc88"
}
],
- "digest": "sha256:c1a51e98d44a45fdb13e73fff46ebbeaed9367dab13e0acf0f35a3ee11c617c4"
+ "digest": "sha256:6cc6ad9fdbd1cdda1a80b7961ec0d805ce3a91591b60c1e5db8a8a5d8e5e9940"
}
diff --git a/examples/talking-head-aroll/README.md b/examples/talking-head-aroll/README.md
index 14898861..6505320d 100644
--- a/examples/talking-head-aroll/README.md
+++ b/examples/talking-head-aroll/README.md
@@ -17,7 +17,7 @@ separately imported `official-ugc-v1.svs` mapping. It emits an ordered Prompt Pr
Seedance Speech Program while compiling `main.svml`; neither operation is a Runtime task.
The four generated videos are normalized and concatenated into one Speech Spine. One canonical
audio projection goes through WhisperX, direct Script alignment produces the complete SemanticMap,
-Gemini plans Cue cuts and the `important` field without seeing timing, and the resulting Caption
+Gemini plans Cue cuts without seeing timing, and the resulting Caption
VisualTrack joins the peer speech visual/audio Tracks in Film. HyperFrames renders and muxes the
single `final.video` target.
diff --git a/examples/talking-head-aroll/main.svml b/examples/talking-head-aroll/main.svml
index 9dd0c1d0..04284064 100644
--- a/examples/talking-head-aroll/main.svml
+++ b/examples/talking-head-aroll/main.svml
@@ -103,10 +103,10 @@
<!-- Gemini sees display words and these requirements, never audio or timing. -->
<caption-fine:Style id="primary-caption" recipe={studio.caption.primary}/>
- <caption:Program id="caption-program" words={story.caption.words} default={primary-caption}/>
+ <caption:Program id="caption-program" display={story.caption} default={primary-caption}/>
<caption-ai:Planner
id="caption-plan"
- words={story.caption.words}
+ display={story.caption}
program={caption-program}
model="gemini-2.5-flash"
/>
@@ -114,8 +114,8 @@
<!-- Caption becomes an ordinary self-contained VisualTrack after joining the measured map. -->
<caption-fine:Track
id="captions"
- narrative={story}
- words={story.caption.words}
+ display={story.caption}
+ correspondence={story.caption.correspondence}
map={timing.map}
space={speech.space}
program={caption-program}
diff --git a/examples/talking-head-aroll/studio.svs b/examples/talking-head-aroll/studio.svs
index 8535f8f4..bc935e45 100644
--- a/examples/talking-head-aroll/studio.svs
+++ b/examples/talking-head-aroll/studio.svs
@@ -34,10 +34,6 @@
caption.primary {
cue-min-words: 2;
cue-max-words: 7;
- important-min-per-cue: 1;
- important-max-per-cue: 2;
- important-fill: #FFF16A;
- important-scale: 1.12;
stack-order: 70;
x: 0.08;
y: 0.74;
diff --git a/examples/talking-head-aroll/svml.packages.lock b/examples/talking-head-aroll/svml.packages.lock
index 056080d1..680d9f1f 100644
--- a/examples/talking-head-aroll/svml.packages.lock
+++ b/examples/talking-head-aroll/svml.packages.lock
@@ -9,17 +9,17 @@
{
"name": "@narratage/caption-fine",
"version": "0.0.0-dev",
- "digest": "sha256:0b803c8876f1838b25ff8d5a3511bd2f5510751940d4f2793db1eab4a9cc98cd"
+ "digest": "sha256:bd101448eeae8bdd8a3f9e7db98d85a2900020bf7ee6a4c7317e6d3826f4073d"
},
{
"name": "@narratage/caption-gemini",
"version": "0.0.0-dev",
- "digest": "sha256:2a8f75b53ffc67401aeb94f5d2b79afc4871ad2de11dab731ac65d74147b6aab"
+ "digest": "sha256:418af81230bf3ac8a7e307484b6098f283575bbae0b18a94cb525c3b52ce26e8"
},
{
"name": "@narratage/caption",
"version": "0.0.0-dev",
- "digest": "sha256:56883d4bed9bbf939d2c1267df7594b59902c240e18281e259f5eb0b5bca89e1"
+ "digest": "sha256:3a569665a5bc60fde2f049786f76928e0729b4fd723c91a38d568014f53cdbb1"
},
{
"name": "@narratage/component-kit",
@@ -84,7 +84,7 @@
{
"name": "@narratage/narrative",
"version": "0.0.0-dev",
- "digest": "sha256:61e1d8b149008c83cabb05b532c5fae49d0aadf479a405dd9b7bf98ff3dd4a09"
+ "digest": "sha256:d68aca30ab820e2ef794fc81d80aadb1c150937311048a6646df8b2dab3b99b0"
},
{
"name": "@narratage/program-space",
@@ -119,7 +119,7 @@
{
"name": "@narratage/script",
"version": "0.0.0-dev",
- "digest": "sha256:60860383d3df5645cc437e9a516b62db524a37d5dbe1acfa9a10008c4de5f709"
+ "digest": "sha256:d0b7b906c6527d7e3dd6bd71e227b10b468b75f0d6a1cc1a4ed6f03a343c7956"
},
{
"name": "@narratage/seedance-speaker",
@@ -144,7 +144,7 @@
{
"name": "@narratage/speech-alignment",
"version": "0.0.0-dev",
- "digest": "sha256:277ffa2f5453636c61f6eef5713ad9fc4d3f71c2529456232f37df4ad6686b20"
+ "digest": "sha256:9f54aefc38b06344902d0a9f1bb0e76f03cd9e9e29d1bd8d79263979c73da6d2"
},
{
"name": "@narratage/speech-basis",
@@ -159,7 +159,7 @@
{
"name": "@narratage/speech-spine",
"version": "0.0.0-dev",
- "digest": "sha256:99ba2c670a48d35eb25e538c6099c2b31f3a20801771d81642c8e78efe13c8b4"
+ "digest": "sha256:d7f8c6b8c274a78b7c62c67e2dcf4b5010d4fb74f04690b17649df0183ec6efb"
},
{
"name": "@narratage/speech",
@@ -174,7 +174,7 @@
{
"name": "@narratage/text",
"version": "0.0.0-dev",
- "digest": "sha256:0558cd7cde9d4f0b8d6d880da23212f83d2328f1e531afc6eb7567e527125964"
+ "digest": "sha256:5dbe71a52a2b77f29a77ee7cff790081044cbecbbc410ca085769dd27be2566c"
},
{
"name": "@narratage/validation",
@@ -208,8 +208,8 @@
"name": "@narratage/caption",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:2bde33270a3381610524b375782fe1bfd8cd73e75b9c93251e02b3d88b480ad0",
- "closureDigest": "sha256:bce7573a505437d7e87900be19ba9b9e2470efd2b34b1504a56df285c38174e6"
+ "facetsDigest": "sha256:1b92b73928d937b1825a4f026a5864ddd0d4bd6af73ae7a4699c4dbd0ce3a8e0",
+ "closureDigest": "sha256:98690f96d5049bd1627db8b69774abc83dbe4593dd7fd40038ff6da43b6d091a"
},
{
"specifier": "@narratage/caption-fine",
@@ -217,8 +217,8 @@
"name": "@narratage/caption-fine",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:24be8a66a292efc633e44781cee2ae391932e5b6a0ee917e2e14e02ef07f6afc",
- "closureDigest": "sha256:cf273f4247731bd0dca0c693af35eae3ea5ee7c21357d8baac51442cc54e218c"
+ "facetsDigest": "sha256:acb8acf3319ff676999190b6440776fb8ce65949d7890f7ad5f1f1bcffcbaaee",
+ "closureDigest": "sha256:7ef6b0f639f236c21c2107148343719ac69da211b9ef1f28ccbfee283ae134f3"
},
{
"specifier": "@narratage/caption-gemini",
@@ -226,8 +226,8 @@
"name": "@narratage/caption-gemini",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:7a4037e49c13ceb17587d43b01b663c46c1a3af451d340fcac52d64b5aba4671",
- "closureDigest": "sha256:40b5f28cb10b36aeed3871743fa43485a93b33315e0ac3ec817b5bae08188bb8"
+ "facetsDigest": "sha256:bdb5bc2561b4f91f72db885edbcef4caaeb8665861009166d0965c2d97182666",
+ "closureDigest": "sha256:33f7f84ca4e31c17262831816af63d7336fd1d706e79e02372f30f807254adfc"
},
{
"specifier": "@narratage/composition",
@@ -236,7 +236,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:ec680d429d88ec1c55531125a18c1620fea0c81600de55ba6d43c4fbf873e480",
- "closureDigest": "sha256:4a528d83d47df3588bd8223ed4f13b5497f54d8b9f1e50b3fba52ff0de7aacb9"
+ "closureDigest": "sha256:0d3642864d00b894d390e68f4ef575baa278414935ea227202fb3b572ecf1efb"
},
{
"specifier": "@narratage/estimate",
@@ -244,8 +244,8 @@
"name": "@narratage/estimate",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:eefe70d98cbf8e501d8102fc30d95778193b7985bf684a28b59a041b03ea10b6",
- "closureDigest": "sha256:06ca52046a5fc0ebaecab78a060923a62f1d79e9505be02ec99df920cabb4c2a"
+ "facetsDigest": "sha256:e12b59419cfadb67a2066ef31977f2f23aa5540dbadb20215599f1f7eab16824",
+ "closureDigest": "sha256:17dde773aa24a177093ced2060e14216f7b47e1732688965f6354ca5132d521e"
},
{
"specifier": "@narratage/film",
@@ -254,7 +254,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:61839fefb8cf723e9b55f4ac0932928b2dba5de4935ca03c0c4335246c6ab5d8",
- "closureDigest": "sha256:fe2b5a6656a1fa1e7df66a13b5a6b35a7231c085ddab77508c2ceb686c6f48c3"
+ "closureDigest": "sha256:d5b125075c494cd26a4d428d4a6caed63ce911771ef28fcebf361602e85e6e73"
},
{
"specifier": "@narratage/generation",
@@ -272,7 +272,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:34bcacd52b6d396bf53f3b9709b43d54377a48c3b0332180079c3488a7fbe6ec",
- "closureDigest": "sha256:2473311ab583320ab5c6cb383dd1cb20f902b1891af2821a78fe9ed7e571b966"
+ "closureDigest": "sha256:278ff28b7121f641e2df2bee6a891b3f4365aa8c58a08ee87c03adcc81d6d58c"
},
{
"specifier": "@narratage/media",
@@ -281,7 +281,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:c883f03447734dc30a6c239c828083e6e9708258e457f845c2cfa5a2f95c0387",
- "closureDigest": "sha256:ec5993223ae3961246e318e1c036a73dfbc5c5918b4ae7aaebea242d6c9a8b9b"
+ "closureDigest": "sha256:0906fb66319369378e604c5afdf4a466d215a575c679cd9ff244d68c6f8986c1"
},
{
"specifier": "@narratage/media-pipeline",
@@ -289,8 +289,8 @@
"name": "@narratage/media-pipeline",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:c93d4a7ced859956130610ebd14539663bbaed6152f3309eb7cb786cefd40458",
- "closureDigest": "sha256:661de1ca6a2907fb93758800c63439de8fabce230238bfb78e0dbdcf1a5c17a3"
+ "facetsDigest": "sha256:1b66155f5949570bfb11b31b20645615188cd6b0ffd034b1386b20f81a9bed42",
+ "closureDigest": "sha256:e4c966ba99f2f0f8a08bb732c7ca431a46e7af4c6a5eeec204dff7c11194af92"
},
{
"specifier": "@narratage/narrative",
@@ -298,8 +298,8 @@
"name": "@narratage/narrative",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:f4d042f2b821793beae3e33d1a35da83d45129e9e1bd33c54092a8abb75cccd7",
- "closureDigest": "sha256:a324b56d3f1550ae1b94f3d178d1e184f7c04aa8fde796f8f10fa034d9778d9c"
+ "facetsDigest": "sha256:778a963cda86b3e5e44359e66b6ba0b47cde31952c6ec2e09d0a1220df71bed0",
+ "closureDigest": "sha256:f793162590dcc0637ce25aec9b232d974bd4872fb2f8c1914dfd30c38c94f895"
},
{
"specifier": "@narratage/program-space",
@@ -325,8 +325,8 @@
"name": "@narratage/render-hyperframes",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:8a8726172d2396d54f8f99f483523827353ef1acb8fa3b050cd62ae16d9cd8ef",
- "closureDigest": "sha256:03f37df3468dab6d4a9dfa0b727c36c040dbe251526cecdf45edf49e39a27804"
+ "facetsDigest": "sha256:146a78ac53e7d5bf6910c62534e729d7567043636068ac9026e6ee6002cbc947",
+ "closureDigest": "sha256:1c80bcca6dff9ceea096c5b0c0b15536ec5ba1f68f92e6ee2cb087f12c4a3eaa"
},
{
"specifier": "@narratage/run-text",
@@ -335,7 +335,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:fbcab7cea86223fe00ea04b51c2160bb534aafbb99067f5877e6825b0ce4eefc",
- "closureDigest": "sha256:a6d37eabad64b436b87c5edf4a4200412415b303be3c571fc1af6f6c3ad289db"
+ "closureDigest": "sha256:0658c375a7b94becfb9e3b25fa79971ecc09f109ac835e6972dac2f57f4c31fe"
},
{
"specifier": "@narratage/script",
@@ -343,8 +343,8 @@
"name": "@narratage/script",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:c4500a649b05a9ba4b07bef7ee9cdc9d1d3ebaff76b9a678361131a7fca982f3",
- "closureDigest": "sha256:ad9cf9128e425d6bff194b83a64e7ae719df5d04a7622f8dba6228232ca0a5b4"
+ "facetsDigest": "sha256:1407d1c367c4131e3419837a66d0b81ae7a13ff44413540f8f63ef9663f6792f",
+ "closureDigest": "sha256:95a11a979adffded4e7a34565f7aceedc5e059fef38a3af3da24b65ed9c2df6a"
},
{
"specifier": "@narratage/seedance",
@@ -352,8 +352,8 @@
"name": "@narratage/seedance",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:bcb5da66886c2e21be9193492db40e3585991cb3846ee3b014a1848b4a00986e",
- "closureDigest": "sha256:e6d4aa8d6351bf7652760f94315dac192a484270cf5dbe33beb630da9f8086c3"
+ "facetsDigest": "sha256:e33699a3f4b87279ca22fa2c3422012013a3b91368355cffd10a5a529b6a7ca4",
+ "closureDigest": "sha256:11a501219f1f0f2d0f8ded90f24892ff16f8230a3e89136b8ebc71d842ef0e94"
},
{
"specifier": "@narratage/seedance-speaker",
@@ -361,8 +361,8 @@
"name": "@narratage/seedance-speaker",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:a1d08a41b12a83684754d636d4a0290957bbc970b7d49643774fa56597ec5865",
- "closureDigest": "sha256:90bd5bc73e49b63b93c8964042b4855c0d53af9d441bbb1eb4f10a1e20930041"
+ "facetsDigest": "sha256:2ee6645b4e1707f33fd3db99b285098644fff13b4329e0a107612b800b087b95",
+ "closureDigest": "sha256:d6e27a8ba274c2e9a05d1d35619fad87a65dd5406925ed05761263df0ebb297e"
},
{
"specifier": "@narratage/semantic-map",
@@ -370,8 +370,8 @@
"name": "@narratage/semantic-map",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:40f349ffdc8a2f276c971f2e62a22d6e162d97ed47f608e33d5a2c6e51cd93a4",
- "closureDigest": "sha256:5da5da001f31dfb0e09b96bf8fb3135ddc5e011f396b9ca4c69e46132e6f01c1"
+ "facetsDigest": "sha256:ee0976d45ddcf1c3ad79e0e2b1dc3d3481035a11bc1752e437407aa1e2055f34",
+ "closureDigest": "sha256:099227a8ca88bc795efe84b06862fb914e6ed52c2778dbee521378b1a9ab232d"
},
{
"specifier": "@narratage/speech",
@@ -379,8 +379,8 @@
"name": "@narratage/speech",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:606702bab5ea6bbab372be70e5d9d032975721c88754e64001929d20aa690bf7",
- "closureDigest": "sha256:c07b24e334b2efc6af8b0c7f8a729a40f00fd73e51693e68118984963a9a8ca0"
+ "facetsDigest": "sha256:0c006d9930a89bf8eb30db761c533ce478fcfc2f6c3697ffb76e0684284f63c1",
+ "closureDigest": "sha256:3af835a772697839c4f2eff6da7f44636767ebc54071353a917ce9ad3a2be542"
},
{
"specifier": "@narratage/speech-alignment",
@@ -388,8 +388,8 @@
"name": "@narratage/speech-alignment",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:ba23d1fe5fccd3c2210994a9d6e245bfed0b283e144e3040bbfcb621f440aefb",
- "closureDigest": "sha256:502f3454cbdeb1a7769cbeb25b0ec792afc5180f73e40129ec745978362c6b18"
+ "facetsDigest": "sha256:75e2dc61bb3cc417c5be0a04712b41bf9769c1940ec1764378d1a117bfd004d8",
+ "closureDigest": "sha256:fb2641832716592148a15a5f6cee2c05e8aa26311ad35d84a58ce7affcb5e74c"
},
{
"specifier": "@narratage/speech-basis",
@@ -397,8 +397,8 @@
"name": "@narratage/speech-basis",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:c98ce13f17cc1a5ebb225d3335db8804a49f70965732deb46ad133941d58f02b",
- "closureDigest": "sha256:57a24ce23026e4e4b1ca0d30ad58f55ce5840c252843653ebd5012105ef894d2"
+ "facetsDigest": "sha256:cc802b2e713bd0e92e838eb1803bed926e07658972b18aa6da596a89017bffd0",
+ "closureDigest": "sha256:2da1a922a375087e76ada57d92cfa9510a6e119218ca12756c59202ccc46e137"
},
{
"specifier": "@narratage/speech-evidence",
@@ -406,8 +406,8 @@
"name": "@narratage/speech-evidence",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:0de6289f6e0bc6b139e4c2fe063bde23877158f69be0935d506e83d2771a9af3",
- "closureDigest": "sha256:805278f1a0cf261172314edf17896df0e836622d4a9bd7519d5b2d17fa42ce84"
+ "facetsDigest": "sha256:4814f163a676c156b5a436182a041c3ea7c3154459b48108897395d015d65098",
+ "closureDigest": "sha256:3c253b02645a0fde101f85b0997a116485227fed74dad05fe58c8d7bfe5f0a7a"
},
{
"specifier": "@narratage/speech-spine",
@@ -415,8 +415,8 @@
"name": "@narratage/speech-spine",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:76bb403ab59040cd460c140b1f458a86305907a5c16076a75f745341a494092d",
- "closureDigest": "sha256:592f7aed7f9d27366d579f2bc9d6d6a2ffde05b68e7e47dd08d8b27f306a3370"
+ "facetsDigest": "sha256:2d5b77c3e4fe6e1f2737134659877198999ba4e7b43f6ba6c5477db615632817",
+ "closureDigest": "sha256:902ed264e7876cef0b96bdd73d75301761984a4f772e25083280b22a7335f4cb"
},
{
"specifier": "@narratage/svs",
@@ -442,9 +442,9 @@
"name": "@narratage/whisperx",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:d2c6ba803cd4fce5260bb40d5187bc71e0e5540d576d98a55689e36f01a40cde",
- "closureDigest": "sha256:88a8d09bc98716c2ac5ab47a6fdbdd02e9422d9287aff2782aa9cdac1c870b97"
+ "facetsDigest": "sha256:c53108f29b771a1238fe124428927d200c0dadef6a25724f77ce62c8d80a4738",
+ "closureDigest": "sha256:c6634d4fdc7dfafb8b16e4219018957528f27e581c892a9344f894c820bde296"
}
],
- "digest": "sha256:d15e5202ee8594ed492642182f70499e1eaeb238045f3c1e7134ddd0460ad1ef"
+ "digest": "sha256:9459e7a8b5c06ff90185db72ec3e4fc24d832e2e02082543f80d2413a08f55f8"
}
diff --git a/examples/talking-head-aroll/svml.runtime-packages.lock b/examples/talking-head-aroll/svml.runtime-packages.lock
index 0f848e7c..ed48c98e 100644
--- a/examples/talking-head-aroll/svml.runtime-packages.lock
+++ b/examples/talking-head-aroll/svml.runtime-packages.lock
@@ -34,12 +34,12 @@
{
"name": "@narratage/caption-gemini",
"version": "0.0.0-dev",
- "digest": "sha256:8a631e9345ea2c5901f3785058fde176a8280b9f20dd77e3333d651d56991797"
+ "digest": "sha256:418af81230bf3ac8a7e307484b6098f283575bbae0b18a94cb525c3b52ce26e8"
},
{
"name": "@narratage/caption",
"version": "0.0.0-dev",
- "digest": "sha256:dc090e7d96c7a6c3a53c4aabd0874f1b743f02ecd7e05c81f50fef8cfa88ae6a"
+ "digest": "sha256:3a569665a5bc60fde2f049786f76928e0729b4fd723c91a38d568014f53cdbb1"
},
{
"name": "@narratage/component-kit",
@@ -99,7 +99,7 @@
{
"name": "@narratage/narrative",
"version": "0.0.0-dev",
- "digest": "sha256:85e98e61021310c2dd6473189dcbcc9daa54039277169cbb5ed51ed70277a894"
+ "digest": "sha256:d68aca30ab820e2ef794fc81d80aadb1c150937311048a6646df8b2dab3b99b0"
},
{
"name": "@narratage/program-space",
@@ -114,7 +114,7 @@
{
"name": "@narratage/provider-google-vertex",
"version": "0.0.0-dev",
- "digest": "sha256:13e1058f0d4058cb6790a969a3ed54b778d8e818305eac1625bd939c4eb81c09"
+ "digest": "sha256:4ff2b97f210866712da258f09fc315ee296096c5fd29b23f471eb0e1becea733"
},
{
"name": "@narratage/provider-hyperframes-local",
@@ -159,7 +159,7 @@
{
"name": "@narratage/semantic-map",
"version": "0.0.0-dev",
- "digest": "sha256:4293518a23ead1320f81d5d38e60bef96ea7f27149495bc890b9e4c275572864"
+ "digest": "sha256:425c1ea0d89039fc070de2a5de13b7d414d7a3a7d4c2e155445603fad3d60572"
},
{
"name": "@narratage/source",
@@ -169,7 +169,7 @@
{
"name": "@narratage/speech-alignment",
"version": "0.0.0-dev",
- "digest": "sha256:277ffa2f5453636c61f6eef5713ad9fc4d3f71c2529456232f37df4ad6686b20"
+ "digest": "sha256:9f54aefc38b06344902d0a9f1bb0e76f03cd9e9e29d1bd8d79263979c73da6d2"
},
{
"name": "@narratage/speech-evidence",
@@ -181,15 +181,10 @@
"version": "0.0.0-dev",
"digest": "sha256:1296f5241ab87621cbed2019ffd09e51a2ecd79a49848715ade628a9d722d8c1"
},
- {
- "name": "@narratage/svs",
- "version": "0.0.0-dev",
- "digest": "sha256:8973ec01d5c9390da4e5a3f74c7e5af0a4bb49e15796e98e748b75ce62363d9d"
- },
{
"name": "@narratage/text",
"version": "0.0.0-dev",
- "digest": "sha256:99605f90eeca6d07428bd3e90741c3ddb36c75f8cd6a0e0958112d0994d13fd9"
+ "digest": "sha256:5dbe71a52a2b77f29a77ee7cff790081044cbecbbc410ca085769dd27be2566c"
},
{
"name": "@narratage/validation",
@@ -855,7 +850,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:b2636492ab0b0b500d674899ab2ff14fc61c7f2b22ef44eb8f816f141b21ec4d",
- "closureDigest": "sha256:308ab3eba0e8ab28b601890f48b342645e16f2b6809efc8ddd27ef4a9568ee31"
+ "closureDigest": "sha256:a9da145f8ca7d78926bb835ac5e641d177632164cd5466050bab83fc18ba4603"
},
{
"specifier": "@narratage/provider-hyperframes-local",
@@ -864,7 +859,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:b14b721bdc0c9d5d9ced1c345107c92a3a5878f3383dea2ecd3e310442c2a40c",
- "closureDigest": "sha256:cd38be1844a865ae960decea036fb307d90e23a26f6b2b3654dd9add0d8a81f9"
+ "closureDigest": "sha256:2ee2076c51c8e5e3570287da0b6abbf055d664c099a05b7993130d12c2ccf667"
},
{
"specifier": "@narratage/provider-kie",
@@ -882,7 +877,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:701abcc923bf8a711c20b6e308d8cc8be27bf560f249b8c5a801e1dbd9cb3979",
- "closureDigest": "sha256:14ffbed83708b1a4dbf62c2087f42813c728d483f71969a0959b1587c3793051"
+ "closureDigest": "sha256:9c5ef78ea75970ea2d295a7dd9fcc60c890f190e46f2d3ca918a8a61077d0fef"
},
{
"specifier": "@narratage/provider-whisperx-local",
@@ -891,8 +886,8 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:3686b84c42cc2b1bcaccf57cbdcda25a15c7d9d3778fb92d25437f9880ed8f5d",
- "closureDigest": "sha256:df22e2ec9d843674c4fdbcb6d6d243f63cf426033e4586106917508a7023a508"
+ "closureDigest": "sha256:76b93481c5df50da73ebead3050abd139b350d0bedf8b802a08dc540f61ccc88"
}
],
- "digest": "sha256:c1a51e98d44a45fdb13e73fff46ebbeaed9367dab13e0acf0f35a3ee11c617c4"
+ "digest": "sha256:6cc6ad9fdbd1cdda1a80b7961ec0d805ce3a91591b60c1e5db8a8a5d8e5e9940"
}
diff --git a/packages/caption-fine/README.md b/packages/caption-fine/README.md
index 5ac243ad..55b2b66d 100644
--- a/packages/caption-fine/README.md
+++ b/packages/caption-fine/README.md
@@ -3,23 +3,25 @@
The first official fine-grained Caption Style family. It turns an SVS Recipe into one complete
`CaptionStyle` and lowers timed Cues into one ordinary `VisualTrack`.
-The accepted target is deliberately plain: semantic Cue bounds plus one deterministic static
-appearance shared by every word in the Cue. Fine will declare no planning fields. `important`,
-random word sizing, karaoke state and per-word motion are not part of this family. The current
-executable baseline still contains the temporary optional `important` experiment; that is a known
-implementation delta, not a compatibility promise.
+Fine means short semantic Cues with deterministic geometry, typography and paint. Every Word in a
+Cue has one uniform static appearance. This family declares no planning fields and has no
+`important`, karaoke, random sizing or per-Word motion semantics.
```xml
<fine:Style id="primary" recipe={studio.caption.primary}/>
-<caption:Program id="captions" words={story.caption.words} default={primary}>
- <caption:Use words={story.caption.selection.callout} style={callout}/>
-</caption:Program>
+<caption:Program id="captions" display={story.caption} default={primary}/>
-<fine:Track id="captions-track" narrative={story} words={story.caption.words} map={timing.map}
- program={captions} plan={caption-plan.plan} space={speech.space}/>
+<fine:Track
+ id="captions-track"
+ display={story.caption}
+ correspondence={story.caption.correspondence}
+ map={timing.map}
+ program={captions}
+ plan={caption-plan.plan}
+ space={speech.space}
+/>
```
-One `fine:Track` handles the default and all ordered overrides and emits one ordinary
-`VisualTrack`. The common `@narratage/caption` package does not know this package's fonts, boxes,
-colors or placement rules.
+One Track handles the default and ordered Style replacements and emits one peer `VisualTrack`.
+Common Caption, Media, Composition and Core know none of Fine's Recipe fields or layout policy.
diff --git a/packages/caption-fine/src/component.ts b/packages/caption-fine/src/component.ts
index ee6062b2..d0e126e8 100644
--- a/packages/caption-fine/src/component.ts
+++ b/packages/caption-fine/src/component.ts
@@ -1,6 +1,6 @@
import type { CaptionProgram, TimedCaptionProjection } from "@narratage/caption";
import type { ComponentPackage } from "@narratage/component-kit";
-import type { CaptionWordSequence } from "@narratage/narrative";
+import type { CaptionDisplaySequence } from "@narratage/narrative";
import type { ProgramSpace } from "@narratage/program-space";
import type { StoredValue } from "@narratage/protocol";
import { canonicalize } from "@narratage/protocol";
@@ -22,7 +22,7 @@ export const captionFineComponent = {
outputs: { track: { kind: "inline", value: canonicalize(renderFineCaption(
inline<TimedCaptionProjection>(inputs.caption?.value, "TimedCaptionProjection"),
inline<CaptionProgram>(inputs.program?.value, "CaptionProgram"),
- inline<CaptionWordSequence>(inputs.words?.value, "CaptionWordSequence"),
+ inline<CaptionDisplaySequence>(inputs.display?.value, "CaptionDisplaySequence"),
inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),
)) } },
needs: {},
diff --git a/packages/caption-fine/src/fragment.ts b/packages/caption-fine/src/fragment.ts
index f60381cf..a1c7cec4 100644
--- a/packages/caption-fine/src/fragment.ts
+++ b/packages/caption-fine/src/fragment.ts
@@ -14,9 +14,9 @@ const operation = (id: string) => ({ kind: "fragment-operation" as const, operat
export const fineCaptionTrackFragment = sealGraphFragment({
name: "@narratage/caption-fine/track@1",
inputs: [
- { name: "narrative", type: narrativeTypes.narrative },
+ { name: "display", type: narrativeTypes.captionDisplay },
+ { name: "correspondence", type: narrativeTypes.captionCorrespondence },
{ name: "map", type: semanticMapTypes.complete },
- { name: "words", type: narrativeTypes.captionWordSequence },
{ name: "plan", type: captionTypes.plan },
{ name: "program", type: captionTypes.program },
{ name: "space", type: programSpaceTypes.programSpace },
@@ -26,7 +26,7 @@ export const fineCaptionTrackFragment = sealGraphFragment({
id: "caption:temporalize-plan",
producer: captionProducers.temporalizePlan,
inputs: {
- narrative: input("narrative"), map: input("map"), words: input("words"),
+ display: input("display"), correspondence: input("correspondence"), map: input("map"),
plan: input("plan"), program: input("program"),
},
result: { kind: "output", name: "caption" },
@@ -36,7 +36,7 @@ export const fineCaptionTrackFragment = sealGraphFragment({
producer: captionFineProducers.render,
inputs: {
caption: operation("caption:temporalize-plan"), program: input("program"),
- words: input("words"), space: input("space"),
+ display: input("display"), space: input("space"),
},
result: { kind: "output", name: "track" },
},
@@ -45,7 +45,7 @@ export const fineCaptionTrackFragment = sealGraphFragment({
name: "track",
type: compositionTypes.visualTrack,
root: operation("caption-fine:render"),
- semanticInputs: ["narrative", "map", "words", "plan", "program", "space"],
+ semanticInputs: ["display", "correspondence", "map", "plan", "program", "space"],
fidelity: "exact",
}],
});
diff --git a/packages/caption-fine/src/manifest.ts b/packages/caption-fine/src/manifest.ts
index ef37437d..acbabbc2 100644
--- a/packages/caption-fine/src/manifest.ts
+++ b/packages/caption-fine/src/manifest.ts
@@ -12,8 +12,8 @@ export const captionFineModuleRef = { name: "@narratage/caption-fine", version:
export const captionFineProducers = {
render: { module: captionFineModuleRef, name: "render-fine-caption" },
} satisfies Record<string, ProducerRef>;
-export const captionFineStyleSurfaceImplementationDigest = digestOf("@narratage/caption-fine/style-surface@1");
-export const captionFineTrackSurfaceImplementationDigest = digestOf("@narratage/caption-fine/track-surface@1");
+export const captionFineStyleSurfaceImplementationDigest = digestOf("@narratage/caption-fine/field-free-style-surface@1");
+export const captionFineTrackSurfaceImplementationDigest = digestOf("@narratage/caption-fine/whole-atom-track-surface@1");
export const captionFineManifest: ModuleManifest = {
format: "svml.module@1",
@@ -45,7 +45,7 @@ export const captionFineManifest: ModuleManifest = {
inputs: [
{ name: "caption", type: captionTypes.timedProjection },
{ name: "program", type: captionTypes.program },
- { name: "words", type: narrativeTypes.captionWordSequence },
+ { name: "display", type: narrativeTypes.captionDisplay },
{ name: "space", type: programSpaceTypes.programSpace },
],
outputs: [{ name: "track", type: compositionTypes.visualTrack }],
diff --git a/packages/caption-fine/src/render.ts b/packages/caption-fine/src/render.ts
index 37412ad2..c6dc0dc3 100644
--- a/packages/caption-fine/src/render.ts
+++ b/packages/caption-fine/src/render.ts
@@ -1,12 +1,8 @@
-import { assertCaptionProgramForWords, assertTimedCaptionProjection } from "@narratage/caption";
-import type {
- CaptionFieldAssignment,
- CaptionProgram,
- TimedCaptionProjection,
-} from "@narratage/caption";
+import { assertCaptionProgramForDisplay, assertTimedCaptionProjection } from "@narratage/caption";
+import type { CaptionProgram, TimedCaptionProjection } from "@narratage/caption";
import { assertVisualTrackIdentity, sealVisualTrack } from "@narratage/composition";
import type { VisualElement, VisualStyleDeclaration, VisualTrack } from "@narratage/composition";
-import type { CaptionWordSequence } from "@narratage/narrative";
+import type { CaptionDisplaySequence } from "@narratage/narrative";
import { assertProgramSpaceIdentity, programSpaceFrameCount } from "@narratage/program-space";
import type { ProgramSpace } from "@narratage/program-space";
import { digestOf } from "@narratage/protocol";
@@ -14,54 +10,37 @@ import { digestOf } from "@narratage/protocol";
import { assertFineCaptionParameters, FINE_CAPTION_FAMILY } from "./style.js";
import type { FineCaptionParameters } from "./types.js";
-export const renderFineCaptionImplementationDigest = digestOf("@narratage/caption-fine/render@1");
+export const renderFineCaptionImplementationDigest = digestOf("@narratage/caption-fine/render-field-free-atoms@1");
function frameAt(space: ProgramSpace, seconds: number): number {
return Math.round(seconds * space.frameRate.numerator / space.frameRate.denominator);
}
-function captionPieces(
- display: string,
- wordIds: readonly string[],
- wordText: ReadonlyMap<string, string>,
-): string[] {
- let cursor = 0;
- const starts = wordIds.map((wordId) => {
+function captionPieces(wordIds: readonly string[], wordText: ReadonlyMap<string, string>): string[] {
+ return wordIds.map((wordId, index) => {
const text = wordText.get(wordId);
if (text === undefined) throw new Error(`Fine Caption Cue references unknown word ${wordId}`);
- const start = display.indexOf(text, cursor);
- if (start < cursor) throw new Error(`Fine Caption Cue cannot locate word ${wordId} in its authored display`);
- cursor = start + text.length;
- return start;
+ return index === wordIds.length - 1 ? text : `${text} `;
});
- return starts.map((start, index) => display.slice(index === 0 ? 0 : start, starts[index + 1] ?? display.length));
}
-function textStyle(parameters: FineCaptionParameters, important: boolean): VisualStyleDeclaration[] {
+function textStyle(parameters: FineCaptionParameters): VisualStyleDeclaration[] {
return [
- { name: "color", value: important ? parameters.important.fill : parameters.typography.fill },
+ { name: "color", value: parameters.typography.fill },
{ name: "font-family", value: parameters.typography.fontFamily },
{ name: "font-size", value: `${parameters.typography.fontSizePx}px` },
{ name: "font-weight", value: parameters.typography.fontWeight },
{ name: "line-height", value: parameters.typography.lineHeight },
{ name: "white-space", value: "pre-wrap" },
- ...(important ? [{ name: "transform", value: `scale(${parameters.important.scale})` }] : []),
];
}
-function isImportant(fields: readonly CaptionFieldAssignment[], wordId: string): boolean {
- return fields.some((field) =>
- field.declarationId === "important" && field.wordId === wordId && field.value === "true");
-}
-
function cueElements(
- display: string,
wordIds: readonly string[],
- fields: readonly CaptionFieldAssignment[],
parameters: FineCaptionParameters,
wordText: ReadonlyMap<string, string>,
): VisualElement[] {
- const pieces = captionPieces(display, wordIds, wordText);
+ const pieces = captionPieces(wordIds, wordText);
return [
{
id: "placement",
@@ -95,18 +74,14 @@ function cueElements(
],
},
...wordIds.map((wordId, index): VisualElement => {
- const important = isImportant(fields, wordId);
return {
id: `word-${index + 1}`,
parent: "cue",
order: index + 2,
kind: "text",
text: pieces[index]!,
- style: textStyle(parameters, important),
- attributes: [
- { name: "data-caption-word", value: wordId },
- ...(important ? [{ name: "data-caption-field-important", value: "true" }] : []),
- ],
+ style: textStyle(parameters),
+ attributes: [{ name: "data-caption-word", value: wordId }],
};
}),
];
@@ -115,14 +90,16 @@ function cueElements(
export function renderFineCaption(
projection: TimedCaptionProjection,
program: CaptionProgram,
- words: CaptionWordSequence,
+ display: CaptionDisplaySequence,
space: ProgramSpace,
): VisualTrack {
assertTimedCaptionProjection(projection);
- assertCaptionProgramForWords(program, words);
+ assertCaptionProgramForDisplay(program, display);
+ if (projection.displaySequenceId !== display.id) throw new Error("Fine Caption received another display sequence");
assertProgramSpaceIdentity(space);
const styles = new Map(program.styles.map((style) => [style.id, style]));
- const wordText = new Map(words.words.map((word) => [word.id, word.text]));
+ const wordText = new Map(display.words.map((word) => [word.id, word.text]));
+ const atomById = new Map(display.atoms.map((atom) => [atom.id, atom]));
for (const style of styles.values()) {
if (style.rendering.family !== FINE_CAPTION_FAMILY) {
throw new Error(`Fine Caption cannot render Style family ${style.rendering.family}`);
@@ -130,22 +107,24 @@ export function renderFineCaption(
assertFineCaptionParameters(style.rendering.parameters as unknown as FineCaptionParameters);
}
const totalFrames = programSpaceFrameCount(space);
- const presents = projection.regions.flatMap((region) => {
- if (region.styleId === undefined || region.wordIds === undefined || region.wordIds.length === 0) {
- throw new Error(`Fine Caption region ${region.id} lacks resolved Style or word identities`);
- }
- const style = styles.get(region.styleId);
- if (style === undefined) throw new Error(`Fine Caption region ${region.id} references unknown Style ${region.styleId}`);
+ const presents = projection.cues.flatMap((cue) => {
+ const atoms = cue.atoms.map((timing) => atomById.get(timing.atomId));
+ if (atoms.some((atom) => atom === undefined)) throw new Error(`Fine Caption Cue ${cue.id} references unknown Atom`);
+ const resolvedAtoms = atoms.map((atom) => atom!);
+ const wordIds = resolvedAtoms.flatMap((atom) => atom.wordIds);
+ const style = styles.get(cue.styleId);
+ if (style === undefined) throw new Error(`Fine Caption Cue ${cue.id} references unknown Style ${cue.styleId}`);
const parameters = style.rendering.parameters as unknown as FineCaptionParameters;
- const startFrame = Math.max(0, frameAt(space, region.startSec));
- const measuredEnd = Math.min(totalFrames, frameAt(space, region.endSec));
+ if (cue.fields.length > 0) throw new Error(`Fine Caption Cue ${cue.id} contains unsupported fields`);
+ const startFrame = Math.max(0, frameAt(space, cue.startSec));
+ const measuredEnd = Math.min(totalFrames, frameAt(space, cue.endSec));
const endFrameExclusive = Math.min(totalFrames, Math.max(startFrame + 1, measuredEnd));
- if (!region.display || startFrame >= totalFrames || endFrameExclusive <= startFrame) return [];
+ if (startFrame >= totalFrames || endFrameExclusive <= startFrame) return [];
return [{
- id: region.id,
+ id: cue.id,
span: { startFrame, endFrameExclusive },
- stacking: { order: parameters.stackingOrder, tieBreak: `${program.id}:${region.id}` },
- elements: cueElements(region.display, region.wordIds, region.fields ?? [], parameters, wordText),
+ stacking: { order: parameters.stackingOrder, tieBreak: `${program.id}:${cue.id}` },
+ elements: cueElements(wordIds, parameters, wordText),
}];
});
const track = sealVisualTrack({
diff --git a/packages/caption-fine/src/style.ts b/packages/caption-fine/src/style.ts
index 1618dfd6..17f01f09 100644
--- a/packages/caption-fine/src/style.ts
+++ b/packages/caption-fine/src/style.ts
@@ -47,7 +47,6 @@ function padding(value: string): { readonly x: number; readonly y: number } {
export function fineCaptionParameters(recipe: SvsRecipe): FineCaptionParameters {
const expected = [
"align", "background", "cue-max-words", "cue-min-words", "fill", "font",
- "important-fill", "important-max-per-cue", "important-min-per-cue", "important-scale",
"line-height", "padding", "radius", "size", "stack-order", "weight", "width", "x", "y",
];
if (Object.keys(recipe.properties).sort().join("\0") !== expected.sort().join("\0")) {
@@ -76,7 +75,6 @@ export function fineCaptionParameters(recipe: SvsRecipe): FineCaptionParameters
paddingYPx: pad.y,
radiusPx: number(recipe, "radius"),
},
- important: { fill: color(recipe, "important-fill"), scale: number(recipe, "important-scale") },
};
assertFineCaptionParameters(parameters);
return parameters;
@@ -90,15 +88,15 @@ export function assertFineCaptionParameters(value: FineCaptionParameters): void
const numeric = [
value.placement.x, value.placement.y, value.placement.width, value.typography.fontSizePx,
value.typography.fontWeight, value.typography.lineHeight, value.box.paddingXPx, value.box.paddingYPx,
- value.box.radiusPx, value.important.scale,
+ value.box.radiusPx,
];
if (numeric.some((item) => !Number.isFinite(item) || item < 0)
|| value.placement.x > 1 || value.placement.y > 1 || value.placement.width <= 0 || value.placement.width > 1
- || value.typography.fontSizePx <= 0 || value.typography.lineHeight <= 0 || value.important.scale <= 0) {
+ || value.typography.fontSizePx <= 0 || value.typography.lineHeight <= 0) {
throw new Error("Fine Caption parameters contain invalid numeric bounds");
}
if (!value.typography.fontFamily.trim()) throw new Error("Fine Caption font family is empty");
- for (const colorValue of [value.typography.fill, value.box.background, value.important.fill]) {
+ for (const colorValue of [value.typography.fill, value.box.background]) {
if (!/^#[0-9a-f]{6}(?:[0-9a-f]{2})?$/iu.test(colorValue)) throw new Error("Fine Caption color is invalid");
}
}
@@ -106,14 +104,9 @@ export function assertFineCaptionParameters(value: FineCaptionParameters): void
export function fineCaptionStyle(id: string, recipe: SvsRecipe): CaptionStyleIntent {
const minimumWords = integer(recipe, "cue-min-words");
const maximumWords = integer(recipe, "cue-max-words");
- const importantMinimum = integer(recipe, "important-min-per-cue");
- const importantMaximum = integer(recipe, "important-max-per-cue");
if (minimumWords <= 0 || maximumWords < minimumWords) {
throw new Error("Fine Caption Recipe Cue word bounds are invalid");
}
- if (importantMinimum < 0 || importantMaximum < importantMinimum || importantMaximum > maximumWords) {
- throw new Error("Fine Caption Recipe important bounds are invalid");
- }
return sealCaptionStyle({
contract: "svml.caption-style@1",
id,
@@ -123,13 +116,7 @@ export function fineCaptionStyle(id: string, recipe: SvsRecipe): CaptionStyleInt
maximumWords,
instruction: `Split into complete semantic phrases of ${minimumWords} to ${maximumWords} display words. Never cross punctuation when avoidable.`,
},
- fields: importantMaximum === 0 ? [] : [{
- id: "important",
- value: { kind: "boolean" },
- instruction: "Select the words whose emphasis best communicates this Cue. The words need not be contiguous.",
- minimumPerCue: importantMinimum,
- maximumPerCue: importantMaximum,
- }],
+ fields: [],
},
rendering: {
family: FINE_CAPTION_FAMILY,
diff --git a/packages/caption-fine/src/surface.ts b/packages/caption-fine/src/surface.ts
index 491adc85..e2bcaddc 100644
--- a/packages/caption-fine/src/surface.ts
+++ b/packages/caption-fine/src/surface.ts
@@ -68,14 +68,19 @@ export const decodeFineCaptionStyleSurface: StructuredSurfaceHandler = ({ elemen
};
export const decodeFineCaptionTrackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
- attributes(element, ["id", "narrative", "map", "words", "program", "plan", "space"]);
+ attributes(element, ["id", "display", "correspondence", "map", "program", "plan", "space"]);
if (element.children.some((child) => child.kind === "element" || child.value.trim())) {
throw new Error(`${element.name} does not accept children`);
}
const id = stringAttribute(element, "id");
- const narrative = reference(element, "narrative", narrativeTypes.narrative, resolveReference);
+ const display = reference(element, "display", narrativeTypes.captionDisplay, resolveReference);
+ const correspondence = reference(
+ element,
+ "correspondence",
+ narrativeTypes.captionCorrespondence,
+ resolveReference,
+ );
const map = reference(element, "map", semanticMapTypes.complete, resolveReference);
- const words = reference(element, "words", narrativeTypes.captionWordSequence, resolveReference);
const program = reference(element, "program", captionTypes.program, resolveReference);
const plan = reference(element, "plan", captionTypes.plan, resolveReference);
const space = reference(element, "space", programSpaceTypes.programSpace, resolveReference);
@@ -85,7 +90,7 @@ export const decodeFineCaptionTrackSurface: StructuredSurfaceHandler = ({ elemen
id,
fragment: fineCaptionTrackFragment.id,
inputs: {
- narrative: narrative.ref, map: map.ref, words: words.ref,
+ display: display.ref, correspondence: correspondence.ref, map: map.ref,
program: program.ref, plan: plan.ref, space: space.ref,
},
outputs: { track: `${id}.track` },
diff --git a/packages/caption-fine/src/types.ts b/packages/caption-fine/src/types.ts
index e52464d0..38c464e8 100644
--- a/packages/caption-fine/src/types.ts
+++ b/packages/caption-fine/src/types.ts
@@ -20,8 +20,4 @@ export type FineCaptionParameters = {
readonly paddingYPx: number;
readonly radiusPx: number;
};
- readonly important: {
- readonly fill: string;
- readonly scale: number;
- };
};
diff --git a/packages/caption-fine/test/caption-fine.test.ts b/packages/caption-fine/test/caption-fine.test.ts
index c0376ff1..c011d180 100644
--- a/packages/caption-fine/test/caption-fine.test.ts
+++ b/packages/caption-fine/test/caption-fine.test.ts
@@ -5,7 +5,7 @@ import { resolveCaptionProgram } from "@narratage/caption";
import type { TimedCaptionProjection } from "@narratage/caption";
import { fineCaptionStyle, renderFineCaption } from "@narratage/caption-fine";
import { sealProgramSpace } from "@narratage/program-space";
-import { captionWordSequence, parseScript } from "@narratage/script";
+import { captionDisplaySequence, parseScript } from "@narratage/script";
import type { SvsRecipe } from "@narratage/svs";
const recipe: SvsRecipe = {
@@ -14,10 +14,6 @@ const recipe: SvsRecipe = {
properties: {
"cue-min-words": 1,
"cue-max-words": 5,
- "important-min-per-cue": 0,
- "important-max-per-cue": 2,
- "important-fill": "#FFE044",
- "important-scale": 1.15,
"stack-order": 70,
x: 0.08,
y: 0.76,
@@ -34,28 +30,23 @@ const recipe: SvsRecipe = {
},
};
-test("one Fine renderer handles the default Style, per-word fields and one peer VisualTrack", () => {
+test("one Fine renderer handles uniform Cue appearance as one peer VisualTrack", () => {
const narrative = parseScript("fine.svml", "<line>Meaning becomes visible.</line>");
- const words = captionWordSequence(narrative, "story.caption.words");
+ const display = captionDisplaySequence(narrative, "story.caption");
const style = fineCaptionStyle("primary", recipe);
- const program = resolveCaptionProgram(words, "captions", style, []);
- const wordIds = words.words.map((word) => word.id);
+ const program = resolveCaptionProgram(display, "captions", style, []);
const projection: TimedCaptionProjection = {
contract: "svml.timed-caption-projection@1",
- text: "Meaning becomes visible.",
- regions: [{
+ displaySequenceId: display.id,
+ cues: [{
id: "cue:1",
runId: program.runs[0]!.id,
styleId: style.id,
- display: "Meaning becomes visible.",
segmentId: "line",
- kind: "identity",
- sourceTokenIds: narrative.tokens.map((token) => token.id),
startSec: 0,
endSec: 2,
- refinements: [],
- wordIds,
- fields: [{ declarationId: "important", wordId: wordIds[1]!, value: "true" }],
+ atoms: display.atoms.map((atom) => ({ atomId: atom.id, startSec: 0, endSec: 2 })),
+ fields: [],
}],
};
const space = sealProgramSpace({
@@ -63,14 +54,15 @@ test("one Fine renderer handles the default Style, per-word fields and one peer
durationSec: 2,
frameRate: { numerator: 30, denominator: 1 },
});
- const track = renderFineCaption(projection, program, words, space);
+ const track = renderFineCaption(projection, program, display, space);
assert.equal(track.contract, "svml.visual-track@1");
assert.equal(track.presents.length, 1);
const wordElements = track.presents[0]!.elements.filter((element) => element.kind === "text");
assert.equal(wordElements.length, 3);
assert.deepEqual(wordElements.map((element) => element.kind === "text" ? element.text : undefined),
["Meaning ", "becomes ", "visible."]);
- assert.equal(wordElements[1]!.style.some((declaration) =>
- declaration.name === "color" && declaration.value === "#FFE044"), true);
- assert.equal(wordElements[0]!.style.some((declaration) => declaration.name === "transform"), false);
+ assert.equal(wordElements.every((element) => element.style.some((declaration) =>
+ declaration.name === "color" && declaration.value === "#FFFFFF")), true);
+ assert.equal(wordElements.every((element) =>
+ element.style.every((declaration) => declaration.name !== "transform")), true);
});
diff --git a/packages/caption-gemini/README.md b/packages/caption-gemini/README.md
index 33cfc01c..a1cd4a8b 100644
--- a/packages/caption-gemini/README.md
+++ b/packages/caption-gemini/README.md
@@ -1,30 +1,25 @@
# `@narratage/caption-gemini`
-Exact Gemini Caption planning, without transcript correction, Style selection or timing authority.
+Exact Gemini Caption planning, without wording, Style-selection or timing authority.
```xml
<caption-ai:Planner
id="caption-plan"
- words={story.caption.words}
+ display={story.caption}
program={caption-program}
model="gemini-2.5-flash"
/>
```
-Gemini receives the immutable ordered display words and the planning requirements of each resolved
-Program run. It never receives WhisperX text, timestamps or the pronunciation side of Dual Text,
-and it cannot return replacement wording.
+Gemini receives one `atoms: string[][]` value per resolved Style run. The outer array is the
+ordered Cue-cut universe; each inner array is one indivisible display Atom; its strings are
+immutable display Words with authored punctuation. The pronunciation side of Dual Text, WhisperX,
+timestamps and internal ids never enter the prompt.
-Within each run the response may only:
+A response may consume consecutive Atoms with `atom_count` and assign declared fields using
+one-based `atom_number` plus `word_number` inside the Cue. The package restores stable ids and
+rejects rewrites, incomplete coverage, invalid coordinates, fields or values before producing a
+`CaptionPlan`.
-- choose `after_word_id` boundaries that partition the run exactly once and in order;
-- attach declared field values to `word_id` values inside that Cue.
-
-Fields are generic per-word attributes. A Style family may declare no fields, one `important`
-boolean, several independent booleans, enums or bounded numbers. A word may receive zero, one or
-multiple fields. The model package validates word identities, Cue coverage, Cue word bounds, field
-types and per-Cue cardinality before a `CaptionPlan` enters the graph.
-
-Model choice remains author-visible. Network execution is separate: a Runtime explicitly binds the
-exact Gemini capability to `@narratage/provider-google-vertex` or another compatible Endpoint.
-Credentials, queues and transport never enter `.svml` or BuildState.
+Model choice is author-visible. A Runtime separately binds the exact capability to Google Vertex
+or another explicitly configured Endpoint; credentials, queue and transport remain outside SVML.
diff --git a/packages/caption-gemini/src/component.ts b/packages/caption-gemini/src/component.ts
index 294dd7d1..c1204f18 100644
--- a/packages/caption-gemini/src/component.ts
+++ b/packages/caption-gemini/src/component.ts
@@ -1,6 +1,6 @@
import type { ComponentPackage } from "@narratage/component-kit";
import type { CaptionProgram } from "@narratage/caption";
-import type { CaptionWordSequence } from "@narratage/narrative";
+import type { CaptionDisplaySequence } from "@narratage/narrative";
import type { StoredValue } from "@narratage/protocol";
import { canonicalize } from "@narratage/protocol";
@@ -25,7 +25,7 @@ export const captionGeminiComponent = {
request: {
kind: "inline",
value: canonicalize(compileCaptionGeminiRequest(
- inline<CaptionWordSequence>(inputs.words?.value, "CaptionWordSequence"),
+ inline<CaptionDisplaySequence>(inputs.display?.value, "CaptionDisplaySequence"),
inline<CaptionProgram>(inputs.captionProgram?.value, "CaptionProgram"),
inline<CaptionGeminiProgram>(inputs.program?.value, "CaptionGeminiProgram"),
)),
diff --git a/packages/caption-gemini/src/fragment.ts b/packages/caption-gemini/src/fragment.ts
index 9a713350..e14790e0 100644
--- a/packages/caption-gemini/src/fragment.ts
+++ b/packages/caption-gemini/src/fragment.ts
@@ -10,7 +10,7 @@ const operation = (id: string) => ({ kind: "fragment-operation" as const, operat
export const captionGeminiPlanningFragment = sealGraphFragment({
name: "@narratage/caption-gemini/planning@1",
inputs: [
- { name: "words", type: narrativeTypes.captionWordSequence },
+ { name: "display", type: narrativeTypes.captionDisplay },
{ name: "captionProgram", type: captionTypes.program },
{ name: "program", type: captionGeminiTypes.program },
],
@@ -18,7 +18,7 @@ export const captionGeminiPlanningFragment = sealGraphFragment({
{
id: "caption-gemini:compile",
producer: captionGeminiProducers.compile,
- inputs: { words: input("words"), captionProgram: input("captionProgram"), program: input("program") },
+ inputs: { display: input("display"), captionProgram: input("captionProgram"), program: input("program") },
result: { kind: "output", name: "request" },
},
{
@@ -32,7 +32,7 @@ export const captionGeminiPlanningFragment = sealGraphFragment({
name: "plan",
type: captionTypes.plan,
root: operation("caption-gemini:plan"),
- semanticInputs: ["words", "captionProgram", "program"],
+ semanticInputs: ["display", "captionProgram", "program"],
fidelity: "exact",
}],
});
diff --git a/packages/caption-gemini/src/manifest.ts b/packages/caption-gemini/src/manifest.ts
index 500a6a02..9c3ca54e 100644
--- a/packages/caption-gemini/src/manifest.ts
+++ b/packages/caption-gemini/src/manifest.ts
@@ -16,11 +16,11 @@ export const captionGeminiProducers = {
request: { module: captionGeminiModuleRef, name: "request-caption-gemini-plan" },
} satisfies Record<string, ProducerRef>;
export const captionGeminiImplementationDigests = {
- compile: digestOf("@narratage/caption-gemini/compile-request@1"),
- request: digestOf("@narratage/caption-gemini/request-plan@1"),
+ compile: digestOf("@narratage/caption-gemini/compile-readable-atoms-structured-fields@1"),
+ request: digestOf("@narratage/caption-gemini/request-readable-atoms-plan@1"),
programValidator: digestOf("@narratage/caption-gemini/validate-program@1"),
- requestValidator: digestOf("@narratage/caption-gemini/validate-request@1"),
- plannerSurface: digestOf("@narratage/caption-gemini/planner-surface@1"),
+ requestValidator: digestOf("@narratage/caption-gemini/validate-readable-atoms-request@1"),
+ plannerSurface: digestOf("@narratage/caption-gemini/display-planner-surface@1"),
} as const;
const string = { kind: "string", minLength: 1 } as const satisfies ValueSchema;
@@ -50,7 +50,11 @@ const fieldDeclaration = object({
});
const planningRun = object({
id: { schema: string }, styleId: { schema: string },
- wordIds: { schema: { kind: "array", minItems: 1, items: string } },
+ atoms: { schema: { kind: "array", minItems: 1, items: object({
+ id: { schema: string }, words: { schema: { kind: "array", minItems: 1, items: object({
+ id: { schema: string }, text: { schema: string },
+ }) } },
+ }) } },
cueMinimumWords: { schema: nonNegativeInteger }, cueMaximumWords: { schema: nonNegativeInteger },
cueInstruction: { schema: string }, fields: { schema: { kind: "array", items: fieldDeclaration } },
});
@@ -61,7 +65,6 @@ export const captionGeminiProgramSchema: ValueSchema = object({
export const captionGeminiRequestSchema: ValueSchema = object({
contract: { schema: { kind: "literal", value: "svml.caption-gemini-request@1" } },
model: { schema: model },
- words: { schema: { kind: "array", minItems: 1, items: object({ id: { schema: string }, text: { schema: string } }) } },
runs: { schema: { kind: "array", minItems: 1, items: planningRun } },
systemInstruction: { schema: string }, prompt: { schema: string },
temperature: { schema: { kind: "literal", value: 0.2 } },
@@ -106,7 +109,7 @@ export const captionGeminiManifest: ModuleManifest = {
{
name: captionGeminiProducers.compile.name,
inputs: [
- { name: "words", type: narrativeTypes.captionWordSequence },
+ { name: "display", type: narrativeTypes.captionDisplay },
{ name: "captionProgram", type: captionTypes.program },
{ name: "program", type: captionGeminiTypes.program },
],
diff --git a/packages/caption-gemini/src/plan.ts b/packages/caption-gemini/src/plan.ts
index a0c46749..8331d9cb 100644
--- a/packages/caption-gemini/src/plan.ts
+++ b/packages/caption-gemini/src/plan.ts
@@ -42,28 +42,34 @@ function cueFromRaw(
raw: unknown,
runId: string,
cueIndex: number,
- runWordIds: readonly string[],
+ runAtoms: CaptionGeminiRequest["runs"][number]["atoms"],
cursor: number,
declarations: ReadonlyMap<string, CaptionFieldDeclaration>,
): { readonly cue: CaptionPlannedCue; readonly nextCursor: number } {
const value = object(raw, `Caption cue ${runId}/${cueIndex + 1}`);
- exactKeys(value, ["after_word_id", "fields"], `Caption cue ${runId}/${cueIndex + 1}`);
- assert(typeof value.after_word_id === "string", `Caption cue ${runId}/${cueIndex + 1} endpoint is invalid`);
- const endpoint = runWordIds.indexOf(value.after_word_id, cursor);
- assert(endpoint >= cursor, `Caption cue ${runId}/${cueIndex + 1} endpoint is outside its remaining run`);
- const wordIds = runWordIds.slice(cursor, endpoint + 1);
+ exactKeys(value, ["atom_count", "fields"], `Caption cue ${runId}/${cueIndex + 1}`);
+ assert(Number.isSafeInteger(value.atom_count) && (value.atom_count as number) > 0,
+ `Caption cue ${runId}/${cueIndex + 1} atom count is invalid`);
+ const nextCursor = cursor + (value.atom_count as number);
+ assert(nextCursor <= runAtoms.length, `Caption cue ${runId}/${cueIndex + 1} exceeds its remaining atoms`);
+ const cueAtoms = runAtoms.slice(cursor, nextCursor);
assert(Array.isArray(value.fields), `Caption cue ${runId}/${cueIndex + 1} fields are invalid`);
const assignments = value.fields.map((rawField, fieldIndex) => {
const field = object(rawField, `Caption field ${runId}/${cueIndex + 1}/${fieldIndex + 1}`);
- exactKeys(field, ["declaration_id", "word_id", "value"], "Caption field assignment");
+ exactKeys(field, ["declaration_id", "atom_number", "word_number", "value"], "Caption field assignment");
assert(typeof field.declaration_id === "string", "Caption field declaration id is invalid");
const declaration = declarations.get(field.declaration_id);
assert(declaration !== undefined, `Caption field ${field.declaration_id} was not declared for run ${runId}`);
- assert(typeof field.word_id === "string" && wordIds.includes(field.word_id),
- `Caption field ${declaration.id} word lies outside its cue`);
+ assert(Number.isSafeInteger(field.atom_number)
+ && (field.atom_number as number) >= 1 && (field.atom_number as number) <= cueAtoms.length,
+ `Caption field ${declaration.id} Atom number lies outside its cue`);
+ const atom = cueAtoms[(field.atom_number as number) - 1]!;
+ assert(Number.isSafeInteger(field.word_number)
+ && (field.word_number as number) >= 1 && (field.word_number as number) <= atom.words.length,
+ `Caption field ${declaration.id} word number lies outside its Atom`);
return {
declarationId: declaration.id,
- wordId: field.word_id,
+ wordId: atom.words[(field.word_number as number) - 1]!.id,
value: fieldValue(declaration, field.value),
};
});
@@ -75,8 +81,8 @@ function cueFromRaw(
`Caption field ${declaration.id} count ${count} violates ${declaration.minimumPerCue}..${declaration.maximumPerCue} per cue`);
}
return {
- cue: { id: `${runId}:cue:${cueIndex + 1}`, wordIds, fields: assignments },
- nextCursor: endpoint + 1,
+ cue: { id: `${runId}:cue:${cueIndex + 1}`, atomIds: cueAtoms.map((atom) => atom.id), fields: assignments },
+ nextCursor,
};
}
@@ -88,29 +94,25 @@ export function sealCaptionGeminiPlan(
const root = object(response, "Caption Gemini response");
exactKeys(root, ["runs"], "Caption Gemini response");
assert(Array.isArray(root.runs), "Caption Gemini response runs are invalid");
- const rawRuns = new Map<string, Record<string, unknown>>();
+ assert(root.runs.length === request.runs.length, "Caption Gemini response does not contain the exact run set");
+ const rawRuns: Record<string, unknown>[] = [];
for (const rawRun of root.runs) {
const run = object(rawRun, "Caption Gemini response run");
- exactKeys(run, ["run_id", "cues"], "Caption Gemini response run");
- assert(typeof run.run_id === "string" && !rawRuns.has(run.run_id), "Caption Gemini response run id is invalid or repeated");
- rawRuns.set(run.run_id, run);
+ exactKeys(run, ["cues"], "Caption Gemini response run");
+ rawRuns.push(run);
}
- assert(rawRuns.size === request.runs.length, "Caption Gemini response does not contain the exact run set");
- const runs = request.runs.map((expectedRun) => {
- const rawRun = rawRuns.get(expectedRun.id);
+ const runs = request.runs.map((expectedRun, runIndex) => {
+ const rawRun = rawRuns[runIndex];
assert(rawRun !== undefined && Array.isArray(rawRun.cues) && rawRun.cues.length > 0,
`Caption run ${expectedRun.id} has no Cues`);
const declarations = new Map(expectedRun.fields.map((field) => [field.id, field]));
let cursor = 0;
const cues = rawRun.cues.map((rawCue, cueIndex) => {
- const planned = cueFromRaw(rawCue, expectedRun.id, cueIndex, expectedRun.wordIds, cursor, declarations);
+ const planned = cueFromRaw(rawCue, expectedRun.id, cueIndex, expectedRun.atoms, cursor, declarations);
cursor = planned.nextCursor;
- assert(planned.cue.wordIds.length >= expectedRun.cueMinimumWords
- && planned.cue.wordIds.length <= expectedRun.cueMaximumWords,
- `Caption Cue ${planned.cue.id} violates ${expectedRun.cueMinimumWords}..${expectedRun.cueMaximumWords} words`);
return planned.cue;
});
- assert(cursor === expectedRun.wordIds.length, `Caption run ${expectedRun.id} Cues do not cover its final word`);
+ assert(cursor === expectedRun.atoms.length, `Caption run ${expectedRun.id} Cues do not cover its final atom`);
return { id: expectedRun.id, styleId: expectedRun.styleId, cues };
});
return sealCaptionPlan({
diff --git a/packages/caption-gemini/src/request.ts b/packages/caption-gemini/src/request.ts
index 81ba6a07..dfa9b2b2 100644
--- a/packages/caption-gemini/src/request.ts
+++ b/packages/caption-gemini/src/request.ts
@@ -1,6 +1,6 @@
-import { assertCaptionProgramForWords } from "@narratage/caption";
+import { assertCaptionProgramForDisplay } from "@narratage/caption";
import type { CaptionFieldDeclaration, CaptionProgram } from "@narratage/caption";
-import type { CaptionWordSequence } from "@narratage/narrative";
+import type { CaptionDisplaySequence } from "@narratage/narrative";
import { verifyCaptionGeminiProgram } from "./program.js";
import type {
@@ -13,43 +13,46 @@ function assert(condition: unknown, message: string): asserts condition {
if (!condition) throw new Error(message);
}
-function fieldInstruction(field: CaptionFieldDeclaration): string {
- const value = field.value.kind === "boolean"
- ? 'the string "true"; omit the assignment when the field does not apply'
- : field.value.kind === "enum"
- ? `one of ${field.value.values.map((item) => JSON.stringify(item)).join(", ")}`
- : "a finite base-10 number encoded as a string";
- return `${field.id}: ${field.minimumPerCue}..${field.maximumPerCue} word assignments per cue; value=${value}. ${field.instruction}`;
+function fieldPromptDeclaration(field: CaptionFieldDeclaration) {
+ return {
+ id: field.id,
+ value: field.value.kind === "enum"
+ ? { kind: "enum", values: [...field.value.values] }
+ : field.value.kind === "number"
+ ? {
+ kind: "number",
+ ...(field.value.minimum === undefined ? {} : { minimum: field.value.minimum }),
+ ...(field.value.maximum === undefined ? {} : { maximum: field.value.maximum }),
+ }
+ : { kind: "boolean", sparse_true: true },
+ per_cue: { minimum: field.minimumPerCue, maximum: field.maximumPerCue },
+ instruction: field.instruction,
+ };
}
-export function captionGeminiSystemInstruction(runs: readonly CaptionPlanningRun[]): string {
- const styleRules = new Map<string, CaptionPlanningRun>();
- runs.forEach((run) => styleRules.set(run.styleId, run));
+export function captionGeminiSystemInstruction(): string {
return [
"You are a caption cue and per-word attribute planner. Display words are immutable author truth.",
"Return only response-schema JSON. Never return corrected text, replacement text, pronunciation text, timestamps, Markdown, or explanations.",
"Each run is a hard boundary. Plan every run independently; a cue may never cross a run boundary.",
- "Within each run, cues must preserve and exactly partition word_ids. Return only each cue's final after_word_id; the final cue must end at the run's final word.",
- "Every field assignment targets exactly one word inside its cue and uses a field declared by that run's style.",
- ...[...styleRules.values()].flatMap((run) => [
- `Style ${run.styleId} cue judgment (${run.cueMinimumWords}..${run.cueMaximumWords} words): ${run.cueInstruction.trim()}`,
- ...run.fields.map((field) => `Style ${run.styleId} field ${fieldInstruction(field)}`),
- ]),
- "Echo every run exactly once. Use an empty fields array when no declaration applies. Never invent ids.",
+ "Each inner array in atoms is one indivisible display atom; each string inside it is one immutable display word including its authored punctuation.",
+ "Group consecutive whole atoms into cues; never split, rewrite, omit, repeat, or reorder an atom or word.",
+ "Count the inner-array words when applying cue_words. If one atom alone exceeds the preferred maximum, keep it whole as its own cue.",
+ "For every cue, atom_count is the number of consecutive unread atoms it consumes. The atom counts must partition the run exactly.",
+ "Every field assignment targets one display word by one-based atom_number inside its cue and one-based word_number inside that atom.",
+ "Return every run exactly once and in input order. Use an empty fields array when no declaration applies.",
].join("\n");
}
-function promptContent(request: Pick<CaptionGeminiRequest, "words" | "runs">): string {
+function promptContent(request: Pick<CaptionGeminiRequest, "runs">): string {
return JSON.stringify({
- display_words: request.words,
runs: request.runs.map((run) => ({
- run_id: run.id,
- style_id: run.styleId,
- word_ids: run.wordIds,
- cue_minimum_words: run.cueMinimumWords,
- cue_maximum_words: run.cueMaximumWords,
+ atoms: run.atoms.map((atom) => atom.words.map((word) => word.text)),
+ cue_words: { minimum: run.cueMinimumWords, maximum: run.cueMaximumWords },
cue_instruction: run.cueInstruction,
- field_declarations: run.fields,
+ ...(run.fields.length === 0 ? {} : {
+ field_declarations: run.fields.map((field) => fieldPromptDeclaration(field)),
+ }),
})),
}, null, 2);
}
@@ -58,11 +61,13 @@ function requestContent(value: CaptionGeminiRequest) {
return {
contract: "svml.caption-gemini-request@1" as const,
model: value.model,
- words: value.words.map((word) => ({ ...word })),
runs: value.runs.map((run) => ({
id: run.id,
styleId: run.styleId,
- wordIds: [...run.wordIds],
+ atoms: run.atoms.map((atom) => ({
+ id: atom.id,
+ words: atom.words.map((word) => ({ ...word })),
+ })),
cueMinimumWords: run.cueMinimumWords,
cueMaximumWords: run.cueMaximumWords,
cueInstruction: run.cueInstruction.trim(),
@@ -75,20 +80,34 @@ function requestContent(value: CaptionGeminiRequest) {
}
export function compileCaptionGeminiRequest(
- words: CaptionWordSequence,
+ display: CaptionDisplaySequence,
captionProgram: CaptionProgram,
program: CaptionGeminiProgram,
): CaptionGeminiRequest {
- assertCaptionProgramForWords(captionProgram, words);
+ assertCaptionProgramForDisplay(captionProgram, display);
verifyCaptionGeminiProgram(program);
const styles = new Map(captionProgram.styles.map((style) => [style.id, style]));
+ const wordById = new Map(display.words.map((word) => [word.id, word]));
+ const atomById = new Map(display.atoms.map((atom) => [atom.id, atom]));
const runs = captionProgram.runs.map((run) => {
const style = styles.get(run.styleId);
assert(style !== undefined, `Caption run ${run.id} references an unknown Style`);
return {
id: run.id,
styleId: run.styleId,
- wordIds: [...run.wordIds],
+ atoms: [...new Set(run.wordIds.map((wordId) => wordById.get(wordId)?.atomId))].map((atomId) => {
+ assert(atomId !== undefined, `Caption run ${run.id} references an unknown display word`);
+ const atom = atomById.get(atomId);
+ assert(atom !== undefined, `Caption run ${run.id} references unknown Atom ${atomId}`);
+ return {
+ id: atom.id,
+ words: atom.wordIds.map((wordId) => {
+ const word = wordById.get(wordId);
+ assert(word !== undefined, `Caption Atom ${atom.id} references unknown word ${wordId}`);
+ return { id: word.id, text: word.text };
+ }),
+ };
+ }),
cueMinimumWords: style.planning.cue.minimumWords,
cueMaximumWords: style.planning.cue.maximumWords,
cueInstruction: style.planning.cue.instruction,
@@ -98,10 +117,9 @@ export function compileCaptionGeminiRequest(
const base = {
contract: "svml.caption-gemini-request@1" as const,
model: program.model,
- words: words.words.map((word) => ({ id: word.id, text: word.text })),
runs,
};
- const systemInstruction = captionGeminiSystemInstruction(runs);
+ const systemInstruction = captionGeminiSystemInstruction();
const prompt = promptContent(base);
const request = requestContent({ ...base, systemInstruction, prompt, temperature: 0.2 });
verifyCaptionGeminiRequest(request);
@@ -114,22 +132,24 @@ export function verifyCaptionGeminiRequest(value: unknown): asserts value is Cap
assert(request.contract === "svml.caption-gemini-request@1", "Caption Gemini request contract is invalid");
assert(request.model === "gemini-2.5-flash" || request.model === "gemini-3.1-pro-preview", "Caption Gemini request model is unsupported");
assert(request.temperature === 0.2, "Caption Gemini temperature is not the package-owned value");
- assert(Array.isArray(request.words) && request.words.length > 0, "Caption Gemini request words are empty");
- const wordIds = request.words.map((word) => word.id);
- assert(wordIds.every((id) => typeof id === "string" && id.length > 0) && new Set(wordIds).size === wordIds.length,
- "Caption Gemini word ids are invalid or repeated");
- assert(request.words.every((word) => typeof word.text === "string" && word.text.length > 0),
- "Caption Gemini display word text is invalid");
- const wordSet = new Set(wordIds);
- const planned = request.runs.flatMap((run) => run.wordIds);
- assert(request.runs.length > 0 && planned.length === wordIds.length && new Set(planned).size === wordIds.length
- && planned.every((id) => wordSet.has(id)), "Caption Gemini runs are not an exact display-word partition");
- assert(request.runs.every((run) => run.wordIds.length > 0
+ assert(Array.isArray(request.runs) && request.runs.length > 0, "Caption Gemini request runs are empty");
+ const plannedAtoms = request.runs.flatMap((run: CaptionPlanningRun) =>
+ run.atoms.map((atom: CaptionPlanningRun["atoms"][number]) => atom.id));
+ const plannedWords = request.runs.flatMap((run: CaptionPlanningRun) =>
+ run.atoms.flatMap((atom: CaptionPlanningRun["atoms"][number]) =>
+ atom.words.map((word: CaptionPlanningRun["atoms"][number]["words"][number]) => word.id)));
+ assert(plannedAtoms.length > 0 && new Set(plannedAtoms).size === plannedAtoms.length
+ && plannedWords.length > 0 && new Set(plannedWords).size === plannedWords.length,
+ "Caption Gemini atoms repeat or omit internal display-word identities");
+ assert(request.runs.every((run) => run.atoms.length > 0
+ && run.atoms.every((atom: CaptionPlanningRun["atoms"][number]) =>
+ atom.id.length > 0 && atom.words.length > 0
+ && atom.words.every((word) => word.id.length > 0 && word.text.trim().length > 0))
&& Number.isSafeInteger(run.cueMinimumWords) && run.cueMinimumWords > 0
&& Number.isSafeInteger(run.cueMaximumWords) && run.cueMaximumWords >= run.cueMinimumWords
&& run.cueInstruction.trim().length > 0),
"Caption Gemini request contains an empty run");
- assert(request.systemInstruction === captionGeminiSystemInstruction(request.runs),
+ assert(request.systemInstruction === captionGeminiSystemInstruction(),
"Caption Gemini system instruction differs from the model package");
assert(request.prompt === promptContent(request), "Caption Gemini prompt differs from immutable display facts");
}
diff --git a/packages/caption-gemini/src/surface.ts b/packages/caption-gemini/src/surface.ts
index 84c2691a..e896d302 100644
--- a/packages/caption-gemini/src/surface.ts
+++ b/packages/caption-gemini/src/surface.ts
@@ -29,7 +29,7 @@ function reference(
}
export const decodeCaptionGeminiPlannerSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
- const required = ["id", "words", "program", "model"];
+ const required = ["id", "display", "program", "model"];
if (required.some((name) => element.attributes[name] === undefined)
|| Object.keys(element.attributes).some((name) => !required.includes(name))) {
throw new Error(`${element.name} requires ${required.join(", ")}`);
@@ -43,7 +43,7 @@ export const decodeCaptionGeminiPlannerSurface: StructuredSurfaceHandler = ({ el
if (model !== "gemini-2.5-flash" && model !== "gemini-3.1-pro-preview") {
throw new Error(`${element.name}.model is unsupported`);
}
- const words = reference(element, "words", narrativeTypes.captionWordSequence, resolveReference);
+ const display = reference(element, "display", narrativeTypes.captionDisplay, resolveReference);
const captionProgram = reference(element, "program", captionTypes.program, resolveReference);
const optionsId = `${id}.gemini`;
const options = sealCaptionGeminiProgram({
@@ -56,7 +56,7 @@ export const decodeCaptionGeminiPlannerSurface: StructuredSurfaceHandler = ({ el
id,
fragment: captionGeminiPlanningFragment.id,
inputs: {
- words: words.ref,
+ display: display.ref,
captionProgram: captionProgram.ref,
program: { kind: "record", id: optionsId },
},
diff --git a/packages/caption-gemini/src/types.ts b/packages/caption-gemini/src/types.ts
index b6c8231d..2e63a32d 100644
--- a/packages/caption-gemini/src/types.ts
+++ b/packages/caption-gemini/src/types.ts
@@ -26,15 +26,16 @@ export type CaptionGeminiProgram = {
readonly model: CaptionGeminiModel;
};
-export type CaptionPlanningWord = {
+export type CaptionPlanningAtom = {
readonly id: string;
- readonly text: string;
+ /** Stable identities stay inside the compiled request; the model sees only the word surfaces. */
+ readonly words: readonly { readonly id: string; readonly text: string }[];
};
export type CaptionPlanningRun = {
readonly id: string;
readonly styleId: string;
- readonly wordIds: readonly string[];
+ readonly atoms: readonly CaptionPlanningAtom[];
readonly cueMinimumWords: number;
readonly cueMaximumWords: number;
readonly cueInstruction: string;
@@ -44,7 +45,6 @@ export type CaptionPlanningRun = {
export type CaptionGeminiRequest = {
readonly contract: "svml.caption-gemini-request@1";
readonly model: CaptionGeminiModel;
- readonly words: readonly CaptionPlanningWord[];
readonly runs: readonly CaptionPlanningRun[];
readonly systemInstruction: string;
readonly prompt: string;
@@ -53,12 +53,15 @@ export type CaptionGeminiRequest = {
export type RawCaptionGeminiResponse = {
readonly runs: readonly {
- readonly run_id: string;
readonly cues: readonly {
- readonly after_word_id: string;
+ /** Number of consecutive unread atoms consumed by this Cue. */
+ readonly atom_count: number;
readonly fields: readonly {
readonly declaration_id: string;
- readonly word_id: string;
+ /** One-based Atom position inside this Cue. */
+ readonly atom_number: number;
+ /** One-based display-word position inside that Atom. */
+ readonly word_number: number;
readonly value: string;
}[];
}[];
diff --git a/packages/caption-gemini/test/planning.test.ts b/packages/caption-gemini/test/planning.test.ts
index a26f664f..8ed268ab 100644
--- a/packages/caption-gemini/test/planning.test.ts
+++ b/packages/caption-gemini/test/planning.test.ts
@@ -9,7 +9,7 @@ import {
sealCaptionGeminiProgram,
} from "@narratage/caption-gemini";
import type { CaptionGeminiRequest, RawCaptionGeminiResponse } from "@narratage/caption-gemini";
-import { captionWordSequence, parseScript } from "@narratage/script";
+import { captionDisplaySequence, parseScript } from "@narratage/script";
function style(id: string, fields: readonly CaptionFieldDeclaration[] = []): CaptionStyleIntent {
return sealCaptionStyle({
@@ -30,12 +30,12 @@ function options() {
function validResponse(request: CaptionGeminiRequest): RawCaptionGeminiResponse {
return {
runs: request.runs.map((run) => ({
- run_id: run.id,
cues: [{
- after_word_id: run.wordIds.at(-1)!,
+ atom_count: run.atoms.length,
fields: run.fields.length === 0 ? [] : [{
declaration_id: run.fields[0]!.id,
- word_id: run.wordIds[0]!,
+ atom_number: 1,
+ word_number: 1,
value: "true",
}],
}],
@@ -43,74 +43,156 @@ function validResponse(request: CaptionGeminiRequest): RawCaptionGeminiResponse
};
}
-test("Gemini sees only immutable left-side display words and resolved Style runs", () => {
- const narrative = parseScript("roles.svml", `
+test("Gemini sees only immutable display atoms and resolved Style runs", () => {
+ const parsed = parseScript("roles.svml", `
<opening>Meet <SVML | semantic video markup language>.</opening>
<answer><ALICE>Meaning becomes the source.</answer>
`);
- const words = captionWordSequence(narrative, "story.caption.words");
+ const display = captionDisplaySequence(parsed, "story.caption");
const alice = style("alice", [{
id: "important", value: { kind: "boolean" }, instruction: "Select up to one important word.",
minimumPerCue: 0, maximumPerCue: 1,
}]);
- const program = resolveCaptionProgram(words, "captions", style("normal"), [{
+ const program = resolveCaptionProgram(display, "captions", style("normal"), [{
id: "alice-use",
words: {
- contract: "svml.caption-word-subset@1", id: "role:ALICE", sequenceId: words.id,
- wordIds: words.words.filter((word) => word.role === "ALICE").map((word) => word.id),
+ contract: "svml.caption-display-word-subset@1", id: "role:ALICE", sequenceId: display.id,
+ wordIds: display.words.filter((word) => word.role === "ALICE").map((word) => word.id),
},
style: alice,
}]);
- const request = compileCaptionGeminiRequest(words, program, options());
+ const request = compileCaptionGeminiRequest(display, program, options());
assert.deepEqual(request.runs.map((run) => run.styleId), ["normal", "alice"]);
- assert.equal(request.words.some((word) => word.text === "SVML"), true);
- assert.equal(request.words.some((word) => word.text === "semantic"), false);
+ assert.equal(request.runs.some((run) => run.atoms.some((atom) =>
+ atom.words.some((word) => word.text === "SVML"))), true);
assert.equal(request.prompt.includes("semantic video markup language"), false);
assert.equal("whisperx" in request, false);
- const foreignWords = { ...words, id: "other.caption.words" };
- assert.throws(() => compileCaptionGeminiRequest(foreignWords, program, options()), /does not partition/u);
+ const foreignDisplay = { ...display, id: "other.caption" };
+ assert.throws(() => compileCaptionGeminiRequest(foreignDisplay, program, options()), /does not partition/u);
});
-test("Gemini may only cut runs and assign declared fields to words inside each Cue", () => {
- const narrative = parseScript("plan.svml", "<line>Meaning becomes the source.</line>");
- const words = captionWordSequence(narrative, "story.caption.words");
- const program = resolveCaptionProgram(words, "captions", style("emphasis", [{
+test("Gemini cuts only whole atoms and assigns fields by Atom and Word coordinates", () => {
+ const parsed = parseScript("plan.svml", "<line>Meaning becomes the source.</line>");
+ const display = captionDisplaySequence(parsed, "story.caption");
+ const program = resolveCaptionProgram(display, "captions", style("emphasis", [{
id: "important", value: { kind: "boolean" }, instruction: "Select exactly one important word.",
minimumPerCue: 1, maximumPerCue: 1,
}]), []);
- const request = compileCaptionGeminiRequest(words, program, options());
+ const request = compileCaptionGeminiRequest(display, program, options());
const plan = sealCaptionGeminiPlan(request, validResponse(request));
- assert.deepEqual(plan.runs.flatMap((run) => run.cues.flatMap((cue) => cue.wordIds)),
- request.runs.flatMap((run) => run.wordIds));
+ assert.deepEqual(JSON.parse(request.prompt).runs[0].field_declarations[0], {
+ id: "important",
+ value: { kind: "boolean", sparse_true: true },
+ per_cue: { minimum: 1, maximum: 1 },
+ instruction: "Select exactly one important word.",
+ });
+ assert.deepEqual(plan.runs.flatMap((run) => run.cues.flatMap((cue) => cue.atomIds)),
+ request.runs.flatMap((run) => run.atoms.map((atom) => atom.id)));
+
+ const outsideAtom = structuredClone(validResponse(request));
+ (outsideAtom.runs[0]!.cues[0]!.fields[0]! as { atom_number: number }).atom_number = 100;
+ assert.throws(() => sealCaptionGeminiPlan(request, outsideAtom), /Atom number lies outside/u);
+
+ const outsideWord = structuredClone(validResponse(request));
+ (outsideWord.runs[0]!.cues[0]!.fields[0]! as { word_number: number }).word_number = 100;
+ assert.throws(() => sealCaptionGeminiPlan(request, outsideWord), /word number lies outside/u);
+});
+
+test("a field-free Style requires only Cue boundaries", () => {
+ const parsed = parseScript(
+ "field-free.svml",
+ "<line>The quiet morning light moves softly across the empty studio windows.</line>",
+ );
+ const display = captionDisplaySequence(parsed, "story.caption");
+ const program = resolveCaptionProgram(display, "captions", style("fine"), []);
+ const request = compileCaptionGeminiRequest(display, program, options());
+ assert.deepEqual(request.runs.map((run) => run.fields), [[]]);
+ assert.doesNotMatch(request.prompt, /field_declarations/u);
- const outside = structuredClone(validResponse(request));
- (outside.runs[0]!.cues[0]!.fields[0]! as { word_id: string }).word_id = "foreign";
- assert.throws(() => sealCaptionGeminiPlan(request, outside), /outside its cue/u);
+ const count = request.runs[0]!.atoms.length;
+ const plan = sealCaptionGeminiPlan(request, { runs: [{ cues: [
+ { atom_count: 4, fields: [] },
+ { atom_count: 3, fields: [] },
+ { atom_count: count - 7, fields: [] },
+ ] }] });
+ assert.deepEqual(plan.runs[0]!.cues.map((cue) => cue.fields), [[], [], []]);
+ assert.deepEqual(plan.runs[0]!.cues.flatMap((cue) => cue.atomIds),
+ request.runs[0]!.atoms.map((atom) => atom.id));
});
-test("one word may carry multiple independent, non-contiguous field assignments", () => {
- const narrative = parseScript("multi.svml", "<line>one two three four five.</line>");
- const words = captionWordSequence(narrative, "story.caption.words");
+test("one Word may carry multiple independent field assignments", () => {
+ const parsed = parseScript("multi.svml", "<line>one two three four five.</line>");
+ const display = captionDisplaySequence(parsed, "story.caption");
const fields: CaptionFieldDeclaration[] = ["best", "medium"].map((id) => ({
id, value: { kind: "boolean" }, instruction: `Independently select ${id} words.`,
minimumPerCue: 0, maximumPerCue: 2,
}));
- const program = resolveCaptionProgram(words, "captions", style("multi", fields), []);
- const request = compileCaptionGeminiRequest(words, program, options());
- const ids = request.runs[0]!.wordIds;
- const plan = sealCaptionGeminiPlan(request, { runs: [{
- run_id: request.runs[0]!.id,
- cues: [{
- after_word_id: ids.at(-1)!,
- fields: [
- { declaration_id: "best", word_id: ids[0]!, value: "true" },
- { declaration_id: "best", word_id: ids[2]!, value: "true" },
- { declaration_id: "medium", word_id: ids[1]!, value: "true" },
- { declaration_id: "medium", word_id: ids[2]!, value: "true" },
- ],
- }],
- }] });
- assert.equal(plan.runs[0]!.cues[0]!.fields.filter((field) => field.wordId === ids[2]).length, 2);
+ const program = resolveCaptionProgram(display, "captions", style("multi", fields), []);
+ const request = compileCaptionGeminiRequest(display, program, options());
+ const plan = sealCaptionGeminiPlan(request, { runs: [{ cues: [{
+ atom_count: request.runs[0]!.atoms.length,
+ fields: [
+ { declaration_id: "best", atom_number: 3, word_number: 1, value: "true" },
+ { declaration_id: "medium", atom_number: 3, word_number: 1, value: "true" },
+ ],
+ }] }] });
+ const target = request.runs[0]!.atoms[2]!.words[0]!.id;
+ assert.equal(plan.runs[0]!.cues[0]!.fields.filter((field) => field.wordId === target).length, 2);
+});
+
+test("Dual Text becomes one readable indivisible string-array Atom", () => {
+ const parsed = parseScript("atoms.svml", "<line>new york <city is beautiful | fuck></line>");
+ const display = captionDisplaySequence(parsed, "story.caption");
+ const program = resolveCaptionProgram(display, "captions", style("fine"), []);
+ const request = compileCaptionGeminiRequest(display, program, options());
+
+ assert.deepEqual(request.runs[0]!.atoms.map((atom) => atom.words.map((word) => word.text)), [
+ ["new"], ["york"], ["city", "is", "beautiful"],
+ ]);
+ assert.deepEqual(JSON.parse(request.prompt).runs[0].atoms, [
+ ["new"], ["york"], ["city", "is", "beautiful"],
+ ]);
+ assert.equal(request.prompt.includes("fuck"), false);
+
+ const plan = sealCaptionGeminiPlan(request, { runs: [{ cues: [
+ { atom_count: 2, fields: [] },
+ { atom_count: 1, fields: [] },
+ ] }] });
+ assert.deepEqual(plan.runs[0]!.cues.map((cue) => cue.atomIds.length), [2, 1]);
+ assert.throws(() => sealCaptionGeminiPlan(request, { runs: [{ cues: [
+ { atom_count: 4, fields: [] },
+ ] }] }), /exceeds its remaining atoms/u);
+});
+
+test("punctuation is sent once, as immutable Word text inside atoms", () => {
+ const parsed = parseScript("punctuation.svml", "<line>45% back-and-forth damn! 300,000</line>");
+ const display = captionDisplaySequence(parsed, "story.caption");
+ const program = resolveCaptionProgram(display, "captions", style("fine"), []);
+ const request = compileCaptionGeminiRequest(display, program, options());
+ assert.deepEqual(JSON.parse(request.prompt).runs[0].atoms, [
+ ["45%"], ["back-and-forth"], ["damn!"], ["300,000"],
+ ]);
+ assert.equal(JSON.parse(request.prompt).runs[0].text, undefined);
+ assert.equal(JSON.parse(request.prompt).runs[0].words, undefined);
+});
+
+test("one indivisible Atom may exceed the preferred Cue maximum", () => {
+ const parsed = parseScript("oversized.svml", "<line><one two three four | something></line>");
+ const display = captionDisplaySequence(parsed, "story.caption");
+ const compact = sealCaptionStyle({
+ contract: "svml.caption-style@1",
+ id: "compact",
+ planning: {
+ cue: { minimumWords: 1, maximumWords: 3, instruction: "Prefer at most three visible words." },
+ fields: [],
+ },
+ rendering: { family: "test-caption@1", parameters: {} },
+ });
+ const program = resolveCaptionProgram(display, "captions", compact, []);
+ const request = compileCaptionGeminiRequest(display, program, options());
+ assert.deepEqual(request.runs[0]!.atoms[0]!.words.map((word) => word.text), ["one", "two", "three", "four"]);
+ const plan = sealCaptionGeminiPlan(request, { runs: [{ cues: [{ atom_count: 1, fields: [] }] }] });
+ assert.equal(plan.runs[0]!.cues[0]!.atomIds.length, 1);
});
diff --git a/packages/caption/README.md b/packages/caption/README.md
index 69cec005..47ef9ebd 100644
--- a/packages/caption/README.md
+++ b/packages/caption/README.md
@@ -1,44 +1,30 @@
# `@narratage/caption`
-Style-family-neutral Caption contracts and timing.
+Style-family-neutral Caption contracts and deterministic whole-Atom timing.
-The package owns four pieces of common language:
+The package owns:
-- a complete ordered `CaptionWordSequence` supplied explicitly by the Script package;
-- one total `CaptionProgram`, with an explicit default Style and ordered whole-Style overrides;
-- a generic `CaptionPlan`, which partitions words into Cues and assigns declared fields;
-- the deterministic join from planned display words to an independent `CompleteSemanticMap`.
+- total Style assignment over an explicit `CaptionDisplaySequence`;
+- generic `CaptionPlan` validation: Cue boundaries use Atom ids, fields use Word ids;
+- the join of `CaptionCorrespondence` and `CompleteSemanticMap` into timed Cues and Atoms.
-It does **not** own a concrete font, box model, field meaning, renderer, LLM or Provider. A Style
-family such as `@narratage/caption-fine` creates complete Styles and renders the resulting Program.
+It owns no Script parser, concrete font, box model, field meaning, renderer, LLM, Provider or
+Runtime policy.
```xml
<caption-fine:Style id="plain" recipe={studio.caption.plain}/>
<caption-fine:Style id="impact" recipe={studio.caption.impact}/>
-<caption:Program id="captions" words={story.caption.words} default={plain}>
+<caption:Program id="captions" display={story.caption} default={plain}>
<caption:Use role="ALICE" style={impact}/>
<caption:Use words={story.caption.selection.special} style={plain}/>
</caption:Program>
```
-The default covers the whole word universe. Authors do not create a fake `@whole` Selection or its
-complement. Each `Use` replaces one complete Style on an explicit word subset; later matching rules
-win. `role=` is author-surface sugar for a subset of the same ordered word universe. It is not a
-second time-range language.
+The default covers all display Words. Ordered `Use` rules replace whole Styles; later matches win.
+No rule may split a multiword Atom. `role=` is author-surface sugar for an explicit display-word
+subset, not a second time-range language.
-The Program stores only sequence identity and word-id-to-Style runs. It does not copy the upstream
-word payload. Planner, timing and rendering packages receive `story.caption.words` through their own
-explicit graph inputs.
-
-One Style is an indivisible pair:
-
-- common planning requirements: Cue word bounds and generic per-word field declarations;
-- package-owned rendering family and opaque resolved parameters.
-
-The planner has only two freedoms: partition each resolved run without reordering or rewriting its
-words, and assign zero or more declared fields to each word. It never sees audio or time. The common
-timing Producer later joins the validated Plan to Script's Caption projection and the measured
-semantic map.
-
-The package contains no LLM, Provider, credential, queue, visual renderer or Core authority.
+Planner freedom is limited to grouping consecutive whole Atoms into Cues and assigning declared
+fields to Words inside those Cues. Timing later gives each whole Atom the measured envelope of its
+explicit speech-token correspondence. It never invents per-Word timing.
diff --git a/packages/caption/src/component.ts b/packages/caption/src/component.ts
index 75a9bd06..f0c621eb 100644
--- a/packages/caption/src/component.ts
+++ b/packages/caption/src/component.ts
@@ -1,11 +1,10 @@
import type { ComponentPackage } from "@narratage/component-kit";
-import type { CaptionWordSequence, Narrative } from "@narratage/narrative";
+import type { CaptionCorrespondence, CaptionDisplaySequence } from "@narratage/narrative";
import type { CompleteSemanticMap } from "@narratage/semantic-map";
import type { StoredValue } from "@narratage/protocol";
import { canonicalize } from "@narratage/protocol";
import {
- captionImplementationDigest,
captionPlanImplementationDigest,
captionProducers,
captionTypes,
@@ -13,7 +12,7 @@ import {
} from "./manifest.js";
import { assertCaptionPlan } from "./plan.js";
import { assertCaptionProgram, assertCaptionStyle } from "./style.js";
-import { temporalizeCaption, temporalizeCaptionPlan } from "./temporalize.js";
+import { temporalizeCaptionPlan } from "./temporalize.js";
import { assertTimedCaptionProjection } from "./temporalize.js";
import type {
CaptionPlan,
@@ -31,22 +30,6 @@ function inline<T>(value: StoredValue | undefined, subject: string): T {
export const captionComponent = {
name: "@narratage/caption",
producers: [
- {
- producer: captionProducers.temporalize,
- implementationDigest: captionImplementationDigest,
- handler: ({ inputs }) => ({
- outputs: {
- caption: {
- kind: "inline",
- value: canonicalize(temporalizeCaption(
- inline<Narrative>(inputs.narrative?.value, "Narrative"),
- inline<CompleteSemanticMap>(inputs.map?.value, "CompleteSemanticMap"),
- )),
- },
- },
- needs: {},
- }),
- },
{
producer: captionProducers.temporalizePlan,
implementationDigest: captionPlanImplementationDigest,
@@ -55,9 +38,9 @@ export const captionComponent = {
caption: {
kind: "inline",
value: canonicalize(temporalizeCaptionPlan(
- inline<Narrative>(inputs.narrative?.value, "Narrative"),
+ inline<CaptionDisplaySequence>(inputs.display?.value, "CaptionDisplaySequence"),
+ inline<CaptionCorrespondence>(inputs.correspondence?.value, "CaptionCorrespondence"),
inline<CompleteSemanticMap>(inputs.map?.value, "CompleteSemanticMap"),
- inline<CaptionWordSequence>(inputs.words?.value, "CaptionWordSequence"),
inline<CaptionProgram>(inputs.program?.value, "CaptionProgram"),
inline<CaptionPlan>(inputs.plan?.value, "CaptionPlan"),
)),
diff --git a/packages/caption/src/display.ts b/packages/caption/src/display.ts
index 871c0391..35d920d4 100644
--- a/packages/caption/src/display.ts
+++ b/packages/caption/src/display.ts
@@ -1,89 +1,109 @@
import type {
- CaptionWord,
- CaptionWordSequence,
- CaptionWordSubset,
- Narrative,
+ CaptionCorrespondence,
+ CaptionDisplaySequence,
+ CaptionDisplayWord,
+ CaptionDisplayWordSubset,
} from "@narratage/narrative";
function assert(condition: unknown, message: string): asserts condition {
if (!condition) throw new Error(message);
}
-export function assertCaptionWordSequence(value: CaptionWordSequence): void {
- assert(value.contract === "svml.caption-word-sequence@1" && value.id.length > 0,
- "CaptionWordSequence identity is invalid");
- assert(value.words.length > 0, "CaptionWordSequence is empty");
- const ids = new Set<string>();
+export function assertCaptionDisplaySequence(value: CaptionDisplaySequence): void {
+ assert(value.contract === "svml.caption-display-sequence@1" && value.id.length > 0,
+ "CaptionDisplaySequence identity is invalid");
+ assert(value.atoms.length > 0 && value.words.length > 0,
+ "CaptionDisplaySequence is empty");
+ const words = new Map<string, CaptionDisplayWord>();
value.words.forEach((word, index) => {
- assert(word.id.length > 0 && !ids.has(word.id) && word.index === index,
- "CaptionWord identity is invalid or repeated");
- ids.add(word.id);
- assert(word.regionId.length > 0 && word.segmentId.length > 0 && word.turnId.length > 0 && word.text.length > 0,
- `CaptionWord ${word.id} context is invalid`);
- assert(Number.isSafeInteger(word.displayStart) && Number.isSafeInteger(word.displayEnd)
- && word.displayStart >= 0 && word.displayEnd > word.displayStart,
- `CaptionWord ${word.id} display range is invalid`);
- assert(Number.isSafeInteger(word.sourceTokenStart) && Number.isSafeInteger(word.sourceTokenEndExclusive)
- && word.sourceTokenStart >= 0 && word.sourceTokenEndExclusive > word.sourceTokenStart,
- `CaptionWord ${word.id} source range is invalid`);
- assert(word.correspondence === "exact" || word.correspondence === "region-envelope",
- `CaptionWord ${word.id} correspondence is invalid`);
+ assert(word.id.length > 0 && !words.has(word.id) && word.index === index,
+ "Caption display-word identity is invalid or repeated");
+ assert(word.atomId.length > 0 && word.segmentId.length > 0 && word.turnId.length > 0 && word.text.length > 0,
+ `Caption display word ${word.id} context is invalid`);
+ words.set(word.id, word);
});
+ const planned: string[] = [];
+ const atomIds = new Set<string>();
+ value.atoms.forEach((atom, index) => {
+ assert(atom.id.length > 0 && !atomIds.has(atom.id) && atom.index === index,
+ "Caption Atom identity is invalid or repeated");
+ atomIds.add(atom.id);
+ assert(atom.segmentId.length > 0 && atom.turnId.length > 0 && atom.wordIds.length > 0,
+ `Caption Atom ${atom.id} context is invalid`);
+ for (const wordId of atom.wordIds) {
+ const word = words.get(wordId);
+ assert(word !== undefined && word.atomId === atom.id,
+ `Caption Atom ${atom.id} references a foreign display word`);
+ assert(word.segmentId === atom.segmentId && word.turnId === atom.turnId && word.role === atom.role,
+ `Caption Atom ${atom.id} disagrees with its display-word context`);
+ planned.push(wordId);
+ }
+ });
+ assert(planned.join("\0") === value.words.map((word) => word.id).join("\0"),
+ "Caption Atoms do not partition their display words exactly once and in order");
}
-export function assertCaptionWordSubset(value: CaptionWordSubset, sequence: CaptionWordSequence): void {
- assertCaptionWordSequence(sequence);
- assert(value.contract === "svml.caption-word-subset@1" && value.id.length > 0,
- "CaptionWordSubset identity is invalid");
- assert(value.sequenceId === sequence.id, `CaptionWordSubset ${value.id} belongs to another word sequence`);
+export function assertCaptionCorrespondence(
+ value: CaptionCorrespondence,
+ sequence: CaptionDisplaySequence,
+): void {
+ assertCaptionDisplaySequence(sequence);
+ assert(value.contract === "svml.caption-correspondence@1"
+ && value.displaySequenceId === sequence.id,
+ "CaptionCorrespondence belongs to another display sequence");
+ assert(value.atoms.length === sequence.atoms.length,
+ "CaptionCorrespondence does not cover the exact Atom sequence");
+ const sourceIds = new Set<string>();
+ value.atoms.forEach((mapping, index) => {
+ assert(mapping.atomId === sequence.atoms[index]!.id && mapping.sourceTokenIds.length > 0,
+ "CaptionCorrespondence changes Atom order or contains an empty speech range");
+ for (const tokenId of mapping.sourceTokenIds) {
+ assert(tokenId.length > 0 && !sourceIds.has(tokenId),
+ `CaptionCorrespondence repeats speech token ${tokenId}`);
+ sourceIds.add(tokenId);
+ }
+ });
+}
+
+export function assertCaptionDisplayWordSubset(
+ value: CaptionDisplayWordSubset,
+ sequence: CaptionDisplaySequence,
+): void {
+ assertCaptionDisplaySequence(sequence);
+ assert(value.contract === "svml.caption-display-word-subset@1" && value.id.length > 0,
+ "CaptionDisplayWordSubset identity is invalid");
+ assert(value.sequenceId === sequence.id,
+ `CaptionDisplayWordSubset ${value.id} belongs to another display sequence`);
const positions = new Map(sequence.words.map((word) => [word.id, word.index]));
let previous = -1;
const seen = new Set<string>();
for (const id of value.wordIds) {
const position = positions.get(id);
assert(position !== undefined && position > previous && !seen.has(id),
- `CaptionWordSubset ${value.id} is not an ordered subset of ${sequence.id}`);
+ `CaptionDisplayWordSubset ${value.id} is not an ordered subset of ${sequence.id}`);
previous = position;
seen.add(id);
}
+ const selected = new Set(value.wordIds);
+ for (const atom of sequence.atoms) {
+ const count = atom.wordIds.filter((id) => selected.has(id)).length;
+ assert(count === 0 || count === atom.wordIds.length,
+ `CaptionDisplayWordSubset ${value.id} splits indivisible Atom ${atom.id}`);
+ }
}
/** Author-surface sugar for Role selection. The Program itself receives only the resolved subset. */
-export function captionWordsForRole(sequence: CaptionWordSequence, role: string): CaptionWordSubset {
- assertCaptionWordSequence(sequence);
+export function captionWordsForRole(
+ sequence: CaptionDisplaySequence,
+ role: string,
+): CaptionDisplayWordSubset {
+ assertCaptionDisplaySequence(sequence);
const normalized = role.trim();
assert(normalized.length > 0, "Caption Role is empty");
return {
- contract: "svml.caption-word-subset@1",
+ contract: "svml.caption-display-word-subset@1",
id: `role:${normalized}`,
sequenceId: sequence.id,
wordIds: sequence.words.filter((word) => word.role === normalized).map((word) => word.id),
};
}
-
-export function displayTextForWords(
- narrative: Narrative,
- universe: readonly CaptionWord[],
- selected: readonly CaptionWord[],
-): string {
- const byRegion = new Map<string, CaptionWord[]>();
- for (const word of selected) {
- const values = byRegion.get(word.regionId) ?? [];
- values.push(word);
- byRegion.set(word.regionId, values);
- }
- const parts: string[] = [];
- for (const region of narrative.captionProjection.regions) {
- const words = byRegion.get(region.id);
- if (words === undefined || words.length === 0) continue;
- const ordered = [...words].sort((left, right) => left.displayStart - right.displayStart);
- const all = universe
- .filter((word) => word.regionId === region.id)
- .sort((left, right) => left.displayStart - right.displayStart);
- const first = ordered[0]!;
- const last = ordered.at(-1)!;
- const after = all.find((word) => word.displayStart > last.displayStart);
- parts.push(region.display.slice(first.displayStart, after?.displayStart ?? region.display.length).trim());
- }
- return parts.join(" ").replace(/\s+/gu, " ").replace(/\s+([,.;:!?])/gu, "$1").trim();
-}
diff --git a/packages/caption/src/error.ts b/packages/caption/src/error.ts
index 1dacbfec..c1fcb54c 100644
--- a/packages/caption/src/error.ts
+++ b/packages/caption/src/error.ts
@@ -1,9 +1,9 @@
-export class CaptionProjectionError extends Error {
+export class CaptionTimingError extends Error {
readonly code: string;
constructor(code: string, message: string) {
super(message);
- this.name = "CaptionProjectionError";
+ this.name = "CaptionTimingError";
this.code = code;
}
}
diff --git a/packages/caption/src/fragment.ts b/packages/caption/src/fragment.ts
index 5ee08798..7fc98c99 100644
--- a/packages/caption/src/fragment.ts
+++ b/packages/caption/src/fragment.ts
@@ -7,34 +7,13 @@ import { captionProducers, captionTypes } from "./manifest.js";
const input = (name: string) => ({ kind: "fragment-input" as const, name });
const operation = (id: string) => ({ kind: "fragment-operation" as const, operation: id });
-export const captionTimingFragment = sealGraphFragment({
- name: "@narratage/caption/timing@1",
- inputs: [
- { name: "narrative", type: narrativeTypes.narrative },
- { name: "map", type: semanticMapTypes.complete },
- ],
- operations: [{
- id: "temporalize-caption",
- producer: captionProducers.temporalize,
- inputs: { narrative: input("narrative"), map: input("map") },
- result: { kind: "output", name: "caption" },
- }],
- exports: [{
- name: "caption",
- type: captionTypes.timedProjection,
- root: operation("temporalize-caption"),
- semanticInputs: ["narrative", "map"],
- fidelity: "exact",
- }],
-});
-
-/** Cue and field facts meet measured speech time here, before any Style renderer. */
+/** Cue and field facts meet only proven whole-Atom speech time here. */
export const plannedCaptionTimingFragment = sealGraphFragment({
name: "@narratage/caption/planned-timing@1",
inputs: [
- { name: "narrative", type: narrativeTypes.narrative },
+ { name: "display", type: narrativeTypes.captionDisplay },
+ { name: "correspondence", type: narrativeTypes.captionCorrespondence },
{ name: "map", type: semanticMapTypes.complete },
- { name: "words", type: narrativeTypes.captionWordSequence },
{ name: "plan", type: captionTypes.plan },
{ name: "program", type: captionTypes.program },
],
@@ -42,7 +21,7 @@ export const plannedCaptionTimingFragment = sealGraphFragment({
id: "temporalize-caption-plan",
producer: captionProducers.temporalizePlan,
inputs: {
- narrative: input("narrative"), map: input("map"), words: input("words"),
+ display: input("display"), correspondence: input("correspondence"), map: input("map"),
plan: input("plan"), program: input("program"),
},
result: { kind: "output", name: "caption" },
@@ -51,7 +30,7 @@ export const plannedCaptionTimingFragment = sealGraphFragment({
name: "caption",
type: captionTypes.timedProjection,
root: operation("temporalize-caption-plan"),
- semanticInputs: ["narrative", "map", "words", "plan", "program"],
+ semanticInputs: ["display", "correspondence", "map", "plan", "program"],
fidelity: "exact",
}],
});
diff --git a/packages/caption/src/index.ts b/packages/caption/src/index.ts
index e2d8af5e..34d1e511 100644
--- a/packages/caption/src/index.ts
+++ b/packages/caption/src/index.ts
@@ -1,14 +1,13 @@
-export { CaptionProjectionError } from "./error.js";
+export { CaptionTimingError } from "./error.js";
export {
- assertCaptionWordSequence,
- assertCaptionWordSubset,
+ assertCaptionCorrespondence,
+ assertCaptionDisplaySequence,
+ assertCaptionDisplayWordSubset,
captionWordsForRole,
- displayTextForWords,
} from "./display.js";
export { captionComponent } from "./component.js";
-export { captionTimingFragment, plannedCaptionTimingFragment } from "./fragment.js";
+export { plannedCaptionTimingFragment } from "./fragment.js";
export {
- captionImplementationDigest,
captionProgramSchema,
captionProgramSurfaceImplementationDigest,
captionPlanImplementationDigest,
@@ -25,12 +24,12 @@ export { decodeCaptionProgramSurface } from "./surface.js";
export { assertCaptionPlan, assertCaptionPlanForProgram, sealCaptionPlan } from "./plan.js";
export {
assertCaptionProgram,
- assertCaptionProgramForWords,
+ assertCaptionProgramForDisplay,
assertCaptionStyle,
resolveCaptionProgram,
sealCaptionProgram,
sealCaptionStyle,
} from "./style.js";
export type { CaptionStyleApplication } from "./style.js";
-export { assertTimedCaptionProjection, temporalizeCaption, temporalizeCaptionPlan } from "./temporalize.js";
+export { assertTimedCaptionProjection, temporalizeCaptionPlan } from "./temporalize.js";
export type * from "./types.js";
diff --git a/packages/caption/src/manifest.ts b/packages/caption/src/manifest.ts
index 4e3ac56b..0e26bda3 100644
--- a/packages/caption/src/manifest.ts
+++ b/packages/caption/src/manifest.ts
@@ -5,7 +5,6 @@ import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@narrata
export const captionModuleRef = { name: "@narratage/caption", version: "0.0.0-dev" } as const;
export const captionProducers = {
- temporalize: { module: captionModuleRef, name: "temporalize-caption" },
temporalizePlan: { module: captionModuleRef, name: "temporalize-caption-plan" },
} satisfies Record<string, ProducerRef>;
export const captionTypes = {
@@ -14,15 +13,14 @@ export const captionTypes = {
plan: { module: captionModuleRef, name: "CaptionPlan" },
timedProjection: { module: captionModuleRef, name: "TimedCaptionProjection" },
} satisfies Record<string, TypeRef>;
-export const captionImplementationDigest = digestOf("@narratage/caption/temporalize@1");
-export const captionPlanImplementationDigest = digestOf("@narratage/caption/temporalize-plan@1");
+export const captionPlanImplementationDigest = digestOf("@narratage/caption/temporalize-whole-atoms-plan@1");
export const captionValidatorDigests = {
style: digestOf("@narratage/caption/validate-style@1"),
- program: digestOf("@narratage/caption/validate-program@1"),
- plan: digestOf("@narratage/caption/validate-plan@1"),
- timedProjection: digestOf("@narratage/caption/validate-timed-projection@1"),
+ program: digestOf("@narratage/caption/validate-display-program@1"),
+ plan: digestOf("@narratage/caption/validate-atom-plan@1"),
+ timedProjection: digestOf("@narratage/caption/validate-timed-atoms@1"),
} as const;
-export const captionProgramSurfaceImplementationDigest = digestOf("@narratage/caption/program-surface@1");
+export const captionProgramSurfaceImplementationDigest = digestOf("@narratage/caption/display-program-surface@1");
const string = { kind: "string", minLength: 1 } as const;
const number = { kind: "number", minimum: 0 } as const;
@@ -36,7 +34,7 @@ const captionFieldAssignment = object({
});
const captionPlannedCue = object({
id: { schema: string },
- wordIds: { schema: { kind: "array", minItems: 1, items: string } },
+ atomIds: { schema: { kind: "array", minItems: 1, items: string } },
fields: { schema: { kind: "array", items: captionFieldAssignment } },
});
const captionPlannedRun = object({
@@ -48,24 +46,19 @@ export const captionPlanSchema: ValueSchema = object({
runs: { schema: { kind: "array", minItems: 1, items: captionPlannedRun } },
});
-const timedCaptionRefinement = object({
- id: { schema: string }, display: { schema: string }, displayStart: { schema: integer },
- displayEnd: { schema: integer }, sourceTokenIds: { schema: { kind: "array", items: string } },
- startSec: { schema: number }, endSec: { schema: number },
- relation: { schema: { kind: "literal", value: "exact" } },
+const timedCaptionAtom = object({
+ atomId: { schema: string }, startSec: { schema: number }, endSec: { schema: number },
});
-const timedCaptionRegion = object({
- id: { schema: string }, runId: { schema: string, optional: true }, styleId: { schema: string, optional: true },
- display: { schema: { kind: "string" } }, segmentId: { schema: string },
- kind: { schema: { kind: "string", enum: ["identity", "alias", "hidden"] } },
- sourceTokenIds: { schema: { kind: "array", items: string } }, startSec: { schema: number },
- endSec: { schema: number }, refinements: { schema: { kind: "array", items: timedCaptionRefinement } },
- wordIds: { schema: { kind: "array", items: string }, optional: true },
- fields: { schema: { kind: "array", items: captionFieldAssignment }, optional: true },
+const timedCaptionCue = object({
+ id: { schema: string }, runId: { schema: string }, styleId: { schema: string }, segmentId: { schema: string },
+ startSec: { schema: number }, endSec: { schema: number },
+ atoms: { schema: { kind: "array", minItems: 1, items: timedCaptionAtom } },
+ fields: { schema: { kind: "array", items: captionFieldAssignment } },
});
export const timedCaptionProjectionSchema: ValueSchema = object({
contract: { schema: { kind: "literal", value: "svml.timed-caption-projection@1" } },
- text: { schema: { kind: "string" } }, regions: { schema: { kind: "array", items: timedCaptionRegion } },
+ displaySequenceId: { schema: string },
+ cues: { schema: { kind: "array", minItems: 1, items: timedCaptionCue } },
});
const captionFieldValueSchema: ValueSchema = {
@@ -103,7 +96,7 @@ const captionProgramRun = object({
});
export const captionProgramSchema: ValueSchema = object({
contract: { schema: { kind: "literal", value: "svml.caption-program@1" } },
- id: { schema: string }, wordSequenceId: { schema: string }, defaultStyleId: { schema: string },
+ id: { schema: string }, displaySequenceId: { schema: string }, defaultStyleId: { schema: string },
styles: { schema: { kind: "array", minItems: 1, items: captionStyleSchema } },
runs: { schema: { kind: "array", minItems: 1, items: captionProgramRun } },
});
@@ -136,19 +129,12 @@ export const captionManifest: ModuleManifest = {
digest: captionProgramSurfaceImplementationDigest },
}],
producers: [
- {
- name: captionProducers.temporalize.name,
- inputs: [{ name: "narrative", type: narrativeTypes.narrative }, { name: "map", type: semanticMapTypes.complete }],
- outputs: [{ name: "caption", type: captionTypes.timedProjection }], needs: [],
- implementation: { kind: "registered", locator: "@narratage/caption/temporalize",
- digest: captionImplementationDigest },
- },
{
name: captionProducers.temporalizePlan.name,
inputs: [
- { name: "narrative", type: narrativeTypes.narrative },
+ { name: "display", type: narrativeTypes.captionDisplay },
+ { name: "correspondence", type: narrativeTypes.captionCorrespondence },
{ name: "map", type: semanticMapTypes.complete },
- { name: "words", type: narrativeTypes.captionWordSequence },
{ name: "program", type: captionTypes.program },
{ name: "plan", type: captionTypes.plan },
],
diff --git a/packages/caption/src/plan.ts b/packages/caption/src/plan.ts
index ce21d2e3..4bd523eb 100644
--- a/packages/caption/src/plan.ts
+++ b/packages/caption/src/plan.ts
@@ -1,18 +1,20 @@
+import type { CaptionDisplaySequence } from "@narratage/narrative";
+
import type { CaptionFieldDeclaration, CaptionPlan, CaptionProgram } from "./types.js";
function assert(condition: unknown, message: string): asserts condition {
if (!condition) throw new Error(message);
}
-function content(value: CaptionPlan) {
+function content(value: CaptionPlan): CaptionPlan {
return {
- contract: "svml.caption-plan@1" as const,
+ contract: "svml.caption-plan@1",
runs: value.runs.map((run) => ({
id: run.id,
styleId: run.styleId,
cues: run.cues.map((cue) => ({
id: cue.id,
- wordIds: [...cue.wordIds],
+ atomIds: [...cue.atomIds],
fields: cue.fields.map((field) => ({ ...field })),
})),
})),
@@ -20,8 +22,7 @@ function content(value: CaptionPlan) {
}
export function sealCaptionPlan(value: CaptionPlan): CaptionPlan {
- const normalized = content(value);
- const plan = normalized;
+ const plan = content(value);
assertCaptionPlan(plan);
return plan;
}
@@ -32,7 +33,7 @@ export function assertCaptionPlan(value: unknown): asserts value is CaptionPlan
assert(plan.contract === "svml.caption-plan@1", "Unsupported CaptionPlan contract");
assert(Array.isArray(plan.runs) && plan.runs.length > 0, "CaptionPlan runs are empty");
const runIds = new Set<string>();
- const wordIds = new Set<string>();
+ const atomIds = new Set<string>();
const cueIds = new Set<string>();
for (const run of plan.runs) {
assert(run.id.length > 0 && !runIds.has(run.id) && run.styleId.length > 0,
@@ -40,18 +41,17 @@ export function assertCaptionPlan(value: unknown): asserts value is CaptionPlan
runIds.add(run.id);
assert(run.cues.length > 0, `CaptionPlan run ${run.id} has no Cues`);
for (const cue of run.cues) {
- assert(cue.id.length > 0 && !cueIds.has(cue.id) && cue.wordIds.length > 0,
+ assert(cue.id.length > 0 && !cueIds.has(cue.id) && cue.atomIds.length > 0,
`CaptionPlan run ${run.id} contains an empty or repeated Cue`);
cueIds.add(cue.id);
- assert(new Set(cue.wordIds).size === cue.wordIds.length, `CaptionPlan Cue ${cue.id} repeats a display word`);
- for (const wordId of cue.wordIds) {
- assert(!wordIds.has(wordId), `CaptionPlan repeats display word ${wordId}`);
- wordIds.add(wordId);
+ assert(new Set(cue.atomIds).size === cue.atomIds.length, `CaptionPlan Cue ${cue.id} repeats an Atom`);
+ for (const atomId of cue.atomIds) {
+ assert(!atomIds.has(atomId), `CaptionPlan repeats Atom ${atomId}`);
+ atomIds.add(atomId);
}
for (const field of cue.fields) {
assert(field.declarationId.length > 0 && field.wordId.length > 0 && field.value.length > 0,
`CaptionPlan Cue ${cue.id} field is invalid`);
- assert(cue.wordIds.includes(field.wordId), `CaptionPlan Cue ${cue.id} field lies outside its Cue`);
}
}
}
@@ -72,27 +72,39 @@ function assertFieldValue(declaration: CaptionFieldDeclaration, value: string):
}
}
-/** Validate planner freedom against one resolved Program; no Provider-specific assumptions. */
-export function assertCaptionPlanForProgram(plan: CaptionPlan, program: CaptionProgram): void {
+/** Validate planner freedom against one resolved Program and its exact authored Atom universe. */
+export function assertCaptionPlanForProgram(
+ plan: CaptionPlan,
+ program: CaptionProgram,
+ display: CaptionDisplaySequence,
+): void {
assertCaptionPlan(plan);
+ assert(program.displaySequenceId === display.id, "CaptionPlan uses another display sequence");
assert(plan.runs.length === program.runs.length, "CaptionPlan does not contain the exact Program run set");
const styles = new Map(program.styles.map((style) => [style.id, style]));
+ const atomByWord = new Map(display.words.map((word) => [word.id, word.atomId]));
+ const atomById = new Map(display.atoms.map((atom) => [atom.id, atom]));
for (let index = 0; index < program.runs.length; index += 1) {
const expected = program.runs[index]!;
const actual = plan.runs[index]!;
assert(actual.id === expected.id && actual.styleId === expected.styleId,
`CaptionPlan run ${actual.id} differs from Program order or Style`);
- assert(actual.cues.flatMap((cue) => cue.wordIds).join("\0") === expected.wordIds.join("\0"),
- `CaptionPlan run ${actual.id} does not partition its Program words exactly in order`);
+ const expectedAtoms: string[] = [];
+ for (const wordId of expected.wordIds) {
+ const atomId = atomByWord.get(wordId);
+ assert(atomId !== undefined, `Caption Program run ${expected.id} references unknown display word ${wordId}`);
+ if (expectedAtoms.at(-1) !== atomId) expectedAtoms.push(atomId);
+ }
+ assert(expectedAtoms.length > 0 && actual.cues.flatMap((cue) => cue.atomIds).join("\0") === expectedAtoms.join("\0"),
+ `CaptionPlan run ${actual.id} does not partition its Program Atoms exactly in order`);
const style = styles.get(expected.styleId);
assert(style !== undefined, `Caption Program Style ${expected.styleId} is absent`);
const declarations = new Map(style.planning.fields.map((field) => [field.id, field]));
for (const cue of actual.cues) {
- assert(cue.wordIds.length >= style.planning.cue.minimumWords
- && cue.wordIds.length <= style.planning.cue.maximumWords,
- `Caption Cue ${cue.id} violates Style ${style.id} word-count bounds`);
+ const cueWords = new Set(cue.atomIds.flatMap((atomId) => atomById.get(atomId)?.wordIds ?? []));
const unique = new Set<string>();
for (const field of cue.fields) {
+ assert(cueWords.has(field.wordId), `Caption field ${field.declarationId} lies outside Cue ${cue.id}`);
const declaration = declarations.get(field.declarationId);
assert(declaration !== undefined, `Caption field ${field.declarationId} is not declared by Style ${style.id}`);
const key = `${field.declarationId}\0${field.wordId}`;
diff --git a/packages/caption/src/style.ts b/packages/caption/src/style.ts
index c55d00cb..28f59363 100644
--- a/packages/caption/src/style.ts
+++ b/packages/caption/src/style.ts
@@ -1,7 +1,11 @@
-import type { CaptionWord, CaptionWordSequence, CaptionWordSubset } from "@narratage/narrative";
+import type {
+ CaptionDisplaySequence,
+ CaptionDisplayWord,
+ CaptionDisplayWordSubset,
+} from "@narratage/narrative";
import { canonicalize, digestOf } from "@narratage/protocol";
-import { assertCaptionWordSequence, assertCaptionWordSubset } from "./display.js";
+import { assertCaptionDisplaySequence, assertCaptionDisplayWordSubset } from "./display.js";
import type {
CaptionFieldDeclaration,
CaptionProgram,
@@ -10,7 +14,7 @@ import type {
export type CaptionStyleApplication = {
readonly id: string;
- readonly words: CaptionWordSubset;
+ readonly words: CaptionDisplayWordSubset;
readonly style: CaptionStyleIntent;
};
@@ -80,8 +84,6 @@ export function assertCaptionStyle(value: CaptionStyleIntent): void {
`Caption Style ${value.id} field minimum is invalid`);
assert(Number.isSafeInteger(field.maximumPerCue) && field.maximumPerCue >= field.minimumPerCue,
`Caption Style ${value.id} field maximum is invalid`);
- assert(field.maximumPerCue <= cue.maximumWords,
- `Caption Style ${value.id} field maximum exceeds its Cue word maximum`);
if (field.value.kind === "enum") {
assert(field.value.values.length > 0 && new Set(field.value.values).size === field.value.values.length,
`Caption Style ${value.id} field enum is invalid`);
@@ -108,7 +110,7 @@ export function sealCaptionProgram(value: CaptionProgram): CaptionProgram {
export function assertCaptionProgram(value: CaptionProgram): void {
assert(value.contract === "svml.caption-program@1" && ID.test(value.id), "Caption Program identity is invalid");
- assert(value.wordSequenceId.length > 0 && value.runs.length > 0 && value.styles.length > 0,
+ assert(value.displaySequenceId.length > 0 && value.runs.length > 0 && value.styles.length > 0,
"Caption Program is empty");
const styles = new Map(value.styles.map((style) => [style.id, style]));
assert(styles.size === value.styles.length && styles.has(value.defaultStyleId), "Caption Program styles are invalid");
@@ -121,37 +123,35 @@ export function assertCaptionProgram(value: CaptionProgram): void {
assert(new Set(planned).size === planned.length, "Caption Program runs repeat a display word");
assert(value.runs.every((run) => styles.has(run.styleId) && run.wordIds.length > 0),
"Caption Program run style is invalid");
- for (const run of value.runs) {
- const style = styles.get(run.styleId)!;
- const fewestCues = Math.ceil(run.wordIds.length / style.planning.cue.maximumWords);
- const mostCues = Math.floor(run.wordIds.length / style.planning.cue.minimumWords);
- assert(fewestCues <= mostCues,
- `Caption Program run ${run.id} cannot satisfy Style ${style.id} Cue word bounds`);
- }
}
/** Bind a Program's immutable display universe to the exact Script-produced sequence. */
-export function assertCaptionProgramForWords(value: CaptionProgram, sequence: CaptionWordSequence): void {
+export function assertCaptionProgramForDisplay(value: CaptionProgram, sequence: CaptionDisplaySequence): void {
assertCaptionProgram(value);
- assertCaptionWordSequence(sequence);
- assert(value.wordSequenceId === sequence.id
+ assertCaptionDisplaySequence(sequence);
+ assert(value.displaySequenceId === sequence.id
&& value.runs.flatMap((run) => run.wordIds).join("\0") === sequence.words.map((word) => word.id).join("\0"),
- "Caption Program does not partition its CaptionWordSequence exactly once and in order");
+ "Caption Program does not partition its CaptionDisplaySequence exactly once and in order");
+ const runByWord = new Map(value.runs.flatMap((run) => run.wordIds.map((id) => [id, run.id] as const)));
+ for (const atom of sequence.atoms) {
+ assert(new Set(atom.wordIds.map((id) => runByWord.get(id))).size === 1,
+ `Caption Program splits indivisible Atom ${atom.id}`);
+ }
}
/** Resolve an explicit default Style plus ordered whole-Style replacements. Later applications win. */
export function resolveCaptionProgram(
- sequence: CaptionWordSequence,
+ sequence: CaptionDisplaySequence,
id: string,
defaultStyle: CaptionStyleIntent,
applications: readonly CaptionStyleApplication[],
): CaptionProgram {
- assertCaptionWordSequence(sequence);
+ assertCaptionDisplaySequence(sequence);
assertCaptionStyle(defaultStyle);
applications.forEach((application) => {
assert(ID.test(application.id), "Caption Style Application identity is invalid");
assertCaptionStyle(application.style);
- assertCaptionWordSubset(application.words, sequence);
+ assertCaptionDisplayWordSubset(application.words, sequence);
assert(application.words.wordIds.length > 0, `Caption application ${application.id} selects no visible display word`);
});
const styles = new Map<string, CaptionStyleIntent>([[defaultStyle.id, defaultStyle]]);
@@ -169,6 +169,11 @@ export function resolveCaptionProgram(
});
return { word, styleId };
});
+ const assignmentByWord = new Map(assignments.map((assignment) => [assignment.word.id, assignment.styleId]));
+ for (const atom of sequence.atoms) {
+ assert(new Set(atom.wordIds.map((wordId) => assignmentByWord.get(wordId))).size === 1,
+ `Caption Style applications split indivisible Atom ${atom.id}`);
+ }
const runs: Array<{ id: string; styleId: string; wordIds: string[]; turnId: string; segmentId: string }> = [];
for (const assignment of assignments) {
const current = runs.at(-1);
@@ -188,13 +193,13 @@ export function resolveCaptionProgram(
const program = sealCaptionProgram({
contract: "svml.caption-program@1",
id,
- wordSequenceId: sequence.id,
+ displaySequenceId: sequence.id,
defaultStyleId: defaultStyle.id,
styles: [...styles.values()],
runs: runs.map(({ turnId: _turn, segmentId: _segment, ...run }) => run),
});
- assertCaptionProgramForWords(program, sequence);
+ assertCaptionProgramForDisplay(program, sequence);
return program;
}
-export type { CaptionWord };
+export type { CaptionDisplayWord };
diff --git a/packages/caption/src/surface.ts b/packages/caption/src/surface.ts
index fa63d4e7..74387b9f 100644
--- a/packages/caption/src/surface.ts
+++ b/packages/caption/src/surface.ts
@@ -1,5 +1,5 @@
import { narrativeTypes } from "@narratage/narrative";
-import type { CaptionWordSequence, CaptionWordSubset } from "@narratage/narrative";
+import type { CaptionDisplaySequence, CaptionDisplayWordSubset } from "@narratage/narrative";
import type {
StructuredElement,
StructuredSurfaceHandler,
@@ -67,14 +67,14 @@ function localName(name: string): string {
/**
* Resolve one complete word assignment at author-compilation time. Script has already projected
- * Selection syntax to CaptionWordSubset data; this Surface lowers Role sugar from the same sequence.
+ * Selection syntax to CaptionDisplayWordSubset data; this Surface lowers Role sugar from the same sequence.
*/
export const decodeCaptionProgramSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
- attributes(element, ["id", "words", "default"]);
+ attributes(element, ["id", "display", "default"]);
const id = stringAttribute(element, "id");
- const sequence = inline<CaptionWordSequence>(
- reference(element, "words", narrativeTypes.captionWordSequence, resolveReference),
- `${element.name}.words`,
+ const sequence = inline<CaptionDisplaySequence>(
+ reference(element, "display", narrativeTypes.captionDisplay, resolveReference),
+ `${element.name}.display`,
);
const defaultStyle = inline<CaptionStyleIntent>(
reference(element, "default", captionTypes.style, resolveReference),
@@ -94,8 +94,8 @@ export const decodeCaptionProgramSurface: StructuredSurfaceHandler = ({ element,
throw new Error(`${child.name} requires exactly one of role or words`);
}
const words = role === undefined
- ? inline<CaptionWordSubset>(
- reference(child, "words", narrativeTypes.captionWordSubset, resolveReference),
+ ? inline<CaptionDisplayWordSubset>(
+ reference(child, "words", narrativeTypes.captionDisplayWordSubset, resolveReference),
`${child.name}.words`,
)
: captionWordsForRole(sequence, role);
diff --git a/packages/caption/src/temporalize.ts b/packages/caption/src/temporalize.ts
index 8661b25d..6835dcf2 100644
--- a/packages/caption/src/temporalize.ts
+++ b/packages/caption/src/temporalize.ts
@@ -1,192 +1,85 @@
-import type { CaptionWord, CaptionWordSequence, Narrative } from "@narratage/narrative";
+import type { CaptionCorrespondence, CaptionDisplaySequence } from "@narratage/narrative";
import { tokenSpanSeconds } from "@narratage/semantic-map";
import type { CompleteSemanticMap } from "@narratage/semantic-map";
-import { CaptionProjectionError } from "./error.js";
-import type {
- CaptionPlan,
- CaptionProgram,
- TimedCaptionProjection,
- TimedCaptionRefinement,
- TimedCaptionRegion,
-} from "./types.js";
+import { CaptionTimingError } from "./error.js";
+import { assertCaptionCorrespondence, assertCaptionDisplaySequence } from "./display.js";
import { assertCaptionPlanForProgram } from "./plan.js";
-import { displayTextForWords } from "./display.js";
-import { assertCaptionProgramForWords } from "./style.js";
+import { assertCaptionProgramForDisplay } from "./style.js";
+import type { CaptionPlan, CaptionProgram, TimedCaptionProjection } from "./types.js";
-export function temporalizeCaption(
- narrative: Narrative,
+/** Join whole authored Caption Atoms to measured speech time. No display-word time is invented. */
+export function temporalizeCaptionPlan(
+ display: CaptionDisplaySequence,
+ correspondence: CaptionCorrespondence,
map: CompleteSemanticMap,
+ program: CaptionProgram,
+ plan: CaptionPlan,
): TimedCaptionProjection {
- const sourceTiming = (startToken: number, endTokenExclusive: number, owner: string) => {
- const source = narrative.tokens.slice(startToken, endTokenExclusive);
- const window = tokenSpanSeconds(map, source.map((token) => token.id));
- if (window === undefined) {
- throw new CaptionProjectionError(
- "CAPTION_SPEECH_COVERAGE",
- `${owner} is not covered by the complete speech time map.`,
- );
+ assertCaptionDisplaySequence(display);
+ assertCaptionCorrespondence(correspondence, display);
+ assertCaptionProgramForDisplay(program, display);
+ assertCaptionPlanForProgram(plan, program, display);
+ const atomById = new Map(display.atoms.map((atom) => [atom.id, atom]));
+ const sourceByAtom = new Map(correspondence.atoms.map((item) => [item.atomId, item.sourceTokenIds]));
+ const cues = plan.runs.flatMap((run) => run.cues.map((cue) => {
+ const atoms = cue.atomIds.map((atomId) => {
+ const atom = atomById.get(atomId);
+ const sourceTokenIds = sourceByAtom.get(atomId);
+ if (atom === undefined || sourceTokenIds === undefined) {
+ throw new CaptionTimingError("CAPTION_ATOM", `Caption Cue ${cue.id} references unknown Atom ${atomId}.`);
+ }
+ const window = tokenSpanSeconds(map, sourceTokenIds);
+ if (window === undefined) {
+ throw new CaptionTimingError(
+ "CAPTION_SPEECH_COVERAGE",
+ `Caption Atom ${atomId} is absent from the complete speech map.`,
+ );
+ }
+ return { atom, timing: { atomId, startSec: window.startSec, endSec: window.endSec } };
+ });
+ if (new Set(atoms.map((item) => item.atom.segmentId)).size !== 1) {
+ throw new CaptionTimingError("CAPTION_PLAN_SEGMENT", `Caption Cue ${cue.id} crosses a Script Segment.`);
}
return {
- sourceTokenIds: source.map((token) => token.id),
- startSec: window.startSec,
- endSec: window.endSec,
- };
- };
-
- const regions: TimedCaptionRegion[] = narrative.captionProjection.regions.map((region) => {
- const refinements: TimedCaptionRefinement[] = region.refinements.map((refinement) => ({
- id: refinement.id,
- display: refinement.display,
- displayStart: refinement.displayStart,
- displayEnd: refinement.displayEnd,
- ...sourceTiming(refinement.startToken, refinement.endTokenExclusive, refinement.id),
- relation: refinement.relation,
- }));
- return {
- id: region.id,
- display: region.display,
- segmentId: region.segmentId,
- kind: region.kind,
- ...sourceTiming(region.startToken, region.endTokenExclusive, region.id),
- refinements,
+ id: cue.id,
+ runId: run.id,
+ styleId: run.styleId,
+ segmentId: atoms[0]!.atom.segmentId,
+ startSec: atoms[0]!.timing.startSec,
+ endSec: atoms.at(-1)!.timing.endSec,
+ atoms: atoms.map((item) => item.timing),
+ fields: cue.fields.map((field) => ({ ...field })),
};
- });
- return {
+ }));
+ const result: TimedCaptionProjection = {
contract: "svml.timed-caption-projection@1",
- text: narrative.captionProjection.text,
- regions,
+ displaySequenceId: display.id,
+ cues,
};
-}
-
-function cleanDisplay(value: string): string {
- return value.replace(/\s+/gu, " ").replace(/\s+([,.;:!?])/gu, "$1").trim();
+ assertTimedCaptionProjection(result);
+ return result;
}
export function assertTimedCaptionProjection(projection: TimedCaptionProjection): void {
- if (projection.contract !== "svml.timed-caption-projection@1") {
+ if (projection.contract !== "svml.timed-caption-projection@1" || projection.displaySequenceId.length === 0) {
throw new Error("Unsupported TimedCaptionProjection contract.");
}
-}
-
-/** Join planner-neutral Cue/field facts with the one authoritative Script map. */
-export function temporalizeCaptionPlan(
- narrative: Narrative,
- map: CompleteSemanticMap,
- sequence: CaptionWordSequence,
- program: CaptionProgram,
- plan: CaptionPlan,
-): TimedCaptionProjection {
- assertCaptionProgramForWords(program, sequence);
- assertCaptionPlanForProgram(plan, program);
- const wordById = new Map(sequence.words.map((word) => [word.id, word]));
- const regionById = new Map(narrative.captionProjection.regions.map((region) => [region.id, region]));
- const wordsByRegion = new Map<string, CaptionWord[]>();
- for (const word of sequence.words) {
- const values = wordsByRegion.get(word.regionId) ?? [];
- values.push(word);
- wordsByRegion.set(word.regionId, values);
- }
- const wordWindow = (word: CaptionWord) => {
- const sourceIds = narrative.tokens
- .slice(word.sourceTokenStart, word.sourceTokenEndExclusive)
- .map((token) => token.id);
- const window = tokenSpanSeconds(map, sourceIds);
- if (window === undefined) {
- throw new CaptionProjectionError(
- "CAPTION_SPEECH_COVERAGE",
- `Caption display word ${word.id} is absent from the complete speech map.`,
- );
- }
- if (word.correspondence === "exact") return window;
- const region = regionById.get(word.regionId);
- if (region === undefined || region.display.length === 0) {
- throw new CaptionProjectionError(
- "CAPTION_DISPLAY_REGION",
- `Caption display word ${word.id} has no owning display region.`,
- );
- }
- const siblings = wordsByRegion.get(word.regionId) ?? [];
- const next = siblings.find((candidate) => candidate.displayStart > word.displayStart);
- const duration = window.endSec - window.startSec;
- return {
- startSec: window.startSec + duration * word.displayStart / region.display.length,
- endSec: window.startSec + duration * (next?.displayStart ?? region.display.length) / region.display.length,
- };
- };
- const visible = new Set(sequence.words.map((word) => word.id));
- const plannedIds = plan.runs.flatMap((run) => run.cues.flatMap((cue) => cue.wordIds));
- if (plannedIds.length !== visible.size || new Set(plannedIds).size !== visible.size
- || plannedIds.some((id) => !visible.has(id))) {
- throw new CaptionProjectionError(
- "CAPTION_PLAN_COVERAGE",
- "CaptionPlan Cues must be disjoint and cover every display word exactly once.",
- );
- }
- const programRuns = new Map(program.runs.map((run) => [run.id, run]));
- const regions: TimedCaptionRegion[] = [];
- for (const run of plan.runs) {
- const expectedRun = programRuns.get(run.id);
- if (expectedRun === undefined || expectedRun.styleId !== run.styleId) {
- throw new CaptionProjectionError("CAPTION_PLAN_RUN", `Caption run ${run.id} differs from its Program.`);
+ const cueIds = new Set<string>();
+ const atomIds = new Set<string>();
+ for (const cue of projection.cues) {
+ if (cue.id.length === 0 || cueIds.has(cue.id) || cue.runId.length === 0 || cue.styleId.length === 0
+ || cue.segmentId.length === 0 || cue.atoms.length === 0
+ || !Number.isFinite(cue.startSec) || !Number.isFinite(cue.endSec) || cue.endSec < cue.startSec) {
+ throw new Error("TimedCaptionProjection contains an invalid Cue");
}
- const runWords = run.cues.flatMap((cue) => cue.wordIds);
- if (runWords.join("\0") !== expectedRun.wordIds.join("\0")) {
- throw new CaptionProjectionError("CAPTION_PLAN_RUN", `Caption run ${run.id} changes its Program word order.`);
- }
- for (const cue of run.cues) {
- const source = cue.wordIds.map((id) => wordById.get(id));
- if (source.some((word) => word === undefined)) {
- throw new CaptionProjectionError("CAPTION_PLAN_WORD", `Caption Cue ${cue.id} references an unknown display word.`);
- }
- const words = source as CaptionWord[];
- if (words.some((word, index) => index > 0 && word.index <= words[index - 1]!.index)) {
- throw new CaptionProjectionError("CAPTION_PLAN_ORDER", `Caption Cue ${cue.id} changes display word order.`);
- }
- if (new Set(words.map((word) => word.segmentId)).size !== 1) {
- throw new CaptionProjectionError("CAPTION_PLAN_SEGMENT", `Caption Cue ${cue.id} crosses a Script Segment.`);
+ cueIds.add(cue.id);
+ for (const atom of cue.atoms) {
+ if (atom.atomId.length === 0 || atomIds.has(atom.atomId)
+ || !Number.isFinite(atom.startSec) || !Number.isFinite(atom.endSec) || atom.endSec < atom.startSec) {
+ throw new Error("TimedCaptionProjection contains an invalid or repeated Atom timing");
}
- const sourceIndexes = [...new Set(words.flatMap((word) => Array.from(
- { length: word.sourceTokenEndExclusive - word.sourceTokenStart },
- (_, offset) => word.sourceTokenStart + offset,
- )))].sort((left, right) => left - right);
- const sourceTokens = sourceIndexes.map((index) => narrative.tokens[index]);
- if (sourceTokens.some((token) => token === undefined)
- || tokenSpanSeconds(map, sourceTokens.map((token) => token!.id)) === undefined) {
- throw new CaptionProjectionError(
- "CAPTION_SPEECH_COVERAGE",
- `Caption Cue ${cue.id} is absent from the complete speech map.`,
- );
- }
- const firstWindow = wordWindow(words[0]!);
- const lastWindow = wordWindow(words.at(-1)!);
- for (const field of cue.fields) {
- if (!cue.wordIds.includes(field.wordId)) {
- throw new CaptionProjectionError(
- "CAPTION_PLAN_FIELD",
- `Caption field ${field.declarationId} lies outside Cue ${cue.id}.`,
- );
- }
- }
- regions.push({
- id: cue.id,
- runId: run.id,
- styleId: run.styleId,
- display: displayTextForWords(narrative, sequence.words, words),
- segmentId: words[0]!.segmentId,
- kind: words.some((word) => word.correspondence === "region-envelope") ? "alias" : "identity",
- sourceTokenIds: sourceTokens.map((token) => token!.id),
- startSec: firstWindow.startSec,
- endSec: lastWindow.endSec,
- refinements: [],
- wordIds: [...cue.wordIds],
- fields: cue.fields.map((field) => ({ ...field })),
- });
+ atomIds.add(atom.atomId);
}
}
- return {
- contract: "svml.timed-caption-projection@1",
- text: cleanDisplay(regions.map((region) => region.display).join(" ")),
- regions,
- };
}
diff --git a/packages/caption/src/types.ts b/packages/caption/src/types.ts
index b5eedfee..affcb81d 100644
--- a/packages/caption/src/types.ts
+++ b/packages/caption/src/types.ts
@@ -31,7 +31,7 @@ export type CaptionFieldAssignment = {
export type CaptionPlannedCue = {
readonly id: string;
- readonly wordIds: readonly string[];
+ readonly atomIds: readonly string[];
readonly fields: readonly CaptionFieldAssignment[];
};
@@ -69,40 +69,30 @@ export type CaptionProgramRun = {
export type CaptionProgram = {
readonly contract: "svml.caption-program@1";
readonly id: string;
- readonly wordSequenceId: string;
+ readonly displaySequenceId: string;
readonly defaultStyleId: string;
readonly styles: readonly CaptionStyleIntent[];
readonly runs: readonly CaptionProgramRun[];
};
-export type TimedCaptionRefinement = {
- readonly id: string;
- readonly display: string;
- readonly displayStart: number;
- readonly displayEnd: number;
- readonly sourceTokenIds: readonly string[];
- readonly startSec: number;
- readonly endSec: number;
- readonly relation: "exact";
-};
-
-export type TimedCaptionRegion = {
+export type TimedCaptionCue = {
readonly id: string;
- readonly runId?: string;
- readonly styleId?: string;
- readonly display: string;
+ readonly runId: string;
+ readonly styleId: string;
readonly segmentId: string;
- readonly kind: "identity" | "alias" | "hidden";
- readonly sourceTokenIds: readonly string[];
readonly startSec: number;
readonly endSec: number;
- readonly refinements: readonly TimedCaptionRefinement[];
- readonly wordIds?: readonly string[];
- readonly fields?: readonly CaptionFieldAssignment[];
+ /** Only whole-Atom timing is proven. No display-word timing exists here. */
+ readonly atoms: readonly {
+ readonly atomId: string;
+ readonly startSec: number;
+ readonly endSec: number;
+ }[];
+ readonly fields: readonly CaptionFieldAssignment[];
};
export type TimedCaptionProjection = {
readonly contract: "svml.timed-caption-projection@1";
- readonly text: string;
- readonly regions: readonly TimedCaptionRegion[];
+ readonly displaySequenceId: string;
+ readonly cues: readonly TimedCaptionCue[];
};
diff --git a/packages/caption/test/caption.test.ts b/packages/caption/test/caption.test.ts
index eb45d632..11db4031 100644
--- a/packages/caption/test/caption.test.ts
+++ b/packages/caption/test/caption.test.ts
@@ -6,7 +6,6 @@ import {
resolveCaptionProgram,
sealCaptionPlan,
sealCaptionStyle,
- temporalizeCaption,
temporalizeCaptionPlan,
} from "@narratage/caption";
import type { CaptionFieldDeclaration, CaptionStyleIntent } from "@narratage/caption";
@@ -19,8 +18,9 @@ import { locateSpeechTiming } from "@narratage/speech-alignment";
import { sealAlignedTranscriptEvidence } from "@narratage/speech-evidence";
import type { AlignedTranscriptSegment } from "@narratage/speech-evidence";
import {
+ captionCorrespondence,
+ captionDisplaySequence,
captionSelectionWordSubset,
- captionWordSequence,
parseScript,
} from "@narratage/script";
@@ -70,46 +70,49 @@ function style(id: string, fields: readonly CaptionFieldDeclaration[] = []): Cap
});
}
-test("Script projects an ordered word universe and explicit Selection word subset", () => {
- const narrative = parseScript(
- "selection.svml",
- "<line>Keep this @special one sentence different @/special and return.</line>",
+test("Script emits punctuation-preserving display Words grouped into whole Atoms", () => {
+ const parsed = parseScript(
+ "punctuation.svml",
+ "<line>45% back-and-forth damn! 300,000 don't U.S.A.</line>",
);
- const words = captionWordSequence(narrative, "story.caption.words");
- const special = captionSelectionWordSubset(words, narrative.selections[0]!);
- assert.deepEqual(words.words.map((word) => word.text), ["Keep", "this", "one", "sentence", "different", "and", "return"]);
- assert.deepEqual(special.wordIds.map((id) => words.words.find((word) => word.id === id)!.text),
- ["one", "sentence", "different"]);
+ const display = captionDisplaySequence(parsed, "story.caption");
+ const correspondence = captionCorrespondence(parsed, display.id);
- const program = resolveCaptionProgram(words, "captions", style("normal"), [{
- id: "special-use",
- words: special,
- style: style("special"),
- }]);
- assert.deepEqual(program.runs.map((run) => [run.styleId, run.wordIds.length]), [
- ["normal", 2], ["special", 3], ["normal", 2],
+ assert.deepEqual(display.words.map((word) => word.text), [
+ "45%", "back-and-forth", "damn!", "300,000", "don't", "U.S.A.",
]);
- assert.equal("words" in program, false, "Program must reference the word edge rather than copy its payload");
+ assert.deepEqual(display.atoms.map((atom) => atom.wordIds.length), [1, 1, 1, 1, 1, 1]);
+ assert.deepEqual(correspondence.atoms.map((item) => item.atomId), display.atoms.map((atom) => atom.id));
});
-test("Role replacement is a word subset and the explicit default covers roleless prose", () => {
- const narrative = parseScript("roles.svml", `
- <intro>Roleless words stay readable.</intro>
- <answer><ALICE>Only this sentence changes.</answer>
- `);
- const words = captionWordSequence(narrative, "story.caption.words");
- const aliceIds = words.words.filter((word) => word.role === "ALICE").map((word) => word.id);
- const program = resolveCaptionProgram(words, "captions", style("normal"), [{
+test("Selection and Role project to ordered whole-Atom display-word subsets", () => {
+ const parsed = parseScript(
+ "selection.svml",
+ "<intro>Roleless words stay readable.</intro><answer><ALICE>Keep @special this sentence @/special different.</answer>",
+ );
+ const display = captionDisplaySequence(parsed, "story.caption");
+ const correspondence = captionCorrespondence(parsed, display.id);
+ const special = captionSelectionWordSubset(parsed, display, correspondence, parsed.selections[0]!);
+ assert.deepEqual(special.wordIds.map((id) => display.words.find((word) => word.id === id)!.text),
+ ["this", "sentence"]);
+
+ const aliceIds = display.words.filter((word) => word.role === "ALICE").map((word) => word.id);
+ const program = resolveCaptionProgram(display, "captions", style("normal"), [{
id: "alice-use",
- words: { contract: "svml.caption-word-subset@1", id: "role:ALICE", sequenceId: words.id, wordIds: aliceIds },
+ words: {
+ contract: "svml.caption-display-word-subset@1",
+ id: "role:ALICE",
+ sequenceId: display.id,
+ wordIds: aliceIds,
+ },
style: style("alice"),
}]);
assert.deepEqual(program.runs.map((run) => run.styleId), ["normal", "alice"]);
});
-test("Caption Plan validates Cue word bounds and independent per-word fields", () => {
- const narrative = parseScript("plan.svml", "<line>one two three four.</line>");
- const words = captionWordSequence(narrative, "story.caption.words");
+test("Caption Plan partitions Atoms and assigns independent fields to display Words", () => {
+ const parsed = parseScript("plan.svml", "<line>one two three four.</line>");
+ const display = captionDisplaySequence(parsed, "story.caption");
const important: CaptionFieldDeclaration = {
id: "important",
value: { kind: "boolean" },
@@ -117,46 +120,59 @@ test("Caption Plan validates Cue word bounds and independent per-word fields", (
minimumPerCue: 0,
maximumPerCue: 2,
};
- const program = resolveCaptionProgram(words, "captions", style("fine", [important]), []);
+ const program = resolveCaptionProgram(display, "captions", style("fine", [important]), []);
const run = program.runs[0]!;
const plan = sealCaptionPlan({
contract: "svml.caption-plan@1",
runs: [{ id: run.id, styleId: run.styleId, cues: [{
id: "cue:1",
- wordIds: run.wordIds,
+ atomIds: display.atoms.map((atom) => atom.id),
fields: [
- { declarationId: "important", wordId: run.wordIds[0]!, value: "true" },
- { declarationId: "important", wordId: run.wordIds[2]!, value: "true" },
+ { declarationId: "important", wordId: display.words[0]!.id, value: "true" },
+ { declarationId: "important", wordId: display.words[2]!.id, value: "true" },
],
}] }],
});
- assert.doesNotThrow(() => assertCaptionPlanForProgram(plan, program));
+ assert.doesNotThrow(() => assertCaptionPlanForProgram(plan, program, display));
});
-test("Program rejects a Style run that no legal Cue partition can satisfy", () => {
- const narrative = parseScript("impossible.svml", "<line>one two three four five six.</line>");
- const words = captionWordSequence(narrative, "story.caption.words");
- const impossible = sealCaptionStyle({
+test("Cue word bounds are planner preferences and cannot split an oversized authored Atom", () => {
+ const parsed = parseScript("impossible.svml", "<line><one two three four | something></line>");
+ const display = captionDisplaySequence(parsed, "story.caption");
+ const compact = sealCaptionStyle({
contract: "svml.caption-style@1",
- id: "impossible",
- planning: { cue: { minimumWords: 4, maximumWords: 5, instruction: "Use four or five words." }, fields: [] },
+ id: "compact",
+ planning: { cue: { minimumWords: 1, maximumWords: 3, instruction: "Prefer at most three words." }, fields: [] },
rendering: { family: "test-caption@1", parameters: {} },
});
- assert.throws(() => resolveCaptionProgram(words, "captions", impossible, []), /cannot satisfy/u);
+ const program = resolveCaptionProgram(display, "captions", compact, []);
+ assert.equal(display.atoms.length, 1);
+ assert.equal(display.atoms[0]!.wordIds.length, 4);
+ assert.doesNotThrow(() => assertCaptionPlanForProgram(sealCaptionPlan({
+ contract: "svml.caption-plan@1",
+ runs: [{ id: program.runs[0]!.id, styleId: compact.id, cues: [{
+ id: "cue:1", atomIds: [display.atoms[0]!.id], fields: [],
+ }] }],
+ }), program, display));
});
-test("planned display words join measured speech timing only after planning", () => {
- const narrative = parseScript("timed.svml", "<line><that was insane | what the fuck></line>");
- const words = captionWordSequence(narrative, "story.caption.words");
- const program = resolveCaptionProgram(words, "captions", style("fine"), []);
+test("Dual Text exposes one whole timed display Atom and never invents internal word timing", () => {
+ const parsed = parseScript("timed.svml", "<line><that was insane | what the fuck></line>");
+ const display = captionDisplaySequence(parsed, "story.caption");
+ const correspondence = captionCorrespondence(parsed, display.id);
+ assert.deepEqual(display.atoms.map((atom) => atom.wordIds.length), [3]);
+ assert.deepEqual(display.words.map((word) => word.text), ["that", "was", "insane"]);
+ assert.equal(correspondence.atoms[0]!.sourceTokenIds.length, 3);
+
+ const program = resolveCaptionProgram(display, "captions", style("fine"), []);
const run = program.runs[0]!;
const plan = sealCaptionPlan({
contract: "svml.caption-plan@1",
- runs: [{ id: run.id, styleId: run.styleId, cues: run.wordIds.map((wordId, index) => ({
- id: `cue:${index + 1}`, wordIds: [wordId], fields: [],
- })) }],
+ runs: [{ id: run.id, styleId: run.styleId, cues: [{
+ id: "cue:1", atomIds: [display.atoms[0]!.id], fields: [],
+ }] }],
});
- const map = locate(narrative, 1, [{
+ const map = locate(parsed, 1, [{
sourceSegmentId: "line", startSec: 0, endSec: 1,
words: [
{ text: "what", startSec: 0.1, endSec: 0.25 },
@@ -165,15 +181,26 @@ test("planned display words join measured speech timing only after planning", ()
],
chars: [],
}]);
- const projection = temporalizeCaptionPlan(narrative, map, words, program, plan);
- assert.deepEqual(projection.regions.map((region) => region.display), ["that", "was", "insane"]);
- assert.equal(projection.regions[0]!.endSec <= projection.regions[1]!.startSec, true);
- assert.deepEqual(projection.regions.map((region) => region.wordIds?.length), [1, 1, 1]);
- assert.equal(temporalizeCaption(narrative, map).text, "that was insane");
+ const projection = temporalizeCaptionPlan(display, correspondence, map, program, plan);
+ assert.deepEqual(projection.cues, [{
+ id: "cue:1",
+ runId: run.id,
+ styleId: run.styleId,
+ segmentId: "line",
+ startSec: 0.1,
+ endSec: 22 / 30,
+ atoms: [{ atomId: display.atoms[0]!.id, startSec: 0.1, endSec: 22 / 30 }],
+ fields: [],
+ }]);
+ assert.equal("words" in projection.cues[0]!, false);
});
-test("a Selection cutting an indivisible Dual Text word fails before Caption planning", () => {
- const narrative = parseScript("partial.svml", "<line><lmao | laughed my @part ass out @/part></line>");
- const words = captionWordSequence(narrative, "story.caption.words");
- assert.throws(() => captionSelectionWordSubset(words, narrative.selections[0]!), /owns only part/u);
+test("a Selection cutting an indivisible Dual Text Atom fails before Caption planning", () => {
+ const parsed = parseScript("partial.svml", "<line><lmao | laughed my @part ass out @/part></line>");
+ const display = captionDisplaySequence(parsed, "story.caption");
+ const correspondence = captionCorrespondence(parsed, display.id);
+ assert.throws(
+ () => captionSelectionWordSubset(parsed, display, correspondence, parsed.selections[0]!),
+ /owns only part/u,
+ );
});
diff --git a/packages/narrative/src/index.ts b/packages/narrative/src/index.ts
index 523c2e82..4004ce89 100644
--- a/packages/narrative/src/index.ts
+++ b/packages/narrative/src/index.ts
@@ -1,3 +1,3 @@
export { narrativeDependency, narrativeManifest, narrativeManifestDigest, narrativeModuleRef, narrativeTypes } from "./manifest.js";
-export { captionProjectionSchema, captionWordSequenceSchema, captionWordSubsetSchema, narrativeDialogueExcerptSchema, narrativeExcerptSchema, narrativeMomentSchema, narrativeSchema, narrativeSelectionSchema, narrativeSpeechExcerptSchema } from "./schema.js";
+export { captionCorrespondenceSchema, captionDisplaySequenceSchema, captionDisplayWordSubsetSchema, narrativeDialogueExcerptSchema, narrativeExcerptSchema, narrativeMomentSchema, narrativeSchema, narrativeSelectionSchema, narrativeSpeechExcerptSchema } from "./schema.js";
export type * from "./types.js";
diff --git a/packages/narrative/src/manifest.ts b/packages/narrative/src/manifest.ts
index e5200af9..a80f36e9 100644
--- a/packages/narrative/src/manifest.ts
+++ b/packages/narrative/src/manifest.ts
@@ -1,9 +1,9 @@
import { digestOf } from "@narratage/protocol";
import type { ModuleManifest, TypeRef } from "@narratage/protocol";
import {
- captionProjectionSchema,
- captionWordSequenceSchema,
- captionWordSubsetSchema,
+ captionCorrespondenceSchema,
+ captionDisplaySequenceSchema,
+ captionDisplayWordSubsetSchema,
narrativeDialogueExcerptSchema,
narrativeExcerptSchema,
narrativeSchema,
@@ -20,9 +20,9 @@ export const narrativeTypes = {
speechExcerpt: { module: narrativeModuleRef, name: "NarrativeSpeechExcerpt" },
selection: { module: narrativeModuleRef, name: "NarrativeSelection" },
moment: { module: narrativeModuleRef, name: "NarrativeMoment" },
- captionProjection: { module: narrativeModuleRef, name: "CaptionProjection" },
- captionWordSequence: { module: narrativeModuleRef, name: "CaptionWordSequence" },
- captionWordSubset: { module: narrativeModuleRef, name: "CaptionWordSubset" },
+ captionDisplay: { module: narrativeModuleRef, name: "CaptionDisplaySequence" },
+ captionCorrespondence: { module: narrativeModuleRef, name: "CaptionCorrespondence" },
+ captionDisplayWordSubset: { module: narrativeModuleRef, name: "CaptionDisplayWordSubset" },
} satisfies Record<string, TypeRef>;
export const narrativeManifest: ModuleManifest = {
format: "svml.module@1", name: narrativeModuleRef.name, version: narrativeModuleRef.version,
@@ -34,9 +34,9 @@ export const narrativeManifest: ModuleManifest = {
{ name: narrativeTypes.speechExcerpt.name, schema: narrativeSpeechExcerptSchema },
{ name: narrativeTypes.selection.name, schema: narrativeSelectionSchema },
{ name: narrativeTypes.moment.name, schema: narrativeMomentSchema },
- { name: narrativeTypes.captionProjection.name, schema: captionProjectionSchema },
- { name: narrativeTypes.captionWordSequence.name, schema: captionWordSequenceSchema },
- { name: narrativeTypes.captionWordSubset.name, schema: captionWordSubsetSchema },
+ { name: narrativeTypes.captionDisplay.name, schema: captionDisplaySequenceSchema },
+ { name: narrativeTypes.captionCorrespondence.name, schema: captionCorrespondenceSchema },
+ { name: narrativeTypes.captionDisplayWordSubset.name, schema: captionDisplayWordSubsetSchema },
],
capabilities: [], surfaces: [], producers: [],
};
diff --git a/packages/narrative/src/schema.ts b/packages/narrative/src/schema.ts
index 8d4d20d1..fa2382ef 100644
--- a/packages/narrative/src/schema.ts
+++ b/packages/narrative/src/schema.ts
@@ -30,20 +30,6 @@ export const narrativeSchema: ValueSchema = object({
occurrence: { schema: integer }, anchorId: { schema: string },
}) } },
}) } },
- captionProjection: { schema: object({
- contract: { schema: { kind: "literal", value: "svml.caption-projection@1" } },
- text: { schema: { kind: "string" } },
- regions: { schema: { kind: "array", items: object({
- id: { schema: string }, display: { schema: { kind: "string" } }, segmentId: { schema: string },
- startToken: { schema: integer }, endTokenExclusive: { schema: integer },
- kind: { schema: { kind: "string", enum: ["identity", "alias", "hidden"] } },
- refinements: { schema: { kind: "array", items: object({
- id: { schema: string }, display: { schema: string }, displayStart: { schema: integer },
- displayEnd: { schema: integer }, startToken: { schema: integer }, endTokenExclusive: { schema: integer },
- relation: { schema: { kind: "literal", value: "exact" } },
- }) } },
- }) } },
- }) },
semanticIndex: { schema: object({
contract: { schema: { kind: "literal", value: "svml.semantic-index@1" } },
anchors: { schema: { kind: "array", minItems: 2, items: object({
@@ -74,33 +60,30 @@ export const narrativeSpeechExcerptSchema: ValueSchema = object({
kind: { schema: { kind: "literal", value: "segment" } }, id: { schema: string },
tokenStart: { schema: integer }, tokenEndExclusive: { schema: integer }, speech: { schema: string },
});
-export const captionProjectionSchema: ValueSchema = object({
- contract: { schema: { kind: "literal", value: "svml.caption-projection@1" } },
- text: { schema: { kind: "string" } },
- regions: { schema: { kind: "array", items: object({
- id: { schema: string }, display: { schema: { kind: "string" } }, segmentId: { schema: string },
- startToken: { schema: integer }, endTokenExclusive: { schema: integer },
- kind: { schema: { kind: "string", enum: ["identity", "alias", "hidden"] } },
- refinements: { schema: { kind: "array", items: object({
- id: { schema: string }, display: { schema: string }, displayStart: { schema: integer },
- displayEnd: { schema: integer }, startToken: { schema: integer }, endTokenExclusive: { schema: integer },
- relation: { schema: { kind: "literal", value: "exact" } },
- }) } },
- }) } },
-});
-const captionWord = object({
- id: { schema: string }, index: { schema: integer }, regionId: { schema: string }, segmentId: { schema: string },
+const captionDisplayWord = object({
+ id: { schema: string }, index: { schema: integer }, atomId: { schema: string }, segmentId: { schema: string },
turnId: { schema: string }, role: { schema: string, optional: true }, text: { schema: string },
- displayStart: { schema: integer }, displayEnd: { schema: integer }, sourceTokenStart: { schema: integer },
- sourceTokenEndExclusive: { schema: integer },
- correspondence: { schema: { kind: "string", enum: ["exact", "region-envelope"] } },
});
-export const captionWordSequenceSchema: ValueSchema = object({
- contract: { schema: { kind: "literal", value: "svml.caption-word-sequence@1" } },
- id: { schema: string }, words: { schema: { kind: "array", minItems: 1, items: captionWord } },
+const captionDisplayAtom = object({
+ id: { schema: string }, index: { schema: integer }, segmentId: { schema: string },
+ turnId: { schema: string }, role: { schema: string, optional: true },
+ wordIds: { schema: { kind: "array", minItems: 1, items: string } },
+});
+export const captionDisplaySequenceSchema: ValueSchema = object({
+ contract: { schema: { kind: "literal", value: "svml.caption-display-sequence@1" } },
+ id: { schema: string },
+ atoms: { schema: { kind: "array", minItems: 1, items: captionDisplayAtom } },
+ words: { schema: { kind: "array", minItems: 1, items: captionDisplayWord } },
+});
+export const captionCorrespondenceSchema: ValueSchema = object({
+ contract: { schema: { kind: "literal", value: "svml.caption-correspondence@1" } },
+ displaySequenceId: { schema: string },
+ atoms: { schema: { kind: "array", minItems: 1, items: object({
+ atomId: { schema: string }, sourceTokenIds: { schema: { kind: "array", minItems: 1, items: string } },
+ }) } },
});
-export const captionWordSubsetSchema: ValueSchema = object({
- contract: { schema: { kind: "literal", value: "svml.caption-word-subset@1" } },
+export const captionDisplayWordSubsetSchema: ValueSchema = object({
+ contract: { schema: { kind: "literal", value: "svml.caption-display-word-subset@1" } },
id: { schema: string }, sequenceId: { schema: string },
wordIds: { schema: { kind: "array", items: string } },
});
diff --git a/packages/narrative/src/types.ts b/packages/narrative/src/types.ts
index 2bd73d0a..c9d2038b 100644
--- a/packages/narrative/src/types.ts
+++ b/packages/narrative/src/types.ts
@@ -59,61 +59,48 @@ export type NarrativeMomentRef = NarrativeMoment & {
readonly contract: "svml.narrative-moment@1";
};
-export type CaptionRefinement = {
+/** One author-visible word surface. Punctuation owned by the surface is preserved. */
+export type CaptionDisplayWord = {
readonly id: string;
- readonly display: string;
- readonly displayStart: number;
- readonly displayEnd: number;
- readonly startToken: number;
- readonly endTokenExclusive: number;
- readonly relation: "exact";
-};
-
-export type CaptionRegion = {
- readonly id: string;
- readonly display: string;
+ readonly index: number;
+ readonly atomId: string;
readonly segmentId: string;
- readonly startToken: number;
- readonly endTokenExclusive: number;
- readonly kind: "identity" | "alias" | "hidden";
- readonly refinements: readonly CaptionRefinement[];
-};
-
-export type CaptionProjection = {
- readonly contract: "svml.caption-projection@1";
+ readonly turnId: string;
+ readonly role?: string;
readonly text: string;
- readonly regions: readonly CaptionRegion[];
};
-/** Whole authored display projection. It contains no pronunciation replacement text. */
-export type CaptionProjectionRef = CaptionProjection;
-
-/** One visible display word. Pronunciation-only text never enters this value. */
-export type CaptionWord = {
+/** One indivisible Cue-planning unit. Fields may still address its ordered words. */
+export type CaptionDisplayAtom = {
readonly id: string;
readonly index: number;
- readonly regionId: string;
readonly segmentId: string;
readonly turnId: string;
readonly role?: string;
- readonly text: string;
- readonly displayStart: number;
- readonly displayEnd: number;
- readonly sourceTokenStart: number;
- readonly sourceTokenEndExclusive: number;
- readonly correspondence: "exact" | "region-envelope";
+ readonly wordIds: readonly string[];
};
-/** The complete ordered visible-word universe for one Script Caption projection. */
-export type CaptionWordSequence = {
- readonly contract: "svml.caption-word-sequence@1";
+/** Complete visible Caption truth. It contains no pronunciation or timing facts. */
+export type CaptionDisplaySequence = {
+ readonly contract: "svml.caption-display-sequence@1";
readonly id: string;
- readonly words: readonly CaptionWord[];
+ readonly atoms: readonly CaptionDisplayAtom[];
+ readonly words: readonly CaptionDisplayWord[];
+};
+
+/** Author-declared whole-Atom correspondence to spoken Script tokens; never an inferred refinement. */
+export type CaptionCorrespondence = {
+ readonly contract: "svml.caption-correspondence@1";
+ readonly displaySequenceId: string;
+ readonly atoms: readonly {
+ readonly atomId: string;
+ readonly sourceTokenIds: readonly string[];
+ }[];
};
-/** One ordered subset of an exact CaptionWordSequence, projected by Script structure. */
-export type CaptionWordSubset = {
- readonly contract: "svml.caption-word-subset@1";
+/** One ordered subset of an exact CaptionDisplaySequence, projected by Script structure. */
+export type CaptionDisplayWordSubset = {
+ readonly contract: "svml.caption-display-word-subset@1";
readonly id: string;
readonly sequenceId: string;
readonly wordIds: readonly string[];
@@ -134,7 +121,6 @@ export type Narrative = {
readonly turns: readonly NarrativeTurn[];
readonly selections: readonly NarrativeSelection[];
readonly moments: readonly NarrativeMoment[];
- readonly captionProjection: CaptionProjection;
readonly semanticIndex: {
readonly contract: "svml.semantic-index@1";
readonly anchors: readonly SemanticAnchor[];
diff --git a/packages/provider-google-vertex/src/provider.ts b/packages/provider-google-vertex/src/provider.ts
index fb758b07..cdf5fe66 100644
--- a/packages/provider-google-vertex/src/provider.ts
+++ b/packages/provider-google-vertex/src/provider.ts
@@ -19,7 +19,7 @@ export const googleVertexProviderModuleRef = {
version: "0.0.0-dev",
} as const;
export const googleVertexProviderImplementationDigest = digestOf(
- "@narratage/provider-google-vertex/caption-gemini@1:@google/[email protected]",
+ "@narratage/provider-google-vertex/caption-atom-word-coordinates@1:@google/[email protected]",
);
type GenerateCaptionContentInput = {
@@ -61,34 +61,34 @@ const captionResponseSchema: Schema = {
items: {
type: Type.OBJECT,
properties: {
- run_id: { type: Type.STRING },
cues: {
type: Type.ARRAY,
items: {
type: Type.OBJECT,
properties: {
- after_word_id: { type: Type.STRING },
+ atom_count: { type: Type.INTEGER, minimum: 1 },
fields: {
type: Type.ARRAY,
items: {
type: Type.OBJECT,
properties: {
declaration_id: { type: Type.STRING },
- word_id: { type: Type.STRING },
+ atom_number: { type: Type.INTEGER, minimum: 1 },
+ word_number: { type: Type.INTEGER, minimum: 1 },
value: { type: Type.STRING },
},
- required: ["declaration_id", "word_id", "value"],
- propertyOrdering: ["declaration_id", "word_id", "value"],
+ required: ["declaration_id", "atom_number", "word_number", "value"],
+ propertyOrdering: ["declaration_id", "atom_number", "word_number", "value"],
},
},
},
- required: ["after_word_id", "fields"],
- propertyOrdering: ["after_word_id", "fields"],
+ required: ["atom_count", "fields"],
+ propertyOrdering: ["atom_count", "fields"],
},
},
},
- required: ["run_id", "cues"],
- propertyOrdering: ["run_id", "cues"],
+ required: ["cues"],
+ propertyOrdering: ["cues"],
},
},
},
diff --git a/packages/provider-google-vertex/test/provider.test.ts b/packages/provider-google-vertex/test/provider.test.ts
index be874440..29d3d53b 100644
--- a/packages/provider-google-vertex/test/provider.test.ts
+++ b/packages/provider-google-vertex/test/provider.test.ts
@@ -21,11 +21,11 @@ import {
googleVertexProviderImplementationDigest,
} from "@narratage/provider-google-vertex";
import type { GenerateCaptionContent } from "@narratage/provider-google-vertex";
-import { captionWordSequence, parseScript } from "@narratage/script";
+import { captionDisplaySequence, parseScript } from "@narratage/script";
function request(): CaptionGeminiRequest {
const narrative = parseScript("provider.svml", "<line><ALICE>Meaning becomes the source.</line>");
- const words = captionWordSequence(narrative, "story.caption.words");
+ const display = captionDisplaySequence(narrative, "story.caption");
const style = sealCaptionStyle({
contract: "svml.caption-style@1",
id: "important",
@@ -41,8 +41,8 @@ function request(): CaptionGeminiRequest {
},
rendering: { family: "test-caption@1", parameters: {} },
});
- const captionProgram = resolveCaptionProgram(words, "captions", style, []);
- return compileCaptionGeminiRequest(words, captionProgram, sealCaptionGeminiProgram({
+ const captionProgram = resolveCaptionProgram(display, "captions", style, []);
+ return compileCaptionGeminiRequest(display, captionProgram, sealCaptionGeminiProgram({
contract: "svml.caption-gemini-program@1",
model: "gemini-2.5-flash",
}));
@@ -50,10 +50,9 @@ function request(): CaptionGeminiRequest {
function response(requestValue: CaptionGeminiRequest): RawCaptionGeminiResponse {
return { runs: requestValue.runs.map((run) => ({
- run_id: run.id,
cues: [{
- after_word_id: run.wordIds.at(-1)!,
- fields: [{ declaration_id: "important", word_id: run.wordIds[0]!, value: "important" }],
+ atom_count: run.atoms.length,
+ fields: [{ declaration_id: "important", atom_number: 1, word_number: 1, value: "important" }],
}],
})) };
}
diff --git a/packages/script/README.md b/packages/script/README.md
index 1c4f130e..9336b1d8 100644
--- a/packages/script/README.md
+++ b/packages/script/README.md
@@ -41,10 +41,11 @@ The Surface exports one full Narrative plus narrow, immutable views:
- `script.segment.<id>.dialogue` is display-independent dialogue, including optional Role cues and
right-side Dual Text pronunciation, for a speech-video model;
- `script.segment.<id>.speech` is pronunciation-only text for duration estimation;
-- `script.caption` is the whole left-side `CaptionProjection` and is not split by Segment for the
- Caption planner;
+- `script.caption` is the complete left-side `CaptionDisplaySequence`;
+- `script.caption.correspondence` maps each whole display Atom to its authored speech-token range;
+- `script.caption.selection.<id>` is the display-word subset wholly owned by one Selection;
- `script.selection.<id>` is a reusable explicit Selection.
Seedance consumes the dialogue excerpt, Speech Spine consumes the Segment excerpt, and Caption
-consumes the whole Narrative/Caption projection. None imports Script's parser AST. Another authoring
-package may produce the same contracts.
+consumes the explicit display and correspondence edges. None imports Script's parser AST. Another
+authoring package may produce the same contracts.
diff --git a/packages/script/src/index.ts b/packages/script/src/index.ts
index 222917db..59a17b6f 100644
--- a/packages/script/src/index.ts
+++ b/packages/script/src/index.ts
@@ -1,9 +1,9 @@
export { ScriptSyntaxError } from "./error.js";
export { formatScript } from "./format.js";
export {
- captionProjectionType,
- captionWordSequenceType,
- captionWordSubsetType,
+ captionCorrespondenceType,
+ captionDisplayType,
+ captionDisplayWordSubsetType,
narrativeDialogueExcerptType,
narrativeExcerptType,
narrativeMomentType,
@@ -16,11 +16,12 @@ export {
scriptSurfaceImplementationDigest,
} from "./manifest.js";
export {
- captionProjectionValue,
+ captionCorrespondence,
+ captionCorrespondenceValue,
+ captionDisplaySequence,
+ captionDisplaySequenceValue,
captionSelectionWordSubset,
captionSelectionWordSubsetValue,
- captionWordSequence,
- captionWordSequenceValue,
narrativeDialogueExcerptValue,
narrativeSegmentExcerptValue,
narrativeMomentValue,
diff --git a/packages/script/src/manifest.ts b/packages/script/src/manifest.ts
index 706b9e71..ad1fe2b9 100644
--- a/packages/script/src/manifest.ts
+++ b/packages/script/src/manifest.ts
@@ -9,12 +9,12 @@ export const narrativeDialogueExcerptType: TypeRef = narrativeTypes.dialogueExce
export const narrativeSpeechExcerptType: TypeRef = narrativeTypes.speechExcerpt;
export const narrativeSelectionType: TypeRef = narrativeTypes.selection;
export const narrativeMomentType: TypeRef = narrativeTypes.moment;
-export const captionProjectionType: TypeRef = narrativeTypes.captionProjection;
-export const captionWordSequenceType: TypeRef = narrativeTypes.captionWordSequence;
-export const captionWordSubsetType: TypeRef = narrativeTypes.captionWordSubset;
+export const captionDisplayType: TypeRef = narrativeTypes.captionDisplay;
+export const captionCorrespondenceType: TypeRef = narrativeTypes.captionCorrespondence;
+export const captionDisplayWordSubsetType: TypeRef = narrativeTypes.captionDisplayWordSubset;
export { narrativeSchema };
-export const scriptSurfaceImplementationDigest = digestOf("@narratage/script/surface@1");
+export const scriptSurfaceImplementationDigest = digestOf("@narratage/script/display-atoms-surface@1");
export const scriptManifest: ModuleManifest = {
format: "svml.module@1",
@@ -35,9 +35,9 @@ export const scriptManifest: ModuleManifest = {
narrativeSpeechExcerptType,
narrativeSelectionType,
narrativeMomentType,
- captionProjectionType,
- captionWordSequenceType,
- captionWordSubsetType,
+ captionDisplayType,
+ captionCorrespondenceType,
+ captionDisplayWordSubsetType,
],
implementation: {
kind: "trusted-frontend-surface",
diff --git a/packages/script/src/narrative.ts b/packages/script/src/narrative.ts
index 360550eb..ab378f2a 100644
--- a/packages/script/src/narrative.ts
+++ b/packages/script/src/narrative.ts
@@ -1,23 +1,54 @@
import { canonicalize } from "@narratage/core";
import type { CanonicalValue } from "@narratage/protocol";
-import type { CaptionRegion, CaptionWord, CaptionWordSequence, CaptionWordSubset } from "@narratage/narrative";
+import type {
+ CaptionCorrespondence,
+ CaptionDisplayAtom,
+ CaptionDisplaySequence,
+ CaptionDisplayWord,
+ CaptionDisplayWordSubset,
+} from "@narratage/narrative";
-import type { ParsedNarrative } from "./types.js";
+import type { ParsedCaptionRegion, ParsedNarrative } from "./types.js";
-const DISPLAY_WORD =
+const LEXICAL_WORD =
/[\p{Script=Han}\p{Script=Hiragana}\p{Script=Katakana}]\p{M}*|[\p{L}\p{M}\p{N}]+(?:['’.-][\p{L}\p{M}\p{N}]+)*/gu;
+const DISPLAY_SURFACE = /\S+/gu;
-function displayWords(value: string): Array<{ readonly text: string; readonly start: number; readonly end: number }> {
- const result: Array<{ text: string; start: number; end: number }> = [];
- DISPLAY_WORD.lastIndex = 0;
+function lexicalCount(value: string): number {
+ LEXICAL_WORD.lastIndex = 0;
+ let count = 0;
+ while (LEXICAL_WORD.exec(value)) count += 1;
+ return count;
+}
+
+/**
+ * The official Script reader is English-first: authored whitespace is the primary display-word
+ * boundary and punctuation inside a surface is preserved. A punctuation-only surface is attached
+ * to its left neighbour; a leading one is attached to the first following word. This is lexical
+ * ownership, not semantic interpretation.
+ */
+function displayWords(value: string): string[] {
+ const chunks: string[] = [];
+ DISPLAY_SURFACE.lastIndex = 0;
let match: RegExpExecArray | null;
- while ((match = DISPLAY_WORD.exec(value))) {
- result.push({ text: match[0], start: match.index, end: match.index + match[0].length });
+ while ((match = DISPLAY_SURFACE.exec(value))) chunks.push(match[0]);
+ const result: string[] = [];
+ let leading = "";
+ for (const chunk of chunks) {
+ if (lexicalCount(chunk) > 0) {
+ result.push(leading ? `${leading} ${chunk}` : chunk);
+ leading = "";
+ } else if (result.length > 0) {
+ result[result.length - 1] = `${result.at(-1)!} ${chunk}`;
+ } else {
+ leading = leading ? `${leading} ${chunk}` : chunk;
+ }
}
+ if (leading) result.push(leading);
return result;
}
-function turnForRegion(parsed: ParsedNarrative, region: CaptionRegion): ParsedNarrative["turns"][number] {
+function turnForRegion(parsed: ParsedNarrative, region: ParsedCaptionRegion): ParsedNarrative["turns"][number] {
const turn = parsed.turns.find((candidate) =>
candidate.segmentId === region.segmentId
&& candidate.tokenStart < region.endTokenExclusive
@@ -26,74 +57,134 @@ function turnForRegion(parsed: ParsedNarrative, region: CaptionRegion): ParsedNa
return turn;
}
-/** Script owns display-word projection because only it still knows authored source structure. */
-export function captionWordSequence(parsed: ParsedNarrative, id: string): CaptionWordSequence {
- const words: CaptionWord[] = [];
+function projectCaption(
+ parsed: ParsedNarrative,
+ id: string,
+): { readonly display: CaptionDisplaySequence; readonly correspondence: CaptionCorrespondence } {
+ const atoms: CaptionDisplayAtom[] = [];
+ const words: CaptionDisplayWord[] = [];
+ const correspondence: Array<{ atomId: string; sourceTokenIds: string[] }> = [];
for (const region of parsed.captionProjection.regions) {
if (region.kind === "hidden") continue;
const turn = turnForRegion(parsed, region);
- for (const word of displayWords(region.display)) {
- const exact = region.refinements.find((refinement) =>
- refinement.displayStart === word.start && refinement.displayEnd === word.end);
- words.push({
- id: `${region.id}:word:${words.length + 1}`,
- index: words.length,
- regionId: region.id,
+ const surfaces = displayWords(region.display);
+ if (surfaces.length === 0) throw new Error(`Caption region ${region.id} contains no visible display surface`);
+ const groups = region.kind === "alias" ? [surfaces] : surfaces.map((surface) => [surface]);
+ let sourceCursor = region.startToken;
+ for (const group of groups) {
+ const atomId = `${id}:atom:${atoms.length + 1}`;
+ const atomWordIds = group.map((surface, groupIndex) => {
+ const wordId = `${atomId}:word:${groupIndex + 1}`;
+ words.push({
+ id: wordId,
+ index: words.length,
+ atomId,
+ segmentId: region.segmentId,
+ turnId: turn.id,
+ ...(turn.role === undefined ? {} : { role: turn.role }),
+ text: surface,
+ });
+ return wordId;
+ });
+ atoms.push({
+ id: atomId,
+ index: atoms.length,
segmentId: region.segmentId,
turnId: turn.id,
...(turn.role === undefined ? {} : { role: turn.role }),
- text: word.text,
- displayStart: word.start,
- displayEnd: word.end,
- sourceTokenStart: exact?.startToken ?? region.startToken,
- sourceTokenEndExclusive: exact?.endTokenExclusive ?? region.endTokenExclusive,
- correspondence: exact === undefined ? "region-envelope" : "exact",
+ wordIds: atomWordIds,
});
+ const sourceEnd = region.kind === "alias"
+ ? region.endTokenExclusive
+ : sourceCursor + group.reduce((count, surface) => count + lexicalCount(surface), 0);
+ const sourceTokenIds = parsed.tokens.slice(sourceCursor, sourceEnd).map((token) => token.id);
+ if (sourceTokenIds.length === 0) throw new Error(`Caption Atom ${atomId} has no authored speech correspondence`);
+ correspondence.push({ atomId, sourceTokenIds });
+ sourceCursor = sourceEnd;
+ }
+ if (sourceCursor !== region.endTokenExclusive) {
+ throw new Error(`Caption identity region ${region.id} does not structurally partition its authored speech`);
}
}
- if (words.length === 0) throw new Error("Caption display projection contains no visible words");
- return { contract: "svml.caption-word-sequence@1", id, words };
+ if (atoms.length === 0 || words.length === 0) throw new Error("Caption display contains no visible words");
+ return {
+ display: {
+ contract: "svml.caption-display-sequence@1",
+ id,
+ atoms,
+ words,
+ },
+ correspondence: {
+ contract: "svml.caption-correspondence@1",
+ displaySequenceId: id,
+ atoms: correspondence,
+ },
+ };
+}
+
+export function captionDisplaySequence(parsed: ParsedNarrative, id: string): CaptionDisplaySequence {
+ return projectCaption(parsed, id).display;
+}
+
+export function captionCorrespondence(parsed: ParsedNarrative, id: string): CaptionCorrespondence {
+ return projectCaption(parsed, id).correspondence;
}
-/** Project one authored Selection into visible words before public values lose source positions. */
+/** Project one authored Selection into whole visible Atoms before public values lose source positions. */
export function captionSelectionWordSubset(
- sequence: CaptionWordSequence,
+ parsed: ParsedNarrative,
+ sequence: CaptionDisplaySequence,
+ correspondence: CaptionCorrespondence,
selection: ParsedNarrative["selections"][number],
-): CaptionWordSubset {
+): CaptionDisplayWordSubset {
const ranges = selection.occurrences.map((occurrence) => ({
start: occurrence.open.boundary.tokenIndex,
endExclusive: occurrence.close.boundary.tokenIndex,
}));
+ const tokenIndex = new Map(parsed.tokens.map((token) => [token.id, token.index]));
+ const atomById = new Map(sequence.atoms.map((atom) => [atom.id, atom]));
const wordIds: string[] = [];
- for (const word of sequence.words) {
+ for (const mapping of correspondence.atoms) {
+ const indexes = mapping.sourceTokenIds.map((id) => tokenIndex.get(id));
+ if (indexes.some((index) => index === undefined)) throw new Error(`Caption Atom ${mapping.atomId} references an unknown speech token`);
+ const start = Math.min(...indexes as number[]);
+ const endExclusive = Math.max(...indexes as number[]) + 1;
const intersects = ranges.some((range) =>
- word.sourceTokenStart < range.endExclusive && word.sourceTokenEndExclusive > range.start);
+ start < range.endExclusive && endExclusive > range.start);
const contained = ranges.some((range) =>
- word.sourceTokenStart >= range.start && word.sourceTokenEndExclusive <= range.endExclusive);
+ start >= range.start && endExclusive <= range.endExclusive);
if (intersects && !contained) {
- throw new Error(
- `Selection ${selection.id} owns only part of Caption word ${word.id}; split the Dual Text atom`,
- );
+ throw new Error(`Selection ${selection.id} owns only part of Caption Atom ${mapping.atomId}`);
+ }
+ if (contained) {
+ const atom = atomById.get(mapping.atomId);
+ if (atom === undefined) throw new Error(`Caption correspondence references unknown Atom ${mapping.atomId}`);
+ wordIds.push(...atom.wordIds);
}
- if (contained) wordIds.push(word.id);
}
return {
- contract: "svml.caption-word-subset@1",
+ contract: "svml.caption-display-word-subset@1",
id: selection.id,
sequenceId: sequence.id,
wordIds,
};
}
-export function captionWordSequenceValue(parsed: ParsedNarrative, id: string): CanonicalValue {
- return canonicalize(captionWordSequence(parsed, id));
+export function captionDisplaySequenceValue(parsed: ParsedNarrative, id: string): CanonicalValue {
+ return canonicalize(captionDisplaySequence(parsed, id));
+}
+
+export function captionCorrespondenceValue(parsed: ParsedNarrative, id: string): CanonicalValue {
+ return canonicalize(captionCorrespondence(parsed, id));
}
export function captionSelectionWordSubsetValue(
- sequence: CaptionWordSequence,
+ parsed: ParsedNarrative,
+ sequence: CaptionDisplaySequence,
+ correspondence: CaptionCorrespondence,
selection: ParsedNarrative["selections"][number],
): CanonicalValue {
- return canonicalize(captionSelectionWordSubset(sequence, selection));
+ return canonicalize(captionSelectionWordSubset(parsed, sequence, correspondence, selection));
}
function cleanProjection(value: string): string {
@@ -193,15 +284,6 @@ export function narrativeSpeechExcerptValue(
return canonicalize(content);
}
-export function captionProjectionValue(parsed: ParsedNarrative): CanonicalValue {
- const content = {
- contract: parsed.captionProjection.contract,
- text: parsed.captionProjection.text,
- regions: parsed.captionProjection.regions.map(({ range: _range, ...region }) => region),
- } as const;
- return canonicalize(content);
-}
-
export function narrativeSelectionValue(selection: ParsedNarrative["selections"][number]): CanonicalValue {
const content = {
contract: "svml.narrative-selection@1",
@@ -270,27 +352,6 @@ export function narrativeValue(parsed: ParsedNarrative): CanonicalValue {
anchorId: occurrence.boundary.anchorId,
})),
})),
- captionProjection: {
- contract: parsed.captionProjection.contract,
- text: parsed.captionProjection.text,
- regions: parsed.captionProjection.regions.map((region) => ({
- id: region.id,
- display: region.display,
- segmentId: region.segmentId,
- startToken: region.startToken,
- endTokenExclusive: region.endTokenExclusive,
- kind: region.kind,
- refinements: region.refinements.map((refinement) => ({
- id: refinement.id,
- display: refinement.display,
- displayStart: refinement.displayStart,
- displayEnd: refinement.displayEnd,
- startToken: refinement.startToken,
- endTokenExclusive: refinement.endTokenExclusive,
- relation: refinement.relation,
- })),
- })),
- },
semanticIndex: parsed.semanticIndex,
serializations: parsed.serializations,
});
diff --git a/packages/script/src/parser.ts b/packages/script/src/parser.ts
index e9a27727..81c87e23 100644
--- a/packages/script/src/parser.ts
+++ b/packages/script/src/parser.ts
@@ -5,7 +5,6 @@ import type {
Affinity,
MarkerBoundary,
ParsedAtom,
- ParsedCaptionRefinement,
ParsedCaptionRegion,
ParsedMoment,
ParsedMomentOccurrence,
@@ -58,89 +57,6 @@ function normalizeWord(value: string): string {
.replace(/[^\p{L}\p{M}\p{N}]+/gu, "");
}
-type WordLexeme = {
- readonly text: string;
- readonly normalized: string;
- readonly start: number;
- readonly end: number;
-};
-
-function wordLexemes(value: string): WordLexeme[] {
- const lexemes: WordLexeme[] = [];
- WORD.lastIndex = 0;
- let match: RegExpExecArray | null;
- while ((match = WORD.exec(value))) {
- const normalized = normalizeWord(match[0]);
- if (!normalized) continue;
- lexemes.push({
- text: match[0],
- normalized,
- start: match.index,
- end: match.index + match[0].length,
- });
- }
- return lexemes;
-}
-
-function exactCaptionRefinements(
- regionId: string,
- display: string,
- speechTokens: readonly ParsedToken[],
-): ParsedCaptionRefinement[] {
- const displayWords = wordLexemes(display);
- if (!displayWords.length || !speechTokens.length) return [];
-
- let matches: Array<{ readonly displayIndex: number; readonly speechIndex: number }>;
- if (
- displayWords.length === speechTokens.length
- && displayWords.every((word, index) => word.normalized === speechTokens[index]!.normalized)
- ) {
- matches = displayWords.map((_, index) => ({ displayIndex: index, speechIndex: index }));
- } else {
- const displayCounts = new Map<string, number>();
- const speechCounts = new Map<string, number>();
- for (const word of displayWords) displayCounts.set(word.normalized, (displayCounts.get(word.normalized) ?? 0) + 1);
- for (const token of speechTokens) speechCounts.set(token.normalized, (speechCounts.get(token.normalized) ?? 0) + 1);
- const candidates = displayWords.flatMap((word, displayIndex) => {
- if (displayCounts.get(word.normalized) !== 1 || speechCounts.get(word.normalized) !== 1) return [];
- const speechIndex = speechTokens.findIndex((token) => token.normalized === word.normalized);
- return speechIndex < 0 ? [] : [{ displayIndex, speechIndex }];
- });
-
- // Keep a longest monotonic chain. Exact words that cross after an alias
- // rewrite are individually plausible, but cannot form a temporal caption
- // refinement without reversing display order.
- const chains: Array<Array<{ readonly displayIndex: number; readonly speechIndex: number }>> = [];
- for (let index = 0; index < candidates.length; index += 1) {
- let best: Array<{ readonly displayIndex: number; readonly speechIndex: number }> = [];
- for (let before = 0; before < index; before += 1) {
- if (candidates[before]!.speechIndex < candidates[index]!.speechIndex && chains[before]!.length > best.length) {
- best = chains[before]!;
- }
- }
- chains.push([...best, candidates[index]!]);
- }
- matches = chains.reduce<typeof candidates>(
- (best, chain) => chain.length > best.length ? chain : best,
- [],
- );
- }
-
- return matches.map(({ displayIndex, speechIndex }, index) => {
- const word = displayWords[displayIndex]!;
- const token = speechTokens[speechIndex]!;
- return {
- id: `${regionId}:exact:${index + 1}`,
- display: word.text,
- displayStart: word.start,
- displayEnd: word.end,
- startToken: token.index,
- endTokenExclusive: token.index + 1,
- relation: "exact",
- };
- });
-}
-
function cleanProjection(value: string): string {
return value
.replace(/\s+/gu, " ")
@@ -418,7 +334,6 @@ export function parseScript(
startToken: tokenStart,
endTokenExclusive: tokens.length,
kind: "identity",
- refinements: exactCaptionRefinements(id, display, tokens.slice(tokenStart)),
range: { start: sourceOffset + start, end: sourceOffset + end },
});
}
@@ -461,15 +376,13 @@ export function parseScript(
if (tokens.length > startToken) {
const display = cleanProjection(caption);
const id = `caption-region:${captionRegions.length + 1}`;
- const speechTokens = tokens.slice(startToken);
captionRegions.push({
id,
display,
segmentId: current!.id,
startToken,
endTokenExclusive: tokens.length,
- kind: display ? (cleanProjection(speechTokens.map((token) => token.text).join(" ")) === display ? "identity" : "alias") : "hidden",
- refinements: exactCaptionRefinements(id, display, speechTokens),
+ kind: display ? "alias" : "hidden",
range: { start: sourceOffset + absoluteStart, end: sourceOffset + absoluteStart + raw.length },
});
}
@@ -753,7 +666,6 @@ export function parseScript(
}),
})),
captionProjection: {
- contract: "svml.caption-projection@1",
text: captionSegments.join("\n"),
regions: captionRegions,
},
diff --git a/packages/script/src/surface.ts b/packages/script/src/surface.ts
index f252db5d..b37de41b 100644
--- a/packages/script/src/surface.ts
+++ b/packages/script/src/surface.ts
@@ -2,9 +2,11 @@ import { canonicalize } from "@narratage/core";
import { ScriptSyntaxError } from "./error.js";
import {
- captionProjectionValue,
+ captionCorrespondence,
+ captionCorrespondenceValue,
+ captionDisplaySequence,
+ captionDisplaySequenceValue,
captionSelectionWordSubsetValue,
- captionWordSequence,
narrativeDialogueExcerptValue,
narrativeSegmentExcerptValue,
narrativeMomentValue,
@@ -14,9 +16,9 @@ import {
narrativeValue,
} from "./narrative.js";
import {
- captionProjectionType,
- captionWordSequenceType,
- captionWordSubsetType,
+ captionCorrespondenceType,
+ captionDisplayType,
+ captionDisplayWordSubsetType,
narrativeDialogueExcerptType,
narrativeExcerptType,
narrativeMomentType,
@@ -81,8 +83,9 @@ export function decodeScriptSurface(input: ScriptSurfaceInput): ScriptSurfaceOut
input.source.slice(input.contentStart, close.start),
input.contentStart,
);
- const wordsId = `${rawId}.caption.words`;
- const words = captionWordSequence(parsed, wordsId);
+ const displayId = `${rawId}.caption`;
+ const display = captionDisplaySequence(parsed, displayId);
+ const correspondence = captionCorrespondence(parsed, displayId);
return {
nextOffset: close.end,
records: [
@@ -113,21 +116,26 @@ export function decodeScriptSurface(input: ScriptSurfaceInput): ScriptSurfaceOut
},
]),
{
- id: `${rawId}.caption`,
- type: captionProjectionType,
- value: { kind: "inline" as const, value: captionProjectionValue(parsed) },
+ id: displayId,
+ type: captionDisplayType,
+ value: { kind: "inline" as const, value: captionDisplaySequenceValue(parsed, displayId) },
range: { start: input.openingStart, end: close.end },
},
{
- id: wordsId,
- type: captionWordSequenceType,
- value: { kind: "inline" as const, value: canonicalize(words) },
+ id: `${displayId}.correspondence`,
+ type: captionCorrespondenceType,
+ value: { kind: "inline" as const, value: captionCorrespondenceValue(parsed, displayId) },
range: { start: input.openingStart, end: close.end },
},
...parsed.selections.map((selection) => ({
id: `${rawId}.caption.selection.${selection.id}`,
- type: captionWordSubsetType,
- value: { kind: "inline" as const, value: captionSelectionWordSubsetValue(words, selection) },
+ type: captionDisplayWordSubsetType,
+ value: { kind: "inline" as const, value: captionSelectionWordSubsetValue(
+ parsed,
+ display,
+ correspondence,
+ selection,
+ ) },
range: {
start: selection.occurrences[0]!.open.range.start,
end: selection.occurrences.at(-1)!.close.range.end,
diff --git a/packages/script/src/types.ts b/packages/script/src/types.ts
index 29d1bcc5..429ea71e 100644
--- a/packages/script/src/types.ts
+++ b/packages/script/src/types.ts
@@ -1,7 +1,7 @@
import type { CanonicalValue, SourceRange, StoredValue, TypeRef } from "@narratage/protocol";
-import type { CaptionProjection, CaptionRefinement, CaptionRegion, Narrative, NarrativeMomentOccurrence, NarrativeSegment, NarrativeSelectionOccurrence, NarrativeToken, NarrativeTurn } from "@narratage/narrative";
+import type { Narrative, NarrativeMomentOccurrence, NarrativeSegment, NarrativeSelectionOccurrence, NarrativeToken, NarrativeTurn } from "@narratage/narrative";
-export type { CaptionProjection, Narrative, SemanticAnchor } from "@narratage/narrative";
+export type { Narrative, SemanticAnchor } from "@narratage/narrative";
export type Affinity = "left" | "right";
@@ -59,19 +59,27 @@ export type ParsedMoment = {
readonly occurrences: readonly ParsedMomentOccurrence[];
};
-export type ParsedCaptionRefinement = CaptionRefinement;
-export type ParsedCaptionRegion = CaptionRegion & { readonly range: SourceRange };
+export type ParsedCaptionRegion = {
+ readonly id: string;
+ readonly display: string;
+ readonly segmentId: string;
+ readonly startToken: number;
+ readonly endTokenExclusive: number;
+ readonly kind: "identity" | "alias" | "hidden";
+ readonly range: SourceRange;
+};
export type ParsedNarrative = Omit<
Narrative,
- "segments" | "tokens" | "turns" | "selections" | "moments" | "captionProjection"
+ "segments" | "tokens" | "turns" | "selections" | "moments"
> & {
readonly segments: readonly ParsedSegment[];
readonly tokens: readonly ParsedToken[];
readonly turns: readonly ParsedTurn[];
readonly selections: readonly ParsedSelection[];
readonly moments: readonly ParsedMoment[];
- readonly captionProjection: Omit<CaptionProjection, "regions"> & {
+ readonly captionProjection: {
+ readonly text: string;
readonly regions: readonly ParsedCaptionRegion[];
};
};
diff --git a/packages/script/test/script.test.ts b/packages/script/test/script.test.ts
index b81571b0..07b03c1b 100644
--- a/packages/script/test/script.test.ts
+++ b/packages/script/test/script.test.ts
@@ -6,6 +6,8 @@ import { digestOf } from "@narratage/core";
import type { Narrative } from "@narratage/narrative";
import {
ScriptSyntaxError,
+ captionCorrespondence,
+ captionDisplaySequence,
formatScript,
narrativeValue,
parseScript,
@@ -88,7 +90,7 @@ test("selections, moments and Dual Text preserve separate semantic projections",
["anchorId", "occurrence"]);
});
-test("Caption Projection owns speech ranges without inventing alias word timing", () => {
+test("every explicit Dual Text is one whole display Atom without inferred internal correspondence", () => {
const parsed = parseScript(
"caption.svml",
`<line>
@@ -99,18 +101,12 @@ test("Caption Projection owns speech ranges without inventing alias word timing"
< | um>
</line>`,
);
- const [identity, discount, replacement, interrupted, hidden] = parsed.captionProjection.regions;
-
- assert.equal(parsed.captionProjection.contract, "svml.caption-projection@1");
- assert.equal(identity?.kind, "identity");
- assert.deepEqual(identity?.refinements.map((item) => item.display), ["test", "this"]);
- assert.equal(discount?.kind, "alias");
- assert.deepEqual(discount?.refinements.map((item) => item.display), ["off"]);
- assert.deepEqual(replacement?.refinements, []);
- assert.deepEqual(interrupted?.refinements.map((item) => item.display), ["what", "the"]);
- assert.equal(hidden?.kind, "hidden");
- assert.equal(hidden?.display, "");
- assert.equal(hidden?.endTokenExclusive! - hidden?.startToken!, 1);
+ const display = captionDisplaySequence(parsed, "story.caption");
+ const correspondence = captionCorrespondence(parsed, display.id);
+
+ assert.deepEqual(display.atoms.map((atom) => atom.wordIds.length), [2, 2, 3, 2]);
+ assert.deepEqual(correspondence.atoms.map((mapping) => mapping.sourceTokenIds.length), [2, 3, 3, 3]);
+ assert.equal(display.words.some((word) => word.text === "um"), false);
});
test("mismatched named Segment closes are rejected", () => {
diff --git a/packages/speech-alignment/README.md b/packages/speech-alignment/README.md
index d2d4e66d..1ecc2082 100644
--- a/packages/speech-alignment/README.md
+++ b/packages/speech-alignment/README.md
@@ -25,4 +25,5 @@ contracts and protocol utilities; it has no Core, Driver, Provider, Artifact, qu
authority.
Caption display is outside this package. `@narratage/caption` composes the resulting complete speech
-map with Script's authored Caption Projection and keeps presentation estimates local to the track.
+map with Script's explicit whole-Atom `CaptionCorrespondence`; it never asks this locator to infer
+display text or display-Word timing.
diff --git a/packages/speech-spine/test/video-pipeline-fixture.ts b/packages/speech-spine/test/video-pipeline-fixture.ts
index fd671a4c..5b6494e4 100644
--- a/packages/speech-spine/test/video-pipeline-fixture.ts
+++ b/packages/speech-spine/test/video-pipeline-fixture.ts
@@ -1,14 +1,24 @@
import { videoContractManifests } from "../../test-support/video-domain.js";
-import { captionManifest, captionTimingFragment } from "@narratage/caption";
+import {
+ captionManifest,
+ captionTypes,
+ captionValidatorDigests,
+ plannedCaptionTimingFragment,
+ resolveCaptionProgram,
+ sealCaptionPlan,
+ sealCaptionStyle,
+} from "@narratage/caption";
import { narrativeDependency, narrativeTypes } from "@narratage/narrative";
import { speechDependency, speechTypes } from "@narratage/speech";
import {
createResolvedClosure,
+ canonicalize,
digestOf,
link,
sealBuildRequest,
sealCompiledGraph,
sealRecord,
+ sealTypeValidationReceipt,
sealTypedModule,
start,
} from "@narratage/core";
@@ -31,7 +41,12 @@ import type {
ValueSchema,
} from "@narratage/protocol";
import { resolveRealization, sealRealizationOverlay } from "@narratage/run";
-import { narrativeValue, parseScript } from "@narratage/script";
+import {
+ captionCorrespondence,
+ captionDisplaySequence,
+ narrativeValue,
+ parseScript,
+} from "@narratage/script";
import { speechAlignmentManifest } from "@narratage/speech-alignment";
import { speechBasisManifest, speechBasisProjectionFragment } from "@narratage/speech-basis";
import { mediaPipelineManifest } from "@narratage/media-pipeline";
@@ -256,22 +271,63 @@ export const videoClosure = createResolvedClosure([
function authorProgram(): LinkedProgram {
const parsed = parseScript("pipeline.svml", "<line><that was insane | what the fuck></line>");
+ const display = captionDisplaySequence(parsed, "caption:display");
+ const correspondence = captionCorrespondence(parsed, display.id);
+ const style = sealCaptionStyle({
+ contract: "svml.caption-style@1",
+ id: "fixture",
+ planning: {
+ cue: { minimumWords: 1, maximumWords: 7, instruction: "Use one authored Atom." },
+ fields: [],
+ },
+ rendering: { family: "fixture-caption@1", parameters: {} },
+ });
+ const captionProgram = resolveCaptionProgram(display, "caption-program", style, []);
+ const captionPlan = sealCaptionPlan({
+ contract: "svml.caption-plan@1",
+ runs: [{
+ id: captionProgram.runs[0]!.id,
+ styleId: style.id,
+ cues: [{ id: "caption:cue:1", atomIds: display.atoms.map((atom) => atom.id), fields: [] }],
+ }],
+ });
+ const origin = {
+ kind: "authored" as const,
+ sourceDigest: digestOf("source:video-pipeline"),
+ frontendClosureDigest: digestOf("frontend:script"),
+ sourceName: "pipeline.svml",
+ };
const narrative = sealRecord({
id: "narrative:root",
type: narrativeTypes.narrative,
value: { kind: "inline", value: narrativeValue(parsed) },
conformance: "exact",
- origin: {
- kind: "authored",
- sourceDigest: digestOf("source:video-pipeline"),
- frontendClosureDigest: digestOf("frontend:script"),
- sourceName: "pipeline.svml",
- },
+ origin,
});
+ const programRecord = sealRecord({ id: "caption:program", type: captionTypes.program,
+ value: { kind: "inline", value: canonicalize(captionProgram) }, conformance: "exact", origin });
+ const planRecord = sealRecord({ id: "caption:plan", type: captionTypes.plan,
+ value: { kind: "inline", value: canonicalize(captionPlan) }, conformance: "exact", origin });
+ const captionRecords = [
+ sealRecord({ id: "caption:display", type: narrativeTypes.captionDisplay,
+ value: { kind: "inline", value: canonicalize(display) }, conformance: "exact", origin }),
+ sealRecord({ id: "caption:correspondence", type: narrativeTypes.captionCorrespondence,
+ value: { kind: "inline", value: canonicalize(correspondence) }, conformance: "exact", origin }),
+ { ...programRecord, validation: sealTypeValidationReceipt({
+ type: programRecord.type,
+ recordDigest: programRecord.digest,
+ validatorDigest: captionValidatorDigests.program,
+ }) },
+ { ...planRecord, validation: sealTypeValidationReceipt({
+ type: planRecord.type,
+ recordDigest: planRecord.digest,
+ validatorDigest: captionValidatorDigests.plan,
+ }) },
+ ];
return link(videoClosure, [sealTypedModule({
id: "author:video-pipeline",
closureDigest: videoClosure.digest,
- records: [narrative],
+ records: [narrative, ...captionRecords],
})]);
}
@@ -303,12 +359,15 @@ export function createVideoGraph(program: LinkedProgram): CompiledGraph {
audio: { kind: "logical-output", id: videoOutputs.audio },
},
});
- const caption = elaborateGraphFragment(program, captionTimingFragment, {
+ const caption = elaborateGraphFragment(program, plannedCaptionTimingFragment, {
id: "opening.caption",
- fragment: captionTimingFragment.id,
+ fragment: plannedCaptionTimingFragment.id,
inputs: {
- narrative,
+ display: { kind: "record", id: "caption:display" },
+ correspondence: { kind: "record", id: "caption:correspondence" },
map: { kind: "logical-output", id: videoOutputs.map },
+ program: { kind: "record", id: "caption:program" },
+ plan: { kind: "record", id: "caption:plan" },
},
});
const contribution = mergeFragmentContributions(
diff --git a/packages/speech-spine/test/video-pipeline.test.ts b/packages/speech-spine/test/video-pipeline.test.ts
index 00efc6db..fc37eebe 100644
--- a/packages/speech-spine/test/video-pipeline.test.ts
+++ b/packages/speech-spine/test/video-pipeline.test.ts
@@ -446,8 +446,8 @@ test("the Speech Spine pipeline resumes without repeating paid calls", async ()
captionRecord?.value.kind === "inline" ? captionRecord.value.value : {},
);
assert.deepEqual(
- [value.text, value.regions[0]?.display, value.regions[0]?.startSec, value.regions[0]?.endSec],
- ["that was insane", "that was insane", 0.1, 0.72],
+ [value.cues[0]?.startSec, value.cues[0]?.endSec, value.cues[0]?.atoms.length],
+ [0.1, 0.72, 1],
);
const mapRecordId = selectedRecord(completed.state, videoOutputs.map);
@@ -498,7 +498,7 @@ test("Targets prune official Fragments while shared SpeechBasis generation stays
assert.equal(producerCount(visual, speechBasisProducers.projectVisual), 1);
assert.equal(producerCount(visual, speechBasisProducers.projectAudio), 0);
assert.equal(producerCount(visual, whisperXProducers.request), 0);
- assert.equal(producerCount(visual, captionProducers.temporalize), 0);
+ assert.equal(producerCount(visual, captionProducers.temporalizePlan), 0);
const map = createVideoBuild({ targets: ["map"] });
assert.equal(producerCount(map, speechBasisProducers.projectAudio), 1);
@@ -506,7 +506,7 @@ test("Targets prune official Fragments while shared SpeechBasis generation stays
assert.equal(producerCount(map, whisperXProducers.request), 1);
assert.equal(producerCount(map, whisperXProducers.normalize), 1);
assert.equal(producerCount(map, speechAlignmentProducers.locate), 1);
- assert.equal(producerCount(map, captionProducers.temporalize), 0);
+ assert.equal(producerCount(map, captionProducers.temporalizePlan), 0);
const bothTakeProjections = createVideoBuild({ targets: ["audio", "visual"] });
assert.equal(producerCount(bothTakeProjections, videoProducers.requestSeedanceMini), 1);
@@ -571,7 +571,7 @@ test("an Existing SpeechBasis cuts generation while a visual substitute cuts the
assert.equal(producerCount(captionFromExisting, videoProducers.assembleBasis), 0);
assert.equal(producerCount(captionFromExisting, speechBasisProducers.projectAudio), 1);
assert.equal(producerCount(captionFromExisting, whisperXProducers.request), 1);
- assert.equal(producerCount(captionFromExisting, captionProducers.temporalize), 1);
+ assert.equal(producerCount(captionFromExisting, captionProducers.temporalizePlan), 1);
const black: VisualTrack = sealVisualTrack({
contract: "svml.visual-track@1",
diff --git a/packages/text/test/text.test.ts b/packages/text/test/text.test.ts
index 4acba223..c595b5f9 100644
--- a/packages/text/test/text.test.ts
+++ b/packages/text/test/text.test.ts
@@ -1,5 +1,5 @@
import { narrativeManifest } from "@narratage/narrative";
-import type { CaptionProjection, Narrative, NarrativeDialogueExcerpt, NarrativeExcerpt, NarrativeSpeechExcerpt } from "@narratage/narrative";
+import type { Narrative, NarrativeDialogueExcerpt, NarrativeExcerpt, NarrativeSpeechExcerpt } from "@narratage/narrative";
import assert from "node:assert/strict";
import test from "node:test";
@@ -62,9 +62,9 @@ test("Text learns <script> only from an imported Script Manifest", async () => {
assert.equal(result.module.records.some((record) =>
record.id === "story.segment.opening.speech" && record.type.name === "NarrativeSpeechExcerpt"), true);
assert.equal(result.module.records.some((record) =>
- record.id === "story.caption" && record.type.name === "CaptionProjection"), true);
+ record.id === "story.caption" && record.type.name === "CaptionDisplaySequence"), true);
assert.equal(result.module.records.some((record) =>
- record.id === "story.caption.words" && record.type.name === "CaptionWordSequence"), true);
+ record.id === "story.caption.correspondence" && record.type.name === "CaptionCorrespondence"), true);
assert.equal(result.sourceMaps.length, 1);
assert.doesNotThrow(() => link(scriptContext().closure, [result.module]));
});
diff --git a/packages/video-cli/test/cli.test.ts b/packages/video-cli/test/cli.test.ts
index 3e35b6b7..c8f5ceae 100644
--- a/packages/video-cli/test/cli.test.ts
+++ b/packages/video-cli/test/cli.test.ts
@@ -88,11 +88,11 @@ test("official video CLI checks a real Script source through the Node compiler h
}, kind: "record" },
{ name: "story.caption", type: {
module: { name: "@narratage/narrative", version: "0.0.0-dev" },
- name: "CaptionProjection",
+ name: "CaptionDisplaySequence",
}, kind: "record" },
- { name: "story.caption.words", type: {
+ { name: "story.caption.correspondence", type: {
module: { name: "@narratage/narrative", version: "0.0.0-dev" },
- name: "CaptionWordSequence",
+ name: "CaptionCorrespondence",
}, kind: "record" },
{ name: "story.segment.opening", type: {
module: { name: "@narratage/narrative", version: "0.0.0-dev" },
diff --git a/spec/caption-fine.md b/spec/caption-fine.md
index 6f1afa43..516bfd75 100644
--- a/spec/caption-fine.md
+++ b/spec/caption-fine.md
@@ -1,7 +1,6 @@
# Fine Caption Style Family
-Status: accepted direction; parameter surface deliberately not frozen; current implementation has
-the deltas listed below.
+Status: implemented baseline; parameter surface remains deliberately small and pre-release.
## Purpose
@@ -47,8 +46,8 @@ glow, long shadow, backdrop sampling, case rewriting, random variation and field
must not enter merely to reproduce the old engine's option count.
Cue timing is not a fifth Style dimension. It comes from the independent `TimedCaptionProjection`:
-the first and last included display-word windows determine the Cue window. Fine adds no hidden hold,
-lead, fade or provider-dependent timing.
+the first and last included whole-Atom windows determine the Cue window. Fine adds no hidden hold,
+lead, fade or provider-dependent timing, and it receives no display-Word timestamps.
## Font boundary
@@ -58,16 +57,8 @@ components. `@narratage/media` already defines `FontArtifactRef`; adding an auth
font files and connecting that value explicitly is separate work. It must not be smuggled into the
Fine redesign as a Caption-specific Media contract.
-## Current implementation delta
+## Remaining visual evidence
-Before this Style family can be treated as its accepted design, its package alone must:
-
-- remove `important-min-per-cue`, `important-max-per-cue`, `important-fill` and `important-scale`;
-- emit an empty field declaration list;
-- remove field-dependent word color/scale and related attributes;
-- replace the current left/top box with a deterministic bounded layout after its small parameter
- set is reviewed;
-- add real browser/pixel evidence for placement, wrapping and basic paint.
-
-Quickstarts continue to describe the executable baseline until this delta is implemented. They are
-not a reason to preserve the temporary field.
+The field-free contract is executable. Before freezing the family's visual surface, add real
+browser/pixel evidence for placement, wrapping and basic paint and review whether the current
+bounded left/top geometry is the smallest useful public parameter set.
diff --git a/spec/caption-program.md b/spec/caption-program.md
index d9f5c848..80cfa43e 100644
--- a/spec/caption-program.md
+++ b/spec/caption-program.md
@@ -1,179 +1,168 @@
# Caption Program
-Status: pre-release contract under implementation. Public names may still be polished before
-package publication; the semantic laws below are the target for executable tests.
+Status: pre-release executable contract.
## 1. Scope
-Caption is an ordinary video component family. It owns the transformation from immutable authored
-display words to one self-contained `VisualTrack`. It has no authority over Script wording,
-pronunciation, speech alignment, global time, Composition order, Provider selection or Runtime
-reuse policy.
+Caption is an ordinary video component family. It turns immutable authored display text into one
+self-contained `VisualTrack`. It has no authority over Script wording, pronunciation, speech
+alignment, Provider selection, Runtime realization or Composition order.
-The graph has two independent branches:
+Its two independent graph branches are:
```text
-CaptionWordSequence + CaptionProgram ── planner ────── CaptionPlan
-speech audio ────────────────────────────────── evidence/map ─ CompleteSemanticMap
+CaptionDisplaySequence + CaptionProgram ── planner ─────────── CaptionPlan
+speech audio ────────────────────────────── evidence/locator ── CompleteSemanticMap
-Narrative + Words + Program + Plan + Map ── temporalize ── TimedCaptionProjection
-TimedCaptionProjection + Words + Program ── Style renderer ── VisualTrack
+Display + Correspondence + Plan + Program + Map ── timing ─── TimedCaptionProjection
+TimedCaptionProjection + Display + Program ─────── renderer ── VisualTrack
```
-The branches meet only at temporalization. Planning contains no timestamps, and alignment contains
-no Caption Style or Cue judgment.
+Planning never receives timestamps. Alignment never receives Cue or Style judgment. The branches
+meet only at the explicit timing component.
-## 2. Display truth and word subsets
+## 2. Display truth: Atom and Word
-Caption planning sees only the visible left side of Script. For Dual Text:
+Script emits one `CaptionDisplaySequence` with two ordered views:
-```svml
-<SVML | semantic video markup language>
-```
-
-Caption sees `SVML`; speech generation and alignment see `semantic video markup language`. The
-right side must never enter a Caption planning request.
-
-Script deterministically projects every visible `CaptionProjection` region into one explicit,
-ordered `CaptionWordSequence`. This is the complete Caption word universe `W`; it is not a bag of
-strings. Each `CaptionWord` has a stable id and global index, display text and source offsets,
-Segment/Turn ownership, optional Role, and a source speech-token range. A word's correspondence is
-either:
-
-- `exact`: its own source speech-token range is proven;
-- `region-envelope`: only the enclosing display region's speech range is proven.
+- `Atom` is the smallest unit a planner may place wholly into one Cue and the smallest display unit
+ that can receive proven speech timing;
+- `Word` is the smallest target for optional Style-owned fields.
-The package does not invent word timing for the second case at this stage.
+Every Word belongs to exactly one Atom. Atoms partition Words exactly once and in order. Text and
+punctuation remain author truth: `45%`, `back-and-forth`, `damn!`, `300,000`, `don't` and `U.S.A.`
+cross this edge unchanged. The official Script reader is English-first and uses authored whitespace
+as its normal display-word boundary; another author package may publish another policy while
+producing the same public contracts.
-Script additionally emits ordered `CaptionWordSubset` values for explicit Selections. Role
-ownership is already explicit on each word, so the Caption author Surface can derive the equivalent
-Role subset directly from the sequence. A subset contains only ids from the exact
-`CaptionWordSequence` and preserves their global order. Selection-to-word projection is a
-Script/source-structure concern: Caption never reads token indices, source ranges or marker affinity
-from a general `NarrativeSelection`.
+For ordinary text, one display Word is normally one Atom. Explicit Dual Text is always one whole
+Atom, even when both sides happen to contain matching words:
-This creates two deliberately different projections of one authored Selection:
+```svml
+<New York City | new york city>
+```
-- the general `NarrativeSelection` contains only semantic start/end anchor identities and can be
- located in time by any consumer;
-- the Caption-specific `CaptionWordSubset` names visible display words wholly owned by that
- Selection and can be used before time exists.
+The display sequence contains one Atom with three Words: `New`, `York`, `City`. It does not claim
+that any one display Word corresponds to any one spoken token. Hidden Dual Text (`< | um>`) produces
+no display Atom.
-The two values are not interchangeable and neither is hidden metadata on the other. A Selection
-that cuts only part of an indivisible Dual Text display region cannot become a Caption word subset;
-the author must split the Dual Text atom.
+This is intentional information discipline: Script uses all correspondence the author explicitly
+provided and invents none that the author did not provide.
-## 3. Complete Style intent
+## 3. Speech correspondence is a separate edge
-A `CaptionStyle` is a complete author declaration with two inseparable payloads:
+`CaptionCorrespondence` maps every display Atom to one or more authored speech-token identities.
+It is separate from `CaptionDisplaySequence`; display consumers do not carry speech lineage they do
+not need.
-- `planning`: universal Cue word-count bounds plus zero or more Style-owned typed per-word field
- declarations and their instructions;
-- `rendering`: the owning Style family identity plus that family's complete, Recipe-resolved
- parameters.
+For ordinary text, each Atom maps to the speech tokens structurally owned by that display surface.
+For Dual Text, the whole display Atom maps to the whole right-side speech range. There is no public
+display-Word-to-speech-Word mapping and no guessed internal timing.
-Styles are declared separately from where they apply. The common Caption contract owns Cue
-partitioning and field assignment shapes, but it does not own fonts, colors, box layout, animation
-modes or the meaning of any field id. Those belong to concrete Style families. A Style family may
-declare no fields at all: the accepted `@narratage/caption-fine` direction uses only Cue partitioning
-and one uniform static appearance for every word in a Cue.
+Script also emits `CaptionDisplayWordSubset` values for explicit Selections. Role selectors are
+author-surface sugar over the same display sequence. A subset is ordered and may select zero or
+more whole Atoms; selecting only part of a multiword Atom is an authoring error. These values are
+not temporal `NarrativeSelection` values and do not add another `when` language.
-A field declaration has a stable id, value schema, natural-language instruction and
-minimum/maximum assignment count per Cue. Boolean, enum and bounded numeric values are supported.
-One word may receive zero, one or multiple independent fields; fields need not be contiguous.
+## 4. Complete Style intent
-A Style is a whole value. Applying another Style replaces it completely. The language does not
-implicitly merge arbitrary visual properties, Cue rules or fields from two Styles. Planning and
-rendering cannot be selected independently: replacing a Style replaces both. This prevents a
-planner from emitting a field that the selected renderer does not understand.
+A `CaptionStyle` is a complete author declaration with:
-SVS stores named Recipe data. A Style package parses the Recipe into its own complete parameters
-and emits the common `CaptionStyle` envelope. SVS does not understand Caption fields or rendering.
+- universal planning requirements: preferred Cue word bounds and zero or more typed per-Word field
+ declarations;
+- one rendering-family identity and that family's complete Recipe-resolved parameters.
-## 4. Total assignment and ordered replacement
+The common Caption package owns the Cue and field data shapes, not fonts, boxes, colors, animation
+or the meaning of a field id. A Style family may declare no fields. Boolean, enum and bounded-number
+fields are available to other families; a Word may carry zero, one or several independent fields.
-Every `CaptionProgram` names one explicit `default` Style. That Style initially owns every word in
-`W`, including roleless Turns. The word universe itself is the whole set: authors must not write
-`@whole ... @/whole` merely to style all captions, and the compiler must not invent such an authored
-time Selection.
+SVS stores named Recipe data. A Style package interprets that Recipe and emits a complete Style.
+Applying another Style replaces the whole Style; arbitrary planning and rendering pieces are never
+implicitly merged.
-`words={story.caption.words}` is a real graph input, not payload to copy through every downstream
-value. The Program stores the sequence identity and ordered word-id-to-Style runs only. Planner,
-temporalizer and Style renderer each declare their own explicit edge to the same word sequence when
-they need word text or source correspondence.
+## 5. Total assignment and ordered replacement
-The Program then applies zero or more ordered whole-Style replacements:
+Every `CaptionProgram` has one explicit default Style covering the complete display universe.
+Authors do not need a fake `@whole` Selection or a complement.
```xml
-<caption:Program id="captions" words={story.caption.words} default={plain}>
+<caption:Program id="captions" display={story.caption} default={plain}>
<caption:Use role="ALICE" style={alice}/>
<caption:Use words={story.caption.selection.special} style={impact}/>
</caption:Program>
```
-An author Surface may offer `role="ALICE"` as sugar, but it must resolve to the explicit Role word
-subset before the Program is constructed. It is not a temporal Selection and it introduces no
-fourth `when` language.
+Uses apply in source order and the last matching whole-Style replacement wins. The Program stores
+the display-sequence identity and ordered Word-id runs, not copied text. A run never crosses a
+Segment or Turn boundary. All runs are disjoint and partition the display Words exactly once, and
+no run may split an Atom.
+
+`display={story.caption}` is a real graph edge. Planner, timing and rendering declare their own
+edges to the same value only when they need it.
-Rules run in source order; for every word, the last matching rule wins. A rule that selects no
-visible word is an error. Role is optional per Script Turn and resets at every Segment boundary. A
-roleless Turn therefore receives the default unless an explicit word subset replaces it. Authors
-never spell a complement merely to preserve the default.
+## 6. Planner contract
-After assignment, one deterministic resolver creates maximal contiguous runs. A run never crosses
-a Segment or Turn boundary and every word in it has one Style. Runs are disjoint, ordered and
-partition the complete display universe exactly once. It does not create one graph node per Style.
+A planner receives resolved runs. For each run it may only:
-## 5. Planner contract
+1. group consecutive whole Atoms into ordered Cues;
+2. assign declared fields to Words inside each Cue.
-A planner consumes resolved runs rather than interpreting Role, Selection or Style precedence. For
-each run it may only:
+Cues must partition every run exactly once. Preferred minimum/maximum Cue word counts guide the
+planner but cannot invalidate author text; a single multiword Atom may exceed the maximum and must
+remain whole. The response contains no replacement text, pronunciation, timestamp or Style.
-1. split the word sequence into one or more ordered Cues, respecting the Style's common minimum and
- maximum word count;
-2. assign declared fields to words inside each Cue.
+The Gemini planner sends the readable form only once:
+
+```json
+{
+ "atoms": [
+ ["new"],
+ ["york"],
+ ["city", "is", "beautiful"]
+ ]
+}
+```
-The result must contain every Program run exactly once. Cues must preserve order and partition the
-run's words exactly once. Field ids, word ids, value schemas and per-Cue cardinalities fail closed.
-There is no response field for text, pronunciation, time or Style.
+The outer array is the Cue-cut universe. Each inner array is one indivisible Atom. Strings inside
+it are immutable field targets. Stable ids remain inside the package. A response consumes Atoms by
+`atom_count`; a field uses one-based `atom_number` and `word_number` within its Cue.
-`@narratage/caption-gemini` is one explicit implementation of this contract. It is not part of the
-Caption package or Core. Other packages may implement the same output law without being called
-Gemini, but the author graph must explicitly choose them.
+## 7. Timing projection
-## 6. Timing projection
+`CompleteSemanticMap` remains the only global speech timing truth. The timing component resolves
+each Atom's explicit speech-token range through `CaptionCorrespondence` and gives that whole Atom
+one measured window. A Cue begins at its first Atom and ends at its last Atom.
-`CompleteSemanticMap` remains the only global speech timing truth. Exact display words reuse their
-source token windows. For a `region-envelope` alias, a Caption temporalizer may allocate local
-windows inside that region's envelope. Those local boundaries remain downstream Caption facts and
-are never written back into the global map.
+`TimedCaptionProjection` therefore contains timed Cues and timed Atoms only. It contains no
+per-display-Word timestamps, no `estimated` quality label and no proportional character split. For
+a multiword Dual Text Atom, downstream renderers may choose how to lay out or animate its internal
+Words, but they cannot claim invented word timing. Nothing is written back into SemanticMap.
-Multiple Cues inside one alias must receive ordered, non-overlapping local windows. They must not
-each copy the same whole region envelope.
+## 8. Rendering and composition
-## 7. Rendering and composition
+Each timed Cue selects the complete rendering payload of its resolved Style. One Style-family
+renderer returns one ordinary `VisualTrack`. In the first contract, all Styles in one Program use
+the same rendering family.
-Each timed Cue selects the full rendering payload of its resolved Style. One Style-family renderer
-consumes all runs of that family, including the default and every override, and returns exactly one
-ordinary `VisualTrack`. The renderer may interpret its own declared fields, but cannot sample or
-mutate another Track. In v1 every Style in one Program must name the same rendering family; mixing
-families requires separate Programs/Tracks until a real cross-family composition contract exists.
+`@narratage/caption-fine` is the first official family. It declares no fields and uses one uniform
+static appearance for all Words in a Cue. Another package may define karaoke or field-dependent
+paint without modifying Script, Caption, Composition or Core.
-Film does not recognize Caption specially. It folds the returned Track alongside Speech, B-roll,
-Text or any third-party Track satisfying the same public contract.
+Film does not recognize Caption specially. It folds the resulting peer Track beside Speech,
+B-roll, Text and third-party Tracks.
-## 8. Explicit non-goals
+## 9. Explicit non-goals
-Caption Program v1 does not define:
+Caption v1 does not define:
- Script correction or alternative wording;
+- inferred Dual Text word correspondence;
- speaker diarization or speaker identity;
-- implicit model routing;
-- automatic cache reuse or prompt-based memoization;
-- hidden Runtime fallback;
+- implicit model or Provider routing;
+- automatic cache reuse;
+- estimated display-word timing;
- a global Caption layer in Composition;
-- field-specific visual semantics in Core;
-- a condition language parallel to Selection, Moment or Caption word subsets.
+- field-specific semantics in Core.
-Reuse, Existing Values and substitutes are selected by the external BuildRequest/realization
-closure. They do not change Caption data or planner semantics.
+Targets, Candidates and realizations remain external run intent. They do not change Caption data or
+planner semantics.
diff --git a/spec/script-surface.md b/spec/script-surface.md
index 16b7e653..e3fd033e 100644
--- a/spec/script-surface.md
+++ b/spec/script-surface.md
@@ -158,7 +158,7 @@ B: It’s 8:30.
在 `speech` 和 `caption` 投影中,`<A>`、`<B>` 都被移除。Role Cue 不是
speaker 数据模型,不建立人物实体,不选择音色,也不对字幕隐式分组。外部
Program 可以显式写 `role="A"` 查询这些 spoken turn。Caption Program 将这类
-查询直接降低为 `CaptionWordSubset`;它不是时间 Selection,也不伪造 Selection
+查询直接降低为 `CaptionDisplayWordSubset`;它不是时间 Selection,也不伪造 Selection
Record。仅有 `<A>` 本身不触发任何样式、人物、音色或素材行为。
为消除歧义:
@@ -541,14 +541,15 @@ temporal marker、Role Cue 或结构标签内部。注释可出现在 atom 之
- 每个 Segment 的有序 spoken turn,以及各 turn 的稳定 identity、可选 Role Cue、
token range 和 source range;
- plain / Dual Text / Slot atom 及其源码范围;
-- caption atom 到一个或多个 speech token 的显式映射;
+- Caption display Atom、其内部有序 display Word,以及 Atom 到一个或多个 speech token
+ 的独立显式映射;
- 每个 Selection 的一个或多个有序 occurrence,以及各 occurrence 已解析的
`startAnchorId` / `endAnchorId`;
- 每个 Moment 的一个或多个有序 occurrence,以及各 occurrence 已解析的
`anchorId`;
- Parser/Source Map 内部另行保留 affinity、token/source range,公开值不携带它们;
-- 完整有序的 `CaptionWordSequence`,以及每个显式 Selection 对应的
- `CaptionWordSubset`。字幕无需也不得从公开 Selection 反推词下标。
+- 完整有序的 `CaptionDisplaySequence`、独立的 `CaptionCorrespondence`,以及每个显式
+ Selection 对应的 `CaptionDisplayWordSubset`。字幕无需也不得从公开 Selection 反推词下标。
外部 Program 的 `role="label"` selector 可以从上述词全集投影自己的词子集;
同一 label 的多次 Turn 是多个有序、互不相邻的命中。它是 Caption author surface
@@ -558,13 +559,15 @@ SelectionSet/MomentSet 是消费者边界,不是 Segment 的子对象。典型
关系是:
```text
-Script ──compile──> Narrative IR ──project──> dialogue / speech / caption
+Script ──compile──> Narrative IR ──project──> dialogue / speech
├──resolve──> NarrativeSelection[] / NarrativeMoment[]
- └──project──> CaptionWordSequence / CaptionWordSubset[]
+ └──project──> CaptionDisplaySequence
+ + CaptionCorrespondence
+ + CaptionDisplayWordSubset[]
NarrativeSelection A ──during──> consumer.port_1
NarrativeMoment P ──at──────> consumer.port_2
-CaptionWordSubset C ──words───> caption.Program
+CaptionDisplayWordSubset C ──words──> caption.Program
```
一个 Ranking、B-roll 或其他节点可以有多个动态端口,各端口接受不同