Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.
TypeScript agentic-aiaiai-agentsai-videocompilerdslffmpeggenerative-aillmmarkup-languagemonorepoplugin-systemprogramming-languagetext-to-videotypescriptvideovideo-automationvideo-clonevideo-editingvideo-generation
feat: make Script authoritative for speech timing
kashorin07b2924cc37a
diff --git a/README.md b/README.mdindex c322f81c..348d2a56 100644--- a/README.md+++ b/README.md@@ -11,7 +11,7 @@ HyperFrames HTML document.│├─ typed Plan IR ─────────────────────────────> Canvas view├─ Basis Producer ─────> TemporalBasisProduction- ├─ Locator ────────────> ExactSemanticMap+ ├─ Locator ────────────> CompleteSemanticMap├─ Basis + Map ────────> TemporalBinding ─────> Timeline view└─ flat Track[] ───────> Composition ─────────> HyperFrames HTML```@@ -29,8 +29,10 @@ architecture end to end:Selections and zero-width Moments;- an exact `SemanticIndex` with independent word and Segment endpoints(`2M + 2N` identities);-- replaceable `TemporalBasisProduction` and `ExactSemanticMap` Components,+- replaceable `TemporalBasisProduction` and `CompleteSemanticMap` Components,validated and bound before any Track lowering;+- direct monotonic many-to-many alignment from authoritative Script tokens to+ noisy `SpeechTimingEvidence`, without a corrected-transcript stage;- hard cut, gap and audio/visual crossfade joins with distinct source-to-programmaps;- typed `.svc` content modules and `.svs` classes, including nested Component@@ -40,6 +42,8 @@ architecture end to end:- isolated `.svk` projectors, verified offline capability artifacts and finite,auditable `composite-v1` expansion;- manifest-enforced temporal `one` / `each` / `set` consumption;+- a post-locate `CaptionPlan` that may group cues and add typed style annotations+ but cannot rewrite Script text or timing;- flat media, ranking, B-roll, caption, text and audio Tracks in one absoluteProgramSpace;- Film as the only `Track[]` consumer and HyperFrames HTML as the sole formal@@ -81,10 +85,10 @@ pnpm svml render build/index.html --out build/video.mp4````--artifacts` is required only when the reachable Plan contains capability-Components. Exact alignment is an explicit typed value referenced by the-Locator in source; it is not an ambient CLI argument. `estimate` returns an-`EstimatedSemanticMap` for preview and cannot be supplied where final HTML-requires an `ExactSemanticMap`.+Components. Speech timing is an explicit typed value referenced by the Locator+in source; it is not an ambient CLI argument. `estimate` produces estimated+timing evidence and the same structurally complete `CompleteSemanticMap` type;+per-anchor quality records what was measured, derived or estimated.`fmt` currently canonicalizes only the Script body and refuses a rewrite if thereparsed semantic IR differs. `canvas` needs no materialized time evidence.@@ -108,8 +112,8 @@ Canvas owns only a human-readable view.- [`examples/regen-ranking/regen-ranking.svml`](examples/regen-ranking/regen-ranking.svml)reconstructs a pinned production video with five ranking items, five B-roll- items, captions, title, BGM and sound effects. It compiles to 29 Plan nodes,- 33 typed edges, 1083 frames, 153 visual fragments and 14 audio fragments.+ items, captions, title, BGM and sound effects. It compiles to 30 Plan nodes,+ 35 typed edges and 1083 frames.- [`examples/flat-track-launch/flat-track-launch.svml`](examples/flat-track-launch/flat-track-launch.svml)is the comprehensive language fixture: explicit Basis/Locator, nested SVS,overlapping Presents, manual ProgramSpans, a Moment, B-roll source audio anddiff --git a/examples/composite-speech-program/README.md b/examples/composite-speech-program/README.mdindex 1d416eb5..054c5345 100644--- a/examples/composite-speech-program/README.md+++ b/examples/composite-speech-program/README.md@@ -13,7 +13,7 @@ For `<speech-program id="voice">`, expansion creates:```textvoice::basis → TemporalBasisProduction + ProgramBoundVideoSequence│- └──────────────> voice::locator → ExactSemanticMap+ └──────────────> voice::locator → CompleteSemanticMapvoice.production = voice::basis.productionvoice.map = voice::locator.mapdiff --git a/examples/composite-speech-program/alignment.json b/examples/composite-speech-program/alignment.jsonindex 5cc43427..a23de902 100644--- a/examples/composite-speech-program/alignment.json+++ b/examples/composite-speech-program/alignment.json@@ -1,8 +1,9 @@{- "contract": "svml.speech-alignment.v1",+ "contract": "svml.speech-timing-evidence.v1","durationSec": 9.5,"fps": 30,- "words": [+ "quality": "measured",+ "units": [{ "text": "Hello", "startSec": 0.2, "endSec": 0.8, "segmentId": "intro" },{ "text": "this", "startSec": 1.0, "endSec": 1.5, "segmentId": "intro" },{ "text": "is", "startSec": 1.6, "endSec": 1.9, "segmentId": "intro" },diff --git a/examples/composite-speech-program/minimal.svml b/examples/composite-speech-program/minimal.svmlindex a15d9958..70370f95 100644--- a/examples/composite-speech-program/minimal.svml+++ b/examples/composite-speech-program/minimal.svml@@ -16,9 +16,9 @@<video id="host" src="../regen-ranking/assets/avatar-1.mp4"/><audio id="host-audio" src="../regen-ranking/assets/avatar-1.mp3"/>- <alignment id="alignment" src="./alignment.json"/>+ <timing id="timing" src="./alignment.json"/>- <speech-program id="voice" script={script} evidence={alignment} class="house.default">+ <speech-program id="voice" script={script} evidence={timing} class="house.default"><segment id="intro" visual={host} audio={host-audio} sourceWindow="0s .. 5s"/><segment id="close" visual={host} audio={host-audio} sourceWindow="5s .. 10s"/><join after="intro" overlap="500ms" audio="crossfade" visual="dissolve"/>diff --git a/examples/composite-speech-program/speech-program.svk b/examples/composite-speech-program/speech-program.svkindex 0dc9970f..8b1277db 100644--- a/examples/composite-speech-program/speech-program.svk+++ b/examples/composite-speech-program/speech-program.svk@@ -1,8 +1,8 @@<kernel name="speech-program" abi="1" profile="composite-v1"><port name="script" direction="input" type="NarrativeIR" cardinality="one"/>- <port name="evidence" direction="input" type="AlignmentEvidence" cardinality="one"/>+ <port name="evidence" direction="input" type="SpeechTimingEvidence" cardinality="one"/><port name="production" direction="output" type="TemporalBasisProduction" cardinality="one"/>- <port name="map" direction="output" type="ExactSemanticMap" cardinality="one"/>+ <port name="map" direction="output" type="CompleteSemanticMap" cardinality="one"/><port name="facets.visual" direction="output" type="ProgramBoundVideoSequence" cardinality="one"/><param name="join" type="enum(cut,crossfade,gap)" default="cut"/>diff --git a/examples/flat-track-launch/README.md b/examples/flat-track-launch/README.mdindex 4984925a..d6db0113 100644--- a/examples/flat-track-launch/README.md+++ b/examples/flat-track-launch/README.md@@ -5,14 +5,14 @@ providers and exercises the contracts together:- independent Script Segment endpoints (`2M + 2N`);- replaceable Basis Producer and Locator Components;-- one explicit `ProgramBasis` plus `ExactSemanticMap`;+- one explicit `ProgramBasis` plus `CompleteSemanticMap`;- flat Tracks and absolute `z`, with Film as the only `Track[]` consumer;- one continuous A-roll mapping with overlapping Present windows;- manual `ProgramSpan`, a `Moment`, B-roll audio and a visual crossfade;- a disconnected SelectionSet driving one caption style rule;- top-level and nested Component fields configured by `.svs`.-It reuses the checked-in local artifacts and exact alignment evidence from the+It reuses the checked-in local artifacts and measured speech timing evidence from theadjacent `regen-ranking` example. No provider is called.```shdiff --git a/examples/flat-track-launch/flat-track-launch.svml b/examples/flat-track-launch/flat-track-launch.svmlindex a3d82c5f..2a513e20 100644--- a/examples/flat-track-launch/flat-track-launch.svml+++ b/examples/flat-track-launch/flat-track-launch.svml@@ -6,6 +6,7 @@<import from="../../stdlib/broll-track.svk"/><import from="../../stdlib/ranking-column.svk"/><import from="../../stdlib/text-track.svk"/>+ <import from="../../stdlib/caption-planner.svk"/><import from="../../stdlib/caption-track.svk"/><import from="../../stdlib/audio-track.svk"/><import from="../../stdlib/film.svk"/>@@ -61,7 +62,7 @@<audio id="ranking-appear" src="../regen-ranking/assets/ranking-appear.wav"/><audio id="ranking-move" src="../regen-ranking/assets/ranking-move.wav"/><audio id="music" src="../regen-ranking/assets/background.mp3"/>- <alignment id="alignment" src="../regen-ranking/evidence/alignment.json"/>+ <timing id="timing" src="../regen-ranking/evidence/alignment.json"/><!-- The Basis Producer and Locator are replaceable Components with typed outputs. --><speech-assemble id="voice" fps="30" defaultJoin="cut">@@ -69,7 +70,7 @@<segment id="scene-2" script={script.segment.scene-2} visual={avatar-2} audio={avatar-2-audio} sourceWindow="0s .. 12.033333333333333s"/><segment id="scene-3" script={script.segment.scene-3} visual={avatar-3} audio={avatar-3-audio} sourceWindow="0s .. 12.033333333333333s"/></speech-assemble>- <speech-locator id="location" script={script} basis={voice.production} evidence={alignment}/>+ <speech-locator id="location" script={script} basis={voice.production} evidence={timing}/><!-- One source mapping; Present only changes its projection in ProgramSpace. --><media-track id="aroll" z="20">@@ -107,8 +108,10 @@</text-track><!-- The repeated emphasis Selection is disconnected and consumed as one set. -->- <caption-track id="captions" z="500" script={script} semantic={location.map} class="house.captions">+ <caption-planner id="caption-plan" script={script} semantic={location.map} maxWords="3"/>+ <caption-track id="captions" z="500" script={script} semantic={location.map} plan={caption-plan.plan} class="house.captions"><style id="emphasis" during={script.selection.emphasis} class="house.caption-emphasis"/>+ <style id="narrator" role="NARRATOR" activeColor="#73fbd3"/></caption-track><audio-track id="background-music">diff --git a/examples/regen-ranking/evidence/alignment.json b/examples/regen-ranking/evidence/alignment.jsonindex b2b45d70..fd7153c2 100644--- a/examples/regen-ranking/evidence/alignment.json+++ b/examples/regen-ranking/evidence/alignment.json@@ -1,35 +1,42 @@{- "contract": "svml.speech-alignment.v1",+ "contract": "svml.speech-timing-evidence.v1","durationSec": 36.1,"fps": 30,- "words": [+ "quality": "measured",+ "units": [{- "text": "Photoshop",+ "text": "Photo shop","startSec": 0.08,"endSec": 1,"segmentId": "scene-1"},{- "text": "Powerful",+ "text": "power full but","startSec": 1,- "endSec": 1.53,+ "endSec": 1.9,"segmentId": "scene-1"},{- "text": "but",- "startSec": 1.53,- "endSec": 1.9,+ "text": "on",+ "startSec": 1.9,+ "endSec": 1.945,"segmentId": "scene-1"},{- "text": "only",- "startSec": 1.9,+ "text": "lee",+ "startSec": 1.945,"endSec": 1.99,"segmentId": "scene-1"},{- "text": "if",+ "text": "of","startSec": 1.99,+ "endSec": 1.995,+ "segmentId": "scene-1"+ },+ {+ "text": "uh",+ "startSec": 1.995,"endSec": 2,"segmentId": "scene-1"},@@ -832,295 +839,12 @@}],"provenance": {- "method": "whisper.cpp word timestamps + deterministic intended-script LCS projection",- "source": "existing pinned avatar audio; no generation call",- "captionCues": {- "kind": "reference-recovery",- "source": "regen ranking Job cmrxqbub800012tjoi2ev2u24",- "note": "Cue boundaries are evidence emitted by Locate, not authored SVML timing."- },+ "method": "pinned speech timing with deliberate recognition noise",+ "source": "existing avatar audio; wording is measurement evidence, never corrected transcript truth","timingOverrides": {"kind": "reference-recovery","source": "scene transitions in regen ranking Job cmrxqbub800012tjoi2ev2u24","note": "Only boundaries visible in the frozen reference video are overridden."}- },- "captionCues": [- {- "startWord": 0,- "endWordExclusive": 1- },- {- "startWord": 1,- "endWordExclusive": 2- },- {- "startWord": 2,- "endWordExclusive": 5- },- {- "startWord": 5,- "endWordExclusive": 8- },- {- "startWord": 8,- "endWordExclusive": 11- },- {- "startWord": 11,- "endWordExclusive": 12- },- {- "startWord": 12,- "endWordExclusive": 14- },- {- "startWord": 14,- "endWordExclusive": 16- },- {- "startWord": 16,- "endWordExclusive": 18- },- {- "startWord": 18,- "endWordExclusive": 20- },- {- "startWord": 20,- "endWordExclusive": 21- },- {- "startWord": 21,- "endWordExclusive": 22- },- {- "startWord": 22,- "endWordExclusive": 24- },- {- "startWord": 24,- "endWordExclusive": 26- },- {- "startWord": 26,- "endWordExclusive": 27- },- {- "startWord": 27,- "endWordExclusive": 28- },- {- "startWord": 28,- "endWordExclusive": 31- },- {- "startWord": 31,- "endWordExclusive": 34- },- {- "startWord": 34,- "endWordExclusive": 35- },- {- "startWord": 35,- "endWordExclusive": 38- },- {- "startWord": 38,- "endWordExclusive": 40- },- {- "startWord": 40,- "endWordExclusive": 42- },- {- "startWord": 42,- "endWordExclusive": 43- },- {- "startWord": 43,- "endWordExclusive": 44- },- {- "startWord": 44,- "endWordExclusive": 47- },- {- "startWord": 47,- "endWordExclusive": 50- },- {- "startWord": 50,- "endWordExclusive": 53- },- {- "startWord": 53,- "endWordExclusive": 56- },- {- "startWord": 56,- "endWordExclusive": 57- },- {- "startWord": 57,- "endWordExclusive": 59- },- {- "startWord": 59,- "endWordExclusive": 62- },- {- "startWord": 62,- "endWordExclusive": 65- },- {- "startWord": 65,- "endWordExclusive": 67- },- {- "startWord": 67,- "endWordExclusive": 68- },- {- "startWord": 68,- "endWordExclusive": 70- },- {- "startWord": 70,- "endWordExclusive": 73- },- {- "startWord": 73,- "endWordExclusive": 75- },- {- "startWord": 75,- "endWordExclusive": 76- },- {- "startWord": 76,- "endWordExclusive": 77- },- {- "startWord": 77,- "endWordExclusive": 80- },- {- "startWord": 80,- "endWordExclusive": 83- },- {- "startWord": 83,- "endWordExclusive": 86- },- {- "startWord": 86,- "endWordExclusive": 87- },- {- "startWord": 87,- "endWordExclusive": 89- },- {- "startWord": 89,- "endWordExclusive": 90- },- {- "startWord": 90,- "endWordExclusive": 93- },- {- "startWord": 93,- "endWordExclusive": 95- },- {- "startWord": 95,- "endWordExclusive": 98- },- {- "startWord": 98,- "endWordExclusive": 100- },- {- "startWord": 100,- "endWordExclusive": 103- },- {- "startWord": 103,- "endWordExclusive": 105- },- {- "startWord": 105,- "endWordExclusive": 106- },- {- "startWord": 106,- "endWordExclusive": 107- },- {- "startWord": 107,- "endWordExclusive": 108- },- {- "startWord": 108,- "endWordExclusive": 109- },- {- "startWord": 109,- "endWordExclusive": 111- },- {- "startWord": 111,- "endWordExclusive": 112- },- {- "startWord": 112,- "endWordExclusive": 114- },- {- "startWord": 114,- "endWordExclusive": 116- },- {- "startWord": 116,- "endWordExclusive": 118- },- {- "startWord": 118,- "endWordExclusive": 120- },- {- "startWord": 120,- "endWordExclusive": 122- },- {- "startWord": 122,- "endWordExclusive": 123- },- {- "startWord": 123,- "endWordExclusive": 125- },- {- "startWord": 125,- "endWordExclusive": 127- },- {- "startWord": 127,- "endWordExclusive": 129- },- {- "startWord": 129,- "endWordExclusive": 131- },- {- "startWord": 131,- "endWordExclusive": 133- },- {- "startWord": 133,- "endWordExclusive": 135- }- ]+ }}diff --git a/examples/regen-ranking/evidence/caption-cues.json b/examples/regen-ranking/evidence/caption-cues.jsondeleted file mode 100644index 232bdad8..00000000--- a/examples/regen-ranking/evidence/caption-cues.json+++ /dev/null@@ -1,81 +0,0 @@-{- "contract": "svml.caption-cues.v1",- "captionCues": [- { "startWord": 0, "endWordExclusive": 1 },- { "startWord": 1, "endWordExclusive": 2 },- { "startWord": 2, "endWordExclusive": 5 },- { "startWord": 5, "endWordExclusive": 8 },- { "startWord": 8, "endWordExclusive": 11 },- { "startWord": 11, "endWordExclusive": 12 },- { "startWord": 12, "endWordExclusive": 14 },- { "startWord": 14, "endWordExclusive": 16 },- { "startWord": 16, "endWordExclusive": 18 },- { "startWord": 18, "endWordExclusive": 20 },- { "startWord": 20, "endWordExclusive": 21 },- { "startWord": 21, "endWordExclusive": 22 },- { "startWord": 22, "endWordExclusive": 24 },- { "startWord": 24, "endWordExclusive": 26 },- { "startWord": 26, "endWordExclusive": 27 },- { "startWord": 27, "endWordExclusive": 28 },- { "startWord": 28, "endWordExclusive": 31 },- { "startWord": 31, "endWordExclusive": 34 },- { "startWord": 34, "endWordExclusive": 35 },- { "startWord": 35, "endWordExclusive": 38 },- { "startWord": 38, "endWordExclusive": 40 },- { "startWord": 40, "endWordExclusive": 42 },-- { "startWord": 42, "endWordExclusive": 43 },- { "startWord": 43, "endWordExclusive": 44 },- { "startWord": 44, "endWordExclusive": 47 },- { "startWord": 47, "endWordExclusive": 50 },- { "startWord": 50, "endWordExclusive": 53 },- { "startWord": 53, "endWordExclusive": 56 },- { "startWord": 56, "endWordExclusive": 57 },- { "startWord": 57, "endWordExclusive": 59 },- { "startWord": 59, "endWordExclusive": 62 },- { "startWord": 62, "endWordExclusive": 65 },- { "startWord": 65, "endWordExclusive": 67 },- { "startWord": 67, "endWordExclusive": 68 },- { "startWord": 68, "endWordExclusive": 70 },- { "startWord": 70, "endWordExclusive": 73 },- { "startWord": 73, "endWordExclusive": 75 },- { "startWord": 75, "endWordExclusive": 76 },- { "startWord": 76, "endWordExclusive": 77 },- { "startWord": 77, "endWordExclusive": 80 },- { "startWord": 80, "endWordExclusive": 83 },- { "startWord": 83, "endWordExclusive": 86 },- { "startWord": 86, "endWordExclusive": 87 },- { "startWord": 87, "endWordExclusive": 89 },-- { "startWord": 89, "endWordExclusive": 90 },- { "startWord": 90, "endWordExclusive": 93 },- { "startWord": 93, "endWordExclusive": 95 },- { "startWord": 95, "endWordExclusive": 98 },- { "startWord": 98, "endWordExclusive": 100 },- { "startWord": 100, "endWordExclusive": 103 },- { "startWord": 103, "endWordExclusive": 105 },- { "startWord": 105, "endWordExclusive": 106 },- { "startWord": 106, "endWordExclusive": 107 },- { "startWord": 107, "endWordExclusive": 108 },- { "startWord": 108, "endWordExclusive": 109 },- { "startWord": 109, "endWordExclusive": 111 },- { "startWord": 111, "endWordExclusive": 112 },- { "startWord": 112, "endWordExclusive": 114 },- { "startWord": 114, "endWordExclusive": 116 },- { "startWord": 116, "endWordExclusive": 118 },- { "startWord": 118, "endWordExclusive": 120 },- { "startWord": 120, "endWordExclusive": 122 },- { "startWord": 122, "endWordExclusive": 123 },- { "startWord": 123, "endWordExclusive": 125 },- { "startWord": 125, "endWordExclusive": 127 },- { "startWord": 127, "endWordExclusive": 129 },- { "startWord": 129, "endWordExclusive": 131 },- { "startWord": 131, "endWordExclusive": 133 },- { "startWord": 133, "endWordExclusive": 135 }- ],- "provenance": {- "kind": "reference-recovery",- "source": "regen ranking Job cmrxqbub800012tjoi2ev2u24",- "note": "Cue boundaries are evidence emitted by Locate, not authored SVML timing."- }-}diff --git a/examples/regen-ranking/evidence/timing-overrides.json b/examples/regen-ranking/evidence/timing-overrides.jsonindex 548b1138..405808b8 100644--- a/examples/regen-ranking/evidence/timing-overrides.json+++ b/examples/regen-ranking/evidence/timing-overrides.json@@ -1,6 +1,6 @@{- "contract": "svml.alignment-overrides.v1",- "words": [+ "contract": "svml.speech-timing-overrides.v1",+ "units": [{ "index": 69, "endSec": 20.000 },{ "index": 70, "startSec": 20.000, "endSec": 20.033 },{ "index": 71, "startSec": 20.033, "endSec": 20.133 },diff --git a/examples/regen-ranking/reference.json b/examples/regen-ranking/reference.jsonindex 9360a94e..cc68f6fe 100644--- a/examples/regen-ranking/reference.json+++ b/examples/regen-ranking/reference.json@@ -15,8 +15,8 @@"plan": {"root": "regen-ranking","values": 20,- "instances": 9,- "edges": 33+ "instances": 10,+ "edges": 35},"selectionFrames": {"handsome-1": [[791, 831]],@@ -26,7 +26,7 @@"instagram-post": [[977, 1009]]},"locatedOutput": {- "visualFragments": 153,+ "visualFragments": 154,"audioFragments": 14},"renderComparison": {diff --git a/examples/regen-ranking/regen-ranking.svml b/examples/regen-ranking/regen-ranking.svmlindex 5149f76a..e1a9ce76 100644--- a/examples/regen-ranking/regen-ranking.svml+++ b/examples/regen-ranking/regen-ranking.svml@@ -6,6 +6,7 @@<import from="../../stdlib/ranking-column.svk"/><import from="../../stdlib/broll-track.svk"/><import from="../../stdlib/text-track.svk"/>+ <import from="../../stdlib/caption-planner.svk"/><import from="../../stdlib/caption-track.svk"/><import from="../../stdlib/audio-track.svk"/><import from="../../stdlib/film.svk"/>@@ -62,7 +63,7 @@<audio id="ranking-appear" src="./assets/ranking-appear.wav"/><audio id="ranking-move" src="./assets/ranking-move.wav"/><audio id="background-music" src="./assets/background.mp3"/>- <alignment id="alignment" src="./evidence/alignment.json"/>+ <timing id="timing" src="./evidence/alignment.json"/><speech-assemble id="voice" fps="30" defaultJoin="cut"><segment id="scene-1" script={script.segment.scene-1} visual={avatar-1} audio={avatar-1-audio} sourceWindow="0s .. 12.033333333333333s"/>@@ -70,7 +71,7 @@<segment id="scene-3" script={script.segment.scene-3} visual={avatar-3} audio={avatar-3-audio} sourceWindow="0s .. 12.033333333333333s"/></speech-assemble>- <speech-locator id="location" script={script} basis={voice.production} evidence={alignment}/>+ <speech-locator id="location" script={script} basis={voice.production} evidence={timing}/><media-track id="base" z="20"><item id="scene-1" source={voice.segment.scene-1.visual} fit="cover"/>@@ -129,15 +130,17 @@<item id="instagram-post" source={broll-1} during={script.selection.instagram-post} fit="contain" zoom="1.02"/></broll-track>+ <caption-planner id="caption-plan" script={script} semantic={location.map} maxWords="3"/>+<caption-trackid="captions"script={script}semantic={location.map}+ plan={caption-plan.plan}z="500"x="0.37"y="0.54"width="0.64"- maxWords="3"hold="0.08"font="Poppins, Inter, Arial, sans-serif"size="50"diff --git a/spec/script-surface-v1.md b/spec/script-surface-v1.mdindex 4f44850a..c150c64c 100644--- a/spec/script-surface-v1.md+++ b/spec/script-surface-v1.md@@ -356,9 +356,9 @@ gap B.start > A.end若未来需要真正重排或并行 speech,应发布显式的非线性叙事模型;v1 不让普通Range 在不同 Locator 下反向或消失。-同一份 SelectionSet 可先在 `EstimatedSemanticMap` 上得到预览区间,再在-`ExactSemanticMap` 上得到成片区间。两者使用完全相同的 identity、不同的精度-类型。Script 本身不含秒数、帧号或采样点。+同一份 SelectionSet 在不同 fulfillment 产生的 `CompleteSemanticMap` 上可得到+预览或成片区间。Map 使用相同 identity;每个 anchor 以 `measured`、`derived` 或+`estimated` 记录证据质量。Script 本身不含秒数、帧号或采样点。Script Surface 不绑定帧率、采样率或渲染器。后端一旦选择物理时钟,必须只量化一次并让所有消费者复用同一整数边界;后端时钟变化不改变本语言表面。@@ -413,7 +413,7 @@ overlap 或 gap 落在不同点。Dual Text 的 speech 侧,但均不得切入 v1 speech token。- Moment 只选择 Semantic Anchor Index 中已有的一个候选点,不新增 anchoridentity;因此 `2M + 2N` 的精确计数不因 Moment 数量改变。-- MomentSet 与 SelectionSet 复用同一份 Estimated/Exact SemanticMap 和一次性+- MomentSet 与 SelectionSet 复用同一份 CompleteSemanticMap 和一次性帧量化;区别只在最终载体是 `Point[]` 而不是 `Range[]`。消费端保持两个互不相混的类型:diff --git a/src/cli.ts b/src/cli.tsindex 972b13c0..266dcb85 100644--- a/src/cli.ts+++ b/src/cli.ts@@ -169,7 +169,7 @@ async function run(): Promise<void> {return;}if (args.command === "estimate") {- if (!file) throw new Error("usage: svml estimate <file.svml> [--out alignment.json]");+ if (!file) throw new Error("usage: svml estimate <file.svml> [--out timing.json]");const numeric = (name: string): number | undefined => {const value = option(args, name);if (value === undefined) return undefined;@@ -188,9 +188,6 @@ async function run(): Promise<void> {...(numeric("empty-segment-duration") !== undefined? { emptySegmentSec: numeric("empty-segment-duration")! }: {}),- ...(numeric("max-caption-words") !== undefined- ? { maxCaptionWords: numeric("max-caption-words")! }- : {}),...(numeric("fps") !== undefined ? { fps: numeric("fps")! } : {}),});const payload = {@@ -288,7 +285,7 @@ usage:svml script <file.svml> [--out narrative.json]svml plan <file.svml> [--out plan.json]svml fmt <file.svml> [--check|--write|--out formatted.svml]- svml estimate <file.svml> [--out alignment.json] [--fps 30]+ svml estimate <file.svml> [--out timing.json] [--fps 30]svml canvas <file.svml> [--out canvas.json]svml timeline <file.svml> [--out timeline.json] [--lock svml.lock] [--artifacts artifacts.json]svml compile <file.svml> --out index.html [--lock svml.lock] [--artifacts artifacts.json]diff --git a/src/compiler.ts b/src/compiler.tsindex 1f753ff8..78cde6dd 100644--- a/src/compiler.ts+++ b/src/compiler.ts@@ -16,9 +16,9 @@ import {} from "./kernel.js";import { createLock, freezeLock, verifyLockFile, type SvmlLock } from "./lock.js";import type {- AlignmentEvidence,+ CompleteSemanticMap,AttributeValue,- ExactSemanticMap,+ SpeechTimingEvidence,LocatedScript,PlanIR,SourceDocument,@@ -34,7 +34,7 @@ import type {import { projectKernelIsolated } from "./sandbox.js";import { validateKernelProjection } from "./runtime-validation.js";import { bindSemanticMap, locateScript } from "./script/locate.js";-import { estimateAlignment, type EstimateOptions } from "./script/estimate.js";+import { estimateSpeechTiming, type EstimateOptions } from "./script/estimate.js";import { parseScript } from "./script/parse.js";import { loadSourceClosure } from "./source/modules.js";import { childElements, textContent } from "./source/query.js";@@ -42,7 +42,7 @@ import { applyStyleSheets } from "./source/styles.js";import { sha256, stableJson } from "./util.js";import {createProgramBasis,- semanticMapFromAlignment,+ semanticMapFromTiming,validateSemanticMap,validateTemporalBasisProduction,} from "./temporal.js";@@ -52,7 +52,7 @@ export type Compilation = {narrative: ReturnType<typeof parseScript>;plan: PlanIR;basis: TemporalBasisProduction;- semanticMap: ExactSemanticMap;+ semanticMap: CompleteSemanticMap;located: LocatedScript;html: string;target: HyperframesDocument;@@ -211,13 +211,13 @@ export async function estimateSource(file: string,options: EstimateOptions = {},): Promise<{- evidence: AlignmentEvidence;+ evidence: SpeechTimingEvidence;basis: ReturnType<typeof createProgramBasis>;- map: ReturnType<typeof semanticMapFromAlignment>;+ map: ReturnType<typeof semanticMapFromTiming>;located: LocatedScript;}> {const checked = await checkSource(file);- const evidence = estimateAlignment(checked.narrative, options);+ const evidence = estimateSpeechTiming(checked.narrative, options);const basis = createProgramBasis({fps: evidence.fps,durationFrames: Math.round(evidence.durationSec * evidence.fps),@@ -226,11 +226,10 @@ export async function estimateSource(evidence,})),});- const map = semanticMapFromAlignment(+ const map = semanticMapFromTiming(checked.narrative,basis,evidence,- { precision: "estimated" },);return {evidence,@@ -264,7 +263,7 @@ export async function compileSource(args: {const basisPort = rootManifest.ports.find((port) =>port.direction === "input" && port.type.split("|").includes("TemporalBasisProduction"));const semanticPort = rootManifest.ports.find((port) =>- port.direction === "input" && port.type.split("|").includes("ExactSemanticMap"));+ port.direction === "input" && port.type.split("|").includes("CompleteSemanticMap"));if (!basisPort || !semanticPort) {fail("runtime_composition_contract", "Composition does not expose basis/map input ports.");}@@ -278,7 +277,7 @@ export async function compileSource(args: {const absoluteSource = value.source? pathToFileURL(resolve(dirname(value.module), value.source)).href: undefined;- const data = value.type === "AlignmentEvidence" && value.source+ const data = value.type === "SpeechTimingEvidence" && value.source? JSON.parse(await readFile(resolve(dirname(value.module), value.source), "utf8")): undefined;return [@@ -390,7 +389,8 @@ export async function compileSource(args: {const projection = validateKernelProjection(await projectKernelIsolated(manifest, context));if (located) {for (const port of manifest.ports.filter((candidate) =>- candidate.direction === "output" && candidate.type === "Track")) {+ candidate.direction === "output"+ && (candidate.type === "Track" || candidate.type === "CaptionTrack"))) {validateTrackOutput(instance.id, port.name, projection.outputs[port.name], located.durationFrames);}}@@ -428,12 +428,12 @@ export async function compileSource(args: {) as TemporalBasisProduction | undefined;const semanticMap = resolveValue(rootElement.attributes[semanticPort.name],- ) as ExactSemanticMap | undefined;+ ) as CompleteSemanticMap | undefined;if (production?.contract !== "svml.temporal-basis-production.v1") {fail("runtime_basis_missing", "Composition root must select one TemporalBasisProduction.");}- if (semanticMap?.contract !== "svml.exact-semantic-map.v1") {- fail("runtime_semantic_map_missing", "Composition root must select one ExactSemanticMap.");+ if (semanticMap?.contract !== "svml.complete-semantic-map.v1") {+ fail("runtime_semantic_map_missing", "Composition root must select one CompleteSemanticMap.");}validateTemporalBasisProduction(production);validateSemanticMap(checked.narrative, production.basis, semanticMap);diff --git a/src/format.ts b/src/format.tsindex 7fec0a8e..480b23b5 100644--- a/src/format.ts+++ b/src/format.ts@@ -48,8 +48,8 @@ function semanticNarrative(narrative: NarrativeIR): unknown {captionAtoms: narrative.captionAtoms.map((atom) => ({display: atom.display,segmentId: atom.segmentId,- startWord: atom.startWord,- endWordExclusive: atom.endWordExclusive,+ startToken: atom.startToken,+ endTokenExclusive: atom.endTokenExclusive,})),projections: narrative.projections,};diff --git a/src/model.ts b/src/model.tsindex 27946dee..d84fa62c 100644--- a/src/model.ts+++ b/src/model.ts@@ -64,9 +64,27 @@ export type SourceDocument = {};export type ScriptAtom =- | { kind: "text"; speech: string; caption: string; start: number; end: number }+ | {+ kind: "text";+ speech: string;+ caption: string;+ start: number;+ end: number;+ tokenStart: number;+ tokenEndExclusive: number;+ }| { kind: "role"; label: string; start: number; end: number };+export type SpokenTurn = {+ id: string;+ segmentId: string;+ role?: string;+ tokenStart: number;+ tokenEndExclusive: number;+ sourceStart: number;+ sourceEnd: number;+};+export type ScriptToken = {id: string;index: number;@@ -153,8 +171,8 @@ export type CaptionAtom = {id: string;display: string;segmentId: string;- startWord: number;- endWordExclusive: number;+ startToken: number;+ endTokenExclusive: number;sourceStart: number;sourceEnd: number;};@@ -162,6 +180,7 @@ export type CaptionAtom = {export type NarrativeIR = {segments: ScriptSegment[];tokens: ScriptToken[];+ turns: SpokenTurn[];selections: Record<string, SelectionOccurrence[]>;moments: Record<string, MomentOccurrence[]>;captionAtoms: CaptionAtom[];@@ -206,19 +225,14 @@ export type SemanticMapBase = {evidenceDigests: string[];locatorDigest: string;quantizationPolicy: "nearest-frame";- captionCues?: AlignmentCaptionCue[];mapDigest: string;};-export type EstimatedSemanticMap = SemanticMapBase & {- contract: "svml.estimated-semantic-map.v1";-};--export type ExactSemanticMap = SemanticMapBase & {- contract: "svml.exact-semantic-map.v1";+export type CompleteSemanticMap = SemanticMapBase & {+ contract: "svml.complete-semantic-map.v1";};-export type SemanticMap = EstimatedSemanticMap | ExactSemanticMap;+export type SemanticMap = CompleteSemanticMap;export type SourceToProgramMap = {id: string;@@ -260,35 +274,53 @@ export type TemporalBasisProduction = {productionDigest: string;};-export type AlignmentWord = {+export type SpeechTimingUnit = {text: string;startSec: number;endSec: number;segmentId: string;};-export type AlignmentSegment = {+export type SpeechTimingSegment = {id: string;startSec: number;endSec: number;};-export type AlignmentCaptionCue = {- id?: string;- startWord: number;- endWordExclusive: number;-};--export type AlignmentEvidence = {- contract: "svml.speech-alignment.v1";+export type SpeechTimingEvidence = {+ contract: "svml.speech-timing-evidence.v1";durationSec: number;fps: number;- words: AlignmentWord[];- segments: AlignmentSegment[];- captionCues?: AlignmentCaptionCue[];+ quality: "measured" | "estimated";+ units: SpeechTimingUnit[];+ segments: SpeechTimingSegment[];provenance?: Record<string, unknown>;};+export type CaptionCue = {+ id: string;+ startToken: number;+ endTokenExclusive: number;+};++export type CaptionAnnotation = {+ id: string;+ kind: string;+ startToken: number;+ endTokenExclusive: number;+ value?: string;+};++export type CaptionPlan = {+ contract: "svml.caption-plan.v1";+ semanticIndexDigest: string;+ basisDigest: string;+ cues: CaptionCue[];+ annotations: CaptionAnnotation[];+ plannerDigest: string;+ planDigest: string;+};+export type ProgramRange = {startFrame: number;endFrameExclusive: number;@@ -306,17 +338,10 @@ export type LocatedMoment = {frames: number[];};-export type LocatedCaptionCue = ProgramRange & {- id: string;- startWord: number;- endWordExclusive: number;-};-export type LocatedCaptionAtom = CaptionAtom & ProgramRange;export type LocatedScript = {contract: "svml.temporal-binding.v1";- precision: "estimated" | "exact";basisDigest: string;semanticMapDigest: string;durationFrames: number;@@ -327,12 +352,11 @@ export type LocatedScript = {selections: Record<string, LocatedSelection>;moments: Record<string, LocatedMoment>;captionAtoms: LocatedCaptionAtom[];- captionCues?: LocatedCaptionCue[];};export type PlanValue = {id: string;- type: "Image" | "Video" | "Audio" | "Text" | "AlignmentEvidence";+ type: "Image" | "Video" | "Audio" | "Text" | "SpeechTimingEvidence";source?: string;value?: string;identity: string;diff --git a/src/plan.ts b/src/plan.tsindex fb25fdad..894056b6 100644--- a/src/plan.ts+++ b/src/plan.ts@@ -24,7 +24,7 @@ const VALUE_TYPES: Record<string, PlanValue["type"]> = {video: "Video",audio: "Audio",text: "Text",- alignment: "AlignmentEvidence",+ timing: "SpeechTimingEvidence",};function idOf(element: SourceElement): string {@@ -507,7 +507,7 @@ export function buildPlan(const basisInputs = rootManifest.ports.filter((port) =>port.direction === "input" && port.type.split("|").includes("TemporalBasisProduction"));const semanticInputs = rootManifest.ports.filter((port) =>- port.direction === "input" && port.type.split("|").includes("ExactSemanticMap"));+ port.direction === "input" && port.type.split("|").includes("CompleteSemanticMap"));if (basisInputs.length !== 1|| basisInputs[0]?.cardinality !== "one"@@ -516,10 +516,11 @@ export function buildPlan() {fail("composition_temporal_contract",- `Composition Kernel "${rootManifest.name}" must select exactly one TemporalBasisProduction and one ExactSemanticMap.`,+ `Composition Kernel "${rootManifest.name}" must select exactly one TemporalBasisProduction and one CompleteSemanticMap.`,);}- const trackEdges = reachableEdges.filter((edge) => edge.type === "Track");+ const trackEdges = reachableEdges.filter((edge) =>+ edge.type === "Track" || edge.type === "CaptionTrack");const illegalTrackConsumer = trackEdges.find((edge) => edge.to !== root);if (illegalTrackConsumer) {fail(@@ -535,6 +536,15 @@ export function buildPlan(`Composition "${root}" references Track "${duplicateTrack.from}" more than once.`,);}+ const captionTracks = reachableInstances.filter((instance) =>+ kernels.get(instance.kernel)?.ports.some((port) =>+ port.direction === "output" && port.type === "CaptionTrack"));+ if (captionTracks.length > 1) {+ fail(+ "caption_track_cardinality",+ `A Composition may contain at most one CaptionTrack; received ${captionTracks.length}.`,+ );+ }const usedKernels = new Set(reachableInstances.map((instance) => instance.kernel));return {contract: "svml.plan.v1",diff --git a/src/script/estimate.ts b/src/script/estimate.tsindex 4f8a7d3c..6e8c1496 100644--- a/src/script/estimate.ts+++ b/src/script/estimate.ts@@ -1,8 +1,7 @@import type {- AlignmentCaptionCue,- AlignmentEvidence,NarrativeIR,ScriptToken,+ SpeechTimingEvidence,} from "../model.js";export type EstimateOptions = {@@ -10,7 +9,6 @@ export type EstimateOptions = {wordGapSec?: number;segmentPaddingSec?: number;emptySegmentSec?: number;- maxCaptionWords?: number;fps?: number;};@@ -27,33 +25,14 @@ function syllables(token: ScriptToken): number {return Math.max(1, groups - silentE);}-function captionCues(narrative: NarrativeIR, maxWords: number): AlignmentCaptionCue[] {- const output: AlignmentCaptionCue[] = [];- for (const segment of narrative.segments) {- for (- let startWord = segment.tokenStart;- startWord < segment.tokenEnd;- startWord += maxWords- ) {- output.push({- id: `estimate-cue-${output.length + 1}`,- startWord,- endWordExclusive: Math.min(segment.tokenEnd, startWord + maxWords),- });- }- }- return output;-}--export function estimateAlignment(+export function estimateSpeechTiming(narrative: NarrativeIR,options: EstimateOptions = {},-): AlignmentEvidence {+): SpeechTimingEvidence {const syllablesPerSecond = options.syllablesPerSecond ?? 6.5;const wordGapSec = options.wordGapSec ?? 0.045;const segmentPaddingSec = options.segmentPaddingSec ?? 0.12;const emptySegmentSec = options.emptySegmentSec ?? 1;- const maxCaptionWords = Math.max(1, Math.floor(options.maxCaptionWords ?? 3));const fps = Math.max(1, Math.round(options.fps ?? 30));for (const [name, value] of Object.entries({syllablesPerSecond,@@ -67,8 +46,8 @@ export function estimateAlignment(}let cursor = 0;- const words: AlignmentEvidence["words"] = [];- const segments: AlignmentEvidence["segments"] = [];+ const units: SpeechTimingEvidence["units"] = [];+ const segments: SpeechTimingEvidence["segments"] = [];for (const segment of narrative.segments) {const segmentTokens = narrative.tokens.slice(segment.tokenStart, segment.tokenEnd);const startSec = cursor;@@ -82,7 +61,7 @@ export function estimateAlignment(for (const [index, token] of segmentTokens.entries()) {const wordStart = cursor;cursor += syllables(token) / syllablesPerSecond;- words.push({+ units.push({text: token.text,startSec: wordStart,endSec: cursor,@@ -97,12 +76,12 @@ export function estimateAlignment(const durationFrames = Math.max(1, Math.ceil(cursor * fps));const durationSec = durationFrames / fps;return {- contract: "svml.speech-alignment.v1",+ contract: "svml.speech-timing-evidence.v1",durationSec,fps,- words,+ quality: "estimated",+ units,segments,- captionCues: captionCues(narrative, maxCaptionWords),provenance: {method: "svml.syllable-estimate.v1",measured: false,@@ -110,7 +89,6 @@ export function estimateAlignment(wordGapSec,segmentPaddingSec,emptySegmentSec,- maxCaptionWords,},};}diff --git a/src/script/locate.ts b/src/script/locate.tsindex 40a39211..040acbb4 100644--- a/src/script/locate.ts+++ b/src/script/locate.ts@@ -1,6 +1,6 @@import { fail } from "../diagnostics.js";import type {- ExactSemanticMap,+ CompleteSemanticMap,LocatedScript,MarkerBoundary,NarrativeIR,@@ -61,77 +61,13 @@ export function boundaryAnchor(?? fail("locate_boundary_empty", "Script has no structural boundary candidate.");}-function validateCaptionCues(- narrative: NarrativeIR,- locatedWords: LocatedScript["words"],- locatedAtoms: LocatedScript["captionAtoms"],- map: SemanticMap,- basis: ProgramBasis,-): LocatedScript["captionCues"] {- const cues = map.captionCues;- if (!cues) return undefined;- const located = cues.map((cue, index) => {- if (!Number.isInteger(cue.startWord) || !Number.isInteger(cue.endWordExclusive)) {- fail("locate_caption_cue_index", `Caption cue ${index + 1} must use integer token indexes.`);- }- if (- cue.startWord < 0- || cue.endWordExclusive <= cue.startWord- || cue.endWordExclusive > locatedWords.length- ) {- fail(- "locate_caption_cue_range",- `Caption cue ${index + 1} has invalid token range ${cue.startWord}..${cue.endWordExclusive}.`,- );- }- const previous = cues[index - 1];- const expectedStart = previous?.endWordExclusive ?? 0;- if (cue.startWord !== expectedStart) {- fail(- "locate_caption_cue_partition",- `Caption cue ${index + 1} starts at token ${cue.startWord}; expected ${expectedStart}.`,- );- }- const first = locatedWords[cue.startWord];- const last = locatedWords[cue.endWordExclusive - 1];- if (!first || !last || first.segmentId !== last.segmentId) {- fail("locate_caption_cue_segment", `Caption cue ${index + 1} crosses Script Segments.`);- }- for (const atom of locatedAtoms) {- const overlaps = cue.startWord < atom.endWordExclusive- && cue.endWordExclusive > atom.startWord;- const contains = cue.startWord <= atom.startWord- && cue.endWordExclusive >= atom.endWordExclusive;- if (overlaps && !contains) {- fail(- "locate_caption_partial_dual",- `Caption cue ${index + 1} cuts through Dual Text atom "${atom.id}".`,- );- }- }- return {- id: cue.id ?? `cue-${index + 1}`,- startWord: cue.startWord,- endWordExclusive: cue.endWordExclusive,- ...range(first.startFrame, last.endFrameExclusive, basis, `Caption cue ${index + 1}`),- };- });- if ((located.at(-1)?.endWordExclusive ?? 0) !== narrative.tokens.length) {- fail(- "locate_caption_cue_partition",- `Caption cues end at token ${located.at(-1)?.endWordExclusive ?? 0}; expected ${narrative.tokens.length}.`,- );- }- return located;-}-export function locateScript(narrative: NarrativeIR,basis: ProgramBasis,- map: ExactSemanticMap,+ map: CompleteSemanticMap,): LocatedScript {- if (map.contract !== "svml.exact-semantic-map.v1") {- fail("locate_exact_required", "Located HTML compilation requires ExactSemanticMap.");+ if (map.contract !== "svml.complete-semantic-map.v1") {+ fail("locate_complete_required", "HTML compilation requires CompleteSemanticMap.");}return bindSemanticMap(narrative, basis, map);}@@ -194,8 +130,8 @@ export function bindSemanticMap(],));const captionAtoms = narrative.captionAtoms.map((atom) => {- const first = words[atom.startWord];- const last = words[atom.endWordExclusive - 1];+ const first = words[atom.startToken];+ const last = words[atom.endTokenExclusive - 1];if (!first || !last || first.segmentId !== last.segmentId) {fail("locate_caption_atom", `Caption atom "${atom.id}" has an invalid speech span.`);}@@ -204,10 +140,8 @@ export function bindSemanticMap(...range(first.startFrame, last.endFrameExclusive, basis, `Caption atom "${atom.id}"`),};});- const captionCues = validateCaptionCues(narrative, words, captionAtoms, map, basis);return {contract: "svml.temporal-binding.v1",- precision: map.contract === "svml.exact-semantic-map.v1" ? "exact" : "estimated",basisDigest: basis.basisDigest,semanticMapDigest: map.mapDigest,durationFrames: basis.durationFrames,@@ -218,6 +152,5 @@ export function bindSemanticMap(selections,moments,captionAtoms,- ...(captionCues ? { captionCues } : {}),};}diff --git a/src/script/parse.ts b/src/script/parse.tsindex 4f626dab..09cb6a48 100644--- a/src/script/parse.ts+++ b/src/script/parse.ts@@ -291,13 +291,7 @@ export function parseScript(}return;}- current.atoms.push({- kind: "text",- speech,- caption,- start: sourceOffset + start,- end: sourceOffset + end,- });+ const tokenStart = tokens.length;WORD.lastIndex = 0;let match: RegExpExecArray | null;while ((match = WORD.exec(speech))) {@@ -318,6 +312,15 @@ export function parseScript(sourceEnd: sourceOffset + start + match.index + match[0].length,});}+ current.atoms.push({+ kind: "text",+ speech,+ caption,+ start: sourceOffset + start,+ end: sourceOffset + end,+ tokenStart,+ tokenEndExclusive: tokens.length,+ });};const consumeSpeechSide = (@@ -325,7 +328,7 @@ export function parseScript(absoluteStart: number,caption: string,): void => {- const startWord = tokens.length;+ const startToken = tokens.length;let partStart = 0;let index = 0;let emittedCaption = false;@@ -370,14 +373,14 @@ export function parseScript(partStart = index;}addLiteral(raw.slice(partStart), absoluteStart + partStart);- const endWordExclusive = tokens.length;- if (endWordExclusive > startWord) {+ const endTokenExclusive = tokens.length;+ if (endTokenExclusive > startToken) {captionAtoms.push({id: `caption-atom-${captionAtoms.length + 1}`,display: cleanProjection(caption),segmentId: current!.id,- startWord,- endWordExclusive,+ startToken,+ endTokenExclusive,sourceStart: sourceOffset + absoluteStart,sourceEnd: sourceOffset + absoluteStart + raw.length,});@@ -553,16 +556,39 @@ export function parseScript(const speechSegments: string[] = [];const captionSegments: string[] = [];const dialogueTurns: string[] = [];+ const turns: NarrativeIR["turns"] = [];for (const segment of segments) {const speechParts: string[] = [];const captionParts: string[] = [];let activeRole: string | undefined;let turnParts: string[] = [];+ let turnTokenStart: number | undefined;+ let turnTokenEndExclusive: number | undefined;+ let turnSourceStart: number | undefined;+ let turnSourceEnd: number | undefined;const flushTurn = (): void => {const body = joinProjection(turnParts);- if (!body) return;- dialogueTurns.push(activeRole ? `${activeRole}: ${body}` : body);+ if (body) dialogueTurns.push(activeRole ? `${activeRole}: ${body}` : body);+ if (+ turnTokenStart !== undefined+ && turnTokenEndExclusive !== undefined+ && turnTokenEndExclusive > turnTokenStart+ ) {+ turns.push({+ id: `turn-${turns.length + 1}`,+ segmentId: segment.id,+ ...(activeRole ? { role: activeRole } : {}),+ tokenStart: turnTokenStart,+ tokenEndExclusive: turnTokenEndExclusive,+ sourceStart: turnSourceStart ?? segment.sourceStart,+ sourceEnd: turnSourceEnd ?? segment.sourceEnd,+ });+ }turnParts = [];+ turnTokenStart = undefined;+ turnTokenEndExclusive = undefined;+ turnSourceStart = undefined;+ turnSourceEnd = undefined;};for (const atom of segment.atoms) {if (atom.kind === "role") {@@ -572,6 +598,12 @@ export function parseScript(speechParts.push(atom.speech);captionParts.push(atom.caption);turnParts.push(atom.speech);+ if (atom.tokenEndExclusive > atom.tokenStart) {+ turnTokenStart ??= atom.tokenStart;+ turnTokenEndExclusive = atom.tokenEndExclusive;+ turnSourceStart ??= atom.start;+ turnSourceEnd = atom.end;+ }}}flushTurn();@@ -626,6 +658,7 @@ export function parseScript(return {segments,tokens,+ turns,selections,moments,captionAtoms,diff --git a/src/temporal.ts b/src/temporal.tsindex 3e12b189..6f1d36a8 100644--- a/src/temporal.ts+++ b/src/temporal.ts@@ -1,12 +1,11 @@import { fail } from "./diagnostics.js";import type {- AlignmentEvidence,- ExactSemanticMap,- EstimatedSemanticMap,+ CompleteSemanticMap,NarrativeIR,ProgramBasis,SemanticAnchorPoint,SemanticMap,+ SpeechTimingEvidence,TemporalBasisProduction,} from "./model.js";import { normalizeWord, sha256, stableJson } from "./util.js";@@ -222,17 +221,17 @@ function frameAt(seconds: number, basis: ProgramBasis): number {return Math.round(seconds * basisFps(basis));}-function semanticMapDigest(map: Omit<SemanticMap, "mapDigest">): string {+function semanticMapDigest(map: Omit<CompleteSemanticMap, "mapDigest">): string {return sha256(stableJson(map));}-function validateAlignment(+function validateSpeechTiming(narrative: NarrativeIR,basis: ProgramBasis,- evidence: AlignmentEvidence,+ evidence: SpeechTimingEvidence,): void {- if (evidence.contract !== "svml.speech-alignment.v1") {- fail("locator_evidence_contract", `Unsupported alignment contract "${evidence.contract}".`);+ if (evidence.contract !== "svml.speech-timing-evidence.v1") {+ fail("locator_evidence_contract", `Unsupported speech timing contract "${evidence.contract}".`);}if (!Number.isFinite(evidence.fps)@@ -240,40 +239,37 @@ function validateAlignment(|| !Number.isFinite(evidence.durationSec)|| evidence.durationSec <= 0) {- fail("locator_evidence_clock", "Alignment evidence requires positive fps and duration.");+ fail("locator_evidence_clock", "SpeechTimingEvidence requires positive fps and duration.");+ }+ if (evidence.quality !== "measured" && evidence.quality !== "estimated") {+ fail("locator_evidence_quality", "SpeechTimingEvidence quality must be measured or estimated.");}if (Math.abs(evidence.fps - basisFps(basis)) > EPSILON) {fail("locator_basis_frame_rate",- `Alignment fps ${evidence.fps} does not match ProgramBasis fps ${basisFps(basis)}.`,+ `Speech timing fps ${evidence.fps} does not match ProgramBasis fps ${basisFps(basis)}.`,);}if (frameAt(evidence.durationSec, basis) !== basis.durationFrames) {fail("locator_basis_duration",- `Alignment duration resolves to ${frameAt(evidence.durationSec, basis)} frames; ProgramBasis has ${basis.durationFrames}.`,- );- }- if (evidence.words.length !== narrative.tokens.length) {- fail(- "locator_word_cardinality",- `Script has ${narrative.tokens.length} tokens but alignment has ${evidence.words.length}.`,+ `Speech timing duration resolves to ${frameAt(evidence.durationSec, basis)} frames; ProgramBasis has ${basis.durationFrames}.`,);}if (evidence.segments.length !== narrative.segments.length) {fail("locator_segment_cardinality",- `Script has ${narrative.segments.length} Segments but alignment has ${evidence.segments.length}.`,+ `Script has ${narrative.segments.length} Segments but speech timing has ${evidence.segments.length}.`,);}const segments = new Map(evidence.segments.map((segment) => [segment.id, segment]));if (segments.size !== evidence.segments.length) {- fail("locator_segment_duplicate", "Alignment evidence repeats a Segment id.");+ fail("locator_segment_duplicate", "SpeechTimingEvidence repeats a Segment id.");}for (const [index, segment] of narrative.segments.entries()) {const aligned = segments.get(segment.id);- if (!aligned) fail("locator_segment_missing", `Alignment is missing Segment "${segment.id}".`);+ if (!aligned) fail("locator_segment_missing", `Speech timing is missing Segment "${segment.id}".`);if (!Number.isFinite(aligned.startSec)|| !Number.isFinite(aligned.endSec)@@ -302,99 +298,200 @@ function validateAlignment(}}- const previousBySegment = new Map<string, AlignmentEvidence["words"][number]>();- for (const [index, token] of narrative.tokens.entries()) {- const word = evidence.words[index];- if (!word) fail("locator_word_missing", `Alignment is missing token ${index + 1}.`);- if (normalizeWord(word.text) !== token.normalized || word.segmentId !== token.segmentId) {- fail(- "locator_word_mismatch",- `Token ${index + 1} does not match alignment word "${word.text}" in Segment "${word.segmentId}".`,- );- }- const segment = segments.get(word.segmentId);+ const previousBySegment = new Map<string, SpeechTimingEvidence["units"][number]>();+ for (const [index, unit] of evidence.units.entries()) {+ const segment = segments.get(unit.segmentId);if (!segment- || !Number.isFinite(word.startSec)- || !Number.isFinite(word.endSec)- || word.startSec < segment.startSec - EPSILON- || word.endSec < word.startSec- || word.endSec > segment.endSec + EPSILON+ || typeof unit.text !== "string"+ || !Number.isFinite(unit.startSec)+ || !Number.isFinite(unit.endSec)+ || unit.startSec < segment.startSec - EPSILON+ || unit.endSec < unit.startSec+ || unit.endSec > segment.endSec + EPSILON) {fail("locator_word_window",- `Alignment token ${index + 1} falls outside Segment "${word.segmentId}".`,+ `Speech timing unit ${index + 1} falls outside Segment "${unit.segmentId}".`,);}- const previous = previousBySegment.get(word.segmentId);- if (previous && word.startSec < previous.endSec - EPSILON) {+ const previous = previousBySegment.get(unit.segmentId);+ if (previous && unit.startSec < previous.endSec - EPSILON) {fail("locator_word_overlap",- `Alignment token ${index + 1} overlaps the preceding token in Segment "${word.segmentId}".`,+ `Speech timing unit ${index + 1} overlaps the preceding unit in Segment "${unit.segmentId}".`,);}- previousBySegment.set(word.segmentId, word);+ previousBySegment.set(unit.segmentId, unit);}}-export function semanticMapFromAlignment(- narrative: NarrativeIR,- basis: ProgramBasis,- evidence: AlignmentEvidence,- options: { precision: "exact"; locatorDigest?: string },-): ExactSemanticMap;-export function semanticMapFromAlignment(- narrative: NarrativeIR,+type TokenFrame = {+ startFrame: number;+ endFrameExclusive: number;+ quality: "estimated" | "derived" | "measured";+};++function lcsMatches(+ tokens: NarrativeIR["tokens"],+ units: SpeechTimingEvidence["units"],+): Array<[number, number]> {+ const rows = Array.from({ length: tokens.length + 1 }, () =>+ Array<number>(units.length + 1).fill(0));+ for (let left = tokens.length - 1; left >= 0; left -= 1) {+ for (let right = units.length - 1; right >= 0; right -= 1) {+ rows[left]![right] = tokens[left]!.normalized === normalizeWord(units[right]!.text)+ ? rows[left + 1]![right + 1]! + 1+ : Math.max(rows[left + 1]![right]!, rows[left]![right + 1]!);+ }+ }+ const matches: Array<[number, number]> = [];+ let left = 0;+ let right = 0;+ while (left < tokens.length && right < units.length) {+ if (+ tokens[left]!.normalized === normalizeWord(units[right]!.text)+ && rows[left]![right] === rows[left + 1]![right + 1]! + 1+ ) {+ matches.push([left, right]);+ left += 1;+ right += 1;+ } else if (rows[left + 1]![right]! >= rows[left]![right + 1]!) {+ left += 1;+ } else {+ right += 1;+ }+ }+ return matches;+}++function distributeFrames(+ tokens: NarrativeIR["tokens"],+ startFrame: number,+ endFrameExclusive: number,+ quality: TokenFrame["quality"],+): TokenFrame[] {+ if (!tokens.length) return [];+ const weights = tokens.map((token) => Math.max(1, [...token.normalized].length));+ const total = weights.reduce((sum, weight) => sum + weight, 0);+ let accumulated = 0;+ return tokens.map((_, index) => {+ const start = startFrame + Math.round((endFrameExclusive - startFrame) * accumulated / total);+ accumulated += weights[index]!;+ const end = index === tokens.length - 1+ ? endFrameExclusive+ : startFrame + Math.round((endFrameExclusive - startFrame) * accumulated / total);+ return { startFrame: start, endFrameExclusive: Math.max(start, end), quality };+ });+}++function alignSegment(+ tokens: NarrativeIR["tokens"],+ units: SpeechTimingEvidence["units"],+ segmentStartFrame: number,+ segmentEndFrame: number,basis: ProgramBasis,- evidence: AlignmentEvidence,- options: { precision: "estimated"; locatorDigest?: string },-): EstimatedSemanticMap;-export function semanticMapFromAlignment(+ evidenceQuality: SpeechTimingEvidence["quality"],+): TokenFrame[] {+ const output: Array<TokenFrame | undefined> = Array(tokens.length).fill(undefined);+ const matches = lcsMatches(tokens, units);+ let tokenCursor = 0;+ let unitCursor = 0;+ for (let matchIndex = 0; matchIndex <= matches.length; matchIndex += 1) {+ const match = matches[matchIndex];+ const tokenEnd = match?.[0] ?? tokens.length;+ const unitEnd = match?.[1] ?? units.length;+ const tokenRun = tokens.slice(tokenCursor, tokenEnd);+ const unitRun = units.slice(unitCursor, unitEnd);+ if (tokenRun.length) {+ const previous = tokenCursor > 0 ? output[tokenCursor - 1] : undefined;+ const nextUnit = match ? units[unitEnd] : undefined;+ const startFrame = unitRun.length+ ? frameAt(unitRun[0]!.startSec, basis)+ : previous?.endFrameExclusive ?? segmentStartFrame;+ const endFrame = unitRun.length+ ? frameAt(unitRun.at(-1)!.endSec, basis)+ : nextUnit+ ? frameAt(nextUnit.startSec, basis)+ : segmentEndFrame;+ const quality = evidenceQuality === "estimated" || !unitRun.length+ ? "estimated" as const+ : "derived" as const;+ for (const [offset, value] of distributeFrames(+ tokenRun,+ startFrame,+ Math.max(startFrame, endFrame),+ quality,+ ).entries()) output[tokenCursor + offset] = value;+ }+ if (match) {+ const [tokenIndex, unitIndex] = match;+ const unit = units[unitIndex]!;+ output[tokenIndex] = {+ startFrame: frameAt(unit.startSec, basis),+ endFrameExclusive: frameAt(unit.endSec, basis),+ quality: evidenceQuality === "measured" ? "measured" : "estimated",+ };+ tokenCursor = tokenIndex + 1;+ unitCursor = unitIndex + 1;+ }+ }+ return output.map((value, index) => value+ ?? fail("locator_token_unmapped", `Script token ${index + 1} could not be aligned.`));+}++export function semanticMapFromTiming(narrative: NarrativeIR,basis: ProgramBasis,- evidence: AlignmentEvidence,- options: {- precision: "estimated" | "exact";- locatorDigest?: string;- },-): EstimatedSemanticMap | ExactSemanticMap {+ evidence: SpeechTimingEvidence,+ options: { locatorDigest?: string; evidenceDigest?: string } = {},+): CompleteSemanticMap {validateProgramBasis(basis);- validateAlignment(narrative, basis, evidence);+ validateSpeechTiming(narrative, basis, evidence);const segmentEvidence = new Map(evidence.segments.map((segment) => [segment.id, segment]));- const wordEvidence = new Map(narrative.tokens.map((token, index) => [token.id, evidence.words[index]!]));- const quality = options.precision === "exact" ? "measured" : "estimated";+ const tokenFrames = new Map<string, TokenFrame>();+ for (const segment of narrative.segments) {+ const measuredSegment = segmentEvidence.get(segment.id)!;+ const tokens = narrative.tokens.slice(segment.tokenStart, segment.tokenEnd);+ const units = evidence.units.filter((unit) => unit.segmentId === segment.id);+ const aligned = alignSegment(+ tokens,+ units,+ frameAt(measuredSegment.startSec, basis),+ frameAt(measuredSegment.endSec, basis),+ basis,+ evidence.quality,+ );+ tokens.forEach((token, index) => tokenFrames.set(token.id, aligned[index]!));+ }const anchors: SemanticAnchorPoint[] = narrative.semanticIndex.anchors.map((anchor) => {const segment = segmentEvidence.get(anchor.segmentId);- if (!segment) fail("locator_segment_missing", `Alignment is missing Segment "${anchor.segmentId}".`);- const word = anchor.tokenId ? wordEvidence.get(anchor.tokenId) : undefined;- const seconds = anchor.kind === "segment-start"- ? segment.startSec+ if (!segment) fail("locator_segment_missing", `Speech timing is missing Segment "${anchor.segmentId}".`);+ const token = anchor.tokenId ? tokenFrames.get(anchor.tokenId) : undefined;+ const frame = anchor.kind === "segment-start"+ ? frameAt(segment.startSec, basis): anchor.kind === "segment-end"- ? segment.endSec+ ? frameAt(segment.endSec, basis): anchor.kind === "token-start"- ? word?.startSec- : word?.endSec;- if (seconds === undefined) {- fail("locator_anchor_missing", `No alignment time exists for anchor "${anchor.id}".`);+ ? token?.startFrame+ : token?.endFrameExclusive;+ if (frame === undefined) {+ fail("locator_anchor_missing", `No speech timing exists for anchor "${anchor.id}".`);}return {identity: anchor.id,- point: { basisDigest: basis.basisDigest, frame: frameAt(seconds, basis) },- quality,+ point: { basisDigest: basis.basisDigest, frame },+ quality: token?.quality ?? evidence.quality,};});- const contract = options.precision === "exact"- ? "svml.exact-semantic-map.v1" as const- : "svml.estimated-semantic-map.v1" as const;const payload = {- contract,+ contract: "svml.complete-semantic-map.v1" as const,semanticIndexDigest: narrative.semanticIndex.digest,basisDigest: basis.basisDigest,anchors,- evidenceDigests: [sha256(stableJson(evidence))],- locatorDigest: options.locatorDigest ?? sha256(`svml.${options.precision}-speech-locator.v1`),+ evidenceDigests: [options.evidenceDigest ?? sha256(stableJson(evidence))],+ locatorDigest: options.locatorDigest ?? sha256("svml.script-timing-locator.v1"),quantizationPolicy: "nearest-frame" as const,- ...(evidence.captionCues ? { captionCues: evidence.captionCues } : {}),};const map = { ...payload, mapDigest: semanticMapDigest(payload) };validateSemanticMap(narrative, basis, map);@@ -408,8 +505,7 @@ export function validateSemanticMap(): void {validateProgramBasis(basis);if (- map.contract !== "svml.estimated-semantic-map.v1"- && map.contract !== "svml.exact-semantic-map.v1"+ map.contract !== "svml.complete-semantic-map.v1") {fail("semantic_map_contract", "Unsupported SemanticMap contract.");}@@ -446,8 +542,8 @@ export function validateSemanticMap() {fail("semantic_map_point", `SemanticMap anchor "${actual.identity}" is outside ProgramBasis.`);}- if (map.contract === "svml.exact-semantic-map.v1" && actual.quality === "estimated") {- fail("semantic_map_precision", `ExactSemanticMap anchor "${actual.identity}" is estimated.`);+ if (!["measured", "derived", "estimated"].includes(actual.quality)) {+ fail("semantic_map_quality", `SemanticMap anchor "${actual.identity}" has invalid quality.`);}if (pointById.has(actual.identity)) {fail("semantic_map_duplicate", `SemanticMap repeats anchor "${actual.identity}".`);diff --git a/src/views.ts b/src/views.tsindex 3753de35..51278874 100644--- a/src/views.ts+++ b/src/views.ts@@ -31,7 +31,6 @@ export type TimelineView = {segments: Compilation["located"]["segments"];selections: Compilation["located"]["selections"];moments: Compilation["located"]["moments"];- captionCues: Compilation["located"]["captionCues"];visuals: Compilation["target"]["visuals"];audios: Compilation["target"]["audios"];};@@ -71,7 +70,6 @@ export function projectTimeline(compilation: Compilation): TimelineView {segments: compilation.located.segments,selections: compilation.located.selections,moments: compilation.located.moments,- captionCues: compilation.located.captionCues,visuals: compilation.target.visuals,audios: compilation.target.audios,};diff --git a/stdlib/caption-planner.svk b/stdlib/caption-planner.svknew file mode 100644index 00000000..934cb4b5--- /dev/null+++ b/stdlib/caption-planner.svk@@ -0,0 +1,13 @@+<kernel name="caption-planner" abi="1" implementation="./runtime/caption-planner.mjs">+ <param name="maxWords" style="max-words" type="number" default="3"/>+ <port name="script" direction="input" type="NarrativeIR" cardinality="one"/>+ <port name="semantic" direction="input" type="CompleteSemanticMap" cardinality="one"/>+ <port name="plan" direction="output" type="CaptionPlan" cardinality="one"/>+ <child name="annotate" cardinality="zero-or-more">+ <field name="id" type="id" required="true"/>+ <field name="kind" type="string" required="true"/>+ <field name="value" type="string"/>+ <field name="during" type="SelectionSet|ProgramSpan" consume="set"/>+ <field name="role" type="string"/>+ </child>+</kernel>diff --git a/stdlib/caption-track.svk b/stdlib/caption-track.svkindex d96c40ea..2af1e667 100644--- a/stdlib/caption-track.svk+++ b/stdlib/caption-track.svk@@ -3,7 +3,6 @@<param name="x" type="number" default="0.37"/><param name="y" type="number" default="0.54"/><param name="width" type="number" default="0.64"/>- <param name="maxWords" style="max-words" type="number" default="3"/><param name="hold" type="number" default="0.08"/><param name="cueFade" style="cue-fade" type="number" default="0.12"/><param name="font" style="font-family" type="string" default="Poppins, Inter, Arial, sans-serif"/>@@ -16,15 +15,24 @@<param name="strokeColor" style="stroke-color" type="string" default="rgba(0,0,0,.82)"/><param name="shadow" type="string" default="0 2px 9px rgba(0,0,0,.5), 0 0 2px rgba(0,0,0,.68)"/><port name="script" direction="input" type="NarrativeIR" cardinality="one"/>- <port name="semantic" direction="input" type="ExactSemanticMap" cardinality="one"/>- <port name="track" direction="output" type="Track" cardinality="one"/>+ <port name="semantic" direction="input" type="CompleteSemanticMap" cardinality="one"/>+ <port name="plan" direction="input" type="CaptionPlan" cardinality="one"/>+ <port name="track" direction="output" type="CaptionTrack" cardinality="one"/><child name="style" cardinality="zero-or-more"><field name="id" type="id" required="true"/>- <field name="during" type="SelectionSet|ProgramSpan" consume="set" required="true"/>+ <field name="during" type="SelectionSet|ProgramSpan" consume="set"/>+ <field name="role" type="string"/>+ <field name="annotation" type="string"/><field name="color" style="fill" type="string"/><field name="activeColor" style="active-fill" type="string"/><field name="background" type="string"/><field name="weight" type="number"/><field name="scale" type="number"/></child>+ <child name="mute" cardinality="zero-or-more">+ <field name="id" type="id" required="true"/>+ <field name="during" type="SelectionSet|ProgramSpan" consume="set"/>+ <field name="role" type="string"/>+ <field name="annotation" type="string"/>+ </child></kernel>diff --git a/stdlib/film.svk b/stdlib/film.svkindex 5c91952d..9c36f6ff 100644--- a/stdlib/film.svk+++ b/stdlib/film.svk@@ -4,10 +4,10 @@<param name="fps" style="frame-rate" type="number"/><param name="background" type="string" default="#000000"/><port name="basis" direction="input" type="TemporalBasisProduction" cardinality="one"/>- <port name="semantic" direction="input" type="ExactSemanticMap" cardinality="one"/>- <port name="track" direction="input" type="Track" cardinality="zero-or-more"/>+ <port name="semantic" direction="input" type="CompleteSemanticMap" cardinality="one"/>+ <port name="track" direction="input" type="Track|CaptionTrack" cardinality="zero-or-more"/><port name="document" direction="output" type="HyperframesDocument" cardinality="one"/><child name="track" cardinality="zero-or-more">- <field name="ref" type="Track" required="true"/>+ <field name="ref" type="Track|CaptionTrack" required="true"/></child></kernel>diff --git a/stdlib/runtime/caption-planner.mjs b/stdlib/runtime/caption-planner.mjsnew file mode 100644index 00000000..0fb78dbb--- /dev/null+++ b/stdlib/runtime/caption-planner.mjs@@ -0,0 +1,95 @@+import { elements, numberAttr, stringAttr } from "./helpers.mjs";++function fail(message) {+ throw new Error(message);+}++export default {+ abiVersion: "1",+ project(context) {+ const script = context.resolve(context.element.attributes.script);+ const semantic = context.resolve(context.element.attributes.semantic);+ if (!script?.semanticIndex) fail("caption-planner requires NarrativeIR");+ if (semantic?.contract !== "svml.complete-semantic-map.v1") {+ fail("caption-planner requires CompleteSemanticMap");+ }+ if (semantic.semanticIndexDigest !== script.semanticIndex.digest) {+ fail("caption-planner SemanticMap does not match Script");+ }+ const points = new Map(semantic.anchors.map(anchor => [anchor.identity, anchor.point.frame]));+ const words = script.tokens.map(token => ({+ ...token,+ startFrame: points.get(token.startAnchorId),+ endFrameExclusive: points.get(token.endAnchorId),+ }));+ const maxWords = Math.max(1, Math.round(numberAttr(context.element, "maxWords", 3)));+ const atomsByStart = new Map(script.captionAtoms.map(atom => [atom.startToken, atom]));+ const cues = [];+ for (const segment of script.segments) {+ const units = [];+ let token = segment.tokenStart;+ while (token < segment.tokenEnd) {+ const atom = atomsByStart.get(token);+ if (atom) {+ units.push({ startToken: atom.startToken, endTokenExclusive: atom.endTokenExclusive });+ token = atom.endTokenExclusive;+ } else {+ units.push({ startToken: token, endTokenExclusive: token + 1 });+ token += 1;+ }+ }+ for (let index = 0; index < units.length; index += maxWords) {+ const group = units.slice(index, index + maxWords);+ cues.push({+ id: `cue-${cues.length + 1}`,+ startToken: group[0].startToken,+ endTokenExclusive: group.at(-1).endTokenExclusive,+ });+ }+ }+ const frameRangesForRole = role => script.turns+ .filter(turn => turn.role === role)+ .map(turn => ({+ startFrame: words[turn.tokenStart]?.startFrame,+ endFrameExclusive: words[turn.tokenEndExclusive - 1]?.endFrameExclusive,+ }))+ .filter(range => Number.isInteger(range.startFrame) && Number.isInteger(range.endFrameExclusive));+ const annotations = [];+ for (const item of elements(context, "annotate")) {+ const role = item.attributes.role;+ const during = item.attributes.during;+ if ((role === undefined) === (during === undefined)) {+ fail(`<annotate id="${stringAttr(item, "id")}"> requires exactly one of role or during`);+ }+ const ranges = role !== undefined+ ? frameRangesForRole(stringAttr(item, "role"))+ : context.selection(during, "annotate.during");+ for (const [occurrence, range] of ranges.entries()) {+ const selected = words.filter(word =>+ word.startFrame < range.endFrameExclusive+ && word.endFrameExclusive > range.startFrame);+ if (!selected.length) continue;+ annotations.push({+ id: `${stringAttr(item, "id")}:${occurrence + 1}`,+ kind: stringAttr(item, "kind"),+ startToken: selected[0].index,+ endTokenExclusive: selected.at(-1).index + 1,+ ...(typeof item.attributes.value === "string" ? { value: item.attributes.value } : {}),+ });+ }+ }+ const plannerDigest = context.digest({+ implementation: context.instance.executionDigest,+ maxWords,+ });+ const payload = {+ contract: "svml.caption-plan.v1",+ semanticIndexDigest: script.semanticIndex.digest,+ basisDigest: semantic.basisDigest,+ cues,+ annotations,+ plannerDigest,+ };+ return { outputs: { plan: { ...payload, planDigest: context.digest(payload) } } };+ },+};diff --git a/stdlib/runtime/caption-track.mjs b/stdlib/runtime/caption-track.mjsindex 20658cc1..d140dc6b 100644--- a/stdlib/runtime/caption-track.mjs+++ b/stdlib/runtime/caption-track.mjs@@ -1,66 +1,101 @@import { elements, html, numberAttr, stringAttr, trackOutput } from "./helpers.mjs";+function fail(message) {+ throw new Error(message);+}+export default {abiVersion: "1",project(context) {const script = context.resolve(context.element.attributes.script);const semantic = context.resolve(context.element.attributes.semantic);- if (semantic?.contract !== "svml.exact-semantic-map.v1") {- throw new Error("caption-track requires ExactSemanticMap");+ const plan = context.resolve(context.element.attributes.plan);+ if (semantic?.contract !== "svml.complete-semantic-map.v1") {+ fail("caption-track requires CompleteSemanticMap");+ }+ if (plan?.contract !== "svml.caption-plan.v1") fail("caption-track requires CaptionPlan");+ if (+ semantic.basisDigest !== context.program.basisDigest+ || plan.basisDigest !== semantic.basisDigest+ || plan.semanticIndexDigest !== script.semanticIndex.digest+ ) fail("caption-track inputs do not share one Script and ProgramBasis");+ if (!Array.isArray(plan.cues) || !Array.isArray(plan.annotations)) {+ fail("CaptionPlan requires cues and annotations arrays");}- if (semantic.basisDigest !== context.program.basisDigest) {- throw new Error("caption-track SemanticMap does not match the selected ProgramBasis");+ const { planDigest, ...canonicalPlan } = plan;+ if (context.digest(canonicalPlan) !== planDigest) fail("CaptionPlan digest does not match its contents");+ for (const [index, annotation] of plan.annotations.entries()) {+ if (+ typeof annotation.id !== "string"+ || typeof annotation.kind !== "string"+ || !Number.isInteger(annotation.startToken)+ || !Number.isInteger(annotation.endTokenExclusive)+ || annotation.startToken < 0+ || annotation.endTokenExclusive <= annotation.startToken+ || annotation.endTokenExclusive > script.tokens.length+ ) fail(`CaptionPlan annotation ${index + 1} has an invalid Script token range`);}- const points = new Map(semantic.anchors.map((anchor) => [anchor.identity, anchor.point.frame]));+ const points = new Map(semantic.anchors.map(anchor => [anchor.identity, anchor.point.frame]));const frameRange = (startFrame, endFrameExclusive) => ({startFrame,endFrameExclusive,startSec: startFrame / context.fps,endSec: endFrameExclusive / context.fps,});- const locatedWords = script.tokens.map((token) => ({+ const words = script.tokens.map(token => ({...token,...frameRange(points.get(token.startAnchorId), points.get(token.endAnchorId)),}));- const captionAtoms = script.captionAtoms.map((atom) => ({+ const captionAtoms = script.captionAtoms.map(atom => ({...atom,...frameRange(- locatedWords[atom.startWord].startFrame,- locatedWords[atom.endWordExclusive - 1].endFrameExclusive,+ words[atom.startToken].startFrame,+ words[atom.endTokenExclusive - 1].endFrameExclusive,),}));- const captionCues = semantic.captionCues?.map((cue, index) => ({- id: cue.id ?? `cue-${index + 1}`,- ...cue,- ...frameRange(- locatedWords[cue.startWord].startFrame,- locatedWords[cue.endWordExclusive - 1].endFrameExclusive,- ),- }));- const located = {- words: locatedWords,- captionAtoms,- captionCues,- durationFrames: context.program.durationFrames,+ const roleRanges = role => script.turns+ .filter(turn => turn.role === role)+ .map(turn => frameRange(+ words[turn.tokenStart].startFrame,+ words[turn.tokenEndExclusive - 1].endFrameExclusive,+ ));+ const annotationRanges = kind => plan.annotations+ .filter(annotation => annotation.kind === kind)+ .map(annotation => frameRange(+ words[annotation.startToken].startFrame,+ words[annotation.endTokenExclusive - 1].endFrameExclusive,+ ));+ const selectorRanges = (element, label, globalAllowed) => {+ const selectors = ["during", "role", "annotation"]+ .filter(name => element.attributes[name] !== undefined);+ if (selectors.length > 1 || (!globalAllowed && selectors.length !== 1)) {+ fail(`<${label} id="${stringAttr(element, "id")}"> requires ${globalAllowed ? "at most" : "exactly"} one selector`);+ }+ if (!selectors.length) return [frameRange(0, context.program.durationFrames)];+ if (selectors[0] === "during") {+ return context.selection(element.attributes.during, `${label}.during`);+ }+ if (selectors[0] === "role") return roleRanges(stringAttr(element, "role"));+ return annotationRanges(stringAttr(element, "annotation"));};- const z = numberAttr(context.element, "z");- const maxWords = Math.max(1, Math.round(numberAttr(context.element, "maxWords", 3)));- const words = located.words;- const styledRanges = elements(context, "style").map((style) => ({+ const styledRanges = elements(context, "style").map(style => ({id: stringAttr(style, "id"),- ranges: context.selection(style.attributes.during, "style.during"),+ ranges: selectorRanges(style, "style", true),color: style.attributes.color,activeColor: style.attributes.activeColor,background: style.attributes.background,weight: style.attributes.weight,scale: style.attributes.scale,}));+ const mutedRanges = elements(context, "mute")+ .flatMap(mute => selectorRanges(mute, "mute", false));+ const overlaps = (unit, range) =>+ unit.startFrame < range.endFrameExclusive+ && unit.endFrameExclusive > range.startFrame;const inlineStyle = (word, active) => {const declarations = {};for (const rule of styledRanges) {- if (!rule.ranges.some((range) =>- word.startFrame < range.endFrameExclusive- && word.endFrameExclusive > range.startFrame)) continue;+ if (!rule.ranges.some(range => overlaps(word, range))) continue;if (typeof rule.color === "string") declarations.color = rule.color;if (active && typeof rule.activeColor === "string") declarations.color = rule.activeColor;if (typeof rule.background === "string") declarations.background = rule.background;@@ -71,45 +106,50 @@ export default {.map(([name, value]) => `${name}:${String(value)}`).join(";");return body ? ` style="${html(body)}"` : "";};- const atomsByStart = new Map(located.captionAtoms.map((atom) => [atom.startWord, atom]));- const displayUnits = (startWord, endWordExclusive) => {+ const atomsByStart = new Map(captionAtoms.map(atom => [atom.startToken, atom]));+ const displayUnits = (startToken, endTokenExclusive) => {const units = [];- let index = startWord;- while (index < endWordExclusive) {+ let index = startToken;+ while (index < endTokenExclusive) {const atom = atomsByStart.get(index);if (atom) {- if (atom.endWordExclusive > endWordExclusive) {- throw new Error(`caption cue cuts through Dual Text atom "${atom.id}"`);+ if (atom.endTokenExclusive > endTokenExclusive) {+ fail(`caption cue cuts through Dual Text atom "${atom.id}"`);}- if (atom.display) {+ if (atom.display && !mutedRanges.some(range => overlaps(atom, range))) {units.push({text: atom.display,startFrame: atom.startFrame,endFrameExclusive: atom.endFrameExclusive,});}- index = atom.endWordExclusive;+ index = atom.endTokenExclusive;continue;}const word = words[index];- if (word) units.push(word);+ if (word && !mutedRanges.some(range => overlaps(word, range))) units.push(word);index += 1;}return units;};- const groups = located.captionCues?.map((cue) => ({- cue,- words: displayUnits(cue.startWord, cue.endWordExclusive),- })) ?? [];- if (groups.length === 0) {- const units = displayUnits(0, words.length);- for (let index = 0; index < units.length; index += maxWords) {- groups.push({- words: units.slice(index, index + maxWords),- });- }- }+ let expectedStart = 0;+ const groups = plan.cues.map((cue, index) => {+ if (+ !Number.isInteger(cue.startToken)+ || !Number.isInteger(cue.endTokenExclusive)+ || cue.startToken !== expectedStart+ || cue.endTokenExclusive <= cue.startToken+ || cue.endTokenExclusive > words.length+ ) fail(`CaptionPlan cue ${index + 1} is not a complete ordered token partition`);+ const first = words[cue.startToken];+ const last = words[cue.endTokenExclusive - 1];+ if (first.segmentId !== last.segmentId) fail(`CaptionPlan cue ${index + 1} crosses Segments`);+ expectedStart = cue.endTokenExclusive;+ return { cue, words: displayUnits(cue.startToken, cue.endTokenExclusive) };+ }).filter(entry => entry.words.length);+ if (expectedStart !== words.length) fail("CaptionPlan does not cover every Script token");const visuals = [];+ const z = numberAttr(context.element, "z");for (const [groupIndex, entry] of groups.entries()) {const group = entry.words;const nextCueStart = groups[groupIndex + 1]?.words[0]?.startFrame;@@ -117,8 +157,8 @@ export default {const next = group[activeIndex + 1];const endFrameExclusive = next?.startFrame?? Math.min(- located.durationFrames,- nextCueStart ?? located.durationFrames,+ context.program.durationFrames,+ nextCueStart ?? context.program.durationFrames,word.endFrameExclusive + Math.round(numberAttr(context.element, "hold", 0.08) * context.fps),);if (endFrameExclusive <= word.startFrame) continue;@@ -126,14 +166,8 @@ export default {const active = index === activeIndex;return `<span class="${active ? "active" : ""}"${inlineStyle(item, active)}>${html(item.text)}</span>`;}).join("");- const fragmentDuration = Math.max(- 1 / context.fps,- (endFrameExclusive - word.startFrame) / context.fps,- );- const fadeSec = Math.min(- numberAttr(context.element, "cueFade", 0.12),- fragmentDuration * 0.45,- );+ const fragmentDuration = Math.max(1 / context.fps, (endFrameExclusive - word.startFrame) / context.fps);+ const fadeSec = Math.min(numberAttr(context.element, "cueFade", 0.12), fragmentDuration * 0.45);const fadePercent = Math.min(45, fadeSec / fragmentDuration * 100);const animationName = `svml-caption-${context.instance.id}-${groupIndex}-${activeIndex}`.replaceAll(/[^A-Za-z0-9_-]/gu, "-");@@ -167,11 +201,7 @@ export default {"text-align": "center","-webkit-text-stroke": `${context.width * numberAttr(context.element, "strokeWidth", 0.002)}px ${stringAttr(context.element, "strokeColor", "rgba(0,0,0,.82)")}`,"paint-order": "stroke fill",- "text-shadow": stringAttr(- context.element,- "shadow",- "0 2px 9px rgba(0,0,0,.5), 0 0 2px rgba(0,0,0,.68)",- ),+ "text-shadow": stringAttr(context.element, "shadow", "0 2px 9px rgba(0,0,0,.5), 0 0 2px rgba(0,0,0,.68)"),animation: `${animationName} ${fragmentDuration}s linear both`,},css: `@keyframes ${animationName} { ${opacityKeyframes} }`,@@ -179,13 +209,11 @@ export default {}}const highlight = stringAttr(context.element, "highlight", "#ffd34d");- const gap = Math.max(- 12,- Math.round(- numberAttr(context.element, "size", 50)- * (0.34 + Math.max(0, -numberAttr(context.element, "tracking", -1) / numberAttr(context.element, "size", 50))),- ),- );+ const fontSize = numberAttr(context.element, "size", 50);+ const gap = Math.max(12, Math.round(fontSize * (0.34 + Math.max(+ 0,+ -numberAttr(context.element, "tracking", -1) / fontSize,+ ))));return trackOutput(visuals, [], [`.svml-caption-line .active { color: ${highlight}; }`,`.svml-caption-line {diff --git a/stdlib/runtime/film.mjs b/stdlib/runtime/film.mjsindex 0aa411bb..5552a7cc 100644--- a/stdlib/runtime/film.mjs+++ b/stdlib/runtime/film.mjs@@ -11,8 +11,8 @@ export default {if (production?.contract !== "svml.temporal-basis-production.v1") {throw new Error("film requires TemporalBasisProduction");}- if (semantic?.contract !== "svml.exact-semantic-map.v1") {- throw new Error("film requires ExactSemanticMap");+ if (semantic?.contract !== "svml.complete-semantic-map.v1") {+ throw new Error("film requires CompleteSemanticMap");}if (semantic.basisDigest !== production.basis.basisDigest) {throw new Error("film basis and SemanticMap do not share a basisDigest");diff --git a/stdlib/runtime/speech-locator.mjs b/stdlib/runtime/speech-locator.mjsindex 547da910..427efe95 100644--- a/stdlib/runtime/speech-locator.mjs+++ b/stdlib/runtime/speech-locator.mjs@@ -9,6 +9,150 @@ function fail(message) {throw new Error(message);}+function lcsMatches(tokens, units) {+ const rows = Array.from({ length: tokens.length + 1 }, () =>+ Array(units.length + 1).fill(0));+ for (let left = tokens.length - 1; left >= 0; left -= 1) {+ for (let right = units.length - 1; right >= 0; right -= 1) {+ rows[left][right] = tokens[left].normalized === normalizeWord(units[right].text)+ ? rows[left + 1][right + 1] + 1+ : Math.max(rows[left + 1][right], rows[left][right + 1]);+ }+ }+ const matches = [];+ let left = 0;+ let right = 0;+ while (left < tokens.length && right < units.length) {+ if (+ tokens[left].normalized === normalizeWord(units[right].text)+ && rows[left][right] === rows[left + 1][right + 1] + 1+ ) {+ matches.push([left, right]);+ left += 1;+ right += 1;+ } else if (rows[left + 1][right] >= rows[left][right + 1]) {+ left += 1;+ } else {+ right += 1;+ }+ }+ return matches;+}++function distributeFrames(tokens, startFrame, endFrameExclusive, quality) {+ const weights = tokens.map(token => Math.max(1, [...token.normalized].length));+ const total = weights.reduce((sum, value) => sum + value, 0);+ let accumulated = 0;+ return tokens.map((_, index) => {+ const start = startFrame + Math.round((endFrameExclusive - startFrame) * accumulated / total);+ accumulated += weights[index];+ const end = index === tokens.length - 1+ ? endFrameExclusive+ : startFrame + Math.round((endFrameExclusive - startFrame) * accumulated / total);+ return { startFrame: start, endFrameExclusive: Math.max(start, end), quality };+ });+}++function alignSegment(tokens, units, segment, fps, evidenceQuality) {+ const output = Array(tokens.length).fill(undefined);+ const matches = lcsMatches(tokens, units);+ let tokenCursor = 0;+ let unitCursor = 0;+ for (let matchIndex = 0; matchIndex <= matches.length; matchIndex += 1) {+ const match = matches[matchIndex];+ const tokenEnd = match?.[0] ?? tokens.length;+ const unitEnd = match?.[1] ?? units.length;+ const tokenRun = tokens.slice(tokenCursor, tokenEnd);+ const unitRun = units.slice(unitCursor, unitEnd);+ if (tokenRun.length) {+ const previous = tokenCursor > 0 ? output[tokenCursor - 1] : undefined;+ const nextUnit = match ? units[unitEnd] : undefined;+ const startFrame = unitRun.length+ ? Math.round(unitRun[0].startSec * fps)+ : previous?.endFrameExclusive ?? Math.round(segment.startSec * fps);+ const endFrame = unitRun.length+ ? Math.round(unitRun.at(-1).endSec * fps)+ : nextUnit+ ? Math.round(nextUnit.startSec * fps)+ : Math.round(segment.endSec * fps);+ const quality = evidenceQuality === "estimated" || !unitRun.length+ ? "estimated"+ : "derived";+ for (const [offset, value] of distributeFrames(+ tokenRun,+ startFrame,+ Math.max(startFrame, endFrame),+ quality,+ ).entries()) output[tokenCursor + offset] = value;+ }+ if (match) {+ const [tokenIndex, unitIndex] = match;+ const unit = units[unitIndex];+ output[tokenIndex] = {+ startFrame: Math.round(unit.startSec * fps),+ endFrameExclusive: Math.round(unit.endSec * fps),+ quality: evidenceQuality === "measured" ? "measured" : "estimated",+ };+ tokenCursor = tokenIndex + 1;+ unitCursor = unitIndex + 1;+ }+ }+ if (output.some(value => !value)) fail(`could not align every Script token in Segment ${segment.id}`);+ return output;+}++function validate(narrative, basis, evidence) {+ if (evidence?.contract !== "svml.speech-timing-evidence.v1") {+ fail("speech-locator requires SpeechTimingEvidence");+ }+ if (!['measured', 'estimated'].includes(evidence.quality)) fail("invalid speech timing quality");+ const fps = basis.frameRate.numerator / basis.frameRate.denominator;+ if (Math.abs(evidence.fps - fps) > 1e-6) fail("speech timing fps does not match ProgramBasis");+ if (Math.round(evidence.durationSec * fps) !== basis.durationFrames) {+ fail("speech timing duration does not match ProgramBasis");+ }+ if (evidence.segments.length !== narrative.segments.length) {+ fail("speech timing Segment cardinality mismatch");+ }+ const segments = new Map(evidence.segments.map(segment => [segment.id, segment]));+ if (segments.size !== evidence.segments.length) fail("speech timing repeats a Segment id");+ for (const [index, scriptSegment] of narrative.segments.entries()) {+ const segment = segments.get(scriptSegment.id);+ if (!segment) fail(`speech timing is missing Segment ${scriptSegment.id}`);+ if (+ !Number.isFinite(segment.startSec)+ || !Number.isFinite(segment.endSec)+ || segment.startSec < 0+ || segment.endSec < segment.startSec+ || segment.endSec > evidence.durationSec + 1e-6+ ) fail(`invalid speech timing window for Segment ${scriptSegment.id}`);+ const previousScript = narrative.segments[index - 1];+ const previous = previousScript ? segments.get(previousScript.id) : undefined;+ if (previous && (segment.startSec < previous.startSec - 1e-6 || segment.endSec < previous.endSec - 1e-6)) {+ fail(`speech timing reorders Segment ${scriptSegment.id}`);+ }+ }+ const previousBySegment = new Map();+ for (const [index, unit] of evidence.units.entries()) {+ const segment = segments.get(unit.segmentId);+ if (+ !segment+ || typeof unit.text !== "string"+ || !Number.isFinite(unit.startSec)+ || !Number.isFinite(unit.endSec)+ || unit.startSec < segment.startSec - 1e-6+ || unit.endSec < unit.startSec+ || unit.endSec > segment.endSec + 1e-6+ ) fail(`speech timing unit ${index + 1} falls outside its Segment`);+ const previous = previousBySegment.get(unit.segmentId);+ if (previous && unit.startSec < previous.endSec - 1e-6) {+ fail(`speech timing units overlap inside Segment ${unit.segmentId}`);+ }+ previousBySegment.set(unit.segmentId, unit);+ }+ return { fps, segments };+}+export default {abiVersion: "1",project(context) {@@ -20,60 +164,35 @@ export default {if (production?.contract !== "svml.temporal-basis-production.v1") {fail("speech-locator requires TemporalBasisProduction");}- if (evidence?.contract !== "svml.speech-alignment.v1") fail("unsupported alignment evidence");const basis = production.basis;- const fps = basis.frameRate.numerator / basis.frameRate.denominator;- if (Math.round(evidence.durationSec * fps) !== basis.durationFrames) {- fail("alignment duration does not match ProgramBasis");- }- if (evidence.words.length !== narrative.tokens.length) fail("alignment token cardinality mismatch");- if (evidence.segments.length !== narrative.segments.length) fail("alignment Segment cardinality mismatch");- const segments = new Map(evidence.segments.map(segment => [segment.id, segment]));- if (segments.size !== evidence.segments.length) fail("alignment repeats a Segment id");- const words = new Map();- const previousBySegment = new Map();- for (const [index, token] of narrative.tokens.entries()) {- const word = evidence.words[index];- const segment = segments.get(token.segmentId);- if (!word || !segment || word.segmentId !== token.segmentId || normalizeWord(word.text) !== token.normalized) {- fail(`alignment token ${index + 1} does not match Script`);- }- if (word.startSec < segment.startSec || word.endSec < word.startSec || word.endSec > segment.endSec) {- fail(`alignment token ${index + 1} falls outside Segment`);- }- const previous = previousBySegment.get(word.segmentId);- if (previous && word.startSec < previous.endSec - 1e-6) fail(`alignment overlaps inside Segment ${word.segmentId}`);- previousBySegment.set(word.segmentId, word);- words.set(token.id, word);- }- for (const [index, scriptSegment] of narrative.segments.entries()) {+ const { fps, segments } = validate(narrative, basis, evidence);+ const tokenFrames = new Map();+ for (const scriptSegment of narrative.segments) {const segment = segments.get(scriptSegment.id);- if (!segment) fail(`alignment is missing Segment ${scriptSegment.id}`);- const previousScript = narrative.segments[index - 1];- const previous = previousScript ? segments.get(previousScript.id) : undefined;- if (previous && (segment.startSec < previous.startSec || segment.endSec < previous.endSec)) {- fail(`alignment reorders Segment ${scriptSegment.id}`);- }+ const tokens = narrative.tokens.slice(scriptSegment.tokenStart, scriptSegment.tokenEnd);+ const units = evidence.units.filter(unit => unit.segmentId === scriptSegment.id);+ const aligned = alignSegment(tokens, units, segment, fps, evidence.quality);+ tokens.forEach((token, index) => tokenFrames.set(token.id, aligned[index]));}const anchors = narrative.semanticIndex.anchors.map(anchor => {const segment = segments.get(anchor.segmentId);- const word = anchor.tokenId ? words.get(anchor.tokenId) : undefined;- const seconds = anchor.kind === "segment-start"- ? segment?.startSec+ const token = anchor.tokenId ? tokenFrames.get(anchor.tokenId) : undefined;+ const frame = anchor.kind === "segment-start"+ ? Math.round(segment.startSec * fps): anchor.kind === "segment-end"- ? segment?.endSec+ ? Math.round(segment.endSec * fps): anchor.kind === "token-start"- ? word?.startSec- : word?.endSec;- if (seconds === undefined) fail(`alignment is missing anchor ${anchor.id}`);+ ? token?.startFrame+ : token?.endFrameExclusive;+ if (frame === undefined) fail(`speech timing is missing anchor ${anchor.id}`);return {identity: anchor.id,- point: { basisDigest: basis.basisDigest, frame: Math.round(seconds * fps) },- quality: "measured",+ point: { basisDigest: basis.basisDigest, frame },+ quality: token?.quality ?? evidence.quality,};});const payload = {- contract: "svml.exact-semantic-map.v1",+ contract: "svml.complete-semantic-map.v1",semanticIndexDigest: narrative.semanticIndex.digest,basisDigest: basis.basisDigest,anchors,@@ -83,12 +202,7 @@ export default {quantizationPolicy: "nearest-frame",}),quantizationPolicy: "nearest-frame",- ...(evidence.captionCues ? { captionCues: evidence.captionCues } : {}),- };- return {- outputs: {- map: { ...payload, mapDigest: context.digest(payload) },- },};+ return { outputs: { map: { ...payload, mapDigest: context.digest(payload) } } };},};diff --git a/stdlib/speech-locator.svk b/stdlib/speech-locator.svkindex 34b36f0c..a7bd5758 100644--- a/stdlib/speech-locator.svk+++ b/stdlib/speech-locator.svk@@ -1,6 +1,6 @@<kernel name="speech-locator" abi="1" implementation="./runtime/speech-locator.mjs"><port name="script" direction="input" type="NarrativeIR" cardinality="one"/><port name="basis" direction="input" type="TemporalBasisProduction" cardinality="one"/>- <port name="evidence" direction="input" type="AlignmentEvidence" cardinality="one"/>- <port name="map" direction="output" type="ExactSemanticMap" cardinality="one"/>+ <port name="evidence" direction="input" type="SpeechTimingEvidence" cardinality="one"/>+ <port name="map" direction="output" type="CompleteSemanticMap" cardinality="one"/></kernel>diff --git a/test/artifacts.test.ts b/test/artifacts.test.tsindex 987a4725..b7a362b0 100644--- a/test/artifacts.test.ts+++ b/test/artifacts.test.ts@@ -103,7 +103,7 @@ test("capability instances require exact verified artifacts and never invoke a p);});-test("a Locator capability returns ExactSemanticMap through the same typed Artifact ABI", async () => {+test("a Locator capability returns CompleteSemanticMap through the same typed Artifact ABI", async () => {const source = fileURLToPath(new URL("../examples/composite-speech-program/minimal.svml", import.meta.url),);@@ -134,13 +134,13 @@ test("a Locator capability returns ExactSemanticMap through the same typed Artif</script><video id="host" src="${avatar}"/><audio id="host-audio" src="${audio}"/>- <alignment id="alignment" src="${evidence}"/>+ <timing id="timing" src="${evidence}"/><speech-assemble id="voice" fps="30" defaultJoin="cut"><segment id="intro" script={script.segment.intro} visual={host} audio={host-audio} sourceWindow="0s .. 5s"/><segment id="close" script={script.segment.close} visual={host} audio={host-audio} sourceWindow="5s .. 10s"/><join after="intro" overlap="500ms" audio="crossfade" visual="dissolve"/></speech-assemble>- <pinned-locator id="location" script={script} basis={voice.production} evidence={alignment}/>+ <pinned-locator id="location" script={script} basis={voice.production} evidence={timing}/><film id="film" basis={voice.production} semantic={location.map}/></svml>\n`, "utf8");const checked = await checkSource(svml);@@ -153,7 +153,7 @@ test("a Locator capability returns ExactSemanticMap through the same typed ArtifexecutionDigest: instance?.executionDigest,outputs: {map: {- type: "ExactSemanticMap",+ type: "CompleteSemanticMap",value: reference.semanticMap,contentDigest: sha256(stableJson(reference.semanticMap)),},diff --git a/test/composite.test.ts b/test/composite.test.tsindex a3ea210a..3c4c7613 100644--- a/test/composite.test.ts+++ b/test/composite.test.ts@@ -29,6 +29,8 @@ test("composite-v1 expands finite internal Basis/Locator instances with auditedassert.match(internal?.sourceMap?.definitionSite.file ?? "", /speech-program\.svk$/u);const compilation = await compileSource({ file: fixture });+ assert.equal(compilation.plan.kernels.some((kernel) =>+ kernel.ports.some((port) => port.type === "CaptionTrack")), false);assert.equal(compilation.located.durationFrames, 285);assert.equal(compilation.basis.sourceMaps.length, 4);assert.equal(compilation.basis.audio[0]?.fadeOutFrames, 15);diff --git a/test/fixtures/artifacts/alignment.json b/test/fixtures/artifacts/alignment.jsonindex 2b301f1d..eaf3c0ce 100644--- a/test/fixtures/artifacts/alignment.json+++ b/test/fixtures/artifacts/alignment.json@@ -1,8 +1,9 @@{- "contract": "svml.speech-alignment.v1",+ "contract": "svml.speech-timing-evidence.v1","durationSec": 1,"fps": 30,- "words": [+ "quality": "measured",+ "units": [{"text": "Hello","startSec": 0,diff --git a/test/fixtures/artifacts/artifact-film.svk b/test/fixtures/artifacts/artifact-film.svkindex b1611e71..8908a8f4 100644--- a/test/fixtures/artifacts/artifact-film.svk+++ b/test/fixtures/artifacts/artifact-film.svk@@ -6,6 +6,6 @@><port name="image" direction="input" type="Image" cardinality="one"/><port name="basis" direction="input" type="TemporalBasisProduction" cardinality="one"/>- <port name="semantic" direction="input" type="ExactSemanticMap" cardinality="one"/>+ <port name="semantic" direction="input" type="CompleteSemanticMap" cardinality="one"/><port name="document" direction="output" type="HyperframesDocument" cardinality="one"/></kernel>diff --git a/test/fixtures/artifacts/main.svml b/test/fixtures/artifacts/main.svmlindex e8a49887..219bb0c2 100644--- a/test/fixtures/artifacts/main.svml+++ b/test/fixtures/artifacts/main.svml@@ -9,11 +9,11 @@</script><audio id="voice-audio" src="./alignment.json"/>- <alignment id="alignment" src="./alignment.json"/>+ <timing id="timing" src="./alignment.json"/><speech-assemble id="speech" fps="30"><segment id="main" script={script.segment.main} audio={voice-audio} sourceWindow="0s .. 1s"/></speech-assemble>- <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/>+ <speech-locator id="location" script={script} basis={speech.production} evidence={timing}/><pinned-image id="poster" prompt="A pinned result; never call a provider."/><artifact-film id="film" image={poster.image} basis={speech.production} semantic={location.map}/>diff --git a/test/fixtures/artifacts/pinned-locator.svk b/test/fixtures/artifacts/pinned-locator.svkindex 6296311c..855cdabb 100644--- a/test/fixtures/artifacts/pinned-locator.svk+++ b/test/fixtures/artifacts/pinned-locator.svk@@ -1,6 +1,6 @@<kernel name="pinned-locator" abi="1" profile="capability-v1" capability="semantic-locate"><port name="script" direction="input" type="NarrativeIR" cardinality="one"/><port name="basis" direction="input" type="TemporalBasisProduction" cardinality="one"/>- <port name="evidence" direction="input" type="AlignmentEvidence" cardinality="one"/>- <port name="map" direction="output" type="ExactSemanticMap" cardinality="one"/>+ <port name="evidence" direction="input" type="SpeechTimingEvidence" cardinality="one"/>+ <port name="map" direction="output" type="CompleteSemanticMap" cardinality="one"/></kernel>diff --git a/test/fixtures/collision/alignment.json b/test/fixtures/collision/alignment.jsonindex 413c7b61..f1dd6006 100644--- a/test/fixtures/collision/alignment.json+++ b/test/fixtures/collision/alignment.json@@ -1,8 +1,9 @@{- "contract": "svml.speech-alignment.v1",+ "contract": "svml.speech-timing-evidence.v1","durationSec": 2,"fps": 30,- "words": [+ "quality": "measured",+ "units": [{ "text": "Left", "startSec": 0.1, "endSec": 0.5, "segmentId": "main" },{ "text": "and", "startSec": 0.6, "endSec": 0.9, "segmentId": "main" },{ "text": "right", "startSec": 1, "endSec": 1.5, "segmentId": "main" }diff --git a/test/fixtures/collision/main.svml b/test/fixtures/collision/main.svmlindex 0925685b..b915620f 100644--- a/test/fixtures/collision/main.svml+++ b/test/fixtures/collision/main.svml@@ -12,7 +12,7 @@<video id="avatar" src="./avatar.mp4"/><audio id="voice-audio" src="./avatar.mp4"/>- <alignment id="alignment" src="./alignment.json"/>+ <timing id="timing" src="./alignment.json"/><image id="left-image" src="./left/image.png"/><image id="right-image" src="./right/image.png"/>@@ -20,7 +20,7 @@<segment id="main" script={script.segment.main} visual={avatar} audio={voice-audio} sourceWindow="0s .. 2s"/></speech-assemble>- <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/>+ <speech-locator id="location" script={script} basis={speech.production} evidence={timing}/><broll-track id="broll" z="100"><item id="left" source={left-image} during={script.selection.left}/>diff --git a/test/fixtures/disconnected/alignment.json b/test/fixtures/disconnected/alignment.jsonindex a5401de8..5d28c064 100644--- a/test/fixtures/disconnected/alignment.json+++ b/test/fixtures/disconnected/alignment.json@@ -1,8 +1,9 @@{- "contract": "svml.speech-alignment.v1",+ "contract": "svml.speech-timing-evidence.v1","durationSec": 3,"fps": 30,- "words": [+ "quality": "measured",+ "units": [{ "text": "Alpha", "startSec": 0, "endSec": 0.7, "segmentId": "main" },{ "text": "gap", "startSec": 1, "endSec": 1.5, "segmentId": "main" },{ "text": "omega", "startSec": 2, "endSec": 2.7, "segmentId": "main" }diff --git a/test/fixtures/disconnected/captions.svml b/test/fixtures/disconnected/captions.svmlindex 9081f409..eeb0c7d5 100644--- a/test/fixtures/disconnected/captions.svml+++ b/test/fixtures/disconnected/captions.svml@@ -1,22 +1,31 @@<svml version="1"><import from="../../../stdlib/speech-assemble.svk"/><import from="../../../stdlib/speech-locator.svk"/>+ <import from="../../../stdlib/caption-planner.svk"/><import from="../../../stdlib/caption-track.svk"/><import from="../../../stdlib/film.svk"/><script>- <segment id="main">@pick Alpha @/pick gap @pick omega @/pick.</segment>+ <segment id="main">+ <A> @pick Alpha @/pick @silent gap @/silent+ <B> @pick omega @/pick.+ </segment></script><audio id="voice-audio" src="../presentation/long.mp4"/>- <alignment id="alignment" src="./alignment.json"/>+ <timing id="timing" src="./alignment.json"/><speech-assemble id="speech" fps="30"><segment id="main" script={script.segment.main} audio={voice-audio} sourceWindow="0s .. 3s"/></speech-assemble>- <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/>+ <speech-locator id="location" script={script} basis={speech.production} evidence={timing}/>- <caption-track id="captions" script={script} semantic={location.map} z="10" maxWords="3">- <style id="picked" during={script.selection.pick} color="#ff0000" activeColor="#00ff00" scale="1.1"/>+ <caption-planner id="caption-plan" script={script} semantic={location.map} maxWords="3">+ <annotate id="picked" kind="accent" during={script.selection.pick}/>+ </caption-planner>+ <caption-track id="captions" script={script} semantic={location.map} plan={caption-plan.plan} z="10">+ <style id="speaker-a" role="A" background="#0000ff"/>+ <style id="picked" annotation="accent" color="#ff0000" activeColor="#00ff00" scale="1.1"/>+ <mute id="silent" during={script.selection.silent}/></caption-track><film id="film" basis={speech.production} semantic={location.map}>diff --git a/test/fixtures/disconnected/duplicate-captions.svml b/test/fixtures/disconnected/duplicate-captions.svmlnew file mode 100644index 00000000..39286f14--- /dev/null+++ b/test/fixtures/disconnected/duplicate-captions.svml@@ -0,0 +1,22 @@+<svml version="1">+ <import from="../../../stdlib/speech-assemble.svk"/>+ <import from="../../../stdlib/speech-locator.svk"/>+ <import from="../../../stdlib/caption-planner.svk"/>+ <import from="../../../stdlib/caption-track.svk"/>+ <import from="../../../stdlib/film.svk"/>++ <script><segment id="main">Alpha omega.</segment></script>+ <audio id="voice-audio" src="../presentation/long.mp4"/>+ <timing id="timing" src="./alignment.json"/>+ <speech-assemble id="speech" fps="30">+ <segment id="main" script={script.segment.main} audio={voice-audio} sourceWindow="0s .. 3s"/>+ </speech-assemble>+ <speech-locator id="location" script={script} basis={speech.production} evidence={timing}/>+ <caption-planner id="caption-plan" script={script} semantic={location.map}/>+ <caption-track id="captions-a" script={script} semantic={location.map} plan={caption-plan.plan} z="10"/>+ <caption-track id="captions-b" script={script} semantic={location.map} plan={caption-plan.plan} z="11"/>+ <film id="film" basis={speech.production} semantic={location.map}>+ <track ref={captions-a.track}/>+ <track ref={captions-b.track}/>+ </film>+</svml>diff --git a/test/fixtures/disconnected/each.svml b/test/fixtures/disconnected/each.svmlindex 5cc168df..d2f60278 100644--- a/test/fixtures/disconnected/each.svml+++ b/test/fixtures/disconnected/each.svml@@ -10,12 +10,12 @@<video id="avatar" src="../presentation/long.mp4"/><audio id="voice-audio" src="../presentation/long.mp4"/>- <alignment id="alignment" src="./alignment.json"/>+ <timing id="timing" src="./alignment.json"/><speech-assemble id="speech" fps="30"><segment id="main" script={script.segment.main} visual={avatar} audio={voice-audio} sourceWindow="0s .. 3s"/></speech-assemble>- <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/>+ <speech-locator id="location" script={script} basis={speech.production} evidence={timing}/><media-track id="media" z="1"><item id="picked" source={avatar} during={script.selection.pick}/>diff --git a/test/fixtures/disconnected/one.svml b/test/fixtures/disconnected/one.svmlindex f7e737f4..86ceafbf 100644--- a/test/fixtures/disconnected/one.svml+++ b/test/fixtures/disconnected/one.svml@@ -11,12 +11,12 @@<video id="avatar" src="../presentation/long.mp4"/><audio id="voice-audio" src="../presentation/long.mp4"/><image id="icon" src="../collision/left/image.png"/>- <alignment id="alignment" src="./alignment.json"/>+ <timing id="timing" src="./alignment.json"/><speech-assemble id="speech" fps="30"><segment id="main" script={script.segment.main} visual={avatar} audio={voice-audio} sourceWindow="0s .. 3s"/></speech-assemble>- <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/>+ <speech-locator id="location" script={script} basis={speech.production} evidence={timing}/><ranking-column id="ranking" z="1"><item id="picked" rank="1" image={icon} during={script.selection.pick}/>diff --git a/test/fixtures/disconnected/set.svml b/test/fixtures/disconnected/set.svmlindex 5f78cd9a..ff951c22 100644--- a/test/fixtures/disconnected/set.svml+++ b/test/fixtures/disconnected/set.svml@@ -10,12 +10,12 @@<video id="avatar" src="../presentation/long.mp4"/><audio id="voice-audio" src="../presentation/long.mp4"/>- <alignment id="alignment" src="./alignment.json"/>+ <timing id="timing" src="./alignment.json"/><speech-assemble id="speech" fps="30"><segment id="main" script={script.segment.main} visual={avatar} audio={voice-audio} sourceWindow="0s .. 3s"/></speech-assemble>- <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/>+ <speech-locator id="location" script={script} basis={speech.production} evidence={timing}/><selection-set id="styles"><item id="highlight" during={script.selection.pick}/>diff --git a/test/fixtures/media-basis/main.svml b/test/fixtures/media-basis/main.svmlindex 0bc1f7bc..7f0b36d0 100644--- a/test/fixtures/media-basis/main.svml+++ b/test/fixtures/media-basis/main.svml@@ -6,9 +6,9 @@<script><segment id="main">Hello.</segment></script><video id="visual" src="../collision/avatar.mp4"/><audio id="audio" src="../collision/avatar.mp4"/>- <alignment id="alignment" src="../artifacts/alignment.json"/>+ <timing id="timing" src="../artifacts/alignment.json"/><media-basis id="program" visual={visual} audio={audio} duration="1s"/>- <speech-locator id="location" script={script} basis={program.production} evidence={alignment}/>+ <speech-locator id="location" script={script} basis={program.production} evidence={timing}/><film id="film" basis={program.production} semantic={location.map}/></svml>diff --git a/test/fixtures/modules/alignment.json b/test/fixtures/modules/alignment.jsonindex 5d086519..02f6b18e 100644--- a/test/fixtures/modules/alignment.json+++ b/test/fixtures/modules/alignment.json@@ -1,8 +1,9 @@{- "contract": "svml.speech-alignment.v1",+ "contract": "svml.speech-timing-evidence.v1","durationSec": 1,"fps": 30,- "words": [+ "quality": "measured",+ "units": [{"text": "Hello","startSec": 0.1,diff --git a/test/fixtures/modules/main-renamed-alias.svml b/test/fixtures/modules/main-renamed-alias.svmlindex 743ba685..25f8b8d8 100644--- a/test/fixtures/modules/main-renamed-alias.svml+++ b/test/fixtures/modules/main-renamed-alias.svml@@ -2,6 +2,7 @@<import from="../../../stdlib/speech-assemble.svk"/><import from="../../../stdlib/speech-locator.svk"/><import from="../../../stdlib/broll-track.svk"/>+ <import from="../../../stdlib/caption-planner.svk"/><import as="subtitles" from="../../../stdlib/caption-track.svk"/><import from="../../../stdlib/film.svk"/><import as="assets" from="./brand.svc"/>@@ -15,19 +16,20 @@<video id="avatar" src="./avatar.mp4"/><audio id="voice-audio" src="./avatar.mp4"/>- <alignment id="alignment" src="./alignment.json"/>+ <timing id="timing" src="./alignment.json"/><speech-assemble id="speech" fps="30"><segment id="main" script={script.segment.main} visual={avatar} audio={voice-audio} sourceWindow="0s .. 1s"/></speech-assemble>- <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/>+ <speech-locator id="location" script={script} basis={speech.production} evidence={timing}/><broll-track id="broll" class="house.full"><item id="poster" source={assets.poster.image} during={script.selection.poster}/></broll-track>- <subtitles id="captions" script={script} semantic={location.map} class="house.launch" size="48"/>+ <caption-planner id="caption-plan" script={script} semantic={location.map}/>+ <subtitles id="captions" script={script} semantic={location.map} plan={caption-plan.plan} class="house.launch" size="48"/><film id="main-film" class="house.vertical" basis={speech.production} semantic={location.map}><track ref={broll.track}/>diff --git a/test/fixtures/modules/main.svml b/test/fixtures/modules/main.svmlindex 178a9e20..68e0afc4 100644--- a/test/fixtures/modules/main.svml+++ b/test/fixtures/modules/main.svml@@ -2,6 +2,7 @@<import from="../../../stdlib/speech-assemble.svk"/><import from="../../../stdlib/speech-locator.svk"/><import from="../../../stdlib/broll-track.svk"/>+ <import from="../../../stdlib/caption-planner.svk"/><import from="../../../stdlib/caption-track.svk"/><import from="../../../stdlib/film.svk"/><import as="brand" from="./brand.svc"/>@@ -15,19 +16,20 @@<video id="avatar" src="./avatar.mp4"/><audio id="voice-audio" src="./avatar.mp4"/>- <alignment id="alignment" src="./alignment.json"/>+ <timing id="timing" src="./alignment.json"/><speech-assemble id="speech" fps="30"><segment id="main" script={script.segment.main} visual={avatar} audio={voice-audio} sourceWindow="0s .. 1s"/></speech-assemble>- <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/>+ <speech-locator id="location" script={script} basis={speech.production} evidence={timing}/><broll-track id="broll" class="house.full"><item id="poster" source={brand.poster.image} during={script.selection.poster}/></broll-track>- <caption-track id="captions" script={script} semantic={location.map} class="house.launch" size="48"/>+ <caption-planner id="caption-plan" script={script} semantic={location.map}/>+ <caption-track id="captions" script={script} semantic={location.map} plan={caption-plan.plan} class="house.launch" size="48"/><film id="main-film" class="house.vertical" basis={speech.production} semantic={location.map}><track ref={broll.track}/>diff --git a/test/fixtures/presentation/alignment.json b/test/fixtures/presentation/alignment.jsonindex ec4e5903..581dec35 100644--- a/test/fixtures/presentation/alignment.json+++ b/test/fixtures/presentation/alignment.json@@ -1,8 +1,9 @@{- "contract": "svml.speech-alignment.v1",+ "contract": "svml.speech-timing-evidence.v1","durationSec": 3,"fps": 30,- "words": [+ "quality": "measured",+ "units": [{ "text": "One", "startSec": 0.2, "endSec": 0.6, "segmentId": "main" },{ "text": "two", "startSec": 1.1, "endSec": 1.5, "segmentId": "main" },{ "text": "three", "startSec": 2.1, "endSec": 2.7, "segmentId": "main" }diff --git a/test/fixtures/presentation/empty-present.svml b/test/fixtures/presentation/empty-present.svmlindex 9b46e3e2..7644c3e8 100644--- a/test/fixtures/presentation/empty-present.svml+++ b/test/fixtures/presentation/empty-present.svml@@ -12,12 +12,12 @@<video id="long-video" src="./long.mp4"/><audio id="voice-audio" src="./long.mp4"/>- <alignment id="alignment" src="./alignment.json"/>+ <timing id="timing" src="./alignment.json"/><speech-assemble id="speech" fps="30"><segment id="main" script={script.segment.main} visual={long-video} audio={voice-audio} sourceWindow="0s .. 3s"/></speech-assemble>- <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/>+ <speech-locator id="location" script={script} basis={speech.production} evidence={timing}/><media-track id="aroll" z="200"><item id="host" source={long-video} during="full">diff --git a/test/fixtures/presentation/main.svml b/test/fixtures/presentation/main.svmlindex c41502d1..e74783c6 100644--- a/test/fixtures/presentation/main.svml+++ b/test/fixtures/presentation/main.svml@@ -14,13 +14,13 @@<video id="long-video" src="./long.mp4"/><audio id="voice-audio" src="./long.mp4"/><audio id="pop-sound" src="./pop.wav"/>- <alignment id="alignment" src="./alignment.json"/>+ <timing id="timing" src="./alignment.json"/><speech-assemble id="speech" fps="30"><segment id="main" script={script.segment.main} visual={long-video} audio={voice-audio} sourceWindow="0s .. 3s"/></speech-assemble>- <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/>+ <speech-locator id="location" script={script} basis={speech.production} evidence={timing}/><media-track id="aroll" z="200"><item id="host" source={long-video} during="full" fit="cover">diff --git a/test/fixtures/unsafe/alignment.json b/test/fixtures/unsafe/alignment.jsonindex b6b288ff..568a3efb 100644--- a/test/fixtures/unsafe/alignment.json+++ b/test/fixtures/unsafe/alignment.json@@ -1,8 +1,9 @@{- "contract": "svml.speech-alignment.v1",+ "contract": "svml.speech-timing-evidence.v1","durationSec": 1,"fps": 30,- "words": [+ "quality": "measured",+ "units": [{ "text": "Hello", "startSec": 0.1, "endSec": 0.8, "segmentId": "main" }],"segments": [diff --git a/test/fixtures/unsafe/main.svml b/test/fixtures/unsafe/main.svmlindex fb265c2f..b7f7b962 100644--- a/test/fixtures/unsafe/main.svml+++ b/test/fixtures/unsafe/main.svml@@ -8,11 +8,11 @@</script><audio id="voice-audio" src="./alignment.json"/>- <alignment id="alignment" src="./alignment.json"/>+ <timing id="timing" src="./alignment.json"/><speech-assemble id="speech" fps="30"><segment id="main" script={script.segment.main} audio={voice-audio} sourceWindow="0s .. 1s"/></speech-assemble>- <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/>+ <speech-locator id="location" script={script} basis={speech.production} evidence={timing}/><unsafe id="main" basis={speech.production} semantic={location.map}/></svml>diff --git a/test/fixtures/unsafe/unsafe.svk b/test/fixtures/unsafe/unsafe.svkindex dbc423bd..d7ca9692 100644--- a/test/fixtures/unsafe/unsafe.svk+++ b/test/fixtures/unsafe/unsafe.svk@@ -1,5 +1,5 @@<kernel name="unsafe" abi="1" root="true" implementation="./unsafe.mjs"><port name="basis" direction="input" type="TemporalBasisProduction" cardinality="one"/>- <port name="semantic" direction="input" type="ExactSemanticMap" cardinality="one"/>+ <port name="semantic" direction="input" type="CompleteSemanticMap" cardinality="one"/><port name="document" direction="output" type="HyperframesDocument" cardinality="one"/></kernel>diff --git a/test/modules.test.ts b/test/modules.test.tsindex 74df9f57..0d901df5 100644--- a/test/modules.test.ts+++ b/test/modules.test.ts@@ -16,11 +16,11 @@ test(".svc content DAG and .svs parameter classes bind into one source closure",assert.deepEqual(checked.plan.instances.map((instance) => instance.id).sort(),- ["brand.poster", "broll", "captions", "location", "main-film", "speech"],+ ["brand.poster", "broll", "caption-plan", "captions", "location", "main-film", "speech"],);assert.deepEqual(checked.plan.values.map((value) => value.id).sort(),- ["alignment", "avatar", "brand.logo", "voice-audio"],+ ["avatar", "brand.logo", "timing", "voice-audio"],);const logo = checked.plan.values.find((value) => value.id === "brand.logo");assert.equal(logo?.localId, "logo");diff --git a/test/presentation.test.ts b/test/presentation.test.tsindex 60c990b1..327d359c 100644--- a/test/presentation.test.ts+++ b/test/presentation.test.ts@@ -94,4 +94,18 @@ test("disconnected SelectionSet drives one caption style rule without losing occassert.ok(captions.some((fragment) => /color:#ff0000/u.test(fragment.html ?? "")));assert.ok(captions.some((fragment) => /color:#00ff00/u.test(fragment.html ?? "")));assert.ok(captions.some((fragment) => /scale\(1\.1\)/u.test(fragment.html ?? "")));+ assert.ok(captions.some((fragment) => /background:#0000ff/u.test(fragment.html ?? "")));+ assert.ok(captions.every((fragment) => !/gap/u.test(fragment.html ?? "")));+ const plan = compilation.projections.get("caption-plan")?.outputs.plan as {+ annotations?: Array<{ kind: string }>;+ };+ assert.deepEqual(plan.annotations?.map((annotation) => annotation.kind), ["accent", "accent"]);+});++test("a Composition rejects a second CaptionTrack but permits no CaptionTrack", async () => {+ const fixture = fileURLToPath(new URL("./fixtures/disconnected/", import.meta.url));+ await assert.rejects(+ compileSource({ file: `${fixture}/duplicate-captions.svml` }),+ /caption_track_cardinality/u,+ );});diff --git a/test/regen-ranking.test.ts b/test/regen-ranking.test.tsindex a6d64b0f..6bbd53d0 100644--- a/test/regen-ranking.test.ts+++ b/test/regen-ranking.test.ts@@ -52,6 +52,9 @@ test("regen ranking compiles to the frozen production timing contract", async ()assert.equal(document.audios?.length, reference.locatedOutput.audioFragments);assert.match(compilation.html, /data-composition-id="regen-ranking"/u);assert.match(compilation.html, /data-duration="36\.1"/u);+ assert.match(compilation.html, />Powerful</u);+ assert.doesNotMatch(compilation.html, /power full but|>uh</u);+ assert.ok(compilation.semanticMap.anchors.some((anchor) => anchor.quality === "derived"));const canvas = projectCanvas(compilation.plan);assert.equal(canvas.root, reference.plan.root);diff --git a/test/script.test.ts b/test/script.test.tsindex 6b3271c9..6fa6bb8c 100644--- a/test/script.test.ts+++ b/test/script.test.ts@@ -1,26 +1,26 @@import assert from "node:assert/strict";import test from "node:test";-import type { AlignmentEvidence, NarrativeIR } from "../src/model.js";+import type { NarrativeIR, SpeechTimingEvidence } from "../src/model.js";import { locateScript } from "../src/script/locate.js";-import { estimateAlignment } from "../src/script/estimate.js";+import { estimateSpeechTiming } from "../src/script/estimate.js";import { parseScript } from "../src/script/parse.js";import { formatDocumentScript } from "../src/format.js";import { parseDocumentSource } from "../src/source/parse-document.js";import {createProgramBasis,- semanticMapFromAlignment,+ semanticMapFromTiming,validateSemanticMap,} from "../src/temporal.js";import { sha256, stableJson } from "../src/util.js";-function locateAlignment(narrative: NarrativeIR, evidence: AlignmentEvidence) {+function locateTiming(narrative: NarrativeIR, evidence: SpeechTimingEvidence) {const basis = createProgramBasis({fps: evidence.fps,durationFrames: Math.round(evidence.durationSec * evidence.fps),outcomeDigest: sha256(stableJson(evidence)),});- const map = semanticMapFromAlignment(narrative, basis, evidence, { precision: "exact" });- assert.equal(map.contract, "svml.exact-semantic-map.v1");+ const map = semanticMapFromTiming(narrative, basis, evidence);+ assert.equal(map.contract, "svml.complete-semantic-map.v1");return locateScript(narrative, basis, map);}@@ -35,6 +35,9 @@ test("Script Surface projects role, dual text, selections and moments", () => {assert.equal(narrative.projections.dialogue, "A: I just laughed my ass out.");assert.equal(narrative.projections.speech, "I just laughed my ass out.");assert.equal(narrative.projections.caption, "I just lmao.");+ assert.deepEqual(narrative.turns.map((turn) => [turn.role, turn.tokenStart, turn.tokenEndExclusive]), [+ ["A", 0, 6],+ ]);assert.equal(narrative.selections.whole?.length, 1);assert.equal(narrative.selections.laugh?.length, 1);assert.equal(narrative.selections.punch?.length, 1);@@ -42,8 +45,8 @@ test("Script Surface projects role, dual text, selections and moments", () => {assert.deepEqual(narrative.captionAtoms.map((atom) => [atom.display,- atom.startWord,- atom.endWordExclusive,+ atom.startToken,+ atom.endTokenExclusive,]),[["lmao", 2, 6]],);@@ -88,11 +91,12 @@ test("overlapping Segments retain distinct boundary affinity on one ProgramBasis~@left! @right!<segment id="b">Beta.</segment>`);- const evidence: AlignmentEvidence = {- contract: "svml.speech-alignment.v1",+ const evidence: SpeechTimingEvidence = {+ contract: "svml.speech-timing-evidence.v1",durationSec: 8,fps: 30,- words: [+ quality: "measured",+ units: [{ text: "Alpha", startSec: 0.2, endSec: 4.8, segmentId: "a" },{ text: "Beta", startSec: 4.6, endSec: 7.8, segmentId: "b" },],@@ -101,7 +105,7 @@ test("overlapping Segments retain distinct boundary affinity on one ProgramBasis{ id: "b", startSec: 4.5, endSec: 8 },],};- const located = locateAlignment(narrative, evidence);+ const located = locateTiming(narrative, evidence);assert.deepEqual(located.moments.left?.frames, [150]);assert.deepEqual(located.moments.right?.frames, [135]);assert.equal(located.segments.a?.endFrameExclusive, 150);@@ -113,11 +117,12 @@ test("SemanticMap rejects source-order reversal, wrong basis and digest tamperin"fixture.svml",`<segment id="a">Alpha.</segment><segment id="b">Beta.</segment>`,);- const evidence: AlignmentEvidence = {- contract: "svml.speech-alignment.v1",+ const evidence: SpeechTimingEvidence = {+ contract: "svml.speech-timing-evidence.v1",durationSec: 5,fps: 30,- words: [+ quality: "measured",+ units: [{ text: "Alpha", startSec: 1.1, endSec: 1.8, segmentId: "a" },{ text: "Beta", startSec: 0.2, endSec: 0.8, segmentId: "b" },],@@ -132,13 +137,13 @@ test("SemanticMap rejects source-order reversal, wrong basis and digest tamperinoutcomeDigest: sha256("reordered"),});assert.throws(- () => semanticMapFromAlignment(narrative, basis, evidence, { precision: "exact" }),+ () => semanticMapFromTiming(narrative, basis, evidence),/locator_segment_order/u,);- const validEvidence: AlignmentEvidence = {+ const validEvidence: SpeechTimingEvidence = {...evidence,- words: [+ units: [{ text: "Alpha", startSec: 0.1, endSec: 0.8, segmentId: "a" },{ text: "Beta", startSec: 1.1, endSec: 1.8, segmentId: "b" },],@@ -147,11 +152,10 @@ test("SemanticMap rejects source-order reversal, wrong basis and digest tamperin{ id: "b", startSec: 1, endSec: 2 },],};- const validMap = semanticMapFromAlignment(+ const validMap = semanticMapFromTiming(narrative,basis,validEvidence,- { precision: "exact" },);assert.throws(() => validateSemanticMap(narrative, basis, {@@ -173,16 +177,17 @@ test("SemanticMap rejects source-order reversal, wrong basis and digest tamperin);});-test("Dual Text owns its complete speech span and caption cues cannot split it", () => {+test("Dual Text owns its complete speech span without entering SemanticMap", () => {const narrative = parseScript("fixture.svml",`<segment id="one">I just <lmao | laughed my ass out>.</segment>`,);- const evidence: AlignmentEvidence = {- contract: "svml.speech-alignment.v1",+ const evidence: SpeechTimingEvidence = {+ contract: "svml.speech-timing-evidence.v1",durationSec: 2,fps: 30,- words: [+ quality: "measured",+ units: [{ text: "I", startSec: 0, endSec: 0.1, segmentId: "one" },{ text: "just", startSec: 0.2, endSec: 0.4, segmentId: "one" },{ text: "laughed", startSec: 0.5, endSec: 0.8, segmentId: "one" },@@ -192,7 +197,7 @@ test("Dual Text owns its complete speech span and caption cues cannot split it",],segments: [{ id: "one", startSec: 0, endSec: 2 }],};- const located = locateAlignment(narrative, evidence);+ const located = locateTiming(narrative, evidence);assert.deepEqual(located.captionAtoms.map((atom) => [atom.display,@@ -201,16 +206,6 @@ test("Dual Text owns its complete speech span and caption cues cannot split it",]),[["lmao", 15, 51]],);- assert.throws(- () => locateAlignment(narrative, {- ...evidence,- captionCues: [- { startWord: 0, endWordExclusive: 4 },- { startWord: 4, endWordExclusive: 6 },- ],- }),- /locate_caption_partial_dual/u,- );});test("Slots bind after parsing and escaped reserved syntax stays literal", () => {@@ -260,57 +255,52 @@ test("selection endpoints must close", () => {);});-test("caption cue evidence is a complete ordered word partition", () => {+test("direct Script-to-timing alignment handles N:1, 1:N, extras and missing units", () => {const narrative = parseScript("fixture.svml",- `<segment id="one">Alpha beta gamma.</segment>`,+ `<segment id="one">We really love hyper frames cannot now.</segment>`,);- const evidence: AlignmentEvidence = {- contract: "svml.speech-alignment.v1",- durationSec: 1.5,+ const evidence: SpeechTimingEvidence = {+ contract: "svml.speech-timing-evidence.v1",+ durationSec: 2.4,fps: 30,- words: [- { text: "Alpha", startSec: 0, endSec: 0.4, segmentId: "one" },- { text: "beta", startSec: 0.5, endSec: 0.9, segmentId: "one" },- { text: "gamma", startSec: 1, endSec: 1.4, segmentId: "one" },- ],- segments: [{ id: "one", startSec: 0, endSec: 1.5 }],- captionCues: [- { id: "opening", startWord: 0, endWordExclusive: 1 },- { id: "payoff", startWord: 1, endWordExclusive: 3 },+ quality: "measured",+ units: [+ { text: "We", startSec: 0.1, endSec: 0.2, segmentId: "one" },+ { text: "reallylovehyper", startSec: 0.4, endSec: 0.9, segmentId: "one" },+ { text: "frames", startSec: 1, endSec: 1.3, segmentId: "one" },+ { text: "can", startSec: 1.4, endSec: 1.55, segmentId: "one" },+ { text: "not", startSec: 1.56, endSec: 1.8, segmentId: "one" },+ { text: "um", startSec: 1.81, endSec: 1.85, segmentId: "one" },+ { text: "now", startSec: 1.9, endSec: 2.1, segmentId: "one" },],+ segments: [{ id: "one", startSec: 0, endSec: 2.4 }],};- const located = locateAlignment(narrative, evidence);+ const basis = createProgramBasis({+ fps: evidence.fps,+ durationFrames: Math.round(evidence.durationSec * evidence.fps),+ outcomeDigest: sha256(stableJson(evidence)),+ });+ const map = semanticMapFromTiming(narrative, basis, evidence);+ const located = locateScript(narrative, basis, map);+ assert.deepEqual(located.words.map((word) => word.text),+ ["We", "really", "love", "hyper", "frames", "cannot", "now"]);assert.deepEqual(- located.captionCues?.map((cue) => [- cue.id,- cue.startWord,- cue.endWordExclusive,- cue.startFrame,- cue.endFrameExclusive,- ]),- [- ["opening", 0, 1, 0, 12],- ["payoff", 1, 3, 15, 42],- ],+ map.anchors.filter((anchor) => anchor.identity.includes(":token:") && anchor.identity.endsWith(":start"))+ .map((anchor) => anchor.quality),+ ["measured", "derived", "derived", "derived", "measured", "derived", "measured"],);-- assert.throws(- () => locateAlignment(narrative, {- ...evidence,- captionCues: [{ startWord: 1, endWordExclusive: 3 }],- }),- /locate_caption_cue_partition/u,+ assert.deepEqual(+ [located.words[1]?.startFrame, located.words[3]?.endFrameExclusive],+ [12, 27],);- assert.throws(- () => locateAlignment(narrative, {- ...evidence,- words: evidence.words.map((word, index) =>- index === 1 ? { ...word, startSec: 0.2 } : word),- }),- /locator_word_overlap/u,- );+ const missing = locateTiming(narrative, {+ ...evidence,+ units: [evidence.units[0]!, evidence.units[2]!, evidence.units.at(-1)!],+ });+ assert.equal(missing.words[1]?.startFrame, 6);+ assert.equal(missing.words[3]?.endFrameExclusive, 30);});test("Script normalizes newlines and NFC while tokenizing East Asian writing per character", () => {@@ -340,30 +330,25 @@ test("reserved @ syntax and non-canonical ids fail closed", () => {);});-test("syllable Estimate produces complete deterministic alignment including silence", () => {+test("syllable Estimate produces deterministic SpeechTimingEvidence including silence", () => {const narrative = parseScript("fixture.svml", `<segment id="speech">Hello 中文.</segment><segment id="silence"/>`);- const first = estimateAlignment(narrative, {+ const first = estimateSpeechTiming(narrative, {fps: 30,syllablesPerSecond: 4,emptySegmentSec: 2,- maxCaptionWords: 2,});- const second = estimateAlignment(narrative, {+ const second = estimateSpeechTiming(narrative, {fps: 30,syllablesPerSecond: 4,emptySegmentSec: 2,- maxCaptionWords: 2,});assert.deepEqual(first, second);- assert.deepEqual(first.words.map((word) => word.text), ["Hello", "中", "文"]);+ assert.deepEqual(first.units.map((unit) => unit.text), ["Hello", "中", "文"]);assert.equal(first.segments[1]?.endSec! - first.segments[1]?.startSec!, 2);- assert.deepEqual(- first.captionCues?.map((cue) => [cue.startWord, cue.endWordExclusive]),- [[0, 2], [2, 3]],- );+ assert.equal(first.quality, "estimated");assert.equal(first.provenance?.method, "svml.syllable-estimate.v1");});diff --git a/tools/build-speech-timing.mjs b/tools/build-speech-timing.mjsnew file mode 100644index 00000000..836d21d2--- /dev/null+++ b/tools/build-speech-timing.mjs@@ -0,0 +1,88 @@+#!/usr/bin/env node+import { readFile, writeFile } from "node:fs/promises";+import { resolve } from "node:path";++const args = process.argv.slice(2);+function takeOption(name) {+ const index = args.indexOf(name);+ if (index < 0) return undefined;+ const value = args[index + 1];+ if (!value) throw new Error(`${name} requires a value.`);+ args.splice(index, 2);+ return value;+}++const timingOverrideFile = takeOption("--timing-overrides");+const fps = Number(takeOption("--fps") ?? 30);+const segmentFrames = Number(takeOption("--segment-frames") ?? 361);+const [scriptFile, outputFile, ...whisperFiles] = args;+if (!scriptFile || !outputFile || whisperFiles.length === 0) {+ throw new Error(+ "usage: build-speech-timing.mjs <script.json> <output.json> <whisper-1.json> ... "+ + "[--fps 30] [--segment-frames 361] [--timing-overrides <overrides.json>]",+ );+}+if (!Number.isFinite(fps) || fps <= 0 || !Number.isInteger(segmentFrames) || segmentFrames <= 0) {+ throw new Error("fps and segment-frames must define a positive clock.");+}++const script = JSON.parse(await readFile(resolve(scriptFile), "utf8"));+if (!Array.isArray(script.segments) || script.segments.length !== whisperFiles.length) {+ throw new Error("Script Segment count must match the supplied Whisper result count.");+}++const units = [];+for (const [segmentIndex, segment] of script.segments.entries()) {+ const whisper = JSON.parse(await readFile(resolve(whisperFiles[segmentIndex]), "utf8"));+ const offsetSec = segmentIndex * segmentFrames / fps;+ const segmentEndSec = (segmentIndex + 1) * segmentFrames / fps;+ for (const entry of whisper.transcription ?? []) {+ const text = String(entry.text ?? "").trim();+ if (!text) continue;+ const startSec = Number(entry.offsets?.from) / 1000 + offsetSec;+ const endSec = Number(entry.offsets?.to) / 1000 + offsetSec;+ units.push({+ text,+ startSec: Math.max(offsetSec, Math.min(segmentEndSec, startSec)),+ endSec: Math.max(offsetSec, Math.min(segmentEndSec, endSec)),+ segmentId: segment.id,+ });+ }+}++const evidence = {+ contract: "svml.speech-timing-evidence.v1",+ durationSec: script.segments.length * segmentFrames / fps,+ fps,+ quality: "measured",+ units,+ segments: script.segments.map((segment, index) => ({+ id: segment.id,+ startSec: index * segmentFrames / fps,+ endSec: (index + 1) * segmentFrames / fps,+ })),+ provenance: {+ method: "whisper.cpp raw word timestamps",+ source: "measured units are not corrected or projected to Script wording",+ },+};++if (timingOverrideFile) {+ const overrideDocument = JSON.parse(await readFile(resolve(timingOverrideFile), "utf8"));+ if (!Array.isArray(overrideDocument.units)) {+ throw new Error(`${timingOverrideFile} must contain a units array.`);+ }+ for (const override of overrideDocument.units) {+ const index = Number(override.index);+ const unit = evidence.units[index];+ if (!Number.isInteger(index) || !unit) {+ throw new Error(`${timingOverrideFile} contains invalid unit index ${String(override.index)}.`);+ }+ if (override.startSec !== undefined) unit.startSec = Number(override.startSec);+ if (override.endSec !== undefined) unit.endSec = Number(override.endSec);+ }+ evidence.provenance.timingOverrides = overrideDocument.provenance+ ?? "manual timing evidence correction";+}++await writeFile(resolve(outputFile), `${JSON.stringify(evidence, null, 2)}\n`);diff --git a/tools/build-whisper-evidence.mjs b/tools/build-whisper-evidence.mjsdeleted file mode 100644index 14f114fc..00000000--- a/tools/build-whisper-evidence.mjs+++ /dev/null@@ -1,168 +0,0 @@-#!/usr/bin/env node-import { readFile, writeFile } from "node:fs/promises";-import { resolve } from "node:path";--const args = process.argv.slice(2);-function takeOption(name) {- const index = args.indexOf(name);- if (index < 0) return undefined;- const value = args[index + 1];- if (!value) throw new Error(`${name} requires a file path.`);- args.splice(index, 2);- return value;-}-const cueFile = takeOption("--caption-cues");-const timingOverrideFile = takeOption("--timing-overrides");-const [sourceFile, outputFile, ...whisperFiles] = args;-if (!sourceFile || !outputFile || whisperFiles.length === 0) {- throw new Error(- "usage: build-whisper-evidence.mjs <tokens.json> <output.json> <whisper-1.json> ... "- + "[--caption-cues <cues.json>] [--timing-overrides <overrides.json>]",- );-}--const intended = JSON.parse(await readFile(resolve(sourceFile), "utf8"));-const normalize = (value) => String(value)- .normalize("NFKC")- .toLowerCase()- .replace(/[^\p{L}\p{M}\p{N}]+/gu, "");--const lcs = (left, right) => {- const table = Array.from({ length: left.length + 1 }, () =>- Array(right.length + 1).fill(0));- for (let leftIndex = left.length - 1; leftIndex >= 0; leftIndex -= 1) {- for (let rightIndex = right.length - 1; rightIndex >= 0; rightIndex -= 1) {- table[leftIndex][rightIndex] = left[leftIndex] === right[rightIndex]- ? 1 + table[leftIndex + 1][rightIndex + 1]- : Math.max(table[leftIndex + 1][rightIndex], table[leftIndex][rightIndex + 1]);- }- }- const matches = [];- let leftIndex = 0;- let rightIndex = 0;- while (leftIndex < left.length && rightIndex < right.length) {- if (left[leftIndex] === right[rightIndex]) {- matches.push([leftIndex, rightIndex]);- leftIndex += 1;- rightIndex += 1;- } else if (table[leftIndex + 1][rightIndex] >= table[leftIndex][rightIndex + 1]) {- leftIndex += 1;- } else {- rightIndex += 1;- }- }- return matches;-};--const evidenceWords = [];-const segmentFrames = 361;-for (let segmentIndex = 0; segmentIndex < intended.segments.length; segmentIndex += 1) {- const segment = intended.segments[segmentIndex];- const whisper = JSON.parse(await readFile(resolve(whisperFiles[segmentIndex]), "utf8"));- const measured = whisper.transcription- .filter((entry) => String(entry.text).trim())- .map((entry) => ({- text: String(entry.text).trim(),- startSec: Number(entry.offsets.from) / 1000,- endSec: Number(entry.offsets.to) / 1000,- }));- const target = intended.words.filter((word) => word.segmentId === segment.id);- const matches = lcs(- target.map((word) => normalize(word.text)),- measured.map((word) => normalize(word.text)),- );- const anchors = [[-1, -1], ...matches, [target.length, measured.length]];- const located = Array(target.length);- for (const [targetIndex, measuredIndex] of matches) {- located[targetIndex] = {- text: target[targetIndex].text,- startSec: measured[measuredIndex].startSec,- endSec: measured[measuredIndex].endSec,- };- }- for (let anchorIndex = 0; anchorIndex < anchors.length - 1; anchorIndex += 1) {- const [leftTarget, leftMeasured] = anchors[anchorIndex];- const [rightTarget, rightMeasured] = anchors[anchorIndex + 1];- const missing = rightTarget - leftTarget - 1;- if (missing <= 0) continue;- const measuredSlice = measured.slice(leftMeasured + 1, rightMeasured);- const startSec = measuredSlice[0]?.startSec- ?? measured[leftMeasured]?.endSec- ?? 0;- const endSec = measuredSlice.at(-1)?.endSec- ?? measured[rightMeasured]?.startSec- ?? segmentFrames / 30;- const weights = target- .slice(leftTarget + 1, rightTarget)- .map((word) => Math.max(1, normalize(word.text).length));- const totalWeight = weights.reduce((sum, value) => sum + value, 0);- let cursor = startSec;- for (let index = 0; index < missing; index += 1) {- const duration = (endSec - startSec) * weights[index] / totalWeight;- const targetIndex = leftTarget + 1 + index;- located[targetIndex] = {- text: target[targetIndex].text,- startSec: cursor,- endSec: index === missing - 1 ? endSec : cursor + duration,- };- cursor += duration;- }- }- const offsetSec = segmentIndex * segmentFrames / 30;- const segmentEndSec = (segmentIndex + 1) * segmentFrames / 30;- for (const word of located) {- evidenceWords.push({- text: word.text,- startSec: Math.max(offsetSec, Math.min(segmentEndSec, word.startSec + offsetSec)),- endSec: Math.max(offsetSec, Math.min(segmentEndSec, word.endSec + offsetSec)),- segmentId: segment.id,- });- }-}--const evidence = {- contract: "svml.speech-alignment.v1",- durationSec: intended.segments.length * segmentFrames / 30,- fps: 30,- words: evidenceWords,- segments: intended.segments.map((segment, index) => ({- id: segment.id,- startSec: index * segmentFrames / 30,- endSec: (index + 1) * segmentFrames / 30,- })),- provenance: {- method: "whisper.cpp word timestamps + deterministic intended-script LCS projection",- source: "existing pinned avatar audio; no generation call",- },-};-if (cueFile) {- const cueDocument = JSON.parse(await readFile(resolve(cueFile), "utf8"));- evidence.captionCues = Array.isArray(cueDocument)- ? cueDocument- : cueDocument.captionCues;- if (!Array.isArray(evidence.captionCues)) {- throw new Error(`${cueFile} must contain an array or a captionCues array.`);- }- evidence.provenance.captionCues = cueDocument.provenance- ?? "pinned cue partition from the reference Locate result";-}-if (timingOverrideFile) {- const overrideDocument = JSON.parse(- await readFile(resolve(timingOverrideFile), "utf8"),- );- if (!Array.isArray(overrideDocument.words)) {- throw new Error(`${timingOverrideFile} must contain a words array.`);- }- for (const override of overrideDocument.words) {- const index = Number(override.index);- const word = evidence.words[index];- if (!Number.isInteger(index) || !word) {- throw new Error(`${timingOverrideFile} contains invalid word index ${String(override.index)}.`);- }- if (override.startSec !== undefined) word.startSec = Number(override.startSec);- if (override.endSec !== undefined) word.endSec = Number(override.endSec);- }- evidence.provenance.timingOverrides = overrideDocument.provenance- ?? "reference transition recovery";-}-await writeFile(resolve(outputFile), `${JSON.stringify(evidence, null, 2)}\n`);