Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.
TypeScript agentic-aiaiai-agentsai-videocompilerdslffmpeggenerative-aillmmarkup-languagemonorepoplugin-systemprogramming-languagetext-to-videotypescriptvideovideo-automationvideo-clonevideo-editingvideo-generation
feat: implement decoupled SVML v1 compiler
kashorin61f8fc848faa
diff --git a/README.md b/README.mdindex d6c4d77c..05899a86 100644--- a/README.md+++ b/README.md@@ -1,56 +1,60 @@# SVMLSemantic Video Markup Language is a semantic source language for information-flow-video. Script creates stable semantic addresses; a Composition selects one-content-addressed ProgramBasis and one imported Locator Component whose SemanticMap-binds those addresses to that basis; Track Components then lower the program into-one deterministic HyperFrames document.+video. Script creates stable semantic addresses. A Composition selects one+content-addressed `ProgramBasis` and one exact `SemanticMap`; flat Track+Components then lower every audiovisual contribution into one deterministic+HyperFrames HTML document.```text-.svml + .svc + .svs + .svk + lock + evidence- │- ├─ Plan IR ─────────────────────> Canvas view- ├─ Basis + SemanticMap ─────────> TemporalBinding- ├─ TemporalBinding ─────────────> Timeline view- └─ flat Track[] ────────────────> HyperFrames HTML+.svml + transitive .svk/.svs/.svc imports + lock + evidence+ │+ ├─ typed Plan IR ─────────────────────────────> Canvas view+ ├─ Basis Producer ─────> TemporalBasisProduction+ ├─ Locator ────────────> ExactSemanticMap+ ├─ Basis + Map ────────> TemporalBinding ─────> Timeline view+ └─ flat Track[] ───────> Composition ─────────> HyperFrames HTML```-Canvas and Timeline are views of the same source closure. They are not additional-authoring truths. Provider tasks, pins, takes and caches are runtime state and do-not enter the source language.+Canvas and Timeline are views of the same source closure, not additional+authoring truths. Provider tasks, pins, takes, caches and queue state belong to a+Runtime Host and do not enter the source language.++## Implemented v1 slice++The TypeScript compiler and local standard library implement the current v1+architecture end to end:++- readable Script with Role Cues, Dual Text, Slots, closed/disconnected+ Selections and zero-width Moments;+- an exact `SemanticIndex` with independent word and Segment endpoints+ (`2M + 2N` identities);+- replaceable `TemporalBasisProduction` and `ExactSemanticMap` Components,+ validated and bound before any Track lowering;+- hard cut, gap and audio/visual crossfade joins with distinct source-to-program+ maps;+- typed `.svc` content modules and `.svs` classes, including nested Component+ fields;+- stable Plan identity, separate execution digests and a full temporal+ `svml.lock`;+- isolated `.svk` projectors, verified offline capability artifacts and finite,+ auditable `composite-v1` expansion;+- manifest-enforced temporal `one` / `each` / `set` consumption;+- flat media, ranking, B-roll, caption, text and audio Tracks in one absolute+ ProgramSpace;+- Film as the only `Track[]` consumer and HyperFrames HTML as the sole formal+ video compilation target.-## Current prototype+There is no live provider adapter in this repository. A reachable+`profile="capability-v1"` Component must be satisfied by an exact,+content-verified `svml.artifacts.v1` binding; compilation never falls back to a+provider call. Existing local `Image`, `Video`, `Audio` and Evidence values work+directly.-The repository contains a strict TypeScript compiler and an intentionally small-stdlib. The deterministic slice is working end to end:+SVML has not been publicly released. There is no legacy v1 compatibility layer:+the current architecture and Script Surface are the only v1 target.-- an early Script parser with dialogue, speech and caption projections;-- closed, possibly disconnected Selection sets and zero-width Moments;-- `.svc` content modules and typed `.svs` parameter classes;-- stable Plan identities, separate execution digests and a verified `svml.lock`;-- isolated `.svk` component lowering with scoped CSS and a JSON-only ABI;-- typed `.svk` ports, parameters and recursive child-content schemas;-- manifest-enforced temporal `one` / `each` / `set` consumption;-- a temporary Speech Spine prototype, media/presentation mapping, ranking, B-roll,- captions, text and audio;-- frame-exact Located IR, Canvas/Timeline views and HyperFrames HTML.--The current runtime deliberately has no live generation provider host. A-`profile="capability-v1"` Component remains visible in Plan, but compilation requires-an exact, content-verified `svml.artifacts.v1` binding for its outputs and never-falls back to a provider call. Direct existing `Image`, `Video` and `Audio` values-work as usual. This keeps the first end-to-end reference deterministic while-preserving the eventual capability boundary.--SVML has not been publicly released, so the repository does not preserve the early-prototype as a compatibility contract. The sole v1 target is the architecture draft-and Script Surface v1: a replaceable Basis Component, an imported Locator Component,-and `2M + 2N` independent Segment/word endpoint identities. The executable prototype-still uses shared adjacent Segment cuts and a temporary `speech-spine`; it is an-incomplete implementation scheduled for direct replacement, not an older language-version that v1 must support.--## Commands+## Quick startRequires Node.js 22 and pnpm.@@ -61,70 +65,73 @@ pnpm testpnpm buildpnpm svml check examples/regen-ranking/regen-ranking.svml-pnpm svml fmt examples/regen-ranking/regen-ranking.svml --check+pnpm svml script examples/regen-ranking/regen-ranking.svml --out narrative.json+pnpm svml plan examples/regen-ranking/regen-ranking.svml --out plan.jsonpnpm svml canvas examples/regen-ranking/regen-ranking.svml --out canvas.json-pnpm svml estimate examples/regen-ranking/regen-ranking.svml \- --out estimated-alignment.json-pnpm svml timeline examples/regen-ranking/regen-ranking.svml \- --evidence examples/regen-ranking/evidence/alignment.json \- --out timeline.json+pnpm svml estimate examples/regen-ranking/regen-ranking.svml --out estimate.json+pnpm svml timeline examples/regen-ranking/regen-ranking.svml --out timeline.json++# Produce a full frozen lock, including the selected Basis, Map and Evidence.pnpm svml lock examples/regen-ranking/regen-ranking.svml --out svml.lock+pnpm svml compile examples/regen-ranking/regen-ranking.svml \- --evidence examples/regen-ranking/evidence/alignment.json \--lock svml.lock \--out build/index.htmlpnpm svml render build/index.html --out build/video.mp4```-`--artifacts` is needed only when the reachable Plan contains capability Components.-`fmt` currently canonicalizes only the prototype Script body and refuses any rewrite-whose reparsed semantic IR differs.-`canvas` requires no timing or provider result. `estimate` deterministically-creates provisional syllable-based alignment; `timeline` and `compile` consume-either that preview evidence or measured Speech Spine evidence. The current-prototype verifies source closure and bound capability artifacts. It does not yet-claim complete temporal reproducibility because Evidence, Locator, ProgramBasis and-SemanticMap digests are not all part of the executable lock path.--## Reference implementation--[`examples/regen-ranking/regen-ranking.svml`](examples/regen-ranking/regen-ranking.svml)-reconstructs a real production video using the temporary Speech Spine prototype,-a five-item ranking track, five B-roll items, captions, title, music and sound effects. The media files-are intentionally gitignored; their production provenance and frozen assertions-are recorded in-[`examples/regen-ranking/reference.json`](examples/regen-ranking/reference.json).-The regression test compiles the example to 27 Canvas nodes, 29 typed topology-edges, 1083 frames, 155 visual fragments and 14 audio fragments.-The default syllable estimator predicts 36.267 seconds for the frozen 36.1-second-speech program (0.46% duration error); it remains preview evidence, not measured-alignment.--[`examples/flat-track-launch/flat-track-launch.svml`](examples/flat-track-launch/flat-track-launch.svml)-is the readable fixture for the v1 architecture target: separate Basis and-Locator Components, Script Surface v1, continuous A-roll Present changes, B-roll-transition audio, disconnected caption selections, absolute z and Film as the sole-`Track[]` consumer. It is a draft fixture not yet accepted by the early compiler-prototype.--[`examples/composite-speech-program/minimal.svml`](examples/composite-speech-program/minimal.svml)-and its adjacent `speech-program.svk` define the proposed `composite-v1` author-surface: one transparent Component replaces repeated Basis/Locator wiring while-both internal instances remain independently visible in Plan and lock. Its-`SpeechProgram` output is only a typed alias bundle, per-Segment joins are author-data consumed by one assembler, and the Component has no ambient document access.--## Specification--- [Script Surface v1](spec/script-surface-v1.md) — the sole draft target for the- human-readable Script, including independent Segment endpoints and `2M + 2N`.-- [Source Architecture Draft](spec/source-architecture-draft.md) — the current- draft for `.svk` components, `.svs` parameter sheets, `.svc` content modules,- Script-dependent generation, five public output boundaries, `composite-v1`, flat- Tracks, and a Composition root.-- [Compiler prototype record](docs/compiler-prototype.md) — implemented- boundaries, real-video evidence and remaining work.-- [External Runtime Host Architecture Draft](docs/runtime-host-architecture-draft.md)- — portable runtime, Artifact, Effect/Receipt, Worker, queue, provider and- self-hosting boundaries; TemporalBasisProduction acquisition remains- intentionally deferred.+`--artifacts` is required only when the reachable Plan contains capability+Components. Exact alignment is an explicit typed value referenced by the+Locator in source; it is not an ambient CLI argument. `estimate` returns an+`EstimatedSemanticMap` for preview and cannot be supplied where final HTML+requires an `ExactSemanticMap`.++`fmt` currently canonicalizes only the Script body and refuses a rewrite if the+reparsed semantic IR differs. `canvas` needs no materialized time evidence.+`timeline`, `compile` and `lock` execute the deterministic local Plan and validate+the selected Basis/Map pair.++## Four source files++| Suffix | Responsibility |+|---|---|+| `.svml` | One film's Script, local values, Component calls, Tracks and Composition |+| `.svk` | An importable Component vocabulary, public typed ABI and lowering |+| `.svs` | Typed parameter classes for public Component parameters and fields |+| `.svc` | Context-free reusable content values and content subgraphs |++The compiler owns universal language laws. Libraries own replaceable vocabulary+and algorithms. A Runtime Host owns effects, credentials, queues and storage.+Canvas owns only a human-readable view.++## Executable examples++- [`examples/regen-ranking/regen-ranking.svml`](examples/regen-ranking/regen-ranking.svml)+ reconstructs a pinned production video with five ranking items, five B-roll+ items, captions, title, BGM and sound effects. It compiles to 29 Plan nodes,+ 33 typed edges, 1083 frames, 153 visual fragments and 14 audio fragments.+- [`examples/flat-track-launch/flat-track-launch.svml`](examples/flat-track-launch/flat-track-launch.svml)+ is the comprehensive language fixture: explicit Basis/Locator, nested SVS,+ overlapping Presents, manual ProgramSpans, a Moment, B-roll source audio and+ transition SFX, disconnected caption styling and absolute z.+- [`examples/composite-speech-program/minimal.svml`](examples/composite-speech-program/minimal.svml)+ proves that an author-facing `speech-program` can hide repetitive wiring while+ expanding to separately auditable `voice::basis` and `voice::locator`+ instances. The compiler has no special knowledge of that Component name.+- [`examples/media-basis`](test/fixtures/media-basis) proves that precomposed+ media can implement the same Basis contract and reuse the same Locator and+ Composition path.++The local media used by the examples are already-materialized artifacts; no+generation provider is invoked by tests or compilation.++## Specification and implementation record++- [Script Surface v1](spec/script-surface-v1.md)+- [Source Architecture v1 draft](spec/source-architecture-draft.md)+- [Compiler implementation record](docs/compiler-prototype.md)+- [External Runtime Host architecture](docs/runtime-host-architecture-draft.md)++VLM, bbox, face tracking and visual reverse-location are intentionally outside+SVML v1. They may be implemented later as explicit post-processing extensions;+the semantic audio timeline remains the language's locating foundation.diff --git a/examples/composite-speech-program/README.md b/examples/composite-speech-program/README.mdindex b8e0a439..1d416eb5 100644--- a/examples/composite-speech-program/README.md+++ b/examples/composite-speech-program/README.md@@ -1,49 +1,40 @@-# Composite Speech Program draft fixture+# Composite Speech Program executable fixture-This directory demonstrates the proposed, unimplemented `composite-v1` SVK-profile.+This directory proves the implemented `composite-v1` profile.- `minimal.svml` is the author-facing source.-- `speech-program.svk` defines a callable Component that statically expands- `speech-assemble` and `speech-locator`.-- `house.svs` configures only the Composite's public join shorthand.+- `speech-program.svk` statically expands `speech-assemble` and+ `speech-locator`.+- `house.svs` configures only public Composite parameters.+- `alignment.json` is explicit typed Evidence.-For `<speech-program id="voice">`, canonical expansion creates the internal-instances `voice::basis` and `voice::locator`. Their implementations, digests,-Evidence and Artifacts remain independently visible in Plan and lock. The outer-ports are aliases:+For `<speech-program id="voice">`, expansion creates:```text-voice.program.production = voice::basis.production-voice.program.semantic = voice::locator.map-voice.production = voice::basis.production-voice.map = voice::locator.map-voice.facets.visual = voice::basis.facets.visual-```+voice::basis → TemporalBasisProduction + ProgramBoundVideoSequence+ │+ └──────────────> voice::locator → ExactSemanticMap-`voice.program` is a typed port bundle whose fields alias the two internal-outputs. It does not copy audio, facets, Evidence or Artifacts, and it does not-replace the compiler's basis/map affinity check. The imported Film facade accepts-the bundle as author-facing shorthand; after Composite expansion the Plan still-contains the selected Basis and ExactSemanticMap separately.+voice.production = voice::basis.production+voice.map = voice::locator.map+voice.facets.visual = voice::basis.facets.visual+```-The default join and each `<join after="...">` are author data. They normalize to-per-boundary `JoinSpec` values consumed by one `speech-assemble` instance; they do-not select a language-level family or mode. `after` names a stable Segment id, and-the following Segment comes from source order. A join cannot declare both `gap`-and `overlap`.+The two internal instances, implementation digests, Evidence dependencies and+outputs remain visible in Plan and lock. The outer ports are aliases; the+Composite does not copy Artifacts or merge execution boundaries.-The v1 stdlib publishes `speech-program` as its chosen author surface. It remains-an ordinary imported Composite, never a compiler keyword or a required singleton.-Alternative Locators use explicit low-level wiring or another Composite instead-of extending a closed `locator="..."` enum.+`join` and `joinDuration` are author shorthand forwarded to one assembler. Each+explicit `<join after="...">` may override the corresponding boundary. They do+not select a compiler family or language-level mode.The bracket expression in `ports.script.segment[item.id]` is a stable id-keyed-lookup, not a positional array lookup. The `for` iterates only over finite children-declared at the call site.--The Composite receives its Script through an explicit typed input. Reading-`document.script` or any other ambient caller state is forbidden.--The current compiler must reject or clearly report unsupported `composite-v1`;-these files are architecture fixtures, not executable regression inputs.+lookup over finite declared children, not positional indexing. The Composite+receives Script and Evidence through explicit typed inputs and cannot read+ambient document state.++```sh+pnpm svml check examples/composite-speech-program/minimal.svml+pnpm svml compile examples/composite-speech-program/minimal.svml \+ --out /tmp/svml-composite/index.html+```diff --git a/examples/composite-speech-program/alignment.json b/examples/composite-speech-program/alignment.jsonnew file mode 100644index 00000000..5cc43427--- /dev/null+++ b/examples/composite-speech-program/alignment.json@@ -0,0 +1,21 @@+{+ "contract": "svml.speech-alignment.v1",+ "durationSec": 9.5,+ "fps": 30,+ "words": [+ { "text": "Hello", "startSec": 0.2, "endSec": 0.8, "segmentId": "intro" },+ { "text": "this", "startSec": 1.0, "endSec": 1.5, "segmentId": "intro" },+ { "text": "is", "startSec": 1.6, "endSec": 1.9, "segmentId": "intro" },+ { "text": "SVML", "startSec": 2.0, "endSec": 2.8, "segmentId": "intro" },+ { "text": "Write", "startSec": 4.7, "endSec": 5.2, "segmentId": "close" },+ { "text": "the", "startSec": 5.3, "endSec": 5.6, "segmentId": "close" },+ { "text": "script", "startSec": 5.7, "endSec": 6.2, "segmentId": "close" },+ { "text": "get", "startSec": 6.4, "endSec": 6.7, "segmentId": "close" },+ { "text": "the", "startSec": 6.8, "endSec": 7.1, "segmentId": "close" },+ { "text": "video", "startSec": 7.2, "endSec": 8.0, "segmentId": "close" }+ ],+ "segments": [+ { "id": "intro", "startSec": 0, "endSec": 5 },+ { "id": "close", "startSec": 4.5, "endSec": 9.5 }+ ]+}diff --git a/examples/composite-speech-program/minimal.svml b/examples/composite-speech-program/minimal.svmlindex 02a13028..a15d9958 100644--- a/examples/composite-speech-program/minimal.svml+++ b/examples/composite-speech-program/minimal.svml@@ -1,12 +1,7 @@-<!--- Proposed composite-v1 author surface. The current compiler does not implement- this profile or the complete Script Surface v1 yet.---><svml version="1">-<import from="./speech-program.svk"/>- <import from="@svml/std/media-track.svk"/>- <import from="@svml/std/film.svk"/>+ <import from="../../stdlib/media-track.svk"/>+ <import from="../../stdlib/film.svk"/><import as="house" from="./house.svs"/><script>@@ -19,42 +14,21 @@</segment></script>- <video id="host" src="./host.mp4"/>-- <speech-program- id="voice"- script={script}- class="house.default"- >- <segment- id="intro"- source={host}- sourceWindow="0s .. 5s"- />- <segment- id="close"- source={host}- sourceWindow="5s .. 10s"- />+ <video id="host" src="../regen-ranking/assets/avatar-1.mp4"/>+ <audio id="host-audio" src="../regen-ranking/assets/avatar-1.mp3"/>+ <alignment id="alignment" src="./alignment.json"/>- <join- after="intro"- overlap="500ms"- audio="crossfade"- visual="dissolve"- />+ <speech-program id="voice" script={script} evidence={alignment} class="house.default">+ <segment id="intro" visual={host} audio={host-audio} sourceWindow="0s .. 5s"/>+ <segment id="close" visual={host} audio={host-audio} sourceWindow="5s .. 10s"/>+ <join after="intro" overlap="500ms" audio="crossfade" visual="dissolve"/></speech-program>- <media-track id="aroll">- <item- source={voice.facets.visual}- during="full"- playback="program-map"- />+ <media-track id="aroll" z="100">+ <item id="host" source={voice.facets.visual} during="full" playback="program-map"/></media-track>- <film program={voice.program}>+ <film id="main" basis={voice.production} semantic={voice.map} width="1080" height="1920"><track ref={aroll.track}/></film>-</svml>diff --git a/examples/composite-speech-program/speech-program.svk b/examples/composite-speech-program/speech-program.svkindex a00423d6..0dc9970f 100644--- a/examples/composite-speech-program/speech-program.svk+++ b/examples/composite-speech-program/speech-program.svk@@ -1,62 +1,18 @@-<!-- Proposed composite-v1 syntax; not implemented by the current compiler. -->-<kernel- name="speech-program"- abi="1"- profile="composite-v1"->+<kernel name="speech-program" abi="1" profile="composite-v1">+ <port name="script" direction="input" type="NarrativeIR" cardinality="one"/>+ <port name="evidence" direction="input" type="AlignmentEvidence" cardinality="one"/>+ <port name="production" direction="output" type="TemporalBasisProduction" cardinality="one"/>+ <port name="map" direction="output" type="ExactSemanticMap" cardinality="one"/>+ <port name="facets.visual" direction="output" type="ProgramBoundVideoSequence" cardinality="one"/>- <port- name="script"- direction="input"- type="Script"- cardinality="one"- />-- <port- name="production"- direction="output"- type="TemporalBasisProduction"- cardinality="one"- />-- <port- name="program"- direction="output"- type="SpeechProgram"- cardinality="one"- />-- <port- name="map"- direction="output"- type="ExactSemanticMap"- cardinality="one"- />-- <port- name="facets.visual"- direction="output"- type="ProgramBoundVideo"- cardinality="zero-or-one"- />-- <param- name="join"- type="enum(cut,crossfade,gap)"- default="cut"- />-- <param- name="joinDuration"- style="join-duration"- type="duration"- default="0ms"- />+ <param name="join" type="enum(cut,crossfade,gap)" default="cut"/>+ <param name="joinDuration" style="join-duration" type="duration" default="0ms"/><child name="segment" cardinality="one-or-more"><field name="id" type="id" required="true"/>- <field name="source" type="Video" required="true"/>- <field name="sourceWindow" type="SourceRange"/>+ <field name="visual" type="Video"/>+ <field name="audio" type="Audio" required="true"/>+ <field name="sourceWindow" type="SourceRange" required="true"/></child><child name="join" cardinality="zero-or-more">@@ -67,32 +23,20 @@<field name="visual" type="enum(cut,dissolve)"/></child>- <import- as="basis-component"- from="@svml/std/speech-assemble.svk"- />-- <import- as="locator-component"- from="@svml/std/speech-locator.svk"- />+ <import as="assemble" from="../../stdlib/speech-assemble.svk"/>+ <import as="locate" from="../../stdlib/speech-locator.svk"/><compose>-- <basis-component- id="basis"- defaultJoin={params.join}- defaultJoinDuration={params.joinDuration}- >+ <assemble id="basis" defaultJoin={params.join} defaultJoinDuration={params.joinDuration}><for each={children.segment} as="item"><segmentid={item.id}script={ports.script.segment[item.id]}- source={item.source}+ visual={item.visual}+ audio={item.audio}sourceWindow={item.sourceWindow}/></for>-<for each={children.join} as="join"><joinafter={join.after}@@ -102,36 +46,12 @@visual={join.visual}/></for>- </basis-component>-- <locator-component- id="locator"- script={ports.script}- basis={basis.production}- />-- <export- port="production"- value={basis.production}- />+ </assemble>- <export- port="map"- value={locator.map}- />-- <export port="program">- <value- production={basis.production}- semantic={locator.map}- />- </export>-- <export- port="facets.visual"- value={basis.facets.visual}- />+ <locate id="locator" script={ports.script} basis={basis.production} evidence={ports.evidence}/>+ <export port="production" value={basis.production}/>+ <export port="map" value={locator.map}/>+ <export port="facets.visual" value={basis.facets.visual}/></compose>-</kernel>diff --git a/examples/flat-track-launch/README.md b/examples/flat-track-launch/README.mdindex 9940f1ec..4984925a 100644--- a/examples/flat-track-launch/README.md+++ b/examples/flat-track-launch/README.md@@ -1,20 +1,23 @@-# Flat Track Launch architecture fixture+# Flat Track Launch executable fixture-This source is the readable fixture for the unimplemented Source Architecture-Draft. It deliberately exercises v1 contracts that the early compiler does not-yet accept:+This is the end-to-end v1 architecture fixture. It compiles without generation+providers and exercises the contracts together:-- Script Surface v1 independent Segment endpoints (`2M + 2N`);-- a Basis Component that overlaps adjacent Segment media;-- an imported Locator Component that produces the selected ExactSemanticMap;-- one ProgramBasis and one absolute ProgramSpace;-- flat visual/audio Track contributions with no Track nesting;-- continuous A-roll source mapping across overlapping Present windows;-- B-roll transition audio inside the B-roll Track;-- disconnected caption selections, Moments and manual ProgramSpans;-- Film as the sole `Track[]` consumer.+- independent Script Segment endpoints (`2M + 2N`);+- replaceable Basis Producer and Locator Components;+- one explicit `ProgramBasis` plus `ExactSemanticMap`;+- flat Tracks and absolute `z`, with Film as the only `Track[]` consumer;+- one continuous A-roll mapping with overlapping Present windows;+- manual `ProgramSpan`, a `Moment`, B-roll audio and a visual crossfade;+- a disconnected SelectionSet driving one caption style rule;+- top-level and nested Component fields configured by `.svs`.-The `@svml/std/*` imports and local assets are illustrative package paths. The-adjacent `launch.svs` makes every ProgramSpace placement explicit. Do not use this-fixture as the executable regression until the flat Track milestone lands; use-`../regen-ranking/regen-ranking.svml` for the current compiler.+It reuses the checked-in local artifacts and exact alignment evidence from the+adjacent `regen-ranking` example. No provider is called.++```sh+pnpm svml compile examples/flat-track-launch/flat-track-launch.svml \+ --out /tmp/svml-flat-track-launch/index.html+pnpm svml render examples/flat-track-launch/flat-track-launch.svml \+ --out /tmp/svml-flat-track-launch.mp4+```diff --git a/examples/flat-track-launch/flat-track-launch.svml b/examples/flat-track-launch/flat-track-launch.svmlindex 22bccfbb..a3d82c5f 100644--- a/examples/flat-track-launch/flat-track-launch.svml+++ b/examples/flat-track-launch/flat-track-launch.svml@@ -1,263 +1,122 @@-<!--- Architecture fixture for Source Architecture Draft.- It is intentionally not accepted by the current early compiler prototype.---><svml version="1">- <import from="@svml/std/speech-assemble.svk"/>- <import from="@svml/std/speech-locator.svk"/>- <import from="@svml/std/media-track.svk"/>- <import from="@svml/std/broll-track.svk"/>- <import from="@svml/std/ranking-tier-list.svk"/>- <import from="@svml/std/text-track.svk"/>- <import from="@svml/std/caption-track.svk"/>- <import from="@svml/std/audio-track.svk"/>- <import from="@svml/std/film.svk"/>-+ <import from="../../stdlib/speech-assemble.svk"/>+ <import from="../../stdlib/speech-locator.svk"/>+ <import from="../../stdlib/media-track.svk"/>+ <import from="../../stdlib/broll-track.svk"/>+ <import from="../../stdlib/ranking-column.svk"/>+ <import from="../../stdlib/text-track.svk"/>+ <import from="../../stdlib/caption-track.svk"/>+ <import from="../../stdlib/audio-track.svk"/>+ <import from="../../stdlib/film.svk"/><import as="house" from="./launch.svs"/><script>@whole- @prelude- <segment id="prelude"/>- @/prelude+ <segment id="scene-1">+ <NARRATOR> @photoshop @emphasis Photoshop @/emphasis.+ <SPEAKER> Powerful, but only if you know how to use it. Otherwise, your new profile+ picture becomes a three-hour design project @/photoshop.+ <NARRATOR> @facetune Facetune.+ <SPEAKER> Fast and great for touch-ups. But it still leaves you with the same awkward+ pose and the same dorm-room background @/facetune.+ </segment>- <segment id="intro">- <HOST> @promise @emphasis Video creation is about to change @/emphasis.- @/promise~ @ranking! @rank-svml SVML starts with meaning, not pixels.- @/rank-svml+ <segment id="scene-2">+ <NARRATOR> @remini Remini.+ <SPEAKER> Great for blurry photos. It restores detail, but it can't fix the pose,+ outfit, or background. You get the same bad photo in HD @/remini.+ <NARRATOR> @chatgpt ChatGPT.+ <SPEAKER> More flexible. It can create a new shot, but getting your face and the+ details right becomes a prompt-writing group project @/chatgpt.</segment>- <segment id="close">- <HOST> @reveal One semantic source becomes @emphasis every frame @/emphasis,- every track, and every version. @/reveal+ <segment id="scene-3">+ <NARRATOR> @ranking! @regen ReGen.+ <SPEAKER> Pick a photo you love, and the app @handsome-1 rebuilds it around you with a+ @/handsome-1~ @handsome-2 @emphasis better pose @/emphasis, outfit, and setting. @/handsome-2+ From one selfie, you can make @dating-photo a dating photo, @/dating-photo~+ @linkedin-headshot LinkedIn headshot, @/linkedin-headshot~+ @instagram-post or Instagram post, @/instagram-post+ so you get what you wanted instead of fixing a bad shot @/regen.</segment>@/whole~</script>- <!-- Plain values can fan out to Basis and Track Components. -->- <video id="host-long" src="./assets/host-50s.mp4"/>- <image id="poster" src="./assets/poster.png"/>- <video id="product-demo" src="./assets/product-demo.mp4"/>- <image id="rank-icon-svml" src="./assets/svml.png"/>- <image id="rank-icon-editors" src="./assets/editors.png"/>- <audio id="whoosh" src="./assets/whoosh.wav"/>- <audio id="music" src="./assets/music.mp3"/>-- <!--- Basis production assembles physical media only. It emits ProgramBasis,- source maps, program audio and a program-bound visual facet. The Locator- Component separately binds this Script's 2M + 2N identities to the basis.- -->- <speech-assemble id="voice" defaultJoin="cut">- <segment- id="prelude"- script={script.segment.prelude}- source={host-long}- sourceWindow="0s .. 10s"- />- <segment- id="intro"- script={script.segment.intro}- source={host-long}- sourceWindow="10s .. 30s"- />- <segment- id="close"- script={script.segment.close}- source={host-long}- sourceWindow="30s .. 50s"- />-- <join- after="prelude"- overlap="500ms"- audio="crossfade"- visual="dissolve"- />-- <join- after="intro"- overlap="500ms"- audio="crossfade"- visual="dissolve"- />+ <!-- These local artifacts are inputs; the example invokes no generation model. -->+ <video id="avatar-1" src="../regen-ranking/assets/avatar-1.mp4"/>+ <video id="avatar-2" src="../regen-ranking/assets/avatar-2.mp4"/>+ <video id="avatar-3" src="../regen-ranking/assets/avatar-3.mp4"/>+ <audio id="avatar-1-audio" src="../regen-ranking/assets/avatar-1.mp3"/>+ <audio id="avatar-2-audio" src="../regen-ranking/assets/avatar-2.mp3"/>+ <audio id="avatar-3-audio" src="../regen-ranking/assets/avatar-3.mp3"/>+ <image id="ranking-icon-1" src="../regen-ranking/assets/ranking-icon-1.jpg"/>+ <image id="ranking-icon-2" src="../regen-ranking/assets/ranking-icon-2.jpg"/>+ <image id="ranking-icon-3" src="../regen-ranking/assets/ranking-icon-3.jpg"/>+ <image id="ranking-icon-4" src="../regen-ranking/assets/ranking-icon-4.jpg"/>+ <image id="ranking-icon-5" src="../regen-ranking/assets/ranking-icon-5.jpg"/>+ <image id="broll-1" src="../regen-ranking/assets/broll-5.png"/>+ <audio id="ranking-appear" src="../regen-ranking/assets/ranking-appear.wav"/>+ <audio id="ranking-move" src="../regen-ranking/assets/ranking-move.wav"/>+ <audio id="music" src="../regen-ranking/assets/background.mp3"/>+ <alignment id="alignment" src="../regen-ranking/evidence/alignment.json"/>++ <!-- The Basis Producer and Locator are replaceable Components with typed outputs. -->+ <speech-assemble id="voice" fps="30" defaultJoin="cut">+ <segment id="scene-1" script={script.segment.scene-1} visual={avatar-1} audio={avatar-1-audio} sourceWindow="0s .. 12.033333333333333s"/>+ <segment id="scene-2" script={script.segment.scene-2} visual={avatar-2} audio={avatar-2-audio} sourceWindow="0s .. 12.033333333333333s"/>+ <segment id="scene-3" script={script.segment.scene-3} visual={avatar-3} audio={avatar-3-audio} sourceWindow="0s .. 12.033333333333333s"/></speech-assemble>-- <!--- SemanticMap is the fifth SVK output role. A simple speech-locator may expose- one atomic capability; a modular alternative can consume explicit Evidence- Value Components while preserving this exact output contract.- -->- <speech-locator- id="location"- script={script}- basis={voice.production}- />-- <!--- One program-bound host facet, one continuous source mapping. Present changes- only x/y/size/mask/z; it never restarts or seeks the video.- -->- <media-track id="aroll" z="200">- <item- id="host"- source={voice.facets.visual}- during={script.selection.whole}- playback="program-map"- class="house.host-full"- >- <!-- Ten seconds without words: manual projections remain legal. -->- <present- id="prelude-top-left"- during={script.selection.prelude}- window="start .. start+2.2s"- z="210"- class="house.face-top-left"- exit="fade 200ms"- />- <present- id="prelude-top-right"- during={script.selection.prelude}- window="start+2s .. start+4.2s"- z="211"- class="house.face-top-right"- enter="fade 200ms"- exit="fade 200ms"- />- <present- id="prelude-bottom-right"- during={script.selection.prelude}- window="start+4s .. start+6.2s"- z="212"- class="house.face-bottom-right"- enter="fade 200ms"- exit="fade 200ms"- />- <present- id="prelude-bottom-left"- during={script.selection.prelude}- window="start+6s .. start+8.2s"- z="213"- class="house.face-bottom-left"- enter="fade 200ms"- exit="fade 200ms"- />- <present- id="prelude-center"- during={script.selection.prelude}- window="start+8s .. end"- z="214"- class="house.face-center"- enter="fade 200ms"- />-- <present- id="promise-split"- during={script.selection.promise}- z="220"- class="house.host-lower-split"- />- <present- id="reveal-pip"- during={script.selection.reveal}- window="start-200ms .. end"- z="221"- class="house.host-corner-circle"- enter="fade 200ms"- />+ <speech-locator id="location" script={script} basis={voice.production} evidence={alignment}/>++ <!-- One source mapping; Present only changes its projection in ProgramSpace. -->+ <media-track id="aroll" z="20">+ <item id="host" source={voice.facets.visual} during={script.selection.whole} class="house.host-full">+ <present id="opening-left" during="0s .. 2.2s" z="31" class="house.face-top-left" exit="fade 200ms"/>+ <present id="opening-right" during="2s .. 4.2s" z="32" class="house.face-top-right" enter="fade 200ms" exit="fade 200ms"/>+ <present id="opening-low" during="4s .. 6.2s" z="33" class="house.face-bottom-right" enter="fade 200ms" exit="fade 200ms"/>+ <present id="opening-full" during="6s .. 8s" z="34" class="house.face-center" enter="fade 200ms"/>+ <present id="photoshop-split" during={script.selection.photoshop} z="35" class="house.host-lower-split"/>+ <present id="regen-pip" during={script.selection.regen} window="start-200ms .. end" z="36" class="house.host-corner-circle" enter="fade 200ms"/></item></media-track>- <!-- B-roll owns item composition, visual transition, source audio and SFX. -->- <broll-track id="broll" z="100" class="house.broll-full">- <item- id="poster"- source={poster}- during={script.selection.promise}- playback="hold"- />- <item- id="product-demo"- source={product-demo}- during={script.selection.reveal}- playback="stretch"- sourceAudio="on"- sourceGain="0.35"- />- <transition- from="poster"- to="product-demo"- kind="crossfade"- duration="200ms"- sound={whoosh}- soundGain="0.7"- />+ <!-- B-roll keeps its own source audio and transition sound in the same Track. -->+ <broll-track id="broll" z="100">+ <item id="poster" source={broll-1} during={script.selection.handsome-1} playback="hold" class="house.broll-full"/>+ <item id="product-demo" source={avatar-3} during={script.selection.handsome-2} playback="stretch" sourceDuration="12.033333333333333" audio="source" volume="0.15" class="house.broll-full"/>+ <transition from="poster" to="product-demo" kind="crossfade" duration="200ms" sound={ranking-appear} volume="0.7"/></broll-track>- <!-- Ranking is another flat Track, not a child of either media Track. -->- <ranking-tier-list- id="ranking"- z="300"- at={script.moment.ranking}- class="house.ranking"- >- <item- id="svml"- label="SVML"- rank="1"- icon={rank-icon-svml}- during={script.selection.rank-svml}- />- <item- id="editors"- label="Pixel-first editors"- rank="2"- icon={rank-icon-editors}- during={script.selection.reveal}- />- </ranking-tier-list>-- <!-- Manual ProgramSpan is intentionally legal. -->+ <!-- A Moment gates one independent Track; every item still owns its Selection. -->+ <ranking-column id="ranking" z="300" at={script.moment.ranking} class="house.ranking"+ appearSound={ranking-appear} moveSound={ranking-move}>+ <item id="regen" rank="1" image={ranking-icon-1} during={script.selection.regen}/>+ <item id="chatgpt" rank="2" image={ranking-icon-2} during={script.selection.chatgpt}/>+ <item id="remini" rank="3" image={ranking-icon-3} during={script.selection.remini}/>+ <item id="facetune" rank="4" image={ranking-icon-4} during={script.selection.facetune}/>+ <item id="photoshop" rank="5" image={ranking-icon-5} during={script.selection.photoshop}/>+ </ranking-column>++ <!-- Manual ProgramSpans remain a deliberate escape hatch. --><text-track id="overlays" z="400">- <item id="opening-title" during="0s .. 3s" class="house.title">- Semantic video starts with meaning.- </item>- <item id="ending-arrow" during="end-2s .. end" class="house.arrow">- ↓- </item>+ <item id="opening-title" text="Semantic video starts with meaning." during="0s .. 3s" class="house.title"/>+ <item id="ending-arrow" text="↓" during="end-2s .. end" class="house.arrow"/></text-track>- <!-- The repeated emphasis Selection is disconnected and consumed as a set. -->- <caption-track id="captions" z="500" script={script} class="house.captions">- <style during={script.selection.emphasis} class="house.caption-emphasis"/>+ <!-- The repeated emphasis Selection is disconnected and consumed as one set. -->+ <caption-track id="captions" z="500" script={script} semantic={location.map} class="house.captions">+ <style id="emphasis" during={script.selection.emphasis} class="house.caption-emphasis"/></caption-track><audio-track id="background-music">- <item- id="music"- source={music}- during="full"- volume="0.28"- bus="music"- duckUnder="voice"- />+ <item id="music" source={music} during="full" volume="0.28" bus="music"/></audio-track>- <!--- Child order is deliberately not z order. Film is the only Track[] consumer.- Grain and master mix are Film-level because they need the complete program.- -->- <film- id="launch"- basis={voice.production}- semantic={location.map}- class="house.vertical"- globalEffect="grain-soft"- masterMix="house.launch-mix"- >+ <!-- Child order is not paint order: absolute z across flat Tracks is authoritative. -->+ <film id="launch" basis={voice.production} semantic={location.map} class="house.vertical"><track ref={captions.track}/><track ref={background-music.track}/><track ref={broll.track}/>diff --git a/examples/flat-track-launch/launch.svs b/examples/flat-track-launch/launch.svsindex f90b811c..a1099acd 100644--- a/examples/flat-track-launch/launch.svs+++ b/examples/flat-track-launch/launch.svs@@ -1,7 +1,8 @@<sheet version="1" id="house">film.vertical {- resolution: 1080x1920;+ width: 1080;+ height: 1920;frame-rate: 30;background: #09090B;}@@ -41,15 +42,6 @@mask: circle;}- media-track.face-bottom-left {- x: 0.05;- y: 0.64;- width: 0.30;- height: 0.30;- fit: cover;- mask: circle;- }-media-track.face-center {x: 0.35;y: 0.35;@@ -84,21 +76,24 @@fit: cover;}- ranking-tier-list.ranking {+ ranking-column.ranking {x: 0.06;y: 0.14;- width: 0.88;- row-gap: 0.018;- board-color: #141418;- accent: #8B5CF6;+ width: 0.42;+ row-height: 0.071;+ gap: 0.018;+ stage-x: 0.66;+ stage-y: 0.73;+ stage-size: 0.33;}text-track.title {x: 0.06;y: 0.08;width: 0.88;- font-family: Inter;- font-size: 64px;+ height: 0.12;+ font-family: Inter, Arial, sans-serif;+ font-size: 64;font-weight: 850;fill: #FFFFFF;}@@ -107,7 +102,8 @@x: 0.42;y: 0.80;width: 0.16;- font-size: 120px;+ height: 0.14;+ font-size: 120;fill: #A78BFA;}@@ -115,14 +111,15 @@x: 0.08;y: 0.78;width: 0.84;- font-family: Inter;- font-size: 56px;+ font-family: Inter, Arial, sans-serif;+ font-size: 56;font-weight: 800;fill: #FFFFFF;}caption-track.caption-emphasis {fill: #FDE047;+ active-fill: #FFFFFF;scale: 1.08;}diff --git a/examples/regen-ranking/reference.json b/examples/regen-ranking/reference.jsonindex 00b38ee8..9360a94e 100644--- a/examples/regen-ranking/reference.json+++ b/examples/regen-ranking/reference.json@@ -14,9 +14,9 @@},"plan": {"root": "regen-ranking",- "values": 19,- "instances": 8,- "edges": 29+ "values": 20,+ "instances": 9,+ "edges": 33},"selectionFrames": {"handsome-1": [[791, 831]],@@ -26,7 +26,22 @@"instagram-post": [[977, 1009]]},"locatedOutput": {- "visualFragments": 155,+ "visualFragments": 153,"audioFragments": 14+ },+ "renderComparison": {+ "reference": "/svml-reference/regen-ranking-column-east-asian/final.mp4",+ "durationSec": 36.1,+ "video": {+ "width": 1080,+ "height": 1920,+ "fps": 30,+ "ssimAll": 0.9658+ },+ "audio": {+ "sampleRate": 48000,+ "channels": 2,+ "apsnrDb": [174.207, 174.208]+ }}}diff --git a/examples/regen-ranking/regen-ranking.svml b/examples/regen-ranking/regen-ranking.svmlindex 32f6fc6a..5149f76a 100644--- a/examples/regen-ranking/regen-ranking.svml+++ b/examples/regen-ranking/regen-ranking.svml@@ -1,6 +1,7 @@<svml version="1">- <import from="../../stdlib/speech-spine.svk"/>+ <import from="../../stdlib/speech-assemble.svk"/>+ <import from="../../stdlib/speech-locator.svk"/><import from="../../stdlib/media-track.svk"/><import from="../../stdlib/ranking-column.svk"/><import from="../../stdlib/broll-track.svk"/>@@ -61,17 +62,20 @@<audio id="ranking-appear" src="./assets/ranking-appear.wav"/><audio id="ranking-move" src="./assets/ranking-move.wav"/><audio id="background-music" src="./assets/background.mp3"/>+ <alignment id="alignment" src="./evidence/alignment.json"/>- <speech-spine id="speech">- <segment id="scene-1" script={script.segment.scene-1} visual={avatar-1} audio={avatar-1-audio}/>- <segment id="scene-2" script={script.segment.scene-2} visual={avatar-2} audio={avatar-2-audio}/>- <segment id="scene-3" script={script.segment.scene-3} visual={avatar-3} audio={avatar-3-audio}/>- </speech-spine>+ <speech-assemble id="voice" fps="30" defaultJoin="cut">+ <segment id="scene-1" script={script.segment.scene-1} visual={avatar-1} audio={avatar-1-audio} sourceWindow="0s .. 12.033333333333333s"/>+ <segment id="scene-2" script={script.segment.scene-2} visual={avatar-2} audio={avatar-2-audio} sourceWindow="0s .. 12.033333333333333s"/>+ <segment id="scene-3" script={script.segment.scene-3} visual={avatar-3} audio={avatar-3-audio} sourceWindow="0s .. 12.033333333333333s"/>+ </speech-assemble>++ <speech-locator id="location" script={script} basis={voice.production} evidence={alignment}/><media-track id="base" z="20">- <item id="scene-1" source={speech.segment.scene-1.visual} fit="cover"/>- <item id="scene-2" source={speech.segment.scene-2.visual} fit="cover"/>- <item id="scene-3" source={speech.segment.scene-3.visual} fit="cover"/>+ <item id="scene-1" source={voice.segment.scene-1.visual} fit="cover"/>+ <item id="scene-2" source={voice.segment.scene-2.visual} fit="cover"/>+ <item id="scene-3" source={voice.segment.scene-3.visual} fit="cover"/></media-track><ranking-column@@ -128,6 +132,7 @@<caption-trackid="captions"script={script}+ semantic={location.map}z="500"x="0.37"y="0.54"@@ -148,7 +153,8 @@<filmid="regen-ranking"- speech={speech.speech}+ basis={voice.production}+ semantic={location.map}width="1080"height="1920"fps="30"diff --git a/src/artifacts.ts b/src/artifacts.tsindex 73dfb249..00639bc4 100644--- a/src/artifacts.ts+++ b/src/artifacts.ts@@ -8,9 +8,9 @@ import type { KernelProjection } from "./runtime-contract.js";import { sha256, stableJson } from "./util.js";type ArtifactOutput = {- type: "Image" | "Video" | "Audio" | "Text";+ type: string;uri?: string;- value?: string;+ value?: unknown;contentDigest: string;};@@ -51,6 +51,16 @@ async function resolveOutput(}return { type: "Text", value: output.value, contentDigest: digest };}+ if (!["Image", "Video", "Audio"].includes(output.type)) {+ if (output.uri !== undefined || output.value === undefined) {+ fail("artifact_typed_value", `${output.type} artifact output requires value and forbids uri.`);+ }+ const digest = sha256(stableJson(output.value));+ if (digest !== output.contentDigest) {+ fail("artifact_digest", `${output.type} artifact contentDigest does not match its value.`);+ }+ return output.value as Record<string, unknown>;+ }if (typeof output.uri !== "string" || output.value !== undefined) {fail("artifact_media", `${output.type} artifact output requires uri and forbids value.`);}diff --git a/src/cli.ts b/src/cli.tsindex 72347360..972b13c0 100644--- a/src/cli.ts+++ b/src/cli.ts@@ -94,7 +94,12 @@ async function run(): Promise<void> {}if (args.command === "lock") {if (!file) throw new Error("usage: svml lock <file.svml> [--out svml.lock]");- const result = await checkSource(file);+ const result = await compileSource({+ file,+ ...(option(args, "artifacts")+ ? { artifactsFile: option(args, "artifacts")! }+ : {}),+ });const output = option(args, "out") ?? resolve(dirname(resolve(file)), "svml.lock");await writeJson(output, result.lock);process.stdout.write(`${JSON.stringify({@@ -188,9 +193,14 @@ async function run(): Promise<void> {: {}),...(numeric("fps") !== undefined ? { fps: numeric("fps")! } : {}),});+ const payload = {+ basis: result.basis,+ map: result.map,+ timeline: result.located,+ };const output = option(args, "out");- if (output) await writeJson(output, result.evidence);- else process.stdout.write(`${JSON.stringify(result.evidence, null, 2)}\n`);+ if (output) await writeJson(output, payload);+ else process.stdout.write(`${JSON.stringify(payload, null, 2)}\n`);return;}if (args.command === "canvas") {@@ -204,12 +214,10 @@ async function run(): Promise<void> {}if (args.command === "timeline") {if (!file) {- throw new Error("usage: svml timeline <file.svml> --evidence alignment.json [--out timeline.json]");+ throw new Error("usage: svml timeline <file.svml> [--out timeline.json]");}- const evidenceFile = option(args, "evidence", true);const compilation = await compileSource({file,- evidenceFile: evidenceFile!,...(option(args, "lock") ? { lockFile: option(args, "lock")! } : {}),...(option(args, "artifacts")? { artifactsFile: option(args, "artifacts")! }@@ -223,13 +231,11 @@ async function run(): Promise<void> {}if (args.command === "compile") {if (!file) {- throw new Error("usage: svml compile <file.svml> --evidence alignment.json --out index.html");+ throw new Error("usage: svml compile <file.svml> --out index.html");}- const evidenceFile = option(args, "evidence", true);const outputFile = option(args, "out", true);const compilation = await compileSource({file,- evidenceFile: evidenceFile!,outputFile: outputFile!,...(option(args, "lock") ? { lockFile: option(args, "lock")! } : {}),...(option(args, "artifacts")@@ -278,14 +284,14 @@ async function run(): Promise<void> {usage:svml check <file.svml>- svml lock <file.svml> [--out svml.lock]+ svml lock <file.svml> [--out svml.lock] [--artifacts artifacts.json]svml script <file.svml> [--out narrative.json]svml plan <file.svml> [--out plan.json]svml fmt <file.svml> [--check|--write|--out formatted.svml]svml estimate <file.svml> [--out alignment.json] [--fps 30]svml canvas <file.svml> [--out canvas.json]- svml timeline <file.svml> --evidence alignment.json [--out timeline.json] [--lock svml.lock] [--artifacts artifacts.json]- svml compile <file.svml> --evidence alignment.json --out index.html [--lock svml.lock] [--artifacts artifacts.json]+ svml timeline <file.svml> [--out timeline.json] [--lock svml.lock] [--artifacts artifacts.json]+ svml compile <file.svml> --out index.html [--lock svml.lock] [--artifacts artifacts.json]svml render <index.html> --out video.mp4The current runtime is deterministic-only: it has no generation provider host.diff --git a/src/compiler.ts b/src/compiler.tsindex 2b3b5d8d..1f753ff8 100644--- a/src/compiler.ts+++ b/src/compiler.ts@@ -2,6 +2,7 @@ import { copyFile, readFile, writeFile, mkdir } from "node:fs/promises";import { basename, dirname, resolve } from "node:path";import { fileURLToPath, pathToFileURL } from "node:url";import { fail } from "./diagnostics.js";+import { expandComposites } from "./composite.js";import {resolveArtifactFile,verifyArtifactExecutionDigests,@@ -13,13 +14,16 @@ import {loadKernels,type KernelManifest,} from "./kernel.js";-import { createLock, verifyLockFile, type SvmlLock } from "./lock.js";+import { createLock, freezeLock, verifyLockFile, type SvmlLock } from "./lock.js";import type {AlignmentEvidence,AttributeValue,+ ExactSemanticMap,+ LocatedScript,PlanIR,SourceDocument,SourceElement,+ TemporalBasisProduction,} from "./model.js";import { buildPlan, topologicalInstances } from "./plan.js";import type {@@ -29,24 +33,34 @@ import type {} from "./runtime-contract.js";import { projectKernelIsolated } from "./sandbox.js";import { validateKernelProjection } from "./runtime-validation.js";-import { locateScript } from "./script/locate.js";+import { bindSemanticMap, locateScript } from "./script/locate.js";import { estimateAlignment, type EstimateOptions } from "./script/estimate.js";import { parseScript } from "./script/parse.js";import { loadSourceClosure } from "./source/modules.js";import { childElements, textContent } from "./source/query.js";import { applyStyleSheets } from "./source/styles.js";-import { sha256 } from "./util.js";+import { sha256, stableJson } from "./util.js";+import {+ createProgramBasis,+ semanticMapFromAlignment,+ validateSemanticMap,+ validateTemporalBasisProduction,+} from "./temporal.js";export type Compilation = {document: SourceDocument;narrative: ReturnType<typeof parseScript>;plan: PlanIR;- located: ReturnType<typeof locateScript>;+ basis: TemporalBasisProduction;+ semanticMap: ExactSemanticMap;+ located: LocatedScript;html: string;target: HyperframesDocument;outputFile?: string;projections: Map<string, KernelProjection>;lock: SvmlLock;+ sourceClosureVerified: boolean;+ temporalEvidenceVerified: boolean;lockVerified: boolean;artifactsVerified: boolean;artifactManifestHash?: string;@@ -97,6 +111,43 @@ function isHyperframesDocument(value: unknown): value is HyperframesDocument {&& (value as HyperframesDocument).contract === "svml.hyperframes-document.v1";}+function validateTrackOutput(+ instanceId: string,+ portName: string,+ value: unknown,+ durationFrames: number,+): void {+ if (!value || typeof value !== "object") {+ fail("track_output_shape", `Track output "${instanceId}.${portName}" is not an object.`);+ }+ const track = value as { visuals?: unknown; audios?: unknown; styles?: unknown };+ if (!Array.isArray(track.visuals) || !Array.isArray(track.audios) || !Array.isArray(track.styles)) {+ fail(+ "track_output_shape",+ `Track output "${instanceId}.${portName}" requires flat visuals, audios and styles arrays.`,+ );+ }+ for (const contribution of [...track.visuals, ...track.audios] as Array<{+ id?: unknown;+ startFrame?: unknown;+ endFrameExclusive?: unknown;+ }>) {+ if (+ typeof contribution.id !== "string"+ || !Number.isInteger(contribution.startFrame)+ || !Number.isInteger(contribution.endFrameExclusive)+ || (contribution.startFrame as number) < 0+ || (contribution.endFrameExclusive as number) > durationFrames+ || (contribution.endFrameExclusive as number) <= (contribution.startFrame as number)+ ) {+ fail(+ "track_contribution_window",+ `Track "${instanceId}" contribution "${String(contribution.id)}" has invalid Program range ${String(contribution.startFrame)}..${String(contribution.endFrameExclusive)} for duration ${durationFrames}.`,+ );+ }+ }+}+async function stageDocumentAssets(document: HyperframesDocument,outputFile: string,@@ -136,7 +187,8 @@ export async function checkSource(file: string): Promise<{}> {const closure = await loadSourceClosure(file);const kernels = await loadKernels(closure);- const document = await applyStyleSheets(closure, kernels);+ const styled = await applyStyleSheets(closure, kernels);+ const document = await applyStyleSheets(expandComposites(styled, kernels), kernels);const narrative = parseScript(document.file,document.scriptSource,@@ -160,29 +212,44 @@ export async function estimateSource(options: EstimateOptions = {},): Promise<{evidence: AlignmentEvidence;- located: ReturnType<typeof locateScript>;+ basis: ReturnType<typeof createProgramBasis>;+ map: ReturnType<typeof semanticMapFromAlignment>;+ located: LocatedScript;}> {const checked = await checkSource(file);const evidence = estimateAlignment(checked.narrative, options);+ const basis = createProgramBasis({+ fps: evidence.fps,+ durationFrames: Math.round(evidence.durationSec * evidence.fps),+ outcomeDigest: sha256(JSON.stringify({+ kind: "svml.estimated-program-basis.v1",+ evidence,+ })),+ });+ const map = semanticMapFromAlignment(+ checked.narrative,+ basis,+ evidence,+ { precision: "estimated" },+ );return {evidence,- located: locateScript(checked.narrative, evidence),+ basis,+ map,+ located: bindSemanticMap(checked.narrative, basis, map),};}export async function compileSource(args: {file: string;- evidenceFile: string;outputFile?: string;lockFile?: string;artifactsFile?: string;}): Promise<Compilation> {const checked = await checkSource(args.file);- if (args.lockFile) await verifyLockFile(resolve(args.lockFile), checked.lock);- const evidence = JSON.parse(- await readFile(resolve(args.evidenceFile), "utf8"),- ) as AlignmentEvidence;- const located = locateScript(checked.narrative, evidence);+ const suppliedLock = args.lockFile+ ? await verifyLockFile(resolve(args.lockFile), checked.lock)+ : undefined;const kernels = await loadKernels(checked.document);const artifacts = await resolveArtifactFile(args.artifactsFile, checked.plan, kernels);await attachExecutionDigests(checked.plan, kernels, artifacts.outputDigests);@@ -192,35 +259,57 @@ export async function compileSource(args: {if (!root) fail("runtime_root_missing", `Root instance "${checked.plan.root}" is missing.`);const rootElement = elements.get(root.id);if (!rootElement) fail("runtime_root_element", `Root element "${root.id}" is missing.`);+ const rootManifest = kernels.get(root.kernel);+ if (!rootManifest) fail("runtime_kernel_missing", `Kernel "${root.kernel}" is missing.`);+ const basisPort = rootManifest.ports.find((port) =>+ port.direction === "input" && port.type.split("|").includes("TemporalBasisProduction"));+ const semanticPort = rootManifest.ports.find((port) =>+ port.direction === "input" && port.type.split("|").includes("ExactSemanticMap"));+ if (!basisPort || !semanticPort) {+ fail("runtime_composition_contract", "Composition does not expose basis/map input ports.");+ }const width = Number(root.attributes.width ?? 1080);const height = Number(root.attributes.height ?? 1920);if (!Number.isFinite(width) || !Number.isFinite(height)) {fail("runtime_dimensions", "Film width and height must be finite numbers.");}- const values = new Map<string, RuntimeValue>(checked.plan.values.map((value) => {+ const valueEntries = await Promise.all(checked.plan.values.map(async (value) => {const absoluteSource = value.source? pathToFileURL(resolve(dirname(value.module), value.source)).href: undefined;+ const data = value.type === "AlignmentEvidence" && value.source+ ? JSON.parse(await readFile(resolve(dirname(value.module), value.source), "utf8"))+ : undefined;return [value.id,- { ...value, ...(absoluteSource ? { absoluteSource } : {}) },- ];+ {+ ...value,+ ...(absoluteSource ? { absoluteSource } : {}),+ ...(data !== undefined ? { data } : {}),+ },+ ] as const;}));+ const values = new Map<string, RuntimeValue>(valueEntries);const results = new Map<string, KernelProjection>();+ let located: LocatedScript | undefined;const resolvePath = (path: string): unknown => {if (path === "script") {- return { narrative: checked.narrative, located };+ return checked.narrative;}if (path.startsWith("script.selection.")) {+ if (!located) fail("runtime_temporal_unbound", `Temporal reference "${path}" is not bound yet.`);return located.selections[path.slice("script.selection.".length)];}if (path.startsWith("script.moment.")) {+ if (!located) fail("runtime_temporal_unbound", `Temporal reference "${path}" is not bound yet.`);return located.moments[path.slice("script.moment.".length)];}if (path.startsWith("script.segment.")) {- return located.segments[path.slice("script.segment.".length)];+ return checked.narrative.segments.find(+ (segment) => segment.id === path.slice("script.segment.".length),+ );}const declarationIds = [...checked.plan.values.map((value) => value.id),@@ -253,7 +342,7 @@ export async function compileSource(args: {};};- for (const instance of topologicalInstances(checked.plan)) {+ const executeInstance = async (instance: PlanIR["instances"][number]): Promise<void> => {const manifest = kernels.get(instance.kernel);if (!manifest) fail("runtime_kernel_missing", `Kernel "${instance.kernel}" is missing.`);if (manifest.abiVersion !== "1") {@@ -268,7 +357,7 @@ export async function compileSource(args: {fail("artifact_projection_missing", `Capability "${instance.id}" has no resolved outputs.`);}results.set(instance.id, projection);- continue;+ return;}const sourceElement = elements.get(instance.id);if (!sourceElement) fail("runtime_instance_element", `Element "${instance.id}" is missing.`);@@ -285,19 +374,73 @@ export async function compileSource(args: {const context = {instance,element: resolvedElement,- narrative: checked.narrative,- located,- alignment: evidence,- plan: checked.plan,temporalContracts: kernelTemporalContracts(manifest),- fps: evidence.fps,+ ...(located ? {+ program: {+ basisDigest: located.basisDigest,+ fps: located.fps,+ durationFrames: located.durationFrames,+ durationSec: located.durationSec,+ },+ } : {}),+ fps: located?.fps ?? Number(root.attributes.fps ?? 30),width,height,};- results.set(- instance.id,- validateKernelProjection(await projectKernelIsolated(manifest, context)),- );+ const projection = validateKernelProjection(await projectKernelIsolated(manifest, context));+ if (located) {+ for (const port of manifest.ports.filter((candidate) =>+ candidate.direction === "output" && candidate.type === "Track")) {+ validateTrackOutput(instance.id, port.name, projection.outputs[port.name], located.durationFrames);+ }+ }+ results.set(instance.id, projection);+ };++ const ordered = topologicalInstances(checked.plan);+ const instanceIds = ordered.map((instance) => instance.id)+ .sort((left, right) => right.length - left.length);+ const producerOf = (value: AttributeValue | undefined, label: string): string => {+ if (!value || typeof value !== "object" || value.kind !== "reference") {+ fail("runtime_composition_reference", `Composition ${label} must be an instance output reference.`);+ }+ const owner = instanceIds.find((id) => value.path === id || value.path.startsWith(`${id}.`));+ if (!owner) fail("runtime_composition_reference", `Composition ${label} has no producer.`);+ return owner;+ };+ const preBinding = new Set<string>();+ const addPreBinding = (id: string): void => {+ if (preBinding.has(id)) return;+ preBinding.add(id);+ const instance = ordered.find((candidate) => candidate.id === id);+ for (const dependency of instance?.dependencies ?? []) {+ if (ordered.some((candidate) => candidate.id === dependency)) addPreBinding(dependency);+ }+ };+ addPreBinding(producerOf(rootElement.attributes[basisPort.name], basisPort.name));+ addPreBinding(producerOf(rootElement.attributes[semanticPort.name], semanticPort.name));+ for (const instance of ordered) {+ if (preBinding.has(instance.id)) await executeInstance(instance);+ }++ const production = resolveValue(+ rootElement.attributes[basisPort.name],+ ) as TemporalBasisProduction | undefined;+ const semanticMap = resolveValue(+ rootElement.attributes[semanticPort.name],+ ) as ExactSemanticMap | undefined;+ if (production?.contract !== "svml.temporal-basis-production.v1") {+ fail("runtime_basis_missing", "Composition root must select one TemporalBasisProduction.");+ }+ if (semanticMap?.contract !== "svml.exact-semantic-map.v1") {+ fail("runtime_semantic_map_missing", "Composition root must select one ExactSemanticMap.");+ }+ validateTemporalBasisProduction(production);+ validateSemanticMap(checked.narrative, production.basis, semanticMap);+ located = locateScript(checked.narrative, production.basis, semanticMap);++ for (const instance of ordered) {+ if (!preBinding.has(instance.id)) await executeInstance(instance);}const rootProjection = results.get(checked.plan.root);@@ -308,6 +451,78 @@ export async function compileSource(args: {`Root Kernel "${root.kernel}" did not output svml.hyperframes-document.v1.`,);}+ const sourceDigests = new Map<string, string>();+ for (const value of values.values()) {+ const source = value.absoluteSource ?? value.source;+ if (source && value.contentDigest) sourceDigests.set(source, value.contentDigest);+ }+ const collectArtifactSources = (value: unknown): void => {+ if (Array.isArray(value)) {+ value.forEach(collectArtifactSources);+ return;+ }+ if (!value || typeof value !== "object") return;+ const record = value as Record<string, unknown>;+ if (typeof record.source === "string" && typeof record.contentDigest === "string") {+ sourceDigests.set(record.source, record.contentDigest);+ }+ Object.values(record).forEach(collectArtifactSources);+ };+ results.forEach((projection) => collectArtifactSources(projection.outputs));+ const canonicalTarget = {+ ...documentOutput,+ visuals: documentOutput.visuals.map((fragment) => ({+ ...fragment,+ ...(fragment.source+ ? { source: sourceDigests.get(fragment.source) ?? fragment.source }+ : {}),+ })),+ audios: documentOutput.audios.map((fragment) => ({+ ...fragment,+ source: sourceDigests.get(fragment.source) ?? fragment.source,+ })),+ };+ const planDigest = sha256(stableJson({+ root: checked.plan.root,+ instances: checked.plan.instances.map((instance) => ({+ identity: instance.identity,+ executionDigest: instance.executionDigest,+ })),+ edges: checked.plan.edges,+ expansions: checked.plan.expansions,+ }));+ const frozenLock = freezeLock(checked.lock, {+ planDigest,+ ...(artifacts.manifestHash ? { artifactManifestHash: artifacts.manifestHash } : {}),+ basisDigest: production.basis.basisDigest,+ productionDigest: production.productionDigest,+ semanticIndexDigest: semanticMap.semanticIndexDigest,+ semanticMapDigest: semanticMap.mapDigest,+ locatorDigest: semanticMap.locatorDigest,+ evidenceDigests: semanticMap.evidenceDigests,+ quantizationPolicy: semanticMap.quantizationPolicy,+ anchors: semanticMap.anchors.map((anchor) => ({+ identity: anchor.identity,+ frame: anchor.point.frame,+ quality: anchor.quality,+ })),+ programSpace: {+ width: documentOutput.width,+ height: documentOutput.height,+ fps: documentOutput.fps,+ durationFrames: documentOutput.durationFrames,+ },+ targetDigest: sha256(stableJson(canonicalTarget)),+ htmlDigest: sha256(emitHyperframesHtml(canonicalTarget)),+ }, artifacts.verified);+ const temporalEvidenceVerified = !!suppliedLock?.execution+ && stableJson(suppliedLock) === stableJson(frozenLock);+ if (suppliedLock?.execution && !temporalEvidenceVerified) {+ fail(+ "lock_execution_mismatch",+ `${resolve(args.lockFile!)} does not match the compiled Basis, SemanticMap or HyperFrames target.`,+ );+ }const outputFile = args.outputFile ? resolve(args.outputFile) : undefined;const materializedDocument = outputFile? await stageDocumentAssets(documentOutput, outputFile)@@ -321,20 +536,24 @@ export async function compileSource(args: {document: checked.document,narrative: checked.narrative,plan: checked.plan,+ basis: production,+ semanticMap,located,html,target: materializedDocument,...(outputFile ? { outputFile } : {}),projections: results,- lock: checked.lock,- lockVerified: args.lockFile !== undefined,+ lock: frozenLock,+ sourceClosureVerified: args.lockFile !== undefined,+ temporalEvidenceVerified,+ lockVerified: temporalEvidenceVerified,artifactsVerified: artifacts.verified,...(artifacts.manifestHash? { artifactManifestHash: artifacts.manifestHash }: {}),reproducible:- args.lockFile !== undefined- && checked.lock.reproducible+ temporalEvidenceVerified+ && frozenLock.reproducible&& artifacts.verified,};}diff --git a/src/composite.ts b/src/composite.tsnew file mode 100644index 00000000..b03e6e5c--- /dev/null+++ b/src/composite.ts@@ -0,0 +1,285 @@+import { pathToFileURL } from "node:url";+import { fail } from "./diagnostics.js";+import type { KernelManifest, KernelRegistry } from "./kernel.js";+import { validateKernelShape } from "./plan.js";+import type {+ AttributeValue,+ Reference,+ SourceDocument,+ SourceElement,+ SourceNode,+} from "./model.js";+import { childElements } from "./source/query.js";+import { sha256, stableJson } from "./util.js";++type LoopBinding = {+ name: string;+ value: SourceElement;+};++function reference(value: AttributeValue | undefined): Reference | undefined {+ return value && typeof value === "object" && value.kind === "reference"+ ? value+ : undefined;+}++function canonicalNode(node: SourceNode): unknown {+ if (node.kind === "text") return { kind: "text", value: node.value };+ return {+ kind: "element",+ name: node.name,+ attributes: node.attributes,+ children: node.children.map(canonicalNode),+ };+}++function localReference(path: string, outerId: string, localIds: Set<string>): Reference | undefined {+ const owner = [...localIds]+ .sort((left, right) => right.length - left.length)+ .find((candidate) => path === candidate || path.startsWith(`${candidate}.`));+ if (!owner) return undefined;+ return {+ kind: "reference",+ path: `${outerId}::${owner}${path.slice(owner.length)}`,+ };+}++function evaluate(+ value: AttributeValue,+ args: {+ outer: SourceElement;+ outerId: string;+ localIds: Set<string>;+ loop?: LoopBinding;+ },+): AttributeValue | undefined {+ const ref = reference(value);+ if (!ref) return value;+ const computedScript = /^ports\.([A-Za-z_][A-Za-z0-9_.:-]*)\.segment\[([A-Za-z_][A-Za-z0-9_.:-]*)\.id\]$/u+ .exec(ref.path);+ if (computedScript) {+ if (!args.loop || args.loop.name !== computedScript[2]) {+ fail("composite_expression", `Composite expression "${ref.path}" has no matching loop.`);+ }+ const script = reference(args.outer.attributes[computedScript[1]!]);+ const id = args.loop.value.attributes.id;+ if (!script || typeof id !== "string") {+ fail("composite_expression", `Composite expression "${ref.path}" requires Script and child id.`);+ }+ return { kind: "reference", path: `${script.path}.segment.${id}` };+ }+ const [head, ...tail] = ref.path.split(".");+ if (head === "params" || head === "ports") {+ const name = tail.join(".");+ return args.outer.attributes[name];+ }+ if (args.loop && head === args.loop.name) {+ return args.loop.value.attributes[tail.join(".")];+ }+ return localReference(ref.path, args.outerId, args.localIds) ?? ref;+}++function instantiateNode(+ template: SourceNode,+ args: {+ outer: SourceElement;+ outerId: string;+ localIds: Set<string>;+ aliases: ReadonlyMap<string, string>;+ expansionDigest: string;+ definitionFile: string;+ componentRoot: boolean;+ loop?: LoopBinding;+ },+): SourceNode[] {+ if (template.kind === "text") return template.value.trim() ? [{ ...template }] : [];+ if (template.name === "for") {+ const each = reference(template.attributes.each)?.path;+ const alias = template.attributes.as;+ const match = /^children\.([A-Za-z_][A-Za-z0-9_.:-]*)$/u.exec(each ?? "");+ if (!match || typeof alias !== "string") {+ fail("composite_for", "Composite <for> requires each={children.name} and a quoted as name.");+ }+ return childElements(args.outer, match[1]).flatMap((value) =>+ template.children.flatMap((child) => instantiateNode(child, {+ ...args,+ componentRoot: false,+ loop: { name: alias, value },+ })));+ }+ const name = args.componentRoot+ ? args.aliases.get(template.name) ?? template.name+ : template.name;+ const attributes = Object.fromEntries(Object.entries(template.attributes).flatMap(([key, value]) => {+ const evaluated = evaluate(value, args);+ if (evaluated === undefined) return [];+ if (key === "id" && args.componentRoot) {+ if (typeof evaluated !== "string") fail("composite_id", "Composite internal id must be literal.");+ return [[key, `${args.outerId}::${evaluated}`]];+ }+ return [[key, evaluated]];+ }));+ return [{+ ...template,+ name,+ attributes,+ children: template.children.flatMap((child) => instantiateNode(child, {+ ...args,+ componentRoot: false,+ })),+ origin: {+ file: args.outer.origin?.file ?? "",+ ...(typeof attributes.id === "string" ? { localId: attributes.id } : {}),+ expansionDigest: args.expansionDigest,+ callSite: {+ file: args.outer.origin?.file ?? "",+ start: args.outer.start,+ end: args.outer.end,+ },+ definitionSite: {+ file: args.definitionFile,+ start: template.start,+ end: template.end,+ },+ },+ }];+}++function rewriteReference(value: AttributeValue, aliases: ReadonlyMap<string, string>): AttributeValue {+ const ref = reference(value);+ if (!ref) return value;+ let path = ref.path;+ for (let pass = 0; pass < 32; pass += 1) {+ const owner = [...aliases.keys()]+ .sort((left, right) => right.length - left.length)+ .find((candidate) => path === candidate || path.startsWith(`${candidate}.`));+ if (!owner) break;+ const next = `${aliases.get(owner)!}${path.slice(owner.length)}`;+ if (next === path) break;+ path = next;+ }+ return { kind: "reference", path };+}++function rewriteNode(node: SourceNode, aliases: ReadonlyMap<string, string>): SourceNode {+ if (node.kind === "text") return node;+ return {+ ...node,+ attributes: Object.fromEntries(Object.entries(node.attributes)+ .map(([name, value]) => [name, rewriteReference(value, aliases)])),+ children: node.children.map((child) => rewriteNode(child, aliases)),+ };+}++function expandOne(+ element: SourceElement,+ manifest: KernelManifest,+): {+ nodes: SourceElement[];+ aliases: Record<string, string>;+ expansion: NonNullable<SourceDocument["expansions"]>[number];+} {+ const outerId = element.attributes.id;+ if (typeof outerId !== "string") fail("composite_id", `<${manifest.name}> requires a stable id.`);+ validateKernelShape(element, manifest);+ const compose = manifest.compose;+ if (!compose) fail("composite_shape", `Composite "${manifest.name}" has no compose body.`);+ const templates = childElements(compose).filter((child) => child.name !== "export");+ const localIds = new Set(templates.flatMap((template) => {+ const id = template.attributes.id;+ return typeof id === "string" ? [id] : [];+ }));+ const expansionDigest = sha256(stableJson({+ contract: "svml.composite-expansion.v1",+ manifestHash: manifest.sourceHash,+ instanceId: outerId,+ attributes: element.attributes,+ children: element.children.map(canonicalNode),+ }));+ const importAliases = new Map((manifest.compositeImports ?? []).map((item) => {+ if (!item.kernelName) fail("composite_import", `Composite import "${item.alias}" is unresolved.`);+ return [item.alias, item.kernelName];+ }));+ const nodes = templates.flatMap((template) => instantiateNode(template, {+ outer: element,+ outerId,+ localIds,+ aliases: importAliases,+ expansionDigest,+ definitionFile: manifest.sourcePath,+ componentRoot: true,+ })) as SourceElement[];+ const exports: Record<string, string> = {};+ for (const item of childElements(compose, "export")) {+ const port = item.attributes.port;+ const value = item.attributes.value;+ if (typeof port !== "string" || !reference(value)) {+ fail("composite_export", "Composite <export> requires port and reference value attributes.");+ }+ const evaluated = evaluate(value!, { outer: element, outerId, localIds });+ const target = reference(evaluated)?.path;+ if (!target || exports[port]) fail("composite_export", `Invalid or duplicate export "${port}".`);+ exports[port] = target;+ }+ for (const port of manifest.ports.filter((candidate) => candidate.direction === "output")) {+ if (!exports[port.name]) {+ fail("composite_export_missing", `Composite "${manifest.name}" does not export "${port.name}".`);+ }+ }+ return {+ nodes,+ aliases: Object.fromEntries(Object.entries(exports).map(([port, target]) => [+ `${outerId}.${port}`,+ target,+ ])),+ expansion: {+ instanceId: outerId,+ component: manifest.name,+ manifestHash: manifest.sourceHash,+ expansionDigest,+ internalIds: nodes.map((node) => String(node.attributes.id)),+ exports,+ },+ };+}++export function expandComposites(+ document: SourceDocument,+ kernels: KernelRegistry,+): SourceDocument {+ let children = document.root.children;+ const aliases = new Map<string, string>();+ const expansions: NonNullable<SourceDocument["expansions"]> = [];+ for (let pass = 0; pass < 32; pass += 1) {+ let changed = false;+ const next = children.flatMap((node): SourceNode[] => {+ if (node.kind !== "element") return [node];+ const manifest = kernels.get(node.name);+ if (manifest?.profile !== "composite-v1") return [node];+ changed = true;+ const expanded = expandOne(node, manifest);+ Object.entries(expanded.aliases).forEach(([from, to]) => aliases.set(from, to));+ expansions.push(expanded.expansion);+ return expanded.nodes;+ });+ children = next.map((node) => rewriteNode(node, aliases));+ if (!changed) {+ const compositeModules = [...kernels.values()]+ .filter((manifest) => manifest.profile === "composite-v1")+ .map((manifest) => ({+ kind: "svk" as const,+ uri: pathToFileURL(manifest.sourcePath).href,+ contentHash: manifest.sourceHash,+ dependencies: (manifest.compositeImports ?? []).map((item) =>+ pathToFileURL(item.sourcePath).href),+ }));+ return {+ ...document,+ root: { ...document.root, children },+ modules: [...(document.modules ?? []), ...compositeModules],+ expansions,+ };+ }+ }+ fail("composite_expansion_limit", "Composite expansion exceeded 32 finite passes.");+}diff --git a/src/digests.ts b/src/digests.tsindex dd31e506..23d5a94a 100644--- a/src/digests.ts+++ b/src/digests.ts@@ -108,6 +108,7 @@ export async function attachExecutionDigests(abi: manifest.abiVersion,manifest: manifest.sourceHash,implementation: manifest.implementationHash,+ expansion: instance.expansionDigest,attributes: Object.fromEntries(Object.entries(instance.attributes).filter(([name]) => !["id", "class"].includes(name)).map(([name, value]) => [name, canonicalValue(value)])),diff --git a/src/hyperframes.ts b/src/hyperframes.tsindex fdd2d8b7..bf1c15b5 100644--- a/src/hyperframes.ts+++ b/src/hyperframes.ts@@ -96,8 +96,6 @@ export function emitHyperframesHtml(args: {}): string {const orderedVisuals = [...args.visuals].sort((left, right) =>left.z - right.z- || (left.layer ?? 0) - (right.layer ?? 0)- || left.startFrame - right.startFrame|| left.id.localeCompare(right.id));const visuals = orderedVisuals.map((fragment, index) =>visualHtml(fragment, args.fps, index));diff --git a/src/kernel.ts b/src/kernel.tsindex 7f34a0ec..f7bf3022 100644--- a/src/kernel.ts+++ b/src/kernel.ts@@ -18,7 +18,7 @@ export type KernelPort = {export type KernelParameter = {name: string;styleName: string;- type: "string" | "number" | "boolean";+ type: string;defaultValue?: AttributeValue;};@@ -46,12 +46,14 @@ export type KernelManifest = {sourceHash: string;implementationPath?: string;implementationHash: string;- profile: "isolated-projector-v1" | "capability-v1";+ profile: "isolated-projector-v1" | "capability-v1" | "composite-v1";capability?: string;permissions: string[];ports: KernelPort[];parameters: KernelParameter[];children: KernelChildSchema[];+ compositeImports?: Array<{ alias: string; sourcePath: string; kernelName?: string }>;+ compose?: SourceElement;};export type KernelRegistry = Map<string, KernelManifest>;@@ -124,7 +126,14 @@ function booleanAttribute(function parseParameter(sourcePath: string, element: SourceElement): KernelParameter {const name = stringAttribute(element, "name");const type = stringAttribute(element, "type");- if (!name || !type || !["string", "number", "boolean"].includes(type)) {+ if (+ !name+ || !type+ || !(+ ["string", "number", "boolean", "duration"].includes(type)+ || /^enum\([A-Za-z_][A-Za-z0-9_.:-]*(?:,[A-Za-z_][A-Za-z0-9_.:-]*)+\)$/u.test(type)+ )+ ) {fail("kernel_manifest_parameter", `${sourcePath} contains an invalid <param>.`);}let defaultValue = element.attributes.default;@@ -141,6 +150,22 @@ function parseParameter(sourcePath: string, element: SourceElement): KernelParam`${sourcePath} parameter ${name} has a non-boolean default.`,);}+ if (type === "duration" && (+ typeof defaultValue !== "string"+ || !/^[+-]?\d+(?:\.\d+)?(?:ms|s)$/u.test(defaultValue)+ )) {+ fail(+ "kernel_manifest_parameter_default",+ `${sourcePath} parameter ${name} has an invalid duration default.`,+ );+ }+ const enumValues = /^enum\((.+)\)$/u.exec(type)?.[1]?.split(",");+ if (enumValues && (typeof defaultValue !== "string" || !enumValues.includes(defaultValue))) {+ fail(+ "kernel_manifest_parameter_default",+ `${sourcePath} parameter ${name} has an invalid enum default.`,+ );+ }if (type === "string" && typeof defaultValue !== "string") {defaultValue = String(defaultValue);}@@ -212,7 +237,7 @@ function parseManifest(sourcePath: string, source: string): KernelManifest {if (!name || !/^[A-Za-z_][A-Za-z0-9_.:-]*$/u.test(name)) {fail("kernel_manifest_name", `${sourcePath} has an invalid or missing Kernel name.`);}- if (profile !== "isolated-projector-v1" && profile !== "capability-v1") {+ if (!["isolated-projector-v1", "capability-v1", "composite-v1"].includes(profile ?? "")) {fail("kernel_manifest_profile", `${sourcePath} has unsupported profile "${profile ?? ""}".`);}if (profile === "isolated-projector-v1" && !implementation) {@@ -262,7 +287,28 @@ function parseManifest(sourcePath: string, source: string): KernelManifest {if (new Set(children.map((child) => child.name)).size !== children.length) {fail("kernel_manifest_child_duplicate", `${sourcePath} repeats a root child schema.`);}- const known = new Set(["port", "param", "child"]);+ const compositeImports = childElements(root, "import").map((element) => ({+ alias: stringAttribute(element, "as") ?? "",+ sourcePath: resolve(dirname(sourcePath), stringAttribute(element, "from") ?? ""),+ }));+ const compose = childElements(root, "compose")[0];+ if (profile === "composite-v1") {+ if (implementation || capability || !compose || !compositeImports.length) {+ fail(+ "kernel_composite_shape",+ `${sourcePath} composite-v1 requires imports and one <compose>, without implementation/capability.`,+ );+ }+ if (compositeImports.some((item) => !item.alias)) {+ fail("kernel_composite_import", `${sourcePath} Composite imports require explicit aliases.`);+ }+ }+ const known = new Set([+ "port",+ "param",+ "child",+ ...(profile === "composite-v1" ? ["import", "compose"] : []),+ ]);const unknown = childElements(root).find((element) => !known.has(element.name));if (unknown) {fail("kernel_manifest_element", `${sourcePath} contains unknown <${unknown.name}>.`);@@ -275,7 +321,7 @@ function parseManifest(sourcePath: string, source: string): KernelManifest {sourceHash: sha256(source),...(implementation ? { implementationPath: resolve(dirname(sourcePath), implementation) } : {}),implementationHash: "",- profile,+ profile: profile as KernelManifest["profile"],...(capability ? { capability } : {}),permissions: (stringAttribute(root, "permissions", "") ?? "").split(/[\s,]+/u)@@ -283,27 +329,70 @@ function parseManifest(sourcePath: string, source: string): KernelManifest {ports,parameters,children,+ ...(profile === "composite-v1" ? { compositeImports, compose } : {}),};}export async function loadKernels(document: SourceDocument): Promise<KernelRegistry> {const registry: KernelRegistry = new Map();const imports = childElements(document.root, "import");- for (const element of imports) {- const from = attributeString(element, "from");- if (!from.endsWith(".svk")) continue;- const sourcePath = resolve(dirname(document.file), from);+ const loading = new Set<string>();+ const definitions = new Map<string, KernelManifest>();+ const load = async (sourcePath: string): Promise<KernelManifest> => {+ sourcePath = resolve(sourcePath);+ if (loading.has(sourcePath)) {+ fail("kernel_import_cycle", `Composite Kernel import cycle at ${sourcePath}.`);+ }+ const byPath = definitions.get(sourcePath);+ if (byPath) return byPath;+ loading.add(sourcePath);const manifest = parseManifest(sourcePath, await readFile(sourcePath, "utf8"));- manifest.implementationHash = manifest.implementationPath- ? sha256(await readFile(manifest.implementationPath))- : sha256(`capability\0${manifest.capability ?? manifest.name}`);- if (registry.has(manifest.name)) {+ if (manifest.profile === "composite-v1") {+ for (const imported of manifest.compositeImports ?? []) {+ const nested = await load(imported.sourcePath);+ imported.kernelName = nested.name;+ }+ manifest.implementationHash = sha256(`composite\0${manifest.sourceHash}`);+ } else {+ manifest.implementationHash = manifest.implementationPath+ ? sha256(await readFile(manifest.implementationPath))+ : sha256(`capability\0${manifest.capability ?? manifest.name}`);+ }+ definitions.set(sourcePath, manifest);+ loading.delete(sourcePath);+ return manifest;+ };+ const bind = (binding: string, manifest: KernelManifest): void => {+ if (!/^[A-Za-z_][A-Za-z0-9_.:-]*$/u.test(binding)) {+ fail("kernel_import_alias", `Kernel import alias "${binding}" is invalid.`);+ }+ const existing = registry.get(binding);+ if (existing && existing.sourcePath !== manifest.sourcePath) {fail("kernel_duplicate_binding",- `Kernel "${manifest.name}" is imported more than once; use explicit aliases once namespace syntax is enabled.`,+ `Kernel binding "${binding}" is defined by both ${existing.sourcePath} and ${manifest.sourcePath}.`,);}- registry.set(manifest.name, manifest);+ registry.set(binding, binding === manifest.name ? manifest : { ...manifest, name: binding });+ };+ const bindCompositeDependencies = (manifest: KernelManifest): void => {+ for (const imported of manifest.compositeImports ?? []) {+ const nested = definitions.get(resolve(imported.sourcePath));+ if (!nested) fail("kernel_import_resolution", `Kernel import ${imported.sourcePath} is unresolved.`);+ bind(nested.name, nested);+ bindCompositeDependencies(nested);+ }+ };+ for (const element of imports) {+ const from = attributeString(element, "from");+ if (!from.endsWith(".svk")) continue;+ const manifest = await load(resolve(dirname(document.file), from));+ bindCompositeDependencies(manifest);+ const alias = element.attributes.as;+ if (alias !== undefined && typeof alias !== "string") {+ fail("kernel_import_alias", `${document.file} Kernel import alias must be a string.`);+ }+ bind(alias ?? manifest.name, manifest);}return registry;}diff --git a/src/lock.ts b/src/lock.tsindex d2d04450..2e9b870b 100644--- a/src/lock.ts+++ b/src/lock.ts@@ -23,6 +23,31 @@ export type SvmlLock = {};reproducible: boolean;modules: NonNullable<SourceDocument["modules"]>;+ expansions: PlanIR["expansions"];+ execution?: {+ planDigest: string;+ artifactManifestHash?: string;+ basisDigest: string;+ productionDigest: string;+ semanticIndexDigest: string;+ semanticMapDigest: string;+ locatorDigest: string;+ evidenceDigests: string[];+ quantizationPolicy: string;+ anchors: Array<{+ identity: string;+ frame: number;+ quality: string;+ }>;+ programSpace: {+ width: number;+ height: number;+ fps: number;+ durationFrames: number;+ };+ targetDigest: string;+ htmlDigest: string;+ };materials: Array<{identity: string;type: string;@@ -89,12 +114,13 @@ export function createLock(name: "hyperframes",version: "0.7.84",},- reproducible: materials.every((material) => material.status === "content"),+ reproducible: false,modules: (document.modules ?? []).map((module) => ({...module,uri: lockUri(module.uri),dependencies: module.dependencies.map(lockUri),})).sort((left, right) => left.uri.localeCompare(right.uri)),+ expansions: plan.expansions,materials,kernels: manifests.map((manifest) => ({name: manifest.name,@@ -112,10 +138,24 @@ export function createLock(};}+export function freezeLock(+ sourceLock: SvmlLock,+ execution: NonNullable<SvmlLock["execution"]>,+ artifactsVerified: boolean,+): SvmlLock {+ return {+ ...sourceLock,+ reproducible:+ artifactsVerified+ && sourceLock.materials.every((material) => material.status === "content"),+ execution,+ };+}+export async function verifyLockFile(file: string,expected: SvmlLock,-): Promise<void> {+): Promise<SvmlLock> {let actual: unknown;try {actual = JSON.parse(await readFile(file, "utf8"));@@ -128,10 +168,17 @@ export async function verifyLockFile(if (!actual || typeof actual !== "object" || (actual as { contract?: string }).contract !== "svml.lock.v1") {fail("lock_contract", `${file} is not an svml.lock.v1 document.`);}- if (stableJson(actual) !== stableJson(expected)) {+ const parsed = actual as SvmlLock;+ const sourceOnly: SvmlLock = {+ ...parsed,+ reproducible: false,+ };+ delete sourceOnly.execution;+ if (stableJson(sourceOnly) !== stableJson(expected)) {fail("lock_mismatch",`${file} does not match the resolved source closure; regenerate it with "svml lock".`,);}+ return parsed;}diff --git a/src/model.ts b/src/model.tsindex 1e783e22..27946dee 100644--- a/src/model.ts+++ b/src/model.ts@@ -23,6 +23,9 @@ export type SourceElement = {origin?: {file: string;localId?: string;+ expansionDigest?: string;+ callSite?: { file: string; start: number; end: number };+ definitionSite?: { file: string; start: number; end: number };};parameterSources?: Record<string, ParameterSource[]>;};@@ -37,7 +40,7 @@ export type SourceText = {export type SourceNode = SourceElement | SourceText;export type SourceModuleRecord = {- kind: "svml" | "svc" | "svs";+ kind: "svml" | "svc" | "svs" | "svk";uri: string;contentHash: string;dependencies: string[];@@ -50,6 +53,14 @@ export type SourceDocument = {scriptSource: string;scriptOffset: number;modules?: SourceModuleRecord[];+ expansions?: Array<{+ instanceId: string;+ component: string;+ manifestHash: string;+ expansionDigest: string;+ internalIds: string[];+ exports: Record<string, string>;+ }>;};export type ScriptAtom =@@ -60,6 +71,9 @@ export type ScriptToken = {id: string;index: number;segmentId: string;+ segmentTokenIndex: number;+ startAnchorId: string;+ endAnchorId: string;text: string;normalized: string;sourceStart: number;@@ -69,7 +83,7 @@ export type ScriptToken = {export type MarkerBoundary = {tokenIndex: number;segmentId?: string;- structuralCut?: number;+ structuralPosition: number;};export type SelectionOccurrence = {@@ -98,6 +112,8 @@ export type MomentOccurrence = {export type ScriptSegment = {id: string;index: number;+ startAnchorId: string;+ endAnchorId: string;atoms: ScriptAtom[];tokenStart: number;tokenEnd: number;@@ -105,6 +121,34 @@ export type ScriptSegment = {sourceEnd: number;};+export type SemanticAnchorKind =+ | "segment-start"+ | "token-start"+ | "token-end"+ | "segment-end";++export type SemanticAnchorIdentity = {+ id: string;+ kind: SemanticAnchorKind;+ segmentId: string;+ tokenId?: string;+ segmentTokenIndex?: number;+};++export type SemanticIndex = {+ contract: "svml.semantic-index.v1";+ anchors: SemanticAnchorIdentity[];+ tokens: Array<{+ id: string;+ segmentId: string;+ segmentTokenIndex: number;+ normalized: string;+ startAnchorId: string;+ endAnchorId: string;+ }>;+ digest: string;+};+export type CaptionAtom = {id: string;display: string;@@ -121,6 +165,7 @@ export type NarrativeIR = {selections: Record<string, SelectionOccurrence[]>;moments: Record<string, MomentOccurrence[]>;captionAtoms: CaptionAtom[];+ semanticIndex: SemanticIndex;projections: {dialogue: string;speech: string;@@ -128,6 +173,93 @@ export type NarrativeIR = {};};+export type Rational = {+ numerator: number;+ denominator: number;+};++export type ProgramBasis = {+ contract: "svml.program-basis.v1";+ frameRate: Rational;+ originFrame: 0;+ durationFrames: number;+ basisDigest: string;+};++export type ProgramPoint = {+ basisDigest: string;+ frame: number;+};++export type SemanticPointQuality = "estimated" | "derived" | "measured";++export type SemanticAnchorPoint = {+ identity: string;+ point: ProgramPoint;+ quality: SemanticPointQuality;+};++export type SemanticMapBase = {+ semanticIndexDigest: string;+ basisDigest: string;+ anchors: SemanticAnchorPoint[];+ evidenceDigests: string[];+ locatorDigest: string;+ quantizationPolicy: "nearest-frame";+ captionCues?: AlignmentCaptionCue[];+ mapDigest: string;+};++export type EstimatedSemanticMap = SemanticMapBase & {+ contract: "svml.estimated-semantic-map.v1";+};++export type ExactSemanticMap = SemanticMapBase & {+ contract: "svml.exact-semantic-map.v1";+};++export type SemanticMap = EstimatedSemanticMap | ExactSemanticMap;++export type SourceToProgramMap = {+ id: string;+ sourceId: string;+ sourceDigest: string;+ sourceStartFrame: number;+ sourceEndFrameExclusive: number;+ programStartFrame: number;+ programEndFrameExclusive: number;+};++export type ProgramAudioContribution = {+ id: string;+ sourceId: string;+ sourceDigest: string;+ source: string;+ sourceMapId: string;+ sourceStartFrame: number;+ sourceEndFrameExclusive: number;+ programStartFrame: number;+ programEndFrameExclusive: number;+ gain: number;+ fadeInFrames?: number;+ fadeOutFrames?: number;+};++export type TemporalBasisProduction = {+ contract: "svml.temporal-basis-production.v1";+ basis: ProgramBasis;+ alignmentSubjects: Array<{+ id: string;+ sourceId: string;+ sourceDigest: string;+ sourceMapId: string;+ }>;+ sourceMaps: SourceToProgramMap[];+ audio: ProgramAudioContribution[];+ facets: Record<string, unknown>;+ productionDigest: string;+};+export type AlignmentWord = {text: string;startSec: number;@@ -183,6 +315,10 @@ export type LocatedCaptionCue = ProgramRange & {export type LocatedCaptionAtom = CaptionAtom & ProgramRange;export type LocatedScript = {+ contract: "svml.temporal-binding.v1";+ precision: "estimated" | "exact";+ basisDigest: string;+ semanticMapDigest: string;durationFrames: number;durationSec: number;fps: number;@@ -196,7 +332,7 @@ export type LocatedScript = {export type PlanValue = {id: string;- type: "Image" | "Video" | "Audio" | "Text";+ type: "Image" | "Video" | "Audio" | "Text" | "AlignmentEvidence";source?: string;value?: string;identity: string;@@ -220,6 +356,11 @@ export type PlanInstance = {children: SourceNode[];dependencies: string[];executionDigest?: string;+ expansionDigest?: string;+ sourceMap?: {+ callSite: { file: string; start: number; end: number };+ definitionSite: { file: string; start: number; end: number };+ };};export type PlanIR = {@@ -232,7 +373,7 @@ export type PlanIR = {abiVersion: string;manifestHash: string;implementationHash: string;- profile: "isolated-projector-v1" | "capability-v1";+ profile: "isolated-projector-v1" | "capability-v1" | "composite-v1";capability?: string;ports: Array<{name: string;@@ -244,7 +385,7 @@ export type PlanIR = {parameters: Array<{name: string;styleName: string;- type: "string" | "number" | "boolean";+ type: string;defaultValue?: AttributeValue;}>;children: Array<{@@ -262,6 +403,7 @@ export type PlanIR = {}>;}>;root: string;+ expansions: NonNullable<SourceDocument["expansions"]>;edges: Array<{from: string;to: string;diff --git a/src/plan.ts b/src/plan.tsindex 88819f7e..fb25fdad 100644--- a/src/plan.ts+++ b/src/plan.ts@@ -24,11 +24,12 @@ const VALUE_TYPES: Record<string, PlanValue["type"]> = {video: "Video",audio: "Audio",text: "Text",+ alignment: "AlignmentEvidence",};function idOf(element: SourceElement): string {const id = element.attributes.id;- if (typeof id !== "string" || !/^[A-Za-z_][A-Za-z0-9_.-]*$/u.test(id)) {+ if (typeof id !== "string" || !/^[A-Za-z_][A-Za-z0-9_.:-]*$/u.test(id)) {fail("plan_declaration_id", `<${element.name}> requires a stable id.`);}return id;@@ -75,9 +76,17 @@ function scalarAccepts(type: string, value: AttributeValue): boolean {if (type === "number") return typeof value === "number" && Number.isFinite(value);if (type === "boolean") return typeof value === "boolean";if (type === "id") {- return typeof value === "string" && /^[A-Za-z_][A-Za-z0-9_.-]*$/u.test(value);+ return typeof value === "string" && /^[A-Za-z_][A-Za-z0-9_.:-]*$/u.test(value);}if (type === "ProgramSpan" || type === "ProgramPoint") return typeof value === "string";+ if (type === "duration") {+ return typeof value === "string" && /^[+-]?\d+(?:\.\d+)?(?:ms|s)$/u.test(value);+ }+ if (type === "SourceRange") {+ return typeof value === "string" && /^\s*\d+(?:\.\d+)?(?:ms|s)\s*\.\.\s*\d+(?:\.\d+)?(?:ms|s)\s*$/u.test(value);+ }+ const enumValues = /^enum\((.+)\)$/u.exec(type)?.[1]?.split(",");+ if (enumValues) return typeof value === "string" && enumValues.includes(value);return false;}@@ -144,7 +153,7 @@ function validateChild(}}-function validateKernelShape(element: SourceElement, manifest: KernelManifest): void {+export function validateKernelShape(element: SourceElement, manifest: KernelManifest): void {const inputPorts = manifest.ports.filter((port) => port.direction === "input");const allowed = new Set(["id",@@ -338,6 +347,17 @@ export function buildPlan(effectiveParameters,children: element.children,dependencies: [...new Set(dependencies)],+ ...(element.origin?.expansionDigest+ ? { expansionDigest: element.origin.expansionDigest }+ : {}),+ ...(element.origin?.callSite && element.origin.definitionSite+ ? {+ sourceMap: {+ callSite: element.origin.callSite,+ definitionSite: element.origin.definitionSite,+ },+ }+ : {}),});}@@ -482,6 +502,39 @@ export function buildPlan(};visit(root);const reachableInstances = instances.filter((instance) => reachable.has(instance.id));+ const reachableEdges = edges.filter((edge) => reachable.has(edge.from) && reachable.has(edge.to));+ const rootManifest = kernels.get(rootCandidates[0]!.kernel)!;+ const basisInputs = rootManifest.ports.filter((port) =>+ port.direction === "input" && port.type.split("|").includes("TemporalBasisProduction"));+ const semanticInputs = rootManifest.ports.filter((port) =>+ port.direction === "input" && port.type.split("|").includes("ExactSemanticMap"));+ if (+ basisInputs.length !== 1+ || basisInputs[0]?.cardinality !== "one"+ || semanticInputs.length !== 1+ || semanticInputs[0]?.cardinality !== "one"+ ) {+ fail(+ "composition_temporal_contract",+ `Composition Kernel "${rootManifest.name}" must select exactly one TemporalBasisProduction and one ExactSemanticMap.`,+ );+ }+ const trackEdges = reachableEdges.filter((edge) => edge.type === "Track");+ const illegalTrackConsumer = trackEdges.find((edge) => edge.to !== root);+ if (illegalTrackConsumer) {+ fail(+ "track_terminal_contract",+ `Track "${illegalTrackConsumer.from}" is consumed by non-Composition instance "${illegalTrackConsumer.to}".`,+ );+ }+ const duplicateTrack = trackEdges.find((edge, index) =>+ trackEdges.findIndex((candidate) => candidate.from === edge.from && candidate.to === edge.to) !== index);+ if (duplicateTrack) {+ fail(+ "duplicate_track_in_composition",+ `Composition "${root}" references Track "${duplicateTrack.from}" more than once.`,+ );+ }const usedKernels = new Set(reachableInstances.map((instance) => instance.kernel));return {contract: "svml.plan.v1",@@ -503,7 +556,8 @@ export function buildPlan(})).sort((left, right) => left.name.localeCompare(right.name)),root,- edges: edges.filter((edge) => reachable.has(edge.from) && reachable.has(edge.to)),+ expansions: document.expansions ?? [],+ edges: reachableEdges,};}diff --git a/src/runtime-contract.ts b/src/runtime-contract.tsindex e3f14565..49ac1bca 100644--- a/src/runtime-contract.ts+++ b/src/runtime-contract.ts@@ -1,9 +1,5 @@import type {- AlignmentEvidence,AttributeValue,- LocatedScript,- NarrativeIR,- PlanIR,PlanInstance,PlanValue,ProgramRange,@@ -16,7 +12,6 @@ export type VisualFragment = {startFrame: number;endFrameExclusive: number;z: number;- layer?: number;source?: string;mediaStartSec?: number;playbackRate?: number;@@ -65,19 +60,20 @@ export type HyperframesDocument = {export type RuntimeValue = PlanValue & {absoluteSource?: string;+ data?: unknown;};export type KernelContext = {instance: PlanInstance;element: SourceElement;- narrative: NarrativeIR;- located: LocatedScript;- alignment: AlignmentEvidence;- plan: PlanIR;- values: Map<string, RuntimeValue>;- results: Map<string, KernelProjection>;resolve: (value: AttributeValue | undefined) => unknown;- resolvePath: (path: string) => unknown;+ digest: (value: unknown) => string;+ program?: {+ basisDigest: string;+ fps: number;+ durationFrames: number;+ durationSec: number;+ };temporalContracts: Record<string, {kind: "selection" | "moment";consume: "one" | "each" | "set";diff --git a/src/runtime-validation.ts b/src/runtime-validation.tsindex 284e522f..72b789bf 100644--- a/src/runtime-validation.ts+++ b/src/runtime-validation.ts@@ -149,7 +149,7 @@ function normalizeSurface(value: unknown): void {}const audioIds = (surface.audios ?? []).map((fragment) => fragment.id);if (new Set(audioIds).size !== audioIds.length) {- fail("kernel_audio_duplicate_id", "A projected audio tree repeats a fragment id.");+ fail("kernel_audio_duplicate_id", "A projected audio contribution set repeats a fragment id.");}const styles = Array.isArray(surface.styles) ? surface.styles : [];if (surface.visuals) surface.visuals = surface.visuals.map((fragment) =>diff --git a/src/sandbox-worker.mjs b/src/sandbox-worker.mjsindex ee285b1a..b38f2b69 100644--- a/src/sandbox-worker.mjs+++ b/src/sandbox-worker.mjs@@ -1,4 +1,5 @@import { readFile, realpath } from "node:fs/promises";+import { createHash } from "node:crypto";import { dirname, relative, resolve } from "node:path";import { fileURLToPath, pathToFileURL } from "node:url";import vm from "node:vm";@@ -25,6 +26,7 @@ async function main() {name: "svml-kernel-projector",codeGeneration: { strings: false, wasm: false },});+ context.__svmlSha256 = value => createHash("sha256").update(value).digest("hex");new vm.Script(`Object.defineProperty(Math, "random", {value() { throw new Error("SVK projector cannot use ambient randomness"); },@@ -84,9 +86,14 @@ async function main() {(() => {const context = JSON.parse(globalThis.__svmlInput);context.resolve = value => value;- context.resolvePath = () => {- throw new Error("resolvePath is unavailable in isolated projector ABI v1");+ const stableJson = value => {+ if (Array.isArray(value)) return "[" + value.map(stableJson).join(",") + "]";+ if (value && typeof value === "object") {+ return "{" + Object.keys(value).sort().map(key => JSON.stringify(key) + ":" + stableJson(value[key])).join(",") + "}";+ }+ return JSON.stringify(value);};+ context.digest = value => globalThis.__svmlSha256(stableJson(value));const frameRange = (startSec, endSec) => {const startFrame = Math.round(startSec * context.fps);const endFrameExclusive = Math.round(endSec * context.fps);@@ -101,10 +108,10 @@ async function main() {const endpoint = raw => {const value = String(raw).trim();if (value === "start") return 0;- if (value === "end") return context.located.durationSec;+ if (value === "end") return context.program.durationSec;const relative = /^(start|end)([+-])(\\d+(?:\\.\\d+)?)(ms|s)$/.exec(value);if (relative) {- const base = relative[1] === "start" ? 0 : context.located.durationSec;+ const base = relative[1] === "start" ? 0 : context.program.durationSec;const amount = Number(relative[3]) * (relative[4] === "ms" ? 0.001 : 1);return base + (relative[2] === "+" ? amount : -amount);}@@ -123,7 +130,7 @@ async function main() {const contract = temporalContract(path, "selection");let ranges;if (typeof value === "string") {- if (value === "full") ranges = [frameRange(0, context.located.durationSec)];+ if (value === "full") ranges = [frameRange(0, context.program.durationSec)];else {const parts = value.split("..");if (parts.length !== 2) throw new Error("manual span requires start .. end");diff --git a/src/script/locate.ts b/src/script/locate.tsindex 949eaae1..40a39211 100644--- a/src/script/locate.ts+++ b/src/script/locate.ts@@ -1,23 +1,29 @@import { fail } from "../diagnostics.js";import type {- AlignmentEvidence,+ ExactSemanticMap,LocatedScript,MarkerBoundary,NarrativeIR,+ ProgramBasis,ProgramRange,+ SemanticMap,} from "../model.js";-import { normalizeWord } from "../util.js";+import { basisFps, semanticPoints, validateSemanticMap } from "../temporal.js";-function frameAt(seconds: number, fps: number): number {- return Math.round(seconds * fps);-}--function range(startSec: number, endSec: number, fps: number): ProgramRange {- const startFrame = frameAt(startSec, fps);- const endFrameExclusive = frameAt(endSec, fps);- if (endFrameExclusive < startFrame) {- fail("locate_negative_range", `Range ${startSec}..${endSec} resolves backwards.`);+function range(+ startFrame: number,+ endFrameExclusive: number,+ basis: ProgramBasis,+ label: string,+ requirePositive = false,+): ProgramRange {+ if (endFrameExclusive < startFrame || (requirePositive && endFrameExclusive === startFrame)) {+ fail(+ requirePositive ? "locate_nonpositive_range" : "locate_negative_range",+ `${label} resolves to non-positive range ${startFrame}..${endFrameExclusive}.`,+ );}+ const fps = basisFps(basis);return {startFrame,endFrameExclusive,@@ -26,180 +32,72 @@ function range(startSec: number, endSec: number, fps: number): ProgramRange {};}-function boundaryTime(+export function boundaryAnchor(+ narrative: NarrativeIR,boundary: MarkerBoundary,affinity: "left" | "right",- narrative: NarrativeIR,- evidence: AlignmentEvidence,-): number {- const left = boundary.tokenIndex > 0 ? evidence.words[boundary.tokenIndex - 1] : undefined;- const right = evidence.words[boundary.tokenIndex];- if (affinity === "left" && left) return left.endSec;- if (affinity === "right" && right) return right.startSec;-- const segment = boundary.segmentId- ? evidence.segments.find((item) => item.id === boundary.segmentId)- : undefined;- if (segment) return affinity === "left" ? segment.endSec : segment.startSec;+): string {+ if (boundary.segmentId) {+ const segment = narrative.segments.find((item) => item.id === boundary.segmentId);+ if (!segment) fail("locate_boundary_segment", `Unknown Segment "${boundary.segmentId}".`);+ if (affinity === "left") {+ const token = narrative.tokens[boundary.tokenIndex - 1];+ if (token && token.segmentId === segment.id) return token.endAnchorId;+ return segment.startAnchorId;+ }+ const token = narrative.tokens[boundary.tokenIndex];+ if (token && token.segmentId === segment.id) return token.startAnchorId;+ return segment.endAnchorId;+ }- const cut = boundary.structuralCut ?? 0;- if (cut <= 0) return 0;- if (cut >= narrative.segments.length) return evidence.durationSec;- const rightSegment = evidence.segments.find((item) => item.id === narrative.segments[cut]?.id);- const leftSegment = evidence.segments.find((item) => item.id === narrative.segments[cut - 1]?.id);- if (affinity === "right" && rightSegment) return rightSegment.startSec;- if (leftSegment) return leftSegment.endSec;- fail("locate_unresolved_boundary", `Could not resolve structural cut ${cut}.`);+ const position = boundary.structuralPosition;+ if (affinity === "left") {+ return narrative.segments[position - 1]?.endAnchorId+ ?? narrative.segments[position]?.startAnchorId+ ?? fail("locate_boundary_empty", "Script has no structural boundary candidate.");+ }+ return narrative.segments[position]?.startAnchorId+ ?? narrative.segments[position - 1]?.endAnchorId+ ?? fail("locate_boundary_empty", "Script has no structural boundary candidate.");}-export function locateScript(+function validateCaptionCues(narrative: NarrativeIR,- evidence: AlignmentEvidence,-): LocatedScript {- if (evidence.contract !== "svml.speech-alignment.v1") {- fail("locate_alignment_contract", `Unsupported alignment contract "${evidence.contract}".`);- }- if (- !Number.isFinite(evidence.fps)- || evidence.fps <= 0- || !Number.isFinite(evidence.durationSec)- || evidence.durationSec <= 0- ) {- fail("locate_alignment_clock", "Alignment evidence requires positive finite fps and durationSec.");- }- if (narrative.tokens.length !== evidence.words.length) {- fail(- "locate_alignment_cardinality",- `Script has ${narrative.tokens.length} words but alignment has ${evidence.words.length}.`,- );- }- for (let index = 0; index < evidence.words.length; index += 1) {- const word = evidence.words[index];- if (!word) continue;- if (- !Number.isFinite(word.startSec)- || !Number.isFinite(word.endSec)- || word.startSec < 0- || word.endSec < word.startSec- || word.endSec > evidence.durationSec- ) {- fail(- "locate_alignment_word_window",- `Alignment word ${index + 1} has invalid window ${word.startSec}..${word.endSec}.`,- );- }- const previous = evidence.words[index - 1];- if (previous && word.startSec < previous.endSec - 1e-6) {- fail(- "locate_alignment_word_overlap",- `Alignment word ${index + 1} starts at ${word.startSec} before word ${index} ends at ${previous.endSec}.`,- );- }- const segment = evidence.segments.find((item) => item.id === word.segmentId);- if (- !segment- || word.startSec < segment.startSec - 1e-6- || word.endSec > segment.endSec + 1e-6- ) {- fail(- "locate_alignment_word_segment",- `Alignment word ${index + 1} falls outside segment "${word.segmentId}".`,- );- }- }- for (let index = 0; index < narrative.tokens.length; index += 1) {- const token = narrative.tokens[index];- const word = evidence.words[index];- if (!token || !word) continue;- if (token.normalized !== normalizeWord(word.text)) {- fail(- "locate_alignment_word_mismatch",- `Word ${index + 1} is "${token.text}" in Script and "${word.text}" in alignment evidence.`,- );- }- if (token.segmentId !== word.segmentId) {- fail(- "locate_alignment_segment_mismatch",- `Word ${index + 1} belongs to ${token.segmentId} in Script and ${word.segmentId} in evidence.`,- );- }- }- const fps = evidence.fps;- const words = narrative.tokens.map((token, index) => {- const word = evidence.words[index];- if (!word) fail("locate_alignment_missing_word", `Missing aligned word ${index + 1}.`);- return { ...token, ...range(word.startSec, word.endSec, fps) };- });- const segments = Object.fromEntries(evidence.segments.map((segment) => [- segment.id,- range(segment.startSec, segment.endSec, fps),- ]));- const selections = Object.fromEntries(Object.entries(narrative.selections).map(([id, occurrences]) => [- id,- {- id,- ranges: occurrences.map((occurrence) => range(- boundaryTime(occurrence.open.boundary, occurrence.open.affinity, narrative, evidence),- boundaryTime(occurrence.close.boundary, occurrence.close.affinity, narrative, evidence),- fps,- )),- },- ]));- const moments = Object.fromEntries(Object.entries(narrative.moments).map(([id, occurrences]) => [- id,- {- id,- frames: occurrences.map((occurrence) => frameAt(- boundaryTime(occurrence.boundary, occurrence.affinity, narrative, evidence),- fps,- )),- },- ]));- const captionAtoms = narrative.captionAtoms.map((atom) => {- const first = words[atom.startWord];- const last = words[atom.endWordExclusive - 1];- if (!first || !last || first.segmentId !== last.segmentId) {- fail("locate_caption_atom", `Caption atom "${atom.id}" has an invalid speech span.`);- }- return {- ...atom,- ...range(first.startSec, last.endSec, fps),- };- });- const captionCues = evidence.captionCues?.map((cue, index) => {+ locatedWords: LocatedScript["words"],+ locatedAtoms: LocatedScript["captionAtoms"],+ map: SemanticMap,+ basis: ProgramBasis,+): LocatedScript["captionCues"] {+ const cues = map.captionCues;+ if (!cues) return undefined;+ const located = cues.map((cue, index) => {if (!Number.isInteger(cue.startWord) || !Number.isInteger(cue.endWordExclusive)) {- fail("locate_caption_cue_index", `Caption cue ${index + 1} must use integer word indexes.`);+ fail("locate_caption_cue_index", `Caption cue ${index + 1} must use integer token indexes.`);}if (cue.startWord < 0|| cue.endWordExclusive <= cue.startWord- || cue.endWordExclusive > words.length+ || cue.endWordExclusive > locatedWords.length) {fail("locate_caption_cue_range",- `Caption cue ${index + 1} has invalid word range ${cue.startWord}..${cue.endWordExclusive}.`,+ `Caption cue ${index + 1} has invalid token range ${cue.startWord}..${cue.endWordExclusive}.`,);}- const previous = evidence.captionCues?.[index - 1];+ const previous = cues[index - 1];const expectedStart = previous?.endWordExclusive ?? 0;if (cue.startWord !== expectedStart) {fail("locate_caption_cue_partition",- `Caption cue ${index + 1} starts at word ${cue.startWord}; expected ${expectedStart}.`,+ `Caption cue ${index + 1} starts at token ${cue.startWord}; expected ${expectedStart}.`,);}- const first = words[cue.startWord];- const last = words[cue.endWordExclusive - 1];- if (!first || !last) {- fail("locate_caption_cue_word", `Caption cue ${index + 1} references a missing word.`);- }- if (first.segmentId !== last.segmentId) {- fail(- "locate_caption_cue_segment",- `Caption cue ${index + 1} crosses Script segments ${first.segmentId} and ${last.segmentId}.`,- );+ const first = locatedWords[cue.startWord];+ const last = locatedWords[cue.endWordExclusive - 1];+ if (!first || !last || first.segmentId !== last.segmentId) {+ fail("locate_caption_cue_segment", `Caption cue ${index + 1} crosses Script Segments.`);}- for (const atom of captionAtoms) {+ for (const atom of locatedAtoms) {const overlaps = cue.startWord < atom.endWordExclusive&& cue.endWordExclusive > atom.startWord;const contains = cue.startWord <= atom.startWord@@ -215,21 +113,105 @@ export function locateScript(id: cue.id ?? `cue-${index + 1}`,startWord: cue.startWord,endWordExclusive: cue.endWordExclusive,- ...range(first.startSec, last.endSec, fps),+ ...range(first.startFrame, last.endFrameExclusive, basis, `Caption cue ${index + 1}`),};});- if (- captionCues- && (captionCues.at(-1)?.endWordExclusive ?? 0) !== words.length- ) {+ if ((located.at(-1)?.endWordExclusive ?? 0) !== narrative.tokens.length) {fail("locate_caption_cue_partition",- `Caption cues end at word ${captionCues.at(-1)?.endWordExclusive ?? 0}; expected ${words.length}.`,+ `Caption cues end at token ${located.at(-1)?.endWordExclusive ?? 0}; expected ${narrative.tokens.length}.`,);}+ return located;+}++export function locateScript(+ narrative: NarrativeIR,+ basis: ProgramBasis,+ map: ExactSemanticMap,+): LocatedScript {+ if (map.contract !== "svml.exact-semantic-map.v1") {+ fail("locate_exact_required", "Located HTML compilation requires ExactSemanticMap.");+ }+ return bindSemanticMap(narrative, basis, map);+}++export function bindSemanticMap(+ narrative: NarrativeIR,+ basis: ProgramBasis,+ map: SemanticMap,+): LocatedScript {+ validateSemanticMap(narrative, basis, map);+ const points = semanticPoints(map);+ const point = (identity: string): number => {+ const value = points.get(identity);+ if (value === undefined) fail("locate_anchor_missing", `SemanticMap is missing "${identity}".`);+ return value;+ };+ const fps = basisFps(basis);+ const words = narrative.tokens.map((token) => ({+ ...token,+ ...range(+ point(token.startAnchorId),+ point(token.endAnchorId),+ basis,+ `Token "${token.text}"`,+ ),+ }));+ const segments = Object.fromEntries(narrative.segments.map((segment) => [+ segment.id,+ range(+ point(segment.startAnchorId),+ point(segment.endAnchorId),+ basis,+ `Segment "${segment.id}"`,+ ),+ ]));+ const selections = Object.fromEntries(Object.entries(narrative.selections).map(+ ([id, occurrences]) => [+ id,+ {+ id,+ ranges: occurrences.map((occurrence, index) => range(+ point(boundaryAnchor(narrative, occurrence.open.boundary, occurrence.open.affinity)),+ point(boundaryAnchor(narrative, occurrence.close.boundary, occurrence.close.affinity)),+ basis,+ `Selection "${id}" occurrence ${index + 1}`,+ true,+ )),+ },+ ],+ ));+ const moments = Object.fromEntries(Object.entries(narrative.moments).map(+ ([id, occurrences]) => [+ id,+ {+ id,+ frames: occurrences.map((occurrence) => point(+ boundaryAnchor(narrative, occurrence.boundary, occurrence.affinity),+ )),+ },+ ],+ ));+ const captionAtoms = narrative.captionAtoms.map((atom) => {+ const first = words[atom.startWord];+ const last = words[atom.endWordExclusive - 1];+ if (!first || !last || first.segmentId !== last.segmentId) {+ fail("locate_caption_atom", `Caption atom "${atom.id}" has an invalid speech span.`);+ }+ return {+ ...atom,+ ...range(first.startFrame, last.endFrameExclusive, basis, `Caption atom "${atom.id}"`),+ };+ });+ const captionCues = validateCaptionCues(narrative, words, captionAtoms, map, basis);return {- durationFrames: frameAt(evidence.durationSec, fps),- durationSec: frameAt(evidence.durationSec, fps) / fps,+ contract: "svml.temporal-binding.v1",+ precision: map.contract === "svml.exact-semantic-map.v1" ? "exact" : "estimated",+ basisDigest: basis.basisDigest,+ semanticMapDigest: map.mapDigest,+ durationFrames: basis.durationFrames,+ durationSec: basis.durationFrames / fps,fps,words,segments,diff --git a/src/script/parse.ts b/src/script/parse.tsindex a3cbdc5f..4f626dab 100644--- a/src/script/parse.ts+++ b/src/script/parse.ts@@ -8,7 +8,7 @@ import type {ScriptToken,SelectionOccurrence,} from "../model.js";-import { normalizeWord } from "../util.js";+import { normalizeWord, sha256, stableJson } from "../util.js";type Marker = {id: string;@@ -77,15 +77,27 @@ function joinProjection(parts: string[]): string {function markerBoundary(tokenIndex: number,segment: MutableSegment | undefined,- structuralCut: number,+ structuralPosition: number,): MarkerBoundary {return {tokenIndex,- structuralCut,+ structuralPosition,...(segment ? { segmentId: segment.id } : {}),};}+function segmentAnchorId(segmentId: string, edge: "start" | "end"): string {+ return `segment:${segmentId}:${edge}`;+}++function tokenAnchorId(+ segmentId: string,+ segmentTokenIndex: number,+ edge: "start" | "end",+): string {+ return `segment:${segmentId}:token:${segmentTokenIndex + 1}:${edge}`;+}+export function parseScript(file: string,scriptSource: string,@@ -105,7 +117,7 @@ export function parseScript(const captionAtoms: NarrativeIR["captionAtoms"] = [];let current: MutableSegment | undefined;let offset = 0;- let structuralCut = 0;+ let structuralPosition = 0;let lineStart = true;const location = (at: number) => sourceLocation(file, scriptSource, at);@@ -196,7 +208,7 @@ export function parseScript(literalPieces(raw, absoluteStart, dual).map((piece) => piece.value).join("");const addMarker = (marker: Marker, at: number): void => {- const boundary = markerBoundary(tokens.length, current, structuralCut);+ const boundary = markerBoundary(tokens.length, current, structuralPosition);if (marker.kind === "moment") {if (openSelections.has(marker.id) || selections[marker.id]) {fail(@@ -292,10 +304,14 @@ export function parseScript(const normalized = normalizeWord(match[0]);if (!normalized) continue;const index = tokens.length;+ const segmentTokenIndex = index - current.tokenStart;tokens.push({id: `w${index + 1}`,index,segmentId: current.id,+ segmentTokenIndex,+ startAnchorId: tokenAnchorId(current.id, segmentTokenIndex, "start"),+ endAnchorId: tokenAnchorId(current.id, segmentTokenIndex, "end"),text: match[0],normalized,sourceStart: sourceOffset + start + match.index,@@ -402,11 +418,13 @@ export function parseScript(if (segmentOpen[3] === "/") {segments.push({...current,+ startAnchorId: segmentAnchorId(current.id, "start"),+ endAnchorId: segmentAnchorId(current.id, "end"),tokenEnd: tokens.length,sourceEnd: sourceOffset + offset,});current = undefined;- structuralCut += 1;+ structuralPosition += 1;}continue;}@@ -419,11 +437,13 @@ export function parseScript(offset += segmentClose[0].length;segments.push({...current,+ startAnchorId: segmentAnchorId(current.id, "start"),+ endAnchorId: segmentAnchorId(current.id, "end"),tokenEnd: tokens.length,sourceEnd: sourceOffset + offset,});current = undefined;- structuralCut += 1;+ structuralPosition += 1;lineStart = true;continue;}@@ -561,12 +581,58 @@ export function parseScript(if (caption) captionSegments.push(caption);}+ const semanticAnchors = segments.flatMap((segment) => [+ {+ id: segment.startAnchorId,+ kind: "segment-start" as const,+ segmentId: segment.id,+ },+ ...tokens.slice(segment.tokenStart, segment.tokenEnd).flatMap((token) => [+ {+ id: token.startAnchorId,+ kind: "token-start" as const,+ segmentId: segment.id,+ tokenId: token.id,+ segmentTokenIndex: token.segmentTokenIndex,+ },+ {+ id: token.endAnchorId,+ kind: "token-end" as const,+ segmentId: segment.id,+ tokenId: token.id,+ segmentTokenIndex: token.segmentTokenIndex,+ },+ ]),+ {+ id: segment.endAnchorId,+ kind: "segment-end" as const,+ segmentId: segment.id,+ },+ ]);+ const semanticTokens = tokens.map((token) => ({+ id: token.id,+ segmentId: token.segmentId,+ segmentTokenIndex: token.segmentTokenIndex,+ normalized: token.normalized,+ startAnchorId: token.startAnchorId,+ endAnchorId: token.endAnchorId,+ }));+ const semanticIndexPayload = {+ contract: "svml.semantic-index.v1" as const,+ anchors: semanticAnchors,+ tokens: semanticTokens,+ };+return {segments,tokens,selections,moments,captionAtoms,+ semanticIndex: {+ ...semanticIndexPayload,+ digest: sha256(stableJson(semanticIndexPayload)),+ },projections: {dialogue: dialogueTurns.join("\n"),speech: speechSegments.join("\n"),diff --git a/src/source/modules.ts b/src/source/modules.tsindex b2ce8318..1b9a683c 100644--- a/src/source/modules.ts+++ b/src/source/modules.ts@@ -78,7 +78,7 @@ function cloneNode(if (localId) attributes.id = `${args.prefix}${localId}`;if (args.topLevel- && ["image", "video", "audio"].includes(node.name)+ && ["image", "video", "audio", "alignment"].includes(node.name)&& typeof attributes.src === "string"&& !isAbsolute(attributes.src)&& !isExternalSource(attributes.src)diff --git a/src/source/parse-document.ts b/src/source/parse-document.tsindex dc1a45c4..80896096 100644--- a/src/source/parse-document.ts+++ b/src/source/parse-document.ts@@ -73,7 +73,7 @@ function parseReference(cursor: Cursor): AttributeValue {}const path = cursor.source.slice(start, cursor.offset).trim();cursor.offset += 1;- if (!/^[A-Za-z_][A-Za-z0-9_.:-]*$/u.test(path)) {+ if (!/^[A-Za-z_][A-Za-z0-9_.:-]*(?:\[[A-Za-z_][A-Za-z0-9_.:-]*\])?$/u.test(path)) {fail("source_invalid_reference",`Invalid reference path "${path}".`,diff --git a/src/source/styles.ts b/src/source/styles.tsindex a858e893..2e08053c 100644--- a/src/source/styles.ts+++ b/src/source/styles.ts@@ -10,6 +10,8 @@ import type {SourceNode,} from "../model.js";import type {+ KernelChildSchema,+ KernelField,KernelManifest,KernelParameter,KernelRegistry,@@ -57,8 +59,20 @@ function typedValue(if (typeof value !== "boolean") {fail("style_parameter_type", `${source} expects boolean ${parameter.styleName}.`);}- } else if (typeof value !== "string") {- fail("style_parameter_type", `${source} expects string ${parameter.styleName}.`);+ } else {+ if (typeof value !== "string") {+ fail("style_parameter_type", `${source} expects string ${parameter.styleName}.`);+ }+ if (+ parameter.type === "duration"+ && !/^[+-]?\d+(?:\.\d+)?(?:ms|s)$/u.test(value)+ ) {+ fail("style_parameter_type", `${source} expects duration ${parameter.styleName}.`);+ }+ const enumValues = /^enum\((.+)\)$/u.exec(parameter.type)?.[1]?.split(",");+ if (enumValues && !enumValues.includes(value)) {+ fail("style_parameter_type", `${source} expects ${parameter.type} ${parameter.styleName}.`);+ }}return value;}@@ -140,6 +154,7 @@ function findRule(token: string,element: SourceElement,sheets: Sheet[],+ selector: string,): StyleRule {const dot = token.indexOf(".");if (dot > 0) {@@ -148,18 +163,18 @@ function findRule(const sheet = sheets.find((candidate) => candidate.binding === binding);if (!sheet) fail("style_unknown_binding", `Unknown style binding "${binding}".`);const rule = sheet.rules.find(- (candidate) => candidate.selector === element.name && candidate.className === className,+ (candidate) => candidate.selector === selector && candidate.className === className,);if (!rule) {fail("style_unknown_class",- `${sheet.file} does not define ${element.name}.${className}.`,+ `${sheet.file} does not define ${selector}.${className}.`,);}return rule;}const matches = sheets.flatMap((sheet) => sheet.rules.filter(- (candidate) => candidate.selector === element.name && candidate.className === token,+ (candidate) => candidate.selector === selector && candidate.className === token,));if (matches.length !== 1) {fail(@@ -180,6 +195,23 @@ function parameterFor();}+function fieldFor(+ schemas: KernelChildSchema[],+ elementName: string,+ styleName: string,+): KernelField | undefined {+ for (const schema of schemas) {+ if (schema.name === elementName) {+ const field = schema.fields.find((candidate) =>+ candidate.styleName === styleName || candidate.name === styleName);+ if (field) return field;+ }+ const nested = fieldFor(schema.children, elementName, styleName);+ if (nested) return nested;+ }+ return undefined;+}+function styleElement(element: SourceElement,sheets: Sheet[],@@ -187,7 +219,16 @@ function styleElement(ownerKernel?: KernelManifest,): SourceElement {const manifest = kernels.get(element.name) ?? ownerKernel;+ if (element.parameterSources) {+ return {+ ...element,+ children: element.children.map((child) => child.kind === "element"+ ? styleElement(child, sheets, kernels, manifest)+ : child),+ };+ }const topLevelKernel = kernels.get(element.name);+ const selector = topLevelKernel?.name ?? ownerKernel?.name ?? element.name;const classValue = element.attributes.class;if (classValue !== undefined && typeof classValue !== "string") {fail("style_class_type", `<${element.name}> class must be a string.`);@@ -195,19 +236,27 @@ function styleElement(const attributes: Record<string, AttributeValue> = {};const parameterSources: Record<string, ParameterSource[]> = {};for (const token of (classValue ?? "").split(/\s+/u).filter(Boolean)) {- const rule = findRule(token, element, sheets);+ const rule = findRule(token, element, sheets, selector);for (const property of rule.properties) {const parameter = topLevelKernel? parameterFor(topLevelKernel, property.name): undefined;- if (topLevelKernel && !parameter) {+ const field = !topLevelKernel && ownerKernel+ ? fieldFor(ownerKernel.children, element.name, property.name)+ : undefined;+ if ((topLevelKernel || ownerKernel) && !parameter && !field) {fail("style_unknown_parameter",`${rule.source} configures unknown public parameter "${property.name}".`,);}- const name = parameter?.name ?? property.name;- const value = typedValue(property.raw, parameter, rule.source);+ const schema = parameter ?? (field ? {+ name: field.name,+ styleName: field.styleName,+ type: field.type,+ } : undefined);+ const name = schema?.name ?? property.name;+ const value = typedValue(property.raw, schema, rule.source);attributes[name] = value;(parameterSources[name] ??= []).push({kind: "svs-class",diff --git a/src/temporal.ts b/src/temporal.tsnew file mode 100644index 00000000..3e12b189--- /dev/null+++ b/src/temporal.ts@@ -0,0 +1,484 @@+import { fail } from "./diagnostics.js";+import type {+ AlignmentEvidence,+ ExactSemanticMap,+ EstimatedSemanticMap,+ NarrativeIR,+ ProgramBasis,+ SemanticAnchorPoint,+ SemanticMap,+ TemporalBasisProduction,+} from "./model.js";+import { normalizeWord, sha256, stableJson } from "./util.js";++const EPSILON = 1e-6;++export function basisFps(basis: ProgramBasis): number {+ return basis.frameRate.numerator / basis.frameRate.denominator;+}++function integer(value: number, label: string): void {+ if (!Number.isInteger(value)) fail("temporal_integer", `${label} must be an integer.`);+}++function normalizeFrameRate(fps: number): ProgramBasis["frameRate"] {+ if (!Number.isFinite(fps) || fps <= 0) {+ fail("basis_frame_rate", `ProgramBasis fps must be positive and finite; received ${fps}.`);+ }+ if (Number.isInteger(fps)) return { numerator: fps, denominator: 1 };+ const denominator = 1_000_000;+ const numerator = Math.round(fps * denominator);+ const divisor = gcd(numerator, denominator);+ return { numerator: numerator / divisor, denominator: denominator / divisor };+}++function gcd(left: number, right: number): number {+ left = Math.abs(left);+ right = Math.abs(right);+ while (right) [left, right] = [right, left % right];+ return left || 1;+}++export function createProgramBasis(args: {+ fps: number;+ durationFrames: number;+ outcomeDigest: string;+}): ProgramBasis {+ integer(args.durationFrames, "ProgramBasis durationFrames");+ if (args.durationFrames <= 0) {+ fail("basis_duration", "ProgramBasis durationFrames must be positive.");+ }+ const payload = {+ contract: "svml.program-basis.v1" as const,+ frameRate: normalizeFrameRate(args.fps),+ originFrame: 0 as const,+ durationFrames: args.durationFrames,+ outcomeDigest: args.outcomeDigest,+ };+ return {+ contract: payload.contract,+ frameRate: payload.frameRate,+ originFrame: payload.originFrame,+ durationFrames: payload.durationFrames,+ basisDigest: sha256(stableJson(payload)),+ };+}++export function validateProgramBasis(basis: ProgramBasis): void {+ if (basis.contract !== "svml.program-basis.v1") {+ fail("basis_contract", `Unsupported ProgramBasis contract "${basis.contract}".`);+ }+ integer(basis.frameRate.numerator, "ProgramBasis frameRate numerator");+ integer(basis.frameRate.denominator, "ProgramBasis frameRate denominator");+ integer(basis.durationFrames, "ProgramBasis durationFrames");+ if (+ basis.originFrame !== 0+ || basis.frameRate.numerator <= 0+ || basis.frameRate.denominator <= 0+ || basis.durationFrames <= 0+ || !/^[a-f0-9]{64}$/u.test(basis.basisDigest)+ ) {+ fail("basis_shape", "ProgramBasis has an invalid clock or digest.");+ }+}++function canonicalFacet(value: unknown): unknown {+ if (Array.isArray(value)) return value.map(canonicalFacet);+ if (!value || typeof value !== "object") return value;+ return Object.fromEntries(Object.entries(value as Record<string, unknown>)+ .filter(([key]) => !["source", "absoluteSource", "basisDigest"].includes(key))+ .map(([key, nested]) => [key, canonicalFacet(nested)]));+}++export function basisOutcome(production: TemporalBasisProduction): unknown {+ return {+ contract: "svml.program-basis-outcome.v1",+ frameRate: production.basis.frameRate,+ originFrame: production.basis.originFrame,+ durationFrames: production.basis.durationFrames,+ alignmentSubjects: production.alignmentSubjects,+ sourceMaps: production.sourceMaps,+ audio: production.audio.map(({ source: _source, ...contribution }) => contribution),+ facets: canonicalFacet(production.facets),+ };+}++function digest(value: string, label: string): void {+ if (!/^[a-f0-9]{64}$/u.test(value)) {+ fail("temporal_digest", `${label} must be a lowercase SHA-256 digest.`);+ }+}++function uniqueIds(values: Array<{ id: string }>, label: string): void {+ if (new Set(values.map((value) => value.id)).size !== values.length) {+ fail("basis_identity_duplicate", `${label} contains duplicate ids.`);+ }+}++function validateProgramWindow(+ start: number,+ end: number,+ durationFrames: number,+ label: string,+): void {+ integer(start, `${label} start`);+ integer(end, `${label} end`);+ if (start < 0 || end <= start || end > durationFrames) {+ fail("basis_program_window", `${label} has invalid Program range ${start}..${end}.`);+ }+}++function validateFacetAffinity(value: unknown, basisDigest: string, path = "facets"): void {+ if (Array.isArray(value)) {+ value.forEach((nested, index) => validateFacetAffinity(nested, basisDigest, `${path}[${index}]`));+ return;+ }+ if (!value || typeof value !== "object") return;+ const record = value as Record<string, unknown>;+ if ("basisDigest" in record && record.basisDigest !== basisDigest) {+ fail("basis_facet_affinity", `${path} is bound to a different ProgramBasis.`);+ }+ for (const [key, nested] of Object.entries(record)) {+ validateFacetAffinity(nested, basisDigest, `${path}.${key}`);+ }+}++export function validateTemporalBasisProduction(production: TemporalBasisProduction): void {+ if (production.contract !== "svml.temporal-basis-production.v1") {+ fail("basis_production_contract", "Unsupported TemporalBasisProduction contract.");+ }+ validateProgramBasis(production.basis);+ digest(production.productionDigest, "productionDigest");+ uniqueIds(production.alignmentSubjects, "alignmentSubjects");+ uniqueIds(production.sourceMaps, "sourceMaps");+ uniqueIds(production.audio, "Program audio");+ const maps = new Map(production.sourceMaps.map((map) => [map.id, map]));+ for (const map of production.sourceMaps) {+ digest(map.sourceDigest, `Source map "${map.id}" digest`);+ integer(map.sourceStartFrame, `Source map "${map.id}" source start`);+ integer(map.sourceEndFrameExclusive, `Source map "${map.id}" source end`);+ if (map.sourceStartFrame < 0 || map.sourceEndFrameExclusive <= map.sourceStartFrame) {+ fail("basis_source_window", `Source map "${map.id}" has an invalid source range.`);+ }+ validateProgramWindow(+ map.programStartFrame,+ map.programEndFrameExclusive,+ production.basis.durationFrames,+ `Source map "${map.id}"`,+ );+ }+ for (const subject of production.alignmentSubjects) {+ digest(subject.sourceDigest, `Alignment subject "${subject.id}" digest`);+ const map = maps.get(subject.sourceMapId);+ if (+ !map+ || map.sourceId !== subject.sourceId+ || map.sourceDigest !== subject.sourceDigest+ ) {+ fail("basis_alignment_subject", `Alignment subject "${subject.id}" has no matching source map.`);+ }+ }+ for (const contribution of production.audio) {+ digest(contribution.sourceDigest, `Program audio "${contribution.id}" digest`);+ const map = maps.get(contribution.sourceMapId);+ if (+ !map+ || map.sourceId !== contribution.sourceId+ || map.sourceDigest !== contribution.sourceDigest+ ) {+ fail("basis_audio_source_map", `Program audio "${contribution.id}" has no matching source map.`);+ }+ validateProgramWindow(+ contribution.programStartFrame,+ contribution.programEndFrameExclusive,+ production.basis.durationFrames,+ `Program audio "${contribution.id}"`,+ );+ const duration = contribution.programEndFrameExclusive - contribution.programStartFrame;+ for (const [name, value] of [+ ["fadeInFrames", contribution.fadeInFrames ?? 0],+ ["fadeOutFrames", contribution.fadeOutFrames ?? 0],+ ] as const) {+ integer(value, `Program audio "${contribution.id}" ${name}`);+ if (value < 0 || value > duration) {+ fail("basis_audio_fade", `Program audio "${contribution.id}" has invalid ${name}.`);+ }+ }+ if (!Number.isFinite(contribution.gain) || contribution.gain < 0) {+ fail("basis_audio_gain", `Program audio "${contribution.id}" has invalid gain.`);+ }+ }+ validateFacetAffinity(production.facets, production.basis.basisDigest);+ const expected = sha256(stableJson(basisOutcome(production)));+ if (production.basis.basisDigest !== expected) {+ fail(+ "basis_digest",+ `ProgramBasis digest ${production.basis.basisDigest} does not match canonical outcome ${expected}.`,+ );+ }+}++function frameAt(seconds: number, basis: ProgramBasis): number {+ return Math.round(seconds * basisFps(basis));+}++function semanticMapDigest(map: Omit<SemanticMap, "mapDigest">): string {+ return sha256(stableJson(map));+}++function validateAlignment(+ narrative: NarrativeIR,+ basis: ProgramBasis,+ evidence: AlignmentEvidence,+): void {+ if (evidence.contract !== "svml.speech-alignment.v1") {+ fail("locator_evidence_contract", `Unsupported alignment contract "${evidence.contract}".`);+ }+ if (+ !Number.isFinite(evidence.fps)+ || evidence.fps <= 0+ || !Number.isFinite(evidence.durationSec)+ || evidence.durationSec <= 0+ ) {+ fail("locator_evidence_clock", "Alignment evidence requires positive fps and duration.");+ }+ if (Math.abs(evidence.fps - basisFps(basis)) > EPSILON) {+ fail(+ "locator_basis_frame_rate",+ `Alignment fps ${evidence.fps} does not match ProgramBasis fps ${basisFps(basis)}.`,+ );+ }+ if (frameAt(evidence.durationSec, basis) !== basis.durationFrames) {+ fail(+ "locator_basis_duration",+ `Alignment duration resolves to ${frameAt(evidence.durationSec, basis)} frames; ProgramBasis has ${basis.durationFrames}.`,+ );+ }+ if (evidence.words.length !== narrative.tokens.length) {+ fail(+ "locator_word_cardinality",+ `Script has ${narrative.tokens.length} tokens but alignment has ${evidence.words.length}.`,+ );+ }+ if (evidence.segments.length !== narrative.segments.length) {+ fail(+ "locator_segment_cardinality",+ `Script has ${narrative.segments.length} Segments but alignment has ${evidence.segments.length}.`,+ );+ }++ const segments = new Map(evidence.segments.map((segment) => [segment.id, segment]));+ if (segments.size !== evidence.segments.length) {+ fail("locator_segment_duplicate", "Alignment evidence repeats a Segment id.");+ }+ for (const [index, segment] of narrative.segments.entries()) {+ const aligned = segments.get(segment.id);+ if (!aligned) fail("locator_segment_missing", `Alignment is missing Segment "${segment.id}".`);+ if (+ !Number.isFinite(aligned.startSec)+ || !Number.isFinite(aligned.endSec)+ || aligned.startSec < 0+ || aligned.endSec < aligned.startSec+ || aligned.endSec > evidence.durationSec + EPSILON+ ) {+ fail(+ "locator_segment_window",+ `Segment "${segment.id}" has invalid window ${aligned.startSec}..${aligned.endSec}.`,+ );+ }+ const previousScript = narrative.segments[index - 1];+ const previous = previousScript ? segments.get(previousScript.id) : undefined;+ if (+ previous+ && (+ aligned.startSec < previous.startSec - EPSILON+ || aligned.endSec < previous.endSec - EPSILON+ )+ ) {+ fail(+ "locator_segment_order",+ `Segment "${segment.id}" violates source-order start/end monotonicity.`,+ );+ }+ }++ const previousBySegment = new Map<string, AlignmentEvidence["words"][number]>();+ for (const [index, token] of narrative.tokens.entries()) {+ const word = evidence.words[index];+ if (!word) fail("locator_word_missing", `Alignment is missing token ${index + 1}.`);+ if (normalizeWord(word.text) !== token.normalized || word.segmentId !== token.segmentId) {+ fail(+ "locator_word_mismatch",+ `Token ${index + 1} does not match alignment word "${word.text}" in Segment "${word.segmentId}".`,+ );+ }+ const segment = segments.get(word.segmentId);+ if (+ !segment+ || !Number.isFinite(word.startSec)+ || !Number.isFinite(word.endSec)+ || word.startSec < segment.startSec - EPSILON+ || word.endSec < word.startSec+ || word.endSec > segment.endSec + EPSILON+ ) {+ fail(+ "locator_word_window",+ `Alignment token ${index + 1} falls outside Segment "${word.segmentId}".`,+ );+ }+ const previous = previousBySegment.get(word.segmentId);+ if (previous && word.startSec < previous.endSec - EPSILON) {+ fail(+ "locator_word_overlap",+ `Alignment token ${index + 1} overlaps the preceding token in Segment "${word.segmentId}".`,+ );+ }+ previousBySegment.set(word.segmentId, word);+ }+}++export function semanticMapFromAlignment(+ narrative: NarrativeIR,+ basis: ProgramBasis,+ evidence: AlignmentEvidence,+ options: { precision: "exact"; locatorDigest?: string },+): ExactSemanticMap;+export function semanticMapFromAlignment(+ narrative: NarrativeIR,+ basis: ProgramBasis,+ evidence: AlignmentEvidence,+ options: { precision: "estimated"; locatorDigest?: string },+): EstimatedSemanticMap;+export function semanticMapFromAlignment(+ narrative: NarrativeIR,+ basis: ProgramBasis,+ evidence: AlignmentEvidence,+ options: {+ precision: "estimated" | "exact";+ locatorDigest?: string;+ },+): EstimatedSemanticMap | ExactSemanticMap {+ validateProgramBasis(basis);+ validateAlignment(narrative, basis, evidence);+ const segmentEvidence = new Map(evidence.segments.map((segment) => [segment.id, segment]));+ const wordEvidence = new Map(narrative.tokens.map((token, index) => [token.id, evidence.words[index]!]));+ const quality = options.precision === "exact" ? "measured" : "estimated";+ const anchors: SemanticAnchorPoint[] = narrative.semanticIndex.anchors.map((anchor) => {+ const segment = segmentEvidence.get(anchor.segmentId);+ if (!segment) fail("locator_segment_missing", `Alignment is missing Segment "${anchor.segmentId}".`);+ const word = anchor.tokenId ? wordEvidence.get(anchor.tokenId) : undefined;+ const seconds = anchor.kind === "segment-start"+ ? segment.startSec+ : anchor.kind === "segment-end"+ ? segment.endSec+ : anchor.kind === "token-start"+ ? word?.startSec+ : word?.endSec;+ if (seconds === undefined) {+ fail("locator_anchor_missing", `No alignment time exists for anchor "${anchor.id}".`);+ }+ return {+ identity: anchor.id,+ point: { basisDigest: basis.basisDigest, frame: frameAt(seconds, basis) },+ quality,+ };+ });+ const contract = options.precision === "exact"+ ? "svml.exact-semantic-map.v1" as const+ : "svml.estimated-semantic-map.v1" as const;+ const payload = {+ contract,+ semanticIndexDigest: narrative.semanticIndex.digest,+ basisDigest: basis.basisDigest,+ anchors,+ evidenceDigests: [sha256(stableJson(evidence))],+ locatorDigest: options.locatorDigest ?? sha256(`svml.${options.precision}-speech-locator.v1`),+ quantizationPolicy: "nearest-frame" as const,+ ...(evidence.captionCues ? { captionCues: evidence.captionCues } : {}),+ };+ const map = { ...payload, mapDigest: semanticMapDigest(payload) };+ validateSemanticMap(narrative, basis, map);+ return map;+}++export function validateSemanticMap(+ narrative: NarrativeIR,+ basis: ProgramBasis,+ map: SemanticMap,+): void {+ validateProgramBasis(basis);+ if (+ map.contract !== "svml.estimated-semantic-map.v1"+ && map.contract !== "svml.exact-semantic-map.v1"+ ) {+ fail("semantic_map_contract", "Unsupported SemanticMap contract.");+ }+ if (map.semanticIndexDigest !== narrative.semanticIndex.digest) {+ fail("semantic_map_index", "SemanticMap was produced for a different SemanticIndex.");+ }+ if (map.basisDigest !== basis.basisDigest) {+ fail("semantic_map_basis", "SemanticMap was produced for a different ProgramBasis.");+ }+ const { mapDigest, ...canonicalMap } = map;+ if (semanticMapDigest(canonicalMap) !== mapDigest) {+ fail("semantic_map_digest", "SemanticMap digest does not match its canonical contents.");+ }+ if (map.anchors.length !== narrative.semanticIndex.anchors.length) {+ fail(+ "semantic_map_cardinality",+ `SemanticMap has ${map.anchors.length} anchors; expected ${narrative.semanticIndex.anchors.length}.`,+ );+ }+ const pointById = new Map<string, number>();+ for (const [index, expected] of narrative.semanticIndex.anchors.entries()) {+ const actual = map.anchors[index];+ if (!actual || actual.identity !== expected.id) {+ fail(+ "semantic_map_identity",+ `SemanticMap anchor ${index + 1} must be "${expected.id}".`,+ );+ }+ integer(actual.point.frame, `SemanticMap point ${actual.identity}`);+ if (+ actual.point.basisDigest !== basis.basisDigest+ || actual.point.frame < 0+ || actual.point.frame > basis.durationFrames+ ) {+ fail("semantic_map_point", `SemanticMap anchor "${actual.identity}" is outside ProgramBasis.`);+ }+ if (map.contract === "svml.exact-semantic-map.v1" && actual.quality === "estimated") {+ fail("semantic_map_precision", `ExactSemanticMap anchor "${actual.identity}" is estimated.`);+ }+ if (pointById.has(actual.identity)) {+ fail("semantic_map_duplicate", `SemanticMap repeats anchor "${actual.identity}".`);+ }+ pointById.set(actual.identity, actual.point.frame);+ }+ for (const segment of narrative.segments) {+ const ordered = [+ segment.startAnchorId,+ ...narrative.tokens.slice(segment.tokenStart, segment.tokenEnd)+ .flatMap((token) => [token.startAnchorId, token.endAnchorId]),+ segment.endAnchorId,+ ];+ for (let index = 1; index < ordered.length; index += 1) {+ if (pointById.get(ordered[index]!)! < pointById.get(ordered[index - 1]!)!) {+ fail("semantic_map_segment_order", `SemanticMap runs backwards inside Segment "${segment.id}".`);+ }+ }+ }+ for (let index = 1; index < narrative.segments.length; index += 1) {+ const previous = narrative.segments[index - 1]!;+ const current = narrative.segments[index]!;+ if (+ pointById.get(current.startAnchorId)! < pointById.get(previous.startAnchorId)!+ || pointById.get(current.endAnchorId)! < pointById.get(previous.endAnchorId)!+ ) {+ fail("semantic_map_source_order", `SemanticMap reorders Segment "${current.id}".`);+ }+ }+}++export function semanticPoints(map: SemanticMap): ReadonlyMap<string, number> {+ return new Map(map.anchors.map((anchor) => [anchor.identity, anchor.point.frame]));+}diff --git a/stdlib/broll-track.svk b/stdlib/broll-track.svkindex 18995248..f15c0bf8 100644--- a/stdlib/broll-track.svk+++ b/stdlib/broll-track.svk@@ -4,7 +4,7 @@<port name="track" direction="output" type="Track" cardinality="one"/><child name="item" cardinality="one-or-more"><field name="id" type="id" required="true"/>- <field name="source" type="Image|Video" required="true"/>+ <field name="source" type="Image|Video|ProgramBoundImage|ProgramBoundVideo" required="true"/><field name="during" type="SelectionSet|ProgramSpan" consume="each"/><field name="window" type="string"/><field name="playback" type="string"/>@@ -20,22 +20,23 @@<field name="height" type="number"/><field name="fit" type="string"/><field name="position" type="string"/>- <field name="shape" type="string"/>+ <field name="shape" style="mask" type="string"/><field name="radius" type="number"/><field name="opacity" type="number"/><field name="background" type="string"/>+ <field name="z" type="number"/><child name="present" cardinality="zero-or-more"><field name="id" type="id" required="true"/><field name="during" type="SelectionSet|ProgramSpan" consume="each"/><field name="window" type="string"/>- <field name="layer" type="number"/>+ <field name="z" type="number"/><field name="x" type="number"/><field name="y" type="number"/><field name="width" type="number"/><field name="height" type="number"/><field name="fit" type="string"/><field name="position" type="string"/>- <field name="shape" type="string"/>+ <field name="shape" style="mask" type="string"/><field name="radius" type="number"/><field name="opacity" type="number"/><field name="background" type="string"/>diff --git a/stdlib/caption-track.svk b/stdlib/caption-track.svkindex a077aa14..d96c40ea 100644--- a/stdlib/caption-track.svk+++ b/stdlib/caption-track.svk@@ -16,5 +16,15 @@<param name="strokeColor" style="stroke-color" type="string" default="rgba(0,0,0,.82)"/><param name="shadow" type="string" default="0 2px 9px rgba(0,0,0,.5), 0 0 2px rgba(0,0,0,.68)"/><port name="script" direction="input" type="NarrativeIR" cardinality="one"/>+ <port name="semantic" direction="input" type="ExactSemanticMap" cardinality="one"/><port name="track" direction="output" type="Track" cardinality="one"/>+ <child name="style" cardinality="zero-or-more">+ <field name="id" type="id" required="true"/>+ <field name="during" type="SelectionSet|ProgramSpan" consume="set" required="true"/>+ <field name="color" style="fill" type="string"/>+ <field name="activeColor" style="active-fill" type="string"/>+ <field name="background" type="string"/>+ <field name="weight" type="number"/>+ <field name="scale" type="number"/>+ </child></kernel>diff --git a/stdlib/film.svk b/stdlib/film.svkindex d2ac17f2..5c91952d 100644--- a/stdlib/film.svk+++ b/stdlib/film.svk@@ -3,7 +3,8 @@<param name="height" type="number"/><param name="fps" style="frame-rate" type="number"/><param name="background" type="string" default="#000000"/>- <port name="speech" direction="input" type="SpeechSpine" cardinality="one"/>+ <port name="basis" direction="input" type="TemporalBasisProduction" cardinality="one"/>+ <port name="semantic" direction="input" type="ExactSemanticMap" cardinality="one"/><port name="track" direction="input" type="Track" cardinality="zero-or-more"/><port name="document" direction="output" type="HyperframesDocument" cardinality="one"/><child name="track" cardinality="zero-or-more">diff --git a/stdlib/media-basis.svk b/stdlib/media-basis.svknew file mode 100644index 00000000..3a415a82--- /dev/null+++ b/stdlib/media-basis.svk@@ -0,0 +1,9 @@+<kernel name="media-basis" abi="1" implementation="./runtime/media-basis.mjs">+ <param name="fps" type="number" default="30"/>+ <param name="duration" type="duration"/>+ <param name="sourceStart" type="duration" default="0ms"/>+ <port name="audio" direction="input" type="Audio" cardinality="one"/>+ <port name="visual" direction="input" type="Video" cardinality="zero-or-one"/>+ <port name="production" direction="output" type="TemporalBasisProduction" cardinality="one"/>+ <port name="facets.visual" direction="output" type="ProgramBoundVideo" cardinality="zero-or-one"/>+</kernel>diff --git a/stdlib/media-track.svk b/stdlib/media-track.svkindex 13fdeced..b32a5a17 100644--- a/stdlib/media-track.svk+++ b/stdlib/media-track.svk@@ -4,7 +4,7 @@<port name="track" direction="output" type="Track" cardinality="one"/><child name="item" cardinality="one-or-more"><field name="id" type="id" required="true"/>- <field name="source" type="Image|Video" required="true"/>+ <field name="source" type="Image|Video|ProgramBoundImage|ProgramBoundVideo|ProgramBoundVideoSequence" required="true"/><field name="during" type="SelectionSet|ProgramSpan" consume="each"/><field name="window" type="string"/><field name="playback" type="string"/>@@ -16,22 +16,23 @@<field name="height" type="number"/><field name="fit" type="string"/><field name="position" type="string"/>- <field name="shape" type="string"/>+ <field name="shape" style="mask" type="string"/><field name="radius" type="number"/><field name="opacity" type="number"/><field name="background" type="string"/>+ <field name="z" type="number"/><child name="present" cardinality="zero-or-more"><field name="id" type="id" required="true"/><field name="during" type="SelectionSet|ProgramSpan" consume="each"/><field name="window" type="string"/>- <field name="layer" type="number"/>+ <field name="z" type="number"/><field name="x" type="number"/><field name="y" type="number"/><field name="width" type="number"/><field name="height" type="number"/><field name="fit" type="string"/><field name="position" type="string"/>- <field name="shape" type="string"/>+ <field name="shape" style="mask" type="string"/><field name="radius" type="number"/><field name="opacity" type="number"/><field name="background" type="string"/>diff --git a/stdlib/ranking-column.svk b/stdlib/ranking-column.svkindex 84f66d08..dc528c42 100644--- a/stdlib/ranking-column.svk+++ b/stdlib/ranking-column.svk@@ -12,6 +12,7 @@<param name="moveGain" style="move-gain" type="number" default="0.1"/><port name="appearSound" direction="input" type="Audio" cardinality="zero-or-one"/><port name="moveSound" direction="input" type="Audio" cardinality="zero-or-one"/>+ <port name="at" direction="input" type="MomentSet|ProgramPoint" cardinality="zero-or-one" consume="one"/><port name="item" direction="input" type="RankingItem" cardinality="one-or-more"/><port name="track" direction="output" type="Track" cardinality="one"/><child name="item" cardinality="one-or-more">diff --git a/stdlib/runtime/audio-track.mjs b/stdlib/runtime/audio-track.mjsindex 77339779..565e4e74 100644--- a/stdlib/runtime/audio-track.mjs+++ b/stdlib/runtime/audio-track.mjs@@ -24,7 +24,7 @@ export default {return {startFrame,endFrameExclusive: Math.min(- context.located.durationFrames,+ context.program.durationFrames,startFrame + durationFrames,),};diff --git a/stdlib/runtime/broll-track.mjs b/stdlib/runtime/broll-track.mjsindex 9c2ed5a5..4e2c50dc 100644--- a/stdlib/runtime/broll-track.mjs+++ b/stdlib/runtime/broll-track.mjs@@ -34,7 +34,7 @@ export default {if (String(source.type).toLowerCase() !== expectedKind.toLowerCase()) {throw new Error(`broll item "${id}" expected ${expectedKind}, received ${source.type}`);}- const presentations = resolveItemPresentations(context, item, source);+ const presentations = resolveItemPresentations(context, item, source, z);for (const [index, presentation] of presentations.entries()) {const range = presentation.range;const zoom = numberAttr(item, "zoom", 1);@@ -55,9 +55,10 @@ export default {source: source.source,startFrame: range.startFrame,endFrameExclusive: range.endFrameExclusive,- z,- layer: presentation.layer,- muted: !(item.attributes.audio === true || item.attributes.audio === "source"),+ z: presentation.z,+ // HyperFrames receives source audio as an explicit sibling contribution.+ // Keep the visual element muted so one source can never play twice.+ muted: true,...timing,style: {...presentationStyle(presentation.element, item, { fit: "contain" }),@@ -83,12 +84,14 @@ export default {presentation.parentRange,]));for (const [parentIndex, range] of parents) {+ const timing = mediaTiming(item, { parentRange: range, range }, source, context.fps);audios.push({id: `${context.instance.id}:${id}:${parentIndex}:audio`,source: source.source,startFrame: range.startFrame,endFrameExclusive: range.endFrameExclusive,- mediaStartSec: numberAttr(item, "mediaStart", 0),+ mediaStartSec: timing.mediaStartSec,+ playbackRate: timing.playbackRate,volume: numberAttr(item, "volume", 1),bus: "source",});@@ -122,7 +125,7 @@ export default {const cue = incoming.startFrame;const startFrame = cue - Math.floor(durationFrames * cueRatio);const endFrameExclusive = startFrame + durationFrames;- if (startFrame < 0 || endFrameExclusive > context.located.durationFrames) {+ if (startFrame < 0 || endFrameExclusive > context.program.durationFrames) {throw new Error(`broll transition ${from} -> ${to} leaves the master timeline`);}if (occupiedWindows.some((window) =>diff --git a/stdlib/runtime/caption-track.mjs b/stdlib/runtime/caption-track.mjsindex 85c9d95b..20658cc1 100644--- a/stdlib/runtime/caption-track.mjs+++ b/stdlib/runtime/caption-track.mjs@@ -1,13 +1,76 @@-import { html, numberAttr, stringAttr, trackOutput } from "./helpers.mjs";+import { elements, html, numberAttr, stringAttr, trackOutput } from "./helpers.mjs";export default {abiVersion: "1",project(context) {const script = context.resolve(context.element.attributes.script);- const located = script.located;+ const semantic = context.resolve(context.element.attributes.semantic);+ if (semantic?.contract !== "svml.exact-semantic-map.v1") {+ throw new Error("caption-track requires ExactSemanticMap");+ }+ if (semantic.basisDigest !== context.program.basisDigest) {+ throw new Error("caption-track SemanticMap does not match the selected ProgramBasis");+ }+ const points = new Map(semantic.anchors.map((anchor) => [anchor.identity, anchor.point.frame]));+ const frameRange = (startFrame, endFrameExclusive) => ({+ startFrame,+ endFrameExclusive,+ startSec: startFrame / context.fps,+ endSec: endFrameExclusive / context.fps,+ });+ const locatedWords = script.tokens.map((token) => ({+ ...token,+ ...frameRange(points.get(token.startAnchorId), points.get(token.endAnchorId)),+ }));+ const captionAtoms = script.captionAtoms.map((atom) => ({+ ...atom,+ ...frameRange(+ locatedWords[atom.startWord].startFrame,+ locatedWords[atom.endWordExclusive - 1].endFrameExclusive,+ ),+ }));+ const captionCues = semantic.captionCues?.map((cue, index) => ({+ id: cue.id ?? `cue-${index + 1}`,+ ...cue,+ ...frameRange(+ locatedWords[cue.startWord].startFrame,+ locatedWords[cue.endWordExclusive - 1].endFrameExclusive,+ ),+ }));+ const located = {+ words: locatedWords,+ captionAtoms,+ captionCues,+ durationFrames: context.program.durationFrames,+ };const z = numberAttr(context.element, "z");const maxWords = Math.max(1, Math.round(numberAttr(context.element, "maxWords", 3)));const words = located.words;+ const styledRanges = elements(context, "style").map((style) => ({+ id: stringAttr(style, "id"),+ ranges: context.selection(style.attributes.during, "style.during"),+ color: style.attributes.color,+ activeColor: style.attributes.activeColor,+ background: style.attributes.background,+ weight: style.attributes.weight,+ scale: style.attributes.scale,+ }));+ const inlineStyle = (word, active) => {+ const declarations = {};+ for (const rule of styledRanges) {+ if (!rule.ranges.some((range) =>+ word.startFrame < range.endFrameExclusive+ && word.endFrameExclusive > range.startFrame)) continue;+ if (typeof rule.color === "string") declarations.color = rule.color;+ if (active && typeof rule.activeColor === "string") declarations.color = rule.activeColor;+ if (typeof rule.background === "string") declarations.background = rule.background;+ if (typeof rule.weight === "number") declarations["font-weight"] = rule.weight;+ if (typeof rule.scale === "number") declarations.transform = `scale(${rule.scale})`;+ }+ const body = Object.entries(declarations)+ .map(([name, value]) => `${name}:${String(value)}`).join(";");+ return body ? ` style="${html(body)}"` : "";+ };const atomsByStart = new Map(located.captionAtoms.map((atom) => [atom.startWord, atom]));const displayUnits = (startWord, endWordExclusive) => {const units = [];@@ -58,8 +121,11 @@ export default {nextCueStart ?? located.durationFrames,word.endFrameExclusive + Math.round(numberAttr(context.element, "hold", 0.08) * context.fps),);- const body = group.map((item, index) =>- `<span class="${index === activeIndex ? "active" : ""}">${html(item.text)}</span>`).join("");+ if (endFrameExclusive <= word.startFrame) continue;+ const body = group.map((item, index) => {+ const active = index === activeIndex;+ return `<span class="${active ? "active" : ""}"${inlineStyle(item, active)}>${html(item.text)}</span>`;+ }).join("");const fragmentDuration = Math.max(1 / context.fps,(endFrameExclusive - word.startFrame) / context.fps,diff --git a/stdlib/runtime/film.mjs b/stdlib/runtime/film.mjsindex 80187e75..0aa411bb 100644--- a/stdlib/runtime/film.mjs+++ b/stdlib/runtime/film.mjs@@ -6,16 +6,33 @@ export default {const visuals = [];const audios = [];const styles = [];- const speech = resolveAttr(context, context.element, "speech");- const speechTrack = speech.track ?? speech.speech?.track;- if (speechTrack) {- visuals.push(...(speechTrack.visuals ?? []));- audios.push(...(speechTrack.audios ?? []));- styles.push(...(speechTrack.styles ?? []));- } else if (speech.speech?.primaryAudio) {- audios.push(...speech.speech.primaryAudio);- } else if (speech.primaryAudio) {- audios.push(...speech.primaryAudio);+ const production = resolveAttr(context, context.element, "basis");+ const semantic = resolveAttr(context, context.element, "semantic");+ if (production?.contract !== "svml.temporal-basis-production.v1") {+ throw new Error("film requires TemporalBasisProduction");+ }+ if (semantic?.contract !== "svml.exact-semantic-map.v1") {+ throw new Error("film requires ExactSemanticMap");+ }+ if (semantic.basisDigest !== production.basis.basisDigest) {+ throw new Error("film basis and SemanticMap do not share a basisDigest");+ }+ const basisFps = production.basis.frameRate.numerator+ / production.basis.frameRate.denominator;+ for (const contribution of production.audio ?? []) {+ audios.push({+ id: contribution.id,+ source: contribution.source,+ startFrame: contribution.programStartFrame,+ endFrameExclusive: contribution.programEndFrameExclusive,+ mediaStartSec: contribution.sourceStartFrame / basisFps,+ playbackRate: (contribution.sourceEndFrameExclusive - contribution.sourceStartFrame)+ / Math.max(1, contribution.programEndFrameExclusive - contribution.programStartFrame),+ volume: contribution.gain,+ ...(contribution.fadeInFrames ? { fadeInSec: contribution.fadeInFrames / basisFps } : {}),+ ...(contribution.fadeOutFrames ? { fadeOutSec: contribution.fadeOutFrames / basisFps } : {}),+ bus: "speech",+ });}for (const trackElement of elements(context, "track")) {const track = resolveAttr(context, trackElement, "ref");@@ -23,9 +40,9 @@ export default {audios.push(...(track.audios ?? []));styles.push(...(track.styles ?? []));}- const fps = numberAttr(context.element, "fps", context.fps);- if (fps !== context.fps) {- throw new Error(`film frame-rate ${fps} does not match Located clock ${context.fps}`);+ const fps = numberAttr(context.element, "fps", basisFps);+ if (fps !== basisFps) {+ throw new Error(`film frame-rate ${fps} does not match ProgramBasis ${basisFps}`);}return {outputs: {@@ -35,7 +52,7 @@ export default {width: numberAttr(context.element, "width", context.width),height: numberAttr(context.element, "height", context.height),fps,- durationFrames: context.located.durationFrames,+ durationFrames: production.basis.durationFrames,background: stringAttr(context.element, "background", "#000000"),visuals,audios,diff --git a/stdlib/runtime/helpers.mjs b/stdlib/runtime/helpers.mjsindex 6b44645e..9626e99b 100644--- a/stdlib/runtime/helpers.mjs+++ b/stdlib/runtime/helpers.mjs@@ -30,13 +30,24 @@ export function resolveAttr(context, element, name) {export function material(value, expected) {if (!value || typeof value !== "object") throw new Error(`expected ${expected ?? "media"} material`);- const type = value.type ?? value.kind;- if (expected && type && String(type).toLowerCase() !== expected.toLowerCase()) {- throw new Error(`expected ${expected}, received ${String(type)}`);+ const declaredType = String(value.type ?? value.kind ?? "");+ const programBound = /^ProgramBound(?:Image|Video|Audio)$/u.test(declaredType);+ const type = programBound ? declaredType.slice("ProgramBound".length) : declaredType;+ if (expected && type && type.toLowerCase() !== expected.toLowerCase()) {+ throw new Error(`expected ${expected}, received ${declaredType}`);}const source = value.absoluteSource ?? value.source;if (typeof source !== "string") throw new Error(`${expected ?? "media"} material has no source`);- return { ...value, source };+ return { ...value, type, declaredType, programBound, source };+}++export function mediaMaterials(value, expected) {+ if (value?.type === "ProgramBoundVideoSequence") {+ const segments = Object.values(value.segments ?? {}).map((segment) => material(segment, expected));+ if (!segments.length) throw new Error("ProgramBoundVideoSequence has no media segments");+ return segments;+ }+ return [material(value, expected)];}function unitSeconds(raw) {@@ -219,7 +230,7 @@ export function presentationAnimation(element, range, fps, id) {};}-export function resolveItemPresentations(context, item, source) {+export function resolveItemPresentations(context, item, source, defaultZ, options = {}) {const rawParents = item.attributes.during? context.selection(item.attributes.during, "item.during"): source.range@@ -228,8 +239,10 @@ export function resolveItemPresentations(context, item, source) {if (!rawParents.length) {throw new Error(`item "${stringAttr(item, "id")}" has no temporal range`);}- const parents = rawParents.map((range) =>- projectWindow(range, item.attributes.window, context.fps));+ const parents = rawParents+ .map((range) => projectWindow(range, item.attributes.window, context.fps))+ .map((range) => source.range ? intersectRange(range, source.range) : range)+ .filter(Boolean);const presents = elements({ element: item }, "present");const explicit = [];for (const [presentIndex, present] of presents.entries()) {@@ -247,11 +260,14 @@ export function resolveItemPresentations(context, item, source) {parentRange,range,element: present,- layer: numberAttr(present, "layer", presentIndex + 1),+ z: numberAttr(present, "z", numberAttr(item, "z", defaultZ)),explicit: true,});}}+ if (!options.allowEmptyPresent && !explicit.some((value) => value.id === stringAttr(present, "id"))) {+ throw new Error(`presentation_empty: Present "${stringAttr(present, "id")}" has no intersection with its Item`);+ }}const defaults = [];for (const [parentIndex, parentRange] of parents.entries()) {@@ -268,18 +284,43 @@ export function resolveItemPresentations(context, item, source) {parentRange,range,element: undefined,- layer: 0,+ z: numberAttr(item, "z", defaultZ),explicit: false,});}}+ for (let left = 0; left < explicit.length; left += 1) {+ for (let right = left + 1; right < explicit.length; right += 1) {+ const a = explicit[left];+ const b = explicit[right];+ if (+ a.z === b.z+ && a.range.startFrame < b.range.endFrameExclusive+ && b.range.startFrame < a.range.endFrameExclusive+ ) {+ throw new Error(`presentation_z_ambiguous: Present "${a.id}" and "${b.id}" overlap at z=${a.z}`);+ }+ }+ }return [...defaults, ...explicit];}export function mediaTiming(item, presentation, source, fps) {- const playback = stringAttr(item, "playback", "sync");- const offsetFrames = presentation.range.startFrame - presentation.parentRange.startFrame;- const mediaStart = numberAttr(item, "mediaStart", 0);+ const playback = stringAttr(item, "playback", source.programBound ? "program-map" : "sync");+ const mappingRange = source.programBound && source.range+ ? source.range+ : presentation.parentRange;+ const offsetFrames = presentation.range.startFrame - mappingRange.startFrame;+ const mediaStart = numberAttr(item, "mediaStart", source.mediaStartSec ?? 0);+ if (playback === "program-map") {+ if (!source.programBound) {+ throw new Error(`playback "program-map" requires ProgramBound media`);+ }+ return {+ mediaStartSec: mediaStart + offsetFrames / fps * (source.playbackRate ?? 1),+ playbackRate: source.playbackRate ?? 1,+ };+ }if (playback === "sync") {return {mediaStartSec: mediaStart + offsetFrames / fps,@@ -307,3 +348,29 @@ export function mediaTiming(item, presentation, source, fps) {if (playback === "hold" && source.type === "Image") return {};throw new Error(`unsupported playback "${playback}" for ${source.type}`);}++export function programBoundAnimation(source, range, fps, id) {+ if (!source.programBound || (!source.fadeInFrames && !source.fadeOutFrames)) return {};+ const totalFrames = Math.max(1, range.endFrameExclusive - range.startFrame);+ const sourceStart = source.range.startFrame;+ const sourceEnd = source.range.endFrameExclusive;+ const fadeInEnd = sourceStart + (source.fadeInFrames ?? 0);+ const fadeOutStart = sourceEnd - (source.fadeOutFrames ?? 0);+ const points = new Map([[0, 1], [100, 1]]);+ const percent = (frameValue) => Math.max(0, Math.min(100,+ (frameValue - range.startFrame) / totalFrames * 100));+ if (source.fadeInFrames && range.startFrame < fadeInEnd) {+ points.set(0, Math.max(0, (range.startFrame - sourceStart) / source.fadeInFrames));+ points.set(percent(Math.min(range.endFrameExclusive, fadeInEnd)), 1);+ }+ if (source.fadeOutFrames && range.endFrameExclusive > fadeOutStart) {+ points.set(percent(Math.max(range.startFrame, fadeOutStart)), 1);+ points.set(100, Math.max(0, (sourceEnd - range.endFrameExclusive) / source.fadeOutFrames));+ }+ const name = `svml-program-bound-${id}`.replaceAll(/[^A-Za-z0-9_-]/gu, "-");+ return {+ animation: `${name} ${totalFrames / fps}s linear both`,+ css: `@keyframes ${name} { ${[...points].sort((a, b) => a[0] - b[0])+ .map(([at, opacity]) => `${at}% { opacity:${opacity}; }`).join(" ")} }`,+ };+}diff --git a/stdlib/runtime/media-basis.mjs b/stdlib/runtime/media-basis.mjsnew file mode 100644index 00000000..369bf3b0--- /dev/null+++ b/stdlib/runtime/media-basis.mjs@@ -0,0 +1,133 @@+import { material, numberAttr, resolveAttr, stringAttr } from "./helpers.mjs";++function seconds(value) {+ const match = /^(\d+(?:\.\d+)?)(ms|s)$/u.exec(String(value).trim());+ if (!match) throw new Error(`invalid duration "${String(value)}"`);+ return Number(match[1]) * (match[2] === "ms" ? 0.001 : 1);+}++function canonicalFacet(value) {+ if (Array.isArray(value)) return value.map(canonicalFacet);+ if (!value || typeof value !== "object") return value;+ return Object.fromEntries(Object.entries(value)+ .filter(([key]) => !["source", "absoluteSource", "basisDigest"].includes(key))+ .map(([key, nested]) => [key, canonicalFacet(nested)]));+}++function frameRate(fps) {+ if (!Number.isFinite(fps) || fps <= 0) throw new Error("fps must be positive and finite");+ if (Number.isInteger(fps)) return { numerator: fps, denominator: 1 };+ let numerator = Math.round(fps * 1_000_000);+ let denominator = 1_000_000;+ let left = numerator;+ let right = denominator;+ while (right) [left, right] = [right, left % right];+ numerator /= left;+ denominator /= left;+ return { numerator, denominator };+}++export default {+ abiVersion: "1",+ project(context) {+ const fps = numberAttr(context.element, "fps", 30);+ const rationalFps = frameRate(fps);+ const durationSec = seconds(stringAttr(context.element, "duration"));+ const sourceStartSec = seconds(stringAttr(context.element, "sourceStart", "0ms"));+ const durationFrames = Math.round(durationSec * fps);+ if (durationFrames <= 0) throw new Error("media-basis duration must resolve to positive frames");+ const sourceStartFrame = Math.round(sourceStartSec * fps);+ const sourceEndFrameExclusive = sourceStartFrame + durationFrames;+ const audioValue = material(resolveAttr(context, context.element, "audio"), "audio");+ const visualValue = context.element.attributes.visual+ ? material(resolveAttr(context, context.element, "visual"), "video")+ : undefined;+ const sourceMaps = [{+ id: "program:audio-map",+ sourceId: audioValue.id,+ sourceDigest: audioValue.contentDigest,+ sourceStartFrame,+ sourceEndFrameExclusive,+ programStartFrame: 0,+ programEndFrameExclusive: durationFrames,+ }];+ if (visualValue) {+ sourceMaps.push({+ id: "program:visual-map",+ sourceId: visualValue.id,+ sourceDigest: visualValue.contentDigest,+ sourceStartFrame,+ sourceEndFrameExclusive,+ programStartFrame: 0,+ programEndFrameExclusive: durationFrames,+ });+ }+ const alignmentSubjects = [{+ id: "program:audio",+ sourceId: audioValue.id,+ sourceDigest: audioValue.contentDigest,+ sourceMapId: "program:audio-map",+ }];+ const audio = [{+ id: "program:audio",+ sourceId: audioValue.id,+ sourceDigest: audioValue.contentDigest,+ source: audioValue.source,+ sourceMapId: "program:audio-map",+ sourceStartFrame,+ sourceEndFrameExclusive,+ programStartFrame: 0,+ programEndFrameExclusive: durationFrames,+ gain: 1,+ }];+ const unboundVisual = visualValue ? {+ type: "ProgramBoundVideo",+ source: visualValue.source,+ sourceId: visualValue.id,+ sourceDigest: visualValue.contentDigest,+ range: {+ startFrame: 0,+ endFrameExclusive: durationFrames,+ startSec: 0,+ endSec: durationFrames / fps,+ },+ mediaStartSec: sourceStartSec,+ playbackRate: 1,+ sourceMapId: "program:visual-map",+ } : undefined;+ const outcome = {+ contract: "svml.program-basis-outcome.v1",+ frameRate: rationalFps,+ originFrame: 0,+ durationFrames,+ alignmentSubjects,+ sourceMaps,+ audio: audio.map(({ source: _source, ...contribution }) => contribution),+ facets: canonicalFacet({ ...(unboundVisual ? { visual: unboundVisual } : {}) }),+ };+ const basisDigest = context.digest(outcome);+ const basis = {+ contract: "svml.program-basis.v1",+ frameRate: outcome.frameRate,+ originFrame: 0,+ durationFrames,+ basisDigest,+ };+ const facets = unboundVisual+ ? { visual: { ...unboundVisual, basisDigest } }+ : {};+ const production = {+ contract: "svml.temporal-basis-production.v1",+ basis,+ alignmentSubjects,+ sourceMaps,+ audio,+ facets,+ productionDigest: context.digest({+ implementation: context.instance.executionDigest,+ outcome,+ }),+ };+ return { outputs: { production, facets } };+ },+};diff --git a/stdlib/runtime/media-track.mjs b/stdlib/runtime/media-track.mjsindex f50140e4..ad28359d 100644--- a/stdlib/runtime/media-track.mjs+++ b/stdlib/runtime/media-track.mjs@@ -1,9 +1,10 @@import {elements,- material,+ mediaMaterials,mediaTiming,numberAttr,presentationAnimation,+ programBoundAnimation,presentationStyle,resolveItemPresentations,resolveAttr,@@ -18,9 +19,13 @@ export default {const visuals = [];for (const item of elements(context, "item")) {const id = stringAttr(item, "id");- const source = material(resolveAttr(context, item, "source"));- const presentations = resolveItemPresentations(context, item, source);+ const sources = mediaMaterials(resolveAttr(context, item, "source"));+ const itemVisuals = [];+ const resolvedPresentIds = new Set();+ for (const [sourceIndex, source] of sources.entries()) {+ const presentations = resolveItemPresentations(context, item, source, z, { allowEmptyPresent: true });for (const [index, presentation] of presentations.entries()) {+ if (presentation.explicit) resolvedPresentIds.add(presentation.id);const range = presentation.range;const animation = presentationAnimation(presentation.element,@@ -28,26 +33,46 @@ export default {context.fps,`${context.instance.id}-${id}-${presentation.id}-${index}`,);+ const sourceAnimation = programBoundAnimation(+ source,+ range,+ context.fps,+ `${context.instance.id}-${id}-${sourceIndex}-${presentation.id}-${index}`,+ );const timing = source.type === "Video"? mediaTiming(item, presentation, source, context.fps): {};- visuals.push({- id: `${context.instance.id}:${id}:${index}`,+ itemVisuals.push({+ id: `${context.instance.id}:${id}:${sourceIndex}:${index}`,kind: source.type === "Image" ? "image" : "video",source: source.source,startFrame: range.startFrame,endFrameExclusive: range.endFrameExclusive,- z,- layer: presentation.layer,+ z: presentation.z,muted: true,...timing,style: {...presentationStyle(presentation.element, item),- ...(animation.animation ? { animation: animation.animation } : {}),+ ...((animation.animation || sourceAnimation.animation) ? {+ animation: [animation.animation, sourceAnimation.animation].filter(Boolean).join(", "),+ } : {}),},- ...(animation.css ? { css: animation.css } : {}),+ ...((animation.css || sourceAnimation.css) ? {+ css: [animation.css, sourceAnimation.css].filter(Boolean).join("\n"),+ } : {}),});}+ }+ if (!itemVisuals.length) {+ throw new Error(`item_empty: Item "${id}" has no intersection with its source`);+ }+ for (const present of elements({ element: item }, "present")) {+ const presentId = stringAttr(present, "id");+ if (!resolvedPresentIds.has(presentId)) {+ throw new Error(`presentation_empty: Present "${presentId}" has no intersection with its Item`);+ }+ }+ visuals.push(...itemVisuals);}return trackOutput(visuals);},diff --git a/stdlib/runtime/ranking-column.mjs b/stdlib/runtime/ranking-column.mjsindex 8117ff54..8b4fe5eb 100644--- a/stdlib/runtime/ranking-column.mjs+++ b/stdlib/runtime/ranking-column.mjs@@ -141,6 +141,9 @@ export default {const stageSize = Math.round(context.width * numberAttr(context.element, "stageSize", 0.33));const stageCx = Math.round(context.width * numberAttr(context.element, "stageX", 0.66));const stageCy = Math.round(context.height * numberAttr(context.element, "stageY", 0.73));+ const boardStartFrame = context.element.attributes.at+ ? context.moment(context.element.attributes.at, "at")[0]+ : 0;const cellLeft = x + pad + rowH + Math.max(8, Math.round(rowH * 0.16));const rows = Array.from({ length: total }, (_, index) => {const top = pad + index * (rowH + gap);@@ -150,8 +153,8 @@ export default {const visuals = [{id: `${context.instance.id}:board`,kind: "html",- startFrame: 0,- endFrameExclusive: context.located.durationFrames,+ startFrame: boardStartFrame,+ endFrameExclusive: context.program.durationFrames,z,html: rows,style: {@@ -178,15 +181,20 @@ export default {const rank = Math.max(1, Math.min(total, numberAttr(item, "rank")));const base = context.selection(item.attributes.during, "item.during")[0];const range = projectWindow(base, item.attributes.window, context.fps);+ const visibleRange = {+ ...range,+ startFrame: Math.max(boardStartFrame, range.startFrame),+ };const top = y + pad + (rank - 1) * (rowH + gap);- const endDuration = (context.located.durationFrames - range.endFrameExclusive) / context.fps;+ const settledStartFrame = Math.max(boardStartFrame, range.endFrameExclusive);+ const endDuration = (context.program.durationFrames - settledStartFrame) / context.fps;if (endDuration > 0) {visuals.push({id: `${context.instance.id}:${id}:settled`,kind: "image",source: image.source,- startFrame: range.endFrameExclusive,- endFrameExclusive: context.located.durationFrames,+ startFrame: settledStartFrame,+ endFrameExclusive: context.program.durationFrames,z: z + 1,style: {left: `${cellLeft}px`,@@ -205,8 +213,9 @@ export default {},});}+ if (visibleRange.endFrameExclusive <= visibleRange.startFrame) continue;const keyframes = `svml-rank-${context.instance.id}-${id}`.replaceAll(/[^A-Za-z0-9_-]/gu, "-");- const durationFrames = Math.max(1, range.endFrameExclusive - range.startFrame);+ const durationFrames = Math.max(1, visibleRange.endFrameExclusive - visibleRange.startFrame);const duration = durationFrames / context.fps;const targetCx = cellLeft + rowH / 2;const targetCy = top + rowH / 2;@@ -214,8 +223,8 @@ export default {id: `${context.instance.id}:${id}:active`,kind: "image",source: image.source,- startFrame: range.startFrame,- endFrameExclusive: range.endFrameExclusive,+ startFrame: visibleRange.startFrame,+ endFrameExclusive: visibleRange.endFrameExclusive,z: z + 2,style: {left: `${stageCx - stageSize / 2}px`,@@ -254,10 +263,10 @@ ${activeKeyframes({audios.push({id: `${context.instance.id}:${id}:appear`,source: appear.source,- startFrame: range.startFrame,+ startFrame: visibleRange.startFrame,endFrameExclusive: Math.min(- context.located.durationFrames,- range.startFrame + Math.round(0.247021 * context.fps),+ context.program.durationFrames,+ visibleRange.startFrame + Math.round(0.247021 * context.fps),),volume: numberAttr(context.element, "appearGain", 0.09),bus: "sfx",@@ -268,8 +277,8 @@ ${activeKeyframes({audios.push({id: `${context.instance.id}:${id}:move`,source: move.source,- startFrame: Math.max(range.startFrame, range.endFrameExclusive - Math.round(0.55 * context.fps)),- endFrameExclusive: Math.min(context.located.durationFrames, range.endFrameExclusive - Math.round(0.55 * context.fps) + moveFrames),+ startFrame: Math.max(visibleRange.startFrame, visibleRange.endFrameExclusive - Math.round(0.55 * context.fps)),+ endFrameExclusive: Math.min(context.program.durationFrames, visibleRange.endFrameExclusive - Math.round(0.55 * context.fps) + moveFrames),volume: numberAttr(context.element, "moveGain", 0.1),bus: "sfx",});diff --git a/stdlib/runtime/speech-assemble.mjs b/stdlib/runtime/speech-assemble.mjsnew file mode 100644index 00000000..7384308b--- /dev/null+++ b/stdlib/runtime/speech-assemble.mjs@@ -0,0 +1,310 @@+import { elements, material, numberAttr, resolveAttr, stringAttr } from "./helpers.mjs";++function seconds(raw) {+ const match = /^([+-]?\d+(?:\.\d+)?)(ms|s)$/u.exec(String(raw).trim());+ if (!match) throw new Error(`invalid duration "${String(raw)}"`);+ return Number(match[1]) * (match[2] === "ms" ? 0.001 : 1);+}++function sourceWindow(raw) {+ const parts = String(raw).split("..");+ if (parts.length !== 2) throw new Error(`sourceWindow requires start .. end, received "${raw}"`);+ const startSec = seconds(parts[0]);+ const endSec = seconds(parts[1]);+ if (startSec < 0 || endSec <= startSec) throw new Error(`invalid sourceWindow "${raw}"`);+ return { startSec, endSec, durationSec: endSec - startSec };+}++function defaultJoin(element) {+ const kind = stringAttr(element, "defaultJoin", "cut");+ const duration = seconds(stringAttr(element, "defaultJoinDuration", "0ms"));+ if (kind === "cut") return { temporal: "cut", durationSec: 0, audio: "cut", visual: "cut" };+ if (kind === "crossfade") {+ if (duration <= 0) throw new Error("crossfade defaultJoinDuration must be positive");+ return { temporal: "overlap", durationSec: duration, audio: "crossfade", visual: "dissolve" };+ }+ if (kind === "gap") {+ if (duration <= 0) throw new Error("gap defaultJoinDuration must be positive");+ return { temporal: "gap", durationSec: duration, audio: "cut", visual: "cut" };+ }+ throw new Error(`unsupported defaultJoin "${kind}"`);+}++function explicitJoin(element, fallback) {+ const overlap = element.attributes.overlap;+ const gap = element.attributes.gap;+ if (overlap !== undefined && gap !== undefined) {+ throw new Error(`<join after="${stringAttr(element, "after")}"> cannot declare both overlap and gap`);+ }+ const temporal = overlap !== undefined ? "overlap" : gap !== undefined ? "gap" : "cut";+ const durationSec = overlap !== undefined+ ? seconds(overlap)+ : gap !== undefined+ ? seconds(gap)+ : 0;+ if (temporal !== "cut" && durationSec <= 0) throw new Error(`${temporal} duration must be positive`);+ const audio = stringAttr(element, "audio", temporal === "overlap" ? fallback.audio : "cut");+ const visual = stringAttr(element, "visual", temporal === "overlap" ? fallback.visual : "cut");+ if (!["cut", "crossfade"].includes(audio)) throw new Error(`unsupported audio join "${audio}"`);+ if (!["cut", "dissolve"].includes(visual)) throw new Error(`unsupported visual join "${visual}"`);+ if (temporal !== "overlap" && (audio === "crossfade" || visual === "dissolve")) {+ throw new Error(`${audio}/${visual} requires a temporal overlap`);+ }+ return {+ temporal,+ durationSec,+ audio,+ visual,+ };+}++function frame(secondsValue, fps) {+ return Math.round(secondsValue * fps);+}++function rationalFrameRate(fps) {+ if (!Number.isFinite(fps) || fps <= 0) throw new Error("fps must be positive and finite");+ if (Number.isInteger(fps)) return { numerator: fps, denominator: 1 };+ let numerator = Math.round(fps * 1_000_000);+ let denominator = 1_000_000;+ let left = numerator;+ let right = denominator;+ while (right) [left, right] = [right, left % right];+ numerator /= left;+ denominator /= left;+ return { numerator, denominator };+}++function canonicalFacet(value) {+ if (Array.isArray(value)) return value.map(canonicalFacet);+ if (!value || typeof value !== "object") return value;+ return Object.fromEntries(Object.entries(value)+ .filter(([key]) => !["source", "absoluteSource", "basisDigest"].includes(key))+ .map(([key, nested]) => [key, canonicalFacet(nested)]));+}++export default {+ abiVersion: "1",+ project(context) {+ const fps = numberAttr(context.element, "fps", 30);+ const rationalFps = rationalFrameRate(fps);+ const segmentElements = elements(context, "segment");+ const ids = segmentElements.map(segment => stringAttr(segment, "id"));+ if (new Set(ids).size !== ids.length) throw new Error("speech-assemble Segment ids must be unique");+ const fallback = defaultJoin(context.element);+ const overrides = new Map();+ for (const join of elements(context, "join")) {+ const after = stringAttr(join, "after");+ if (overrides.has(after)) throw new Error(`duplicate join after "${after}"`);+ if (!ids.includes(after) || ids.at(-1) === after) {+ throw new Error(`join after "${after}" must name a non-final Segment`);+ }+ overrides.set(after, explicitJoin(join, fallback));+ }++ const segments = [];+ const joins = [];+ let nextStartSec = 0;+ for (const [index, element] of segmentElements.entries()) {+ const id = ids[index];+ const script = resolveAttr(context, element, "script");+ if (!script || script.id !== id) throw new Error(`Segment "${id}" does not match its ScriptSegment`);+ const audio = material(resolveAttr(context, element, "audio"), "audio");+ const visual = element.attributes.visual+ ? material(resolveAttr(context, element, "visual"), "video")+ : undefined;+ const window = sourceWindow(stringAttr(element, "sourceWindow"));+ const startSec = nextStartSec;+ const endSec = startSec + window.durationSec;+ segments.push({ id, script, audio, visual, window, startSec, endSec });+ if (index >= segmentElements.length - 1) continue;+ const join = overrides.get(id) ?? fallback;+ joins.push({ after: id, ...join });+ nextStartSec = join.temporal === "overlap"+ ? endSec - join.durationSec+ : join.temporal === "gap"+ ? endSec + join.durationSec+ : endSec;+ if (nextStartSec < startSec) throw new Error(`join after "${id}" overlaps more than the Segment duration`);+ }++ const durationFrames = Math.max(...segments.map(segment => frame(segment.endSec, fps)));+ const frameRate = rationalFps;+ const sourceMaps = [];+ const alignmentSubjects = [];+ const audio = [];+ const unboundSegmentOutputs = {};+ for (const [index, segment] of segments.entries()) {+ const programStartFrame = frame(segment.startSec, fps);+ const programEndFrameExclusive = frame(segment.endSec, fps);+ const sourceStartFrame = frame(segment.window.startSec, fps);+ const sourceEndFrameExclusive = frame(segment.window.endSec, fps);+ const previousJoin = joins[index - 1];+ const nextJoin = joins[index];+ const audioMap = {+ id: `segment:${segment.id}:audio-map`,+ sourceId: segment.audio.id,+ sourceDigest: segment.audio.contentDigest,+ sourceStartFrame,+ sourceEndFrameExclusive,+ programStartFrame,+ programEndFrameExclusive,+ };+ sourceMaps.push(audioMap);+ alignmentSubjects.push({+ id: `segment:${segment.id}:speech`,+ sourceId: segment.audio.id,+ sourceDigest: segment.audio.contentDigest,+ sourceMapId: audioMap.id,+ });++ let audioEndFrameExclusive = programEndFrameExclusive;+ if (nextJoin?.temporal === "overlap" && nextJoin.audio === "cut") {+ audioEndFrameExclusive = frame(segments[index + 1].startSec, fps);+ }+ const fadeInFrames = previousJoin?.temporal === "overlap" && previousJoin.audio === "crossfade"+ ? frame(previousJoin.durationSec, fps)+ : 0;+ const fadeOutFrames = nextJoin?.temporal === "overlap" && nextJoin.audio === "crossfade"+ ? frame(nextJoin.durationSec, fps)+ : 0;+ const audioContribution = {+ id: `segment:${segment.id}:program-audio`,+ sourceId: segment.audio.id,+ sourceDigest: segment.audio.contentDigest,+ source: segment.audio.source,+ sourceMapId: audioMap.id,+ sourceStartFrame,+ sourceEndFrameExclusive: sourceStartFrame + (audioEndFrameExclusive - programStartFrame),+ programStartFrame,+ programEndFrameExclusive: audioEndFrameExclusive,+ gain: 1,+ ...(fadeInFrames ? { fadeInFrames } : {}),+ ...(fadeOutFrames ? { fadeOutFrames } : {}),+ };+ audio.push(audioContribution);++ const audioRange = {+ startFrame: programStartFrame,+ endFrameExclusive: audioEndFrameExclusive,+ startSec: programStartFrame / fps,+ endSec: audioEndFrameExclusive / fps,+ };+ const output = {+ audio: {+ type: "ProgramBoundAudio",+ source: segment.audio.source,+ sourceId: segment.audio.id,+ sourceDigest: segment.audio.contentDigest,+ range: audioRange,+ mediaStartSec: segment.window.startSec,+ playbackRate: 1,+ sourceMapId: audioMap.id,+ ...(fadeInFrames ? { fadeInFrames } : {}),+ ...(fadeOutFrames ? { fadeOutFrames } : {}),+ },+ };+ if (segment.visual) {+ const visualMap = {+ id: `segment:${segment.id}:visual-map`,+ sourceId: segment.visual.id,+ sourceDigest: segment.visual.contentDigest,+ sourceStartFrame,+ sourceEndFrameExclusive,+ programStartFrame,+ programEndFrameExclusive,+ };+ sourceMaps.push(visualMap);+ let visualEndFrameExclusive = programEndFrameExclusive;+ if (nextJoin?.temporal === "overlap" && nextJoin.visual === "cut") {+ visualEndFrameExclusive = frame(segments[index + 1].startSec, fps);+ }+ const visualFadeInFrames = previousJoin?.temporal === "overlap" && previousJoin.visual === "dissolve"+ ? frame(previousJoin.durationSec, fps)+ : 0;+ const visualFadeOutFrames = nextJoin?.temporal === "overlap" && nextJoin.visual === "dissolve"+ ? frame(nextJoin.durationSec, fps)+ : 0;+ output.visual = {+ type: "ProgramBoundVideo",+ source: segment.visual.source,+ sourceId: segment.visual.id,+ sourceDigest: segment.visual.contentDigest,+ range: {+ startFrame: programStartFrame,+ endFrameExclusive: visualEndFrameExclusive,+ startSec: programStartFrame / fps,+ endSec: visualEndFrameExclusive / fps,+ },+ mediaStartSec: segment.window.startSec,+ playbackRate: 1,+ sourceMapId: visualMap.id,+ ...(visualFadeInFrames ? { fadeInFrames: visualFadeInFrames } : {}),+ ...(visualFadeOutFrames ? { fadeOutFrames: visualFadeOutFrames } : {}),+ };+ }+ unboundSegmentOutputs[segment.id] = output;+ }+ const unboundFacets = {+ visual: {+ type: "ProgramBoundVideoSequence",+ segments: Object.fromEntries(Object.entries(unboundSegmentOutputs)+ .filter(([, value]) => value.visual)+ .map(([id, value]) => [id, value.visual])),+ },+ };+ const outcome = {+ contract: "svml.program-basis-outcome.v1",+ frameRate,+ originFrame: 0,+ durationFrames,+ alignmentSubjects,+ sourceMaps,+ audio: audio.map(({ source: _source, ...contribution }) => contribution),+ facets: canonicalFacet(unboundFacets),+ };+ const basisDigest = context.digest(outcome);+ const basis = {+ contract: "svml.program-basis.v1",+ frameRate,+ originFrame: 0,+ durationFrames,+ basisDigest,+ };+ const segmentOutputs = Object.fromEntries(Object.entries(unboundSegmentOutputs).map(([id, output]) => [+ id,+ {+ audio: { ...output.audio, basisDigest },+ ...(output.visual ? { visual: { ...output.visual, basisDigest } } : {}),+ },+ ]));+ const facets = {+ visual: {+ type: "ProgramBoundVideoSequence",+ basisDigest,+ segments: Object.fromEntries(Object.entries(segmentOutputs)+ .filter(([, value]) => value.visual)+ .map(([id, value]) => [id, value.visual])),+ },+ };+ const production = {+ contract: "svml.temporal-basis-production.v1",+ basis,+ alignmentSubjects,+ sourceMaps,+ audio,+ facets,+ productionDigest: context.digest({+ implementation: context.instance.executionDigest,+ outcome,+ }),+ };+ return {+ outputs: {+ production,+ facets: production.facets,+ segment: segmentOutputs,+ },+ };+ },+};diff --git a/stdlib/runtime/speech-locator.mjs b/stdlib/runtime/speech-locator.mjsnew file mode 100644index 00000000..547da910--- /dev/null+++ b/stdlib/runtime/speech-locator.mjs@@ -0,0 +1,94 @@+function normalizeWord(value) {+ return String(value)+ .normalize("NFKC")+ .toLocaleLowerCase("en")+ .replace(/[^\p{L}\p{M}\p{N}]+/gu, "");+}++function fail(message) {+ throw new Error(message);+}++export default {+ abiVersion: "1",+ project(context) {+ const narrative = context.element.attributes.script;+ const production = context.element.attributes.basis;+ const evidenceValue = context.element.attributes.evidence;+ const evidence = evidenceValue?.data ?? evidenceValue;+ if (!narrative?.semanticIndex) fail("speech-locator requires NarrativeIR");+ if (production?.contract !== "svml.temporal-basis-production.v1") {+ fail("speech-locator requires TemporalBasisProduction");+ }+ if (evidence?.contract !== "svml.speech-alignment.v1") fail("unsupported alignment evidence");+ const basis = production.basis;+ const fps = basis.frameRate.numerator / basis.frameRate.denominator;+ if (Math.round(evidence.durationSec * fps) !== basis.durationFrames) {+ fail("alignment duration does not match ProgramBasis");+ }+ if (evidence.words.length !== narrative.tokens.length) fail("alignment token cardinality mismatch");+ if (evidence.segments.length !== narrative.segments.length) fail("alignment Segment cardinality mismatch");+ const segments = new Map(evidence.segments.map(segment => [segment.id, segment]));+ if (segments.size !== evidence.segments.length) fail("alignment repeats a Segment id");+ const words = new Map();+ const previousBySegment = new Map();+ for (const [index, token] of narrative.tokens.entries()) {+ const word = evidence.words[index];+ const segment = segments.get(token.segmentId);+ if (!word || !segment || word.segmentId !== token.segmentId || normalizeWord(word.text) !== token.normalized) {+ fail(`alignment token ${index + 1} does not match Script`);+ }+ if (word.startSec < segment.startSec || word.endSec < word.startSec || word.endSec > segment.endSec) {+ fail(`alignment token ${index + 1} falls outside Segment`);+ }+ const previous = previousBySegment.get(word.segmentId);+ if (previous && word.startSec < previous.endSec - 1e-6) fail(`alignment overlaps inside Segment ${word.segmentId}`);+ previousBySegment.set(word.segmentId, word);+ words.set(token.id, word);+ }+ for (const [index, scriptSegment] of narrative.segments.entries()) {+ const segment = segments.get(scriptSegment.id);+ if (!segment) fail(`alignment is missing Segment ${scriptSegment.id}`);+ const previousScript = narrative.segments[index - 1];+ const previous = previousScript ? segments.get(previousScript.id) : undefined;+ if (previous && (segment.startSec < previous.startSec || segment.endSec < previous.endSec)) {+ fail(`alignment reorders Segment ${scriptSegment.id}`);+ }+ }+ const anchors = narrative.semanticIndex.anchors.map(anchor => {+ const segment = segments.get(anchor.segmentId);+ const word = anchor.tokenId ? words.get(anchor.tokenId) : undefined;+ const seconds = anchor.kind === "segment-start"+ ? segment?.startSec+ : anchor.kind === "segment-end"+ ? segment?.endSec+ : anchor.kind === "token-start"+ ? word?.startSec+ : word?.endSec;+ if (seconds === undefined) fail(`alignment is missing anchor ${anchor.id}`);+ return {+ identity: anchor.id,+ point: { basisDigest: basis.basisDigest, frame: Math.round(seconds * fps) },+ quality: "measured",+ };+ });+ const payload = {+ contract: "svml.exact-semantic-map.v1",+ semanticIndexDigest: narrative.semanticIndex.digest,+ basisDigest: basis.basisDigest,+ anchors,+ evidenceDigests: [evidenceValue.contentDigest ?? context.digest(evidence)],+ locatorDigest: context.digest({+ implementation: context.instance.executionDigest,+ quantizationPolicy: "nearest-frame",+ }),+ quantizationPolicy: "nearest-frame",+ ...(evidence.captionCues ? { captionCues: evidence.captionCues } : {}),+ };+ return {+ outputs: {+ map: { ...payload, mapDigest: context.digest(payload) },+ },+ };+ },+};diff --git a/stdlib/runtime/speech-spine.mjs b/stdlib/runtime/speech-spine.mjsdeleted file mode 100644index f98fc712..00000000--- a/stdlib/runtime/speech-spine.mjs+++ /dev/null@@ -1,49 +0,0 @@-import { elements, material, resolveAttr, stringAttr } from "./helpers.mjs";--export default {- abiVersion: "1",- project(context) {- const segmentOutputs = {};- const audios = [];- for (const segment of elements(context, "segment")) {- const id = stringAttr(segment, "id");- const located = resolveAttr(context, segment, "script");- const visual = material(resolveAttr(context, segment, "visual"), "video");- const audioValue = segment.attributes.audio- ? material(resolveAttr(context, segment, "audio"), "audio")- : { source: visual.source };- segmentOutputs[id] = {- range: located,- visual: {- type: "Video",- source: visual.source,- range: located,- },- audio: {- type: "Audio",- source: audioValue.source,- range: located,- },- };- audios.push({- id: `${context.instance.id}:speech:${id}`,- source: audioValue.source,- startFrame: located.startFrame,- endFrameExclusive: located.endFrameExclusive,- volume: 1,- bus: "speech",- });- }- return {- outputs: {- segment: segmentOutputs,- speech: {- segments: segmentOutputs,- primaryAudio: audios,- },- track: { visuals: [], audios, styles: [] },- },- audios,- };- },-};diff --git a/stdlib/speech-assemble.svk b/stdlib/speech-assemble.svknew file mode 100644index 00000000..20a5b5c2--- /dev/null+++ b/stdlib/speech-assemble.svk@@ -0,0 +1,23 @@+<kernel name="speech-assemble" abi="1" implementation="./runtime/speech-assemble.mjs">+ <param name="fps" type="number" default="30"/>+ <param name="defaultJoin" style="default-join" type="string" default="cut"/>+ <param name="defaultJoinDuration" style="default-join-duration" type="string" default="0ms"/>+ <port name="production" direction="output" type="TemporalBasisProduction" cardinality="one"/>+ <port name="facets.visual" direction="output" type="ProgramBoundVideoSequence" cardinality="zero-or-one"/>+ <port name="segment.*.visual" direction="output" type="ProgramBoundVideo" cardinality="zero-or-one"/>+ <port name="segment.*.audio" direction="output" type="ProgramBoundAudio" cardinality="one"/>+ <child name="segment" cardinality="one-or-more">+ <field name="id" type="id" required="true"/>+ <field name="script" type="ScriptSegment" required="true"/>+ <field name="visual" type="Video"/>+ <field name="audio" type="Audio" required="true"/>+ <field name="sourceWindow" type="string" required="true"/>+ </child>+ <child name="join" cardinality="zero-or-more">+ <field name="after" type="id" required="true"/>+ <field name="overlap" type="string"/>+ <field name="gap" type="string"/>+ <field name="audio" type="string"/>+ <field name="visual" type="string"/>+ </child>+</kernel>diff --git a/stdlib/speech-locator.svk b/stdlib/speech-locator.svknew file mode 100644index 00000000..34b36f0c--- /dev/null+++ b/stdlib/speech-locator.svk@@ -0,0 +1,6 @@+<kernel name="speech-locator" abi="1" implementation="./runtime/speech-locator.mjs">+ <port name="script" direction="input" type="NarrativeIR" cardinality="one"/>+ <port name="basis" direction="input" type="TemporalBasisProduction" cardinality="one"/>+ <port name="evidence" direction="input" type="AlignmentEvidence" cardinality="one"/>+ <port name="map" direction="output" type="ExactSemanticMap" cardinality="one"/>+</kernel>diff --git a/stdlib/speech-spine.svk b/stdlib/speech-spine.svkdeleted file mode 100644index e2ab8e4c..00000000--- a/stdlib/speech-spine.svk+++ /dev/null@@ -1,14 +0,0 @@-<kernel name="speech-spine" abi="1" implementation="./runtime/speech-spine.mjs">- <port name="segment" direction="input" type="SegmentBinding" cardinality="one-or-more"/>- <port name="segment" direction="output" type="SegmentMaterial" cardinality="one-or-more"/>- <port name="segment.*.visual" direction="output" type="Video" cardinality="zero-or-one"/>- <port name="segment.*.audio" direction="output" type="Audio" cardinality="one"/>- <port name="speech" direction="output" type="SpeechSpine" cardinality="one"/>- <port name="track" direction="output" type="Track" cardinality="one"/>- <child name="segment" cardinality="one-or-more">- <field name="id" type="id" required="true"/>- <field name="script" type="ScriptSegment" required="true"/>- <field name="visual" type="Video" required="true"/>- <field name="audio" type="Audio"/>- </child>-</kernel>diff --git a/stdlib/text-track.svk b/stdlib/text-track.svkindex f2a1a488..e25358b8 100644--- a/stdlib/text-track.svk+++ b/stdlib/text-track.svk@@ -10,12 +10,12 @@<field name="y" type="number"/><field name="width" type="number"/><field name="height" type="number"/>- <field name="font" type="string"/>- <field name="size" type="number"/>- <field name="weight" type="number"/>- <field name="lineHeight" type="number"/>- <field name="letterSpacing" type="number"/>- <field name="color" type="string"/>+ <field name="font" style="font-family" type="string"/>+ <field name="size" style="font-size" type="number"/>+ <field name="weight" style="font-weight" type="number"/>+ <field name="lineHeight" style="line-height" type="number"/>+ <field name="letterSpacing" style="letter-spacing" type="number"/>+ <field name="color" style="fill" type="string"/><field name="background" type="string"/><field name="uppercase" type="boolean"/></child>diff --git a/test/artifacts.test.ts b/test/artifacts.test.tsindex a9174d03..987a4725 100644--- a/test/artifacts.test.ts+++ b/test/artifacts.test.ts@@ -6,7 +6,7 @@ import { join } from "node:path";import test from "node:test";import { fileURLToPath, pathToFileURL } from "node:url";import { checkSource, compileSource } from "../src/compiler.js";-import { sha256 } from "../src/util.js";+import { sha256, stableJson } from "../src/util.js";const fixture = fileURLToPath(new URL("./fixtures/artifacts/", import.meta.url));const pinnedImage = fileURLToPath(@@ -27,7 +27,6 @@ test("capability instances require exact verified artifacts and never invoke a pawait assert.rejects(compileSource({file,- evidenceFile: `${fixture}/alignment.json`,}),/artifact_file_required/u,);@@ -54,16 +53,27 @@ test("capability instances require exact verified artifacts and never invoke a pconst compilation = await compileSource({file,- evidenceFile: `${fixture}/alignment.json`,lockFile: lock,artifactsFile: artifacts,outputFile: output,});assert.equal(compilation.artifactsVerified, true);- assert.equal(compilation.reproducible, true);+ assert.equal(compilation.sourceClosureVerified, true);+ assert.equal(compilation.temporalEvidenceVerified, false);+ assert.equal(compilation.reproducible, false);assert.match(compilation.html, /bound-image/u);assert.match(compilation.html, /\.\/assets\/[a-f0-9]{16}-image\.png/u);+ await writeFile(lock, `${JSON.stringify(compilation.lock, null, 2)}\n`, "utf8");+ const frozen = await compileSource({+ file,+ lockFile: lock,+ artifactsFile: artifacts,+ });+ assert.equal(frozen.lockVerified, true);+ assert.equal(frozen.temporalEvidenceVerified, true);+ assert.equal(frozen.reproducible, true);+const alternateArtifacts = join(directory, "alternate-artifacts.json");await writeFile(alternateArtifacts, `${JSON.stringify({contract: "svml.artifacts.v1",@@ -81,8 +91,6 @@ test("capability instances require exact verified artifacts and never invoke a p}, null, 2)}\n`, "utf8");const alternate = await compileSource({file,- evidenceFile: `${fixture}/alignment.json`,- lockFile: lock,artifactsFile: alternateArtifacts,});assert.equal(@@ -94,3 +102,65 @@ test("capability instances require exact verified artifacts and never invoke a palternate.plan.instances.find((instance) => instance.id === "film")?.executionDigest,);});++test("a Locator capability returns ExactSemanticMap through the same typed Artifact ABI", async () => {+ const source = fileURLToPath(+ new URL("../examples/composite-speech-program/minimal.svml", import.meta.url),+ );+ const reference = await compileSource({ file: source });+ const directory = await mkdtemp(join(tmpdir(), "svml-locator-artifact-"));+ const svml = join(directory, "main.svml");+ const assemble = fileURLToPath(new URL("../stdlib/speech-assemble.svk", import.meta.url));+ const film = fileURLToPath(new URL("../stdlib/film.svk", import.meta.url));+ const locator = fileURLToPath(+ new URL("./fixtures/artifacts/pinned-locator.svk", import.meta.url),+ );+ const avatar = fileURLToPath(+ new URL("../examples/regen-ranking/assets/avatar-1.mp4", import.meta.url),+ );+ const audio = fileURLToPath(+ new URL("../examples/regen-ranking/assets/avatar-1.mp3", import.meta.url),+ );+ const evidence = fileURLToPath(+ new URL("../examples/composite-speech-program/alignment.json", import.meta.url),+ );+ await writeFile(svml, `<svml version="1">+ <import from="${assemble}"/>+ <import from="${locator}"/>+ <import from="${film}"/>+ <script>+ <segment id="intro"><HOST> Hello, this is SVML.</segment>+ <segment id="close"><HOST> Write the script, get the video.</segment>+ </script>+ <video id="host" src="${avatar}"/>+ <audio id="host-audio" src="${audio}"/>+ <alignment id="alignment" src="${evidence}"/>+ <speech-assemble id="voice" fps="30" defaultJoin="cut">+ <segment id="intro" script={script.segment.intro} visual={host} audio={host-audio} sourceWindow="0s .. 5s"/>+ <segment id="close" script={script.segment.close} visual={host} audio={host-audio} sourceWindow="5s .. 10s"/>+ <join after="intro" overlap="500ms" audio="crossfade" visual="dissolve"/>+ </speech-assemble>+ <pinned-locator id="location" script={script} basis={voice.production} evidence={alignment}/>+ <film id="film" basis={voice.production} semantic={location.map}/>+</svml>\n`, "utf8");+ const checked = await checkSource(svml);+ const instance = checked.plan.instances.find((item) => item.id === "location");+ const artifacts = join(directory, "artifacts.json");+ await writeFile(artifacts, `${JSON.stringify({+ contract: "svml.artifacts.v1",+ bindings: [{+ instanceIdentity: instance?.identity,+ executionDigest: instance?.executionDigest,+ outputs: {+ map: {+ type: "ExactSemanticMap",+ value: reference.semanticMap,+ contentDigest: sha256(stableJson(reference.semanticMap)),+ },+ },+ }],+ }, null, 2)}\n`, "utf8");+ const compilation = await compileSource({ file: svml, artifactsFile: artifacts });+ assert.equal(compilation.semanticMap.mapDigest, reference.semanticMap.mapDigest);+ assert.equal(compilation.semanticMap.basisDigest, compilation.basis.basis.basisDigest);+});diff --git a/test/composite.test.ts b/test/composite.test.tsnew file mode 100644index 00000000..a3ea210a--- /dev/null+++ b/test/composite.test.ts@@ -0,0 +1,54 @@+import assert from "node:assert/strict";+import test from "node:test";+import { fileURLToPath } from "node:url";+import { checkSource, compileSource } from "../src/compiler.js";++const fixture = fileURLToPath(+ new URL("../examples/composite-speech-program/minimal.svml", import.meta.url),+);++test("composite-v1 expands finite internal Basis/Locator instances with audited aliases", async () => {+ const checked = await checkSource(fixture);+ assert.deepEqual(+ checked.plan.instances.map((instance) => instance.id).sort(),+ ["aroll", "main", "voice::basis", "voice::locator"],+ );+ assert.equal(checked.plan.expansions.length, 1);+ assert.deepEqual(checked.plan.expansions[0]?.internalIds, [+ "voice::basis",+ "voice::locator",+ ]);+ assert.deepEqual(checked.plan.expansions[0]?.exports, {+ production: "voice::basis.production",+ map: "voice::locator.map",+ "facets.visual": "voice::basis.facets.visual",+ });+ assert.equal(checked.lock.expansions[0]?.expansionDigest.length, 64);+ const internal = checked.plan.instances.find((instance) => instance.id === "voice::basis");+ assert.match(internal?.sourceMap?.callSite.file ?? "", /minimal\.svml$/u);+ assert.match(internal?.sourceMap?.definitionSite.file ?? "", /speech-program\.svk$/u);++ const compilation = await compileSource({ file: fixture });+ assert.equal(compilation.located.durationFrames, 285);+ assert.equal(compilation.basis.sourceMaps.length, 4);+ assert.equal(compilation.basis.audio[0]?.fadeOutFrames, 15);+ assert.equal(compilation.basis.audio[1]?.fadeInFrames, 15);++ const visuals = compilation.projections.get("aroll")?.visuals ?? [];+ assert.deepEqual(+ visuals.map((visual) => [visual.startFrame, visual.endFrameExclusive, visual.mediaStartSec]),+ [[0, 150, 0], [135, 285, 5]],+ );+ assert.ok(visuals.every((visual) => visual.z === 100));+ assert.ok(visuals.every((visual) => /svml-program-bound/u.test(visual.css ?? "")));+});++test("a precomposed media Basis reuses the same Locator and Composition contracts", async () => {+ const file = fileURLToPath(new URL("./fixtures/media-basis/main.svml", import.meta.url));+ const compilation = await compileSource({ file });+ assert.equal(compilation.basis.basis.durationFrames, 30);+ assert.equal(compilation.basis.alignmentSubjects.length, 1);+ assert.equal(compilation.basis.sourceMaps.length, 2);+ assert.equal(compilation.semanticMap.anchors.length, 4);+ assert.equal(compilation.target.audios.length, 1);+});diff --git a/test/fixtures/artifacts/artifact-film.mjs b/test/fixtures/artifacts/artifact-film.mjsindex 15d72e75..5368690a 100644--- a/test/fixtures/artifacts/artifact-film.mjs+++ b/test/fixtures/artifacts/artifact-film.mjs@@ -2,6 +2,7 @@ export default {abiVersion: "1",project(context) {const image = context.resolve(context.element.attributes.image);+ const production = context.resolve(context.element.attributes.basis);return {outputs: {document: {@@ -9,15 +10,15 @@ export default {id: context.instance.id,width: 64,height: 64,- fps: context.fps,- durationFrames: context.located.durationFrames,+ fps: production.basis.frameRate.numerator / production.basis.frameRate.denominator,+ durationFrames: production.basis.durationFrames,background: "#000000",visuals: [{id: "bound-image",kind: "image",source: image.absoluteSource ?? image.source,startFrame: 0,- endFrameExclusive: context.located.durationFrames,+ endFrameExclusive: production.basis.durationFrames,z: 0,}],audios: [],diff --git a/test/fixtures/artifacts/artifact-film.svk b/test/fixtures/artifacts/artifact-film.svkindex 8f93ad04..b1611e71 100644--- a/test/fixtures/artifacts/artifact-film.svk+++ b/test/fixtures/artifacts/artifact-film.svk@@ -5,5 +5,7 @@implementation="./artifact-film.mjs"><port name="image" direction="input" type="Image" cardinality="one"/>+ <port name="basis" direction="input" type="TemporalBasisProduction" cardinality="one"/>+ <port name="semantic" direction="input" type="ExactSemanticMap" cardinality="one"/><port name="document" direction="output" type="HyperframesDocument" cardinality="one"/></kernel>diff --git a/test/fixtures/artifacts/main.svml b/test/fixtures/artifacts/main.svmlindex 730bb240..e8a49887 100644--- a/test/fixtures/artifacts/main.svml+++ b/test/fixtures/artifacts/main.svml@@ -1,11 +1,20 @@<svml version="1"><import from="./pinned-image.svk"/><import from="./artifact-film.svk"/>+ <import from="../../../stdlib/speech-assemble.svk"/>+ <import from="../../../stdlib/speech-locator.svk"/><script><segment id="main">Hello.</segment></script>+ <audio id="voice-audio" src="./alignment.json"/>+ <alignment id="alignment" src="./alignment.json"/>+ <speech-assemble id="speech" fps="30">+ <segment id="main" script={script.segment.main} audio={voice-audio} sourceWindow="0s .. 1s"/>+ </speech-assemble>+ <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/>+<pinned-image id="poster" prompt="A pinned result; never call a provider."/>- <artifact-film id="film" image={poster.image}/>+ <artifact-film id="film" image={poster.image} basis={speech.production} semantic={location.map}/></svml>diff --git a/test/fixtures/artifacts/pinned-locator.svk b/test/fixtures/artifacts/pinned-locator.svknew file mode 100644index 00000000..6296311c--- /dev/null+++ b/test/fixtures/artifacts/pinned-locator.svk@@ -0,0 +1,6 @@+<kernel name="pinned-locator" abi="1" profile="capability-v1" capability="semantic-locate">+ <port name="script" direction="input" type="NarrativeIR" cardinality="one"/>+ <port name="basis" direction="input" type="TemporalBasisProduction" cardinality="one"/>+ <port name="evidence" direction="input" type="AlignmentEvidence" cardinality="one"/>+ <port name="map" direction="output" type="ExactSemanticMap" cardinality="one"/>+</kernel>diff --git a/test/fixtures/collision/main.svml b/test/fixtures/collision/main.svmlindex f813653f..0925685b 100644--- a/test/fixtures/collision/main.svml+++ b/test/fixtures/collision/main.svml@@ -1,5 +1,6 @@<svml version="1">- <import from="../../../stdlib/speech-spine.svk"/>+ <import from="../../../stdlib/speech-assemble.svk"/>+ <import from="../../../stdlib/speech-locator.svk"/><import from="../../../stdlib/broll-track.svk"/><import from="../../../stdlib/film.svk"/>@@ -10,12 +11,16 @@</script><video id="avatar" src="./avatar.mp4"/>+ <audio id="voice-audio" src="./avatar.mp4"/>+ <alignment id="alignment" src="./alignment.json"/><image id="left-image" src="./left/image.png"/><image id="right-image" src="./right/image.png"/>- <speech-spine id="speech">- <segment id="main" script={script.segment.main} visual={avatar}/>- </speech-spine>+ <speech-assemble id="speech" fps="30">+ <segment id="main" script={script.segment.main} visual={avatar} audio={voice-audio} sourceWindow="0s .. 2s"/>+ </speech-assemble>++ <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/><broll-track id="broll" z="100"><item id="left" source={left-image} during={script.selection.left}/>@@ -23,7 +28,7 @@<transition from="left" to="right" kind="crossfade" duration="200ms"/></broll-track>- <film id="collision" speech={speech.speech}>+ <film id="collision" basis={speech.production} semantic={location.map}><track ref={broll.track}/></film></svml>diff --git a/test/fixtures/disconnected/captions.svml b/test/fixtures/disconnected/captions.svmlnew file mode 100644index 00000000..9081f409--- /dev/null+++ b/test/fixtures/disconnected/captions.svml@@ -0,0 +1,25 @@+<svml version="1">+ <import from="../../../stdlib/speech-assemble.svk"/>+ <import from="../../../stdlib/speech-locator.svk"/>+ <import from="../../../stdlib/caption-track.svk"/>+ <import from="../../../stdlib/film.svk"/>++ <script>+ <segment id="main">@pick Alpha @/pick gap @pick omega @/pick.</segment>+ </script>++ <audio id="voice-audio" src="../presentation/long.mp4"/>+ <alignment id="alignment" src="./alignment.json"/>+ <speech-assemble id="speech" fps="30">+ <segment id="main" script={script.segment.main} audio={voice-audio} sourceWindow="0s .. 3s"/>+ </speech-assemble>+ <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/>++ <caption-track id="captions" script={script} semantic={location.map} z="10" maxWords="3">+ <style id="picked" during={script.selection.pick} color="#ff0000" activeColor="#00ff00" scale="1.1"/>+ </caption-track>++ <film id="film" basis={speech.production} semantic={location.map}>+ <track ref={captions.track}/>+ </film>+</svml>diff --git a/test/fixtures/disconnected/each.svml b/test/fixtures/disconnected/each.svmlindex ce79c2e5..5cc168df 100644--- a/test/fixtures/disconnected/each.svml+++ b/test/fixtures/disconnected/each.svml@@ -1,5 +1,6 @@<svml version="1">- <import from="../../../stdlib/speech-spine.svk"/>+ <import from="../../../stdlib/speech-assemble.svk"/>+ <import from="../../../stdlib/speech-locator.svk"/><import from="../../../stdlib/media-track.svk"/><import from="../../../stdlib/film.svk"/>@@ -8,16 +9,19 @@</script><video id="avatar" src="../presentation/long.mp4"/>+ <audio id="voice-audio" src="../presentation/long.mp4"/>+ <alignment id="alignment" src="./alignment.json"/>- <speech-spine id="speech">- <segment id="main" script={script.segment.main} visual={avatar}/>- </speech-spine>+ <speech-assemble id="speech" fps="30">+ <segment id="main" script={script.segment.main} visual={avatar} audio={voice-audio} sourceWindow="0s .. 3s"/>+ </speech-assemble>+ <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/><media-track id="media" z="1"><item id="picked" source={avatar} during={script.selection.pick}/></media-track>- <film id="film" speech={speech.speech}>+ <film id="film" basis={speech.production} semantic={location.map}><track ref={media.track}/></film></svml>diff --git a/test/fixtures/disconnected/one.svml b/test/fixtures/disconnected/one.svmlindex 7f0d83cf..f7e737f4 100644--- a/test/fixtures/disconnected/one.svml+++ b/test/fixtures/disconnected/one.svml@@ -1,5 +1,6 @@<svml version="1">- <import from="../../../stdlib/speech-spine.svk"/>+ <import from="../../../stdlib/speech-assemble.svk"/>+ <import from="../../../stdlib/speech-locator.svk"/><import from="../../../stdlib/ranking-column.svk"/><import from="../../../stdlib/film.svk"/>@@ -8,17 +9,20 @@</script><video id="avatar" src="../presentation/long.mp4"/>+ <audio id="voice-audio" src="../presentation/long.mp4"/><image id="icon" src="../collision/left/image.png"/>+ <alignment id="alignment" src="./alignment.json"/>- <speech-spine id="speech">- <segment id="main" script={script.segment.main} visual={avatar}/>- </speech-spine>+ <speech-assemble id="speech" fps="30">+ <segment id="main" script={script.segment.main} visual={avatar} audio={voice-audio} sourceWindow="0s .. 3s"/>+ </speech-assemble>+ <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/><ranking-column id="ranking" z="1"><item id="picked" rank="1" image={icon} during={script.selection.pick}/></ranking-column>- <film id="film" speech={speech.speech}>+ <film id="film" basis={speech.production} semantic={location.map}><track ref={ranking.track}/></film></svml>diff --git a/test/fixtures/disconnected/set.svml b/test/fixtures/disconnected/set.svmlindex dccdcc62..5f78cd9a 100644--- a/test/fixtures/disconnected/set.svml+++ b/test/fixtures/disconnected/set.svml@@ -1,5 +1,6 @@<svml version="1">- <import from="../../../stdlib/speech-spine.svk"/>+ <import from="../../../stdlib/speech-assemble.svk"/>+ <import from="../../../stdlib/speech-locator.svk"/><import from="./selection-set.svk"/><import from="../../../stdlib/film.svk"/>@@ -8,16 +9,19 @@</script><video id="avatar" src="../presentation/long.mp4"/>+ <audio id="voice-audio" src="../presentation/long.mp4"/>+ <alignment id="alignment" src="./alignment.json"/>- <speech-spine id="speech">- <segment id="main" script={script.segment.main} visual={avatar}/>- </speech-spine>+ <speech-assemble id="speech" fps="30">+ <segment id="main" script={script.segment.main} visual={avatar} audio={voice-audio} sourceWindow="0s .. 3s"/>+ </speech-assemble>+ <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/><selection-set id="styles"><item id="highlight" during={script.selection.pick}/></selection-set>- <film id="film" speech={speech.speech}>+ <film id="film" basis={speech.production} semantic={location.map}><track ref={styles.track}/></film></svml>diff --git a/test/fixtures/media-basis/main.svml b/test/fixtures/media-basis/main.svmlnew file mode 100644index 00000000..0bc1f7bc--- /dev/null+++ b/test/fixtures/media-basis/main.svml@@ -0,0 +1,14 @@+<svml version="1">+ <import from="../../../stdlib/media-basis.svk"/>+ <import from="../../../stdlib/speech-locator.svk"/>+ <import from="../../../stdlib/film.svk"/>++ <script><segment id="main">Hello.</segment></script>+ <video id="visual" src="../collision/avatar.mp4"/>+ <audio id="audio" src="../collision/avatar.mp4"/>+ <alignment id="alignment" src="../artifacts/alignment.json"/>++ <media-basis id="program" visual={visual} audio={audio} duration="1s"/>+ <speech-locator id="location" script={script} basis={program.production} evidence={alignment}/>+ <film id="film" basis={program.production} semantic={location.map}/>+</svml>diff --git a/test/fixtures/modules/house.svs b/test/fixtures/modules/house.svsindex e06a57ad..ee3945d1 100644--- a/test/fixtures/modules/house.svs+++ b/test/fixtures/modules/house.svs@@ -9,6 +9,12 @@fill: #fefefe;}+ subtitles.launch {+ z: 510;+ font-size: 44;+ fill: #fefefe;+ }+film.vertical {width: 1080;height: 1920;diff --git a/test/fixtures/modules/main-renamed-alias.svml b/test/fixtures/modules/main-renamed-alias.svmlindex 42b8d0ef..743ba685 100644--- a/test/fixtures/modules/main-renamed-alias.svml+++ b/test/fixtures/modules/main-renamed-alias.svml@@ -1,7 +1,8 @@<svml version="1">- <import from="../../../stdlib/speech-spine.svk"/>+ <import from="../../../stdlib/speech-assemble.svk"/>+ <import from="../../../stdlib/speech-locator.svk"/><import from="../../../stdlib/broll-track.svk"/>- <import from="../../../stdlib/caption-track.svk"/>+ <import as="subtitles" from="../../../stdlib/caption-track.svk"/><import from="../../../stdlib/film.svk"/><import as="assets" from="./brand.svc"/><import as="house" from="./house.svs"/>@@ -13,18 +14,22 @@</script><video id="avatar" src="./avatar.mp4"/>+ <audio id="voice-audio" src="./avatar.mp4"/>+ <alignment id="alignment" src="./alignment.json"/>- <speech-spine id="speech">- <segment id="main" script={script.segment.main} visual={avatar}/>- </speech-spine>+ <speech-assemble id="speech" fps="30">+ <segment id="main" script={script.segment.main} visual={avatar} audio={voice-audio} sourceWindow="0s .. 1s"/>+ </speech-assemble>++ <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/><broll-track id="broll" class="house.full"><item id="poster" source={assets.poster.image} during={script.selection.poster}/></broll-track>- <caption-track id="captions" script={script} class="house.launch" size="48"/>+ <subtitles id="captions" script={script} semantic={location.map} class="house.launch" size="48"/>- <film id="main-film" class="house.vertical" speech={speech.speech}>+ <film id="main-film" class="house.vertical" basis={speech.production} semantic={location.map}><track ref={broll.track}/><track ref={captions.track}/></film>diff --git a/test/fixtures/modules/main.svml b/test/fixtures/modules/main.svmlindex 671a4afc..178a9e20 100644--- a/test/fixtures/modules/main.svml+++ b/test/fixtures/modules/main.svml@@ -1,5 +1,6 @@<svml version="1">- <import from="../../../stdlib/speech-spine.svk"/>+ <import from="../../../stdlib/speech-assemble.svk"/>+ <import from="../../../stdlib/speech-locator.svk"/><import from="../../../stdlib/broll-track.svk"/><import from="../../../stdlib/caption-track.svk"/><import from="../../../stdlib/film.svk"/>@@ -13,18 +14,22 @@</script><video id="avatar" src="./avatar.mp4"/>+ <audio id="voice-audio" src="./avatar.mp4"/>+ <alignment id="alignment" src="./alignment.json"/>- <speech-spine id="speech">- <segment id="main" script={script.segment.main} visual={avatar}/>- </speech-spine>+ <speech-assemble id="speech" fps="30">+ <segment id="main" script={script.segment.main} visual={avatar} audio={voice-audio} sourceWindow="0s .. 1s"/>+ </speech-assemble>++ <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/><broll-track id="broll" class="house.full"><item id="poster" source={brand.poster.image} during={script.selection.poster}/></broll-track>- <caption-track id="captions" script={script} class="house.launch" size="48"/>+ <caption-track id="captions" script={script} semantic={location.map} class="house.launch" size="48"/>- <film id="main-film" class="house.vertical" speech={speech.speech}>+ <film id="main-film" class="house.vertical" basis={speech.production} semantic={location.map}><track ref={broll.track}/><track ref={captions.track}/></film>diff --git a/test/fixtures/presentation/empty-present.svml b/test/fixtures/presentation/empty-present.svmlnew file mode 100644index 00000000..9b46e3e2--- /dev/null+++ b/test/fixtures/presentation/empty-present.svml@@ -0,0 +1,31 @@+<svml version="1">+ <import from="../../../stdlib/speech-assemble.svk"/>+ <import from="../../../stdlib/speech-locator.svk"/>+ <import from="../../../stdlib/media-track.svk"/>+ <import from="../../../stdlib/film.svk"/>++ <script>+ <segment id="main">+ One @pop! two three.+ </segment>+ </script>++ <video id="long-video" src="./long.mp4"/>+ <audio id="voice-audio" src="./long.mp4"/>+ <alignment id="alignment" src="./alignment.json"/>++ <speech-assemble id="speech" fps="30">+ <segment id="main" script={script.segment.main} visual={long-video} audio={voice-audio} sourceWindow="0s .. 3s"/>+ </speech-assemble>+ <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/>++ <media-track id="aroll" z="200">+ <item id="host" source={long-video} during="full">+ <present id="outside" during="end+1s .. end+2s" z="230"/>+ </item>+ </media-track>++ <film id="presentation" basis={speech.production} semantic={location.map}>+ <track ref={aroll.track}/>+ </film>+</svml>diff --git a/test/fixtures/presentation/main.svml b/test/fixtures/presentation/main.svmlindex 3875d687..c41502d1 100644--- a/test/fixtures/presentation/main.svml+++ b/test/fixtures/presentation/main.svml@@ -1,5 +1,6 @@<svml version="1">- <import from="../../../stdlib/speech-spine.svk"/>+ <import from="../../../stdlib/speech-assemble.svk"/>+ <import from="../../../stdlib/speech-locator.svk"/><import from="../../../stdlib/media-track.svk"/><import from="../../../stdlib/audio-track.svk"/><import from="../../../stdlib/film.svk"/>@@ -11,18 +12,22 @@</script><video id="long-video" src="./long.mp4"/>+ <audio id="voice-audio" src="./long.mp4"/><audio id="pop-sound" src="./pop.wav"/>+ <alignment id="alignment" src="./alignment.json"/>- <speech-spine id="speech">- <segment id="main" script={script.segment.main} visual={long-video}/>- </speech-spine>+ <speech-assemble id="speech" fps="30">+ <segment id="main" script={script.segment.main} visual={long-video} audio={voice-audio} sourceWindow="0s .. 3s"/>+ </speech-assemble>++ <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/><media-track id="aroll" z="200"><item id="host" source={long-video} during="full" fit="cover"><presentid="top-circle"during="start .. start+1.2s"- layer="10"+ z="210"x="0.05"y="0.05"width="0.3"@@ -32,18 +37,13 @@<presentid="lower-split"during="start+1s .. start+2s"- layer="20"+ z="220"x="0"y="0.5"width="1"height="0.5"enter="fade 200ms"/>- <present- id="outside"- during="end+1s .. end+2s"- layer="30"- /></item></media-track>@@ -51,7 +51,7 @@<item id="pop" source={pop-sound} at={script.moment.pop} duration="200ms" bus="sfx"/></audio-track>- <film id="presentation" speech={speech.speech}>+ <film id="presentation" basis={speech.production} semantic={location.map}><track ref={aroll.track}/><track ref={sfx.track}/></film>diff --git a/test/fixtures/unsafe/main.svml b/test/fixtures/unsafe/main.svmlindex 7b384f4d..fb265c2f 100644--- a/test/fixtures/unsafe/main.svml+++ b/test/fixtures/unsafe/main.svml@@ -1,9 +1,18 @@<svml version="1"><import from="./unsafe.svk"/>+ <import from="../../../stdlib/speech-assemble.svk"/>+ <import from="../../../stdlib/speech-locator.svk"/><script><segment id="main">Hello.</segment></script>- <unsafe id="main"/>+ <audio id="voice-audio" src="./alignment.json"/>+ <alignment id="alignment" src="./alignment.json"/>+ <speech-assemble id="speech" fps="30">+ <segment id="main" script={script.segment.main} audio={voice-audio} sourceWindow="0s .. 1s"/>+ </speech-assemble>+ <speech-locator id="location" script={script} basis={speech.production} evidence={alignment}/>++ <unsafe id="main" basis={speech.production} semantic={location.map}/></svml>diff --git a/test/fixtures/unsafe/unsafe.svk b/test/fixtures/unsafe/unsafe.svkindex 8e7466e7..dbc423bd 100644--- a/test/fixtures/unsafe/unsafe.svk+++ b/test/fixtures/unsafe/unsafe.svk@@ -1,3 +1,5 @@<kernel name="unsafe" abi="1" root="true" implementation="./unsafe.mjs">+ <port name="basis" direction="input" type="TemporalBasisProduction" cardinality="one"/>+ <port name="semantic" direction="input" type="ExactSemanticMap" cardinality="one"/><port name="document" direction="output" type="HyperframesDocument" cardinality="one"/></kernel>diff --git a/test/flat-track-launch.test.ts b/test/flat-track-launch.test.tsnew file mode 100644index 00000000..67211e76--- /dev/null+++ b/test/flat-track-launch.test.ts@@ -0,0 +1,62 @@+import assert from "node:assert/strict";+import test from "node:test";+import { fileURLToPath } from "node:url";+import { compileSource } from "../src/compiler.js";+import type { SourceElement } from "../src/model.js";++const example = fileURLToPath(new URL("../examples/flat-track-launch/flat-track-launch.svml", import.meta.url));++function elements(nodes: SourceElement["children"], name: string): SourceElement[] {+ return nodes.filter((node): node is SourceElement => node.kind === "element" && node.name === name);+}++test("the flat Track launch fixture closes the complete v1 compilation contract", async () => {+ const compilation = await compileSource({ file: example });++ assert.equal(compilation.located.durationFrames, 1083);+ assert.equal(+ compilation.narrative.semanticIndex.anchors.length,+ compilation.narrative.tokens.length * 2 + compilation.narrative.segments.length * 2,+ );+ assert.equal(compilation.located.selections.emphasis?.ranges.length, 2);+ assert.equal(compilation.target.width, 1080);+ assert.equal(compilation.target.height, 1920);++ const arroll = compilation.plan.instances.find((instance) => instance.id === "aroll");+ const item = elements(arroll?.children ?? [], "item")[0];+ const openingLeft = elements(item?.children ?? [], "present")+ .find((present) => present.attributes.id === "opening-left");+ assert.equal(openingLeft?.attributes.shape, "circle");+ assert.equal(openingLeft?.attributes.width, 0.3);++ const overlays = compilation.plan.instances.find((instance) => instance.id === "overlays");+ const title = elements(overlays?.children ?? [], "item")+ .find((candidate) => candidate.attributes.id === "opening-title");+ assert.equal(title?.attributes.size, 64);+ assert.equal(title?.attributes.color, "#FFFFFF");++ const broll = compilation.projections.get("broll");+ const productVisual = broll?.visuals?.find((fragment) => fragment.id.includes(":product-demo:"));+ const productAudio = broll?.audios?.find((fragment) => fragment.id.includes(":product-demo:"));+ assert.equal(productAudio?.playbackRate, productVisual?.playbackRate);+ assert.equal(productVisual?.muted, true);+ assert.equal(broll?.audios?.some((fragment) => fragment.bus === "sfx"), true);++ const board = compilation.projections.get("ranking")?.visuals+ ?.find((fragment) => fragment.id === "ranking:board");+ assert.equal(board?.startFrame, compilation.located.moments.ranking?.frames[0]);++ const text = compilation.projections.get("overlays")?.visuals ?? [];+ assert.deepEqual(+ text.map((fragment) => [fragment.id, fragment.startFrame, fragment.endFrameExclusive]),+ [+ ["overlays:opening-title:0", 0, 90],+ ["overlays:ending-arrow:0", 1023, 1083],+ ],+ );++ const captions = compilation.projections.get("captions")?.visuals ?? [];+ assert.ok(captions.some((fragment) => /#FDE047/u.test(fragment.html ?? "")));+ assert.ok(captions.some((fragment) => /scale\(1\.08\)/u.test(fragment.html ?? "")));+ assert.match(compilation.html, /data-composition-id="launch"/u);+});diff --git a/test/kernel-contracts.test.ts b/test/kernel-contracts.test.tsindex d4811fe5..495b7ae9 100644--- a/test/kernel-contracts.test.ts+++ b/test/kernel-contracts.test.ts@@ -10,7 +10,7 @@ const brollKernel = fileURLToPath(new URL("../stdlib/broll-track.svk", import.meta.url),);const unsafeKernel = fileURLToPath(- new URL("./fixtures/unsafe/unsafe.svk", import.meta.url),+ new URL("./fixtures/artifacts/artifact-film.svk", import.meta.url),);async function checkBody(body: string): Promise<ReturnType<typeof checkSource>> {@@ -22,7 +22,7 @@ async function checkBody(body: string): Promise<ReturnType<typeof checkSource>><script><segment id="main">Hello.</segment></script><audio id="sound" src="./missing.wav"/>${body}- <unsafe id="root"/>+ <artifact-film id="root" image={sound} basis={sound} semantic={sound}/></svml>`, "utf8");return checkSource(file);diff --git a/test/modules.test.ts b/test/modules.test.tsindex 94033472..74df9f57 100644--- a/test/modules.test.ts+++ b/test/modules.test.ts@@ -16,11 +16,11 @@ test(".svc content DAG and .svs parameter classes bind into one source closure",assert.deepEqual(checked.plan.instances.map((instance) => instance.id).sort(),- ["brand.poster", "broll", "captions", "main-film", "speech"],+ ["brand.poster", "broll", "captions", "location", "main-film", "speech"],);assert.deepEqual(checked.plan.values.map((value) => value.id).sort(),- ["avatar", "brand.logo"],+ ["alignment", "avatar", "brand.logo", "voice-audio"],);const logo = checked.plan.values.find((value) => value.id === "brand.logo");assert.equal(logo?.localId, "logo");@@ -42,7 +42,6 @@ test(".svc content DAG and .svs parameter classes bind into one source closure",const compilation = await compileSource({file,- evidenceFile: `${fixtures}/alignment.json`,});assert.match(compilation.html, /data-composition-id="main-film"/u);assert.match(compilation.html, /#112233/u);@@ -75,10 +74,14 @@ test("svml.lock verifies the exact source closure and Kernel implementations", aconst verified = await compileSource({file,- evidenceFile: `${fixtures}/alignment.json`,lockFile: lock,});- assert.equal(verified.lockVerified, true);+ assert.equal(verified.sourceClosureVerified, true);+ assert.equal(verified.lockVerified, false);+ await writeFile(lock, `${JSON.stringify(verified.lock, null, 2)}\n`, "utf8");+ const frozen = await compileSource({ file, lockFile: lock });+ assert.equal(frozen.lockVerified, true);+ assert.equal(frozen.lock.execution?.htmlDigest.length, 64);const changed = structuredClone(checked.lock);changed.kernels[0]!.implementationHash = "0".repeat(64);@@ -86,7 +89,6 @@ test("svml.lock verifies the exact source closure and Kernel implementations", aawait assert.rejects(compileSource({file,- evidenceFile: `${fixtures}/alignment.json`,lockFile: lock,}),(error: unknown) =>@@ -109,7 +111,6 @@ test("asset staging uses content-addressed names and never overwrites equal baseawait mkdir(directory, { recursive: true });await compileSource({file: `${collision}/main.svml`,- evidenceFile: `${collision}/alignment.json`,outputFile: output,});diff --git a/test/presentation.test.ts b/test/presentation.test.tsindex 9b79882b..60c990b1 100644--- a/test/presentation.test.ts+++ b/test/presentation.test.ts@@ -4,11 +4,10 @@ import { fileURLToPath } from "node:url";import { compileSource } from "../src/compiler.js";import type { VisualFragment } from "../src/runtime-contract.js";-test("Present ranges intersect the Item, overlap by layer, and keep one source-time map", async () => {+test("Present ranges intersect the Item, overlap by absolute z, and keep one source-time map", async () => {const fixture = fileURLToPath(new URL("./fixtures/presentation/", import.meta.url));const compilation = await compileSource({file: `${fixture}/main.svml`,- evidenceFile: `${fixture}/alignment.json`,});const visuals = compilation.projections.get("aroll")?.visuals ?? [];assert.equal(visuals.length, 3);@@ -16,13 +15,13 @@ test("Present ranges intersect the Item, overlap by layer, and keep one source-tvisuals.map((fragment) => [fragment.startFrame,fragment.endFrameExclusive,- fragment.layer,+ fragment.z,fragment.mediaStartSec,]),[- [60, 90, 0, 2],- [0, 36, 10, 0],- [30, 60, 20, 1],+ [60, 90, 200, 2],+ [0, 36, 210, 0],+ [30, 60, 220, 1],],);assert.equal(visuals[1]?.style?.["border-radius"], "50%");@@ -38,11 +37,18 @@ test("Present ranges intersect the Item, overlap by layer, and keep one source-t);});+test("a Present with no parent intersection fails instead of disappearing", async () => {+ const fixture = fileURLToPath(new URL("./fixtures/presentation/", import.meta.url));+ await assert.rejects(+ compileSource({ file: `${fixture}/empty-present.svml` }),+ /presentation_empty/u,+ );+});+test("B-roll crossfade extends only the visual surfaces and leaves source audio independent", async () => {const fixture = fileURLToPath(new URL("./fixtures/collision/", import.meta.url));const compilation = await compileSource({file: `${fixture}/main.svml`,- evidenceFile: `${fixture}/alignment.json`,});const broll = compilation.projections.get("broll");const visuals = (broll?.visuals ?? []) as VisualFragment[];@@ -59,7 +65,6 @@ test("temporal one/each cardinality is declared by the Kernel contract", async (const fixture = fileURLToPath(new URL("./fixtures/disconnected/", import.meta.url));const each = await compileSource({file: `${fixture}/each.svml`,- evidenceFile: `${fixture}/alignment.json`,});assert.equal(each.projections.get("media")?.visuals?.length, 2);const media = each.plan.kernels.find((kernel) => kernel.name === "media-track");@@ -67,7 +72,6 @@ test("temporal one/each cardinality is declared by the Kernel contract", async (const set = await compileSource({file: `${fixture}/set.svml`,- evidenceFile: `${fixture}/alignment.json`,});assert.deepEqual(set.projections.get("styles")?.diagnostics, [{code: "selection_set_count",@@ -77,8 +81,17 @@ test("temporal one/each cardinality is declared by the Kernel contract", async (await assert.rejects(compileSource({file: `${fixture}/one.svml`,- evidenceFile: `${fixture}/alignment.json`,}),/temporal cardinality expected one, received 2/u,);});++test("disconnected SelectionSet drives one caption style rule without losing occurrences", async () => {+ const fixture = fileURLToPath(new URL("./fixtures/disconnected/", import.meta.url));+ const compilation = await compileSource({ file: `${fixture}/captions.svml` });+ const captions = compilation.projections.get("captions")?.visuals ?? [];+ assert.ok(captions.length > 0);+ assert.ok(captions.some((fragment) => /color:#ff0000/u.test(fragment.html ?? "")));+ assert.ok(captions.some((fragment) => /color:#00ff00/u.test(fragment.html ?? "")));+ assert.ok(captions.some((fragment) => /scale\(1\.1\)/u.test(fragment.html ?? "")));+});diff --git a/test/regen-ranking.test.ts b/test/regen-ranking.test.tsindex a5a85e03..a6d64b0f 100644--- a/test/regen-ranking.test.ts+++ b/test/regen-ranking.test.ts@@ -20,7 +20,6 @@ test("regen ranking compiles to the frozen production timing contract", async ()) as Reference;const compilation = await compileSource({file: `${example}/regen-ranking.svml`,- evidenceFile: `${example}/evidence/alignment.json`,});assert.equal(compilation.located.fps, reference.clock.fps);diff --git a/test/runtime-security.test.ts b/test/runtime-security.test.tsindex 1c11f003..be80afba 100644--- a/test/runtime-security.test.ts+++ b/test/runtime-security.test.ts@@ -42,7 +42,6 @@ test("isolated projector ABI rejects Node builtin imports", async () => {await assert.rejects(compileSource({file: `${fixture}/main.svml`,- evidenceFile: `${fixture}/alignment.json`,}),(error: unknown) =>error instanceof SvmlErrordiff --git a/test/script.test.ts b/test/script.test.tsindex fb4a4aa3..6b3271c9 100644--- a/test/script.test.ts+++ b/test/script.test.ts@@ -1,11 +1,28 @@import assert from "node:assert/strict";import test from "node:test";-import type { AlignmentEvidence } from "../src/model.js";+import type { AlignmentEvidence, NarrativeIR } from "../src/model.js";import { locateScript } from "../src/script/locate.js";import { estimateAlignment } from "../src/script/estimate.js";import { parseScript } from "../src/script/parse.js";import { formatDocumentScript } from "../src/format.js";import { parseDocumentSource } from "../src/source/parse-document.js";+import {+ createProgramBasis,+ semanticMapFromAlignment,+ validateSemanticMap,+} from "../src/temporal.js";+import { sha256, stableJson } from "../src/util.js";++function locateAlignment(narrative: NarrativeIR, evidence: AlignmentEvidence) {+ const basis = createProgramBasis({+ fps: evidence.fps,+ durationFrames: Math.round(evidence.durationSec * evidence.fps),+ outcomeDigest: sha256(stableJson(evidence)),+ });+ const map = semanticMapFromAlignment(narrative, basis, evidence, { precision: "exact" });+ assert.equal(map.contract, "svml.exact-semantic-map.v1");+ return locateScript(narrative, basis, map);+}test("Script Surface projects role, dual text, selections and moments", () => {const narrative = parseScript("fixture.svml", `@@ -32,6 +49,130 @@ test("Script Surface projects role, dual text, selections and moments", () => {);});+test("Script v1 builds independent 2M + 2N anchor identities", () => {+ const narrative = parseScript("fixture.svml", `+ <segment id="a">Alpha beta.</segment>+ <segment id="b"/>+ <segment id="c">Gamma.</segment>+ `);+ assert.equal(narrative.tokens.length, 3);+ assert.equal(narrative.segments.length, 3);+ assert.equal(narrative.semanticIndex.anchors.length, 2 * 3 + 2 * 3);+ assert.deepEqual(+ narrative.semanticIndex.anchors.map((anchor) => anchor.id),+ [+ "segment:a:start",+ "segment:a:token:1:start",+ "segment:a:token:1:end",+ "segment:a:token:2:start",+ "segment:a:token:2:end",+ "segment:a:end",+ "segment:b:start",+ "segment:b:end",+ "segment:c:start",+ "segment:c:token:1:start",+ "segment:c:token:1:end",+ "segment:c:end",+ ],+ );+ assert.notEqual(+ narrative.segments[0]?.endAnchorId,+ narrative.segments[1]?.startAnchorId,+ );+ assert.match(narrative.semanticIndex.digest, /^[a-f0-9]{64}$/u);+});++test("overlapping Segments retain distinct boundary affinity on one ProgramBasis", () => {+ const narrative = parseScript("fixture.svml", `+ <segment id="a">Alpha.</segment>+ ~@left! @right!+ <segment id="b">Beta.</segment>+ `);+ const evidence: AlignmentEvidence = {+ contract: "svml.speech-alignment.v1",+ durationSec: 8,+ fps: 30,+ words: [+ { text: "Alpha", startSec: 0.2, endSec: 4.8, segmentId: "a" },+ { text: "Beta", startSec: 4.6, endSec: 7.8, segmentId: "b" },+ ],+ segments: [+ { id: "a", startSec: 0, endSec: 5 },+ { id: "b", startSec: 4.5, endSec: 8 },+ ],+ };+ const located = locateAlignment(narrative, evidence);+ assert.deepEqual(located.moments.left?.frames, [150]);+ assert.deepEqual(located.moments.right?.frames, [135]);+ assert.equal(located.segments.a?.endFrameExclusive, 150);+ assert.equal(located.segments.b?.startFrame, 135);+});++test("SemanticMap rejects source-order reversal, wrong basis and digest tampering", () => {+ const narrative = parseScript(+ "fixture.svml",+ `<segment id="a">Alpha.</segment><segment id="b">Beta.</segment>`,+ );+ const evidence: AlignmentEvidence = {+ contract: "svml.speech-alignment.v1",+ durationSec: 5,+ fps: 30,+ words: [+ { text: "Alpha", startSec: 1.1, endSec: 1.8, segmentId: "a" },+ { text: "Beta", startSec: 0.2, endSec: 0.8, segmentId: "b" },+ ],+ segments: [+ { id: "a", startSec: 1, endSec: 2 },+ { id: "b", startSec: 0, endSec: 1 },+ ],+ };+ const basis = createProgramBasis({+ fps: 30,+ durationFrames: 150,+ outcomeDigest: sha256("reordered"),+ });+ assert.throws(+ () => semanticMapFromAlignment(narrative, basis, evidence, { precision: "exact" }),+ /locator_segment_order/u,+ );++ const validEvidence: AlignmentEvidence = {+ ...evidence,+ words: [+ { text: "Alpha", startSec: 0.1, endSec: 0.8, segmentId: "a" },+ { text: "Beta", startSec: 1.1, endSec: 1.8, segmentId: "b" },+ ],+ segments: [+ { id: "a", startSec: 0, endSec: 1 },+ { id: "b", startSec: 1, endSec: 2 },+ ],+ };+ const validMap = semanticMapFromAlignment(+ narrative,+ basis,+ validEvidence,+ { precision: "exact" },+ );+ assert.throws(+ () => validateSemanticMap(narrative, basis, {+ ...validMap,+ anchors: validMap.anchors.map((anchor, index) => index === 0+ ? { ...anchor, point: { ...anchor.point, frame: anchor.point.frame + 1 } }+ : anchor),+ }),+ /semantic_map_digest/u,+ );+ const otherBasis = createProgramBasis({+ fps: 30,+ durationFrames: 150,+ outcomeDigest: sha256("other"),+ });+ assert.throws(+ () => validateSemanticMap(narrative, otherBasis, validMap),+ /semantic_map_basis/u,+ );+});+test("Dual Text owns its complete speech span and caption cues cannot split it", () => {const narrative = parseScript("fixture.svml",@@ -51,7 +192,7 @@ test("Dual Text owns its complete speech span and caption cues cannot split it",],segments: [{ id: "one", startSec: 0, endSec: 2 }],};- const located = locateScript(narrative, evidence);+ const located = locateAlignment(narrative, evidence);assert.deepEqual(located.captionAtoms.map((atom) => [atom.display,@@ -61,7 +202,7 @@ test("Dual Text owns its complete speech span and caption cues cannot split it",[["lmao", 15, 51]],);assert.throws(- () => locateScript(narrative, {+ () => locateAlignment(narrative, {...evidence,captionCues: [{ startWord: 0, endWordExclusive: 4 },@@ -139,7 +280,7 @@ test("caption cue evidence is a complete ordered word partition", () => {{ id: "payoff", startWord: 1, endWordExclusive: 3 },],};- const located = locateScript(narrative, evidence);+ const located = locateAlignment(narrative, evidence);assert.deepEqual(located.captionCues?.map((cue) => [cue.id,@@ -155,7 +296,7 @@ test("caption cue evidence is a complete ordered word partition", () => {);assert.throws(- () => locateScript(narrative, {+ () => locateAlignment(narrative, {...evidence,captionCues: [{ startWord: 1, endWordExclusive: 3 }],}),@@ -163,12 +304,12 @@ test("caption cue evidence is a complete ordered word partition", () => {);assert.throws(- () => locateScript(narrative, {+ () => locateAlignment(narrative, {...evidence,words: evidence.words.map((word, index) =>index === 1 ? { ...word, startSec: 0.2 } : word),}),- /locate_alignment_word_overlap/u,+ /locator_word_overlap/u,);});