Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.
TypeScript agentic-aiaiai-agentsai-videocompilerdslffmpeggenerative-aillmmarkup-languagemonorepoplugin-systemprogramming-languagetext-to-videotypescriptvideovideo-automationvideo-clonevideo-editingvideo-generation
refactor: unify adaptive timeline and presentation composition
kashorinb679e3219251
diff --git a/docs/guide/author-packages.md b/docs/guide/author-packages.mdindex 426f1329..85ca51dd 100644--- a/docs/guide/author-packages.md+++ b/docs/guide/author-packages.md@@ -35,7 +35,7 @@ appearance and meaningful events. A transition can accept a Moment for when thea Selection for how long the scene appears. A pure animation can accept authored event times instead.Project those inputs onto the chosen clock, then use the resulting schedule to draw the scene.-For an existing spoken performance, consume its SemanticTrack so picture sampling follows the same+For an existing spoken performance, consume its Timeline so picture sampling follows the sametakes and source positions as the speech. Other video inputs enter the timeline as normalized media.The [responsive explainer example](https://github.com/hypit-ai/hypit/tree/main/examples/semantic-composition/packages/responsive-explainer)shows a continuously playing video moving from full screen to a side portrait inside an HTML scene.diff --git a/docs/guide/component-anatomy.md b/docs/guide/component-anatomy.mdindex 86f76e37..22fa9db3 100644--- a/docs/guide/component-anatomy.md+++ b/docs/guide/component-anatomy.md@@ -38,7 +38,7 @@ seconds or frames can express the reading rhythm. A duration such as `12f` can dof a transition in either case.Media presentation, spatial layout and time each have their own inputs. A normalized clip supplies-sampleable media; a SemanticTrack can supply the prepared performance and its source positions.+sampleable media; a Timeline can supply the prepared performance and its source positions.A Frame can locate the scene, while its internal HTML/CSS or element tree coordinates videos,text, masks and graphics. Group content that shares behavior. Independent contributions can stayas peer Tracks with their own paint order.diff --git a/docs/guide/studio-temporal-windows.md b/docs/guide/studio-temporal-windows.mdindex cc9abbf3..9e922f88 100644--- a/docs/guide/studio-temporal-windows.md+++ b/docs/guide/studio-temporal-windows.md@@ -11,16 +11,16 @@ author choice (Selection / Segment / Moment / program time)-> domain consumption (Program / Schedule / Track)```-Script's semantic ruler contains `2M + 2N + 2` ordered points: Token and Segment boundaries plus+Script's semantic ruler contains `2M + 2N + 2` author points: Token and Segment boundaries plusdistinct Program start/end anchors. Program anchors resolve at frame `0` and `frameCount` in the-SemanticTrack; they are author points and therefore never have to be forged inside a SemanticTake.+Timeline; they are author points and therefore never have to be forged inside a SemanticTake.+Their Script order need not match physical time when Takes overlap or are reordered.This does not make `during="program"` writable: that projection remains structurally fixed.`@hypit/temporal-markup` owns the SVML author forms and lowers them before a domain component runs.The compiler only composes records, components and fragments; it does not recognize `during`, `at`-or any time grammar. A declared `ProgramSpace` supplies authored animation time; for a performance,-the graph projects it through the public Semantic Track producer,-so domain components consume only ProgramSpace and projected time values and remain independent from+or any time grammar. One Timeline supplies the complete range and any placed semantic evidence.+Domain components receive that same Timeline and projected time values, remaining independent fromboth semantic lookup and Studio.## Runtime values@@ -32,7 +32,7 @@ The identity chain is author-visible rather than synthesized by Studio:- `<script id="story">` becomes `Narrative.id = story`; every Selection, Moment, Segment excerptand CaptionDocument from that Script carries `narrativeId = story`;-- the selected Semantic Track id, or the declared Space id, becomes `ProgramSpace.id`; every terminal Track carries that exact+- the selected Timeline id becomes `ProgramSpace.id`; every terminal Track carries that exact`programSpaceId`;- every projected Instant carries the space identity; semantic sources also carry the Narrativeidentity. Each includes the consumer's public domain identity (normally its SVML `id`) as `subjectId`.@@ -87,7 +87,7 @@ the same public Instant/Window protocol and carrying its executed lineage to itsFilm and Script interpretation also remain outside Studio core. `film-studio` declares which Filmreference supplies the time axis and which child references are terminal Tracks; `script-studio` ownsScript source observation and marker relocation. Studio selects the Script source map whose-`narrativeId` exactly matches the active SemanticTrack and delegates the edit back to that companion.+`narrativeId` exactly matches the active Timeline and delegates the edit back to that companion.An authored Space supplies the physical timeline without a Script lane.The binding name is not used as a global address. Markup retains the exact author element and inputdiff --git a/docs/quickstart/composition.md b/docs/quickstart/composition.mdindex 3e3c9ab4..8780c07f 100644--- a/docs/quickstart/composition.md+++ b/docs/quickstart/composition.md@@ -19,9 +19,15 @@ retains its own timed appearances and paint order; sound is included through the```svml<space:Canvas id="vertical" width="1080" height="1920"/>-<film:Film id="main" canvas={vertical} semantic={speech.semantic} appearance={recipes.film.vertical}>+<import as="sound" from="@hypit/sound@1"/>+<sound:Style id="voice-style"/>+<sound:Track id="voice" timeline={speech.timeline}>+ <sound:Use style={voice-style}/>+</sound:Track>++<film:Film id="main" canvas={vertical} timeline={speech.timeline} appearance={recipes.film.vertical}><film:Track source={performance.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={voice.audio}/><film:Track source={captions.track}/><film:Track source={product-broll.visual}/><film:Track source={titles.track}/>@@ -32,8 +38,7 @@ retains its own timed appearances and paint order; sound is included through the|---|---|---|| `id` | yes | Unique identifier || `canvas` | yes | Explicit CanvasSpace shared with Track layout |-| `semantic` | one time source | Selected performance's SemanticTrack; supplies duration and frame rate |-| `space` | one time source | Authored ProgramSpace; use either `semantic` or `space` |+| `timeline` | yes | Complete Timeline, with any prepared material and semantic anchors || `appearance` | yes | SVS Film Recipe — the canvas clear color |### film:Track@@ -48,8 +53,8 @@ Common Track sources:| Source | Type | From ||---|---|---|-| `{performance.visual}` | VisualTrack | `media-track:Track` — presentation of the semantic performance |-| `{speech.audio}` | AudioTrack | `speech:Track` — synchronized audio |+| `{performance.visual}` | VisualTrack | `performance:Track` — presentation of existing footage |+| `{voice.audio}` | AudioTrack | `sound:Track` — presentation of existing audio || `{captions.track}` | VisualTrack | a Caption Style-family Track — timed captions || `{cards.visual}` | VisualTrack | `media-track:Track` — media overlays or B-roll || `{titles.track}` | VisualTrack | `text:Track` — text overlays |@@ -82,15 +87,14 @@ not prescribe that boundary.Compiles the Composition into a finished video via the HyperFrames renderer.```svml-<render:Video id="final" composition={main.composition} semantic={speech.semantic}/>+<render:Video id="final" composition={main.composition} timeline={speech.timeline}/>```| Attribute | Required | Description ||---|---|---|| `id` | yes | Unique identifier || `composition` | yes | Composition from `film:Film` |-| `semantic` | one time source | The composition's selected SemanticTrack |-| `space` | one time source | The composition's authored ProgramSpace; use either `semantic` or `space` |+| `timeline` | yes | Complete Timeline, with any prepared material and semantic anchors |The renderer:@@ -111,13 +115,14 @@ their presentation. A chat animation or diagram can instead direct its own readithe film clock and use it in the scene, Film and Render:```svml-<import as="time" from="@hypit/program-space@1"/>-<time:Space id="animation" frame-rate="30" duration="8s"/>+<import as="time" from="@hypit/timeline-author@1"/>+<time:Clock id="animation-clock" frame-rate="30"/>+<time:Timeline id="animation" clock={animation-clock} end="8s"/><!-- scene.track is produced by a component using this same clock. -->-<film:Film id="main" canvas={canvas} space={animation} appearance={recipes.film.main}>+<film:Film id="main" canvas={canvas} timeline={animation.timeline} appearance={recipes.film.main}><film:Track source={scene.track}/></film:Film>-<render:Video id="final" composition={main.composition} space={animation}/>+<render:Video id="final" composition={main.composition} timeline={animation.timeline}/>```The scene's events can use authored seconds or frames. In a spoken composition, the same behavior@@ -141,12 +146,13 @@ illustration. The complete runnable project is `examples/podcast/`; the commands<import as="gpt" from="@hypit/gpt-image@1"/><import as="seedance" from="@hypit/seedance@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="whisperx" from="@hypit/whisperx@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/><import as="fonts" from="@hypit/fonts-open@1"/><import as="media-track" from="@hypit/media-track@1"/>+ <import as="performance" from="@hypit/performance@1"/><import as="text" from="@hypit/typography-track@1"/><import as="space" from="@hypit/spatial@1"/><import as="program" from="@hypit/program-space@1"/>@@ -196,39 +202,46 @@ illustration. The complete runnable project is `examples/podcast/`; the commandsvideo="primary-moving" audio="none" span-authority="video" clock={clock}/><whisperx:SemanticTake id="opening-semantic" narrative={story}segment={story.segment.opening} media={take-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={opening-semantic.take}/>- </speech:Track>-<media-track:Track id="performance" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={speech-frame}- appearance={recipes.media.performance}/>-</media-track:Track>+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={opening-semantic.take}/>+ </time:Timeline>+<performance:Style id="performance-style" frame={speech-frame} appearance={recipes.media.performance}/>+ <performance:Track id="performance" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={performance-style} during="program"/>+ </performance:Track><!-- 4. Tracks: captions, Media, text --><fonts:Stack id="caption-font" family="inter" weight="700" style="normal"/><fonts:Stack id="title-font" family="inter" weight="900" style="normal"/><caption-fine:Style id="base-caption" recipe={recipes.caption.base} font={caption-font}/>- <caption:Program id="caption-program" document={story.caption} narrative={story}- default={base-caption}/>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={base-caption}/>+ </caption-fine:Track>- <media-track:Track id="cards" semantic={speech.semantic} canvas={vertical}>+ <media-track:Track id="cards" timeline={speech.timeline} canvas={vertical}><media-track:Item media={motion-media.media} during={story.selection.demo}frame={card-frame} appearance={recipes.media.card} motion={recipes.motion.card}/></media-track:Track><text:Style id="title-style" recipe={recipes.text.title} font={title-font}/>- <text:Track id="titles" semantic={speech.semantic}>+ <text:Track id="titles" timeline={speech.timeline}><text:Area id="meaning" placement={title-frame} style={title-style} during="program">MEANING</text:Area></text:Track><!-- 5. Film: compose all tracks -->- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <import as="sound" from="@hypit/sound@1"/>+<sound:Style id="voice-style"/>+<sound:Track id="voice" timeline={speech.timeline}>+ <sound:Use style={voice-style}/>+</sound:Track>++<film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={performance.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={voice.audio}/><film:Track source={cards.visual}/><film:Track source={captions.track}/><film:Track source={titles.track}/>@@ -236,7 +249,7 @@ illustration. The complete runnable project is `examples/podcast/`; the commands<!-- 6. Render: compile to MP4 --><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>```diff --git a/docs/quickstart/generation.md b/docs/quickstart/generation.mdindex 3c3c5ff4..61d982bd 100644--- a/docs/quickstart/generation.md+++ b/docs/quickstart/generation.md@@ -347,6 +347,6 @@ generation:</seedance:ReferenceVideo>```-Each `seedance:ReferenceVideo` produces a `{*.video}` output that feeds into `speech:Track` in the+Each `seedance:ReferenceVideo` produces a `{*.video}` output that feeds into `time:Timeline` in thenext stage. Different takes can use different reference images while sharing the same voice timbreand prompt Recipe.diff --git a/docs/quickstart/images.md b/docs/quickstart/images.mdindex c36cfe7d..59a52be1 100644--- a/docs/quickstart/images.md+++ b/docs/quickstart/images.md@@ -111,7 +111,7 @@ served by a supporting HypiHub Endpoint:```Normalize `cutout.video` to prepare it for the timeline. If it establishes the spoken program,-align that prepared media to its Script Segment and assemble it through Speech Track. Present the+align that prepared media to its Script Segment and assemble it through Timeline assembly. Present thepicture with Media Track or a project scene, at the chosen location and paint order. As B-roll,the normalized cutout can enter Media Track directly. Matting changes the picture's background;the composition determines its role.diff --git a/docs/quickstart/preview.md b/docs/quickstart/preview.mdindex 8d28aa6c..3f0caf2a 100644--- a/docs/quickstart/preview.md+++ b/docs/quickstart/preview.md@@ -26,7 +26,7 @@ that Run. To keep generated material while editing, select completed Outputs wit| `--port <number>` | Request a browser-server port; the default is `5179`. |The selected target must lead to one Film and its time source. A spoken performance supplies a-SemanticTrack; an animation can supply an authored ProgramSpace. Both support visual components and+Timeline; an animation can supply an authored ProgramSpace. Both support visual components andproperty editing. The material needed to display the composition must already be available throughthe Run. Studio can perform media preparation supported by the selected Runtime; submit generationand encoded renders through `hypit build`.diff --git a/docs/quickstart/script.md b/docs/quickstart/script.mdindex ea563047..4244546f 100644--- a/docs/quickstart/script.md+++ b/docs/quickstart/script.md@@ -111,7 +111,7 @@ is rejected. Escape it as `\@` when the at-sign should be shown.`||` is the **Caption Cue Break** syntax. It records a boundary between complete Alignment Units;it cannot appear inside Dual Text or split an N:M unit. Cue timing is still obtained later by-joining the CaptionDocument to the SemanticTrack.+joining the CaptionDocument to the Timeline.### Flat token attributes@@ -190,7 +190,7 @@ Selections are not required to nest like XML tags. They can cross each other:Selection markers are zero-width and never appear in any text projection. They compile into one`NarrativeSelection` with `startAnchorId` and `endAnchorId`. Script itself contains no seconds or-frame numbers — timing comes from SemanticTrack alignment.+frame numbers — timing comes from Timeline alignment.Other components reference Selections via `{story.selection.problem}` to bind visual content tosemantic moments in the narrative.diff --git a/docs/quickstart/styles.md b/docs/quickstart/styles.mdindex bae3ba53..3dfe5bd1 100644--- a/docs/quickstart/styles.md+++ b/docs/quickstart/styles.md@@ -60,7 +60,7 @@ Referenced by `film:Film` via the `appearance` attribute:```svml<space:Canvas id="vertical" width="1080" height="1920"/>-<film:Film id="main" canvas={vertical} semantic={speech.semantic} appearance={recipes.film.vertical}>+<film:Film id="main" canvas={vertical} timeline={speech.timeline} appearance={recipes.film.vertical}>```## Caption Fine@@ -133,17 +133,18 @@ caption.bob {}```-Then assign them via `caption:Program`:+Then select them with timed Uses in the Track:```svml<fonts:Stack id="caption-font" family="inter" weight="600" style="normal"/><caption-fine:Style id="default-caption" recipe={recipes.caption.dialogue} font={caption-font}/><caption-fine:Style id="alice-caption" recipe={recipes.caption.alice} font={caption-font}/><caption-fine:Style id="bob-caption" recipe={recipes.caption.bob} font={caption-font}/>-<caption:Program id="caption-program" document={story.caption} default={default-caption}>- <caption:Use role="ALICE" style={alice-caption}/>- <caption:Use role="BOB" style={bob-caption}/>-</caption:Program>+<caption-fine:Track id="captions" document={story.caption} timeline={speech.timeline}>+ <caption-fine:Use style={default-caption}/>+ <caption-fine:Use role="ALICE" style={alice-caption}/>+ <caption-fine:Use role="BOB" style={bob-caption}/>+</caption-fine:Track>```## Media Track@@ -400,5 +401,5 @@ This file is imported once in the `.svml` source and its values are referenced t<caption-fine:Style id="primary-caption" recipe={recipes.caption.primary} font={caption-font}/><space:Canvas id="vertical" width="720" height="1280"/>-<film:Film id="main" canvas={vertical} semantic={speech.semantic} appearance={recipes.film.vertical}>+<film:Film id="main" canvas={vertical} timeline={speech.timeline} appearance={recipes.film.vertical}>```diff --git a/docs/quickstart/timing.md b/docs/quickstart/timing.mdindex 64848a1f..022d4cab 100644--- a/docs/quickstart/timing.md+++ b/docs/quickstart/timing.md@@ -1,19 +1,19 @@---title: Timing & Assembly-description: Per-take normalization and alignment, followed by SemanticTrack assembly.+description: Per-take normalization and alignment, followed by Timeline assembly.----For spoken video, a `SemanticTrack` connects the authored Script to the actual performance. This is+For spoken video, a `Timeline` connects the authored Script to the actual performance. This isthe natural time source for captions, word-triggered graphics and coverage. Build it in segment-sized pieces:1. normalize each accepted A/V take into one exact frame domain;2. align that normalized media with its authored Script Segment to create a self-contained `SemanticTake`;-3. assemble the Semantic Takes in program order with `speech:Track`.+3. assemble the Semantic Takes in program order with `time:Timeline`.-Every Take is already semantic before it enters the Speech Track. Pictures can be presented by+Every Take is already semantic before it enters the Timeline assembly. Pictures can be presented byMedia Track or a project component independently of this semantic and audio assembly.-A purely visual animation can instead declare its own duration and frame rate in a ProgramSpace.+A purely visual animation uses the same Timeline with an explicit end and zero Takes.See [an authored film clock](./composition.md#a-film-drawn-entirely-by-components). It can use secondsor frames for events; spoken work can use Script Selections and Moments for the same visual behavior.@@ -21,8 +21,9 @@ or frames for events; spoken work can use Script Selections and Moments for the<import as="program" from="@hypit/program-space@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>-<import as="speech" from="@hypit/speech-track@1"/>+<import as="time" from="@hypit/timeline-author@1"/><import as="media-track" from="@hypit/media-track@1"/>+ <import as="performance" from="@hypit/performance@1"/><import as="space" from="@hypit/spatial@1"/><import as="recipes" source="./recipes.svs"/>```@@ -30,7 +31,7 @@ or frames for events; spoken work can use Script Selections and Moments for the## Normalize each takeNormalization makes video, audio, duration and frame rate one explicit `SynchronizedMedia` fact.-The Clock is authored once and shared by every Take that will enter the same SemanticTrack.+The Clock is authored once and shared by every Take that will enter the same Timeline.```svml<program:Clock id="clock" frame-rate="30"/>@@ -64,43 +65,52 @@ window, and all of that Segment's structural anchors. There are two anchors forfor each word. Acoustic evidence is an implementation input to this step; downstream components seethe completed `SemanticTake`, not a second evidence-shaped timing structure.-## Assemble the SemanticTrack+## Assemble the Timeline-`speech:Track` concatenates already-semantic Takes in document order and provides the semantic-timeline and its original sound. Media presents the performance separately:+`time:Timeline` places prepared Takes sequentially by default, or at authored `at` positions,+and provides the complete Timeline. Performance and Sound present its pictures and audio separately:```svml<space:Canvas id="vertical" width="1080" height="1920"/><space:Frame id="speech-frame" within={vertical}left="0%" top="0%" right="100%" bottom="100%"/>-<speech:Track id="speech">- <speech:Take source={opening-semantic.take}/>- <speech:Take source={answer-semantic.take}/>-</speech:Track>-<media-track:Track id="performance" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={speech-frame}- appearance={recipes.media.performance}/>-</media-track:Track>+<time:Timeline id="speech" clock={clock}>+ <time:Take source={opening-semantic.take}/>+ <time:Take source={answer-semantic.take}/>+</time:Timeline>+<import as="sound" from="@hypit/sound@1"/>+<sound:Style id="voice-style"/>+<sound:Track id="voice" timeline={speech.timeline}>+ <sound:Use style={voice-style}/>+</sound:Track>+<performance:Style id="performance-style" frame={speech-frame} appearance={recipes.media.performance}/>+ <performance:Track id="performance" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={performance-style} during="program"/>+ </performance:Track>```| Output | Type | Meaning ||---|---|---|-| `{speech.semantic}` | SemanticTrack | Global semantic and frame-domain authority |-| `{speech.audio}` | AudioTrack | Same-source sound, aligned to the semantic items |+| `{speech.timeline}` | Timeline | Global semantic and frame-domain authority |+| `{voice.audio}` | AudioTrack | Sound presentation of the placed Takes |-Picture consumers and the original audio use the same prepared Takes and source positions.-`SemanticTrack` derives global frames by prefix-summing the local Take lengths, and also supplies the-program duration and frame domain required by Film and Render.+Performance and Sound use the same prepared Takes and source positions.+Each Take's global frames are its placement start plus its local frames. The first omitted `at` is+zero; later omitted `at` follows the preceding Take's end. Use `at="previous.end+2s"` for a gap,+`at="previous.end-12f"` for overlap, or an absolute position. Timeline `end` defaults to the latest+content end; `end="content.end+2s"` reserves a tail and `end="30s"` declares a fixed extent.+All positions resolve to exact frames and the extent contains all Takes. Empty Timelines require+an authored positive end. No placeholder media fills uncovered time.## Consume semantic timeSelections, Moments and whole Segments remain authored Script identities. A downstream component-receives the SemanticTrack once and projects those identities into frames only when it builds its+receives the Timeline once and projects those identities into frames only when it builds itsdeterministic Track:```svml-<media-track:Track id="cards" semantic={speech.semantic} canvas={vertical}>+<media-track:Track id="cards" timeline={speech.timeline} canvas={vertical}><media-track:Item image={card.image} extent={card-extent}during={story.selection.demo} frame={card-frame}appearance={recipes.media.card} motion={recipes.motion.card}/>@@ -108,28 +118,30 @@ deterministic Track:<caption-fine:Track id="captions"document={story.caption}- semantic={speech.semantic}- program={caption-program}/>+ timeline={speech.timeline}+>+ <caption-fine:Use style={primary-caption}/>+</caption-fine:Track><film:Film id="main" canvas={vertical}- semantic={speech.semantic} appearance={recipes.film.vertical}>+ timeline={speech.timeline} appearance={recipes.film.vertical}><film:Track source={performance.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={voice.audio}/><film:Track source={cards.visual}/><film:Track source={captions.track}/></film:Film><render:Video id="final"- composition={main.composition} semantic={speech.semantic}/>+ composition={main.composition} timeline={speech.timeline}/>```Use `during={story.segment.answer}` for a whole Segment, a Selection for an authored range, a Moment-for a point event, and `during="program"` for the complete SemanticTrack domain. Components consume-`semantic={speech.semantic}`.+for a point event, and `during="program"` for the complete Timeline domain. Components consume+`timeline={speech.timeline}`.```text-prepared Takes → Speech Track ── .semantic → Media / project scene → .visual ─┐- │ └──→ Caption / semantic graphics ───┤- └───── .audio ────────────────────────────────────┤- Film+prepared Takes → Timeline → Performance / project scene → visual ─┐+ ├→ Caption / semantic graphics → visual ┤+ └→ Sound → audio ───────────────────────┤+ Film```diff --git a/docs/quickstart/tracks.md b/docs/quickstart/tracks.mdindex fb74f447..44e1dc90 100644--- a/docs/quickstart/tracks.md+++ b/docs/quickstart/tracks.md@@ -12,7 +12,7 @@ covers Caption, Media, Typography and Audio Track authoring.Caption uses a Script-owned document and a replaceable Style family:```text-Script CaptionDocument → Caption Program → SemanticTrack timing → Style-family Track+Script CaptionDocument + Timeline → complete timed Cues → Use presentation``````svml@@ -105,7 +105,7 @@ Recipe—not a second renderer.With `wrap: word`, Fine wraps between complete Alignment Units and falls back inside a singleover-wide display unit so it cannot escape the Region. `max-words-per-line` constructs explicit rows;when `max-lines` is present, a Cue that would construct more rows is rejected instead of having its-text or Paint clipped. Cue boundaries come from Script segments, turns, Style changes and authored+text or Paint clipped. Cue boundaries come from Script segments, turns and authored`||`.Fine is the uniform-flow family: every token follows the same Recipe and may differ only by time,@@ -117,37 +117,22 @@ CJK dialogue can be written directly. For a display-only emoji that still followauthor the correspondence explicitly, such as `<🌐 | globe>`; the system will not invent a spokenword for a bare symbol.-### caption:Program--The Program starts from the complete ordered CaptionDocument emitted by Script. One explicit default-Style covers every Alignment Unit. Ordered `Use` rules replace the Style on a Role or a semantic-Selection, with the last match winning.--```svml-<caption:Program id="caption-program" document={story.caption} narrative={story}- default={primary-caption}>- <caption:Use role="ALICE" style={alice-caption}/>- <caption:Use role="BOB" style={bob-caption}/>- <caption:Use selection={story.selection.product-demo}- style={dialogue-caption}/>- <caption:Mute selection={story.selection.private}/>-</caption:Program>-```--`role=` is a convenient query for a word subset, not a temporal condition. `selection=` projects-the public semantic Selection to complete Caption Alignment Units before timing. Partial ownership-of an indivisible N:M Dual Text unit is rejected. `Mute` uses the same projection and does not-regroup authored Cues.-### caption-fine:Track+Caption content comes from Script and Timeline. Uses choose presentation in time; later Uses replace earlier treatments inside their windows, including Hidden.+```svml-<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+<caption:Hidden id="hidden"/>+<caption-fine:Track id="captions" document={story.caption} timeline={speech.timeline}>+ <caption-fine:Use style={primary-caption}/>+ <caption-fine:Use role="ALICE" style={alice-caption}/>+ <caption-fine:Use role="BOB" style={bob-caption}/>+ <caption-fine:Use during={story.selection.product-demo} style={dialogue-caption}/>+ <caption-fine:Use during={story.selection.private} style={hidden}/>+</caption-fine:Track>```-Caption joins each complete unit to the independent SemanticTrack. Fine then renders all default-and override Styles into one ordinary peer `VisualTrack`: `{captions.track}`.+`||` organizes Cues. A window can start inside a Cue while retaining its complete text and original word timing. `role` filters the speaker independently of time. `at`/`for`, `until`/`for` and `start`/`end` use the same time language as other tracks.## Media overlays and B-roll@@ -181,7 +166,7 @@ Placement is an explicit Spatial Frame edge; appearance and motion remain reusab<pipeline:Normalize id="product-media" source={product-motion.video}video="primary-moving" audio="none" span-authority="video" frame-rate="30"/>-<media-track:Track id="product-broll" semantic={speech.semantic} canvas={vertical}>+<media-track:Track id="product-broll" timeline={speech.timeline} canvas={vertical}><media-track:Item media={product-media.media} frame={product-frame}during={story.selection.product-demo}appearance={recipes.media.product}@@ -215,8 +200,8 @@ image or a video.## Audio tracks-`@hypit/audio-track` places explicitly prepared audio on the same ProgramSpace as the visual-Tracks. A `Clip` consumes `SynchronizedMedia`; normalize a declared or generated audio Blob first,+`@hypit/audio-track` places explicitly prepared audio on the same Timeline as the visual+Tracks. An `Item` consumes `SynchronizedMedia`; normalize a declared or generated audio Blob first,then choose its exact program window and occupancy:```svml@@ -228,8 +213,8 @@ then choose its exact program window and occupancy:<pipeline:Normalize id="music-media" source={music}video="none" audio="default" span-authority="audio" frame-rate="30"/>-<audio:Track id="music-bed" semantic={speech.semantic}>- <audio:Clip source={music-media.media} during="program"+<audio:Track id="music-bed" timeline={speech.timeline}>+ <audio:Item source={music-media.media} during="program"playback="loop-end" gain="0.28" fade-in="600ms" fade-out="800ms"/></audio:Track>```@@ -237,16 +222,16 @@ then choose its exact program window and occupancy:| Attribute | Required | Description ||---|---|---|| `Track.id` | yes | Stable Audio Track identity |-| `Track.semantic` | yes | SemanticTrack that defines the exact sample and frame domain |-| `Clip.source` | yes | Explicitly selected and normalized `SynchronizedMedia` |+| `Track.timeline` | yes | Timeline that defines the exact sample and frame domain |+| `Item.source` | yes | Explicitly selected and normalized `SynchronizedMedia` || `during`, `at`/`for`, or `start`/`end` | exactly one form | Whole-program, Selection, Moment, or explicit window || `playback` | no | `once`, `once-end`, `loop`, `loop-end`, or bounded `stretch` || `trim-start`, `trim-end` | no | Exact source trim |-| `gain`, `fade-in`, `fade-out` | no | Explicit per-clip mix values |+| `gain`, `fade-in`, `fade-out` | no | Explicit per-item mix values |The package performs no automatic extraction, normalization, ducking, or bus routing. Multiple-Clips in one Track and multiple peer Audio Tracks remain independent inputs to Film. The output is-`{music-bed.track}`, an ordinary `AudioTrack`.+Items in one Track and multiple peer Audio Tracks remain independent inputs to Film. The output is+`{music-bed.audio}`, an ordinary `AudioTrack`.## Text overlays@@ -267,7 +252,7 @@ Container for text items.left="6%" top="6%" right="94%" bottom="16%"/><fonts:Stack id="title-font" family="inter" weight="900" style="normal"/><text:Style id="title-style" recipe={recipes.text.title} font={title-font}/>-<text:Track id="titles" semantic={speech.semantic}>+<text:Track id="titles" timeline={speech.timeline}><text:Area id="title" placement={title-frame} style={title-style} during="program">EDIT MEANING, NOT TIMELINES</text:Area>@@ -277,7 +262,7 @@ Container for text items.| Attribute | Required | Description ||---|---|---|| `id` | yes | Unique identifier |-| `semantic` | yes | SemanticTrack from `speech:Track` — also resolves Selection-based item timing |+| `semantic` | yes | Timeline from `time:Timeline` — also resolves Selection-based item timing |### text:Point, text:Area and text:Path@@ -303,7 +288,7 @@ or a Selection reference for semantic timing:```svml<text:Style id="callout-style" recipe={recipes.text.callout} font={title-font}/>-<text:Track id="callout" semantic={speech.semantic}>+<text:Track id="callout" timeline={speech.timeline}><text:Area id="callout-copy" placement={callout-frame}style={callout-style} during={story.selection.callout}>EXACTLY THE RIGHT MOMENT@@ -315,7 +300,7 @@ Graph-produced copy remains visible as an edge:```svml<wording:Value id="headline">EXACTLY THE RIGHT MOMENT</wording:Value>-<text:Track id="callout" semantic={speech.semantic}>+<text:Track id="callout" timeline={speech.timeline}><text:Area id="callout-copy" content={headline}placement={callout-frame} style={callout-style} during="program"/></text:Track>@@ -350,7 +335,7 @@ family is refused by name.| Attribute | Takes ||---|---|-| `semantic` | the SemanticTrack the board is timed against |+| `semantic` | the Timeline the board is timed against || `frame` | a `space:Frame` — the board's declared placement || `during` | the timing form declared by the selected component || `style` | the matching style record, and only that variant's |@@ -377,7 +362,7 @@ Each variant takes its own, at least one, and ids must be unique within a board.```svml<ranking:ColumnStyle id="board-style" recipe={recipes.ranking.board} font={ui-font}/>-<ranking:Column id="board" semantic={speech.semantic} canvas={vertical} frame={board-frame}+<ranking:Column id="board" timeline={speech.timeline} canvas={vertical} frame={board-frame}during={story.selection.board} style={board-style}><ranking:ColumnItem id="row-regen" rank="1" label="ReGen" icon={icon-regen}during={story.selection.regen-reveal}/>@@ -426,7 +411,7 @@ give both and it is refused. `size`, `color`, `align`, `block` and `padding` are```svml<space:Frame id="deck-frame" within={vertical} left="44%" top="60%" right="98%" bottom="88%"/>-<deck:DepthStack id="deck" semantic={speech.semantic} canvas={vertical}+<deck:DepthStack id="deck" timeline={speech.timeline} canvas={vertical}frame={deck-frame} appearance={recipes.deck.stack} until={story.moment.done}><deck:Card id="card-spatial" source={icon-spatial} extent={square} at={story.moment.deal-one}/><deck:Card id="card-type" source={icon-type} extent={square} at={story.moment.deal-two}/>@@ -445,14 +430,14 @@ effect bound to its own window.<import as="screen" from="@hypit/screen-overlay@1"/>```-`screen:Track` takes `id`, `canvas` and one time source: `semantic` or `space`. Its children are the effects, at least one, each+`screen:Track` takes `id`, `canvas` and `timeline`. Its children are the effects, at least one, eachempty, each with a required `z` for stacking order and a window that is one of:| Window | Written ||---|---|| The whole programme | `during="program"` |-| A Selection | `during={story.selection.x}` on an item whose Track has `semantic={speech.semantic}` |-| A Moment, for a length | `at={story.moment.x} for="12f"` on an item whose Track has `semantic={speech.semantic}` |+| A Selection | `during={story.selection.x}` on an item whose Track has `timeline={speech.timeline}` |+| A Moment, for a length | `at={story.moment.x} for="12f"` on an item whose Track has `timeline={speech.timeline}` || An explicit span | `start="…" end="…"`, optionally against a `selection=` or `moment=` |Lengths are `12f`, `250ms` or `1.5s`. A Selection@@ -466,7 +451,7 @@ required attributes, such as `color` / `intensity` / `attack` / `hold` / `decay`None have defaults: an effect states its whole shape or is refused.```svml-<screen:Track id="effects" semantic={speech.semantic} canvas={vertical}>+<screen:Track id="effects" timeline={speech.timeline} canvas={vertical}><screen:Flash during={story.selection.overlay} z="80"color="#ffffff" intensity="0.6" attack="2" hold="2" decay="6"/></screen:Track>@@ -487,8 +472,7 @@ optional metadata line.card's whole appearance — background, border, radius, tail, avatar, the three text rows, and theenter/hold/exit motion — and every key has a default, so a recipe may set only what it changes.-`comment:Track` takes `id`, `canvas` and one time source: `semantic` for performance time or `space`-for authored animation.+`comment:Track` takes `id`, `canvas` and `timeline` for the complete work.`comment:Sticker` requires `id`, `frame` and `style`, and takes the same windows as a screen overlayabove. Its copy is either the `comment=` attribute or the element's own text — both is refused. The@@ -497,7 +481,7 @@ image, and there is no `z`: stacking order comes from the recipe's `stack-order````svml<comment:Style id="social" recipe={recipes.comment} font={ui-font}/>-<comment:Track id="comments" canvas={vertical} semantic={speech.semantic}>+<comment:Track id="comments" canvas={vertical} timeline={speech.timeline}><comment:Sticker id="one" frame={comment-frame} style={social} avatar={viewer-avatar}author="@viewer" meta="Featured" during={story.selection.reaction}>Wait, it pinned the caption to the word, not the second.@@ -526,10 +510,12 @@ All four track families together in one source file:<fonts:Stack id="caption-font" family="inter" weight="700" style="normal"/><fonts:Stack id="title-font" family="inter" weight="900" style="normal"/><caption-fine:Style id="base-caption" recipe={recipes.caption.base} font={caption-font}/>-<caption:Program id="caption-program" document={story.caption} narrative={story}- default={base-caption}/>+<caption:Hidden id="hidden"/>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={base-caption}/>+ </caption-fine:Track><!-- Shared placement is an explicit edge, separate from Text appearance. --><space:Canvas id="vertical" width="1080" height="1920"/>@@ -541,14 +527,14 @@ All four track families together in one source file:<!-- Media: one ordinary Item used editorially as B-roll --><pipeline:Normalize id="card-media" source={motion.video}video="primary-moving" audio="none" span-authority="video" frame-rate="30"/>-<media-track:Track id="cards" semantic={speech.semantic} canvas={vertical}>+<media-track:Track id="cards" timeline={speech.timeline} canvas={vertical}><media-track:Item media={card-media.media} frame={card-frame}during={story.selection.demo} appearance={recipes.media.card} motion={recipes.motion.card}/></media-track:Track><!-- Text: persistent title overlay --><text:Style id="title-style" recipe={recipes.text.title} font={title-font}/>-<text:Track id="titles" semantic={speech.semantic}>+<text:Track id="titles" timeline={speech.timeline}><text:Area id="meaning" placement={title-frame} style={title-style} during="program">MEANING</text:Area>@@ -558,19 +544,25 @@ All four track families together in one source file:<media:Audio id="music" src="./assets/music.wav"/><pipeline:Normalize id="music-media" source={music}video="none" audio="default" span-authority="audio" frame-rate="30"/>-<audio:Track id="music-bed" semantic={speech.semantic}>- <audio:Clip source={music-media.media} during="program"+<audio:Track id="music-bed" timeline={speech.timeline}>+ <audio:Item source={music-media.media} during="program"playback="loop-end" gain="0.28" fade-in="600ms" fade-out="800ms"/></audio:Track><!-- All peer tracks feed into Film -->-<film:Film id="main" canvas={vertical} semantic={speech.semantic} appearance={recipes.film.vertical}>+<import as="sound" from="@hypit/sound@1"/>+<sound:Style id="voice-style"/>+<sound:Track id="voice" timeline={speech.timeline}>+ <sound:Use style={voice-style}/>+</sound:Track>++<film:Film id="main" canvas={vertical} timeline={speech.timeline} appearance={recipes.film.vertical}><film:Track source={performance.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={voice.audio}/><film:Track source={cards.visual}/><film:Track source={captions.track}/><film:Track source={titles.track}/>- <film:Track source={music-bed.track}/>+ <film:Track source={music-bed.audio}/></film:Film>```diff --git a/docs/zh/guide/author-packages.md b/docs/zh/guide/author-packages.mdindex 3c850ce5..3162dcd4 100644--- a/docs/zh/guide/author-packages.md+++ b/docs/zh/guide/author-packages.md@@ -22,7 +22,7 @@ npm run build公开另一个视频作者真正会改的决定:内容、素材、位置、外观和有意义的事件。转场可以接收一个 Moment,决定何时改变布局;接收一个 Selection,决定整个场景何时存在。纯动画则可以接收作者指定的事件时间。将这些输入投影到选定时钟,再按得到的调度绘制场景。-呈现已有说话表演时,消费它的 SemanticTrack,让画面采样与口播使用相同的 Take 和素材位置。其他视频输入以归一化媒体进入时间线。[响应式讲解场景](https://github.com/hypit-ai/hypit/tree/main/examples/semantic-composition/packages/responsive-explainer) 展示持续播放的视频如何在 HTML 场景里从全屏移到侧边竖屏。[聊天示例](https://github.com/hypit-ai/hypit/tree/main/examples/semantic-composition/packages/chat-scene) 展示同一个事件接口如何接受作者时间或 Script Moment。+呈现已有说话表演时,消费它的 Timeline,让画面采样与口播使用相同的 Take 和素材位置。其他视频输入以归一化媒体进入时间线。[响应式讲解场景](https://github.com/hypit-ai/hypit/tree/main/examples/semantic-composition/packages/responsive-explainer) 展示持续播放的视频如何在 HTML 场景里从全屏移到侧边竖屏。[聊天示例](https://github.com/hypit-ai/hypit/tree/main/examples/semantic-composition/packages/chat-scene) 展示同一个事件接口如何接受作者时间或 Script Moment。Style 一类 Surface 在裸作者 id 下公开其值,例如 `style={board-style}`;独立输出可以使用 `.visual`、`.audio`、`.track` 等有意义的后缀。在组件自己的 vocabulary 和 README 中说明名称与可用值。diff --git a/docs/zh/guide/component-anatomy.md b/docs/zh/guide/component-anatomy.mdindex 451fd79d..2fdf8ad3 100644--- a/docs/zh/guide/component-anatomy.md+++ b/docs/zh/guide/component-anatomy.md@@ -28,7 +28,7 @@ description: 将场景的创作接口变成可复用的图贡献。对于说话视频,接受 Selection 和 Moment 来表达随词语发生的行为。表演产生后,投影提供准确区间与时刻。作者决定节奏的动画可以使用秒或帧;`12f` 这样的时长在两种情况下都能表达一次转场的长度。-素材呈现、空间布局与时间各有自己的输入。归一化视频提供可采样素材,SemanticTrack 可以提供准备好的表演及其素材位置。Frame 定位场景,内部 HTML/CSS 或元素树协调视频、文字、遮罩和图形。共享行为的内容组织在一起,独立贡献可以继续作为具有自己绘制顺序的 Track。+素材呈现、空间布局与时间各有自己的输入。归一化视频提供可采样素材,Timeline 可以提供准备好的表演及其素材位置。Frame 定位场景,内部 HTML/CSS 或元素树协调视频、文字、遮罩和图形。共享行为的内容组织在一起,独立贡献可以继续作为具有自己绘制顺序的 Track。字幕组件可以沿用已有字幕文档、样式选择与语义时间。细粒度字幕族能表达的样式可以直接使用;新的样式族则可以消费相同字幕关系,绘制不同的视觉呈现。diff --git a/docs/zh/guide/studio-temporal-windows.md b/docs/zh/guide/studio-temporal-windows.mdindex 3c83f23c..ebddee1d 100644--- a/docs/zh/guide/studio-temporal-windows.md+++ b/docs/zh/guide/studio-temporal-windows.md@@ -11,15 +11,15 @@ description: 作者选择如何成为 Instant 或 Window,并准确回到真正-> 消费层(Programme / Schedule / Track)```-Script 的语义标尺包含严格有序的 `2M + 2N + 2` 个点:Token 与 Segment 的首尾,再加彼此-独立的 Program start/end。Program 锚点由 SemanticTrack 在 `0` 和 `frameCount` 解析,不需要-伪造进某个 SemanticTake。它们是 Selection/Moment 可选择的作者语义点;这不会让+Script 的语义标尺包含 `2M + 2N + 2` 个点:Token 与 Segment 的首尾,再加彼此+独立的 Program start/end。Program 锚点由 Timeline 在 `0` 和 `frameCount` 解析,不需要+伪造进某个 SemanticTake。Take 交叠或重排时,Script 顺序不一定等于实际时间顺序。它们是 Selection/Moment 可选择的作者语义点;这不会让`during="program"` 变成可写,后者依然是结构固定的投影。`@hypit/temporal-markup` 统一拥有 SVML 时间语法,并在领域组件运行前把它降低成普通的投影组件和运行图边。编译器只会组合 Record、Component 和 Fragment,不认识 `during`、`at`,-各 Track 也不再各写一套找帧算法。纯动画可直接声明 `ProgramSpace`;表演时间则通过-Semantic Track 的公共 producer 显式投影。领域组件只消费 ProgramSpace 与时间投影结果,+各 Track 也不再各写一套找帧算法。同一个 Timeline 提供全篇范围和其中已有的语义证据,+领域组件接收同一个 Timeline 与时间投影结果,与选择层和 Studio 都解耦。## 运行时对象@@ -34,7 +34,7 @@ Semantic Track 的公共 producer 显式投影。领域组件只消费 ProgramSp- `<script id="story">` 产生 `Narrative.id = story`,它派生出的 Selection、Moment、Segmentexcerpt 与 CaptionDocument 全部携带 `narrativeId = story`;-- 被 Film 选择的 Semantic Track 或直接声明的 Space 的 `id` 就是 `ProgramSpace.id`,所有终端 Track 都携带同一个+- 被 Film 选择的 Timeline 的 `id` 就是 `ProgramSpace.id`,所有终端 Track 都携带同一个`programSpaceId`;- 每个 Instant 保留 Space 身份,来自语义的来源额外保留 Narrative 身份;消费者的公开领域身份(通常就是 SVML `id`)作为 `subjectId`。@@ -85,7 +85,7 @@ Companion 仍负责实体外观和 Inspector 展示,但不再声明通用时Instant/Window,并由 Companion 把真实执行谱系连到实体,就自动获得同一套时间行为。消费端也不是只读取 `frame`:每个官方组件在接收 Instant/Window 时核对 `subjectId`、-`ProgramSpace.id`。Script 与 SemanticTrack 的对应关系在语义投影时核对。Space 是运行图上的显式输入边;不通过全局状态、+`ProgramSpace.id`。Script 与 Timeline 的对应关系在语义投影时核对。Timeline 是运行图上的显式输入边;不通过全局状态、当前 Film 或 renderer 上下文补猜。字幕 Cue 保持独立:`CaptionDocument -> TimedCaptionProjection -> FineCaptionSchedule` 直接从@@ -95,7 +95,7 @@ Semantic token evidence 得到只读 Cue 时间,不伪装成可拖动的 TempoFilm 与 Script 的解释也不留在 Studio 核心:`film-studio` 声明 Film 的时间来源和终端 Track引用规则;`script-studio` 拥有 Script Source map 与 marker 移动。Studio 只选择-`narrativeId` 与当前 SemanticTrack 一致的 Script,再把语义修改委托回该 Companion。+`narrativeId` 与当前 Timeline 一致的 Script,再把语义修改委托回该 Companion。直接声明的 Space 提供物理时间线,不需要 Script 轨道。`at="2s"` 表示作者编排的事件时刻,可以单独用于 Instant,也可以与 `for` 配合形成 Window;其时间修改回写 SVML。diff --git a/docs/zh/quickstart/composition.md b/docs/zh/quickstart/composition.mdindex 6010a0a5..4c8e5afe 100644--- a/docs/zh/quickstart/composition.md+++ b/docs/zh/quickstart/composition.md@@ -19,9 +19,15 @@ Composition。然后渲染器将该 Composition 编译为 MP4 视频。```svml<space:Canvas id="vertical" width="1080" height="1920"/>-<film:Film id="main" canvas={vertical} semantic={speech.semantic} appearance={recipes.film.vertical}>+<import as="sound" from="@hypit/sound@1"/>+<sound:Style id="voice-style"/>+<sound:Track id="voice" timeline={speech.timeline}>+ <sound:Use style={voice-style}/>+</sound:Track>++<film:Film id="main" canvas={vertical} timeline={speech.timeline} appearance={recipes.film.vertical}><film:Track source={performance.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={voice.audio}/><film:Track source={captions.track}/><film:Track source={product-broll.visual}/><film:Track source={titles.track}/>@@ -32,8 +38,7 @@ Composition。然后渲染器将该 Composition 编译为 MP4 视频。|---|---|---|| `id` | 是 | 唯一标识符 || `canvas` | 是 | 与 Track 布局共享的显式 CanvasSpace |-| `semantic` | 二选一 | 选定表演的 SemanticTrack,提供时长与帧率 |-| `space` | 二选一 | 作者声明的 ProgramSpace;与 `semantic` 选择其一 |+| `timeline` | 是 | 完整时间轴及其中的素材放置和语义锚点 || `appearance` | 是 | SVS Film Recipe——画布清除颜色 |### film:Track@@ -48,8 +53,8 @@ Composition。然后渲染器将该 Composition 编译为 MP4 视频。| 来源 | 类型 | 来自 ||---|---|---|-| `{performance.visual}` | VisualTrack | `media-track:Track`——语义表演的画面呈现 |-| `{speech.audio}` | AudioTrack | `speech:Track`——同步音频 |+| `{performance.visual}` | VisualTrack | `performance:Track`——已有表演的画面呈现 |+| `{voice.audio}` | AudioTrack | `sound:Track`——已有声音的呈现 || `{captions.track}` | VisualTrack | Caption 样式族 Track——定时字幕 || `{cards.visual}` | VisualTrack | `media-track:Track`——Media 叠加层或 B-roll || `{titles.track}` | VisualTrack | `text:Track`——文字叠加层 |@@ -80,15 +85,14 @@ Film 收集对等的 Track。每个 Track 可以包含多个独立定时、独通过 HyperFrames 渲染器将 Composition 编译为最终视频。```svml-<render:Video id="final" composition={main.composition} semantic={speech.semantic}/>+<render:Video id="final" composition={main.composition} timeline={speech.timeline}/>```| 属性 | 必填 | 说明 ||---|---|---|| `id` | 是 | 唯一标识符 || `composition` | 是 | 来自 `film:Film` 的 Composition |-| `semantic` | 二选一 | Composition 选定的 SemanticTrack |-| `space` | 二选一 | Composition 使用的 ProgramSpace;与 `semantic` 选择其一 |+| `timeline` | 是 | 完整时间轴及其中的素材放置和语义锚点 |渲染器:@@ -107,13 +111,14 @@ Build Target,也可以直接接到媒体裁切、音频/帧提取或模型参也可以自行安排阅读节奏:声明影片时钟,让场景组件、Film 和 Render 使用它。```svml-<import as="time" from="@hypit/program-space@1"/>-<time:Space id="animation" frame-rate="30" duration="8s"/>+<import as="time" from="@hypit/timeline-author@1"/>+<time:Clock id="animation-clock" frame-rate="30"/>+<time:Timeline id="animation" clock={animation-clock} end="8s"/><!-- scene.track 由使用同一时钟的组件产生。 -->-<film:Film id="main" canvas={canvas} space={animation} appearance={recipes.film.main}>+<film:Film id="main" canvas={canvas} timeline={animation.timeline} appearance={recipes.film.main}><film:Track source={scene.track}/></film:Film>-<render:Video id="final" composition={main.composition} space={animation}/>+<render:Video id="final" composition={main.composition} timeline={animation.timeline}/>```场景事件可以采用作者指定的秒数或帧数。在口播编排中,同一种表现也可以跟随投影后的 Script@@ -135,12 +140,13 @@ Build Target,也可以直接接到媒体裁切、音频/帧提取或模型参<import as="gpt" from="@hypit/gpt-image@1"/><import as="seedance" from="@hypit/seedance@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="whisperx" from="@hypit/whisperx@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/><import as="fonts" from="@hypit/fonts-open@1"/><import as="media-track" from="@hypit/media-track@1"/>+ <import as="performance" from="@hypit/performance@1"/><import as="text" from="@hypit/typography-track@1"/><import as="space" from="@hypit/spatial@1"/><import as="program" from="@hypit/program-space@1"/>@@ -190,39 +196,46 @@ Build Target,也可以直接接到媒体裁切、音频/帧提取或模型参video="primary-moving" audio="none" span-authority="video" clock={clock}/><whisperx:SemanticTake id="opening-semantic" narrative={story}segment={story.segment.opening} media={take-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={opening-semantic.take}/>- </speech:Track>-<media-track:Track id="performance" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={speech-frame}- appearance={recipes.media.performance}/>-</media-track:Track>+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={opening-semantic.take}/>+ </time:Timeline>+<performance:Style id="performance-style" frame={speech-frame} appearance={recipes.media.performance}/>+ <performance:Track id="performance" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={performance-style} during="program"/>+ </performance:Track><!-- 4. Tracks: captions, Media, text --><fonts:Stack id="caption-font" family="inter" weight="700" style="normal"/><fonts:Stack id="title-font" family="inter" weight="900" style="normal"/><caption-fine:Style id="base-caption" recipe={recipes.caption.base} font={caption-font}/>- <caption:Program id="caption-program" document={story.caption} narrative={story}- default={base-caption}/>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={base-caption}/>+ </caption-fine:Track>- <media-track:Track id="cards" semantic={speech.semantic} canvas={vertical}>+ <media-track:Track id="cards" timeline={speech.timeline} canvas={vertical}><media-track:Item media={motion-media.media} during={story.selection.demo}frame={card-frame} appearance={recipes.media.card} motion={recipes.motion.card}/></media-track:Track><text:Style id="title-style" recipe={recipes.text.title} font={title-font}/>- <text:Track id="titles" semantic={speech.semantic}>+ <text:Track id="titles" timeline={speech.timeline}><text:Area id="meaning" placement={title-frame} style={title-style} during="program">MEANING</text:Area></text:Track><!-- 5. Film: compose all tracks -->- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <import as="sound" from="@hypit/sound@1"/>+<sound:Style id="voice-style"/>+<sound:Track id="voice" timeline={speech.timeline}>+ <sound:Use style={voice-style}/>+</sound:Track>++<film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={performance.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={voice.audio}/><film:Track source={cards.visual}/><film:Track source={captions.track}/><film:Track source={titles.track}/>@@ -230,7 +243,7 @@ Build Target,也可以直接接到媒体裁切、音频/帧提取或模型参<!-- 6. Render: compile to MP4 --><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>```diff --git a/docs/zh/quickstart/generation.md b/docs/zh/quickstart/generation.mdindex ec16c1b4..68c94dc3 100644--- a/docs/zh/quickstart/generation.md+++ b/docs/zh/quickstart/generation.md@@ -306,5 +306,5 @@ dialogue/action 由普通 Text 模块组装,结果再像其他生成任务一</seedance:ReferenceVideo>```-每个 `seedance:ReferenceVideo` 产出 `{*.video}`,进入下一阶段的 `speech:Track`。不同 Take+每个 `seedance:ReferenceVideo` 产出 `{*.video}`,进入下一阶段的 `time:Timeline`。不同 Take可以使用不同参考图,同时共享相同的音色与 Prompt Recipe。diff --git a/docs/zh/quickstart/images.md b/docs/zh/quickstart/images.mdindex 358d6854..b7c5e793 100644--- a/docs/zh/quickstart/images.md+++ b/docs/zh/quickstart/images.md@@ -95,4 +95,4 @@ description: 在图片流向生成器或 Track 之前,对它做合成、校正<matte:Portrait id="cutout" source={performance.video}/>```-将 `cutout.video` 归一化,准备进入时间线。如果它建立说话节目的语义骨架,再把准备好的媒体与 Script Segment 对齐,通过 Speech Track 装配。画面由 Media Track 或项目场景按选择的位置、绘制顺序呈现。作为 B-roll 时,归一化后的抠像可以直接进入 Media Track。抠像改变画面背景,具体角色由编排决定。+将 `cutout.video` 归一化,准备进入时间线。如果它建立说话节目的语义骨架,再把准备好的媒体与 Script Segment 对齐,通过 Timeline assembly 装配。画面由 Media Track 或项目场景按选择的位置、绘制顺序呈现。作为 B-roll 时,归一化后的抠像可以直接进入 Media Track。抠像改变画面背景,具体角色由编排决定。diff --git a/docs/zh/quickstart/preview.md b/docs/zh/quickstart/preview.mdindex ee635031..ea835940 100644--- a/docs/zh/quickstart/preview.md+++ b/docs/zh/quickstart/preview.md@@ -21,7 +21,7 @@ hypit studio --run build.svrun| `--workspace <directory>` | 设置 Source 访问与修改的项目边界。 || `--port <number>` | 指定浏览器服务端口,默认请求 `5179`。 |-所选目标需要指向一个 Film 及其时间来源。说话表演可以提供 SemanticTrack,动画可以使用作者声明的 ProgramSpace,两者都支持视觉组件与属性编辑。显示编排所需的素材应已通过 Run 提供。Studio 可以完成所选 Runtime 支持的媒体准备;生成和编码渲染通过 `hypit build` 提交。+所选目标需要指向一个 Film 及其时间来源。说话表演可以提供 Timeline,动画可以使用作者声明的 ProgramSpace,两者都支持视觉组件与属性编辑。显示编排所需的素材应已通过 Run 提供。Studio 可以完成所选 Runtime 支持的媒体准备;生成和编码渲染通过 `hypit build` 提交。## 浏览项目diff --git a/docs/zh/quickstart/script.md b/docs/zh/quickstart/script.mdindex 5f198fbc..f700b349 100644--- a/docs/zh/quickstart/script.md+++ b/docs/zh/quickstart/script.md@@ -103,7 +103,7 @@ dialogue `Text` 包含 Role Cue 前缀,speech `Text` 和 CaptionDocument 会如果确实要显示 at-sign,请写成 `\@`。`||` 是 **Caption Cue Break** 语法,只能位于完整对齐单元之间,不能写进 Dual Text 或切开-N:M 单元。字幕稍后才把 CaptionDocument 与 SemanticTrack 汇合得到帧时间。+N:M 单元。字幕稍后才把 CaptionDocument 与 Timeline 汇合得到帧时间。### 扁平词属性@@ -178,7 +178,7 @@ Selection 不要求像 XML 标签那样嵌套,它们可以互相交叉:Selection 标记是零宽度的,不会出现在任何文本投影中。它们编译为带有`startAnchorId`/`endAnchorId` 的 `NarrativeSelection`。Script 本身不包含秒数或帧号——时间-信息来自 SemanticTrack 对齐。+信息来自 Timeline 对齐。其他组件通过 `{story.selection.problem}` 引用 Selection,将视觉内容绑定到叙事中的语义时刻。diff --git a/docs/zh/quickstart/styles.md b/docs/zh/quickstart/styles.mdindex fc459854..042af264 100644--- a/docs/zh/quickstart/styles.md+++ b/docs/zh/quickstart/styles.md@@ -57,7 +57,7 @@ film.vertical {```svml<space:Canvas id="vertical" width="1080" height="1920"/>-<film:Film id="main" canvas={vertical} semantic={speech.semantic} appearance={recipes.film.vertical}>+<film:Film id="main" canvas={vertical} timeline={speech.timeline} appearance={recipes.film.vertical}>```## Caption Fine@@ -130,17 +130,18 @@ caption.bob {}```-然后通过 `caption:Program` 进行分配:+然后在 Track 中通过 Use 选择呈现样式:```svml<fonts:Stack id="caption-font" family="inter" weight="600" style="normal"/><caption-fine:Style id="default-caption" recipe={recipes.caption.dialogue} font={caption-font}/><caption-fine:Style id="alice-caption" recipe={recipes.caption.alice} font={caption-font}/><caption-fine:Style id="bob-caption" recipe={recipes.caption.bob} font={caption-font}/>-<caption:Program id="caption-program" document={story.caption} default={default-caption}>- <caption:Use role="ALICE" style={alice-caption}/>- <caption:Use role="BOB" style={bob-caption}/>-</caption:Program>+<caption-fine:Track id="captions" document={story.caption} timeline={speech.timeline}>+ <caption-fine:Use style={default-caption}/>+ <caption-fine:Use role="ALICE" style={alice-caption}/>+ <caption-fine:Use role="BOB" style={bob-caption}/>+</caption-fine:Track>```## Media Track@@ -380,5 +381,5 @@ Caption Recipe 不再重复家族、字重或字形。CJK 与 Emoji 即使由多<caption-fine:Style id="primary-caption" recipe={recipes.caption.primary} font={caption-font}/><space:Canvas id="vertical" width="720" height="1280"/>-<film:Film id="main" canvas={vertical} semantic={speech.semantic} appearance={recipes.film.vertical}>+<film:Film id="main" canvas={vertical} timeline={speech.timeline} appearance={recipes.film.vertical}>```diff --git a/docs/zh/quickstart/timing.md b/docs/zh/quickstart/timing.mdindex f4430dc8..27fedbb2 100644--- a/docs/zh/quickstart/timing.md+++ b/docs/zh/quickstart/timing.md@@ -1,24 +1,25 @@---title: 时序与装配-description: 逐 Take 归一化与语义对齐,然后装配为 SemanticTrack。+description: 逐 Take 归一化与语义对齐,然后装配为 Timeline。----对于说话视频,`SemanticTrack` 把作者的 Script 与实际表演联系起来,是字幕、随词语出现的图形和覆盖画面的自然时间来源。它按 Segment 粒度构建:+对于说话视频,`Timeline` 把作者的 Script 与实际表演联系起来,是字幕、随词语出现的图形和覆盖画面的自然时间来源。它按 Segment 粒度构建:1. 把每个已接受的音视频 Take 归一化到同一个精确帧域;2. 将归一化媒体与对应的 Script Segment 对齐,得到自包含的 `SemanticTake`;-3. 用 `speech:Track` 按节目顺序装配这些 Semantic Take。+3. 用 `time:Timeline` 按节目顺序装配这些 Semantic Take。-每个 Take 在进入 Speech Track 之前就已经具有语义。画面由 Media Track 或项目组件呈现,与这里的语义和音频装配分别表达。+每个 Take 在进入 Timeline assembly 之前就已经具有语义。画面由 Media Track 或项目组件呈现,与这里的语义和音频装配分别表达。-纯视觉动画也可以通过 ProgramSpace 声明自己的时长与帧率,见 [纯组件绘制的影片](./composition.md)。其事件可以使用秒或帧;说话视频则可以用 Script Selection 和 Moment 驱动相同的视觉行为。+纯视觉动画使用同一种 Timeline 声明:指定结束时间,不放入 Take,见 [纯组件绘制的影片](./composition.md)。其事件可以使用秒或帧;说话视频则可以用 Script Selection 和 Moment 驱动相同的视觉行为。```svml<import as="program" from="@hypit/program-space@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>-<import as="speech" from="@hypit/speech-track@1"/>+<import as="time" from="@hypit/timeline-author@1"/><import as="media-track" from="@hypit/media-track@1"/>+ <import as="performance" from="@hypit/performance@1"/><import as="space" from="@hypit/spatial@1"/><import as="recipes" source="./recipes.svs"/>```@@ -26,7 +27,7 @@ description: 逐 Take 归一化与语义对齐,然后装配为 SemanticTrack## 逐 Take 归一化归一化把视频、音频、时长和帧率变成一个明确的 `SynchronizedMedia` 事实。同一条-SemanticTrack 内的所有 Take 共享作者显式声明的 Clock。+Timeline 内的所有 Take 共享作者显式声明的 Clock。```svml<program:Clock id="clock" frame-rate="30"/>@@ -57,40 +58,49 @@ Hypit 不会根据 Script 文本或音频自动检测、分流语言。结构锚点:Segment 有两个锚点,每个词也有两个锚点。声学证据只是这一步的实现输入;下游组件看到的是完成后的 `SemanticTake`,而不是第二套 evidence 形状的时间结构。-## 装配 SemanticTrack+## 装配 Timeline-`speech:Track` 按文档顺序拼接已经语义化的 Take,提供语义时间线及原声音频;Media 独立呈现表演画面:+`time:Timeline` 默认顺接已经准备好的 Take,也允许通过 `at` 自由放置,提供完整 Timeline。+Performance 和 Sound 分别呈现其中的画面和声音:```svml<space:Canvas id="vertical" width="1080" height="1920"/><space:Frame id="speech-frame" within={vertical}left="0%" top="0%" right="100%" bottom="100%"/>-<speech:Track id="speech">- <speech:Take source={opening-semantic.take}/>- <speech:Take source={answer-semantic.take}/>-</speech:Track>-<media-track:Track id="performance" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={speech-frame}- appearance={recipes.media.performance}/>-</media-track:Track>+<time:Timeline id="speech" clock={clock}>+ <time:Take source={opening-semantic.take}/>+ <time:Take source={answer-semantic.take}/>+</time:Timeline>+<import as="sound" from="@hypit/sound@1"/>+<sound:Style id="voice-style"/>+<sound:Track id="voice" timeline={speech.timeline}>+ <sound:Use style={voice-style}/>+</sound:Track>+<performance:Style id="performance-style" frame={speech-frame} appearance={recipes.media.performance}/>+ <performance:Track id="performance" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={performance-style} during="program"/>+ </performance:Track>```| 输出 | 类型 | 含义 ||---|---|---|-| `{speech.semantic}` | SemanticTrack | 全局语义与帧域真相 |-| `{speech.audio}` | AudioTrack | 与语义 item 对齐的同源声音 |+| `{speech.timeline}` | Timeline | 全局语义与帧域真相 |+| `{voice.audio}` | AudioTrack | Sound 对已有 Take 声音的呈现 |-画面组件和原声音频使用同一批素材及源位置。`SemanticTrack` 通过局部 Take 长度的前缀和-得到全局帧位置,同时提供 Film 和 Render 所需的节目时长与帧域。+Performance 和 Sound 使用同一批素材及源位置。每个 Take 的全局位置由放置起点加局部位置得到。+第一段省略 `at` 表示从零开始,后续省略则顺接上一段。`at="previous.end+2s"` 留出间隔,+`at="previous.end-12f"` 表示交叠,也可以写绝对位置。Timeline 的 `end` 默认取所有 Take 的最晚终点;+`end="content.end+2s"` 留出片尾,`end="30s"` 指定完整时长。位置须落在精确帧上,完整范围须容纳所有 Take。+零 Take 的 Timeline 需要明确的正时长,空隔不生成占位素材。## 消费语义时间Selection、Moment 与完整 Segment 始终是 Script 中的作者身份。下游组件只接收一次-SemanticTrack,并在构建确定性 Track 时把这些身份投影成帧:+Timeline,并在构建确定性 Track 时把这些身份投影成帧:```svml-<media-track:Track id="cards" semantic={speech.semantic} canvas={vertical}>+<media-track:Track id="cards" timeline={speech.timeline} canvas={vertical}><media-track:Item image={card.image} extent={card-extent}during={story.selection.demo} frame={card-frame}appearance={recipes.media.card} motion={recipes.motion.card}/>@@ -98,26 +108,28 @@ SemanticTrack,并在构建确定性 Track 时把这些身份投影成帧:<caption-fine:Track id="captions"document={story.caption}- semantic={speech.semantic}- program={caption-program}/>+ timeline={speech.timeline}+>+ <caption-fine:Use style={primary-caption}/>+</caption-fine:Track><film:Film id="main" canvas={vertical}- semantic={speech.semantic} appearance={recipes.film.vertical}>+ timeline={speech.timeline} appearance={recipes.film.vertical}><film:Track source={performance.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={voice.audio}/><film:Track source={cards.visual}/><film:Track source={captions.track}/></film:Film><render:Video id="final"- composition={main.composition} semantic={speech.semantic}/>+ composition={main.composition} timeline={speech.timeline}/>```-整段使用 `during={story.segment.answer}`,作者范围使用 Selection,点事件使用 Moment,完整节目使用 `during="program"`。组件统一消费 `semantic={speech.semantic}`。+整段使用 `during={story.segment.answer}`,作者范围使用 Selection,点事件使用 Moment,完整节目使用 `during="program"`。组件统一消费 `timeline={speech.timeline}`。```text-prepared Takes → Speech Track ── .semantic → Media / project scene → .visual ─┐- │ └──→ Caption / semantic graphics ───┤- └───── .audio ────────────────────────────────────┤- Film+prepared Takes → Timeline → Performance / project scene → visual ─┐+ ├→ Caption / semantic graphics → visual ┤+ └→ Sound → audio ───────────────────────┤+ Film```diff --git a/docs/zh/quickstart/tracks.md b/docs/zh/quickstart/tracks.mdindex 2dcc6397..a1a31753 100644--- a/docs/zh/quickstart/tracks.md+++ b/docs/zh/quickstart/tracks.md@@ -12,7 +12,7 @@ Track 的作者语法。字幕由 Script 产生的单一文档与可替换的样式族组成:```text-Script CaptionDocument → Caption Program → SemanticTrack 定时 → 样式族 Track+Script CaptionDocument + Timeline → 完整 Cue 定时 → Use 呈现``````svml@@ -23,7 +23,7 @@ Script CaptionDocument → Caption Program → SemanticTrack 定时 → 样式```公共 Caption 负责 CaptionDocument、完整 Alignment Unit 的 Selection/Role 投影、样式分配与-SemanticTrack 时间拼接。Fine 是一种样式族,负责自己的几何、字形/Cue/Pill Paint 与局部动画。+Timeline 时间拼接。Fine 是一种样式族,负责自己的几何、字形/Cue/Pill Paint 与局部动画。### caption-fine:Style@@ -73,35 +73,22 @@ Fine 是“统一文字流”字幕:同一 Cue 的每个 token 遵守同一 ReCJK 口播可以直接书写。若一个只负责显示的 emoji 仍需跟随语音计时,应显式写出对应,例如 `<🌐 | globe>`;系统不会替裸符号虚构一个口播词。-### caption:Program--Program 消费 Script 显式输出的完整有序 CaptionDocument。一个必填的默认 Style 自动覆盖-所有 Alignment Unit。`Use` 按源码顺序替换整个 Style,后命中者获胜。--```svml-<caption:Program id="caption-program" document={story.caption} narrative={story}- default={primary-caption}>- <caption:Use role="ALICE" style={alice-caption}/>- <caption:Use role="BOB" style={bob-caption}/>- <caption:Use selection={story.selection.product-demo}- style={dialogue-caption}/>- <caption:Mute selection={story.selection.private}/>-</caption:Program>-```--`role=` 是词子集查询,不是时间条件。`selection=` 把公开的语义 Selection 投影为完整的-Caption Alignment Unit;如果切到 N:M Dual Text 的中间,会直接拒绝。`Mute` 消费同一份投影,-不会重新分 Cue。-### caption-fine:Track+字幕内容来自 Script 和 Timeline;Use 决定某段时间的呈现方式。后声明的 Use 在窗口内覆盖前面的样式,隐藏也遵守这个规则。+```svml-<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+<caption:Hidden id="hidden"/>+<caption-fine:Track id="captions" document={story.caption} timeline={speech.timeline}>+ <caption-fine:Use style={primary-caption}/>+ <caption-fine:Use role="ALICE" style={alice-caption}/>+ <caption-fine:Use role="BOB" style={bob-caption}/>+ <caption-fine:Use during={story.selection.product-demo} style={dialogue-caption}/>+ <caption-fine:Use during={story.selection.private} style={hidden}/>+</caption-fine:Track>```-Caption 把每个完整 unit 与独立 SemanticTrack 对齐,Fine 再把默认/覆盖样式渲染成普通的对等-`VisualTrack`:`{captions.track}`。+`||` 决定 Cue 分组。窗口可以从 Cue 中间开始,完整文字和原来的逐词时间仍然保留。`role` 按说话人过滤内容,独立于时间窗口。也可以使用 `at`/`for`、`until`/`for` 或 `start`/`end`。## Media 叠加层与 B-roll@@ -133,7 +120,7 @@ B-roll 是通用 Media Track 的一种剪辑用途,不是独立 Track 家族<pipeline:Normalize id="product-media" source={product-motion.video}video="primary-moving" audio="none" span-authority="video" frame-rate="30"/>-<media-track:Track id="product-broll" semantic={speech.semantic} canvas={vertical}>+<media-track:Track id="product-broll" timeline={speech.timeline} canvas={vertical}><media-track:Item media={product-media.media} frame={product-frame}during={story.selection.product-demo}appearance={recipes.media.product}@@ -158,7 +145,7 @@ B-roll 是通用 Media Track 的一种剪辑用途,不是独立 Track 家族## Audio Track-`@hypit/audio-track` 把显式准备好的音频放进与视觉 Track 相同的 ProgramSpace。`Clip`+`@hypit/audio-track` 把显式准备好的音频放进与视觉 Track 相同的 Timeline。`Item`消费 `SynchronizedMedia`;先规范化已声明或生成的音频 Blob,再选择精确节目窗口与占用方式:```svml@@ -170,8 +157,8 @@ B-roll 是通用 Media Track 的一种剪辑用途,不是独立 Track 家族<pipeline:Normalize id="music-media" source={music}video="none" audio="default" span-authority="audio" frame-rate="30"/>-<audio:Track id="music-bed" semantic={speech.semantic}>- <audio:Clip source={music-media.media} during="program"+<audio:Track id="music-bed" timeline={speech.timeline}>+ <audio:Item source={music-media.media} during="program"playback="loop-end" gain="0.28" fade-in="600ms" fade-out="800ms"/></audio:Track>```@@ -179,15 +166,15 @@ B-roll 是通用 Media Track 的一种剪辑用途,不是独立 Track 家族| 属性 | 必填 | 描述 ||---|---|---|| `Track.id` | 是 | 稳定的 Audio Track 身份 |-| `Track.semantic` | 是 | 定义精确采样域与帧域的 SemanticTrack |-| `Clip.source` | 是 | 显式选流并规范化后的 `SynchronizedMedia` |+| `Track.timeline` | 是 | 定义精确采样域与帧域的 Timeline |+| `Item.source` | 是 | 显式选流并规范化后的 `SynchronizedMedia` || `during`、`at`/`for` 或 `start`/`end` | 三种形式选一 | 全节目、Selection、Moment 或显式窗口 || `playback` | 否 | `once`、`once-end`、`loop`、`loop-end` 或有界 `stretch` || `trim-start`、`trim-end` | 否 | 精确源裁切 |-| `gain`、`fade-in`、`fade-out` | 否 | 显式的单 Clip 混音值 |+| `gain`、`fade-in`、`fade-out` | 否 | 显式的单 Item 混音值 |-该包不会自动提取、规范化、duck 或分配 bus。同一 Track 内的多个 Clip 与多个对等 Audio-Track 都会作为独立输入进入 Film。输出 `{music-bed.track}` 是普通 `AudioTrack`。+该包不会自动提取、规范化、duck 或分配 bus。同一 Track 内的多个 Item 与多个对等 Audio+Track 都会作为独立输入进入 Film。输出 `{music-bed.audio}` 是普通 `AudioTrack`。## 文字叠加层@@ -208,7 +195,7 @@ Track 都会作为独立输入进入 Film。输出 `{music-bed.track}` 是普通left="6%" top="6%" right="94%" bottom="16%"/><fonts:Stack id="title-font" family="inter" weight="900" style="normal"/><text:Style id="title-style" recipe={recipes.text.title} font={title-font}/>-<text:Track id="titles" semantic={speech.semantic}>+<text:Track id="titles" timeline={speech.timeline}><text:Area id="title" placement={title-frame} style={title-style} during="program">EDIT MEANING, NOT TIMELINES</text:Area>@@ -218,7 +205,7 @@ Track 都会作为独立输入进入 Film。输出 `{music-bed.track}` 是普通| 属性 | 必填 | 描述 ||---|---|---|| `id` | 是 | 唯一标识符 |-| `semantic` | 是 | 来自 `speech:Track` 的 SemanticTrack——也用于解析基于 Selection 的项目计时 |+| `semantic` | 是 | 来自 `time:Timeline` 的 Timeline——也用于解析基于 Selection 的项目计时 |### text:Point、text:Area 与 text:Path@@ -243,7 +230,7 @@ Track 都会作为独立输入进入 Film。输出 `{music-bed.track}` 是普通```svml<text:Style id="callout-style" recipe={recipes.text.callout} font={title-font}/>-<text:Track id="callout" semantic={speech.semantic}>+<text:Track id="callout" timeline={speech.timeline}><text:Area id="callout-copy" placement={callout-frame}style={callout-style} during={story.selection.callout}>EXACTLY THE RIGHT MOMENT@@ -255,7 +242,7 @@ Track 都会作为独立输入进入 Film。输出 `{music-bed.track}` 是普通```svml<wording:Value id="headline">EXACTLY THE RIGHT MOMENT</wording:Value>-<text:Track id="callout" semantic={speech.semantic}>+<text:Track id="callout" timeline={speech.timeline}><text:Area id="callout-copy" content={headline}placement={callout-frame} style={callout-style} during="program"/></text:Track>@@ -287,7 +274,7 @@ Run 时,继续使用内联 `P`/`Span`/`Break`。| 属性 | 取值 ||---|---|-| `semantic` | 板据以计时的 SemanticTrack |+| `semantic` | 板据以计时的 Timeline || `frame` | 一个 `space:Frame`——选中组件声明的板面位置 || `during` | 选中组件声明的时间形式 || `style` | 对应的样式记录,且只接受本变体的 |@@ -306,7 +293,7 @@ Run 时,继续使用内联 `P`/`Span`/`Break`。```svml<ranking:ColumnStyle id="board-style" recipe={recipes.ranking.board} font={ui-font}/>-<ranking:Column id="board" semantic={speech.semantic} canvas={vertical} frame={board-frame}+<ranking:Column id="board" timeline={speech.timeline} canvas={vertical} frame={board-frame}during={story.selection.board} style={board-style}><ranking:ColumnItem id="row-regen" rank="1" label="ReGen" icon={icon-regen}during={story.selection.regen-reveal}/>@@ -348,7 +335,7 @@ DepthStack 的直接子元素,自闭合,至少一张,按书写顺序发出```svml<space:Frame id="deck-frame" within={vertical} left="44%" top="60%" right="98%" bottom="88%"/>-<deck:DepthStack id="deck" semantic={speech.semantic} canvas={vertical}+<deck:DepthStack id="deck" timeline={speech.timeline} canvas={vertical}frame={deck-frame} appearance={recipes.deck.stack} until={story.moment.done}><deck:Card id="card-spatial" source={icon-spatial} extent={square} at={story.moment.deal-one}/><deck:Card id="card-type" source={icon-type} extent={square} at={story.moment.deal-two}/>@@ -365,13 +352,13 @@ DepthStack 的直接子元素,自闭合,至少一张,按书写顺序发出<import as="screen" from="@hypit/screen-overlay@1"/>```-`screen:Track` 接受 `id`、`canvas`,以及 `semantic` 或 `space` 之一作为时间来源。它的子元素就是各个效果,至少一个,各自为空,都必须带 `z` 决定层叠顺序,并且各有一个时间窗,形式是以下之一:+`screen:Track` 接受 `id`、`canvas`,以及 统一的 `timeline` 时间来源。它的子元素就是各个效果,至少一个,各自为空,都必须带 `z` 决定层叠顺序,并且各有一个时间窗,形式是以下之一:| 时间窗 | 写法 ||---|---|| 整个节目 | `during="program"` |-| 一个 Selection | 在带有 `semantic={speech.semantic}` 的 Track 内写 `during={story.selection.x}` |-| 一个 Moment,持续一段时长 | 在带有 `semantic={speech.semantic}` 的 Track 内写 `at={story.moment.x} for="12f"` |+| 一个 Selection | 在带有 `timeline={speech.timeline}` 的 Track 内写 `during={story.selection.x}` |+| 一个 Moment,持续一段时长 | 在带有 `timeline={speech.timeline}` 的 Track 内写 `at={story.moment.x} for="12f"` || 显式区间 | `start="…" end="…"`,可另外指定 `selection=` 或 `moment=` |时长写作 `12f`、`250ms` 或 `1.5s`。一个 Selection@@ -380,7 +367,7 @@ DepthStack 的直接子元素,自闭合,至少一张,按书写顺序发出可用的效果有十一种——`Flash`、`ColorWash`、`Vignette`、`ScanLines`、`DirectionalMatte`、`WhipVeil`、`GlitchVeil`、`Grain`、`LightLeak`、`Bokeh` 与 `TVStatic`——每种各有自己的必填属性,例如 `Flash` 的 `color` / `intensity` / `attack` / `hold` / `decay`,或 `Vignette` 的 `center-x` / `center-y` / `radius-x` / `radius-y` / `softness` / `color` / `opacity`。它们都没有默认值:一个效果要么把自己的形状说全,要么被拒绝。```svml-<screen:Track id="effects" semantic={speech.semantic} canvas={vertical}>+<screen:Track id="effects" timeline={speech.timeline} canvas={vertical}><screen:Flash during={story.selection.overlay} z="80"color="#ffffff" intensity="0.6" attack="2" hold="2" decay="6"/></screen:Track>@@ -404,7 +391,7 @@ DepthStack 的直接子元素,自闭合,至少一张,按书写顺序发出```svml<comment:Style id="social" recipe={recipes.comment} font={ui-font}/>-<comment:Track id="comments" canvas={vertical} semantic={speech.semantic}>+<comment:Track id="comments" canvas={vertical} timeline={speech.timeline}><comment:Sticker id="one" frame={comment-frame} style={social} avatar={viewer-avatar}author="@viewer" meta="Featured" during={story.selection.reaction}>原来它把字幕钉在词上,而不是钉在秒上。@@ -433,9 +420,11 @@ DepthStack 的直接子元素,自闭合,至少一张,按书写顺序发出<fonts:Stack id="caption-font" family="inter" weight="700" style="normal"/><fonts:Stack id="title-font" family="inter" weight="900" style="normal"/><caption-fine:Style id="base-caption" recipe={recipes.caption.base} font={caption-font}/>-<caption:Program id="caption-program" document={story.caption} narrative={story} default={base-caption}/>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={base-caption}/>+ </caption-fine:Track><!-- 共享位置是显式边,与 Media/Text 外观分开。 --><space:Canvas id="vertical" width="1080" height="1920"/>@@ -447,14 +436,14 @@ DepthStack 的直接子元素,自闭合,至少一张,按书写顺序发出<!-- Media:Selection 期间显示一个普通 Item --><pipeline:Normalize id="card-media" source={motion.video}video="primary-moving" audio="none" span-authority="video" frame-rate="30"/>-<media-track:Track id="cards" semantic={speech.semantic} canvas={vertical}>+<media-track:Track id="cards" timeline={speech.timeline} canvas={vertical}><media-track:Item media={card-media.media} frame={card-frame}during={story.selection.demo} appearance={recipes.media.card} motion={recipes.motion.card}/></media-track:Track><!-- Text: persistent title overlay --><text:Style id="title-style" recipe={recipes.text.title} font={title-font}/>-<text:Track id="titles" semantic={speech.semantic}>+<text:Track id="titles" timeline={speech.timeline}><text:Area id="meaning" placement={title-frame} style={title-style} during="program">MEANING</text:Area>@@ -464,19 +453,25 @@ DepthStack 的直接子元素,自闭合,至少一张,按书写顺序发出<media:Audio id="music" src="./assets/music.wav"/><pipeline:Normalize id="music-media" source={music}video="none" audio="default" span-authority="audio" frame-rate="30"/>-<audio:Track id="music-bed" semantic={speech.semantic}>- <audio:Clip source={music-media.media} during="program"+<audio:Track id="music-bed" timeline={speech.timeline}>+ <audio:Item source={music-media.media} during="program"playback="loop-end" gain="0.28" fade-in="600ms" fade-out="800ms"/></audio:Track><!-- 所有对等 Track 都进入 Film -->-<film:Film id="main" canvas={vertical} semantic={speech.semantic} appearance={recipes.film.vertical}>+<import as="sound" from="@hypit/sound@1"/>+<sound:Style id="voice-style"/>+<sound:Track id="voice" timeline={speech.timeline}>+ <sound:Use style={voice-style}/>+</sound:Track>++<film:Film id="main" canvas={vertical} timeline={speech.timeline} appearance={recipes.film.vertical}><film:Track source={performance.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={voice.audio}/><film:Track source={cards.visual}/><film:Track source={captions.track}/><film:Track source={titles.track}/>- <film:Track source={music-bed.track}/>+ <film:Track source={music-bed.audio}/></film:Film>```diff --git a/examples/interview/README.md b/examples/interview/README.mdindex f8873a69..1c77b171 100644--- a/examples/interview/README.md+++ b/examples/interview/README.md@@ -65,7 +65,10 @@ After `vertical` is declared, add its measured timeline and connect it to the ex```svml<space:RegionTimeline id="wife-heads" within={vertical} recipe={heads.heads.wife}/><caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program} regions={wife-heads}/>+ timeline={speech.timeline} regions={wife-heads}>+ <caption-fine:Use style={caption-boy-style}/>+ <caption-fine:Use role="WIFE" style={caption-wife-style}/>+</caption-fine:Track>```Replace the existing `captions` declaration with the connected one. The Recipe contains a WIFE Rolediff --git a/examples/interview/reference.svml b/examples/interview/reference.svmlindex cf0ac87d..a8d3fe0f 100644--- a/examples/interview/reference.svml+++ b/examples/interview/reference.svml@@ -2,6 +2,8 @@<svml><import as="media-track" from="@hypit/media-track@1"/>+ <import as="sound" from="@hypit/sound@1"/>+ <import as="performance" from="@hypit/performance@1"/><import from="@hypit/script@1"/><import as="text" from="@hypit/text@1"/><import as="media" from="@hypit/media@1"/>@@ -11,7 +13,7 @@<import as="program" from="@hypit/program-space@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="audio-track" from="@hypit/audio-track@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/>@@ -248,27 +250,32 @@<whisperx:SemanticTake id="bitcoin-rule-semantic" narrative={story}segment={story.segment.bitcoin-rule} media={bitcoin-rule-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={manifest-rule-semantic.take}/>- <speech:Take source={real-estate-rule-semantic.take}/>- <speech:Take source={bitcoin-rule-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={full-frame}- appearance={recipes.media.performance}/>- </media-track:Track>+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={manifest-rule-semantic.take}/>+ <time:Take source={real-estate-rule-semantic.take}/>+ <time:Take source={bitcoin-rule-semantic.take}/>+ </time:Timeline>- <audio-track:Track id="music-bed" semantic={speech.semantic}>- <audio-track:Clip source={soundtrack-media.media} during="program"+ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={full-frame} appearance={recipes.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track>++ <audio-track:Track id="music-bed" timeline={speech.timeline}>+ <audio-track:Item source={soundtrack-media.media} during="program"playback="once-start" gain="0.12" fade-in="300ms" fade-out="600ms"/></audio-track:Track>- <audio-track:Track id="rule-reveal-sfx" semantic={speech.semantic}>- <audio-track:Clip id="manifest-reveal-sfx" source={rule-reveal-sound-media.media}+ <audio-track:Track id="rule-reveal-sfx" timeline={speech.timeline}>+ <audio-track:Item id="manifest-reveal-sfx" source={rule-reveal-sound-media.media}at={story.moment.manifest} for="20f" playback="once-start" gain="0.20"/>- <audio-track:Clip id="real-estate-reveal-sfx" source={rule-reveal-sound-media.media}+ <audio-track:Item id="real-estate-reveal-sfx" source={rule-reveal-sound-media.media}at={story.moment.real-estate} for="20f" playback="once-start" gain="0.20"/>- <audio-track:Clip id="bitcoin-reveal-sfx" source={rule-reveal-sound-media.media}+ <audio-track:Item id="bitcoin-reveal-sfx" source={rule-reveal-sound-media.media}at={story.moment.bitcoin} for="20f" playback="once-start" gain="0.20"/></audio-track:Track>@@ -277,22 +284,22 @@font={caption-font}/><caption-fine:Style id="caption-wife-style" recipe={recipes.caption.wife}font={caption-font}/>- <caption:Program id="caption-program" document={story.caption}- narrative={story} default={caption-boy-style}>- <caption:Use role="WIFE" style={caption-wife-style}/>- </caption:Program>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={caption-boy-style}/>+ <caption-fine:Use role="WIFE" style={caption-wife-style}/>+ </caption-fine:Track><emoji:Style id="rule-reveal-style" recipe={recipes.emoji.strip}/>- <emoji:Track id="rule-reveal" semantic={speech.semantic} canvas={vertical}+ <emoji:Track id="rule-reveal" timeline={speech.timeline} canvas={vertical}style={rule-reveal-style} placeholder={rule-placeholder} during="program"><emoji:Item id="manifest" icon={manifest-icon} at={story.moment.manifest}/><emoji:Item id="real-estate" icon={real-estate-icon} at={story.moment.real-estate}/><emoji:Item id="bitcoin" icon={bitcoin-icon} at={story.moment.bitcoin}/></emoji:Track>- <screen:Track id="rule-reveal-flashes" semantic={speech.semantic} canvas={vertical}>+ <screen:Track id="rule-reveal-flashes" timeline={speech.timeline} canvas={vertical}><screen:Flash id="manifest-flash" at={story.moment.manifest} for="8f" z="80"color="#FF4D6D" intensity="0.28" attack="2" hold="1" decay="5"/><screen:Flash id="real-estate-flash" at={story.moment.real-estate} for="8f" z="80"@@ -301,17 +308,17 @@color="#F59E0B" intensity="0.28" attack="2" hold="1" decay="5"/></screen:Track>- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={captions.track}/><film:Track source={rule-reveal.track}/><film:Track source={rule-reveal-flashes.track}/>- <film:Track source={music-bed.track}/>- <film:Track source={rule-reveal-sfx.track}/>+ <film:Track source={music-bed.audio}/>+ <film:Track source={rule-reveal-sfx.audio}/></film:Film><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>diff --git a/examples/interview/swap-host.svml b/examples/interview/swap-host.svmlindex 50faf7be..2d1187ca 100644--- a/examples/interview/swap-host.svml+++ b/examples/interview/swap-host.svml@@ -2,6 +2,8 @@<svml><import as="media-track" from="@hypit/media-track@1"/>+ <import as="sound" from="@hypit/sound@1"/>+ <import as="performance" from="@hypit/performance@1"/><import from="@hypit/script@1"/><import as="text" from="@hypit/text@1"/><import as="media" from="@hypit/media@1"/>@@ -10,7 +12,7 @@<import as="program" from="@hypit/program-space@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="audio-track" from="@hypit/audio-track@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/>@@ -179,27 +181,32 @@<whisperx:SemanticTake id="bitcoin-rule-semantic" narrative={story}segment={story.segment.bitcoin-rule} media={bitcoin-rule-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={manifest-rule-semantic.take}/>- <speech:Take source={real-estate-rule-semantic.take}/>- <speech:Take source={bitcoin-rule-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={full-frame}- appearance={recipes.media.performance}/>- </media-track:Track>-- <audio-track:Track id="music-bed" semantic={speech.semantic}>- <audio-track:Clip source={soundtrack-media.media} during="program"+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={manifest-rule-semantic.take}/>+ <time:Take source={real-estate-rule-semantic.take}/>+ <time:Take source={bitcoin-rule-semantic.take}/>+ </time:Timeline>++ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={full-frame} appearance={recipes.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track>++ <audio-track:Track id="music-bed" timeline={speech.timeline}>+ <audio-track:Item source={soundtrack-media.media} during="program"playback="once-start" gain="0.12" fade-in="300ms" fade-out="600ms"/></audio-track:Track>- <audio-track:Track id="rule-reveal-sfx" semantic={speech.semantic}>- <audio-track:Clip id="manifest-reveal-sfx" source={rule-reveal-sound-media.media}+ <audio-track:Track id="rule-reveal-sfx" timeline={speech.timeline}>+ <audio-track:Item id="manifest-reveal-sfx" source={rule-reveal-sound-media.media}at={story.moment.manifest} for="20f" playback="once-start" gain="0.20"/>- <audio-track:Clip id="real-estate-reveal-sfx" source={rule-reveal-sound-media.media}+ <audio-track:Item id="real-estate-reveal-sfx" source={rule-reveal-sound-media.media}at={story.moment.real-estate} for="20f" playback="once-start" gain="0.20"/>- <audio-track:Clip id="bitcoin-reveal-sfx" source={rule-reveal-sound-media.media}+ <audio-track:Item id="bitcoin-reveal-sfx" source={rule-reveal-sound-media.media}at={story.moment.bitcoin} for="20f" playback="once-start" gain="0.20"/></audio-track:Track>@@ -208,22 +215,22 @@font={caption-font}/><caption-fine:Style id="caption-wojak-style" recipe={recipes.caption.wojak}font={caption-font}/>- <caption:Program id="caption-program" document={story.caption}- narrative={story} default={caption-chad-style}>- <caption:Use role="WOJAK" style={caption-wojak-style}/>- </caption:Program>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program} regions={wojak-heads}/>+ timeline={speech.timeline} regions={wojak-heads}>+ <caption-fine:Use style={caption-chad-style}/>+ <caption-fine:Use role="WOJAK" style={caption-wojak-style}/>+ </caption-fine:Track><emoji:Style id="rule-reveal-style" recipe={recipes.emoji.strip}/>- <emoji:Track id="rule-reveal" semantic={speech.semantic} canvas={vertical}+ <emoji:Track id="rule-reveal" timeline={speech.timeline} canvas={vertical}style={rule-reveal-style} placeholder={rule-placeholder.image} during="program"><emoji:Item id="manifest" icon={manifest-icon.image} at={story.moment.manifest}/><emoji:Item id="real-estate" icon={real-estate-icon.image} at={story.moment.real-estate}/><emoji:Item id="bitcoin" icon={bitcoin-icon.image} at={story.moment.bitcoin}/></emoji:Track>- <screen:Track id="rule-reveal-flashes" semantic={speech.semantic} canvas={vertical}>+ <screen:Track id="rule-reveal-flashes" timeline={speech.timeline} canvas={vertical}><screen:Flash id="manifest-flash" at={story.moment.manifest} for="8f" z="80"color="#FF4D6D" intensity="0.28" attack="2" hold="1" decay="5"/><screen:Flash id="real-estate-flash" at={story.moment.real-estate} for="8f" z="80"@@ -232,17 +239,17 @@color="#F59E0B" intensity="0.28" attack="2" hold="1" decay="5"/></screen:Track>- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={captions.track}/><film:Track source={rule-reveal.track}/><film:Track source={rule-reveal-flashes.track}/>- <film:Track source={music-bed.track}/>- <film:Track source={rule-reveal-sfx.track}/>+ <film:Track source={music-bed.audio}/>+ <film:Track source={rule-reveal-sfx.audio}/></film:Film><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>diff --git a/examples/interview/swap-lang.svml b/examples/interview/swap-lang.svmlindex c2da1934..734f5b3c 100644--- a/examples/interview/swap-lang.svml+++ b/examples/interview/swap-lang.svml@@ -2,6 +2,8 @@<svml><import as="media-track" from="@hypit/media-track@1"/>+ <import as="sound" from="@hypit/sound@1"/>+ <import as="performance" from="@hypit/performance@1"/><import from="@hypit/script@1"/><import as="text" from="@hypit/text@1"/><import as="media" from="@hypit/media@1"/>@@ -11,7 +13,7 @@<import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/><import as="adjust" from="@hypit/semantic-take-adjust@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="audio-track" from="@hypit/audio-track@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/>@@ -183,27 +185,32 @@<adjust:Anchor at={story.moment.aja} frame="63"/></adjust:SemanticTake>- <speech:Track id="speech">- <speech:Take source={manifest-rule-corrected.take}/>- <speech:Take source={real-estate-rule-semantic.take}/>- <speech:Take source={bitcoin-rule-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={full-frame}- appearance={recipes.media.performance}/>- </media-track:Track>-- <audio-track:Track id="music-bed" semantic={speech.semantic}>- <audio-track:Clip source={soundtrack-media.media} during="program"+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={manifest-rule-corrected.take}/>+ <time:Take source={real-estate-rule-semantic.take}/>+ <time:Take source={bitcoin-rule-semantic.take}/>+ </time:Timeline>++ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={full-frame} appearance={recipes.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track>++ <audio-track:Track id="music-bed" timeline={speech.timeline}>+ <audio-track:Item source={soundtrack-media.media} during="program"playback="once-start" gain="0.12" fade-in="300ms" fade-out="600ms"/></audio-track:Track>- <audio-track:Track id="rule-reveal-sfx" semantic={speech.semantic}>- <audio-track:Clip id="manifest-reveal-sfx" source={rule-reveal-sound-media.media}+ <audio-track:Track id="rule-reveal-sfx" timeline={speech.timeline}>+ <audio-track:Item id="manifest-reveal-sfx" source={rule-reveal-sound-media.media}at={story.moment.manifest} for="20f" playback="once-start" gain="0.20"/>- <audio-track:Clip id="real-estate-reveal-sfx" source={rule-reveal-sound-media.media}+ <audio-track:Item id="real-estate-reveal-sfx" source={rule-reveal-sound-media.media}at={story.moment.real-estate} for="20f" playback="once-start" gain="0.20"/>- <audio-track:Clip id="bitcoin-reveal-sfx" source={rule-reveal-sound-media.media}+ <audio-track:Item id="bitcoin-reveal-sfx" source={rule-reveal-sound-media.media}at={story.moment.bitcoin} for="20f" playback="once-start" gain="0.20"/></audio-track:Track>@@ -212,22 +219,22 @@font={caption-font}/><caption-fine:Style id="caption-wife-style" recipe={recipes.caption.wife}font={caption-font}/>- <caption:Program id="caption-program" document={story.caption}- narrative={story} default={caption-boy-style}>- <caption:Use role="WIFE" style={caption-wife-style}/>- </caption:Program>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program} regions={wife-heads}/>+ timeline={speech.timeline} regions={wife-heads}>+ <caption-fine:Use style={caption-boy-style}/>+ <caption-fine:Use role="WIFE" style={caption-wife-style}/>+ </caption-fine:Track><emoji:Style id="rule-reveal-style" recipe={recipes.emoji.strip}/>- <emoji:Track id="rule-reveal" semantic={speech.semantic} canvas={vertical}+ <emoji:Track id="rule-reveal" timeline={speech.timeline} canvas={vertical}style={rule-reveal-style} placeholder={rule-placeholder.image} during="program"><emoji:Item id="manifest" icon={manifest-icon.image} at={story.moment.manifest}/><emoji:Item id="real-estate" icon={real-estate-icon.image} at={story.moment.real-estate}/><emoji:Item id="bitcoin" icon={bitcoin-icon.image} at={story.moment.bitcoin}/></emoji:Track>- <screen:Track id="rule-reveal-flashes" semantic={speech.semantic} canvas={vertical}>+ <screen:Track id="rule-reveal-flashes" timeline={speech.timeline} canvas={vertical}><screen:Flash id="manifest-flash" at={story.moment.manifest} for="8f" z="80"color="#FF4D6D" intensity="0.28" attack="2" hold="1" decay="5"/><screen:Flash id="real-estate-flash" at={story.moment.real-estate} for="8f" z="80"@@ -236,17 +243,17 @@color="#F59E0B" intensity="0.28" attack="2" hold="1" decay="5"/></screen:Track>- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={captions.track}/><film:Track source={rule-reveal.track}/><film:Track source={rule-reveal-flashes.track}/>- <film:Track source={music-bed.track}/>- <film:Track source={rule-reveal-sfx.track}/>+ <film:Track source={music-bed.audio}/>+ <film:Track source={rule-reveal-sfx.audio}/></film:Film><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>diff --git a/examples/interview/swap-ride.svml b/examples/interview/swap-ride.svmlindex 6604de4e..9a24116c 100644--- a/examples/interview/swap-ride.svml+++ b/examples/interview/swap-ride.svml@@ -2,6 +2,8 @@<svml><import as="media-track" from="@hypit/media-track@1"/>+ <import as="sound" from="@hypit/sound@1"/>+ <import as="performance" from="@hypit/performance@1"/><import from="@hypit/script@1"/><import as="text" from="@hypit/text@1"/><import as="media" from="@hypit/media@1"/>@@ -10,7 +12,7 @@<import as="program" from="@hypit/program-space@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="audio-track" from="@hypit/audio-track@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/>@@ -184,27 +186,32 @@<whisperx:SemanticTake id="first-place-rule-semantic" narrative={story}segment={story.segment.first-place-rule} media={first-place-rule-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={uber-rule-semantic.take}/>- <speech:Take source={shortcuts-rule-semantic.take}/>- <speech:Take source={first-place-rule-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={full-frame}- appearance={recipes.media.performance}/>- </media-track:Track>-- <audio-track:Track id="music-bed" semantic={speech.semantic}>- <audio-track:Clip source={soundtrack-media.media} during="program"+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={uber-rule-semantic.take}/>+ <time:Take source={shortcuts-rule-semantic.take}/>+ <time:Take source={first-place-rule-semantic.take}/>+ </time:Timeline>++ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={full-frame} appearance={recipes.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track>++ <audio-track:Track id="music-bed" timeline={speech.timeline}>+ <audio-track:Item source={soundtrack-media.media} during="program"playback="once-start" gain="0.12" fade-in="300ms" fade-out="600ms"/></audio-track:Track>- <audio-track:Track id="rule-reveal-sfx" semantic={speech.semantic}>- <audio-track:Clip id="uber-reveal-sfx" source={rule-reveal-sound-media.media}+ <audio-track:Track id="rule-reveal-sfx" timeline={speech.timeline}>+ <audio-track:Item id="uber-reveal-sfx" source={rule-reveal-sound-media.media}at={story.moment.uber} for="20f" playback="once-start" gain="0.20"/>- <audio-track:Clip id="shortcuts-reveal-sfx" source={rule-reveal-sound-media.media}+ <audio-track:Item id="shortcuts-reveal-sfx" source={rule-reveal-sound-media.media}at={story.moment.shortcuts} for="20f" playback="once-start" gain="0.20"/>- <audio-track:Clip id="first-place-reveal-sfx" source={rule-reveal-sound-media.media}+ <audio-track:Item id="first-place-reveal-sfx" source={rule-reveal-sound-media.media}at={story.moment.first-place} for="20f" playback="once-start" gain="0.20"/></audio-track:Track>@@ -213,24 +220,24 @@font={caption-font}/><caption-fine:Style id="caption-ada-style" recipe={recipes.caption.ada}font={caption-font}/>- <caption:Program id="caption-program" document={story.caption}- narrative={story} default={caption-leon-style}>- <caption:Use role="ADA" style={caption-ada-style}/>- </caption:Program>+<space:RegionTimeline id="ada-heads" within={vertical}recipe={tracking.heads.ada}/><caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program} regions={ada-heads}/>+ timeline={speech.timeline} regions={ada-heads}>+ <caption-fine:Use style={caption-leon-style}/>+ <caption-fine:Use role="ADA" style={caption-ada-style}/>+ </caption-fine:Track><emoji:Style id="rule-reveal-style" recipe={recipes.emoji.strip}/>- <emoji:Track id="rule-reveal" semantic={speech.semantic} canvas={vertical}+ <emoji:Track id="rule-reveal" timeline={speech.timeline} canvas={vertical}style={rule-reveal-style} placeholder={rule-placeholder.image} during="program"><emoji:Item id="uber" icon={uber-icon.image} at={story.moment.uber}/><emoji:Item id="shortcuts" icon={shortcuts-icon.image} at={story.moment.shortcuts}/><emoji:Item id="first-place" icon={first-place-icon.image} at={story.moment.first-place}/></emoji:Track>- <screen:Track id="rule-reveal-flashes" semantic={speech.semantic} canvas={vertical}>+ <screen:Track id="rule-reveal-flashes" timeline={speech.timeline} canvas={vertical}><screen:Flash id="uber-flash" at={story.moment.uber} for="8f" z="80"color="#2A9DFF" intensity="0.28" attack="2" hold="1" decay="5"/><screen:Flash id="shortcuts-flash" at={story.moment.shortcuts} for="8f" z="80"@@ -239,17 +246,17 @@color="#F2B832" intensity="0.28" attack="2" hold="1" decay="5"/></screen:Track>- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={captions.track}/><film:Track source={rule-reveal.track}/><film:Track source={rule-reveal-flashes.track}/>- <film:Track source={music-bed.track}/>- <film:Track source={rule-reveal-sfx.track}/>+ <film:Track source={music-bed.audio}/>+ <film:Track source={rule-reveal-sfx.audio}/></film:Film><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>diff --git a/examples/minimal-author-package/packages/example-component/preview/preview.svml b/examples/minimal-author-package/packages/example-component/preview/preview.svmlindex 25f31f6c..34f1e212 100644--- a/examples/minimal-author-package/packages/example-component/preview/preview.svml+++ b/examples/minimal-author-package/packages/example-component/preview/preview.svml@@ -4,11 +4,13 @@Candidates are supplied, and a normal Build of the same Run can regenerate the catalogue frame. --><svml><import as="media-track" from="@hypit/media-track@1"/>+ <import as="sound" from="@hypit/sound@1"/>+ <import as="performance" from="@hypit/performance@1"/><import from="@hypit/script@1"/><import as="wording" from="@hypit/text@1"/><import as="seedance" from="@hypit/seedance@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="whisperx" from="@hypit/whisperx@1"/><import as="space" from="@hypit/spatial@1"/><import as="program" from="@hypit/program-space@1"/>@@ -29,19 +31,24 @@<seedance:TextVideo id="take" model="mini" prompt={direction} duration="4" generate-audio="true"/><pipeline:Normalize id="take-media" source={take.video} recipe={recipes.media.aroll} clock={clock}/><whisperx:SemanticTake id="showcase-semantic" narrative={story} segment={story.segment.showcase} media={take-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={showcase-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={speech-frame}- appearance={recipes.media.performance}/>- </media-track:Track>+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={showcase-semantic.take}/>+ </time:Timeline>- <example:Box id="box" semantic={speech.semantic}/>- <film:Film id="main" canvas={vertical} semantic={speech.semantic} appearance={recipes.film.vertical}>+ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={speech-frame} appearance={recipes.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track>++ <example:Box id="box" timeline={speech.timeline}/>+ <film:Film id="main" canvas={vertical} timeline={speech.timeline} appearance={recipes.film.vertical}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={box.track}/></film:Film>- <render:Video id="final" composition={main.composition} semantic={speech.semantic}/>+ <render:Video id="final" composition={main.composition} timeline={speech.timeline}/></svml>diff --git a/examples/minimal-author-package/packages/example-component/src/component.ts b/examples/minimal-author-package/packages/example-component/src/component.tsindex e117a9a1..a8261d87 100644--- a/examples/minimal-author-package/packages/example-component/src/component.ts+++ b/examples/minimal-author-package/packages/example-component/src/component.ts@@ -1,5 +1,5 @@import { sealVisualTrack } from "@hypit/hypit/composition";-import type { ProgramSpace } from "@hypit/hypit/program-space";+import type { Timeline } from "@hypit/hypit/timeline";import { canonicalize } from "@hypit/hypit/author-kit";import type { BlobRef, ComponentPackage, ProducerHandler, TypedRecord } from "@hypit/hypit/author-kit";import { VISUAL_IR_V1 } from "@hypit/hypit/visual-ir";@@ -13,10 +13,10 @@ const output = (value: unknown) => ({ kind: "inline" as const, value: canonicalifunction blob(record: TypedRecord | undefined): BlobRef | undefined {return record?.value.kind === "blob" ? record.value : undefined;}-function render(kind: string, space: ProgramSpace, media?: BlobRef) {- const frames = Math.max(1, Math.round(space.durationSec * space.frameRate.numerator / space.frameRate.denominator));+function render(kind: string, timeline: Timeline, media?: BlobRef) {+ const frames = Math.max(1, Math.round(timeline.durationSec * timeline.frameRate.numerator / timeline.frameRate.denominator));return sealVisualTrack({- programSpaceId: space.id,+ programSpaceId: timeline.id,visualIr: VISUAL_IR_V1,id: `example-${kind}`,presents: [{ id: `present-${kind}`, span: { startFrame: 0, endFrameExclusive: frames }, stacking: { order: 0, tieBreak: kind }, elements: [{@@ -26,7 +26,7 @@ function render(kind: string, space: ProgramSpace, media?: BlobRef) {] }],});}-const producer = (kind: string): { readonly handler: ProducerHandler } => ({ handler: ({ inputs }) => ({ outputs: { track: output(render(kind, inline<ProgramSpace>(inputs.space), blob(inputs.media))) }, needs: {} }) });+const producer = (kind: string): { readonly handler: ProducerHandler } => ({ handler: ({ inputs }) => ({ outputs: { track: output(render(kind, inline<Timeline>(inputs.timeline), blob(inputs.media))) }, needs: {} }) });const appendItems: { readonly handler: ProducerHandler } = { handler: ({ inputs }) => ({ outputs: { set: output({ items: [inline<{ items: readonly unknown[] }>(inputs.previous).items, inline<{ items: readonly unknown[] }>(inputs.item).items].flat() }) }, needs: {} }) };export const exampleComponent = {diff --git a/examples/minimal-author-package/packages/example-component/src/fragment.ts b/examples/minimal-author-package/packages/example-component/src/fragment.tsindex c82e584c..4047ccfe 100644--- a/examples/minimal-author-package/packages/example-component/src/fragment.ts+++ b/examples/minimal-author-package/packages/example-component/src/fragment.ts@@ -1,8 +1,7 @@import { compositionTypes } from "@hypit/hypit/composition";import { artifactTypes } from "@hypit/hypit/artifact";import { sealGraphFragment } from "@hypit/hypit/author-kit";-import { semanticTrackProducers, semanticTrackTypes } from "@hypit/hypit/semantic-track";-import { programSpaceTypes } from "@hypit/hypit/program-space";+import { timelineTypes } from "@hypit/hypit/timeline";import { exampleProducers } from "./manifest.js";import { exampleTypes } from "./manifest.js";@@ -10,12 +9,11 @@ const input = (name: string) => ({ kind: "fragment-input" as const, name });const operation = (id: string) => ({ kind: "fragment-operation" as const, operation: id });export function createExampleFragment(producer: typeof exampleProducers[keyof typeof exampleProducers], id: string, media = false) {- const inputs = [{ name: "semantic", type: semanticTrackTypes.track }, ...(media ? [{ name: "media", type: artifactTypes.blob }] : [])];- const renderInputs = { space: operation("space"), ...(media ? { media: input("media") } : {}) };+ const inputs = [{ name: "timeline", type: timelineTypes.track }, ...(media ? [{ name: "media", type: artifactTypes.blob }] : [])];+ const renderInputs = { timeline: input("timeline"), ...(media ? { media: input("media") } : {}) };return sealGraphFragment({inputs,operations: [- { id: "space", producer: semanticTrackProducers.projectProgramSpace, inputs: { track: input("semantic") }, result: { kind: "output", name: "space" } },{ id, producer, inputs: renderInputs, result: { kind: "output", name: "track" } },],exports: [{ name: "track", type: compositionTypes.visualTrack, root: operation(id) }],diff --git a/examples/minimal-author-package/packages/example-component/src/manifest.ts b/examples/minimal-author-package/packages/example-component/src/manifest.tsindex 5814d8f9..dc4520ff 100644--- a/examples/minimal-author-package/packages/example-component/src/manifest.ts+++ b/examples/minimal-author-package/packages/example-component/src/manifest.ts@@ -1,8 +1,7 @@import { readFile } from "node:fs/promises";import { compositionTypes } from "@hypit/hypit/composition";import type { ModuleManifest, ProducerRef, TypeRef } from "@hypit/hypit/author-kit";-import { programSpaceTypes } from "@hypit/hypit/program-space";-import { semanticTrackTypes } from "@hypit/hypit/semantic-track";+import { timelineTypes } from "@hypit/hypit/timeline";import { temporalTypes } from "@hypit/hypit/temporal";import { mediaTypes } from "@hypit/hypit/media";import { svsRecipeType } from "@hypit/hypit/svs";@@ -35,8 +34,7 @@ export const exampleManifest: ModuleManifest = {{ module: artifactTypes.blob.module },{ module: compositionTypes.visualTrack.module },{ module: mediaTypes.fontStack.module },- { module: programSpaceTypes.programSpace.module },- { module: semanticTrackTypes.track.module },+ { module: timelineTypes.track.module },{ module: svsRecipeType.module },{ module: temporalTypes.window.module },],@@ -46,7 +44,7 @@ export const exampleManifest: ModuleManifest = {name: producer.name,inputs: producer === exampleProducers.appendItems? [{ name: "previous", type: exampleTypes.itemSet }, { name: "item", type: exampleTypes.itemSet }]- : [{ name: "space", type: programSpaceTypes.programSpace }, ...(producer === exampleProducers.renderMedia ? [{ name: "media", type: artifactTypes.blob }] : [])],+ : [{ name: "timeline", type: timelineTypes.track }, ...(producer === exampleProducers.renderMedia ? [{ name: "media", type: artifactTypes.blob }] : [])],outputs: producer === exampleProducers.appendItems? [{ name: "set", type: exampleTypes.itemSet }]: [{ name: "track", type: compositionTypes.visualTrack }],@@ -56,11 +54,11 @@ export const exampleManifest: ModuleManifest = {const vocabulary = (summary: string, example: string) => ({summary,- appearance: "A deterministic, self-contained example surface rendered on the supplied ProgramSpace.",+ appearance: "A deterministic, self-contained example surface rendered on the supplied Timeline.",preview: previewImage("Box.png"),attributes: [{ name: "id", kind: "identifier" as const, required: true, summary: "Names this instance." },- { name: "semantic", kind: "reference" as const, required: true, accepts: [semanticTrackTypes.track], summary: "Selects the semantic timing track." },+ { name: "timeline", kind: "reference" as const, required: true, accepts: [timelineTypes.track], summary: "Selects the complete Timeline." },],ports: [{ name: "track", type: compositionTypes.visualTrack, summary: "The terminal VisualTrack." }],example,@@ -68,9 +66,9 @@ const vocabulary = (summary: string, example: string) => ({});export const exampleMarkupSurfaces = [- { name: "box", tag: "Box", mode: "structured", outputs: [exampleTypes.box, compositionTypes.visualTrack], vocabulary: { ...vocabulary("A framed box surface.", "<example:Box id=\"box\" semantic={speech.semantic}/>") , preview: previewImage("Box.png") } },- { name: "text", tag: "Text", mode: "structured", outputs: [exampleTypes.text, temporalTypes.instantSpec, temporalTypes.windowSpec, temporalTypes.instant, temporalTypes.window, compositionTypes.visualTrack], vocabulary: { ...vocabulary("A text-bearing surface.", "<example:Text id=\"title\" semantic={speech.semantic}>Hello</example:Text>"), preview: previewImage("Box.png") } },- { name: "media-slot", tag: "MediaSlot", mode: "structured", outputs: [exampleTypes.mediaSlot, compositionTypes.visualTrack], vocabulary: { ...vocabulary("A media slot whose content is a graph input.", "<example:MediaSlot id=\"shot\" semantic={speech.semantic} media={shot-media}/>") , attributes: [...vocabulary("", "").attributes, { name: "media", kind: "reference" as const, required: false, accepts: [artifactTypes.blob], summary: "Optional image or video Blob Artifact." }], preview: previewImage("Box.png") } },+ { name: "box", tag: "Box", mode: "structured", outputs: [exampleTypes.box, compositionTypes.visualTrack], vocabulary: { ...vocabulary("A framed box surface.", "<example:Box id=\"box\" timeline={speech.timeline}/>") , preview: previewImage("Box.png") } },+ { name: "text", tag: "Text", mode: "structured", outputs: [exampleTypes.text, temporalTypes.instantSpec, temporalTypes.windowSpec, temporalTypes.instant, temporalTypes.window, compositionTypes.visualTrack], vocabulary: { ...vocabulary("A text-bearing surface.", "<example:Text id=\"title\" timeline={speech.timeline}>Hello</example:Text>"), preview: previewImage("Box.png") } },+ { name: "media-slot", tag: "MediaSlot", mode: "structured", outputs: [exampleTypes.mediaSlot, compositionTypes.visualTrack], vocabulary: { ...vocabulary("A media slot whose content is a graph input.", "<example:MediaSlot id=\"shot\" timeline={speech.timeline} media={shot-media}/>") , attributes: [...vocabulary("", "").attributes, { name: "media", kind: "reference" as const, required: false, accepts: [artifactTypes.blob], summary: "Optional image or video Blob Artifact." }], preview: previewImage("Box.png") } },{ name: "style", tag: "Style", mode: "structured", outputs: [exampleTypes.style], vocabulary: {summary: "Decodes one SVS Recipe and exact FontStackRef into a Style value.",attributes: [diff --git a/examples/minimal-author-package/packages/example-component/src/surface.ts b/examples/minimal-author-package/packages/example-component/src/surface.tsindex c73fc119..150eec26 100644--- a/examples/minimal-author-package/packages/example-component/src/surface.ts+++ b/examples/minimal-author-package/packages/example-component/src/surface.ts@@ -28,7 +28,7 @@ export const decodeExampleSurface: StructuredSurfaceHandler = ({ element, resolvif (recipe.record?.value.kind !== "inline" || font.record?.value.kind !== "inline") throw new Error("Style references must resolve to inline records.");return { records: [{ id, type: exampleTypes.style, value: { kind: "inline", value: { recipe: recipe.record.value.value, fonts: font.record.value.value } }, range: element.range }], components: [], fragments: [], exports: [id] };}- const semantic = reference(element, "semantic", resolveReference);+ const timeline = reference(element, "timeline", resolveReference);const surface = exampleMarkupSurfaces.find((item) => item.tag === element.name.split(":").at(-1));if (surface === undefined) throw new Error(`Unknown example surface ${element.name}`);const fragment = surface.name === "box" ? exampleBoxFragment : surface.name === "text" ? exampleTextFragment : exampleMediaFragment;@@ -36,6 +36,6 @@ export const decodeExampleSurface: StructuredSurfaceHandler = ({ element, resolvconst record: SurfaceRecordDraft = { id: `${id}.value`, type, value: { kind: "inline", value: { id } }, range: element.range };const media = element.attributes.media === undefined ? undefined : reference(element, "media", resolveReference);if (media !== undefined && (media.type.module.name !== artifactTypes.blob.module.name || media.type.name !== artifactTypes.blob.name)) throw new Error(`${element.name}.media must be a Blob Artifact.`);- const component: SurfaceComponentDraft = { id, fragment: fragment.id, inputs: { semantic: semantic.ref, ...(media === undefined ? {} : { media: media.ref }) }, outputs: { track: `${id}.track` }, range: element.range };+ const component: SurfaceComponentDraft = { id, fragment: fragment.id, inputs: { timeline: timeline.ref, ...(media === undefined ? {} : { media: media.ref }) }, outputs: { track: `${id}.track` }, range: element.range };return { records: [record], components: [component], fragments: [fragment], exports: [`${id}.value`, `${id}.track`] };};diff --git a/examples/podcast/reference.svml b/examples/podcast/reference.svmlindex 5bf26ed7..f43f60e6 100644--- a/examples/podcast/reference.svml+++ b/examples/podcast/reference.svml@@ -2,6 +2,7 @@<svml><import from="@hypit/script@1"/>+ <import as="sound" from="@hypit/sound@1"/><import as="asset" from="@hypit/media@1"/><import as="mimo" from="@hypit/mimo-speech@1"/><import as="text" from="@hypit/text@1"/>@@ -10,12 +11,13 @@<import as="program" from="@hypit/program-space@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="audio-track" from="@hypit/audio-track@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/><import as="fonts" from="@hypit/fonts-open@1"/><import as="media-track" from="@hypit/media-track@1"/>+ <import as="performance" from="@hypit/performance@1"/><import as="space" from="@hypit/spatial@1"/><import as="film" from="@hypit/film@1"/><import as="render" from="@hypit/render-hyperframes@1"/>@@ -373,22 +375,27 @@segment={story.segment.arms-are-asking} media={arms-are-asking-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={opening-semantic.take}/>- <speech:Take source={daily-creatine-semantic.take}/>- <speech:Take source={arms-are-asking-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={full-frame}- appearance={recipes.media.performance}/>- </media-track:Track>+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={opening-semantic.take}/>+ <time:Take source={daily-creatine-semantic.take}/>+ <time:Take source={arms-are-asking-semantic.take}/>+ </time:Timeline>++ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={full-frame} appearance={recipes.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track>- <audio-track:Track id="music-bed" semantic={speech.semantic}>- <audio-track:Clip source={soundtrack-media.media} during="program"+ <audio-track:Track id="music-bed" timeline={speech.timeline}>+ <audio-track:Item source={soundtrack-media.media} during="program"playback="once-start" gain="0.12" fade-in="300ms" fade-out="600ms"/></audio-track:Track>- <media-track:Track id="lifestyle" semantic={speech.semantic} canvas={vertical}>+ <media-track:Track id="lifestyle" timeline={speech.timeline} canvas={vertical}><media-track:Item id="lifestyle-cutaway" media={lifestyle-media.media}frame={full-frame} during={story.selection.lifestyle}appearance={recipes.media.lifestyle}/>@@ -399,22 +406,22 @@font={caption-font}/><caption-fine:Style id="caption-guy-style" recipe={recipes.caption.original-guy}font={caption-font}/>- <caption:Program id="caption-program" document={story.caption}- narrative={story} default={caption-style}>- <caption:Use role="GUY" style={caption-guy-style}/>- </caption:Program>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={caption-style}/>+ <caption-fine:Use role="GUY" style={caption-guy-style}/>+ </caption-fine:Track>- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={lifestyle.visual}/><film:Track source={captions.track}/>- <film:Track source={music-bed.track}/>+ <film:Track source={music-bed.audio}/></film:Film><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>diff --git a/examples/podcast/swap-app.svml b/examples/podcast/swap-app.svmlindex b2910948..6703ff1e 100644--- a/examples/podcast/swap-app.svml+++ b/examples/podcast/swap-app.svml@@ -2,6 +2,7 @@<svml><import from="@hypit/script@1"/>+ <import as="sound" from="@hypit/sound@1"/><import as="asset" from="@hypit/media@1"/><import as="text" from="@hypit/text@1"/><import as="gpt" from="@hypit/gpt-image@1"/>@@ -9,12 +10,13 @@<import as="program" from="@hypit/program-space@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="audio-track" from="@hypit/audio-track@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/><import as="fonts" from="@hypit/fonts-open@1"/><import as="media-track" from="@hypit/media-track@1"/>+ <import as="performance" from="@hypit/performance@1"/><import as="space" from="@hypit/spatial@1"/><import as="film" from="@hypit/film@1"/><import as="render" from="@hypit/render-hyperframes@1"/>@@ -187,22 +189,27 @@<whisperx:SemanticTake id="brain-is-asking-semantic" narrative={story}segment={story.segment.brain-is-asking} media={brain-is-asking-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={opening-semantic.take}/>- <speech:Take source={daily-cheatgpt-semantic.take}/>- <speech:Take source={brain-is-asking-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={full-frame}- appearance={recipes.media.performance}/>- </media-track:Track>-- <audio-track:Track id="music-bed" semantic={speech.semantic}>- <audio-track:Clip source={soundtrack-media.media} during="program"+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={opening-semantic.take}/>+ <time:Take source={daily-cheatgpt-semantic.take}/>+ <time:Take source={brain-is-asking-semantic.take}/>+ </time:Timeline>++ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={full-frame} appearance={recipes.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track>++ <audio-track:Track id="music-bed" timeline={speech.timeline}>+ <audio-track:Item source={soundtrack-media.media} during="program"playback="once-start" gain="0.12" fade-in="300ms" fade-out="600ms"/></audio-track:Track>- <media-track:Track id="lifestyle" semantic={speech.semantic} canvas={vertical}>+ <media-track:Track id="lifestyle" timeline={speech.timeline} canvas={vertical}><media-track:Item id="lifestyle-cutaway" media={lifestyle-media.media}frame={full-frame} during={story.selection.lifestyle}appearance={recipes.media.lifestyle}/>@@ -213,22 +220,22 @@font={caption-font}/><caption-fine:Style id="caption-nerd-style" recipe={recipes.caption.nerd}font={caption-font}/>- <caption:Program id="caption-program" document={story.caption}- narrative={story} default={caption-jock-style}>- <caption:Use role="NERD" style={caption-nerd-style}/>- </caption:Program>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={caption-jock-style}/>+ <caption-fine:Use role="NERD" style={caption-nerd-style}/>+ </caption-fine:Track>- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={lifestyle.visual}/><film:Track source={captions.track}/>- <film:Track source={music-bed.track}/>+ <film:Track source={music-bed.audio}/></film:Film><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>diff --git a/examples/podcast/swap-host.svml b/examples/podcast/swap-host.svmlindex 7aa9809c..4a40017f 100644--- a/examples/podcast/swap-host.svml+++ b/examples/podcast/swap-host.svml@@ -2,6 +2,7 @@<svml><import from="@hypit/script@1"/>+ <import as="sound" from="@hypit/sound@1"/><import as="asset" from="@hypit/media@1"/><import as="text" from="@hypit/text@1"/><import as="gpt" from="@hypit/gpt-image@1"/>@@ -9,12 +10,13 @@<import as="program" from="@hypit/program-space@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="audio-track" from="@hypit/audio-track@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/><import as="fonts" from="@hypit/fonts-open@1"/><import as="media-track" from="@hypit/media-track@1"/>+ <import as="performance" from="@hypit/performance@1"/><import as="space" from="@hypit/spatial@1"/><import as="film" from="@hypit/film@1"/><import as="render" from="@hypit/render-hyperframes@1"/>@@ -195,22 +197,27 @@<whisperx:SemanticTake id="arms-are-asking-semantic" narrative={story}segment={story.segment.arms-are-asking} media={arms-are-asking-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={opening-semantic.take}/>- <speech:Take source={daily-creatine-semantic.take}/>- <speech:Take source={arms-are-asking-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={full-frame}- appearance={recipes.media.performance}/>- </media-track:Track>-- <audio-track:Track id="music-bed" semantic={speech.semantic}>- <audio-track:Clip source={soundtrack-media.media} during="program"+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={opening-semantic.take}/>+ <time:Take source={daily-creatine-semantic.take}/>+ <time:Take source={arms-are-asking-semantic.take}/>+ </time:Timeline>++ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={full-frame} appearance={recipes.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track>++ <audio-track:Track id="music-bed" timeline={speech.timeline}>+ <audio-track:Item source={soundtrack-media.media} during="program"playback="once-start" gain="0.12" fade-in="300ms" fade-out="600ms"/></audio-track:Track>- <media-track:Track id="lifestyle" semantic={speech.semantic} canvas={vertical}>+ <media-track:Track id="lifestyle" timeline={speech.timeline} canvas={vertical}><media-track:Item id="lifestyle-cutaway" media={lifestyle-media.media}frame={full-frame} during={story.selection.lifestyle}appearance={recipes.media.lifestyle}/>@@ -221,22 +228,22 @@font={caption-font}/><caption-fine:Style id="caption-doge-style" recipe={recipes.caption.doge}font={caption-font}/>- <caption:Program id="caption-program" document={story.caption}- narrative={story} default={caption-style}>- <caption:Use role="DOGE" style={caption-doge-style}/>- </caption:Program>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={caption-style}/>+ <caption-fine:Use role="DOGE" style={caption-doge-style}/>+ </caption-fine:Track>- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={lifestyle.visual}/><film:Track source={captions.track}/>- <film:Track source={music-bed.track}/>+ <film:Track source={music-bed.audio}/></film:Film><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>diff --git a/examples/podcast/swap-item.svml b/examples/podcast/swap-item.svmlindex 245a3dbf..8b907cb1 100644--- a/examples/podcast/swap-item.svml+++ b/examples/podcast/swap-item.svml@@ -2,6 +2,7 @@<svml><import from="@hypit/script@1"/>+ <import as="sound" from="@hypit/sound@1"/><import as="asset" from="@hypit/media@1"/><import as="text" from="@hypit/text@1"/><import as="gpt" from="@hypit/gpt-image@1"/>@@ -9,12 +10,13 @@<import as="program" from="@hypit/program-space@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="audio-track" from="@hypit/audio-track@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/><import as="fonts" from="@hypit/fonts-open@1"/><import as="media-track" from="@hypit/media-track@1"/>+ <import as="performance" from="@hypit/performance@1"/><import as="space" from="@hypit/spatial@1"/><import as="film" from="@hypit/film@1"/><import as="render" from="@hypit/render-hyperframes@1"/>@@ -195,22 +197,27 @@<whisperx:SemanticTake id="skin-is-asking-semantic" narrative={story}segment={story.segment.skin-is-asking} media={skin-is-asking-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={opening-semantic.take}/>- <speech:Take source={daily-retinol-semantic.take}/>- <speech:Take source={skin-is-asking-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={full-frame}- appearance={recipes.media.performance}/>- </media-track:Track>-- <audio-track:Track id="music-bed" semantic={speech.semantic}>- <audio-track:Clip source={soundtrack-media.media} during="program"+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={opening-semantic.take}/>+ <time:Take source={daily-retinol-semantic.take}/>+ <time:Take source={skin-is-asking-semantic.take}/>+ </time:Timeline>++ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={full-frame} appearance={recipes.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track>++ <audio-track:Track id="music-bed" timeline={speech.timeline}>+ <audio-track:Item source={soundtrack-media.media} during="program"playback="once-start" gain="0.12" fade-in="300ms" fade-out="600ms"/></audio-track:Track>- <media-track:Track id="lifestyle" semantic={speech.semantic} canvas={vertical}>+ <media-track:Track id="lifestyle" timeline={speech.timeline} canvas={vertical}><media-track:Item id="lifestyle-cutaway" media={lifestyle-media.media}frame={full-frame} during={story.selection.lifestyle}appearance={recipes.media.lifestyle}/>@@ -221,22 +228,22 @@font={caption-font}/><caption-fine:Style id="caption-tboy-style" recipe={recipes.caption.retinol-tboy}font={caption-font}/>- <caption:Program id="caption-program" document={story.caption}- narrative={story} default={caption-idol-style}>- <caption:Use role="TBOY" style={caption-tboy-style}/>- </caption:Program>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={caption-idol-style}/>+ <caption-fine:Use role="TBOY" style={caption-tboy-style}/>+ </caption-fine:Track>- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={lifestyle.visual}/><film:Track source={captions.track}/>- <film:Track source={music-bed.track}/>+ <film:Track source={music-bed.audio}/></film:Film><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>diff --git a/examples/ranking-football/reference-banana-from-swap/reference-banana-from-swap.svml b/examples/ranking-football/reference-banana-from-swap/reference-banana-from-swap.svmlindex cf2eb685..bd4f6bbc 100644--- a/examples/ranking-football/reference-banana-from-swap/reference-banana-from-swap.svml+++ b/examples/ranking-football/reference-banana-from-swap/reference-banana-from-swap.svml@@ -2,14 +2,16 @@<svml><import from="@hypit/script@1"/>+ <import as="sound" from="@hypit/sound@1"/><import as="text" from="@hypit/text@1"/><import as="asset" from="@hypit/media@1"/><import as="gpt" from="@hypit/gpt-image@1"/><import as="seedance" from="@hypit/seedance@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="media-track" from="@hypit/media-track@1"/>+ <import as="performance" from="@hypit/performance@1"/><import as="audio-track" from="@hypit/audio-track@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/>@@ -194,16 +196,21 @@<whisperx:SemanticTake id="messi-semantic" narrative={story}segment={story.segment.messi} media={messi-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={ronaldo-semantic.take}/>- <speech:Take source={messi-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={speech-frame}- appearance={recipes.media.performance}/>- </media-track:Track>- <audio-track:Track id="music-bed" semantic={speech.semantic}>- <audio-track:Clip source={soundtrack-media.media} during="program"+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={ronaldo-semantic.take}/>+ <time:Take source={messi-semantic.take}/>+ </time:Timeline>++ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={speech-frame} appearance={recipes.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track>+ <audio-track:Track id="music-bed" timeline={speech.timeline}>+ <audio-track:Item source={soundtrack-media.media} during="program"playback="once-start" gain="0.12" fade-in="300ms" fade-out="600ms"/></audio-track:Track>@@ -211,7 +218,7 @@<fonts:Stack id="tier-font" family="inter" weight="400" style="normal"/><ranking:TierBoardStyle id="tier-style" recipe={recipes.ranking.football}font={tier-font}/>- <ranking:TierBoard id="football-tiers" semantic={speech.semantic}+ <ranking:TierBoard id="football-tiers" timeline={speech.timeline}canvas={vertical} frame={tier-frame}during="program" style={tier-style}appear-sound={ranking-appear-media.media}@@ -233,7 +240,7 @@<!-- Every Item consumes a projected Selection window. Later overlappingjokes use a higher stack order through their appearance Recipe. -->- <media-track:Track id="broll" semantic={speech.semantic} canvas={vertical}>+ <media-track:Track id="broll" timeline={speech.timeline} canvas={vertical}><media-track:Item id="hair-gel" image={broll-hair-gel.image}extent={hair-gel-extent} during={story.selection.hair-gel}frame={hair-gel-frame} appearance={recipes.media.hair-gel}@@ -270,21 +277,22 @@<caption-fine:Style id="caption-style" recipe={recipes.caption.primary}font={display-font}/>- <caption:Program id="caption-program" document={story.caption}- narrative={story} default={caption-style}/>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={caption-style}/>+ </caption-fine:Track>- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={football-tiers.visual}/><film:Track source={football-tiers.audio}/><film:Track source={broll.visual}/><film:Track source={captions.track}/>- <film:Track source={music-bed.track}/>+ <film:Track source={music-bed.audio}/></film:Film><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>diff --git a/examples/ranking-football/reference-banana/reference-banana.svml b/examples/ranking-football/reference-banana/reference-banana.svmlindex cf2eb685..bd4f6bbc 100644--- a/examples/ranking-football/reference-banana/reference-banana.svml+++ b/examples/ranking-football/reference-banana/reference-banana.svml@@ -2,14 +2,16 @@<svml><import from="@hypit/script@1"/>+ <import as="sound" from="@hypit/sound@1"/><import as="text" from="@hypit/text@1"/><import as="asset" from="@hypit/media@1"/><import as="gpt" from="@hypit/gpt-image@1"/><import as="seedance" from="@hypit/seedance@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="media-track" from="@hypit/media-track@1"/>+ <import as="performance" from="@hypit/performance@1"/><import as="audio-track" from="@hypit/audio-track@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/>@@ -194,16 +196,21 @@<whisperx:SemanticTake id="messi-semantic" narrative={story}segment={story.segment.messi} media={messi-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={ronaldo-semantic.take}/>- <speech:Take source={messi-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={speech-frame}- appearance={recipes.media.performance}/>- </media-track:Track>- <audio-track:Track id="music-bed" semantic={speech.semantic}>- <audio-track:Clip source={soundtrack-media.media} during="program"+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={ronaldo-semantic.take}/>+ <time:Take source={messi-semantic.take}/>+ </time:Timeline>++ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={speech-frame} appearance={recipes.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track>+ <audio-track:Track id="music-bed" timeline={speech.timeline}>+ <audio-track:Item source={soundtrack-media.media} during="program"playback="once-start" gain="0.12" fade-in="300ms" fade-out="600ms"/></audio-track:Track>@@ -211,7 +218,7 @@<fonts:Stack id="tier-font" family="inter" weight="400" style="normal"/><ranking:TierBoardStyle id="tier-style" recipe={recipes.ranking.football}font={tier-font}/>- <ranking:TierBoard id="football-tiers" semantic={speech.semantic}+ <ranking:TierBoard id="football-tiers" timeline={speech.timeline}canvas={vertical} frame={tier-frame}during="program" style={tier-style}appear-sound={ranking-appear-media.media}@@ -233,7 +240,7 @@<!-- Every Item consumes a projected Selection window. Later overlappingjokes use a higher stack order through their appearance Recipe. -->- <media-track:Track id="broll" semantic={speech.semantic} canvas={vertical}>+ <media-track:Track id="broll" timeline={speech.timeline} canvas={vertical}><media-track:Item id="hair-gel" image={broll-hair-gel.image}extent={hair-gel-extent} during={story.selection.hair-gel}frame={hair-gel-frame} appearance={recipes.media.hair-gel}@@ -270,21 +277,22 @@<caption-fine:Style id="caption-style" recipe={recipes.caption.primary}font={display-font}/>- <caption:Program id="caption-program" document={story.caption}- narrative={story} default={caption-style}/>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={caption-style}/>+ </caption-fine:Track>- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={football-tiers.visual}/><film:Track source={football-tiers.audio}/><film:Track source={broll.visual}/><film:Track source={captions.track}/>- <film:Track source={music-bed.track}/>+ <film:Track source={music-bed.audio}/></film:Film><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>diff --git a/examples/ranking-football/reference-gpt.svml b/examples/ranking-football/reference-gpt.svmlindex efaa5664..e1da5295 100644--- a/examples/ranking-football/reference-gpt.svml+++ b/examples/ranking-football/reference-gpt.svml@@ -2,14 +2,16 @@<svml><import from="@hypit/script@1"/>+ <import as="sound" from="@hypit/sound@1"/><import as="text" from="@hypit/text@1"/><import as="asset" from="@hypit/media@1"/><import as="gpt" from="@hypit/gpt-image@1"/><import as="seedance" from="@hypit/seedance@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="media-track" from="@hypit/media-track@1"/>+ <import as="performance" from="@hypit/performance@1"/><import as="audio-track" from="@hypit/audio-track@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/>@@ -194,16 +196,21 @@<whisperx:SemanticTake id="messi-semantic" narrative={story}segment={story.segment.messi} media={messi-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={ronaldo-semantic.take}/>- <speech:Take source={messi-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={speech-frame}- appearance={recipes.media.performance}/>- </media-track:Track>- <audio-track:Track id="music-bed" semantic={speech.semantic}>- <audio-track:Clip source={soundtrack-media.media} during="program"+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={ronaldo-semantic.take}/>+ <time:Take source={messi-semantic.take}/>+ </time:Timeline>++ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={speech-frame} appearance={recipes.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track>+ <audio-track:Track id="music-bed" timeline={speech.timeline}>+ <audio-track:Item source={soundtrack-media.media} during="program"playback="once-start" gain="0.12" fade-in="300ms" fade-out="600ms"/></audio-track:Track>@@ -211,7 +218,7 @@<fonts:Stack id="tier-font" family="inter" weight="400" style="normal"/><ranking:TierBoardStyle id="tier-style" recipe={recipes.ranking.football}font={tier-font}/>- <ranking:TierBoard id="football-tiers" semantic={speech.semantic}+ <ranking:TierBoard id="football-tiers" timeline={speech.timeline}canvas={vertical} frame={tier-frame}during="program" style={tier-style}appear-sound={ranking-appear-media.media}@@ -233,7 +240,7 @@<!-- Every Item consumes a projected Selection window. Later overlappingjokes use a higher stack order through their appearance Recipe. -->- <media-track:Track id="broll" semantic={speech.semantic} canvas={vertical}>+ <media-track:Track id="broll" timeline={speech.timeline} canvas={vertical}><media-track:Item id="hair-gel" image={broll-hair-gel.image}extent={hair-gel-extent} during={story.selection.hair-gel}frame={hair-gel-frame} appearance={recipes.media.hair-gel}@@ -270,21 +277,22 @@<caption-fine:Style id="caption-style" recipe={recipes.caption.primary}font={display-font}/>- <caption:Program id="caption-program" document={story.caption}- narrative={story} default={caption-style}/>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={caption-style}/>+ </caption-fine:Track>- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={football-tiers.visual}/><film:Track source={football-tiers.audio}/><film:Track source={broll.visual}/><film:Track source={captions.track}/>- <film:Track source={music-bed.track}/>+ <film:Track source={music-bed.audio}/></film:Film><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>diff --git a/examples/ranking-football/reference.svml b/examples/ranking-football/reference.svmlindex 7beb87d0..660fec11 100644--- a/examples/ranking-football/reference.svml+++ b/examples/ranking-football/reference.svml@@ -2,6 +2,7 @@<svml><import from="@hypit/script@1"/>+ <import as="sound" from="@hypit/sound@1"/><import as="text" from="@hypit/text@1"/><import as="asset" from="@hypit/media@1"/><import as="mimo" from="@hypit/mimo-speech@1"/>@@ -9,8 +10,9 @@<import as="seedance" from="@hypit/seedance@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="media-track" from="@hypit/media-track@1"/>+ <import as="performance" from="@hypit/performance@1"/><import as="audio-track" from="@hypit/audio-track@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/>@@ -225,16 +227,21 @@<whisperx:SemanticTake id="messi-semantic" narrative={story}segment={story.segment.messi} media={messi-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={ronaldo-semantic.take}/>- <speech:Take source={messi-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={speech-frame}- appearance={recipes.media.performance}/>- </media-track:Track>- <audio-track:Track id="music-bed" semantic={speech.semantic}>- <audio-track:Clip source={soundtrack-media.media} during="program"+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={ronaldo-semantic.take}/>+ <time:Take source={messi-semantic.take}/>+ </time:Timeline>++ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={speech-frame} appearance={recipes.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track>+ <audio-track:Track id="music-bed" timeline={speech.timeline}>+ <audio-track:Item source={soundtrack-media.media} during="program"playback="once-start" gain="0.12" fade-in="300ms" fade-out="600ms"/></audio-track:Track>@@ -242,7 +249,7 @@<fonts:Stack id="tier-font" family="inter" weight="400" style="normal"/><ranking:TierBoardStyle id="tier-style" recipe={recipes.ranking.football}font={tier-font}/>- <ranking:TierBoard id="football-tiers" semantic={speech.semantic}+ <ranking:TierBoard id="football-tiers" timeline={speech.timeline}canvas={vertical} frame={tier-frame}during="program" style={tier-style}appear-sound={ranking-appear-media.media}@@ -264,7 +271,7 @@<!-- Every Item consumes a projected Selection window. Later overlappingjokes use a higher stack order through their appearance Recipe. -->- <media-track:Track id="broll" semantic={speech.semantic} canvas={vertical}>+ <media-track:Track id="broll" timeline={speech.timeline} canvas={vertical}><media-track:Item id="hair-gel" image={broll-hair-gel.image}extent={hair-gel-extent} during={story.selection.hair-gel}frame={hair-gel-frame} appearance={recipes.media.hair-gel}@@ -301,21 +308,22 @@<caption-fine:Style id="caption-style" recipe={recipes.caption.primary}font={display-font}/>- <caption:Program id="caption-program" document={story.caption}- narrative={story} default={caption-style}/>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={caption-style}/>+ </caption-fine:Track>- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={football-tiers.visual}/><film:Track source={football-tiers.audio}/><film:Track source={broll.visual}/><film:Track source={captions.track}/>- <film:Track source={music-bed.track}/>+ <film:Track source={music-bed.audio}/></film:Film><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>diff --git a/examples/ranking-football/swap-effect-banana-reference-sync/swap-effect-banana-reference-sync.svml b/examples/ranking-football/swap-effect-banana-reference-sync/swap-effect-banana-reference-sync.svmlindex 6a835470..d73d4c8a 100644--- a/examples/ranking-football/swap-effect-banana-reference-sync/swap-effect-banana-reference-sync.svml+++ b/examples/ranking-football/swap-effect-banana-reference-sync/swap-effect-banana-reference-sync.svml@@ -2,14 +2,16 @@<svml><import from="@hypit/script@1"/>+ <import as="sound" from="@hypit/sound@1"/><import as="text" from="@hypit/text@1"/><import as="asset" from="@hypit/media@1"/><import as="gpt" from="@hypit/gpt-image@1"/><import as="seedance" from="@hypit/seedance@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="media-track" from="@hypit/media-track@1"/>+ <import as="performance" from="@hypit/performance@1"/><import as="audio-track" from="@hypit/audio-track@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/>@@ -194,16 +196,21 @@<whisperx:SemanticTake id="messi-semantic" narrative={story}segment={story.segment.messi} media={messi-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={ronaldo-semantic.take}/>- <speech:Take source={messi-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={speech-frame}- appearance={recipes.media.performance}/>- </media-track:Track>- <audio-track:Track id="music-bed" semantic={speech.semantic}>- <audio-track:Clip source={soundtrack-media.media} during="program"+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={ronaldo-semantic.take}/>+ <time:Take source={messi-semantic.take}/>+ </time:Timeline>++ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={speech-frame} appearance={recipes.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track>+ <audio-track:Track id="music-bed" timeline={speech.timeline}>+ <audio-track:Item source={soundtrack-media.media} during="program"playback="once-start" gain="0.12" fade-in="300ms" fade-out="600ms"/></audio-track:Track>@@ -211,7 +218,7 @@<fonts:Stack id="column-font" family="inter" weight="700" style="normal"/><ranking:ColumnStyle id="column-style" recipe={recipes.ranking.football-column}font={column-font}/>- <ranking:Column id="football-column" semantic={speech.semantic}+ <ranking:Column id="football-column" timeline={speech.timeline}canvas={vertical} frame={column-frame} during="program" style={column-style}appear-sound={ranking-appear-media.media}move-sound={ranking-move-media.media}>@@ -232,7 +239,7 @@<!-- Every Item consumes a projected Selection window. Later overlappingjokes use a higher stack order through their appearance Recipe. -->- <media-track:Track id="broll" semantic={speech.semantic} canvas={vertical}>+ <media-track:Track id="broll" timeline={speech.timeline} canvas={vertical}><media-track:Item id="hair-gel" image={broll-hair-gel.image}extent={hair-gel-extent} during={story.selection.hair-gel}frame={hair-gel-frame} appearance={recipes.media.hair-gel}@@ -269,21 +276,22 @@<caption-fine:Style id="caption-style" recipe={recipes.caption.primary}font={display-font}/>- <caption:Program id="caption-program" document={story.caption}- narrative={story} default={caption-style}/>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={caption-style}/>+ </caption-fine:Track>- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={football-column.visual}/><film:Track source={football-column.audio}/><film:Track source={broll.visual}/><film:Track source={captions.track}/>- <film:Track source={music-bed.track}/>+ <film:Track source={music-bed.audio}/></film:Film><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>diff --git a/examples/ranking-football/swap-effect-banana/swap-effect-banana.svml b/examples/ranking-football/swap-effect-banana/swap-effect-banana.svmlindex 8d6ecb11..929979eb 100644--- a/examples/ranking-football/swap-effect-banana/swap-effect-banana.svml+++ b/examples/ranking-football/swap-effect-banana/swap-effect-banana.svml@@ -2,14 +2,16 @@<svml><import from="@hypit/script@1"/>+ <import as="sound" from="@hypit/sound@1"/><import as="text" from="@hypit/text@1"/><import as="asset" from="@hypit/media@1"/><import as="gpt" from="@hypit/gpt-image@1"/><import as="seedance" from="@hypit/seedance@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="media-track" from="@hypit/media-track@1"/>+ <import as="performance" from="@hypit/performance@1"/><import as="audio-track" from="@hypit/audio-track@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/>@@ -176,16 +178,21 @@<whisperx:SemanticTake id="ronaldo-semantic" narrative={story}segment={story.segment.ronaldo} media={ronaldo-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={messi-semantic.take}/>- <speech:Take source={ronaldo-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={speech-frame}- appearance={recipes.media.performance}/>- </media-track:Track>- <audio-track:Track id="music-bed" semantic={speech.semantic}>- <audio-track:Clip source={soundtrack-media.media} during="program"+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={messi-semantic.take}/>+ <time:Take source={ronaldo-semantic.take}/>+ </time:Timeline>++ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={speech-frame} appearance={recipes.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track>+ <audio-track:Track id="music-bed" timeline={speech.timeline}>+ <audio-track:Item source={soundtrack-media.media} during="program"playback="once-start" gain="0.12" fade-in="300ms" fade-out="600ms"/></audio-track:Track>@@ -193,7 +200,7 @@<fonts:Stack id="column-font" family="inter" weight="700" style="normal"/><ranking:ColumnStyle id="column-style" recipe={recipes.ranking.football-column}font={column-font}/>- <ranking:Column id="football-column" semantic={speech.semantic}+ <ranking:Column id="football-column" timeline={speech.timeline}canvas={vertical} frame={column-frame} during="program" style={column-style}appear-sound={ranking-appear-media.media}move-sound={ranking-move-media.media}>@@ -219,7 +226,7 @@icon={messi-icon.image} during={story.segment.messi}/></ranking:Column>- <media-track:Track id="broll" semantic={speech.semantic} canvas={vertical}>+ <media-track:Track id="broll" timeline={speech.timeline} canvas={vertical}><media-track:Item id="pitch-pov" image={broll-pitch-pov.image}extent={pitch-pov-extent} during={story.selection.pitch-pov}frame={pitch-pov-frame} appearance={recipes.media.pitch-pov}@@ -252,21 +259,22 @@<caption-fine:Style id="caption-style" recipe={recipes.caption.primary}font={display-font}/>- <caption:Program id="caption-program" document={story.caption}- narrative={story} default={caption-style}/>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={caption-style}/>+ </caption-fine:Track>- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={football-column.visual}/><film:Track source={football-column.audio}/><film:Track source={broll.visual}/><film:Track source={captions.track}/>- <film:Track source={music-bed.track}/>+ <film:Track source={music-bed.audio}/></film:Film><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>diff --git a/examples/ranking-football/swap-effect.svml b/examples/ranking-football/swap-effect.svmlindex 1a38d239..e092fd63 100644--- a/examples/ranking-football/swap-effect.svml+++ b/examples/ranking-football/swap-effect.svml@@ -2,14 +2,16 @@<svml><import from="@hypit/script@1"/>+ <import as="sound" from="@hypit/sound@1"/><import as="text" from="@hypit/text@1"/><import as="asset" from="@hypit/media@1"/><import as="gpt" from="@hypit/gpt-image@1"/><import as="seedance" from="@hypit/seedance@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="media-track" from="@hypit/media-track@1"/>+ <import as="performance" from="@hypit/performance@1"/><import as="audio-track" from="@hypit/audio-track@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/>@@ -176,16 +178,21 @@<whisperx:SemanticTake id="ronaldo-semantic" narrative={story}segment={story.segment.ronaldo} media={ronaldo-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={messi-semantic.take}/>- <speech:Take source={ronaldo-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={speech-frame}- appearance={recipes.media.performance}/>- </media-track:Track>- <audio-track:Track id="music-bed" semantic={speech.semantic}>- <audio-track:Clip source={soundtrack-media.media} during="program"+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={messi-semantic.take}/>+ <time:Take source={ronaldo-semantic.take}/>+ </time:Timeline>++ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={speech-frame} appearance={recipes.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track>+ <audio-track:Track id="music-bed" timeline={speech.timeline}>+ <audio-track:Item source={soundtrack-media.media} during="program"playback="once-start" gain="0.12" fade-in="300ms" fade-out="600ms"/></audio-track:Track>@@ -193,7 +200,7 @@<fonts:Stack id="column-font" family="inter" weight="700" style="normal"/><ranking:ColumnStyle id="column-style" recipe={recipes.ranking.football-column}font={column-font}/>- <ranking:Column id="football-column" semantic={speech.semantic}+ <ranking:Column id="football-column" timeline={speech.timeline}canvas={vertical} frame={column-frame} during="program" style={column-style}appear-sound={ranking-appear-media.media}move-sound={ranking-move-media.media}>@@ -219,7 +226,7 @@icon={messi-icon.image} during={story.segment.messi}/></ranking:Column>- <media-track:Track id="broll" semantic={speech.semantic} canvas={vertical}>+ <media-track:Track id="broll" timeline={speech.timeline} canvas={vertical}><media-track:Item id="pitch-pov" image={broll-pitch-pov.image}extent={pitch-pov-extent} during={story.selection.pitch-pov}frame={pitch-pov-frame} appearance={recipes.media.pitch-pov}@@ -252,21 +259,22 @@<caption-fine:Style id="caption-style" recipe={recipes.caption.primary}font={display-font}/>- <caption:Program id="caption-program" document={story.caption}- narrative={story} default={caption-style}/>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={caption-style}/>+ </caption-fine:Track>- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={football-column.visual}/><film:Track source={football-column.audio}/><film:Track source={broll.visual}/><film:Track source={captions.track}/>- <film:Track source={music-bed.track}/>+ <film:Track source={music-bed.audio}/></film:Film><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>diff --git a/examples/ranking-football/swap-host.svml b/examples/ranking-football/swap-host.svmlindex 4174148a..0ea12add 100644--- a/examples/ranking-football/swap-host.svml+++ b/examples/ranking-football/swap-host.svml@@ -2,14 +2,16 @@<svml><import from="@hypit/script@1"/>+ <import as="sound" from="@hypit/sound@1"/><import as="text" from="@hypit/text@1"/><import as="asset" from="@hypit/media@1"/><import as="gpt" from="@hypit/gpt-image@1"/><import as="seedance" from="@hypit/seedance@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="media-track" from="@hypit/media-track@1"/>+ <import as="performance" from="@hypit/performance@1"/><import as="audio-track" from="@hypit/audio-track@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/>@@ -194,16 +196,21 @@<whisperx:SemanticTake id="messi-semantic" narrative={story}segment={story.segment.messi} media={messi-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={ronaldo-semantic.take}/>- <speech:Take source={messi-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={speech-frame}- appearance={recipes.media.performance}/>- </media-track:Track>- <audio-track:Track id="music-bed" semantic={speech.semantic}>- <audio-track:Clip source={soundtrack-media.media} during="program"+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={ronaldo-semantic.take}/>+ <time:Take source={messi-semantic.take}/>+ </time:Timeline>++ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={speech-frame} appearance={recipes.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track>+ <audio-track:Track id="music-bed" timeline={speech.timeline}>+ <audio-track:Item source={soundtrack-media.media} during="program"playback="once-start" gain="0.12" fade-in="300ms" fade-out="600ms"/></audio-track:Track>@@ -211,7 +218,7 @@<fonts:Stack id="tier-font" family="inter" weight="400" style="normal"/><ranking:TierBoardStyle id="tier-style" recipe={recipes.ranking.football}font={tier-font}/>- <ranking:TierBoard id="football-tiers" semantic={speech.semantic}+ <ranking:TierBoard id="football-tiers" timeline={speech.timeline}canvas={vertical} frame={tier-frame}during="program" style={tier-style}appear-sound={ranking-appear-media.media}@@ -233,7 +240,7 @@<!-- Every Item consumes a projected Selection window. Later overlappingjokes use a higher stack order through their appearance Recipe. -->- <media-track:Track id="broll" semantic={speech.semantic} canvas={vertical}>+ <media-track:Track id="broll" timeline={speech.timeline} canvas={vertical}><media-track:Item id="hair-gel" image={broll-hair-gel.image}extent={hair-gel-extent} during={story.selection.hair-gel}frame={hair-gel-frame} appearance={recipes.media.hair-gel}@@ -270,21 +277,22 @@<caption-fine:Style id="caption-style" recipe={recipes.caption.primary}font={display-font}/>- <caption:Program id="caption-program" document={story.caption}- narrative={story} default={caption-style}/>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={caption-style}/>+ </caption-fine:Track>- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={football-tiers.visual}/><film:Track source={football-tiers.audio}/><film:Track source={broll.visual}/><film:Track source={captions.track}/>- <film:Track source={music-bed.track}/>+ <film:Track source={music-bed.audio}/></film:Film><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>diff --git a/examples/ranking-football/swap-topic.svml b/examples/ranking-football/swap-topic.svmlindex 019d801e..a0cc7abf 100644--- a/examples/ranking-football/swap-topic.svml+++ b/examples/ranking-football/swap-topic.svml@@ -2,14 +2,16 @@<svml><import from="@hypit/script@1"/>+ <import as="sound" from="@hypit/sound@1"/><import as="text" from="@hypit/text@1"/><import as="asset" from="@hypit/media@1"/><import as="gpt" from="@hypit/gpt-image@1"/><import as="seedance" from="@hypit/seedance@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="media-track" from="@hypit/media-track@1"/>+ <import as="performance" from="@hypit/performance@1"/><import as="audio-track" from="@hypit/audio-track@1"/><import as="caption" from="@hypit/caption@1"/><import as="caption-fine" from="@hypit/caption-fine@1"/>@@ -219,16 +221,21 @@<whisperx:SemanticTake id="sam-semantic" narrative={story}segment={story.segment.sam} media={sam-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={elon-semantic.take}/>- <speech:Take source={sam-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={speech-frame}- appearance={recipes.media.performance}/>- </media-track:Track>- <audio-track:Track id="music-bed" semantic={speech.semantic}>- <audio-track:Clip source={soundtrack-media.media} during="program"+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={elon-semantic.take}/>+ <time:Take source={sam-semantic.take}/>+ </time:Timeline>++ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={speech-frame} appearance={recipes.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={vertical}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track>+ <audio-track:Track id="music-bed" timeline={speech.timeline}>+ <audio-track:Item source={soundtrack-media.media} during="program"playback="once-start" gain="0.12" fade-in="300ms" fade-out="600ms"/></audio-track:Track>@@ -236,7 +243,7 @@<fonts:Stack id="tier-font" family="inter" weight="400" style="normal"/><ranking:TierBoardStyle id="tier-style" recipe={recipes.ranking.tech}font={tier-font}/>- <ranking:TierBoard id="tech-tiers" semantic={speech.semantic}+ <ranking:TierBoard id="tech-tiers" timeline={speech.timeline}canvas={vertical} frame={tier-frame}during="program" style={tier-style}appear-sound={ranking-appear-media.media}@@ -254,7 +261,7 @@icon={sam-icon.image} during={story.segment.sam}/></ranking:TierBoard>- <media-track:Track id="broll" semantic={speech.semantic} canvas={vertical}>+ <media-track:Track id="broll" timeline={speech.timeline} canvas={vertical}><media-track:Item id="money-rain" image={broll-money-rain.image}extent={money-rain-extent} during={story.selection.money-rain}frame={money-rain-frame} appearance={recipes.media.money-rain}@@ -311,21 +318,22 @@<caption-fine:Style id="caption-style" recipe={recipes.caption.primary}font={display-font}/>- <caption:Program id="caption-program" document={story.caption}- narrative={story} default={caption-style}/>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={caption-style}/>+ </caption-fine:Track>- <film:Film id="main" canvas={vertical} semantic={speech.semantic}+ <film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={tech-tiers.visual}/><film:Track source={tech-tiers.audio}/><film:Track source={broll.visual}/><film:Track source={captions.track}/>- <film:Track source={music-bed.track}/>+ <film:Track source={music-bed.audio}/></film:Film><render:Video id="final" composition={main.composition}- semantic={speech.semantic}/>+ timeline={speech.timeline}/></svml>diff --git a/examples/semantic-composition/README.md b/examples/semantic-composition/README.mdindex 9478bf92..22caacb3 100644--- a/examples/semantic-composition/README.md+++ b/examples/semantic-composition/README.md@@ -33,3 +33,9 @@ there is no hosted generation account in this example. The final Output is `finaRun can be opened in Studio, where its component lane and physical clock work without a Script lane.For an independent project, install `@hypit/hypit`, copy the component into `packages/`, replace its workspacedependency with the installed Hypit version, and build the package normally.++## Sound presentation Styles++`packages/sound-styles` defines a project crossfade with explicit outgoing and incoming Segments.+It uses Sound Uses over the same Timeline and preserves original source playback through the blend.+Its README explains authoring and the public fragment interface.diff --git a/examples/semantic-composition/chat.svml b/examples/semantic-composition/chat.svmlindex 513a767b..68396b8e 100644--- a/examples/semantic-composition/chat.svml+++ b/examples/semantic-composition/chat.svml@@ -1,6 +1,6 @@<?svml using="@hypit/markup@1"?><svml>- <import as="time" from="@hypit/program-space@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="spatial" from="@hypit/spatial@1"/><import as="fonts" from="@hypit/fonts-open@1"/><import as="chat" from="@example/chat-scene@1"/>@@ -8,18 +8,19 @@<import as="render" from="@hypit/render-hyperframes@1"/><import as="style" source="./chat.svs"/>- <time:Space id="animation" frame-rate="30" duration="8s"/>+ <time:Clock id="animation-clock" frame-rate="30"/>+<time:Timeline id="animation" clock={animation-clock} end="8s"/><spatial:Canvas id="canvas" width="540" height="960"/><fonts:Stack id="font" family="inter" weight="600" style="normal"/>- <chat:Scene id="conversation" space={animation} canvas={canvas} font={font}+ <chat:Scene id="conversation" timeline={animation.timeline} canvas={canvas} font={font}during="program" title="Launch crew"><chat:Message id="question" sender="Maya" side="left" at="0.5s" text="Are we ready to launch?"/><chat:Message id="reply" sender="Leo" side="right" at="2s" text="The video is ready."/><chat:Message id="turn" sender="Maya" side="left" at="3.5s" text="Great. One tiny change..."/><chat:Message id="reveal" sender="Leo" side="right" at="5.2s" text="I left the whole scene editable."/></chat:Scene>- <film:Film id="main" canvas={canvas} space={animation} appearance={style.film.main}>+ <film:Film id="main" canvas={canvas} timeline={animation.timeline} appearance={style.film.main}><film:Track source={conversation.track}/></film:Film>- <render:Video id="final" composition={main.composition} space={animation}/>+ <render:Video id="final" composition={main.composition} timeline={animation.timeline}/></svml>diff --git a/examples/semantic-composition/packages/chat-scene/README.md b/examples/semantic-composition/packages/chat-scene/README.mdindex 4870ec8d..b753f925 100644--- a/examples/semantic-composition/packages/chat-scene/README.md+++ b/examples/semantic-composition/packages/chat-scene/README.md@@ -6,20 +6,20 @@ Content lives in Source. Every Message has its own `at`; there is no fixed messaFor speech-led work, bind the message to the words that introduce it:```svml-<chat:Scene id="conversation" semantic={speech.semantic} canvas={canvas} font={font}+<chat:Scene id="conversation" timeline={speech.timeline} canvas={canvas} font={font}during={story.selection.demo} title="Conversation"><chat:Message id="answer" sender="Maya" side="left" text="Here it is."at={story.moment.answer}/></chat:Scene>```-The same component can receive `space={animation}` and `at="2s"` for authored animation. A Scene+The same component can receive a zero-Take `timeline={animation.timeline}` and `at="2s"` for authored animation. A Scenehas an outer Window; its message Instants trigger persistent state. `entrance-frames` controls arrivaland scrolling duration independently of those triggers (default 10). Messages stay visible or scrollup as the conversation develops. Message times are on the film clock and must fall inside the Scene.The Surface uses the shared temporal helpers and expands children to a finite create/append/render-graph. The rendering function receives ProgramSpace, Window and Instants, so it knows nothing about+graph. The rendering function receives Timeline, Window and Instants, so it knows nothing aboutScript parsing, speech models or Runtime. The browser program evaluates any requested frame directly,including frames reached by scrubbing, range rendering or concurrent workers.diff --git a/examples/semantic-composition/packages/chat-scene/src/activation.ts b/examples/semantic-composition/packages/chat-scene/src/activation.tsindex 32db0e5e..cd8bb506 100644--- a/examples/semantic-composition/packages/chat-scene/src/activation.ts+++ b/examples/semantic-composition/packages/chat-scene/src/activation.ts@@ -3,14 +3,13 @@ import type { ComponentPackage, FragmentOperation, ModuleManifest, StructuredSurimport { compositionTypes } from "@hypit/hypit/composition";import { mediaTypes } from "@hypit/hypit/media";import type { FontStackRef } from "@hypit/hypit/media";-import { programSpaceTypes } from "@hypit/hypit/program-space";-import type { ProgramSpace } from "@hypit/hypit/program-space";-import { semanticTrackTypes } from "@hypit/hypit/semantic-track";+import type { Timeline } from "@hypit/hypit/timeline";+import { timelineTypes } from "@hypit/hypit/timeline";import { spatialTypes } from "@hypit/hypit/spatial";import type { CanvasSpace } from "@hypit/hypit/spatial";import { assertTemporalInstantFor, temporalTypes } from "@hypit/hypit/temporal";import type { TemporalInstant, TemporalWindow } from "@hypit/hypit/temporal";-import { createTemporalInstantProjection, createTemporalSpace, createTemporalWindowProjection, resolveTemporalContext,+import { createTemporalInstantProjection, createTemporalWindowProjection, resolveTemporalContext,temporalContextAttributeVocabulary, temporalInstantAttributeNames, temporalInstantAttributeVocabulary,temporalWindowAttributeNames, temporalWindowAttributeVocabulary } from "@hypit/hypit/temporal-markup";import { renderChat } from "./render.js";@@ -20,14 +19,13 @@ const module = { name: "@example/chat-scene", version: "1" } as const;const types = Object.fromEntries(["Options", "Message", "Messages"].map(name => [name, { module, name }])) as Record<"Options" | "Message" | "Messages", TypeRef>;const producers = Object.fromEntries(["empty", "append", "render"].map(name => [name, { module, name }])) as Record<"empty" | "append" | "render", { module: typeof module; name: string }>;export const manifest: ModuleManifest = { format: "hypit.module@1", ...module,- dependencies: [compositionTypes.visualTrack, mediaTypes.fontStack, programSpaceTypes.programSpace,- semanticTrackTypes.track, spatialTypes.canvas, temporalTypes.instant].map(type => ({ module: type.module })),+ dependencies: [compositionTypes.visualTrack, mediaTypes.fontStack, timelineTypes.track, spatialTypes.canvas, temporalTypes.instant].map(type => ({ module: type.module })),types: Object.values(types).map(type => ({ name: type.name })), capabilities: [], producers: [{ name: "empty", inputs: [], outputs: [{ name: "messages", type: types.Messages }], needs: [] },{ name: "append", inputs: [{ name: "messages", type: types.Messages }, { name: "message", type: types.Message },- { name: "at", type: temporalTypes.instant }, { name: "space", type: programSpaceTypes.programSpace }], outputs: [{ name: "messages", type: types.Messages }], needs: [] },+ { name: "at", type: temporalTypes.instant }, { name: "timeline", type: timelineTypes.track }], outputs: [{ name: "messages", type: types.Messages }], needs: [] },{ name: "render", inputs: [{ name: "messages", type: types.Messages }, { name: "options", type: types.Options },- { name: "space", type: programSpaceTypes.programSpace }, { name: "canvas", type: spatialTypes.canvas },+ { name: "timeline", type: timelineTypes.track }, { name: "canvas", type: spatialTypes.canvas },{ name: "window", type: temporalTypes.window }, { name: "font", type: mediaTypes.fontStack }], outputs: [{ name: "track", type: compositionTypes.visualTrack }], needs: [] },],};@@ -41,20 +39,19 @@ const component: ComponentPackage = { producers: [{ producer: producers.append, handler: ({ inputs }) => {const message = inline<Omit<Message, "at">>(inputs.message), at = inline<TemporalInstant>(inputs.at);const messages = inline<Message[]>(inputs.messages);- assertTemporalInstantFor(at, { subjectId: message.id, space: inline<ProgramSpace>(inputs.space) });+ assertTemporalInstantFor(at, { subjectId: message.id, space: inline<Timeline>(inputs.timeline) });if (messages.some(item => item.id === message.id) || (messages.at(-1)?.at.frame ?? -1) > at.frame) throw new Error("Chat messages need unique ids and chronological arrival times.");return { outputs: { messages: value([...messages, { ...message, at }]) }, needs: {} };} },- { producer: producers.render, handler: ({ inputs }) => ({ outputs: { track: value(renderChat(inline<ProgramSpace>(inputs.space),+ { producer: producers.render, handler: ({ inputs }) => ({ outputs: { track: value(renderChat(inline<Timeline>(inputs.timeline),inline<CanvasSpace>(inputs.canvas), inline<TemporalWindow>(inputs.window), inline<FontStackRef>(inputs.font),inline<Message[]>(inputs.messages), inline<ChatOptions>(inputs.options))) }, needs: {} }) },] };export const decodeSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {- assertAttributes(element, ["id", "semantic", "space", "canvas", "font", "title", "entrance-frames", ...temporalWindowAttributeNames]);+ assertAttributes(element, ["id", "timeline", "canvas", "font", "title", "entrance-frames", ...temporalWindowAttributeNames]);const id = textAttribute(element, "id"), context = resolveTemporalContext({ element, resolveReference });- const time = createTemporalSpace({ id, element, ...context });- const window = createTemporalWindowProjection({ id: `${id}.window`, subjectId: id, element, ...context, space: time.space, resolveReference });+ const window = createTemporalWindowProjection({ id: `${id}.window`, subjectId: id, element, ...context, resolveReference });const reference = (name: string, type: TypeRef): SurfaceResolvedReference => {const raw = element.attributes[name];if (typeof raw !== "object" || raw.kind !== "reference") throw new Error(`${name} must be a reference.`);@@ -64,10 +61,10 @@ export const decodeSurface: StructuredSurfaceHandler = ({ element, resolveRefere};const options: ChatOptions = { id, title: textAttribute(element, "title"), entranceFrames: Number(element.attributes["entrance-frames"] ?? "10") };const records = [...window.records, { id: `${id}.options`, type: types.Options, value: value(options), range: element.range }];- const components = [...time.components, ...window.components], fragments = [...time.fragments, ...window.fragments];- const inputs = [{ name: "space", type: programSpaceTypes.programSpace }, { name: "canvas", type: spatialTypes.canvas },+ const components = [...window.components], fragments = [...window.fragments];+ const inputs = [{ name: "timeline", type: timelineTypes.track }, { name: "canvas", type: spatialTypes.canvas },{ name: "font", type: mediaTypes.fontStack }, { name: "window", type: temporalTypes.window }, { name: "options", type: types.Options }];- const bindings: Record<string, SurfaceResolvedReference["ref"]> = { space: time.space.ref, canvas: reference("canvas", spatialTypes.canvas).ref,+ const bindings: Record<string, SurfaceResolvedReference["ref"]> = { timeline: context.timeline.ref, canvas: reference("canvas", spatialTypes.canvas).ref,font: reference("font", mediaTypes.fontStack).ref, window: window.ref, options: { kind: "record", id: `${id}.options` } };const input = (name: string) => ({ kind: "fragment-input" as const, name });const operation = (name: string) => ({ kind: "fragment-operation" as const, operation: name });@@ -79,24 +76,24 @@ export const decodeSurface: StructuredSurfaceHandler = ({ element, resolveRefereassertAttributes(child, ["id", "sender", "text", "side", ...temporalInstantAttributeNames]); assertEmptyElement(child);const messageId = textAttribute(child, "id"), side = textAttribute(child, "side");if (side !== "left" && side !== "right") throw new Error("Message side must be left or right.");- const at = createTemporalInstantProjection({ id: `${id}.${messageId}`, subjectId: messageId, element: child, ...context, space: time.space, resolveReference });+ const at = createTemporalInstantProjection({ id: `${id}.${messageId}`, subjectId: messageId, element: child, ...context, resolveReference });records.push(...at.records); components.push(...at.components); fragments.push(...at.fragments);const key = `message-${++index}`;records.push({ id: `${id}.${key}`, type: types.Message, value: value({ id: messageId, sender: textAttribute(child, "sender"), text: textAttribute(child, "text"), side }), range: child.range });inputs.push({ name: key, type: types.Message }, { name: `${key}-at`, type: temporalTypes.instant });bindings[key] = { kind: "record", id: `${id}.${key}` }; bindings[`${key}-at`] = at.ref;- operations.push({ id: key, producer: producers.append, inputs: { messages: operation(previous), message: input(key), at: input(`${key}-at`), space: input("space") }, result: { kind: "output", name: "messages" } });+ operations.push({ id: key, producer: producers.append, inputs: { messages: operation(previous), message: input(key), at: input(`${key}-at`), timeline: input("timeline") }, result: { kind: "output", name: "messages" } });previous = key;}if (!index) throw new Error("Chat Scene requires a Message.");operations.push({ id: "render", producer: producers.render, inputs: { messages: operation(previous), options: input("options"),- space: input("space"), canvas: input("canvas"), font: input("font"), window: input("window") }, result: { kind: "output", name: "track" } });+ timeline: input("timeline"), canvas: input("canvas"), font: input("font"), window: input("window") }, result: { kind: "output", name: "track" } });const fragment = sealGraphFragment({ inputs, operations, exports: [{ name: "track", type: compositionTypes.visualTrack, root: operation("render") }] });return { records, fragments: [...fragments, fragment], components: [...components,{ id, fragment: fragment.id, inputs: bindings, outputs: { track: `${id}.track` }, range: element.range }], exports: [`${id}.track`] };};const declaration = { name: "scene", tag: "Scene", mode: "structured" as const,- outputs: [compositionTypes.visualTrack, programSpaceTypes.programSpace, temporalTypes.window, temporalTypes.instant, temporalTypes.windowSpec, temporalTypes.instantSpec, ...Object.values(types)],+ outputs: [compositionTypes.visualTrack, timelineTypes.track, temporalTypes.window, temporalTypes.instant, temporalTypes.windowSpec, temporalTypes.instantSpec, ...Object.values(types)],vocabulary: { summary: "A conversation whose message arrivals and scrolling form one visual scene.", attributes: [...temporalContextAttributeVocabulary, ...temporalWindowAttributeVocabulary,...["id", "title", "canvas", "font"].map(name => ({ name, kind: "expression" as const, required: true, summary: name })),@@ -104,7 +101,7 @@ const declaration = { name: "scene", tag: "Scene", mode: "structured" as const,], children: [{ tag: "Message", cardinality: "many" as const, summary: "One authored message and the event that reveals it.", attributes: [...["id", "sender", "text", "side"].map(name => ({ name, kind: "literal" as const, required: true, summary: name })), ...temporalInstantAttributeVocabulary,] }], ports: [{ name: "track", type: compositionTypes.visualTrack, summary: "The complete conversation scene." }],- example: '<chat:Scene id="chat" semantic={speech.semantic} canvas={canvas} font={font} during="program" title="Conversation"><chat:Message id="answer" sender="Maya" side="left" text="Here it is." at={story.moment.answer}/></chat:Scene>',+ example: '<chat:Scene id="chat" timeline={speech.timeline} canvas={canvas} font={font} during="program" title="Conversation"><chat:Message id="answer" sender="Maya" side="left" text="Here it is." at={story.moment.answer}/></chat:Scene>',},};export const hypitPackage = { format: "hypit.node-package@1" as const, modules: [{ manifest }], components: [component],diff --git a/examples/semantic-composition/packages/chat-scene/src/render.ts b/examples/semantic-composition/packages/chat-scene/src/render.tsindex eebc8808..cf62ae0e 100644--- a/examples/semantic-composition/packages/chat-scene/src/render.ts+++ b/examples/semantic-composition/packages/chat-scene/src/render.ts@@ -2,7 +2,7 @@ import { sealVisualTrack } from "@hypit/hypit/composition";import type { VisualElement } from "@hypit/hypit/composition";import { browserProgram } from "@hypit/hypit/hyperframes";import type { FontStackRef } from "@hypit/hypit/media";-import type { ProgramSpace } from "@hypit/hypit/program-space";+import type { Timeline } from "@hypit/hypit/timeline";import type { CanvasSpace } from "@hypit/hypit/spatial";import { assertTemporalWindowFor } from "@hypit/hypit/temporal";import type { TemporalInstant, TemporalWindow } from "@hypit/hypit/temporal";@@ -11,9 +11,9 @@ export type Message = { id: string; sender: string; text: string; side: "left" |export type ChatOptions = { id: string; title: string; entranceFrames: number };/** Layout, arrival and scrolling share one visual component; timing is already resolved. */-export function renderChat(space: ProgramSpace, canvas: CanvasSpace, window: TemporalWindow,+export function renderChat(timeline: Timeline, canvas: CanvasSpace, window: TemporalWindow,font: FontStackRef, messages: readonly Message[], options: ChatOptions) {- assertTemporalWindowFor(window, { subjectId: options.id, space });+ assertTemporalWindowFor(window, { subjectId: options.id, space: timeline });if (!Number.isSafeInteger(options.entranceFrames) || options.entranceFrames < 1) throw new Error("Chat entrance-frames must be a positive integer.");if (messages.some(message => message.at.frame < window.span.startFrame || message.at.frame >= window.span.endFrameExclusive)) throw new Error("Chat messages must appear inside the scene's window.");let order = 0;@@ -56,7 +56,7 @@ export function renderChat(space: ProgramSpace, canvas: CanvasSpace, window: Tem[...dots.children].forEach((dot,i)=>dot.style.transform='translateY('+(-3*(1+Math.sin(frame*.18-i)))+'px)');};`,});- return sealVisualTrack({ id: options.id, programSpaceId: space.id, visualIr: "hypit.visual-ir@1",+ return sealVisualTrack({ id: options.id, programSpaceId: timeline.id, visualIr: "hypit.visual-ir@1",presents: [{ id: options.id, span: window.span, stacking: { order: 0, tieBreak: options.id },elements: [{ id: "chat", kind: "program", order: 0, program,style: [{ name: "position", value: "absolute" }, { name: "inset", value: 0 },diff --git a/examples/semantic-composition/packages/chat-scene/test/render.test.ts b/examples/semantic-composition/packages/chat-scene/test/render.test.tsindex 1c338503..55eec771 100644--- a/examples/semantic-composition/packages/chat-scene/test/render.test.ts+++ b/examples/semantic-composition/packages/chat-scene/test/render.test.ts@@ -4,23 +4,23 @@ import type { BrowserProgram } from "@hypit/hypit/hyperframes";import type { FontStackRef } from "@hypit/media";import type { ProgramSpace } from "@hypit/program-space";import { composeTemporalWindow, projectMomentInstant, projectProgramInstant } from "@hypit/temporal";-import { semanticTrackFixture } from "../../../../../test/semantic-track-fixture.js";+import { timelineFixture } from "../../../../../test/timeline-fixture.js";import { fixtureResource } from "../../../../../test/fixture-resource.js";import { renderChat } from "../src/render.js";const space: ProgramSpace = { id: "animation", durationSec: 8, frameRate: { numerator: 30, denominator: 1 } };const font: FontStackRef = { faces: [{ sources: [{ artifact: { kind: "blob", resource: fixtureResource("chat-font"), size: 32, mediaType: "font/woff2" } }], weight: 600, style: "normal" }] };-const point = (id: string, frame: number) => projectProgramInstant({ itemId: id, subjectId: id, space,+const point = (id: string, frame: number) => projectProgramInstant({ itemId: id, subjectId: id, timeline: { ...space, items: [] },projection: { ref: "absolute", at: { unit: "frames", value: frame } }, authority: { kind: "fixed" } });const window = composeTemporalWindow({ id: "chat", subjectId: "chat" }, point("chat", 0), point("chat", 240));test("the same chat renderer consumes authored and word-bound events; changing performance moves only the word-bound message", () => {const scene = (wordFrame: number) => {- const semantic = semanticTrackFixture(space, { anchors: [{ identity: "answer", frame: wordFrame }] });- const at = projectMomentInstant({ itemId: "reply", subjectId: "reply", semantic,+ const semantic = timelineFixture(space, { anchors: [{ identity: "answer", frame: wordFrame }] });+ const at = projectMomentInstant({ itemId: "reply", subjectId: "reply", timeline: semantic,moment: { narrativeId: semantic.narrativeId, id: "answer", anchorId: "answer" },projection: { ref: "moment.cue" }, authority: { kind: "semantic", boundary: "cue" } });- return renderChat(space, { widthPx: 540, heightPx: 960, origin: "top-left", xDirection: "right", yDirection: "down", pixelAspect: "square" }, window, font, [+ return renderChat(semantic, { widthPx: 540, heightPx: 960, origin: "top-left", xDirection: "right", yDirection: "down", pixelAspect: "square" }, window, font, [{ id: "opening", sender: "Maya", text: "Ready?", side: "left", at: point("opening", 15) },{ id: "reply", sender: "Leo", text: "Ready.", side: "right", at },], { id: "chat", title: "Launch", entranceFrames: 10 });diff --git a/examples/semantic-composition/packages/performance-styles/README.md b/examples/semantic-composition/packages/performance-styles/README.mdnew file mode 100644index 00000000..851a8526--- /dev/null+++ b/examples/semantic-composition/packages/performance-styles/README.md@@ -0,0 +1,42 @@+# Project-authored Performance Styles++This package demonstrates two ordinary project families: a viewport moving between Frames and a+crossfade between explicitly selected Segments. Both export the common PerformanceStyle, so they+work alongside the ordinary fixed Style inside one Performance Track.++```svml+<import as="performance" from="@hypit/performance@1"/>+<import as="styles" from="@example/performance-styles@1"/>+<performance:Style id="full" frame={layout.full} appearance={recipes.media.presenter}/>+<performance:Style id="side" frame={layout.side} appearance={recipes.media.presenter}/>+<styles:Move id="full-to-side" from={layout.full} to={layout.side}/>+<styles:Crossfade id="mix" outgoing={story.segment.previous}+ incoming={story.segment.next} frame={layout.side}/>+<performance:Track id="presenter" timeline={program.timeline} canvas={canvas}>+ <performance:Use style={full}/>+ <performance:Use moment={story.moment.explain}+ start="moment.cue" end="program.end" style={side}/>+ <performance:Use at={story.moment.explain} for="2s" style={full-to-side}/>+ <performance:Use start-source={story.segment.next} start="segment.start"+ end-source={story.segment.previous} end="segment.end" style={mix}/>+</performance:Track>+```++The two-second movement overrides the beginning of the already declared side treatment. Changing+its duration requires no reference to a Use id. Shared endpoint Frames keep the fixed and moving+layouts aligned. All timing belongs to Uses; each Style is independently reusable.++Crossfade requires known outgoing/incoming Segment identities. A known audio-only source supplies+transparent picture, giving a fade in or out; an unknown reference reports the actual error. This+example uses isolated additive blending of weighted pictures. `projectTimelineMedia` preserves the+original source positions; animation progress comes from the original Use Window.++Read `activation.ts` for the complete author Surface, bound fragment and Producer wiring, and+`render.ts` for the browser program. A project can add its own parameters or additional events without+changing Performance. Compile with `npm run build`; install the package in the video project's+ordinary package dependencies. The rendering test exercises these same functions with actual video.++The package also includes `src/studio.ts`. Move exposes its `from` and `to` Frame coordinates;+Crossfade exposes its Frame and read-only source references. These parameter Companions match the+project's own Style Surfaces and are composed into any consuming Performance Use. The Track+Companion and Studio require no knowledge of these Style names.diff --git a/examples/semantic-composition/packages/performance-styles/package.json b/examples/semantic-composition/packages/performance-styles/package.jsonnew file mode 100644index 00000000..844505d9--- /dev/null+++ b/examples/semantic-composition/packages/performance-styles/package.json@@ -0,0 +1,25 @@+{+ "name": "@example/performance-styles",+ "version": "0.0.0-dev",+ "private": true,+ "type": "module",+ "hypit": {+ "activation": "./dist/activation.js"+ },+ "exports": {+ ".": "./dist/render.js"+ },+ "files": [+ "dist",+ "README.md"+ ],+ "scripts": {+ "build": "tsc -p tsconfig.json",+ "prepack": "npm run build"+ },+ "devDependencies": {+ "@hypit/hypit": "workspace:*",+ "typescript": "5.9.3",+ "@types/node": "24.10.1"+ }+}diff --git a/examples/semantic-composition/packages/performance-styles/src/activation.ts b/examples/semantic-composition/packages/performance-styles/src/activation.tsnew file mode 100644index 00000000..1f2c8d56--- /dev/null+++ b/examples/semantic-composition/packages/performance-styles/src/activation.ts@@ -0,0 +1,57 @@+import { styleStudioFacet } from "./studio.js";+import { assertAttributes, assertEmptyElement, canonicalize, createMarkupSurfaceHostFacet, sameType, sealGraphFragment, textAttribute } from "@hypit/hypit/author-kit";+import type { ComponentPackage, ModuleManifest, StructuredSurfaceHandler, SurfaceResolvedReference } from "@hypit/hypit/author-kit";+import { compositionTypes } from "@hypit/hypit/composition";+import { performanceStyle, performanceTypes, performanceModuleRef } from "@hypit/hypit/performance";+import { timelineTypes } from "@hypit/hypit/timeline";+import type { Timeline } from "@hypit/hypit/timeline";+import { spatialTypes } from "@hypit/hypit/spatial";+import type { SpatialFrame } from "@hypit/hypit/spatial";+import { temporalTypes } from "@hypit/hypit/temporal";+import type { TemporalWindow } from "@hypit/hypit/temporal";+import { narrativeTypes } from "@hypit/hypit/narrative";+import type { NarrativeExcerpt } from "@hypit/hypit/narrative";+import { movingFrame, crossfade } from "./render.js";++const module = { name: "@example/performance-styles", version: "1" } as const;+const common = [{ name: "timeline", type: timelineTypes.track }, { name: "canvas", type: spatialTypes.canvas }, { name: "window", type: temporalTypes.window }];+const moveInputs = [...common, { name: "from", type: spatialTypes.frame }, { name: "to", type: spatialTypes.frame }];+const mixInputs = [...common, { name: "outgoing", type: narrativeTypes.excerpt }, { name: "incoming", type: narrativeTypes.excerpt }, { name: "frame", type: spatialTypes.frame }];+const families = [{ name: "move", tag: "Move", ports: moveInputs }, { name: "crossfade", tag: "Crossfade", ports: mixInputs }];+export const manifest: ModuleManifest = { format: "hypit.module@1", ...module,+ dependencies: [performanceModuleRef, compositionTypes.visualTrack.module, timelineTypes.track.module, spatialTypes.canvas.module, temporalTypes.window.module, narrativeTypes.excerpt.module].map(module => ({ module })),+ types: [], capabilities: [], producers: families.map(family => ({ name: family.name, inputs: family.ports, outputs: [{ name: "visual", type: compositionTypes.visualTrack }], needs: [] })) };+const fragments = families.map(family => sealGraphFragment({ inputs: family.ports,+ operations: [{ id: "render", producer: { module, name: family.name }, inputs: Object.fromEntries(family.ports.map(port => [port.name, { kind: "fragment-input" as const, name: port.name }])), result: { kind: "output", name: "visual" } }],+ exports: [{ name: "visual", type: compositionTypes.visualTrack, root: { kind: "fragment-operation", operation: "render" } }] }));+const component: ComponentPackage = { producers: families.map(family => ({ producer: { module, name: family.name }, handler: ({ inputs }) => {+ const values = Object.fromEntries(Object.entries(inputs).map(([key, record]) => {+ if (record.value.kind !== "inline") throw new Error(`${key} must be inline`);+ return [key, record.value.value];+ }));+ const visual = family.name === "move" ? movingFrame(values.timeline as unknown as Timeline, values.window as unknown as TemporalWindow, values.from as unknown as SpatialFrame, values.to as unknown as SpatialFrame)+ : crossfade(values.timeline as unknown as Timeline, values.window as unknown as TemporalWindow, values.outgoing as unknown as NarrativeExcerpt, values.incoming as unknown as NarrativeExcerpt, values.frame as unknown as SpatialFrame);+ return { outputs: { visual: { kind: "inline", value: canonicalize(visual) } }, needs: {} };+} })) };+const hostFacets = families.map((family,index) => {+ const params = family.ports.slice(common.length);+ const handler: StructuredSurfaceHandler = ({ element, resolveReference }) => {+ assertAttributes(element, ["id", ...params.map(port => port.name)]); assertEmptyElement(element);+ const bindings: Record<string, SurfaceResolvedReference> = {};+ for (const param of params) {+ const raw = element.attributes[param.name];+ if (typeof raw !== "object" || raw.kind !== "reference") throw new Error(`${param.name} must be a reference`);+ const value = resolveReference(raw.path);+ if (value === undefined || !sameType(value.type,param.type)) throw new Error(`${param.name} has the wrong Type`);+ bindings[param.name] = value;+ }+ return { records: [{ id: textAttribute(element,"id"), type: performanceTypes.style,+ value: { kind: "inline", value: canonicalize(performanceStyle(fragments[index]!,bindings)) }, range: element.range }], components: [], fragments: [] };+ };+ return createMarkupSurfaceHostFacet({ module, declaration: { name: family.name, tag: family.tag, mode: "structured", outputs: [performanceTypes.style], vocabulary: {+ summary: `Project-owned ${family.name} presentation Style.`, example: `<styles:${family.tag} id="look" ${params.map(port => `${port.name}={${port.name}}`).join(' ')}/>`,+ attributes: [{ name: "id", kind: "identifier", required: true, summary: "Style identity." }, ...params.map(port => ({ name: port.name, kind: "reference" as const, required: true, accepts: [port.type], summary: `Explicit ${port.name} input.` }))],+ } }, handler });+});+export const hypitPackage = { format: "hypit.node-package@1" as const, modules: [{ manifest }], components: [component], hostFacets: [...hostFacets, styleStudioFacet(module)] };+export default hypitPackage;diff --git a/examples/semantic-composition/packages/performance-styles/src/render.ts b/examples/semantic-composition/packages/performance-styles/src/render.tsnew file mode 100644index 00000000..32daf180--- /dev/null+++ b/examples/semantic-composition/packages/performance-styles/src/render.ts@@ -0,0 +1,54 @@+import { sealVisualTrack } from "@hypit/hypit/composition";+import type { VisualElement } from "@hypit/hypit/composition";+import { browserProgram } from "@hypit/hypit/hyperframes";+import { projectTimelineMedia } from "@hypit/hypit/timeline";+import type { Timeline } from "@hypit/hypit/timeline";+import type { TemporalWindow } from "@hypit/hypit/temporal";+import type { SpatialFrame } from "@hypit/hypit/spatial";+import type { NarrativeExcerpt } from "@hypit/hypit/narrative";++/** Project behavior, not a registered special effect: the original Use window owns progress. */+export function movingFrame(timeline: Timeline, window: TemporalWindow, from: SpatialFrame, to: SpatialFrame) {+ return render(timeline, window, { from, to });+}+export function crossfade(timeline: Timeline, window: TemporalWindow, outgoing: NarrativeExcerpt, incoming: NarrativeExcerpt, frame: SpatialFrame) {+ for (const ref of [outgoing, incoming]) {+ if (ref.narrativeId !== timeline.narrativeId || !timeline.items.some(item => item.take.segment.segmentId === ref.id)) {+ throw new Error(`Unknown crossfade source ${ref.id}.`);+ }+ }+ return render(timeline, window, { outgoing: outgoing.id, incoming: incoming.id, from: frame, to: frame });+}+function render(timeline: Timeline, window: TemporalWindow, options: { from: SpatialFrame; to: SpatialFrame; outgoing?: string; incoming?: string }) {+ const id = window.subjectId;+ const clips = projectTimelineMedia(timeline, window.span).filter(clip => clip.media.visual !== undefined+ && (options.outgoing === undefined || clip.segmentId === options.outgoing || clip.segmentId === options.incoming));+ const videos: VisualElement[] = clips.map((clip, index) => ({+ id: `source-${index}`, parent: "scene", order: index + 1, kind: "video", muted: true,+ artifact: clip.media.visual!.artifact,+ attributes: [{ name: "data-source-role", value: options.incoming === clip.segmentId ? "incoming" : "outgoing" }],+ style: [{ name: "position", value: "absolute" }, { name: "inset", value: 0 }, { name: "width", value: "100%" }, { name: "height", value: "100%" }, { name: "object-fit", value: "cover" }],+ sampling: { sourceFrameRate: clip.media.timeline.frameRate, sourceFrameCount: clip.media.timeline.frameCount, segments: [{+ target: { startFrame: clip.span.startFrame - window.span.startFrame, endFrameExclusive: clip.span.endFrameExclusive - window.span.startFrame },+ sourceFrame: { numerator: clip.source.startFrame, denominator: 1 }, rate: { numerator: 1, denominator: 1 },+ }] },+ }));+ const program = browserProgram({+ html: `<div class="viewport">${videos.map((video,index) => `<div class="source" data-role="${clips[index]!.segmentId === options.incoming ? "incoming" : "outgoing"}">{{${video.id}}}</div>`).join("")}</div>`,+ css: `.viewport{position:absolute;overflow:hidden;isolation:isolate}.source{position:absolute;inset:0;${options.outgoing === undefined ? '' : 'mix-blend-mode:plus-lighter'}}`,+ data: { from: options.from, to: options.to, mixing: options.outgoing !== undefined, duration: window.span.endFrameExclusive - window.span.startFrame },+ setup: `const view=root.querySelector('.viewport'), sources=view.querySelectorAll('.source');+ return frame => {+ const p=Math.max(0,Math.min(1,frame/Math.max(1,data.duration-1)));+ const q=p*p*(3-2*p), a=data.from, b=data.to;+ Object.assign(view.style,{left:(a.xPx+(b.xPx-a.xPx)*q)+'px',top:(a.yPx+(b.yPx-a.yPx)*q)+'px',+ width:(a.widthPx+(b.widthPx-a.widthPx)*q)+'px',height:(a.heightPx+(b.heightPx-a.heightPx)*q)+'px'});+ if(data.mixing) for(const child of sources) {+ const role=child.dataset.role;+ child.style.opacity=String(role==='incoming'?p:1-p);+ }+ };`,+ });+ return sealVisualTrack({ id, programSpaceId: timeline.id, visualIr: "hypit.visual-ir@1", presents: videos.length ? [{ id, span: window.span,+ stacking: { order: 0, tieBreak: id }, elements: [{ id: "scene", kind: "program", order: 0, style: [{ name: "position", value: "absolute" }, { name: "inset", value: 0 }], program }, ...videos] }] : [] });+}diff --git a/examples/semantic-composition/packages/performance-styles/src/studio.ts b/examples/semantic-composition/packages/performance-styles/src/studio.tsnew file mode 100644index 00000000..99cd0b9a--- /dev/null+++ b/examples/semantic-composition/packages/performance-styles/src/studio.ts@@ -0,0 +1,28 @@+import { createStudioCompanionHostFacet } from "@hypit/hypit/studio-adapter";+import type { StudioParameterCompanion } from "@hypit/hypit/studio-adapter";++/** The project Style owns its inputs; Performance only owns its Use interval. */+export function styleStudioFacet(module: { readonly name: string; readonly version: string }) {+ const coordinates = ["left", "top", "right", "bottom", "x", "y", "width", "height"];+ const definitions = [+ { surface: "move", frames: ["from", "to"], sources: [] },+ { surface: "crossfade", frames: ["frame"], sources: ["outgoing", "incoming"] },+ ];+ const parameters: StudioParameterCompanion[] = definitions.map(({ surface, frames, sources }) => ({+ id: surface, match: { module, surface },+ bindings: [+ ...frames.map(name => ({ name, referenced: coordinates.map(name => ({ name, writable: true })) })),+ ...sources.map(name => ({ name })),+ ],+ inspector: [+ ...frames.flatMap(frame => coordinates.map(name => ({+ binding: `${frame}.${name}`, label: name, domain: "where" as const,+ page: { id: frame, label: frame }, section: { id: "frame", label: "Frame" },+ control: "number" as const, number: { suffixes: ["%", "px"], step: 1 },+ }))),+ ...sources.map(name => ({ binding: name, label: name, domain: "how" as const,+ section: { id: "sources", label: "Sources" }, control: "text" as const })),+ ],+ }));+ return createStudioCompanionHostFacet({ parameters });+}diff --git a/examples/semantic-composition/packages/performance-styles/tsconfig.json b/examples/semantic-composition/packages/performance-styles/tsconfig.jsonnew file mode 100644index 00000000..6d5a18e7--- /dev/null+++ b/examples/semantic-composition/packages/performance-styles/tsconfig.json@@ -0,0 +1,18 @@+{+ "compilerOptions": {+ "target": "ES2023",+ "module": "NodeNext",+ "moduleResolution": "NodeNext",+ "lib": ["ES2023"],+ "strict": true,+ "noUncheckedIndexedAccess": true,+ "exactOptionalPropertyTypes": true,+ "verbatimModuleSyntax": true,+ "declaration": true,+ "sourceMap": true,+ "rootDir": "src",+ "outDir": "dist",+ "skipLibCheck": true+ },+ "include": ["src/**/*.ts"]+}diff --git a/examples/semantic-composition/packages/responsive-explainer/README.md b/examples/semantic-composition/packages/responsive-explainer/README.mdindex 67e35d58..c5146ca4 100644--- a/examples/semantic-composition/packages/responsive-explainer/README.md+++ b/examples/semantic-composition/packages/responsive-explainer/README.md@@ -8,16 +8,23 @@ With prepared Takes, a Canvas and an exact FontStack already available:```svml<import as="explainer" from="@example/responsive-explainer@1"/>-<speech:Track id="speech">- <speech:Take source={opening.take}/>- <speech:Take source={explanation.take}/>-</speech:Track>-<explainer:Scene id="scene" semantic={speech.semantic} canvas={canvas} font={font}+<time:Clock id="clock" frame-rate="30"/>+<time:Timeline id="speech" clock={clock}>+ <time:Take source={opening.take}/>+ <time:Take source={explanation.take}/>+</time:Timeline>+<explainer:Scene id="scene" timeline={speech.timeline} canvas={canvas} font={font}during="program" reveal={story.moment.demonstrate} title="Make room for meaning"transition-frames="24" stack-order="0"/>-<film:Film id="main" canvas={canvas} semantic={speech.semantic} appearance={look.film}>+<import as="sound" from="@hypit/sound@1"/>+<sound:Style id="voice-style"/>+<sound:Track id="voice" timeline={speech.timeline}>+ <sound:Use style={voice-style}/>+</sound:Track>++<film:Film id="main" canvas={canvas} timeline={speech.timeline} appearance={look.film}><film:Track source={scene.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={voice.audio}/><film:Track source={captions.track}/></film:Film>```@@ -28,11 +35,11 @@ the Script or using another delivery changes the projected frame while retaining`transition-frames` is the duration of that change, separate from the scene's lifetime. `stack-order`places this scene among other contributions. The caller supplies its title and exact fonts.-`projectSemanticMedia` selects all intersecting prepared Takes. Each video's target interval is+`projectTimelineMedia` selects all intersecting prepared Takes. Each video's target interval isPresent-local and its source offset is preserved. The parent viewport changes size and position;playback is never restarted by the layout change. The component owns the diagram and the viewport,so its internal overlap, rounded clipping and backdrop blur are ordinary HTML/CSS relationships.-The original audio stays independently selectable as `speech.audio`.+Sound independently presents the existing audio as `voice.audio`.`src/activation.ts` declares the Module, Surface, semantic projections and Producer Fragment.`src/render.ts` emits a browser program with HTML slots for typed video and text children. Its renderdiff --git a/examples/semantic-composition/packages/responsive-explainer/src/activation.ts b/examples/semantic-composition/packages/responsive-explainer/src/activation.tsindex 26469636..254cbf90 100644--- a/examples/semantic-composition/packages/responsive-explainer/src/activation.ts+++ b/examples/semantic-composition/packages/responsive-explainer/src/activation.ts@@ -5,8 +5,8 @@ import { compositionTypes } from "@hypit/hypit/composition";import { mediaTypes } from "@hypit/hypit/media";import type { FontStackRef } from "@hypit/hypit/media";import { narrativeTypes } from "@hypit/hypit/narrative";-import { semanticTrackTypes } from "@hypit/hypit/semantic-track";-import type { SemanticTrack } from "@hypit/hypit/semantic-track";+import { timelineTypes } from "@hypit/hypit/timeline";+import type { Timeline } from "@hypit/hypit/timeline";import { spatialTypes } from "@hypit/hypit/spatial";import type { CanvasSpace } from "@hypit/hypit/spatial";import { temporalTypes } from "@hypit/hypit/temporal";@@ -20,12 +20,12 @@ const module = { name: "@example/responsive-explainer", version: "1" } as const;const optionsType = { module, name: "ExplainerOptions" };const producer = { module, name: "compose" };const inputs = [- { name: "semantic", type: semanticTrackTypes.track }, { name: "canvas", type: spatialTypes.canvas },+ { name: "timeline", type: timelineTypes.track }, { name: "canvas", type: spatialTypes.canvas },{ name: "window", type: temporalTypes.window }, { name: "reveal", type: temporalTypes.instant },{ name: "font", type: mediaTypes.fontStack }, { name: "options", type: optionsType },];export const manifest: ModuleManifest = { format: "hypit.module@1", ...module,- dependencies: [compositionTypes.visualTrack, mediaTypes.fontStack, semanticTrackTypes.track,+ dependencies: [compositionTypes.visualTrack, mediaTypes.fontStack, timelineTypes.track,spatialTypes.canvas, temporalTypes.window].map(type => ({ module: type.module })),types: [{ name: optionsType.name }], capabilities: [],producers: [{ name: producer.name, inputs, outputs: [{ name: "visual", type: compositionTypes.visualTrack }], needs: [] }],@@ -40,12 +40,12 @@ const inline = <T>(record: { value: { kind: string; value?: unknown } } | undefireturn record.value.value as T;};const component: ComponentPackage = { producers: [{ producer, handler: ({ inputs }) => ({ needs: {}, outputs: {- visual: { kind: "inline", value: canonicalize(renderExplainer(inline<SemanticTrack>(inputs.semantic),+ visual: { kind: "inline", value: canonicalize(renderExplainer(inline<Timeline>(inputs.timeline),inline<CanvasSpace>(inputs.canvas), inline<TemporalWindow>(inputs.window), inline<TemporalInstant>(inputs.reveal),inline<FontStackRef>(inputs.font).faces, inline<ExplainerOptions>(inputs.options))) },} }) }] };export const decodeSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {- assertAttributes(element, ["id", "semantic", "canvas", "font", "reveal", "title", "transition-frames", "stack-order", ...temporalWindowAttributeNames]);+ assertAttributes(element, ["id", "timeline", "canvas", "font", "reveal", "title", "transition-frames", "stack-order", ...temporalWindowAttributeNames]);assertEmptyElement(element);const id = textAttribute(element, "id");const reference = (name: string, type: TypeRef): SurfaceResolvedReference => {@@ -55,19 +55,19 @@ export const decodeSurface: StructuredSurfaceHandler = ({ element, resolveRefereif (value === undefined || !sameType(type, value.type)) throw new Error(`${name} has the wrong type.`);return value;};- const semantic = reference("semantic", semanticTrackTypes.track);- const window = createTemporalWindowProjection({ id: `${id}.window`, element, semantic, resolveReference });+ const timeline = reference("timeline", timelineTypes.track);+ const window = createTemporalWindowProjection({ id: `${id}.window`, element, timeline, resolveReference });// Reveal has its own Moment; the outer Window independently controls this scene's lifetime.reference("reveal", narrativeTypes.moment);const reveal = createTemporalInstantProjection({ id: `${id}.reveal`,- element: { ...element, attributes: { at: element.attributes.reveal! } }, semantic, resolveReference });+ element: { ...element, attributes: { at: element.attributes.reveal! } }, timeline, resolveReference });const options: ExplainerOptions = { id, title: textAttribute(element, "title"),transitionFrames: Number(textAttribute(element, "transition-frames")), stackingOrder: Number(textAttribute(element, "stack-order")) };return { records: [...window.records, ...reveal.records, { id: `${id}.options`, type: optionsType,value: { kind: "inline", value: canonicalize(options) }, range: element.range }],fragments: [...window.fragments, ...reveal.fragments, fragment],components: [...window.components, ...reveal.components, { id, fragment: fragment.id,- inputs: { semantic: semantic.ref, canvas: reference("canvas", spatialTypes.canvas).ref,+ inputs: { timeline: timeline.ref, canvas: reference("canvas", spatialTypes.canvas).ref,font: reference("font", mediaTypes.fontStack).ref, window: window.ref, reveal: reveal.ref,options: { kind: "record", id: `${id}.options` } }, outputs: { visual: `${id}.visual` }, range: element.range }],exports: [`${id}.visual`] };@@ -77,12 +77,12 @@ const declaration = { name: "scene", tag: "Scene", mode: "structured" as const,vocabulary: { summary: "A continuously playing performance makes room for a diagram on a semantic Moment.",attributes: [...["id", "title", "transition-frames", "stack-order"].map(name => ({ name, kind: "literal" as const, required: true, summary: name })),- ...[{ name: "semantic", type: semanticTrackTypes.track }, { name: "canvas", type: spatialTypes.canvas },+ ...[{ name: "timeline", type: timelineTypes.track }, { name: "canvas", type: spatialTypes.canvas },{ name: "font", type: mediaTypes.fontStack }, { name: "reveal", type: narrativeTypes.moment }].map(({ name, type }) => ({ name, kind: "reference" as const, required: true, accepts: [type], summary: name })),...temporalWindowAttributeVocabulary,], ports: [{ name: "visual", type: compositionTypes.visualTrack, summary: "The coordinated scene." }],- example: '<explainer:Scene id="scene" semantic={speech.semantic} canvas={canvas} font={font} during="program" reveal={story.moment.reveal} title="How it works" transition-frames="18" stack-order="0"/>',+ example: '<explainer:Scene id="scene" timeline={speech.timeline} canvas={canvas} font={font} during="program" reveal={story.moment.reveal} title="How it works" transition-frames="18" stack-order="0"/>',},};export const hypitPackage = { format: "hypit.node-package@1" as const, modules: [{ manifest }], components: [component],diff --git a/examples/semantic-composition/packages/responsive-explainer/src/render.ts b/examples/semantic-composition/packages/responsive-explainer/src/render.tsindex d3af26d4..2cd84925 100644--- a/examples/semantic-composition/packages/responsive-explainer/src/render.ts+++ b/examples/semantic-composition/packages/responsive-explainer/src/render.ts@@ -2,22 +2,22 @@ import { sealVisualTrack } from "@hypit/hypit/composition";import type { VisualElement } from "@hypit/hypit/composition";import { browserProgram } from "@hypit/hypit/hyperframes";import type { FontArtifactRef } from "@hypit/hypit/media";-import { projectSemanticMedia, projectSemanticProgramSpace } from "@hypit/hypit/semantic-track";-import type { SemanticTrack } from "@hypit/hypit/semantic-track";+import { projectTimelineMedia, projectTimelineSpace } from "@hypit/hypit/timeline";+import type { Timeline } from "@hypit/hypit/timeline";import type { CanvasSpace } from "@hypit/hypit/spatial";import type { TemporalInstant, TemporalWindow } from "@hypit/hypit/temporal";export type ExplainerOptions = { id: string; title: string; transitionFrames: number; stackingOrder: number };/** One scene owns the moving performance viewport and the diagram it makes room for. */-export function renderExplainer(semantic: SemanticTrack, canvas: CanvasSpace, window: TemporalWindow,+export function renderExplainer(semantic: Timeline, canvas: CanvasSpace, window: TemporalWindow,reveal: TemporalInstant, fonts: readonly FontArtifactRef[], options: ExplainerOptions) {- const space = projectSemanticProgramSpace(semantic);+ const space = projectTimelineSpace(semantic);if (window.start.source.spaceId !== space.id || reveal.source.spaceId !== space.id)throw new Error("Explainer timing must belong to its semantic performance.");if (!Number.isSafeInteger(options.transitionFrames) || options.transitionFrames < 1)throw new Error("Explainer transitionFrames must be a positive integer.");- const clips = projectSemanticMedia(semantic, window.span).filter(clip => clip.media.visual !== undefined);+ const clips = projectTimelineMedia(semantic, window.span).filter(clip => clip.media.visual !== undefined);const videos: VisualElement[] = clips.map((clip, i) => ({id: `take-${i}`, parent: "scene", kind: "video", order: i + 1,artifact: clip.media.visual!.artifact, muted: true,diff --git a/examples/semantic-composition/packages/sound-styles/README.md b/examples/semantic-composition/packages/sound-styles/README.mdnew file mode 100644index 00000000..39969457--- /dev/null+++ b/examples/semantic-composition/packages/sound-styles/README.md@@ -0,0 +1,36 @@+# Project sound Style: explicit crossfade++This ordinary project package demonstrates `soundStyle(fragment, bindings)`. It publishes one+`Crossfade` Style with explicit outgoing and incoming Segment inputs. It uses public+`@hypit/hypit/*` imports and returns ordinary AudioTrack data. Copy it into a video's `packages/`,+select the installed Hypit version as its development dependency, build it and declare it in the+project's ordinary package configuration.++```svml+<import as="sound" from="@hypit/sound@1"/>+<import as="mix" from="@example/sound-styles@1"/>+<sound:Style id="normal"/>+<mix:Crossfade id="handoff"+ outgoing={story.segment.first} incoming={story.segment.second}/>+<sound:Track id="voice" timeline={program.timeline}>+ <sound:Use style={normal}/>+ <sound:Use style={handoff}+ start-source={story.segment.second} start="segment.start"+ end-source={story.segment.first} end="segment.end"/>+</sound:Track>+```++The Window describes an actual overlap, even though its endpoints are reversed in Script order.+The two Style inputs declare roles; temporal endpoints do not infer them. Outgoing gain decreases+from 1 to 0 and incoming gain increases from 0 to 1 across that Window. Playback stays at each+Take's existing source position. A known source with no audio contributes nothing; an unknown source+is an error. Both roles must name distinct Segments.++Linear crossfade is this family's chosen behavior, not a universal acoustic rule. A different+project family can use another envelope or more sources without changing Timeline or the renderer.+For intentional simultaneous speech, a family can return both explicit `sourceSound` results at+unity gain. Independently included Sound Tracks can also coexist.++The implementation is in [render.ts](src/render.ts); [activation.ts](src/activation.ts) declares the+Surface, typed inputs, fragment and Producer. Build with `pnpm --filter @example/sound-styles build`+inside this repository after public declarations are available.diff --git a/examples/semantic-composition/packages/sound-styles/package.json b/examples/semantic-composition/packages/sound-styles/package.jsonnew file mode 100644index 00000000..66c3c125--- /dev/null+++ b/examples/semantic-composition/packages/sound-styles/package.json@@ -0,0 +1,25 @@+{+ "name": "@example/sound-styles",+ "version": "0.0.0-dev",+ "private": true,+ "type": "module",+ "hypit": {+ "activation": "./dist/activation.js"+ },+ "exports": {+ ".": "./dist/render.js"+ },+ "files": [+ "dist",+ "README.md"+ ],+ "scripts": {+ "build": "tsc -p tsconfig.json",+ "prepack": "npm run build"+ },+ "devDependencies": {+ "@hypit/hypit": "workspace:*",+ "typescript": "5.9.3",+ "@types/node": "24.10.1"+ }+}diff --git a/examples/semantic-composition/packages/sound-styles/src/activation.ts b/examples/semantic-composition/packages/sound-styles/src/activation.tsnew file mode 100644index 00000000..f4f23250--- /dev/null+++ b/examples/semantic-composition/packages/sound-styles/src/activation.ts@@ -0,0 +1,53 @@+import { assertAttributes, assertEmptyElement, canonicalize, createMarkupSurfaceHostFacet, sameType, sealGraphFragment, textAttribute } from "@hypit/hypit/author-kit";+import type { ComponentPackage, ModuleManifest, StructuredSurfaceHandler, SurfaceResolvedReference } from "@hypit/hypit/author-kit";+import { compositionTypes } from "@hypit/hypit/composition";+import { soundStyle, soundTypes, soundModuleRef } from "@hypit/hypit/sound";+import { timelineTypes } from "@hypit/hypit/timeline";+import type { Timeline } from "@hypit/hypit/timeline";+import { temporalTypes } from "@hypit/hypit/temporal";+import type { TemporalWindow } from "@hypit/hypit/temporal";+import { narrativeTypes } from "@hypit/hypit/narrative";+import type { NarrativeExcerpt } from "@hypit/hypit/narrative";+import { crossfadeSound } from "./render.js";++const module = { name: "@example/sound-styles", version: "1" } as const;+const common = [{ name: "timeline", type: timelineTypes.track }, { name: "window", type: temporalTypes.window }];+const mixInputs = [...common, { name: "outgoing", type: narrativeTypes.excerpt }, { name: "incoming", type: narrativeTypes.excerpt }];+const families = [{ name: "crossfade", tag: "Crossfade", ports: mixInputs }];+export const manifest: ModuleManifest = { format: "hypit.module@1", ...module,+ dependencies: [soundModuleRef, compositionTypes.audioTrack.module, timelineTypes.track.module, temporalTypes.window.module, narrativeTypes.excerpt.module].map(module => ({ module })),+ types: [], capabilities: [], producers: families.map(family => ({ name: family.name, inputs: family.ports, outputs: [{ name: "audio", type: compositionTypes.audioTrack }], needs: [] })) };+const fragments = families.map(family => sealGraphFragment({ inputs: family.ports,+ operations: [{ id: "render", producer: { module, name: family.name }, inputs: Object.fromEntries(family.ports.map(port => [port.name, { kind: "fragment-input" as const, name: port.name }])), result: { kind: "output", name: "audio" } }],+ exports: [{ name: "audio", type: compositionTypes.audioTrack, root: { kind: "fragment-operation", operation: "render" } }] }));+const component: ComponentPackage = { producers: families.map(family => ({ producer: { module, name: family.name }, handler: ({ inputs }) => {+ const values = Object.fromEntries(Object.entries(inputs).map(([key, record]) => {+ if (record.value.kind !== "inline") throw new Error(`${key} must be inline`);+ return [key, record.value.value];+ }));+ const audio = crossfadeSound(values.timeline as unknown as Timeline, values.window as unknown as TemporalWindow,+ values.outgoing as unknown as NarrativeExcerpt, values.incoming as unknown as NarrativeExcerpt);+ return { outputs: { audio: { kind: "inline", value: canonicalize(audio) } }, needs: {} };+} })) };+const hostFacets = families.map((family,index) => {+ const params = family.ports.slice(common.length);+ const handler: StructuredSurfaceHandler = ({ element, resolveReference }) => {+ assertAttributes(element, ["id", ...params.map(port => port.name)]); assertEmptyElement(element);+ const bindings: Record<string, SurfaceResolvedReference> = {};+ for (const param of params) {+ const raw = element.attributes[param.name];+ if (typeof raw !== "object" || raw.kind !== "reference") throw new Error(`${param.name} must be a reference`);+ const value = resolveReference(raw.path);+ if (value === undefined || !sameType(value.type,param.type)) throw new Error(`${param.name} has the wrong Type`);+ bindings[param.name] = value;+ }+ return { records: [{ id: textAttribute(element,"id"), type: soundTypes.style,+ value: { kind: "inline", value: canonicalize(soundStyle(fragments[index]!,bindings)) }, range: element.range }], components: [], fragments: [] };+ };+ return createMarkupSurfaceHostFacet({ module, declaration: { name: family.name, tag: family.tag, mode: "structured", outputs: [soundTypes.style], vocabulary: {+ summary: `Project-owned ${family.name} presentation Style.`, example: `<styles:${family.tag} id="look" ${params.map(port => `${port.name}={${port.name}}`).join(' ')}/>`,+ attributes: [{ name: "id", kind: "identifier", required: true, summary: "Style identity." }, ...params.map(port => ({ name: port.name, kind: "reference" as const, required: true, accepts: [port.type], summary: `Explicit ${port.name} input.` }))],+ } }, handler });+});+export const hypitPackage = { format: "hypit.node-package@1" as const, modules: [{ manifest }], components: [component], hostFacets };+export default hypitPackage;diff --git a/examples/semantic-composition/packages/sound-styles/src/render.ts b/examples/semantic-composition/packages/sound-styles/src/render.tsnew file mode 100644index 00000000..b7ca0bc7--- /dev/null+++ b/examples/semantic-composition/packages/sound-styles/src/render.ts@@ -0,0 +1,14 @@+import { sealAudioTrack } from "@hypit/hypit/composition";+import { sourceSound } from "@hypit/hypit/sound";+import type { Timeline } from "@hypit/hypit/timeline";+import type { TemporalWindow } from "@hypit/hypit/temporal";+import type { NarrativeExcerpt } from "@hypit/hypit/narrative";++/** The family explicitly chooses the two roles; the Window only controls the blend progress. */+export function crossfadeSound(timeline: Timeline, window: TemporalWindow,+ outgoing: NarrativeExcerpt, incoming: NarrativeExcerpt) {+ if (outgoing.id === incoming.id && outgoing.narrativeId === incoming.narrativeId) throw new Error("Crossfade needs two distinct sources.");+ const left = sourceSound(timeline, window, outgoing, { gain: 1, endGain: 0 });+ const right = sourceSound(timeline, window, incoming, { gain: 0, endGain: 1 });+ return sealAudioTrack({ id: window.subjectId, programSpaceId: timeline.id, clips: [...left.clips, ...right.clips] });+}diff --git a/examples/semantic-composition/packages/sound-styles/tsconfig.json b/examples/semantic-composition/packages/sound-styles/tsconfig.jsonnew file mode 100644index 00000000..6d5a18e7--- /dev/null+++ b/examples/semantic-composition/packages/sound-styles/tsconfig.json@@ -0,0 +1,18 @@+{+ "compilerOptions": {+ "target": "ES2023",+ "module": "NodeNext",+ "moduleResolution": "NodeNext",+ "lib": ["ES2023"],+ "strict": true,+ "noUncheckedIndexedAccess": true,+ "exactOptionalPropertyTypes": true,+ "verbatimModuleSyntax": true,+ "declaration": true,+ "sourceMap": true,+ "rootDir": "src",+ "outDir": "dist",+ "skipLibCheck": true+ },+ "include": ["src/**/*.ts"]+}diff --git a/package.json b/package.jsonindex c2c9f475..b0e31e3e 100644--- a/package.json+++ b/package.json@@ -58,9 +58,9 @@"types": "./dist/public/runtime-kit.d.ts","import": "./packages/runtime-kit/src/index.ts"},- "./semantic-track": {- "types": "./dist/public/semantic-track.d.ts",- "import": "./packages/semantic-track/src/index.ts"+ "./timeline": {+ "types": "./dist/public/timeline.d.ts",+ "import": "./packages/timeline/src/index.ts"},"./spatial": {"types": "./dist/public/spatial.d.ts",@@ -97,6 +97,14 @@"./hyperframes": {"types": "./dist/public/hyperframes.d.ts","import": "./packages/hyperframes/src/index.ts"+ },+ "./performance": {+ "types": "./dist/public/performance.d.ts",+ "import": "./packages/performance/src/index.ts"+ },+ "./sound": {+ "types": "./dist/public/sound.d.ts",+ "import": "./packages/sound/src/index.ts"}},"bin": {@@ -169,12 +177,12 @@"node": ">=22.12.0"},"devDependencies": {- "@hypit/browser-capture": "workspace:*","@hypit/artifact": "workspace:*","@hypit/audio-track": "workspace:*","@hypit/audio-track-studio": "workspace:*","@hypit/author-kit": "workspace:*","@hypit/background-removal": "workspace:*",+ "@hypit/browser-capture": "workspace:*","@hypit/build-result": "workspace:*","@hypit/build-result-fs": "workspace:*","@hypit/build-result-kit": "workspace:*",@@ -209,6 +217,9 @@"@hypit/minimax-h3": "workspace:*","@hypit/nano-banana": "workspace:*","@hypit/narrative": "workspace:*",+ "@hypit/performance": "workspace:*",+ "@hypit/performance-studio": "workspace:*",+ "@hypit/sound-studio": "workspace:*","@hypit/program-space": "workspace:*","@hypit/protocol": "workspace:*","@hypit/provider-hyperframes-aws-lambda": "workspace:*",@@ -236,19 +247,19 @@"@hypit/seedance": "workspace:*","@hypit/seedance-kits": "workspace:*","@hypit/seedream": "workspace:*",- "@hypit/semantic-track": "workspace:*",+ "@hypit/sound": "workspace:*","@hypit/spatial": "workspace:*","@hypit/speech": "workspace:*","@hypit/speech-alignment": "workspace:*","@hypit/speech-evidence": "workspace:*",- "@hypit/speech-track": "workspace:*",- "@hypit/speech-track-studio": "workspace:*","@hypit/store-sqlite": "workspace:*","@hypit/studio-adapter": "workspace:*","@hypit/svs": "workspace:*","@hypit/temporal": "workspace:*","@hypit/temporal-markup": "workspace:*","@hypit/text": "workspace:*",+ "@hypit/timeline": "workspace:*",+ "@hypit/timeline-author": "workspace:*","@hypit/typography-track": "workspace:*","@hypit/typography-track-studio": "workspace:*","@hypit/visual-ir": "workspace:*",diff --git a/packages/audio-track-studio/README.md b/packages/audio-track-studio/README.mdindex fbaef546..c13735fb 100644--- a/packages/audio-track-studio/README.md+++ b/packages/audio-track-studio/README.md@@ -1,3 +1,26 @@# `@hypit/audio-track-studio`Hypit Studio Companion for `@hypit/audio-track`. The domain package does not depend on this package.++Each Item has a title and waveform body. An explicit author `id` supplies its+title; otherwise the Companion displays the exact `source` reference. Display+names do not replace the Item's identity or writeback endpoint. This is the same+rule as Media Items. For example, `id="soundtrack" source={music.media}` shows+`soundtrack`; omitting the id shows `music.media`. Here `.media` is the source+output port, not an Audio-specific name suffix.++When groups placement, source trim, playback rates and fades. How contains gain.+Gain displays as a percentage (0–6400%) and writes back as a scalar; duration controls preserve their authored `f`, `ms` or `s` unit.+Omitted gain and fade defaults remain editable; the first edit writes an explicit+attribute. Playback uses the standard parameter controls and saves when editing ends. Stretch also+asks for minimum and maximum rate multipliers (1 means original speed). Switching+to once/loop removes these bounds in the same edit. The Surface retains final+validation of positive, ordered rates. No arbitrary bounds are supplied for the author.++The Companion declares a schema-described `attributes` group. Studio provides+record alternatives, completion hints and grouped source serialization without knowing+any Audio mode names.++Selecting an overlapping Item raises it in the editing lane while all sounds+continue to mix. Timing gestures use the same temporal projection bindings as+Media Track.diff --git a/packages/audio-track-studio/src/index.ts b/packages/audio-track-studio/src/index.tsindex e1778f69..22a90c30 100644--- a/packages/audio-track-studio/src/index.ts+++ b/packages/audio-track-studio/src/index.ts@@ -1,8 +1,8 @@-import { audioTrackModuleRef, audioTrackTypes } from "@hypit/audio-track";+import { audioItemDefaults, audioTrackModuleRef, audioTrackTypes } from "@hypit/audio-track";import type { AudioTrackProgram } from "@hypit/audio-track";import { compositionTypes } from "@hypit/composition";import type { StudioTrackCompanion, StudioTrackCompanionContext, StudioEntityDraft } from "@hypit/studio-adapter";-import { artifactPreview, childEntities, previewLayer, requiredSurfaceValue, temporalLineageFor, temporalSemanticSource } from "@hypit/studio-adapter";+import { artifactPreview, authoredItemTitle, childEntities, previewLayer, requiredSurfaceValue, temporalLineageFor, temporalSemanticSource } from "@hypit/studio-adapter";function projectAudio(context: StudioTrackCompanionContext): readonly StudioEntityDraft[] {const program = requiredSurfaceValue(context, "program") as AudioTrackProgram;@@ -21,7 +21,10 @@ function projectAudio(context: StudioTrackCompanionContext): readonly StudioEnticonst semanticSource = temporalSemanticSource(temporal);return {...entity,- display: { ...entity.display, layers: [previewLayer(item.preview, "waveform")] },+ display: {+ title: authoredItemTitle(context, entity.authoredId, item.sourceTypes, ["source"]),+ layers: [previewLayer(item.preview, "waveform")],+ },...(semanticSource?.id === undefined? {}: { markerId: semanticSource.id }),@@ -39,30 +42,38 @@ export const audioTrackStudioTrackCompanions: readonly StudioTrackCompanion[] ={ name: "source" },{ name: "trim-start", writable: true },{ name: "trim-end", writable: true },- { name: "playback", writable: true },- { name: "min-rate", writable: true },- { name: "max-rate", writable: true },- { name: "gain", writable: true },- { name: "fade-in", writable: true },- { name: "fade-out", writable: true },+ { name: "playback-settings", writable: true,+ attributes: ["playback", "min-rate", "max-rate"],+ fallback: { playback: audioItemDefaults.playback },+ schema: { kind: "oneOf", variants: ["once", "once-start", "once-end", "loop", "loop-start", "loop-end", "stretch"].map(mode => ({+ kind: "object", fields: {+ playback: { schema: { kind: "literal", value: mode } },+ ...(mode === "stretch" ? {+ "min-rate": { schema: { kind: "number", minimum: 0, maximum: 100 } },+ "max-rate": { schema: { kind: "number", minimum: 0, maximum: 100 } },+ } : {}),+ },+ })) },+ },+ { name: "gain", writable: true, fallback: audioItemDefaults.gain },+ { name: "fade-in", writable: true, fallback: audioItemDefaults["fade-in"] },+ { name: "fade-out", writable: true, fallback: audioItemDefaults["fade-out"] },],inspector: [{- binding: "playback", label: "Playback", domain: "when",+ binding: "playback-settings", label: "Playback", domain: "when",page: { id: "playback", label: "Playback" }, section: { id: "playback", label: "Playback" },- control: "select", options: ["once", "once-start", "once-end", "loop", "loop-start", "loop-end", "stretch"],+ control: "record",+ summary: "Rates are multipliers (1 = original speed). Stretch requires both rate bounds.",},...(["trim-start", "trim-end"] as const).map((binding) => ({binding, label: binding === "trim-start" ? "Trim Start" : "Trim End", domain: "when" as const,page: { id: "playback", label: "Playback" }, section: { id: "trim", label: "Trim" }, control: "number" as const,number: { suffixes: ["ms", "s", "f"], minimum: 0 },})),- ...(["min-rate", "max-rate", "gain"] as const).map((binding) => ({- binding, label: binding === "min-rate" ? "Minimum Rate" : binding === "max-rate" ? "Maximum Rate" : "Gain",- domain: "how" as const, page: { id: "mix", label: "Mix" }, section: { id: "mix", label: "Mix" },- control: "number" as const,- unit: "%", number: { scale: 100, minimum: 0, step: 1 },- })),+ { binding: "gain", label: "Gain", domain: "how",+ page: { id: "mix", label: "Mix" }, section: { id: "mix", label: "Mix" },+ control: "number", unit: "%", number: { scale: 100, minimum: 0, maximum: 6400, step: 1 } },...(["fade-in", "fade-out"] as const).map((binding) => ({binding, label: binding === "fade-in" ? "Fade In" : "Fade Out", domain: "when" as const,page: { id: "fade", label: "Fade" }, section: { id: "fade", label: "Fade" }, control: "number" as const,diff --git a/packages/audio-track/README.md b/packages/audio-track/README.mdindex 22b570fe..d2c39750 100644--- a/packages/audio-track/README.md+++ b/packages/audio-track/README.md@@ -1,37 +1,37 @@# `@hypit/audio-track`-The Track Surface accepts `semantic={speech.semantic}` or `space={animation}`. Semantic context-resolves Script Selections and Moments; authored space supports clock-based animation. Shared `at`+The Track Surface accepts `timeline={program.timeline}`. The same Timeline supports authored+positions and, where prepared Takes supply evidence, Script Selections and Moments. Shared `at`inputs accept a Moment or a time such as `2s`; `at` with `for` produces a Window where required.Place music, ambience and sound effects in the assembled video. A reveal sound can follow a ScriptMoment; music can occupy a Segment or the program; an explicitly timed sound can use clock placement.-Performance audio normally comes from [Speech Track](../speech-track/README.md) and can continue+Performance audio is presented by [Sound](../sound/README.md) from the assembled Timeline and can continueunder these independent sounds and [Media Track](../media-track/README.md) coverage.-Each Clip consumes normalized audio-bearing media and a projected Window. Gain, fades, trim and+Each Item consumes normalized audio-bearing media and a projected Window. Gain, fades, trim andplayback express the mix. Normalization preserves the source level; choose the balance and any-music ducking for the actual performance. The resulting `.track` is a peer AudioTrack in Film.+music ducking for the actual performance. The resulting `.audio` is a peer AudioTrack in Film.-## Authoring clips+## Authoring items-The Markup Track selects `semantic` or `space`; its Fragment receives the resulting ProgramSpace.-Each Clip accepts one shared temporal Window form: `during="program"`, a Selection or Segment,+The Markup Track selects `timeline`; its Fragment receives the same Timeline.+Each Item accepts one shared temporal Window form: `during="program"`, a Selection or Segment,a Moment paired with `for`, or explicit `start`/`end` expressions. The source must already contain-normalized audio. This excerpt assumes imports, Script, SemanticTrack, Clock and source media exist:+normalized audio. This excerpt assumes imports, Script, Timeline, Clock and source media exist:```svml<pipeline:Normalize id="hit-media" source={hit-source}video="none" audio="default" span-authority="audio" clock={clock}/>-<audio:Track id="effects" semantic={speech.semantic}>- <audio:Clip source={hit-media.media} at={story.moment.reveal} for="600ms"+<audio:Track id="effects" timeline={speech.timeline}>+ <audio:Item source={hit-media.media} at={story.moment.reveal} for="600ms"playback="once" gain="0.45" fade-in="0f" fade-out="3f"/></audio:Track>```-The Track publishes `.program` and `.track`; include the latter as a peer Film Track. `gain` is a+The Track publishes `.program` and `.audio`; include the latter as a peer Film Track. `gain` is alinear multiplier, defaulting to `1`; fade durations default to `0f`. Trim boundaries are source-durations (`f`, integer `ms`, or fractional `s`), distinct from the Clip's destination Window.+durations (`f`, integer `ms`, or fractional `s`), distinct from the Item's destination Window.`trim-end` is the exclusive source endpoint, not an amount to subtract from the tail. For example,`trim-start="1s" trim-end="3s"` selects the source interval from one to three seconds.@@ -41,3 +41,10 @@ requires authored `min-rate` and `max-rate` bounds. Other playback modes rejectChoose occupancy from the intended sound; the Track does not silently loop music or stretch a hitto fill a longer Window. Its source level is retained unless gain or another explicit audio operationchanges it.++Media Track and Audio Track both place independent Items on the same Timeline.+Overlapping pictures compose by draw order; overlapping sounds mix together.+Give an Item an `id` when it needs a recognizable author name. Studio otherwise+displays its source reference; the generated internal identity still owns+selection and writeback. Reusing a source in two Items keeps two independent+placements, even when their displayed names match.diff --git a/packages/audio-track/package.json b/packages/audio-track/package.jsonindex 1de78800..09d1b7d6 100644--- a/packages/audio-track/package.json+++ b/packages/audio-track/package.json@@ -18,7 +18,7 @@"@hypit/narrative": "workspace:*","@hypit/program-space": "workspace:*","@hypit/protocol": "workspace:*",- "@hypit/semantic-track": "workspace:*",+ "@hypit/timeline": "workspace:*","@hypit/temporal": "workspace:*","@hypit/temporal-markup": "workspace:*","@hypit/markup": "workspace:*"diff --git a/packages/audio-track/src/component.ts b/packages/audio-track/src/component.tsindex 75b3e707..bb2db021 100644--- a/packages/audio-track/src/component.ts+++ b/packages/audio-track/src/component.ts@@ -1,8 +1,8 @@+import type { Timeline } from "@hypit/timeline";import type { ComponentPackage } from "@hypit/component-kit";import type { SynchronizedMedia } from "@hypit/media";import { canonicalize } from "@hypit/protocol";import type { StoredValue } from "@hypit/protocol";-import type { ProgramSpace } from "@hypit/program-space";import { audioTrackProducers, audioTrackTypes } from "./manifest.js";import { appendProjectedAudioItem, assertAudioTrackProgram, createAudioTrackSet, finalizeAudioTrack, renderAudioTrack } from "./program.js";@@ -15,14 +15,14 @@ function inline<T>(value: StoredValue | undefined, label: string): T {}const output = (value: unknown) => ({ kind: "inline" as const, value: canonicalize(value) });-const base = (inputs: [AudioTrackSet, AudioTrackHeader, ProgramSpace, SynchronizedMedia, AudioClipSpec, TemporalWindow]) => output(appendProjectedAudioItem(...inputs));+const base = (inputs: [AudioTrackSet, AudioTrackHeader, Timeline, SynchronizedMedia, AudioClipSpec, TemporalWindow]) => output(appendProjectedAudioItem(...inputs));export const audioTrackComponent = {producers: [{ producer: audioTrackProducers.createSet, handler: () => ({ outputs: { set: output(createAudioTrackSet()) }, needs: {} }) },{ producer: audioTrackProducers.appendItem, handler: ({ inputs }) => ({ outputs: { set: base([inline<AudioTrackSet>(inputs.set?.value, "AudioTrackSet"), inline<AudioTrackHeader>(inputs.header?.value, "AudioTrackHeader"),- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"), inline<SynchronizedMedia>(inputs.media?.value, "SynchronizedMedia"),+ inline<Timeline>(inputs.timeline?.value, "Timeline"), inline<SynchronizedMedia>(inputs.media?.value, "SynchronizedMedia"),inline<AudioClipSpec>(inputs.spec?.value, "AudioClipSpec"),inline<TemporalWindow>(inputs.window?.value, "TemporalWindow"),]) }, needs: {} }) },@@ -30,7 +30,7 @@ export const audioTrackComponent = {inline<AudioTrackSet>(inputs.set?.value, "AudioTrackSet"), inline<AudioTrackHeader>(inputs.header?.value, "AudioTrackHeader"),)) }, needs: {} }) },{ producer: audioTrackProducers.render, handler: ({ inputs }) => ({ outputs: { track: output(renderAudioTrack(- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"), inline<AudioTrackProgram>(inputs.program?.value, "AudioTrackProgram"),+ inline<Timeline>(inputs.timeline?.value, "Timeline"), inline<AudioTrackProgram>(inputs.program?.value, "AudioTrackProgram"),)) }, needs: {} }) },],validators: [{diff --git a/packages/audio-track/src/fragment.ts b/packages/audio-track/src/fragment.tsindex dcf88dd7..fb71107a 100644--- a/packages/audio-track/src/fragment.ts+++ b/packages/audio-track/src/fragment.ts@@ -1,4 +1,5 @@-import { programSpaceTypes } from "@hypit/program-space";+import { timelineTypes } from "@hypit/timeline";+import { compositionTypes } from "@hypit/composition";import { sealGraphFragment } from "@hypit/elaborator";import type { FragmentOperation } from "@hypit/elaborator";@@ -32,7 +33,7 @@ export function createAudioTrackFragment(items: readonly AudioTrackFragmentItem[id,producer: audioTrackProducers.appendItem,inputs: {- set: operation(current), header: input("header"), space: input("space"),+ set: operation(current), header: input("header"), timeline: input("timeline"),media: input(item.mediaName), spec: input(item.specName),window: input(item.windowName),},@@ -42,18 +43,18 @@ export function createAudioTrackFragment(items: readonly AudioTrackFragmentItem[});operations.push({ id: "audio:program", producer: audioTrackProducers.finalize, inputs: { set: operation(current), header: input("header") }, result: { kind: "output", name: "program" } },- { id: "audio:track", producer: audioTrackProducers.render, inputs: { space: input("space"), program: operation("audio:program") }, result: { kind: "output", name: "track" } },+ { id: "audio:track", producer: audioTrackProducers.render, inputs: { timeline: input("timeline"), program: operation("audio:program") }, result: { kind: "output", name: "track" } },);return sealGraphFragment({inputs: [{ name: "header", type: audioTrackTypes.header },- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },...[...inputTypes].map(([inputName, type]) => ({ name: inputName, type })),],operations,exports: [{ name: "program", type: audioTrackTypes.program, root: operation("audio:program") },- { name: "track", type: compositionTypes.audioTrack, root: operation("audio:track") },+ { name: "audio", type: compositionTypes.audioTrack, root: operation("audio:track") },],});}diff --git a/packages/audio-track/src/index.ts b/packages/audio-track/src/index.tsindex 2ccb7784..690f48bf 100644--- a/packages/audio-track/src/index.ts+++ b/packages/audio-track/src/index.ts@@ -2,6 +2,6 @@ export { audioTrackComponent } from "./component.js";export { createAudioTrackFragment, programAudioTrackFragment } from "./fragment.js";export type { AudioTrackFragmentItem } from "./fragment.js";export { audioClipSpecSchema, audioTrackDependency, audioTrackHeaderSchema, audioTrackManifest, audioTrackMarkupSurfaces, audioTrackModuleRef, audioTrackProducers, audioTrackProgramSchema, audioTrackSetSchema, audioTrackTypes } from "./manifest.js";-export { decodeAudioTrackSurface } from "./surface.js";+export { decodeAudioTrackSurface, audioItemDefaults } from "./surface.js";export { appendProjectedAudioItem, assertAudioClipSpec, assertAudioTrackHeader, assertAudioTrackProgram, assertAudioTrackSet, createAudioTrackSet, finalizeAudioTrack, renderAudioTrack, sealAudioClipSpec, sealAudioTrackHeader, sealAudioTrackProgram } from "./program.js";export type * from "./types.js";diff --git a/packages/audio-track/src/manifest.ts b/packages/audio-track/src/manifest.tsindex 83c50c36..08197648 100644--- a/packages/audio-track/src/manifest.ts+++ b/packages/audio-track/src/manifest.ts@@ -1,11 +1,12 @@+import { timelineTypes } from "@hypit/timeline";import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";import { compositionDependency, compositionTypes } from "@hypit/composition";import { mediaDependency, mediaTypes } from "@hypit/media";import { narrativeDependency } from "@hypit/narrative";-import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";+import { blobRefObjectSchema } from "@hypit/protocol";import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";-import { semanticTrackDependency } from "@hypit/semantic-track";+import { timelineDependency } from "@hypit/timeline";import { temporalDependency, temporalTypes } from "@hypit/temporal";import { temporalWindowAttributeVocabulary } from "@hypit/temporal-markup";@@ -68,7 +69,7 @@ export const audioTrackProgramSchema: ValueSchema = object({const baseInputs = [{ name: "set", type: audioTrackTypes.set }, { name: "header", type: audioTrackTypes.header },- { name: "space", type: programSpaceTypes.programSpace }, { name: "media", type: mediaTypes.synchronized },+ { name: "timeline", type: timelineTypes.track }, { name: "media", type: mediaTypes.synchronized },{ name: "spec", type: audioTrackTypes.clipSpec }, { name: "window", type: temporalTypes.window },] as const;@@ -76,20 +77,20 @@ export const audioTrackMarkupSurfaces = [{name: "track", tag: "Track", mode: "structured",outputs: [audioTrackTypes.header, audioTrackTypes.clipSpec, temporalTypes.instantSpec, temporalTypes.windowSpec, temporalTypes.instant, temporalTypes.window, audioTrackTypes.program, compositionTypes.audioTrack],vocabulary: {- summary: "One Audio Track: explicitly prepared audio Clips placed on the selected film time axis and lowered to one ordinary peer AudioTrack.",+ summary: "One Audio Track: explicitly prepared audio Items placed on the selected film time axis and lowered to one ordinary peer AudioTrack.",attributes: [- { name: "id", kind: "identifier", required: true, summary: "Names this Audio Track and prefixes the identity of every Clip that does not name itself." },+ { name: "id", kind: "identifier", required: true, summary: "Names this Audio Track and prefixes the identity of every Item that does not name itself." },...temporalContextAttributeVocabulary,],children: [- { tag: "Clip", cardinality: "many",+ { tag: "Item", cardinality: "many",summary: "Places one SynchronizedMedia source on an exact program window with its own trim, occupancy and mix.",attributes: [{ name: "id", kind: "identifier", required: false,- summary: "Names this Clip; the Track derives `<track>.clip.<index>` when it is absent." },+ summary: "Names this Item; the Track derives `<track>.item.<index>` when it is absent." },{ name: "source", kind: "reference", required: true,accepts: [mediaTypes.synchronized],- summary: "Selects the explicitly prepared audio this Clip plays." },+ summary: "Selects the explicitly prepared audio this Item plays." },...temporalWindowAttributeVocabulary,{ name: "trim-start", kind: "literal", required: false,summary: "Sets the source start position; defaults to the source beginning." },@@ -103,7 +104,7 @@ export const audioTrackMarkupSurfaces = [{{ name: "max-rate", kind: "literal", required: false,summary: "Bounds the fastest rate bounded stretch may use." },{ name: "gain", kind: "literal", required: false,- summary: "Scales this Clip by a linear gain; defaults to `1`." },+ summary: "Scales this Item by a linear gain; defaults to `1`." },{ name: "fade-in", kind: "literal", required: false,summary: "Fixes the exact fade-in length; defaults to `0f`." },{ name: "fade-out", kind: "literal", required: false,@@ -112,15 +113,15 @@ export const audioTrackMarkupSurfaces = [{],ports: [{ name: "program", type: audioTrackTypes.program, summary: "The resolved sample-exact item list this Track renders from." },- { name: "track", type: compositionTypes.audioTrack, summary: "The rendered AudioTrack that Film composes with its peers." },+ { name: "audio", type: compositionTypes.audioTrack, summary: "The rendered AudioTrack that Film composes with its peers." },],- example: `<audio:Track id="music-bed" semantic={speech.semantic}>- <audio:Clip source={music-media.media} during="program"+ example: `<audio:Track id="music-bed" timeline={speech.timeline}>+ <audio:Item source={music-media.media} during="program"playback="loop-end" gain="0.28" fade-in="600ms" fade-out="800ms"/></audio:Track>`,notes: [- "A Track requires at least one Clip, and neither a Track nor a Clip accepts text content.",- "A Clip states exactly one window form: `during`, `at` with `for`, `until` with `for`, or `start` with `end`.",+ "A Track requires at least one Item, and neither a Track nor a Item accepts text content.",+ "A Item states exactly one window form: `during`, `at` with `for`, `until` with `for`, or `start` with `end`.","A point expression is `program.start`, `program.end`, `selection.start`, `selection.end`, `segment.start`, `segment.end` or `moment.cue`, each optionally offset by `+` or `-` and a duration, or a bare duration read as an absolute position.","Bind selection, segment and/or moment only when the start/end expressions use them; different endpoints can use different bindings.","`min-rate` and `max-rate` are rejected unless `playback` is `stretch`.",@@ -133,7 +134,7 @@ export const audioTrackManifest: ModuleManifest = {format: "hypit.module@1",name: audioTrackModuleRef.name,version: audioTrackModuleRef.version,- dependencies: [mediaDependency, narrativeDependency, programSpaceDependency, semanticTrackDependency, temporalDependency, compositionDependency],+ dependencies: [mediaDependency, narrativeDependency, timelineDependency, temporalDependency, compositionDependency],types: [{ name: audioTrackTypes.header.name },{ name: audioTrackTypes.clipSpec.name },@@ -145,7 +146,7 @@ export const audioTrackManifest: ModuleManifest = {{ name: audioTrackProducers.createSet.name, inputs: [], outputs: [{ name: "set", type: audioTrackTypes.set }], needs: [] },{ name: audioTrackProducers.appendItem.name, inputs: [...baseInputs], outputs: [{ name: "set", type: audioTrackTypes.set }], needs: [] },{ name: audioTrackProducers.finalize.name, inputs: [{ name: "set", type: audioTrackTypes.set }, { name: "header", type: audioTrackTypes.header }], outputs: [{ name: "program", type: audioTrackTypes.program }], needs: [] },- { name: audioTrackProducers.render.name, inputs: [{ name: "space", type: programSpaceTypes.programSpace }, { name: "program", type: audioTrackTypes.program }], outputs: [{ name: "track", type: compositionTypes.audioTrack }], needs: [] },+ { name: audioTrackProducers.render.name, inputs: [{ name: "timeline", type: timelineTypes.track }, { name: "program", type: audioTrackTypes.program }], outputs: [{ name: "track", type: compositionTypes.audioTrack }], needs: [] },],};export const audioTrackDependency = { module: audioTrackModuleRef } as const;diff --git a/packages/audio-track/src/program.ts b/packages/audio-track/src/program.tsindex d6ecae2e..fbb6f2eb 100644--- a/packages/audio-track/src/program.ts+++ b/packages/audio-track/src/program.ts@@ -1,13 +1,9 @@+import type { Timeline } from "@hypit/timeline";import { assertAudioTrackIdentity, sealAudioTrack } from "@hypit/composition";import type { AudioClip, AudioTrack } from "@hypit/composition";import { synchronizedMediaSampleFrames, verifySynchronizedMedia } from "@hypit/media";import type { SynchronizedMedia } from "@hypit/media";-import {- assertProgramSpaceIdentity,- programFrameSampleBoundary,- programSpaceSampleFrames,-} from "@hypit/program-space";-import type { ProgramSpace } from "@hypit/program-space";+import { assertProgramSpaceIdentity, programFrameSampleBoundary, programSpaceSampleFrames } from "@hypit/program-space";import { canonicalize, isResourceId } from "@hypit/protocol";import { assertTemporalWindowFor, temporalDurationInSamples } from "@hypit/temporal";import type { ProjectedWindow, TemporalDuration } from "@hypit/temporal";@@ -94,23 +90,23 @@ function sourceFacts(media: SynchronizedMedia): AudioItemProgram["source"] {function realizedItems(set: AudioTrackSet,header: AudioTrackHeader,- space: ProgramSpace,+ timeline: Timeline,media: SynchronizedMedia,spec: AudioClipSpec,window: ProjectedWindow,): AudioTrackSet {assertAudioTrackSet(set);assertAudioTrackHeader(header);- assertProgramSpaceIdentity(space);+ assertProgramSpaceIdentity(timeline);assertAudioClipSpec(spec);- assertTemporalWindowFor(window, { subjectId: spec.id, space });+ assertTemporalWindowFor(window, { subjectId: spec.id, space: timeline });const source = sourceFacts(media);- const trimStart = spec.trim.start === undefined ? 0 : temporalDurationInSamples(spec.trim.start, space);- const trimEnd = spec.trim.end === undefined ? source.sampleFrames : temporalDurationInSamples(spec.trim.end, space);+ const trimStart = spec.trim.start === undefined ? 0 : temporalDurationInSamples(spec.trim.start, timeline);+ const trimEnd = spec.trim.end === undefined ? source.sampleFrames : temporalDurationInSamples(spec.trim.end, timeline);assert(trimStart >= 0 && trimStart < source.sampleFrames, `Audio Clip ${spec.id} trim start is outside its source.`);assert(trimEnd > trimStart && trimEnd <= source.sampleFrames, `Audio Clip ${spec.id} trim end is outside its source.`);- const fadeInSamples = temporalDurationInSamples(spec.mix.fadeIn, space);- const fadeOutSamples = temporalDurationInSamples(spec.mix.fadeOut, space);+ const fadeInSamples = temporalDurationInSamples(spec.mix.fadeIn, timeline);+ const fadeOutSamples = temporalDurationInSamples(spec.mix.fadeOut, timeline);const addition = {id: window.id,subjectId: spec.id,@@ -129,12 +125,12 @@ function realizedItems(export function appendProjectedAudioItem(set: AudioTrackSet,header: AudioTrackHeader,- space: ProgramSpace,+ timeline: Timeline,media: SynchronizedMedia,spec: AudioClipSpec,window: ProjectedWindow,): AudioTrackSet {- return realizedItems(set, header, space, media, spec, window);+ return realizedItems(set, header, timeline, media, spec, window);}function normalizeProgram(value: AudioTrackProgram): AudioTrackProgram {@@ -183,9 +179,9 @@ export function assertAudioTrackProgram(value: AudioTrackProgram): void {}}-function terminalClip(item: AudioItemProgram, space: ProgramSpace): AudioClip {- const windowStart = programFrameSampleBoundary(space, item.window.startFrame, 48_000);- const windowEnd = programFrameSampleBoundary(space, item.window.endFrameExclusive, 48_000);+function terminalClip(item: AudioItemProgram, timeline: Timeline): AudioClip {+ const windowStart = programFrameSampleBoundary(timeline, item.window.startFrame, 48_000);+ const windowEnd = programFrameSampleBoundary(timeline, item.window.endFrameExclusive, 48_000);const windowLength = windowEnd - windowStart;const effectiveLength = item.trim.endSampleExclusive - item.trim.startSample;let targetStart = windowStart;@@ -237,18 +233,18 @@ function terminalClip(item: AudioItemProgram, space: ProgramSpace): AudioClip {};}-export function renderAudioTrack(space: ProgramSpace, program: AudioTrackProgram): AudioTrack {- assertProgramSpaceIdentity(space);+export function renderAudioTrack(timeline: Timeline, program: AudioTrackProgram): AudioTrack {+ assertProgramSpaceIdentity(timeline);assertAudioTrackProgram(program);- const totalSamples = programSpaceSampleFrames(space, 48_000);+ const totalSamples = programSpaceSampleFrames(timeline, 48_000);const track = sealAudioTrack({- programSpaceId: space.id,+ programSpaceId: timeline.id,id: program.id,- clips: program.items.map((item) => terminalClip(item, space)),+ clips: program.items.map((item) => terminalClip(item, timeline)),});for (const clip of track.clips) {- assert(clip.target.endSampleExclusive <= totalSamples, `Audio Clip ${clip.id} is outside ProgramSpace.`);+ assert(clip.target.endSampleExclusive <= totalSamples, `Audio Clip ${clip.id} is outside Timeline.`);}- assertAudioTrackIdentity(track, space);+ assertAudioTrackIdentity(track, timeline);return track;}diff --git a/packages/audio-track/src/surface.ts b/packages/audio-track/src/surface.tsindex d7639bd2..c63af5d1 100644--- a/packages/audio-track/src/surface.ts+++ b/packages/audio-track/src/surface.ts@@ -1,4 +1,4 @@-import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";+import { resolveTemporalContext } from "@hypit/temporal-markup";import { mediaTypes } from "@hypit/media";import type {StructuredElement,@@ -85,8 +85,10 @@ function numeric(element: StructuredElement, name: string, fallback?: number): nreturn value;}+export const audioItemDefaults = { playback: "once", gain: 1, "fade-in": "0f", "fade-out": "0f" } as const;+function occupancy(element: StructuredElement): AudioOccupancy {- switch (text(element, "playback", "once")) {+ switch (text(element, "playback", audioItemDefaults.playback)) {case "once":case "once-start": return { mode: "once", align: "start" };case "once-end": return { mode: "once", align: "end" };@@ -104,10 +106,9 @@ function occupancy(element: StructuredElement): AudioOccupancy {}export const decodeAudioTrackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {- allowed(element, ["id", "semantic", "space"]);+ allowed(element, ["id", "timeline"]);const id = text(element, "id");const context = resolveTemporalContext({ element, resolveReference });- const time = createTemporalSpace({ id: id, element, ...context });const headerId = `${id}.header`;const records: SurfaceRecordDraft[] = [{id: headerId,@@ -115,20 +116,20 @@ export const decodeAudioTrackSurface: StructuredSurfaceHandler = ({ element, resvalue: { kind: "inline", value: sealAudioTrackHeader({ id }) },range: element.range,}];- const temporalComponents: SurfaceComponentDraft[] = [...time.components];- const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [...time.fragments];+ const temporalComponents: SurfaceComponentDraft[] = [];+ const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [];const fragmentItems: Parameters<typeof createAudioTrackFragment>[0][number][] = [];const inputs: Record<string, { kind: "record"; id: string } | { kind: "component-output"; component: string; output: string }> = {header: { kind: "record", id: headerId },- space: time.space.ref,+ timeline: context.timeline.ref,};let itemIndex = 0;for (const child of element.children) {if (child.kind === "text") {- if (child.value.trim()) throw new Error(`${element.name} accepts only Clip children.`);+ if (child.value.trim()) throw new Error(`${element.name} accepts only Item children.`);continue;}- if (!child.name.endsWith(":Clip") && child.name !== "Clip") throw new Error(`${element.name} accepts only Clip children.`);+ if (!child.name.endsWith(":Item") && child.name !== "Item") throw new Error(`${element.name} accepts only Item children.`);if (child.children.some((node) => node.kind === "element" || node.value.trim())) throw new Error(`${child.name} must be empty.`);allowed(child, ["id", "source", ...temporalWindowAttributeNames,@@ -136,9 +137,9 @@ export const decodeAudioTrackSurface: StructuredSurfaceHandler = ({ element, res]);itemIndex += 1;const suffix = String(itemIndex).padStart(4, "0");- const clipId = optionalText(child, "id") ?? `${id}.clip.${suffix}`;+ const clipId = optionalText(child, "id") ?? `${id}.item.${suffix}`;const source = resolved(child.attributes.source, `${child.name}.source`, mediaTypes.synchronized, resolveReference);- const temporal = createTemporalWindowProjection({ id: clipId, element: child, ...context, space: time.space, resolveReference });+ const temporal = createTemporalWindowProjection({ id: clipId, element: child, ...context, resolveReference });records.push(...temporal.records);temporalComponents.push(...temporal.components);temporalFragments.push(...temporal.fragments);@@ -155,22 +156,22 @@ export const decodeAudioTrackSurface: StructuredSurfaceHandler = ({ element, res},occupancy: playback,mix: {- gain: numeric(child, "gain", 1),- fadeIn: duration(text(child, "fade-in", "0f"), `${child.name}.fade-in`),- fadeOut: duration(text(child, "fade-out", "0f"), `${child.name}.fade-out`),+ gain: numeric(child, "gain", audioItemDefaults.gain),+ fadeIn: duration(text(child, "fade-in", audioItemDefaults["fade-in"]), `${child.name}.fade-in`),+ fadeOut: duration(text(child, "fade-out", audioItemDefaults["fade-out"]), `${child.name}.fade-out`),},});const windowName = `item-${suffix}-window`;inputs[windowName] = temporal.ref;const mediaName = `item-${suffix}-media`;const specName = `item-${suffix}-spec`;- const specId = `${id}.clip.${suffix}.spec`;+ const specId = `${id}.item.${suffix}.spec`;records.push({ id: specId, type: audioTrackTypes.clipSpec, value: { kind: "inline", value: clipSpec }, range: child.range });inputs[mediaName] = source.ref;inputs[specName] = { kind: "record", id: specId };fragmentItems.push({ mediaName, specName, windowName });}- if (fragmentItems.length === 0) throw new Error(`${element.name} requires at least one Clip.`);+ if (fragmentItems.length === 0) throw new Error(`${element.name} requires at least one Item.`);const fragment = createAudioTrackFragment(fragmentItems);return {records,@@ -178,10 +179,10 @@ export const decodeAudioTrackSurface: StructuredSurfaceHandler = ({ element, resid,fragment: fragment.id,inputs,- outputs: { program: `${id}.program`, track: `${id}.track` },+ outputs: { program: `${id}.program`, audio: `${id}.audio` },range: element.range,}],fragments: [...temporalFragments, fragment],- exports: [`${id}.program`, `${id}.track`],+ exports: [`${id}.program`, `${id}.audio`],};};diff --git a/packages/audio-track/test/audio-track.test.ts b/packages/audio-track/test/audio-track.test.tsindex da17e58b..54d6244f 100644--- a/packages/audio-track/test/audio-track.test.ts+++ b/packages/audio-track/test/audio-track.test.ts@@ -1,7 +1,8 @@+import { sealTimeline } from "@hypit/timeline";import assert from "node:assert/strict";import test from "node:test";import { fixtureResource } from "../../../test/fixture-resource.js";-import { semanticTrackFixture } from "../../../test/semantic-track-fixture.js";+import { timelineFixture } from "../../../test/timeline-fixture.js";import { projectMomentWindow, projectProgramWindow, projectSelectionWindow } from "../../../test/temporal-fixture.js";import type { TemporalWindowProjection } from "../../../test/temporal-fixture.js";@@ -32,7 +33,7 @@ import { narrativeManifest } from "@hypit/narrative";import { compileAudioProgramPlan } from "@hypit/media-pipeline";import { programSpaceDependency, programSpaceManifest, programSpaceTypes, sealProgramSpace } from "@hypit/program-space";import type { ModuleManifest } from "@hypit/protocol";-import { projectSemanticProgramSpace, semanticTrackDependency, semanticTrackManifest, semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";+import { projectTimelineSpace, timelineDependency, timelineManifest, timelineProducers, timelineTypes } from "@hypit/timeline";import { temporalProducers } from "@hypit/temporal";import { speechEvidenceManifest } from "@hypit/speech-evidence";import { speechManifest } from "@hypit/speech";@@ -43,11 +44,11 @@ import { MarkupSurfaceRegistry, createMarkupAuthorFrontend } from "@hypit/markupimport { createRecordAdmitter, TypeValidatorRegistry } from "@hypit/validation";import { visualIrManifest } from "@hypit/visual-ir";-const space = sealProgramSpace({ id: "test-space", durationSec: 10,+const space = sealTimeline({ items: [], id: "test-space", durationSec: 10,frameRate: { numerator: 30, denominator: 1 },});const header = sealAudioTrackHeader({ id: "sound" });-const semantic = semanticTrackFixture(space, { narrativeId: "test-narrative", anchors: [+const semantic = timelineFixture(space, { narrativeId: "test-narrative", anchors: [{ identity: "a", frame: 30 }, { identity: "b", frame: 60 },{ identity: "c", frame: 90 }, { identity: "d", frame: 120 },] });@@ -87,32 +88,32 @@ function spec(overrides: Partial<TestAudioClipSpec> = {}): TestAudioClipSpec {}function appendProgramAudioItem(- set: AudioTrackSet, trackHeader: typeof header, semanticTrack: typeof semantic,+ set: AudioTrackSet, trackHeader: typeof header, timeline: typeof semantic,source: SynchronizedMedia, authored: TestAudioClipSpec,): AudioTrackSet {const { projection, ...clip } = authored;- return appendProjectedAudioItem(set, trackHeader, projectSemanticProgramSpace(semanticTrack), source, clip, projectProgramWindow({- itemId: clip.id, semantic: semanticTrack, projection,+ return appendProjectedAudioItem(set, trackHeader, timeline, source, clip, projectProgramWindow({+ itemId: clip.id, semantic: timeline, projection,}));}function appendSelectionAudioItem(- set: AudioTrackSet, trackHeader: typeof header, semanticTrack: typeof semantic,+ set: AudioTrackSet, trackHeader: typeof header, timeline: typeof semantic,source: SynchronizedMedia, selection: NarrativeSelectionRef, authored: TestAudioClipSpec,): AudioTrackSet {const { projection, ...clip } = authored;- return appendProjectedAudioItem(set, trackHeader, projectSemanticProgramSpace(semanticTrack), source, clip, projectSelectionWindow({- itemId: clip.id, semantic: semanticTrack, selection, projection,+ return appendProjectedAudioItem(set, trackHeader, timeline, source, clip, projectSelectionWindow({+ itemId: clip.id, semantic: timeline, selection, projection,}));}function appendMomentAudioItem(- set: AudioTrackSet, trackHeader: typeof header, semanticTrack: typeof semantic,+ set: AudioTrackSet, trackHeader: typeof header, timeline: typeof semantic,source: SynchronizedMedia, moment: NarrativeMomentRef, authored: TestAudioClipSpec,): AudioTrackSet {const { projection, ...clip } = authored;- return appendProjectedAudioItem(set, trackHeader, projectSemanticProgramSpace(semanticTrack), source, clip, projectMomentWindow({- itemId: clip.id, semantic: semanticTrack, moment, projection,+ return appendProjectedAudioItem(set, trackHeader, timeline, source, clip, projectMomentWindow({+ itemId: clip.id, semantic: timeline, moment, projection,}));}@@ -248,9 +249,9 @@ test("dynamic Fragment keeps every material and temporal dependency as an explic{ mediaName: "impact", specName: "impact-spec", windowName: "impact-window" },]);assert.deepEqual(fragment.inputs.map((input) => input.name), [- "header", "impact", "impact-spec", "impact-window", "music", "music-spec", "music-window", "space", "voice", "voice-spec", "voice-window",+ "header", "impact", "impact-spec", "impact-window", "music", "music-spec", "music-window", "timeline", "voice", "voice-spec", "voice-window",]);- assert.equal(fragment.exports[1]?.name, "track");+ assert.ok(fragment.exports.some((output) => output.name === "audio" && output.type.name === compositionTypes.audioTrack.name));});test("the self-described Audio Surface parses into the same finite Producer graph", async () => {@@ -258,13 +259,13 @@ test("the self-described Audio Surface parses into the same finite Producer grapconst fixtureSurfaceDigest = fixtureResource("example.audio-inputs/surface@1");const fixtureSurface = {name: "inputs", tag: "Inputs", mode: "structured",- outputs: [mediaTypes.synchronized, semanticTrackTypes.track],+ outputs: [mediaTypes.synchronized, timelineTypes.track],} as const;const fixtureManifest: ModuleManifest = {format: "hypit.module@1",name: fixtureModule.name,version: fixtureModule.version,- dependencies: [mediaDependency, semanticTrackDependency],+ dependencies: [mediaDependency, timelineDependency],types: [],capabilities: [],producers: [],@@ -278,7 +279,7 @@ test("the self-described Audio Surface parses into the same finite Producer grapsvsManifest,spatialManifest,speechEvidenceManifest,- semanticTrackManifest,+ timelineManifest,temporalManifest,visualIrManifest,compositionManifest,@@ -289,7 +290,7 @@ test("the self-described Audio Surface parses into the same finite Producer grapregistry.registerStructured({ module: fixtureModule, declaration: fixtureSurface, handler: ({ element }) => ({records: [{ id: "source", type: mediaTypes.synchronized, value: { kind: "inline", value: media("surface", 48_000) }, range: element.range },- { id: "semantic", type: semanticTrackTypes.track, value: { kind: "inline", value: semantic }, range: element.range },+ { id: "semantic", type: timelineTypes.track, value: { kind: "inline", value: semantic }, range: element.range },],components: [],fragments: [],@@ -315,8 +316,8 @@ test("the self-described Audio Surface parses into the same finite Producer grap<import as="fixture" from="example.audio-inputs@1"/><import as="audio" from="@hypit/audio-track@1"/><fixture:Inputs/>- <audio:Track id="sound" semantic={semantic}>- <audio:Clip source={source} during="program" playback="loop-end" gain="0.5" fade-in="2f" fade-out="3f"/>+ <audio:Track id="sound" timeline={semantic}>+ <audio:Item source={source} during="program" playback="loop-end" gain="0.5" fade-in="2f" fade-out="3f"/></audio:Track></svml>`,},@@ -325,7 +326,7 @@ test("the self-described Audio Surface parses into the same finite Producer grapadmitRecord: createRecordAdmitter(validators),resolveSource() { throw new Error("Audio fixture has no source imports."); },});- const trackExport = resolveCompiledSourceExport(compiled, "sound.track", compositionTypes.audioTrack);+ const trackExport = resolveCompiledSourceExport(compiled, "sound.audio", compositionTypes.audioTrack);assert.equal(trackExport.ref.kind, "logical-output");const build = start(compiled.program, compiled.graph, sealBuildRequest({targets: [{ output: trackExport.ref.kind === "logical-output" ? trackExport.ref.id : "" }],@@ -337,7 +338,6 @@ test("the self-described Audio Surface parses into the same finite Producer graptemporalProducers.projectProgramInstant.name,temporalProducers.projectProgramInstant.name,temporalProducers.composeWindow.name,- semanticTrackProducers.projectProgramSpace.name,- audioTrackProducers.render.name,+ audioTrackProducers.render.name,].sort());});diff --git a/packages/author-kit/README.md b/packages/author-kit/README.mdindex b62afe92..4ad17dc9 100644--- a/packages/author-kit/README.md+++ b/packages/author-kit/README.md@@ -4,7 +4,7 @@ Small public framework API for an Author Package.It joins four stable authoring boundaries: nominal Module declarations, deterministic componenthandlers, sealed Graph Fragments and Markup Surface handlers. Domain facts still come from their-owners—for example `@hypit/composition`, `@hypit/semantic-track` and `@hypit/spatial`. This package+owners—for example `@hypit/composition`, `@hypit/timeline` and `@hypit/spatial`. This packagedoes not re-export the video vocabulary or turn those packages into one central object.`@hypit/hypit/author-kit` contains no registry, package search, installer, Runtime, Provider or Studiodiff --git a/packages/caption-fine-studio/README.md b/packages/caption-fine-studio/README.mdindex 967250b5..70f5fd7b 100644--- a/packages/caption-fine-studio/README.md+++ b/packages/caption-fine-studio/README.md@@ -1,3 +1,25 @@-# `@hypit/caption-fine-studio`+# Caption Fine Studio Companion-Hypit Studio Companion for `@hypit/caption-fine`. It projects the public Caption Schedule and owns the Inspector layout.+One Caption Track contains Cue blocks and a bottom Uses band declared through the public ABI.+Each Cue has a numbered title such as `#15` and complete subtitle text in its body.+Cue ids remain the underlying identities; the number is a display label. Cues retain their measured+times, including content currently hidden by a Use. Their read-only Range describes the content, independently of its visual treatment.++The Uses band shows each authored Window once, with its Style name. A whole-Timeline Use remains one+block across Segment changes and gaps. Hidden and overridden Uses stay selectable. The Companion+reads the Track's `content`, `schedule` and `program` exports; it never reconstructs author windows+from the rendered words.++Use entities retain the actual child Source range and standard Temporal lineage. Timeline gestures+therefore use the same writeback rules as Performance and Sound. Selecting a Use exposes its+referenced Style's font and Recipe fields. Shared Styles remain shared; changing a time window+changes presentation without editing Script Cue breaks.++Studio owns Track bands, block chrome, overlap selection and generic Inspector controls. This Companion owns+Caption facts, field grouping and Source bindings. Content, Uses and their renderer stay independent+of central Studio domain logic.++Fine Style parameters are a separate `parameters` Companion matched to the Fine Style Surface.+The Use band opts its `style` reference into the selected object's Companion. Custom Styles can+provide their own field table through their project package. Supported omitted defaults appear+in the Inspector and are inserted into the owning Recipe only when edited.diff --git a/packages/caption-fine-studio/package.json b/packages/caption-fine-studio/package.jsonindex 0a6eb21d..971479a4 100644--- a/packages/caption-fine-studio/package.json+++ b/packages/caption-fine-studio/package.json@@ -15,6 +15,8 @@"@hypit/composition": "workspace:*","@hypit/narrative": "workspace:*","@hypit/studio-adapter": "workspace:*",- "@hypit/fonts-open": "workspace:*"+ "@hypit/fonts-open": "workspace:*",+ "@hypit/caption": "workspace:*",+ "@hypit/timeline": "workspace:*"}}diff --git a/packages/caption-fine-studio/src/activation.ts b/packages/caption-fine-studio/src/activation.tsindex b78ec14f..bda72776 100644--- a/packages/caption-fine-studio/src/activation.ts+++ b/packages/caption-fine-studio/src/activation.ts@@ -1,7 +1,7 @@-import { createStudioTrackCompanionHostFacet } from "@hypit/studio-adapter";-import { captionFineStudioTrackCompanions } from "./index.js";+import { createStudioCompanionHostFacet } from "@hypit/studio-adapter";+import { captionFineStudioTrackCompanions, captionFineStudioParameterCompanions } from "./index.js";export default {format: "hypit.node-package@1" as const,- hostFacets: [createStudioTrackCompanionHostFacet(captionFineStudioTrackCompanions)],+ hostFacets: [createStudioCompanionHostFacet({ tracks: captionFineStudioTrackCompanions, parameters: captionFineStudioParameterCompanions })],};diff --git a/packages/caption-fine-studio/src/index.ts b/packages/caption-fine-studio/src/index.tsindex eb56c1b1..2393076b 100644--- a/packages/caption-fine-studio/src/index.ts+++ b/packages/caption-fine-studio/src/index.ts@@ -1,5 +1,7 @@+import { temporalTypes } from "@hypit/temporal";+import type { CaptionProgram, TimedCaptionProjection } from "@hypit/caption";import { openFontStudioFields } from "@hypit/fonts-open/studio";-import { captionFineMarkupSurfaces, captionFineModuleRef } from "@hypit/caption-fine";+import { fineCaptionEditableDefaults, captionFineMarkupSurfaces, captionFineModuleRef } from "@hypit/caption-fine";import type { FineCaptionSchedule } from "@hypit/caption-fine";import { compositionTypes } from "@hypit/composition";import { narrativeTypes } from "@hypit/narrative";@@ -10,11 +12,11 @@ import type {StudioEntityDraft,StudioInspectorFieldDeclaration,} from "@hypit/studio-adapter";-import { requiredReferencedValue, requiredSurfaceValue, textLayer } from "@hypit/studio-adapter";+import { authoredChildFor, temporalLineageFor, temporalSemanticSource, requiredReferencedValue, requiredSurfaceValue } from "@hypit/studio-adapter";const styleSurface = captionFineMarkupSurfaces.find((surface) => surface.name === "style");-const fontInspector = openFontStudioFields("program");+const fontInspector = openFontStudioFields("style");const recipeVocabulary = styleSurface?.vocabulary.attributes.find((attribute) => attribute.name === "recipe")?.recipe ?? [];@@ -95,7 +97,7 @@ export const captionFineInspectorFields: readonly StudioInspectorFieldDeclaratioif (placement === undefined) throw new Error(`Caption Studio has no explicit Inspector declaration for ${property.name}.`);const options = valuesFor(property);return {- binding: `program.${property.name}`,+ binding: `style.${property.name}`,label: title(property.name),...placement,summary: property.summary,@@ -118,39 +120,40 @@ function cueText(document: CaptionDocument | undefined, unitIds: readonly string.trim();}-function projectCaption(context: StudioTrackCompanionContext): readonly StudioEntityDraft[] {+export function projectCaptionContents(context: StudioTrackCompanionContext): readonly StudioEntityDraft[] {+ const content = requiredSurfaceValue(context, "content") as TimedCaptionProjection;const schedule = requiredSurfaceValue(context, "schedule") as FineCaptionSchedule;const document = requiredReferencedValue(context, "document", narrativeTypes.captionDocument) as CaptionDocument;- if (document.id !== schedule.documentId) throw new Error("Caption Studio Schedule belongs to another CaptionDocument.");- const rendered = new Map(context.generic().flatMap((entity) =>- entity.presentId === undefined ? [] : [[entity.presentId, entity] as const]));- return schedule.cues.map((cue): StudioEntityDraft => {- const base = rendered.get(cue.id);- const label = cueText(document, cue.units.map((unit) => unit.unitId));- const authoredId = context.placement?.id ?? cue.id;+ if (document.id !== content.documentId) throw new Error("Caption content belongs to another CaptionDocument.");+ return content.cues.map((cue, index): StudioEntityDraft => ({+ id: `${context.track.outputRef}:cue:${cue.id}`, authoredId: cue.id,+ display: { title: `#${index + 1}`, layers: [{ kind: "text", role: "content", text: cueText(document, cue.units.map(unit => unit.unitId)) }] },+ startFrame: cue.startFrame, endFrameExclusive: cue.endFrameExclusive, stackOrder: 0,+ renderIds: schedule.cues.filter(item => item.cueId === cue.id).map(item => item.id),+ presentation: { entity: "caption-cue", chrome: "standard" },+ inspector: [{ id: "range", label: "Range", domain: "when", section: { id: "cue", label: "Cue" }, value: `${cue.startFrame}–${cue.endFrameExclusive}`, unit: "f" }],+ }));+}++export function projectCaption(context: StudioTrackCompanionContext): readonly StudioEntityDraft[] {+ const program = requiredSurfaceValue(context, "program") as CaptionProgram;+ const uses: StudioEntityDraft[] = program.uses.map((use, index) => {+ const authoredId = use.window.subjectId;+ const child = authoredChildFor(context, authoredId, [temporalTypes.windowSpec]);+ if (child === undefined) throw new Error(`Caption Use ${authoredId} has no author provenance.`);+ const temporal = temporalLineageFor(context, authoredId, "window");+ const semantic = temporalSemanticSource(temporal);return {- ...(base ?? {- id: `${context.track.outputRef}:entity:${cue.id}`,- authoredId: cue.id,- display: { title: "Caption", layers: [] },- startFrame: cue.visibleStartFrame,- endFrameExclusive: cue.visibleEndFrameExclusive,- stackOrder: 0,- presentId: cue.id,- renderIds: [cue.id],- }),- authoredId,- ...(context.placement === undefined ? {} : { elementRange: context.placement.range }),- display: {- title: "Caption",- layers: label.length === 0 ? [] : [textLayer(label)],- },- startFrame: cue.visibleStartFrame,- endFrameExclusive: cue.visibleEndFrameExclusive,- presentation: { entity: "caption-cue", chrome: "standard" },- parameterReferences: { program: cue.styleId },+ id: `${context.track.outputRef}:use:${authoredId}`, authoredId,+ display: { title: use.styleId, layers: [] },+ ...use.window.span, stackOrder: index, elementRange: child.range,+ ...(temporal === undefined ? {} : { temporal }),+ ...(semantic === undefined ? {} : { markerId: semantic.id }),+ presentation: { entity: "caption-use", chrome: "standard" }, band: "uses",+ inspector: [{ id: "range", label: "Range", domain: "when", section: { id: "placement", label: "Placement" }, value: `${use.window.span.startFrame}–${use.window.span.endFrameExclusive}`, unit: "f" }],};});+ return [...projectCaptionContents(context), ...uses];}export const captionFineStudioTrackCompanions: readonly StudioTrackCompanion[] = [@@ -158,15 +161,24 @@ export const captionFineStudioTrackCompanions: readonly StudioTrackCompanion[] =id: "track", role: "track",output: { type: compositionTypes.visualTrack, surface: "track", modules: [captionFineModuleRef] },family: "caption", tone: "magenta", icon: "captions",- bindings: [- {- name: "program",- referenced: [fontInspector.binding],- recipe: { through: ["recipe"], bindings: recipeVocabulary.map(({ name }) => ({ name })) },- },- ],- inspector: [...fontInspector.fields, ...captionFineInspectorFields],- requiredValues: ["schedule"], project: projectCaption,- lane: { heightPx: 48 },+ lane: { heightPx: 36 },+ bands: [{+ id: "uses", placement: "after", heightPx: 15, display: "label",+ bindings: [{ name: "style", companion: true }],+ inspector: [{ binding: "style", label: "Style", domain: "how", section: { id: "style", label: "Style" }, control: "text" }],+ }],+ requiredValues: ["content", "schedule", "program"], project: projectCaption,+},];++export const captionFineStudioParameterCompanions: readonly import("@hypit/studio-adapter").StudioParameterCompanion[] = [{+ id: "style", match: { module: captionFineModuleRef, surface: "style" },+ bindings: [fontInspector.binding, { name: "recipe", recipe: { bindings: recipeVocabulary.map(({ name }) => ({ name,+ ...(Object.hasOwn(fineCaptionEditableDefaults, name) ? { fallback: fineCaptionEditableDefaults[name as keyof typeof fineCaptionEditableDefaults] } : {}),+ })) } }],+ inspector: [+ ...fontInspector.fields.map(field => ({ ...field, binding: field.binding.replace(/^style\./u, "") })),+ ...captionFineInspectorFields.map(field => ({ ...field, binding: field.binding.replace(/^style\./u, "recipe.") })),+ ],+}];diff --git a/packages/caption-fine-studio/test/adapter.test.ts b/packages/caption-fine-studio/test/adapter.test.tsindex 01b5b1cf..3cb1c4f3 100644--- a/packages/caption-fine-studio/test/adapter.test.ts+++ b/packages/caption-fine-studio/test/adapter.test.ts@@ -7,10 +7,10 @@ import { narrativeTypes } from "@hypit/narrative";import type { CaptionDocument } from "@hypit/narrative";import type { StudioTrackCompanionContext, StudioEntityDraft } from "@hypit/studio-adapter";-import { captionFineInspectorFields, captionFineStudioTrackCompanions } from "../src/index.js";+import { captionFineInspectorFields, projectCaptionContents } from "../src/index.js";test("Caption Companion owns Inspector grouping", () => {- const declared = (name: string) => captionFineInspectorFields.find((field) => field.binding === `program.${name}`);+ const declared = (name: string) => captionFineInspectorFields.find((field) => field.binding === `style.${name}`);assert.equal(declared("x")?.domain, "where");assert.equal(declared("x")?.section.id, "region");assert.equal(declared("cue-shadow-blur")?.domain, "how");@@ -24,7 +24,7 @@ test("Caption Companion projects Cue text and Style from public domain values",narrativeId: "story",documentId: "story.caption",cues: [{- id: "cue-1",+ id: "cue-1", cueId: "cue-1", visibility: [{startFrame:8,endFrameExclusive:24}],styleId: "caption-alt",semanticStartFrame: 10,semanticEndFrameExclusive: 20,@@ -54,7 +54,7 @@ test("Caption Companion projects Cue text and Style from public domain values",candidateOrigin: "source", role: "track",trace: {surface: "track", module: { name: "@hypit/caption-fine", version: "1" }, authoredId: "captions",- outputPorts: [{ name: "schedule", ref: "captions.schedule", type: "FineCaptionSchedule" }],+ outputPorts: [{ name: "schedule", ref: "captions.schedule", type: "FineCaptionSchedule" }, { name: "content", ref: "captions.content", type: "TimedCaptionProjection" }],references: [{input: "document", name: "story.caption", ref: "story.caption", type: "CaptionDocument",typeRef: narrativeTypes.captionDocument,@@ -63,17 +63,37 @@ test("Caption Companion projects Cue text and Style from public domain values",value: { visualIr: "hypit.visual-ir@1", id: "captions", presents: [] },},spans: [{ id: "cue-1", startFrame: 8, endFrameExclusive: 24, stackOrder: 70 }],- values: new Map<string, unknown>([["captions.schedule", schedule], ["story.caption", document]]),+ values: new Map<string, unknown>([["captions.schedule", schedule], ["captions.content", {documentId:document.id, cues:[{id:"cue-1",startFrame:10,endFrameExclusive:20,units:schedule.cues[0]!.units}]}], ["story.caption", document]]),temporalBindings: [],semantic: { spaceId: "speech", narrativeId: "story", presentation: { family: "speech", tone: "teal", icon: "timeline", lane: { heightPx: 45 } }, anchors: [], segments: [], tokens: [], selections: [], moments: [], provenance: { output: "speech", outputRef: "speech", origin: "source", status: "resolved", errors: [] } },generic: () => [base],} satisfies StudioTrackCompanionContext;- const adapter = captionFineStudioTrackCompanions.find((candidate) => candidate.id === "track")!;- const [cue] = adapter.project!(context);+ const [cue] = projectCaptionContents(context);assert.deepEqual(cue?.display, {- title: "Caption",+ title: "#1",layers: [{ kind: "text", role: "content", text: "真实字幕" }],});- assert.deepEqual(cue?.parameterReferences, { program: "caption-alt" });- assert.deepEqual([cue?.startFrame, cue?.endFrameExclusive], [8, 24]);+ assert.equal(cue?.presentation?.chrome, "standard");+ assert.equal(cue?.parameterReferences, undefined);+ assert.deepEqual([cue?.startFrame, cue?.endFrameExclusive], [10, 20]);+});++test("Caption Uses keep authored windows and child ownership even with no rendered content", async () => {+ const { projectCaption } = await import("../src/index.js");+ const { temporalTypes } = await import("@hypit/temporal");+ const { timelineFixture } = await import("../../../test/timeline-fixture.js");+ const { projectProgramWindow } = await import("../../../test/temporal-fixture.js");+ const timeline = timelineFixture({id:"film",durationSec:6,frameRate:{numerator:30,denominator:1}}, {segments:[{id:"a",frameCount:90},{id:"b",frameCount:90}]});+ const uses = ["base", "hidden"].map(id=>({styleId:id,window:projectProgramWindow({itemId:id,semantic:timeline,projection:{start:{ref:"program.start"},end:{ref:"program.end"}}})}));+ const context = {+ track:{outputRef:"captions.track",trace:{references:[{input:"document",typeRef:narrativeTypes.captionDocument,ref:"document"}],outputPorts:[{name:"content",ref:"content"},{name:"schedule",ref:"schedule"},{name:"program",ref:"program"}]}},+ placement:{children:uses.map((use,i)=>({range:{start:i*10,end:i*10+8},referenceAttributes:{style:use.styleId},values:[{type:temporalTypes.windowSpec,value:{id:use.window.subjectId}}]}))},+ values:new Map<string,unknown>([["document",{id:"document",units:[],words:[]}],["content",{documentId:"document",cues:[]}],["schedule",{cues:[]}],["program",{uses}]]),+ spans:[],temporalBindings:[],+ } as unknown as StudioTrackCompanionContext;+ const entities=projectCaption(context);+ assert.deepEqual(entities.map(item=>[item.display.title,item.startFrame,item.endFrameExclusive,item.elementRange]),[+ ["base",0,180,{start:0,end:8}],["hidden",0,180,{start:10,end:18}],+ ]);+ assert.ok(entities.every(item=>item.band==="uses" && item.presentation?.chrome==="standard"));});diff --git a/packages/caption-fine/README.md b/packages/caption-fine/README.mdindex 585a32cc..57330b64 100644--- a/packages/caption-fine/README.md+++ b/packages/caption-fine/README.md@@ -7,9 +7,11 @@ does not carry a private visual role.```xml<caption-fine:Style id="primary" recipe={recipes.caption.primary} font={caption-font}/>-<caption:Program id="captions" document={story.caption} narrative={story} default={primary}/>+<caption-fine:Track id="captions-track" document={story.caption}- semantic={speech.semantic} program={captions}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={primary}/>+ </caption-fine:Track>```An optional Spatial Region Timeline replaces only the moving placement point. The timeline is authored@@ -20,7 +22,9 @@ and a later Run can reuse the same media and alignment while changing placement:```xml<space:RegionTimeline id="heads" within={vertical} recipe={tracking.heads.default}/><caption-fine:Track id="captions-track" document={story.caption}- semantic={speech.semantic} program={captions} regions={heads}/>+ timeline={speech.timeline} regions={heads}>+ <caption-fine:Use style={primary}/>+ </caption-fine:Track>```With `regions`, every Cue must carry one Script Role. When the Region Timeline contains a measured region for that@@ -45,16 +49,17 @@ One SVS Recipe freezes three public dimensions:Caption first projects authored Script units onto semantic Word timing. Fine then produces anexplicit visible Schedule and renders that Schedule. Lead and tail never change the semantic Word-times used by Karaoke.+times used by Karaoke. Later Uses mask earlier presentation, including Hidden. The final visibility+is clipped to the winning Use Window while the original Cue envelope and animations are preserved.The Fine Schedule preserves the Caption projection's ProgramSpace, Narrative and document identities.The renderer rejects any mismatched Space or document. Studio may expose lead, tail and handoff asordinary parameter edits, but Cue rectangles remain read-only semantic evidence.-Script segments, turns, Style changes and `||` author Cue grouping. Fine rejects word-specific Style-runs. A caption whose Cue contains structural roles or+Script segments, turns and `||` organize complete Cues. Timed Uses select presentation without+changing that grouping. Fine applies uniform rules to the words of a Cue. A caption whose Cue contains structural roles orrelationships—such as an independently arranged oversized keyword and supporting phrase—can use-a new project Caption family. Reuse the common [Caption document, Program and timing](../caption/README.md),+a new project Caption family. Reuse the common [Caption content, timed Uses and timing](../caption/README.md),and implement the new schedule and rendering behavior in that package. This is ordinary componentauthorship; different colors or fonts alone can remain Fine Style choices.@@ -119,5 +124,5 @@ The Hypit Skill's Caption craft page owns grouping and visual direction.primary face or a `<caption-fine:Fallback font={...}/>` child, just like a bundled face. See[Media font assets](../media/README.md#font-files) and [the open catalog](../fonts-open/README.md).-The [Fine Studio Companion](../caption-fine-studio/src/index.ts) reads the same schedule and per-Cue-Style references. It presents the actual Cue timing and exposes supported Style edits in the Inspector.+The [Fine Studio Companion](../caption-fine-studio/src/index.ts) reads the same schedule and authored Use+Style references. It presents the actual Cue timing and exposes Use timing and supported Style edits in the Inspector.diff --git a/packages/caption-fine/package.json b/packages/caption-fine/package.jsonindex 3938d09d..857573ad 100644--- a/packages/caption-fine/package.json+++ b/packages/caption-fine/package.json@@ -19,10 +19,12 @@"@hypit/narrative": "workspace:*","@hypit/program-space": "workspace:*","@hypit/protocol": "workspace:*",- "@hypit/semantic-track": "workspace:*",+ "@hypit/timeline": "workspace:*","@hypit/spatial": "workspace:*","@hypit/svs": "workspace:*",- "@hypit/markup": "workspace:*"+ "@hypit/markup": "workspace:*",+ "@hypit/temporal": "workspace:*",+ "@hypit/temporal-markup": "workspace:*"},"devDependencies": {"@hypit/script": "workspace:*"diff --git a/packages/caption-fine/src/component.ts b/packages/caption-fine/src/component.tsindex 330a1309..163745dd 100644--- a/packages/caption-fine/src/component.ts+++ b/packages/caption-fine/src/component.ts@@ -1,7 +1,7 @@+import type { Timeline } from "@hypit/timeline";import type { CaptionProgram, TimedCaptionProjection } from "@hypit/caption";import type { ComponentPackage } from "@hypit/component-kit";import type { CaptionDocument } from "@hypit/narrative";-import type { ProgramSpace } from "@hypit/program-space";import type { StoredValue } from "@hypit/protocol";import { canonicalize } from "@hypit/protocol";import type { SpatialRegionTimeline } from "@hypit/spatial";@@ -36,7 +36,7 @@ export const captionFineComponent = {inline<FineCaptionSchedule>(inputs.schedule?.value, "FineCaptionSchedule"),inline<CaptionProgram>(inputs.program?.value, "CaptionProgram"),inline<CaptionDocument>(inputs.document?.value, "CaptionDocument"),- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ inline<Timeline>(inputs.timeline?.value, "Timeline"),)) } },needs: {},}),@@ -48,7 +48,7 @@ export const captionFineComponent = {inline<FineCaptionSchedule>(inputs.schedule?.value, "FineCaptionSchedule"),inline<CaptionProgram>(inputs.program?.value, "CaptionProgram"),inline<CaptionDocument>(inputs.document?.value, "CaptionDocument"),- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ inline<Timeline>(inputs.timeline?.value, "Timeline"),inline<SpatialRegionTimeline>(inputs.regions?.value, "SpatialRegionTimeline"),)) } },needs: {},diff --git a/packages/caption-fine/src/fragment.ts b/packages/caption-fine/src/fragment.tsindex 79b88f98..2ed28db1 100644--- a/packages/caption-fine/src/fragment.ts+++ b/packages/caption-fine/src/fragment.ts@@ -2,7 +2,7 @@ import { captionProducers, captionTypes } from "@hypit/caption";import { compositionTypes } from "@hypit/composition";import { sealGraphFragment } from "@hypit/elaborator";import { narrativeTypes } from "@hypit/narrative";-import { semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";+import { timelineTypes } from "@hypit/timeline";import { spatialTypes } from "@hypit/spatial";import { captionFineProducers, captionFineTypes } from "./manifest.js";@@ -14,21 +14,15 @@ const operation = (id: string) => ({ kind: "fragment-operation" as const, operatexport const fineCaptionTrackFragment = sealGraphFragment({inputs: [{ name: "document", type: narrativeTypes.captionDocument },- { name: "semantic", type: semanticTrackTypes.track },+ { name: "timeline", type: timelineTypes.track },{ name: "program", type: captionTypes.program },],operations: [- {- id: "caption:space",- producer: semanticTrackProducers.projectProgramSpace,- inputs: { track: input("semantic") },- result: { kind: "output", name: "space" },- },{id: "caption:temporalize-document",producer: captionProducers.temporalizeDocument,inputs: {- document: input("document"), semantic: input("semantic"), program: input("program"),+ document: input("document"), timeline: input("timeline"),},result: { kind: "output", name: "caption" },},@@ -45,7 +39,7 @@ export const fineCaptionTrackFragment = sealGraphFragment({producer: captionFineProducers.render,inputs: {schedule: operation("caption-fine:schedule"), program: input("program"),- document: input("document"), space: operation("caption:space"),+ document: input("document"), timeline: input("timeline"),},result: { kind: "output", name: "track" },},@@ -68,22 +62,16 @@ export const fineCaptionTrackFragment = sealGraphFragment({export const fineCaptionRegionTrackFragment = sealGraphFragment({inputs: [{ name: "document", type: narrativeTypes.captionDocument },- { name: "semantic", type: semanticTrackTypes.track },+ { name: "timeline", type: timelineTypes.track },{ name: "program", type: captionTypes.program },{ name: "regions", type: spatialTypes.regionTimeline },],operations: [- {- id: "caption:space",- producer: semanticTrackProducers.projectProgramSpace,- inputs: { track: input("semantic") },- result: { kind: "output", name: "space" },- },{id: "caption:temporalize-document",producer: captionProducers.temporalizeDocument,inputs: {- document: input("document"), semantic: input("semantic"), program: input("program"),+ document: input("document"), timeline: input("timeline"),},result: { kind: "output", name: "caption" },},@@ -100,7 +88,7 @@ export const fineCaptionRegionTrackFragment = sealGraphFragment({producer: captionFineProducers.renderWithRegions,inputs: {schedule: operation("caption-fine:schedule"), program: input("program"),- document: input("document"), space: operation("caption:space"), regions: input("regions"),+ document: input("document"), timeline: input("timeline"), regions: input("regions"),},result: { kind: "output", name: "track" },},diff --git a/packages/caption-fine/src/index.ts b/packages/caption-fine/src/index.tsindex 5ff635f6..4f510ed6 100644--- a/packages/caption-fine/src/index.ts+++ b/packages/caption-fine/src/index.ts@@ -17,3 +17,5 @@ export {} from "./style.js";export { decodeFineCaptionStyleSurface, decodeFineCaptionTrackSurface } from "./surface.js";export type * from "./types.js";++export { fineCaptionEditableDefaults } from "./recipe.js";diff --git a/packages/caption-fine/src/manifest.ts b/packages/caption-fine/src/manifest.tsindex 68d994e4..266ccc09 100644--- a/packages/caption-fine/src/manifest.ts+++ b/packages/caption-fine/src/manifest.ts@@ -1,12 +1,15 @@+import { temporalDependency, temporalTypes } from "@hypit/temporal";+import { temporalWindowAttributeVocabulary } from "@hypit/temporal-markup";+import type { Timeline } from "@hypit/timeline";import { readFile } from "node:fs/promises";import { captionManifest, captionModuleRef, captionTypes } from "@hypit/caption";import { compositionDependency, compositionTypes } from "@hypit/composition";import { mediaDependency, mediaTypes } from "@hypit/media";import { narrativeDependency, narrativeTypes } from "@hypit/narrative";-import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";+import type { ModuleManifest, ProducerRef } from "@hypit/protocol";-import { semanticTrackDependency, semanticTrackTypes } from "@hypit/semantic-track";+import { timelineDependency, timelineTypes } from "@hypit/timeline";import { spatialDependency, spatialTypes } from "@hypit/spatial";import { svsRecipeType } from "@hypit/svs";@@ -35,7 +38,7 @@ export const captionFineMarkupSurfaces = [summary: "Resolves one SVS Recipe and one exact font stack into a complete fine-grained Caption Style.",attributes: [{ name: "id", kind: "identifier", required: true,- summary: "Names this Style so a Caption Program can assign it." },+ summary: "Names this Style so a timed Use can select it." },{ name: "recipe", kind: "reference", required: true, accepts: [svsRecipeType],summary: "Chooses the Recipe that carries Cue geometry, Paint and local motion.",recipe: [@@ -295,9 +298,9 @@ export const captionFineMarkupSurfaces = [},},{- name: "track", tag: "Track", mode: "structured", outputs: [captionFineTypes.schedule, compositionTypes.visualTrack],+ name: "track", tag: "Track", mode: "structured", outputs: [captionFineTypes.schedule, compositionTypes.visualTrack, captionTypes.program, captionTypes.timedProjection, captionTypes.header, captionTypes.filter, temporalTypes.instantSpec, temporalTypes.windowSpec, temporalTypes.instant, temporalTypes.window],vocabulary: {- summary: "Joins the authored CaptionDocument against the SemanticTrack and renders it as one ordinary peer VisualTrack.",+ summary: "Joins the authored CaptionDocument against the Timeline and renders it as one ordinary peer VisualTrack.",appearance: "One block of caption text wrapped into lines inside a rounded Cue box, placed at a Recipe-chosen point on the Canvas and spanning a fraction of its width. Cues follow the speech one after another, each arriving and leaving with its own motion, and the Words of a Cue either stand there together from its first Frame or uncover as they are spoken. As the speech advances, the Word being spoken, or every Word up to it, is repainted in the active Paint, snapping at the Word boundary or sweeping across the glyphs, and it may take a rule beneath it, a rounded highlight capsule behind it and a brief pop at the moment it becomes the spoken one. That capsule either stands alone on each Atom or grows as one continuous run over everything already read, following the Words across line breaks.\n\nEvery Cue uses the ordered exact font stack its Style names, with fallback faces supplying glyphs for other scripts. Emphasis varies by timed unit through the active Paint. Lines follow the common flow, so a line cannot be given a separate face, colour or weight of its own. A caption whose lines are set in different typefaces is outside what this Track can draw.",preview: previewImage("Track.png"),attributes: [@@ -305,23 +308,30 @@ export const captionFineMarkupSurfaces = [summary: "Names this Track so its rendered output can be placed in a Film." },{ name: "document", kind: "reference", required: true, accepts: [narrativeTypes.captionDocument],summary: "Chooses the Script-owned CaptionDocument." },- { name: "semantic", kind: "reference", required: true, accepts: [semanticTrackTypes.track],- summary: "Chooses the continuous SemanticTrack whose Word anchors give every Cue its time." },- { name: "program", kind: "reference", required: true, accepts: [captionTypes.program],- summary: "Chooses the Style assignment that decides which Style each run is rendered in." },+ { name: "timeline", kind: "reference", required: true, accepts: [timelineTypes.track],+ summary: "Chooses the continuous Timeline whose Word anchors give every Cue its time." },{ name: "regions", kind: "reference", required: false, accepts: [spatialTypes.regionTimeline],summary: "Optionally follows external, frame-exact regions whose ids equal Script Roles, placing a Cue at its measured speaker region's top center and hiding it on null Frames." },],+ children: [{ tag: "Use", cardinality: "many", summary: "Presents complete Cues inside a time window. Later matching Uses replace earlier ones.", attributes: [+ { name: "id", kind: "identifier", required: false, summary: "Optional occurrence identity." },+ { name: "style", kind: "reference", required: true, accepts: [captionTypes.style], summary: "Complete Caption Style, including Hidden." },+ { name: "role", kind: "literal", required: false, summary: "Optional speaker filter, independent of the time window." },+ ...temporalWindowAttributeVocabulary,+ ] }],ports: [{ name: "schedule", type: captionFineTypes.schedule,summary: "The visible Cue schedule consumed by the renderer and other declared tools." },{ name: "track", type: compositionTypes.visualTrack,summary: "The rendered Caption as one self-contained VisualTrack." },],- example: `<caption-fine:Track id="captions" document={story.caption} semantic={speech.semantic} program={caption-program}/>`,+ example: `<caption-fine:Track id="captions" document={story.caption} timeline={speech.timeline}>+ <caption-fine:Use style={primary-caption}/>+ <caption-fine:Use during={story.selection.answer} style={answer-caption}/>+</caption-fine:Track>`,notes: [- "The element is empty; it accepts no children and no text.",- "One Track renders the Program's default Style and every ordered replacement together.",+ "Use without time attributes covers the whole Timeline. Role filters content without changing the window.",+ "Script Cue breaks organize text; Use windows change presentation without cutting Cues or restarting word timing. A hidden Use still replaces earlier presentation.","Without regions, Recipe x/y placement is unchanged. With regions, a measured region overrides x/y for that Frame; a null Frame on that Role track hides the Cue, while a missing Role track retains authored x/y.","The tracked point replaces Recipe x/y while width and the Recipe anchor still decide the Caption box geometry; anchor-x=center and anchor-y=bottom place the box immediately above the measured region.",],@@ -339,9 +349,8 @@ export const captionFineManifest: ModuleManifest = {compositionDependency,mediaDependency,narrativeDependency,- programSpaceDependency,- semanticTrackDependency,- spatialDependency,+ timelineDependency,+ spatialDependency, temporalDependency,],types: [{ name: captionFineTypes.schedule.name }],capabilities: [],@@ -362,7 +371,7 @@ export const captionFineManifest: ModuleManifest = {{ name: "schedule", type: captionFineTypes.schedule },{ name: "program", type: captionTypes.program },{ name: "document", type: narrativeTypes.captionDocument },- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },],outputs: [{ name: "track", type: compositionTypes.visualTrack }],needs: [],@@ -373,7 +382,7 @@ export const captionFineManifest: ModuleManifest = {{ name: "schedule", type: captionFineTypes.schedule },{ name: "program", type: captionTypes.program },{ name: "document", type: narrativeTypes.captionDocument },- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },{ name: "regions", type: spatialTypes.regionTimeline },],outputs: [{ name: "track", type: compositionTypes.visualTrack }],diff --git a/packages/caption-fine/src/recipe.ts b/packages/caption-fine/src/recipe.tsindex 3a215938..cd6d88c5 100644--- a/packages/caption-fine/src/recipe.ts+++ b/packages/caption-fine/src/recipe.ts@@ -118,3 +118,10 @@ export const fineCaptionRecipeSchema: ValueSchema = {required.has(name) ? { schema: schemaFor(name) } : { schema: schemaFor(name), optional: true },])),};++/** Static authored defaults; dependent paint values retain their decoder relationships. */+export const fineCaptionEditableDefaults = {+ opacity: 1, "stroke-width": 0, "shadow-opacity": 0, "glow-opacity": 0,+ karaoke: "off", "karaoke-transition": "step", "active-box": "off",+ "underline": "off", "active-underline": "off",+} as const;diff --git a/packages/caption-fine/src/render.ts b/packages/caption-fine/src/render.tsindex 6b330af8..ea1414b2 100644--- a/packages/caption-fine/src/render.ts+++ b/packages/caption-fine/src/render.ts@@ -1,3 +1,4 @@+import type { Timeline } from "@hypit/timeline";import { assertCaptionProgramForDocument } from "@hypit/caption";import type { CaptionProgram } from "@hypit/caption";import { assertVisualTrackIdentity, sealVisualTrack } from "@hypit/composition";@@ -14,7 +15,6 @@ import type {} from "@hypit/composition";import type { CaptionAlignmentUnit, CaptionDocument } from "@hypit/narrative";import { assertProgramSpaceIdentity, programSpaceFrameCount } from "@hypit/program-space";-import type { ProgramSpace } from "@hypit/program-space";import { assertSpatialRegionTimeline } from "@hypit/spatial";import type { CanvasSpace, SpatialFrame, SpatialRegionTimeline } from "@hypit/spatial";@@ -639,7 +639,7 @@ function structuralRowCount(/*** Acoustic Word windows may overlap. Preserve those measurements in the- * SemanticTrack, but give every current-only visual state one unambiguous+ * Timeline, but give every current-only visual state one unambiguous* owner: as soon as the next authored unit starts, the previous one stops.*/function exclusiveActivationFrames(@@ -788,8 +788,8 @@ function cueElements(const next = atoms[prefixIndex + 1];const nextStart = next === undefined ? durationFrames : atomFrames.get(next.id)?.start;if (nextStart === undefined) throw new Error("Fine Caption is missing timing for the next Atom");- // A no-break space inside an atom keeps its words on one row; the boundaries between atoms- // take an ordinary space. Both are spaced only where the surfaces meeting there call for one.+ // A no-break timeline inside an atom keeps its words on one row; the boundaries between atoms+ // take an ordinary timeline. Both are spaced only where the surfaces meeting there call for one.const prefixText = joinSurfaces(atoms.slice(0, prefixIndex + 1).map((prefixAtom, index) => {for (const wordId of prefixAtom.wordIds) {@@ -1069,33 +1069,32 @@ export function renderFineCaption(schedule: FineCaptionSchedule,program: CaptionProgram,document: CaptionDocument,- space: ProgramSpace,+ timeline: Timeline,regions?: SpatialRegionTimeline,): VisualTrack {assertFineCaptionSchedule(schedule);assertCaptionProgramForDocument(program, document);if (schedule.documentId !== document.id) throw new Error("Fine Caption received another CaptionDocument");- if (program.wordRuns.length !== 0) {- throw new Error("Fine Caption accepts one uniform token rule and cannot consume word-specific Style runs");- }- assertProgramSpaceIdentity(space);- if (schedule.spaceId !== space.id || schedule.narrativeId !== document.narrativeId) {++ assertProgramSpaceIdentity(timeline);+ if (schedule.spaceId !== timeline.id || schedule.narrativeId !== document.narrativeId) {throw new Error("Fine Caption inputs belong to different ProgramSpaces or Narratives");}const styles = new Map(program.styles.map((style) => [style.id, style]));const wordText = new Map(document.words.map((word) => [word.id, word.text]));const atomById = new Map(document.units.map((atom) => [atom.id, atom]));for (const style of styles.values()) {+ if (style.rendering === null) continue;if (style.rendering.family !== FINE_CAPTION_FAMILY) {throw new Error(`Fine Caption cannot render Style family ${style.rendering.family}`);}assertFineCaptionParameters(style.rendering.parameters as unknown as FineCaptionParameters);}- const totalFrames = programSpaceFrameCount(space);+ const totalFrames = programSpaceFrameCount(timeline);const regionTracks = regions === undefined ? undefined : (() => {assertSpatialRegionTimeline(regions);if (regions.frameCount !== totalFrames) {- throw new Error(`Fine Caption regions cover ${regions.frameCount} Frames but ProgramSpace has ${totalFrames}`);+ throw new Error(`Fine Caption regions cover ${regions.frameCount} Frames but Timeline has ${totalFrames}`);}return new Map(regions.tracks.map((track) => [track.id, track]));})();@@ -1105,6 +1104,7 @@ export function renderFineCaption(const resolvedAtoms = atoms.map((atom) => atom!);const style = styles.get(cue.styleId);if (style === undefined) throw new Error(`Fine Caption Cue ${cue.id} references unknown Style ${cue.styleId}`);+ if (style.rendering === null) return [];const parameters = style.rendering.parameters as unknown as FineCaptionParameters;const startFrame = Math.max(0, cue.visibleStartFrame);const measuredEnd = Math.min(totalFrames, cue.visibleEndFrameExclusive);@@ -1130,16 +1130,17 @@ export function renderFineCaption(return [{id: cue.id,span: { startFrame, endFrameExclusive },+ visibility: cue.visibility,stacking: { order: parameters.stackingOrder, tieBreak: `${program.id}:${cue.id}` },elements: cueElements(resolvedAtoms, atomFrames, parameters, wordText, durationFrames, cue.styleId, trackedPlacement),}];});const track = sealVisualTrack({- programSpaceId: space.id,+ programSpaceId: timeline.id,visualIr: "hypit.visual-ir@1",id: program.id,presents,});- assertVisualTrackIdentity(track, space);+ assertVisualTrackIdentity(track, timeline);return track;}diff --git a/packages/caption-fine/src/schedule.ts b/packages/caption-fine/src/schedule.tsindex 4d5f207f..c1b1af80 100644--- a/packages/caption-fine/src/schedule.ts+++ b/packages/caption-fine/src/schedule.ts@@ -1,4 +1,4 @@-import { applyCaptionMute, assertCaptionProgramForDocument, assertTimedCaptionProjection } from "@hypit/caption";+import { assertCaptionProgramForDocument, assertTimedCaptionProjection, captionUseVisibility } from "@hypit/caption";import type { CaptionProgram, TimedCaptionProjection } from "@hypit/caption";import type { CaptionDocument } from "@hypit/narrative";@@ -27,13 +27,11 @@ export function scheduleFineCaption(if (projection.narrativeId !== document.narrativeId) {throw new Error("Fine Caption Schedule received a CaptionDocument from another Narrative");}- if (program.wordRuns.length !== 0) {- throw new Error("Fine Caption accepts one uniform token rule and cannot consume word-specific Style runs");- }- const visibleProjection = applyCaptionMute(projection, program, document);-const parameters = new Map<string, FineCaptionParameters>();+ const activeStyles = new Set(program.uses.map(use => use.styleId));for (const style of program.styles) {+ if (!activeStyles.has(style.id)) continue;+ if (style.rendering === null) continue;if (style.rendering.family !== FINE_CAPTION_FAMILY) {throw new Error(`Fine Caption cannot schedule Style family ${style.rendering.family}`);}@@ -42,39 +40,41 @@ export function scheduleFineCaption(parameters.set(style.id, value);}- const desired = visibleProjection.cues.map((cue): FineCaptionScheduledCue => {- const style = parameters.get(cue.styleId);- if (style === undefined) throw new Error(`Fine Caption Cue ${cue.id} references unknown Style ${cue.styleId}`);- const visibleStartFrame = Math.max(0, cue.startFrame - style.timing.leadFrames);- const visibleEndFrameExclusive = cue.endFrameExclusive + style.timing.tailFrames;- return {- id: cue.id,- styleId: cue.styleId,- semanticStartFrame: cue.startFrame,- semanticEndFrameExclusive: cue.endFrameExclusive,- visibleStartFrame,- visibleEndFrameExclusive,- units: cue.units,- };- });-+ const units = new Map(document.units.map(unit => [unit.id, unit]));const cues: FineCaptionScheduledCue[] = [];- for (const wanted of desired) {- let cue = wanted;- const previous = cues.at(-1);- if (previous !== undefined) {- const previousStyle = parameters.get(previous.styleId)!;- const currentStyle = parameters.get(cue.styleId)!;- if (previousStyle.timing.handoff === "cut" || currentStyle.timing.handoff === "cut") {- const previousEnd = Math.max(- previous.semanticEndFrameExclusive,- Math.min(previous.visibleEndFrameExclusive, cue.semanticStartFrame),- );- cues[cues.length - 1] = { ...previous, visibleEndFrameExclusive: previousEnd };+ for (const [index, use] of program.uses.entries()) {+ if (use.window.start.source.spaceId !== projection.spaceId) throw new Error("Caption Use belongs to another Timeline");+ const style = parameters.get(use.styleId);+ if (style === undefined) continue; // Hidden still participates in coverage below.+ const desired = projection.cues.filter(cue => use.role === undefined || units.get(cue.units[0]!.unitId)?.role === use.role).map((cue): FineCaptionScheduledCue => ({+ id: `${use.window.subjectId}:${cue.id}`, cueId: cue.id, styleId: use.styleId,+ semanticStartFrame: cue.startFrame, semanticEndFrameExclusive: cue.endFrameExclusive,+ visibleStartFrame: Math.max(0, cue.startFrame - style.timing.leadFrames),+ visibleEndFrameExclusive: cue.endFrameExclusive + style.timing.tailFrames,+ units: cue.units, visibility: [],+ }));+ // Resolve each speaker's ordinary Cue handoff before masking by Use windows.+ desired.sort((a, b) => a.semanticStartFrame - b.semanticStartFrame);+ const previousByRole = new Map<string | undefined, number>();+ const envelopes: FineCaptionScheduledCue[] = [];+ for (const wanted of desired) {+ let cue = wanted;+ const role = units.get(cue.units[0]!.unitId)?.role;+ const previousIndex = previousByRole.get(role);+ const previous = previousIndex === undefined ? undefined : envelopes[previousIndex];+ if (previous !== undefined && style.timing.handoff === "cut" && previous.semanticEndFrameExclusive <= cue.semanticStartFrame) {+ const previousEnd = Math.max(previous.semanticEndFrameExclusive, Math.min(previous.visibleEndFrameExclusive, cue.semanticStartFrame));+ envelopes[previousIndex!] = { ...previous, visibleEndFrameExclusive: previousEnd };cue = { ...cue, visibleStartFrame: Math.min(cue.semanticStartFrame, Math.max(cue.visibleStartFrame, previousEnd)) };}+ previousByRole.set(role, envelopes.length);+ envelopes.push(cue);+ }+ for (const cue of envelopes) {+ const role = units.get(cue.units[0]!.unitId)?.role;+ const visibility = captionUseVisibility(program, index, role, { startFrame: cue.visibleStartFrame, endFrameExclusive: cue.visibleEndFrameExclusive });+ if (visibility.length) cues.push({ ...cue, visibility });}- cues.push(cue);}const ids = new Set<string>();@@ -110,6 +110,10 @@ export function assertFineCaptionSchedule(value: FineCaptionSchedule): void {throw new Error("Fine Caption Schedule contains an invalid Cue");}ids.add(cue.id);+ if (!cue.cueId || cue.visibility.length === 0 || cue.visibility.some(span =>+ !Number.isSafeInteger(span.startFrame) || !Number.isSafeInteger(span.endFrameExclusive)+ || span.startFrame < cue.visibleStartFrame || span.endFrameExclusive > cue.visibleEndFrameExclusive+ || span.endFrameExclusive <= span.startFrame)) throw new Error("Fine Caption visibility is invalid");for (const [label, frame] of [["semantic start", cue.semanticStartFrame],["semantic end", cue.semanticEndFrameExclusive],diff --git a/packages/caption-fine/src/style.ts b/packages/caption-fine/src/style.tsindex 5e4a2de8..2adaa3e0 100644--- a/packages/caption-fine/src/style.ts+++ b/packages/caption-fine/src/style.ts@@ -8,6 +8,7 @@ import type { SvsRecipe } from "@hypit/svs";import type { FineCaptionGlyphPaint, FineCaptionParameters } from "./types.js";import {fineCaptionOneShotMotions,+ fineCaptionEditableDefaults,fineCaptionOptionalRecipeProperties,fineCaptionRequiredRecipeProperties,} from "./recipe.js";@@ -104,14 +105,14 @@ function glyphPaint(recipe: SvsRecipe, prefix: "" | "active-", base?: FineCaptioreturn {fill: color(recipe, `${prefix}fill`, prefix === "active-" ? "#FFD54A" : base?.fill),...(gradient === undefined ? {} : { gradient }),- opacity: number(recipe, `${prefix}opacity`, base?.opacity ?? 1),+ opacity: number(recipe, `${prefix}opacity`, base?.opacity ?? fineCaptionEditableDefaults.opacity),stroke: {color: color(recipe, `${prefix}stroke-color`, baseStroke?.color ?? "#000000"),- widthPx: number(recipe, `${prefix}stroke-width`, baseStroke?.widthPx ?? 0),+ widthPx: number(recipe, `${prefix}stroke-width`, baseStroke?.widthPx ?? fineCaptionEditableDefaults["stroke-width"]),},shadow: {color: color(recipe, `${prefix}shadow-color`, baseShadow?.color ?? "#000000"),- opacity: number(recipe, `${prefix}shadow-opacity`, baseShadow?.opacity ?? 0),+ opacity: number(recipe, `${prefix}shadow-opacity`, baseShadow?.opacity ?? fineCaptionEditableDefaults["shadow-opacity"]),offsetXPx: number(recipe, `${prefix}shadow-x`, baseShadow?.offsetXPx ?? 0),offsetYPx: number(recipe, `${prefix}shadow-y`, baseShadow?.offsetYPx ?? 0),blurPx: number(recipe, `${prefix}shadow-blur`, baseShadow?.blurPx ?? 0),@@ -125,7 +126,7 @@ function glyphPaint(recipe: SvsRecipe, prefix: "" | "active-", base?: FineCaptio},glow: {color: color(recipe, `${prefix}glow-color`, baseGlow?.color ?? "#FFFFFF"),- opacity: number(recipe, `${prefix}glow-opacity`, baseGlow?.opacity ?? 0),+ opacity: number(recipe, `${prefix}glow-opacity`, baseGlow?.opacity ?? fineCaptionEditableDefaults["glow-opacity"]),blurPx: number(recipe, `${prefix}glow-blur`, baseGlow?.blurPx ?? 0),spreadPx: number(recipe, `${prefix}glow-spread`, baseGlow?.spreadPx ?? 0),},@@ -216,13 +217,13 @@ export function fineCaptionParameters(basePaint,activePaint: glyphPaint(recipe, "active-", basePaint),underline: {- mode: choice(recipe, "underline", ["off", "always"] as const, "off"),+ mode: choice(recipe, "underline", ["off", "always"] as const, fineCaptionEditableDefaults["underline"]),color: color(recipe, "underline-color", basePaint.fill),thicknessPx: number(recipe, "underline-thickness", 2),offsetPx: number(recipe, "underline-offset", 4),},activeUnderline: {- mode: choice(recipe, "active-underline", ["off", "current", "trail"] as const, "off"),+ mode: choice(recipe, "active-underline", ["off", "current", "trail"] as const, fineCaptionEditableDefaults["active-underline"]),color: color(recipe, "active-underline-color", "#FFD54A"),thicknessPx: number(recipe, "active-underline-thickness", 3),offsetPx: number(recipe, "active-underline-offset", 4),@@ -244,11 +245,11 @@ export function fineCaptionParameters(},},karaoke: {- mode: choice(recipe, "karaoke", ["off", "current", "trail"] as const, "off"),- transition: choice(recipe, "karaoke-transition", ["step", "wipe"] as const, "step"),+ mode: choice(recipe, "karaoke", ["off", "current", "trail"] as const, fineCaptionEditableDefaults["karaoke"]),+ transition: choice(recipe, "karaoke-transition", ["step", "wipe"] as const, fineCaptionEditableDefaults["karaoke-transition"]),},activeBox: {- mode: choice(recipe, "active-box", ["off", "current", "trail"] as const, "off"),+ mode: choice(recipe, "active-box", ["off", "current", "trail"] as const, fineCaptionEditableDefaults["active-box"]),continuity: choice(recipe, "active-box-continuity", ["isolated", "joined"] as const, "isolated"),background: color(recipe, "active-box-background", "#FFD54A"),borderColor: color(recipe, "active-box-border-color", "#00000000"),diff --git a/packages/caption-fine/src/surface.ts b/packages/caption-fine/src/surface.tsindex 0390f2ba..9303a3cb 100644--- a/packages/caption-fine/src/surface.ts+++ b/packages/caption-fine/src/surface.ts@@ -1,8 +1,8 @@-import { captionTypes } from "@hypit/caption";+import { captionTypes, captionProducers } from "@hypit/caption";import { assertFontArtifactRef, assertFontStackRef, mediaTypes } from "@hypit/media";import type { FontArtifactRef, FontStackRef } from "@hypit/media";import { narrativeTypes } from "@hypit/narrative";-import { semanticTrackTypes } from "@hypit/semantic-track";+import { timelineTypes } from "@hypit/timeline";import { spatialTypes } from "@hypit/spatial";import { svsRecipeType } from "@hypit/svs";import type { SvsRecipe } from "@hypit/svs";@@ -13,7 +13,16 @@ import type {MarkupAttributeValue,} from "@hypit/markup";-import { fineCaptionRegionTrackFragment, fineCaptionTrackFragment } from "./fragment.js";+import { sealGraphFragment } from "@hypit/elaborator";+import type { FragmentOperation } from "@hypit/elaborator";+import type { SurfaceRecordDraft, SurfaceComponentDraft } from "@hypit/markup";+import { assertEmptyElement, optionalTextAttribute } from "@hypit/markup";+import { createTemporalWindowProjection, resolveTemporalContext, temporalWindowAttributeNames } from "@hypit/temporal-markup";+import { temporalTypes } from "@hypit/temporal";+import { compositionTypes } from "@hypit/composition";+import { captionFineProducers, captionFineTypes } from "./manifest.js";+const input = (name: string) => ({ kind: "fragment-input" as const, name });+const operation = (id: string) => ({ kind: "fragment-operation" as const, operation: id });import { fineCaptionStyle } from "./style.js";function sameType(left: SurfaceResolvedReference["type"], right: SurfaceResolvedReference["type"]): boolean {@@ -114,39 +123,57 @@ export const decodeFineCaptionStyleSurface: StructuredSurfaceHandler = ({ elemen};export const decodeFineCaptionTrackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {- attributes(element, ["id", "document", "semantic", "program"], ["regions"]);- if (element.children.some((child) => child.kind === "element" || child.value.trim())) {- throw new Error(`${element.name} does not accept children`);- }+ attributes(element, ["id", "document", "timeline"], ["regions"]);const id = stringAttribute(element, "id");const document = reference(element, "document", narrativeTypes.captionDocument, resolveReference);- const semantic = reference(element, "semantic", semanticTrackTypes.track, resolveReference);- const program = reference(element, "program", captionTypes.program, resolveReference);- if (element.attributes.regions !== undefined) {- const regions = reference(element, "regions", spatialTypes.regionTimeline, resolveReference);- return {- records: [],- components: [{- id,- fragment: fineCaptionRegionTrackFragment.id,- inputs: { document: document.ref, semantic: semantic.ref, program: program.ref, regions: regions.ref },- outputs: { schedule: `${id}.schedule`, track: `${id}.track` },- range: element.range,- }],- fragments: [fineCaptionRegionTrackFragment],- };+ const context = resolveTemporalContext({ element, resolveReference });+ const regions = element.attributes.regions === undefined ? undefined : reference(element, "regions", spatialTypes.regionTimeline, resolveReference);+ const records: SurfaceRecordDraft[] = [{ id: `${id}.header`, type: captionTypes.header,+ value: { kind: "inline", value: { id } }, range: element.range }];+ const components: SurfaceComponentDraft[] = [];+ const fragments: ReturnType<typeof sealGraphFragment>[] = [];+ const inputs: { name: string; type: SurfaceResolvedReference["type"] }[] = [{ name: "document", type: narrativeTypes.captionDocument }, { name: "timeline", type: timelineTypes.track }, { name: "header", type: captionTypes.header }];+ const bindings: Record<string, SurfaceResolvedReference["ref"]> = { document: document.ref, timeline: context.timeline.ref, header: { kind: "record", id: `${id}.header` } };+ const operations: FragmentOperation[] = [{ id: "create", producer: captionProducers.create,+ inputs: { document: input("document"), header: input("header") }, result: { kind: "output", name: "program" } }];+ let previous = "create", index = 0;+ for (const child of element.children) {+ if (child.kind === "text") { if (child.value.trim()) throw new Error("Caption Track accepts Use children."); continue; }+ if (localName(child.name) !== "Use") throw new Error("Caption Track accepts Use children.");+ attributes(child, ["style"], ["id", "role", ...temporalWindowAttributeNames]);+ assertEmptyElement(child);+ index += 1;+ const useId = optionalTextAttribute(child, "id") ?? `${id}.use.${index}`;+ const hasTime = temporalWindowAttributeNames.some(name => child.attributes[name] !== undefined);+ const temporal = createTemporalWindowProjection({ id: useId, ...context, resolveReference,+ element: hasTime ? child : { ...child, attributes: { ...child.attributes, during: "program" } } });+ records.push(...temporal.records); components.push(...temporal.components); fragments.push(...temporal.fragments);+ const style = reference(child, "style", captionTypes.style, resolveReference);+ const role = optionalTextAttribute(child, "role");+ const filterId = `${useId}.filter`;+ records.push({ id: filterId, type: captionTypes.filter, value: { kind: "inline", value: role === undefined ? {} : { role } }, range: child.range });+ const key = `use-${index}`;+ inputs.push({ name: `${key}-window`, type: temporalTypes.window }, { name: `${key}-style`, type: captionTypes.style }, { name: `${key}-filter`, type: captionTypes.filter });+ bindings[`${key}-window`] = temporal.ref; bindings[`${key}-style`] = style.ref; bindings[`${key}-filter`] = { kind: "record", id: filterId };+ operations.push({ id: key, producer: captionProducers.append,+ inputs: { program: operation(previous), window: input(`${key}-window`), style: input(`${key}-style`), filter: input(`${key}-filter`) }, result: { kind: "output", name: "program" } });+ previous = key;}- return {- records: [],- components: [{- id,- fragment: fineCaptionTrackFragment.id,- inputs: {- document: document.ref, semantic: semantic.ref, program: program.ref,- },- outputs: { schedule: `${id}.schedule`, track: `${id}.track` },- range: element.range,- }],- fragments: [fineCaptionTrackFragment],- };+ operations.push({ id: "content", producer: captionProducers.temporalizeDocument,+ inputs: { document: input("document"), timeline: input("timeline") }, result: { kind: "output", name: "caption" } });+ operations.push({ id: "schedule", producer: captionFineProducers.schedule,+ inputs: { caption: operation("content"), document: input("document"), program: operation(previous) }, result: { kind: "output", name: "schedule" } });+ if (regions !== undefined) { inputs.push({ name: "regions", type: spatialTypes.regionTimeline }); bindings.regions = regions.ref; }+ operations.push({ id: "render", producer: regions === undefined ? captionFineProducers.render : captionFineProducers.renderWithRegions,+ inputs: { schedule: operation("schedule"), document: input("document"), timeline: input("timeline"), program: operation(previous), ...(regions === undefined ? {} : { regions: input("regions") }) }, result: { kind: "output", name: "track" } });+ const collector = sealGraphFragment({ inputs, operations, exports: [+ { name: "content", type: captionTypes.timedProjection, root: operation("content") },+ { name: "program", type: captionTypes.program, root: operation(previous) },+ { name: "schedule", type: captionFineTypes.schedule, root: operation("schedule") },+ { name: "track", type: compositionTypes.visualTrack, root: operation("render") },+ ] });+ fragments.push(collector);+ components.push({ id, fragment: collector.id, inputs: bindings,+ outputs: { content: `${id}.content`, program: `${id}.program`, schedule: `${id}.schedule`, track: `${id}.track` }, range: element.range });+ return { records, components, fragments, exports: [`${id}.content`, `${id}.program`, `${id}.schedule`, `${id}.track`] };};diff --git a/packages/caption-fine/src/types.ts b/packages/caption-fine/src/types.tsindex cb388334..d6eb1160 100644--- a/packages/caption-fine/src/types.ts+++ b/packages/caption-fine/src/types.ts@@ -173,6 +173,8 @@ export type FineCaptionParameters = {};export type FineCaptionScheduledCue = {+ readonly cueId: string;+ readonly visibility: readonly { readonly startFrame: number; readonly endFrameExclusive: number }[];readonly id: string;readonly styleId: string;readonly semanticStartFrame: number;diff --git a/packages/caption-fine/test/caption-fine.test.ts b/packages/caption-fine/test/caption-fine.test.tsindex c2a331d1..75b3614b 100644--- a/packages/caption-fine/test/caption-fine.test.ts+++ b/packages/caption-fine/test/caption-fine.test.ts@@ -1,3 +1,5 @@+import { projectProgramWindow } from "../../../test/temporal-fixture.js";+import { sealTimeline } from "@hypit/timeline";import assert from "node:assert/strict";import test from "node:test";@@ -72,9 +74,10 @@ function fixture(script = "<line>one two || three four</line>") {id: "captions",documentId: document.id,styles: [style],- runs: [{ id: "captions:run:1", styleId: style.id, unitIds: document.units.map((unit) => unit.id) }],- wordRuns: [],- mutedUnitIds: [],+ uses: [{ styleId: style.id, window: projectProgramWindow({ itemId: "captions.use.1",+ semantic: sealTimeline({ id: "test-space", durationSec: 3, frameRate: { numerator: 30, denominator: 1 }, items: [] }),+ projection: { start: { ref: "program.start" }, end: { ref: "program.end" } },+ }) }],};const [first, second, third, fourth] = document.units;assert.ok(first && second && third && fourth);@@ -84,14 +87,14 @@ function fixture(script = "<line>one two || three four</line>") {documentId: document.id,cues: [{- id: "captions:cue:1", styleId: style.id, startFrame: 10, endFrameExclusive: 30,+ id: "captions:cue:1", startFrame: 10, endFrameExclusive: 30,units: [{ unitId: first.id, startFrame: 10, endFrameExclusive: 20 },{ unitId: second.id, startFrame: 20, endFrameExclusive: 30 },],},{- id: "captions:cue:2", styleId: style.id, startFrame: 36, endFrameExclusive: 56,+ id: "captions:cue:2", startFrame: 36, endFrameExclusive: 56,units: [{ unitId: third.id, startFrame: 36, endFrameExclusive: 46 },{ unitId: fourth.id, startFrame: 46, endFrameExclusive: 56 },@@ -127,7 +130,7 @@ test("Fine Caption schedules visibility outside semantic Word timing and cuts on]);const track = renderFineCaption(schedule, program, document,- sealProgramSpace({ id: "test-space", durationSec: 3, frameRate: { numerator: 30, denominator: 1 } }));+ sealTimeline({ items: [], id: "test-space", durationSec: 3, frameRate: { numerator: 30, denominator: 1 } }));assert.deepEqual(track.presents.map((present) => present.span), [{ startFrame: 6, endFrameExclusive: 36 },{ startFrame: 36, endFrameExclusive: 62 },@@ -157,7 +160,7 @@ test("Fine Caption uniformly springs the whole Cue through exact scales", () =>scheduleFineCaption(projection, animatedProgram, document),animatedProgram,document,- sealProgramSpace({ id: "test-space", durationSec: 3,+ sealTimeline({ items: [], id: "test-space", durationSec: 3,frameRate: { numerator: 30, denominator: 1 } }),);const cueMotion = track.presents[0]?.elements.find((element) => element.id === "cue-motion");@@ -189,7 +192,7 @@ test("Fine Caption follows measured Role regions and hides null Frames", () => {const { document, program, projection } = fixture(`<line><BOY>one two || three four</line>`);- const space = sealProgramSpace({ id: "test-space", durationSec: 3,+ const space = sealTimeline({ items: [], id: "test-space", durationSec: 3,frameRate: { numerator: 30, denominator: 1 } });const regions: SpatialRegionTimeline = {canvas: { widthPx: 1080, heightPx: 1920, origin: "top-left", xDirection: "right", yDirection: "down", pixelAspect: "square" },@@ -217,7 +220,7 @@ test("Fine Caption keeps authored placement when a Cue Role has no measured tracconst { document, program, projection } = fixture(`<line><BOY>one two || three four</line>`);- const space = sealProgramSpace({ id: "test-space", durationSec: 3,+ const space = sealTimeline({ items: [], id: "test-space", durationSec: 3,frameRate: { numerator: 30, denominator: 1 } });const regions: SpatialRegionTimeline = {canvas: { widthPx: 1080, heightPx: 1920, origin: "top-left", xDirection: "right", yDirection: "down", pixelAspect: "square" },@@ -241,18 +244,24 @@ test("Fine Caption rejects a Cue that exceeds its structural row budget insteadconst schedule = scheduleFineCaption(projection, constrainedProgram, document);assert.throws(() => renderFineCaption(schedule, constrainedProgram, document,- sealProgramSpace({ id: "test-space", durationSec: 3,+ sealTimeline({ items: [], id: "test-space", durationSec: 3,frameRate: { numerator: 30, denominator: 1 } })),/constructs 2 rows.+maximum is 1/u,);});-test("Fine Caption applies authored mute before it schedules the visible envelope", () => {+test("Fine Caption gives a hidden Style no Cue, background or decoration", () => {const { document, program, projection } = fixture();- const muted: CaptionProgram = { ...program, mutedUnitIds: [projection.cues[0]!.units[0]!.unitId] };- const schedule = scheduleFineCaption(projection, muted, document);- assert.deepEqual(schedule.cues[0]?.units, [projection.cues[0]!.units[1]]);- assert.equal(schedule.cues[0]?.semanticStartFrame, projection.cues[0]!.units[1]!.startFrame);+ const hidden = { id: "hidden", rendering: null };+ const hiddenProgram: CaptionProgram = { ...program, styles: [hidden],+ uses: program.uses.map(use => ({ ...use, styleId: hidden.id })) };+ const hiddenProjection = { ...projection, cues: [] };+ const schedule = scheduleFineCaption(hiddenProjection, hiddenProgram, document);+ const visual = renderFineCaption(schedule, hiddenProgram, document,+ sealTimeline({ items: [], id: "test-space", durationSec: 3,+ frameRate: { numerator: 30, denominator: 1 } }));+ assert.deepEqual(schedule.cues, []);+ assert.deepEqual(visual.presents, []);});test("Fine Caption gives overlapping acoustic Words one current Karaoke owner", () => {@@ -277,7 +286,7 @@ test("Fine Caption gives overlapping acoustic Words one current Karaoke owner",scheduleFineCaption(overlapped, karaokeProgram, document),karaokeProgram,document,- sealProgramSpace({ id: "test-space", durationSec: 3,+ sealTimeline({ items: [], id: "test-space", durationSec: 3,frameRate: { numerator: 30, denominator: 1 } }),);const firstActive = track.presents[0]?.elements.find((element) => element.id === "atom-1-active");@@ -290,19 +299,6 @@ test("Fine Caption gives overlapping acoustic Words one current Karaoke owner",assert.equal(opacityAt(secondActive, 14), 1);});-test("Fine Caption rejects token-specific Style runs instead of disguising a structural caption", () => {- const { document, program, projection } = fixture();- const word = document.words[0]!;- const styled: CaptionProgram = {- ...program,- wordRuns: [{ id: "captions:word-run:1", styleId: "plain", wordIds: [word.id] }],- };- assert.throws(- () => scheduleFineCaption(projection, styled, document),- /one uniform token rule/u,- );-});-function unevenChineseCaption(properties: SvsRecipe["properties"] = {}) {const { document, program, projection } = fixture("<line>你真好看</line>");const { "max-lines": _lines, "max-words-per-line": _words, ...flow } = recipe.properties;@@ -319,7 +315,7 @@ function unevenChineseCaption(properties: SvsRecipe["properties"] = {}) {};const styled = { ...program, styles: [style] };const track = renderFineCaption(scheduleFineCaption(timed, styled, document), styled, document,- sealProgramSpace({ id: "test-space", durationSec: 3, frameRate: { numerator: 30, denominator: 1 } }));+ sealTimeline({ items: [], id: "test-space", durationSec: 3, frameRate: { numerator: 30, denominator: 1 } }));return { track, windows };}@@ -390,7 +386,7 @@ test("Typewriter keeps combining graphemes whole and preserves unequal-width Latproperties: { ...recipe.properties, "atom-reveal": "typewriter" } }, [font]);const styled = { ...program, styles: [style] };const track = renderFineCaption(scheduleFineCaption(projection, styled, document), styled, document,- sealProgramSpace({ id: "test-space", durationSec: 3, frameRate: { numerator: 30, denominator: 1 } }));+ sealTimeline({ items: [], id: "test-space", durationSec: 3, frameRate: { numerator: 30, denominator: 1 } }));const text = track.presents[0]!.elements.filter((element) => element.kind === "text");assert.deepEqual(text.map((element) => element.text), ["W", "i", "e\u0301"]);assert.equal(numericStyleAt(text[0]!, 4, "opacity"), 1);@@ -398,3 +394,127 @@ test("Typewriter keeps combining graphemes whole and preserves unequal-width Latassert.equal(numericStyleAt(text[1]!, 9, "opacity"), 1);assert.equal(numericStyleAt(text[2]!, 14, "opacity"), 1);});++test("Caption handoff follows placed time and retains simultaneous spoken envelopes", () => {+ const { document, program, projection } = fixture();+ const first = projection.cues[0]!;+ const second = projection.cues[1]!;+ const overlapping = { ...second, startFrame: 20, endFrameExclusive: 40,+ units: second.units.map(unit => ({ ...unit, startFrame: unit.startFrame - 16,+ endFrameExclusive: unit.endFrameExclusive - 16 })) };+ // Reverse declaration order: placement decides temporal neighbors, not Script order.+ const schedule = scheduleFineCaption({ ...projection, cues: [overlapping, first] }, program, document);+ assert.deepEqual(schedule.cues.map(cue => [cue.semanticStartFrame, cue.semanticEndFrameExclusive]), [[10, 30], [20, 40]]);+ assert.deepEqual(schedule.cues.map(cue => [cue.visibleStartFrame, cue.visibleEndFrameExclusive]), [[6, 36], [16, 46]]);+ const track = renderFineCaption(schedule, program, document,+ sealTimeline({ items: [], id: "test-space", durationSec: 3, frameRate: { numerator: 30, denominator: 1 } }));+ assert.equal(track.presents.filter(present => present.span.startFrame <= 25 && present.span.endFrameExclusive > 25).length, 2);+});+++test("Caption Surface wires one Timeline to both ordinary and tracked-region rendering", async () => {+ const { decodeFineCaptionTrackSurface } = await import("../src/surface.js");+ const { timelineTypes } = await import("@hypit/timeline");+ const { narrativeTypes } = await import("@hypit/narrative");+ const { captionTypes } = await import("@hypit/caption");+ const { spatialTypes } = await import("@hypit/spatial");+ const types = { timeline: timelineTypes.track, document: narrativeTypes.captionDocument,+ program: captionTypes.program, regions: spatialTypes.regionTimeline };+ for (const tracking of [false, true]) {+ const names = tracking ? ["timeline", "document", "regions"] as const+ : ["timeline", "document"] as const;+ const attributes = Object.fromEntries(names.map(name => [name, { kind: "reference" as const, path: name }]));+ const result = await decodeFineCaptionTrackSurface({ sourceName: "caption.svml",+ element: { kind: "element", name: "fine:Track", attributes: { id: "captions", ...attributes },+ children: [], range: { start: 0, end: 1 } },+ resolveReference(path) {+ const type = types[path as keyof typeof types];+ return type === undefined ? undefined : { path, type, ref: { kind: "record", id: path } };+ },+ resolveAsset() { throw new Error("Caption assembly uses declared inputs."); },+ });+ const component = result.components[0]!;+ const fragment = result.fragments!.find(fragment => fragment.id === component.fragment)!;+ assert.deepEqual(Object.keys(component.inputs).sort(), fragment.inputs.map(input => input.name).sort());+ assert.deepEqual(component.inputs.timeline, { kind: "record", id: "timeline" });+ assert.equal(fragment.operations.some(operation => operation.producer.name === "project-program-space"), false);+ }+});++test("a Use beginning inside a Cue changes presentation without changing words or animation time", () => {+ const { document, program } = fixture("<line>test1 || test2 @select test3 || test4 @/select</line>");+ const timeline = sealTimeline({ id: "test-space", durationSec: 3, frameRate: { numerator: 30, denominator: 1 }, items: [] });+ const unit = (index: number) => ({ unitId: document.units[index]!.id, startFrame: 10 + index * 10, endFrameExclusive: 20 + index * 10 });+ const projection: TimedCaptionProjection = { spaceId: timeline.id, narrativeId: "story", documentId: document.id, cues: [+ { id: "c1", startFrame: 10, endFrameExclusive: 20, units: [unit(0)] },+ { id: "c2", startFrame: 20, endFrameExclusive: 40, units: [unit(1), unit(2)] },+ { id: "c3", startFrame: 40, endFrameExclusive: 50, units: [unit(3)] },+ ] };+ const emphasis = fineCaptionStyle("emphasis", { ...recipe, properties: { ...recipe.properties, "active-fill": "#FF6347", karaoke: "trail" } }, [font]);+ const window = (id:string,start:number,end:number) => projectProgramWindow({itemId:id,semantic:timeline,projection:{start:{ref:"absolute",at:{unit:"frames",value:start}},end:{ref:"absolute",at:{unit:"frames",value:end}}}});+ const changed: CaptionProgram = { ...program, styles: [...program.styles, emphasis], uses: [...program.uses, { styleId: emphasis.id, window: window("emphasis",30,50) }] };+ const schedule = scheduleFineCaption(projection, changed, document);+ const parts = schedule.cues.filter(cue=>cue.cueId==="c2");+ assert.equal(parts.length,2);+ assert.deepEqual(parts.map(cue=>cue.units), [[unit(1),unit(2)],[unit(1),unit(2)]]);+ assert.deepEqual(parts.map(cue=>cue.visibility), [[{startFrame:20,endFrameExclusive:30}],[{startFrame:30,endFrameExclusive:40}]]);+ assert.deepEqual(parts.map(cue=>cue.visibleStartFrame),[20,20]);+ const track = renderFineCaption(schedule,changed,document,timeline);+ const entered = track.presents.find(present=>present.id === parts[1]!.id)!;+ assert.equal(entered.span.startFrame,20);+ assert.deepEqual(entered.visibility,[{startFrame:30,endFrameExclusive:40}]);+ // Rendering the same Style over the whole timeline has identical animation data for this Cue.+ const whole: CaptionProgram = {...changed,uses:[{styleId:emphasis.id,window:window("whole",0,90)}]};+ const original = renderFineCaption(scheduleFineCaption(projection,whole,document),whole,document,timeline).presents.find(p=>p.id.endsWith(":c2"))!;+ assert.deepEqual(entered.elements,original.elements);++ const hidden: CaptionProgram = {...changed,styles:[...changed.styles,{id:"hidden",rendering:null}],uses:[...changed.uses,{styleId:"hidden",window:window("hide",32,35)}]};+ const resumed = scheduleFineCaption(projection,hidden,document).cues.find(cue=>cue.id===parts[1]!.id)!;+ assert.deepEqual(resumed.visibility,[{startFrame:30,endFrameExclusive:32},{startFrame:35,endFrameExclusive:40}]);+ assert.deepEqual(resumed.units,parts[1]!.units);+ assert.equal(resumed.visibleStartFrame,20);+});++test("Caption time windows select presentation, while Role filters preserve simultaneous speakers", () => {+ const { document, program, projection } = fixture("<line><A>one two || <B>three four</line>");+ const timeline = sealTimeline({id:"test-space",durationSec:3,frameRate:{numerator:30,denominator:1},items:[]});+ const overlap: TimedCaptionProjection = {...projection,cues:[projection.cues[0]!,{+ ...projection.cues[1]!,startFrame:15,endFrameExclusive:35,+ units:projection.cues[1]!.units.map((u,i)=>({...u,startFrame:15+i*10,endFrameExclusive:25+i*10})),+ }]};+ const hidden: CaptionProgram = {...program,styles:[...program.styles,{id:"hidden",rendering:null}],uses:[...program.uses,{+ role:"A",styleId:"hidden",window:projectProgramWindow({itemId:"hideA",semantic:timeline,projection:{start:{ref:"program.start"},end:{ref:"program.end"}}}),+ }]};+ const schedule = scheduleFineCaption(overlap,hidden,document);+ assert.equal(schedule.cues.length,1);+ assert.deepEqual(schedule.cues[0]!.units,overlap.cues[1]!.units);+ const noContent = scheduleFineCaption({...projection,cues:[]},program,document);+ assert.deepEqual(noContent.cues,[]);+ assert.deepEqual(scheduleFineCaption(projection,{...program,uses:[],styles:[]},document).cues,[]);+});++test("Caption Track Uses share the temporal author language and reject the removed Program edge", async () => {+ const { decodeFineCaptionTrackSurface } = await import("../src/surface.js");+ const { parseStructuredElement } = await import("@hypit/markup");+ const { timelineTypes } = await import("@hypit/timeline");+ const { narrativeTypes } = await import("@hypit/narrative");+ const { captionTypes } = await import("@hypit/caption");+ const refs: Record<string, import("@hypit/protocol").TypeRef> = { timeline: timelineTypes.track, document: narrativeTypes.captionDocument, style: captionTypes.style, selection:narrativeTypes.selection, moment:narrativeTypes.moment };+ const decode = async (body:string,extra="") => decodeFineCaptionTrackSurface({sourceName:"caption.svml",+ element:parseStructuredElement({name:"caption.svml",text:`<fine:Track id="captions" document={document} timeline={timeline} ${extra}>${body}</fine:Track>`},0).element,+ resolveReference(path){const type=refs[path];return type===undefined?undefined:{path,type,ref:{kind:"record" as const,id:path}};},+ resolveAsset(){throw new Error("No assets");},+ });+ for (const time of ['', 'during={selection}', 'at={moment} for="2s"', 'until={moment} for="12f"', 'start="2s" end="5s"']) {+ const result=await decode(`<fine:Use style={style} ${time}/>`);+ const track=result.components.find(component=>component.id==="captions")!;+ assert.ok(track.outputs.program);+ assert.ok(track.outputs.content);+ const fragment=result.fragments!.find(fragment=>fragment.id===track.fragment)!;+ assert.equal(fragment.operations.find(op=>op.id==="content")?.result.name,"caption");+ assert.ok(fragment.inputs.some(port=>port.name.endsWith("-window")));+ assert.equal(fragment.inputs.some(port=>port.name==="program"),false);+ }+ await assert.rejects(()=>decode('', 'program={old}'),/requires id, document, timeline/);+ await assert.rejects(()=>decode('<fine:Use style={style} selection={selection}/>'),/requires exactly one/);+});diff --git a/packages/caption/README.md b/packages/caption/README.mdindex 5f86413a..b8649c05 100644--- a/packages/caption/README.md+++ b/packages/caption/README.md@@ -1,77 +1,65 @@# `@hypit/caption`-External components use `@hypit/hypit/caption` from their `@hypit/hypit` development dependency. The package-owns the types and helpers below; Source imports retain the `@hypit/caption@1` Module identity.---The deterministic Caption contract. Script emits one `CaptionDocument` containing display words,-N:M alignment units and authored `||` cue breaks. Caption applies Styles to complete units selected-by a Role or a semantic Selection, then joins those units to a `SemanticTrack` for frame timing.--```xml-<caption:Program id="captions" document={story.caption} narrative={story} default={plain}>- <caption:Use role="HOST" style={impact}/>- <caption:Use selection={story.selection.emphasis} style={plain}/>- <caption:Use attribute="emphasis" style={keyword}/>-</caption:Program>+External components import `@hypit/hypit/caption`. Source uses the `@hypit/caption@1` Module+identity for shared declarations such as `Hidden`.++Caption has two independent structures: Script organizes displayed words into Cues, and timed Uses+choose how those Cues appear. A Use may begin inside a Cue. It changes presentation without changing+that Cue's text or restarting its word timing.++`CaptionDocument` owns displayed words, display/pronunciation associations, word attributes and `||`+breaks. `Timeline` owns placed Takes and measured word times. `temporalizeCaptionDocument(document,+timeline)` joins their identities and returns `TimedCaptionProjection`: complete Cues with original+unit times. Segment, speaking-turn and explicit Cue boundaries organize this content. No Style is+needed for this operation. Unplaced material contributes no Cue.++A rendering family's Track accepts `document`, `timeline` and ordered `Use` children. Fine provides:++```svml+<caption:Hidden id="hidden"/>+<caption-fine:Track id="captions" document={story.caption} timeline={film.timeline}>+ <caption-fine:Use style={plain}/>+ <caption-fine:Use role="GUEST" style={guest}/>+ <caption-fine:Use during={story.selection.demo} style={hidden}/>+ <caption-fine:Use at={story.moment.key} for="2s" style={impact}/>+</caption-fine:Track>```-There is no planner, model call or frame range in this authoring contract. A Selection is projected-to complete alignment units before timing; a Selection that cuts through an N:M unit is rejected.-Script-native word attributes are resolved separately as local word Style runs and do not create-another timing source. They are part of the common Caption contract for structural Caption families;-`@hypit/caption-fine` deliberately rejects them because Fine requires one uniform token rule per Cue.--`TimedCaptionProjection` is intentionally not a `TemporalWindow` collection. Cue timing is a-read-only projection of token evidence, not an author-controlled placement window. It nevertheless-carries the exact `spaceId`, `narrativeId` and `documentId`, so another SemanticTrack or Script cannot-be substituted silently.--## Extending with another rendering family--The common public values are defined in [types.ts](src/types.ts), with exports in-[index.ts](src/index.ts). A family supplies:--- a Style decoder producing `CaptionStyleIntent` with its own `rendering.family` and validated- `rendering.parameters`;-- an optional family schedule Type and deterministic scheduling Producer;-- a rendering Producer returning the ordinary `VisualTrack` Type;-- a Surface/Fragment connecting the common document, program and timing to those Producers.--`caption:Program` remains the common author Surface. `Use` selects exactly one of `role`, `selection`-or `attribute`; `Mute` selects a Role or Selection. For Role/Selection applications, the last matching-`Use` in Source order selects the complete Style for each unit. It replaces the previous choice-without merging Style parameters. A Selection can cover a phrase, a whole Segment, several Segments-or the program through its structural anchors; there is no separate Segment selector.--Word attributes produce separate `wordRuns`. Conflicting Styles on the same attributed word are-rejected; these runs do not use the Role/Selection override rule. A family chooses whether and how-it supports word-specific runs.--Mute applications combine their selected unit ids independently of Style order. They suppress those-display words, preserving the speech and semantic timeline. Visible lead and tail still belong to-the family schedule, so muting a word is not a time-based mask of every neighboring Cue.--Reuse `captionProducers.temporalizeDocument` with inputs `document`, `semantic` and `program`, or the-corresponding exported timing Fragment. It emits a `TimedCaptionProjection` whose Cues contain-`styleId`, `startFrame`, `endFrameExclusive` and measured `units`. Each unit carries `unitId` and its-own boundaries. Recover display Words and their associations from the same `CaptionDocument`; do-not infer N:M timing by evenly dividing a Cue or splitting its rendered string.--Scheduling may derive visible intervals while preserving those measured boundaries. Use-`assertCaptionProgramForDocument`, `assertTimedCaptionProjection` and, for visible scheduling,-`applyCaptionMute` as appropriate. Retain `spaceId`, `narrativeId` and `documentId` in a schedule so-the renderer can check its inputs. The renderer should consume that schedule without silently-changing Cue grouping, lead or tail again.--`@hypit/caption-fine` provides a working example in its `src/fragment.ts`, `src/schedule.ts` and-`src/render.ts`: one shared timing join, then family schedule and render operations. A new structural-family replaces Fine's uniform-word layout and parameters, not the common speech truth. Its own-Manifest, activation and vocabulary belong to the project package; no change to this common package-is required merely to add a new visual family.--That family can render a standalone Caption Track or coordinate caption text with graphics in a-shared scene. Its output uses ordinary VisualTrack elements or a renderer program with explicit-resources. Caption retains the authored words and their semantic relationship while the component-owns the visual boundary. The Program does not automatically route different families to different-renderers; choose the implementing component and its coverage explicitly.+Time attributes come from `@hypit/temporal-markup`: `during`, `at`/`for`, `until`/`for`, and+`start`/`end`, including semantic references and explicit frame/second expressions. Omitted time+attributes mean the whole Timeline. `role` filters content independently of time. Later matching+Uses replace earlier presentation inside their windows, including a Hidden Style. Separate Tracks+remain independent and can intentionally display simultaneous captions.++## Rendering-family extension++A family owns its Style, schedule, renderer and Track Surface. It can use ordinary VisualTrack+objects or an explicit browser program; no central renderer dispatch is required.++The public helpers and Types are in [index.ts](src/index.ts):++- `CaptionStyleIntent` carries a family identifier and parameters, or `rendering: null` for Hidden.+- `CaptionProgram` is the Track's internal collection of ordered, resolved Uses and referenced Styles.+ It is not a separate author element. `create-caption-uses` and `append-caption-use` assemble it from+ typed Windows. The Track may export it for its Companion, like Performance and Sound.+- `captionUseVisibility(program, index, role, envelope)` intersects a Cue envelope with a Use and+ subtracts later matching windows. Hidden participates even though it renders nothing.+- `captionProducers.temporalizeDocument` takes only `document` and `timeline`. Its output retains+ `spaceId`, `narrativeId`, `documentId`, Cue identities and measured alignment-unit boundaries.++Derive layout and animation from complete Cue content and original timing. Apply Use coverage as a+visibility mask. Fine keeps the original Present span and element animations, with separate+`visibility` intervals; changing or briefly hiding a Style does not restart karaoke, typing or motion.+A family may define lead/tail and handoff behavior, but its resulting visibility stays inside the+winning Use window. Empty content produces no drawing.++Word attributes remain on `CaptionDocument.words`. A structural family can interpret an explicit+attribute as a keyword role while retaining complete display/alignment units. Time selection does+not split `<display|speech>` text, and elapsed Cue progress is not a replacement for word timing.+The content query helpers for Role, Selection and attributes remain available for components that+actually need authored word subsets; those queries do not define the Use time language.++See Fine's [Surface](../caption-fine/src/surface.ts), [schedule](../caption-fine/src/schedule.ts) and+[renderer](../caption-fine/src/render.ts) for a concrete implementation. New family behavior belongs+in its own project package. The selected family interprets its own Styles; mixed-family dispatch,+when useful, is an explicit component responsibility.diff --git a/packages/caption/package.json b/packages/caption/package.jsonindex 3984594b..91987b8c 100644--- a/packages/caption/package.json+++ b/packages/caption/package.json@@ -15,8 +15,9 @@"@hypit/elaborator": "workspace:*","@hypit/narrative": "workspace:*","@hypit/protocol": "workspace:*",- "@hypit/semantic-track": "workspace:*",- "@hypit/markup": "workspace:*"+ "@hypit/timeline": "workspace:*",+ "@hypit/markup": "workspace:*",+ "@hypit/temporal": "workspace:*"},"devDependencies": {"@hypit/program-space": "workspace:*",diff --git a/packages/caption/src/activation.ts b/packages/caption/src/activation.tsindex 58a14603..cce2649c 100644--- a/packages/caption/src/activation.ts+++ b/packages/caption/src/activation.ts@@ -1,7 +1,7 @@import { createMarkupSurfaceHostFacet } from "@hypit/markup";import {captionComponent, captionManifest, captionModuleRef,- decodeCaptionProgramSurface,+ decodeHiddenCaptionStyleSurface,captionMarkupSurfaces,} from "./index.js";@@ -11,7 +11,7 @@ export const hypitPackage = {components: [captionComponent],hostFacets: [createMarkupSurfaceHostFacet({ module: captionModuleRef,- declaration: captionMarkupSurfaces.find((item) => item.name === "program")!, handler: decodeCaptionProgramSurface }),+ declaration: captionMarkupSurfaces.find((item) => item.name === "hidden")!, handler: decodeHiddenCaptionStyleSurface }),],};export default hypitPackage;diff --git a/packages/caption/src/component.ts b/packages/caption/src/component.tsindex 2494e1f5..09690c72 100644--- a/packages/caption/src/component.ts+++ b/packages/caption/src/component.ts@@ -1,11 +1,12 @@+import type { TemporalWindow } from "@hypit/temporal";import type { ComponentPackage } from "@hypit/component-kit";import type { CaptionDocument } from "@hypit/narrative";-import type { SemanticTrack } from "@hypit/semantic-track";+import type { Timeline } from "@hypit/timeline";import type { StoredValue } from "@hypit/protocol";import { canonicalize } from "@hypit/protocol";import { captionProducers, captionTypes } from "./manifest.js";-import { assertCaptionProgram } from "./style.js";+import { appendCaptionUse, assertCaptionProgram, assertCaptionStyle } from "./style.js";import { assertTimedCaptionProjection, temporalizeCaptionDocument } from "./temporalize.js";import type { CaptionProgram, CaptionStyleIntent, TimedCaptionProjection } from "./types.js";@@ -16,12 +17,26 @@ function inline<T>(value: StoredValue | undefined, subject: string): T {export const captionComponent = {producers: [{+ producer: captionProducers.create,+ handler: ({ inputs }) => ({ outputs: { program: { kind: "inline", value: canonicalize({+ id: inline<{id:string}>(inputs.header?.value, "CaptionTrackSpec").id,+ documentId: inline<CaptionDocument>(inputs.document?.value, "CaptionDocument").id,+ styles: [], uses: [],+ }) } }, needs: {} }),+ }, {+ producer: captionProducers.append,+ handler: ({ inputs }) => ({ outputs: { program: { kind: "inline", value: canonicalize(appendCaptionUse(+ inline<CaptionProgram>(inputs.program?.value, "CaptionProgram"),+ inline<TemporalWindow>(inputs.window?.value, "TemporalWindow"),+ inline<CaptionStyleIntent>(inputs.style?.value, "CaptionStyle"),+ inline<{role?:string}>(inputs.filter?.value, "CaptionContentFilter").role,+ )) } }, needs: {} }),+ }, {producer: captionProducers.temporalizeDocument,handler: ({ inputs }) => ({outputs: { caption: { kind: "inline", value: canonicalize(temporalizeCaptionDocument(inline<CaptionDocument>(inputs.document?.value, "CaptionDocument"),- inline<SemanticTrack>(inputs.semantic?.value, "SemanticTrack"),- inline<CaptionProgram>(inputs.program?.value, "CaptionProgram"),+ inline<Timeline>(inputs.timeline?.value, "Timeline"),)) } },needs: {},}),@@ -30,7 +45,7 @@ export const captionComponent = {{ type: captionTypes.style, handler: ({ value }) => {if (value.kind !== "inline") throw new Error("CaptionStyle must be inline");const style = value.value as CaptionStyleIntent;- if (!style.id || !style.rendering?.family) throw new Error("CaptionStyle is invalid");+ assertCaptionStyle(style);} },{ type: captionTypes.program, handler: ({ value }) => assertCaptionProgram(inline<CaptionProgram>(value, "CaptionProgram")) },{ type: captionTypes.timedProjection, handler: ({ value }) => assertTimedCaptionProjection(inline<TimedCaptionProjection>(value, "TimedCaptionProjection")) },diff --git a/packages/caption/src/display.ts b/packages/caption/src/display.tsindex 15961d25..17ef1530 100644--- a/packages/caption/src/display.ts+++ b/packages/caption/src/display.ts@@ -1,3 +1,4 @@+import { narrativeSelectionTokenRange } from "@hypit/narrative";import type { CaptionDocument, Narrative, NarrativeSelection } from "@hypit/narrative";function assert(condition: unknown, message: string): asserts condition {@@ -38,30 +39,14 @@ export type CaptionUnitSubset = {readonly unitIds: readonly string[];};-function tokenBoundary(narrative: Narrative, anchorId: string, owner: string): number {- const anchor = narrative.semanticIndex.anchors.find((candidate) => candidate.id === anchorId);- if (anchor === undefined) throw new Error(`${owner} names unknown semantic anchor ${anchorId}`);- if (anchor.kind === "program-start") return 0;- if (anchor.kind === "program-end") return narrative.tokens.length;- const segment = narrative.segments.find((candidate) => candidate.id === anchor.segmentId);- if (segment === undefined) throw new Error(`${owner} names an anchor outside its Segment`);- if (anchor.kind === "segment-start") return segment.tokenStart;- if (anchor.kind === "segment-end") return segment.tokenEndExclusive;- const tokenIndex = narrative.tokens.findIndex((token) => token.id === anchor.tokenId);- if (tokenIndex < 0) throw new Error(`${owner} names an anchor without a Narrative token`);- return anchor.kind === "token-start" ? tokenIndex : tokenIndex + 1;-}-/** Project a semantic Selection to complete authored N:M Caption units. */export function captionUnitsForSelection(- document: CaptionDocument,narrative: Narrative,selection: NarrativeSelection,): CaptionUnitSubset {+ const document = narrative.caption;assertCaptionDocument(document);- const start = tokenBoundary(narrative, selection.startAnchorId, `Selection ${selection.id}`);- const end = tokenBoundary(narrative, selection.endAnchorId, `Selection ${selection.id}`);- assert(end >= start, `Selection ${selection.id} is backwards`);+ const { tokenStart: start, tokenEndExclusive: end } = narrativeSelectionTokenRange(narrative, selection);const tokenPositions = new Map(narrative.tokens.map((token, index) => [token.id, index]));const unitIds: string[] = [];for (const unit of document.units) {diff --git a/packages/caption/src/fragment.ts b/packages/caption/src/fragment.tsindex 8298d0ba..2e1759fc 100644--- a/packages/caption/src/fragment.ts+++ b/packages/caption/src/fragment.ts@@ -1,5 +1,5 @@import { narrativeTypes } from "@hypit/narrative";-import { semanticTrackTypes } from "@hypit/semantic-track";+import { timelineTypes } from "@hypit/timeline";import { sealGraphFragment } from "@hypit/elaborator";import { captionProducers, captionTypes } from "./manifest.js";@@ -10,17 +10,13 @@ const operation = (id: string) => ({ kind: "fragment-operation" as const, operatexport const captionTimingFragment = sealGraphFragment({inputs: [{ name: "document", type: narrativeTypes.captionDocument },- { name: "semantic", type: semanticTrackTypes.track },- { name: "program", type: captionTypes.program },+ { name: "timeline", type: timelineTypes.track },],operations: [{id: "temporalize-caption-document",producer: captionProducers.temporalizeDocument,- inputs: { document: input("document"), semantic: input("semantic"), program: input("program") },+ inputs: { document: input("document"), timeline: input("timeline") },result: { kind: "output", name: "caption" },}],exports: [{ name: "caption", type: captionTypes.timedProjection, root: operation("temporalize-caption-document") }],});--/** Kept as the fragment's public name for callers that only know the old export. */-export const plannedCaptionTimingFragment = captionTimingFragment;diff --git a/packages/caption/src/index.ts b/packages/caption/src/index.tsindex 71e0400e..a76760e0 100644--- a/packages/caption/src/index.ts+++ b/packages/caption/src/index.ts@@ -8,7 +8,7 @@ export {export type { CaptionUnitSubset } from "./display.js";export { captionWordsForAttribute } from "./display.js";export { captionComponent } from "./component.js";-export { captionTimingFragment, plannedCaptionTimingFragment } from "./fragment.js";+export { captionTimingFragment } from "./fragment.js";export {captionProgramSchema,captionStyleSchema,@@ -19,15 +19,16 @@ export {captionProducers,captionTypes,} from "./manifest.js";-export { decodeCaptionProgramSurface } from "./surface.js";+export { decodeHiddenCaptionStyleSurface } from "./surface.js";export {assertCaptionProgram,assertCaptionProgramForDocument,assertCaptionStyle,- resolveCaptionProgram,+ appendCaptionUse,sealCaptionProgram,sealCaptionStyle,} from "./style.js";-export type { CaptionMuteApplication, CaptionStyleApplication, CaptionWordStyleApplication } from "./style.js";-export { applyCaptionMute, assertTimedCaptionProjection, temporalizeCaptionDocument } from "./temporalize.js";+export { assertTimedCaptionProjection, temporalizeCaptionDocument } from "./temporalize.js";export type * from "./types.js";++export { captionUseVisibility } from "./visibility.js";diff --git a/packages/caption/src/manifest.ts b/packages/caption/src/manifest.tsindex af353071..21ca61b0 100644--- a/packages/caption/src/manifest.ts+++ b/packages/caption/src/manifest.ts@@ -1,12 +1,17 @@+import { temporalDependency, temporalTypes, temporalWindowSchema } from "@hypit/temporal";import { narrativeDependency, narrativeTypes } from "@hypit/narrative";-import { semanticTrackDependency, semanticTrackTypes } from "@hypit/semantic-track";+import { timelineDependency, timelineTypes } from "@hypit/timeline";import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";export const captionModuleRef = { name: "@hypit/caption", version: "1" } as const;export const captionProducers = {+ create: { module: captionModuleRef, name: "create-caption-uses" },+ append: { module: captionModuleRef, name: "append-caption-use" },temporalizeDocument: { module: captionModuleRef, name: "temporalize-caption-document" },} satisfies Record<string, ProducerRef>;export const captionTypes = {+ header: { module: captionModuleRef, name: "CaptionTrackSpec" },+ filter: { module: captionModuleRef, name: "CaptionContentFilter" },style: { module: captionModuleRef, name: "CaptionStyle" },program: { module: captionModuleRef, name: "CaptionProgram" },timedProjection: { module: captionModuleRef, name: "TimedCaptionProjection" },@@ -17,21 +22,17 @@ const integer = { kind: "number", integer: true, minimum: 0 } as const;const object = (fields: Readonly<Record<string, { readonly schema: ValueSchema; readonly optional?: boolean }>>): ValueSchema => ({ kind: "object", fields });export const captionStyleSchema: ValueSchema = object({id: { schema: string },- rendering: { schema: object({+ rendering: { schema: { kind: "oneOf", variants: [{ kind: "null" }, object({family: { schema: string },parameters: { schema: { kind: "object", fields: {}, allowUnknown: true } },- }) },+ })] } },});export const captionProgramSchema: ValueSchema = object({id: { schema: string }, documentId: { schema: string },- styles: { schema: { kind: "array", minItems: 1, items: captionStyleSchema } },- runs: { schema: { kind: "array", minItems: 1, items: object({- id: { schema: string }, styleId: { schema: string }, unitIds: { schema: { kind: "array", minItems: 1, items: string } },+ styles: { schema: { kind: "array", items: captionStyleSchema } },+ uses: { schema: { kind: "array", items: object({+ window: { schema: temporalWindowSchema }, styleId: { schema: string }, role: { schema: string, optional: true },}) } },- wordRuns: { schema: { kind: "array", items: object({- id: { schema: string }, styleId: { schema: string }, wordIds: { schema: { kind: "array", minItems: 1, items: string } },- }) } },- mutedUnitIds: { schema: { kind: "array", items: string } },});const timedUnit = object({ unitId: { schema: string }, startFrame: { schema: integer }, endFrameExclusive: { schema: integer } });export const timedCaptionProjectionSchema: ValueSchema = object({@@ -39,58 +40,40 @@ export const timedCaptionProjectionSchema: ValueSchema = object({narrativeId: { schema: string },documentId: { schema: string },cues: { schema: { kind: "array", items: object({- id: { schema: string }, styleId: { schema: string }, startFrame: { schema: integer }, endFrameExclusive: { schema: integer },+ id: { schema: string }, startFrame: { schema: integer }, endFrameExclusive: { schema: integer },units: { schema: { kind: "array", minItems: 1, items: timedUnit } },}) } },});export const captionMarkupSurfaces = [{- name: "program", tag: "Program", mode: "structured", outputs: [captionTypes.program],+ name: "hidden", tag: "Hidden", mode: "structured", outputs: [captionTypes.style],vocabulary: {- summary: "Assigns rendering Styles to the Script-owned CaptionDocument using authored semantic selections.",- attributes: [- { name: "id", kind: "identifier", required: true, summary: "Names the CaptionProgram." },- { name: "document", kind: "reference", required: true, accepts: [narrativeTypes.captionDocument], summary: "The complete Script-owned CaptionDocument." },- { name: "narrative", kind: "reference", required: true, accepts: [narrativeTypes.narrative], summary: "The semantic anchor universe used to project selections." },- { name: "default", kind: "reference", required: true, accepts: [captionTypes.style], summary: "The default Style." },- ],- children: [- { tag: "Use", cardinality: "many", summary: "Applies a complete Style by Role or semantic Selection, or supplies a word-attribute Style for a supporting family.", attributes: [- { name: "style", kind: "reference", required: true, accepts: [captionTypes.style], summary: "The Style to apply." },- { name: "role", kind: "literal", required: false, summary: "Selects all Caption units authored for this Role." },- { name: "selection", kind: "reference", required: false, accepts: [narrativeTypes.selection], summary: "Selects complete Caption units contained by a semantic Selection." },- { name: "attribute", kind: "literal", required: false, summary: "Selects display words carrying this Script-native attribute." },- ] },- { tag: "Mute", cardinality: "many", summary: "Hides selected complete Caption units.", attributes: [- { name: "role", kind: "literal", required: false, summary: "Selects all Caption units authored for this Role." },- { name: "selection", kind: "reference", required: false, accepts: [narrativeTypes.selection], summary: "Selects complete Caption units contained by a semantic Selection." },- ] },- ],- example: `<caption:Program id="captions" document={story.caption} narrative={story} default={plain}>- <caption:Use role="GUEST" style={guest}/>- <caption:Use selection={story.selection.answer} style={answer}/>- <caption:Mute selection={story.selection.demonstration}/>-</caption:Program>`,- notes: [- "Use selects exactly one of role, selection or attribute; Mute selects exactly one of role or selection.",- "For Role and Selection applications, the last matching Use in Source order selects the complete Style; properties are not merged.",- "Selections may cover a phrase, a whole Segment, several Segments or the program, preserving complete display/pronunciation units.",- "All Mute selections hide their words independently of Style order. They do not mute audio; the family still controls neighboring Cue lead and tail.",- "Word-attribute runs are separate from Role/Selection overrides and require a family that implements them; Fine does not. Conflicting Styles on one attributed word are rejected.",- ],+ summary: "Declares a Caption Style that contributes no rendering when selected by Use.",+ attributes: [{ name: "id", kind: "identifier", required: true, summary: "Names the hidden Style." }],+ example: '<caption:Hidden id="hidden"/>',+ notes: ["Hidden is a complete Style choice; it uses ordinary Use precedence and changes no speech or semantic anchors."],},}] as const;export const captionManifest: ModuleManifest = {format: "hypit.module@1", name: captionModuleRef.name, version: captionModuleRef.version,- dependencies: [narrativeDependency, semanticTrackDependency], types: [+ dependencies: [narrativeDependency, timelineDependency, temporalDependency], types: [+ { name: captionTypes.header.name }, { name: captionTypes.filter.name },{ name: captionTypes.style.name }, { name: captionTypes.program.name }, { name: captionTypes.timedProjection.name },], capabilities: [], producers: [{+ name: captionProducers.create.name,+ inputs: [{ name: "document", type: narrativeTypes.captionDocument }, { name: "header", type: captionTypes.header }],+ outputs: [{ name: "program", type: captionTypes.program }], needs: [],+ }, {+ name: captionProducers.append.name,+ inputs: [{ name: "program", type: captionTypes.program }, { name: "window", type: temporalTypes.window },+ { name: "style", type: captionTypes.style }, { name: "filter", type: captionTypes.filter }],+ outputs: [{ name: "program", type: captionTypes.program }], needs: [],+ }, {name: captionProducers.temporalizeDocument.name,inputs: [{ name: "document", type: narrativeTypes.captionDocument },- { name: "semantic", type: semanticTrackTypes.track },- { name: "program", type: captionTypes.program },+ { name: "timeline", type: timelineTypes.track },],outputs: [{ name: "caption", type: captionTypes.timedProjection }], needs: [],}],diff --git a/packages/caption/src/style.ts b/packages/caption/src/style.tsindex 58a477d0..f37242c5 100644--- a/packages/caption/src/style.ts+++ b/packages/caption/src/style.ts@@ -1,20 +1,10 @@-import type { CaptionDocument, Narrative } from "@hypit/narrative";+import type { CaptionDocument } from "@hypit/narrative";import { canonicalStringify, canonicalize } from "@hypit/protocol";-import {- assertCaptionDocument,- captionUnitsForRole,- type CaptionUnitSubset,-} from "./display.js";-import type {- CaptionMuteApplication,- CaptionProgram,- CaptionStyleApplication,- CaptionStyleIntent,- CaptionWordStyleApplication,-} from "./types.js";--export type { CaptionMuteApplication, CaptionStyleApplication, CaptionWordStyleApplication } from "./types.js";+import { assertCaptionDocument } from "./display.js";+import { assertTemporalWindowFor } from "@hypit/temporal";+import type { TemporalWindow } from "@hypit/temporal";+import type { CaptionProgram, CaptionStyleIntent } from "./types.js";const ID = /^[A-Za-z][A-Za-z0-9_.-]{0,127}$/u;function assert(condition: unknown, message: string): asserts condition { if (!condition) throw new Error(message); }@@ -22,7 +12,8 @@ function assert(condition: unknown, message: string): asserts condition { if (!cexport function sealCaptionStyle(value: CaptionStyleIntent): CaptionStyleIntent {const result = canonicalize({id: value.id,- rendering: { family: value.rendering.family.trim(), parameters: canonicalize(value.rendering.parameters) },+ rendering: value.rendering === null ? null+ : { family: value.rendering.family.trim(), parameters: canonicalize(value.rendering.parameters) },}) as unknown as CaptionStyleIntent;assertCaptionStyle(result);return result;@@ -30,6 +21,7 @@ export function sealCaptionStyle(value: CaptionStyleIntent): CaptionStyleIntentexport function assertCaptionStyle(value: CaptionStyleIntent): void {assert(ID.test(value.id), "Caption Style identity is invalid");+ if (value.rendering === null) return;assert(value.rendering.family.trim().length > 0, `Caption Style ${value.id} rendering family is empty`);canonicalize(value.rendering.parameters);}@@ -41,130 +33,27 @@ export function sealCaptionProgram(value: CaptionProgram): CaptionProgram {}export function assertCaptionProgram(value: CaptionProgram): void {- assert(ID.test(value.id), "Caption Program identity is invalid");- assert(value.documentId.length > 0 && value.styles.length > 0 && value.runs.length > 0,- "Caption Program is empty");- const styles = new Map(value.styles.map((style) => [style.id, style]));- assert(styles.size === value.styles.length, "Caption Program styles are repeated");+ assert(ID.test(value.id) && value.documentId.length > 0, "Caption Track identity is invalid");+ const styles = new Map(value.styles.map(style => [style.id, style]));+ assert(styles.size === value.styles.length, "Caption Styles are repeated");value.styles.forEach(assertCaptionStyle);- assert(new Set(value.runs.map((run) => run.id)).size === value.runs.length,- "Caption Program run ids are repeated");- const units = value.runs.flatMap((run) => run.unitIds);- assert(new Set(units).size === units.length, "Caption Program runs repeat a unit");- assert(value.runs.every((run) => styles.has(run.styleId) && run.unitIds.length > 0),- "Caption Program run is invalid");- assert(new Set(value.wordRuns.map((run) => run.id)).size === value.wordRuns.length,- "Caption Program word run ids are repeated");- const words = value.wordRuns.flatMap((run) => run.wordIds);- assert(new Set(words).size === words.length, "Caption Program word runs repeat a word");- assert(value.wordRuns.every((run) => styles.has(run.styleId) && run.wordIds.length > 0),- "Caption Program word run is invalid");- assert(new Set(value.mutedUnitIds).size === value.mutedUnitIds.length,- "Caption Program muted units are repeated");+ assert(new Set(value.uses.map(use => use.window.subjectId)).size === value.uses.length, "Caption Use ids are repeated");+ for (const use of value.uses) {+ assertTemporalWindowFor(use.window, { subjectId: use.window.subjectId });+ assert(styles.has(use.styleId), "Caption Use references an unknown Style");+ assert(use.role === undefined || use.role.trim().length > 0, "Caption Role is empty");+ }}-export function assertCaptionProgramForDocument(value: CaptionProgram, document: CaptionDocument): void {assertCaptionProgram(value);assertCaptionDocument(document);- assert(value.documentId === document.id, "Caption Program belongs to another CaptionDocument");- const expected = document.units.map((unit) => unit.id);- assert(value.runs.flatMap((run) => run.unitIds).join("\0") === expected.join("\0"),- "Caption Program must partition every Caption unit exactly once and in order");- const known = new Set(expected);- assert(value.mutedUnitIds.every((id) => known.has(id)), "Caption Program mutes an unknown unit");- const knownWords = new Set(document.words.map((word) => word.id));- assert(value.wordRuns.every((run) => run.wordIds.every((id) => knownWords.has(id))),- "Caption Program styles an unknown display word");+ assert(value.documentId === document.id, "Caption Uses belong to another CaptionDocument");}--function orderedSubset(value: CaptionUnitSubset, document: CaptionDocument): string[] {- assertCaptionDocument(document);- assert(value.documentId === document.id, "Caption unit subset belongs to another document");- const order = new Map(document.units.map((unit, index) => [unit.id, index]));- const indices = value.unitIds.map((id) => order.get(id));- assert(indices.length > 0 && indices.every((index) => index !== undefined),- "Caption unit subset contains an unknown unit");- assert(indices.every((index, position) => position === 0 || index! > indices[position - 1]!),- "Caption unit subset must be ordered and contain no repeated unit");- return [...value.unitIds];-}--export function resolveCaptionProgram(- document: CaptionDocument,- narrative: Narrative,- id: string,- defaultStyle: CaptionStyleIntent,- applications: readonly CaptionStyleApplication[],- mutes: readonly CaptionMuteApplication[] = [],- wordApplications: readonly CaptionWordStyleApplication[] = [],-): CaptionProgram {- assertCaptionDocument(document);- assertCaptionStyle(defaultStyle);- const styles = new Map<string, CaptionStyleIntent>([[defaultStyle.id, defaultStyle]]);- const selected = applications.map((application) => {- assert(ID.test(application.id), "Caption Style Application identity is invalid");- assertCaptionStyle(application.style);- const unitIds = orderedSubset({ documentId: document.id, unitIds: application.unitIds }, document);- const previous = styles.get(application.style.id);- assert(previous === undefined || canonicalStringify(previous) === canonicalStringify(application.style),- `Caption Style ${application.style.id} has conflicting definitions`);- styles.set(application.style.id, application.style);- return { ...application, unitIds };- });- const muted = mutes.flatMap((application) => {- assert(ID.test(application.id), "Caption Mute identity is invalid");- return orderedSubset({ documentId: document.id, unitIds: application.unitIds }, document);+export function appendCaptionUse(program: CaptionProgram, window: TemporalWindow, style: CaptionStyleIntent, role?: string): CaptionProgram {+ const previous = program.styles.find(item => item.id === style.id);+ assert(previous === undefined || canonicalStringify(previous) === canonicalStringify(style), `Caption Style ${style.id} has conflicting definitions`);+ return sealCaptionProgram({ ...program,+ styles: previous === undefined ? [...program.styles, style] : program.styles,+ uses: [...program.uses, { window, styleId: style.id, ...(role === undefined ? {} : { role }) }],});- const wordChosen = new Map<string, string>();- for (const application of wordApplications) {- assert(ID.test(application.id), "Caption Word Style Application identity is invalid");- assert(application.attribute.trim().length > 0, "Caption Word Style Application attribute is empty");- assertCaptionStyle(application.style);- const knownWordIds = new Set(document.words.map((word) => word.id));- for (const wordId of application.wordIds) {- assert(knownWordIds.has(wordId), `Caption Word Style Application references unknown word ${wordId}`);- const previous = wordChosen.get(wordId);- assert(previous === undefined || previous === application.style.id,- `Caption word ${wordId} receives conflicting styles`);- wordChosen.set(wordId, application.style.id);- }- const previous = styles.get(application.style.id);- assert(previous === undefined || canonicalStringify(previous) === canonicalStringify(application.style),- `Caption Style ${application.style.id} has conflicting definitions`);- styles.set(application.style.id, application.style);- }- const chosen = new Map(document.units.map((unit) => [unit.id, defaultStyle.id]));- for (const application of selected) for (const unitId of application.unitIds) chosen.set(unitId, application.style.id);- const runs: CaptionProgram["runs"][number][] = [];- for (const unit of document.units) {- const styleId = chosen.get(unit.id)!;- const previous = runs.at(-1);- if (previous === undefined || previous.styleId !== styleId) {- runs.push({ id: `${id}:run:${runs.length + 1}`, styleId, unitIds: [unit.id] });- } else {- runs[runs.length - 1] = { ...previous, unitIds: [...previous.unitIds, unit.id] };- }- }- const wordRuns: CaptionProgram["wordRuns"][number][] = [];- for (const [wordId, styleId] of wordChosen) {- const previous = wordRuns.at(-1);- if (previous === undefined || previous.styleId !== styleId) {- wordRuns.push({ id: `${id}:word-run:${wordRuns.length + 1}`, styleId, wordIds: [wordId] });- } else {- wordRuns[wordRuns.length - 1] = { ...previous, wordIds: [...previous.wordIds, wordId] };- }- }- const program = sealCaptionProgram({- id,- documentId: document.id,- styles: [...styles.values()].filter((style) => [...chosen.values(), ...wordChosen.values()].includes(style.id)),- runs,- wordRuns,- mutedUnitIds: [...new Set(muted)],- });- assertCaptionProgramForDocument(program, document);- void narrative;- return program;}--export { captionUnitsForRole };diff --git a/packages/caption/src/surface.ts b/packages/caption/src/surface.tsindex f0a3738b..9717a39e 100644--- a/packages/caption/src/surface.ts+++ b/packages/caption/src/surface.ts@@ -1,96 +1,16 @@-import { narrativeTypes } from "@hypit/narrative";-import type { CaptionDocument, Narrative, NarrativeSelection } from "@hypit/narrative";-import type { StructuredElement, StructuredSurfaceHandler, SurfaceResolvedReference, MarkupAttributeValue } from "@hypit/markup";-+import type { StructuredSurfaceHandler } from "@hypit/markup";+import { assertAttributes, assertEmptyElement, textAttribute } from "@hypit/markup";import { captionTypes } from "./manifest.js";-import { captionUnitsForRole, captionUnitsForSelection, captionWordsForAttribute } from "./display.js";-import { resolveCaptionProgram } from "./style.js";-import type { CaptionMuteApplication, CaptionStyleApplication, CaptionStyleIntent, CaptionWordStyleApplication } from "./types.js";--function sameType(left: SurfaceResolvedReference["type"], right: SurfaceResolvedReference["type"]): boolean {- return left.module.name === right.module.name && left.module.version === right.module.version && left.name === right.name;-}-function attributes(element: StructuredElement, required: readonly string[], optional: readonly string[] = []): void {- const allowed = new Set([...required, ...optional]);- const actual = Object.keys(element.attributes);- if (required.some((name) => element.attributes[name] === undefined) || actual.some((name) => !allowed.has(name))) {- throw new Error(`${element.name} requires ${required.join(", ")}${optional.length ? `; optional: ${optional.join(", ")}` : ""}`);- }-}-function stringAttribute(element: StructuredElement, name: string): string {- const value = element.attributes[name];- if (typeof value !== "string" || !value.trim()) throw new Error(`${element.name}.${name} must be a non-empty string`);- return value.trim();-}-function optionalString(element: StructuredElement, name: string): string | undefined {- const value = element.attributes[name];- if (value === undefined) return undefined;- if (typeof value !== "string" || !value.trim()) throw new Error(`${element.name}.${name} must be a non-empty string`);- return value.trim();-}-function reference(- element: StructuredElement,- name: string,- expected: SurfaceResolvedReference["type"],- resolveReference: (path: string) => SurfaceResolvedReference | undefined,-): SurfaceResolvedReference {- const raw: MarkupAttributeValue | undefined = element.attributes[name];- if (typeof raw !== "object" || raw.kind !== "reference") throw new Error(`${element.name}.${name} must be a whole-value reference`);- const value = resolveReference(raw.path);- if (value === undefined) throw new Error(`${element.name}.${name} cannot resolve ${raw.path}`);- if (!sameType(value.type, expected)) throw new Error(`${element.name}.${name} has the wrong type`);- return value;-}-function inline<T>(referenceValue: SurfaceResolvedReference, subject: string): T {- if (referenceValue.record?.value.kind !== "inline") throw new Error(`${subject} must reference an authored inline Record`);- return referenceValue.record.value.value as unknown as T;-}-function localName(name: string): string { const index = name.lastIndexOf(":"); return index < 0 ? name : name.slice(index + 1); }+import { sealCaptionStyle } from "./style.js";-export const decodeCaptionProgramSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {- attributes(element, ["id", "document", "narrative", "default"]);- const id = stringAttribute(element, "id");- const document = inline<CaptionDocument>(reference(element, "document", narrativeTypes.captionDocument, resolveReference), `${element.name}.document`);- const narrative = inline<Narrative>(reference(element, "narrative", narrativeTypes.narrative, resolveReference), `${element.name}.narrative`);- const defaultStyle = inline<CaptionStyleIntent>(reference(element, "default", captionTypes.style, resolveReference), `${element.name}.default`);- const applications: CaptionStyleApplication[] = [];- const wordApplications: CaptionWordStyleApplication[] = [];- const mutes: CaptionMuteApplication[] = [];- for (const child of element.children) {- if (child.kind === "text") { if (child.value.trim()) throw new Error(`${element.name} accepts only Use or Mute children`); continue; }- const childName = localName(child.name);- if (childName !== "Use" && childName !== "Mute") throw new Error(`${element.name} accepts only Use or Mute children`);- attributes(child, childName === "Use" ? ["style"] : [], ["role", "selection", "attribute"]);- const role = optionalString(child, "role");- const attribute = optionalString(child, "attribute");- const selectionRaw = child.attributes.selection;- const selectors = [role !== undefined, selectionRaw !== undefined, attribute !== undefined].filter(Boolean).length;- if (selectors !== 1) throw new Error(`${child.name} requires exactly one of role, selection or attribute`);- if (childName === "Mute" && attribute !== undefined) throw new Error(`${child.name} cannot mute a token attribute`);- if (attribute !== undefined) {- const wordIds = captionWordsForAttribute(document, attribute);- wordApplications.push({- id: `${id}.word-use.${wordApplications.length + 1}`,- attribute,- wordIds,- style: inline<CaptionStyleIntent>(reference(child, "style", captionTypes.style, resolveReference), `${child.name}.style`),- });- continue;- }- const unitIds = role !== undefined- ? captionUnitsForRole(document, role).unitIds- : captionUnitsForSelection(- document,- narrative,- inline<NarrativeSelection>(reference(child, "selection", narrativeTypes.selection, resolveReference), `${child.name}.selection`),- ).unitIds;- if (childName === "Mute") mutes.push({ id: `${id}.mute.${mutes.length + 1}`, unitIds });- else applications.push({- id: `${id}.use.${applications.length + 1}`,- unitIds,- style: inline<CaptionStyleIntent>(reference(child, "style", captionTypes.style, resolveReference), `${child.name}.style`),- });+export const decodeHiddenCaptionStyleSurface: StructuredSurfaceHandler = ({ element }) => {+ assertAttributes(element, ["id"]);+ assertEmptyElement(element);+ const id = textAttribute(element, "id");+ if (element.children.some(child => child.kind !== "text" || child.value.trim())) {+ throw new Error(`${element.name} accepts no children`);}- const program = resolveCaptionProgram(document, narrative, id, defaultStyle, applications, mutes, wordApplications);- return { records: [{ id, type: captionTypes.program, value: { kind: "inline", value: program }, range: element.range }], components: [], fragments: [] };+ return { records: [{ id, type: captionTypes.style,+ value: { kind: "inline", value: sealCaptionStyle({ id, rendering: null }) }, range: element.range }],+ components: [], fragments: [] };};diff --git a/packages/caption/src/temporalize.ts b/packages/caption/src/temporalize.tsindex a439f0f2..aef2d854 100644--- a/packages/caption/src/temporalize.ts+++ b/packages/caption/src/temporalize.ts@@ -1,64 +1,55 @@import type { CaptionDocument } from "@hypit/narrative";-import { projectSemanticProgramSpace, tokenFrameSpan } from "@hypit/semantic-track";-import type { SemanticTrack } from "@hypit/semantic-track";+import { projectTimelineSpace, tokenFrameSpan } from "@hypit/timeline";+import type { Timeline } from "@hypit/timeline";import { CaptionTimingError } from "./error.js";import { assertCaptionDocument } from "./display.js";-import { assertCaptionProgramForDocument } from "./style.js";-import type { CaptionProgram, TimedCaptionProjection, TimedCaptionCue, TimedCaptionUnit } from "./types.js";+import type { TimedCaptionProjection, TimedCaptionCue, TimedCaptionUnit } from "./types.js";export function temporalizeCaptionDocument(document: CaptionDocument,- semantic: SemanticTrack,- program: CaptionProgram,+ semantic: Timeline,): TimedCaptionProjection {assertCaptionDocument(document);- assertCaptionProgramForDocument(program, document);- const space = projectSemanticProgramSpace(semantic);- if (document.narrativeId !== semantic.narrativeId) {- throw new CaptionTimingError("CAPTION_NARRATIVE", "CaptionDocument and SemanticTrack belong to different Narratives.");+ const space = projectTimelineSpace(semantic);+ if (semantic.narrativeId !== undefined && document.narrativeId !== semantic.narrativeId) {+ throw new CaptionTimingError("CAPTION_NARRATIVE", "CaptionDocument and Timeline belong to different Narratives.");}- const styleByUnit = new Map(program.runs.flatMap((run) => run.unitIds.map((unitId) => [unitId, run.styleId] as const)));- const muted = new Set(program.mutedUnitIds);const breaks = new Set(document.cueBreaks.map((cueBreak) => cueBreak.afterUnitId));- const timed: Array<{ unit: CaptionDocument["units"][number]; styleId: string; timing: TimedCaptionUnit }> = [];+ const timed: Array<{ unit: CaptionDocument["units"][number]; timing: TimedCaptionUnit }> = [];for (const unit of document.units) {- if (muted.has(unit.id)) continue;- const styleId = styleByUnit.get(unit.id);- if (styleId === undefined) throw new CaptionTimingError("CAPTION_UNIT", `Caption unit ${unit.id} has no Style.`);const window = tokenFrameSpan(semantic, unit.sourceTokenIds);- if (window === undefined) throw new CaptionTimingError("CAPTION_SPEECH_COVERAGE", `Caption unit ${unit.id} is absent from the SemanticTrack.`);+ if (window === undefined) continue;if (window.endFrameExclusive < window.startFrame) {throw new CaptionTimingError("CAPTION_SPEECH_ORDER", `Caption unit ${unit.id} references speech Tokens in reverse order.`);}const startFrame = window.startFrame;const endFrameExclusive = Math.max(startFrame + 1, window.endFrameExclusive);- timed.push({ unit, styleId, timing: { unitId: unit.id, startFrame, endFrameExclusive } });+ timed.push({ unit, timing: { unitId: unit.id, startFrame, endFrameExclusive } });}const cues: TimedCaptionCue[] = [];- let current: { styleId: string; units: TimedCaptionUnit[]; segmentId: string; turnId: string } | undefined;+ let current: { units: TimedCaptionUnit[]; segmentId: string; turnId: string } | undefined;const flush = (): void => {if (current === undefined || current.units.length === 0) return;cues.push({- id: `${program.id}:cue:${cues.length + 1}`,- styleId: current.styleId,+ id: `${document.id}:cue:${cues.length + 1}`,startFrame: current.units[0]!.startFrame,endFrameExclusive: current.units.at(-1)!.endFrameExclusive,units: current.units,});current = undefined;};+ const documentOrder = new Map(document.units.map((unit, index) => [unit.id, index]));for (const [index, entry] of timed.entries()) {const previousDocumentUnit = index === 0 ? undefined : timed[index - 1]!.unit;const mustBreak = current !== undefined && (- current.styleId !== entry.styleId- || current.segmentId !== entry.unit.segmentId+ current.segmentId !== entry.unit.segmentId|| current.turnId !== entry.unit.turnId- || (previousDocumentUnit !== undefined && breaks.has(previousDocumentUnit.id))+ || (previousDocumentUnit !== undefined && (breaks.has(previousDocumentUnit.id)+ || documentOrder.get(entry.unit.id)! !== documentOrder.get(previousDocumentUnit.id)! + 1)));if (mustBreak) flush();if (current === undefined) current = {- styleId: entry.styleId,units: [],segmentId: entry.unit.segmentId,turnId: entry.unit.turnId,@@ -68,7 +59,7 @@ export function temporalizeCaptionDocument(flush();const result: TimedCaptionProjection = {spaceId: space.id,- narrativeId: semantic.narrativeId,+ narrativeId: document.narrativeId,documentId: document.id,cues,};@@ -76,32 +67,6 @@ export function temporalizeCaptionDocument(return result;}-export function applyCaptionMute(- projection: TimedCaptionProjection,- program: CaptionProgram,- document: CaptionDocument,-): TimedCaptionProjection {- assertCaptionDocument(document);- assertCaptionProgramForDocument(program, document);- if (projection.documentId !== document.id || projection.narrativeId !== document.narrativeId) {- throw new Error("Caption Mute received another CaptionDocument or Narrative");- }- const muted = new Set(program.mutedUnitIds);- const cues = projection.cues.flatMap((cue) => {- const units = cue.units.filter((unit) => !muted.has(unit.unitId));- if (units.length === 0) return [];- return [{ ...cue, units, startFrame: units[0]!.startFrame, endFrameExclusive: units.at(-1)!.endFrameExclusive }];- });- const result = {- spaceId: projection.spaceId,- narrativeId: projection.narrativeId,- documentId: projection.documentId,- cues,- };- assertTimedCaptionProjection(result);- return result;-}-export function assertTimedCaptionProjection(projection: TimedCaptionProjection): void {if (projection.spaceId.length === 0 || projection.narrativeId.length === 0 || projection.documentId.length === 0) {throw new Error("TimedCaptionProjection provenance is invalid");@@ -109,7 +74,7 @@ export function assertTimedCaptionProjection(projection: TimedCaptionProjection)const cueIds = new Set<string>();const unitIds = new Set<string>();for (const cue of projection.cues) {- if (cue.id.length === 0 || cueIds.has(cue.id) || cue.styleId.length === 0 || cue.units.length === 0+ if (cue.id.length === 0 || cueIds.has(cue.id) || cue.units.length === 0|| !Number.isSafeInteger(cue.startFrame) || !Number.isSafeInteger(cue.endFrameExclusive)|| cue.startFrame < 0 || cue.endFrameExclusive <= cue.startFrame) throw new Error("TimedCaptionProjection contains an invalid Cue");cueIds.add(cue.id);diff --git a/packages/caption/src/types.ts b/packages/caption/src/types.tsindex 5af696ae..69de4f6f 100644--- a/packages/caption/src/types.ts+++ b/packages/caption/src/types.ts@@ -2,50 +2,24 @@ import type { CanonicalValue } from "@hypit/protocol";export type CaptionStyleIntent = {readonly id: string;+ /** null selects no rendering; later Uses can select a visible Style again. */readonly rendering: {readonly family: string;readonly parameters: CanonicalValue;- };+ } | null;};-export type CaptionStyleApplication = {- readonly id: string;- readonly unitIds: readonly string[];- readonly style: CaptionStyleIntent;-};--export type CaptionWordStyleApplication = {- readonly id: string;- readonly attribute: string;- readonly wordIds: readonly string[];- readonly style: CaptionStyleIntent;-};--export type CaptionMuteApplication = {- readonly id: string;- readonly unitIds: readonly string[];-};--export type CaptionProgramRun = {- readonly id: string;- readonly styleId: string;- readonly unitIds: readonly string[];-};--export type CaptionProgramWordRun = {- readonly id: string;+/** Ordered, resolved Uses owned by a Caption Track, not a separate author element. */+export type CaptionUse = {+ readonly window: import("@hypit/temporal").TemporalWindow;readonly styleId: string;- readonly wordIds: readonly string[];+ readonly role?: string;};-export type CaptionProgram = {readonly id: string;readonly documentId: string;readonly styles: readonly CaptionStyleIntent[];- readonly runs: readonly CaptionProgramRun[];- /** Optional local word styles resolved from Script-native word attributes. */- readonly wordRuns: readonly CaptionProgramWordRun[];- readonly mutedUnitIds: readonly string[];+ readonly uses: readonly CaptionUse[];};export type TimedCaptionUnit = {@@ -56,14 +30,13 @@ export type TimedCaptionUnit = {export type TimedCaptionCue = {readonly id: string;- readonly styleId: string;readonly startFrame: number;readonly endFrameExclusive: number;readonly units: readonly TimedCaptionUnit[];};export type TimedCaptionProjection = {- /** Semantic ProgramSpace from which every unit frame was measured. */+ /** Semantic Timeline from which every unit frame was measured. */readonly spaceId: string;readonly narrativeId: string;readonly documentId: string;diff --git a/packages/caption/src/visibility.ts b/packages/caption/src/visibility.tsnew file mode 100644index 00000000..aadb47c5--- /dev/null+++ b/packages/caption/src/visibility.ts@@ -0,0 +1,23 @@+type FrameSpan = { readonly startFrame: number; readonly endFrameExclusive: number };+import type { CaptionProgram } from "./types.js";++/** Later Uses replace this Cue's presentation inside their windows, including Hidden. */+export function captionUseVisibility(program: CaptionProgram, index: number, role: string | undefined, envelope: FrameSpan): FrameSpan[] {+ const use = program.uses[index]!;+ if (use.role !== undefined && use.role !== role) return [];+ const startFrame = Math.max(envelope.startFrame, use.window.span.startFrame);+ const endFrameExclusive = Math.min(envelope.endFrameExclusive, use.window.span.endFrameExclusive);+ let visible = endFrameExclusive > startFrame ? [{ startFrame, endFrameExclusive }] : [];+ for (const later of program.uses.slice(index + 1)) {+ if (later.role !== undefined && later.role !== role) continue;+ const cut = later.window.span;+ visible = visible.flatMap(span => {+ if (cut.startFrame >= span.endFrameExclusive || cut.endFrameExclusive <= span.startFrame) return [span];+ return [+ ...(cut.startFrame > span.startFrame ? [{ startFrame: span.startFrame, endFrameExclusive: cut.startFrame }] : []),+ ...(cut.endFrameExclusive < span.endFrameExclusive ? [{ startFrame: cut.endFrameExclusive, endFrameExclusive: span.endFrameExclusive }] : []),+ ];+ });+ }+ return visible;+}diff --git a/packages/caption/test/caption.test.ts b/packages/caption/test/caption.test.tsindex 1586d712..9228ebea 100644--- a/packages/caption/test/caption.test.ts+++ b/packages/caption/test/caption.test.ts@@ -6,7 +6,6 @@ import {assertCaptionProgramForDocument,captionUnitsForRole,captionUnitsForSelection,- resolveCaptionProgram,sealCaptionStyle,} from "@hypit/caption";import { captionDocument, narrativeValue, parseScript } from "@hypit/script";@@ -17,31 +16,11 @@ test("Caption uses complete semantic selections and authored cue breaks", () =>const narrative = narrativeValue(parsed, "story") as unknown as Narrative;const document = captionDocument(parsed, "story.caption", "story");const selection = narrative.selections[0]!;- const subset = captionUnitsForSelection(document, narrative, selection);+ const subset = captionUnitsForSelection(narrative, selection);assert.equal(subset.unitIds.length, 2);- const style = sealCaptionStyle({ id: "plain", rendering: { family: "test", parameters: canonicalize({}) } });- const program = resolveCaptionProgram(document, narrative, "captions", style, [], []);- assertCaptionProgramForDocument(program, document);assert.equal(document.cueBreaks.length, 1);});-test("Caption styles one Role across non-contiguous authored Cues", () => {- const parsed = parseScript("caption-roles.svml", "<line><GUY>one || <GIRL>two || <GUY>three</line>");- const narrative = narrativeValue(parsed, "story") as unknown as Narrative;- const document = captionDocument(parsed, "story.caption", "story");- const plain = sealCaptionStyle({ id: "plain", rendering: { family: "test", parameters: canonicalize({}) } });- const guy = sealCaptionStyle({ id: "guy", rendering: { family: "test", parameters: canonicalize({}) } });- const program = resolveCaptionProgram(- document,- narrative,- "captions",- plain,- [{ id: "guy-role", unitIds: captionUnitsForRole(document, "GUY").unitIds, style: guy }],- );- assertCaptionProgramForDocument(program, document);- assert.deepEqual(program.runs.map((run) => run.styleId), ["guy", "plain", "guy"]);-});-test("Caption selects complete Segments and the program through structural anchors", () => {const parsed = parseScript("caption-ranges.svml", `~@whole@opening <intro><HOST>One idea.</intro> @/opening@@ -51,48 +30,8 @@ test("Caption selects complete Segments and the program through structural anchoconst document = captionDocument(parsed, "story.caption", "story");const whole = narrative.selections.find((selection) => selection.id === "whole")!;const opening = narrative.selections.find((selection) => selection.id === "opening")!;- assert.deepEqual(captionUnitsForSelection(document, narrative, whole).unitIds,+ assert.deepEqual(captionUnitsForSelection(narrative, whole).unitIds,document.units.map((unit) => unit.id));- assert.deepEqual(captionUnitsForSelection(document, narrative, opening).unitIds,+ assert.deepEqual(captionUnitsForSelection(narrative, opening).unitIds,document.units.filter((unit) => unit.segmentId === "intro").map((unit) => unit.id));});--test("Caption applications use authored order and styling does not undo a display mute", () => {- const parsed = parseScript("caption-coverage.svml",- "<line><HOST>First. <GUEST>@answer Another view. @/answer</line>");- const narrative = narrativeValue(parsed, "story") as unknown as Narrative;- const document = captionDocument(parsed, "story.caption", "story");- const makeStyle = (id: string) => sealCaptionStyle({ id, rendering: { family: "test", parameters: {} } });- const base = makeStyle("base"), guest = makeStyle("guest"), accent = makeStyle("accent");- const selected = captionUnitsForSelection(document, narrative, narrative.selections[0]!).unitIds;- const roleUse = { id: "guest-role", unitIds: captionUnitsForRole(document, "GUEST").unitIds, style: guest };- const rangeUse = { id: "answer-range", unitIds: selected, style: accent };- const resolve = (applications: typeof roleUse[]) => resolveCaptionProgram(document, narrative, "captions",- base, applications, [{ id: "hidden-answer", unitIds: selected }]);- const rangeLast = resolve([roleUse, rangeUse]);- assert.deepEqual(rangeLast.runs.map((run) => run.styleId), ["base", "accent"]);- const roleLast = resolve([rangeUse, roleUse]);- assert.deepEqual(roleLast.runs.map((run) => run.styleId), ["base", "guest"]);- assert.deepEqual(rangeLast.mutedUnitIds, selected);- assert.deepEqual(roleLast.mutedUnitIds, selected);-});--test("Caption keeps Selection styles and adds flat word-attribute styles separately", () => {- const parsed = parseScript("caption-attributes.svml", "<line>@focus really{emphasis} matters @/focus</line>");- const narrative = narrativeValue(parsed, "story") as unknown as Narrative;- const document = captionDocument(parsed, "story.caption", "story");- const plain = sealCaptionStyle({ id: "plain", rendering: { family: "test", parameters: canonicalize({ level: "plain" }) } });- const emphasis = sealCaptionStyle({ id: "emphasis", rendering: { family: "test", parameters: canonicalize({ level: "emphasis" }) } });- const selection = narrative.selections[0]!;- const program = resolveCaptionProgram(- document,- narrative,- "captions",- plain,- [{ id: "selection", unitIds: captionUnitsForSelection(document, narrative, selection).unitIds, style: plain }],- [],- [{ id: "word", attribute: "emphasis", wordIds: [document.words[1]!.id], style: emphasis }],- );- assert.equal(program.runs.length, 1);- assert.deepEqual(program.wordRuns, [{ id: "captions:word-run:1", styleId: "emphasis", wordIds: [document.words[1]!.id] }]);-});diff --git a/packages/caption/test/chinese-pipeline.test.ts b/packages/caption/test/chinese-pipeline.test.tsindex baca93a2..fc0b1a0d 100644--- a/packages/caption/test/chinese-pipeline.test.ts+++ b/packages/caption/test/chinese-pipeline.test.ts@@ -4,10 +4,9 @@ import type { Narrative } from "@hypit/narrative";import { captionDocument, narrativeValue, parseScript } from "@hypit/script";import { alignSemanticTake } from "../../speech-alignment/src/index.js";import { interpretWhisperXTranscript } from "../../whisperx/src/index.js";-import { appendSpeechTrackTake, assembleSpeechTrack, createSpeechTrackSet, sealSpeechTrackHeader } from "../../speech-track/src/index.js";-import { selectionFrameSpan } from "@hypit/semantic-track";+import { appendTimelineAuthorTake, assembleTimelineAuthor, createTimelineAuthorSet, sealTimelineAuthorHeader } from "../../timeline-author/src/index.js";+import { selectionFrameSpan } from "@hypit/timeline";import { temporalizeCaptionDocument } from "../src/index.js";-import type { CaptionProgram } from "../src/index.js";import { fixtureResource } from "../../../test/fixture-resource.js";test("Chinese Script, WhisperX response, semantic assembly and captions preserve words, cue breaks and timing", () => {@@ -19,7 +18,7 @@ test("Chinese Script, WhisperX response, semantic assembly and captions preserve// These fixtures test transport and mapping, not acoustic recognition accuracy.const phrases = ["用ElevenLabs做视频真方便", "今年二零二六年这个很好"];const durations = [4000, 3000];- let set = createSpeechTrackSet();+ let set = createTimelineAuthorSet();const evidenceWindows: Array<Array<[number, number]>> = [];for (const [index, segment] of narrative.segments.entries()) {let cursor = 100;@@ -40,16 +39,10 @@ test("Chinese Script, WhisperX response, semantic assembly and captions preserveaudio: { artifact: { kind: "blob", resource: fixtureResource(`zh:${index}`), size: 1, mediaType: "audio/wav" } },}, { passages: interpretWhisperXTranscript({ language: "zh", segments: [{ start: 0,end: durations[index]! / 1000, words }] }, sampleFrames) });- set = appendSpeechTrackTake(set, take);+ set = appendTimelineAuthorTake(set, take);}- const semantic = assembleSpeechTrack(sealSpeechTrackHeader({ id: "speech" }), set);- const program: CaptionProgram = {- id: "captions", documentId: document.id,- styles: [{ id: "plain", rendering: { family: "test-caption@1", parameters: {} } }],- runs: [{ id: "all", styleId: "plain", unitIds: document.units.map(unit => unit.id) }],- wordRuns: [], mutedUnitIds: [],- };- const projection = temporalizeCaptionDocument(document, semantic, program);+ const semantic = assembleTimelineAuthor(sealTimelineAuthorHeader({ id: "speech" }), set, { frameRate: { numerator: 1000, denominator: 1 } });+ const projection = temporalizeCaptionDocument(document, semantic);const wordText = new Map(document.words.map(word => [word.id, word.text]));const unitText = new Map(document.units.map(unit => [unit.id, unit.wordIds.map(id => wordText.get(id)!).join("")]));assert.deepEqual(projection.cues.map(cue => cue.units.map(unit => unitText.get(unit.unitId)).join("")),@@ -68,4 +61,5 @@ test("Chinese Script, WhisperX response, semantic assembly and captions preserveassert.deepEqual(windowsFor("這"), [[4000 + second[7]![0], 4000 + second[7]![1]]]);assert.deepEqual(windowsFor("個"), [[4000 + second[8]![0], 4000 + second[8]![1]]]);assert.equal(document.units[0]!.role, "HOST");+});diff --git a/packages/comment-sticker/README.md b/packages/comment-sticker/README.mdindex 5ab877a5..0d1e6bf3 100644--- a/packages/comment-sticker/README.md+++ b/packages/comment-sticker/README.md@@ -1,7 +1,7 @@# `@hypit/comment-sticker`-The Track Surface accepts `semantic={speech.semantic}` or `space={animation}`. Semantic context-resolves Script Selections and Moments; authored space supports clock-based animation. Shared `at`+The Track Surface accepts `timeline={program.timeline}`. The same Timeline supports authored+positions and, where prepared Takes supply evidence, Script Selections and Moments. Shared `at`inputs accept a Moment or a time such as `2s`; `at` with `for` produces a Window where required.An author package for timed social-comment cards. It publishes an ordinary `VisualTrack` containing@@ -21,7 +21,7 @@ The author surface keeps the three independent concerns visible:<comment:Style id="social-comment" recipe={styles.comment} font={fonts.ui}/>-<comment:Track id="comments" canvas={video.canvas} semantic={speech.semantic}>+<comment:Track id="comments" canvas={video.canvas} timeline={speech.timeline}><comment:Stickerid="opening-comment"comment={comment-copy}diff --git a/packages/comment-sticker/package.json b/packages/comment-sticker/package.jsonindex 22263cec..12c44b38 100644--- a/packages/comment-sticker/package.json+++ b/packages/comment-sticker/package.json@@ -15,7 +15,7 @@"@hypit/narrative": "workspace:*","@hypit/protocol": "workspace:*","@hypit/program-space": "workspace:*",- "@hypit/semantic-track": "workspace:*",+ "@hypit/timeline": "workspace:*","@hypit/spatial": "workspace:*","@hypit/svs": "workspace:*","@hypit/temporal": "workspace:*",diff --git a/packages/comment-sticker/src/component.ts b/packages/comment-sticker/src/component.tsindex 75388761..abb1c5a3 100644--- a/packages/comment-sticker/src/component.ts+++ b/packages/comment-sticker/src/component.ts@@ -1,7 +1,7 @@+import type { Timeline } from "@hypit/timeline";import type { ComponentPackage, ProducerHandlerContext } from "@hypit/component-kit";import { canonicalize } from "@hypit/protocol";import type { BlobRef, StoredValue } from "@hypit/protocol";-import type { ProgramSpace } from "@hypit/program-space";import type { CanvasSpace, SpatialFrame } from "@hypit/spatial";import type { Text } from "@hypit/text";@@ -28,7 +28,7 @@ function common(inputs: Record<string, { readonly value: StoredValue } | undefinreturn {set: inline<CommentStickerSet>(inputs.set?.value, "CommentStickerSet"),header: inline<CommentStickerHeader>(inputs.header?.value, "CommentStickerHeader"),- space: inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ timeline: inline<Timeline>(inputs.timeline?.value, "Timeline"),frame: inline<SpatialFrame>(inputs.frame?.value, "SpatialFrame"),style: inline<CommentStickerStyle>(inputs.style?.value, "CommentStickerStyle"),spec: inline<CommentStickerItemSpec>(inputs.spec?.value, "CommentStickerItemSpec"),@@ -69,7 +69,7 @@ export const commentStickerComponent = {handler: ({ inputs }: ProducerHandlerContext) => {const values = common(inputs);return { outputs: { set: output(appendProjectedCommentSticker(- values.set, values.header, values.space, values.frame, values.style, values.spec, values.content, values.window,+ values.set, values.header, values.timeline, values.frame, values.style, values.spec, values.content, values.window,avatar ? inline<BlobRef>(inputs.avatar?.value, "Comment Sticker avatar") : undefined,)) }, needs: {} };},@@ -85,7 +85,7 @@ export const commentStickerComponent = {producer: commentStickerProducers.render,handler: ({ inputs }) => ({ outputs: { track: output(renderCommentSticker(inline<CanvasSpace>(inputs.canvas?.value, "CanvasSpace"),- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ inline<Timeline>(inputs.timeline?.value, "Timeline"),inline<CommentStickerProgram>(inputs.program?.value, "CommentStickerProgram"),)) }, needs: {} }),},diff --git a/packages/comment-sticker/src/fragment.ts b/packages/comment-sticker/src/fragment.tsindex 3aac04e7..9668f17b 100644--- a/packages/comment-sticker/src/fragment.ts+++ b/packages/comment-sticker/src/fragment.ts@@ -1,4 +1,5 @@-import { programSpaceTypes } from "@hypit/program-space";+import { timelineTypes } from "@hypit/timeline";+import { artifactTypes } from "@hypit/artifact";import { compositionTypes } from "@hypit/composition";import { sealGraphFragment } from "@hypit/elaborator";@@ -82,7 +83,7 @@ export function createCommentStickerFragment(items: readonly CommentStickerFragminputs: {set: operation(current),header: input("header"),- space: input("space"),+ timeline: input("timeline"),frame: input(item.frameName),style: input(item.styleName),window: input(item.windowName),@@ -104,7 +105,7 @@ export function createCommentStickerFragment(items: readonly CommentStickerFragm{id: "comment:track",producer: commentStickerProducers.render,- inputs: { canvas: input("canvas"), space: input("space"), program: operation("comment:program") },+ inputs: { canvas: input("canvas"), timeline: input("timeline"), program: operation("comment:program") },result: { kind: "output", name: "track" },},);@@ -112,7 +113,7 @@ export function createCommentStickerFragment(items: readonly CommentStickerFragminputs: [{ name: "canvas", type: spatialTypes.canvas },{ name: "header", type: commentStickerTypes.header },- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },...[...types].map(([inputName, type]) => ({ name: inputName, type })),],operations,diff --git a/packages/comment-sticker/src/manifest.ts b/packages/comment-sticker/src/manifest.tsindex 3d871a77..6a255f56 100644--- a/packages/comment-sticker/src/manifest.ts+++ b/packages/comment-sticker/src/manifest.ts@@ -1,3 +1,4 @@+import { timelineTypes } from "@hypit/timeline";import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";import { readFile } from "node:fs/promises";@@ -5,9 +6,9 @@ import { artifactDependency, artifactTypes } from "@hypit/artifact";import { compositionDependency, compositionTypes } from "@hypit/composition";import { fontArtifactSchema, mediaDependency, mediaTypes } from "@hypit/media";import { narrativeDependency } from "@hypit/narrative";-import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";+import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";-import { semanticTrackDependency } from "@hypit/semantic-track";+import { timelineDependency } from "@hypit/timeline";import { spatialDependency, spatialFrameSchema, spatialTypes } from "@hypit/spatial";import { svsRecipeType } from "@hypit/svs";import { temporalDependency, temporalTypes } from "@hypit/temporal";@@ -134,7 +135,7 @@ export const commentStickerProgramSchema: ValueSchema = object({});const appendInputs = [{ name: "set", type: commentStickerTypes.set }, { name: "header", type: commentStickerTypes.header },- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },{ name: "frame", type: spatialTypes.frame }, { name: "style", type: commentStickerTypes.style },{ name: "spec", type: commentStickerTypes.itemSpec },{ name: "content", type: commentStickerTypes.content }, { name: "window", type: temporalTypes.window },@@ -311,7 +312,7 @@ export const commentStickerMarkupSurfaces = [summary: "That Program rendered as one VisualTrack." },],example: `-<comment:Track id="comments" canvas={vertical} semantic={speech.semantic}>+<comment:Track id="comments" canvas={vertical} timeline={speech.timeline}><comment:Sticker id="one" frame={comment-frame} style={social} avatar={viewer-avatar}author="@viewer" meta="Featured" during={story.selection.reaction}>Wait, it pinned the caption to the word, not the second.@@ -334,7 +335,7 @@ export const commentStickerManifest: ModuleManifest = {format: "hypit.module@1",name: commentStickerModuleRef.name,version: commentStickerModuleRef.version,- dependencies: [artifactDependency, narrativeDependency, programSpaceDependency, semanticTrackDependency, spatialDependency, temporalDependency, mediaDependency, compositionDependency, textDependency],+ dependencies: [artifactDependency, narrativeDependency, timelineDependency, spatialDependency, temporalDependency, mediaDependency, compositionDependency, textDependency],types: [{ name: commentStickerTypes.header.name },{ name: commentStickerTypes.style.name },@@ -366,7 +367,7 @@ export const commentStickerManifest: ModuleManifest = {needs: [],})),{ name: commentStickerProducers.finalize.name, inputs: [{ name: "set", type: commentStickerTypes.set }, { name: "header", type: commentStickerTypes.header }], outputs: [{ name: "program", type: commentStickerTypes.program }], needs: [] },- { name: commentStickerProducers.render.name, inputs: [{ name: "canvas", type: spatialTypes.canvas }, { name: "space", type: programSpaceTypes.programSpace }, { name: "program", type: commentStickerTypes.program }], outputs: [{ name: "track", type: compositionTypes.visualTrack }], needs: [] },+ { name: commentStickerProducers.render.name, inputs: [{ name: "canvas", type: spatialTypes.canvas }, { name: "timeline", type: timelineTypes.track }, { name: "program", type: commentStickerTypes.program }], outputs: [{ name: "track", type: compositionTypes.visualTrack }], needs: [] },],};diff --git a/packages/comment-sticker/src/program.ts b/packages/comment-sticker/src/program.tsindex b44b458a..67d3320c 100644--- a/packages/comment-sticker/src/program.ts+++ b/packages/comment-sticker/src/program.ts@@ -1,3 +1,4 @@+import type { Timeline } from "@hypit/timeline";import {assertVisualTrackIdentity,sealVisualTrack,@@ -12,7 +13,6 @@ import type {} from "@hypit/composition";import { assertFontArtifactRef } from "@hypit/media";import { assertProgramSpaceIdentity } from "@hypit/program-space";-import type { ProgramSpace } from "@hypit/program-space";import { canonicalize, isResourceId } from "@hypit/protocol";import type { BlobRef } from "@hypit/protocol";import { assertCanvasSpace, assertSpatialFrame } from "@hypit/spatial";@@ -195,7 +195,7 @@ export function assertCommentStickerSet(value: CommentStickerSet): void {function realized(set: CommentStickerSet,header: CommentStickerHeader,- space: ProgramSpace,+ timeline: Timeline,frame: SpatialFrame,style: CommentStickerStyle,spec: CommentStickerItemSpec,@@ -229,7 +229,7 @@ function realized(export function appendProjectedCommentSticker(set: CommentStickerSet,header: CommentStickerHeader,- space: ProgramSpace,+ timeline: Timeline,frame: SpatialFrame,style: CommentStickerStyle,spec: CommentStickerItemSpec,@@ -237,8 +237,8 @@ export function appendProjectedCommentSticker(window: ProjectedWindow,avatar?: BlobRef,): CommentStickerSet {- assertTemporalWindowFor(window, { subjectId: spec.id, space });- return realized(set, header, space, frame, style, spec, content, window, avatar);+ assertTemporalWindowFor(window, { subjectId: spec.id, space: timeline });+ return realized(set, header, timeline, frame, style, spec, content, window, avatar);}export function assertCommentStickerProgram(value: CommentStickerProgram): void {@@ -589,12 +589,12 @@ function stickerElements(item: CommentStickerItemProgram): readonly VisualElemenreturn elements;}-export function renderCommentSticker(canvas: CanvasSpace, space: ProgramSpace, program: CommentStickerProgram): VisualTrack {+export function renderCommentSticker(canvas: CanvasSpace, timeline: Timeline, program: CommentStickerProgram): VisualTrack {assertCanvasSpace(canvas);- assertProgramSpaceIdentity(space);+ assertProgramSpaceIdentity(timeline);assertCommentStickerProgram(program);const track = sealVisualTrack({- programSpaceId: space.id,+ programSpaceId: timeline.id,visualIr: "hypit.visual-ir@1",id: program.id,presents: program.items.map((item) => ({@@ -605,6 +605,6 @@ export function renderCommentSticker(canvas: CanvasSpace, space: ProgramSpace, pelements: stickerElements(item),})),});- assertVisualTrackIdentity(track, space);+ assertVisualTrackIdentity(track, timeline);return track;}diff --git a/packages/comment-sticker/src/surface.ts b/packages/comment-sticker/src/surface.tsindex 9ec9e638..27fb3ebd 100644--- a/packages/comment-sticker/src/surface.ts+++ b/packages/comment-sticker/src/surface.ts@@ -1,4 +1,4 @@-import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";+import { resolveTemporalContext } from "@hypit/temporal-markup";import { artifactTypes } from "@hypit/artifact";import { mediaTypes } from "@hypit/media";import type { FontStackRef } from "@hypit/media";@@ -88,11 +88,10 @@ export const decodeCommentStickerStyleSurface: StructuredSurfaceHandler = ({ ele};export const decodeCommentStickerTrackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {- allowed(element, ["id", "canvas", "semantic"], ["id", "canvas", "semantic"]);+ allowed(element, ["id", "canvas", "timeline"], ["id", "canvas", "timeline"]);const id = text(element, "id");const canvas = reference(element.attributes.canvas, `${element.name}.canvas`, spatialTypes.canvas, resolveReference);const context = resolveTemporalContext({ element, resolveReference });- const time = createTemporalSpace({ id: id, element, ...context });const headerId = `${id}.__header`;const records: SurfaceRecordDraft[] = [{id: headerId,@@ -100,9 +99,9 @@ export const decodeCommentStickerTrackSurface: StructuredSurfaceHandler = ({ elevalue: { kind: "inline", value: sealCommentStickerHeader({ id }) },range: element.range,}];- const temporalComponents: SurfaceComponentDraft[] = [...time.components];- const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [...time.fragments];- const inputs: Record<string, typeof canvas.ref> = { canvas: canvas.ref, header: { kind: "record", id: headerId }, space: time.space.ref };+ const temporalComponents: SurfaceComponentDraft[] = [];+ const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [];+ const inputs: Record<string, typeof canvas.ref> = { canvas: canvas.ref, header: { kind: "record", id: headerId }, timeline: context.timeline.ref };const items: Parameters<typeof createCommentStickerFragment>[0][number][] = [];for (const child of element.children) {if (child.kind === "text") {@@ -113,7 +112,7 @@ export const decodeCommentStickerTrackSurface: StructuredSurfaceHandler = ({ eleif (child.children.some((node) => node.kind === "element")) throw new Error(`${child.name} accepts plain comment text only.`);allowed(child, ["id", "comment", "frame", "style", "avatar", "author", "header", "meta", ...TIMING], ["id", "frame", "style"]);const itemId = text(child, "id");- const temporal = createTemporalWindowProjection({ id: itemId, element: child, ...context, space: time.space, resolveReference });+ const temporal = createTemporalWindowProjection({ id: itemId, element: child, ...context, resolveReference });records.push(...temporal.records); temporalComponents.push(...temporal.components); temporalFragments.push(...temporal.fragments);const suffix = String(items.length + 1).padStart(4, "0");const frame = reference(child.attributes.frame, `${child.name}.frame`, spatialTypes.frame, resolveReference);diff --git a/packages/comment-sticker/test/comment-sticker.test.ts b/packages/comment-sticker/test/comment-sticker.test.tsindex 8c1e8c82..5cfd9ebe 100644--- a/packages/comment-sticker/test/comment-sticker.test.ts+++ b/packages/comment-sticker/test/comment-sticker.test.ts@@ -1,7 +1,8 @@+import { sealTimeline } from "@hypit/timeline";import assert from "node:assert/strict";import test from "node:test";import { fixtureResource } from "../../../test/fixture-resource.js";-import { semanticTrackFixture } from "../../../test/semantic-track-fixture.js";+import { timelineFixture } from "../../../test/timeline-fixture.js";import { projectProgramWindow } from "../../../test/temporal-fixture.js";import { artifactTypes } from "@hypit/artifact";@@ -9,7 +10,7 @@ import { compileHyperframesDocument } from "@hypit/hyperframes";import type { FontArtifactRef, FontStackRef } from "@hypit/media";import { mediaTypes } from "@hypit/media";import { sealProgramSpace } from "@hypit/program-space";-import { semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";+import { timelineProducers, timelineTypes } from "@hypit/timeline";import { spatialTypes } from "@hypit/spatial";import { svsRecipeType } from "@hypit/svs";import type { SvsRecipe } from "@hypit/svs";@@ -50,9 +51,9 @@ const style = decodeCommentStickerStyle(recipe, fonts, "social-comment");const frame = { xPx: 80, yPx: 140, widthPx: 920, heightPx: 360 };const canvas = { widthPx: 1080, heightPx: 1920,origin: "top-left" as const, xDirection: "right" as const, yDirection: "down" as const, pixelAspect: "square" as const };-const space = sealProgramSpace({ id: "test-space", durationSec: 3, frameRate: { numerator: 30, denominator: 1 } });+const space = sealTimeline({ items: [], id: "test-space", durationSec: 3, frameRate: { numerator: 30, denominator: 1 } });const header = sealCommentStickerHeader({ id: "comments" });-const semantic = semanticTrackFixture(space);+const semantic = timelineFixture(space);function item(id: string) {return sealCommentStickerItemSpec({@@ -170,7 +171,7 @@ test("Track Surface lowers mixed program and semantic Stickers to a finite expliconst blob = { kind: "blob" as const, resource: fixtureResource("comment-avatar"), size: 128, mediaType: "image/png" };const refs = new Map<string, SurfaceResolvedReference>([["video.canvas", authored("video.canvas", spatialTypes.canvas, canvas)],- ["video.semantic", authored("video.semantic", semanticTrackTypes.track, semantic)],+ ["video.timeline", authored("video.timeline", timelineTypes.track, semantic)],["layout.comment", authored("layout.comment", spatialTypes.frame, frame)],["social", authored("social", commentStickerTypes.style, style)],["avatar", authored("avatar", artifactTypes.blob, blob)],@@ -178,7 +179,7 @@ test("Track Surface lowers mixed program and semantic Stickers to a finite expli]);const result = await decodeCommentStickerTrackSurface({sourceName: "fixture.svml",- element: parsed(`<comment:Track id="comments" canvas={video.canvas} semantic={video.semantic}>+ element: parsed(`<comment:Track id="comments" canvas={video.canvas} timeline={video.timeline}><comment:Sticker id="one" comment={copy} frame={layout.comment} style={social} avatar={avatar} author="@viewer" meta="Featured" during="program"/></comment:Track>`),resolveReference: (path) => refs.get(path),@@ -195,8 +196,7 @@ test("Track Surface lowers mixed program and semantic Stickers to a finite explitemporalProducers.projectProgramInstant.name,temporalProducers.projectProgramInstant.name,temporalProducers.composeWindow.name,- semanticTrackProducers.projectProgramSpace.name,- commentStickerProducers.render.name,+ commentStickerProducers.render.name,].sort());assert.equal(result.components.find((component) => component.outputs.track !== undefined)?.outputs.track, "comments.track");});diff --git a/packages/composition/README.md b/packages/composition/README.mdindex eadb048f..30f695f0 100644--- a/packages/composition/README.md+++ b/packages/composition/README.md@@ -51,3 +51,24 @@ A Composition carries its id, Canvas with clear color, and peer Tracks. The TracProgramSpace identity; assembly and rendering receive the corresponding time domain separately.`@hypit/film` is one author-facing way to assemble it;`@hypit/render-hyperframes` consumes it to produce a video.++### Visibility without a new playback origin++A VisualPresent may supply `visibility`, an ordered list of non-overlapping subranges in program+frames, all inside its `span`. Omission means the full span; an empty list means never visible.+`span` continues to define local animation and media-sampling time. This permits rule overrides or+other partial visibility without cutting a program into restarted copies. HyperFrames applies the+visibility independently when seeking; custom HTML and typed media keep their original clocks.++## Audio presentation on the program clock++AudioClip optionally carries `gainEnvelope: { sample, gain }[]` and+`audibility: { startSample, endSampleExclusive }[]`. Both use absolute 48 kHz program samples.+Envelope points have strictly increasing sample positions and interpolate linearly, holding the+outer endpoint gains. They multiply the existing Clip gain and fades. Audible subranges are ordered,+disjoint and inside the original target; omission means the whole target, an empty list means silence.+`audioEnvelopeGainAt` evaluates the envelope; `assertAudioPresentation` checks these values.++Original target and source sampling remain intact when presentation is partially hidden. Range+rendering applies envelopes and masks before cropping. These fields express generic sound behavior;+source selection and Style/Use precedence belong to author components such as Sound.diff --git a/packages/composition/src/audio-presentation.ts b/packages/composition/src/audio-presentation.tsnew file mode 100644index 00000000..5f5e500f--- /dev/null+++ b/packages/composition/src/audio-presentation.ts@@ -0,0 +1,44 @@+/** Gain and audibility are measured on the complete 48 kHz program clock. */+export type AudioSampleSpan = { readonly startSample: number; readonly endSampleExclusive: number };+export type AudioGainPoint = { readonly sample: number; readonly gain: number };+export type AudioPresentation = {+ /** Linear interpolation; hold the first/last gain outside the supplied points. */+ readonly gainEnvelope?: readonly AudioGainPoint[];+ /** Half-open audible regions; omission means the complete target, [] means silence. */+ readonly audibility?: readonly AudioSampleSpan[];+};++export function assertAudioPresentation(value: AudioPresentation, target: AudioSampleSpan,+ totalSamples = Number.MAX_SAFE_INTEGER): void {+ if (value.gainEnvelope !== undefined) {+ if (!Array.isArray(value.gainEnvelope) || value.gainEnvelope.length === 0) throw new Error("Audio gain envelope needs points.");+ let previous = -1;+ for (const point of value.gainEnvelope) {+ if (!Number.isSafeInteger(point.sample) || point.sample <= previous || point.sample > totalSamples+ || !Number.isFinite(point.gain) || point.gain < 0 || point.gain > 64) throw new Error("Audio gain envelope point is invalid.");+ previous = point.sample;+ }+ }+ if (value.audibility !== undefined) {+ if (!Array.isArray(value.audibility)) throw new Error("Audio audibility must be sample intervals.");+ let previous = target.startSample;+ for (const span of value.audibility) {+ if (!Number.isSafeInteger(span.startSample) || span.startSample < previous+ || !Number.isSafeInteger(span.endSampleExclusive) || span.endSampleExclusive <= span.startSample+ || span.endSampleExclusive > target.endSampleExclusive) throw new Error("Audio audibility is outside its target or overlaps.");+ previous = span.endSampleExclusive;+ }+ }+}++export function audioEnvelopeGainAt(points: readonly AudioGainPoint[] | undefined, sample: number): number {+ if (points === undefined) return 1;+ const first = points[0];+ if (first === undefined) throw new Error("Audio gain envelope needs points.");+ if (sample <= first.sample) return first.gain;+ for (let index = 1; index < points.length; index++) {+ const left = points[index - 1]!, right = points[index]!;+ if (sample <= right.sample) return left.gain + (right.gain - left.gain) * (sample - left.sample) / (right.sample - left.sample);+ }+ return points.at(-1)!.gain;+}diff --git a/packages/composition/src/index.ts b/packages/composition/src/index.tsindex 83341bbf..bc932704 100644--- a/packages/composition/src/index.ts+++ b/packages/composition/src/index.ts@@ -16,3 +16,6 @@ export {export { assertAudioTrackIdentity, assertCompositionIdentity, assertVisualTrackIdentity, sealAudioTrack, sealComposition, sealVisualTrack } from "./track.js";export { animatableLocalStyles } from "./track.js";export type * from "./track.js";++export * from "./audio-presentation.js";+export { audioSampleSpanSchema, audioGainEnvelopeSchema } from "./schema.js";diff --git a/packages/composition/src/schema.ts b/packages/composition/src/schema.tsindex a09c246e..dc0d460a 100644--- a/packages/composition/src/schema.ts+++ b/packages/composition/src/schema.ts@@ -170,13 +170,16 @@ export const visualSurfaceSchema: ValueSchema = object({ ...base, kind: { schemaexport const visualProgramSchema: ValueSchema = object({ ...base, kind: { schema: { kind: "literal", value: "program" } }, program: { schema: object({ format: { schema: string }, payload: { schema: object({}, true) }, artifacts: { schema: { kind: "array", items: mediaBlobRef } } }) } });export const visualElementSchema: ValueSchema = { kind: "oneOf", variants: [visualBoxSchema, visualMaskSchema, visualTextSchema, textFlowElement, pathTextElement, visualImageSchema, visualVideoSchema, visualSurfaceSchema, visualProgramSchema] };const span = object({ startFrame: { schema: integer }, endFrameExclusive: { schema: integer } });-const present = object({ id: { schema: string }, span: { schema: span }, stacking: { schema: object({ order: { schema: signedInteger }, tieBreak: { schema: string } }) }, elements: { schema: { kind: "array", minItems: 1, items: visualElementSchema } } });+const present = object({ id: { schema: string }, subjectId: { schema: string, optional: true }, span: { schema: span }, visibility: { schema: { kind: "array", items: span }, optional: true }, stacking: { schema: object({ order: { schema: signedInteger }, tieBreak: { schema: string } }) }, elements: { schema: { kind: "array", minItems: 1, items: visualElementSchema } } });export const visualTrackSchema: ValueSchema = object({ kind: { schema: { kind: "literal", value: "visual" } }, programSpaceId: { schema: string }, visualIr: { schema: { kind: "literal", value: VISUAL_IR_V1 } }, id: { schema: string }, presents: { schema: { kind: "array", items: present } } });-const audioSampleSpan = object({ startSample: { schema: integer }, endSampleExclusive: { schema: { kind: "number", integer: true, minimum: 1 } } });+export const audioSampleSpanSchema = object({ startSample: { schema: integer }, endSampleExclusive: { schema: { kind: "number", integer: true, minimum: 1 } } });+export const audioGainEnvelopeSchema: ValueSchema = { kind: "array", minItems: 1, items: object({+ sample: { schema: integer }, gain: { schema: { kind: "number", minimum: 0, maximum: 64 } },+}) };const audioClip = object({id: { schema: string },artifact: { schema: audioBlobRef },- target: { schema: audioSampleSpan },+ target: { schema: audioSampleSpanSchema },source: { schema: object({sampleFrames: { schema: { kind: "number", integer: true, minimum: 1 } },startSample: { schema: integer },@@ -189,6 +192,8 @@ const audioClip = object({gain: { schema: { kind: "number", minimum: 0, maximum: 64 } },fadeInSamples: { schema: integer },fadeOutSamples: { schema: integer },+ gainEnvelope: { schema: audioGainEnvelopeSchema, optional: true },+ audibility: { schema: { kind: "array", items: audioSampleSpanSchema }, optional: true },});export const audioTrackSchema: ValueSchema = object({ kind: { schema: { kind: "literal", value: "audio" } }, programSpaceId: { schema: string }, id: { schema: string }, clips: { schema: { kind: "array", items: audioClip } } });export const compositionSchema: ValueSchema = object({ id: { schema: string }, canvas: { schema: object({ width: { schema: integer }, height: { schema: integer }, clearColor: { schema: string } }) }, tracks: { schema: { kind: "array", items: { kind: "oneOf", variants: [visualTrackSchema, audioTrackSchema] } } } });diff --git a/packages/composition/src/track.ts b/packages/composition/src/track.tsindex f0aa1d4d..214ffc67 100644--- a/packages/composition/src/track.ts+++ b/packages/composition/src/track.ts@@ -1,3 +1,5 @@+import { assertAudioPresentation } from "./audio-presentation.js";+import type { AudioPresentation } from "./audio-presentation.js";import { canonicalStringify, isResourceId } from "@hypit/protocol";import type { BlobRef, CanonicalValue } from "@hypit/protocol";@@ -361,6 +363,8 @@ export type VisualPresent = {/** Optional domain entity implemented by this renderer Present. */readonly subjectId?: string;readonly span: FrameSpan;+ /** Optional visible subranges in program frames; span remains the animation and sampling origin. */+ readonly visibility?: readonly FrameSpan[];/** Absolute paint position for this Present, not for its authoring Track. */readonly stacking: {readonly order: number;@@ -380,7 +384,7 @@ export type VisualTrack = {readonly presents: readonly VisualPresent[];};-export type AudioClip = {+export type AudioClip = AudioPresentation & {readonly id: string;/** Optional domain entity implemented by this renderer Clip. */readonly subjectId?: string;@@ -935,6 +939,12 @@ function assertPresent(present: VisualPresent, programSpace: ProgramSpace | undeassertNonEmpty(present.id, `${trackId} Present id`);if (present.subjectId !== undefined) assertNonEmpty(present.subjectId, `${trackId}.${present.id} subjectId`);assertFrameSpan(present.span, totalFrames, `${trackId}.${present.id}.span`);+ let previousEnd = present.span.startFrame;+ for (const span of present.visibility ?? []) {+ assertFrameSpan(span, present.span.endFrameExclusive, `${trackId}.${present.id}.visibility`);+ if (span.startFrame < previousEnd) throw new Error(`${trackId}.${present.id} visibility must be ordered within its span.`);+ previousEnd = span.endFrameExclusive;+ }assertNonEmpty(present.stacking.tieBreak, `${trackId}.${present.id} stacking tieBreak`);if (!Number.isSafeInteger(present.stacking.order)) {throw new Error(`${trackId}.${present.id} has invalid stacking order.`);@@ -1236,6 +1246,7 @@ function visualTrackContent(value: Omit<VisualTrack, "kind">): VisualTrack {id: present.id,...(present.subjectId === undefined ? {} : { subjectId: present.subjectId }),span: { ...present.span },+ ...(present.visibility === undefined ? {} : { visibility: present.visibility.map(span => ({ ...span })) }),stacking: { ...present.stacking },elements: [...present.elements].map(normalizeElement).sort((a, b) => a.order - b.order || a.id.localeCompare(b.id)),}))@@ -1263,6 +1274,8 @@ function audioTrackContent(value: Omit<AudioTrack, "kind">): AudioTrack {gain: clip.gain,fadeInSamples: clip.fadeInSamples,fadeOutSamples: clip.fadeOutSamples,+ ...(clip.gainEnvelope === undefined ? {} : { gainEnvelope: clip.gainEnvelope.map(point => ({ ...point })) }),+ ...(clip.audibility === undefined ? {} : { audibility: clip.audibility.map(span => ({ ...span })) }),})).sort((a, b) => a.target.startSample - b.target.startSample || a.id.localeCompare(b.id)),};@@ -1337,6 +1350,7 @@ export function assertAudioTrackIdentity(track: AudioTrack, programSpace?: Progrif (!Number.isFinite(clip.gain) || clip.gain < 0 || clip.gain > 64) {throw new Error(`${track.id}.${clip.id} gain is invalid.`);}+ assertAudioPresentation(clip, clip.target, totalSamples);const targetLength = clip.target.endSampleExclusive - clip.target.startSample;if (!Number.isSafeInteger(clip.fadeInSamples) || clip.fadeInSamples < 0 || clip.fadeInSamples > targetLength|| !Number.isSafeInteger(clip.fadeOutSamples) || clip.fadeOutSamples < 0diff --git a/packages/deck-track/README.md b/packages/deck-track/README.mdindex 53461abb..e6e89043 100644--- a/packages/deck-track/README.md+++ b/packages/deck-track/README.md@@ -1,7 +1,7 @@# `@hypit/deck-track`-The Track Surface accepts `semantic={speech.semantic}` or `space={animation}`. Semantic context-resolves Script Selections and Moments; authored space supports clock-based animation. Shared `at`+The Track Surface accepts `timeline={program.timeline}`. The same Timeline supports authored+positions and, where prepared Takes supply evidence, Script Selections and Moments. Shared `at`inputs accept a Moment or a time such as `2s`; `at` with `for` produces a Window where required.The official, deliberately narrow depth-stack Deck author package.@@ -19,7 +19,7 @@ Media layer lowerer, but Deck is not a Media mode and exports only an ordinary p<deck:DepthStackid="proof-stack"- semantic={speech.semantic}+ timeline={speech.timeline}canvas={vertical}frame={layout.proof-stack}until={story.selection.proof}diff --git a/packages/deck-track/package.json b/packages/deck-track/package.jsonindex 2b1d5eb2..c4ffe019 100644--- a/packages/deck-track/package.json+++ b/packages/deck-track/package.json@@ -20,7 +20,7 @@"@hypit/narrative": "workspace:*","@hypit/program-space": "workspace:*","@hypit/protocol": "workspace:*",- "@hypit/semantic-track": "workspace:*",+ "@hypit/timeline": "workspace:*","@hypit/spatial": "workspace:*","@hypit/svs": "workspace:*","@hypit/temporal": "workspace:*",diff --git a/packages/deck-track/src/component.ts b/packages/deck-track/src/component.tsindex 587bffe3..f86ed711 100644--- a/packages/deck-track/src/component.ts+++ b/packages/deck-track/src/component.ts@@ -1,7 +1,7 @@+import type { Timeline } from "@hypit/timeline";import type { ComponentPackage, ProducerHandlerContext } from "@hypit/component-kit";import { canonicalize } from "@hypit/protocol";import type { StoredValue } from "@hypit/protocol";-import type { ProgramSpace } from "@hypit/program-space";import type { CanvasSpace, SpatialFrame } from "@hypit/spatial";import type { TemporalInstant } from "@hypit/temporal";import type { MediaLayerSet } from "@hypit/media-track";@@ -32,7 +32,7 @@ function finalizeInputs(inputs: ProducerHandlerContext["inputs"]) {header: inline<DepthStackHeader>(inputs.header?.value, "DepthStackHeader"),frame: inline<SpatialFrame>(inputs.frame?.value, "SpatialFrame"),spec: inline<DepthStackSpec>(inputs.spec?.value, "DepthStackSpec"),- space: inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ timeline: inline<Timeline>(inputs.timeline?.value, "Timeline"),};}@@ -53,7 +53,7 @@ export const depthStackComponent = {producer: depthStackProducers.appendCard,handler: ({ inputs }) => ({ outputs: { set: output(appendDepthStackCard(inline<DepthStackCardSet>(inputs.set?.value, "DepthStackCardSet"),- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ inline<Timeline>(inputs.timeline?.value, "Timeline"),inline<MediaLayerSet>(inputs.material?.value, "MediaLayerSet"),inline<DepthStackCardLabel>(inputs.label?.value, "DepthStackCardLabel"),inline<DepthStackCardSpec>(inputs.spec?.value, "DepthStackCardSpec"),@@ -66,7 +66,7 @@ export const depthStackComponent = {const value = finalizeInputs(inputs);return { outputs: { program: output(finalizeDepthStack(value.set, value.header, value.frame, value.spec,- inline<TemporalInstant>(inputs.terminal?.value, "TemporalInstant"), value.space,+ inline<TemporalInstant>(inputs.terminal?.value, "TemporalInstant"), value.timeline,)) }, needs: {} };},},@@ -74,7 +74,7 @@ export const depthStackComponent = {producer: depthStackProducers.render,handler: ({ inputs }) => ({ outputs: { track: output(renderDepthStack(inline<CanvasSpace>(inputs.canvas?.value, "CanvasSpace"),- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ inline<Timeline>(inputs.timeline?.value, "Timeline"),inline<DepthStackProgram>(inputs.program?.value, "DepthStackProgram"),)) }, needs: {} }),},diff --git a/packages/deck-track/src/fragment.ts b/packages/deck-track/src/fragment.tsindex 1e052e2a..5ecc0150 100644--- a/packages/deck-track/src/fragment.ts+++ b/packages/deck-track/src/fragment.ts@@ -1,4 +1,5 @@-import { programSpaceTypes } from "@hypit/program-space";+import { timelineTypes } from "@hypit/timeline";+import { artifactTypes } from "@hypit/artifact";import { compositionTypes } from "@hypit/composition";import { sealGraphFragment } from "@hypit/elaborator";@@ -42,7 +43,7 @@ export function createDepthStackFragment({ name: "canvas", type: spatialTypes.canvas },{ name: "frame", type: spatialTypes.frame },{ name: "header", type: depthStackTypes.header },- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },{ name: "spec", type: depthStackTypes.spec },];const operations: FragmentOperation[] = [@@ -107,7 +108,7 @@ export function createDepthStackFragment(producer: depthStackProducers.appendCard,inputs: {set: cardSet,- space: input("space"),+ timeline: input("timeline"),material: operation(sampleId),label: input(card.labelName),spec: input(card.cardSpecName),@@ -126,7 +127,7 @@ export function createDepthStackFragment(header: input("header"),frame: input("frame"),spec: input("spec"),- space: input("space"),+ timeline: input("timeline"),terminal: input(terminalName),},result: { kind: "output", name: "program" },@@ -134,7 +135,7 @@ export function createDepthStackFragment(operations.push({id: "track",producer: depthStackProducers.render,- inputs: { canvas: input("canvas"), space: input("space"), program: operation("program") },+ inputs: { canvas: input("canvas"), timeline: input("timeline"), program: operation("program") },result: { kind: "output", name: "track" },});return sealGraphFragment({diff --git a/packages/deck-track/src/lower.ts b/packages/deck-track/src/lower.tsindex bfa4c38c..7ec9eab9 100644--- a/packages/deck-track/src/lower.ts+++ b/packages/deck-track/src/lower.ts@@ -1,3 +1,4 @@+import type { Timeline } from "@hypit/timeline";import {assertVisualTrackIdentity,sealVisualTrack,@@ -20,7 +21,6 @@ import type {MediaSampleLayerProgram,MediaVisualTrim,} from "@hypit/media-track";-import type { ProgramSpace } from "@hypit/program-space";import { assertCanvasSpace } from "@hypit/spatial";import type { CanvasSpace, SpatialFrame } from "@hypit/spatial";@@ -170,17 +170,17 @@ function samplingFor(input: {readonly relation: number;readonly stageStart: number;readonly stageDuration: number;- readonly space: ProgramSpace;+ readonly timeline: Timeline;}): VisualTimedSampling | undefined {const timing = sourceTiming(input.layer);if (timing === undefined) return undefined;- assert(timing.frameRate.numerator === input.space.frameRate.numerator- && timing.frameRate.denominator === input.space.frameRate.denominator,- `DepthStack Card ${input.card.id} timed material must be normalized to ProgramSpace.`);+ assert(timing.frameRate.numerator === input.timeline.frameRate.numerator+ && timing.frameRate.denominator === input.timeline.frameRate.denominator,+ `DepthStack Card ${input.card.id} timed material must be normalized to Timeline.`);const trim = trimFor(input.layer, timing.frameCount);if (input.relation === 0) {return resolveVisualSampling({- space: input.space,+ timeline: input.timeline,sourceFrameRate: timing.frameRate,sourceFrameCount: timing.frameCount,targetFrameCount: input.stageDuration,@@ -282,10 +282,10 @@ function labelElement(card: DepthStackCard, parent: string, order: number): Visuexport function renderDepthStack(canvas: CanvasSpace,- space: ProgramSpace,+ timeline: Timeline,program: DepthStackProgram,): VisualTrack {- assertDepthStackProgramIdentity(program, space);+ assertDepthStackProgramIdentity(program, timeline);assertCanvasSpace(canvas);const presents: VisualTrack["presents"][number][] = [];for (let stageIndex = 0; stageIndex < program.cards.length; stageIndex += 1) {@@ -322,7 +322,7 @@ export function renderDepthStack(const samplingAnimationOverrides: Record<string, VisualAnimation | null> = {};for (const layer of card.material.layers) {if (layer.kind !== "sample") continue;- const sampling = samplingFor({ program, card, layer, relation, stageStart, stageDuration, space });+ const sampling = samplingFor({ program, card, layer, relation, stageStart, stageDuration, timeline });if (sampling !== undefined) samplingOverrides[layer.id] = sampling;if (relation !== 0 && layer.samplingMotion !== undefined) samplingAnimationOverrides[layer.id] = null;}@@ -337,7 +337,7 @@ export function renderDepthStack(stacking: { order: 0, tieBreak: card.id },sounds: [],};- const material = [...lowerMediaItemElements(mediaItem, space, {+ const material = [...lowerMediaItemElements(mediaItem, timeline, {placementParent: group.poseParent,lifecycleAnimationOverride: null,samplingOverrides,@@ -360,11 +360,11 @@ export function renderDepthStack(}}const track = sealVisualTrack({- programSpaceId: space.id,+ programSpaceId: timeline.id,visualIr: "hypit.visual-ir@1",id: program.id,presents,});- assertVisualTrackIdentity(track, space);+ assertVisualTrackIdentity(track, timeline);return track;}diff --git a/packages/deck-track/src/manifest.ts b/packages/deck-track/src/manifest.tsindex 05ace84e..86df0925 100644--- a/packages/deck-track/src/manifest.ts+++ b/packages/deck-track/src/manifest.ts@@ -1,3 +1,4 @@+import { timelineTypes } from "@hypit/timeline";import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";import { readFile } from "node:fs/promises";@@ -19,9 +20,9 @@ import {mediaTrackTypes,} from "@hypit/media-track";import { narrativeDependency, narrativeTypes } from "@hypit/narrative";-import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";+import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";-import { semanticTrackDependency } from "@hypit/semantic-track";+import { timelineDependency } from "@hypit/timeline";import { spatialDependency, spatialFrameSchema, spatialTypes } from "@hypit/spatial";import { svsRecipeType } from "@hypit/svs";import { temporalDependency, temporalTypes } from "@hypit/temporal";@@ -118,7 +119,7 @@ export const depthStackProgramSchema: ValueSchema = object({const finalizeInputs = [{ name: "set", type: depthStackTypes.cardSet }, { name: "header", type: depthStackTypes.header },{ name: "frame", type: spatialTypes.frame }, { name: "spec", type: depthStackTypes.spec },- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },] as const;export const depthStackMarkupSurfaces = [@@ -386,7 +387,7 @@ export const depthStackMarkupSurfaces = [],example: `<deck:DepthStackid="proof-stack"- semantic={speech.semantic}+ timeline={speech.timeline}canvas={vertical}frame={layout.proof-stack}until={story.selection.proof}@@ -408,7 +409,7 @@ export const depthStackMarkupSurfaces = [export const depthStackManifest: ModuleManifest = {format: "hypit.module@1", name: depthStackModuleRef.name, version: depthStackModuleRef.version,- dependencies: [narrativeDependency, programSpaceDependency, semanticTrackDependency, spatialDependency, temporalDependency, mediaDependency, mediaTrackDependency, compositionDependency, textDependency],+ dependencies: [narrativeDependency, timelineDependency, spatialDependency, temporalDependency, mediaDependency, mediaTrackDependency, compositionDependency, textDependency],types: [{ name: depthStackTypes.header.name },{ name: depthStackTypes.spec.name },@@ -422,7 +423,7 @@ export const depthStackManifest: ModuleManifest = {{ name: depthStackProducers.bindLabelText.name, inputs: [{ name: "style", type: depthStackTypes.cardLabelStyle }, { name: "content", type: textTypes.text }], outputs: [{ name: "label", type: depthStackTypes.cardLabel }], needs: [] },{ name: depthStackProducers.createCards.name, inputs: [], outputs: [{ name: "set", type: depthStackTypes.cardSet }], needs: [] },{ name: depthStackProducers.appendCard.name, inputs: [- { name: "set", type: depthStackTypes.cardSet }, { name: "space", type: programSpaceTypes.programSpace }, { name: "material", type: mediaTrackTypes.layerSet },+ { name: "set", type: depthStackTypes.cardSet }, { name: "timeline", type: timelineTypes.track }, { name: "material", type: mediaTrackTypes.layerSet },{ name: "label", type: depthStackTypes.cardLabel }, { name: "spec", type: depthStackTypes.cardSpec },{ name: "activation", type: temporalTypes.instant },], outputs: [{ name: "set", type: depthStackTypes.cardSet }], needs: [] },@@ -430,7 +431,7 @@ export const depthStackManifest: ModuleManifest = {inputs: [...finalizeInputs, { name: "terminal", type: temporalTypes.instant }],outputs: [{ name: "program", type: depthStackTypes.program }], needs: [] },{ name: depthStackProducers.render.name, inputs: [- { name: "canvas", type: spatialTypes.canvas }, { name: "space", type: programSpaceTypes.programSpace },+ { name: "canvas", type: spatialTypes.canvas }, { name: "timeline", type: timelineTypes.track },{ name: "program", type: depthStackTypes.program },], outputs: [{ name: "track", type: compositionTypes.visualTrack }], needs: [] },],diff --git a/packages/deck-track/src/program.ts b/packages/deck-track/src/program.tsindex 281e80fd..410a3ec9 100644--- a/packages/deck-track/src/program.ts+++ b/packages/deck-track/src/program.ts@@ -1,3 +1,4 @@+import type { Timeline } from "@hypit/timeline";import {assertFontArtifactRef,} from "@hypit/media";@@ -6,11 +7,7 @@ import {assertMediaLayerSet,assertMediaLifecycleMotion,} from "@hypit/media-track";-import {- assertProgramSpaceIdentity,- programSpaceFrameCount,-} from "@hypit/program-space";-import type { ProgramSpace } from "@hypit/program-space";+import { assertProgramSpaceIdentity, programSpaceFrameCount } from "@hypit/program-space";import { canonicalize } from "@hypit/protocol";import { verifyText } from "@hypit/text";import type { Text } from "@hypit/text";@@ -229,13 +226,13 @@ function appendDepthStackCardAtFrame(export function appendDepthStackCard(set: DepthStackCardSet,- space: ProgramSpace,+ timeline: Timeline,material: DepthStackCardSet["cards"][number]["material"],label: DepthStackCardLabel,spec: DepthStackCardSpec,activation: TemporalInstant,): DepthStackCardSet {- assertTemporalInstantFor(activation, { subjectId: spec.id, space });+ assertTemporalInstantFor(activation, { subjectId: spec.id, space: timeline });return appendDepthStackCardAtFrame(set, material, label, spec, activation.frame);}@@ -245,13 +242,13 @@ function finalizeDepthStackAtFrame(frame: SpatialFrame,spec: DepthStackSpec,terminalFrame: number,- space: ProgramSpace,+ timeline: Timeline,): DepthStackProgram {assertDepthStackCardSet(set);assertDepthStackHeader(header);assertSpatialFrame(frame);assertDepthStackSpec(spec);- assertProgramSpaceIdentity(space);+ assertProgramSpaceIdentity(timeline);assert(set.cards.length > 0, "DepthStack requires at least one Card.");const first = set.cards[0]!;const program: DepthStackProgram = {@@ -263,7 +260,7 @@ function finalizeDepthStackAtFrame(spec: structuredClone(spec),cards: structuredClone(set.cards),};- assertDepthStackProgramIdentity(program, space);+ assertDepthStackProgramIdentity(program, timeline);return canonicalize(program) as unknown as DepthStackProgram;}@@ -273,10 +270,10 @@ export function finalizeDepthStack(frame: SpatialFrame,spec: DepthStackSpec,terminal: TemporalInstant,- space: ProgramSpace,+ timeline: Timeline,): DepthStackProgram {- assertTemporalInstantFor(terminal, { subjectId: header.id, space });- return finalizeDepthStackAtFrame(set, header, frame, spec, terminal.frame, space);+ assertTemporalInstantFor(terminal, { subjectId: header.id, space: timeline });+ return finalizeDepthStackAtFrame(set, header, frame, spec, terminal.frame, timeline);}export function assertDepthStackProgram(program: DepthStackProgram): void {@@ -307,10 +304,10 @@ export function assertDepthStackProgram(program: DepthStackProgram): void {}}-export function assertDepthStackProgramIdentity(program: DepthStackProgram, space: ProgramSpace): void {- assertProgramSpaceIdentity(space);+export function assertDepthStackProgramIdentity(program: DepthStackProgram, timeline: Timeline): void {+ assertProgramSpaceIdentity(timeline);assertDepthStackProgram(program);- assert(program.terminalFrame <= programSpaceFrameCount(space), "DepthStack terminal is outside ProgramSpace.");+ assert(program.terminalFrame <= programSpaceFrameCount(timeline), "DepthStack terminal is outside Timeline.");}function modulo(value: number, size: number): number {diff --git a/packages/deck-track/src/surface.ts b/packages/deck-track/src/surface.tsindex fa16fc28..7f2353db 100644--- a/packages/deck-track/src/surface.ts+++ b/packages/deck-track/src/surface.ts@@ -1,4 +1,4 @@-import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";+import { resolveTemporalContext } from "@hypit/temporal-markup";import { artifactTypes } from "@hypit/artifact";import { mediaTypes } from "@hypit/media";import type { FontStackRef } from "@hypit/media";@@ -181,28 +181,27 @@ export const decodeDepthStackLabelSurface: StructuredSurfaceHandler = ({ element};export const decodeDepthStackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {- allowed(element, ["id", "semantic", "space", "canvas", "frame", "appearance", "until", "until-boundary"]);+ allowed(element, ["id", "timeline", "canvas", "frame", "appearance", "until", "until-boundary"]);const id = text(element, "id");const context = resolveTemporalContext({ element, resolveReference });- const time = createTemporalSpace({ id: id, element, ...context });const canvas = reference(element.attributes.canvas, `${element.name}.canvas`, spatialTypes.canvas, resolveReference);const frame = reference(element.attributes.frame, `${element.name}.frame`, spatialTypes.frame, resolveReference);const appearance = recipe(element.attributes.appearance, `${element.name}.appearance`, resolveReference);const terminal = createTemporalInstantProjection({- id: `${id}.terminal`, subjectId: id, element, ...context, space: time.space, resolveReference,+ id: `${id}.terminal`, subjectId: id, element, ...context, resolveReference,semanticAttribute: "until", boundaryAttribute: "until-boundary", boundaryFallback: "end", projectedAttribute: false,});const records: SurfaceRecordDraft[] = [...terminal.records];- const temporalComponents: SurfaceComponentDraft[] = [...time.components, ...terminal.components];- const temporalFragments = [...time.fragments, ...terminal.fragments];+ const temporalComponents: SurfaceComponentDraft[] = [...terminal.components];+ const temporalFragments = [...terminal.fragments];const headerId = `${id}.header`;const specId = `${id}.spec`;records.push({ id: headerId, type: depthStackTypes.header, value: { kind: "inline", value: sealDepthStackHeader({ id }) }, range: element.range },{ id: specId, type: depthStackTypes.spec, value: { kind: "inline", value: decodeDepthStackSpec(appearance) }, range: element.range },);- const inputs: Record<string, typeof time.space.ref> = {- canvas: canvas.ref, frame: frame.ref, header: { kind: "record", id: headerId }, space: time.space.ref,+ const inputs: Record<string, typeof context.timeline.ref> = {+ canvas: canvas.ref, frame: frame.ref, header: { kind: "record", id: headerId }, timeline: context.timeline.ref,spec: { kind: "record", id: specId }, terminal: terminal.ref,};const cards: DepthStackFragmentCard[] = [];@@ -225,7 +224,7 @@ export const decodeDepthStackSurface: StructuredSurfaceHandler = ({ element, res: reference(child.attributes.extent, `${child.name}.extent`, spatialTypes.extent, resolveReference);if (sourceKind === "still" && extent === undefined) throw new Error(`${child.name}.extent is required for a still image.`);if (sourceKind !== "still" && extent !== undefined) throw new Error(`${child.name}.extent belongs only to a still image.`);- const activation = createTemporalInstantProjection({ id: `${id}.${cardId}`, subjectId: cardId, element: child, ...context, space: time.space, resolveReference });+ const activation = createTemporalInstantProjection({ id: `${id}.${cardId}`, subjectId: cardId, element: child, ...context, resolveReference });records.push(...activation.records); temporalComponents.push(...activation.components); temporalFragments.push(...activation.fragments);const cardAppearance = child.attributes.appearance === undefined? appearance : recipe(child.attributes.appearance, `${child.name}.appearance`, resolveReference);@@ -249,7 +248,7 @@ export const decodeDepthStackSurface: StructuredSurfaceHandler = ({ element, resinputs[sampleSpecName] = { kind: "record", id: sampleId };inputs[cardSpecName] = { kind: "record", id: cardSpecId };inputs[activationName] = activation.ref;- let labelRef: typeof time.space.ref;+ let labelRef: typeof context.timeline.ref;if (child.attributes.label === undefined) {const labelId = `${id}.card.${suffix}.label-none`;records.push({ id: labelId, type: depthStackTypes.cardLabel, value: { kind: "inline", value: noDepthStackCardLabel() }, range: child.range });diff --git a/packages/deck-track/test/deck-track.test.ts b/packages/deck-track/test/deck-track.test.tsindex 751ff0e6..0945bdd1 100644--- a/packages/deck-track/test/deck-track.test.ts+++ b/packages/deck-track/test/deck-track.test.ts@@ -1,8 +1,9 @@+import { sealTimeline } from "@hypit/timeline";import assert from "node:assert/strict";import test from "node:test";import { videoContractManifests } from "../../../test/support/video-domain.js";import { fixtureResource } from "../../../test/fixture-resource.js";-import { semanticTrackFixture } from "../../../test/semantic-track-fixture.js";+import { timelineFixture } from "../../../test/timeline-fixture.js";import { compositionDependency, compositionTypes } from "@hypit/composition";import type { VisualElement, VisualTimedSampling } from "@hypit/composition";@@ -39,7 +40,7 @@ import type { MediaLayerSet } from "@hypit/media-track";import { narrativeTypes } from "@hypit/narrative";import { sealProgramSpace } from "@hypit/program-space";import type { ModuleManifest } from "@hypit/protocol";-import { semanticTrackTypes } from "@hypit/semantic-track";+import { timelineTypes } from "@hypit/timeline";import { temporalTypes } from "@hypit/temporal";import type { TemporalInstant } from "@hypit/temporal";import { sealCanvasSpace, sealSpatialFrame, spatialTypes } from "@hypit/spatial";@@ -55,10 +56,10 @@ import type {import { artifactTypes } from "@hypit/artifact";import { mediaTrackManifest, mediaTrackTypes } from "@hypit/media-track";-const space = sealProgramSpace({ id: "test-space", durationSec: 2,+const space = sealTimeline({ items: [], id: "test-space", durationSec: 2,frameRate: { numerator: 30, denominator: 1 },});-const semantic = semanticTrackFixture(space);+const semantic = timelineFixture(space);const canvas = sealCanvasSpace({widthPx: 360,heightPx: 640,@@ -420,7 +421,7 @@ test("the author Surface keeps every source, trigger, terminal, Frame and optionrecord: { value: { kind: "inline", value: { path, properties } } } as never,});const references = new Map<string, SurfaceResolvedReference>([- ["semantic", plain("semantic", semanticTrackTypes.track)],+ ["semantic", plain("semantic", timelineTypes.track)],["canvas", plain("canvas", spatialTypes.canvas)], ["frame", plain("frame", spatialTypes.frame)],["first", plain("first", artifactTypes.blob)], ["first-extent", plain("first-extent", spatialTypes.extent)],["second", plain("second", mediaTypes.synchronized)], ["one", plain("one", narrativeTypes.moment)],@@ -431,7 +432,7 @@ test("the author Surface keeps every source, trigger, terminal, Frame and optionconst result = await decodeDepthStackSurface({sourceName: "deck.svml",element: node("deck:DepthStack", {- id: "proof", semantic: ref("semantic"), canvas: ref("canvas"), frame: ref("frame"),+ id: "proof", timeline: ref("semantic"), canvas: ref("canvas"), frame: ref("frame"),appearance: ref("deck-style"), until: ref("terminal"),}, [node("deck:Card", { id: "one", source: ref("first"), extent: ref("first-extent"), at: ref("one") }),diff --git a/packages/film-studio/package.json b/packages/film-studio/package.jsonindex e53f4218..8a059a1c 100644--- a/packages/film-studio/package.json+++ b/packages/film-studio/package.json@@ -9,7 +9,7 @@"dependencies": {"@hypit/composition": "workspace:*","@hypit/film": "workspace:*",- "@hypit/semantic-track": "workspace:*",+ "@hypit/timeline": "workspace:*","@hypit/studio-adapter": "workspace:*","@hypit/program-space": "workspace:*"}diff --git a/packages/film-studio/src/index.ts b/packages/film-studio/src/index.tsindex e961d120..f14f7cd6 100644--- a/packages/film-studio/src/index.ts+++ b/packages/film-studio/src/index.ts@@ -2,13 +2,13 @@ import { programSpaceTypes } from "@hypit/program-space";import type { StudioFilmCompanion } from "@hypit/studio-adapter";import { compositionTypes } from "@hypit/composition";import { filmModuleRef } from "@hypit/film";-import { semanticTrackTypes } from "@hypit/semantic-track";+import { timelineTypes } from "@hypit/timeline";/** Film owns the author vocabulary that selects its time source and peer Tracks. */export const filmStudioCompanions: readonly StudioFilmCompanion[] = [{id: "film",match: { module: filmModuleRef, surface: "film", outputType: compositionTypes.composition },- timeSources: [{ attribute: "semantic", type: semanticTrackTypes.track }, { attribute: "space", type: programSpaceTypes.programSpace }],+ timeSources: [{ attribute: "timeline", type: timelineTypes.track }],tracks: {childSurface: "Track",sourceAttribute: "source",diff --git a/packages/film/README.md b/packages/film/README.mdindex 29ae3f1c..c9a66bb5 100644--- a/packages/film/README.md+++ b/packages/film/README.md@@ -1,8 +1,7 @@# `@hypit/film`-The Surface accepts `semantic={speech.semantic}` for performance time or `space={animation}` for an-authored ProgramSpace. Its Fragment receives `space` directly; rendering does not require Script-or prepared performance media. Both time sources use the same frame and audio pipeline.+The Surface, Fragment and assembly Producers receive the same Timeline through a `timeline` input.+Partial, overlapping or absent semantic coverage all use that complete range.Official package-owned assembly layer between peer Tracks and the generic Composition waist.Film is an ordinary author component and an optional Target, not a Core root or a privileged video@@ -21,9 +20,9 @@ belongs to each Present's absolute stacking key.The official Structured Surface validates an imported generic SVS Recipe into a nominal`FilmProgram`, receives CanvasSpace and the selected time source through separate explicit edges,-projects ProgramSpace when that source is a SemanticTrack, type-checks+passes the Timeline directly, type-checkseach `<film:Track source={...}/>` reference and generates the finite fold. CanvasSpace is the only-dimension truth, ProgramSpace is the only frame-rate truth, and FilmProgram owns only assembly+dimension truth, Timeline is the only time-range truth, and FilmProgram owns only assemblyidentity and clear color.Child order is organizational: Track and Present identity, timing and absolute stacking remain intheir own typed values. Final rendering is a separate author package and capability.@@ -31,10 +30,16 @@ their own typed values. Final rendering is a separate author package and capabilFor example, after the named inputs are declared:```svml-<film:Film id="main" canvas={vertical} semantic={speech.semantic}+<import as="sound" from="@hypit/sound@1"/>+<sound:Style id="voice-style"/>+<sound:Track id="voice" timeline={speech.timeline}>+ <sound:Use style={voice-style}/>+</sound:Track>++<film:Film id="main" canvas={vertical} timeline={speech.timeline}appearance={recipes.film.vertical}><film:Track source={performance.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={voice.audio}/><film:Track source={coverage.visual}/><film:Track source={captions.track}/><film:Track source={music.track}/>diff --git a/packages/film/package.json b/packages/film/package.jsonindex 72014143..94b4f8dc 100644--- a/packages/film/package.json+++ b/packages/film/package.json@@ -16,7 +16,7 @@"@hypit/elaborator": "workspace:*","@hypit/program-space": "workspace:*","@hypit/protocol": "workspace:*",- "@hypit/semantic-track": "workspace:*",+ "@hypit/timeline": "workspace:*","@hypit/spatial": "workspace:*","@hypit/svs": "workspace:*","@hypit/markup": "workspace:*",diff --git a/packages/film/src/component.ts b/packages/film/src/component.tsindex 68368e20..d6b41c46 100644--- a/packages/film/src/component.ts+++ b/packages/film/src/component.ts@@ -1,5 +1,5 @@+import type { Timeline } from "@hypit/timeline";import type { ComponentPackage } from "@hypit/component-kit";-import type { ProgramSpace } from "@hypit/program-space";import type { CanvasSpace } from "@hypit/spatial";import type { AudioTrack, VisualTrack } from "@hypit/composition";import type { StoredValue } from "@hypit/protocol";@@ -28,7 +28,7 @@ export const filmComponent = {handler: ({ inputs }) => ({outputs: { set: { kind: "inline", value: canonicalize(appendFilmVisualTrack(inline<FilmTrackSet>(inputs.set?.value, "FilmTrackSet"),- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ inline<Timeline>(inputs.timeline?.value, "Timeline"),inline<VisualTrack>(inputs.track?.value, "VisualTrack"),)) } },needs: {},@@ -39,7 +39,7 @@ export const filmComponent = {handler: ({ inputs }) => ({outputs: { set: { kind: "inline", value: canonicalize(appendFilmAudioTrack(inline<FilmTrackSet>(inputs.set?.value, "FilmTrackSet"),- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ inline<Timeline>(inputs.timeline?.value, "Timeline"),inline<AudioTrack>(inputs.track?.value, "AudioTrack"),)) } },needs: {},@@ -51,7 +51,7 @@ export const filmComponent = {outputs: { composition: { kind: "inline", value: canonicalize(compileFilmComposition(inline<FilmProgram>(inputs.program?.value, "FilmProgram"),inline<CanvasSpace>(inputs.canvas?.value, "CanvasSpace"),- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ inline<Timeline>(inputs.timeline?.value, "Timeline"),inline<FilmTrackSet>(inputs.set?.value, "FilmTrackSet"),)) } },needs: {},diff --git a/packages/film/src/fragment.ts b/packages/film/src/fragment.tsindex 305e1743..ccc68c52 100644--- a/packages/film/src/fragment.ts+++ b/packages/film/src/fragment.ts@@ -1,4 +1,5 @@-import { programSpaceTypes } from "@hypit/program-space";+import { timelineTypes } from "@hypit/timeline";+import { spatialTypes } from "@hypit/spatial";import { compositionTypes } from "@hypit/composition";import type { Track } from "@hypit/composition";@@ -12,7 +13,7 @@ const input = (name: string) => ({ kind: "fragment-input" as const, name });const operation = (id: string) => ({ kind: "fragment-operation" as const, operation: id });function assertTrackInputs(tracks: readonly FilmTrackInput[]): FilmTrackInput[] {- const names = new Set(["program", "canvas", "space"]);+ const names = new Set(["program", "canvas", "timeline"]);return [...tracks].map((track) => ({ name: track.name.trim(), kind: track.kind })).sort((left, right) => left.name.localeCompare(right.name))@@ -45,7 +46,7 @@ export function createFilmAssemblyFragment(options: FilmAssemblyFragmentOptions)operations.push({id,producer: track.kind === "visual" ? filmProducers.appendVisualTrack : filmProducers.appendAudioTrack,- inputs: { set: operation(current), space: input("space"), track: input(track.name) },+ inputs: { set: operation(current), timeline: input("timeline"), track: input(track.name) },result: { kind: "output" as const, name: "set" },});current = id;@@ -56,7 +57,7 @@ export function createFilmAssemblyFragment(options: FilmAssemblyFragmentOptions)inputs: {program: input("program"),canvas: input("canvas"),- space: input("space"),+ timeline: input("timeline"),set: operation(current),},result: { kind: "output" as const, name: "composition" },@@ -65,7 +66,7 @@ export function createFilmAssemblyFragment(options: FilmAssemblyFragmentOptions)inputs: [{ name: "program", type: filmTypes.program },{ name: "canvas", type: spatialTypes.canvas },- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },...tracks.map((track) => ({name: track.name,type: track.kind === "visual" ? compositionTypes.visualTrack : compositionTypes.audioTrack,diff --git a/packages/film/src/manifest.ts b/packages/film/src/manifest.tsindex b063f4c5..e163450a 100644--- a/packages/film/src/manifest.ts+++ b/packages/film/src/manifest.ts@@ -1,8 +1,10 @@+import { timelineTypes } from "@hypit/timeline";+import type { Timeline } from "@hypit/timeline";import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";import { audioTrackSchema, compositionDependency, compositionTypes, visualTrackSchema } from "@hypit/composition";-import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";+import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";-import { semanticTrackDependency } from "@hypit/semantic-track";+import { timelineDependency } from "@hypit/timeline";import { spatialDependency, spatialTypes } from "@hypit/spatial";import { svsManifest, svsModuleRef, svsRecipeType } from "@hypit/svs";@@ -47,7 +49,7 @@ export const filmMarkupSurfaces = [{outputs: [filmTypes.program],vocabulary: {summary:- "Assembles any number of peer VisualTrack and AudioTrack references into one Composition against a Canvas and a SemanticTrack.",+ "Assembles any number of peer VisualTrack and AudioTrack references into one Composition against a Canvas and a Timeline.",appearance:"One flat fill of the entire Canvas, in the single hexadecimal color the Recipe's `background` carries, lying behind everything else in the Frame. It covers the full Canvas width and height, holds that one color from the first Frame to the last, and never moves, fades or changes. Wherever nothing is painted over it, that color is what the Frame shows; the Film puts no mark of its own on top of it.",attributes: [@@ -79,9 +81,9 @@ export const filmMarkupSurfaces = [{summary: "The assembled Composition, addressed as `<id>.composition`." },],example: [- '<film:Film id="main" canvas={vertical} semantic={speech.semantic} appearance={recipes.film.vertical}>',+ '<film:Film id="main" canvas={vertical} timeline={speech.timeline} appearance={recipes.film.vertical}>'," <film:Track source={performance.visual}/>",- " <film:Track source={speech.audio}/>",+ " <film:Track source={voice.audio}/>"," <film:Track source={captions.track}/>","</film:Film>",].join("\n"),@@ -99,8 +101,7 @@ export const filmManifest: ModuleManifest = {name: filmModuleRef.name,version: filmModuleRef.version,dependencies: [- programSpaceDependency,- semanticTrackDependency,+ timelineDependency,spatialDependency,compositionDependency,{ module: svsModuleRef },@@ -121,7 +122,7 @@ export const filmManifest: ModuleManifest = {name: filmProducers.appendVisualTrack.name,inputs: [{ name: "set", type: filmTypes.trackSet },- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },{ name: "track", type: compositionTypes.visualTrack },],outputs: [{ name: "set", type: filmTypes.trackSet }],@@ -131,7 +132,7 @@ export const filmManifest: ModuleManifest = {name: filmProducers.appendAudioTrack.name,inputs: [{ name: "set", type: filmTypes.trackSet },- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },{ name: "track", type: compositionTypes.audioTrack },],outputs: [{ name: "set", type: filmTypes.trackSet }],@@ -142,7 +143,7 @@ export const filmManifest: ModuleManifest = {inputs: [{ name: "program", type: filmTypes.program },{ name: "canvas", type: spatialTypes.canvas },- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },{ name: "set", type: filmTypes.trackSet },],outputs: [{ name: "composition", type: compositionTypes.composition }],diff --git a/packages/film/src/program.ts b/packages/film/src/program.tsindex 6f940d6a..148ecc3b 100644--- a/packages/film/src/program.ts+++ b/packages/film/src/program.ts@@ -1,5 +1,5 @@+import type { Timeline } from "@hypit/timeline";import { assertProgramSpaceIdentity } from "@hypit/program-space";-import type { ProgramSpace } from "@hypit/program-space";import { assertCanvasSpace } from "@hypit/spatial";import type { CanvasSpace } from "@hypit/spatial";import { assertAudioTrackIdentity, assertCompositionIdentity, assertVisualTrackIdentity, sealComposition } from "@hypit/composition";@@ -64,7 +64,7 @@ export function createFilmTrackSet(): FilmTrackSet {});}-function appendTrack(set: FilmTrackSet, programSpace: ProgramSpace, track: Track): FilmTrackSet {+function appendTrack(set: FilmTrackSet, programSpace: Timeline, track: Track): FilmTrackSet {assertFilmTrackSetIdentity(set);assertProgramSpaceIdentity(programSpace);if (track.kind === "visual") assertVisualTrackIdentity(track, programSpace);@@ -78,18 +78,18 @@ function appendTrack(set: FilmTrackSet, programSpace: ProgramSpace, track: Track});}-export function appendFilmVisualTrack(set: FilmTrackSet, programSpace: ProgramSpace, track: VisualTrack): FilmTrackSet {+export function appendFilmVisualTrack(set: FilmTrackSet, programSpace: Timeline, track: VisualTrack): FilmTrackSet {return appendTrack(set, programSpace, track);}-export function appendFilmAudioTrack(set: FilmTrackSet, programSpace: ProgramSpace, track: AudioTrack): FilmTrackSet {+export function appendFilmAudioTrack(set: FilmTrackSet, programSpace: Timeline, track: AudioTrack): FilmTrackSet {return appendTrack(set, programSpace, track);}export function compileFilmComposition(program: FilmProgram,canvas: CanvasSpace,- programSpace: ProgramSpace,+ programSpace: Timeline,set: FilmTrackSet,): Composition {assertFilmProgramIdentity(program);diff --git a/packages/film/src/recipe.ts b/packages/film/src/recipe.tsindex 2677137ed..38db50f7 100644--- a/packages/film/src/recipe.ts+++ b/packages/film/src/recipe.ts@@ -1,7 +1,8 @@+import type { Timeline } from "@hypit/timeline";import type { SvsRecipe } from "@hypit/svs";/**- * The Film Recipe owns Film appearance only. Canvas geometry and ProgramSpace+ * The Film Recipe owns Film appearance only. Canvas geometry and Timeline* remain separate graph inputs and cannot be smuggled into a stylesheet.*/diff --git a/packages/film/src/surface.ts b/packages/film/src/surface.tsindex 45618313..22566d22 100644--- a/packages/film/src/surface.ts+++ b/packages/film/src/surface.ts@@ -1,4 +1,4 @@-import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";+import { resolveTemporalContext } from "@hypit/temporal-markup";import { spatialTypes } from "@hypit/spatial";import { compositionTypes } from "@hypit/composition";import type { AudioTrack, Track, VisualTrack } from "@hypit/composition";@@ -93,13 +93,13 @@ function trackKind(reference: SurfaceResolvedReference): "visual" | "audio" {}export const decodeFilmSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {- exactAttributes(element, ["id", "canvas", element.attributes.space === undefined ? "semantic" : "space", "appearance"]);+ exactAttributes(element, ["id", "canvas", "timeline", "appearance"]);const id = stringAttribute(element, "id");const canvas = requiredReference(element, "canvas", resolveReference);if (!sameType(canvas.type, spatialTypes.canvas)) {throw new Error(`${element.name}.canvas must reference CanvasSpace`);}- const time = createTemporalSpace({ id, element, ...resolveTemporalContext({ element, resolveReference }) });+ const context = resolveTemporalContext({ element, resolveReference });const appearanceReference = requiredReference(element, "appearance", resolveReference);const appearance = filmAppearanceFromRecipe(recipe(appearanceReference, `${element.name}.appearance`).properties,@@ -129,18 +129,18 @@ export const decodeFilmSurface: StructuredSurfaceHandler = ({ element, resolveRevalue: { kind: "inline", value: program },range: element.range,}],- components: [...time.components, {+ components: [{id,fragment: fragment.id,inputs: {program: { kind: "record", id: programId },canvas: canvas.ref,- space: time.space.ref,+ timeline: context.timeline.ref,...Object.fromEntries(tracks.map((track) => [track.name, track.source.ref])),},outputs: { composition: `${id}.composition` },range: element.range,}],- fragments: [...time.fragments, fragment],+ fragments: [fragment],};};diff --git a/packages/film/test/film.test.ts b/packages/film/test/film.test.tsindex e098b06e..0764e3b5 100644--- a/packages/film/test/film.test.ts+++ b/packages/film/test/film.test.ts@@ -1,8 +1,9 @@+import { sealTimeline } from "@hypit/timeline";import { compositionComponent, spatialComponent, videoContractManifests } from "../../../test/support/video-domain.js";import { registerTypeValidatorFacets } from "@hypit/component-kit";import { programSpaceTypes, sealProgramSpace } from "@hypit/program-space";-import { projectSemanticProgramSpace, semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";-import type { SemanticTrack } from "@hypit/semantic-track";+import { projectTimelineSpace, timelineProducers, timelineTypes } from "@hypit/timeline";+import type { Timeline } from "@hypit/timeline";import { compositionTypes, sealAudioTrack, sealVisualTrack } from "@hypit/composition";import type { Composition, Track } from "@hypit/composition";import type { FontArtifactRef } from "@hypit/media";@@ -10,7 +11,7 @@ import { sealCanvasSpace, spatialTypes } from "@hypit/spatial";import assert from "node:assert/strict";import test from "node:test";import { fixtureResource } from "../../../test/fixture-resource.js";-import { semanticTrackFixture } from "../../../test/semantic-track-fixture.js";+import { timelineFixture } from "../../../test/timeline-fixture.js";import {createResolvedClosure,@@ -34,10 +35,10 @@ import { renderTypographyTrack, sealTypographyTrackProgram, stillTextMotion, typimport type { TextStyle } from "@hypit/typography-track";import { admitRecord, TypeValidatorRegistry } from "@hypit/validation";-const space = sealProgramSpace({ id: "test-space", durationSec: 4,+const space = sealTimeline({ items: [], id: "test-space", durationSec: 4,frameRate: { numerator: 30, denominator: 1 },});-const semantic = semanticTrackFixture(space);+const semantic = timelineFixture(space);function validatorRegistry(): TypeValidatorRegistry {const registry = new TypeValidatorRegistry();@@ -133,8 +134,9 @@ const closure = createResolvedClosure([typographyTrackManifest,]);const records = await Promise.all([- sealRecord({ id: "space", type: programSpaceTypes.programSpace, value: stored(space) }),- sealRecord({ id: "semantic", type: semanticTrackTypes.track, value: stored(semantic) }),+ sealRecord({ id: "render-space", type: programSpaceTypes.programSpace, value: stored(projectTimelineSpace(space)) }),+ sealRecord({ id: "timeline", type: timelineTypes.track, value: stored(space) }),+ sealRecord({ id: "semantic", type: timelineTypes.track, value: stored(semantic) }),sealRecord({ id: "canvas", type: spatialTypes.canvas, value: stored(canvas) }),sealRecord({ id: "film-program", type: filmTypes.program, value: stored(filmProgram) }),sealRecord({ id: "text-program", type: typographyTrackTypes.program, value: stored(textProgram) }),@@ -147,7 +149,7 @@ const textInstance = elaborateGraphFragment(linked, typographyTrackFragment, {id: "title",fragment: typographyTrackFragment.id,inputs: {- space: { kind: "record", id: "space" },+ timeline: { kind: "record", id: "timeline" },program: { kind: "record", id: "text-program" },},});@@ -167,7 +169,7 @@ const filmInstance = elaborateGraphFragment(linked, filmFragment, {inputs: {program: { kind: "record", id: "film-program" },canvas: { kind: "record", id: "canvas" },- space: { kind: "record", id: "space" },+ timeline: { kind: "record", id: "timeline" },title: { kind: "logical-output", id: "title.track" },background: { kind: "record", id: "background" },audio: { kind: "record", id: "audio" },@@ -181,7 +183,7 @@ const hyperframesInstance = elaborateGraphFragment(linked, hyperframesDocumentFrfragment: hyperframesDocumentFragment.id,inputs: {composition: { kind: "logical-output", id: "main.composition" },- space: { kind: "record", id: "space" },+ space: { kind: "record", id: "render-space" },},});const hyperframesContribution = bindAuthorFragment(hyperframesInstance, {@@ -227,13 +229,13 @@ test("Film stops at Composition and Hyperframes remains an ordinary downstream Ftest("the Driver folds peer Tracks, then independently compiles the Composition", async () => {const registry = new ProducerRegistry();- registry.registerProducer(semanticTrackProducers.projectProgramSpace, ({ inputs }) => ({- outputs: { space: stored(projectSemanticProgramSpace(inline(inputs.track) as SemanticTrack)) },+ registry.registerProducer(timelineProducers.projectProgramSpace, ({ inputs }) => ({+ outputs: { space: stored(projectTimelineSpace(inline(inputs.track) as Timeline)) },needs: {},}));registry.registerProducer(typographyTrackProducers.render, ({ inputs }) => ({outputs: { track: stored(renderTypographyTrack(- inline(inputs.space) as typeof space,+ inline(inputs.timeline) as typeof space,inline(inputs.program) as typeof textProgram,)) },needs: {},@@ -245,7 +247,7 @@ test("the Driver folds peer Tracks, then independently compiles the Composition"registry.registerProducer(filmProducers.appendVisualTrack, ({ inputs }) => ({outputs: { set: stored(appendFilmVisualTrack(inline(inputs.set) as never,- inline(inputs.space) as typeof space,+ inline(inputs.timeline) as typeof space,inline(inputs.track) as never,)) },needs: {},@@ -253,7 +255,7 @@ test("the Driver folds peer Tracks, then independently compiles the Composition"registry.registerProducer(filmProducers.appendAudioTrack, ({ inputs }) => ({outputs: { set: stored(appendFilmAudioTrack(inline(inputs.set) as never,- inline(inputs.space) as typeof space,+ inline(inputs.timeline) as typeof space,inline(inputs.track) as never,)) },needs: {},@@ -262,7 +264,7 @@ test("the Driver folds peer Tracks, then independently compiles the Composition"outputs: { composition: stored(compileFilmComposition(inline(inputs.program) as never,inline(inputs.canvas) as typeof canvas,- inline(inputs.space) as typeof space,+ inline(inputs.timeline) as typeof space,inline(inputs.set) as never,)) },needs: {},diff --git a/packages/film/test/surface.test.ts b/packages/film/test/surface.test.tsindex 75aca533..bfe13c5a 100644--- a/packages/film/test/surface.test.ts+++ b/packages/film/test/surface.test.ts@@ -1,13 +1,14 @@+import { sealTimeline } from "@hypit/timeline";import { compositionComponent, spatialComponent, videoContractManifests } from "../../../test/support/video-domain.js";import { registerTypeValidatorFacets } from "@hypit/component-kit";import { programSpaceDependency, programSpaceTypes, sealProgramSpace } from "@hypit/program-space";-import { semanticTrackDependency, semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";+import { timelineDependency, timelineProducers, timelineTypes } from "@hypit/timeline";import { compositionDependency, compositionTypes, sealAudioTrack, sealVisualTrack } from "@hypit/composition";import type { Track } from "@hypit/composition";import assert from "node:assert/strict";import test from "node:test";import { fixtureResource } from "../../../test/fixture-resource.js";-import { semanticTrackFixture } from "../../../test/semantic-track-fixture.js";+import { timelineFixture } from "../../../test/timeline-fixture.js";import { createResolvedClosure, sealBuildRequest, start } from "@hypit/core";import {@@ -42,14 +43,14 @@ const fixtureModule = { name: "example.film-fixture", version: "1" } as const;const fixtureSurfaceDigest = fixtureResource("example.film-fixture/inputs-surface@1");const fixtureSurface = {name: "inputs", tag: "Inputs", mode: "structured",- outputs: [semanticTrackTypes.track, compositionTypes.visualTrack, compositionTypes.audioTrack],+ outputs: [timelineTypes.track, compositionTypes.visualTrack, compositionTypes.audioTrack],} as const;const fixtureManifest: ModuleManifest = {format: "hypit.module@1",name: fixtureModule.name,version: fixtureModule.version,dependencies: [- semanticTrackDependency,+ timelineDependency,compositionDependency,],types: [],@@ -57,10 +58,10 @@ const fixtureManifest: ModuleManifest = {producers: [],};-const space = sealProgramSpace({ id: "test-space", durationSec: 2,+const space = sealTimeline({ items: [], id: "test-space", durationSec: 2,frameRate: { numerator: 30, denominator: 1 },});-const semantic = semanticTrackFixture(space);+const semantic = timelineFixture(space);const visual = sealVisualTrack({ programSpaceId: "test-space",visualIr: "hypit.visual-ir@1",id: "visual",@@ -103,7 +104,7 @@ async function compileFilm(options: { readonly styles?: string } = {}) {const surfaces = new MarkupSurfaceRegistry();surfaces.registerStructured({ module: fixtureModule, declaration: fixtureSurface, handler: ({ element }) => ({records: [- { id: "semantic", type: semanticTrackTypes.track, value: { kind: "inline", value: semantic }, range: element.range },+ { id: "semantic", type: timelineTypes.track, value: { kind: "inline", value: semantic }, range: element.range },{ id: "visual", type: compositionTypes.visualTrack, value: { kind: "inline", value: visual }, range: element.range },{ id: "audio", type: compositionTypes.audioTrack, value: { kind: "inline", value: audio }, range: element.range },],@@ -139,7 +140,7 @@ async function compileFilm(options: { readonly styles?: string } = {}) {<import as="recipes" source="./recipes.svs"/><fixture:Inputs/><space:Canvas id="vertical" width="1080" height="1920"/>- <film:Film id="main" canvas={vertical} semantic={semantic} appearance={recipes.film.vertical}>+ <film:Film id="main" canvas={vertical} timeline={semantic} appearance={recipes.film.vertical}><film:Track source={visual}/><film:Track source={audio}/></film:Film></svml>`),@@ -170,8 +171,7 @@ test("the official Film Surface validates SVS and lowers dynamic peer Tracks", afilmProducers.appendAudioTrack.name,filmProducers.appendVisualTrack.name,filmProducers.compileComposition.name,- semanticTrackProducers.projectProgramSpace.name,- ].sort());+ ].sort());});test("Film rejects an invalid package-owned Recipe during check", async () => {diff --git a/packages/hyperframes/README.md b/packages/hyperframes/README.mdindex 2f5c8ff1..a027367e 100644--- a/packages/hyperframes/README.md+++ b/packages/hyperframes/README.md@@ -61,7 +61,7 @@ material preparation before rendering; browser resources belong in the program'sUse `hyperframesResourceUri(artifact.resource)` in resource-bearing markup or CSS.Ordinary child sampling follows the Present clock. Reframing the parent leaves source playback-unchanged. `projectSemanticMedia` from `@hypit/hypit/semantic-track` gives a component selected prepared+unchanged. `projectTimelineMedia` from `@hypit/hypit/timeline` gives a component selected preparedclips with their exact program and source spans. Speech audio is an independently selected Track.`program.format` is explicit: this backend reports an unsupported format rather than interpretingdiff --git a/packages/hyperframes/src/document.ts b/packages/hyperframes/src/document.tsindex a11b786c..9ece71dc 100644--- a/packages/hyperframes/src/document.ts+++ b/packages/hyperframes/src/document.ts@@ -461,7 +461,7 @@ function renderVisualPresent(emittedFilterIds,stableId,});- return `<div class="clip hypit-visual-present" data-hypit-track-id="${escapeHtml(track.id)}" data-hypit-present-id="${escapeHtml(present.id)}" data-hypit-stack-order="${present.stacking.order}" data-hypit-stack-tie="${escapeHtml(present.stacking.tieBreak)}" data-track-index="${stackIndex}" data-start="${start}" data-duration="${duration}" style="position:absolute;inset:0;z-index:${stackIndex};overflow:hidden;pointer-events:none">${contents}</div>`;+ return `<div class="clip hypit-visual-present" data-hypit-track-id="${escapeHtml(track.id)}" data-hypit-present-id="${escapeHtml(present.id)}" data-hypit-stack-order="${present.stacking.order}" data-hypit-stack-tie="${escapeHtml(present.stacking.tieBreak)}" data-track-index="${stackIndex}" data-start="${start}" data-duration="${duration}" style="position:absolute;inset:0;z-index:${stackIndex};overflow:hidden;pointer-events:none">${present.visibility === undefined ? contents : `<div data-hypit-visibility="${escapeHtml(JSON.stringify(present.visibility))}" style="position:absolute;inset:0">${contents}</div>`}</div>`;}function renderAnimationRules(track: VisualTrack, present: VisualPresent, stableId: StableDomId): string[] {@@ -667,6 +667,18 @@ function emitHtml(composition: Composition, programSpace: ProgramSpace): string}]));const programCss = programs.map(entry => `@scope (#${entry.id}) { ${entry.program.css ?? ""} }`).join("\n");const programRuntime = programs.length === 0 ? "" : `<script>${browserProgramScript(programs, numerator, denominator)}</script>`;+ const visibilityRuntime = visuals.some(({ present }) => present.visibility !== undefined) ? `<script>+(() => {+ const entries = [...document.querySelectorAll('[data-hypit-visibility]')].map(element => ({ element, spans: JSON.parse(element.dataset.hypitVisibility) }));+ const apply = time => {+ const frame = Math.round(Number(time || 0) * ${numerator} / ${denominator});+ // A presentation mask controls painting, not layout or animation existence.+ // display:none removes CSS animations and prevents hidden text from being measured.+ for (const { element, spans } of entries) element.style.opacity = spans.some(s => frame >= s.startFrame && frame < s.endFrameExclusive) ? '' : '0';+ };+ apply(0);+ window.addEventListener('hf-seek', event => apply(event.detail?.time));+})();</script>` : "";const duration = frameSeconds(programSpaceFrameCount(programSpace), numerator, denominator);const fps = fpsRational(numerator, denominator);const frameCount = programSpaceFrameCount(programSpace);@@ -693,7 +705,8 @@ function emitHtml(composition: Composition, programSpace: ProgramSpace): string<body><div data-composition-id="${escapeHtml(composition.id)}" data-start="0" data-no-timeline data-width="${composition.canvas.width}" data-height="${composition.canvas.height}" data-duration="${duration}" data-fps="${fps}" data-hypit-frame-count="${frameCount}">${visualHtml}- </div>${programRuntime}${animationRuntime}${textRuntime}+ </div>${programRuntime}+ ${visibilityRuntime}${animationRuntime}${textRuntime}</body></html>`;diff --git a/packages/interview-emoji-reveal/README.md b/packages/interview-emoji-reveal/README.mdindex 49eb1660..f800b148 100644--- a/packages/interview-emoji-reveal/README.md+++ b/packages/interview-emoji-reveal/README.md@@ -1,7 +1,7 @@# Emoji Reveal-The Track Surface accepts `semantic={speech.semantic}` or `space={animation}`. Semantic context-resolves Script Selections and Moments; authored space supports clock-based animation. Shared `at`+The Track Surface accepts `timeline={program.timeline}`. The same Timeline supports authored+positions and, where prepared Takes supply evidence, Script Selections and Moments. Shared `at`inputs accept a Moment or a time such as `2s`; `at` with `for` produces a Window where required.Reusable Hypit author vocabulary for a top-of-frame icon answer strip.@@ -21,7 +21,7 @@ answers remain and later slots remain unanswered.<emoji:Style id="emoji-strip" recipe={styles.emoji-strip}/>-<emoji:Track id="rules" semantic={speech.semantic} canvas={vertical}+<emoji:Track id="rules" timeline={speech.timeline} canvas={vertical}style={emoji-strip} placeholder={question-icon} during="program"><emoji:Item id="manifest" icon={manifest-icon} preset="true"/><emoji:Item id="real-estate" icon={real-estate-icon} at={story.moment.real-estate}/>@@ -30,7 +30,7 @@ answers remain and later slots remain unanswered.````Selection`, `Segment`, numeric instants and a `boundary` fallback are deliberately not part of an-Item's vocabulary. The component consumes only `ProgramSpace`, the projected outer Window and the+Item's vocabulary. The component consumes the shared `Timeline`, the projected outer Window and theprojected Instants; it does not interpret Script semantics itself and has no Studio dependency.The bundled preview icons come from [Tabler Icons](https://tabler.io/icons), distributed under thediff --git a/packages/interview-emoji-reveal/package.json b/packages/interview-emoji-reveal/package.jsonindex 5e775b70..2b395123 100644--- a/packages/interview-emoji-reveal/package.json+++ b/packages/interview-emoji-reveal/package.json@@ -14,7 +14,7 @@"@hypit/narrative": "workspace:*","@hypit/program-space": "workspace:*","@hypit/protocol": "workspace:*",- "@hypit/semantic-track": "workspace:*",+ "@hypit/timeline": "workspace:*","@hypit/spatial": "workspace:*","@hypit/svs": "workspace:*","@hypit/temporal": "workspace:*",diff --git a/packages/interview-emoji-reveal/src/component.ts b/packages/interview-emoji-reveal/src/component.tsindex 14a71a97..7f154d00 100644--- a/packages/interview-emoji-reveal/src/component.ts+++ b/packages/interview-emoji-reveal/src/component.ts@@ -1,7 +1,7 @@+import type { Timeline } from "@hypit/timeline";import type { ComponentPackage } from "@hypit/component-kit";import { canonicalize } from "@hypit/protocol";import type { BlobRef, StoredValue } from "@hypit/protocol";-import type { ProgramSpace } from "@hypit/program-space";import type { CanvasSpace } from "@hypit/spatial";import type { TemporalInstant, TemporalWindow } from "@hypit/temporal";@@ -23,7 +23,7 @@ export const emojiRevealComponent = {producers: [{ producer: emojiRevealProducers.createSet, handler: () => ({ outputs: { set: output(createEmojiRevealSet()) }, needs: {} }) },{ producer: emojiRevealProducers.appendItem, handler: ({ inputs }) => ({ outputs: { set: output(appendEmojiRevealItem(- inline<EmojiRevealSet>(inputs.set?.value, "EmojiRevealSet"), inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ inline<EmojiRevealSet>(inputs.set?.value, "EmojiRevealSet"), inline<Timeline>(inputs.timeline?.value, "Timeline"),inline<EmojiRevealItemSpec>(inputs.spec?.value, "EmojiRevealItemSpec"), blob(inputs.icon?.value, "Emoji Reveal icon"),inline<TemporalInstant>(inputs.activation?.value, "TemporalInstant"),)) }, needs: {} }) },@@ -32,12 +32,12 @@ export const emojiRevealComponent = {inline<EmojiRevealItemSpec>(inputs.spec?.value, "EmojiRevealItemSpec"), blob(inputs.icon?.value, "Emoji Reveal icon"),)) }, needs: {} }) },{ producer: emojiRevealProducers.finalize, handler: ({ inputs }) => ({ outputs: { program: output(finalizeEmojiReveal(- inline<EmojiRevealHeader>(inputs.header?.value, "EmojiRevealHeader"), inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ inline<EmojiRevealHeader>(inputs.header?.value, "EmojiRevealHeader"), inline<Timeline>(inputs.timeline?.value, "Timeline"),inline<TemporalWindow>(inputs.outer?.value, "TemporalWindow"), inline<EmojiRevealStyle>(inputs.style?.value, "EmojiRevealStyle"),blob(inputs.placeholder?.value, "Emoji Reveal placeholder"), inline<EmojiRevealSet>(inputs.set?.value, "EmojiRevealSet"),)) }, needs: {} }) },{ producer: emojiRevealProducers.render, handler: ({ inputs }) => ({ outputs: { track: output(renderEmojiReveal(- inline<CanvasSpace>(inputs.canvas?.value, "CanvasSpace"), inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ inline<CanvasSpace>(inputs.canvas?.value, "CanvasSpace"), inline<Timeline>(inputs.timeline?.value, "Timeline"),inline<EmojiRevealProgram>(inputs.program?.value, "EmojiRevealProgram"),)) }, needs: {} }) },],diff --git a/packages/interview-emoji-reveal/src/fragment.ts b/packages/interview-emoji-reveal/src/fragment.tsindex 39650658..8849e8b4 100644--- a/packages/interview-emoji-reveal/src/fragment.ts+++ b/packages/interview-emoji-reveal/src/fragment.ts@@ -1,4 +1,5 @@-import { programSpaceTypes } from "@hypit/program-space";+import { timelineTypes } from "@hypit/timeline";+import { compositionTypes } from "@hypit/composition";import { artifactTypes } from "@hypit/artifact";import { sealGraphFragment } from "@hypit/elaborator";@@ -29,23 +30,23 @@ export function createEmojiRevealFragment(items: readonly EmojiRevealFragmentIte: emojiRevealProducers.appendItem, inputs: {set: operation(current), spec: input(item.specName), icon: input(item.iconName),...(item.activationName === undefined ? {} : {- space: input("space"), activation: input(item.activationName),+ timeline: input("timeline"), activation: input(item.activationName),}),}, result: { kind: "output", name: "set" } });current = id;});operations.push({ id: "emoji:program", producer: emojiRevealProducers.finalize, inputs: {- header: input("header"), space: input("space"), outer: input("outer"), style: input("style"),+ header: input("header"), timeline: input("timeline"), outer: input("outer"), style: input("style"),placeholder: input("placeholder"), set: operation(current),}, result: { kind: "output", name: "program" } },{ id: "emoji:track", producer: emojiRevealProducers.render, inputs: {- canvas: input("canvas"), space: input("space"), program: operation("emoji:program"),+ canvas: input("canvas"), timeline: input("timeline"), program: operation("emoji:program"),}, result: { kind: "output", name: "track" } },);return sealGraphFragment({inputs: [- { name: "header", type: emojiRevealTypes.header }, { name: "space", type: programSpaceTypes.programSpace },+ { name: "header", type: emojiRevealTypes.header }, { name: "timeline", type: timelineTypes.track },{ name: "canvas", type: spatialTypes.canvas }, { name: "outer", type: temporalTypes.window }, { name: "style", type: emojiRevealTypes.style },{ name: "placeholder", type: artifactTypes.blob },...items.flatMap((item) => [diff --git a/packages/interview-emoji-reveal/src/manifest.ts b/packages/interview-emoji-reveal/src/manifest.tsindex 6902de71..4f163bda 100644--- a/packages/interview-emoji-reveal/src/manifest.ts+++ b/packages/interview-emoji-reveal/src/manifest.ts@@ -1,12 +1,13 @@+import { timelineTypes } from "@hypit/timeline";import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";import { readFile } from "node:fs/promises";import { artifactDependency, artifactTypes } from "@hypit/artifact";import { compositionDependency, compositionTypes } from "@hypit/composition";import { narrativeDependency, narrativeTypes } from "@hypit/narrative";-import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";+import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";-import { semanticTrackDependency } from "@hypit/semantic-track";+import { timelineDependency } from "@hypit/timeline";import { spatialDependency, spatialTypes } from "@hypit/spatial";import { svsRecipeType } from "@hypit/svs";import { temporalDependency, temporalInstantSchema, temporalTypes, temporalWindowSchema } from "@hypit/temporal";@@ -107,7 +108,7 @@ export const emojiRevealMarkupSurfaces = [recipe: recipe.map(([name, fallback, summary]) => ({ name, required: false, fallback, summary })) },],example: `<emoji:Style id="emoji-strip" recipe={styles.emoji-strip}/>` ,- notes: ["The Style owns appearance and placement, but never semantic timing."],+ notes: ["The Style owns appearance and placement, but never event timing."],} },{ name: "track", tag: "Track", mode: "structured",outputs: [emojiRevealTypes.header, emojiRevealTypes.itemSpec, temporalTypes.instantSpec, temporalTypes.windowSpec, temporalTypes.instant, temporalTypes.window, emojiRevealTypes.program, compositionTypes.visualTrack],@@ -133,7 +134,7 @@ export const emojiRevealMarkupSurfaces = [{ name: "program", type: emojiRevealTypes.program, summary: "The adaptive strip with its projected outer Window and fully traced Moment activations." },{ name: "track", type: compositionTypes.visualTrack, summary: "That Program rendered as an ordinary VisualTrack." },],- example: `<emoji:Track id="rules" semantic={speech.semantic} canvas={vertical} style={emoji-strip} placeholder={question-icon} during="program">+ example: `<emoji:Track id="rules" timeline={speech.timeline} canvas={vertical} style={emoji-strip} placeholder={question-icon} during="program"><emoji:Item id="manifest" icon={manifest-icon} preset="true"/><emoji:Item id="real-estate" icon={real-estate-icon} at={story.moment.real-estate}/><emoji:Item id="bitcoin" icon={bitcoin-icon} at={story.moment.bitcoin}/>@@ -148,7 +149,7 @@ export const emojiRevealMarkupSurfaces = [export const emojiRevealManifest: ModuleManifest = {format: "hypit.module@1", name: emojiRevealModuleRef.name, version: emojiRevealModuleRef.version,- dependencies: [artifactDependency, narrativeDependency, programSpaceDependency, semanticTrackDependency, spatialDependency, temporalDependency, compositionDependency],+ dependencies: [artifactDependency, narrativeDependency, timelineDependency, spatialDependency, temporalDependency, compositionDependency],types: [{ name: emojiRevealTypes.header.name }, { name: emojiRevealTypes.style.name },{ name: emojiRevealTypes.itemSpec.name }, { name: emojiRevealTypes.set.name }, { name: emojiRevealTypes.program.name },@@ -157,16 +158,16 @@ export const emojiRevealManifest: ModuleManifest = {producers: [{ name: emojiRevealProducers.createSet.name, inputs: [], outputs: [{ name: "set", type: emojiRevealTypes.set }], needs: [] },{ name: emojiRevealProducers.appendItem.name,- inputs: [{ name: "set", type: emojiRevealTypes.set }, { name: "space", type: programSpaceTypes.programSpace }, { name: "spec", type: emojiRevealTypes.itemSpec }, { name: "icon", type: artifactTypes.blob }, { name: "activation", type: temporalTypes.instant }],+ inputs: [{ name: "set", type: emojiRevealTypes.set }, { name: "timeline", type: timelineTypes.track }, { name: "spec", type: emojiRevealTypes.itemSpec }, { name: "icon", type: artifactTypes.blob }, { name: "activation", type: temporalTypes.instant }],outputs: [{ name: "set", type: emojiRevealTypes.set }], needs: [] },{ name: emojiRevealProducers.appendPresetItem.name,inputs: [{ name: "set", type: emojiRevealTypes.set }, { name: "spec", type: emojiRevealTypes.itemSpec }, { name: "icon", type: artifactTypes.blob }],outputs: [{ name: "set", type: emojiRevealTypes.set }], needs: [] },{ name: emojiRevealProducers.finalize.name,- inputs: [{ name: "header", type: emojiRevealTypes.header }, { name: "space", type: programSpaceTypes.programSpace }, { name: "outer", type: temporalTypes.window }, { name: "style", type: emojiRevealTypes.style }, { name: "placeholder", type: artifactTypes.blob }, { name: "set", type: emojiRevealTypes.set }],+ inputs: [{ name: "header", type: emojiRevealTypes.header }, { name: "timeline", type: timelineTypes.track }, { name: "outer", type: temporalTypes.window }, { name: "style", type: emojiRevealTypes.style }, { name: "placeholder", type: artifactTypes.blob }, { name: "set", type: emojiRevealTypes.set }],outputs: [{ name: "program", type: emojiRevealTypes.program }], needs: [] },{ name: emojiRevealProducers.render.name,- inputs: [{ name: "canvas", type: spatialTypes.canvas }, { name: "space", type: programSpaceTypes.programSpace }, { name: "program", type: emojiRevealTypes.program }],+ inputs: [{ name: "canvas", type: spatialTypes.canvas }, { name: "timeline", type: timelineTypes.track }, { name: "program", type: emojiRevealTypes.program }],outputs: [{ name: "track", type: compositionTypes.visualTrack }], needs: [] },],};diff --git a/packages/interview-emoji-reveal/src/program.ts b/packages/interview-emoji-reveal/src/program.tsindex d7805c70..77b7d901 100644--- a/packages/interview-emoji-reveal/src/program.ts+++ b/packages/interview-emoji-reveal/src/program.ts@@ -1,7 +1,7 @@+import type { Timeline } from "@hypit/timeline";import { assertVisualTrackIdentity, sealVisualTrack } from "@hypit/composition";import type { VisualAnimation, VisualElement, VisualTrack } from "@hypit/composition";import { assertProgramSpaceIdentity } from "@hypit/program-space";-import type { ProgramSpace } from "@hypit/program-space";import { canonicalize } from "@hypit/protocol";import type { BlobRef } from "@hypit/protocol";import { assertCanvasSpace } from "@hypit/spatial";@@ -67,11 +67,11 @@ export function assertEmojiRevealSet(value: EmojiRevealSet): void {}}-export function appendEmojiRevealItem(set: EmojiRevealSet, space: ProgramSpace, spec: EmojiRevealItemSpec, icon: BlobRef, activation: TemporalInstant): EmojiRevealSet {- assertEmojiRevealSet(set); assertProgramSpaceIdentity(space); assertEmojiRevealItemSpec(spec);+export function appendEmojiRevealItem(set: EmojiRevealSet, timeline: Timeline, spec: EmojiRevealItemSpec, icon: BlobRef, activation: TemporalInstant): EmojiRevealSet {+ assertEmojiRevealSet(set); assertProgramSpaceIdentity(timeline); assertEmojiRevealItemSpec(spec);assert(!spec.preset, `Emoji Reveal Item ${spec.id} with preset=true cannot have an activation.`);assertIconImage(icon, `Emoji Reveal Item ${spec.id} icon`);- assertTemporalInstantFor(activation, { subjectId: spec.id, space });+ assertTemporalInstantFor(activation, { subjectId: spec.id, space: timeline });assert(!set.items.some((item) => item.spec.id === spec.id), `Duplicate Emoji Reveal Item ${spec.id}.`);return { items: [...set.items, { spec: structuredClone(spec), icon: structuredClone(icon), activation: structuredClone(activation) }] };}@@ -85,11 +85,11 @@ export function appendPresetEmojiRevealItem(set: EmojiRevealSet, spec: EmojiReve}export function finalizeEmojiReveal(- header: EmojiRevealHeader, space: ProgramSpace, outer: TemporalWindow, style: EmojiRevealStyle, placeholder: BlobRef, set: EmojiRevealSet,+ header: EmojiRevealHeader, timeline: Timeline, outer: TemporalWindow, style: EmojiRevealStyle, placeholder: BlobRef, set: EmojiRevealSet,): EmojiRevealProgram {- assertEmojiRevealHeader(header); assertProgramSpaceIdentity(space); assertEmojiRevealStyle(style); assertEmojiRevealSet(set);+ assertEmojiRevealHeader(header); assertProgramSpaceIdentity(timeline); assertEmojiRevealStyle(style); assertEmojiRevealSet(set);assertIconImage(placeholder, "Emoji Reveal placeholder");- assertTemporalWindowFor(outer, { subjectId: header.id, space });+ assertTemporalWindowFor(outer, { subjectId: header.id, space: timeline });assert(set.items.length > 0, "Emoji Reveal requires at least one Item.");let previous = outer.span.startFrame - 1;let revealStarted = false;@@ -106,7 +106,7 @@ export function finalizeEmojiReveal(previous = frame;}const program: EmojiRevealProgram = {- id: header.id, programSpaceId: space.id, outer: structuredClone(outer), style: structuredClone(style),+ id: header.id, programSpaceId: timeline.id, outer: structuredClone(outer), style: structuredClone(style),placeholder: structuredClone(placeholder), items: structuredClone(set.items),};assertEmojiRevealProgram(program);@@ -183,10 +183,10 @@ function iconElement(input: {};}-export function renderEmojiReveal(canvas: CanvasSpace, space: ProgramSpace, program: EmojiRevealProgram): VisualTrack {- assertCanvasSpace(canvas); assertProgramSpaceIdentity(space); assertEmojiRevealProgram(program);- assert(program.programSpaceId === space.id, "EmojiRevealProgram belongs to another ProgramSpace.");- assertTemporalWindowFor(program.outer, { subjectId: program.id, space });+export function renderEmojiReveal(canvas: CanvasSpace, timeline: Timeline, program: EmojiRevealProgram): VisualTrack {+ assertCanvasSpace(canvas); assertProgramSpaceIdentity(timeline); assertEmojiRevealProgram(program);+ assert(program.programSpaceId === timeline.id, "EmojiRevealProgram belongs to another Timeline.");+ assertTemporalWindowFor(program.outer, { subjectId: program.id, space: timeline });const { style } = program;const count = program.items.length;const width = style.paddingXPx * 2 + style.slotSizePx * count + style.gapPx * (count - 1);@@ -234,9 +234,9 @@ export function renderEmojiReveal(canvas: CanvasSpace, space: ProgramSpace, prog);});const track = sealVisualTrack({- programSpaceId: space.id, visualIr: "hypit.visual-ir@1", id: program.id,+ programSpaceId: timeline.id, visualIr: "hypit.visual-ir@1", id: program.id,presents: [{ id: program.id, subjectId: program.id, span: { ...program.outer.span },stacking: { order: style.stackingOrder, tieBreak: program.id }, elements }],});- assertVisualTrackIdentity(track, space); return track;+ assertVisualTrackIdentity(track, timeline); return track;}diff --git a/packages/interview-emoji-reveal/src/surface.ts b/packages/interview-emoji-reveal/src/surface.tsindex 756ca2fe..6fc9a254 100644--- a/packages/interview-emoji-reveal/src/surface.ts+++ b/packages/interview-emoji-reveal/src/surface.ts@@ -1,4 +1,4 @@-import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";+import { resolveTemporalContext } from "@hypit/temporal-markup";import {assertAttributes, assertEmptyElement, optionalTextAttribute, textAttribute, type MarkupAttributeValue, type StructuredSurfaceHandler,type SurfaceComponentDraft, type SurfaceRecordDraft, type SurfaceResolvedReference,@@ -48,24 +48,23 @@ export const decodeEmojiRevealStyleSurface: StructuredSurfaceHandler = ({ elemen};export const decodeEmojiRevealTrackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {- assertAttributes(element, ["id", "semantic", "space", "canvas", "style", "placeholder", ...temporalWindowAttributeNames]);+ assertAttributes(element, ["id", "timeline", "canvas", "style", "placeholder", ...temporalWindowAttributeNames]);const id = textAttribute(element, "id");const context = resolveTemporalContext({ element, resolveReference });- const time = createTemporalSpace({ id: id, element, ...context });const canvas = reference(element.attributes.canvas, `${element.name}.canvas`, spatialTypes.canvas, resolveReference);const style = reference(element.attributes.style, `${element.name}.style`, emojiRevealTypes.style, resolveReference);const placeholder = reference(element.attributes.placeholder, `${element.name}.placeholder`, artifactTypes.blob, resolveReference);- const outer = createTemporalWindowProjection({ id, subjectId: id, element, ...context, space: time.space, resolveReference });+ const outer = createTemporalWindowProjection({ id, subjectId: id, element, ...context, resolveReference });const headerId = `${id}.header`;const records: SurfaceRecordDraft[] = [...outer.records, {id: headerId, type: emojiRevealTypes.header,value: { kind: "inline", value: sealEmojiRevealHeader({ id }) as unknown as CanonicalValue }, range: element.range,}];- const temporalComponents: SurfaceComponentDraft[] = [...time.components, ...outer.components];- const temporalFragments = [...time.fragments, ...outer.fragments];+ const temporalComponents: SurfaceComponentDraft[] = [...outer.components];+ const temporalFragments = [...outer.fragments];const items: { specName: string; iconName: string; activationName?: string }[] = [];- const inputs: Record<string, typeof time.space.ref> = {- header: { kind: "record", id: headerId }, space: time.space.ref, canvas: canvas.ref, style: style.ref,+ const inputs: Record<string, typeof context.timeline.ref> = {+ header: { kind: "record", id: headerId }, timeline: context.timeline.ref, canvas: canvas.ref, style: style.ref,placeholder: placeholder.ref, outer: outer.ref,};const ids = new Set<string>();@@ -92,7 +91,7 @@ export const decodeEmojiRevealTrackSurface: StructuredSurfaceHandler = ({ elemenelse {const activation = createTemporalInstantProjection({id: `${id}.item.${suffix}.activation`, subjectId: itemId,- element: child, ...context, space: time.space, resolveReference, semanticAttribute: "at", projectedAttribute: false,+ element: child, ...context, resolveReference, semanticAttribute: "at", projectedAttribute: false,});records.push(...activation.records); temporalComponents.push(...activation.components); temporalFragments.push(...activation.fragments);const activationName = `item-${suffix}-activation`;diff --git a/packages/media-execution/README.md b/packages/media-execution/README.mdindex ca5a3799..bf9682b9 100644--- a/packages/media-execution/README.md+++ b/packages/media-execution/README.md@@ -26,3 +26,8 @@ instead of reimplementing media semantics per deployment.This is not an author package, Provider, queue or Core extension. It performs no endpoint selection,credential lookup or SVML parsing.++Audio gain envelopes and audible subranges are evaluated per sample after tempo/looping and Clip+fades, before the requested range is cropped. Original source and envelope progress therefore survive+local silence and range rendering. This is generic AudioProgramPlan execution, independent of author+component or Style names.diff --git a/packages/media-execution/src/audio-presentation.ts b/packages/media-execution/src/audio-presentation.tsnew file mode 100644index 00000000..4d9a9d05--- /dev/null+++ b/packages/media-execution/src/audio-presentation.ts@@ -0,0 +1,23 @@+import type { AudioProgramClip } from "@hypit/media-pipeline";++/** Per-sample gain after source playback/fades, before any requested render-range crop. */+export function audioPresentationFilter(clip: AudioProgramClip): string[] {+ if (clip.gainEnvelope === undefined && clip.audibility === undefined) return [];+ const sample = `(n+${clip.targetStartSample})`;+ let gain = "1";+ const points = clip.gainEnvelope;+ if (points !== undefined) {+ gain = String(points.at(-1)!.gain);+ for (let index = points.length - 1; index > 0; index--) {+ const left = points[index - 1]!, right = points[index]!;+ const line = `(${left.gain}+(${right.gain}-${left.gain})*(${sample}-${left.sample})/${right.sample-left.sample})`;+ gain = `if(lt(${sample},${right.sample}),${line},${gain})`;+ }+ gain = `if(lt(${sample},${points[0]!.sample}),${points[0]!.gain},${gain})`;+ }+ if (clip.audibility !== undefined) {+ const active = clip.audibility.map(span => `(gte(${sample},${span.startSample})*lt(${sample},${span.endSampleExclusive}))`).join("+") || "0";+ gain = `(${gain})*(${active})`;+ }+ return [`aeval=exprs='val(0)*(${gain})|val(1)*(${gain})'`];+}diff --git a/packages/media-execution/src/execute.ts b/packages/media-execution/src/execute.tsindex 613a3a14..f5e5ccd8 100644--- a/packages/media-execution/src/execute.ts+++ b/packages/media-execution/src/execute.ts@@ -1,3 +1,4 @@+import { audioPresentationFilter } from "./audio-presentation.js";import { spawn } from "node:child_process";import { randomUUID } from "node:crypto";import { createReadStream } from "node:fs";@@ -556,6 +557,7 @@ function audioClipFilter(clip: AudioProgramClip, inputIndex: number, outputIndex...(clip.fadeOutSamples === 0 ? [] : [`afade=t=out:start_sample=${length - clip.fadeOutSamples}:nb_samples=${clip.fadeOutSamples}`,]),+ ...audioPresentationFilter(clip),// Crop after tempo, looping and envelopes, keeping their original phase.`atrim=start_sample=${left - clip.targetStartSample}:end_sample=${right - clip.targetStartSample}`,"asetpts=N/SR/TB",diff --git a/packages/media-pipeline/README.md b/packages/media-pipeline/README.mdindex 4c3856c4..da47f642 100644--- a/packages/media-pipeline/README.md+++ b/packages/media-pipeline/README.md@@ -37,7 +37,7 @@ This package is the shared execution vocabulary. Speech authoring makes the bounalignment produce one `SemanticTake`. Media Track consumes that prepared value; it never owns animplicit visual-only normalization policy.-The frame clock is a separate program fact and can be shared by every Normalize and Speech Track:+The frame clock is a separate program fact and can be shared by every Normalize and Timeline assembly:```svml<program:Clock id="clock" frame-rate="30"/>@@ -111,3 +111,7 @@ resampling, mixing, encoding or muxing is never a Producer shortcut: it is an exNeed. The plan fixes exact 48 kHz sample boundaries, gain/fade/playback parameters and the absenceof hidden normalization/limiting. The local FFmpeg and AWS Lambda Providers consume the same sharedexecution body and return the same public contracts.++AudioProgramPlan preserves AudioClip `gainEnvelope` and `audibility` on the full program sample+clock. The [Composition definition](../composition/README.md#audio-presentation-on-the-program-clock)+owns their semantics; the plan does not reinterpret source choice or presentation precedence.diff --git a/packages/media-pipeline/src/audio-plan.ts b/packages/media-pipeline/src/audio-plan.tsindex e07cb5e8..6172a5c2 100644--- a/packages/media-pipeline/src/audio-plan.ts+++ b/packages/media-pipeline/src/audio-plan.ts@@ -1,6 +1,6 @@import { programFrameSampleBoundary, programSpaceFrameCount } from "@hypit/program-space";import type { ProgramSpace } from "@hypit/program-space";-import { assertCompositionIdentity } from "@hypit/composition";+import { assertCompositionIdentity, assertAudioPresentation } from "@hypit/composition";import type { Composition } from "@hypit/composition";import { canonicalize, isResourceId } from "@hypit/protocol";@@ -62,6 +62,7 @@ export function verifyAudioProgramPlan(value: unknown): asserts value is AudioPrassert(clip.pitch === "preserve", `AudioProgramPlan clip ${clip.id} pitch policy is invalid`);assert(Number.isFinite(clip.gain) && clip.gain >= 0 && clip.gain <= 64,`AudioProgramPlan clip ${clip.id} gain is invalid`);+ assertAudioPresentation(clip, { startSample: clip.targetStartSample, endSampleExclusive: clip.targetEndSampleExclusive }, item.sampleFrames);const length = clip.targetEndSampleExclusive - clip.targetStartSample;assert(Number.isSafeInteger(clip.fadeInSamples) && clip.fadeInSamples >= 0 && clip.fadeInSamples <= length&& Number.isSafeInteger(clip.fadeOutSamples) && clip.fadeOutSamples >= 0 && clip.fadeOutSamples <= length,@@ -100,6 +101,8 @@ export function compileAudioProgramPlan(composition: Composition, programSpace:gain: clip.gain,fadeInSamples: clip.fadeInSamples,fadeOutSamples: clip.fadeOutSamples,+ ...(clip.gainEnvelope === undefined ? {} : { gainEnvelope: clip.gainEnvelope }),+ ...(clip.audibility === undefined ? {} : { audibility: clip.audibility }),};}));return sealAudioProgramPlan({diff --git a/packages/media-pipeline/src/manifest.ts b/packages/media-pipeline/src/manifest.tsindex d14e9c7c..118a4c3c 100644--- a/packages/media-pipeline/src/manifest.ts+++ b/packages/media-pipeline/src/manifest.ts@@ -3,7 +3,7 @@ import { mediaDependency, mediaTypes } from "@hypit/media";import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";import { svsModuleRef, svsRecipeType } from "@hypit/svs";import { speechDependency, speechTypes } from "@hypit/speech";-import { compositionDependency, compositionTypes } from "@hypit/composition";+import { compositionDependency, compositionTypes, audioSampleSpanSchema, audioGainEnvelopeSchema } from "@hypit/composition";import { artifactTypes } from "@hypit/artifact";import type {CapabilityRef,@@ -237,6 +237,8 @@ export const audioProgramPlanSchema: ValueSchema = {gain: { schema: { kind: "number", minimum: 0, maximum: 64 } },fadeInSamples: { schema: integer },fadeOutSamples: { schema: integer },+ gainEnvelope: { schema: audioGainEnvelopeSchema, optional: true },+ audibility: { schema: { kind: "array", items: audioSampleSpanSchema }, optional: true },},} } },mix: { schema: {diff --git a/packages/media-pipeline/src/types.ts b/packages/media-pipeline/src/types.tsindex ded0c04b..21e02ca0 100644--- a/packages/media-pipeline/src/types.ts+++ b/packages/media-pipeline/src/types.ts@@ -1,3 +1,4 @@+import type { AudioPresentation } from "@hypit/composition";import type { BlobRef } from "@hypit/protocol";import type { MediaFrameRange, MediaInspection, MediaRational, MediaStreamSelection, RenderedVisual, TimelineAudio } from "@hypit/media";@@ -134,7 +135,7 @@ export type ProjectSpeechEvidenceAudioNeed = {readonly evidenceSampleFrames: number;};-export type AudioProgramClip = {+export type AudioProgramClip = AudioPresentation & {readonly id: string;readonly artifact: BlobRef;readonly targetStartSample: number;diff --git a/packages/media-track-studio/README.md b/packages/media-track-studio/README.mdindex fe0c8047..68c225ce 100644--- a/packages/media-track-studio/README.md+++ b/packages/media-track-studio/README.md@@ -1,3 +1,12 @@# `@hypit/media-track-studio`Hypit Studio Companion for `@hypit/media-track`.++Each Item has a title and image or video body. An explicit author `id` supplies+its title; otherwise the Companion displays its `image`, `media` or `surface`+reference. Display names do not replace the Item's identity or writeback+endpoint. Source-audio gain uses the same percentage display and scalar+writeback as Audio Track.++Media and Audio Track share selection and temporal editing behavior. Media+composes pictures by draw order; Audio Track mixes overlapping sounds.diff --git a/packages/media-track-studio/src/index.ts b/packages/media-track-studio/src/index.tsindex 3a2d887f..8d556ac1 100644--- a/packages/media-track-studio/src/index.ts+++ b/packages/media-track-studio/src/index.ts@@ -1,4 +1,4 @@-import { mediaTrackMarkupSurfaces, mediaTrackModuleRef, mediaTrackTypes } from "@hypit/media-track";+import { mediaAppearanceDefaults, mediaTrackMarkupSurfaces, mediaTrackModuleRef, mediaTrackTypes } from "@hypit/media-track";import type { MediaLayerProgram, MediaTrackProgram } from "@hypit/media-track";import { compositionTypes } from "@hypit/composition";import { sameType } from "@hypit/protocol";@@ -10,7 +10,7 @@ import type {StudioMaterialPreview,StudioSourceBindingDeclaration,} from "@hypit/studio-adapter";-import { artifactPreview, childEntities, previewLayer, requiredSurfaceValue, temporalLineageFor, temporalSemanticSource } from "@hypit/studio-adapter";+import { artifactPreview, authoredItemTitle, childEntities, previewLayer, requiredSurfaceValue, temporalLineageFor, temporalSemanticSource } from "@hypit/studio-adapter";const frameParameters: readonly StudioSourceBindingDeclaration[] = [{ name: "within" },@@ -70,13 +70,15 @@ const mediaPercentProperties = new Set(["frame-x", "frame-y", "content-x", "contconst mediaPixelProperties = new Set(["fit-offset-x", "fit-offset-y", "radius", "blur", "border-width"]);function mediaRecipe(name: "appearance" | "motion"): {- readonly bindings: readonly { readonly name: string }[];+ readonly bindings: readonly import("@hypit/studio-adapter").StudioRecipeBindingDeclaration[];readonly inspector: readonly StudioInspectorFieldDeclaration[];} {const attribute = mediaItemAttributes.find((candidate) => candidate.name === name);const properties = attribute !== undefined && "recipe" in attribute ? attribute.recipe : [];return {- bindings: properties.map(({ name: property }) => ({ name: property })),+ bindings: properties.map(({ name: property }) => ({ name: property,+ ...(Object.hasOwn(mediaAppearanceDefaults, property) ? { fallback: mediaAppearanceDefaults[property as keyof typeof mediaAppearanceDefaults] } : {}),+ })),inspector: properties.map((property) => {const placement = mediaPlacement.get(property.name);if (placement === undefined) throw new Error(`Media Studio has no explicit Inspector declaration for ${property.name}.`);@@ -99,7 +101,8 @@ function mediaRecipe(name: "appearance" | "motion"): {};}-const mediaAppearanceRecipe = mediaRecipe("appearance");+export const mediaAppearanceStudioFields = mediaRecipe("appearance");+const mediaAppearanceRecipe = mediaAppearanceStudioFields;const mediaMotionRecipe = mediaRecipe("motion");function materialPreview(@@ -153,15 +156,16 @@ function projectMedia(context: StudioTrackCompanionContext): readonly StudioEnticonst semanticSource = temporalSemanticSource(temporal);return {...entity,+ display: {+ ...entity.display,+ title: authoredItemTitle(context, entity.authoredId, item.sourceTypes, ["image", "media", "surface"]),+ ...(item.preview === undefined ? {} : {+ layers: [previewLayer(item.preview, facet === "audio" ? "waveform" : item.preview.kind === "video" ? "storyboard" : "repeat-x")],+ }),+ },...(semanticSource?.id === undefined? {}: { markerId: semanticSource.id }),- ...(item.preview === undefined ? {} : {- display: {- ...entity.display,- layers: [previewLayer(item.preview, facet === "audio" ? "waveform" : item.preview.kind === "video" ? "storyboard" : "repeat-x")],- },- }),...(temporal === undefined ? {} : { temporal }),};});@@ -178,6 +182,7 @@ const commonInspector: readonly StudioInspectorFieldDeclaration[] = [{binding: "audio-gain", label: "Audio Gain", domain: "how",page: { id: "audio", label: "Audio" }, section: { id: "audio", label: "Audio" }, control: "number",+ unit: "%", number: { scale: 100, minimum: 0, step: 1 },},{binding: "until-boundary", label: "Until Boundary", domain: "when",diff --git a/packages/media-track/README.md b/packages/media-track/README.mdindex 47d93cd5..a83a1421 100644--- a/packages/media-track/README.md+++ b/packages/media-track/README.md@@ -1,61 +1,28 @@# `@hypit/media-track`-The Track Surface accepts `semantic={speech.semantic}` or `space={animation}`. Semantic context-resolves Script Selections and Moments; authored space supports clock-based animation. Shared `at`+The Track Surface accepts `timeline={program.timeline}`. The same Timeline supports authored+positions and, where prepared Takes supply evidence, Script Selections and Moments. Shared `at`inputs accept a Moment or a time such as `2s`; `at` with `for` produces a Window where required.-`Performance` still needs `semantic`, which supplies its ordered prepared footage.+[Performance](../performance/README.md) separately obtains existing footage and source positions from Timeline.Place images, prepared video or compositable surfaces over the semantic timeline. B-roll is a common-use: the picture can illustrate a phrase while the A-roll's speech continues. Performance presents the semantic material as a full picture, inset or cutout.+use: the picture can illustrate a phrase while the A-roll's speech continues.Media Items own independently timed pictures;a Sequence owns one visual slot whose Members replace each other through Handoffs.-The Markup Track takes `canvas` and a time source (`semantic` or `space`). Its Surface projects+The Markup Track takes `canvas` and `timeline`. Its Surface projectsauthored Selections, Moments, Segments or clock expressions into Windows and Instants. The component-consumes those projected times with Frames, media and appearance; its Fragment receives ProgramSpace.+consumes those projected times with Frames, media and appearance; its Fragment receives the same Timeline.Moving media enters after [normalization](../media-pipeline/README.md), with the intended streams andframe clock already selected. Still images use their actual intrinsic Extent. Placement, fitting,sampling and motion remain separate authored choices.-## Display a semantic performance+## Present existing Timeline footage-```svml-<media:Track id="performance" semantic={speech.semantic} canvas={canvas}>- <media:Performance during="program" frame={layout.full}- appearance={look.performance}/>-</media:Track>-```--`Performance` displays the prepared material in its enclosing Track's `semantic` input. `during`-selects the whole program, a Segment or a Selection; the other Window forms work as usual. One-Performance can span any number of Takes. Each plays at its original program position: if a Take-begins at program second 5, a Window from seconds 7 to 10 shows source seconds 2 to 5. An audio-only-interval contributes no picture. The presentation shares one frame, appearance and lifecycle across-Take boundaries.--Its appearance Recipe needs `stack-order` and can set fit, crop anchors, opacity, filters, clipping,-padding, border, shadow and frame paint. `motion` moves the framed presentation, and `Sampling`-children transform the fitted picture over the complete selected Window, without restarting at-Take boundaries. `Sound` children attach prepared effects to its entrance or exit. Its original-sound is independently available as `speech.audio` for Film. Use an Item with direct `media` for-independent playback or source trim; Performance preserves the semantic assembly's source timing.--```svml-<media:Performance during={story.segment.explanation} frame={layout.full}- appearance={look.performance}>- <media:Sampling at="start" zoom="1"/>- <media:Sampling at="end" zoom="1.08"/>-</media:Performance>-```--This excerpt belongs inside the same Media Track. Its selected passage determines the appearance-interval, while the two Sampling keys direct a gradual push-in inside the unchanged Frame.--For a fixed card, full picture, circle or cutout this is ordinary Media presentation. If video and-graphics must resize, mask or move together, write a project component that owns that relationship.-`projectSemanticMedia` supplies the same material spans to its drawing function. The component can-use a browser program from `@hypit/hypit/hyperframes` while Caption and independent overlays remain peers.+[Performance](../performance/README.md) obtains existing footage from Timeline and applies broad+and local Use rules. Use Media Items here for independently supplied assets and playback choices.+Both reuse the same geometry and fitting capabilities.## Place the frame, then fit its contents@@ -115,10 +82,10 @@ a deliberate pan or rotation may reveal the backing or lower layers.## Author Items and replacement SequencesThese excerpts assume the imports, named assets, normalized media, Frames, Extents, Script,-SemanticTrack and Recipes are already declared:+Timeline and Recipes are already declared:```svml-<media-track:Track id="coverage" semantic={speech.semantic} canvas={vertical}>+<media-track:Track id="coverage" timeline={speech.timeline} canvas={vertical}><media-track:Item image={photo} extent={photo-extent} frame={full-frame}during={story.selection.example} appearance={recipes.media.still}/><media-track:Item media={clip-media.media} frame={full-frame}@@ -132,7 +99,7 @@ Appearance owns stack order, fit, clipping and source sampling; a `motion` Recipmovement. `Sampling` children can move the sampled content inside the Frame independently.```svml-<media-track:Track id="cards" semantic={speech.semantic} canvas={vertical}>+<media-track:Track id="cards" timeline={speech.timeline} canvas={vertical}><media-track:Sequence id="steps" frame={card-frame} appearance={recipes.media.card}until={story.selection.demo} until-boundary="end"><media-track:Member id="one" image={first} extent={card-extent}@@ -177,7 +144,7 @@ separately controls which source frames it occupies there. `playback` defaults t`trim-start` and `trim-end` are a pair of integer source-frame boundaries, with an exclusive end.Omit both to use the full source. The playback rules apply to the trimmed range when present.For `stretch`, the source-frame advance is `sourceLength / targetLength`. For native playback, moving-media has already been normalized to ProgramSpace's frame rate, so one program frame advances one+media has already been normalized to Timeline's frame rate, so one program frame advances onesource frame. These are source sampling choices; they do not move a Selection's semantic boundaries.A two-second Item can use the beginning of a four-second generated clip without requesting adiff --git a/packages/media-track/package.json b/packages/media-track/package.jsonindex 3a144622..3b951db2 100644--- a/packages/media-track/package.json+++ b/packages/media-track/package.json@@ -19,7 +19,7 @@"@hypit/narrative": "workspace:*","@hypit/program-space": "workspace:*","@hypit/protocol": "workspace:*",- "@hypit/semantic-track": "workspace:*",+ "@hypit/timeline": "workspace:*","@hypit/spatial": "workspace:*","@hypit/svs": "workspace:*","@hypit/markup": "workspace:*",diff --git a/packages/media-track/preview/preview.svml b/packages/media-track/preview/preview.svmlindex d2608ffe..8c70de52 100644--- a/packages/media-track/preview/preview.svml+++ b/packages/media-track/preview/preview.svml@@ -6,7 +6,7 @@<import from="@hypit/script@1"/><import as="media" from="@hypit/media@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="whisperx" from="@hypit/whisperx@1"/><import as="media-track" from="@hypit/media-track@1"/><import as="space" from="@hypit/spatial@1"/>@@ -32,16 +32,16 @@video="primary-moving" audio="none" span-authority="video"/><whisperx:SemanticTake id="showcase-time" narrative={story}segment={story.segment.empty} media={duration-media.media}/>- <speech:Track id="speech"><speech:Take source={showcase-time.take}/></speech:Track>+ <time:Timeline id="speech" clock={clock}><time:Take source={showcase-time.take}/></time:Timeline>- <media-track:Track id="showcase" semantic={speech.semantic} canvas={vertical}>+ <media-track:Track id="showcase" timeline={speech.timeline} canvas={vertical}><media-track:Item id="rounded-card" image={picture-a} extent={picture-size} during="program"frame={card-a} appearance={recipes.media.card} motion={recipes.motion.card}/><media-track:Item id="flush-card" image={picture-b} extent={picture-size} during="program"frame={card-b} appearance={recipes.media.wide} motion={recipes.motion.card}/></media-track:Track>- <film:Film id="main" canvas={vertical} semantic={speech.semantic} appearance={recipes.film.vertical}>+ <film:Film id="main" canvas={vertical} timeline={speech.timeline} appearance={recipes.film.vertical}><film:Track source={showcase.visual}/></film:Film>- <render:Video id="final" composition={main.composition} semantic={speech.semantic}/>+ <render:Video id="final" composition={main.composition} timeline={speech.timeline}/></svml>diff --git a/packages/media-track/src/author.ts b/packages/media-track/src/author.tsindex 72b16404..d58e3890 100644--- a/packages/media-track/src/author.ts+++ b/packages/media-track/src/author.ts@@ -182,12 +182,18 @@ export function decodeMediaSampleSpec(}/** Decode visual-only sample styling without choosing playback or source trim. */+/** Public authored defaults shared by decoding and optional editor fields. */+export const mediaAppearanceDefaults = {+ opacity: 1, blur: 0, brightness: 1, contrast: 1, saturation: 1,+ clip: "frame", radius: 0,+} as const;+export function decodeMediaSampleAppearance(recipe: SvsRecipe): MediaSampleAppearance {return {- opacity: number(recipe, "opacity", 1),+ opacity: number(recipe, "opacity", mediaAppearanceDefaults.opacity),filter: {- blurPx: number(recipe, "blur", 0), brightness: number(recipe, "brightness", 1),- contrast: number(recipe, "contrast", 1), saturation: number(recipe, "saturation", 1),+ blurPx: number(recipe, "blur", mediaAppearanceDefaults.blur), brightness: number(recipe, "brightness", mediaAppearanceDefaults.brightness),+ contrast: number(recipe, "contrast", mediaAppearanceDefaults.contrast), saturation: number(recipe, "saturation", mediaAppearanceDefaults.saturation),},};}@@ -236,10 +242,10 @@ export function decodeMediaMotion(recipe: SvsRecipe | undefined): MediaLifecycle}export function decodeMediaPresentation(recipe: SvsRecipe): MediaFramePresentation {- const clip = oneOf(recipe, "clip", ["none", "frame", "rounded"] as const, "frame");+ const clip = oneOf(recipe, "clip", ["none", "frame", "rounded"] as const, mediaAppearanceDefaults.clip);const borderWidth = number(recipe, "border-width", 0);return {- clip: clip === "rounded" ? { kind: "rounded", radiusPx: number(recipe, "radius", 0) } : { kind: clip },+ clip: clip === "rounded" ? { kind: "rounded", radiusPx: number(recipe, "radius", mediaAppearanceDefaults.radius) } : { kind: clip },padding: padding(recipe),...(borderWidth === 0 ? {} : { border: {widthPx: borderWidth,@@ -278,7 +284,7 @@ export function decodeMediaFramePaint(recipe: SvsRecipe, id: string): MediaPaintexport function decodeMediaPaintSpec(recipe: SvsRecipe, id: string): MediaPaintLayerSpec {assertKeys(recipe, ["paint", "opacity"]);return sealMediaPaintLayerSpec({- id, paint: paint(recipe), opacity: number(recipe, "opacity", 1),+ id, paint: paint(recipe), opacity: number(recipe, "opacity", mediaAppearanceDefaults.opacity),});}diff --git a/packages/media-track/src/component.ts b/packages/media-track/src/component.tsindex 3fbe8b5b..9f379def 100644--- a/packages/media-track/src/component.ts+++ b/packages/media-track/src/component.ts@@ -1,10 +1,8 @@-import { appendMediaPerformance } from "./performance.js";-import type { SemanticTrack } from "@hypit/semantic-track";+import type { Timeline } from "@hypit/timeline";import type { ComponentPackage, ProducerHandlerContext } from "@hypit/component-kit";import type { CompositableSurfaceRef, SynchronizedMedia } from "@hypit/media";import { canonicalize } from "@hypit/protocol";import type { BlobRef, StoredValue } from "@hypit/protocol";-import type { ProgramSpace } from "@hypit/program-space";import type { CanvasSpace, ContentFit, IntrinsicExtent, SpatialFrame, SpatialPath } from "@hypit/spatial";import type { TemporalInstant, TemporalWindow } from "@hypit/temporal";@@ -54,7 +52,7 @@ function itemInputs(inputs: ProducerHandlerContext["inputs"]) {return {set: inline<MediaTrackSet>(inputs.set?.value, "MediaTrackSet"),header: inline<MediaTrackHeader>(inputs.header?.value, "MediaTrackHeader"),- space: inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ timeline: inline<Timeline>(inputs.timeline?.value, "Timeline"),canvas: inline<CanvasSpace>(inputs.canvas?.value, "CanvasSpace"),layers: inline<MediaLayerSet>(inputs.layers?.value, "MediaLayerSet"),frame: inline<SpatialFrame>(inputs.frame?.value, "SpatialFrame"),@@ -67,7 +65,7 @@ function itemInputs(inputs: ProducerHandlerContext["inputs"]) {function memberInputs(inputs: ProducerHandlerContext["inputs"]) {return {members: inline<MediaSequenceMemberSet>(inputs.members?.value, "MediaSequenceMemberSet"),- space: inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ timeline: inline<Timeline>(inputs.timeline?.value, "Timeline"),layers: inline<MediaLayerSet>(inputs.layers?.value, "MediaLayerSet"),spec: inline<MediaSequenceMemberSpec>(inputs.spec?.value, "MediaSequenceMemberSpec"),activation: inline<TemporalInstant>(inputs.activation?.value, "TemporalInstant"),@@ -78,7 +76,7 @@ function sequenceInputs(inputs: ProducerHandlerContext["inputs"]) {return {set: inline<MediaTrackSet>(inputs.set?.value, "MediaTrackSet"),header: inline<MediaTrackHeader>(inputs.header?.value, "MediaTrackHeader"),- space: inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ timeline: inline<Timeline>(inputs.timeline?.value, "Timeline"),canvas: inline<CanvasSpace>(inputs.canvas?.value, "CanvasSpace"),members: inline<MediaSequenceMemberSet>(inputs.members?.value, "MediaSequenceMemberSet"),frame: inline<SpatialFrame>(inputs.frame?.value, "SpatialFrame"),@@ -90,10 +88,6 @@ function sequenceInputs(inputs: ProducerHandlerContext["inputs"]) {export const mediaTrackComponent = {producers: [- { producer: mediaTrackProducers.appendPerformance, handler: ({ inputs }) => ({ outputs: { set: output(appendMediaPerformance(- itemInputs(inputs), inline<SemanticTrack>(inputs.semantic?.value, "SemanticTrack"),- inline<ContentFit>(inputs.fit?.value, "ContentFit"), inline<MediaSampleLayerSpec>(inputs.sampleSpec?.value, "MediaSampleLayerSpec"),- )) }, needs: {} }) },{ producer: mediaTrackProducers.createLayers, handler: () => ({ outputs: { layers: output(createMediaLayerSet()) }, needs: {} }) },{ producer: mediaTrackProducers.appendPaintLayer, handler: ({ inputs }) => ({ outputs: { layers: output(appendMediaPaintLayer(inline<MediaLayerSet>(inputs.layers?.value, "MediaLayerSet"),@@ -121,7 +115,7 @@ export const mediaTrackComponent = {{ producer: mediaTrackProducers.createSet, handler: () => ({ outputs: { set: output(createMediaTrackSet()) }, needs: {} }) },{ producer: mediaTrackProducers.appendItem, handler: ({ inputs }) => {const value = itemInputs(inputs);- return { outputs: { set: output(appendMediaItem(value.set, value.header, value.space, value.canvas, value.layers, value.frame, value.spec, value.sounds, value.window)) }, needs: {} };+ return { outputs: { set: output(appendMediaItem(value.set, value.header, value.timeline, value.canvas, value.layers, value.frame, value.spec, value.sounds, value.window)) }, needs: {} };} },{ producer: mediaTrackProducers.bindItemClipPath, handler: ({ inputs }) => ({ outputs: { spec: output(bindMediaItemClipPath(inline<MediaItemSpec>(inputs.spec?.value, "MediaItemSpec"),@@ -135,24 +129,24 @@ export const mediaTrackComponent = {{ producer: mediaTrackProducers.appendMember, handler: ({ inputs }) => {const value = memberInputs(inputs);return { outputs: { members: output(appendMediaSequenceMember(- value.members, value.space, value.layers, value.spec, value.activation,+ value.members, value.timeline, value.layers, value.spec, value.activation,)) }, needs: {} };} },{ producer: mediaTrackProducers.appendSequence, handler: ({ inputs }) => {const value = sequenceInputs(inputs);return { outputs: { set: output(appendMediaSequence(- value.set, value.header, value.space, value.canvas, value.members, value.frame, value.spec, value.sounds, value.terminal,+ value.set, value.header, value.timeline, value.canvas, value.members, value.frame, value.spec, value.sounds, value.terminal,)) }, needs: {} };} },{ producer: mediaTrackProducers.finalize, handler: ({ inputs }) => ({ outputs: { program: output(finalizeMediaTrack(inline<MediaTrackSet>(inputs.set?.value, "MediaTrackSet"), inline<MediaTrackHeader>(inputs.header?.value, "MediaTrackHeader"),- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ inline<Timeline>(inputs.timeline?.value, "Timeline"),)) }, needs: {} }) },{ producer: mediaTrackProducers.projectVisual, handler: ({ inputs }) => ({ outputs: { track: output(projectMediaVisualTrack(- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"), inline<MediaTrackProgram>(inputs.program?.value, "MediaTrackProgram"),+ inline<Timeline>(inputs.timeline?.value, "Timeline"), inline<MediaTrackProgram>(inputs.program?.value, "MediaTrackProgram"),)) }, needs: {} }) },{ producer: mediaTrackProducers.projectAudio, handler: ({ inputs }) => ({ outputs: { track: output(projectMediaAudioTrack(- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"), inline<MediaTrackProgram>(inputs.program?.value, "MediaTrackProgram"),+ inline<Timeline>(inputs.timeline?.value, "Timeline"), inline<MediaTrackProgram>(inputs.program?.value, "MediaTrackProgram"),)) }, needs: {} }) },],validators: [{diff --git a/packages/media-track/src/fragment.ts b/packages/media-track/src/fragment.tsindex 196ba6f4..1793e02f 100644--- a/packages/media-track/src/fragment.ts+++ b/packages/media-track/src/fragment.ts@@ -1,4 +1,5 @@-import { programSpaceTypes } from "@hypit/program-space";+import { timelineTypes } from "@hypit/timeline";+import { artifactTypes } from "@hypit/artifact";import { compositionTypes } from "@hypit/composition";import { sealGraphFragment } from "@hypit/elaborator";@@ -14,7 +15,7 @@ const operation = (id: string) => ({ kind: "fragment-operation" as const, operatexport const stillMediaTrackFragment = sealGraphFragment({inputs: [{ name: "header", type: mediaTrackTypes.header },- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },{ name: "canvas", type: spatialTypes.canvas },{ name: "source", type: artifactTypes.blob },{ name: "extent", type: spatialTypes.extent },@@ -32,14 +33,14 @@ export const stillMediaTrackFragment = sealGraphFragment({{ id: "set", producer: mediaTrackProducers.createSet, inputs: {}, result: { kind: "output", name: "set" } },{ id: "sounds", producer: mediaTrackProducers.createSounds, inputs: {}, result: { kind: "output", name: "sounds" } },{ id: "append", producer: mediaTrackProducers.appendItem, inputs: {- set: operation("set"), header: input("header"), space: input("space"), canvas: input("canvas"), layers: operation("sample"),+ set: operation("set"), header: input("header"), timeline: input("timeline"), canvas: input("canvas"), layers: operation("sample"),frame: input("frame"), spec: input("item-spec"), sounds: operation("sounds"), window: input("window"),}, result: { kind: "output", name: "set" } },{ id: "finalize", producer: mediaTrackProducers.finalize, inputs: {- set: operation("append"), header: input("header"), space: input("space"),+ set: operation("append"), header: input("header"), timeline: input("timeline"),}, result: { kind: "output", name: "program" } },{ id: "visual", producer: mediaTrackProducers.projectVisual, inputs: {- space: input("space"), program: operation("finalize"),+ timeline: input("timeline"), program: operation("finalize"),}, result: { kind: "output", name: "track" } },],exports: [@@ -49,9 +50,9 @@ export const stillMediaTrackFragment = sealGraphFragment({});export const renderMediaTrackFragment = sealGraphFragment({- inputs: [{ name: "space", type: programSpaceTypes.programSpace }, { name: "program", type: mediaTrackTypes.program }],+ inputs: [{ name: "timeline", type: timelineTypes.track }, { name: "program", type: mediaTrackTypes.program }],operations: [- { id: "visual", producer: mediaTrackProducers.projectVisual, inputs: { space: input("space"), program: input("program") }, result: { kind: "output", name: "track" } },+ { id: "visual", producer: mediaTrackProducers.projectVisual, inputs: { timeline: input("timeline"), program: input("program") }, result: { kind: "output", name: "track" } },],exports: [{ name: "track", type: compositionTypes.visualTrack, root: operation("visual") }],});diff --git a/packages/media-track/src/index.ts b/packages/media-track/src/index.tsindex ebe07acb..da0b0a56 100644--- a/packages/media-track/src/index.ts+++ b/packages/media-track/src/index.ts@@ -1,6 +1,7 @@export { mediaTrackComponent } from "./component.js";export {decodeMediaFit,+ decodeMediaItemSpec,decodeMediaFramePaint,decodeMediaMotion,decodeMediaPresentation,@@ -46,7 +47,6 @@ export {} from "./sounds.js";export { appendMediaItem, appendMediaSequence, bindMediaItemClipPath, bindMediaSequenceClipPath, assertMediaItemSpec, assertMediaTrackHeader, assertMediaTrackProgram, assertMediaTrackProgramIdentity, assertMediaTrackSet, createMediaTrackSet, finalizeMediaTrack, projectMediaAudioTrack, projectMediaVisualTrack, sealMediaItemSpec, sealMediaTrackHeader, sealMediaTrackProgram } from "./program.js";export { resolveVisualSampling } from "./sampling.js";-export { appendMediaPerformance } from "./performance.js";export { decodeMediaSamplingKeyframe, decodeMediaTrackSurface } from "./surface.js";export {appendMediaSequenceMember,@@ -63,3 +63,5 @@ export {sequenceMemberHandoffAnimation,} from "./sequence.js";export type * from "./types.js";++export { mediaAppearanceDefaults } from "./author.js";diff --git a/packages/media-track/src/lower.ts b/packages/media-track/src/lower.tsindex 8629a3e2..438bfaa1 100644--- a/packages/media-track/src/lower.ts+++ b/packages/media-track/src/lower.ts@@ -1,10 +1,10 @@+import type { Timeline } from "@hypit/timeline";import type {VisualAnimation,VisualElement,VisualStyleDeclaration,VisualTimedSampling,} from "@hypit/composition";-import type { ProgramSpace } from "@hypit/program-space";import { fitContent } from "@hypit/spatial";import type { SpatialFrame, SpatialPath } from "@hypit/spatial";@@ -133,7 +133,7 @@ function sampleElements(item: MediaItemProgram,parent: string,order: { value: number },- space: ProgramSpace,+ timeline: Timeline,samplingOverrides: Readonly<Record<string, VisualTimedSampling>>,samplingAnimationOverrides: Readonly<Record<string, VisualAnimation | null>>,): VisualElement[] {@@ -209,7 +209,7 @@ function sampleElements(}];}const sampling = samplingOverrides[layer.id] ?? layer.sampling ?? resolveVisualSampling({- space,+ timeline,sourceFrameRate: sourceTiming.frameRate,sourceFrameCount: sourceTiming.frameCount,targetFrameCount: durationFrames,@@ -244,12 +244,12 @@ function layerElements(item: MediaItemProgram,parent: string,order: { value: number },- space: ProgramSpace,+ timeline: Timeline,samplingOverrides: Readonly<Record<string, VisualTimedSampling>>,samplingAnimationOverrides: Readonly<Record<string, VisualAnimation | null>>,): VisualElement[] {if (layer.kind === "sample") {- return sampleElements(layer, item, parent, order, space, samplingOverrides, samplingAnimationOverrides);+ return sampleElements(layer, item, parent, order, timeline, samplingOverrides, samplingAnimationOverrides);}return [{id: layer.id,@@ -271,7 +271,7 @@ function layerElements(/** Lower one independent Item through the fixed wrapper stack from the Media spec. */export function lowerMediaItemElements(item: MediaItemProgram,- space: ProgramSpace,+ timeline: Timeline,options: {readonly includeHandoffWrapper?: boolean;readonly handoffAnimation?: VisualAnimation;@@ -326,7 +326,7 @@ export function lowerMediaItemElements(item,frame,order,- space,+ timeline,options.samplingOverrides ?? {},options.samplingAnimationOverrides ?? {},));diff --git a/packages/media-track/src/manifest.ts b/packages/media-track/src/manifest.tsindex e3bb781e..c0fd9cfb 100644--- a/packages/media-track/src/manifest.ts+++ b/packages/media-track/src/manifest.ts@@ -1,3 +1,4 @@+import type { Timeline } from "@hypit/timeline";import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";import { visualTimedSamplingSchema } from "@hypit/composition";import { readFile } from "node:fs/promises";@@ -10,9 +11,9 @@ import {mediaTypes,} from "@hypit/media";import { narrativeDependency, narrativeTypes } from "@hypit/narrative";-import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";+import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";-import { semanticTrackDependency, semanticTrackTypes } from "@hypit/semantic-track";+import { timelineDependency, timelineTypes } from "@hypit/timeline";import {contentFitSchema,intrinsicExtentSchema,@@ -48,7 +49,6 @@ export const mediaTrackTypes = {} satisfies Record<string, TypeRef>;export const mediaTrackProducers = {- appendPerformance: { module: mediaTrackModuleRef, name: "append-media-performance" },createLayers: { module: mediaTrackModuleRef, name: "create-media-layer-set" },appendPaintLayer: { module: mediaTrackModuleRef, name: "append-media-paint-layer" },appendStillLayer: { module: mediaTrackModuleRef, name: "append-still-media-layer" },@@ -256,18 +256,18 @@ export const mediaTrackHeaderSchema: ValueSchema = object({const itemInputs = [{ name: "set", type: mediaTrackTypes.set }, { name: "header", type: mediaTrackTypes.header },- { name: "space", type: programSpaceTypes.programSpace }, { name: "canvas", type: spatialTypes.canvas },+ { name: "timeline", type: timelineTypes.track }, { name: "canvas", type: spatialTypes.canvas },{ name: "layers", type: mediaTrackTypes.layerSet },{ name: "frame", type: spatialTypes.frame }, { name: "spec", type: mediaTrackTypes.itemSpec },{ name: "sounds", type: mediaTrackTypes.soundSet }, { name: "window", type: temporalTypes.window },] as const;const memberInputs = [- { name: "members", type: mediaTrackTypes.memberSet }, { name: "space", type: programSpaceTypes.programSpace }, { name: "layers", type: mediaTrackTypes.layerSet },+ { name: "members", type: mediaTrackTypes.memberSet }, { name: "timeline", type: timelineTypes.track }, { name: "layers", type: mediaTrackTypes.layerSet },{ name: "spec", type: mediaTrackTypes.memberSpec }, { name: "activation", type: temporalTypes.instant },] as const;const sequenceInputs = [{ name: "set", type: mediaTrackTypes.set }, { name: "header", type: mediaTrackTypes.header },- { name: "space", type: programSpaceTypes.programSpace }, { name: "canvas", type: spatialTypes.canvas },+ { name: "timeline", type: timelineTypes.track }, { name: "canvas", type: spatialTypes.canvas },{ name: "members", type: mediaTrackTypes.memberSet },{ name: "frame", type: spatialTypes.frame }, { name: "spec", type: mediaTrackTypes.sequenceSpec },{ name: "sounds", type: mediaTrackTypes.soundSet }, { name: "terminal", type: temporalTypes.instant },@@ -368,7 +368,7 @@ export const mediaTrackMarkupSurfaces = [{temporalTypes.instantSpec, temporalTypes.windowSpec, temporalTypes.instant, temporalTypes.window, mediaTrackTypes.program,compositionTypes.visualTrack, compositionTypes.audioTrack],vocabulary: {- summary: "Show the semantic performance, place independent media Items, and compose replacement Sequences on a shared film time axis and Canvas. Publishes a VisualTrack and, when audio is authored, an AudioTrack.",+ summary: "Show the timeline performance, place independent media Items, and compose replacement Sequences on a shared film time axis and Canvas. Publishes a VisualTrack and, when audio is authored, an AudioTrack.",appearance: "Pictures and layered compositions occupy authored Frames on the Canvas. Each source is scaled and aligned inside its Frame's border and padding; a contain fit can leave space, while cover can crop. Paint and sampled Layers draw in Source order under a shared rectangular, rounded or Path clip, with optional border and shadows. Lifecycle motion moves the framed unit; Sampling pans, zooms or rotates the picture inside it. Each Item has an independent window. A Sequence shares one outer Frame while Members replace its contents through cut, crossfade, push, wipe, cover or page-turn Handoffs. Source playback determines whether moving content covers the whole assigned span. Absolute stack order decides which overlapping unit draws in front.",preview: previewImage("Track.png"),attributes: [@@ -379,23 +379,6 @@ export const mediaTrackMarkupSurfaces = [{summary: "Chooses the Canvas every Frame on this Track is measured inside." },],children: [- { tag: "Performance", cardinality: "many",- summary: "Shows the enclosing Track's semantic performance during a chosen interval, preserving each Take's original source position under one frame and continuous motion. Accepts Sampling and Sound children.",- attributes: [- { name: "id", kind: "identifier", required: false,- summary: "Names this presentation; otherwise its identity follows the shared Item/Performance order." },- { name: "frame", kind: "reference", required: true, accepts: [spatialTypes.frame],- summary: "Places the performance picture on the Canvas." },- { name: "appearance", kind: "reference", required: true, accepts: [svsRecipeType],- summary: "Uses the Media fitting, picture styling, frame paint, clipping, border and shadow properties. Source time follows the semantic performance.",- recipe: appearanceRecipeProperties.filter(property => !["playback", "trim-start", "trim-end"].includes(property.name)) },- { name: "motion", kind: "reference", required: false, accepts: [svsRecipeType],- summary: "Moves the whole framed presentation over its Window, across Take boundaries.",- recipe: motionRecipeProperties },- { name: "clip", kind: "reference", required: false, accepts: [spatialTypes.path],- summary: "Clips the presentation to an authored Path." },- ...temporalWindowAttributeVocabulary,- ] },{ tag: "Item", cardinality: "many",summary: "One independently timed picture on its own Frame, from a direct source or ordered Paint and Layer children, carrying its own Sampling and Sound children.",attributes: [@@ -458,19 +441,18 @@ export const mediaTrackMarkupSurfaces = [{example: `<pipeline:Normalize id="cutaway-bags" source={bags-take.video}video="primary-moving" audio="none" span-authority="video" clock={clock}/>-<media-track:Track id="cutaways" semantic={speech.semantic} canvas={vertical}>- <media-track:Performance during="program" frame={full} appearance={recipes.media.presenter}/>+<media-track:Track id="cutaways" timeline={speech.timeline} canvas={vertical}>+ <media-track:Item media={prepared.media} during="program" frame={full} appearance={recipes.media.clip}/><media-track:Item id="bags" media={cutaway-bags.media} during={story.selection.bags}frame={full} appearance={recipes.media.cutaway} motion={recipes.motion.cut}/></media-track:Track>`,notes: [- "A Track requires at least one Performance, Item or Sequence and accepts no text content.",- "Performance and Item use one window form: `during`, `at` with `for`, `until` with `for`, or `start` with `end`. Bind `selection`, `segment` and/or `moment` only when the start/end expressions use them; different endpoints can use different bindings.",- "Performance uses the Track's existing semantic input. `during=\"program\"` shows all its prepared Takes in order; a Segment or Selection shows that interval at the corresponding source positions. Speech Track.audio independently supplies the original sound. Sampling pans, zooms or rotates the fitted picture across the whole selected Window.",+ "A Track requires at least one Item or Sequence and accepts no text content.",+ "Item use one window form: `during`, `at` with `for`, `until` with `for`, or `start` with `end`. Bind `selection`, `segment` and/or `moment` only when the start/end expressions use them; different endpoints can use different bindings.","A point expression is `program.start`, `program.end`, `selection.start`, `selection.end`, `segment.start`, `segment.end` or `moment.cue`, each optionally offset by `+` or `-` and a duration, or a bare duration read as an absolute position.","A unit that names a direct source names exactly one of `image`, `media` or `surface`; `extent` is required with `image` and refused otherwise.","`audio-gain` is refused without selected source audio.",- "`clip` is refused on a Performance, Item or Sequence whose Recipe already states a clip.",+ "`clip` is refused on a Item or Sequence whose Recipe already states a clip.","An `appearance` or `motion` Recipe is refused when it carries a property outside its own set.","`playback`, `trim-start` and `trim-end` are refused on durationless still material, and `trim-start` and `trim-end` are written together or not at all.","An `enter` or an `exit` operator requires its own `enter-frames` or `exit-frames`.",@@ -575,8 +557,7 @@ export const mediaTrackManifest: ModuleManifest = {artifactDependency,mediaDependency,narrativeDependency,- programSpaceDependency,- semanticTrackDependency,+ timelineDependency,temporalDependency,spatialDependency,compositionDependency,@@ -606,18 +587,15 @@ export const mediaTrackManifest: ModuleManifest = {{ name: mediaTrackProducers.createSounds.name, inputs: [], outputs: [{ name: "sounds", type: mediaTrackTypes.soundSet }], needs: [] },{ name: mediaTrackProducers.appendSound.name, inputs: [{ name: "sounds", type: mediaTrackTypes.soundSet }, { name: "source", type: mediaTypes.synchronized }, { name: "spec", type: mediaTrackTypes.soundSpec }], outputs: [{ name: "sounds", type: mediaTrackTypes.soundSet }], needs: [] },{ name: mediaTrackProducers.createSet.name, inputs: [], outputs: [{ name: "set", type: mediaTrackTypes.set }], needs: [] },- { name: mediaTrackProducers.appendPerformance.name, inputs: [...itemInputs,- { name: "semantic", type: semanticTrackTypes.track }, { name: "fit", type: spatialTypes.fit },- { name: "sampleSpec", type: mediaTrackTypes.sampleLayerSpec }], outputs: [{ name: "set", type: mediaTrackTypes.set }], needs: [] },{ name: mediaTrackProducers.appendItem.name, inputs: itemInputs, outputs: [{ name: "set", type: mediaTrackTypes.set }], needs: [] },{ name: mediaTrackProducers.bindItemClipPath.name, inputs: [{ name: "spec", type: mediaTrackTypes.itemSpec }, { name: "path", type: spatialTypes.path }], outputs: [{ name: "spec", type: mediaTrackTypes.itemSpec }], needs: [] },{ name: mediaTrackProducers.bindSequenceClipPath.name, inputs: [{ name: "spec", type: mediaTrackTypes.sequenceSpec }, { name: "path", type: spatialTypes.path }], outputs: [{ name: "spec", type: mediaTrackTypes.sequenceSpec }], needs: [] },{ name: mediaTrackProducers.createMembers.name, inputs: [], outputs: [{ name: "members", type: mediaTrackTypes.memberSet }], needs: [] },{ name: mediaTrackProducers.appendMember.name, inputs: memberInputs, outputs: [{ name: "members", type: mediaTrackTypes.memberSet }], needs: [] },{ name: mediaTrackProducers.appendSequence.name, inputs: sequenceInputs, outputs: [{ name: "set", type: mediaTrackTypes.set }], needs: [] },- { name: mediaTrackProducers.finalize.name, inputs: [{ name: "set", type: mediaTrackTypes.set }, { name: "header", type: mediaTrackTypes.header }, { name: "space", type: programSpaceTypes.programSpace }], outputs: [{ name: "program", type: mediaTrackTypes.program }], needs: [] },- { name: mediaTrackProducers.projectVisual.name, inputs: [{ name: "space", type: programSpaceTypes.programSpace }, { name: "program", type: mediaTrackTypes.program }], outputs: [{ name: "track", type: compositionTypes.visualTrack }], needs: [] },- { name: mediaTrackProducers.projectAudio.name, inputs: [{ name: "space", type: programSpaceTypes.programSpace }, { name: "program", type: mediaTrackTypes.program }], outputs: [{ name: "track", type: compositionTypes.audioTrack }], needs: [] },+ { name: mediaTrackProducers.finalize.name, inputs: [{ name: "set", type: mediaTrackTypes.set }, { name: "header", type: mediaTrackTypes.header }, { name: "timeline", type: timelineTypes.track }], outputs: [{ name: "program", type: mediaTrackTypes.program }], needs: [] },+ { name: mediaTrackProducers.projectVisual.name, inputs: [{ name: "timeline", type: timelineTypes.track }, { name: "program", type: mediaTrackTypes.program }], outputs: [{ name: "track", type: compositionTypes.visualTrack }], needs: [] },+ { name: mediaTrackProducers.projectAudio.name, inputs: [{ name: "timeline", type: timelineTypes.track }, { name: "program", type: mediaTrackTypes.program }], outputs: [{ name: "track", type: compositionTypes.audioTrack }], needs: [] },],};diff --git a/packages/media-track/src/program.ts b/packages/media-track/src/program.tsindex 9a533137..d2c3bf0f 100644--- a/packages/media-track/src/program.ts+++ b/packages/media-track/src/program.ts@@ -1,3 +1,4 @@+import type { Timeline } from "@hypit/timeline";import {assertAudioTrackIdentity,assertVisualTrackIdentity,@@ -5,13 +6,7 @@ import {sealVisualTrack,} from "@hypit/composition";import type { AudioClip, AudioTrack, VisualTrack } from "@hypit/composition";-import {- assertProgramSpaceIdentity,- programFrameSampleBoundary,- programSpaceFrameCount,- programSpaceSampleFrames,-} from "@hypit/program-space";-import type { ProgramSpace } from "@hypit/program-space";+import { assertProgramSpaceIdentity, programFrameSampleBoundary, programSpaceFrameCount, programSpaceSampleFrames } from "@hypit/program-space";import { canonicalize, isResourceId } from "@hypit/protocol";import { assertCanvasSpace, assertSpatialFrame } from "@hypit/spatial";import type { CanvasSpace } from "@hypit/spatial";@@ -125,7 +120,7 @@ export function assertMediaTrackSet(value: MediaTrackSet): void {function realizedItems(set: MediaTrackSet,header: MediaTrackHeader,- space: ProgramSpace,+ timeline: Timeline,canvas: CanvasSpace,layers: MediaLayerSet,frame: MediaItemProgram["frame"],@@ -135,13 +130,13 @@ function realizedItems(): MediaTrackSet {assertMediaTrackSet(set);assertMediaTrackHeader(header);- assertProgramSpaceIdentity(space);+ assertProgramSpaceIdentity(timeline);assertCanvasSpace(canvas);assertMediaLayerSet(layers);assert(layers.layers.length > 0, `Media Item ${spec.id} requires at least one layer.`);assertSpatialFrame(frame);assertMediaItemSpec(spec);- assertTemporalWindowFor(window, { subjectId: spec.id, space });+ assertTemporalWindowFor(window, { subjectId: spec.id, space: timeline });assertMediaSoundSet(sounds);assert(!sounds.sounds.some((sound) => sound.trigger.kind === "handoff"),`Media Item ${spec.id} cannot own a Handoff sound.`);@@ -177,7 +172,7 @@ function realizedItems(export function appendMediaItem(set: MediaTrackSet,header: MediaTrackHeader,- space: ProgramSpace,+ timeline: Timeline,canvas: CanvasSpace,layers: MediaLayerSet,frame: MediaItemProgram["frame"],@@ -185,29 +180,29 @@ export function appendMediaItem(sounds: MediaSoundSet,window: ProjectedWindow,): MediaTrackSet {- return realizedItems(set, header, space, canvas, layers, frame, spec, sounds, window);+ return realizedItems(set, header, timeline, canvas, layers, frame, spec, sounds, window);}-function assertSampleLayerForSpace(layer: MediaSampleLayerProgram, space: ProgramSpace, label: string): void {+function assertSampleLayerForSpace(layer: MediaSampleLayerProgram, timeline: Timeline, label: string): void {assertMediaVisualSource(layer.source, `${label}.source`);const timing = layer.source.kind === "timed" ? { frameRate: layer.source.frameRate, frameCount: layer.source.frameCount }: layer.source.kind === "surface" && layer.source.surface.timing.kind === "frames"? layer.source.surface.timing : undefined;if (timing === undefined) return;- assert(timing.frameRate.numerator === space.frameRate.numerator- && timing.frameRate.denominator === space.frameRate.denominator,- `${label} is not normalized to ProgramSpace frame rate.`);+ assert(timing.frameRate.numerator === timeline.frameRate.numerator+ && timing.frameRate.denominator === timeline.frameRate.denominator,+ `${label} is not normalized to Timeline frame rate.`);if (layer.trim !== undefined) assertMediaVisualTrim(layer.trim, timing.frameCount, `${label}.trim`);assert(layer.occupancy !== undefined, `${label} requires occupancy.`);assertMediaVisualOccupancy(layer.occupancy, `${label}.occupancy`);}-function assertItem(item: MediaItemProgram, space: ProgramSpace, label: string): void {+function assertItem(item: MediaItemProgram, timeline: Timeline, label: string): void {assertMediaIdentity(item.id, `${label}.id`);assertMediaIdentity(item.subjectId, `${label}.subjectId`);assert(Number.isSafeInteger(item.span.startFrame) && Number.isSafeInteger(item.span.endFrameExclusive)&& item.span.startFrame >= 0 && item.span.endFrameExclusive > item.span.startFrame- && item.span.endFrameExclusive <= programSpaceFrameCount(space), `${label}.span is invalid.`);+ && item.span.endFrameExclusive <= programSpaceFrameCount(timeline), `${label}.span is invalid.`);assertSpatialFrame(item.frame);assertMediaFramePresentation(item.presentation, `${label}.presentation`);assertMediaLifecycleMotion(item.motion, item.span.endFrameExclusive - item.span.startFrame, `${label}.motion`);@@ -217,7 +212,7 @@ function assertItem(item: MediaItemProgram, space: ProgramSpace, label: string):assert(item.layers.length > 0, `${label} requires layers.`);const layerSet: MediaLayerSet = { layers: item.layers };assertMediaLayerSet(layerSet);- for (const layer of item.layers) if (layer.kind === "sample") assertSampleLayerForSpace(layer, space, `${label}.${layer.id}`);+ for (const layer of item.layers) if (layer.kind === "sample") assertSampleLayerForSpace(layer, timeline, `${label}.${layer.id}`);if (item.sourceAudio !== undefined) {const selected = item.layers.find((layer) => layer.id === item.sourceAudio!.fromLayer);assert(selected?.kind === "sample" && selected.source.kind === "timed" && selected.source.audio !== undefined,@@ -230,7 +225,7 @@ function assertItem(item: MediaItemProgram, space: ProgramSpace, label: string):function appendMediaSequenceAtFrame(set: MediaTrackSet,header: MediaTrackHeader,- space: ProgramSpace,+ timeline: Timeline,canvas: CanvasSpace,members: MediaSequenceMemberSet,frame: MediaSequenceProgram["frame"],@@ -241,7 +236,7 @@ function appendMediaSequenceAtFrame(assertMediaTrackSet(set);assertMediaSequenceMemberSet(members);assertMediaSequenceSpec(spec);- const sequence = resolveMediaSequence(header, space, canvas, members, frame, spec, sounds, terminalFrame);+ const sequence = resolveMediaSequence(header, timeline, canvas, members, frame, spec, sounds, terminalFrame);assert(!set.items.some((item) => item.id === sequence.id)&& !set.sequences.some((item) => item.id === sequence.id),`Media Track ${header.id} already contains ${sequence.id}.`);@@ -251,7 +246,7 @@ function appendMediaSequenceAtFrame(export function appendMediaSequence(set: MediaTrackSet,header: MediaTrackHeader,- space: ProgramSpace,+ timeline: Timeline,canvas: CanvasSpace,members: MediaSequenceMemberSet,frame: MediaSequenceProgram["frame"],@@ -259,13 +254,13 @@ export function appendMediaSequence(sounds: MediaSoundSet,terminal: TemporalInstant,): MediaTrackSet {- assertTemporalInstantFor(terminal, { subjectId: spec.id, space });- return appendMediaSequenceAtFrame(set, header, space, canvas, members, frame, spec, sounds, terminal.frame);+ assertTemporalInstantFor(terminal, { subjectId: spec.id, space: timeline });+ return appendMediaSequenceAtFrame(set, header, timeline, canvas, members, frame, spec, sounds, terminal.frame);}-function assertSequence(sequence: MediaSequenceProgram, space: ProgramSpace, label: string): void {+function assertSequence(sequence: MediaSequenceProgram, timeline: Timeline, label: string): void {assertMediaIdentity(sequence.id, `${label}.id`);- assert(sequence.span.startFrame >= 0 && sequence.span.endFrameExclusive <= programSpaceFrameCount(space)+ assert(sequence.span.startFrame >= 0 && sequence.span.endFrameExclusive <= programSpaceFrameCount(timeline)&& sequence.span.endFrameExclusive > sequence.span.startFrame&& sequence.terminalFrame === sequence.span.endFrameExclusive, `${label}.span is invalid.`);assertSpatialFrame(sequence.frame);@@ -288,7 +283,7 @@ function assertSequence(sequence: MediaSequenceProgram, space: ProgramSpace, labconst layerSet: MediaLayerSet = { layers: member.layers };assertMediaLayerSet(layerSet);for (const layer of member.layers) if (layer.kind === "sample") {- assertSampleLayerForSpace(layer, space, `${label}.members.${index}.${layer.id}`);+ assertSampleLayerForSpace(layer, timeline, `${label}.members.${index}.${layer.id}`);}if (member.sourceAudio !== undefined) {const selected = member.layers.find((layer) => layer.id === member.sourceAudio!.fromLayer);@@ -323,13 +318,13 @@ function normalizeProgram(value: MediaTrackProgram): MediaTrackProgram {};}-export function sealMediaTrackProgram(value: MediaTrackProgram, space: ProgramSpace): MediaTrackProgram {+export function sealMediaTrackProgram(value: MediaTrackProgram, timeline: Timeline): MediaTrackProgram {const normalized = normalizeProgram(value);- assertMediaTrackProgramIdentity(normalized, space);+ assertMediaTrackProgramIdentity(normalized, timeline);return canonicalize(normalized) as unknown as MediaTrackProgram;}-export function finalizeMediaTrack(set: MediaTrackSet, header: MediaTrackHeader, space: ProgramSpace): MediaTrackProgram {+export function finalizeMediaTrack(set: MediaTrackSet, header: MediaTrackHeader, timeline: Timeline): MediaTrackProgram {assertMediaTrackSet(set);assertMediaTrackHeader(header);assert(set.items.length + set.sequences.length > 0, "Media Track requires at least one Item or Sequence.");@@ -338,7 +333,7 @@ export function finalizeMediaTrack(set: MediaTrackSet, header: MediaTrackHeader,id: header.id,items: set.items,sequences: set.sequences,- }, space);+ }, timeline);}export function assertMediaTrackProgram(value: MediaTrackProgram): void {@@ -347,26 +342,26 @@ export function assertMediaTrackProgram(value: MediaTrackProgram): void {&& value.items.length + value.sequences.length > 0, "MediaTrackProgram is empty.");}-export function assertMediaTrackProgramIdentity(value: MediaTrackProgram, space: ProgramSpace): void {+export function assertMediaTrackProgramIdentity(value: MediaTrackProgram, timeline: Timeline): void {assertMediaTrackProgram(value);- assertProgramSpaceIdentity(space);+ assertProgramSpaceIdentity(timeline);const ids = new Set<string>();for (const item of value.items) {assert(!ids.has(item.id), `MediaTrackProgram repeats ${item.id}.`);ids.add(item.id);- assertItem(item, space, `Media Item ${item.id}`);+ assertItem(item, timeline, `Media Item ${item.id}`);}for (const sequence of value.sequences) {assert(!ids.has(sequence.id), `MediaTrackProgram repeats ${sequence.id}.`);ids.add(sequence.id);- assertSequence(sequence, space, `Media Sequence ${sequence.id}`);+ assertSequence(sequence, timeline, `Media Sequence ${sequence.id}`);}}-export function projectMediaVisualTrack(space: ProgramSpace, program: MediaTrackProgram): VisualTrack {- assertMediaTrackProgramIdentity(program, space);+export function projectMediaVisualTrack(timeline: Timeline, program: MediaTrackProgram): VisualTrack {+ assertMediaTrackProgramIdentity(program, timeline);const track = sealVisualTrack({- programSpaceId: space.id,+ programSpaceId: timeline.id,visualIr: "hypit.visual-ir@1",id: program.id,presents: [@@ -375,12 +370,12 @@ export function projectMediaVisualTrack(space: ProgramSpace, program: MediaTracksubjectId: item.subjectId,span: { ...item.span },stacking: { ...item.stacking },- elements: lowerMediaItemElements(item, space),+ elements: lowerMediaItemElements(item, timeline),})),- ...program.sequences.flatMap((sequence) => lowerMediaSequencePresents(sequence, space)),+ ...program.sequences.flatMap((sequence) => lowerMediaSequencePresents(sequence, timeline)),],});- assertVisualTrackIdentity(track, space);+ assertVisualTrackIdentity(track, timeline);return track;}@@ -394,7 +389,7 @@ function sourceSampleBoundary(frame: number, frameCount: number, sampleFrames: nfunction sourceAudioClip(value: Pick<MediaItemProgram, "id" | "subjectId" | "span" | "layers" | "sourceAudio">,- space: ProgramSpace,+ timeline: Timeline,fades: { readonly inSamples: number; readonly outSamples: number } = { inSamples: 0, outSamples: 0 },): AudioClip | undefined {if (value.sourceAudio === undefined) return undefined;@@ -409,8 +404,8 @@ function sourceAudioClip(const trim = layer.trim ?? { startFrame: 0, endFrameExclusive: source.frameCount };const sourceStart = sourceSampleBoundary(trim.startFrame, source.frameCount, audio.sampleFrames);const sourceEnd = sourceSampleBoundary(trim.endFrameExclusive, source.frameCount, audio.sampleFrames);- const targetWindowStart = programFrameSampleBoundary(space, value.span.startFrame, 48_000);- const targetWindowEnd = programFrameSampleBoundary(space, value.span.endFrameExclusive, 48_000);+ const targetWindowStart = programFrameSampleBoundary(timeline, value.span.startFrame, 48_000);+ const targetWindowEnd = programFrameSampleBoundary(timeline, value.span.endFrameExclusive, 48_000);const targetFrames = value.span.endFrameExclusive - value.span.startFrame;const sourceFrames = trim.endFrameExclusive - trim.startFrame;let targetStart = targetWindowStart;@@ -423,10 +418,10 @@ function sourceAudioClip(if (layer.occupancy.mode === "once" || layer.occupancy.mode === "hold") {const playedFrames = Math.min(targetFrames, sourceFrames);if (layer.occupancy.align === "start") {- targetEnd = programFrameSampleBoundary(space, value.span.startFrame + playedFrames, 48_000);+ targetEnd = programFrameSampleBoundary(timeline, value.span.startFrame + playedFrames, 48_000);clipSourceEnd = sourceSampleBoundary(trim.startFrame + playedFrames, source.frameCount, audio.sampleFrames);} else {- targetStart = programFrameSampleBoundary(space, value.span.endFrameExclusive - playedFrames, 48_000);+ targetStart = programFrameSampleBoundary(timeline, value.span.endFrameExclusive - playedFrames, 48_000);clipSourceStart = sourceSampleBoundary(trim.endFrameExclusive - playedFrames, source.frameCount, audio.sampleFrames);}} else if (layer.occupancy.mode === "loop") {@@ -466,12 +461,12 @@ function soundClip(source: MediaSoundSource,gain: number,point: number,- space: ProgramSpace,+ timeline: Timeline,): AudioClip {- const total = programSpaceSampleFrames(space, 48_000);- const targetStart = programFrameSampleBoundary(space, point, 48_000);+ const total = programSpaceSampleFrames(timeline, 48_000);+ const targetStart = programFrameSampleBoundary(timeline, point, 48_000);const audible = Math.min(source.sampleFrames, Math.max(0, total - targetStart));- assert(audible > 0, `Media sound ${id} begins outside ProgramSpace.`);+ assert(audible > 0, `Media sound ${id} begins outside Timeline.`);return {id,artifact: structuredClone(source.artifact),@@ -493,18 +488,18 @@ function soundClip(function edgeSoundClips(value: Pick<MediaItemProgram, "id" | "span" | "motion" | "sounds">,- space: ProgramSpace,+ timeline: Timeline,): AudioClip[] {return value.sounds.flatMap((sound) => {if (sound.trigger.kind === "handoff") return [];const point = sound.trigger.kind === "enter"? value.span.startFrame: value.span.endFrameExclusive - (value.motion.exit?.durationFrames ?? 0);- return [soundClip(`${value.id}:${sound.id}`, sound.source, sound.gain, point, space)];+ return [soundClip(`${value.id}:${sound.id}`, sound.source, sound.gain, point, timeline)];});}-function sequenceAudioClips(sequence: MediaSequenceProgram, space: ProgramSpace): AudioClip[] {+function sequenceAudioClips(sequence: MediaSequenceProgram, timeline: Timeline): AudioClip[] {const sourceClips = sequence.members.flatMap((member, index) => {if (member.sourceAudio === undefined) return [];const incoming = sequence.handoffs[index - 1];@@ -515,12 +510,12 @@ function sequenceAudioClips(sequence: MediaSequenceProgram, space: ProgramSpace)? outgoing.span.endFrameExclusive : member.logicalSpan.endFrameExclusive,};const fadeInSamples = incoming?.audio === "crossfade"- ? programFrameSampleBoundary(space, incoming.span.endFrameExclusive, 48_000)- - programFrameSampleBoundary(space, incoming.span.startFrame, 48_000)+ ? programFrameSampleBoundary(timeline, incoming.span.endFrameExclusive, 48_000)+ - programFrameSampleBoundary(timeline, incoming.span.startFrame, 48_000): 0;const fadeOutSamples = outgoing?.audio === "crossfade"- ? programFrameSampleBoundary(space, outgoing.span.endFrameExclusive, 48_000)- - programFrameSampleBoundary(space, outgoing.span.startFrame, 48_000)+ ? programFrameSampleBoundary(timeline, outgoing.span.endFrameExclusive, 48_000)+ - programFrameSampleBoundary(timeline, outgoing.span.startFrame, 48_000): 0;const clip = sourceAudioClip({id: `${sequence.id}:${member.id}`,@@ -528,7 +523,7 @@ function sequenceAudioClips(sequence: MediaSequenceProgram, space: ProgramSpace)span,layers: member.layers,sourceAudio: member.sourceAudio,- }, space, { inSamples: fadeInSamples, outSamples: fadeOutSamples });+ }, timeline, { inSamples: fadeInSamples, outSamples: fadeOutSamples });return clip === undefined ? [] : [clip];});const handoffSounds = sequence.sounds.flatMap((sound) => {@@ -537,7 +532,7 @@ function sequenceAudioClips(sequence: MediaSequenceProgram, space: ProgramSpace)const handoff = sequence.handoffs.find((candidate) => candidate.id === handoffId)!;return [soundClip(`${sequence.id}:${sound.id}`, sound.source, sound.gain,- sequence.members.find((member) => member.id === handoff.toMemberId)!.activationFrame, space,+ sequence.members.find((member) => member.id === handoff.toMemberId)!.activationFrame, timeline,)];});return [@@ -547,19 +542,19 @@ function sequenceAudioClips(sequence: MediaSequenceProgram, space: ProgramSpace)span: sequence.span,motion: sequence.motion,sounds: sequence.sounds,- }, space),+ }, timeline),...handoffSounds,];}-export function projectMediaAudioTrack(space: ProgramSpace, program: MediaTrackProgram): AudioTrack {- assertMediaTrackProgramIdentity(program, space);+export function projectMediaAudioTrack(timeline: Timeline, program: MediaTrackProgram): AudioTrack {+ assertMediaTrackProgramIdentity(program, timeline);const clips = program.items.flatMap((item) => {- const source = sourceAudioClip(item, space);- return [...(source === undefined ? [] : [source]), ...edgeSoundClips(item, space)];- }).concat(program.sequences.flatMap((sequence) => sequenceAudioClips(sequence, space)));+ const source = sourceAudioClip(item, timeline);+ return [...(source === undefined ? [] : [source]), ...edgeSoundClips(item, timeline)];+ }).concat(program.sequences.flatMap((sequence) => sequenceAudioClips(sequence, timeline)));assert(clips.length > 0, `Media Program ${program.id} has no explicitly authored audio projection.`);- const track = sealAudioTrack({ programSpaceId: space.id, id: `${program.id}:audio`, clips });- assertAudioTrackIdentity(track, space);+ const track = sealAudioTrack({ programSpaceId: timeline.id, id: `${program.id}:audio`, clips });+ assertAudioTrackIdentity(track, timeline);return track;}diff --git a/packages/media-track/src/sampling.ts b/packages/media-track/src/sampling.tsindex 74a573ee..bd8036c5 100644--- a/packages/media-track/src/sampling.ts+++ b/packages/media-track/src/sampling.ts@@ -1,10 +1,10 @@+import type { Timeline } from "@hypit/timeline";import type {VisualSamplingRational,VisualSamplingSegment,VisualTimedSampling,} from "@hypit/composition";import { assertProgramSpaceIdentity } from "@hypit/program-space";-import type { ProgramSpace } from "@hypit/program-space";import type { MediaVisualOccupancy, MediaVisualTrim } from "./types.js";@@ -58,21 +58,21 @@ function assertOccupancy(value: MediaVisualOccupancy): void {* frame. This avoids a second, renderer-specific frame quantizer.*/export function resolveVisualSampling(input: {- readonly space: ProgramSpace;+ readonly timeline: Timeline;readonly sourceFrameRate: { readonly numerator: number; readonly denominator: number };readonly sourceFrameCount: number;readonly targetFrameCount: number;readonly trim?: MediaVisualTrim;readonly occupancy: MediaVisualOccupancy;}): VisualTimedSampling {- assertProgramSpaceIdentity(input.space);+ assertProgramSpaceIdentity(input.timeline);assert(Number.isSafeInteger(input.sourceFrameCount) && input.sourceFrameCount > 0,"Timed visual source frame count is invalid.");assert(Number.isSafeInteger(input.targetFrameCount) && input.targetFrameCount > 0,"Timed visual target frame count is invalid.");- assert(input.sourceFrameRate.numerator === input.space.frameRate.numerator- && input.sourceFrameRate.denominator === input.space.frameRate.denominator,- "Timed visual source must be normalized to ProgramSpace frame rate before Media authoring.");+ assert(input.sourceFrameRate.numerator === input.timeline.frameRate.numerator+ && input.sourceFrameRate.denominator === input.timeline.frameRate.denominator,+ "Timed visual source must be normalized to Timeline frame rate before Media authoring.");assertOccupancy(input.occupancy);const trim = input.trim ?? { startFrame: 0, endFrameExclusive: input.sourceFrameCount };assert(Number.isSafeInteger(trim.startFrame) && Number.isSafeInteger(trim.endFrameExclusive)diff --git a/packages/media-track/src/sequence.ts b/packages/media-track/src/sequence.tsindex eed17c7c..ecfcdbb2 100644--- a/packages/media-track/src/sequence.ts+++ b/packages/media-track/src/sequence.ts@@ -1,3 +1,4 @@+import type { Timeline } from "@hypit/timeline";import type {VisualAnimation,VisualEasing,@@ -5,7 +6,6 @@ import type {VisualStyleDeclaration,} from "@hypit/composition";import { assertProgramSpaceIdentity, programSpaceFrameCount } from "@hypit/program-space";-import type { ProgramSpace } from "@hypit/program-space";import { canonicalize } from "@hypit/protocol";import { assertCanvasSpace, assertSpatialFrame } from "@hypit/spatial";import type { CanvasSpace } from "@hypit/spatial";@@ -109,12 +109,12 @@ function appendMediaSequenceMemberAtFrame(export function appendMediaSequenceMember(set: MediaSequenceMemberSet,- space: ProgramSpace,+ timeline: Timeline,layers: MediaLayerSet,spec: MediaSequenceMemberSpec,activation: TemporalInstant,): MediaSequenceMemberSet {- assertTemporalInstantFor(activation, { subjectId: spec.id, space });+ assertTemporalInstantFor(activation, { subjectId: spec.id, space: timeline });return appendMediaSequenceMemberAtFrame(set, layers, spec, activation.frame);}@@ -180,7 +180,7 @@ function resolvedHandoff(export function resolveMediaSequence(header: MediaTrackHeader,- space: ProgramSpace,+ timeline: Timeline,canvas: CanvasSpace,memberSet: MediaSequenceMemberSet,frame: MediaSequenceProgram["frame"],@@ -188,7 +188,7 @@ export function resolveMediaSequence(sounds: MediaSoundSet,terminalFrame: number,): MediaSequenceProgram {- assertProgramSpaceIdentity(space);+ assertProgramSpaceIdentity(timeline);assertCanvasSpace(canvas);assertMediaSequenceMemberSet(memberSet);assertMediaSequenceSpec(spec);@@ -205,7 +205,7 @@ export function resolveMediaSequence(terminalFrame,triggers: memberSet.members.map((member) => ({ id: member.id, frame: member.activationFrame })),});- assert(terminalFrame <= programSpaceFrameCount(space), `Media Sequence ${spec.id} exits ProgramSpace.`);+ assert(terminalFrame <= programSpaceFrameCount(timeline), `Media Sequence ${spec.id} exits Timeline.`);const handoffs = spec.handoffs.map((handoff, index) => {const from = memberSet.members[index]!;const to = memberSet.members[index + 1]!;@@ -360,7 +360,7 @@ export function sequenceMemberHandoffAnimation(export function lowerMediaSequencePresents(sequence: MediaSequenceProgram,- space: ProgramSpace,+ timeline: Timeline,): readonly VisualPresent[] {return sequence.members.map((member, index) => {const incoming = sequence.handoffs[index - 1];@@ -389,7 +389,7 @@ export function lowerMediaSequencePresents(subjectId: sequence.id,span: { ...item.span },stacking: { ...item.stacking },- elements: lowerMediaItemElements(item, space, {+ elements: lowerMediaItemElements(item, timeline, {includeHandoffWrapper: true,...(handoffAnimation === undefined ? {} : { handoffAnimation }),lifecycleAnimationOverride: lifecycleAnimationWindow(diff --git a/packages/media-track/src/surface.ts b/packages/media-track/src/surface.tsindex bfbbbda2..ffc8aa44 100644--- a/packages/media-track/src/surface.ts+++ b/packages/media-track/src/surface.ts@@ -1,4 +1,4 @@-import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";+import { resolveTemporalContext } from "@hypit/temporal-markup";import {assertEmptyElement as empty,assertAttributes as allowed,@@ -85,7 +85,6 @@ type FragmentLayer =type FragmentSound = { readonly sourceName: string; readonly specName: string };type FragmentItem = {- readonly performance?: { readonly fitName: string; readonly sampleName: string };readonly suffix: string;readonly windowName: string;readonly frameName: string;@@ -167,11 +166,11 @@ function createMediaTrackSurfaceFragment(inputTypes: readonly { readonly name: sspec = operation(bindId);}const common = {- set: operation(set), header: input("header"), space: input("space"), canvas: input("canvas"),+ set: operation(set), header: input("header"), timeline: input("timeline"), canvas: input("canvas"),layers: operation(layers), frame: input(item.frameName), spec, sounds: operation(sounds), window: input(item.windowName),};- operations.push({ id: appendId, producer: item.performance === undefined ? mediaTrackProducers.appendItem : mediaTrackProducers.appendPerformance,- inputs: { ...common, ...(item.performance === undefined ? {} : { semantic: input("semantic"), fit: input(item.performance.fitName), sampleSpec: input(item.performance.sampleName) }) }, result: { kind: "output", name: "set" } });+ operations.push({ id: appendId, producer: mediaTrackProducers.appendItem,+ inputs: common, result: { kind: "output", name: "set" } });set = appendId;}for (const sequence of sequences) {@@ -184,7 +183,7 @@ function createMediaTrackSurfaceFragment(inputTypes: readonly { readonly name: sconst layers = appendLayers(operations, memberPrefix, member.layers);const appendId = `${memberPrefix}:append`;operations.push({ id: appendId, producer: mediaTrackProducers.appendMember, inputs: {- members: operation(members), space: input("space"), layers: operation(layers), spec: input(member.specName), activation: input(member.instantName),+ members: operation(members), timeline: input("timeline"), layers: operation(layers), spec: input(member.specName), activation: input(member.instantName),}, result: { kind: "output", name: "members" } });members = appendId;}@@ -199,7 +198,7 @@ function createMediaTrackSurfaceFragment(inputTypes: readonly { readonly name: sspec = operation(bindId);}const common = {- set: operation(set), header: input("header"), space: input("space"), canvas: input("canvas"),+ set: operation(set), header: input("header"), timeline: input("timeline"), canvas: input("canvas"),members: operation(members), frame: input(sequence.frameName), spec, sounds: operation(sounds),};operations.push({ id: appendId, producer: mediaTrackProducers.appendSequence,@@ -207,10 +206,10 @@ function createMediaTrackSurfaceFragment(inputTypes: readonly { readonly name: sset = appendId;}operations.push(- { id: "track:finalize", producer: mediaTrackProducers.finalize, inputs: { set: operation(set), header: input("header"), space: input("space") }, result: { kind: "output", name: "program" } },- { id: "track:visual", producer: mediaTrackProducers.projectVisual, inputs: { space: input("space"), program: operation("track:finalize") }, result: { kind: "output", name: "track" } },+ { id: "track:finalize", producer: mediaTrackProducers.finalize, inputs: { set: operation(set), header: input("header"), timeline: input("timeline") }, result: { kind: "output", name: "program" } },+ { id: "track:visual", producer: mediaTrackProducers.projectVisual, inputs: { timeline: input("timeline"), program: operation("track:finalize") }, result: { kind: "output", name: "track" } },);- if (audio) operations.push({ id: "track:audio", producer: mediaTrackProducers.projectAudio, inputs: { space: input("space"), program: operation("track:finalize") }, result: { kind: "output", name: "track" } });+ if (audio) operations.push({ id: "track:audio", producer: mediaTrackProducers.projectAudio, inputs: { timeline: input("timeline"), program: operation("track:finalize") }, result: { kind: "output", name: "track" } });return sealGraphFragment({inputs: inputTypes,operations,@@ -495,39 +494,33 @@ function sound(}export const decodeMediaTrackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {- allowed(element, ["id", "semantic", "space", "canvas"]);+ allowed(element, ["id", "timeline", "canvas"]);const trackId = text(element, "id");const state = builder();const context = resolveTemporalContext({ element, resolveReference });- const time = createTemporalSpace({ id: trackId, element, ...context });- state.addReference("space", time.space);+ state.addReference("timeline", context.timeline);state.addReference("canvas", reference(element.attributes.canvas, `${element.name}.canvas`, spatialTypes.canvas, resolveReference));const headerId = `${trackId}.header`;state.addRecord("header", headerId, mediaTrackTypes.header,sealMediaTrackHeader({ id: trackId }), element.range);const items: FragmentItem[] = [];const sequences: FragmentSequence[] = [];- const temporalComponents: SurfaceComponentDraft[] = [...time.components];- const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [...time.fragments];+ const temporalComponents: SurfaceComponentDraft[] = [];+ const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [];let itemIndex = 0;let sequenceIndex = 0;let hasAudio = false;for (const child of element.children) {if (child.kind === "text") {- if (child.value.trim().length > 0) throw new Error(`${element.name} accepts only Performance, Item and Sequence children.`);+ if (child.value.trim().length > 0) throw new Error(`${element.name} accepts only Item and Sequence children.`);continue;}- const isPerformance = child.name.endsWith(":Performance") || child.name === "Performance";- if (isPerformance) {- if (context.semantic === undefined) throw new Error(`${child.name} requires semantic performance input.`);- if (state.inputs.semantic === undefined) state.addReference("semantic", context.semantic);- }- if (isPerformance || child.name.endsWith(":Item") || child.name === "Item") {+ if (child.name.endsWith(":Item") || child.name === "Item") {itemIndex += 1;const itemSuffix = suffix(itemIndex);allowed(child, ["id", "frame", "appearance", "motion",- ...(isPerformance ? [] : ["image", "media", "surface", "extent", "source-audio", "audio-gain"]),+ "image", "media", "surface", "extent", "source-audio", "audio-gain","clip", ...temporalWindowAttributeNames,]);const id = optionalText(child, "id") ?? `${trackId}.item.${itemSuffix}`;@@ -539,23 +532,12 @@ export const decodeMediaTrackSurface: StructuredSurfaceHandler = ({ element, res}const motionRecipe = child.attributes.motion === undefined ? undefined: recipe(reference(child.attributes.motion, `${child.name}.motion`, svsRecipeType, resolveReference), `${child.name}.motion`);- const temporal = createTemporalWindowProjection({ id, element: child, ...context, space: time.space, resolveReference });+ const temporal = createTemporalWindowProjection({ id, element: child, ...context, resolveReference });state.records.push(...temporal.records); temporalComponents.push(...temporal.components); temporalFragments.push(...temporal.fragments);- const performance = isPerformance ? {- fitName: `item-${itemSuffix}-fit`, sampleName: `item-${itemSuffix}-sample`,- } : undefined;- const directSource = isPerformance ? undefined : declaredVisualSource(child, resolveReference);- validateUnitChildren(child, isPerformance || directSource !== undefined, true);- if (performance !== undefined) {- state.addRecord(performance.fitName, `${id}.fit`, spatialTypes.fit, decodeMediaFit(appearance), child.range);- if (appearance.properties.playback !== undefined || appearance.properties["trim-start"] !== undefined || appearance.properties["trim-end"] !== undefined) {- throw new Error(`${child.name} follows the performance's source time. Select its interval with during or start/end.`);- }- state.addRecord(performance.sampleName, `${id}.sample`, mediaTrackTypes.sampleLayerSpec,- decodeMediaSampleSpec(appearance, "content", "timed", decodeSamplingMotion(child, true), true), child.range);- }+ const directSource = declaredVisualSource(child, resolveReference);+ validateUnitChildren(child, directSource !== undefined, true);const layers = unitLayers(state, { trackId, unitSuffix: `item-${itemSuffix}`, element: child, appearance,- ...(directSource === undefined ? {} : { directSource }), allowFramePaint: true, allowEmpty: isPerformance }, resolveReference);+ ...(directSource === undefined ? {} : { directSource }), allowFramePaint: true, allowEmpty: false }, resolveReference);const selectedAudio = sourceAudio(child);const specName = `item-${itemSuffix}-spec`;const windowName = `item-${itemSuffix}-window`;@@ -572,12 +554,12 @@ export const decodeMediaTrackSurface: StructuredSurfaceHandler = ({ element, res}}if (selectedAudio !== undefined || fragmentSounds.length > 0) hasAudio = true;- items.push({ suffix: itemSuffix, windowName, frameName, specName, ...(performance === undefined ? {} : { performance }),+ items.push({ suffix: itemSuffix, windowName, frameName, specName,...(clipPathName === undefined ? {} : { clipPathName }), layers, sounds: fragmentSounds });continue;}if (!(child.name.endsWith(":Sequence") || child.name === "Sequence")) {- throw new Error(`${element.name} accepts only Performance, Item and Sequence children.`);+ throw new Error(`${element.name} accepts only Item and Sequence children.`);}sequenceIndex += 1;const sequenceSuffix = suffix(sequenceIndex);@@ -611,7 +593,7 @@ export const decodeMediaTrackSurface: StructuredSurfaceHandler = ({ element, resfor (const [index, member] of memberElements.entries()) {const memberSuffix = suffix(index + 1);allowed(member, ["id", "image", "media", "surface", "extent", "appearance", ...temporalInstantAttributeNames, "source-audio", "audio-gain"]);- const temporal = createTemporalInstantProjection({ id: `${id}.${memberIds[index]}`, subjectId: memberIds[index]!, element: member, ...context, space: time.space, resolveReference });+ const temporal = createTemporalInstantProjection({ id: `${id}.${memberIds[index]}`, subjectId: memberIds[index]!, element: member, ...context, resolveReference });state.records.push(...temporal.records); temporalComponents.push(...temporal.components); temporalFragments.push(...temporal.fragments);const memberAppearance = member.attributes.appearance === undefined ? appearance: recipe(reference(member.attributes.appearance, `${member.name}.appearance`, svsRecipeType, resolveReference), `${member.name}.appearance`);@@ -641,7 +623,7 @@ export const decodeMediaTrackSurface: StructuredSurfaceHandler = ({ element, res}if (sequenceSounds.length > 0) hasAudio = true;const terminalTemporal = createTemporalInstantProjection({- id: `${id}.terminal`, subjectId: id, element: child, ...context, space: time.space, resolveReference,+ id: `${id}.terminal`, subjectId: id, element: child, ...context, resolveReference,semanticAttribute: "until", boundaryAttribute: "until-boundary", projectedAttribute: false,boundaryFallback: "end",});@@ -656,7 +638,7 @@ export const decodeMediaTrackSurface: StructuredSurfaceHandler = ({ element, ressequences.push({ suffix: sequenceSuffix, terminalName, frameName, specName,...(clipPathName === undefined ? {} : { clipPathName }), members, sounds: sequenceSounds });}- if (items.length === 0 && sequences.length === 0) throw new Error(`${element.name} requires at least one Performance, Item or Sequence.`);+ if (items.length === 0 && sequences.length === 0) throw new Error(`${element.name} requires at least one Item or Sequence.`);const fragment = createMediaTrackSurfaceFragment(state.inputTypes, items, sequences, hasAudio);return {records: state.records,diff --git a/packages/media-track/test/media-track.test.ts b/packages/media-track/test/media-track.test.tsindex 8e6d5f32..e14c10ff 100644--- a/packages/media-track/test/media-track.test.ts+++ b/packages/media-track/test/media-track.test.ts@@ -1,7 +1,9 @@+import { appendMediaPerformance } from "@hypit/performance";+import { sealTimeline } from "@hypit/timeline";import assert from "node:assert/strict";import test from "node:test";import { fixtureResource } from "../../../test/fixture-resource.js";-import { semanticTrackFixture } from "../../../test/semantic-track-fixture.js";+import { timelineFixture } from "../../../test/timeline-fixture.js";import { projectMomentWindow, projectProgramWindow, projectSegmentWindow, projectSelectionWindow } from "../../../test/temporal-fixture.js";import type { TemporalWindowProjection } from "../../../test/temporal-fixture.js";@@ -13,7 +15,6 @@ import type { CompositableSurfaceRef, SynchronizedMedia } from "@hypit/media";import { artifactTypes } from "@hypit/artifact";import { narrativeTypes } from "@hypit/narrative";import {- appendMediaPerformance,appendMediaPaintLayer,appendMediaSound,appendMediaItem,@@ -57,14 +58,14 @@ import type { NarrativeMomentRef, NarrativeSelectionRef } from "@hypit/narrativeimport { sealProgramSpace } from "@hypit/program-space";import { programSpaceTypes } from "@hypit/program-space";import type { BlobRef } from "@hypit/protocol";-import { semanticTrackTypes } from "@hypit/semantic-track";+import { timelineTypes } from "@hypit/timeline";import { spatialTypes } from "@hypit/spatial";import { svsRecipeType } from "@hypit/svs";import type { SvsRecipe } from "@hypit/svs";import type { StructuredElement, StructuredNode, SurfaceResolvedReference, MarkupAttributeValue } from "@hypit/markup";import type { TemporalWindow } from "@hypit/temporal";-const space = sealProgramSpace({ id: "test-space", durationSec: 4,+const space = sealTimeline({ items: [], id: "test-space", durationSec: 4,frameRate: { numerator: 30, denominator: 1 },});const canvas = {@@ -76,7 +77,7 @@ const canvas = {pixelAspect: "square" as const,};const header = sealMediaTrackHeader({ id: "proof" });-const semantic = semanticTrackFixture(space, {+const semantic = timelineFixture(space, {segments: [{ id: "opening", frameCount: 30 },{ id: "answer", frameCount: 60 },@@ -133,44 +134,44 @@ function itemSpec(overrides: Partial<TestMediaItemSpec> = {}): TestMediaItemSpec}function appendProgramMediaItem(- set: ReturnType<typeof createMediaTrackSet>, trackHeader: typeof header, semanticTrack: typeof semantic,+ set: ReturnType<typeof createMediaTrackSet>, trackHeader: typeof header, timeline: typeof semantic,canvasSpace: typeof canvas, layers: MediaLayerSet, frameValue: typeof frame, authored: MediaItemSpec | TestMediaItemSpec,sounds: ReturnType<typeof createMediaSoundSet>,) {const projection = "projection" in authored ? authored.projection : defaultItemProjection;const { projection: _ignored, ...spec } = authored as TestMediaItemSpec;return appendMediaItem(set, trackHeader, space, canvasSpace, layers, frameValue, spec, sounds,- projectProgramWindow({ itemId: spec.id, semantic: semanticTrack, projection }));+ projectProgramWindow({ itemId: spec.id, semantic: timeline, projection }));}function appendSelectionMediaItem(- set: ReturnType<typeof createMediaTrackSet>, trackHeader: typeof header, semanticTrack: typeof semantic,+ set: ReturnType<typeof createMediaTrackSet>, trackHeader: typeof header, timeline: typeof semantic,canvasSpace: typeof canvas, layers: MediaLayerSet, frameValue: typeof frame, selection: NarrativeSelectionRef,authored: TestMediaItemSpec, sounds: ReturnType<typeof createMediaSoundSet>,) {const { projection, ...spec } = authored;return appendMediaItem(set, trackHeader, space, canvasSpace, layers, frameValue, spec, sounds,- projectSelectionWindow({ itemId: spec.id, semantic: semanticTrack, selection, projection }));+ projectSelectionWindow({ itemId: spec.id, semantic: timeline, selection, projection }));}function appendSegmentMediaItem(- set: ReturnType<typeof createMediaTrackSet>, trackHeader: typeof header, semanticTrack: typeof semantic,+ set: ReturnType<typeof createMediaTrackSet>, trackHeader: typeof header, timeline: typeof semantic,canvasSpace: typeof canvas, layers: MediaLayerSet, frameValue: typeof frame, segment: { narrativeId: string; kind: "segment"; id: string; tokenStart: number; tokenEndExclusive: number },authored: TestMediaItemSpec, sounds: ReturnType<typeof createMediaSoundSet>,) {const { projection, ...spec } = authored;return appendMediaItem(set, trackHeader, space, canvasSpace, layers, frameValue, spec, sounds,- projectSegmentWindow({ itemId: spec.id, semantic: semanticTrack, segment, projection }));+ projectSegmentWindow({ itemId: spec.id, semantic: timeline, segment, projection }));}function appendMomentMediaItem(- set: ReturnType<typeof createMediaTrackSet>, trackHeader: typeof header, semanticTrack: typeof semantic,+ set: ReturnType<typeof createMediaTrackSet>, trackHeader: typeof header, timeline: typeof semantic,canvasSpace: typeof canvas, layers: MediaLayerSet, frameValue: typeof frame, moment: NarrativeMomentRef,authored: TestMediaItemSpec, sounds: ReturnType<typeof createMediaSoundSet>,) {const { projection, ...spec } = authored;return appendMediaItem(set, trackHeader, space, canvasSpace, layers, frameValue, spec, sounds,- projectMomentWindow({ itemId: spec.id, semantic: semanticTrack, moment, projection }));+ projectMomentWindow({ itemId: spec.id, semantic: timeline, moment, projection }));}function stillLayers(): MediaLayerSet {@@ -692,20 +693,20 @@ test("visual-only normalized video never creates an implicit audio branch", () =test("timed visual occupancy resolves every alignment into exact source-frame segments", () => {const sampling = (occupancy: Parameters<typeof resolveVisualSampling>[0]["occupancy"]) =>- resolveVisualSampling({ space, sourceFrameRate: { numerator: 30, denominator: 1 }, sourceFrameCount: 30, targetFrameCount: 50, occupancy });+ resolveVisualSampling({ timeline: space, sourceFrameRate: { numerator: 30, denominator: 1 }, sourceFrameCount: 30, targetFrameCount: 50, occupancy });assert.deepEqual(sampling({ mode: "once", align: "start" }).segments.map((item) => item.target), [{ startFrame: 0, endFrameExclusive: 30 }]);assert.deepEqual(sampling({ mode: "once", align: "end" }).segments.map((item) => item.target), [{ startFrame: 20, endFrameExclusive: 50 }]);assert.deepEqual(sampling({ mode: "loop", align: "end" }).segments[0]?.sourceFrame, { numerator: 10, denominator: 1 });assert.deepEqual(sampling({ mode: "stretch" }).segments[0]?.rate, { numerator: 3, denominator: 5 });assert.throws(() => resolveVisualSampling({- space, sourceFrameRate: { numerator: 24, denominator: 1 }, sourceFrameCount: 24, targetFrameCount: 30,+ timeline: space, sourceFrameRate: { numerator: 24, denominator: 1 }, sourceFrameCount: 24, targetFrameCount: 30,occupancy: { mode: "once", align: "start" },- }), /normalized to ProgramSpace frame rate/u);+ }), /normalized to Timeline frame rate/u);});test("the graph keeps every source, extent, fit, frame, time and appearance input explicit", () => {assert.deepEqual(stillMediaTrackFragment.inputs.map((input) => input.name), [- "canvas", "extent", "fit", "frame", "header", "item-spec", "sample-spec", "source", "space", "window",+ "canvas", "extent", "fit", "frame", "header", "item-spec", "sample-spec", "source", "timeline", "window",]);});@@ -726,7 +727,7 @@ test("the Media author Surface emits explicit graph edges for layers, semantic t});const references = new Map<string, SurfaceResolvedReference>([["canvas", plain("canvas", spatialTypes.canvas)],- ["semantic", plain("semantic", semanticTrackTypes.track)],+ ["semantic", plain("semantic", timelineTypes.track)],["frame", plain("frame", spatialTypes.frame)],["clip-path", plain("clip-path", spatialTypes.path)],["still", plain("still", artifactTypes.blob)],@@ -748,7 +749,7 @@ test("the Media author Surface emits explicit graph edges for layers, semantic t["motion", appearance("motion", { enter: "slide", "enter-frames": 12, "enter-easing": "ease-out", "enter-direction": "up", "enter-origin": "outside-canvas" })],["handoff", appearance("handoff", { operator: "crossfade", "duration-frames": 10, "boundary-ratio": 0.5, audio: "cut" })],]);- const root = node("media:Track", { id: "editorial", semantic: ref("semantic"), canvas: ref("canvas") }, [+ const root = node("media:Track", { id: "editorial", timeline: ref("semantic"), canvas: ref("canvas") }, [node("media:Item", { id: "still-card", image: ref("still"), extent: ref("extent"), frame: ref("frame"), clip: ref("clip-path"), appearance: ref("still-style"), during: "program" }),node("media:Item", { id: "segment-card", image: ref("still"), extent: ref("extent"), frame: ref("frame"), appearance: ref("still-style"), during: ref("answer-segment") }),node("media:Item", { id: "proof", frame: ref("frame"), appearance: ref("card-style"), motion: ref("motion"), during: ref("selection"), "source-audio": "video", "audio-gain": "0.8" }, [@@ -759,12 +760,12 @@ test("the Media author Surface emits explicit graph edges for layers, semantic t]),node("media:Sound", { id: "proof-enter", source: ref("sfx"), at: "enter", gain: "0.5" }),]),- node("media:Performance", { id: "presenter", frame: ref("frame"), clip: ref("clip-path"), appearance: ref("still-style"), motion: ref("motion"), during: ref("answer-segment") }, [+ node("media:Item", { image: ref("still"), extent: ref("extent"), id: "presenter", frame: ref("frame"), clip: ref("clip-path"), appearance: ref("still-style"), motion: ref("motion"), during: ref("answer-segment") }, [node("media:Sampling", { at: "start", zoom: "1" }),node("media:Sampling", { at: "end", zoom: "1.08", x: "6" }),node("media:Sound", { id: "presenter-enter", source: ref("sfx"), at: "enter" }),]),- node("media:Performance", { id: "inset", frame: ref("frame"), appearance: ref("still-style"), at: "12f", for: "2s" }),+ node("media:Item", { image: ref("still"), extent: ref("extent"), id: "inset", frame: ref("frame"), appearance: ref("still-style"), at: "12f", for: "2s" }),node("media:Sequence", { id: "steps", frame: ref("frame"), appearance: ref("sequence-style"), until: ref("terminal"), "until-boundary": "end" }, [node("media:Member", { id: "one", image: ref("still"), extent: ref("extent"), at: ref("cue1") }),node("media:Member", { id: "two", surface: ref("surface"), appearance: ref("surface-style"), at: ref("cue2") }),@@ -785,10 +786,6 @@ test("the Media author Surface emits explicit graph edges for layers, semantic tconst producers = fragment.operations.map((entry) => entry.producer.name);assert.ok(producers.includes("append-still-media-layer"));assert.ok(producers.includes("append-media-item"));- const performanceOperation = fragment.operations.find(entry => entry.producer.name === "append-media-performance");- assert.deepEqual(performanceOperation?.inputs.semantic, { kind: "fragment-input", name: "semantic" });- assert.equal(fragment.inputs.filter(entry => entry.type.name === semanticTrackTypes.track.name).length, 1);- assert.equal(fragment.operations.filter(entry => entry.producer.name === "append-media-performance").length, 2);assert.ok(producers.includes("append-timed-media-layer"));assert.ok(producers.includes("append-surface-media-layer"));assert.ok(producers.includes("append-media-paint-layer"));@@ -811,14 +808,6 @@ test("the Media author Surface emits explicit graph edges for layers, semantic tassert.ok(fragment.inputs.some((entry) => entry.type.name === spatialTypes.path.name));const itemSpecs = result.records.filter((entry) => entry.type.name === "MediaItemSpec");assert.equal(itemSpecs.length, 5);- const performanceSample = result.records.find(entry => entry.id === "presenter.sample");- assert.ok(performanceSample?.value.kind === "inline");- assert.deepEqual((performanceSample.value.value as unknown as { samplingMotion: unknown }).samplingMotion, {- keyframes: [- { atProgress: 0, zoom: 1, offsetX: 0, offsetY: 0, rotationDeg: 0 },- { atProgress: 1, zoom: 1.08, offsetX: 6, offsetY: 0, rotationDeg: 0 },- ],- });const selected = itemSpecs[2]!.value.kind === "inline"? itemSpecs[2]!.value.value as unknown as MediaItemSpec: undefined;@@ -997,15 +986,15 @@ test("Sequence fails atomically on malformed order, topology, envelope and threetest("Performance keeps source positions and continuous picture motion across Takes", () => {- const performance = { ...semantic, items: semantic.items.map(item => ({ take: { ...item.take,+ const performance = { ...semantic, items: semantic.items.map(item => ({ startFrame: item.startFrame, take: { ...item.take,media: { ...item.take.media, visual: { artifact: { ...source, mediaType: "video/mp4" }, width: 720, height: 1280 } },} })) };const window = projectProgramWindow({ itemId: "performance", semantic: performance,projection: { start: { ref: "absolute", at: { unit: "frames", value: 20 } },end: { ref: "absolute", at: { unit: "frames", value: 100 } } } });- const set = appendMediaPerformance({ set: createMediaTrackSet(), header, space, canvas, frame,+ const set = appendMediaPerformance({ set: createMediaTrackSet(), header, timeline: performance, canvas, frame,layers: createMediaLayerSet(), spec: itemSpec({ id: "performance" }), sounds: createMediaSoundSet(), window },- performance, fit, sealMediaSampleLayerSpec({ id: "content", appearance, occupancy: { mode: "once", align: "start" },+ fit, sealMediaSampleLayerSpec({ id: "content", appearance, occupancy: { mode: "once", align: "start" },samplingMotion: { keyframes: [{ atProgress: 0, zoom: 1, offsetX: 0, offsetY: 0, rotationDeg: 0 },{ atProgress: 1, zoom: 1.08, offsetX: 0, offsetY: -18, rotationDeg: 0 },diff --git a/packages/narrative/README.md b/packages/narrative/README.mdindex 79b8f41f..2c362f6f 100644--- a/packages/narrative/README.md+++ b/packages/narrative/README.md@@ -4,4 +4,26 @@ External components use `@hypit/hypit/narrative` from their `@hypit/hypit` develowns the types and helpers below; Source imports retain the `@hypit/narrative@1` Module identity.-Public, provider-neutral contracts for authored speech and semantic selections.+Public, provider-neutral values for complete authored content and its semantic references.++A `Narrative` owns speech Tokens, Segments, Turns, semantic Anchors, Selections and Moments, plus+its `caption: CaptionDocument`. The document supplies display Words, speech/display correspondence+and Cue breaks. These are authored relationships; frame timing and visual Styles belong downstream.+Script exports the complete value as `story` and the same document as the narrow `story.caption`+Record. A third-party author package can produce the same values without importing the Script parser.++## Query content through semantic references++- `narrativeAnchorTokenBoundary(narrative, anchorId)` resolves a boundary in authored token order.+- `narrativeSelectionTokenRange(narrative, selection)` returns the half-open authored token range.+- `narrativeTokensForSelection(narrative, selection)` returns those Tokens in authored order.++These pure queries use existing Anchors and Tokens, with no stored secondary index. An exported+Selection retains its Narrative identity; content lookup checks that identity. Structural anchors+remain distinct even when they resolve to the same token boundary, and an empty Segment can select+no Tokens. Authored order is independent of gaps, overlap or placement order in a Timeline.++Caption uses this content query and the Narrative's display correspondence to select complete+subtitle units. Timeline projection uses the same semantic references to locate events in physical+time. Neither operation substitutes for the other. Token lookup returns Tokens, not reconstructed+source prose: whitespace, comments and source-preserving edits remain the author language's concern.diff --git a/packages/narrative/src/identity.ts b/packages/narrative/src/identity.tsindex 1af8abc2..bda2c6a5 100644--- a/packages/narrative/src/identity.ts+++ b/packages/narrative/src/identity.ts@@ -25,6 +25,17 @@ export function assertNarrativeIdentity(value: Narrative): void {if (value.segments.length === 0) throw new Error("Narrative must contain at least one Segment.");const segmentIds = unique(value.segments.map((item) => item.id), "Narrative Segment id");const tokenIds = unique(value.tokens.map((item) => item.id), "Narrative Token id");+ assertCaptionDocumentIdentity(value.caption);+ if (value.caption.narrativeId !== value.id) throw new Error("Narrative CaptionDocument belongs to another Narrative.");+ const turns = new Map(value.turns.map((turn) => [turn.id, turn]));+ const tokens = new Map(value.tokens.map((token) => [token.id, token]));+ for (const unit of value.caption.units) {+ const turn = turns.get(unit.turnId);+ if (turn === undefined || turn.segmentId !== unit.segmentId || turn.role !== unit.role+ || unit.sourceTokenIds.some((id) => tokens.get(id)?.segmentId !== unit.segmentId)) {+ throw new Error(`Caption unit ${unit.id} disagrees with its authored speech.`);+ }+ }unique(value.turns.map((item) => item.id), "Narrative Turn id");unique(value.selections.map((item) => item.id), "Narrative Selection id");unique(value.moments.map((item) => item.id), "Narrative Moment id");diff --git a/packages/narrative/src/index.ts b/packages/narrative/src/index.tsindex 8621493e..92035ad0 100644--- a/packages/narrative/src/index.ts+++ b/packages/narrative/src/index.ts@@ -9,3 +9,6 @@ export {} from "./identity.js";export { captionDocumentSchema, narrativeExcerptSchema, narrativeMomentSchema, narrativeSchema, narrativeSelectionSchema } from "./schema.js";export type * from "./types.js";++export { narrativeAnchorTokenBoundary, narrativeSelectionTokenRange, narrativeTokensForSelection } from "./selection.js";+export type { NarrativeTokenRange } from "./selection.js";diff --git a/packages/narrative/src/schema.ts b/packages/narrative/src/schema.tsindex 534b545b..03c56a02 100644--- a/packages/narrative/src/schema.ts+++ b/packages/narrative/src/schema.ts@@ -23,33 +23,6 @@ const semanticAnchor: ValueSchema = { kind: "oneOf", variants: [kind: { schema: { kind: "literal", value: "program-end" } },}),] };-export const narrativeSchema: ValueSchema = object({- id: { schema: string },- segments: { schema: { kind: "array", minItems: 1, items: object({- id: { schema: string }, startAnchorId: { schema: string },- endAnchorId: { schema: string }, tokenStart: { schema: integer }, tokenEndExclusive: { schema: integer },- }) } },- tokens: { schema: { kind: "array", items: object({- id: { schema: string }, segmentId: { schema: string },- startAnchorId: { schema: string }, endAnchorId: { schema: string },- text: { schema: string }, normalized: { schema: string },- }) } },- turns: { schema: { kind: "array", items: object({- id: { schema: string }, segmentId: { schema: string }, role: { schema: string, optional: true },- tokenStart: { schema: integer }, tokenEndExclusive: { schema: integer },- }) } },- selections: { schema: { kind: "array", items: object({- id: { schema: string }, startAnchorId: { schema: string }, endAnchorId: { schema: string },- }) } },- moments: { schema: { kind: "array", items: object({- id: { schema: string }, anchorId: { schema: string },- }) } },- semanticIndex: { schema: object({-- anchors: { schema: { kind: "array", minItems: 2, items: semanticAnchor } },- }) },-});-export const narrativeExcerptSchema: ValueSchema = object({narrativeId: { schema: string },kind: { schema: { kind: "literal", value: "segment" } }, id: { schema: string },@@ -77,6 +50,34 @@ export const captionDocumentSchema: ValueSchema = object({words: { schema: { kind: "array", items: captionDisplayWord } },cueBreaks: { schema: { kind: "array", items: captionCueBreak } },});+export const narrativeSchema: ValueSchema = object({+ id: { schema: string },+ caption: { schema: captionDocumentSchema },+ segments: { schema: { kind: "array", minItems: 1, items: object({+ id: { schema: string }, startAnchorId: { schema: string },+ endAnchorId: { schema: string }, tokenStart: { schema: integer }, tokenEndExclusive: { schema: integer },+ }) } },+ tokens: { schema: { kind: "array", items: object({+ id: { schema: string }, segmentId: { schema: string },+ startAnchorId: { schema: string }, endAnchorId: { schema: string },+ text: { schema: string }, normalized: { schema: string },+ }) } },+ turns: { schema: { kind: "array", items: object({+ id: { schema: string }, segmentId: { schema: string }, role: { schema: string, optional: true },+ tokenStart: { schema: integer }, tokenEndExclusive: { schema: integer },+ }) } },+ selections: { schema: { kind: "array", items: object({+ id: { schema: string }, startAnchorId: { schema: string }, endAnchorId: { schema: string },+ }) } },+ moments: { schema: { kind: "array", items: object({+ id: { schema: string }, anchorId: { schema: string },+ }) } },+ semanticIndex: { schema: object({++ anchors: { schema: { kind: "array", minItems: 2, items: semanticAnchor } },+ }) },+});+export const narrativeMomentSchema: ValueSchema = object({narrativeId: { schema: string },id: { schema: string },diff --git a/packages/narrative/src/selection.ts b/packages/narrative/src/selection.tsnew file mode 100644index 00000000..83057e4a--- /dev/null+++ b/packages/narrative/src/selection.ts@@ -0,0 +1,52 @@+import type { Narrative, NarrativeSelection, NarrativeSelectionRef, NarrativeToken } from "./types.js";++/** A range in authored token order, independent of material placement and frame timing. */+export type NarrativeTokenRange = {+ readonly narrativeId: string;+ readonly tokenStart: number;+ readonly tokenEndExclusive: number;+};++/** Resolve a semantic anchor to a boundary in the authored token sequence. */+export function narrativeAnchorTokenBoundary(narrative: Narrative, anchorId: string): number {+ const anchor = narrative.semanticIndex.anchors.find((item) => item.id === anchorId);+ if (anchor === undefined) throw new Error(`Narrative ${narrative.id} has no anchor ${anchorId}.`);+ if (anchor.kind === "program-start") return 0;+ if (anchor.kind === "program-end") return narrative.tokens.length;+ const segment = narrative.segments.find((item) => item.id === anchor.segmentId);+ if (segment === undefined) throw new Error(`Anchor ${anchorId} has no authored Segment.`);+ if (anchor.kind === "segment-start") return segment.tokenStart;+ if (anchor.kind === "segment-end") return segment.tokenEndExclusive;+ const index = narrative.tokens.findIndex((item) => item.id === anchor.tokenId);+ if (index < 0) throw new Error(`Anchor ${anchorId} has no authored token.`);+ return anchor.kind === "token-start" ? index : index + 1;+}++export function narrativeSelectionTokenRange(+ narrative: Narrative,+ selection: NarrativeSelection | NarrativeSelectionRef,+): NarrativeTokenRange {+ if ("narrativeId" in selection && selection.narrativeId !== narrative.id) {+ throw new Error(`Selection ${selection.id} belongs to another Narrative.`);+ }+ const anchors = narrative.semanticIndex.anchors;+ const start = anchors.findIndex((item) => item.id === selection.startAnchorId);+ const end = anchors.findIndex((item) => item.id === selection.endAnchorId);+ if (start < 0 || end < 0 || end < start) {+ throw new Error(`Selection ${selection.id} has invalid authored anchor order.`);+ }+ return {+ narrativeId: narrative.id,+ tokenStart: narrativeAnchorTokenBoundary(narrative, selection.startAnchorId),+ tokenEndExclusive: narrativeAnchorTokenBoundary(narrative, selection.endAnchorId),+ };+}++/** Select content without consulting a Timeline or reconstructing prose from token strings. */+export function narrativeTokensForSelection(+ narrative: Narrative,+ selection: NarrativeSelection | NarrativeSelectionRef,+): readonly NarrativeToken[] {+ const range = narrativeSelectionTokenRange(narrative, selection);+ return narrative.tokens.slice(range.tokenStart, range.tokenEndExclusive);+}diff --git a/packages/narrative/src/types.ts b/packages/narrative/src/types.tsindex 9878a3b9..e45d3034 100644--- a/packages/narrative/src/types.ts+++ b/packages/narrative/src/types.ts@@ -113,12 +113,14 @@ export type SemanticAnchor =readonly tokenId?: string;};+/** Complete authored content; local views are projections of this value. */export type Narrative = {/** Author-visible `<script id>`; every exported semantic view retains it. */readonly id: string;readonly segments: readonly NarrativeSegment[];readonly tokens: readonly NarrativeToken[];readonly turns: readonly NarrativeTurn[];+ readonly caption: CaptionDocument;readonly selections: readonly NarrativeSelection[];readonly moments: readonly NarrativeMoment[];readonly semanticIndex: {diff --git a/packages/performance-studio/README.md b/packages/performance-studio/README.mdnew file mode 100644index 00000000..958ee551--- /dev/null+++ b/packages/performance-studio/README.md@@ -0,0 +1,25 @@+# Performance Studio Companion++The Companion contributes one Performance Track with a content area and an internal Uses band:++- **Performance** shows the Timeline's placed picture Takes, with source thumbnails. These are available contents, independent of which Use presents or hides them. Audio-only Takes have no picture item.+- **Uses** shows each authored Style application as a named interval in the bottom label band. Declaration order sets overlap order; selecting an item raises it for editing without changing the authored priority. Covered and empty applications remain available.++`performance:Track.program` exports the ordered windows and unresolved visuals. The Companion reads that value instead of reconstructing author rules from the final visibility masks. The Timeline supplies the content lane. Typed temporal values identify each Use's exact author element even when its id was omitted.++One Performance label spans the content and Uses band; selection targets a picture Take or a Use.+A Take shows its Segment and placed frame range. A Use shows its resolved range and Style reference.+These are small read-only facts within When and How, alongside the editable fields in the same domains.+The displayed range uses an exclusive end frame.++Selecting a Use exposes its ordinary Style's referenced Frame and Media appearance Recipe. Frame edits write to SVML; appearance edits write to SVS. A shared Style or Recipe remains shared. Temporal gestures follow the same Selection/Moment/clock projection and source-writeback path as other components.++Project Styles receive the same Use interval and title. Their parameters are contributed through `createStudioCompanionHostFacet({ parameters })`, matching+the Style's own Module and Surface. The Track opts its `style` reference into this declaration.+Ordinary Style fields live in their own parameter Companion. See the project+[performance Styles](../../examples/semantic-composition/packages/performance-styles/README.md) for an example.++The internal `bands`, field bindings and content chrome are public Studio Companion declarations. Uses are entities in the same Track; independent child Tracks still use `attachments`. Studio does not recognize Performance names or scene-specific layouts. Caption and Sound can describe their own content and applications through the same interface.++Use picture selection follows exact rendered Present ids supplied by Performance. Take content uses+its Segment identity rather than array position, so reordering Takes preserves selection identity.diff --git a/packages/performance-studio/package.json b/packages/performance-studio/package.jsonnew file mode 100644index 00000000..5c086166--- /dev/null+++ b/packages/performance-studio/package.json@@ -0,0 +1,21 @@+{+ "name": "@hypit/performance-studio",+ "version": "0.0.0-dev",+ "license": "SEE LICENSE IN LICENSE",+ "private": true,+ "type": "module",+ "exports": {+ ".": "./src/index.ts"+ },+ "hypit": {+ "activation": "./src/activation.ts"+ },+ "dependencies": {+ "@hypit/performance": "workspace:*",+ "@hypit/timeline": "workspace:*",+ "@hypit/temporal": "workspace:*",+ "@hypit/composition": "workspace:*",+ "@hypit/studio-adapter": "workspace:*",+ "@hypit/media-track-studio": "workspace:*"+ }+}diff --git a/packages/performance-studio/src/activation.ts b/packages/performance-studio/src/activation.tsnew file mode 100644index 00000000..9f144954--- /dev/null+++ b/packages/performance-studio/src/activation.ts@@ -0,0 +1,6 @@+import { createStudioCompanionHostFacet } from "@hypit/studio-adapter";+import { performanceStudioTrackCompanions, performanceStudioParameterCompanions } from "./index.js";+export default {+ format: "hypit.node-package@1" as const,+ hostFacets: [createStudioCompanionHostFacet({ tracks: performanceStudioTrackCompanions, parameters: performanceStudioParameterCompanions })],+};diff --git a/packages/performance-studio/src/index.ts b/packages/performance-studio/src/index.tsnew file mode 100644index 00000000..4ae1d1c8--- /dev/null+++ b/packages/performance-studio/src/index.ts@@ -0,0 +1,84 @@+import { performanceModuleRef, performancePresentId } from "@hypit/performance";+import type { PerformanceSet } from "@hypit/performance";+import { projectTimelineMedia, timelineTypes } from "@hypit/timeline";+import type { Timeline } from "@hypit/timeline";+import { temporalTypes } from "@hypit/temporal";+import { compositionTypes } from "@hypit/composition";+import { mediaAppearanceStudioFields } from "@hypit/media-track-studio";+import { artifactPreview, authoredChildFor, previewLayer, requiredReferencedValue, requiredSurfaceValue, temporalLineageFor, temporalSemanticSource } from "@hypit/studio-adapter";+import type { StudioTrackCompanion, StudioTrackCompanionContext, StudioEntityDraft, StudioSourceBindingDeclaration } from "@hypit/studio-adapter";++const frameFields = ["left", "top", "right", "bottom", "x", "y", "width", "height"];+const frameBindings: readonly StudioSourceBindingDeclaration[] = frameFields.map(name => ({ name, writable: true }));+// Ordinary Performance uses Timeline playback. The rest is the same Media appearance vocabulary.+const playbackFields = new Set(["playback", "trim-start", "trim-end"]);+const appearanceBindings = mediaAppearanceStudioFields.bindings.filter(field => !playbackFields.has(field.name));+const appearanceInspector = mediaAppearanceStudioFields.inspector.filter(field =>+ !playbackFields.has(field.binding.slice("appearance.".length)));++export function projectPerformance(context: StudioTrackCompanionContext): readonly StudioEntityDraft[] {+ const timeline = requiredReferencedValue(context, "timeline", timelineTypes.track) as Timeline;+ const program = requiredSurfaceValue(context, "program") as PerformanceSet;+ // Source content keeps its own placement. A Use changes its presentation, not its media clock.+ const content: StudioEntityDraft[] = projectTimelineMedia(timeline).flatMap((clip, index) => {+ if (clip.media.visual === undefined) return [];+ return [{+ id: `${context.track.outputRef}:content:${clip.segmentId}`,+ authoredId: clip.segmentId,+ display: { title: clip.segmentId, layers: [previewLayer(artifactPreview("video", clip.media.visual.artifact.resource), "storyboard")] },+ ...clip.span, stackOrder: index,+ inspector: [+ { id: "segment", label: "Segment", domain: "how", section: { id: "content", label: "Content" }, value: clip.segmentId },+ { id: "range", label: "Range", domain: "when", section: { id: "placement", label: "Placement" }, value: `${clip.span.startFrame}–${clip.span.endFrameExclusive}`, unit: "f", summary: "Placed interval; the end frame is exclusive." },+ ],+ presentation: { entity: "performance-content", chrome: "standard" },+ }];+ });+ const uses: StudioEntityDraft[] = program.uses.map((use, index) => {+ const authoredId = use.window.subjectId;+ const child = authoredChildFor(context, authoredId, [temporalTypes.windowSpec]);+ if (child === undefined) throw new Error(`Performance Use ${authoredId} has no author provenance.`);+ const style = child.referenceAttributes.style;+ const temporal = temporalLineageFor(context, authoredId, "window");+ const semantic = temporalSemanticSource(temporal);+ return {+ id: `${context.track.outputRef}:use:${authoredId}`, authoredId,+ display: { title: style ?? authoredId, layers: [] },+ ...use.window.span, stackOrder: index,+ elementRange: child.range,+ ...(temporal === undefined ? {} : { temporal }),+ ...(semantic === undefined ? {} : { markerId: semantic.id }),+ presentation: { entity: "performance-use", chrome: "standard" },+ band: "uses",+ renderIds: use.visual.presents.map(present => performancePresentId(authoredId, present.id)),+ inspector: [{ id: "range", label: "Range", domain: "when", section: { id: "placement", label: "Placement" }, value: `${use.window.span.startFrame}–${use.window.span.endFrameExclusive}`, unit: "f", summary: "Resolved Use interval; the end frame is exclusive. Timeline gestures follow this Use's author timing." }],+ };+ });+ return [...content, ...uses];+}++export const performanceStudioTrackCompanions: readonly StudioTrackCompanion[] = [{+ id: "visual", role: "track",+ output: { type: compositionTypes.visualTrack, surface: "track", modules: [performanceModuleRef] },+ family: "performance", label: "Performance", tone: "blue", icon: "video",+ lane: { heightPx: 60 },+ requiredValues: ["program"],+ bands: [{+ id: "uses", placement: "after", heightPx: 15, display: "label",+ bindings: [{ name: "style", companion: true }],+ inspector: [{ binding: "style", label: "Style", domain: "how", section: { id: "style", label: "Style" }, control: "text", summary: "The referenced Style is shared by every Use that selects it." }],+ }],+ project: projectPerformance,+}];++export const performanceStudioParameterCompanions: readonly import("@hypit/studio-adapter").StudioParameterCompanion[] = [{+ id: "style", match: { module: performanceModuleRef, surface: "style" },+ bindings: [ { name: "frame", referenced: frameBindings },+ { name: "appearance", recipe: { bindings: appearanceBindings } } ],+ inspector: [+ ...frameFields.map(name => ({ binding: `frame.${name}`, label: name[0]!.toUpperCase() + name.slice(1),+ domain: "where" as const, page: { id: "frame", label: "Frame" }, section: { id: "placement", label: "Placement" },+ control: "number" as const, number: { suffixes: ["%", "px"], step: 1 } })),+ ...appearanceInspector,+ ],+}];diff --git a/packages/performance-studio/test/companion.test.ts b/packages/performance-studio/test/companion.test.tsnew file mode 100644index 00000000..139d9da4--- /dev/null+++ b/packages/performance-studio/test/companion.test.ts@@ -0,0 +1,47 @@+import assert from "node:assert/strict";+import test from "node:test";+import { sealVisualTrack } from "@hypit/composition";+import { temporalTypes } from "@hypit/temporal";+import { timelineTypes } from "@hypit/timeline";+import type { StudioTrackCompanionContext } from "@hypit/studio-adapter";+import { projectPerformance } from "../src/index.js";+import { timelineFixture } from "../../../test/timeline-fixture.js";+import { projectProgramWindow } from "../../../test/temporal-fixture.js";++const base = timelineFixture({ id: "film", durationSec: 6, frameRate: { numerator: 30, denominator: 1 } }, {+ segments: [{id:"first",frameCount:90},{id:"second",frameCount:90}],+});+function context(picture: boolean): StudioTrackCompanionContext {+ const timeline = {...base, items: base.items.map((item,index)=>({...item,startFrame:index ? 60:15,take:{...item.take,+ media:{...item.take.media,...(picture?{visual:{artifact:{kind:"blob",resource:`res_picture${index}`,mediaType:"video/mp4",size:1},width:64,height:64}}:{})},+ }}))};+ const program = {uses:["base","override","hidden"].map(id=>({+ window:projectProgramWindow({itemId:id,semantic:base,projection:{start:{ref:"absolute",at:{unit:"frames",value:0}},end:{ref:"absolute",at:{unit:"frames",value:180}}}}),+ visual:sealVisualTrack({id,programSpaceId:base.id,visualIr:"hypit.visual-ir@1",presents:[]}),+ }))};+ return {+ track:{outputRef:"p.visual",trace:{references:[{input:"timeline",typeRef:timelineTypes.track,ref:"film"}],outputPorts:[{name:"program",ref:"p.program"}]}},+ placement:{children:program.uses.map((use,index)=>({range:{start:index*10,end:index*10+5},referenceAttributes:{style:`styles.${use.window.subjectId}`},values:[{type:temporalTypes.windowSpec,value:{id:use.window.subjectId}}]}))},+ values:new Map<string,unknown>([["film",timeline],["p.program",program]]),spans:[],temporalBindings:[],+ } as unknown as StudioTrackCompanionContext;+}++test("Use band keeps fully overridden and empty rules with exact child and Style ownership",()=>{+ const uses=projectPerformance(context(false));+ assert.deepEqual(uses.map(use=>[use.display.title,use.band,use.elementRange]),[+ ["styles.base","uses",{start:0,end:5}],+ ["styles.override","uses",{start:10,end:15}],+ ["styles.hidden","uses",{start:20,end:25}],+ ]);+ assert.ok(uses.every(use=>use.presentation?.chrome==="standard"));+ assert.ok(uses.every(use=>use.inspector?.[0]?.value === "0–180"));+});++test("content follows placed picture Takes including gaps and overlaps; audio-only contributes no picture",()=>{+ const content=projectPerformance(context(true)).filter(entity=>entity.band===undefined);+ assert.deepEqual(content.map(entity=>[entity.authoredId,entity.startFrame,entity.endFrameExclusive]),[+ ["first",15,105],["second",60,150],+ ]);+ assert.deepEqual(content[0]?.inspector?.map(field=>field.value), ["first", "15–105"]);+ assert.equal(projectPerformance(context(false)).filter(entity=>entity.band===undefined).length,0);+});diff --git a/packages/performance/README.md b/packages/performance/README.mdnew file mode 100644index 00000000..a3b65230--- /dev/null+++ b/packages/performance/README.md@@ -0,0 +1,83 @@+# Performance++Performance presents footage already placed on a Timeline. Media Items receive independent assets;+Performance Uses select how the existing footage appears. [Sound](../sound/README.md) presents existing audio. Include the wanted visual and audio outputs+separately in Film.++```svml+<import as="performance" from="@hypit/performance@1"/>+<performance:Style id="full" frame={layout.full} appearance={recipes.media.presenter}/>+<performance:Style id="side" frame={layout.side} appearance={recipes.media.presenter}/>+<performance:Track id="presenter" timeline={program.timeline} canvas={canvas}>+ <performance:Use style={full}/>+ <performance:Use during={story.selection.explanation} style={side}/>+</performance:Track>+```++Select `presenter.visual` in Film. A Style's appearance uses Media fitting, clipping, radius, padding,+border, shadows and sample appearance. `stack-order` places its Presents among Film contributions.+A Use with no time selector covers the complete Timeline. Later Uses replace earlier rules locally;+they do not add another copy of the picture. No matching Use means no picture. An empty winning+Style stays empty. Separate Performance Tracks can independently show the same material.++## Time, sources and visibility++Use accepts shared `during`, `at` + `for`, `until` + `for`, or `start` + `end`. A Window says when a+presentation applies. Timeline determines source positions: a Take placed at second 5 shows source+second 2 at program second 7. Styles receive the original Use Window even when partially covered.+Visibility does not restart playback or animation. The ordinary Style presents available videos in+Timeline declaration order, later above earlier; no Takes, audio-only Takes and source gaps supply+no picture. The Timeline can extend past all material without filling that time with a clip.++Two independently bound endpoints can describe a physical overlap that is backwards in Script order:++```svml+<performance:Use style={mix}+ start-source={story.segment.next} start="segment.start"+ end-source={story.segment.previous} end="segment.end"/>+```++The concrete Style determines which sources are outgoing/incoming. Window endpoints do not select+those roles implicitly. There is no automatic crossfade when source spans overlap.++## Write a project Style++`performanceStyle(fragment, bindings)` creates an authored Style. The fragment takes `timeline`,+`canvas`, and `window`, plus any family-defined typed inputs, and exports `visual: VisualTrack`.+`bindings` binds those extra inputs to resolved author references. A Style Surface returns this as+an inline Record of `performanceTypes.style`. At each Use, the Track expands the fragment and turns+every binding into a normal graph edge, including references to generated outputs. No family+registry or effect-name dispatch is involved. The Style is an author-time definition, not a+runtime-produced choice of executable code.++```ts+const style = performanceStyle(fragment, {+ from: resolveFrame("from"),+ to: resolveFrame("to"),+});+// Each value is a SurfaceResolvedReference; all fragment inputs except the three+// occurrence inputs above must be bound. Your Surface owns its parameter names.+```++A family may bind Segment references, Moments, prepared assets, Frames, Recipes or computed outputs.+Its Producers use the Timeline and shared projections to obtain exact source spans and event times.+`projectTimelineMedia` returns zero, one or several sources with their identities and native source+positions. Known sources without visual material differ from invalid references. Rendering can use+ordinary structural elements, Media helpers or a browser program with HTML, CSS and frame code.+`ordinaryPerformance` provides reusable fixed framing; a whole coordinated diagram/presenter scene+can instead be an independent component.++The [project Style example](../../examples/semantic-composition/packages/performance-styles/README.md)+implements movement and two-source blending through these public interfaces. It demonstrates an+extension path, not a required library of built-in transitions.++## Implementation boundary++`resolvePerformance` resolves last-declared Window coverage. It retains each Present's original+`span` and intersects its optional `visibility` subranges with the winning intervals. The renderer+uses `span` for animation and sampling and `visibility` only for display. Empty Uses still participate+in coverage. Nothing in this package changes audio, Script, Timeline placement or generation.++`Track.visual` is the resolved picture. `Track.program` retains the ordered Use windows and their+unresolved VisualTracks, including covered and empty applications. The+[Studio Companion](../performance-studio/README.md) uses this program to expose individual Uses.diff --git a/packages/performance/package.json b/packages/performance/package.jsonnew file mode 100644index 00000000..71fdd718--- /dev/null+++ b/packages/performance/package.json@@ -0,0 +1,30 @@+{+ "name": "@hypit/performance",+ "version": "0.0.0-dev",+ "license": "SEE LICENSE IN LICENSE",+ "private": true,+ "type": "module",+ "exports": {+ ".": "./src/index.ts"+ },+ "hypit": {+ "activation": "./src/activation.ts"+ },+ "dependencies": {+ "@hypit/artifact": "workspace:*",+ "@hypit/component-kit": "workspace:*",+ "@hypit/composition": "workspace:*",+ "@hypit/elaborator": "workspace:*",+ "@hypit/media": "workspace:*",+ "@hypit/narrative": "workspace:*",+ "@hypit/program-space": "workspace:*",+ "@hypit/protocol": "workspace:*",+ "@hypit/timeline": "workspace:*",+ "@hypit/spatial": "workspace:*",+ "@hypit/svs": "workspace:*",+ "@hypit/markup": "workspace:*",+ "@hypit/temporal": "workspace:*",+ "@hypit/temporal-markup": "workspace:*",+ "@hypit/media-track": "workspace:*"+ }+}diff --git a/packages/performance/src/activation.ts b/packages/performance/src/activation.tsnew file mode 100644index 00000000..4be09bf7--- /dev/null+++ b/packages/performance/src/activation.ts@@ -0,0 +1,11 @@+import { createMarkupSurfaceHostFacet } from "@hypit/markup";+import { performanceManifest, performanceModuleRef, performanceMarkupSurfaces } from "./manifest.js";+import { performanceComponent } from "./component.js";+import { decodePerformanceStyleSurface, decodePerformanceTrackSurface } from "./surface.js";+export const hypitPackage = {+ format: "hypit.node-package@1" as const,+ modules: [{ manifest: performanceManifest }], components: [performanceComponent],+ hostFacets: performanceMarkupSurfaces.map(declaration => createMarkupSurfaceHostFacet({ module: performanceModuleRef, declaration,+ handler: declaration.name === "style" ? decodePerformanceStyleSurface : decodePerformanceTrackSurface })),+};+export default hypitPackage;diff --git a/packages/performance/src/component.ts b/packages/performance/src/component.tsnew file mode 100644index 00000000..64722e16--- /dev/null+++ b/packages/performance/src/component.ts@@ -0,0 +1,46 @@+import { canonicalize } from "@hypit/protocol";+import type { StoredValue } from "@hypit/protocol";+import type { ComponentPackage } from "@hypit/component-kit";+import type { Timeline } from "@hypit/timeline";+import { sealVisualTrack } from "@hypit/composition";+import type { VisualTrack } from "@hypit/composition";+import type { CanvasSpace, SpatialFrame, ContentFit } from "@hypit/spatial";+import type { TemporalWindow } from "@hypit/temporal";+import { createMediaLayerSet, createMediaSoundSet, createMediaTrackSet, finalizeMediaTrack, projectMediaVisualTrack } from "@hypit/media-track";+import type { MediaItemSpec, MediaSampleLayerSpec } from "@hypit/media-track";+import { appendMediaPerformance } from "./media.js";+import { appendPerformanceUse, resolvePerformance } from "./program.js";+import type { PerformanceSet } from "./program.js";+import { performanceProducers } from "./manifest.js";++function inline<T>(value: StoredValue | undefined): T {+ if (value?.kind !== "inline") throw new Error("Performance input must be inline.");+ return value.value as unknown as T;+}+const output = (value: unknown) => ({ kind: "inline" as const, value: canonicalize(value) });+export function ordinaryPerformance(input: {+ timeline: Timeline; canvas: CanvasSpace; window: TemporalWindow; frame: SpatialFrame;+ fit: ContentFit; sample: MediaSampleLayerSpec; spec: MediaItemSpec;+}): VisualTrack {+ const id = input.window.subjectId;+ const header = { id };+ const set = appendMediaPerformance({ ...input, header, set: createMediaTrackSet(), layers: createMediaLayerSet(),+ sounds: createMediaSoundSet(), spec: { ...input.spec, id } }, input.fit, input.sample);+ if (set.items.length === 0) return sealVisualTrack({ id, programSpaceId: input.timeline.id, visualIr: "hypit.visual-ir@1", presents: [] });+ return projectMediaVisualTrack(input.timeline, finalizeMediaTrack(set, header, input.timeline));+}+export const performanceComponent: ComponentPackage = {+ producers: [+ { producer: performanceProducers.create, handler: () => ({ outputs: { set: output({ uses: [] }) }, needs: {} }) },+ { producer: performanceProducers.append, handler: ({ inputs }) => ({ outputs: { set: output(appendPerformanceUse(+ inline<PerformanceSet>(inputs.set?.value), inline<TemporalWindow>(inputs.window?.value), inline<VisualTrack>(inputs.visual?.value),+ )) }, needs: {} }) },+ { producer: performanceProducers.resolve, handler: ({ inputs }) => ({ outputs: { visual: output(resolvePerformance(+ inline<Timeline>(inputs.timeline?.value), inline<{ id: string }>(inputs.header?.value).id, inline<PerformanceSet>(inputs.set?.value),+ )) }, needs: {} }) },+ { producer: performanceProducers.ordinary, handler: ({ inputs }) => ({ outputs: { visual: output(ordinaryPerformance({+ timeline: inline(inputs.timeline?.value), canvas: inline(inputs.canvas?.value), window: inline(inputs.window?.value),+ frame: inline(inputs.frame?.value), fit: inline(inputs.fit?.value), sample: inline(inputs.sample?.value), spec: inline(inputs.spec?.value),+ })) }, needs: {} }) },+ ],+};diff --git a/packages/performance/src/index.ts b/packages/performance/src/index.tsnew file mode 100644index 00000000..cae602fd--- /dev/null+++ b/packages/performance/src/index.ts@@ -0,0 +1,6 @@+export * from "./style.js";+export * from "./program.js";+export * from "./component.js";+export * from "./surface.js";+export * from "./manifest.js";+export { appendMediaPerformance } from "./media.js";diff --git a/packages/performance/src/manifest.ts b/packages/performance/src/manifest.tsnew file mode 100644index 00000000..e0018bf1--- /dev/null+++ b/packages/performance/src/manifest.ts@@ -0,0 +1,53 @@+import { compositionDependency, compositionTypes } from "@hypit/composition";+import { timelineDependency, timelineTypes } from "@hypit/timeline";+import { spatialDependency, spatialTypes } from "@hypit/spatial";+import { temporalDependency, temporalTypes } from "@hypit/temporal";+import { mediaTrackDependency, mediaTrackTypes } from "@hypit/media-track";+import { svsRecipeType } from "@hypit/svs";+import { temporalWindowAttributeVocabulary } from "@hypit/temporal-markup";+import type { ModuleManifest, ProducerRef, TypeRef } from "@hypit/protocol";++export const performanceModuleRef = { name: "@hypit/performance", version: "1" } as const;+export const performanceTypes = Object.fromEntries(["Style", "Set", "Header"].map(name => [name.toLowerCase(), { module: performanceModuleRef, name: `Performance${name}` }])) as Record<"style" | "set" | "header", TypeRef>;+export const performanceProducers = Object.fromEntries(["create", "append", "resolve", "ordinary"].map(name => [name, { module: performanceModuleRef, name }])) as Record<"create" | "append" | "resolve" | "ordinary", ProducerRef>;+export const ordinaryInputs = [+ { name: "timeline", type: timelineTypes.track }, { name: "canvas", type: spatialTypes.canvas },+ { name: "window", type: temporalTypes.window }, { name: "frame", type: spatialTypes.frame },+ { name: "fit", type: spatialTypes.fit }, { name: "sample", type: mediaTrackTypes.sampleLayerSpec },+ { name: "spec", type: mediaTrackTypes.itemSpec },+];+export const performanceManifest: ModuleManifest = {+ format: "hypit.module@1", ...performanceModuleRef,+ dependencies: [compositionDependency, timelineDependency, spatialDependency, temporalDependency, mediaTrackDependency],+ types: Object.values(performanceTypes).map(type => ({ name: type.name })), capabilities: [],+ producers: [+ { name: "create", inputs: [], outputs: [{ name: "set", type: performanceTypes.set }], needs: [] },+ { name: "append", inputs: [{ name: "set", type: performanceTypes.set }, { name: "window", type: temporalTypes.window }, { name: "visual", type: compositionTypes.visualTrack }], outputs: [{ name: "set", type: performanceTypes.set }], needs: [] },+ { name: "resolve", inputs: [{ name: "set", type: performanceTypes.set }, { name: "timeline", type: timelineTypes.track }, { name: "header", type: performanceTypes.header }], outputs: [{ name: "visual", type: compositionTypes.visualTrack }], needs: [] },+ { name: "ordinary", inputs: ordinaryInputs, outputs: [{ name: "visual", type: compositionTypes.visualTrack }], needs: [] },+ ],+};+export const performanceMarkupSurfaces = [+ { name: "style", tag: "Style", mode: "structured", outputs: [performanceTypes.style, spatialTypes.fit, mediaTrackTypes.sampleLayerSpec, mediaTrackTypes.itemSpec], vocabulary: {+ summary: "Defines ordinary A-roll framing using Media appearance properties.",+ example: `<performance:Style id="full" frame={layout.full} appearance={recipes.media.presenter}/>`,+ attributes: [+ { name: "id", kind: "identifier", required: true, summary: "Style name." },+ { name: "frame", kind: "reference", required: true, accepts: [spatialTypes.frame], summary: "Destination Frame." },+ { name: "appearance", kind: "reference", required: true, accepts: [svsRecipeType], summary: "Media appearance Recipe: fitting, clipping, border and color." },+ ],+ } },+ { name: "track", tag: "Track", mode: "structured", outputs: [compositionTypes.visualTrack, performanceTypes.set, performanceTypes.header, temporalTypes.instantSpec, temporalTypes.windowSpec, temporalTypes.instant, temporalTypes.window], vocabulary: {+ summary: "Presents Timeline footage through ordered Use rules, retaining source playback time.",+ example: `<performance:Track id="presenter" timeline={program.timeline} canvas={canvas}><performance:Use style={full}/></performance:Track>`,+ attributes: [+ { name: "id", kind: "identifier", required: true, summary: "Presentation name." },+ { name: "timeline", kind: "reference", required: true, accepts: [timelineTypes.track], summary: "Placed material and film time." },+ { name: "canvas", kind: "reference", required: true, accepts: [spatialTypes.canvas], summary: "Composition Canvas." },+ ], children: [{ tag: "Use", cardinality: "many", summary: "Last matching Use wins. No time selector means the whole Timeline.", attributes: [+ { name: "id", kind: "identifier", required: false, summary: "Optional occurrence identity." },+ { name: "style", kind: "reference", required: true, accepts: [performanceTypes.style], summary: "Ordinary or project-authored presentation Style." },+ ...temporalWindowAttributeVocabulary,+ ] }],+ } },+] as const;diff --git a/packages/media-track/src/performance.ts b/packages/performance/src/media.tssimilarity index 64%rename from packages/media-track/src/performance.tsrename to packages/performance/src/media.tsindex e934ecd5..8cdeec1c 100644--- a/packages/media-track/src/performance.ts+++ b/packages/performance/src/media.ts@@ -1,22 +1,21 @@-import { projectSemanticMedia } from "@hypit/semantic-track";-import type { SemanticTrack } from "@hypit/semantic-track";+import { projectTimelineMedia } from "@hypit/timeline";+import type { Timeline } from "@hypit/timeline";import type { ContentFit } from "@hypit/spatial";-import { appendTimedMediaLayer, createMediaLayerSet } from "./layers.js";-import { appendMediaItem } from "./program.js";-import type { MediaSampleLayerSpec } from "./types.js";+import { appendTimedMediaLayer, createMediaLayerSet } from "@hypit/media-track";+import { appendMediaItem } from "@hypit/media-track";+import type { MediaSampleLayerSpec } from "@hypit/media-track";type ItemArguments = Parameters<typeof appendMediaItem>;type PerformanceInput = {- set: ItemArguments[0]; header: ItemArguments[1]; space: ItemArguments[2]; canvas: ItemArguments[3];+ set: ItemArguments[0]; header: ItemArguments[1]; timeline: ItemArguments[2]; canvas: ItemArguments[3];layers: ItemArguments[4]; frame: ItemArguments[5]; spec: ItemArguments[6];sounds: ItemArguments[7]; window: ItemArguments[8];};/** One framed unit and lifecycle containing all selected performance spans. */-export function appendMediaPerformance(input: PerformanceInput, semantic: SemanticTrack, fit: ContentFit, sample: MediaSampleLayerSpec) {- if (semantic.id !== input.space.id) throw new Error("Media performance must belong to the composition's ProgramSpace.");+export function appendMediaPerformance(input: PerformanceInput, fit: ContentFit, sample: MediaSampleLayerSpec) {const layers = [...input.layers.layers];- for (const clip of projectSemanticMedia(semantic, input.window.span)) {+ for (const clip of projectTimelineMedia(input.timeline, input.window.span)) {if (clip.media.visual === undefined) continue;const [layer] = appendTimedMediaLayer(createMediaLayerSet(), clip.media, fit, {...sample, id: `${input.spec.id}:${clip.segmentId}`, trim: clip.source, occupancy: { mode: "once", align: "start" },@@ -32,6 +31,6 @@ export function appendMediaPerformance(input: PerformanceInput, semantic: Semant} });}if (layers.length === 0) return input.set;- return appendMediaItem(input.set, input.header, input.space, input.canvas, { layers }, input.frame,+ return appendMediaItem(input.set, input.header, input.timeline, input.canvas, { layers }, input.frame,input.spec, input.sounds, input.window);}diff --git a/packages/performance/src/program.ts b/packages/performance/src/program.tsnew file mode 100644index 00000000..82baf843--- /dev/null+++ b/packages/performance/src/program.ts@@ -0,0 +1,51 @@+import { sealVisualTrack, assertVisualTrackIdentity } from "@hypit/composition";+import type { FrameSpan, VisualTrack } from "@hypit/composition";+import { assertTemporalWindowFor } from "@hypit/temporal";+import type { TemporalWindow } from "@hypit/temporal";+import type { Timeline } from "@hypit/timeline";+import { VISUAL_IR_V1 } from "@hypit/visual-ir";++export type PerformanceUse = { readonly window: TemporalWindow; readonly visual: VisualTrack };+export type PerformanceSet = { readonly uses: readonly PerformanceUse[] };++export function appendPerformanceUse(set: PerformanceSet, window: TemporalWindow, visual: VisualTrack): PerformanceSet {+ assertVisualTrackIdentity(visual);+ if (visual.programSpaceId !== window.start.source.spaceId) throw new Error("Performance Use and picture belong to different Timelines.");+ return { uses: [...set.uses, { window, visual }] };+}++export function intersectVisibility(left: readonly FrameSpan[], right: readonly FrameSpan[]): FrameSpan[] {+ return left.flatMap(a => right.flatMap(b => {+ const startFrame = Math.max(a.startFrame, b.startFrame), endFrameExclusive = Math.min(a.endFrameExclusive, b.endFrameExclusive);+ return endFrameExclusive > startFrame ? [{ startFrame, endFrameExclusive }] : [];+ }));+}++export function performancePresentId(useId: string, presentId: string): string {+ return `${useId}:${presentId}`;+}++/** Keep original animation/sample origins. Only the visibility mask changes. */+export function resolvePerformance(timeline: Timeline, id: string, set: PerformanceSet): VisualTrack {+ const presents: VisualTrack["presents"][number][] = [];+ for (const [index, use] of set.uses.entries()) {+ assertTemporalWindowFor(use.window, { subjectId: use.window.subjectId, space: timeline });+ if (use.visual.programSpaceId !== timeline.id || use.window.start.source.spaceId !== timeline.id) throw new Error("Performance uses another Timeline.");+ let visible = [use.window.span];+ for (const later of set.uses.slice(index + 1)) {+ visible = visible.flatMap(span => {+ const cut = later.window.span;+ if (cut.endFrameExclusive <= span.startFrame || cut.startFrame >= span.endFrameExclusive) return [span];+ return [+ ...(cut.startFrame > span.startFrame ? [{ startFrame: span.startFrame, endFrameExclusive: cut.startFrame }] : []),+ ...(cut.endFrameExclusive < span.endFrameExclusive ? [{ startFrame: cut.endFrameExclusive, endFrameExclusive: span.endFrameExclusive }] : []),+ ];+ });+ }+ for (const present of use.visual.presents) {+ const visibility = intersectVisibility(visible, present.visibility ?? [present.span]);+ if (visibility.length) presents.push({ ...present, id: performancePresentId(use.window.subjectId, present.id), visibility });+ }+ }+ return sealVisualTrack({ id, programSpaceId: timeline.id, visualIr: VISUAL_IR_V1, presents });+}diff --git a/packages/performance/src/style.ts b/packages/performance/src/style.tsnew file mode 100644index 00000000..cf7863cd--- /dev/null+++ b/packages/performance/src/style.ts@@ -0,0 +1,35 @@+import { sealGraphFragment } from "@hypit/elaborator";+import type { GraphFragment } from "@hypit/elaborator";+import { compositionTypes } from "@hypit/composition";+import { timelineTypes } from "@hypit/timeline";+import { spatialTypes } from "@hypit/spatial";+import { temporalTypes } from "@hypit/temporal";+import { sameType } from "@hypit/protocol";+import type { SurfaceResolvedReference } from "@hypit/markup";++/** Author-time closure: bound references become ordinary graph edges at each Use. */+export type PerformanceStyle = {+ readonly fragment: GraphFragment;+ readonly bindings: Readonly<Record<string, { readonly type: SurfaceResolvedReference["type"]; readonly ref: SurfaceResolvedReference["ref"] }>>;+};+export function performanceStyle(fragment: GraphFragment, bindings: PerformanceStyle["bindings"] = {}): PerformanceStyle {+ fragment = sealGraphFragment(fragment);+ const common = { timeline: timelineTypes.track, canvas: spatialTypes.canvas, window: temporalTypes.window };+ for (const [name, type] of Object.entries(common)) {+ const port = fragment.inputs.find(port => port.name === name);+ if (port === undefined || !sameType(port.type, type)) throw new Error(`Performance Style requires ${name} input.`);+ if (bindings[name] !== undefined) throw new Error(`Performance supplies ${name} at each Use.`);+ }+ for (const port of fragment.inputs) {+ if (port.name in common) continue;+ const binding = bindings[port.name];+ if (binding === undefined || !sameType(port.type, binding.type)) throw new Error(`Performance Style binding ${port.name} has the wrong Type.`);+ }+ for (const name of Object.keys(bindings)) {+ if (!fragment.inputs.some(port => port.name === name)) throw new Error(`Performance Style has unknown binding ${name}.`);+ }+ if (fragment.exports.length !== 1 || fragment.exports[0]?.name !== "visual" || !sameType(fragment.exports[0].type, compositionTypes.visualTrack)) {+ throw new Error("Performance Style must export visual: VisualTrack.");+ }+ return { fragment, bindings: Object.fromEntries(Object.entries(bindings).map(([name, value]) => [name, { ref: value.ref, type: value.type }])) };+}diff --git a/packages/performance/src/surface.ts b/packages/performance/src/surface.tsnew file mode 100644index 00000000..59545120--- /dev/null+++ b/packages/performance/src/surface.ts@@ -0,0 +1,99 @@+import { assertAttributes, assertEmptyElement, textAttribute, optionalTextAttribute } from "@hypit/markup";+import type { StructuredSurfaceHandler, SurfaceResolvedReference, SurfaceRecordDraft, SurfaceComponentDraft, StructuredElement } from "@hypit/markup";+import { canonicalize, sameType } from "@hypit/protocol";+import type { TypeRef } from "@hypit/protocol";+import { sealGraphFragment } from "@hypit/elaborator";+import type { FragmentOperation } from "@hypit/elaborator";+import { createTemporalWindowProjection, resolveTemporalContext, temporalWindowAttributeNames } from "@hypit/temporal-markup";+import { temporalTypes } from "@hypit/temporal";+import { compositionTypes } from "@hypit/composition";+import { timelineTypes } from "@hypit/timeline";+import { spatialTypes } from "@hypit/spatial";+import { svsRecipeType } from "@hypit/svs";+import type { SvsRecipe } from "@hypit/svs";+import { decodeMediaFit, decodeMediaSampleSpec, decodeMediaItemSpec, mediaTrackTypes } from "@hypit/media-track";+import { performanceStyle } from "./style.js";+import type { PerformanceStyle } from "./style.js";+import { performanceTypes, performanceProducers, ordinaryInputs } from "./manifest.js";++const input = (name: string) => ({ kind: "fragment-input" as const, name });+const operation = (id: string) => ({ kind: "fragment-operation" as const, operation: id });+function reference(element: StructuredElement, name: string, type: TypeRef, resolve: (path: string) => SurfaceResolvedReference | undefined): SurfaceResolvedReference {+ const raw = element.attributes[name];+ if (typeof raw !== "object" || raw.kind !== "reference") throw new Error(`${element.name}.${name} must be a reference.`);+ const found = resolve(raw.path);+ if (found === undefined || !sameType(found.type, type)) throw new Error(`${element.name}.${name} has the wrong Type.`);+ return found;+}+function authored<T>(value: SurfaceResolvedReference): T {+ if (value.record?.value.kind !== "inline") throw new Error("A presentation Style or Recipe must be an authored inline value.");+ return value.record.value.value as unknown as T;+}+export const ordinaryPerformanceFragment = sealGraphFragment({+ inputs: ordinaryInputs,+ operations: [{ id: "ordinary", producer: performanceProducers.ordinary,+ inputs: Object.fromEntries(ordinaryInputs.map(port => [port.name, input(port.name)])), result: { kind: "output", name: "visual" } }],+ exports: [{ name: "visual", type: compositionTypes.visualTrack, root: operation("ordinary") }],+});+export const decodePerformanceStyleSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {+ assertAttributes(element, ["id", "frame", "appearance"]); assertEmptyElement(element);+ const id = textAttribute(element, "id");+ const frame = reference(element, "frame", spatialTypes.frame, resolveReference);+ const recipe = authored<SvsRecipe>(reference(element, "appearance", svsRecipeType, resolveReference));+ if (["playback", "trim-start", "trim-end"].some(key => recipe.properties[key] !== undefined)) throw new Error("Performance uses Timeline source positions; select its presentation interval on Use.");+ const records: SurfaceRecordDraft[] = [];+ const bind = (name: string, type: TypeRef, value: unknown) => {+ const recordId = `${id}.${name}`;+ records.push({ id: recordId, type, value: { kind: "inline", value: canonicalize(value) }, range: element.range });+ return { ref: { kind: "record" as const, id: recordId }, type };+ };+ const style = performanceStyle(ordinaryPerformanceFragment, {+ frame,+ fit: bind("fit", spatialTypes.fit, decodeMediaFit(recipe)),+ sample: bind("sample", mediaTrackTypes.sampleLayerSpec, decodeMediaSampleSpec(recipe, "content", "timed", undefined, true)),+ spec: bind("spec", mediaTrackTypes.itemSpec, decodeMediaItemSpec(recipe, { id, motion: { sustain: [] } })),+ });+ records.push({ id, type: performanceTypes.style, value: { kind: "inline", value: canonicalize(style) }, range: element.range });+ return { records, components: [], fragments: [] };+};++export const decodePerformanceTrackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {+ assertAttributes(element, ["id", "timeline", "canvas"]);+ const id = textAttribute(element, "id"), context = resolveTemporalContext({ element, resolveReference });+ const canvas = reference(element, "canvas", spatialTypes.canvas, resolveReference);+ const records: SurfaceRecordDraft[] = [{ id: `${id}.header`, type: performanceTypes.header, value: { kind: "inline", value: { id } }, range: element.range }];+ const components: SurfaceComponentDraft[] = [];+ const fragments = [] as ReturnType<typeof sealGraphFragment>[];+ const collectorInputs = [{ name: "timeline", type: timelineTypes.track }, { name: "header", type: performanceTypes.header }];+ const bindings: Record<string, SurfaceResolvedReference["ref"]> = { timeline: context.timeline.ref, header: { kind: "record", id: `${id}.header` } };+ const operations: FragmentOperation[] = [{ id: "empty", producer: performanceProducers.create, inputs: {}, result: { kind: "output", name: "set" } }];+ let previous = "empty", index = 0;+ for (const child of element.children) {+ if (child.kind === "text") { if (child.value.trim()) throw new Error("Performance Track accepts Use children."); continue; }+ if (child.name.split(":").at(-1) !== "Use") throw new Error("Performance Track accepts Use children.");+ assertAttributes(child, ["id", "style", ...temporalWindowAttributeNames]); assertEmptyElement(child);+ const useId = optionalTextAttribute(child, "id") ?? `${id}.use.${index + 1}`;+ index += 1;+ const hasTime = temporalWindowAttributeNames.some(name => child.attributes[name] !== undefined);+ const temporal = createTemporalWindowProjection({ id: useId, ...context, resolveReference,+ element: hasTime ? child : { ...child, attributes: { ...child.attributes, during: "program" } } });+ records.push(...temporal.records); components.push(...temporal.components); fragments.push(...temporal.fragments);+ const value = authored<PerformanceStyle>(reference(child, "style", performanceTypes.style, resolveReference));+ const style = performanceStyle(value.fragment, value.bindings);+ fragments.push(style.fragment);+ const visualId = `${useId}.visual`;+ components.push({ id: `${useId}.__style`, fragment: style.fragment.id,+ inputs: { ...Object.fromEntries(Object.entries(style.bindings).map(([key, bound]) => [key, bound.ref])),+ timeline: context.timeline.ref, canvas: canvas.ref, window: temporal.ref }, outputs: { visual: visualId }, range: child.range });+ const suffix = `use-${operations.length}`;+ collectorInputs.push({ name: `${suffix}-window`, type: temporalTypes.window }, { name: `${suffix}-visual`, type: compositionTypes.visualTrack });+ bindings[`${suffix}-window`] = temporal.ref; bindings[`${suffix}-visual`] = { kind: "component-output", component: `${useId}.__style`, output: "visual" };+ operations.push({ id: suffix, producer: performanceProducers.append, inputs: { set: operation(previous), window: input(`${suffix}-window`), visual: input(`${suffix}-visual`) }, result: { kind: "output", name: "set" } });+ previous = suffix;+ }+ operations.push({ id: "resolve", producer: performanceProducers.resolve, inputs: { set: operation(previous), timeline: input("timeline"), header: input("header") }, result: { kind: "output", name: "visual" } });+ const collector = sealGraphFragment({ inputs: collectorInputs, operations, exports: [{ name: "visual", type: compositionTypes.visualTrack, root: operation("resolve") }, { name: "program", type: performanceTypes.set, root: operation(previous) }] });+ fragments.push(collector);+ components.push({ id, fragment: collector.id, inputs: bindings, outputs: { visual: `${id}.visual`, program: `${id}.program` }, range: element.range });+ return { records, components, fragments, exports: [`${id}.visual`, `${id}.program`] };+};diff --git a/packages/performance/test/performance.test.ts b/packages/performance/test/performance.test.tsnew file mode 100644index 00000000..fa4cabd9--- /dev/null+++ b/packages/performance/test/performance.test.ts@@ -0,0 +1,85 @@+import assert from "node:assert/strict";+import test from "node:test";+import { sealVisualTrack, sealComposition } from "@hypit/composition";+import { compileHyperframesDocument } from "@hypit/hyperframes";+import { resolvePerformance, appendPerformanceUse, ordinaryPerformance, performanceStyle, decodePerformanceTrackSurface } from "../src/index.js";+import { timelineFixture } from "../../../test/timeline-fixture.js";+import { projectProgramWindow } from "../../../test/temporal-fixture.js";+import { decodeMediaItemSpec, decodeMediaSampleSpec, decodeMediaFit } from "@hypit/media-track";+import type { Timeline } from "@hypit/timeline";+import type { StructuredElement, SurfaceResolvedReference } from "@hypit/markup";+import { timelineTypes } from "@hypit/timeline";+import { spatialTypes } from "@hypit/spatial";+import { performanceTypes } from "../src/manifest.js";+import { ordinaryPerformanceFragment } from "../src/surface.js";+import { mediaTrackTypes } from "@hypit/media-track";+import { canonicalize } from "@hypit/protocol";+import { movingFrame, crossfade } from "../../../examples/semantic-composition/packages/performance-styles/src/render.js";++const timeline: Timeline = { id: "film", durationSec: 6, frameRate: { numerator: 30, denominator: 1 }, items: [] };+const window = (id: string,start: number,end: number, source=timeline) => projectProgramWindow({ itemId: id, semantic: source,+ projection: { start: { ref: "absolute", at: { unit: "frames", value: start } }, end: { ref: "absolute", at: { unit: "frames", value: end } } } });+const picture = (id: string,start=0,end=180) => sealVisualTrack({ id, visualIr: "hypit.visual-ir@1",programSpaceId: timeline.id,+ presents: [{ id,span: {startFrame:start,endFrameExclusive:end},stacking: {order:0,tieBreak:id},elements:[{id:"root",kind:"box",order:0,style:[{name:"background",value:"red"}]}] }] });+const canvas = { widthPx: 64,heightPx:64, origin: "top-left", xDirection: "right", yDirection: "down", pixelAspect: "square" } as const;+const frame = {xPx:0,yPx:0,widthPx:64,heightPx:64 };+const recipe = { path: "look", properties: { "stack-order": 0, fit: "cover", clip:"frame" } };+const ordinary = (source: Timeline) => ordinaryPerformance({ timeline:source,canvas,frame,window:window("use",0,180,source),+ fit:decodeMediaFit(recipe),sample:decodeMediaSampleSpec(recipe,"content","timed",undefined,true),spec:decodeMediaItemSpec(recipe,{id:"template",motion:{sustain:[]}}) });++test("Use overrides preserve original spans and an empty winner does not reveal earlier content", () => {+ let set = appendPerformanceUse({uses:[]},window("base",0,180),picture("base"));+ set = appendPerformanceUse(set,window("side",60,180),picture("side",60));+ set = appendPerformanceUse(set,window("move",60,90),picture("move",60,90));+ const empty = sealVisualTrack({id:"hidden",visualIr:"hypit.visual-ir@1",programSpaceId:timeline.id,presents:[]});+ set = appendPerformanceUse(set,window("hide",120,150),empty);+ const result = resolvePerformance(timeline,"presenter",set);+ assert.deepEqual(result.presents.map(p=>[p.span,p.visibility]),[+ [{startFrame:0,endFrameExclusive:180},[{startFrame:0,endFrameExclusive:60}]],+ [{startFrame:60,endFrameExclusive:90},[{startFrame:60,endFrameExclusive:90}]],+ [{startFrame:60,endFrameExclusive:180},[{startFrame:90,endFrameExclusive:120},{startFrame:150,endFrameExclusive:180}]],+ ]);+ const document = compileHyperframesDocument(sealComposition({id:"render",canvas:{width:64,height:64,clearColor:"#000000"},tracks:[result]}),timeline);+ assert.match(document.html,/data-hypit-visibility/);+ assert.match(document.html,/hf-seek/);+});++test("ordinary Performance supports no Takes, audio-only Takes, overlap and gaps without resampling origins", () => {+ assert.equal(ordinary(timeline).presents.length,0);+ const audio = timelineFixture(timeline,{segments:[{id:"a",frameCount:90},{id:"b",frameCount:90}]});+ assert.equal(ordinary(audio).presents.length,0);+ const source = { ...audio, items: audio.items.map((item,i)=>({...item,startFrame: i===0 ? 15:60,take:{...item.take,media:{...item.take.media,visual:{ artifact:{kind:"blob" as const,resource:`res_video${i}` as const,size:1,mediaType:"video/mp4"},width:64,height:64}}}})) };+ const result=ordinary(source);+ const videos=result.presents[0]!.elements.filter(e=>e.kind==="video");+ assert.equal(videos.length,2);+ assert.deepEqual(videos.map(v=>(v.kind === "video" ? v.sampling : undefined)?.segments[0]?.target),[{startFrame:15,endFrameExclusive:105},{startFrame:60,endFrameExclusive:150}]);+ assert.deepEqual(videos.map(v=>(v.kind === "video" ? v.sampling : undefined)?.segments[0]?.sourceFrame),[{numerator:0,denominator:1},{numerator:0,denominator:1}]);+ const excerpt=(id:string)=>({kind:"segment" as const,narrativeId:source.narrativeId,id,tokenStart:0,tokenEndExclusive:0});+ const mixed=crossfade(source,window("mix",60,105,source),excerpt("a"),excerpt("b"),frame);+ assert.equal(mixed.presents[0]!.elements.filter(e=>e.kind==="video").length,2);+ const move=movingFrame(source,window("move",30,120,source),frame,{...frame,xPx:32,widthPx:32});+ const doc=compileHyperframesDocument(sealComposition({id:"movement",canvas:{width:64,height:64,clearColor:"#000000"},tracks:[move]}),source);+ assert.match(doc.html,/source-0/);+ assert.match(doc.html,/viewport/);+ const single={...source,items:[source.items[0]!,{...source.items[1]!,take:audio.items[1]!.take}]};+ assert.equal(crossfade(single,window("mix",60,105,single),excerpt("a"),excerpt("b"),frame).presents[0]!.elements.filter(e=>e.kind==="video").length,1);+ assert.throws(()=>crossfade(source,window("mix",60,105,source),excerpt("missing"),excerpt("b"),frame),/Unknown/);+});++test("Style-bound inputs become explicit graph edges at each Use; a baseline needs no temporal selector", async () => {+ const reference=(id:string,type:SurfaceResolvedReference["type"]):SurfaceResolvedReference=>({path:id,type,ref:{kind:"record",id}});+ const bindings={frame:reference("frame",spatialTypes.frame),fit:reference("fit",spatialTypes.fit),sample:reference("sample",mediaTrackTypes.sampleLayerSpec),spec:reference("spec",mediaTrackTypes.itemSpec)};+ const style=performanceStyle(ordinaryPerformanceFragment,bindings);+ const refs = new Map<string, SurfaceResolvedReference>([+ ["timeline", reference("timeline",timelineTypes.track)],+ ["canvas", reference("canvas",spatialTypes.canvas)],+ ["style", { ...reference("style",performanceTypes.style), record: { id:"style", type:performanceTypes.style,+ value:{kind:"inline",value:canonicalize(style)} } }],+ ]);+ const range={source:"test.svml",start:0,end:1};+ const root:StructuredElement={kind:"element",name:"performance:Track",attributes:{id:"p",timeline:{kind:"reference",path:"timeline"},canvas:{kind:"reference",path:"canvas"}},range,children:[{kind:"element",name:"performance:Use",attributes:{style:{kind:"reference",path:"style"}},children:[],range}]};+ const result=await decodePerformanceTrackSurface({element:root,sourceName:"test.svml",resolveReference:path=>refs.get(path),resolveAsset:async()=>{throw new Error("unexpected");}});+ assert.equal(result.components.find(c=>c.id==="p.use.1.__style")?.inputs.frame?.kind,"record");+ assert.deepEqual(result.components.find(c=>c.id==="p.use.1.__style")?.inputs.frame,bindings.frame.ref);+ assert.deepEqual(result.components.at(-1)?.outputs,{visual:"p.visual",program:"p.program"});+});diff --git a/packages/program-space/README.md b/packages/program-space/README.mdindex df386df5..db249aee 100644--- a/packages/program-space/README.md+++ b/packages/program-space/README.md@@ -1,20 +1,11 @@# `@hypit/program-space`-The time and frame domain shared by a compiled film. ProgramSpace contains `id`, `durationSec` and-`frameRate`. Script identity belongs to semantic data; the physical film clock also serves animation-with no speech or source media.--For spoken work, SemanticTrack projects the selected performance's real duration and frame rate.-For authored animation, `Space` declares them directly:--```svml-<import as="time" from="@hypit/program-space@1"/>-<time:Space id="animation" frame-rate="30" duration="8s"/>-```--`duration` accepts seconds, milliseconds or frames (`8s`, `8000ms`, `240f`); the result must end on an-exact frame boundary. The rate may be rational, such as `30000/1001`. `Space` publishes an ordinary-inline ProgramSpace value and performs no media generation.--`Clock` is the duration-free frame-rate input for media normalization. A clock does not prescribe-how long a performance will be; its actual normalized material establishes that later.+`ProgramClock` supplies a positive rational frame rate before material exists. The `Clock` Surface+publishes it for normalization and Timeline assembly; the same Clock Surface is available from+`@hypit/timeline-author`.++`ProgramSpace` is the lightweight physical range consumed by rendering: `id`, `durationSec` and+`frameRate`. Its positive duration ends on an exact frame boundary. The Timeline projects this+view, whether it contains speaking Takes, wordless media, gaps or no Takes at all. Authors declare+that complete work through [Timeline](../timeline-author/README.md), then components obtain the+range and any semantic evidence they need. The range itself carries no media or word table.diff --git a/packages/program-space/src/activation.ts b/packages/program-space/src/activation.tsindex d8c2d877..d7511bb7 100644--- a/packages/program-space/src/activation.ts+++ b/packages/program-space/src/activation.ts@@ -1,6 +1,6 @@import { createMarkupSurfaceHostFacet } from "@hypit/markup";-import { decodeClockSurface, decodeSpaceSurface } from "./surface.js";+import { decodeClockSurface } from "./surface.js";import { programSpaceManifest, programSpaceMarkupSurfaces, programSpaceModuleRef } from "./index.js";export const hypitPackage = {@@ -10,10 +10,6 @@ export const hypitPackage = {module: programSpaceModuleRef,declaration: programSpaceMarkupSurfaces.find((item) => item.name === "clock")!,handler: decodeClockSurface,- }), createMarkupSurfaceHostFacet({- module: programSpaceModuleRef,- declaration: programSpaceMarkupSurfaces.find((item) => item.name === "space")!,- handler: decodeSpaceSurface,})],};export default hypitPackage;diff --git a/packages/program-space/src/index.ts b/packages/program-space/src/index.tsindex 9abc2eb5..da941a3d 100644--- a/packages/program-space/src/index.ts+++ b/packages/program-space/src/index.ts@@ -40,17 +40,6 @@ export const programSpaceManifest: ModuleManifest = {export const programSpaceDependency = { module: programSpaceModuleRef } as const;export const programSpaceMarkupSurfaces = [{ name: "clock", tag: "Clock", mode: "structured", outputs: [programSpaceTypes.clock] },- { name: "space", tag: "Space", mode: "structured", outputs: [programSpaceTypes.programSpace],- vocabulary: {- summary: "Declares a film's frame rate and authored duration, for animation whose timing is directed without a performance.",- attributes: [- { name: "id", kind: "identifier", required: true, summary: "Names the shared film time axis." },- { name: "frame-rate", kind: "literal", required: true, summary: "Frames per second, such as 30 or 30000/1001." },- { name: "duration", kind: "literal", required: true, summary: "Total duration in seconds, milliseconds or frames, such as 8s or 240f; ends on a frame boundary." },- ],- example: '<time:Space id="animation" frame-rate="30" duration="8s"/>',- },- },] as const;export function sealProgramClock(value: ProgramClock): ProgramClock {@@ -105,3 +94,5 @@ export function assertProgramSpaceIdentity(programSpace: ProgramSpace): void {}programSpaceFrameCount(programSpace);}++export { decodeClockSurface } from "./surface.js";diff --git a/packages/program-space/src/surface.ts b/packages/program-space/src/surface.tsindex 7b064749..3f0e3999 100644--- a/packages/program-space/src/surface.ts+++ b/packages/program-space/src/surface.ts@@ -5,7 +5,7 @@ import {type StructuredSurfaceHandler,} from "@hypit/markup";-import { assertProgramSpaceIdentity, programSpaceTypes, sealProgramClock, type ProgramClock } from "./index.js";+import { programSpaceTypes, sealProgramClock, type ProgramClock } from "./index.js";function frameRate(element: StructuredElement): ProgramClock["frameRate"] {const value = text(element, "frame-rate");@@ -34,22 +34,3 @@ export const decodeClockSurface: StructuredSurfaceHandler = ({ element }) => {fragments: [],};};--export const decodeSpaceSurface: StructuredSurfaceHandler = ({ element }) => {- const expected = ["duration", "frame-rate", "id"];- if (Object.keys(element.attributes).sort().join("\0") !== expected.join("\0")) {- throw new Error(`${element.name} requires exactly id, frame-rate and duration.`);- }- empty(element);- const id = text(element, "id");- const rate = frameRate(element);- const duration = text(element, "duration");- const match = /^(\d+(?:\.\d+)?)(s|ms|f)$/u.exec(duration);- if (match === null) throw new Error(`${element.name}.duration must be a duration such as 8s, 8000ms or 240f.`);- const value = Number(match[1]);- const durationSec = match[2] === "f" ? value * rate.denominator / rate.numerator- : match[2] === "ms" ? value / 1000 : value;- const space = { id, durationSec, frameRate: rate };- assertProgramSpaceIdentity(space);- return { records: [{ id, type: programSpaceTypes.programSpace, value: { kind: "inline", value: space }, range: element.range }], components: [], fragments: [] };-};diff --git a/packages/provider-hyperframes-local/test/alpha.test.ts b/packages/provider-hyperframes-local/test/alpha.test.tsindex 627ad4cc..34b88457 100644--- a/packages/provider-hyperframes-local/test/alpha.test.ts+++ b/packages/provider-hyperframes-local/test/alpha.test.ts@@ -1,3 +1,4 @@+import { appendMediaPerformance } from "@hypit/performance";import assert from "node:assert/strict";import test from "node:test";import { mkdir, mkdtemp, rm } from "node:fs/promises";@@ -8,16 +9,16 @@ import { compileHyperframesDocument } from "@hypit/hyperframes";import type { Narrative } from "@hypit/narrative";import { parseScript, narrativeValue } from "@hypit/script";import { materializeSemanticTake } from "@hypit/speech";-import { appendSpeechTrackTake, assembleSpeechTrack, createSpeechTrackSet,- sealSpeechTrackHeader } from "@hypit/speech-track";-import { projectSemanticProgramSpace } from "@hypit/semantic-track";-import { appendMediaPerformance, appendMediaItem, appendTimedMediaLayer, createMediaLayerSet, createMediaSoundSet, createMediaTrackSet,+import { appendTimelineAuthorTake, assembleTimelineAuthor, createTimelineAuthorSet,+ sealTimelineAuthorHeader } from "@hypit/timeline-author";+import { projectTimelineSpace } from "@hypit/timeline";+import { appendMediaItem, appendTimedMediaLayer, createMediaLayerSet, createMediaSoundSet, createMediaTrackSet,finalizeMediaTrack, projectMediaVisualTrack, sealMediaItemSpec, sealMediaSampleLayerSpec, sealMediaTrackHeader } from "@hypit/media-track";import { projectProgramWindow } from "../../../test/temporal-fixture.js";import { MemoryResourceStore, ffmpegBytes, normalizeTestVideo, transparentVideoFixture, writeTestArtifact } from "../../../test/alpha-video-fixture.js";import { renderHyperframesVisual } from "../src/index.js";-test("transparent normalized media composites through both SemanticTake/Speech Track and Media Track in Chrome", {+test("transparent normalized media composites through both SemanticTake/Timeline and Media Track in Chrome", {skip: process.env.HYPIT_BROWSER_TESTS !== "1",}, async () => {const directory = process.env.HYPIT_ALPHA_PROOF_DIR ?? await mkdtemp(join(tmpdir(), "hypit-alpha-render-"));@@ -44,9 +45,9 @@ test("transparent normalized media composites through both SemanticTake/Speech Tconst fit = { sizing: "contain" as const, framePoint: { x: 0.5, y: 0.5 }, contentPoint: { x: 0.5, y: 0.5 },offsetPx: { x: 0, y: 0 }, constraint: "bounded" as const };const frame = { xPx: 0, yPx: 0, widthPx: 96, heightPx: 64 };- const set = appendSpeechTrackTake(createSpeechTrackSet(), take);- const semantic = assembleSpeechTrack(sealSpeechTrackHeader({ id: "speech" }), set);- const space = projectSemanticProgramSpace(semantic);+ const set = appendTimelineAuthorTake(createTimelineAuthorSet(), take);+ const semantic = assembleTimelineAuthor(sealTimelineAuthorHeader({ id: "speech" }), set, { frameRate: media.timeline.frameRate });+ const space = semantic;const header = sealMediaTrackHeader({ id: "broll" });const layers = appendTimedMediaLayer(createMediaLayerSet(), broll, fit, sealMediaSampleLayerSpec({ id: "cutout",occupancy: { mode: "once", align: "start" },@@ -62,11 +63,11 @@ test("transparent normalized media composites through both SemanticTake/Speech T}));const performanceHeader = sealMediaTrackHeader({ id: "performance" });const performanceSpec = { ...spec, id: "performance", stackingOrder: 1 };- const performance = appendMediaPerformance({ set: createMediaTrackSet(), header: performanceHeader, space,+ const performance = appendMediaPerformance({ set: createMediaTrackSet(), header: performanceHeader, timeline: semantic,canvas: { widthPx: 192, heightPx: 64, origin: "top-left", xDirection: "right", yDirection: "down", pixelAspect: "square" },layers: createMediaLayerSet(), frame, spec: performanceSpec, sounds: createMediaSoundSet(),window: projectProgramWindow({ itemId: performanceSpec.id, semantic, projection: { start: { ref: "program.start" }, end: { ref: "program.end" } } }),- }, semantic, fit, sealMediaSampleLayerSpec({ id: "content", occupancy: { mode: "once", align: "start" },+ }, fit, sealMediaSampleLayerSpec({ id: "content", occupancy: { mode: "once", align: "start" },appearance: { opacity: 1, filter: { blurPx: 0, brightness: 1, contrast: 1, saturation: 1 } } }));const speechVisual = projectMediaVisualTrack(space, finalizeMediaTrack(performance, performanceHeader, space));const brollVisual = projectMediaVisualTrack(space, finalizeMediaTrack(items, header, space));diff --git a/packages/provider-hyperframes-local/test/performance.test.ts b/packages/provider-hyperframes-local/test/performance.test.tsnew file mode 100644index 00000000..198848ac--- /dev/null+++ b/packages/provider-hyperframes-local/test/performance.test.ts@@ -0,0 +1,59 @@+import assert from "node:assert/strict";+import test from "node:test";+import { spawnSync } from "node:child_process";+import { mkdtemp, readFile, writeFile, rm } from "node:fs/promises";+import { tmpdir } from "node:os";+import { join } from "node:path";+import { MemoryResourceStore } from "@hypit/driver-node";+import { sealComposition, sealVisualTrack } from "@hypit/composition";+import { compileHyperframesDocument } from "@hypit/hyperframes";+import { appendPerformanceUse, resolvePerformance } from "@hypit/performance";+import { timelineFixture } from "../../../test/timeline-fixture.js";+import { projectProgramWindow } from "../../../test/temporal-fixture.js";+import { movingFrame, crossfade } from "../../../examples/semantic-composition/packages/performance-styles/src/render.js";+import { renderHyperframesVisual } from "../src/index.js";++test("project Performance Styles render movement, empty override, resumed playback, tail and two-source blending", async () => {+ const root=await mkdtemp(join(tmpdir(),"hypit-performance-"));+ try {+ const resources=new MemoryResourceStore();+ const makeVideo=async(name:string,color:number[])=>{+ const path=join(root,`${name}.mp4`);+ const raw=Buffer.from(Array.from({length:8*64*64},()=>color).flat());+ const result=spawnSync("ffmpeg",["-v","error","-y","-f","rawvideo","-pix_fmt","rgb24","-s","64x64","-r","30","-i","pipe:0","-c:v","libx264","-crf","0","-pix_fmt","yuv420p",path],{input:raw});+ assert.equal(result.status,0,result.stderr.toString());+ return resources.put(await readFile(path),"video/mp4");+ };+ const a=await makeVideo("red",[220,20,20]),b=await makeVideo("blue",[20,20,220]);+ const fixture=timelineFixture({id:"film",durationSec:16/30,frameRate:{numerator:30,denominator:1}},{segments:[{id:"a",frameCount:8},{id:"b",frameCount:8}]});+ const timeline={...fixture,durationSec:14/30,items:fixture.items.map((item,i)=>({...item,startFrame:i*4,take:{...item.take,media:{timeline:item.take.media.timeline,visual:{artifact:i===0?a:b,width:64,height:64}}}}))};+ const window=(id:string,start:number,end:number)=>projectProgramWindow({itemId:id,semantic:timeline,projection:{start:{ref:"absolute",at:{unit:"frames",value:start}},end:{ref:"absolute",at:{unit:"frames",value:end}}}});+ const full={xPx:0,yPx:0,widthPx:64,heightPx:64},side={...full,xPx:32,widthPx:32};+ const base=window("base",0,14),settled=window("side",4,14),move=window("move",4,8),hide=window("hide",8,9);+ let set=appendPerformanceUse({uses:[]},base,movingFrame(timeline,base,full,full));+ set=appendPerformanceUse(set,settled,movingFrame(timeline,settled,side,side));+ set=appendPerformanceUse(set,move,movingFrame(timeline,move,full,side));+ set=appendPerformanceUse(set,hide,sealVisualTrack({id:"hidden",programSpaceId:timeline.id,visualIr:"hypit.visual-ir@1",presents:[]}));+ const render=async(name:string,track:ReturnType<typeof resolvePerformance>)=>{+ const document=compileHyperframesDocument(sealComposition({id:name,canvas:{width:64,height:64,clearColor:"#000000"},tracks:[track]}),timeline);+ const video=await renderHyperframesVisual({document},{resources,workers:1,quality:"high",processTimeoutMs:120000});+ const file=join(root,`${name}.mp4`);await writeFile(file,(await resources.get(video.artifact.resource))!);+ const decoded=spawnSync("ffmpeg",["-v","error","-i",file,"-f","rawvideo","-pix_fmt","rgb24","pipe:1"]);+ assert.equal(decoded.status,0,decoded.stderr.toString());return decoded.stdout;+ };+ const pixels=await render("move",resolvePerformance(timeline,"presenter",set));+ const pixel=(data:Buffer,f:number,x:number)=>[...data.subarray((f*64*64+32*64+x)*3,(f*64*64+32*64+x)*3+3)];+ assert.ok(pixel(pixels,2,16)[0]!>170,"initial full frame");+ assert.ok(pixel(pixels,7,16).every(c=>c<25),"moving view reaches right side");+ assert.ok(pixel(pixels,7,48)[2]!>170,"incoming footage remains visible");+ assert.ok(pixel(pixels,8,48).every(c=>c<25),"empty Use suppresses underlying rules");+ assert.ok(pixel(pixels,9,48)[2]!>170,"settled view resumes");+ assert.ok(pixel(pixels,12,48).every(c=>c<25),"tail requires no fabricated clip");+ const excerpt=(id:string)=>({kind:"segment" as const,narrativeId:fixture.narrativeId,id,tokenStart:0,tokenEndExclusive:0});+ const mix=crossfade(timeline,move,excerpt("a"),excerpt("b"),full);+ const blended=await render("mix",mix);+ assert.ok(pixel(blended,4,32)[0]!>170,"outgoing endpoint");+ assert.ok(pixel(blended,7,32)[2]!>170,"incoming endpoint");+ assert.ok(pixel(blended,5,32)[0]!>90 && pixel(blended,5,32)[2]!>40,"both sources contribute inside overlap");+ } finally { await rm(root,{recursive:true,force:true}); }+});diff --git a/packages/provider-media-local/README.md b/packages/provider-media-local/README.mdindex 11667791..6625557f 100644--- a/packages/provider-media-local/README.md+++ b/packages/provider-media-local/README.md@@ -37,7 +37,7 @@ These are this FFmpeg implementation's intermediate encodings; Source and Trackto use ordinary SynchronizedMedia. Transform currently emits opaque MP4, so perform trim/retimebefore matting when the resulting clip needs transparency.-The result is ordinary SynchronizedMedia. Speech Track and Media Track consume its transparent+The result is ordinary SynchronizedMedia. Timeline assembly and Media Track consume its transparentpicture through their existing visual outputs; choosing the A-roll or B-roll role belongs to theSource. Local HyperFrames extracts alpha-preserving PNGs and composites them against the authoredCanvas and lower visual layers before encoding the final MP4.diff --git a/packages/provider-media-local/test/provider.test.ts b/packages/provider-media-local/test/provider.test.tsindex fd46e53c..e13d226a 100644--- a/packages/provider-media-local/test/provider.test.ts+++ b/packages/provider-media-local/test/provider.test.ts@@ -903,3 +903,43 @@ test("audio range preserves loop phase, tempo and intersected fades from the fulassert.ok(maximumError <= 1, `Selected audio differs by ${maximumError} PCM units`);} finally { await rm(root, { recursive: true, force: true }); }});++test("audio envelopes and audible regions preserve sample phase through gaps and range renders", { skip: !hasMediaBinaries }, async () => {+ const root = await mkdtemp(join(tmpdir(), "hypit-audio-presentation-"));+ try {+ const resources = new MemoryResourceStore();+ const source = await resources.put(rampWav(4800), "audio/wav");+ const plan = sealAudioProgramPlan({+ frameRate: { numerator: 30, denominator: 1 }, frameCount: 12, sampleRate: 48000, sampleFrames: 19200,+ clips: [{ id: "voice", artifact: source, targetStartSample: 1600, targetEndSampleExclusive: 17600,+ sourceSampleFrames: 4800, sourceStartSample: 0, sourceEndSampleExclusive: 4800,+ sourceLoop: true, sourcePhaseSample: 0, playbackRate: 1, pitch: "preserve", gain: 1,+ fadeInSamples: 0, fadeOutSamples: 0,+ gainEnvelope: [{ sample: 1600, gain: 0 }, { sample: 17600, gain: 1 }],+ audibility: [{ startSample: 1600, endSampleExclusive: 4800 }, { startSample: 8000, endSampleExclusive: 17600 }],+ }], mix: { normalize: false, limiter: "none" },+ });+ const pcm = async (name: string, range?: { startFrame: number; endFrameExclusive: number }) => {+ const value = await fulfillInline(resources, need(`need:${name}`, mediaPipelineCapabilities.renderAudio,+ mediaTypes.timelineAudio, canonicalize({ plan, ...(range === undefined ? {} : { range }) })));+ verifyTimelineAudio(value);+ const path = join(root, `${name}.wav`), raw = join(root, `${name}.raw`);+ await writeFile(path, (await resources.get((value as unknown as TimelineAudio).artifact.resource))!);+ await run("ffmpeg", ["-v", "error", "-y", "-i", path, "-f", "s16le", raw]);+ return readFile(raw);+ };+ const full = await pcm("full");+ for (const sample of [0, 1599, 4800, 6400, 7999, 17600, 18000]) assert.equal(full.readInt16LE(sample * 4), 0);+ for (const sample of [1600, 2401, 4799, 8000, 8001, 12000, 17599]) {+ const local = sample - 1600;+ const expected = (local % 4800 + 1) * local / 16000;+ assert.ok(Math.abs(full.readInt16LE(sample * 4) - expected) <= 1, `Wrong envelope or source phase at ${sample}`);+ }+ const selected = await pcm("selected", { startFrame: 4, endFrameExclusive: 10 });+ const expected = full.subarray(6400 * 4, 16000 * 4);+ assert.equal(selected.length, expected.length);+ for (let index = 0; index < selected.length; index += 2) {+ assert.ok(Math.abs(selected.readInt16LE(index) - expected.readInt16LE(index)) <= 1);+ }+ } finally { await rm(root, { recursive: true, force: true }); }+});diff --git a/packages/ranking-studio/README.md b/packages/ranking-studio/README.mdindex 7e4d8f65..05dce9c3 100644--- a/packages/ranking-studio/README.md+++ b/packages/ranking-studio/README.md@@ -2,6 +2,10 @@Hypit Studio Companion for the Ranking module's Column, Tier and Top Three surfaces.+Each surface has separate visual and audio Companion declarations in this one+package. Film chooses `.visual` and `.audio` independently; selecting the visual+output does not connect its sound.+The root lane represents the board; `attachments` expose reveal or activationevents on a detail lane. Both use Studio's ordinary item selection and overlapbehavior. Their time ranges remain the authored event ranges.@@ -12,3 +16,17 @@ entering or settled icon selects the same detail entity. The board and presetitems without a detail event remain associated with the root. Studio measuresthe visible parts at the current frame; the Companion does not duplicate motiongeometry or infer identities from rendered ID strings.++The audio Companion presents a `Ranking Sounds` lane. Each emitted clip shows+its item and phase (`Appear` or `Move`) with a waveform. It joins the board's+exported `.events` to the rendered audio clips by exact event identity, so a+phase without a supplied sound contributes no audio item. `Move` is the motion+into the board, not an exit sound.++Selecting a sound exposes the event trigger as a read-only fact and the shared+Ranking Style's sound gains and fade-in length as editable fields. Gain is+displayed as a percentage; edits write the original linear gain to the Style's+recipe. These fields affect every matching event using that recipe. Event+timing comes from the Ranking animation; changing the visual event timing+updates its sound too. The visual Companion owns layout and motion controls,+while the audio Companion owns the sound controls.diff --git a/packages/ranking-studio/src/index.ts b/packages/ranking-studio/src/index.tsindex e4aa2fb1..af79fe17 100644--- a/packages/ranking-studio/src/index.ts+++ b/packages/ranking-studio/src/index.ts@@ -1,7 +1,8 @@import { openFontStudioFields } from "@hypit/fonts-open/studio";import { rankingMarkupSurfaces, rankingModuleRef, rankingTypes } from "@hypit/ranking";-import type { RankingProgram, RankingSchedule } from "@hypit/ranking";+import type { RankingProgram, RankingSchedule, RankingSoundEventPlan } from "@hypit/ranking";import { compositionTypes } from "@hypit/composition";+import type { AudioTrack } from "@hypit/composition";import type { StudioTrackCompanion, StudioTrackCompanionContext, StudioEntityDraft, StudioInspectorFieldDeclaration, StudioSourceBindingDeclaration } from "@hypit/studio-adapter";import { artifactPreview, authoredChildFor, previewLayer, requiredSurfaceValue, temporalLineageFor, temporalSemanticSource } from "@hypit/studio-adapter";@@ -13,9 +14,12 @@ const frameParameters: readonly StudioSourceBindingDeclaration[] = [...["left", "top", "right", "bottom", "x", "y", "width", "height"].map((name) => ({ name, writable: true })),];-function rankingProperties(surface: "column-style" | "tier-style" | "top-three-style") {- return rankingMarkupSurfaces.find((candidate) => candidate.name === surface)- ?.vocabulary.attributes.find((attribute) => attribute.name === "recipe")?.recipe ?? [];+const soundProperties = new Set(["appear-gain", "move-gain", "sound-fade-frames"]);++function rankingProperties(surface: "column-style" | "tier-style" | "top-three-style", audio = false) {+ return (rankingMarkupSurfaces.find((candidate) => candidate.name === surface)+ ?.vocabulary.attributes.find((attribute) => attribute.name === "recipe")?.recipe ?? [])+ .filter((property) => soundProperties.has(property.name) === audio);}function title(name: string): string {@@ -92,11 +96,11 @@ function rankingInspector(surface: "column-style" | "tier-style" | "top-three-st})];}-function rankingStyle(surface: "column-style" | "tier-style" | "top-three-style"): StudioSourceBindingDeclaration {+function rankingStyle(surface: "column-style" | "tier-style" | "top-three-style", audio = false): StudioSourceBindingDeclaration {return {name: "style",- referenced: [fontInspector.binding],- recipe: { through: ["recipe"], bindings: rankingProperties(surface).map(({ name, schema, fallback }) => ({+ ...(audio ? {} : { referenced: [fontInspector.binding] }),+ recipe: { through: ["recipe"], bindings: rankingProperties(surface, audio).map(({ name, schema, fallback }) => ({name,schema,...(fallback === undefined ? {} : { fallback }),@@ -159,10 +163,57 @@ function projectRanking(context: StudioTrackCompanionContext): readonly StudioEnconst commonBindings: readonly StudioSourceBindingDeclaration[] = [{ name: "frame", referenced: frameParameters },- { name: "appear-sound" },- { name: "move-sound" },];+function projectRankingAudio(context: StudioTrackCompanionContext): readonly StudioEntityDraft[] {+ const events = requiredSurfaceValue(context, "events") as RankingSoundEventPlan;+ const byId = new Map(events.events.map((event) => [event.id, event]));+ const clips = new Map((context.track.value as AudioTrack).clips.map((clip) => [clip.id, clip]));+ const boardId = context.placement?.id ?? context.track.outputRef;+ return context.spans.map((span): StudioEntityDraft => {+ const event = byId.get(span.id);+ const clip = clips.get(span.id);+ if (event === undefined || clip === undefined) throw new Error(`Ranking sound ${span.id} has no event or audio clip.`);+ const child = authoredChildFor(context, event.itemId, [rankingTypes.itemSpec, rankingTypes.textItemShell]);+ const label = child?.attributes.label ?? event.itemId;+ const phase = event.kind === "appear" ? "Appear" : "Move";+ return {+ id: `${context.track.outputRef}:entity:${event.id}`,+ authoredId: boardId,+ display: { title: `${label} · ${phase}`, layers: [previewLayer(artifactPreview("audio", clip.artifact.resource), "waveform")] },+ startFrame: span.startFrame, endFrameExclusive: span.endFrameExclusive, stackOrder: span.stackOrder,+ ...(context.placement === undefined ? {} : { elementRange: context.placement.range }),+ presentation: { entity: "ranking-sound", chrome: "standard" },+ inspector: [{+ id: "trigger", label: "Trigger", domain: "when", page: { id: "sound", label: "Sound" }, section: { id: "event", label: "Event" },+ value: `${label} · ${phase} · ${event.frame}f`,+ summary: "Follows the Ranking item's animation event. Adjust its visual timing to move the sound with it.",+ }],+ };+ });+}++function rankingAudioCompanion(surface: "column" | "tier" | "top-three"): StudioTrackCompanion {+ const style = `${surface}-style` as const;+ return {+ id: `${surface}-audio`, role: "track",+ output: { type: compositionTypes.audioTrack, surface, modules: [rankingModuleRef] },+ family: "audio", tone: "green", label: "Ranking Sounds", icon: "waveform",+ lane: { heightPx: 48 }, requiredValues: ["events"],+ bindings: [{ name: "appear-sound" }, ...(surface === "top-three" ? [] : [{ name: "move-sound" }]), rankingStyle(style, true)],+ inspector: rankingProperties(style, true).map((property) => ({+ binding: `style.${property.name}`, label: property.name === "sound-fade-frames" ? "Fade In" : title(property.name),+ ...rankingInspectorPlacement[property.name as keyof typeof rankingInspectorPlacement],+ control: "number",+ ...(property.name === "sound-fade-frames"+ ? { unit: "f", number: { minimum: 0, step: 1 } }+ : { unit: "%", number: { scale: 100, minimum: 0, step: 1 } }),+ summary: "Shared by every matching sound event using this Ranking style.",+ })),+ project: projectRankingAudio,+ };+}+const frameInspector: readonly StudioInspectorFieldDeclaration[] = frameParameters.filter(({ writable }) => writable === true).map(({ name }) => ({@@ -171,6 +222,7 @@ const frameInspector: readonly StudioInspectorFieldDeclaration[] = frameParamete}));export const rankingStudioTrackCompanions: readonly StudioTrackCompanion[] = [+ ...(["column", "tier", "top-three"] as const).map(rankingAudioCompanion),{id: "column", role: "track",output: { type: compositionTypes.visualTrack, surface: "column", modules: [rankingModuleRef] },diff --git a/packages/ranking-studio/test/companion.test.ts b/packages/ranking-studio/test/companion.test.tsnew file mode 100644index 00000000..fd52fac4--- /dev/null+++ b/packages/ranking-studio/test/companion.test.ts@@ -0,0 +1,42 @@+import assert from "node:assert/strict";+import test from "node:test";+import { compositionTypes } from "@hypit/composition";+import type { StudioTrackCompanionContext } from "@hypit/studio-adapter";+import { rankingStudioTrackCompanions } from "../src/index.js";++test("each Ranking surface has independent visual and audio Companions", () => {+ for (const surface of ["column", "tier", "top-three"]) {+ const companions = rankingStudioTrackCompanions.filter((entry) => entry.output.surface === surface);+ assert.equal(companions.length, 2);+ const audio = companions.find((entry) => entry.output.type.name === compositionTypes.audioTrack.name)!;+ const visual = companions.find((entry) => entry.output.type.name === compositionTypes.visualTrack.name)!;+ assert.equal(audio.label, "Ranking Sounds");+ assert.ok(audio.inspector?.some((field) => field.binding === "style.appear-gain" && field.number?.scale === 100));+ assert.ok(visual.inspector?.every((field) => !field.binding.includes("gain") && !field.binding.includes("sound-fade")));+ }+});++test("audio uses exact event identities and emitted clips, with shared recipe ownership", () => {+ const audio = rankingStudioTrackCompanions.find((entry) => entry.id === "tier-audio")!;+ const context = {+ track: { outputRef: "board.audio", value: { clips: [+ { id: "opaque-event-id", artifact: { resource: "res_sound" } },+ ] }, trace: { outputPorts: [{ name: "events", ref: "board.events" }] } },+ placement: { id: "board", range: { start: 0, end: 100 }, children: [+ { id: "player", attributes: { label: "Player One" }, values: [] },+ ] },+ values: new Map([["board.events", { events: [+ { id: "opaque-event-id", itemId: "player", kind: "move", frame: 90 },+ { id: "unprovided-sound", itemId: "player", kind: "appear", frame: 0 },+ ] }]]),+ spans: [{ id: "opaque-event-id", startFrame: 90, endFrameExclusive: 99, stackOrder: 0 }],+ } as unknown as StudioTrackCompanionContext;+ const entities = audio.project!(context);+ assert.equal(entities.length, 1);+ assert.equal(entities[0]!.display.title, "Player One · Move");+ assert.equal(entities[0]!.authoredId, "board");+ assert.deepEqual(entities[0]!.elementRange, context.placement!.range);+ assert.equal(entities[0]!.temporal, undefined);+ assert.equal(entities[0]!.inspector![0]!.value, "Player One · Move · 90f");+ assert.deepEqual([entities[0]!.startFrame, entities[0]!.endFrameExclusive], [90, 99]);+});diff --git a/packages/ranking/README.md b/packages/ranking/README.mdindex 63d88e54..146588ba 100644--- a/packages/ranking/README.md+++ b/packages/ranking/README.md@@ -1,7 +1,7 @@# `@hypit/ranking`-The Track Surface accepts `semantic={speech.semantic}` or `space={animation}`. Semantic context-resolves Script Selections and Moments; authored space supports clock-based animation. Shared `at`+The Track Surface accepts `timeline={program.timeline}`. The same Timeline supports authored+positions and, where prepared Takes supply evidence, Script Selections and Moments. Shared `at`inputs accept a Moment or a time such as `2s`; `at` with `for` produces a Window where required.Ranking boards that follow the argument of a video: introduce an item, give its verdict, and retain@@ -20,12 +20,12 @@ where the selected schedule requires succession. Ending a reveal does not remove## Bind the ranking to the target Script-This excerpt assumes the Script, SemanticTrack, Canvas, Frame and Style have been declared:+This excerpt assumes the Script, Timeline, Canvas, Frame and Style have been declared:```svml<import as="ranking" from="@hypit/ranking@1"/>-<ranking:Column id="priorities" semantic={speech.semantic} canvas={vertical}+<ranking:Column id="priorities" timeline={speech.timeline} canvas={vertical}frame={layout.ranking} during={story.segment.ranking} style={ranking-style}><ranking:ColumnItem rank="1" label="Winner" during={story.selection.winner}/><ranking:ColumnItem rank="2" preset="true" label="Already placed"/>@@ -38,7 +38,9 @@ Window. TopThree instead accepts item `at` projections and a board `terminal` prcan supply the intended trigger.Each board publishes `.schedule`, `.program` and `.visual`. Optional normalized `appear-sound` or-`move-sound` inputs add `.audio` where that form supports them. Include the wanted visual and audio+`move-sound` inputs add `.events` and `.audio` where that form supports them.+The event plan drives the audio from the same appearance and movement timing+as the visual animation. Include the wanted visual and audiooutputs as peers in Film. Labels can be literal or graph Text; Styles are typed SVS Recipes withexact fonts. Use `hypit vocabulary @hypit/ranking --tag Column` (or another declared tag) for itsattributes, children, outputs and configured example.@@ -50,7 +52,7 @@ The following files are included in the Distribution and show how the responsibi| File | What to learn from it || --- | --- || [surface.ts](src/surface.ts) | `rankingSurface` resolves author inputs, projects the outer and item times through `createTemporalWindowProjection` / `createTemporalInstantProjection`, and retains the returned drafts and references. |-| [fragment.ts](src/fragment.ts) | `createRankingFragment` receives ProgramSpace and wires typed content, time, layout and Style inputs into finite operations. |+| [fragment.ts](src/fragment.ts) | `createRankingFragment` receives Timeline and wires typed content, time, layout and Style inputs into finite operations. || [schedule.ts](src/schedule.ts) | Compute reveal, activation and settled spans from the projected times. || [component.ts](src/component.ts) and [render.ts](src/render.ts) | Build the ranking program and produce picture and optional sound from that schedule. || [manifest.ts](src/manifest.ts) and [activation.ts](src/activation.ts) | Publish Types, Producers, Surface vocabulary and package contributions. |diff --git a/packages/ranking/package.json b/packages/ranking/package.jsonindex 9cc4bc49..6a4ffcb5 100644--- a/packages/ranking/package.json+++ b/packages/ranking/package.json@@ -19,7 +19,7 @@"@hypit/narrative": "workspace:*","@hypit/program-space": "workspace:*","@hypit/protocol": "workspace:*",- "@hypit/semantic-track": "workspace:*",+ "@hypit/timeline": "workspace:*","@hypit/spatial": "workspace:*","@hypit/svs": "workspace:*","@hypit/temporal": "workspace:*",diff --git a/packages/ranking/src/component.ts b/packages/ranking/src/component.tsindex 3be13405..14442f37 100644--- a/packages/ranking/src/component.ts+++ b/packages/ranking/src/component.ts@@ -1,6 +1,6 @@+import type { Timeline } from "@hypit/timeline";import type { ComponentPackage, ProducerHandlerContext } from "@hypit/component-kit";import type { SynchronizedMedia } from "@hypit/media";-import type { ProgramSpace } from "@hypit/program-space";import { canonicalize } from "@hypit/protocol";import type { BlobRef, StoredValue } from "@hypit/protocol";import type { TemporalInstant, TemporalWindow } from "@hypit/temporal";@@ -96,7 +96,7 @@ export const rankingComponent = {producer: rankingProducers.schedule,handler: ({ inputs }) => ({ outputs: { schedule: output(buildTriggeredRankingSchedule({header: inline(inputs.header?.value, "RankingHeader"), items: inline(inputs.items?.value, "RankingItemSpecSet"),- space: inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ timeline: inline<Timeline>(inputs.timeline?.value, "Timeline"),outer: inline<TemporalWindow>(inputs.outer?.value, "TemporalWindow"),candidates: inline(inputs.candidates?.value, "TriggeredRankingCandidateSet"),terminal: inline<TemporalInstant>(inputs.terminal?.value, "TemporalInstant"),@@ -119,7 +119,7 @@ export const rankingComponent = {handler: ({ inputs }) => ({ outputs: { schedule: output(buildTierBoardSchedule({header: inline<RankingHeader>(inputs.header?.value, "RankingHeader"),items: inline<RankingItemSpecSet>(inputs.items?.value, "RankingItemSpecSet"),- space: inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ timeline: inline<Timeline>(inputs.timeline?.value, "Timeline"),outer: inline<TemporalWindow>(inputs.outer?.value, "TemporalWindow"),windows: inline<TierBoardWindowSet>(inputs.windows?.value, "TierBoardWindowSet"),})) }, needs: {} }),@@ -141,7 +141,7 @@ export const rankingComponent = {handler: ({ inputs }) => ({ outputs: { schedule: output(buildColumnSchedule({header: inline<RankingHeader>(inputs.header?.value, "RankingHeader"),items: inline<RankingItemSpecSet>(inputs.items?.value, "RankingItemSpecSet"),- space: inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ timeline: inline<Timeline>(inputs.timeline?.value, "Timeline"),outer: inline<TemporalWindow>(inputs.outer?.value, "TemporalWindow"),windows: inline<ColumnWindowSet>(inputs.windows?.value, "ColumnWindowSet"),})) }, needs: {} }),@@ -239,7 +239,7 @@ export const rankingComponent = {{producer: rankingProducers.renderAudio,handler: ({ inputs }) => ({ outputs: { track: output(renderRankingAudio(- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ inline<Timeline>(inputs.timeline?.value, "Timeline"),inline<RankingSoundEventPlan>(inputs.events?.value, "RankingSoundEventPlan"),inline<RankingSoundStyle>(inputs.style?.value, "RankingSoundStyle"),inline<RankingSoundSet>(inputs.sounds?.value, "RankingSoundSet"),@@ -252,7 +252,7 @@ export const rankingComponent = {] as const).map(([producer, render, label]) => ({producer,handler: ({ inputs }: ProducerHandlerContext) => ({ outputs: { track: output(render(- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"), inline(inputs.program?.value, label) as never,+ inline<Timeline>(inputs.timeline?.value, "Timeline"), inline(inputs.program?.value, label) as never,)) }, needs: {} }),})),],diff --git a/packages/ranking/src/fragment.ts b/packages/ranking/src/fragment.tsindex 27d68f1e..f9d85505 100644--- a/packages/ranking/src/fragment.ts+++ b/packages/ranking/src/fragment.ts@@ -1,4 +1,5 @@-import { programSpaceTypes } from "@hypit/program-space";+import { timelineTypes } from "@hypit/timeline";+import { compositionTypes } from "@hypit/composition";import { sealGraphFragment } from "@hypit/elaborator";import type { FragmentOperation, GraphFragment } from "@hypit/elaborator";@@ -56,7 +57,7 @@ export function createRankingFragment(const selected = definition(variant);const inputs: Array<GraphFragment["inputs"][number]> = [{ name: "header", type: rankingTypes.header },- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },{ name: "outer", type: temporalTypes.window },...(variant === "top-three" ? [{ name: "terminal", type: temporalTypes.instant },@@ -133,7 +134,7 @@ export function createRankingFragment(operations.push({id: "schedule",producer: variant === "column" ? rankingProducers.columnSchedule : rankingProducers.tierSchedule,- inputs: { header: input("header"), items: specs, space: input("space"), outer: input("outer"), windows: candidates },+ inputs: { header: input("header"), items: specs, timeline: input("timeline"), outer: input("outer"), windows: candidates },result: { kind: "output", name: "schedule" },});} else {@@ -141,7 +142,7 @@ export function createRankingFragment(id: "schedule",producer: rankingProducers.schedule,inputs: {- header: input("header"), items: specs, space: input("space"),+ header: input("header"), items: specs, timeline: input("timeline"),outer: input("outer"), candidates, terminal: input("terminal"),},result: { kind: "output", name: "schedule" },@@ -160,7 +161,7 @@ export function createRankingFragment(operations.push({id: "visual",producer: selected.render,- inputs: { space: input("space"), program: operation("program") },+ inputs: { timeline: input("timeline"), program: operation("program") },result: { kind: "output", name: "track" },});const hasAudio = sound.appearName !== undefined || sound.moveName !== undefined;@@ -189,7 +190,7 @@ export function createRankingFragment(}operations.push({id: "audio", producer: rankingProducers.renderAudio,- inputs: { space: input("space"), events: operation("events"), style: input("sound-style"), sounds },+ inputs: { timeline: input("timeline"), events: operation("events"), style: input("sound-style"), sounds },result: { kind: "output", name: "track" },});}@@ -200,7 +201,10 @@ export function createRankingFragment({ name: "schedule", type: rankingTypes.schedule, root: operation("schedule") },{ name: "program", type: selected.program, root: operation("program") },{ name: "visual", type: compositionTypes.visualTrack, root: operation("visual") },- ...(hasAudio ? [{ name: "audio", type: compositionTypes.audioTrack, root: operation("audio") }] : []),+ ...(hasAudio ? [+ { name: "events", type: rankingTypes.soundEvents, root: operation("events") },+ { name: "audio", type: compositionTypes.audioTrack, root: operation("audio") },+ ] : []),],});}diff --git a/packages/ranking/src/manifest.ts b/packages/ranking/src/manifest.tsindex 47cbe036..74bab5a7 100644--- a/packages/ranking/src/manifest.ts+++ b/packages/ranking/src/manifest.ts@@ -1,3 +1,4 @@+import { timelineTypes } from "@hypit/timeline";import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";import { readFile } from "node:fs/promises";@@ -5,9 +6,9 @@ import { artifactDependency } from "@hypit/artifact";import { compositionDependency, compositionTypes } from "@hypit/composition";import { mediaDependency, mediaTypes } from "@hypit/media";import { narrativeDependency, narrativeTypes } from "@hypit/narrative";-import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";+import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";-import { semanticTrackDependency } from "@hypit/semantic-track";+import { timelineDependency } from "@hypit/timeline";import { spatialDependency, spatialFrameSchema, spatialTypes } from "@hypit/spatial";import { svsRecipeType } from "@hypit/svs";import { temporalDependency, temporalInstantSchema, temporalTypes, temporalWindowSchema } from "@hypit/temporal";@@ -487,7 +488,7 @@ const allRankingMarkupSurfaces = ["Every other property is refused by name.",],} },- { name: "tier", tag: "TierBoard", mode: "structured", outputs: [rankingTypes.header, rankingTypes.itemSpec, temporalTypes.instantSpec, temporalTypes.windowSpec, temporalTypes.instant, temporalTypes.window, rankingTypes.schedule, rankingTypes.tierProgram, compositionTypes.visualTrack, compositionTypes.audioTrack],+ { name: "tier", tag: "TierBoard", mode: "structured", outputs: [rankingTypes.header, rankingTypes.itemSpec, temporalTypes.instantSpec, temporalTypes.windowSpec, temporalTypes.instant, temporalTypes.window, rankingTypes.schedule, rankingTypes.tierProgram, compositionTypes.visualTrack, rankingTypes.soundEvents, compositionTypes.audioTrack],vocabulary: {summary: "Places Items into tier rows from preset state or explicit reveal Selections, and publishes the board and the Tracks it renders to.",appearance:@@ -537,11 +538,13 @@ const allRankingMarkupSurfaces = [summary: "The resolved board: its Frame, Style, Schedule and ordered Items." },{ name: "visual", type: compositionTypes.visualTrack,summary: "The rendered board, an ordinary peer VisualTrack." },+ { name: "events", type: rankingTypes.soundEvents,+ summary: "Animation-linked sound events, published when a sound is authored." },{ name: "audio", type: compositionTypes.audioTrack,summary: "The rendered board sound, published only when a sound is authored." },],example: `<ranking:TierBoardStyle id="tier-style" recipe={recipes.ranking.tier} font={ui-font}/>-<ranking:TierBoard id="tiers" semantic={speech.semantic} canvas={vertical} frame={board-frame}+<ranking:TierBoard id="tiers" timeline={speech.timeline} canvas={vertical} frame={board-frame}during="program"style={tier-style}><ranking:TierItem id="row-regen" tier="s" preset="true" icon={icon-regen}/>@@ -555,7 +558,7 @@ const allRankingMarkupSurfaces = ["Authoring either sound also connects the Style's `.sound` output, so `style` must name a TierBoardStyle written in this Source.",],} },- { name: "column", tag: "Column", mode: "structured", outputs: [rankingTypes.header, rankingTypes.itemSpec, rankingTypes.textItemShell, temporalTypes.instantSpec, temporalTypes.windowSpec, temporalTypes.instant, temporalTypes.window, rankingTypes.schedule, rankingTypes.columnProgram, compositionTypes.visualTrack, compositionTypes.audioTrack],+ { name: "column", tag: "Column", mode: "structured", outputs: [rankingTypes.header, rankingTypes.itemSpec, rankingTypes.textItemShell, temporalTypes.instantSpec, temporalTypes.windowSpec, temporalTypes.instant, temporalTypes.window, rankingTypes.schedule, rankingTypes.columnProgram, compositionTypes.visualTrack, rankingTypes.soundEvents, compositionTypes.audioTrack],vocabulary: {summary: "Places rows by explicit rank, reveals each non-preset row in its own Segment or Selection, and publishes the board and the Tracks it renders to.",appearance:@@ -605,11 +608,13 @@ const allRankingMarkupSurfaces = [summary: "The resolved board: its Frame, Style, Schedule and ordered Items." },{ name: "visual", type: compositionTypes.visualTrack,summary: "The rendered board, an ordinary peer VisualTrack." },+ { name: "events", type: rankingTypes.soundEvents,+ summary: "Animation-linked sound events, published when a sound is authored." },{ name: "audio", type: compositionTypes.audioTrack,summary: "The rendered board sound, published only when a sound is authored." },],example: `<ranking:ColumnStyle id="board-style" recipe={recipes.ranking.board} font={ui-font}/>-<ranking:Column id="board" semantic={speech.semantic} canvas={vertical} frame={board-frame}+<ranking:Column id="board" timeline={speech.timeline} canvas={vertical} frame={board-frame}during={story.segment.ranking}style={board-style}><ranking:ColumnItem id="row-regen" rank="1" label="ReGen" icon={icon-regen} during={story.selection.regen}/>@@ -624,7 +629,7 @@ const allRankingMarkupSurfaces = ["Authoring either sound also connects the Style's `.sound` output, so `style` must name a ColumnStyle written in this Source.",],} },- { name: "top-three", tag: "TopThree", mode: "structured", outputs: [rankingTypes.header, rankingTypes.itemSpec, rankingTypes.textItemShell, temporalTypes.instantSpec, temporalTypes.windowSpec, temporalTypes.instant, temporalTypes.window, rankingTypes.schedule, rankingTypes.topThreeProgram, compositionTypes.visualTrack, compositionTypes.audioTrack],+ { name: "top-three", tag: "TopThree", mode: "structured", outputs: [rankingTypes.header, rankingTypes.itemSpec, rankingTypes.textItemShell, temporalTypes.instantSpec, temporalTypes.windowSpec, temporalTypes.instant, temporalTypes.window, rankingTypes.schedule, rankingTypes.topThreeProgram, compositionTypes.visualTrack, rankingTypes.soundEvents, compositionTypes.audioTrack],vocabulary: {summary: "Fills a podium one slot at a time at the Moment owned by each Item, and publishes the board and the Tracks it renders to.",appearance:@@ -667,11 +672,13 @@ const allRankingMarkupSurfaces = [summary: "The resolved board: its Frame, Style, Schedule and ordered Items." },{ name: "visual", type: compositionTypes.visualTrack,summary: "The rendered board, an ordinary peer VisualTrack." },+ { name: "events", type: rankingTypes.soundEvents,+ summary: "Animation-linked sound events, published when a sound is authored." },{ name: "audio", type: compositionTypes.audioTrack,summary: "The rendered board sound, published only when a sound is authored." },],example: `<ranking:TopThreeStyle id="podium-style" recipe={recipes.ranking.podium} font={ui-font}/>-<ranking:TopThree id="podium" semantic={speech.semantic} frame={board-frame}+<ranking:TopThree id="podium" timeline={speech.timeline} frame={board-frame}during={story.selection.board} terminal={story.moment.done}style={podium-style}><ranking:TopThreeItem id="slot-gold" label="ReGen" icon={icon-regen} at={story.moment.regen}/>@@ -692,7 +699,7 @@ export const rankingMarkupSurfaces = allRankingMarkupSurfaces;export const rankingManifest: ModuleManifest = {format: "hypit.module@1", name: rankingModuleRef.name, version: rankingModuleRef.version,- dependencies: [artifactDependency, mediaDependency, narrativeDependency, programSpaceDependency, semanticTrackDependency, spatialDependency, temporalDependency, compositionDependency, textDependency],+ dependencies: [artifactDependency, mediaDependency, narrativeDependency, timelineDependency, spatialDependency, temporalDependency, compositionDependency, textDependency],types: [{ name: rankingTypes.header.name },{ name: rankingTypes.itemSpec.name },@@ -729,7 +736,7 @@ export const rankingManifest: ModuleManifest = {], outputs: [{ name: "set", type: rankingTypes.triggeredCandidates }], needs: [] },{ name: rankingProducers.schedule.name, inputs: [{ name: "header", type: rankingTypes.header }, { name: "items", type: rankingTypes.itemSpecs },- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },{ name: "outer", type: temporalTypes.window }, { name: "candidates", type: rankingTypes.triggeredCandidates },{ name: "terminal", type: temporalTypes.instant },], outputs: [{ name: "schedule", type: rankingTypes.schedule }], needs: [] },@@ -742,7 +749,7 @@ export const rankingManifest: ModuleManifest = {], outputs: [{ name: "set", type: rankingTypes.tierWindows }], needs: [] },{ name: rankingProducers.tierSchedule.name, inputs: [{ name: "header", type: rankingTypes.header }, { name: "items", type: rankingTypes.itemSpecs },- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },{ name: "outer", type: temporalTypes.window }, { name: "windows", type: rankingTypes.tierWindows },], outputs: [{ name: "schedule", type: rankingTypes.schedule }], needs: [] },{ name: rankingProducers.createColumnWindows.name, inputs: [], outputs: [@@ -754,7 +761,7 @@ export const rankingManifest: ModuleManifest = {], outputs: [{ name: "set", type: rankingTypes.columnWindows }], needs: [] },{ name: rankingProducers.columnSchedule.name, inputs: [{ name: "header", type: rankingTypes.header }, { name: "items", type: rankingTypes.itemSpecs },- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },{ name: "outer", type: temporalTypes.window }, { name: "windows", type: rankingTypes.columnWindows },], outputs: [{ name: "schedule", type: rankingTypes.schedule }], needs: [] },...([@@ -788,7 +795,7 @@ export const rankingManifest: ModuleManifest = {{ name: eventProducer.name, inputs: [{ name: "schedule", type: rankingTypes.schedule }, { name: "style", type: styleType }, { name: "specs", type: rankingTypes.itemSpecs },], outputs: [{ name: "events", type: rankingTypes.soundEvents }], needs: [] },- { name: renderProducer.name, inputs: [{ name: "space", type: programSpaceTypes.programSpace }, { name: "program", type: programType }], outputs: [{ name: "track", type: compositionTypes.visualTrack }], needs: [] },+ { name: renderProducer.name, inputs: [{ name: "timeline", type: timelineTypes.track }, { name: "program", type: programType }], outputs: [{ name: "track", type: compositionTypes.visualTrack }], needs: [] },]),{ name: rankingProducers.createSounds.name, inputs: [], outputs: [{ name: "sounds", type: rankingTypes.sounds }], needs: [] },...([@@ -798,7 +805,7 @@ export const rankingManifest: ModuleManifest = {name: producer.name, inputs: [{ name: "sounds", type: rankingTypes.sounds }, { name: "media", type: mediaTypes.synchronized }], outputs: [{ name: "sounds", type: rankingTypes.sounds }], needs: [],})),{ name: rankingProducers.renderAudio.name, inputs: [- { name: "space", type: programSpaceTypes.programSpace }, { name: "events", type: rankingTypes.soundEvents },+ { name: "timeline", type: timelineTypes.track }, { name: "events", type: rankingTypes.soundEvents },{ name: "style", type: rankingTypes.soundStyle }, { name: "sounds", type: rankingTypes.sounds },], outputs: [{ name: "track", type: compositionTypes.audioTrack }], needs: [] },],diff --git a/packages/ranking/src/render.ts b/packages/ranking/src/render.tsindex f35ac53b..43c3ebe4 100644--- a/packages/ranking/src/render.ts+++ b/packages/ranking/src/render.ts@@ -1,3 +1,4 @@+import type { Timeline } from "@hypit/timeline";import {assertAudioTrackIdentity,assertVisualTrackIdentity,@@ -14,12 +15,7 @@ import type {VisualTrack,} from "@hypit/composition";import { synchronizedMediaSampleFrames, verifySynchronizedMedia } from "@hypit/media";-import {- assertProgramSpaceIdentity,- programFrameSampleBoundary,- programSpaceSampleFrames,-} from "@hypit/program-space";-import type { ProgramSpace } from "@hypit/program-space";+import { assertProgramSpaceIdentity, programFrameSampleBoundary, programSpaceSampleFrames } from "@hypit/program-space";import {assertColumnProgram,@@ -253,14 +249,14 @@ function tierItemAnimation(})));}-function sealTrack(space: ProgramSpace, id: string, presents: readonly VisualPresent[]): VisualTrack {- const value = sealVisualTrack({ programSpaceId: space.id, visualIr: "hypit.visual-ir@1", id, presents });- assertVisualTrackIdentity(value, space);+function sealTrack(timeline: Timeline, id: string, presents: readonly VisualPresent[]): VisualTrack {+ const value = sealVisualTrack({ programSpaceId: timeline.id, visualIr: "hypit.visual-ir@1", id, presents });+ assertVisualTrackIdentity(value, timeline);return value;}-export function renderTierBoard(space: ProgramSpace, program: TierBoardProgram): VisualTrack {- assertProgramSpaceIdentity(space);+export function renderTierBoard(timeline: Timeline, program: TierBoardProgram): VisualTrack {+ assertProgramSpaceIdentity(timeline);assertTierBoardProgram(program);const { canvas, frame, style, schedule } = program;const geometry = tierBoardGeometry(frame.widthPx, frame.heightPx, style);@@ -370,7 +366,7 @@ export function renderTierBoard(space: ProgramSpace, program: TierBoardProgram):elements: itemElements(),}));}- return sealTrack(space, program.id, presents);+ return sealTrack(timeline, program.id, presents);}function rowY(frameY: number, frameHeight: number, padding: number, count: number, index: number, height: number, gap: number): number {@@ -463,8 +459,8 @@ function columnRevealAnimation(input: {]);}-export function renderColumn(space: ProgramSpace, program: ColumnProgram): VisualTrack {- assertProgramSpaceIdentity(space);+export function renderColumn(timeline: Timeline, program: ColumnProgram): VisualTrack {+ assertProgramSpaceIdentity(timeline);assertColumnProgram(program);const { canvas, frame, style, schedule } = program;const presents: VisualPresent[] = [];@@ -562,7 +558,7 @@ export function renderColumn(space: ProgramSpace, program: ColumnProgram): VisuatieBreak: `${program.id}:item:${String(item.rank).padStart(4, "0")}:${item.id}:settled`, elements: itemElements(),}));}- return sealTrack(space, program.id, presents);+ return sealTrack(timeline, program.id, presents);}function topSlotX(program: TopThreeProgram, index: number): number {@@ -580,8 +576,8 @@ function activeAccentAnimation(duration: number, activeFrames: number): VisualAn]);}-export function renderTopThree(space: ProgramSpace, program: TopThreeProgram): VisualTrack {- assertProgramSpaceIdentity(space);+export function renderTopThree(timeline: Timeline, program: TopThreeProgram): VisualTrack {+ assertProgramSpaceIdentity(timeline);assertTopThreeProgram(program);const { frame, style, schedule } = program;const boardElements: VisualElement[] = [absoluteBox({@@ -652,33 +648,33 @@ export function renderTopThree(space: ProgramSpace, program: TopThreeProgram): VtieBreak: `${program.id}:item:${String(index).padStart(4, "0")}:${item.id}:settled`, elements: itemElements(false),}));}- return sealTrack(space, program.id, presents);+ return sealTrack(timeline, program.id, presents);}export function renderRankingAudio(- space: ProgramSpace,+ timeline: Timeline,plan: RankingSoundEventPlan,style: RankingSoundStyle,sounds: RankingSoundSet,): AudioTrack {- assertProgramSpaceIdentity(space);+ assertProgramSpaceIdentity(timeline);assertRankingSoundEventPlan(plan);assertRankingSoundStyle(style);assert(sounds.appear !== undefined || sounds.move !== undefined, "Ranking audio requires at least one authored sound.");if (sounds.appear !== undefined) verifySynchronizedMedia(sounds.appear);if (sounds.move !== undefined) verifySynchronizedMedia(sounds.move);- const totalSamples = programSpaceSampleFrames(space, 48_000);- const fadeSamples = programFrameSampleBoundary(space, style.fadeFrames, 48_000);+ const totalSamples = programSpaceSampleFrames(timeline, 48_000);+ const fadeSamples = programFrameSampleBoundary(timeline, style.fadeFrames, 48_000);const clips: AudioClip[] = [];for (const event of plan.events) {const media = sounds[event.kind];if (media === undefined) continue;const audio = media.audio;assert(audio !== undefined, `Ranking ${event.kind} sound has no normalized audio.`);- const startSample = programFrameSampleBoundary(space, event.frame, 48_000);+ const startSample = programFrameSampleBoundary(timeline, event.frame, 48_000);const sourceSampleFrames = synchronizedMediaSampleFrames(media);const length = Math.min(sourceSampleFrames, totalSamples - startSample);- assert(length > 0, `Ranking sound ${event.id} starts after ProgramSpace.`);+ assert(length > 0, `Ranking sound ${event.id} starts after Timeline.`);assert(fadeSamples <= length, `Ranking sound ${event.id} fade exceeds its audible interval.`);clips.push({id: event.id,@@ -694,7 +690,7 @@ export function renderRankingAudio(if (sounds[kind] !== undefined) assert(clips.some((clip) => clip.id.endsWith(`:${kind}`)),`Ranking ${kind} sound has no matching visual event.`);}- const track = sealAudioTrack({ programSpaceId: space.id, id: `${plan.id}.audio`, clips });- assertAudioTrackIdentity(track, space);+ const track = sealAudioTrack({ programSpaceId: timeline.id, id: `${plan.id}.audio`, clips });+ assertAudioTrackIdentity(track, timeline);return track;}diff --git a/packages/ranking/src/schedule.ts b/packages/ranking/src/schedule.tsindex ae905bff..cf628013 100644--- a/packages/ranking/src/schedule.ts+++ b/packages/ranking/src/schedule.ts@@ -1,13 +1,10 @@+import type { Timeline } from "@hypit/timeline";import {assertFontArtifactRef,verifySynchronizedMedia,} from "@hypit/media";import type { SynchronizedMedia } from "@hypit/media";-import {- assertProgramSpaceIdentity,- programSpaceFrameCount,-} from "@hypit/program-space";-import type { ProgramSpace } from "@hypit/program-space";+import { assertProgramSpaceIdentity, programSpaceFrameCount } from "@hypit/program-space";import { canonicalize, isResourceId } from "@hypit/protocol";import { verifyText } from "@hypit/text";import type { Text } from "@hypit/text";@@ -334,7 +331,7 @@ export function appendTriggeredRankingCandidate(export function buildTriggeredRankingSchedule(input: {readonly header: RankingHeader;readonly items: RankingItemSpecSet;- readonly space: ProgramSpace;+ readonly timeline: Timeline;readonly outer: TemporalWindow;readonly candidates: TriggeredRankingCandidateSet;readonly terminal: TemporalInstant;@@ -345,10 +342,10 @@ export function buildTriggeredRankingSchedule(input: {assert(input.header.variant === "top-three", "Only TopThree uses the triggered Ranking schedule.");assert(input.items.items.length > 0, "Ranking requires at least one Item.");assertTriggeredRankingCandidateSet(input.candidates);- assertTemporalWindowFor(input.outer, { subjectId: input.header.id, space: input.space });- assertTemporalInstantFor(input.terminal, { subjectId: input.header.id, space: input.space });+ assertTemporalWindowFor(input.outer, { subjectId: input.header.id, space: input.timeline });+ assertTemporalInstantFor(input.terminal, { subjectId: input.header.id, space: input.timeline });for (const candidate of input.candidates.entries) {- assertTemporalInstantFor(candidate.activation, { subjectId: candidate.itemId, space: input.space });+ assertTemporalInstantFor(candidate.activation, { subjectId: candidate.itemId, space: input.timeline });}const expected = new Set(input.items.items.map((item) => item.id));const received = new Set(input.candidates.entries.map((entry) => entry.itemId));@@ -379,7 +376,7 @@ export function buildTriggeredRankingSchedule(input: {terminalFrame: resolved.terminalFrame,entries,};- assertRankingSchedule(result, input.space);+ assertRankingSchedule(result, input.timeline);return canonicalize(result) as unknown as TriggeredRankingSchedule;}@@ -448,7 +445,7 @@ export function appendColumnWindow(function buildWindowedRankingSchedule(input: {readonly header: RankingHeader;readonly items: RankingItemSpecSet;- readonly space: ProgramSpace;+ readonly timeline: Timeline;readonly outer: TemporalWindow;readonly windows: RankingWindowSet;readonly variant: "tier-board" | "column";@@ -460,7 +457,7 @@ function buildWindowedRankingSchedule(input: {assert(input.items.variant === input.variant, `${label} Schedule requires ${label} Items.`);assert(input.items.items.length > 0, `${label} requires at least one Item.`);assertRankingWindowSet(input.windows, `${label}WindowSet`);- assertTemporalWindowFor(input.outer, { subjectId: input.header.id, space: input.space });+ assertTemporalWindowFor(input.outer, { subjectId: input.header.id, space: input.timeline });frame(input.outer.span.startFrame, `${label} outer window start`);frame(input.outer.span.endFrameExclusive, `${label} outer window end`);assert(input.outer.span.endFrameExclusive > input.outer.span.startFrame, `${label} outer window is empty.`);@@ -474,7 +471,7 @@ function buildWindowedRankingSchedule(input: {|| left.window.span.endFrameExclusive - right.window.span.endFrameExclusive|| left.itemId.localeCompare(right.itemId));for (const entry of orderedWindows) {- assertTemporalWindowFor(entry.window, { subjectId: entry.itemId, space: input.space });+ assertTemporalWindowFor(entry.window, { subjectId: entry.itemId, space: input.timeline });assert(entry.window.span.startFrame >= input.outer.span.startFrame&& entry.window.span.endFrameExclusive <= input.outer.span.endFrameExclusive,`${label} Item ${entry.itemId} window is outside the outer window.`);@@ -524,7 +521,7 @@ function buildWindowedRankingSchedule(input: {export function buildTierBoardSchedule(input: {readonly header: RankingHeader;readonly items: RankingItemSpecSet;- readonly space: ProgramSpace;+ readonly timeline: Timeline;readonly outer: TemporalWindow;readonly windows: TierBoardWindowSet;}): TierBoardSchedule {@@ -534,7 +531,7 @@ export function buildTierBoardSchedule(input: {export function buildColumnSchedule(input: {readonly header: RankingHeader;readonly items: RankingItemSpecSet;- readonly space: ProgramSpace;+ readonly timeline: Timeline;readonly outer: TemporalWindow;readonly windows: ColumnWindowSet;}): ColumnSchedule {@@ -626,12 +623,12 @@ function assertWindowedRankingSchedule(value: TierBoardSchedule | ColumnSchedule}}-export function assertRankingSchedule(value: RankingSchedule, space?: ProgramSpace): void {+export function assertRankingSchedule(value: RankingSchedule, timeline?: Timeline): void {if (value.variant === "top-three") assertTriggeredRankingSchedule(value);else assertWindowedRankingSchedule(value);- if (space !== undefined) {- assertProgramSpaceIdentity(space);- assert(value.outer.endFrameExclusive <= programSpaceFrameCount(space), "RankingSchedule exceeds ProgramSpace.");+ if (timeline !== undefined) {+ assertProgramSpaceIdentity(timeline);+ assert(value.outer.endFrameExclusive <= programSpaceFrameCount(timeline), "RankingSchedule exceeds Timeline.");}}diff --git a/packages/ranking/src/surface.ts b/packages/ranking/src/surface.tsindex c600df9b..33f4628a 100644--- a/packages/ranking/src/surface.ts+++ b/packages/ranking/src/surface.ts@@ -1,4 +1,4 @@-import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";+import { resolveTemporalContext } from "@hypit/temporal-markup";import {assertEmptyElement as empty,assertAttributes as allowed,@@ -199,7 +199,7 @@ const variantDefinition = {function rankingSurface(variant: RankingVariant): StructuredSurfaceHandler {return ({ element, resolveReference }) => {- const common = ["id", "semantic", "space", "frame", "during", "terminal", "style", "appear-sound", "move-sound"];+ const common = ["id", "timeline", "frame", "during", "terminal", "style", "appear-sound", "move-sound"];const attributes = variant === "column" || variant === "tier-board"? [...common.filter((name) => name !== "terminal"), "canvas"]: common;@@ -207,31 +207,30 @@ function rankingSurface(variant: RankingVariant): StructuredSurfaceHandler {const id = text(element, "id");const selected = variantDefinition[variant];const context = resolveTemporalContext({ element, resolveReference });- const time = createTemporalSpace({ id, element, ...context });const canvas = variant === "column" || variant === "tier-board"? reference(element.attributes.canvas, `${element.name}.canvas`, spatialTypes.canvas, resolveReference): undefined;const frame = reference(element.attributes.frame, `${element.name}.frame`, spatialTypes.frame, resolveReference);- const outerTemporal = createTemporalWindowProjection({ id: `${id}.outer`, subjectId: id, element, ...context, space: time.space, resolveReference });+ const outerTemporal = createTemporalWindowProjection({ id: `${id}.outer`, subjectId: id, element, ...context, resolveReference });const terminalTemporal = variant === "top-three"? createTemporalInstantProjection({- id: `${id}.terminal`, subjectId: id, element, ...context, space: time.space, resolveReference,+ id: `${id}.terminal`, subjectId: id, element, ...context, resolveReference,semanticAttribute: "terminal", projectedAttribute: false,}): undefined;const styleRaw = element.attributes.style;const style = reference(styleRaw, `${element.name}.style`, selected.style, resolveReference);const records: SurfaceRecordDraft[] = [...outerTemporal.records, ...(terminalTemporal?.records ?? [])];- const temporalComponents: SurfaceComponentDraft[] = [...time.components, ...outerTemporal.components, ...(terminalTemporal?.components ?? [])];- const temporalFragments = [...time.fragments, ...outerTemporal.fragments, ...(terminalTemporal?.fragments ?? [])];+ const temporalComponents: SurfaceComponentDraft[] = [...outerTemporal.components, ...(terminalTemporal?.components ?? [])];+ const temporalFragments = [...outerTemporal.fragments, ...(terminalTemporal?.fragments ?? [])];const headerId = `${id}.header`;records.push({id: headerId, type: rankingTypes.header,value: { kind: "inline", value: sealRankingHeader({ id, variant }) as unknown as CanonicalValue },range: element.range,});- const inputs: Record<string, typeof time.space.ref> = {- header: { kind: "record", id: headerId }, space: time.space.ref, frame: frame.ref,+ const inputs: Record<string, typeof context.timeline.ref> = {+ header: { kind: "record", id: headerId }, timeline: context.timeline.ref, frame: frame.ref,outer: outerTemporal.ref, style: style.ref,...(canvas === undefined ? {} : { canvas: canvas.ref }),...(terminalTemporal === undefined ? {} : { terminal: terminalTemporal.ref }),@@ -260,8 +259,8 @@ function rankingSurface(variant: RankingVariant): StructuredSurfaceHandler {const contentName = authored.content === undefined ? undefined : `item-${suffix}-content`;if (authored.content !== undefined) inputs[contentName!] = authored.content.ref;const itemTemporal = !authored.timed ? undefined : variant === "top-three"- ? createTemporalInstantProjection({ id: `${id}.item.${suffix}`, subjectId: spec.id, element: child, ...context, space: time.space, resolveReference })- : createTemporalWindowProjection({ id: `${id}.item.${suffix}`, subjectId: spec.id, element: child, ...context, space: time.space, resolveReference });+ ? createTemporalInstantProjection({ id: `${id}.item.${suffix}`, subjectId: spec.id, element: child, ...context, resolveReference })+ : createTemporalWindowProjection({ id: `${id}.item.${suffix}`, subjectId: spec.id, element: child, ...context, resolveReference });if (itemTemporal !== undefined) {records.push(...itemTemporal.records);temporalComponents.push(...itemTemporal.components);@@ -307,12 +306,12 @@ function rankingSurface(variant: RankingVariant): StructuredSurfaceHandler {id, fragment: fragment.id, inputs,outputs: {schedule: `${id}.schedule`, program: `${id}.program`, visual: `${id}.visual`,- ...(sound.appearName === undefined && sound.moveName === undefined ? {} : { audio: `${id}.audio` }),+ ...(sound.appearName === undefined && sound.moveName === undefined ? {} : { events: `${id}.events`, audio: `${id}.audio` }),},range: element.range,}],fragments: [...temporalFragments, fragment],- exports: [`${id}.schedule`, `${id}.program`, `${id}.visual`, ...(sound.appearName === undefined && sound.moveName === undefined ? [] : [`${id}.audio`])],+ exports: [`${id}.schedule`, `${id}.program`, `${id}.visual`, ...(sound.appearName === undefined && sound.moveName === undefined ? [] : [`${id}.events`, `${id}.audio`])],};};}diff --git a/packages/ranking/test/ranking.test.ts b/packages/ranking/test/ranking.test.tsindex 1864c6ec..1e249d0b 100644--- a/packages/ranking/test/ranking.test.ts+++ b/packages/ranking/test/ranking.test.ts@@ -1,8 +1,9 @@+import { sealTimeline } from "@hypit/timeline";import assert from "node:assert/strict";import test from "node:test";import { videoContractManifests } from "../../../test/support/video-domain.js";import { fixtureResource } from "../../../test/fixture-resource.js";-import { semanticTrackFixture } from "../../../test/semantic-track-fixture.js";+import { timelineFixture } from "../../../test/timeline-fixture.js";import { projectMomentInstantFixture, projectSegmentWindow, projectSelectionWindow } from "../../../test/temporal-fixture.js";import { createResolvedClosure } from "@hypit/core";@@ -78,7 +79,7 @@ import type {TopThreeItemSpec,} from "@hypit/ranking";import { narrativeTypes } from "@hypit/narrative";-import { semanticTrackTypes } from "@hypit/semantic-track";+import { timelineTypes } from "@hypit/timeline";import { spatialTypes } from "@hypit/spatial";import { svsRecipeType } from "@hypit/svs";import { sealText, textManifest, textTypes } from "@hypit/text";@@ -91,7 +92,7 @@ import type {MarkupAttributeValue,} from "@hypit/markup";-const space = sealProgramSpace({ id: "test-space", durationSec: 8,+const space = sealTimeline({ items: [], id: "test-space", durationSec: 8,frameRate: { numerator: 30, denominator: 1 },});const canvas = sealCanvasSpace({@@ -104,7 +105,7 @@ const frame = sealSpatialFrame({const tierFrame = sealSpatialFrame({xPx: 40, yPx: 360, widthPx: 720, heightPx: 280,});-const semanticTrack = semanticTrackFixture(space, {+const timeline = timelineFixture(space, {segments: [{ id: "opening", frameCount: 10 },{ id: "ranking", frameCount: 220 },@@ -172,18 +173,18 @@ function schedule(): TriggeredRankingSchedule {let candidates = createTriggeredRankingCandidateSet();for (const [index, value] of values.entries()) {- candidates = appendTriggeredRankingCandidate(candidates, value, semanticTrack, {+ candidates = appendTriggeredRankingCandidate(candidates, value, timeline, {narrativeId: "script",id: `${value.id}-moment`,anchorId: anchors[index]!,});}- return buildRankingSchedule({ header: headerValue, items: specs(headerValue, values), semantic: semanticTrack, outer, candidates, terminal });+ return buildRankingSchedule({ header: headerValue, items: specs(headerValue, values), semantic: timeline, outer, candidates, terminal });}function appendTriggeredRankingCandidate(set: ReturnType<typeof createTriggeredRankingCandidateSet>, spec: RankingItemSpec,- semantic: typeof semanticTrack, moment: NarrativeMomentRef,+ semantic: typeof timeline, moment: NarrativeMomentRef,) {return appendProjectedTriggeredRankingCandidate(set, spec, projectMomentInstantFixture({itemId: spec.id, semantic, moment,@@ -194,7 +195,7 @@ function appendTriggeredRankingCandidate(function buildRankingSchedule(input: {readonly header: RankingHeader;readonly items: RankingItemSpecSet;- readonly semantic: typeof semanticTrack;+ readonly semantic: typeof timeline;readonly outer: NarrativeSelectionRef;readonly candidates: TriggeredRankingCandidateSet;readonly terminal: NarrativeMomentRef;@@ -202,7 +203,7 @@ function buildRankingSchedule(input: {return buildTriggeredRankingSchedule({header: input.header,items: input.items,- space,+ timeline: space,outer: projectSelectionWindow({itemId: input.outer.id, subjectId: input.header.id, semantic: input.semantic, selection: input.outer,projection: { start: { ref: "selection.start" }, end: { ref: "selection.end" } },@@ -217,7 +218,7 @@ function buildRankingSchedule(input: {function appendProjectedColumnWindow(set: ReturnType<typeof createColumnWindowSet>, spec: ColumnItemSpec,- semantic: typeof semanticTrack, selection: NarrativeSelectionRef,+ semantic: typeof timeline, selection: NarrativeSelectionRef,) {return appendColumnWindow(set, spec, projectSelectionWindow({itemId: spec.id, semantic, selection,@@ -227,7 +228,7 @@ function appendProjectedColumnWindow(function appendProjectedTierWindow(set: ReturnType<typeof createTierBoardWindowSet>, spec: TierBoardItemSpec,- semantic: typeof semanticTrack, selectionValue: NarrativeSelectionRef,+ semantic: typeof timeline, selectionValue: NarrativeSelectionRef,) {return appendTierBoardWindow(set, spec, projectSelectionWindow({itemId: spec.id, semantic, selection: selectionValue,@@ -235,7 +236,7 @@ function appendProjectedTierWindow(}));}-function projectColumnSegmentOuterWindow(semantic: typeof semanticTrack, segment: NarrativeExcerpt, subjectId: string): TemporalWindow {+function projectColumnSegmentOuterWindow(semantic: typeof timeline, segment: NarrativeExcerpt, subjectId: string): TemporalWindow {return projectSegmentWindow({itemId: `${segment.id}:outer`, subjectId, semantic, segment,projection: { start: { ref: "segment.start" }, end: { ref: "segment.end" } },@@ -256,16 +257,16 @@ function columnSchedule(owner: RankingHeader,values: readonly ColumnItemSpec[],windows: Readonly<Record<string, NarrativeSelectionRef>>,- outerWindow = projectColumnSegmentOuterWindow(semanticTrack, rankingSegment, owner.id),+ outerWindow = projectColumnSegmentOuterWindow(timeline, rankingSegment, owner.id),): ColumnSchedule {let set = createColumnWindowSet();for (const value of values) {if (value.preset) continue;const timing = windows[value.id];if (timing === undefined) throw new Error(`Missing test Selection for ${value.id}.`);- set = appendProjectedColumnWindow(set, value, semanticTrack, timing);+ set = appendProjectedColumnWindow(set, value, timeline, timing);}- return buildColumnSchedule({ header: owner, items: specs(owner, values), space, outer: outerWindow, windows: set });+ return buildColumnSchedule({ header: owner, items: specs(owner, values), timeline: space, outer: outerWindow, windows: set });}function tierSchedule(@@ -273,7 +274,7 @@ function tierSchedule(values: readonly TierBoardItemSpec[],windows: Readonly<Record<string, NarrativeSelectionRef>>,outerWindow = projectSelectionWindow({- itemId: outer.id, subjectId: owner.id, semantic: semanticTrack, selection: outer,+ itemId: outer.id, subjectId: owner.id, semantic: timeline, selection: outer,projection: { start: { ref: "selection.start" }, end: { ref: "selection.end" } },}),): TierBoardSchedule {@@ -282,9 +283,9 @@ function tierSchedule(if (value.preset) continue;const timing = windows[value.id];if (timing === undefined) throw new Error(`Missing test Selection for ${value.id}.`);- set = appendProjectedTierWindow(set, value, semanticTrack, timing);+ set = appendProjectedTierWindow(set, value, timeline, timing);}- return buildTierBoardSchedule({ header: owner, items: specs(owner, values), space, outer: outerWindow, windows: set });+ return buildTierBoardSchedule({ header: owner, items: specs(owner, values), timeline: space, outer: outerWindow, windows: set });}test("RankingSchedule derives chronological Item order from item-owned Moments and preserves a settled suffix", () => {@@ -302,9 +303,9 @@ test("RankingSchedule requires one item-owned Moment per Item and distinct chronconst owner = header("top-three");const values = [topSpec("a"), topSpec("b")];let missing = createTriggeredRankingCandidateSet();- missing = appendTriggeredRankingCandidate(missing, values[0]!, semanticTrack, { narrativeId: "script", id: "a-at", anchorId: "one" });+ missing = appendTriggeredRankingCandidate(missing, values[0]!, timeline, { narrativeId: "script", id: "a-at", anchorId: "one" });assert.throws(() => buildRankingSchedule({- header: owner, items: specs(owner, values), semantic: semanticTrack, outer, candidates: missing, terminal,+ header: owner, items: specs(owner, values), semantic: timeline, outer, candidates: missing, terminal,}), /item-owned Moments differ/u);assert.throws(() => schedule(owner, values, ["one", "one"]), /strictly increasing/u);});@@ -402,7 +403,7 @@ test("Column consumes explicit disjoint reveal windows without changing them", (set = appendColumnItem(set, semantic[1]!, image("preset"));set = appendColumnItem(set, semantic[2]!);set = appendColumnItem(set, semantic[3]!, image("overlap"));- const outerWindow = projectColumnSegmentOuterWindow(semanticTrack, rankingSegment, owner.id);+ const outerWindow = projectColumnSegmentOuterWindow(timeline, rankingSegment, owner.id);assert.deepEqual(outerWindow.span, { startFrame: 10, endFrameExclusive: 230 });const value = columnSchedule(owner, semantic, {"late-rank-one": late,@@ -436,10 +437,10 @@ test("Column rejects overlapping or out-of-bounds reveal windows", () => {assert.throws(() => columnSchedule(owner, values, { one: early, two: overlapping }),/windows one and two overlap/u);- const outerWindow = projectColumnSegmentOuterWindow(semanticTrack, rankingSegment, owner.id);+ const outerWindow = projectColumnSegmentOuterWindow(timeline, rankingSegment, owner.id);let windows = createColumnWindowSet();const projected = projectSelectionWindow({- itemId: "one", semantic: semanticTrack, selection: early,+ itemId: "one", semantic: timeline, selection: early,projection: { start: { ref: "selection.start" }, end: { ref: "selection.end" } },});windows = appendColumnWindow(windows, values[0]!, {@@ -448,7 +449,7 @@ test("Column rejects overlapping or out-of-bounds reveal windows", () => {span: { startFrame: outerWindow.span.startFrame - 1, endFrameExclusive: projected.span.endFrameExclusive },});assert.throws(() => buildColumnSchedule({- header: owner, items: specs(owner, [values[0]!]), space, outer: outerWindow, windows,+ header: owner, items: specs(owner, [values[0]!]), timeline: space, outer: outerWindow, windows,}), /one window is outside/u);});@@ -525,7 +526,7 @@ test("all three author Surfaces preserve explicit semantic, spatial, font, imagerecord: { value: { kind: "inline", value } } as never,});const references = new Map<string, SurfaceResolvedReference>([- ["semantic", plain("semantic", semanticTrackTypes.track)],+ ["semantic", plain("semantic", timelineTypes.track)],["canvas", plain("canvas", spatialTypes.canvas)],["frame", plain("frame", spatialTypes.frame)],["outer", plain("outer", narrativeTypes.selection)],@@ -563,11 +564,11 @@ test("all three author Surfaces preserve explicit semantic, spatial, font, image}const cases = [[decodeTierBoardSurface, node("ranking:TierBoard", {- id: "tier", semantic: ref("semantic"), canvas: ref("canvas"), frame: ref("frame"), during: "program",+ id: "tier", timeline: ref("semantic"), canvas: ref("canvas"), frame: ref("frame"), during: "program",style: ref("tier-style"),}, [node("ranking:TierItem", { id: "tier-one", tier: "s", entry: "drop", icon: ref("icon-1"), during: ref("ranking-segment") })])],[decodeColumnSurface, node("ranking:Column", {- id: "column", semantic: ref("semantic"), canvas: ref("canvas"), frame: ref("frame"), during: ref("ranking-segment"),+ id: "column", timeline: ref("semantic"), canvas: ref("canvas"), frame: ref("frame"), during: ref("ranking-segment"),style: ref("column-style"),"appear-sound": ref("appear"), "move-sound": ref("move"),}, [@@ -575,7 +576,7 @@ test("all three author Surfaces preserve explicit semantic, spatial, font, imagenode("ranking:ColumnItem", { id: "column-two", rank: "5", preset: "true", label: "Two", icon: ref("icon-2") }),])],[decodeTopThreeSurface, node("ranking:TopThree", {- id: "top", semantic: ref("semantic"), frame: ref("frame"), during: ref("outer"),+ id: "top", timeline: ref("semantic"), frame: ref("frame"), during: ref("outer"),terminal: ref("terminal"), style: ref("top-style"),}, [node("ranking:TopThreeItem", { label: "First", at: ref("moment-one") }), node("ranking:TopThreeItem", { label: "Second", icon: ref("icon-1"), at: ref("moment-two") })])],] as const;@@ -625,7 +626,8 @@ test("all three author Surfaces preserve explicit semantic, spatial, font, imageconst audio = columnFragment.exports.find((output) => output.name === "audio");assert.ok(columnFragment.operations.some((operation) => operation.producer.name === rankingProducers.materializeTextItem.name));assert(audio !== undefined);- assert.deepEqual(Object.keys(column.components.find((component) => component.outputs.visual !== undefined)!.outputs).sort(), ["audio", "program", "schedule", "visual"]);+ assert.ok(columnFragment.exports.some((output) => output.name === "events" && output.type.name === rankingTypes.soundEvents.name));+ assert.deepEqual(Object.keys(column.components.find((component) => component.outputs.visual !== undefined)!.outputs).sort(), ["audio", "events", "program", "schedule", "visual"]);});test("Ranking Surfaces declare their sealed Records and icon Producers consume Blob values", async () => {@@ -656,14 +658,14 @@ test("Ranking author Surfaces fail closed on impossible image and sound combinatconst ref = (path: string): MarkupAttributeValue => ({ kind: "reference", path });const plain = (path: string, type: SurfaceResolvedReference["type"]): SurfaceResolvedReference => ({ path, ref: { kind: "record", id: path }, type });const references = new Map<string, SurfaceResolvedReference>([- ["semantic", plain("semantic", semanticTrackTypes.track)],+ ["semantic", plain("semantic", timelineTypes.track)],["canvas", plain("canvas", spatialTypes.canvas)],["frame", plain("frame", spatialTypes.frame)], ["outer", plain("outer", narrativeTypes.selection)],["icon", plain("icon", mediaTypes.blobArtifact)],["style", plain("style", rankingTypes.tierStyle)], ["style.sound", plain("style.sound", rankingTypes.soundStyle)],["move", plain("move", mediaTypes.synchronized)],]);- const common = { id: "bad", semantic: ref("semantic"), canvas: ref("canvas"), frame: ref("frame"), during: ref("outer"), style: ref("style") };+ const common = { id: "bad", timeline: ref("semantic"), canvas: ref("canvas"), frame: ref("frame"), during: ref("outer"), style: ref("style") };const context = (element: StructuredElement) => ({sourceName: "ranking.svml", element, resolveReference: (path: string) => references.get(path),resolveAsset: async () => { throw new Error("no asset resolution expected"); },diff --git a/packages/render-hyperframes/README.md b/packages/render-hyperframes/README.mdindex f120b6e0..039d8205 100644--- a/packages/render-hyperframes/README.md+++ b/packages/render-hyperframes/README.md@@ -1,14 +1,14 @@# `@hypit/render-hyperframes`-The Surface accepts `semantic={speech.semantic}` for performance time or `space={animation}` for an-authored ProgramSpace. Its Fragment receives `space` directly; rendering does not require Script-or prepared performance media. Both time sources use the same frame and audio pipeline.+The Surface accepts `timeline={program.timeline}` and projects its complete ProgramSpace.+Its Fragment receives that range as `space`; rendering works with partial, overlapping or absent+semantic coverage through the same frame and audio pipeline.Explicit author and capability boundary for final HyperFrames video rendering.The package owns `<render:Video composition={...}/>` and lowers it to ordinary Operations that:-1. obtain the `ProgramSpace` from the semantic input and compile the referenced `Composition` into a `HyperframesDocument`;+1. obtain the `ProgramSpace` from the Timeline input and compile the referenced `Composition` into a `HyperframesDocument`;2. request a silent, frame-exact `RenderedVisual`;3. compile every peer `AudioTrack` into one `AudioProgramPlan`;4. request an exact 48 kHz `TimelineAudio`;@@ -43,7 +43,7 @@ stream with the declared canvas, rational frame rate and frame count.Select a contiguous interval using zero-based, half-open frame bounds:```xml-<render:Video id="preview" composition={main.composition} semantic={speech.semantic}+<render:Video id="preview" composition={main.composition} timeline={speech.timeline}start-frame="240" end-frame-exclusive="360"/>```diff --git a/packages/render-hyperframes/package.json b/packages/render-hyperframes/package.jsonindex 768833fe..fa86dccb 100644--- a/packages/render-hyperframes/package.json+++ b/packages/render-hyperframes/package.json@@ -20,7 +20,7 @@"@hypit/media-pipeline": "workspace:*","@hypit/program-space": "workspace:*","@hypit/protocol": "workspace:*",- "@hypit/semantic-track": "workspace:*",+ "@hypit/timeline": "workspace:*","@hypit/markup": "workspace:*","@hypit/temporal-markup": "workspace:*"},diff --git a/packages/render-hyperframes/src/fragment.ts b/packages/render-hyperframes/src/fragment.tsindex 2f04eef0..f24baf97 100644--- a/packages/render-hyperframes/src/fragment.ts+++ b/packages/render-hyperframes/src/fragment.ts@@ -1,6 +1,7 @@+import { timelineProducers, timelineTypes } from "@hypit/timeline";import { mediaTypes } from "@hypit/media";import { artifactTypes } from "@hypit/artifact";-import { programSpaceTypes } from "@hypit/program-space";+import { compositionTypes } from "@hypit/composition";import { sealGraphFragment } from "@hypit/elaborator";import { hyperframesProducers } from "@hypit/hyperframes";@@ -17,14 +18,16 @@ export function createRenderHyperframesFragment(selectedRange = false) {return sealGraphFragment({inputs: [{ name: "composition", type: compositionTypes.composition },- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },...(selectedRange ? [{ name: "range", type: mediaTypes.frameRange }] : []),],operations: [+ { id: "render-range", producer: timelineProducers.projectProgramSpace,+ inputs: { track: input("timeline") }, result: { kind: "output", name: "space" } },{id: "compile-document",producer: hyperframesProducers.compile,- inputs: { composition: input("composition"), space: input("space") },+ inputs: { composition: input("composition"), space: operation("render-range") },result: { kind: "output", name: "document" },},{@@ -36,7 +39,7 @@ export function createRenderHyperframesFragment(selectedRange = false) {{id: "compile-audio-program",producer: mediaPipelineProducers.planAudio,- inputs: { composition: input("composition"), space: input("space") },+ inputs: { composition: input("composition"), space: operation("render-range") },result: { kind: "output", name: "plan" },},{diff --git a/packages/render-hyperframes/src/manifest.ts b/packages/render-hyperframes/src/manifest.tsindex 273847bf..9d89e049 100644--- a/packages/render-hyperframes/src/manifest.ts+++ b/packages/render-hyperframes/src/manifest.ts@@ -2,7 +2,7 @@ import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";import { artifactDependency, artifactTypes } from "@hypit/artifact";import { mediaDependency, mediaTypes } from "@hypit/media";import { compositionDependency, compositionTypes } from "@hypit/composition";-import { semanticTrackDependency } from "@hypit/semantic-track";+import { timelineDependency } from "@hypit/timeline";import {hyperframesManifest,hyperframesModuleRef,@@ -48,9 +48,9 @@ export const renderHyperframesMarkupSurfaces = [{{ name: "video", type: artifactTypes.blob,summary: "The final muxed video Artifact, addressed as `<id>.video`." },],- example: '<render:Video id="final" composition={main.composition} semantic={speech.semantic}/>',+ example: '<render:Video id="final" composition={main.composition} timeline={speech.timeline}/>',notes: [- "Supply id, composition and one time source: semantic or space. Write both frame bounds to select a range. The element is empty.",+ "Supply id, composition and timeline. Write both frame bounds to select a range. The element is empty.","The visual render, the audio render and the mux are three separate Needs, each realized by a Provider this package does not choose.","The published Artifact carries no duration or lineage metadata, so a consumer that needs stream facts requests explicit media inspection.",],@@ -66,7 +66,7 @@ export const renderHyperframesManifest: ModuleManifest = {artifactDependency,mediaDependency,compositionDependency,- semanticTrackDependency,+ timelineDependency,{ module: hyperframesModuleRef },{ module: mediaPipelineModuleRef },],diff --git a/packages/render-hyperframes/src/surface.ts b/packages/render-hyperframes/src/surface.tsindex c14c633c..a85d122d 100644--- a/packages/render-hyperframes/src/surface.ts+++ b/packages/render-hyperframes/src/surface.ts@@ -1,4 +1,4 @@-import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";+import { resolveTemporalContext } from "@hypit/temporal-markup";import { compositionTypes } from "@hypit/composition";import { mediaTypes, verifyMediaFrameRange } from "@hypit/media";import type {@@ -25,7 +25,7 @@ function referenceAttribute(element: StructuredElement, name: string): string {export const decodeHyperframesRenderSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {const names = Object.keys(element.attributes).sort();- if (names.some((name) => !["composition", "id", "semantic", "space", "start-frame", "end-frame-exclusive"].includes(name))) {+ if (names.some((name) => !["composition", "id", "timeline", "start-frame", "end-frame-exclusive"].includes(name))) {throw new Error(`${element.name} contains unsupported attributes`);}if (element.children.some((child) => child.kind === "element" || child.value.trim().length > 0)) {@@ -40,7 +40,7 @@ export const decodeHyperframesRenderSurface: StructuredSurfaceHandler = ({ eleme|| composition.type.name !== compositionTypes.composition.name) {throw new Error(`${element.name}.composition must reference Composition`);}- const time = createTemporalSpace({ id, element, ...resolveTemporalContext({ element, resolveReference }) });+ const context = resolveTemporalContext({ element, resolveReference });const selected = element.attributes["start-frame"] !== undefined || element.attributes["end-frame-exclusive"] !== undefined;const frame = (name: string): number => {const text = stringAttribute(element, name);@@ -53,13 +53,13 @@ export const decodeHyperframesRenderSurface: StructuredSurfaceHandler = ({ elemeconst rangeId = `${id}.frame-range`;return {records: range === undefined ? [] : [{ id: rangeId, type: mediaTypes.frameRange, value: { kind: "inline", value: range }, range: element.range }],- components: [...time.components, {+ components: [{id,fragment: fragment.id,- inputs: { composition: composition.ref, space: time.space.ref, ...(selected ? { range: { kind: "record" as const, id: rangeId } } : {}) },+ inputs: { composition: composition.ref, timeline: context.timeline.ref, ...(selected ? { range: { kind: "record" as const, id: rangeId } } : {}) },outputs: { video: `${id}.video` },range: element.range,}],- fragments: [...time.fragments, fragment],+ fragments: [fragment],};};diff --git a/packages/render-hyperframes/test/render.test.ts b/packages/render-hyperframes/test/render.test.tsindex e4a9d4fb..05ffa6f9 100644--- a/packages/render-hyperframes/test/render.test.ts+++ b/packages/render-hyperframes/test/render.test.ts@@ -1,3 +1,4 @@+import { sealTimeline } from "@hypit/timeline";import { compositionComponent, videoContractManifests } from "../../../test/support/video-domain.js";import { artifactTypes } from "@hypit/artifact";import {@@ -11,7 +12,7 @@ import type { Composition } from "@hypit/composition";import assert from "node:assert/strict";import test from "node:test";import { fixtureResource } from "../../../test/fixture-resource.js";-import { semanticTrackFixture } from "../../../test/semantic-track-fixture.js";+import { timelineFixture } from "../../../test/timeline-fixture.js";import {createResolvedClosure,@@ -79,15 +80,15 @@ import {MarkupSurfaceRegistry,} from "@hypit/markup";import {- semanticTrackComponent,- semanticTrackDependency,- semanticTrackTypes,-} from "@hypit/semantic-track";+ timelineComponent,+ timelineDependency,+ timelineTypes,+} from "@hypit/timeline";-const space = sealProgramSpace({ id: "test-space", durationSec: 2,+const space = sealTimeline({ items: [], id: "test-space", durationSec: 2,frameRate: { numerator: 30, denominator: 1 },});-const semantic = semanticTrackFixture(space);+const semantic = timelineFixture(space);const composition = sealComposition({id: "render-test",canvas: { width: 1080, height: 1920, clearColor: "#000000" },@@ -114,7 +115,7 @@ const compositionRecord = await admitRecord(closure, sealRecord({}), validatorRegistry());const spaceRecord = await admitRecord(closure, sealRecord({id: "space",- type: programSpaceTypes.programSpace,+ type: timelineTypes.track,value: stored(space),}), validatorRegistry());const linked = link(closure, [compositionRecord, spaceRecord]);@@ -123,7 +124,7 @@ const instance = elaborateGraphFragment(linked, renderHyperframesFragment, {fragment: renderHyperframesFragment.id,inputs: {composition: { kind: "record", id: compositionRecord.id },- space: { kind: "record", id: spaceRecord.id },+ timeline: { kind: "record", id: spaceRecord.id },},});const contribution = bindAuthorFragment(instance, { video: "final.video" });@@ -140,7 +141,7 @@ function producerRegistry(): ProducerRegistry {registerProducerFacets(registry, mediaPipelineComponent.producers);registerProducerFacets(registry, hyperframesComponent.producers);registerProducerFacets(registry, renderHyperframesComponent.producers);- registerProducerFacets(registry, semanticTrackComponent.producers);+ registerProducerFacets(registry, timelineComponent.producers);return registry;}@@ -156,6 +157,7 @@ function validatorRegistry(): TypeValidatorRegistry {test("HyperFrames rendering is an explicit exact Need after ordinary document compilation", async () => {assert.deepEqual(build().plan.steps.map((step) => step.producer.name).sort(), [hyperframesProducers.compile.name,+ "project-program-space",renderHyperframesProducers.requestVisual.name,mediaPipelineProducers.planAudio.name,mediaPipelineProducers.renderAudio.name,@@ -311,13 +313,13 @@ const fixtureModule = { name: "example.composition-fixture", version: "1" } as cconst fixtureSurfaceDigest = fixtureResource("example.composition-fixture/surface@1");const fixtureSurface = {name: "composition", tag: "Composition", mode: "structured",- outputs: [compositionTypes.composition, semanticTrackTypes.track],+ outputs: [compositionTypes.composition, timelineTypes.track],} as const;const fixtureManifest: ModuleManifest = {format: "hypit.module@1",name: fixtureModule.name,version: fixtureModule.version,- dependencies: [compositionDependency, semanticTrackDependency],+ dependencies: [compositionDependency, timelineDependency],types: [],capabilities: [],producers: [],@@ -344,7 +346,7 @@ test(`the ${selectedRange ? "selected" : "full"} rendered video remains an ordinsurfaces.registerStructured({ module: fixtureModule, declaration: fixtureSurface, handler: ({ element }) => ({records: [{ id: "composition", type: compositionTypes.composition, value: stored(composition), range: element.range },- { id: "semantic", type: semanticTrackTypes.track, value: stored(semantic), range: element.range },+ { id: "semantic", type: timelineTypes.track, value: stored(semantic), range: element.range },],components: [],fragments: [],@@ -374,7 +376,7 @@ test(`the ${selectedRange ? "selected" : "full"} rendered video remains an ordin<import as="render" from="@hypit/render-hyperframes@1"/><import as="media" from="@hypit/media-pipeline@1"/><fixture:Composition/>- <render:Video id="final" composition={composition} semantic={semantic} ${selectedRange ? 'start-frame="15" end-frame-exclusive="45"' : ""}/>+ <render:Video id="final" composition={composition} timeline={semantic} ${selectedRange ? 'start-frame="15" end-frame-exclusive="45"' : ""}/><media:ExtractFrame id="poster" source={final.video} video="primary-moving" at="last"/></svml>`),closure: sourceClosure,@@ -413,7 +415,7 @@ test("selected render frames reach both visual and audio Needs through the ordinconst selected = elaborateGraphFragment(program, fragment, {id: "selected", fragment: fragment.id, inputs: {composition: { kind: "record", id: compositionRecord.id },- space: { kind: "record", id: spaceRecord.id },+ timeline: { kind: "record", id: spaceRecord.id },range: { kind: "record", id: rangeRecord.id },},});diff --git a/packages/screen-overlay/README.md b/packages/screen-overlay/README.mdindex 73df602d..926f67f4 100644--- a/packages/screen-overlay/README.md+++ b/packages/screen-overlay/README.md@@ -1,12 +1,12 @@# `@hypit/screen-overlay`-The Track Surface accepts `semantic={speech.semantic}` or `space={animation}`. Semantic context-resolves Script Selections and Moments; authored space supports clock-based animation. Shared `at`+The Track Surface accepts `timeline={program.timeline}`. The same Timeline supports authored+positions and, where prepared Takes supply evidence, Script Selections and Moments. Shared `at`inputs accept a Moment or a time such as `2s`; `at` with `for` produces a Window where required.Official provider-free self-contained screen-paint package.-Every item consumes its typed parameters, explicit Temporal source, CanvasSpace, ProgramSpace,+Every item consumes its typed parameters, explicit Temporal source, CanvasSpace, Timeline,seed and local frame. Flash, ColorWash, Vignette, ScanLines, DirectionalMatte, WhipVeil,GlitchVeil, Grain, LightLeak, Bokeh and TVStatic lower to ordinary absolute-stack `VisualTrack`Presents. These effects generate their own pixels over the selected interval.diff --git a/packages/screen-overlay/package.json b/packages/screen-overlay/package.jsonindex efb9570f..7846019d 100644--- a/packages/screen-overlay/package.json+++ b/packages/screen-overlay/package.json@@ -13,7 +13,7 @@"@hypit/narrative": "workspace:*","@hypit/protocol": "workspace:*","@hypit/program-space": "workspace:*",- "@hypit/semantic-track": "workspace:*",+ "@hypit/timeline": "workspace:*","@hypit/spatial": "workspace:*","@hypit/temporal": "workspace:*","@hypit/temporal-markup": "workspace:*",diff --git a/packages/screen-overlay/src/component.ts b/packages/screen-overlay/src/component.tsindex 3e44cb7b..783c1852 100644--- a/packages/screen-overlay/src/component.ts+++ b/packages/screen-overlay/src/component.ts@@ -1,7 +1,7 @@+import type { Timeline } from "@hypit/timeline";import type { ComponentPackage } from "@hypit/component-kit";import { canonicalize } from "@hypit/protocol";import type { StoredValue } from "@hypit/protocol";-import type { ProgramSpace } from "@hypit/program-space";import type { CanvasSpace } from "@hypit/spatial";import { screenOverlayProducers, screenOverlayTypes } from "./manifest.js";import { appendProjectedScreenOverlay, assertScreenOverlayProgram, createScreenOverlaySet, finalizeScreenOverlay, renderScreenOverlay } from "./program.js";@@ -12,9 +12,9 @@ const output = (value: unknown) => ({ kind: "inline" as const, value: canonicaliexport const screenOverlayComponent = {producers: [{ producer: screenOverlayProducers.createSet, handler: () => ({ outputs: { set: output(createScreenOverlaySet()) }, needs: {} }) },- { producer: screenOverlayProducers.appendItem, handler: ({ inputs }) => ({ outputs: { set: output(appendProjectedScreenOverlay(inline<ScreenOverlaySet>(inputs.set?.value, "ScreenOverlaySet"), inline<ScreenOverlayHeader>(inputs.header?.value, "ScreenOverlayHeader"), inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"), inline<ScreenOverlayItemSpec>(inputs.spec?.value, "ScreenOverlayItemSpec"), inline<TemporalWindow>(inputs.window?.value, "TemporalWindow"))) }, needs: {} }) },+ { producer: screenOverlayProducers.appendItem, handler: ({ inputs }) => ({ outputs: { set: output(appendProjectedScreenOverlay(inline<ScreenOverlaySet>(inputs.set?.value, "ScreenOverlaySet"), inline<ScreenOverlayHeader>(inputs.header?.value, "ScreenOverlayHeader"), inline<Timeline>(inputs.timeline?.value, "Timeline"), inline<ScreenOverlayItemSpec>(inputs.spec?.value, "ScreenOverlayItemSpec"), inline<TemporalWindow>(inputs.window?.value, "TemporalWindow"))) }, needs: {} }) },{ producer: screenOverlayProducers.finalize, handler: ({ inputs }) => ({ outputs: { program: output(finalizeScreenOverlay(inline<ScreenOverlaySet>(inputs.set?.value, "ScreenOverlaySet"), inline<ScreenOverlayHeader>(inputs.header?.value, "ScreenOverlayHeader"))) }, needs: {} }) },- { producer: screenOverlayProducers.render, handler: ({ inputs }) => ({ outputs: { track: output(renderScreenOverlay(inline<CanvasSpace>(inputs.canvas?.value, "CanvasSpace"), inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"), inline<ScreenOverlayProgram>(inputs.program?.value, "ScreenOverlayProgram"))) }, needs: {} }) },+ { producer: screenOverlayProducers.render, handler: ({ inputs }) => ({ outputs: { track: output(renderScreenOverlay(inline<CanvasSpace>(inputs.canvas?.value, "CanvasSpace"), inline<Timeline>(inputs.timeline?.value, "Timeline"), inline<ScreenOverlayProgram>(inputs.program?.value, "ScreenOverlayProgram"))) }, needs: {} }) },],validators: [{ type: screenOverlayTypes.program, handler: ({ value }) => assertScreenOverlayProgram(inline<ScreenOverlayProgram>(value, "ScreenOverlayProgram")) }],} satisfies ComponentPackage;diff --git a/packages/screen-overlay/src/fragment.ts b/packages/screen-overlay/src/fragment.tsindex 7c5b1713..c6972a30 100644--- a/packages/screen-overlay/src/fragment.ts+++ b/packages/screen-overlay/src/fragment.ts@@ -1,4 +1,5 @@-import { programSpaceTypes } from "@hypit/program-space";+import { timelineTypes } from "@hypit/timeline";+import { compositionTypes } from "@hypit/composition";import { sealGraphFragment } from "@hypit/elaborator";import type { FragmentOperation } from "@hypit/elaborator";@@ -22,16 +23,16 @@ export function createScreenOverlayFragment(items: readonly ScreenOverlayFragmenconst id = `overlay:set:append:${String(index + 1).padStart(4, "0")}`;operations.push({ id, producer: screenOverlayProducers.appendItem,inputs: { set: operation(current), header: input("header"), spec: input(item.specName),- space: input("space"), window: input(item.windowName) }, result: { kind: "output", name: "set" } });+ timeline: input("timeline"), window: input(item.windowName) }, result: { kind: "output", name: "set" } });current = id;});operations.push({ id: "overlay:program", producer: screenOverlayProducers.finalize, inputs: { set: operation(current), header: input("header") }, result: { kind: "output", name: "program" } },- { id: "overlay:track", producer: screenOverlayProducers.render, inputs: { canvas: input("canvas"), space: input("space"), program: operation("overlay:program") }, result: { kind: "output", name: "track" } },+ { id: "overlay:track", producer: screenOverlayProducers.render, inputs: { canvas: input("canvas"), timeline: input("timeline"), program: operation("overlay:program") }, result: { kind: "output", name: "track" } },);return sealGraphFragment({ inputs: [{ name: "canvas", type: spatialTypes.canvas }, { name: "header", type: screenOverlayTypes.header },- { name: "space", type: programSpaceTypes.programSpace }, ...[...types].map(([inputName, type]) => ({ name: inputName, type })),+ { name: "timeline", type: timelineTypes.track }, ...[...types].map(([inputName, type]) => ({ name: inputName, type })),], operations, exports: [{ name: "program", type: screenOverlayTypes.program, root: operation("overlay:program") },{ name: "track", type: compositionTypes.visualTrack, root: operation("overlay:track") },diff --git a/packages/screen-overlay/src/manifest.ts b/packages/screen-overlay/src/manifest.tsindex d1717642..69db475c 100644--- a/packages/screen-overlay/src/manifest.ts+++ b/packages/screen-overlay/src/manifest.ts@@ -1,11 +1,12 @@+import { timelineTypes } from "@hypit/timeline";import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";import { readFile } from "node:fs/promises";import { compositionDependency, compositionTypes } from "@hypit/composition";import { narrativeDependency } from "@hypit/narrative";-import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";+import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";-import { semanticTrackDependency } from "@hypit/semantic-track";+import { timelineDependency } from "@hypit/timeline";import { spatialDependency, spatialTypes } from "@hypit/spatial";import { temporalDependency, temporalTypes } from "@hypit/temporal";import { temporalWindowAttributeVocabulary } from "@hypit/temporal-markup";@@ -65,7 +66,7 @@ export const screenOverlayHeaderSchema: ValueSchema = object({ id: { schema: strexport const screenOverlayItemSpecSchema: ValueSchema = itemSpec;export const screenOverlaySetSchema: ValueSchema = object({ items: { schema: { kind: "array", items: item } } });export const screenOverlayProgramSchema: ValueSchema = object({ id: { schema: string }, items: { schema: { kind: "array", minItems: 1, items: item } } });-const appendInputs = [{ name: "set", type: screenOverlayTypes.set }, { name: "header", type: screenOverlayTypes.header }, { name: "space", type: programSpaceTypes.programSpace }, { name: "spec", type: screenOverlayTypes.itemSpec }, { name: "window", type: temporalTypes.window }] as const;+const appendInputs = [{ name: "set", type: screenOverlayTypes.set }, { name: "header", type: screenOverlayTypes.header }, { name: "timeline", type: timelineTypes.track }, { name: "spec", type: screenOverlayTypes.itemSpec }, { name: "window", type: temporalTypes.window }] as const;const itemAttributes = [{ name: "id", kind: "identifier", required: false,@@ -269,7 +270,7 @@ export const screenOverlayMarkupSurfaces = [{ name: "track", type: compositionTypes.visualTrack,summary: "The rendered overlay, an ordinary peer VisualTrack." },],- example: `<screen:Track id="effects" semantic={speech.semantic} canvas={vertical}>+ example: `<screen:Track id="effects" timeline={speech.timeline} canvas={vertical}><screen:Flash during={story.selection.overlay} z="80"color="#ffffff" intensity="0.6" attack="2" hold="2" decay="6"/></screen:Track>`,@@ -288,7 +289,7 @@ export const screenOverlayMarkupSurfaces = [export const screenOverlayManifest: ModuleManifest = {format: "hypit.module@1", name: screenOverlayModuleRef.name, version: screenOverlayModuleRef.version,- dependencies: [narrativeDependency, programSpaceDependency, semanticTrackDependency, spatialDependency, temporalDependency, compositionDependency],+ dependencies: [narrativeDependency, timelineDependency, spatialDependency, temporalDependency, compositionDependency],types: [{ name: screenOverlayTypes.header.name },{ name: screenOverlayTypes.itemSpec.name },@@ -299,7 +300,7 @@ export const screenOverlayManifest: ModuleManifest = {{ name: screenOverlayProducers.createSet.name, inputs: [], outputs: [{ name: "set", type: screenOverlayTypes.set }], needs: [] },{ name: screenOverlayProducers.appendItem.name, inputs: [...appendInputs], outputs: [{ name: "set", type: screenOverlayTypes.set }], needs: [] },{ name: screenOverlayProducers.finalize.name, inputs: [{ name: "set", type: screenOverlayTypes.set }, { name: "header", type: screenOverlayTypes.header }], outputs: [{ name: "program", type: screenOverlayTypes.program }], needs: [] },- { name: screenOverlayProducers.render.name, inputs: [{ name: "canvas", type: spatialTypes.canvas }, { name: "space", type: programSpaceTypes.programSpace }, { name: "program", type: screenOverlayTypes.program }], outputs: [{ name: "track", type: compositionTypes.visualTrack }], needs: [] },+ { name: screenOverlayProducers.render.name, inputs: [{ name: "canvas", type: spatialTypes.canvas }, { name: "timeline", type: timelineTypes.track }, { name: "program", type: screenOverlayTypes.program }], outputs: [{ name: "track", type: compositionTypes.visualTrack }], needs: [] },],};export const screenOverlayDependency = { module: screenOverlayModuleRef } as const;diff --git a/packages/screen-overlay/src/program.ts b/packages/screen-overlay/src/program.tsindex 46721b52..43302769 100644--- a/packages/screen-overlay/src/program.ts+++ b/packages/screen-overlay/src/program.ts@@ -1,7 +1,7 @@+import type { Timeline } from "@hypit/timeline";import { assertVisualTrackIdentity, sealVisualTrack } from "@hypit/composition";import type { VisualAnimation, VisualElement, VisualStyleDeclaration, VisualTrack } from "@hypit/composition";import { assertProgramSpaceIdentity } from "@hypit/program-space";-import type { ProgramSpace } from "@hypit/program-space";import { canonicalize } from "@hypit/protocol";import { assertCanvasSpace } from "@hypit/spatial";import type { CanvasSpace } from "@hypit/spatial";@@ -107,11 +107,11 @@ export function assertScreenOverlaySet(value: ScreenOverlaySet): void {}function realized(- set: ScreenOverlaySet, header: ScreenOverlayHeader, space: ProgramSpace, spec: ScreenOverlayItemSpec,+ set: ScreenOverlaySet, header: ScreenOverlayHeader, timeline: Timeline, spec: ScreenOverlayItemSpec,window: ProjectedWindow,): ScreenOverlaySet {assertScreenOverlaySet(set); assertScreenOverlayHeader(header); assertScreenOverlayItemSpec(spec);- assertTemporalWindowFor(window, { subjectId: spec.id, space });+ assertTemporalWindowFor(window, { subjectId: spec.id, space: timeline });const addition = {id: window.id,subjectId: spec.id,@@ -128,11 +128,11 @@ function realized(export function appendProjectedScreenOverlay(set: ScreenOverlaySet,header: ScreenOverlayHeader,- space: ProgramSpace,+ timeline: Timeline,spec: ScreenOverlayItemSpec,window: ProjectedWindow,): ScreenOverlaySet {- return realized(set, header, space, spec, window);+ return realized(set, header, timeline, spec, window);}export function sealScreenOverlayProgram(value: ScreenOverlayProgram): ScreenOverlayProgram {const normalized = { id: value.id,@@ -307,15 +307,15 @@ function overlayElements(content: ScreenOverlayComponent, canvas: CanvasSpace, d}}-export function renderScreenOverlay(canvas: CanvasSpace, space: ProgramSpace, program: ScreenOverlayProgram): VisualTrack {- assertCanvasSpace(canvas); assertProgramSpaceIdentity(space); assertScreenOverlayProgram(program);+export function renderScreenOverlay(canvas: CanvasSpace, timeline: Timeline, program: ScreenOverlayProgram): VisualTrack {+ assertCanvasSpace(canvas); assertProgramSpaceIdentity(timeline); assertScreenOverlayProgram(program);const track = sealVisualTrack({- programSpaceId: space.id, visualIr: "hypit.visual-ir@1", id: program.id,+ programSpaceId: timeline.id, visualIr: "hypit.visual-ir@1", id: program.id,presents: program.items.map((item) => ({id: item.id, subjectId: item.subjectId, span: { ...item.span }, stacking: { ...item.stacking },elements: overlayElements(item.content, canvas, item.span.endFrameExclusive - item.span.startFrame),})),});- assertVisualTrackIdentity(track, space);+ assertVisualTrackIdentity(track, timeline);return track;}diff --git a/packages/screen-overlay/src/surface.ts b/packages/screen-overlay/src/surface.tsindex 7713d184..9b686cf8 100644--- a/packages/screen-overlay/src/surface.ts+++ b/packages/screen-overlay/src/surface.ts@@ -1,4 +1,4 @@-import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";+import { resolveTemporalContext } from "@hypit/temporal-markup";import { spatialTypes } from "@hypit/spatial";import type { StructuredElement, StructuredSurfaceHandler, SurfaceComponentDraft, SurfaceRecordDraft, SurfaceResolvedReference, MarkupAttributeValue } from "@hypit/markup";import { createTemporalWindowProjection, temporalWindowAttributeNames } from "@hypit/temporal-markup";@@ -57,16 +57,15 @@ function content(element: StructuredElement): { readonly value: ScreenOverlayCom}export const decodeScreenOverlaySurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {- allowed(element, ["id", "canvas", "semantic"]); const id = text(element, "id");+ allowed(element, ["id", "canvas", "timeline"]); const id = text(element, "id");const canvas = ref(element.attributes.canvas, `${element.name}.canvas`, spatialTypes.canvas, resolveReference);const context = resolveTemporalContext({ element, resolveReference });- const time = createTemporalSpace({ id: id, element, ...context });const headerId = `${id}.header`; const records: SurfaceRecordDraft[] = [{ id: headerId, type: screenOverlayTypes.header,value: { kind: "inline", value: sealScreenOverlayHeader({ id }) }, range: element.range }];- const temporalComponents: SurfaceComponentDraft[] = [...time.components];- const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [...time.fragments];+ const temporalComponents: SurfaceComponentDraft[] = [];+ const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [];const fragmentItems: Parameters<typeof createScreenOverlayFragment>[0][number][] = [];- const inputs: Record<string, typeof canvas.ref> = { canvas: canvas.ref, header: { kind: "record", id: headerId }, space: time.space.ref };+ const inputs: Record<string, typeof canvas.ref> = { canvas: canvas.ref, header: { kind: "record", id: headerId }, timeline: context.timeline.ref };let index = 0;for (const child of element.children) {if (child.kind === "text") { if (child.value.trim()) throw new Error(`${element.name} accepts only component children.`); continue; }@@ -76,7 +75,7 @@ export const decodeScreenOverlaySurface: StructuredSurfaceHandler = ({ element,const itemSpec = sealScreenOverlayItemSpec({id: optionalText(child, "id") ?? `${id}.${decoded.value.kind}.${suffix}`, content: decoded.value,stackingOrder: integer(child, "z") });- const temporal = createTemporalWindowProjection({ id: itemSpec.id, element: child, ...context, space: time.space, resolveReference });+ const temporal = createTemporalWindowProjection({ id: itemSpec.id, element: child, ...context, resolveReference });records.push(...temporal.records); temporalComponents.push(...temporal.components); temporalFragments.push(...temporal.fragments);const specId = `${id}.item.${suffix}.spec`; const specName = `item-${suffix}-spec`;records.push({ id: specId, type: screenOverlayTypes.itemSpec, value: { kind: "inline", value: itemSpec }, range: child.range });diff --git a/packages/screen-overlay/test/screen-overlay.test.ts b/packages/screen-overlay/test/screen-overlay.test.tsindex a2dae0d5..1ced2bc3 100644--- a/packages/screen-overlay/test/screen-overlay.test.ts+++ b/packages/screen-overlay/test/screen-overlay.test.ts@@ -1,7 +1,8 @@+import { sealTimeline } from "@hypit/timeline";import assert from "node:assert/strict";import test from "node:test";import { fixtureResource } from "../../../test/fixture-resource.js";-import { semanticTrackFixture } from "../../../test/semantic-track-fixture.js";+import { timelineFixture } from "../../../test/timeline-fixture.js";import { projectProgramWindow } from "../../../test/temporal-fixture.js";import { artifactManifest } from "@hypit/artifact";@@ -29,7 +30,7 @@ import {sealScreenOverlayItemSpec,} from "@hypit/screen-overlay";import type { ScreenOverlayComponent } from "@hypit/screen-overlay";-import { semanticTrackDependency, semanticTrackManifest, semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";+import { timelineDependency, timelineManifest, timelineProducers, timelineTypes } from "@hypit/timeline";import { speechEvidenceManifest } from "@hypit/speech-evidence";import { speechManifest } from "@hypit/speech";import { sealCanvasSpace, spatialComponent, spatialDependency, spatialManifest, spatialTypes } from "@hypit/spatial";@@ -43,10 +44,10 @@ const canvas = sealCanvasSpace({widthPx: 1080, heightPx: 1920,origin: "top-left", xDirection: "right", yDirection: "down", pixelAspect: "square",});-const space = sealProgramSpace({ id: "test-space", durationSec: 2, frameRate: { numerator: 30, denominator: 1 },+const space = sealTimeline({ items: [], id: "test-space", durationSec: 2, frameRate: { numerator: 30, denominator: 1 },});const header = sealScreenOverlayHeader({ id: "screen" });-const semantic = semanticTrackFixture(space);+const semantic = timelineFixture(space);const components: readonly ScreenOverlayComponent[] = [{ kind: "flash", color: "#ffffff", intensity: 0.9, attackFrames: 2, holdFrames: 3, decayFrames: 5 },{ kind: "color-wash", color: "#2244ff", opacity: 0.2 },@@ -124,7 +125,7 @@ test("overlay Tracks interleave with peer Tracks only through absolute stacking"test("the Screen Overlay Fragment publishes its Program and peer VisualTrack", () => {const fragment = createScreenOverlayFragment([{ specName: "spec", windowName: "window" }]);- assert.deepEqual(fragment.inputs.map((input) => input.name), ["canvas", "header", "space", "spec", "window"]);+ assert.deepEqual(fragment.inputs.map((input) => input.name), ["canvas", "header", "spec", "timeline", "window"]);assert.deepEqual(fragment.exports.map((output) => [output.name, output.type.name]), [["program", "ScreenOverlayProgram"],["track", "VisualTrack"],@@ -136,13 +137,13 @@ test("the self-described Screen Surface parses into a finite peer-Track graph",const fixtureSurfaceDigest = fixtureResource("example.screen-inputs/surface@1");const fixtureSurface = {name: "inputs", tag: "Inputs", mode: "structured",- outputs: [spatialTypes.canvas, semanticTrackTypes.track],+ outputs: [spatialTypes.canvas, timelineTypes.track],} as const;const fixtureManifest: ModuleManifest = {format: "hypit.module@1",name: fixtureModule.name,version: fixtureModule.version,- dependencies: [spatialDependency, semanticTrackDependency],+ dependencies: [spatialDependency, timelineDependency],types: [],capabilities: [],producers: [],@@ -154,7 +155,7 @@ test("the self-described Screen Surface parses into a finite peer-Track graph",programSpaceManifest,speechManifest,speechEvidenceManifest,- semanticTrackManifest,+ timelineManifest,spatialManifest,svsManifest,temporalManifest,@@ -167,7 +168,7 @@ test("the self-described Screen Surface parses into a finite peer-Track graph",registry.registerStructured({ module: fixtureModule, declaration: fixtureSurface, handler: ({ element }) => ({records: [{ id: "canvas", type: spatialTypes.canvas, value: { kind: "inline", value: canvas }, range: element.range },- { id: "semantic", type: semanticTrackTypes.track, value: { kind: "inline", value: semantic }, range: element.range },+ { id: "semantic", type: timelineTypes.track, value: { kind: "inline", value: semantic }, range: element.range },],components: [],fragments: [],@@ -193,7 +194,7 @@ test("the self-described Screen Surface parses into a finite peer-Track graph",<import as="fixture" from="example.screen-inputs@1"/><import as="screen" from="@hypit/screen-overlay@1"/><fixture:Inputs/>- <screen:Track id="screen-fx" canvas={canvas} semantic={semantic}>+ <screen:Track id="screen-fx" canvas={canvas} timeline={semantic}><screen:Flash during="program" z="70" color="#ffffff" intensity="0.9" attack="2" hold="3" decay="5"/></screen:Track></svml>`,@@ -215,7 +216,6 @@ test("the self-described Screen Surface parses into a finite peer-Track graph",temporalProducers.projectProgramInstant.name,temporalProducers.projectProgramInstant.name,temporalProducers.composeWindow.name,- semanticTrackProducers.projectProgramSpace.name,- screenOverlayProducers.render.name,+ screenOverlayProducers.render.name,].sort());});diff --git a/packages/script-studio/package.json b/packages/script-studio/package.jsonindex d7377766..9372b261 100644--- a/packages/script-studio/package.json+++ b/packages/script-studio/package.json@@ -4,10 +4,16 @@"license": "SEE LICENSE IN LICENSE","private": true,"type": "module",- "exports": { ".": "./src/index.ts" },- "hypit": { "activation": "./src/activation.ts" },+ "exports": {+ ".": "./src/index.ts"+ },+ "hypit": {+ "activation": "./src/activation.ts"+ },"dependencies": {"@hypit/script": "workspace:*",- "@hypit/studio-adapter": "workspace:*"+ "@hypit/studio-adapter": "workspace:*",+ "@hypit/narrative": "workspace:*",+ "@hypit/protocol": "workspace:*"}}diff --git a/packages/script-studio/src/index.ts b/packages/script-studio/src/index.tsindex b81640c4..ab571c98 100644--- a/packages/script-studio/src/index.ts+++ b/packages/script-studio/src/index.ts@@ -1,9 +1,11 @@+import { projectScriptTimeline } from "./projection.js";import { adjustScriptMoment, adjustScriptSelection, parseScript, scriptModuleRef } from "@hypit/script";import type { StudioScriptCompanion } from "@hypit/studio-adapter";export const scriptStudioCompanions: readonly StudioScriptCompanion[] = [{id: "script",match: { module: scriptModuleRef, surface: "script" },+ project: projectScriptTimeline,observe(input) {const narrativeId = input.attributes.id ?? "script";if (typeof narrativeId !== "string" || narrativeId.length === 0) return undefined;diff --git a/packages/script-studio/src/projection.ts b/packages/script-studio/src/projection.tsnew file mode 100644index 00000000..962a56aa--- /dev/null+++ b/packages/script-studio/src/projection.ts@@ -0,0 +1,105 @@+import { narrativeTypes } from "@hypit/narrative";+import { sameType } from "@hypit/protocol";+import type { StudioScriptProjectionInput, StudioScriptProjection } from "@hypit/studio-adapter";++type NarrativeValue = {+ readonly id?: string;+ readonly segments?: readonly {+ readonly id: string;+ readonly startAnchorId: string;+ readonly endAnchorId: string;+ }[];+ readonly tokens?: readonly {+ readonly id: string;+ readonly segmentId: string;+ readonly startAnchorId: string;+ readonly endAnchorId: string;+ readonly text: string;+ }[];+ readonly selections?: readonly {+ readonly id: string;+ readonly startAnchorId: string;+ readonly endAnchorId: string;+ }[];+ readonly moments?: readonly {+ readonly id: string;+ readonly anchorId: string;+ }[];+ readonly semanticIndex?: {+ readonly anchors?: readonly {+ readonly id: string;+ readonly kind: "program-start" | "segment-start" | "segment-end" | "token-start" | "token-end" | "program-end";+ readonly segmentId?: string;+ readonly tokenId?: string;+ }[];+ };+};++export function projectScriptTimeline(input: StudioScriptProjectionInput): StudioScriptProjection {+ const records = input.values.filter((item) =>+ sameType(item.type, narrativeTypes.narrative)+ && (item.value as NarrativeValue).id === input.source.narrativeId);+ if (records.length > 1) {+ throw new Error(`Studio Narrative id ${input.source.narrativeId} resolves to more than one authored value.`);+ }+ const record = records[0];+ const narrative = record?.value as NarrativeValue | undefined;+ if (narrative === undefined) throw new Error("Studio Narrative is not an inline authored value.");++ const segmentRanges = new Map((input.source.segments ?? []).map((item) => [item.id, item.range]));+ const tokenRanges = new Map((input.source.tokens ?? []).map((item) => [item.id, item.range]));+ const frame = (id: string): number | undefined => input.anchors.get(id);++ const segments = (narrative.segments ?? []).flatMap((segment) => {+ const startFrame = frame(segment.startAnchorId);+ const endFrame = frame(segment.endAnchorId);+ if (startFrame === undefined || endFrame === undefined) return [];+ return [{+ id: segment.id,+ startFrame,+ endFrameExclusive: Math.max(startFrame + 1, endFrame),+ ...(segmentRanges.has(segment.id) ? { range: segmentRanges.get(segment.id)! } : {}),+ }];+ });+ const tokens = (narrative.tokens ?? []).flatMap((token) => {+ const startFrame = frame(token.startAnchorId);+ const endFrame = frame(token.endAnchorId);+ if (startFrame === undefined || endFrame === undefined) return [];+ return [{+ id: token.id,+ segmentId: token.segmentId,+ text: token.text,+ startFrame,+ endFrameExclusive: Math.max(startFrame + 1, endFrame),+ ...(tokenRanges.has(token.id) ? { range: tokenRanges.get(token.id)! } : {}),+ }];+ });+ const anchors = (narrative.semanticIndex?.anchors ?? []).flatMap((anchor) => {+ const at = frame(anchor.id);+ if (at === undefined) return [];+ return [{+ id: anchor.id,+ kind: anchor.kind,+ frame: at,+ ...(anchor.segmentId === undefined ? {} : { segmentId: anchor.segmentId }),+ ...(anchor.tokenId === undefined ? {} : { tokenId: anchor.tokenId }),+ }];+ });+ const selections = (narrative.selections ?? []).flatMap((selection) => {+ const startFrame = frame(selection.startAnchorId);+ const endFrameExclusive = frame(selection.endAnchorId);+ if (startFrame === undefined || endFrameExclusive === undefined || endFrameExclusive <= startFrame) return [];+ return [{+ id: selection.id,+ startAnchorId: selection.startAnchorId,+ endAnchorId: selection.endAnchorId,+ startFrame,+ endFrameExclusive,+ }];+ });+ const moments = (narrative.moments ?? []).flatMap((moment) => {+ const at = frame(moment.anchorId);+ return at === undefined ? [] : [{ id: moment.id, anchorId: moment.anchorId, frame: at }];+ });+ return { anchors, segments, tokens, selections, moments };+}diff --git a/packages/script/README.md b/packages/script/README.mdindex 2efc304c..764b6824 100644--- a/packages/script/README.md+++ b/packages/script/README.md@@ -53,8 +53,8 @@ The Surface exports one full Narrative plus narrow, immutable views:- `script.moment.<id>` is a reusable explicit Moment.`@hypit/caption` projects `script.selection.<id>` or a Role onto complete Caption Alignment Units;-it then joins those units to a SemanticTrack for frame timing. Seedance consumes dialogue `Text`,-Estimate and TTS consume speech `Text`, and Speech Track consumes the Segment excerpt. None imports+it then joins those units to a Timeline for frame timing. Seedance consumes dialogue `Text`,+Estimate and TTS consume speech `Text`, and semantic preparation consumes the Segment excerpt. None importsScript's parser AST. Another authoring package may produce the same ordinary Text, Narrative andCaptionDocument contracts.@@ -157,3 +157,16 @@ distinct boundaries writable.Writeback reparses the result to retain the intended identities and unchanged narrative/captioncontent. These checks concern authored order, not frame order. A consumer's temporal projectiondetermines whether the resulting Instant or Window is usable.++## Complete authored content and narrow exports++The root `story` Record contains the complete Narrative: speech structure, semantic references and+its CaptionDocument. `story.caption`, `story.selection.<id>`, Segment excerpts and dialogue/speech+Text outputs remain explicit narrow exports derived from that content. Caption rendering can read+the document and Timeline. Timed Uses change its presentation; their semantic references are+projected through the same Timeline as other components. Word attributes remain in the document+for families that give those words structural visual roles.++For content lookup, use the Narrative package's `narrativeTokensForSelection` or+`narrativeSelectionTokenRange`. These query authored order; Timeline separately locates the same+anchors in the assembled video. No timestamps or display Styles enter the Script value.diff --git a/packages/script/src/manifest.ts b/packages/script/src/manifest.tsindex 23d50c78..9d2dd50b 100644--- a/packages/script/src/manifest.ts+++ b/packages/script/src/manifest.ts@@ -32,7 +32,7 @@ export const scriptMarkupSurfaces = [text: "The element's own content is the Script body: named Segments holding prose, Role Cues, Dual Text, flat token attributes, and zero-width Selection and Moment markers. It carries no timecode, no media reference and no generation parameter.",ports: [{ name: "", type: narrativeType,- summary: "The whole authored Narrative, addressed by the element's own id." },+ summary: "The complete authored Narrative, including speech structure, semantic anchors and its CaptionDocument, addressed by the element's own id." },{ name: "segment.<id>", type: narrativeExcerptType,summary: "One Segment as a narrow Excerpt, used to associate a generated Take with that Segment." },{ name: "segment.<id>.dialogue", type: textTypes.text,@@ -40,7 +40,7 @@ export const scriptMarkupSurfaces = [{ name: "segment.<id>.speech", type: textTypes.text,summary: "One Segment as pronunciation only, with Role Cue labels dropped." },{ name: "caption", type: captionDocumentType,- summary: "The complete Script-owned CaptionDocument: display Words, N:M Alignment Units, Cue breaks and speech correspondence." },+ summary: "The Narrative's CaptionDocument exported as a narrow view: display Words, N:M Alignment Units, Cue breaks and speech correspondence." },{ name: "selection.<id>", type: narrativeSelectionType,summary: "One named range over the Narrative, reusable wherever a Selection is read." },{ name: "moment.<id>", type: narrativeMomentType,diff --git a/packages/script/src/narrative.ts b/packages/script/src/narrative.tsindex fbe8c311..4afecaec 100644--- a/packages/script/src/narrative.ts+++ b/packages/script/src/narrative.ts@@ -180,6 +180,7 @@ export function narrativeMomentValue(moment: ParsedNarrative["moments"][number],export function narrativeValue(parsed: ParsedNarrative, id: string): CanonicalValue {return canonicalize({id,+ caption: captionDocument(parsed, `${id}.caption`, id),segments: parsed.segments.map((segment) => ({id: segment.id,startAnchorId: segment.startAnchorId,diff --git a/packages/script/src/surface.ts b/packages/script/src/surface.tsindex 096cdca8..80d0f46f 100644--- a/packages/script/src/surface.ts+++ b/packages/script/src/surface.ts@@ -1,6 +1,5 @@import { ScriptSyntaxError } from "./error.js";import {- captionDocumentValue,narrativeDialogueTextValue,narrativeSegmentExcerptValue,narrativeMomentValue,@@ -73,6 +72,8 @@ export function decodeScriptSurface(input: ScriptSurfaceInput): ScriptSurfaceOutinput.source.slice(input.contentStart, close.start),input.contentStart,);+ const narrative = narrativeValue(parsed, rawId);+ const caption = (narrative as { readonly caption: import("@hypit/protocol").CanonicalValue }).caption;const captionId = `${rawId}.caption`;return {nextOffset: close.end,@@ -80,7 +81,7 @@ export function decodeScriptSurface(input: ScriptSurfaceInput): ScriptSurfaceOut{id: rawId,type: narrativeType,- value: { kind: "inline", value: narrativeValue(parsed, rawId) },+ value: { kind: "inline", value: narrative },range: { start: input.openingStart, end: close.end },},...parsed.segments.map((segment) => ({@@ -106,7 +107,7 @@ export function decodeScriptSurface(input: ScriptSurfaceInput): ScriptSurfaceOut{id: captionId,type: captionDocumentType,- value: { kind: "inline" as const, value: captionDocumentValue(parsed, captionId, rawId) },+ value: { kind: "inline" as const, value: caption },range: { start: input.openingStart, end: close.end },},...parsed.selections.map((selection) => ({diff --git a/packages/script/src/types.ts b/packages/script/src/types.tsindex 254e195f..4fa948d4 100644--- a/packages/script/src/types.ts+++ b/packages/script/src/types.ts@@ -77,7 +77,7 @@ export type ParsedCaptionRegion = {export type ParsedNarrative = Omit<Narrative,- "id" | "segments" | "tokens" | "turns" | "selections" | "moments"+ "id" | "segments" | "tokens" | "turns" | "selections" | "moments" | "caption"> & {/** Exact Script body range, used only for source-preserving Program-boundary edits. */readonly sourceRange: SourceRange;diff --git a/packages/script/test/narrative-views.test.ts b/packages/script/test/narrative-views.test.tsnew file mode 100644index 00000000..dbd141f8--- /dev/null+++ b/packages/script/test/narrative-views.test.ts@@ -0,0 +1,64 @@+import assert from "node:assert/strict";+import test from "node:test";+import { decodeScriptSurface, narrativeValue, parseScript } from "@hypit/script";+import { assertNarrativeIdentity, narrativeAnchorTokenBoundary, narrativeSelectionTokenRange,+ narrativeTokensForSelection, narrativeTypes } from "@hypit/narrative";+import type { Narrative, NarrativeSelectionRef } from "@hypit/narrative";+import { captionUnitsForSelection, captionTypes, decodeHiddenCaptionStyleSurface, sealCaptionStyle } from "@hypit/caption";+import type { CaptionProgram } from "@hypit/caption";+import { parseStructuredElement } from "@hypit/markup";+import type { SurfaceResolvedReference } from "@hypit/markup";++const body = `<intro><HOST>Try @brand <hypit|Hai-Pit> @/brand today. ||</intro>+<gap/>+<closing><HOST>现在 @name <声工坊|voice workshop> @/name 好用。<|indeed></closing>`;++function authored() {+ return narrativeValue(parseScript("views.svml", body), "story") as unknown as Narrative;+}++test("Script exports complete author content and a caption view from that same value", () => {+ const source = `<script id="story">${body}</script>`;+ const result = decodeScriptSurface({ sourceName: "views.svml", source, tag: "script",+ attributes: { id: "story" }, openingStart: 0, contentStart: source.indexOf(">") + 1 });+ const root = result.records.find((record) => record.id === "story")!;+ const caption = result.records.find((record) => record.id === "story.caption")!;+ assert.equal(root.value.kind, "inline");+ assert.equal(caption.value.kind, "inline");+ if (root.value.kind !== "inline" || caption.value.kind !== "inline") return;+ const narrative = root.value.value as unknown as Narrative;+ assertNarrativeIdentity(narrative);+ assert.strictEqual(narrative.caption, caption.value.value);+ assert.match(narrative.caption.words.map((word) => word.text).join(""), /hypit/);+ assert.match(narrative.caption.words.map((word) => word.text).join(""), /声工坊/);+ assert.ok(!narrative.caption.words.some((word) => word.text.includes("indeed")));+ assert.ok(narrative.tokens.some((token) => token.text === "indeed"));+ assert.equal(narrative.caption.cueBreaks.length, 1);+});++test("Narrative selections query authored speech while Caption preserves display correspondence", () => {+ const narrative = authored();+ const brand = narrative.selections.find((selection) => selection.id === "brand")!;+ assert.equal(narrativeTokensForSelection(narrative, brand).map((token) => token.text).join(" "), "Hai-Pit");+ const name = narrative.selections.find((selection) => selection.id === "name")!;+ const selected = new Set(captionUnitsForSelection(narrative, name).unitIds);+ assert.equal(narrative.caption.words.filter((word) => selected.has(word.unitId)).map((word) => word.text).join(""), "声工坊");+ const unit = narrative.caption.units.find((unit) => selected.has(unit.id))!;+ const first = narrative.tokens.find((token) => token.id === unit.sourceTokenIds[0])!;+ const last = narrative.tokens.find((token) => token.id === unit.sourceTokenIds.at(-1))!;+ assert.notEqual(first.id, last.id);+ assert.throws(() => captionUnitsForSelection(narrative, {+ id: "partial", startAnchorId: first.endAnchorId, endAnchorId: last.endAnchorId,+ }), /partially selects/);+ const foreign: NarrativeSelectionRef = { ...brand, narrativeId: "other" };+ assert.throws(() => narrativeTokensForSelection(narrative, foreign), /another Narrative/);+});++test("Content queries preserve structural boundaries even when no words lie between them", () => {+ const narrative = authored();+ const gap = narrative.segments.find((segment) => segment.id === "gap")!;+ assert.deepEqual(narrativeTokensForSelection(narrative, { id: "gap", startAnchorId: gap.startAnchorId, endAnchorId: gap.endAnchorId }), []);+ assert.equal(narrativeAnchorTokenBoundary(narrative, gap.startAnchorId), narrativeAnchorTokenBoundary(narrative, gap.endAnchorId));+ assert.throws(() => narrativeSelectionTokenRange(narrative, { id: "backwards", startAnchorId: gap.endAnchorId, endAnchorId: gap.startAnchorId }), /anchor order/);+ assert.deepEqual(narrativeTokensForSelection(narrative, { id: "whole", startAnchorId: "program:start", endAnchorId: "program:end" }), narrative.tokens);+});diff --git a/packages/semantic-track/README.md b/packages/semantic-track/README.mddeleted file mode 100644index 259528da..00000000--- a/packages/semantic-track/README.md+++ /dev/null@@ -1,21 +0,0 @@-# @hypit/semantic-track--`SemanticTrack` is the single global semantic-timeline fact. It is the continuous program skeleton:-every item is a real segment-local `SemanticTake`, and global frame positions are the prefix sum of-their normalized local frame counts. There are no gap items or empty items. A Take may carry only-audio or only visual media; its word array may be empty. An empty Script Segment such as-`<empty></empty>` can therefore retain a real media span and both Segment anchors without speech.-An empty word array is not an empty Track item. Visual placement is deliberately not-part of this semantic truth.--The package exposes pure projections to `ProgramSpace` and `AudioTrack`, plus the-selection/segment/moment lookup functions used by semantic components. It performs no media I/O,-mixing, transcription or rendering.--## Prepared material in a semantic window--`projectSemanticMedia(track, window?)` returns each intersecting Take's `media`, `segmentId`, global-`span` and local `source` frame span. The default window is the whole performance. An interval-beginning inside a Take preserves its source offset. Audio-only Takes remain represented so the-consumer can choose the streams it displays. This projection makes no layout or layering decisions;-Media and custom visual components can consume the same prepared performance.diff --git a/packages/semantic-track/src/activation.ts b/packages/semantic-track/src/activation.tsdeleted file mode 100644index b2ae4dc4..00000000--- a/packages/semantic-track/src/activation.ts+++ /dev/null@@ -1,8 +0,0 @@-import { semanticTrackComponent, semanticTrackManifest } from "./index.js";--export const hypitPackage = {- format: "hypit.node-package@1" as const,- modules: [{ manifest: semanticTrackManifest }],- components: [semanticTrackComponent],-};-export default hypitPackage;diff --git a/packages/semantic-track/src/component.ts b/packages/semantic-track/src/component.tsdeleted file mode 100644index 65c1319b..00000000--- a/packages/semantic-track/src/component.ts+++ /dev/null@@ -1,24 +0,0 @@-import type { ComponentPackage } from "@hypit/component-kit";-import type { StoredValue } from "@hypit/protocol";-import { canonicalize } from "@hypit/protocol";--import { semanticTrackProducers, semanticTrackTypes } from "./manifest.js";-import { assertSemanticTrackIdentity } from "./identity.js";-import { projectSemanticAudioTrack, projectSemanticProgramSpace } from "./projection.js";-import type { SemanticTrack } from "./types.js";--function track(value: StoredValue | undefined): SemanticTrack {- if (value?.kind !== "inline") throw new Error("SemanticTrack must be inline.");- return value.value as unknown as SemanticTrack;-}--export const semanticTrackComponent = {- producers: [- { producer: semanticTrackProducers.projectProgramSpace, handler: ({ inputs }) => ({ outputs: { space: { kind: "inline", value: canonicalize(projectSemanticProgramSpace(track(inputs.track?.value))) } }, needs: {} }) },- { producer: semanticTrackProducers.projectAudio, handler: ({ inputs }) => ({ outputs: { audio: { kind: "inline", value: canonicalize(projectSemanticAudioTrack(track(inputs.track?.value))) } }, needs: {} }) },- ],- validators: [{- type: semanticTrackTypes.track,- handler: ({ value }) => assertSemanticTrackIdentity(track(value)),- }],-} satisfies ComponentPackage;diff --git a/packages/semantic-track/src/identity.ts b/packages/semantic-track/src/identity.tsdeleted file mode 100644index 05b01055..00000000--- a/packages/semantic-track/src/identity.ts+++ /dev/null@@ -1,58 +0,0 @@-import { assertSemanticTakeIdentity } from "@hypit/speech";--import type { SemanticTrack, SemanticTrackSpan } from "./types.js";--export function sealSemanticTrack(value: SemanticTrack): SemanticTrack {- const track = structuredClone(value);- assertSemanticTrackIdentity(track);- return track;-}--export function assertSemanticTrackIdentity(track: SemanticTrack): void {- if (!track.id.trim() || !track.narrativeId.trim() || track.items.length === 0) {- throw new Error("SemanticTrack must have identities and contain at least one item.");- }- const segmentIds = new Set<string>();- const tokenIds = new Set<string>();- const anchorIds = new Set<string>();- let frameRate: SemanticTrack["items"][number]["take"]["media"]["timeline"]["frameRate"] | undefined;- for (const item of track.items) {- assertSemanticTakeIdentity(item.take);- if (item.take.narrativeId !== track.narrativeId) {- throw new Error(`SemanticTrack ${track.id} mixes Narrative ${item.take.narrativeId} into ${track.narrativeId}.`);- }- const rate = item.take.media.timeline.frameRate;- if (frameRate === undefined) frameRate = rate;- else if (frameRate.numerator !== rate.numerator || frameRate.denominator !== rate.denominator) {- throw new Error("SemanticTrack items must use one frame rate.");- }- const segmentId = item.take.segment.segmentId;- if (segmentIds.has(segmentId)) throw new Error(`SemanticTrack repeats Segment ${segmentId}.`);- segmentIds.add(segmentId);- for (const token of item.take.tokens) {- if (tokenIds.has(token.tokenId)) throw new Error(`SemanticTrack repeats Token ${token.tokenId}.`);- tokenIds.add(token.tokenId);- }- for (const anchor of item.take.anchors) {- if (anchor.identity === "program:start" || anchor.identity === "program:end") {- throw new Error(`SemanticTake cannot declare reserved Program Anchor ${anchor.identity}.`);- }- if (anchorIds.has(anchor.identity)) throw new Error(`SemanticTrack repeats Anchor ${anchor.identity}.`);- anchorIds.add(anchor.identity);- }- }-}--export function semanticTrackSpans(track: SemanticTrack): readonly SemanticTrackSpan[] {- assertSemanticTrackIdentity(track);- let frame = 0;- return track.items.map((item) => {- const startFrame = frame;- frame += item.take.media.timeline.frameCount;- return { item, startFrame, endFrameExclusive: frame };- });-}--export function semanticTrackFrameCount(track: SemanticTrack): number {- return semanticTrackSpans(track).at(-1)!.endFrameExclusive;-}diff --git a/packages/semantic-track/src/index.ts b/packages/semantic-track/src/index.tsdeleted file mode 100644index e1245f55..00000000--- a/packages/semantic-track/src/index.ts+++ /dev/null@@ -1,6 +0,0 @@-export { semanticTrackComponent } from "./component.js";-export { assertSemanticTrackIdentity, sealSemanticTrack, semanticTrackFrameCount, semanticTrackSpans } from "./identity.js";-export { assertNarrativeMomentIdentity, assertNarrativeSelectionIdentity, momentFrame, segmentFrameSpan, selectionFrameSpan, semanticAnchorFrames, tokenFrameSpan } from "./location.js";-export { semanticTrackDependency, semanticTrackManifest, semanticTrackModuleRef, semanticTrackProducers, semanticTrackSchema, semanticTrackTypes } from "./manifest.js";-export { projectSemanticAudioTrack, projectSemanticProgramSpace, projectSemanticMedia } from "./projection.js";-export type * from "./types.js";diff --git a/packages/semantic-track/src/manifest.ts b/packages/semantic-track/src/manifest.tsdeleted file mode 100644index a47eceb0..00000000--- a/packages/semantic-track/src/manifest.ts+++ /dev/null@@ -1,38 +0,0 @@-import { compositionDependency, compositionTypes } from "@hypit/composition";-import { narrativeDependency } from "@hypit/narrative";-import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";-import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";-import { semanticTakeSchema, speechDependency } from "@hypit/speech";--export const semanticTrackModuleRef = { name: "@hypit/semantic-track", version: "1" } as const;-export const semanticTrackTypes = {- track: { module: semanticTrackModuleRef, name: "SemanticTrack" },-} satisfies Record<string, TypeRef>;-export const semanticTrackProducers = {- projectProgramSpace: { module: semanticTrackModuleRef, name: "project-program-space" },- projectAudio: { module: semanticTrackModuleRef, name: "project-audio-track" },-} satisfies Record<string, ProducerRef>;--const string = { kind: "string", minLength: 1 } as const;-const object = (fields: Readonly<Record<string, { readonly schema: ValueSchema; readonly optional?: boolean }>>): ValueSchema => ({ kind: "object", fields });-export const semanticTrackSchema: ValueSchema = object({- id: { schema: string },- narrativeId: { schema: string },- items: { schema: { kind: "array", minItems: 1, items: object({- take: { schema: semanticTakeSchema },- }) } },-});--export const semanticTrackManifest: ModuleManifest = {- format: "hypit.module@1",- name: semanticTrackModuleRef.name,- version: semanticTrackModuleRef.version,- dependencies: [speechDependency, narrativeDependency, programSpaceDependency, compositionDependency],- types: [{ name: semanticTrackTypes.track.name }],- capabilities: [],- producers: [- { name: semanticTrackProducers.projectProgramSpace.name, inputs: [{ name: "track", type: semanticTrackTypes.track }], outputs: [{ name: "space", type: programSpaceTypes.programSpace }], needs: [] },- { name: semanticTrackProducers.projectAudio.name, inputs: [{ name: "track", type: semanticTrackTypes.track }], outputs: [{ name: "audio", type: compositionTypes.audioTrack }], needs: [] },- ],-};-export const semanticTrackDependency = { module: semanticTrackModuleRef } as const;diff --git a/packages/semantic-track/src/types.ts b/packages/semantic-track/src/types.tsdeleted file mode 100644index e3ccf79d..00000000--- a/packages/semantic-track/src/types.ts+++ /dev/null@@ -1,26 +0,0 @@-import type { SemanticTake } from "@hypit/speech";--export type SemanticTrackItem = {- readonly take: SemanticTake;-};--/**- * The continuous program skeleton. Every item is a real SemanticTake and the- * global frame domain is exactly their prefix sum: no gap or empty item exists.- */-export type SemanticTrack = {- readonly id: string;- readonly narrativeId: string;- readonly items: readonly SemanticTrackItem[];-};--export type SemanticTrackSpan = {- readonly item: SemanticTrackItem;- readonly startFrame: number;- readonly endFrameExclusive: number;-};--export type LocatedFrameSpan = {- readonly startFrame: number;- readonly endFrameExclusive: number;-};diff --git a/packages/sound-studio/README.md b/packages/sound-studio/README.mdnew file mode 100644index 00000000..718597bb--- /dev/null+++ b/packages/sound-studio/README.md@@ -0,0 +1,22 @@+# Sound Studio Companion++Sound contributes one Track with content and a bottom Uses band through the public Companion interface:++- The content lane shows placed audio Takes with waveforms. Each Take retains its actual interval, including gaps and overlaps. Picture-only Takes contribute no sound item.+- The Uses label band shows each authored Style application. Later declarations appear over earlier ones; selection raises an item for editing without changing that priority. Silent and fully covered Uses remain selectable.++One Sound label spans the waveform content and Uses band. Selecting a Take shows its Segment and placed frame range.+Selecting a Use shows its Style and resolved frame range, alongside the ordinary Style's authored+`gain` and `end-gain` parameters. Gain is displayed as a percentage: 100% writes the linear value 1.+The end gain defaults to the start gain. The parameter Companion exposes this dependent default;+the first edit inserts an explicit attribute into the Style.++The fields live in When and How. Shared Style edits affect all Uses selecting that Style.+Temporal gestures follow the existing Selection/Moment/clock projection and exact source endpoint+rules. Project Styles expose their own parameters through `createStudioCompanionHostFacet({ parameters })`,+matching their own Module and Surface. The Track opts `style` into this owner with `companion: true`.++`Sound Track.program` exports the already-computed ordered Use set before presentation coverage+is resolved. `Sound Track.audio` exports the effective audio arrangement. The Companion reads the+former to preserve editing choices even when they currently produce no sound; rendering consumes+the latter. Studio does not reconstruct the rule set from audible output or inspect Sound internals.diff --git a/packages/sound-studio/package.json b/packages/sound-studio/package.jsonnew file mode 100644index 00000000..e81518a1--- /dev/null+++ b/packages/sound-studio/package.json@@ -0,0 +1,20 @@+{+ "name": "@hypit/sound-studio",+ "version": "0.0.0-dev",+ "license": "SEE LICENSE IN LICENSE",+ "private": true,+ "type": "module",+ "exports": {+ ".": "./src/index.ts"+ },+ "hypit": {+ "activation": "./src/activation.ts"+ },+ "dependencies": {+ "@hypit/timeline": "workspace:*",+ "@hypit/temporal": "workspace:*",+ "@hypit/composition": "workspace:*",+ "@hypit/studio-adapter": "workspace:*",+ "@hypit/sound": "workspace:*"+ }+}diff --git a/packages/sound-studio/src/activation.ts b/packages/sound-studio/src/activation.tsnew file mode 100644index 00000000..105d10a8--- /dev/null+++ b/packages/sound-studio/src/activation.ts@@ -0,0 +1,6 @@+import { createStudioCompanionHostFacet } from "@hypit/studio-adapter";+import { soundStudioTrackCompanions, soundStudioParameterCompanions } from "./index.js";+export default {+ format: "hypit.node-package@1" as const,+ hostFacets: [createStudioCompanionHostFacet({ tracks: soundStudioTrackCompanions, parameters: soundStudioParameterCompanions })],+};diff --git a/packages/sound-studio/src/index.ts b/packages/sound-studio/src/index.tsnew file mode 100644index 00000000..1674e12c--- /dev/null+++ b/packages/sound-studio/src/index.ts@@ -0,0 +1,74 @@+import { soundModuleRef } from "@hypit/sound";+import type { SoundSet } from "@hypit/sound";+import { projectTimelineMedia, timelineTypes } from "@hypit/timeline";+import type { Timeline } from "@hypit/timeline";+import { temporalTypes } from "@hypit/temporal";+import { compositionTypes } from "@hypit/composition";+import { artifactPreview, authoredChildFor, previewLayer, requiredReferencedValue, requiredSurfaceValue, temporalLineageFor, temporalSemanticSource } from "@hypit/studio-adapter";+import type { StudioTrackCompanion, StudioTrackCompanionContext, StudioEntityDraft } from "@hypit/studio-adapter";++export function projectSound(context: StudioTrackCompanionContext): readonly StudioEntityDraft[] {+ const timeline = requiredReferencedValue(context, "timeline", timelineTypes.track) as Timeline;+ const program = requiredSurfaceValue(context, "program") as SoundSet;+ // Source content keeps its own placement. A Use changes its presentation, not its media clock.+ const content: StudioEntityDraft[] = projectTimelineMedia(timeline).flatMap((clip, index) => {+ if (clip.media.audio === undefined) return [];+ return [{+ id: `${context.track.outputRef}:content:${clip.segmentId}`,+ authoredId: clip.segmentId,+ display: { title: clip.segmentId, layers: [previewLayer(artifactPreview("audio", clip.media.audio.artifact.resource), "waveform")] },+ ...clip.span, stackOrder: index,+ inspector: [+ { id: "segment", label: "Segment", domain: "how", section: { id: "content", label: "Content" }, value: clip.segmentId },+ { id: "range", label: "Range", domain: "when", section: { id: "placement", label: "Placement" }, value: `${clip.span.startFrame}–${clip.span.endFrameExclusive}`, unit: "f", summary: "Placed interval; the end frame is exclusive." },+ ],+ presentation: { entity: "sound-content", chrome: "standard" },+ }];+ });+ const uses: StudioEntityDraft[] = program.uses.map((use, index) => {+ const authoredId = use.window.subjectId;+ const child = authoredChildFor(context, authoredId, [temporalTypes.windowSpec]);+ if (child === undefined) throw new Error(`Sound Use ${authoredId} has no author provenance.`);+ const style = child.referenceAttributes.style;+ const temporal = temporalLineageFor(context, authoredId, "window");+ const semantic = temporalSemanticSource(temporal);+ return {+ id: `${context.track.outputRef}:use:${authoredId}`, authoredId,+ display: { title: style ?? authoredId, layers: [] },+ ...use.window.span, stackOrder: index,+ elementRange: child.range,+ ...(temporal === undefined ? {} : { temporal }),+ ...(semantic === undefined ? {} : { markerId: semantic.id }),+ presentation: { entity: "sound-use", chrome: "standard" },+ band: "uses",+ inspector: [{ id: "range", label: "Range", domain: "when", section: { id: "placement", label: "Placement" }, value: `${use.window.span.startFrame}–${use.window.span.endFrameExclusive}`, unit: "f", summary: "Resolved Use interval; the end frame is exclusive. Timeline gestures follow this Use's author timing." }],+ };+ });+ return [...content, ...uses];+}++export const soundStudioTrackCompanions: readonly StudioTrackCompanion[] = [{+ id: "audio", role: "track",+ output: { type: compositionTypes.audioTrack, surface: "track", modules: [soundModuleRef] },+ family: "sound", label: "Sound", tone: "green", icon: "waveform",+ lane: { heightPx: 48 },+ requiredValues: ["program"],+ bands: [{+ id: "uses", placement: "after", heightPx: 15, display: "label",+ bindings: [{ name: "style", companion: true }],+ inspector: [{ binding: "style", label: "Style", domain: "how", section: { id: "sound", label: "Sound" }, control: "text", summary: "This Style is shared by every Use that selects it." }],+ }],+ project: projectSound,+}];++export const soundStudioParameterCompanions: readonly import("@hypit/studio-adapter").StudioParameterCompanion[] = [{+ id: "style", match: { module: soundModuleRef, surface: "style" },+ bindings: [ { name: "gain", writable: true, fallback: 1 },+ { name: "end-gain", writable: true, fallback: authored => Number(authored.gain ?? 1) } ],+ inspector: (["gain", "end-gain"] as const).map(name => ({+ binding: name, label: name === "gain" ? "Gain" : "End Gain", domain: "how",+ section: { id: "sound", label: "Sound" }, control: "number", unit: "%",+ number: { scale: 100, minimum: 0, maximum: 6400, step: 1 },+ summary: name === "gain" ? "Linear gain at the start of each Use. 100% preserves the source level." : "Linear gain at the end of each Use. The Use interval owns the ramp.",+ })),+}];diff --git a/packages/sound-studio/test/companion.test.ts b/packages/sound-studio/test/companion.test.tsnew file mode 100644index 00000000..fbc29342--- /dev/null+++ b/packages/sound-studio/test/companion.test.ts@@ -0,0 +1,47 @@+import assert from "node:assert/strict";+import test from "node:test";+import { sealAudioTrack } from "@hypit/composition";+import { temporalTypes } from "@hypit/temporal";+import { timelineTypes } from "@hypit/timeline";+import type { StudioTrackCompanionContext } from "@hypit/studio-adapter";+import { projectSound } from "../src/index.js";+import { timelineFixture } from "../../../test/timeline-fixture.js";+import { projectProgramWindow } from "../../../test/temporal-fixture.js";++const base = timelineFixture({ id: "film", durationSec: 6, frameRate: { numerator: 30, denominator: 1 } }, {+ segments: [{id:"first",frameCount:90},{id:"second",frameCount:90}],+});+function context(audio: boolean): StudioTrackCompanionContext {+ const timeline = {...base, items: base.items.map((item,index)=>({...item,startFrame:index ? 60:15,take:{...item.take,+ media: audio ? item.take.media : {timeline:item.take.media.timeline,visual:{artifact:{kind:"blob",resource:`res_picture${index}`,mediaType:"video/mp4",size:1},width:64,height:64}},+ }}))};+ const program = {uses:["base","override","hidden"].map(id=>({+ window:projectProgramWindow({itemId:id,semantic:base,projection:{start:{ref:"absolute",at:{unit:"frames",value:0}},end:{ref:"absolute",at:{unit:"frames",value:180}}}}),+ audio:sealAudioTrack({id,programSpaceId:base.id,clips:[]}),+ }))};+ return {+ track:{outputRef:"p.audio",trace:{references:[{input:"timeline",typeRef:timelineTypes.track,ref:"film"}],outputPorts:[{name:"program",ref:"p.program"}]}},+ placement:{children:program.uses.map((use,index)=>({range:{start:index*10,end:index*10+5},referenceAttributes:{style:`styles.${use.window.subjectId}`},values:[{type:temporalTypes.windowSpec,value:{id:use.window.subjectId}}]}))},+ values:new Map<string,unknown>([["film",timeline],["p.program",program]]),spans:[],temporalBindings:[],+ } as unknown as StudioTrackCompanionContext;+}++test("Use band keeps fully overridden and silent rules with exact child and Style ownership",()=>{+ const uses=projectSound(context(false));+ assert.deepEqual(uses.map(use=>[use.display.title,use.band,use.elementRange]),[+ ["styles.base","uses",{start:0,end:5}],+ ["styles.override","uses",{start:10,end:15}],+ ["styles.hidden","uses",{start:20,end:25}],+ ]);+ assert.ok(uses.every(use=>use.presentation?.chrome==="standard"));+ assert.ok(uses.every(use=>use.inspector?.[0]?.value === "0–180"));+});++test("content follows audio-only Takes including gaps and overlaps; picture-only contributes no sound",()=>{+ const content=projectSound(context(true)).filter(entity=>entity.band===undefined);+ assert.deepEqual(content.map(entity=>[entity.authoredId,entity.startFrame,entity.endFrameExclusive]),[+ ["first",15,105],["second",60,150],+ ]);+ assert.deepEqual(content[0]?.inspector?.map(field=>field.value), ["first", "15–105"]);+ assert.equal(projectSound(context(false)).filter(entity=>entity.band===undefined).length,0);+});diff --git a/packages/sound/README.md b/packages/sound/README.mdnew file mode 100644index 00000000..6611aaa1--- /dev/null+++ b/packages/sound/README.md@@ -0,0 +1,81 @@+# Sound++Sound presents audio already placed on a Timeline. Audio Track takes independent material such as+music and effects. Sound obtains existing sources and playback positions from Timeline and chooses+how they are heard. Visual Performance and Caption remain independent presentations.++```svml+<import as="sound" from="@hypit/sound@1"/>+<sound:Style id="normal"/>+<sound:Style id="fade-in" gain="0" end-gain="1"/>+<sound:Style id="silent" gain="0"/>+<sound:Track id="voice" timeline={program.timeline}>+ <sound:Use style={normal}/>+ <sound:Use at="0f" for="2s" style={fade-in}/>+ <sound:Use during={story.selection.demonstration} style={silent}/>+</sound:Track>+```++Include `voice.audio` in Film to hear this presentation. Timeline exports `.timeline`, retaining+the material for consumers; Sound selects and presents its audio explicitly.++Sound also exports `.program`: the ordered Use windows and their audio presentations before+coverage is resolved. The Studio Companion consumes this value to keep silent and covered Uses+available for editing. `.audio` remains the effective arrangement for Film.++## Two separate choices++The last declared matching Use selects the complete presentation locally. No selector means the+whole Timeline. No matching Use contributes no sound; an empty winning Style remains silent.+Later Uses can restore sound inside an earlier silent interval. Separate Sound Tracks can coexist.++Within the ordinary Style, the last Timeline-declared active source with audio wins. Declaration+order is independent of start-time order. A video-only Take supplies no sound candidate; a recorded+silent audio source is still a source. Gaps and zero-Take Timelines have no sound to present. When a+later source ends, an earlier still-active source resumes at its current source position.++`gain` and `end-gain` are linear amplitude multipliers from 0 to 64. `gain` defaults to 1 and+`end-gain` defaults to `gain`. They describe a linear ramp over the original Use Window. Equal values+produce constant gain; zero produces silence. Changing the Window changes the ramp duration.+The interval's right endpoint describes the gain reached at the boundary, while sample coverage is+half-open. Matching adjacent Style endpoints is an authoring choice.++## Time and continuity++Use accepts shared `during`, `at` + `for`, `until` + `for`, and `start` + `end`, including independently+bound `start-source` / `end-source`. A source role and a time endpoint are distinct inputs.++A Use starting at program second 7 does not restart a Take placed at second 5: its sound is already+at source second 2. A partially overridden Use keeps its original gain-ramp progress. Resolution+intersects audible subranges and preserves original Clip targets, sampling and envelopes. The same+holds for a partial audio render. Sound does not move Takes, change words or retime semantic anchors.++## Write a project Style++The public TypeScript entry is `@hypit/hypit/sound`.+`soundStyle(fragment, bindings)` creates an authored Style. Its fragment takes `timeline` and+`window`, plus explicitly bound family inputs, and exports `audio: AudioTrack`. Bound references+become ordinary graph edges at each Use. The fragment may produce an empty Track, one source or+multiple sources; the renderer consumes the resulting ordinary AudioTrack.++`ordinarySound(timeline, window, { gain, endGain })` supplies ordinary last-source presentation.+`sourceSound(timeline, window, segment, { gain, endGain })` selects one explicit Narrative Segment,+retaining its existing source and target positions. A known Segment without audio returns an empty+Track; a foreign or unknown Segment is an error. Neither helper invents sound outside source coverage.+A family can publish richer program-sample `gainEnvelope` points through the common AudioClip data.++The [project crossfade example](../../examples/semantic-composition/packages/sound-styles/README.md)+binds `outgoing` and `incoming` independently. Its two gains move in opposite directions across the+Use Window. There is no transition-mode registry or automatic source pairing. Other families can+own different curves or source combinations through the same fragment interface.++## Rendering boundary++[Composition](../composition/README.md) owns generic AudioClip presentation fields: optional+`gainEnvelope` points and `audibility` subranges, both on the complete 48 kHz program clock. Sound+resolves Uses into those values. The media pipeline preserves them; the execution package applies+them per sample after source playback and existing fades, before render-range cropping.++The [Sound Companion](../sound-studio/README.md) presents placed audio content and authored Uses.+Studio playback consumes the same gain envelopes, audible regions and fades through its generic+AudioTrack player.diff --git a/packages/sound/package.json b/packages/sound/package.jsonnew file mode 100644index 00000000..e8f39c4f--- /dev/null+++ b/packages/sound/package.json@@ -0,0 +1,25 @@+{+ "name": "@hypit/sound",+ "version": "0.0.0-dev",+ "license": "SEE LICENSE IN LICENSE",+ "private": true,+ "type": "module",+ "exports": {+ ".": "./src/index.ts"+ },+ "hypit": {+ "activation": "./src/activation.ts"+ },+ "dependencies": {+ "@hypit/component-kit": "workspace:*",+ "@hypit/composition": "workspace:*",+ "@hypit/elaborator": "workspace:*",+ "@hypit/markup": "workspace:*",+ "@hypit/narrative": "workspace:*",+ "@hypit/program-space": "workspace:*",+ "@hypit/protocol": "workspace:*",+ "@hypit/temporal": "workspace:*",+ "@hypit/temporal-markup": "workspace:*",+ "@hypit/timeline": "workspace:*"+ }+}diff --git a/packages/sound/src/activation.ts b/packages/sound/src/activation.tsnew file mode 100644index 00000000..3c8f57b8--- /dev/null+++ b/packages/sound/src/activation.ts@@ -0,0 +1,11 @@+import { createMarkupSurfaceHostFacet } from "@hypit/markup";+import { soundManifest, soundModuleRef, soundMarkupSurfaces } from "./manifest.js";+import { soundComponent } from "./component.js";+import { decodeSoundStyleSurface, decodeSoundTrackSurface } from "./surface.js";+export const hypitPackage = {+ format: "hypit.node-package@1" as const,+ modules: [{ manifest: soundManifest }], components: [soundComponent],+ hostFacets: soundMarkupSurfaces.map(declaration => createMarkupSurfaceHostFacet({ module: soundModuleRef, declaration,+ handler: declaration.name === "style" ? decodeSoundStyleSurface : decodeSoundTrackSurface })),+};+export default hypitPackage;diff --git a/packages/sound/src/component.ts b/packages/sound/src/component.tsnew file mode 100644index 00000000..88d2ae2d--- /dev/null+++ b/packages/sound/src/component.ts@@ -0,0 +1,28 @@+import { canonicalize } from "@hypit/protocol";+import type { StoredValue } from "@hypit/protocol";+import type { ComponentPackage } from "@hypit/component-kit";+import type { Timeline } from "@hypit/timeline";+import type { AudioTrack } from "@hypit/composition";+import type { TemporalWindow } from "@hypit/temporal";+import { ordinarySound, appendSoundUse, resolveSound } from "./program.js";+import type { SoundSet, SoundGain } from "./program.js";+import { soundProducers } from "./manifest.js";+function inline<T>(value: StoredValue | undefined): T {+ if (value?.kind !== "inline") throw new Error("Sound input must be inline.");+ return value.value as unknown as T;+}+const output = (value: unknown) => ({ kind: "inline" as const, value: canonicalize(value) });+export const soundComponent: ComponentPackage = {+ producers: [+ { producer: soundProducers.create, handler: () => ({ outputs: { set: output({ uses: [] }) }, needs: {} }) },+ { producer: soundProducers.append, handler: ({ inputs }) => ({ outputs: { set: output(appendSoundUse(+ inline<SoundSet>(inputs.set?.value), inline<TemporalWindow>(inputs.window?.value), inline<AudioTrack>(inputs.audio?.value),+ )) }, needs: {} }) },+ { producer: soundProducers.resolve, handler: ({ inputs }) => ({ outputs: { audio: output(resolveSound(+ inline<Timeline>(inputs.timeline?.value), inline<{ id: string }>(inputs.header?.value).id, inline<SoundSet>(inputs.set?.value),+ )) }, needs: {} }) },+ { producer: soundProducers.ordinary, handler: ({ inputs }) => ({ outputs: { audio: output(ordinarySound(+ inline<Timeline>(inputs.timeline?.value), inline<TemporalWindow>(inputs.window?.value), inline<SoundGain>(inputs.spec?.value),+ )) }, needs: {} }) },+ ],+};diff --git a/packages/sound/src/index.ts b/packages/sound/src/index.tsnew file mode 100644index 00000000..cf2c580a--- /dev/null+++ b/packages/sound/src/index.ts@@ -0,0 +1,5 @@+export * from "./style.js";+export * from "./program.js";+export * from "./component.js";+export * from "./surface.js";+export * from "./manifest.js";diff --git a/packages/sound/src/manifest.ts b/packages/sound/src/manifest.tsnew file mode 100644index 00000000..5bc01760--- /dev/null+++ b/packages/sound/src/manifest.ts@@ -0,0 +1,47 @@+import { compositionDependency, compositionTypes } from "@hypit/composition";+import { timelineDependency, timelineTypes } from "@hypit/timeline";+import { temporalDependency, temporalTypes } from "@hypit/temporal";+import { temporalWindowAttributeVocabulary } from "@hypit/temporal-markup";+import type { ModuleManifest, ProducerRef, TypeRef } from "@hypit/protocol";++export const soundModuleRef = { name: "@hypit/sound", version: "1" } as const;+export const soundTypes = Object.fromEntries(["Style", "Set", "Header", "Gain"].map(name => [name.toLowerCase(), { module: soundModuleRef, name: `Sound${name}` }])) as Record<"style" | "set" | "header" | "gain", TypeRef>;+export const soundProducers = Object.fromEntries(["create", "append", "resolve", "ordinary"].map(name => [name, { module: soundModuleRef, name }])) as Record<"create" | "append" | "resolve" | "ordinary", ProducerRef>;+export const ordinaryInputs = [+ { name: "timeline", type: timelineTypes.track }, { name: "window", type: temporalTypes.window },+ { name: "spec", type: soundTypes.gain },+];+export const soundManifest: ModuleManifest = {+ format: "hypit.module@1", ...soundModuleRef,+ dependencies: [compositionDependency, timelineDependency, temporalDependency],+ types: Object.values(soundTypes).map(type => ({ name: type.name })), capabilities: [],+ producers: [+ { name: "create", inputs: [], outputs: [{ name: "set", type: soundTypes.set }], needs: [] },+ { name: "append", inputs: [{ name: "set", type: soundTypes.set }, { name: "window", type: temporalTypes.window }, { name: "audio", type: compositionTypes.audioTrack }], outputs: [{ name: "set", type: soundTypes.set }], needs: [] },+ { name: "resolve", inputs: [{ name: "set", type: soundTypes.set }, { name: "timeline", type: timelineTypes.track }, { name: "header", type: soundTypes.header }], outputs: [{ name: "audio", type: compositionTypes.audioTrack }], needs: [] },+ { name: "ordinary", inputs: ordinaryInputs, outputs: [{ name: "audio", type: compositionTypes.audioTrack }], needs: [] },+ ],+};+export const soundMarkupSurfaces = [+ { name: "style", tag: "Style", mode: "structured", outputs: [soundTypes.style, soundTypes.gain], vocabulary: {+ summary: "Presents existing Timeline sound, choosing the last declared active audio source. Gain may change linearly across each Use.",+ example: `<sound:Style id="fade-in" gain="0" end-gain="1"/>`,+ attributes: [+ { name: "id", kind: "identifier", required: true, summary: "Style name." },+ { name: "gain", kind: "literal", required: false, summary: "Linear gain at the beginning of a Use; defaults to 1. Zero silences the presentation." },+ { name: "end-gain", kind: "literal", required: false, summary: "Linear gain at the end of a Use; defaults to gain. The original Use interval owns the ramp." },+ ],+ } },+ { name: "track", tag: "Track", mode: "structured", outputs: [compositionTypes.audioTrack, soundTypes.set, soundTypes.header, temporalTypes.instantSpec, temporalTypes.windowSpec, temporalTypes.instant, temporalTypes.window], vocabulary: {+ summary: "Presents Timeline sound through ordered Uses. Later Uses replace earlier presentation locally without restarting source playback or gain curves.",+ example: `<sound:Track id="voice" timeline={program.timeline}><sound:Use style={normal}/><sound:Use at="0f" for="2s" style={fade-in}/></sound:Track>`,+ attributes: [+ { name: "id", kind: "identifier", required: true, summary: "Sound presentation name." },+ { name: "timeline", kind: "reference", required: true, accepts: [timelineTypes.track], summary: "Complete film time and placed sources." },+ ], children: [{ tag: "Use", cardinality: "many", summary: "Last matching Use wins, including silence. No time selector covers the complete Timeline.", attributes: [+ { name: "id", kind: "identifier", required: false, summary: "Optional occurrence identity." },+ { name: "style", kind: "reference", required: true, accepts: [soundTypes.style], summary: "Ordinary or project-authored sound Style. Explicit source bindings belong to the Style." },+ ...temporalWindowAttributeVocabulary,+ ] }],+ } },+] as const;diff --git a/packages/sound/src/program.ts b/packages/sound/src/program.tsnew file mode 100644index 00000000..8195b3aa--- /dev/null+++ b/packages/sound/src/program.ts@@ -0,0 +1,84 @@+import { sealAudioTrack, assertAudioTrackIdentity } from "@hypit/composition";+import type { AudioTrack, AudioClip, AudioSampleSpan, AudioGainPoint } from "@hypit/composition";+import { assertTemporalWindowFor } from "@hypit/temporal";+import type { TemporalWindow } from "@hypit/temporal";+import { projectTimelineAudio } from "@hypit/timeline";+import type { Timeline } from "@hypit/timeline";+import type { NarrativeExcerpt } from "@hypit/narrative";+import { programFrameSampleBoundary } from "@hypit/program-space";++export type SoundUse = { readonly window: TemporalWindow; readonly audio: AudioTrack };+export type SoundSet = { readonly uses: readonly SoundUse[] };+export type SoundGain = { readonly gain: number; readonly endGain: number };++export function soundWindow(timeline: Timeline, window: TemporalWindow): AudioSampleSpan {+ assertTemporalWindowFor(window, { subjectId: window.subjectId, space: timeline });+ return { startSample: programFrameSampleBoundary(timeline, window.span.startFrame, 48000),+ endSampleExclusive: programFrameSampleBoundary(timeline, window.span.endFrameExclusive, 48000) };+}+export function intersectAudioSpans(left: readonly AudioSampleSpan[], right: readonly AudioSampleSpan[]): AudioSampleSpan[] {+ return left.flatMap(a => right.flatMap(b => {+ const startSample = Math.max(a.startSample, b.startSample), endSampleExclusive = Math.min(a.endSampleExclusive, b.endSampleExclusive);+ return endSampleExclusive > startSample ? [{ startSample, endSampleExclusive }] : [];+ }));+}+function subtract(spans: readonly AudioSampleSpan[], cut: AudioSampleSpan): AudioSampleSpan[] {+ return spans.flatMap(span => {+ if (cut.endSampleExclusive <= span.startSample || cut.startSample >= span.endSampleExclusive) return [span];+ return [+ ...(cut.startSample > span.startSample ? [{ startSample: span.startSample, endSampleExclusive: cut.startSample }] : []),+ ...(cut.endSampleExclusive < span.endSampleExclusive ? [{ startSample: cut.endSampleExclusive, endSampleExclusive: span.endSampleExclusive }] : []),+ ];+ });+}+function envelope(span: AudioSampleSpan, gain: SoundGain): readonly AudioGainPoint[] {+ if (![gain.gain, gain.endGain].every(value => Number.isFinite(value) && value >= 0 && value <= 64)) throw new Error("Sound gain must be from 0 to 64.");+ return [{ sample: span.startSample, gain: gain.gain }, { sample: span.endSampleExclusive, gain: gain.endGain }];+}+const unity = { gain: 1, endGain: 1 };++/** Default source choice is declaration order, independently of the order of Use rules. */+export function ordinarySound(timeline: Timeline, window: TemporalWindow, gain: SoundGain = unity): AudioTrack {+ const span = soundWindow(timeline, window), gainEnvelope = envelope(span, gain);+ const sources = new Map(projectTimelineAudio(timeline).clips.map(clip => [clip.id, clip]));+ const clips = timeline.items.flatMap(item => { const clip = sources.get(item.take.segment.segmentId); return clip ? [clip] : []; });+ const result: AudioClip[] = [];+ for (const [index, clip] of clips.entries()) {+ let audibility = intersectAudioSpans([span], [clip.target]);+ for (const later of clips.slice(index + 1)) audibility = subtract(audibility, later.target);+ if (audibility.length && (gain.gain !== 0 || gain.endGain !== 0)) result.push({ ...clip, gainEnvelope, audibility });+ }+ return sealAudioTrack({ id: window.subjectId, programSpaceId: timeline.id, clips: result });+}++/** Explicit source selection for project Styles, with its original placement/sampling retained. */+export function sourceSound(timeline: Timeline, window: TemporalWindow, segment: NarrativeExcerpt, gain: SoundGain = unity): AudioTrack {+ if (segment.kind !== "segment" || segment.narrativeId !== timeline.narrativeId) throw new Error("Sound source must be a Segment of this Timeline's Narrative.");+ if (!timeline.items.some(item => item.take.segment.segmentId === segment.id)) throw new Error(`Unknown Sound Segment ${segment.id}.`);+ const span = soundWindow(timeline, window), gainEnvelope = envelope(span, gain);+ const clip = projectTimelineAudio(timeline).clips.find(clip => clip.id === segment.id);+ const audibility = clip ? intersectAudioSpans([span], [clip.target]) : [];+ return sealAudioTrack({ id: `${window.subjectId}:${segment.id}`, programSpaceId: timeline.id,+ clips: clip && audibility.length && (gain.gain !== 0 || gain.endGain !== 0) ? [{ ...clip, gainEnvelope, audibility }] : [] });+}+export function appendSoundUse(set: SoundSet, window: TemporalWindow, audio: AudioTrack): SoundSet {+ assertAudioTrackIdentity(audio);+ if (audio.programSpaceId !== window.start.source.spaceId) throw new Error("Sound Use and audio belong to different Timelines.");+ return { uses: [...set.uses, { window, audio }] };+}+/** Resolve presentation coverage without changing original targets, source clocks or envelopes. */+export function resolveSound(timeline: Timeline, id: string, set: SoundSet): AudioTrack {+ const windows = set.uses.map(use => { assertAudioTrackIdentity(use.audio, timeline); return soundWindow(timeline, use.window); });+ const clips: AudioClip[] = [];+ for (const [index, use] of set.uses.entries()) {+ let active = [windows[index]!];+ for (const cut of windows.slice(index + 1)) active = subtract(active, cut);+ for (const clip of use.audio.clips) {+ const audibility = intersectAudioSpans(active, clip.audibility ?? [clip.target]);+ if (audibility.length) clips.push({ ...clip, id: `${index + 1}:${clip.id}`, audibility });+ }+ }+ const result = sealAudioTrack({ id, programSpaceId: timeline.id, clips });+ assertAudioTrackIdentity(result, timeline);+ return result;+}diff --git a/packages/sound/src/style.ts b/packages/sound/src/style.tsnew file mode 100644index 00000000..ea14991c--- /dev/null+++ b/packages/sound/src/style.ts@@ -0,0 +1,34 @@+import { sealGraphFragment } from "@hypit/elaborator";+import type { GraphFragment } from "@hypit/elaborator";+import { compositionTypes } from "@hypit/composition";+import { timelineTypes } from "@hypit/timeline";+import { temporalTypes } from "@hypit/temporal";+import { sameType } from "@hypit/protocol";+import type { SurfaceResolvedReference } from "@hypit/markup";++/** Author-time closure: bound references become ordinary graph edges at each Use. */+export type SoundStyle = {+ readonly fragment: GraphFragment;+ readonly bindings: Readonly<Record<string, { readonly type: SurfaceResolvedReference["type"]; readonly ref: SurfaceResolvedReference["ref"] }>>;+};+export function soundStyle(fragment: GraphFragment, bindings: SoundStyle["bindings"] = {}): SoundStyle {+ fragment = sealGraphFragment(fragment);+ const common = { timeline: timelineTypes.track, window: temporalTypes.window };+ for (const [name, type] of Object.entries(common)) {+ const port = fragment.inputs.find(port => port.name === name);+ if (port === undefined || !sameType(port.type, type)) throw new Error(`Sound Style requires ${name} input.`);+ if (bindings[name] !== undefined) throw new Error(`Sound supplies ${name} at each Use.`);+ }+ for (const port of fragment.inputs) {+ if (port.name in common) continue;+ const binding = bindings[port.name];+ if (binding === undefined || !sameType(port.type, binding.type)) throw new Error(`Sound Style binding ${port.name} has the wrong Type.`);+ }+ for (const name of Object.keys(bindings)) {+ if (!fragment.inputs.some(port => port.name === name)) throw new Error(`Sound Style has unknown binding ${name}.`);+ }+ if (fragment.exports.length !== 1 || fragment.exports[0]?.name !== "audio" || !sameType(fragment.exports[0].type, compositionTypes.audioTrack)) {+ throw new Error("Sound Style must export audio: AudioTrack.");+ }+ return { fragment, bindings: Object.fromEntries(Object.entries(bindings).map(([name, value]) => [name, { ref: value.ref, type: value.type }])) };+}diff --git a/packages/sound/src/surface.ts b/packages/sound/src/surface.tsnew file mode 100644index 00000000..e09d4134--- /dev/null+++ b/packages/sound/src/surface.ts@@ -0,0 +1,90 @@+import { assertAttributes, assertEmptyElement, textAttribute, optionalTextAttribute } from "@hypit/markup";+import type { StructuredSurfaceHandler, SurfaceResolvedReference, SurfaceRecordDraft, SurfaceComponentDraft, StructuredElement } from "@hypit/markup";+import { canonicalize, sameType } from "@hypit/protocol";+import type { TypeRef } from "@hypit/protocol";+import { sealGraphFragment } from "@hypit/elaborator";+import type { FragmentOperation } from "@hypit/elaborator";+import { createTemporalWindowProjection, resolveTemporalContext, temporalWindowAttributeNames } from "@hypit/temporal-markup";+import { temporalTypes } from "@hypit/temporal";+import { compositionTypes } from "@hypit/composition";+import { timelineTypes } from "@hypit/timeline";+import { soundStyle } from "./style.js";+import type { SoundStyle } from "./style.js";+import { soundTypes, soundProducers, ordinaryInputs } from "./manifest.js";++const input = (name: string) => ({ kind: "fragment-input" as const, name });+const operation = (id: string) => ({ kind: "fragment-operation" as const, operation: id });+function reference(element: StructuredElement, name: string, type: TypeRef, resolve: (path: string) => SurfaceResolvedReference | undefined): SurfaceResolvedReference {+ const raw = element.attributes[name];+ if (typeof raw !== "object" || raw.kind !== "reference") throw new Error(`${element.name}.${name} must be a reference.`);+ const found = resolve(raw.path);+ if (found === undefined || !sameType(found.type, type)) throw new Error(`${element.name}.${name} has the wrong Type.`);+ return found;+}+function authored<T>(value: SurfaceResolvedReference): T {+ if (value.record?.value.kind !== "inline") throw new Error("A presentation Style or Recipe must be an authored inline value.");+ return value.record.value.value as unknown as T;+}+export const decodeSoundTrackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {+ assertAttributes(element, ["id", "timeline"]);+ const id = textAttribute(element, "id"), context = resolveTemporalContext({ element, resolveReference });+ const records: SurfaceRecordDraft[] = [{ id: `${id}.header`, type: soundTypes.header, value: { kind: "inline", value: { id } }, range: element.range }];+ const components: SurfaceComponentDraft[] = [];+ const fragments = [] as ReturnType<typeof sealGraphFragment>[];+ const collectorInputs = [{ name: "timeline", type: timelineTypes.track }, { name: "header", type: soundTypes.header }];+ const bindings: Record<string, SurfaceResolvedReference["ref"]> = { timeline: context.timeline.ref, header: { kind: "record", id: `${id}.header` } };+ const operations: FragmentOperation[] = [{ id: "empty", producer: soundProducers.create, inputs: {}, result: { kind: "output", name: "set" } }];+ let previous = "empty", index = 0;+ for (const child of element.children) {+ if (child.kind === "text") { if (child.value.trim()) throw new Error("Sound Track accepts Use children."); continue; }+ if (child.name.split(":").at(-1) !== "Use") throw new Error("Sound Track accepts Use children.");+ assertAttributes(child, ["id", "style", ...temporalWindowAttributeNames]); assertEmptyElement(child);+ const useId = optionalTextAttribute(child, "id") ?? `${id}.use.${index + 1}`;+ index += 1;+ const hasTime = temporalWindowAttributeNames.some(name => child.attributes[name] !== undefined);+ const temporal = createTemporalWindowProjection({ id: useId, ...context, resolveReference,+ element: hasTime ? child : { ...child, attributes: { ...child.attributes, during: "program" } } });+ records.push(...temporal.records); components.push(...temporal.components); fragments.push(...temporal.fragments);+ const value = authored<SoundStyle>(reference(child, "style", soundTypes.style, resolveReference));+ const style = soundStyle(value.fragment, value.bindings);+ fragments.push(style.fragment);+ const audioId = `${useId}.audio`;+ components.push({ id: `${useId}.__style`, fragment: style.fragment.id,+ inputs: { ...Object.fromEntries(Object.entries(style.bindings).map(([key, bound]) => [key, bound.ref])),+ timeline: context.timeline.ref, window: temporal.ref }, outputs: { audio: audioId }, range: child.range });+ const suffix = `use-${operations.length}`;+ collectorInputs.push({ name: `${suffix}-window`, type: temporalTypes.window }, { name: `${suffix}-audio`, type: compositionTypes.audioTrack });+ bindings[`${suffix}-window`] = temporal.ref; bindings[`${suffix}-audio`] = { kind: "component-output", component: `${useId}.__style`, output: "audio" };+ operations.push({ id: suffix, producer: soundProducers.append, inputs: { set: operation(previous), window: input(`${suffix}-window`), audio: input(`${suffix}-audio`) }, result: { kind: "output", name: "set" } });+ previous = suffix;+ }+ operations.push({ id: "resolve", producer: soundProducers.resolve, inputs: { set: operation(previous), timeline: input("timeline"), header: input("header") }, result: { kind: "output", name: "audio" } });+ const collector = sealGraphFragment({ inputs: collectorInputs, operations, exports: [{ name: "audio", type: compositionTypes.audioTrack, root: operation("resolve") }, { name: "program", type: soundTypes.set, root: operation(previous) }] });+ fragments.push(collector);+ components.push({ id, fragment: collector.id, inputs: bindings, outputs: { audio: `${id}.audio`, program: `${id}.program` }, range: element.range });+ return { records, components, fragments, exports: [`${id}.audio`, `${id}.program`] };+};++export const ordinarySoundFragment = sealGraphFragment({+ inputs: ordinaryInputs,+ operations: [{ id: "ordinary", producer: soundProducers.ordinary,+ inputs: Object.fromEntries(ordinaryInputs.map(port => [port.name, input(port.name)])), result: { kind: "output", name: "audio" } }],+ exports: [{ name: "audio", type: compositionTypes.audioTrack, root: operation("ordinary") }],+});+export const decodeSoundStyleSurface: StructuredSurfaceHandler = ({ element }) => {+ assertAttributes(element, ["id", "gain", "end-gain"]); assertEmptyElement(element);+ const id = textAttribute(element, "id");+ const number = (name: string, fallback: number): number => {+ const raw = optionalTextAttribute(element, name);+ const value = raw === undefined ? fallback : raw.trim() === "" ? NaN : Number(raw);+ if (!Number.isFinite(value) || value < 0 || value > 64) throw new Error(`Sound Style ${name} must be a gain from 0 to 64.`);+ return value;+ };+ const gain = number("gain", 1), endGain = number("end-gain", gain);+ const specId = `${id}.gain`;+ const style = soundStyle(ordinarySoundFragment, { spec: { ref: { kind: "record", id: specId }, type: soundTypes.gain } });+ return { records: [+ { id: specId, type: soundTypes.gain, value: { kind: "inline", value: { gain, endGain } }, range: element.range },+ { id, type: soundTypes.style, value: { kind: "inline", value: canonicalize(style) }, range: element.range },+ ], components: [], fragments: [] };+};diff --git a/packages/sound/test/sound.test.ts b/packages/sound/test/sound.test.tsnew file mode 100644index 00000000..807d2887--- /dev/null+++ b/packages/sound/test/sound.test.ts@@ -0,0 +1,94 @@+import assert from "node:assert/strict";+import test from "node:test";+import { assertAudioTrackIdentity, sealAudioTrack, audioEnvelopeGainAt, sealComposition } from "@hypit/composition";+import { compileAudioProgramPlan, verifyAudioProgramPlan } from "@hypit/media-pipeline";+import { ordinarySound, sourceSound, appendSoundUse, resolveSound, soundStyle, soundTypes, decodeSoundTrackSurface } from "../src/index.js";+import { timelineFixture } from "../../../test/timeline-fixture.js";+import { projectProgramWindow } from "../../../test/temporal-fixture.js";+import type { Timeline } from "@hypit/timeline";+import { timelineTypes } from "@hypit/timeline";+import type { NarrativeExcerpt } from "@hypit/narrative";+import { narrativeTypes } from "@hypit/narrative";+import type { StructuredElement, SurfaceResolvedReference } from "@hypit/markup";+import { canonicalize } from "@hypit/protocol";+import { sealGraphFragment } from "@hypit/elaborator";+import { compositionTypes } from "@hypit/composition";+import { temporalTypes } from "@hypit/temporal";+import { crossfadeSound } from "../../../examples/semantic-composition/packages/sound-styles/src/render.js";++const clock = { id: "film", durationSec: 6, frameRate: { numerator: 30, denominator: 1 } };+const base = timelineFixture(clock, { segments: [{ id: "a", frameCount: 90 }, { id: "b", frameCount: 90 }] });+const timeline: Timeline = { ...base, items: base.items.map((item,index)=>({...item,startFrame:index===0 ? 15:60})) };+const window = (id: string, start: number, end: number, source=timeline) => projectProgramWindow({ itemId: id, semantic: source,+ projection: { start: { ref: "absolute", at: { unit: "frames", value: start } }, end: { ref: "absolute", at: { unit: "frames", value: end } } } });+const segment = (id:string):NarrativeExcerpt => ({kind:"segment",narrativeId:base.narrativeId,id,tokenStart:0,tokenEndExclusive:0});+const at = (frame:number)=>frame*1600;++test("ordinary sound chooses the last declared active audio source, with gaps and absence intact",()=>{+ const sound=ordinarySound(timeline,window("base",0,180));+ assertAudioTrackIdentity(sound,timeline);+ assert.deepEqual(sound.clips.map(c=>[c.id,c.audibility]),[+ ["a",[{startSample:at(15),endSampleExclusive:at(60)}]],+ ["b",[{startSample:at(60),endSampleExclusive:at(150)}]],+ ]);+ assert.deepEqual(sound.clips.map(c=>c.source.startSample),[0,0]);+ const reverse={...timeline,items:[timeline.items[1]!,timeline.items[0]!]};+ const reversed=ordinarySound(reverse,window("base",0,180,reverse));+ assert.deepEqual(reversed.clips.find(c=>c.id==="b")!.audibility,[{startSample:at(105),endSampleExclusive:at(150)}]);+ const noAudio={...timeline,items:timeline.items.map((item,index)=> index===0?item:{...item,take:{...item.take,media:{timeline:item.take.media.timeline,visual:{artifact:{kind:"blob" as const,resource:"res_visual" as const,size:1,mediaType:"video/mp4"},width:64,height:64}}}})};+ assert.deepEqual(ordinarySound(noAudio,window("base",0,180,noAudio)).clips[0]!.audibility,[{startSample:at(15),endSampleExclusive:at(105)}]);+ const empty={...clock,items:[]};+ assert.equal(ordinarySound(empty,window("empty",0,180,empty)).clips.length,0);+});++test("Use silence and later restoration preserve original playback and fade progress",()=>{+ const broad=window("broad",0,180), fade=window("fade",15,150), hidden=window("hidden",45,75), restore=window("restore",60,65);+ let set=appendSoundUse({uses:[]},broad,ordinarySound(timeline,broad));+ set=appendSoundUse(set,fade,ordinarySound(timeline,fade,{gain:0,endGain:1}));+ set=appendSoundUse(set,hidden,ordinarySound(timeline,hidden,{gain:0,endGain:0}));+ set=appendSoundUse(set,restore,ordinarySound(timeline,restore));+ const result=resolveSound(timeline,"voice",set);+ const resumed=result.clips.find(c=>c.id==="2:b")!;+ assert.deepEqual(resumed.target,{startSample:at(60),endSampleExclusive:at(150)});+ assert.equal(resumed.source.startSample,0);+ assert.deepEqual(resumed.audibility,[{startSample:at(75),endSampleExclusive:at(150)}]);+ assert.equal(audioEnvelopeGainAt(resumed.gainEnvelope,at(75)),60/135);+ assert.deepEqual(result.clips.find(c=>c.id==="4:b")!.audibility,[{startSample:at(60),endSampleExclusive:at(65)}]);+ const plan=compileAudioProgramPlan(sealComposition({id:"film",canvas:{width:64,height:64,clearColor:"#000000"},tracks:[result]}),timeline);+ verifyAudioProgramPlan(plan);+ assert.deepEqual(plan.clips.find(c=>c.id==="voice:2:b")!.gainEnvelope,resumed.gainEnvelope);+ assert.deepEqual(plan.clips.find(c=>c.id==="voice:2:b")!.audibility,resumed.audibility);+ assert.equal(resolveSound(timeline,"silent",{uses:[]}).clips.length,0);+});++test("a project crossfade binds source roles independently from its physical Window",()=>{+ const span=window("mix",60,105);+ const mixed=crossfadeSound(timeline,span,segment("a"),segment("b"));+ assertAudioTrackIdentity(mixed,timeline);+ assert.deepEqual(mixed.clips.map(c=>c.id),["a","b"]);+ for(const c of mixed.clips) assert.equal(audioEnvelopeGainAt(c.gainEnvelope,at(82.5)),0.5);+ assert.equal(mixed.clips.find(c=>c.id==="a")!.target.startSample,at(15));+ const swapped=crossfadeSound(timeline,span,segment("b"),segment("a"));+ assert.equal(audioEnvelopeGainAt(swapped.clips.find(c=>c.id==="a")!.gainEnvelope,at(60)),0);+ const missingSound={...timeline,items:timeline.items.map((item,index)=>index===0?item:{...item,take:{...item.take,media:{timeline:item.take.media.timeline,visual:{artifact:{kind:"blob" as const,resource:"res_visual" as const,size:1,mediaType:"video/mp4"},width:64,height:64}}}})};+ assert.equal(crossfadeSound(missingSound,window("mix",60,105,missingSound),segment("a"),segment("b")).clips.length,1);+ assert.throws(()=>sourceSound(timeline,span,segment("unknown")),/Unknown/);+ assert.throws(()=>sourceSound(timeline,span,{...segment("a"),narrativeId:"other"}),/Narrative/);+ assert.throws(()=>crossfadeSound(timeline,span,segment("a"),segment("a")),/distinct/);+});++test("Sound Style bindings expand as graph edges and broad Use needs no selector",async()=>{+ const common=[{name:"timeline",type:timelineTypes.track},{name:"window",type:temporalTypes.window}];+ const extra=[{name:"outgoing",type:narrativeTypes.excerpt},{name:"incoming",type:narrativeTypes.excerpt}];+ const fragment=sealGraphFragment({inputs:[...common,...extra],operations:[{id:"render",producer:{module:{name:"@example/test",version:"1"},name:"mix"},inputs:Object.fromEntries([...common,...extra].map(p=>[p.name,{kind:"fragment-input" as const,name:p.name}])),result:{kind:"output",name:"audio"}}],exports:[{name:"audio",type:compositionTypes.audioTrack,root:{kind:"fragment-operation",operation:"render"}}]});+ const ref=(id:string,type:SurfaceResolvedReference["type"]):SurfaceResolvedReference=>({path:id,type,ref:{kind:"record",id}});+ const bindings={outgoing:ref("outgoing",narrativeTypes.excerpt),incoming:ref("incoming",narrativeTypes.excerpt)};+ const style=soundStyle(fragment,bindings);+ const refs=new Map<string,SurfaceResolvedReference>([["timeline",ref("timeline",timelineTypes.track)],["style",{...ref("style",soundTypes.style),record:{id:"style",type:soundTypes.style,value:{kind:"inline",value:canonicalize(style)}}}]]);+ const range={source:"test.svml",start:0,end:1};+ const root:StructuredElement={kind:"element",name:"sound:Track",attributes:{id:"voice",timeline:{kind:"reference",path:"timeline"}},range,children:[{kind:"element",name:"sound:Use",attributes:{style:{kind:"reference",path:"style"}},children:[],range}]};+ const result=await decodeSoundTrackSurface({element:root,sourceName:"test.svml",resolveReference:path=>refs.get(path),resolveAsset:async()=>{throw new Error("unexpected");}});+ assert.deepEqual(result.components.find(c=>c.id==="voice.use.1.__style")!.inputs.outgoing,bindings.outgoing.ref);+ assert.deepEqual(result.components.at(-1)!.outputs,{audio:"voice.audio",program:"voice.program"});+ assert.throws(()=>soundStyle(fragment,{...bindings,window:ref("window",temporalTypes.window)}),/supplies window/);+});diff --git a/packages/speech-alignment/src/local.ts b/packages/speech-alignment/src/local.tsindex f88ea689..7bcce6b6 100644--- a/packages/speech-alignment/src/local.ts+++ b/packages/speech-alignment/src/local.ts@@ -16,8 +16,16 @@ function segmentNarrative(narrative: Narrative, excerpt: NarrativeExcerpt): Narrthrow new Error(`NarrativeExcerpt ${excerpt.id} does not describe its authored Segment.`);}const tokens = narrative.tokens.slice(segment.tokenStart, segment.tokenEndExclusive);+ const units = narrative.caption.units.filter((unit) => unit.segmentId === segment.id);+ const unitIds = new Set(units.map((unit) => unit.id));return {id: narrative.id,+ caption: {+ ...narrative.caption,+ units,+ words: narrative.caption.words.filter((word) => unitIds.has(word.unitId)),+ cueBreaks: narrative.caption.cueBreaks.filter((cue) => unitIds.has(cue.afterUnitId)),+ },segments: [{ ...segment, tokenStart: 0, tokenEndExclusive: tokens.length }],tokens,turns: narrative.turnsdiff --git a/packages/speech-alignment/src/locate.ts b/packages/speech-alignment/src/locate.tsindex da7a546b..dea234d0 100644--- a/packages/speech-alignment/src/locate.ts+++ b/packages/speech-alignment/src/locate.ts@@ -456,7 +456,7 @@ export function locateAlignedSegmentTiming(const tokensById = new Map(timedTokens.map((token) => [token.tokenId, token]));const segmentsById = new Map(timedSegments.map((segment) => [segment.segmentId, segment]));const anchors: LocalSemanticTimePoint[] = narrative.semanticIndex.anchors.flatMap((anchor): LocalSemanticTimePoint[] => {- // Program boundaries belong to the assembled SemanticTrack, never to one Segment-local Take.+ // Program boundaries belong to the assembled Timeline, never to one Segment-local Take.if (anchor.kind === "program-start" || anchor.kind === "program-end") return [];if (anchor.kind === "segment-start" || anchor.kind === "segment-end") {const segment = segmentsById.get(anchor.segmentId)!;diff --git a/packages/speech-track-studio/README.md b/packages/speech-track-studio/README.mddeleted file mode 100644index efe7227b..00000000--- a/packages/speech-track-studio/README.md+++ /dev/null@@ -1,3 +0,0 @@-# `@hypit/speech-track-studio`--Hypit Studio Companion for the original audio projection of `@hypit/speech-track`.diff --git a/packages/speech-track-studio/package.json b/packages/speech-track-studio/package.jsondeleted file mode 100644index 0ddfbe0b..00000000--- a/packages/speech-track-studio/package.json+++ /dev/null@@ -1,14 +0,0 @@-{- "name": "@hypit/speech-track-studio",- "version": "0.0.0-dev",- "license": "SEE LICENSE IN LICENSE",- "private": true,- "type": "module",- "exports": { ".": "./src/index.ts" },- "hypit": { "activation": "./src/activation.ts" },- "dependencies": {- "@hypit/speech-track": "workspace:*",- "@hypit/composition": "workspace:*",- "@hypit/studio-adapter": "workspace:*"- }-}diff --git a/packages/speech-track-studio/src/activation.ts b/packages/speech-track-studio/src/activation.tsdeleted file mode 100644index 065b7e4f..00000000--- a/packages/speech-track-studio/src/activation.ts+++ /dev/null@@ -1,7 +0,0 @@-import { createStudioTrackCompanionHostFacet } from "@hypit/studio-adapter";-import { speechTrackStudioTrackCompanions } from "./index.js";--export default {- format: "hypit.node-package@1" as const,- hostFacets: [createStudioTrackCompanionHostFacet(speechTrackStudioTrackCompanions)],-};diff --git a/packages/speech-track-studio/src/index.ts b/packages/speech-track-studio/src/index.tsdeleted file mode 100644index 67776ca7..00000000--- a/packages/speech-track-studio/src/index.ts+++ /dev/null@@ -1,40 +0,0 @@-import { speechTrackModuleRef } from "@hypit/speech-track";-import { compositionTypes } from "@hypit/composition";-import type { StudioTrackCompanion, StudioTrackCompanionContext, StudioEntityDraft } from "@hypit/studio-adapter";-import { artifactPreview, previewLayer } from "@hypit/studio-adapter";-type SpeechAudioTrack = {- readonly clips?: readonly {- readonly id: string;- readonly subjectId?: string;- readonly artifact?: { readonly resource?: string };- }[];-};--function projectSpeechAudio(context: StudioTrackCompanionContext): readonly StudioEntityDraft[] {- const clips = new Map(((context.track.value as SpeechAudioTrack).clips ?? [])- .map((clip) => [clip.id, clip] as const));- return context.generic().map((entity) => {- const id = entity.presentId ?? entity.authoredId;- const clip = clips.get(id);- const subjectId = clip?.subjectId;- return {- ...entity,- authoredId: subjectId ?? id,- display: {- title: subjectId ?? entity.authoredId,- layers: clip?.artifact?.resource === undefined ? [] : [previewLayer(artifactPreview("audio", clip.artifact.resource), "waveform")],- },- presentation: { entity: "audio-clip", chrome: "standard" },- };- });-}--export const speechTrackStudioTrackCompanions: readonly StudioTrackCompanion[] = [- {- id: "audio", role: "track",- output: { type: compositionTypes.audioTrack, surface: "track", modules: [speechTrackModuleRef] },- family: "speech-audio", tone: "green", label: "Speech Audio", icon: "waveform",- bindings: [], inspector: [], project: projectSpeechAudio,- lane: { heightPx: 48 },- },-];diff --git a/packages/speech-track/README.md b/packages/speech-track/README.mddeleted file mode 100644index a7462ff4..00000000--- a/packages/speech-track/README.md+++ /dev/null@@ -1,51 +0,0 @@-# `@hypit/speech-track`--Assemble ordered SemanticTakes into a semantic timeline and its original sound. The performance-carrying the Script is the A-roll, whether its picture fills the screen, appears in an inset,-is a transparent cutout, or is covered by other pictures. This role establishes time; the visual-composition decides where and how that material appears.--```svml-<speech:Track id="speech">- <speech:Take source={opening.take}/>- <speech:Take source={answer.take}/>-</speech:Track>-```--The Takes are already normalized and semantic. Source order and actual prepared lengths establish-the shared ProgramSpace. Local word positions and Segment boundaries become global positions.-One Take may contain several speakers or camera cuts; a wordless Take supplies its media boundaries.--| Output | Use |-| --- | --- |-| `.semantic` | Semantic time and prepared materials for Film, Caption, Media and project components |-| `.audio` | Original performance sound at its assembled positions |--Select `.audio` in Film when that sound belongs in the result. This explicit convenience projection-keeps the prepared source at its original speed and gain; a Take without audio contributes no clip.-For independent music, effects or authored mixing choices, use [Audio Track](../audio-track/README.md).--## Present the performance--[Media Track](../media-track/README.md#display-a-semantic-performance) can consume the assembled-performance through a Media Performance:--```svml-<media:Track id="performance" semantic={speech.semantic} canvas={canvas}>- <media:Performance during="program" frame={layout.full}- appearance={look.performance}/>-</media:Track>-```--Include `performance.visual` and `speech.audio` in Film. The Media Recipe supplies `stack-order`,-fit, clipping, rounded corners, border and other presentation choices. A selected window that starts-inside a Take begins at the matching source frame. Presentation continues across Take boundaries.--A project component can also consume `.semantic` and use `projectSemanticMedia` from-`@hypit/hypit/semantic-track` to obtain selected material spans and their source positions. When video and-graphics share layout or motion, that component can own both in one visual program. Independent-Caption or other overlays can remain separate Tracks.--[Media Pipeline](../media-pipeline/README.md) owns preparation;-[Semantic Track](../semantic-track/README.md) owns the timeline and its projections.-The [Studio Companion](../speech-track-studio/src/index.ts) exposes the original audio clips.diff --git a/packages/speech-track/src/activation.ts b/packages/speech-track/src/activation.tsdeleted file mode 100644index 7ca63b1d..00000000--- a/packages/speech-track/src/activation.ts+++ /dev/null@@ -1,19 +0,0 @@-import { createMarkupSurfaceHostFacet } from "@hypit/markup";-import {- decodeSpeechTrackSurface, speechTrackComponent, speechTrackManifest,- speechTrackModuleRef,- speechTrackMarkupSurfaces,-} from "./index.js";--export const hypitPackage = {- format: "hypit.node-package@1" as const,- modules: [{- manifest: speechTrackManifest,- }],- components: [speechTrackComponent],- hostFacets: [createMarkupSurfaceHostFacet({- module: speechTrackModuleRef,- declaration: speechTrackMarkupSurfaces.find((item) => item.name === "track")!, handler: decodeSpeechTrackSurface,- })],-};-export default hypitPackage;diff --git a/packages/speech-track/src/index.ts b/packages/speech-track/src/index.tsdeleted file mode 100644index eff8f838..00000000--- a/packages/speech-track/src/index.ts+++ /dev/null@@ -1,7 +0,0 @@-/** Official Speech Track authoring and deterministic SemanticTrack assembly. */-export { speechTrackComponent } from "./component.js";-export { createSpeechTrackFragment } from "./fragment.js";-export { speechTrackManifest, speechTrackMarkupSurfaces, speechTrackModuleRef, speechTrackProducers, speechTrackTypes, speechTrackHeaderSchema, speechTrackSetSchema } from "./manifest.js";-export { appendSpeechTrackTake, assembleSpeechTrack, assertSpeechTrackHeader, assertSpeechTrackSet, createSpeechTrackSet, sealSpeechTrackHeader } from "./program.js";-export { decodeSpeechTrackSurface } from "./surface.js";-export type * from "./types.js";diff --git a/packages/speech-track/src/manifest.ts b/packages/speech-track/src/manifest.tsdeleted file mode 100644index e55ffad2..00000000--- a/packages/speech-track/src/manifest.ts+++ /dev/null@@ -1,73 +0,0 @@-import { semanticTakeSchema, speechDependency, speechTypes } from "@hypit/speech";-import { compositionDependency, compositionTypes } from "@hypit/composition";-import { semanticTrackDependency, semanticTrackTypes } from "@hypit/semantic-track";-import type { ModuleManifest, ValueSchema } from "@hypit/protocol";-export const speechTrackModuleRef = { name: "@hypit/speech-track", version: "1" } as const;-export const speechTrackTypes = {- header: { module: speechTrackModuleRef, name: "SpeechTrackHeader" },- trackSet: { module: speechTrackModuleRef, name: "SpeechTrackSet" },-} as const;-export const speechTrackProducers = {- createSet: { module: speechTrackModuleRef, name: "create-track-set" },- appendTake: { module: speechTrackModuleRef, name: "append-track-take" },- assembleTrack: { module: speechTrackModuleRef, name: "assemble-semantic-track" },-} as const;-export const speechTrackHeaderSchema: ValueSchema = { kind: "object", fields: { id: { schema: { kind: "string", minLength: 1 } } } };-export const speechTrackSetSchema: ValueSchema = { kind: "object", fields: { takes: { schema: { kind: "array", items: { kind: "object", fields: { semantic: { schema: semanticTakeSchema } } } } } } };-export const speechTrackMarkupSurfaces = [{- name: "track", tag: "Track", mode: "structured",- outputs: [speechTrackTypes.header, semanticTrackTypes.track, compositionTypes.audioTrack],- vocabulary: {- summary: "Assembles ordered SemanticTakes into one semantic timeline and its original performance audio.",- attributes: [{ name: "id", kind: "identifier", required: true, summary: "Names the semantic assembly and its outputs." }],- children: [{ tag: "Take", cardinality: "many", summary: "A prepared SemanticTake in performance order.",- attributes: [{ name: "source", kind: "reference", required: true, accepts: [speechTypes.semanticTake], summary: "The normalized, aligned performance." }] }],- ports: [- { name: "semantic", type: semanticTrackTypes.track, summary: "The semantic timeline, retaining its prepared material." },- { name: "audio", type: compositionTypes.audioTrack, summary: "Original performance audio aligned to that timeline; select it explicitly in Film." },- ],- example: '<speech:Track id="speech"><speech:Take source={opening.take}/><speech:Take source={answer.take}/></speech:Track>',- notes: ["Use Media or a project visual component to present the performance. Those components choose placement and motion independently of semantic assembly."],- },-}] as const;-export const speechTrackManifest: ModuleManifest = {- format: "hypit.module@1",- name: speechTrackModuleRef.name,- version: speechTrackModuleRef.version,- dependencies: [- speechDependency,- compositionDependency,- semanticTrackDependency,- ],- types: [- { name: speechTrackTypes.header.name },- { name: speechTrackTypes.trackSet.name },- ],- capabilities: [],- producers: [- {- name: speechTrackProducers.createSet.name,- inputs: [],- outputs: [{ name: "set", type: speechTrackTypes.trackSet }],- needs: [],- },- {- name: speechTrackProducers.appendTake.name,- inputs: [- { name: "set", type: speechTrackTypes.trackSet },- { name: "take", type: speechTypes.semanticTake },- ],- outputs: [{ name: "set", type: speechTrackTypes.trackSet }],- needs: [],- },- {- name: speechTrackProducers.assembleTrack.name,- inputs: [- { name: "header", type: speechTrackTypes.header },- { name: "set", type: speechTrackTypes.trackSet },- ],- outputs: [{ name: "track", type: semanticTrackTypes.track }],- needs: [],- },- ],-};diff --git a/packages/speech-track/src/program.ts b/packages/speech-track/src/program.tsdeleted file mode 100644index c49ff5f4..00000000--- a/packages/speech-track/src/program.ts+++ /dev/null@@ -1,64 +0,0 @@-import { assertSemanticTakeIdentity } from "@hypit/speech";-import type { SemanticTake } from "@hypit/speech";-import { sealSemanticTrack } from "@hypit/semantic-track";-import type { SemanticTrack } from "@hypit/semantic-track";-import type { SpeechTrackHeader, SpeechTrackSet, SpeechTrackTake } from "./types.js";-function assert(condition: unknown, message: string): asserts condition {- if (!condition) throw new Error(message);-}-export function sealSpeechTrackHeader(value: SpeechTrackHeader): SpeechTrackHeader { return { id: value.id }; }-export function assertSpeechTrackHeader(value: SpeechTrackHeader): void {- assert(value.id.trim().length > 0, "SpeechTrackHeader id must not be empty");-}-function sealSpeechTrackSet(value: SpeechTrackSet): SpeechTrackSet { return structuredClone(value); }-function assertTake(take: SpeechTrackTake): void { assertSemanticTakeIdentity(take.semantic); }-export function assertSpeechTrackSet(value: SpeechTrackSet): void {- const segments = new Set<string>();- for (const take of value.takes) {- assertSemanticTakeIdentity(take.semantic);- const segmentId = take.semantic.segment.segmentId;- assert(!segments.has(segmentId), `Speech Track repeats Segment ${segmentId}`);- segments.add(segmentId);- }-}--export function createSpeechTrackSet(): SpeechTrackSet {- return sealSpeechTrackSet({- takes: [],- });-}--function appendTake(- set: SpeechTrackSet,- take: SpeechTrackTake,-): SpeechTrackSet {- assertSpeechTrackSet(set);- assertTake(take);- assert(!set.takes.some((item) => item.semantic.segment.segmentId === take.semantic.segment.segmentId),- `Speech Track repeats Segment ${take.semantic.segment.segmentId}`);- return sealSpeechTrackSet({- takes: [...set.takes, take],- });-}--export function appendSpeechTrackTake(set: SpeechTrackSet, semantic: SemanticTake): SpeechTrackSet {- return appendTake(set, { semantic });-}--export function assembleSpeechTrack(- header: SpeechTrackHeader,- set: SpeechTrackSet,-): SemanticTrack {- assertSpeechTrackHeader(header);- assertSpeechTrackSet(set);- assert(set.takes.length > 0, "Speech Track must contain at least one Take");- for (const take of set.takes) assertTake(take);- const narrativeId = set.takes[0]!.semantic.narrativeId;- assert(set.takes.every((take) => take.semantic.narrativeId === narrativeId),- "Speech Track cannot mix Takes from different Narratives");- return sealSemanticTrack({- id: header.id,- narrativeId,- items: set.takes.map((item) => ({ take: item.semantic })),- });-}diff --git a/packages/speech-track/src/surface.ts b/packages/speech-track/src/surface.tsdeleted file mode 100644index b7055bb9..00000000--- a/packages/speech-track/src/surface.ts+++ /dev/null@@ -1,35 +0,0 @@-import { assertAttributes, assertEmptyElement, localName, textAttribute, type StructuredSurfaceHandler } from "@hypit/markup";-import { sameType } from "@hypit/protocol";-import { speechTypes } from "@hypit/speech";-import { createSpeechTrackFragment } from "./fragment.js";-import { speechTrackTypes } from "./manifest.js";--export const decodeSpeechTrackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {- assertAttributes(element, ["id"]);- const id = textAttribute(element, "id");- const takes = element.children.flatMap(child => {- if (child.kind === "text") {- if (child.value.trim()) throw new Error("Speech Track accepts only Take children.");- return [];- }- if (localName(child.name) !== "Take") throw new Error("Speech Track accepts only Take children.");- assertAttributes(child, ["source"]);- assertEmptyElement(child);- const source = child.attributes.source;- if (typeof source !== "object" || source.kind !== "reference") throw new Error("Speech Take source must be a reference.");- const resolved = resolveReference(source.path);- if (resolved === undefined || !sameType(resolved.type, speechTypes.semanticTake)) throw new Error("Speech Take source must be a SemanticTake.");- return [resolved];- });- const inputs = takes.map((_, index) => ({ takeName: `take-${index + 1}` }));- const fragment = createSpeechTrackFragment({ takes: inputs });- return {- records: [{ id: `${id}.header`, type: speechTrackTypes.header, value: { kind: "inline", value: { id } }, range: element.range }],- fragments: [fragment],- components: [{ id, fragment: fragment.id, inputs: {- header: { kind: "record", id: `${id}.header` },- ...Object.fromEntries(takes.map((take, index) => [inputs[index]!.takeName, take.ref])),- }, outputs: { semantic: `${id}.semantic`, audio: `${id}.audio` }, range: element.range }],- exports: [`${id}.semantic`, `${id}.audio`],- };-};diff --git a/packages/speech-track/src/types.ts b/packages/speech-track/src/types.tsdeleted file mode 100644index d8e58683..00000000--- a/packages/speech-track/src/types.ts+++ /dev/null@@ -1,6 +0,0 @@-import type { SemanticTake } from "@hypit/speech";-export type SpeechTrackHeader = { readonly id: string };-export type SpeechTrackTake = { readonly semantic: SemanticTake };-export type SpeechTrackSet = { readonly takes: readonly SpeechTrackTake[] };-export type SpeechTrackInput = { readonly takeName: string };-export type SpeechTrackFragmentOptions = { readonly takes: readonly SpeechTrackInput[] };diff --git a/packages/speech-track/test/program.test.ts b/packages/speech-track/test/program.test.tsdeleted file mode 100644index 68ddaf4c..00000000--- a/packages/speech-track/test/program.test.ts+++ /dev/null@@ -1,58 +0,0 @@-import { projectSemanticAudioTrack, projectSemanticMedia } from "@hypit/semantic-track";-import assert from "node:assert/strict";-import test from "node:test";-import { fixtureResource } from "../../../test/fixture-resource.js";--import type { SynchronizedMedia } from "@hypit/media";-import type { SemanticTake } from "@hypit/speech";-import {- appendSpeechTrackTake,- assembleSpeechTrack,- createSpeechTrackSet,- sealSpeechTrackHeader,-} from "@hypit/speech-track";--function semantic(id: string, visual: boolean): SemanticTake {- const media: SynchronizedMedia = {- timeline: { frameRate: { numerator: 30, denominator: 1 }, frameCount: 30 },- ...(visual ? { visual: {- artifact: { kind: "blob", resource: fixtureResource(`${id}:video`), size: 1, mediaType: "video/mp4" },- width: 720,- height: 1280,- } } : {}),- audio: { artifact: { kind: "blob", resource: fixtureResource(`${id}:audio`), size: 1, mediaType: "audio/wav" } },- };- return {- narrativeId: "test-narrative",- media,- segment: {- segmentId: id,- startAnchorId: `segment:${id}:start`,- endAnchorId: `segment:${id}:end`,- startFrame: 0,- endFrameExclusive: 30,- },- tokens: [],- anchors: [- { identity: `segment:${id}:start`, frame: 0 },- { identity: `segment:${id}:end`, frame: 30 },- ],- };-}--test("semantic assembly retains audio and material timing without requiring placement", () => {- const track = assembleSpeechTrack(sealSpeechTrackHeader({ id: "speech" }),- appendSpeechTrackTake(appendSpeechTrackTake(createSpeechTrackSet(), semantic("voiceover", false)), semantic("answer", true)));- assert.deepEqual(track.items.map(item => item.take.segment.segmentId), ["voiceover", "answer"]);- const audio = projectSemanticAudioTrack(track);- assert.deepEqual(audio.clips.map(clip => clip.target), [- { startSample: 0, endSampleExclusive: 48000 }, { startSample: 48000, endSampleExclusive: 96000 },- ]);- const selected = projectSemanticMedia(track, { startFrame: 20, endFrameExclusive: 45 });- assert.deepEqual(selected.map(({ span, source }) => ({ span, source })), [- { span: { startFrame: 20, endFrameExclusive: 30 }, source: { startFrame: 20, endFrameExclusive: 30 } },- { span: { startFrame: 30, endFrameExclusive: 45 }, source: { startFrame: 0, endFrameExclusive: 15 } },- ]);- assert.equal(selected[0]!.media.visual, undefined);- assert.equal(selected[1]!.media.visual?.artifact.resource, fixtureResource("answer:video"));-});diff --git a/packages/speech/README.md b/packages/speech/README.mdindex 632c5e5c..afbc2255 100644--- a/packages/speech/README.md+++ b/packages/speech/README.md@@ -6,8 +6,8 @@ owns the types and helpers below; Source imports retain the `@hypit/speech@1` MoPublic contracts for speech duration, normalized semantic Takes and provider-neutral evidence audio.A `SemanticTake` contains one normalized media product, one authored Segment, its words and local-frame anchors. Ordered Takes are assembled by `@hypit/speech-track` into a `SemanticTrack`, which is-the program's continuous semantic skeleton. `SpeechEvidenceAudio` carries canonical 16 kHz WAV bytes+frame anchors. Ordered Takes are assembled by `@hypit/timeline-author` into a `Timeline`, which is+the complete Program range and Take placements, including gaps or overlap. `SpeechEvidenceAudio` carries canonical 16 kHz WAV bytesand their exact sample count; it never carries Script or Segment identity.The package also owns deterministic boundary materialization for a Segment with no Tokens. It mapsdiff --git a/packages/studio-adapter/README.md b/packages/studio-adapter/README.mdindex ea8324be..3d91cf3d 100644--- a/packages/studio-adapter/README.md+++ b/packages/studio-adapter/README.md@@ -106,7 +106,8 @@ ordered layers, `startFrame`, `endFrameExclusive` and `stackOrder`. Useful optio| Field/helper | Purpose || --- | --- |-| `presentation` | Name the entity's role and choose `standard`, `group` or `point` chrome |+| `selectionGroup` | Link disjoint displayed intervals of one author entity; selection highlights them together without changing their independent timing |+| `presentation` | Name the entity's role and choose `standard`, `group`, `point` or `compact` chrome (a single-line primary label) || `textLayer(text)` | Put explicit domain text in the timeline body || `previewLayer(artifactPreview(kind, resource), layout)` | Show a declared image/video/audio Resource with a finite layout; Studio resolves transport || `renderIds` | Relate an entity to its actual rendered elements |@@ -114,6 +115,7 @@ ordered layers, `startFrame`, `endFrameExclusive` and `stackOrder`. Useful optio| `temporal` | Carry the executed Instant/Window lineage and its edit authority || `lane` and Companion `attachments` | Put child entities on a declared additional lane, with its own bindings and Inspector || `childEntities` / `authoredChildFor` | Resolve children from public identity and exact Spec Types rather than source order guesses |+| `authoredItemTitle` | Prefer an explicit author id, then a source reference from the attributes chosen by the Companion; presentation leaves editing identity unchanged |Use the program's frame space and half-open intervals. Persistent visibility and its activation aredifferent facts: a board item can remain visible until the board ends while its reveal occupies only@@ -152,12 +154,11 @@ physical package identity, so executable package code cannot impersonate anofficial Companion. The Source closure selects project packages; Studio does notscan `node_modules` for plugins or use Runtime Profiles to select Companions.-The same host facet can contribute Film and Script boundary companions. A Film companion declares-the accepted `timeSources` (author attribute and Type) and its terminal Tracks. Film accepts either-a SemanticTrack or a declared ProgramSpace. Track Companion context includes `semantic` when the-time source supplies a semantic timeline; pure animation leaves it undefined. A Script companion owns-raw source observation and marker adjustment. Studio core only matches and invokes these declarations;-it does not import either domain package.+The same host facet can contribute Film, Script and parameter Companions. A Film Companion+identifies its Timeline and terminal Tracks. The Timeline supplies the program range; semantic+rows are populated when it carries Script anchors. A Script Companion owns source observation,+projection of its values into Studio's segment/word/marker rows, and marker adjustment. Studio+consumes that projection without reading the Script package's Narrative representation.For a domain item whose Spec is consumed beside a Window or Instant, call`temporalLineageFor(context, item.id, "window")` using the actual projection input name. Attach the@@ -172,8 +173,8 @@ Inspector editing deliberately has two declarations:- `bindings` names exact author endpoints, including explicit reference pathsinto authored elements or SVS Recipes. A binding is not visible by itself;-- `inspector` selects writable bindings and gives them a `where`, `how` or- `when` domain, an optional package-owned page, a section and one of Studio's+- `inspector` selects bindings and gives them a `where`, `when` or+ `how` domain, an optional package-owned page, a section and one of Studio'sfinite controls (`text`, `number`, `boolean`, `select`, `color`, `list` or`record`). A domain Recipe may declare a shared canonical-value schema; thecompanion chooses its presentation while Studio derives and validates the@@ -181,8 +182,11 @@ Inspector editing deliberately has two declarations:This keeps source traversal, timeline inverses and editor presentation fromsilently becoming one policy. Studio resolves the declarations against the-current Source closure, publishes only real writable fields, renders all DOM-and CSS itself, and commits changes through `parameter.adjust`.+current Source closure. Visible read-only bindings render as text alongside editable fields.+A projected entity can also supply `inspector` values with an `id`, `label`, `domain`, `section`+and `value`, such as its resolved interval. These facts need no Source endpoint.+Only resolved fields with an `edit` endpoint accept `parameter.adjust`. Studio renders all DOM+and CSS itself; grouping does not grant edit authority.Fields can declare `number` display scaling, supported suffixes, limits and step; `unit` alone isonly a label. Select options can carry separate scalar values, labels, descriptions and color/font@@ -205,3 +209,118 @@ is selected, its field reaches the correct Source or Recipe, and changing it recFor semantic handles, verify the exact marker and its other consumers too. Read-only derived timingis preferable to an invented inverse. The Companion explains the component; its Producers remainresponsible for identical video behavior in Studio, seeking and encoded rendering.++Compact chrome uses `display.title` as its complete single-line content: a Cue can put its subtitle+text there and a Use can put its Style name there. It has no separate thumbnail/body region or+inline duration. Time remains in the tooltip and Companion-selected Inspector facts. Tone and lane+height are independent declarations; compact does not recognize Caption or Use names.++Choose lane tones explicitly. The `blue-muted`, `green-muted`, `magenta-muted`+and `orange-muted` palette entries provide quieter rows within the same color+family. For example, content and its presentation Uses can share a hue while+the Uses take the muted tone. Studio does not derive tone from attachment depth+or the entity name.++### Internal bands and child Tracks++A Track can place independently timed entities in internal `bands`. These strips share its label+and meet without gaps. Use them for aspects of the same content, such as a bottom strip of+presentation rules. Keep `attachments` for independently represented child objects with their own+labels, such as a ranking board's reveals. These declarations affect Studio presentation only.++```ts+{+ lane: { heightPx: 60 },+ bands: [{+ id: "rules",+ placement: "after",+ heightPx: 15,+ display: "label",+ bindings: [{ name: "style" }],+ inspector: [{+ binding: "style", label: "Style", domain: "how", control: "text",+ section: { id: "presentation", label: "Presentation" },+ }],+ }],+}+```++The projection puts `band: "rules"` on the corresponding entity drafts. Other entities remain in+the main content area. A band can be `before` or `after` that area; declarations retain their order+within each placement. `display: "label"` fills the strip with entity titles, while `"content"`+retains the entity's normal chrome and body. An optional `tone` selects another palette entry;+omission inherits the Track tone. Heights are independent, so a label band can match the 15px+standard item header.++An entity selects either an internal `band` or a child `lane`. Bands retain their own intervals,+overlap ordering, selection, Inspector declarations and temporal writeback. A rule spanning three+content blocks stays one entity; Studio does not split it at content boundaries. Band entities stay+in the Track's `clips` collection, so ordinary selection and editing use the same identities.++### Parameters owned by referenced objects++A consumer opts a reference into its owner's parameter Companion:++```ts+bindings: [{ name: "style", companion: true }]+```++The package that authors the referenced object contributes through+`createStudioCompanionHostFacet({ parameters: [...] })`:++```ts+{+ id: "slide",+ match: { module: myModule, surface: "slide" },+ bindings: [{ name: "distance", writable: true, fallback: 0.25 }],+ inspector: [{+ binding: "distance", label: "Travel", domain: "where",+ section: { id: "path", label: "Path" }, control: "number",+ unit: "%", number: { scale: 100 },+ }],+}+```++Studio resolves the actual reference and matches its Module/Surface. It prefixes this object's+bindings and fields under the consumer reference; the example becomes `style.distance`.+Nested `referenced` and `recipe` bindings use the same composition. The Use retains its own+Window and time gestures. Several Uses referencing one Style edit the same source object.+A project Style can therefore publish controls without replacing the Performance, Sound or+Caption Track Companion. No parameter Companion means the reference remains visible with only+its consumer-declared fields.++A source or Recipe binding may declare a typed `fallback`, or a function of the authored+properties for a dependent default. For example, Sound's end gain follows its start gain until+authored explicitly. The field displays the fallback; the first edit inserts the attribute/property+in its owning SVML/SVS file. Source recompilation remains the owner of current values.+Only expose omitted defaults whose insertion preserves valid author semantics; geometry inputs+whose legality depends on a different Frame form can remain tied to the authored form.++Fields without `page` remain visible alongside the selected named page within Where, When or How.+Read-only and editable fields can share a section. Each package chooses a small useful field set.++### Edit related attributes together++A source binding can expose several scalar attributes as one `record` field:++```ts+{ name: "layout", writable: true, attributes: ["mode", "columns"],+ fallback: { mode: "flow" },+ schema: { kind: "oneOf", variants: [+ { kind: "object", fields: { mode: { schema: { kind: "literal", value: "flow" } } } },+ { kind: "object", fields: {+ mode: { schema: { kind: "literal", value: "grid" } },+ columns: { schema: { kind: "number", integer: true, minimum: 1 } },+ } },+ ] } }+```++The synthetic binding name is editor vocabulary; it creates no `layout` Source attribute.+The field's `control: "record"` saves a complete, valid value when editing ends. Object alternatives with a shared, distinct+literal field provide a mode selector; selecting one prepares that alternative's fields.+Incomplete values stay in the editor with a completion hint until the required fields are filled.+Only the declared attributes are replaced together. Omitted members are removed, while unrelated+attributes, references and child content survive. Existing references within the group remain+read-only. Domain validation still owns valid author values; existing transactions reject an+invalid edit without leaving invalid Source behind. Source grouping is available on direct and+referenced element bindings and stays data-only across the editor boundary.diff --git a/packages/studio-adapter/src/index.ts b/packages/studio-adapter/src/index.tsindex b910551a..6864b9c7 100644--- a/packages/studio-adapter/src/index.ts+++ b/packages/studio-adapter/src/index.ts@@ -21,18 +21,21 @@ export type StudioIcon =/** Studio-owned visual palette. Domain families never become CSS selectors. */export type StudioTimelineTone =| "blue"+ | "blue-muted"| "green"+ | "green-muted"| "teal"| "violet"| "magenta"+ | "magenta-muted"| "orange"| "orange-muted"| "neutral";export type StudioTimelinePresentation = {readonly entity: string;- /** Studio-owned shell. It never decides whether title, time or body exists. */- readonly chrome: "standard" | "group" | "point";+ /** Studio-owned shell. Compact presents the primary label in one line, with time in details. */+ readonly chrome: "standard" | "group" | "point" | "compact";};export type StudioTemporalSource = {@@ -140,6 +143,12 @@ export type StudioNumberPresentation = {readonly step?: number;};+/** Attribute ranges are relative to the owning element's source preimage. */+export type StudioAttributeGroupEdit = {+ readonly insertionOffset: number;+ readonly ranges: Readonly<Record<string, Range | null>>;+};+/** A real author endpoint. Its existence never implies Inspector visibility. */export type StudioSourceBinding = {readonly id: string;@@ -162,10 +171,11 @@ export type StudioSourceBinding = {readonly prefix?: string;readonly suffix?: string;};+ readonly attributes?: StudioAttributeGroupEdit;readonly disabledReason?: string;};-export type StudioInspectorDomain = "where" | "how" | "when";+export type StudioInspectorDomain = "where" | "when" | "how";export type StudioInspectorPageDeclaration = {readonly id: string;@@ -177,7 +187,7 @@ export type StudioInspectorSectionDeclaration = {readonly label: string;};-/** One visible, writable field selected from a Companion's source bindings. */+/** One visible field selected from a Companion's source bindings. */export type StudioInspectorFieldDeclaration = {readonly binding: string;readonly label: string;@@ -196,14 +206,25 @@ export type StudioInspectorFieldDeclaration = {readonly swatches?: readonly string[];};-/** Resolved Inspector DTO. Studio renders it and executes its exact source write. */-export type StudioInspectorField = Omit<StudioInspectorFieldDeclaration, "control"> & {+/** A selected entity's package-owned fact, with no author write endpoint. */+export type StudioInspectorValue = Pick<StudioInspectorFieldDeclaration,+ "label" | "domain" | "page" | "section" | "summary" | "unit"> & {+ readonly id: string;+ readonly value: CanonicalValue;+};++/** Display and edit authority are independent of the Where / When / How grouping. */+export type StudioInspectorField = Omit<StudioInspectorFieldDeclaration, "binding" | "control"> & {+ readonly binding?: string;readonly id: string;readonly control: StudioParameterControl;readonly value: CanonicalValue;readonly schema?: ValueSchema;- readonly language: StudioParameterLanguage;- readonly source: StudioSourceBinding["source"];+ readonly edit?: {+ readonly language: StudioParameterLanguage;+ readonly source: StudioSourceBinding["source"];+ readonly attributes?: StudioAttributeGroupEdit;+ };};/** Companion-owned source allowlist. It contains no editor presentation. */@@ -211,6 +232,12 @@ export type StudioSourceBindingDeclaration = {readonly name: string;readonly writable?: boolean;readonly schema?: ValueSchema;+ /** Typed value used when this editable attribute is omitted. */+ readonly fallback?: CanonicalValue | ((authored: Readonly<Record<string, CanonicalValue>>) => CanonicalValue | undefined);+ /** Compose the referenced object's package-owned parameter Companion. */+ readonly companion?: boolean;+ /** Edit these scalar attributes together as one schema-described record. */+ readonly attributes?: readonly string[];/** Optional declaration for the authored element named by a reference. */readonly referenced?: readonly StudioSourceBindingDeclaration[];/**@@ -234,7 +261,7 @@ export type StudioRecipeBindingDeclaration = {/** Domain-owned public Recipe value structure, never editor presentation. */readonly schema?: ValueSchema;/** Typed public fallback; Studio writes the property only after the author changes it. */- readonly fallback?: CanonicalValue;+ readonly fallback?: CanonicalValue | ((authored: Readonly<Record<string, CanonicalValue>>) => CanonicalValue | undefined);};export type StudioTimelineGesture =@@ -323,7 +350,7 @@ export type StudioDisplayLayer =};export type StudioEntityDisplay = {- /** First row. Studio always places computed time immediately after it. */+ /** Primary label: a header in standard chrome, the complete single-line content in compact chrome. */readonly title: string;/** Ordered back-to-front body layers. */readonly layers: readonly StudioDisplayLayer[];@@ -486,7 +513,7 @@ export type StudioResolvedTrack = {};export type StudioViewRole =- | "semantic-track"+ | "timeline"| "supporting-value"| "track";@@ -514,10 +541,16 @@ export type StudioEntityDraft = {readonly renderIds?: readonly string[];/** Resolved author references that differ per derived entity, such as one Cue's actual Style. */readonly parameterReferences?: Readonly<Record<string, string>>;+ /** Disjoint displayed intervals belonging to one selectable author entity. */+ readonly selectionGroup?: string;+ /** Deliberately selected facts, presented beside bound Inspector fields. */+ readonly inspector?: readonly StudioInspectorValue[];readonly presentation?: StudioTimelinePresentation;readonly temporal?: StudioTemporalLineage;- /** Studio-local lane partition; omitted means the root lane. */+ /** Independent child Track partition; omitted means this Track. */readonly lane?: string;+ /** Internal band of this Track; mutually exclusive with lane. */+ readonly band?: string;};export type StudioTrackCompanionContext = {@@ -550,6 +583,8 @@ export type StudioTrackCompanion = {/** Opts root timeline entities into the component's package-owned Surface preview. */readonly poster?: { readonly source: "surface-preview" };readonly attachments?: readonly StudioLaneAttachment[];+ /** Internal bands share this Track’s label and contain independently timed entities. */+ readonly bands?: readonly StudioTrackBand[];/** Same-Surface output values required to project this Track for Studio. */readonly requiredValues?: readonly string[];readonly lane?: StudioLaneDescription;@@ -560,11 +595,20 @@ export type StudioTrackCompanion = {readonly project?: (context: StudioTrackCompanionContext) => readonly StudioEntityDraft[];};+/** Parameters owned by an authored object, independently of its consumers. */+export type StudioParameterCompanion = {+ readonly id: string;+ readonly match: { readonly module: ModuleRef; readonly surface: string };+ readonly bindings: readonly StudioSourceBindingDeclaration[];+ readonly inspector: readonly StudioInspectorFieldDeclaration[];+};+export type StudioCompanionContribution = {readonly format: "hypit.studio-companions@1";readonly tracks: readonly StudioTrackCompanion[];readonly films?: readonly StudioFilmCompanion[];readonly scripts?: readonly StudioScriptCompanion[];+ readonly parameters?: readonly StudioParameterCompanion[];};/** Declarative Film boundary. Studio follows only the references named here. */@@ -605,6 +649,13 @@ export type StudioScriptAdjustment =| { readonly kind: "selection"; readonly id: string; readonly startAnchorId: string; readonly endAnchorId: string }| { readonly kind: "moment"; readonly id: string; readonly anchorId: string };+export type StudioScriptProjection = Pick<StudioSemanticTimeline, "anchors" | "segments" | "tokens" | "selections" | "moments">;+export type StudioScriptProjectionInput = {+ readonly source: StudioScriptSourceMap;+ readonly values: readonly StudioObservedValue[];+ readonly anchors: ReadonlyMap<string, number>;+};+/** Script grammar companion. It owns source observation and semantic inverse edits. */export type StudioScriptCompanion = {readonly id: string;@@ -619,6 +670,7 @@ export type StudioScriptCompanion = {readonly nextOffset?: number;readonly attributes: Readonly<Record<string, unknown>>;}) => Omit<StudioScriptSourceMap, "companion"> | undefined;+ readonly project?: (input: StudioScriptProjectionInput) => StudioScriptProjection;readonly adjust: (input: {readonly sourceName: string;readonly source: string;@@ -642,6 +694,7 @@ export function createStudioCompanionHostFacet(input: {readonly tracks?: readonly StudioTrackCompanion[];readonly films?: readonly StudioFilmCompanion[];readonly scripts?: readonly StudioScriptCompanion[];+ readonly parameters?: readonly StudioParameterCompanion[];}): StudioCompanionHostFacet {return {abi: studioCompanionHostAbi,@@ -650,6 +703,7 @@ export function createStudioCompanionHostFacet(input: {tracks: input.tracks ?? [],...(input.films === undefined ? {} : { films: input.films }),...(input.scripts === undefined ? {} : { scripts: input.scripts }),+ ...(input.parameters === undefined ? {} : { parameters: input.parameters }),},};}@@ -658,6 +712,7 @@ export type StudioPackageContribution = {readonly tracks: readonly StudioTrackCompanion[];readonly films: readonly StudioFilmCompanion[];readonly scripts: readonly StudioScriptCompanion[];+ readonly parameters: readonly StudioParameterCompanion[];};function qualify(owner: string, local: string, subject: string): string {@@ -675,6 +730,7 @@ export function studioContributionFromPackage(const tracks: StudioTrackCompanion[] = [];const films: StudioFilmCompanion[] = [];const scripts: StudioScriptCompanion[] = [];+ const parameters: StudioParameterCompanion[] = [];for (const facet of facets) {if (facet.abi !== studioCompanionHostAbi) continue;const contribution = facet.implementation as Partial<StudioCompanionContribution>;@@ -689,12 +745,15 @@ export function studioContributionFromPackage(...film,id: qualify(owner, film.id, "Studio Film companion"),})));+ parameters.push(...(contribution.parameters ?? []).map((parameter) => ({+ ...parameter, id: qualify(owner, parameter.id, "Studio Parameter companion"),+ })));scripts.push(...(contribution.scripts ?? []).map((script) => ({...script,id: qualify(owner, script.id, "Studio Script companion"),})));}- return { tracks, films, scripts };+ return { tracks, films, scripts, parameters };}/**@@ -708,6 +767,18 @@ export function studioTrackCompanionsFromPackage(return studioContributionFromPackage(owner, facets).tracks;}+/** A contiguous internal strip, not a child Track. Declaration order is visual order. */+export type StudioTrackBand = {+ readonly id: string;+ readonly placement: "before" | "after";+ readonly heightPx: number;+ /** Label bands use the whole strip for each entity's title; content bands retain its chrome. */+ readonly display: "label" | "content";+ readonly tone?: StudioTimelineTone;+ readonly bindings?: readonly StudioSourceBindingDeclaration[];+ readonly inspector?: readonly StudioInspectorFieldDeclaration[];+};+export type StudioLaneAttachment = {readonly id: string;readonly family: StudioTrackFamily;@@ -836,6 +907,22 @@ export function temporalSemanticSource(lineage: StudioTemporalLineage | undefine: undefined;}+/** A package chooses which source references can name an otherwise unnamed Item. */+export function authoredItemTitle(+ context: StudioTrackCompanionContext,+ authoredId: string,+ sourceTypes: readonly TypeRef[],+ sourceAttributes: readonly string[],+): string {+ const child = authoredChildFor(context, authoredId, sourceTypes);+ if (child?.attributes.id) return child.attributes.id;+ for (const attribute of sourceAttributes) {+ const reference = child?.referenceAttributes[attribute];+ if (reference) return reference;+ }+ return authoredId;+}+export function childEntities(context: StudioTrackCompanionContext,items: readonly {diff --git a/packages/studio/INSPECTOR.md b/packages/studio/INSPECTOR.mdindex dd710e0f..5e81ae12 100644--- a/packages/studio/INSPECTOR.md+++ b/packages/studio/INSPECTOR.md@@ -18,7 +18,20 @@ its color is How, and the object's frame is Where. One field has one owner and o## Controls describe the editing task`bindings` exposes actual author endpoints; `inspector` chooses which to show. A resolved field is-shown only when its binding is writable. The finite controls are independent of the three domains:+shown when selected by the Companion, including read-only bindings. The interface orders available+domains as Where, When, How. Read-only values and editable controls share the same sections.++A projected entity may supply a few computed `inspector` facts without binding them to Source:++```ts+inspector: [{ id: "range", label: "Range", domain: "when",+ section: { id: "placement", label: "Placement" }, value: "15–105", unit: "f" }]+```++The resolved DTO carries an `edit: { language, source }` only for writable author fields.+The UI uses plain selectable text otherwise; the server requires that endpoint for writes.+Choose facts that explain the selected object or its controls. Component headers organize lanes;+the selected entities own their details. The finite controls are independent of the three domains:| Control | Use || --- | --- |@@ -30,7 +43,7 @@ shown only when its binding is writable. The finite controls are independent of| `list` | An ordered value described by an array schema, such as a palette or repeated settings. || `record` | Named values described by an object schema, such as one shadow's parameters. |-Lists and records retain Apply/Reset drafts. Their schema owns value shape. There is no separate+Lists and records save complete, valid values when editing ends. Incomplete edits show a completion hint; their schema owns value shape. There is no separatepalette database, font control protocol, generic CSS editor, or component-supplied DOM renderer.Use text for an expression whose parts cannot meaningfully be edited as one number.diff --git a/packages/studio/README.md b/packages/studio/README.mdindex 929180e8..4c4c142e 100644--- a/packages/studio/README.md+++ b/packages/studio/README.md@@ -73,13 +73,17 @@ transient processing require the selected Runtime.## Opening and editing a sessionThe selected targets must reach one Film and its resolved time source. Studio rejects several-distinct Films in one view; use separate Runs/sessions for those. A SemanticTrack supplies the-Script lane and performance timing, including wordless Segments with media spans. A declared-ProgramSpace supplies authored animation time without a Script lane. Both show their component-tracks and use the same rendering and parameter-editing machinery. When the snapshot contains-a semantic timeline, its lane and label stay pinned directly below the time ruler while-component tracks scroll. Its presence follows the resolved semantic timeline, including-wordless content, rather than a component name or the presence of spoken words.+distinct Films in one view; use separate Runs/sessions for those. A Timeline supplies the+placed material and semantic timing, including wordless Segments. The ruler always shows program+time. When placed Segments are present, its left Timeline header spans the time ticks and the+Segment, Word and Selection/Moment bands. Empty Word or marker bands are omitted based on the+whole work, not the visible window or playhead. With no Segments, only the ordinary time ruler remains.+The complete ruler area stays pinned while component tracks scroll.++Semantic objects retain declaration-order drawing within each band. Their fine borders distinguish+adjacent and overlapping intervals without inventing time gaps. Selection raises an object above+its peers; playback highlighting changes color without changing that order. Right-click an overlap+to choose a covered object. These are editor presentation rules, independent of Film paint order.Open the URL printed by Vite. Studio requests port 5179 by default, accepts `--port`,and Vite can choose another available port when it is occupied. Reuse that process@@ -90,11 +94,14 @@ package code or imports that introduce new packages. Browser refresh does not reserver-side package modules.The Source pane can edit the selected `.svml`, `.svs` or `.svrun` file. The Inspector-shows project facts when no entity is selected and only declared writable fields for+shows project facts when no entity is selected and declared facts and author fields forthe selected entity. A reference can resolve to a shared Frame or Recipe, so one edit-may affect several consumers. Structured fields use Apply/Reset for their local draft.+may affect several consumers. Structured fields save together when editing ends and the value is complete; missing required values stay in the editor with a completion hint.Check save status; source conflicts reject stale edits rather than overwrite newer files.+The Timeline owns the editor's complete range; displayed objects do not extend it. Take placement+and complete extent are reference information in Studio.+Timeline gestures use explicit temporal authority. Moving a shared Selection orMoment edits Script and moves its consumers after recompilation. A parameter-basedhandle edits its exact authored parameter. Fixed or derived values with no supported@@ -137,12 +144,11 @@ at runtime. Ship the Companion's compiled JavaScript with its component package.The companion owns what its Track means: matching, required same-Surface values,entities, lane range, finite chrome, title and ordered text/material layers,source bindings, Inspector fields and executed temporal lineage. Inspector fields-select real writable bindings and organize them under the Studio-owned-`Where / How / When` domains, optional companion-owned pages and sections. A+select real bindings or package-owned facts and organize them under the Studio-owned+`Where / When / How` domains, optional companion-owned pages and sections. Asource binding is never shown merely because Studio can reach it. Material layers carry a-Resource id or Surface identity, never a Studio HTTP URL. Studio always owns-time formatting and transport resolution, so chrome and material cannot hide a-title or its time.+Resource id or Surface identity, never a Studio HTTP URL. Studio owns time formatting and transport resolution. Compact chrome shows one primary label;+computed time remains available in its tooltip rather than competing with that label.Studio owns session-wide behavior and chrome: Companion assembly, single-row overlap display,fallback defaults, selection treatment, playback, zoom,scrolling, the finite Inspector control set and source mutation transport. A@@ -184,3 +190,11 @@ semantics.Read [`@hypit/studio-adapter`](../studio-adapter/README.md) for the Companion ABI,project activation example, value projection and Inspector declarations.++## Composition audio preview++The preview consumes generic AudioClips: source sampling, fixed gain, program-sample gain envelopes,+audibility regions and target-relative fades. Web Audio gain nodes schedule the independent factors+on the same clock, including silence and amplification above unity. Visual video elements stay muted;+the Film's selected AudioTracks own composition sound. Seeking is silent. This path is independent of+which component emitted a clip.diff --git a/packages/studio/package.json b/packages/studio/package.jsonindex a81658d2..bb1844a3 100644--- a/packages/studio/package.json+++ b/packages/studio/package.json@@ -12,34 +12,34 @@"@hypit/build-result": "workspace:*","@hypit/caption-fine-studio": "workspace:*","@hypit/cli": "workspace:*",+ "@hypit/comment-sticker-studio": "workspace:*","@hypit/compiler-node": "workspace:*","@hypit/component-kit": "workspace:*","@hypit/composition": "workspace:*",- "@hypit/comment-sticker-studio": "workspace:*","@hypit/core": "workspace:*",- "@hypit/driver-node": "workspace:*","@hypit/deck-track-studio": "workspace:*",+ "@hypit/driver-node": "workspace:*","@hypit/elaborator": "workspace:*","@hypit/film-studio": "workspace:*","@hypit/hyperframes": "workspace:*","@hypit/markup": "workspace:*","@hypit/media-track-studio": "workspace:*",- "@hypit/narrative": "workspace:*","@hypit/package-loader-node": "workspace:*",+ "@hypit/performance-studio": "workspace:*",+ "@hypit/sound-studio": "workspace:*","@hypit/program-space": "workspace:*","@hypit/protocol": "workspace:*","@hypit/ranking-studio": "workspace:*","@hypit/run": "workspace:*","@hypit/runtime": "workspace:*","@hypit/runtime-host-node": "workspace:*",- "@hypit/script-studio": "workspace:*","@hypit/screen-overlay-studio": "workspace:*",+ "@hypit/script-studio": "workspace:*","@hypit/speech": "workspace:*",- "@hypit/semantic-track": "workspace:*",+ "@hypit/studio-adapter": "workspace:*","@hypit/svs": "workspace:*","@hypit/temporal": "workspace:*",- "@hypit/studio-adapter": "workspace:*",- "@hypit/speech-track-studio": "workspace:*",+ "@hypit/timeline": "workspace:*","@hypit/typography-track-studio": "workspace:*","@hypit/validation": "workspace:*","@hypit/video-cli": "workspace:*",diff --git a/packages/studio/src/companion-assembly.ts b/packages/studio/src/companion-assembly.tsindex 07ebbfd7..cd52bf6e 100644--- a/packages/studio/src/companion-assembly.ts+++ b/packages/studio/src/companion-assembly.ts@@ -34,6 +34,7 @@ export async function loadStudioCompanionRegistry(input: {{films: contributions.flatMap((item) => item.films),scripts: contributions.flatMap((item) => item.scripts),+ parameters: contributions.flatMap((item) => item.parameters),},);}diff --git a/packages/studio/src/domain.ts b/packages/studio/src/domain.tsindex 3de6d4ac..d9c390e9 100644--- a/packages/studio/src/domain.ts+++ b/packages/studio/src/domain.ts@@ -33,7 +33,7 @@ export type StudioDomain = {/*** Assemble Studio from the same recursive package selection as the official CLI.- * The application owns this video-domain view; packages never register Studio metadata.+ * The application assembles the video-domain view; selected packages contribute Studio Companions through host facets.*/export async function loadStudioDomain(input: {readonly run: string;diff --git a/packages/studio/src/fallback-companions.ts b/packages/studio/src/fallback-companions.tsindex 71170612..7d789f1b 100644--- a/packages/studio/src/fallback-companions.ts+++ b/packages/studio/src/fallback-companions.ts@@ -1,7 +1,7 @@import type { StudioTrackCompanion, StudioTrackCompanionContext, StudioEntityDraft } from "@hypit/studio-adapter";import { artifactPreview, previewLayer, temporalLineageFor, temporalSemanticSource } from "@hypit/studio-adapter";import { compositionTypes } from "@hypit/composition";-import { semanticTrackTypes } from "@hypit/semantic-track";+import { timelineTypes } from "@hypit/timeline";type TerminalVisualTrack = {readonly presents?: readonly {@@ -79,8 +79,8 @@ function projectTerminalAudio(context: StudioTrackCompanionContext): readonly St/** Cross-domain terminal protocols understood even when no Companion is installed. */export const fallbackStudioTrackCompanions: readonly StudioTrackCompanion[] = [{- id: "@hypit/studio#semantic-track", role: "semantic-track", output: { type: semanticTrackTypes.track },- family: "semantic", tone: "teal", label: "Semantic", icon: "brand",+ id: "@hypit/studio#timeline", role: "timeline", output: { type: timelineTypes.track },+ family: "semantic", tone: "teal", label: "Timeline", icon: "brand",lane: { heightPx: 45 },},{diff --git a/packages/studio/src/parameter-values.ts b/packages/studio/src/parameter-values.tsindex 5b26e8a9..e86b6830 100644--- a/packages/studio/src/parameter-values.ts+++ b/packages/studio/src/parameter-values.ts@@ -1,6 +1,6 @@import type { CanonicalValue, ValueSchema } from "@hypit/protocol";import { formatSvsValue } from "@hypit/svs";-import type { StudioInspectorField, StudioParameterOption, StudioNumberPresentation } from "@hypit/studio-adapter";+import type { StudioInspectorField, StudioParameterControl, StudioParameterOption, StudioNumberPresentation } from "@hypit/studio-adapter";export const parameterOption = (option: StudioParameterOption): Exclude<StudioParameterOption, string> =>typeof option === "string" ? { value: option, label: option } : option;@@ -119,3 +119,45 @@ export function serializeParameterValue(value: CanonicalValue, language: "svml"if (typeof value === "number" || typeof value === "boolean") return String(value);throw new Error(`${language.toUpperCase()} parameter bindings currently accept scalar values only.`);}++/** Record alternatives are selected by an explicitly literal-valued field. */+export function parameterRecordVariants(schema: ValueSchema): readonly {+ key: string; value: CanonicalValue; schema: Extract<ValueSchema, { kind: "object" }>;+}[] {+ if (schema.kind !== "oneOf" || !schema.variants.every(variant => variant.kind === "object")) return [];+ const records = schema.variants as readonly Extract<ValueSchema, { kind: "object" }>[];+ const key = Object.keys(records[0]?.fields ?? {}).find(name => records.every(record => record.fields[name]?.schema.kind === "literal")+ && new Set(records.map(record => JSON.stringify((record.fields[name]!.schema as { value: CanonicalValue }).value))).size === records.length);+ return key === undefined ? [] : records.map(record => ({ key, value: (record.fields[key]!.schema as { value: CanonicalValue }).value, schema: record }));+}++export function parameterRecordSchema(schema: ValueSchema, value: CanonicalValue): Extract<ValueSchema, { kind: "object" }> | undefined {+ if (schema.kind === "object") return schema;+ if (value === null || Array.isArray(value) || typeof value !== "object") return undefined;+ return parameterRecordVariants(schema).find(variant => (value as Readonly<Record<string, CanonicalValue>>)[variant.key] === variant.value)?.schema;+}++/** Replace only a Companion-declared group; unrelated attributes and child content survive. */+export function serializeAttributeGroup(field: StudioInspectorField, value: CanonicalValue): string {+ const group = field.edit?.attributes;+ if (!group || value === null || Array.isArray(value) || typeof value !== "object") throw new Error("Expected an attribute record.");+ const entries = Object.entries(value);+ if (entries.some(([name]) => !Object.hasOwn(group.ranges, name))) throw new Error("Attribute is outside this group.");+ let text = field.edit!.source.preimage;+ const ranges = Object.values(group.ranges).filter((range): range is NonNullable<typeof range> => range !== null);+ for (const range of ranges.sort((a, b) => b.start - a.start)) text = text.slice(0, range.start) + text.slice(range.end);+ const attributes = entries.map(([name, held]) => ` ${name}="${serializeParameterValue(held, field.edit!.language)}"`).join("");+ return text.slice(0, group.insertionOffset) + attributes + text.slice(group.insertionOffset);+}++export function parameterControlForSchema(schema: ValueSchema | undefined): StudioParameterControl | undefined {+ if (schema === undefined) return undefined;+ if (schema.kind === "boolean") return "boolean";+ if (schema.kind === "number") return "number";+ if (schema.kind === "string") return schema.enum === undefined+ ? schema.format === "color" ? "color" : "text"+ : "select";+ if (schema.kind === "array") return "list";+ if (schema.kind === "object" || parameterRecordVariants(schema).length > 0) return "record";+ return undefined;+}diff --git a/packages/studio/src/parameters.ts b/packages/studio/src/parameters.tsindex e0720cfe..e1d69cb0 100644--- a/packages/studio/src/parameters.ts+++ b/packages/studio/src/parameters.ts@@ -5,7 +5,6 @@ import type {StudioInspectorFieldDeclaration,StudioSourceBinding,StudioSourceBindingDeclaration,- StudioParameterControl,StudioRecipeReferenceBindingDeclaration,StudioEntityDraft,StudioEditSource,@@ -17,6 +16,11 @@ import type {StudioTimelineGesture,} from "@hypit/studio-adapter";import { parseSvs } from "@hypit/svs";+import { parseOpeningTag } from "@hypit/markup";+import { parameterControlForSchema, parameterRecordSchema } from "./parameter-values.js";+import type { CanonicalValue } from "@hypit/protocol";++import type { StudioCompanionRegistry } from "./studio-registry.js";import type { Range } from "./shared.js";import type { Placement } from "./observe.js";@@ -30,6 +34,7 @@ export type StudioSourceFile = {};type AuthorElement = {+ readonly range: Range;readonly authorElement?: string;readonly authorEndpoints: Readonly<Record<string, string>>;readonly id?: string;@@ -48,6 +53,7 @@ function authoredElements(placements: readonly Placement[]): readonly AuthorElem...(placement.authorElement === undefined ? {} : { authorElement: placement.authorElement }),authorEndpoints: placement.authorEndpoints ?? {},...(placement.id === undefined ? {} : { id: placement.id }),+ range: placement.range,sourcePath: placement.sourcePath,attributes: placement.attributes,references: placement.referenceAttributes,@@ -60,6 +66,7 @@ function authoredElements(placements: readonly Placement[]): readonly AuthorElem...(child.authorElement === undefined ? {} : { authorElement: child.authorElement }),authorEndpoints: child.authorEndpoints ?? {},...(child.id === undefined ? {} : { id: child.id }),+ range: child.range,sourcePath: child.sourcePath,attributes: child.attributes,references: child.referenceAttributes,@@ -175,7 +182,8 @@ function recipeParameters(input: {return input.recipe.bindings.flatMap((declaration): readonly StudioSourceBinding[] => {const property = recipe.properties.find((candidate) => candidate.name === declaration.name);if (property === undefined) {- if (declaration.fallback === undefined) return [];+ const fallback = typeof declaration.fallback === "function" ? declaration.fallback(recipe.value.properties) : declaration.fallback;+ if (fallback === undefined) return [];const close = recipe.range.end - 1;const lineStart = Math.max(source.text.lastIndexOf("\n", close - 1), source.text.lastIndexOf("\r", close - 1)) + 1;const closeIndent = source.text.slice(lineStart, close);@@ -189,7 +197,7 @@ function recipeParameters(input: {id: `${input.draft.id}:${input.referenceName}:${declaration.name}`,binding: `${input.referenceName}.${declaration.name}`,name: declaration.name,- value: declaration.fallback,+ value: fallback,...(declaration.schema === undefined ? {} : { schema: declaration.schema }),language: "svs" as const,writable: declaration.writable ?? true,@@ -220,6 +228,57 @@ function recipeParameters(input: {});}+function attributeGroup(file: StudioSourceFile, target: AuthorElement, draft: StudioEntityDraft,+ declaration: StudioSourceBindingDeclaration, binding: string, root: string): readonly StudioSourceBinding[] {+ if (declaration.schema === undefined) throw new Error(`Attribute group ${binding} needs a schema.`);+ const tag = parseOpeningTag({ name: file.path, text: file.text }, target.range.start);+ const sourceRange = { start: target.range.start, end: tag.end };+ const preimage = file.text.slice(sourceRange.start, sourceRange.end);+ const opening = /^<[\w:.-]+/u.exec(preimage);+ if (!opening) throw new Error(`Cannot locate opening tag for ${binding}.`);+ const fallback = typeof declaration.fallback === "function" ? declaration.fallback(target.attributes) : declaration.fallback;+ const value: Record<string, CanonicalValue> = fallback !== null && typeof fallback === "object" && !Array.isArray(fallback) ? { ...(fallback as Readonly<Record<string, CanonicalValue>>) } : {};+ const ranges: Record<string, Range | null> = {};+ let writable = declaration.writable === true;+ for (const name of declaration.attributes!) {+ const range = target.attributeValueRanges[name];+ ranges[name] = null;+ if (range === undefined) continue;+ if (target.references[name] !== undefined) { writable = false; value[name] = target.references[name]!; continue; }+ value[name] = target.attributes[name] ?? target.references[name] ?? "";+ const before = file.text.slice(target.range.start, range.start);+ const escaped = name.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");+ const prefix = new RegExp(`\\s+${escaped}\\s*=\\s*["']$`, "u").exec(before);+ if (!prefix) throw new Error(`Cannot locate scalar attribute ${name}.`);+ ranges[name] = { start: prefix.index, end: range.end - target.range.start + 1 };+ }+ const schema = parameterRecordSchema(declaration.schema, value);+ for (const [name, held] of Object.entries(value)) {+ if (target.references[name] !== undefined || typeof held !== "string") continue;+ const kind = schema?.fields[name]?.schema.kind;+ if (kind === "number" && held.trim() !== "") value[name] = Number(held);+ if (kind === "boolean" && (held === "true" || held === "false")) value[name] = held === "true";+ }+ return [{ id: `${draft.id}:${binding}`, binding, name: declaration.name, value, schema: declaration.schema,+ language: file.language, writable, source: { path: relative(root, sourceAbsolute(root, target.sourcePath)),+ range: sourceRange, preimage }, attributes: { insertionOffset: opening[0].length, ranges } }];+}++function omittedAttribute(file: StudioSourceFile, target: AuthorElement, draft: StudioEntityDraft,+ declaration: StudioSourceBindingDeclaration, binding: string, root: string): readonly StudioSourceBinding[] {+ const value = typeof declaration.fallback === "function" ? declaration.fallback(target.attributes) : declaration.fallback;+ if (value === undefined) return [];+ const tag = /^<[\w:.-]+/u.exec(file.text.slice(target.range.start));+ if (tag === null) throw new Error(`Cannot locate opening tag for ${binding}.`);+ const at = target.range.start + tag[0].length;+ return [{ id: `${draft.id}:${binding}`, binding, name: declaration.name, value,+ ...(declaration.schema === undefined ? {} : { schema: declaration.schema }),+ language: file.language, writable: declaration.writable === true,+ source: { path: relative(root, sourceAbsolute(root, target.sourcePath)),+ range: { start: at, end: at }, preimage: "", prefix: ` ${declaration.name}="`, suffix: '"' },+ }];+}+function referencedParameters(input: {readonly root: string;readonly files: readonly StudioSourceFile[];@@ -238,16 +297,25 @@ function referencedParameters(input: {const file = sourceFor(input.root, target.sourcePath, input.files);if (file === undefined) return [];return input.declarations.flatMap((declaration) => {+ if (declaration.attributes) return attributeGroup(file, target, input.draft, declaration, `${input.referenceName}.${declaration.name}`, input.root);const nestedPath = target.references[declaration.name];- const nested = nestedPath === undefined || declaration.referenced === undefined ? [] : referencedParameters({+ const referenced = nestedPath === undefined || declaration.referenced === undefined ? [] : referencedParameters({root: input.root, files: input.files, draft: input.draft, placements: input.placements,referenceName: `${input.referenceName}.${declaration.name}`,referencePath: nestedPath,...(target.resolvedReferences[declaration.name] === undefined ? {} : { referenceRef: target.resolvedReferences[declaration.name] }),declarations: declaration.referenced,});+ const recipe = nestedPath === undefined || declaration.recipe === undefined ? [] : recipeParameters({+ root: input.root, files: input.files, draft: input.draft, placements: input.placements,+ current: file, referenceName: `${input.referenceName}.${declaration.name}`,+ referencePath: nestedPath,+ ...(target.resolvedReferences[declaration.name] === undefined ? {} : { referenceRef: target.resolvedReferences[declaration.name] }), recipe: declaration.recipe,+ through: declaration.recipe.through ?? [],+ });+ const nested = [...referenced, ...recipe];const range = target.attributeValueRanges[declaration.name];- if (range === undefined) return nested;+ if (range === undefined) return [...omittedAttribute(file, target, input.draft, declaration, `${input.referenceName}.${declaration.name}`, input.root), ...nested];const preimage = file.text.slice(range.start, range.end);const reference = target.references[declaration.name];const value = reference === undefined ? (target.attributes[declaration.name] ?? preimage) : reference;@@ -288,6 +356,37 @@ function parameterReference(return ref === undefined ? { path } : { path, ref };}+/** Resolve only references opted into by the consuming Companion. */+export function composeParameterDeclarations(input: {+ readonly placement: StudioPlacement | undefined;+ readonly draft: StudioEntityDraft;+ readonly placements: readonly Placement[];+ readonly registry: StudioCompanionRegistry;+ readonly bindings: readonly StudioSourceBindingDeclaration[];+ readonly inspector: readonly StudioInspectorFieldDeclaration[];+}): { bindings: readonly StudioSourceBindingDeclaration[]; inspector: readonly StudioInspectorFieldDeclaration[] } {+ const inspector = [...input.inspector];+ const element = input.placement === undefined ? undefined : elementFor(input.placement, input.draft);+ const expand = (declarations: readonly StudioSourceBindingDeclaration[], owner: AuthorElement | undefined,+ prefix: string, seen: ReadonlySet<string>): readonly StudioSourceBindingDeclaration[] => declarations.map(declaration => {+ const path = prefix ? `${prefix}.${declaration.name}` : declaration.name;+ const reference = prefix ? owner?.references[declaration.name] : input.draft.parameterReferences?.[declaration.name] ?? owner?.references[declaration.name];+ const resolved = !prefix && input.draft.parameterReferences?.[declaration.name] !== undefined ? undefined : owner?.resolvedReferences[declaration.name];+ const target = input.placements.find(candidate => resolved !== undefined+ ? candidate.records.includes(resolved) || candidate.outputs.includes(resolved)+ : candidate.id === reference && candidate.sourcePath === owner?.sourcePath);+ const companion = declaration.companion && target !== undefined+ ? input.registry.parameterCompanionFor(target.module, target.surface) : undefined;+ const key = target?.authorElement ?? `${target?.sourcePath}:${target?.range.start}`;+ if (companion !== undefined && seen.has(key)) throw new Error(`Circular Studio parameter reference at ${path}.`);+ if (companion !== undefined) inspector.push(...companion.inspector.map(field => ({ ...field, binding: `${path}.${field.binding}` })));+ const next = target === undefined ? undefined : elementFor(target, { ...input.draft, authoredId: target.id ?? "", elementRange: target.range });+ const nested = [...(declaration.referenced ?? []), ...(companion?.bindings ?? [])];+ return { ...declaration, referenced: expand(nested, next, path, new Set([...seen, key])) };+ });+ return { bindings: expand(input.bindings, element, "", new Set()), inspector };+}+/*** Expose only attributes a package explicitly registered. The source range is* still discovered by the generic markup frontend, while the meaning and@@ -310,8 +409,9 @@ export function sourceBindingsForDraft(input: {const direct = input.declarations.flatMap((declaration) => {const name = declaration.name;+ if (declaration.attributes) return attributeGroup(file, element, input.draft, declaration, name, input.root);const range = element.attributeValueRanges[name];- if (range === undefined) return [];+ if (range === undefined) return omittedAttribute(file, element, input.draft, declaration, name, input.root);const preimage = file.text.slice(range.start, range.end);const reference = element.references[name];const value = reference === undefined@@ -372,29 +472,17 @@ export function sourceBindingsForDraft(input: {return [...direct, ...recipes];}-function controlForSchema(schema: StudioSourceBinding["schema"]): StudioParameterControl | undefined {- if (schema === undefined) return undefined;- if (schema.kind === "boolean") return "boolean";- if (schema.kind === "number") return "number";- if (schema.kind === "string") return schema.enum === undefined- ? schema.format === "color" ? "color" : "text"- : "select";- if (schema.kind === "array") return "list";- if (schema.kind === "object") return "record";- return undefined;-}--/** Resolve the Companion's visible field table against real writable bindings. */+/** Resolve the Companion's visible field table against real bindings and entity facts. */export function inspectorFieldsForBindings(draft: StudioEntityDraft,bindings: readonly StudioSourceBinding[],declarations: readonly StudioInspectorFieldDeclaration[],): readonly StudioInspectorField[] {const byBinding = new Map(bindings.map((binding) => [binding.binding, binding] as const));- return declarations.flatMap((declaration): readonly StudioInspectorField[] => {+ const fields = declarations.flatMap((declaration): readonly StudioInspectorField[] => {const binding = byBinding.get(declaration.binding);- if (binding === undefined || !binding.writable) return [];- const control = declaration.control ?? controlForSchema(binding.schema);+ if (binding === undefined) return [];+ const control = declaration.control ?? (binding.attributes ? "record" : parameterControlForSchema(binding.schema));if (control === undefined) {throw new Error(`Studio Inspector binding ${declaration.binding} has neither a control nor a supported public schema.`);}@@ -408,10 +496,12 @@ export function inspectorFieldsForBindings(...(declaration.options !== undefined || schemaOptions === undefined? {}: { options: schemaOptions }),- language: binding.language,- source: binding.source,+ ...(binding.writable ? { edit: { language: binding.language, source: binding.source, ...(binding.attributes ? { attributes: binding.attributes } : {}) } } : {}),}];});+ return [...fields, ...(draft.inspector ?? []).map((fact): StudioInspectorField => ({+ ...fact, id: `${draft.id}:inspector:fact:${fact.id}`, control: "text",+ }))];}/** Runtime authority, rather than a Companion allowlist, makes timing fields writable. */diff --git a/packages/studio/src/preview/render.ts b/packages/studio/src/preview/render.tsindex 8f0f7df3..37c0263f 100644--- a/packages/studio/src/preview/render.ts+++ b/packages/studio/src/preview/render.ts@@ -45,7 +45,11 @@ export function renderPreview(input: RenderInput): string {data-loop="${clip.source.loop}"data-phase="${clip.source.phaseSample / 48_000}"data-playback-rate="${clip.playbackRate}"- data-gain="${clip.gain}"></audio>`;+ data-gain="${clip.gain}"+ data-presentation="${encodeURIComponent(JSON.stringify({+ gainEnvelope: clip.gainEnvelope, audibility: clip.audibility,+ fadeInSamples: clip.fadeInSamples, fadeOutSamples: clip.fadeOutSamples,+ }))}"></audio>`;}).join("");return injectRuntimeShim(html, audio);}diff --git a/packages/studio/src/preview/runtime-shim.ts b/packages/studio/src/preview/runtime-shim.tsindex 31f31372..877da92b 100644--- a/packages/studio/src/preview/runtime-shim.ts+++ b/packages/studio/src/preview/runtime-shim.ts@@ -1,3 +1,5 @@+import { audioEnvelopeGainAt } from "@hypit/composition";+/*** Studio's frame driver for a compiled HyperFrames document.*@@ -48,6 +50,7 @@ function shim(): string {rate: parseFloat(element.getAttribute('data-playback-rate') || '1') || 1});}+ var audioContext;var programmeAudio = [];for (var element of document.querySelectorAll('.hypit-studio-audio')) {programmeAudio.push({@@ -59,10 +62,51 @@ function shim(): string {loop: element.getAttribute('data-loop') === 'true',phase: parseFloat(element.getAttribute('data-phase') || '0') || 0,rate: parseFloat(element.getAttribute('data-playback-rate') || '1') || 1,- gain: parseFloat(element.getAttribute('data-gain') || '1') || 1+ gain: Number(element.getAttribute('data-gain') ?? '1'),+ presentation: JSON.parse(decodeURIComponent(element.getAttribute('data-presentation') || '%7B%7D'))});}+ function scheduleEnvelope(param, points, sample, now) {+ param.cancelScheduledValues(now);+ param.setValueAtTime(audioEnvelopeGainAt(points, sample), now);+ for (var point of points || []) {+ if (point.sample > sample) param.linearRampToValueAtTime(point.gain, now + (point.sample - sample) / 48000);+ }+ }++ // Each factor has its own GainNode so intersecting ramps multiply exactly.+ // These are the same absolute-sample envelopes consumed by the render mixer.+ function placeAudioGain(record, seconds) {+ if (!record.nodes) {+ var source = audioContext.createMediaElementSource(record.element);+ record.nodes = Array.from({ length: 5 }, function () { return audioContext.createGain(); });+ var previous = source;+ for (var node of record.nodes) { previous.connect(node); previous = node; }+ previous.connect(audioContext.destination);+ record.element.volume = 1;+ }+ var sample = seconds * 48000;+ var now = audioContext.currentTime;+ var p = record.presentation;+ var start = record.start * 48000, end = (record.start + record.duration) * 48000;+ record.nodes[0].gain.setValueAtTime(record.gain, now);+ scheduleEnvelope(record.nodes[1].gain, p.gainEnvelope, sample, now);+ scheduleEnvelope(record.nodes[2].gain, p.fadeInSamples > 0+ ? [{ sample: start, gain: 0 }, { sample: start + p.fadeInSamples, gain: 1 }] : undefined, sample, now);+ scheduleEnvelope(record.nodes[3].gain, p.fadeOutSamples > 0+ ? [{ sample: end - p.fadeOutSamples, gain: 1 }, { sample: end, gain: 0 }] : undefined, sample, now);+ var gate = record.nodes[4].gain;+ gate.cancelScheduledValues(now);+ gate.setValueAtTime(p.audibility === undefined || p.audibility.some(function (span) {+ return sample >= span.startSample && sample < span.endSampleExclusive;+ }) ? 1 : 0, now);+ for (var span of p.audibility || []) {+ if (span.startSample > sample) gate.setValueAtTime(1, now + (span.startSample - sample) / 48000);+ if (span.endSampleExclusive > sample) gate.setValueAtTime(0, now + (span.endSampleExclusive - sample) / 48000);+ }+ }+function seekDecoded(element, target) {if (Math.abs(element.currentTime - target) < 0.0005 && element.readyState >= 2) return Promise.resolve();return new Promise(function (resolve) {@@ -148,7 +192,7 @@ function shim(): string {} else {if (Math.abs(element.currentTime - target) > 0.08) element.currentTime = target;element.playbackRate = record.rate;- element.volume = Math.max(0, Math.min(1, record.gain));+ placeAudioGain(record, currentSeconds + frameSeconds / 2);if (element.paused) {var started = element.play();if (started) started.catch(function () {});@@ -169,6 +213,8 @@ function shim(): string {};window.__hypitPlayFrame = function (frame) {playing = true;+ if (programmeAudio.length && !audioContext) audioContext = new AudioContext();+ if (audioContext && audioContext.state === 'suspended') audioContext.resume();return apply(frame / fps, false);};window.__hypitFrameReady = apply(0, true);@@ -178,7 +224,7 @@ function shim(): string {}export function injectRuntimeShim(html: string, audio = ""): string {- const script = audio + shim();+ const script = audio + shim().replace("(function () {", `(function () {\n${audioEnvelopeGainAt.toString()}`);const at = html.lastIndexOf("</body>");return at < 0 ? html + script : html.slice(0, at) + script + html.slice(at);}diff --git a/packages/studio/src/programme.ts b/packages/studio/src/programme.tsindex e14e9c33..2be0fb99 100644--- a/packages/studio/src/programme.ts+++ b/packages/studio/src/programme.ts@@ -4,13 +4,13 @@ import { sameType } from "@hypit/protocol";import type { Composition } from "@hypit/composition";import { compositionTypes } from "@hypit/composition";import {- projectSemanticProgramSpace,+ projectTimelineSpace,semanticAnchorFrames,- semanticTrackSpans,-} from "@hypit/semantic-track";-import type { SemanticTrack } from "@hypit/semantic-track";-import { semanticTrackTypes } from "@hypit/semantic-track";-import { assertProgramSpaceIdentity, programSpaceTypes } from "@hypit/program-space";+ timelineSpans,+} from "@hypit/timeline";+import type { Timeline } from "@hypit/timeline";+import { timelineTypes } from "@hypit/timeline";+import { assertProgramSpaceIdentity } from "@hypit/program-space";import type { ProgramSpace } from "@hypit/program-space";import type { StudioResolvedTrack, StudioTemporalBinding } from "@hypit/studio-adapter";import type { RuntimeHostTransientExecution } from "@hypit/runtime-host-node";@@ -36,7 +36,6 @@ export type Preview = {readonly values: ReadonlyMap<string, unknown>;readonly temporalBindings: ReadonlyMap<string, readonly StudioTemporalBinding[]>;readonly composition: Composition;- readonly timing: "measured" | "authored";readonly narrativeId?: string;readonly timingOutput?: { readonly name: string; readonly ref: string };readonly timingCandidateId?: string;@@ -173,11 +172,11 @@ export async function preview(input: {const timingType = timingOutput?.typeRef ?? timingRecord?.type;const timingValue = selectedValue(executed.state, input.timeRef);if (timingValue?.kind !== "inline" || timingType === undefined) throw new Error("Studio requires a resolved film time source.");- const semantic = sameType(timingType, semanticTrackTypes.track) ? timingValue.value as unknown as SemanticTrack : undefined;- if (semantic === undefined && !sameType(timingType, programSpaceTypes.programSpace)) throw new Error("Unsupported Studio time source.");- const spans = semantic === undefined ? [] : semanticTrackSpans(semantic);- const anchors = semantic === undefined ? new Map<string, number>() : semanticAnchorFrames(semantic);- const space = semantic === undefined ? timingValue.value as unknown as ProgramSpace : projectSemanticProgramSpace(semantic);+ if (!sameType(timingType, timelineTypes.track)) throw new Error("Studio requires the declared Timeline.");+ const semantic = timingValue.value as unknown as Timeline;+ const spans = timelineSpans(semantic);+ const anchors = semanticAnchorFrames(semantic);+ const space = projectTimelineSpace(semantic);assertProgramSpaceIdentity(space);const rate = space.frameRate;const compositionValue = selectedValue(executed.state, input.compositionRef);@@ -227,22 +226,8 @@ export async function preview(input: {const stored = selectedValue(executed.state, target.ref);if (stored?.kind === "inline") values.set(target.ref, stored.value);}- // Author Records are already deterministic inline values from this exact- // compilation. Studio Track Companions may need them to name a resolved projection- // (for example Caption cue text); exposing them here avoids recomputing the- // domain value or turning a Record into a fake graph target.- for (const placement of input.source.observations.placements) {- for (const value of placement.values) {- const suffixes = [`::record::${value.id}`, `::output::${value.id}`];- for (const reference of input.projections.flatMap((projection) => projection.trace.references)) {- if (suffixes.some((suffix) => reference.ref.endsWith(suffix))) values.set(reference.ref, value.value);- }- }- }- // Script is a raw Surface, so its inline Records are not among the structured- // element observations above. They are still ordinary Author values from- // the same compilation and are needed to turn Caption atom ids back into the- // text the author actually sees.+ // Author records from this compilation already carry exact qualified references.+ // Expose referenced inline values to Companions without guessing their owner by suffix.for (const record of input.source.compiled.program.records) {if (record.value.kind !== "inline") continue;if (input.projections.some((projection) => projection.trace.references@@ -257,8 +242,7 @@ export async function preview(input: {values,temporalBindings,composition,- timing: semantic === undefined ? "authored" : "measured",- ...(semantic === undefined ? {} : { narrativeId: semantic.narrativeId }),+ ...(semantic.narrativeId === undefined ? {} : { narrativeId: semantic.narrativeId }),timingOutput: { name: timingOutput?.name ?? space.id, ref: input.timeRef },...(timingCandidateId === undefined ? {} : { timingCandidateId }),timingCandidateOrigin: timingCandidateId === undefined ? "source" : "run",diff --git a/packages/studio/src/server.ts b/packages/studio/src/server.tsindex a624945d..e84bfbd5 100644--- a/packages/studio/src/server.ts+++ b/packages/studio/src/server.ts@@ -15,7 +15,7 @@ import type { ServedFile } from "./compile.js";import type { StudioDomain } from "./domain.js";import type { StudioCompanionRegistry } from "./studio-registry.js";import { loadStudioRun } from "./run.js";-import { parameterAuthorValue, parameterOption, serializeParameterValue, validateParameterValue } from "./parameter-values.js";+import { parameterAuthorValue, parameterOption, serializeParameterValue, serializeAttributeGroup, validateParameterValue } from "./parameter-values.js";import { readStudioSession } from "./session.js";import type { Range, StudioFailure, StudioLibraryRequest, StudioLibraryView, StudioMutation, StudioSnapshot } from "./shared.js";import { createStudioStoryboard } from "./storyboard.js";@@ -402,7 +402,7 @@ export function studioPlugin(options: StudioPluginOptions): Plugin {if (mutation.type === "timeline.adjust") return timelinePatches(mutation);const clip = currentClip(mutation.entityId);const parameter = clip.inspector.find((candidate) => candidate.id === mutation.parameterId);- if (parameter === undefined) {+ if (parameter?.edit === undefined) {throw new Error(`Entity ${mutation.entityId} has no writable parameter ${mutation.parameterId}.`);}const authorValue = parameterAuthorValue(parameter, mutation.value);@@ -420,10 +420,14 @@ export function studioPlugin(options: StudioPluginOptions): Plugin {throw new Error(`${parameter.label} does not accept ${mutation.value}.`);}if (parameter.control === "color") validateParameterValue(authorValue, { kind: "string", format: "color" }, parameter.label);- const encoded = serializeParameterValue(authorValue, parameter.language);- const replacement = `${parameter.source.prefix ?? ""}${encoded}${parameter.source.suffix ?? ""}`;- return replacement === parameter.source.preimage ? [] : [{- ...parameter.source,+ if (parameter.edit.attributes) {+ const replacement = serializeAttributeGroup(parameter, authorValue);+ return replacement === parameter.edit.source.preimage ? [] : [{ ...parameter.edit.source, replacement }];+ }+ const encoded = serializeParameterValue(authorValue, parameter.edit.language);+ const replacement = `${parameter.edit.source.prefix ?? ""}${encoded}${parameter.edit.source.suffix ?? ""}`;+ return replacement === parameter.edit.source.preimage ? [] : [{+ ...parameter.edit.source,replacement,}];};diff --git a/packages/studio/src/shared.ts b/packages/studio/src/shared.tsindex 3148c6d8..5155147d 100644--- a/packages/studio/src/shared.ts+++ b/packages/studio/src/shared.ts@@ -9,6 +9,7 @@ import type {StudioIcon,StudioEditHandle,StudioLaneDescription,+ StudioTrackBand,StudioMaterialPreview,StudioInspectorField,StudioInspectorDomain,@@ -63,6 +64,7 @@ export type StudioTrackBinding = {readonly companion: string;/** Studio-local partition key for an attached projection. */readonly attachmentId?: string;+ readonly bands?: readonly Omit<StudioTrackBand, "bindings" | "inspector">[];readonly lane: StudioLaneDescription;readonly authoredTag?: string;readonly references: readonly { readonly name: string; readonly type: string }[];@@ -95,6 +97,8 @@ export type Clip = {readonly presentId?: string;/** The authored id this Present is named after, when it names one. */readonly authoredId: string;+ readonly selectionGroup?: string;+ readonly band?: string;/** The Script marker that placed it, when something said put it there. */readonly markerId?: string;readonly display: StudioEntityDisplay;@@ -281,8 +285,6 @@ export type StudioSnapshot = {/** The Tracks, compiled into the document the renderer photographs. */readonly preview: { readonly kind: "hyperframes"; readonly srcdoc: string };readonly provenance: {- /** Where the timeline came from. The frame domain always shares its source. */- readonly timing: "measured" | "authored";readonly picture: "resolved";/** What the badges above are standing for, in one sentence. */readonly note: string;diff --git a/packages/studio/src/snapshot.ts b/packages/studio/src/snapshot.tsindex c23c3f90..25178cc0 100644--- a/packages/studio/src/snapshot.ts+++ b/packages/studio/src/snapshot.ts@@ -1,16 +1,15 @@+import { composeParameterDeclarations } from "./parameters.js";/*** Turn a built programme into what the panels read.*- * A timeline is the Presents of every Track, and a Present carries a span, an- * identity and a stacking order - the whole contract, for every package. So the- * timeline is assembled without knowing what made any of it, and a package that- * grows a new Track shows up without this file changing.+ * The Timeline owns the work range. Companions project selectable entities onto+ * that range without extending it or requiring component-specific editor code.*/import { relative } from "node:path";import type { MarkupSurfaceRegistryLike } from "@hypit/markup";import { compositionTypes } from "@hypit/composition";-import { narrativeTypes } from "@hypit/narrative";+import { programSpaceFrameCount } from "@hypit/program-space";import { sameModule, sameType } from "@hypit/protocol";import type {@@ -138,43 +137,10 @@ function scriptMap(};}-type NarrativeValue = {- readonly id?: string;- readonly segments?: readonly {- readonly id: string;- readonly startAnchorId: string;- readonly endAnchorId: string;- }[];- readonly tokens?: readonly {- readonly id: string;- readonly segmentId: string;- readonly startAnchorId: string;- readonly endAnchorId: string;- readonly text: string;- }[];- readonly selections?: readonly {- readonly id: string;- readonly startAnchorId: string;- readonly endAnchorId: string;- }[];- readonly moments?: readonly {- readonly id: string;- readonly anchorId: string;- }[];- readonly semanticIndex?: {- readonly anchors?: readonly {- readonly id: string;- readonly kind: "program-start" | "segment-start" | "segment-end" | "token-start" | "token-end" | "program-end";- readonly segmentId?: string;- readonly tokenId?: string;- }[];- };-};-/*** Project the compiled Narrative into the frame domain that the preview is* already using. No frontend timing is invented here: if an anchor is absent- * from the built SemanticTrack, the corresponding item is simply not drawable yet.+ * from the built Timeline, the corresponding item is simply not drawable yet.*/function semanticTimeline(registry: StudioCompanionRegistry,@@ -182,77 +148,13 @@ function semanticTimeline(script: ScriptMap | undefined,): SemanticTimeline | undefined {if (built.narrativeId === undefined) return undefined;- const records = built.source.compiled.program.records.filter((item) =>- sameType(item.type, narrativeTypes.narrative)- && item.value.kind === "inline"- && (item.value.value as NarrativeValue).id === built.narrativeId);- if (records.length > 1) {- throw new Error(`Studio Narrative id ${built.narrativeId} resolves to more than one authored value.`);- }- const record = records[0];- const narrative = record?.value.kind === "inline"- ? record.value.value as NarrativeValue- : undefined;- if (narrative === undefined) throw new Error("Studio Narrative is not an inline authored value.");-- const segmentRanges = new Map((script?.segments ?? []).map((item) => [item.id, item.range]));- const tokenRanges = new Map((script?.tokens ?? []).map((item) => [item.id, item.range]));- const frame = (id: string): number | undefined => built.anchors.get(id);-- const segments = (narrative.segments ?? []).flatMap((segment) => {- const startFrame = frame(segment.startAnchorId);- const endFrame = frame(segment.endAnchorId);- if (startFrame === undefined || endFrame === undefined) return [];- return [{- id: segment.id,- startFrame,- endFrameExclusive: Math.max(startFrame + 1, endFrame),- ...(segmentRanges.has(segment.id) ? { range: segmentRanges.get(segment.id)! } : {}),- }];+ if (script === undefined) throw new Error("Timeline's Script has no Studio source mapping.");+ const projected = registry.projectScript({ source: script, anchors: built.anchors,+ values: built.source.compiled.program.records.flatMap(record => record.value.kind === "inline"+ ? [{ id: record.id, type: record.type, value: record.value.value }] : []),});- const tokens = (narrative.tokens ?? []).flatMap((token) => {- const startFrame = frame(token.startAnchorId);- const endFrame = frame(token.endAnchorId);- if (startFrame === undefined || endFrame === undefined) return [];- return [{- id: token.id,- segmentId: token.segmentId,- text: token.text,- startFrame,- endFrameExclusive: Math.max(startFrame + 1, endFrame),- ...(tokenRanges.has(token.id) ? { range: tokenRanges.get(token.id)! } : {}),- }];- });- const anchors = (narrative.semanticIndex?.anchors ?? []).flatMap((anchor) => {- const at = frame(anchor.id);- if (at === undefined) return [];- return [{- id: anchor.id,- kind: anchor.kind,- frame: at,- ...(anchor.segmentId === undefined ? {} : { segmentId: anchor.segmentId }),- ...(anchor.tokenId === undefined ? {} : { tokenId: anchor.tokenId }),- }];- });- const selections = (narrative.selections ?? []).flatMap((selection) => {- const startFrame = frame(selection.startAnchorId);- const endFrameExclusive = frame(selection.endAnchorId);- if (startFrame === undefined || endFrameExclusive === undefined || endFrameExclusive <= startFrame) return [];- return [{- id: selection.id,- startAnchorId: selection.startAnchorId,- endAnchorId: selection.endAnchorId,- startFrame,- endFrameExclusive,- }];- });- const moments = (narrative.moments ?? []).flatMap((moment) => {- const at = frame(moment.anchorId);- return at === undefined ? [] : [{ id: moment.id, anchorId: moment.anchorId, frame: at }];- });- if (segments.length === 0) throw new Error("Studio SemanticTrack resolves no authored Segment anchors.");const provenance: CandidateProvenance = {- output: built.timingOutput?.name ?? "SemanticTrack",+ output: built.timingOutput?.name ?? "Timeline",...(built.timingOutput?.ref === undefined ? {} : { outputRef: built.timingOutput.ref }),...(built.timingCandidateId === undefined ? {} : { candidateId: built.timingCandidateId }),origin: built.timingCandidateOrigin,@@ -262,17 +164,10 @@ function semanticTimeline(return {spaceId: built.space.id,narrativeId: built.narrativeId,- // Semantic is a Studio lane with its own registered meaning. Do not copy- // the authored Speech Track id into this label: it is the timebase, not a- // second Speech output.+ // The generic ruler presentation is independent of the authored Timeline id.presentation: registry.semanticTimelinePresentation(),- // Narrative order is the semantic ruler. Frame ties are common and must- // not erase the discrete 2M+2N+2 anchor ordering used by writeback.- anchors,- segments: segments.sort((left, right) => left.startFrame - right.startFrame || left.id.localeCompare(right.id)),- tokens: tokens.sort((left, right) => left.startFrame - right.startFrame || left.id.localeCompare(right.id)),- selections: selections.sort((left, right) => left.startFrame - right.startFrame || left.id.localeCompare(right.id)),- moments: moments.sort((left, right) => left.frame - right.frame || left.id.localeCompare(right.id)),+ // Preserve the Companion's ordered anchors, including coincident boundaries.+ ...projected,provenance,};}@@ -338,13 +233,18 @@ export function snapshot(registry: StudioCompanionRegistry, built: Preview, inpusemantic,generic,}).map((draft) => {+ const declarations = composeParameterDeclarations({+ placement, draft, placements: built.source.observations.placements, registry,+ bindings: registry.bindingDeclarations(item, placement, draft.lane, draft.band),+ inspector: registry.inspectorDeclarations(item, placement, draft.lane, draft.band),+ });const bindings = sourceBindingsForDraft({root: input.workspaceRoot,files: input.sourceFiles,placement,draft,declarations: [- ...registry.bindingDeclarations(item, placement, draft.lane),+ ...declarations.bindings,...temporalBindingDeclarations(draft.temporal),],placements: built.source.observations.placements,@@ -352,7 +252,7 @@ export function snapshot(registry: StudioCompanionRegistry, built: Preview, inpuconst inspector = inspectorFieldsForBindings(draft,bindings,- registry.inspectorDeclarations(item, placement, draft.lane),+ declarations.inspector,);const editHandles = resolveTimelineEditHandles(bindings,@@ -404,17 +304,7 @@ export function snapshot(registry: StudioCompanionRegistry, built: Preview, inpu// root that produced them in this list.const rows = tracks.map((track, row) => ({ ...track, row }));- const declared = (built.space as { durationSec?: number; frameRate?: { numerator: number; denominator: number } })- ?.durationSec;- const frameCount = Math.max(- 1,- declared === undefined- ? 0- : Math.round(declared * input.frameRate.numerator / input.frameRate.denominator),- ...rows.flatMap((track) => track.clips.map((clip) => clip.endFrameExclusive)),- ...(semantic?.segments ?? []).map((segment) => segment.endFrameExclusive),- ...(semantic?.tokens ?? []).map((token) => token.endFrameExclusive),- );+ const frameCount = programSpaceFrameCount(built.space);return {revision: input.revision,source: {@@ -442,7 +332,6 @@ export function snapshot(registry: StudioCompanionRegistry, built: Preview, inpu...(semantic === undefined ? {} : { semantic }),preview: input.preview,provenance: {- timing: built.timing,picture: "resolved",note: note(built),},diff --git a/packages/studio/src/studio-preflight.ts b/packages/studio/src/studio-preflight.tsindex bef6a443..81220650 100644--- a/packages/studio/src/studio-preflight.ts+++ b/packages/studio/src/studio-preflight.ts@@ -1,7 +1,7 @@import type { Candidate } from "@hypit/protocol";import { sameType } from "@hypit/protocol";import type { StudioFilmCompanion } from "@hypit/studio-adapter";-import { semanticTrackTypes } from "@hypit/semantic-track";+import { timelineTypes } from "@hypit/timeline";import type { CompiledSource } from "./compile.js";import type { Placement } from "./observe.js";@@ -137,7 +137,7 @@ export function inspectStudioRun(});const timeOutput = times[0];if (times.length !== 1) issues.push("Film requires one traceable time source.");- const semanticOutput = timeOutput !== undefined && sameType(timeOutput.typeRef, semanticTrackTypes.track) ? timeOutput : undefined;+ const semanticOutput = timeOutput !== undefined && sameType(timeOutput.typeRef, timelineTypes.track) ? timeOutput : undefined;let companionValueRefs: readonly string[] = [];try {companionValueRefs = unique(filmTrackRefs.flatMap((ref) =>@@ -149,7 +149,7 @@ export function inspectStudioRun(const projectionRefs = unique([...filmTrackRefs,- ...(semanticOutput !== undefined && sameType(semanticOutput.typeRef, semanticTrackTypes.track) ? [semanticOutput.ref] : []),+ ...(semanticOutput !== undefined && sameType(semanticOutput.typeRef, timelineTypes.track) ? [semanticOutput.ref] : []),...companionValueRefs,]);const projections: StudioViewRequirement[] = [];diff --git a/packages/studio/src/studio-registry.ts b/packages/studio/src/studio-registry.tsindex 320619cf..d554a85b 100644--- a/packages/studio/src/studio-registry.ts+++ b/packages/studio/src/studio-registry.ts@@ -1,5 +1,6 @@import type {StudioTrackCompanion,+ StudioParameterCompanion,StudioTrackCompanionContext,StudioEditHandle,StudioEntityDraft,@@ -8,6 +9,7 @@ import type {StudioViewRole,StudioResolvedTrack,StudioScriptAdjustment,+ StudioScriptProjectionInput,StudioScriptCompanion,StudioScriptSourceMap,StudioSourceBindingDeclaration,@@ -18,7 +20,7 @@ import { parameterOption } from "./parameter-values.js";import { compositionTypes } from "@hypit/composition";import { sameModule, sameType } from "@hypit/protocol";import type { ModuleRef, TypeRef } from "@hypit/protocol";-import { semanticTrackTypes } from "@hypit/semantic-track";+import { timelineTypes } from "@hypit/timeline";import type { Placement } from "./observe.js";import type { Clip, StudioTrackBinding } from "./shared.js";@@ -29,9 +31,9 @@ const flatLane = {heightPx: 52,};-const tones = new Set(["blue", "green", "teal", "violet", "magenta", "orange", "orange-muted", "neutral"]);+const tones = new Set(["blue", "blue-muted", "green", "green-muted", "teal", "violet", "magenta", "magenta-muted", "orange", "orange-muted", "neutral"]);const icons = new Set(["brand", "captions", "component", "layers", "ranking", "text", "timeline", "video", "waveform"]);-const chromes = new Set(["standard", "group", "point"]);+const chromes = new Set(["standard", "group", "point", "compact"]);const layouts = new Set(["repeat-x", "cover", "contain", "storyboard", "waveform"]);const inspectorDomains = new Set(["where", "how", "when"]);const inspectorControls = new Set<string>(studioParameterControls);@@ -100,6 +102,17 @@ function validateCompanionVocabulary(companion: StudioTrackCompanion): void {throw new Error(`Studio Track Companion ${companion.id} selects unsupported icon ${companion.icon}`);}validateInspector(`Studio Track Companion ${companion.id}`, companion.bindings, companion.inspector);+ const bandIds = new Set<string>();+ for (const band of companion.bands ?? []) {+ if (!band.id.trim() || bandIds.has(band.id)) throw new Error(`Studio Companion ${companion.id} has a duplicate or empty band id`);+ bandIds.add(band.id);+ if (!Number.isFinite(band.heightPx) || band.heightPx <= 0+ || !["before", "after"].includes(band.placement) || !["label", "content"].includes(band.display)) {+ throw new Error(`Studio Companion ${companion.id} band ${band.id} has invalid layout`);+ }+ if (band.tone !== undefined && !tones.has(band.tone)) throw new Error(`Studio Companion ${companion.id} band ${band.id} has unsupported tone`);+ validateInspector(`Studio Companion ${companion.id} band ${band.id}`, band.bindings, band.inspector);+ }for (const attachment of companion.attachments ?? []) {if (attachment.tone !== undefined && !tones.has(attachment.tone)) {throw new Error(`Studio Track Companion ${companion.id} attachment ${attachment.id} selects unsupported tone ${attachment.tone}`);@@ -112,6 +125,10 @@ function validateCompanionVocabulary(companion: StudioTrackCompanion): void {}function validateDraftVocabulary(companion: StudioTrackCompanion, draft: StudioEntityDraft): void {+ if (draft.band !== undefined && (draft.lane !== undefined || !companion.bands?.some(band => band.id === draft.band))) {+ throw new Error(`Studio Companion ${companion.id} entity ${draft.id} must select one declared band within its own Track`);+ }+if (draft.display === undefined || typeof draft.display.title !== "string" || draft.display.title.trim().length === 0) {throw new Error(`Studio Track Companion ${companion.id} entity ${draft.id} has no display title`);}@@ -168,6 +185,7 @@ function specificity(companion: StudioTrackCompanion): number {/** Immutable interpretation assembled for one project session. */export class StudioCompanionRegistry {+ readonly #parameters: readonly StudioParameterCompanion[];readonly #tracks: readonly StudioTrackCompanion[];readonly #films: readonly StudioFilmCompanion[];readonly #scripts: readonly StudioScriptCompanion[];@@ -177,6 +195,7 @@ export class StudioCompanionRegistry {options: {readonly films?: readonly StudioFilmCompanion[];readonly scripts?: readonly StudioScriptCompanion[];+ readonly parameters?: readonly StudioParameterCompanion[];} = {},) {const ids = new Set<string>();@@ -197,11 +216,23 @@ export class StudioCompanionRegistry {ids.add(companion.id);}}+ this.#parameters = Object.freeze([...(options.parameters ?? [])]);+ for (const companion of this.#parameters) {+ if (ids.has(companion.id)) throw new Error(`Studio Parameter Companion id is repeated: ${companion.id}`);+ ids.add(companion.id);+ validateInspector(`Studio Parameter Companion ${companion.id}`, companion.bindings, companion.inspector);+ }this.#tracks = Object.freeze([...tracks]);this.#films = Object.freeze(films);this.#scripts = Object.freeze(scripts);}+ parameterCompanionFor(module: ModuleRef, surface: string): StudioParameterCompanion | undefined {+ const found = this.#parameters.filter(companion => sameModule(companion.match.module, module) && companion.match.surface === surface);+ if (found.length > 1) throw new Error(`Studio Parameter companions are ambiguous for ${module.name}#${surface}.`);+ return found[0];+ }+filmCompanionFor(type: TypeRef,placement: Pick<Placement, "surface" | "module"> | undefined,@@ -234,6 +265,12 @@ export class StudioCompanionRegistry {: { ...found, companion: companion.id };}+ projectScript(input: StudioScriptProjectionInput) {+ const companion = this.#scripts.find(item => item.id === input.source.companion);+ if (companion?.project === undefined) throw new Error(`Script Companion ${input.source.companion} does not provide a timeline projection.`);+ return companion.project(input);+ }+adjustScript(input: {readonly companion: string;readonly sourceName: string;@@ -286,7 +323,7 @@ export class StudioCompanionRegistry {readonly icon: StudioTrackBinding["icon"];readonly lane: StudioTrackBinding["lane"];} {- const companion = this.trackCompanionFor(semanticTrackTypes.track, undefined, []);+ const companion = this.trackCompanionFor(timelineTypes.track, undefined, []);return {family: companion?.family ?? "semantic",tone: companion?.tone ?? "teal",@@ -334,7 +371,7 @@ export class StudioCompanionRegistry {bindTrack(track: StudioResolvedTrack): StudioTrackBinding {const companion = this.#trackCompanion(track);// A Companion may provide a deliberate presentation label for a facet- // (e.g. Speech Visual / Speech Audio). Authored ids remain the fallback+ // (e.g. a component’s picture and sound outputs). Authored ids remain the fallback// for ordinary user-named tracks.const label = companion.label ?? track.trace.authoredId;return {@@ -346,6 +383,7 @@ export class StudioCompanionRegistry {icon: companion.icon ?? (sameType(track.typeRef, compositionTypes.audioTrack) ? "waveform" : "layers"),companion: companion.id,lane: companion.lane ?? flatLane,+ ...(companion.bands === undefined ? {} : { bands: companion.bands.map(({ bindings: _bindings, inspector: _inspector, ...band }) => band) }),...(track.trace.placement === undefined ? {} : { authoredTag: track.trace.placement }),references: track.trace.references.map(({ name, type }) => ({ name, type })),};@@ -360,7 +398,7 @@ export class StudioCompanionRegistry {: { kind: "preview", role: "decoration", preview, layout: "repeat-x" };const projected: readonly StudioEntityDraft[] = companion.poster?.source !== "surface-preview" || surfaceLayer === undefined? drafts- : drafts.map((draft) => draft.lane !== undefined+ : drafts.map((draft) => draft.lane !== undefined || draft.band !== undefined? draft: {...draft,@@ -377,8 +415,10 @@ export class StudioCompanionRegistry {track: StudioResolvedTrack,placement: Placement | undefined,lane?: string,+ band?: string,) {const companion = this.#trackCompanion(track);+ if (band !== undefined) return companion.bands?.find(item => item.id === band)?.bindings ?? [];if (lane !== undefined) {return companion.attachments?.find((attachment) => attachment.id === lane)?.bindings ?? [];}@@ -389,8 +429,10 @@ export class StudioCompanionRegistry {track: StudioResolvedTrack,placement: Placement | undefined,lane?: string,+ band?: string,) {const companion = this.#trackCompanion(track);+ if (band !== undefined) return companion.bands?.find(item => item.id === band)?.inspector ?? [];if (lane !== undefined) {return companion.attachments?.find((attachment) => attachment.id === lane)?.inspector ?? [];}@@ -410,6 +452,8 @@ export function sealStudioClip(id: draft.id.startsWith(`${outputRef}:`) ? draft.id : `${outputRef}:${draft.id}`,...(draft.presentId === undefined ? {} : { presentId: draft.presentId }),authoredId: draft.authoredId,+ ...(draft.band === undefined ? {} : { band: draft.band }),+ ...(draft.selectionGroup === undefined ? {} : { selectionGroup: draft.selectionGroup }),...(draft.markerId === undefined ? {} : { markerId: draft.markerId }),display: draft.display,startFrame: draft.startFrame,diff --git a/packages/studio/src/style.css b/packages/studio/src/style.cssindex 28031391..370f1d03 100644--- a/packages/studio/src/style.css+++ b/packages/studio/src/style.css@@ -28,6 +28,8 @@--tone-orange-muted-rail: #e7834c;--tone-green: #38853e;--tone-green-rail: #75d079;+ --tone-green-muted: #285c30;+ --tone-green-muted-rail: #73a478;--tone-teal: #008889;--tone-teal-rail: #00d4d5;/* Semantic ruler: brightness separates content from its own band; hue marks state. */@@ -35,6 +37,7 @@--semantic-word: #005551;/* Author intent uses a gold annotation hue; brand pink stays in the chrome. */--semantic-intent: #a28b2b;+ --semantic-moment: var(--brand-500);/* Same perceived active level as a selected/covered semantic cell. */--semantic-current: #009f98;/* Ordered marker depth: one hue, decreasing lightness. */@@ -44,10 +47,14 @@--intent-depth-3: #846d00;--tone-blue: #1777b8;--tone-blue-rail: #58c0ff;+ --tone-blue-muted: #175079;+ --tone-blue-muted-rail: #5795bf;--tone-violet: #6f63b8;--tone-violet-rail: #b3a6ff;--tone-magenta: #995398;--tone-magenta-rail: #ea93e8;+ --tone-magenta-muted: #683b68;+ --tone-magenta-muted-rail: #b57ab4;--tone-neutral: #6a737c;--tone-neutral-rail: #aeb9c4;--timeline-label-width: 188px;@@ -689,6 +696,7 @@ button:focus-visible, textarea:focus-visible {caret-color: var(--interaction-accent);cursor: text !important;}+.parameter-display { width: 100%; padding: 5px 7px; color: var(--text-2); font-size: 11px; line-height: 1.2; text-align: right; overflow-wrap: anywhere; user-select: text; font-variant-numeric: tabular-nums; }.parameter-value:focus { border-color: #6a6e77; outline: 0; background: #131417; }.control-boolean .parameter-right { justify-content: flex-end; }.control-boolean .parameter-value {@@ -745,8 +753,8 @@ button:focus-visible, textarea:focus-visible {.parameter-color-value { width: min(100%, 102px); }.control-list,.control-record { grid-template-columns: minmax(0, 1fr); align-items: stretch; gap: 6px; }-.control-list .parameter-right,-.control-record .parameter-right { grid-column: 1 / -1; width: 100%; }+.control-list > .parameter-right,+.control-record > .parameter-right { grid-column: 1 / -1; width: 100%; }.parameter-structured { min-width: 0; width: 100%; }.parameter-structured-summary {min-height: 20px;@@ -786,20 +794,6 @@ button:focus-visible, textarea:focus-visible {.parameter-list-actions button:not(:disabled):focus-visible { outline: 1px solid #62666f; }.parameter-list-actions .icon { width: 11px; height: 11px; }.parameter-record-fields { min-width: 0; display: flex; flex-direction: column; gap: 5px; }-.parameter-record-field {- min-width: 0;- display: grid;- grid-template-columns: 58px minmax(0, 1fr);- align-items: center;- gap: 7px;-}-.parameter-record-field > span:first-child {- overflow: hidden;- color: #6f737c;- font-size: 10px;- text-overflow: ellipsis;- white-space: nowrap;-}.parameter-structured-footer {min-height: 30px;display: flex;@@ -818,10 +812,10 @@ button:focus-visible, textarea:focus-visible {font: 10px/1 var(--font-ui);}.parameter-structured-footer button:disabled { color: #454850; background: #16171a; }-.parameter-structured-footer button.primary:not(:disabled) { border-color: #555962; background: #292b30; color: #e0e2e6; }.parameter-structured-add { display: inline-flex; align-items: center; gap: 4px; }.parameter-structured-add .icon { width: 11px; height: 11px; }-.parameter-structured-decisions { display: flex; align-items: center; gap: 4px; margin-left: auto; }+.parameter-structured-notice { margin-top: 6px; color: var(--text-3); font: 10px/1.4 var(--font-ui); }+.parameter-structured-notice[hidden] { display: none; }.parameter-select { position: relative; width: 100%; min-width: 0; }.parameter-select-trigger {width: 100%;@@ -1016,10 +1010,13 @@ button:focus-visible, textarea:focus-visible {transform: translateY(-50%);}.track-tone-blue { --track: var(--tone-blue); --rail: var(--tone-blue-rail); }+.track-tone-blue-muted { --track: var(--tone-blue-muted); --rail: var(--tone-blue-muted-rail); }.track-tone-green { --track: var(--tone-green); --rail: var(--tone-green-rail); }+.track-tone-green-muted { --track: var(--tone-green-muted); --rail: var(--tone-green-muted-rail); }.track-tone-teal { --track: var(--tone-teal); --rail: var(--tone-teal-rail); }.track-tone-violet { --track: var(--tone-violet); --rail: var(--tone-violet-rail); }.track-tone-magenta { --track: var(--tone-magenta); --rail: var(--tone-magenta-rail); }+.track-tone-magenta-muted { --track: var(--tone-magenta-muted); --rail: var(--tone-magenta-muted-rail); }.track-tone-orange { --track: var(--tone-orange); --rail: var(--tone-orange-rail); }.track-tone-orange-muted { --track: var(--tone-orange-muted); --rail: var(--tone-orange-muted-rail); }.track-tone-neutral { --track: var(--tone-neutral); --rail: var(--tone-neutral-rail); }@@ -1083,8 +1080,10 @@ button:focus-visible, textarea:focus-visible {height: var(--semantic-band-height);overflow: hidden;background: var(--lane);+ box-shadow: inset 0 -1px var(--line);+ isolation: isolate;}-.semantic-band-intent { border-bottom: 0; }+.semantic-band:last-child { box-shadow: none; }.lane-attached {border-left: 0;background: var(--lane);@@ -1104,12 +1103,12 @@ button:focus-visible, textarea:focus-visible {border: 0;border-radius: 0;background: var(--semantic-cell);- box-shadow: none;+ box-shadow: inset 0 0 0 1px #ffffff12;color: var(--text);text-align: left;cursor: default;}-.semantic-cell:focus-visible { outline: 0; }+.semantic-cell:focus-visible { outline: 0; box-shadow: inset 0 0 0 1px #ffffff60; }.semantic-segment {--semantic-cell: var(--semantic-segment);}@@ -1120,11 +1119,9 @@ button:focus-visible, textarea:focus-visible {--semantic-cell: var(--semantic-intent);color: #fff;}-.semantic-cell.selected,-.semantic-cell.live { z-index: 4; filter: brightness(1.22); }-.semantic-segment.selected { box-shadow: none; }-.semantic-segment.live.selected { filter: brightness(1.22); }-.semantic-moment.selected { z-index: 4; filter: brightness(1.22); }+.semantic-cell.live { filter: brightness(1.12); }+.semantic-cell.selected { z-index: 4; filter: brightness(1.16); box-shadow: inset 0 0 0 1px #ffffff60; }+.semantic-moment.selected { z-index: 5; filter: brightness(1.16); box-shadow: inset 0 0 0 1px #ffffff60; }.semantic-segment-label {flex: 1 1 auto;min-width: 0;@@ -1170,21 +1167,19 @@ button:focus-visible, textarea:focus-visible {.semantic-word .semantic-cell-content { padding: 0 5px; }.semantic-moment {position: absolute;- top: 4px;- width: 14px;- height: 14px;- transform: translateX(-7px) rotate(45deg);+ top: calc((var(--semantic-band-height) - 9px) / 2);+ z-index: 1;+ width: 9px;+ height: 9px;+ transform: translateX(-50%) scaleX(.6) rotate(45deg);border-radius: 0;padding: 0;- border: 0;- background: var(--semantic-intent);+ border: 1px solid var(--lane);+ background: var(--semantic-moment);}.semantic-moment:focus-visible { outline: 0; }-.track-label-semantic { position: sticky; top: var(--timeline-ruler-height); z-index: 8; border-bottom: 1px solid var(--line); background: #111216; }-.track-label-semantic::before {- border-radius: 0;- background: #181a1f;-}+.track-label-semantic { position: sticky; top: 0; z-index: 8; border-bottom: 1px solid var(--line); background: #111216; }+.track-label-semantic::before { display: none; }.track-label-attached {/* The branch marker occupies the same 18px prefix as a root icon;attached and root labels therefore share one exact text column. */@@ -1207,7 +1202,6 @@ button:focus-visible, textarea:focus-visible {display: block;padding: 0;border: 0;- border-bottom: 1px solid color-mix(in srgb, var(--track) 48%, var(--lane));border-radius: var(--timeline-item-radius);background: var(--item-body);box-shadow: none;@@ -1215,8 +1209,7 @@ button:focus-visible, textarea:focus-visible {text-align: left;cursor: default;}-/* The edge affordance is intentionally visual-only for now. Actual trim and- writeback remain disabled until their SVML mapping is implemented. */+/* Only Companion-declared editable intervals enable edge gestures. */#app .clip::before,#app .clip::after {content: "";@@ -1244,10 +1237,19 @@ button:focus-visible, textarea:focus-visible {}.clip.editing { box-shadow: 0 0 0 1px #fff; }.clip:focus-visible { outline: 0; }+/* Draw above thumbnails and text without changing interval geometry. */+.clip-boundary {+ position: absolute;+ z-index: 5;+ inset: 0;+ border: 1px solid #ffffff20;+ border-radius: inherit;+ pointer-events: none;+}.clip-selection {position: absolute;z-index: 6;- inset: calc(-1 * var(--timeline-selection-stroke));+ inset: 0;display: none;border: var(--timeline-selection-stroke) solid var(--timeline-selection-color);border-radius: var(--timeline-selection-radius);@@ -1256,20 +1258,21 @@ button:focus-visible, textarea:focus-visible {.clip-selection::before, .clip-selection::after {content: "";position: absolute;- top: -1px;- bottom: -1px;+ top: 0;+ bottom: 0;width: var(--timeline-selection-edge);background: var(--timeline-selection-color);}.clip-selection::before {- left: calc(-1 * var(--timeline-selection-stroke));+ left: 0;border-radius: var(--timeline-selection-radius) 0 0 var(--timeline-selection-radius);}.clip-selection::after {- right: calc(-1 * var(--timeline-selection-stroke));+ right: 0;border-radius: 0 var(--timeline-selection-radius) var(--timeline-selection-radius) 0;}.clip.selected > .clip-selection { display: block; }+.clip.selected > .clip-boundary { border-color: var(--timeline-selection-color); }.clip-head {position: absolute;z-index: 3;@@ -1288,6 +1291,7 @@ button:focus-visible, textarea:focus-visible {}.clip-body {position: absolute;+ z-index: 1;top: var(--timeline-item-header-height);right: 0;bottom: 0;@@ -1346,10 +1350,13 @@ button:focus-visible, textarea:focus-visible {font-variant-numeric: tabular-nums;}.clip-tone-blue { --track: var(--tone-blue); }+.clip-tone-blue-muted { --track: var(--tone-blue-muted); }.clip-tone-green { --track: var(--tone-green); }+.clip-tone-green-muted { --track: var(--tone-green-muted); }.clip-tone-teal { --track: var(--tone-teal); }.clip-tone-violet { --track: var(--tone-violet); }.clip-tone-magenta { --track: var(--tone-magenta); }+.clip-tone-magenta-muted { --track: var(--tone-magenta-muted); }.clip-tone-orange { --track: var(--tone-orange); }.clip-tone-orange-muted { --track: var(--tone-orange-muted); }.clip-tone-neutral { --track: var(--tone-neutral); }@@ -1514,3 +1521,68 @@ button:focus-visible, textarea:focus-visible {.artifact-card.selected .artifact-preview { background: var(--artifact-selection); border-radius: 3px; }.artifact-card.selected .artifact-name-text { color: #f1f2f5; background: var(--artifact-selection); }.library-continuation { grid-column: 1 / -1; height: 28px; display: grid; place-items: center; color: #777e89; font-size: 10px; }++.timeline-overlap-menu {+ position: fixed;+ z-index: 100;+ min-width: 160px;+ max-width: 280px;+ max-height: 240px;+ overflow: auto;+ padding: 4px;+ border: 1px solid var(--line-strong);+ border-radius: 5px;+ background: var(--panel-raised);+ box-shadow: 0 6px 20px #0006;+}+.timeline-overlap-menu button {+ display: block;+ width: 100%;+ padding: 6px 9px;+ border: 0;+ border-radius: 2px;+ background: transparent;+ color: var(--text);+ font-family: inherit;+ font-size: 12px;+ line-height: 1.4;+ text-align: left;+ overflow-wrap: anywhere;+}+.timeline-overlap-menu button:hover,+.timeline-overlap-menu button:focus-visible { background: var(--field); outline: none; }++/* Compact authored rules keep interval geometry without a thumbnail body. */+.clip-chrome-compact > .clip-head {+ height: 100%;+ border-bottom: 0;+ border-radius: var(--timeline-item-radius);+ background: var(--item-head);+}+.clip-chrome-compact > .clip-body,+.clip-chrome-compact .clip-meta { display: none; }+.clip-chrome-compact .clip-name { font-size: 11px; font-weight: 500; }++/* Track-internal bands share one label and meet without child-track gutters. */+.timeline-track-banded { padding: var(--timeline-item-inset-y) 0; }+.timeline-track-banded > .track-band { border-bottom: 0; }+.timeline-track-banded > .track-band > .clip {+ height: 100%;+ margin-top: 0;+ border-radius: 0;+}+.timeline-track-banded > .track-band:first-child > .clip {+ border-top-left-radius: var(--timeline-item-radius);+ border-top-right-radius: var(--timeline-item-radius);+}+.timeline-track-banded > .track-band:last-child > .clip {+ border-bottom-left-radius: var(--timeline-item-radius);+ border-bottom-right-radius: var(--timeline-item-radius);+}+.track-band .clip-head, .track-band .clip-body { border-radius: inherit; }+.track-band-label .clip-head {+ height: 100%;+ border-bottom: 0;+ background: var(--item-head);+}+.track-band-label .clip-body, .track-band-label .clip-meta { display: none; }diff --git a/packages/studio/src/ui/main.ts b/packages/studio/src/ui/main.tsindex 6b7572cf..643899f3 100644--- a/packages/studio/src/ui/main.ts+++ b/packages/studio/src/ui/main.ts@@ -1,6 +1,6 @@import type { Clip, StudioFailure, StudioInspectorDomain, StudioSnapshot } from "../shared.js";import type { CanonicalValue, ValueSchema } from "@hypit/protocol";-import { parameterAuthorValue, parameterNumber, parameterOption, validateParameterValue } from "../parameter-values.js";+import { parameterAuthorValue, parameterControlForSchema, parameterNumber, parameterOption, parameterRecordSchema, parameterRecordVariants, validateParameterValue } from "../parameter-values.js";import { createCodePane } from "./code.js";import { icon } from "./icons.js";import { createLibraryPane } from "./library.js";@@ -154,7 +154,7 @@ const domainPresentation: Readonly<Record<StudioInspectorDomain, { readonly labehow: { label: "How", icon: "how" },when: { label: "When", icon: "when" },};-const domainOrder: readonly StudioInspectorDomain[] = ["where", "how", "when"];+const domainOrder: readonly StudioInspectorDomain[] = ["where", "when", "how"];const inspectorDomainByEntity = new Map<string, StudioInspectorDomain>();const inspectorPageByEntity = new Map<string, string>();@@ -230,8 +230,8 @@ document.addEventListener("keydown", (event) => {}, true);function selectControl(- entityId: string,parameter: Clip["inspector"][number],+ change: (value: CanonicalValue) => void,): HTMLElement {const control = document.createElement("div");control.className = "parameter-select parameter-control";@@ -286,7 +286,7 @@ function selectControl(}close(false);trigger.focus();- commitControl(entityId, parameter, option.value);+ change(option.value);});return item;});@@ -437,7 +437,8 @@ function fieldLabel(name: string): string {function blankValue(schema: ValueSchema): CanonicalValue {if (schema.kind === "string") return "";- if (schema.kind === "number" || schema.kind === "null" || schema.kind === "literal" || schema.kind === "oneOf") return null;+ if (schema.kind === "literal") return schema.value;+ if (schema.kind === "number" || schema.kind === "null" || schema.kind === "oneOf") return null;if (schema.kind === "boolean") return false;if (schema.kind === "array") return [];if (schema.kind === "object") return Object.fromEntries(Object.entries(schema.fields)@@ -452,31 +453,12 @@ function scalarDraftControl(held: CanonicalValue | undefined,change: (value: CanonicalValue) => void,): HTMLElement {- if (schema.kind === "string" && schema.format === "color") {- return colorValueControl(label, typeof held === "string" ? held : "", change, true);- }- const input = document.createElement("input");- input.className = "parameter-value parameter-structured-value";- input.setAttribute("aria-label", label);- input.spellcheck = false;- if (schema.kind === "boolean") {- input.type = "checkbox";- input.checked = held === true;- input.addEventListener("change", () => change(input.checked));- return input;- }- input.type = "text";- if (schema.kind === "number") input.inputMode = "decimal";- input.value = held === undefined || held === null ? "" : textValue(held);- input.addEventListener("input", () => {- if (schema.kind === "number") {- const number = Number(input.value);- change(input.value.trim().length === 0 || !Number.isFinite(number) ? null : number);- } else {- change(input.value);- }- });- return input;+ return parameterControl("", {+ id: label, label, domain: "how", section: { id: "record", label: "" },+ control: parameterControlForSchema(schema) ?? "text",+ ...(schema.kind === "string" && schema.enum !== undefined ? { options: schema.enum } : {}),+ schema, value: held ?? null,+ }, change);}function recordDraftControl(@@ -490,30 +472,34 @@ function recordDraftControl(const fields = document.createElement("div");fields.className = "parameter-record-fields";for (const [name, field] of Object.entries(schema.fields)) {- const row = document.createElement("label");- row.className = "parameter-record-field";- const copy = document.createElement("span");- copy.textContent = fieldLabel(name);- row.append(copy, scalarDraftControl(fieldLabel(name), field.schema, record[name], (next) => {+ if (field.schema.kind === "literal") continue;+ fields.append(scalarDraftControl(fieldLabel(name), field.schema, record[name], (next) => {record = { ...record, [name]: next };change(record);}));- fields.append(row);}return fields;}-function structuredControl(entityId: string, parameter: Clip["inspector"][number]): HTMLElement {- const schema = parameter.schema;+function structuredControl(entityId: string, parameter: Clip["inspector"][number], change: (value: CanonicalValue) => void): HTMLElement {+ const declaredSchema = parameter.schema;const shell = document.createElement("div");shell.className = "parameter-structured";- if (schema === undefined || (schema.kind !== "array" && schema.kind !== "object")) return shell;+ if (declaredSchema === undefined || (declaredSchema.kind !== "array" && declaredSchema.kind !== "object" && !parameterRecordVariants(declaredSchema).length)) return shell;let draft = structuredClone(parameter.value);- let refreshDecisions = (): void => {};+ const notice = document.createElement("div");+ notice.className = "parameter-structured-notice";+ notice.setAttribute("role", "status");+ const save = (): void => {+ const changed = !sameValue(draft, parameter.value);+ notice.hidden = !changed || valid();+ notice.textContent = "Complete the required fields with valid values to save.";+ if (changed && valid()) change(draft);+ };const valid = (): boolean => {try {- validateParameterValue(draft, schema, parameter.label);+ validateParameterValue(draft, declaredSchema, parameter.label);return true;} catch {return false;@@ -522,6 +508,21 @@ function structuredControl(entityId: string, parameter: Clip["inspector"][numberconst render = (): void => {shell.replaceChildren();+ const variants = parameterRecordVariants(declaredSchema);+ const schema = declaredSchema.kind === "array" ? declaredSchema : parameterRecordSchema(declaredSchema, draft);+ if (schema === undefined) return;+ if (variants.length) {+ const active = variants.find(variant => variant.schema === schema)!;+ shell.append(parameterControl(entityId, {+ ...parameter, control: "select", value: variants.indexOf(active),+ options: variants.map((variant, index) => ({ value: index, label: textValue(variant.value) })),+ }, (next) => {+ if (next === variants.indexOf(active)) return;+ draft = blankValue(variants[Number(next)]!.schema);+ render();+ save();+ }));+ }const summary = document.createElement("div");summary.className = "parameter-structured-summary";const count = document.createElement("span");@@ -529,7 +530,7 @@ function structuredControl(entityId: string, parameter: Clip["inspector"][number? `${draft.length} ${draft.length === 1 ? "item" : "items"}`: "Structured value";summary.append(count);- shell.append(summary);+ if (schema.kind === "array") shell.append(summary);if (schema.kind === "array") {const list = Array.isArray(draft) ? draft : [];@@ -542,12 +543,12 @@ function structuredControl(entityId: string, parameter: Clip["inspector"][number? recordDraftControl(schema.items, item, (next) => {const current = Array.isArray(draft) ? draft : [];draft = current.map((candidate, heldIndex) => heldIndex === index ? next : candidate);- refreshDecisions();+ save();}): scalarDraftControl(`${parameter.label} ${index + 1}`, schema.items, item, (next) => {const current = Array.isArray(draft) ? draft : [];draft = current.map((candidate, heldIndex) => heldIndex === index ? next : candidate);- refreshDecisions();+ save();});const actions = document.createElement("span");actions.className = "parameter-list-actions";@@ -557,6 +558,7 @@ function structuredControl(entityId: string, parameter: Clip["inspector"][numberreordered.splice(index + offset, 0, moving!);draft = reordered;render();+ save();};const up = document.createElement("button");up.type = "button";@@ -579,6 +581,7 @@ function structuredControl(entityId: string, parameter: Clip["inspector"][numberconst current = Array.isArray(draft) ? draft : [];draft = current.filter((_, heldIndex) => heldIndex !== index);render();+ save();});actions.append(up, down, remove);row.append(editor, actions);@@ -588,7 +591,7 @@ function structuredControl(entityId: string, parameter: Clip["inspector"][number} else {shell.append(recordDraftControl(schema, draft, (next) => {draft = next;- refreshDecisions();+ save();}));}@@ -605,69 +608,81 @@ function structuredControl(entityId: string, parameter: Clip["inspector"][numberconst current = Array.isArray(draft) ? draft : [];draft = [...current, blankValue(schema.items)];render();+ save();});footer.append(add);}- const decisions = document.createElement("span");- decisions.className = "parameter-structured-decisions";- const reset = document.createElement("button");- reset.type = "button";- reset.textContent = "Reset";- reset.disabled = sameValue(draft, parameter.value);- reset.addEventListener("click", () => {- draft = structuredClone(parameter.value);- render();- });- const apply = document.createElement("button");- apply.type = "button";- apply.className = "primary";- apply.textContent = "Apply";- apply.addEventListener("click", () => commitControl(entityId, parameter, draft));- refreshDecisions = () => {- reset.disabled = sameValue(draft, parameter.value);- apply.disabled = reset.disabled || !valid();- };- refreshDecisions();- decisions.append(reset, apply);- footer.append(decisions);- shell.append(footer);+ if (schema.kind === "array") shell.append(footer);+ notice.hidden = sameValue(draft, parameter.value) || valid();+ shell.append(notice);};render();return shell;}-function parameterControl(entityId: string, parameter: Clip["inspector"][number]): HTMLElement {+function parameterControl(+ entityId: string,+ parameter: Clip["inspector"][number],+ change?: (value: CanonicalValue) => void,+): HTMLElement {+ const editable = parameter.edit !== undefined || change !== undefined;+ const commit = change ?? ((value: CanonicalValue) => commitControl(entityId, parameter, value));+ if (editable && parameter.control === "record" && parameter.schema !== undefined && parameterRecordVariants(parameter.schema).length) {+ return structuredControl(entityId, parameter, commit);+ }const row = document.createElement("div");- row.className = `parameter-row parameter-editable control-${parameter.control}`;+ row.className = `parameter-row ${!editable ? "parameter-readonly" : "parameter-editable"} control-${parameter.control}`;const name = document.createElement("span");name.className = "parameter-label";name.textContent = parameter.label;name.title = parameter.summary ?? parameter.label;const right = document.createElement("span");right.className = "parameter-right parameter-control";+ if (!editable) {+ const value = document.createElement("span");+ value.className = "parameter-display";+ let displayValue = textValue(parameter.value);+ let unitLabel = parameter.unit;+ if (parameter.control === "number") {+ try {+ const numeric = parameterNumber(change && parameter.value === null ? 0 : parameter.value, parameter.number);+ displayValue = String(numeric.value);+ unitLabel = numeric.suffix || parameter.unit;+ } catch { /* Keep the actual value visible when it is not a scalar. */ }+ } else if (parameter.control === "select") {+ displayValue = parameter.options?.map(parameterOption).find(option => option.value === parameter.value)?.label ?? displayValue;+ }+ value.textContent = displayValue;+ right.append(value);+ if (unitLabel !== undefined) {+ const unit = document.createElement("small"); unit.textContent = unitLabel; right.append(unit);+ }+ row.append(name, right);+ return row;+ }let unitLabel = parameter.unit;if (parameter.control === "select") {- right.append(selectControl(entityId, parameter));+ right.append(selectControl(parameter, commit));} else if (parameter.control === "color") {right.append(colorValueControl(parameter.label, textValue(parameter.value), (next) => {- commitControl(entityId, parameter, next);+ commit(next);}));if (parameter.swatches?.length) {const palette = document.createElement("span"); palette.className = "parameter-swatches";for (const color of parameter.swatches) {const swatch = document.createElement("button"); swatch.type = "button";swatch.style.backgroundColor = color; swatch.title = color; swatch.setAttribute("aria-label", `Use ${color}`);- swatch.addEventListener("click", () => commitControl(entityId, parameter, color)); palette.append(swatch);+ swatch.addEventListener("click", () => commit(color)); palette.append(swatch);}right.append(palette);}} else if (parameter.control === "list" || parameter.control === "record") {- right.append(structuredControl(entityId, parameter));+ right.append(structuredControl(entityId, parameter, commit));} else {if (parameter.multiline && parameter.control === "text") {const value = document.createElement("textarea"); value.className = "parameter-value parameter-multiline";value.value = textValue(parameter.value); value.setAttribute("aria-label", parameter.label);- value.addEventListener("change", () => commitControl(entityId, parameter, value.value)); right.append(value);+ value.addEventListener("change", () => commit(value.value)); right.append(value);row.append(name, right); return row;}const value = document.createElement("input");@@ -678,8 +693,8 @@ function parameterControl(entityId: string, parameter: Clip["inspector"][number]if (value.type === "checkbox") value.checked = parameter.value === true || parameter.value === "true";else if (parameter.control === "number") {try {- const numeric = parameterNumber(parameter.value, parameter.number);- value.type = "number"; value.inputMode = "decimal"; value.value = String(numeric.value);+ const numeric = parameterNumber(change && parameter.value === null ? 0 : parameter.value, parameter.number);+ value.type = "number"; value.inputMode = "decimal"; value.value = change && parameter.value === null ? "" : String(numeric.value);const schema = parameter.schema?.kind === "number" ? parameter.schema : undefined;const scale = parameter.number?.scale ?? 1;const minimum = parameter.number?.minimum ?? (schema?.minimum === undefined ? undefined : schema.minimum * scale);@@ -698,11 +713,12 @@ function parameterControl(entityId: string, parameter: Clip["inspector"][number]if (!value.title) value.title = parameter.summary ?? parameter.label;value.addEventListener("change", () => {if (value.type === "checkbox") {- commitControl(entityId, parameter, value.checked);+ commit(value.checked);} else if (parameter.control === "number") {- if (value.reportValidity()) commitControl(entityId, parameter, value.valueAsNumber);+ if (change) commit(Number.isFinite(value.valueAsNumber) ? value.valueAsNumber : null);+ else if (value.reportValidity()) commit(value.valueAsNumber);} else {- commitControl(entityId, parameter, value.value);+ commit(value.value);}});right.append(value);@@ -802,7 +818,7 @@ function renderInspector(snapshot: StudioSnapshot, clipId: string | undefined):defaultWorkspaceHeading();const empty = document.createElement("div");empty.className = "inspector-empty";- empty.textContent = "No adjustable parameters";+ empty.textContent = "No details for this selection";inspector.replaceChildren(empty);return;}@@ -813,17 +829,22 @@ function renderInspector(snapshot: StudioSnapshot, clipId: string | undefined):const domainFields = clip.inspector.filter((field) => field.domain === activeDomain);const pages = new Map<string, { readonly label: string; readonly fields: typeof domainFields }>();- for (const field of domainFields) {- const id = field.page?.id ?? "default";+ const commonFields = domainFields.filter(field => field.page === undefined);+ for (const field of domainFields.filter(field => field.page !== undefined)) {+ const id = field.page!.id;const page = pages.get(id) ?? { label: field.page?.label ?? "", fields: [] };pages.set(id, { ...page, fields: [...page.fields, field] });}const pageIds = [...pages.keys()];+ if (pageIds.length === 0) {+ inspector.replaceChildren(...parameterGroups(clip.id, commonFields));+ return;+ }const memoryKey = `${clip.id}:${activeDomain}`;const rememberedPage = inspectorPageByEntity.get(memoryKey);const activePage = rememberedPage !== undefined && pages.has(rememberedPage) ? rememberedPage : pageIds[0]!;inspectorPageByEntity.set(memoryKey, activePage);- const page = pages.get(activePage)!;+ const page = pages.get(activePage);const subtabs = document.createElement("div");subtabs.className = "inspector-subtabs";if (pageIds.length > 1) {@@ -841,7 +862,7 @@ function renderInspector(snapshot: StudioSnapshot, clipId: string | undefined):return button;}));}- inspector.replaceChildren(...(pageIds.length > 1 ? [subtabs] : []), ...parameterGroups(clip.id, page.fields));+ inspector.replaceChildren(...(pageIds.length > 1 ? [subtabs] : []), ...parameterGroups(clip.id, [...commonFields, ...(page?.fields ?? [])]));}function renderSemanticInspector(snapshot: StudioSnapshot, segmentId: string): void {@@ -850,7 +871,7 @@ function renderSemanticInspector(snapshot: StudioSnapshot, segmentId: string): vdefaultWorkspaceHeading();const empty = document.createElement("div");empty.className = "inspector-empty";- empty.textContent = "No adjustable parameters";+ empty.textContent = "No details for this selection";inspector.replaceChildren(empty);}diff --git a/packages/studio/src/ui/timeline.ts b/packages/studio/src/ui/timeline.tsindex 7db4254a..4095662e 100644--- a/packages/studio/src/ui/timeline.ts+++ b/packages/studio/src/ui/timeline.ts@@ -147,11 +147,20 @@ export function createTimeline(store: Store): Timeline {let paintFrame = 0;let rebuildFrame = 0;const readableWordLabels = new Set<string>();+ let selectedWord: string | undefined;+ let overlapMenu: HTMLElement | undefined;+ const closeOverlapMenu = () => { overlapMenu?.remove(); overlapMenu = undefined; };+ element.addEventListener("pointerdown", event => {+ if (!overlapMenu?.contains(event.target as Node)) closeOverlapMenu();+ });+ element.addEventListener("keydown", event => { if (event.key === "Escape") closeOverlapMenu(); });+ body.addEventListener("scroll", closeOverlapMenu);let clipNodes: readonly {readonly node: HTMLElement;readonly start: number;readonly end: number;readonly id: string;+ readonly selectionGroup?: string;}[] = [];let semanticNodes: readonly {readonly node: HTMLElement;@@ -301,7 +310,7 @@ export function createTimeline(store: Store): Timeline {const from = place(preview.startFrame, state.snapshot.space.frameCount, zoom.window());const to = place(preview.endFrameExclusive, state.snapshot.space.frameCount, zoom.window());activeEdit.node.style.left = `${from * 100}%`;- activeEdit.node.style.width = `max(2px, calc(${Math.max(0, to - from) * 100}% - ${itemMetrics.gapPx}px))`;+ activeEdit.node.style.width = `max(2px, ${Math.max(0, to - from) * 100}%)`;}return;}@@ -479,29 +488,32 @@ export function createTimeline(store: Store): Timeline {.sort((left, right) => (left.binding.lane.order ?? 0) - (right.binding.lane.order ?? 0));const buildSemanticLane = (snapshot: StudioSnapshot): void => {- if (snapshot.semantic === undefined) {+ if (snapshot.semantic === undefined || snapshot.semantic.segments.length === 0) {return;}const presentation = snapshot.semantic.presentation;- // This is one semantic ruler group, not a regular Film lane. Its three- // bands share the same inner inset as ordinary items, so the first pixel- // of the semantic content aligns with the first pixel of the label card.- const laneHeight = presentation.lane.heightPx;- const bandHeight = Math.max(1, (laneHeight - itemMetrics.insetYPx * 2) / 3);+ const bandHeight = Math.max(1, (presentation.lane.heightPx - itemMetrics.insetYPx * 2) / 3);+ const bands = [+ { kind: "segment", height: bandHeight },+ ...(snapshot.semantic.tokens.length > 0 ? [{ kind: "word", height: bandHeight }] : []),+ ...(snapshot.semantic.selections.length + snapshot.semantic.moments.length > 0+ ? [{ kind: "intent", height: bandHeight }] : []),+ ];+ const laneHeight = bands.reduce((height, band) => height + band.height, itemMetrics.insetYPx * 2);const label = createTrackLabel(- presentation.label ?? "Semantic",+ presentation.label ?? "Timeline",presentation.tone,presentation.icon,- `${snapshot.semantic.segments.length} take${snapshot.semantic.segments.length === 1 ? "" : "s"}`,+ `${snapshot.semantic.segments.length} segment${snapshot.semantic.segments.length === 1 ? "" : "s"}`,laneHeight,);label.classList.add("track-label-semantic");+ label.style.height = `calc(var(--timeline-ruler-height) + ${laneHeight}px)`;labels.append(label);const lane = document.createElement("div");lane.className = `lane semantic-lane track-tone-${presentation.tone}`;lane.style.height = `${laneHeight}px`;- lane.style.setProperty("--semantic-band-height", `${bandHeight}px`);const laneWidth = lanes.clientWidth;const nextSemanticNodes: {node: HTMLElement;@@ -511,12 +523,15 @@ export function createTimeline(store: Store): Timeline {kind: "segment" | "word" | "selection" | "moment";}[] = [];- const bands = ["segment", "word", "intent"] as const;- for (const [index, kind] of bands.entries()) {+ let bandTop = itemMetrics.insetYPx;+ for (const { kind, height } of bands) {const band = document.createElement("div");band.className = `semantic-band semantic-band-${kind}`;- band.style.top = `${itemMetrics.insetYPx + index * bandHeight}px`;+ band.style.top = `${bandTop}px`;+ band.style.setProperty("--semantic-band-height", `${height}px`);+ band.setAttribute("aria-label", kind === "intent" ? "Selections and Moments" : kind === "word" ? "Words" : "Segments");lane.append(band);+ bandTop += height;}const segmentBand = lane.querySelector<HTMLElement>(".semantic-band-segment")!;const wordBand = lane.querySelector<HTMLElement>(".semantic-band-word")!;@@ -535,7 +550,7 @@ export function createTimeline(store: Store): Timeline {node.dataset.semanticSegment = segment.id;node.style.left = `${from * 100}%`;const segmentWidth = Math.max(0, to - from) * 100;- node.style.width = `max(2px, calc(${segmentWidth}% - ${itemMetrics.gapPx}px))`;+ node.style.width = `max(2px, ${segmentWidth}%)`;node.title = segment.id;const segmentLabel = document.createElement("span");segmentLabel.className = "semantic-segment-label";@@ -575,12 +590,12 @@ export function createTimeline(store: Store): Timeline {word.setAttribute("aria-label", token.text);word.style.left = `${wordFrom * 100}%`;const wordWidth = Math.max(0, wordTo - wordFrom);- word.style.width = `max(1px, calc(${wordWidth * 100}% - ${itemMetrics.gapPx}px))`;+ word.style.width = `max(1px, ${wordWidth * 100}%)`;word.title = token.text;const text = document.createElement("span");text.className = "semantic-word-label";text.textContent = token.text;- const wordWidthPx = Math.max(1, wordWidth * laneWidth - itemMetrics.gapPx);+ const wordWidthPx = Math.max(1, wordWidth * laneWidth);const textWidthPx = measuredText(token.text);const contentWidthPx = Math.max(0, wordWidthPx - wordLabelPaddingPx * 2);const wasReadable = readableWordLabels.has(token.id);@@ -596,6 +611,8 @@ export function createTimeline(store: Store): Timeline {word.append(wordContent);word.addEventListener("click", (event) => {event.stopPropagation();+ store.clearSelection();+ selectedWord = token.id;store.seek(token.startFrame, "timeline");});nextSemanticNodes.push({ node: word, id: token.id, start: token.startFrame, end: token.endFrameExclusive, kind: "word" });@@ -611,7 +628,7 @@ export function createTimeline(store: Store): Timeline {node.tabIndex = 0;node.setAttribute("aria-label", `Selection ${selection.id}`);node.style.left = `${from * 100}%`;- node.style.width = `max(2px, calc(${Math.max(0, to - from) * 100}% - ${itemMetrics.gapPx}px))`;+ node.style.width = `max(2px, ${Math.max(0, to - from) * 100}%)`;node.title = selection.id;const selectionLabel = document.createElement("span");selectionLabel.className = "semantic-selection-label";@@ -645,6 +662,33 @@ export function createTimeline(store: Store): Timeline {intentBand.append(node);nextSemanticNodes.push({ node, id: moment.id, start: moment.frame, end: moment.frame + 1, kind: "moment" });}+ lane.addEventListener("contextmenu", event => {+ const hits = nextSemanticNodes.filter(({ node }) => {+ const rect = node.getBoundingClientRect();+ return event.clientX >= rect.left && event.clientX < rect.right+ && event.clientY >= rect.top && event.clientY < rect.bottom;+ });+ if (hits.length < 2) return;+ event.preventDefault();+ closeOverlapMenu();+ const menu = document.createElement("div");+ menu.className = "timeline-overlap-menu";+ menu.setAttribute("role", "menu");+ menu.setAttribute("aria-label", "Overlapping timeline objects");+ for (const hit of [...hits].reverse()) {+ const option = document.createElement("button");+ option.type = "button";+ option.setAttribute("role", "menuitem");+ option.textContent = hit.node.getAttribute("aria-label") ?? hit.id;+ option.addEventListener("click", () => { closeOverlapMenu(); hit.node.click(); });+ menu.append(option);+ }+ element.append(menu);+ menu.style.left = `${Math.max(0, Math.min(event.clientX, window.innerWidth - menu.offsetWidth - 8))}px`;+ menu.style.top = `${Math.max(0, Math.min(event.clientY, window.innerHeight - menu.offsetHeight - 8))}px`;+ overlapMenu = menu;+ menu.querySelector("button")?.focus();+ });rows.append(lane);semanticNodes = nextSemanticNodes;};@@ -652,118 +696,139 @@ export function createTimeline(store: Store): Timeline {const buildTrack = (snapshot: StudioSnapshot,track: StudioSnapshot["tracks"][number],- nextClipNodes: { node: HTMLElement; start: number; end: number; id: string }[],+ nextClipNodes: { node: HTMLElement; start: number; end: number; id: string; selectionGroup?: string }[],attached = false,): void => {+ const trackBands = track.binding.bands ?? [];+ const banded = trackBands.length > 0;+ const layout = [+ ...trackBands.filter(band => band.placement === "before"),+ { id: undefined, heightPx: track.binding.lane.heightPx, display: "content" as const, tone: track.binding.tone },+ ...trackBands.filter(band => band.placement === "after"),+ ];+ const totalHeight = layout.reduce((sum, band) => sum + band.heightPx, 0)+ + (banded ? 2 * itemMetrics.insetYPx : 0);const tone = track.binding.tone;- const laneHeight = track.binding.lane.heightPx;const displayedItems = track.clips.length;const label = createTrackLabel(displayTrackName(track),tone,track.binding.icon,`${displayedItems} item${displayedItems === 1 ? "" : "s"}`,- laneHeight,+ totalHeight,attached,);label.classList.add(`track-facet-${track.binding.facet}`);labels.append(label);- const lane = document.createElement("div");- lane.className = `lane track-tone-${tone} track-facet-${track.binding.facet}${attached ? " lane-attached" : ""}`;- lane.style.height = `${laneHeight}px`;- lane.style.setProperty("--lane-height", `${laneHeight}px`);- const laneWidth = lanes.clientWidth;+ const trackBody = document.createElement("div");+ trackBody.className = banded ? "timeline-track timeline-track-banded" : "timeline-track";+ trackBody.dataset.track = track.id;+ trackBody.style.height = `${totalHeight}px`;const materialMounts: {readonly target: HTMLElement;readonly preview: Extract<StudioSnapshot["tracks"][number]["clips"][number]["display"]["layers"][number], { readonly kind: "preview" }>["preview"];}[] = [];- // Stable order preserves Companion projection order when stack levels tie.- for (const clip of [...track.clips].sort((a, b) => a.stackOrder - b.stackOrder)) {- const from = place(clip.startFrame, snapshot.space.frameCount, zoom.window());- const to = place(clip.endFrameExclusive, snapshot.space.frameCount, zoom.window());- if (to <= 0 || from >= 1) continue;- const node = document.createElement("button");- node.type = "button";- node.className = `clip clip-tone-${tone} clip-facet-${track.binding.facet} clip-chrome-${clip.presentation.chrome}`;- node.classList.toggle("clip-editable", clip.editHandles.some((handle) => handle.enabled));- node.dataset.clip = clip.id;- node.setAttribute("aria-label", clip.display.title);- node.style.left = `${from * 100}%`;- node.style.width = `max(2px, calc(${Math.max(0, to - from) * 100}% - ${itemMetrics.gapPx}px))`;- const visibleWidthPx = visibleItemWidth(from, to, laneWidth);- node.classList.toggle("clip-preview-wide", visibleWidthPx >= 92);- node.title = `${clip.display.title} · ${clip.startFrame}-${clip.endFrameExclusive}f`;- node.innerHTML = `<span class="clip-head"><span class="clip-name"></span><span class="clip-meta"></span></span><span class="clip-body"><span class="clip-layers" aria-hidden="true"></span><span class="clip-phases"></span></span><span class="clip-selection" aria-hidden="true"></span>`;- const layers = node.querySelector<HTMLElement>(".clip-layers")!;- clip.display.layers.forEach((layer, index) => {- const layerNode = document.createElement("span");- layerNode.className = `clip-layer clip-layer-${layer.kind} clip-layer-role-${layer.role}`;- layerNode.style.zIndex = String(index + 1);- if (layer.kind === "text") {- layerNode.textContent = layer.text;- } else {- layerNode.classList.add(`clip-layer-layout-${layer.layout}`);- materialMounts.push({ target: layerNode, preview: layer.preview });+ for (const band of layout) {+ const tone = band.tone ?? track.binding.tone;+ const laneHeight = band.heightPx;+ const lane = document.createElement("div");+ lane.className = `lane track-tone-${tone} track-facet-${track.binding.facet}${attached ? " lane-attached" : ""}`;+ lane.style.height = `${laneHeight}px`;+ if (banded) lane.classList.add("track-band", `track-band-${band.display}`);+ if (band.id !== undefined) lane.dataset.band = band.id;+ lane.style.setProperty("--lane-height", `${laneHeight}px`);+ const laneWidth = lanes.clientWidth;+ // Stable order preserves Companion projection order when stack levels tie.+ for (const clip of track.clips.filter(clip => clip.band === band.id).sort((a, b) => a.stackOrder - b.stackOrder)) {+ const from = place(clip.startFrame, snapshot.space.frameCount, zoom.window());+ const to = place(clip.endFrameExclusive, snapshot.space.frameCount, zoom.window());+ if (to <= 0 || from >= 1) continue;+ const node = document.createElement("button");+ node.type = "button";+ node.className = `clip clip-tone-${tone} clip-facet-${track.binding.facet} clip-chrome-${clip.presentation.chrome}`;+ node.classList.toggle("clip-editable", clip.editHandles.some((handle) => handle.enabled));+ node.dataset.clip = clip.id;+ node.setAttribute("aria-label", clip.display.title);+ node.style.left = `${from * 100}%`;+ node.style.width = `max(2px, ${Math.max(0, to - from) * 100}%)`;+ const visibleWidthPx = visibleItemWidth(from, to, laneWidth);+ node.classList.toggle("clip-preview-wide", visibleWidthPx >= 92);+ node.title = `${clip.display.title} · ${clip.startFrame}-${clip.endFrameExclusive}f`;+ node.innerHTML = `<span class="clip-head"><span class="clip-name"></span><span class="clip-meta"></span></span><span class="clip-body"><span class="clip-layers" aria-hidden="true"></span><span class="clip-phases"></span></span><span class="clip-boundary" aria-hidden="true"></span><span class="clip-selection" aria-hidden="true"></span>`;+ const layers = node.querySelector<HTMLElement>(".clip-layers")!;+ clip.display.layers.forEach((layer, index) => {+ const layerNode = document.createElement("span");+ layerNode.className = `clip-layer clip-layer-${layer.kind} clip-layer-role-${layer.role}`;+ layerNode.style.zIndex = String(index + 1);+ if (layer.kind === "text") {+ layerNode.textContent = layer.text;+ } else {+ layerNode.classList.add(`clip-layer-layout-${layer.layout}`);+ materialMounts.push({ target: layerNode, preview: layer.preview });+ }+ layers.append(layerNode);+ });+ const phaseLayer = node.querySelector<HTMLElement>(".clip-phases")!;+ for (const phase of clip.temporal?.phases ?? []) {+ if (phase.endFrameExclusive <= clip.startFrame || phase.startFrame >= clip.endFrameExclusive) continue;+ const phaseNode = document.createElement("span");+ phaseNode.className = `clip-phase clip-phase-${phase.role}`;+ phaseNode.style.left = `${(phase.startFrame - clip.startFrame) / Math.max(1, clip.endFrameExclusive - clip.startFrame) * 100}%`;+ phaseNode.style.width = `${(phase.endFrameExclusive - phase.startFrame) / Math.max(1, clip.endFrameExclusive - clip.startFrame) * 100}%`;+ phaseLayer.append(phaseNode);}- layers.append(layerNode);- });- const phaseLayer = node.querySelector<HTMLElement>(".clip-phases")!;- for (const phase of clip.temporal?.phases ?? []) {- if (phase.endFrameExclusive <= clip.startFrame || phase.startFrame >= clip.endFrameExclusive) continue;- const phaseNode = document.createElement("span");- phaseNode.className = `clip-phase clip-phase-${phase.role}`;- phaseNode.style.left = `${(phase.startFrame - clip.startFrame) / Math.max(1, clip.endFrameExclusive - clip.startFrame) * 100}%`;- phaseNode.style.width = `${(phase.endFrameExclusive - phase.startFrame) / Math.max(1, clip.endFrameExclusive - clip.startFrame) * 100}%`;- phaseLayer.append(phaseNode);- }- const metaText = clip.presentation.chrome === "point"- ? `${(clip.startFrame / fps(snapshot)).toFixed(2)}s`- : `${((clip.endFrameExclusive - clip.startFrame) / fps(snapshot)).toFixed(2)}s`;- const headerLabel = node.querySelector<HTMLElement>(".clip-name")!;- node.querySelector(".clip-meta")!.textContent = metaText;- headerLabel.textContent = clip.display.title;- node.addEventListener("pointerdown", (event) => {- const rect = node.getBoundingClientRect();- const edge = Math.min(8, Math.max(4, rect.width / 3));- const nearStart = event.clientX - rect.left <= edge;- const nearEnd = rect.right - event.clientX <= edge;- const handle = clip.editHandles.find((candidate) =>- candidate.enabled- && ((candidate.gesture === "trim-start" && nearStart)- || (candidate.gesture === "trim-end" && nearEnd)- || (candidate.gesture === "move" && !nearStart && !nearEnd)));- if (handle !== undefined- && (handle.gesture === "move" || handle.gesture === "trim-start" || handle.gesture === "trim-end")) {- event.stopPropagation();- const pointerFrame = handle.coordinate === "semantic-anchor"- ? rawFrameAt(event.clientX)- : frameAt(event.clientX);- activeEdit = {- clip,- handle,- startFrame: pointerFrame,- pointerId: event.pointerId,- node,- originalLeft: node.style.left,- originalWidth: node.style.width,- };- node.classList.add("editing");- try { lanes.setPointerCapture(event.pointerId); } catch { /* local pointer */ }+ const metaText = clip.presentation.chrome === "point"+ ? `${(clip.startFrame / fps(snapshot)).toFixed(2)}s`+ : `${((clip.endFrameExclusive - clip.startFrame) / fps(snapshot)).toFixed(2)}s`;+ const headerLabel = node.querySelector<HTMLElement>(".clip-name")!;+ node.querySelector(".clip-meta")!.textContent = metaText;+ headerLabel.textContent = clip.display.title;+ node.addEventListener("pointerdown", (event) => {+ const rect = node.getBoundingClientRect();+ const edge = Math.min(8, Math.max(4, rect.width / 3));+ const nearStart = event.clientX - rect.left <= edge;+ const nearEnd = rect.right - event.clientX <= edge;+ const handle = clip.editHandles.find((candidate) =>+ candidate.enabled+ && ((candidate.gesture === "trim-start" && nearStart)+ || (candidate.gesture === "trim-end" && nearEnd)+ || (candidate.gesture === "move" && !nearStart && !nearEnd)));+ if (handle !== undefined+ && (handle.gesture === "move" || handle.gesture === "trim-start" || handle.gesture === "trim-end")) {+ event.stopPropagation();+ const pointerFrame = handle.coordinate === "semantic-anchor"+ ? rawFrameAt(event.clientX)+ : frameAt(event.clientX);+ activeEdit = {+ clip,+ handle,+ startFrame: pointerFrame,+ pointerId: event.pointerId,+ node,+ originalLeft: node.style.left,+ originalWidth: node.style.width,+ };+ node.classList.add("editing");+ try { lanes.setPointerCapture(event.pointerId); } catch { /* local pointer */ }+ store.select(clip.id, "timeline");+ return;+ }store.select(clip.id, "timeline");- return;- }- store.select(clip.id, "timeline");- });- node.addEventListener("dblclick", () => store.seek(clip.startFrame, "timeline"));- nextClipNodes.push({ node, start: clip.startFrame, end: clip.endFrameExclusive, id: clip.id });- lane.append(node);+ });+ node.addEventListener("dblclick", () => store.seek(clip.startFrame, "timeline"));+ nextClipNodes.push({ node, start: clip.startFrame, end: clip.endFrameExclusive, id: clip.id, ...(clip.selectionGroup === undefined ? {} : { selectionGroup: clip.selectionGroup }) });+ lane.append(node);+ }+ trackBody.append(lane);}- rows.append(lane);+ rows.append(trackBody);+ // Cached storyboards render synchronously and need connected, measurable targets.for (const item of materialMounts) mountMaterialPreview(item.target, item.preview);};const build = (snapshot: StudioSnapshot): void => {+ closeOverlapMenu();const scrollTop = body.scrollTop;labels.replaceChildren();rows.replaceChildren();@@ -771,9 +836,9 @@ export function createTimeline(store: Store): Timeline {const corner = document.createElement("div");corner.className = "label-corner";corner.textContent = "";- labels.append(corner);+ if (snapshot.semantic === undefined || snapshot.semantic.segments.length === 0) labels.append(corner);buildSemanticLane(snapshot);- const nextClipNodes: { node: HTMLElement; start: number; end: number; id: string }[] = [];+ const nextClipNodes: { node: HTMLElement; start: number; end: number; id: string; selectionGroup?: string }[] = [];for (const track of snapshot.tracks) {const attachedTo = track.binding.lane.attachedTo;if (attachedTo !== undefined) continue;@@ -804,8 +869,10 @@ export function createTimeline(store: Store): Timeline {playhead.classList.toggle("outside", position < 0 || position > lanes.clientWidth);playheadGrip.classList.toggle("outside", position < 0 || position > lanes.clientWidth);timelineTime.textContent = frameTimecode(snapshot, head.frame);+ const chosen = selection.kind === "clip" ? store.clip(selection.clipId) : undefined;for (const item of clipNodes) {- const selected = selection.kind === "clip" && selection.clipId === item.id;+ const selected = chosen !== undefined && (chosen.id === item.id+ || (chosen.selectionGroup !== undefined && chosen.selectionGroup === item.selectionGroup));item.node.classList.toggle("selected", selected);item.node.setAttribute("aria-pressed", String(selected));item.node.classList.toggle("live", head.frame >= item.start && head.frame < item.end);@@ -815,7 +882,8 @@ export function createTimeline(store: Store): Timeline {(item.kind === "segment" || item.kind === "selection")&& head.frame >= item.start && head.frame < item.end);item.node.classList.toggle("current", item.kind === "word" && head.frame >= item.start && head.frame < item.end);- const selected = (item.kind === "segment"+ const selected = (item.kind === "word" && selectedWord === item.id && selection.kind === "none")+ || (item.kind === "segment"&& selection.kind === "semantic-segment" && selection.segmentId === item.id)|| (item.kind === "selection"&& selection.kind === "semantic-selection" && selection.selectionId === item.id)@@ -862,6 +930,7 @@ export function createTimeline(store: Store): Timeline {zoom.subscribe(scheduleBuild);store.subscribe((value) => {state = value;+ if (value.selection.kind !== "none") selectedWord = undefined;if (value.snapshot.revision !== built) {built = value.snapshot.revision;build(value.snapshot);diff --git a/packages/studio/test/attribute-groups.test.ts b/packages/studio/test/attribute-groups.test.tsnew file mode 100644index 00000000..4b75323f--- /dev/null+++ b/packages/studio/test/attribute-groups.test.ts@@ -0,0 +1,64 @@+import assert from "node:assert/strict";+import test from "node:test";+import { parseStructuredElement } from "@hypit/markup";+import { audioTrackModuleRef } from "@hypit/audio-track";+import type { StudioPlacement, StudioEntityDraft, StudioTrackCompanion } from "@hypit/studio-adapter";+import { audioTrackStudioTrackCompanions } from "../../audio-track-studio/src/index.js";+import { sourceBindingsForDraft, inspectorFieldsForBindings } from "../src/parameters.js";+import { serializeAttributeGroup, validateParameterValue } from "../src/parameter-values.js";++function fields(text: string, companion: Pick<StudioTrackCompanion, "bindings" | "inspector"> = audioTrackStudioTrackCompanions[0]!) {+ const element = parseStructuredElement({ name: "main.svml", text }, 0).element;+ const placement = { ...element, id: "music", tag: element.name, surface: "item", module: audioTrackModuleRef,+ sourcePath: "main.svml", references: [], referenceAttributes: {}, records: [], outputs: [], outputPorts: [], values: [], children: [],+ } as unknown as StudioPlacement;+ const draft: StudioEntityDraft = { id: "music", authoredId: "music", display: { title: "Music", layers: [] }, startFrame: 0, endFrameExclusive: 90, stackOrder: 0 };+ return inspectorFieldsForBindings(draft, sourceBindingsForDraft({ root: "/project", files: [{ path: "main.svml", text, language: "svml" }],+ placement, draft, declarations: companion.bindings! }), companion.inspector!);+}++test("a grouped mode edit adds and removes dependent attributes without changing other author facts", () => {+ let text = '<a:Item id="music" source={song.media} gain="0.4"/>';+ let field = fields(text).find(field => field.binding === "playback-settings")!;+ assert.deepEqual(field.value, { playback: "once" });+ const stretch = { playback: "stretch", "min-rate": 0.8, "max-rate": 1.2 };+ validateParameterValue(stretch, field.schema!, "Playback");+ assert.throws(() => validateParameterValue({ playback: "stretch" }, field.schema!, "Playback"));+ text = serializeAttributeGroup(field, stretch);+ field = fields(text).find(field => field.binding === "playback-settings")!;+ assert.deepEqual(field.value, stretch);+ text = serializeAttributeGroup(field, { playback: "loop" });+ assert.equal(text, '<a:Item playback="loop" id="music" source={song.media} gain="0.4"/>');+ for (let count = 0; count < 3; count++) {+ field = fields(text).find(field => field.binding === "playback-settings")!;+ text = serializeAttributeGroup(field, stretch);+ field = fields(text).find(field => field.binding === "playback-settings")!;+ text = serializeAttributeGroup(field, { playback: "loop" });+ }+ assert.equal(text, '<a:Item playback="loop" id="music" source={song.media} gain="0.4"/>');+});++test("ordinary Audio defaults have real first-edit endpoints", () => {+ const held = fields('<a:Item id="music" source={song.media}/>');+ assert.deepEqual(held.filter(field => ["gain", "fade-in", "fade-out"].includes(field.binding!)).map(field => [field.binding, field.value, field.edit?.source.prefix]),+ [["gain", 1, ' gain="'], ["fade-in", "0f", ' fade-in="'], ["fade-out", "0f", ' fade-out="']]);+});+++test("record attributes decode declared booleans and preserve strings before a grouped write", () => {+ const companion: Pick<StudioTrackCompanion, "bindings" | "inspector"> = {+ bindings: [{ name: "layout", attributes: ["enabled", "columns", "align"], writable: true,+ schema: { kind: "object", fields: {+ enabled: { schema: { kind: "boolean" } },+ columns: { schema: { kind: "number", minimum: 1 } },+ align: { schema: { kind: "string", enum: ["left", "right"] } },+ } } }],+ inspector: [{ binding: "layout", label: "Layout", domain: "where",+ section: { id: "layout", label: "Layout" }, control: "record" }],+ };+ const field = fields('<custom:Item id="music" enabled="false" columns="2" align="left"/>', companion)[0]!;+ assert.deepEqual(field.value, { enabled: false, columns: 2, align: "left" });+ const next = { enabled: true, columns: 3, align: "right" };+ validateParameterValue(next, field.schema!, "Layout");+ assert.deepEqual(fields(serializeAttributeGroup(field, next), companion)[0]!.value, next);+});diff --git a/packages/studio/test/audio-preview.test.ts b/packages/studio/test/audio-preview.test.tsnew file mode 100644index 00000000..5f0a5da4--- /dev/null+++ b/packages/studio/test/audio-preview.test.ts@@ -0,0 +1,59 @@+import assert from "node:assert/strict";+import test from "node:test";+import vm from "node:vm";+import { injectRuntimeShim } from "../src/preview/runtime-shim.js";++class Param {+ events: { kind: "set" | "ramp"; value: number; time: number }[] = [];+ cancelScheduledValues(time: number) { this.events = this.events.filter(event => event.time < time); }+ setValueAtTime(value: number, time: number) { this.events.push({ kind: "set", value, time }); }+ linearRampToValueAtTime(value: number, time: number) { this.events.push({ kind: "ramp", value, time }); }+}+function player(gain: number, presentation: object = {}) {+ const nodes: { gain: Param; connect: () => void }[] = [];+ const attrs: Record<string,string> = {+ "data-start": "0", "data-duration": "4", "data-media-start": "0", "data-media-end": "4",+ "data-gain": String(gain), "data-presentation": encodeURIComponent(JSON.stringify(presentation)),+ };+ const audio = { getAttribute: (name: string) => attrs[name] ?? null, currentTime: 0, readyState: 4,+ paused: true, volume: 0, muted: false, pause() { this.paused = true; },+ play() { this.paused = false; return Promise.resolve(); }, addEventListener() {}, removeEventListener() {} };+ const root = { style: {}, getAttribute: (name: string) => ({ "data-fps": "30", "data-composition-id": "test" })[name] ?? null };+ const window: Record<string, any> = { addEventListener() {}, dispatchEvent() {} };+ const html = injectRuntimeShim("").trim();+ vm.runInNewContext(html.slice("<script>".length, -"</script>".length), {+ window, document: { querySelector: () => root, querySelectorAll: (selector: string) => selector === ".hypit-studio-audio" ? [audio] : [] },+ CustomEvent: class {}, AudioContext: class {+ currentTime = 10; destination = {}; state = "running";+ createMediaElementSource() { return { connect() {} }; }+ createGain() { const node = { gain: new Param(), connect() {} }; nodes.push(node); return node; }+ },+ });+ return { audio, nodes, play: (frame: number) => window.__hypitPlayFrame(frame) };+}++test("preview preserves silence and gain above unity through the audio graph", async () => {+ for (const gain of [0, 0.3, 2]) {+ const p = player(gain); await p.play(30);+ assert.equal(p.audio.volume, 1);+ assert.equal(p.nodes[0]!.gain.events.at(-1)!.value, gain);+ }+});++test("preview schedules independent envelopes, fades and half-open audible regions on the sample clock", async () => {+ const p = player(2, {+ fadeInSamples: 96000, fadeOutSamples: 96000,+ gainEnvelope: [{ sample: 0, gain: 1 }, { sample: 192000, gain: 0 }],+ audibility: [{ startSample: 48000, endSampleExclusive: 96000 }, { startSample: 144000, endSampleExclusive: 192000 }],+ });+ await p.play(30);+ const time = 1 + 1/60;+ assert.equal(p.nodes[1]!.gain.events[0]!.value, 1 - time/4);+ assert.equal(p.nodes[2]!.gain.events[0]!.value, time/2);+ assert.equal(p.nodes[3]!.gain.events[0]!.value, 1);+ assert.deepEqual(p.nodes[4]!.gain.events.map(event => event.value), [1, 0, 1, 0]);+ assert.ok(p.nodes[1]!.gain.events.some(event => event.kind === "ramp"));+ await p.play(60);+ assert.equal(p.nodes.length, 5, "reuse one audio graph across frames");+ assert.equal(p.nodes[4]!.gain.events[0]!.value, 0, "gap is silent");+});diff --git a/packages/studio/test/material-companions.test.ts b/packages/studio/test/material-companions.test.tsnew file mode 100644index 00000000..bc7e4720--- /dev/null+++ b/packages/studio/test/material-companions.test.ts@@ -0,0 +1,55 @@+import assert from "node:assert/strict";+import test from "node:test";+import { compositionTypes } from "@hypit/composition";+import { audioTrackTypes } from "@hypit/audio-track";+import { mediaTrackTypes } from "@hypit/media-track";+import type { StudioTrackCompanionContext } from "@hypit/studio-adapter";+import { audioTrackStudioTrackCompanions } from "../../audio-track-studio/src/index.js";+import { mediaTrackStudioTrackCompanions } from "../../media-track-studio/src/index.js";++for (const kind of ["audio", "media"] as const) {+ test(`${kind} Item titles use explicit author names or exact source references without changing identity`, () => {+ const sourceType = kind === "audio" ? audioTrackTypes.clipSpec : mediaTrackTypes.itemSpec;+ const subjects = ["my-chosen-name", "track.item.0002", "track.item.0003"];+ const child = (id: string, index: number) => ({+ ...(index === 0 ? { id } : {}),+ attributes: index === 0 ? { id } : {},+ referenceAttributes: { [kind === "audio" ? "source" : "image"]: "assets.shared.material" },+ range: { start: index * 10, end: index * 10 + 9 },+ values: [{ type: sourceType, value: { id } }],+ });+ const program = {+ items: subjects.map((id, index) => ({+ id: `projected.${index}`, subjectId: id,+ window: { startFrame: index * 10, endFrameExclusive: index * 10 + 20 },+ span: { startFrame: index * 10, endFrameExclusive: index * 10 + 20 },+ stacking: { order: index },+ source: { artifact: { resource: "res_audio" } },+ layers: [{ kind: "sample", id: "picture", source: { kind: "still", artifact: { resource: "res_image" } } }],+ })), sequences: [],+ };+ const context = {+ track: { outputRef: `track.${kind}`, typeRef: kind === "audio" ? compositionTypes.audioTrack : compositionTypes.visualTrack,+ trace: { outputPorts: [{ name: "program", ref: "track.program" }] } },+ placement: { children: subjects.map(child) },+ values: new Map([["track.program", program]]), spans: [], temporalBindings: [],+ } as unknown as StudioTrackCompanionContext;+ const companion = kind === "audio" ? audioTrackStudioTrackCompanions[0]!+ : mediaTrackStudioTrackCompanions.find((entry) => entry.id === "visual")!;+ const entities = companion.project!(context);+ assert.deepEqual(entities.map((entry) => entry.display.title), ["my-chosen-name", "assets.shared.material", "assets.shared.material"]);+ assert.deepEqual(entities.map((entry) => entry.authoredId), subjects);+ assert.equal(new Set(entities.map((entry) => entry.id)).size, 3);+ assert.deepEqual(entities.map((entry) => entry.elementRange), subjects.map(child).map((entry) => entry.range));+ assert.deepEqual(entities.map((entry) => [entry.startFrame, entry.endFrameExclusive]), [[0, 20], [10, 30], [20, 40]]);+ });+}++test("audio playback is one attribute group and gain stays a bounded mixing control", () => {+ const companion = audioTrackStudioTrackCompanions[0]!;+ const group = companion.bindings!.find(binding => binding.name === "playback-settings")!;+ assert.deepEqual(group.attributes, ["playback", "min-rate", "max-rate"]);+ assert.equal(companion.inspector!.find(field => field.binding === group.name)?.domain, "when");+ const gain = companion.inspector!.find(field => field.binding === "gain")!;+ assert.equal(gain.domain, "how"); assert.equal(gain.number?.scale, 100); assert.equal(gain.number?.maximum, 6400);+});diff --git a/packages/studio/test/parameter-companions.test.ts b/packages/studio/test/parameter-companions.test.tsnew file mode 100644index 00000000..65d3500f--- /dev/null+++ b/packages/studio/test/parameter-companions.test.ts@@ -0,0 +1,41 @@+import assert from "node:assert/strict";+import test from "node:test";+import type { StudioPlacement, StudioEntityDraft, StudioParameterCompanion } from "@hypit/studio-adapter";+import { studioContributionFromPackage, createStudioCompanionHostFacet } from "@hypit/studio-adapter";+import { composeParameterDeclarations, sourceBindingsForDraft, inspectorFieldsForBindings } from "../src/parameters.js";+import { StudioCompanionRegistry } from "../src/studio-registry.js";+import { parameterAuthorValue, serializeParameterValue } from "../src/parameter-values.js";++const module = { name: "@project/pan", version: "1" } as const;+const companion: StudioParameterCompanion = {+ id: "pan", match: { module, surface: "pan" },+ bindings: [{ name: "distance", writable: true, fallback: 0.25 }],+ inspector: [{ binding: "distance", label: "Travel", domain: "where", section: { id: "path", label: "Path" },+ control: "number", number: { scale: 100 }, unit: "%" }],+};+const text = '<x:Track id="view" style={pan}/><p:Pan id="pan"/>';+function placement(id: string, start: number, end: number): StudioPlacement {+ return { id, module, surface: id === "pan" ? "pan" : "track", tag: id === "pan" ? "p:Pan" : "x:Track",+ sourcePath: "main.svml", range: { start, end }, records: [id], values: [], outputs: [], outputPorts: [], children: [],+ attributes: { id }, attributeValueRanges: {}, references: [], referenceAttributes: {}, referenceTypes: {} };+}+test("a project object contributes its own parameters to a consumer and inserts an omitted scalar on first edit", () => {+ const contribution = studioContributionFromPackage("@project/pan", [createStudioCompanionHostFacet({ parameters: [companion] })]);+ const registry = new StudioCompanionRegistry([], { parameters: contribution.parameters });+ const split = text.indexOf('<p:Pan');+ const owner = { ...placement("view", 0, split), referenceAttributes: { style: "pan" }, resolvedReferenceAttributes: { style: "pan" } };+ const style = placement("pan", split, text.length);+ const draft: StudioEntityDraft = { id: "use", authoredId: "view", display: { title: "Pan", layers: [] }, startFrame: 0, endFrameExclusive: 30, stackOrder: 0 };+ const declarations = composeParameterDeclarations({ registry, placement: owner, placements: [owner, style], draft,+ bindings: [{ name: "style", companion: true }], inspector: [] });+ const fields = inspectorFieldsForBindings(draft, sourceBindingsForDraft({ root: "/workspace", files: [{ path: "main.svml", text, language: "svml" }],+ placement: owner, placements: [owner, style], draft, declarations: declarations.bindings }), declarations.inspector);+ assert.equal(fields.length, 1);+ const field = fields[0]!;+ assert.equal(field.label, "Travel"); assert.equal(field.value, 0.25);+ const source = field.edit!.source;+ const written = source.prefix! + serializeParameterValue(parameterAuthorValue(field, 50), field.edit!.language) + source.suffix!;+ assert.equal(text.slice(0, source.range.start) + written + text.slice(source.range.end),+ '<x:Track id="view" style={pan}/><p:Pan distance="0.5" id="pan"/>');+ assert.equal(registry.parameterCompanionFor({ name: "@other/pan", version: "1" }, "pan"), undefined);+});diff --git a/packages/studio/test/parameter-values.test.ts b/packages/studio/test/parameter-values.test.tsindex 1d57606c..ac65e9d8 100644--- a/packages/studio/test/parameter-values.test.ts+++ b/packages/studio/test/parameter-values.test.ts@@ -1,11 +1,11 @@import assert from "node:assert/strict";import test from "node:test";import type { StudioInspectorField } from "@hypit/studio-adapter";-import { parameterAuthorValue, parameterNumber, parameterOption, serializeParameterValue, validateParameterValue } from "../src/parameter-values.js";+import { parameterAuthorValue, parameterControlForSchema, parameterNumber, parameterOption, serializeParameterValue, validateParameterValue } from "../src/parameter-values.js";const field = (overrides: Partial<StudioInspectorField>): StudioInspectorField => ({id: "width", binding: "frame.width", label: "Width", domain: "where", section: { id: "size", label: "Size" },- control: "number", value: "78%", language: "svml", source: { path: "main.svml", range: { start: 0, end: 3 }, preimage: "78%" },+ control: "number", value: "78%", edit: { language: "svml", source: { path: "main.svml", range: { start: 0, end: 3 }, preimage: "78%" } },number: { suffixes: ["%", "px"] }, ...overrides,});@@ -23,7 +23,7 @@ test("numeric editing separates units and restores the authored unit without con});test("percentage display and schema validation apply on opposite sides of the translation", () => {- const opacity = field({ value: .78, language: "svs", unit: "%", number: { scale: 100, minimum: 0, maximum: 100 },+ const opacity = field({ value: .78, unit: "%", number: { scale: 100, minimum: 0, maximum: 100 },schema: { kind: "number", minimum: 0, maximum: 1 } });assert.equal(parameterNumber(opacity.value, opacity.number).value, 78);assert.equal(parameterNumber(.55, opacity.number).value, 55);@@ -48,3 +48,13 @@ test("SVML parameter text escapes attribute delimiters while SVS keeps its own cassert.equal(serializeParameterValue(text, "svml"), "Say "hello" & <look> at Bob's");assert.equal(serializeParameterValue(text, "svs"), JSON.stringify(text));});+++test("top-level and nested parameter fields share schema-to-control selection", () => {+ assert.equal(parameterControlForSchema({ kind: "string", enum: ["left", "right"] }), "select");+ assert.equal(parameterControlForSchema({ kind: "string", format: "color" }), "color");+ assert.equal(parameterControlForSchema({ kind: "boolean" }), "boolean");+ assert.equal(parameterControlForSchema({ kind: "number" }), "number");+ assert.equal(parameterControlForSchema({ kind: "object", fields: {} }), "record");+ assert.equal(parameterControlForSchema({ kind: "array", items: { kind: "string" } }), "list");+});diff --git a/packages/studio/test/parameters.test.ts b/packages/studio/test/parameters.test.tsindex f9d8051c..b1e8d368 100644--- a/packages/studio/test/parameters.test.ts+++ b/packages/studio/test/parameters.test.ts@@ -278,7 +278,7 @@ test("nested declared references reach the font attribute, not the referring Sty});test("a derived entity follows its actual Style and Companion-owned Recipe presentation", () => {- const main = "<caption-fine:Track id=\"captions\" program={caption-program}/>";+ const main = "<scene:Track id=\"captions\" layout={baseline-layout}/>";const sheet = `<sheet version="1">caption.alt { x: 0.4; handoff: overlap; colors: ["#FF3F56", "#FFA72D"]; }</sheet>`;@@ -304,12 +304,12 @@ test("a derived entity follows its actual Style and Companion-owned Recipe prese});const track = placement({id: "captions", module: "@hypit/caption-fine", surface: "track",- references: { program: "caption-program" },- resolvedReferences: { program: "main::record::caption-program" },+ references: { layout: "baseline-layout" },+ resolvedReferences: { layout: "main::record::baseline-layout" },});- const program = placement({- id: "caption-program", module: "@hypit/caption", surface: "program",- references: {}, outputs: ["main::record::caption-program"],+ const layout = placement({+ id: "baseline-layout", module: "@example/layout", surface: "layout",+ references: {}, outputs: ["main::record::baseline-layout"],});const style = placement({id: "alternate-caption", module: "@hypit/caption-fine", surface: "style",@@ -339,7 +339,7 @@ test("a derived entity follows its actual Style and Companion-owned Recipe preseid: "captions:cue:2", authoredId: "captions", presentId: "cue:2", display: { title: "cue:2", layers: [] },startFrame: 0, endFrameExclusive: 10, stackOrder: 70,elementRange: track.range,- parameterReferences: { program: "alternate-caption" },+ parameterReferences: { layout: "alternate-caption" },} as const;const parameters = sourceBindingsForDraft({root: "/workspace",@@ -348,10 +348,10 @@ test("a derived entity follows its actual Style and Companion-owned Recipe prese{ path: "recipes.svs", text: sheet, language: "svs" },],placement: track,- placements: [track, program, style],+ placements: [track, layout, style],draft,declarations: [{- name: "program",+ name: "layout",recipe: {through: ["recipe"],bindings: [@@ -365,19 +365,19 @@ test("a derived entity follows its actual Style and Companion-owned Recipe prese});const inspector = inspectorFieldsForBindings(draft, parameters, [{- binding: "program.x", label: "X", domain: "where", page: { id: "placement", label: "Placement" },+ binding: "layout.x", label: "X", domain: "where", page: { id: "placement", label: "Placement" },section: { id: "region", label: "Region" }, control: "number",},{- binding: "program.handoff", label: "Handoff", domain: "when", page: { id: "cue", label: "Cue" },+ binding: "layout.handoff", label: "Handoff", domain: "when", page: { id: "cue", label: "Cue" },section: { id: "envelope", label: "Envelope" }, control: "select", options: ["cut", "overlap"],},{- binding: "program.colors", label: "Colors", domain: "how", page: { id: "paint", label: "Paint" },+ binding: "layout.colors", label: "Colors", domain: "how", page: { id: "paint", label: "Paint" },section: { id: "palette", label: "Palette" },},{- binding: "program.fallback-colors", label: "Fallback Colors", domain: "how", page: { id: "paint", label: "Paint" },+ binding: "layout.fallback-colors", label: "Fallback Colors", domain: "how", page: { id: "paint", label: "Paint" },section: { id: "palette", label: "Palette" },},]);@@ -385,14 +385,38 @@ test("a derived entity follows its actual Style and Companion-owned Recipe preseassert.deepEqual(inspector.map(({ binding, domain, section, control, options }) => ({binding, domain, section: section.id, control, options,})), [- { binding: "program.x", domain: "where", section: "region", control: "number", options: undefined },- { binding: "program.handoff", domain: "when", section: "envelope", control: "select", options: ["cut", "overlap"] },- { binding: "program.colors", domain: "how", section: "palette", control: "list", options: undefined },- { binding: "program.fallback-colors", domain: "how", section: "palette", control: "list", options: undefined },+ { binding: "layout.x", domain: "where", section: "region", control: "number", options: undefined },+ { binding: "layout.handoff", domain: "when", section: "envelope", control: "select", options: ["cut", "overlap"] },+ { binding: "layout.colors", domain: "how", section: "palette", control: "list", options: undefined },+ { binding: "layout.fallback-colors", domain: "how", section: "palette", control: "list", options: undefined },]);assert.deepEqual(inspector[2]?.value, ["#FF3F56", "#FFA72D"]);assert.deepEqual(inspector[3]?.value, ["#000000"]);- assert.equal(inspector[3]?.source.preimage, "");- assert.match(inspector[3]?.source.prefix ?? "", /fallback-colors/u);+ assert.equal(inspector[3]?.edit?.source.preimage, "");+ assert.match(inspector[3]?.edit?.source.prefix ?? "", /fallback-colors/u);assert.ok(parameters.every((parameter) => parameter.source.path === "recipes.svs"));});+++test("Inspector keeps selected read-only bindings and computed facts beside editable fields", () => {+ const source = { path: "main.svml", range: { start: 0, end: 4 }, preimage: "wide" };+ const fields = inspectorFieldsForBindings({+ id: "use", authoredId: "use", display: { title: "Use", layers: [] },+ startFrame: 15, endFrameExclusive: 90, stackOrder: 0,+ inspector: [{ id: "range", label: "Range", domain: "when", section: { id: "placement", label: "Placement" }, value: "15–90", unit: "f" }],+ }, [+ { id: "style", binding: "style", name: "style", value: "wide", language: "svml", writable: false, source },+ { id: "width", binding: "frame.width", name: "width", value: "78%", language: "svml", writable: true, source },+ { id: "private", binding: "private", name: "private", value: "unexposed", language: "svml", writable: false, source },+ ], [+ { binding: "style", label: "Style", domain: "how", section: { id: "style", label: "Style" }, control: "text" },+ { binding: "frame.width", label: "Width", domain: "where", section: { id: "placement", label: "Placement" }, control: "number" },+ ]);+ assert.equal(fields.length, 3);+ assert.equal(fields[0]?.value, "wide");+ assert.equal(fields[0]?.edit, undefined);+ assert.equal(fields[1]?.edit?.source, source);+ assert.equal(fields[2]?.value, "15–90");+ assert.equal(fields[2]?.edit, undefined);+ assert.equal(fields[2]?.binding, undefined);+});diff --git a/packages/studio/test/preflight.test.ts b/packages/studio/test/preflight.test.tsindex 28b5d575..4500c7a8 100644--- a/packages/studio/test/preflight.test.ts+++ b/packages/studio/test/preflight.test.ts@@ -3,7 +3,7 @@ import assert from "node:assert/strict";import test from "node:test";import { compositionTypes } from "@hypit/composition";-import { semanticTrackTypes } from "@hypit/semantic-track";+import { timelineTypes } from "@hypit/timeline";import type { StudioFilmCompanion, StudioPlacement } from "@hypit/studio-adapter";import type { CompiledSource } from "../src/compile.js";@@ -15,7 +15,7 @@ const filmModule = { name: "@example/film", version: "1" } as const;const companion: StudioFilmCompanion = {id: "film",match: { module: filmModule, surface: "film", outputType: compositionTypes.composition },- timeSources: [{ attribute: "semantic", type: semanticTrackTypes.track }, { attribute: "space", type: programSpaceTypes.programSpace }],+ timeSources: [{ attribute: "semantic", type: timelineTypes.track }, { attribute: "space", type: programSpaceTypes.programSpace }],tracks: { childSurface: "Track", sourceAttribute: "source", types: [compositionTypes.visualTrack] },};diff --git a/packages/studio/test/registry.test.ts b/packages/studio/test/registry.test.tsindex e0e15cb3..982be984 100644--- a/packages/studio/test/registry.test.ts+++ b/packages/studio/test/registry.test.ts@@ -20,3 +20,32 @@ test("Companion origin matching includes the exact module version", () => {[],), undefined);});++test("internal bands keep entities in their Track and own their Inspector bindings", () => {+ const companion: StudioTrackCompanion = {+ id: "cards", role: "track", family: "cards", tone: "blue",+ output: { type: compositionTypes.visualTrack },+ bands: [{ id: "rules", placement: "after", heightPx: 15, display: "label",+ bindings: [{ name: "style" }],+ inspector: [{ binding: "style", label: "Style", domain: "how", section: { id: "style", label: "Style" }, control: "text" }],+ }],+ attachments: [{ id: "objects", family: "objects", icon: "layers", facet: "visual", lane: { heightPx: 40 } }],+ project: () => [{ id: "rule", authoredId: "rule", band: "rules",+ display: { title: "Layout", layers: [] }, startFrame: 0, endFrameExclusive: 120, stackOrder: 0 }],+ };+ const registry = new StudioCompanionRegistry([companion]);+ const track = { typeRef: compositionTypes.visualTrack, outputRef: "cards.visual", name: "cards", type: "VisualTrack",+ candidateOrigin: "source", role: "track", value: {}, trace: { references: [], outputPorts: [] } } as const;+ assert.deepEqual(registry.trackAttachments(track).map(item => item.attachmentId), ["objects"]);+ assert.deepEqual(registry.bindTrack(track).bands, [{ id: "rules", placement: "after", heightPx: 15, display: "label" }]);+ assert.deepEqual(registry.bindingDeclarations(track, undefined, undefined, "rules"), [{ name: "style" }]);+ assert.equal(registry.inspectorDeclarations(track, undefined, undefined, "rules")[0]?.label, "Style");+ const context = { track, values: new Map(), spans: [], temporalBindings: [], semantic: undefined, generic: () => [] };+ const [draft] = registry.projectTrack(context);+ assert.equal(draft?.lane, undefined);+ assert.equal(draft?.band, "rules");+ const invalid = new StudioCompanionRegistry([{ ...companion, project: () => [{ ...draft!, band: "missing" }] }]);+ assert.throws(() => invalid.projectTrack(context), /declared band/);+ const ambiguous = new StudioCompanionRegistry([{ ...companion, project: () => [{ ...draft!, lane: "objects" }] }]);+ assert.throws(() => ambiguous.projectTrack(context), /declared band/);+});diff --git a/packages/studio/test/semantic-gesture.test.ts b/packages/studio/test/semantic-gesture.test.tsindex bed257dd..45fbe017 100644--- a/packages/studio/test/semantic-gesture.test.ts+++ b/packages/studio/test/semantic-gesture.test.ts@@ -81,7 +81,7 @@ test("clock expressions retain units until edited and edited values reproject tofor (const frameRate of [{ numerator: 30, denominator: 1 }, { numerator: 60, denominator: 1 }, { numerator: 30000, denominator: 1001 }]) {const space = { id: "clock", durationSec: 300 * frameRate.denominator / frameRate.numerator, frameRate };const locate = (value: string) => projectProgramInstant({- itemId: "point", subjectId: "point", space,+ itemId: "point", subjectId: "point", timeline: { ...space, items: [] },projection: parseTemporalInstant(value, "point"), authority: { kind: "parameter", binding: "instant", relation: "direct" },}).frame;assert.equal(locate("2s"), Math.round(2 * frameRate.numerator / frameRate.denominator));diff --git a/packages/studio/test/snapshot-range.test.ts b/packages/studio/test/snapshot-range.test.tsnew file mode 100644index 00000000..9a2ba351--- /dev/null+++ b/packages/studio/test/snapshot-range.test.ts@@ -0,0 +1,25 @@+import assert from "node:assert/strict";+import test from "node:test";+import { compositionTypes } from "@hypit/composition";+import { snapshot } from "../src/snapshot.js";+import type { Preview } from "../src/programme.js";+import { StudioCompanionRegistry } from "../src/studio-registry.js";+import { fallbackStudioTrackCompanions } from "../src/fallback-companions.js";++test("a pure-MG work keeps its declared extent even when an editor entity extends beyond it", () => {+ const built = { source: { observations: { placements: [], sourceMaps: [] } },+ tracks: [{ outputRef: "art.visual", name: "art.visual", typeRef: compositionTypes.visualTrack,+ trace: { outputPorts: [], references: [] }, candidateOrigin: "source", value: { presents: [{ id: "scene", span: { startFrame: 0, endFrameExclusive: 150 }, stacking: { order: 0 } }] } }],+ space: { id: "work", durationSec: 3, frameRate: { numerator: 30, denominator: 1 } },+ values: new Map(), temporalBindings: new Map(),+ } as unknown as Preview;+ const result = snapshot(new StudioCompanionRegistry(fallbackStudioTrackCompanions), built, {+ revision: 1, path: "main.svml", text: "", run: { path: "main.svrun", targets: [], satisfactions: [] },+ canvas: { width: 1080, height: 1920, clearColor: "#000000" }, frameRate: built.space.frameRate,+ preview: { kind: "hyperframes", srcdoc: "" }, workspaceRoot: "/project", sourceFiles: [], surfaces: {} as never,+ });+ assert.equal(result.space.frameCount, 90); assert.equal(result.space.durationSec, 3);+ assert.equal(result.tracks[0]!.clips[0]!.endFrameExclusive, 150);+ assert.equal(result.semantic, undefined);+ assert.equal("timing" in result.provenance, false);+});diff --git a/packages/studio/test/temporal-graph.test.ts b/packages/studio/test/temporal-graph.test.tsindex 7b90abb2..83cdca74 100644--- a/packages/studio/test/temporal-graph.test.ts+++ b/packages/studio/test/temporal-graph.test.ts@@ -17,7 +17,7 @@ const step = (id: string, owner: string, name: string, inputs: ProducerStep["inptest("Studio reads Instant lineage and author authority from executed graph edges", () => {const program = [- record("semantic", "@hypit/semantic-track", "SemanticTrack", { id: "speech" }),+ record("semantic", "@hypit/timeline", "Timeline", { id: "speech" }),record("moment", "@hypit/narrative", "NarrativeMoment", { id: "cue", anchorId: "cue-anchor" }),record("point-spec", "@hypit/temporal", "TemporalInstantSpec", {id: "deck.card", subjectId: "card", projection: { ref: "moment.cue" }, authority: { kind: "semantic", boundary: "cue" },diff --git a/packages/temporal-markup/EDITING.md b/packages/temporal-markup/EDITING.mdindex f3409586..83cc9381 100644--- a/packages/temporal-markup/EDITING.md+++ b/packages/temporal-markup/EDITING.md@@ -132,8 +132,8 @@ handles allow it, a choice lists the other anchors at the same frame; horizontalcannot distinguish them. That explicit choice uses the same Script adjustment operation as dragging.Segment overlap or gaps can change temporal order without changing Script order. Stop construction-must not assume contiguous Segments. The current SemanticTrack implementation still concatenates-Take durations; introducing explicit Take placement is separate work.+must not assume contiguous Segments. Timeline places each prepared Take independently; semantic+stops use those placed anchor frames, including coincident anchors from overlapping Takes.## Selection, projection, and consumptiondiff --git a/packages/temporal-markup/README.md b/packages/temporal-markup/README.mdindex f2b586b5..fd6e8794 100644--- a/packages/temporal-markup/README.md+++ b/packages/temporal-markup/README.md@@ -2,11 +2,11 @@Shared SVML author-time projection helpers for `@hypit/temporal`. The Surface lowers authoredSelection, Segment, Moment or explicit clock expressions to ordinary Instant projections and Window-composition. A domain consumer receives the projected values and explicit ProgramSpace; it does not+composition. A domain consumer receives the projected values and the shared Timeline; it does notlocate Script words or infer a semantic source inside its renderer.The package exports `createTemporalWindowProjection`, `createTemporalInstantProjection`,-`resolveTemporalContext`, `createTemporalSpace`, attribute vocabulary, and the exact duration/instant parsers. These are helpers for component+`resolveTemporalContext`, attribute vocabulary, and the exact duration/instant parsers. These are helpers for componentSurfaces, not standalone author tags or a new Track.The editing behavior is specified in [Author-directed time editing](EDITING.md).@@ -15,15 +15,15 @@ write target for each supported gesture.## Time context-A Track Surface can accept `semantic={speech.semantic}` or `space={animation}`. Resolve that choice-with `resolveTemporalContext({ element, resolveReference })`, and pass the result to-`createTemporalSpace({ id, element, ...context })`. Preserve its component and fragment drafts.-Supply `...context, space: time.space` to each temporal projection helper, and pass `time.space.ref`-to the domain Fragment. Its input Type is `programSpaceTypes.programSpace`.+A Track Surface accepts `timeline={program.timeline}`. Resolve it with+`resolveTemporalContext({ element, resolveReference })` and pass that context to each temporal+projection helper. Wire `context.timeline.ref` directly into the component Fragment's Timeline port.+Preserve the projections' returned records, components and fragments.-A semantic context locates Script references and supplies the performance's time axis. A declared-space supplies a clock for authored animation. Literal times and program boundaries work in either-context. A Script reference needs `semantic` to locate it; the renderer never invents that mapping.+The context contains one required `timeline` reference. Program time and Script references both+project against it. Pure animation uses a zero-Take Timeline with an authored extent. Script events+use the anchors provided by its placed Takes. Drawing Producers receive that same Timeline and the+projected Instants/Windows; they need no Script parser or separate time-range input.## Window forms@@ -70,10 +70,26 @@ can deliberately admit only that form; consumers need not expose unrelated tempoThe Surface supplies an author-facing `subjectId` for the actual item whose time is being projected,separately from graph-qualified operation ids. The graph wires the resulting Instant or Window and-ProgramSpace into the consumer. Keep projection outside the domain Producer: it consumes resolved+Timeline into the consumer. Keep projection outside the domain Producer: it consumes resolvedtime and implements its own schedule or state, while the shared temporal protocol retains where thattime came from. An outer lifetime and child activations are separate inputs when a component persistsbetween events.Script's `@`, `~@`, close markers and Moment syntax belong to `@hypit/script`; media playback belongsto `@hypit/media-track`; a graphic component's reveal or preset semantics belong to that component.++## Independently bound endpoints++For a Window whose endpoints refer to different semantic sources, use `start-source` and+`end-source`. The expression still states the kind and boundary; each binding supplies that source:++```svml+<example:Item start-source={story.segment.next} start="segment.start"+ end-source={story.segment.previous} end="segment.end"/>+```++This describes an overlapping physical interval without authoring a backwards Script Selection.+The same endpoint bindings accept Selections or Moments with their corresponding expressions.+`segment`, `selection` and `moment` remain convenient shared bindings when both expressions use the+same source. Explicit expressions retain local parameter writeback; these bindings do not move the+referenced Script anchors when the expression's offset is edited.diff --git a/packages/temporal-markup/package.json b/packages/temporal-markup/package.jsonindex c6fec5eb..f06671c5 100644--- a/packages/temporal-markup/package.json+++ b/packages/temporal-markup/package.json@@ -11,7 +11,7 @@"@hypit/elaborator": "workspace:*","@hypit/markup": "workspace:*","@hypit/narrative": "workspace:*",- "@hypit/semantic-track": "workspace:*",+ "@hypit/timeline": "workspace:*","@hypit/temporal": "workspace:*","@hypit/program-space": "workspace:*"}diff --git a/packages/temporal-markup/src/index.ts b/packages/temporal-markup/src/index.tsindex a1ccf22e..4a22f32b 100644--- a/packages/temporal-markup/src/index.ts+++ b/packages/temporal-markup/src/index.ts@@ -9,10 +9,8 @@ import type {SurfaceResolvedReference,} from "@hypit/markup";import { narrativeTypes } from "@hypit/narrative";-import { programSpaceTypes } from "@hypit/program-space";-import { createTemporalSpace } from "./space.js";export * from "./space.js";-import { semanticTrackTypes } from "@hypit/semantic-track";+import { timelineTypes } from "@hypit/timeline";import {temporalProducers,temporalTypes,@@ -41,6 +39,9 @@ export const temporalWindowAttributeVocabulary: readonly SurfaceAttributeVocabulsummary: "Sets a projected start expression; write it together with end." },{ name: "end", kind: "literal", required: false,summary: "Sets a projected end expression; write it together with start." },+ ...["start-source", "end-source"].map(name => ({ name, kind: "reference" as const, required: false,+ accepts: [narrativeTypes.selection, narrativeTypes.excerpt, narrativeTypes.moment],+ summary: "Binds this endpoint's semantic source independently; its expression determines the boundary." })),{ name: "selection", kind: "reference", required: false, accepts: [narrativeTypes.selection],summary: "Resolves selection.start or selection.end used by a projected start/end expression." },{ name: "segment", kind: "reference", required: false, accepts: [narrativeTypes.excerpt],@@ -205,12 +206,13 @@ function sourceReference(element: StructuredElement,kind: SourceKind,resolveReference: ResolveReference,+ attribute = kind as string,): SurfaceResolvedReference | undefined {if (kind === "program") return undefined;const expected = kind === "selection"? narrativeTypes.selection: kind === "segment" ? narrativeTypes.excerpt : narrativeTypes.moment;- return resolve(element.attributes[kind], `${element.name}.${kind}`, expected, resolveReference);+ return resolve(element.attributes[attribute], `${element.name}.${attribute}`, expected, resolveReference);}function projectionFragment(input: {@@ -224,8 +226,7 @@ function projectionFragment(input: {? [["instant", input.start] as const]: [["start", input.start] as const, ["end", input.end] as const];const inputs = [- { name: "space", type: programSpaceTypes.programSpace },- ...(endpoints.some(([, endpoint]) => endpoint.source !== "program") ? [{ name: "semantic", type: semanticTrackTypes.track }] : []),+ { name: "timeline", type: timelineTypes.track },...endpoints.map(([name]) => ({ name: `${name}-spec`, type: temporalTypes.instantSpec })),...endpoints.flatMap(([name, endpoint]) => endpoint.source === "program" ? [] : [{name: `${name}-${endpoint.source}`,@@ -241,7 +242,7 @@ function projectionFragment(input: {: endpoint.source === "segment" ? temporalProducers.projectSegmentInstant: temporalProducers.projectMomentInstant,inputs: {- ...(endpoint.source === "program" ? { space: fragmentInput("space") } : { semantic: fragmentInput("semantic") }),+ timeline: fragmentInput("timeline"),spec: fragmentInput(`${name}-spec`),...(endpoint.source === "program" ? {} : {[endpoint.source]: fragmentInput(`${name}-${endpoint.source}`),@@ -274,15 +275,10 @@ function projectionDraft(input: {readonly id: string;readonly subjectId?: string;readonly element: StructuredElement;- readonly semantic?: SurfaceResolvedReference;- readonly space?: SurfaceResolvedReference;+ readonly timeline: SurfaceResolvedReference;readonly start: InstantDraft;readonly end?: InstantDraft;}): TemporalMarkupProjection {- if ([input.start, input.end].some((endpoint) => endpoint !== undefined && endpoint.source !== "program") && input.semantic === undefined) {- throw new Error(`${input.element.name} needs semantic to resolve this Script reference.`);- }- const time = createTemporalSpace(input);const output = input.end === undefined ? "instant" : "window";const subjectId = input.subjectId ?? input.id;if (subjectId.length === 0) throw new Error("Temporal projection subjectId must not be empty.");@@ -291,8 +287,7 @@ function projectionDraft(input: {const records: SurfaceRecordDraft[] = [];const componentInputs: SurfaceComponentDraft["inputs"] extends infer _T? Record<string, SurfaceResolvedReference["ref"] | { readonly kind: "record"; readonly id: string }>- : never = { space: time.space.ref, ...(input.semantic === undefined ? {} : { semantic: input.semantic.ref }) };- if ([input.start, input.end].every((endpoint) => endpoint === undefined || endpoint.source === "program")) delete componentInputs.semantic;+ : never = { timeline: input.timeline.ref };const endpoints = input.end === undefined? [["instant", input.start] as const]: [["start", input.start] as const, ["end", input.end] as const];@@ -327,14 +322,14 @@ function projectionDraft(input: {}return {records,- components: [...time.components, {+ components: [{id: componentId,fragment: fragment.id,inputs: componentInputs,outputs: { [output]: `${input.id}.__temporal.${output}` },range: input.element.range,}],- fragments: [...time.fragments, fragment],+ fragments: [fragment],ref: { kind: "component-output", component: componentId, output },};}@@ -344,8 +339,7 @@ export function createTemporalWindowProjection(input: {readonly id: string;readonly subjectId?: string;readonly element: StructuredElement;- readonly semantic?: SurfaceResolvedReference;- readonly space?: SurfaceResolvedReference;+ readonly timeline: SurfaceResolvedReference;readonly resolveReference: ResolveReference;}): TemporalMarkupProjection {const { element, resolveReference } = input;@@ -422,13 +416,15 @@ export function createTemporalWindowProjection(input: {const endExpression = parseTemporalInstant(end, `${element.name}.end`);const startSource = expressionSource(startExpression);const endSource = expressionSource(endExpression);+ const startBinding = element.attributes["start-source"] === undefined ? startSource : "start-source";+ const endBinding = element.attributes["end-source"] === undefined ? endSource : "end-source";rejectUnusedTemporalAttributes(element, temporalWindowAttributeNames, ["start", "end",- ...(startSource === "program" ? [] : [startSource]),- ...(endSource === "program" ? [] : [endSource]),+ ...(startSource === "program" ? [] : [startBinding]),+ ...(endSource === "program" ? [] : [endBinding]),]);- const startReference = startSource === "program" ? undefined : sourceReference(element, startSource, resolveReference);- const endReference = endSource === "program" ? undefined : sourceReference(element, endSource, resolveReference);+ const startReference = sourceReference(element, startSource, resolveReference, startBinding);+ const endReference = sourceReference(element, endSource, resolveReference, endBinding);return projectionDraft({...input,start: {@@ -451,8 +447,7 @@ export function createTemporalInstantProjection(input: {readonly id: string;readonly subjectId?: string;readonly element: StructuredElement;- readonly semantic?: SurfaceResolvedReference;- readonly space?: SurfaceResolvedReference;+ readonly timeline: SurfaceResolvedReference;readonly resolveReference: ResolveReference;readonly semanticAttribute?: string;readonly boundaryAttribute?: string;diff --git a/packages/temporal-markup/src/space.ts b/packages/temporal-markup/src/space.tsindex 0f96e24a..61a476d7 100644--- a/packages/temporal-markup/src/space.ts+++ b/packages/temporal-markup/src/space.ts@@ -1,59 +1,28 @@-import { sealGraphFragment } from "@hypit/elaborator";import type { StructuredElement, SurfaceAttributeVocabulary, SurfaceResolvedReference } from "@hypit/markup";-import { programSpaceTypes } from "@hypit/program-space";-import { semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";+import { timelineTypes } from "@hypit/timeline";export type TemporalContext = {- readonly semantic?: SurfaceResolvedReference;- readonly space?: SurfaceResolvedReference;+ readonly timeline: SurfaceResolvedReference;};export const temporalContextAttributeVocabulary = [- { name: "semantic", kind: "reference", required: false, accepts: [semanticTrackTypes.track],- summary: "Uses the performance's time axis and resolves Script Selections and Moments." },- { name: "space", kind: "reference", required: false, accepts: [programSpaceTypes.programSpace],- summary: "Uses a declared film time axis for authored animation. Supply semantic or space." },+ { name: "timeline", kind: "reference", required: true, accepts: [timelineTypes.track],+ summary: "The complete film Timeline, with any placed material and semantic anchors." },] as const satisfies readonly SurfaceAttributeVocabulary[];-/** Select the author's film time source. */+/** One author-facing Timeline; projections share its range and evidence. */export function resolveTemporalContext(input: {readonly element: StructuredElement;readonly resolveReference: (path: string) => SurfaceResolvedReference | undefined;}): TemporalContext {const { element, resolveReference } = input;- if (Number(element.attributes.semantic !== undefined) + Number(element.attributes.space !== undefined) !== 1) {- throw new Error(`${element.name} requires one time source: semantic or space.`);- }- const name = element.attributes.semantic === undefined ? "space" : "semantic";- const raw = element.attributes[name];- if (typeof raw !== "object" || raw.kind !== "reference") throw new Error(`${element.name}.${name} must be a reference.`);+ const raw = element.attributes.timeline;+ if (typeof raw !== "object" || raw.kind !== "reference") throw new Error(`${element.name}.timeline must be a reference.`);const found = resolveReference(raw.path);- const expected = name === "semantic" ? semanticTrackTypes.track : programSpaceTypes.programSpace;+ const expected = timelineTypes.track;if (found === undefined || found.type.module.name !== expected.module.name|| found.type.module.version !== expected.module.version || found.type.name !== expected.name) {- throw new Error(`${element.name}.${name} must reference ${expected.name}.`);+ throw new Error(`${element.name}.timeline must reference a Timeline.`);}- return { [name]: found };-}--/** Give consumers a ProgramSpace while preserving semantic context for word-bound projections. */-export function createTemporalSpace(input: TemporalContext & { readonly id: string; readonly element: StructuredElement }) {- if (input.space !== undefined) return { records: [], components: [], fragments: [], space: input.space };- if (input.semantic === undefined) throw new Error(`${input.element.name} requires a film time source.`);- const fragment = sealGraphFragment({- inputs: [{ name: "semantic", type: semanticTrackTypes.track }],- operations: [{ id: "project-space", producer: semanticTrackProducers.projectProgramSpace,- inputs: { track: { kind: "fragment-input", name: "semantic" } }, result: { kind: "output", name: "space" } }],- exports: [{ name: "space", type: programSpaceTypes.programSpace,- root: { kind: "fragment-operation", operation: "project-space" } }],- });- const id = `${input.id}.__space`;- const space: SurfaceResolvedReference = { path: `${id}.space`, type: programSpaceTypes.programSpace,- ref: { kind: "component-output", component: id, output: "space" } };- return {- records: [],- components: [{ id, fragment: fragment.id, inputs: { semantic: input.semantic.ref }, outputs: { space: `${id}.space` }, range: input.element.range }],- fragments: [fragment],- space,- };+ return { timeline: found };}diff --git a/packages/temporal-markup/test/temporal-markup.test.ts b/packages/temporal-markup/test/temporal-markup.test.tsindex 195c168f..2903250e 100644--- a/packages/temporal-markup/test/temporal-markup.test.ts+++ b/packages/temporal-markup/test/temporal-markup.test.ts@@ -4,7 +4,7 @@ import test from "node:test";import type { MarkupAttributeValue, StructuredElement, SurfaceResolvedReference } from "@hypit/markup";import { narrativeTypes } from "@hypit/narrative";import { programSpaceTypes } from "@hypit/program-space";-import { semanticTrackTypes } from "@hypit/semantic-track";+import { timelineTypes } from "@hypit/timeline";import type { TemporalInstantSpec } from "@hypit/temporal";import { createTemporalInstantProjection, createTemporalWindowProjection, resolveTemporalContext } from "../src/index.js";@@ -15,7 +15,7 @@ const resolved = (path: string, type: SurfaceResolvedReference["type"]): Surfacepath, type, ref: { kind: "record", id: path },});-const semantic = resolved("semantic", semanticTrackTypes.track);+const timeline = resolved("timeline", timelineTypes.track);const references = new Map([["selection", resolved("selection", narrativeTypes.selection)],["segment", resolved("segment", narrativeTypes.excerpt)],@@ -31,21 +31,21 @@ const specs = (projection: ReturnType<typeof createTemporalWindowProjection>) =>test("the author bridge owns the four Window forms and the explicit Instant fallback", () => {assert.deepEqual(specs(createTemporalWindowProjection({- id: "during", element: element({ during: reference("selection") }), semantic, resolveReference,+ id: "during", element: element({ during: reference("selection") }), timeline, resolveReference,})).map((spec) => spec?.authority), [{ kind: "semantic", boundary: "start" },{ kind: "semantic", boundary: "end" },]);assert.deepEqual(specs(createTemporalWindowProjection({- id: "at", element: element({ at: reference("moment"), for: "8f" }), semantic, resolveReference,+ id: "at", element: element({ at: reference("moment"), for: "8f" }), timeline, resolveReference,})).map((spec) => spec?.authority), [{ kind: "semantic", boundary: "cue" },{ kind: "parameter", binding: "for", relation: "after-start" },]);assert.deepEqual(specs(createTemporalWindowProjection({- id: "until", element: element({ until: reference("moment"), for: "8f" }), semantic, resolveReference,+ id: "until", element: element({ until: reference("moment"), for: "8f" }), timeline, resolveReference,})).map((spec) => spec?.authority), [{ kind: "parameter", binding: "for", relation: "before-end" },{ kind: "semantic", boundary: "cue" },@@ -54,7 +54,7 @@ test("the author bridge owns the four Window forms and the explicit Instant fallassert.deepEqual(specs(createTemporalWindowProjection({id: "explicit",element: element({ start: "moment.cue+3f", end: "program.end", moment: reference("moment") }),- semantic, resolveReference,+ timeline, resolveReference,})).map((spec) => spec?.authority), [{ kind: "parameter", binding: "start", relation: "direct" },{ kind: "parameter", binding: "end", relation: "direct" },@@ -62,33 +62,37 @@ test("the author bridge owns the four Window forms and the explicit Instant fallconst instant = createTemporalInstantProjection({id: "fallback", element: element({ instant: "moment.cue+3f", moment: reference("moment") }),- semantic, resolveReference,+ timeline, resolveReference,});assert.deepEqual(specs(instant as ReturnType<typeof createTemporalWindowProjection>)[0]?.authority,{ kind: "parameter", binding: "instant", relation: "direct" });assert.throws(() => createTemporalInstantProjection({- id: "ambiguous", element: element({ at: "moment.cue+3f" }), semantic, resolveReference,+ id: "ambiguous", element: element({ at: "moment.cue+3f" }), timeline, resolveReference,}), /exact duration/u);});-test("authored at/for projects on a declared clock while Script events keep their semantic dependency", () => {+test("authored time and Script events use the same Timeline with different projections", () => {const space = resolved("animation", programSpaceTypes.programSpace);- const at = createTemporalInstantProjection({ id: "message", element: element({ at: "2.5s" }), space, resolveReference });+ const at = createTemporalInstantProjection({ id: "message", element: element({ at: "2.5s" }), timeline, resolveReference });const absolute = specs(at as ReturnType<typeof createTemporalWindowProjection>)[0]!;assert.deepEqual(absolute.projection, { ref: "absolute", at: { unit: "seconds", numerator: 5, denominator: 2 } });assert.deepEqual(absolute.authority, { kind: "parameter", binding: "at", relation: "direct" });- assert.equal(at.fragments.some(fragment => fragment.inputs.some(input => input.name === "semantic")), false);+ assert.equal(at.fragments.at(-1)!.inputs.some(input => input.name === "timeline"), true);+ assert.deepEqual(at.components.at(-1)!.inputs.timeline, timeline.ref);const window = specs(createTemporalWindowProjection({- id: "bubble", element: element({ at: "2.5s", for: "8f" }), space, resolveReference,+ id: "bubble", element: element({ at: "2.5s", for: "8f" }), timeline, resolveReference,}));assert.deepEqual(window[1]?.projection, { ref: "absolute", at: { unit: "seconds", numerator: 5, denominator: 2 }, offset: { unit: "frames", value: 8 } });- const bound = createTemporalInstantProjection({ id: "message", element: element({ at: reference("moment") }), semantic, resolveReference });- assert.equal(bound.fragments.some(fragment => fragment.inputs.some(input => input.name === "semantic")), true);- assert.throws(() => createTemporalInstantProjection({ id: "message", element: element({ at: reference("moment") }), space, resolveReference }), /semantic/u);+ const bound = createTemporalInstantProjection({ id: "message", element: element({ at: reference("moment") }), timeline, resolveReference });+ assert.equal(bound.fragments.at(-1)!.inputs.some(input => input.name === "timeline"), true);+ assert.deepEqual(bound.components.at(-1)!.inputs.timeline, timeline.ref);- const context = resolveTemporalContext({ element: element({ space: reference("animation") }), resolveReference: path => path === "animation" ? space : undefined });- assert.deepEqual(context, { space });+ const context = resolveTemporalContext({ element: element({ timeline: reference("timeline") }), resolveReference: path => path === "timeline" ? timeline : undefined });+ assert.deepEqual(context, { timeline });+ assert.throws(() => resolveTemporalContext({+ element: element({ timeline: reference("animation") }), resolveReference: () => space,+ }), /must reference a Timeline/);});@@ -96,8 +100,21 @@ test("bare references and explicit offsets expose the same local parameter edit"for (const instant of ["moment.cue", "moment.cue + 0f", "moment.cue - 2f", "20f"]) {const projection = createTemporalInstantProjection({id: "event", element: element({ instant: instant!, ...(instant!.startsWith("moment") ? { moment: reference("moment") } : {}) }),- semantic, resolveReference,+ timeline, resolveReference,});assert.deepEqual(specs(projection)[0]!.authority, { kind: "parameter", binding: "instant", relation: "direct" });}});++test("Window endpoints can bind distinct Segments without reversing a Script Selection", () => {+ const before = resolved("before", narrativeTypes.excerpt), after = resolved("after", narrativeTypes.excerpt);+ const projection = createTemporalWindowProjection({ id: "overlap", timeline,+ element: element({ start: "segment.start", "start-source": reference("after"), end: "segment.end", "end-source": reference("before") }),+ resolveReference: path => path === "before" ? before : path === "after" ? after : undefined,+ });+ assert.deepEqual(projection.components[0]!.inputs["start-segment"], after.ref);+ assert.deepEqual(projection.components[0]!.inputs["end-segment"], before.ref);+ assert.deepEqual(specs(projection).map(spec => spec?.authority), [+ { kind: "parameter", binding: "start", relation: "direct" }, { kind: "parameter", binding: "end", relation: "direct" },+ ]);+});diff --git a/packages/temporal/README.md b/packages/temporal/README.mdindex b2790316..38c0f5ee 100644--- a/packages/temporal/README.md+++ b/packages/temporal/README.md@@ -12,20 +12,20 @@ or editor metadata. A graph-qualified projection `id` and its author-facing `subon purpose.Author syntax does not live here. `@hypit/temporal-markup` lowers SVML timing forms into ordinary-Instant projection and Window composition operations. The graph supplies ProgramSpace separately;-domain components receive only ProgramSpace plus the resulting Instant or Window and never locate a+Instant projection and Window composition operations. The graph supplies one Timeline;+domain components receive that Timeline plus the resulting Instant or Window and never locate aSelection, Segment or Moment themselves.Every official consumer verifies the projection at its public boundary: `subjectId` must name the-domain object being built, both endpoints must retain one ProgramSpace, and that identity-must equal the explicitly supplied ProgramSpace. ProgramSpace is therefore an ordinary graph input+domain object being built, both endpoints must retain one time-range identity, and that identity+must equal the explicitly supplied Timeline. Timeline is therefore an ordinary graph inputto the consumer, not ambient renderer state.Temporal rejects Instants outside ProgramSpace and Windows that are reversed or empty. It does notclip or repair author time. The package also provides sibling-window validation and triggered-stagescheduling, but no renderer, Provider, media policy or Studio behavior.-Program-bound projection receives ProgramSpace directly. Semantic-bound projection receives the-SemanticTrack that locates its Script anchors. Both produce the same Instant type; program-bound+All projection Producers receive Timeline. Program-bound projection uses its complete range;+semantic-bound projection locates its Script anchors. Both produce the same Instant type; program-boundsources have no Narrative identity. A Window can combine a semantic endpoint with a program endpointon the same time axis. `absolute` expressions may include a duration offset, as used by `at`/`for`.diff --git a/packages/temporal/package.json b/packages/temporal/package.jsonindex a854d8c4..b80897fb 100644--- a/packages/temporal/package.json+++ b/packages/temporal/package.json@@ -15,6 +15,6 @@"@hypit/narrative": "workspace:*","@hypit/program-space": "workspace:*","@hypit/protocol": "workspace:*",- "@hypit/semantic-track": "workspace:*"+ "@hypit/timeline": "workspace:*"}}diff --git a/packages/temporal/src/component.ts b/packages/temporal/src/component.tsindex 404446e6..45e3b71b 100644--- a/packages/temporal/src/component.ts+++ b/packages/temporal/src/component.ts@@ -2,8 +2,7 @@ import type { ComponentPackage, ProducerHandlerContext } from "@hypit/component-import { canonicalize } from "@hypit/protocol";import type { StoredValue } from "@hypit/protocol";import type { NarrativeExcerpt, NarrativeMomentRef, NarrativeSelectionRef } from "@hypit/narrative";-import type { ProgramSpace } from "@hypit/program-space";-import type { SemanticTrack } from "@hypit/semantic-track";+import type { Timeline } from "@hypit/timeline";import {composeTemporalWindow,@@ -31,14 +30,14 @@ export const temporalComponent = {{ producer: temporalProducers.projectProgramInstant, handler: ({ inputs }) => ({ outputs: { instant: output(projectProgramInstant({itemId: instantSpec(inputs).id,subjectId: instantSpec(inputs).subjectId,- space: inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ timeline: inline<Timeline>(inputs.timeline?.value, "Timeline"),projection: instantSpec(inputs).projection,authority: instantSpec(inputs).authority,})) }, needs: {} }) },{ producer: temporalProducers.projectSelectionInstant, handler: ({ inputs }) => ({ outputs: { instant: output(projectSelectionInstant({itemId: instantSpec(inputs).id,subjectId: instantSpec(inputs).subjectId,- semantic: inline<SemanticTrack>(inputs.semantic?.value, "SemanticTrack"),+ timeline: inline<Timeline>(inputs.timeline?.value, "Timeline"),selection: inline<NarrativeSelectionRef>(inputs.selection?.value, "NarrativeSelection"),projection: instantSpec(inputs).projection,authority: instantSpec(inputs).authority,@@ -46,7 +45,7 @@ export const temporalComponent = {{ producer: temporalProducers.projectSegmentInstant, handler: ({ inputs }) => ({ outputs: { instant: output(projectSegmentInstant({itemId: instantSpec(inputs).id,subjectId: instantSpec(inputs).subjectId,- semantic: inline<SemanticTrack>(inputs.semantic?.value, "SemanticTrack"),+ timeline: inline<Timeline>(inputs.timeline?.value, "Timeline"),segment: inline<NarrativeExcerpt>(inputs.segment?.value, "NarrativeExcerpt"),projection: instantSpec(inputs).projection,authority: instantSpec(inputs).authority,@@ -54,7 +53,7 @@ export const temporalComponent = {{ producer: temporalProducers.projectMomentInstant, handler: ({ inputs }) => ({ outputs: { instant: output(projectMomentInstant({itemId: instantSpec(inputs).id,subjectId: instantSpec(inputs).subjectId,- semantic: inline<SemanticTrack>(inputs.semantic?.value, "SemanticTrack"),+ timeline: inline<Timeline>(inputs.timeline?.value, "Timeline"),moment: inline<NarrativeMomentRef>(inputs.moment?.value, "NarrativeMoment"),projection: instantSpec(inputs).projection,authority: instantSpec(inputs).authority,diff --git a/packages/temporal/src/index.ts b/packages/temporal/src/index.tsindex c1007296..afa4295b 100644--- a/packages/temporal/src/index.ts+++ b/packages/temporal/src/index.ts@@ -1,6 +1,6 @@import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";import { narrativeDependency, narrativeTypes } from "@hypit/narrative";-import { semanticTrackDependency, semanticTrackTypes } from "@hypit/semantic-track";+import { timelineDependency, timelineTypes } from "@hypit/timeline";import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";const string = { kind: "string", minLength: 1 } as const;@@ -65,7 +65,7 @@ export const temporalManifest: ModuleManifest = {format: "hypit.module@1",name: temporalModuleRef.name,version: temporalModuleRef.version,- dependencies: [narrativeDependency, programSpaceDependency, semanticTrackDependency],+ dependencies: [narrativeDependency, programSpaceDependency, timelineDependency],types: [{ name: temporalTypes.instantSpec.name },{ name: temporalTypes.instant.name },@@ -75,16 +75,16 @@ export const temporalManifest: ModuleManifest = {capabilities: [],producers: [{ name: temporalProducers.projectProgramInstant.name,- inputs: [{ name: "space", type: programSpaceTypes.programSpace }, { name: "spec", type: temporalTypes.instantSpec }],+ inputs: [{ name: "timeline", type: timelineTypes.track }, { name: "spec", type: temporalTypes.instantSpec }],outputs: [{ name: "instant", type: temporalTypes.instant }], needs: [] },{ name: temporalProducers.projectSelectionInstant.name,- inputs: [{ name: "semantic", type: semanticTrackTypes.track }, { name: "selection", type: narrativeTypes.selection }, { name: "spec", type: temporalTypes.instantSpec }],+ inputs: [{ name: "timeline", type: timelineTypes.track }, { name: "selection", type: narrativeTypes.selection }, { name: "spec", type: temporalTypes.instantSpec }],outputs: [{ name: "instant", type: temporalTypes.instant }], needs: [] },{ name: temporalProducers.projectSegmentInstant.name,- inputs: [{ name: "semantic", type: semanticTrackTypes.track }, { name: "segment", type: narrativeTypes.excerpt }, { name: "spec", type: temporalTypes.instantSpec }],+ inputs: [{ name: "timeline", type: timelineTypes.track }, { name: "segment", type: narrativeTypes.excerpt }, { name: "spec", type: temporalTypes.instantSpec }],outputs: [{ name: "instant", type: temporalTypes.instant }], needs: [] },{ name: temporalProducers.projectMomentInstant.name,- inputs: [{ name: "semantic", type: semanticTrackTypes.track }, { name: "moment", type: narrativeTypes.moment }, { name: "spec", type: temporalTypes.instantSpec }],+ inputs: [{ name: "timeline", type: timelineTypes.track }, { name: "moment", type: narrativeTypes.moment }, { name: "spec", type: temporalTypes.instantSpec }],outputs: [{ name: "instant", type: temporalTypes.instant }], needs: [] },{ name: temporalProducers.composeWindow.name,inputs: [{ name: "spec", type: temporalTypes.windowSpec }, { name: "start", type: temporalTypes.instant }, { name: "end", type: temporalTypes.instant }],@@ -108,3 +108,5 @@ export const temporalWindowSchema: ValueSchema = object({end: { schema: temporalInstantSchema },span: { schema: frameSpanSchema },});++export { durationInFrames } from "./rational.js";diff --git a/packages/temporal/src/location.ts b/packages/temporal/src/location.tsindex 47c6322f..992e6731 100644--- a/packages/temporal/src/location.ts+++ b/packages/temporal/src/location.ts@@ -2,11 +2,11 @@ import type { NarrativeExcerpt, NarrativeMomentRef, NarrativeSelectionRef } fromimport { programSpaceFrameCount } from "@hypit/program-space";import {momentFrame,- projectSemanticProgramSpace,+ projectTimelineSpace,segmentFrameSpan,selectionFrameSpan,-} from "@hypit/semantic-track";-import type { SemanticTrack } from "@hypit/semantic-track";+} from "@hypit/timeline";+import type { Timeline } from "@hypit/timeline";import type {LocatedMoment,@@ -22,10 +22,10 @@ function assertLocatedFrame(frame: number, totalFrames: number, label: string):}export function locateSelection(- semantic: SemanticTrack,+ semantic: Timeline,selection: NarrativeSelectionRef,): LocatedSelection {- const space = projectSemanticProgramSpace(semantic);+ const space = projectTimelineSpace(semantic);const totalFrames = programSpaceFrameCount(space);const span = selectionFrameSpan(semantic, selection);assertLocatedFrame(span.startFrame, totalFrames, `NarrativeSelection ${selection.id} start`);@@ -34,26 +34,26 @@ export function locateSelection(}export function locateMoment(- semantic: SemanticTrack,+ semantic: Timeline,moment: NarrativeMomentRef,): LocatedMoment {- const space = projectSemanticProgramSpace(semantic);+ const space = projectTimelineSpace(semantic);const totalFrames = programSpaceFrameCount(space);const frame = momentFrame(semantic, moment);assertLocatedFrame(frame, totalFrames, `NarrativeMoment ${moment.id} cue`);return { id: moment.id, cue: { frame } };}-export function locateProgram(semantic: SemanticTrack): LocatedProgram {- const space = projectSemanticProgramSpace(semantic);+export function locateProgram(semantic: Timeline): LocatedProgram {+ const space = projectTimelineSpace(semantic);return { id: "program", start: { frame: 0 }, end: { frame: programSpaceFrameCount(space) } };}export function locateSegment(- semantic: SemanticTrack,+ semantic: Timeline,segment: NarrativeExcerpt,): LocatedSegment {- const space = projectSemanticProgramSpace(semantic);+ const space = projectTimelineSpace(semantic);const totalFrames = programSpaceFrameCount(space);const span = segmentFrameSpan(semantic, segment);assertLocatedFrame(span.startFrame, totalFrames, `Narrative Segment ${segment.id} start`);diff --git a/packages/temporal/src/projection.ts b/packages/temporal/src/projection.tsindex 5619328b..e689b445 100644--- a/packages/temporal/src/projection.ts+++ b/packages/temporal/src/projection.ts@@ -1,8 +1,8 @@import type { NarrativeExcerpt, NarrativeMomentRef, NarrativeSelectionRef } from "@hypit/narrative";import { assertProgramSpaceIdentity, programSpaceFrameCount } from "@hypit/program-space";import type { ProgramSpace } from "@hypit/program-space";-import { projectSemanticProgramSpace } from "@hypit/semantic-track";-import type { SemanticTrack } from "@hypit/semantic-track";+import { projectTimelineSpace } from "@hypit/timeline";+import type { Timeline } from "@hypit/timeline";import { locateMoment, locateProgram, locateSegment, locateSelection } from "./location.js";import { add, compare, durationInFrames, quantizeBoundary, rational } from "./rational.js";@@ -169,17 +169,17 @@ function projectedInstant(export function projectSelectionInstant(input: {readonly itemId: string;readonly subjectId: string;- readonly semantic: SemanticTrack;+ readonly timeline: Timeline;readonly selection: NarrativeSelectionRef;readonly projection: TemporalInstantExpression;readonly authority: TemporalInstantSpec["authority"];}): ProjectedInstant {- const space = projectSemanticProgramSpace(input.semantic);- const program = locateProgram(input.semantic);- const selection = locateSelection(input.semantic, input.selection);+ const space = projectTimelineSpace(input.timeline);+ const program = locateProgram(input.timeline);+ const selection = locateSelection(input.timeline, input.selection);return projectedInstant({ id: input.itemId, subjectId: input.subjectId, projection: input.projection, authority: input.authority },- { spaceId: space.id, narrativeId: input.semantic.narrativeId, kind: "selection", id: selection.id },+ { spaceId: space.id, narrativeId: input.selection.narrativeId, kind: "selection", id: selection.id },projectTemporalInstant(input.projection, { program, selection }, space),);}@@ -187,17 +187,17 @@ export function projectSelectionInstant(input: {export function projectMomentInstant(input: {readonly itemId: string;readonly subjectId: string;- readonly semantic: SemanticTrack;+ readonly timeline: Timeline;readonly moment: NarrativeMomentRef;readonly projection: TemporalInstantExpression;readonly authority: TemporalInstantSpec["authority"];}): ProjectedInstant {- const space = projectSemanticProgramSpace(input.semantic);- const program = locateProgram(input.semantic);- const moment = locateMoment(input.semantic, input.moment);+ const space = projectTimelineSpace(input.timeline);+ const program = locateProgram(input.timeline);+ const moment = locateMoment(input.timeline, input.moment);return projectedInstant({ id: input.itemId, subjectId: input.subjectId, projection: input.projection, authority: input.authority },- { spaceId: space.id, narrativeId: input.semantic.narrativeId, kind: "moment", id: moment.id },+ { spaceId: space.id, narrativeId: input.moment.narrativeId, kind: "moment", id: moment.id },projectTemporalInstant(input.projection, { program, moment }, space),);}@@ -205,11 +205,11 @@ export function projectMomentInstant(input: {export function projectProgramInstant(input: {readonly itemId: string;readonly subjectId: string;- readonly space: ProgramSpace;+ readonly timeline: Timeline;readonly projection: TemporalInstantExpression;readonly authority: TemporalInstantSpec["authority"];}): ProjectedInstant {- const space = input.space;+ const space = projectTimelineSpace(input.timeline);const program = { id: "program" as const, start: { frame: 0 }, end: { frame: programSpaceFrameCount(space) } };return projectedInstant({ id: input.itemId, subjectId: input.subjectId, projection: input.projection, authority: input.authority },@@ -221,17 +221,17 @@ export function projectProgramInstant(input: {export function projectSegmentInstant(input: {readonly itemId: string;readonly subjectId: string;- readonly semantic: SemanticTrack;+ readonly timeline: Timeline;readonly segment: NarrativeExcerpt;readonly projection: TemporalInstantExpression;readonly authority: TemporalInstantSpec["authority"];}): ProjectedInstant {- const space = projectSemanticProgramSpace(input.semantic);- const program = locateProgram(input.semantic);- const segment = locateSegment(input.semantic, input.segment);+ const space = projectTimelineSpace(input.timeline);+ const program = locateProgram(input.timeline);+ const segment = locateSegment(input.timeline, input.segment);return projectedInstant({ id: input.itemId, subjectId: input.subjectId, projection: input.projection, authority: input.authority },- { spaceId: space.id, narrativeId: input.semantic.narrativeId, kind: "segment", id: segment.id },+ { spaceId: space.id, narrativeId: input.segment.narrativeId, kind: "segment", id: segment.id },projectTemporalInstant(input.projection, { program, segment }, space),);}diff --git a/packages/temporal/src/rational.ts b/packages/temporal/src/rational.tsindex 40e3bd76..c538abc3 100644--- a/packages/temporal/src/rational.ts+++ b/packages/temporal/src/rational.ts@@ -1,4 +1,4 @@-import type { ProgramSpace } from "@hypit/program-space";+import type { ProgramClock } from "@hypit/program-space";import type { TemporalDuration } from "./types.js";@@ -51,9 +51,9 @@ function safeInteger(value: number, label: string): bigint {return BigInt(value);}-export function durationInFrames(duration: TemporalDuration, space: ProgramSpace): Rational {- const frameNumerator = safeInteger(space.frameRate.numerator, "ProgramSpace frame-rate numerator");- const frameDenominator = safeInteger(space.frameRate.denominator, "ProgramSpace frame-rate denominator");+export function durationInFrames(duration: TemporalDuration, space: ProgramClock): Rational {+ const frameNumerator = safeInteger(space.frameRate.numerator, "ProgramClock frame-rate numerator");+ const frameDenominator = safeInteger(space.frameRate.denominator, "ProgramClock frame-rate denominator");switch (duration.unit) {case "frames":return rational(safeInteger(duration.value, "Frame duration"));diff --git a/packages/temporal/test/temporal.test.ts b/packages/temporal/test/temporal.test.tsindex 6c91d4b9..80a57cfd 100644--- a/packages/temporal/test/temporal.test.ts+++ b/packages/temporal/test/temporal.test.ts@@ -1,11 +1,11 @@-import { projectSemanticProgramSpace } from "@hypit/semantic-track";+import { projectTimelineSpace } from "@hypit/timeline";import assert from "node:assert/strict";import test from "node:test";import type { NarrativeMomentRef, NarrativeSelectionRef } from "@hypit/narrative";import { programFrameSampleBoundary, programSpaceSampleFrames } from "@hypit/program-space";import type { ProgramSpace } from "@hypit/program-space";-import { semanticTrackFixture } from "../../../test/semantic-track-fixture.js";+import { timelineFixture } from "../../../test/timeline-fixture.js";import {assertWindowRelation,@@ -26,7 +26,7 @@ const space: ProgramSpace = {frameRate: { numerator: 30, denominator: 1 },};-const semantic = semanticTrackFixture(space, { narrativeId: "test-narrative",+const semantic = timelineFixture(space, { narrativeId: "test-narrative",segments: [{ id: "opening", frameCount: 60 },{ id: "answer", frameCount: 60 },@@ -53,36 +53,36 @@ const seconds = (numerator: number, denominator = 1) => ({ unit: "seconds" as coconst fixed = { kind: "fixed" as const };type Projection = import("../src/index.js").TemporalInstantExpression;const projectProgramInstantFixture = (input: { itemId: string; semantic: typeof semantic; projection: Projection }) =>- projectProgramInstant({ ...input, space: projectSemanticProgramSpace(input.semantic), subjectId: input.itemId, authority: fixed });+ projectProgramInstant({ ...input, timeline: input.semantic, subjectId: input.itemId, authority: fixed });const projectMomentInstantFixture = (input: { itemId: string; semantic: typeof semantic; moment: NarrativeMomentRef; projection: Projection }) =>- projectMomentInstant({ ...input, subjectId: input.itemId, authority: fixed });+ projectMomentInstant({ ...input, timeline: input.semantic, subjectId: input.itemId, authority: fixed });const projectSelectionInstantFixture = (input: { itemId: string; semantic: typeof semantic; selection: NarrativeSelectionRef; projection: Projection }) =>- projectSelectionInstant({ ...input, subjectId: input.itemId, authority: fixed });+ projectSelectionInstant({ ...input, timeline: input.semantic, subjectId: input.itemId, authority: fixed });const projectSelectionWindow = (input: {itemId: string; semantic: typeof semantic; selection: NarrativeSelectionRef;projection: { start: Projection; end: Projection };}) => composeTemporalWindow({ id: input.itemId, subjectId: input.itemId },- projectSelectionInstant({ itemId: `${input.itemId}.start`, subjectId: input.itemId, semantic: input.semantic, selection: input.selection, projection: input.projection.start, authority: fixed }),+ projectSelectionInstant({ itemId: `${input.itemId}.start`, subjectId: input.itemId, timeline: input.semantic, selection: input.selection, projection: input.projection.start, authority: fixed }),input.projection.end.ref.startsWith("program.") || input.projection.end.ref === "absolute"- ? projectProgramInstant({ itemId: `${input.itemId}.end`, subjectId: input.itemId, space: projectSemanticProgramSpace(input.semantic), projection: input.projection.end, authority: fixed })- : projectSelectionInstant({ itemId: `${input.itemId}.end`, subjectId: input.itemId, semantic: input.semantic, selection: input.selection, projection: input.projection.end, authority: fixed }));+ ? projectProgramInstant({ itemId: `${input.itemId}.end`, subjectId: input.itemId, timeline: input.semantic, projection: input.projection.end, authority: fixed })+ : projectSelectionInstant({ itemId: `${input.itemId}.end`, subjectId: input.itemId, timeline: input.semantic, selection: input.selection, projection: input.projection.end, authority: fixed }));const projectMomentWindow = (input: {itemId: string; semantic: typeof semantic; moment: NarrativeMomentRef;projection: { start: Projection; end: Projection };}) => composeTemporalWindow({ id: input.itemId, subjectId: input.itemId },- projectMomentInstant({ itemId: `${input.itemId}.start`, subjectId: input.itemId, semantic: input.semantic, moment: input.moment, projection: input.projection.start, authority: fixed }),- projectMomentInstant({ itemId: `${input.itemId}.end`, subjectId: input.itemId, semantic: input.semantic, moment: input.moment, projection: input.projection.end, authority: fixed }));+ projectMomentInstant({ itemId: `${input.itemId}.start`, subjectId: input.itemId, timeline: input.semantic, moment: input.moment, projection: input.projection.start, authority: fixed }),+ projectMomentInstant({ itemId: `${input.itemId}.end`, subjectId: input.itemId, timeline: input.semantic, moment: input.moment, projection: input.projection.end, authority: fixed }));const projectSegmentWindow = (input: {itemId: string; semantic: typeof semantic; segment: import("@hypit/narrative").NarrativeExcerpt;projection: { start: Projection; end: Projection };}) => composeTemporalWindow({ id: input.itemId, subjectId: input.itemId },- projectSegmentInstant({ itemId: `${input.itemId}.start`, subjectId: input.itemId, semantic: input.semantic, segment: input.segment, projection: input.projection.start, authority: fixed }),- projectSegmentInstant({ itemId: `${input.itemId}.end`, subjectId: input.itemId, semantic: input.semantic, segment: input.segment, projection: input.projection.end, authority: fixed }));+ projectSegmentInstant({ itemId: `${input.itemId}.start`, subjectId: input.itemId, timeline: input.semantic, segment: input.segment, projection: input.projection.start, authority: fixed }),+ projectSegmentInstant({ itemId: `${input.itemId}.end`, subjectId: input.itemId, timeline: input.semantic, segment: input.segment, projection: input.projection.end, authority: fixed }));const projectProgramWindow = (input: {itemId: string; semantic: typeof semantic; projection: { start: Projection; end: Projection };}) => composeTemporalWindow({ id: input.itemId, subjectId: input.itemId },- projectProgramInstant({ itemId: `${input.itemId}.start`, subjectId: input.itemId, space: projectSemanticProgramSpace(input.semantic), projection: input.projection.start, authority: fixed }),- projectProgramInstant({ itemId: `${input.itemId}.end`, subjectId: input.itemId, space: projectSemanticProgramSpace(input.semantic), projection: input.projection.end, authority: fixed }));+ projectProgramInstant({ itemId: `${input.itemId}.start`, subjectId: input.itemId, timeline: input.semantic, projection: input.projection.start, authority: fixed }),+ projectProgramInstant({ itemId: `${input.itemId}.end`, subjectId: input.itemId, timeline: input.semantic, projection: input.projection.end, authority: fixed }));test("one Selection projects exact local points and stable source identity", () => {const result = projectSelectionWindow({@@ -130,7 +130,7 @@ test("points preserve source identity and admit both ProgramSpace boundaries", (}).frame, 0);});-test("Script Program Anchors resolve through the SemanticTrack without entering a SemanticTake", () => {+test("Script Program Anchors resolve through the Timeline without entering a SemanticTake", () => {assert.deepEqual(locateSelection(semantic, selection("whole", "program:start", "program:end")), {id: "whole", start: { frame: 0 }, end: { frame: 300 },});@@ -197,7 +197,7 @@ test("program and absolute projections use exact rational frame-rate arithmetic"};const result = projectProgramWindow({itemId: "absolute",- semantic: semanticTrackFixture(ntsc),+ semantic: timelineFixture(ntsc),projection: {start: { ref: "absolute", at: { unit: "milliseconds", value: 500 } },end: { ref: "absolute", at: seconds(1) },@@ -207,7 +207,7 @@ test("program and absolute projections use exact rational frame-rate arithmetic"});test("an authored clock needs no narrative and combines absolute time with frame offsets", () => {- const point = (itemId: string, projection: Projection) => projectProgramInstant({ itemId, subjectId: "bubble", space, projection, authority: fixed });+ const point = (itemId: string, projection: Projection) => projectProgramInstant({ itemId, subjectId: "bubble", timeline: { ...space, items: [] }, projection, authority: fixed });const start = point("start", { ref: "absolute", at: seconds(5, 2) });const end = point("end", { ref: "absolute", at: seconds(5, 2), offset: frames(8) });assert.deepEqual(composeTemporalWindow({ id: "bubble", subjectId: "bubble" }, start, end).span,@@ -220,7 +220,7 @@ test("shared physical time preserves semantic ownership and rejects another filmitemId: "reveal", semantic, moment: moment("answer", "b"), projection: { ref: "moment.cue" },});const end = projectProgramInstant({- itemId: "end", subjectId: "reveal", space,+ itemId: "end", subjectId: "reveal", timeline: { ...space, items: [] },projection: { ref: "absolute", at: seconds(3) }, authority: fixed,});const window = composeTemporalWindow({ id: "reveal", subjectId: "reveal" }, cue, end);diff --git a/packages/timeline-author/README.md b/packages/timeline-author/README.mdnew file mode 100644index 00000000..66d85e19--- /dev/null+++ b/packages/timeline-author/README.md@@ -0,0 +1,68 @@+# `@hypit/timeline-author`++Declare one complete Timeline and place prepared Takes within it. Spoken work, mixed spoken/MG+work and pure authored animation use the same declaration. The separate+[Timeline value package](../timeline/README.md) owns the data and projections; this author package+owns the markup and deterministic assembly.++```svml+<import as="time" from="@hypit/timeline-author@1"/>+<time:Clock id="clock" frame-rate="30"/>+<time:Timeline id="program" clock={clock} end="content.end+2s">+ <time:Take source={opening.take} at="2s"/>+ <time:Take source={explanation.take} at="previous.end+3s"/>+ <time:Take source={closing.take} at="previous.end-12f"/>+</time:Timeline>+```++The first Take begins at two seconds. The second starts three seconds after the preceding Take ends.+The third overlaps its predecessor by twelve frames. The complete Timeline ends two seconds after+all Takes have ended. Durations come from prepared material, so authoring these relations does not+require knowing generated lengths beforehand or serializing generation.++| Declaration | Meaning |+| --- | --- |+| `clock={clock}` | Required frame-rate input, also usable by upstream normalization. |+| Take `source={clip.take}` | Complete normalized, semantically prepared local Take. A wordless Take is valid real media. |+| First omitted `at` | `0f`. |+| Later omitted `at` | `previous.end`. |+| `at="20s"` | Absolute start. |+| `at="previous.end+2s"` | Start relative to the preceding Take declaration's end. Negative offsets permit overlap. |+| Omitted `end` | `content.end`, the maximum end across all placed Takes. |+| `end="content.end+2s"` | Reserve a trailing interval after all Takes finish. |+| `end="30s"` | Fixed complete extent, including every placed Take. |++Positions and offsets accept seconds, milliseconds and frames, and must resolve to exact frame+boundaries on the Clock. Negative placements, a missing predecessor, or a fixed end shorter than+placed content produce an error. A positive predecessor offset does not promise globally empty+time if another long Take remains active. Placement is translation at native speed; trimming or+retiming material belongs before its semantic preparation.++For ordinary sequential assembly, omit all Take `at` attributes and Timeline `end`. For wholly+authored animation, supply the extent and no Takes:++```svml+<time:Timeline id="animation" clock={clock} end="30s"/>+```++This produces no blank video or fabricated Script. Empty content cannot supply an implicit positive+extent. Program boundaries remain available for MG, media, titles and sound.++| Output | Use |+| --- | --- |+| `.timeline` | Complete time range and placed material/evidence, consumed through `timeline={program.timeline}`. |++[Performance](../performance/README.md) presents footage already+held by the Timeline. It can show it full-frame, in an inset or as a cutout; project components can+own coordinated video and graphics. Sampling a new presentation Window preserves each Take's source+position. Overlap supplies multiple sources, not an automatic dissolve. Independent Media Items+still receive their own assets and may coexist. Caption reads the authored words and placed timing.++[Sound](../sound/README.md) presents existing Timeline audio through ordered Uses.+[Audio Track](../audio-track/README.md) handles independent music and effects. Include the wanted+presentation outputs explicitly in Film. Timeline itself exports only `.timeline`; holding material+does not add a picture or sound contribution.++Timeline placement is authored with `at` and `end`. Studio shows placed Takes and the complete range+as reference information. Use-window and Script-marker controls edit presentation timing and semantic+anchors respectively; they do not move Takes or change the Timeline extent.diff --git a/packages/speech-track/package.json b/packages/timeline-author/package.jsonsimilarity index 70%rename from packages/speech-track/package.jsonrename to packages/timeline-author/package.jsonindex 826a6597..fc5702cc 100644--- a/packages/speech-track/package.json+++ b/packages/timeline-author/package.json@@ -1,5 +1,5 @@{- "name": "@hypit/speech-track",+ "name": "@hypit/timeline-author","version": "0.0.0-dev","license": "SEE LICENSE IN LICENSE","private": true,@@ -12,12 +12,14 @@},"dependencies": {"@hypit/component-kit": "workspace:*",- "@hypit/composition": "workspace:*","@hypit/elaborator": "workspace:*","@hypit/protocol": "workspace:*","@hypit/speech": "workspace:*",- "@hypit/semantic-track": "workspace:*",- "@hypit/markup": "workspace:*"+ "@hypit/timeline": "workspace:*",+ "@hypit/markup": "workspace:*",+ "@hypit/program-space": "workspace:*",+ "@hypit/temporal": "workspace:*",+ "@hypit/temporal-markup": "workspace:*"},"devDependencies": {"@hypit/core": "workspace:*",diff --git a/packages/speech-track/preview/Track.png b/packages/timeline-author/preview/Track.pngsimilarity index 100%rename from packages/speech-track/preview/Track.pngrename to packages/timeline-author/preview/Track.pngdiff --git a/packages/timeline-author/src/activation.ts b/packages/timeline-author/src/activation.tsnew file mode 100644index 00000000..8d4f3a9f--- /dev/null+++ b/packages/timeline-author/src/activation.ts@@ -0,0 +1,22 @@+import { decodeClockSurface } from "@hypit/program-space";+import { createMarkupSurfaceHostFacet } from "@hypit/markup";+import {+ decodeTimelineAuthorSurface, timelineAuthorComponent, timelineAuthorManifest,+ timelineAuthorModuleRef,+ timelineAuthorMarkupSurfaces,+} from "./index.js";++export const hypitPackage = {+ format: "hypit.node-package@1" as const,+ modules: [{+ manifest: timelineAuthorManifest,+ }],+ components: [timelineAuthorComponent],+ hostFacets: [createMarkupSurfaceHostFacet({+ module: timelineAuthorModuleRef,+ declaration: timelineAuthorMarkupSurfaces.find((item) => item.name === "timeline")!, handler: decodeTimelineAuthorSurface,+ }), createMarkupSurfaceHostFacet({ module: timelineAuthorModuleRef,+ declaration: timelineAuthorMarkupSurfaces.find(item => item.name === "clock")!, handler: decodeClockSurface,+ })],+};+export default hypitPackage;diff --git a/packages/speech-track/src/component.ts b/packages/timeline-author/src/component.tssimilarity index 52%rename from packages/speech-track/src/component.tsrename to packages/timeline-author/src/component.tsindex 2c75cf97..784238fa 100644--- a/packages/speech-track/src/component.ts+++ b/packages/timeline-author/src/component.ts@@ -1,42 +1,44 @@+import type { ProgramClock } from "@hypit/program-space";import type { ComponentPackage } from "@hypit/component-kit";import type { StoredValue } from "@hypit/protocol";import { canonicalize } from "@hypit/protocol";import type { SemanticTake } from "@hypit/speech";-import { speechTrackProducers } from "./manifest.js";-import { appendSpeechTrackTake, assembleSpeechTrack, createSpeechTrackSet } from "./program.js";-import type { SpeechTrackHeader, SpeechTrackSet } from "./types.js";+import { timelineAuthorProducers } from "./manifest.js";+import { appendTimelineAuthorTake, assembleTimelineAuthor, createTimelineAuthorSet } from "./program.js";+import type { TimelineAuthorHeader, TimelineAuthorSet } from "./types.js";function inline<T>(value: StoredValue | undefined, subject: string): T {if (value?.kind !== "inline") throw new Error(`${subject} must be inline`);return value.value as unknown as T;}-export const speechTrackComponent = {+export const timelineAuthorComponent = {producers: [{- producer: speechTrackProducers.createSet,+ producer: timelineAuthorProducers.createSet,handler: () => ({- outputs: { set: { kind: "inline", value: canonicalize(createSpeechTrackSet()) } },+ outputs: { set: { kind: "inline", value: canonicalize(createTimelineAuthorSet()) } },needs: {},}),},{- producer: speechTrackProducers.appendTake,+ producer: timelineAuthorProducers.appendTake,handler: ({ inputs }) => ({- outputs: { set: { kind: "inline", value: canonicalize(appendSpeechTrackTake(- inline<SpeechTrackSet>(inputs.set?.value, "SpeechTrackSet"),+ outputs: { set: { kind: "inline", value: canonicalize(appendTimelineAuthorTake(+ inline<TimelineAuthorSet>(inputs.set?.value, "TimelineAuthorSet"),inline<SemanticTake>(inputs.take?.value, "SemanticTake"),)) } },needs: {},}),},{- producer: speechTrackProducers.assembleTrack,+ producer: timelineAuthorProducers.assembleTrack,handler: ({ inputs }) => ({- outputs: { track: { kind: "inline", value: canonicalize(assembleSpeechTrack(- inline<SpeechTrackHeader>(inputs.header?.value, "SpeechTrackHeader"),- inline<SpeechTrackSet>(inputs.set?.value, "SpeechTrackSet"),+ outputs: { track: { kind: "inline", value: canonicalize(assembleTimelineAuthor(+ inline<TimelineAuthorHeader>(inputs.header?.value, "TimelineAuthorHeader"),+ inline<TimelineAuthorSet>(inputs.set?.value, "TimelineAuthorSet"),+ inline<ProgramClock>(inputs.clock?.value, "ProgramClock"),)) } },needs: {},}),diff --git a/packages/speech-track/src/fragment.ts b/packages/timeline-author/src/fragment.tssimilarity index 57%rename from packages/speech-track/src/fragment.tsrename to packages/timeline-author/src/fragment.tsindex 5604d365..10c6caaa 100644--- a/packages/speech-track/src/fragment.ts+++ b/packages/timeline-author/src/fragment.ts@@ -1,35 +1,35 @@+import { programSpaceTypes } from "@hypit/program-space";import { speechTypes } from "@hypit/speech";-import { compositionTypes } from "@hypit/composition";import { sealGraphFragment } from "@hypit/elaborator";import type { FragmentOperation } from "@hypit/elaborator";-import { semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";+import { timelineTypes } from "@hypit/timeline";import type { TypeRef } from "@hypit/protocol";-import { speechTrackProducers, speechTrackTypes } from "./manifest.js";-import type { SpeechTrackFragmentOptions } from "./types.js";+import { timelineAuthorProducers, timelineAuthorTypes } from "./manifest.js";+import type { TimelineAuthorFragmentOptions } from "./types.js";const input = (name: string) => ({ kind: "fragment-input" as const, name });const operation = (id: string) => ({ kind: "fragment-operation" as const, operation: id });-export function createSpeechTrackFragment(options: SpeechTrackFragmentOptions) {- if (options.takes.length === 0) throw new Error("Speech Track requires at least one Take");+export function createTimelineAuthorFragment(options: TimelineAuthorFragmentOptions) {const declaredInputs = new Map<string, TypeRef>();const addInput = (name: string, type: TypeRef): void => {- if (!name) throw new Error("Speech Track Fragment input names must not be empty");+ if (!name) throw new Error("Timeline Fragment input names must not be empty");const previous = declaredInputs.get(name);if (previous !== undefined && (previous.module.name !== type.module.name|| previous.module.version !== type.module.version || previous.name !== type.name)) {- throw new Error(`Speech Track Fragment input ${name} is reused with another Type`);+ throw new Error(`Timeline Fragment input ${name} is reused with another Type`);}declaredInputs.set(name, type);};- addInput("header", speechTrackTypes.header);+ addInput("header", timelineAuthorTypes.header);+ addInput("clock", programSpaceTypes.clock);for (const take of options.takes) {addInput(take.takeName, speechTypes.semanticTake);}const operations: FragmentOperation[] = [{id: "track:set:empty",- producer: speechTrackProducers.createSet,+ producer: timelineAuthorProducers.createSet,inputs: {},result: { kind: "output", name: "set" },}];@@ -38,7 +38,7 @@ export function createSpeechTrackFragment(options: SpeechTrackFragmentOptions) {const id = `track:set:append:${String(index + 1).padStart(4, "0")}`;operations.push({id,- producer: speechTrackProducers.appendTake,+ producer: timelineAuthorProducers.appendTake,inputs: {set: operation(current),take: input(take.takeName),@@ -50,25 +50,16 @@ export function createSpeechTrackFragment(options: SpeechTrackFragmentOptions) {operations.push({id: "track:semantic",- producer: speechTrackProducers.assembleTrack,- inputs: { header: input("header"), set: operation(current) },+ producer: timelineAuthorProducers.assembleTrack,+ inputs: { header: input("header"), set: operation(current), clock: input("clock") },result: { kind: "output", name: "track" },},- {- id: "track:audio-track",- producer: semanticTrackProducers.projectAudio,- inputs: { track: operation("track:semantic") },- result: { kind: "output", name: "audio" },- },);return sealGraphFragment({inputs: [...declaredInputs.entries()].map(([name, type]) => ({ name, type })),operations,exports: [- { name: "semantic", type: semanticTrackTypes.track, root: operation("track:semantic") },- {- name: "audio", type: compositionTypes.audioTrack, root: operation("track:audio-track"),- },+ { name: "timeline", type: timelineTypes.track, root: operation("track:semantic") },],});}diff --git a/packages/timeline-author/src/index.ts b/packages/timeline-author/src/index.tsnew file mode 100644index 00000000..89cd8715--- /dev/null+++ b/packages/timeline-author/src/index.ts@@ -0,0 +1,7 @@+/** Official Timeline authoring and deterministic Timeline assembly. */+export { timelineAuthorComponent } from "./component.js";+export { createTimelineAuthorFragment } from "./fragment.js";+export { timelineAuthorManifest, timelineAuthorMarkupSurfaces, timelineAuthorModuleRef, timelineAuthorProducers, timelineAuthorTypes, timelineAuthorHeaderSchema, timelineAuthorSetSchema } from "./manifest.js";+export { appendTimelineAuthorTake, assembleTimelineAuthor, assertTimelineAuthorHeader, assertTimelineAuthorSet, createTimelineAuthorSet, sealTimelineAuthorHeader } from "./program.js";+export { decodeTimelineAuthorSurface } from "./surface.js";+export type * from "./types.js";diff --git a/packages/timeline-author/src/manifest.ts b/packages/timeline-author/src/manifest.tsnew file mode 100644index 00000000..48d6117f--- /dev/null+++ b/packages/timeline-author/src/manifest.ts@@ -0,0 +1,78 @@+import { programSpaceDependency, programSpaceTypes, programSpaceMarkupSurfaces } from "@hypit/program-space";+import { semanticTakeSchema, speechDependency, speechTypes } from "@hypit/speech";+import { timelineDependency, timelineTypes } from "@hypit/timeline";+import type { ModuleManifest, ValueSchema } from "@hypit/protocol";+export const timelineAuthorModuleRef = { name: "@hypit/timeline-author", version: "1" } as const;+export const timelineAuthorTypes = {+ header: { module: timelineAuthorModuleRef, name: "TimelineAuthorHeader" },+ trackSet: { module: timelineAuthorModuleRef, name: "TimelineAuthorSet" },+} as const;+export const timelineAuthorProducers = {+ createSet: { module: timelineAuthorModuleRef, name: "create-track-set" },+ appendTake: { module: timelineAuthorModuleRef, name: "append-track-take" },+ assembleTrack: { module: timelineAuthorModuleRef, name: "assemble-timeline" },+} as const;+export const timelineAuthorHeaderSchema: ValueSchema = { kind: "object", fields: { id: { schema: { kind: "string", minLength: 1 } }, at: { optional: true, schema: { kind: "array", items: { kind: "string" } } }, end: { optional: true, schema: { kind: "string" } } } };+export const timelineAuthorSetSchema: ValueSchema = { kind: "object", fields: { takes: { schema: { kind: "array", items: { kind: "object", fields: { semantic: { schema: semanticTakeSchema } } } } } } };+export const timelineAuthorMarkupSurfaces = [{+ name: "timeline", tag: "Timeline", mode: "structured",+ outputs: [timelineAuthorTypes.header, timelineTypes.track],+ vocabulary: {+ summary: "Declares one complete Timeline, with zero or more prepared Takes placed within it.",+ attributes: [+ { name: "id", kind: "identifier", required: true, summary: "Names the Timeline output." },+ { name: "clock", kind: "reference", required: true, accepts: [programSpaceTypes.clock], summary: "Frame rate, available before material generation." },+ { name: "end", kind: "literal", required: false, summary: "Program end: fixed time or content.end with an optional offset; defaults to content.end." },+ ],+ children: [{ tag: "Take", cardinality: "many", summary: "A prepared SemanticTake in performance order.",+ attributes: [{ name: "source", kind: "reference", required: true, accepts: [speechTypes.semanticTake], summary: "The normalized, aligned performance." },+ { name: "at", kind: "literal", required: false, summary: "Absolute position or previous.end with an offset. Defaults to zero for the first Take and previous.end thereafter." }] }],+ ports: [+ { name: "timeline", type: timelineTypes.track, summary: "The complete time range, retaining placed material and any semantic evidence." },+ ],+ example: '<time:Clock id="clock" frame-rate="30"/><time:Timeline id="program" clock={clock} end="content.end+2s"><time:Take source={opening.take} at="2s"/><time:Take source={answer.take}/></time:Timeline>',+ notes: ["Performance, Sound and Caption present existing pictures, audio and words independently. Include their contributions explicitly in Film."],+ },+}, programSpaceMarkupSurfaces[0]] as const;+export const timelineAuthorManifest: ModuleManifest = {+ format: "hypit.module@1",+ name: timelineAuthorModuleRef.name,+ version: timelineAuthorModuleRef.version,+ dependencies: [+ programSpaceDependency,+ speechDependency,+ timelineDependency,+ ],+ types: [+ { name: timelineAuthorTypes.header.name },+ { name: timelineAuthorTypes.trackSet.name },+ ],+ capabilities: [],+ producers: [+ {+ name: timelineAuthorProducers.createSet.name,+ inputs: [],+ outputs: [{ name: "set", type: timelineAuthorTypes.trackSet }],+ needs: [],+ },+ {+ name: timelineAuthorProducers.appendTake.name,+ inputs: [+ { name: "set", type: timelineAuthorTypes.trackSet },+ { name: "take", type: speechTypes.semanticTake },+ ],+ outputs: [{ name: "set", type: timelineAuthorTypes.trackSet }],+ needs: [],+ },+ {+ name: timelineAuthorProducers.assembleTrack.name,+ inputs: [+ { name: "header", type: timelineAuthorTypes.header },+ { name: "clock", type: programSpaceTypes.clock },+ { name: "set", type: timelineAuthorTypes.trackSet },+ ],+ outputs: [{ name: "track", type: timelineTypes.track }],+ needs: [],+ },+ ],+};diff --git a/packages/timeline-author/src/program.ts b/packages/timeline-author/src/program.tsnew file mode 100644index 00000000..811fd689--- /dev/null+++ b/packages/timeline-author/src/program.ts@@ -0,0 +1,107 @@+import { assertProgramClockIdentity, type ProgramClock } from "@hypit/program-space";+import { durationInFrames } from "@hypit/temporal";+import { parseTemporalDuration } from "@hypit/temporal-markup";+import { assertSemanticTakeIdentity } from "@hypit/speech";+import type { SemanticTake } from "@hypit/speech";+import { sealTimeline } from "@hypit/timeline";+import type { Timeline } from "@hypit/timeline";+import type { TimelineAuthorHeader, TimelineAuthorSet, TimelineAuthorTake } from "./types.js";+function assert(condition: unknown, message: string): asserts condition {+ if (!condition) throw new Error(message);+}+export function sealTimelineAuthorHeader(value: TimelineAuthorHeader): TimelineAuthorHeader { return structuredClone(value); }+export function assertTimelineAuthorHeader(value: TimelineAuthorHeader): void {+ assert(value.id.trim().length > 0, "TimelineAuthorHeader id must not be empty");+}+function sealTimelineAuthorSet(value: TimelineAuthorSet): TimelineAuthorSet { return structuredClone(value); }+function assertTake(take: TimelineAuthorTake): void { assertSemanticTakeIdentity(take.semantic); }+export function assertTimelineAuthorSet(value: TimelineAuthorSet): void {+ const segments = new Set<string>();+ for (const take of value.takes) {+ assertSemanticTakeIdentity(take.semantic);+ const segmentId = take.semantic.segment.segmentId;+ assert(!segments.has(segmentId), `Timeline repeats Segment ${segmentId}`);+ segments.add(segmentId);+ }+}++export function createTimelineAuthorSet(): TimelineAuthorSet {+ return sealTimelineAuthorSet({+ takes: [],+ });+}++function appendTake(+ set: TimelineAuthorSet,+ take: TimelineAuthorTake,+): TimelineAuthorSet {+ assertTimelineAuthorSet(set);+ assertTake(take);+ assert(!set.takes.some((item) => item.semantic.segment.segmentId === take.semantic.segment.segmentId),+ `Timeline repeats Segment ${take.semantic.segment.segmentId}`);+ return sealTimelineAuthorSet({+ takes: [...set.takes, take],+ });+}++export function appendTimelineAuthorTake(set: TimelineAuthorSet, semantic: SemanticTake): TimelineAuthorSet {+ return appendTake(set, { semantic });+}++export function assembleTimelineAuthor(+ header: TimelineAuthorHeader,+ set: TimelineAuthorSet,+ clock: ProgramClock,+): Timeline {+ assertTimelineAuthorHeader(header);+ assertTimelineAuthorSet(set);+ assertProgramClockIdentity(clock);+ if (header.at !== undefined && header.at.length !== set.takes.length) {+ throw new Error("Timeline placement declarations must correspond to its Takes.");+ }+ const narrativeId = set.takes[0]?.semantic.narrativeId;+ let previousEnd = 0;+ let contentEnd = 0;+ const items = set.takes.map((item, index) => {+ const at = header.at?.[index] ?? (index === 0 ? "0f" : "previous.end");+ const startFrame = placementFrame(at, clock, index === 0 ? undefined : previousEnd, "previous.end");+ previousEnd = startFrame + item.semantic.media.timeline.frameCount;+ contentEnd = Math.max(contentEnd, previousEnd);+ return { take: item.semantic, startFrame };+ });+ const endFrame = placementFrame(header.end ?? "content.end", clock, contentEnd, "content.end");+ assert(endFrame > 0, "Timeline needs a positive end; an empty Timeline requires an authored extent.");+ assert(endFrame >= contentEnd, "Timeline end precedes placed content; choose an end that includes its Takes.");+ return sealTimeline({+ id: header.id, frameRate: clock.frameRate,+ durationSec: endFrame * clock.frameRate.denominator / clock.frameRate.numerator,+ ...(narrativeId === undefined ? {} : { narrativeId }), items,+ });+}++/** Parse local placement syntax before any upstream material needs to run. */+export function placementExpression(expression: string, reference: string) {+ const value = expression.trim();+ if (!value.startsWith(reference)) return { relative: false, sign: 1,+ duration: parseTemporalDuration(value, "Timeline position") };+ const offset = value.slice(reference.length).trim();+ if (offset === "") return { relative: true, sign: 1,+ duration: parseTemporalDuration("0f", "Timeline offset") };+ const match = /^([+-])\s*(.+)$/u.exec(offset);+ if (match === null) throw new Error(`Invalid Timeline position ${expression}.`);+ return { relative: true, sign: match[1] === "-" ? -1 : 1,+ duration: parseTemporalDuration(match[2]!, "Timeline offset") };+}++/** Resolve local assembly expressions before a complete Timeline exists. */+export function placementFrame(expression: string, clock: ProgramClock, base: number | undefined, reference: string): number {+ const parsed = placementExpression(expression, reference);+ if (parsed.relative && base === undefined) throw new Error(`${reference} has no preceding Take.`);+ const frames = durationInFrames(parsed.duration, clock);+ if (frames.denominator !== 1n || frames.numerator > BigInt(Number.MAX_SAFE_INTEGER)) {+ throw new Error(`Timeline duration ${expression} must land on an exact frame boundary.`);+ }+ const frame = (parsed.relative ? base! : 0) + parsed.sign * Number(frames.numerator);+ if (!Number.isSafeInteger(frame) || frame < 0) throw new Error(`Timeline position ${expression} must resolve to a non-negative frame.`);+ return frame;+}diff --git a/packages/timeline-author/src/surface.ts b/packages/timeline-author/src/surface.tsnew file mode 100644index 00000000..b71ca696--- /dev/null+++ b/packages/timeline-author/src/surface.ts@@ -0,0 +1,46 @@+import { placementExpression } from "./program.js";+import { programSpaceTypes } from "@hypit/program-space";+import { assertAttributes, assertEmptyElement, localName, textAttribute, type StructuredSurfaceHandler } from "@hypit/markup";+import { sameType } from "@hypit/protocol";+import { speechTypes } from "@hypit/speech";+import { createTimelineAuthorFragment } from "./fragment.js";+import { timelineAuthorTypes } from "./manifest.js";++export const decodeTimelineAuthorSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {+ assertAttributes(element, ["id", "clock", "end"]);+ const id = textAttribute(element, "id");+ const rawClock = element.attributes.clock;+ const clock = typeof rawClock === "object" && rawClock.kind === "reference" ? resolveReference(rawClock.path) : undefined;+ if (clock === undefined || !sameType(clock.type, programSpaceTypes.clock)) throw new Error(`${element.name}.clock must reference a Clock.`);+ if (element.attributes.end !== undefined) placementExpression(textAttribute(element, "end"), "content.end");+ const at: string[] = [];+ const takes = element.children.flatMap(child => {+ if (child.kind === "text") {+ if (child.value.trim()) throw new Error("Timeline accepts only Take children.");+ return [];+ }+ if (localName(child.name) !== "Take") throw new Error("Timeline accepts only Take children.");+ assertAttributes(child, ["source", "at"]);+ at.push(child.attributes.at === undefined ? (at.length === 0 ? "0f" : "previous.end") : textAttribute(child, "at"));+ const position = placementExpression(at.at(-1)!, "previous.end");+ if (at.length === 1 && position.relative) throw new Error("The first Timeline Take has no previous.end.");+ assertEmptyElement(child);+ const source = child.attributes.source;+ if (typeof source !== "object" || source.kind !== "reference") throw new Error("Timeline Take source must be a reference.");+ const resolved = resolveReference(source.path);+ if (resolved === undefined || !sameType(resolved.type, speechTypes.semanticTake)) throw new Error("Timeline Take source must be a SemanticTake.");+ return [resolved];+ });+ const inputs = takes.map((_, index) => ({ takeName: `take-${index + 1}` }));+ const fragment = createTimelineAuthorFragment({ takes: inputs });+ return {+ records: [{ id: `${id}.header`, type: timelineAuthorTypes.header, value: { kind: "inline", value: { id, at, ...(element.attributes.end === undefined ? {} : { end: textAttribute(element, "end") }) } }, range: element.range }],+ fragments: [fragment],+ components: [{ id, fragment: fragment.id, inputs: {+ clock: clock.ref,+ header: { kind: "record", id: `${id}.header` },+ ...Object.fromEntries(takes.map((take, index) => [inputs[index]!.takeName, take.ref])),+ }, outputs: { timeline: `${id}.timeline` }, range: element.range }],+ exports: [`${id}.timeline`],+ };+};diff --git a/packages/timeline-author/src/types.ts b/packages/timeline-author/src/types.tsnew file mode 100644index 00000000..521951f4--- /dev/null+++ b/packages/timeline-author/src/types.ts@@ -0,0 +1,6 @@+import type { SemanticTake } from "@hypit/speech";+export type TimelineAuthorHeader = { readonly id: string; readonly at?: readonly string[]; readonly end?: string };+export type TimelineAuthorTake = { readonly semantic: SemanticTake };+export type TimelineAuthorSet = { readonly takes: readonly TimelineAuthorTake[] };+export type TimelineAuthorInput = { readonly takeName: string };+export type TimelineAuthorFragmentOptions = { readonly takes: readonly TimelineAuthorInput[] };diff --git a/packages/timeline-author/test/program.test.ts b/packages/timeline-author/test/program.test.tsnew file mode 100644index 00000000..58ee30f6--- /dev/null+++ b/packages/timeline-author/test/program.test.ts@@ -0,0 +1,132 @@+import { projectTimelineAudio, projectTimelineMedia } from "@hypit/timeline";+import assert from "node:assert/strict";+import test from "node:test";+import { fixtureResource } from "../../../test/fixture-resource.js";++import type { SynchronizedMedia } from "@hypit/media";+import type { SemanticTake } from "@hypit/speech";+import {+ appendTimelineAuthorTake,+ assembleTimelineAuthor,+ createTimelineAuthorSet,+ sealTimelineAuthorHeader,+} from "@hypit/timeline-author";++function semantic(id: string, visual: boolean): SemanticTake {+ const media: SynchronizedMedia = {+ timeline: { frameRate: { numerator: 30, denominator: 1 }, frameCount: 30 },+ ...(visual ? { visual: {+ artifact: { kind: "blob", resource: fixtureResource(`${id}:video`), size: 1, mediaType: "video/mp4" },+ width: 720,+ height: 1280,+ } } : {}),+ audio: { artifact: { kind: "blob", resource: fixtureResource(`${id}:audio`), size: 1, mediaType: "audio/wav" } },+ };+ return {+ narrativeId: "test-narrative",+ media,+ segment: {+ segmentId: id,+ startAnchorId: `segment:${id}:start`,+ endAnchorId: `segment:${id}:end`,+ startFrame: 0,+ endFrameExclusive: 30,+ },+ tokens: [],+ anchors: [+ { identity: `segment:${id}:start`, frame: 0 },+ { identity: `segment:${id}:end`, frame: 30 },+ ],+ };+}++test("semantic assembly retains audio and material timing without requiring placement", () => {+ const track = assembleTimelineAuthor(sealTimelineAuthorHeader({ id: "speech" }),+ appendTimelineAuthorTake(appendTimelineAuthorTake(createTimelineAuthorSet(), semantic("voiceover", false)), semantic("answer", true)), { frameRate: { numerator: 30, denominator: 1 } });+ assert.deepEqual(track.items.map(item => item.take.segment.segmentId), ["voiceover", "answer"]);+ const audio = projectTimelineAudio(track);+ assert.deepEqual(audio.clips.map(clip => clip.target), [+ { startSample: 0, endSampleExclusive: 48000 }, { startSample: 48000, endSampleExclusive: 96000 },+ ]);+ const selected = projectTimelineMedia(track, { startFrame: 20, endFrameExclusive: 45 });+ assert.deepEqual(selected.map(({ span, source }) => ({ span, source })), [+ { span: { startFrame: 20, endFrameExclusive: 30 }, source: { startFrame: 20, endFrameExclusive: 30 } },+ { span: { startFrame: 30, endFrameExclusive: 45 }, source: { startFrame: 0, endFrameExclusive: 15 } },+ ]);+ assert.equal(selected[0]!.media.visual, undefined);+ assert.equal(selected[1]!.media.visual?.artifact.resource, fixtureResource("answer:video"));+});++const clock = { frameRate: { numerator: 30, denominator: 1 } };+function placed(end: string | undefined, at: readonly string[], lengths: readonly number[]) {+ let set = createTimelineAuthorSet();+ for (const [i, length] of lengths.entries()) {+ const take = semantic(`take-${i}`, true);+ set = appendTimelineAuthorTake(set, { ...take,+ media: { ...take.media, timeline: { ...take.media.timeline, frameCount: length } },+ segment: { ...take.segment, endFrameExclusive: length },+ anchors: [take.anchors[0]!, { ...take.anchors[1]!, frame: length }],+ });+ }+ return assembleTimelineAuthor({ id: "program", at, ...(end === undefined ? {} : { end }) }, set, clock);+}++test("one Timeline supports leading, interior and trailing gaps without source material", () => {+ const timeline = placed("30s", ["2s", "22s"], [180, 180]);+ assert.equal(timeline.durationSec, 30);+ assert.deepEqual(timeline.items.map(item => item.startFrame), [60, 660]);+ for (const [startFrame, endFrameExclusive] of [[0, 60], [240, 660], [840, 900]]) {+ assert.deepEqual(projectTimelineMedia(timeline, { startFrame: startFrame!, endFrameExclusive: endFrameExclusive! }), []);+ }+ assert.deepEqual(projectTimelineAudio(timeline).clips.map(clip => clip.target), [+ { startSample: 96000, endSampleExclusive: 384000 },+ { startSample: 1056000, endSampleExclusive: 1344000 },+ ]);+});++test("overlap retains both sources; content end is the latest end rather than declaration order", () => {+ const timeline = placed("content.end+2s", ["2s", "previous.end-12f", "3s"], [180, 60, 30]);+ assert.deepEqual(timeline.items.map(item => item.startFrame), [60, 228, 90]);+ assert.equal(timeline.durationSec, 348 / 30);+ const samples = projectTimelineMedia(timeline, { startFrame: 228, endFrameExclusive: 240 });+ assert.deepEqual(samples.map(sample => sample.source), [+ { startFrame: 168, endFrameExclusive: 180 }, { startFrame: 0, endFrameExclusive: 12 },+ ]);+ assert.equal(projectTimelineAudio(timeline).clips.length, 3);+});++test("a zero-Take Timeline supplies its Clock and extent without a Narrative or audio", () => {+ const timeline = assembleTimelineAuthor({ id: "animation", end: "30s" }, createTimelineAuthorSet(), clock);+ assert.equal(timeline.narrativeId, undefined);+ assert.equal(timeline.durationSec, 30);+ assert.deepEqual(timeline.items, []);+ assert.deepEqual(projectTimelineMedia(timeline), []);+ assert.deepEqual(projectTimelineAudio(timeline).clips, []);+ assert.throws(() => assembleTimelineAuthor({ id: "animation" }, createTimelineAuthorSet(), clock), /positive end/);+});++test("placement rejects missing predecessors, negative starts, fractional frames and clipped content", () => {+ assert.throws(() => placed(undefined, ["previous.end"], [30]), /no preceding/);+ assert.throws(() => placed(undefined, ["0f", "previous.end-2s"], [30, 30]), /non-negative/);+ assert.throws(() => placed(undefined, ["0.01s"], [30]), /exact frame/);+ assert.throws(() => placed("1s", ["2s"], [30]), /precedes placed content/);+});++test("word and Selection locations translate with the Take without reordering authored endpoints", async () => {+ const { semanticAnchorFrames, selectionFrameSpan, momentFrame, tokenFrameSpan } = await import("@hypit/timeline");+ const base = semantic("opening", true);+ const take = { ...base, tokens: [{ tokenId: "word", segmentId: "opening", text: "Hello",+ startAnchorId: "word:start", endAnchorId: "word:end", startFrame: 5, endFrameExclusive: 20 }],+ anchors: [...base.anchors, { identity: "word:start", frame: 5 }, { identity: "word:end", frame: 20 }] };+ const timeline = assembleTimelineAuthor({ id: "program", at: ["2s"], end: "5s" },+ appendTimelineAuthorTake(createTimelineAuthorSet(), take), clock);+ assert.deepEqual(tokenFrameSpan(timeline, ["word"]), { startFrame: 65, endFrameExclusive: 80 });+ assert.equal(momentFrame(timeline, { narrativeId: take.narrativeId, id: "cue", anchorId: "word:start" }), 65);+ assert.deepEqual(selectionFrameSpan(timeline, { narrativeId: take.narrativeId, id: "selection",+ startAnchorId: "word:end", endAnchorId: "word:start" }), { startFrame: 80, endFrameExclusive: 65 });+ const anchors = semanticAnchorFrames(timeline);+ assert.equal(anchors.get("program:start"), 0);+ assert.equal(anchors.get("program:end"), 150);+ assert.equal(anchors.get(base.segment.startAnchorId), 60);+ assert.deepEqual(take.tokens[0], timeline.items[0]!.take.tokens[0]);+});diff --git a/packages/speech-track/test/surface.test.ts b/packages/timeline-author/test/surface.test.tssimilarity index 69%rename from packages/speech-track/test/surface.test.tsrename to packages/timeline-author/test/surface.test.tsindex 7c13cd6c..b9703b5c 100644--- a/packages/speech-track/test/surface.test.ts+++ b/packages/timeline-author/test/surface.test.ts@@ -1,3 +1,4 @@+import { programSpaceTypes } from "@hypit/program-space";import assert from "node:assert/strict";import test from "node:test";import { fixtureResource } from "../../../test/fixture-resource.js";@@ -5,7 +6,7 @@ import { fixtureResource } from "../../../test/fixture-resource.js";import type { StructuredElement, SurfaceResolvedReference } from "@hypit/markup";import type { SemanticTake } from "@hypit/speech";import { speechTypes } from "@hypit/speech";-import { decodeSpeechTrackSurface } from "@hypit/speech-track";+import { decodeTimelineAuthorSurface } from "@hypit/timeline-author";const take: SemanticTake = {narrativeId: "test-narrative",@@ -35,6 +36,7 @@ const take: SemanticTake = {};function resolved(path: string): SurfaceResolvedReference | undefined {+ if (path === "clock") return { path, type: programSpaceTypes.clock, ref: { kind: "record", id: path } };if (path === "opening.take") return {path,ref: { kind: "record", id: path },@@ -44,26 +46,26 @@ function resolved(path: string): SurfaceResolvedReference | undefined {return undefined;}-test("Speech Track accepts only already-semantic Segment Takes", async () => {+test("Timeline accepts only already-semantic Segment Takes", async () => {const element: StructuredElement = {kind: "element",- name: "speech:Track",- attributes: { id: "speech" },- children: [{ kind: "element", name: "speech:Take",+ name: "time:Timeline",+ attributes: { id: "speech", clock: { kind: "reference", path: "clock" } },+ children: [{ kind: "element", name: "time:Take",attributes: { source: { kind: "reference", path: "opening.take" } }, children: [], range: { start: 20, end: 60 } }],range: { start: 0, end: 70 },};- const output = await decodeSpeechTrackSurface({+ const output = await decodeTimelineAuthorSurface({sourceName: "main.svml",element,resolveReference: resolved,- resolveAsset() { throw new Error("Speech Track does not resolve assets"); },+ resolveAsset() { throw new Error("Timeline does not resolve assets"); },});assert.deepEqual(output.components[0]?.outputs, {- semantic: "speech.semantic",- audio: "speech.audio",+ timeline: "speech.timeline",});- assert.equal(output.exports?.includes("speech.audio") ?? false, true);+ assert.deepEqual(output.exports, ["speech.timeline"]);+ assert.deepEqual(output.fragments?.[0]?.exports.map(port => port.name), ["timeline"]);assert.equal(output.records.length, 1);- assert.deepEqual(output.fragments?.[0]?.inputs.map(input => input.name), ["header", "take-1"]);+ assert.deepEqual(output.fragments?.[0]?.inputs.map(input => input.name), ["clock", "header", "take-1"]);});diff --git a/packages/timeline/README.md b/packages/timeline/README.mdnew file mode 100644index 00000000..c1d1422d--- /dev/null+++ b/packages/timeline/README.md@@ -0,0 +1,31 @@+# `@hypit/timeline`++One Timeline describes the complete film time range and the prepared Takes placed within it.+Semantic anchors exist wherever those Takes supply them; no second kind of time axis is needed+for authored animation. The value stores `id`, `durationSec`, `frameRate`, optional `narrativeId`,+and `items` containing `take` and `startFrame`. A zero-Take Timeline has no Narrative identity.++Each Take retains its normalized local media and semantic evidence. Global positions are its+placement start plus local positions. Items can leave gaps or overlap; their declaration order is+preserved. Frame rates and Narrative ownership agree, identities remain unique, and every complete+Take fits inside the positive Program range. Assembly is owned by+[Timeline authoring](../timeline-author/README.md).++`projectTimelineSpace` supplies the lightweight `ProgramSpace` range used by rendering.+`semanticAnchorFrames`, `selectionFrameSpan`, `segmentFrameSpan`, `tokenFrameSpan` and `momentFrame`+locate evidence without copying another word table. Program boundaries are available even with zero+Takes. Selection endpoints retain their authored identities; projection/consumption determines+whether a requested physical Window is usable.++## Project prepared material++`projectTimelineMedia(timeline, window?)` returns every intersecting Take's `media`, `segmentId`,+global `span` and local `source` frame span. The default Window is the complete Timeline. Gaps+return no source; overlaps return all intersecting sources. Audio-only Takes remain represented.+Starting presentation inside a Take retains the matching source frame. This projection chooses no+layout, winning picture, held frame or transition.++`projectTimelineAudio` places original audio at each Take's position at native speed and gain.+Takes without audio contribute no clip; gaps are silent and overlapping Takes contribute simultaneous+clips. Film explicitly includes that audio contribution. These functions perform no media I/O,+transcription, rendering or generation.diff --git a/packages/semantic-track/package.json b/packages/timeline/package.jsonsimilarity index 93%rename from packages/semantic-track/package.jsonrename to packages/timeline/package.jsonindex b59dcf44..97a1986f 100644--- a/packages/semantic-track/package.json+++ b/packages/timeline/package.json@@ -1,5 +1,5 @@{- "name": "@hypit/semantic-track",+ "name": "@hypit/timeline","version": "0.0.0-dev","license": "SEE LICENSE IN LICENSE","private": true,diff --git a/packages/timeline/src/activation.ts b/packages/timeline/src/activation.tsnew file mode 100644index 00000000..ce78077a--- /dev/null+++ b/packages/timeline/src/activation.ts@@ -0,0 +1,8 @@+import { timelineComponent, timelineManifest } from "./index.js";++export const hypitPackage = {+ format: "hypit.node-package@1" as const,+ modules: [{ manifest: timelineManifest }],+ components: [timelineComponent],+};+export default hypitPackage;diff --git a/packages/timeline/src/component.ts b/packages/timeline/src/component.tsnew file mode 100644index 00000000..ef3e1b1a--- /dev/null+++ b/packages/timeline/src/component.ts@@ -0,0 +1,24 @@+import type { ComponentPackage } from "@hypit/component-kit";+import type { StoredValue } from "@hypit/protocol";+import { canonicalize } from "@hypit/protocol";++import { timelineProducers, timelineTypes } from "./manifest.js";+import { assertTimelineIdentity } from "./identity.js";+import { projectTimelineAudio, projectTimelineSpace } from "./projection.js";+import type { Timeline } from "./types.js";++function track(value: StoredValue | undefined): Timeline {+ if (value?.kind !== "inline") throw new Error("Timeline must be inline.");+ return value.value as unknown as Timeline;+}++export const timelineComponent = {+ producers: [+ { producer: timelineProducers.projectProgramSpace, handler: ({ inputs }) => ({ outputs: { space: { kind: "inline", value: canonicalize(projectTimelineSpace(track(inputs.track?.value))) } }, needs: {} }) },+ { producer: timelineProducers.projectAudio, handler: ({ inputs }) => ({ outputs: { audio: { kind: "inline", value: canonicalize(projectTimelineAudio(track(inputs.track?.value))) } }, needs: {} }) },+ ],+ validators: [{+ type: timelineTypes.track,+ handler: ({ value }) => assertTimelineIdentity(track(value)),+ }],+} satisfies ComponentPackage;diff --git a/packages/timeline/src/identity.ts b/packages/timeline/src/identity.tsnew file mode 100644index 00000000..8ed4ec9a--- /dev/null+++ b/packages/timeline/src/identity.ts@@ -0,0 +1,60 @@+import { assertProgramSpaceIdentity, programSpaceFrameCount } from "@hypit/program-space";+import { assertSemanticTakeIdentity } from "@hypit/speech";++import type { Timeline, TimelineSpan } from "./types.js";++export function sealTimeline(value: Timeline): Timeline {+ const track = structuredClone(value);+ assertTimelineIdentity(track);+ return track;+}++export function assertTimelineIdentity(track: Timeline): void {+ assertProgramSpaceIdentity(track);+ const frameCount = programSpaceFrameCount(track);+ if (track.items.length > 0 && !track.narrativeId?.trim()) throw new Error("A Timeline with Takes requires their Narrative identity.");+ if (track.items.length === 0 && track.narrativeId !== undefined) throw new Error("An empty Timeline has no Narrative identity.");+ const segmentIds = new Set<string>();+ const tokenIds = new Set<string>();+ const anchorIds = new Set<string>();+ const frameRate = track.frameRate;+ for (const item of track.items) {+ assertSemanticTakeIdentity(item.take);+ if (!Number.isSafeInteger(item.startFrame) || item.startFrame < 0+ || item.startFrame + item.take.media.timeline.frameCount > frameCount) {+ throw new Error(`Timeline placement for ${item.take.segment.segmentId} is outside its Program range.`);+ }+ if (item.take.narrativeId !== track.narrativeId) {+ throw new Error(`Timeline ${track.id} mixes Narrative ${item.take.narrativeId} into ${track.narrativeId}.`);+ }+ const rate = item.take.media.timeline.frameRate;+ if (frameRate.numerator !== rate.numerator || frameRate.denominator !== rate.denominator) {+ throw new Error("Timeline items must use one frame rate.");+ }+ const segmentId = item.take.segment.segmentId;+ if (segmentIds.has(segmentId)) throw new Error(`Timeline repeats Segment ${segmentId}.`);+ segmentIds.add(segmentId);+ for (const token of item.take.tokens) {+ if (tokenIds.has(token.tokenId)) throw new Error(`Timeline repeats Token ${token.tokenId}.`);+ tokenIds.add(token.tokenId);+ }+ for (const anchor of item.take.anchors) {+ if (anchor.identity === "program:start" || anchor.identity === "program:end") {+ throw new Error(`SemanticTake cannot declare reserved Program Anchor ${anchor.identity}.`);+ }+ if (anchorIds.has(anchor.identity)) throw new Error(`Timeline repeats Anchor ${anchor.identity}.`);+ anchorIds.add(anchor.identity);+ }+ }+}++export function timelineSpans(track: Timeline): readonly TimelineSpan[] {+ assertTimelineIdentity(track);+ return track.items.map(item => ({ item, startFrame: item.startFrame,+ endFrameExclusive: item.startFrame + item.take.media.timeline.frameCount }));+}++export function timelineFrameCount(track: Timeline): number {+ assertTimelineIdentity(track);+ return programSpaceFrameCount(track);+}diff --git a/packages/timeline/src/index.ts b/packages/timeline/src/index.tsnew file mode 100644index 00000000..6f1cf7a0--- /dev/null+++ b/packages/timeline/src/index.ts@@ -0,0 +1,6 @@+export { timelineComponent } from "./component.js";+export { assertTimelineIdentity, sealTimeline, timelineFrameCount, timelineSpans } from "./identity.js";+export { assertNarrativeMomentIdentity, assertNarrativeSelectionIdentity, momentFrame, segmentFrameSpan, selectionFrameSpan, semanticAnchorFrames, tokenFrameSpan } from "./location.js";+export { timelineDependency, timelineManifest, timelineModuleRef, timelineProducers, timelineSchema, timelineTypes } from "./manifest.js";+export { projectTimelineAudio, projectTimelineSpace, projectTimelineMedia } from "./projection.js";+export type * from "./types.js";diff --git a/packages/semantic-track/src/location.ts b/packages/timeline/src/location.tssimilarity index 64%rename from packages/semantic-track/src/location.tsrename to packages/timeline/src/location.tsindex 6bfd633f..c7849d5f 100644--- a/packages/semantic-track/src/location.ts+++ b/packages/timeline/src/location.ts@@ -1,7 +1,7 @@import type { NarrativeExcerpt, NarrativeMomentRef, NarrativeSelectionRef } from "@hypit/narrative";-import { assertSemanticTrackIdentity, semanticTrackFrameCount, semanticTrackSpans } from "./identity.js";-import type { LocatedFrameSpan, SemanticTrack } from "./types.js";+import { assertTimelineIdentity, timelineFrameCount, timelineSpans } from "./identity.js";+import type { LocatedFrameSpan, Timeline } from "./types.js";export function assertNarrativeSelectionIdentity(selection: NarrativeSelectionRef): void {if (!selection.narrativeId || !selection.id || !selection.startAnchorId || !selection.endAnchorId) throw new Error("NarrativeSelection is invalid");@@ -11,33 +11,33 @@ export function assertNarrativeMomentIdentity(moment: NarrativeMomentRef): voidif (!moment.narrativeId || !moment.id || !moment.anchorId) throw new Error("NarrativeMoment is invalid");}-function assertNarrativeOwner(track: SemanticTrack, narrativeId: string, label: string): void {+function assertNarrativeOwner(track: Timeline, narrativeId: string, label: string): void {if (narrativeId !== track.narrativeId) {- throw new Error(`${label} belongs to Narrative ${narrativeId}, not SemanticTrack ${track.id}.`);+ throw new Error(`${label} belongs to Narrative ${narrativeId}, not Timeline ${track.id}.`);}}-export function semanticAnchorFrames(track: SemanticTrack): ReadonlyMap<string, number> {- assertSemanticTrackIdentity(track);+export function semanticAnchorFrames(track: Timeline): ReadonlyMap<string, number> {+ assertTimelineIdentity(track);return new Map([["program:start", 0] as const,- ...semanticTrackSpans(track).flatMap(({ item, startFrame }) =>+ ...timelineSpans(track).flatMap(({ item, startFrame }) =>item.take.anchors.map((anchor) => [anchor.identity, startFrame + anchor.frame] as const)),- ["program:end", semanticTrackFrameCount(track)] as const,+ ["program:end", timelineFrameCount(track)] as const,]);}function frameFor(frames: ReadonlyMap<string, number>, anchorId: string, owner: string): number {const frame = frames.get(anchorId);- if (frame === undefined) throw new Error(`${owner} names Anchor ${anchorId}, which this SemanticTrack does not contain.`);+ if (frame === undefined) throw new Error(`${owner} names Anchor ${anchorId}, which this Timeline does not contain.`);return frame;}export function selectionFrameSpan(- track: SemanticTrack,+ track: Timeline,selection: NarrativeSelectionRef,): LocatedFrameSpan {- assertSemanticTrackIdentity(track);+ assertTimelineIdentity(track);assertNarrativeSelectionIdentity(selection);assertNarrativeOwner(track, selection.narrativeId, `NarrativeSelection ${selection.id}`);const frames = semanticAnchorFrames(track);@@ -47,19 +47,19 @@ export function selectionFrameSpan(};}-export function segmentFrameSpan(track: SemanticTrack, segment: NarrativeExcerpt): LocatedFrameSpan {- assertSemanticTrackIdentity(track);+export function segmentFrameSpan(track: Timeline, segment: NarrativeExcerpt): LocatedFrameSpan {+ assertTimelineIdentity(track);if (segment.kind !== "segment" || !segment.id) throw new Error("Narrative Segment excerpt is invalid");assertNarrativeOwner(track, segment.narrativeId, `Narrative Segment ${segment.id}`);- const found = semanticTrackSpans(track).find(({ item }) => item.take.segment.segmentId === segment.id);- if (found === undefined) throw new Error(`SemanticTrack does not contain Segment ${segment.id}.`);+ const found = timelineSpans(track).find(({ item }) => item.take.segment.segmentId === segment.id);+ if (found === undefined) throw new Error(`Timeline does not contain Segment ${segment.id}.`);return { startFrame: found.startFrame, endFrameExclusive: found.endFrameExclusive };}-export function tokenFrameSpan(track: SemanticTrack, tokenIds: readonly string[]): LocatedFrameSpan | undefined {- assertSemanticTrackIdentity(track);+export function tokenFrameSpan(track: Timeline, tokenIds: readonly string[]): LocatedFrameSpan | undefined {+ assertTimelineIdentity(track);if (tokenIds.length === 0) return undefined;- const timing = new Map(semanticTrackSpans(track).flatMap(({ item, startFrame }) =>+ const timing = new Map(timelineSpans(track).flatMap(({ item, startFrame }) =>item.take.tokens.map((token) => [token.tokenId, {startFrame: startFrame + token.startFrame,endFrameExclusive: startFrame + token.endFrameExclusive,@@ -72,8 +72,8 @@ export function tokenFrameSpan(track: SemanticTrack, tokenIds: readonly string[]};}-export function momentFrame(track: SemanticTrack, moment: NarrativeMomentRef): number {- assertSemanticTrackIdentity(track);+export function momentFrame(track: Timeline, moment: NarrativeMomentRef): number {+ assertTimelineIdentity(track);assertNarrativeMomentIdentity(moment);assertNarrativeOwner(track, moment.narrativeId, `NarrativeMoment ${moment.id}`);const frames = semanticAnchorFrames(track);diff --git a/packages/timeline/src/manifest.ts b/packages/timeline/src/manifest.tsnew file mode 100644index 00000000..58cc1ce6--- /dev/null+++ b/packages/timeline/src/manifest.ts@@ -0,0 +1,39 @@+import { compositionDependency, compositionTypes } from "@hypit/composition";+import { narrativeDependency } from "@hypit/narrative";+import { programSpaceDependency, programSpaceSchema, programSpaceTypes } from "@hypit/program-space";+import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";+import { semanticTakeSchema, speechDependency } from "@hypit/speech";++export const timelineModuleRef = { name: "@hypit/timeline", version: "1" } as const;+export const timelineTypes = {+ track: { module: timelineModuleRef, name: "Timeline" },+} satisfies Record<string, TypeRef>;+export const timelineProducers = {+ projectProgramSpace: { module: timelineModuleRef, name: "project-program-space" },+ projectAudio: { module: timelineModuleRef, name: "project-audio-track" },+} satisfies Record<string, ProducerRef>;++const string = { kind: "string", minLength: 1 } as const;+const object = (fields: Readonly<Record<string, { readonly schema: ValueSchema; readonly optional?: boolean }>>): ValueSchema => ({ kind: "object", fields });+export const timelineSchema: ValueSchema = object({+ ...(programSpaceSchema.kind === "object" ? programSpaceSchema.fields : {}),+ narrativeId: { schema: string, optional: true },+ items: { schema: { kind: "array", items: object({+ startFrame: { schema: { kind: "number", integer: true, minimum: 0 } },+ take: { schema: semanticTakeSchema },+ }) } },+});++export const timelineManifest: ModuleManifest = {+ format: "hypit.module@1",+ name: timelineModuleRef.name,+ version: timelineModuleRef.version,+ dependencies: [speechDependency, narrativeDependency, programSpaceDependency, compositionDependency],+ types: [{ name: timelineTypes.track.name }],+ capabilities: [],+ producers: [+ { name: timelineProducers.projectProgramSpace.name, inputs: [{ name: "track", type: timelineTypes.track }], outputs: [{ name: "space", type: programSpaceTypes.programSpace }], needs: [] },+ { name: timelineProducers.projectAudio.name, inputs: [{ name: "track", type: timelineTypes.track }], outputs: [{ name: "audio", type: compositionTypes.audioTrack }], needs: [] },+ ],+};+export const timelineDependency = { module: timelineModuleRef } as const;diff --git a/packages/semantic-track/src/projection.ts b/packages/timeline/src/projection.tssimilarity index 65%rename from packages/semantic-track/src/projection.tsrename to packages/timeline/src/projection.tsindex 86be22c8..5f8c9c8f 100644--- a/packages/semantic-track/src/projection.ts+++ b/packages/timeline/src/projection.ts@@ -7,18 +7,18 @@ import {} from "@hypit/program-space";import type { ProgramSpace } from "@hypit/program-space";-import { assertSemanticTrackIdentity, semanticTrackFrameCount, semanticTrackSpans } from "./identity.js";-import type { SemanticTrack } from "./types.js";+import { assertTimelineIdentity, timelineFrameCount, timelineSpans } from "./identity.js";+import type { Timeline } from "./types.js";/** Prepared material in an authored program interval, with its original source position retained. */-export function projectSemanticMedia(track: SemanticTrack, window = {- startFrame: 0, endFrameExclusive: semanticTrackFrameCount(track),+export function projectTimelineMedia(track: Timeline, window = {+ startFrame: 0, endFrameExclusive: timelineFrameCount(track),}) {- assertSemanticTrackIdentity(track);+ assertTimelineIdentity(track);if (!Number.isSafeInteger(window.startFrame) || !Number.isSafeInteger(window.endFrameExclusive)|| window.startFrame < 0 || window.endFrameExclusive <= window.startFrame- || window.endFrameExclusive > semanticTrackFrameCount(track)) throw new Error("Semantic media window is outside the performance.");- return semanticTrackSpans(track).flatMap(({ item, startFrame, endFrameExclusive }) => {+ || window.endFrameExclusive > timelineFrameCount(track)) throw new Error("Semantic media window is outside the performance.");+ return timelineSpans(track).flatMap(({ item, startFrame, endFrameExclusive }) => {const start = Math.max(window.startFrame, startFrame);const end = Math.min(window.endFrameExclusive, endFrameExclusive);return end <= start ? [] : [{@@ -30,19 +30,19 @@ export function projectSemanticMedia(track: SemanticTrack, window = {});}-export function projectSemanticProgramSpace(track: SemanticTrack): ProgramSpace {- assertSemanticTrackIdentity(track);- const frameRate = track.items[0]!.take.media.timeline.frameRate;+export function projectTimelineSpace(track: Timeline): ProgramSpace {+ assertTimelineIdentity(track);+ const frameRate = track.frameRate;return sealProgramSpace({id: track.id,- durationSec: semanticTrackFrameCount(track) * frameRate.denominator / frameRate.numerator,+ durationSec: track.durationSec,frameRate,});}-export function projectSemanticAudioTrack(track: SemanticTrack): AudioTrack {- const space = projectSemanticProgramSpace(track);- const clips = semanticTrackSpans(track).flatMap(({ item, startFrame, endFrameExclusive }) => {+export function projectTimelineAudio(track: Timeline): AudioTrack {+ const space = projectTimelineSpace(track);+ const clips = timelineSpans(track).flatMap(({ item, startFrame, endFrameExclusive }) => {const audio = item.take.media.audio;if (audio === undefined) return [];const sourceSampleFrames = synchronizedMediaSampleFrames(item.take.media);diff --git a/packages/timeline/src/types.ts b/packages/timeline/src/types.tsnew file mode 100644index 00000000..382d1f1e--- /dev/null+++ b/packages/timeline/src/types.ts@@ -0,0 +1,24 @@+import type { ProgramSpace } from "@hypit/program-space";+import type { SemanticTake } from "@hypit/speech";++export type TimelineItem = {+ readonly take: SemanticTake;+ readonly startFrame: number;+};++/** One complete film time range, with prepared material placed where it belongs. */+export type Timeline = ProgramSpace & {+ readonly narrativeId?: string;+ readonly items: readonly TimelineItem[];+};++export type TimelineSpan = {+ readonly item: TimelineItem;+ readonly startFrame: number;+ readonly endFrameExclusive: number;+};++export type LocatedFrameSpan = {+ readonly startFrame: number;+ readonly endFrameExclusive: number;+};diff --git a/packages/typography-track/README.md b/packages/typography-track/README.mdindex 202c0b5a..30b2040f 100644--- a/packages/typography-track/README.md+++ b/packages/typography-track/README.md@@ -1,14 +1,14 @@# `@hypit/typography-track`-The Track Surface accepts `semantic={speech.semantic}` or `space={animation}`. Semantic context-resolves Script Selections and Moments; authored space supports clock-based animation. Shared `at`+The Track Surface accepts `timeline={program.timeline}`. The same Timeline supports authored+positions and, where prepared Takes supply evidence, Script Selections and Moments. Shared `at`inputs accept a Moment or a time such as `2s`; `at` with `for` produces a Window where required.Place independently authored titles, labels, verdicts and other text in the video. Their lifetimecan follow a Script Selection or Moment even when their wording differs from the speech. For wordsdisplayed as they are spoken, use a [Caption family](../caption-fine/README.md).-The Track takes `semantic` or `space` to supply ProgramSpace. Items use explicit Point, Frame or Path+The Track takes the shared `timeline`. Items use explicit Point, Frame or Pathplacement, exact fonts and Styles. Timing can follow a Segment, Selection, Moment with duration, thewhole program or explicit clock expressions. `.program` describes the text presentation; `.track`is the VisualTrack to include in Film. Each item has its own lifetime and stacking order.@@ -26,7 +26,7 @@ Content has two explicit author forms:<copy:Value id="headline-copy">A useful idea, clearly shown.</copy:Value>-<typo:Track id="titles" semantic={speech.semantic}>+<typo:Track id="titles" timeline={speech.timeline}><typo:Area id="headline" content={headline-copy}placement={layout.headline} style={title-style} during="program"/>diff --git a/packages/typography-track/package.json b/packages/typography-track/package.jsonindex fc7d9056..e7597c09 100644--- a/packages/typography-track/package.json+++ b/packages/typography-track/package.json@@ -18,7 +18,7 @@"@hypit/narrative": "workspace:*","@hypit/protocol": "workspace:*","@hypit/program-space": "workspace:*",- "@hypit/semantic-track": "workspace:*",+ "@hypit/timeline": "workspace:*","@hypit/spatial": "workspace:*","@hypit/svs": "workspace:*","@hypit/temporal": "workspace:*",diff --git a/packages/typography-track/src/component.ts b/packages/typography-track/src/component.tsindex a8062631..7bbf4289 100644--- a/packages/typography-track/src/component.ts+++ b/packages/typography-track/src/component.ts@@ -1,5 +1,5 @@+import type { Timeline } from "@hypit/timeline";import type { ComponentPackage } from "@hypit/component-kit";-import type { ProgramSpace } from "@hypit/program-space";import type { CompositableSurfaceRef } from "@hypit/media";import type { SpatialFrame, SpatialPath, SpatialPoint } from "@hypit/spatial";import type { StoredValue } from "@hypit/protocol";@@ -58,7 +58,7 @@ export const typographyTrackComponent = {handler: ({ inputs }) => ({ outputs: { set: output(appendProjectedTextItem(inline<TypographyTrackSet>(inputs.set?.value, "TypographyTrackSet"),inline<TypographyTrackHeader>(inputs.header?.value, "TypographyTrackHeader"),- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ inline<Timeline>(inputs.timeline?.value, "Timeline"),inline<TextPlacement>(inputs.placement?.value, "TextPlacement"),inline<TextItemSpec>(inputs.spec?.value, "TextItemSpec"),inline<TextStyle>(inputs.style?.value, "TextStyle"),@@ -76,14 +76,14 @@ export const typographyTrackComponent = {{producer: typographyTrackProducers.render,handler: ({ inputs }) => ({ outputs: { track: output(renderTypographyTrack(- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ inline<Timeline>(inputs.timeline?.value, "Timeline"),inline<TypographyTrackProgram>(inputs.program?.value, "TypographyTrackProgram"),)) }, needs: {} }),},{producer: typographyTrackProducers.renderMask,handler: ({ inputs }) => ({ outputs: { track: output(renderTextMaskTrack(- inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),+ inline<Timeline>(inputs.timeline?.value, "Timeline"),inline<TypographyTrackProgram>(inputs.program?.value, "TypographyTrackProgram"),inline<CompositableSurfaceRef>(inputs.material?.value, "CompositableSurfaceRef"),inline<TextMaskSpec>(inputs.spec?.value, "TextMaskSpec"),diff --git a/packages/typography-track/src/fragment.ts b/packages/typography-track/src/fragment.tsindex 73f60745..073fc984 100644--- a/packages/typography-track/src/fragment.ts+++ b/packages/typography-track/src/fragment.ts@@ -1,4 +1,5 @@-import { programSpaceTypes } from "@hypit/program-space";+import { timelineTypes } from "@hypit/timeline";+import { compositionTypes } from "@hypit/composition";import type { VisualTrack } from "@hypit/composition";import { sealGraphFragment } from "@hypit/elaborator";@@ -11,14 +12,14 @@ const operation = (id: string) => ({ kind: "fragment-operation" as const, operat/** Provider-free official lowering from TypographyTrackProgram to one peer VisualTrack. */export const typographyTrackFragment = sealGraphFragment({inputs: [- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },{ name: "program", type: typographyTrackTypes.program },],operations: [{id: "render",producer: typographyTrackProducers.render,- inputs: { space: input("space"), program: input("program") },+ inputs: { timeline: input("timeline"), program: input("program") },result: { kind: "output", name: "track" },},],diff --git a/packages/typography-track/src/manifest.ts b/packages/typography-track/src/manifest.tsindex 2411455b..fcc812f0 100644--- a/packages/typography-track/src/manifest.ts+++ b/packages/typography-track/src/manifest.ts@@ -1,9 +1,11 @@+import { timelineTypes } from "@hypit/timeline";+import type { Timeline } from "@hypit/timeline";import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";import { readFile } from "node:fs/promises";import { narrativeDependency } from "@hypit/narrative";-import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";-import { semanticTrackDependency } from "@hypit/semantic-track";++import { timelineDependency } from "@hypit/timeline";import {compositionDependency,compositionTypes,@@ -423,7 +425,7 @@ export const typographyTrackMarkupSurfaces = [},{ name: "track", tag: "Track", mode: "structured", outputs: [typographyTrackTypes.header, typographyTrackTypes.itemSpec, typographyTrackTypes.plainItemSpec, typographyTrackTypes.motion, typographyTrackTypes.set, typographyTrackTypes.placement, temporalTypes.instantSpec, temporalTypes.windowSpec, temporalTypes.instant, temporalTypes.window, typographyTrackTypes.program, compositionTypes.visualTrack],vocabulary: {- summary: "One Typography Track: independently placed and timed text items on a shared ProgramSpace, lowered to one addressable TypographyTrackProgram and one peer VisualTrack.",+ summary: "One Typography Track: independently placed and timed text items on a shared Timeline, lowered to one addressable TypographyTrackProgram and one peer VisualTrack.",appearance: "Text alone on an otherwise empty Canvas: the glyphs, plus whatever Paint the Style puts around them — fills, outlines, glows, shadows and rounded, bordered, optionally tailed boxes drawn behind the frame, paragraph, line, run, word or grapheme. Each item holds its own region of the Canvas: a Point item is one unwrapped block that hugs its text and hangs off a single coordinate by its inline and block anchors, an Area item flows and wraps inside a rectangle under its own alignment, columns, clipping and overflow, and a Path item strings the glyphs along a curve, on one side of it, turning with it or standing upright. Items switch on and off at their own frame windows and overlap in the stacking order their Styles declare, so titles, labels and captions can occupy different corners at once and outlast or outlive one another. While an item is on screen it plays its Motion: the whole block translating, scaling, rotating, skewing, fading, blurring, recoloring or wiping open from an edge, and its paragraphs, lines, runs, words or graphemes arriving one behind another in a staggered run.",preview: previewImage("Track.png"),attributes: [@@ -475,7 +477,7 @@ export const typographyTrackMarkupSurfaces = [{ name: "track", type: compositionTypes.visualTrack,summary: "The rendered text, an ordinary peer VisualTrack." },],- example: `<text:Track id="titles" semantic={speech.semantic}>+ example: `<text:Track id="titles" timeline={speech.timeline}><text:Area id="title" placement={title-frame} style={title-style} during="program">EDIT MEANING, NOT TIMELINES</text:Area>@@ -514,7 +516,7 @@ export const typographyTrackMarkupSurfaces = [{ name: "track", type: compositionTypes.visualTrack,summary: "The masked picture, an ordinary peer VisualTrack." },],- example: `<text:Mask id="masked-titles" semantic={speech.semantic} text={mask-shape.program} material={material}/>`,+ example: `<text:Mask id="masked-titles" timeline={speech.timeline} text={mask-shape.program} material={material}/>`,notes: ["A Mask is written empty and accepts no children.","The material must be a still Surface; a timed material is refused and materializes through an independent package.",@@ -529,7 +531,7 @@ export const typographyTrackManifest: ModuleManifest = {format: "hypit.module@1",name: typographyTrackModuleRef.name,version: typographyTrackModuleRef.version,- dependencies: [narrativeDependency, programSpaceDependency, semanticTrackDependency, spatialDependency, mediaDependency, compositionDependency, textDependency, temporalDependency],+ dependencies: [narrativeDependency, timelineDependency, spatialDependency, mediaDependency, compositionDependency, textDependency, temporalDependency],types: [{ name: typographyTrackTypes.style.name },{ name: typographyTrackTypes.motion.name },@@ -548,9 +550,9 @@ export const typographyTrackManifest: ModuleManifest = {{ name: typographyTrackProducers.bindArea.name, inputs: [{ name: "frame", type: spatialTypes.frame }], outputs: [{ name: "placement", type: typographyTrackTypes.placement }], needs: [] },{ name: typographyTrackProducers.bindPath.name, inputs: [{ name: "path", type: spatialTypes.path }], outputs: [{ name: "placement", type: typographyTrackTypes.placement }], needs: [] },{ name: typographyTrackProducers.createSet.name, inputs: [], outputs: [{ name: "set", type: typographyTrackTypes.set }], needs: [] },- { name: typographyTrackProducers.appendItem.name, inputs: [{ name: "set", type: typographyTrackTypes.set }, { name: "header", type: typographyTrackTypes.header }, { name: "space", type: programSpaceTypes.programSpace }, { name: "placement", type: typographyTrackTypes.placement }, { name: "spec", type: typographyTrackTypes.itemSpec }, { name: "style", type: typographyTrackTypes.style }, { name: "motion", type: typographyTrackTypes.motion }, { name: "window", type: temporalTypes.window }], outputs: [{ name: "set", type: typographyTrackTypes.set }], needs: [] },+ { name: typographyTrackProducers.appendItem.name, inputs: [{ name: "set", type: typographyTrackTypes.set }, { name: "header", type: typographyTrackTypes.header }, { name: "timeline", type: timelineTypes.track }, { name: "placement", type: typographyTrackTypes.placement }, { name: "spec", type: typographyTrackTypes.itemSpec }, { name: "style", type: typographyTrackTypes.style }, { name: "motion", type: typographyTrackTypes.motion }, { name: "window", type: temporalTypes.window }], outputs: [{ name: "set", type: typographyTrackTypes.set }], needs: [] },{ name: typographyTrackProducers.finalize.name, inputs: [{ name: "header", type: typographyTrackTypes.header }, { name: "set", type: typographyTrackTypes.set }], outputs: [{ name: "program", type: typographyTrackTypes.program }], needs: [] },- { name: typographyTrackProducers.render.name, inputs: [{ name: "space", type: programSpaceTypes.programSpace }, { name: "program", type: typographyTrackTypes.program }], outputs: [{ name: "track", type: compositionTypes.visualTrack }], needs: [] },- { name: typographyTrackProducers.renderMask.name, inputs: [{ name: "space", type: programSpaceTypes.programSpace }, { name: "program", type: typographyTrackTypes.program }, { name: "material", type: mediaTypes.compositableSurface }, { name: "spec", type: typographyTrackTypes.maskSpec }], outputs: [{ name: "track", type: compositionTypes.visualTrack }], needs: [] },+ { name: typographyTrackProducers.render.name, inputs: [{ name: "timeline", type: timelineTypes.track }, { name: "program", type: typographyTrackTypes.program }], outputs: [{ name: "track", type: compositionTypes.visualTrack }], needs: [] },+ { name: typographyTrackProducers.renderMask.name, inputs: [{ name: "timeline", type: timelineTypes.track }, { name: "program", type: typographyTrackTypes.program }, { name: "material", type: mediaTypes.compositableSurface }, { name: "spec", type: typographyTrackTypes.maskSpec }], outputs: [{ name: "track", type: compositionTypes.visualTrack }], needs: [] },],};diff --git a/packages/typography-track/src/program.ts b/packages/typography-track/src/program.tsindex bc77e22e..1987dbcd 100644--- a/packages/typography-track/src/program.ts+++ b/packages/typography-track/src/program.ts@@ -1,5 +1,5 @@+import type { Timeline } from "@hypit/timeline";import { assertProgramSpaceIdentity, programSpaceFrameCount } from "@hypit/program-space";-import type { ProgramSpace } from "@hypit/program-space";import {assertSpatialFrame,assertSpatialPath,@@ -304,7 +304,7 @@ function append(export function appendProjectedTextItem(set: TypographyTrackSet,header: TypographyTrackHeader,- space: ProgramSpace,+ timeline: Timeline,placement: TextPlacement,spec: TextItemSpec,style: TextStyle,@@ -312,7 +312,7 @@ export function appendProjectedTextItem(window: ProjectedWindow,): TypographyTrackSet {assertTextPlacement(placement);- assertTemporalWindowFor(window, { subjectId: spec.id, space });+ assertTemporalWindowFor(window, { subjectId: spec.id, space: timeline });return append(set, header, spec, style, motion, placement.geometry, window);}@@ -332,18 +332,18 @@ export function sealTypographyTrackProgram(value: TypographyTrackProgram): Typogreturn programContent(value);}-export function assertTypographyTrackProgramIdentity(program: TypographyTrackProgram, space: ProgramSpace): void {- assertProgramSpaceIdentity(space);+export function assertTypographyTrackProgramIdentity(program: TypographyTrackProgram, timeline: Timeline): void {+ assertProgramSpaceIdentity(timeline);nonEmpty(program.id, "TypographyTrackProgram id");if (program.items.length === 0) throw new Error("TypographyTrackProgram has no Items.");- const total = programSpaceFrameCount(space);+ const total = programSpaceFrameCount(timeline);const ids = new Set<string>();for (const item of program.items) {nonEmpty(item.id, "Text Item id");if (ids.has(item.id)) throw new Error(`TypographyTrackProgram repeats ${item.id}.`);ids.add(item.id);if (item.span.startFrame < 0 || item.span.endFrameExclusive <= item.span.startFrame || item.span.endFrameExclusive > total) {- throw new Error(`${item.id} is outside ProgramSpace.`);+ throw new Error(`${item.id} is outside Timeline.`);}assertGeometry(item.geometry);assertDocument(item.document, `${item.id} document`);@@ -556,10 +556,10 @@ function elements(item: TextItem): VisualElement[] {return [root, motion, terminalTextElement(item, "motion", 2)];}-export function renderTypographyTrack(space: ProgramSpace, program: TypographyTrackProgram): VisualTrack {- assertTypographyTrackProgramIdentity(program, space);+export function renderTypographyTrack(timeline: Timeline, program: TypographyTrackProgram): VisualTrack {+ assertTypographyTrackProgramIdentity(program, timeline);const track = sealVisualTrack({- programSpaceId: space.id,+ programSpaceId: timeline.id,visualIr: "hypit.visual-ir@1",id: program.id,presents: program.items.map((item) => ({@@ -569,24 +569,24 @@ export function renderTypographyTrack(space: ProgramSpace, program: TypographyTrelements: elements(item),})),});- assertVisualTrackIdentity(track, space);+ assertVisualTrackIdentity(track, timeline);return track;}export function renderTextMaskTrack(- space: ProgramSpace,+ timeline: Timeline,program: TypographyTrackProgram,material: CompositableSurfaceRef,spec: TextMaskSpec,): VisualTrack {- assertTypographyTrackProgramIdentity(program, space);+ assertTypographyTrackProgramIdentity(program, timeline);assertCompositableSurfaceRef(material);assertTextMaskSpec(spec);if (material.timing.kind !== "still") {throw new Error("Official Text Mask requires one explicit still material Surface; timed materials use an independent package.");}const track = sealVisualTrack({- programSpaceId: space.id,+ programSpaceId: timeline.id,visualIr: "hypit.visual-ir@1",id: spec.id,presents: program.items.map((item) => {@@ -617,6 +617,6 @@ export function renderTextMaskTrack(};}),});- assertVisualTrackIdentity(track, space);+ assertVisualTrackIdentity(track, timeline);return track;}diff --git a/packages/typography-track/src/surface.ts b/packages/typography-track/src/surface.tsindex 3e4aec29..e944631f 100644--- a/packages/typography-track/src/surface.ts+++ b/packages/typography-track/src/surface.ts@@ -1,5 +1,6 @@-import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";-import { programSpaceTypes } from "@hypit/program-space";+import { timelineTypes } from "@hypit/timeline";+import { resolveTemporalContext } from "@hypit/temporal-markup";+import { compositionTypes } from "@hypit/composition";import type {VisualColorPaint,@@ -649,7 +650,7 @@ function createTrackFragment(id: string, items: readonly FragmentItem[]): GraphF}const append = `text:set:append:${String(index + 1).padStart(4, "0")}`;const common = {- set: operation(current), header: input("header"), space: input("space"), placement: operation(bind),+ set: operation(current), header: input("header"), timeline: input("timeline"), placement: operation(bind),spec: item.contentName === undefined ? input(item.specName) : operation(materialized),style: input(item.styleName), motion: input(item.motionName), window: input(item.windowName),};@@ -658,10 +659,10 @@ function createTrackFragment(id: string, items: readonly FragmentItem[]): GraphF}operations.push({ id: "text:finalize", producer: typographyTrackProducers.finalize, inputs: { header: input("header"), set: operation(current) }, result: { kind: "output", name: "program" } },- { id: "text:render", producer: typographyTrackProducers.render, inputs: { space: input("space"), program: operation("text:finalize") }, result: { kind: "output", name: "track" } },+ { id: "text:render", producer: typographyTrackProducers.render, inputs: { timeline: input("timeline"), program: operation("text:finalize") }, result: { kind: "output", name: "track" } },);const inputEntries = [- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },{ name: "header", type: typographyTrackTypes.header },...items.flatMap((item) => [{ name: item.geometryName, type: item.placementKind === "point" ? spatialTypes.point : item.placementKind === "area" ? spatialTypes.frame : spatialTypes.path },@@ -688,10 +689,9 @@ function createTrackFragment(id: string, items: readonly FragmentItem[]): GraphF}export const decodeTypographyTrackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {- allowed(element, ["id", "semantic", "space"], ["id"]);+ allowed(element, ["id", "timeline"], ["id"]);const id = text(element, "id");const context = resolveTemporalContext({ element, resolveReference });- const time = createTemporalSpace({ id: id, element, ...context });const headerId = `${id}.__header`;const records: SurfaceRecordDraft[] = [{id: headerId, type: typographyTrackTypes.header,@@ -699,8 +699,8 @@ export const decodeTypographyTrackSurface: StructuredSurfaceHandler = ({ element}];const defaultMotionId = `${id}.__still-motion`;records.push({ id: defaultMotionId, type: typographyTrackTypes.motion, value: { kind: "inline", value: stillTextMotion(defaultMotionId) }, range: element.range });- const temporalComponents: SurfaceComponentDraft[] = [...time.components];- const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [...time.fragments];+ const temporalComponents: SurfaceComponentDraft[] = [];+ const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [];const items: Array<FragmentItem & {readonly geometry: SurfaceResolvedReference;readonly style: SurfaceResolvedReference;@@ -731,7 +731,7 @@ export const decodeTypographyTrackSurface: StructuredSurfaceHandler = ({ element"id", "content", "placement", "style", "motion", ...temporalWindowAttributeNames,], ["id", "placement", "style"]);const itemId = text(child, "id");- const temporal = createTemporalWindowProjection({ id: itemId, element: child, ...context, space: time.space, resolveReference });+ const temporal = createTemporalWindowProjection({ id: itemId, element: child, ...context, resolveReference });records.push(...temporal.records); temporalComponents.push(...temporal.components); temporalFragments.push(...temporal.fragments);const placementKind = form.toLowerCase() as "point" | "area" | "path";const geometry = reference(child.attributes.placement, `${child.name}.placement`, placementKind === "point" ? spatialTypes.point : placementKind === "area" ? spatialTypes.frame : spatialTypes.path, resolveReference);@@ -777,7 +777,7 @@ export const decodeTypographyTrackSurface: StructuredSurfaceHandler = ({ elementcomponents: [...temporalComponents, {id, fragment: fragment.id,inputs: {- space: time.space.ref, header: { kind: "record", id: headerId },+ timeline: context.timeline.ref, header: { kind: "record", id: headerId },...Object.fromEntries(items.flatMap((item) => [[item.geometryName, item.geometry.ref], [item.specName, { kind: "record" as const, id: item.specId }],[item.windowName, item.window],@@ -794,7 +794,7 @@ export const decodeTypographyTrackSurface: StructuredSurfaceHandler = ({ elementfunction createMaskFragment(id: string): GraphFragment {const inputs = [- { name: "space", type: programSpaceTypes.programSpace },+ { name: "timeline", type: timelineTypes.track },{ name: "program", type: typographyTrackTypes.program },{ name: "material", type: mediaTypes.compositableSurface },{ name: "spec", type: typographyTrackTypes.maskSpec },@@ -804,7 +804,7 @@ function createMaskFragment(id: string): GraphFragment {operations: [{id: "text-mask:render", producer: typographyTrackProducers.renderMask,- inputs: { space: input("space"), program: input("program"), material: input("material"), spec: input("spec") },+ inputs: { timeline: input("timeline"), program: input("program"), material: input("material"), spec: input("spec") },result: { kind: "output", name: "track" },},],@@ -815,11 +815,10 @@ function createMaskFragment(id: string): GraphFragment {}export const decodeTypographyMaskSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {- allowed(element, ["id", "semantic", "space", "text", "material", "mode", "fit"], ["id", "text", "material"]);+ allowed(element, ["id", "timeline", "text", "material", "mode", "fit"], ["id", "text", "material"]);empty(element);const id = text(element, "id");const context = resolveTemporalContext({ element, resolveReference });- const time = createTemporalSpace({ id: id, element, ...context });const program = reference(element.attributes.text, `${element.name}.text`, typographyTrackTypes.program, resolveReference);const material = reference(element.attributes.material, `${element.name}.material`, mediaTypes.compositableSurface, resolveReference);const specId = `${id}.__spec`;@@ -831,12 +830,12 @@ export const decodeTypographyMaskSurface: StructuredSurfaceHandler = ({ element,const fragment = createMaskFragment(id);return {records: [{ id: specId, type: typographyTrackTypes.maskSpec, value: { kind: "inline", value: spec }, range: element.range }],- components: [...time.components, {+ components: [{id, fragment: fragment.id,- inputs: { space: time.space.ref, program: program.ref, material: material.ref, spec: { kind: "record", id: specId } },+ inputs: { timeline: context.timeline.ref, program: program.ref, material: material.ref, spec: { kind: "record", id: specId } },outputs: { track: `${id}.track` }, range: element.range,}],- fragments: [...time.fragments, fragment],+ fragments: [fragment],exports: [`${id}.track`],};};diff --git a/packages/typography-track/test/typography-track.test.ts b/packages/typography-track/test/typography-track.test.tsindex 678469a0..df255ad5 100644--- a/packages/typography-track/test/typography-track.test.ts+++ b/packages/typography-track/test/typography-track.test.ts@@ -1,7 +1,8 @@+import { sealTimeline } from "@hypit/timeline";import assert from "node:assert/strict";import test from "node:test";import { fixtureResource } from "../../../test/fixture-resource.js";-import { semanticTrackFixture } from "../../../test/semantic-track-fixture.js";+import { timelineFixture } from "../../../test/timeline-fixture.js";import { projectSelectionWindow } from "../../../test/temporal-fixture.js";import { spatialComponent, videoContractManifests } from "../../../test/support/video-domain.js";@@ -16,7 +17,7 @@ import type { NarrativeSelectionRef } from "@hypit/narrative";import { sealProgramSpace } from "@hypit/program-space";import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";import type { ModuleManifest } from "@hypit/protocol";-import { semanticTrackDependency, semanticTrackTypes } from "@hypit/semantic-track";+import { timelineDependency, timelineTypes } from "@hypit/timeline";import {appendProjectedTextItem,bindAreaTextPlacement,@@ -49,10 +50,10 @@ import { sealText, textComponent, textDependency, textManifest, textTypes } fromimport { MarkupSurfaceRegistry, createMarkupAuthorFrontend } from "@hypit/markup";import { createRecordAdmitter, TypeValidatorRegistry } from "@hypit/validation";-const space = sealProgramSpace({ id: "test-space", durationSec: 5,+const space = sealTimeline({ items: [], id: "test-space", durationSec: 5,frameRate: { numerator: 30, denominator: 1 },});-const semantic = semanticTrackFixture(space, { anchors: [+const semantic = timelineFixture(space, { anchors: [{ identity: "selection:start", frame: 30 },{ identity: "selection:end", frame: 60 },] });@@ -258,7 +259,7 @@ test("Text Mask explicitly consumes one authored Text Program and one owned stil})), /must be materialized by an independent package/u);});-test("TypographyTrackProgram rejects a frame span outside ProgramSpace", () => {+test("TypographyTrackProgram rejects a frame span outside Timeline", () => {const program = sealTypographyTrackProgram({id: "invalid-text",@@ -272,10 +273,10 @@ test("TypographyTrackProgram rejects a frame span outside ProgramSpace", () => {motion: stillTextMotion(),}],});- assert.throws(() => renderTypographyTrack(space, program), /outside ProgramSpace/u);+ assert.throws(() => renderTypographyTrack(space, program), /outside Timeline/u);});-test("Selection Text consumes explicit Selection, SemanticTrack, Style, Motion and Placement edges", () => {+test("Selection Text consumes explicit Selection, Timeline, Style, Motion and Placement edges", () => {const selection: NarrativeSelectionRef = {narrativeId: "script",id: "callout",@@ -312,7 +313,7 @@ test("the self-described Markup Surfaces compile Style, Motion and all three spaname: "inputs", tag: "Inputs", mode: "structured",outputs: [svsRecipeType,- semanticTrackTypes.track,+ timelineTypes.track,spatialTypes.point,spatialTypes.frame,spatialTypes.path,@@ -326,7 +327,7 @@ test("the self-described Markup Surfaces compile Style, Motion and all three spaname: fixtureModule.name,version: fixtureModule.version,dependencies: [- semanticTrackDependency,+ timelineDependency,spatialDependency,mediaDependency,{@@ -384,7 +385,7 @@ test("the self-described Markup Surfaces compile Style, Motion and all three spa} },range: element.range,},- { id: "semantic", type: semanticTrackTypes.track, value: { kind: "inline", value: semantic }, range: element.range },+ { id: "semantic", type: timelineTypes.track, value: { kind: "inline", value: semantic }, range: element.range },{ id: "title-point", type: spatialTypes.point, value: { kind: "inline", value: { xPx: 540, yPx: 120 } }, range: element.range },{ id: "body-frame", type: spatialTypes.frame, value: { kind: "inline", value: { xPx: 80, yPx: 220, widthPx: 920, heightPx: 520 } }, range: element.range },{ id: "arc", type: spatialTypes.path, value: { kind: "inline", value: { commands: [@@ -435,17 +436,17 @@ test("the self-described Markup Surfaces compile Style, Motion and all three spa<text:Keyframe at="1" opacity="1"/></text:Sequence></text:Motion>- <text:Track id="titles" semantic={semantic}>+ <text:Track id="titles" timeline={semantic}><text:Point id="hook" content={copy} placement={title-point} style={poster} during="program"/><text:Area id="body" placement={body-frame} style={poster} motion={arrive} during="program"><text:P id="first">Rich <text:Span style={poster}>inline text</text:Span><text:Break/>wraps.</text:P></text:Area><text:Path id="arc-title" placement={arc} style={poster} during="program">Along the path</text:Path></text:Track>- <text:Track id="mask-shape" semantic={semantic}>+ <text:Track id="mask-shape" timeline={semantic}><text:Area id="mask-word" placement={body-frame} style={mask-shape-style} during="program">MASK</text:Area></text:Track>- <text:Mask id="masked-titles" semantic={semantic} text={mask-shape.program} material={material}/>+ <text:Mask id="masked-titles" timeline={semantic} text={mask-shape.program} material={material}/></svml>`,},closure,@@ -607,13 +608,13 @@ test("a paragraph's source indentation is not part of its words", async () => {const fixtureSurface = {name: "inputs", tag: "Inputs", mode: "structured",outputs: [- svsRecipeType, semanticTrackTypes.track, spatialTypes.frame,+ svsRecipeType, timelineTypes.track, spatialTypes.frame,mediaTypes.fontArtifact, textTypes.text,],} as const;const fixtureManifest: ModuleManifest = {format: "hypit.module@1", name: fixtureModule.name, version: fixtureModule.version,- dependencies: [semanticTrackDependency, spatialDependency, mediaDependency,+ dependencies: [timelineDependency, spatialDependency, mediaDependency,{ module: { name: svsManifest.name, version: svsManifest.version } }, textDependency],types: [], capabilities: [], producers: [],};@@ -627,7 +628,7 @@ test("a paragraph's source indentation is not part of its words", async () => {path: "text.editorial", properties: { "stack-order": 70, size: 44, "line-height": 1.15,"inline-size": "fixed", "block-size": "fixed", wrap: "word", overflow: "shrink", "minimum-scale": 0.65 } } },range: element.range },- { id: "semantic", type: semanticTrackTypes.track, value: { kind: "inline", value: semantic }, range: element.range },+ { id: "semantic", type: timelineTypes.track, value: { kind: "inline", value: semantic }, range: element.range },{ id: "body-frame", type: spatialTypes.frame, value: { kind: "inline", value: { xPx: 80, yPx: 220, widthPx: 920, heightPx: 520 } }, range: element.range },{ id: "exact-font", type: mediaTypes.fontArtifact, value: { kind: "inline", value: exactTestFont }, range: element.range },],@@ -654,7 +655,7 @@ test("a paragraph's source indentation is not part of its words", async () => {<text:Style id="poster" recipe={editorial} font={exact-font}><text:Fill color="#f8fafc"/></text:Style>- <text:Track id="titles" semantic={semantic}>+ <text:Track id="titles" timeline={semantic}><text:Area id="body" placement={body-frame} style={poster} during="program"><text:P id="first">Top 5 Most Populardiff --git a/packages/video-cli/src/studio-distribution.ts b/packages/video-cli/src/studio-distribution.tsindex 59b59204..e76ccb4b 100644--- a/packages/video-cli/src/studio-distribution.ts+++ b/packages/video-cli/src/studio-distribution.ts@@ -11,9 +11,10 @@ export const videoStudioCompanionPackages = ["@hypit/deck-track-studio","@hypit/film-studio","@hypit/media-track-studio",+ "@hypit/performance-studio","@hypit/ranking-studio","@hypit/screen-overlay-studio","@hypit/script-studio",- "@hypit/speech-track-studio",+ "@hypit/sound-studio","@hypit/typography-track-studio",] as const;diff --git a/packages/video-cli/test/packages.ts b/packages/video-cli/test/packages.tsindex e6eb38a6..6add8072 100644--- a/packages/video-cli/test/packages.ts+++ b/packages/video-cli/test/packages.ts@@ -18,10 +18,10 @@ import runMarkup from "../../run-markup/src/activation.js";import script from "../../script/src/activation.js";import seedance from "../../seedance/src/activation.js";import speechAlignment from "../../speech-alignment/src/activation.js";-import speechTrack from "../../speech-track/src/activation.js";+import timelineAuthor from "../../timeline-author/src/activation.js";import speech from "../../speech/src/activation.js";import speechEvidence from "../../speech-evidence/src/activation.js";-import semanticTrack from "../../semantic-track/src/activation.js";+import timeline from "../../timeline/src/activation.js";import spatial from "../../spatial/src/activation.js";import temporal from "../../temporal/src/activation.js";import visualIr from "../../visual-ir/src/activation.js";@@ -43,7 +43,7 @@ export const videoTestPackages = [bind("@hypit/program-space", programSpace),bind("@hypit/speech", speech),bind("@hypit/speech-evidence", speechEvidence),- bind("@hypit/semantic-track", semanticTrack),+ bind("@hypit/timeline", timeline),bind("@hypit/visual-ir", visualIr),bind("@hypit/composition", composition),bind("@hypit/svs", svs),@@ -56,7 +56,7 @@ export const videoTestPackages = [bind("@hypit/caption", caption),bind("@hypit/caption-fine", captionFine),bind("@hypit/speech-alignment", speechAlignment),- bind("@hypit/speech-track", speechTrack),+ bind("@hypit/timeline-author", timelineAuthor),bind("@hypit/whisperx", whisperX),bind("@hypit/spatial", spatial),bind("@hypit/temporal", temporal),diff --git a/packages/volcengine-matting/README.md b/packages/volcengine-matting/README.mdindex 4432e7c2..cb92c3db 100644--- a/packages/volcengine-matting/README.md+++ b/packages/volcengine-matting/README.md@@ -30,20 +30,20 @@ formats. The service's flat-background MP4 option is a different output treatmenThis excerpt assumes the performance and Script exist. Normalize keeps the transparent pictureand prepares the selected embedded audio on the program clock. WhisperX and semantic alignmentassociate that prepared performance with the Script. Use `opening-semantic.take` in-[Speech Track](../speech-track/README.md); its screen position and stack order are independent+[Timeline assembly](../timeline-author/README.md); its screen position and stack order are independentof its role as A-roll.For B-roll, normalize `cutout.video` with `audio="none"` when its sound is unwanted, then use`cutout-media.media` in [Media Track](../media-track/README.md). No SemanticTake is needed forthat overlay. Existing transparency can enter Normalize directly.-For example, this alternative uses the existing program's SemanticTrack and an authored Selection:+For example, this alternative uses the existing program's Timeline and an authored Selection:```svml<import as="media-track" from="@hypit/media-track@1"/><pipeline:Normalize id="overlay-media" source={cutout.video} clock={clock}video="primary-moving" audio="none" span-authority="video"/>-<media-track:Track id="overlay" semantic={speech.semantic} canvas={canvas}>+<media-track:Track id="overlay" timeline={speech.timeline} canvas={canvas}><media-track:Item media={overlay-media.media} during={story.selection.example}frame={overlay-frame} appearance={look.media.overlay}/></media-track:Track>diff --git a/packages/whisperx/README.md b/packages/whisperx/README.mdindex 8e8c767f..3986d612 100644--- a/packages/whisperx/README.md+++ b/packages/whisperx/README.md@@ -33,7 +33,7 @@ so this branch requests no evidence audio and no WhisperX capability:segment={story.segment.pause} media={pause-media.media}/>```-There is no whole-program WhisperX pass. Speech Track only receives already-semantic Takes and later+There is no whole-program WhisperX pass. Timeline assembly only receives already-semantic Takes and latertranslates their local frames when assembling the final ProgramSpace and complete semantic map.`@hypit/provider-hypihub` is the default concrete adapter; it uploads the canonical evidence audiodiff --git a/packages/whisperx/test/whisperx.test.ts b/packages/whisperx/test/whisperx.test.tsindex 077f2e6a..694adf3f 100644--- a/packages/whisperx/test/whisperx.test.ts+++ b/packages/whisperx/test/whisperx.test.ts@@ -38,6 +38,7 @@ test("WhisperX receives normalized bytes without authored Segment truth", () =>test("the real-media Surface materializes an empty Segment from its media domain", async () => {const narrative: Narrative = {id: "wordless-real",+ caption: { id: "wordless-real.caption", narrativeId: "wordless-real", units: [], words: [], cueBreaks: [] },segments: [{id: "pause",startAnchorId: "pause:start",diff --git a/pnpm-lock.yaml b/pnpm-lock.yamlindex ee03080d..e9103ca3 100644--- a/pnpm-lock.yaml+++ b/pnpm-lock.yaml@@ -153,6 +153,12 @@ importers:'@hypit/narrative':specifier: workspace:*version: link:packages/narrative+ '@hypit/performance':+ specifier: workspace:*+ version: link:packages/performance+ '@hypit/performance-studio':+ specifier: workspace:*+ version: link:packages/performance-studio'@hypit/program-space':specifier: workspace:*version: link:packages/program-space@@ -234,9 +240,12 @@ importers:'@hypit/seedream':specifier: workspace:*version: link:packages/seedream- '@hypit/semantic-track':+ '@hypit/sound':+ specifier: workspace:*+ version: link:packages/sound+ '@hypit/sound-studio':specifier: workspace:*- version: link:packages/semantic-track+ version: link:packages/sound-studio'@hypit/spatial':specifier: workspace:*version: link:packages/spatial@@ -249,12 +258,6 @@ importers:'@hypit/speech-evidence':specifier: workspace:*version: link:packages/speech-evidence- '@hypit/speech-track':- specifier: workspace:*- version: link:packages/speech-track- '@hypit/speech-track-studio':- specifier: workspace:*- version: link:packages/speech-track-studio'@hypit/store-sqlite':specifier: workspace:*version: link:packages/store-sqlite@@ -273,6 +276,12 @@ importers:'@hypit/text':specifier: workspace:*version: link:packages/text+ '@hypit/timeline':+ specifier: workspace:*+ version: link:packages/timeline+ '@hypit/timeline-author':+ specifier: workspace:*+ version: link:packages/timeline-author'@hypit/typography-track':specifier: workspace:*version: link:packages/typography-track@@ -325,6 +334,18 @@ importers:specifier: 5.9.3version: 5.9.3+ examples/semantic-composition/packages/performance-styles:+ devDependencies:+ '@hypit/hypit':+ specifier: workspace:*+ version: link:../../../..+ '@types/node':+ specifier: 24.10.1+ version: 24.10.1+ typescript:+ specifier: 5.9.3+ version: 5.9.3+examples/semantic-composition/packages/responsive-explainer:devDependencies:'@hypit/hypit':@@ -337,6 +358,18 @@ importers:specifier: 5.9.3version: 5.9.3+ examples/semantic-composition/packages/sound-styles:+ devDependencies:+ '@hypit/hypit':+ specifier: workspace:*+ version: link:../../../..+ '@types/node':+ specifier: 24.10.1+ version: 24.10.1+ typescript:+ specifier: 5.9.3+ version: 5.9.3+packages/artifact:dependencies:'@hypit/protocol':@@ -369,15 +402,15 @@ importers:'@hypit/protocol':specifier: workspace:*version: link:../protocol- '@hypit/semantic-track':- specifier: workspace:*- version: link:../semantic-track'@hypit/temporal':specifier: workspace:*version: link:../temporal'@hypit/temporal-markup':specifier: workspace:*version: link:../temporal-markup+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timelinedevDependencies:'@hypit/artifact':specifier: workspace:*@@ -523,9 +556,12 @@ importers:'@hypit/protocol':specifier: workspace:*version: link:../protocol- '@hypit/semantic-track':+ '@hypit/temporal':+ specifier: workspace:*+ version: link:../temporal+ '@hypit/timeline':specifier: workspace:*- version: link:../semantic-track+ version: link:../timelinedevDependencies:'@hypit/program-space':specifier: workspace:*@@ -572,15 +608,21 @@ importers:'@hypit/protocol':specifier: workspace:*version: link:../protocol- '@hypit/semantic-track':- specifier: workspace:*- version: link:../semantic-track'@hypit/spatial':specifier: workspace:*version: link:../spatial'@hypit/svs':specifier: workspace:*version: link:../svs+ '@hypit/temporal':+ specifier: workspace:*+ version: link:../temporal+ '@hypit/temporal-markup':+ specifier: workspace:*+ version: link:../temporal-markup+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timelinedevDependencies:'@hypit/script':specifier: workspace:*@@ -588,6 +630,9 @@ importers:packages/caption-fine-studio:dependencies:+ '@hypit/caption':+ specifier: workspace:*+ version: link:../caption'@hypit/caption-fine':specifier: workspace:*version: link:../caption-fine@@ -603,6 +648,9 @@ importers:'@hypit/studio-adapter':specifier: workspace:*version: link:../studio-adapter+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timelinepackages/cli:dependencies:@@ -694,9 +742,6 @@ importers:'@hypit/protocol':specifier: workspace:*version: link:../protocol- '@hypit/semantic-track':- specifier: workspace:*- version: link:../semantic-track'@hypit/spatial':specifier: workspace:*version: link:../spatial@@ -712,6 +757,9 @@ importers:'@hypit/text':specifier: workspace:*version: link:../text+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timelinedevDependencies:'@hypit/hyperframes':specifier: workspace:*@@ -896,9 +944,6 @@ importers:'@hypit/protocol':specifier: workspace:*version: link:../protocol- '@hypit/semantic-track':- specifier: workspace:*- version: link:../semantic-track'@hypit/spatial':specifier: workspace:*version: link:../spatial@@ -914,6 +959,9 @@ importers:'@hypit/text':specifier: workspace:*version: link:../text+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timelinedevDependencies:'@hypit/core':specifier: workspace:*@@ -1017,9 +1065,6 @@ importers:'@hypit/protocol':specifier: workspace:*version: link:../protocol- '@hypit/semantic-track':- specifier: workspace:*- version: link:../semantic-track'@hypit/spatial':specifier: workspace:*version: link:../spatial@@ -1029,6 +1074,9 @@ importers:'@hypit/temporal-markup':specifier: workspace:*version: link:../temporal-markup+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timelinedevDependencies:'@hypit/core':specifier: workspace:*@@ -1063,12 +1111,12 @@ importers:'@hypit/program-space':specifier: workspace:*version: link:../program-space- '@hypit/semantic-track':- specifier: workspace:*- version: link:../semantic-track'@hypit/studio-adapter':specifier: workspace:*version: link:../studio-adapter+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timelinepackages/fonts-open:dependencies:@@ -1655,9 +1703,6 @@ importers:'@hypit/protocol':specifier: workspace:*version: link:../protocol- '@hypit/semantic-track':- specifier: workspace:*- version: link:../semantic-track'@hypit/spatial':specifier: workspace:*version: link:../spatial@@ -1670,6 +1715,9 @@ importers:'@hypit/temporal-markup':specifier: workspace:*version: link:../temporal-markup+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timelinepackages/markup:dependencies:@@ -1787,9 +1835,6 @@ importers:'@hypit/protocol':specifier: workspace:*version: link:../protocol- '@hypit/semantic-track':- specifier: workspace:*- version: link:../semantic-track'@hypit/spatial':specifier: workspace:*version: link:../spatial@@ -1802,6 +1847,9 @@ importers:'@hypit/temporal-markup':specifier: workspace:*version: link:../temporal-markup+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timelinepackages/media-track-studio:dependencies:@@ -1926,6 +1974,75 @@ importers:specifier: workspace:*version: link:../protocol+ packages/performance:+ dependencies:+ '@hypit/artifact':+ specifier: workspace:*+ version: link:../artifact+ '@hypit/component-kit':+ specifier: workspace:*+ version: link:../component-kit+ '@hypit/composition':+ specifier: workspace:*+ version: link:../composition+ '@hypit/elaborator':+ specifier: workspace:*+ version: link:../elaborator+ '@hypit/markup':+ specifier: workspace:*+ version: link:../markup+ '@hypit/media':+ specifier: workspace:*+ version: link:../media+ '@hypit/media-track':+ specifier: workspace:*+ version: link:../media-track+ '@hypit/narrative':+ specifier: workspace:*+ version: link:../narrative+ '@hypit/program-space':+ specifier: workspace:*+ version: link:../program-space+ '@hypit/protocol':+ specifier: workspace:*+ version: link:../protocol+ '@hypit/spatial':+ specifier: workspace:*+ version: link:../spatial+ '@hypit/svs':+ specifier: workspace:*+ version: link:../svs+ '@hypit/temporal':+ specifier: workspace:*+ version: link:../temporal+ '@hypit/temporal-markup':+ specifier: workspace:*+ version: link:../temporal-markup+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timeline++ packages/performance-studio:+ dependencies:+ '@hypit/composition':+ specifier: workspace:*+ version: link:../composition+ '@hypit/media-track-studio':+ specifier: workspace:*+ version: link:../media-track-studio+ '@hypit/performance':+ specifier: workspace:*+ version: link:../performance+ '@hypit/studio-adapter':+ specifier: workspace:*+ version: link:../studio-adapter+ '@hypit/temporal':+ specifier: workspace:*+ version: link:../temporal+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timeline+packages/program-space:dependencies:'@hypit/markup':@@ -2319,9 +2436,6 @@ importers:'@hypit/protocol':specifier: workspace:*version: link:../protocol- '@hypit/semantic-track':- specifier: workspace:*- version: link:../semantic-track'@hypit/spatial':specifier: workspace:*version: link:../spatial@@ -2337,6 +2451,9 @@ importers:'@hypit/text':specifier: workspace:*version: link:../text+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timelinedevDependencies:'@hypit/core':specifier: workspace:*@@ -2401,12 +2518,12 @@ importers:'@hypit/protocol':specifier: workspace:*version: link:../protocol- '@hypit/semantic-track':- specifier: workspace:*- version: link:../semantic-track'@hypit/temporal-markup':specifier: workspace:*version: link:../temporal-markup+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timelinedevDependencies:'@hypit/core':specifier: workspace:*@@ -2602,9 +2719,6 @@ importers:'@hypit/protocol':specifier: workspace:*version: link:../protocol- '@hypit/semantic-track':- specifier: workspace:*- version: link:../semantic-track'@hypit/spatial':specifier: workspace:*version: link:../spatial@@ -2614,6 +2728,9 @@ importers:'@hypit/temporal-markup':specifier: workspace:*version: link:../temporal-markup+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timelinedevDependencies:'@hypit/artifact':specifier: workspace:*@@ -2670,6 +2787,12 @@ importers:packages/script-studio:dependencies:+ '@hypit/narrative':+ specifier: workspace:*+ version: link:../narrative+ '@hypit/protocol':+ specifier: workspace:*+ version: link:../protocol'@hypit/script':specifier: workspace:*version: link:../script@@ -2752,7 +2875,7 @@ importers:specifier: workspace:*version: link:../speech- packages/semantic-track:+ packages/sound:dependencies:'@hypit/component-kit':specifier: workspace:*@@ -2760,9 +2883,12 @@ importers:'@hypit/composition':specifier: workspace:*version: link:../composition- '@hypit/media':+ '@hypit/elaborator':specifier: workspace:*- version: link:../media+ version: link:../elaborator+ '@hypit/markup':+ specifier: workspace:*+ version: link:../markup'@hypit/narrative':specifier: workspace:*version: link:../narrative@@ -2772,9 +2898,33 @@ importers:'@hypit/protocol':specifier: workspace:*version: link:../protocol- '@hypit/speech':+ '@hypit/temporal':specifier: workspace:*- version: link:../speech+ version: link:../temporal+ '@hypit/temporal-markup':+ specifier: workspace:*+ version: link:../temporal-markup+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timeline++ packages/sound-studio:+ dependencies:+ '@hypit/composition':+ specifier: workspace:*+ version: link:../composition+ '@hypit/sound':+ specifier: workspace:*+ version: link:../sound+ '@hypit/studio-adapter':+ specifier: workspace:*+ version: link:../studio-adapter+ '@hypit/temporal':+ specifier: workspace:*+ version: link:../temporal+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timelinepackages/source:dependencies:@@ -2856,55 +3006,6 @@ importers:specifier: workspace:*version: link:../protocol- packages/speech-track:- dependencies:- '@hypit/component-kit':- specifier: workspace:*- version: link:../component-kit- '@hypit/composition':- specifier: workspace:*- version: link:../composition- '@hypit/elaborator':- specifier: workspace:*- version: link:../elaborator- '@hypit/markup':- specifier: workspace:*- version: link:../markup- '@hypit/protocol':- specifier: workspace:*- version: link:../protocol- '@hypit/semantic-track':- specifier: workspace:*- version: link:../semantic-track- '@hypit/speech':- specifier: workspace:*- version: link:../speech- devDependencies:- '@hypit/core':- specifier: workspace:*- version: link:../core- '@hypit/script':- specifier: workspace:*- version: link:../script- '@hypit/text':- specifier: workspace:*- version: link:../text- '@hypit/validation':- specifier: workspace:*- version: link:../validation-- packages/speech-track-studio:- dependencies:- '@hypit/composition':- specifier: workspace:*- version: link:../composition- '@hypit/speech-track':- specifier: workspace:*- version: link:../speech-track- '@hypit/studio-adapter':- specifier: workspace:*- version: link:../studio-adapter-packages/store-sqlite:dependencies:'@hypit/core':@@ -2967,12 +3068,12 @@ importers:'@hypit/media-track-studio':specifier: workspace:*version: link:../media-track-studio- '@hypit/narrative':- specifier: workspace:*- version: link:../narrative'@hypit/package-loader-node':specifier: workspace:*version: link:../package-loader-node+ '@hypit/performance-studio':+ specifier: workspace:*+ version: link:../performance-studio'@hypit/program-space':specifier: workspace:*version: link:../program-space@@ -2997,15 +3098,12 @@ importers:'@hypit/script-studio':specifier: workspace:*version: link:../script-studio- '@hypit/semantic-track':+ '@hypit/sound-studio':specifier: workspace:*- version: link:../semantic-track+ version: link:../sound-studio'@hypit/speech':specifier: workspace:*version: link:../speech- '@hypit/speech-track-studio':- specifier: workspace:*- version: link:../speech-track-studio'@hypit/studio-adapter':specifier: workspace:*version: link:../studio-adapter@@ -3015,6 +3113,9 @@ importers:'@hypit/temporal':specifier: workspace:*version: link:../temporal+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timeline'@hypit/typography-track-studio':specifier: workspace:*version: link:../typography-track-studio@@ -3073,9 +3174,9 @@ importers:'@hypit/protocol':specifier: workspace:*version: link:../protocol- '@hypit/semantic-track':+ '@hypit/timeline':specifier: workspace:*- version: link:../semantic-track+ version: link:../timelinepackages/temporal-markup:dependencies:@@ -3091,12 +3192,12 @@ importers:'@hypit/program-space':specifier: workspace:*version: link:../program-space- '@hypit/semantic-track':- specifier: workspace:*- version: link:../semantic-track'@hypit/temporal':specifier: workspace:*version: link:../temporal+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timelinepackages/text:dependencies:@@ -3119,6 +3220,73 @@ importers:specifier: workspace:*version: link:../svs+ packages/timeline:+ dependencies:+ '@hypit/component-kit':+ specifier: workspace:*+ version: link:../component-kit+ '@hypit/composition':+ specifier: workspace:*+ version: link:../composition+ '@hypit/media':+ specifier: workspace:*+ version: link:../media+ '@hypit/narrative':+ specifier: workspace:*+ version: link:../narrative+ '@hypit/program-space':+ specifier: workspace:*+ version: link:../program-space+ '@hypit/protocol':+ specifier: workspace:*+ version: link:../protocol+ '@hypit/speech':+ specifier: workspace:*+ version: link:../speech++ packages/timeline-author:+ dependencies:+ '@hypit/component-kit':+ specifier: workspace:*+ version: link:../component-kit+ '@hypit/elaborator':+ specifier: workspace:*+ version: link:../elaborator+ '@hypit/markup':+ specifier: workspace:*+ version: link:../markup+ '@hypit/program-space':+ specifier: workspace:*+ version: link:../program-space+ '@hypit/protocol':+ specifier: workspace:*+ version: link:../protocol+ '@hypit/speech':+ specifier: workspace:*+ version: link:../speech+ '@hypit/temporal':+ specifier: workspace:*+ version: link:../temporal+ '@hypit/temporal-markup':+ specifier: workspace:*+ version: link:../temporal-markup+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timeline+ devDependencies:+ '@hypit/core':+ specifier: workspace:*+ version: link:../core+ '@hypit/script':+ specifier: workspace:*+ version: link:../script+ '@hypit/text':+ specifier: workspace:*+ version: link:../text+ '@hypit/validation':+ specifier: workspace:*+ version: link:../validation+packages/transport-aws-lambda:dependencies:'@aws-sdk/client-lambda':@@ -3154,9 +3322,6 @@ importers:'@hypit/protocol':specifier: workspace:*version: link:../protocol- '@hypit/semantic-track':- specifier: workspace:*- version: link:../semantic-track'@hypit/spatial':specifier: workspace:*version: link:../spatial@@ -3172,6 +3337,9 @@ importers:'@hypit/text':specifier: workspace:*version: link:../text+ '@hypit/timeline':+ specifier: workspace:*+ version: link:../timeline'@hypit/visual-ir':specifier: workspace:*version: link:../visual-irdiff --git a/skills/hypit/SKILL.md b/skills/hypit/SKILL.mdindex cdd0f49d..68b122ed 100644--- a/skills/hypit/SKILL.md+++ b/skills/hypit/SKILL.md@@ -30,21 +30,28 @@ When setup, generation or rendering takes time, explain what is running or whatwork. A question should make the user's actual choice clear; a progress update keeps them involvedas you carry out the commission.-Most work Hypit is asked to clone is creator-led, phone-captured short-form UGC: spoken expression is-carried by one or more A-roll performances while B-roll, Caption and MG reshape what the viewer sees.-A-roll is defined by the performance establishing semantic time, regardless of its picture size,-placement or stack order.-Use this as a practical prior when evidence is incomplete, not as a definition of valid work. When the-reference or Brief establishes independently narrated, speechless, animated or another kind of work,-direct that form on its own terms.+## Give the work useful structure-For creator-led social video, favor compelling casting, a distinctive, appealing voice, and a-performance with a clear attitude toward the subject and listener. Outgoing or restrained, let that-relationship shape the voice, expression and timing, with a few decisive details. Give humor,-contrast or surprise an expression the audience can experience in the piece itself, and carry that-intention into the material requests.+Understand the piece as objects and relationships changing over time: what persists, what changes,+what draws attention, and what each change accomplishes. Express the relationships that should+survive a new product, wording or performance. Give those behaviors useful names and inputs; the+components can complete their local layout and motion. Hypit supplies shared authoring and execution+interfaces for both existing and project-defined components.++The work has one Timeline and a canvas. Placed Takes add semantic anchors where they belong;+components give the picture useful structure. Time can contain speech, gaps, overlaps or wholly+authored animation. Space can contain independent contributions or a coordinated scene with its own+internal tree. [System relationships](references/production/system.md) explains this adaptive+organization and connects it to materials and execution.-Prefer to author the video's meaning in Script and let the accepted performance give it time.+Choose component boundaries through shared behavior. An ordinary media presentation can use Media;+a playing video and a diagram exchanging space can share a scene that owns their layout and motion.+Independent Caption or overlays can remain peers. Use an existing component when its behavior fits;+create a project component when a new relationship deserves its own expression. A one-off scene is+normal production work. [Component design](references/production/component-design.md) helps turn the+visual idea into useful parts and author controls.++For spoken work, prefer to author meaning in Script and let the accepted performance give it time.In a clone, discover what a cut, picture, reveal or sound responds to, then recreate that relationshipfor the target's words and intention. Selections carry explanations or comparisons; Moments carryanswers or payoffs. This is Hypit's strong production prior: a new wording or delivery moves the@@ -52,6 +59,30 @@ presentation with its meaning. Preserve what each event responds to. In an authogive messages, reveals and state changes their own reading rhythm. Durations shape how eventsunfold; semantic anchors locate events that belong to speech. A piece can use both relationships.+A-roll is the performance supplying a spoken passage and its local timing. Its sound and semantic role can+continue while its picture fills the frame, moves into an inset, or gives way to a demonstration.+Choose what leads the picture from the current idea. Spatial grouping and timing are independent+choices, so a whole scene can respond to a Moment and separate components can share that Moment.++## Direct the material++Material direction carries practical knowledge of how generation models respond to prompts and+references. Use that experience to turn the creative answer into effective requests. Composition+uses your understanding of the work to organize the resulting material into meaningful behavior.++Keep the whole creative intention in view, then give each part the direction it can realize.+Translate planned composition into the generated material's appearance, framing and performance;+give later graphics their content, placement and events through components. High-level style and+attitude remain useful when they describe what that material should look, sound or perform like.+[Direction and its inputs](references/production/system.md#give-each-part-the-direction-it-can-realize)+explains this division.++For creator-led social video, favor compelling casting, a distinctive, appealing voice, and a+performance with a clear attitude toward the subject and listener. Outgoing or restrained, let that+relationship shape the voice, expression and timing, with a few decisive details. Give humor,+contrast or surprise an expression the audience can experience in the piece itself, and translate+the chosen idea into sensory and performable direction.+Before writing or adapting an image prompt, read[Image direction](references/playbooks/craft/image-direction.md) and apply it to the intended picture,including when starting from an existing project's prompt. UGC, podcast and street-interview images@@ -86,7 +117,7 @@ path and reconsider it when actual progress changes its usefulness. Reference incomponent work can progress alongsidethe environment and generation work they do not depend on.-## Understand and direct+## Understand and adaptUnderstand the whole reference and the designed behavior that makes it work. Follow its argument orstory from opening to close: what should the viewer feel, learn or decide, and how does the piece@@ -125,16 +156,8 @@ lets B-roll cover the explanation it supports. The composition is ready when itscarry the Treatment clearly and compellingly. Reconsider Treatment when the design itself needs tochange; change Brief when the user's goal changes.-Give useful relationships clear names and leave their expression to the component. Choose component-boundaries through shared behavior. Reuse Media for ordinary presentation; let a-video viewport and graphics share one component when they move or arrange together. Reuse Fine-Caption for flowing speech text, or author a Caption family for a new relationship among words and-graphics, carrying the same Script wording and semantic timing. Independent contributions can-remain peers. [Component design](references/production/component-design.md)-explains how semantic events make these compositions reusable.--Create project components as normal production work. Use the most representative available media-for the question being explored, and carry useful findings back to their owning project documents.+Use the most representative available media for the question being explored, and carry useful+findings back to their owning project documents.Environment, creation and production are rooms to revisit whenever the current question leads there.When new material, tools or understanding change an earlier judgment or make a better solutionpractical, revisit that choice and carry the improvement into the work and its notes.@@ -186,7 +209,8 @@ practical, revisit that choice and carry the improvement into the work and its n| When the question is about | Read || --- | --- |-| how Source, media, Tracks, execution and Results connect | `references/production/system.md` |+| how to express a work through materials, components and authored relationships, and how execution realizes it | `references/production/system.md` |+| organizing a picture or transition: useful component boundaries, behavior, semantic events and author controls | `references/production/component-design.md` || locating an existing `hypit` installation, or installing or updating the executable after Skill setup | `references/environment/distribution.md` || what this machine can do, credentials, Model/Provider/Endpoint choices or shared capacity | `references/environment/profile.md` || a new model, another service or Key for an existing model, or a project Model/Provider extension | `references/environment/model-and-provider.md` |@@ -202,10 +226,11 @@ practical, revisit that choice and carry the improvement into the work and its n| deciding who is A-roll, recurring voice identity, covered performance or independent narration | `references/playbooks/craft/voice-and-performance.md` || designing B-roll coverage, montage, short display windows or editorial handoffs | `references/playbooks/craft/b-roll.md` || grouping Chinese, English or mixed-script Caption, reading rhythm, styling or placement | `references/playbooks/craft/captions.md` |-| applying Caption Styles by speaker or passage, overriding a treatment, or hiding selected captions | `references/production/caption-program.md` |+| applying Caption Styles by speaker or passage, overriding a treatment, or hiding selected captions | `references/production/caption-presentation.md` || designing MG, a board, cards, graphic state, hierarchy, palette or reveals | `references/playbooks/craft/graphic-compositions.md` || deciding which generated images or videos should depend on which references | `references/playbooks/craft/generated-dependencies.md` || directing music, sound effects, ambience, gain, ducking or the completed mix | `references/playbooks/craft/sound-mix.md` |+| presenting existing Timeline sound, local gain, silence, fades or explicit source blending | `references/production/sound.md` || writing Sources, Recipes, and Runs, reusing produced work, adding a component | `references/production/authoring.md` || imports, output references, literal values or Recipe rules | `references/production/source-syntax.md` || choosing an existing Prompt Kit, assembling its wording or authoring a new one | `references/production/prompt-kits.md` |@@ -219,8 +244,9 @@ practical, revisit that choice and carry the improvement into the work and its n| choosing or finding fonts, using local font files, multilingual text, Emoji or Typography | `references/production/fonts-and-text.md` || which installed Surface to use, or whether to write a project component | `references/production/vocabulary.md` || sharing a component, Prompt Kit, Model, or Provider across projects | `references/production/component-sharing.md` |-| the A-roll semantic timeline and composing Media, Audio, Caption, Text, MG and Effect Tracks | `references/production/tracks.md` |-| designing a good component: its visual idea, semantic behavior, useful controls and fit in the work | `references/production/component-design.md` |+| placing Takes, gaps, overlap, complete duration or a pure MG work on one Timeline | `references/production/timeline.md` |+| composing Performance, Media, Audio, Caption, Text, MG and Effect Tracks | `references/production/tracks.md` |+| presenting existing Timeline footage with broad or local Uses, a moving viewport or a custom Performance Style | `references/production/performance.md` || writing a project Track with new layout, semantic events or persistent state | `references/production/track-authoring.md` || drawing a component's elements, animation, resources or prepared surfaces | `references/production/component-visuals.md` || writing a Caption family with new word relationships, scheduling or layout | `references/production/caption-authoring.md` |diff --git a/skills/hypit/references/creation/project-files.md b/skills/hypit/references/creation/project-files.mdindex 551fa6c9..129082a7 100644--- a/skills/hypit/references/creation/project-files.md+++ b/skills/hypit/references/creation/project-files.md@@ -136,7 +136,8 @@ referenced in place; its documentary role does not require a duplicate.- `TIMELINE.md` connects what happens when with what it does for the viewer. Preserve the content,placement, entry, changes, persistence and exit of the reference's visual systems, their word oraction relationships, source times and useful evidence paths. Concurrent picture, speech, Caption,- Typography, MG, Effect and Audio behavior belong to the same connected account.+ Typography, MG, Effect and Audio behavior belong to the same connected account. These are reference+ timeline notes; the target's actual placements belong to the `time:Timeline` declaration in Source.- `transcript.json` is word-level speech evidence. It is evidence, not the director's interpretation.- `evidence/` contains only media worth reopening, with ordinary human-readable names.- `BRIEF.md` preserves the user's goal, facts, constraints, requested changes anddiff --git a/skills/hypit/references/creation/reference-video.md b/skills/hypit/references/creation/reference-video.mdindex ab96f12b..23290c27 100644--- a/skills/hypit/references/creation/reference-video.md+++ b/skills/hypit/references/creation/reference-video.md@@ -40,6 +40,16 @@ replace an item, add to a collection, or temporarily cover it. Document the systits changes in time. Camera boundaries help navigation; meaning and continuity determine whichthings belong together.+Build an account of behavior that can become an editable production. A fixed screen image may be+the evidence the passage needs; a changing comparison also needs its arrivals, states and handoffs.+Retain what the objects do together as well as what each looks like. Several implementations can+express the observed behavior, so the account need not guess the original author's source code or+component structure.+The useful result is an explanatory model for this commission: which relationships should remain+when its person, product or wording changes, and which choices should be redesigned. Record the+observed behavior before choosing how the target will implement it. A picture, text and motion that+work together may become one component; repeated appearances may be states of that same object.+## Make time visibleUse `hypit transcribe` when understanding a spoken reference. Its wording and word-level times give@@ -129,7 +139,7 @@ distant moments relate. Keep the reference's facts and your interpretation distiordinary prose.`TIMELINE.md` carries time-locatable realization. Organize it into sections named by source-media-time and a meaningful phase. Within each, connect the active layers and their detailed behavior to+time and a meaningful phase. Within each, connect the active objects and their detailed behavior tothe words or actions they serve. An account should let someone find the event, understand itsexpression and implement an appropriate counterpart. For example:@@ -171,6 +181,12 @@ For generated camera imagery, [image direction](../playbooks/craft/image-directiturns the observed appearance into a coherent styling and scene direction, with the decisive detailsthe target needs. The reference notes retain the observations behind those choices.+For the designed picture, use [component design](../production/component-design.md) to turn the+target's relationships into useful objects, parameters and events. A system can span several shots;+one shot can contain several independent systems. Choose the new component boundaries from shared+behavior, with the details required to realize it. [Direction and its inputs](../production/system.md#give-each-part-the-direction-it-can-realize)+connects the whole creative design to its separate material and composition instructions.+The recorded account should explain the whole piece from opening to close and make its distinctvisual systems and their changes locatable. It should be concrete enough to direct the new work:what to preserve or adapt, how it is expressed, and why it belongs. Continue investigating andiff --git a/skills/hypit/references/creation/script-and-time.md b/skills/hypit/references/creation/script-and-time.mdindex c61bcc97..54cdb339 100644--- a/skills/hypit/references/creation/script-and-time.md+++ b/skills/hypit/references/creation/script-and-time.md@@ -8,6 +8,8 @@ and edge cases.[Source syntax](../production/source-syntax.md) covers the surrounding imports, references, Recipesand Runs; [Tracks](../production/tracks.md) covers the consumers of Script meaning.+[Timeline authoring](../production/timeline.md) places Takes, allows gaps and overlaps, and declares+the complete work, including pure MG with no Script.[Media preparation](../production/media.md) explains connecting actual footage to a Segment, and[Runs](../production/runs.md) explains targeting material and reusing produced Takes.@@ -45,7 +47,8 @@ Alignment Unit. `||` ends one on-screen Caption Cue after that unit and lets theA Cue is a timed block of displayed speech, not a line of text. Its Caption family and Recipe maywrap that block over one or more lines, reveal or highlight its words, and give the block an entrance,-exit or handoff. Segment boundaries, Role turns and Caption Style changes already separate Cues.+exit or handoff. Segment boundaries and Role turns already separate Cues. Caption Use windows can change a+Style midway through a Cue while preserving its content and original word times.Use `||` when the same Segment, turn and Style still needs another deliberate reading handoff. The[Caption craft](../playbooks/craft/captions.md#design-cue-rhythm-with-the-caption-system) owns how thepicture, language, family and Recipe shape that decision.@@ -65,8 +68,8 @@ The author-facing forms are:| `@proof ... @/proof` | A Selection: one named semantic range. || `@claim!` | A Moment: one named semantic point. |-A Script contains one or more uniquely named lower-case Segments. A paired Segment carries prose;-the self-closing form carries a wordless passage. A Role Cue is a bare turn marker rather than a+A Script contains one or more uniquely named lower-case Segments. A paired Segment can carry prose+or be wordless; the self-closing form also carries a wordless passage. A Role Cue is a bare turn marker rather than apaired element: the next Role Cue begins the next turn, and closing the Segment ends the final turnand resets its Role. When a Segment uses Roles, place the first Role before that Segment's firstspoken text. One Segment can contain several Role turns without requiring several generated Takes.@@ -184,8 +187,10 @@ Script also represents passages without speech:`empty` is an ordinary Segment name; a name such as `product-detail` can express the passage's role.No words does not mean no semantics: the Segment retains its identity and start/end anchors. Itsassociated normalized media determines the duration, and its SemanticTake has an empty word array.-The same SemanticTrack and Track timing vocabulary apply to a wordless passage or an entire+The same Timeline and Track timing vocabulary apply to a wordless passage or an entirepiece made from prepared media. The empty tag itself declares neither a zero-length interval nor a duration.+For an interval made only of component animation, use Timeline placement and extent instead; it+needs no media-backed Segment. Spoken, wordless-media and graphics-only passages can share one work.Choose Segment boundaries from natural production passages and delivery length, not from everypicture cut. One Segment and Take can carry several speaking turns, camera cuts or a split-screen@@ -204,7 +209,7 @@ A chat animation, diagram or kinetic-text piece can instead be drawn entirely bymessages and changes still carry meaning; the author chooses when the audience receives them andhow long they need to read. Keep content and event timing together in the owning component's Source.An event can have an identity such as `question` or `reveal` and an authored `at="2.6s"` without-inventing spoken words or a media-backed Segment. Film time is declared through ProgramSpace;+inventing spoken words or a media-backed Segment. Film time is declared through a Timeline with an explicit end and zero Takes;[composition and rendering](../production/rendering.md#compose-an-authored-animation) shows the form.Choose timing per relationship, not once for the whole video. A spoken Moment can introduce a chat@@ -252,7 +257,7 @@ applies this judgment to the selected model's request range.Measurement balances the intended speaking density and sizes generation; it supplies no timelineanchors. Once a Take is accepted, normalize it, align its actual speech to its Script Segment, and-assemble the resulting SemanticTakes into the SemanticTrack. The literal duration answers how much+assemble the resulting SemanticTakes into the Timeline. The literal duration answers how muchmedia to request. The aligned words answer where Caption, B-roll, MG, and Effects belong in that actualmedia. Alignment measures real word positions inside that media envelope; it does not reproduce anestimated distribution of words.@@ -263,7 +268,7 @@ Speech-rate measurement has no role there; the resulting media still determines## Bind meaning to Script identitiesFor a picture, Caption treatment, MG state, sound, or effect that belongs to spoken meaning, author a-Selection or Moment and use the consuming component's Surface to project it through the SemanticTrack.+Selection or Moment and use the consuming component's Surface to project it through the Timeline.Use explicit seconds for genuinely clock-based or speechless design.Selections may overlap, cross, or span Segments; they are named semantic ranges rather than nested@@ -296,9 +301,9 @@ B-roll windows that should also own the pause between words, choose which neighbowns that gap through the explicit affinities; [B-roll craft](../playbooks/craft/b-roll.md) shows theshared-boundary forms.-The consuming component's Surface projects the authored identity through the real SemanticTrack.+The consuming component's Surface projects the authored identity through the real Timeline.For example, `during={story.selection.proof}` on a visual Item makes that Surface construct a Window.-A persistent MG reveal can use `at={story.moment.claim}` to consume an Instant; an Audio Clip uses+A persistent MG reveal can use `at={story.moment.claim}` to consume an Instant; an Audio Item uses`at={story.moment.claim} for="600ms"` to occupy a Window. The component's Fragment and Producersconsume that value and own playback, visible duration, animation, and state behavior; Script suppliesthe meaning and its Anchors.@@ -343,7 +348,7 @@ behavior.Reference archives keep original seconds and explain which original words or content events an itemserves. The target Source names the intended relation against the target Script. After the target's-actual audio is aligned, the SemanticTrack supplies its frames. Do not copy a reference timestamp into+actual audio is aligned, the Timeline supplies its frames. Do not copy a reference timestamp intothe target or preserve an incidental lead/lag unless that offset itself is part of the design.## Connect reference understanding to audiovisual compositiondiff --git a/skills/hypit/references/playbooks/craft/b-roll.md b/skills/hypit/references/playbooks/craft/b-roll.mdindex af9cdee3..f4946501 100644--- a/skills/hypit/references/playbooks/craft/b-roll.md+++ b/skills/hypit/references/playbooks/craft/b-roll.md@@ -69,7 +69,7 @@ the relationship, not a required montage duration or a fixed amount of extra rooA montage can stay up after its subject finishes speaking. A partner can start responding while thelast lifestyle picture is still visible, with their camera returning later. This incoming voicebefore its picture forms a J-cut relationship and gives the montage room.-The B-roll does not need to contain that speech: the speaking Track's audio already continues beneath.+The B-roll does not need to contain that speech: the Sound output included in Film continues beneath.Choose the endpoint by the thought, reaction and visual reading time. Extending coverage across aRole turn is legal; a Selection can also cross a Segment boundary. Avoid cutting the final scene@@ -114,6 +114,8 @@ Keep the decisive action visible and let small changes finish; camera movement aserve that sensation. Synchronized generated sound, recorded sound or separately authored sound caneach be appropriate. [Sound and mix](sound-mix.md) covers their relationship.-When a work is genuinely speech-free, its visual, musical and action relationships still need-authored semantic boundaries. [Media preparation](../../production/media.md#empty-segments-use-their-media-boundaries)-owns how actual media supplies them.+For speech-free work, author timing from the visual, musical and action relationships. Independent+Media Items can occupy chosen Windows. A named wordless Segment is useful when a real media passage+should supply reusable start/end anchors; [media preparation](../../production/media.md#empty-segments-use-their-media-boundaries)+explains that option. [Timeline](../../production/timeline.md) also supports graphics-only intervals+and complete animation without media-backed Segments.diff --git a/skills/hypit/references/playbooks/craft/caption-tracking.md b/skills/hypit/references/playbooks/craft/caption-tracking.mdindex 23e1083f..db3a3989 100644--- a/skills/hypit/references/playbooks/craft/caption-tracking.md+++ b/skills/hypit/references/playbooks/craft/caption-tracking.md@@ -41,9 +41,9 @@ the first or largest face alone can attach the guest's Caption to the intervieweTransform the useful observations into ordinary project data, with these decisions explicit:-- **Clock:** resample onto the actual ProgramSpace frame rate. For concatenated normalized Takes,- the local-to-global offset is the sum of preceding Takes' actual frame counts, not their requested- generation durations. Final-video measurements are already on the global timeline.+- **Clock:** resample onto the Timeline's frame rate. A Take-local observation becomes global by+ adding that Take's actual placement start, including any authored gap or overlap. Sequential+ placement uses preceding prepared durations. Final-video measurements already use the global clock.- **Geometry:** convert detector edges to `[x, y, width, height]` using width `right - left` and height`bottom - top`. If measurement used another resolution, crop, inset or split-screen panel, map thebox through that actual placement before normalizing it to the final Canvas.@@ -83,7 +83,9 @@ these transformations reproducible when needed, taking explicit media, clock and```svml<space:RegionTimeline id="heads" within={vertical} recipe={tracking.heads.default}/><caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program} regions={heads}/>+ timeline={speech.timeline} regions={heads}>+ <caption-fine:Use style={caption-style}/>+</caption-fine:Track>```The RegionTimeline's ids match Script Roles. Fine places a single-Role Cue at that region's topdiff --git a/skills/hypit/references/playbooks/craft/captions.md b/skills/hypit/references/playbooks/craft/captions.mdindex a9dfa9b2..8a1185f6 100644--- a/skills/hypit/references/playbooks/craft/captions.md+++ b/skills/hypit/references/playbooks/craft/captions.md@@ -30,7 +30,7 @@ energy inform that judgment rather than prescribing a fixed Cue length or one ca## Caption families, Styles and parametersScript publishes one `CaptionDocument`: ordered display words, complete Alignment Units, speaking-Roles, word attributes and authored Cue handoffs. Caption joins it to the real SemanticTrack. That+Roles, word attributes and authored Cue handoffs. Caption joins it to the real Timeline. Thatcommon typed relationship is what makes the result Caption; an official renderer is not thedefinition. A project package can provide another Caption family while keeping the same Script wordsand measured speech evidence.@@ -49,14 +49,14 @@ project `.svs`; a set that has earned reuse across works can become an ordinaryunder its owner's scope. Such a collection preserves the family, Recipe and intended use togetherrather than turning isolated parameter values into universal defaults.-Caption Program chooses where those Styles apply: a default, a speaking Role, a semantic passage or-a local emphasis. [Caption styling and coverage](../../production/caption-program.md) explains-overrides, Segment-wide selections, word attributes and hiding selected captions with Mute.+Caption Track Uses choose when those Styles apply, optionally filtered by speaker. Cue content+stays complete when presentation changes midway through a phrase. [Caption styling and coverage](../../production/caption-presentation.md) explains+overrides, Segment-wide selections, word attributes and hiding selected captions with a Hidden Style.## Keep wording and timing in their ownersScript owns display words, speaking Roles, Dual Text units, attributes and `||` Cue Breaks. Its-CaptionDocument contains no seconds or frames. Caption joins that truth to the actual SemanticTrack;+CaptionDocument contains no seconds or frames. Caption joins that truth to the actual Timeline;a visual family then presents the timed units. Reuse those units rather than retyping spoken wordsinto independent Typography merely because it can draw the desired shape.@@ -72,12 +72,12 @@ explicit Style Recipes. Role overrides can distinguish podcast hosts; a trackedmoving placement point without changing the verbal pipeline.```svml-<caption:Program id="caption-program" document={story.caption} narrative={story}- default={primary-caption}>- <caption:Use role="GUEST" style={guest-caption}/>-</caption:Program>+<caption-fine:Track id="captions" document={story.caption}- semantic={speech.semantic} program={caption-program}/>+ timeline={speech.timeline}>+ <caption-fine:Use style={primary-caption}/>+ <caption-fine:Use role="GUEST" style={guest-caption}/>+ </caption-fine:Track>```This excerpt assumes the imported Surfaces, fonts and Styles already declared in the Source.diff --git a/skills/hypit/references/playbooks/craft/compositing.md b/skills/hypit/references/playbooks/craft/compositing.mdindex da4e10fd..d185732d 100644--- a/skills/hypit/references/playbooks/craft/compositing.md+++ b/skills/hypit/references/playbooks/craft/compositing.md@@ -78,7 +78,7 @@ evidence, and the feeling of the shot rather than from a fixed layout recipe.| Full-frame | A `9:16` character-and-scene image and reference-generated performance are the common fit for vertical creator footage. Frame the face, body and real setting for the later composite. || Upper or lower split | A square camera image and performance often cover a `9:8` half of a vertical Canvas cleanly, with a small top/bottom crop. A close, intimate phone view may instead be a vertically generated performance whose other half is covered. || Foreground cutout | A vertical performance often preserves a natural amount of the speaking body. Remove its background, normalize the alpha-bearing result, and use that same prepared performance as the SemanticTake. |-| Circular picture-in-picture | A square source with the face near its own center is a useful starting point. Present the semantic performance through Media Performance with a square Frame and rounded clip whose radius is half the side. |+| Circular picture-in-picture | A square source with the face near its own center is a useful starting point. Present the semantic performance through Performance with a square Frame and rounded clip whose radius is half the side. || Audio-only A-roll | The performance contributes semantic time and sound but no picture. Another Track fills the Segment with the intended demonstration, POV, B-roll, Typography or MG. |For a non-overlapping upper/lower split, equal stacking order can express two peer regions; the@@ -118,7 +118,7 @@ chroma-backed camera image→ moving-video matting→ alpha-preserving normalization→ SemanticTake- → Speech Track cutout presentation+ → Timeline placement → Performance or project-scene presentation```The chroma backdrop establishes subject separation rather than the final story setting. The final@@ -128,11 +128,11 @@ identity, framing and useful performance state; the matting section below owns t## Inset, cutout and flattened composite are distinct choices-A speaking rectangular inset, circular inset or normalized cutout can be presented directly by-Speech Track. A circular inset is a geometric crop: use a square Frame and a rounded clip whose+A speaking rectangular inset, circular inset or normalized cutout can be presented by+Performance or a project scene. A circular inset is a geometric crop: use a square Frame and a rounded clip whoseradius is half that square's side. The camera background remains inside the circle. A cutout instead-follows the person's silhouette and needs suitable transparency in the source. Speech Track presents-the Take while it carries the Segment's speaking performance. Media Track can use the same prepared+follows the person's silhouette and needs suitable transparency in the source. Timeline assembly retains+the Take's positions; its visual component presents the prepared picture. Media Track can use the same preparedmedia when the work intentionally gives it an independent Window, playback, trim or visual role.A speaking person in either presentation remains A-roll when their performance carries the main@@ -164,8 +164,8 @@ require another kind of Take or Track. Choose which role the cutout serves in th| Role | Consume the prepared output || --- | --- |-| A speaking performance establishing semantic time | `cutout-media.media` → SemanticTake → Speech Track; show its prepared picture through Media or a project component and include `speech.audio` when wanted. |-| A visual overlay on an existing semantic timeline | `cutout-media.media` → a Media Item's `media` input; select its Window and include the Track's visual output in Film. |+| A speaking performance with local semantic timing | `cutout-media.media` → SemanticTake → Timeline assembly; show its placed picture through Performance or a project component and present its placed audio through Sound. |+| An independent visual overlay | `cutout-media.media` → a Media Item's `media` input; select its Window on the Timeline and include the Track's visual output in Film. |When the intended artifact is a fixed flattened arrangement of still images, use[Image Compose](../../production/image-operations.md#flatten-a-fixed-still-image-arrangement).diff --git a/skills/hypit/references/playbooks/craft/image-direction.md b/skills/hypit/references/playbooks/craft/image-direction.mdindex e6029eb7..8d0da5de 100644--- a/skills/hypit/references/playbooks/craft/image-direction.md+++ b/skills/hypit/references/playbooks/craft/image-direction.md@@ -46,6 +46,24 @@ sleeve may be incidental. If the sleeve exposes a featured watch, preserve thatthe details the new work depends on, and let its new person and camera realize the shared style.More painstaking description does not by itself make the direction more faithful or attractive.+## Describe the picture being made++Translate the complete design into what this image should contain. A later comparison board may+motivate placing a person on the right. In the image prompt, describe that placement and a natural+scene relationship, such as a table entering the lower-left corner. The board's own content and+animation belong to its component. The image receives the visible consequence of the design.++High-level anchors still direct the picture: soft goth specifies styling, idol-like beauty specifies+the kind and strength of appeal, and a Korean street-side café specifies a recognizable setting.+Keep that compression. For an action or interaction, choose what actually appears. If water should+be poured, direct that action; if only a gesture is intended, describe the hand gesture itself.+Resolve figurative or uncertain wording when taking it literally would add the wrong prop or event.++The requested output determines what belongs. A photographed sign or a generated graphic can include+its intended lettering and symbols. A photographic source for a later composite receives its own+person, scene and camera view. [Direction and its inputs](../../production/system.md#give-each-part-the-direction-it-can-realize)+explains how these local instructions retain the whole design.+## Four paragraphs, four responsibilitiesFor a complete character-and-scene camera image, use this order:diff --git a/skills/hypit/references/playbooks/craft/sound-mix.md b/skills/hypit/references/playbooks/craft/sound-mix.mdindex 5d8eb973..72854889 100644--- a/skills/hypit/references/playbooks/craft/sound-mix.md+++ b/skills/hypit/references/playbooks/craft/sound-mix.md@@ -55,7 +55,7 @@ every available effect to every cut quickly exhausts the listener. A tactile prodepend on delicate contact and material texture. Preserve those small sounds and match their onsetto the visible action rather than burying them under a generic impact.-An independent sound can use an Audio Clip at a Script Moment with a deliberate duration. A sound+An independent sound can use an Audio Item at a Script Moment with a deliberate duration. A soundintrinsic to a Media Item or Sequence transition can use that component's enter, exit or Handofftrigger. The component owns that event relationship. [Tracks](../../production/tracks.md) gives aClip example; `@hypit/audio-track` and `@hypit/media-track` own exact playback and trigger syntax.@@ -70,3 +70,8 @@ an audible transition even when the picture changes gently.Give speech its complete phrase and let effect tails finish naturally. Use fades and overlaps tocarry the intended handoff. Keep the mix coherent across the piece: clear speech, a consistent soundpalette, and emphasis reserved for the moments that deserve it.++For an existing performed passage, [Sound](../../production/sound.md) supplies local Uses for gain,+silence and source handoffs while retaining its playback position. A crossfade Style names the two+sources it blends; the selected Window controls its duration. Independent music and effects keep+their own Audio Tracks.diff --git a/skills/hypit/references/playbooks/craft/video-direction.md b/skills/hypit/references/playbooks/craft/video-direction.mdindex fc0db715..7682854e 100644--- a/skills/hypit/references/playbooks/craft/video-direction.md+++ b/skills/hypit/references/playbooks/craft/video-direction.md@@ -80,8 +80,10 @@ owns how the premise becomes a viewer experience; action directs the person's paThe prompt describes what this generation should make visible over time. Give a performer physicalrelationships with the camera, people, props and parts of the setting that actually exist in the-generated scene. A Caption, icon, product card or other MG composed later can influence framing, but-it is not an object for the performer to touch, watch or reveal inside the generated video.+generated scene. Translate a later graphic's framing needs into the camera view, gaze or gesture+the video should perform. Give the later Caption, icon or product card its own content and events+through composition. A prop or display intended to exist within the generated scene can be described+as that actual object and connected through the relevant references.Capable video models can turn figurative wording into literal objects, events or transformations. Useconcrete visible language for intended gaze, gesture, movement, camera behavior and cuts when a@@ -89,6 +91,12 @@ metaphor would introduce the wrong scene content. Social attitude and aestheticuseful when they direct performance; an imagined object or event belongs in the prompt when it shouldtruly appear in the generated world.+Give the performer an attitude they can express, such as questioning a claim with skeptical interest.+The broader purpose of persuading an audience belongs in Treatment; translate it into this person's+delivery and interaction. Decide whether a described action really happens or whether a gesture is+intended, then state that choice. [Direction and its inputs](../../production/system.md#give-each-part-the-direction-it-can-realize)+connects those performance choices to the whole composition.+Natural emphatic gestures are usually more reliable than asking fingers to display an exact number.Let speech, Caption or MG convey the quantity while pointing and hand actions serve the performance.An encounter also has edges: someone interrupted can first be occupied, and someone finishing candiff --git a/skills/hypit/references/playbooks/craft/voice-and-performance.md b/skills/hypit/references/playbooks/craft/voice-and-performance.mdindex 71d72d61..9dc866c2 100644--- a/skills/hypit/references/playbooks/craft/voice-and-performance.md+++ b/skills/hypit/references/playbooks/craft/voice-and-performance.md@@ -11,9 +11,10 @@ seconds. In Hypit, A-roll names the performance that makes a spoken Segment timea visible A-roll whose picture and sound are performed together, or an audio-only A-roll whoseindependent speech carries the passage while other Tracks supply the picture.-The A-roll does not fit itself onto an earlier program timeline. By performing the Segment, it gives-that passage real time; the other Segment performances together create the program's semantic time.-[Tracks](../../production/tracks.md#semantic-takes-supply-the-media-timeline) owns the exact+Performing the Segment gives that passage its duration and local word times. Timeline assembly+places it within the complete work, ordinarily after the previous Take, or at an authored position.+The work can also contain graphics-only passages before, between or after those performances.+[Tracks](../../production/tracks.md#place-prepared-takes-on-the-timeline) owns the exactSemanticTake assembly and projection mechanism.This gives a practical question: **who is speaking this Segment?** For visible A-roll, that person's@@ -27,17 +28,16 @@ own their own lines even while another picture covers them. Visual handoffs, covand Effects are authored against the time that performance established. If an edit changes theperformance's actual length, prepare and align the edited media before assembly.-A wordless Segment is the important edge of the same model. Nobody speaks, so it has no speaking-A-roll, but it still has authored meaning. Its prepared media supplies the Segment's start and end,-and its SemanticTake carries an empty word array into the same SemanticTrack. Here semantic means the-authored identity and relationships of the passage, not a requirement that it contain speech.+A wordless Segment can identify a real performed passage, such as a dance or reaction. Its prepared+media supplies start/end boundaries without spoken words. A graphics-only interval can instead+occupy the Timeline directly, with no Take. Choose from the content the passage actually uses.## Keep the performance role separate from the pictureA-roll is not the bottom layer, the largest rectangle, or the currently visible face. ItsSemanticTake retains the role when its picture is full-frame, one half of a split, a circular inset,a moving cutout, or completely covered by B-roll or MG. A large app recording can be the primary-picture while a small presenter in a corner supplies the semantic timeline and speech.+picture while a small presenter in a corner supplies the passage's words and timing.The covering B-roll may even show the same person doing a silent lifestyle action. Seeing a personwhile hearing words does not by itself make that picture the speaking Take or create a new speech@@ -61,11 +61,11 @@ the listening rhythm. For ordinary creator speech, podcast turns, interviews andmakes a direct join between the prepared Segment Takes a practical starting point. Visual coverageand effects can still cross that seam without changing the underlying speech time.-This is a directing judgment, not a ban on another temporal design. Deliberate interruption,-overlapping dialogue, musical phrasing or another work may call for a different assembly. A project-component can own that operation and publish the explicit semantic, visual and audio outputs its-consumers need. [Track authoring](../../production/track-authoring.md#author-an-alternative-performance-assembly)-explains the type boundary; the default Speech Track remains the simple lossless assembly.+Deliberate interruption, overlapping dialogue or musical phrasing can use the Timeline's explicit+Take positions. [Timeline authoring](../../production/timeline.md) supports gaps and overlaps directly.+The sources retain their local timing; a crossfade or other coordinated picture treatment belongs+to the visual component owning that behavior. Sequential placement remains the concise ordinary+choice for creator speech.## Give a recurring person one accepted voicediff --git a/skills/hypit/references/playbooks/craft/voice-direction.md b/skills/hypit/references/playbooks/craft/voice-direction.mdindex 6774d0dc..6b04503c 100644--- a/skills/hypit/references/playbooks/craft/voice-direction.md+++ b/skills/hypit/references/playbooks/craft/voice-direction.md@@ -35,6 +35,12 @@ presentation, language or accent, pitch, texture and articulation can sharpen thChoose the few that settle the casting; this is not a field list to fill out. Cultural or charactershorthand earns its place through the sound it evokes.+Translate the character's role in the piece into audible qualities. Wanting the viewer to trust a+claim can inform clear articulation and thoughtful emphasis; the request should carry that chosen+sound and manner of speaking. Music, graphic reveals and the later mix receive their own direction.+[Direction and its inputs](../../production/system.md#give-each-part-the-direction-it-can-realize)+explains this relationship between the whole design and the individual request.+For example:```textdiff --git a/skills/hypit/references/playbooks/index.md b/skills/hypit/references/playbooks/index.mdindex 762106a8..bce093c0 100644--- a/skills/hypit/references/playbooks/index.md+++ b/skills/hypit/references/playbooks/index.md@@ -24,7 +24,7 @@ not need a matching playbook before it can be understood or made.| --- | --- || [Directing generated images](craft/image-direction.md) | phone-video capture, compelling people, camera relationships and settings; four-paragraph image direction and reference use || [Directing a voice](craft/voice-direction.md) | casting an appealing voice, choosing its vocal character, writing Voice Design direction or selecting a sample line |-| [Voice and performance](craft/voice-and-performance.md) | how Segment performances create semantic time, who is A-roll, recurring voice identity, visual A-roll presentations, or genuinely independent narration |+| [Voice and performance](craft/voice-and-performance.md) | how performances supply local word timing, who is A-roll, recurring voice identity, visual A-roll presentations, or genuinely independent narration || [Directing generated video](craft/video-direction.md) | choosing a generation relationship, directing visible performance or silent action, camera behavior, cuts, or request duration || [Reference relationships](craft/generated-dependencies.md) | deciding which images or Takes should depend on which references || [Visual continuity](craft/visual-continuity.md) | complementary camera views, gaze, prop state, or what must remain coherent through a cut |diff --git a/skills/hypit/references/production/authoring.md b/skills/hypit/references/production/authoring.mdindex e8ff0dbd..078313ab 100644--- a/skills/hypit/references/production/authoring.md+++ b/skills/hypit/references/production/authoring.md@@ -16,8 +16,8 @@ and [Caption authoring](caption-authoring.md) explain creating a new project comBegin with the work the Treatment describes, not with a package inventory. Identify:-- the Script passages and speaking roles;-- the A-roll performances that give those passages semantic time, and the B-roll that supports them;+- the Script passages and speaking roles, where the work has speech;+- the performances and other material the work needs, and their placement on its complete Timeline;- the Caption, Typography, MG, Effect, and Audio systems that act with them;- the semantic relations that should follow words, phrases, pauses, or content events;- the genuinely clock-based events;@@ -36,8 +36,9 @@ other layers will follow,These pages establish what the authored relationship should accomplish. Installed vocabulary thensupplies the exact language for expressing it.-Author those relationships explicitly. Script Selections and Moments carry meaning through produced-speech into real time. Track and Film elements arrange the resulting picture and sound. One Source+Author those relationships explicitly. Script Selections and Moments carry meaning through placed+performances into real time; authored positions locate independently timed events on the same Timeline.+Track and Film elements arrange the resulting picture and sound. One Sourcecarries the whole creative program; its cuts are relationships inside that program.When a role first leads to an installed Surface in the current work, query its actual declarationdiff --git a/skills/hypit/references/production/caption-authoring.md b/skills/hypit/references/production/caption-authoring.mdindex d8d546f9..cce9d200 100644--- a/skills/hypit/references/production/caption-authoring.md+++ b/skills/hypit/references/production/caption-authoring.md@@ -2,7 +2,7 @@Read this when a work needs a new relationship among its spoken words, layout and motion.[Caption craft](../playbooks/craft/captions.md) owns readability and directing judgment;-[Caption styling and coverage](caption-program.md) owns applying Styles and Mutes;+[Caption styling and coverage](caption-presentation.md) owns applying visible and hidden Styles;[Track authoring](track-authoring.md) owns project package wiring.[Fonts and text](fonts-and-text.md) explains exact font resources and fallbacks;[component visuals](component-visuals.md) explains the final drawing representation.@@ -45,40 +45,39 @@ and evaluate its state at the requested frame. Keep separately useful overlays a## Keep the existing text and timing chain```text-Script → CaptionDocument (display words, alignment units, authored Cue breaks)-Styles + Role / Selection / word-attribute applications → CaptionProgram-CaptionDocument + CaptionProgram + SemanticTrack → TimedCaptionProjection-projection + family parameters → explicit family schedule-schedule + document + program + ProgramSpace → VisualTrack+Script → CaptionDocument (displayed words, Cue breaks and word attributes)+CaptionDocument + Timeline → complete timed Cues and original word times+Track Uses → resolved time windows, Styles and optional speaker filters+complete Cues + Uses + family parameters → family schedule → VisualTrack```-Use `@hypit/hypit/caption` for the common Program and timing helpers and `@hypit/hypit/narrative` for the Script's-document and unit types. The common Caption layer handles display-versus-spoken-wording, N:M alignment units and complete-unit Selection boundaries. Do not transcribe again,-rebuild timing by splitting a string, or use the video model's requested duration as speech evidence.-The package README explains the extension inputs and points to their owning implementations.--The ordinary Program Surface remains useful with a new family's Styles:+Use `@hypit/hypit/caption` for content timing and Use coverage, and `@hypit/hypit/narrative` for+Script document types. A family Track accepts `document`, `timeline` and ordered `Use` children:```svml-<caption:Program id="captions" document={story.caption} narrative={story}- default={base-style}>- <caption:Use role="GUEST" style={guest-style}/>- <caption:Use selection={story.selection.punchline} style={punchline-style}/>-</caption:Program>+<keyword:Track id="captions" document={story.caption} timeline={film.timeline}>+ <keyword:Use style={base-style}/>+ <keyword:Use role="GUEST" style={guest-style}/>+ <keyword:Use during={story.selection.punchline} style={punchline-style}/>+</keyword:Track>```-These names assume Styles have already been declared by the chosen family. They are complete Style-applications, not CSS inheritance. For word-specific roles, Script can mark `useful{emphasis}` and-`caption:Use attribute="emphasis"` can select a Style. The new family must deliberately interpret-the resulting word runs; Fine rejects them. A structural word role should not be guessed by the-renderer from spelling, capitalization or “the third word.”+These names assume the project family and its Styles have been declared. Reuse the common time+projection helpers for `during`, `at`/`for`, `until`/`for` and `start`/`end`. A Use has the same+meaning regardless of the family. `role` filters whose content it presents within that window.++For a keyword layout, Script can mark `useful{emphasis}`. The family reads that attribute from the+CaptionDocument and gives it a visual role within the complete Cue. The word role and the time+window answer different questions; an emphasis attribute does not invent another time language.## Design a family-specific scheduleSeparate measured speech time from visible time. Preserve each alignment unit's measured boundariesand identity; resolve lead, tail, stagger, hold and handoff into an explicit schedule before rendering.-The renderer draws that schedule and does not silently add another tail or reorder Cues.+The renderer draws that schedule. Preserve a Cue's original animation origin and apply the+winning Use window as a separate visibility mask. A window starting midway through a Cue still+receives the complete Cue and original word times. It can change appearance without restarting+karaoke or text reveal.For a “large keyword plus supporting phrase” family, the schedule might identify each Cue's displaywords, its emphasized word ids, their measured units and the visible interval of the two groups.@@ -103,29 +102,30 @@ effect from following spoken characters. Keep within-glyph wiping an explicit viUse uneven unit durations and a pause in a short example to check that the chosen effect followsthe intended clock.-`caption:Mute` suppresses selected display units while preserving speech. A new family should use-the common mute application when scheduling, as Fine does. Validate that schedule, document, narrative-and ProgramSpace belong together; do not silently accept timing from another video.+`caption:Hidden` supplies a Style with `rendering: null`. It takes part in later-Use precedence,+clearing this Track's presentation in its window; a later visible Use can restore a smaller window.+Use `captionUseVisibility` to resolve that coverage for each Cue's speaker. The content projection+retains all Cues regardless of Style. Empty or uncovered time naturally produces no drawing.## Give Style, layout and rendering clear ownersThe Style Surface validates a Recipe and exact font references, then emits the common Caption Style-shape with the new family's name and parameters. The Track Surface accepts the document, semantic-timeline and Caption Program; its Fragment performs the common timing join and its own schedule and-render operations. Register the new family's Producers and any new schedule Type in its own package.+shape with the new family's name and parameters. The Track Surface accepts the document,+Timeline and timed Use children; its Fragment assembles the Uses, performs the common timing join,+and runs its own schedule and render operations. Register the new family's Producers and any new schedule Type in its own package.The renderer owns typography, structural relationships, stacking and motion. Use the existing textshaping and Visual IR facilities with explicit font resources, including selected local font files,so the same faces reach the rendering machine.-Read `@hypit/caption-fine`'s `fragment.ts`, `schedule.ts` and `render.ts` as separate implementation+Read `@hypit/caption-fine`'s `surface.ts`, `schedule.ts` and `render.ts` as separate implementationexamples. Reuse the common parts and replace the actual family behavior, including its parametervalidation; renaming Fine while retaining its uniform-word assumption will not implement a structuralkeyword treatment.-The common Program does not automatically dispatch Styles to different renderers. Fine rejects a-foreign family. A new family can provide its own ordinary and emphasized treatments in one Program;-if the work combines separate Caption Tracks, author their coverage or mutes so the same words do-not appear twice. Define any mixed-family support explicitly in the component that implements it.+The chosen family interprets its own Styles. Fine handles its uniform-flow Styles; a structural+family handles its own layouts. Each Track is independent, so multiple Tracks can intentionally+show captions together or use complementary coverage. Mixed-family rendering, if useful, belongs+to the component that implements it.If this family supports spatial tracking, take an explicit RegionTimeline and map it into the actualcomposition. Define subject matching and absent-region behavior. Detection belongs to the measurement@@ -143,6 +143,7 @@ and authored word roles editable in the project. The package adds its renderingconsuming the existing Script, Caption and semantic-time owners.For live Cue entities and Inspector editing, add a [Studio Companion](studio.md#give-a-project-component-a-useful-companion)-that consumes the family's actual schedule, CaptionDocument and Program. Preserve the selected-Style reference for each Cue rather than exposing one misleading global Style. Caption Fine's+that presents complete Cue content and authored Uses as separate lanes. Read the Track's resolved+Use collection, retain each child's Source range and temporal lineage, and expose the referenced+Style on that Use. Caption Fine'sCompanion is a useful example of these relationships; the new family's layout stays in its renderer.diff --git a/skills/hypit/references/production/caption-presentation.md b/skills/hypit/references/production/caption-presentation.mdnew file mode 100644index 00000000..d21f4f35--- /dev/null+++ b/skills/hypit/references/production/caption-presentation.md@@ -0,0 +1,66 @@+# Caption presentation++Read this to apply subtitle treatments, change them during a passage, or hide them temporarily.+[Caption craft](../playbooks/craft/captions.md) owns visual direction;+[Caption authoring](caption-authoring.md) explains creating a new family.++**Script organizes content; Use organizes presentation in time.** `||` determines which words belong+together. The Track joins `document={story.caption}` to the placed `timeline`, preserving displayed+text and pronunciation associations. Each Use selects a complete Style inside a time window.++```svml+<caption:Hidden id="hidden"/>+<caption-fine:Track id="captions" document={story.caption} timeline={film.timeline}>+ <caption-fine:Use style={base-style}/>+ <caption-fine:Use role="GUEST" style={guest-style}/>+ <caption-fine:Use during={story.selection.answer} style={answer-style}/>+ <caption-fine:Use during={story.selection.demonstration} style={hidden}/>+ <caption-fine:Use at={story.moment.key} for="2s" style={impact-style}/>+</caption-fine:Track>+```++Styles are declared by the chosen family. Fine Styles take a Recipe and exact fonts. A new structural+caption can use a project family with its own layout and behavior.++## One time language++| Intent | Use |+| --- | --- |+| A treatment across the whole Timeline | `style={base}` with no time attributes |+| Follow a semantic passage | `during={story.selection.answer}` |+| Follow a placed Segment | `during={story.segment.answer}` |+| Start at a meaningful word, for a fixed duration | `at={story.moment.key} for="2s"` |+| End at a meaningful event | `until={story.moment.key} for="12f"` |+| Explicit window | `start="8s" end="10s"` |+| A particular speaker | Add `role="GUEST"` to any of these |++[Performance timing](performance.md) explains the shared expressions and semantic references.+Use semantic boundaries when the treatment follows what is being said. Absolute times remain useful+for an explicitly timed presentation. A Role filters whose content is presented, independently of the+window: it can cover several turns and does not change the Window shown in Studio.++The last matching Use in Source order wins locally. Each match replaces the whole Style; properties+are not merged. A hidden Use clears this Track's subtitle presentation throughout that window,+including neighboring Cue lead/tail. Later Uses can restore a smaller interval. Audio and content+stay unchanged. With no matching Use, nothing is drawn. With no subtitle content, a Use has nothing+to draw. Independent Caption Tracks can intentionally show multiple presentations at the same time.++## A style change can happen inside a Cue++```text+test1 || test2 @select test3 || test4 @/select+```++This contains three Cues: `test1`, `test2 test3`, and `test4`. The Selection starts before `test3`;+it does not split `test2 test3`. With a broad ordinary Use and a later emphasized Use during `select`,+the complete second Cue changes Style at that boundary, then the third Cue uses the same emphasis.++Word times remain the original times. Karaoke or text reveal therefore continues at the appropriate+word when the Style changes or returns after a hidden interval. The Window limits visibility; it does+not restart the Cue's animation clock. Changing only one word's visual role is a different design+choice: Fine provides current/trail unit emphasis, while a custom family can interpret authored word+attributes for structural layouts. Display/pronunciation units remain complete.++In Studio, the content row shows Cues and the Uses row shows authored windows. Select a Use to edit+its time or referenced Style. Moving the Use does not move `||`. Inspect the resulting placement,+handoffs and emphasis alongside the other visuals while reusing the existing media.diff --git a/skills/hypit/references/production/caption-program.md b/skills/hypit/references/production/caption-program.mddeleted file mode 100644index f5a47201..00000000--- a/skills/hypit/references/production/caption-program.md+++ /dev/null@@ -1,93 +0,0 @@-# Caption styling and coverage--Read this to apply a caption treatment to speakers or passages, change it locally, or leave selected-words without captions. [Caption craft](../playbooks/craft/captions.md) owns the visual direction;-[Caption authoring](caption-authoring.md) explains creating a new rendering family.--The Script owns displayed words and Cue breaks. A family's Style owns a resolved visual treatment.-`caption:Program` assigns those Styles to the words, then the family's Track joins them to the actual-SemanticTrack and draws them. Changing the appearance can reuse the existing performance and timing.--## Choose the words that receive a treatment--| Intended scope | Authoring |-| --- | --- |-| Ordinary treatment across the video | Set the Program's `default` Style. |-| A speaking character, across their turns and Segments | `Use role="GUEST" style={guest-style}` selects the Script Role. |-| A phrase, one Segment or a passage spanning several Segments | Name a Script Selection around that material and use `Use selection={story.selection.name} style={...}`. |-| A display word with a particular visual role | Mark a Script attribute and use `Use attribute="emphasis" style={...}` with a family that implements word-specific layout. |-| Selected words that should have no caption | `Mute role="GUEST"` or `Mute selection={story.selection.name}`. Speech and the semantic timeline continue. |--A Selection can cover a whole Segment; the Program consumes that Selection rather than a separate-`segment` selector. A Role selects the authored speaker, regardless of which person is currently-visible. For one speaker's particular reply, place a Selection around that reply. Each `Use` selects-one of `role`, `selection` or `attribute`; each `Mute` selects one of `role` or `selection`.--## Apply broad choices and local overrides--The following excerpt assumes the named Styles are already declared in the chosen family:--```svml-<script id="story">- @opening-look- <opening>- <HOST> One simple idea.- <GUEST> @answer It changes the whole picture. @/answer- </opening>- @/opening-look-- <demonstration>- <HOST> @clear-picture Let me show you how it works. @/clear-picture- </demonstration>- <closing><HOST> Try it with your product.</closing>-</script>--<caption:Program id="caption-program" document={story.caption} narrative={story}- default={base-style}>- <caption:Use role="GUEST" style={guest-style}/>- <caption:Use selection={story.selection.opening-look} style={opening-style}/>- <caption:Use selection={story.selection.answer} style={answer-style}/>- <caption:Mute selection={story.selection.clear-picture}/>-</caption:Program>-```--Role and Selection applications are evaluated in Source order. **The last matching `Use` wins** for-each complete display unit. In this example, the opening treatment replaces the guest treatment in-the opening passage, then the answer treatment replaces it in the selected reply. The closing keeps-the default. This gives the author an explicit order for broad treatments and particular moments;-selector specificity or proximity does not choose the winner.--Each match chooses a complete Style. To make an answer differ only in color, declare an answer Style-with the intended complete treatment; Program does not merge its properties with the previous Style.-The resulting Style boundaries form Cue handoffs, alongside Segment, Role-turn and authored `||`-boundaries. A selection-wide Style change is useful when a phrase becomes its own caption state.--`Use attribute="emphasis"` has a different purpose: it describes local word roles within a Cue for a-family that can arrange them. Fine keeps a uniform flow and does not implement word-specific Style-runs. An oversized keyword beside supporting words belongs naturally to a custom family consuming-those runs. Conflicting Styles applied to the same attributed word produce an error; the common-word-attribute mechanism does not use the Role/Selection override rule.--## Let coverage serve the composition--Mute is a display selection, useful when another visual already carries the words, a demonstration-needs the space, or another Caption family presents that passage. All Mute selections contribute to-the hidden words; a later Style application does not make them visible again. It changes neither-audio routing nor the performance. For a different version of the video, revise the Program's Mutes.--The family's lead, tail and handoff still determine the visible envelope of neighboring Cues. When-a passage needs a clear picture, select complete Cues and choose their neighboring lead/tail so the-previous or following caption also releases that space. Selecting a word for Mute removes that word;-it does not by itself make every caption pixel disappear during the word's time.--One family can implement several related treatments through its own Styles. When independent-families are useful in one video, give each its own Program and Track, with complementary coverage.-The common Program assigns Styles; it does not dispatch them to renderers automatically. A scene-whose words and graphics share behavior can instead own their rendering together through the common-Caption timing inputs, as described in [Caption authoring](caption-authoring.md).--Use semantic boundaries for the relationship: a Selection follows the intended words after a Script-or performance change. Keep a Dual Text unit complete when selecting it. Fine's moving placement-can independently follow measured head regions through [Caption tracking](../playbooks/craft/caption-tracking.md).-Inspect the resulting handoffs and placement with the other visuals while reusing the production's-media.diff --git a/skills/hypit/references/production/component-design.md b/skills/hypit/references/production/component-design.mdindex 85d98863..14482990 100644--- a/skills/hypit/references/production/component-design.md+++ b/skills/hypit/references/production/component-design.md@@ -1,8 +1,8 @@# Designing a good component-Read this when a video needs a new visual role, shared state or expressive structure. Design it with-the taste of a director and the care of someone who will use it in a real composition. A useful-component makes a particular idea legible and gives its author clear ways to direct that idea.+Read this when deciding how to organize a picture, a transition or a recurring visual system.+Design it with the taste of a director and the care of someone who will use it in a real composition.+A useful component makes a particular idea legible and gives its author clear ways to direct it.[Graphic composition](../playbooks/craft/graphic-compositions.md) owns visual hierarchy and motion;[Track authoring](track-authoring.md) owns the implementation. A new Caption family also draws on@@ -15,16 +15,31 @@ ranking accumulates, or a phrase lands with particular emphasis. Explain what apthere, what changes, and what the viewer should retain afterward. That explanation suggests boththe visual design and the component boundary.+Existing components are useful expressions of familiar relationships. Use one when its behavior+fits, or define a project component for the role this work needs. Its content and timing can vary+while the relationship remains clear. A new component can combine existing helpers with its own+drawing and animation; ordinary authoring includes both using and extending that vocabulary.+A board whose rows share layout and persistent state benefits from one component. An independentphoto and title can remain peer Tracks. An object that survives several camera cuts keeps the sameauthored identity while its state develops. Let shared behavior define the unit.A performance moving from full screen into a side viewport and a diagram filling the released spacecan share one component. It owns their relative layout, overlap, masking and coordinated motion.-A-roll names the performance carrying semantic time; it does not reserve the picture for Speech-Track. Ordinary presentation can reuse Media; a new relationship can be authored directly. A+A-roll supplies the performed passage; visual ownership follows the behavior being designed.+Existing footage can use [Performance Styles](performance.md); independent assets use Media. Aone-off scene is a useful component too. Caption and independent overlays can remain separate.+Decompose a scene further where its parts have meaningful independent responsibilities. Keep+coordinated geometry, state and transitions together where that makes their behavior clear. Internal+functions and subcomponents can organize this code without each becoming another Track or package.+The useful division preserves the relationship through change; more layers alone do not improve it.++Consider the change the user actually wants. If the product changes, the comparison's content may+change while its explanatory behavior remains. If delivery changes, its spoken reveal should move+with the relevant Moment. If a presenter and a diagram exchange space, their shared layout should+remain coordinated. These relationships suggest the inputs and boundaries worth authoring.+Choose visual grouping and timing independently. A combined scene can respond to spoken Moments;separate Tracks can share an authored event. What belongs together on the canvas does not decidewhat gives that behavior its time.@@ -35,8 +50,9 @@ behavior can keep these visuals together; [Caption authoring](caption-authoring.specialized text inputs and the same freedom to compose.Give a reusable scene the prepared performance, its outer Window and the Moment that changes its-layout. Replacing a product or rewriting the Script then changes content and semantic anchors while-preserving the behavior. The final frame positions come from the selected Takes.+layout through the Timeline and shared temporal projections. Replacing a product or rewriting the+Script then changes content and semantic anchors while preserving the behavior. The final frame+positions come from the placed Takes.Sketch its intended Source use with this video's actual content. The tag and its children shouldread like a concise account of the visual idea: which subjects are compared, which answer is already@@ -46,7 +62,7 @@ helps reveal a missing relationship before it becomes rendering code.## Let meaning drive the behaviorPrefer Script Selections, Moments and Segments for events that respond to the argument or performance.-The Surface projects them through the accepted SemanticTrack; the component consumes the resulting+The Surface projects them through the accepted Timeline; the component consumes the resultingWindows or Instants. A different delivery can then move the event while preserving its purpose.Explicit time remains useful for an authored lead, a short entrance or another clock-based decision.In a pure MG piece, name the events that carry its meaning and direct their reading rhythm. A usefuldiff --git a/skills/hypit/references/production/component-visuals.md b/skills/hypit/references/production/component-visuals.mdindex 42fe37aa..eb0218b1 100644--- a/skills/hypit/references/production/component-visuals.md+++ b/skills/hypit/references/production/component-visuals.md@@ -11,7 +11,7 @@ visible lifetime. For a reveal card, those decisions become a box and text, a dean entrance animation and the interval for which the answer remains visible.A component can first compute a domain program, such as a list of answers and their reveal times.-Its drawing Producer receives that program, ProgramSpace, Frames, Style and explicit assets, then+Its drawing Producer receives that program, Timeline, Frames, Style and explicit assets, thenreturns a VisualTrack. A simple component can produce the Track directly when no separate programwould help its behavior or editing.@@ -19,7 +19,7 @@ The public representation has these parts:| Part | What it means || --- | --- |-| Track | Named picture contribution belonging to one ProgramSpace |+| Track | Named picture contribution belonging to one Timeline || Present | An independently timed and stacked appearance || `span` | Program frames from `startFrame` through the frame before `endFrameExclusive` || `stacking` | Absolute paint order among Presents; higher order appears above lower order |@@ -109,11 +109,11 @@ video sampling and font handling while participating in the program's HTML layouused by the program belong in its `artifacts` list; `hyperframesResourceUri` supplies their resourceURLs. The rendering environment materializes those references.-For a semantic performance, `projectSemanticMedia(semantic, window.span)` from-`@hypit/hypit/semantic-track` returns each intersecting Take's prepared media, program span and source+For a semantic performance, `projectTimelineMedia(semantic, window.span)` from+`@hypit/hypit/timeline` returns each intersecting Take's prepared media, program span and sourcespan. Subtract the outer Window's start to obtain Present-local video sampling intervals. Preservethe returned source offset: moving or reframing a video changes its presentation while playback-continues from the same place. Original sound can remain `speech.audio` in Film.+continues from the same place. A separate Sound presentation can continue the existing audio in Film.A reusable scene might expose `during={story.selection.explanation}` for its lifetime and`reveal={story.moment.demonstrate}` for its layout change. The Surface projects these independently;diff --git a/skills/hypit/references/production/examples/production.svml b/skills/hypit/references/production/examples/production.svmlindex 93d33bdc..b7fb76b5 100644--- a/skills/hypit/references/production/examples/production.svml+++ b/skills/hypit/references/production/examples/production.svml@@ -1,12 +1,14 @@<?svml using="@hypit/markup@1"?><svml><import as="media-track" from="@hypit/media-track@1"/>+ <import as="sound" from="@hypit/sound@1"/>+ <import as="performance" from="@hypit/performance@1"/><import from="@hypit/script@1"/><import as="text" from="@hypit/text@1"/><import as="seedance" from="@hypit/seedance@1"/><import as="pipeline" from="@hypit/media-pipeline@1"/><import as="whisperx" from="@hypit/whisperx@1"/>- <import as="speech" from="@hypit/speech-track@1"/>+ <import as="time" from="@hypit/timeline-author@1"/><import as="space" from="@hypit/spatial@1"/><import as="program" from="@hypit/program-space@1"/><import as="fonts" from="@hypit/fonts-open@1"/>@@ -35,28 +37,33 @@video="primary-moving" audio="default" span-authority="video"/><whisperx:SemanticTake id="opening-semantic" narrative={story}segment={story.segment.opening} media={performance-media.media} language="en"/>- <speech:Track id="speech">- <speech:Take source={opening-semantic.take}/>- </speech:Track>- <media-track:Track id="speech-picture" semantic={speech.semantic} canvas={canvas}>- <media-track:Performance during="program" frame={full}- appearance={look.media.performance}/>- </media-track:Track>+ <time:Timeline id="speech" clock={clock}>+ <time:Take source={opening-semantic.take}/>+ </time:Timeline>++ <sound:Style id="speech-sound-style"/>+ <sound:Track id="speech-sound" timeline={speech.timeline}>+ <sound:Use style={speech-sound-style}/>+ </sound:Track>+ <performance:Style id="speech-picture-style" frame={full} appearance={look.media.performance}/>+ <performance:Track id="speech-picture" timeline={speech.timeline} canvas={canvas}>+ <performance:Use style={speech-picture-style} during="program"/>+ </performance:Track><fonts:Stack id="title-font" family="inter" weight="700" style="normal" emoji="color"/><typo:Style id="title-style" recipe={look.text.title} font={title-font}><typo:Fill color="#F1E7D8"/></typo:Style>- <typo:Track id="titles" semantic={speech.semantic}>+ <typo:Track id="titles" timeline={speech.timeline}><typo:Area id="proof-label" placement={title-frame} style={title-style}during={story.selection.proof}>The useful part</typo:Area></typo:Track>- <film:Film id="main" canvas={canvas} semantic={speech.semantic} appearance={look.film.main}>+ <film:Film id="main" canvas={canvas} timeline={speech.timeline} appearance={look.film.main}><film:Track source={speech-picture.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={speech-sound.audio}/><film:Track source={titles.track}/></film:Film>- <render:Video id="final" composition={main.composition} semantic={speech.semantic}/>- <render:Video id="detail" composition={main.composition} semantic={speech.semantic}+ <render:Video id="final" composition={main.composition} timeline={speech.timeline}/>+ <render:Video id="detail" composition={main.composition} timeline={speech.timeline}start-frame="30" end-frame-exclusive="90"/></svml>diff --git a/skills/hypit/references/production/fonts-and-text.md b/skills/hypit/references/production/fonts-and-text.mdindex 40ab89f3..920e2cdb 100644--- a/skills/hypit/references/production/fonts-and-text.md+++ b/skills/hypit/references/production/fonts-and-text.md@@ -87,7 +87,7 @@ wanted.Typography is useful for titles, labels, verdicts and copy that follows its own display rhythm.Script Caption remains appropriate when the displayed wording follows the performance.-The following excerpt assumes the named Fonts, layout Frames, SemanticTrack and Recipes exist:+The following excerpt assumes the named Fonts, layout Frames, Timeline and Recipes exist:```svml<import as="copy" from="@hypit/text@1"/>@@ -97,7 +97,7 @@ The following excerpt assumes the named Fonts, layout Frames, SemanticTrack and<typo:Fill color="#F1E7D8"/></typo:Style><copy:Value id="headline">A useful idea, clearly shown.</copy:Value>-<typo:Track id="titles" semantic={speech.semantic}>+<typo:Track id="titles" timeline={speech.timeline}><typo:Area id="opening-title" content={headline} placement={title-frame}style={headline-style} during={story.selection.proof}/></typo:Track>diff --git a/skills/hypit/references/production/media.md b/skills/hypit/references/production/media.mdindex 2550550e..e727c0e1 100644--- a/skills/hypit/references/production/media.md+++ b/skills/hypit/references/production/media.md@@ -6,10 +6,10 @@ For a source video at a link, read [video download](video-downloads.md).For acquiring website screenshots, page recordings or local HTML graphics, read[browser capture](browser-capture.md).-Media carries the visible performance and sound. Preparing it for a video answers two further-questions: how it occupies the program's time, and which Script passage it realizes. Normalization-establishes the frame clock and selected picture/audio streams. A SemanticTake adds the association-with Script and the locations of its words and boundaries.+Media carries picture and sound. Normalization establishes its frame clock, local duration and+selected picture/audio streams. A SemanticTake adds the association with a Script passage and the+local locations of its words and boundaries. [Timeline assembly](timeline.md) then determines where+the prepared Take belongs in the complete work.## Files and generated Outputs@@ -81,21 +81,22 @@ For an already declared Script and prepared spoken performance:WhisperX supplies timed speech evidence; alignment locates the authored Script in that evidence.Script remains the wording authority, and this step establishes where its words occur in theperformance. The result `opening-semantic.take` contains the same media and that Segment's local-timing. Speech Track assembles Takes in their intended order and translates their local positions-into program time.+timing. [Timeline assembly](timeline.md) places Takes sequentially or at authored starts and+translates their local positions into Program time.Use the performed language supported by the selected package and Endpoint.Audio-only A-roll follows the same path with audio-only media. Voice Clone produces the Segment'sperformed audio from its Script and the person's Voice Reference. Normalize that output with`video="none"`, `audio="default"` and `span-authority="audio"`, then align it to the same Segment.-The resulting SemanticTake publishes semantic time and sound through Speech Track without inventing+The resulting SemanticTake publishes semantic time and sound through Timeline assembly without inventinga visual performance; the work's Media, Typography or MG Tracks supply the picture.## Empty Segments use their media boundariesAn ordinary Script Segment such as `<empty></empty>` can carry a passage without words. Its preparedmedia gives it a duration, so the resulting SemanticTake has the Segment's start/end Anchors and no-timed Tokens. It enters the same Speech Track assembly as a spoken Take.+timed Tokens. It enters the same Timeline assembly as a spoken Take. An interval with no performance+material can instead be left open by Timeline placement or extent, with graphics authored there directly.The semantic Surface maps an empty Segment directly to its prepared media domain:@@ -106,7 +107,7 @@ The semantic Surface maps an empty Segment directly to its prepared media domainThis performs boundary materialization without a transcription request because the Segment has noTokens. Action, music or visual rhythm can determine the media duration. The ordinary SemanticTake-enters Speech Track alongside spoken Takes.+enters Timeline assembly alongside spoken Takes.## Give a still a duration when that is its rolediff --git a/skills/hypit/references/production/performance.md b/skills/hypit/references/production/performance.mdnew file mode 100644index 00000000..c658407c--- /dev/null+++ b/skills/hypit/references/production/performance.md@@ -0,0 +1,69 @@+# Presenting an existing performance++Read this to place existing A-roll full frame, in an inset, beside graphics, or in a changing layout.+[Timeline](timeline.md) places the prepared Takes; Performance directs their picture. Media Items+instead take independently supplied material, just as Typography supplies text while Caption uses+Script's words. A-roll may be audio-only, leaving the picture to other components.++## Broad treatment, local Uses++```svml+<import as="performance" from="@hypit/performance@1"/>+<performance:Style id="full" frame={layout.full} appearance={recipes.media.presenter}/>+<performance:Style id="side" frame={layout.side} appearance={recipes.media.presenter}/>+<performance:Track id="presenter" timeline={program.timeline} canvas={canvas}>+ <performance:Use style={full}/>+ <performance:Use during={story.selection.explanation} style={side}/>+</performance:Track>+```++The excerpt assumes the Timeline, Canvas, Frames and Media appearance Recipe exist. The ordinary+Style shares Media fitting, crop anchors, rounded clipping, border and sample appearance. Include+`presenter.visual` and a [Sound](sound.md) presentation separately in Film. Sound obtains the same+placed material and handles gain, silence and explicit source blending.++An unqualified Use sets the broad treatment. Later matching Uses replace the whole choice locally;+this is the same broad/local rule as Caption styling. Performance selects frame Windows; Caption+selects authored display units. Separate presentation Tracks remain independent.++Use supports `during`, `at` + `for`, `until` + `for`, and `start` + `end`. A Moment can start a+fixed-duration change. No matching rule contributes no picture; an intentionally empty winning+rule stays empty. Source gaps and audio-only Takes need no visual stand-in. Extend Timeline's end+for a code-authored closing passage, and place its graphics there directly.++## Give a new visual behavior a Style++A recurring presenter motion can be a project Style. It receives the Timeline, Canvas and original+Use Window, plus its own explicit parameters. Think of a specific reusable behavior, such as a+full-frame viewport moving into a side Frame. The family owns its trajectory and exposes the inputs+that make it useful; arbitrary Styles do not automatically morph into one another.++Declare the initial treatment, then the settled treatment starting at the event, then the short+movement Use last. It covers the start of the settled treatment and naturally reveals that treatment+when finished. Matching endpoint geometry comes from shared Frames or parameters. The source video+continues playing throughout; partial coverage does not reset its animation clock.++An overlap transition is also a Use. For two Takes, its Window can run from the next Segment's start+to the previous Segment's end:++```svml+<performance:Use style={mix}+ start-source={story.segment.next} start="segment.start"+ end-source={story.segment.previous} end="segment.end"/>+```++Define `mix` in the chosen family with explicit outgoing/incoming sources. Those roles are separate+from the occurrence Window. The ordinary Style layers simultaneous footage in declaration order;+an overlap by itself does not create a transition.++For implementation, a Style uses `performanceStyle(fragment, bindings)` from+`@hypit/hypit/performance`. The fragment receives `timeline`, `canvas`, `window` and family-owned typed+inputs, and exports `visual: VisualTrack`. Bind extra inputs to resolved author references; they+become explicit graph dependencies at each Use. Use `projectTimelineMedia` for source identities,+program spans and source offsets, and the shared temporal helpers for additional events. All existing+structural/browser drawing capabilities remain available. [Track authoring](track-authoring.md) and+[component drawing](component-visuals.md) explain that implementation path.++When presenter and graphics share a larger coordinated scene, give that scene its own component.+Choose the abstraction by the behavior worth reusing; a project Style does not have to become an+official preset before it can be used.diff --git a/skills/hypit/references/production/rendering.md b/skills/hypit/references/production/rendering.mdindex a3e65297..97546ac3 100644--- a/skills/hypit/references/production/rendering.md+++ b/skills/hypit/references/production/rendering.md@@ -15,19 +15,25 @@ With the named inputs already declared:<import as="film" from="@hypit/film@1"/><import as="render" from="@hypit/render-hyperframes@1"/>-<film:Film id="main" canvas={canvas} semantic={speech.semantic}+<import as="sound" from="@hypit/sound@1"/>+<sound:Style id="voice-style"/>+<sound:Track id="voice" timeline={speech.timeline}>+ <sound:Use style={voice-style}/>+</sound:Track>++<film:Film id="main" canvas={canvas} timeline={speech.timeline}appearance={look.film.main}><film:Track source={performance.visual}/>- <film:Track source={speech.audio}/>+ <film:Track source={voice.audio}/><film:Track source={coverage.visual}/><film:Track source={captions.track}/><film:Track source={music.track}/></film:Film>-<render:Video id="final" composition={main.composition} semantic={speech.semantic}/>+<render:Video id="final" composition={main.composition} timeline={speech.timeline}/>```An example Film Recipe is `film.main { background: #18212A; }`. Canvas supplies the picture dimensions;-the SemanticTrack supplies program time. Include each wanted audio output explicitly. A covering+the Timeline supplies program time. Include each wanted audio output explicitly. A coveringpicture leaves the included performance audio audible. Layer order is authored in the Tracks'Presents, so moving these Film children does not reorder the picture.@@ -37,34 +43,35 @@ A compatible Composition from another component can also feed the render Surface## Compose an authored animationA Film needs a time axis, whether or not it contains speech or prepared media. For a speech-led piece,-continue to pass `semantic={speech.semantic}`: it provides both the real performance time and the-context in which Tracks resolve Script references. For a pure MG piece, declare a ProgramSpace and-pass `space` to the components, Film and Render:+continue to pass `timeline={speech.timeline}`: it provides both the real performance time and the+context in which Tracks resolve Script references. For a pure MG piece, use the same Timeline with+an explicit end and zero Takes, and pass it to the components, Film and Render:```svml-<import as="time" from="@hypit/program-space@1"/>-<time:Space id="animation" frame-rate="30" duration="8s"/>+<import as="time" from="@hypit/timeline-author@1"/>+<time:Clock id="animation-clock" frame-rate="30"/>+<time:Timeline id="animation" clock={animation-clock} end="8s"/><!-- conversation is the project's own visual component. -->-<chat:Scene id="conversation" space={animation} canvas={canvas} font={font}+<chat:Scene id="conversation" timeline={animation.timeline} canvas={canvas} font={font}during="program" title="Launch crew"><chat:Message id="question" sender="Maya" side="left" at="0.5s" text="Ready?"/><chat:Message id="answer" sender="Leo" side="right" at="2s" text="Let's go."/></chat:Scene>-<film:Film id="main" canvas={canvas} space={animation} appearance={look.film.main}>+<film:Film id="main" canvas={canvas} timeline={animation.timeline} appearance={look.film.main}><film:Track source={conversation.track}/></film:Film>-<render:Video id="final" composition={main.composition} space={animation}/>+<render:Video id="final" composition={main.composition} timeline={animation.timeline}/>```-The example assumes the Canvas, font, Film Recipe and project package are declared. `time:Space`+The example assumes the Canvas, font, Film Recipe and project package are declared. Timeline's `end`accepts seconds, milliseconds or frames and must end on a frame boundary. `Clock` remains theseparate duration-free input for normalizing real media. Drawing code produces the picture at eachrequested frame; the Film's background supplies the canvas color. With no AudioTrack, the deliveredvideo is silent. Render ranges and worker settings apply in the same way as for spoken work.-Media, Typography, Audio and the graphic Tracks accept this same time context. A Media Performance-specifically needs its SemanticTrack because that is where its prepared footage resides. Caption+Media, Typography, Audio and the graphic Tracks accept this same Timeline. Performance obtains+any available prepared footage from it. Captionuses its speech-linked document; authored chat text belongs to the chat scene. The working example`examples/semantic-composition/chat.svml` and its `@example/chat-scene` package show the completecode-only composition, including scrolling and arbitrary message arrivals.@@ -72,7 +79,7 @@ code-only composition, including scrolling and arbitrary message arrivals.## Choose a render interval in frames```svml-<render:Video id="detail" composition={main.composition} semantic={speech.semantic}+<render:Video id="detail" composition={main.composition} timeline={speech.timeline}start-frame="240" end-frame-exclusive="360"/>```diff --git a/skills/hypit/references/production/review.md b/skills/hypit/references/production/review.mdindex 322e9de8..14097436 100644--- a/skills/hypit/references/production/review.md+++ b/skills/hypit/references/production/review.md@@ -11,8 +11,10 @@ project Companions. [Builds and Results](builds.md) explains retrieval and expliCompose MG, Caption, B-roll, Typography and Effects around the directed material and existing Outputs.Watch their actual arrangement in Studio or a rendered Result: is the comparison clear, is theCaption readable, does the B-roll cover the intended explanation, and do entries and exits land on-the right words or actions? Adjust the layout, timing or component behavior that will make the-passage work better. Judge a local change in the surrounding composition.+the right words or actions? Does a persistent object retain its identity and state through the+handoff, and does moving a video viewport keep its intended playback? Adjust the layout, timing or+component behavior that will make the passage work better. Judge a local change in the surrounding+composition.Work from the authored Script and the production's existing semantic timing when locating a word,reveal or handoff. Those relationships identify the passages whose composition needs attention;diff --git a/skills/hypit/references/production/sound.md b/skills/hypit/references/production/sound.mdnew file mode 100644index 00000000..85f5d81a--- /dev/null+++ b/skills/hypit/references/production/sound.md@@ -0,0 +1,76 @@+# Presenting existing sound++Use Sound to present audio from Takes already placed on the [Timeline](timeline.md). Use+[Audio Track](tracks.md#sound-text-and-caption-answer-different-events) for independently supplied+music and effects. Sound chooses how existing sources are heard; [Performance](performance.md)+chooses how their pictures appear. Neither changes the Script or the Take placements.++## Broad treatment and local Uses++```svml+<import as="sound" from="@hypit/sound@1"/>+<sound:Style id="normal"/>+<sound:Style id="opening" gain="0" end-gain="1"/>+<sound:Style id="silent" gain="0"/>+<sound:Track id="voice" timeline={program.timeline}>+ <sound:Use style={normal}/>+ <sound:Use at="0f" for="2s" style={opening}/>+ <sound:Use during={story.selection.demonstration} style={silent}/>+</sound:Track>+```++Select `voice.audio` in Film to hear this presentation. Timeline retains source material; Sound+chooses how it is heard. Independent music and effects remain separate Film contributions.++The first Use sets a broad treatment. The last matching Use replaces it locally, including when+the winner is silent. A later normal Use can restore sound inside a muted passage. No matching+Use means no sound from this Track. This is the same broad/local presentation rule as Performance+and Caption, with time Windows as Sound's domain.++The ordinary Style chooses the last Timeline-declared active source with audio. A Take without an+audio member contributes no candidate. No Take means no original sound; it does not affect separate+music or effects. A recording that contains silence is still an audio source.++`gain` is a linear amplitude multiplier, normally 1; `end-gain` defaults to the same value. Different+values create a linear change over the Use's full Window. Use `gain="0"` for silence, or+`gain="1" end-gain="0"` for a fade-out. A leading Timeline gap has no source to fade: attach an+opening fade to the first Take's actual start when it begins later than the program.++All shared time forms apply: `during`, `at` + `for`, `until` + `for`, and `start` + `end`. Word-related+changes can follow Selections or Moments; independently timed changes can use seconds or frames.+The Window governs treatment progress, while the Take placement governs source playback. After a+local override, both resume at the progress appropriate to the current time.++## Explicit source relationships++For a crossfade, a project Style selects its outgoing and incoming Segments explicitly. An ordinary+Use then selects the overlap Window. A Style declaration might look like:++```svml+<mix:Crossfade id="handoff"+ outgoing={story.segment.first} incoming={story.segment.second}/>+<sound:Track id="voice" timeline={program.timeline}>+ <sound:Use style={normal}/>+ <sound:Use style={handoff}+ start-source={story.segment.second} start="segment.start"+ end-source={story.segment.first} end="segment.end"/>+</sound:Track>+```++Here `mix:Crossfade` is a project-defined Style, not a built-in Sound tag. Its complete example ships+at `examples/semantic-composition/packages/sound-styles/` in the Hypit Distribution. The interval+locates the effect; the bound sources determine its roles. Sound does not infer a mix from a visual+transition. A family for intentional simultaneous speech can retain both sources at full gain.++## Author a new sound Style++Use `soundStyle(fragment, bindings)` from `@hypit/hypit/sound`. Its fragment receives `timeline` and+the original `window`, plus the family's typed bindings, and exports `audio: AudioTrack`. A Surface+publishes this definition as an inline `soundTypes.style` value. Each Use expands normal graph edges.++`sourceSound(timeline, window, segment, { gain, endGain })` selects one existing source, and+`ordinarySound(timeline, window, { gain, endGain })` provides the usual last-source behavior. Known+sources without audio return no clips. A custom family can combine explicit sources or author a+richer gain envelope using ordinary AudioClip data. No new renderer mode is needed.+See [component authoring](track-authoring.md) for packaging and [sound direction](../playbooks/craft/sound-mix.md)+for deciding what the audible change contributes to the work.diff --git a/skills/hypit/references/production/source-syntax.md b/skills/hypit/references/production/source-syntax.mdindex 1ca4f6d4..0a6e2ba6 100644--- a/skills/hypit/references/production/source-syntax.md+++ b/skills/hypit/references/production/source-syntax.md@@ -56,7 +56,7 @@ Output paths are package vocabulary, not inferred from the element's tag:- an imported `media:Image id="logo"` publishes `{logo}`;- `gpt:Image id="portrait"` publishes `{portrait.image}`;- `pipeline:Normalize id="prepared"` publishes `{prepared.media}`;-- `speech:Track id="speech"` publishes `{speech.semantic}` and `{speech.audio}`.+- `time:Timeline id="speech"` publishes `{speech.timeline}`. Performance and Sound separately publish the picture and sound contributions selected in Film.Inspect `hypit vocabulary` for the actual Surface's exports and use the published path it reports.Public bindings can be authored Records or realizable Logical Outputs. Script,diff --git a/skills/hypit/references/production/spatial.md b/skills/hypit/references/production/spatial.mdindex fb3a34ce..31b7a01f 100644--- a/skills/hypit/references/production/spatial.md+++ b/skills/hypit/references/production/spatial.md@@ -106,13 +106,14 @@ already available:```svml<space:AnchoredFrame id="presenter-frame" within={canvas}x="94%" y="94%" width="320px" height="320px" anchor="bottom-right"/>-<speech:Track id="speech">- <speech:Take source={opening.take}/>-</speech:Track>-<media:Track id="presenter" semantic={speech.semantic} canvas={canvas}>- <media:Performance during="program" frame={presenter-frame}- appearance={look.presenter}/>-</media:Track>+<time:Clock id="clock" frame-rate="30"/>+<time:Timeline id="speech" clock={clock}>+ <time:Take source={opening.take}/>+</time:Timeline>+<performance:Style id="presenter-style" frame={presenter-frame} appearance={look.presenter}/>+ <performance:Track id="presenter" timeline={speech.timeline} canvas={canvas}>+ <performance:Use style={presenter-style} during="program"/>+ </performance:Track>``````svs@@ -154,19 +155,19 @@ for a moving crop or a slow push-in while a card's outline stays still:```This excerpt belongs inside Media Track, with prepared media, a Selection, Frame and Recipe already-available. Sampling fields apply to a direct-source Item or Member, a sampled Layer, or a Performance.+available. Sampling fields apply to a direct-source Item or Member, or a sampled Layer.`x` and `y` are pixel offsets, `rotate` is indegrees, and `at` follows the source unit's active span from `start` to `end`, with percentages forintermediate keys. Sampling acts after the static fit; its movement can expose space inside theFrame. Choose the crop and motion together for the intended coverage.-A Performance uses the Track's prepared semantic material, with Sampling and whole-frame motion-running across its full display Window, including Take boundaries. A scene that coordinates a moving-performance viewport with surrounding graphics can own that motion in its-[component program](component-visuals.md#compose-video-and-graphics-in-one-browser-program), while-retaining the prepared video's source positions.+[Performance](performance.md) applies fixed or custom Styles to existing Timeline footage. A moving+Style retains the original Use Window across partial coverage and Take boundaries. A scene that+coordinates the viewport with surrounding graphics can own the shared motion in its+[component program](component-visuals.md#compose-video-and-graphics-in-one-browser-program), retaining+source playback positions.-[Tracks](tracks.md) explains the Media/Speech inputs and timing roles. Read their installed+[Tracks](tracks.md) explains the Media and Performance inputs and timing roles. Read their installedvocabulary for the complete appearance and motion fields.## Carry measured regions through the same geometrydiff --git a/skills/hypit/references/production/studio-companions.md b/skills/hypit/references/production/studio-companions.mdindex 5df8f4b1..06c40b4e 100644--- a/skills/hypit/references/production/studio-companions.md+++ b/skills/hypit/references/production/studio-companions.md@@ -36,7 +36,23 @@ it explains another authoring relationship, such as activation within a persisteAn activation rectangle may end before the object disappears. Keep the activation's editable rangeand the object's continuing picture identity distinct. The installed Ranking Companion demonstrates-this with board, reveal and activation lanes.+this with the board and its independent reveals.++## Put related rules in a Band++A Track's `bands` are internal strips sharing its label. For example, Caption shows Cue content+with a bottom band for Style Uses; Performance and Sound show placed Takes with their Uses below.+The rule is a selectable entity with its own Window, even when overridden or currently empty.+It remains one Use across gaps and content boundaries.++```ts+bands: [{ id: "uses", placement: "after", heightPx: 15, display: "label" }]+// The projected Use entity selects: band: "uses"+```++Use `attachments` for independent child objects such as Ranking reveals. Use a Band for another+aspect of the same Track such as its presentation choices. Both use ordinary entity selection,+overlap order, field declarations and time editing. Each entity chooses one declared band or lane.## Connect the picture to the entity@@ -56,8 +72,7 @@ then be adjusted through its declared Inspector fields.Declare `bindings` for the actual author endpoints, then choose visible `inspector` fields. Abinding can follow a shared authored Frame or Style through `referenced`, or an SVS Recipe through-`recipe`. Use `parameterReferences` for an entity's actual reference, such as the Style selected-for this Caption Cue. Shared values retain their shared effect when edited.+`recipe`. Use `parameterReferences` for an entity's actual reference, when a derived entity points to a different authored object. Shared values retain their shared effect when edited.Where describes placement and layout. When describes timing, playback and motion. How describesappearance, content and sound. Each Companion chooses useful pages and sections within them.@@ -69,7 +84,7 @@ appearance, content and sound. Each Companion chooses useful pages and sections| `boolean` | An actual two-state choice. || `select` | Preset, font, alignment or another finite choice, with readable labels. || `color` | Exact hex color, optionally with suggested `swatches`. |-| `list` / `record` | A schema-described ordered list or named set of values, edited as one Apply/Reset draft. |+| `list` / `record` | A schema-described ordered list or named set of values, saved together when editing ends and the value is complete. |For a Surface with a literal `label` attribute, a minimal declaration is:@@ -87,6 +102,24 @@ An opacity stored as `0.78` can use `unit: "%", number: { scale: 100, minimum: 0the control shows 78 and entering 42 writes 0.42. `unit` alone is a display label. Suffix handlingretains an authored unit; it does not convert percentages to pixels.+A Style can own its controls independently of the Track that consumes it. The Track declares+`{ name: "style", companion: true }`; the Style's package contributes+`createStudioCompanionHostFacet({ parameters: [{ id, match: { module, surface }, bindings, inspector }] })`.+Its field names are local to that Style. Studio follows the reference and combines those fields+with the Use's time controls. This works for a new project motion Style as well as a familiar+framed presentation. Shared Style edits continue to affect its other Uses.++A binding's typed `fallback` exposes an omitted default; first editing it writes the property or+attribute into its owning source. A fallback function can express a dependent default from the+authored properties. Choose useful valid defaults in the component's own vocabulary. Read-only+facts and controls share Where, When and How; unpaged fields remain visible beside the active page.++Related scalar attributes can share one `record` draft through a binding's `attributes` list and+schema. Use object alternatives with a literal mode field when each choice needs different inputs;+Once its required fields are complete, editing writes the chosen group together and removes unused members. Audio playback demonstrates+this for its mode and stretch bounds. The package owns those alternatives; see+the installed Studio adapter README, “Edit related attributes together,” for the exact declaration.+Select options may be plain strings or `{ value, label, description, preview }` entries. Color andfont previews are small visual hints; the actual option value is written. A font hint uses a familyavailable to the editor, while the production font remains an explicit resource as described in@@ -121,7 +154,7 @@ Use the installed `packages/studio-adapter/README.md` for the complete minimal Cactivation example, `packages/studio/INSPECTOR.md` for field declarations, and`packages/temporal-markup/EDITING.md` for exact time-form behavior. These are package referencesinside the installed Distribution. Ranking, Caption Fine and Media Track Companions demonstrate-persistent events, per-Cue styles and material occupancy respectively.+persistent events, Cue content with Style Uses, and material occupancy respectively.Try the component in its actual Run: select a meaningful picture part, inspect the correspondingtimeline entity, change an exposed value and inspect the owning Source and resulting picture.diff --git a/skills/hypit/references/production/studio.md b/skills/hypit/references/production/studio.mdindex 26ca0f6c..5fb18f42 100644--- a/skills/hypit/references/production/studio.md+++ b/skills/hypit/references/production/studio.md@@ -15,8 +15,8 @@ hypit studio --run build.svrunSupply `--runtime <profile>` when intentionally using a different Profile from the project's`hypit runtime use` selection. Studio needs a Run whose selected targets reach one Film with resolved-composition and time. Speech-led work includes a Script lane; a pure animation's ProgramSpace supplies-the clock for its component lanes. An image-generation-only Run is not a Film view; several distinct+composition and time. The Timeline supplies the clock and any placed Script anchors for component+lanes, including a pure animation with no Takes. An image-generation-only Run is not a Film view; several distinctFilms need separate Runs or sessions.Open the URL actually printed by the process and retain it with the Run it serves. The default@@ -54,14 +54,17 @@ include when the user wants to continue editing on another machine.| Source | The exact Run, Author and imported Source/Recipe files. Select a file and use Edit source; changes save automatically, and Cmd/Ctrl+S saves immediately. Check save/error state. This is not a project filesystem browser. || Preview | The selected Film composition rendered by HyperFrames in the browser. Play or seek with the transport or timeline. Selecting a component-declared visual part selects its corresponding timeline entity; adjust its exposed position in the Inspector. || Timeline | Semantic Segments, Selections and Moments, plus the component-projected Track entities and their visible intervals, materials or event lanes. A rectangle may describe occupancy, activation or persistent visibility; read the component's meaning. |-| Inspector | With nothing selected, project, Canvas, time and Run facts. For a selected entity, only its declared adjustable fields, organized under Where, How and When where applicable. |+| Inspector | With nothing selected, project, Canvas, time and Run facts. For a selected entity, its declared read-only facts and adjustable fields, organized under Where, When and How where applicable. || Tasks | One card per Build, grouped into ongoing and finished. Active status and progress come from the selected Runtime; completed, failed and cancelled Builds come from project Results. Cards retain the source Run, times and any failure or attention reason. || Artifacts | Image, video and audio file Outputs from project Results, including those already published by ongoing Builds. Use the sidebar to choose all media, videos, images or audio. View media on a Build opens its Outputs; opening the Artifacts tab returns to project media. Composite Outputs such as normalized media and Semantic Takes stay intact and do not add their internal files to this gallery. Click a card to view it in the central preview; video and audio have playback and a time slider. Back to composition returns to the existing composition position. Previewing a file does not select it as a Candidate in the Run. |A declared lane stays one row even when items overlap. Later items cover earlier ones at equalstacking order; selecting an item brings its full rectangle forward within that lane. This changeseditor selection, not the Film's paint order. The same behavior applies to attached child lanes.-When semantic time is present, its lane stays below the time ruler while the other lanes scroll.+The time ruler and its Segment, Word and Selection/Moment bands form one pinned Timeline area.+Empty information bands are omitted; a work without Segments keeps the ordinary time ruler.+Select an overlapping object to bring it forward, or right-click the overlap to choose one covered+by its peers. This changes editor selection, not the composition.Open either library tab or click Refresh to read its latest state. These lists do not poll.Refresh replaces the view; scrolling to the end loads more. Media categories query matching files@@ -92,8 +95,8 @@ for inspecting the composition as encoded media; choose the view that helps answInspector fields and timeline handles write back to actual Source endpoints. A field can live in areferenced Frame, Style or SVS Recipe rather than on the selected Track tag. Editing a shared Recipechanges every consumer that uses it; create and select a separate authored instance when the design-needs independent variation. Lists and records have a local draft with Apply and Reset; Apply writes-the complete validated value. Check save status and the resulting preview.+needs independent variation. Lists and records save their complete validated value when editing ends. Required fields that are+still incomplete remain in the editor with a completion hint. Check save status and the resulting preview.Where groups position, size, fitting and layout. When groups timing, playback, trims and motion.How groups content, typography, color, effects and audio levels. Components name their own pagesdiff --git a/skills/hypit/references/production/system.md b/skills/hypit/references/production/system.mdindex 821fefc6..57ba12b2 100644--- a/skills/hypit/references/production/system.md+++ b/skills/hypit/references/production/system.md@@ -2,18 +2,48 @@Read this for the relationships that connect authoring, media, composition and execution.+> Give semantics to the timeline, rather than the timeline to semantics.+>+> Give layers to the canvas, rather than the canvas to layers.++The work has a complete time range and a canvas. Add semantic attachments where events belong to+speech, and introduce component boundaries where they express useful shared behavior. A silent+ending can contain only graphics. A moving presenter and diagram can share one scene. Neither time+nor space needs to be completely partitioned by the same kind of object.+SVML expresses what belongs together in a video: which media carries a passage, which picturesupports an idea, and which words or events a Caption, graphic or sound follows. Those relationshipslet the production survive revision. When a performance becomes longer, a graphic attached to thesame phrase can still appear with that phrase.-Semantic authoring is the preferred starting point for a Hypit production, including a clone.+Treat components as vocabulary you can use and extend. A comparison, a scrolling conversation or a+presenter making room for a diagram can each have named content, events and behavior. Define the+parts that make this work understandable and useful to change. Existing Media, Caption and other+components can supply familiar behavior; a project component supplies a new role through the same+interfaces. [Component design](component-design.md) explains how to find those boundaries.++For spoken work, semantic authoring is the preferred starting point, including a clone.Understand which idea, question, answer or action each layer serves. Express that relationship inthe target's Script and components, then project it through the new media. For example, a productpicture follows the explanation of its benefit and a graphic settles when the speaker gives theverdict. The reference's seconds locate evidence; the target's actual performance supplies its time.-## Script gives meaning; media gives it time+## Materials meet composition at the Timeline++```text+Script + directed or supplied material+ ↓ normalization and, for Script performance, semantic preparation+ prepared material with local time+ ↓ placement+ Timeline: full extent, Takes and semantic anchors+ ↓ selected content, event times and source playback positions+ components → Film → rendered video+```++Independent images, B-roll and music bring their own assets to their components. Caption also uses+the Script's display text. The Timeline supplies their common time context; a Frame supplies spatial+placement. The diagram describes relationships, not a serial production checklist: independent+material requests and component implementation can progress together.Script names the work's Segments, words, Selections and Moments. A Segment identifies a passage;a Selection identifies a range within the Script; a Moment identifies a point. The author can refer@@ -21,15 +51,19 @@ to them before a performance has been generated.Once the media exists, normalization gives it a shared frame clock. A **SemanticTake** associatesthat media with one Script Segment and records its word and boundary positions in local frames.-Speech Track assembles the Takes, translating their positions into a **SemanticTrack** for the-complete program. Its **ProgramSpace** supplies the program's time axis. A pure graphic animation can-instead declare a ProgramSpace directly and draw each frame through components; it needs no performance media.+The **Timeline** places prepared Takes within the complete work. Their global word and boundary+positions are local positions plus the Take's placement start. It can contain gaps or overlapping+Takes, or no Takes at all. A pure MG work uses this same declaration with an authored end.++[Timeline authoring](timeline.md) explains sequential defaults, relative and absolute placement,+head/tail space and the unified component input.In spoken work, the performance carrying the main Script is the A-roll. A single presenter, aconversation with several speakers, and an independent narration can each carry this spine.-Covering the speaker with B-roll does not change whose words establish time. Speech Track assembles-the relevant Takes into one semantic timeline and publishes their original audio. Media or a project-component obtains the prepared picture from that semantic performance and decides how to present it.+Covering the speaker with B-roll does not change whose words establish time. Timeline assembly places+the relevant Takes and retains their source material. Performance or a project+component presents the placed picture; Sound presents the placed audio. Film explicitly selects+their contributions.The speaker can occupy a small circular inset or appear as a cutout above a full-screen demonstration.That speaking performance still supplies the SemanticTake. A-roll describes this semantic role;screen area and stacking belong to its visual presentation.@@ -37,12 +71,38 @@ screen area and stacking belong to its visual presentation.This is how “show the proof while she explains the result” becomes a precise interval in theproduced video. A wordless Segment works through the same relationship: its media supplies thespan, with start and end boundaries and no spoken words to locate.+A gap instead reserves time without a Take. Use Timeline placement and extent for such passages;+their graphics can be authored directly.[Script and semantic time](../creation/script-and-time.md) explains the authoring forms.[Media preparation](media.md) explains normalization and constructing Takes.## Components turn those relationships into picture and sound+Choose content ownership as well as presentation:++| Content relationship | Useful starting point |+| --- | --- |+| An independent image, video or surface with its own placement | Media |+| Existing Timeline footage shown full frame, inset, hidden or moving | Performance |+| Existing Timeline audio | Sound |+| Independently supplied music and effects | Audio Track |+| Independently authored text | Typography |+| Existing Script words presented at their performed times | Caption |+| A newly coordinated visual role with shared layout, state or motion | A project component |++These are useful abstractions to compose and extend. A custom Performance Style can direct footage;+a scene owning a presenter and diagram can expose their combined behavior. [Performance](performance.md)+and [component design](component-design.md) explain those choices. [Sound](sound.md) applies local+treatments to existing audio through the same Use model.++Content, Use and Style answer different questions: what is available, when a treatment applies,+and what that treatment does. Performance, Sound and Caption share timed Uses: a broad Use sets+the treatment, and later matching Uses replace it locally, including hidden or silent presentation.+The original source playback and word timing continue beneath those choices. Independent Tracks+can coexist when the work calls for simultaneous presentations. [Caption presentation](caption-presentation.md)+explains how complete Cues retain their content while their styling changes.+Choose each relationship at the scale that makes it useful. Script describes what an event follows;Frames describe placement; a component owns shared layout, state and motion. The dependency graphsays which values that work needs. These structures can differ: a Source element can produce several@@ -63,19 +123,59 @@ component can own the shared behavior of a more specific scene. [Component desigexplains that choice; [drawing a component](component-visuals.md) shows both structural elements andHTML/CSS programs with frame-driven behavior.+The component's Surface can accept a Moment, Selection or direct time and use shared projection+helpers to provide the resulting Instant or Window to its behavior. The author expresses the+relationship in one place; the implementation handles its conversion to frames. When presenting+speech media, its playback mapping also identifies the right Take and source frame. Moving its+viewport can therefore leave the performance playing continuously.++A local HTML/CSS program can own the needed layout, state and motion while participating in the+same composition as ordinary Media and Caption. It does not need to become an intermediate video.+The scene's internal implementation can use further functions and parts without making each one a+separate Track. This lets an author expose the useful controls and implement the rest locally.+Film assembles the wanted Tracks into a **Composition**, and rendering turns that composition intothe delivered video. Including performance audio letsit continue under B-roll; a silent covering picture changes only the visible layer.+A component coordinating graphics and sound can publish both contributions; Film selects each+explicitly. Their shared event can preserve synchronization without making audio inclusion implicit.[Tracks](tracks.md) explains the available roles. [Spatial layout](spatial.md) and[fonts and text](fonts-and-text.md) explain their shared inputs. A new role can be implemented as a[project component](track-authoring.md) and included through the same composition model.+## Give each part the direction it can realize++The Agent connects the complete design to the responsibilities of its parts. Treatment retains the+creative reasoning; each production input expresses the choice its recipient should realize.++| Recipient | Useful direction |+| --- | --- |+| Image generation | The intended picture: appearance, style, camera relationship, visible activity, setting and reference facts. |+| Video generation | The performed passage: dialogue, attitude, audible and visible expression, actual interaction, camera behavior and cuts. |+| Voice generation | The intended vocal character and delivery, with the wording and references appropriate to that request. |+| Visual and audio components | The selected materials, content, layout, presentation, mix and authored events they should arrange. |++A planned graphic may affect where the camera places a person. Give the image request that camera+relationship and give the graphic component its own placement and events. This preserves the shared+design through different inputs rather than copying the whole Treatment into each request.++High-level language earns its place through the sensory or behavioral choices it communicates.+Preserve recognizable styling and performable attitudes; resolve an uncertain action or a figurative+idea into the output actually intended. [Image direction](../playbooks/craft/image-direction.md),+[video direction](../playbooks/craft/video-direction.md) and+[voice direction](../playbooks/craft/voice-direction.md) own those concrete judgments. The+[Prompt Kit](prompt-kits.md) preserves useful wording and assembles the chosen direction.+## Source describes the work; Run selects this executionThe Author Source connects the Script, media requests, components and deliverables. Recipes holdreusable authored choices such as Styles or prompt configurations. The connections form a dependencygraph: completing the final video requires the values used to make it.+The author graph describes those dependencies; the run graph incorporates this execution's selected+Targets and Candidates. Visual grouping, paint order and execution dependency remain separate+relationships: one scene can require several material requests, while one material can supply+several appearances.A **Run** selects which public Outputs to complete through **Targets**, and can choose a different**Candidate** for an Output. That Candidate might be a supplied file, a prior Result, or a computation@@ -88,6 +188,10 @@ reuse explicitly.A submitted execution is a **Build**. Its **Result** retains completed public Outputs for inspection,delivery and future selection, including useful Outputs completed before a later failure.+Runtime maintains the running work; Result files keep its products available independently of that+process. If an attempt fails, a new Run can select those products for a new Build. When changing the+video, locate the owning choice—material direction, placement, presentation or execution selection—+and preserve the other relationships that still serve the work.[Source syntax](source-syntax.md) explains declarations and references. [Runs](runs.md) explainsselection and substitutes. [Builds and Results](builds.md) explains submission and retrieval.@@ -106,3 +210,10 @@ relationships while using the execution setup selected for it.[Runtime profiles](../environment/profile.md) explains those choices. [Rendering](rendering.md)explains whole and partial video outputs. [Studio](studio.md) provides interactive inspection andediting of the same authored work with its selected media.++In Studio, a component's Companion makes its authored choices recognizable and editable. Content+and Uses can occupy one Track with an internal Band; independent child objects can have their own+lanes. A Style can supply its own controls through a parameter Companion. Editing a Use changes its+application; editing a shared Style, Frame or Selection changes that shared source and its consumers.+[Companion authoring](studio-companions.md) explains how project components supply these views and+writeback relationships through the same public interfaces.diff --git a/skills/hypit/references/production/timeline.md b/skills/hypit/references/production/timeline.mdnew file mode 100644index 00000000..dcf0dd44--- /dev/null+++ b/skills/hypit/references/production/timeline.md@@ -0,0 +1,65 @@+# Placing Takes on the Timeline++Use this when placing prepared Takes, leaving passages for MG, or creating a work without speech.+[Media preparation](media.md) produces local SemanticTakes; [Script and time](../creation/script-and-time.md)+owns Selection/Moment meaning, and [Tracks](tracks.md) explains presentation.++A Timeline contains the complete time range and the Takes placed within it. Semantic anchors exist+where those Takes supply them. For an event responding to speech, bind to its Selection or Moment;+use authored time for independently directed rhythm. Both belong to this one Timeline.+The project's reference `TIMELINE.md` records observations and their meaning; the production+`time:Timeline` declaration places the new work's material. Reference times locate evidence, while+production placements determine the rendered work.++```svml+<import as="time" from="@hypit/timeline-author@1"/>+<time:Clock id="clock" frame-rate="30"/>+<time:Timeline id="program" clock={clock} end="content.end+2s">+ <time:Take source={opening.take} at="2s"/>+ <time:Take source={explanation.take} at="previous.end+3s"/>+ <time:Take source={closing.take} at="previous.end-12f"/>+</time:Timeline>+```++First omitted `at` means zero; later omitted `at` means the preceding Take's end. Omitted Timeline+`end` means the latest end of all Takes. Ordinary sequential assembly therefore needs only the+Clock and Take references. Material lengths resolve when preparation finishes; generation can run+concurrently while these placement relations wait for their inputs.++`previous.end` refers to the preceding declaration. `content.end` is the maximum of all placed+ends, even when the latest-ending Take is not the last declaration. Absolute positions such as `20s`+work too. `end="30s"` reserves a fixed complete duration; `end="content.end+2s"` reserves a tail.+Reordering Take declarations changes placements that depend on `previous.end`, including omitted+`at` values, and can therefore move later dependent Takes.+Seconds, milliseconds and frames must resolve to exact frames on the Clock. All complete Takes fit+inside the authored extent. Take placement retains native speed and local semantic evidence.++A gap contains no performance source. A wordless Segment instead has actual prepared media, as with+someone dancing. MG can occupy either passage according to the intended picture. Pure MG needs no+placeholder media or Script:++```svml+<time:Timeline id="animation" clock={clock} end="30s"/>+```++The output `.timeline` supplies Film and component contexts through `timeline={program.timeline}`.+Component Surfaces, Fragments and Producers use that same Timeline; shared temporal helpers+project both literal times and Script references against it.+Timeline exports only `.timeline`. [Sound](sound.md) presents its existing audio and supplies an+AudioTrack to Film. Its ordinary Style chooses the last declared active audio source; a project+Style can explicitly mix sources. Material remains available independently of its presentation.++[Performance](performance.md) obtains existing footage from the Timeline. Ordinary Media Items take their own+assets. Caption obtains existing words, while Typography takes authored text. A custom scene can+consume the same placed footage and projected event times. New display Windows preserve the source+frame corresponding to the current Program time; moving a viewport does not restart playback.++Overlapping Takes make both sources available. They do not prescribe a dissolve or choose a picture.+Performance places its sampled layers in declaration order; for a coordinated blend or layout,+use a scene that owns that behavior. Simultaneous Caption Cues retain their spoken intervals; use+Role-based styles or separate Caption presentations when their placement should differ.++Caption also receives a Script-derived CaptionDocument: the document supplies display text and cue+structure, while Timeline supplies placed timing. Independent B-roll and music receive Timeline for+placement and their own assets for content; semantic preparation is needed for a performed Script,+not for every asset used in a composition.diff --git a/skills/hypit/references/production/track-authoring.md b/skills/hypit/references/production/track-authoring.mdindex fac2477e..27f51ba1 100644--- a/skills/hypit/references/production/track-authoring.md+++ b/skills/hypit/references/production/track-authoring.md@@ -24,7 +24,7 @@ the component more usefully than a long list of decorative parameters.This existing answer-strip vocabulary illustrates a concrete choice:```svml-<emoji:Track id="answers" semantic={speech.semantic} canvas={canvas}+<emoji:Track id="answers" timeline={speech.timeline} canvas={canvas}style={answer-style} placeholder={question-icon} during="program"><emoji:Item id="known" icon={known-icon} preset="true"/><emoji:Item id="surprise" icon={answer-icon} at={story.moment.reveal}/>@@ -37,15 +37,15 @@ repeated updates or simultaneous events.## Keep selection, projection and consumption distinct-Script names meaning: a Selection, Segment or Moment. The semantic timeline supplies where that-meaning occurred in the produced media. The consuming component's Surface lowers its authored timing+Script names meaning: a Selection, Segment or Moment. The Timeline supplies where that+event falls after its prepared media is placed. The consuming component's Surface lowers its authored timingform into a projection subgraph, which resolves an Instant or Window in the program's frame space.The component's own Fragment and Producers then decide what to do with that value.```text-Script Selection / Segment / Moment + SemanticTrack+Script Selection / Segment / Moment + Timeline↓ component Surface creates projection- ProgramSpace + Window / Instant+ Timeline + Window / Instant↓ component Fragment / Producer consumptionmedia occupancy, graphic state, sound or effect```@@ -54,11 +54,11 @@ This separation lets one Moment drive an answer reveal, a short sound and a flasreceives its own appropriate projection, but all refer to the same authored event. Changed speechmoves the event without asking each component to search for a word or inspect the Script parser.-For a component that also serves authored animation, `resolveTemporalContext` and `createTemporalSpace`-from `@hypit/hypit/temporal-markup` supply the chosen film clock. Clock expressions project directly through-that space; Script references use its semantic context. The drawing code consumes the resolved-values in either case. The installed `examples/semantic-composition/packages/chat-scene` demonstrates-repeated content children with their own events and a program drawn entirely in code.+Use `resolveTemporalContext` from `@hypit/hypit/temporal-markup` to resolve the component's+`timeline` input. Pass that same Timeline to its Fragment and Producers, alongside projected+Instants/Windows. A component can combine literal times and word-bound events without switching time+models. The installed `examples/semantic-composition/packages/chat-scene` demonstrates repeated+content children with their own events and a picture drawn entirely in code.For a Window consumer, `during={story.selection.example}` can take both semantic boundaries.`at={story.moment.answer} for="8f"` starts a short effect at a Moment. An Instant consumer may accept@@ -85,31 +85,22 @@ The interview's project emoji strip illustrates this design: one outer Window, oitems, a placeholder asset and one icon per answer. Each Moment changes its own slot. The Track doesnot decide when the spoken answer occurred; it consumes the already projected event.-## Author an alternative performance assembly+## Use the Timeline's existing placement model-The installed Speech Track expresses one particular and useful assembly: every Segment-local-SemanticTake keeps its complete span, and Source order forms one continuous prefix-sum-`SemanticTrack`. Its original audio follows that assembly. Visual components consume its prepared materials.+The [Timeline](timeline.md) places intact prepared Takes at native speed. Sequential placement,+head/interior/tail space and simultaneous Takes share the same data type. A project scene consumes+its range, media samples and semantic events through ordinary projection helpers.-A work may intentionally need interruptions, overlapping dialogue, a time-consuming transition or-another performance-time relationship. That is a legitimate project Track when it makes the edited-media, semantic positions, visual projection and audio ownership explicit. If its output still is the-standard `SemanticTrack`, it must satisfy that type's continuous item semantics so existing consumers-can trust it. A genuinely different timeline meaning belongs to a project-owned type with matching-projection Surfaces or an explicit adapter to the standard type. Source then selects that component-normally; neither Core nor the default Speech Track needs a special case.--Design the author Surface from the intended relationship rather than exposing low-level overlap-arithmetic by default. The implementation can accept prepared performances, edit or combine them at-its declared boundary, and publish only the outputs it can define unambiguously. Caption, MG, Media-and Effects receive its declared semantic projections rather than inferring overlap from picture;-sound routing makes deliberate simultaneous speech visible rather than accidental.+A different material operation, such as trimming or retiming a performance, changes the local media+and its preparation. A display operation, such as crossfading two available sources, belongs to the+visual scope owning them. Keep these decisions distinct so a layout change preserves source timing+and a material change establishes truthful local evidence.## Make spatial decisions equally explicitCanvasSpace identifies the coordinate system. SpatialFrame, anchors, extents and fitting describewhere content goes and how its intrinsic shape occupies that place. RegionTimeline supplies authored-per-frame regions in a chosen Canvas. These are distinct from ProgramSpace, which owns the clock.+per-frame regions in a chosen Canvas. These are distinct from Timeline, which owns the complete time range.Keep image dimensions, placement and crop transformations visible so a measured head or a reservedMG area maps into the actual composition. Do not hardcode a global 9:16 canvas, fixed speaker side,@@ -132,7 +123,7 @@ that answers the current question. A new component may use fewer operations or dThe Surface exposes author intent and lowers its supported temporal forms through shared projectionhelpers such as `createTemporalWindowProjection` and `createTemporalInstantProjection`. The Fragment-wires those projections, explicit ProgramSpace/Canvas and authored values into the component's+wires those projections, the shared Timeline/Canvas and authored values into the component'sProducers. The Producer consumes the resulting time and geometry and owns the visual/state behavior;Studio consumes that published meaning for presentation and editing.@@ -154,13 +145,12 @@ The useful pieces of a project package are:| Vocabulary and preview | Explain the role and show a recognizable, configured example || Optional Studio Companion | Project meaningful editor entities, real parameter bindings and temporal lineage without changing video rendering |-Resolve the Track's `semantic` or `space` through `resolveTemporalContext`, then use-`createTemporalSpace` once to obtain the shared ProgramSpace. A spoken composition normally supplies-`semantic={speech.semantic}`; an authored animation can supply `space={animation}`. This choice-belongs to the Surface. The visual Producer consumes space and projected time, not Script syntax.+Resolve the Track's `timeline` through `resolveTemporalContext` and wire `context.timeline.ref`+directly to its Timeline input. Words and source footage are available where prepared Takes provide+them. The visual Producer consumes Timeline and projected times, not Script syntax.For each Window, call `createTemporalWindowProjection`; for an event, use-`createTemporalInstantProjection`. Pass the resolved context, the shared `space`, the child element+`createTemporalInstantProjection`. Pass the resolved context, the child elementand `resolveReference`. Preserve all returned `records`, `components` and `fragments`, and wire thereturned `ref` into the domain Fragment. The common `at` accepts a Moment or an authored durationfrom program start. Selection and Segment boundaries use `boundary="start"` or `boundary="end"`.@@ -169,7 +159,7 @@ A Moment needs the semantic context that locates its Script anchor.Keep each child's `subjectId` meaningful to the component while qualifying graph ids by its owningTrack, so multiple instances can coexist. A finite create/append/finalize graph supports any authorednumber of messages or cards with ordinary fixed Producer ports. The exact helpers and vocabulary-live in `@hypit/hypit/temporal-markup`; the `@example/chat-scene` package demonstrates both time sources.+live in `@hypit/hypit/temporal-markup`; the `@example/chat-scene` package demonstrates authored and semantic events on one Timeline.A scene may publish computed event times when another component needs them, just as it publishes aTrack. This shares pre-render data. When the author already specifies a common trigger, consumers@@ -206,3 +196,12 @@ A designed still alone cannot establish that behavior. For stateful MG, a few mechecks around activation and exit are more useful than snapshots of every implementation detail.Also check a second instance, a differently sized Frame and direct seeking into the middle whenthose exercise the new behavior.++## A reusable performance presentation++When the behavior presents existing Timeline footage, [Performance Styles](performance.md) offer+an occurrence model with broad and local Uses. A Style binds a typed fragment's additional inputs;+each Use supplies Timeline, Canvas and its original Window. The fragment exports a VisualTrack.+This supports project-owned HTML behavior and explicit extra events or material without introducing+a new Track protocol. A larger scene that coordinates footage and independent graphics can remain+its own component. Both paths use the same rendering capabilities.diff --git a/skills/hypit/references/production/tracks.md b/skills/hypit/references/production/tracks.mdindex 4d7d45dd..2e61a7ae 100644--- a/skills/hypit/references/production/tracks.md+++ b/skills/hypit/references/production/tracks.md@@ -11,10 +11,11 @@ implementation, read [Track authoring](track-authoring.md); for a new Caption fa| Role | Owns | Common published values || --- | --- | --- |-| SemanticTrack | The ordered media timeline and aligned semantic locations; it draws nothing | Usually `speech.semantic` |-| Speech Track | Ordered semantic performances and their original sound | `.semantic`, `.audio` |-| Media Track | Framed images, semantic performances, prepared moving media, or authored surfaces; B-roll is one use | `.program`, `.visual`, optional `.audio` |-| Audio Track | Independently timed music, ambience and effects | `.program`, `.track` |+| Timeline | Complete time range and placed Takes, with any semantic evidence; it draws nothing | `.timeline` |+| Performance | Existing Timeline footage, with broad and local presentation Uses | `.visual` |+| Media Track | Independently supplied images, prepared moving media, or authored surfaces; B-roll is one use | `.program`, `.visual`, optional `.audio` |+| Sound | Existing Timeline audio, with broad and local presentation Uses | `.audio` |+| Audio Track | Independently timed music, ambience and effects | `.program`, `.audio` || Caption | Script-derived display text presented with speech | Fine publishes `.schedule`, `.track` || Typography / Text | Titles, labels and other independently authored text | `.program`, `.track` || Semantic MG | A board, comparison, reveal or another system responding to meaning | Package-owned; often `.program` and a visual Track |@@ -35,9 +36,9 @@ exposes an appropriate projection form on its own Surface:```Here `during` belongs to `media-track:Item`. While lowering that element, Media Track's Surface-resolves the Selection and SemanticTrack, creates the temporal projection Records and Fragment, and+resolves the Selection and Timeline, creates the temporal projection Records and Fragment, andwires the projected `Window` into Media Track's own Fragment. The projection subgraph resolves the-Selection's start and end Anchors against the selected SemanticTrack and composes the resulting+Selection's start and end Anchors against the selected Timeline and composes the resultingInstants into that Window.The Media Track's internal Producers consume the Window together with the media, Frame and authored@@ -52,7 +53,7 @@ projection forms its behavior can honestly consume.```textScript Selection / Moment / Segment- ↓ component Surface projection through SemanticTrack+ ↓ component Surface projection through TimelineWindow / Instant↓ component Fragment and Producersoccupancy, playback, state, sound or visual behavior@@ -63,26 +64,17 @@ inspect or deliver and changes none of these relationships. [Script and time](..owns Selection, Moment and shared projection spellings; each selected Surface's vocabulary says whichforms that component exposes.-## Semantic Takes supply the media timeline+## Place prepared Takes on the Timeline-Script Segments have authored order and meaning but no program positions. Each spoken Segment is-associated with the local media performance carrying it; normalization establishes that media's-frame envelope, and alignment locates the Segment's words and anchors inside it. That self-contained-value is a SemanticTake. Speech Track places the Takes in Source order and translates every local-position by the lengths before it. The resulting SemanticTrack is the program's semantic time.+Each SemanticTake holds local prepared media and the Script Segment's local word/boundary positions.+[Timeline authoring](timeline.md) places those Takes: sequentially by default, or with authored+starts that leave gaps or overlap. The complete Timeline can extend before and after all Takes.+A wholly authored animation uses the same Timeline with zero Takes and an explicit end.-```text-Segment A + its local media → SemanticTake A ┐-Segment B + its local media → SemanticTake B ├─ Source order → SemanticTrack-Segment C + its local media → SemanticTake C ┘-```--Speech Track is therefore not placing A-roll on an independently maintained master timeline. The-ordered Takes create the timeline that Caption, MG, Media, Typography, Audio and Effects project-through. The official Speech Track uses each complete local media span and joins the next Take at its-end; it creates no gap or duration-consuming overlap. An edit that trims or retimes a performance-belongs before its alignment. A visual handoff that does not change the performance media remains an-ordinary visual composition on the resulting program time.+Caption, MG, Media, Typography, Audio and Effects share `timeline={program.timeline}`. Word-related+events retain Selection/Moment bindings; independent events can use authored positions. Source+overlap provides simultaneous content. Its visible presentation or crossfade belongs to the visual+component, not assembly. Original audio follows the same placements and remains silent in gaps.For the usual spoken production, the performance carrying each Segment is the A-roll. In a podcast,the participants' turns belong to that shared performance timeline; each visible speaker does not@@ -92,40 +84,39 @@ assembled work. [Voice and performance](../playbooks/craft/voice-and-performancebetween visible A-roll, covered visible A-roll and audio-only A-roll.A-roll can be a circular presenter inset or a moving cutout over a demonstration. Its role is-semantic: it carries the Script. Speech Track assembles its time and original sound; the visual-component decides how to present its prepared material.+semantic: it carries the Script. Timeline assembly places its prepared material and semantic evidence; Performance and Sound+independently decide how its pictures and audio are presented.```svml-<speech:Track id="speech">- <speech:Take source={opening-take}/>- <speech:Take source={answer-take}/>-</speech:Track>-<media-track:Track id="performance" semantic={speech.semantic} canvas={canvas}>- <media-track:Performance during="program" frame={layout.full}- appearance={look.performance}/>-</media-track:Track>+<time:Clock id="clock" frame-rate="30"/>+<time:Timeline id="speech" clock={clock}>+ <time:Take source={opening-take}/>+ <time:Take source={answer-take}/>+</time:Timeline>+<sound:Style id="voice-style"/>+<sound:Track id="voice" timeline={speech.timeline}>+ <sound:Use style={voice-style}/>+</sound:Track>+<performance:Style id="performance-style" frame={layout.full} appearance={look.performance}/>+ <performance:Track id="performance" timeline={speech.timeline} canvas={canvas}>+ <performance:Use style={performance-style} during="program"/>+ </performance:Track>```The imports, Canvas, Frame, Recipe and prepared SemanticTakes exist in this excerpt. The MediaRecipe specifies `stack-order`, fitting and frame presentation. Select `performance.visual` and-`speech.audio` in Film. The material is generated once and remains the same performance.--The Track's `semantic` supplies the performance and the time context for all its children.-`Performance` displays that performance's picture; `Item` places an independently supplied image,-media source or layered composition. Both use `during` to choose their display interval.+`voice.audio` from the Sound Track in Film. The material is generated once and remains the same performance.-`Performance during="program"` shows all the Takes in order under one frame and lifecycle.-Use `during={story.segment.explanation}` or a Selection to show only the corresponding passage.-If a Take begins at program second 5, a Window from seconds 7 to 10 shows source seconds 2 to 5;-the source offset preserves synchronization with `speech.audio`. Media fitting, clipping, decoration,-`motion` and `Sampling` control presentation. Use an Item with direct `media` for independent-playback or source trim.+[Performance](performance.md) owns broad and local presentation Uses. Its Timeline supplies source+positions; an interval from program seconds 7 to 10 samples source seconds 2 to 5 when that Take+starts at second 5. Ordinary Styles reuse Media geometry and appearance; project Styles can define+movement and mixing. Media Items retain independent playback and source trim.A moving performance viewport and its neighboring diagram may belong in one component because theyshare a layout change. That component consumes prepared material and projected Moments or Windows,then draws their shared state. Independent Caption and coverage remain separate when useful.Its Surface can expose a Scene, Item or another useful authoring unit. Inside the component,-`projectSemanticMedia` supplies the intersecting prepared Takes, their program spans and their source+`projectTimelineMedia` supplies the intersecting prepared Takes, their program spans and their sourceoffsets, so changing the layout preserves playback alignment.[Component design](component-design.md) explains where to draw these boundaries;[drawing a component](component-visuals.md#compose-video-and-graphics-in-one-browser-program) explains@@ -133,18 +124,18 @@ HTML, CSS and frame-driven code inside one visual contribution.Semantic structure also supports a wordless passage. It has no speaking A-roll, yet a named emptyScript Segment receives its actual boundaries from prepared media and remains addressable through-the assembled SemanticTrack. Semantic identity here means that the work can refer to the passage and+the assembled Timeline. Semantic identity here means that the work can refer to the passage andits boundaries; it does not classify the passage or require spoken words.[Media preparation](media.md#empty-segments-use-their-media-boundaries) owns that construction.-Every Take contributes its Segment boundaries to the assembled semantic time. When its prepared-media has no audio stream, Speech Track emits no audio clip for that Take. A visual consumer skips-an audio-only Take while preserving its place in semantic time.+Every Take contributes its placed Segment boundaries to the Timeline. When its prepared+media has no audio stream, Sound has no audio candidate from that Take. A visual consumer skips+the missing source picture of an audio-only Take while its time and speech remain available.## Coverage has a Window and a separate playback choice```svml-<media-track:Track id="coverage" semantic={speech.semantic} canvas={canvas}>+<media-track:Track id="coverage" timeline={speech.timeline} canvas={canvas}><media-track:Item image={portrait.image} extent={portrait-extent}frame={layout.full} during={story.selection.example} appearance={look.still}/><media-track:Item media={prepared-broll.media}@@ -184,12 +175,19 @@ the same speech twice. A silent covering image does not mute the performance bel## Sound, text and Caption answer different events-An Audio Clip takes normalized audio-bearing media. `gain` is a linear multiplier; fades and trim+[Sound](sound.md) presents audio already placed on the Timeline, including local silence, gain+changes and explicit source blends. Audio Track supplies independent sources.++An Audio Item takes normalized audio-bearing media. `gain` is a linear multiplier; fades and trimare explicit. Normalizing input media does not automatically balance or duck the complete mix.+Like Media Track, Audio Track places independent Items and accepts the shared timing forms.+Its `.audio` output connects to Film. Overlapping Audio Items mix; visual Items compose by draw+order. An explicit Item `id` gives Studio a recognizable name; unnamed Items display their source+reference while retaining independent timing and editing identities.```svml-<audio:Track id="effects" semantic={speech.semantic}>- <audio:Clip source={reveal-sound.media} at={story.moment.reveal} for="600ms"+<audio:Track id="effects" timeline={speech.timeline}>+ <audio:Item source={reveal-sound.media} at={story.moment.reveal} for="600ms"playback="once" gain="0.45" fade-in="0f" fade-out="3f"/></audio:Track>```@@ -201,8 +199,8 @@ valid for events independent of speech even in a speaking video.Typography uses exact fonts, its own Style and Point/Area/Path placement. Graph Text and rich inlinecontent are alternatives. A rich Span Style replaces that run's typography rather than cascadingCSS properties. A headline held over several sentences is generally Typography. Caption instead-reads `story.caption`, applies a Caption Program and joins to `speech.semantic`; Role overrides,-Selection overrides and authored `||` breaks retain Script as the wording owner.+joins `story.caption` to `speech.timeline`; timed Uses choose presentation while authored `||`+breaks retain Script as the content-grouping owner.[Fonts and text](fonts-and-text.md) covers exact faces, fallbacks, Emoji and Typography placement.@@ -210,8 +208,7 @@ Selection overrides and authored `||` breaks retain Script as the wording owner.`@hypit/caption-fine` is the usual fine-grained Caption renderer. Its Recipe exposes placement,font and size, glyph and box Paint, wrapping, active-word coloring, Cue and token motion, lead/tail-and handoff. Script supplies display wording, Roles and `||` Cue breaks; `@hypit/caption` applies-Styles and joins that document to the actual semantic timing. Fine publishes a visible `.schedule`+and handoff. Script supplies display wording, Roles and `||` Cue breaks; `@hypit/caption` joins that document to the actual semantic timing. Fine publishes a visible `.schedule`and a rendered `.track`.Use those controls for the appearance and rhythm they express. When the design needs a different@@ -222,8 +219,7 @@ can be chosen directly for a new visual role; making one is ordinary productionFor the selected family's exact attributes and Recipes, use `hypit vocabulary @hypit/caption-fine`and its installed README. [Caption craft](../playbooks/craft/captions.md) explains reading rhythm;-[Caption styling and coverage](caption-program.md) explains Role and Selection overrides, whole-Segments, word-specific styles and Mute;+[Caption styling and coverage](caption-presentation.md) explains timed Uses, Role filters, mid-Cue Style changes and hiding;[Caption tracking](../playbooks/craft/caption-tracking.md) explains measured moving placement.## Persistent MG is more than another timed image@@ -245,9 +241,9 @@ uses that same separation when designing a new semantic component.## Assemble the intended peers-Film's domain assembly receives a Canvas, a ProgramSpace and a Film Recipe. The current `film:Film`-Surface takes `canvas`, `appearance` and a time source: `semantic` derives the space from the-performance; `space` selects an authored ProgramSpace. Pure MG can consist entirely of visual+Film's domain assembly receives a Canvas, a Timeline and a Film Recipe. The current `film:Film`+Surface takes `canvas`, `appearance` and `timeline={program.timeline}`. Its Timeline supplies the+complete range and any semantic evidence. Pure MG can consist entirely of visualcomponents. [Rendering](rendering.md#compose-an-authored-animation) shows the complete time setup.Include each desired visual and audiooutput explicitly. Listing a Track later does not move it to the front: absolute stacking is authoreddiff --git a/test/support/video-domain.ts b/test/support/video-domain.tsindex 5ed82987..2f193a81 100644--- a/test/support/video-domain.ts+++ b/test/support/video-domain.ts@@ -3,7 +3,7 @@ import { compositionComponent, compositionManifest } from "@hypit/composition";import { mediaComponent, mediaManifest } from "@hypit/media";import { narrativeManifest } from "@hypit/narrative";import { programSpaceManifest } from "@hypit/program-space";-import { semanticTrackManifest } from "@hypit/semantic-track";+import { timelineManifest } from "@hypit/timeline";import { spatialComponent, spatialManifest } from "@hypit/spatial";import { speechManifest } from "@hypit/speech";import { speechEvidenceManifest } from "@hypit/speech-evidence";@@ -20,7 +20,7 @@ export const videoContractManifests = [speechManifest,speechEvidenceManifest,svsManifest,- semanticTrackManifest,+ timelineManifest,spatialManifest,temporalManifest,visualIrManifest,diff --git a/test/temporal-fixture.ts b/test/temporal-fixture.tsindex f673e37b..6e52ce26 100644--- a/test/temporal-fixture.ts+++ b/test/temporal-fixture.ts@@ -1,6 +1,6 @@-import { projectSemanticProgramSpace } from "@hypit/semantic-track";+import { projectTimelineSpace } from "@hypit/timeline";import type { NarrativeExcerpt, NarrativeMomentRef, NarrativeSelectionRef } from "@hypit/narrative";-import type { SemanticTrack } from "@hypit/semantic-track";+import type { Timeline } from "@hypit/timeline";import {composeTemporalWindow,projectMomentInstant,@@ -20,69 +20,69 @@ const fixed = { kind: "fixed" as const };export function projectProgramInstantFixture(input: {readonly itemId: string;readonly subjectId?: string;- readonly semantic: SemanticTrack;+ readonly semantic: Timeline;readonly projection: TemporalInstantExpression;}) {- return projectProgramInstant({ ...input, space: projectSemanticProgramSpace(input.semantic), subjectId: input.subjectId ?? input.itemId, authority: fixed });+ return projectProgramInstant({ ...input, timeline: input.semantic, subjectId: input.subjectId ?? input.itemId, authority: fixed });}export function projectMomentInstantFixture(input: {readonly itemId: string;readonly subjectId?: string;- readonly semantic: SemanticTrack;+ readonly semantic: Timeline;readonly moment: NarrativeMomentRef;readonly projection: TemporalInstantExpression;}) {- return projectMomentInstant({ ...input, subjectId: input.subjectId ?? input.itemId, authority: fixed });+ return projectMomentInstant({ ...input, timeline: input.semantic, subjectId: input.subjectId ?? input.itemId, authority: fixed });}export function projectProgramWindow(input: {readonly itemId: string;readonly subjectId?: string;- readonly semantic: SemanticTrack;+ readonly semantic: Timeline;readonly projection: TemporalWindowProjection;}) {const subjectId = input.subjectId ?? input.itemId;return composeTemporalWindow({ id: input.itemId, subjectId },- projectProgramInstant({ itemId: `${input.itemId}.start`, subjectId, space: projectSemanticProgramSpace(input.semantic), projection: input.projection.start, authority: fixed }),- projectProgramInstant({ itemId: `${input.itemId}.end`, subjectId, space: projectSemanticProgramSpace(input.semantic), projection: input.projection.end, authority: fixed }));+ projectProgramInstant({ itemId: `${input.itemId}.start`, subjectId, timeline: input.semantic, projection: input.projection.start, authority: fixed }),+ projectProgramInstant({ itemId: `${input.itemId}.end`, subjectId, timeline: input.semantic, projection: input.projection.end, authority: fixed }));}export function projectSelectionWindow(input: {readonly itemId: string;readonly subjectId?: string;- readonly semantic: SemanticTrack;+ readonly semantic: Timeline;readonly selection: NarrativeSelectionRef;readonly projection: TemporalWindowProjection;}) {const subjectId = input.subjectId ?? input.itemId;return composeTemporalWindow({ id: input.itemId, subjectId },- projectSelectionInstant({ itemId: `${input.itemId}.start`, subjectId, semantic: input.semantic, selection: input.selection, projection: input.projection.start, authority: fixed }),- projectSelectionInstant({ itemId: `${input.itemId}.end`, subjectId, semantic: input.semantic, selection: input.selection, projection: input.projection.end, authority: fixed }));+ projectSelectionInstant({ itemId: `${input.itemId}.start`, subjectId, timeline: input.semantic, selection: input.selection, projection: input.projection.start, authority: fixed }),+ projectSelectionInstant({ itemId: `${input.itemId}.end`, subjectId, timeline: input.semantic, selection: input.selection, projection: input.projection.end, authority: fixed }));}export function projectSegmentWindow(input: {readonly itemId: string;readonly subjectId?: string;- readonly semantic: SemanticTrack;+ readonly semantic: Timeline;readonly segment: NarrativeExcerpt;readonly projection: TemporalWindowProjection;}) {const subjectId = input.subjectId ?? input.itemId;return composeTemporalWindow({ id: input.itemId, subjectId },- projectSegmentInstant({ itemId: `${input.itemId}.start`, subjectId, semantic: input.semantic, segment: input.segment, projection: input.projection.start, authority: fixed }),- projectSegmentInstant({ itemId: `${input.itemId}.end`, subjectId, semantic: input.semantic, segment: input.segment, projection: input.projection.end, authority: fixed }));+ projectSegmentInstant({ itemId: `${input.itemId}.start`, subjectId, timeline: input.semantic, segment: input.segment, projection: input.projection.start, authority: fixed }),+ projectSegmentInstant({ itemId: `${input.itemId}.end`, subjectId, timeline: input.semantic, segment: input.segment, projection: input.projection.end, authority: fixed }));}export function projectMomentWindow(input: {readonly itemId: string;readonly subjectId?: string;- readonly semantic: SemanticTrack;+ readonly semantic: Timeline;readonly moment: NarrativeMomentRef;readonly projection: TemporalWindowProjection;}) {const subjectId = input.subjectId ?? input.itemId;return composeTemporalWindow({ id: input.itemId, subjectId },- projectMomentInstant({ itemId: `${input.itemId}.start`, subjectId, semantic: input.semantic, moment: input.moment, projection: input.projection.start, authority: fixed }),- projectMomentInstant({ itemId: `${input.itemId}.end`, subjectId, semantic: input.semantic, moment: input.moment, projection: input.projection.end, authority: fixed }));+ projectMomentInstant({ itemId: `${input.itemId}.start`, subjectId, timeline: input.semantic, moment: input.moment, projection: input.projection.start, authority: fixed }),+ projectMomentInstant({ itemId: `${input.itemId}.end`, subjectId, timeline: input.semantic, moment: input.moment, projection: input.projection.end, authority: fixed }));}diff --git a/test/semantic-track-fixture.ts b/test/timeline-fixture.tssimilarity index 82%rename from test/semantic-track-fixture.tsrename to test/timeline-fixture.tsindex 075ee84c..13288be5 100644--- a/test/semantic-track-fixture.ts+++ b/test/timeline-fixture.ts@@ -1,5 +1,5 @@import type { ProgramSpace } from "@hypit/program-space";-import type { SemanticTrack } from "@hypit/semantic-track";+import type { Timeline } from "@hypit/timeline";type FixtureSegment = {readonly id: string;@@ -13,13 +13,13 @@ type FixtureAnchor = {const audioArtifact = {kind: "blob" as const,- resource: "res_semantic-track-fixture" as const,+ resource: "res_timeline-fixture" as const,size: 1,mediaType: "audio/wav",};-/** Minimal real SemanticTrack for tests whose subject is timing rather than speech recognition. */-export function semanticTrackFixture(+/** Minimal real Timeline for tests whose subject is timing rather than speech recognition. */+export function timelineFixture(space: ProgramSpace,options: {readonly id?: string;@@ -27,11 +27,11 @@ export function semanticTrackFixture(readonly segments?: readonly FixtureSegment[];readonly anchors?: readonly FixtureAnchor[];} = {},-): SemanticTrack {+): Timeline & { readonly narrativeId: string } {const totalFrames = Math.round(space.durationSec * space.frameRate.numerator / space.frameRate.denominator);const segments = options.segments ?? [{ id: "fixture", frameCount: totalFrames }];if (segments.reduce((sum, segment) => sum + segment.frameCount, 0) !== totalFrames) {- throw new Error("SemanticTrack fixture segments must fill ProgramSpace exactly.");+ throw new Error("Timeline fixture segments must fill ProgramSpace exactly.");}const anchors = options.anchors ?? [];const structuralAnchorIds = new Set(segments.flatMap((segment) => [@@ -40,7 +40,8 @@ export function semanticTrackFixture(]));let startFrame = 0;return {- // A SemanticTrack is the author-visible identity of the ProgramSpace it projects.+ // A Timeline is the author-visible identity of the ProgramSpace it projects.+ ...space,id: options.id ?? space.id,narrativeId: options.narrativeId ?? "script",items: segments.map((segment, index) => {@@ -72,8 +73,9 @@ export function semanticTrackFixture(...localAnchors.filter((anchor) => anchor.identity !== startAnchorId && anchor.identity !== endAnchorId),],};+ const placement = { take, startFrame };startFrame = endFrameExclusive;- return { take };+ return placement;}),};}