Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

feat(composition): support authored clocks alongside semantic timelines

kashorine90b070c339c
diff --git a/docs/guide/studio-temporal-windows.md b/docs/guide/studio-temporal-windows.md
index 0a192add..11da8501 100644
--- a/docs/guide/studio-temporal-windows.md
+++ b/docs/guide/studio-temporal-windows.md
@@ -8,7 +8,7 @@ description: How author choice becomes an Instant or Window and returns to its r
Time has three layers:
```text
-semantic choice (Program / Selection / Segment / Moment)
+author choice (Selection / Segment / Moment / program time)
-> projection (TemporalInstant / TemporalWindow)
-> domain consumption (Program / Schedule / Track)
```
@@ -20,7 +20,8 @@ This does not make `during="program"` writable: that projection remains structur
`@hypit/temporal-markup` owns the SVML author forms and lowers them before a domain component runs.
The compiler only composes records, components and fragments; it does not recognize `during`, `at`
-or any time grammar. The graph also projects `ProgramSpace` through the public Semantic Track producer,
+or any time grammar. A declared `ProgramSpace` supplies authored animation time; for a performance,
+the graph projects it through the public Semantic Track producer,
so domain components consume only ProgramSpace and projected time values and remain independent from
both semantic lookup and Studio.
@@ -33,10 +34,10 @@ The identity chain is author-visible rather than synthesized by Studio:
- `<script id="story">` becomes `Narrative.id = story`; every Selection, Moment, Segment excerpt
and CaptionDocument from that Script carries `narrativeId = story`;
-- the selected Semantic Track id becomes `ProgramSpace.id`; every terminal Track carries that exact
+- the selected Semantic Track id, or the declared Space id, becomes `ProgramSpace.id`; every terminal Track carries that exact
`programSpaceId`;
-- every projected Instant/Window carries both identities plus the consumer's public domain identity
- (normally its SVML `id`) as `subjectId`.
+- every projected Instant carries the space identity; semantic sources also carry the Narrative
+ identity. Each includes the consumer's public domain identity (normally its SVML `id`) as `subjectId`.
The projection record `id` may be qualified to stay unique inside an expanded graph; it is not the
author identity. `subjectId` is kept separate and remains the exact board, card, item or sequence
@@ -60,10 +61,11 @@ Out-of-range Instants and reversed or empty Windows are rejected, not clipped or
| `during={Segment}` / `during="program"` | fixed | fixed | read-only |
| `at={Moment} for="…"` | semantic Moment cue | parameter `for` | Script and/or SVML atomically |
| `until={Moment} for="…"` | parameter `for` | semantic Moment cue | Script and/or SVML atomically |
+| `at="2s" for="…"` | parameter `at` | parameter `for` | SVML timing attributes |
| `start="…" end="…"` | parameter `start` | parameter `end` | SVML timing attributes |
-A point consumer uses `at={Moment}` (or a chosen Selection boundary) for semantic authority. The
-explicit fallback is `instant="…"`. Text such as `at="moment.cue+3f"` is rejected because it hides
+A point consumer uses `at={Moment}` (or a chosen Selection boundary) for semantic authority, or
+`at="2s"` for authored time. `instant="…"` accepts a projected expression. Text such as `at="moment.cue+3f"` is rejected because it hides
whether the author intended to move the Moment or the offset.
## Studio inverse
@@ -85,9 +87,10 @@ not declare common timeline inverse functions. A new component becomes time-edit
the same public Instant/Window protocol and carrying its executed lineage to its Studio entity.
Film and Script interpretation also remain outside Studio core. `film-studio` declares which Film
-reference is the semantic axis and which child references are terminal Tracks; `script-studio` owns
+reference supplies the time axis and which child references are terminal Tracks; `script-studio` owns
Script source observation and marker relocation. Studio selects the Script source map whose
-`narrativeId` exactly matches the active ProgramSpace and delegates the edit back to that companion.
+`narrativeId` exactly matches the active SemanticTrack and delegates the edit back to that companion.
+An authored Space supplies the physical timeline without a Script lane.
The binding name is not used as a global address. Markup retains the exact author element and input
range while decoding, Elaborator hygienizes that endpoint with its Source unit, and the compiled
diff --git a/docs/quickstart/composition.md b/docs/quickstart/composition.md
index 00e7aa98..9d5614c3 100644
--- a/docs/quickstart/composition.md
+++ b/docs/quickstart/composition.md
@@ -16,9 +16,8 @@ Composition. The renderer then compiles that Composition into an MP4 video.
## film:Film
-Assembles all Tracks into a single Composition. Film itself has no domain knowledge — it does not
-know what a caption is, what Media is, or what speech is. It takes any VisualTrack or AudioTrack
-and layers them by stacking order.
+Assembles selected VisualTracks and AudioTracks into a single Composition. Each visual contribution
+retains its own timed appearances and paint order; sound is included through the selected AudioTracks.
```svml
<space:Canvas id="vertical" width="1080" height="1920"/>
@@ -35,7 +34,8 @@ and layers them by stacking order.
|---|---|---|
| `id` | yes | Unique identifier |
| `canvas` | yes | Explicit CanvasSpace shared with Track layout |
-| `semantic` | yes | SemanticTrack from `speech:Track` — defines duration and frame rate |
+| `semantic` | one time source | Selected performance's SemanticTrack; supplies duration and frame rate |
+| `space` | one time source | Authored ProgramSpace; use either `semantic` or `space` |
| `appearance` | yes | SVS Film Recipe — the canvas clear color |
### film:Track
@@ -58,9 +58,9 @@ Common Track sources:
### Track stacking
-Tracks are **flat** — there is no nesting or grouping. Z-ordering is determined entirely by the
-`stack-order` property in each Track's SVS Recipe. Lower values go behind; higher values render on
-top.
+Film collects peer Tracks. Each Track can contain several independently timed and ordered
+appearances, called Presents. Many components expose that paint order as `stack-order` in their
+Recipe: lower values paint behind higher values. Reordering Film children leaves this order intact.
Typical stacking order:
@@ -71,9 +71,11 @@ Typical stacking order:
| 70 | Captions |
| 90 | Text overlays |
-One component can emit multiple visual elements (Presents) at different z-positions, which
-interleave with other components' Presents. The final render flattens all Presents, sorts by
-absolute stacking key, and paints them onto one canvas.
+Presents from different components can interleave. Each Present also owns an internal element tree:
+several videos, text and graphics can share layout, masks or coordinated motion. A project component
+can use an HTML/CSS browser program for such a scene. Independent Caption or coverage can remain
+peer contributions. Group the content whose behavior belongs together; its size and media type do
+not prescribe that boundary.
**Output:** `{main.composition}` — the complete Composition, passed to the renderer.
@@ -89,7 +91,8 @@ Compiles the Composition into a finished video via the HyperFrames renderer.
|---|---|---|
| `id` | yes | Unique identifier |
| `composition` | yes | Composition from `film:Film` |
-| `semantic` | yes | SemanticTrack from `speech:Track` |
+| `semantic` | one time source | The composition's selected SemanticTrack |
+| `space` | one time source | The composition's authored ProgramSpace; use either `semantic` or `space` |
The renderer:
@@ -103,6 +106,27 @@ The renderer:
This is the most common Build Target, and it can also be connected directly to later Blob consumers
such as media trimming, audio/frame extraction or a model reference input.
+## A film drawn entirely by components
+
+For spoken work, Script Selections and Moments preserve the relationship between the words and
+their presentation. A chat animation or diagram can instead direct its own reading rhythm. Declare
+the film clock and use it in the scene, Film and Render:
+
+```svml
+<import as="time" from="@hypit/program-space@1"/>
+<time:Space id="animation" frame-rate="30" duration="8s"/>
+<!-- scene.track is produced by a component using this same clock. -->
+<film:Film id="main" canvas={canvas} space={animation} appearance={recipes.film.main}>
+ <film:Track source={scene.track}/>
+</film:Film>
+<render:Video id="final" composition={main.composition} space={animation}/>
+```
+
+The scene's events can use authored seconds or frames. In a spoken composition, the same behavior
+can follow projected Script events instead. The component draws the picture directly; no background
+image or silent performance is needed to establish duration. With no selected AudioTrack, the
+delivered video is silent.
+
## Full pipeline walkthrough
The complete data flow from Script to rendered video. The Sources below are an abridged
diff --git a/docs/quickstart/tracks.md b/docs/quickstart/tracks.md
index 959b7ec1..642f3df6 100644
--- a/docs/quickstart/tracks.md
+++ b/docs/quickstart/tracks.md
@@ -404,10 +404,10 @@ same Frame and moves the whole stack.
### deck:DepthStack
-`id`, `semantic`, `canvas`, `frame` and `appearance` are all required, as is `until`, which says
-what ends the deck: the literal `"program.end"`, a Moment, or a Selection. Only with a Selection may
-you add `until-boundary="start" | "end"` to choose which edge of it ends the deck; the default is
-`end`, and giving the attribute in the other two cases is refused rather than ignored.
+`id`, `canvas`, `frame` and `appearance` are required, together with `semantic` for performance time
+or `space` for authored animation. `until` says
+what ends the deck: a Moment, a Selection or Segment boundary, or an authored time such as `8s`.
+With a Selection or Segment, `until-boundary="start" | "end"` chooses its edge; the default is `end`.
### deck:Card
@@ -489,7 +489,8 @@ optional metadata line.
card's whole appearance — background, border, radius, tail, avatar, the three text rows, and the
enter/hold/exit motion — and every key has a default, so a recipe may set only what it changes.
-`comment:Track` takes `id`, `canvas` and `semantic`, all three required.
+`comment:Track` takes `id`, `canvas` and one time source: `semantic` for performance time or `space`
+for authored animation.
`comment:Sticker` requires `id`, `frame` and `style`, and takes the same windows as a screen overlay
above. Its copy is either the `comment=` attribute or the element's own text — both is refused. The
@@ -575,5 +576,5 @@ All four track families together in one source file:
</film:Film>
```
-The `stack-order` in each SVS Recipe determines z-ordering: speech visual at 10, media at 40,
-captions at 70, text at 90. Higher values render on top.
+In this example, the Recipes place the performance picture at 10, media at 40, captions at 70 and
+text at 90. Higher values paint on top; the author chooses these relationships for the composition.
diff --git a/docs/zh/guide/studio-temporal-windows.md b/docs/zh/guide/studio-temporal-windows.md
index 8402de84..c30e16b2 100644
--- a/docs/zh/guide/studio-temporal-windows.md
+++ b/docs/zh/guide/studio-temporal-windows.md
@@ -8,7 +8,7 @@ description: 作者选择如何成为 Instant 或 Window,并准确回到真正
时间只有三层:
```text
-选择层(Program / Selection / Segment / Moment)
+选择层(Selection / Segment / Moment / 编排时间)
-> 投影层(TemporalInstant / TemporalWindow)
-> 消费层(Programme / Schedule / Track)
```
@@ -20,8 +20,9 @@ Script 的语义标尺包含严格有序的 `2M + 2N + 2` 个点:Token 与 Seg
`@hypit/temporal-markup` 统一拥有 SVML 时间语法,并在领域组件运行前把它降低成普通的投影
组件和运行图边。编译器只会组合 Record、Component 和 Fragment,不认识 `during`、`at`,
-各 Track 也不再各写一套找帧算法。运行图还会通过 Semantic Track 的公共 producer 显式投影
-`ProgramSpace`;领域组件只消费 ProgramSpace 与时间投影结果,与选择层和 Studio 都解耦。
+各 Track 也不再各写一套找帧算法。纯动画可直接声明 `ProgramSpace`;表演时间则通过
+Semantic Track 的公共 producer 显式投影。领域组件只消费 ProgramSpace 与时间投影结果,
+与选择层和 Studio 都解耦。
## 运行时对象
@@ -35,10 +36,10 @@ Script 的语义标尺包含严格有序的 `2M + 2N + 2` 个点:Token 与 Seg
- `<script id="story">` 产生 `Narrative.id = story`,它派生出的 Selection、Moment、Segment
excerpt 与 CaptionDocument 全部携带 `narrativeId = story`;
-- 被 Film 激活的 Semantic Track 的 `id` 就是 `ProgramSpace.id`,所有终端 Track 都携带同一个
+- 被 Film 选择的 Semantic Track 或直接声明的 Space 的 `id` 就是 `ProgramSpace.id`,所有终端 Track 都携带同一个
`programSpaceId`;
-- 每个 Instant/Window 同时保留这两个身份,并用消费者的公开领域身份(通常就是 SVML `id`)
- 作为 `subjectId`。
+- 每个 Instant 保留 Space 身份,来自语义的来源额外保留 Narrative 身份;消费者的公开领域
+ 身份(通常就是 SVML `id`)作为 `subjectId`。
投影 record 的 `id` 可以为了展开后的图内唯一性带父级前缀,但它不是作者身份;`subjectId`
独立保留组件公共 Program 发布的 board、card、item 或 sequence 身份,Companion 只沿这一
@@ -86,7 +87,7 @@ Companion 仍负责实体外观和 Inspector 展示,但不再声明通用时
Instant/Window,并由 Companion 把真实执行谱系连到实体,就自动获得同一套时间行为。
消费端也不是只读取 `frame`:每个官方组件在接收 Instant/Window 时核对 `subjectId`、
-`ProgramSpace.id` 与 `narrativeId`。为此 Space 是运行图上的显式输入边;不通过全局状态、
+`ProgramSpace.id`。Script 与 SemanticTrack 的对应关系在语义投影时核对。Space 是运行图上的显式输入边;不通过全局状态、
当前 Film 或 renderer 上下文补猜。
字幕 Cue 保持独立:`CaptionDocument -> TimedCaptionProjection -> FineCaptionSchedule` 直接从
@@ -94,9 +95,11 @@ Semantic token evidence 得到只读 Cue 时间,不伪装成可拖动的 Tempo
`spaceId / narrativeId / documentId` 的来源校验,但只有 Fine 的 lead、tail、handoff 作为普通
参数允许修改。
-Film 与 Script 的解释也不留在 Studio 核心:`film-studio` 声明 Film 的语义轴和终端 Track
+Film 与 Script 的解释也不留在 Studio 核心:`film-studio` 声明 Film 的时间来源和终端 Track
引用规则;`script-studio` 拥有 Script Source map 与 marker 移动。Studio 只选择
-`narrativeId` 与当前 ProgramSpace 严格一致的 Script,再把语义修改委托回该 Companion。
+`narrativeId` 与当前 SemanticTrack 一致的 Script,再把语义修改委托回该 Companion。
+直接声明的 Space 提供物理时间线,不需要 Script 轨道。`at="2s"` 表示作者编排的事件时刻,
+可以单独用于 Instant,也可以与 `for` 配合形成 Window;其时间修改回写 SVML。
这里的 binding 名不是全局地址。Markup 在降低语法时保留作者元素和输入范围,Elaborator
把端点与 Record、Component、Output 一起按 Source unit hygienize;编译结果中的
diff --git a/docs/zh/quickstart/composition.md b/docs/zh/quickstart/composition.md
index 02f39dfb..0bf178ec 100644
--- a/docs/zh/quickstart/composition.md
+++ b/docs/zh/quickstart/composition.md
@@ -16,8 +16,8 @@ Composition。然后渲染器将该 Composition 编译为 MP4 视频。
## film:Film
-将所有 Track 组装为单一的 Composition。Film 本身没有领域知识——它不知道什么是字幕、
-什么是 Media、什么是语音。它接收任何 VisualTrack 或 AudioTrack,并按堆叠顺序将它们分层。
+将选定的 VisualTrack 和 AudioTrack 组装为单一的 Composition。每份视觉贡献保留自己的
+定时呈现和绘制顺序;声音通过选定的 AudioTrack 加入。
```svml
<space:Canvas id="vertical" width="1080" height="1920"/>
@@ -34,7 +34,8 @@ Composition。然后渲染器将该 Composition 编译为 MP4 视频。
|---|---|---|
| `id` | 是 | 唯一标识符 |
| `canvas` | 是 | 与 Track 布局共享的显式 CanvasSpace |
-| `semantic` | 是 | 来自 `speech:Track` 的 SemanticTrack——定义时长和帧率 |
+| `semantic` | 二选一 | 选定表演的 SemanticTrack,提供时长与帧率 |
+| `space` | 二选一 | 作者声明的 ProgramSpace;与 `semantic` 选择其一 |
| `appearance` | 是 | SVS Film Recipe——画布清除颜色 |
### film:Track
@@ -57,8 +58,9 @@ Composition。然后渲染器将该 Composition 编译为 MP4 视频。
### Track 堆叠
-Track 是**扁平的**——没有嵌套或分组。Z 轴排序完全由每个 Track 的 SVS Recipe 中的
-`stack-order` 属性决定。较低的值在后面;较高的值渲染在上面。
+Film 收集对等的 Track。每个 Track 可以包含多个独立定时、独立排序的呈现,称为 Present。
+许多组件通过 Recipe 的 `stack-order` 暴露绘制顺序:较低的值在后面,较高的值在前面。
+调整 Film 子元素的书写顺序不会改变这一绘制顺序。
典型的堆叠顺序:
@@ -69,8 +71,9 @@ Track 是**扁平的**——没有嵌套或分组。Z 轴排序完全由每个 T
| 70 | 字幕 |
| 90 | 文字叠加层 |
-一个组件可以在不同的 z 位置发出多个视觉元素(Presents),这些元素会与其他组件的 Presents
-交错排列。最终渲染会将所有 Presents 展平,按绝对堆叠键排序,然后绘制到一个画布上。
+不同组件的 Present 可以交错排列。每个 Present 内部又拥有自己的元素树:多个视频、文字
+和图形可以共享布局、遮罩或协同运动。项目组件可以用 HTML/CSS 浏览器程序实现这样的场景,
+独立字幕或覆盖画面仍可以作为对等贡献。根据共同的表现关系分组,尺寸和素材类型不决定边界。
**输出:** `{main.composition}`——完整的 Composition,传递给渲染器。
@@ -86,7 +89,8 @@ Track 是**扁平的**——没有嵌套或分组。Z 轴排序完全由每个 T
|---|---|---|
| `id` | 是 | 唯一标识符 |
| `composition` | 是 | 来自 `film:Film` 的 Composition |
-| `semantic` | 是 | 来自 `speech:Track` 的 SemanticTrack |
+| `semantic` | 二选一 | Composition 选定的 SemanticTrack |
+| `space` | 二选一 | Composition 使用的 ProgramSpace;与 `semantic` 选择其一 |
渲染器:
@@ -99,6 +103,24 @@ Track 是**扁平的**——没有嵌套或分组。Z 轴排序完全由每个 T
**输出:** `{final.video}`——以普通内容寻址 `BlobArtifact` 表示的最终视频。这是最常见的
Build Target,也可以直接接到媒体裁切、音频/帧提取或模型参考输入等后续 Blob 消费者。
+## 完全由组件绘制的影片
+
+对于口播作品,Script 的 Selection 和 Moment 保留话语与呈现之间的关系。聊天动画或图解
+也可以自行安排阅读节奏:声明影片时钟,让场景组件、Film 和 Render 使用它。
+
+```svml
+<import as="time" from="@hypit/program-space@1"/>
+<time:Space id="animation" frame-rate="30" duration="8s"/>
+<!-- scene.track 由使用同一时钟的组件产生。 -->
+<film:Film id="main" canvas={canvas} space={animation} appearance={recipes.film.main}>
+ <film:Track source={scene.track}/>
+</film:Film>
+<render:Video id="final" composition={main.composition} space={animation}/>
+```
+
+场景事件可以采用作者指定的秒数或帧数。在口播编排中,同一种表现也可以跟随投影后的 Script
+事件。组件直接绘制画面,声明时长无需背景图片或静音表演;未选择 AudioTrack 时,交付视频无声。
+
## 完整的管线流程
从 Script 到渲染视频的完整数据流。下面的 Source 是删节示意;可运行的完整项目见
diff --git a/docs/zh/quickstart/tracks.md b/docs/zh/quickstart/tracks.md
index 220619d3..ba7d93a9 100644
--- a/docs/zh/quickstart/tracks.md
+++ b/docs/zh/quickstart/tracks.md
@@ -482,4 +482,5 @@ Track 接受 `id`、`canvas` 与 `semantic`。时长写作 `12f`、`250ms` 或 `
</film:Film>
```
-每个 SVS Recipe 中的 `stack-order` 决定 z 轴排序:语音视觉层为 10,Media 为 40,字幕为 70,文字为 90。数值越高,渲染层越靠上。
+本例的 Recipe 将表演画面放在 10,Media 放在 40,字幕放在 70,文字放在 90。
+数值越高,绘制位置越靠前;作者按作品需要选择这些关系。
diff --git a/examples/semantic-composition/README.md b/examples/semantic-composition/README.md
index 1282b299..5f61a596 100644
--- a/examples/semantic-composition/README.md
+++ b/examples/semantic-composition/README.md
@@ -12,3 +12,24 @@ name is registered in Core or in the renderer.
Copy the package into a video's `packages/`, select the installed Hypit version for its development
dependency, build it and declare it in the project's ordinary package configuration. The package
README shows its Source use and explains the timing and material boundaries.
+
+## Authored chat animation
+
+`chat.svml` is an eight-second composition drawn entirely in a project component. It needs no Script,
+WhisperX, image generation or source video. Four messages arrive on authored times; the same
+`@example/chat-scene` component accepts Script Moments when used beside a speaking performance.
+
+From the repository after installing its dependencies:
+
+```bash
+pnpm build:public-types
+pnpm --filter @example/chat-scene build
+node bin/hypit.mjs check examples/semantic-composition/chat.svml --workspace examples/semantic-composition
+node bin/hypit.mjs build examples/semantic-composition/chat.svrun --workspace examples/semantic-composition --runtime examples/semantic-composition/hypit.runtime.json --follow
+```
+
+The Profile selects local HyperFrames and FFmpeg. Use the machine's prepared browser/media tools;
+there is no hosted generation account in this example. The final Output is `final.video`. The same
+Run can be opened in Studio, where its component lane and physical clock work without a Script lane.
+For an independent project, install `hypit`, copy the component into `packages/`, replace its workspace
+dependency with the installed Hypit version, and build the package normally.
diff --git a/examples/semantic-composition/chat.svml b/examples/semantic-composition/chat.svml
new file mode 100644
index 00000000..513a767b
--- /dev/null
+++ b/examples/semantic-composition/chat.svml
@@ -0,0 +1,25 @@
+<?svml using="@hypit/markup@1"?>
+<svml>
+ <import as="time" from="@hypit/program-space@1"/>
+ <import as="spatial" from="@hypit/spatial@1"/>
+ <import as="fonts" from="@hypit/fonts-open@1"/>
+ <import as="chat" from="@example/chat-scene@1"/>
+ <import as="film" from="@hypit/film@1"/>
+ <import as="render" from="@hypit/render-hyperframes@1"/>
+ <import as="style" source="./chat.svs"/>
+
+ <time:Space id="animation" frame-rate="30" duration="8s"/>
+ <spatial:Canvas id="canvas" width="540" height="960"/>
+ <fonts:Stack id="font" family="inter" weight="600" style="normal"/>
+ <chat:Scene id="conversation" space={animation} canvas={canvas} font={font}
+ during="program" title="Launch crew">
+ <chat:Message id="question" sender="Maya" side="left" at="0.5s" text="Are we ready to launch?"/>
+ <chat:Message id="reply" sender="Leo" side="right" at="2s" text="The video is ready."/>
+ <chat:Message id="turn" sender="Maya" side="left" at="3.5s" text="Great. One tiny change..."/>
+ <chat:Message id="reveal" sender="Leo" side="right" at="5.2s" text="I left the whole scene editable."/>
+ </chat:Scene>
+ <film:Film id="main" canvas={canvas} space={animation} appearance={style.film.main}>
+ <film:Track source={conversation.track}/>
+ </film:Film>
+ <render:Video id="final" composition={main.composition} space={animation}/>
+</svml>
diff --git a/examples/semantic-composition/chat.svrun b/examples/semantic-composition/chat.svrun
new file mode 100644
index 00000000..9f54c331
--- /dev/null
+++ b/examples/semantic-composition/chat.svrun
@@ -0,0 +1,6 @@
+<?svml using="@hypit/run-markup@1"?>
+
+<svrun version="1">
+ <author source="./chat.svml"/>
+ <target output="final.video"/>
+</svrun>
diff --git a/examples/semantic-composition/chat.svs b/examples/semantic-composition/chat.svs
new file mode 100644
index 00000000..6a4a5677
--- /dev/null
+++ b/examples/semantic-composition/chat.svs
@@ -0,0 +1,4 @@
+<?svml using="@hypit/svs@1"?>
+<sheet version="1">
+ film.main { background: #142b32; }
+</sheet>
diff --git a/examples/semantic-composition/hypit.runtime.json b/examples/semantic-composition/hypit.runtime.json
new file mode 100644
index 00000000..c55e7b03
--- /dev/null
+++ b/examples/semantic-composition/hypit.runtime.json
@@ -0,0 +1,8 @@
+{
+ "format": "hypit.runtime-local@1",
+ "dataRoot": ".hypit/runtime",
+ "endpoints": {
+ "media.local": { "use": "@hypit/provider-media-local", "config": { "defaultConcurrency": 2 } },
+ "hyperframes.local": { "use": "@hypit/provider-hyperframes-local", "config": { "workers": 2, "defaultConcurrency": 1 } }
+ }
+}
diff --git a/examples/semantic-composition/packages/chat-scene/README.md b/examples/semantic-composition/packages/chat-scene/README.md
new file mode 100644
index 00000000..4d319c65
--- /dev/null
+++ b/examples/semantic-composition/packages/chat-scene/README.md
@@ -0,0 +1,29 @@
+# Chat scene
+
+A project component for a conversation whose arrivals, bubble layout and scrolling belong together.
+Content lives in Source. Every Message has its own `at`; there is no fixed message count.
+
+For speech-led work, bind the message to the words that introduce it:
+
+```svml
+<chat:Scene id="conversation" semantic={speech.semantic} canvas={canvas} font={font}
+ during={story.selection.demo} title="Conversation">
+ <chat:Message id="answer" sender="Maya" side="left" text="Here it is."
+ at={story.moment.answer}/>
+</chat:Scene>
+```
+
+The same component can receive `space={animation}` and `at="2s"` for authored animation. A Scene
+has an outer Window; its message Instants trigger persistent state. `entrance-frames` controls arrival
+and scrolling duration independently of those triggers (default 10). Messages stay visible or scroll
+up as the conversation develops. Message times are on the film clock and must fall inside the Scene.
+
+The Surface uses the shared temporal helpers and expands children to a finite create/append/render
+graph. The rendering function receives ProgramSpace, Window and Instants, so it knows nothing about
+Script parsing, speech models or Runtime. The browser program evaluates any requested frame directly,
+including frames reached by scrubbing, range rendering or concurrent workers.
+
+Compile with `npm run build`. The package uses the public `hypit/*` interfaces and can live in any
+video project's `packages/`. It is an example to adapt, not an official component to install for every
+animation. The complete runnable Source is [chat.svml](../../chat.svml); see the surrounding
+example README for commands. `font` receives a FontStack, for example from `fonts:Stack`.
diff --git a/examples/semantic-composition/packages/chat-scene/package.json b/examples/semantic-composition/packages/chat-scene/package.json
new file mode 100644
index 00000000..0faf8c02
--- /dev/null
+++ b/examples/semantic-composition/packages/chat-scene/package.json
@@ -0,0 +1,11 @@
+{
+ "name": "@example/chat-scene",
+ "version": "0.0.0-dev",
+ "private": true,
+ "type": "module",
+ "hypit": { "activation": "./dist/activation.js" },
+ "exports": { ".": "./dist/render.js" },
+ "files": ["dist", "README.md"],
+ "scripts": { "build": "tsc -p tsconfig.json", "prepack": "npm run build" },
+ "devDependencies": { "hypit": "workspace:*", "typescript": "5.9.3", "@types/node": "24.10.1" }
+}
diff --git a/examples/semantic-composition/packages/chat-scene/src/activation.ts b/examples/semantic-composition/packages/chat-scene/src/activation.ts
new file mode 100644
index 00000000..a480725e
--- /dev/null
+++ b/examples/semantic-composition/packages/chat-scene/src/activation.ts
@@ -0,0 +1,112 @@
+import { assertAttributes, assertEmptyElement, canonicalize, createMarkupSurfaceHostFacet, sameType, sealGraphFragment, textAttribute } from "hypit/author-kit";
+import type { ComponentPackage, FragmentOperation, ModuleManifest, StructuredSurfaceHandler, SurfaceResolvedReference, TypeRef } from "hypit/author-kit";
+import { compositionTypes } from "hypit/composition";
+import { mediaTypes } from "hypit/media";
+import type { FontStackRef } from "hypit/media";
+import { programSpaceTypes } from "hypit/program-space";
+import type { ProgramSpace } from "hypit/program-space";
+import { semanticTrackTypes } from "hypit/semantic-track";
+import { spatialTypes } from "hypit/spatial";
+import type { CanvasSpace } from "hypit/spatial";
+import { assertTemporalInstantFor, temporalTypes } from "hypit/temporal";
+import type { TemporalInstant, TemporalWindow } from "hypit/temporal";
+import { createTemporalInstantProjection, createTemporalSpace, createTemporalWindowProjection, resolveTemporalContext,
+ temporalContextAttributeVocabulary, temporalInstantAttributeNames, temporalInstantAttributeVocabulary,
+ temporalWindowAttributeNames, temporalWindowAttributeVocabulary } from "hypit/temporal-markup";
+import { renderChat } from "./render.js";
+import type { ChatOptions, Message } from "./render.js";
+
+const module = { name: "@example/chat-scene", version: "1" } as const;
+const types = Object.fromEntries(["Options", "Message", "Messages"].map(name => [name, { module, name }])) as Record<"Options" | "Message" | "Messages", TypeRef>;
+const producers = Object.fromEntries(["empty", "append", "render"].map(name => [name, { module, name }])) as Record<"empty" | "append" | "render", { module: typeof module; name: string }>;
+export const manifest: ModuleManifest = { format: "hypit.module@1", ...module,
+ dependencies: [compositionTypes.visualTrack, mediaTypes.fontStack, programSpaceTypes.programSpace,
+ semanticTrackTypes.track, spatialTypes.canvas, temporalTypes.instant].map(type => ({ module: type.module })),
+ types: Object.values(types).map(type => ({ name: type.name })), capabilities: [], producers: [
+ { name: "empty", inputs: [], outputs: [{ name: "messages", type: types.Messages }], needs: [] },
+ { name: "append", inputs: [{ name: "messages", type: types.Messages }, { name: "message", type: types.Message },
+ { name: "at", type: temporalTypes.instant }, { name: "space", type: programSpaceTypes.programSpace }], outputs: [{ name: "messages", type: types.Messages }], needs: [] },
+ { name: "render", inputs: [{ name: "messages", type: types.Messages }, { name: "options", type: types.Options },
+ { name: "space", type: programSpaceTypes.programSpace }, { name: "canvas", type: spatialTypes.canvas },
+ { name: "window", type: temporalTypes.window }, { name: "font", type: mediaTypes.fontStack }], outputs: [{ name: "track", type: compositionTypes.visualTrack }], needs: [] },
+ ],
+};
+const inline = <T>(record: { value: { kind: string; value?: unknown } } | undefined): T => {
+ if (record?.value.kind !== "inline") throw new Error("Chat inputs must be inline values.");
+ return record.value.value as T;
+};
+const value = (data: unknown) => ({ kind: "inline" as const, value: canonicalize(data) });
+const component: ComponentPackage = { producers: [
+ { producer: producers.empty, handler: () => ({ outputs: { messages: value([]) }, needs: {} }) },
+ { producer: producers.append, handler: ({ inputs }) => {
+ const message = inline<Omit<Message, "at">>(inputs.message), at = inline<TemporalInstant>(inputs.at);
+ const messages = inline<Message[]>(inputs.messages);
+ assertTemporalInstantFor(at, { subjectId: message.id, space: inline<ProgramSpace>(inputs.space) });
+ if (messages.some(item => item.id === message.id) || (messages.at(-1)?.at.frame ?? -1) > at.frame) throw new Error("Chat messages need unique ids and chronological arrival times.");
+ return { outputs: { messages: value([...messages, { ...message, at }]) }, needs: {} };
+ } },
+ { producer: producers.render, handler: ({ inputs }) => ({ outputs: { track: value(renderChat(inline<ProgramSpace>(inputs.space),
+ inline<CanvasSpace>(inputs.canvas), inline<TemporalWindow>(inputs.window), inline<FontStackRef>(inputs.font),
+ inline<Message[]>(inputs.messages), inline<ChatOptions>(inputs.options))) }, needs: {} }) },
+] };
+
+export const decodeSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
+ assertAttributes(element, ["id", "semantic", "space", "canvas", "font", "title", "entrance-frames", ...temporalWindowAttributeNames]);
+ const id = textAttribute(element, "id"), context = resolveTemporalContext({ element, resolveReference });
+ const time = createTemporalSpace({ id, element, ...context });
+ const window = createTemporalWindowProjection({ id: `${id}.window`, subjectId: id, element, ...context, space: time.space, resolveReference });
+ const reference = (name: string, type: TypeRef): SurfaceResolvedReference => {
+ const raw = element.attributes[name];
+ if (typeof raw !== "object" || raw.kind !== "reference") throw new Error(`${name} must be a reference.`);
+ const found = resolveReference(raw.path);
+ if (found === undefined || !sameType(found.type, type)) throw new Error(`${name} has the wrong Type.`);
+ return found;
+ };
+ const options: ChatOptions = { id, title: textAttribute(element, "title"), entranceFrames: Number(element.attributes["entrance-frames"] ?? "10") };
+ const records = [...window.records, { id: `${id}.options`, type: types.Options, value: value(options), range: element.range }];
+ const components = [...time.components, ...window.components], fragments = [...time.fragments, ...window.fragments];
+ const inputs = [{ name: "space", type: programSpaceTypes.programSpace }, { name: "canvas", type: spatialTypes.canvas },
+ { name: "font", type: mediaTypes.fontStack }, { name: "window", type: temporalTypes.window }, { name: "options", type: types.Options }];
+ const bindings: Record<string, SurfaceResolvedReference["ref"]> = { space: time.space.ref, canvas: reference("canvas", spatialTypes.canvas).ref,
+ font: reference("font", mediaTypes.fontStack).ref, window: window.ref, options: { kind: "record", id: `${id}.options` } };
+ const input = (name: string) => ({ kind: "fragment-input" as const, name });
+ const operation = (name: string) => ({ kind: "fragment-operation" as const, operation: name });
+ const operations: FragmentOperation[] = [{ id: "empty", producer: producers.empty, inputs: {}, result: { kind: "output", name: "messages" } }];
+ let previous = "empty", index = 0;
+ for (const child of element.children) {
+ if (child.kind === "text") { if (child.value.trim()) throw new Error("Chat Scene accepts Message children."); continue; }
+ if (child.name.split(":").at(-1) !== "Message") throw new Error("Chat Scene accepts Message children.");
+ assertAttributes(child, ["id", "sender", "text", "side", ...temporalInstantAttributeNames]); assertEmptyElement(child);
+ const messageId = textAttribute(child, "id"), side = textAttribute(child, "side");
+ if (side !== "left" && side !== "right") throw new Error("Message side must be left or right.");
+ const at = createTemporalInstantProjection({ id: `${id}.${messageId}`, subjectId: messageId, element: child, ...context, space: time.space, resolveReference });
+ records.push(...at.records); components.push(...at.components); fragments.push(...at.fragments);
+ const key = `message-${++index}`;
+ records.push({ id: `${id}.${key}`, type: types.Message, value: value({ id: messageId, sender: textAttribute(child, "sender"), text: textAttribute(child, "text"), side }), range: child.range });
+ inputs.push({ name: key, type: types.Message }, { name: `${key}-at`, type: temporalTypes.instant });
+ bindings[key] = { kind: "record", id: `${id}.${key}` }; bindings[`${key}-at`] = at.ref;
+ operations.push({ id: key, producer: producers.append, inputs: { messages: operation(previous), message: input(key), at: input(`${key}-at`), space: input("space") }, result: { kind: "output", name: "messages" } });
+ previous = key;
+ }
+ if (!index) throw new Error("Chat Scene requires a Message.");
+ operations.push({ id: "render", producer: producers.render, inputs: { messages: operation(previous), options: input("options"),
+ space: input("space"), canvas: input("canvas"), font: input("font"), window: input("window") }, result: { kind: "output", name: "track" } });
+ const fragment = sealGraphFragment({ inputs, operations, exports: [{ name: "track", type: compositionTypes.visualTrack, root: operation("render") }] });
+ return { records, fragments: [...fragments, fragment], components: [...components,
+ { id, fragment: fragment.id, inputs: bindings, outputs: { track: `${id}.track` }, range: element.range }], exports: [`${id}.track`] };
+};
+const declaration = { name: "scene", tag: "Scene", mode: "structured" as const,
+ outputs: [compositionTypes.visualTrack, programSpaceTypes.programSpace, temporalTypes.window, temporalTypes.instant, temporalTypes.windowSpec, temporalTypes.instantSpec, ...Object.values(types)],
+ vocabulary: { summary: "A conversation whose message arrivals and scrolling form one visual scene.", attributes: [
+ ...temporalContextAttributeVocabulary, ...temporalWindowAttributeVocabulary,
+ ...["id", "title", "canvas", "font"].map(name => ({ name, kind: "expression" as const, required: true, summary: name })),
+ { name: "entrance-frames", kind: "literal" as const, required: false, summary: "Arrival and scrolling duration; defaults to 10 frames." },
+ ], children: [{ tag: "Message", cardinality: "many" as const, summary: "One authored message and the event that reveals it.", attributes: [
+ ...["id", "sender", "text", "side"].map(name => ({ name, kind: "literal" as const, required: true, summary: name })), ...temporalInstantAttributeVocabulary,
+ ] }], ports: [{ name: "track", type: compositionTypes.visualTrack, summary: "The complete conversation scene." }],
+ example: '<chat:Scene id="chat" semantic={speech.semantic} canvas={canvas} font={font} during="program" title="Conversation"><chat:Message id="answer" sender="Maya" side="left" text="Here it is." at={story.moment.answer}/></chat:Scene>',
+ },
+};
+export const hypitPackage = { format: "hypit.node-package@1" as const, modules: [{ manifest }], components: [component],
+ hostFacets: [createMarkupSurfaceHostFacet({ module, declaration, handler: decodeSurface })] };
+export default hypitPackage;
diff --git a/examples/semantic-composition/packages/chat-scene/src/render.ts b/examples/semantic-composition/packages/chat-scene/src/render.ts
new file mode 100644
index 00000000..877d671c
--- /dev/null
+++ b/examples/semantic-composition/packages/chat-scene/src/render.ts
@@ -0,0 +1,64 @@
+import { sealVisualTrack } from "hypit/composition";
+import type { VisualElement } from "hypit/composition";
+import { browserProgram } from "hypit/hyperframes";
+import type { FontStackRef } from "hypit/media";
+import type { ProgramSpace } from "hypit/program-space";
+import type { CanvasSpace } from "hypit/spatial";
+import { assertTemporalWindowFor } from "hypit/temporal";
+import type { TemporalInstant, TemporalWindow } from "hypit/temporal";
+
+export type Message = { id: string; sender: string; text: string; side: "left" | "right"; at: TemporalInstant };
+export type ChatOptions = { id: string; title: string; entranceFrames: number };
+
+/** Layout, arrival and scrolling share one visual component; timing is already resolved. */
+export function renderChat(space: ProgramSpace, canvas: CanvasSpace, window: TemporalWindow,
+ font: FontStackRef, messages: readonly Message[], options: ChatOptions) {
+ assertTemporalWindowFor(window, { subjectId: options.id, space });
+ if (!Number.isSafeInteger(options.entranceFrames) || options.entranceFrames < 1) throw new Error("Chat entrance-frames must be a positive integer.");
+ if (messages.some(message => message.at.frame < window.span.startFrame || message.at.frame >= window.span.endFrameExclusive)) throw new Error("Chat messages must appear inside the scene's window.");
+ let order = 0;
+ const text = (id: string, value: string, size: number, color: string): VisualElement => ({
+ id, kind: "text", parent: "chat", order: ++order, text: value, fonts: font.faces,
+ style: [{ name: "font-size", value: `${size}px` }, { name: "line-height", value: 1.3 }, { name: "color", value: color }],
+ });
+ const children = [text("title", options.title, 30, "#f6ead9"), text("subtitle", "A small change of plan", 16, "#b8c6c5"),
+ ...messages.flatMap((message, index) => [text(`sender-${index}`, message.sender, 16, "#a9b8bd"), text(`text-${index}`, message.text, 27, "#f4efe6")])];
+ const program = browserProgram({
+ html: `<header><div class="status"></div><div>{{title}}<div class="subtitle">{{subtitle}}</div></div></header>
+ <div class="viewport"><div class="messages">${messages.map((message, index) => `<article class="${message.side}" data-message="${index}">
+ <div class="sender">{{sender-${index}}}</div><div class="bubble">{{text-${index}}}</div></article>`).join("")}</div></div>
+ <footer><div class="dots"><i></i><i></i><i></i></div></footer>`,
+ css: `:scope{background:#142b32;overflow:hidden}
+ header{position:absolute;inset:0 0 auto;padding:48px 38px 32px;display:flex;align-items:center;gap:18px;border-bottom:1px solid #35505a;background:#1c353e}
+ .status{width:17px;height:17px;border-radius:50%;background:#abda82;box-shadow:0 0 0 7px #304b43}
+ .subtitle{margin-top:6px}.viewport{position:absolute;inset:164px 32px 96px;overflow:hidden}
+ .messages{position:relative}article{margin:0 0 28px;max-width:84%;transform-origin:left bottom}
+ article.right{margin-left:auto;transform-origin:right bottom}.sender{margin:0 10px 8px}
+ .right .sender{text-align:right}.bubble{padding:20px 23px;border-radius:23px 23px 23px 5px;background:#304953;box-shadow:0 6px 0 #10262d}
+ .right .bubble{background:#47694d;border-radius:23px 23px 5px 23px}footer{position:absolute;inset:auto 0 0;height:90px;background:#1c353e}
+ .dots{display:flex;gap:9px;margin:35px 40px}.dots i{width:10px;height:10px;background:#a6b8b9;border-radius:50%}`,
+ data: { times: messages.map(message => message.at.frame - window.span.startFrame), entrance: options.entranceFrames },
+ setup: `const cards=[...root.querySelectorAll('article')], list=root.querySelector('.messages'), viewport=root.querySelector('.viewport'), dots=root.querySelector('.dots');
+ return frame=>{
+ let last=-1;
+ cards.forEach((card,i)=>{
+ const age=frame-data.times[i], visible=age>=0;
+ card.style.display=visible?'block':'none';
+ if(visible){last=i;const t=Math.min(1,age/data.entrance), p=1-Math.pow(1-t,3);
+ card.style.opacity=String(p);card.style.transform='translateY('+(18*(1-p))+'px) scale('+(0.96+0.04*p)+')';}
+ });
+ const bottom=last<0?0:cards[last].offsetTop+cards[last].offsetHeight;
+ const previous=last<1?0:cards[last-1].offsetTop+cards[last-1].offsetHeight;
+ const t=last<0?1:Math.max(0,Math.min(1,(frame-data.times[last])/data.entrance));
+ const before=Math.max(0,previous-viewport.clientHeight), after=Math.max(0,bottom-viewport.clientHeight);
+ list.style.transform='translateY('+(-(before+(after-before)*(1-Math.pow(1-t,3))))+'px)';
+ dots.style.opacity=last===cards.length-1?'0':'1';
+ [...dots.children].forEach((dot,i)=>dot.style.transform='translateY('+(-3*(1+Math.sin(frame*.18-i)))+'px)');
+ };`,
+ });
+ return sealVisualTrack({ id: options.id, programSpaceId: space.id, visualIr: "hypit.visual-ir@1",
+ presents: [{ id: options.id, span: window.span, stacking: { order: 0, tieBreak: options.id },
+ elements: [{ id: "chat", kind: "program", order: 0, program,
+ style: [{ name: "position", value: "absolute" }, { name: "inset", value: 0 },
+ { name: "width", value: `${canvas.widthPx}px` }, { name: "height", value: `${canvas.heightPx}px` }] }, ...children] }] });
+}
diff --git a/examples/semantic-composition/packages/chat-scene/test/render.test.ts b/examples/semantic-composition/packages/chat-scene/test/render.test.ts
new file mode 100644
index 00000000..ed7ed686
--- /dev/null
+++ b/examples/semantic-composition/packages/chat-scene/test/render.test.ts
@@ -0,0 +1,37 @@
+import assert from "node:assert/strict";
+import test from "node:test";
+import type { BrowserProgram } from "hypit/hyperframes";
+import type { FontStackRef } from "@hypit/media";
+import type { ProgramSpace } from "@hypit/program-space";
+import { composeTemporalWindow, projectMomentInstant, projectProgramInstant } from "@hypit/temporal";
+import { semanticTrackFixture } from "../../../../../test/semantic-track-fixture.js";
+import { fixtureResource } from "../../../../../test/fixture-resource.js";
+import { renderChat } from "../src/render.js";
+
+const space: ProgramSpace = { id: "animation", durationSec: 8, frameRate: { numerator: 30, denominator: 1 } };
+const font: FontStackRef = { faces: [{ sources: [{ artifact: { kind: "blob", resource: fixtureResource("chat-font"), size: 32, mediaType: "font/woff2" } }], weight: 600, style: "normal" }] };
+const point = (id: string, frame: number) => projectProgramInstant({ itemId: id, subjectId: id, space,
+ projection: { ref: "absolute", at: { unit: "frames", value: frame } }, authority: { kind: "fixed" } });
+const window = composeTemporalWindow({ id: "chat", subjectId: "chat" }, point("chat", 0), point("chat", 240));
+
+test("the same chat renderer consumes authored and word-bound events; changing performance moves only the word-bound message", () => {
+ const scene = (wordFrame: number) => {
+ const semantic = semanticTrackFixture(space, { anchors: [{ identity: "answer", frame: wordFrame }] });
+ const at = projectMomentInstant({ itemId: "reply", subjectId: "reply", semantic,
+ moment: { narrativeId: semantic.narrativeId, id: "answer", anchorId: "answer" },
+ projection: { ref: "moment.cue" }, authority: { kind: "semantic", boundary: "cue" } });
+ return renderChat(space, { widthPx: 540, heightPx: 960, origin: "top-left", xDirection: "right", yDirection: "down", pixelAspect: "square" }, window, font, [
+ { id: "opening", sender: "Maya", text: "Ready?", side: "left", at: point("opening", 15) },
+ { id: "reply", sender: "Leo", text: "Ready.", side: "right", at },
+ ], { id: "chat", title: "Launch", entranceFrames: 10 });
+ };
+ const program = (value: ReturnType<typeof scene>) => {
+ const element = value.presents[0]!.elements.find(element => element.kind === "program")!;
+ assert.equal(element.kind, "program");
+ return element.program.payload as unknown as BrowserProgram;
+ };
+ const before = program(scene(60)), after = program(scene(90));
+ assert.equal(before.html, after.html);
+ assert.deepEqual(before.data, { times: [15, 60], entrance: 10 });
+ assert.deepEqual(after.data, { times: [15, 90], entrance: 10 });
+});
diff --git a/examples/semantic-composition/packages/chat-scene/tsconfig.json b/examples/semantic-composition/packages/chat-scene/tsconfig.json
new file mode 100644
index 00000000..6d5a18e7
--- /dev/null
+++ b/examples/semantic-composition/packages/chat-scene/tsconfig.json
@@ -0,0 +1,18 @@
+{
+ "compilerOptions": {
+ "target": "ES2023",
+ "module": "NodeNext",
+ "moduleResolution": "NodeNext",
+ "lib": ["ES2023"],
+ "strict": true,
+ "noUncheckedIndexedAccess": true,
+ "exactOptionalPropertyTypes": true,
+ "verbatimModuleSyntax": true,
+ "declaration": true,
+ "sourceMap": true,
+ "rootDir": "src",
+ "outDir": "dist",
+ "skipLibCheck": true
+ },
+ "include": ["src/**/*.ts"]
+}
diff --git a/package.json b/package.json
index e1509497..11c2288f 100644
--- a/package.json
+++ b/package.json
@@ -139,7 +139,11 @@
"examples/semantic-composition/packages/*/src/**/*",
"examples/semantic-composition/packages/*/package.json",
"examples/semantic-composition/packages/*/tsconfig.json",
- "examples/semantic-composition/packages/*/README.md"
+ "examples/semantic-composition/packages/*/README.md",
+ "examples/semantic-composition/*.svml",
+ "examples/semantic-composition/*.svrun",
+ "examples/semantic-composition/*.svs",
+ "examples/semantic-composition/hypit.runtime.json"
],
"scripts": {
"build:public-types": "node scripts/build-public-types.mjs",
diff --git a/packages/audio-track/README.md b/packages/audio-track/README.md
index 3810c7ba..22b570fe 100644
--- a/packages/audio-track/README.md
+++ b/packages/audio-track/README.md
@@ -1,5 +1,9 @@
# `@hypit/audio-track`
+The Track Surface accepts `semantic={speech.semantic}` or `space={animation}`. Semantic context
+resolves Script Selections and Moments; authored space supports clock-based animation. Shared `at`
+inputs accept a Moment or a time such as `2s`; `at` with `for` produces a Window where required.
+
Place music, ambience and sound effects in the assembled video. A reveal sound can follow a Script
Moment; music can occupy a Segment or the program; an explicitly timed sound can use clock placement.
Performance audio normally comes from [Speech Track](../speech-track/README.md) and can continue
@@ -11,7 +15,7 @@ music ducking for the actual performance. The resulting `.track` is a peer Audio
## Authoring clips
-The current Markup Track requires `semantic`; its Fragment derives ProgramSpace from that input.
+The Markup Track selects `semantic` or `space`; its Fragment receives the resulting ProgramSpace.
Each Clip accepts one shared temporal Window form: `during="program"`, a Selection or Segment,
a Moment paired with `for`, or explicit `start`/`end` expressions. The source must already contain
normalized audio. This excerpt assumes imports, Script, SemanticTrack, Clock and source media exist:
diff --git a/packages/audio-track/src/fragment.ts b/packages/audio-track/src/fragment.ts
index ae0c8696..dcf88dd7 100644
--- a/packages/audio-track/src/fragment.ts
+++ b/packages/audio-track/src/fragment.ts
@@ -1,8 +1,8 @@
+import { programSpaceTypes } from "@hypit/program-space";
import { compositionTypes } from "@hypit/composition";
import { sealGraphFragment } from "@hypit/elaborator";
import type { FragmentOperation } from "@hypit/elaborator";
import { mediaTypes } from "@hypit/media";
-import { semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";
import { temporalTypes } from "@hypit/temporal";
import { audioTrackProducers, audioTrackTypes } from "./manifest.js";
@@ -20,7 +20,6 @@ export function createAudioTrackFragment(items: readonly AudioTrackFragmentItem[
if (items.length === 0) throw new Error("Audio Track Fragment requires at least one Item.");
const inputTypes = new Map<string, (typeof audioTrackTypes.clipSpec | typeof mediaTypes.synchronized | typeof temporalTypes.window)>();
const operations: FragmentOperation[] = [
- { id: "audio:space", producer: semanticTrackProducers.projectProgramSpace, inputs: { track: input("semantic") }, result: { kind: "output", name: "space" } },
{ id: "audio:set:empty", producer: audioTrackProducers.createSet, inputs: {}, result: { kind: "output", name: "set" } },
];
let current = "audio:set:empty";
@@ -33,7 +32,7 @@ export function createAudioTrackFragment(items: readonly AudioTrackFragmentItem[
id,
producer: audioTrackProducers.appendItem,
inputs: {
- set: operation(current), header: input("header"), space: operation("audio:space"),
+ set: operation(current), header: input("header"), space: input("space"),
media: input(item.mediaName), spec: input(item.specName),
window: input(item.windowName),
},
@@ -43,12 +42,12 @@ export function createAudioTrackFragment(items: readonly AudioTrackFragmentItem[
});
operations.push(
{ id: "audio:program", producer: audioTrackProducers.finalize, inputs: { set: operation(current), header: input("header") }, result: { kind: "output", name: "program" } },
- { id: "audio:track", producer: audioTrackProducers.render, inputs: { space: operation("audio:space"), program: operation("audio:program") }, result: { kind: "output", name: "track" } },
+ { id: "audio:track", producer: audioTrackProducers.render, inputs: { space: input("space"), program: operation("audio:program") }, result: { kind: "output", name: "track" } },
);
return sealGraphFragment({
inputs: [
{ name: "header", type: audioTrackTypes.header },
- { name: "semantic", type: semanticTrackTypes.track },
+ { name: "space", type: programSpaceTypes.programSpace },
...[...inputTypes].map(([inputName, type]) => ({ name: inputName, type })),
],
operations,
diff --git a/packages/audio-track/src/manifest.ts b/packages/audio-track/src/manifest.ts
index 75db20f0..83c50c36 100644
--- a/packages/audio-track/src/manifest.ts
+++ b/packages/audio-track/src/manifest.ts
@@ -1,10 +1,11 @@
+import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";
import { compositionDependency, compositionTypes } from "@hypit/composition";
import { mediaDependency, mediaTypes } from "@hypit/media";
import { narrativeDependency } from "@hypit/narrative";
import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";
import { blobRefObjectSchema } from "@hypit/protocol";
import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";
-import { semanticTrackDependency, semanticTrackTypes } from "@hypit/semantic-track";
+import { semanticTrackDependency } from "@hypit/semantic-track";
import { temporalDependency, temporalTypes } from "@hypit/temporal";
import { temporalWindowAttributeVocabulary } from "@hypit/temporal-markup";
@@ -75,10 +76,10 @@ export const audioTrackMarkupSurfaces = [{
name: "track", tag: "Track", mode: "structured",
outputs: [audioTrackTypes.header, audioTrackTypes.clipSpec, temporalTypes.instantSpec, temporalTypes.windowSpec, temporalTypes.instant, temporalTypes.window, audioTrackTypes.program, compositionTypes.audioTrack],
vocabulary: {
- summary: "One Audio Track: explicitly prepared audio Clips placed on a shared SemanticTrack and lowered to one ordinary peer AudioTrack.",
+ summary: "One Audio Track: explicitly prepared audio Clips placed on the selected film time axis and lowered to one ordinary peer AudioTrack.",
attributes: [
{ name: "id", kind: "identifier", required: true, summary: "Names this Audio Track and prefixes the identity of every Clip that does not name itself." },
- { name: "semantic", kind: "reference", required: true, accepts: [semanticTrackTypes.track], summary: "Selects the continuous semantic frame domain every Clip window resolves into." },
+ ...temporalContextAttributeVocabulary,
],
children: [
{ tag: "Clip", cardinality: "many",
diff --git a/packages/audio-track/src/surface.ts b/packages/audio-track/src/surface.ts
index 37e8b1dc..d7639bd2 100644
--- a/packages/audio-track/src/surface.ts
+++ b/packages/audio-track/src/surface.ts
@@ -1,5 +1,5 @@
+import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";
import { mediaTypes } from "@hypit/media";
-import { semanticTrackTypes } from "@hypit/semantic-track";
import type {
StructuredElement,
StructuredSurfaceHandler,
@@ -104,9 +104,10 @@ function occupancy(element: StructuredElement): AudioOccupancy {
}
export const decodeAudioTrackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
- allowed(element, ["id", "semantic"]);
+ allowed(element, ["id", "semantic", "space"]);
const id = text(element, "id");
- const semantic = resolved(element.attributes.semantic, `${element.name}.semantic`, semanticTrackTypes.track, resolveReference);
+ const context = resolveTemporalContext({ element, resolveReference });
+ const time = createTemporalSpace({ id: id, element, ...context });
const headerId = `${id}.header`;
const records: SurfaceRecordDraft[] = [{
id: headerId,
@@ -114,12 +115,12 @@ export const decodeAudioTrackSurface: StructuredSurfaceHandler = ({ element, res
value: { kind: "inline", value: sealAudioTrackHeader({ id }) },
range: element.range,
}];
- const temporalComponents: SurfaceComponentDraft[] = [];
- const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [];
+ const temporalComponents: SurfaceComponentDraft[] = [...time.components];
+ const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [...time.fragments];
const fragmentItems: Parameters<typeof createAudioTrackFragment>[0][number][] = [];
const inputs: Record<string, { kind: "record"; id: string } | { kind: "component-output"; component: string; output: string }> = {
header: { kind: "record", id: headerId },
- semantic: semantic.ref,
+ space: time.space.ref,
};
let itemIndex = 0;
for (const child of element.children) {
@@ -137,7 +138,7 @@ export const decodeAudioTrackSurface: StructuredSurfaceHandler = ({ element, res
const suffix = String(itemIndex).padStart(4, "0");
const clipId = optionalText(child, "id") ?? `${id}.clip.${suffix}`;
const source = resolved(child.attributes.source, `${child.name}.source`, mediaTypes.synchronized, resolveReference);
- const temporal = createTemporalWindowProjection({ id: clipId, element: child, semantic, resolveReference });
+ const temporal = createTemporalWindowProjection({ id: clipId, element: child, ...context, space: time.space, resolveReference });
records.push(...temporal.records);
temporalComponents.push(...temporal.components);
temporalFragments.push(...temporal.fragments);
diff --git a/packages/audio-track/test/audio-track.test.ts b/packages/audio-track/test/audio-track.test.ts
index 43f82cc0..da17e58b 100644
--- a/packages/audio-track/test/audio-track.test.ts
+++ b/packages/audio-track/test/audio-track.test.ts
@@ -43,12 +43,11 @@ import { MarkupSurfaceRegistry, createMarkupAuthorFrontend } from "@hypit/markup
import { createRecordAdmitter, TypeValidatorRegistry } from "@hypit/validation";
import { visualIrManifest } from "@hypit/visual-ir";
-const space = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 10,
+const space = sealProgramSpace({ id: "test-space", durationSec: 10,
frameRate: { numerator: 30, denominator: 1 },
});
const header = sealAudioTrackHeader({ id: "sound" });
-const semantic = semanticTrackFixture(space, { anchors: [
+const semantic = semanticTrackFixture(space, { narrativeId: "test-narrative", anchors: [
{ identity: "a", frame: 30 }, { identity: "b", frame: 60 },
{ identity: "c", frame: 90 }, { identity: "d", frame: 120 },
] });
@@ -249,7 +248,7 @@ test("dynamic Fragment keeps every material and temporal dependency as an explic
{ mediaName: "impact", specName: "impact-spec", windowName: "impact-window" },
]);
assert.deepEqual(fragment.inputs.map((input) => input.name), [
- "header", "impact", "impact-spec", "impact-window", "music", "music-spec", "music-window", "semantic", "voice", "voice-spec", "voice-window",
+ "header", "impact", "impact-spec", "impact-window", "music", "music-spec", "music-window", "space", "voice", "voice-spec", "voice-window",
]);
assert.equal(fragment.exports[1]?.name, "track");
});
diff --git a/packages/caption-fine/README.md b/packages/caption-fine/README.md
index 6c7a8692..65e741f0 100644
--- a/packages/caption-fine/README.md
+++ b/packages/caption-fine/README.md
@@ -64,9 +64,9 @@ caption.primary {
x: 0.5; y: 0.9; width: 0.84; height: 0.22;
anchor-x: center; anchor-y: bottom;
align: center; block-align: end; inline-size: fixed;
- wrap: word; max-lines: 2; max-words-per-line: 4;
+ wrap: word;
size: 58; line-height: 1; fill: #FFFFFF;
- background: #00000000; padding: 0; radius: 0;
+ background: #00000000; padding: "0"; radius: 0;
karaoke: current; active-fill: #FFD54A;
cue-enter: spring; cue-enter-frames: 4;
cue-enter-start-scale: 0.75;
@@ -75,5 +75,34 @@ caption.primary {
}
```
+## Language, spacing and line layout
+
+The same Caption pipeline serves English and Chinese. Script emits English lexical words and
+individual Han characters as Display Words; punctuation stays with its display word. Fine uses
+those units for timing and active Paint, while authored Cues remain complete reading phrases.
+Dual Text retains its complete alignment unit even when it displays or speaks several words.
+
+`word-gap` applies between Latin words and at Chinese/Latin boundaries. Adjacent Han characters and
+full-width punctuation carry no extra word gap. `letter-spacing` controls glyph tracking. Exact
+font fallbacks supply the required glyphs; the layout does not select a font by language.
+
+| Control | Behavior |
+| --- | --- |
+| `width`, `size`, `padding`, `letter-spacing`, `word-gap` | Determine the available space and the text's occupied width. |
+| `wrap: word` | Flows at display-unit boundaries and permits an over-wide word to break. Han units are already characters. |
+| `wrap: grapheme` | Also permits breaking inside a Latin word. |
+| `max-words-per-line` | Optional counted row breaks between complete alignment units. Counts Display Words, normally characters for Chinese; it does not make new Cues. An indivisible Dual Text unit can exceed this count. |
+| `max-lines` | Requires `max-words-per-line`; rejects too many counted rows. It does not measure browser wrapping or guarantee one physical line. |
+
+Both count limits are omitted above so the example flows by available width. For a compact
+single-line treatment, author coherent Cues and choose a font, size and width that fit those Cues.
+`karaoke: off` keeps a complete Cue steady; `current` and `trail` follow its timed units, which are
+normally individual characters in Chinese. The Hypit Skill's Caption craft page owns grouping and
+visual direction.
+
+`font` accepts an exact face or ordered stack. A local file declared through `media:Font` can be the
+primary face or a `<caption-fine:Fallback font={...}/>` child, just like a bundled face. See
+[Media font assets](../media/README.md#font-files) and [the open catalog](../fonts-open/README.md).
+
The [Fine Studio Companion](../caption-fine-studio/src/index.ts) reads the same schedule and per-Cue
Style references. It presents the actual Cue timing and exposes supported Style edits in the Inspector.
diff --git a/packages/caption-fine/src/manifest.ts b/packages/caption-fine/src/manifest.ts
index 8dfe83a7..72259364 100644
--- a/packages/caption-fine/src/manifest.ts
+++ b/packages/caption-fine/src/manifest.ts
@@ -62,9 +62,9 @@ export const captionFineMarkupSurfaces = [
{ name: "wrap", required: false, values: ["word", "grapheme"], fallback: "word",
summary: "Wraps at Word boundaries, falling back for an over-wide Word, or permits a break at any grapheme." },
{ name: "max-lines", required: false,
- summary: "Rejects a Cue whose explicitly constructed rows exceed this budget; it requires max-words-per-line and never clips Paint." },
+ summary: "Rejects too many counted rows; requires max-words-per-line. Does not measure browser wrapping or guarantee a single physical line." },
{ name: "max-words-per-line", required: false,
- summary: "Forces a new line after this many display Words while preserving authored Cue boundaries." },
+ summary: "Optional counted row breaks between complete alignment units. Counts display Words, normally individual characters in Chinese; preserves Cues. Omit for width-based flow." },
{ name: "direction", required: false, values: ["ltr", "rtl"], fallback: "ltr",
summary: "Sets the writing direction the Words are laid out in." },
{ name: "line-height", required: true,
@@ -72,7 +72,7 @@ export const captionFineMarkupSurfaces = [
{ name: "letter-spacing", required: false, fallback: "0",
summary: "Adds pixels of tracking between glyphs." },
{ name: "word-gap", required: false, fallback: "a quarter of size",
- summary: "Sets the pixel gap between neighbouring Words." },
+ summary: "Sets pixel spacing between Latin words and at Chinese/Latin boundaries. Adjacent Han characters and full-width punctuation have no extra word gap; use letter-spacing for tracking." },
{ name: "size", required: true,
summary: "Sets the type size in pixels." },
{ name: "kerning", required: false, values: ["auto", "normal", "none"], fallback: "auto",
@@ -298,7 +298,7 @@ export const captionFineMarkupSurfaces = [
name: "track", tag: "Track", mode: "structured", outputs: [captionFineTypes.schedule, compositionTypes.visualTrack],
vocabulary: {
summary: "Joins the authored CaptionDocument against the SemanticTrack and renders it as one ordinary peer VisualTrack.",
- appearance: "One block of caption text wrapped into lines inside a rounded Cue box, placed at a Recipe-chosen point on the Canvas and spanning a fraction of its width. Cues follow the speech one after another, each arriving and leaving with its own motion, and the Words of a Cue either stand there together from its first Frame or uncover as they are spoken. As the speech advances, the Word being spoken, or every Word up to it, is repainted in the active Paint, snapping at the Word boundary or sweeping across the glyphs, and it may take a rule beneath it, a rounded highlight capsule behind it and a brief pop at the moment it becomes the spoken one. That capsule either stands alone on each Atom or grows as one continuous run over everything already read, following the Words across line breaks.\n\nEvery Word of every Cue is drawn in one typeface, the one its Style names: emphasis varies by Word through the active Paint, never by typeface, and lines exist only where the text wraps, so a line cannot be given a face, colour or weight of its own. A caption whose lines are set in different typefaces is outside what this Track can draw.",
+ appearance: "One block of caption text wrapped into lines inside a rounded Cue box, placed at a Recipe-chosen point on the Canvas and spanning a fraction of its width. Cues follow the speech one after another, each arriving and leaving with its own motion, and the Words of a Cue either stand there together from its first Frame or uncover as they are spoken. As the speech advances, the Word being spoken, or every Word up to it, is repainted in the active Paint, snapping at the Word boundary or sweeping across the glyphs, and it may take a rule beneath it, a rounded highlight capsule behind it and a brief pop at the moment it becomes the spoken one. That capsule either stands alone on each Atom or grows as one continuous run over everything already read, following the Words across line breaks.\n\nEvery Cue uses the ordered exact font stack its Style names, with fallback faces supplying glyphs for other scripts. Emphasis varies by timed unit through the active Paint. Lines follow the common flow, so a line cannot be given a separate face, colour or weight of its own. A caption whose lines are set in different typefaces is outside what this Track can draw.",
preview: previewImage("Track.png"),
attributes: [
{ name: "id", kind: "identifier", required: true,
diff --git a/packages/caption-fine/src/render.ts b/packages/caption-fine/src/render.ts
index 6cf7d0d4..eb4131af 100644
--- a/packages/caption-fine/src/render.ts
+++ b/packages/caption-fine/src/render.ts
@@ -1067,8 +1067,7 @@ export function renderFineCaption(
throw new Error("Fine Caption accepts one uniform token rule and cannot consume word-specific Style runs");
}
assertProgramSpaceIdentity(space);
- if (schedule.spaceId !== space.id || schedule.narrativeId !== space.narrativeId
- || document.narrativeId !== space.narrativeId) {
+ if (schedule.spaceId !== space.id || schedule.narrativeId !== document.narrativeId) {
throw new Error("Fine Caption inputs belong to different ProgramSpaces or Narratives");
}
const styles = new Map(program.styles.map((style) => [style.id, style]));
diff --git a/packages/caption-fine/test/caption-fine.test.ts b/packages/caption-fine/test/caption-fine.test.ts
index cae31e6e..af5d227d 100644
--- a/packages/caption-fine/test/caption-fine.test.ts
+++ b/packages/caption-fine/test/caption-fine.test.ts
@@ -127,7 +127,7 @@ test("Fine Caption schedules visibility outside semantic Word timing and cuts on
]);
const track = renderFineCaption(schedule, program, document,
- sealProgramSpace({ id: "test-space", narrativeId: "story", durationSec: 3, frameRate: { numerator: 30, denominator: 1 } }));
+ sealProgramSpace({ id: "test-space", durationSec: 3, frameRate: { numerator: 30, denominator: 1 } }));
assert.deepEqual(track.presents.map((present) => present.span), [
{ startFrame: 6, endFrameExclusive: 36 },
{ startFrame: 36, endFrameExclusive: 62 },
@@ -157,7 +157,7 @@ test("Fine Caption uniformly springs the whole Cue through exact scales", () =>
scheduleFineCaption(projection, animatedProgram, document),
animatedProgram,
document,
- sealProgramSpace({ id: "test-space", narrativeId: "story", durationSec: 3,
+ sealProgramSpace({ id: "test-space", durationSec: 3,
frameRate: { numerator: 30, denominator: 1 } }),
);
const cueMotion = track.presents[0]?.elements.find((element) => element.id === "cue-motion");
@@ -189,7 +189,7 @@ test("Fine Caption follows measured Role regions and hides null Frames", () => {
const { document, program, projection } = fixture(`<line>
<BOY>one two || three four
</line>`);
- const space = sealProgramSpace({ id: "test-space", narrativeId: "story", durationSec: 3,
+ const space = sealProgramSpace({ id: "test-space", durationSec: 3,
frameRate: { numerator: 30, denominator: 1 } });
const regions: SpatialRegionTimeline = {
canvas: { widthPx: 1080, heightPx: 1920, origin: "top-left", xDirection: "right", yDirection: "down", pixelAspect: "square" },
@@ -217,7 +217,7 @@ test("Fine Caption keeps authored placement when a Cue Role has no measured trac
const { document, program, projection } = fixture(`<line>
<BOY>one two || three four
</line>`);
- const space = sealProgramSpace({ id: "test-space", narrativeId: "story", durationSec: 3,
+ const space = sealProgramSpace({ id: "test-space", durationSec: 3,
frameRate: { numerator: 30, denominator: 1 } });
const regions: SpatialRegionTimeline = {
canvas: { widthPx: 1080, heightPx: 1920, origin: "top-left", xDirection: "right", yDirection: "down", pixelAspect: "square" },
@@ -241,7 +241,7 @@ test("Fine Caption rejects a Cue that exceeds its structural row budget instead
const schedule = scheduleFineCaption(projection, constrainedProgram, document);
assert.throws(
() => renderFineCaption(schedule, constrainedProgram, document,
- sealProgramSpace({ id: "test-space", narrativeId: "story", durationSec: 3,
+ sealProgramSpace({ id: "test-space", durationSec: 3,
frameRate: { numerator: 30, denominator: 1 } })),
/constructs 2 rows.+maximum is 1/u,
);
@@ -277,7 +277,7 @@ test("Fine Caption gives overlapping acoustic Words one current Karaoke owner",
scheduleFineCaption(overlapped, karaokeProgram, document),
karaokeProgram,
document,
- sealProgramSpace({ id: "test-space", narrativeId: "story", durationSec: 3,
+ sealProgramSpace({ id: "test-space", durationSec: 3,
frameRate: { numerator: 30, denominator: 1 } }),
);
const firstActive = track.presents[0]?.elements.find((element) => element.id === "atom-1-active");
diff --git a/packages/caption/README.md b/packages/caption/README.md
index b872989d..4197201e 100644
--- a/packages/caption/README.md
+++ b/packages/caption/README.md
@@ -39,9 +39,18 @@ The common public values are defined in [types.ts](src/types.ts), with exports i
- a Surface/Fragment connecting the common document, program and timing to those Producers.
`caption:Program` remains the common author Surface. `Use` selects exactly one of `role`, `selection`
-or `attribute`; `Mute` selects a Role or Selection. Overlapping unit Style applications are resolved
-in authored order, while word attributes produce separate `wordRuns`. This is explicit application,
-not CSS inheritance. A family chooses whether and how it supports word-specific runs.
+or `attribute`; `Mute` selects a Role or Selection. For Role/Selection applications, the last matching
+`Use` in Source order selects the complete Style for each unit. It replaces the previous choice
+without merging Style parameters. A Selection can cover a phrase, a whole Segment, several Segments
+or the program through its structural anchors; there is no separate Segment selector.
+
+Word attributes produce separate `wordRuns`. Conflicting Styles on the same attributed word are
+rejected; these runs do not use the Role/Selection override rule. A family chooses whether and how
+it supports word-specific runs.
+
+Mute applications combine their selected unit ids independently of Style order. They suppress those
+display words, preserving the speech and semantic timeline. Visible lead and tail still belong to
+the family schedule, so muting a word is not a time-based mask of every neighboring Cue.
Reuse `captionProducers.temporalizeDocument` with inputs `document`, `semantic` and `program`, or the
corresponding exported timing Fragment. It emits a `TimedCaptionProjection` whose Cues contain
@@ -60,3 +69,9 @@ changing Cue grouping, lead or tail again.
family replaces Fine's uniform-word layout and parameters, not the common speech truth. Its own
Manifest, activation and vocabulary belong to the project package; no change to this common package
is required merely to add a new visual family.
+
+That family can render a standalone Caption Track or coordinate caption text with graphics in a
+shared scene. Its output uses ordinary VisualTrack elements or a renderer program with explicit
+resources. Caption retains the authored words and their semantic relationship while the component
+owns the visual boundary. The Program does not automatically route different families to different
+renderers; choose the implementing component and its coverage explicitly.
diff --git a/packages/caption/src/display.ts b/packages/caption/src/display.ts
index 938016a0..15961d25 100644
--- a/packages/caption/src/display.ts
+++ b/packages/caption/src/display.ts
@@ -41,6 +41,8 @@ export type CaptionUnitSubset = {
function tokenBoundary(narrative: Narrative, anchorId: string, owner: string): number {
const anchor = narrative.semanticIndex.anchors.find((candidate) => candidate.id === anchorId);
if (anchor === undefined) throw new Error(`${owner} names unknown semantic anchor ${anchorId}`);
+ if (anchor.kind === "program-start") return 0;
+ if (anchor.kind === "program-end") return narrative.tokens.length;
const segment = narrative.segments.find((candidate) => candidate.id === anchor.segmentId);
if (segment === undefined) throw new Error(`${owner} names an anchor outside its Segment`);
if (anchor.kind === "segment-start") return segment.tokenStart;
diff --git a/packages/caption/src/manifest.ts b/packages/caption/src/manifest.ts
index f6e41408..af353071 100644
--- a/packages/caption/src/manifest.ts
+++ b/packages/caption/src/manifest.ts
@@ -55,7 +55,7 @@ export const captionMarkupSurfaces = [{
{ name: "default", kind: "reference", required: true, accepts: [captionTypes.style], summary: "The default Style." },
],
children: [
- { tag: "Use", cardinality: "many", summary: "Applies a Style to one Role or one complete semantic Selection.", attributes: [
+ { tag: "Use", cardinality: "many", summary: "Applies a complete Style by Role or semantic Selection, or supplies a word-attribute Style for a supporting family.", attributes: [
{ name: "style", kind: "reference", required: true, accepts: [captionTypes.style], summary: "The Style to apply." },
{ name: "role", kind: "literal", required: false, summary: "Selects all Caption units authored for this Role." },
{ name: "selection", kind: "reference", required: false, accepts: [narrativeTypes.selection], summary: "Selects complete Caption units contained by a semantic Selection." },
@@ -66,7 +66,18 @@ export const captionMarkupSurfaces = [{
{ name: "selection", kind: "reference", required: false, accepts: [narrativeTypes.selection], summary: "Selects complete Caption units contained by a semantic Selection." },
] },
],
- example: '<caption:Program id="captions" document={story.caption} narrative={story} default={plain}/> ',
+ example: `<caption:Program id="captions" document={story.caption} narrative={story} default={plain}>
+ <caption:Use role="GUEST" style={guest}/>
+ <caption:Use selection={story.selection.answer} style={answer}/>
+ <caption:Mute selection={story.selection.demonstration}/>
+</caption:Program>`,
+ notes: [
+ "Use selects exactly one of role, selection or attribute; Mute selects exactly one of role or selection.",
+ "For Role and Selection applications, the last matching Use in Source order selects the complete Style; properties are not merged.",
+ "Selections may cover a phrase, a whole Segment, several Segments or the program, preserving complete display/pronunciation units.",
+ "All Mute selections hide their words independently of Style order. They do not mute audio; the family still controls neighboring Cue lead and tail.",
+ "Word-attribute runs are separate from Role/Selection overrides and require a family that implements them; Fine does not. Conflicting Styles on one attributed word are rejected.",
+ ],
},
}] as const;
diff --git a/packages/caption/src/temporalize.ts b/packages/caption/src/temporalize.ts
index 05befba6..a439f0f2 100644
--- a/packages/caption/src/temporalize.ts
+++ b/packages/caption/src/temporalize.ts
@@ -15,7 +15,7 @@ export function temporalizeCaptionDocument(
assertCaptionDocument(document);
assertCaptionProgramForDocument(program, document);
const space = projectSemanticProgramSpace(semantic);
- if (document.narrativeId !== space.narrativeId) {
+ if (document.narrativeId !== semantic.narrativeId) {
throw new CaptionTimingError("CAPTION_NARRATIVE", "CaptionDocument and SemanticTrack belong to different Narratives.");
}
const styleByUnit = new Map(program.runs.flatMap((run) => run.unitIds.map((unitId) => [unitId, run.styleId] as const)));
@@ -68,7 +68,7 @@ export function temporalizeCaptionDocument(
flush();
const result: TimedCaptionProjection = {
spaceId: space.id,
- narrativeId: space.narrativeId,
+ narrativeId: semantic.narrativeId,
documentId: document.id,
cues,
};
diff --git a/packages/caption/test/caption.test.ts b/packages/caption/test/caption.test.ts
index 8b2eed9e..1586d712 100644
--- a/packages/caption/test/caption.test.ts
+++ b/packages/caption/test/caption.test.ts
@@ -42,6 +42,41 @@ test("Caption styles one Role across non-contiguous authored Cues", () => {
assert.deepEqual(program.runs.map((run) => run.styleId), ["guy", "plain", "guy"]);
});
+test("Caption selects complete Segments and the program through structural anchors", () => {
+ const parsed = parseScript("caption-ranges.svml", `~@whole
+@opening <intro><HOST>One idea.</intro> @/opening
+<answer><GUEST>Another view.</answer>
+@/whole~`);
+ const narrative = narrativeValue(parsed, "story") as unknown as Narrative;
+ const document = captionDocument(parsed, "story.caption", "story");
+ const whole = narrative.selections.find((selection) => selection.id === "whole")!;
+ const opening = narrative.selections.find((selection) => selection.id === "opening")!;
+ assert.deepEqual(captionUnitsForSelection(document, narrative, whole).unitIds,
+ document.units.map((unit) => unit.id));
+ assert.deepEqual(captionUnitsForSelection(document, narrative, opening).unitIds,
+ document.units.filter((unit) => unit.segmentId === "intro").map((unit) => unit.id));
+});
+
+test("Caption applications use authored order and styling does not undo a display mute", () => {
+ const parsed = parseScript("caption-coverage.svml",
+ "<line><HOST>First. <GUEST>@answer Another view. @/answer</line>");
+ const narrative = narrativeValue(parsed, "story") as unknown as Narrative;
+ const document = captionDocument(parsed, "story.caption", "story");
+ const makeStyle = (id: string) => sealCaptionStyle({ id, rendering: { family: "test", parameters: {} } });
+ const base = makeStyle("base"), guest = makeStyle("guest"), accent = makeStyle("accent");
+ const selected = captionUnitsForSelection(document, narrative, narrative.selections[0]!).unitIds;
+ const roleUse = { id: "guest-role", unitIds: captionUnitsForRole(document, "GUEST").unitIds, style: guest };
+ const rangeUse = { id: "answer-range", unitIds: selected, style: accent };
+ const resolve = (applications: typeof roleUse[]) => resolveCaptionProgram(document, narrative, "captions",
+ base, applications, [{ id: "hidden-answer", unitIds: selected }]);
+ const rangeLast = resolve([roleUse, rangeUse]);
+ assert.deepEqual(rangeLast.runs.map((run) => run.styleId), ["base", "accent"]);
+ const roleLast = resolve([rangeUse, roleUse]);
+ assert.deepEqual(roleLast.runs.map((run) => run.styleId), ["base", "guest"]);
+ assert.deepEqual(rangeLast.mutedUnitIds, selected);
+ assert.deepEqual(roleLast.mutedUnitIds, selected);
+});
+
test("Caption keeps Selection styles and adds flat word-attribute styles separately", () => {
const parsed = parseScript("caption-attributes.svml", "<line>@focus really{emphasis} matters @/focus</line>");
const narrative = narrativeValue(parsed, "story") as unknown as Narrative;
diff --git a/packages/comment-sticker/README.md b/packages/comment-sticker/README.md
index 796c4714..5ab877a5 100644
--- a/packages/comment-sticker/README.md
+++ b/packages/comment-sticker/README.md
@@ -1,7 +1,11 @@
# `@hypit/comment-sticker`
-An optional author package for timed social-comment cards. It lowers every sticker to one ordinary,
-self-contained `VisualTrack`; it has no privileged Film behavior and never samples a sibling Track.
+The Track Surface accepts `semantic={speech.semantic}` or `space={animation}`. Semantic context
+resolves Script Selections and Moments; authored space supports clock-based animation. Shared `at`
+inputs accept a Moment or a time such as `2s`; `at` with `for` produces a Window where required.
+
+An author package for timed social-comment cards. It publishes an ordinary `VisualTrack` containing
+the cards' independently timed appearances.
The author surface keeps the three independent concerns visible:
diff --git a/packages/comment-sticker/src/fragment.ts b/packages/comment-sticker/src/fragment.ts
index 452b4a8f..3aac04e7 100644
--- a/packages/comment-sticker/src/fragment.ts
+++ b/packages/comment-sticker/src/fragment.ts
@@ -1,9 +1,9 @@
+import { programSpaceTypes } from "@hypit/program-space";
import { artifactTypes } from "@hypit/artifact";
import { compositionTypes } from "@hypit/composition";
import { sealGraphFragment } from "@hypit/elaborator";
import type { FragmentOperation } from "@hypit/elaborator";
import type { TypeRef } from "@hypit/protocol";
-import { semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";
import { spatialTypes } from "@hypit/spatial";
import { textTypes } from "@hypit/text";
import { temporalTypes } from "@hypit/temporal";
@@ -33,12 +33,6 @@ export function createCommentStickerFragment(items: readonly CommentStickerFragm
if (items.length === 0) throw new Error("Comment Sticker Fragment requires at least one Item.");
const types = new Map<string, TypeRef>();
const operations: FragmentOperation[] = [
- {
- id: "comment:space",
- producer: semanticTrackProducers.projectProgramSpace,
- inputs: { track: input("semantic") },
- result: { kind: "output", name: "space" },
- },
{
id: "comment:set:empty",
producer: commentStickerProducers.createSet,
@@ -88,7 +82,7 @@ export function createCommentStickerFragment(items: readonly CommentStickerFragm
inputs: {
set: operation(current),
header: input("header"),
- space: operation("comment:space"),
+ space: input("space"),
frame: input(item.frameName),
style: input(item.styleName),
window: input(item.windowName),
@@ -110,7 +104,7 @@ export function createCommentStickerFragment(items: readonly CommentStickerFragm
{
id: "comment:track",
producer: commentStickerProducers.render,
- inputs: { canvas: input("canvas"), space: operation("comment:space"), program: operation("comment:program") },
+ inputs: { canvas: input("canvas"), space: input("space"), program: operation("comment:program") },
result: { kind: "output", name: "track" },
},
);
@@ -118,7 +112,7 @@ export function createCommentStickerFragment(items: readonly CommentStickerFragm
inputs: [
{ name: "canvas", type: spatialTypes.canvas },
{ name: "header", type: commentStickerTypes.header },
- { name: "semantic", type: semanticTrackTypes.track },
+ { name: "space", type: programSpaceTypes.programSpace },
...[...types].map(([inputName, type]) => ({ name: inputName, type })),
],
operations,
diff --git a/packages/comment-sticker/src/manifest.ts b/packages/comment-sticker/src/manifest.ts
index b8d83abb..3d871a77 100644
--- a/packages/comment-sticker/src/manifest.ts
+++ b/packages/comment-sticker/src/manifest.ts
@@ -1,3 +1,4 @@
+import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";
import { readFile } from "node:fs/promises";
import { artifactDependency, artifactTypes } from "@hypit/artifact";
@@ -6,7 +7,7 @@ import { fontArtifactSchema, mediaDependency, mediaTypes } from "@hypit/media";
import { narrativeDependency } from "@hypit/narrative";
import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";
import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";
-import { semanticTrackDependency, semanticTrackTypes } from "@hypit/semantic-track";
+import { semanticTrackDependency } from "@hypit/semantic-track";
import { spatialDependency, spatialFrameSchema, spatialTypes } from "@hypit/spatial";
import { svsRecipeType } from "@hypit/svs";
import { temporalDependency, temporalTypes } from "@hypit/temporal";
@@ -276,8 +277,7 @@ export const commentStickerMarkupSurfaces = [
summary: "Names the Track and prefixes every binding it publishes." },
{ name: "canvas", kind: "reference", required: true, accepts: [spatialTypes.canvas],
summary: "Chooses the Canvas the cards are laid out on." },
- { name: "semantic", kind: "reference", required: true, accepts: [semanticTrackTypes.track],
- summary: "Chooses the SemanticTrack that owns the Program frame domain and resolves every temporal binding." },
+ ...temporalContextAttributeVocabulary,
],
children: [
{ tag: "Sticker", cardinality: "many",
diff --git a/packages/comment-sticker/src/surface.ts b/packages/comment-sticker/src/surface.ts
index 2cc2e31d..9ec9e638 100644
--- a/packages/comment-sticker/src/surface.ts
+++ b/packages/comment-sticker/src/surface.ts
@@ -1,7 +1,7 @@
+import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";
import { artifactTypes } from "@hypit/artifact";
import { mediaTypes } from "@hypit/media";
import type { FontStackRef } from "@hypit/media";
-import { semanticTrackTypes } from "@hypit/semantic-track";
import { spatialTypes } from "@hypit/spatial";
import { svsRecipeType } from "@hypit/svs";
import type { SvsRecipe } from "@hypit/svs";
@@ -91,7 +91,8 @@ export const decodeCommentStickerTrackSurface: StructuredSurfaceHandler = ({ ele
allowed(element, ["id", "canvas", "semantic"], ["id", "canvas", "semantic"]);
const id = text(element, "id");
const canvas = reference(element.attributes.canvas, `${element.name}.canvas`, spatialTypes.canvas, resolveReference);
- const semantic = reference(element.attributes.semantic, `${element.name}.semantic`, semanticTrackTypes.track, resolveReference);
+ const context = resolveTemporalContext({ element, resolveReference });
+ const time = createTemporalSpace({ id: id, element, ...context });
const headerId = `${id}.__header`;
const records: SurfaceRecordDraft[] = [{
id: headerId,
@@ -99,9 +100,9 @@ export const decodeCommentStickerTrackSurface: StructuredSurfaceHandler = ({ ele
value: { kind: "inline", value: sealCommentStickerHeader({ id }) },
range: element.range,
}];
- const temporalComponents: SurfaceComponentDraft[] = [];
- const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [];
- const inputs: Record<string, typeof canvas.ref> = { canvas: canvas.ref, header: { kind: "record", id: headerId }, semantic: semantic.ref };
+ const temporalComponents: SurfaceComponentDraft[] = [...time.components];
+ const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [...time.fragments];
+ const inputs: Record<string, typeof canvas.ref> = { canvas: canvas.ref, header: { kind: "record", id: headerId }, space: time.space.ref };
const items: Parameters<typeof createCommentStickerFragment>[0][number][] = [];
for (const child of element.children) {
if (child.kind === "text") {
@@ -112,7 +113,7 @@ export const decodeCommentStickerTrackSurface: StructuredSurfaceHandler = ({ ele
if (child.children.some((node) => node.kind === "element")) throw new Error(`${child.name} accepts plain comment text only.`);
allowed(child, ["id", "comment", "frame", "style", "avatar", "author", "header", "meta", ...TIMING], ["id", "frame", "style"]);
const itemId = text(child, "id");
- const temporal = createTemporalWindowProjection({ id: itemId, element: child, semantic, resolveReference });
+ const temporal = createTemporalWindowProjection({ id: itemId, element: child, ...context, space: time.space, resolveReference });
records.push(...temporal.records); temporalComponents.push(...temporal.components); temporalFragments.push(...temporal.fragments);
const suffix = String(items.length + 1).padStart(4, "0");
const frame = reference(child.attributes.frame, `${child.name}.frame`, spatialTypes.frame, resolveReference);
diff --git a/packages/comment-sticker/test/comment-sticker.test.ts b/packages/comment-sticker/test/comment-sticker.test.ts
index fd9eae00..8c1e8c82 100644
--- a/packages/comment-sticker/test/comment-sticker.test.ts
+++ b/packages/comment-sticker/test/comment-sticker.test.ts
@@ -50,7 +50,7 @@ const style = decodeCommentStickerStyle(recipe, fonts, "social-comment");
const frame = { xPx: 80, yPx: 140, widthPx: 920, heightPx: 360 };
const canvas = { widthPx: 1080, heightPx: 1920,
origin: "top-left" as const, xDirection: "right" as const, yDirection: "down" as const, pixelAspect: "square" as const };
-const space = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative", durationSec: 3, frameRate: { numerator: 30, denominator: 1 } });
+const space = sealProgramSpace({ id: "test-space", durationSec: 3, frameRate: { numerator: 30, denominator: 1 } });
const header = sealCommentStickerHeader({ id: "comments" });
const semantic = semanticTrackFixture(space);
diff --git a/packages/composition/test/render.test.ts b/packages/composition/test/render.test.ts
index ce2f758e..1face20e 100644
--- a/packages/composition/test/render.test.ts
+++ b/packages/composition/test/render.test.ts
@@ -15,8 +15,7 @@ import {
} from "../src/index.js";
import type { VisualTrack } from "../src/index.js";
-const space = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 2,
+const space = sealProgramSpace({ id: "test-space", durationSec: 2,
frameRate: { numerator: 30, denominator: 1 },
});
diff --git a/packages/composition/test/track.test.ts b/packages/composition/test/track.test.ts
index 136cb43e..7ebbf09b 100644
--- a/packages/composition/test/track.test.ts
+++ b/packages/composition/test/track.test.ts
@@ -46,8 +46,7 @@ const audio: BlobRef = {
};
function fixture() {
- const programSpace = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 4,
+ const programSpace = sealProgramSpace({ id: "test-space", durationSec: 4,
frameRate: { numerator: 30, denominator: 1 },
});
const visual = sealVisualTrack({ programSpaceId: "test-space",
@@ -220,8 +219,7 @@ test("the Type owner rejects an invalid VisualTrack at the shared admission gate
test("Composition validates Track frame ranges against the explicitly connected ProgramSpace", () => {
const { programSpace, visual } = fixture();
- const foreign = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 1,
+ const foreign = sealProgramSpace({ id: "test-space", durationSec: 1,
frameRate: { numerator: 24, denominator: 1 },
});
const composition = sealComposition({
diff --git a/packages/deck-track/README.md b/packages/deck-track/README.md
index e0297df1..53461abb 100644
--- a/packages/deck-track/README.md
+++ b/packages/deck-track/README.md
@@ -1,5 +1,9 @@
# `@hypit/deck-track`
+The Track Surface accepts `semantic={speech.semantic}` or `space={animation}`. Semantic context
+resolves Script Selections and Moments; authored space supports clock-based animation. Shared `at`
+inputs accept a Moment or a time such as `2s`; `at` with `for` produces a Window where required.
+
The official, deliberately narrow depth-stack Deck author package.
`DepthStack` owns ordered Cards, semantic activation points, a finite visible neighborhood,
diff --git a/packages/deck-track/src/fragment.ts b/packages/deck-track/src/fragment.ts
index 8e318df1..1e052e2a 100644
--- a/packages/deck-track/src/fragment.ts
+++ b/packages/deck-track/src/fragment.ts
@@ -1,10 +1,10 @@
+import { programSpaceTypes } from "@hypit/program-space";
import { artifactTypes } from "@hypit/artifact";
import { compositionTypes } from "@hypit/composition";
import { sealGraphFragment } from "@hypit/elaborator";
import type { FragmentOperation, GraphFragment } from "@hypit/elaborator";
import { mediaTypes } from "@hypit/media";
import { mediaTrackProducers, mediaTrackTypes } from "@hypit/media-track";
-import { semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";
import { spatialTypes } from "@hypit/spatial";
import { temporalTypes } from "@hypit/temporal";
@@ -42,16 +42,10 @@ export function createDepthStackFragment(
{ name: "canvas", type: spatialTypes.canvas },
{ name: "frame", type: spatialTypes.frame },
{ name: "header", type: depthStackTypes.header },
- { name: "semantic", type: semanticTrackTypes.track },
+ { name: "space", type: programSpaceTypes.programSpace },
{ name: "spec", type: depthStackTypes.spec },
];
const operations: FragmentOperation[] = [
- {
- id: "space",
- producer: semanticTrackProducers.projectProgramSpace,
- inputs: { track: input("semantic") },
- result: { kind: "output", name: "space" },
- },
{
id: "cards",
producer: depthStackProducers.createCards,
@@ -113,7 +107,7 @@ export function createDepthStackFragment(
producer: depthStackProducers.appendCard,
inputs: {
set: cardSet,
- space: operation("space"),
+ space: input("space"),
material: operation(sampleId),
label: input(card.labelName),
spec: input(card.cardSpecName),
@@ -132,7 +126,7 @@ export function createDepthStackFragment(
header: input("header"),
frame: input("frame"),
spec: input("spec"),
- space: operation("space"),
+ space: input("space"),
terminal: input(terminalName),
},
result: { kind: "output", name: "program" },
@@ -140,7 +134,7 @@ export function createDepthStackFragment(
operations.push({
id: "track",
producer: depthStackProducers.render,
- inputs: { canvas: input("canvas"), space: operation("space"), program: operation("program") },
+ inputs: { canvas: input("canvas"), space: input("space"), program: operation("program") },
result: { kind: "output", name: "track" },
});
return sealGraphFragment({
diff --git a/packages/deck-track/src/manifest.ts b/packages/deck-track/src/manifest.ts
index 4d8af303..05ace84e 100644
--- a/packages/deck-track/src/manifest.ts
+++ b/packages/deck-track/src/manifest.ts
@@ -1,3 +1,4 @@
+import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";
import { readFile } from "node:fs/promises";
import { artifactTypes } from "@hypit/artifact";
@@ -20,7 +21,7 @@ import {
import { narrativeDependency, narrativeTypes } from "@hypit/narrative";
import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";
import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";
-import { semanticTrackDependency, semanticTrackTypes } from "@hypit/semantic-track";
+import { semanticTrackDependency } from "@hypit/semantic-track";
import { spatialDependency, spatialFrameSchema, spatialTypes } from "@hypit/spatial";
import { svsRecipeType } from "@hypit/svs";
import { temporalDependency, temporalTypes } from "@hypit/temporal";
@@ -168,8 +169,7 @@ export const depthStackMarkupSurfaces = [
attributes: [
{ name: "id", kind: "identifier", required: true,
summary: "Names this deck so its Program and Track can be referenced elsewhere in the Source." },
- { name: "semantic", kind: "reference", required: true, accepts: [semanticTrackTypes.track],
- summary: "Chooses the SemanticTrack that owns the frame domain and places each Card's Moment." },
+ ...temporalContextAttributeVocabulary,
{ name: "canvas", kind: "reference", required: true, accepts: [spatialTypes.canvas],
summary: "Chooses the Canvas the deck is rendered into." },
{ name: "frame", kind: "reference", required: true, accepts: [spatialTypes.frame],
@@ -314,15 +314,15 @@ export const depthStackMarkupSurfaces = [
{ name: "playback-past", required: false, values: ["hold-tail", "continue", "hide"], fallback: "hold-tail",
summary: "Decides what a Card shows once its material is spent: its last frame held, its own timeline running on, or nothing." },
] },
- { name: "until", kind: "expression", required: true, values: ["program.end"],
- accepts: [narrativeTypes.moment, narrativeTypes.selection],
- summary: "Ends the deck at the literal `program.end`, at a Moment, or at a Selection." },
+ { name: "until", kind: "expression", required: true,
+ accepts: [narrativeTypes.moment, narrativeTypes.selection, narrativeTypes.excerpt],
+ summary: "Ends the deck at a Moment, a Selection or Segment boundary, or an authored time such as 8s." },
{ name: "until-boundary", kind: "literal", required: false, values: ["start", "end"],
- summary: "Chooses which edge of the ending Selection ends the deck." },
+ summary: "Chooses which edge of the ending Selection or Segment ends the deck." },
],
children: [
{ tag: "Card", cardinality: "many",
- summary: "One card of the stack, dealt at its own Moment in document order.",
+ summary: "One card of the stack, dealt at its own event in document order.",
attributes: [
{ name: "id", kind: "identifier", required: true,
summary: "Names this Card within the deck." },
@@ -399,7 +399,7 @@ export const depthStackMarkupSurfaces = [
notes: [
"The deck requires at least one Card, and a Card is empty.",
"`extent` is required for a still image source and refused for a Synchronized Medium or a Compositable Surface.",
- "`until-boundary` defaults to `end` and is refused unless `until` names a Selection.",
+ "`until-boundary` defaults to `end` when `until` names a Selection or Segment.",
"Both Recipes are closed: any property outside the ones listed here is refused, and a Card without its own `appearance` reads the deck's Recipe for fit, sampling, frame Paint and playback.",
],
} },
diff --git a/packages/deck-track/src/surface.ts b/packages/deck-track/src/surface.ts
index 7e864903..fa16fc28 100644
--- a/packages/deck-track/src/surface.ts
+++ b/packages/deck-track/src/surface.ts
@@ -1,9 +1,9 @@
+import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";
import { artifactTypes } from "@hypit/artifact";
import { mediaTypes } from "@hypit/media";
import type { FontStackRef } from "@hypit/media";
import { mediaTrackTypes } from "@hypit/media-track";
import type { TypeRef } from "@hypit/protocol";
-import { semanticTrackTypes } from "@hypit/semantic-track";
import { spatialTypes } from "@hypit/spatial";
import { svsRecipeType } from "@hypit/svs";
import { createTemporalInstantProjection, temporalInstantAttributeNames } from "@hypit/temporal-markup";
@@ -181,27 +181,28 @@ export const decodeDepthStackLabelSurface: StructuredSurfaceHandler = ({ element
};
export const decodeDepthStackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
- allowed(element, ["id", "semantic", "canvas", "frame", "appearance", "until", "until-boundary"]);
+ allowed(element, ["id", "semantic", "space", "canvas", "frame", "appearance", "until", "until-boundary"]);
const id = text(element, "id");
- const semantic = reference(element.attributes.semantic, `${element.name}.semantic`, semanticTrackTypes.track, resolveReference);
+ const context = resolveTemporalContext({ element, resolveReference });
+ const time = createTemporalSpace({ id: id, element, ...context });
const canvas = reference(element.attributes.canvas, `${element.name}.canvas`, spatialTypes.canvas, resolveReference);
const frame = reference(element.attributes.frame, `${element.name}.frame`, spatialTypes.frame, resolveReference);
const appearance = recipe(element.attributes.appearance, `${element.name}.appearance`, resolveReference);
const terminal = createTemporalInstantProjection({
- id: `${id}.terminal`, subjectId: id, element, semantic, resolveReference,
+ id: `${id}.terminal`, subjectId: id, element, ...context, space: time.space, resolveReference,
semanticAttribute: "until", boundaryAttribute: "until-boundary", boundaryFallback: "end", projectedAttribute: false,
});
const records: SurfaceRecordDraft[] = [...terminal.records];
- const temporalComponents: SurfaceComponentDraft[] = [...terminal.components];
- const temporalFragments = [...terminal.fragments];
+ const temporalComponents: SurfaceComponentDraft[] = [...time.components, ...terminal.components];
+ const temporalFragments = [...time.fragments, ...terminal.fragments];
const headerId = `${id}.header`;
const specId = `${id}.spec`;
records.push(
{ id: headerId, type: depthStackTypes.header, value: { kind: "inline", value: sealDepthStackHeader({ id }) }, range: element.range },
{ id: specId, type: depthStackTypes.spec, value: { kind: "inline", value: decodeDepthStackSpec(appearance) }, range: element.range },
);
- const inputs: Record<string, typeof semantic.ref> = {
- canvas: canvas.ref, frame: frame.ref, header: { kind: "record", id: headerId }, semantic: semantic.ref,
+ const inputs: Record<string, typeof time.space.ref> = {
+ canvas: canvas.ref, frame: frame.ref, header: { kind: "record", id: headerId }, space: time.space.ref,
spec: { kind: "record", id: specId }, terminal: terminal.ref,
};
const cards: DepthStackFragmentCard[] = [];
@@ -224,7 +225,7 @@ export const decodeDepthStackSurface: StructuredSurfaceHandler = ({ element, res
: reference(child.attributes.extent, `${child.name}.extent`, spatialTypes.extent, resolveReference);
if (sourceKind === "still" && extent === undefined) throw new Error(`${child.name}.extent is required for a still image.`);
if (sourceKind !== "still" && extent !== undefined) throw new Error(`${child.name}.extent belongs only to a still image.`);
- const activation = createTemporalInstantProjection({ id: `${id}.${cardId}`, subjectId: cardId, element: child, semantic, resolveReference });
+ const activation = createTemporalInstantProjection({ id: `${id}.${cardId}`, subjectId: cardId, element: child, ...context, space: time.space, resolveReference });
records.push(...activation.records); temporalComponents.push(...activation.components); temporalFragments.push(...activation.fragments);
const cardAppearance = child.attributes.appearance === undefined
? appearance : recipe(child.attributes.appearance, `${child.name}.appearance`, resolveReference);
@@ -248,7 +249,7 @@ export const decodeDepthStackSurface: StructuredSurfaceHandler = ({ element, res
inputs[sampleSpecName] = { kind: "record", id: sampleId };
inputs[cardSpecName] = { kind: "record", id: cardSpecId };
inputs[activationName] = activation.ref;
- let labelRef: typeof semantic.ref;
+ let labelRef: typeof time.space.ref;
if (child.attributes.label === undefined) {
const labelId = `${id}.card.${suffix}.label-none`;
records.push({ id: labelId, type: depthStackTypes.cardLabel, value: { kind: "inline", value: noDepthStackCardLabel() }, range: child.range });
diff --git a/packages/deck-track/test/deck-track.test.ts b/packages/deck-track/test/deck-track.test.ts
index db282e0f..751ff0e6 100644
--- a/packages/deck-track/test/deck-track.test.ts
+++ b/packages/deck-track/test/deck-track.test.ts
@@ -55,8 +55,7 @@ import type {
import { artifactTypes } from "@hypit/artifact";
import { mediaTrackManifest, mediaTrackTypes } from "@hypit/media-track";
-const space = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 2,
+const space = sealProgramSpace({ id: "test-space", durationSec: 2,
frameRate: { numerator: 30, denominator: 1 },
});
const semantic = semanticTrackFixture(space);
@@ -208,7 +207,7 @@ function momentPoint(id: string, frameValue: number): TemporalInstant {
return {
id: `${id}::moment`,
subjectId: id,
- source: { spaceId: space.id, narrativeId: space.narrativeId, kind: "moment", id },
+ source: { spaceId: space.id, narrativeId: "script", kind: "moment", id },
projection: { ref: "moment.cue" },
frame: frameValue,
authority: { kind: "fixed" },
@@ -219,7 +218,7 @@ function programPoint(subjectId: string, endFrame: number): TemporalInstant {
return {
id: "program::program",
subjectId,
- source: { spaceId: space.id, narrativeId: space.narrativeId, kind: "program", id: "program" },
+ source: { spaceId: space.id, narrativeId: "script", kind: "program", id: "program" },
projection: { ref: "program.end" },
frame: endFrame,
authority: { kind: "fixed" },
diff --git a/packages/film-studio/package.json b/packages/film-studio/package.json
index 2880d5a6..e53f4218 100644
--- a/packages/film-studio/package.json
+++ b/packages/film-studio/package.json
@@ -10,6 +10,7 @@
"@hypit/composition": "workspace:*",
"@hypit/film": "workspace:*",
"@hypit/semantic-track": "workspace:*",
- "@hypit/studio-adapter": "workspace:*"
+ "@hypit/studio-adapter": "workspace:*",
+ "@hypit/program-space": "workspace:*"
}
}
diff --git a/packages/film-studio/src/index.ts b/packages/film-studio/src/index.ts
index fdbca4f9..e961d120 100644
--- a/packages/film-studio/src/index.ts
+++ b/packages/film-studio/src/index.ts
@@ -1,13 +1,14 @@
+import { programSpaceTypes } from "@hypit/program-space";
import type { StudioFilmCompanion } from "@hypit/studio-adapter";
import { compositionTypes } from "@hypit/composition";
import { filmModuleRef } from "@hypit/film";
import { semanticTrackTypes } from "@hypit/semantic-track";
-/** Film alone owns the author vocabulary that selects one semantic axis and its peer Tracks. */
+/** Film owns the author vocabulary that selects its time source and peer Tracks. */
export const filmStudioCompanions: readonly StudioFilmCompanion[] = [{
id: "film",
match: { module: filmModuleRef, surface: "film", outputType: compositionTypes.composition },
- semantic: { attribute: "semantic", type: semanticTrackTypes.track },
+ timeSources: [{ attribute: "semantic", type: semanticTrackTypes.track }, { attribute: "space", type: programSpaceTypes.programSpace }],
tracks: {
childSurface: "Track",
sourceAttribute: "source",
diff --git a/packages/film/README.md b/packages/film/README.md
index 5fdf2d3d..29ae3f1c 100644
--- a/packages/film/README.md
+++ b/packages/film/README.md
@@ -1,5 +1,9 @@
# `@hypit/film`
+The Surface accepts `semantic={speech.semantic}` for performance time or `space={animation}` for an
+authored ProgramSpace. Its Fragment receives `space` directly; rendering does not require Script
+or prepared performance media. Both time sources use the same frame and audio pipeline.
+
Official package-owned assembly layer between peer Tracks and the generic Composition waist.
Film is an ordinary author component and an optional Target, not a Core root or a privileged video
type.
@@ -16,8 +20,8 @@ remains order-independent because the public contract canonicalizes Tracks and v
belongs to each Present's absolute stacking key.
The official Structured Surface validates an imported generic SVS Recipe into a nominal
-`FilmProgram`, receives CanvasSpace and SemanticTrack through separate explicit edges, projects
-ProgramSpace from that SemanticTrack, type-checks
+`FilmProgram`, receives CanvasSpace and the selected time source through separate explicit edges,
+projects ProgramSpace when that source is a SemanticTrack, type-checks
each `<film:Track source={...}/>` reference and generates the finite fold. CanvasSpace is the only
dimension truth, ProgramSpace is the only frame-rate truth, and FilmProgram owns only assembly
identity and clear color.
diff --git a/packages/film/package.json b/packages/film/package.json
index 305ea3c1..72014143 100644
--- a/packages/film/package.json
+++ b/packages/film/package.json
@@ -19,7 +19,8 @@
"@hypit/semantic-track": "workspace:*",
"@hypit/spatial": "workspace:*",
"@hypit/svs": "workspace:*",
- "@hypit/markup": "workspace:*"
+ "@hypit/markup": "workspace:*",
+ "@hypit/temporal-markup": "workspace:*"
},
"devDependencies": {
"@hypit/core": "workspace:*",
diff --git a/packages/film/src/fragment.ts b/packages/film/src/fragment.ts
index f3e26c20..305e1743 100644
--- a/packages/film/src/fragment.ts
+++ b/packages/film/src/fragment.ts
@@ -1,4 +1,4 @@
-import { semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";
+import { programSpaceTypes } from "@hypit/program-space";
import { spatialTypes } from "@hypit/spatial";
import { compositionTypes } from "@hypit/composition";
import type { Track } from "@hypit/composition";
@@ -12,7 +12,7 @@ const input = (name: string) => ({ kind: "fragment-input" as const, name });
const operation = (id: string) => ({ kind: "fragment-operation" as const, operation: id });
function assertTrackInputs(tracks: readonly FilmTrackInput[]): FilmTrackInput[] {
- const names = new Set(["program", "canvas", "semantic"]);
+ const names = new Set(["program", "canvas", "space"]);
return [...tracks]
.map((track) => ({ name: track.name.trim(), kind: track.kind }))
.sort((left, right) => left.name.localeCompare(right.name))
@@ -32,12 +32,6 @@ function assertTrackInputs(tracks: readonly FilmTrackInput[]): FilmTrackInput[]
export function createFilmAssemblyFragment(options: FilmAssemblyFragmentOptions) {
const tracks = assertTrackInputs(options.tracks);
const operations: FragmentOperation[] = [
- {
- id: "film:space",
- producer: semanticTrackProducers.projectProgramSpace,
- inputs: { track: input("semantic") },
- result: { kind: "output" as const, name: "space" },
- },
{
id: "track-set:empty",
producer: filmProducers.createTrackSet,
@@ -51,7 +45,7 @@ export function createFilmAssemblyFragment(options: FilmAssemblyFragmentOptions)
operations.push({
id,
producer: track.kind === "visual" ? filmProducers.appendVisualTrack : filmProducers.appendAudioTrack,
- inputs: { set: operation(current), space: operation("film:space"), track: input(track.name) },
+ inputs: { set: operation(current), space: input("space"), track: input(track.name) },
result: { kind: "output" as const, name: "set" },
});
current = id;
@@ -62,7 +56,7 @@ export function createFilmAssemblyFragment(options: FilmAssemblyFragmentOptions)
inputs: {
program: input("program"),
canvas: input("canvas"),
- space: operation("film:space"),
+ space: input("space"),
set: operation(current),
},
result: { kind: "output" as const, name: "composition" },
@@ -71,7 +65,7 @@ export function createFilmAssemblyFragment(options: FilmAssemblyFragmentOptions)
inputs: [
{ name: "program", type: filmTypes.program },
{ name: "canvas", type: spatialTypes.canvas },
- { name: "semantic", type: semanticTrackTypes.track },
+ { name: "space", type: programSpaceTypes.programSpace },
...tracks.map((track) => ({
name: track.name,
type: track.kind === "visual" ? compositionTypes.visualTrack : compositionTypes.audioTrack,
diff --git a/packages/film/src/manifest.ts b/packages/film/src/manifest.ts
index d37b0578..b063f4c5 100644
--- a/packages/film/src/manifest.ts
+++ b/packages/film/src/manifest.ts
@@ -1,7 +1,8 @@
+import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";
import { audioTrackSchema, compositionDependency, compositionTypes, visualTrackSchema } from "@hypit/composition";
import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";
import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";
-import { semanticTrackDependency, semanticTrackTypes } from "@hypit/semantic-track";
+import { semanticTrackDependency } from "@hypit/semantic-track";
import { spatialDependency, spatialTypes } from "@hypit/spatial";
import { svsManifest, svsModuleRef, svsRecipeType } from "@hypit/svs";
@@ -55,9 +56,7 @@ export const filmMarkupSurfaces = [{
{ name: "canvas", kind: "reference", required: true,
accepts: [spatialTypes.canvas],
summary: "Selects the CanvasSpace that decides the Composition's dimensions." },
- { name: "semantic", kind: "reference", required: true,
- accepts: [semanticTrackTypes.track],
- summary: "Selects the SemanticTrack that decides the Composition's duration and frame rate." },
+ ...temporalContextAttributeVocabulary,
{ name: "appearance", kind: "reference", required: true,
accepts: [svsRecipeType],
summary: "Selects the SVS Recipe that decides the clear color behind every Track.",
diff --git a/packages/film/src/surface.ts b/packages/film/src/surface.ts
index 41d5ae2a..45618313 100644
--- a/packages/film/src/surface.ts
+++ b/packages/film/src/surface.ts
@@ -1,4 +1,4 @@
-import { semanticTrackTypes } from "@hypit/semantic-track";
+import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";
import { spatialTypes } from "@hypit/spatial";
import { compositionTypes } from "@hypit/composition";
import type { AudioTrack, Track, VisualTrack } from "@hypit/composition";
@@ -93,16 +93,13 @@ function trackKind(reference: SurfaceResolvedReference): "visual" | "audio" {
}
export const decodeFilmSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
- exactAttributes(element, ["id", "canvas", "semantic", "appearance"]);
+ exactAttributes(element, ["id", "canvas", element.attributes.space === undefined ? "semantic" : "space", "appearance"]);
const id = stringAttribute(element, "id");
const canvas = requiredReference(element, "canvas", resolveReference);
if (!sameType(canvas.type, spatialTypes.canvas)) {
throw new Error(`${element.name}.canvas must reference CanvasSpace`);
}
- const semantic = requiredReference(element, "semantic", resolveReference);
- if (!sameType(semantic.type, semanticTrackTypes.track)) {
- throw new Error(`${element.name}.semantic must reference SemanticTrack`);
- }
+ const time = createTemporalSpace({ id, element, ...resolveTemporalContext({ element, resolveReference }) });
const appearanceReference = requiredReference(element, "appearance", resolveReference);
const appearance = filmAppearanceFromRecipe(
recipe(appearanceReference, `${element.name}.appearance`).properties,
@@ -132,18 +129,18 @@ export const decodeFilmSurface: StructuredSurfaceHandler = ({ element, resolveRe
value: { kind: "inline", value: program },
range: element.range,
}],
- components: [{
+ components: [...time.components, {
id,
fragment: fragment.id,
inputs: {
program: { kind: "record", id: programId },
canvas: canvas.ref,
- semantic: semantic.ref,
+ space: time.space.ref,
...Object.fromEntries(tracks.map((track) => [track.name, track.source.ref])),
},
outputs: { composition: `${id}.composition` },
range: element.range,
}],
- fragments: [fragment],
+ fragments: [...time.fragments, fragment],
};
};
diff --git a/packages/film/test/film.test.ts b/packages/film/test/film.test.ts
index e8cef9a7..e098b06e 100644
--- a/packages/film/test/film.test.ts
+++ b/packages/film/test/film.test.ts
@@ -34,8 +34,7 @@ import { renderTypographyTrack, sealTypographyTrackProgram, stillTextMotion, typ
import type { TextStyle } from "@hypit/typography-track";
import { admitRecord, TypeValidatorRegistry } from "@hypit/validation";
-const space = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 4,
+const space = sealProgramSpace({ id: "test-space", durationSec: 4,
frameRate: { numerator: 30, denominator: 1 },
});
const semantic = semanticTrackFixture(space);
@@ -148,7 +147,7 @@ const textInstance = elaborateGraphFragment(linked, typographyTrackFragment, {
id: "title",
fragment: typographyTrackFragment.id,
inputs: {
- semantic: { kind: "record", id: "semantic" },
+ space: { kind: "record", id: "space" },
program: { kind: "record", id: "text-program" },
},
});
@@ -168,7 +167,7 @@ const filmInstance = elaborateGraphFragment(linked, filmFragment, {
inputs: {
program: { kind: "record", id: "film-program" },
canvas: { kind: "record", id: "canvas" },
- semantic: { kind: "record", id: "semantic" },
+ space: { kind: "record", id: "space" },
title: { kind: "logical-output", id: "title.track" },
background: { kind: "record", id: "background" },
audio: { kind: "record", id: "audio" },
@@ -213,7 +212,6 @@ function producerModules(target: string): string[] {
test("Film stops at Composition and Hyperframes remains an ordinary downstream Fragment", () => {
assert.deepEqual(producerNames("title.track"), [
typographyTrackProducers.render.name,
- semanticTrackProducers.projectProgramSpace.name,
]);
assert.equal(producerModules("main.composition").includes(hyperframesProducers.compile.module.name), false);
assert.deepEqual(producerNames("main.composition").filter((name) => name.startsWith("append-")).sort(), [
diff --git a/packages/film/test/surface.test.ts b/packages/film/test/surface.test.ts
index 96969399..75aca533 100644
--- a/packages/film/test/surface.test.ts
+++ b/packages/film/test/surface.test.ts
@@ -57,8 +57,7 @@ const fixtureManifest: ModuleManifest = {
producers: [],
};
-const space = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 2,
+const space = sealProgramSpace({ id: "test-space", durationSec: 2,
frameRate: { numerator: 30, denominator: 1 },
});
const semantic = semanticTrackFixture(space);
diff --git a/packages/hyperframes/test/document.test.ts b/packages/hyperframes/test/document.test.ts
index f3216209..a7745a3d 100644
--- a/packages/hyperframes/test/document.test.ts
+++ b/packages/hyperframes/test/document.test.ts
@@ -22,8 +22,7 @@ const fixtureFont: FontArtifactRef = {
};
function fixture() {
- const programSpace = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 1001 / 1000,
+ const programSpace = sealProgramSpace({ id: "test-space", durationSec: 1001 / 1000,
frameRate: { numerator: 30_000, denominator: 1_001 },
});
const picture = {
@@ -118,8 +117,7 @@ test("HyperFrames flattens generic peer visual Track Presents without absorbing
test("Text shrink preserves authored hug sizing and trims metrics inside the content box", () => {
const programSpace = sealProgramSpace({
id: "text-space",
- narrativeId: "text-narrative",
- durationSec: 1,
+ durationSec: 1,
frameRate: { numerator: 30, denominator: 1 },
});
const track = sealVisualTrack({
@@ -325,8 +323,7 @@ test("HyperFrames clips a long animation by Present visibility instead of reject
});
test("content-bound fonts and typed compositable Surfaces cross the same Artifact boundary", () => {
- const space = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 1,
+ const space = sealProgramSpace({ id: "test-space", durationSec: 1,
frameRate: { numerator: 30, denominator: 1 },
});
const font: FontArtifactRef = {
@@ -400,8 +397,7 @@ test("content-bound fonts and typed compositable Surfaces cross the same Artifac
});
test("exact timed sampling lowers loop boundaries and held frames without zero-rate browser media", () => {
- const programSpace = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 8 / 30,
+ const programSpace = sealProgramSpace({ id: "test-space", durationSec: 8 / 30,
frameRate: { numerator: 30, denominator: 1 },
});
const artifact = {
@@ -465,8 +461,7 @@ test("a Track naming five takes still lowers to DOM identities a Windows path ca
const trackId = "speech-visual:opening-monologue-take-1+opening-monologue-take-2"
+ "+opening-monologue-take-3+opening-monologue-take-4+opening-monologue-take-5";
const presentId = `${trackId}:clip-1`;
- const programSpace = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 8 / 30,
+ const programSpace = sealProgramSpace({ id: "test-space", durationSec: 8 / 30,
frameRate: { numerator: 30, denominator: 1 },
});
const track = sealVisualTrack({ programSpaceId: "test-space",
diff --git a/packages/hyperframes/test/project.test.ts b/packages/hyperframes/test/project.test.ts
index 54cc8e63..03f17d20 100644
--- a/packages/hyperframes/test/project.test.ts
+++ b/packages/hyperframes/test/project.test.ts
@@ -10,7 +10,7 @@ import { sealComposition, sealVisualTrack } from "@hypit/composition";
import { sealProgramSpace } from "@hypit/program-space";
test("staging cancels and joins sibling reads before exposing a failure", async () => {
- const space = sealProgramSpace({ id: "space", narrativeId: "story", durationSec: 1,
+ const space = sealProgramSpace({ id: "space", durationSec: 1,
frameRate: { numerator: 30, denominator: 1 } });
const track = sealVisualTrack({ id: "images", visualIr: "hypit.visual-ir@1", programSpaceId: space.id,
presents: ["a", "b"].map((id, order) => ({ id, span: { startFrame: 0, endFrameExclusive: 30 }, stacking: { order, tieBreak: id },
diff --git a/packages/interview-emoji-reveal/README.md b/packages/interview-emoji-reveal/README.md
index aa0f5955..49eb1660 100644
--- a/packages/interview-emoji-reveal/README.md
+++ b/packages/interview-emoji-reveal/README.md
@@ -1,5 +1,9 @@
# Emoji Reveal
+The Track Surface accepts `semantic={speech.semantic}` or `space={animation}`. Semantic context
+resolves Script Selections and Moments; authored space supports clock-based animation. Shared `at`
+inputs accept a Moment or a time such as `2s`; `at` with `for` produces a Window where required.
+
Reusable Hypit author vocabulary for a top-of-frame icon answer strip.
The strip owns one outer projected `TemporalWindow`. An Item is either preset from the first frame or
@@ -27,7 +31,7 @@ answers remain and later slots remain unanswered.
`Selection`, `Segment`, numeric instants and a `boundary` fallback are deliberately not part of an
Item's vocabulary. The component consumes only `ProgramSpace`, the projected outer Window and the
-projected Moment Instants; it does not interpret Script semantics itself and has no Studio dependency.
+projected Instants; it does not interpret Script semantics itself and has no Studio dependency.
The bundled preview icons come from [Tabler Icons](https://tabler.io/icons), distributed under the
MIT license and retrieved through the Iconify API. Their SVG files are the design sources and are
diff --git a/packages/interview-emoji-reveal/src/fragment.ts b/packages/interview-emoji-reveal/src/fragment.ts
index 696660e4..39650658 100644
--- a/packages/interview-emoji-reveal/src/fragment.ts
+++ b/packages/interview-emoji-reveal/src/fragment.ts
@@ -1,8 +1,8 @@
+import { programSpaceTypes } from "@hypit/program-space";
import { compositionTypes } from "@hypit/composition";
import { artifactTypes } from "@hypit/artifact";
import { sealGraphFragment } from "@hypit/elaborator";
import type { FragmentOperation } from "@hypit/elaborator";
-import { semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";
import { spatialTypes } from "@hypit/spatial";
import { temporalTypes } from "@hypit/temporal";
@@ -19,7 +19,6 @@ const operation = (id: string) => ({ kind: "fragment-operation" as const, operat
export function createEmojiRevealFragment(items: readonly EmojiRevealFragmentItem[]) {
if (items.length === 0) throw new Error("Emoji Reveal Fragment requires at least one Item.");
const operations: FragmentOperation[] = [
- { id: "emoji:space", producer: semanticTrackProducers.projectProgramSpace, inputs: { track: input("semantic") }, result: { kind: "output", name: "space" } },
{ id: "emoji:set:empty", producer: emojiRevealProducers.createSet, inputs: {}, result: { kind: "output", name: "set" } },
];
let current = "emoji:set:empty";
@@ -30,23 +29,23 @@ export function createEmojiRevealFragment(items: readonly EmojiRevealFragmentIte
: emojiRevealProducers.appendItem, inputs: {
set: operation(current), spec: input(item.specName), icon: input(item.iconName),
...(item.activationName === undefined ? {} : {
- space: operation("emoji:space"), activation: input(item.activationName),
+ space: input("space"), activation: input(item.activationName),
}),
}, result: { kind: "output", name: "set" } });
current = id;
});
operations.push(
{ id: "emoji:program", producer: emojiRevealProducers.finalize, inputs: {
- header: input("header"), space: operation("emoji:space"), outer: input("outer"), style: input("style"),
+ header: input("header"), space: input("space"), outer: input("outer"), style: input("style"),
placeholder: input("placeholder"), set: operation(current),
}, result: { kind: "output", name: "program" } },
{ id: "emoji:track", producer: emojiRevealProducers.render, inputs: {
- canvas: input("canvas"), space: operation("emoji:space"), program: operation("emoji:program"),
+ canvas: input("canvas"), space: input("space"), program: operation("emoji:program"),
}, result: { kind: "output", name: "track" } },
);
return sealGraphFragment({
inputs: [
- { name: "header", type: emojiRevealTypes.header }, { name: "semantic", type: semanticTrackTypes.track },
+ { name: "header", type: emojiRevealTypes.header }, { name: "space", type: programSpaceTypes.programSpace },
{ name: "canvas", type: spatialTypes.canvas }, { name: "outer", type: temporalTypes.window }, { name: "style", type: emojiRevealTypes.style },
{ name: "placeholder", type: artifactTypes.blob },
...items.flatMap((item) => [
diff --git a/packages/interview-emoji-reveal/src/manifest.ts b/packages/interview-emoji-reveal/src/manifest.ts
index 07bfe3ea..6902de71 100644
--- a/packages/interview-emoji-reveal/src/manifest.ts
+++ b/packages/interview-emoji-reveal/src/manifest.ts
@@ -1,3 +1,4 @@
+import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";
import { readFile } from "node:fs/promises";
import { artifactDependency, artifactTypes } from "@hypit/artifact";
@@ -5,7 +6,7 @@ import { compositionDependency, compositionTypes } from "@hypit/composition";
import { narrativeDependency, narrativeTypes } from "@hypit/narrative";
import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";
import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";
-import { semanticTrackDependency, semanticTrackTypes } from "@hypit/semantic-track";
+import { semanticTrackDependency } from "@hypit/semantic-track";
import { spatialDependency, spatialTypes } from "@hypit/spatial";
import { svsRecipeType } from "@hypit/svs";
import { temporalDependency, temporalInstantSchema, temporalTypes, temporalWindowSchema } from "@hypit/temporal";
@@ -116,17 +117,17 @@ export const emojiRevealMarkupSurfaces = [
preview: previewImage("Track.png"),
attributes: [
{ name: "id", kind: "identifier", required: true, summary: "Names the reveal Program and Track." },
- { name: "semantic", kind: "reference", required: true, accepts: [semanticTrackTypes.track], summary: "Chooses the SemanticTrack whose frame domain projects the outer Window and every Moment." },
+ ...temporalContextAttributeVocabulary,
{ name: "canvas", kind: "reference", required: true, accepts: [spatialTypes.canvas], summary: "Chooses the Canvas used for normalized top placement." },
{ name: "style", kind: "reference", required: true, accepts: [emojiRevealTypes.style], summary: "Chooses the strip Style." },
{ name: "placeholder", kind: "reference", required: true, accepts: [artifactTypes.blob], summary: "Supplies the one image drawn in every unrevealed slot." },
...temporalWindowAttributeVocabulary,
],
- children: [{ tag: "Item", cardinality: "many", summary: "One answer slot, either settled from the start or revealed at exactly one semantic Moment.", attributes: [
+ children: [{ tag: "Item", cardinality: "many", summary: "One answer slot, either settled from the start or revealed at one event.", attributes: [
{ name: "id", kind: "identifier", required: true, summary: "Names this answer slot and the timing subject it owns." },
{ name: "icon", kind: "reference", required: true, accepts: [artifactTypes.blob], summary: "Supplies this answer as an image Artifact from the same visual icon family as the placeholder." },
{ name: "preset", kind: "literal", required: false, values: ["true", "false"], summary: "Settles this Item from the start of the Track Window; preset Items have no at." },
- { name: "at", kind: "reference", required: false, accepts: [narrativeTypes.moment], summary: "Chooses this non-preset Item's reveal Moment; required unless preset is true." },
+ { name: "at", kind: "expression", required: false, accepts: [narrativeTypes.moment], summary: "Chooses this non-preset Item's reveal Moment or authored time, such as 2s or 12f; required unless preset is true." },
] }],
ports: [
{ name: "program", type: emojiRevealTypes.program, summary: "The adaptive strip with its projected outer Window and fully traced Moment activations." },
@@ -139,7 +140,7 @@ export const emojiRevealMarkupSurfaces = [
</emoji:Track>`,
notes: [
"Item order is display order: preset Items come first, then non-preset Items in strict chronological reveal order.",
- "A non-preset Item.at accepts only a Moment: Selection boundaries, numeric instants and boundary fallbacks are intentionally absent.",
+ "A non-preset Item.at chooses one reveal trigger: a Script Moment or an authored time. The answer then persists to the outer Window's end.",
"The outer Track Window is separate from the child reveal Moments and uses the shared temporal Window protocol.",
],
} },
diff --git a/packages/interview-emoji-reveal/src/surface.ts b/packages/interview-emoji-reveal/src/surface.ts
index f62b6fc5..756ca2fe 100644
--- a/packages/interview-emoji-reveal/src/surface.ts
+++ b/packages/interview-emoji-reveal/src/surface.ts
@@ -1,3 +1,4 @@
+import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";
import {
assertAttributes, assertEmptyElement, optionalTextAttribute, textAttribute, type MarkupAttributeValue, type StructuredSurfaceHandler,
type SurfaceComponentDraft, type SurfaceRecordDraft, type SurfaceResolvedReference,
@@ -5,7 +6,6 @@ import {
import { artifactTypes } from "@hypit/artifact";
import { narrativeTypes } from "@hypit/narrative";
import { sameType, type CanonicalValue, type TypeRef } from "@hypit/protocol";
-import { semanticTrackTypes } from "@hypit/semantic-track";
import { spatialTypes } from "@hypit/spatial";
import { svsRecipeType } from "@hypit/svs";
import type { SvsRecipe } from "@hypit/svs";
@@ -48,23 +48,24 @@ export const decodeEmojiRevealStyleSurface: StructuredSurfaceHandler = ({ elemen
};
export const decodeEmojiRevealTrackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
- assertAttributes(element, ["id", "semantic", "canvas", "style", "placeholder", ...temporalWindowAttributeNames]);
+ assertAttributes(element, ["id", "semantic", "space", "canvas", "style", "placeholder", ...temporalWindowAttributeNames]);
const id = textAttribute(element, "id");
- const semantic = reference(element.attributes.semantic, `${element.name}.semantic`, semanticTrackTypes.track, resolveReference);
+ const context = resolveTemporalContext({ element, resolveReference });
+ const time = createTemporalSpace({ id: id, element, ...context });
const canvas = reference(element.attributes.canvas, `${element.name}.canvas`, spatialTypes.canvas, resolveReference);
const style = reference(element.attributes.style, `${element.name}.style`, emojiRevealTypes.style, resolveReference);
const placeholder = reference(element.attributes.placeholder, `${element.name}.placeholder`, artifactTypes.blob, resolveReference);
- const outer = createTemporalWindowProjection({ id, subjectId: id, element, semantic, resolveReference });
+ const outer = createTemporalWindowProjection({ id, subjectId: id, element, ...context, space: time.space, resolveReference });
const headerId = `${id}.header`;
const records: SurfaceRecordDraft[] = [...outer.records, {
id: headerId, type: emojiRevealTypes.header,
value: { kind: "inline", value: sealEmojiRevealHeader({ id }) as unknown as CanonicalValue }, range: element.range,
}];
- const temporalComponents: SurfaceComponentDraft[] = [...outer.components];
- const temporalFragments = [...outer.fragments];
+ const temporalComponents: SurfaceComponentDraft[] = [...time.components, ...outer.components];
+ const temporalFragments = [...time.fragments, ...outer.fragments];
const items: { specName: string; iconName: string; activationName?: string }[] = [];
- const inputs: Record<string, typeof semantic.ref> = {
- header: { kind: "record", id: headerId }, semantic: semantic.ref, canvas: canvas.ref, style: style.ref,
+ const inputs: Record<string, typeof time.space.ref> = {
+ header: { kind: "record", id: headerId }, space: time.space.ref, canvas: canvas.ref, style: style.ref,
placeholder: placeholder.ref, outer: outer.ref,
};
const ids = new Set<string>();
@@ -89,10 +90,9 @@ export const decodeEmojiRevealTrackSurface: StructuredSurfaceHandler = ({ elemen
inputs[specName] = { kind: "record", id: specId }; inputs[iconName] = icon.ref;
if (preset) items.push({ specName, iconName });
else {
- reference(child.attributes.at, `${child.name}.at`, narrativeTypes.moment, resolveReference);
const activation = createTemporalInstantProjection({
id: `${id}.item.${suffix}.activation`, subjectId: itemId,
- element: child, semantic, resolveReference, semanticAttribute: "at", projectedAttribute: false,
+ element: child, ...context, space: time.space, resolveReference, semanticAttribute: "at", projectedAttribute: false,
});
records.push(...activation.records); temporalComponents.push(...activation.components); temporalFragments.push(...activation.fragments);
const activationName = `item-${suffix}-activation`;
diff --git a/packages/media-pipeline/src/audio-plan.ts b/packages/media-pipeline/src/audio-plan.ts
index a7bbaa52..e07cb5e8 100644
--- a/packages/media-pipeline/src/audio-plan.ts
+++ b/packages/media-pipeline/src/audio-plan.ts
@@ -24,7 +24,6 @@ export function verifyAudioProgramPlan(value: unknown): asserts value is AudioPr
assert(item.sampleRate === 48_000, "AudioProgramPlan sample rate must be 48000");
const planSpace = {
id: "audio-program-plan",
- narrativeId: "audio-program-plan",
durationSec: item.frameCount * item.frameRate.denominator / item.frameRate.numerator,
frameRate: item.frameRate,
};
diff --git a/packages/media-track/README.md b/packages/media-track/README.md
index 0eeb92e6..99da1fdc 100644
--- a/packages/media-track/README.md
+++ b/packages/media-track/README.md
@@ -1,13 +1,18 @@
# `@hypit/media-track`
+The Track Surface accepts `semantic={speech.semantic}` or `space={animation}`. Semantic context
+resolves Script Selections and Moments; authored space supports clock-based animation. Shared `at`
+inputs accept a Moment or a time such as `2s`; `at` with `for` produces a Window where required.
+`Performance` still needs `semantic`, which supplies its ordered prepared footage.
+
Place images, prepared video or compositable surfaces over the semantic timeline. B-roll is a common
use: the picture can illustrate a phrase while the A-roll's speech continues. Performance presents the semantic material as a full picture, inset or cutout.
Media Items own independently timed pictures;
a Sequence owns one visual slot whose Members replace each other through Handoffs.
-The Markup Track takes `semantic` and `canvas`. Its Surface projects authored Selections, Moments,
-Segments or clock expressions into Windows and Instants. The component consumes those projected
-times with Frames, media and appearance; the Fragment obtains ProgramSpace from the SemanticTrack.
+The Markup Track takes `canvas` and a time source (`semantic` or `space`). Its Surface projects
+authored Selections, Moments, Segments or clock expressions into Windows and Instants. The component
+consumes those projected times with Frames, media and appearance; its Fragment receives ProgramSpace.
Moving media enters after [normalization](../media-pipeline/README.md), with the intended streams and
frame clock already selected. Still images use their actual intrinsic Extent. Placement, fitting,
diff --git a/packages/media-track/package.json b/packages/media-track/package.json
index 89a87631..3a144622 100644
--- a/packages/media-track/package.json
+++ b/packages/media-track/package.json
@@ -23,7 +23,7 @@
"@hypit/spatial": "workspace:*",
"@hypit/svs": "workspace:*",
"@hypit/markup": "workspace:*",
- "@hypit/temporal": "workspace:*"
- ,"@hypit/temporal-markup": "workspace:*"
+ "@hypit/temporal": "workspace:*",
+ "@hypit/temporal-markup": "workspace:*"
}
}
diff --git a/packages/media-track/src/fragment.ts b/packages/media-track/src/fragment.ts
index a935bb1a..196ba6f4 100644
--- a/packages/media-track/src/fragment.ts
+++ b/packages/media-track/src/fragment.ts
@@ -1,7 +1,7 @@
+import { programSpaceTypes } from "@hypit/program-space";
import { artifactTypes } from "@hypit/artifact";
import { compositionTypes } from "@hypit/composition";
import { sealGraphFragment } from "@hypit/elaborator";
-import { semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";
import { spatialTypes } from "@hypit/spatial";
import { temporalTypes } from "@hypit/temporal";
@@ -14,7 +14,7 @@ const operation = (id: string) => ({ kind: "fragment-operation" as const, operat
export const stillMediaTrackFragment = sealGraphFragment({
inputs: [
{ name: "header", type: mediaTrackTypes.header },
- { name: "semantic", type: semanticTrackTypes.track },
+ { name: "space", type: programSpaceTypes.programSpace },
{ name: "canvas", type: spatialTypes.canvas },
{ name: "source", type: artifactTypes.blob },
{ name: "extent", type: spatialTypes.extent },
@@ -31,18 +31,15 @@ export const stillMediaTrackFragment = sealGraphFragment({
}, result: { kind: "output", name: "layers" } },
{ id: "set", producer: mediaTrackProducers.createSet, inputs: {}, result: { kind: "output", name: "set" } },
{ id: "sounds", producer: mediaTrackProducers.createSounds, inputs: {}, result: { kind: "output", name: "sounds" } },
- { id: "space", producer: semanticTrackProducers.projectProgramSpace, inputs: {
- track: input("semantic"),
- }, result: { kind: "output", name: "space" } },
{ id: "append", producer: mediaTrackProducers.appendItem, inputs: {
- set: operation("set"), header: input("header"), space: operation("space"), canvas: input("canvas"), layers: operation("sample"),
+ set: operation("set"), header: input("header"), space: input("space"), canvas: input("canvas"), layers: operation("sample"),
frame: input("frame"), spec: input("item-spec"), sounds: operation("sounds"), window: input("window"),
}, result: { kind: "output", name: "set" } },
{ id: "finalize", producer: mediaTrackProducers.finalize, inputs: {
- set: operation("append"), header: input("header"), space: operation("space"),
+ set: operation("append"), header: input("header"), space: input("space"),
}, result: { kind: "output", name: "program" } },
{ id: "visual", producer: mediaTrackProducers.projectVisual, inputs: {
- space: operation("space"), program: operation("finalize"),
+ space: input("space"), program: operation("finalize"),
}, result: { kind: "output", name: "track" } },
],
exports: [
@@ -52,10 +49,9 @@ export const stillMediaTrackFragment = sealGraphFragment({
});
export const renderMediaTrackFragment = sealGraphFragment({
- inputs: [{ name: "semantic", type: semanticTrackTypes.track }, { name: "program", type: mediaTrackTypes.program }],
+ inputs: [{ name: "space", type: programSpaceTypes.programSpace }, { name: "program", type: mediaTrackTypes.program }],
operations: [
- { id: "space", producer: semanticTrackProducers.projectProgramSpace, inputs: { track: input("semantic") }, result: { kind: "output", name: "space" } },
- { id: "visual", producer: mediaTrackProducers.projectVisual, inputs: { space: operation("space"), program: input("program") }, result: { kind: "output", name: "track" } },
+ { id: "visual", producer: mediaTrackProducers.projectVisual, inputs: { space: input("space"), program: input("program") }, result: { kind: "output", name: "track" } },
],
exports: [{ name: "track", type: compositionTypes.visualTrack, root: operation("visual") }],
});
diff --git a/packages/media-track/src/manifest.ts b/packages/media-track/src/manifest.ts
index b7a597e9..e3bb781e 100644
--- a/packages/media-track/src/manifest.ts
+++ b/packages/media-track/src/manifest.ts
@@ -1,3 +1,4 @@
+import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";
import { visualTimedSamplingSchema } from "@hypit/composition";
import { readFile } from "node:fs/promises";
@@ -367,14 +368,13 @@ export const mediaTrackMarkupSurfaces = [{
temporalTypes.instantSpec, temporalTypes.windowSpec, temporalTypes.instant, temporalTypes.window, mediaTrackTypes.program,
compositionTypes.visualTrack, compositionTypes.audioTrack],
vocabulary: {
- summary: "Show the semantic performance, place independent media Items, and compose replacement Sequences on a shared SemanticTrack and Canvas. Publishes a VisualTrack and, when audio is authored, an AudioTrack.",
+ summary: "Show the semantic performance, place independent media Items, and compose replacement Sequences on a shared film time axis and Canvas. Publishes a VisualTrack and, when audio is authored, an AudioTrack.",
appearance: "Pictures and layered compositions occupy authored Frames on the Canvas. Each source is scaled and aligned inside its Frame's border and padding; a contain fit can leave space, while cover can crop. Paint and sampled Layers draw in Source order under a shared rectangular, rounded or Path clip, with optional border and shadows. Lifecycle motion moves the framed unit; Sampling pans, zooms or rotates the picture inside it. Each Item has an independent window. A Sequence shares one outer Frame while Members replace its contents through cut, crossfade, push, wipe, cover or page-turn Handoffs. Source playback determines whether moving content covers the whole assigned span. Absolute stack order decides which overlapping unit draws in front.",
preview: previewImage("Track.png"),
attributes: [
{ name: "id", kind: "identifier", required: true,
summary: "Names this Track and prefixes the identity of every Item, Sequence, layer and sound that does not name itself." },
- { name: "semantic", kind: "reference", required: true, accepts: [semanticTrackTypes.track],
- summary: "Supplies semantic time for every child and the prepared material shown by Performance." },
+ ...temporalContextAttributeVocabulary,
{ name: "canvas", kind: "reference", required: true, accepts: [spatialTypes.canvas],
summary: "Chooses the Canvas every Frame on this Track is measured inside." },
],
@@ -440,9 +440,9 @@ export const mediaTrackMarkupSurfaces = [{
recipe: motionRecipeProperties },
{ name: "clip", kind: "reference", required: false, accepts: [spatialTypes.path],
summary: "Clips the Sequence to an authored Path." },
- { name: "until", kind: "reference", required: true,
+ { name: "until", kind: "expression", required: true,
accepts: [narrativeTypes.moment, narrativeTypes.selection, narrativeTypes.excerpt],
- summary: "Ends the Sequence at a Moment or a chosen Selection/Segment boundary." },
+ summary: "Ends the Sequence at a Moment, a Selection/Segment boundary, or an authored time such as 8s." },
{ name: "until-boundary", kind: "literal", required: false, values: ["start", "end"],
summary: "Chooses which edge of the ending Selection or Segment ends the Sequence; defaults to `end` and is refused for a Moment." },
] },
@@ -476,13 +476,13 @@ export const mediaTrackMarkupSurfaces = [{
"An `enter` or an `exit` operator requires its own `enter-frames` or `exit-frames`.",
"An Item written with a direct source accepts `<Sampling>` and `<Sound>` children; an Item written without one accepts `<Paint>`, `<Layer>` and `<Sound>` children, and requires at least one Paint or Layer.",
"A Sequence requires at least two `<Member>` children, exactly one `<Handoff>` for every adjacent Member pair, and accepts `<Sound>` children.",
- "Sequence `until` is a semantic reference, not a literal `program.end`. `until-boundary` defaults to end for a Selection or Segment and is refused for a Moment.",
+ "Sequence `until` selects a Moment, a Selection or Segment boundary, or an authored time such as 8s. `until-boundary` defaults to end for a Selection or Segment.",
"A Member is one picture in the replacement order:",
[
"| Attribute | Kind | Required | Meaning |",
"|---|---|---|---|",
"| `id` | identifier | no | Names this Member; the Sequence derives `<sequence>.member.<index>` otherwise |",
- "| `at` | semantic reference | one timing form | A Moment, Selection or Segment that activates this Member |",
+ "| `at` | reference or time | one timing form | A Moment, Selection or Segment boundary, or an authored time that activates this Member |",
"| `boundary` | literal (start, end) | with a Selection or Segment | Which edge activates this Member; refused for a Moment |",
"| `instant` | expression | one timing form | Explicit point expression instead of `at`; bind selection, segment or moment if the expression uses it |",
"| `appearance` | reference (@hypit/svs@1#Recipe) | no | This Member's own Recipe in place of the Sequence's |",
diff --git a/packages/media-track/src/surface.ts b/packages/media-track/src/surface.ts
index 8db36ba8..bfbbbda2 100644
--- a/packages/media-track/src/surface.ts
+++ b/packages/media-track/src/surface.ts
@@ -1,3 +1,4 @@
+import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";
import {
assertEmptyElement as empty,
assertAttributes as allowed,
@@ -16,7 +17,6 @@ import { compositionTypes } from "@hypit/composition";
import { sealGraphFragment } from "@hypit/elaborator";
import type { FragmentOperation } from "@hypit/elaborator";
import { mediaTypes } from "@hypit/media";
-import { semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";
import { spatialTypes } from "@hypit/spatial";
import { svsRecipeType } from "@hypit/svs";
import type { SvsRecipe } from "@hypit/svs";
@@ -150,7 +150,6 @@ function appendSounds(operations: FragmentOperation[], prefix: string, sounds: r
function createMediaTrackSurfaceFragment(inputTypes: readonly { readonly name: string; readonly type: TypeRef }[], items: readonly FragmentItem[], sequences: readonly FragmentSequence[], audio: boolean) {
const operations: FragmentOperation[] = [
- { id: "track:space", producer: semanticTrackProducers.projectProgramSpace, inputs: { track: input("semantic") }, result: { kind: "output", name: "space" } },
{ id: "track:set:empty", producer: mediaTrackProducers.createSet, inputs: {}, result: { kind: "output", name: "set" } },
];
let set = "track:set:empty";
@@ -168,7 +167,7 @@ function createMediaTrackSurfaceFragment(inputTypes: readonly { readonly name: s
spec = operation(bindId);
}
const common = {
- set: operation(set), header: input("header"), space: operation("track:space"), canvas: input("canvas"),
+ set: operation(set), header: input("header"), space: input("space"), canvas: input("canvas"),
layers: operation(layers), frame: input(item.frameName), spec, sounds: operation(sounds), window: input(item.windowName),
};
operations.push({ id: appendId, producer: item.performance === undefined ? mediaTrackProducers.appendItem : mediaTrackProducers.appendPerformance,
@@ -185,7 +184,7 @@ function createMediaTrackSurfaceFragment(inputTypes: readonly { readonly name: s
const layers = appendLayers(operations, memberPrefix, member.layers);
const appendId = `${memberPrefix}:append`;
operations.push({ id: appendId, producer: mediaTrackProducers.appendMember, inputs: {
- members: operation(members), space: operation("track:space"), layers: operation(layers), spec: input(member.specName), activation: input(member.instantName),
+ members: operation(members), space: input("space"), layers: operation(layers), spec: input(member.specName), activation: input(member.instantName),
}, result: { kind: "output", name: "members" } });
members = appendId;
}
@@ -200,7 +199,7 @@ function createMediaTrackSurfaceFragment(inputTypes: readonly { readonly name: s
spec = operation(bindId);
}
const common = {
- set: operation(set), header: input("header"), space: operation("track:space"), canvas: input("canvas"),
+ set: operation(set), header: input("header"), space: input("space"), canvas: input("canvas"),
members: operation(members), frame: input(sequence.frameName), spec, sounds: operation(sounds),
};
operations.push({ id: appendId, producer: mediaTrackProducers.appendSequence,
@@ -208,10 +207,10 @@ function createMediaTrackSurfaceFragment(inputTypes: readonly { readonly name: s
set = appendId;
}
operations.push(
- { id: "track:finalize", producer: mediaTrackProducers.finalize, inputs: { set: operation(set), header: input("header"), space: operation("track:space") }, result: { kind: "output", name: "program" } },
- { id: "track:visual", producer: mediaTrackProducers.projectVisual, inputs: { space: operation("track:space"), program: operation("track:finalize") }, result: { kind: "output", name: "track" } },
+ { id: "track:finalize", producer: mediaTrackProducers.finalize, inputs: { set: operation(set), header: input("header"), space: input("space") }, result: { kind: "output", name: "program" } },
+ { id: "track:visual", producer: mediaTrackProducers.projectVisual, inputs: { space: input("space"), program: operation("track:finalize") }, result: { kind: "output", name: "track" } },
);
- if (audio) operations.push({ id: "track:audio", producer: mediaTrackProducers.projectAudio, inputs: { space: operation("track:space"), program: operation("track:finalize") }, result: { kind: "output", name: "track" } });
+ if (audio) operations.push({ id: "track:audio", producer: mediaTrackProducers.projectAudio, inputs: { space: input("space"), program: operation("track:finalize") }, result: { kind: "output", name: "track" } });
return sealGraphFragment({
inputs: inputTypes,
operations,
@@ -496,19 +495,20 @@ function sound(
}
export const decodeMediaTrackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
- allowed(element, ["id", "semantic", "canvas"]);
+ allowed(element, ["id", "semantic", "space", "canvas"]);
const trackId = text(element, "id");
const state = builder();
- const semantic = reference(element.attributes.semantic, `${element.name}.semantic`, semanticTrackTypes.track, resolveReference);
- state.addReference("semantic", semantic);
+ const context = resolveTemporalContext({ element, resolveReference });
+ const time = createTemporalSpace({ id: trackId, element, ...context });
+ state.addReference("space", time.space);
state.addReference("canvas", reference(element.attributes.canvas, `${element.name}.canvas`, spatialTypes.canvas, resolveReference));
const headerId = `${trackId}.header`;
state.addRecord("header", headerId, mediaTrackTypes.header,
sealMediaTrackHeader({ id: trackId }), element.range);
const items: FragmentItem[] = [];
const sequences: FragmentSequence[] = [];
- const temporalComponents: SurfaceComponentDraft[] = [];
- const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [];
+ const temporalComponents: SurfaceComponentDraft[] = [...time.components];
+ const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [...time.fragments];
let itemIndex = 0;
let sequenceIndex = 0;
let hasAudio = false;
@@ -518,6 +518,10 @@ export const decodeMediaTrackSurface: StructuredSurfaceHandler = ({ element, res
continue;
}
const isPerformance = child.name.endsWith(":Performance") || child.name === "Performance";
+ if (isPerformance) {
+ if (context.semantic === undefined) throw new Error(`${child.name} requires semantic performance input.`);
+ if (state.inputs.semantic === undefined) state.addReference("semantic", context.semantic);
+ }
if (isPerformance || child.name.endsWith(":Item") || child.name === "Item") {
itemIndex += 1;
const itemSuffix = suffix(itemIndex);
@@ -535,7 +539,7 @@ export const decodeMediaTrackSurface: StructuredSurfaceHandler = ({ element, res
}
const motionRecipe = child.attributes.motion === undefined ? undefined
: recipe(reference(child.attributes.motion, `${child.name}.motion`, svsRecipeType, resolveReference), `${child.name}.motion`);
- const temporal = createTemporalWindowProjection({ id, element: child, semantic, resolveReference });
+ const temporal = createTemporalWindowProjection({ id, element: child, ...context, space: time.space, resolveReference });
state.records.push(...temporal.records); temporalComponents.push(...temporal.components); temporalFragments.push(...temporal.fragments);
const performance = isPerformance ? {
fitName: `item-${itemSuffix}-fit`, sampleName: `item-${itemSuffix}-sample`,
@@ -607,7 +611,7 @@ export const decodeMediaTrackSurface: StructuredSurfaceHandler = ({ element, res
for (const [index, member] of memberElements.entries()) {
const memberSuffix = suffix(index + 1);
allowed(member, ["id", "image", "media", "surface", "extent", "appearance", ...temporalInstantAttributeNames, "source-audio", "audio-gain"]);
- const temporal = createTemporalInstantProjection({ id: `${id}.${memberIds[index]}`, subjectId: memberIds[index]!, element: member, semantic, resolveReference });
+ const temporal = createTemporalInstantProjection({ id: `${id}.${memberIds[index]}`, subjectId: memberIds[index]!, element: member, ...context, space: time.space, resolveReference });
state.records.push(...temporal.records); temporalComponents.push(...temporal.components); temporalFragments.push(...temporal.fragments);
const memberAppearance = member.attributes.appearance === undefined ? appearance
: recipe(reference(member.attributes.appearance, `${member.name}.appearance`, svsRecipeType, resolveReference), `${member.name}.appearance`);
@@ -637,7 +641,7 @@ export const decodeMediaTrackSurface: StructuredSurfaceHandler = ({ element, res
}
if (sequenceSounds.length > 0) hasAudio = true;
const terminalTemporal = createTemporalInstantProjection({
- id: `${id}.terminal`, subjectId: id, element: child, semantic, resolveReference,
+ id: `${id}.terminal`, subjectId: id, element: child, ...context, space: time.space, resolveReference,
semanticAttribute: "until", boundaryAttribute: "until-boundary", projectedAttribute: false,
boundaryFallback: "end",
});
diff --git a/packages/media-track/test/media-track.test.ts b/packages/media-track/test/media-track.test.ts
index 6be77299..8e6d5f32 100644
--- a/packages/media-track/test/media-track.test.ts
+++ b/packages/media-track/test/media-track.test.ts
@@ -64,8 +64,7 @@ import type { SvsRecipe } from "@hypit/svs";
import type { StructuredElement, StructuredNode, SurfaceResolvedReference, MarkupAttributeValue } from "@hypit/markup";
import type { TemporalWindow } from "@hypit/temporal";
-const space = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 4,
+const space = sealProgramSpace({ id: "test-space", durationSec: 4,
frameRate: { numerator: 30, denominator: 1 },
});
const canvas = {
@@ -233,7 +232,7 @@ function sequenceMembers(
members = appendProjectedMediaSequenceMember(members, space, layerFactory(index), spec, {
id: `${spec.id}::test`,
subjectId: spec.id,
- source: { spaceId: space.id, narrativeId: space.narrativeId, kind: "program", id: "program" },
+ source: { spaceId: space.id, narrativeId: "script", kind: "program", id: "program" },
projection: { ref: "program.start" },
frame: activationFrame,
authority: { kind: "fixed" },
@@ -246,8 +245,8 @@ function testProgramWindow(endFrameExclusive: number): TemporalWindow {
return {
id: "program::test",
subjectId: "program",
- start: { id: "program::test.start", subjectId: "program", source: { spaceId: space.id, narrativeId: space.narrativeId, kind: "program", id: "program" }, projection: { ref: "program.start" }, frame: 0, authority: { kind: "fixed" } },
- end: { id: "program::test.end", subjectId: "program", source: { spaceId: space.id, narrativeId: space.narrativeId, kind: "program", id: "program" }, projection: { ref: "program.end" }, frame: endFrameExclusive, authority: { kind: "fixed" } },
+ start: { id: "program::test.start", subjectId: "program", source: { spaceId: space.id, narrativeId: "script", kind: "program", id: "program" }, projection: { ref: "program.start" }, frame: 0, authority: { kind: "fixed" } },
+ end: { id: "program::test.end", subjectId: "program", source: { spaceId: space.id, narrativeId: "script", kind: "program", id: "program" }, projection: { ref: "program.end" }, frame: endFrameExclusive, authority: { kind: "fixed" } },
span: { startFrame: 0, endFrameExclusive },
};
}
@@ -258,7 +257,7 @@ function appendMediaSequence(
sounds: ReturnType<typeof createMediaSoundSet>, terminalFrame: number,
) {
return appendProjectedMediaSequence(set, trackHeader, spaceValue, canvasValue, members, frameValue, spec, sounds, {
- id: `${spec.id}::terminal`, subjectId: spec.id, source: { spaceId: space.id, narrativeId: space.narrativeId, kind: "program", id: "program" },
+ id: `${spec.id}::terminal`, subjectId: spec.id, source: { spaceId: space.id, narrativeId: "script", kind: "program", id: "program" },
projection: { ref: "program.end" }, frame: terminalFrame, authority: { kind: "fixed" },
});
}
@@ -269,7 +268,7 @@ function appendMediaSequenceMember(
return appendProjectedMediaSequenceMember(set, space, layers, spec, {
id: `${spec.id}::test`,
subjectId: spec.id,
- source: { spaceId: space.id, narrativeId: space.narrativeId, kind: "program", id: "program" },
+ source: { spaceId: space.id, narrativeId: "script", kind: "program", id: "program" },
projection: { ref: "program.start" },
frame: activationFrame,
authority: { kind: "fixed" },
@@ -540,7 +539,7 @@ test("still and animated typed Surfaces use the same layer law without browser f
test("Media Items consume singular Selection and Moment sources without becoming an exclusive lane", () => {
const selection: NarrativeSelectionRef = {
- narrativeId: space.narrativeId,
+ narrativeId: "script",
id: "mentions",
startAnchorId: "a",
endAnchorId: "b",
@@ -552,7 +551,7 @@ test("Media Items consume singular Selection and Moment sources without becoming
}), createMediaSoundSet());
assert.deepEqual(selected.items.map((item) => item.span), [{ startFrame: 15, endFrameExclusive: 45 }]);
const moment: NarrativeMomentRef = {
- narrativeId: space.narrativeId,
+ narrativeId: "script",
id: "cue", anchorId: "b",
};
const overlapping = appendMomentMediaItem(selected, header, semantic, canvas, stillLayers(), frame,
@@ -581,7 +580,7 @@ test("Media Items consume singular Selection and Moment sources without becoming
test("a Media Item can consume one whole Narrative Segment without a synthetic Selection", () => {
const result = appendSegmentMediaItem(
createMediaTrackSet(), header, semantic, canvas, stillLayers(), frame,
- { narrativeId: space.narrativeId, kind: "segment", id: "answer", tokenStart: 0, tokenEndExclusive: 1 },
+ { narrativeId: "script", kind: "segment", id: "answer", tokenStart: 0, tokenEndExclusive: 1 },
itemSpec({
id: "whole-answer",
projection: { start: { ref: "segment.start" }, end: { ref: "segment.end" } },
@@ -706,7 +705,7 @@ test("timed visual occupancy resolves every alignment into exact source-frame se
test("the graph keeps every source, extent, fit, frame, time and appearance input explicit", () => {
assert.deepEqual(stillMediaTrackFragment.inputs.map((input) => input.name), [
- "canvas", "extent", "fit", "frame", "header", "item-spec", "sample-spec", "semantic", "source", "window",
+ "canvas", "extent", "fit", "frame", "header", "item-spec", "sample-spec", "source", "space", "window",
]);
});
@@ -765,6 +764,7 @@ test("the Media author Surface emits explicit graph edges for layers, semantic t
node("media:Sampling", { at: "end", zoom: "1.08", x: "6" }),
node("media:Sound", { id: "presenter-enter", source: ref("sfx"), at: "enter" }),
]),
+ node("media:Performance", { id: "inset", frame: ref("frame"), appearance: ref("still-style"), at: "12f", for: "2s" }),
node("media:Sequence", { id: "steps", frame: ref("frame"), appearance: ref("sequence-style"), until: ref("terminal"), "until-boundary": "end" }, [
node("media:Member", { id: "one", image: ref("still"), extent: ref("extent"), at: ref("cue1") }),
node("media:Member", { id: "two", surface: ref("surface"), appearance: ref("surface-style"), at: ref("cue2") }),
@@ -788,6 +788,7 @@ test("the Media author Surface emits explicit graph edges for layers, semantic t
const performanceOperation = fragment.operations.find(entry => entry.producer.name === "append-media-performance");
assert.deepEqual(performanceOperation?.inputs.semantic, { kind: "fragment-input", name: "semantic" });
assert.equal(fragment.inputs.filter(entry => entry.type.name === semanticTrackTypes.track.name).length, 1);
+ assert.equal(fragment.operations.filter(entry => entry.producer.name === "append-media-performance").length, 2);
assert.ok(producers.includes("append-timed-media-layer"));
assert.ok(producers.includes("append-surface-media-layer"));
assert.ok(producers.includes("append-media-paint-layer"));
@@ -803,13 +804,13 @@ test("the Media author Surface emits explicit graph edges for layers, semantic t
&& "subjectId" in record.value.value
? [String((record.value.value as { readonly subjectId: unknown }).subjectId)]
: []));
- assert.deepEqual([...temporalSubjects].sort(), ["one", "presenter", "proof", "segment-card", "steps", "still-card", "two"]);
+ assert.deepEqual([...temporalSubjects].sort(), ["inset", "one", "presenter", "proof", "segment-card", "steps", "still-card", "two"]);
assert.ok(producers.includes("append-media-sound"));
assert.ok(fragment.inputs.some((entry) => entry.type.name === artifactTypes.blob.name));
assert.ok(fragment.inputs.some((entry) => entry.type.name === mediaTypes.synchronized.name));
assert.ok(fragment.inputs.some((entry) => entry.type.name === spatialTypes.path.name));
const itemSpecs = result.records.filter((entry) => entry.type.name === "MediaItemSpec");
- assert.equal(itemSpecs.length, 4);
+ assert.equal(itemSpecs.length, 5);
const performanceSample = result.records.find(entry => entry.id === "presenter.sample");
assert.ok(performanceSample?.value.kind === "inline");
assert.deepEqual((performanceSample.value.value as unknown as { samplingMotion: unknown }).samplingMotion, {
diff --git a/packages/program-space/README.md b/packages/program-space/README.md
index 4fd6d43a..df386df5 100644
--- a/packages/program-space/README.md
+++ b/packages/program-space/README.md
@@ -1,7 +1,20 @@
# `@hypit/program-space`
-The provider-neutral time and frame domain shared by a compiled program.
+The time and frame domain shared by a compiled film. ProgramSpace contains `id`, `durationSec` and
+`frameRate`. Script identity belongs to semantic data; the physical film clock also serves animation
+with no speech or source media.
-`Clock` is the authored, duration-free frame-rate fact for a video project. It is declared once and
-consumed by media normalization and Speech Track; `ProgramSpace` is the later duration-bearing
-domain assembled from the semantic Takes.
+For spoken work, SemanticTrack projects the selected performance's real duration and frame rate.
+For authored animation, `Space` declares them directly:
+
+```svml
+<import as="time" from="@hypit/program-space@1"/>
+<time:Space id="animation" frame-rate="30" duration="8s"/>
+```
+
+`duration` accepts seconds, milliseconds or frames (`8s`, `8000ms`, `240f`); the result must end on an
+exact frame boundary. The rate may be rational, such as `30000/1001`. `Space` publishes an ordinary
+inline ProgramSpace value and performs no media generation.
+
+`Clock` is the duration-free frame-rate input for media normalization. A clock does not prescribe
+how long a performance will be; its actual normalized material establishes that later.
diff --git a/packages/program-space/src/activation.ts b/packages/program-space/src/activation.ts
index d7511bb7..d8c2d877 100644
--- a/packages/program-space/src/activation.ts
+++ b/packages/program-space/src/activation.ts
@@ -1,6 +1,6 @@
import { createMarkupSurfaceHostFacet } from "@hypit/markup";
-import { decodeClockSurface } from "./surface.js";
+import { decodeClockSurface, decodeSpaceSurface } from "./surface.js";
import { programSpaceManifest, programSpaceMarkupSurfaces, programSpaceModuleRef } from "./index.js";
export const hypitPackage = {
@@ -10,6 +10,10 @@ export const hypitPackage = {
module: programSpaceModuleRef,
declaration: programSpaceMarkupSurfaces.find((item) => item.name === "clock")!,
handler: decodeClockSurface,
+ }), createMarkupSurfaceHostFacet({
+ module: programSpaceModuleRef,
+ declaration: programSpaceMarkupSurfaces.find((item) => item.name === "space")!,
+ handler: decodeSpaceSurface,
})],
};
export default hypitPackage;
diff --git a/packages/program-space/src/index.ts b/packages/program-space/src/index.ts
index ba773ef7..9abc2eb5 100644
--- a/packages/program-space/src/index.ts
+++ b/packages/program-space/src/index.ts
@@ -5,10 +5,8 @@ export type ProgramClock = {
};
export type ProgramSpace = {
- /** Author-visible identity of the SemanticTrack defining this time axis. */
+ /** Author-visible identity of this film time axis. */
readonly id: string;
- /** Author-visible Script identity behind the semantic time axis. */
- readonly narrativeId: string;
readonly durationSec: number;
readonly frameRate: { readonly numerator: number; readonly denominator: number };
};
@@ -25,7 +23,6 @@ export const programSpaceSchema: ValueSchema = {
kind: "object",
fields: {
id: { schema: string },
- narrativeId: { schema: string },
durationSec: { schema: number },
frameRate: { schema: { kind: "object", fields: {
numerator: { schema: integer }, denominator: { schema: integer },
@@ -43,6 +40,17 @@ export const programSpaceManifest: ModuleManifest = {
export const programSpaceDependency = { module: programSpaceModuleRef } as const;
export const programSpaceMarkupSurfaces = [
{ name: "clock", tag: "Clock", mode: "structured", outputs: [programSpaceTypes.clock] },
+ { name: "space", tag: "Space", mode: "structured", outputs: [programSpaceTypes.programSpace],
+ vocabulary: {
+ summary: "Declares a film's frame rate and authored duration, for animation whose timing is directed without a performance.",
+ attributes: [
+ { name: "id", kind: "identifier", required: true, summary: "Names the shared film time axis." },
+ { name: "frame-rate", kind: "literal", required: true, summary: "Frames per second, such as 30 or 30000/1001." },
+ { name: "duration", kind: "literal", required: true, summary: "Total duration in seconds, milliseconds or frames, such as 8s or 240f; ends on a frame boundary." },
+ ],
+ example: '<time:Space id="animation" frame-rate="30" duration="8s"/>',
+ },
+ },
] as const;
export function sealProgramClock(value: ProgramClock): ProgramClock {
@@ -90,7 +98,7 @@ export function programFrameSampleBoundary(
}
export function assertProgramSpaceIdentity(programSpace: ProgramSpace): void {
const { numerator, denominator } = programSpace.frameRate;
- if (!programSpace.id.trim() || !programSpace.narrativeId.trim()
+ if (!programSpace.id.trim()
|| !Number.isSafeInteger(numerator) || numerator <= 0 || !Number.isSafeInteger(denominator)
|| denominator <= 0 || !Number.isFinite(programSpace.durationSec) || programSpace.durationSec <= 0) {
throw new Error("ProgramSpace is invalid.");
diff --git a/packages/program-space/src/surface.ts b/packages/program-space/src/surface.ts
index 3f0e3999..7b064749 100644
--- a/packages/program-space/src/surface.ts
+++ b/packages/program-space/src/surface.ts
@@ -5,7 +5,7 @@ import {
type StructuredSurfaceHandler,
} from "@hypit/markup";
-import { programSpaceTypes, sealProgramClock, type ProgramClock } from "./index.js";
+import { assertProgramSpaceIdentity, programSpaceTypes, sealProgramClock, type ProgramClock } from "./index.js";
function frameRate(element: StructuredElement): ProgramClock["frameRate"] {
const value = text(element, "frame-rate");
@@ -34,3 +34,22 @@ export const decodeClockSurface: StructuredSurfaceHandler = ({ element }) => {
fragments: [],
};
};
+
+export const decodeSpaceSurface: StructuredSurfaceHandler = ({ element }) => {
+ const expected = ["duration", "frame-rate", "id"];
+ if (Object.keys(element.attributes).sort().join("\0") !== expected.join("\0")) {
+ throw new Error(`${element.name} requires exactly id, frame-rate and duration.`);
+ }
+ empty(element);
+ const id = text(element, "id");
+ const rate = frameRate(element);
+ const duration = text(element, "duration");
+ const match = /^(\d+(?:\.\d+)?)(s|ms|f)$/u.exec(duration);
+ if (match === null) throw new Error(`${element.name}.duration must be a duration such as 8s, 8000ms or 240f.`);
+ const value = Number(match[1]);
+ const durationSec = match[2] === "f" ? value * rate.denominator / rate.numerator
+ : match[2] === "ms" ? value / 1000 : value;
+ const space = { id, durationSec, frameRate: rate };
+ assertProgramSpaceIdentity(space);
+ return { records: [{ id, type: programSpaceTypes.programSpace, value: { kind: "inline", value: space }, range: element.range }], components: [], fragments: [] };
+};
diff --git a/packages/provider-hyperframes-local/test/provider.test.ts b/packages/provider-hyperframes-local/test/provider.test.ts
index f7871787..b8557c9c 100644
--- a/packages/provider-hyperframes-local/test/provider.test.ts
+++ b/packages/provider-hyperframes-local/test/provider.test.ts
@@ -20,8 +20,7 @@ const liveEnabled = process.env.HYPIT_BROWSER_TESTS === "1";
const hasFfprobe = spawnSync("ffprobe", ["-version"], { stdio: "ignore", windowsHide: true }).status === 0;
function documentFixture(surface?: CompositableSurfaceRef) {
- const programSpace = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 1,
+ const programSpace = sealProgramSpace({ id: "test-space", durationSec: 1,
frameRate: { numerator: 12, denominator: 1 },
});
const track = sealVisualTrack({ programSpaceId: "test-space",
diff --git a/packages/provider-hyperframes-local/test/range.test.ts b/packages/provider-hyperframes-local/test/range.test.ts
index 8e69eb7d..9c6d7f55 100644
--- a/packages/provider-hyperframes-local/test/range.test.ts
+++ b/packages/provider-hyperframes-local/test/range.test.ts
@@ -24,8 +24,7 @@ import type { HyperframesRenderProgress } from "../src/index.js";
import { distributeFrameRange, sourceFrameAt, sourceWindows, videoSlots } from "../src/sampling.js";
function documentFor(artifact: BlobRef) {
- const space = sealProgramSpace({ id: "range-space", narrativeId: "range-narrative",
- durationSec: 12 / 30, frameRate: { numerator: 30, denominator: 1 } });
+ const space = sealProgramSpace({ id: "range-space", durationSec: 12 / 30, frameRate: { numerator: 30, denominator: 1 } });
const track = sealVisualTrack({ id: "video", visualIr: "hypit.visual-ir@1", programSpaceId: space.id,
presents: [{ id: "sample", span: { startFrame: 0, endFrameExclusive: 12 },
stacking: { order: 0, tieBreak: "sample" }, elements: [{ id: "video", order: 0, kind: "video", artifact,
diff --git a/packages/ranking/README.md b/packages/ranking/README.md
index 16017899..f1408c65 100644
--- a/packages/ranking/README.md
+++ b/packages/ranking/README.md
@@ -1,8 +1,12 @@
# `@hypit/ranking`
+The Track Surface accepts `semantic={speech.semantic}` or `space={animation}`. Semantic context
+resolves Script Selections and Moments; authored space supports clock-based animation. Shared `at`
+inputs accept a Moment or a time such as `2s`; `at` with `for` produces a Window where required.
+
Ranking boards that follow the argument of a video: introduce an item, give its verdict, and retain
-its place while the next item is discussed. The board consumes the performance's SemanticTrack,
-Frames, Styles, fonts, text and optional icons or sound.
+its place while the next item is discussed. Its Surface binds the performance's semantic time or
+authored time; the board consumes projected timing, Frames, Styles, fonts, text and optional icons or sound.
| Form | Temporal behavior |
| --- | --- |
@@ -46,7 +50,7 @@ The following files are included in the Distribution and show how the responsibi
| File | What to learn from it |
| --- | --- |
| [surface.ts](src/surface.ts) | `rankingSurface` resolves author inputs, projects the outer and item times through `createTemporalWindowProjection` / `createTemporalInstantProjection`, and retains the returned drafts and references. |
-| [fragment.ts](src/fragment.ts) | `createRankingFragment` projects ProgramSpace and wires typed content, time, layout and Style inputs into finite operations. |
+| [fragment.ts](src/fragment.ts) | `createRankingFragment` receives ProgramSpace and wires typed content, time, layout and Style inputs into finite operations. |
| [schedule.ts](src/schedule.ts) | Compute reveal, activation and settled spans from the projected times. |
| [component.ts](src/component.ts) and [render.ts](src/render.ts) | Build the ranking program and produce picture and optional sound from that schedule. |
| [manifest.ts](src/manifest.ts) and [activation.ts](src/activation.ts) | Publish Types, Producers, Surface vocabulary and package contributions. |
diff --git a/packages/ranking/src/fragment.ts b/packages/ranking/src/fragment.ts
index 790e75cd..27d68f1e 100644
--- a/packages/ranking/src/fragment.ts
+++ b/packages/ranking/src/fragment.ts
@@ -1,8 +1,8 @@
+import { programSpaceTypes } from "@hypit/program-space";
import { compositionTypes } from "@hypit/composition";
import { sealGraphFragment } from "@hypit/elaborator";
import type { FragmentOperation, GraphFragment } from "@hypit/elaborator";
import { mediaTypes } from "@hypit/media";
-import { semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";
import { spatialTypes } from "@hypit/spatial";
import { textTypes } from "@hypit/text";
import { temporalTypes } from "@hypit/temporal";
@@ -56,7 +56,7 @@ export function createRankingFragment(
const selected = definition(variant);
const inputs: Array<GraphFragment["inputs"][number]> = [
{ name: "header", type: rankingTypes.header },
- { name: "semantic", type: semanticTrackTypes.track },
+ { name: "space", type: programSpaceTypes.programSpace },
{ name: "outer", type: temporalTypes.window },
...(variant === "top-three" ? [
{ name: "terminal", type: temporalTypes.instant },
@@ -66,8 +66,6 @@ export function createRankingFragment(
{ name: "style", type: selected.style },
];
const operations: FragmentOperation[] = [
- { id: "space", producer: semanticTrackProducers.projectProgramSpace,
- inputs: { track: input("semantic") }, result: { kind: "output", name: "space" } },
{ id: "specs", producer: rankingProducers.createSpecs, inputs: { header: input("header") }, result: { kind: "output", name: "set" } },
{ id: "resolved", producer: selected.create, inputs: {}, result: { kind: "output", name: "set" } },
{ id: "candidates", producer: variant === "column"
@@ -135,7 +133,7 @@ export function createRankingFragment(
operations.push({
id: "schedule",
producer: variant === "column" ? rankingProducers.columnSchedule : rankingProducers.tierSchedule,
- inputs: { header: input("header"), items: specs, space: operation("space"), outer: input("outer"), windows: candidates },
+ inputs: { header: input("header"), items: specs, space: input("space"), outer: input("outer"), windows: candidates },
result: { kind: "output", name: "schedule" },
});
} else {
@@ -143,7 +141,7 @@ export function createRankingFragment(
id: "schedule",
producer: rankingProducers.schedule,
inputs: {
- header: input("header"), items: specs, space: operation("space"),
+ header: input("header"), items: specs, space: input("space"),
outer: input("outer"), candidates, terminal: input("terminal"),
},
result: { kind: "output", name: "schedule" },
@@ -162,7 +160,7 @@ export function createRankingFragment(
operations.push({
id: "visual",
producer: selected.render,
- inputs: { space: operation("space"), program: operation("program") },
+ inputs: { space: input("space"), program: operation("program") },
result: { kind: "output", name: "track" },
});
const hasAudio = sound.appearName !== undefined || sound.moveName !== undefined;
@@ -191,7 +189,7 @@ export function createRankingFragment(
}
operations.push({
id: "audio", producer: rankingProducers.renderAudio,
- inputs: { space: operation("space"), events: operation("events"), style: input("sound-style"), sounds },
+ inputs: { space: input("space"), events: operation("events"), style: input("sound-style"), sounds },
result: { kind: "output", name: "track" },
});
}
diff --git a/packages/ranking/src/manifest.ts b/packages/ranking/src/manifest.ts
index 262ef0db..47cbe036 100644
--- a/packages/ranking/src/manifest.ts
+++ b/packages/ranking/src/manifest.ts
@@ -1,3 +1,4 @@
+import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";
import { readFile } from "node:fs/promises";
import { artifactDependency } from "@hypit/artifact";
@@ -6,7 +7,7 @@ import { mediaDependency, mediaTypes } from "@hypit/media";
import { narrativeDependency, narrativeTypes } from "@hypit/narrative";
import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";
import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";
-import { semanticTrackDependency, semanticTrackTypes } from "@hypit/semantic-track";
+import { semanticTrackDependency } from "@hypit/semantic-track";
import { spatialDependency, spatialFrameSchema, spatialTypes } from "@hypit/spatial";
import { svsRecipeType } from "@hypit/svs";
import { temporalDependency, temporalInstantSchema, temporalTypes, temporalWindowSchema } from "@hypit/temporal";
@@ -495,14 +496,13 @@ const allRankingMarkupSurfaces = [
attributes: [
{ name: "id", kind: "identifier", required: true,
summary: "Names this board so its Schedule, Program and Tracks can be referenced elsewhere in the Source." },
- { name: "semantic", kind: "reference", required: true, accepts: [semanticTrackTypes.track],
- summary: "Chooses the SemanticTrack that owns the frame domain and resolves the board and Item windows." },
+ ...temporalContextAttributeVocabulary,
{ name: "canvas", kind: "reference", required: true, accepts: [spatialTypes.canvas],
summary: "Chooses the Canvas coordinate space used by the independent explanation stage." },
{ name: "frame", kind: "reference", required: true, accepts: [spatialTypes.frame],
summary: "Chooses the compact Frame occupied only by the tier table." },
{ name: "during", kind: "expression", required: true, values: ["program"], accepts: [narrativeTypes.selection, narrativeTypes.excerpt],
- summary: "Spans the complete SemanticTrack when written as program, or projects the referenced Segment or Selection into the board lifetime." },
+ summary: "Spans the complete film clock when written as program, or projects the referenced Segment or Selection into the board lifetime." },
{ name: "style", kind: "reference", required: true, accepts: [rankingTypes.tierStyle],
summary: "Chooses the TierBoardStyle this board is drawn in, and only that variant's." },
{ name: "appear-sound", kind: "reference", required: false, accepts: [mediaTypes.synchronized],
@@ -564,14 +564,13 @@ const allRankingMarkupSurfaces = [
attributes: [
{ name: "id", kind: "identifier", required: true,
summary: "Names this board so its Schedule, Program and Tracks can be referenced elsewhere in the Source." },
- { name: "semantic", kind: "reference", required: true, accepts: [semanticTrackTypes.track],
- summary: "Chooses the SemanticTrack that owns the frame domain and resolves every reveal window." },
+ ...temporalContextAttributeVocabulary,
{ name: "canvas", kind: "reference", required: true, accepts: [spatialTypes.canvas],
summary: "Chooses the Canvas coordinate space used by the independent reveal stage." },
{ name: "frame", kind: "reference", required: true, accepts: [spatialTypes.frame],
summary: "Chooses the fixed Frame occupied by the vertical rank rail." },
{ name: "during", kind: "expression", required: true, values: ["program"], accepts: [narrativeTypes.selection, narrativeTypes.excerpt],
- summary: "Spans the complete SemanticTrack when written as program, or projects the referenced Segment or Selection into the Column lifetime." },
+ summary: "Spans the complete film clock when written as program, or projects the referenced Segment or Selection into the Column lifetime." },
{ name: "style", kind: "reference", required: true, accepts: [rankingTypes.columnStyle],
summary: "Chooses the ColumnStyle this board is drawn in, and only that variant's." },
{ name: "appear-sound", kind: "reference", required: false, accepts: [mediaTypes.synchronized],
@@ -634,14 +633,13 @@ const allRankingMarkupSurfaces = [
attributes: [
{ name: "id", kind: "identifier", required: true,
summary: "Names this board so its Schedule, Program and Tracks can be referenced elsewhere in the Source." },
- { name: "semantic", kind: "reference", required: true, accepts: [semanticTrackTypes.track],
- summary: "Chooses the SemanticTrack that owns the frame domain and resolves every Item Moment." },
+ ...temporalContextAttributeVocabulary,
{ name: "frame", kind: "reference", required: true, accepts: [spatialTypes.frame],
summary: "Chooses the Frame the whole board occupies." },
- { name: "during", kind: "reference", required: true, accepts: [narrativeTypes.selection],
- summary: "Chooses the Selection the board is on screen for." },
- { name: "terminal", kind: "reference", required: true, accepts: [narrativeTypes.moment],
- summary: "Chooses the Moment the board settles on and ends after." },
+ { name: "during", kind: "expression", required: true, values: ["program"], accepts: [narrativeTypes.selection, narrativeTypes.excerpt],
+ summary: "Chooses the board's whole-program, Selection or Segment lifetime." },
+ { name: "terminal", kind: "expression", required: true, accepts: [narrativeTypes.moment],
+ summary: "Chooses the terminal Moment or authored time, such as 8s." },
{ name: "style", kind: "reference", required: true, accepts: [rankingTypes.topThreeStyle],
summary: "Chooses the TopThreeStyle this board is drawn in, and only that variant's." },
{ name: "appear-sound", kind: "reference", required: false, accepts: [mediaTypes.synchronized],
diff --git a/packages/ranking/src/surface.ts b/packages/ranking/src/surface.ts
index 4ac2b85e..c600df9b 100644
--- a/packages/ranking/src/surface.ts
+++ b/packages/ranking/src/surface.ts
@@ -1,3 +1,4 @@
+import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";
import {
assertEmptyElement as empty,
assertAttributes as allowed,
@@ -14,7 +15,6 @@ import {
import { sameType, type CanonicalValue, type TypeRef } from "@hypit/protocol";
import { mediaTypes } from "@hypit/media";
import type { FontArtifactRef, FontStackRef } from "@hypit/media";
-import { semanticTrackTypes } from "@hypit/semantic-track";
import { spatialTypes } from "@hypit/spatial";
import { svsRecipeType } from "@hypit/svs";
import type { SvsRecipe } from "@hypit/svs";
@@ -199,38 +199,39 @@ const variantDefinition = {
function rankingSurface(variant: RankingVariant): StructuredSurfaceHandler {
return ({ element, resolveReference }) => {
- const common = ["id", "semantic", "frame", "during", "terminal", "style", "appear-sound", "move-sound"];
+ const common = ["id", "semantic", "space", "frame", "during", "terminal", "style", "appear-sound", "move-sound"];
const attributes = variant === "column" || variant === "tier-board"
? [...common.filter((name) => name !== "terminal"), "canvas"]
: common;
allowed(element, attributes);
const id = text(element, "id");
const selected = variantDefinition[variant];
- const semantic = reference(element.attributes.semantic, `${element.name}.semantic`, semanticTrackTypes.track, resolveReference);
+ const context = resolveTemporalContext({ element, resolveReference });
+ const time = createTemporalSpace({ id, element, ...context });
const canvas = variant === "column" || variant === "tier-board"
? reference(element.attributes.canvas, `${element.name}.canvas`, spatialTypes.canvas, resolveReference)
: undefined;
const frame = reference(element.attributes.frame, `${element.name}.frame`, spatialTypes.frame, resolveReference);
- const outerTemporal = createTemporalWindowProjection({ id: `${id}.outer`, subjectId: id, element, semantic, resolveReference });
+ const outerTemporal = createTemporalWindowProjection({ id: `${id}.outer`, subjectId: id, element, ...context, space: time.space, resolveReference });
const terminalTemporal = variant === "top-three"
? createTemporalInstantProjection({
- id: `${id}.terminal`, subjectId: id, element, semantic, resolveReference,
+ id: `${id}.terminal`, subjectId: id, element, ...context, space: time.space, resolveReference,
semanticAttribute: "terminal", projectedAttribute: false,
})
: undefined;
const styleRaw = element.attributes.style;
const style = reference(styleRaw, `${element.name}.style`, selected.style, resolveReference);
const records: SurfaceRecordDraft[] = [...outerTemporal.records, ...(terminalTemporal?.records ?? [])];
- const temporalComponents: SurfaceComponentDraft[] = [...outerTemporal.components, ...(terminalTemporal?.components ?? [])];
- const temporalFragments = [...outerTemporal.fragments, ...(terminalTemporal?.fragments ?? [])];
+ const temporalComponents: SurfaceComponentDraft[] = [...time.components, ...outerTemporal.components, ...(terminalTemporal?.components ?? [])];
+ const temporalFragments = [...time.fragments, ...outerTemporal.fragments, ...(terminalTemporal?.fragments ?? [])];
const headerId = `${id}.header`;
records.push({
id: headerId, type: rankingTypes.header,
value: { kind: "inline", value: sealRankingHeader({ id, variant }) as unknown as CanonicalValue },
range: element.range,
});
- const inputs: Record<string, typeof semantic.ref> = {
- header: { kind: "record", id: headerId }, semantic: semantic.ref, frame: frame.ref,
+ const inputs: Record<string, typeof time.space.ref> = {
+ header: { kind: "record", id: headerId }, space: time.space.ref, frame: frame.ref,
outer: outerTemporal.ref, style: style.ref,
...(canvas === undefined ? {} : { canvas: canvas.ref }),
...(terminalTemporal === undefined ? {} : { terminal: terminalTemporal.ref }),
@@ -259,8 +260,8 @@ function rankingSurface(variant: RankingVariant): StructuredSurfaceHandler {
const contentName = authored.content === undefined ? undefined : `item-${suffix}-content`;
if (authored.content !== undefined) inputs[contentName!] = authored.content.ref;
const itemTemporal = !authored.timed ? undefined : variant === "top-three"
- ? createTemporalInstantProjection({ id: `${id}.item.${suffix}`, subjectId: spec.id, element: child, semantic, resolveReference })
- : createTemporalWindowProjection({ id: `${id}.item.${suffix}`, subjectId: spec.id, element: child, semantic, resolveReference });
+ ? createTemporalInstantProjection({ id: `${id}.item.${suffix}`, subjectId: spec.id, element: child, ...context, space: time.space, resolveReference })
+ : createTemporalWindowProjection({ id: `${id}.item.${suffix}`, subjectId: spec.id, element: child, ...context, space: time.space, resolveReference });
if (itemTemporal !== undefined) {
records.push(...itemTemporal.records);
temporalComponents.push(...itemTemporal.components);
diff --git a/packages/ranking/test/ranking.test.ts b/packages/ranking/test/ranking.test.ts
index 12fbf0d2..55c8e3b6 100644
--- a/packages/ranking/test/ranking.test.ts
+++ b/packages/ranking/test/ranking.test.ts
@@ -91,8 +91,7 @@ import type {
MarkupAttributeValue,
} from "@hypit/markup";
-const space = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 8,
+const space = sealProgramSpace({ id: "test-space", durationSec: 8,
frameRate: { numerator: 30, denominator: 1 },
});
const canvas = sealCanvasSpace({
@@ -128,19 +127,19 @@ const semanticTrack = semanticTrackFixture(space, {
],
});
const outer: NarrativeSelectionRef = {
- narrativeId: space.narrativeId,
+ narrativeId: "script",
id: "ranking-window",
startAnchorId: "outer-start",
endAnchorId: "outer-end",
};
const terminal: NarrativeMomentRef = {
- narrativeId: space.narrativeId,
+ narrativeId: "script",
id: "ranking-complete",
anchorId: "terminal",
};
-const rankingSegment: NarrativeExcerpt = { narrativeId: space.narrativeId, kind: "segment", id: "ranking", tokenStart: 0, tokenEndExclusive: 1 };
+const rankingSegment: NarrativeExcerpt = { narrativeId: "script", kind: "segment", id: "ranking", tokenStart: 0, tokenEndExclusive: 1 };
const selection = (id: string, startAnchorId: string, endAnchorId: string): NarrativeSelectionRef => ({
- narrativeId: space.narrativeId, id, startAnchorId, endAnchorId,
+ narrativeId: "script", id, startAnchorId, endAnchorId,
});
const early = selection("early", "early-start", "early-end");
const overlapping = selection("overlapping", "overlap-start", "overlap-end");
@@ -174,7 +173,7 @@ function schedule(
let candidates = createTriggeredRankingCandidateSet();
for (const [index, value] of values.entries()) {
candidates = appendTriggeredRankingCandidate(candidates, value, semanticTrack, {
- narrativeId: space.narrativeId,
+ narrativeId: "script",
id: `${value.id}-moment`,
anchorId: anchors[index]!,
});
@@ -303,7 +302,7 @@ test("RankingSchedule requires one item-owned Moment per Item and distinct chron
const owner = header("top-three");
const values = [topSpec("a"), topSpec("b")];
let missing = createTriggeredRankingCandidateSet();
- missing = appendTriggeredRankingCandidate(missing, values[0]!, semanticTrack, { narrativeId: space.narrativeId, id: "a-at", anchorId: "one" });
+ missing = appendTriggeredRankingCandidate(missing, values[0]!, semanticTrack, { narrativeId: "script", id: "a-at", anchorId: "one" });
assert.throws(() => buildRankingSchedule({
header: owner, items: specs(owner, values), semantic: semanticTrack, outer, candidates: missing, terminal,
}), /item-owned Moments differ/u);
diff --git a/packages/render-hyperframes/README.md b/packages/render-hyperframes/README.md
index 6c663424..f120b6e0 100644
--- a/packages/render-hyperframes/README.md
+++ b/packages/render-hyperframes/README.md
@@ -1,5 +1,9 @@
# `@hypit/render-hyperframes`
+The Surface accepts `semantic={speech.semantic}` for performance time or `space={animation}` for an
+authored ProgramSpace. Its Fragment receives `space` directly; rendering does not require Script
+or prepared performance media. Both time sources use the same frame and audio pipeline.
+
Explicit author and capability boundary for final HyperFrames video rendering.
The package owns `<render:Video composition={...}/>` and lowers it to ordinary Operations that:
diff --git a/packages/render-hyperframes/package.json b/packages/render-hyperframes/package.json
index cc19b262..768833fe 100644
--- a/packages/render-hyperframes/package.json
+++ b/packages/render-hyperframes/package.json
@@ -21,7 +21,8 @@
"@hypit/program-space": "workspace:*",
"@hypit/protocol": "workspace:*",
"@hypit/semantic-track": "workspace:*",
- "@hypit/markup": "workspace:*"
+ "@hypit/markup": "workspace:*",
+ "@hypit/temporal-markup": "workspace:*"
},
"devDependencies": {
"@hypit/core": "workspace:*",
diff --git a/packages/render-hyperframes/src/fragment.ts b/packages/render-hyperframes/src/fragment.ts
index e938d6ce..2f04eef0 100644
--- a/packages/render-hyperframes/src/fragment.ts
+++ b/packages/render-hyperframes/src/fragment.ts
@@ -1,6 +1,6 @@
import { mediaTypes } from "@hypit/media";
import { artifactTypes } from "@hypit/artifact";
-import { semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";
+import { programSpaceTypes } from "@hypit/program-space";
import { compositionTypes } from "@hypit/composition";
import { sealGraphFragment } from "@hypit/elaborator";
import { hyperframesProducers } from "@hypit/hyperframes";
@@ -17,20 +17,14 @@ export function createRenderHyperframesFragment(selectedRange = false) {
return sealGraphFragment({
inputs: [
{ name: "composition", type: compositionTypes.composition },
- { name: "semantic", type: semanticTrackTypes.track },
+ { name: "space", type: programSpaceTypes.programSpace },
...(selectedRange ? [{ name: "range", type: mediaTypes.frameRange }] : []),
],
operations: [
- {
- id: "project-space",
- producer: semanticTrackProducers.projectProgramSpace,
- inputs: { track: input("semantic") },
- result: { kind: "output", name: "space" },
- },
{
id: "compile-document",
producer: hyperframesProducers.compile,
- inputs: { composition: input("composition"), space: operation("project-space") },
+ inputs: { composition: input("composition"), space: input("space") },
result: { kind: "output", name: "document" },
},
{
@@ -42,7 +36,7 @@ export function createRenderHyperframesFragment(selectedRange = false) {
{
id: "compile-audio-program",
producer: mediaPipelineProducers.planAudio,
- inputs: { composition: input("composition"), space: operation("project-space") },
+ inputs: { composition: input("composition"), space: input("space") },
result: { kind: "output", name: "plan" },
},
{
diff --git a/packages/render-hyperframes/src/manifest.ts b/packages/render-hyperframes/src/manifest.ts
index 045b57ee..273847bf 100644
--- a/packages/render-hyperframes/src/manifest.ts
+++ b/packages/render-hyperframes/src/manifest.ts
@@ -1,7 +1,8 @@
+import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";
import { artifactDependency, artifactTypes } from "@hypit/artifact";
import { mediaDependency, mediaTypes } from "@hypit/media";
import { compositionDependency, compositionTypes } from "@hypit/composition";
-import { semanticTrackDependency, semanticTrackTypes } from "@hypit/semantic-track";
+import { semanticTrackDependency } from "@hypit/semantic-track";
import {
hyperframesManifest,
hyperframesModuleRef,
@@ -32,16 +33,14 @@ export const renderHyperframesMarkupSurfaces = [{
outputs: [mediaTypes.frameRange],
vocabulary: {
summary:
- "Renders one Composition on one SemanticTrack into a final video, publishing the muxed result as a BlobArtifact.",
+ "Renders one Composition on its selected film time axis into a final video, publishing the muxed result as a BlobArtifact.",
attributes: [
{ name: "id", kind: "identifier", required: true,
summary: "Names the render component and the final video this element publishes." },
{ name: "composition", kind: "reference", required: true,
accepts: [compositionTypes.composition],
summary: "Selects the Composition this element compiles, renders and muxes." },
- { name: "semantic", kind: "reference", required: true,
- accepts: [semanticTrackTypes.track],
- summary: "Selects the SemanticTrack whose derived duration and frame rate every rendered Product is bound to." },
+ ...temporalContextAttributeVocabulary,
{ name: "start-frame", kind: "literal", required: false, summary: "First original programme frame to render; requires end-frame-exclusive." },
{ name: "end-frame-exclusive", kind: "literal", required: false, summary: "First excluded frame; requires start-frame." },
],
@@ -51,7 +50,7 @@ export const renderHyperframesMarkupSurfaces = [{
],
example: '<render:Video id="final" composition={main.composition} semantic={speech.semantic}/>',
notes: [
- "id, composition and semantic are required; write both frame bounds to select a range; the element accepts no children and no text content.",
+ "Supply id, composition and one time source: semantic or space. Write both frame bounds to select a range. The element is empty.",
"The visual render, the audio render and the mux are three separate Needs, each realized by a Provider this package does not choose.",
"The published Artifact carries no duration or lineage metadata, so a consumer that needs stream facts requests explicit media inspection.",
],
diff --git a/packages/render-hyperframes/src/surface.ts b/packages/render-hyperframes/src/surface.ts
index 7ece8af5..c14c633c 100644
--- a/packages/render-hyperframes/src/surface.ts
+++ b/packages/render-hyperframes/src/surface.ts
@@ -1,4 +1,4 @@
-import { semanticTrackTypes } from "@hypit/semantic-track";
+import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";
import { compositionTypes } from "@hypit/composition";
import { mediaTypes, verifyMediaFrameRange } from "@hypit/media";
import type {
@@ -25,7 +25,7 @@ function referenceAttribute(element: StructuredElement, name: string): string {
export const decodeHyperframesRenderSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
const names = Object.keys(element.attributes).sort();
- if (names.some((name) => !["composition", "id", "semantic", "start-frame", "end-frame-exclusive"].includes(name))) {
+ if (names.some((name) => !["composition", "id", "semantic", "space", "start-frame", "end-frame-exclusive"].includes(name))) {
throw new Error(`${element.name} contains unsupported attributes`);
}
if (element.children.some((child) => child.kind === "element" || child.value.trim().length > 0)) {
@@ -40,14 +40,7 @@ export const decodeHyperframesRenderSurface: StructuredSurfaceHandler = ({ eleme
|| composition.type.name !== compositionTypes.composition.name) {
throw new Error(`${element.name}.composition must reference Composition`);
}
- const semanticPath = referenceAttribute(element, "semantic");
- const semantic = resolveReference(semanticPath);
- if (semantic === undefined
- || semantic.type.module.name !== semanticTrackTypes.track.module.name
- || semantic.type.module.version !== semanticTrackTypes.track.module.version
- || semantic.type.name !== semanticTrackTypes.track.name) {
- throw new Error(`${element.name}.semantic must reference SemanticTrack`);
- }
+ const time = createTemporalSpace({ id, element, ...resolveTemporalContext({ element, resolveReference }) });
const selected = element.attributes["start-frame"] !== undefined || element.attributes["end-frame-exclusive"] !== undefined;
const frame = (name: string): number => {
const text = stringAttribute(element, name);
@@ -60,13 +53,13 @@ export const decodeHyperframesRenderSurface: StructuredSurfaceHandler = ({ eleme
const rangeId = `${id}.frame-range`;
return {
records: range === undefined ? [] : [{ id: rangeId, type: mediaTypes.frameRange, value: { kind: "inline", value: range }, range: element.range }],
- components: [{
+ components: [...time.components, {
id,
fragment: fragment.id,
- inputs: { composition: composition.ref, semantic: semantic.ref, ...(selected ? { range: { kind: "record" as const, id: rangeId } } : {}) },
+ inputs: { composition: composition.ref, space: time.space.ref, ...(selected ? { range: { kind: "record" as const, id: rangeId } } : {}) },
outputs: { video: `${id}.video` },
range: element.range,
}],
- fragments: [fragment],
+ fragments: [...time.fragments, fragment],
};
};
diff --git a/packages/render-hyperframes/test/render.test.ts b/packages/render-hyperframes/test/render.test.ts
index 2a9a6a59..e4a9d4fb 100644
--- a/packages/render-hyperframes/test/render.test.ts
+++ b/packages/render-hyperframes/test/render.test.ts
@@ -84,8 +84,7 @@ import {
semanticTrackTypes,
} from "@hypit/semantic-track";
-const space = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 2,
+const space = sealProgramSpace({ id: "test-space", durationSec: 2,
frameRate: { numerator: 30, denominator: 1 },
});
const semantic = semanticTrackFixture(space);
@@ -113,18 +112,18 @@ const compositionRecord = await admitRecord(closure, sealRecord({
type: compositionTypes.composition,
value: stored(composition),
}), validatorRegistry());
-const semanticRecord = await admitRecord(closure, sealRecord({
- id: "semantic",
- type: semanticTrackTypes.track,
- value: stored(semantic),
+const spaceRecord = await admitRecord(closure, sealRecord({
+ id: "space",
+ type: programSpaceTypes.programSpace,
+ value: stored(space),
}), validatorRegistry());
-const linked = link(closure, [compositionRecord, semanticRecord]);
+const linked = link(closure, [compositionRecord, spaceRecord]);
const instance = elaborateGraphFragment(linked, renderHyperframesFragment, {
id: "final",
fragment: renderHyperframesFragment.id,
inputs: {
composition: { kind: "record", id: compositionRecord.id },
- semantic: { kind: "record", id: semanticRecord.id },
+ space: { kind: "record", id: spaceRecord.id },
},
});
const contribution = bindAuthorFragment(instance, { video: "final.video" });
@@ -156,7 +155,6 @@ function validatorRegistry(): TypeValidatorRegistry {
test("HyperFrames rendering is an explicit exact Need after ordinary document compilation", async () => {
assert.deepEqual(build().plan.steps.map((step) => step.producer.name).sort(), [
- "project-program-space",
hyperframesProducers.compile.name,
renderHyperframesProducers.requestVisual.name,
mediaPipelineProducers.planAudio.name,
@@ -410,12 +408,12 @@ test("selected render frames reach both visual and audio Needs through the ordin
const range = { startFrame: 15, endFrameExclusive: 45 };
const rangeRecord = await admitRecord(closure, sealRecord({ id: "selection", type: mediaTypes.frameRange,
value: stored(range) }), validatorRegistry());
- const program = link(closure, [compositionRecord, semanticRecord, rangeRecord]);
+ const program = link(closure, [compositionRecord, spaceRecord, rangeRecord]);
const fragment = createRenderHyperframesFragment(true);
const selected = elaborateGraphFragment(program, fragment, {
id: "selected", fragment: fragment.id, inputs: {
composition: { kind: "record", id: compositionRecord.id },
- semantic: { kind: "record", id: semanticRecord.id },
+ space: { kind: "record", id: spaceRecord.id },
range: { kind: "record", id: rangeRecord.id },
},
});
diff --git a/packages/screen-overlay/README.md b/packages/screen-overlay/README.md
index cc0056c2..73df602d 100644
--- a/packages/screen-overlay/README.md
+++ b/packages/screen-overlay/README.md
@@ -1,11 +1,17 @@
# `@hypit/screen-overlay`
+The Track Surface accepts `semantic={speech.semantic}` or `space={animation}`. Semantic context
+resolves Script Selections and Moments; authored space supports clock-based animation. Shared `at`
+inputs accept a Moment or a time such as `2s`; `at` with `for` produces a Window where required.
+
Official provider-free self-contained screen-paint package.
-Every item consumes only its typed parameters, explicit Temporal source, CanvasSpace, ProgramSpace,
+Every item consumes its typed parameters, explicit Temporal source, CanvasSpace, ProgramSpace,
seed and local frame. Flash, ColorWash, Vignette, ScanLines, DirectionalMatte, WhipVeil,
GlitchVeil, Grain, LightLeak, Bokeh and TVStatic lower to ordinary absolute-stack `VisualTrack`
-Presents. No item can read lower pixels, sibling Tracks, Film state or hidden audio.
+Presents. These effects generate their own pixels over the selected interval.
-Gaussian blur, color adjustment, zoom blur and cross-Track blend modes intentionally do not exist in
-this package. Their honest form is an explicit-input media/surface transformation.
+For a transition or filter shared by several pictures, a component can own the participating content
+and its compositing scope. Structural styles and a local [browser program](../hyperframes/README.md#local-browser-programs)
+support filtering, backdrop filtering and blending. A media/surface transformation is useful when
+the intended output is a newly prepared asset. Choose the operation according to what it changes.
diff --git a/packages/screen-overlay/src/fragment.ts b/packages/screen-overlay/src/fragment.ts
index fd9ac05c..7c5b1713 100644
--- a/packages/screen-overlay/src/fragment.ts
+++ b/packages/screen-overlay/src/fragment.ts
@@ -1,7 +1,7 @@
+import { programSpaceTypes } from "@hypit/program-space";
import { compositionTypes } from "@hypit/composition";
import { sealGraphFragment } from "@hypit/elaborator";
import type { FragmentOperation } from "@hypit/elaborator";
-import { semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";
import { spatialTypes } from "@hypit/spatial";
import { temporalTypes } from "@hypit/temporal";
import { screenOverlayProducers, screenOverlayTypes } from "./manifest.js";
@@ -13,7 +13,6 @@ export function createScreenOverlayFragment(items: readonly ScreenOverlayFragmen
if (items.length === 0) throw new Error("Screen Overlay Fragment requires at least one Item.");
const types = new Map<string, typeof screenOverlayTypes.itemSpec | typeof temporalTypes.window>();
const operations: FragmentOperation[] = [
- { id: "overlay:space", producer: semanticTrackProducers.projectProgramSpace, inputs: { track: input("semantic") }, result: { kind: "output", name: "space" } },
{ id: "overlay:set:empty", producer: screenOverlayProducers.createSet, inputs: {}, result: { kind: "output", name: "set" } },
];
let current = "overlay:set:empty";
@@ -23,16 +22,16 @@ export function createScreenOverlayFragment(items: readonly ScreenOverlayFragmen
const id = `overlay:set:append:${String(index + 1).padStart(4, "0")}`;
operations.push({ id, producer: screenOverlayProducers.appendItem,
inputs: { set: operation(current), header: input("header"), spec: input(item.specName),
- space: operation("overlay:space"), window: input(item.windowName) }, result: { kind: "output", name: "set" } });
+ space: input("space"), window: input(item.windowName) }, result: { kind: "output", name: "set" } });
current = id;
});
operations.push(
{ id: "overlay:program", producer: screenOverlayProducers.finalize, inputs: { set: operation(current), header: input("header") }, result: { kind: "output", name: "program" } },
- { id: "overlay:track", producer: screenOverlayProducers.render, inputs: { canvas: input("canvas"), space: operation("overlay:space"), program: operation("overlay:program") }, result: { kind: "output", name: "track" } },
+ { id: "overlay:track", producer: screenOverlayProducers.render, inputs: { canvas: input("canvas"), space: input("space"), program: operation("overlay:program") }, result: { kind: "output", name: "track" } },
);
return sealGraphFragment({ inputs: [
{ name: "canvas", type: spatialTypes.canvas }, { name: "header", type: screenOverlayTypes.header },
- { name: "semantic", type: semanticTrackTypes.track }, ...[...types].map(([inputName, type]) => ({ name: inputName, type })),
+ { name: "space", type: programSpaceTypes.programSpace }, ...[...types].map(([inputName, type]) => ({ name: inputName, type })),
], operations, exports: [
{ name: "program", type: screenOverlayTypes.program, root: operation("overlay:program") },
{ name: "track", type: compositionTypes.visualTrack, root: operation("overlay:track") },
diff --git a/packages/screen-overlay/src/manifest.ts b/packages/screen-overlay/src/manifest.ts
index 9307b704..d1717642 100644
--- a/packages/screen-overlay/src/manifest.ts
+++ b/packages/screen-overlay/src/manifest.ts
@@ -1,10 +1,11 @@
+import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";
import { readFile } from "node:fs/promises";
import { compositionDependency, compositionTypes } from "@hypit/composition";
import { narrativeDependency } from "@hypit/narrative";
import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";
import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";
-import { semanticTrackDependency, semanticTrackTypes } from "@hypit/semantic-track";
+import { semanticTrackDependency } from "@hypit/semantic-track";
import { spatialDependency, spatialTypes } from "@hypit/spatial";
import { temporalDependency, temporalTypes } from "@hypit/temporal";
import { temporalWindowAttributeVocabulary } from "@hypit/temporal-markup";
@@ -85,8 +86,7 @@ export const screenOverlayMarkupSurfaces = [
summary: "Names this overlay so its Program and Track can be referenced elsewhere in the Source." },
{ name: "canvas", kind: "reference", required: true, accepts: [spatialTypes.canvas],
summary: "Chooses the Canvas every item is painted across." },
- { name: "semantic", kind: "reference", required: true, accepts: [semanticTrackTypes.track],
- summary: "Chooses the SemanticTrack that owns the frame domain and resolves temporal bindings." },
+ ...temporalContextAttributeVocabulary,
],
children: [
{ tag: "Flash", cardinality: "many",
diff --git a/packages/screen-overlay/src/surface.ts b/packages/screen-overlay/src/surface.ts
index 457488c1..7713d184 100644
--- a/packages/screen-overlay/src/surface.ts
+++ b/packages/screen-overlay/src/surface.ts
@@ -1,4 +1,4 @@
-import { semanticTrackTypes } from "@hypit/semantic-track";
+import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";
import { spatialTypes } from "@hypit/spatial";
import type { StructuredElement, StructuredSurfaceHandler, SurfaceComponentDraft, SurfaceRecordDraft, SurfaceResolvedReference, MarkupAttributeValue } from "@hypit/markup";
import { createTemporalWindowProjection, temporalWindowAttributeNames } from "@hypit/temporal-markup";
@@ -59,13 +59,14 @@ function content(element: StructuredElement): { readonly value: ScreenOverlayCom
export const decodeScreenOverlaySurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
allowed(element, ["id", "canvas", "semantic"]); const id = text(element, "id");
const canvas = ref(element.attributes.canvas, `${element.name}.canvas`, spatialTypes.canvas, resolveReference);
- const semantic = ref(element.attributes.semantic, `${element.name}.semantic`, semanticTrackTypes.track, resolveReference);
+ const context = resolveTemporalContext({ element, resolveReference });
+ const time = createTemporalSpace({ id: id, element, ...context });
const headerId = `${id}.header`; const records: SurfaceRecordDraft[] = [{ id: headerId, type: screenOverlayTypes.header,
value: { kind: "inline", value: sealScreenOverlayHeader({ id }) }, range: element.range }];
- const temporalComponents: SurfaceComponentDraft[] = [];
- const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [];
+ const temporalComponents: SurfaceComponentDraft[] = [...time.components];
+ const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [...time.fragments];
const fragmentItems: Parameters<typeof createScreenOverlayFragment>[0][number][] = [];
- const inputs: Record<string, typeof canvas.ref> = { canvas: canvas.ref, header: { kind: "record", id: headerId }, semantic: semantic.ref };
+ const inputs: Record<string, typeof canvas.ref> = { canvas: canvas.ref, header: { kind: "record", id: headerId }, space: time.space.ref };
let index = 0;
for (const child of element.children) {
if (child.kind === "text") { if (child.value.trim()) throw new Error(`${element.name} accepts only component children.`); continue; }
@@ -75,7 +76,7 @@ export const decodeScreenOverlaySurface: StructuredSurfaceHandler = ({ element,
const itemSpec = sealScreenOverlayItemSpec({
id: optionalText(child, "id") ?? `${id}.${decoded.value.kind}.${suffix}`, content: decoded.value,
stackingOrder: integer(child, "z") });
- const temporal = createTemporalWindowProjection({ id: itemSpec.id, element: child, semantic, resolveReference });
+ const temporal = createTemporalWindowProjection({ id: itemSpec.id, element: child, ...context, space: time.space, resolveReference });
records.push(...temporal.records); temporalComponents.push(...temporal.components); temporalFragments.push(...temporal.fragments);
const specId = `${id}.item.${suffix}.spec`; const specName = `item-${suffix}-spec`;
records.push({ id: specId, type: screenOverlayTypes.itemSpec, value: { kind: "inline", value: itemSpec }, range: child.range });
diff --git a/packages/screen-overlay/test/screen-overlay.test.ts b/packages/screen-overlay/test/screen-overlay.test.ts
index 6858cb05..a2dae0d5 100644
--- a/packages/screen-overlay/test/screen-overlay.test.ts
+++ b/packages/screen-overlay/test/screen-overlay.test.ts
@@ -43,8 +43,7 @@ const canvas = sealCanvasSpace({
widthPx: 1080, heightPx: 1920,
origin: "top-left", xDirection: "right", yDirection: "down", pixelAspect: "square",
});
-const space = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 2, frameRate: { numerator: 30, denominator: 1 },
+const space = sealProgramSpace({ id: "test-space", durationSec: 2, frameRate: { numerator: 30, denominator: 1 },
});
const header = sealScreenOverlayHeader({ id: "screen" });
const semantic = semanticTrackFixture(space);
@@ -125,7 +124,7 @@ test("overlay Tracks interleave with peer Tracks only through absolute stacking"
test("the Screen Overlay Fragment publishes its Program and peer VisualTrack", () => {
const fragment = createScreenOverlayFragment([{ specName: "spec", windowName: "window" }]);
- assert.deepEqual(fragment.inputs.map((input) => input.name), ["canvas", "header", "semantic", "spec", "window"]);
+ assert.deepEqual(fragment.inputs.map((input) => input.name), ["canvas", "header", "space", "spec", "window"]);
assert.deepEqual(fragment.exports.map((output) => [output.name, output.type.name]), [
["program", "ScreenOverlayProgram"],
["track", "VisualTrack"],
diff --git a/packages/semantic-track/src/projection.ts b/packages/semantic-track/src/projection.ts
index 8258236a..86be22c8 100644
--- a/packages/semantic-track/src/projection.ts
+++ b/packages/semantic-track/src/projection.ts
@@ -35,7 +35,6 @@ export function projectSemanticProgramSpace(track: SemanticTrack): ProgramSpace
const frameRate = track.items[0]!.take.media.timeline.frameRate;
return sealProgramSpace({
id: track.id,
- narrativeId: track.narrativeId,
durationSec: semanticTrackFrameCount(track) * frameRate.denominator / frameRate.numerator,
frameRate,
});
diff --git a/packages/spatial/README.md b/packages/spatial/README.md
index 3786f8bd..81b6bad5 100644
--- a/packages/spatial/README.md
+++ b/packages/spatial/README.md
@@ -40,7 +40,8 @@ position unchanged. Fitting calculates rectangles; the consuming component owns
`content-x`, `content-y`, `fit-offset-x`, `fit-offset-y`, and `fit-constraint`. Defaults are `contain`,
center points (`0.5`), zero pixel offsets and `bounded`. These alignment fractions place the source
inside the supplied destination. Canvas placement remains the job of Frame / AnchoredFrame /
-AspectFrame. Media Track and Speech Track use this same decoder.
+AspectFrame. Media Track uses this decoder for Items, Performances and Sequence members; project
+components can use the same fitting functions where that presentation fits their role.
## Measured regions
diff --git a/packages/speech-alignment/src/local.ts b/packages/speech-alignment/src/local.ts
index 7ad62e3e..f88ea689 100644
--- a/packages/speech-alignment/src/local.ts
+++ b/packages/speech-alignment/src/local.ts
@@ -46,7 +46,6 @@ export function alignSemanticTake(
const localNarrative = segmentNarrative(narrative, excerpt);
const programSpace = sealProgramSpace({
id: `${excerpt.narrativeId}.segment.${excerpt.id}`,
- narrativeId: excerpt.narrativeId,
durationSec: media.timeline.frameCount * media.timeline.frameRate.denominator
/ media.timeline.frameRate.numerator,
frameRate: media.timeline.frameRate,
diff --git a/packages/speech-alignment/test/speech-align.test.ts b/packages/speech-alignment/test/speech-align.test.ts
index 2aa321f3..50352c62 100644
--- a/packages/speech-alignment/test/speech-align.test.ts
+++ b/packages/speech-alignment/test/speech-align.test.ts
@@ -81,8 +81,7 @@ function speechBasis(
durationSec = 2,
): AlignmentBasis {
if (narrative.segments.length !== 1) throw new Error("Test alignment requires one Segment.");
- const programSpace = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec,
+ const programSpace = sealProgramSpace({ id: "test-space", durationSec,
frameRate: { numerator: 1_000, denominator: 1 },
});
const audioDigest = fixtureResource(`fixture:audio:${narrative.segments.map((segment) => segment.id).join("+")}:${durationSec}`);
@@ -327,8 +326,7 @@ test("a collapsed WhisperX word is assigned the available interval between its n
test("three Script words may share the two video frames covered by one evidence word", () => {
const narrative = parseScript("pigeonhole.svml", "<line>alpha beta gamma</line>");
const original = speechBasis(narrative, 1);
- const programSpace = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 1,
+ const programSpace = sealProgramSpace({ id: "test-space", durationSec: 1,
frameRate: { numerator: 32, denominator: 1 },
});
const basis: AlignmentBasis = {
@@ -354,8 +352,7 @@ test("Evidence is interpreted only through the explicitly connected alignment cl
basis,
words: [{ text: "Hello", startSec: 0.1, endSec: 0.4 }, { text: "world", startSec: 0.5, endSec: 0.9 }],
});
- const anotherSpace = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 2,
+ const anotherSpace = sealProgramSpace({ id: "test-space", durationSec: 2,
frameRate: { numerator: 30, denominator: 1 },
});
const anotherBasis: AlignmentBasis = {
@@ -370,8 +367,7 @@ test("Evidence is interpreted only through the explicitly connected alignment cl
test("the final map is quantized once into the selected ProgramSpace", () => {
const narrative = parseScript("frames.svml", "<line>Hello.</line>");
const original = speechBasis(narrative, 1);
- const programSpace = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 1,
+ const programSpace = sealProgramSpace({ id: "test-space", durationSec: 1,
frameRate: { numerator: 30, denominator: 1 },
});
const basis: AlignmentBasis = {
diff --git a/packages/speech/test/evidence.test.ts b/packages/speech/test/evidence.test.ts
index c0ad5c60..8ac3dc74 100644
--- a/packages/speech/test/evidence.test.ts
+++ b/packages/speech/test/evidence.test.ts
@@ -11,8 +11,7 @@ test("speech evidence uses integer rational boundary projection rather than floa
assert.equal(speechEvidenceSampleBoundary(480_000), 160_000);
assert.equal(speechEvidenceSampleBoundary(480_001), 160_000);
assert.equal(speechEvidenceSampleBoundary(480_002), 160_001);
- const ntsc = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 1.001,
+ const ntsc = sealProgramSpace({ id: "test-space", durationSec: 1.001,
frameRate: { numerator: 30_000, denominator: 1_001 },
});
assert.equal(programSpaceSampleFrames(ntsc, 48_000), 48_048);
diff --git a/packages/studio-adapter/README.md b/packages/studio-adapter/README.md
index 0beeb826..8d47bcad 100644
--- a/packages/studio-adapter/README.md
+++ b/packages/studio-adapter/README.md
@@ -137,8 +137,10 @@ official Companion. The Source closure selects project packages; Studio does not
scan `node_modules` for plugins or use Runtime Profiles to select Companions.
The same host facet can contribute Film and Script boundary companions. A Film companion declares
-the references that select one semantic axis and its terminal Tracks. A Script companion owns raw
-source observation and marker adjustment. Studio core only matches and invokes these declarations;
+the accepted `timeSources` (author attribute and Type) and its terminal Tracks. Film accepts either
+a SemanticTrack or a declared ProgramSpace. Track Companion context includes `semantic` when the
+time source supplies a semantic timeline; pure animation leaves it undefined. A Script companion owns
+raw source observation and marker adjustment. Studio core only matches and invokes these declarations;
it does not import either domain package.
For a domain item whose Spec is consumed beside a Window or Instant, call
diff --git a/packages/studio-adapter/src/index.ts b/packages/studio-adapter/src/index.ts
index 7ffbc1d0..024a6685 100644
--- a/packages/studio-adapter/src/index.ts
+++ b/packages/studio-adapter/src/index.ts
@@ -37,7 +37,7 @@ export type StudioTimelinePresentation = {
export type StudioTemporalSource = {
readonly spaceId: string;
- readonly narrativeId: string;
+ readonly narrativeId?: string;
readonly kind: "program" | "selection" | "segment" | "moment";
readonly id: string;
};
@@ -504,7 +504,7 @@ export type StudioTrackCompanionContext = {
readonly values: ReadonlyMap<string, unknown>;
/** Temporal values in this Track's actual executed dependency closure. */
readonly temporalBindings: readonly StudioTemporalBinding[];
- readonly semantic: StudioSemanticTimeline;
+ readonly semantic: StudioSemanticTimeline | undefined;
readonly generic: () => readonly StudioEntityDraft[];
};
@@ -550,7 +550,7 @@ export type StudioFilmCompanion = {
readonly surface: string;
readonly outputType: TypeRef;
};
- readonly semantic: { readonly attribute: string; readonly type: TypeRef };
+ readonly timeSources: readonly { readonly attribute: string; readonly type: TypeRef }[];
readonly tracks: {
readonly childSurface: string;
readonly sourceAttribute: string;
diff --git a/packages/studio/README.md b/packages/studio/README.md
index ebec05f9..f99e5857 100644
--- a/packages/studio/README.md
+++ b/packages/studio/README.md
@@ -28,9 +28,11 @@ transient processing require the selected Runtime.
## Opening and editing a session
-The selected targets must reach one Film and its semantic projection. Studio rejects
-several distinct Films in one view; use separate Runs/sessions for those. The semantic
-axis can include ordinary wordless Script Segments with media spans.
+The selected targets must reach one Film and its resolved time source. Studio rejects several
+distinct Films in one view; use separate Runs/sessions for those. A SemanticTrack supplies the
+Script lane and performance timing, including wordless Segments with media spans. A declared
+ProgramSpace supplies authored animation time without a Script lane. Both show their component
+tracks and use the same rendering and parameter-editing machinery.
Open the URL printed by Vite. Studio requests port 5179 by default, accepts `--port`,
and Vite can choose another available port when it is occupied. Reuse that process
diff --git a/packages/studio/src/parameters.ts b/packages/studio/src/parameters.ts
index 15373e31..174ed6c5 100644
--- a/packages/studio/src/parameters.ts
+++ b/packages/studio/src/parameters.ts
@@ -449,7 +449,7 @@ export function resolveTimelineEditHandles(
? [endpoint.authority.source]
: []);
const first = sources[0];
- if (first === undefined
+ if (first === undefined || first.narrativeId === undefined
|| first.spaceId !== semantic?.spaceId
|| first.narrativeId !== semantic.narrativeId
|| !sources.every((candidate) => candidate.kind === first.kind
diff --git a/packages/studio/src/programme.ts b/packages/studio/src/programme.ts
index e338d969..e14e9c33 100644
--- a/packages/studio/src/programme.ts
+++ b/packages/studio/src/programme.ts
@@ -10,6 +10,7 @@ import {
} from "@hypit/semantic-track";
import type { SemanticTrack } from "@hypit/semantic-track";
import { semanticTrackTypes } from "@hypit/semantic-track";
+import { assertProgramSpaceIdentity, programSpaceTypes } from "@hypit/program-space";
import type { ProgramSpace } from "@hypit/program-space";
import type { StudioResolvedTrack, StudioTemporalBinding } from "@hypit/studio-adapter";
import type { RuntimeHostTransientExecution } from "@hypit/runtime-host-node";
@@ -35,7 +36,8 @@ export type Preview = {
readonly values: ReadonlyMap<string, unknown>;
readonly temporalBindings: ReadonlyMap<string, readonly StudioTemporalBinding[]>;
readonly composition: Composition;
- readonly timing: "measured";
+ readonly timing: "measured" | "authored";
+ readonly narrativeId?: string;
readonly timingOutput?: { readonly name: string; readonly ref: string };
readonly timingCandidateId?: string;
readonly timingCandidateOrigin: "run" | "source" | "none";
@@ -75,7 +77,7 @@ function selectedValue(
output: string,
): StoredValue | undefined {
const selection = state.plan.outputBindings.find((item) => item.output === output);
- if (selection === undefined) return undefined;
+ if (selection === undefined) return state.program.records.find((item) => item.id === output)?.value;
const executed = state.records.find((item) => item.id === selection.record)?.value;
if (executed !== undefined) return executed;
// Inline Source and Run candidates both belong to this freshly recompiled
@@ -109,6 +111,7 @@ export async function preview(input: {
readonly domain: StudioDomain;
readonly outputRefs: readonly string[];
readonly compositionRef: string;
+ readonly timeRef: string;
readonly projections: readonly StudioViewRequirement[];
/** Runtime-owned execution for capabilities explicitly safe outside a Build. */
readonly transientExecution?: RuntimeHostTransientExecution;
@@ -165,17 +168,17 @@ export async function preview(input: {
const satisfactions = new Map(
input.run.run.graph.satisfactions.map((item) => [item.output, item.candidate]),
);
- const timingOutput = targets.find((target) => sameType(target.typeRef, semanticTrackTypes.track));
- const timingValue = timingOutput === undefined
- ? undefined
- : selectedValue(executed.state, timingOutput.ref);
- if (timingOutput === undefined || timingValue?.kind !== "inline") {
- throw new Error("Studio requires a resolved SemanticTrack projection.");
- }
- const semantic = timingValue.value as unknown as SemanticTrack;
- const spans = semanticTrackSpans(semantic);
- const anchors = semanticAnchorFrames(semantic);
- const space = projectSemanticProgramSpace(semantic);
+ const timingOutput = input.source.exports.find((target) => target.ref === input.timeRef);
+ const timingRecord = executed.state.program.records.find((record) => record.id === input.timeRef);
+ const timingType = timingOutput?.typeRef ?? timingRecord?.type;
+ const timingValue = selectedValue(executed.state, input.timeRef);
+ if (timingValue?.kind !== "inline" || timingType === undefined) throw new Error("Studio requires a resolved film time source.");
+ const semantic = sameType(timingType, semanticTrackTypes.track) ? timingValue.value as unknown as SemanticTrack : undefined;
+ if (semantic === undefined && !sameType(timingType, programSpaceTypes.programSpace)) throw new Error("Unsupported Studio time source.");
+ const spans = semantic === undefined ? [] : semanticTrackSpans(semantic);
+ const anchors = semantic === undefined ? new Map<string, number>() : semanticAnchorFrames(semantic);
+ const space = semantic === undefined ? timingValue.value as unknown as ProgramSpace : projectSemanticProgramSpace(semantic);
+ assertProgramSpaceIdentity(space);
const rate = space.frameRate;
const compositionValue = selectedValue(executed.state, input.compositionRef);
if (compositionValue?.kind !== "inline") {
@@ -247,15 +250,16 @@ export async function preview(input: {
values.set(record.id, record.value.value);
}
}
- const timingCandidateId = satisfactions.get(timingOutput.ref);
+ const timingCandidateId = satisfactions.get(input.timeRef);
return {
source: input.source,
tracks,
values,
temporalBindings,
composition,
- timing: "measured",
- timingOutput: { name: timingOutput.name, ref: timingOutput.ref },
+ timing: semantic === undefined ? "authored" : "measured",
+ ...(semantic === undefined ? {} : { narrativeId: semantic.narrativeId }),
+ timingOutput: { name: timingOutput?.name ?? space.id, ref: input.timeRef },
...(timingCandidateId === undefined ? {} : { timingCandidateId }),
timingCandidateOrigin: timingCandidateId === undefined ? "source" : "run",
served,
diff --git a/packages/studio/src/server.ts b/packages/studio/src/server.ts
index 2b67ab58..b22b43eb 100644
--- a/packages/studio/src/server.ts
+++ b/packages/studio/src/server.ts
@@ -293,7 +293,7 @@ export function studioPlugin(options: StudioPluginOptions): Plugin {
}
const current = snapshot;
const script = current?.script;
- if (current === undefined || script === undefined) throw new Error("Studio has no writable Script source map.");
+ if (current === undefined || script === undefined || current.semantic === undefined) throw new Error("Studio has no writable Script source map.");
if (handle.semantic.narrativeId !== current.semantic.narrativeId
|| script.narrativeId !== current.semantic.narrativeId) {
throw new Error("The timeline entity and writable Script do not belong to the selected Narrative.");
@@ -342,11 +342,11 @@ export function studioPlugin(options: StudioPluginOptions): Plugin {
const anchorId = endpoint.authority.boundary === "start"
? semanticTarget.startAnchorId
: semanticTarget.endAnchorId;
- return snapshot?.semantic.anchors.find((anchor) => anchor.id === anchorId)?.frame;
+ return snapshot?.semantic?.anchors.find((anchor) => anchor.id === anchorId)?.frame;
}
if (endpoint.source.kind === "moment" && semanticTarget.kind === "moment"
&& endpoint.source.id === handle.semantic.id) {
- return snapshot?.semantic.anchors.find((anchor) => anchor.id === semanticTarget.anchorId)?.frame;
+ return snapshot?.semantic?.anchors.find((anchor) => anchor.id === semanticTarget.anchorId)?.frame;
}
return undefined;
};
@@ -357,14 +357,14 @@ export function studioPlugin(options: StudioPluginOptions): Plugin {
const id = endpoint.source.id;
if (id === undefined) return undefined;
if (endpoint.reference === "selection.start" || endpoint.reference === "selection.end") {
- const selection = snapshot?.semantic.selections.find((candidate) => candidate.id === id);
+ const selection = snapshot?.semantic?.selections.find((candidate) => candidate.id === id);
return endpoint.reference === "selection.start" ? selection?.startFrame : selection?.endFrameExclusive;
}
if (endpoint.reference === "segment.start" || endpoint.reference === "segment.end") {
- const segment = snapshot?.semantic.segments.find((candidate) => candidate.id === id);
+ const segment = snapshot?.semantic?.segments.find((candidate) => candidate.id === id);
return endpoint.reference === "segment.start" ? segment?.startFrame : segment?.endFrameExclusive;
}
- return snapshot?.semantic.moments.find((candidate) => candidate.id === id)?.frame;
+ return snapshot?.semantic?.moments.find((candidate) => candidate.id === id)?.frame;
};
const projectedPointValue = (endpoint: StudioTemporalInstantProjection, desired: number): string => {
if (endpoint.reference === "absolute") return frame(desired);
diff --git a/packages/studio/src/session.ts b/packages/studio/src/session.ts
index ae30ee2a..564778f7 100644
--- a/packages/studio/src/session.ts
+++ b/packages/studio/src/session.ts
@@ -73,6 +73,7 @@ export async function readStudioSession(input: {
domain: input.domain,
outputRefs,
compositionRef: inspection.filmComposition,
+ timeRef: inspection.timeRef,
projections: inspection.projections,
...(input.transientExecution === undefined ? {} : { transientExecution: input.transientExecution }),
});
diff --git a/packages/studio/src/shared.ts b/packages/studio/src/shared.ts
index 4df1c9df..d3c8f3d8 100644
--- a/packages/studio/src/shared.ts
+++ b/packages/studio/src/shared.ts
@@ -263,18 +263,13 @@ export type StudioSnapshot = {
* The special Studio lane. This is not another VisualTrack: it is the
* Narrative's segment/word geometry projected onto the same frame domain.
*/
- readonly semantic: SemanticTimeline;
- /**
- * The picture, ready to mount. Real material is the base layer rather than the
- * whole picture: the Tracks above it have not been rendered into that file, so
- * they are composited over it exactly as they are in `hyperframes` mode.
- */
+ readonly semantic?: SemanticTimeline;
/** The Tracks, compiled into the document the renderer photographs. */
readonly preview: { readonly kind: "hyperframes"; readonly srcdoc: string };
readonly provenance: {
/** Where the timeline came from. The frame domain always shares its source. */
- readonly timing: "measured";
- readonly picture: "measured";
+ readonly timing: "measured" | "authored";
+ readonly picture: "resolved";
/** What the badges above are standing for, in one sentence. */
readonly note: string;
};
diff --git a/packages/studio/src/snapshot.ts b/packages/studio/src/snapshot.ts
index d74969aa..c23c3f90 100644
--- a/packages/studio/src/snapshot.ts
+++ b/packages/studio/src/snapshot.ts
@@ -180,13 +180,14 @@ function semanticTimeline(
registry: StudioCompanionRegistry,
built: Preview,
script: ScriptMap | undefined,
-): SemanticTimeline {
+): SemanticTimeline | undefined {
+ if (built.narrativeId === undefined) return undefined;
const records = built.source.compiled.program.records.filter((item) =>
sameType(item.type, narrativeTypes.narrative)
&& item.value.kind === "inline"
- && (item.value.value as NarrativeValue).id === built.space.narrativeId);
+ && (item.value.value as NarrativeValue).id === built.narrativeId);
if (records.length > 1) {
- throw new Error(`Studio Narrative id ${built.space.narrativeId} resolves to more than one authored value.`);
+ throw new Error(`Studio Narrative id ${built.narrativeId} resolves to more than one authored value.`);
}
const record = records[0];
const narrative = record?.value.kind === "inline"
@@ -260,7 +261,7 @@ function semanticTimeline(
};
return {
spaceId: built.space.id,
- narrativeId: built.space.narrativeId,
+ narrativeId: built.narrativeId,
// Semantic is a Studio lane with its own registered meaning. Do not copy
// the authored Speech Track id into this label: it is the timebase, not a
// second Speech output.
@@ -302,7 +303,7 @@ export function snapshot(registry: StudioCompanionRegistry, built: Preview, inpu
readonly surfaces: MarkupSurfaceRegistryLike;
}): StudioSnapshot {
const located = authored(built.source.observations.placements);
- const script = scriptMap(built.source.observations.sourceMaps, built, built.space.narrativeId);
+ const script = built.narrativeId === undefined ? undefined : scriptMap(built.source.observations.sourceMaps, built, built.narrativeId);
const semantic = semanticTimeline(registry, built, script);
const tracks: Track[] = [];
for (const item of built.tracks) {
@@ -438,16 +439,16 @@ export function snapshot(registry: StudioCompanionRegistry, built: Preview, inpu
durationSec: frameCount * input.frameRate.denominator / input.frameRate.numerator,
},
tracks: rows,
- semantic,
+ ...(semantic === undefined ? {} : { semantic }),
preview: input.preview,
provenance: {
- timing: "measured",
- picture: "measured",
+ timing: built.timing,
+ picture: "resolved",
note: note(built),
},
};
}
function note(_built: Preview): string {
- return "Timings and material are the Candidates selected by the Run Source.";
+ return "Composition and timing are resolved from the selected Run Source.";
}
diff --git a/packages/studio/src/studio-preflight.ts b/packages/studio/src/studio-preflight.ts
index 14601db6..bef6a443 100644
--- a/packages/studio/src/studio-preflight.ts
+++ b/packages/studio/src/studio-preflight.ts
@@ -9,6 +9,7 @@ import type { RunPlan } from "./run.js";
import type { StudioCompanionRegistry } from "./studio-registry.js";
import {
outputFor,
+ referencedValueFor,
placementFor,
roleFor,
traceFor,
@@ -30,6 +31,7 @@ export type StudioViewRequirement = {
export type StudioInspection = {
readonly renderTargets: readonly string[];
readonly filmComposition: string;
+ readonly timeRef: string;
readonly projections: readonly StudioViewRequirement[];
};
@@ -118,7 +120,7 @@ export function inspectStudioRun(
}
}
- // The selected Film companion alone declares membership and semantic-axis vocabulary.
+ // The selected Film companion declares membership and time-source vocabulary.
const filmTrackRefs = unique(chosen.placement.children.flatMap((child) => {
if (localName(child.tag) !== film.tracks.childSurface) return [];
const source = child.resolvedReferenceAttributes?.[film.tracks.sourceAttribute];
@@ -129,11 +131,13 @@ export function inspectStudioRun(
? [output.ref]
: [];
});
- const semanticOutput = outputFor(source,
- chosen.placement.resolvedReferenceAttributes?.[film.semantic.attribute] ?? "");
- if (semanticOutput === undefined || !sameType(semanticOutput.typeRef, film.semantic.type)) {
- issues.push("Film has no traceable SemanticTrack reference");
- }
+ const times = film.timeSources.flatMap((sourceType) => {
+ const value = referencedValueFor(source, chosen.placement!.resolvedReferenceAttributes?.[sourceType.attribute] ?? "");
+ return value !== undefined && sameType(value.typeRef, sourceType.type) ? [value] : [];
+ });
+ const timeOutput = times[0];
+ if (times.length !== 1) issues.push("Film requires one traceable time source.");
+ const semanticOutput = timeOutput !== undefined && sameType(timeOutput.typeRef, semanticTrackTypes.track) ? timeOutput : undefined;
let companionValueRefs: readonly string[] = [];
try {
companionValueRefs = unique(filmTrackRefs.flatMap((ref) =>
@@ -188,6 +192,7 @@ export function inspectStudioRun(
return {
renderTargets: targets.map((item) => item.ref),
filmComposition: chosen.compositionRef,
+ timeRef: timeOutput!.ref,
projections,
};
}
diff --git a/packages/studio/src/studio-trace.ts b/packages/studio/src/studio-trace.ts
index d729a36e..1b566bc3 100644
--- a/packages/studio/src/studio-trace.ts
+++ b/packages/studio/src/studio-trace.ts
@@ -30,7 +30,7 @@ export function outputFor(source: CompiledSource, ref: string): StudioOutput | u
return source.exports.find((item) => item.ref === ref);
}
-function referencedValueFor(source: CompiledSource, ref: string): StudioOutput | undefined {
+export function referencedValueFor(source: CompiledSource, ref: string): StudioOutput | undefined {
const output = outputFor(source, ref);
if (output !== undefined) return output;
const record = source.compiled.program.records.find((item) => item.id === ref);
diff --git a/packages/studio/src/temporal-graph.ts b/packages/studio/src/temporal-graph.ts
index fd6c6e19..c5965386 100644
--- a/packages/studio/src/temporal-graph.ts
+++ b/packages/studio/src/temporal-graph.ts
@@ -46,7 +46,7 @@ function signed(value: unknown): string {
function expression(value: unknown): string {
const point = value as PointExpression;
- if (point.ref === "absolute") return duration(point.at);
+ if (point.ref === "absolute") return `${duration(point.at)}${point.offset === undefined ? "" : signed(point.offset)}`;
if (typeof point.ref !== "string") return "?";
return point.offset === undefined ? point.ref : `${point.ref}${signed(point.offset)}`;
}
@@ -60,12 +60,12 @@ function source(value: unknown): StudioTemporalSource | undefined {
} | undefined;
if (held?.kind !== "program" && held?.kind !== "selection"
&& held?.kind !== "segment" && held?.kind !== "moment") return undefined;
- if (typeof held.spaceId !== "string" || typeof held.narrativeId !== "string" || typeof held.id !== "string") {
+ if (typeof held.spaceId !== "string" || typeof held.id !== "string") {
return undefined;
}
return {
spaceId: held.spaceId,
- narrativeId: held.narrativeId,
+ ...(typeof held.narrativeId === "string" ? { narrativeId: held.narrativeId } : {}),
kind: held.kind,
id: held.id,
};
diff --git a/packages/studio/src/ui/main.ts b/packages/studio/src/ui/main.ts
index 59361efe..6a77ec44 100644
--- a/packages/studio/src/ui/main.ts
+++ b/packages/studio/src/ui/main.ts
@@ -775,7 +775,7 @@ function renderInspector(snapshot: StudioSnapshot, clipId: string | undefined):
}
function renderSemanticInspector(snapshot: StudioSnapshot, segmentId: string): void {
- const segment = snapshot.semantic.segments.find((item) => item.id === segmentId);
+ const segment = snapshot.semantic?.segments.find((item) => item.id === segmentId);
if (segment === undefined) { inspector.replaceChildren(); return; }
defaultWorkspaceHeading();
const empty = document.createElement("div");
@@ -785,7 +785,7 @@ function renderSemanticInspector(snapshot: StudioSnapshot, segmentId: string): v
}
function renderSemanticSelectionInspector(snapshot: StudioSnapshot, selectionId: string): void {
- const selection = snapshot.semantic.selections.find((item) => item.id === selectionId);
+ const selection = snapshot.semantic?.selections.find((item) => item.id === selectionId);
if (selection === undefined) { inspector.replaceChildren(); return; }
defaultWorkspaceHeading();
const empty = document.createElement("div");
@@ -795,7 +795,7 @@ function renderSemanticSelectionInspector(snapshot: StudioSnapshot, selectionId:
}
function renderSemanticMomentInspector(snapshot: StudioSnapshot, momentId: string): void {
- const moment = snapshot.semantic.moments.find((item) => item.id === momentId);
+ const moment = snapshot.semantic?.moments.find((item) => item.id === momentId);
if (moment === undefined) { inspector.replaceChildren(); return; }
defaultWorkspaceHeading();
const empty = document.createElement("div");
@@ -819,13 +819,13 @@ store.subscribe(({ snapshot, selection, playhead }) => {
const origin = selection.kind === "none" ? undefined : selection.origin;
const chosen = selection.kind === "clip" ? store.clip(selection.clipId) : undefined;
const chosenSegment = selection.kind === "semantic-segment"
- ? snapshot.semantic.segments.find((item) => item.id === selection.segmentId)
+ ? snapshot.semantic?.segments.find((item) => item.id === selection.segmentId)
: undefined;
const chosenSelection = selection.kind === "semantic-selection"
- ? snapshot.semantic.selections.find((item) => item.id === selection.selectionId)
+ ? snapshot.semantic?.selections.find((item) => item.id === selection.selectionId)
: undefined;
const chosenMoment = selection.kind === "semantic-moment"
- ? snapshot.semantic.moments.find((item) => item.id === selection.momentId)
+ ? snapshot.semantic?.moments.find((item) => item.id === selection.momentId)
: undefined;
// Rebuilding this every frame of playback would be DOM churn for no change.
const describes = `${snapshot.revision}:${selection.kind}:${chosen?.id ?? chosenSegment?.id ?? chosenSelection?.id ?? chosenMoment?.id ?? ""}`;
diff --git a/packages/studio/src/ui/selection.ts b/packages/studio/src/ui/selection.ts
index 3da75a52..e5a924eb 100644
--- a/packages/studio/src/ui/selection.ts
+++ b/packages/studio/src/ui/selection.ts
@@ -111,15 +111,15 @@ export function createStore(): Store {
selection = { kind: "none" };
}
if (held.kind === "semantic-segment"
- && !snapshot.semantic.segments.some((segment) => segment.id === held.segmentId)) {
+ && !snapshot.semantic?.segments.some((segment) => segment.id === held.segmentId)) {
selection = { kind: "none" };
}
if (held.kind === "semantic-selection"
- && !snapshot.semantic.selections.some((item) => item.id === held.selectionId)) {
+ && !snapshot.semantic?.selections.some((item) => item.id === held.selectionId)) {
selection = { kind: "none" };
}
if (held.kind === "semantic-moment"
- && !snapshot.semantic.moments.some((item) => item.id === held.momentId)) {
+ && !snapshot.semantic?.moments.some((item) => item.id === held.momentId)) {
selection = { kind: "none" };
}
playhead = { frame: clamp(playhead.frame), origin: playhead.origin };
@@ -132,17 +132,17 @@ export function createStore(): Store {
emit();
},
selectSemanticSegment(segmentId, origin) {
- if (snapshot?.semantic.segments.some((segment) => segment.id === segmentId) !== true) return;
+ if (snapshot?.semantic?.segments.some((segment) => segment.id === segmentId) !== true) return;
selection = { kind: "semantic-segment", segmentId, origin };
emit();
},
selectSemanticSelection(selectionId, origin) {
- if (snapshot?.semantic.selections.some((item) => item.id === selectionId) !== true) return;
+ if (snapshot?.semantic?.selections.some((item) => item.id === selectionId) !== true) return;
selection = { kind: "semantic-selection", selectionId, origin };
emit();
},
selectSemanticMoment(momentId, origin) {
- if (snapshot?.semantic.moments.some((item) => item.id === momentId) !== true) return;
+ if (snapshot?.semantic?.moments.some((item) => item.id === momentId) !== true) return;
selection = { kind: "semantic-moment", momentId, origin };
emit();
},
diff --git a/packages/studio/src/ui/timeline.ts b/packages/studio/src/ui/timeline.ts
index db253a9c..d516a149 100644
--- a/packages/studio/src/ui/timeline.ts
+++ b/packages/studio/src/ui/timeline.ts
@@ -189,7 +189,7 @@ export function createTimeline(store: Store): Timeline {
const next = Math.abs(candidate - frame);
if (next < distance) { nearest = candidate; distance = next; }
};
- for (const anchor of state.snapshot.semantic.anchors) consider(anchor.frame);
+ for (const anchor of state.snapshot.semantic?.anchors ?? []) consider(anchor.frame);
for (const clip of state.snapshot.tracks.flatMap((track) => track.clips)) {
consider(clip.startFrame);
consider(clip.endFrameExclusive);
@@ -234,7 +234,7 @@ export function createTimeline(store: Store): Timeline {
preferredIndex?: number,
): number | undefined => {
if (state === undefined) return undefined;
- const anchors = state.snapshot.semantic.anchors;
+ const anchors = state.snapshot.semantic?.anchors ?? [];
const preferredKind = preferredIndex === undefined ? undefined : anchors[preferredIndex]?.kind;
return [...indices].sort((left, right) => {
const leftAnchor = anchors[left]!;
@@ -254,7 +254,7 @@ export function createTimeline(store: Store): Timeline {
| undefined => {
if (state === undefined || edit.handle.semantic === undefined) return undefined;
const semantic = edit.handle.semantic;
- const anchors = state.snapshot.semantic.anchors;
+ const anchors = state.snapshot.semantic?.anchors ?? [];
if (semantic.kind === "moment") {
const currentIndex = anchors.findIndex((anchor) => anchor.id === semantic.anchorId);
const anchorIndex = nearestAnchorIndex(
@@ -355,9 +355,9 @@ export function createTimeline(store: Store): Timeline {
if (state === undefined) return undefined;
const target = semanticTarget(edit, nextFrame);
if (target?.kind === "selection" && edit.handle.semantic?.kind === "selection") {
- const current = state.snapshot.semantic.selections.find((selection) => selection.id === edit.handle.semantic!.id);
- const nextStart = state.snapshot.semantic.anchors.find((anchor) => anchor.id === target.startAnchorId)?.frame;
- const nextEnd = state.snapshot.semantic.anchors.find((anchor) => anchor.id === target.endAnchorId)?.frame;
+ const current = state.snapshot.semantic?.selections.find((selection) => selection.id === edit.handle.semantic!.id);
+ const nextStart = state.snapshot.semantic?.anchors.find((anchor) => anchor.id === target.startAnchorId)?.frame;
+ const nextEnd = state.snapshot.semantic?.anchors.find((anchor) => anchor.id === target.endAnchorId)?.frame;
if (current === undefined || nextStart === undefined || nextEnd === undefined) return undefined;
const instantBoundary = edit.handle.temporal?.kind === "instant"
&& edit.handle.temporal.authority.kind === "semantic"
@@ -374,8 +374,8 @@ export function createTimeline(store: Store): Timeline {
};
}
if (target?.kind === "moment" && edit.handle.semantic?.kind === "moment") {
- const current = state.snapshot.semantic.moments.find((moment) => moment.id === edit.handle.semantic!.id);
- const next = state.snapshot.semantic.anchors.find((anchor) => anchor.id === target.anchorId)?.frame;
+ const current = state.snapshot.semantic?.moments.find((moment) => moment.id === edit.handle.semantic!.id);
+ const next = state.snapshot.semantic?.anchors.find((anchor) => anchor.id === target.anchorId)?.frame;
if (current === undefined || next === undefined) return undefined;
const delta = next - current.frame;
if (edit.handle.gesture === "trim-start") {
@@ -604,7 +604,7 @@ export function createTimeline(store: Store): Timeline {
.sort((left, right) => (left.binding.lane.order ?? 0) - (right.binding.lane.order ?? 0));
const buildSemanticLane = (snapshot: StudioSnapshot): void => {
- if (snapshot.semantic.segments.length === 0) {
+ if (snapshot.semantic === undefined || snapshot.semantic.segments.length === 0) {
return;
}
const presentation = snapshot.semantic.presentation;
@@ -897,7 +897,7 @@ export function createTimeline(store: Store): Timeline {
const pointerAnchorIndex = handle.semantic !== undefined
? nearestAnchorIndex(
pointerFrame,
- state?.snapshot.semantic.anchors.map((_, index) => index) ?? [],
+ state?.snapshot.semantic?.anchors.map((_, index) => index) ?? [],
)
: undefined;
activeEdit = {
diff --git a/packages/studio/start.ts b/packages/studio/start.ts
index e339a44a..8f928071 100644
--- a/packages/studio/start.ts
+++ b/packages/studio/start.ts
@@ -22,7 +22,7 @@ function usage(message?: string): never {
[--port <number>] [--workspace <directory>] [--package-root <directory>]
Studio opens one explicit Run Source, requires a Film/Render target and a
-resolved deterministic semantic projection, and writes only the selected file
+resolved deterministic composition, and writes only the selected file
inside that exact Run and Author Source closure.
`);
process.exit(1);
diff --git a/packages/studio/test/parameters.test.ts b/packages/studio/test/parameters.test.ts
index f8e9becb..658a3496 100644
--- a/packages/studio/test/parameters.test.ts
+++ b/packages/studio/test/parameters.test.ts
@@ -147,7 +147,7 @@ test("at/for and until/for derive complementary semantic and duration inverses",
]);
});
-test("absolute Window edits use only the Companion's exact parameter vocabulary", () => {
+test("absolute Window edits work without a semantic lane and use the Companion's parameter vocabulary", () => {
const source = (start: number, end: number) => ({
endpoint: `main::${start}`,
path: "main.svml",
@@ -170,18 +170,18 @@ test("absolute Window edits use only the Companion's exact parameter vocabulary"
projection: {
kind: "window",
start: {
- kind: "instant", expression: "1f", reference: "absolute", frame: 1, source: { ...temporalIdentity, kind: "program", id: "program" },
+ kind: "instant", expression: "1f", reference: "absolute", frame: 1, source: { spaceId: "animation", kind: "program", id: "program" },
authority: { kind: "parameter", binding: "from", relation: "direct" },
},
end: {
- kind: "instant", expression: "20f", reference: "absolute", frame: 20, source: { ...temporalIdentity, kind: "program", id: "program" },
+ kind: "instant", expression: "20f", reference: "absolute", frame: 20, source: { spaceId: "animation", kind: "program", id: "program" },
authority: { kind: "parameter", binding: "until", relation: "direct" },
},
startFrame: 1, endFrameExclusive: 20,
},
phases: [],
},
- semantic,
+ undefined,
);
assert.deepEqual(handles.map((handle) => ({
diff --git a/packages/studio/test/preflight.test.ts b/packages/studio/test/preflight.test.ts
index a8f53306..28b5d575 100644
--- a/packages/studio/test/preflight.test.ts
+++ b/packages/studio/test/preflight.test.ts
@@ -1,3 +1,4 @@
+import { programSpaceTypes } from "@hypit/program-space";
import assert from "node:assert/strict";
import test from "node:test";
@@ -14,7 +15,7 @@ const filmModule = { name: "@example/film", version: "1" } as const;
const companion: StudioFilmCompanion = {
id: "film",
match: { module: filmModule, surface: "film", outputType: compositionTypes.composition },
- semantic: { attribute: "semantic", type: semanticTrackTypes.track },
+ timeSources: [{ attribute: "semantic", type: semanticTrackTypes.track }, { attribute: "space", type: programSpaceTypes.programSpace }],
tracks: { childSurface: "Track", sourceAttribute: "source", types: [compositionTypes.visualTrack] },
};
diff --git a/packages/temporal-markup/README.md b/packages/temporal-markup/README.md
index 322760d0..68884cb5 100644
--- a/packages/temporal-markup/README.md
+++ b/packages/temporal-markup/README.md
@@ -5,10 +5,22 @@ Selection, Segment, Moment or explicit clock expressions to ordinary Instant pro
composition. A domain consumer receives the projected values and explicit ProgramSpace; it does not
locate Script words or infer a semantic source inside its renderer.
-The package exports `createTemporalWindowProjection`, `createTemporalInstantProjection`, their
-attribute vocabulary, and the exact duration/instant parsers. These are helpers for component
+The package exports `createTemporalWindowProjection`, `createTemporalInstantProjection`,
+`resolveTemporalContext`, `createTemporalSpace`, attribute vocabulary, and the exact duration/instant parsers. These are helpers for component
Surfaces, not standalone author tags or a new Track.
+## Time context
+
+A Track Surface can accept `semantic={speech.semantic}` or `space={animation}`. Resolve that choice
+with `resolveTemporalContext({ element, resolveReference })`, and pass the result to
+`createTemporalSpace({ id, element, ...context })`. Preserve its component and fragment drafts.
+Supply `...context, space: time.space` to each temporal projection helper, and pass `time.space.ref`
+to the domain Fragment. Its input Type is `programSpaceTypes.programSpace`.
+
+A semantic context locates Script references and supplies the performance's time axis. A declared
+space supplies a clock for authored animation. Literal times and program boundaries work in either
+context. A Script reference needs `semantic` to locate it; the renderer never invents that mapping.
+
## Window forms
A Surface using the shared Window vocabulary accepts one complete form:
@@ -18,6 +30,7 @@ A Surface using the shared Window vocabulary accepts one complete form:
| `during="program"` | The complete program Window. |
| `during={story.segment.opening}` | A Segment's Window. |
| `during={story.selection.proof}` | A Selection's Window, including its authored endpoint affinity. |
+| `at="2s" for="8f"` | Start two seconds into the program and last eight frames. |
| `at={story.moment.reveal} for="8f"` | Start at a Moment and last eight program frames. |
| `until={story.moment.reveal} for="250ms"` | End at a Moment after a span of 250 milliseconds. |
| `start="program.start" end="moment.cue" moment={story.moment.reveal}` | Compose independently authored endpoints. |
@@ -41,6 +54,7 @@ An Instant consumer has a different job: an event or activation with one tempora
| `at={story.moment.reveal}` | The authored Moment. |
| `at={story.selection.proof} boundary="start"` | An explicitly chosen Selection boundary. |
| `at={story.segment.opening} boundary="end"` | An explicitly chosen Segment boundary. |
+| `at="2.6s"` | An authored event 2.6 seconds from program start. |
| `instant="program.start + 8f"` | A projected clock expression. |
The domain component decides what happens after that point. An answer may remain visible, a Sequence
diff --git a/packages/temporal-markup/package.json b/packages/temporal-markup/package.json
index dc7a2bb4..c6fec5eb 100644
--- a/packages/temporal-markup/package.json
+++ b/packages/temporal-markup/package.json
@@ -12,6 +12,7 @@
"@hypit/markup": "workspace:*",
"@hypit/narrative": "workspace:*",
"@hypit/semantic-track": "workspace:*",
- "@hypit/temporal": "workspace:*"
+ "@hypit/temporal": "workspace:*",
+ "@hypit/program-space": "workspace:*"
}
}
diff --git a/packages/temporal-markup/src/index.ts b/packages/temporal-markup/src/index.ts
index b606e8f3..a1ccf22e 100644
--- a/packages/temporal-markup/src/index.ts
+++ b/packages/temporal-markup/src/index.ts
@@ -9,6 +9,9 @@ import type {
SurfaceResolvedReference,
} from "@hypit/markup";
import { narrativeTypes } from "@hypit/narrative";
+import { programSpaceTypes } from "@hypit/program-space";
+import { createTemporalSpace } from "./space.js";
+export * from "./space.js";
import { semanticTrackTypes } from "@hypit/semantic-track";
import {
temporalProducers,
@@ -28,10 +31,10 @@ export const temporalWindowAttributeVocabulary: readonly SurfaceAttributeVocabul
{ name: "during", kind: "expression", required: false, values: ["program"],
accepts: [narrativeTypes.selection, narrativeTypes.excerpt],
summary: "Uses the whole Program, or the exact window of a referenced Selection or Segment." },
- { name: "at", kind: "reference", required: false, accepts: [narrativeTypes.moment],
- summary: "Uses a semantic Moment as the start authority; write it together with for." },
- { name: "until", kind: "reference", required: false, accepts: [narrativeTypes.moment],
- summary: "Uses a semantic Moment as the end authority; write it together with for." },
+ { name: "at", kind: "expression", required: false, accepts: [narrativeTypes.moment],
+ summary: "Starts at a semantic Moment or authored time (2s, 60f); write it together with for." },
+ { name: "until", kind: "expression", required: false, accepts: [narrativeTypes.moment],
+ summary: "Ends at a semantic Moment or authored time; write it together with for." },
{ name: "for", kind: "literal", required: false,
summary: "Sets the projected duration paired with at or until, such as 12f, 250ms or 1.5s." },
{ name: "start", kind: "literal", required: false,
@@ -49,11 +52,11 @@ export const temporalWindowAttributeNames = temporalWindowAttributeVocabulary.ma
/** Complete author vocabulary for a semantic or explicitly projected Instant. */
export const temporalInstantAttributeVocabulary: readonly SurfaceAttributeVocabulary[] = [
- { name: "at", kind: "reference", required: false,
+ { name: "at", kind: "expression", required: false,
accepts: [narrativeTypes.moment, narrativeTypes.selection, narrativeTypes.excerpt],
- summary: "Uses a semantic Moment or a chosen Selection or Segment boundary as the Instant authority." },
+ summary: "Uses a semantic Moment, a chosen Selection or Segment boundary, or an authored time such as 2s." },
{ name: "instant", kind: "literal", required: false,
- summary: "Uses an explicitly projected point expression when no semantic point owns the timing." },
+ summary: "Uses a projected point expression, such as moment.cue + 12f or program.start + 2s." },
{ name: "boundary", kind: "literal", required: false, values: ["start", "end"],
summary: "Chooses the start or end boundary when at references a Selection or Segment." },
{ name: "selection", kind: "reference", required: false, accepts: [narrativeTypes.selection],
@@ -221,7 +224,8 @@ function projectionFragment(input: {
? [["instant", input.start] as const]
: [["start", input.start] as const, ["end", input.end] as const];
const inputs = [
- { name: "semantic", type: semanticTrackTypes.track },
+ { name: "space", type: programSpaceTypes.programSpace },
+ ...(endpoints.some(([, endpoint]) => endpoint.source !== "program") ? [{ name: "semantic", type: semanticTrackTypes.track }] : []),
...endpoints.map(([name]) => ({ name: `${name}-spec`, type: temporalTypes.instantSpec })),
...endpoints.flatMap(([name, endpoint]) => endpoint.source === "program" ? [] : [{
name: `${name}-${endpoint.source}`,
@@ -237,7 +241,7 @@ function projectionFragment(input: {
: endpoint.source === "segment" ? temporalProducers.projectSegmentInstant
: temporalProducers.projectMomentInstant,
inputs: {
- semantic: fragmentInput("semantic"),
+ ...(endpoint.source === "program" ? { space: fragmentInput("space") } : { semantic: fragmentInput("semantic") }),
spec: fragmentInput(`${name}-spec`),
...(endpoint.source === "program" ? {} : {
[endpoint.source]: fragmentInput(`${name}-${endpoint.source}`),
@@ -270,13 +274,15 @@ function projectionDraft(input: {
readonly id: string;
readonly subjectId?: string;
readonly element: StructuredElement;
- readonly semantic: SurfaceResolvedReference;
+ readonly semantic?: SurfaceResolvedReference;
+ readonly space?: SurfaceResolvedReference;
readonly start: InstantDraft;
readonly end?: InstantDraft;
}): TemporalMarkupProjection {
- if (!sameType(input.semantic.type, semanticTrackTypes.track)) {
- throw new Error(`${input.element.name}.semantic has the wrong Type.`);
+ if ([input.start, input.end].some((endpoint) => endpoint !== undefined && endpoint.source !== "program") && input.semantic === undefined) {
+ throw new Error(`${input.element.name} needs semantic to resolve this Script reference.`);
}
+ const time = createTemporalSpace(input);
const output = input.end === undefined ? "instant" : "window";
const subjectId = input.subjectId ?? input.id;
if (subjectId.length === 0) throw new Error("Temporal projection subjectId must not be empty.");
@@ -285,7 +291,8 @@ function projectionDraft(input: {
const records: SurfaceRecordDraft[] = [];
const componentInputs: SurfaceComponentDraft["inputs"] extends infer _T
? Record<string, SurfaceResolvedReference["ref"] | { readonly kind: "record"; readonly id: string }>
- : never = { semantic: input.semantic.ref };
+ : never = { space: time.space.ref, ...(input.semantic === undefined ? {} : { semantic: input.semantic.ref }) };
+ if ([input.start, input.end].every((endpoint) => endpoint === undefined || endpoint.source === "program")) delete componentInputs.semantic;
const endpoints = input.end === undefined
? [["instant", input.start] as const]
: [["start", input.start] as const, ["end", input.end] as const];
@@ -320,14 +327,14 @@ function projectionDraft(input: {
}
return {
records,
- components: [{
+ components: [...time.components, {
id: componentId,
fragment: fragment.id,
inputs: componentInputs,
outputs: { [output]: `${input.id}.__temporal.${output}` },
range: input.element.range,
}],
- fragments: [fragment],
+ fragments: [...time.fragments, fragment],
ref: { kind: "component-output", component: componentId, output },
};
}
@@ -337,7 +344,8 @@ export function createTemporalWindowProjection(input: {
readonly id: string;
readonly subjectId?: string;
readonly element: StructuredElement;
- readonly semantic: SurfaceResolvedReference;
+ readonly semantic?: SurfaceResolvedReference;
+ readonly space?: SurfaceResolvedReference;
readonly resolveReference: ResolveReference;
}): TemporalMarkupProjection {
const { element, resolveReference } = input;
@@ -382,23 +390,24 @@ export function createTemporalWindowProjection(input: {
if (at !== undefined || until !== undefined) {
const name = at !== undefined ? "at" : "until";
rejectUnusedTemporalAttributes(element, temporalWindowAttributeNames, [name, "for"]);
- const moment = resolve(at ?? until, `${element.name}.${name}`, narrativeTypes.moment, resolveReference);
+ const raw = at ?? until;
+ const moment = typeof raw === "string" ? undefined : resolve(raw, `${element.name}.${name}`, narrativeTypes.moment, resolveReference);
const length = parseTemporalDuration(text(element, "for"), `${element.name}.for`);
const cue: InstantDraft = {
- expression: { ref: "moment.cue" },
- authority: { kind: "semantic", boundary: "cue" },
- source: "moment",
- reference: moment,
+ expression: typeof raw === "string" ? { ref: "absolute", at: parseTemporalDuration(raw, `${element.name}.${name}`) } : { ref: "moment.cue" },
+ authority: moment === undefined ? { kind: "parameter", binding: name, relation: "direct" } : { kind: "semantic", boundary: "cue" },
+ source: moment === undefined ? "program" : "moment",
+ ...(moment === undefined ? {} : { reference: moment }),
};
const derived: InstantDraft = {
- expression: { ref: "moment.cue", offset: at !== undefined ? length : negate(length) },
+ expression: { ...cue.expression, offset: at !== undefined ? length : negate(length) },
authority: {
kind: "parameter",
binding: "for",
relation: at !== undefined ? "after-start" : "before-end",
},
- source: "moment",
- reference: moment,
+ source: cue.source,
+ ...(moment === undefined ? {} : { reference: moment }),
};
return projectionDraft({
...input,
@@ -437,12 +446,13 @@ export function createTemporalWindowProjection(input: {
});
}
-/** Decode exactly one semantic at={Moment} or projected instant="...". */
+/** Decode a semantic reference, an authored at time, or a projected instant expression. */
export function createTemporalInstantProjection(input: {
readonly id: string;
readonly subjectId?: string;
readonly element: StructuredElement;
- readonly semantic: SurfaceResolvedReference;
+ readonly semantic?: SurfaceResolvedReference;
+ readonly space?: SurfaceResolvedReference;
readonly resolveReference: ResolveReference;
readonly semanticAttribute?: string;
readonly boundaryAttribute?: string;
@@ -465,7 +475,11 @@ export function createTemporalInstantProjection(input: {
}
if (at !== undefined) {
if (typeof at === "string") {
- throw new Error(`${input.element.name}.${semanticAttribute} must be a semantic reference; use ${projectedAttribute || "instant"} for a projected point.`);
+ rejectUnusedTemporalAttributes(input.element, universe, [semanticAttribute]);
+ return projectionDraft({ ...input, start: {
+ expression: { ref: "absolute", at: parseTemporalDuration(at, `${input.element.name}.${semanticAttribute}`) },
+ authority: { kind: "parameter", binding: semanticAttribute, relation: "direct" }, source: "program",
+ } });
}
const found = input.resolveReference(at.path);
if (found === undefined) throw new Error(`${input.element.name}.${semanticAttribute} is unresolved.`);
diff --git a/packages/temporal-markup/src/space.ts b/packages/temporal-markup/src/space.ts
new file mode 100644
index 00000000..0f96e24a
--- /dev/null
+++ b/packages/temporal-markup/src/space.ts
@@ -0,0 +1,59 @@
+import { sealGraphFragment } from "@hypit/elaborator";
+import type { StructuredElement, SurfaceAttributeVocabulary, SurfaceResolvedReference } from "@hypit/markup";
+import { programSpaceTypes } from "@hypit/program-space";
+import { semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";
+
+export type TemporalContext = {
+ readonly semantic?: SurfaceResolvedReference;
+ readonly space?: SurfaceResolvedReference;
+};
+
+export const temporalContextAttributeVocabulary = [
+ { name: "semantic", kind: "reference", required: false, accepts: [semanticTrackTypes.track],
+ summary: "Uses the performance's time axis and resolves Script Selections and Moments." },
+ { name: "space", kind: "reference", required: false, accepts: [programSpaceTypes.programSpace],
+ summary: "Uses a declared film time axis for authored animation. Supply semantic or space." },
+] as const satisfies readonly SurfaceAttributeVocabulary[];
+
+/** Select the author's film time source. */
+export function resolveTemporalContext(input: {
+ readonly element: StructuredElement;
+ readonly resolveReference: (path: string) => SurfaceResolvedReference | undefined;
+}): TemporalContext {
+ const { element, resolveReference } = input;
+ if (Number(element.attributes.semantic !== undefined) + Number(element.attributes.space !== undefined) !== 1) {
+ throw new Error(`${element.name} requires one time source: semantic or space.`);
+ }
+ const name = element.attributes.semantic === undefined ? "space" : "semantic";
+ const raw = element.attributes[name];
+ if (typeof raw !== "object" || raw.kind !== "reference") throw new Error(`${element.name}.${name} must be a reference.`);
+ const found = resolveReference(raw.path);
+ const expected = name === "semantic" ? semanticTrackTypes.track : programSpaceTypes.programSpace;
+ if (found === undefined || found.type.module.name !== expected.module.name
+ || found.type.module.version !== expected.module.version || found.type.name !== expected.name) {
+ throw new Error(`${element.name}.${name} must reference ${expected.name}.`);
+ }
+ return { [name]: found };
+}
+
+/** Give consumers a ProgramSpace while preserving semantic context for word-bound projections. */
+export function createTemporalSpace(input: TemporalContext & { readonly id: string; readonly element: StructuredElement }) {
+ if (input.space !== undefined) return { records: [], components: [], fragments: [], space: input.space };
+ if (input.semantic === undefined) throw new Error(`${input.element.name} requires a film time source.`);
+ const fragment = sealGraphFragment({
+ inputs: [{ name: "semantic", type: semanticTrackTypes.track }],
+ operations: [{ id: "project-space", producer: semanticTrackProducers.projectProgramSpace,
+ inputs: { track: { kind: "fragment-input", name: "semantic" } }, result: { kind: "output", name: "space" } }],
+ exports: [{ name: "space", type: programSpaceTypes.programSpace,
+ root: { kind: "fragment-operation", operation: "project-space" } }],
+ });
+ const id = `${input.id}.__space`;
+ const space: SurfaceResolvedReference = { path: `${id}.space`, type: programSpaceTypes.programSpace,
+ ref: { kind: "component-output", component: id, output: "space" } };
+ return {
+ records: [],
+ components: [{ id, fragment: fragment.id, inputs: { semantic: input.semantic.ref }, outputs: { space: `${id}.space` }, range: input.element.range }],
+ fragments: [fragment],
+ space,
+ };
+}
diff --git a/packages/temporal-markup/test/temporal-markup.test.ts b/packages/temporal-markup/test/temporal-markup.test.ts
index 2ce514ef..16795d0e 100644
--- a/packages/temporal-markup/test/temporal-markup.test.ts
+++ b/packages/temporal-markup/test/temporal-markup.test.ts
@@ -3,16 +3,18 @@ import test from "node:test";
import type { MarkupAttributeValue, StructuredElement, SurfaceResolvedReference } from "@hypit/markup";
import { narrativeTypes } from "@hypit/narrative";
+import { programSpaceTypes } from "@hypit/program-space";
import { semanticTrackTypes } from "@hypit/semantic-track";
import type { TemporalInstantSpec } from "@hypit/temporal";
-import { createTemporalInstantProjection, createTemporalWindowProjection } from "../src/index.js";
+import { createTemporalInstantProjection, createTemporalWindowProjection, resolveTemporalContext } from "../src/index.js";
const range = { source: "main.svml", start: 0, end: 1 };
const reference = (path: string): MarkupAttributeValue => ({ kind: "reference", path });
const resolved = (path: string, type: SurfaceResolvedReference["type"]): SurfaceResolvedReference => ({
path, type, ref: { kind: "record", id: path },
});
+
const semantic = resolved("semantic", semanticTrackTypes.track);
const references = new Map([
["selection", resolved("selection", narrativeTypes.selection)],
@@ -66,5 +68,25 @@ test("the author bridge owns the four Window forms and the explicit Instant fall
{ kind: "parameter", binding: "instant", relation: "direct" });
assert.throws(() => createTemporalInstantProjection({
id: "ambiguous", element: element({ at: "moment.cue+3f" }), semantic, resolveReference,
- }), /semantic reference.*projected point/u);
+ }), /exact duration/u);
+});
+
+test("authored at/for projects on a declared clock while Script events keep their semantic dependency", () => {
+ const space = resolved("animation", programSpaceTypes.programSpace);
+ const at = createTemporalInstantProjection({ id: "message", element: element({ at: "2.5s" }), space, resolveReference });
+ const absolute = specs(at as ReturnType<typeof createTemporalWindowProjection>)[0]!;
+ assert.deepEqual(absolute.projection, { ref: "absolute", at: { unit: "seconds", numerator: 5, denominator: 2 } });
+ assert.deepEqual(absolute.authority, { kind: "parameter", binding: "at", relation: "direct" });
+ assert.equal(at.fragments.some(fragment => fragment.inputs.some(input => input.name === "semantic")), false);
+
+ const window = specs(createTemporalWindowProjection({
+ id: "bubble", element: element({ at: "2.5s", for: "8f" }), space, resolveReference,
+ }));
+ assert.deepEqual(window[1]?.projection, { ref: "absolute", at: { unit: "seconds", numerator: 5, denominator: 2 }, offset: { unit: "frames", value: 8 } });
+ const bound = createTemporalInstantProjection({ id: "message", element: element({ at: reference("moment") }), semantic, resolveReference });
+ assert.equal(bound.fragments.some(fragment => fragment.inputs.some(input => input.name === "semantic")), true);
+ assert.throws(() => createTemporalInstantProjection({ id: "message", element: element({ at: reference("moment") }), space, resolveReference }), /semantic/u);
+
+ const context = resolveTemporalContext({ element: element({ space: reference("animation") }), resolveReference: path => path === "animation" ? space : undefined });
+ assert.deepEqual(context, { space });
});
diff --git a/packages/temporal/README.md b/packages/temporal/README.md
index d8fcd25a..b2790316 100644
--- a/packages/temporal/README.md
+++ b/packages/temporal/README.md
@@ -6,7 +6,7 @@ The public values are `TemporalInstant` and `TemporalWindow`. An Instant retains
exact projection expression, resolved ProgramSpace frame and author authority. A Window is composed
from two independently traced Instants; it has no synthetic single source.
-Every source names its exact `ProgramSpace` and `Narrative`, and every projection retains the
+Every source names its exact `ProgramSpace`; semantic sources also name their `Narrative`. Every projection retains the
consumer's public domain identity as `subjectId`. These are public provenance fields, not generated hashes
or editor metadata. A graph-qualified projection `id` and its author-facing `subjectId` are separate
on purpose.
@@ -17,10 +17,15 @@ domain components receive only ProgramSpace plus the resulting Instant or Window
Selection, Segment or Moment themselves.
Every official consumer verifies the projection at its public boundary: `subjectId` must name the
-domain object being built, both endpoints must retain one ProgramSpace/Narrative, and that identity
+domain object being built, both endpoints must retain one ProgramSpace, and that identity
must equal the explicitly supplied ProgramSpace. ProgramSpace is therefore an ordinary graph input
to the consumer, not ambient renderer state.
Temporal rejects Instants outside ProgramSpace and Windows that are reversed or empty. It does not
clip or repair author time. The package also provides sibling-window validation and triggered-stage
scheduling, but no renderer, Provider, media policy or Studio behavior.
+
+Program-bound projection receives ProgramSpace directly. Semantic-bound projection receives the
+SemanticTrack that locates its Script anchors. Both produce the same Instant type; program-bound
+sources have no Narrative identity. A Window can combine a semantic endpoint with a program endpoint
+on the same time axis. `absolute` expressions may include a duration offset, as used by `at`/`for`.
diff --git a/packages/temporal/src/component.ts b/packages/temporal/src/component.ts
index 50405e67..404446e6 100644
--- a/packages/temporal/src/component.ts
+++ b/packages/temporal/src/component.ts
@@ -2,6 +2,7 @@ import type { ComponentPackage, ProducerHandlerContext } from "@hypit/component-
import { canonicalize } from "@hypit/protocol";
import type { StoredValue } from "@hypit/protocol";
import type { NarrativeExcerpt, NarrativeMomentRef, NarrativeSelectionRef } from "@hypit/narrative";
+import type { ProgramSpace } from "@hypit/program-space";
import type { SemanticTrack } from "@hypit/semantic-track";
import {
@@ -30,7 +31,7 @@ export const temporalComponent = {
{ producer: temporalProducers.projectProgramInstant, handler: ({ inputs }) => ({ outputs: { instant: output(projectProgramInstant({
itemId: instantSpec(inputs).id,
subjectId: instantSpec(inputs).subjectId,
- semantic: inline<SemanticTrack>(inputs.semantic?.value, "SemanticTrack"),
+ space: inline<ProgramSpace>(inputs.space?.value, "ProgramSpace"),
projection: instantSpec(inputs).projection,
authority: instantSpec(inputs).authority,
})) }, needs: {} }) },
diff --git a/packages/temporal/src/index.ts b/packages/temporal/src/index.ts
index 968c3b07..c1007296 100644
--- a/packages/temporal/src/index.ts
+++ b/packages/temporal/src/index.ts
@@ -1,7 +1,7 @@
import type { ModuleManifest, ProducerRef, TypeRef, ValueSchema } from "@hypit/protocol";
import { narrativeDependency, narrativeTypes } from "@hypit/narrative";
import { semanticTrackDependency, semanticTrackTypes } from "@hypit/semantic-track";
-import { programSpaceDependency } from "@hypit/program-space";
+import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";
const string = { kind: "string", minLength: 1 } as const;
const unsignedInteger = { kind: "number", integer: true, minimum: 0 } as const;
@@ -16,11 +16,11 @@ const instant: ValueSchema = { kind: "oneOf", variants: [
...["program.start", "program.end", "selection.start", "selection.end", "segment.start", "segment.end", "moment.cue"].map((ref) => object({
ref: { schema: { kind: "literal", value: ref } }, offset: { schema: duration, optional: true },
})),
- object({ ref: { schema: { kind: "literal", value: "absolute" } }, at: { schema: duration } }),
+ object({ ref: { schema: { kind: "literal", value: "absolute" } }, at: { schema: duration }, offset: { schema: duration, optional: true } }),
] };
const sourceSchema: ValueSchema = { kind: "oneOf", variants: [
...["program", "selection", "segment", "moment"].map((kind) => object({
- spaceId: { schema: string }, narrativeId: { schema: string },
+ spaceId: { schema: string }, narrativeId: { schema: string, optional: true },
kind: { schema: { kind: "literal", value: kind } }, id: { schema: string },
})),
] };
@@ -75,7 +75,7 @@ export const temporalManifest: ModuleManifest = {
capabilities: [],
producers: [
{ name: temporalProducers.projectProgramInstant.name,
- inputs: [{ name: "semantic", type: semanticTrackTypes.track }, { name: "spec", type: temporalTypes.instantSpec }],
+ inputs: [{ name: "space", type: programSpaceTypes.programSpace }, { name: "spec", type: temporalTypes.instantSpec }],
outputs: [{ name: "instant", type: temporalTypes.instant }], needs: [] },
{ name: temporalProducers.projectSelectionInstant.name,
inputs: [{ name: "semantic", type: semanticTrackTypes.track }, { name: "selection", type: narrativeTypes.selection }, { name: "spec", type: temporalTypes.instantSpec }],
diff --git a/packages/temporal/src/projection.ts b/packages/temporal/src/projection.ts
index 32746ba1..5619328b 100644
--- a/packages/temporal/src/projection.ts
+++ b/packages/temporal/src/projection.ts
@@ -36,7 +36,7 @@ export function assertTemporalInstantFor(
if (!expected.subjectId || instant.subjectId !== expected.subjectId) {
throw new Error(`Temporal Instant subject ${instant.subjectId} does not match ${expected.subjectId}.`);
}
- if (!instant.source.spaceId || !instant.source.narrativeId) {
+ if (!instant.source.spaceId) {
throw new Error("Temporal Instant has no ProgramSpace identity.");
}
if (!Number.isSafeInteger(instant.frame) || instant.frame < 0) {
@@ -44,9 +44,8 @@ export function assertTemporalInstantFor(
}
if (expected.space !== undefined) {
assertProgramSpaceIdentity(expected.space);
- if (instant.source.spaceId !== expected.space.id
- || instant.source.narrativeId !== expected.space.narrativeId) {
- throw new Error("Temporal Instant belongs to a different ProgramSpace or Narrative.");
+ if (instant.source.spaceId !== expected.space.id) {
+ throw new Error("Temporal Instant belongs to a different ProgramSpace.");
}
if (instant.frame > programSpaceFrameCount(expected.space)) {
throw new Error("Temporal Instant falls outside its ProgramSpace.");
@@ -66,9 +65,8 @@ export function assertTemporalWindowFor(
}
assertTemporalInstantFor(window.start, expected);
assertTemporalInstantFor(window.end, expected);
- if (window.start.source.spaceId !== window.end.source.spaceId
- || window.start.source.narrativeId !== window.end.source.narrativeId) {
- throw new Error("Temporal Window endpoints belong to different ProgramSpaces or Narratives.");
+ if (window.start.source.spaceId !== window.end.source.spaceId) {
+ throw new Error("Temporal Window endpoints belong to different ProgramSpaces.");
}
if (window.span.startFrame !== window.start.frame
|| window.span.endFrameExclusive !== window.end.frame
@@ -89,7 +87,10 @@ function evaluateInstant(
environment: InstantEnvironment,
space: ProgramSpace,
): Rational {
- if (expression.ref === "absolute") return durationInFrames(expression.at, space);
+ if (expression.ref === "absolute") {
+ const at = durationInFrames(expression.at, space);
+ return expression.offset === undefined ? at : add(at, durationInFrames(expression.offset, space));
+ }
let base: number;
switch (expression.ref) {
case "program.start": base = environment.program.start.frame; break;
@@ -178,7 +179,7 @@ export function projectSelectionInstant(input: {
const selection = locateSelection(input.semantic, input.selection);
return projectedInstant(
{ id: input.itemId, subjectId: input.subjectId, projection: input.projection, authority: input.authority },
- { spaceId: space.id, narrativeId: space.narrativeId, kind: "selection", id: selection.id },
+ { spaceId: space.id, narrativeId: input.semantic.narrativeId, kind: "selection", id: selection.id },
projectTemporalInstant(input.projection, { program, selection }, space),
);
}
@@ -196,7 +197,7 @@ export function projectMomentInstant(input: {
const moment = locateMoment(input.semantic, input.moment);
return projectedInstant(
{ id: input.itemId, subjectId: input.subjectId, projection: input.projection, authority: input.authority },
- { spaceId: space.id, narrativeId: space.narrativeId, kind: "moment", id: moment.id },
+ { spaceId: space.id, narrativeId: input.semantic.narrativeId, kind: "moment", id: moment.id },
projectTemporalInstant(input.projection, { program, moment }, space),
);
}
@@ -204,15 +205,15 @@ export function projectMomentInstant(input: {
export function projectProgramInstant(input: {
readonly itemId: string;
readonly subjectId: string;
- readonly semantic: SemanticTrack;
+ readonly space: ProgramSpace;
readonly projection: TemporalInstantExpression;
readonly authority: TemporalInstantSpec["authority"];
}): ProjectedInstant {
- const space = projectSemanticProgramSpace(input.semantic);
- const program = locateProgram(input.semantic);
+ const space = input.space;
+ const program = { id: "program" as const, start: { frame: 0 }, end: { frame: programSpaceFrameCount(space) } };
return projectedInstant(
{ id: input.itemId, subjectId: input.subjectId, projection: input.projection, authority: input.authority },
- { spaceId: space.id, narrativeId: space.narrativeId, kind: "program", id: program.id },
+ { spaceId: space.id, kind: "program", id: program.id },
projectTemporalInstant(input.projection, { program }, space),
);
}
@@ -230,7 +231,7 @@ export function projectSegmentInstant(input: {
const segment = locateSegment(input.semantic, input.segment);
return projectedInstant(
{ id: input.itemId, subjectId: input.subjectId, projection: input.projection, authority: input.authority },
- { spaceId: space.id, narrativeId: space.narrativeId, kind: "segment", id: segment.id },
+ { spaceId: space.id, narrativeId: input.semantic.narrativeId, kind: "segment", id: segment.id },
projectTemporalInstant(input.projection, { program, segment }, space),
);
}
@@ -244,8 +245,8 @@ export function composeTemporalWindow(
if (!spec.subjectId || start.subjectId !== spec.subjectId || end.subjectId !== spec.subjectId) {
throw new Error("Temporal Window endpoints must explicitly name the Window subject.");
}
- if (start.source.spaceId !== end.source.spaceId || start.source.narrativeId !== end.source.narrativeId) {
- throw new Error("Temporal Window endpoints must belong to one ProgramSpace and Narrative.");
+ if (start.source.spaceId !== end.source.spaceId) {
+ throw new Error("Temporal Window endpoints must belong to one ProgramSpace.");
}
if (end.frame < start.frame) throw new Error("Temporal Window endpoints are reversed.");
if (end.frame === start.frame) throw new Error("Temporal Window endpoints produce a zero window.");
diff --git a/packages/temporal/src/types.ts b/packages/temporal/src/types.ts
index c8cba2fa..54c2b46d 100644
--- a/packages/temporal/src/types.ts
+++ b/packages/temporal/src/types.ts
@@ -40,12 +40,13 @@ export type TemporalInstantExpression =
| { readonly ref: "segment.start"; readonly offset?: TemporalDuration }
| { readonly ref: "segment.end"; readonly offset?: TemporalDuration }
| { readonly ref: "moment.cue"; readonly offset?: TemporalDuration }
- | { readonly ref: "absolute"; readonly at: TemporalDuration };
+ | { readonly ref: "absolute"; readonly at: TemporalDuration; readonly offset?: TemporalDuration };
/** Runtime dependency used to resolve an Instant. This is not its authoring authority. */
export type TemporalSource = {
readonly spaceId: string;
- readonly narrativeId: string;
+ /** Present when the boundary originates in a Script. */
+ readonly narrativeId?: string;
readonly kind: "program" | "selection" | "segment" | "moment";
readonly id: string;
};
diff --git a/packages/temporal/test/temporal.test.ts b/packages/temporal/test/temporal.test.ts
index 1008b694..6c91d4b9 100644
--- a/packages/temporal/test/temporal.test.ts
+++ b/packages/temporal/test/temporal.test.ts
@@ -1,3 +1,4 @@
+import { projectSemanticProgramSpace } from "@hypit/semantic-track";
import assert from "node:assert/strict";
import test from "node:test";
@@ -8,6 +9,7 @@ import { semanticTrackFixture } from "../../../test/semantic-track-fixture.js";
import {
assertWindowRelation,
+ assertTemporalInstantFor,
composeTemporalWindow,
locateSelection,
projectMomentInstant,
@@ -20,12 +22,11 @@ import {
const space: ProgramSpace = {
id: "test-space",
- narrativeId: "test-narrative",
durationSec: 10,
frameRate: { numerator: 30, denominator: 1 },
};
-const semantic = semanticTrackFixture(space, {
+const semantic = semanticTrackFixture(space, { narrativeId: "test-narrative",
segments: [
{ id: "opening", frameCount: 60 },
{ id: "answer", frameCount: 60 },
@@ -52,7 +53,7 @@ const seconds = (numerator: number, denominator = 1) => ({ unit: "seconds" as co
const fixed = { kind: "fixed" as const };
type Projection = import("../src/index.js").TemporalInstantExpression;
const projectProgramInstantFixture = (input: { itemId: string; semantic: typeof semantic; projection: Projection }) =>
- projectProgramInstant({ ...input, subjectId: input.itemId, authority: fixed });
+ projectProgramInstant({ ...input, space: projectSemanticProgramSpace(input.semantic), subjectId: input.itemId, authority: fixed });
const projectMomentInstantFixture = (input: { itemId: string; semantic: typeof semantic; moment: NarrativeMomentRef; projection: Projection }) =>
projectMomentInstant({ ...input, subjectId: input.itemId, authority: fixed });
const projectSelectionInstantFixture = (input: { itemId: string; semantic: typeof semantic; selection: NarrativeSelectionRef; projection: Projection }) =>
@@ -63,7 +64,7 @@ const projectSelectionWindow = (input: {
}) => composeTemporalWindow({ id: input.itemId, subjectId: input.itemId },
projectSelectionInstant({ itemId: `${input.itemId}.start`, subjectId: input.itemId, semantic: input.semantic, selection: input.selection, projection: input.projection.start, authority: fixed }),
input.projection.end.ref.startsWith("program.") || input.projection.end.ref === "absolute"
- ? projectProgramInstant({ itemId: `${input.itemId}.end`, subjectId: input.itemId, semantic: input.semantic, projection: input.projection.end, authority: fixed })
+ ? projectProgramInstant({ itemId: `${input.itemId}.end`, subjectId: input.itemId, space: projectSemanticProgramSpace(input.semantic), projection: input.projection.end, authority: fixed })
: projectSelectionInstant({ itemId: `${input.itemId}.end`, subjectId: input.itemId, semantic: input.semantic, selection: input.selection, projection: input.projection.end, authority: fixed }));
const projectMomentWindow = (input: {
itemId: string; semantic: typeof semantic; moment: NarrativeMomentRef;
@@ -80,8 +81,8 @@ const projectSegmentWindow = (input: {
const projectProgramWindow = (input: {
itemId: string; semantic: typeof semantic; projection: { start: Projection; end: Projection };
}) => composeTemporalWindow({ id: input.itemId, subjectId: input.itemId },
- projectProgramInstant({ itemId: `${input.itemId}.start`, subjectId: input.itemId, semantic: input.semantic, projection: input.projection.start, authority: fixed }),
- projectProgramInstant({ itemId: `${input.itemId}.end`, subjectId: input.itemId, semantic: input.semantic, projection: input.projection.end, authority: fixed }));
+ projectProgramInstant({ itemId: `${input.itemId}.start`, subjectId: input.itemId, space: projectSemanticProgramSpace(input.semantic), projection: input.projection.start, authority: fixed }),
+ projectProgramInstant({ itemId: `${input.itemId}.end`, subjectId: input.itemId, space: projectSemanticProgramSpace(input.semantic), projection: input.projection.end, authority: fixed }));
test("one Selection projects exact local points and stable source identity", () => {
const result = projectSelectionWindow({
@@ -104,7 +105,7 @@ test("points preserve source identity and admit both ProgramSpace boundaries", (
}), {
id: "terminal::program",
subjectId: "terminal",
- source: { spaceId: "test-space", narrativeId: "test-narrative", kind: "program", id: "program" },
+ source: { spaceId: "test-space", kind: "program", id: "program" },
projection: { ref: "program.end" },
authority: fixed,
frame: 300,
@@ -191,8 +192,7 @@ test("Instant endpoints outside ProgramSpace are rejected before Window composit
test("program and absolute projections use exact rational frame-rate arithmetic", () => {
const ntsc: ProgramSpace = {
- id: "test-space", narrativeId: "test-narrative",
- durationSec: 1.001,
+ id: "test-space", durationSec: 1.001,
frameRate: { numerator: 30_000, denominator: 1_001 },
};
const result = projectProgramWindow({
@@ -206,10 +206,39 @@ test("program and absolute projections use exact rational frame-rate arithmetic"
assert.deepEqual(result.span, { startFrame: 15, endFrameExclusive: 30 });
});
+test("an authored clock needs no narrative and combines absolute time with frame offsets", () => {
+ const point = (itemId: string, projection: Projection) => projectProgramInstant({ itemId, subjectId: "bubble", space, projection, authority: fixed });
+ const start = point("start", { ref: "absolute", at: seconds(5, 2) });
+ const end = point("end", { ref: "absolute", at: seconds(5, 2), offset: frames(8) });
+ assert.deepEqual(composeTemporalWindow({ id: "bubble", subjectId: "bubble" }, start, end).span,
+ { startFrame: 75, endFrameExclusive: 83 });
+ assert.equal(start.source.narrativeId, undefined);
+});
+
+test("shared physical time preserves semantic ownership and rejects another film clock", () => {
+ const cue = projectMomentInstantFixture({
+ itemId: "reveal", semantic, moment: moment("answer", "b"), projection: { ref: "moment.cue" },
+ });
+ const end = projectProgramInstant({
+ itemId: "end", subjectId: "reveal", space,
+ projection: { ref: "absolute", at: seconds(3) }, authority: fixed,
+ });
+ const window = composeTemporalWindow({ id: "reveal", subjectId: "reveal" }, cue, end);
+ assert.deepEqual(window.span, { startFrame: 60, endFrameExclusive: 90 });
+ assert.equal(window.start.source.narrativeId, "test-narrative");
+ assert.equal(window.end.source.narrativeId, undefined);
+ assert.throws(() => projectMomentInstantFixture({
+ itemId: "reveal", semantic, moment: { ...moment("answer", "b"), narrativeId: "another-story" },
+ projection: { ref: "moment.cue" },
+ }), /belongs to Narrative another-story/u);
+ assert.throws(() => assertTemporalInstantFor(cue, {
+ subjectId: "reveal", space: { ...space, id: "another-film" },
+ }), /different ProgramSpace/u);
+});
+
test("frame and authored durations enter one exact sample-boundary rule", () => {
const ntsc: ProgramSpace = {
- id: "test-space", narrativeId: "test-narrative",
- durationSec: 1.001,
+ id: "test-space", durationSec: 1.001,
frameRate: { numerator: 30_000, denominator: 1_001 },
};
assert.equal(programFrameSampleBoundary(ntsc, 15, 48_000), 24_024);
@@ -218,8 +247,7 @@ test("frame and authored durations enter one exact sample-boundary rule", () =>
assert.equal(temporalDurationInSamples({ unit: "milliseconds", value: 125 }, ntsc), 6_000);
assert.equal(temporalDurationInSamples(seconds(1, 3), ntsc), 16_000);
const twentyFour: ProgramSpace = {
- id: "test-space", narrativeId: "test-narrative",
- durationSec: 1,
+ id: "test-space", durationSec: 1,
frameRate: { numerator: 24, denominator: 1 },
};
assert.equal(programFrameSampleBoundary(twentyFour, 1, 44_100), 1_838,
diff --git a/packages/typography-track/README.md b/packages/typography-track/README.md
index dcfc42d0..202c0b5a 100644
--- a/packages/typography-track/README.md
+++ b/packages/typography-track/README.md
@@ -1,10 +1,14 @@
# `@hypit/typography-track`
+The Track Surface accepts `semantic={speech.semantic}` or `space={animation}`. Semantic context
+resolves Script Selections and Moments; authored space supports clock-based animation. Shared `at`
+inputs accept a Moment or a time such as `2s`; `at` with `for` produces a Window where required.
+
Place independently authored titles, labels, verdicts and other text in the video. Their lifetime
can follow a Script Selection or Moment even when their wording differs from the speech. For words
displayed as they are spoken, use a [Caption family](../caption-fine/README.md).
-The Track takes `semantic` and derives ProgramSpace from it. Items use explicit Point, Frame or Path
+The Track takes `semantic` or `space` to supply ProgramSpace. Items use explicit Point, Frame or Path
placement, exact fonts and Styles. Timing can follow a Segment, Selection, Moment with duration, the
whole program or explicit clock expressions. `.program` describes the text presentation; `.track`
is the VisualTrack to include in Film. Each item has its own lifetime and stacking order.
diff --git a/packages/typography-track/src/fragment.ts b/packages/typography-track/src/fragment.ts
index 54a6e304..73f60745 100644
--- a/packages/typography-track/src/fragment.ts
+++ b/packages/typography-track/src/fragment.ts
@@ -1,7 +1,7 @@
+import { programSpaceTypes } from "@hypit/program-space";
import { compositionTypes } from "@hypit/composition";
import type { VisualTrack } from "@hypit/composition";
import { sealGraphFragment } from "@hypit/elaborator";
-import { semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";
import { typographyTrackProducers, typographyTrackTypes } from "./manifest.js";
@@ -11,20 +11,14 @@ const operation = (id: string) => ({ kind: "fragment-operation" as const, operat
/** Provider-free official lowering from TypographyTrackProgram to one peer VisualTrack. */
export const typographyTrackFragment = sealGraphFragment({
inputs: [
- { name: "semantic", type: semanticTrackTypes.track },
+ { name: "space", type: programSpaceTypes.programSpace },
{ name: "program", type: typographyTrackTypes.program },
],
operations: [
- {
- id: "space",
- producer: semanticTrackProducers.projectProgramSpace,
- inputs: { track: input("semantic") },
- result: { kind: "output", name: "space" },
- },
{
id: "render",
producer: typographyTrackProducers.render,
- inputs: { space: operation("space"), program: input("program") },
+ inputs: { space: input("space"), program: input("program") },
result: { kind: "output", name: "track" },
},
],
diff --git a/packages/typography-track/src/manifest.ts b/packages/typography-track/src/manifest.ts
index 9cdfc332..2411455b 100644
--- a/packages/typography-track/src/manifest.ts
+++ b/packages/typography-track/src/manifest.ts
@@ -1,8 +1,9 @@
+import { temporalContextAttributeVocabulary } from "@hypit/temporal-markup";
import { readFile } from "node:fs/promises";
import { narrativeDependency } from "@hypit/narrative";
import { programSpaceDependency, programSpaceTypes } from "@hypit/program-space";
-import { semanticTrackDependency, semanticTrackTypes } from "@hypit/semantic-track";
+import { semanticTrackDependency } from "@hypit/semantic-track";
import {
compositionDependency,
compositionTypes,
@@ -428,8 +429,7 @@ export const typographyTrackMarkupSurfaces = [
attributes: [
{ name: "id", kind: "identifier", required: true,
summary: "Names this Track and prefixes the identity of every item spec it seals." },
- { name: "semantic", kind: "reference", required: true, accepts: [semanticTrackTypes.track],
- summary: "Chooses the SemanticTrack that owns the frame domain and resolves every item window." },
+ ...temporalContextAttributeVocabulary,
],
children: [
{ tag: "Point", cardinality: "many",
@@ -500,8 +500,7 @@ export const typographyTrackMarkupSurfaces = [
attributes: [
{ name: "id", kind: "identifier", required: true,
summary: "Names this Mask, under which its VisualTrack is published." },
- { name: "semantic", kind: "reference", required: true, accepts: [semanticTrackTypes.track],
- summary: "Chooses the SemanticTrack whose frame domain the masked Track is rendered into." },
+ ...temporalContextAttributeVocabulary,
{ name: "text", kind: "reference", required: true, accepts: [typographyTrackTypes.program],
summary: "Chooses the authored Text Program whose items give the mask its shape and timing." },
{ name: "material", kind: "reference", required: true, accepts: [mediaTypes.compositableSurface],
diff --git a/packages/typography-track/src/surface.ts b/packages/typography-track/src/surface.ts
index ecd55a58..3e4aec29 100644
--- a/packages/typography-track/src/surface.ts
+++ b/packages/typography-track/src/surface.ts
@@ -1,4 +1,5 @@
-import { semanticTrackProducers, semanticTrackTypes } from "@hypit/semantic-track";
+import { createTemporalSpace, resolveTemporalContext } from "@hypit/temporal-markup";
+import { programSpaceTypes } from "@hypit/program-space";
import { compositionTypes } from "@hypit/composition";
import type {
VisualColorPaint,
@@ -626,7 +627,6 @@ type FragmentItem = {
function createTrackFragment(id: string, items: readonly FragmentItem[]): GraphFragment {
const operations: FragmentOperation[] = [
- { id: "text:space", producer: semanticTrackProducers.projectProgramSpace, inputs: { track: input("semantic") }, result: { kind: "output", name: "space" } },
{ id: "text:set:empty", producer: typographyTrackProducers.createSet, inputs: {}, result: { kind: "output", name: "set" } },
];
let current = "text:set:empty";
@@ -649,7 +649,7 @@ function createTrackFragment(id: string, items: readonly FragmentItem[]): GraphF
}
const append = `text:set:append:${String(index + 1).padStart(4, "0")}`;
const common = {
- set: operation(current), header: input("header"), space: operation("text:space"), placement: operation(bind),
+ set: operation(current), header: input("header"), space: input("space"), placement: operation(bind),
spec: item.contentName === undefined ? input(item.specName) : operation(materialized),
style: input(item.styleName), motion: input(item.motionName), window: input(item.windowName),
};
@@ -658,10 +658,10 @@ function createTrackFragment(id: string, items: readonly FragmentItem[]): GraphF
}
operations.push(
{ id: "text:finalize", producer: typographyTrackProducers.finalize, inputs: { header: input("header"), set: operation(current) }, result: { kind: "output", name: "program" } },
- { id: "text:render", producer: typographyTrackProducers.render, inputs: { space: operation("text:space"), program: operation("text:finalize") }, result: { kind: "output", name: "track" } },
+ { id: "text:render", producer: typographyTrackProducers.render, inputs: { space: input("space"), program: operation("text:finalize") }, result: { kind: "output", name: "track" } },
);
const inputEntries = [
- { name: "semantic", type: semanticTrackTypes.track },
+ { name: "space", type: programSpaceTypes.programSpace },
{ name: "header", type: typographyTrackTypes.header },
...items.flatMap((item) => [
{ name: item.geometryName, type: item.placementKind === "point" ? spatialTypes.point : item.placementKind === "area" ? spatialTypes.frame : spatialTypes.path },
@@ -688,9 +688,10 @@ function createTrackFragment(id: string, items: readonly FragmentItem[]): GraphF
}
export const decodeTypographyTrackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
- allowed(element, ["id", "semantic"], ["id", "semantic"]);
+ allowed(element, ["id", "semantic", "space"], ["id"]);
const id = text(element, "id");
- const semantic = reference(element.attributes.semantic, `${element.name}.semantic`, semanticTrackTypes.track, resolveReference);
+ const context = resolveTemporalContext({ element, resolveReference });
+ const time = createTemporalSpace({ id: id, element, ...context });
const headerId = `${id}.__header`;
const records: SurfaceRecordDraft[] = [{
id: headerId, type: typographyTrackTypes.header,
@@ -698,8 +699,8 @@ export const decodeTypographyTrackSurface: StructuredSurfaceHandler = ({ element
}];
const defaultMotionId = `${id}.__still-motion`;
records.push({ id: defaultMotionId, type: typographyTrackTypes.motion, value: { kind: "inline", value: stillTextMotion(defaultMotionId) }, range: element.range });
- const temporalComponents: SurfaceComponentDraft[] = [];
- const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [];
+ const temporalComponents: SurfaceComponentDraft[] = [...time.components];
+ const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [...time.fragments];
const items: Array<FragmentItem & {
readonly geometry: SurfaceResolvedReference;
readonly style: SurfaceResolvedReference;
@@ -730,7 +731,7 @@ export const decodeTypographyTrackSurface: StructuredSurfaceHandler = ({ element
"id", "content", "placement", "style", "motion", ...temporalWindowAttributeNames,
], ["id", "placement", "style"]);
const itemId = text(child, "id");
- const temporal = createTemporalWindowProjection({ id: itemId, element: child, semantic, resolveReference });
+ const temporal = createTemporalWindowProjection({ id: itemId, element: child, ...context, space: time.space, resolveReference });
records.push(...temporal.records); temporalComponents.push(...temporal.components); temporalFragments.push(...temporal.fragments);
const placementKind = form.toLowerCase() as "point" | "area" | "path";
const geometry = reference(child.attributes.placement, `${child.name}.placement`, placementKind === "point" ? spatialTypes.point : placementKind === "area" ? spatialTypes.frame : spatialTypes.path, resolveReference);
@@ -776,7 +777,7 @@ export const decodeTypographyTrackSurface: StructuredSurfaceHandler = ({ element
components: [...temporalComponents, {
id, fragment: fragment.id,
inputs: {
- semantic: semantic.ref, header: { kind: "record", id: headerId },
+ space: time.space.ref, header: { kind: "record", id: headerId },
...Object.fromEntries(items.flatMap((item) => [
[item.geometryName, item.geometry.ref], [item.specName, { kind: "record" as const, id: item.specId }],
[item.windowName, item.window],
@@ -793,7 +794,7 @@ export const decodeTypographyTrackSurface: StructuredSurfaceHandler = ({ element
function createMaskFragment(id: string): GraphFragment {
const inputs = [
- { name: "semantic", type: semanticTrackTypes.track },
+ { name: "space", type: programSpaceTypes.programSpace },
{ name: "program", type: typographyTrackTypes.program },
{ name: "material", type: mediaTypes.compositableSurface },
{ name: "spec", type: typographyTrackTypes.maskSpec },
@@ -801,13 +802,9 @@ function createMaskFragment(id: string): GraphFragment {
return sealGraphFragment({
inputs,
operations: [
- {
- id: "text-mask:space", producer: semanticTrackProducers.projectProgramSpace,
- inputs: { track: input("semantic") }, result: { kind: "output", name: "space" },
- },
{
id: "text-mask:render", producer: typographyTrackProducers.renderMask,
- inputs: { space: operation("text-mask:space"), program: input("program"), material: input("material"), spec: input("spec") },
+ inputs: { space: input("space"), program: input("program"), material: input("material"), spec: input("spec") },
result: { kind: "output", name: "track" },
},
],
@@ -818,10 +815,11 @@ function createMaskFragment(id: string): GraphFragment {
}
export const decodeTypographyMaskSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
- allowed(element, ["id", "semantic", "text", "material", "mode", "fit"], ["id", "semantic", "text", "material"]);
+ allowed(element, ["id", "semantic", "space", "text", "material", "mode", "fit"], ["id", "text", "material"]);
empty(element);
const id = text(element, "id");
- const semantic = reference(element.attributes.semantic, `${element.name}.semantic`, semanticTrackTypes.track, resolveReference);
+ const context = resolveTemporalContext({ element, resolveReference });
+ const time = createTemporalSpace({ id: id, element, ...context });
const program = reference(element.attributes.text, `${element.name}.text`, typographyTrackTypes.program, resolveReference);
const material = reference(element.attributes.material, `${element.name}.material`, mediaTypes.compositableSurface, resolveReference);
const specId = `${id}.__spec`;
@@ -833,12 +831,12 @@ export const decodeTypographyMaskSurface: StructuredSurfaceHandler = ({ element,
const fragment = createMaskFragment(id);
return {
records: [{ id: specId, type: typographyTrackTypes.maskSpec, value: { kind: "inline", value: spec }, range: element.range }],
- components: [{
+ components: [...time.components, {
id, fragment: fragment.id,
- inputs: { semantic: semantic.ref, program: program.ref, material: material.ref, spec: { kind: "record", id: specId } },
+ inputs: { space: time.space.ref, program: program.ref, material: material.ref, spec: { kind: "record", id: specId } },
outputs: { track: `${id}.track` }, range: element.range,
}],
- fragments: [fragment],
+ fragments: [...time.fragments, fragment],
exports: [`${id}.track`],
};
};
diff --git a/packages/typography-track/test/typography-track.test.ts b/packages/typography-track/test/typography-track.test.ts
index 18e7ff21..678469a0 100644
--- a/packages/typography-track/test/typography-track.test.ts
+++ b/packages/typography-track/test/typography-track.test.ts
@@ -49,8 +49,7 @@ import { sealText, textComponent, textDependency, textManifest, textTypes } from
import { MarkupSurfaceRegistry, createMarkupAuthorFrontend } from "@hypit/markup";
import { createRecordAdmitter, TypeValidatorRegistry } from "@hypit/validation";
-const space = sealProgramSpace({ id: "test-space", narrativeId: "test-narrative",
- durationSec: 5,
+const space = sealProgramSpace({ id: "test-space", durationSec: 5,
frameRate: { numerator: 30, denominator: 1 },
});
const semantic = semanticTrackFixture(space, { anchors: [
@@ -278,7 +277,7 @@ test("TypographyTrackProgram rejects a frame span outside ProgramSpace", () => {
test("Selection Text consumes explicit Selection, SemanticTrack, Style, Motion and Placement edges", () => {
const selection: NarrativeSelectionRef = {
- narrativeId: space.narrativeId,
+ narrativeId: "script",
id: "callout",
startAnchorId: "selection:start",
endAnchorId: "selection:end",
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index 7c43a580..373efa1f 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -310,6 +310,18 @@ importers:
specifier: 5.9.3
version: 5.9.3
+ examples/semantic-composition/packages/chat-scene:
+ devDependencies:
+ '@types/node':
+ specifier: 24.10.1
+ version: 24.10.1
+ hypit:
+ specifier: workspace:*
+ version: link:../../../..
+ typescript:
+ specifier: 5.9.3
+ version: 5.9.3
+
examples/semantic-composition/packages/responsive-explainer:
devDependencies:
'@types/node':
@@ -1005,6 +1017,9 @@ importers:
'@hypit/svs':
specifier: workspace:*
version: link:../svs
+ '@hypit/temporal-markup':
+ specifier: workspace:*
+ version: link:../temporal-markup
devDependencies:
'@hypit/core':
specifier: workspace:*
@@ -1036,6 +1051,9 @@ importers:
'@hypit/film':
specifier: workspace:*
version: link:../film
+ '@hypit/program-space':
+ specifier: workspace:*
+ version: link:../program-space
'@hypit/semantic-track':
specifier: workspace:*
version: link:../semantic-track
@@ -2370,6 +2388,9 @@ importers:
'@hypit/semantic-track':
specifier: workspace:*
version: link:../semantic-track
+ '@hypit/temporal-markup':
+ specifier: workspace:*
+ version: link:../temporal-markup
devDependencies:
'@hypit/core':
specifier: workspace:*
@@ -3051,6 +3072,9 @@ importers:
'@hypit/narrative':
specifier: workspace:*
version: link:../narrative
+ '@hypit/program-space':
+ specifier: workspace:*
+ version: link:../program-space
'@hypit/semantic-track':
specifier: workspace:*
version: link:../semantic-track
diff --git a/test/semantic-track-fixture.ts b/test/semantic-track-fixture.ts
index a26513b5..075ee84c 100644
--- a/test/semantic-track-fixture.ts
+++ b/test/semantic-track-fixture.ts
@@ -23,6 +23,7 @@ export function semanticTrackFixture(
space: ProgramSpace,
options: {
readonly id?: string;
+ readonly narrativeId?: string;
readonly segments?: readonly FixtureSegment[];
readonly anchors?: readonly FixtureAnchor[];
} = {},
@@ -41,7 +42,7 @@ export function semanticTrackFixture(
return {
// A SemanticTrack is the author-visible identity of the ProgramSpace it projects.
id: options.id ?? space.id,
- narrativeId: space.narrativeId,
+ narrativeId: options.narrativeId ?? "script",
items: segments.map((segment, index) => {
const endFrameExclusive = startFrame + segment.frameCount;
const startAnchorId = `segment:${segment.id}:start`;
@@ -52,7 +53,7 @@ export function semanticTrackFixture(
&& (index === segments.length - 1 ? anchor.frame <= endFrameExclusive : anchor.frame < endFrameExclusive))
.map((anchor) => ({ identity: anchor.identity, frame: anchor.frame - startFrame }));
const take = {
- narrativeId: space.narrativeId,
+ narrativeId: options.narrativeId ?? "script",
media: {
timeline: { frameRate: space.frameRate, frameCount: segment.frameCount },
audio: { artifact: audioArtifact },
diff --git a/test/temporal-fixture.ts b/test/temporal-fixture.ts
index bf471ace..f673e37b 100644
--- a/test/temporal-fixture.ts
+++ b/test/temporal-fixture.ts
@@ -1,3 +1,4 @@
+import { projectSemanticProgramSpace } from "@hypit/semantic-track";
import type { NarrativeExcerpt, NarrativeMomentRef, NarrativeSelectionRef } from "@hypit/narrative";
import type { SemanticTrack } from "@hypit/semantic-track";
import {
@@ -22,7 +23,7 @@ export function projectProgramInstantFixture(input: {
readonly semantic: SemanticTrack;
readonly projection: TemporalInstantExpression;
}) {
- return projectProgramInstant({ ...input, subjectId: input.subjectId ?? input.itemId, authority: fixed });
+ return projectProgramInstant({ ...input, space: projectSemanticProgramSpace(input.semantic), subjectId: input.subjectId ?? input.itemId, authority: fixed });
}
export function projectMomentInstantFixture(input: {
@@ -43,8 +44,8 @@ export function projectProgramWindow(input: {
}) {
const subjectId = input.subjectId ?? input.itemId;
return composeTemporalWindow({ id: input.itemId, subjectId },
- projectProgramInstant({ itemId: `${input.itemId}.start`, subjectId, semantic: input.semantic, projection: input.projection.start, authority: fixed }),
- projectProgramInstant({ itemId: `${input.itemId}.end`, subjectId, semantic: input.semantic, projection: input.projection.end, authority: fixed }));
+ projectProgramInstant({ itemId: `${input.itemId}.start`, subjectId, space: projectSemanticProgramSpace(input.semantic), projection: input.projection.start, authority: fixed }),
+ projectProgramInstant({ itemId: `${input.itemId}.end`, subjectId, space: projectSemanticProgramSpace(input.semantic), projection: input.projection.end, authority: fixed }));
}
export function projectSelectionWindow(input: {