Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

feat(runtime): add project-owned build results

kashorin0664b5bac002
diff --git a/docs/guide/providers.md b/docs/guide/providers.md
index ca6b69ad..9fabefaf 100644
--- a/docs/guide/providers.md
+++ b/docs/guide/providers.md
@@ -47,7 +47,7 @@ packages or the CLI:
## 3. Implement the Provider
The Provider handles Commands from the Scheduler: request submission, polling, download and
-ArtifactStore persistence.
+admission into the current Build's working byte area.
```typescript
// src/provider.ts
@@ -215,7 +215,7 @@ hypit doctor hypit.runtime.json
| Package | Pattern |
|---|---|
-| `provider-kie` | Remote API: upload, paid submission, checkpointed polling, bounded download, immediate ArtifactStore persistence |
+| `provider-kie` | Remote API: upload, paid submission, checkpointed polling, bounded download into the current Build workspace |
| `provider-media-local` | Local process: shell-free ffprobe/ffmpeg with bounded execution |
| `provider-whisperx-local` | Local HTTP service with a warm model, single-admit concurrency |
| `provider-hyperframes-local` | Local process: Chrome rendering with worker parallelism and output probe validation |
diff --git a/docs/guide/runtime.md b/docs/guide/runtime.md
index 0a30698a..2de2b87c 100644
--- a/docs/guide/runtime.md
+++ b/docs/guide/runtime.md
@@ -11,7 +11,8 @@ Hypit keeps three decisions separate:
|---|---|
| Workspace | Which source files and local assets compilation may read |
| Runtime Profile | Which environmental packages may be used |
-| Local Runtime | How Builds are queued, executed, stored and observed |
+| Local Runtime | How active Builds are queued, executed and observed |
+| Project Build Results | Which completed public Outputs belong to each Build |
Core is a domain neutral state machine. It accepts facts, derives Commands and verifies returned
Records. It does not start processes, read credentials, choose Providers or know that a Build makes
@@ -31,10 +32,6 @@ The Profile contains only environmental choices that genuinely vary:
"use": "@hypit/runtime-local",
"config": {
"dataRoot": ".hypit/runtimes/local",
- "artifacts": {
- "use": "@hypit/artifact-store-fs",
- "config": { "path": "artifacts" }
- },
"credentials": {
"env": { "use": "@hypit/credential-store-env" }
},
@@ -46,7 +43,6 @@ The Profile contains only environmental choices that genuinely vary:
}
```
-* `artifacts` selects where generated bytes are stored.
* `credentials` selects stores for explicit credential references.
* `endpoints` selects exact Provider implementations and their configuration.
@@ -63,13 +59,28 @@ Selecting a Profile writes a project pointer. Execution data stays under `dataRo
runtimes/
local/
runtime.sqlite
- artifacts/
+ work/
worker/
programs/
+ results/
+ <build-id>/
+ result.json
+ files/
+ values/
```
-`check` and `plan` do not create Runtime data. SQLite stores Build facts, Operations and queue state;
-Artifact bytes stay in the selected Artifact Store; secrets stay in the selected Credential Stores.
+`check` and `plan` do not create Runtime data. SQLite holds only active execution facts, Provider
+Operations and queue state. Each active Build has a private working byte directory. On an ordinary
+complete, failed or cancelled path, once the Result is safely written, its heavy SQLite execution
+rows and working bytes are removed. If writing the Result itself fails, the working state is kept for
+local repair. One small terminal dispatch row may remain for Runtime status. It is not the history
+source.
+
+Historical content lives in the project, not the Profile: `.hypit/results/<build-id>/result.json`
+names final Targets and every public Author Output that actually completed on their route. Media is
+under that Result's `files/`; structured values are under `values/`. `builds`, `history`, `inspect`,
+`get` and `build-record` read these Results directly and never need Runtime SQLite or a selected byte
+store.
The Workspace is resolved independently from the explicit `--workspace`, the project containing the
Runtime pointer, or the entry source directory. Runtime configuration cannot widen source access.
@@ -96,7 +107,7 @@ hypit programs down
```
`programs up` first prepares the selected Endpoint packages' upstream npm dependencies, then operates
-their declared programs. The commands do not open SQLite, Artifact Stores or Credential Stores.
+their declared programs. The commands do not open SQLite or Credential Stores.
## Lifecycle
@@ -111,8 +122,8 @@ hypit runtime down
`runtime up` asks npm to prepare the selected adapters' exact upstream packages in the shared
machine home, prepares declared Managed Programs, then starts the local Worker. `build` performs no
provisioning: it runs cheap read-only preflight, submits work only when ready, and ensures the Worker
-is available. `status`, `queue` and `cancel` observe or control durable work. A terminal Build is not
-resumed. Reusing an earlier result is an explicit Candidate in a new `.svrun`, not hidden Runtime
+is available. `status`, `queue` and `cancel` observe or control active work. A terminal Build is not
+resumed. Reusing an earlier Result is an explicit Candidate in a new `.svrun`, not hidden Runtime
behavior.
A running Worker loads installed Component packages as Builds first require them. A later Build may
diff --git a/docs/quickstart.md b/docs/quickstart.md
index 47cdab73..5d48f1bd 100644
--- a/docs/quickstart.md
+++ b/docs/quickstart.md
@@ -83,7 +83,7 @@ A working video project normally has four authored or configured inputs:
| `main.svml` | What video are you making? |
| `recipes.svs` | Which reusable Recipe values does it use? |
| `build.svrun` | Which outputs and Candidates does this Run select? |
-| `hypit.runtime.json` | Which machine, stores and Provider endpoints execute it? |
+| `hypit.runtime.json` | Which credentials, Provider endpoints and services execute it? |
The short form is:
@@ -111,7 +111,8 @@ hypit plan build.svrun
A project with `package.json` owns its third-party packages. A plain creative folder needs no Node
project and uses official packages from the installed Distribution. Source and exported files remain in the
-project. Runtime state and Artifacts live under the selected Profile's `dataRoot`.
+project. Build Results live under the project's `.hypit/results`; active execution state lives under
+the selected Profile's `dataRoot`.
`runtime use` stores only a local pointer at `.hypit/runtime`. Source imports select author
packages; the Profile independently selects Runtime packages through `use`.
@@ -145,8 +146,8 @@ hypit get <build-id> \
--to output/final.mp4
```
-The Runtime archives accepted intermediate Records and media. `get` makes a human-readable copy; it
-does not decide whether an intermediate result should have been retained.
+The Build Result saves every public Author Output completed on the Target route. `get` makes an
+optional human-readable copy; Targets do not double as a retention list.
## Local tools used by real Builds
diff --git a/docs/quickstart/run.md b/docs/quickstart/run.md
index afbe256e..52224ffe 100644
--- a/docs/quickstart/run.md
+++ b/docs/quickstart/run.md
@@ -7,7 +7,7 @@ description: Declaring build targets, reusing results and configuring the runtim
The Author Source defines the video. A Run Source chooses which of its public outputs to produce and
which explicit Candidates, if any, should satisfy them. The Runtime Profile chooses the machine,
-stores and Provider endpoints that execute the resulting plan.
+credentials, Provider endpoints and services that execute the resulting plan.
Select the project Runtime once:
@@ -65,9 +65,10 @@ Every `.svrun` file begins with its processing instruction:
### Targets
-A Target is simply an output you want from this Build. It may be a generated image, a video take, a
-timing map, a Track or the final render. The compiler only executes Operations needed for the chosen
-Targets; unrelated branches are left alone.
+A Target is the Build's final intention, normally the finished video or another real deliverable.
+It is not a retention list. The compiler executes only the route needed for the Targets, and every
+public Author Output that actually completes on that route is written into the same Build Result.
+Internal Operation values remain execution details.
### Multiple targets
@@ -78,18 +79,18 @@ You can demand multiple outputs from one Build:
<target output="captions.track"/>
```
-Different build intentions should be separate `.svrun` files. They can point to the same Author
-Source without duplicating it. For example, `images.svrun` may target image outputs while
-`film.svrun` targets the final video.
+Use multiple Targets only when one execution genuinely has several final goals. Different build
+intentions should be separate `.svrun` files. They can point to the same Author Source without
+duplicating it.
## Reusing results
-Hypit has no implicit cache. Reusing a result is explicit Run Graph authoring — you declare
-historical Records as zero-input Candidates and connect them through Satisfaction edges.
+Hypit has no implicit cache. Reusing a result is explicit Run Graph authoring — you declare one
+named Output from one earlier Build Result as a zero-input Candidate and connect it through a
+Satisfaction edge.
As soon as a generated image or take is accepted, reuse it explicitly in the next `.svrun` with
-`build-record` and `satisfy`, then inspect the frozen plan before starting paid downstream work.
-Core has no Pin state or fidelity label.
+`build-record` and `satisfy`, then inspect the plan before starting paid downstream work.
```svml
<?svml using="@hypit/run-markup@1"?>
@@ -116,25 +117,23 @@ Core has no Pin state or fidelity label.
### Finding reusable output
-Query an output name as it appeared in each historical Build's frozen Host Catalog:
+Query an output name across the project's local Build Results:
```bash
hypit history hook-take.video
```
-`history` reports only public Logical Outputs that the Build actually produced. It
-does not list merely declared-but-unbuilt aliases or authored Record aliases that cannot back a
-`build-record` Candidate. If the old name is unknown, list accepted output names from Builds whose
-Catalog recorded an exact source path:
+`history` reports only public Author Outputs that the Build actually produced. It does not list
+declared-but-unbuilt outputs or internal Operation values. If the old name is unknown, list Outputs
+from Results whose manifest names the exact Author Source path:
```bash
hypit history --source ./main.svml
```
-An output name is a human locator inside one immutable historical Catalog, not its identity. The
-historical Core Build, Logical Output and Record digests carry identity. If the current source
-renames `hook-take.video` to `opening-shot.video`, keep the old name on `<build-record>` and use the
-current name on `<satisfy>`:
+An output name is a human locator inside one Build Result. The pair `build + output` is the exact
+address. If the current source renames `hook-take.video` to `opening-shot.video`, keep the old name
+on `<build-record>` and use the current name on `<satisfy>`:
```svml
<build-record id="approved-opening"
@@ -142,13 +141,14 @@ current name on `<satisfy>`:
<satisfy output="opening-shot.video" candidate="approved-opening"/>
```
-Hypit never infers that two names mean the same author intent. Every `build` invocation receives
-a fresh Build id, which the CLI prints and the Runtime archives. Source identity never reclaims an
-earlier Build. A later Run reuses an accepted result only by naming that historical Build id here.
+Hypit never infers that two names mean the same author intent. Every `build` invocation receives a
+fresh Build id and its own Result directory, even when nothing changed. A later Run reuses an Output
+only by naming the earlier Build id and Output here. If that earlier Output already forwards to an
+older one, resolution follows the explicit chain; no bytes are copied into the new Result.
### build-record
-Declares a zero-input Candidate backed by a historical Record from a previous Build:
+Declares a zero-input Candidate backed by one named Output from a previous Build Result:
| Attribute | Description |
|---|---|
@@ -183,9 +183,9 @@ to one current Logical Output:
<satisfy output="opening-shot.video" candidate="approved-opening"/>
```
-The file is read relative to the `.svrun`, content-addressed and archived with the Build. There is
-no special Pin state, filename cache or hidden history lookup. A black video, preview image or
-human-supplied result uses the same mechanism.
+The file is read relative to the `.svrun`. If it becomes a completed public Output on the Target
+route, it is written into the Build Result like any generated media. There is no hidden history
+lookup. A black video, preview image or human-supplied result uses the same mechanism.
## Runtime Profile
@@ -252,6 +252,11 @@ only with `--workspace`. `--package-root` locates installed packages and never w
```text
.hypit/
+ results/
+ <build-id>/
+ result.json
+ files/
+ values/
output/
```
@@ -305,7 +310,7 @@ processes declared by Endpoints.
### 4. Submit the Build
```bash
-hypit build build.svrun --follow
+hypit build build.svrun --name first-cut --follow
```
Without `--follow`, `build` returns after durable submission. The detached Worker continues. With
@@ -326,12 +331,13 @@ A plain `status <build-id>` prints one snapshot. `status --watch` exits at termi
| `--runtime` | One-command Runtime Profile override; normally select it once with `runtime use` |
| `--package-root` | Host directory containing the installed packages |
| `--workspace` | Explicit Source Workspace override |
+| `--name` | Optional human-facing name for this independent Build |
| `--follow` | Wait for terminal state as an observer; durable execution remains with the Worker |
Each invocation creates a fresh Build id, even when the Author and Run Sources are unchanged. That
is necessary for non-deterministic generation: cross-Build reuse belongs only to explicit Candidates
-in a Run Source. Once submitted, that one Build is durable. A Worker restart continues its accepted
-Records and the same external task checkpoints; it never turns another invocation into that Build.
+in a Run Source. While a Build is active, a Worker restart continues its accepted execution facts
+and the same external task checkpoints; it never turns another invocation into that Build.
### 5. Inspect and retrieve results
@@ -339,8 +345,8 @@ Records and the same external task checkpoints; it never turns another invocatio
hypit inspect <build-id>
```
-`inspect` reports durable Build state, demanded outputs, and accepted Records. Retrieve the selected
-archived Artifact only after those facts are correct:
+`inspect` reads the project-owned Result directly and lists final Targets plus every completed
+public Output saved along their execution route:
```bash
hypit get <build-id> \
@@ -348,17 +354,16 @@ hypit get <build-id> \
--to examples/talking-head-aroll/output/final.mp4
```
-Every accepted intermediate Record and Artifact is archived before the Build completes. `get` makes
-a copy of an already durable Record. Blob Artifacts are streamed from the selected Store, checked
-against their declared size and SHA-256 digest, and only then atomically replace the destination;
-exporting a large MP4 does not buffer the complete file in CLI memory.
+`get` resolves one exact `build + output` address and makes an optional copy. A file Output is copied
+from that Result; a structured Output is read from its Result value file; a forwarded historical
+Output is followed to its declared earlier Result. The Runtime Profile is not involved.
-The terminal Build result prints the exact `get --name …` command for every targeted source alias;
+The terminal Build result prints the exact `get --name …` command for every file Target;
there is no need to inspect opaque Record ids just to export `final.video`.
### 6. Reuse in a new Build
-Create a new `.svrun` file that references the completed Build's Records (see [Reusing results](#reusing-results)
+Create a new `.svrun` file that references the completed Build's Outputs (see [Reusing results](#reusing-results)
above), then submit it:
```bash
diff --git a/docs/zh/guide/providers.md b/docs/zh/guide/providers.md
index a39539c0..6b2fb827 100644
--- a/docs/zh/guide/providers.md
+++ b/docs/zh/guide/providers.md
@@ -44,7 +44,7 @@ Provider 包依赖 Runtime 端口与共享能力词汇,不依赖精确模型
## 3. 实现 Provider
-Provider 处理来自 Scheduler 的 Command:提交请求、轮询、下载以及 ArtifactStore 持久化。
+Provider 处理来自 Scheduler 的 Command:提交请求、轮询、下载,并把字节交给当前 Build 的临时工作区。
```typescript
// src/provider.ts
@@ -207,7 +207,7 @@ hypit doctor hypit.runtime.json
| 包 | 模式 |
|---|---|
-| `provider-kie` | 远程 API:上传、付费提交、带检查点的轮询、有界下载、即时 ArtifactStore 持久化 |
+| `provider-kie` | 远程 API:上传、付费提交、带检查点的轮询、有界下载、写入当前 Build 工作区 |
| `provider-media-local` | 本地进程:不经 shell 的 ffprobe/ffmpeg,执行有界 |
| `provider-whisperx-local` | 本地 HTTP 服务:带热模型,单次准入并发 |
| `provider-hyperframes-local` | 本地进程:Chrome 渲染,带 worker 并行与输出探测校验 |
diff --git a/docs/zh/guide/runtime.md b/docs/zh/guide/runtime.md
index d0a00ad3..478f1b88 100644
--- a/docs/zh/guide/runtime.md
+++ b/docs/zh/guide/runtime.md
@@ -11,7 +11,8 @@ Hypit 把三个决定分开:
|---|---|
| Workspace | 编译能读取哪些 Source 和本地素材 |
| Runtime Profile | 允许使用哪些环境包 |
-| Local Runtime | Build 如何排队、执行、保存和被观察 |
+| Local Runtime | 活跃 Build 如何排队、执行和被观察 |
+| 项目 Build Results | 每个 Build 实际完成了哪些公开 Output |
Core 是领域无关的状态机。它接收事实、派生 Command、验证 Record,不启动进程、不读取凭证、
不选择 Provider,也不知道 Build 最终是不是视频。
@@ -30,10 +31,6 @@ Profile 只保留真正会随环境变化的选择:
"use": "@hypit/runtime-local",
"config": {
"dataRoot": ".hypit/runtimes/local",
- "artifacts": {
- "use": "@hypit/artifact-store-fs",
- "config": { "path": "artifacts" }
- },
"credentials": {
"env": { "use": "@hypit/credential-store-env" }
},
@@ -45,8 +42,7 @@ Profile 只保留真正会随环境变化的选择:
}
```
-`artifacts` 决定产物字节放在哪里;`credentials` 选择凭证存储;`endpoints` 选择明确的
-Provider 实现。
+`credentials` 选择凭证存储;`endpoints` 选择明确的 Provider 实现。
安装包只增加一种可选实现,不会自动激活。Profile 不包含 Workspace、作者 import 或隐藏的
创作路由。
@@ -61,13 +57,26 @@ Provider 实现。
runtimes/
local/
runtime.sqlite
- artifacts/
+ work/
worker/
programs/
+ results/
+ <build-id>/
+ result.json
+ files/
+ values/
```
-`check` 和 `plan` 不创建 Runtime 数据。SQLite 保存 Build 事实、Operation 与队列;产物字节进入
-Artifact Store;密钥只留在 Credential Store。
+`check` 和 `plan` 不创建 Runtime 数据。SQLite 只保存活跃 Build 的执行事实、Provider
+Operation 与队列状态;每个活跃 Build 有独立的临时字节目录。在普通的完整、失败或取消路径上,
+Result 安全写完后,Definition/Facts、Operation 载荷、Catalog 和临时字节都会删除。如果失败的
+正是 Result 写入本身,则保留执行状态供本地修复。Runtime 可以留一条很小的终态调度回执供
+`status` 使用,但它不是历史权威。
+
+历史内容属于项目:`.hypit/results/<build-id>/result.json` 记录最终 Target,以及这条执行路线
+上真正完成的所有公开 Author Output。媒体放在该 Result 的 `files/`,结构化值放在 `values/`。
+`builds`、`history`、`inspect`、`get` 和 `build-record` 都直接读取这些 Result,不依赖 Runtime
+SQLite,也不需要选择历史产物存储。
Workspace 独立由显式 `--workspace`、Runtime 指针所在项目或入口 Source 目录确定。Runtime 配置
不能扩大源码读取范围。
@@ -93,7 +102,7 @@ hypit programs down
```
`programs up` 会先准备所选 Endpoint 包的上游 npm 依赖,再操作其声明的程序。这些命令不打开
-SQLite、Artifact Store 或 Credential Store。
+SQLite 或 Credential Store。
## 生命周期
@@ -107,8 +116,8 @@ hypit runtime down
`runtime up` 先让 npm 把所选 Adapter 的精确上游包准备到机器共享目录,再准备 Managed
Program 并启动本地 Worker。`build` 不做部署:它执行便宜只读预检,只在就绪后提交,并确保
-Worker 可用。`status`、`queue`、`cancel` 用来观察和控制耐久工作。终态 Build 不恢复;复用
-以前的结果必须在新的 `.svrun` 中显式写 Candidate。
+Worker 可用。`status`、`queue`、`cancel` 用来观察和控制活跃工作。终态 Build 不恢复;复用
+以前的 Result 必须在新的 `.svrun` 中显式写 Candidate。
运行中的 Worker 会在 Build 第一次需要某个已安装 Component 包时加载它。后续 Build 可以在
不重启 Worker 的情况下新增 Component 包;Worker 会沿完整依赖闭包只加载尚未见过的部分。
diff --git a/docs/zh/quickstart/run.md b/docs/zh/quickstart/run.md
index 4d0cb142..a52562aa 100644
--- a/docs/zh/quickstart/run.md
+++ b/docs/zh/quickstart/run.md
@@ -5,7 +5,7 @@ description: 声明 Build 目标、复用结果以及配置运行时环境。
# Run Source 与 Build
-Author Source 定义视频本身。Run Source 从中挑选要产出哪些公开输出,以及是否用明确的 Candidate 来满足它们。Runtime Profile 则选择执行这份计划的机器、Store 与 Provider endpoint。
+Author Source 定义视频本身。Run Source 从中挑选最终目标,以及是否用明确的 Candidate 来满足它们。Runtime Profile 则选择执行这份计划的机器、凭据、Provider Endpoint 与服务。
先为项目选择一次 Runtime:
@@ -60,7 +60,7 @@ Run Source 与 Runtime Profile 不会悄悄改写视频。创作性的模型选
### Target
-Target 就是你希望这次 Build 产出的东西。它可以是一张生成的图片、一个视频镜头、一份时序映射、一条 Track,或者最终成片。编译器只执行选定 Target 所需的 Operation;无关的分支不会被触碰。
+Target 表达这次 Build 的最终意图,通常是成片或另一个真正的交付物;它不是“要保存哪些东西”的列表。编译器只执行通向 Target 的路线,而这条路上真正完成的每个公开 Author Output 都会自动进入同一个 Build Result。内部 Operation 值不进入 Result。
### 多个 Target
@@ -71,13 +71,13 @@ Target 就是你希望这次 Build 产出的东西。它可以是一张生成的
<target output="captions.track"/>
```
-不同的运行意图写成不同的 `.svrun` 文件即可,它们可以共同指向同一个 Author Source。例如 `images.svrun` 只请求图片,`film.svrun` 请求最终视频,无需在文件内部再造一层集合。
+只有一次执行确实存在多个最终目标时才写多个 Target。不同运行意图写成不同的 `.svrun` 文件即可,它们可以共同指向同一个 Author Source。
## 复用结果
-Hypit 没有隐式缓存。复用结果是显式的运行图编写——你将历史 Record 声明为零输入 Candidate,并通过 Satisfaction 边将它们连接起来。
+Hypit 没有隐式缓存。复用结果是显式的运行图编写:把某个旧 Build Result 里的一个具名 Output 声明为零输入 Candidate,再通过 Satisfaction 边连接到当前输出。
-生成图片或 Take 一经验收,就在下一份 `.svrun` 中用 `build-record` 与 `satisfy` 显式复用,并在启动付费下游工作前检查冻结 plan。Core 没有 Pin 状态或 fidelity 标签。
+生成图片或 Take 一完成,就能在下一份 `.svrun` 中用 `build-record` 与 `satisfy` 显式复用,并在启动付费下游工作前检查 plan。
```svml
<?svml using="@hypit/run-markup@1"?>
@@ -104,21 +104,19 @@ Hypit 没有隐式缓存。复用结果是显式的运行图编写——你将
### 查找可复用输出
-按照输出在各个历史 Build 冻结 Catalog 中的旧名字查询:
+按照输出在项目本地 Build Result 中的名字查询:
```bash
hypit history hook-take.video
```
-`history` 只列出该 Build 确实产出过的公开 Logical Output。仅仅在源码中声明但没有运行出来的别名,以及不能作为 `build-record` Candidate 的 authored Record 别名,都不会混入结果。如果忘了旧名字,可以按 Catalog 当时记录的精确源码路径列出真正验收过的输出名:
+`history` 只列出该 Build 确实完成的公开 Author Output。只声明但没有运行出来的输出和内部 Operation 值不会混入结果。如果忘了旧名字,可以按 Result 里记录的精确 Author Source 路径查询:
```bash
hypit history --source ./main.svml
```
-输出名只是某个不可变历史 Catalog 内供人查找的名字,不是产物身份。真正身份由历史 Core
-Build、Logical Output 和 Record 摘要共同确定。假如当前源码把 `hook-take.video` 改名为
-`opening-shot.video`,`<build-record>` 仍写历史旧名,`<satisfy>` 写当前新名:
+输出名是某个 Build Result 内供人查找的名字;`build + output` 这对地址已经足够精确。假如当前源码把 `hook-take.video` 改名为 `opening-shot.video`,`<build-record>` 仍写历史旧名,`<satisfy>` 写当前新名:
```svml
<build-record id="approved-opening"
@@ -126,13 +124,11 @@ Build、Logical Output 和 Record 摘要共同确定。假如当前源码把 `ho
<satisfy output="opening-shot.video" candidate="approved-opening"/>
```
-Hypit 永远不会猜测两个名字代表同一份作者意图。每次执行 `build` 都会得到一个新的
-Build id,由 CLI 打印并由 Runtime 归档。源码身份绝不会重新认领旧 Build;后续 Run 只有在
-这里明确写出历史 Build id 时,才会复用它已经接受的结果。
+Hypit 永远不会猜测两个名字代表同一份作者意图。每次执行 `build` 都会得到新的 Build id 和独立 Result 目录,即使源码完全没变。后续 Run 只有明确写出旧 Build id 与 Output 时才复用;若旧 Output 本身继续转发到更老的 Result,就沿显式关系向前解析,不把文件复制进新 Result。
### build-record
-声明一个由先前 Build 的历史 Record 支持的零输入 Candidate:
+声明一个由先前 Build Result 的具名 Output 支持的零输入 Candidate:
| 属性 | 说明 |
|---|---|
@@ -162,7 +158,7 @@ Core 不再给 Candidate 标注 `exact` 或 `substitute`。选择 Candidate 本
<satisfy output="opening-shot.video" candidate="approved-opening"/>
```
-文件相对于 `.svrun` 读取,按内容寻址,并随 Build 归档。系统没有额外的 Pin 状态、文件名缓存或隐式历史查找;黑场、预览图与人工交付的产物也使用同一个机制。
+文件相对于 `.svrun` 读取。如果它在 Target 路线上成为已经完成的公开 Output,就像生成媒体一样写入 Build Result。系统没有隐式历史查找;黑场、预览图与人工交付的产物也使用同一个机制。
## Runtime Profile
@@ -251,6 +247,9 @@ Runtime Profile 无权改变这条源码边界。`--package-root` 只定位已
output/
```
+每次 Build 的权威结果位于 `.hypit/results/<build-id>/`:`result.json` 记录名字、状态、Target
+和公开 Output,媒体在 `files/`,结构化值在 `values/`。
+
`status`、`builds` 等只读归档命令不会在状态尚不存在时初始化 Runtime 数据库。
共享只读素材库不必复制进项目,也不必放宽 Source 边界:
@@ -310,7 +309,7 @@ status` 用于观察,`programs up|status|down` 只管理外部程序。
### 4. 提交 Build
```bash
-hypit build build.svrun --follow
+hypit build build.svrun --name first-cut --follow
```
不带 `--follow` 时,Build 在耐久提交后退出,后台 Worker 继续。带 `--follow` 时终端也只是观察者,并会报告 phase / Operation 数量变化;Ctrl-C 不会取消任务。
@@ -328,11 +327,12 @@ hypit status <build-id> --watch
| `--runtime` | 单次命令的 Runtime Profile 覆盖;通常用 `runtime use` 选择一次即可 |
| `--package-root` | 存放已安装包的 Host 目录 |
| `--workspace` | 显式 Source Workspace 覆盖项 |
+| `--name` | 给这次独立 Build 一个供人阅读的名字 |
| `--follow` | 将 Build 进度流式输出到终端 |
每次执行都会创建新的 Build id,即使 Author Source 和 Run Source 完全没变。这是非确定性生成
所要求的边界:跨 Build 复用只能由 Run Source 里的显式 Candidate 决定。一次 Build 提交后
-具有耐久性;Worker 重启会继续它已经接受的 Record 和同一外部任务的 checkpoint,但不会让
+处于活跃状态时具有耐久性;Worker 重启会继续它已经接受的执行事实和同一外部任务,但不会让
另一次命令变成这个 Build。
### 5. 检查并获取结果
@@ -341,7 +341,7 @@ hypit status <build-id> --watch
hypit inspect <build-id>
```
-`inspect` 会显示耐久 Build 状态、所需输出与已接受的 Record。确认这些事实正确后,再获取所选归档 Artifact:
+`inspect` 直接读取项目里的 Build Result,列出最终 Target,以及通向它们的路线上实际完成的所有公开 Output:
```bash
hypit get <build-id> \
@@ -349,14 +349,13 @@ hypit get <build-id> \
--to examples/talking-head-aroll/output/final.mp4
```
-每个被接受的中间 Record 和 Artifact 在 Build 完成前都会被归档。`get` 会复制一份已持久化的
-Record。Blob Artifact 会从所选 Store 流式读取,逐步校验长度与 SHA-256,完整通过后才原子替换目标路径;导出大 MP4 不会把整段视频塞进 CLI 内存。
-Build 的最终输出会为每个目标别名打印精确的 `get --name …` 命令,不必为了导出
-`final.video` 去查不透明的 Record id。
+`get` 解析一个精确的 `build + output` 地址并按需复制。文件从所属 Result 复制,结构化 Output
+从 Result 的值文件读取,历史转发则沿显式关系找到前一个 Result;整个过程不需要 Runtime Profile。
+Build 的最终输出会为每个文件 Target 打印精确的 `get --name …` 命令。
### 6. 在新 Build 中复用
-创建一个引用已完成 Build 的 Record 的新 `.svrun` 文件(参见上文 [复用结果](#复用结果)),然后提交:
+创建一个引用已完成 Build Output 的新 `.svrun` 文件(参见上文 [复用结果](#复用结果)),然后提交:
```bash
hypit build reuse-generated.svrun --follow
diff --git a/package.json b/package.json
index 8e6608b6..bb9f8e1b 100644
--- a/package.json
+++ b/package.json
@@ -61,6 +61,7 @@
"@hypit/artifact": "workspace:*",
"@hypit/artifact-store-fs": "workspace:*",
"@hypit/artifact-store-s3": "workspace:*",
+ "@hypit/build-result": "workspace:*",
"@hypit/audio-track": "workspace:*",
"@hypit/audio-track-studio": "workspace:*",
"@hypit/background-removal": "workspace:*",
diff --git a/packages/artifact-store-fs/README.md b/packages/artifact-store-fs/README.md
index 264c3f01..961ae25e 100644
--- a/packages/artifact-store-fs/README.md
+++ b/packages/artifact-store-fs/README.md
@@ -1,7 +1,8 @@
# `@hypit/artifact-store-fs`
-Filesystem implementation of the Runtime `ArtifactStore` port. It keeps generated bytes under a
-content-addressed path and exposes streaming reads, writes and explicit garbage collection.
+Filesystem implementation of the Runtime's transient byte port. The local Runtime gives each Build
+its own working directory and removes it after the project Build Result has accepted the public
+Outputs.
-The Runtime Profile selects this package and supplies its path. It stores no Build state, credentials or
-author source. Code embedding the Runtime directly may construct `FileArtifactStore` itself.
+The Runtime Profile does not select this package or its path. It stores no Build Result, credentials
+or author source. Code embedding the Runtime directly may still construct `FileArtifactStore`.
diff --git a/packages/artifact-store-fs/src/store.ts b/packages/artifact-store-fs/src/store.ts
index 096613bf..a4c9c280 100644
--- a/packages/artifact-store-fs/src/store.ts
+++ b/packages/artifact-store-fs/src/store.ts
@@ -41,7 +41,8 @@ export class FileArtifactStore implements ArtifactStore {
if (mediaType.trim().length === 0) throw new Error("Artifact mediaType must not be empty");
const incoming = join(this.root, ".incoming");
await mkdir(incoming, { recursive: true });
- const temporary = join(incoming, randomUUID());
+ const resource = `res_${randomUUID()}`;
+ const temporary = join(incoming, resource);
const handle = await openFile(temporary, "wx");
const hash = createHash("sha256");
let size = 0;
@@ -78,7 +79,7 @@ export class FileArtifactStore implements ArtifactStore {
}
}
await rm(temporary, { force: true });
- return { kind: "blob", digest, size, mediaType };
+ return { kind: "blob", resource, digest, size, mediaType };
}
async get(digest: Digest): Promise<Uint8Array | undefined> {
diff --git a/packages/artifact-store-s3/src/store.ts b/packages/artifact-store-s3/src/store.ts
index 20366073..f0a2b97c 100644
--- a/packages/artifact-store-s3/src/store.ts
+++ b/packages/artifact-store-s3/src/store.ts
@@ -127,7 +127,7 @@ export class S3ArtifactStore implements ArtifactStore {
: { ExpectedBucketOwner: this.#expectedBucketOwner }),
} as const;
await this.#client.put(input);
- return { kind: "blob", digest, size: copy.byteLength, mediaType };
+ return { kind: "blob", resource: `res_${randomUUID()}`, digest, size: copy.byteLength, mediaType };
}
async get(digest: Digest): Promise<Uint8Array | undefined> {
@@ -256,7 +256,7 @@ export class S3ArtifactStore implements ArtifactStore {
} finally {
await client.delete?.({ Bucket: this.#bucket, Key: staging, ...owner }).catch(() => undefined);
}
- return { kind: "blob", digest, size, mediaType };
+ return { kind: "blob", resource: `res_${randomUUID()}`, digest, size, mediaType };
}
}
diff --git a/packages/build-result/package.json b/packages/build-result/package.json
new file mode 100644
index 00000000..34660ee8
--- /dev/null
+++ b/packages/build-result/package.json
@@ -0,0 +1,13 @@
+{
+ "name": "@hypit/build-result",
+ "version": "0.0.0-dev",
+ "license": "SEE LICENSE IN LICENSE",
+ "private": true,
+ "type": "module",
+ "exports": {
+ ".": "./src/index.ts"
+ },
+ "dependencies": {
+ "@hypit/protocol": "workspace:*"
+ }
+}
diff --git a/packages/build-result/src/index.ts b/packages/build-result/src/index.ts
new file mode 100644
index 00000000..f8b40482
--- /dev/null
+++ b/packages/build-result/src/index.ts
@@ -0,0 +1,9 @@
+export {
+ FileBuildResult,
+ buildResultDirectory,
+ listBuildResults,
+ materializeBuildResultOutput,
+ readBuildResult,
+ resolveBuildResultOutput,
+} from "./store.js";
+export type * from "./types.js";
diff --git a/packages/build-result/src/store.ts b/packages/build-result/src/store.ts
new file mode 100644
index 00000000..9ec1b60c
--- /dev/null
+++ b/packages/build-result/src/store.ts
@@ -0,0 +1,425 @@
+import { randomUUID } from "node:crypto";
+import {
+ copyFile,
+ mkdir,
+ open,
+ readFile,
+ readdir,
+ rename,
+ rm,
+ stat,
+ writeFile,
+} from "node:fs/promises";
+import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
+
+import type { BlobRef, BuildState, StoredValue, TypedRecord } from "@hypit/protocol";
+
+import type {
+ BuildResultArtifactSource,
+ BuildResultFileRef,
+ BuildResultFinish,
+ BuildResultJsonValue,
+ BuildResultManifest,
+ BuildResultOutput,
+ BuildResultSeed,
+ BuildResultSync,
+ HistoricalBuildOutputRef,
+ ResolvedBuildResultOutput,
+} from "./types.js";
+
+type WriterState = {
+ readonly resources: Readonly<Record<string, string>>;
+};
+
+const manifestName = "result.json";
+const writerStateName = ".writer.json";
+
+function assert(condition: unknown, message: string): asserts condition {
+ if (!condition) throw new Error(message);
+}
+
+function safeName(value: string): string {
+ const safe = value.trim().replace(/[^a-zA-Z0-9._-]+/gu, "-").replace(/^-+|-+$/gu, "");
+ return safe.length === 0 ? "output" : safe;
+}
+
+function valueFileName(output: string): string {
+ return `${encodeURIComponent(output)}.json`;
+}
+
+function mediaExtension(mediaType: string): string {
+ const subtype = mediaType.split("/", 2)[1]?.split(";", 1)[0]?.trim().toLowerCase();
+ if (subtype === undefined || subtype.length === 0) return ".bin";
+ const conventional = subtype === "jpeg" ? "jpg" : subtype === "x-wav" ? "wav" : subtype;
+ const suffix = conventional.includes("+") ? conventional.slice(0, conventional.indexOf("+")) : conventional;
+ const safe = suffix.replace(/[^a-z0-9]+/gu, "-").replace(/^-+|-+$/gu, "");
+ return safe.length === 0 ? ".bin" : `.${safe}`;
+}
+
+function isBlobRef(value: unknown): value is BlobRef {
+ if (value === null || Array.isArray(value) || typeof value !== "object") return false;
+ const item = value as Readonly<Record<string, unknown>>;
+ return item.kind === "blob"
+ && typeof item.digest === "string"
+ && typeof item.size === "number"
+ && Number.isSafeInteger(item.size)
+ && item.size >= 0
+ && typeof item.mediaType === "string"
+ && item.mediaType.length > 0;
+}
+
+function scalar(value: unknown): value is null | boolean | number | string {
+ return value === null || ["boolean", "number", "string"].includes(typeof value);
+}
+
+function resourceIdentity(artifact: BlobRef): string {
+ assert(artifact.resource !== undefined,
+ "Build resource has no instance identity; the producing Store must return BlobRef.resource");
+ return artifact.resource;
+}
+
+async function exists(path: string): Promise<boolean> {
+ return await stat(path).then((item) => item.isFile(), () => false);
+}
+
+async function writeJsonAtomic(path: string, value: unknown): Promise<void> {
+ await mkdir(dirname(path), { recursive: true });
+ const temporary = `${path}.part-${randomUUID()}`;
+ try {
+ await writeFile(temporary, `${JSON.stringify(value, null, 2)}\n`, { flag: "wx" });
+ await rename(temporary, path);
+ } catch (error) {
+ await rm(temporary, { force: true });
+ throw error;
+ }
+}
+
+async function copyArtifactAtomic(
+ source: BuildResultArtifactSource,
+ artifact: BlobRef,
+ destination: string,
+): Promise<void> {
+ if (await exists(destination)) return;
+ await mkdir(dirname(destination), { recursive: true });
+ const input = await source.open(artifact);
+ if (input === undefined) throw new Error(`Build resource ${resourceIdentity(artifact)} is unavailable`);
+ const temporary = `${destination}.part-${randomUUID()}`;
+ const output = await open(temporary, "wx");
+ try {
+ for await (const value of input) {
+ const chunk = Uint8Array.from(value);
+ await output.write(chunk);
+ }
+ await output.close();
+ await rename(temporary, destination);
+ } catch (error) {
+ await output.close().catch(() => undefined);
+ await rm(temporary, { force: true });
+ throw error;
+ }
+}
+
+function outputRecord(state: BuildState, output: string): TypedRecord | undefined {
+ const selection = state.plan.selections.find((item) => item.output === output);
+ if (selection === undefined) return undefined;
+ return state.records.find((item) => item.id === selection.record);
+}
+
+function outputCandidate(state: BuildState, output: string): string | undefined {
+ return state.plan.selections.find((item) => item.output === output)?.candidate;
+}
+
+export function buildResultDirectory(root: string, build: string): string {
+ assert(build.trim().length > 0 && !build.includes("/") && !build.includes("\\"), "Build id is not a directory name");
+ return join(resolve(root), build);
+}
+
+export async function readBuildResult(directory: string): Promise<BuildResultManifest | undefined> {
+ const path = join(resolve(directory), manifestName);
+ const text = await readFile(path, "utf8").catch((error: unknown) => {
+ if (error instanceof Error && "code" in error && error.code === "ENOENT") return undefined;
+ throw error;
+ });
+ if (text === undefined) return undefined;
+ const value = JSON.parse(text) as BuildResultManifest;
+ assert(value.format === "hypit.build-result@1", `${path} is not a Hypit Build Result`);
+ return value;
+}
+
+function containedResultPath(directory: string, path: string): string {
+ assert(!isAbsolute(path), `Build Result path ${path} must be relative`);
+ const root = resolve(directory);
+ const absolute = resolve(root, path);
+ const relation = relative(root, absolute);
+ assert(relation !== ".." && !relation.startsWith(`..${sep}`),
+ `Build Result path ${path} leaves ${root}`);
+ return absolute;
+}
+
+export async function listBuildResults(root: string): Promise<readonly BuildResultManifest[]> {
+ const directory = resolve(root);
+ const entries = await readdir(directory, { withFileTypes: true }).catch((error: unknown) => {
+ if (error instanceof Error && "code" in error && error.code === "ENOENT") return [];
+ throw error;
+ });
+ const manifests = await Promise.all(entries
+ .filter((entry) => entry.isDirectory())
+ .map(async (entry) => {
+ const manifest = await readBuildResult(join(directory, entry.name));
+ assert(manifest !== undefined, `${join(directory, entry.name)} has no result.json`);
+ return manifest;
+ }));
+ return manifests.sort((left, right) => right.startedAt - left.startedAt || left.id.localeCompare(right.id));
+}
+
+export async function resolveBuildResultOutput(
+ root: string,
+ build: string,
+ output: string,
+): Promise<ResolvedBuildResultOutput | undefined> {
+ const seen = new Set<string>();
+ let currentBuild = build;
+ let currentOutput = output;
+ while (true) {
+ const address = `${currentBuild}\u0000${currentOutput}`;
+ assert(!seen.has(address), `Build Output forwarding repeats ${currentBuild} / ${currentOutput}`);
+ seen.add(address);
+ const directory = buildResultDirectory(root, currentBuild);
+ const manifest = await readBuildResult(directory);
+ if (manifest === undefined) return undefined;
+ const entry = manifest.outputs[currentOutput];
+ if (entry === undefined) return undefined;
+ if (entry.value.kind === "build-output") {
+ currentBuild = entry.value.build;
+ currentOutput = entry.value.output;
+ continue;
+ }
+ if (entry.value.kind === "json") {
+ const value = JSON.parse(await readFile(containedResultPath(directory, entry.value.path), "utf8")) as BuildResultJsonValue;
+ return {
+ build: currentBuild,
+ output: currentOutput,
+ directory,
+ type: entry.type,
+ value: { ...entry.value, value },
+ };
+ }
+ return {
+ build: currentBuild,
+ output: currentOutput,
+ directory,
+ type: entry.type,
+ value: entry.value,
+ };
+ }
+}
+
+export async function materializeBuildResultOutput(
+ root: string,
+ build: string,
+ output: string,
+ destination: string,
+): Promise<{
+ readonly build: string;
+ readonly output: string;
+ readonly path: string;
+ readonly kind: "file" | "json" | "inline";
+}> {
+ const resolvedOutput = await resolveBuildResultOutput(root, build, output);
+ if (resolvedOutput === undefined) throw new Error(`Build ${build} has no Output ${output}`);
+ const target = resolve(destination);
+ await mkdir(dirname(target), { recursive: true });
+ const temporary = `${target}.part-${randomUUID()}`;
+ try {
+ if (resolvedOutput.value.kind === "inline") {
+ await writeFile(temporary, `${JSON.stringify(resolvedOutput.value.value, null, 2)}\n`, { flag: "wx" });
+ } else {
+ const sourceDirectory = resolvedOutput.value.kind === "build-file" && resolvedOutput.value.build !== undefined
+ ? buildResultDirectory(root, resolvedOutput.value.build)
+ : resolvedOutput.directory;
+ await copyFile(containedResultPath(sourceDirectory, resolvedOutput.value.path), temporary);
+ }
+ await rename(temporary, target);
+ } catch (error) {
+ await rm(temporary, { force: true });
+ throw error;
+ }
+ return {
+ build: resolvedOutput.build,
+ output: resolvedOutput.output,
+ path: target,
+ kind: resolvedOutput.value.kind === "build-file"
+ ? "file"
+ : resolvedOutput.value.kind === "json" ? "json" : "inline",
+ };
+}
+
+export class FileBuildResult {
+ readonly directory: string;
+
+ private constructor(directory: string) {
+ this.directory = resolve(directory);
+ }
+
+ static async create(root: string, seed: BuildResultSeed): Promise<FileBuildResult> {
+ const directory = buildResultDirectory(root, seed.id);
+ assert(seed.name === undefined || seed.name.trim().length > 0, "Build Result name must not be empty");
+ await mkdir(directory, { recursive: true });
+ const existing = await readBuildResult(directory);
+ if (existing !== undefined) throw new Error(`Build Result ${seed.id} already exists`);
+ const now = seed.startedAt ?? Date.now();
+ const manifest: BuildResultManifest = {
+ format: "hypit.build-result@1",
+ id: seed.id,
+ ...(seed.name === undefined ? {} : { name: seed.name }),
+ source: seed.source,
+ ...(seed.run === undefined ? {} : { run: seed.run }),
+ targets: [...seed.targets],
+ startedAt: now,
+ updatedAt: now,
+ status: "running",
+ outputs: {},
+ };
+ await writeJsonAtomic(join(directory, manifestName), manifest);
+ await writeJsonAtomic(join(directory, writerStateName), {
+ resources: {},
+ aliases: seed.aliases,
+ reuses: seed.reuses ?? [],
+ });
+ return new FileBuildResult(directory);
+ }
+
+ static async open(directory: string): Promise<FileBuildResult> {
+ const absolute = resolve(directory);
+ const manifest = await readBuildResult(absolute);
+ if (manifest === undefined) throw new Error(`${absolute} has no Build Result`);
+ return new FileBuildResult(absolute);
+ }
+
+ async read(): Promise<BuildResultManifest> {
+ const manifest = await readBuildResult(this.directory);
+ assert(manifest !== undefined, `${this.directory} has no Build Result`);
+ return manifest;
+ }
+
+ async sync(input: BuildResultSync): Promise<BuildResultManifest> {
+ const manifest = await this.read();
+ if (manifest.status !== "running") return manifest;
+ const rawWriter = await readFile(join(this.directory, writerStateName), "utf8");
+ const writer = JSON.parse(rawWriter) as WriterState & {
+ readonly aliases: readonly { readonly name: string; readonly output: string }[];
+ readonly reuses: readonly { readonly candidate: string; readonly build: string; readonly output: string }[];
+ };
+ const resources = new Map(Object.entries(writer.resources));
+ const outputs: Record<string, BuildResultOutput> = { ...manifest.outputs };
+ const materialize = async (
+ artifact: BlobRef,
+ preferredName: string,
+ ): Promise<BuildResultFileRef> => {
+ if (artifact.origin !== undefined) {
+ return {
+ kind: "build-file",
+ build: artifact.origin.build,
+ path: artifact.origin.path,
+ size: artifact.size,
+ mediaType: artifact.mediaType,
+ };
+ }
+ const identity = resourceIdentity(artifact);
+ let relativePath = resources.get(identity);
+ if (relativePath === undefined) {
+ const base = safeName(preferredName);
+ let candidate = join("files", `${base}${mediaExtension(artifact.mediaType)}`);
+ let counter = 2;
+ const occupied = new Set(resources.values());
+ while (occupied.has(candidate)) {
+ candidate = join("files", `${base}-${counter}${mediaExtension(artifact.mediaType)}`);
+ counter += 1;
+ }
+ relativePath = candidate;
+ resources.set(identity, relativePath);
+ }
+ await copyArtifactAtomic(input.artifacts, artifact, join(this.directory, relativePath));
+ return { kind: "build-file", path: relativePath, size: artifact.size, mediaType: artifact.mediaType };
+ };
+ const convert = async (value: unknown, preferredName: string): Promise<BuildResultJsonValue> => {
+ if (isBlobRef(value)) return await materialize(value, preferredName);
+ if (scalar(value)) return value;
+ if (Array.isArray(value)) {
+ return await Promise.all(value.map(async (item, index) => await convert(item, `${preferredName}-${index + 1}`)));
+ }
+ assert(typeof value === "object", `Build Output ${preferredName} contains a non-canonical value`);
+ return Object.fromEntries(await Promise.all(Object.entries(value as Readonly<Record<string, unknown>>)
+ .map(async ([key, item]) => [key, await convert(item, `${preferredName}-${key}`)] as const)));
+ };
+ const outputValue = async (name: string, value: StoredValue): Promise<BuildResultOutput["value"]> => {
+ if (value.kind === "blob") return await materialize(value, name);
+ if (scalar(value.value)) return { kind: "inline", value: value.value };
+ const relativePath = join("values", valueFileName(name));
+ await writeJsonAtomic(join(this.directory, relativePath), await convert(value.value, name));
+ return { kind: "json", path: relativePath };
+ };
+ const reuses = new Map(writer.reuses.map((item) => [item.candidate, item]));
+ const outputRank = (record: TypedRecord, reused: { readonly build: string; readonly output: string } | undefined): number => reused !== undefined
+ ? 0
+ : record.value.kind === "blob" ? 1 : scalar(record.value.value) ? 2 : 3;
+ const ready = writer.aliases.flatMap((alias) => {
+ const record = outputRecord(input.state, alias.output);
+ if (record === undefined) return [];
+ const reused = reuses.get(outputCandidate(input.state, alias.output) ?? "");
+ return [{ alias, record, reused }];
+ }).sort((left, right) => {
+ return outputRank(left.record, left.reused) - outputRank(right.record, right.reused)
+ || left.alias.name.localeCompare(right.alias.name);
+ });
+ for (const { alias, record, reused } of ready) {
+ const value: BuildResultOutput["value"] = reused === undefined
+ ? await outputValue(alias.name, record.value)
+ : {
+ kind: "build-output",
+ build: reused.build,
+ output: reused.output,
+ } satisfies HistoricalBuildOutputRef;
+ outputs[alias.name] = { type: record.type, value };
+ }
+ const now = Date.now();
+ const status = input.state.status === "complete"
+ ? "complete"
+ : input.state.status === "failed" ? "failed" : manifest.status;
+ const updated: BuildResultManifest = {
+ ...manifest,
+ updatedAt: now,
+ status,
+ ...(status === "running" ? {} : { finishedAt: manifest.finishedAt ?? now }),
+ outputs,
+ };
+ await writeJsonAtomic(join(this.directory, writerStateName), {
+ ...writer,
+ resources: Object.fromEntries(resources),
+ });
+ await writeJsonAtomic(join(this.directory, manifestName), updated);
+ if (status !== "running") await rm(join(this.directory, writerStateName), { force: true });
+ return updated;
+ }
+
+ async finish(input: BuildResultFinish): Promise<BuildResultManifest> {
+ const manifest = await this.read();
+ if (manifest.status !== "running") {
+ if (manifest.status !== input.status || input.failure === undefined || manifest.failure !== undefined) return manifest;
+ const updated = { ...manifest, failure: input.failure, updatedAt: Date.now() };
+ await writeJsonAtomic(join(this.directory, manifestName), updated);
+ return updated;
+ }
+ const now = Date.now();
+ const updated: BuildResultManifest = {
+ ...manifest,
+ status: input.status,
+ updatedAt: now,
+ finishedAt: manifest.finishedAt ?? now,
+ ...(input.failure === undefined ? {} : { failure: input.failure }),
+ };
+ await writeJsonAtomic(join(this.directory, manifestName), updated);
+ await rm(join(this.directory, writerStateName), { force: true });
+ return updated;
+ }
+}
diff --git a/packages/build-result/src/types.ts b/packages/build-result/src/types.ts
new file mode 100644
index 00000000..912a4788
--- /dev/null
+++ b/packages/build-result/src/types.ts
@@ -0,0 +1,108 @@
+import type {
+ BlobRef,
+ BuildState,
+ TypeRef,
+} from "@hypit/protocol";
+
+export type BuildResultStatus = "running" | "complete" | "failed" | "cancelled";
+
+export type BuildResultFileRef = {
+ readonly kind: "build-file";
+ /** Omitted for this Build; present for a directly carried historical file. */
+ readonly build?: string;
+ /** Forward-only path relative to the owning Build's result directory. */
+ readonly path: string;
+ readonly size: number;
+ readonly mediaType: string;
+};
+
+export type HistoricalBuildOutputRef = {
+ readonly kind: "build-output";
+ readonly build: string;
+ readonly output: string;
+};
+
+export type BuildResultJsonValue =
+ | null
+ | boolean
+ | number
+ | string
+ | readonly BuildResultJsonValue[]
+ | BuildResultFileRef
+ | HistoricalBuildOutputRef
+ | { readonly [key: string]: BuildResultJsonValue };
+
+export type BuildResultOutputValue =
+ | BuildResultFileRef
+ | HistoricalBuildOutputRef
+ | { readonly kind: "inline"; readonly value: null | boolean | number | string }
+ | { readonly kind: "json"; readonly path: string };
+
+export type BuildResultOutput = {
+ readonly type: TypeRef;
+ readonly value: BuildResultOutputValue;
+};
+
+export type BuildResultManifest = {
+ readonly format: "hypit.build-result@1";
+ readonly id: string;
+ readonly name?: string;
+ readonly source: { readonly path: string };
+ readonly run?: { readonly path: string };
+ /** Author-facing names of the Build's actual final goals. */
+ readonly targets: readonly string[];
+ readonly startedAt: number;
+ readonly updatedAt: number;
+ readonly finishedAt?: number;
+ readonly status: BuildResultStatus;
+ readonly failure?: string;
+ readonly outputs: Readonly<Record<string, BuildResultOutput>>;
+};
+
+export type BuildResultAlias = {
+ readonly name: string;
+ readonly output: string;
+};
+
+export type BuildResultReuse = {
+ /** Resolved Candidate id stored in the Build Plan. */
+ readonly candidate: string;
+ readonly build: string;
+ readonly output: string;
+};
+
+export type BuildResultSeed = {
+ readonly id: string;
+ readonly name?: string;
+ readonly source: { readonly path: string };
+ readonly run?: { readonly path: string };
+ readonly targets: readonly string[];
+ readonly aliases: readonly BuildResultAlias[];
+ readonly reuses?: readonly BuildResultReuse[];
+ readonly startedAt?: number;
+};
+
+export type BuildResultArtifactSource = {
+ open(artifact: BlobRef): Promise<AsyncIterable<Uint8Array> | undefined>;
+};
+
+export type BuildResultSync = {
+ readonly state: BuildState;
+ readonly artifacts: BuildResultArtifactSource;
+};
+
+export type BuildResultFinish = {
+ readonly status: Exclude<BuildResultStatus, "running">;
+ readonly failure?: string;
+};
+
+export type ResolvedBuildResultOutput = {
+ readonly build: string;
+ readonly output: string;
+ readonly directory: string;
+ readonly type: TypeRef;
+ readonly value:
+ | BuildResultFileRef
+ | { readonly kind: "inline"; readonly value: null | boolean | number | string }
+ | { readonly kind: "json"; readonly path: string; readonly value: BuildResultJsonValue };
+};
diff --git a/packages/build-result/test/result.test.ts b/packages/build-result/test/result.test.ts
new file mode 100644
index 00000000..adcd997f
--- /dev/null
+++ b/packages/build-result/test/result.test.ts
@@ -0,0 +1,141 @@
+import assert from "node:assert/strict";
+import { mkdtemp, readFile, readdir, rm } from "node:fs/promises";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import test from "node:test";
+
+import { FileBuildResult } from "@hypit/build-result";
+import type { BlobRef, BuildState, TypeRef } from "@hypit/protocol";
+
+const videoType: TypeRef = {
+ module: { name: "example.media", version: "1" },
+ name: "Video",
+};
+const takeType: TypeRef = {
+ module: { name: "example.speech", version: "1" },
+ name: "SemanticTake",
+};
+
+function state(input: {
+ readonly status?: BuildState["status"];
+ readonly records: BuildState["records"];
+ readonly selections: BuildState["plan"]["selections"];
+}): BuildState {
+ return {
+ status: input.status ?? "active",
+ records: input.records,
+ plan: { selections: input.selections },
+ } as unknown as BuildState;
+}
+
+test("one same-Build resource backs a public video and a SemanticTake payload", async () => {
+ const root = await mkdtemp(join(tmpdir(), "hypit-build-result-"));
+ const bytes = new Uint8Array([1, 2, 3, 4]);
+ const video: BlobRef = {
+ kind: "blob",
+ resource: "res_same-build-video",
+ digest: `sha256:${"1".repeat(64)}`,
+ size: bytes.byteLength,
+ mediaType: "video/mp4",
+ };
+ try {
+ const result = await FileBuildResult.create(root, {
+ id: "bld_one",
+ source: { path: "/project/main.svml" },
+ run: { path: "/project/build.svrun" },
+ targets: ["final.video"],
+ aliases: [
+ { name: "shot.video", output: "logical:shot-video" },
+ { name: "shot.take", output: "logical:shot-take" },
+ { name: "unused.video", output: "logical:unused" },
+ ],
+ });
+ const manifest = await result.sync({
+ state: state({
+ records: [
+ { id: "record:video", type: videoType, value: video },
+ {
+ id: "record:take",
+ type: takeType,
+ value: { kind: "inline", value: { media: { visual: { artifact: video } } } },
+ },
+ ],
+ selections: [
+ { output: "logical:shot-video", candidate: "candidate:generated-video", record: "record:video" },
+ { output: "logical:shot-take", candidate: "candidate:aligned-take", record: "record:take" },
+ { output: "logical:unused", candidate: "candidate:unused", record: "record:unused" },
+ ],
+ }),
+ artifacts: {
+ async open(artifact) {
+ assert.equal(artifact.resource, video.resource);
+ return (async function* () { yield bytes; })();
+ },
+ },
+ });
+ assert.deepEqual(Object.keys(manifest.outputs).sort(), ["shot.take", "shot.video"]);
+ assert.equal(manifest.outputs["shot.video"]?.value.kind, "build-file");
+ assert.equal(manifest.outputs["shot.take"]?.value.kind, "json");
+ const videoPath = manifest.outputs["shot.video"]?.value.kind === "build-file"
+ ? manifest.outputs["shot.video"].value.path
+ : undefined;
+ const takePath = manifest.outputs["shot.take"]?.value.kind === "json"
+ ? manifest.outputs["shot.take"].value.path
+ : undefined;
+ assert.ok(videoPath);
+ assert.ok(takePath);
+ assert.equal(videoPath, "files/shot.video.mp4");
+ const take = JSON.parse(await readFile(join(result.directory, takePath), "utf8")) as {
+ readonly media: { readonly visual: { readonly artifact: { readonly path: string } } };
+ };
+ assert.equal(take.media.visual.artifact.path, videoPath);
+ assert.deepEqual(await readdir(join(result.directory, "files")), [videoPath.split("/").at(-1)]);
+
+ const failed = await result.finish({ status: "failed", failure: "final render failed" });
+ assert.equal(failed.status, "failed");
+ assert.equal(failed.failure, "final render failed");
+ assert.deepEqual(Object.keys(failed.outputs).sort(), ["shot.take", "shot.video"]);
+ } finally {
+ await rm(root, { recursive: true, force: true });
+ }
+});
+
+test("an explicitly reused public output is a forward reference and copies no bytes", async () => {
+ const root = await mkdtemp(join(tmpdir(), "hypit-build-result-reuse-"));
+ try {
+ const result = await FileBuildResult.create(root, {
+ id: "bld_new",
+ source: { path: "/project/main.svml" },
+ targets: ["final.video"],
+ aliases: [{ name: "shot.video", output: "logical:shot-video" }],
+ reuses: [{ candidate: "candidate:prior", build: "bld_old", output: "opening.video" }],
+ });
+ const manifest = await result.sync({
+ state: state({
+ records: [{
+ id: "record:prior",
+ type: videoType,
+ value: {
+ kind: "blob",
+ digest: `sha256:${"2".repeat(64)}`,
+ size: 10,
+ mediaType: "video/mp4",
+ },
+ }],
+ selections: [{ output: "logical:shot-video", candidate: "candidate:prior", record: "record:prior" }],
+ }),
+ artifacts: {
+ async open() {
+ throw new Error("a forwarded output must not copy its historical bytes");
+ },
+ },
+ });
+ assert.deepEqual(manifest.outputs["shot.video"]?.value, {
+ kind: "build-output",
+ build: "bld_old",
+ output: "opening.video",
+ });
+ } finally {
+ await rm(root, { recursive: true, force: true });
+ }
+});
diff --git a/packages/cli/package.json b/packages/cli/package.json
index 88c15b36..651c1bea 100644
--- a/packages/cli/package.json
+++ b/packages/cli/package.json
@@ -8,6 +8,7 @@
".": "./src/index.ts"
},
"dependencies": {
+ "@hypit/build-result": "workspace:*",
"@hypit/compiler-node": "workspace:*",
"@hypit/elaborator": "workspace:*",
"@hypit/package-loader-node": "workspace:*",
@@ -18,5 +19,10 @@
"@hypit/source": "workspace:*",
"@hypit/workspace": "workspace:*",
"typescript": "5.9.3"
+ },
+ "devDependencies": {
+ "@hypit/markup": "workspace:*",
+ "@hypit/run-markup": "workspace:*",
+ "@hypit/workspace-fs-node": "workspace:*"
}
}
diff --git a/packages/cli/src/main.ts b/packages/cli/src/main.ts
index 839690b1..ea845cd3 100644
--- a/packages/cli/src/main.ts
+++ b/packages/cli/src/main.ts
@@ -1,10 +1,17 @@
import { createHash, randomBytes, randomUUID } from "node:crypto";
import { createServer } from "node:http";
import { spawn } from "node:child_process";
-import { basename, dirname, isAbsolute, relative, resolve, sep } from "node:path";
+import { basename, dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
import { mkdir, readFile, stat, writeFile } from "node:fs/promises";
import { deterministicSpeechDurations, deterministicSpeechDurationsFromGraph } from "@hypit/compiler-node";
+import {
+ buildResultDirectory,
+ listBuildResults,
+ materializeBuildResultOutput,
+ readBuildResult,
+ resolveBuildResultOutput,
+} from "@hypit/build-result";
import type { NodeCompiledSourceClosure } from "@hypit/compiler-node";
import type { NodeRuntimeHost } from "@hypit/runtime-host-node";
import {
@@ -18,14 +25,7 @@ import type { BuildState, CapabilityRef, TypeRef } from "@hypit/protocol";
import { parseSourceHeader } from "@hypit/source";
import {
- acceptedArchivedOutputs,
- collectArtifacts,
- findArchivedArtifact,
- inspectBuild,
- materializeArtifact,
- materializeRecord,
pinnedRecords,
- selectArchivedRecord,
summarizeBuildCatalog,
} from "./archive.js";
import { unreachedGenerations } from "./reachability.js";
@@ -38,7 +38,6 @@ import type {
CliManagedProgramReport,
CliRuntime,
CliRuntimeArchiveControl,
- CliRuntimeArtifactAccess,
CliRuntimeController,
} from "./runtime-port.js";
import { writeCliHelp, writeCliOutput } from "./output.js";
@@ -394,7 +393,7 @@ function parseArgs(argv: readonly string[]): ParsedArgs {
continue;
}
if (item === "--out") {
- throw new Error("--out was removed: Build always archives accepted Records; use `get <build-id> --to <path>` for an optional copy");
+ throw new Error("--out was removed: Build Results save completed public Outputs automatically; use `get <build-id> --name <output> --to <path>` for an optional copy");
}
if (item === "--record") {
const value = rest[index + 1];
@@ -412,7 +411,7 @@ function parseArgs(argv: readonly string[]): ParsedArgs {
}
if (item === "--name") {
const value = rest[index + 1];
- if (value === undefined || value.startsWith("--")) throw new Error("--name requires a source output name");
+ if (value === undefined || value.startsWith("--")) throw new Error("--name requires a name");
name = value;
index += 1;
continue;
@@ -580,7 +579,7 @@ function assertCommandOptions(args: ParsedArgs): void {
add("--runtime");
break;
case "get":
- add("--runtime", "--name", "--record", "--output", "--artifact", "--to");
+ add("--workspace", "--name", "--record", "--output", "--artifact", "--to");
break;
case "image":
// A package asset needs credentials and nothing else; no Runtime Profile applies.
@@ -595,17 +594,19 @@ function assertCommandOptions(args: ParsedArgs): void {
break;
case "builds":
case "history":
- case "inspect":
- add("--runtime");
+ add("--workspace");
if (args.command === "history") add("--source", "--pin", "--exclude-targets");
break;
+ case "inspect":
+ add("--workspace");
+ break;
case "check":
case "plan":
add("--runtime", "--package-root", "--workspace", "--asset-root");
break;
case "build":
add("--runtime", "--package-root", "--workspace", "--asset-root", "--follow",
- "--max-wait-ms");
+ "--max-wait-ms", "--name");
break;
}
const invalid = args.seenOptions.find((item) => !allowed.has(item));
@@ -633,12 +634,12 @@ function usage(): string {
" hypit queue [--runtime profile.json] [--watch]",
" hypit check <self-described-source> [--runtime profile.json] [--workspace workspace] [--asset-root directory]",
" hypit plan <run-source> [--runtime profile.json] [--workspace workspace] [--asset-root directory]",
- " hypit build <run-source> [--runtime profile.json] [--workspace workspace] [--asset-root directory] [--follow]",
+ " hypit build <run-source> [--name name] [--runtime profile.json] [--workspace workspace] [--asset-root directory] [--follow]",
" hypit status <build-id> [--runtime profile.json] [--watch]",
- " hypit builds [--runtime profile.json]",
- " hypit history [source-output-name] [--runtime profile.json] [--source author.svml] [--pin] [--exclude-targets]",
- " hypit inspect <build-id> [--runtime profile.json]",
- " hypit get <build-id> [--runtime profile.json] [--name source-name|--record record-id|--output logical-output-id|--artifact digest] [--to path]",
+ " hypit builds [--workspace project]",
+ " hypit history [source-output-name] [--workspace project] [--source author.svml] [--pin] [--exclude-targets]",
+ " hypit inspect <build-id> [--workspace project]",
+ " hypit get <build-id> --name output-name [--workspace project] [--to path]",
" hypit cancel <build-id> [--runtime profile.json] [--reason text]",
" hypit auth status|login|logout <endpoint-instance> [--runtime profile.json] [--slot name] [--from secret-file]",
" hypit image --prompt <text|text-file> --to <path.png> [--model package] [--aspect-ratio r] [--resolution r]",
@@ -731,12 +732,6 @@ async function loadRuntimeArchive(
return await host.openArchive({ readOnly });
}
-async function loadRuntimeArtifactAccess(
- host: NodeRuntimeHost,
-): Promise<CliRuntimeArtifactAccess> {
- return await host.openArtifacts();
-}
-
function displayType(type: TypeRef): string {
return `${type.module.name}@${type.module.version}/${type.name}`;
}
@@ -891,12 +886,12 @@ async function observeBuild(
}
if (observedActivity) {
const status = await runtime.status(current.id);
- if (status.build === undefined || status.dispatch === undefined) {
+ if (status.dispatch === undefined) {
throw new Error(`Build ${current.id} disappeared while it was being observed`);
}
current = {
id: current.id,
- state: status.build.state,
+ state: status.build?.state ?? current.state,
status: submissionStatus(status.dispatch),
dispatch: status.dispatch,
};
@@ -1424,8 +1419,124 @@ export async function runCli(
}
return;
}
- if (args.command === "status" || args.command === "builds" || args.command === "inspect"
- || args.command === "history" || args.command === "get" || args.command === "cancel" || args.command === "queue"
+ if (args.command === "builds" || args.command === "history" || args.command === "inspect" || args.command === "get") {
+ const resultsRoot = join(commandProjectRoot(), ".hypit", "results");
+ if (args.command === "builds") {
+ const manifests = await listBuildResults(resultsRoot);
+ const shown = manifests.slice(0, args.verbose ? undefined : 20);
+ const builds = shown.map((manifest) => ({
+ build: manifest.id,
+ ...(manifest.name === undefined ? {} : { name: manifest.name }),
+ status: manifest.status,
+ startedAt: manifest.startedAt,
+ source: manifest.source,
+ ...(manifest.run === undefined ? {} : { run: manifest.run }),
+ targets: manifest.targets,
+ outputs: Object.keys(manifest.outputs),
+ }));
+ writeOperational({ builds }, "Build results", "info", [["Builds", String(manifests.length)]],
+ builds.map((item) => {
+ const source = basename(item.run?.path ?? item.source.path);
+ const label = item.name === undefined ? item.build : `${item.name} · ${item.build}`;
+ return `${label}: ${item.status} · ${source} · ${item.outputs.length} outputs`;
+ }));
+ return;
+ }
+ if (args.command === "history") {
+ const manifests = await listBuildResults(resultsRoot);
+ const source = args.source === undefined ? undefined : resolve(args.source);
+ const entries = manifests.flatMap((manifest) => {
+ if (source !== undefined && manifest.source.path !== source) return [];
+ return Object.entries(manifest.outputs).flatMap(([name, output]) => {
+ if (args.file !== undefined && name !== args.file) return [];
+ if (args.excludeTargets && manifest.targets.includes(name)) return [];
+ return [{
+ build: manifest.id,
+ createdAt: manifest.startedAt,
+ status: manifest.status,
+ source: manifest.source,
+ ...(manifest.run === undefined ? {} : { run: manifest.run }),
+ output: { name, type: output.type, value: output.value },
+ }];
+ });
+ }).sort((left, right) => right.createdAt - left.createdAt
+ || left.output.name.localeCompare(right.output.name)
+ || left.build.localeCompare(right.build));
+ const pins = args.pin ? pinnedRecords(entries) : [];
+ const shown = entries.slice(0, args.verbose ? undefined : 20);
+ writeOperational({
+ query: {
+ ...(args.file === undefined ? {} : { output: args.file }),
+ ...(source === undefined ? {} : { source }),
+ },
+ entries,
+ ...(args.pin ? { pins } : {}),
+ }, entries.length === 0 ? "No Build Output history" : args.pin ? "Reuse these Outputs" : "Output history",
+ entries.length === 0 ? "warning" : "info", [
+ ...(args.file === undefined ? [] : [["Output", args.file] as const]),
+ ...(source === undefined ? [] : [["Source", source] as const]),
+ ["Outputs", String(entries.length)],
+ ], args.pin
+ ? ["Paste into a Run Source; every reference names one exact Build Output.", ...pins.flatMap((item) => item.markup)]
+ : shown.map((item) => {
+ const created = new Date(item.createdAt).toISOString();
+ return `${item.build}: ${item.output.name} · ${created} · ${item.output.value.kind}`;
+ }));
+ return;
+ }
+ if (args.command === "inspect") {
+ const manifest = await readBuildResult(buildResultDirectory(resultsRoot, args.file!));
+ if (manifest === undefined) throw new Error(`Build Result ${args.file} does not exist`);
+ const outputs = Object.entries(manifest.outputs).map(([name, output]) => ({
+ name,
+ target: manifest.targets.includes(name),
+ type: output.type,
+ value: output.value,
+ }));
+ writeOperational({ result: manifest, outputs }, "Build Result detail", manifest.status === "failed" ? "error" : "info", [
+ ["Build", manifest.id],
+ ...(manifest.name === undefined ? [] : [["Name", manifest.name] as const]),
+ ["Status", manifest.status],
+ ["Targets", String(manifest.targets.length)],
+ ["Outputs", String(outputs.length)],
+ ], [
+ ...(manifest.failure === undefined ? [] : [`Reason ${manifest.failure}`]),
+ ...outputs.map((item) => `${item.target ? "Target" : "Output"} ${item.name} · ${displayType(item.type)} · ${item.value.kind}`),
+ ]);
+ return;
+ }
+ if (args.record !== undefined || args.output !== undefined || args.artifact !== undefined) {
+ throw new Error("get now addresses Build Results by --name; Record, Logical Output id and Artifact digest selectors were removed");
+ }
+ const directory = buildResultDirectory(resultsRoot, args.file!);
+ const manifest = await readBuildResult(directory);
+ if (manifest === undefined) throw new Error(`Build Result ${args.file} does not exist`);
+ const available = Object.keys(manifest.outputs);
+ const name = args.name
+ ?? (manifest.targets.length === 1 && manifest.outputs[manifest.targets[0]!] !== undefined
+ ? manifest.targets[0]
+ : available.length === 1 ? available[0] : undefined);
+ if (name === undefined) throw new Error(`Build ${manifest.id} has several Outputs; select one with --name`);
+ const resolved = await resolveBuildResultOutput(resultsRoot, manifest.id, name);
+ if (resolved === undefined) throw new Error(`Build ${manifest.id} has no Output ${name}`);
+ if (args.to === undefined) {
+ writeOperational({ build: manifest.id, output: name, resolved }, "Build Output", "info", [
+ ["Build", manifest.id],
+ ["Output", name],
+ ["Type", displayType(resolved.type)],
+ ["Storage", resolved.value.kind],
+ ], resolved.build === manifest.id && resolved.output === name
+ ? []
+ : [`Forwards to ${resolved.build} / ${resolved.output}`]);
+ } else {
+ const materialized = await materializeBuildResultOutput(resultsRoot, manifest.id, name, args.to);
+ writeOperational({ build: manifest.id, output: name, resolved, materialized }, "Build Output materialized", "success", [
+ ["Build", manifest.id], ["Output", name], ["Path", materialized.path],
+ ]);
+ }
+ return;
+ }
+ if (args.command === "status" || args.command === "cancel" || args.command === "queue"
) {
if (args.runtime === undefined) {
throw new Error(
@@ -1501,85 +1612,6 @@ export async function runCli(
await writeQueue();
await new Promise((resolveWait) => setTimeout(resolveWait, 1_000));
}
- } else if (args.command === "builds") {
- const entries = await runtime.builds();
- const inspected = args.json || args.verbose ? entries : entries.slice(0, 20);
- const builds = await Promise.all(inspected.map(async (entry) => {
- const status = await runtime.status(entry.build);
- return {
- build: entry.build,
- createdAt: entry.createdAt,
- status: status.dispatch === undefined
- ? status.build?.state.status
- : submissionStatus(status.dispatch),
- ...(status.build === undefined ? {} : { coreStatus: status.build.state.status }),
- ...summarizeBuildCatalog(entry, status.build?.state),
- };
- }));
- writeOperational({ builds }, "Build archive", "info", [["Builds", String(entries.length)]],
- builds.map((item) => {
- const source = basename(item.run?.path ?? item.source.path);
- const targets = item.targets.length === 0 ? "no named target" : item.targets.join(", ");
- return `${item.build}: ${item.status ?? "unknown"} · ${source} · ${targets}`;
- }));
- } else if (args.command === "history") {
- const catalogs = await runtime.builds();
- const history = await Promise.all(catalogs.map(async (catalog) => {
- if (args.source !== undefined && catalog.source.path !== args.source) {
- return { entries: [], targetNames: [] };
- }
- if (args.file !== undefined && !catalog.aliases.some((alias) => alias.name === args.file)) {
- return { entries: [], targetNames: [] };
- }
- const status = await runtime.status(catalog.build);
- if (status.build === undefined) return { entries: [], targetNames: [] };
- const targetOutputs = new Set(status.build.state.request.targets.map((target) => target.output));
- return {
- targetNames: catalog.aliases
- .filter((alias) => alias.ref.kind === "logical-output" && targetOutputs.has(alias.ref.id))
- .map((alias) => alias.name),
- entries: acceptedArchivedOutputs(status.build.state, catalog)
- .filter((output) => args.file === undefined || output.name === args.file)
- .map((output) => ({
- build: catalog.build,
- createdAt: catalog.createdAt,
- status: status.dispatch === undefined
- ? status.build!.state.status
- : submissionStatus(status.dispatch),
- source: catalog.source,
- ...(catalog.run === undefined ? {} : { run: catalog.run }),
- output,
- })),
- };
- }));
- const targetNames = new Set(history.flatMap((item) => item.targetNames));
- const entries = history.flatMap((item) => item.entries)
- .filter((entry) => !args.excludeTargets || !targetNames.has(entry.output.name))
- .sort((left, right) => right.createdAt - left.createdAt
- || left.output.name.localeCompare(right.output.name)
- || left.build.localeCompare(right.build));
- const query = {
- ...(args.file === undefined ? {} : { output: args.file }),
- ...(args.source === undefined ? {} : { source: args.source }),
- };
- const pins = args.pin ? pinnedRecords(entries) : [];
- const shown = entries.slice(0, args.verbose ? undefined : 20);
- writeOperational({ query, entries, ...(args.pin ? { pins } : {}) },
- entries.length === 0 ? "No accepted output history" : args.pin ? "Reuse these Records" : "Output history",
- entries.length === 0 ? "warning" : "info", [
- ...(args.file === undefined ? [] : [["Output", args.file] as const]),
- ...(args.source === undefined ? [] : [["Source", args.source] as const]),
- ["Records", String(entries.length)],
- ...(args.pin ? [["Outputs pinned", String(pins.length)] as const] : []),
- ], args.pin
- ? [
- "Paste into a Run Source; the newest accepted Record of each output is selected.",
- ...pins.flatMap((item) => item.markup),
- ]
- : shown.map((item) => {
- const created = new Date(item.createdAt).toISOString();
- return `${item.build}: ${item.output.name} · ${created} · ${item.output.record.id}`;
- }));
} else if (args.command === "status") {
let status = await runtime.status(args.file!);
if (args.watch && status.build !== undefined && status.dispatch !== undefined) {
@@ -1604,15 +1636,22 @@ export async function runCli(
});
status = await runtime.status(args.file!);
}
+ const result = await readBuildResult(
+ buildResultDirectory(join(commandProjectRoot(), ".hypit", "results"), args.file!),
+ );
+ const found = status.build !== undefined || status.dispatch !== undefined || result !== undefined;
const effectiveStatus = status.dispatch === undefined
- ? status.build?.state.status
+ ? status.build?.state.status ?? result?.status
: submissionStatus(status.dispatch);
const machine = {
- build: status.build === undefined ? null : {
- id: status.build.build,
+ build: !found ? null : {
+ id: status.build?.build ?? status.dispatch?.build ?? result!.id,
status: effectiveStatus,
- coreStatus: status.build.state.status,
- diagnostics: status.build.state.diagnostics,
+ ...(status.build === undefined ? {} : {
+ coreStatus: status.build.state.status,
+ diagnostics: status.build.state.diagnostics,
+ }),
+ ...(result === undefined ? {} : { result }),
},
catalog: status.catalog === undefined
? null
@@ -1632,12 +1671,12 @@ export async function runCli(
const notableOperations = status.operations.filter((operation) =>
operation.status !== "completed").slice(0, args.verbose ? undefined : 12);
const terminal = status.dispatch?.phase === "terminal";
- writeOperational(machine, status.build === undefined
+ writeOperational(machine, !found
? "Build not found"
: args.watch
? terminal ? "Build finished" : "Build still running"
: "Build status",
- status.build === undefined
+ !found
? "warning"
: status.dispatch?.terminal === "failed"
? "error"
@@ -1655,137 +1694,7 @@ export async function runCli(
? `${operation.endpoint}: ${operation.status}`
: `${operation.endpoint}: ${formatOperationProgress(operation.progress)}`)
.concat(status.dispatch?.reason === undefined ? [] : [`Reason ${status.dispatch.reason}`]));
- if (status.build === undefined || status.dispatch?.terminal === "failed") io.setExitCode?.(1);
- } else if (args.command === "inspect") {
- const status = await runtime.status(args.file!);
- if (status.build === undefined) throw new Error(`Build ${args.file} does not exist`);
- const archive = inspectBuild(status.build.state, status.catalog);
- const effectiveStatus = status.dispatch === undefined
- ? status.build.state.status
- : submissionStatus(status.dispatch);
- const machine = {
- build: status.build.build,
- status: effectiveStatus,
- coreStatus: status.build.state.status,
- dispatch: status.dispatch,
- archive,
- operations: status.operations.map((operation) => ({
- id: operation.id,
- command: operation.command,
- endpoint: operation.endpoint,
- status: operation.status,
- ...(operation.wakeAt === undefined ? {} : { wakeAt: operation.wakeAt }),
- ...(operation.progress === undefined ? {} : { progress: operation.progress }),
- ...(operation.failure === undefined ? {} : { failure: operation.failure }),
- })),
- };
- const aliases = archive.presentation?.aliases ?? [];
- const aliasByOutput = new Map(aliases.flatMap((alias) =>
- alias.ref.kind === "logical-output" ? [[alias.ref.id, alias] as const] : []));
- const recordById = new Map(archive.records.map((record) => [record.id, record]));
- const targetOutputs = new Set(archive.targets.map((target) => target.output));
- const targetLines = archive.targets.map((target) => {
- const alias = aliasByOutput.get(target.output);
- const record = target.record === undefined ? undefined : recordById.get(target.record);
- const name = alias?.name ?? target.output;
- if (record === undefined) return `Target ${name} · not accepted`;
- return `Target ${name} · ${displayType(record.type)} · ${record.id}`;
- });
- const otherAccepted = archive.demandedOutputs.filter((item) =>
- item.accepted && !targetOutputs.has(item.output));
- const otherLines = args.verbose
- ? otherAccepted.map((item) => {
- const alias = aliasByOutput.get(item.output);
- const record = recordById.get(item.record);
- const name = alias?.name ?? item.output;
- return record === undefined
- ? `Output ${name}`
- : `Output ${name} · ${displayType(record.type)} · ${record.id}`;
- })
- : otherAccepted.length === 0
- ? []
- : [`Other accepted outputs ${otherAccepted.length} · use --verbose to list them`];
- writeOperational(machine, "Build archive detail", "info", [
- ["Build", status.build.build], ["Status", effectiveStatus],
- ...(status.dispatch?.phase === "terminal" || effectiveStatus === archive.status
- ? [] : [["Core", archive.status] as const]),
- ["Targets", String(archive.targets.length)], ["Accepted records", String(archive.records.length)],
- ["Operations", String(status.operations.length)],
- ], [
- ...(status.dispatch?.reason === undefined ? [] : [`Reason ${status.dispatch.reason}`]),
- ...targetLines,
- ...otherLines,
- ]);
- } else if (args.command === "get") {
- const status = await runtime.status(args.file!);
- if (status.build === undefined) throw new Error(`Build ${args.file} does not exist`);
- if (args.artifact !== undefined && (args.name !== undefined || args.record !== undefined || args.output !== undefined)) {
- throw new Error("get accepts one of --name, --record, --output or --artifact");
- }
- if (args.artifact !== undefined) {
- const references = findArchivedArtifact(status.build.state, args.artifact);
- if (args.to === undefined) {
- writeOperational({ build: status.build.build, artifact: args.artifact, references }, "Artifact references", "info", [
- ["Build", status.build.build], ["Artifact", args.artifact], ["References", String(references.length)],
- ]);
- } else {
- const [first] = references;
- if (first === undefined) throw new Error(`Build ${args.file} does not reference Artifact ${args.artifact}`);
- const artifacts = await loadRuntimeArtifactAccess(await runtimeHost(args.runtime!));
- let materialized;
- try {
- materialized = await materializeArtifact(artifacts, first, args.to, `Build ${args.file}`);
- } finally {
- await artifacts.close();
- }
- const machine = {
- build: status.build.build,
- artifact: args.artifact,
- references,
- materialized,
- };
- writeOperational(machine, "Artifact materialized", "success", [
- ["Build", status.build.build], ["Artifact", args.artifact], ["Path", materialized.path],
- ]);
- }
- return;
- }
- const record = selectArchivedRecord(status.build.state, {
- ...(args.name === undefined ? {} : { name: args.name }),
- ...(args.record === undefined ? {} : { record: args.record }),
- ...(args.output === undefined ? {} : { output: args.output }),
- ...(status.catalog === undefined ? {} : { catalog: status.catalog }),
- });
- if (args.to === undefined) {
- const machine = {
- build: status.build.build,
- record,
- artifacts: collectArtifacts(record.value.kind === "blob" ? record.value : record.value.value),
- };
- writeOperational(machine, "Archived Record", "info", [
- ["Build", status.build.build], ["Record", record.id],
- ...(record.value.kind === "inline"
- ? [["Value", inlineValuePreview(record.value.value)] as const]
- : []),
- ["Artifacts", String(machine.artifacts.length)],
- ]);
- } else {
- const artifacts = await loadRuntimeArtifactAccess(await runtimeHost(args.runtime!));
- let materialized;
- try {
- materialized = await materializeRecord(artifacts, record, args.to);
- } finally {
- await artifacts.close();
- }
- const machine = {
- build: status.build.build,
- record: record.id,
- materialized,
- };
- writeOperational(machine, "Record materialized", "success", [
- ["Build", status.build.build], ["Record", record.id], ["Path", materialized.path],
- ]);
- }
+ if (!found || status.dispatch?.terminal === "failed") io.setExitCode?.(1);
} else {
const result = await runtime.cancel(args.file!, args.reason);
const machine = {
@@ -1822,6 +1731,7 @@ export async function runCli(
const effectiveWorkspaceRoot = args.workspaceRoot
?? selectedRuntimeProjectRoot
?? dirname(resolve(args.file!));
+ const projectResultsRoot = join(effectiveWorkspaceRoot, ".hypit", "results");
const sourcePackageRoot = effectivePackageRoot
?? await resolvePackageRoot(effectiveWorkspaceRoot);
const loadedPackageSet = distribution.discoverSourcePackages === undefined
@@ -1921,6 +1831,7 @@ export async function runCli(
frontends: runFrontends,
packageContributions,
runtime: archive,
+ resultsRoot: projectResultsRoot,
});
} finally {
await archive.close();
@@ -1947,6 +1858,19 @@ export async function runCli(
}),
catalog,
attachments: result.compilation.attachments,
+ result: {
+ root: projectResultsRoot,
+ ...(args.name === undefined ? {} : { name: args.name }),
+ reuses: result.compilation.run.document.candidates.flatMap((candidate) => {
+ if (candidate.kind !== "build-record") return [];
+ const resolvedCandidate = result.compilation.run.candidates[candidate.id];
+ return resolvedCandidate === undefined ? [] : [{
+ candidate: resolvedCandidate,
+ build: candidate.build,
+ output: candidate.output,
+ }];
+ }),
+ },
} as const;
const controller = await (await runtimeHost(args.runtime)).controller({
packageRoot: sourcePackageRoot,
@@ -1983,6 +1907,10 @@ export async function runCli(
}),
});
}
+ const terminal = built.dispatch.phase === "terminal";
+ const terminalResult = terminal
+ ? await readBuildResult(buildResultDirectory(projectResultsRoot, built.id))
+ : undefined;
const targetOutputs = new Set(built.state.request.targets.map((target) => target.output));
const presentation = catalog;
const targetAliases = presentation.aliases.filter((alias) =>
@@ -2003,8 +1931,10 @@ export async function runCli(
});
const machine = {
build: built.id,
+ ...(args.name === undefined ? {} : { name: args.name }),
status: built.status,
worker,
+ ...(terminalResult === undefined ? {} : { result: terminalResult }),
goals: built.state.plan.goals.map((goal) => {
const record = built.state.records.find((item) => item.id === goal.record);
return {
@@ -2020,25 +1950,33 @@ export async function runCli(
},
};
const runtimeHint = runtimeNeedsHint ? ` --runtime ${args.runtime}` : "";
- const terminalLines = targetPresentations.length === 0
+ const resultTargets = terminalResult?.targets.flatMap((name) => {
+ const output = terminalResult.outputs[name];
+ return output === undefined ? [] : [{ name, output }];
+ }) ?? [];
+ const terminalLines = resultTargets.length === 0 && targetPresentations.length === 0
? [
...(built.dispatch.reason === undefined ? [] : [`Reason ${built.dispatch.reason}`]),
- `Inspect hypit inspect ${built.id}${runtimeHint}`,
+ `Inspect hypit inspect ${built.id}`,
]
: [
...(built.dispatch.reason === undefined ? [] : [`Reason ${built.dispatch.reason}`]),
- `Inspect hypit inspect ${built.id}${runtimeHint}`,
- ...targetPresentations
- .filter((item) => item.inline !== undefined)
+ `Inspect hypit inspect ${built.id}`,
+ ...resultTargets
+ .filter((item) => item.output.value.kind === "inline")
.slice(0, args.verbose ? undefined : 8)
- .map((item) => `Result ${item.alias.name} = ${item.inline}`),
- ...targetPresentations
- .filter((item) => item.inline === undefined)
+ .map((item) => `Result ${item.name} = ${inlineValuePreview(
+ item.output.value.kind === "inline" ? item.output.value.value : undefined,
+ )}`),
+ ...resultTargets
+ .filter((item) => item.output.value.kind !== "inline")
.slice(0, args.verbose ? undefined : 4)
.map((item) =>
- `Export hypit get ${built.id}${runtimeHint} --name ${item.alias.name} --to <path>`),
+ `Export hypit get ${built.id} --name ${item.name} --to <path>`),
+ ...(resultTargets.length > 0 ? [] : targetPresentations
+ .filter((item) => item.inline !== undefined)
+ .map((item) => `Result ${item.alias.name} = ${item.inline}`)),
];
- const terminal = built.dispatch.phase === "terminal";
writeOperational(machine, args.follow
? terminal ? "Build finished" : "Build still running"
: "Build submitted",
@@ -2047,7 +1985,7 @@ export async function runCli(
["Build", built.id],
["Status", built.status],
["Worker", worker.state === "running" ? String(worker.pid) : worker.state],
- ["Goals", String(machine.goals.length)],
+ ["Goals", String(terminalResult?.targets.length ?? machine.goals.length)],
], terminal ? terminalLines : [
`Watch hypit status ${built.id}${runtimeHint} --watch`,
`Cancel hypit cancel ${built.id}${runtimeHint}`,
@@ -2069,6 +2007,7 @@ export async function runCli(
frontends: runFrontends,
packageContributions,
...(runtime === undefined ? {} : { runtime }),
+ resultsRoot: projectResultsRoot,
});
const result = loaded.compiler.planCompilation(loaded);
const preflight = args.runtime === undefined
diff --git a/packages/cli/src/output.ts b/packages/cli/src/output.ts
index f32025ef..4e7e5e82 100644
--- a/packages/cli/src/output.ts
+++ b/packages/cli/src/output.ts
@@ -452,8 +452,9 @@ function commandHelp(topic: string, colors: Palette): readonly string[] | undefi
colors.accent(colors.strong("hypit build")),
colors.dim("Submit one durable Build and ensure its selected Runtime Worker is available."),
"",
- " hypit build <run-source> [--runtime <profile>] [--asset-root <directory>] [--follow]",
+ " hypit build <run-source> [--name <name>] [--runtime <profile>] [--asset-root <directory>] [--follow]",
"",
+ " --name <name> give this execution a human-facing name",
" --follow observe the Build; the Worker still owns execution",
" --max-wait-ms <ms> bound startup or follow waiting",
],
@@ -501,9 +502,9 @@ function commandHelp(topic: string, colors: Palette): readonly string[] | undefi
],
builds: [
colors.accent(colors.strong("hypit builds")),
- colors.dim("List Builds archived by one Runtime Profile."),
+ colors.dim("List project-owned Build Results without opening a Runtime."),
"",
- " hypit builds [--runtime <profile>]",
+ " hypit builds [--workspace <project>]",
],
status: [
colors.accent(colors.strong("hypit status")),
@@ -515,22 +516,22 @@ function commandHelp(topic: string, colors: Palette): readonly string[] | undefi
],
inspect: [
colors.accent(colors.strong("hypit inspect")),
- colors.dim("Inspect one Build's targets and accepted archive."),
+ colors.dim("Inspect one project-owned Build Result and every saved public Output."),
"",
- " hypit inspect <build-id> [--runtime <profile>]",
+ " hypit inspect <build-id> [--workspace <project>]",
],
get: [
colors.accent(colors.strong("hypit get")),
- colors.dim("Read or copy one archived result; copying never reruns work."),
+ colors.dim("Read or copy one exact named Build Output; copying never reruns work."),
"",
- " hypit get <build-id> [--runtime <profile>] [--name <source-name>|--record <id>|--output <id>|--artifact <digest>] [--to <path>]",
+ " hypit get <build-id> --name <output-name> [--workspace <project>] [--to <path>]",
],
history: [
colors.accent(colors.strong("hypit history")),
- colors.dim("Find accepted historical Logical Outputs without selecting them for a new Run."),
+ colors.dim("Find project Build Outputs from local result manifests."),
"",
- " hypit history <output-name> [--runtime <profile>] [--source <author-source>]",
- " hypit history --source <author-source> [--runtime <profile>]",
+ " hypit history <output-name> [--workspace <project>] [--source <author-source>]",
+ " hypit history --source <author-source> [--workspace <project>]",
],
cancel: [
colors.accent(colors.strong("hypit cancel")),
@@ -596,12 +597,12 @@ export function writeCliHelp(io: CliIo, topic?: string): void {
row("build <run-source>", "submit a durable Build; --follow only observes"),
row("image --prompt … --to <path>", "generate one picture file for a package asset"),
"",
- colors.strong("Archive"),
+ colors.strong("Results"),
row("builds", "list known Builds"),
- row("history [output]", "find accepted historical Logical Outputs"),
+ row("history [output]", "find named historical Build Outputs"),
row("status <build-id> [--watch]", "show or continuously observe one Build"),
- row("inspect <build-id>", "inspect accepted Records and demanded outputs"),
- row("get <build-id>", "read or materialize one archived result"),
+ row("inspect <build-id>", "inspect one project Build Result"),
+ row("get <build-id> --name <output>", "read or materialize one Build Output"),
row("cancel <build-id>", "withdraw one Build and reconcile submitted work"),
"",
colors.strong("Runtime"),
diff --git a/packages/cli/src/run-file.ts b/packages/cli/src/run-file.ts
index 3bdd8a37..072eb59b 100644
--- a/packages/cli/src/run-file.ts
+++ b/packages/cli/src/run-file.ts
@@ -1,3 +1,14 @@
+import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
+
+import {
+ buildResultDirectory,
+ resolveBuildResultOutput,
+} from "@hypit/build-result";
+import type {
+ BuildResultFileRef,
+ BuildResultJsonValue,
+ ResolvedBuildResultOutput,
+} from "@hypit/build-result";
import {
NodeRunCompiler,
} from "@hypit/compiler-node";
@@ -8,6 +19,7 @@ import type {
import type { CliRuntime } from "./runtime-port.js";
import type { NodePackageContribution } from "@hypit/package-loader-node";
import type { WorkspaceSession } from "@hypit/workspace";
+import type { BlobRef, CanonicalValue, StoredValue } from "@hypit/protocol";
import {
installRunFragmentHostFacets,
runFrontendsFromHostFacets,
@@ -37,6 +49,8 @@ function createRunCompiler(options: {
readonly frontends: readonly RunFrontend[];
readonly packageContributions: readonly NodePackageContribution[];
readonly runtime?: Pick<CliRuntime, "status">;
+ readonly resultsRoot?: string;
+ readonly workspace?: WorkspaceSession;
}): NodeRunCompiler {
const fragments = new RunFragmentRegistry();
for (const item of options.packageContributions) {
@@ -52,11 +66,92 @@ function createRunCompiler(options: {
archive.set(id, status);
return status;
};
+ const isResultFile = (value: unknown): value is BuildResultFileRef => {
+ if (value === null || Array.isArray(value) || typeof value !== "object") return false;
+ const item = value as Readonly<Record<string, unknown>>;
+ return item.kind === "build-file" && typeof item.path === "string"
+ && typeof item.size === "number" && typeof item.mediaType === "string";
+ };
+ const isResultOutput = (value: unknown): value is { readonly kind: "build-output"; readonly build: string; readonly output: string } => {
+ if (value === null || Array.isArray(value) || typeof value !== "object") return false;
+ const item = value as Readonly<Record<string, unknown>>;
+ return item.kind === "build-output" && typeof item.build === "string" && typeof item.output === "string";
+ };
+ const isBlob = (value: unknown): value is BlobRef => {
+ if (value === null || Array.isArray(value) || typeof value !== "object") return false;
+ const item = value as Readonly<Record<string, unknown>>;
+ return item.kind === "blob" && typeof item.digest === "string"
+ && typeof item.size === "number" && typeof item.mediaType === "string";
+ };
+ const resultValue = async (
+ value: BuildResultJsonValue,
+ owner: string,
+ ): Promise<CanonicalValue | BlobRef> => {
+ const workspace = options.workspace;
+ const resultsRoot = options.resultsRoot;
+ if (workspace === undefined || resultsRoot === undefined) {
+ throw new Error("Build Result resolution has no Workspace");
+ }
+ const admit = async (file: BuildResultFileRef, defaultBuild: string): Promise<BlobRef> => {
+ const build = file.build ?? defaultBuild;
+ const directory = buildResultDirectory(resultsRoot, build);
+ if (isAbsolute(file.path)) throw new Error(`Build ${build} file path must be relative`);
+ const absolute = resolve(directory, file.path);
+ const relation = relative(directory, absolute);
+ if (relation === ".." || relation.startsWith(`..${sep}`)) {
+ throw new Error(`Build ${build} file path leaves its result directory`);
+ }
+ const sourceRelative = relative(dirname(workspace.entry.id), absolute);
+ const from = sourceRelative.startsWith(".") ? sourceRelative : `.${sep}${sourceRelative}`;
+ const admitted = await workspace.resolveAsset(workspace.entry, { from, mediaType: file.mediaType });
+ return {
+ ...admitted.artifact,
+ origin: { kind: "build-file", build, path: file.path },
+ };
+ };
+ if (isResultFile(value)) return await admit(value, owner);
+ if (isResultOutput(value)) {
+ const forwarded = await resolveBuildResultOutput(resultsRoot, value.build, value.output);
+ if (forwarded === undefined) throw new Error(`Build ${value.build} has no Output ${value.output}`);
+ return await resolvedValue(forwarded);
+ }
+ if (Array.isArray(value)) {
+ return await Promise.all(value.map(async (item) => await resultValue(item, owner))) as CanonicalValue;
+ }
+ if (value !== null && typeof value === "object") {
+ return Object.fromEntries(await Promise.all(Object.entries(value)
+ .map(async ([key, item]) => [key, await resultValue(item, owner)] as const))) as CanonicalValue;
+ }
+ return value;
+ };
+ const resolvedValue = async (resolved: ResolvedBuildResultOutput): Promise<BlobRef | CanonicalValue> => {
+ if (resolved.value.kind === "build-file") return await resultValue(resolved.value, resolved.build);
+ if (resolved.value.kind === "inline") return resolved.value.value;
+ return await resultValue(resolved.value.value, resolved.build);
+ };
+ const storedResult = async (build: string, output: string): Promise<{
+ readonly type: ResolvedBuildResultOutput["type"];
+ readonly value: StoredValue;
+ } | undefined> => {
+ const resultsRoot = options.resultsRoot;
+ if (resultsRoot === undefined) return undefined;
+ const resolved = await resolveBuildResultOutput(resultsRoot, build, output);
+ if (resolved === undefined) return undefined;
+ const value = await resolvedValue(resolved);
+ return {
+ type: resolved.type,
+ value: isBlob(value) ? value : { kind: "inline", value },
+ };
+ };
return new NodeRunCompiler({
authorCompiler: options.authorCompiler,
frontends,
fragments,
- ...(options.runtime === undefined ? {} : {
+ ...(options.resultsRoot !== undefined ? {
+ async resolveBuildRecord(id: string, output: string) {
+ return await storedResult(id, output);
+ },
+ } : options.runtime === undefined ? {} : {
async resolveBuildRecord(id: string, output: string) {
const status = await archived(id);
if (status.build === undefined) return undefined;
@@ -89,8 +184,9 @@ export async function loadRunFile(options: {
readonly frontends: readonly RunFrontend[];
readonly packageContributions: readonly NodePackageContribution[];
readonly runtime?: Pick<CliRuntime, "status">;
+ readonly resultsRoot?: string;
}): Promise<LoadedRunFile> {
- const compiler = createRunCompiler(options);
+ const compiler = createRunCompiler({ ...options, workspace: options.workspace });
const compiled = await compiler.compileSource(options.workspace.entry, options.workspace);
return { path: options.workspace.entry.id, compiler, ...compiled };
}
diff --git a/packages/cli/test/build-programs.test.ts b/packages/cli/test/build-programs.test.ts
index 29778726..c3efa613 100644
--- a/packages/cli/test/build-programs.test.ts
+++ b/packages/cli/test/build-programs.test.ts
@@ -144,12 +144,12 @@ test("Build fails its cheap preflight before submitting or starting programs", a
assert.deepEqual(calls, []);
});
-test("Build never provisions programs after a clean preflight", async () => {
+test("Build accepts a human name and never provisions programs after a clean preflight", async () => {
const calls: string[] = [];
const source = await runSource();
await assert.rejects(
async () => await runCli(
- ["build", source, "--runtime", "/p/hypit.runtime.json"],
+ ["build", source, "--name", "first-cut", "--runtime", "/p/hypit.runtime.json"],
io,
distribution(calls, []),
),
diff --git a/packages/cli/test/build-results.test.ts b/packages/cli/test/build-results.test.ts
new file mode 100644
index 00000000..5ddcd142
--- /dev/null
+++ b/packages/cli/test/build-results.test.ts
@@ -0,0 +1,237 @@
+import assert from "node:assert/strict";
+import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import test from "node:test";
+
+import { FileBuildResult } from "@hypit/build-result";
+import { ModulePackageRegistry, NodeCompiler } from "@hypit/compiler-node";
+import { AuthorFrontendRegistry, sealGraphFragment } from "@hypit/elaborator";
+import { createMarkupAuthorFrontend, MarkupSurfaceRegistry } from "@hypit/markup";
+import type { BuildState, ModuleManifest, ProducerRef, TypeRef } from "@hypit/protocol";
+import { runMarkupFrontend } from "@hypit/run-markup";
+import { NodeFilesystemWorkspace } from "@hypit/workspace-fs-node";
+
+import type { CliDistribution } from "../src/distribution.js";
+import { runCli } from "../src/main.js";
+import { loadRunFile } from "../src/run-file.js";
+
+const valueType: TypeRef = {
+ module: { name: "example.result", version: "1" },
+ name: "Value",
+};
+
+function completedState(): BuildState {
+ return {
+ status: "complete",
+ records: [{ id: "record:stage", type: valueType, value: { kind: "inline", value: "ready" } }],
+ plan: {
+ selections: [{ output: "logical:stage", candidate: "candidate:stage", record: "record:stage" }],
+ },
+ } as unknown as BuildState;
+}
+
+async function fixture(root: string): Promise<void> {
+ const result = await FileBuildResult.create(join(root, ".hypit", "results"), {
+ id: "bld_result",
+ name: "episode-stage",
+ source: { path: join(root, "main.svml") },
+ run: { path: join(root, "build.svrun") },
+ targets: ["final.video"],
+ aliases: [{ name: "stage.value", output: "logical:stage" }],
+ });
+ await result.sync({
+ state: completedState(),
+ artifacts: { async open() { throw new Error("inline Result has no files"); } },
+ });
+}
+
+async function jsonCommand(args: readonly string[], root: string): Promise<unknown> {
+ let output = "";
+ await runCli([...args, "--workspace", root, "--json"], {
+ write(text) { output += text; },
+ }, {} as CliDistribution);
+ return JSON.parse(output);
+}
+
+test("builds, history, inspect and get read project Build Results without opening a Runtime", async () => {
+ const root = await mkdtemp(join(tmpdir(), "hypit-cli-results-"));
+ try {
+ await fixture(root);
+ const builds = await jsonCommand(["builds"], root) as {
+ readonly builds: readonly {
+ readonly build: string;
+ readonly name?: string;
+ readonly status: string;
+ readonly outputs: readonly string[];
+ }[];
+ };
+ assert.equal(builds.builds.length, 1);
+ assert.equal(builds.builds[0]?.build, "bld_result");
+ assert.equal(builds.builds[0]?.name, "episode-stage");
+ assert.equal(builds.builds[0]?.status, "complete");
+ assert.deepEqual(builds.builds[0]?.outputs, ["stage.value"]);
+
+ const history = await jsonCommand(["history", "stage.value"], root) as {
+ readonly entries: readonly { readonly build: string; readonly output: { readonly name: string } }[];
+ };
+ assert.equal(history.entries.length, 1);
+ assert.equal(history.entries[0]?.build, "bld_result");
+ assert.equal(history.entries[0]?.output.name, "stage.value");
+
+ const inspected = await jsonCommand(["inspect", "bld_result"], root) as {
+ readonly result: { readonly id: string; readonly name?: string };
+ readonly outputs: readonly { readonly name: string; readonly target: boolean }[];
+ };
+ assert.equal(inspected.result.id, "bld_result");
+ assert.equal(inspected.result.name, "episode-stage");
+ assert.deepEqual(inspected.outputs, [{
+ name: "stage.value",
+ target: false,
+ type: valueType,
+ value: { kind: "inline", value: "ready" },
+ }]);
+
+ const destination = join(root, "exported.json");
+ await jsonCommand(["get", "bld_result", "--name", "stage.value", "--to", destination], root);
+ assert.equal(await readFile(destination, "utf8"), "\"ready\"\n");
+ } finally {
+ await rm(root, { recursive: true, force: true });
+ }
+});
+
+test("build-record reads one exact result file and preserves its historical origin", async () => {
+ const root = await mkdtemp(join(tmpdir(), "hypit-cli-result-reuse-"));
+ try {
+ const module = { name: "example.result-reuse", version: "1" } as const;
+ const videoType = { module, name: "Video" } satisfies TypeRef;
+ const producer = { module, name: "make-video" } satisfies ProducerRef;
+ const manifest: ModuleManifest = {
+ format: "hypit.module@1",
+ name: module.name,
+ version: module.version,
+ dependencies: [],
+ types: [{ name: videoType.name }],
+ capabilities: [],
+ producers: [{
+ name: producer.name,
+ inputs: [],
+ outputs: [{ name: "video", type: videoType }],
+ needs: [],
+ }],
+ };
+ const fragment = sealGraphFragment({
+ inputs: [],
+ operations: [{
+ id: "make-video",
+ producer,
+ inputs: {},
+ result: { kind: "output", name: "video" },
+ }],
+ exports: [{
+ name: "video",
+ type: videoType,
+ root: { kind: "fragment-operation", operation: "make-video" },
+ }],
+ });
+ const modules = new ModulePackageRegistry();
+ modules.register({ manifest });
+ const surfaces = new MarkupSurfaceRegistry();
+ surfaces.registerStructured({
+ module,
+ declaration: { name: "video", tag: "Video", mode: "structured", outputs: [] },
+ handler: ({ element }) => {
+ const id = element.attributes.id;
+ if (typeof id !== "string") throw new Error("Video id is required");
+ return {
+ records: [],
+ components: [{ id, fragment: fragment.id, inputs: {}, outputs: { video: `${id}.video` }, range: element.range }],
+ fragments: [fragment],
+ };
+ },
+ });
+ const authorFrontends = new AuthorFrontendRegistry();
+ authorFrontends.register(createMarkupAuthorFrontend({
+ registry: surfaces,
+ resolveModule(request) {
+ const resolved = modules.resolve(request.from);
+ if (resolved === undefined) throw new Error(`unknown module ${request.from}`);
+ return resolved;
+ },
+ }));
+ const authorCompiler = new NodeCompiler({
+ modules,
+ frontends: authorFrontends,
+ workspace: new NodeFilesystemWorkspace({ root }),
+ });
+
+ const resultsRoot = join(root, ".hypit", "results");
+ const result = await FileBuildResult.create(resultsRoot, {
+ id: "bld_prior",
+ source: { path: join(root, "main.svml") },
+ run: { path: join(root, "prior.svrun") },
+ targets: ["shot.video"],
+ aliases: [{ name: "shot.video", output: "logical:video" }],
+ });
+ const bytes = new TextEncoder().encode("prior video bytes");
+ await result.sync({
+ state: {
+ status: "complete",
+ records: [{
+ id: "record:video",
+ type: videoType,
+ value: {
+ kind: "blob",
+ resource: "resource:prior-video",
+ digest: "runtime-address:prior-video",
+ size: bytes.byteLength,
+ mediaType: "video/mp4",
+ },
+ }],
+ plan: { selections: [{ output: "logical:video", candidate: "candidate:video", record: "record:video" }] },
+ } as unknown as BuildState,
+ artifacts: {
+ async open() {
+ return (async function* () { yield bytes; })();
+ },
+ },
+ });
+
+ const authorFile = join(root, "main.svml");
+ const runFile = join(root, "reuse.svrun");
+ await writeFile(authorFile, `<?svml using="@hypit/markup@1"?>
+<svml>
+ <import as="media" from="example.result-reuse@1"/>
+ <media:Video id="shot"/>
+</svml>`, "utf8");
+ await writeFile(runFile, `<?svml using="@hypit/run-markup@1"?>
+<svrun version="1">
+ <author source="./main.svml"/>
+ <target output="shot.video"/>
+ <build-record id="prior" build="bld_prior" output="shot.video"/>
+ <satisfy output="shot.video" candidate="prior"/>
+</svrun>`, "utf8");
+
+ const workspace = await authorCompiler.openFile(runFile);
+ const loaded = await loadRunFile({
+ workspace,
+ authorCompiler,
+ frontends: [runMarkupFrontend],
+ packageContributions: [],
+ resultsRoot,
+ });
+ const candidate = loaded.run.graph.candidates[0];
+ assert.equal(candidate?.root.kind, "value");
+ const value = candidate?.root.kind === "value" ? candidate.root.value.value : undefined;
+ assert.equal(value?.kind, "blob");
+ assert.deepEqual(value?.kind === "blob" ? value.origin : undefined, {
+ kind: "build-file",
+ build: "bld_prior",
+ path: "files/shot.video.mp4",
+ });
+ assert.equal(loaded.attachments.length, 1);
+ assert.equal(loaded.compiler.planCompilation(loaded).state.plan.steps.length, 0);
+ } finally {
+ await rm(root, { recursive: true, force: true });
+ }
+});
diff --git a/packages/compiler-node/src/compiler.ts b/packages/compiler-node/src/compiler.ts
index a4c00a51..d77a18ef 100644
--- a/packages/compiler-node/src/compiler.ts
+++ b/packages/compiler-node/src/compiler.ts
@@ -1,4 +1,4 @@
-import { createHash } from "node:crypto";
+import { createHash, randomUUID } from "node:crypto";
import {
compileSourceClosure,
@@ -162,6 +162,7 @@ export class NodeCompiler {
const bytes = Uint8Array.from(request.bytes);
const artifact: BlobRef = {
kind: "blob",
+ resource: `res_${randomUUID()}`,
digest: `sha256:${createHash("sha256").update(bytes).digest("hex")}`,
size: bytes.byteLength,
mediaType: request.mediaType,
diff --git a/packages/compiler-node/test/compiler.test.ts b/packages/compiler-node/test/compiler.test.ts
index 5900e129..67f8de51 100644
--- a/packages/compiler-node/test/compiler.test.ts
+++ b/packages/compiler-node/test/compiler.test.ts
@@ -287,6 +287,7 @@ function memoryWorkspace(sourceText: string, assetBytes: Uint8Array): Workspace
const bytes = Uint8Array.from(assetBytes);
const artifact: BlobRef = {
kind: "blob",
+ resource: "res_memory-reference",
digest: `sha256:${createHash("sha256").update(bytes).digest("hex")}`,
size: bytes.byteLength,
mediaType: request.mediaType,
@@ -602,8 +603,16 @@ test("filesystem and in-memory Workspaces load identical asset bytes", async ()
const filesystem = await assetCompiler({ root }).compileFile(file);
const memory = await assetCompiler({ workspace: memoryWorkspace(source, bytes) }).compileFile("memory:main");
- assert.deepEqual(memory.attachments.map((item) => item.artifact),
- filesystem.attachments.map((item) => item.artifact));
+ const memoryArtifact = memory.attachments[0]?.artifact;
+ const filesystemArtifact = filesystem.attachments[0]?.artifact;
+ assert.ok(memoryArtifact?.resource);
+ assert.ok(filesystemArtifact?.resource);
+ assert.notEqual(memoryArtifact.resource, filesystemArtifact.resource,
+ "separate admissions keep separate resource identities even for equal bytes");
+ assert.deepEqual(
+ { digest: memoryArtifact.digest, size: memoryArtifact.size, mediaType: memoryArtifact.mediaType },
+ { digest: filesystemArtifact.digest, size: filesystemArtifact.size, mediaType: filesystemArtifact.mediaType },
+ );
assert.deepEqual(await readAttachment(memory.attachments[0]),
await readAttachment(filesystem.attachments[0]));
});
diff --git a/packages/core/src/graph.ts b/packages/core/src/graph.ts
index e9536223..a29c6b50 100644
--- a/packages/core/src/graph.ts
+++ b/packages/core/src/graph.ts
@@ -55,7 +55,14 @@ function normalizeRef(ref: GraphValueRef): GraphValueRef {
function normalizedStoredValue(value: StoredValue): StoredValue {
if (value.kind === "inline") return { kind: "inline", value: canonicalize(value.value) };
- return { kind: "blob", digest: value.digest, size: value.size, mediaType: value.mediaType };
+ return {
+ kind: "blob",
+ ...(value.resource === undefined ? {} : { resource: value.resource }),
+ ...(value.origin === undefined ? {} : { origin: { ...value.origin } }),
+ digest: value.digest,
+ size: value.size,
+ mediaType: value.mediaType,
+ };
}
function normalizeRoot(root: CandidateRoot): CandidateRoot {
diff --git a/packages/driver-node/src/artifacts.ts b/packages/driver-node/src/artifacts.ts
index 291b14b1..0865be73 100644
--- a/packages/driver-node/src/artifacts.ts
+++ b/packages/driver-node/src/artifacts.ts
@@ -1,4 +1,4 @@
-import { createHash } from "node:crypto";
+import { createHash, randomUUID } from "node:crypto";
import type { BlobRef, Digest } from "@hypit/protocol";
@@ -11,7 +11,7 @@ export class MemoryArtifactStore implements ArtifactStore {
const copy = Uint8Array.from(bytes);
const digest = `sha256:${createHash("sha256").update(copy).digest("hex")}` as Digest;
this.#values.set(digest, copy);
- return { kind: "blob", digest, size: copy.byteLength, mediaType };
+ return { kind: "blob", resource: `res_${randomUUID()}`, digest, size: copy.byteLength, mediaType };
}
async get(digest: Digest): Promise<Uint8Array | undefined> {
diff --git a/packages/driver-node/src/driver.ts b/packages/driver-node/src/driver.ts
index eab66623..b3b64cf7 100644
--- a/packages/driver-node/src/driver.ts
+++ b/packages/driver-node/src/driver.ts
@@ -48,6 +48,8 @@ export type NodeDriverOptions = {
readonly producers?: ProducerRegistry;
readonly endpoints?: EndpointRegistry;
readonly artifacts?: ArtifactStore;
+ /** Runtime execution may isolate transient bytes by Build without exposing that policy to Endpoints. */
+ readonly artifactsForBuild?: (build: string) => ArtifactStore;
readonly operations?: OperationStore;
readonly credentials?: CredentialStore;
readonly validators?: TypeValidatorRegistryLike;
@@ -89,6 +91,7 @@ export class NodeDriver {
readonly producers: ProducerRegistry;
readonly endpoints: EndpointRegistry;
readonly artifacts: ArtifactStore;
+ readonly #artifactsForBuild: ((build: string) => ArtifactStore) | undefined;
readonly operations: OperationStore | undefined;
readonly credentials: CredentialStore | undefined;
readonly validators: TypeValidatorRegistryLike;
@@ -97,11 +100,16 @@ export class NodeDriver {
this.producers = options.producers ?? new ProducerRegistry();
this.endpoints = options.endpoints ?? new EndpointRegistry();
this.artifacts = options.artifacts ?? new MemoryArtifactStore();
+ this.#artifactsForBuild = options.artifactsForBuild;
this.operations = options.operations;
this.credentials = options.credentials;
this.validators = options.validators ?? new TypeValidatorRegistry();
}
+ #artifactStore(build?: string): ArtifactStore {
+ return build === undefined ? this.artifacts : this.#artifactsForBuild?.(build) ?? this.artifacts;
+ }
+
async #endpointCredentials(
registration: EndpointRegistration,
): Promise<Readonly<Record<string, CredentialValue>>> {
@@ -317,7 +325,7 @@ export class NodeDriver {
const endpointContext = {
command: structuredClone(executable.command),
need: structuredClone(executable.command.need),
- artifacts: this.artifacts,
+ artifacts: this.#artifactStore(context.build),
credentials: await this.#endpointCredentials(executable.registration),
operation: identity.id,
};
@@ -420,7 +428,7 @@ export class NodeDriver {
const result = await executable.registration.handler({
command: structuredClone(executable.command),
need: structuredClone(executable.command.need),
- artifacts: this.artifacts,
+ artifacts: this.#artifactStore(context?.build),
credentials: await this.#endpointCredentials(executable.registration),
});
return { status: "completed", event: await this.#endpointEvent(state, executable, result) };
@@ -493,7 +501,7 @@ export class NodeDriver {
const endpointContext = {
command: structuredClone(executable.command),
need: structuredClone(executable.command.need),
- artifacts: this.artifacts,
+ artifacts: this.#artifactStore(operation.build),
credentials: await this.#endpointCredentials(executable.registration),
operation: current.id,
handle: structuredClone(current.handle as NonNullable<typeof current.handle>),
diff --git a/packages/media/src/schema.ts b/packages/media/src/schema.ts
index 4c5ebaee..3d0b4de4 100644
--- a/packages/media/src/schema.ts
+++ b/packages/media/src/schema.ts
@@ -8,7 +8,12 @@ const integer = { kind: "number", integer: true, minimum: 0 } as const;
const digest = { kind: "string", minLength: 71, maxLength: 71 } as const;
const object = (fields: Readonly<Record<string, { readonly schema: ValueSchema; readonly optional?: boolean }>>): ValueSchema => ({ kind: "object", fields });
const blobArtifactSchema = (mediaTypes?: readonly string[]): ValueSchema => object({
- kind: { schema: { kind: "literal", value: "blob" } }, digest: { schema: digest }, size: { schema: integer },
+ kind: { schema: { kind: "literal", value: "blob" } }, resource: { schema: string, optional: true },
+ origin: { schema: object({
+ kind: { schema: { kind: "literal", value: "build-file" } },
+ build: { schema: string }, path: { schema: string },
+ }), optional: true },
+ digest: { schema: digest }, size: { schema: integer },
mediaType: { schema: mediaTypes === undefined ? string : { kind: "string", enum: mediaTypes } },
});
const rational = object({ numerator: { schema: { kind: "number", integer: true, minimum: 1 } }, denominator: { schema: { kind: "number", integer: true, minimum: 1 } } });
diff --git a/packages/protocol/src/value.ts b/packages/protocol/src/value.ts
index 603068bd..446a5fe5 100644
--- a/packages/protocol/src/value.ts
+++ b/packages/protocol/src/value.ts
@@ -10,6 +10,14 @@ export type CanonicalValue =
export type BlobRef = {
readonly kind: "blob";
+ /** One admitted resource instance. Unlike digest, equal bytes admitted twice keep distinct identities. */
+ readonly resource?: string;
+ /** Exact durable origin carried only when a historical Build file enters a new execution. */
+ readonly origin?: {
+ readonly kind: "build-file";
+ readonly build: string;
+ readonly path: string;
+ };
readonly digest: Digest;
readonly size: number;
readonly mediaType: string;
@@ -75,6 +83,18 @@ export function blobRefObjectSchema(mediaTypes?: readonly string[]): ValueSchema
kind: "object",
fields: {
kind: { schema: { kind: "literal", value: "blob" } },
+ resource: { schema: { kind: "string", minLength: 1 }, optional: true },
+ origin: {
+ schema: {
+ kind: "object",
+ fields: {
+ kind: { schema: { kind: "literal", value: "build-file" } },
+ build: { schema: { kind: "string", minLength: 1 } },
+ path: { schema: { kind: "string", minLength: 1 } },
+ },
+ },
+ optional: true,
+ },
digest: { schema: { kind: "string", minLength: 71, maxLength: 71 } },
size: { schema: { kind: "number", integer: true, minimum: 0 } },
mediaType: {
diff --git a/packages/provider-hyperframes-aws-lambda/README.md b/packages/provider-hyperframes-aws-lambda/README.md
index c6475edc..6f0f6e03 100644
--- a/packages/provider-hyperframes-aws-lambda/README.md
+++ b/packages/provider-hyperframes-aws-lambda/README.md
@@ -18,7 +18,7 @@ remote route owns the same byte verifier as the local Endpoint. Chunking remains
configuration.
On success the Endpoint requires HyperFrames' plan and completed-frame counts to equal the source
-document, then streams the returned S3 object into the selected content-addressed ArtifactStore.
+document, then streams the returned S3 object into the current Build's working byte area.
The Runtime Profile selects the state machine and deployment settings. The Provider does not repeat
the request or selected endpoint as vendor metadata. Container and stream conformance stays in the explicit media
inspection/mux capabilities, so the orchestrating machine does not acquire a hidden FFmpeg
diff --git a/packages/provider-hypihub/README.md b/packages/provider-hypihub/README.md
index e7ad763f..b5633f92 100644
--- a/packages/provider-hypihub/README.md
+++ b/packages/provider-hypihub/README.md
@@ -6,7 +6,7 @@ when its OAuth login is configured.
It maps the currently shipped image/video model capabilities to HypiHub, including image edits and
image-to-video first-frame inputs, submits jobs, polls them, downloads the first-class assets and
-persists them in Hypit's configured ArtifactStore. Image references use HypiHub's documented
+admits them into the current Build's working byte area. Image references use HypiHub's documented
`reference_images` object shape (`[{ "url": "…" }]`); video references use the public
`reference_image_urls`, `reference_videos`, and `reference_audios` fields (with `ref_video_url`
for one video). First/last-frame images use `first_frame` and `last_frame`. It also
diff --git a/packages/provider-kie/README.md b/packages/provider-kie/README.md
index 54bdaed5..8bf8a0c1 100644
--- a/packages/provider-kie/README.md
+++ b/packages/provider-kie/README.md
@@ -87,14 +87,14 @@ Manifests actually imported by the author document; installing KIE does not add
## Paid-operation law
-1. Reference `BlobRef`s are read from the configured ArtifactStore and uploaded through KIE's file
+1. Reference `BlobRef`s are read from the current Build's working byte area and uploaded through KIE's file
stream API. KIE temporary URLs never enter author source or generated Product identity.
2. A successful `createTask` response is persisted as one asynchronous Operation. Because KIE does not document an
idempotency key, an ambiguous network/5xx submission is not automatically retried.
3. Once a `taskId` exists, later Worker polling continues only that same task. Poll/download errors cannot create a new
paid generation.
4. Successful result URLs are converted to short-lived download URLs, bounded while streaming,
- immediately written to the configured content-addressed ArtifactStore, and removed from durable
+ immediately admitted into that Build's working byte area, and removed from durable
result metadata.
5. The selected `BuildDispatchStore` owns shared Build capacity. This Provider contributes one KIE
pool plus exact capability lanes and a conservative create-task interval; it does not introduce Redis or another source of
diff --git a/packages/provider-media-aws-lambda/README.md b/packages/provider-media-aws-lambda/README.md
index 246e9dde..763cce2a 100644
--- a/packages/provider-media-aws-lambda/README.md
+++ b/packages/provider-media-aws-lambda/README.md
@@ -1,19 +1,13 @@
# `@hypit/provider-media-aws-lambda`
-AWS Lambda Endpoint package for the nine exact capabilities declared by
+Retired AWS Lambda Endpoint package for the nine capabilities declared by
`@hypit/media-pipeline` and implemented by `@hypit/media-execution`.
-The Provider invokes one versioned or aliased Lambda ARN synchronously. Source and result bytes stay
-in the configured S3 ArtifactStore bucket; invocation carries only bounded JSON and content-addressed
-references. Every reported result is checked through the Build's ArtifactStore before it can fulfill
-a Need.
+This route required the Runtime and Lambda function to share one selected S3 Artifact Store. Runtime
+Profiles no longer select an Artifact Store: each Build has private transient working resources and
+publishes its public outputs into the project Build Result. Consequently this package cannot be
+activated from a current Runtime Profile.
-Runtime configuration must provide:
-
-- a qualified `functionArn`, never an unversioned mutable function name;
-- the same `bucket` used by the selected S3 ArtifactStore;
-- optional `prefix`, `region` and `defaultConcurrency`.
-
-The package requests `network:aws:lambda` and `network:aws:s3`. It contains no deployment
-credentials, author syntax or media policy fork. The local and Lambda Providers consume the same
-public contracts and shared execution body.
+The source remains only as the description of the former transport. A replacement must own its
+remote staging internally, import completed resources into the current Build working store, and
+leave no Runtime-wide storage choice behind.
diff --git a/packages/provider-media-aws-lambda/src/activation.ts b/packages/provider-media-aws-lambda/src/activation.ts
index c469aee1..5892170e 100644
--- a/packages/provider-media-aws-lambda/src/activation.ts
+++ b/packages/provider-media-aws-lambda/src/activation.ts
@@ -1,63 +1,15 @@
import {
createRuntimeEndpointAdapterFacet,
- runtimeConfigExact,
- runtimeConfigObject,
- runtimeConfigPositiveInteger,
- runtimeConfigString,
} from "@hypit/runtime-kit";
-import type { RuntimeAdapterFactoryContext } from "@hypit/runtime-kit";
-
-import { createAwsLambdaMediaProvider } from "./provider.js";
-import type { CreateAwsLambdaMediaProviderOptions } from "./provider.js";
-
-const QUALIFIED_ARN = /^arn:aws(?:-[a-z]+)*:lambda:([a-z0-9-]+):\d{12}:function:[A-Za-z0-9-_]+:([A-Za-z0-9-_]+)$/u;
-
-function providerOptions(context: RuntimeAdapterFactoryContext): CreateAwsLambdaMediaProviderOptions {
- if (context.pool === undefined) throw new Error("AWS Lambda media Provider Pool is required");
- const config = runtimeConfigObject(context.config, "AWS Lambda media");
- runtimeConfigExact(config, [
- "functionArn", "bucket", "prefix", "region", "defaultConcurrency",
- ], "AWS Lambda media");
- const functionArn = runtimeConfigString(config.functionArn, "AWS Lambda functionArn");
- if (functionArn === undefined) throw new Error("AWS Lambda functionArn is required");
- const arn = QUALIFIED_ARN.exec(functionArn);
- if (arn === null) {
- throw new Error(
- `Lambda functionArn must name a version or alias, as arn:aws:lambda:<region>:<account>:function:<name>:<version>; got ${functionArn}`,
- );
- }
- const bucket = runtimeConfigString(config.bucket, "AWS Lambda bucket");
- if (bucket === undefined) {
- throw new Error(
- "AWS Lambda media bucket is required: the function reads sources and writes results in the"
- + " ArtifactStore's own bucket, because a synchronous invocation cannot carry media",
- );
- }
- const region = runtimeConfigString(config.region, "AWS Lambda region");
- if (region !== undefined && region !== arn[1]) {
- throw new Error(`Lambda region ${region} differs from the region in its ARN (${arn[1]})`);
- }
- const prefix = runtimeConfigString(config.prefix, "AWS Lambda prefix");
- const defaultConcurrency = runtimeConfigPositiveInteger(config.defaultConcurrency, "AWS Lambda defaultConcurrency");
- return {
- instance: context.instance,
- pool: context.pool,
- functionArn,
- bucket,
- ...(prefix === undefined ? {} : { prefix }),
- ...(region === undefined ? {} : { region }),
- ...(defaultConcurrency === undefined ? {} : { defaultConcurrency }),
- };
-}
const awsLambdaMediaRuntimeAdapter = createRuntimeEndpointAdapterFacet({
use: "@hypit/provider-media-aws-lambda",
- activate(context) {
- return { endpoint: createAwsLambdaMediaProvider(providerOptions(context)) };
+ activate(_context) {
+ throw new Error(
+ "@hypit/provider-media-aws-lambda is retired: it depended on a Runtime-wide shared S3 ArtifactStore."
+ + " Use a Provider that owns its remote transport and imports completed resources into the Build working store.",
+ );
},
- // ARN, region and bucket relationships are closed configuration facts. Live
- // Lambda availability remains an operation-time fact until a dedicated
- // read-only deployment probe is added.
});
export const hypitPackage = {
diff --git a/packages/runtime-host-node/README.md b/packages/runtime-host-node/README.md
index 034ba195..3d0dea64 100644
--- a/packages/runtime-host-node/README.md
+++ b/packages/runtime-host-node/README.md
@@ -13,5 +13,5 @@ These functions produce `RuntimeDoctorDiagnostic` values only. They do not const
read secret values, execute commands or choose fallback Providers.
The generic CLI depends only on this port. The official video application selects
-`@hypit/runtime-local` directly; Runtime Profiles vary stores and Endpoints rather than replacing
-the Runtime itself.
+`@hypit/runtime-local` directly; Runtime Profiles vary credentials, Endpoints and their services
+rather than replacing the Runtime itself or selecting Build history storage.
diff --git a/packages/runtime-host-node/package.json b/packages/runtime-host-node/package.json
index 38fa40f1..9d193246 100644
--- a/packages/runtime-host-node/package.json
+++ b/packages/runtime-host-node/package.json
@@ -8,6 +8,7 @@
".": "./src/index.ts"
},
"dependencies": {
+ "@hypit/build-result": "workspace:*",
"@hypit/protocol": "workspace:*",
"@hypit/runtime": "workspace:*",
"@hypit/runtime-kit": "workspace:*",
diff --git a/packages/runtime-host-node/src/index.ts b/packages/runtime-host-node/src/index.ts
index 6ee93057..00b4c511 100644
--- a/packages/runtime-host-node/src/index.ts
+++ b/packages/runtime-host-node/src/index.ts
@@ -3,6 +3,7 @@ import { constants } from "node:fs";
import { homedir } from "node:os";
import { delimiter, isAbsolute, join, resolve } from "node:path";
import type { ArtifactAttachment } from "@hypit/workspace";
+import type { BuildResultReuse } from "@hypit/build-result";
import type { BuildDefinition, BuildState, CapabilityRef, Digest } from "@hypit/protocol";
import type {
BuildCatalogDescriptor,
@@ -79,6 +80,12 @@ export type RuntimeHostExecution = RuntimeHostArchive & RuntimeHostArtifactAcces
readonly componentPackages?: readonly string[];
readonly catalog?: BuildCatalogDescriptor;
readonly attachments?: readonly ArtifactAttachment[];
+ /** Project-owned result destination. Runtime Profile never selects this location. */
+ readonly result?: {
+ readonly root: string;
+ readonly name?: string;
+ readonly reuses?: readonly BuildResultReuse[];
+ };
}, options?: {
readonly follow?: boolean;
readonly pollIntervalMs?: number;
diff --git a/packages/runtime-local/README.md b/packages/runtime-local/README.md
index aec11c5f..9f2cc311 100644
--- a/packages/runtime-local/README.md
+++ b/packages/runtime-local/README.md
@@ -1,7 +1,7 @@
# `@hypit/runtime-local`
-The default local Runtime for Hypit. It owns the Worker, scheduler, SQLite state and Build queue.
-It contains no video, authoring or Provider policy.
+The default local Runtime for Hypit. It owns the Worker, scheduler, active SQLite execution state
+and Build queue. Project-owned Build Results hold terminal public Outputs.
A Runtime Profile selects only the environmental parts that genuinely vary:
@@ -12,10 +12,6 @@ A Runtime Profile selects only the environmental parts that genuinely vary:
"use": "@hypit/runtime-local",
"config": {
"dataRoot": ".hypit/runtimes/local",
- "artifacts": {
- "use": "@hypit/artifact-store-fs",
- "config": { "path": "artifacts" }
- },
"credentials": {
"env": { "use": "@hypit/credential-store-env" }
},
@@ -30,5 +26,8 @@ sharing the configured command limit. Endpoint packages declare their own Provid
Cancellation prevents new work and makes a best effort to cancel an external operation already submitted;
completed output is never rolled back.
+The Profile does not select Build or Artifact history stores. Runtime working bytes are internal and
+Build-local. After a Result is terminal, history is read from `.hypit/results`, not Runtime SQLite.
+
Source imports select author packages. Runtime Profile entries select only code allowed to access files,
credentials, processes or networks. Installing a package changes neither selection.
diff --git a/packages/runtime-local/package.json b/packages/runtime-local/package.json
index 71fece7b..04a7f6e9 100644
--- a/packages/runtime-local/package.json
+++ b/packages/runtime-local/package.json
@@ -11,6 +11,8 @@
"./worker-process": "./src/worker-process.ts"
},
"dependencies": {
+ "@hypit/artifact-store-fs": "workspace:*",
+ "@hypit/build-result": "workspace:*",
"@hypit/component-kit": "workspace:*",
"@hypit/driver-node": "workspace:*",
"@hypit/endpoint-kit": "workspace:*",
@@ -23,7 +25,6 @@
"@hypit/validation": "workspace:*"
},
"devDependencies": {
- "@hypit/artifact-store-fs": "workspace:*",
"@hypit/credential-store-env": "workspace:*",
"@hypit/core": "workspace:*"
}
diff --git a/packages/runtime-local/src/config.ts b/packages/runtime-local/src/config.ts
index a4e6a6c8..bc45cd3b 100644
--- a/packages/runtime-local/src/config.ts
+++ b/packages/runtime-local/src/config.ts
@@ -1,4 +1,4 @@
-import { readFile, stat } from "node:fs/promises";
+import { readFile, rm, stat } from "node:fs/promises";
import { dirname, resolve } from "node:path";
import {
@@ -12,10 +12,10 @@ import type { CanonicalValue, CapabilityRef } from "@hypit/protocol";
import {
CompositeCredentialStore,
} from "@hypit/runtime";
-import type { ArtifactStore, CredentialStore } from "@hypit/runtime";
+import type { CredentialStore } from "@hypit/runtime";
+import { FileArtifactStore } from "@hypit/artifact-store-fs";
import {
isRuntimeAdapterHostFacet,
- runtimeArtifactStoreAdapterHostAbi,
runtimeCredentialStoreAdapterHostAbi,
runtimeEndpointAdapterHostAbi,
RuntimeAdapterRegistry,
@@ -59,7 +59,6 @@ export type RuntimeConfigEntry = {
export type RuntimeConfigDocument = {
readonly format: "hypit.runtime-profile@1";
readonly dataRoot: string;
- readonly artifacts: RuntimeConfigEntry;
readonly credentials: readonly RuntimeConfigEntry[];
readonly endpoints: readonly RuntimeConfigEntry[];
};
@@ -146,8 +145,7 @@ export function parseRuntimeConfig(value: unknown): RuntimeConfigDocument {
throw new Error(`Local Runtime loader cannot activate ${String(runtime.use)}`);
}
const config = object(runtime.config, "$runtime.runtime.config");
- exactKeys(config, ["dataRoot", "artifacts", "credentials", "endpoints"], "$runtime.runtime.config");
- const artifacts = entry(config.artifacts, "artifacts", "$runtime.runtime.config.artifacts");
+ exactKeys(config, ["dataRoot", "credentials", "endpoints"], "$runtime.runtime.config");
const credentials = entries(config.credentials, "$runtime.runtime.config.credentials");
const endpoints = entries(config.endpoints, "$runtime.runtime.config.endpoints", true);
const ids = [...credentials, ...endpoints].map((value) => value.instance);
@@ -155,7 +153,6 @@ export function parseRuntimeConfig(value: unknown): RuntimeConfigDocument {
return {
format: "hypit.runtime-profile@1",
dataRoot: requiredString(config.dataRoot, "$runtime.runtime.config.dataRoot"),
- artifacts,
credentials,
endpoints,
};
@@ -177,7 +174,6 @@ function runtimePackageSelection(document: RuntimeConfigDocument): NodePackageSe
return {
selected: [],
logical: [
- { abi: runtimeArtifactStoreAdapterHostAbi, name: document.artifacts.use },
...document.credentials.map((item) => ({ abi: runtimeCredentialStoreAdapterHostAbi, name: item.use })),
...document.endpoints.map((item) => ({ abi: runtimeEndpointAdapterHostAbi, name: item.use })),
],
@@ -199,7 +195,6 @@ async function installRuntimeAdapters(
): Promise<void> {
const logical = selection.logical?.filter((address) => {
if (address.abi === runtimeEndpointAdapterHostAbi) return !registry.has(address.name, "endpoint");
- if (address.abi === runtimeArtifactStoreAdapterHostAbi) return !registry.has(address.name, "artifact-store");
if (address.abi === runtimeCredentialStoreAdapterHostAbi) return !registry.has(address.name, "credential-store");
return true;
}) ?? [];
@@ -232,7 +227,6 @@ export async function prepareRuntimeConfigPackages(
if (options.distributionPackageRoot === undefined) return [];
const { document } = await openRuntimeConfig(path, options.packageRoot);
const requirements = await distributionExternalPackageRequirements([
- document.artifacts.use,
...document.credentials.map((item) => item.use),
...document.endpoints.map((item) => item.use),
], options.distributionPackageRoot);
@@ -327,20 +321,17 @@ async function openCredentialStores(
}
}
-async function openArtifactStore(
- document: RuntimeConfigDocument,
- root: string,
- hostStateRoot: string,
- registry: RuntimeAdapterRegistry,
-): Promise<RuntimeOpened<ArtifactStore>> {
- const item = document.artifacts;
- return await registry.openArtifactStore(item.use, adapterContext(root, hostStateRoot, item));
-}
-
function statePath(root: string): string {
return resolve(root, "runtime.sqlite");
}
+function buildWorkPath(root: string, build: string): string {
+ if (build.trim().length === 0 || build.includes("/") || build.includes("\\")) {
+ throw new Error("Build id is not a working-directory name");
+ }
+ return resolve(root, "work", build);
+}
+
async function inspectRuntimeConfig(
path: string,
options: RuntimeInspectionOptions,
@@ -362,7 +353,6 @@ async function inspectRuntimeConfig(
return { dataRoot: root, diagnostics: [diagnostic(error, "RUNTIME_PACKAGE_SELECTION_INVALID")] };
}
const storeSelections = [
- { item: document.artifacts, kind: "artifact-store" as const },
...document.credentials.map((item) => ({ item, kind: "credential-store" as const })),
];
for (const selection of storeSelections) {
@@ -467,7 +457,7 @@ export async function preflightRuntimeConfig(
return await inspectRuntimeConfig(path, { ...options, active: false });
}
-/** Active diagnosis may ask selected stores and adapters to verify their configured services. */
+/** Active diagnosis may ask selected credentials and Endpoints to verify their configured services. */
export async function doctorRuntimeConfig(
path: string,
options: LoadRuntimeConfigOptions & { readonly capabilities?: readonly CapabilityRef[] } = {},
@@ -484,10 +474,8 @@ export async function createRuntimeFromConfig(
const registry = options.registry ?? new RuntimeAdapterRegistry();
await installRuntimeAdapters(registry, packageRoot, runtimePackageSelection(document), options.distributionPackageRoot);
const state = new SqliteRuntimeState(statePath(root));
- let artifacts: RuntimeOpened<ArtifactStore> | undefined;
let credentials: Awaited<ReturnType<typeof openCredentialStores>> | undefined;
try {
- artifacts = await openArtifactStore(document, root, hostStateRoot, registry);
credentials = await openCredentialStores(document, root, hostStateRoot, registry);
const endpoints = await Promise.all(document.endpoints.map(async (item) => await registry.createEndpoint(
item.use,
@@ -498,7 +486,11 @@ export async function createRuntimeFromConfig(
buildCatalog: state.catalog,
operationStore: state.operations,
dispatchStore: state.dispatch,
- artifactStore: artifacts.value,
+ artifactStore: new FileArtifactStore(resolve(root, "artifacts")),
+ artifactStoreForBuild: (build) => new FileArtifactStore(buildWorkPath(root, build)),
+ clearBuildArtifacts: async (build) => {
+ await rm(buildWorkPath(root, build), { recursive: true, force: true });
+ },
credentialStore: credentials.store,
loadComponentPackages: async (specifiers) => {
const loaded = await loadNodePackageSelection(specifiers, packageRoot, {
@@ -511,13 +503,11 @@ export async function createRuntimeFromConfig(
endpoints,
close: async () => {
await credentials?.close();
- await artifacts?.close?.();
state.close();
},
});
} catch (error) {
await credentials?.close();
- await artifacts?.close?.();
state.close();
throw error;
}
@@ -542,16 +532,9 @@ export async function createRuntimeArtifactAccessFromConfig(
path: string,
options: LoadRuntimeConfigOptions = {},
): Promise<LocalRuntimeArtifactAccess> {
- const { document, root, packageRoot } = await openRuntimeConfig(path, options.packageRoot);
- const hostStateRoot = resolve(options.hostStateRoot ?? hypitHostStateRoot());
- const registry = options.registry ?? new RuntimeAdapterRegistry();
- await installRuntimeAdapters(registry, packageRoot, {
- selected: [], logical: [{ abi: runtimeArtifactStoreAdapterHostAbi, name: document.artifacts.use }],
- }, options.distributionPackageRoot);
- const opened = await openArtifactStore(document, root, hostStateRoot, registry);
+ const { root } = await openRuntimeConfig(path, options.packageRoot);
return createLocalRuntimeArtifactAccess({
- artifactStore: opened.value,
- ...(opened.close === undefined ? {} : { close: opened.close }),
+ artifactStore: new FileArtifactStore(resolve(root, "artifacts")),
});
}
diff --git a/packages/runtime-local/src/runtime.ts b/packages/runtime-local/src/runtime.ts
index 0fd25c30..7fb2695b 100644
--- a/packages/runtime-local/src/runtime.ts
+++ b/packages/runtime-local/src/runtime.ts
@@ -10,6 +10,7 @@ import {
import {
isStreamingArtifactStore,
} from "@hypit/runtime";
+import { FileBuildResult } from "@hypit/build-result";
import { TypeValidatorRegistry } from "@hypit/validation";
import { createLocalRuntimeArchiveControl, createLocalRuntimeArtifactAccess } from "./control.js";
@@ -86,6 +87,7 @@ export async function createLocalRuntime(
producers,
endpoints,
artifacts: options.artifactStore,
+ ...(options.artifactStoreForBuild === undefined ? {} : { artifactsForBuild: options.artifactStoreForBuild }),
credentials: options.credentialStore,
operations: options.operationStore,
validators,
@@ -93,9 +95,13 @@ export async function createLocalRuntime(
const worker = createDurableLocalWorker(driver, {
stores: {
builds: options.buildStore,
+ ...(buildCatalog === undefined ? {} : { catalog: buildCatalog }),
operations: options.operationStore,
dispatch: options.dispatchStore,
},
+ artifactStore: options.artifactStore,
+ ...(options.artifactStoreForBuild === undefined ? {} : { artifactStoreForBuild: options.artifactStoreForBuild }),
+ ...(options.clearBuildArtifacts === undefined ? {} : { clearBuildArtifacts: options.clearBuildArtifacts }),
installComponentPackages,
});
const credentialControl = createLocalCredentialControl({
@@ -112,12 +118,13 @@ export async function createLocalRuntime(
artifactStore: options.artifactStore,
});
const stageAttachments = async (request: LocalBuildRequest): Promise<void> => {
+ const artifactStore = options.artifactStoreForBuild?.(request.id) ?? options.artifactStore;
for (const item of request.attachments ?? []) {
- if (await options.artifactStore.has(item.artifact.digest)) continue;
+ if (await artifactStore.has(item.artifact.digest)) continue;
const stream = await item.open();
- const stored = isStreamingArtifactStore(options.artifactStore)
- ? await options.artifactStore.putStream(stream, item.artifact.mediaType)
- : await options.artifactStore.put(await (async () => {
+ const stored = isStreamingArtifactStore(artifactStore)
+ ? await artifactStore.putStream(stream, item.artifact.mediaType)
+ : await artifactStore.put(await (async () => {
const chunks: Uint8Array[] = [];
let size = 0;
for await (const chunk of stream) {
@@ -140,28 +147,51 @@ export async function createLocalRuntime(
);
}
};
- const presentation = async (build: string): Promise<LocalBuildSubmission> => {
+ const presentation = async (build: string, previousState?: LocalBuildSubmission["state"]): Promise<LocalBuildSubmission> => {
const [snapshot, dispatch] = await Promise.all([
options.buildStore.read(build),
options.dispatchStore.read(build),
]);
- assert(snapshot !== undefined && dispatch !== undefined, `Build ${build} has no durable Runtime state`);
+ assert(dispatch !== undefined, `Build ${build} has no Runtime dispatch`);
+ const state = snapshot?.state ?? previousState;
+ assert(state !== undefined, `Build ${build} has no execution state`);
const status: LocalBuildSubmission["status"] = dispatch.phase === "terminal"
? dispatch.terminal!
: dispatch.phase;
- return { id: build, state: snapshot.state, status, dispatch };
+ return { id: build, state, status, dispatch };
};
const submit = async (request: LocalBuildRequest): Promise<LocalBuildSubmission> => {
- assert(request.id.trim().length > 0, "Build id must not be empty");
+ assert(request.id.trim().length > 0 && !request.id.includes("/") && !request.id.includes("\\"),
+ "Build id must be one directory-safe name");
if (request.catalog !== undefined) {
assert(buildCatalog !== undefined, "Build supplied Host catalog metadata but no BuildCatalog was selected");
}
await stageAttachments(request);
await options.buildStore.create(request.id, request.definition);
+ const resultRequest = request.result;
+ const result = resultRequest === undefined
+ ? undefined
+ : await (async () => {
+ assert(request.catalog !== undefined, "Build Result requires Author catalog names");
+ const aliases = request.catalog.aliases.flatMap((alias) => alias.ref.kind === "logical-output"
+ ? [{ name: alias.name, output: alias.ref.id }]
+ : []);
+ const names = new Map(aliases.map((alias) => [alias.output, alias.name]));
+ return await FileBuildResult.create(resultRequest.root, {
+ id: request.id,
+ ...(resultRequest.name === undefined ? {} : { name: resultRequest.name }),
+ source: request.catalog.source,
+ ...(request.catalog.run === undefined ? {} : { run: request.catalog.run }),
+ targets: request.definition.request.targets.map((target) => names.get(target.output) ?? target.output),
+ aliases,
+ ...(resultRequest.reuses === undefined ? {} : { reuses: resultRequest.reuses }),
+ });
+ })();
await options.dispatchStore.create({
build: request.id,
componentPackages: [...new Set(request.componentPackages ?? [])].sort(),
+ ...(result === undefined ? {} : { resultDirectory: result.directory }),
});
if (request.catalog !== undefined) {
await buildCatalog!.record(request.id, request.catalog);
@@ -182,7 +212,7 @@ export async function createLocalRuntime(
while (follow.follow === true && !["complete", "failed", "cancelled"].includes(result.status)) {
if (maxWaitMs !== undefined && Date.now() - startedAt + pollIntervalMs > maxWaitMs) return result;
await wait(pollIntervalMs, follow.signal);
- result = await presentation(request.id);
+ result = await presentation(request.id, result.state);
}
return result;
};
diff --git a/packages/runtime-local/src/types.ts b/packages/runtime-local/src/types.ts
index 27be64a6..4f8e6871 100644
--- a/packages/runtime-local/src/types.ts
+++ b/packages/runtime-local/src/types.ts
@@ -25,6 +25,10 @@ export type CreateLocalRuntimeOptions = {
readonly operationStore: OperationStore;
readonly dispatchStore: import("@hypit/runtime").BuildDispatchStore;
readonly artifactStore: ArtifactStore;
+ /** Optional Build-local transient byte area used by Provider execution and Result writing. */
+ readonly artifactStoreForBuild?: (build: string) => ArtifactStore;
+ /** Called only after a terminal Build Result has accepted every public Output completed so far. */
+ readonly clearBuildArtifacts?: (build: string) => Awaitable<void>;
readonly credentialStore: CredentialStore;
readonly components?: readonly ComponentPackage[];
/** Load the complete physical package closure named by a claimed Build. */
diff --git a/packages/runtime-local/src/worker.ts b/packages/runtime-local/src/worker.ts
index a3563aa4..2e2b2fc4 100644
--- a/packages/runtime-local/src/worker.ts
+++ b/packages/runtime-local/src/worker.ts
@@ -1,5 +1,7 @@
import type { BuildState } from "@hypit/protocol";
+import { FileBuildResult } from "@hypit/build-result";
import type {
+ ArtifactStore,
BuildDispatchSnapshot,
RuntimeCommandExecutor,
RuntimeExecutionResult,
@@ -8,14 +10,18 @@ import type {
ScheduledBuildResult,
RuntimeWorkerRunOptions,
} from "@hypit/runtime";
-import { LocalBuildScheduler } from "@hypit/runtime";
+import { isStreamingArtifactStore, LocalBuildScheduler } from "@hypit/runtime";
type LocalWorkerOptions = {
readonly stores: {
readonly builds: import("@hypit/runtime").BuildStore;
+ readonly catalog?: import("@hypit/runtime").BuildCatalog;
readonly operations: import("@hypit/runtime").OperationStore;
readonly dispatch: import("@hypit/runtime").BuildDispatchStore;
};
+ readonly artifactStore: ArtifactStore;
+ readonly artifactStoreForBuild?: (build: string) => ArtifactStore;
+ readonly clearBuildArtifacts?: (build: string) => Promise<void> | void;
readonly installComponentPackages: (specifiers: readonly string[]) => Promise<void>;
};
@@ -140,6 +146,50 @@ class DurableLocalWorker {
this.#executorWithCapacity = new CapacityExecutor(executor, options);
}
+ async #openResult(dispatch: BuildDispatchSnapshot): Promise<FileBuildResult | undefined> {
+ return dispatch.resultDirectory === undefined
+ ? undefined
+ : await FileBuildResult.open(dispatch.resultDirectory);
+ }
+
+ async #syncResult(dispatch: BuildDispatchSnapshot, state: BuildState): Promise<void> {
+ const result = await this.#openResult(dispatch);
+ if (result === undefined) return;
+ const artifactStore = this.#options.artifactStoreForBuild?.(dispatch.build) ?? this.#options.artifactStore;
+ await result.sync({
+ state,
+ artifacts: {
+ open: async (artifact) => {
+ if (isStreamingArtifactStore(artifactStore)) {
+ return await artifactStore.open(artifact.digest);
+ }
+ const bytes = await artifactStore.get(artifact.digest);
+ return bytes === undefined ? undefined : (async function* () { yield bytes; })();
+ },
+ },
+ });
+ }
+
+ async #finishResult(
+ dispatch: BuildDispatchSnapshot,
+ status: "complete" | "failed" | "cancelled",
+ failure?: string,
+ ): Promise<boolean> {
+ const result = await this.#openResult(dispatch);
+ if (result === undefined) return false;
+ await result.finish({ status, ...(failure === undefined ? {} : { failure }) });
+ return true;
+ }
+
+ async #retireExecution(build: string): Promise<void> {
+ await Promise.allSettled([
+ this.#options.clearBuildArtifacts?.(build),
+ this.#options.stores.operations.removeBuild?.(build),
+ this.#options.stores.builds.remove?.(build),
+ this.#options.stores.catalog?.remove?.(build),
+ ].filter((task): task is Promise<void> => task !== undefined));
+ }
+
async #cancel(dispatch: BuildDispatchSnapshot): Promise<BuildDispatchSnapshot> {
const snapshot = await this.#options.stores.builds.read(dispatch.build);
assert(snapshot !== undefined, `Build ${dispatch.build} has no durable state`);
@@ -153,11 +203,14 @@ class DurableLocalWorker {
}
}
await this.#options.stores.dispatch.releaseBuildCapacity(dispatch.build);
- return await this.#options.stores.dispatch.finish(
+ const persisted = await this.#finishResult(dispatch, "cancelled", dispatch.cancellation?.reason ?? "cancelled by Runtime");
+ const finished = await this.#options.stores.dispatch.finish(
dispatch.build,
"cancelled",
dispatch.cancellation?.reason ?? "cancelled by Runtime",
);
+ if (persisted) await this.#retireExecution(dispatch.build);
+ return finished;
}
async #finish(dispatch: BuildDispatchSnapshot, result: ScheduledBuildResult): Promise<BuildDispatchSnapshot> {
@@ -166,19 +219,30 @@ class DurableLocalWorker {
if (current.cancellation !== undefined) return await this.#cancel(current);
if (result.status === "complete") {
await this.#options.stores.dispatch.releaseBuildCapacity(dispatch.build);
- return await this.#options.stores.dispatch.finish(dispatch.build, "complete");
+ const persisted = await this.#finishResult(dispatch, "complete");
+ const finished = await this.#options.stores.dispatch.finish(dispatch.build, "complete");
+ if (persisted) await this.#retireExecution(dispatch.build);
+ return finished;
}
if (result.status === "failed" || result.outcomes.some((item) => item.status === "error")) {
- const reason = result.outcomes.find((item) => item.status === "error")?.message ?? "Core Build failed";
+ const reason = result.outcomes.find((item) => item.status === "error")?.message
+ ?? result.state.diagnostics.at(-1)?.message
+ ?? "Core Build failed";
await this.#options.stores.dispatch.releaseBuildCapacity(dispatch.build);
- return await this.#options.stores.dispatch.finish(dispatch.build, "failed", reason);
+ const persisted = await this.#finishResult(dispatch, "failed", reason);
+ const finished = await this.#options.stores.dispatch.finish(dispatch.build, "failed", reason);
+ if (persisted) await this.#retireExecution(dispatch.build);
+ return finished;
}
const pending = result.outcomes.filter((item) => item.status === "pending");
const deferred = result.outcomes.filter((item) => item.status === "deferred");
if (pending.length === 0 && deferred.length === 0 && result.blocked.length > 0) {
const reason = result.blocked.map((item) => `${item.reason}: ${item.subject}`).join(", ");
await this.#options.stores.dispatch.releaseBuildCapacity(dispatch.build);
- return await this.#options.stores.dispatch.finish(dispatch.build, "failed", reason);
+ const persisted = await this.#finishResult(dispatch, "failed", reason);
+ const finished = await this.#options.stores.dispatch.finish(dispatch.build, "failed", reason);
+ if (persisted) await this.#retireExecution(dispatch.build);
+ return finished;
}
const now = Date.now();
const wakeAt = Math.min(...[
@@ -197,7 +261,11 @@ class DurableLocalWorker {
if (current?.cancellation !== undefined) return await this.#cancel(current);
const reason = error instanceof Error ? error.message : String(error);
await this.#options.stores.dispatch.releaseBuildCapacity(dispatch.build);
- return await this.#options.stores.dispatch.finish(dispatch.build, "failed", reason);
+ await this.#finishResult(dispatch, "failed", reason).catch(() => false);
+ const finished = await this.#options.stores.dispatch.finish(dispatch.build, "failed", reason);
+ // An infrastructure failure may be the Result writer itself. Keep execution bytes and facts so
+ // a local repair can recover already accepted public Outputs instead of deleting their source.
+ return finished;
}
async #runClaimed(dispatches: readonly BuildDispatchSnapshot[]): Promise<readonly BuildDispatchSnapshot[]> {
@@ -212,6 +280,7 @@ class DurableLocalWorker {
await this.#options.installComponentPackages(dispatch.componentPackages);
const snapshot = await this.#options.stores.builds.read(dispatch.build);
assert(snapshot !== undefined, `Dispatch ${dispatch.build} has no Build Definition`);
+ await this.#syncResult(dispatch, snapshot.state);
runnable.push({ dispatch, snapshot });
} catch (error) {
finished.push(await this.#fail(dispatch, error));
@@ -220,6 +289,11 @@ class DurableLocalWorker {
if (runnable.length === 0) return finished;
const scheduler = new LocalBuildScheduler(this.#executorWithCapacity, {
buildStore: this.#options.stores.builds,
+ onStateChange: async (build, state) => {
+ const item = runnable.find((candidate) => candidate.dispatch.build === build);
+ assert(item !== undefined, `Result update refers to unknown Build ${build}`);
+ await this.#syncResult(item.dispatch, state);
+ },
});
try {
const results = await scheduler.run(runnable.map(({ dispatch, snapshot }) => ({
diff --git a/packages/runtime-local/test/config.test.ts b/packages/runtime-local/test/config.test.ts
index 4a71e530..237321f2 100644
--- a/packages/runtime-local/test/config.test.ts
+++ b/packages/runtime-local/test/config.test.ts
@@ -5,11 +5,10 @@ import { join } from "node:path";
import test from "node:test";
import {
- createRuntimeArtifactStoreAdapterFacet,
+ createRuntimeCredentialStoreAdapterFacet,
createRuntimeEndpointAdapterFacet,
RuntimeAdapterRegistry,
} from "@hypit/runtime-kit";
-import { MemoryArtifactStore } from "@hypit/driver-node";
import { SqliteRuntimeState } from "@hypit/store-sqlite";
import {
createRuntimeArchiveFromConfig,
@@ -21,7 +20,6 @@ import {
function profile(config: {
readonly dataRoot?: string;
- readonly artifacts?: Readonly<Record<string, unknown>>;
readonly credentials?: Readonly<Record<string, unknown>>;
readonly endpoints?: Readonly<Record<string, unknown>>;
} = {}) {
@@ -31,7 +29,6 @@ function profile(config: {
use: "@hypit/runtime-local",
config: {
dataRoot: config.dataRoot ?? ".hypit/runtimes/local",
- artifacts: config.artifacts ?? { use: "example.artifacts" },
credentials: config.credentials ?? {},
endpoints: config.endpoints ?? {},
},
@@ -39,13 +36,12 @@ function profile(config: {
};
}
-test("Runtime Profile names the stores and Endpoints used by one local Runtime", () => {
+test("Runtime Profile names credentials and Endpoints, not historical storage", () => {
const parsed = parseRuntimeConfig(profile({
credentials: { secrets: { use: "example.credentials" } },
endpoints: { generation: { use: "example.provider", pool: "shared" } },
}));
assert.equal(parsed.dataRoot, ".hypit/runtimes/local");
- assert.deepEqual(parsed.artifacts, { use: "example.artifacts", instance: "artifacts" });
assert.deepEqual(parsed.credentials, [{ use: "example.credentials", instance: "secrets" }]);
assert.deepEqual(parsed.endpoints, [{ use: "example.provider", instance: "generation", pool: "shared" }]);
});
@@ -54,30 +50,20 @@ test("Runtime Profile rejects source ownership fields", () => {
assert.throws(() => parseRuntimeConfig({ ...profile(), root: "." }), /does not accept root/u);
});
-test("archive inspection opens SQLite only; Artifact access opens the selected Store", async () => {
+test("archive inspection and working Artifact access require no selected ArtifactStore", async () => {
const root = await mkdtemp(join(tmpdir(), "hypit-runtime-slice-"));
const path = join(root, "hypit.runtime.json");
await writeFile(path, JSON.stringify(profile({ dataRoot: "." })));
const state = new SqliteRuntimeState(join(root, "runtime.sqlite"));
state.close();
- let artifactConstructions = 0;
const registry = new RuntimeAdapterRegistry();
- registry.registerFacet(createRuntimeArtifactStoreAdapterFacet({
- use: "example.artifacts",
- validate() {},
- open() {
- artifactConstructions += 1;
- return { value: new MemoryArtifactStore() };
- },
- }));
try {
const archive = await createRuntimeArchiveFromConfig(path, { registry, readOnly: true });
assert.equal((await archive.status("missing")).build, undefined);
await archive.close();
- assert.equal(artifactConstructions, 0);
const artifacts = await createRuntimeArtifactAccessFromConfig(path, { registry });
+ assert.equal(await artifacts.readArtifact("sha256:0000000000000000000000000000000000000000000000000000000000000000"), undefined);
await artifacts.close();
- assert.equal(artifactConstructions, 1);
} finally {
await rm(root, { recursive: true, force: true });
}
@@ -91,11 +77,6 @@ test("doctor reports a down Managed Program", async () => {
endpoints: { speech: { use: "example.speech" } },
})));
const registry = new RuntimeAdapterRegistry();
- registry.registerFacet(createRuntimeArtifactStoreAdapterFacet({
- use: "example.artifacts",
- validate() {},
- open: () => ({ value: new MemoryArtifactStore() }),
- }));
registry.registerFacet(createRuntimeEndpointAdapterFacet({
use: "example.speech",
activate: (context) => ({
@@ -120,16 +101,19 @@ test("doctor reports a down Managed Program", async () => {
}
});
-test("preflight validates stores without running their active doctor", async () => {
+test("preflight validates credential stores without running their active doctor", async () => {
const root = await mkdtemp(join(tmpdir(), "hypit-runtime-preflight-"));
const path = join(root, "hypit.runtime.json");
- await writeFile(path, JSON.stringify(profile({ dataRoot: "." })));
+ await writeFile(path, JSON.stringify(profile({
+ dataRoot: ".",
+ credentials: { secrets: { use: "example.credentials" } },
+ })));
let activeChecks = 0;
const registry = new RuntimeAdapterRegistry();
- registry.registerFacet(createRuntimeArtifactStoreAdapterFacet({
- use: "example.artifacts",
+ registry.registerFacet(createRuntimeCredentialStoreAdapterFacet({
+ use: "example.credentials",
validate() {},
- open: () => ({ value: new MemoryArtifactStore() }),
+ open: () => ({ value: { async resolve() { return undefined; } } }),
doctor: async () => {
activeChecks += 1;
return [];
diff --git a/packages/runtime-local/test/local.test.ts b/packages/runtime-local/test/local.test.ts
index 4bdb9932..ae846882 100644
--- a/packages/runtime-local/test/local.test.ts
+++ b/packages/runtime-local/test/local.test.ts
@@ -1,9 +1,10 @@
import assert from "node:assert/strict";
-import { createHash } from "node:crypto";
import {
mkdir,
mkdtemp,
+ readFile,
rm,
+ stat,
writeFile,
} from "node:fs/promises";
import { tmpdir } from "node:os";
@@ -11,13 +12,13 @@ import { join } from "node:path";
import test from "node:test";
import { FileArtifactStore } from "@hypit/artifact-store-fs";
+import { readBuildResult } from "@hypit/build-result";
import { EnvironmentCredentialStore } from "@hypit/credential-store-env";
-import { MemoryArtifactStore } from "@hypit/driver-node";
import { defineEndpointPackage } from "@hypit/endpoint-kit";
import type { AsyncEndpoint, EndpointPackage } from "@hypit/endpoint-kit";
import type { ComponentPackage } from "@hypit/component-kit";
import { createLocalRuntime } from "@hypit/runtime-local";
-import { defineBuild } from "@hypit/core";
+import { defineBuild, sealBuildRequest } from "@hypit/core";
import {
collectLoadedNodePackageComponents,
loadNodePackageSelection,
@@ -42,12 +43,17 @@ function definition(state: ReturnType<typeof createGreetingBuild>) {
function projectRuntimeFixture(directory: string) {
const state = new SqliteRuntimeState(join(directory, ".hypit", "runtime.sqlite"));
+ const work = join(directory, ".hypit", "work");
return {
buildStore: state.builds,
buildCatalog: state.catalog,
operationStore: state.operations,
dispatchStore: state.dispatch,
artifactStore: new FileArtifactStore(join(directory, ".hypit", "artifacts")),
+ artifactStoreForBuild: (build: string) => new FileArtifactStore(join(work, build)),
+ clearBuildArtifacts: async (build: string) => {
+ await rm(join(work, build), { recursive: true, force: true });
+ },
credentialStore: new EnvironmentCredentialStore(),
close: () => state.close(),
} as const;
@@ -107,6 +113,12 @@ test("project local runtime queues, polls and cancels work with replaceable pack
aliases: [{
name: "final.document",
ref: { kind: "logical-output" as const, id: initial.request.targets[0]!.output },
+ }, {
+ name: "prompt.text",
+ ref: { kind: "logical-output" as const, id: "prompt" },
+ }, {
+ name: "generated.text",
+ ref: { kind: "logical-output" as const, id: "generated" },
}],
};
let promptCalls = 0;
@@ -195,7 +207,12 @@ test("project local runtime queues, polls and cancels work with replaceable pack
components: [components],
endpoints: [endpointPackage],
});
- const first = await firstRuntime.build({ id: "greeting-build", definition: definition(initial), catalog });
+ const first = await firstRuntime.build({
+ id: "greeting-build",
+ definition: definition(initial),
+ catalog,
+ result: { root: join(directory, "results") },
+ });
assert.equal(first.status, "queued");
assert.equal((await firstRuntime.workOnce())?.phase, "waiting");
assert.equal(starts, 1);
@@ -209,8 +226,18 @@ test("project local runtime queues, polls and cancels work with replaceable pack
assert.equal(requestCalls, 1, "the Need request Producer is also persisted");
assert.equal(assembleCalls, 1);
const clientStatus = await firstRuntime.status("greeting-build");
- assert.equal(clientStatus.catalog?.aliases[0]?.name, "final.document");
- assert.deepEqual((await firstRuntime.builds()).map((item) => item.build), ["greeting-build"]);
+ assert.equal(clientStatus.build, undefined);
+ assert.equal(clientStatus.catalog, undefined);
+ assert.deepEqual(clientStatus.operations, []);
+ assert.equal(clientStatus.dispatch?.terminal, "complete");
+ assert.deepEqual(await firstRuntime.builds(), []);
+ const buildResult = await readBuildResult(join(directory, "results", "greeting-build"));
+ assert.deepEqual(Object.keys(buildResult?.outputs ?? {}).sort(), [
+ "final.document",
+ "generated.text",
+ "prompt.text",
+ ]);
+ assert.deepEqual(buildResult?.targets, ["final.document"]);
await firstRuntime.build({ id: "greeting-follow", definition: definition(createGreetingBuild()) });
await firstRuntime.workOnce();
@@ -237,6 +264,116 @@ test("project local runtime queues, polls and cancels work with replaceable pack
}
});
+test("a completed public file moves into its Build Result and leaves no Runtime working copy", async () => {
+ const directory = await mkdtemp(join(tmpdir(), "hypit-local-result-file-"));
+ const id = "public-file-result";
+ const workStore = new FileArtifactStore(join(directory, ".hypit", "work", id));
+ const initial = createGreetingBuild({ generationRealization: "placeholder" });
+ const buildDefinition = defineBuild(initial.program, initial.graph, sealBuildRequest({
+ targets: [{ output: "generated" }],
+ }));
+ const components: ComponentPackage = {
+ producers: [{
+ producer: producers.makePrompt,
+ handler: () => ({ outputs: { prompt: { kind: "inline", value: "make a clip" } }, needs: {} }),
+ }, {
+ producer: producers.placeholderText,
+ handler: async () => ({
+ outputs: { generated: await workStore.put(new TextEncoder().encode("video bytes"), "video/mp4") },
+ needs: {},
+ }),
+ }],
+ };
+ try {
+ const runtime = await createLocalRuntime({
+ ...projectRuntimeFixture(directory),
+ components: [components],
+ });
+ await runtime.build({
+ id,
+ definition: buildDefinition,
+ catalog: {
+ source: { path: join(directory, "main.svml") },
+ aliases: [{ name: "clip.video", ref: { kind: "logical-output", id: "generated" } }],
+ },
+ result: { root: join(directory, "results") },
+ });
+ assert.equal((await runtime.workOnce())?.terminal, "complete");
+ const result = await readBuildResult(join(directory, "results", id));
+ assert.equal(result?.outputs["clip.video"]?.value.kind, "build-file");
+ assert.equal(await readFile(join(directory, "results", id, "files", "clip.video.mp4"), "utf8"), "video bytes");
+ assert.equal(await stat(join(directory, ".hypit", "work", id)).then(() => true, () => false), false);
+ await runtime.close();
+ } finally {
+ await rm(directory, { recursive: true, force: true });
+ }
+});
+
+test("a failed Build keeps public Outputs completed before the failure and retires execution state", async () => {
+ const directory = await mkdtemp(join(tmpdir(), "hypit-local-partial-result-"));
+ const initial = createGreetingBuild();
+ const components: ComponentPackage = {
+ producers: [{
+ producer: producers.makePrompt,
+ handler: () => ({ outputs: { prompt: { kind: "inline", value: "Greet Ada" } }, needs: {} }),
+ }, {
+ producer: producers.requestText,
+ handler: ({ inputs }) => {
+ assert.equal(inputs.prompt?.value.kind, "inline");
+ return { outputs: {}, needs: { generation: { prompt: inputs.prompt.value.value } } };
+ },
+ }],
+ };
+ const endpoint = defineEndpointPackage({
+ module: providerModule,
+ facet: "generation",
+ instance: "generation.failure",
+ pool: "generation.failure",
+ capabilities: [{
+ lifecycle: "asynchronous",
+ capability: capabilities.generation,
+ returns: types.generated,
+ endpoint: {
+ start: () => ({ status: "failed", failure: { code: "REMOTE_FAILED", message: "generation failed" } }),
+ poll: () => { throw new Error("failed work is not polled"); },
+ },
+ }],
+ });
+ try {
+ const runtime = await createLocalRuntime({
+ ...projectRuntimeFixture(directory),
+ components: [components],
+ endpoints: [endpoint],
+ });
+ await runtime.build({
+ id: "partial-result",
+ definition: definition(initial),
+ catalog: {
+ source: { path: join(directory, "main.svml") },
+ aliases: [{ name: "prompt.text", ref: { kind: "logical-output", id: "prompt" } }, {
+ name: "generated.text", ref: { kind: "logical-output", id: "generated" },
+ }, {
+ name: "final.document", ref: { kind: "logical-output", id: "document" },
+ }],
+ },
+ result: { root: join(directory, "results") },
+ });
+ assert.equal((await runtime.workOnce())?.terminal, "failed");
+ const result = await readBuildResult(join(directory, "results", "partial-result"));
+ assert.equal(result?.status, "failed");
+ assert.equal(result?.failure, "generation failed");
+ assert.deepEqual(Object.keys(result?.outputs ?? {}), ["prompt.text"]);
+ assert.deepEqual(result?.outputs["prompt.text"]?.value, { kind: "inline", value: "Greet Ada" });
+ const status = await runtime.status("partial-result");
+ assert.equal(status.build, undefined);
+ assert.deepEqual(status.operations, []);
+ assert.equal(status.dispatch?.terminal, "failed");
+ await runtime.close();
+ } finally {
+ await rm(directory, { recursive: true, force: true });
+ }
+});
+
test("one local Worker admits later Builds while preserving shared Endpoint capacity", async () => {
const directory = await mkdtemp(join(tmpdir(), "hypit-local-parallel-builds-"));
let unrestrictedActive = 0;
@@ -472,61 +609,3 @@ test("project local runtime remembers the complete package closure across increm
await rm(directory, { recursive: true, force: true });
}
});
-
-test("project local runtime accepts an explicitly selected replacement ArtifactStore package", async () => {
- const directory = await mkdtemp(join(tmpdir(), "hypit-local-artifacts-"));
- const artifactStore = new MemoryArtifactStore();
- try {
- const runtime = await createLocalRuntime({
- ...projectRuntimeFixture(directory),
- artifactStore,
- });
- const bytes = new Uint8Array([7, 8, 9]);
- const sourceArtifact = {
- kind: "blob" as const,
- digest: `sha256:${createHash("sha256").update(bytes).digest("hex")}` as const,
- size: bytes.byteLength,
- mediaType: "application/octet-stream",
- };
- assert.equal(await artifactStore.has(sourceArtifact.digest), false);
- await runtime.build({
- id: "source-artifact-staging",
- definition: definition(createGreetingBuild()),
- attachments: [{ artifact: sourceArtifact, open: async () => (async function* () { yield bytes; })() }],
- });
- const failed = await runtime.workOnce();
- assert.equal(failed?.terminal, "failed");
- assert.deepEqual(await artifactStore.get(sourceArtifact.digest), bytes);
- let reopened = false;
- await runtime.build({
- id: "existing-source-artifact",
- definition: definition(createGreetingBuild()),
- attachments: [{
- artifact: sourceArtifact,
- open: async () => {
- reopened = true;
- throw new Error("existing content-addressed bytes must not be reopened");
- },
- }],
- });
- assert.equal(reopened, false);
- const absentBytes = new Uint8Array([10, 11, 12]);
- const absentArtifact = {
- kind: "blob" as const,
- digest: `sha256:${createHash("sha256").update(absentBytes).digest("hex")}` as const,
- size: absentBytes.byteLength,
- mediaType: "application/octet-stream",
- };
- await assert.rejects(runtime.build({
- id: "mismatched-source-artifact",
- definition: definition(createGreetingBuild()),
- attachments: [{
- artifact: absentArtifact,
- open: async () => (async function* () { yield new Uint8Array([0]); })(),
- }],
- }), /does not match its staged bytes/u);
- await runtime.close();
- } finally {
- await rm(directory, { recursive: true, force: true });
- }
-});
diff --git a/packages/runtime-local/test/programs.test.ts b/packages/runtime-local/test/programs.test.ts
index 5a9c65f9..0dcab14a 100644
--- a/packages/runtime-local/test/programs.test.ts
+++ b/packages/runtime-local/test/programs.test.ts
@@ -51,7 +51,6 @@ async function project(program: (root: string) => ManagedProgram) {
use: "@hypit/runtime-local",
config: {
dataRoot: ".",
- artifacts: { use: "example.artifacts" },
credentials: {},
endpoints: {
one: { use: "example.program", pool: "example.local", config: {} },
@@ -231,7 +230,6 @@ test("up creates a fresh Runtime data directory before running commands", async
use: "@hypit/runtime-local",
config: {
dataRoot: "./never-created",
- artifacts: { use: "example.artifacts" },
credentials: {},
endpoints: { one: { use: "example.program", config: {} } },
},
diff --git a/packages/runtime/README.md b/packages/runtime/README.md
index 2fd8d5f2..77b2d61d 100644
--- a/packages/runtime/README.md
+++ b/packages/runtime/README.md
@@ -1,18 +1,18 @@
# `@hypit/runtime`
-Environment neutral execution and storage contracts around Core.
+Environment neutral execution contracts around Core.
The package defines the small ports used by a runtime:
-* `BuildStore` keeps one build definition and its accepted facts.
-* `OperationStore` keeps asynchronous external work while it is being polled.
+* `BuildStore` keeps one active Build definition and its accepted facts until its Result is terminal.
+* `OperationStore` keeps asynchronous external work only while the owning Build is active.
* `BuildDispatchStore` holds the local build queue and shared capacity reservations.
-* `ArtifactStore` stores bytes addressed by their content digest.
+* `ArtifactStore` is the transient byte gateway used during Provider execution.
* `CredentialStore` resolves secrets outside source and build state.
* `RuntimeCommandExecutor` turns the commands regenerated by Core into component or endpoint calls.
* `LocalBuildScheduler` runs ready commands concurrently while respecting declared resource limits.
-These contracts do not choose a filesystem, database, provider or source language. The default local
+These contracts do not define historical Build Results or choose a provider or source language. The default local
implementation lives in `@hypit/runtime-local`; Node component and endpoint execution lives in
`@hypit/driver-node`.
diff --git a/packages/runtime/src/catalog.ts b/packages/runtime/src/catalog.ts
index 16af960e..38384c61 100644
--- a/packages/runtime/src/catalog.ts
+++ b/packages/runtime/src/catalog.ts
@@ -28,4 +28,6 @@ export type BuildCatalog = {
record(build: string, descriptor: BuildCatalogDescriptor): Promise<BuildCatalogEntry>;
read(build: string): Promise<BuildCatalogEntry | undefined>;
list(): Promise<readonly BuildCatalogEntry[]>;
+ /** Result manifests own terminal presentation; Runtime Catalog covers active execution only. */
+ remove?(build: string): Promise<void>;
};
diff --git a/packages/runtime/src/dispatch.ts b/packages/runtime/src/dispatch.ts
index 8b1af305..6f8594b1 100644
--- a/packages/runtime/src/dispatch.ts
+++ b/packages/runtime/src/dispatch.ts
@@ -5,6 +5,8 @@ export type BuildDispatchRequest = {
readonly build: string;
/** Installed component packages loaded when a Worker claims this Build. */
readonly componentPackages: readonly string[];
+ /** Exact per-Build result directory selected by the submitting project. */
+ readonly resultDirectory?: string;
};
export type BuildDispatchSnapshot = BuildDispatchRequest & {
diff --git a/packages/runtime/src/operations.ts b/packages/runtime/src/operations.ts
index d17c62e2..232dec41 100644
--- a/packages/runtime/src/operations.ts
+++ b/packages/runtime/src/operations.ts
@@ -70,4 +70,6 @@ export type OperationStore = {
list(query: OperationQuery): Promise<readonly OperationSnapshot[]>;
/** One local Worker owns execution. A terminal Operation is returned unchanged. */
update(id: string, update: OperationUpdate): Promise<OperationSnapshot>;
+ /** Drop Provider execution payloads after the owning Build Result is terminal. */
+ removeBuild?(build: string): Promise<void>;
};
diff --git a/packages/runtime/src/scheduler.ts b/packages/runtime/src/scheduler.ts
index ae3466c4..5a6b9d8f 100644
--- a/packages/runtime/src/scheduler.ts
+++ b/packages/runtime/src/scheduler.ts
@@ -51,10 +51,12 @@ function buildCommandKey(build: string, command: string): string {
export class LocalBuildScheduler {
readonly #executor: RuntimeCommandExecutor;
readonly #buildStore: BuildSchedulerOptions["buildStore"];
+ readonly #onStateChange: BuildSchedulerOptions["onStateChange"];
constructor(executor: RuntimeCommandExecutor, options: BuildSchedulerOptions = {}) {
this.#executor = executor;
this.#buildStore = options.buildStore;
+ this.#onStateChange = options.onStateChange;
}
async run(requests: readonly ScheduledBuild[]): Promise<readonly ScheduledBuildResult[]> {
@@ -106,6 +108,7 @@ export class LocalBuildScheduler {
const accept = async (build: MutableBuild, event: import("@hypit/protocol").CommandResult): Promise<void> => {
if (this.#buildStore === undefined || build.machine === undefined) {
build.state = reduce(build.state, event);
+ await this.#onStateChange?.(build.id, build.state);
return;
}
const fact = build.machine.evaluate(event);
@@ -116,6 +119,7 @@ export class LocalBuildScheduler {
await this.#buildStore.append(build.id, fact);
build.machine.commit();
build.state = build.machine.view();
+ await this.#onStateChange?.(build.id, build.state);
};
const preparations = async (): Promise<Map<string, readonly RuntimeRunnableCommand[]>> => {
diff --git a/packages/runtime/src/types.ts b/packages/runtime/src/types.ts
index e0daf4e9..6e7bb73c 100644
--- a/packages/runtime/src/types.ts
+++ b/packages/runtime/src/types.ts
@@ -105,6 +105,8 @@ export type BuildStore = {
create(build: string, definition: BuildDefinition): Promise<BuildSnapshot>;
read(build: string): Promise<BuildSnapshot | undefined>;
append(build: string, fact: BuildFact): Promise<void>;
+ /** Drop execution material after the project Build Result has become authoritative. */
+ remove?(build: string): Promise<void>;
};
export type ScheduledBuild =
@@ -132,4 +134,6 @@ export type ScheduledBuildResult = {
export type BuildSchedulerOptions = {
/** Optional durable authority. When present, every admitted Core Fact is appended. */
readonly buildStore?: BuildStore;
+ /** Called after an accepted event changes the materialized Build view. */
+ readonly onStateChange?: (build: string, state: BuildState) => Promise<void>;
};
diff --git a/packages/store-sqlite/src/store.ts b/packages/store-sqlite/src/store.ts
index ee9ffae0..974778bc 100644
--- a/packages/store-sqlite/src/store.ts
+++ b/packages/store-sqlite/src/store.ts
@@ -169,6 +169,13 @@ class SqliteBuildStore implements BuildStore {
`).run(build, fact.command, JSON.stringify(fact));
if (result.changes !== 1) throw new Error(`Build ${build} Fact was not stored`);
}
+
+ async remove(build: string): Promise<void> {
+ transaction(this.#database, () => {
+ this.#database.prepare("DELETE FROM hypit_build_facts WHERE build_id = ?").run(build);
+ this.#database.prepare("DELETE FROM hypit_builds WHERE build_id = ?").run(build);
+ });
+ }
}
function parseCatalogEntry(row: Row): BuildCatalogEntry {
@@ -218,6 +225,10 @@ class SqliteBuildCatalog implements BuildCatalog {
`).all() as Row[];
return rows.map(parseCatalogEntry);
}
+
+ async remove(build: string): Promise<void> {
+ this.#database.prepare("DELETE FROM hypit_build_catalog WHERE build_id = ?").run(build);
+ }
}
class SqliteOperationStore implements OperationStore {
@@ -259,6 +270,10 @@ class SqliteOperationStore implements OperationStore {
return row === undefined ? undefined : parseOperationSnapshot(row);
}
+ async removeBuild(build: string): Promise<void> {
+ this.#database.prepare("DELETE FROM hypit_operations WHERE build_id = ?").run(build);
+ }
+
async list(query: OperationQuery): Promise<readonly OperationSnapshot[]> {
const predicates: string[] = [];
const values: string[] = [];
@@ -335,6 +350,7 @@ function parseDispatchSnapshot(row: Row): BuildDispatchSnapshot {
const snapshot = {
build: row.build_id,
componentPackages,
+ ...(typeof row.result_directory === "string" ? { resultDirectory: row.result_directory } : {}),
createdAt: row.created_at,
availableAt: row.available_at,
phase: row.phase,
@@ -360,10 +376,10 @@ class SqliteBuildDispatchStore implements BuildDispatchStore {
nonNegativeInteger(now, "Dispatch creation time");
this.#database.prepare(`
INSERT INTO hypit_dispatches (
- build_id, component_packages_json, created_at, available_at,
+ build_id, component_packages_json, result_directory, created_at, available_at,
phase, reason, cancel_requested, terminal
- ) VALUES (?, ?, ?, ?, 'queued', NULL, 0, NULL)
- `).run(request.build, JSON.stringify(request.componentPackages), now, now);
+ ) VALUES (?, ?, ?, ?, ?, 'queued', NULL, 0, NULL)
+ `).run(request.build, JSON.stringify(request.componentPackages), request.resultDirectory ?? null, now, now);
return {
...copy(request),
createdAt: now,
@@ -442,7 +458,8 @@ class SqliteBuildDispatchStore implements BuildDispatchStore {
const effective = current.cancellation === undefined ? terminal : "cancelled";
this.#database.prepare(`
UPDATE hypit_dispatches
- SET phase = 'terminal', terminal = ?, reason = ?
+ SET phase = 'terminal', terminal = ?, reason = ?,
+ component_packages_json = '[]', result_directory = NULL
WHERE build_id = ?
`).run(effective, reason ?? current.cancellation?.reason ?? null, build);
return parseDispatchSnapshot(
@@ -641,6 +658,7 @@ export class SqliteRuntimeState {
CREATE TABLE IF NOT EXISTS hypit_dispatches (
build_id TEXT PRIMARY KEY,
component_packages_json TEXT NOT NULL,
+ result_directory TEXT,
created_at INTEGER NOT NULL,
available_at INTEGER NOT NULL,
phase TEXT NOT NULL CHECK (phase IN ('queued', 'running', 'waiting', 'terminal')),
@@ -659,6 +677,13 @@ export class SqliteRuntimeState {
PRIMARY KEY (build_id, command_id)
) STRICT;
`);
+ if (!options.readOnly || emptyReadOnly) {
+ const dispatchColumns = new Set((this.#database.prepare("PRAGMA table_info(hypit_dispatches)").all() as Row[])
+ .flatMap((row) => typeof row.name === "string" ? [row.name] : []));
+ if (!dispatchColumns.has("result_directory")) {
+ this.#database.exec("ALTER TABLE hypit_dispatches ADD COLUMN result_directory TEXT");
+ }
+ }
this.builds = new SqliteBuildStore(database);
this.operations = new SqliteOperationStore(database);
this.dispatch = new SqliteBuildDispatchStore(database);
diff --git a/packages/workspace-fs-node/src/workspace.ts b/packages/workspace-fs-node/src/workspace.ts
index 264ef10c..dfa8f7e7 100644
--- a/packages/workspace-fs-node/src/workspace.ts
+++ b/packages/workspace-fs-node/src/workspace.ts
@@ -1,4 +1,4 @@
-import { createHash } from "node:crypto";
+import { createHash, randomUUID } from "node:crypto";
import { createReadStream } from "node:fs";
import { readFile, realpath } from "node:fs/promises";
import { basename, dirname, isAbsolute, relative, resolve } from "node:path";
@@ -26,7 +26,11 @@ class NodeFilesystemWorkspaceSession implements WorkspaceSession {
readonly entry: SourceUnit;
readonly #assetRoots: readonly string[];
readonly #sourceCache = new Map<string, SourceUnit>();
- readonly #assetIdentity = new Map<string, { readonly digest: BlobRef["digest"]; readonly size: number }>();
+ readonly #assetIdentity = new Map<string, {
+ readonly resource: string;
+ readonly digest: BlobRef["digest"];
+ readonly size: number;
+ }>();
readonly #attachments = new Map<string, ArtifactAttachment>();
private constructor(root: string, entry: SourceUnit, assetRoots: readonly string[]) {
@@ -126,11 +130,12 @@ class NodeFilesystemWorkspaceSession implements WorkspaceSession {
hash.update(chunk);
size += chunk.byteLength;
}
- identity = { digest: `sha256:${hash.digest("hex")}`, size };
+ identity = { resource: `res_${randomUUID()}`, digest: `sha256:${hash.digest("hex")}`, size };
this.#assetIdentity.set(canonical, identity);
}
const artifact: BlobRef = {
kind: "blob",
+ resource: identity.resource,
digest: identity.digest,
size: identity.size,
mediaType: request.mediaType,
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index b2cab32b..15b39378 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -30,6 +30,9 @@ importers:
'@hypit/background-removal':
specifier: workspace:*
version: link:packages/background-removal
+ '@hypit/build-result':
+ specifier: workspace:*
+ version: link:packages/build-result
'@hypit/caption':
specifier: workspace:*
version: link:packages/caption
@@ -437,6 +440,12 @@ importers:
specifier: workspace:*
version: link:../protocol
+ packages/build-result:
+ dependencies:
+ '@hypit/protocol':
+ specifier: workspace:*
+ version: link:../protocol
+
packages/caption:
dependencies:
'@hypit/component-kit':
@@ -534,6 +543,9 @@ importers:
packages/cli:
dependencies:
+ '@hypit/build-result':
+ specifier: workspace:*
+ version: link:../build-result
'@hypit/compiler-node':
specifier: workspace:*
version: link:../compiler-node
@@ -564,6 +576,16 @@ importers:
typescript:
specifier: 5.9.3
version: 5.9.3
+ devDependencies:
+ '@hypit/markup':
+ specifier: workspace:*
+ version: link:../markup
+ '@hypit/run-markup':
+ specifier: workspace:*
+ version: link:../run-markup
+ '@hypit/workspace-fs-node':
+ specifier: workspace:*
+ version: link:../workspace-fs-node
packages/comment-sticker:
dependencies:
@@ -2584,6 +2606,9 @@ importers:
packages/runtime-host-node:
dependencies:
+ '@hypit/build-result':
+ specifier: workspace:*
+ version: link:../build-result
'@hypit/protocol':
specifier: workspace:*
version: link:../protocol
@@ -2614,6 +2639,12 @@ importers:
packages/runtime-local:
dependencies:
+ '@hypit/artifact-store-fs':
+ specifier: workspace:*
+ version: link:../artifact-store-fs
+ '@hypit/build-result':
+ specifier: workspace:*
+ version: link:../build-result
'@hypit/component-kit':
specifier: workspace:*
version: link:../component-kit
@@ -2645,9 +2676,6 @@ importers:
specifier: workspace:*
version: link:../validation
devDependencies:
- '@hypit/artifact-store-fs':
- specifier: workspace:*
- version: link:../artifact-store-fs
'@hypit/core':
specifier: workspace:*
version: link:../core