Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

feat: auto-prepare and auto-start provider services

rpone526fffe45d55
diff --git a/docs/guide/providers.md b/docs/guide/providers.md
index d7ab5b57..aeff1f52 100644
--- a/docs/guide/providers.md
+++ b/docs/guide/providers.md
@@ -30,7 +30,7 @@ Provider packages depend on Runtime ports and the model families they serve, nev
"exports": {
".": "./src/index.ts"
},
- "svml": { "activation": "./src/activation.ts" },
+ "svml": { "activation": "./src/activation.ts", "service": true },
"dependencies": {
"@narratage/endpoint-kit": "workspace:*",
"@narratage/protocol": "workspace:*",
@@ -119,7 +119,43 @@ The `create` function constructs the Endpoint from Runtime config. The `doctor`
diagnostics (credential presence, executable availability) without constructing or calling
anything.
-## 5. Register and lock
+## 5. Declare an external service (if needed)
+
+If the Provider depends on an external program (a Python service, a local server), add
+`"service": true` to the `svml` block in `package.json` (shown in step 2) and export a factory
+from `src/service.ts`:
+
+```typescript
+// src/service.ts
+import type { RuntimeExternalService } from "@narratage/local";
+
+export function createMyExternalService(): RuntimeExternalService {
+ return {
+ id: "my-service",
+ prepare: { command: "uv", args: ["sync", "--project", "services/my-service", "--frozen"] },
+ start: { command: "uv", args: ["run", "--project", "services/my-service", "--frozen", "svml-my-service"] },
+ probe: async () => {
+ // Return { state: "ready" } or { state: "down", detail: "..." }
+ },
+ };
+}
+```
+
+Wire it into the activation descriptor by passing `service` to `createRuntimeEndpointAdapterFacet`:
+
+```typescript
+const adapter = createRuntimeEndpointAdapterFacet({
+ use: "@narratage/provider-my-service",
+ service: createMyExternalService,
+ // create, doctor …
+});
+```
+
+`pnpm install` will run `prepare` automatically, and `narratage build` will call `start` and
+`probe` before the first Operation. Providers that call only remote APIs (no local program) omit
+this step entirely — leave `"service"` out of `package.json`.
+
+## 6. Register and lock
Add the path mapping to `tsconfig.json`:
@@ -135,7 +171,7 @@ pnpm narratage lock-packages <runtime-lock> \
--root .
```
-## 6. Reference from svml.runtime.json
+## 7. Reference from svml.runtime.json
```json
{
diff --git a/docs/guide/services.md b/docs/guide/services.md
index 95fa1267..3b116718 100644
--- a/docs/guide/services.md
+++ b/docs/guide/services.md
@@ -8,13 +8,41 @@ description: Setting up the WhisperX and OpenCV services.
Two Python services support local Builds. They are Runtime deployment packages, not part
of Core or the author language.
+## Automatic setup
+
+You normally do not run any of the commands on this page. `pnpm install` prepares every service
+that an installed Provider declares, and `narratage build` starts them:
+
+```bash
+pnpm install # prepares the Python environments
+pnpm narratage build build.svrun --runtime svml.runtime.json --follow
+```
+
+A Build starts only the services its Runtime Profile declares, leaves them running between Builds
+so a warm model is not reloaded, and stops before its first Operation if one cannot be reached —
+so an unavailable service never costs a paid generation.
+
+Manage them directly when you need to:
+
+```bash
+pnpm narratage services status svml.runtime.json # what is running
+pnpm narratage services up svml.runtime.json # start without building
+pnpm narratage services down svml.runtime.json # stop what this project started
+pnpm narratage build … --no-services # build against what is already running
+```
+
+`pnpm install` skips preparation when `uv` is absent and says so; it never fails the install. The
+rest of this page is what those commands run, for when one of them fails or you are deploying
+outside this repository.
+
## WhisperX
Provides ASR and language-specific alignment for speech timing measurement.
-### Install
+### Install by hand
-WhisperX 3.8.6 supports Python 3.10–3.13. The checked-in lock selects 3.13:
+`pnpm install` runs the first three of these. WhisperX 3.8.6 supports Python 3.10–3.13, and the
+checked-in lock selects 3.13:
```bash
uv python install 3.13
@@ -30,7 +58,10 @@ inference.
The first model start may download ASR and alignment weights. Production should put the Hugging
Face cache on persistent storage.
-### Run
+### Run by hand
+
+`narratage build` and `narratage services up` run this for you, logging to
+`.svml/services/whisperx.log`. To run it in the foreground instead:
```bash
uv run --project services/whisperx --frozen svml-whisperx-service
@@ -71,7 +102,13 @@ pnpm test:whisperx-service
Provides bounded image transforms (e.g. the GPT Image YCrCb denoise preset) through
`@narratage/provider-image-opencv-local`.
-### Install
+OpenCV runs as one bounded process per Need, so there is no program to keep warm — preparing the
+environment is the whole job, and `narratage services status` reports whether the interpreter
+carries a usable `cv2` and `numpy`.
+
+### Install by hand
+
+`pnpm install` runs this:
```bash
uv python install 3.13
diff --git a/docs/quickstart/run.md b/docs/quickstart/run.md
index 856ee76c..6fa74c1b 100644
--- a/docs/quickstart/run.md
+++ b/docs/quickstart/run.md
@@ -245,6 +245,19 @@ prevent one type of work from starving others.
## Build workflow
+### 0. Install
+
+```bash
+pnpm install
+```
+
+This installs Node dependencies **and** prepares the Python services (WhisperX, OpenCV) that local
+Builds need. When `uv` is not on your PATH the Python step is skipped with a warning — install
+[uv](https://docs.astral.sh/uv/) first if you need local alignment or image processing.
+
+`narratage build` will start the required services automatically; you do not need to launch them
+yourself.
+
### 1. Diagnose the environment
```bash
diff --git a/docs/zh/guide/providers.md b/docs/zh/guide/providers.md
index e32fee0c..7517dd75 100644
--- a/docs/zh/guide/providers.md
+++ b/docs/zh/guide/providers.md
@@ -28,7 +28,7 @@ Provider 包依赖 Runtime 端口以及它所服务的模型族,绝不依赖 C
"exports": {
".": "./src/index.ts"
},
- "svml": { "activation": "./src/activation.ts" },
+ "svml": { "activation": "./src/activation.ts", "service": true },
"dependencies": {
"@narratage/endpoint-kit": "workspace:*",
"@narratage/protocol": "workspace:*",
@@ -114,7 +114,42 @@ export default svmlPackage;
`create` 函数根据 Runtime 配置构造 Endpoint。`doctor` 函数返回诊断信息(凭据是否存在、可执行文件是否可用),不构造也不调用任何东西。
-## 5. 注册并锁定
+## 5. 声明外部服务(如需要)
+
+如果 Provider 依赖外部程序(Python 服务、本地服务器),在 `package.json` 的 `svml` 块中添加
+`"service": true`(见第 2 步),并在 `src/service.ts` 中导出工厂函数:
+
+```typescript
+// src/service.ts
+import type { RuntimeExternalService } from "@narratage/local";
+
+export function createMyExternalService(): RuntimeExternalService {
+ return {
+ id: "my-service",
+ prepare: { command: "uv", args: ["sync", "--project", "services/my-service", "--frozen"] },
+ start: { command: "uv", args: ["run", "--project", "services/my-service", "--frozen", "svml-my-service"] },
+ probe: async () => {
+ // 返回 { state: "ready" } 或 { state: "down", detail: "..." }
+ },
+ };
+}
+```
+
+在 activation 描述符中通过 `service` 参数接入:
+
+```typescript
+const adapter = createRuntimeEndpointAdapterFacet({
+ use: "@narratage/provider-my-service",
+ service: createMyExternalService,
+ // create, doctor …
+});
+```
+
+`pnpm install` 会自动执行 `prepare`,`narratage build` 会在第一个 Operation 之前调用 `start`
+和 `probe`。只调用远程 API 的 Provider(没有本地程序)跳过这一步——`package.json` 中不写
+`"service"`。
+
+## 6. 注册并锁定
在 `tsconfig.json` 中添加路径映射:
@@ -130,7 +165,7 @@ pnpm narratage lock-packages <runtime-lock> \
--root .
```
-## 6. 在 svml.runtime.json 中引用
+## 7. 在 svml.runtime.json 中引用
```json
{
diff --git a/docs/zh/guide/services.md b/docs/zh/guide/services.md
index 01c93101..f2165dfe 100644
--- a/docs/zh/guide/services.md
+++ b/docs/zh/guide/services.md
@@ -1,19 +1,44 @@
---
title: 本地服务
-description: 配置 WhisperX 与 OpenCV 边车服务。
+description: 配置 WhisperX 与 OpenCV 服务。
---
# 本地服务
-两个 Python 边车服务为本地 Build 提供支持。它们属于 Runtime 部署包,不属于 Core,也不属于作者语言。
+两个 Python 服务为本地 Build 提供支持。它们属于 Runtime 部署包,不属于 Core,也不属于作者语言。
+
+## 自动配置
+
+通常你不需要手动执行本页的任何命令。`pnpm install` 会准备好所有已安装 Provider 声明的服务,
+`narratage build` 会自动启动它们:
+
+```bash
+pnpm install # 准备 Python 环境
+pnpm narratage build build.svrun --runtime svml.runtime.json --follow
+```
+
+Build 只启动其 Runtime Profile 声明的服务,并在 Build 之间保持运行以避免重新加载热模型。
+如果某个服务无法连通,Build 会在第一个 Operation 之前停止——不会浪费付费生成。
+
+需要时可以直接管理服务:
+
+```bash
+pnpm narratage services status svml.runtime.json # 查看运行状态
+pnpm narratage services up svml.runtime.json # 只启动,不 Build
+pnpm narratage services down svml.runtime.json # 停止本项目启动的服务
+pnpm narratage build … --no-services # 跳过自动启动,使用已运行的服务
+```
+
+当 `uv` 不在 PATH 中时,`pnpm install` 会跳过准备并给出提示,不会导致安装失败。
+本页下面的内容是这些命令实际执行的操作,供某个步骤失败或在仓库外部署时参考。
## WhisperX
提供 ASR 以及针对特定语言的对齐能力,用于测量语音时序。
-### 安装
+### 手动安装
-WhisperX 3.8.6 支持 Python 3.10–3.13。仓库中已签入的锁文件选定 3.13:
+`pnpm install` 会自动执行前三条命令。WhisperX 3.8.6 支持 Python 3.10–3.13,仓库中的锁文件选定 3.13:
```bash
uv python install 3.13
@@ -22,11 +47,15 @@ uv run --project services/whisperx --frozen svml-whisperx-prepare
uv run --project services/whisperx --frozen svml-whisperx-check
```
-`svml-whisperx-prepare` 会在完成 SHA-256 校验后,从固定版本的归档中安装 NLTK 的 `punkt_tab` 句子数据。WhisperX 的对齐流程需要这份数据,并且它绝不会在推理过程中下载。
+`svml-whisperx-prepare` 会在完成 SHA-256 校验后,从固定版本的归档中安装 NLTK 的 `punkt_tab`
+句子数据。WhisperX 的对齐流程需要这份数据,推理过程中不会下载。
首次启动模型时可能会下载 ASR 与对齐权重。生产环境应将 Hugging Face 缓存放在持久化存储上。
-### 运行
+### 手动运行
+
+`narratage build` 和 `narratage services up` 会自动运行此服务,日志写入
+`.svml/services/whisperx.log`。如需前台运行:
```bash
uv run --project services/whisperx --frozen svml-whisperx-service
@@ -49,7 +78,8 @@ curl http://127.0.0.1:8765/health
### 并发
-该服务同一时刻只接纳一次推理。第二个请求会收到 `503 BUSY`。并发由 Narratage Runtime Scheduler 的 lane 配置管控,而不是由服务自身管控。
+该服务同一时刻只接纳一次推理。第二个请求会收到 `503 BUSY`。并发由 Narratage Runtime Scheduler
+的 lane 配置管控,而不是由服务自身管控。
Node Provider(`@narratage/provider-whisperx-local`)必须配置与之匹配的模型、设备、计算类型和批大小。一旦不匹配,会在结果被接受之前失败。
@@ -61,9 +91,15 @@ pnpm test:whisperx-service
## OpenCV 图像服务
-通过 `@narratage/provider-image-opencv-local` 提供有上界的图像变换(例如 GPT Image 的 YCrCb 降噪预设)。
+通过 `@narratage/provider-image-opencv-local` 提供有上界的图像变换(例如 GPT Image 的 YCrCb
+降噪预设)。
+
+OpenCV 以每个 Need 一个有界进程运行,没有需要保持热加载的程序——准备好环境就是全部工作,
+`narratage services status` 会报告解释器是否携带可用的 `cv2` 和 `numpy`。
+
+### 手动安装
-### 安装
+`pnpm install` 会自动执行以下命令:
```bash
uv python install 3.13
diff --git a/docs/zh/quickstart/run.md b/docs/zh/quickstart/run.md
index 7280e347..4b292188 100644
--- a/docs/zh/quickstart/run.md
+++ b/docs/zh/quickstart/run.md
@@ -233,6 +233,18 @@ Runtime Profile(`svml.runtime.json`)告诉系统**在哪里**执行每种类
## Build 工作流
+### 0. 安装
+
+```bash
+pnpm install
+```
+
+这条命令在安装 Node 依赖的同时,也会准备好本地 Build 所需的 Python 服务(WhisperX、OpenCV)。
+如果 `uv` 不在 PATH 中,Python 步骤会跳过并给出提示——需要本地对齐或图像处理时请先安装
+[uv](https://docs.astral.sh/uv/)。
+
+`narratage build` 会自动启动所需的服务,无需手动启动。
+
### 1. 诊断环境
```bash
diff --git a/examples/talking-film-live/svml.runtime-packages.lock b/examples/talking-film-live/svml.runtime-packages.lock
index ad062cc8..9a39c0d7 100644
--- a/examples/talking-film-live/svml.runtime-packages.lock
+++ b/examples/talking-film-live/svml.runtime-packages.lock
@@ -39,7 +39,7 @@
{
"name": "@narratage/caption",
"version": "0.0.0-dev",
- "digest": "sha256:2d6af94e8482e2643d4bd4d3f6b5f26617d3d0598ed426ad2cd3bbea7fbf3e2c"
+ "digest": "sha256:81e7e401a34856a5c0bdaeb0b14a587dccd90b89d06cf2c01c4234c8addfad6c"
},
{
"name": "@narratage/component-kit",
@@ -66,25 +66,10 @@
"version": "0.0.0-dev",
"digest": "sha256:2f377c58c0dbec426d90adcb4794356507b3b4c55eb3d2533f54b44fcc20be03"
},
- {
- "name": "@narratage/gemini-omni",
- "version": "0.0.0-dev",
- "digest": "sha256:81a6cf3606f087085eb539a107a2d0f944a10bfd54ac6b8d0c5eae540d73b252"
- },
{
"name": "@narratage/generation",
"version": "0.0.0-dev",
- "digest": "sha256:280b7f009d0f5c93c171cbf167d21e5ee2cf415fbb0ce885f1b9fa05b1833c29"
- },
- {
- "name": "@narratage/gpt-image",
- "version": "0.0.0-dev",
- "digest": "sha256:03888009b4ecf8be1c31ce0b8e54a5e8ba72073828833ea60ba8975f89ce6809"
- },
- {
- "name": "@narratage/grok-imagine",
- "version": "0.0.0-dev",
- "digest": "sha256:3aa49200686fb185442e52871f1e32ec58ce733e30567cc929fe39b92e91439f"
+ "digest": "sha256:a9d0db9e25f2667602a3cc8735d6317b5ea821a187739d75c4f2c24d06a91381"
},
{
"name": "@narratage/host",
@@ -96,6 +81,11 @@
"version": "0.0.0-dev",
"digest": "sha256:ce3902f4d69c5057a1795fd2d201c189716436c8501f8790d6b1df4ebd2fd691"
},
+ {
+ "name": "@narratage/media-execution",
+ "version": "0.0.0-dev",
+ "digest": "sha256:5caac585710623bfa7435b5f26b9973c70a8dbadb960cc1f83c6003129a349fa"
+ },
{
"name": "@narratage/media-pipeline",
"version": "0.0.0-dev",
@@ -106,25 +96,10 @@
"version": "0.0.0-dev",
"digest": "sha256:d92f53d5060df53a7a671b58b8c0d1305be99384bd34e71bda0f7f1296550f4b"
},
- {
- "name": "@narratage/minimax-h3",
- "version": "0.0.0-dev",
- "digest": "sha256:2662e852ca9d8c61cefdf33895220c7adeedfede13973513e4c9ae3e5940b4de"
- },
- {
- "name": "@narratage/model-kit",
- "version": "0.0.0-dev",
- "digest": "sha256:195208129c357c8b48dd38977557a8c9f87e3bbd3c162e2e51e17db0f56bf874"
- },
- {
- "name": "@narratage/nano-banana",
- "version": "0.0.0-dev",
- "digest": "sha256:254cd027bfc5937a5d09865f574f7512bfcc2070056e842068061f1fc61cce6f"
- },
{
"name": "@narratage/narrative",
"version": "0.0.0-dev",
- "digest": "sha256:866d6570d7fd659cd56540c0be3a28ad43b7de68be464a77591f6e14d8b61564"
+ "digest": "sha256:85e98e61021310c2dd6473189dcbcc9daa54039277169cbb5ed51ed70277a894"
},
{
"name": "@narratage/program-space",
@@ -149,17 +124,17 @@
{
"name": "@narratage/provider-kie",
"version": "0.0.0-dev",
- "digest": "sha256:1333e4b33f80224db0052765ce0f3a9c3f47c9bc03eede4becaea53d5d15f24a"
+ "digest": "sha256:b202786d05553c232855c9273433fb924a15f501941a2d338b0757b24eea3a60"
},
{
"name": "@narratage/provider-media-local",
"version": "0.0.0-dev",
- "digest": "sha256:66ad1aa489a466112f9177ad285de9baf9ce4e8155dba26a8bfdfbbc2aa35be4"
+ "digest": "sha256:e66c1f86f72eb7c5ff7be73e54ac744e1d63c1a8a186b2374dd225e8f98192b9"
},
{
"name": "@narratage/provider-whisperx-local",
"version": "0.0.0-dev",
- "digest": "sha256:e0cf081b8713a757f71d96aa6e7587bdeeb1513ba5d06f747f69c8d82c732a35"
+ "digest": "sha256:86c0b55abfbcf896330d020693d03e85fb5b8d31123b6383fb5b9505a1abf4ef"
},
{
"name": "@narratage/render-hyperframes",
@@ -174,27 +149,17 @@
{
"name": "@narratage/runtime-adapter",
"version": "0.0.0-dev",
- "digest": "sha256:c8871a16a909ec7259180ca2f8db03abb38977675f545dd2b8ec7389b8c8a0ed"
+ "digest": "sha256:532f529daa0334c393c94faa99268e9f216b1e3fb760440e8d4f9b9b1e3ce324"
},
{
"name": "@narratage/runtime",
"version": "0.0.0-dev",
"digest": "sha256:3d5bd79233f3f78633b427740e2efa4ce2720195a91cd50e0ced94f054471c2a"
},
- {
- "name": "@narratage/seedance",
- "version": "0.0.0-dev",
- "digest": "sha256:872fe63f9a1d36d071aec4baa4764cc14e99e807a21a446f0a19788ef37a48ec"
- },
- {
- "name": "@narratage/seedream",
- "version": "0.0.0-dev",
- "digest": "sha256:6ca275e237cd04b0a15d10c690abfe54d08b1cbe458c45bb564a3af1a9aebc7b"
- },
{
"name": "@narratage/semantic-map",
"version": "0.0.0-dev",
- "digest": "sha256:801ede25442f28c00bbe44794b0aa1ae6dd129faa9b46ca9a759782d9cb71c84"
+ "digest": "sha256:4293518a23ead1320f81d5d38e60bef96ea7f27149495bc890b9e4c275572864"
},
{
"name": "@narratage/source",
@@ -204,7 +169,7 @@
{
"name": "@narratage/speech-alignment",
"version": "0.0.0-dev",
- "digest": "sha256:7c3b7403c0de80a4c0fb6d2bbf5722519d71605293956870e5d23a5fe6abd7a5"
+ "digest": "sha256:98c3158c130b90b443283f19c770f358ed81caa915a8e741a699f8ffe3ea9f6d"
},
{
"name": "@narratage/speech-evidence",
@@ -239,7 +204,7 @@
{
"name": "@narratage/whisperx",
"version": "0.0.0-dev",
- "digest": "sha256:72fd08de6dca9bbec329d4f880e176832fba7ce1b89e0992be9bf4bcd87f2b49"
+ "digest": "sha256:92683f7866c852eda0a71c8935a522bbc2579219ebde680b289eabb39f51f77a"
},
{
"name": "@protobufjs/aspromise",
@@ -469,7 +434,7 @@
{
"name": "esbuild",
"version": "0.25.12",
- "digest": "sha256:d45f05635279ab74b586ef526256666bf06353c35733ba7265669098f7b0f221"
+ "digest": "sha256:d428001f327707b220f882523c466e93d30edd744a796697f5c80141d418e520"
},
{
"name": "escalade",
@@ -890,7 +855,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:b2636492ab0b0b500d674899ab2ff14fc61c7f2b22ef44eb8f816f141b21ec4d",
- "closureDigest": "sha256:d042bcb227861027fe461cbc064dcff16f9745944a65829dde382287359d5c33"
+ "closureDigest": "sha256:e56b87f20fc177aebaf4f99c374f7c97363384c2ad9c98f6548342fa5b465c45"
},
{
"specifier": "@narratage/provider-hyperframes-local",
@@ -899,7 +864,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:b14b721bdc0c9d5d9ced1c345107c92a3a5878f3383dea2ecd3e310442c2a40c",
- "closureDigest": "sha256:a2da855432ae1441f10b74ee5cecdcd57e6493e3e527dc02e5c51eb14d12535b"
+ "closureDigest": "sha256:38470648eb8b9d5c44db4ec0676d90fe9e493737e94f61cbe26f00445dd8f799"
},
{
"specifier": "@narratage/provider-kie",
@@ -908,7 +873,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:e61ebb911a33d4c5d0e4e4ee4fb69cdb790f4ced050ef35f7472bfbd72013d9b",
- "closureDigest": "sha256:af79dec340e8f700efbe31c8925af468bab09bd5153831ba89516f8b96193eed"
+ "closureDigest": "sha256:fca0be1ad55d4f5cd6c871d7004865bb6339213d7678ac3b6e795adf0a77c978"
},
{
"specifier": "@narratage/provider-media-local",
@@ -917,7 +882,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:701abcc923bf8a711c20b6e308d8cc8be27bf560f249b8c5a801e1dbd9cb3979",
- "closureDigest": "sha256:64953a2feb5ca13f0f6f8ffa33aeca2e7427e226ce0704168e3cf7e8658e6ff4"
+ "closureDigest": "sha256:b66f4699551aa39acb16a6586dd5921431d2bfa25f6921ac3320b46453fd52ba"
},
{
"specifier": "@narratage/provider-whisperx-local",
@@ -926,8 +891,8 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:3686b84c42cc2b1bcaccf57cbdcda25a15c7d9d3778fb92d25437f9880ed8f5d",
- "closureDigest": "sha256:723c10f0112c7b9d2dff11ba214ae3d595bfcebdf09644703401972818375de0"
+ "closureDigest": "sha256:113812e1354a85f210277b1324d653e8d10984dd9c0d9621f5325d856e317fd2"
}
],
- "digest": "sha256:e04400ffa7fc06fee8659853b74468b9cb300c0d07758f5c4d3b5480a53e35c1"
+ "digest": "sha256:b18d6f2423e1336b5701b775c5e2c523466d051bb7283912459a6a6993630228"
}
diff --git a/examples/talking-head-aroll/svml.runtime-packages.lock b/examples/talking-head-aroll/svml.runtime-packages.lock
index 9cd1f2e0..9a39c0d7 100644
--- a/examples/talking-head-aroll/svml.runtime-packages.lock
+++ b/examples/talking-head-aroll/svml.runtime-packages.lock
@@ -81,6 +81,11 @@
"version": "0.0.0-dev",
"digest": "sha256:ce3902f4d69c5057a1795fd2d201c189716436c8501f8790d6b1df4ebd2fd691"
},
+ {
+ "name": "@narratage/media-execution",
+ "version": "0.0.0-dev",
+ "digest": "sha256:5caac585710623bfa7435b5f26b9973c70a8dbadb960cc1f83c6003129a349fa"
+ },
{
"name": "@narratage/media-pipeline",
"version": "0.0.0-dev",
@@ -124,12 +129,12 @@
{
"name": "@narratage/provider-media-local",
"version": "0.0.0-dev",
- "digest": "sha256:66ad1aa489a466112f9177ad285de9baf9ce4e8155dba26a8bfdfbbc2aa35be4"
+ "digest": "sha256:e66c1f86f72eb7c5ff7be73e54ac744e1d63c1a8a186b2374dd225e8f98192b9"
},
{
"name": "@narratage/provider-whisperx-local",
"version": "0.0.0-dev",
- "digest": "sha256:35b536fe2e6752d4cc92ca368a318214e8e56e0d8356eba56a049d3a626543c5"
+ "digest": "sha256:86c0b55abfbcf896330d020693d03e85fb5b8d31123b6383fb5b9505a1abf4ef"
},
{
"name": "@narratage/render-hyperframes",
@@ -429,7 +434,7 @@
{
"name": "esbuild",
"version": "0.25.12",
- "digest": "sha256:d45f05635279ab74b586ef526256666bf06353c35733ba7265669098f7b0f221"
+ "digest": "sha256:d428001f327707b220f882523c466e93d30edd744a796697f5c80141d418e520"
},
{
"name": "escalade",
@@ -859,7 +864,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:b14b721bdc0c9d5d9ced1c345107c92a3a5878f3383dea2ecd3e310442c2a40c",
- "closureDigest": "sha256:b86d3e37edbdbc3d64cb494897b02697a35e5098e80ce99cf19a87b1f22a6f4b"
+ "closureDigest": "sha256:38470648eb8b9d5c44db4ec0676d90fe9e493737e94f61cbe26f00445dd8f799"
},
{
"specifier": "@narratage/provider-kie",
@@ -877,7 +882,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:701abcc923bf8a711c20b6e308d8cc8be27bf560f249b8c5a801e1dbd9cb3979",
- "closureDigest": "sha256:2b8ffcc53158623170425aa94442671e58c167e210e7413e158fce2cdac87db5"
+ "closureDigest": "sha256:b66f4699551aa39acb16a6586dd5921431d2bfa25f6921ac3320b46453fd52ba"
},
{
"specifier": "@narratage/provider-whisperx-local",
@@ -886,8 +891,8 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:3686b84c42cc2b1bcaccf57cbdcda25a15c7d9d3778fb92d25437f9880ed8f5d",
- "closureDigest": "sha256:d82fe2de72e4ce6fee4f660552b12485617b4254289a0ee4fd50e72a492f44e8"
+ "closureDigest": "sha256:113812e1354a85f210277b1324d653e8d10984dd9c0d9621f5325d856e317fd2"
}
],
- "digest": "sha256:680b29750d6c64627622b37eb9975077d09ba616093170bb08b65b43bbe2947e"
+ "digest": "sha256:b18d6f2423e1336b5701b775c5e2c523466d051bb7283912459a6a6993630228"
}
diff --git a/package.json b/package.json
index ac356656..eeaf7506 100644
--- a/package.json
+++ b/package.json
@@ -5,6 +5,7 @@
"type": "module",
"scripts": {
"check": "tsc -p tsconfig.json --noEmit",
+ "postinstall": "node tools/postinstall-services.mjs",
"docs:dev": "vitepress dev docs",
"docs:build": "vitepress build docs",
"docs:preview": "vitepress preview docs",
diff --git a/packages/cli/src/main.ts b/packages/cli/src/main.ts
index cf4e72ae..76913201 100644
--- a/packages/cli/src/main.ts
+++ b/packages/cli/src/main.ts
@@ -1,7 +1,7 @@
import { dirname, extname, resolve } from "node:path";
import { pathToFileURL } from "node:url";
-import type { LocalRuntime } from "@narratage/local";
+import type { ExternalServiceReport, LocalRuntime } from "@narratage/local";
import type { NodeCompiledSourceClosure } from "@narratage/compiler-node";
import type { BuildCatalogDescriptor } from "@narratage/runtime";
import { parseSourceHeader } from "@narratage/source";
@@ -48,6 +48,8 @@ type ParsedArgs = {
readonly to: string | undefined;
readonly pins: readonly { readonly output: string; readonly build: string }[];
readonly apply: boolean;
+ /** Leave the declared external programs alone; build against what is running. */
+ readonly noServices: boolean;
};
function parseArgs(argv: readonly string[]): ParsedArgs {
@@ -72,6 +74,7 @@ function parseArgs(argv: readonly string[]): ParsedArgs {
let to: string | undefined;
const pins: { output: string; build: string }[] = [];
let apply = false;
+ let noServices = false;
for (let index = 0; index < rest.length; index += 1) {
const item = rest[index];
if (item === "--target") {
@@ -175,6 +178,10 @@ function parseArgs(argv: readonly string[]): ParsedArgs {
apply = true;
continue;
}
+ if (item === "--no-services") {
+ noServices = true;
+ continue;
+ }
if (item === "--max-wait-ms") {
const value = rest[index + 1];
if (value === undefined || value.startsWith("--")) throw new Error("--max-wait-ms requires milliseconds");
@@ -207,6 +214,7 @@ function parseArgs(argv: readonly string[]): ParsedArgs {
to,
pins,
apply,
+ noServices,
};
}
@@ -219,7 +227,7 @@ function usage(): string {
" narratage gc <runtime-profile.json> [--apply]",
" narratage check <self-described-source> [--runtime profile.json] [--package-lock file] [--root directory]",
" narratage plan <run-source> [--runtime profile.json] [--package-lock file]",
- " narratage build <run-source> --runtime profile.json|./svml.runtime.ts [--follow]",
+ " narratage build <run-source> --runtime profile.json|./svml.runtime.ts [--follow] [--no-services]",
" narratage status <build-id> --runtime profile.json|./svml.runtime.ts",
" narratage builds --runtime profile.json|./svml.runtime.ts",
" narratage inspect <build-id> --runtime profile.json|./svml.runtime.ts",
@@ -268,6 +276,42 @@ function createCatalogDescriptor(options: {
};
}
+/**
+ * Starts the external programs the Runtime Profile's Endpoints declare, before
+ * the Build reaches an Operation that would need one. A WhisperX that is not
+ * running otherwise surfaces as a refused connection partway through, after the
+ * paid generation ahead of it has already been spent.
+ *
+ * Only a JSON Profile declares Endpoints this can read. A trusted `.ts` module
+ * builds its Runtime itself and owns whatever its Endpoints talk to, so there is
+ * nothing here to start on its behalf.
+ *
+ * Services are started and left running: a developer submits several Builds
+ * against one warm program, and stopping it between them would pay the model
+ * load every time. `narratage services down` ends them.
+ */
+async function startDeclaredServices(
+ path: string,
+ distribution: CliDistribution,
+): Promise<readonly ExternalServiceReport[] | undefined> {
+ if (extname(path) !== ".json") return undefined;
+ const result = await distribution.externalServices.up(resolve(path), {});
+ const unavailable = result.services.filter((item) => item.state.state !== "ready");
+ if (unavailable.length > 0) {
+ throw new Error([
+ `${unavailable.length} external service${unavailable.length === 1 ? " is" : "s are"} not ready:`,
+ // The report carries two reasons that do not overlap: what the probe saw,
+ // and why the attempt to fix it fell short. Both name a different repair.
+ ...unavailable.map((item) => ` ${item.id}: ${item.state.state}`
+ + `${"detail" in item.state ? ` — ${item.state.detail}` : ""}`
+ + `${item.detail === undefined ? "" : `; ${item.detail}`}`
+ + `${item.logPath === undefined ? "" : ` (log: ${item.logPath})`}`),
+ "Fix the service, or pass --no-services to build against what is already running.",
+ ].join("\n"));
+ }
+ return result.services;
+}
+
async function loadLocalRuntime(path: string, distribution: CliDistribution): Promise<LocalRuntime> {
if (extname(path) === ".json") return await distribution.createRuntimeFromConfig(path);
// A Runtime config is trusted executable deployment code, never an Author Frontend or .svml import.
@@ -366,6 +410,9 @@ export async function runCli(
return;
}
if (args.apply) throw new Error("--apply is only valid for gc");
+ if (args.noServices && args.command !== "build") {
+ throw new Error("--no-services is only valid for build; no other command starts an external program");
+ }
if ((args.record !== undefined || args.output !== undefined || args.name !== undefined
|| args.artifact !== undefined || args.to !== undefined)
&& args.command !== "get") {
@@ -568,6 +615,7 @@ export async function runCli(
}
if (args.command === "build") {
if (args.runtime === undefined) throw new Error("build requires --runtime with a Runtime Profile or trusted local config module");
+ const services = args.noServices ? undefined : await startDeclaredServices(args.runtime, distribution);
const runtime = await loadLocalRuntime(args.runtime, distribution);
try {
const loadedRun = await loadRunFile({
@@ -599,6 +647,9 @@ export async function runCli(
build: built.id,
core: built.state.id,
status: built.status,
+ ...(services === undefined ? {} : {
+ services: services.map((item) => ({ id: item.id, action: item.action })),
+ }),
goals: built.state.plan.goals.map((goal) => {
const record = built.state.records.find((item) => item.id === goal.record);
return {
diff --git a/packages/cli/test/build-services.test.ts b/packages/cli/test/build-services.test.ts
new file mode 100644
index 00000000..e5961ea5
--- /dev/null
+++ b/packages/cli/test/build-services.test.ts
@@ -0,0 +1,124 @@
+import assert from "node:assert/strict";
+import { mkdtemp, writeFile } from "node:fs/promises";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import test from "node:test";
+
+import { runCli } from "../src/main.js";
+import type { CliDistribution, ExternalServiceResult } from "../src/distribution.js";
+
+const io = { write: () => {} };
+
+/**
+ * Enough of a Distribution to reach the build handler: a Run Frontend the
+ * source header names, and a compiler that hands back that source and claims no
+ * Author Frontend, so the CLI takes the Run path.
+ */
+function distribution(
+ calls: string[],
+ services: ExternalServiceResult["services"],
+): CliDistribution {
+ return {
+ name: "test",
+ builtInPackageContributions: [],
+ runFrontends: [{ id: "@narratage/run-text@1" }],
+ createCompiler: () => ({
+ openFile: async (path: string) => ({
+ entry: { name: path, text: '<?svml using="@narratage/run-text@1"?>\n<svrun/>\n' },
+ }),
+ supportsFrontend: () => false,
+ }),
+ externalServices: {
+ up: async (path: string) => {
+ calls.push(`up ${path}`);
+ return { root: "/tmp", services };
+ },
+ down: async () => ({ root: "/tmp", services: [] }),
+ report: async () => ({ root: "/tmp", services: [] }),
+ },
+ } as unknown as CliDistribution;
+}
+
+async function runSource(): Promise<string> {
+ const root = await mkdtemp(join(tmpdir(), "svml-build-services-"));
+ const path = join(root, "build.svrun");
+ await writeFile(path, '<?svml using="@narratage/run-text@1"?>\n<svrun/>\n', "utf8");
+ return path;
+}
+
+test("a declared service that is not ready stops the Build before it spends anything", async () => {
+ const calls: string[] = [];
+ const source = await runSource();
+ await assert.rejects(
+ async () => await runCli(
+ ["build", source, "--runtime", "/p/svml.runtime.json"],
+ io,
+ distribution(calls, [{
+ id: "whisperx",
+ instances: ["whisperx.local"],
+ action: "unchanged",
+ state: { state: "down", detail: "nothing is answering at http://127.0.0.1:8765" },
+ detail: "uv failed: no such project",
+ }]),
+ ),
+ // Both reasons reach the operator: what the probe saw, and why the attempt
+ // to fix it fell short. Each names a different repair.
+ (error: Error) => /1 external service is not ready/u.test(error.message)
+ && /whisperx: down/u.test(error.message)
+ && /nothing is answering/u.test(error.message)
+ && /uv failed: no such project/u.test(error.message)
+ && /--no-services/u.test(error.message),
+ );
+ assert.deepEqual(calls, ["up /p/svml.runtime.json"]);
+});
+
+test("--no-services leaves the declared programs alone", async () => {
+ const calls: string[] = [];
+ const source = await runSource();
+ // Reaching the Runtime is the boundary just past the gate: the stub has no
+ // createRuntimeFromConfig, so arriving there proves the down service was
+ // never consulted rather than merely tolerated.
+ await assert.rejects(
+ async () => await runCli(
+ ["build", source, "--runtime", "/p/svml.runtime.json", "--no-services"],
+ io,
+ distribution(calls, [{
+ id: "whisperx",
+ instances: ["whisperx.local"],
+ state: { state: "down", detail: "nothing is answering" },
+ }]),
+ ),
+ /createRuntimeFromConfig/u,
+ );
+ assert.deepEqual(calls, []);
+});
+
+test("a Build whose services are all ready proceeds to compile", async () => {
+ const calls: string[] = [];
+ const source = await runSource();
+ await assert.rejects(
+ async () => await runCli(
+ ["build", source, "--runtime", "/p/svml.runtime.json"],
+ io,
+ distribution(calls, [{
+ id: "whisperx",
+ instances: ["whisperx.local"],
+ action: "already-running",
+ state: { state: "ready" },
+ }]),
+ ),
+ /createRuntimeFromConfig/u,
+ );
+ assert.deepEqual(calls, ["up /p/svml.runtime.json"]);
+});
+
+test("--no-services belongs to build, the only command that starts a program", async () => {
+ await assert.rejects(
+ async () => await runCli(
+ ["plan", "/p/build.svrun", "--no-services"],
+ io,
+ distribution([], []),
+ ),
+ /--no-services is only valid for build/u,
+ );
+});
diff --git a/packages/provider-image-opencv-local/package.json b/packages/provider-image-opencv-local/package.json
index b4f1838b..29d223da 100644
--- a/packages/provider-image-opencv-local/package.json
+++ b/packages/provider-image-opencv-local/package.json
@@ -7,7 +7,8 @@
".": "./src/index.ts"
},
"svml": {
- "activation": "./src/activation.ts"
+ "activation": "./src/activation.ts",
+ "service": true
},
"dependencies": {
"@narratage/artifact": "workspace:*",
diff --git a/packages/provider-whisperx-local/package.json b/packages/provider-whisperx-local/package.json
index f449c501..c73510fe 100644
--- a/packages/provider-whisperx-local/package.json
+++ b/packages/provider-whisperx-local/package.json
@@ -7,7 +7,8 @@
".": "./src/index.ts"
},
"svml": {
- "activation": "./src/activation.ts"
+ "activation": "./src/activation.ts",
+ "service": true
},
"dependencies": {
"@narratage/endpoint-kit": "workspace:*",
diff --git a/tools/postinstall-services.mjs b/tools/postinstall-services.mjs
new file mode 100644
index 00000000..9e529452
--- /dev/null
+++ b/tools/postinstall-services.mjs
@@ -0,0 +1,178 @@
+/**
+ * Prepares the external programs that installed Providers declare.
+ *
+ * A Provider that needs a separate program — WhisperX, OpenCV — states that in
+ * its own `service.ts`, and marks itself with `"service": true` under `svml` in
+ * its package.json. Nothing here names a Provider, so adding one is a change to
+ * that package alone.
+ *
+ * The prepare command is read from the declaration rather than restated here.
+ * Two statements of "how this environment is built" drift, and the copy that
+ * drifts is the one nobody runs during a Build.
+ *
+ * This runs from `pnpm install` and must never fail it. A missing `uv`, a
+ * Provider whose activation will not import, a prepare that exits non-zero —
+ * each is reported and stepped over. What it prepares is a convenience; what it
+ * must not do is stop someone from installing the repository.
+ *
+ * Reading a declaration means importing TypeScript, so that half runs in a
+ * child under tsx (`--declare`, printing JSON). A child that dies takes its
+ * failure with it.
+ */
+import { spawn } from "node:child_process";
+import { readdir, readFile } from "node:fs/promises";
+import { fileURLToPath } from "node:url";
+
+const packageRoot = new URL("../packages/", import.meta.url);
+const repositoryRoot = fileURLToPath(new URL("../", import.meta.url));
+
+/** Packages that declare an external program, as `{ name, activation }`. */
+async function declaringPackages() {
+ const found = [];
+ for (const entry of await readdir(packageRoot, { withFileTypes: true })) {
+ if (!entry.isDirectory()) continue;
+ let manifest;
+ try {
+ manifest = JSON.parse(await readFile(new URL(`./${entry.name}/package.json`, packageRoot), "utf8"));
+ } catch {
+ continue;
+ }
+ if (manifest.svml?.service !== true) continue;
+ const activation = manifest.svml?.activation;
+ if (typeof activation !== "string") {
+ console.warn(` ${manifest.name} marks itself as a service but declares no activation`);
+ continue;
+ }
+ found.push({
+ name: manifest.name,
+ activation: fileURLToPath(new URL(`./${entry.name}/${activation.replace(/^\.\//u, "")}`, packageRoot)),
+ });
+ }
+ return found.sort((a, b) => a.name.localeCompare(b.name));
+}
+
+/**
+ * Streams stay separate: the declaration child answers on stdout, and Node's
+ * own deprecation notices arrive on stderr. Merging them would leave the JSON
+ * unparseable for a reason that has nothing to do with what was declared.
+ */
+function run(command, args, options = {}) {
+ return new Promise((resolve) => {
+ const child = spawn(command, args, { cwd: repositoryRoot, shell: false, ...options });
+ let out = "";
+ let err = "";
+ if (options.stdio === undefined) {
+ child.stdout.on("data", (chunk) => { out += chunk.toString(); });
+ child.stderr.on("data", (chunk) => { err += chunk.toString(); });
+ }
+ child.on("error", (error) => resolve({ ok: false, out: "", err: error.message }));
+ child.on("close", (code) => resolve({ ok: code === 0, out: out.trim(), err: err.trim(), code }));
+ });
+}
+
+function lastLine(text) {
+ return text.split("\n").filter((line) => line.trim().length > 0).at(-1) ?? "unknown error";
+}
+
+/**
+ * Asks a child running under tsx what each package declares. The parent stays
+ * plain JavaScript so that a repository without tsx built yet still installs.
+ */
+async function declarations(packages) {
+ const result = await run(process.execPath, [
+ "--import", "tsx",
+ fileURLToPath(import.meta.url),
+ "--declare",
+ ...packages.map((item) => `${item.name}=${item.activation}`),
+ ]);
+ if (!result.ok) {
+ console.warn(` could not read service declarations: ${lastLine(result.err)}`);
+ return [];
+ }
+ try {
+ return JSON.parse(result.out);
+ } catch {
+ console.warn(" service declarations were not readable JSON");
+ return [];
+ }
+}
+
+/** Imports the activations named on argv and prints what they declare. */
+async function emitDeclarations(specifiers) {
+ const declared = [];
+ for (const specifier of specifiers) {
+ const separator = specifier.indexOf("=");
+ const name = specifier.slice(0, separator);
+ try {
+ const module = await import(specifier.slice(separator + 1));
+ const facets = (module.default ?? module.svmlPackage)?.hostFacets ?? [];
+ for (const facet of facets) {
+ // A Provider states its program for a Runtime Profile it has not seen
+ // yet. An empty config is what a fresh clone has, and yields the
+ // defaults this repository ships.
+ const service = facet.implementation?.service?.({
+ root: repositoryRoot,
+ instance: "",
+ config: {},
+ });
+ if (service?.prepare === undefined) continue;
+ declared.push({
+ package: name,
+ id: service.id,
+ prepare: { command: service.prepare.command, args: [...service.prepare.args] },
+ });
+ }
+ } catch (error) {
+ declared.push({ package: name, failure: error instanceof Error ? error.message : String(error) });
+ }
+ }
+ process.stdout.write(JSON.stringify(declared));
+}
+
+async function main() {
+ const packages = await declaringPackages();
+ if (packages.length === 0) return;
+
+ console.log(`Preparing external services for ${packages.length} Provider${packages.length === 1 ? "" : "s"}`);
+ const declared = await declarations(packages);
+
+ // One program may be declared by more than one Provider; prepare it once.
+ const unique = new Map();
+ for (const item of declared) {
+ if (item.failure !== undefined) {
+ console.warn(` ${item.package}: ${item.failure}`);
+ continue;
+ }
+ if (!unique.has(item.id)) unique.set(item.id, item);
+ }
+
+ const missing = new Set();
+ let prepared = 0;
+ for (const item of unique.values()) {
+ const { command, args } = item.prepare;
+ if (missing.has(command) || !(await run(command, ["--version"])).ok) {
+ missing.add(command);
+ console.warn(` ${item.id}: skipped, ${command} is not on PATH`);
+ continue;
+ }
+ console.log(` ${item.id}: ${command} ${args.join(" ")}`);
+ const result = await run(command, args, { stdio: "inherit" });
+ if (result.ok) prepared += 1;
+ else console.warn(` ${item.id}: prepare exited ${result.code ?? "abnormally"}; run it by hand to see why`);
+ }
+
+ const skipped = unique.size - prepared;
+ console.log(`Prepared ${prepared} service${prepared === 1 ? "" : "s"}${skipped > 0 ? `, skipped ${skipped}` : ""}`);
+ if (missing.size > 0) {
+ console.log(`Install ${[...missing].join(", ")} and run pnpm install again to finish setup`);
+ }
+}
+
+const declareAt = process.argv.indexOf("--declare");
+try {
+ if (declareAt === -1) await main();
+ else await emitDeclarations(process.argv.slice(declareAt + 1));
+} catch (error) {
+ // Nothing this script does is worth failing an install over.
+ console.warn(`Service preparation stopped: ${error instanceof Error ? error.message : String(error)}`);
+}