Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

feat: add provider-neutral MiMo TTS

kashorine8330033aa6f
diff --git a/README.md b/README.md
index a38a08c5..9caf10f6 100644
--- a/README.md
+++ b/README.md
@@ -70,6 +70,7 @@ The repository implements:
- SQLite Build/Operation stores and filesystem/S3 Artifact stores;
- exact Endpoint binding and scoped credentials;
- eleven exact image/video models declaring their own input ports, reached through the KIE Provider;
+- three exact MiMo TTS models, independently reached through Xiaomi's official Provider;
- local ffprobe/ffmpeg media processing;
- S3 multipart Artifact streaming and AWS Lambda media processing;
- local pinned WhisperX service;
diff --git a/docs/guide/packages.md b/docs/guide/packages.md
index 83cd3edf..521430b4 100644
--- a/docs/guide/packages.md
+++ b/docs/guide/packages.md
@@ -67,7 +67,7 @@ Depends on Layer 1 and 2 but not on any Provider.
@narratage/media media types
@narratage/narrative authored narrative products
@narratage/program-space exact frame/sample domain
-@narratage/generation image/video product contracts
+@narratage/generation image/video/audio generation contracts
@narratage/model-kit model family abstractions
@narratage/seedance Seedance model family + author Surface
@narratage/seedance-speaker Seedance Speaker binding
@@ -77,6 +77,7 @@ Depends on Layer 1 and 2 but not on any Provider.
@narratage/gpt-image GPT Image model family
@narratage/nano-banana Nano Banana model family
@narratage/seedream Seedream model family
+@narratage/mimo-tts three exact Xiaomi MiMo TTS models + author Surfaces
@narratage/estimate duration estimation
@narratage/speech shared speech products
@narratage/speech-basis generated speech A/V product
@@ -112,8 +113,8 @@ Depends on Layer 1 and 2 but not on any Provider.
### Layer 4: Provider (Endpoint) packages
-Privileged external capabilities. Depend on Runtime ports and the model families they serve, never
-on the CLI.
+Privileged external capabilities. Depend on Runtime ports and shared capability vocabularies,
+never on exact-model packages or the CLI.
```text
@narratage/provider-kie KIE generation plus background removal
@@ -124,6 +125,7 @@ on the CLI.
@narratage/provider-hyperframes-aws-lambda recoverable distributed rendering
@narratage/provider-image-opencv-local local OpenCV Raster execution
@narratage/provider-media-aws-lambda synchronous AWS media execution
+@narratage/provider-xiaomi-mimo official Xiaomi MiMo TTS API
```
### Layer 5: Application
diff --git a/docs/guide/providers.md b/docs/guide/providers.md
index 0a205700..51f15e97 100644
--- a/docs/guide/providers.md
+++ b/docs/guide/providers.md
@@ -19,7 +19,8 @@ mkdir -p packages/provider-my-service/src packages/provider-my-service/test
## 2. Write package.json
-Provider packages depend on Runtime ports and the model families they serve, never on the CLI:
+Provider packages depend on Runtime ports and shared capability vocabularies, never on exact-model
+packages or the CLI:
```json
{
@@ -37,7 +38,7 @@ Provider packages depend on Runtime ports and the model families they serve, nev
"@narratage/runtime": "workspace:*",
"@narratage/runtime-adapter": "workspace:*",
"@narratage/runtime-adapter-node": "workspace:*",
- "@narratage/seedance": "workspace:*"
+ "@narratage/generation": "workspace:*"
}
}
```
@@ -71,6 +72,7 @@ Look at existing Providers for reference:
- `packages/provider-hyperframes-local/` — local Chrome rendering
- `packages/provider-hyperframes-aws-lambda/` — recoverable Step Functions/Lambda rendering
- `packages/provider-media-aws-lambda/` — synchronous Lambda media execution over the shared ffmpeg body
+- `packages/provider-xiaomi-mimo/` — immediate official TTS API without importing the MiMo model package
## 4. Write the activation descriptor
@@ -176,3 +178,4 @@ pnpm narratage doctor svml.runtime.json
| `provider-hyperframes-aws-lambda` | Remote recoverable job: deterministic Step Functions submission, polling and S3 streaming |
| `provider-image-opencv-local` | Local Python: bounded OpenCV/NumPy with locked Python environment |
| `provider-media-aws-lambda` | Remote synchronous Lambda: the same five media operations as the local Provider |
+| `provider-xiaomi-mimo` | Remote immediate API: exact MiMo TTS requests to persisted audio Artifacts |
diff --git a/docs/zh/guide/packages.md b/docs/zh/guide/packages.md
index 92f562a1..b39da3a2 100644
--- a/docs/zh/guide/packages.md
+++ b/docs/zh/guide/packages.md
@@ -62,7 +62,7 @@ description: 五个分层、依赖边界、包的结构与 facet。
@narratage/media media types
@narratage/narrative authored narrative products
@narratage/program-space exact frame/sample domain
-@narratage/generation image/video product contracts
+@narratage/generation 图像/视频/音频生成合同
@narratage/model-kit model family abstractions
@narratage/seedance Seedance model family + author Surface
@narratage/seedance-speaker Seedance Speaker binding
@@ -72,6 +72,7 @@ description: 五个分层、依赖边界、包的结构与 facet。
@narratage/gpt-image GPT Image model family
@narratage/nano-banana Nano Banana model family
@narratage/seedream Seedream model family
+@narratage/mimo-tts 三个精确 Xiaomi MiMo TTS 模型及作者 Surface
@narratage/estimate duration estimation
@narratage/speech shared speech products
@narratage/speech-basis generated speech A/V product
@@ -107,7 +108,7 @@ description: 五个分层、依赖边界、包的结构与 facet。
### Layer 4:Provider(Endpoint)包
-具备特权的外部能力。依赖 Runtime 端口和它们所服务的模型族,绝不依赖 CLI。
+具备特权的外部能力。依赖 Runtime 端口与共享能力词汇,不依赖精确模型包,也绝不依赖 CLI。
```text
@narratage/provider-kie KIE 生成与去背景
@@ -118,6 +119,7 @@ description: 五个分层、依赖边界、包的结构与 facet。
@narratage/provider-hyperframes-aws-lambda recoverable distributed rendering
@narratage/provider-image-opencv-local 本地 OpenCV 光栅执行
@narratage/provider-media-aws-lambda synchronous AWS media execution
+@narratage/provider-xiaomi-mimo Xiaomi 官方 MiMo TTS API
```
### Layer 5:应用层
diff --git a/docs/zh/guide/providers.md b/docs/zh/guide/providers.md
index 4f387e23..9cba7666 100644
--- a/docs/zh/guide/providers.md
+++ b/docs/zh/guide/providers.md
@@ -17,7 +17,7 @@ mkdir -p packages/provider-my-service/src packages/provider-my-service/test
## 2. 编写 package.json
-Provider 包依赖 Runtime 端口以及它所服务的模型族,绝不依赖 CLI:
+Provider 包依赖 Runtime 端口与共享能力词汇,不依赖精确模型包,也绝不依赖 CLI:
```json
{
@@ -35,7 +35,7 @@ Provider 包依赖 Runtime 端口以及它所服务的模型族,绝不依赖 C
"@narratage/runtime": "workspace:*",
"@narratage/runtime-adapter": "workspace:*",
"@narratage/runtime-adapter-node": "workspace:*",
- "@narratage/seedance": "workspace:*"
+ "@narratage/generation": "workspace:*"
}
}
```
@@ -68,6 +68,7 @@ export function createMyServiceProvider(options: {
- `packages/provider-hyperframes-local/` — 本地 Chrome 渲染
- `packages/provider-hyperframes-aws-lambda/` — 可恢复的 Step Functions/Lambda 渲染
- `packages/provider-media-aws-lambda/` — 通过共享 ffmpeg 执行体完成同步 Lambda 媒体操作
+- `packages/provider-xiaomi-mimo/` — 不依赖 MiMo 模型包的官方即时 TTS API
## 4. 编写 activation 描述符
@@ -171,3 +172,4 @@ pnpm narratage doctor svml.runtime.json
| `provider-hyperframes-aws-lambda` | 远程可恢复任务:确定性 Step Functions 提交、轮询与 S3 流式落库 |
| `provider-image-opencv-local` | 本地 Python:有界的 OpenCV/NumPy,配合锁定的 Python 环境 |
| `provider-media-aws-lambda` | 远程同步 Lambda:与本地 Provider 相同的五项媒体操作 |
+| `provider-xiaomi-mimo` | 远程即时 API:把精确 MiMo TTS 请求落成持久化音频 Artifact |
diff --git a/examples/talking-film-graph-check/svml.packages.lock b/examples/talking-film-graph-check/svml.packages.lock
index d0fee0b3..742fee0b 100644
--- a/examples/talking-film-graph-check/svml.packages.lock
+++ b/examples/talking-film-graph-check/svml.packages.lock
@@ -604,7 +604,7 @@
{
"name": "@narratage/generation",
"version": "0.0.0-dev",
- "digest": "sha256:af3cee164fcae00d407a657cd8a2491c727b4f82f5465af71ecf7d3101aeee38"
+ "digest": "sha256:4ceaf545fa645de80873c5cc3b9df7655474a262057f805e85eca2679025536e"
},
{
"name": "@narratage/host",
@@ -634,7 +634,7 @@
{
"name": "@narratage/model-kit",
"version": "0.0.0-dev",
- "digest": "sha256:71b23848a90535e0cd74b25f4ef74a9827ce1aa0abdfe4eb34cc1dc3d907ce13"
+ "digest": "sha256:debadc91af0b1c706f6d44214be281fbf05a53b0d53d027066d06a1a8752e477"
},
{
"name": "@narratage/narrative",
@@ -831,8 +831,8 @@
"name": "@narratage/generation",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:e66e286a114071febbf5152de90fad064b7abae825b0f4e20becacb397049171",
- "closureDigest": "sha256:ce410e04dad47fd5ae4b2b1316a642c7b2f6b44ba5c1d1107b6d7666257ef097"
+ "facetsDigest": "sha256:d74a57a3210267e665366513324db5c313b3ca0f770da66e74cc38aee5048efb",
+ "closureDigest": "sha256:67fa8f4123972a33bd2d0e1cc6126a9ed8d3d09cc2745aa058888b735fc85570"
},
{
"specifier": "@narratage/hyperframes",
@@ -921,8 +921,8 @@
"name": "@narratage/seedance",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:fb36bc91792ad09598e09d77fe1a577c357c4e4beede079b7f6ccc0f3478534e",
- "closureDigest": "sha256:c90e652bcf2aa55b525ac778cd8eb9814c592b895a6d2ddb8c957626807a6609"
+ "facetsDigest": "sha256:3b9cebaa0503cb716c3d8314a6cf837b13dd5d88c2b09fd9604ab38e67d7af5e",
+ "closureDigest": "sha256:d8f3669cc7a75d22e30a4e3b304f6a87ff9d33920199e50c043d6ed0e6d4e2be"
},
{
"specifier": "@narratage/semantic-map",
@@ -1033,5 +1033,5 @@
"closureDigest": "sha256:1dd3e16f58411fd77f2daf4a945781a784618cffe7512d83d6181f701c39cadb"
}
],
- "digest": "sha256:50fa78cec0be7a72d5b4a515e90a5506e7031b01159fe2214624f486e6392cd2"
+ "digest": "sha256:53b9f0366ff63d096841760fe4c6b7c5e71a50005564f57eff44539130f45faa"
}
diff --git a/examples/talking-film-live/svml.packages.lock b/examples/talking-film-live/svml.packages.lock
index 736c2457..1f846e86 100644
--- a/examples/talking-film-live/svml.packages.lock
+++ b/examples/talking-film-live/svml.packages.lock
@@ -604,7 +604,7 @@
{
"name": "@narratage/generation",
"version": "0.0.0-dev",
- "digest": "sha256:af3cee164fcae00d407a657cd8a2491c727b4f82f5465af71ecf7d3101aeee38"
+ "digest": "sha256:4ceaf545fa645de80873c5cc3b9df7655474a262057f805e85eca2679025536e"
},
{
"name": "@narratage/host",
@@ -634,7 +634,7 @@
{
"name": "@narratage/model-kit",
"version": "0.0.0-dev",
- "digest": "sha256:71b23848a90535e0cd74b25f4ef74a9827ce1aa0abdfe4eb34cc1dc3d907ce13"
+ "digest": "sha256:debadc91af0b1c706f6d44214be281fbf05a53b0d53d027066d06a1a8752e477"
},
{
"name": "@narratage/narrative",
@@ -831,8 +831,8 @@
"name": "@narratage/generation",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:e66e286a114071febbf5152de90fad064b7abae825b0f4e20becacb397049171",
- "closureDigest": "sha256:ce410e04dad47fd5ae4b2b1316a642c7b2f6b44ba5c1d1107b6d7666257ef097"
+ "facetsDigest": "sha256:d74a57a3210267e665366513324db5c313b3ca0f770da66e74cc38aee5048efb",
+ "closureDigest": "sha256:67fa8f4123972a33bd2d0e1cc6126a9ed8d3d09cc2745aa058888b735fc85570"
},
{
"specifier": "@narratage/hyperframes",
@@ -912,8 +912,8 @@
"name": "@narratage/seedance",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:fb36bc91792ad09598e09d77fe1a577c357c4e4beede079b7f6ccc0f3478534e",
- "closureDigest": "sha256:c90e652bcf2aa55b525ac778cd8eb9814c592b895a6d2ddb8c957626807a6609"
+ "facetsDigest": "sha256:3b9cebaa0503cb716c3d8314a6cf837b13dd5d88c2b09fd9604ab38e67d7af5e",
+ "closureDigest": "sha256:d8f3669cc7a75d22e30a4e3b304f6a87ff9d33920199e50c043d6ed0e6d4e2be"
},
{
"specifier": "@narratage/semantic-map",
@@ -1015,5 +1015,5 @@
"closureDigest": "sha256:1dd3e16f58411fd77f2daf4a945781a784618cffe7512d83d6181f701c39cadb"
}
],
- "digest": "sha256:3e0242e779572cc081d0962465920145ebf93486530226b3a816ed005bbb637f"
+ "digest": "sha256:6c2f003b40c9824af27dfbbf08015ee9a10f1d79231167faeb07d0c7c4389c8f"
}
diff --git a/examples/talking-film-live/svml.runtime-packages.lock b/examples/talking-film-live/svml.runtime-packages.lock
index 248c4075..10f06b2a 100644
--- a/examples/talking-film-live/svml.runtime-packages.lock
+++ b/examples/talking-film-live/svml.runtime-packages.lock
@@ -74,7 +74,7 @@
{
"name": "@narratage/generation",
"version": "0.0.0-dev",
- "digest": "sha256:af3cee164fcae00d407a657cd8a2491c727b4f82f5465af71ecf7d3101aeee38"
+ "digest": "sha256:4ceaf545fa645de80873c5cc3b9df7655474a262057f805e85eca2679025536e"
},
{
"name": "@narratage/host",
@@ -129,7 +129,7 @@
{
"name": "@narratage/provider-kie",
"version": "0.0.0-dev",
- "digest": "sha256:eb6f6128e9df4d7ec1b70bdcbded81e627dd336958b9aabb7212559b93baa7ff"
+ "digest": "sha256:e9047810ec2972880c798ac717c720b33e84dcc8edb7b4421dc8c3dbf55d731e"
},
{
"name": "@narratage/provider-media-local",
@@ -878,7 +878,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:e61ebb911a33d4c5d0e4e4ee4fb69cdb790f4ced050ef35f7472bfbd72013d9b",
- "closureDigest": "sha256:0bd2705a051de96c59b3e9c26f632a6887cb161ffcb84b3786790e59f79cdd93"
+ "closureDigest": "sha256:35e95885907dbc295141ad63c0d2801be742295f4536e574c213a22a99e49530"
},
{
"specifier": "@narratage/provider-media-local",
@@ -899,5 +899,5 @@
"closureDigest": "sha256:90bd3d82d9006322e174a24e616981d6dd942cb9844f676878ab91de38c07e87"
}
],
- "digest": "sha256:cd4864f8925cad18619c3786c266bb70024bd75e6d0aa6cb39b807bced9ebc4c"
+ "digest": "sha256:76ed6d4b9b4ec2dcd8aec5b30a6a3043dd0c637fcf9314cf6abc234393673ec7"
}
diff --git a/examples/talking-head-aroll/svml.packages.lock b/examples/talking-head-aroll/svml.packages.lock
index 9f563b68..010674c4 100644
--- a/examples/talking-head-aroll/svml.packages.lock
+++ b/examples/talking-head-aroll/svml.packages.lock
@@ -604,7 +604,7 @@
{
"name": "@narratage/generation",
"version": "0.0.0-dev",
- "digest": "sha256:af3cee164fcae00d407a657cd8a2491c727b4f82f5465af71ecf7d3101aeee38"
+ "digest": "sha256:4ceaf545fa645de80873c5cc3b9df7655474a262057f805e85eca2679025536e"
},
{
"name": "@narratage/host",
@@ -634,7 +634,7 @@
{
"name": "@narratage/model-kit",
"version": "0.0.0-dev",
- "digest": "sha256:71b23848a90535e0cd74b25f4ef74a9827ce1aa0abdfe4eb34cc1dc3d907ce13"
+ "digest": "sha256:debadc91af0b1c706f6d44214be281fbf05a53b0d53d027066d06a1a8752e477"
},
{
"name": "@narratage/narrative",
@@ -836,8 +836,8 @@
"name": "@narratage/generation",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:e66e286a114071febbf5152de90fad064b7abae825b0f4e20becacb397049171",
- "closureDigest": "sha256:ce410e04dad47fd5ae4b2b1316a642c7b2f6b44ba5c1d1107b6d7666257ef097"
+ "facetsDigest": "sha256:d74a57a3210267e665366513324db5c313b3ca0f770da66e74cc38aee5048efb",
+ "closureDigest": "sha256:67fa8f4123972a33bd2d0e1cc6126a9ed8d3d09cc2745aa058888b735fc85570"
},
{
"specifier": "@narratage/hyperframes",
@@ -926,8 +926,8 @@
"name": "@narratage/seedance",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:fb36bc91792ad09598e09d77fe1a577c357c4e4beede079b7f6ccc0f3478534e",
- "closureDigest": "sha256:c90e652bcf2aa55b525ac778cd8eb9814c592b895a6d2ddb8c957626807a6609"
+ "facetsDigest": "sha256:3b9cebaa0503cb716c3d8314a6cf837b13dd5d88c2b09fd9604ab38e67d7af5e",
+ "closureDigest": "sha256:d8f3669cc7a75d22e30a4e3b304f6a87ff9d33920199e50c043d6ed0e6d4e2be"
},
{
"specifier": "@narratage/seedance-speaker",
@@ -935,8 +935,8 @@
"name": "@narratage/seedance-speaker",
"version": "0.0.0-dev"
},
- "facetsDigest": "sha256:f2f99e778220f5dc90d178a1ac2bc8843395bb14f3cdc63220a5ff87c6fa250b",
- "closureDigest": "sha256:c98428165704676ad121c7c57e00e7370687b9bb8839a5755b90825c2bdf648c"
+ "facetsDigest": "sha256:82648b54b0e363bae552ad852bc4fb261088e20b90e7cf2ab94471f0be68ba35",
+ "closureDigest": "sha256:f09074e9d2145c724f174958dc86e0398a600e2109750da1306339a048df6899"
},
{
"specifier": "@narratage/semantic-map",
@@ -1029,5 +1029,5 @@
"closureDigest": "sha256:1dd3e16f58411fd77f2daf4a945781a784618cffe7512d83d6181f701c39cadb"
}
],
- "digest": "sha256:101eebd953e8e78527a18bc36fa70c989be0ad1865e879c81befe1b41d906191"
+ "digest": "sha256:ec4435e504b092818c851290ff4e4c1528f54e98ed88e9315b180844f4a488dd"
}
diff --git a/examples/talking-head-aroll/svml.runtime-packages.lock b/examples/talking-head-aroll/svml.runtime-packages.lock
index 248c4075..10f06b2a 100644
--- a/examples/talking-head-aroll/svml.runtime-packages.lock
+++ b/examples/talking-head-aroll/svml.runtime-packages.lock
@@ -74,7 +74,7 @@
{
"name": "@narratage/generation",
"version": "0.0.0-dev",
- "digest": "sha256:af3cee164fcae00d407a657cd8a2491c727b4f82f5465af71ecf7d3101aeee38"
+ "digest": "sha256:4ceaf545fa645de80873c5cc3b9df7655474a262057f805e85eca2679025536e"
},
{
"name": "@narratage/host",
@@ -129,7 +129,7 @@
{
"name": "@narratage/provider-kie",
"version": "0.0.0-dev",
- "digest": "sha256:eb6f6128e9df4d7ec1b70bdcbded81e627dd336958b9aabb7212559b93baa7ff"
+ "digest": "sha256:e9047810ec2972880c798ac717c720b33e84dcc8edb7b4421dc8c3dbf55d731e"
},
{
"name": "@narratage/provider-media-local",
@@ -878,7 +878,7 @@
"version": "0.0.0-dev"
},
"facetsDigest": "sha256:e61ebb911a33d4c5d0e4e4ee4fb69cdb790f4ced050ef35f7472bfbd72013d9b",
- "closureDigest": "sha256:0bd2705a051de96c59b3e9c26f632a6887cb161ffcb84b3786790e59f79cdd93"
+ "closureDigest": "sha256:35e95885907dbc295141ad63c0d2801be742295f4536e574c213a22a99e49530"
},
{
"specifier": "@narratage/provider-media-local",
@@ -899,5 +899,5 @@
"closureDigest": "sha256:90bd3d82d9006322e174a24e616981d6dd942cb9844f676878ab91de38c07e87"
}
],
- "digest": "sha256:cd4864f8925cad18619c3786c266bb70024bd75e6d0aa6cb39b807bced9ebc4c"
+ "digest": "sha256:76ed6d4b9b4ec2dcd8aec5b30a6a3043dd0c637fcf9314cf6abc234393673ec7"
}
diff --git a/package.json b/package.json
index c189566d..d9227925 100644
--- a/package.json
+++ b/package.json
@@ -43,6 +43,7 @@
"@narratage/media": "workspace:*",
"@narratage/media-track": "workspace:*",
"@narratage/media-pipeline": "workspace:*",
+ "@narratage/mimo-tts": "workspace:*",
"@narratage/narrative": "workspace:*",
"@narratage/program-space": "workspace:*",
"@narratage/prompt-kit": "workspace:*",
@@ -52,6 +53,7 @@
"@narratage/provider-image-opencv-local": "workspace:*",
"@narratage/provider-kie": "workspace:*",
"@narratage/provider-media-local": "workspace:*",
+ "@narratage/provider-xiaomi-mimo": "workspace:*",
"@narratage/provider-whisperx-local": "workspace:*",
"@narratage/raster": "workspace:*",
"@narratage/render-hyperframes": "workspace:*",
diff --git a/packages/generation/README.md b/packages/generation/README.md
index a05fba10..3b5abc03 100644
--- a/packages/generation/README.md
+++ b/packages/generation/README.md
@@ -1,6 +1,6 @@
# `@narratage/generation`
-Provider-neutral generated-media contracts shared by exact image and video model packages.
+Provider-neutral generated-media contracts shared by exact image, video and audio model packages.
The package owns `GenerationRequest`, `GeneratedImageSet` and `GeneratedVideoSet` identities,
schemas, validators and graph facets. Generated sets are atomic Products: a Provider persists the
diff --git a/packages/generation/src/component.ts b/packages/generation/src/component.ts
index 881a4a80..12e96c7a 100644
--- a/packages/generation/src/component.ts
+++ b/packages/generation/src/component.ts
@@ -1,6 +1,7 @@
import type { ComponentPackage } from "@narratage/component-kit";
import {
+ verifyGeneratedAudioSet,
verifyGeneratedImageSet,
verifyGeneratedVideoSet,
} from "./identity.js";
@@ -18,13 +19,17 @@ function inline(value: { readonly kind: string; readonly value?: unknown }, subj
function primary(
value: { readonly kind: string; readonly value?: unknown },
- kind: "image" | "video",
+ kind: "audio" | "image" | "video",
): import("@narratage/protocol").BlobRef {
- const content = inline(value, `Generated${kind === "image" ? "Image" : "Video"}Set`) as {
+ const label = kind === "audio" ? "Audio" : kind === "image" ? "Image" : "Video";
+ const content = inline(value, `Generated${label}Set`) as {
readonly images?: readonly import("@narratage/protocol").BlobRef[];
readonly videos?: readonly import("@narratage/protocol").BlobRef[];
+ readonly audios?: readonly import("@narratage/protocol").BlobRef[];
};
- const selected = kind === "image" ? content.images?.[0] : content.videos?.[0];
+ const selected = kind === "audio"
+ ? content.audios?.[0]
+ : kind === "image" ? content.images?.[0] : content.videos?.[0];
if (selected === undefined) throw new Error(`Generated ${kind} set has no primary artifact`);
return selected;
}
@@ -33,6 +38,14 @@ function primary(
export const generationComponent = {
name: "@narratage/generation",
producers: [
+ {
+ producer: generationProducers.primaryAudio,
+ implementationDigest: generationProducerDigests.primaryAudio,
+ handler: ({ inputs }) => ({
+ outputs: { audio: primary(inputs.set!.value, "audio") },
+ needs: {},
+ }),
+ },
{
producer: generationProducers.primaryImage,
implementationDigest: generationProducerDigests.primaryImage,
@@ -51,6 +64,13 @@ export const generationComponent = {
},
],
validators: [
+ {
+ type: generationTypes.audioSet,
+ implementationDigest: generationValidatorDigests.audioSet,
+ handler: ({ value }) => {
+ verifyGeneratedAudioSet(inline(value, "GeneratedAudioSet"));
+ },
+ },
{
type: generationTypes.imageSet,
implementationDigest: generationValidatorDigests.imageSet,
diff --git a/packages/generation/src/identity.ts b/packages/generation/src/identity.ts
index 535de33d..5fbe61fe 100644
--- a/packages/generation/src/identity.ts
+++ b/packages/generation/src/identity.ts
@@ -5,6 +5,8 @@ import {
import type { BlobRef } from "@narratage/protocol";
import type {
+ GeneratedAudioSet,
+ GeneratedAudioSetContent,
GeneratedImageSet,
GeneratedImageSetContent,
GeneratedVideoSet,
@@ -68,3 +70,17 @@ export function verifyGeneratedVideoSet(value: unknown): asserts value is Genera
assert(Array.isArray(object.videos) && object.videos.length > 0, "Generated video set is empty");
object.videos.forEach((artifact) => assertGenerationBlobRef(artifact, "video/"));
}
+
+export function sealGeneratedAudioSet(content: GeneratedAudioSetContent): GeneratedAudioSet {
+ assert(content.contract === "svml.generated-audio-set@1", "Generated audio contract is invalid");
+ assert(content.audios.length > 0, "Generated audio set is empty");
+ content.audios.forEach((artifact) => assertGenerationBlobRef(artifact, "audio/"));
+ return canonicalize(content) as unknown as GeneratedAudioSet;
+}
+
+export function verifyGeneratedAudioSet(value: unknown): asserts value is GeneratedAudioSet {
+ const object = plainObject(value, "Generated audio set") as unknown as GeneratedAudioSet;
+ assert(object.contract === "svml.generated-audio-set@1", "Generated audio contract is invalid");
+ assert(Array.isArray(object.audios) && object.audios.length > 0, "Generated audio set is empty");
+ object.audios.forEach((artifact) => assertGenerationBlobRef(artifact, "audio/"));
+}
diff --git a/packages/generation/src/index.ts b/packages/generation/src/index.ts
index 2b286ec2..4c31d71a 100644
--- a/packages/generation/src/index.ts
+++ b/packages/generation/src/index.ts
@@ -1,10 +1,12 @@
export { generationComponent } from "./component.js";
export {
assertGenerationBlobRef,
+ sealGeneratedAudioSet,
sealGeneratedImageSet,
sealGeneratedVideoSet,
sealGenerationRequest,
verifyGeneratedImageSet,
+ verifyGeneratedAudioSet,
verifyGeneratedVideoSet,
} from "./identity.js";
export {
@@ -62,6 +64,7 @@ export {
export type { GenerationPortSubset, GenerationRequest } from "./request.js";
export {
generatedImageSetSchema,
+ generatedAudioSetSchema,
generatedVideoSetSchema,
generationBlobRefSchema,
generationDigestSchema,
diff --git a/packages/generation/src/manifest.ts b/packages/generation/src/manifest.ts
index aa081b8b..1301cac7 100644
--- a/packages/generation/src/manifest.ts
+++ b/packages/generation/src/manifest.ts
@@ -5,25 +5,29 @@ import {
import { digestOf } from "@narratage/protocol";
import type { ModuleManifest, ProducerRef, TypeRef } from "@narratage/protocol";
-import { generatedImageSetSchema, generatedVideoSetSchema } from "./schema.js";
+import { generatedAudioSetSchema, generatedImageSetSchema, generatedVideoSetSchema } from "./schema.js";
export const generationModuleRef = { name: "@narratage/generation", version: "1" } as const;
export const generationTypes = {
+ audioSet: { module: generationModuleRef, name: "GeneratedAudioSet" },
imageSet: { module: generationModuleRef, name: "GeneratedImageSet" },
videoSet: { module: generationModuleRef, name: "GeneratedVideoSet" },
} satisfies Record<string, TypeRef>;
export const generationValidatorDigests = {
+ audioSet: digestOf("@narratage/generation/validate-generated-audio-set@1"),
imageSet: digestOf("@narratage/generation/validate-generated-image-set@1"),
videoSet: digestOf("@narratage/generation/validate-generated-video-set@1"),
};
export const generationProducers = {
+ primaryAudio: { module: generationModuleRef, name: "select-primary-audio" },
primaryImage: { module: generationModuleRef, name: "select-primary-image" },
primaryVideo: { module: generationModuleRef, name: "select-primary-video" },
} satisfies Record<string, ProducerRef>;
export const generationProducerDigests = {
+ primaryAudio: digestOf("@narratage/generation/select-primary-audio@1"),
primaryImage: digestOf("@narratage/generation/select-primary-image@1"),
primaryVideo: digestOf("@narratage/generation/select-primary-video@1"),
} as const;
@@ -34,6 +38,18 @@ export const generationManifest: ModuleManifest = {
version: generationModuleRef.version,
dependencies: [artifactDependency],
types: [
+ {
+ name: generationTypes.audioSet.name,
+ schema: generatedAudioSetSchema,
+ validator: {
+ abi: "svml.type-validator@1",
+ implementation: {
+ kind: "registered",
+ locator: "@narratage/generation/validate-generated-audio-set",
+ digest: generationValidatorDigests.audioSet,
+ },
+ },
+ },
{
name: generationTypes.imageSet.name,
schema: generatedImageSetSchema,
@@ -62,6 +78,17 @@ export const generationManifest: ModuleManifest = {
capabilities: [],
surfaces: [],
producers: [
+ {
+ name: generationProducers.primaryAudio.name,
+ inputs: [{ name: "set", type: generationTypes.audioSet }],
+ outputs: [{ name: "audio", type: artifactTypes.blob }],
+ needs: [],
+ implementation: {
+ kind: "registered",
+ locator: "@narratage/generation/select-primary-audio",
+ digest: generationProducerDigests.primaryAudio,
+ },
+ },
{
name: generationProducers.primaryImage.name,
inputs: [{ name: "set", type: generationTypes.imageSet }],
diff --git a/packages/generation/src/mapping.ts b/packages/generation/src/mapping.ts
index 7f5160d5..aacf8f87 100644
--- a/packages/generation/src/mapping.ts
+++ b/packages/generation/src/mapping.ts
@@ -51,7 +51,7 @@ export type GenerationWireMapping = {
* cannot silently serve another.
*/
readonly capability: CapabilityRef;
- readonly result: "image" | "video";
+ readonly result: "audio" | "image" | "video";
readonly routes: readonly GenerationWireRoute[];
readonly fields: Readonly<Record<string, GenerationFieldMapping>>;
readonly constants?: Readonly<Record<string, CanonicalValue>>;
diff --git a/packages/generation/src/ports.ts b/packages/generation/src/ports.ts
index 3bf89a9a..54bac78f 100644
--- a/packages/generation/src/ports.ts
+++ b/packages/generation/src/ports.ts
@@ -17,7 +17,7 @@ export type GenerationMediaRole = "image" | "video" | "audio";
export const GENERATION_MEDIA_ROLES: readonly GenerationMediaRole[] = ["image", "video", "audio"];
export type GenerationPortScalarKind =
- | { readonly kind: "text"; readonly maxChars: number }
+ | { readonly kind: "text"; readonly maxChars?: number }
| { readonly kind: "token"; readonly minLength: number; readonly maxLength: number }
| { readonly kind: "enum"; readonly values: readonly (string | number)[] }
| {
@@ -81,7 +81,7 @@ export type GenerationPortTable = {
readonly contract: "svml.generation-ports@1";
/** Exact model identity; this is also the Capability name. */
readonly model: string;
- readonly result: "image" | "video";
+ readonly result: "audio" | "image" | "video";
readonly ports: readonly GenerationPort[];
readonly requires: readonly GenerationPortRequirement[];
};
@@ -106,7 +106,9 @@ export function isMediaPort(port: GenerationPort): port is GenerationMediaPort {
function assertScalarKind(value: GenerationPortScalarKind, subject: string): void {
if (value.kind === "text") {
- assert(Number.isSafeInteger(value.maxChars) && value.maxChars > 0, `${subject} maxChars is invalid`);
+ if (value.maxChars !== undefined) {
+ assert(Number.isSafeInteger(value.maxChars) && value.maxChars > 0, `${subject} maxChars is invalid`);
+ }
return;
}
if (value.kind === "token") {
@@ -137,7 +139,8 @@ function assertScalarKind(value: GenerationPortScalarKind, subject: string): voi
export function assertGenerationPortTable(value: GenerationPortTable): void {
assert(value.contract === GENERATION_PORTS_V1, "Generation port table contract is invalid");
assert(typeof value.model === "string" && value.model.trim().length > 0, "Generation port table model is empty");
- assert(value.result === "image" || value.result === "video", `${value.model} port table result is invalid`);
+ assert(value.result === "audio" || value.result === "image" || value.result === "video",
+ `${value.model} port table result is invalid`);
assert(value.ports.length > 0, `${value.model} declares no input port`);
const names = new Set<string>();
diff --git a/packages/generation/src/request.ts b/packages/generation/src/request.ts
index 4dac17ee..02c7a7db 100644
--- a/packages/generation/src/request.ts
+++ b/packages/generation/src/request.ts
@@ -41,7 +41,11 @@ function mediaPrefix(role: GenerationMediaRole): "image/" | "video/" | "audio/"
}
function scalarSchema(value: GenerationPortScalarKind): ValueSchema {
- if (value.kind === "text") return { kind: "string", minLength: 1, maxLength: value.maxChars };
+ if (value.kind === "text") return {
+ kind: "string",
+ minLength: 1,
+ ...(value.maxChars === undefined ? {} : { maxLength: value.maxChars }),
+ };
if (value.kind === "token") return { kind: "string", minLength: value.minLength, maxLength: value.maxLength };
if (value.kind === "boolean") return { kind: "boolean" };
if (value.kind === "number") {
@@ -118,8 +122,10 @@ export function requestSchemaFromPorts(table: GenerationPortTable): ValueSchema
function verifyScalar(value: unknown, kind: GenerationPortScalarKind, subject: string): void {
if (kind.kind === "text") {
- assert(typeof value === "string" && value.length > 0 && value.length <= kind.maxChars,
- `${subject} must be text of 1 to ${kind.maxChars} characters`);
+ assert(typeof value === "string" && value.length > 0, `${subject} must be non-empty text`);
+ if (kind.maxChars !== undefined) {
+ assert(value.length <= kind.maxChars, `${subject} must contain at most ${kind.maxChars} characters`);
+ }
return;
}
if (kind.kind === "token") {
diff --git a/packages/generation/src/schema.ts b/packages/generation/src/schema.ts
index db5e83d5..389a6ccb 100644
--- a/packages/generation/src/schema.ts
+++ b/packages/generation/src/schema.ts
@@ -51,3 +51,15 @@ export const generatedVideoSetSchema = generationObjectSchema({
},
},
});
+
+export const generatedAudioSetSchema = generationObjectSchema({
+ contract: { schema: { kind: "literal", value: "svml.generated-audio-set@1" } },
+ audios: {
+ schema: {
+ kind: "array",
+ minItems: 1,
+ maxItems: 16,
+ items: generationBlobRefSchema,
+ },
+ },
+});
diff --git a/packages/generation/src/types.ts b/packages/generation/src/types.ts
index 429ed2cc..0dacd37d 100644
--- a/packages/generation/src/types.ts
+++ b/packages/generation/src/types.ts
@@ -10,5 +10,11 @@ export type GeneratedVideoSet = {
readonly videos: readonly BlobRef[];
};
+export type GeneratedAudioSet = {
+ readonly contract: "svml.generated-audio-set@1";
+ readonly audios: readonly BlobRef[];
+};
+
export type GeneratedImageSetContent = GeneratedImageSet;
export type GeneratedVideoSetContent = GeneratedVideoSet;
+export type GeneratedAudioSetContent = GeneratedAudioSet;
diff --git a/packages/generation/test/identity.test.ts b/packages/generation/test/identity.test.ts
index 8c208994..4a5e3e56 100644
--- a/packages/generation/test/identity.test.ts
+++ b/packages/generation/test/identity.test.ts
@@ -7,9 +7,11 @@ import {
generationComponent,
generationProducers,
generationTypes,
+ sealGeneratedAudioSet,
sealGeneratedImageSet,
sealGeneratedVideoSet,
verifyGeneratedImageSet,
+ verifyGeneratedAudioSet,
} from "@narratage/generation";
import { TypeValidatorRegistry } from "@narratage/validation";
@@ -27,6 +29,22 @@ test("generated media validators bind artifacts and result contents", async () =
assert.ok(validators.resolve(generationTypes.imageSet));
});
+test("generated audio is an ordinary content-addressed media result", async () => {
+ const store = new MemoryArtifactStore();
+ const audio = await store.put(new Uint8Array([7, 8, 9]), "audio/wav");
+ const set = sealGeneratedAudioSet({ contract: "svml.generated-audio-set@1", audios: [audio] });
+ verifyGeneratedAudioSet(set);
+ const facet = generationComponent.producers.find((item) =>
+ item.producer.name === generationProducers.primaryAudio.name);
+ assert.ok(facet);
+ const result = await facet.handler({ inputs: { set: { value: { kind: "inline", value: set } } } } as never);
+ assert.ok("audio" in result.outputs);
+ assert.deepEqual(result.outputs.audio, audio);
+ const validators = new TypeValidatorRegistry();
+ registerTypeValidatorFacets(validators, generationComponent.validators);
+ assert.ok(validators.resolve(generationTypes.audioSet));
+});
+
test("the primary-video projection returns the ordered Product member as a Blob value", async () => {
const store = new MemoryArtifactStore();
const first = await store.put(new Uint8Array([1, 2, 3]), "video/mp4");
diff --git a/packages/mimo-tts/README.md b/packages/mimo-tts/README.md
new file mode 100644
index 00000000..19be45f0
--- /dev/null
+++ b/packages/mimo-tts/README.md
@@ -0,0 +1,44 @@
+# `@narratage/mimo-tts`
+
+Exact model contracts and author Surfaces for Xiaomi MiMo V2.5 speech synthesis.
+
+The package owns the three distinct model shapes:
+
+- `mimo-v2.5-tts`: explicit built-in voice;
+- `mimo-v2.5-tts-voicedesign`: text-described voice;
+- `mimo-v2.5-tts-voiceclone`: one explicit audio sample.
+
+It contains no API URL, credential, retry, queue or Xiaomi wire encoding. Those belong to a Runtime
+Endpoint such as `@narratage/provider-xiaomi-mimo`. Every model returns the shared
+`GeneratedAudioSet`; author Surfaces expose its primary member as an ordinary `BlobArtifact`.
+
+`optimize_text_preview` is intentionally absent: MiMo may not rewrite Script's exact speech truth.
+
+```xml
+<import as="mimo" from="@narratage/mimo-tts@1"/>
+
+<mimo:Preset id="narration" speech={story.segment.opening.speech} voice="Chloe">
+ Warm, direct and conversational.
+</mimo:Preset>
+
+<mimo:VoiceDesign id="designed" speech={story.segment.answer.speech}>
+ A clear young woman with a grounded, confident delivery.
+</mimo:VoiceDesign>
+
+<mimo:VoiceClone id="cloned" speech={story.segment.payoff.speech} sample={presenter-voice}>
+ Calm and restrained.
+</mimo:VoiceClone>
+```
+
+Official API reference: <https://mimo.mi.com/docs/zh-CN/quick-start/usage-guide/audio/speech-synthesis-v2.5>
+
+Add the author package to the project's explicit package lock before compiling:
+
+```sh
+pnpm narratage lock-packages svml.packages.lock \
+ --package @narratage/script \
+ --package @narratage/mimo-tts \
+ --root .
+```
+
+This selects author syntax and exact model semantics only. It does not select an API service.
diff --git a/packages/mimo-tts/package.json b/packages/mimo-tts/package.json
new file mode 100644
index 00000000..11396de8
--- /dev/null
+++ b/packages/mimo-tts/package.json
@@ -0,0 +1,21 @@
+{
+ "name": "@narratage/mimo-tts",
+ "version": "0.0.0-dev",
+ "private": true,
+ "type": "module",
+ "exports": {
+ ".": "./src/index.ts"
+ },
+ "svml": {
+ "activation": "./src/activation.ts"
+ },
+ "dependencies": {
+ "@narratage/artifact": "workspace:*",
+ "@narratage/elaborator": "workspace:*",
+ "@narratage/generation": "workspace:*",
+ "@narratage/model-kit": "workspace:*",
+ "@narratage/narrative": "workspace:*",
+ "@narratage/protocol": "workspace:*",
+ "@narratage/text": "workspace:*"
+ }
+}
diff --git a/packages/mimo-tts/src/activation.ts b/packages/mimo-tts/src/activation.ts
new file mode 100644
index 00000000..c1dafee2
--- /dev/null
+++ b/packages/mimo-tts/src/activation.ts
@@ -0,0 +1,34 @@
+import { createTextSurfaceHostFacet } from "@narratage/text";
+
+import {
+ decodeMimoPresetSurface,
+ decodeMimoVoiceCloneSurface,
+ decodeMimoVoiceDesignSurface,
+ mimoTtsComponent,
+ mimoTtsManifest,
+ mimoTtsModuleRef,
+ mimoTtsSurfaceDigests,
+} from "./index.js";
+
+export const svmlPackage = {
+ format: "svml.node-package@1" as const,
+ name: "@narratage/mimo-tts",
+ modules: [{ manifest: mimoTtsManifest, specifiers: ["@narratage/mimo-tts", "@narratage/mimo-tts@1"] }],
+ components: [mimoTtsComponent],
+ hostFacets: [
+ createTextSurfaceHostFacet({
+ module: mimoTtsModuleRef, surface: "preset", mode: "structured",
+ implementationDigest: mimoTtsSurfaceDigests.preset, handler: decodeMimoPresetSurface,
+ }),
+ createTextSurfaceHostFacet({
+ module: mimoTtsModuleRef, surface: "voiceDesign", mode: "structured",
+ implementationDigest: mimoTtsSurfaceDigests.voiceDesign, handler: decodeMimoVoiceDesignSurface,
+ }),
+ createTextSurfaceHostFacet({
+ module: mimoTtsModuleRef, surface: "voiceClone", mode: "structured",
+ implementationDigest: mimoTtsSurfaceDigests.voiceClone, handler: decodeMimoVoiceCloneSurface,
+ }),
+ ],
+};
+
+export default svmlPackage;
diff --git a/packages/mimo-tts/src/fragment.ts b/packages/mimo-tts/src/fragment.ts
new file mode 100644
index 00000000..bb2734dd
--- /dev/null
+++ b/packages/mimo-tts/src/fragment.ts
@@ -0,0 +1,36 @@
+import { artifactTypes } from "@narratage/artifact";
+import { sealGraphFragment } from "@narratage/elaborator";
+import { generationProducers } from "@narratage/generation";
+import type { ExactModelEndpoint } from "@narratage/model-kit";
+
+const input = (name: string) => ({ kind: "fragment-input" as const, name });
+const operation = (id: string) => ({ kind: "fragment-operation" as const, operation: id });
+
+/** The model result stays atomic until this deterministic projection exposes one ordinary audio Blob. */
+export function createMimoTtsAudioFragment(endpoint: ExactModelEndpoint) {
+ return sealGraphFragment({
+ name: `@narratage/mimo-tts/${endpoint.key}-primary-audio@1`,
+ inputs: [{ name: "request", type: endpoint.requestType }],
+ operations: [
+ {
+ id: "generate",
+ producer: endpoint.producer,
+ inputs: { request: input("request") },
+ result: { kind: "need", name: "generation", accepts: "exact" },
+ },
+ {
+ id: "select-primary-audio",
+ producer: generationProducers.primaryAudio,
+ inputs: { set: operation("generate") },
+ result: { kind: "output", name: "audio" },
+ },
+ ],
+ exports: [{
+ name: "audio",
+ type: artifactTypes.blob,
+ root: operation("select-primary-audio"),
+ semanticInputs: ["request"],
+ fidelity: "exact",
+ }],
+ });
+}
diff --git a/packages/mimo-tts/src/index.ts b/packages/mimo-tts/src/index.ts
new file mode 100644
index 00000000..f74cde4e
--- /dev/null
+++ b/packages/mimo-tts/src/index.ts
@@ -0,0 +1,136 @@
+import { artifactDependency } from "@narratage/artifact";
+import { sealGenerationPortRequest, sealGenerationPortTable } from "@narratage/generation";
+import type { GenerationPortTable, GenerationPortValue, GenerationRequest } from "@narratage/generation";
+import { defineExactModelModule } from "@narratage/model-kit";
+import { narrativeDependency } from "@narratage/narrative";
+import { digestOf } from "@narratage/protocol";
+
+export const mimoTtsModuleRef = { name: "@narratage/mimo-tts", version: "1" } as const;
+export const mimoTtsModels = [
+ "mimo-v2.5-tts",
+ "mimo-v2.5-tts-voicedesign",
+ "mimo-v2.5-tts-voiceclone",
+] as const;
+export type MimoTtsModel = typeof mimoTtsModels[number];
+
+export const mimoPresetVoices = ["冰糖", "茉莉", "苏打", "白桦", "Mia", "Chloe", "Milo", "Dean"] as const;
+export type MimoPresetVoice = typeof mimoPresetVoices[number];
+
+// Xiaomi documents no fixed character ceiling. Service-side limits stay service-side
+// instead of becoming an invented model constraint in Author Source.
+const spokenText = { kind: "text" } as const;
+const instruction = { kind: "text" } as const;
+
+function table(model: MimoTtsModel): GenerationPortTable {
+ const common = [
+ { name: "text", value: spokenText, minItems: 1, maxItems: 1 },
+ ] as const;
+ if (model === "mimo-v2.5-tts") {
+ return sealGenerationPortTable({
+ contract: "svml.generation-ports@1", model, result: "audio",
+ ports: [
+ ...common,
+ { name: "instruction", value: instruction, minItems: 0, maxItems: 1 },
+ { name: "voice", value: { kind: "enum", values: [...mimoPresetVoices] }, minItems: 1, maxItems: 1 },
+ ],
+ requires: [],
+ });
+ }
+ if (model === "mimo-v2.5-tts-voicedesign") {
+ return sealGenerationPortTable({
+ contract: "svml.generation-ports@1", model, result: "audio",
+ ports: [
+ ...common,
+ { name: "voiceDescription", value: instruction, minItems: 1, maxItems: 1 },
+ ],
+ requires: [],
+ });
+ }
+ return sealGenerationPortTable({
+ contract: "svml.generation-ports@1", model, result: "audio",
+ ports: [
+ ...common,
+ { name: "instruction", value: instruction, minItems: 0, maxItems: 1 },
+ { name: "sample", value: { kind: "media", accepts: ["audio"] }, minItems: 1, maxItems: 1 },
+ ],
+ requires: [],
+ });
+}
+
+export const mimoTtsPorts: Readonly<Record<MimoTtsModel, GenerationPortTable>> = {
+ "mimo-v2.5-tts": table("mimo-v2.5-tts"),
+ "mimo-v2.5-tts-voicedesign": table("mimo-v2.5-tts-voicedesign"),
+ "mimo-v2.5-tts-voiceclone": table("mimo-v2.5-tts-voiceclone"),
+};
+
+export function sealMimoTtsRequest(
+ model: MimoTtsModel,
+ ports: Readonly<Record<string, readonly GenerationPortValue[]>>,
+): GenerationRequest {
+ return sealGenerationPortRequest(mimoTtsPorts[model], ports);
+}
+
+const base = defineExactModelModule({
+ module: mimoTtsModuleRef,
+ endpoints: ([
+ ["preset", "mimo-v2.5-tts", "MimoPresetTtsRequest"],
+ ["voiceDesign", "mimo-v2.5-tts-voicedesign", "MimoVoiceDesignTtsRequest"],
+ ["voiceClone", "mimo-v2.5-tts-voiceclone", "MimoVoiceCloneTtsRequest"],
+ ] as const).map(([key, model, requestTypeName]) => ({
+ key,
+ requestTypeName,
+ producerName: `request-${model}`,
+ ports: mimoTtsPorts[model],
+ })),
+});
+
+export const mimoTtsEndpoints = base.endpoints;
+export const mimoTtsSurfaceDigests = {
+ preset: digestOf("@narratage/mimo-tts/preset-surface@1"),
+ voiceDesign: digestOf("@narratage/mimo-tts/voice-design-surface@1"),
+ voiceClone: digestOf("@narratage/mimo-tts/voice-clone-surface@1"),
+} as const;
+
+export const mimoTtsManifest = {
+ ...base.manifest,
+ dependencies: [...base.manifest.dependencies, artifactDependency, narrativeDependency],
+ surfaces: [
+ {
+ name: "preset", tag: "Preset", mode: "structured",
+ outputs: [mimoTtsEndpoints.preset.requestType],
+ implementation: {
+ kind: "trusted-frontend-surface" as const,
+ locator: "@narratage/mimo-tts/preset-surface",
+ digest: mimoTtsSurfaceDigests.preset,
+ },
+ },
+ {
+ name: "voiceDesign", tag: "VoiceDesign", mode: "structured",
+ outputs: [mimoTtsEndpoints.voiceDesign.requestType],
+ implementation: {
+ kind: "trusted-frontend-surface" as const,
+ locator: "@narratage/mimo-tts/voice-design-surface",
+ digest: mimoTtsSurfaceDigests.voiceDesign,
+ },
+ },
+ {
+ name: "voiceClone", tag: "VoiceClone", mode: "structured",
+ outputs: [mimoTtsEndpoints.voiceClone.requestType],
+ implementation: {
+ kind: "trusted-frontend-surface" as const,
+ locator: "@narratage/mimo-tts/voice-clone-surface",
+ digest: mimoTtsSurfaceDigests.voiceClone,
+ },
+ },
+ ],
+} as const;
+
+export const mimoTtsManifestDigest = digestOf(mimoTtsManifest);
+export const mimoTtsComponent = base.component;
+
+export { createMimoTtsAudioFragment } from "./fragment.js";
+export {
+ decodeMimoPresetSurface,
+ decodeMimoVoiceCloneSurface,
+ decodeMimoVoiceDesignSurface,
+} from "./surface.js";
diff --git a/packages/mimo-tts/src/surface.ts b/packages/mimo-tts/src/surface.ts
new file mode 100644
index 00000000..55ae5a97
--- /dev/null
+++ b/packages/mimo-tts/src/surface.ts
@@ -0,0 +1,163 @@
+import { artifactTypes } from "@narratage/artifact";
+import type { GenerationPortValue } from "@narratage/generation";
+import { narrativeTypes } from "@narratage/narrative";
+import type { BlobRef, CanonicalValue } from "@narratage/protocol";
+import type {
+ StructuredElement,
+ StructuredSurfaceHandler,
+ SurfaceResolvedReference,
+ TextAttributeValue,
+} from "@narratage/text";
+
+import { createMimoTtsAudioFragment } from "./fragment.js";
+import {
+ mimoPresetVoices,
+ mimoTtsEndpoints,
+ sealMimoTtsRequest,
+} from "./index.js";
+import type { MimoPresetVoice, MimoTtsModel } from "./index.js";
+
+function sameType(left: SurfaceResolvedReference["type"], right: SurfaceResolvedReference["type"]): boolean {
+ return left.module.name === right.module.name && left.module.version === right.module.version && left.name === right.name;
+}
+
+function attributes(element: StructuredElement, required: readonly string[], optional: readonly string[] = []): void {
+ const allowed = new Set([...required, ...optional]);
+ const unknown = Object.keys(element.attributes).filter((name) => !allowed.has(name));
+ if (unknown.length > 0 || required.some((name) => element.attributes[name] === undefined)) {
+ throw new Error(`${element.name} requires ${required.join(", ")}`
+ + (optional.length === 0 ? "" : `; optional: ${optional.join(", ")}`));
+ }
+}
+
+function stringAttribute(element: StructuredElement, name: string): string {
+ const value = element.attributes[name];
+ if (typeof value !== "string" || value.trim().length === 0) {
+ throw new Error(`${element.name}.${name} must be a non-empty string`);
+ }
+ return value.trim();
+}
+
+function referencePath(element: StructuredElement, name: string): string {
+ const value: TextAttributeValue | undefined = element.attributes[name];
+ if (typeof value !== "object" || value.kind !== "reference" || value.path.length === 0) {
+ throw new Error(`${element.name}.${name} must be a whole-value reference`);
+ }
+ return value.path;
+}
+
+function resolved(
+ element: StructuredElement,
+ name: string,
+ resolveReference: (path: string) => SurfaceResolvedReference | undefined,
+): SurfaceResolvedReference {
+ const path = referencePath(element, name);
+ const value = resolveReference(path);
+ if (value === undefined) throw new Error(`${element.name}.${name} cannot resolve ${path}`);
+ return value;
+}
+
+function inline(reference: SurfaceResolvedReference, subject: string): CanonicalValue {
+ if (reference.record?.value.kind !== "inline") throw new Error(`${subject} must reference an authored inline value`);
+ return reference.record.value.value;
+}
+
+function speechText(reference: SurfaceResolvedReference, subject: string): string {
+ if (!sameType(reference.type, narrativeTypes.speechExcerpt)) {
+ throw new Error(`${subject} must reference a NarrativeSpeechExcerpt`);
+ }
+ const value = inline(reference, subject) as unknown as Record<string, unknown>;
+ if (value.contract !== "svml.narrative-speech-excerpt@1" || typeof value.speech !== "string"
+ || value.speech.trim().length === 0) {
+ throw new Error(`${subject} NarrativeSpeechExcerpt is invalid`);
+ }
+ return value.speech;
+}
+
+function sample(reference: SurfaceResolvedReference, subject: string): BlobRef {
+ if (!sameType(reference.type, artifactTypes.blob) || reference.record?.value.kind !== "blob") {
+ throw new Error(`${subject} must reference an authored audio BlobArtifact`);
+ }
+ if (!reference.record.value.mediaType.startsWith("audio/")) throw new Error(`${subject} must be audio media`);
+ return reference.record.value;
+}
+
+function body(element: StructuredElement, required: boolean): string | undefined {
+ if (element.children.some((child) => child.kind === "element")) {
+ throw new Error(`${element.name} accepts instruction text only`);
+ }
+ const lines = element.children.map((child) => child.kind === "text" ? child.value : "").join("")
+ .replaceAll("\r\n", "\n").split("\n");
+ while (lines[0]?.trim() === "") lines.shift();
+ while (lines.at(-1)?.trim() === "") lines.pop();
+ const indents = lines.filter((line) => line.trim()).map((line) => /^\s*/u.exec(line)?.[0].length ?? 0);
+ const indent = indents.length === 0 ? 0 : Math.min(...indents);
+ const value = lines.map((line) => line.slice(indent).trimEnd()).join("\n").trim();
+ if (value.length === 0) {
+ if (required) throw new Error(`${element.name} requires a voice description in its body`);
+ return undefined;
+ }
+ return value;
+}
+
+function output(
+ element: StructuredElement,
+ endpoint: (typeof mimoTtsEndpoints)[keyof typeof mimoTtsEndpoints],
+ request: ReturnType<typeof sealMimoTtsRequest>,
+) {
+ const id = stringAttribute(element, "id");
+ const requestId = `${id}.request`;
+ const fragment = createMimoTtsAudioFragment(endpoint);
+ return {
+ records: [{
+ id: requestId,
+ type: endpoint.requestType,
+ value: { kind: "inline" as const, value: request as unknown as CanonicalValue },
+ range: element.range,
+ }],
+ components: [{
+ id,
+ fragment: fragment.id,
+ inputs: { request: { kind: "record" as const, id: requestId } },
+ outputs: { audio: `${id}.audio` },
+ range: element.range,
+ }],
+ fragments: [fragment],
+ };
+}
+
+function common(
+ element: StructuredElement,
+ resolveReference: (path: string) => SurfaceResolvedReference | undefined,
+): readonly GenerationPortValue[] {
+ return [speechText(resolved(element, "speech", resolveReference), `${element.name}.speech`)];
+}
+
+export const decodeMimoPresetSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
+ attributes(element, ["id", "speech", "voice"]);
+ const voice = stringAttribute(element, "voice") as MimoPresetVoice;
+ if (!mimoPresetVoices.includes(voice)) throw new Error(`${element.name}.voice is not a MiMo preset voice`);
+ const instruction = body(element, false);
+ const ports: Record<string, readonly GenerationPortValue[]> = { text: common(element, resolveReference), voice: [voice] };
+ if (instruction !== undefined) ports.instruction = [instruction];
+ return output(element, mimoTtsEndpoints.preset, sealMimoTtsRequest("mimo-v2.5-tts", ports));
+};
+
+export const decodeMimoVoiceDesignSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
+ attributes(element, ["id", "speech"]);
+ return output(element, mimoTtsEndpoints.voiceDesign, sealMimoTtsRequest("mimo-v2.5-tts-voicedesign", {
+ text: common(element, resolveReference),
+ voiceDescription: [body(element, true)!],
+ }));
+};
+
+export const decodeMimoVoiceCloneSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
+ attributes(element, ["id", "speech", "sample"]);
+ const instruction = body(element, false);
+ const ports: Record<string, readonly GenerationPortValue[]> = {
+ text: common(element, resolveReference),
+ sample: [{ role: "audio", artifact: sample(resolved(element, "sample", resolveReference), `${element.name}.sample`) }],
+ };
+ if (instruction !== undefined) ports.instruction = [instruction];
+ return output(element, mimoTtsEndpoints.voiceClone, sealMimoTtsRequest("mimo-v2.5-tts-voiceclone", ports));
+};
diff --git a/packages/mimo-tts/test/mimo-tts.test.ts b/packages/mimo-tts/test/mimo-tts.test.ts
new file mode 100644
index 00000000..bdf49947
--- /dev/null
+++ b/packages/mimo-tts/test/mimo-tts.test.ts
@@ -0,0 +1,121 @@
+import assert from "node:assert/strict";
+import test from "node:test";
+
+import { artifactTypes } from "@narratage/artifact";
+import { generationProducers, generationTypes } from "@narratage/generation";
+import { narrativeTypes } from "@narratage/narrative";
+import { digestOf } from "@narratage/protocol";
+import { parseStructuredElement } from "@narratage/text";
+import type { SurfaceResolvedReference } from "@narratage/text";
+
+import mimoNodePackage from "../src/activation.js";
+import {
+ decodeMimoPresetSurface,
+ decodeMimoVoiceCloneSurface,
+ decodeMimoVoiceDesignSurface,
+ mimoTtsEndpoints,
+ mimoTtsPorts,
+ sealMimoTtsRequest,
+} from "../src/index.js";
+
+function parsed(source: string) {
+ return parseStructuredElement({ name: "mimo.svml", text: source }, 0).element;
+}
+
+function authored(path: string, type: SurfaceResolvedReference["type"], value: unknown): SurfaceResolvedReference {
+ return {
+ path,
+ ref: { kind: "record", id: path },
+ type,
+ record: {
+ id: path,
+ type,
+ value: type.name === artifactTypes.blob.name
+ ? value as never
+ : { kind: "inline", value: value as never },
+ digest: digestOf(value),
+ conformance: "exact",
+ origin: { kind: "authored", sourceDigest: digestOf("mimo-source"), frontendClosureDigest: digestOf("mimo-frontend") },
+ },
+ };
+}
+
+const speech = {
+ contract: "svml.narrative-speech-excerpt@1",
+ kind: "segment",
+ id: "opening",
+ tokenStart: 0,
+ tokenEndExclusive: 4,
+ speech: "Exact authored words stay exact.",
+};
+const sample = { kind: "blob" as const, digest: digestOf("voice-sample"), size: 4, mediaType: "audio/wav" };
+const refs = new Map<string, SurfaceResolvedReference>([
+ ["story.segment.opening.speech", authored("story.segment.opening.speech", narrativeTypes.speechExcerpt, speech)],
+ ["voice", authored("voice", artifactTypes.blob, sample)],
+]);
+const context = (source: string) => ({
+ sourceName: "mimo.svml",
+ element: parsed(source),
+ resolveReference: (path: string) => refs.get(path),
+ resolveAsset: () => { throw new Error("no asset"); },
+});
+
+test("MiMo declares three audio models without Provider facts", () => {
+ assert.deepEqual(Object.values(mimoTtsPorts).map((ports) => ports.result), ["audio", "audio", "audio"]);
+ assert.deepEqual(Object.values(mimoTtsEndpoints).map((endpoint) => endpoint.returns), [
+ generationTypes.audioSet,
+ generationTypes.audioSet,
+ generationTypes.audioSet,
+ ]);
+ assert.equal(JSON.stringify(mimoTtsPorts).includes("xiaomimimo.com"), false);
+ assert.equal(mimoTtsPorts["mimo-v2.5-tts"].ports.find((port) => port.name === "text")?.value.kind, "text");
+ assert.equal("maxChars" in mimoTtsPorts["mimo-v2.5-tts"].ports[0]!.value, false);
+ assert.throws(() => sealMimoTtsRequest("mimo-v2.5-tts-voicedesign", {
+ text: ["Do not rewrite me."],
+ }), /voiceDescription is required/u);
+});
+
+test("one installed author package contributes all three Surfaces without a Runtime Provider", () => {
+ assert.equal(mimoNodePackage.name, "@narratage/mimo-tts");
+ assert.equal(mimoNodePackage.modules[0]?.manifest.version, "1");
+ assert.deepEqual(mimoNodePackage.modules[0]?.manifest.surfaces.map((surface) => surface.name),
+ ["preset", "voiceDesign", "voiceClone"]);
+ assert.equal(JSON.stringify(mimoNodePackage).includes("MIMO_API_KEY"), false);
+});
+
+test("the three author Surfaces are separate components with one ordinary audio output", async () => {
+ const results = await Promise.all([
+ decodeMimoPresetSurface(context(`<mimo:Preset id="preset" speech={story.segment.opening.speech} voice="Chloe">
+ Warm, restrained delivery.
+ </mimo:Preset>`)),
+ decodeMimoVoiceDesignSurface(context(`<mimo:VoiceDesign id="designed" speech={story.segment.opening.speech}>
+ A clear, confident young woman with a grounded conversational tone.
+ </mimo:VoiceDesign>`)),
+ decodeMimoVoiceCloneSurface(context(`<mimo:VoiceClone id="cloned" speech={story.segment.opening.speech} sample={voice}>
+ Calm and direct.
+ </mimo:VoiceClone>`)),
+ ]);
+ for (const result of results) {
+ assert.equal(result.components.length, 1);
+ assert.deepEqual(Object.keys(result.components[0]!.outputs), ["audio"]);
+ assert.deepEqual(result.fragments[0]!.operations.map((operation) => operation.producer.name), [
+ result.fragments[0]!.operations[0]!.producer.name,
+ generationProducers.primaryAudio.name,
+ ]);
+ assert.equal(result.records[0]!.value.kind, "inline");
+ const request = result.records[0]!.value.kind === "inline"
+ ? result.records[0]!.value.value as Record<string, unknown> : {};
+ assert.equal((request.ports as Record<string, unknown[]>).text?.[0], speech.speech);
+ assert.equal(JSON.stringify(request).includes("optimize_text_preview"), false);
+ }
+});
+
+test("voice clone refuses a non-audio sample before any Need exists", async () => {
+ const wrong = { ...sample, mediaType: "video/mp4" };
+ const invalid = new Map(refs);
+ invalid.set("voice", authored("voice", artifactTypes.blob, wrong));
+ assert.throws(() => decodeMimoVoiceCloneSurface({
+ ...context('<mimo:VoiceClone id="bad" speech={story.segment.opening.speech} sample={voice}/>'),
+ resolveReference: (path) => invalid.get(path),
+ }), /must be audio/u);
+});
diff --git a/packages/model-kit/src/index.ts b/packages/model-kit/src/index.ts
index 1d9bee2e..0157ea87 100644
--- a/packages/model-kit/src/index.ts
+++ b/packages/model-kit/src/index.ts
@@ -76,7 +76,9 @@ function endpointRef(module: ModuleRef, spec: ExactModelEndpointSpec) {
const requestType = { module, name: spec.requestTypeName };
const capability = { module, name: spec.ports.model };
const producer = { module, name: spec.producerName };
- const returns = spec.ports.result === "image" ? generationTypes.imageSet : generationTypes.videoSet;
+ const returns = spec.ports.result === "audio"
+ ? generationTypes.audioSet
+ : spec.ports.result === "image" ? generationTypes.imageSet : generationTypes.videoSet;
const implementationDigest = digestOf(`${module.name}/${spec.producerName}@1`);
const validatorDigest = digestOf(`${module.name}/validate-${spec.requestTypeName}@1`);
return { requestType, capability, producer, returns, implementationDigest, validatorDigest };
diff --git a/packages/provider-kie/src/routes.ts b/packages/provider-kie/src/routes.ts
index d4e1cb49..88562e9f 100644
--- a/packages/provider-kie/src/routes.ts
+++ b/packages/provider-kie/src/routes.ts
@@ -11,7 +11,7 @@ import {
sealGeneratedImageSet,
sealGeneratedVideoSet,
} from "@narratage/generation";
-import type { GenerationArtifactUrlResolver, GenerationRequest } from "@narratage/generation";
+import type { GenerationArtifactUrlResolver, GenerationRequest, GenerationWireMapping } from "@narratage/generation";
import { canonicalize } from "@narratage/protocol";
import type { BlobRef, CanonicalValue, CapabilityRef, StoredValue, TypeRef } from "@narratage/protocol";
@@ -40,7 +40,14 @@ function capabilityKey(ref: CapabilityRef): string {
return `${ref.module.name}@${ref.module.version}#${ref.name}`;
}
-const generationRoutes: readonly KieRoute[] = kieModelCatalog.map((mapping) => ({
+const kieGenerationMappings = kieModelCatalog.map((mapping) => {
+ if (mapping.result === "audio") {
+ throw new Error(`KIE route ${mapping.capability.name} declares unsupported audio output`);
+ }
+ return mapping as GenerationWireMapping & { readonly result: "image" | "video" };
+});
+
+const generationRoutes: readonly KieRoute[] = kieGenerationMappings.map((mapping) => ({
key: capabilityKey(mapping.capability),
capability: mapping.capability,
returns: mapping.result === "image" ? generationTypes.imageSet : generationTypes.videoSet,
diff --git a/packages/provider-xiaomi-mimo/README.md b/packages/provider-xiaomi-mimo/README.md
new file mode 100644
index 00000000..31551cb1
--- /dev/null
+++ b/packages/provider-xiaomi-mimo/README.md
@@ -0,0 +1,36 @@
+# `@narratage/provider-xiaomi-mimo`
+
+Immediate Runtime Endpoint for Xiaomi's official MiMo V2.5 TTS API.
+
+This package owns the `chat/completions` wire shape, `api-key` credential, timeout, response bounds,
+voice-sample data URI conversion and ArtifactStore ingestion. It does not import
+`@narratage/mimo-tts`: capabilities and model port names are bound as versioned data, keeping the
+three exact model contracts independent from this particular service.
+
+Runtime Profile example:
+
+```json
+{
+ "use": "@narratage/provider-xiaomi-mimo",
+ "instance": "mimo.official",
+ "lane": "xiaomi-mimo",
+ "config": {
+ "apiKeyEnv": "MIMO_API_KEY",
+ "defaultConcurrency": 2
+ }
+}
+```
+
+The Provider always requests final WAV bytes. Streaming is a service transport optimization and is
+not part of the author model.
+
+Add the Provider independently to the Runtime package lock:
+
+```sh
+pnpm narratage lock-packages svml.runtime-packages.lock \
+ --package @narratage/provider-xiaomi-mimo \
+ --root .
+```
+
+The Provider declares a default concurrency limit on its selected Runtime lane. The shared
+Scheduler owns queuing across Builds; this package does not create a private queue.
diff --git a/packages/provider-xiaomi-mimo/package.json b/packages/provider-xiaomi-mimo/package.json
new file mode 100644
index 00000000..1cdf3ef3
--- /dev/null
+++ b/packages/provider-xiaomi-mimo/package.json
@@ -0,0 +1,20 @@
+{
+ "name": "@narratage/provider-xiaomi-mimo",
+ "version": "0.0.0-dev",
+ "private": true,
+ "type": "module",
+ "exports": {
+ ".": "./src/index.ts"
+ },
+ "svml": {
+ "activation": "./src/activation.ts"
+ },
+ "dependencies": {
+ "@narratage/endpoint-kit": "workspace:*",
+ "@narratage/generation": "workspace:*",
+ "@narratage/protocol": "workspace:*",
+ "@narratage/runtime": "workspace:*",
+ "@narratage/runtime-adapter": "workspace:*",
+ "@narratage/runtime-adapter-node": "workspace:*"
+ }
+}
diff --git a/packages/provider-xiaomi-mimo/src/activation.ts b/packages/provider-xiaomi-mimo/src/activation.ts
new file mode 100644
index 00000000..e351fecc
--- /dev/null
+++ b/packages/provider-xiaomi-mimo/src/activation.ts
@@ -0,0 +1,68 @@
+import { credentialRef } from "@narratage/runtime";
+import {
+ createRuntimeEndpointAdapterFacet,
+ runtimeConfigExact,
+ runtimeConfigObject,
+ runtimeConfigPositiveInteger,
+ runtimeConfigString,
+} from "@narratage/runtime-adapter";
+import { diagnoseRuntimeEnvironmentCredential } from "@narratage/runtime-adapter-node";
+
+import { createXiaomiMimoProvider } from "./provider.js";
+
+const adapter = createRuntimeEndpointAdapterFacet({
+ use: "@narratage/provider-xiaomi-mimo",
+ validate(context) {
+ const config = runtimeConfigObject(context.config, "Xiaomi MiMo");
+ runtimeConfigExact(config, [
+ "apiBaseUrl", "apiKeyEnv", "defaultConcurrency", "requestTimeoutMs",
+ "maxResponseBytes", "maxVoiceSampleBase64Bytes",
+ ], "Xiaomi MiMo");
+ const base = runtimeConfigString(config.apiBaseUrl, "Xiaomi MiMo apiBaseUrl");
+ if (base !== undefined) {
+ const url = new URL(base);
+ if (url.protocol !== "https:" && url.hostname !== "localhost" && url.hostname !== "127.0.0.1") {
+ throw new Error("Xiaomi MiMo apiBaseUrl must use HTTPS or loopback");
+ }
+ }
+ runtimeConfigString(config.apiKeyEnv, "Xiaomi MiMo apiKeyEnv");
+ runtimeConfigPositiveInteger(config.defaultConcurrency, "Xiaomi MiMo defaultConcurrency");
+ runtimeConfigPositiveInteger(config.requestTimeoutMs, "Xiaomi MiMo requestTimeoutMs");
+ runtimeConfigPositiveInteger(config.maxResponseBytes, "Xiaomi MiMo maxResponseBytes");
+ runtimeConfigPositiveInteger(config.maxVoiceSampleBase64Bytes, "Xiaomi MiMo maxVoiceSampleBase64Bytes");
+ },
+ create(context) {
+ const config = runtimeConfigObject(context.config, "Xiaomi MiMo");
+ const apiKeyEnv = runtimeConfigString(config.apiKeyEnv, "Xiaomi MiMo apiKeyEnv");
+ return createXiaomiMimoProvider({
+ instance: context.instance,
+ ...(context.lane === undefined ? {} : { lane: context.lane }),
+ ...(runtimeConfigString(config.apiBaseUrl, "Xiaomi MiMo apiBaseUrl") === undefined
+ ? {} : { apiBaseUrl: config.apiBaseUrl as string }),
+ ...(apiKeyEnv === undefined ? {} : { apiKey: credentialRef("env", apiKeyEnv) }),
+ ...(runtimeConfigPositiveInteger(config.defaultConcurrency, "Xiaomi MiMo defaultConcurrency") === undefined
+ ? {} : { defaultConcurrency: config.defaultConcurrency as number }),
+ ...(runtimeConfigPositiveInteger(config.requestTimeoutMs, "Xiaomi MiMo requestTimeoutMs") === undefined
+ ? {} : { requestTimeoutMs: config.requestTimeoutMs as number }),
+ ...(runtimeConfigPositiveInteger(config.maxResponseBytes, "Xiaomi MiMo maxResponseBytes") === undefined
+ ? {} : { maxResponseBytes: config.maxResponseBytes as number }),
+ ...(runtimeConfigPositiveInteger(config.maxVoiceSampleBase64Bytes, "Xiaomi MiMo maxVoiceSampleBase64Bytes") === undefined
+ ? {} : { maxVoiceSampleBase64Bytes: config.maxVoiceSampleBase64Bytes as number }),
+ });
+ },
+ doctor(context) {
+ const config = runtimeConfigObject(context.config, "Xiaomi MiMo");
+ return diagnoseRuntimeEnvironmentCredential(
+ runtimeConfigString(config.apiKeyEnv, "Xiaomi MiMo apiKeyEnv") ?? "MIMO_API_KEY",
+ "Xiaomi MiMo",
+ );
+ },
+});
+
+export const svmlPackage = {
+ format: "svml.node-package@1" as const,
+ name: "@narratage/provider-xiaomi-mimo",
+ hostFacets: [adapter],
+};
+
+export default svmlPackage;
diff --git a/packages/provider-xiaomi-mimo/src/index.ts b/packages/provider-xiaomi-mimo/src/index.ts
new file mode 100644
index 00000000..20268e1f
--- /dev/null
+++ b/packages/provider-xiaomi-mimo/src/index.ts
@@ -0,0 +1,6 @@
+export {
+ createXiaomiMimoProvider,
+ xiaomiMimoProviderImplementationDigest,
+ xiaomiMimoProviderModuleRef,
+} from "./provider.js";
+export type { CreateXiaomiMimoProviderOptions } from "./provider.js";
diff --git a/packages/provider-xiaomi-mimo/src/provider.ts b/packages/provider-xiaomi-mimo/src/provider.ts
new file mode 100644
index 00000000..782a9fcd
--- /dev/null
+++ b/packages/provider-xiaomi-mimo/src/provider.ts
@@ -0,0 +1,259 @@
+import { defineEndpointPackage } from "@narratage/endpoint-kit";
+import type { EndpointFulfillment, EndpointInvocationContext } from "@narratage/endpoint-kit";
+import {
+ generationTypes,
+ sealGeneratedAudioSet,
+} from "@narratage/generation";
+import type { GenerationMediaValue, GenerationRequest } from "@narratage/generation";
+import { canonicalize, digestOf, isDigest } from "@narratage/protocol";
+import type { CanonicalValue, CapabilityRef, Digest } from "@narratage/protocol";
+import { credentialRef } from "@narratage/runtime";
+import type { CredentialRef } from "@narratage/runtime";
+
+export const xiaomiMimoProviderModuleRef = { name: "@narratage/provider-xiaomi-mimo", version: "1" } as const;
+export const xiaomiMimoProviderImplementationDigest = digestOf(
+ "@narratage/provider-xiaomi-mimo/chat-completions-tts@1",
+);
+
+const mimoModelModule = { name: "@narratage/mimo-tts", version: "1" } as const;
+const modelNames = [
+ "mimo-v2.5-tts",
+ "mimo-v2.5-tts-voicedesign",
+ "mimo-v2.5-tts-voiceclone",
+] as const;
+type Model = typeof modelNames[number];
+
+const capabilities = Object.fromEntries(modelNames.map((name) => [name, {
+ module: mimoModelModule,
+ name,
+}])) as Readonly<Record<Model, CapabilityRef>>;
+
+type Fetch = typeof globalThis.fetch;
+
+export type CreateXiaomiMimoProviderOptions = {
+ readonly instance?: string;
+ readonly lane?: string;
+ readonly apiBaseUrl?: string;
+ readonly apiKey?: CredentialRef;
+ readonly defaultConcurrency?: number;
+ readonly requestTimeoutMs?: number;
+ readonly maxResponseBytes?: number;
+ readonly maxVoiceSampleBase64Bytes?: number;
+ readonly fetch?: Fetch;
+ readonly fetchImplementationDigest?: Digest;
+};
+
+function assert(condition: unknown, message: string): asserts condition {
+ if (!condition) throw new Error(message);
+}
+
+function positiveInteger(value: number, subject: string): number {
+ assert(Number.isSafeInteger(value) && value > 0, `${subject} must be a positive integer`);
+ return value;
+}
+
+function normalizeBaseUrl(value: string): string {
+ const url = new URL(value);
+ assert(url.protocol === "https:" || url.hostname === "localhost" || url.hostname === "127.0.0.1",
+ "Xiaomi MiMo apiBaseUrl must use HTTPS or loopback");
+ return url.href.replace(/\/$/u, "");
+}
+
+function request(value: CanonicalValue, model: Model): GenerationRequest {
+ const result = value as unknown as GenerationRequest;
+ assert(result.contract === "svml.generation-request@1", "Xiaomi MiMo received a non-generation request");
+ assert(result.model === model, `Xiaomi MiMo ${model} received ${result.model}`);
+ assert(result.ports !== null && typeof result.ports === "object", "Xiaomi MiMo request has no ports");
+ return result;
+}
+
+function scalar(req: GenerationRequest, name: string, required = true): string | undefined {
+ const values = req.ports[name];
+ if (values === undefined) {
+ if (required) throw new Error(`${req.model} requires ${name}`);
+ return undefined;
+ }
+ assert(values.length === 1 && typeof values[0] === "string", `${req.model} ${name} must contain one string`);
+ return values[0];
+}
+
+function credential(context: EndpointInvocationContext): string {
+ const value = context.credentials.apiKey?.secret;
+ assert(value !== undefined && value.length > 0, "Xiaomi MiMo API key is unavailable");
+ return value;
+}
+
+async function readVoiceSample(
+ context: EndpointInvocationContext,
+ req: GenerationRequest,
+ maxBase64Bytes: number,
+): Promise<string> {
+ const values = req.ports.sample;
+ assert(values?.length === 1, "MiMo voice clone requires exactly one sample");
+ const value = values[0] as GenerationMediaValue;
+ assert(value.role === "audio", "MiMo voice clone sample must be audio");
+ const supported = new Set(["audio/mpeg", "audio/mp3", "audio/wav", "audio/x-wav"]);
+ assert(supported.has(value.artifact.mediaType), "Xiaomi MiMo accepts only MP3 or WAV voice samples");
+ const encodedSize = 4 * Math.ceil(value.artifact.size / 3);
+ assert(encodedSize <= maxBase64Bytes,
+ `MiMo voice sample exceeds the configured ${maxBase64Bytes}-byte Base64 limit`);
+ const bytes = await context.artifacts.get(value.artifact.digest);
+ assert(bytes !== undefined, `MiMo voice sample ${value.artifact.digest} is unavailable`);
+ assert(bytes.byteLength === value.artifact.size, "MiMo voice sample size differs from its BlobRef");
+ const mediaType = value.artifact.mediaType === "audio/x-wav"
+ ? "audio/wav"
+ : value.artifact.mediaType === "audio/mp3" ? "audio/mpeg" : value.artifact.mediaType;
+ const encoded = Buffer.from(bytes).toString("base64");
+ assert(Buffer.byteLength(encoded, "utf8") <= maxBase64Bytes,
+ `MiMo voice sample exceeds the configured ${maxBase64Bytes}-byte Base64 limit`);
+ return `data:${mediaType};base64,${encoded}`;
+}
+
+function supports(model: Model, value: CanonicalValue): boolean {
+ const req = value as unknown as GenerationRequest;
+ if (req?.contract !== "svml.generation-request@1" || req.model !== model
+ || req.ports === null || typeof req.ports !== "object") return false;
+ const allowed = model === "mimo-v2.5-tts"
+ ? new Set(["text", "instruction", "voice"])
+ : model === "mimo-v2.5-tts-voicedesign"
+ ? new Set(["text", "voiceDescription"])
+ : new Set(["text", "instruction", "sample"]);
+ return Object.keys(req.ports).every((name) => allowed.has(name));
+}
+
+function parseAudio(text: string, maxAudioBytes: number): Uint8Array {
+ let raw: unknown;
+ try {
+ raw = JSON.parse(text);
+ } catch {
+ throw new Error("Xiaomi MiMo returned invalid JSON");
+ }
+ const root = raw as { choices?: Array<{ message?: { audio?: { data?: unknown } } }> };
+ const data = root.choices?.[0]?.message?.audio?.data;
+ assert(typeof data === "string" && data.length > 0, "Xiaomi MiMo returned no audio data");
+ assert(data.length % 4 === 0 && /^[A-Za-z0-9+/]*={0,2}$/u.test(data),
+ "Xiaomi MiMo returned malformed Base64 audio");
+ const bytes = Uint8Array.from(Buffer.from(data, "base64"));
+ assert(bytes.byteLength > 0, "Xiaomi MiMo returned empty audio");
+ assert(bytes.byteLength <= maxAudioBytes, "Xiaomi MiMo audio exceeds the configured limit");
+ return bytes;
+}
+
+async function limitedResponseText(response: Response, maxBytes: number): Promise<string> {
+ const announced = response.headers.get("content-length");
+ if (announced !== null) {
+ const size = Number(announced);
+ assert(Number.isSafeInteger(size) && size >= 0, "Xiaomi MiMo returned invalid Content-Length");
+ assert(size <= maxBytes, "Xiaomi MiMo response exceeds the configured limit");
+ }
+ const reader = response.body?.getReader();
+ if (reader === undefined) {
+ const bytes = new Uint8Array(await response.arrayBuffer());
+ assert(bytes.byteLength <= maxBytes, "Xiaomi MiMo response exceeds the configured limit");
+ return Buffer.from(bytes).toString("utf8");
+ }
+ const chunks: Uint8Array[] = [];
+ let size = 0;
+ while (true) {
+ const item = await reader.read();
+ if (item.done) break;
+ size += item.value.byteLength;
+ if (size > maxBytes) {
+ await reader.cancel();
+ throw new Error("Xiaomi MiMo response exceeds the configured limit");
+ }
+ chunks.push(item.value);
+ }
+ const bytes = new Uint8Array(size);
+ let offset = 0;
+ chunks.forEach((chunk) => { bytes.set(chunk, offset); offset += chunk.byteLength; });
+ return Buffer.from(bytes).toString("utf8");
+}
+
+function fulfillment(value: CanonicalValue, metadata: CanonicalValue): EndpointFulfillment {
+ return { value: { kind: "inline", value }, conformance: "exact", delivery: "executed", metadata };
+}
+
+export function createXiaomiMimoProvider(options: CreateXiaomiMimoProviderOptions = {}) {
+ const apiBaseUrl = normalizeBaseUrl(options.apiBaseUrl ?? "https://api.xiaomimimo.com/v1");
+ const requestTimeoutMs = positiveInteger(options.requestTimeoutMs ?? 180_000, "requestTimeoutMs");
+ const maxResponseBytes = positiveInteger(options.maxResponseBytes ?? 64 * 1024 * 1024, "maxResponseBytes");
+ const maxVoiceSampleBase64Bytes = positiveInteger(
+ options.maxVoiceSampleBase64Bytes ?? 10_000_000,
+ "maxVoiceSampleBase64Bytes",
+ );
+ if (options.fetch !== undefined && options.fetchImplementationDigest === undefined) {
+ throw new Error("custom Xiaomi MiMo fetch requires fetchImplementationDigest");
+ }
+ if (options.fetchImplementationDigest !== undefined && !isDigest(options.fetchImplementationDigest)) {
+ throw new Error("Xiaomi MiMo fetchImplementationDigest is invalid");
+ }
+ const fetcher = options.fetch ?? globalThis.fetch;
+ const transportDigest = options.fetchImplementationDigest ?? digestOf("@narratage/provider-xiaomi-mimo/global-fetch@1");
+ const endpointCapabilities = modelNames.map((model) => ({
+ capability: capabilities[model],
+ returns: generationTypes.audioSet,
+ lifecycle: "immediate" as const,
+ supports: (need: { readonly constraints: CanonicalValue }) => supports(model, need.constraints),
+ handler: async (context: EndpointInvocationContext) => {
+ const req = request(context.need.constraints, model);
+ const text = scalar(req, "text")!;
+ const messages: Array<{ role: "user" | "assistant"; content: string }> = [];
+ const audio: Record<string, CanonicalValue> = { format: "wav" };
+ if (model === "mimo-v2.5-tts") {
+ const instruction = scalar(req, "instruction", false);
+ if (instruction !== undefined) messages.push({ role: "user", content: instruction });
+ audio.voice = scalar(req, "voice")!;
+ } else if (model === "mimo-v2.5-tts-voicedesign") {
+ messages.push({ role: "user", content: scalar(req, "voiceDescription")! });
+ } else {
+ const instruction = scalar(req, "instruction", false);
+ if (instruction !== undefined) messages.push({ role: "user", content: instruction });
+ audio.voice = await readVoiceSample(context, req, maxVoiceSampleBase64Bytes);
+ }
+ messages.push({ role: "assistant", content: text });
+ const controller = new AbortController();
+ const timer = setTimeout(() => controller.abort(new Error("Xiaomi MiMo request timed out")), requestTimeoutMs);
+ let response: Response;
+ let responseText: string;
+ try {
+ response = await fetcher(`${apiBaseUrl}/chat/completions`, {
+ method: "POST",
+ headers: { "content-type": "application/json", "api-key": credential(context) },
+ body: JSON.stringify({ model, messages, audio }),
+ signal: controller.signal,
+ });
+ responseText = await limitedResponseText(response, maxResponseBytes);
+ } finally {
+ clearTimeout(timer);
+ }
+ if (!response.ok) throw new Error(`Xiaomi MiMo returned HTTP ${response.status}: ${responseText.slice(0, 300)}`);
+ const bytes = parseAudio(responseText, maxResponseBytes);
+ const artifact = await context.artifacts.put(bytes, "audio/wav");
+ const result = sealGeneratedAudioSet({ contract: "svml.generated-audio-set@1", audios: [artifact] });
+ return fulfillment(result as unknown as CanonicalValue, canonicalize({
+ provider: "xiaomi-mimo",
+ requestedModel: model,
+ outputBytes: bytes.byteLength,
+ }));
+ },
+ }));
+
+ return defineEndpointPackage({
+ module: xiaomiMimoProviderModuleRef,
+ facet: "tts",
+ instance: options.instance ?? "xiaomi-mimo.default",
+ ...(options.lane === undefined ? {} : { lane: options.lane }),
+ implementation: {
+ locator: "@narratage/provider-xiaomi-mimo/tts",
+ digest: xiaomiMimoProviderImplementationDigest,
+ },
+ permissions: [`network:${new URL(apiBaseUrl).hostname}`],
+ configuration: canonicalize({
+ apiBaseUrl, requestTimeoutMs, maxResponseBytes, maxVoiceSampleBase64Bytes, transportDigest,
+ }),
+ credentials: { apiKey: options.apiKey ?? credentialRef("env", "MIMO_API_KEY") },
+ defaultConcurrency: options.defaultConcurrency ?? 2,
+ capabilities: endpointCapabilities,
+ });
+}
diff --git a/packages/provider-xiaomi-mimo/test/provider.test.ts b/packages/provider-xiaomi-mimo/test/provider.test.ts
new file mode 100644
index 00000000..2b0754f5
--- /dev/null
+++ b/packages/provider-xiaomi-mimo/test/provider.test.ts
@@ -0,0 +1,127 @@
+import assert from "node:assert/strict";
+import test from "node:test";
+
+import { EndpointRegistry, MemoryArtifactStore } from "@narratage/driver-node";
+import { generationTypes } from "@narratage/generation";
+import { mimoTtsEndpoints, sealMimoTtsRequest } from "@narratage/mimo-tts";
+import { digestOf } from "@narratage/protocol";
+import type { CanonicalValue, Need } from "@narratage/protocol";
+import {
+ createXiaomiMimoProvider,
+ xiaomiMimoProviderImplementationDigest,
+} from "@narratage/provider-xiaomi-mimo";
+
+function need(
+ endpoint: (typeof mimoTtsEndpoints)[keyof typeof mimoTtsEndpoints],
+ constraints: CanonicalValue,
+ id: string,
+): Need {
+ return {
+ id: `need:${id}`,
+ capability: endpoint.capability,
+ returns: endpoint.returns,
+ constraints,
+ requestedBy: `derivation:${id}`,
+ result: `record:${id}`,
+ accepts: "exact",
+ conformanceFloor: "exact",
+ requestDigest: digestOf({ capability: endpoint.capability, constraints }),
+ };
+}
+
+async function invoke(request: Need, artifacts: MemoryArtifactStore, fetch: typeof globalThis.fetch) {
+ const provider = createXiaomiMimoProvider({
+ fetch,
+ fetchImplementationDigest: digestOf("xiaomi-mimo:test-fetch"),
+ });
+ const registry = new EndpointRegistry();
+ await provider.install(registry);
+ const resolution = registry.resolve(request);
+ assert.equal(resolution.status, "resolved");
+ assert.equal(resolution.registration.kind, "immediate");
+ return await resolution.registration.handler({
+ command: { kind: "fulfill-need", id: `command:${request.id}`, need: request },
+ need: request,
+ artifacts,
+ credentials: { apiKey: { secret: "test-key" } },
+ });
+}
+
+function audioResponse(bytes: Uint8Array) {
+ return Response.json({ choices: [{ message: { audio: { data: Buffer.from(bytes).toString("base64") } } }] });
+}
+
+test("the official Provider maps all three model contracts without owning them", async () => {
+ const artifacts = new MemoryArtifactStore();
+ const sample = await artifacts.put(new Uint8Array([1, 2, 3, 4]), "audio/wav");
+ const cases = [
+ {
+ endpoint: mimoTtsEndpoints.preset,
+ request: sealMimoTtsRequest("mimo-v2.5-tts", {
+ text: ["Keep every authored word."], instruction: ["Warm and concise."], voice: ["Chloe"],
+ }),
+ assertBody(body: Record<string, unknown>) {
+ assert.deepEqual(body.audio, { format: "wav", voice: "Chloe" });
+ assert.equal((body.messages as unknown[]).length, 2);
+ },
+ },
+ {
+ endpoint: mimoTtsEndpoints.voiceDesign,
+ request: sealMimoTtsRequest("mimo-v2.5-tts-voicedesign", {
+ text: ["Keep every authored word."], voiceDescription: ["A clear, grounded female voice."],
+ }),
+ assertBody(body: Record<string, unknown>) {
+ assert.deepEqual(body.audio, { format: "wav" });
+ assert.equal((body.messages as unknown[]).length, 2);
+ },
+ },
+ {
+ endpoint: mimoTtsEndpoints.voiceClone,
+ request: sealMimoTtsRequest("mimo-v2.5-tts-voiceclone", {
+ text: ["Keep every authored word."], sample: [{ role: "audio", artifact: sample }],
+ }),
+ assertBody(body: Record<string, unknown>) {
+ assert.match((body.audio as Record<string, string>).voice!, /^data:audio\/wav;base64,/u);
+ assert.equal((body.messages as unknown[]).length, 1);
+ },
+ },
+ ];
+ for (const [index, item] of cases.entries()) {
+ let calls = 0;
+ const result = await invoke(
+ need(item.endpoint, item.request as unknown as CanonicalValue, String(index)),
+ artifacts,
+ async (input, init) => {
+ calls += 1;
+ assert.equal(String(input), "https://api.xiaomimimo.com/v1/chat/completions");
+ assert.equal(new Headers(init?.headers).get("api-key"), "test-key");
+ const body = JSON.parse(String(init?.body)) as Record<string, unknown>;
+ assert.equal(body.model, item.request.model);
+ assert.equal((body.messages as Array<Record<string, unknown>>).at(-1)?.role, "assistant");
+ assert.equal((body.messages as Array<Record<string, unknown>>).at(-1)?.content, "Keep every authored word.");
+ assert.equal("optimize_text_preview" in (body.audio as Record<string, unknown>), false);
+ item.assertBody(body);
+ return audioResponse(new Uint8Array([9, 8, 7, index]));
+ },
+ );
+ assert.equal(calls, 1);
+ assert.equal(result.value.kind, "inline");
+ const set = result.value.kind === "inline" ? result.value.value as Record<string, unknown> : {};
+ assert.equal(set.contract, "svml.generated-audio-set@1");
+ const audios = set.audios as Array<{ mediaType: string; digest: string }>;
+ assert.equal(audios[0]?.mediaType, "audio/wav");
+ assert.equal(await artifacts.has(audios[0]!.digest as `sha256:${string}`), true);
+ }
+});
+
+test("Provider configuration owns credentials and queue policy, not model semantics", () => {
+ const provider = createXiaomiMimoProvider({ defaultConcurrency: 3 });
+ const facet = provider.manifest.facets[0];
+ assert(facet?.role === "capability-endpoint");
+ assert.equal(facet.defaultConcurrency, 3);
+ assert.deepEqual(facet.credentialSlots, ["apiKey"]);
+ assert.equal(facet.implementation.digest, xiaomiMimoProviderImplementationDigest);
+ assert.equal(provider.bindings.length, 3);
+ assert.ok(provider.bindings.every((binding) => binding.returns.name === generationTypes.audioSet.name));
+ assert.equal(JSON.stringify(provider.manifest).includes("voiceDescription"), false);
+});
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index de7cb622..b88628ec 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -68,6 +68,9 @@ importers:
'@narratage/media-track':
specifier: workspace:*
version: link:packages/media-track
+ '@narratage/mimo-tts':
+ specifier: workspace:*
+ version: link:packages/mimo-tts
'@narratage/narrative':
specifier: workspace:*
version: link:packages/narrative
@@ -95,6 +98,9 @@ importers:
'@narratage/provider-whisperx-local':
specifier: workspace:*
version: link:packages/provider-whisperx-local
+ '@narratage/provider-xiaomi-mimo':
+ specifier: workspace:*
+ version: link:packages/provider-xiaomi-mimo
'@narratage/ranking':
specifier: workspace:*
version: link:packages/ranking
@@ -1363,6 +1369,30 @@ importers:
specifier: workspace:*
version: link:../text
+ packages/mimo-tts:
+ dependencies:
+ '@narratage/artifact':
+ specifier: workspace:*
+ version: link:../artifact
+ '@narratage/elaborator':
+ specifier: workspace:*
+ version: link:../elaborator
+ '@narratage/generation':
+ specifier: workspace:*
+ version: link:../generation
+ '@narratage/model-kit':
+ specifier: workspace:*
+ version: link:../model-kit
+ '@narratage/narrative':
+ specifier: workspace:*
+ version: link:../narrative
+ '@narratage/protocol':
+ specifier: workspace:*
+ version: link:../protocol
+ '@narratage/text':
+ specifier: workspace:*
+ version: link:../text
+
packages/minimax-h3:
dependencies:
'@narratage/generation':
@@ -1816,6 +1846,27 @@ importers:
specifier: workspace:*
version: link:../driver-node
+ packages/provider-xiaomi-mimo:
+ dependencies:
+ '@narratage/endpoint-kit':
+ specifier: workspace:*
+ version: link:../endpoint-kit
+ '@narratage/generation':
+ specifier: workspace:*
+ version: link:../generation
+ '@narratage/protocol':
+ specifier: workspace:*
+ version: link:../protocol
+ '@narratage/runtime':
+ specifier: workspace:*
+ version: link:../runtime
+ '@narratage/runtime-adapter':
+ specifier: workspace:*
+ version: link:../runtime-adapter
+ '@narratage/runtime-adapter-node':
+ specifier: workspace:*
+ version: link:../runtime-adapter-node
+
packages/ranking:
dependencies:
'@narratage/artifact':
diff --git a/tsconfig.json b/tsconfig.json
index 63e7a8d0..38ccfac1 100644
--- a/tsconfig.json
+++ b/tsconfig.json
@@ -42,6 +42,9 @@
"@narratage/media-track": [
"packages/media-track/src/index.ts"
],
+ "@narratage/mimo-tts": [
+ "packages/mimo-tts/src/index.ts"
+ ],
"@narratage/deck-track": [
"packages/deck-track/src/index.ts"
],
@@ -186,6 +189,9 @@
"@narratage/provider-media-local": [
"packages/provider-media-local/src/index.ts"
],
+ "@narratage/provider-xiaomi-mimo": [
+ "packages/provider-xiaomi-mimo/src/index.ts"
+ ],
"@narratage/provider-hyperframes-aws-lambda": [
"packages/provider-hyperframes-aws-lambda/src/index.ts"
],