Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

fix: preserve request fields and repair local preparation and file I/O (#278)

kashorin2bed7774e5be
diff --git a/.github/workflows/npm-package.yml b/.github/workflows/npm-package.yml
index f10c74e2..d6cb794e 100644
--- a/.github/workflows/npm-package.yml
+++ b/.github/workflows/npm-package.yml
@@ -22,7 +22,11 @@ concurrency:
jobs:
package:
- runs-on: ubuntu-latest
+ runs-on: ${{ matrix.os }}
+ strategy:
+ fail-fast: false
+ matrix:
+ os: [ubuntu-latest, windows-latest]
timeout-minutes: 25
outputs:
version: ${{ steps.version.outputs.version }}
@@ -36,20 +40,23 @@ jobs:
node-version: 24
cache: pnpm
- run: pnpm install --frozen-lockfile
- - name: Prepare the local media toolchain
+ - name: Prepare the local media toolchain on Linux
+ if: runner.os == 'Linux'
run: |
sudo apt-get update
sudo apt-get install -y ffmpeg
+ - name: Prepare the local media toolchain on Windows
+ if: runner.os == 'Windows'
+ run: choco install ffmpeg --yes --no-progress
- name: Read the package version
id: version
run: node --input-type=module -e 'import { readFileSync, appendFileSync } from "node:fs"; const { version } = JSON.parse(readFileSync("package.json", "utf8")); appendFileSync(process.env.GITHUB_OUTPUT, `version=${version}\n`);'
- name: Build the npm package and README
run: npm run pack:distribution
- name: Install, render and export outside the checkout
- env:
- VERSION: ${{ steps.version.outputs.version }}
- run: npm run check:distribution -- "$PWD/dist/release/hypit-hypit-$VERSION.tgz"
+ run: npm run check:distribution -- "dist/release/hypit-hypit-${{ steps.version.outputs.version }}.tgz"
- uses: actions/upload-artifact@v4
+ if: runner.os == 'Linux'
with:
name: npm-package
path: |
diff --git a/bin/hypit.mjs b/bin/hypit.mjs
index a7b59645..7dc29a91 100755
--- a/bin/hypit.mjs
+++ b/bin/hypit.mjs
@@ -1,9 +1,9 @@
#!/usr/bin/env node
import { register } from "tsx/esm/api";
-import { readFileSync } from "node:fs";
-import { dirname, resolve } from "node:path";
-import { fileURLToPath } from "node:url";
+import { readFileSync, realpathSync } from "node:fs";
+import { dirname, resolve, sep } from "node:path";
+import { fileURLToPath, pathToFileURL } from "node:url";
if (process.argv.length === 3 && ["--version", "-v"].includes(process.argv[2])) {
const manifest = JSON.parse(readFileSync(new URL("../package.json", import.meta.url), "utf8"));
@@ -18,27 +18,30 @@ process.emitWarning = function hypitWarning(warning, ...args) {
return emitWarning.call(process, warning, ...args);
};
-const distributionRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
+// Bootstrap and package activation must agree on the physical Distribution root. Windows short
+// paths can survive Node's ordinary resolution while package lookup expands them through libuv.
+const distributionRoot = realpathSync.native(resolve(dirname(fileURLToPath(import.meta.url)), ".."));
+const distributionUrl = pathToFileURL(distributionRoot + sep);
register();
const {
installDistributionPackageResolution,
installExternalPackageResolution,
-} = await import("../packages/package-loader-node/src/distribution-resolution.ts");
+} = await import(new URL("packages/package-loader-node/src/distribution-resolution.ts", distributionUrl).href);
installDistributionPackageResolution([distributionRoot]);
-const { hypitHostPackageRoot } = await import("../packages/runtime-host-node/src/index.ts");
+const { hypitHostPackageRoot } = await import(new URL("packages/runtime-host-node/src/index.ts", distributionUrl).href);
installExternalPackageResolution([hypitHostPackageRoot()]);
const args = process.argv.slice(2);
if (args[0] === "studio" || (args[0] === "help" && args[1] === "studio")) {
- const { runStudio } = await import("../packages/studio/start.ts");
+ const { runStudio } = await import(new URL("packages/studio/start.ts", distributionUrl).href);
try {
await runStudio(args[0] === "help" ? ["--help"] : args.slice(1).filter((arg) => arg !== "--debug"), {
write: (text) => process.stdout.write(text),
});
} catch (error) {
- const { renderCliError } = await import("../packages/cli/src/index.ts");
+ const { renderCliError } = await import(new URL("packages/cli/src/index.ts", distributionUrl).href);
process.stderr.write(renderCliError(error, { debug: args.includes("--debug") }));
process.exitCode = 1;
}
} else {
- await import("../packages/video-cli/src/cli.ts");
+ await import(new URL("packages/video-cli/src/cli.ts", distributionUrl).href);
}
diff --git a/docs/guide/runtime.md b/docs/guide/runtime.md
index 489eb645..295307f0 100644
--- a/docs/guide/runtime.md
+++ b/docs/guide/runtime.md
@@ -73,20 +73,30 @@ request does not silently select another account. Read the chosen Provider's REA
## Prepare only the services needed now
+For local media processing, prepare the selected tools and start the Worker:
+
```bash
-hypit auth status media.local
-hypit doctor --endpoint media.local
hypit runtime up --endpoint media.local
hypit runtime status
```
+To prepare resources without starting a helper or Worker, use
+`hypit programs prepare --endpoint media.local`. Inspect their readiness with
+`hypit programs status --endpoint media.local`; use `hypit doctor --endpoint media.local` when
+checking a configuration or diagnosing a failure. These are available tools, not a required sequence.
+
+Local media processing declares no credentials. For a chosen service that declares credential slots,
+inspect those separately with `hypit auth status <endpoint>`. For example, after choosing HypiHub,
+use `hypit auth status hypihub.default` and connect the account with `hypit auth login hypihub.default`
+when needed.
+
Use the actual Endpoint names from the Profile; repeat `--endpoint` for several. Omission covers the
whole Profile. `doctor` reads configuration and performs the selected Providers' diagnostics without
submitting generation. Read warnings as well as errors: a stored credential or reachable catalogue
is not proof that every request will succeed. With no Runtime selected, doctor checks project Results only.
`runtime up` prepares the selected local dependencies and Managed Programs, then starts the Worker.
-It does not log into or start hosted services. `programs up|status|down` manages those local helpers
+It does not log into or start hosted services. `programs prepare|up|status|down` manages those local resources and helpers
separately. Initial inference setup can require substantial downloads; compare that effort with hosted
execution and choose the route before starting preparation.
diff --git a/docs/zh/guide/runtime.md b/docs/zh/guide/runtime.md
index 6b8bb06b..f42bd245 100644
--- a/docs/zh/guide/runtime.md
+++ b/docs/zh/guide/runtime.md
@@ -69,19 +69,27 @@ hypit runtime init
## 按当前需要准备服务
+本地媒体处理可以直接准备所选工具并启动 Worker:
+
```bash
-hypit auth status media.local
-hypit doctor --endpoint media.local
hypit runtime up --endpoint media.local
hypit runtime status
```
+只准备资源、暂不启动助手或 Worker 时,使用 `hypit programs prepare --endpoint media.local`。
+通过 `hypit programs status --endpoint media.local` 查看就绪情况;检查配置或排查失败时,
+使用 `hypit doctor --endpoint media.local`。按当前问题选择命令,不必依次执行所有检查。
+
+本地媒体处理没有凭据要求。选中的服务若声明了凭据槽,再单独使用
+`hypit auth status <endpoint>` 检查。例如,选择 HypiHub 后使用
+`hypit auth status hypihub.default`,需要连接账户时执行 `hypit auth login hypihub.default`。
+
使用 Profile 里的实际 Endpoint 名称,可重复 `--endpoint` 选择多个;省略时覆盖整个 Profile。
`doctor` 读取配置、运行 Provider 的诊断,不提交生成。除了错误,也要阅读警告:有凭据或能读取
模型目录,不代表每种请求一定成功。没有选定 Runtime 时,doctor 只检查项目 Result。
`runtime up` 准备所选本地依赖和 Managed Program,再启动 Worker;它不登录或启动托管服务。
-`programs up|status|down` 单独管理这些本地助手。首次推理环境准备可能涉及大量下载,
+`programs prepare|up|status|down` 单独管理这些本地资源和助手。首次推理环境准备可能涉及大量下载,
应先比较本地准备成本和托管方式,再选择执行路径。
`plan <run>` 检查当前工作需要的能力和轻量就绪状态。发现 Provider 能力需要可加载的包声明;
diff --git a/packages/build-result/package.json b/packages/build-result/package.json
index e711dc99..b2e385e5 100644
--- a/packages/build-result/package.json
+++ b/packages/build-result/package.json
@@ -9,6 +9,6 @@
},
"dependencies": {
"@hypit/protocol": "workspace:*",
- "koffi": "3.2.1"
+ "@hypit/file-io-node": "workspace:*"
}
}
diff --git a/packages/build-result/src/store.ts b/packages/build-result/src/store.ts
index 1e0a1b4f..3e2245ce 100644
--- a/packages/build-result/src/store.ts
+++ b/packages/build-result/src/store.ts
@@ -43,7 +43,7 @@ import {
encodeBuildResultManifest,
} from "./decode.js";
import { syncBuildResultOutputs } from "./writer.js";
-import { replaceFile } from "./replace-file.js";
+import { replaceFile } from "@hypit/file-io-node";
const manifestName = "result.json";
const writerStateName = ".writer.json";
diff --git a/packages/credential-store-file/README.md b/packages/credential-store-file/README.md
index adfdd8a7..1f363f93 100644
--- a/packages/credential-store-file/README.md
+++ b/packages/credential-store-file/README.md
@@ -47,7 +47,8 @@ report the read error rather than treating corruption as a missing credential.
## Storage
The default directory is `credentials` under the Host state root printed by `hypit paths`.
-`config.path` explicitly selects a directory, resolving relative paths against that Host root.
+`config.path` explicitly selects a directory, including an absolute directory outside that root;
+relative paths resolve against the Host root.
Opening the adapter and reading an absent key do not create files. Keep this directory outside Git.
Each key has its own JSON file containing only `secret` and optional `expiresAt`, as defined by
@@ -56,7 +57,11 @@ CredentialValue. Filenames use reversible lower-case hex encoding of the key's U
case-insensitive filesystems and keeps slashes and reserved names out of filesystem path syntax.
Filesystem path-length limits still apply; an overlong key fails rather than selecting another name.
-Writes create an owner-only temporary file and replace the single destination with rename. Different
-keys do not overwrite each other's updates; concurrent writes to the same key keep the last completed
-replacement. The Store does not enumerate credentials, maintain an index, or record write history.
+Writes create an owner-only temporary file and replace the single destination through `@hypit/file-io-node`.
+On Windows this uses native POSIX replacement semantics, so an open reader keeps the old file
+while a new open sees the replacement; it does not use Node’s `MoveFileExW` path. Different
+keys do not overwrite each other's updates. Reads see a complete document, and the last completed
+replacement of the same key supplies its value. There is no process-local queue or cross-process
+coordination; filesystem errors propagate to the caller rather than being retried or hidden.
+The Store does not enumerate credentials, maintain an index, or record write history.
Permission and I/O errors propagate. Existing directory permissions are not silently changed.
diff --git a/packages/credential-store-file/package.json b/packages/credential-store-file/package.json
index 6a18efb0..79849549 100644
--- a/packages/credential-store-file/package.json
+++ b/packages/credential-store-file/package.json
@@ -16,6 +16,7 @@
},
"dependencies": {
"@hypit/runtime": "workspace:*",
- "@hypit/runtime-kit": "workspace:*"
+ "@hypit/runtime-kit": "workspace:*",
+ "@hypit/file-io-node": "workspace:*"
}
}
diff --git a/packages/credential-store-file/src/activation.ts b/packages/credential-store-file/src/activation.ts
index f72c0335..803a88a4 100644
--- a/packages/credential-store-file/src/activation.ts
+++ b/packages/credential-store-file/src/activation.ts
@@ -1,10 +1,10 @@
+import { join, resolve } from "node:path";
import {
createRuntimeCredentialStoreAdapterFacet,
runtimeConfigExact,
runtimeConfigObject,
runtimeConfigString,
} from "@hypit/runtime-kit";
-import { resolveCredentialDirectory } from "./paths.js";
import { FileCredentialStore } from "./store.js";
const fileCredentialStoreAdapter = createRuntimeCredentialStoreAdapterFacet({
@@ -18,7 +18,7 @@ const fileCredentialStoreAdapter = createRuntimeCredentialStoreAdapterFacet({
const config = runtimeConfigObject(context.config, "file CredentialStore");
const path = runtimeConfigString(config.path, "file credential path");
return { value: new FileCredentialStore(
- resolveCredentialDirectory(context.hostStateRoot, path, "file CredentialStore"),
+ path === undefined ? join(context.hostStateRoot, "credentials") : resolve(context.hostStateRoot, path),
) };
},
});
diff --git a/packages/credential-store-file/src/index.ts b/packages/credential-store-file/src/index.ts
index 97d1909d..481b1203 100644
--- a/packages/credential-store-file/src/index.ts
+++ b/packages/credential-store-file/src/index.ts
@@ -1,2 +1 @@
export { FileCredentialStore } from "./store.js";
-export { resolveCredentialDirectory } from "./paths.js";
diff --git a/packages/credential-store-file/src/paths.ts b/packages/credential-store-file/src/paths.ts
deleted file mode 100644
index 3a8905e5..00000000
--- a/packages/credential-store-file/src/paths.ts
+++ /dev/null
@@ -1,16 +0,0 @@
-import { isAbsolute, join, relative, resolve, sep } from "node:path";
-
-/** Profile `path` is Host-state-relative; it must not write secrets outside that root. */
-export function resolveCredentialDirectory(
- hostStateRoot: string,
- path: string | undefined,
- label: string,
-): string {
- const root = resolve(hostStateRoot);
- const directory = path === undefined ? join(root, "credentials") : resolve(root, path);
- const relation = relative(root, directory);
- if (relation === ".." || relation.startsWith(`..${sep}`) || isAbsolute(relation)) {
- throw new Error(`${label} path must stay inside the Host state root`);
- }
- return directory;
-}
diff --git a/packages/credential-store-file/src/store.ts b/packages/credential-store-file/src/store.ts
index 2beb790f..73f82de7 100644
--- a/packages/credential-store-file/src/store.ts
+++ b/packages/credential-store-file/src/store.ts
@@ -1,5 +1,6 @@
+import { replaceFile } from "@hypit/file-io-node";
import { randomUUID } from "node:crypto";
-import { mkdir, open, rename, rm, stat, unlink } from "node:fs/promises";
+import { mkdir, open, rm, stat, unlink } from "node:fs/promises";
import { join } from "node:path";
import { verifyCredentialRef } from "@hypit/runtime";
import type { CredentialRef, CredentialValue, WritableCredentialStore } from "@hypit/runtime";
@@ -8,26 +9,6 @@ function missing(error: unknown): boolean {
return error instanceof Error && "code" in error && error.code === "ENOENT";
}
-/**
- * One process reads a credential for one Endpoint action while another action's refresh replaces
- * it, so a read handle and a replacement can reach the same document at the same instant. Windows
- * refuses to replace a file any handle still holds. Each document's work takes its turn instead,
- * keyed by path so separate Store instances over one directory share the same order. Keys are
- * independent: a turn on one document never delays another.
- */
-const documentTurns = new Map<string, Promise<unknown>>();
-
-function onDocument<T>(path: string, work: () => Promise<T>): Promise<T> {
- const turn = (documentTurns.get(path) ?? Promise.resolve()).then(work, work);
- // The successor waits for this turn to settle, not to succeed, and the last turn clears the path.
- const settled = turn.then(() => undefined, () => undefined);
- documentTurns.set(path, settled);
- void settled.then(() => {
- if (documentTurns.get(path) === settled) documentTurns.delete(path);
- });
- return turn;
-}
-
function credentialValue(value: unknown): CredentialValue {
if (value === null || typeof value !== "object" || Array.isArray(value)) throw new Error("invalid credential value");
const record = value as Record<string, unknown>;
@@ -64,10 +45,6 @@ export class FileCredentialStore implements WritableCredentialStore {
async resolve(ref: CredentialRef): Promise<CredentialValue | undefined> {
if (!this.owns(ref)) return undefined;
const path = this.#path(ref);
- return await onDocument(path, async () => await this.#read(path));
- }
-
- async #read(path: string): Promise<CredentialValue | undefined> {
try { await this.#privateDirectory(false); } catch (error) {
if (missing(error)) return undefined;
throw error;
@@ -93,25 +70,21 @@ export class FileCredentialStore implements WritableCredentialStore {
async put(ref: CredentialRef, value: CredentialValue): Promise<void> {
const path = this.#path(ref);
const contents = JSON.stringify(credentialValue(value));
- await onDocument(path, async () => {
- await this.#privateDirectory(true);
- const temporary = join(this.directory, `.write-${randomUUID()}.tmp`);
- const file = await open(temporary, "wx", 0o600);
- try {
- try { await file.writeFile(`${contents}\n`, "utf8"); } finally { await file.close(); }
- await rename(temporary, path);
- } finally { await rm(temporary, { force: true }); }
- });
+ await this.#privateDirectory(true);
+ const temporary = join(this.directory, `.write-${randomUUID()}.tmp`);
+ const file = await open(temporary, "wx", 0o600);
+ try {
+ try { await file.writeFile(`${contents}\n`, "utf8"); } finally { await file.close(); }
+ await replaceFile(temporary, path);
+ } finally { await rm(temporary, { force: true }); }
}
async delete(ref: CredentialRef): Promise<boolean> {
const path = this.#path(ref);
// Deletion never opens or decodes the old document.
- return await onDocument(path, async () => {
- try { await unlink(path); return true; } catch (error) {
- if (missing(error)) return false;
- throw error;
- }
- });
+ try { await unlink(path); return true; } catch (error) {
+ if (missing(error)) return false;
+ throw error;
+ }
}
}
diff --git a/packages/credential-store-file/test/concurrency.test.ts b/packages/credential-store-file/test/concurrency.test.ts
index de8bd58a..721e6b56 100644
--- a/packages/credential-store-file/test/concurrency.test.ts
+++ b/packages/credential-store-file/test/concurrency.test.ts
@@ -1,6 +1,6 @@
import assert from "node:assert/strict";
import { spawn } from "node:child_process";
-import { mkdtemp, readdir, rm } from "node:fs/promises";
+import { mkdtemp, open, readdir, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
@@ -21,15 +21,28 @@ type Exit = { code: number; stderr: string };
* would let that layout pass, which is why the barrier is the assertion.
*/
async function placeSimultaneously(directory: string, entries: { key: string; secret: string }[]): Promise<Exit[]> {
- const startAt = Date.now() + 3_000;
- return Promise.all(entries.map(({ key, secret }) => new Promise<Exit>((settle) => {
- const writer = spawn(process.execPath, ["--import", "tsx", fixture, directory, key, secret, String(startAt)], {
- cwd: distribution, stdio: ["ignore", "ignore", "pipe"], windowsHide: true,
+ const children = entries.map(({ key, secret }) => {
+ const writer = spawn(process.execPath, ["--import", "tsx", fixture, directory, key, secret], {
+ cwd: distribution, stdio: ["ignore", "ignore", "pipe", "ipc"], windowsHide: true,
});
let stderr = "";
- writer.stderr.on("data", (chunk) => { stderr += String(chunk); });
- writer.on("close", (code) => settle({ code: code ?? 1, stderr }));
- })));
+ writer.stderr!.on("data", chunk => { stderr += String(chunk); });
+ const ready = new Promise<void>((resolve, reject) => {
+ writer.once("message", () => resolve());
+ writer.once("error", reject);
+ writer.once("close", () => reject(new Error(`writer exited before release: ${stderr}`)));
+ });
+ const exit = new Promise<Exit>((resolve, reject) => {
+ writer.once("error", reject);
+ writer.once("close", code => resolve({ code: code ?? 1, stderr }));
+ });
+ return { writer, ready, exit };
+ });
+ try {
+ await Promise.all(children.map(child => child.ready));
+ for (const child of children) child.writer.send("write");
+ return await Promise.all(children.map(child => child.exit));
+ } finally { for (const child of children) child.writer.kill(); }
}
test("processes placing different keys at one instant keep every credential", { timeout: 120_000 }, async () => {
@@ -60,18 +73,31 @@ test("two processes placing the same key leave one complete value", { timeout: 1
const directory = join(root, "private");
const entries = [{ key: "shared", secret: "first" }, { key: "shared", secret: "second" }];
const exits = await placeSimultaneously(directory, entries);
- // Separate processes share no order, and Windows refuses to replace a document while another
- // replacement of it is in flight. A writer that loses that contest reports why, and one of them
- // places the credential. Within one process this Store orders the work and both writers place.
- assert.ok(exits.some((exit) => exit.code === 0),
- `no writer placed the credential: ${exits.map((exit) => exit.stderr).join(" ")}`);
for (const [index, exit] of exits.entries()) {
- if (exit.code !== 0) assert.notEqual(exit.stderr.trim(), "", `writer ${index} failed without saying why`);
+ assert.equal(exit.code, 0, `writer ${index} failed: ${exit.stderr}`);
}
// Which replacement lands last is the filesystem's business; that the value is one writer's
// whole value, and never a mixture of the two, is this Store's.
const resolved = await new FileCredentialStore(directory).resolve(credentialRef("file", "shared"));
- assert.ok(["first", "second"].includes(resolved?.secret ?? ""), `expected one complete value, got ${JSON.stringify(resolved)}`);
+ assert.ok(entries.some(entry => entry.secret === resolved?.secret), `expected one complete value, got ${JSON.stringify(resolved)}`);
assert.equal((await readdir(directory)).length, 1);
} finally { await rm(root, { recursive: true, force: true }); }
});
+
+
+test("a separate process replaces a credential while its previous document is open", { timeout: 120_000 }, async () => {
+ const directory = await mkdtemp(join(tmpdir(), "hypit-file-credentials-reader-"));
+ try {
+ const store = new FileCredentialStore(directory);
+ const ref = credentialRef("file", "shared");
+ await store.put(ref, { secret: "old" });
+ const [name] = await readdir(directory);
+ const reader = await open(join(directory, name!), "r");
+ try {
+ const [exit] = await placeSimultaneously(directory, [{ key: "shared", secret: "new" }]);
+ assert.equal(exit!.code, 0, exit!.stderr);
+ assert.deepEqual(await store.resolve(ref), { secret: "new" });
+ assert.deepEqual(JSON.parse(await reader.readFile("utf8")), { secret: "old" });
+ } finally { await reader.close(); }
+ } finally { await rm(directory, { recursive: true, force: true }); }
+});
diff --git a/packages/credential-store-file/test/failed-replacement.test.ts b/packages/credential-store-file/test/failed-replacement.test.ts
new file mode 100644
index 00000000..a9c249e0
--- /dev/null
+++ b/packages/credential-store-file/test/failed-replacement.test.ts
@@ -0,0 +1,46 @@
+import assert from "node:assert/strict";
+import fs from "node:fs/promises";
+import { syncBuiltinESMExports } from "node:module";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import test from "node:test";
+import { credentialRef } from "@hypit/runtime";
+import { FileCredentialStore } from "../src/store.js";
+
+test("a refused replacement preserves the old credential, cleans its temporary and reports the I/O error", { skip: process.platform === "win32" }, async t => {
+ const root = await fs.mkdtemp(join(tmpdir(), "hypit-refused-credential-"));
+ t.after(() => fs.rm(root, { recursive: true, force: true }));
+ const store = new FileCredentialStore(root);
+ const ref = credentialRef("file", "example");
+ await store.put(ref, { secret: "original" });
+ const before = await fs.readdir(root);
+ const refusal = Object.assign(new Error("replacement refused"), { code: "EPERM", syscall: "rename" });
+ const rename = t.mock.method(fs, "rename", async () => { throw refusal; });
+ syncBuiltinESMExports();
+ try {
+ await assert.rejects(store.put(ref, { secret: "replacement" }), error => error === refusal);
+ assert.deepEqual(await store.resolve(ref), { secret: "original" });
+ assert.deepEqual(await fs.readdir(root), before);
+ } finally { rename.mock.restore(); syncBuiltinESMExports(); }
+ await store.put(ref, { secret: "next" });
+ assert.deepEqual(await store.resolve(ref), { secret: "next" });
+});
+
+
+test("Windows refuses a read-only destination without deleting the old credential", { skip: process.platform !== "win32" }, async () => {
+ const root = await fs.mkdtemp(join(tmpdir(), "hypit-refused-credential-"));
+ const store = new FileCredentialStore(root);
+ const ref = credentialRef("file", "example");
+ await store.put(ref, { secret: "original" });
+ const before = await fs.readdir(root);
+ const path = join(root, before[0]!);
+ try {
+ await fs.chmod(path, 0o444);
+ await assert.rejects(store.put(ref, { secret: "replacement" }), /Windows file replacement failed/u);
+ assert.deepEqual(await store.resolve(ref), { secret: "original" });
+ assert.deepEqual(await fs.readdir(root), before);
+ } finally {
+ await fs.chmod(path, 0o600);
+ await fs.rm(root, { recursive: true, force: true });
+ }
+});
diff --git a/packages/credential-store-file/test/paths.test.ts b/packages/credential-store-file/test/paths.test.ts
deleted file mode 100644
index 8f10bd00..00000000
--- a/packages/credential-store-file/test/paths.test.ts
+++ /dev/null
@@ -1,25 +0,0 @@
-import assert from "node:assert/strict";
-import { mkdtemp, rm } from "node:fs/promises";
-import { tmpdir } from "node:os";
-import { join, resolve } from "node:path";
-import test from "node:test";
-
-import { resolveCredentialDirectory } from "../src/paths.js";
-
-test("credential directories stay inside the Host state root", async () => {
- const root = await mkdtemp(join(tmpdir(), "hypit-credential-root-"));
- try {
- assert.equal(resolveCredentialDirectory(root, undefined, "file CredentialStore"), join(root, "credentials"));
- assert.equal(resolveCredentialDirectory(root, "nested/keys", "file CredentialStore"), resolve(root, "nested/keys"));
- assert.throws(
- () => resolveCredentialDirectory(root, "..", "file CredentialStore"),
- /file CredentialStore path must stay inside the Host state root/u,
- );
- assert.throws(
- () => resolveCredentialDirectory(root, join("foo", "..", "..", "outside"), "file CredentialStore"),
- /file CredentialStore path must stay inside the Host state root/u,
- );
- } finally {
- await rm(root, { recursive: true, force: true });
- }
-});
diff --git a/packages/credential-store-file/test/store-writer-fixture.ts b/packages/credential-store-file/test/store-writer-fixture.ts
index ee021996..7f06d442 100644
--- a/packages/credential-store-file/test/store-writer-fixture.ts
+++ b/packages/credential-store-file/test/store-writer-fixture.ts
@@ -1,15 +1,16 @@
+import { once } from "node:events";
import { credentialRef } from "@hypit/runtime";
import { FileCredentialStore } from "../src/store.js";
-const [directory, key, secret, startAt] = process.argv.slice(2);
+const [directory, key, secret] = process.argv.slice(2);
-// The parent releases every writer at the same wall-clock instant, because the failure this guards
-// against is a shared document read, modified and replaced: it survives sequential writes and a
-// spread-out schedule, and only loses a key when two replacements overlap.
async function place(): Promise<void> {
const store = new FileCredentialStore(directory!);
- while (Date.now() < Number(startAt)) await new Promise((done) => setTimeout(done, 0));
+ const release = once(process, "message");
+ process.send!({ ready: true });
+ await release;
await store.put(credentialRef("file", key!), { secret: secret! });
+ process.disconnect();
}
place().catch((error: unknown) => {
diff --git a/packages/credential-store-os/runtime/windows-credential.ps1 b/packages/credential-store-os/runtime/windows-credential.ps1
index 4117b3a8..0fc2fe38 100644
--- a/packages/credential-store-os/runtime/windows-credential.ps1
+++ b/packages/credential-store-os/runtime/windows-credential.ps1
@@ -35,8 +35,7 @@ switch ($Operation) {
@{ found = $true; secret = [Convert]::ToBase64String($bytes) } | ConvertTo-Json -Compress
}
"write" {
- $existing = Find-Credential
- if ($null -ne $existing) { $vault.Remove($existing) }
+ # Add replaces the same resource/account; do not destroy the old value before it succeeds.
$bytes = [Convert]::FromBase64String([string]$request.secret)
$secret = [System.Text.Encoding]::UTF8.GetString($bytes)
$credential = New-Object Windows.Security.Credentials.PasswordCredential(
diff --git a/packages/credential-store-os/src/store.ts b/packages/credential-store-os/src/store.ts
index c9feffca..5aac6ba8 100644
--- a/packages/credential-store-os/src/store.ts
+++ b/packages/credential-store-os/src/store.ts
@@ -1,5 +1,5 @@
-import { execFile, spawn } from "node:child_process";
-import { fileURLToPath } from "node:url";
+import { execFile } from "node:child_process";
+import { windowsCredential } from "./windows.js";
import { verifyCredentialRef } from "@hypit/runtime";
import type {
@@ -59,72 +59,6 @@ function macosDeleter(service: string): OsCredentialDeleter {
});
}
-type WindowsCredentialResult = {
- readonly found?: boolean;
- readonly deleted?: boolean;
- readonly secret?: string;
-};
-
-const windowsScript = fileURLToPath(new URL("../runtime/windows-credential.ps1", import.meta.url));
-
-const windowsCredentialTimeoutMs = 10_000;
-
-function windowsCredential(
- operation: "read" | "write" | "delete",
- service: string,
- account: string,
- secret?: string,
-): Promise<WindowsCredentialResult> {
- return new Promise((resolve, reject) => {
- const child = spawn("powershell.exe", [
- "-NoLogo", "-NoProfile", "-NonInteractive", "-ExecutionPolicy", "Bypass",
- "-File", windowsScript, "-Operation", operation,
- ], { shell: false, windowsHide: true, stdio: ["pipe", "pipe", "pipe"] });
- let stdout = "";
- let stderr = "";
- let settled = false;
- const fail = (error: Error): void => {
- if (settled) return;
- settled = true;
- clearTimeout(timer);
- reject(error);
- };
- const timer = setTimeout(() => {
- fail(new Error(`Windows credential ${operation} for ${account} timed out`));
- child.kill("SIGKILL");
- }, windowsCredentialTimeoutMs);
- child.on("error", fail);
- child.stdout.on("data", (chunk: Buffer) => {
- stdout += chunk.toString("utf8");
- if (stdout.length > 4 * 1024 * 1024) fail(new Error("Windows credential response is too large"));
- });
- child.stderr.on("data", (chunk: Buffer) => {
- stderr += chunk.toString("utf8");
- if (stderr.length > 64 * 1024) fail(new Error("Windows credential error is too large"));
- });
- child.on("close", (code) => {
- if (settled) return;
- clearTimeout(timer);
- if (code !== 0) {
- fail(new Error(`Windows credential ${operation} for ${account} failed${stderr.trim().length === 0 ? "" : `: ${stderr.trim()}`}`));
- return;
- }
- try {
- const result = JSON.parse(stdout.replace(/^\uFEFF/u, "").trim()) as WindowsCredentialResult;
- settled = true;
- resolve(result);
- } catch {
- fail(new Error(`Windows credential ${operation} returned an invalid response`));
- }
- });
- child.stdin.end(JSON.stringify({
- service,
- account,
- ...(secret === undefined ? {} : { secret: Buffer.from(secret, "utf8").toString("base64") }),
- }));
- });
-}
-
function windowsBackend(): {
readonly read: OsCredentialReader;
readonly write: OsCredentialWriter;
diff --git a/packages/credential-store-os/src/windows.ts b/packages/credential-store-os/src/windows.ts
new file mode 100644
index 00000000..cc4ab6d4
--- /dev/null
+++ b/packages/credential-store-os/src/windows.ts
@@ -0,0 +1,47 @@
+import { execFile } from "node:child_process";
+import { fileURLToPath } from "node:url";
+
+type WindowsCredentialResult = {
+ readonly found?: boolean;
+ readonly deleted?: boolean;
+ readonly secret?: string;
+};
+
+const windowsScript = fileURLToPath(new URL("../runtime/windows-credential.ps1", import.meta.url));
+
+/** The OS adapter owns this one child; Node owns its timeout, bounded output and termination. */
+export function windowsCredential(
+ operation: "read" | "write" | "delete",
+ service: string,
+ account: string,
+ secret?: string,
+): Promise<WindowsCredentialResult> {
+ return new Promise((resolve, reject) => {
+ const child = execFile("powershell.exe", [
+ "-NoLogo", "-NoProfile", "-NonInteractive", "-ExecutionPolicy", "Bypass",
+ "-File", windowsScript, "-Operation", operation,
+ ], {
+ shell: false, windowsHide: true, encoding: "utf8", timeout: 10_000,
+ maxBuffer: 4 * 1024 * 1024,
+ }, (error, stdout, stderr) => {
+ if (error !== null) {
+ reject(new Error(`Windows credential ${operation} for ${account} failed`
+ + (error.code === undefined ? "" : ` (${error.code})`)
+ + (error.killed ? " (child terminated)" : "")
+ + (stderr.trim().length === 0 ? "" : `: ${stderr.trim()}`)));
+ return;
+ }
+ try { resolve(JSON.parse(stdout.replace(/^\uFEFF/u, "").trim()) as WindowsCredentialResult); }
+ catch { reject(new Error(`Windows credential ${operation} returned an invalid response`)); }
+ });
+ // A closed input pipe is an operation failure too. Do not leave the child waiting for a request.
+ child.stdin!.on("error", () => {
+ child.kill();
+ reject(new Error(`Windows credential ${operation} request could not be written`));
+ });
+ child.stdin!.end(JSON.stringify({
+ service, account,
+ ...(secret === undefined ? {} : { secret: Buffer.from(secret, "utf8").toString("base64") }),
+ }));
+ });
+}
diff --git a/packages/credential-store-os/test/windows.test.ts b/packages/credential-store-os/test/windows.test.ts
new file mode 100644
index 00000000..1eeb1b48
--- /dev/null
+++ b/packages/credential-store-os/test/windows.test.ts
@@ -0,0 +1,50 @@
+import assert from "node:assert/strict";
+import childProcess from "node:child_process";
+import type { ChildProcess, ExecFileOptionsWithStringEncoding } from "node:child_process";
+import { once } from "node:events";
+import { syncBuiltinESMExports } from "node:module";
+import { randomUUID } from "node:crypto";
+import test from "node:test";
+import { credentialRef } from "@hypit/runtime";
+import { OsCredentialStore } from "../src/store.js";
+import { windowsCredential } from "../src/windows.js";
+
+test("an oversized bridge response terminates the child and keeps secret output out of the error", async t => {
+ const execFile = childProcess.execFile;
+ let child: ChildProcess | undefined;
+ let closed: Promise<unknown> | undefined;
+ const launch = t.mock.method(childProcess, "execFile", (
+ _command: string, _args: readonly string[], options: ExecFileOptionsWithStringEncoding,
+ callback: (error: childProcess.ExecFileException | null, stdout: string, stderr: string) => void,
+ ) => {
+ child = execFile(process.execPath, ["-e", 'process.stdin.resume(); process.stdout.write("secret".repeat(800000)); setInterval(() => {}, 1000);'], options, callback);
+ closed = once(child, "close");
+ return child;
+ });
+ syncBuiltinESMExports();
+ try {
+ await assert.rejects(windowsCredential("read", "test", "test"), error => {
+ assert.match(String(error), /Windows credential read.*failed/u);
+ assert.doesNotMatch(String(error), /secret/u);
+ return true;
+ });
+ await closed;
+ assert.equal(child?.killed, true);
+ } finally { launch.mock.restore(); syncBuiltinESMExports(); child?.kill(); }
+});
+
+test("Windows locker replaces credentials and retains the previous value if replacement fails", {
+ skip: process.platform !== "win32", timeout: 60_000,
+}, async () => {
+ const store = new OsCredentialStore({ service: `hypit-test-${randomUUID()}` });
+ const ref = credentialRef("os", "replacement");
+ try {
+ await store.put(ref, { secret: "original" });
+ await store.put(ref, { secret: "updated" });
+ assert.deepEqual(await store.resolve(ref), { secret: "updated" });
+ await assert.rejects(store.put(ref, { secret: "x".repeat(32_000) }));
+ assert.deepEqual(await store.resolve(ref), { secret: "updated" });
+ assert.equal(await store.delete(ref), true);
+ assert.equal(await store.resolve(ref), undefined);
+ } finally { await store.delete(ref); }
+});
diff --git a/packages/credential-store-platform/README.md b/packages/credential-store-platform/README.md
index 5b123bbb..8c2dc153 100644
--- a/packages/credential-store-platform/README.md
+++ b/packages/credential-store-platform/README.md
@@ -53,7 +53,8 @@ credential stored in a locker stays in that locker; a credential stored in a fil
On macOS and Windows the credential is held by the platform locker, with exactly the rules of
`@hypit/credential-store-os`: it is encrypted and access-controlled by the operating system, and it
-is not readable from another machine.
+follows that locker’s access and synchronization policy. Windows Credential Locker may roam
+credentials through the user’s Microsoft account.
On Linux the credential is held by an owner-private document with exactly the rules of
`@hypit/credential-store-file`: unencrypted JSON, one document per key, directory mode `0700`, file
diff --git a/packages/credential-store-platform/src/activation.ts b/packages/credential-store-platform/src/activation.ts
index 3539fdd1..3cd58d7e 100644
--- a/packages/credential-store-platform/src/activation.ts
+++ b/packages/credential-store-platform/src/activation.ts
@@ -1,10 +1,10 @@
+import { join, resolve } from "node:path";
import {
createRuntimeCredentialStoreAdapterFacet,
runtimeConfigExact,
runtimeConfigObject,
runtimeConfigString,
} from "@hypit/runtime-kit";
-import { resolveCredentialDirectory } from "@hypit/credential-store-file";
import { PlatformCredentialStore } from "./store.js";
const platformCredentialStoreAdapter = createRuntimeCredentialStoreAdapterFacet({
@@ -23,7 +23,7 @@ const platformCredentialStoreAdapter = createRuntimeCredentialStoreAdapterFacet(
value: new PlatformCredentialStore({
// Linux uses the same directory the file Store uses by default, so a Profile
// that switches between them on Linux finds the credential it stored.
- directory: resolveCredentialDirectory(context.hostStateRoot, path, "platform CredentialStore"),
+ directory: path === undefined ? join(context.hostStateRoot, "credentials") : resolve(context.hostStateRoot, path),
...(service === undefined ? {} : { service }),
}),
};
diff --git a/packages/file-io-node/README.md b/packages/file-io-node/README.md
new file mode 100644
index 00000000..857fcd97
--- /dev/null
+++ b/packages/file-io-node/README.md
@@ -0,0 +1,20 @@
+# `@hypit/file-io-node`
+
+Node filesystem operations shared by file-backed adapters. This package has no knowledge of
+credentials, Build Results, Runtime or repository formats.
+
+`replaceFile(from, to)` publishes a complete file at a destination on the same filesystem. The
+caller owns creating and closing the source file, its contents and permissions, and temporary-file
+cleanup. Existing readers retain the old file; subsequent opens see the new file. The destination
+may be absent. Concurrent replacements are ordered by the filesystem, with the last replacement
+supplying the current value.
+
+POSIX uses Node's `rename`. Windows uses `SetFileInformationByHandle(FileRenameInfoEx)` with
+`REPLACE_IF_EXISTS | POSIX_SEMANTICS`, the same primitive previously owned by `build-result`.
+This avoids `MoveFileExW` rejecting replacement while another process has the destination open.
+The native structure layout comes from Koffi; constants are Windows API values.
+
+There is no pre-read, queue, lock file, retry or fallback. Windows filesystems must support this
+operation; unsupported operations, access denials and other I/O errors are reported to the caller.
+Read-only destinations are not overridden. The package does not promise power-loss durability or
+coordinate transactions across multiple files.
diff --git a/packages/file-io-node/package.json b/packages/file-io-node/package.json
new file mode 100644
index 00000000..e8f1306d
--- /dev/null
+++ b/packages/file-io-node/package.json
@@ -0,0 +1,13 @@
+{
+ "name": "@hypit/file-io-node",
+ "version": "0.0.0-dev",
+ "license": "SEE LICENSE IN LICENSE",
+ "private": true,
+ "type": "module",
+ "exports": {
+ ".": "./src/index.ts"
+ },
+ "dependencies": {
+ "koffi": "3.2.1"
+ }
+}
diff --git a/packages/file-io-node/src/index.ts b/packages/file-io-node/src/index.ts
new file mode 100644
index 00000000..e103e2be
--- /dev/null
+++ b/packages/file-io-node/src/index.ts
@@ -0,0 +1 @@
+export { replaceFile } from "./replace-file.js";
diff --git a/packages/build-result/src/replace-file-windows.ts b/packages/file-io-node/src/replace-file-windows.ts
similarity index 92%
rename from packages/build-result/src/replace-file-windows.ts
rename to packages/file-io-node/src/replace-file-windows.ts
index 4c07b138..7e8d71d0 100644
--- a/packages/build-result/src/replace-file-windows.ts
+++ b/packages/file-io-node/src/replace-file-windows.ts
@@ -1,4 +1,4 @@
-import { toNamespacedPath } from "node:path";
+import { resolve, toNamespacedPath } from "node:path";
import koffi from "koffi";
const kernel = koffi.load("kernel32.dll");
@@ -35,7 +35,7 @@ function failure(syscall: string, from: string, to: string): Error {
* https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/4217551b-d2c0-42cb-9dc1-69a716cf6d0c
*/
export function replaceWindowsFile(from: string, to: string): void {
- const filename = Buffer.from(toNamespacedPath(to), "utf16le");
+ const filename = Buffer.from(toNamespacedPath(resolve(to)), "utf16le");
const nameOffset = koffi.offsetof(renameInfo, "FileName");
// Win32 requires a NUL-terminated FileName even though FileNameLength excludes
// that terminator. Buffer.alloc leaves the final WCHAR zero-initialized.
@@ -45,7 +45,7 @@ export function replaceWindowsFile(from: string, to: string): void {
info.writeUInt32LE(filename.length, koffi.offsetof(renameInfo, "FileNameLength"));
filename.copy(info, nameOffset);
- const handle = createFile(toNamespacedPath(from), DELETE, SHARE_READ_WRITE_DELETE,
+ const handle = createFile(toNamespacedPath(resolve(from)), DELETE, SHARE_READ_WRITE_DELETE,
null, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0) as number | bigint;
if (handle === -1 || handle === -1n) throw failure("CreateFileW", from, to);
try {
diff --git a/packages/build-result/src/replace-file.ts b/packages/file-io-node/src/replace-file.ts
similarity index 100%
rename from packages/build-result/src/replace-file.ts
rename to packages/file-io-node/src/replace-file.ts
diff --git a/packages/build-result/test/replace-file.test.ts b/packages/file-io-node/test/replace-file.test.ts
similarity index 100%
rename from packages/build-result/test/replace-file.test.ts
rename to packages/file-io-node/test/replace-file.test.ts
diff --git a/packages/generation/README.md b/packages/generation/README.md
index 00b1c83f..199bb485 100644
--- a/packages/generation/README.md
+++ b/packages/generation/README.md
@@ -37,3 +37,9 @@ Media wire mappings (`url`, `urlArray`, `itemObject`) may declare `resourceField
paths against the declared media fields. False, zero and empty strings remain values; omitted fields
remain absent. The resolver implements the service protocol; this package knows no particular
service or classification. Providers using custom transports carry those fields through that boundary.
+
+A Model item field being optional means authors may omit it. If a request supplies that field,
+`mappingSupportsRequest` and final wire compilation require the mapping to carry it through
+`fieldKeys` or `resourceFields`, including an explicit `false`. A service may support requests
+without a particular optional field and refuse requests that supply it; no field is silently dropped.
+Final compilation checks the whole request before resolving or uploading any reference.
diff --git a/packages/generation/src/mapping.ts b/packages/generation/src/mapping.ts
index e844e0ac..5bdf776e 100644
--- a/packages/generation/src/mapping.ts
+++ b/packages/generation/src/mapping.ts
@@ -89,8 +89,9 @@ export function selectWireModelForRequest(
request: GenerationRequest,
pendingPorts: readonly string[] = [],
): string {
- assert(mappingSupportsRequest(mapping, request),
- `${mapping.capability.name} request contains a port this Provider cannot map`);
+ const unsupported = unsupportedRequestField(mapping, request);
+ assert(unsupported === undefined,
+ `${mapping.capability.name} request contains ${unsupported} this Provider cannot map`);
const present = new Set(presentPorts(request));
for (const port of pendingPorts) {
assert(mapping.fields[port] !== undefined,
@@ -149,19 +150,38 @@ export async function compileWireRequest(
return { model, input: canonicalize(input) };
}
-/** Does this request only use ports this mapping can write? Needs no model package. */
-export function mappingSupportsRequest(mapping: GenerationWireMapping, value: unknown): boolean {
+/** Describe only a supplied value the mapping cannot carry; no model table or saved verdict. */
+function unsupportedRequestField(mapping: GenerationWireMapping, value: unknown): string | undefined {
const request = value as GenerationRequest | undefined;
- if (request === undefined || request.ports === null || typeof request.ports !== "object") return false;
- return Object.keys(request.ports).every((port) => mapping.fields[port] !== undefined);
+ if (request === undefined || request === null || request.ports === null || typeof request.ports !== "object") {
+ return "invalid ports";
+ }
+ for (const [port, supplied] of Object.entries(request.ports)) {
+ const field = mapping.fields[port];
+ if (field === undefined) return `port ${port}`;
+ if (field.as !== "url" && field.as !== "urlArray" && field.as !== "itemObject") continue;
+ for (const item of supplied as readonly GenerationMediaValue[]) {
+ for (const name of Object.keys(item.fields ?? {})) {
+ if (!field.resourceFields?.includes(name)
+ && !(field.as === "itemObject" && Object.hasOwn(field.fieldKeys, name))) {
+ return `field ${port}.${name}`;
+ }
+ }
+ }
+ }
+ return undefined;
+}
+
+/** Optional fields may be omitted by authors, but supplied fields must reach the service. */
+export function mappingSupportsRequest(mapping: GenerationWireMapping, value: unknown): boolean {
+ return unsupportedRequestField(mapping, value) === undefined;
}
/**
- * Prove one service mapping covers every port the model declares.
+ * Check structural port coverage and required item fields against the model declaration.
*
- * This is the check the old hand-written per-model translators could not have:
- * a forgotten reference role or item field used to surface only after a paid
- * generation returned the wrong result.
+ * A service may omit optional item capabilities. mappingSupportsRequest and final compilation
+ * additionally refuse requests that actually supply such an unmapped field.
*/
export function assertMappingCoversPorts(
table: GenerationPortTable,
diff --git a/packages/generation/test/mapping-fields.test.ts b/packages/generation/test/mapping-fields.test.ts
new file mode 100644
index 00000000..2a4ce452
--- /dev/null
+++ b/packages/generation/test/mapping-fields.test.ts
@@ -0,0 +1,46 @@
+import assert from "node:assert/strict";
+import test from "node:test";
+import { compileWireRequest, mappingSupportsRequest } from "../src/mapping.js";
+import type { GenerationWireMapping } from "../src/mapping.js";
+import type { GenerationRequest } from "../src/request.js";
+
+const mapping: GenerationWireMapping = {
+ capability: { module: { name: "example", version: "1" }, name: "video" }, result: "video",
+ routes: [{ model: "video" }], fields: { image: { as: "url", field: "image" } },
+};
+const request = (fields?: Record<string, boolean>): GenerationRequest => ({
+ ports: { image: [{ role: "image", artifact: {
+ kind: "blob", resource: "res_image", mediaType: "image/png", size: 1,
+ }, ...(fields === undefined ? {} : { fields }) }] },
+});
+
+test("a supplied optional item field must be mapped before any resource is uploaded", async () => {
+ assert.equal(mappingSupportsRequest(mapping, request()), true);
+ for (const flag of [true, false]) {
+ assert.equal(mappingSupportsRequest(mapping, request({ personReference: flag })), false);
+ let uploads = 0;
+ await assert.rejects(compileWireRequest(mapping, request({ personReference: flag }), async () => {
+ uploads++; return "https://example.test/image";
+ }), /image.personReference/u);
+ assert.equal(uploads, 0);
+ }
+});
+
+test("item fields reach their declared transport or body destination, including false", async () => {
+ for (const flag of [true, false]) {
+ const transport: GenerationWireMapping = { ...mapping, fields: {
+ image: { as: "url", field: "image", resourceFields: ["personReference"] },
+ } };
+ assert.equal(mappingSupportsRequest(transport, request({ personReference: flag })), true);
+ const wire = await compileWireRequest(transport, request({ personReference: flag }), async (_blob, fields) => {
+ assert.deepEqual(fields, { personReference: flag }); return "https://example.test/image";
+ });
+ assert.deepEqual(wire.input, { image: "https://example.test/image" });
+ const body: GenerationWireMapping = { ...mapping, fields: {
+ image: { as: "itemObject", field: "images", urlKey: "url", fieldKeys: { personReference: "person" } },
+ } };
+ assert.equal(mappingSupportsRequest(body, request({ personReference: flag })), true);
+ assert.deepEqual((await compileWireRequest(body, request({ personReference: flag }), async () => "url")).input,
+ { images: [{ url: "url", person: flag }] });
+ }
+});
diff --git a/packages/media-execution/src/toolchain.ts b/packages/media-execution/src/toolchain.ts
index 5e76f6d2..353a9fde 100644
--- a/packages/media-execution/src/toolchain.ts
+++ b/packages/media-execution/src/toolchain.ts
@@ -1,18 +1,14 @@
import { execFile } from "node:child_process";
+import { mediaProcessEnv } from "./process-env.js";
export type MediaToolchainState =
| { readonly state: "ready"; readonly ffprobeVersion: string; readonly ffmpegVersion?: string }
| { readonly state: "down" | "mismatch"; readonly detail: string };
-const REQUIRED_ENCODERS = ["aac", "libx264", "pcm_s16le"] as const;
-const REQUIRED_FILTERS = [
- "aformat", "amix", "aresample", "asetpts", "atempo", "atrim",
- "loop", "pad", "scale", "select", "setpts", "setsar", "trim",
-] as const;
-
-function run(executable: string, args: readonly string[]): Promise<{ ok: boolean; output: string }> {
+function run(executable: string, args: readonly string[], env: NodeJS.ProcessEnv): Promise<{ ok: boolean; output: string }> {
return new Promise((resolve) => {
execFile(executable, [...args], {
+ env,
timeout: 15_000,
shell: false,
windowsHide: true,
@@ -29,22 +25,18 @@ function firstLine(value: string): string {
return value.split(/\r?\n/u, 1)[0]?.trim() ?? "";
}
-function missingNames(output: string, required: readonly string[]): readonly string[] {
- return required.filter((name) => !new RegExp(`(?:^|\\s)${name.replace(/[.*+?^${}()|[\]\\]/gu, "\\$&")}(?:\\s|$)`, "mu")
- .test(output));
-}
-
/**
- * Compatibility probe for a deployment-owned FFmpeg toolchain. It deliberately
- * checks the commands the shared execution body uses instead of requiring one
- * blessed version number. A managed distribution may pin binaries separately;
- * a custom installation remains valid when it supplies the same capability.
+ * Check availability of the selected executables in their execution environment.
+ * This does not predict compatibility with every future media command: the execution
+ * implementation owns those commands and reports the tool's actual failure.
*/
export async function probeMediaToolchain(options: {
readonly ffprobePath: string;
readonly ffmpegPath?: string;
+ readonly environment?: NodeJS.ProcessEnv;
}): Promise<MediaToolchainState> {
- const probeVersion = await run(options.ffprobePath, ["-version"]);
+ const env = options.environment ?? mediaProcessEnv();
+ const probeVersion = await run(options.ffprobePath, ["-version"], env);
if (!probeVersion.ok) {
return { state: "down", detail: `${options.ffprobePath} is unavailable: ${probeVersion.output}` };
}
@@ -52,28 +44,10 @@ export async function probeMediaToolchain(options: {
if (ffprobeVersion.length === 0) return { state: "mismatch", detail: "ffprobe returned no version" };
if (options.ffmpegPath === undefined) return { state: "ready", ffprobeVersion };
- const [ffmpegVersionResult, encoders, filters] = await Promise.all([
- run(options.ffmpegPath, ["-version"]),
- run(options.ffmpegPath, ["-hide_banner", "-encoders"]),
- run(options.ffmpegPath, ["-hide_banner", "-filters"]),
- ]);
+ const ffmpegVersionResult = await run(options.ffmpegPath, ["-version"], env);
if (!ffmpegVersionResult.ok) {
return { state: "down", detail: `${options.ffmpegPath} is unavailable: ${ffmpegVersionResult.output}` };
}
- if (!encoders.ok || !filters.ok) {
- return { state: "mismatch", detail: "ffmpeg could not enumerate its encoders and filters" };
- }
- const missingEncoders = missingNames(encoders.output, REQUIRED_ENCODERS);
- const missingFilters = missingNames(filters.output, REQUIRED_FILTERS);
- if (missingEncoders.length > 0 || missingFilters.length > 0) {
- return {
- state: "mismatch",
- detail: [
- missingEncoders.length === 0 ? "" : `missing encoders ${missingEncoders.join(", ")}`,
- missingFilters.length === 0 ? "" : `missing filters ${missingFilters.join(", ")}`,
- ].filter(Boolean).join("; "),
- };
- }
const ffmpegVersion = firstLine(ffmpegVersionResult.output);
return ffmpegVersion.length === 0
? { state: "mismatch", detail: "ffmpeg returned no version" }
diff --git a/packages/media-execution/test/toolchain.test.ts b/packages/media-execution/test/toolchain.test.ts
new file mode 100644
index 00000000..53aa7b72
--- /dev/null
+++ b/packages/media-execution/test/toolchain.test.ts
@@ -0,0 +1,26 @@
+import assert from "node:assert/strict";
+import childProcess from "node:child_process";
+import type { ExecFileOptionsWithStringEncoding } from "node:child_process";
+import { syncBuiltinESMExports } from "node:module";
+import test from "node:test";
+import { probeMediaToolchain } from "../src/toolchain.js";
+
+test("tool availability uses the execution environment without assuming a task's codec requirements", async t => {
+ const original = childProcess.execFile;
+ const calls: string[][] = [];
+ const launch = t.mock.method(childProcess, "execFile", (
+ command: string, args: readonly string[], options: ExecFileOptionsWithStringEncoding,
+ callback: (error: childProcess.ExecFileException | null, stdout: string, stderr: string) => void,
+ ) => {
+ calls.push([command, ...args]);
+ return original(process.execPath, ["-e", 'process.stdout.write(process.env.SELECTED_TOOL_ENV || "")'], options, callback);
+ });
+ syncBuiltinESMExports();
+ try {
+ assert.deepEqual(await probeMediaToolchain({ ffprobePath: "chosen-probe", ffmpegPath: "chosen-encoder",
+ environment: { SELECTED_TOOL_ENV: "selected tool version" },
+ }), { state: "ready", ffprobeVersion: "selected tool version", ffmpegVersion: "selected tool version" });
+ assert.deepEqual(calls, [["chosen-probe", "-version"], ["chosen-encoder", "-version"]]);
+ assert.equal((await probeMediaToolchain({ ffprobePath: "chosen-probe", environment: {} })).state, "mismatch");
+ } finally { launch.mock.restore(); syncBuiltinESMExports(); }
+});
diff --git a/packages/provider-hyperframes-local/src/program.ts b/packages/provider-hyperframes-local/src/program.ts
index d95725ff..9c64f356 100644
--- a/packages/provider-hyperframes-local/src/program.ts
+++ b/packages/provider-hyperframes-local/src/program.ts
@@ -5,6 +5,7 @@ import { probeMediaToolchain } from "@hypit/media-execution";
import type { ManagedProgram, ManagedProgramState } from "@hypit/runtime-kit";
import { browserCacheDirectory, browserDownloadBaseUrl, browserDownloadUrl, browserExecutablePath, configuredBrowserPath, requireBrowserExecutable, selectedBrowserVersion } from "./browser.js";
import type { BrowserOptions } from "./browser.js";
+import { processEnvironment } from "./process.js";
/**
* This Provider owns browser selection and preparation. Probes never install;
@@ -44,7 +45,7 @@ export function localHyperframesBrowserProgram(
async probe(): Promise<ManagedProgramState> {
const browser = await probeBrowser();
if (browser.state !== "ready") return browser;
- const media = await probeMediaToolchain({ ffprobePath: input.ffprobePath, ...(input.ffmpegPath === undefined ? {} : { ffmpegPath: input.ffmpegPath }) });
+ const media = await probeMediaToolchain({ environment: processEnvironment(), ffprobePath: input.ffprobePath, ...(input.ffmpegPath === undefined ? {} : { ffmpegPath: input.ffmpegPath }) });
return media.state === "ready"
? { state: "ready" }
: { state: media.state, detail: media.detail };
diff --git a/packages/provider-media-local/README.md b/packages/provider-media-local/README.md
index 924d2a48..113f39b9 100644
--- a/packages/provider-media-local/README.md
+++ b/packages/provider-media-local/README.md
@@ -43,7 +43,8 @@ Source. Local HyperFrames extracts alpha-preserving PNGs and composites them aga
Canvas and lower visual layers before encoding the final MP4.
The Runtime Adapter declares the selected `ffmpeg`/`ffprobe` pair as an external, non-daemon Program.
-Its shared probe checks the encoders and filters used by the execution body. Compatible custom paths
+Its probe checks that the selected executables start in the media execution environment. It does not
+guarantee every codec or filter for every task; an unsupported operation reports FFmpeg’s actual error. Custom paths
remain valid; the package neither pins a semantic Capability to one FFmpeg version nor mutates a
system package manager.
diff --git a/packages/runtime-local/src/programs.ts b/packages/runtime-local/src/programs.ts
index 81038c56..21e40604 100644
--- a/packages/runtime-local/src/programs.ts
+++ b/packages/runtime-local/src/programs.ts
@@ -296,6 +296,8 @@ async function bringUpOwned(
const prepared = await prepareOwned(root, program, endpoint, onProgress);
if (prepared.state.state !== "ready") return prepared;
onProgress?.({ id: program.id, phase: "ready" });
+ // A usable tool may have no process. Preserve the preparation action in that case.
+ if (program.start === undefined) return prepared;
return { ...base, action: "already-running", state: initial };
}
if (initial.state === "mismatch") {
diff --git a/packages/runtime-local/test/programs.test.ts b/packages/runtime-local/test/programs.test.ts
index 3ffc6e64..3b3aab02 100644
--- a/packages/runtime-local/test/programs.test.ts
+++ b/packages/runtime-local/test/programs.test.ts
@@ -133,7 +133,7 @@ test("Provider-owned preparation reconciles a cold installation and leaves a rea
}));
try {
assert.equal((await bringManagedProgramsUp(configured.path, configured.options)).programs[0]?.action, "installed");
- assert.equal((await bringManagedProgramsUp(configured.path, configured.options)).programs[0]?.action, "already-running");
+ assert.equal((await bringManagedProgramsUp(configured.path, configured.options)).programs[0]?.action, "unchanged");
assert.equal(await readFile(join(configured.root, "prepared"), "utf8"), "prepared\n");
} finally { await rm(configured.root, { recursive: true, force: true }); }
});
@@ -219,16 +219,40 @@ test("down leaves a running program without a Hypit process record", async () =>
assert.match(result.programs[0]!.detail ?? "", /without a Hypit process record/u);
});
-test("down treats a ready probe-only Program as having nothing to stop", async () => {
- const { path, options } = await project(() => ({
+test("a ready probe-only Program is unchanged on up and has nothing to stop", async (t) => {
+ const { root, path, options } = await project(() => ({
id: "toolchain",
probe: async () => ({ state: "ready" }),
}));
- const result = await takeManagedProgramsDown(path, options);
- assert.equal(result.programs[0]!.action, "nothing-to-stop");
- assert.deepEqual(result.programs[0]!.state, { state: "ready" });
- assert.equal(result.programs[0]!.detail, undefined);
- assert.equal(result.programs[0]!.pid, undefined);
+ t.after(() => rm(root, { recursive: true, force: true }));
+ const up = await bringManagedProgramsUp(path, options);
+ assert.equal(up.programs[0]!.action, "unchanged");
+ assert.equal(up.programs[0]!.pid, undefined);
+ const down = await takeManagedProgramsDown(path, options);
+ assert.equal(down.programs[0]!.action, "nothing-to-stop");
+ assert.deepEqual(down.programs[0]!.state, { state: "ready" });
+ assert.equal(down.programs[0]!.detail, undefined);
+ assert.equal(down.programs[0]!.pid, undefined);
+});
+
+test("up reports resources installed even when a probe-only tool was already usable", async (t) => {
+ const { root, path, options } = await project(root => ({
+ id: "toolchain",
+ probe: async () => ({ state: "ready" }),
+ installation: {
+ probe: async () => {
+ try { await readFile(join(root, "resource")); return { state: "ready" }; }
+ catch { return { state: "down", detail: "resource is not prepared" }; }
+ },
+ commands: [nodeProgram("require('node:fs').writeFileSync(process.argv[1], 'ready')", join(root, "resource"))],
+ },
+ }));
+ t.after(() => rm(root, { recursive: true, force: true }));
+ const up = await bringManagedProgramsUp(path, options);
+ assert.equal(up.programs[0]!.action, "installed");
+ assert.equal(up.programs[0]!.state.state, "ready");
+ assert.equal(up.programs[0]!.pid, undefined);
+ assert.ok(up.programs[0]!.installationLogPath);
});
test("a program with nothing to start is installed once, and installation is the whole job", async () => {
diff --git a/packages/video-cli/test/file-credentials.test.ts b/packages/video-cli/test/file-credentials.test.ts
index 48788b13..e9afb8f0 100644
--- a/packages/video-cli/test/file-credentials.test.ts
+++ b/packages/video-cli/test/file-credentials.test.ts
@@ -21,7 +21,8 @@ test("an explicitly selected file Store supports CLI login, repair and logout wi
const profilePath = join(root, "hypit.runtime.json");
const profile = JSON.parse(await readFile(profilePath, "utf8"));
assert.equal(profile.credentials.platform.use, "@hypit/credential-store-platform", "initialization preserves the existing starter choice");
- profile.credentials = { file: { use: "@hypit/credential-store-file" } };
+ const directory = join(root, "chosen-credentials");
+ profile.credentials = { file: { use: "@hypit/credential-store-file", config: { path: directory } } };
profile.endpoints = { "hypihub.default": { ...profile.endpoints["hypihub.default"], config: {
...profile.endpoints["hypihub.default"].config,
apiKey: { store: "file", key: "test.oauth" },
@@ -35,7 +36,6 @@ test("an explicitly selected file Store supports CLI login, repair and logout wi
const status = (await auth("status")).stdout;
assert.equal(JSON.parse(status).credentials[0].configured, true);
assert.doesNotMatch(status, /test-secret-never-display/u);
- const directory = join(state, "credentials");
const path = join(directory, (await readdir(directory))[0]!);
await writeFile(path, "corrupt-secret-never-display");
await assert.rejects(auth("status"), error => {
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index 1efc138e..265e81c9 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -538,12 +538,12 @@ importers:
packages/build-result:
dependencies:
+ '@hypit/file-io-node':
+ specifier: workspace:*
+ version: link:../file-io-node
'@hypit/protocol':
specifier: workspace:*
version: link:../protocol
- koffi:
- specifier: 3.2.1
- version: 3.2.1
packages/build-result-fs:
dependencies:
@@ -956,6 +956,9 @@ importers:
packages/credential-store-file:
dependencies:
+ '@hypit/file-io-node':
+ specifier: workspace:*
+ version: link:../file-io-node
'@hypit/runtime':
specifier: workspace:*
version: link:../runtime
@@ -1141,6 +1144,12 @@ importers:
specifier: 3.0.0
version: 3.0.0
+ packages/file-io-node:
+ dependencies:
+ koffi:
+ specifier: 3.2.1
+ version: 3.2.1
+
packages/film:
dependencies:
'@hypit/component-kit':
diff --git a/skills/hypit/references/environment/profile.md b/skills/hypit/references/environment/profile.md
index a5738ca4..e7e17db7 100644
--- a/skills/hypit/references/environment/profile.md
+++ b/skills/hypit/references/environment/profile.md
@@ -137,7 +137,7 @@ current directory, `package.json` and explicit paths determine which project the
| Runtime Profile | which environmental packages, Endpoint instances, credentials, and bindings are selected |
| Provider Endpoint | how one capability is supported, diagnosed, invoked, and priced |
| Credential Store | how one explicitly named secret is resolved |
-| Managed Program | how a selected local Endpoint's long-lived helper is prepared and probed |
+| Managed Program | how a selected local Endpoint's resources are prepared and probed, and its helper is started when one exists |
| Project Result repository | where finished Build Results and their public Outputs live |
Changing an Endpoint does not change the Author Source. Result storage is selected separately in
@@ -257,7 +257,9 @@ the Host state root printed by `hypit paths`, and an optional Store `config.path
private directory; ensure a Windows directory's ACL is private to the user. Preserve an existing
Store choice; never switch or migrate credentials just because one Store could not read them.
-Inspect one Endpoint's credential slots without revealing their values:
+Inspect credential slots only when the selected Endpoint declares them. A credential-free Endpoint
+such as local media processing needs no auth command; prepare its tools directly. For an Endpoint
+with declared slots, inspect their state without revealing values:
```bash
hypit auth status <selected-endpoint>
@@ -345,7 +347,15 @@ remains available for a running service; process health alone does not establish
Repeat the flag for several instances. Omission deliberately prepares the whole Profile, even when
a capability is bound elsewhere. Preparation follows each Provider's declared dependencies and
Programs; it does not log into remote accounts. `hypit doctor --endpoint <instance>` actively checks
-that selected service. `plan` already narrows readiness to the Endpoints resolved for the Run.
+that selected service when diagnosis is useful. `plan` already narrows readiness to the Endpoints
+resolved for the Run. Choose the command for the current question; these are not mandatory checks
+before every Build. Resource readiness does not imply that a helper process exists.
+
+For example, local media work can use `hypit programs prepare --endpoint media.local` to prepare
+only, or `hypit runtime up --endpoint media.local` when execution needs the Worker. Neither requires
+an account. Inspect tool readiness with `hypit programs status --endpoint media.local`, and Worker
+state with `hypit runtime status`. Account connection is a separate decision for services that
+actually declare credential slots.
Use `local-tools.md` when a selected local binary or Managed Program needs installation or repair.
Read `../production/builds.md` for how submission uses the prepared environment and Worker.