Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

refactor: make planning and observation boundaries explicit

kashorin1d4124aac744
diff --git a/docs/guide/providers.md b/docs/guide/providers.md
index bdae6104..5de04afe 100644
--- a/docs/guide/providers.md
+++ b/docs/guide/providers.md
@@ -69,7 +69,7 @@ export function createMyServiceProvider(options: {
capabilities: [{
capability: myCapability,
returns: myResultType,
- lane: "generate",
+ capacity: "generate",
lifecycle: "asynchronous",
endpoint: myAsyncEndpoint,
}],
@@ -143,8 +143,10 @@ Build preflight and must never submit generation work.
A Provider that charges declares where it publishes prices, and nothing more: `pricing: { kind: "page",
url }` on `defineEndpointPackage`, pointing at the Provider's own public price page. A Provider that
runs on this machine declares `pricing: { kind: "local" }`. Hypit never copies or interprets prices;
-`hypit plan --runtime <profile>` prints the Endpoint and price page behind each external request so the
-Agent reads the Provider's page before a paid Build. An undeclared price source is reported as unknown.
+`hypit plan --runtime <profile>` prints the Endpoint behind each request. Local work is marked local;
+Provider work carries the Provider's price page so the Agent can read the source before a paid Build.
+An undeclared price source is reported as unknown. When the complete request already exists, planning
+also applies that Endpoint's ordinary `supports` check; it does not maintain a second selection table.
## 5. Declare a Managed Program when needed
diff --git a/docs/zh/guide/providers.md b/docs/zh/guide/providers.md
index a073e95e..ef09e187 100644
--- a/docs/zh/guide/providers.md
+++ b/docs/zh/guide/providers.md
@@ -66,7 +66,7 @@ export function createMyServiceProvider(options: {
capabilities: [{
capability: myCapability,
returns: myResultType,
- lane: "generate",
+ capacity: "generate",
lifecycle: "asynchronous",
endpoint: myAsyncEndpoint,
}],
@@ -133,7 +133,7 @@ Activation 还可以返回 `diagnose(context)`。它只会在用户显式运行
Runtime 会先解析该 Endpoint 自己声明的凭据槽。它可以对真实服务做一次有界只读请求,例如读取
已认证账户的模型目录;Build 预检不能调用它,它也绝不能提交生成任务。
-收费的 Provider 只声明一件事:价格发布在哪里。在 `defineEndpointPackage` 上写 `pricing: { kind: "page", url }`,指向 Provider 自己的公开价格页;在本机运行的 Provider 声明 `pricing: { kind: "local" }`。Hypit 不复制也不解释价格;`hypit plan --runtime <profile>` 会在每个外部请求旁打印承接它的 Endpoint 和价格页,Agent 在付费 Build 之前去 Provider 自己的页面读价。未声明价格来源的会被报告为未知。
+收费的 Provider 只声明一件事:价格发布在哪里。在 `defineEndpointPackage` 上写 `pricing: { kind: "page", url }`,指向 Provider 自己的公开价格页;在本机运行的 Provider 声明 `pricing: { kind: "local" }`。Hypit 不复制也不解释价格;`hypit plan --runtime <profile>` 会列出每个请求的 Endpoint,本地工作明确标成本地,Provider 工作附带它自己的价格页,Agent 在付费 Build 之前去原始页面读价。未声明价格来源的会被报告为未知。当完整请求已经存在时,Plan 也会执行同一个 Endpoint 的普通 `supports` 检查,不维护第二张选择表。
## 5. 按需声明 Managed Program
diff --git a/packages/caption-fine/README.md b/packages/caption-fine/README.md
index 55c844c4..16701aa9 100644
--- a/packages/caption-fine/README.md
+++ b/packages/caption-fine/README.md
@@ -11,7 +11,8 @@ does not carry a private visual role.
semantic={speech.semantic} program={captions}/>
```
-An optional Spatial Region Timeline replaces only the moving placement point:
+An optional Spatial Region Timeline replaces only the moving placement point. The timeline is authored
+numeric input, normally measured from the face before the Build; it is not a face-tracking request:
```xml
<space:RegionTimeline id="heads" within={vertical} recipe={tracking.heads.default}/>
@@ -29,6 +30,10 @@ external evidence: Fine does not detect people, associate identities, smooth mot
missing Frames or invoke a Provider. Without `regions`, the ordinary Recipe `x` and `y` behavior is
unchanged.
+This keeps placement inspectable and editable. If a tracked face needs padding or an above-head anchor,
+transform the measured numbers while authoring the Region Timeline, then give Caption the result. Do
+not hide that transformation in a Provider or ask the Build to rediscover the face.
+
One SVS Recipe freezes three public dimensions:
- **Where**: Region position, anchors, extent, block/inline alignment, wrapping and line limits.
diff --git a/packages/cli/README.md b/packages/cli/README.md
index f76d77a9..58775df6 100644
--- a/packages/cli/README.md
+++ b/packages/cli/README.md
@@ -15,6 +15,13 @@ Human output is compact by default. `--json` emits a stable, bounded command vie
compiler, Runtime or Repository objects. `--verbose` adds bounded operational detail; it never turns
the command into an internal state dump.
+`plan` lists every Endpoint request in the frozen Build graph. Exact-model packages expose their own
+port tables and request-assembly edges, so the CLI can show authored prompt, duration and generation
+settings without searching arbitrary records for a request-shaped object. When an input file will be
+made by an earlier Build step, that direct graph edge stays symbolic until the file exists; the rest
+of the request is still shown. A complete request is resolved through the same Endpoint Registry as
+the Build, including the Endpoint's `supports` check.
+
The implementation follows those same boundaries: `command.ts` defines the exact semantic command
union, while argument parsing and option ownership live in `arguments.ts`; project Result
browsing/export lives under `commands/results.ts`; Runtime,
diff --git a/packages/cli/package.json b/packages/cli/package.json
index fb0d3002..11f81e9f 100644
--- a/packages/cli/package.json
+++ b/packages/cli/package.json
@@ -15,6 +15,8 @@
"@hypit/core": "workspace:*",
"@hypit/driver-node": "workspace:*",
"@hypit/elaborator": "workspace:*",
+ "@hypit/generation": "workspace:*",
+ "@hypit/model-kit": "workspace:*",
"@hypit/package-loader-node": "workspace:*",
"@hypit/protocol": "workspace:*",
"@hypit/run": "workspace:*",
diff --git a/packages/cli/src/build-planning.ts b/packages/cli/src/build-planning.ts
index c4d55bf8..883ede79 100644
--- a/packages/cli/src/build-planning.ts
+++ b/packages/cli/src/build-planning.ts
@@ -1,10 +1,18 @@
import { resolve } from "node:path";
import type { NodeCompiledSourceClosure } from "@hypit/compiler-node";
+import { registerProducerFacets, registerTypeValidatorFacets } from "@hypit/component-kit";
+import { evaluateProducerPlan, ProducerRegistry } from "@hypit/driver-node";
+import { isMediaPort } from "@hypit/generation";
+import type { GenerationPortTable, GenerationRequestDraft } from "@hypit/generation";
+import { exactModelsFromHostFacets, plannedExactModelRequest } from "@hypit/model-kit";
+import type { ExactModelEndpoint } from "@hypit/model-kit";
+import type { NodePackageContribution } from "@hypit/package-loader-node";
+import type { BuildDefinition, BuildState, CanonicalValue, CapabilityRef } from "@hypit/protocol";
import type { NodeRuntimeHost } from "@hypit/runtime-host-node";
import { plannedNeeds } from "@hypit/runtime";
import type { BuildCatalogDescriptor } from "@hypit/runtime";
-import type { BuildState, CapabilityRef } from "@hypit/protocol";
+import { TypeValidatorRegistry } from "@hypit/validation";
export function createCatalogDescriptor(options: {
readonly source: string;
@@ -38,12 +46,22 @@ function capabilityName(capability: CapabilityRef): string {
return `${capability.module.name}@${capability.module.version}#${capability.name}`;
}
+function sameRef(
+ left: { readonly module: { readonly name: string; readonly version: string }; readonly name: string },
+ right: { readonly module: { readonly name: string; readonly version: string }; readonly name: string },
+): boolean {
+ return left.module.name === right.module.name
+ && left.module.version === right.module.version
+ && left.name === right.name;
+}
+
function demandedCapabilities(state: BuildState): readonly CapabilityRef[] {
const found = new Map(plannedNeeds(state).map((need) => [capabilityName(need.capability), need.capability]));
return [...found.entries()].sort(([left], [right]) => left.localeCompare(right)).map(([, value]) => value);
}
export type PlanProviderView = {
+ readonly request: string;
readonly capability: string;
readonly status: "resolved" | "unresolved" | "ambiguous";
readonly endpoint?: string;
@@ -51,70 +69,44 @@ export type PlanProviderView = {
readonly pricing?: { readonly kind: "page"; readonly url: string } | { readonly kind: "local" };
readonly endpoints?: readonly string[];
readonly binding?: string;
+ readonly checked: "request" | "capability";
};
-/** Which Endpoint and price page stand behind each demanded capability; reads the Profile only. */
-export async function describePlanProviders(host: NodeRuntimeHost, state: BuildState): Promise<readonly PlanProviderView[]> {
- const providers = await host.providers(demandedCapabilities(state));
- return providers.map((item) => ({
- capability: capabilityName(item.capability),
- status: item.status,
- ...(item.endpoint === undefined ? {} : { endpoint: item.endpoint }),
- ...(item.use === undefined ? {} : { use: item.use }),
- ...(item.pricing === undefined ? {} : { pricing: item.pricing }),
- ...(item.endpoints === undefined ? {} : { endpoints: item.endpoints }),
- ...(item.binding === undefined ? {} : { binding: item.binding }),
- }));
-}
-
-export async function preflightPlan(host: NodeRuntimeHost, state: BuildState) {
- const capabilities = demandedCapabilities(state);
- const result = await host.preflight({ capabilities });
- return {
- ok: !result.diagnostics.some((item) => item.severity === "error"),
- capabilities: capabilities.map(capabilityName),
- diagnostics: result.diagnostics,
- } as const;
-}
-
-export function assertPreflight(preflight: Awaited<ReturnType<typeof preflightPlan>>): void {
- if (preflight === undefined || preflight.ok) return;
- const errors = preflight.diagnostics.filter((item) => item.severity === "error");
- throw new Error([
- `Runtime preflight failed for ${errors.length} demanded deployment requirement${errors.length === 1 ? "" : "s"}:`,
- ...errors.map((item) => ` ${item.code}${item.subject === undefined ? "" : ` (${item.subject})`}: ${item.message}`),
- "No Build was submitted and no external capability request was made.",
- ].join("\n"));
-}
-
-// ---------------------------------------------------------------------------------------------------
-// The complete bill of needs: every request the Build will make, with its parameters known before
-// anything is paid for.
-
-import { BuildMachine } from "@hypit/core";
-import { registerProducerFacets, registerTypeValidatorFacets } from "@hypit/component-kit";
-import { EndpointRegistry, MemoryResourceStore, NodeDriver, ProducerRegistry } from "@hypit/driver-node";
-import type { NodePackageContribution } from "@hypit/package-loader-node";
-import type { BuildDefinition, CanonicalValue } from "@hypit/protocol";
-import { TypeValidatorRegistry } from "@hypit/validation";
+export type PendingPlanInput = {
+ readonly input: string;
+ readonly record: string;
+ readonly sourceStep?: string;
+ readonly kind?: "image" | "video" | "audio" | "other";
+};
-/** A reading of one request's parameters, taken from the values the request actually carries. */
+/** A reading of one request's authored parameters, never a second request payload. */
export type NeedSummary = {
- /** Scalar parameters by their own names: numbers, short strings, booleans, and `N words` for prose. */
readonly fields: Readonly<Record<string, string | number | boolean>>;
- /** Referenced media counted by kind (image, video, audio, other). */
readonly references: Readonly<Record<string, number>>;
};
+export type EvaluatedPlanNeed = {
+ readonly constraints?: CanonicalValue;
+ readonly summary?: NeedSummary;
+ readonly pending: readonly PendingPlanInput[];
+ readonly issue?: string;
+};
+
+export type EvaluatedPlan = {
+ readonly state: BuildState;
+ readonly needs: ReadonlyMap<string, EvaluatedPlanNeed>;
+};
+
export type PlanNeedView = {
+ readonly request: string;
readonly step: string;
readonly port: string;
readonly capability: string;
readonly endpoint?: string;
- /** Present when the request's parameters were computed at plan time. */
readonly summary?: NeedSummary;
- /** Why the parameters are not known yet, when they are not. */
- readonly unknown?: string;
+ readonly pending: readonly PendingPlanInput[];
+ readonly issue?: string;
+ readonly checked?: "request" | "capability";
};
function mediaKind(mediaType: string): string {
@@ -137,11 +129,19 @@ function countReferences(value: CanonicalValue, references: Record<string, numbe
for (const item of Object.values(record)) countReferences(item, references);
}
+function textMeasure(value: string): string {
+ const trimmed = value.trim();
+ const words = trimmed.split(/\s+/u).filter(Boolean);
+ const cjk = [...trimmed].filter((character) =>
+ /[\p{Script=Han}\p{Script=Hiragana}\p{Script=Katakana}\p{Script=Hangul}]/u.test(character)).length;
+ if (cjk > 0 && words.length <= 1) return `${cjk} chars`;
+ return `${words.length} words`;
+}
+
function scalarSummary(value: CanonicalValue): string | number | boolean | undefined {
if (typeof value === "number" || typeof value === "boolean") return value;
if (typeof value === "string") {
- const words = value.trim().split(/\s+/u).filter((word) => word.length > 0).length;
- return value.length <= 40 && !value.includes("\n") ? value : `${words} words`;
+ return value.length <= 40 && !value.includes("\n") ? value : textMeasure(value);
}
if (value !== null && typeof value === "object" && !Array.isArray(value)) {
const record = value as Record<string, CanonicalValue>;
@@ -150,41 +150,79 @@ function scalarSummary(value: CanonicalValue): string | number | boolean | undef
return undefined;
}
-/**
- * Read a request's parameters off the request itself: every scalar it carries under its own name,
- * every referenced Artifact counted by kind. No model-specific field list lives here; a request
- * shaped as `{ ports }` (an exact-model generation request) reads each port, anything else reads its
- * top-level fields.
- */
-export function summarizeConstraints(constraints: CanonicalValue): NeedSummary {
+function summarizeFields(source: Readonly<Record<string, CanonicalValue>>): Readonly<Record<string, string | number | boolean>> {
const fields: Record<string, string | number | boolean> = {};
- const references: Record<string, number> = {};
- countReferences(constraints, references);
- if (constraints === null || typeof constraints !== "object" || Array.isArray(constraints)) return { fields, references };
- const record = constraints as Record<string, CanonicalValue>;
- const ports = record.ports;
- const source = ports !== null && typeof ports === "object" && !Array.isArray(ports)
- ? ports as Record<string, CanonicalValue>
- : record;
for (const [name, value] of Object.entries(source)) {
const values = Array.isArray(value) ? value : [value];
const scalars = values.map(scalarSummary).filter((item): item is string | number | boolean => item !== undefined);
- if (scalars.length === 0) continue;
- fields[name] = scalars.length === 1 ? scalars[0]! : scalars.map(String).join(", ");
+ if (scalars.length > 0) fields[name] = scalars.length === 1 ? scalars[0]! : scalars.map(String).join(", ");
+ }
+ return fields;
+}
+
+/** Generic requests are summarized only from their declared top-level constraints. */
+export function summarizeConstraints(constraints: CanonicalValue): NeedSummary {
+ const references: Record<string, number> = {};
+ countReferences(constraints, references);
+ if (constraints === null || typeof constraints !== "object" || Array.isArray(constraints)) {
+ return { fields: {}, references };
+ }
+ return { fields: summarizeFields(constraints as Record<string, CanonicalValue>), references };
+}
+
+/** Exact-model requests are summarized from the model package's port table, not an object-shape guess. */
+function summarizeExactModel(
+ table: GenerationPortTable,
+ ports: GenerationRequestDraft["ports"],
+ pending: readonly PendingPlanInput[],
+): NeedSummary {
+ const fields: Record<string, string | number | boolean> = {};
+ const references: Record<string, number> = {};
+ for (const port of table.ports) {
+ const values = ports[port.name] ?? [];
+ if (isMediaPort(port)) {
+ for (const value of values) {
+ if (value === null || typeof value !== "object" || Array.isArray(value)) continue;
+ const role = (value as { readonly role?: unknown }).role;
+ if (role === "image" || role === "video" || role === "audio") references[role] = (references[role] ?? 0) + 1;
+ }
+ continue;
+ }
+ const summarized = values.map((value) => scalarSummary(value as CanonicalValue))
+ .filter((value): value is string | number | boolean => value !== undefined);
+ if (summarized.length > 0) fields[port.name] = summarized.length === 1 ? summarized[0]! : summarized.map(String).join(", ");
+ }
+ for (const item of pending) {
+ const kind = item.kind ?? "other";
+ references[kind] = (references[kind] ?? 0) + 1;
}
return { fields, references };
}
+function exactModels(contributions: readonly NodePackageContribution[]): readonly ExactModelEndpoint[] {
+ return exactModelsFromHostFacets(contributions.flatMap((item) => item.hostFacets ?? []));
+}
+
+function exactModelFor(
+ models: readonly ExactModelEndpoint[],
+ state: BuildState,
+ stepId: string,
+ capability: CapabilityRef,
+): ExactModelEndpoint | undefined {
+ const step = state.plan.steps.find((item) => item.id === stepId);
+ if (step === undefined) return undefined;
+ return models.find((model) => sameRef(model.capability, capability) && sameRef(model.producer, step.producer));
+}
+
/**
- * Run the plan's Producers that need nothing from outside, so every external request exists with
- * its exact constraints before the Build is submitted. Producers that would need an external result,
- * or bytes this planning pass does not hold, are left as they are; their requests stay unknown and
- * say why. Nothing here contacts a Provider or spends anything.
+ * Evaluate deterministic Producers only. A missing upstream Resource is kept as its direct graph
+ * edge, not described as missing author intent and never found by searching for a request-looking
+ * object.
*/
export async function evaluatePlanNeeds(
definition: BuildDefinition,
contributions: readonly NodePackageContribution[],
-): Promise<ReadonlyMap<string, { readonly constraints?: CanonicalValue; readonly unknown?: string }>> {
+): Promise<EvaluatedPlan> {
const producers = new ProducerRegistry();
const validators = new TypeValidatorRegistry();
for (const contribution of contributions) {
@@ -193,115 +231,132 @@ export async function evaluatePlanNeeds(
registerTypeValidatorFacets(validators, component.validators ?? []);
}
}
- const driver = new NodeDriver({ producers, endpoints: new EndpointRegistry(), resources: new MemoryResourceStore(), validators });
- const machine = new BuildMachine(definition);
- const failed = new Map<string, string>();
- let blockedProducers = new Map<string, string>();
- for (let round = 0; round < 10_000; round += 1) {
- const prepared = driver.prepare(machine.view());
- blockedProducers = new Map(prepared.blocked
- .filter((item) => item.reason === "missing-producer")
- .map((item) => [item.command, `no Producer for ${item.subject} is loaded at plan time`]));
- const runnable = prepared.runnable.filter((item) =>
- item.command.kind === "invoke-producer" && !failed.has(item.command.id));
- if (runnable.length === 0) break;
- let advanced = false;
- for (const descriptor of runnable) {
- let result: Awaited<ReturnType<NodeDriver["executeCommand"]>>;
- try {
- result = await driver.executeCommand(prepared.state, descriptor, { build: "plan" });
- } catch (error) {
- failed.set(descriptor.command.id, error instanceof Error ? error.message : String(error));
- continue;
- }
- if (result.status !== "completed" || result.event.kind === "command-failed") {
- failed.set(descriptor.command.id, result.status === "completed" && result.event.kind === "command-failed"
- ? result.event.message
- : "the step did not complete at plan time");
- continue;
- }
- machine.evaluate(result.event);
- machine.commit();
- advanced = true;
- break;
- }
- if (!advanced) break;
- }
- const state = machine.view();
+ const evaluation = await evaluateProducerPlan(definition, producers, validators);
+ const state = evaluation.state;
const known = new Map(state.needs.map((need) => [need.id, need]));
- const stepFailures = new Map<string, string>();
- for (const [command, message] of [...blockedProducers, ...failed]) {
- const step = state.plan.steps.find((item) => command.endsWith(item.id))?.id ?? command;
- stepFailures.set(step, message);
- }
- const result = new Map<string, { readonly constraints?: CanonicalValue; readonly unknown?: string }>();
+ const producedBy = new Map(state.plan.steps.flatMap((step) =>
+ Object.values(step.outputs).map((record) => [record, step.id] as const)));
+ const records = new Set(state.records.map((record) => record.id));
+ const models = exactModels(contributions);
+ const result = new Map<string, EvaluatedPlanNeed>();
+
for (const planned of plannedNeeds(state)) {
const need = known.get(planned.need);
if (need !== undefined) {
- result.set(planned.need, { constraints: need.constraints });
+ const model = exactModelFor(models, state, planned.step, planned.capability);
+ const exact = model === undefined ? undefined : plannedExactModelRequest(state, planned.step, planned.port, model);
+ result.set(planned.need, {
+ constraints: need.constraints,
+ summary: model === undefined || exact === undefined
+ ? summarizeConstraints(need.constraints)
+ : summarizeExactModel(model.ports, exact.ports, []),
+ pending: [],
+ });
+ continue;
+ }
+
+ const model = exactModelFor(models, state, planned.step, planned.capability);
+ const exact = model === undefined ? undefined : plannedExactModelRequest(state, planned.step, planned.port, model);
+ if (model !== undefined && exact !== undefined) {
+ const pending = exact.pendingMedia.map((item): PendingPlanInput => ({
+ input: item.port,
+ record: item.record,
+ ...(item.sourceStep === undefined ? {} : { sourceStep: item.sourceStep }),
+ kind: item.role,
+ }));
+ const issue = evaluation.failures.get(planned.step);
+ result.set(planned.need, {
+ summary: summarizeExactModel(model.ports, exact.ports, pending),
+ pending,
+ ...(issue === undefined ? {} : { issue }),
+ });
continue;
}
- const failure = stepFailures.get(planned.step);
+
const step = state.plan.steps.find((item) => item.id === planned.step);
- const draft = step === undefined ? undefined : requestDraft(state, step);
+ const pending = step === undefined ? [] : Object.entries(step.inputs)
+ .filter(([, record]) => !records.has(record))
+ .map(([input, record]): PendingPlanInput => {
+ const sourceStep = producedBy.get(record);
+ return { input, record, ...(sourceStep === undefined ? {} : { sourceStep }) };
+ });
+ const issue = evaluation.failures.get(planned.step);
result.set(planned.need, {
- ...(draft === undefined ? {} : { constraints: draft }),
- unknown: failure === undefined
- ? (draft === undefined ? "depends on an earlier external result" : "references come from an earlier external result")
- : `its step could not run at plan time: ${failure}`,
+ pending,
+ ...(issue === undefined ? {} : { issue }),
});
}
- return result;
+ return { state, needs: result };
}
-/**
- * When a request step cannot run yet because a reference it needs is still to be generated, the
- * author's own parameters already exist upstream: the request draft the step's inputs were built
- * from. Walk from the step's missing inputs to the steps that produce them, and read the first
- * present record shaped as a request (`{ ports }`). The media it would attach is what is pending.
- */
-function requestDraft(state: BuildState, step: BuildState["plan"]["steps"][number]): CanonicalValue | undefined {
- const records = new Map(state.records.map((item) => [item.id, item]));
- const producedBy = new Map(state.plan.steps.flatMap((item) => Object.values(item.outputs).map((id) => [id, item] as const)));
- const seen = new Set<string>();
- const queue = [step];
- while (queue.length > 0) {
- const current = queue.shift()!;
- if (seen.has(current.id)) continue;
- seen.add(current.id);
- for (const id of Object.values(current.inputs)) {
- const record = records.get(id);
- if (record !== undefined) {
- const value = record.value;
- if (value.kind === "inline" && value.value !== null && typeof value.value === "object"
- && !Array.isArray(value.value) && "ports" in value.value) return value.value;
- continue;
- }
- const producer = producedBy.get(id);
- if (producer !== undefined) queue.push(producer);
- }
- }
- return undefined;
+/** Resolve every planned request independently; concrete constraints exercise Endpoint `supports`. */
+export async function describePlanProviders(
+ host: NodeRuntimeHost,
+ state: BuildState,
+ evaluated: EvaluatedPlan,
+): Promise<readonly PlanProviderView[]> {
+ const requests = plannedNeeds(state).map((planned) => {
+ const constraints = evaluated.needs.get(planned.need)?.constraints;
+ return {
+ request: planned.need,
+ capability: planned.capability,
+ returns: planned.returns,
+ ...(constraints === undefined ? {} : { constraints }),
+ };
+ });
+ return (await host.providers(requests)).map((item) => ({
+ request: item.request,
+ capability: capabilityName(item.capability),
+ status: item.status,
+ checked: item.checked,
+ ...(item.endpoint === undefined ? {} : { endpoint: item.endpoint }),
+ ...(item.use === undefined ? {} : { use: item.use }),
+ ...(item.pricing === undefined ? {} : { pricing: item.pricing }),
+ ...(item.endpoints === undefined ? {} : { endpoints: item.endpoints }),
+ ...(item.binding === undefined ? {} : { binding: item.binding }),
+ }));
+}
+
+export async function preflightPlan(host: NodeRuntimeHost, state: BuildState) {
+ const capabilities = demandedCapabilities(state);
+ const result = await host.preflight({ capabilities });
+ return {
+ ok: !result.diagnostics.some((item) => item.severity === "error"),
+ capabilities: capabilities.map(capabilityName),
+ diagnostics: result.diagnostics,
+ } as const;
+}
+
+export function assertPreflight(preflight: Awaited<ReturnType<typeof preflightPlan>>): void {
+ if (preflight === undefined || preflight.ok) return;
+ const errors = preflight.diagnostics.filter((item) => item.severity === "error");
+ throw new Error([
+ `Runtime preflight failed for ${errors.length} demanded deployment requirement${errors.length === 1 ? "" : "s"}:`,
+ ...errors.map((item) => ` ${item.code}${item.subject === undefined ? "" : ` (${item.subject})`}: ${item.message}`),
+ "No Build was submitted and no external capability request was made.",
+ ].join("\n"));
}
-/** Every request in step order, with the Endpoint behind it and its parameters when known. */
-export async function describePlanNeeds(
+/** Every request in step order, joined to its own Endpoint resolution. */
+export function describePlanNeeds(
state: BuildState,
- evaluated: ReadonlyMap<string, { readonly constraints?: CanonicalValue; readonly unknown?: string }>,
+ evaluated: EvaluatedPlan,
providers: readonly PlanProviderView[],
-): Promise<readonly PlanNeedView[]> {
- const endpoints = new Map(providers.map((item) => [item.capability, item.endpoint]));
+): readonly PlanNeedView[] {
+ const endpoint = new Map(providers.map((item) => [item.request, item]));
return plannedNeeds(state).map((planned) => {
- const capability = capabilityName(planned.capability);
- const endpoint = endpoints.get(capability);
- const found = evaluated.get(planned.need);
+ const provider = endpoint.get(planned.need);
+ const found = evaluated.needs.get(planned.need);
return {
+ request: planned.need,
step: planned.step,
port: planned.port,
- capability,
- ...(endpoint === undefined ? {} : { endpoint }),
- ...(found?.constraints === undefined ? {} : { summary: summarizeConstraints(found.constraints) }),
- ...(found?.unknown === undefined ? {} : { unknown: found.unknown }),
+ capability: capabilityName(planned.capability),
+ ...(provider?.endpoint === undefined ? {} : { endpoint: provider.endpoint }),
+ ...(found?.summary === undefined ? {} : { summary: found.summary }),
+ pending: found?.pending ?? [],
+ ...(found?.issue === undefined ? {} : { issue: found.issue }),
+ ...(provider?.checked === undefined ? {} : { checked: provider.checked }),
};
});
}
diff --git a/packages/cli/src/main.ts b/packages/cli/src/main.ts
index 374608a4..3da3e360 100644
--- a/packages/cli/src/main.ts
+++ b/packages/cli/src/main.ts
@@ -412,13 +412,13 @@ export async function runCli(
}
runtime = await loadRuntime(await runtimeHost(runtimeProfile));
let built = await runtime.build(request);
- const externalRequestCount = result.definition.plan.steps.reduce(
+ const requestCount = result.definition.plan.steps.reduce(
(total, step) => total + Object.keys(step.needs).length,
0,
);
const suppliedOutputCount = runSelections.length;
const workSummary = [
- `${externalRequestCount} external ${externalRequestCount === 1 ? "request" : "requests"}`,
+ `${requestCount} ${requestCount === 1 ? "request" : "requests"}`,
...(suppliedOutputCount === 0 ? [] : [
`${suppliedOutputCount} ${suppliedOutputCount === 1 ? "Output" : "Outputs"} supplied by Run`,
]),
@@ -569,19 +569,12 @@ export async function runCli(
});
const result = loaded.compiler.planCompilation(loaded);
const planHost = runtimeProfile === undefined ? undefined : await runtimeHost(runtimeProfile);
- const preflight = planHost === undefined ? undefined : await preflightPlan(planHost, result.state);
- const providers = planHost === undefined ? undefined : await describePlanProviders(planHost, result.state);
const evaluated = await evaluatePlanNeeds(result.definition, packageContributions);
- const needs = await describePlanNeeds(result.state, evaluated, providers ?? []);
+ const preflight = planHost === undefined ? undefined : await preflightPlan(planHost, evaluated.state);
+ const providers = planHost === undefined ? undefined : await describePlanProviders(planHost, evaluated.state, evaluated);
+ const needs = describePlanNeeds(evaluated.state, evaluated, providers ?? []);
const outputNames = Object.fromEntries(result.compilation.author.exports.flatMap((item) =>
item.ref.kind === "logical-output" ? [[item.ref.id, item.name]] : []));
- const externalRequests = new Map<string, number>();
- for (const step of result.definition.plan.steps) {
- const count = Object.keys(step.needs).length;
- if (count === 0) continue;
- const operation = `${step.producer.module.name}@${step.producer.module.version}/${step.producer.name}`;
- externalRequests.set(operation, (externalRequests.get(operation) ?? 0) + count);
- }
const allChoices = result.selections.flatMap((selection) => {
const output = outputNames[selection.output];
const candidate = loaded.run.satisfactionNames[selection.output];
@@ -589,29 +582,34 @@ export async function runCli(
});
const allUnreached = unreachedGenerations(result.compilation.author.graph, result.state, outputNames)
.map((item) => ({ output: item.name, operation: item.producer }));
- const requestViews = [...externalRequests.entries()].sort(([left], [right]) => left.localeCompare(right))
- .map(([operation, count]) => ({ operation, count }));
const targets = loaded.run.document.targets.map((item) => item.output);
+ const unresolvedRequestCount = providers?.filter((item) => item.status !== "resolved").length ?? 0;
+ const localRequestCount = providers?.filter((item) => item.status === "resolved" && item.pricing?.kind === "local").length ?? 0;
+ const providerRequestCount = providers === undefined ? undefined : providers.length - localRequestCount - unresolvedRequestCount;
+ const requestIssueCount = needs.filter((item) => item.issue !== undefined).length;
writeCliOutput(io, args.presentation, {
kind: "plan",
machine: {
- format: "hypit.cli-plan@2",
- ok: preflight?.ok ?? true,
+ format: "hypit.cli-plan@3",
+ ok: (preflight?.ok ?? true) && unresolvedRequestCount === 0 && requestIssueCount === 0,
run: projectPath(loaded.path, effectiveWorkspaceRoot),
targetCount: targets.length,
targets: targets.slice(0, args.limit),
steps: result.definition.plan.steps.length,
- externalRequestCount: requestViews.reduce((total, item) => total + item.count, 0),
- externalRequests: requestViews.slice(0, args.limit),
- ...(requestViews.length <= args.limit ? {} : { omittedExternalRequests: requestViews.length - args.limit }),
+ requestCount: needs.length,
+ requestIssueCount,
+ ...(providerRequestCount === undefined ? {} : { providerRequestCount }),
+ ...(providers === undefined ? {} : { localRequestCount, unresolvedRequestCount }),
choiceCount: allChoices.length,
choices: allChoices.slice(0, args.limit),
...(allChoices.length <= args.limit ? {} : { omittedChoices: allChoices.length - args.limit }),
...(allUnreached.length === 0 ? {} : { unreached: allUnreached.slice(0, args.limit) }),
...(allUnreached.length <= args.limit ? {} : { omittedUnreached: allUnreached.length - args.limit }),
...(providers === undefined ? {} : {
- providers: providers.slice(0, args.limit),
- ...(providers.length <= args.limit ? {} : { omittedProviders: providers.length - args.limit }),
+ providers: providers.slice(0, Math.max(args.limit, 50)),
+ ...(providers.length <= Math.max(args.limit, 50)
+ ? {}
+ : { omittedProviders: providers.length - Math.max(args.limit, 50) }),
}),
needs: needs.slice(0, Math.max(args.limit, 50)),
...(needs.length <= Math.max(args.limit, 50) ? {} : { omittedNeeds: needs.length - Math.max(args.limit, 50) }),
@@ -630,7 +628,7 @@ export async function runCli(
} }),
},
});
- if (preflight !== undefined && !preflight.ok) io.setExitCode?.(1);
+ if ((preflight !== undefined && !preflight.ok) || unresolvedRequestCount > 0 || requestIssueCount > 0) io.setExitCode?.(1);
} finally {
await planResults?.close();
}
diff --git a/packages/cli/src/output.ts b/packages/cli/src/output.ts
index bbee03a7..8e4a8da9 100644
--- a/packages/cli/src/output.ts
+++ b/packages/cli/src/output.ts
@@ -96,6 +96,7 @@ export type PlanPreflight = {
};
export type PlanProvider = {
+ readonly request: string;
readonly capability: string;
readonly status: "resolved" | "unresolved" | "ambiguous";
readonly endpoint?: string;
@@ -103,18 +104,22 @@ export type PlanProvider = {
readonly pricing?: { readonly kind: "page"; readonly url: string } | { readonly kind: "local" };
readonly endpoints?: readonly string[];
readonly binding?: string;
+ readonly checked: "request" | "capability";
};
export type PlanOutput = {
- readonly format: "hypit.cli-plan@2";
+ readonly format: "hypit.cli-plan@3";
readonly ok: boolean;
readonly run: string;
readonly targetCount: number;
readonly targets: readonly string[];
readonly steps: number;
- readonly externalRequestCount: number;
- readonly externalRequests: readonly { readonly operation: string; readonly count: number }[];
- readonly omittedExternalRequests?: number;
+ readonly requestCount: number;
+ readonly requestIssueCount: number;
+ /** Present when a Runtime Profile was selected. */
+ readonly providerRequestCount?: number;
+ readonly localRequestCount?: number;
+ readonly unresolvedRequestCount?: number;
readonly choiceCount: number;
readonly choices: readonly { readonly output: string; readonly candidate: string }[];
readonly omittedChoices?: number;
@@ -130,6 +135,7 @@ export type PlanOutput = {
};
export type PlanNeed = {
+ readonly request: string;
readonly step: string;
readonly port: string;
readonly capability: string;
@@ -138,7 +144,14 @@ export type PlanNeed = {
readonly fields: Readonly<Record<string, string | number | boolean>>;
readonly references: Readonly<Record<string, number>>;
};
- readonly unknown?: string;
+ readonly pending: readonly {
+ readonly input: string;
+ readonly record: string;
+ readonly sourceStep?: string;
+ readonly kind?: "image" | "video" | "audio" | "other";
+ }[];
+ readonly issue?: string;
+ readonly checked?: "request" | "capability";
};
export type CliMachineView = OperationalMachineView;
@@ -328,11 +341,6 @@ function renderRunCheck(
return `${lines.join("\n")}\n`;
}
-function operationLabel(operation: string): string {
- const name = operation.split("/").at(-1) ?? operation;
- return name.replace(/^request-/u, "").replaceAll("-", " ");
-}
-
/** `@hypit/seedance@1#seedance-2-mini` → `@hypit/seedance#seedance-2-mini`; the module version is verbose detail. */
function capabilityLabel(name: string): string {
return name.replace(/@[^@#]+#/u, "#");
@@ -347,38 +355,44 @@ function stepLabel(step: string): string {
return at === -1 ? decoded : decoded.slice(at + marker.length);
}
-const WORDS = /^(\d+) words$/u;
+const MEASURE = /^(\d+) (words|chars)$/u;
-/** Parameters that count words are ranged across a group rather than splitting it. */
+/** Text lengths are ranged across a group rather than splitting otherwise identical requests. */
function groupKey(need: PlanNeed): string {
- if (need.summary === undefined) return need.unknown === undefined ? "?" : `?${need.unknown}`;
+ const pending = need.pending.map((item) => item.kind ?? "file").sort().join(",");
+ if (need.summary === undefined) return `?${pending}|${need.issue ?? ""}|${need.checked ?? ""}`;
const fields = Object.entries(need.summary.fields)
- .filter(([, value]) => !(typeof value === "string" && WORDS.test(value)))
- .map(([name, value]) => `${name}=${String(value)}`);
+ .map(([name, value]) => {
+ const measured = typeof value === "string" ? MEASURE.exec(value) : null;
+ return measured === null ? `${name}=${String(value)}` : `${name}=<${measured[2]}>`;
+ });
const references = Object.entries(need.summary.references).map(([kind, count]) => `${kind}=${count}`);
- return [...fields, "|", ...references, "|", need.unknown ?? ""].join(" ");
+ return [...fields, "|", ...references, "|", pending, "|", need.issue ?? "", "|", need.checked ?? ""].join(" ");
}
function needSummaryText(needs: readonly PlanNeed[]): string {
const first = needs[0]!;
- if (first.summary === undefined) return first.unknown === undefined ? "parameters unknown" : `parameters unknown: ${first.unknown}`;
const parts: string[] = [];
- for (const [name, value] of Object.entries(first.summary.fields)) {
- if (typeof value === "string" && WORDS.test(value)) {
- const counts = needs.map((need) => Number(WORDS.exec(String(need.summary?.fields[name] ?? "0 words"))?.[1] ?? 0));
+ for (const [name, value] of Object.entries(first.summary?.fields ?? {})) {
+ const measured = typeof value === "string" ? MEASURE.exec(value) : null;
+ if (measured !== null) {
+ const unit = measured[2]!;
+ const counts = needs.map((need) => Number(MEASURE.exec(String(need.summary?.fields[name] ?? `0 ${unit}`))?.[1] ?? 0));
const low = Math.min(...counts);
const high = Math.max(...counts);
- parts.push(`${name} ${low === high ? low : `${low}–${high}`} words`);
+ parts.push(`${name} ${low === high ? low : `${low}–${high}`} ${unit}`);
} else if (typeof value === "boolean") {
parts.push(value ? name : `no ${name}`);
} else {
parts.push(`${name} ${value}`);
}
}
- const references = Object.entries(first.summary.references).sort(([left], [right]) => left.localeCompare(right))
+ const references = Object.entries(first.summary?.references ?? {}).sort(([left], [right]) => left.localeCompare(right))
.map(([kind, count]) => `${count} ${kind}`);
if (references.length > 0) parts.push(`${references.join(" + ")} reference${references.length === 1 && references[0]!.startsWith("1 ") ? "" : "s"}`);
- if (first.unknown !== undefined) parts.push(first.unknown);
+ const pendingCount = first.pending.length;
+ if (pendingCount > 0) parts.push(`${pendingCount === 1 ? "file" : `${pendingCount} files`} produced during Build`);
+ if (first.issue !== undefined) parts.push(`could not inspect: ${first.issue}`);
return parts.length === 0 ? "no parameters" : parts.join(" · ");
}
@@ -391,8 +405,26 @@ function groupedNeedLines(needs: readonly PlanNeed[], colors: Palette, verbose:
}
return [...groups.values()].map((group) => {
const who = group.length === 1 || verbose ? colors.dim(group.map((need) => stepLabel(need.step)).join(", ")) : "";
+ const verification = verbose && group.some((need) => need.checked === "capability")
+ ? colors.dim("Endpoint selected by capability; file compatibility is checked before the request is sent")
+ : "";
const count = `×${group.length}`.padStart(4);
- return ` ${colors.dim(count)} ${needSummaryText(group)}${who.length === 0 ? "" : ` ${who}`}`;
+ return ` ${colors.dim(count)} ${needSummaryText(group)}${who.length === 0 ? "" : ` ${who}`}${verification.length === 0 ? "" : ` ${verification}`}`;
+ });
+}
+
+function providerGroupKey(provider: PlanProvider): string {
+ return JSON.stringify({
+ capability: provider.capability,
+ status: provider.status,
+ endpoint: provider.endpoint,
+ use: provider.use,
+ pricing: provider.pricing,
+ endpoints: provider.endpoints,
+ binding: provider.binding,
+ // A resolved request belongs under its Endpoint regardless of whether its file already exists.
+ // Keep the distinction on each request for verbose honesty, not as another visible Provider.
+ checked: provider.status === "resolved" ? undefined : provider.checked,
});
}
@@ -413,31 +445,31 @@ function renderPlan(
lines.push(...facts([
["Run", shortPath(view.machine.run)],
["Targets", targetSummary],
- ["External requests", String(view.machine.externalRequestCount)],
+ ["Requests", String(view.machine.requestCount)],
+ ...(view.machine.requestIssueCount === 0 ? [] : [["Request issues", String(view.machine.requestIssueCount)] as const]),
+ ...(view.machine.providerRequestCount === undefined ? [] : [["Provider requests", String(view.machine.providerRequestCount)] as const]),
+ ...(view.machine.localRequestCount === undefined ? [] : [["Local requests", String(view.machine.localRequestCount)] as const]),
+ ...((view.machine.unresolvedRequestCount ?? 0) === 0 ? [] : [["Unresolved", String(view.machine.unresolvedRequestCount)] as const]),
...(view.machine.preflight === undefined ? [] : [[
"Preflight", view.machine.preflight.ok ? "ready" : "needs attention",
] as const]),
...(verbose ? [["Steps", String(view.machine.steps)] as const] : []),
], colors));
- if (view.machine.externalRequests.length > 0) {
- lines.push("", colors.strong("External requests"));
- const countWidth = Math.max(...view.machine.externalRequests.map((item) => String(item.count).length));
- for (const item of view.machine.externalRequests) {
- lines.push(` ${colors.warning(String(item.count).padStart(countWidth))} ${verbose ? item.operation : operationLabel(item.operation)}`);
- }
- if ((view.machine.omittedExternalRequests ?? 0) > 0) {
- lines.push(` ${colors.dim(`${view.machine.omittedExternalRequests} more Operations · use --limit <count>`)}`);
- }
- }
if (view.machine.providers !== undefined) {
if (view.machine.providers.length > 0) lines.push("", colors.strong("Providers and price pages"));
- for (const item of view.machine.providers) {
+ const groups = new Map<string, PlanProvider[]>();
+ for (const provider of view.machine.providers) {
+ const key = providerGroupKey(provider);
+ groups.set(key, [...(groups.get(key) ?? []), provider]);
+ }
+ for (const group of groups.values()) {
+ const item = group[0]!;
const where = item.status === "resolved"
? `${item.endpoint ?? ""} ${colors.dim(`(${item.use ?? "?"})${item.binding === undefined ? "" : ", bound in the Profile"}`)}`
: item.status === "ambiguous"
? colors.warning(`${(item.endpoints ?? []).join(", ")} all offer it; add "bindings": { "${item.capability}": "<instance>" } to the Profile`)
: item.binding === undefined
- ? colors.error("no selected Endpoint")
+ ? colors.error(item.checked === "request" ? "no selected Endpoint accepts this request" : "no selected Endpoint")
: colors.error(`bound to ${item.binding}, which does not offer it`);
const price = item.pricing === undefined
? (item.status === "resolved" ? colors.warning("price source unknown") : undefined)
@@ -446,13 +478,23 @@ function renderPlan(
: item.pricing.url;
lines.push(` ${colors.accent(verbose ? item.capability : capabilityLabel(item.capability))}`);
lines.push(` ${where}${price === undefined ? "" : ` · ${price}`}`);
- lines.push(...groupedNeedLines((view.machine.needs ?? []).filter((need) => need.capability === item.capability), colors, verbose));
+ const requests = new Set(group.map((provider) => provider.request));
+ lines.push(...groupedNeedLines((view.machine.needs ?? []).filter((need) => requests.has(need.request)), colors, verbose));
}
if ((view.machine.omittedProviders ?? 0) > 0) {
- lines.push(` ${colors.dim(`${view.machine.omittedProviders} more capabilities · use --limit <count>`)}`);
+ lines.push(` ${colors.dim(`${view.machine.omittedProviders} more requests · use --limit <count>`)}`);
+ }
+ } else if (view.machine.requestCount > 0) {
+ const byCapability = new Map<string, PlanNeed[]>();
+ for (const need of view.machine.needs ?? []) {
+ byCapability.set(need.capability, [...(byCapability.get(need.capability) ?? []), need]);
+ }
+ if (byCapability.size > 0) lines.push("", colors.strong("Requests"));
+ for (const [capability, needs] of byCapability) {
+ lines.push(` ${colors.accent(verbose ? capability : capabilityLabel(capability))}`);
+ lines.push(...groupedNeedLines(needs, colors, verbose));
}
- } else if (view.machine.externalRequestCount > 0) {
- lines.push("", colors.dim("Pass --runtime <profile> to see the Provider and price page behind each external request."));
+ lines.push("", colors.dim("Pass --runtime <profile> to see the Endpoint and price page behind each request."));
}
const unreached = view.machine.unreached ?? [];
if (verbose && unreached.length > 0) {
diff --git a/packages/cli/test/build-programs.test.ts b/packages/cli/test/build-programs.test.ts
index 91c1f87c..c59bde18 100644
--- a/packages/cli/test/build-programs.test.ts
+++ b/packages/cli/test/build-programs.test.ts
@@ -217,7 +217,7 @@ test("Build confirms durable submission before following stable work progress",
const active = output.indexOf("Build still active");
assert.ok(submitted >= 0 && working > submitted && active > working);
assert.match(output, /Target\s+result/u);
- assert.match(output, /Work\s+0 external requests/u);
+ assert.match(output, /Work\s+0 requests/u);
assert.match(output, /Ctrl-C stops watching; the Build continues\./u);
});
diff --git a/packages/cli/test/output.test.ts b/packages/cli/test/output.test.ts
index 456b9f05..386fa676 100644
--- a/packages/cli/test/output.test.ts
+++ b/packages/cli/test/output.test.ts
@@ -57,14 +57,14 @@ test("plan presents useful choices and readable requests without default graph i
const presentation = {
kind: "plan",
machine: {
- format: "hypit.cli-plan@2",
+ format: "hypit.cli-plan@3",
ok: true,
run: "/project/build.svrun",
targetCount: 1,
targets: ["take.video"],
steps: 2,
- externalRequestCount: 1,
- externalRequests: [{ operation: "@hypit/media@1/inspect", count: 1 }],
+ requestCount: 1,
+ requestIssueCount: 0,
preflight: {
ok: true,
capabilityCount: 1,
@@ -79,26 +79,26 @@ test("plan presents useful choices and readable requests without default graph i
} as const;
const output = capture(human, presentation);
assert.match(output, /take\.video\s+← preview/u);
- assert.match(output, /1\s+inspect/u);
+ assert.match(output, /Requests\s+1/u);
assert.match(output, /Preflight\s+ready/u);
assert.doesNotMatch(output, /Runtime\s+ready/u);
assert.doesNotMatch(output, /@hypit\/media|unused\.image|Declared but not reached|logical:|record:|step-/u);
const verbose = capture({ ...human, verbose: true }, presentation);
- assert.match(verbose, /1\s+@hypit\/media@1\/inspect/u);
+ assert.match(verbose, /Requests\s+1/u);
assert.match(verbose, /unused\.image/u);
});
test("plan names the Provider and price page behind each request, and points at --runtime when it cannot", () => {
const base = {
- format: "hypit.cli-plan@2",
+ format: "hypit.cli-plan@3",
ok: true,
run: "/project/build.svrun",
targetCount: 1,
targets: ["take.video"],
steps: 3,
- externalRequestCount: 2,
- externalRequests: [{ operation: "@hypit/seedance@1/request-seedance-2-mini", count: 2 }],
+ requestCount: 2,
+ requestIssueCount: 0,
choiceCount: 0,
choices: [],
} as const;
@@ -106,28 +106,39 @@ test("plan names the Provider and price page behind each request, and points at
assert.match(without, /Pass --runtime <profile>/u);
assert.doesNotMatch(without, /Providers and price pages/u);
- const output = capture(human, { kind: "plan", machine: { ...base, providers: [
+ const output = capture(human, { kind: "plan", machine: { ...base, providerRequestCount: 2, localRequestCount: 1, unresolvedRequestCount: 1, providers: [
{
+ request: "seedance:one",
capability: "@hypit/seedance@1#seedance-2-mini",
+ checked: "capability",
status: "resolved",
endpoint: "hypihub.default",
use: "@hypit/provider-hypihub",
pricing: { kind: "page", url: "https://hypit.ai/commercial/pricing/" },
},
- { capability: "@hypit/media@1#inspect", status: "resolved", endpoint: "media.local", use: "@hypit/provider-media-local", pricing: { kind: "local" } },
- { capability: "@hypit/whisperx@1#whisperx-alignment", status: "resolved", endpoint: "whisperx.remote", use: "@hypit/provider-example" },
- { capability: "@hypit/gpt-image@1#gpt-image-2", status: "unresolved" },
- ] } });
+ { request: "media:one", capability: "@hypit/media@1#inspect", checked: "request", status: "resolved", endpoint: "media.local", use: "@hypit/provider-media-local", pricing: { kind: "local" } },
+ { request: "whisper:one", capability: "@hypit/whisperx@1#whisperx-alignment", checked: "request", status: "resolved", endpoint: "whisperx.remote", use: "@hypit/provider-example" },
+ { request: "image:one", capability: "@hypit/gpt-image@1#gpt-image-2", checked: "request", status: "unresolved" },
+ ], needs: [{
+ request: "seedance:one",
+ step: "video::component::presenter.generate",
+ port: "request",
+ capability: "@hypit/seedance@1#seedance-2-mini",
+ summary: { fields: { prompt: "42 chars", duration: 10, resolution: "720p" }, references: { image: 1 } },
+ pending: [{ input: "referenceImages", record: "presenter:image", sourceStep: "presenter.generate", kind: "image" }],
+ checked: "capability",
+ }] } });
assert.match(output, /Providers and price pages/u);
assert.match(output, /@hypit\/seedance#seedance-2-mini\n\s+hypihub\.default \(@hypit\/provider-hypihub\)\s+·\s+https:\/\/hypit\.ai\/commercial\/pricing\//u);
assert.match(output, /media\.local .*·\s+local, no Provider charge/u);
assert.match(output, /whisperx\.remote .*·\s+price source unknown/u);
- assert.match(output, /gpt-image#gpt-image-2\n\s+no selected Endpoint\n/u);
+ assert.match(output, /gpt-image#gpt-image-2\n\s+no selected Endpoint accepts this request\n/u);
+ assert.match(output, /prompt 42 chars · duration 10 · resolution 720p · 1 image reference · file produced during Build/u);
assert.doesNotMatch(output, /Pass --runtime/u);
assert.doesNotMatch(output, /seedance@1#/u);
const verbose = capture({ ...human, verbose: true }, { kind: "plan", machine: { ...base, providers: [
- { capability: "@hypit/seedance@1#seedance-2-mini", status: "ambiguous", endpoints: ["hypihub.default", "kie.default"] },
+ { request: "seedance:one", capability: "@hypit/seedance@1#seedance-2-mini", checked: "capability", status: "ambiguous", endpoints: ["hypihub.default", "kie.default"] },
] } });
assert.match(verbose, /@hypit\/seedance@1#seedance-2-mini\n\s+hypihub\.default, kie\.default all offer it/u);
});
diff --git a/packages/cli/test/plan-needs.test.ts b/packages/cli/test/plan-needs.test.ts
index 3f5f0b81..55d2b79b 100644
--- a/packages/cli/test/plan-needs.test.ts
+++ b/packages/cli/test/plan-needs.test.ts
@@ -3,16 +3,14 @@ import test from "node:test";
import { summarizeConstraints } from "../src/build-planning.js";
-test("a request summary reads scalars off the request itself and counts references by kind", () => {
+test("a generic request summary reads only declared top-level fields and counts references by kind", () => {
const summary = summarizeConstraints({
- ports: {
- prompt: ["A woman writes on a whiteboard in a bright room with a pool behind the glass wall and plants."],
- duration: [10],
- resolution: ["720p"],
- generateAudio: [true],
- referenceImage: [{ artifact: { resource: "sha:1", mediaType: "image/png", size: 10 }, role: "reference" }],
- referenceAudio: [{ artifact: { resource: "sha:2", mediaType: "audio/wav", size: 10 }, role: "reference" }],
- },
+ prompt: "A woman writes on a whiteboard in a bright room with a pool behind the glass wall and plants.",
+ duration: 10,
+ resolution: "720p",
+ generateAudio: true,
+ referenceImage: { resource: "resource:1", mediaType: "image/png", size: 10 },
+ referenceAudio: { resource: "resource:2", mediaType: "audio/wav", size: 10 },
});
assert.deepEqual(summary.fields, { prompt: "19 words", duration: 10, resolution: "720p", generateAudio: true });
assert.deepEqual(summary.references, { image: 1, audio: 1 });
@@ -20,7 +18,7 @@ test("a request summary reads scalars off the request itself and counts referenc
test("a request that is not a generation request is read from its top-level fields", () => {
const summary = summarizeConstraints({
- audio: { resource: "sha:3", mediaType: "audio/wav", size: 320_000 },
+ audio: { resource: "resource:3", mediaType: "audio/wav", size: 320_000 },
sampleFrames: 160_000,
sampleRate: 16_000,
language: "en",
@@ -28,3 +26,8 @@ test("a request that is not a generation request is read from its top-level fiel
assert.deepEqual(summary.fields, { sampleFrames: 160_000, sampleRate: 16_000, language: "en" });
assert.deepEqual(summary.references, { audio: 1 });
});
+
+test("a Chinese request reports characters instead of pretending the whole prompt is one word", () => {
+ const summary = summarizeConstraints({ prompt: "一个女生在大学教室里拿着手麦说话".repeat(3) });
+ assert.deepEqual(summary.fields, { prompt: "48 chars" });
+});
diff --git a/packages/driver-node/README.md b/packages/driver-node/README.md
index 0283e14f..9e41865e 100644
--- a/packages/driver-node/README.md
+++ b/packages/driver-node/README.md
@@ -15,4 +15,4 @@ Immediate endpoints run once. Asynchronous endpoints start a task, store its han
the selected endpoint. Cancellation is best effort and never rolls back completed work.
The Driver validates returned values before offering a command result to Core. It does not load
-packages, parse source files, own the build queue or know any video vocabulary.
+packages, parse source files, own the active Build work set or know any video vocabulary.
diff --git a/packages/driver-node/src/driver.ts b/packages/driver-node/src/driver.ts
index f6d10b6d..c6413abb 100644
--- a/packages/driver-node/src/driver.ts
+++ b/packages/driver-node/src/driver.ts
@@ -270,7 +270,7 @@ export class NodeDriver {
kind: "command-failed",
command: executable.command.id,
code: "CANCELLED",
- message: `Operation ${snapshot.id} was cancelled by the Runtime pool`,
+ message: `Operation ${snapshot.id} was cancelled by the Runtime capacity controller`,
} as const;
return {
status: "completed",
diff --git a/packages/driver-node/src/index.ts b/packages/driver-node/src/index.ts
index 91db7641..45359007 100644
--- a/packages/driver-node/src/index.ts
+++ b/packages/driver-node/src/index.ts
@@ -1,5 +1,7 @@
export { MemoryResourceStore } from "./resources.js";
export { NodeDriver } from "./driver.js";
+export { evaluateProducerPlan } from "./planning.js";
+export type { ProducerPlanEvaluation } from "./planning.js";
export {
ProducerRegistry,
EndpointRegistry,
diff --git a/packages/driver-node/src/planning.ts b/packages/driver-node/src/planning.ts
new file mode 100644
index 00000000..b3d64a7d
--- /dev/null
+++ b/packages/driver-node/src/planning.ts
@@ -0,0 +1,67 @@
+import { BuildMachine } from "@hypit/core";
+import type { BuildDefinition, BuildState } from "@hypit/protocol";
+import { TypeValidatorRegistry } from "@hypit/validation";
+import type { TypeValidatorRegistryLike } from "@hypit/validation";
+
+import { NodeDriver } from "./driver.js";
+import { EndpointRegistry, ProducerRegistry } from "./registry.js";
+
+export type ProducerPlanEvaluation = {
+ /** Disposable Build view after every currently runnable pure Producer was evaluated once. */
+ readonly state: BuildState;
+ /** Exact Producer Step ids that could not be evaluated, with their direct errors. */
+ readonly failures: ReadonlyMap<string, string>;
+};
+
+/**
+ * Evaluate the closed Build graph without installing any Endpoint.
+ *
+ * Component Producers are deterministic computation. External Needs remain blocked, while every
+ * independent Producer keeps advancing. A failed Producer is attempted once: planning reports the
+ * failure instead of looping or turning it into a Build fact.
+ */
+export async function evaluateProducerPlan(
+ definition: BuildDefinition,
+ producers: ProducerRegistry,
+ validators: TypeValidatorRegistryLike = new TypeValidatorRegistry(),
+): Promise<ProducerPlanEvaluation> {
+ const driver = new NodeDriver({ producers, endpoints: new EndpointRegistry(), validators });
+ const machine = new BuildMachine(definition);
+ const attempted = new Set<string>();
+ const failures = new Map<string, string>();
+
+ while (true) {
+ const prepared = driver.prepare(machine.view());
+ const runnable = prepared.runnable.filter((item) =>
+ item.command.kind === "invoke-producer" && !attempted.has(item.command.id));
+ if (runnable.length === 0) break;
+
+ for (const descriptor of runnable) {
+ if (descriptor.command.kind !== "invoke-producer") continue;
+ attempted.add(descriptor.command.id);
+ try {
+ const result = await driver.executeCommand(machine.view(), descriptor, { build: "plan" });
+ if (result.status !== "completed") {
+ failures.set(descriptor.command.step, "the Producer did not complete synchronously");
+ continue;
+ }
+ if (result.event.kind === "command-failed") {
+ failures.set(descriptor.command.step, result.event.message);
+ continue;
+ }
+ machine.evaluate(result.event);
+ machine.commit();
+ } catch (error) {
+ failures.set(descriptor.command.step, error instanceof Error ? error.message : String(error));
+ }
+ }
+ }
+
+ const final = driver.prepare(machine.view());
+ for (const blocked of final.blocked) {
+ if (blocked.reason !== "missing-producer" || !blocked.command.startsWith("producer:")) continue;
+ failures.set(blocked.command.slice("producer:".length), `no Producer for ${blocked.subject} is loaded`);
+ }
+
+ return { state: machine.view(), failures };
+}
diff --git a/packages/driver-node/test/scheduler.test.ts b/packages/driver-node/test/scheduler.test.ts
index fb17225d..83d3addf 100644
--- a/packages/driver-node/test/scheduler.test.ts
+++ b/packages/driver-node/test/scheduler.test.ts
@@ -193,7 +193,7 @@ function asyncExecutor(
scheduling: {
resources: [
{ id: "pool:fixture.account", limit: 1 },
- { id: "lane:fixture.account/generation", limit: 1 },
+ { id: "capacity:fixture.account/generation", limit: 1 },
],
},
},
diff --git a/packages/endpoint-kit/src/index.ts b/packages/endpoint-kit/src/index.ts
index da76c5e7..f20219ec 100644
--- a/packages/endpoint-kit/src/index.ts
+++ b/packages/endpoint-kit/src/index.ts
@@ -147,7 +147,7 @@ type EndpointCapabilityBase = {
readonly returns: TypeRef;
readonly supports?: (need: Need) => boolean;
/** Stable Provider-local capacity class. Defaults to the capability name. */
- readonly lane?: string;
+ readonly capacity?: string;
/** Exact-capability capacity; the Provider pool keeps its independent total capacity. */
readonly maxConcurrency?: number;
};
@@ -206,24 +206,24 @@ export function defineEndpointPackage(options: DefineEndpointPackageOptions): En
assert(options.capabilities.length > 0, "Endpoint package declares no capability");
const keys = options.capabilities.map((item) => refKey(item.capability));
assert(new Set(keys).size === keys.length, "Endpoint package repeats a capability");
- const lanes = options.capabilities.map((item) => item.lane ?? item.capability.name);
- assert(lanes.every((lane) => lane.trim().length > 0), "Endpoint package lane is empty");
+ const capacities = options.capabilities.map((item) => item.capacity ?? item.capability.name);
+ assert(capacities.every((capacity) => capacity.trim().length > 0), "Endpoint package capacity class is empty");
if (options.pricing?.kind === "page") {
let url: URL | undefined;
try { url = new URL(options.pricing.url); } catch { url = undefined; }
assert(url?.protocol === "https:", "Endpoint pricing page must be an HTTPS URL");
}
- const laneConcurrency = new Map<string, number>();
+ const capacityConcurrency = new Map<string, number>();
for (const capability of options.capabilities) {
- const lane = capability.lane ?? capability.capability.name;
+ const capacity = capability.capacity ?? capability.capability.name;
const concurrency = positiveInteger(
capability.maxConcurrency ?? options.defaultConcurrency ?? 1,
`${capability.capability.name} maxConcurrency`,
);
- const previous = laneConcurrency.get(lane);
+ const previous = capacityConcurrency.get(capacity);
assert(previous === undefined || previous === concurrency,
- `Endpoint package lane ${lane} has conflicting concurrency limits`);
- laneConcurrency.set(lane, concurrency);
+ `Endpoint package capacity ${capacity} has conflicting concurrency limits`);
+ capacityConcurrency.set(capacity, concurrency);
}
const credentials = Object.fromEntries(Object.entries(options.credentials ?? {})
.sort(([left], [right]) => left.localeCompare(right))
@@ -271,11 +271,11 @@ export function defineEndpointPackage(options: DefineEndpointPackageOptions): En
...(options.pricing === undefined ? {} : { pricing: structuredClone(options.pricing) }),
install(registry) {
for (const capability of options.capabilities) {
- const lane = capability.lane ?? capability.capability.name;
+ const capacity = capability.capacity ?? capability.capability.name;
const authorityConcurrency = positiveInteger(options.defaultConcurrency ?? 1, "defaultConcurrency");
- const laneConcurrency = positiveInteger(
+ const exactConcurrency = positiveInteger(
capability.maxConcurrency ?? authorityConcurrency,
- `${lane} maxConcurrency`,
+ `${capacity} maxConcurrency`,
);
const common: EndpointRegistrationOptions = {
...(capability.supports === undefined ? {} : { supports: capability.supports }),
@@ -287,8 +287,8 @@ export function defineEndpointPackage(options: DefineEndpointPackageOptions): En
limit: authorityConcurrency,
},
{
- id: `lane:${options.pool}/${lane}`,
- limit: laneConcurrency,
+ id: `capacity:${options.pool}/${capacity}`,
+ limit: exactConcurrency,
},
],
},
diff --git a/packages/endpoint-kit/test/endpoint.test.ts b/packages/endpoint-kit/test/endpoint.test.ts
index 6f1b221d..b0fbb98c 100644
--- a/packages/endpoint-kit/test/endpoint.test.ts
+++ b/packages/endpoint-kit/test/endpoint.test.ts
@@ -47,7 +47,7 @@ test("one Endpoint definition generates one configured instance and host-neutral
lifecycle: "immediate",
capability: capabilities.generation,
returns: types.generated,
- lane: "text-generation",
+ capacity: "text-generation",
maxConcurrency: 1,
handler: () => ({
value: { kind: "inline", value: "generated" },
@@ -72,7 +72,7 @@ test("one Endpoint definition generates one configured instance and host-neutral
assert.deepEqual(registrations[0]?.options.scheduling, {
resources: [
{ id: "pool:example.personal", limit: 3 },
- { id: "lane:example.personal/text-generation", limit: 1 },
+ { id: "capacity:example.personal/text-generation", limit: 1 },
],
});
});
diff --git a/packages/gemini/README.md b/packages/gemini/README.md
index e6c89cd1..ede2464f 100644
--- a/packages/gemini/README.md
+++ b/packages/gemini/README.md
@@ -1,6 +1,6 @@
# `@hypit/gemini`
-Provider-neutral Gemini text and multimodal requests for Hypit Author Source.
+Provider-neutral Gemini multimodal observation requests.
`<gemini:Generate>` accepts a system instruction, a text prompt and any number of image, video or
audio Artifact references. The model package owns the semantic request and exact model capability;
@@ -13,5 +13,10 @@ the selected Runtime Endpoint owns credentials, uploads and the service-specific
</gemini:Generate>
```
-The generated Text is published as `<id>.text`. Switching between
+The answer is a nominal `VisualObservation`, published as `<id>.observation`. It deliberately is not
+`@hypit/text` Prompt input: Gemini supplies visual evidence to the author; it does not silently write
+production prompts inside a Build. Reference-video work should normally call the creation-time
+`hypit observe` command, inspect the observation, and then author the production Source.
+
+Switching between
`@hypit/provider-hypihub` and `@hypit/provider-vertex` requires only a Runtime Profile change.
diff --git a/packages/gemini/src/component.ts b/packages/gemini/src/component.ts
index 72534072..cd93cacc 100644
--- a/packages/gemini/src/component.ts
+++ b/packages/gemini/src/component.ts
@@ -7,6 +7,7 @@ import { verifyText } from "@hypit/text";
import { geminiModels, geminiProducers, geminiTypes } from "./manifest.js";
import { sealGeminiRequest, verifyGeminiRequest } from "./request.js";
import type { GeminiRequest } from "./request.js";
+import { verifyVisualObservation } from "./observation.js";
function inline<T>(value: StoredValue | undefined, subject: string): T {
if (value?.kind !== "inline") throw new Error(`${subject} must be inline`);
@@ -23,6 +24,7 @@ export const geminiComponent = {
validators: [
{ type: geminiTypes.request, handler: ({ value }) => verifyGeminiRequest(inline(value, "Gemini request")) },
{ type: geminiTypes.draft, handler: ({ value }) => verifyGeminiRequest(inline(value, "Gemini request draft")) },
+ { type: geminiTypes.visualObservation, handler: ({ value }) => verifyVisualObservation(inline(value, "Gemini Visual Observation")) },
],
producers: [
{
@@ -64,7 +66,7 @@ export const geminiComponent = {
handler: ({ inputs }: ProducerHandlerContext) => {
const request = inline<GeminiRequest>(inputs.request?.value, `${model} request`);
verifyGeminiRequest(request);
- return { outputs: {}, needs: { text: canonicalize(request) } };
+ return { outputs: {}, needs: { observation: canonicalize(request) } };
},
})),
],
diff --git a/packages/gemini/src/fragment.ts b/packages/gemini/src/fragment.ts
index a442c80b..14ec2929 100644
--- a/packages/gemini/src/fragment.ts
+++ b/packages/gemini/src/fragment.ts
@@ -40,11 +40,11 @@ export function createGeminiFragment(model: GeminiModel, mediaCount: number) {
});
operations.push({
id: "generate", producer: geminiProducers[model], inputs: { request: operation("finalize-request") },
- result: { kind: "need" as const, name: "text" },
+ result: { kind: "need" as const, name: "observation" },
});
return sealGraphFragment({
inputs,
operations,
- exports: [{ name: "text", type: textTypes.text, root: operation("generate") }],
+ exports: [{ name: "observation", type: geminiTypes.visualObservation, root: operation("generate") }],
});
}
diff --git a/packages/gemini/src/index.ts b/packages/gemini/src/index.ts
index cd120cd1..c4206471 100644
--- a/packages/gemini/src/index.ts
+++ b/packages/gemini/src/index.ts
@@ -14,4 +14,6 @@ export type { GeminiModel } from "./manifest.js";
export { sealGeminiRequest, verifyGeminiRequest } from "./request.js";
export type { GeminiMediaPart, GeminiRequest } from "./request.js";
export type { GeminiGenerateInput, GeminiInlinePart } from "./parts.js";
+export { sealVisualObservation, verifyVisualObservation } from "./observation.js";
+export type { VisualObservation } from "./observation.js";
export { decodeGeminiGenerateSurface } from "./surface.js";
diff --git a/packages/gemini/src/manifest.ts b/packages/gemini/src/manifest.ts
index 13a7b7e4..f68b3c41 100644
--- a/packages/gemini/src/manifest.ts
+++ b/packages/gemini/src/manifest.ts
@@ -10,6 +10,7 @@ export type GeminiModel = typeof geminiModels[number];
export const geminiTypes = {
request: { module: geminiModuleRef, name: "GeminiRequest" },
draft: { module: geminiModuleRef, name: "GeminiRequestDraft" },
+ visualObservation: { module: geminiModuleRef, name: "VisualObservation" },
} satisfies Record<string, TypeRef>;
export const geminiCapabilities = Object.fromEntries(geminiModels.map((model) => [model, {
@@ -46,8 +47,8 @@ export const geminiMarkupSurfaces = [{
summary: "Attaches one image, video or audio Artifact to the request in authored order.",
attributes: [{ name: "media", kind: "reference", required: true, accepts: [artifactTypes.blob],
summary: "Selects the media Artifact Gemini reads." }] }],
- ports: [{ name: "text", type: textTypes.text,
- summary: "The generated answer, addressed as `<id>.text`." }],
+ ports: [{ name: "observation", type: geminiTypes.visualObservation,
+ summary: "What Gemini observed, addressed as `<id>.observation`; it is not generation Prompt Text." }],
example: `<gemini:Generate id="describe" model="gemini-3.1-pro" instruction={instruction} prompt={prompt}>
<gemini:Reference media={reference-video}/>
</gemini:Generate>`,
@@ -63,8 +64,12 @@ export const geminiManifest: ModuleManifest = {
name: geminiModuleRef.name,
version: geminiModuleRef.version,
dependencies: [artifactDependency, textDependency],
- types: [{ name: geminiTypes.request.name }, { name: geminiTypes.draft.name }],
- capabilities: geminiModels.map((model) => ({ name: model, returns: textTypes.text })),
+ types: [
+ { name: geminiTypes.request.name },
+ { name: geminiTypes.draft.name },
+ { name: geminiTypes.visualObservation.name },
+ ],
+ capabilities: geminiModels.map((model) => ({ name: model, returns: geminiTypes.visualObservation })),
producers: [
{
name: geminiProducers.start.name,
@@ -88,7 +93,7 @@ export const geminiManifest: ModuleManifest = {
name: geminiProducers[model].name,
inputs: [{ name: "request", type: geminiTypes.request }],
outputs: [],
- needs: [{ name: "text", capability: geminiCapabilities[model], returns: textTypes.text }],
+ needs: [{ name: "observation", capability: geminiCapabilities[model], returns: geminiTypes.visualObservation }],
})),
],
};
diff --git a/packages/gemini/src/observation.ts b/packages/gemini/src/observation.ts
new file mode 100644
index 00000000..1373eebd
--- /dev/null
+++ b/packages/gemini/src/observation.ts
@@ -0,0 +1,18 @@
+import { canonicalize } from "@hypit/protocol";
+
+/** What Gemini saw in the supplied media. It is evidence for the author, not production Text. */
+export type VisualObservation = { readonly text: string };
+
+export function verifyVisualObservation(value: unknown): asserts value is VisualObservation {
+ if (value === null || typeof value !== "object" || Array.isArray(value)
+ || typeof (value as { readonly text?: unknown }).text !== "string"
+ || (value as { readonly text: string }).text.trim().length === 0) {
+ throw new Error("Visual Observation must contain non-empty text");
+ }
+}
+
+export function sealVisualObservation(text: string): VisualObservation {
+ const result = canonicalize({ text }) as unknown as VisualObservation;
+ verifyVisualObservation(result);
+ return result;
+}
diff --git a/packages/gemini/src/surface.ts b/packages/gemini/src/surface.ts
index c61fde38..5bba6d0e 100644
--- a/packages/gemini/src/surface.ts
+++ b/packages/gemini/src/surface.ts
@@ -66,7 +66,7 @@ export const decodeGeminiGenerateSurface: StructuredSurfaceHandler = ({ element,
prompt: prompt.ref,
...Object.fromEntries(media.map((item, index) => [`media-${String(index + 1).padStart(4, "0")}`, item.ref])),
},
- outputs: { text: `${id}.text` },
+ outputs: { observation: `${id}.observation` },
range: element.range,
}],
};
diff --git a/packages/gemini/test/surface.test.ts b/packages/gemini/test/surface.test.ts
index eb766a36..627e1654 100644
--- a/packages/gemini/test/surface.test.ts
+++ b/packages/gemini/test/surface.test.ts
@@ -35,5 +35,5 @@ test("Gemini Surface builds one provider-neutral need with ordered media edges",
assert.equal(fragmentResult.fragments.length, 1);
const operation = fragmentResult.fragments[0]!.operations.find((item) => item.id === "generate");
assert.deepEqual(operation?.producer, { module: geminiCapabilities["gemini-3.1-pro"].module, name: "request-gemini-3.1-pro" });
- assert.deepEqual(fragmentResult.components[0]?.outputs, { text: "read.text" });
+ assert.deepEqual(fragmentResult.components[0]?.outputs, { observation: "read.observation" });
});
diff --git a/packages/generation/src/index.ts b/packages/generation/src/index.ts
index 1c45dcc8..456f3b21 100644
--- a/packages/generation/src/index.ts
+++ b/packages/generation/src/index.ts
@@ -26,6 +26,7 @@ export type {
export {
assertGenerationPortTable,
generationPort,
+ isMediaPort,
sealGenerationPortTable,
} from "./ports.js";
export type {
diff --git a/packages/model-kit/src/index.ts b/packages/model-kit/src/index.ts
index 12b40cf9..64018f9d 100644
--- a/packages/model-kit/src/index.ts
+++ b/packages/model-kit/src/index.ts
@@ -35,12 +35,33 @@ import {
import type {
CanonicalValue,
CapabilityRef,
+ BuildState,
ModuleManifest,
ModuleRef,
ProducerRef,
TypeRef,
} from "@hypit/protocol";
+export type PlannedExactModelMediaReference = {
+ readonly port: string;
+ readonly role: GenerationMediaBinding["role"];
+ readonly fields?: GenerationMediaBinding["fields"];
+ readonly record: string;
+ readonly sourceStep?: string;
+ /** The Blob already exists in this disposable planning view. */
+ readonly available: boolean;
+};
+
+export type PlannedExactModelRequest = {
+ readonly model: string;
+ /** Every authored scalar and Text port value, plus media values whose Blobs already exist. */
+ readonly ports: GenerationRequestDraft["ports"];
+ /** Declared media edges whose files will be produced by an upstream Build step. */
+ readonly pendingMedia: readonly PlannedExactModelMediaReference[];
+ /** True only when the executable Need already exists and has passed full request validation. */
+ readonly complete: boolean;
+};
+
/**
* One exact model endpoint. The model declares which input ports it accepts;
* its request Schema and semantic validator are derived from that declaration,
@@ -213,6 +234,130 @@ function inlineValue<T>(
return canonicalize(value.value) as unknown as T;
}
+function sameReference(
+ left: { readonly module: ModuleRef; readonly name: string },
+ right: { readonly module: ModuleRef; readonly name: string },
+): boolean {
+ return left.module.name === right.module.name
+ && left.module.version === right.module.version
+ && left.name === right.name;
+}
+
+/**
+ * Read one exact-model request from its declared assembly graph.
+ *
+ * This follows only the exact producers published by the model Host facet. It never searches for
+ * object shapes or guesses which upstream record "looks like" a request. Missing media stays a
+ * symbolic graph edge; scalar parameters and authored Text remain available before the file exists.
+ */
+export function plannedExactModelRequest(
+ state: BuildState,
+ stepId: string,
+ needPort: string,
+ endpoint: ExactModelEndpoint,
+): PlannedExactModelRequest | undefined {
+ const generationStep = state.plan.steps.find((step) => step.id === stepId);
+ if (generationStep === undefined || !sameReference(generationStep.producer, endpoint.producer)) return undefined;
+ const binding = generationStep.needs[needPort];
+ if (binding === undefined) return undefined;
+
+ const knownNeed = state.needs.find((need) => need.id === binding.id);
+ if (knownNeed !== undefined) {
+ verifyRequestAgainstPorts(endpoint.ports, knownNeed.constraints);
+ return {
+ model: endpoint.ports.model,
+ ports: structuredClone((knownNeed.constraints as unknown as GenerationRequestDraft).ports),
+ pendingMedia: [],
+ complete: true,
+ };
+ }
+
+ const records = new Map(state.records.map((record) => [record.id, record]));
+ const producedBy = new Map(state.plan.steps.flatMap((step) =>
+ Object.values(step.outputs).map((record) => [record, step] as const)));
+ const requestRecord = generationStep.inputs.request;
+ if (requestRecord === undefined) return undefined;
+ const finalize = producedBy.get(requestRecord);
+ if (finalize === undefined || !sameReference(finalize.producer, endpoint.finalizeProducer)) return undefined;
+ const finalDraft = finalize.inputs.draft;
+ if (finalDraft === undefined) return undefined;
+
+ const pendingMedia: PlannedExactModelMediaReference[] = [];
+ const visiting = new Set<string>();
+ const rebuildDraft = (recordId: string): GenerationRequestDraft => {
+ if (visiting.has(recordId)) throw new Error(`${endpoint.ports.model} request assembly contains a cycle at ${recordId}`);
+ visiting.add(recordId);
+ try {
+ const record = records.get(recordId);
+ if (record !== undefined) {
+ const draft = inlineValue<GenerationRequestDraft>(record.value, `${endpoint.ports.model} request draft`);
+ verifyRequestDraftAgainstPorts(endpoint.ports, draft);
+ return draft;
+ }
+ const step = producedBy.get(recordId);
+ if (step === undefined) throw new Error(`${endpoint.ports.model} request draft ${recordId} has no source`);
+ const previousId = step.inputs.draft;
+ if (previousId === undefined) throw new Error(`${step.id} does not declare its request draft input`);
+ let draft = rebuildDraft(previousId);
+
+ const textBinding = Object.values(endpoint.textBindings)
+ .find((candidate) => sameReference(step.producer, candidate.producer));
+ if (textBinding !== undefined) {
+ const textId = step.inputs.text;
+ const text = textId === undefined ? undefined : records.get(textId);
+ if (text === undefined) throw new Error(`${step.id} has no authored Text input`);
+ return bindGenerationText(
+ endpoint.ports,
+ draft,
+ textBinding.port,
+ inlineValue<Text>(text.value, `${step.id} Text`),
+ );
+ }
+
+ const mediaBinding = Object.values(endpoint.mediaBindings)
+ .find((candidate) => sameReference(step.producer, candidate.producer));
+ if (mediaBinding === undefined) {
+ throw new Error(`${step.id} is not part of ${endpoint.ports.model}'s declared request assembly`);
+ }
+ const bindingId = step.inputs.binding;
+ const artifactId = step.inputs.artifact;
+ const authoredBinding = bindingId === undefined ? undefined : records.get(bindingId);
+ if (authoredBinding === undefined || artifactId === undefined) {
+ throw new Error(`${step.id} is missing its authored media binding`);
+ }
+ const value = inlineValue<GenerationMediaBinding>(authoredBinding.value, `${step.id} media binding`);
+ const port = endpoint.ports.ports.find((candidate): candidate is GenerationMediaPort =>
+ candidate.name === mediaBinding.port && candidate.value.kind === "media");
+ if (port === undefined) throw new Error(`${step.id} names undeclared media port ${mediaBinding.port}`);
+ verifyGenerationMediaBinding(port, value);
+ const artifact = records.get(artifactId);
+ if (artifact?.value.kind === "blob") {
+ draft = bindGenerationMedia(endpoint.ports, draft, mediaBinding.port, value, artifact.value);
+ } else {
+ pendingMedia.push({
+ port: mediaBinding.port,
+ role: value.role,
+ ...(value.fields === undefined ? {} : { fields: structuredClone(value.fields) }),
+ record: artifactId,
+ ...(producedBy.get(artifactId) === undefined ? {} : { sourceStep: producedBy.get(artifactId)!.id }),
+ available: false,
+ });
+ }
+ return draft;
+ } finally {
+ visiting.delete(recordId);
+ }
+ };
+
+ const draft = rebuildDraft(finalDraft);
+ return {
+ model: endpoint.ports.model,
+ ports: structuredClone(draft.ports),
+ pendingMedia,
+ complete: false,
+ };
+}
+
/**
* Builds the repetitive nominal shell around an exact model request. The model package still owns
* every field, constraint and model name; this helper only wires Type -> Producer -> Need -> Fragment.
diff --git a/packages/model-kit/test/model-kit.test.ts b/packages/model-kit/test/model-kit.test.ts
index fa612974..d5b6dfb8 100644
--- a/packages/model-kit/test/model-kit.test.ts
+++ b/packages/model-kit/test/model-kit.test.ts
@@ -10,7 +10,9 @@ import {
import {
createExactModelPrimaryGenerationFragment,
defineExactModelModule,
+ plannedExactModelRequest,
} from "@hypit/model-kit";
+import type { BuildState } from "@hypit/protocol";
import { sealText } from "@hypit/text";
const ports = sealGenerationPortTable({
@@ -96,3 +98,50 @@ test("the dynamic Fragment exposes every Text and media edge as an explicit sema
"select-primary-image",
]);
});
+
+test("planning follows the model's declared assembly edges and leaves an upstream file symbolic", () => {
+ const endpoint = definition.endpoints.image!;
+ const mediaPort = ports.ports.find((port) => port.name === "images");
+ assert.ok(mediaPort?.value.kind === "media");
+ const state = {
+ records: [
+ { id: "draft:initial", type: endpoint.draftType, value: { kind: "inline", value: sealGenerationRequestDraft(ports, {}) } },
+ { id: "prompt", type: { module: { name: "@hypit/text", version: "1" }, name: "Text" }, value: { kind: "inline", value: sealText("draw the authored scene") } },
+ { id: "binding", type: endpoint.mediaBindings.images!.type, value: { kind: "inline", value: sealGenerationMediaBinding(mediaPort as never, { role: "image" }) } },
+ // Deliberately request-shaped, but not connected to the declared assembly chain.
+ { id: "decoy", type: endpoint.requestType, value: { kind: "inline", value: { ports: { prompt: ["wrong"] } } } },
+ ],
+ needs: [],
+ plan: { steps: [
+ { id: "bind-text", producer: endpoint.textBindings.prompt!.producer, inputs: { draft: "draft:initial", text: "prompt" }, outputs: { draft: "draft:text" }, needs: {} },
+ { id: "bind-image-one", producer: endpoint.mediaBindings.images!.producer, inputs: { draft: "draft:text", binding: "binding", artifact: "image:upstream-one" }, outputs: { draft: "draft:image-one" }, needs: {} },
+ { id: "bind-image-two", producer: endpoint.mediaBindings.images!.producer, inputs: { draft: "draft:image-one", binding: "binding", artifact: "image:upstream-two" }, outputs: { draft: "draft:image-two" }, needs: {} },
+ { id: "finalize", producer: endpoint.finalizeProducer, inputs: { draft: "draft:image-two" }, outputs: { request: "request" }, needs: {} },
+ { id: "generate", producer: endpoint.producer, inputs: { request: "request" }, outputs: {}, needs: { image: { id: "need:image" } } },
+ { id: "make-image-one", producer: { module: { name: "@test/upstream", version: "1" }, name: "make" }, inputs: {}, outputs: { image: "image:upstream-one" }, needs: {} },
+ { id: "make-image-two", producer: { module: { name: "@test/upstream", version: "1" }, name: "make" }, inputs: {}, outputs: { image: "image:upstream-two" }, needs: {} },
+ ] },
+ } as unknown as BuildState;
+
+ assert.deepEqual(plannedExactModelRequest(state, "generate", "image", endpoint), {
+ model: "graph-native-image",
+ ports: { prompt: ["draw the authored scene"] },
+ pendingMedia: [
+ {
+ port: "images",
+ role: "image",
+ record: "image:upstream-one",
+ sourceStep: "make-image-one",
+ available: false,
+ },
+ {
+ port: "images",
+ role: "image",
+ record: "image:upstream-two",
+ sourceStep: "make-image-two",
+ available: false,
+ },
+ ],
+ complete: false,
+ });
+});
diff --git a/packages/provider-hypihub/src/provider.ts b/packages/provider-hypihub/src/provider.ts
index 989e5ccf..02bcfa39 100644
--- a/packages/provider-hypihub/src/provider.ts
+++ b/packages/provider-hypihub/src/provider.ts
@@ -1,14 +1,12 @@
import type { AsyncEndpoint, EndpointFulfillment, EndpointInvocationContext, EndpointPollContext, EndpointStartContext, EndpointOutcome, ImmediateEndpointHandler } from "@hypit/endpoint-kit";
import { defineEndpointPackage, wakeAfter } from "@hypit/endpoint-kit";
-import { geminiCapabilities, geminiModels, verifyGeminiRequest } from "@hypit/gemini";
+import { geminiCapabilities, geminiModels, geminiTypes, sealVisualObservation, verifyGeminiRequest } from "@hypit/gemini";
import type { GeminiRequest } from "@hypit/gemini";
import { canonicalize } from "@hypit/protocol";
import type { BlobRef, CapabilityRef } from "@hypit/protocol";
import { credentialRef } from "@hypit/runtime";
import type { CredentialRef, ResourceStore } from "@hypit/runtime";
import { sealAlignedTranscriptEvidence, speechEvidenceTypes } from "@hypit/speech-evidence";
-import { sealText } from "@hypit/text";
-import { textTypes } from "@hypit/text";
import {
assertWhisperXEvidenceWav,
interpretWhisperXTranscript,
@@ -322,7 +320,7 @@ export function createHypiHubProvider(options: CreateHypiHubProviderOptions = {}
parts.push({ inlineData: { mimeType: item.artifact.mediaType, data: Buffer.from(bytes).toString("base64") } });
}
const value = await generate({ parts, instruction: request.instruction });
- return { value: { kind: "inline", value: canonicalize(sealText(value)) } };
+ return { value: { kind: "inline", value: canonicalize(sealVisualObservation(value)) } };
};
const transcriptionModel = options.transcriptionModel?.trim() || "victor-upmeet/whisperx";
const whisperXEndpoint: ImmediateEndpointHandler = async (context) => {
@@ -370,21 +368,21 @@ export function createHypiHubProvider(options: CreateHypiHubProviderOptions = {}
capabilities: [
...hypiHubRoutes
.map((route) => route.media === "audio"
- ? { capability: route.capability, returns: route.returns, lifecycle: "immediate" as const, handler: audioEndpoint, lane: route.capability.name }
- : { capability: route.capability, returns: route.returns, lifecycle: "asynchronous" as const, endpoint: asyncEndpoint, lane: route.capability.name }),
+ ? { capability: route.capability, returns: route.returns, lifecycle: "immediate" as const, handler: audioEndpoint, capacity: route.capability.name }
+ : { capability: route.capability, returns: route.returns, lifecycle: "asynchronous" as const, endpoint: asyncEndpoint, capacity: route.capability.name }),
...geminiModels.map((model) => ({
capability: geminiCapabilities[model],
- returns: textTypes.text,
+ returns: geminiTypes.visualObservation,
lifecycle: "immediate" as const,
handler: geminiEndpoint,
- lane: "gemini",
+ capacity: "gemini",
})),
{
capability: whisperXCapabilities.alignment,
returns: speechEvidenceTypes.alignedTranscript,
lifecycle: "immediate" as const,
handler: whisperXEndpoint,
- lane: "transcription",
+ capacity: "transcription",
},
],
});
diff --git a/packages/provider-hypihub/test/provider.test.ts b/packages/provider-hypihub/test/provider.test.ts
index 8b315536..ad71d5c5 100644
--- a/packages/provider-hypihub/test/provider.test.ts
+++ b/packages/provider-hypihub/test/provider.test.ts
@@ -4,10 +4,9 @@ import test from "node:test";
import { EndpointRegistry, MemoryResourceStore } from "@hypit/driver-node";
import { defineEndpointPackage } from "@hypit/endpoint-kit";
import type { AsyncEndpoint } from "@hypit/endpoint-kit";
-import { geminiCapabilities, sealGeminiRequest } from "@hypit/gemini";
+import { geminiCapabilities, geminiTypes, sealGeminiRequest } from "@hypit/gemini";
import type { CanonicalValue, Need } from "@hypit/protocol";
import { mimoTtsEndpoints } from "@hypit/mimo-tts";
-import { textTypes } from "@hypit/text";
import { sealSeedanceRequest, seedanceEndpoints } from "@hypit/seedance";
import { sealSpeechEvidenceAudio } from "@hypit/speech";
import { speechEvidenceTypes } from "@hypit/speech-evidence";
@@ -183,7 +182,7 @@ test("HypiHub fulfills Gemini through the Runtime endpoint and uploads every med
const request: Need = {
id: "need:hypihub-gemini",
capability: geminiCapabilities["gemini-3.1-pro"],
- returns: textTypes.text,
+ returns: geminiTypes.visualObservation,
constraints: sealGeminiRequest({
instruction: "Answer briefly.", prompt: "Inspect both references.",
media: [{ artifact: image }, { artifact: video }],
@@ -215,7 +214,7 @@ test("HypiHub fulfills Gemini through the Runtime endpoint and uploads every med
command: { kind: "fulfill-need", id: "command:hypihub-gemini", need: request },
need: request, resources, credentials: { apiKey: { secret: "test-key" } },
});
- assert.deepEqual(result.value, { kind: "inline", value: { value: "provider works" } });
+ assert.deepEqual(result.value, { kind: "inline", value: { text: "provider works" } });
assert.deepEqual(uploads, ["image/png", "video/mp4"]);
assert.deepEqual((generationBody?.contents as readonly unknown[]), [{ role: "user", parts: [
{ text: "Inspect both references." },
diff --git a/packages/provider-kie/README.md b/packages/provider-kie/README.md
index abd97b69..9dd422ac 100644
--- a/packages/provider-kie/README.md
+++ b/packages/provider-kie/README.md
@@ -47,7 +47,7 @@ Declarative activation names an ordinary CredentialRef, not an environment-speci
"config": {
"apiKey": { "store": "os", "key": "kie.api-key" },
"defaultConcurrency": 8,
- "laneConcurrency": {
+ "capabilityConcurrency": {
"seedance-2.5": 4,
"gpt-image-2": 3
}
@@ -67,7 +67,7 @@ const kie = createKieProvider({
instance: "kie.personal",
apiKey: credentialRef("env", "KIE_API_KEY"),
defaultConcurrency: 8,
- laneConcurrency: {
+ capabilityConcurrency: {
"seedance-2.5": 4,
"gpt-image-2": 3,
},
@@ -75,7 +75,7 @@ const kie = createKieProvider({
```
`defaultConcurrency` is the total capacity of this configured KIE Endpoint across all Builds. Each
-optional `laneConcurrency` entry narrows the capacity of one exact KIE model inside that total. By
+optional `capabilityConcurrency` entry narrows the capacity of one exact KIE model inside that total. By
default the Endpoint instance is the shared-resource identity. Set Profile `pool` only when multiple
Endpoint instances really use the same account or deployment quota. Another Provider is independent
unless the Profile explicitly gives it that same real-resource identity.
diff --git a/packages/provider-kie/src/activation.ts b/packages/provider-kie/src/activation.ts
index acada764..dfd3fd9d 100644
--- a/packages/provider-kie/src/activation.ts
+++ b/packages/provider-kie/src/activation.ts
@@ -16,7 +16,7 @@ const kieRuntimeAdapter = createRuntimeEndpointAdapterFacet({
const config = runtimeConfigObject(context.config, "KIE");
runtimeConfigExact(config, [
"apiBaseUrl", "uploadBaseUrl", "apiKey", "defaultConcurrency", "pollIntervalMs",
- "laneConcurrency", "submissionIntervalMs", "requestTimeoutMs", "maxOperationMs", "maxArtifactBytes",
+ "capabilityConcurrency", "submissionIntervalMs", "requestTimeoutMs", "maxOperationMs", "maxArtifactBytes",
], "KIE");
const apiBaseUrl = runtimeConfigString(config.apiBaseUrl, "KIE apiBaseUrl");
const uploadBaseUrl = runtimeConfigString(config.uploadBaseUrl, "KIE uploadBaseUrl");
@@ -30,13 +30,13 @@ const kieRuntimeAdapter = createRuntimeEndpointAdapterFacet({
const apiKey = runtimeConfigCredentialRef(config.apiKey, "KIE apiKey");
if (apiKey === undefined) throw new Error("KIE apiKey CredentialRef is required");
const defaultConcurrency = runtimeConfigPositiveInteger(config.defaultConcurrency, "KIE defaultConcurrency");
- const laneConcurrency = config.laneConcurrency === undefined
+ const capabilityConcurrency = config.capabilityConcurrency === undefined
? undefined
- : Object.fromEntries(Object.entries(runtimeConfigObject(config.laneConcurrency, "KIE laneConcurrency"))
- .map(([lane, value]) => {
- const concurrency = runtimeConfigPositiveInteger(value, `KIE laneConcurrency.${lane}`);
- if (concurrency === undefined) throw new Error(`KIE laneConcurrency.${lane} is required`);
- return [lane, concurrency];
+ : Object.fromEntries(Object.entries(runtimeConfigObject(config.capabilityConcurrency, "KIE capabilityConcurrency"))
+ .map(([capability, value]) => {
+ const concurrency = runtimeConfigPositiveInteger(value, `KIE capabilityConcurrency.${capability}`);
+ if (concurrency === undefined) throw new Error(`KIE capabilityConcurrency.${capability} is required`);
+ return [capability, concurrency];
}));
const pollIntervalMs = runtimeConfigPositiveInteger(config.pollIntervalMs, "KIE pollIntervalMs");
const submissionIntervalMs = runtimeConfigPositiveInteger(config.submissionIntervalMs, "KIE submissionIntervalMs");
@@ -51,7 +51,7 @@ const kieRuntimeAdapter = createRuntimeEndpointAdapterFacet({
...(uploadBaseUrl === undefined ? {} : { uploadBaseUrl }),
apiKey,
...(defaultConcurrency === undefined ? {} : { defaultConcurrency }),
- ...(laneConcurrency === undefined ? {} : { laneConcurrency }),
+ ...(capabilityConcurrency === undefined ? {} : { capabilityConcurrency }),
...(pollIntervalMs === undefined ? {} : { pollIntervalMs }),
...(submissionIntervalMs === undefined ? {} : { submissionIntervalMs }),
...(requestTimeoutMs === undefined ? {} : { requestTimeoutMs }),
diff --git a/packages/provider-kie/src/provider.ts b/packages/provider-kie/src/provider.ts
index 6bdb743d..7afc7dd7 100644
--- a/packages/provider-kie/src/provider.ts
+++ b/packages/provider-kie/src/provider.ts
@@ -39,7 +39,7 @@ export type CreateKieProviderOptions = {
/** Total in-flight capacity shared by every KIE lane. */
readonly defaultConcurrency?: number;
/** Optional KIE lane limits keyed by capability name, for example seedance-2.5. */
- readonly laneConcurrency?: Readonly<Record<string, number>>;
+ readonly capabilityConcurrency?: Readonly<Record<string, number>>;
readonly pollIntervalMs?: number;
readonly submissionIntervalMs?: number;
readonly requestTimeoutMs?: number;
@@ -535,10 +535,10 @@ function endpoint(options: {
export function createKieProvider(config: CreateKieProviderOptions) {
verifyKieRoutes();
- const laneNames = new Set(kieRoutes.map((route) => route.capability.name));
- const laneConcurrency = Object.fromEntries(Object.entries(config.laneConcurrency ?? {}).map(([lane, limit]) => {
- if (!laneNames.has(lane)) throw new Error(`unknown KIE concurrency lane ${lane}`);
- return [lane, positiveInteger(limit, `${lane} laneConcurrency`)];
+ const capacityNames = new Set(kieRoutes.map((route) => route.capability.name));
+ const capabilityConcurrency = Object.fromEntries(Object.entries(config.capabilityConcurrency ?? {}).map(([capability, limit]) => {
+ if (!capacityNames.has(capability)) throw new Error(`unknown KIE capacity ${capability}`);
+ return [capability, positiveInteger(limit, `${capability} capabilityConcurrency`)];
}));
const apiBaseUrl = baseUrl(config.apiBaseUrl ?? "https://api.kie.ai", "apiBaseUrl");
const uploadBaseUrl = baseUrl(config.uploadBaseUrl ?? "https://kieai.redpandaai.co", "uploadBaseUrl");
@@ -574,10 +574,10 @@ export function createKieProvider(config: CreateKieProviderOptions) {
capabilities: kieRoutes.map((route) => ({
capability: route.capability,
returns: route.returns,
- lane: route.capability.name,
- ...(laneConcurrency[route.capability.name] === undefined
+ capacity: route.capability.name,
+ ...(capabilityConcurrency[route.capability.name] === undefined
? {}
- : { maxConcurrency: laneConcurrency[route.capability.name] }),
+ : { maxConcurrency: capabilityConcurrency[route.capability.name] }),
lifecycle: "asynchronous" as const,
endpoint: providerEndpoint,
...(route.supports === undefined ? {} : { supports: route.supports }),
diff --git a/packages/provider-kie/test/provider.test.ts b/packages/provider-kie/test/provider.test.ts
index 7afacff4..0ef26b9b 100644
--- a/packages/provider-kie/test/provider.test.ts
+++ b/packages/provider-kie/test/provider.test.ts
@@ -60,7 +60,7 @@ test("all KIE capabilities share one asynchronous task engine and use exact-mode
const provider = createKieProvider({
fetch: async () => { throw new Error("no request expected"); },
defaultConcurrency: 8,
- laneConcurrency: { "seedance-2-mini": 4 },
+ capabilityConcurrency: { "seedance-2-mini": 4 },
});
const registry = new EndpointRegistry();
await provider.install(registry);
@@ -80,7 +80,7 @@ test("all KIE capabilities share one asynchronous task engine and use exact-mode
assert.deepEqual(seedResolution.registration.scheduling, {
resources: [
{ id: "pool:kie.default", limit: 8 },
- { id: "lane:kie.default/seedance-2-mini", limit: 4 },
+ { id: "capacity:kie.default/seedance-2-mini", limit: 4 },
],
});
});
diff --git a/packages/provider-vertex/src/provider.ts b/packages/provider-vertex/src/provider.ts
index 7c4b6d44..c577e626 100644
--- a/packages/provider-vertex/src/provider.ts
+++ b/packages/provider-vertex/src/provider.ts
@@ -1,12 +1,11 @@
import { defineEndpointPackage } from "@hypit/endpoint-kit";
import type { EndpointInvocationContext, ImmediateEndpointHandler } from "@hypit/endpoint-kit";
-import { geminiCapabilities, geminiModels, verifyGeminiRequest } from "@hypit/gemini";
+import { geminiCapabilities, geminiModels, geminiTypes, sealVisualObservation, verifyGeminiRequest } from "@hypit/gemini";
import type { CapabilityRef } from "@hypit/protocol";
import type { RuntimeDoctorDiagnostic } from "@hypit/runtime-kit";
import type { GeminiRequest } from "@hypit/gemini";
import { canonicalize } from "@hypit/protocol";
import type { CredentialRef } from "@hypit/runtime";
-import { sealText, textTypes } from "@hypit/text";
import { createVertexGeminiGenerator, probeVertexModel, vertexModelId } from "./gemini.js";
@@ -89,7 +88,7 @@ export function createVertexProvider(options: CreateVertexProviderOptions) {
parts.push({ inlineData: { mimeType: item.artifact.mediaType, data: Buffer.from(bytes).toString("base64") } });
}
const value = await generate({ parts, instruction: typed.instruction });
- return { value: { kind: "inline", value: canonicalize(sealText(value)) } };
+ return { value: { kind: "inline", value: canonicalize(sealVisualObservation(value)) } };
};
return defineEndpointPackage({
module: vertexProviderModuleRef,
@@ -108,10 +107,10 @@ export function createVertexProvider(options: CreateVertexProviderOptions) {
defaultConcurrency: options.defaultConcurrency ?? 4,
capabilities: geminiModels.map((model) => ({
capability: geminiCapabilities[model],
- returns: textTypes.text,
+ returns: geminiTypes.visualObservation,
lifecycle: "immediate" as const,
handler,
- lane: model,
+ capacity: model,
})),
});
}
diff --git a/packages/provider-vertex/test/provider.test.ts b/packages/provider-vertex/test/provider.test.ts
index c4ee4366..05727028 100644
--- a/packages/provider-vertex/test/provider.test.ts
+++ b/packages/provider-vertex/test/provider.test.ts
@@ -2,9 +2,8 @@ import assert from "node:assert/strict";
import test from "node:test";
import { EndpointRegistry } from "@hypit/driver-node";
-import { geminiCapabilities } from "@hypit/gemini";
+import { geminiCapabilities, geminiTypes } from "@hypit/gemini";
import { credentialRef } from "@hypit/runtime";
-import { textTypes } from "@hypit/text";
import { createVertexProvider } from "../src/provider.js";
import { vertexModelId } from "../src/gemini.js";
@@ -20,7 +19,7 @@ test("Vertex Runtime provider exposes the same provider-neutral Gemini capabilit
const resolution = registry.resolve({
id: `need:${model}`,
capability: geminiCapabilities[model],
- returns: textTypes.text,
+ returns: geminiTypes.visualObservation,
constraints: { instruction: "x", prompt: "x", media: [] },
result: `record:${model}`,
});
diff --git a/packages/runtime-host-node/src/index.ts b/packages/runtime-host-node/src/index.ts
index 302db859..84a2f398 100644
--- a/packages/runtime-host-node/src/index.ts
+++ b/packages/runtime-host-node/src/index.ts
@@ -130,6 +130,8 @@ export type RuntimeHostDoctorResult = {
/** The selected Endpoint behind one demanded capability, read from the Profile alone. */
export type RuntimeHostCapabilityProvider = {
+ /** Caller-owned identity for this planned request. */
+ readonly request: string;
readonly capability: CapabilityRef;
readonly status: "resolved" | "unresolved" | "ambiguous";
/** Configured Endpoint instance that would serve the capability, when exactly one does. */
@@ -142,6 +144,16 @@ export type RuntimeHostCapabilityProvider = {
readonly endpoints?: readonly string[];
/** The Endpoint instance the Profile's `bindings` name for this capability, when it names one. */
readonly binding?: string;
+ /** `request` means Endpoint `supports` checked the complete Need; `capability` is static selection only. */
+ readonly checked: "request" | "capability";
+};
+
+export type RuntimeHostProviderQuery = {
+ readonly request: string;
+ readonly capability: CapabilityRef;
+ readonly returns: import("@hypit/protocol").TypeRef;
+ /** Omitted only while an upstream Resource in the closed Build graph does not exist yet. */
+ readonly constraints?: import("@hypit/protocol").CanonicalValue;
};
export type ManagedProgramProgress = {
@@ -230,7 +242,7 @@ export type NodeRuntimeHost = {
* Which selected Endpoint would serve each capability and where its Provider publishes prices.
* Reads the Profile and Endpoint declarations only; never resolves a credential or contacts a service.
*/
- providers(capabilities: readonly CapabilityRef[]): Promise<readonly RuntimeHostCapabilityProvider[]>;
+ providers(requests: readonly RuntimeHostProviderQuery[]): Promise<readonly RuntimeHostCapabilityProvider[]>;
/**
* Execute one immediate Need through the selected Endpoint and its credentials, outside any Build.
* The creation-time boundary for observation, transcription and other quick capabilities; it
diff --git a/packages/runtime-local/src/config.ts b/packages/runtime-local/src/config.ts
index 4f0249bc..a4d8a8af 100644
--- a/packages/runtime-local/src/config.ts
+++ b/packages/runtime-local/src/config.ts
@@ -46,7 +46,7 @@ import {
hypitHostStateRoot,
prepareHostPackages,
} from "@hypit/runtime-host-node";
-import type { HostPackageProgress, HostPackageReport, RuntimeHostCapabilityProvider } from "@hypit/runtime-host-node";
+import type { HostPackageProgress, HostPackageReport, RuntimeHostCapabilityProvider, RuntimeHostProviderQuery } from "@hypit/runtime-host-node";
import { SqliteRuntimeState } from "@hypit/store-sqlite";
import { applyEndpointBindings, createLocalRuntime, parseCapabilityKey } from "./runtime.js";
@@ -487,21 +487,29 @@ async function installedEndpointRegistry(
/** Static Endpoint selection per capability. Reads the Profile and activations only; never a credential or a service. */
export async function describeRuntimeConfigProviders(
path: string,
- capabilities: readonly CapabilityRef[],
+ requests: readonly RuntimeHostProviderQuery[],
options: LoadRuntimeConfigOptions = {},
): Promise<readonly RuntimeHostCapabilityProvider[]> {
- if (capabilities.length === 0) return [];
+ if (requests.length === 0) return [];
const { document, root, packageRoot } = await openRuntimeConfig(path, options.packageRoot);
const hostStateRoot = resolve(options.hostStateRoot ?? hypitHostStateRoot());
const registry = options.registry ?? new RuntimeAdapterRegistry();
await installRuntimeAdapters(registry, packageRoot, endpointPackageSelection(document), options.distributionPackageRoot);
const activations = await activatedEndpoints(document, root, hostStateRoot, registry);
const endpoints = await installedEndpointRegistry(document, activations);
- return capabilities.map((capability): RuntimeHostCapabilityProvider => {
- const base = { capability: structuredClone(capability) };
- const binding = document.bindings[capabilityKey(capability)];
+ return requests.map((request): RuntimeHostCapabilityProvider => {
+ const base = {
+ request: request.request,
+ capability: structuredClone(request.capability),
+ checked: request.constraints === undefined ? "capability" as const : "request" as const,
+ };
+ const binding = document.bindings[capabilityKey(request.capability)];
const bound = binding === undefined ? {} : { binding };
- const resolution = endpoints.resolve({ capability });
+ const resolution = endpoints.resolve({
+ capability: request.capability,
+ returns: request.returns,
+ ...(request.constraints === undefined ? {} : { constraints: request.constraints }),
+ });
if (resolution.status === "missing") return { ...base, status: "unresolved", ...bound };
if (resolution.status === "ambiguous") return { ...base, status: "ambiguous", endpoints: resolution.endpointIds, ...bound };
const match = activations.find(({ activation }) => activation.endpoint.instance.id === resolution.registration.id);
diff --git a/packages/runtime-local/src/result-writer.ts b/packages/runtime-local/src/result-writer.ts
index 0ecae893..30c8d52b 100644
--- a/packages/runtime-local/src/result-writer.ts
+++ b/packages/runtime-local/src/result-writer.ts
@@ -41,7 +41,7 @@ function operationEvent(operation: OperationSnapshot): CommandResult | undefined
kind: "command-failed",
command: operation.command,
code: "CANCELLED",
- message: `Operation ${operation.id} was cancelled by the Runtime pool`,
+ message: `Operation ${operation.id} was cancelled by the Runtime capacity controller`,
};
}
diff --git a/packages/runtime-local/test/config.test.ts b/packages/runtime-local/test/config.test.ts
index 5d839e39..126b278e 100644
--- a/packages/runtime-local/test/config.test.ts
+++ b/packages/runtime-local/test/config.test.ts
@@ -17,6 +17,8 @@ import { FileBuildResultRepository } from "@hypit/build-result";
import { MemoryResourceStore } from "@hypit/driver-node";
import { defineEndpointPackage } from "@hypit/endpoint-kit";
import { credentialRef } from "@hypit/runtime";
+import type { CanonicalValue, CapabilityRef, TypeRef } from "@hypit/protocol";
+import type { RuntimeHostProviderQuery } from "@hypit/runtime-host-node";
import { SqliteRuntimeState } from "@hypit/store-sqlite";
import {
createRuntimeControlFromConfig,
@@ -45,6 +47,15 @@ function profile(config: {
};
}
+function providerQuery(
+ request: string,
+ capability: CapabilityRef,
+ returns: TypeRef,
+ constraints?: CanonicalValue,
+): RuntimeHostProviderQuery {
+ return { request, capability, returns, ...(constraints === undefined ? {} : { constraints }) };
+}
+
test("Local Runtime Profile names credentials and Endpoints, not project result storage", () => {
const parsed = parseLocalRuntimeProfile(profile({
credentials: { secrets: { use: "example.credentials" } },
@@ -304,16 +315,64 @@ test("Runtime providers name the selected Endpoint and its declared price source
}),
}));
try {
- assert.deepEqual(await describeRuntimeConfigProviders(path, [generate, render, missing], { registry }), [
+ assert.deepEqual(await describeRuntimeConfigProviders(path, [
+ providerQuery("generate", generate, returns),
+ providerQuery("render", render, returns),
+ providerQuery("missing", missing, returns),
+ ], { registry }), [
{
+ request: "generate",
capability: generate,
+ checked: "capability",
status: "resolved",
endpoint: "paid",
use: "example.paid",
pricing: { kind: "page", url: "https://prices.example/models" },
},
- { capability: render, status: "resolved", endpoint: "local", use: "example.local", pricing: { kind: "local" } },
- { capability: missing, status: "unresolved" },
+ { request: "render", capability: render, checked: "capability", status: "resolved", endpoint: "local", use: "example.local", pricing: { kind: "local" } },
+ { request: "missing", capability: missing, checked: "capability", status: "unresolved" },
+ ]);
+ } finally {
+ await rm(root, { recursive: true, force: true });
+ }
+});
+
+test("Runtime provider inspection applies Endpoint supports when the complete request is available", async () => {
+ const root = await mkdtemp(join(tmpdir(), "hypit-runtime-provider-supports-"));
+ const path = join(root, "hypit.runtime.json");
+ await writeFile(path, JSON.stringify(profile({
+ dataRoot: ".",
+ endpoints: { narrow: { use: "example.narrow" } },
+ })));
+ const capability = { module: { name: "example.model", version: "1" }, name: "generate" } as const;
+ const returns = { module: { name: "example.value", version: "1" }, name: "Output" } as const;
+ const registry = new RuntimeAdapterRegistry();
+ registry.registerFacet(createRuntimeEndpointAdapterFacet({
+ use: "example.narrow",
+ activate: (context) => ({
+ endpoint: defineEndpointPackage({
+ module: { name: "example.narrow-provider", version: "1" },
+ facet: "narrow",
+ instance: context.instance,
+ pool: context.pool ?? context.instance,
+ pricing: { kind: "local" },
+ capabilities: [{
+ capability,
+ returns,
+ lifecycle: "immediate",
+ supports: (need) => (need.constraints as { readonly allowed?: unknown } | null)?.allowed === true,
+ handler: () => ({ value: { kind: "inline", value: null } }),
+ }],
+ }),
+ }),
+ }));
+ try {
+ assert.deepEqual(await describeRuntimeConfigProviders(path, [
+ providerQuery("pending-file", capability, returns),
+ providerQuery("complete", capability, returns, { allowed: false }),
+ ], { registry }), [
+ { request: "pending-file", capability, checked: "capability", status: "resolved", endpoint: "narrow", use: "example.narrow", pricing: { kind: "local" } },
+ { request: "complete", capability, checked: "request", status: "unresolved" },
]);
} finally {
await rm(root, { recursive: true, force: true });
@@ -452,9 +511,12 @@ test("providers, doctor and invoke share one resolver: a contested capability is
try {
const unbound = join(root, "unbound.json");
await writeFile(unbound, JSON.stringify(profile({ dataRoot: ".", endpoints })));
- assert.deepEqual(await describeRuntimeConfigProviders(unbound, [generate, render], { registry }), [
- { capability: generate, status: "ambiguous", endpoints: ["other", "paid"] },
- { capability: render, status: "resolved", endpoint: "local", use: "example.local", pricing: { kind: "local" } },
+ assert.deepEqual(await describeRuntimeConfigProviders(unbound, [
+ providerQuery("generate", generate, returns, null),
+ providerQuery("render", render, returns, null),
+ ], { registry }), [
+ { request: "generate", capability: generate, checked: "request", status: "ambiguous", endpoints: ["other", "paid"] },
+ { request: "render", capability: render, checked: "request", status: "resolved", endpoint: "local", use: "example.local", pricing: { kind: "local" } },
]);
const contested = (await preflightRuntimeConfig(unbound, { registry, capabilities: [generate] })).diagnostics
.filter((item) => item.code === "RUNTIME_CAPABILITY_AMBIGUOUS");
@@ -469,8 +531,8 @@ test("providers, doctor and invoke share one resolver: a contested capability is
const bound = join(root, "bound.json");
await writeFile(bound, JSON.stringify(profile({ dataRoot: ".", endpoints, bindings: { [generateKey]: "other" } })));
- assert.deepEqual(await describeRuntimeConfigProviders(bound, [generate], { registry }), [
- { capability: generate, status: "resolved", endpoint: "other", use: "example.other", pricing: { kind: "local" }, binding: "other" },
+ assert.deepEqual(await describeRuntimeConfigProviders(bound, [providerQuery("generate", generate, returns, null)], { registry }), [
+ { request: "generate", capability: generate, checked: "request", status: "resolved", endpoint: "other", use: "example.other", pricing: { kind: "local" }, binding: "other" },
]);
assert.equal((await preflightRuntimeConfig(bound, { registry, capabilities: [generate] })).diagnostics
.some((item) => item.code === "RUNTIME_CAPABILITY_AMBIGUOUS"), false);
@@ -479,8 +541,8 @@ test("providers, doctor and invoke share one resolver: a contested capability is
const wrong = join(root, "wrong.json");
await writeFile(wrong, JSON.stringify(profile({ dataRoot: ".", endpoints, bindings: { [generateKey]: "local" } })));
- assert.deepEqual(await describeRuntimeConfigProviders(wrong, [generate], { registry }), [
- { capability: generate, status: "unresolved", binding: "local" },
+ assert.deepEqual(await describeRuntimeConfigProviders(wrong, [providerQuery("generate", generate, returns, null)], { registry }), [
+ { request: "generate", capability: generate, checked: "request", status: "unresolved", binding: "local" },
]);
assert.ok((await preflightRuntimeConfig(wrong, { registry, capabilities: [generate] })).diagnostics
.some((item) => item.code === "RUNTIME_BINDING_INVALID"));
diff --git a/packages/spatial/README.md b/packages/spatial/README.md
index f65cd252..1c93760a 100644
--- a/packages/spatial/README.md
+++ b/packages/spatial/README.md
@@ -8,8 +8,11 @@ renderer, Provider or Core behavior.
The package exposes self-described `Canvas`, `Point`, `Path`, `Extent`, `RegionTimeline`, `Frame`,
`AnchoredFrame` and `AspectFrame` author Surfaces plus pure geometry functions and fixed-port Producers.
-`RegionTimeline` accepts already measured data rather than running a detector. One SVS Recipe holds
-the exact ProgramSpace frame count and named tracks whose array positions are Frames:
+`RegionTimeline` accepts already measured data rather than running a detector. Face detection and
+tracking belong to reference observation before authoring: inspect the footage, keep the useful boxes
+as ordinary numbers, and pass that finished timeline to Caption. A Build never starts a detector to
+discover its own layout. One SVS Recipe holds the exact ProgramSpace frame count and named tracks whose
+array positions are Frames:
```svs
heads.default {
diff --git a/packages/spatial/src/manifest.ts b/packages/spatial/src/manifest.ts
index 3e9ad74c..f786845a 100644
--- a/packages/spatial/src/manifest.ts
+++ b/packages/spatial/src/manifest.ts
@@ -193,6 +193,7 @@ export const spatialMarkupSurfaces = [
example: `<space:RegionTimeline id="heads" within={vertical} recipe={tracking.heads.default}/>` ,
notes: [
emptyNote,
+ "Detection and tracking happen before authoring; this Surface consumes finished numeric evidence and never invokes them during a Build.",
"The array index is the ProgramSpace Frame; the Surface performs no timestamp conversion, interpolation, smoothing or identity inference.",
"Each normalized AABB lies inside [0, 1] and is resolved into Canvas pixels during author compilation; null remains explicit absence.",
"Track ids are ordinary external labels; a consumer may interpret them as Script Roles without Spatial knowing what a Role is.",
diff --git a/packages/store-sqlite/test/store.test.ts b/packages/store-sqlite/test/store.test.ts
index 0106280d..b48e9a7b 100644
--- a/packages/store-sqlite/test/store.test.ts
+++ b/packages/store-sqlite/test/store.test.ts
@@ -255,8 +255,8 @@ test("shared capacity resources are acquired atomically across Builds", async ()
try {
const state = new SqliteRuntimeState(join(directory, "runtime.sqlite"));
const pool = { id: "pool:kie.main", limit: 2 };
- const seedance = { id: "lane:kie.main/seedance-2-mini", limit: 1 };
- const minimax = { id: "lane:kie.main/minimax-h3", limit: 2 };
+ const seedance = { id: "capacity:kie.main/seedance-2-mini", limit: 1 };
+ const minimax = { id: "capacity:kie.main/minimax-h3", limit: 2 };
const first = await state.execution.acquireCapacity({
build: "seedance-a",
diff --git a/packages/video-cli/src/creation.ts b/packages/video-cli/src/creation.ts
index 7e595cba..b6473f51 100644
--- a/packages/video-cli/src/creation.ts
+++ b/packages/video-cli/src/creation.ts
@@ -11,18 +11,18 @@ import {
resolveSpeechEstimateLanguage,
speechEstimatePolicyFromAttributes,
} from "@hypit/estimate";
-import { geminiCapabilities, geminiModels, sealGeminiRequest } from "@hypit/gemini";
-import type { GeminiMediaPart, GeminiModel } from "@hypit/gemini";
+import { geminiCapabilities, geminiModels, geminiTypes, sealGeminiRequest, verifyVisualObservation } from "@hypit/gemini";
+import type { GeminiMediaPart, GeminiModel, VisualObservation } from "@hypit/gemini";
import { verifyGeneratedAudioSet } from "@hypit/generation";
import type { GeneratedAudioSet } from "@hypit/generation";
import { mimoTtsEndpoints, sealMimoTtsRequest } from "@hypit/mimo-tts";
import type { CanonicalValue, CapabilityRef, Need, StoredValue } from "@hypit/protocol";
import type { ResourceStore } from "@hypit/runtime";
-import type { RuntimeHostCapabilityProvider } from "@hypit/runtime-host-node";
+import type { RuntimeHostCapabilityProvider, RuntimeHostProviderQuery } from "@hypit/runtime-host-node";
import { sealSpeechEvidenceAudio } from "@hypit/speech";
import { speechEvidenceTypes } from "@hypit/speech-evidence";
import type { AlignedTranscriptEvidence } from "@hypit/speech-evidence";
-import { sealText, textTypes } from "@hypit/text";
+import { sealText } from "@hypit/text";
import { whisperXCapabilities, whisperXRequestForEvidenceAudio } from "@hypit/whisperx";
import type { WhisperXLanguage } from "@hypit/whisperx";
@@ -43,7 +43,7 @@ export type CreationCommand = typeof creationCommands[number];
/** The slice of the Runtime host these commands use; tests hand in a fake. */
export type CreationHost = {
- providers(capabilities: readonly CapabilityRef[]): Promise<readonly RuntimeHostCapabilityProvider[]>;
+ providers(requests: readonly RuntimeHostProviderQuery[]): Promise<readonly RuntimeHostCapabilityProvider[]>;
invoke(need: Need, resources: ResourceStore): Promise<{ readonly value: StoredValue }>;
};
@@ -166,9 +166,14 @@ function capabilityName(capability: CapabilityRef): string {
}
/** Which Endpoint will serve this call and where its Provider publishes prices, before spending. */
-async function selectedProvider(host: CreationHost, capability: CapabilityRef, profile: string): Promise<RuntimeHostCapabilityProvider> {
- const [provider] = await host.providers([capability]);
- const subject = capabilityName(capability);
+async function selectedProvider(host: CreationHost, need: Need, profile: string): Promise<RuntimeHostCapabilityProvider> {
+ const [provider] = await host.providers([{
+ request: need.id,
+ capability: need.capability,
+ returns: need.returns,
+ constraints: need.constraints,
+ }]);
+ const subject = capabilityName(need.capability);
assert(provider !== undefined && provider.status !== "unresolved",
`No Endpoint in ${profile} serves ${subject}; hypit plan --runtime ${profile} shows which Endpoint each capability needs`);
assert(provider.status !== "ambiguous",
@@ -307,8 +312,6 @@ async function observe(argv: readonly string[], io: CliIo, environment: Creation
const to = await destination(parsed, environment.cwd);
const { profile, host } = await environment.openHost(parsed.options.get("--runtime"));
const capability = geminiCapabilities[model];
- const provider = await selectedProvider(host, capability, profile);
- if (!parsed.json) io.write(`Observing with ${model} through ${providerLine(provider)}\n`);
const resources = new MemoryResourceStore();
const media: GeminiMediaPart[] = [];
for (const item of parsed.positionals) {
@@ -318,14 +321,16 @@ async function observe(argv: readonly string[], io: CliIo, environment: Creation
const need: Need = {
id: "need:hypit-observe",
capability,
- returns: textTypes.text,
+ returns: geminiTypes.visualObservation,
constraints: sealGeminiRequest({ instruction, prompt, media }) as unknown as CanonicalValue,
result: "record:hypit-observe",
};
+ const provider = await selectedProvider(host, need, profile);
+ if (!parsed.json) io.write(`Observing with ${model} through ${providerLine(provider)}\n`);
const fulfillment = await host.invoke(need, resources);
assert(fulfillment.value.kind === "inline", "the observation came back by reference");
- const text = (fulfillment.value.value as { readonly value?: unknown } | null)?.value;
- assert(typeof text === "string", "the observation is not Text");
+ verifyVisualObservation(fulfillment.value.value);
+ const text = (fulfillment.value.value as unknown as VisualObservation).text;
await writeNew(to, `${text}\n`);
const view = {
format: "hypit.video-cli-observe@1",
@@ -376,8 +381,6 @@ async function transcribe(argv: readonly string[], io: CliIo, environment: Creat
assert(language === "en" || language === "zh" || language === "es", "--language must be en, zh or es");
const to = await destination(parsed, environment.cwd);
const { profile, host } = await environment.openHost(parsed.options.get("--runtime"));
- const provider = await selectedProvider(host, whisperXCapabilities.alignment, profile);
- if (!parsed.json) io.write(`Transcribing through ${providerLine(provider)}\n`);
const evidence = await speechEvidenceBytes(source);
const resources = new MemoryResourceStore();
const artifact = await resources.put(evidence.bytes, "audio/wav");
@@ -389,6 +392,8 @@ async function transcribe(argv: readonly string[], io: CliIo, environment: Creat
constraints: whisperXRequestForEvidenceAudio(audio, { language: language as WhisperXLanguage }),
result: "record:hypit-transcribe",
};
+ const provider = await selectedProvider(host, need, profile);
+ if (!parsed.json) io.write(`Transcribing through ${providerLine(provider)}\n`);
const fulfillment = await host.invoke(need, resources);
assert(fulfillment.value.kind === "inline", "the transcript came back by reference");
const passages = passagesInSeconds(fulfillment.value.value as unknown as AlignedTranscriptEvidence);
@@ -428,8 +433,6 @@ async function speak(argv: readonly string[], io: CliIo, environment: CreationEn
const endpoint = mimoTtsEndpoints.voiceDesign;
assert(endpoint !== undefined, "@hypit/mimo-tts declares no voice-design endpoint");
const { profile, host } = await environment.openHost(parsed.options.get("--runtime"));
- const provider = await selectedProvider(host, endpoint.capability, profile);
- if (!parsed.json) io.write(`Speaking with ${endpoint.ports.model} through ${providerLine(provider)}\n`);
const resources = new MemoryResourceStore();
const need: Need = {
id: "need:hypit-speak",
@@ -438,6 +441,8 @@ async function speak(argv: readonly string[], io: CliIo, environment: CreationEn
constraints: sealMimoTtsRequest("mimo-v2.5-tts-voicedesign", { text: [text], voiceDescription: [voice] }) as unknown as CanonicalValue,
result: "record:hypit-speak",
};
+ const provider = await selectedProvider(host, need, profile);
+ if (!parsed.json) io.write(`Speaking with ${endpoint.ports.model} through ${providerLine(provider)}\n`);
const fulfillment = await host.invoke(need, resources);
assert(fulfillment.value.kind === "inline", "the speech came back by reference");
verifyGeneratedAudioSet(fulfillment.value.value);
diff --git a/packages/video-cli/test/cli.test.ts b/packages/video-cli/test/cli.test.ts
index d455bfc2..be399f3e 100644
--- a/packages/video-cli/test/cli.test.ts
+++ b/packages/video-cli/test/cli.test.ts
@@ -56,7 +56,7 @@ test("provider-free example plans from installed Source packages", async () => {
readonly steps: number;
readonly targets: readonly string[];
};
- assert.equal(plan.format, "hypit.cli-plan@2");
+ assert.equal(plan.format, "hypit.cli-plan@3");
assert.equal(plan.steps > 0, true);
assert.equal(plan.targets.length, 1);
});
diff --git a/packages/video-cli/test/creation.test.ts b/packages/video-cli/test/creation.test.ts
index 84c6b2e0..07b457da 100644
--- a/packages/video-cli/test/creation.test.ts
+++ b/packages/video-cli/test/creation.test.ts
@@ -5,10 +5,10 @@ import { join } from "node:path";
import test from "node:test";
import { canonicalize } from "@hypit/protocol";
-import type { CapabilityRef, Need } from "@hypit/protocol";
+import type { Need } from "@hypit/protocol";
import type { ResourceStore } from "@hypit/runtime";
import { sealAlignedTranscriptEvidence } from "@hypit/speech-evidence";
-import { sealText } from "@hypit/text";
+import { sealVisualObservation } from "@hypit/gemini";
import { interpretWhisperXTranscript } from "@hypit/whisperx";
import type { WhisperXAlignmentRequest } from "@hypit/whisperx";
@@ -34,8 +34,14 @@ const page = { kind: "page" as const, url: "https://prices.example/models" };
/** A host whose Profile serves everything through one paid Endpoint, answering each Need in kind. */
function host(seen: Need[]): CreationHost {
return {
- providers: async (capabilities: readonly CapabilityRef[]) => capabilities.map((capability) => ({
- capability, status: "resolved" as const, endpoint: "paid.default", use: "@hypit/provider-example", pricing: page,
+ providers: async (requests) => requests.map((request) => ({
+ request: request.request,
+ capability: request.capability,
+ checked: request.constraints === undefined ? "capability" as const : "request" as const,
+ status: "resolved" as const,
+ endpoint: "paid.default",
+ use: "@hypit/provider-example",
+ pricing: page,
})),
invoke: async (need: Need, resources: ResourceStore) => {
seen.push(need);
@@ -52,7 +58,7 @@ function host(seen: Need[]): CreationHost {
return { value: { kind: "inline" as const, value: canonicalize({ audios: [artifact] }) } };
}
const request = need.constraints as { readonly prompt: string; readonly media: readonly unknown[] };
- return { value: { kind: "inline" as const, value: canonicalize(sealText(`saw ${request.media.length} media; asked: ${request.prompt}`)) } };
+ return { value: { kind: "inline" as const, value: canonicalize(sealVisualObservation(`saw ${request.media.length} media; asked: ${request.prompt}`)) } };
},
};
}
@@ -171,7 +177,12 @@ test("a Profile that does not serve the capability stops before anything is spen
const unserved: CreationEnvironment = {
cwd: root,
openHost: async () => ({ profile: join(root, "hypit.runtime.json"), host: {
- providers: async (capabilities) => capabilities.map((capability) => ({ capability, status: "unresolved" as const })),
+ providers: async (requests) => requests.map((request) => ({
+ request: request.request,
+ capability: request.capability,
+ checked: request.constraints === undefined ? "capability" as const : "request" as const,
+ status: "unresolved" as const,
+ })),
invoke: async () => { invoked = true; throw new Error("must not be reached"); },
} }),
};
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index 4cf84f04..6f67fe2b 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -579,6 +579,12 @@ importers:
'@hypit/elaborator':
specifier: workspace:*
version: link:../elaborator
+ '@hypit/generation':
+ specifier: workspace:*
+ version: link:../generation
+ '@hypit/model-kit':
+ specifier: workspace:*
+ version: link:../model-kit
'@hypit/package-loader-node':
specifier: workspace:*
version: link:../package-loader-node
diff --git a/skills/hypit/references/playbooks/craft/captions.md b/skills/hypit/references/playbooks/craft/captions.md
index 9ca2d09c..c90f8c70 100644
--- a/skills/hypit/references/playbooks/craft/captions.md
+++ b/skills/hypit/references/playbooks/craft/captions.md
@@ -66,6 +66,33 @@ exact font → caption-fine:Style → caption:Program
Add `{captions.track}` to `film:Film` as one peer Visual Track. If the format intentionally has no
captions, omit the Caption components entirely.
+## Moving captions consume finished numeric regions
+
+When a caption follows a speaker, measure the face or head before the Build and write the useful
+normalized boxes into an SVS `RegionTimeline`. Intersect that track with the Role's speaking spans:
+the Role has `[x, y, width, height]` only on Frames where that person is speaking and `null` elsewhere.
+Then pass the authored timeline directly to Fine:
+
+```svs
+heads.default {
+ frame-count: 3;
+ tracks: [
+ {"id":"WIFE","regions":[[0.12,0.09,0.20,0.26],null,[0.13,0.10,0.20,0.26]]}
+ ];
+}
+```
+
+```svml
+<space:RegionTimeline id="heads" within={vertical} recipe={tracking.heads.default}/>
+<caption-fine:Track id="captions" document={story.caption}
+ semantic={speech.semantic} program={caption-program} regions={heads}/>
+```
+
+The numbers are authoring evidence, regardless of which observation tool measured them. Adjusting a
+box upward or outward is an ordinary pre-Build data transformation. Do not add face detection,
+tracking, identity association or box expansion to the production graph; Caption should receive the
+final placement evidence rather than rediscovering the face while paid work is running.
+
## A Fine Caption Recipe writes eleven keys or it throws
`caption-fine:Style` requires `align`, `background`, `fill`, `line-height`, `padding`, `radius`,