Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

fix(runtime): Vertex is honest about timeouts, model ids and reachability; pools conflict at doctor time

kashorin402c41cabb01
diff --git a/docs/guide/runtime.md b/docs/guide/runtime.md
index ce9e059f..896f4191 100644
--- a/docs/guide/runtime.md
+++ b/docs/guide/runtime.md
@@ -169,7 +169,7 @@ Runtime pointer, or the entry source directory. Runtime configuration cannot wid
the Worker materializes its state only while advancing it and writes accepted facts back immediately.
There is no Build concurrency setting and Build identity is not a capacity resource.
-Endpoint packages declare the two actual execution queues: one Provider pool and its model lanes.
+The Profile names each Endpoint's pool; the Endpoint package declares the model lanes inside it. Those are the two actual execution queues.
Work from separate Builds shares those exact external capacities. Each resource has one limit: an
immediate call releases its slot when it returns, while an asynchronous Operation keeps the same slot
until it becomes terminal. Unrelated Builds and Commands may advance together.
diff --git a/packages/driver-node/src/registry.ts b/packages/driver-node/src/registry.ts
index c81b40e9..4c9e5014 100644
--- a/packages/driver-node/src/registry.ts
+++ b/packages/driver-node/src/registry.ts
@@ -123,6 +123,31 @@ export class EndpointRegistry implements EndpointRegistrar {
return [...new Set(this.#registrations.map((registration) => registration.id))].sort();
}
+ /**
+ * Shared capacity resources that different registrations size differently. Two Endpoints in one
+ * pool with different concurrency would otherwise only collide inside the scheduler, mid-Build.
+ */
+ capacityConflicts(): readonly { readonly resource: string; readonly limits: readonly number[]; readonly endpointIds: readonly string[] }[] {
+ const limits = new Map<string, Map<number, Set<string>>>();
+ for (const registration of this.#registrations) {
+ for (const resource of registration.scheduling?.resources ?? []) {
+ const byLimit = limits.get(resource.id) ?? new Map<number, Set<string>>();
+ const ids = byLimit.get(resource.limit) ?? new Set<string>();
+ ids.add(registration.id);
+ byLimit.set(resource.limit, ids);
+ limits.set(resource.id, byLimit);
+ }
+ }
+ return [...limits.entries()]
+ .filter(([, byLimit]) => byLimit.size > 1)
+ .map(([resource, byLimit]) => ({
+ resource,
+ limits: [...byLimit.keys()].sort((left, right) => left - right),
+ endpointIds: [...new Set([...byLimit.values()].flatMap((ids) => [...ids]))].sort(),
+ }))
+ .sort((left, right) => left.resource.localeCompare(right.resource));
+ }
+
/** Capabilities offered by more than one Endpoint, with the ids, for a deployment to bind. */
contested(): readonly { readonly capability: CapabilityRef; readonly endpointIds: readonly string[]; readonly bound?: string }[] {
const found: { readonly capability: CapabilityRef; readonly endpointIds: readonly string[]; readonly bound?: string }[] = [];
diff --git a/packages/provider-kie/README.md b/packages/provider-kie/README.md
index 23e949e8..901b4015 100644
--- a/packages/provider-kie/README.md
+++ b/packages/provider-kie/README.md
@@ -42,9 +42,9 @@ requests, not in Provider routing.
Declarative activation names an ordinary CredentialRef, not an environment-specific Provider field:
```json
-{
+"kie.personal": {
"use": "@hypit/provider-kie",
- "instance": "kie.personal",
+ "pool": "generation",
"config": {
"apiKey": { "store": "os", "key": "kie.api-key" },
"defaultConcurrency": 8,
diff --git a/packages/provider-vertex/README.md b/packages/provider-vertex/README.md
index c69ed47e..ff64a77e 100644
--- a/packages/provider-vertex/README.md
+++ b/packages/provider-vertex/README.md
@@ -1,9 +1,53 @@
# `@hypit/provider-vertex`
-Vertex Gemini Runtime Provider and embedded backend for callers that need text, image or video input.
-It fulfills the same provider-neutral `@hypit/gemini` capabilities as HypiHub, so Runtime Profiles
-choose the backend without changing Author Source. The package owns the Google GenAI client and
-credential parsing.
+Vertex Gemini Runtime Provider. It fulfils the same Provider-neutral `@hypit/gemini` capabilities
+as HypiHub, so a Runtime Profile chooses the backend without changing Author Source. The package
+owns the Google GenAI client, credential parsing and the Vertex model ids.
+
+## Runtime Profile
+
+```json
+{
+ "credentials": {
+ "env": { "use": "@hypit/credential-store-env" }
+ },
+ "endpoints": {
+ "vertex.default": {
+ "use": "@hypit/provider-vertex",
+ "pool": "generation",
+ "config": {
+ "project": { "store": "env", "key": "GOOGLE_CLOUD_PROJECT" },
+ "credentials": { "store": "env", "key": "GOOGLE_APPLICATION_CREDENTIALS_JSON" },
+ "location": "global",
+ "requestTimeoutMs": 300000
+ }
+ }
+ }
+}
+```
+
+`project` and `credentials` are both required CredentialRefs. `credentials` is the JSON of a Google
+credential: a service account key, or the `application_default_credentials.json` that
+`gcloud auth application-default login` writes (an `authorized_user` credential). `location`
+defaults to `global`, where the current Gemini models are served. `requestTimeoutMs` bounds each
+request; the default is five minutes.
+
+When another selected Endpoint also offers Gemini (HypiHub does), the Profile's `bindings` say which
+one serves `@hypit/gemini@1#gemini-3.1-pro`; neither Provider hides the capability.
+
+## Model ids
+
+The capability names the model the author chose; this package maps it to the id Vertex publishes
+it under (`gemini-3.1-pro` becomes `gemini-3.1-pro-preview`). A capability without a Vertex id is
+refused at activation rather than sent under a name Vertex may not recognise.
+
+## Doctor
+
+`hypit doctor` asks Vertex for each declared model with the configured project and credentials, so a
+project without the Vertex AI API enabled, without access to the model, or with a wrong location is
+reported before the first paid call, with Vertex's own explanation.
+
+## Embedded use
```ts
import { createVertexGeminiGenerator } from "@hypit/provider-vertex";
@@ -11,6 +55,5 @@ import { createVertexGeminiGenerator } from "@hypit/provider-vertex";
const generate = createVertexGeminiGenerator({
project: process.env.GOOGLE_CLOUD_PROJECT!,
credentials: process.env.GOOGLE_APPLICATION_CREDENTIALS_JSON!,
- model: "gemini-3.1-pro",
});
```
diff --git a/packages/provider-vertex/src/activation.ts b/packages/provider-vertex/src/activation.ts
index 5cd04c3e..8415478d 100644
--- a/packages/provider-vertex/src/activation.ts
+++ b/packages/provider-vertex/src/activation.ts
@@ -7,7 +7,7 @@ import {
runtimeConfigString,
} from "@hypit/runtime-kit";
-import { createVertexProvider } from "./provider.js";
+import { createVertexProvider, diagnoseVertexProvider } from "./provider.js";
const adapter = createRuntimeEndpointAdapterFacet({
use: "@hypit/provider-vertex",
@@ -22,7 +22,7 @@ const adapter = createRuntimeEndpointAdapterFacet({
const location = runtimeConfigString(config.location, "Vertex location");
const defaultConcurrency = runtimeConfigPositiveInteger(config.defaultConcurrency, "Vertex defaultConcurrency");
const requestTimeoutMs = runtimeConfigPositiveInteger(config.requestTimeoutMs, "Vertex requestTimeoutMs");
- return { endpoint: createVertexProvider({
+ const options = {
instance: context.instance,
pool: context.pool,
project,
@@ -30,7 +30,11 @@ const adapter = createRuntimeEndpointAdapterFacet({
...(location === undefined ? {} : { location }),
...(defaultConcurrency === undefined ? {} : { defaultConcurrency }),
...(requestTimeoutMs === undefined ? {} : { requestTimeoutMs }),
- }) };
+ };
+ return {
+ endpoint: createVertexProvider(options),
+ diagnose: async (doctor) => await diagnoseVertexProvider(options, doctor),
+ };
},
});
diff --git a/packages/provider-vertex/src/gemini.ts b/packages/provider-vertex/src/gemini.ts
index 8bb7f442..095095b9 100644
--- a/packages/provider-vertex/src/gemini.ts
+++ b/packages/provider-vertex/src/gemini.ts
@@ -25,12 +25,32 @@ function parseCredentials(value: Record<string, unknown> | string): Record<strin
return parsed as Record<string, unknown>;
}
-/** Vertex Gemini VLM generator for text, image and video inline parts. */
-export function createVertexGeminiGenerator(options: VertexGeminiGeneratorOptions) {
+/**
+ * The Vertex publisher id behind each Hypit Gemini capability. The capability names the model the
+ * author chose; the id Vertex serves it under is this Provider's fact. A capability without an
+ * entry is refused rather than sent under a name Vertex may not recognise.
+ */
+const VERTEX_MODEL_IDS: Readonly<Record<string, string>> = {
+ "gemini-3.1-pro": "gemini-3.1-pro-preview",
+};
+
+export function vertexModelId(capabilityName: string): string {
+ const id = VERTEX_MODEL_IDS[capabilityName];
+ if (id === undefined) throw new Error(`Vertex Provider has no published model id for ${capabilityName}`);
+ return id;
+}
+
+export type VertexClientOptions = {
+ readonly project: string;
+ readonly credentials: Record<string, unknown> | string;
+ readonly location?: string;
+ readonly requestTimeoutMs?: number;
+};
+
+export function createVertexClient(options: VertexClientOptions): GoogleGenAI {
const project = options.project.trim();
assert(project.length > 0, "Vertex project is empty");
- const model = options.model?.trim() || "gemini-3.1-pro";
- const client = new GoogleGenAI({
+ return new GoogleGenAI({
vertexai: true,
project,
location: options.location?.trim() || "global",
@@ -38,7 +58,19 @@ export function createVertexGeminiGenerator(options: VertexGeminiGeneratorOption
credentials: parseCredentials(options.credentials),
scopes: ["https://www.googleapis.com/auth/cloud-platform"],
},
+ httpOptions: { timeout: options.requestTimeoutMs ?? 300_000 },
});
+}
+
+/** Ask Vertex whether this project can reach the model; the service's own message says why not. */
+export async function probeVertexModel(options: VertexClientOptions, modelId: string): Promise<void> {
+ await createVertexClient(options).models.get({ model: modelId });
+}
+
+/** Vertex Gemini VLM generator for text, image and video inline parts. */
+export function createVertexGeminiGenerator(options: VertexGeminiGeneratorOptions) {
+ const model = options.model?.trim() || vertexModelId("gemini-3.1-pro");
+ const client = createVertexClient(options);
return async (input: VertexGeminiGenerateInput): Promise<string> => {
const result = await client.models.generateContent({
diff --git a/packages/provider-vertex/src/provider.ts b/packages/provider-vertex/src/provider.ts
index efce8d00..7c4b6d44 100644
--- a/packages/provider-vertex/src/provider.ts
+++ b/packages/provider-vertex/src/provider.ts
@@ -1,13 +1,14 @@
import { defineEndpointPackage } from "@hypit/endpoint-kit";
import type { EndpointInvocationContext, ImmediateEndpointHandler } from "@hypit/endpoint-kit";
import { geminiCapabilities, geminiModels, verifyGeminiRequest } from "@hypit/gemini";
+import type { CapabilityRef } from "@hypit/protocol";
+import type { RuntimeDoctorDiagnostic } from "@hypit/runtime-kit";
import type { GeminiRequest } from "@hypit/gemini";
import { canonicalize } from "@hypit/protocol";
-import { credentialRef } from "@hypit/runtime";
import type { CredentialRef } from "@hypit/runtime";
import { sealText, textTypes } from "@hypit/text";
-import { createVertexGeminiGenerator } from "./gemini.js";
+import { createVertexGeminiGenerator, probeVertexModel, vertexModelId } from "./gemini.js";
export const vertexProviderModuleRef = { name: "@hypit/provider-vertex", version: "1" } as const;
@@ -27,15 +28,49 @@ function secret(context: EndpointInvocationContext, name: "project" | "credentia
return value;
}
-/**
- * The Vertex publisher id behind each Hypit Gemini capability. The capability names the model the
- * author chose; the id Vertex serves it under is this Provider's fact, not the author's.
- */
-const VERTEX_MODEL_IDS: Readonly<Record<string, string>> = {
- "gemini-3.1-pro": "gemini-3.1-pro-preview",
-};
+function capabilityKey(capability: CapabilityRef): string {
+ return `${capability.module.name}@${capability.module.version}#${capability.name}`;
+}
+
+/** Active doctor: can this project reach each declared model on Vertex? Vertex says why when not. */
+export async function diagnoseVertexProvider(
+ options: Pick<CreateVertexProviderOptions, "location" | "requestTimeoutMs">,
+ context: {
+ readonly credentials: Readonly<Record<string, { readonly secret: string }>>;
+ readonly capabilities?: readonly CapabilityRef[];
+ },
+): Promise<readonly RuntimeDoctorDiagnostic[]> {
+ const project = context.credentials.project?.secret ?? "";
+ const credentials = context.credentials.credentials?.secret ?? "";
+ if (project.trim().length === 0 || credentials.trim().length === 0) {
+ return [{ severity: "error", code: "VERTEX_CREDENTIALS_MISSING", message: "Vertex project and credentials are not both configured" }];
+ }
+ const requested = context.capabilities ?? geminiModels.map((model) => geminiCapabilities[model]);
+ const diagnostics: RuntimeDoctorDiagnostic[] = [];
+ for (const capability of requested) {
+ if (!geminiModels.some((model) => capabilityKey(geminiCapabilities[model]) === capabilityKey(capability))) continue;
+ try {
+ await probeVertexModel({
+ project,
+ credentials,
+ ...(options.location === undefined ? {} : { location: options.location }),
+ ...(options.requestTimeoutMs === undefined ? {} : { requestTimeoutMs: options.requestTimeoutMs }),
+ }, vertexModelId(capability.name));
+ } catch (error) {
+ diagnostics.push({
+ severity: "error",
+ code: "VERTEX_MODEL_UNAVAILABLE",
+ message: `${project} cannot reach ${vertexModelId(capability.name)} on Vertex: ${error instanceof Error ? error.message.slice(0, 400) : String(error)}`,
+ subject: capabilityKey(capability),
+ });
+ }
+ }
+ return diagnostics;
+}
export function createVertexProvider(options: CreateVertexProviderOptions) {
+ // Every declared capability must have a Vertex id before anything is registered.
+ for (const model of geminiModels) vertexModelId(model);
const handler: ImmediateEndpointHandler = async (context) => {
const request = context.need.constraints as unknown;
verifyGeminiRequest(request);
@@ -43,7 +78,7 @@ export function createVertexProvider(options: CreateVertexProviderOptions) {
const generate = createVertexGeminiGenerator({
project: secret(context, "project"),
credentials: secret(context, "credentials"),
- model: VERTEX_MODEL_IDS[context.need.capability.name] ?? context.need.capability.name,
+ model: vertexModelId(context.need.capability.name),
...(options.location === undefined ? {} : { location: options.location }),
...(options.requestTimeoutMs === undefined ? {} : { requestTimeoutMs: options.requestTimeoutMs }),
});
@@ -63,8 +98,8 @@ export function createVertexProvider(options: CreateVertexProviderOptions) {
pool: options.pool ?? options.instance ?? "vertex.default",
pricing: { kind: "page", url: "https://cloud.google.com/vertex-ai/generative-ai/pricing" },
credentials: {
- project: options.project ?? credentialRef("env", "GOOGLE_CLOUD_PROJECT"),
- credentials: options.credentials ?? credentialRef("env", "GOOGLE_APPLICATION_CREDENTIALS_JSON"),
+ project: options.project,
+ credentials: options.credentials,
},
credentialInputs: {
project: { label: "Google Cloud project" },
diff --git a/packages/provider-vertex/test/provider.test.ts b/packages/provider-vertex/test/provider.test.ts
index dc20285d..c4ee4366 100644
--- a/packages/provider-vertex/test/provider.test.ts
+++ b/packages/provider-vertex/test/provider.test.ts
@@ -7,6 +7,7 @@ import { credentialRef } from "@hypit/runtime";
import { textTypes } from "@hypit/text";
import { createVertexProvider } from "../src/provider.js";
+import { vertexModelId } from "../src/gemini.js";
test("Vertex Runtime provider exposes the same provider-neutral Gemini capabilities", async () => {
const registry = new EndpointRegistry();
@@ -31,3 +32,8 @@ test("Vertex Runtime provider exposes the same provider-neutral Gemini capabilit
{ slot: "project", kind: "secret" },
]);
});
+
+test("Vertex model ids fail closed", () => {
+ assert.equal(vertexModelId("gemini-3.1-pro"), "gemini-3.1-pro-preview");
+ assert.throws(() => vertexModelId("gemini-99"), /no published model id for gemini-99/u);
+});
diff --git a/packages/runtime-local/src/config.ts b/packages/runtime-local/src/config.ts
index d68c89a2..4f0249bc 100644
--- a/packages/runtime-local/src/config.ts
+++ b/packages/runtime-local/src/config.ts
@@ -721,6 +721,14 @@ async function inspectRuntimeConfig(
subject: key,
});
}
+ for (const conflict of endpoints.capacityConflicts()) {
+ diagnostics.push({
+ severity: "error",
+ code: "RUNTIME_POOL_CONFLICT",
+ message: `${conflict.endpointIds.join(", ")} share ${conflict.resource} but size it ${conflict.limits.join(" and ")}; give them the same defaultConcurrency or different pools`,
+ subject: conflict.resource,
+ });
+ }
for (const contest of endpoints.contested()) {
if (contest.bound !== undefined) continue;
const key = capabilityKey(contest.capability);
diff --git a/packages/runtime-local/test/config.test.ts b/packages/runtime-local/test/config.test.ts
index 205beb9d..5d839e39 100644
--- a/packages/runtime-local/test/config.test.ts
+++ b/packages/runtime-local/test/config.test.ts
@@ -489,3 +489,45 @@ test("providers, doctor and invoke share one resolver: a contested capability is
await rm(root, { recursive: true, force: true });
}
});
+
+test("doctor reports two Endpoints that share a pool but size it differently", async () => {
+ const root = await mkdtemp(join(tmpdir(), "hypit-runtime-pools-"));
+ const path = join(root, "hypit.runtime.json");
+ await writeFile(path, JSON.stringify(profile({
+ dataRoot: ".",
+ endpoints: {
+ four: { use: "example.four", pool: "generation" },
+ ten: { use: "example.ten", pool: "generation" },
+ },
+ })));
+ const returns = { module: { name: "example.value", version: "1" }, name: "Output" } as const;
+ const registry = new RuntimeAdapterRegistry();
+ for (const [use, name, concurrency] of [["example.four", "four", 4], ["example.ten", "ten", 10]] as const) {
+ registry.registerFacet(createRuntimeEndpointAdapterFacet({
+ use,
+ activate: (context) => ({
+ endpoint: defineEndpointPackage({
+ module: { name: `example.provider.${name}`, version: "1" },
+ facet: name,
+ instance: context.instance,
+ pool: context.pool ?? context.instance,
+ pricing: { kind: "local" },
+ defaultConcurrency: concurrency,
+ capabilities: [{
+ capability: { module: { name: "example.model", version: "1" }, name },
+ returns,
+ lifecycle: "immediate",
+ handler: () => ({ value: { kind: "inline" as const, value: null } }),
+ }],
+ }),
+ }),
+ }));
+ }
+ try {
+ const conflict = (await preflightRuntimeConfig(path, { registry })).diagnostics.find((item) => item.code === "RUNTIME_POOL_CONFLICT");
+ assert.ok(conflict, "the shared pool is reported");
+ assert.match(conflict.message, /four, ten share pool:generation but size it 4 and 10/u);
+ } finally {
+ await rm(root, { recursive: true, force: true });
+ }
+});
diff --git a/packages/video-cli/src/distribution.ts b/packages/video-cli/src/distribution.ts
index 6aaba9bb..9e39ff56 100644
--- a/packages/video-cli/src/distribution.ts
+++ b/packages/video-cli/src/distribution.ts
@@ -28,7 +28,7 @@ export const videoCliDistribution: CliDistribution = {
endpoints: {
"hypihub.default": {
use: "@hypit/provider-hypihub",
- pool: "hypihub.default",
+ pool: "generation",
config: {
baseUrl: "https://hypit.ai",
apiKey: { store: "os", key: "hypihub.oauth" },
@@ -36,11 +36,11 @@ export const videoCliDistribution: CliDistribution = {
},
"media.local": {
use: "@hypit/provider-media-local",
- pool: "media.local",
+ pool: "media",
},
"hyperframes.local": {
use: "@hypit/provider-hyperframes-local",
- pool: "hyperframes.local",
+ pool: "render",
},
},
},