Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.
TypeScript agentic-aiaiai-agentsai-videocompilerdslffmpeggenerative-aillmmarkup-languagemonorepoplugin-systemprogramming-languagetext-to-videotypescriptvideovideo-automationvideo-clonevideo-editingvideo-generation
feat: route Gemini through runtime providers
rpone6264a57f256d
diff --git a/.agents/skills/hypit/references/credentials.md b/.agents/skills/hypit/references/credentials.mdindex dc180901..3e131747 100644--- a/.agents/skills/hypit/references/credentials.md+++ b/.agents/skills/hypit/references/credentials.md@@ -21,7 +21,9 @@ for a HypiHub key when that model is available there. Do not ask them to changeswitch Provider.HypiHub is optional and is never required when the selected Runtime Profile has another Provider.-For Gemini VLM/reference observation, `HYPIT_GEMINI_PROVIDER=auto` (the default) uses HypiHub when+For ordinary `@hypit/gemini` Author Source, the Runtime Profile selects+`@hypit/provider-hypihub` or `@hypit/provider-vertex`; changing that Endpoint never changes Source.+For the reference-video preprocessing observer only, `HYPIT_GEMINI_PROVIDER=auto` (the default) uses HypiHub when`HYPIHUB_API_KEY` is present and otherwise uses Vertex when its two Google variables are present.Set `HYPIT_GEMINI_PROVIDER=hypihub` or `vertex` to select one explicitly. If a user's configured keycannot reach the requested model, point them to [hypit.ai](https://hypit.ai) for a HypiHub key insteaddiff --git a/.agents/skills/hypit/references/runtime.md b/.agents/skills/hypit/references/runtime.mdindex 458fe007..b8866475 100644--- a/.agents/skills/hypit/references/runtime.md+++ b/.agents/skills/hypit/references/runtime.md@@ -12,7 +12,7 @@ Source.Start from the Runtime Profile template in `docs/guide/runtime.md` (the repository does not ship acanonical `examples/*/hypit.runtime.json`). Bind one Endpoint per capability your Sources actually-reach: the picture and video models, speech, alignment, the media Provider and the local renderer.+reach: the picture and video models, Gemini VLM, speech, alignment, the media Provider and the local renderer.Read each Provider's README for the shape of its `config`, and reference credentials through the storerather than writing any secret into the file.@@ -51,10 +51,17 @@ gateway, so a typical profile looks like this:````defaultConcurrency` is the total HypiHub pool shared by its model capabilities. Tune it to the-quota behind the key; the Provider keeps each exact capability in its own lane. An explicit KIE+quota behind the key; image/video capabilities keep exact-model lanes while Gemini requests share+one conservative lane. An explicit KIEprofile remains valid when a project deliberately chooses `@hypit/provider-kie`, but it is not thedefault route.+An Author Source importing `@hypit/gemini` remains Provider-neutral. Bind those capabilities to+`@hypit/provider-hypihub` for HypiHub's upload-backed Gemini endpoint, or to+`@hypit/provider-vertex` for direct Vertex. Never make this Build-time choice with+`HYPIT_GEMINI_PROVIDER`; that environment switch belongs only to the reference-video preprocessing+observer, which runs outside a Build.+Provider-specific input limits are also part of preflight. For the KIE GPT Image 2 route, do notsubmit `4:3`, `3:4` or `4:5`; the route rejects those aspect ratios before upload or paid submission.diff --git a/package.json b/package.jsonindex 00639431..a2bbb537 100644--- a/package.json+++ b/package.json@@ -79,6 +79,7 @@"@hypit/film-studio": "workspace:*","@hypit/fonts-open": "workspace:*","@hypit/generation": "workspace:*",+ "@hypit/gemini": "workspace:*","@hypit/gpt-image": "workspace:*","@hypit/grok-imagine": "workspace:*","@hypit/hyperframes": "workspace:*",diff --git a/packages/gemini/README.md b/packages/gemini/README.mdnew file mode 100644index 00000000..e69db86e--- /dev/null+++ b/packages/gemini/README.md@@ -0,0 +1,17 @@+# `@hypit/gemini`++Provider-neutral Gemini text and multimodal requests for Hypit Author Source.++`<gemini:Generate>` accepts a system instruction, a text prompt and any number of image, video or+audio Artifact references. The model package owns the semantic request and exact model capability;+the selected Runtime Endpoint owns credentials, uploads and the service-specific wire format.++```xml+<gemini:Generate id="describe" model="gemini-3.7-flash"+ instruction={instruction} prompt={prompt}>+ <gemini:Reference media={reference-video}/>+</gemini:Generate>+```++The generated Text is published as `<id>.text`. Switching between+`@hypit/provider-hypihub` and `@hypit/provider-vertex` requires only a Runtime Profile change.diff --git a/packages/gemini/package.json b/packages/gemini/package.jsonnew file mode 100644index 00000000..700a48c4--- /dev/null+++ b/packages/gemini/package.json@@ -0,0 +1,17 @@+{+ "name": "@hypit/gemini",+ "version": "0.0.0-dev",+ "license": "SEE LICENSE IN LICENSE",+ "private": true,+ "type": "module",+ "exports": { ".": "./src/index.ts" },+ "hypit": { "activation": "./src/activation.ts" },+ "dependencies": {+ "@hypit/artifact": "workspace:*",+ "@hypit/component-kit": "workspace:*",+ "@hypit/elaborator": "workspace:*",+ "@hypit/markup": "workspace:*",+ "@hypit/protocol": "workspace:*",+ "@hypit/text": "workspace:*"+ }+}diff --git a/packages/gemini/src/activation.ts b/packages/gemini/src/activation.tsnew file mode 100644index 00000000..38eaa234--- /dev/null+++ b/packages/gemini/src/activation.ts@@ -0,0 +1,22 @@+import { createMarkupSurfaceHostFacet } from "@hypit/markup";++import {+ decodeGeminiGenerateSurface,+ geminiComponent,+ geminiManifest,+ geminiMarkupSurfaces,+ geminiModuleRef,+} from "./index.js";++export const hypitPackage = {+ format: "hypit.node-package@1" as const,+ modules: [{ manifest: geminiManifest }],+ components: [geminiComponent],+ hostFacets: [createMarkupSurfaceHostFacet({+ module: geminiModuleRef,+ declaration: geminiMarkupSurfaces[0],+ handler: decodeGeminiGenerateSurface,+ })],+};++export default hypitPackage;diff --git a/packages/gemini/src/component.ts b/packages/gemini/src/component.tsnew file mode 100644index 00000000..72534072--- /dev/null+++ b/packages/gemini/src/component.ts@@ -0,0 +1,71 @@+import type { ComponentPackage, ProducerHandlerContext } from "@hypit/component-kit";+import { canonicalize } from "@hypit/protocol";+import type { StoredValue } from "@hypit/protocol";+import type { Text } from "@hypit/text";+import { verifyText } from "@hypit/text";++import { geminiModels, geminiProducers, geminiTypes } from "./manifest.js";+import { sealGeminiRequest, verifyGeminiRequest } from "./request.js";+import type { GeminiRequest } from "./request.js";++function inline<T>(value: StoredValue | undefined, subject: string): T {+ if (value?.kind !== "inline") throw new Error(`${subject} must be inline`);+ return value.value as unknown as T;+}++function text(value: StoredValue | undefined, subject: string): string {+ const result = inline<Text>(value, subject);+ verifyText(result);+ return result.value;+}++export const geminiComponent = {+ validators: [+ { type: geminiTypes.request, handler: ({ value }) => verifyGeminiRequest(inline(value, "Gemini request")) },+ { type: geminiTypes.draft, handler: ({ value }) => verifyGeminiRequest(inline(value, "Gemini request draft")) },+ ],+ producers: [+ {+ producer: geminiProducers.start,+ handler: ({ inputs }: ProducerHandlerContext) => ({+ outputs: { draft: { kind: "inline", value: canonicalize(sealGeminiRequest({+ instruction: text(inputs.instruction?.value, "Gemini instruction"),+ prompt: text(inputs.prompt?.value, "Gemini prompt"),+ media: [],+ })) } },+ needs: {},+ }),+ },+ {+ producer: geminiProducers.bindMedia,+ handler: ({ inputs }: ProducerHandlerContext) => {+ const draft = inline<GeminiRequest>(inputs.draft?.value, "Gemini request draft");+ verifyGeminiRequest(draft);+ const artifact = inputs.media?.value;+ if (artifact?.kind !== "blob") throw new Error("Gemini media must be a Blob Artifact");+ return {+ outputs: { draft: { kind: "inline", value: canonicalize(sealGeminiRequest({+ ...draft, media: [...draft.media, { artifact }],+ })) } },+ needs: {},+ };+ },+ },+ {+ producer: geminiProducers.finalize,+ handler: ({ inputs }: ProducerHandlerContext) => {+ const draft = inline<GeminiRequest>(inputs.draft?.value, "Gemini request draft");+ verifyGeminiRequest(draft);+ return { outputs: { request: { kind: "inline", value: canonicalize(draft) } }, needs: {} };+ },+ },+ ...geminiModels.map((model) => ({+ producer: geminiProducers[model],+ handler: ({ inputs }: ProducerHandlerContext) => {+ const request = inline<GeminiRequest>(inputs.request?.value, `${model} request`);+ verifyGeminiRequest(request);+ return { outputs: {}, needs: { text: canonicalize(request) } };+ },+ })),+ ],+} satisfies ComponentPackage;diff --git a/packages/gemini/src/fragment.ts b/packages/gemini/src/fragment.tsnew file mode 100644index 00000000..a442c80b--- /dev/null+++ b/packages/gemini/src/fragment.ts@@ -0,0 +1,50 @@+import { artifactTypes } from "@hypit/artifact";+import { sealGraphFragment } from "@hypit/elaborator";+import type { FragmentOperation } from "@hypit/elaborator";+import { textTypes } from "@hypit/text";++import type { GeminiModel } from "./manifest.js";+import { geminiProducers, geminiTypes } from "./manifest.js";++const input = (name: string) => ({ kind: "fragment-input" as const, name });+const operation = (id: string) => ({ kind: "fragment-operation" as const, operation: id });++export function createGeminiFragment(model: GeminiModel, mediaCount: number) {+ if (!Number.isSafeInteger(mediaCount) || mediaCount < 0) throw new Error("Gemini media count is invalid");+ const inputs = [+ { name: "instruction", type: textTypes.text },+ { name: "prompt", type: textTypes.text },+ ...Array.from({ length: mediaCount }, (_item, index) => ({+ name: `media-${String(index + 1).padStart(4, "0")}`,+ type: artifactTypes.blob,+ })),+ ];+ const operations: FragmentOperation[] = [{+ id: "start-request", producer: geminiProducers.start,+ inputs: { instruction: input("instruction"), prompt: input("prompt") },+ result: { kind: "output" as const, name: "draft" },+ }];+ let draft = operation("start-request");+ for (let index = 0; index < mediaCount; index += 1) {+ const id = `bind-media-${String(index + 1).padStart(4, "0")}`;+ operations.push({+ id, producer: geminiProducers.bindMedia,+ inputs: { draft, media: input(`media-${String(index + 1).padStart(4, "0")}`) },+ result: { kind: "output" as const, name: "draft" },+ });+ draft = operation(id);+ }+ operations.push({+ id: "finalize-request", producer: geminiProducers.finalize, inputs: { draft },+ result: { kind: "output" as const, name: "request" },+ });+ operations.push({+ id: "generate", producer: geminiProducers[model], inputs: { request: operation("finalize-request") },+ result: { kind: "need" as const, name: "text" },+ });+ return sealGraphFragment({+ inputs,+ operations,+ exports: [{ name: "text", type: textTypes.text, root: operation("generate") }],+ });+}diff --git a/packages/gemini/src/index.ts b/packages/gemini/src/index.tsnew file mode 100644index 00000000..cd120cd1--- /dev/null+++ b/packages/gemini/src/index.ts@@ -0,0 +1,17 @@+export { geminiComponent } from "./component.js";+export { createGeminiFragment } from "./fragment.js";+export {+ geminiCapabilities,+ geminiDependency,+ geminiManifest,+ geminiMarkupSurfaces,+ geminiModels,+ geminiModuleRef,+ geminiProducers,+ geminiTypes,+} from "./manifest.js";+export type { GeminiModel } from "./manifest.js";+export { sealGeminiRequest, verifyGeminiRequest } from "./request.js";+export type { GeminiMediaPart, GeminiRequest } from "./request.js";+export type { GeminiGenerateInput, GeminiInlinePart } from "./parts.js";+export { decodeGeminiGenerateSurface } from "./surface.js";diff --git a/packages/gemini/src/manifest.ts b/packages/gemini/src/manifest.tsnew file mode 100644index 00000000..2fba2147--- /dev/null+++ b/packages/gemini/src/manifest.ts@@ -0,0 +1,96 @@+import { artifactDependency, artifactTypes } from "@hypit/artifact";+import type { CapabilityRef, ModuleManifest, ProducerRef, TypeRef } from "@hypit/protocol";+import { textDependency, textTypes } from "@hypit/text";++export const geminiModuleRef = { name: "@hypit/gemini", version: "1" } as const;++export const geminiModels = ["gemini-3.1-pro-preview", "gemini-3.7-flash"] as const;+export type GeminiModel = typeof geminiModels[number];++export const geminiTypes = {+ request: { module: geminiModuleRef, name: "GeminiRequest" },+ draft: { module: geminiModuleRef, name: "GeminiRequestDraft" },+} satisfies Record<string, TypeRef>;++export const geminiCapabilities = Object.fromEntries(geminiModels.map((model) => [model, {+ module: geminiModuleRef,+ name: model,+} satisfies CapabilityRef])) as Record<GeminiModel, CapabilityRef>;++export const geminiProducers = {+ start: { module: geminiModuleRef, name: "start-request" },+ bindMedia: { module: geminiModuleRef, name: "bind-media" },+ finalize: { module: geminiModuleRef, name: "finalize-request" },+ ...Object.fromEntries(geminiModels.map((model) => [model, {+ module: geminiModuleRef,+ name: `request-${model}`,+ } satisfies ProducerRef])),+} as { readonly start: ProducerRef; readonly bindMedia: ProducerRef; readonly finalize: ProducerRef } & Record<GeminiModel, ProducerRef>;++export const geminiMarkupSurfaces = [{+ name: "generate", tag: "Generate", mode: "structured",+ outputs: [geminiTypes.draft],+ vocabulary: {+ summary: "Asks one exact Gemini model to answer a Text prompt with optional image, video or audio references.",+ attributes: [+ { name: "id", kind: "identifier", required: true,+ summary: "Names this request and its generated Text." },+ { name: "model", kind: "literal", required: true, values: [...geminiModels],+ summary: "Selects the exact Gemini capability without choosing its Provider." },+ { name: "instruction", kind: "reference", required: true, accepts: [textTypes.text],+ summary: "Selects the system instruction Text." },+ { name: "prompt", kind: "reference", required: true, accepts: [textTypes.text],+ summary: "Selects the user prompt Text." },+ ],+ children: [{ tag: "Reference", cardinality: "many",+ summary: "Attaches one image, video or audio Artifact to the request in authored order.",+ attributes: [{ name: "media", kind: "reference", required: true, accepts: [artifactTypes.blob],+ summary: "Selects the media Artifact Gemini reads." }] }],+ ports: [{ name: "text", type: textTypes.text,+ summary: "The generated answer, addressed as `<id>.text`." }],+ example: `<gemini:Generate id="describe" model="gemini-3.7-flash" instruction={instruction} prompt={prompt}>+ <gemini:Reference media={reference-video}/>+</gemini:Generate>`,+ notes: [+ "The Source declares no API key, upload URL or Provider-specific file representation.",+ "Every Reference remains a graph edge; the selected Provider decides how its Artifact bytes travel.",+ ],+ },+}] as const;++export const geminiManifest: ModuleManifest = {+ format: "hypit.module@1",+ name: geminiModuleRef.name,+ version: geminiModuleRef.version,+ dependencies: [artifactDependency, textDependency],+ types: [{ name: geminiTypes.request.name }, { name: geminiTypes.draft.name }],+ capabilities: geminiModels.map((model) => ({ name: model, returns: textTypes.text })),+ producers: [+ {+ name: geminiProducers.start.name,+ inputs: [{ name: "instruction", type: textTypes.text }, { name: "prompt", type: textTypes.text }],+ outputs: [{ name: "draft", type: geminiTypes.draft }],+ needs: [],+ },+ {+ name: geminiProducers.bindMedia.name,+ inputs: [{ name: "draft", type: geminiTypes.draft }, { name: "media", type: artifactTypes.blob }],+ outputs: [{ name: "draft", type: geminiTypes.draft }],+ needs: [],+ },+ {+ name: geminiProducers.finalize.name,+ inputs: [{ name: "draft", type: geminiTypes.draft }],+ outputs: [{ name: "request", type: geminiTypes.request }],+ needs: [],+ },+ ...geminiModels.map((model) => ({+ name: geminiProducers[model].name,+ inputs: [{ name: "request", type: geminiTypes.request }],+ outputs: [],+ needs: [{ name: "text", capability: geminiCapabilities[model], returns: textTypes.text }],+ })),+ ],+};++export const geminiDependency = { module: geminiModuleRef } as const;diff --git a/packages/gemini/src/parts.ts b/packages/gemini/src/parts.tsnew file mode 100644index 00000000..23906715--- /dev/null+++ b/packages/gemini/src/parts.ts@@ -0,0 +1,9 @@+/** Provider-neutral inline parts used by embedded Gemini callers before a Runtime Artifact exists. */+export type GeminiInlinePart =+ | { readonly text: string; readonly inlineData?: never }+ | { readonly inlineData: { readonly mimeType: string; readonly data: string }; readonly text?: never };++export type GeminiGenerateInput = {+ readonly parts: readonly GeminiInlinePart[];+ readonly instruction: string;+};diff --git a/packages/gemini/src/request.ts b/packages/gemini/src/request.tsnew file mode 100644index 00000000..ac8870b9--- /dev/null+++ b/packages/gemini/src/request.ts@@ -0,0 +1,43 @@+import { canonicalize } from "@hypit/protocol";+import type { BlobRef } from "@hypit/protocol";++export type GeminiMediaPart = { readonly artifact: BlobRef };+export type GeminiRequest = {+ readonly instruction: string;+ readonly prompt: string;+ readonly media: readonly GeminiMediaPart[];+};++function assert(condition: unknown, message: string): asserts condition {+ if (!condition) throw new Error(message);+}++function object(value: unknown, subject: string): Record<string, unknown> {+ assert(value !== null && typeof value === "object" && !Array.isArray(value), `${subject} must be an object`);+ return value as Record<string, unknown>;+}++export function verifyGeminiRequest(value: unknown): asserts value is GeminiRequest {+ const request = object(value, "Gemini request");+ assert(typeof request.instruction === "string" && request.instruction.trim().length > 0,+ "Gemini request instruction must be non-empty text");+ assert(typeof request.prompt === "string" && request.prompt.trim().length > 0,+ "Gemini request prompt must be non-empty text");+ assert(Array.isArray(request.media), "Gemini request media must be an array");+ for (const [index, raw] of request.media.entries()) {+ const item = object(raw, `Gemini request media[${index}]`);+ const artifact = object(item.artifact, `Gemini request media[${index}].artifact`);+ assert(artifact.kind === "blob" && typeof artifact.digest === "string"+ && typeof artifact.size === "number" && Number.isSafeInteger(artifact.size)+ && typeof artifact.mediaType === "string",+ `Gemini request media[${index}].artifact must be a BlobRef`);+ assert(/^(?:image|video|audio)\//u.test(artifact.mediaType),+ `Gemini request media[${index}] must be image, video or audio media`);+ }+}++export function sealGeminiRequest(value: GeminiRequest): GeminiRequest {+ const result = canonicalize(value) as unknown as GeminiRequest;+ verifyGeminiRequest(result);+ return result;+}diff --git a/packages/gemini/src/surface.ts b/packages/gemini/src/surface.tsnew file mode 100644index 00000000..c61fde38--- /dev/null+++ b/packages/gemini/src/surface.ts@@ -0,0 +1,73 @@+import { artifactTypes } from "@hypit/artifact";+import type {+ MarkupAttributeValue,+ StructuredElement,+ StructuredSurfaceHandler,+ SurfaceResolvedReference,+} from "@hypit/markup";+import type { TypeRef } from "@hypit/protocol";+import { textTypes } from "@hypit/text";++import { createGeminiFragment } from "./fragment.js";+import { geminiModels } from "./manifest.js";+import type { GeminiModel } from "./manifest.js";++function assert(condition: unknown, message: string): asserts condition { if (!condition) throw new Error(message); }+function localName(value: string): string { return value.includes(":") ? value.slice(value.lastIndexOf(":") + 1) : value; }+function sameType(left: TypeRef, right: TypeRef): boolean {+ return left.module.name === right.module.name && left.module.version === right.module.version && left.name === right.name;+}+function reference(value: MarkupAttributeValue | undefined, subject: string,+ expected: TypeRef, resolveReference: (path: string) => SurfaceResolvedReference | undefined): SurfaceResolvedReference {+ assert(typeof value === "object" && value.kind === "reference", `${subject} must be a reference`);+ const resolved = resolveReference(value.path);+ assert(resolved !== undefined && sameType(resolved.type, expected), `${subject} has the wrong type`);+ return resolved;+}+function literal(element: StructuredElement, name: string): string {+ const value = element.attributes[name];+ assert(typeof value === "string" && value.trim().length > 0, `${element.name}.${name} must be non-empty text`);+ return value.trim();+}++export const decodeGeminiGenerateSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {+ const allowed = ["id", "instruction", "model", "prompt"];+ const unknown = Object.keys(element.attributes).find((name) => !allowed.includes(name));+ assert(unknown === undefined, `${element.name} does not accept ${unknown}`);+ const id = literal(element, "id");+ const model = literal(element, "model");+ assert(geminiModels.includes(model as GeminiModel), `${element.name}.model is unsupported`);+ const instruction = reference(element.attributes.instruction, `${element.name}.instruction`, textTypes.text, resolveReference);+ const prompt = reference(element.attributes.prompt, `${element.name}.prompt`, textTypes.text, resolveReference);+ const media: SurfaceResolvedReference[] = [];+ for (const child of element.children) {+ if (child.kind === "text") {+ assert(child.value.trim().length === 0, `${element.name} accepts only Reference children`);+ continue;+ }+ assert(localName(child.name) === "Reference", `${element.name} accepts only Reference children`);+ assert(Object.keys(child.attributes).join(",") === "media", `${child.name} requires only media`);+ assert(!child.children.some((item) => item.kind === "element" || item.value.trim().length > 0), `${child.name} must be empty`);+ const resolved = reference(child.attributes.media, `${child.name}.media`, artifactTypes.blob, resolveReference);+ if (resolved.record?.value.kind === "blob") {+ assert(/^(?:image|video|audio)\//u.test(resolved.record.value.mediaType), `${child.name}.media must be image, video or audio`);+ }+ media.push(resolved);+ }+ const fragment = createGeminiFragment(model as GeminiModel, media.length);+ return {+ records: [],+ fragments: [fragment],+ components: [{+ id,+ fragment: fragment.id,+ inputs: {+ instruction: instruction.ref,+ prompt: prompt.ref,+ ...Object.fromEntries(media.map((item, index) => [`media-${String(index + 1).padStart(4, "0")}`, item.ref])),+ },+ outputs: { text: `${id}.text` },+ range: element.range,+ }],+ };+};diff --git a/packages/gemini/test/surface.test.ts b/packages/gemini/test/surface.test.tsnew file mode 100644index 00000000..6e8d61fc--- /dev/null+++ b/packages/gemini/test/surface.test.ts@@ -0,0 +1,39 @@+import assert from "node:assert/strict";+import test from "node:test";+import { fixtureDigest } from "../../../test/fixture-digest.js";++import { decodeGeminiGenerateSurface, geminiCapabilities } from "@hypit/gemini";+import { artifactTypes } from "@hypit/artifact";+import { textTypes } from "@hypit/text";++test("Gemini Surface builds one provider-neutral need with ordered media edges", async () => {+ const fragmentResult = await decodeGeminiGenerateSurface({+ sourceName: "main.svml",+ element: {+ kind: "element", name: "gemini:Generate",+ attributes: {+ id: "read", model: "gemini-3.7-flash",+ instruction: { kind: "reference", path: "instruction" },+ prompt: { kind: "reference", path: "prompt" },+ },+ children: [{+ kind: "element", name: "gemini:Reference",+ attributes: { media: { kind: "reference", path: "video" } }, children: [], range: { start: 20, end: 40 },+ }], range: { start: 0, end: 50 },+ },+ resolveAsset: () => { throw new Error("not used"); },+ resolveReference: (path) => path === "video" ? {+ path, ref: { kind: "record", id: "video" }, type: artifactTypes.blob,+ record: { id: "video", type: artifactTypes.blob, value: {+ kind: "blob", digest: fixtureDigest("gemini-video"), size: 10, mediaType: "video/mp4",+ } },+ } : {+ path, ref: { kind: "record", id: path }, type: textTypes.text,+ record: { id: path, type: textTypes.text, value: { kind: "inline", value: { value: path } } },+ },+ });+ assert.equal(fragmentResult.fragments.length, 1);+ const operation = fragmentResult.fragments[0]!.operations.find((item) => item.id === "generate");+ assert.deepEqual(operation?.producer, { module: geminiCapabilities["gemini-3.7-flash"].module, name: "request-gemini-3.7-flash" });+ assert.deepEqual(fragmentResult.components[0]?.outputs, { text: "read.text" });+});diff --git a/packages/media/README.md b/packages/media/README.mdindex 7ea4f5fe..be864467 100644--- a/packages/media/README.md+++ b/packages/media/README.md@@ -2,15 +2,16 @@Author-facing media declarations for the neutral `@hypit/media` contract module.-`<media:Image>`, `<media:Audio>` and `<media:Font>` request bytes through the Host-owned source-asset-capability. Image and Audio emit one content-addressed `BlobArtifact`; Font wraps the same immutable+`<media:Image>`, `<media:Audio>`, `<media:Video>` and `<media:Font>` request bytes through the Host-owned source-asset+capability. Image, Audio and Video emit one content-addressed `BlobArtifact`; Font wraps the same immutablebytes with exact weight/style metadata as a one-source `FontArtifactRef`. A logical font face mayalso contain several exact Unicode-range sources when another package, such as`@hypit/fonts-open`, owns a sharded installed font. `FontStackRef` is the generic ordered valuefor one primary face followed by exact glyph fallbacks. Media declarations do not open files themselves,inspect media, call a Provider or promote declared audio to speech. The consuming author packagedecides whether an audio artifact is a voice reference, soundtrack, evidence source or something-else, and which visual component uses an exact Font.+else, whether a video is a generation reference or another input, and which visual component uses+an exact Font.The same module owns the deliberately narrow `SynchronizedMedia` contract used after technicalnormalization. It contains one common `frameRate`/`frameCount`, an optional visual Artifact with itsdiff --git a/packages/media/src/activation.ts b/packages/media/src/activation.tsindex 7d5604f0..13261311 100644--- a/packages/media/src/activation.ts+++ b/packages/media/src/activation.ts@@ -1,7 +1,7 @@import { createMarkupSurfaceHostFacet } from "@hypit/markup";import {- decodeMediaAudioSurface, decodeMediaFontSurface, decodeMediaImageSurface, mediaComponent,+ decodeMediaAudioSurface, decodeMediaFontSurface, decodeMediaImageSurface, decodeMediaVideoSurface, mediaComponent,mediaManifest,mediaModuleRef,mediaMarkupSurfaces,@@ -22,6 +22,11 @@ export const hypitPackage = {declaration: mediaMarkupSurfaces.find((item) => item.name === "audio")!,handler: decodeMediaAudioSurface,}),+ createMarkupSurfaceHostFacet({+ module: mediaModuleRef,+ declaration: mediaMarkupSurfaces.find((item) => item.name === "video")!,+ handler: decodeMediaVideoSurface,+ }),createMarkupSurfaceHostFacet({module: mediaModuleRef,declaration: mediaMarkupSurfaces.find((item) => item.name === "font")!,diff --git a/packages/media/src/index.ts b/packages/media/src/index.tsindex 108b5dc1..5bbfd30b 100644--- a/packages/media/src/index.ts+++ b/packages/media/src/index.ts@@ -2,6 +2,7 @@ export {decodeMediaAudioSurface,decodeMediaFontSurface,decodeMediaImageSurface,+ decodeMediaVideoSurface,} from "./surface.js";export { mediaComponent } from "./component.js";export { sealMediaInspection, sealMediaStreamSelection, sealMuxedMedia, sealRenderedVisual, sealSynchronizedMedia, sealTimelineAudio, synchronizedMediaSampleFrames, verifyMediaInspection, verifyMediaStreamSelection, verifyMuxedMedia, verifyRenderedVisual, verifySynchronizedMedia, verifyTimelineAudio } from "./identity.js";diff --git a/packages/media/src/manifest.ts b/packages/media/src/manifest.tsindex 2a05dcc9..e3a33f24 100644--- a/packages/media/src/manifest.ts+++ b/packages/media/src/manifest.ts@@ -56,6 +56,27 @@ export const mediaMarkupSurfaces = [],},},+ {+ name: "video", tag: "Video", mode: "structured", outputs: [artifactTypes.blob],+ vocabulary: {+ summary: "Requests one authored video file from the Host and publishes it as a content-addressed Artifact.",+ attributes: [+ { name: "id", kind: "identifier", required: true,+ summary: "Names the Artifact Record this element publishes." },+ { name: "src", kind: "literal", required: true,+ summary: "Points at the video file, resolved by the Host against the source that declares it." },+ { name: "media-type", kind: "literal", required: false,+ summary: "States the video media type when the src extension does not name one." },+ ],+ example: `<media:Video id="reference" src="./assets/reference.mp4"/>`,+ notes: [+ "The element is empty; it accepts no children and no text.",+ "`.m4v`, `.mov`, `.mp4` and `.webm` name their own media type; any other file needs `media-type`.",+ "A written `media-type` must begin with `video/`, and the resolved bytes must arrive as a video Artifact.",+ "The Artifact is published under the bare `id`, and the consuming package decides what the video means.",+ ],+ },+ },{name: "font", tag: "Font", mode: "structured", outputs: [mediaTypes.fontArtifact],vocabulary: {diff --git a/packages/media/src/surface.ts b/packages/media/src/surface.tsindex 40f2f987..25651998 100644--- a/packages/media/src/surface.ts+++ b/packages/media/src/surface.ts@@ -27,6 +27,13 @@ const AUDIO_MEDIA_TYPES = new Map([[".wav", "audio/wav"],]);+const VIDEO_MEDIA_TYPES = new Map([+ [".m4v", "video/x-m4v"],+ [".mov", "video/quicktime"],+ [".mp4", "video/mp4"],+ [".webm", "video/webm"],+]);+const FONT_MEDIA_TYPES = new Map([[".otf", "font/otf"],[".ttf", "font/ttf"],@@ -51,7 +58,7 @@ function assertChildrenEmpty(element: StructuredElement): void {function mediaTypeFor(element: StructuredElement,source: string,- kind: "image" | "audio" | "font",+ kind: "image" | "audio" | "video" | "font",known: ReadonlyMap<string, string>,): string {const explicit = element.attributes["media-type"];@@ -74,7 +81,7 @@ function mediaTypeFor(async function decodeMediaAssetSurface(element: StructuredElement,resolveAsset: Parameters<StructuredSurfaceHandler>[0]["resolveAsset"],- kind: "image" | "audio",+ kind: "image" | "audio" | "video",known: ReadonlyMap<string, string>,) {const names = Object.keys(element.attributes).sort();@@ -107,6 +114,9 @@ export const decodeMediaImageSurface: StructuredSurfaceHandler = async ({ elemenexport const decodeMediaAudioSurface: StructuredSurfaceHandler = async ({ element, resolveAsset }) =>await decodeMediaAssetSurface(element, resolveAsset, "audio", AUDIO_MEDIA_TYPES);+export const decodeMediaVideoSurface: StructuredSurfaceHandler = async ({ element, resolveAsset }) =>+ await decodeMediaAssetSurface(element, resolveAsset, "video", VIDEO_MEDIA_TYPES);+export const decodeMediaFontSurface: StructuredSurfaceHandler = async ({ element, resolveAsset }) => {const names = Object.keys(element.attributes).sort();if (names.join(",") !== "id,src,style,weight" && names.join(",") !== "id,media-type,src,style,weight") {diff --git a/packages/media/test/asset-surface.test.ts b/packages/media/test/asset-surface.test.tsnew file mode 100644index 00000000..63ca0653--- /dev/null+++ b/packages/media/test/asset-surface.test.ts@@ -0,0 +1,62 @@+import assert from "node:assert/strict";+import test from "node:test";+import { fixtureDigest } from "../../../test/fixture-digest.js";++import {+ decodeMediaAudioSurface,+ decodeMediaImageSurface,+ decodeMediaVideoSurface,+} from "@hypit/media";++const range = { start: 0, end: 80 };++for (const fixture of [+ { label: "Image", decode: decodeMediaImageSurface, src: "./reference.png", mediaType: "image/png" },+ { label: "Audio", decode: decodeMediaAudioSurface, src: "./reference.wav", mediaType: "audio/wav" },+ { label: "Video", decode: decodeMediaVideoSurface, src: "./reference.mp4", mediaType: "video/mp4" },+] as const) {+ test(`${fixture.label} Surface publishes the resolved BlobArtifact`, async () => {+ const artifact = {+ kind: "blob" as const,+ digest: fixtureDigest(`media:${fixture.label.toLowerCase()}`),+ size: 2_048,+ mediaType: fixture.mediaType,+ };+ const requests: unknown[] = [];+ const result = await fixture.decode({+ sourceName: "main.svml",+ element: {+ kind: "element",+ name: `media:${fixture.label}`,+ attributes: { id: "reference", src: fixture.src },+ children: [],+ range,+ },+ resolveReference: () => undefined,+ resolveAsset: (request) => {+ requests.push(request);+ return { artifact };+ },+ });++ assert.deepEqual(requests, [{ from: fixture.src, mediaType: fixture.mediaType, range }]);+ assert.deepEqual(result.records, [{ id: "reference", type: {+ module: { name: "@hypit/artifact", version: "1" }, name: "BlobArtifact",+ }, value: artifact, range }]);+ });+}++test("Video Surface rejects ambiguous extensions and non-video resolved artifacts", async () => {+ const decode = (attributes: Readonly<Record<string, string>>, mediaType = "video/mp4") => decodeMediaVideoSurface({+ sourceName: "main.svml",+ element: { kind: "element", name: "media:Video", attributes, children: [], range },+ resolveReference: () => undefined,+ resolveAsset: (request) => ({ artifact: {+ kind: "blob", digest: fixtureDigest("media:video-invalid"), size: 10, mediaType,+ } }),+ });++ await assert.rejects(async () => await decode({ id: "video", src: "./clip.bin" }), /known video extension/u);+ await assert.rejects(async () => await decode({ id: "video", src: "./clip.mp4" }, "image/png"), /video artifact/u);+ await assert.rejects(async () => await decode({ id: "video", src: "./clip.bin", "media-type": "image/png" }), /video media type/u);+});diff --git a/packages/provider-hypihub/README.md b/packages/provider-hypihub/README.mdindex 656f1a49..fc3aeda8 100644--- a/packages/provider-hypihub/README.md+++ b/packages/provider-hypihub/README.md@@ -30,8 +30,9 @@ Runtime Profile example:}```-Gemini VLM callers can use the exported `createHypiHubGeminiGenerator` without changing their-provider-facing code. Set `HYPIHUB_API_KEY` (and optionally `HYPIHUB_BASE_URL`; either the origin or+Gemini VLM is also exposed as the provider-neutral `@hypit/gemini` Runtime capability, so an Author+Source can select the exact model while the Runtime chooses HypiHub or Vertex. Existing embedded+callers can still use the exported `createHypiHubGeminiGenerator`. Set `HYPIHUB_API_KEY` (and optionally `HYPIHUB_BASE_URL`; either the origin oran existing `/v1`/`/v1beta` base is accepted) only when choosing HypiHub. The Runtime Provider alsoaccepts the origin or either versioned base and normalizes it to `/v1`; missing or insufficient usercredentials should be resolved at [hypit.ai](https://hypit.ai). Referenced image, audio and videodiff --git a/packages/provider-hypihub/package.json b/packages/provider-hypihub/package.jsonindex 138fbb4d..fc837de1 100644--- a/packages/provider-hypihub/package.json+++ b/packages/provider-hypihub/package.json@@ -8,10 +8,12 @@"hypit": { "activation": "./src/activation.ts" },"dependencies": {"@hypit/endpoint-kit": "workspace:*",+ "@hypit/gemini": "workspace:*","@hypit/generation": "workspace:*","@hypit/protocol": "workspace:*","@hypit/runtime": "workspace:*",- "@hypit/runtime-kit": "workspace:*"+ "@hypit/runtime-kit": "workspace:*",+ "@hypit/text": "workspace:*"},"devDependencies": {"@hypit/driver-node": "workspace:*",diff --git a/packages/provider-hypihub/src/gemini.ts b/packages/provider-hypihub/src/gemini.tsindex dfe67f87..fe880c50 100644--- a/packages/provider-hypihub/src/gemini.ts+++ b/packages/provider-hypihub/src/gemini.ts@@ -1,6 +1,6 @@-export type HypiHubGeminiPart =- | { readonly text: string }- | { readonly inlineData: { readonly mimeType: string; readonly data: string } }+import type { GeminiInlinePart } from "@hypit/gemini";++export type HypiHubGeminiPart = GeminiInlinePart| { readonly fileData: { readonly mimeType?: string; readonly fileUri: string } };export type HypiHubGeminiGeneratorOptions = {@@ -8,6 +8,8 @@ export type HypiHubGeminiGeneratorOptions = {readonly model?: string;readonly baseUrl?: string;readonly requestTimeoutMs?: number;+ readonly maxRateLimitRetries?: number;+ readonly rateLimitRetryDelayMs?: number;readonly fetch?: typeof globalThis.fetch;};@@ -20,6 +22,22 @@ function assert(condition: unknown, message: string): asserts condition {if (!condition) throw new Error(message);}+function positiveInteger(value: number, subject: string): number {+ assert(Number.isSafeInteger(value) && value > 0, `${subject} must be a positive integer`);+ return value;+}++function retryDelay(response: Response, fallback: number, attempt: number): number {+ const retryAfter = response.headers.get("retry-after")?.trim();+ if (retryAfter !== undefined && retryAfter !== "") {+ const seconds = Number(retryAfter);+ if (Number.isFinite(seconds) && seconds >= 0) return Math.max(250, Math.round(seconds * 1_000));+ const date = Date.parse(retryAfter);+ if (Number.isFinite(date)) return Math.max(250, date - Date.now());+ }+ return fallback * (attempt + 1);+}+function partValue(part: HypiHubGeminiPart, uploaded: Map<string, Promise<string>>,upload: (mimeType: string, data: string) => Promise<string>): Promise<Record<string, unknown>> {if ("text" in part) return Promise.resolve({ text: part.text });@@ -54,6 +72,11 @@ export function createHypiHubGeminiGenerator(options: HypiHubGeminiGeneratorOpti.replace(/\/$/u, "");const model = options.model?.trim() || "gemini-3.1-pro-preview";const timeout = options.requestTimeoutMs ?? 120_000;+ const maxRateLimitRetries = options.maxRateLimitRetries ?? 3;+ assert(Number.isSafeInteger(maxRateLimitRetries) && maxRateLimitRetries >= 0,+ "HypiHub Gemini maxRateLimitRetries must be a non-negative integer");+ const rateLimitRetryDelayMs = positiveInteger(options.rateLimitRetryDelayMs ?? 2_000,+ "HypiHub Gemini rateLimitRetryDelayMs");const fetcher = options.fetch ?? globalThis.fetch;return async (input: HypiHubGeminiGenerateInput): Promise<string> => {@@ -84,17 +107,24 @@ export function createHypiHubGeminiGenerator(options: HypiHubGeminiGeneratorOptireturn url;};const parts = await Promise.all(input.parts.map((part) => partValue(part, uploaded, upload)));- const response = await fetcher(`${baseUrl}/v1beta/models/${encodeURIComponent(model)}:generateContent`, {- method: "POST",- signal: controller.signal,- headers: { "x-goog-api-key": apiKey, "content-type": "application/json" },- body: JSON.stringify({- systemInstruction: { parts: [{ text: input.instruction }] },- contents: [{ role: "user", parts }],- generationConfig: { temperature: 1, responseMimeType: "text/plain" },- }),- });- const text = await response.text();+ let response: Response | undefined;+ let text = "";+ for (let attempt = 0; attempt <= maxRateLimitRetries; attempt += 1) {+ response = await fetcher(`${baseUrl}/v1beta/models/${encodeURIComponent(model)}:generateContent`, {+ method: "POST",+ signal: controller.signal,+ headers: { "x-goog-api-key": apiKey, "content-type": "application/json" },+ body: JSON.stringify({+ systemInstruction: { parts: [{ text: input.instruction }] },+ contents: [{ role: "user", parts }],+ generationConfig: { temperature: 1, responseMimeType: "text/plain" },+ }),+ });+ text = await response.text();+ if (response.status !== 429 || attempt === maxRateLimitRetries) break;+ await new Promise((resolve) => setTimeout(resolve, retryDelay(response!, rateLimitRetryDelayMs, attempt)));+ }+ assert(response !== undefined, "HypiHub Gemini made no request");if (!response.ok) {const message = `HypiHub Gemini returned HTTP ${response.status}: ${text.slice(0, 300)}`;if (response.status === 401 || response.status === 403 || response.status === 404) {diff --git a/packages/provider-hypihub/src/provider.ts b/packages/provider-hypihub/src/provider.tsindex b8d158cf..949616de 100644--- a/packages/provider-hypihub/src/provider.ts+++ b/packages/provider-hypihub/src/provider.ts@@ -1,9 +1,14 @@import type { AsyncEndpoint, EndpointFulfillment, EndpointInvocationContext, EndpointPollContext, EndpointStartContext, EndpointOutcome, ImmediateEndpointHandler } from "@hypit/endpoint-kit";import { defineEndpointPackage, wakeAfter } from "@hypit/endpoint-kit";+import { geminiCapabilities, geminiModels, verifyGeminiRequest } from "@hypit/gemini";+import type { GeminiRequest } from "@hypit/gemini";import { canonicalize } from "@hypit/protocol";import type { BlobRef, CapabilityRef } from "@hypit/protocol";import { credentialRef } from "@hypit/runtime";import type { CredentialRef, ArtifactStore } from "@hypit/runtime";+import { sealText } from "@hypit/text";+import { textTypes } from "@hypit/text";+import { createHypiHubGeminiGenerator } from "./gemini.js";import { hypiHubRouteForCapability, hypiHubRoutes } from "./routes.js";export const hypiHubProviderModuleRef = { name: "@hypit/provider-hypihub", version: "1" } as const;@@ -52,6 +57,12 @@ function failure(error: unknown): EndpointOutcome {const message = guidedMessage(error);return { status: "failed", failure: { code: "HYPIHUB_ERROR", message } };}++function inlineGeminiRequest(context: EndpointInvocationContext): GeminiRequest {+ const request = context.need.constraints as unknown;+ verifyGeminiRequest(request);+ return request;+}function jobId(value: Record<string, unknown>): string {const id = value.id ?? value.job_id;assert(typeof id === "string" && id.length > 0, "HypiHub response has no job id");@@ -219,10 +230,31 @@ export function createHypiHubProvider(options: CreateHypiHubProviderOptions = {}throw new Error(guidedMessage(error), { cause: error });}};+ const geminiEndpoint: ImmediateEndpointHandler = async (context) => {+ const request = inlineGeminiRequest(context);+ const generate = createHypiHubGeminiGenerator({+ apiKey: credential(context),+ model: context.need.capability.name,+ ...(options.baseUrl === undefined ? {} : { baseUrl: options.baseUrl }),+ ...(options.requestTimeoutMs === undefined ? {} : { requestTimeoutMs: options.requestTimeoutMs }),+ ...(options.fetch === undefined ? {} : { fetch: options.fetch }),+ maxRateLimitRetries: 5,+ rateLimitRetryDelayMs: 3_000,+ });+ const parts: Array<{ text: string } | { inlineData: { mimeType: string; data: string } }> = [{ text: request.prompt }];+ for (const item of request.media) {+ const bytes = await context.artifacts.get(item.artifact.digest);+ assert(bytes !== undefined, `HypiHub Gemini reference artifact ${item.artifact.digest} is unavailable`);+ parts.push({ inlineData: { mimeType: item.artifact.mediaType, data: Buffer.from(bytes).toString("base64") } });+ }+ const value = await generate({ parts, instruction: request.instruction });+ return { value: { kind: "inline", value: canonicalize(sealText(value)) } };+ };return defineEndpointPackage({module: hypiHubProviderModuleRef, facet: "gateway", instance: options.instance ?? "hypihub.default", pool: options.pool ?? options.instance ?? "hypihub.default",credentials: { apiKey: options.apiKey ?? credentialRef("env", "HYPIHUB_API_KEY") }, credentialInputs: { apiKey: { label: "HypiHub API key" } }, defaultConcurrency: options.defaultConcurrency ?? 4,- capabilities: hypiHubRoutes+ capabilities: [+ ...hypiHubRoutes// HypiHub keys do not necessarily include the MiMo audio models. Keep// official Xiaomi MiMo as the safe default; opting into HypiHub audio is// an explicit Runtime decision for a key that actually has those models.@@ -230,5 +262,14 @@ export function createHypiHubProvider(options: CreateHypiHubProviderOptions = {}.map((route) => route.media === "audio"? { capability: route.capability, returns: route.returns, lifecycle: "immediate" as const, handler: audioEndpoint, lane: route.capability.name }: { capability: route.capability, returns: route.returns, lifecycle: "asynchronous" as const, endpoint: asyncEndpoint, lane: route.capability.name }),+ ...geminiModels.map((model) => ({+ capability: geminiCapabilities[model],+ returns: textTypes.text,+ lifecycle: "immediate" as const,+ handler: geminiEndpoint,+ lane: "gemini",+ maxConcurrency: 1,+ })),+ ],});}diff --git a/packages/provider-hypihub/test/gemini.test.ts b/packages/provider-hypihub/test/gemini.test.tsindex c1d07e7a..f51381ee 100644--- a/packages/provider-hypihub/test/gemini.test.ts+++ b/packages/provider-hypihub/test/gemini.test.ts@@ -46,3 +46,29 @@ test("HypiHub Gemini points unavailable models to HypiHub", async () => {await assert.rejects(() => generate({ instruction: "x", parts: [{ text: "x" }] }),/get a HypiHub key with this model enabled at https:\/\/hypit\.ai/iu);});++test("HypiHub Gemini retries upstream rate limits without reuploading files", async () => {+ let uploads = 0;+ let generations = 0;+ const generate = createHypiHubGeminiGenerator({+ apiKey: "test-key",+ maxRateLimitRetries: 2,+ rateLimitRetryDelayMs: 1,+ fetch: async (input) => {+ const url = String(input);+ if (url.endsWith("/v1/files")) {+ uploads += 1;+ return Response.json({ url: "https://hypit.ai/files/ref" });+ }+ generations += 1;+ return generations < 3+ ? new Response("rate limited", { status: 429, headers: { "retry-after": "0" } })+ : Response.json({ candidates: [{ content: { parts: [{ text: "OK" }] } }] });+ },+ });+ assert.equal(await generate({ instruction: "x", parts: [+ { text: "x" }, { inlineData: { mimeType: "image/png", data: "aW1hZ2U=" } },+ ] }), "OK");+ assert.equal(uploads, 1);+ assert.equal(generations, 3);+});diff --git a/packages/provider-hypihub/test/provider.test.ts b/packages/provider-hypihub/test/provider.test.tsindex f273b6b9..4162fcbb 100644--- a/packages/provider-hypihub/test/provider.test.ts+++ b/packages/provider-hypihub/test/provider.test.ts@@ -3,8 +3,10 @@ import test from "node:test";import { EndpointRegistry, MemoryArtifactStore } from "@hypit/driver-node";import type { AsyncEndpoint } from "@hypit/endpoint-kit";+import { geminiCapabilities, sealGeminiRequest } from "@hypit/gemini";import type { CanonicalValue, Need } from "@hypit/protocol";import { mimoTtsEndpoints } from "@hypit/mimo-tts";+import { textTypes } from "@hypit/text";import { sealSeedanceRequest, seedanceEndpoints } from "@hypit/seedance";import { createHypiHubProvider } from "../src/provider.js";@@ -114,3 +116,51 @@ test("HypiHub uploads referenced Artifacts once, submits their HTTPS URLs, and passert.equal(completed.status, "completed");assert.equal(calls.length, 6);});++test("HypiHub fulfills Gemini through the Runtime endpoint and uploads every media Artifact", async () => {+ const artifacts = new MemoryArtifactStore();+ const image = await artifacts.put(new Uint8Array([1, 2, 3]), "image/png");+ const video = await artifacts.put(new Uint8Array([4, 5, 6]), "video/mp4");+ const request: Need = {+ id: "need:hypihub-gemini",+ capability: geminiCapabilities["gemini-3.7-flash"],+ returns: textTypes.text,+ constraints: sealGeminiRequest({+ instruction: "Answer briefly.", prompt: "Inspect both references.",+ media: [{ artifact: image }, { artifact: video }],+ }) as unknown as CanonicalValue,+ result: "record:hypihub-gemini",+ };+ const uploads: string[] = [];+ let generationBody: Record<string, unknown> | undefined;+ const registry = new EndpointRegistry();+ await createHypiHubProvider({ fetch: async (input, init) => {+ const url = String(input);+ if (url.endsWith("/v1/files")) {+ assert.ok(init?.body instanceof FormData);+ const file = init.body.get("file");+ assert.ok(file instanceof File);+ uploads.push(file.type);+ return Response.json({ url: `https://hypit.ai/files/${uploads.length}` });+ }+ if (url.includes(":generateContent")) {+ generationBody = JSON.parse(String(init?.body)) as Record<string, unknown>;+ return Response.json({ candidates: [{ content: { parts: [{ text: "provider works" }] } }] });+ }+ throw new Error(`Unexpected URL ${url}`);+ } }).install(registry);+ const resolution = registry.resolve(request);+ assert.equal(resolution.status, "resolved");+ assert.equal(resolution.registration.kind, "immediate");+ const result = await resolution.registration.handler({+ command: { kind: "fulfill-need", id: "command:hypihub-gemini", need: request },+ need: request, artifacts, credentials: { apiKey: { secret: "test-key" } },+ });+ assert.deepEqual(result.value, { kind: "inline", value: { value: "provider works" } });+ assert.deepEqual(uploads, ["image/png", "video/mp4"]);+ assert.deepEqual((generationBody?.contents as readonly unknown[]), [{ role: "user", parts: [+ { text: "Inspect both references." },+ { fileData: { mimeType: "image/png", fileUri: "https://hypit.ai/files/1" } },+ { fileData: { mimeType: "video/mp4", fileUri: "https://hypit.ai/files/2" } },+ ] }]);+});diff --git a/packages/provider-vertex/README.md b/packages/provider-vertex/README.mdindex 20a12c11..398de16c 100644--- a/packages/provider-vertex/README.md+++ b/packages/provider-vertex/README.md@@ -1,8 +1,9 @@# `@hypit/provider-vertex`-Vertex Gemini backend for callers that need text, image or video inline-data input. The package owns-the Google GenAI client and credential parsing; callers only select the model and provide a generator-input.+Vertex Gemini Runtime Provider and embedded backend for callers that need text, image or video input.+It fulfills the same provider-neutral `@hypit/gemini` capabilities as HypiHub, so Runtime Profiles+choose the backend without changing Author Source. The package owns the Google GenAI client and+credential parsing.```tsimport { createVertexGeminiGenerator } from "@hypit/provider-vertex";diff --git a/packages/provider-vertex/package.json b/packages/provider-vertex/package.jsonindex 856b3673..82c2fcb4 100644--- a/packages/provider-vertex/package.json+++ b/packages/provider-vertex/package.json@@ -5,7 +5,17 @@"private": true,"type": "module","exports": { ".": "./src/index.ts" },+ "hypit": { "activation": "./src/activation.ts" },"dependencies": {- "@google/genai": "1.52.0"+ "@google/genai": "1.52.0",+ "@hypit/endpoint-kit": "workspace:*",+ "@hypit/gemini": "workspace:*",+ "@hypit/protocol": "workspace:*",+ "@hypit/runtime": "workspace:*",+ "@hypit/runtime-kit": "workspace:*",+ "@hypit/text": "workspace:*"+ },+ "devDependencies": {+ "@hypit/driver-node": "workspace:*"}}diff --git a/packages/provider-vertex/src/activation.ts b/packages/provider-vertex/src/activation.tsnew file mode 100644index 00000000..5cd04c3e--- /dev/null+++ b/packages/provider-vertex/src/activation.ts@@ -0,0 +1,38 @@+import {+ createRuntimeEndpointAdapterFacet,+ runtimeConfigCredentialRef,+ runtimeConfigExact,+ runtimeConfigObject,+ runtimeConfigPositiveInteger,+ runtimeConfigString,+} from "@hypit/runtime-kit";++import { createVertexProvider } from "./provider.js";++const adapter = createRuntimeEndpointAdapterFacet({+ use: "@hypit/provider-vertex",+ activate(context) {+ if (context.pool === undefined) throw new Error("Vertex Provider Pool is required");+ const config = runtimeConfigObject(context.config, "Vertex");+ runtimeConfigExact(config, ["project", "credentials", "location", "defaultConcurrency", "requestTimeoutMs"], "Vertex");+ const project = runtimeConfigCredentialRef(config.project, "Vertex project");+ if (project === undefined) throw new Error("Vertex project CredentialRef is required");+ const credentials = runtimeConfigCredentialRef(config.credentials, "Vertex credentials");+ if (credentials === undefined) throw new Error("Vertex credentials CredentialRef is required");+ const location = runtimeConfigString(config.location, "Vertex location");+ const defaultConcurrency = runtimeConfigPositiveInteger(config.defaultConcurrency, "Vertex defaultConcurrency");+ const requestTimeoutMs = runtimeConfigPositiveInteger(config.requestTimeoutMs, "Vertex requestTimeoutMs");+ return { endpoint: createVertexProvider({+ instance: context.instance,+ pool: context.pool,+ project,+ credentials,+ ...(location === undefined ? {} : { location }),+ ...(defaultConcurrency === undefined ? {} : { defaultConcurrency }),+ ...(requestTimeoutMs === undefined ? {} : { requestTimeoutMs }),+ }) };+ },+});++export const hypitPackage = { format: "hypit.node-package@1" as const, hostFacets: [adapter] };+export default hypitPackage;diff --git a/packages/provider-vertex/src/gemini.ts b/packages/provider-vertex/src/gemini.tsindex b0aade1f..3d07d84d 100644--- a/packages/provider-vertex/src/gemini.ts+++ b/packages/provider-vertex/src/gemini.ts@@ -1,14 +1,9 @@import { GoogleGenAI } from "@google/genai";import type { Content } from "@google/genai";+import type { GeminiGenerateInput, GeminiInlinePart } from "@hypit/gemini";-export type VertexGeminiPart =- | { readonly text?: string; readonly inlineData?: never }- | { readonly inlineData?: { readonly mimeType: string; readonly data: string }; readonly text?: never };--export type VertexGeminiGenerateInput = {- readonly parts: readonly VertexGeminiPart[];- readonly instruction: string;-};+export type VertexGeminiPart = GeminiInlinePart;+export type VertexGeminiGenerateInput = GeminiGenerateInput;export type VertexGeminiGeneratorOptions = {readonly project: string;diff --git a/packages/provider-vertex/src/index.ts b/packages/provider-vertex/src/index.tsindex 49605787..f8ae7798 100644--- a/packages/provider-vertex/src/index.ts+++ b/packages/provider-vertex/src/index.ts@@ -4,3 +4,5 @@ export type {VertexGeminiGeneratorOptions,VertexGeminiPart,} from "./gemini.js";+export { createVertexProvider, vertexProviderModuleRef } from "./provider.js";+export type { CreateVertexProviderOptions } from "./provider.js";diff --git a/packages/provider-vertex/src/provider.ts b/packages/provider-vertex/src/provider.tsnew file mode 100644index 00000000..82c6d02c--- /dev/null+++ b/packages/provider-vertex/src/provider.ts@@ -0,0 +1,73 @@+import { defineEndpointPackage } from "@hypit/endpoint-kit";+import type { EndpointInvocationContext, ImmediateEndpointHandler } from "@hypit/endpoint-kit";+import { geminiCapabilities, geminiModels, verifyGeminiRequest } from "@hypit/gemini";+import type { GeminiRequest } from "@hypit/gemini";+import { canonicalize } from "@hypit/protocol";+import { credentialRef } from "@hypit/runtime";+import type { CredentialRef } from "@hypit/runtime";+import { sealText, textTypes } from "@hypit/text";++import { createVertexGeminiGenerator } from "./gemini.js";++export const vertexProviderModuleRef = { name: "@hypit/provider-vertex", version: "1" } as const;++export type CreateVertexProviderOptions = {+ readonly instance?: string;+ readonly pool?: string;+ readonly project: CredentialRef;+ readonly credentials: CredentialRef;+ readonly location?: string;+ readonly defaultConcurrency?: number;+ readonly requestTimeoutMs?: number;+};++function secret(context: EndpointInvocationContext, name: "project" | "credentials"): string {+ const value = context.credentials[name]?.secret;+ if (typeof value !== "string" || value.trim().length === 0) throw new Error(`Vertex ${name} credential is unavailable`);+ return value;+}++export function createVertexProvider(options: CreateVertexProviderOptions) {+ const handler: ImmediateEndpointHandler = async (context) => {+ const request = context.need.constraints as unknown;+ verifyGeminiRequest(request);+ const typed = request as GeminiRequest;+ const generate = createVertexGeminiGenerator({+ project: secret(context, "project"),+ credentials: secret(context, "credentials"),+ model: context.need.capability.name,+ ...(options.location === undefined ? {} : { location: options.location }),+ ...(options.requestTimeoutMs === undefined ? {} : { requestTimeoutMs: options.requestTimeoutMs }),+ });+ const parts: Array<{ text: string } | { inlineData: { mimeType: string; data: string } }> = [{ text: typed.prompt }];+ for (const item of typed.media) {+ const bytes = await context.artifacts.get(item.artifact.digest);+ if (bytes === undefined) throw new Error(`Vertex Gemini reference artifact ${item.artifact.digest} is unavailable`);+ parts.push({ inlineData: { mimeType: item.artifact.mediaType, data: Buffer.from(bytes).toString("base64") } });+ }+ const value = await generate({ parts, instruction: typed.instruction });+ return { value: { kind: "inline", value: canonicalize(sealText(value)) } };+ };+ return defineEndpointPackage({+ module: vertexProviderModuleRef,+ facet: "gemini",+ instance: options.instance ?? "vertex.default",+ pool: options.pool ?? options.instance ?? "vertex.default",+ credentials: {+ project: options.project ?? credentialRef("env", "GOOGLE_CLOUD_PROJECT"),+ credentials: options.credentials ?? credentialRef("env", "GOOGLE_APPLICATION_CREDENTIALS_JSON"),+ },+ credentialInputs: {+ project: { label: "Google Cloud project" },+ credentials: { label: "Google application credentials", kind: "json" },+ },+ defaultConcurrency: options.defaultConcurrency ?? 4,+ capabilities: geminiModels.map((model) => ({+ capability: geminiCapabilities[model],+ returns: textTypes.text,+ lifecycle: "immediate" as const,+ handler,+ lane: model,+ })),+ });+}diff --git a/packages/provider-vertex/test/provider.test.ts b/packages/provider-vertex/test/provider.test.tsnew file mode 100644index 00000000..a15f5b84--- /dev/null+++ b/packages/provider-vertex/test/provider.test.ts@@ -0,0 +1,33 @@+import assert from "node:assert/strict";+import test from "node:test";++import { EndpointRegistry } from "@hypit/driver-node";+import { geminiCapabilities } from "@hypit/gemini";+import { credentialRef } from "@hypit/runtime";+import { textTypes } from "@hypit/text";++import { createVertexProvider } from "../src/provider.js";++test("Vertex Runtime provider exposes the same provider-neutral Gemini capabilities", async () => {+ const registry = new EndpointRegistry();+ const provider = createVertexProvider({+ project: credentialRef("env", "GOOGLE_CLOUD_PROJECT"),+ credentials: credentialRef("env", "GOOGLE_APPLICATION_CREDENTIALS_JSON"),+ });+ await provider.install(registry);+ for (const model of ["gemini-3.1-pro-preview", "gemini-3.7-flash"] as const) {+ const resolution = registry.resolve({+ id: `need:${model}`,+ capability: geminiCapabilities[model],+ returns: textTypes.text,+ constraints: { instruction: "x", prompt: "x", media: [] },+ result: `record:${model}`,+ });+ assert.equal(resolution.status, "resolved");+ assert.equal(resolution.registration.kind, "immediate");+ }+ assert.deepEqual(provider.credentials.map((item) => ({ slot: item.slot, kind: item.kind })), [+ { slot: "credentials", kind: "json" },+ { slot: "project", kind: "secret" },+ ]);+});diff --git a/packages/reference-video-tools/package.json b/packages/reference-video-tools/package.jsonindex 76459d81..5495b3f8 100644--- a/packages/reference-video-tools/package.json+++ b/packages/reference-video-tools/package.json@@ -16,6 +16,7 @@"@hypit/core": "workspace:*","@hypit/driver-node": "workspace:*","@hypit/estimate": "workspace:*",+ "@hypit/gemini": "workspace:*","@hypit/hyperframes": "workspace:*","@hypit/markup": "workspace:*","@hypit/media": "workspace:*",diff --git a/packages/reference-video-tools/src/tools.ts b/packages/reference-video-tools/src/tools.tsindex 50bebd8c..02e6f1ee 100644--- a/packages/reference-video-tools/src/tools.ts+++ b/packages/reference-video-tools/src/tools.ts@@ -22,7 +22,7 @@ import { describeSchema } from "./contract.js";import { videoCliDistribution } from "@hypit/video-cli";import { createHypiHubGeminiGenerator } from "@hypit/provider-hypihub";import { createVertexGeminiGenerator } from "@hypit/provider-vertex";-import type { VertexGeminiPart } from "@hypit/provider-vertex";+import type { GeminiInlinePart } from "@hypit/gemini";import { authorSource, invokedFrom, nearestPackageRoot, referenceRoot, referenceWords, renderElement, renderPreviews, spokenRange, standInSidecarPath, tokenWindow } from "./authoring.js";import type { RenderElementInput, RenderPreviewsInput, SpokenRange, StandInFocus, StandInSidecar } from "./authoring.js";@@ -300,7 +300,7 @@ type ToolOptions = {readonly retryDelayMs?: number;readonly generate?: GenerateText;};-export type GenerateText = (input: { readonly parts: readonly VertexGeminiPart[]; readonly instruction: string }) => Promise<string>;+export type GenerateText = (input: { readonly parts: readonly GeminiInlinePart[]; readonly instruction: string }) => Promise<string>;type ObservationTask = { readonly key: string; readonly request: Request };function routeStateResult(state: RouteState | undefined): Record<string, unknown> {@@ -850,12 +850,12 @@ const MIME_TYPES: Readonly<Record<string, string>> = {// The Vertex path cannot upload a file and reference it later, so the bytes travel with every// request. One shot's clip is asked about by its own picture, type and sound observations and by the// boundary on each side, so reading and encoding it once per invocation is worth the memory.-function mediaParts(): (path: string) => Promise<VertexGeminiPart> {- const encoded = new Map<string, Promise<VertexGeminiPart>>();+function mediaParts(): (path: string) => Promise<GeminiInlinePart> {+ const encoded = new Map<string, Promise<GeminiInlinePart>>();return (path) => {const held = encoded.get(path);if (held !== undefined) return held;- const part = (async (): Promise<VertexGeminiPart> => {+ const part = (async (): Promise<GeminiInlinePart> => {const bytes = await readBytes(path);const extension = path.toLowerCase().split(".").pop() ?? "";const mimeType = MIME_TYPES[extension];@@ -881,7 +881,7 @@ function permanent(error: unknown): boolean {return /invalid[_ ]argument|permission[_ ]denied|unauthenticated|not[_ ]found|failed[_ ]precondition/iu.test(message(error));}-async function callSafely(retryDelayMs: number, generate: GenerateText, parts: readonly VertexGeminiPart[], instruction: string): Promise<Observation> {+async function callSafely(retryDelayMs: number, generate: GenerateText, parts: readonly GeminiInlinePart[], instruction: string): Promise<Observation> {let last: unknown;for (let attempt = 1; attempt <= 6; attempt += 1) {try { return observation("complete", await generate({ parts, instruction })); }@@ -1461,7 +1461,7 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceVreturn {pending: [],ask: async (_key, { media, prompt, instruction }) => {- const parts: VertexGeminiPart[] = [];+ const parts: GeminiInlinePart[] = [];for (const path of media) parts.push(await mediaPart(path));parts.push({ text: `${prompt}\n\n${WATERMARK_RULE}` });return await callSafely(retryDelayMs, generate, parts, instruction);diff --git a/pnpm-lock.yaml b/pnpm-lock.yamlindex c2f8a6f2..657325dd 100644--- a/pnpm-lock.yaml+++ b/pnpm-lock.yaml@@ -72,6 +72,9 @@ importers:'@hypit/fonts-open':specifier: workspace:*version: link:packages/fonts-open+ '@hypit/gemini':+ specifier: workspace:*+ version: link:packages/gemini'@hypit/generation':specifier: workspace:*version: link:packages/generation@@ -1313,6 +1316,27 @@ importers:specifier: 0.2.0version: 0.2.0+ packages/gemini:+ dependencies:+ '@hypit/artifact':+ specifier: workspace:*+ version: link:../artifact+ '@hypit/component-kit':+ specifier: workspace:*+ version: link:../component-kit+ '@hypit/elaborator':+ specifier: workspace:*+ version: link:../elaborator+ '@hypit/markup':+ specifier: workspace:*+ version: link:../markup+ '@hypit/protocol':+ specifier: workspace:*+ version: link:../protocol+ '@hypit/text':+ specifier: workspace:*+ version: link:../text+packages/generation:dependencies:'@hypit/artifact':@@ -1948,6 +1972,9 @@ importers:'@hypit/endpoint-kit':specifier: workspace:*version: link:../endpoint-kit+ '@hypit/gemini':+ specifier: workspace:*+ version: link:../gemini'@hypit/generation':specifier: workspace:*version: link:../generation@@ -1960,6 +1987,9 @@ importers:'@hypit/runtime-kit':specifier: workspace:*version: link:../runtime-kit+ '@hypit/text':+ specifier: workspace:*+ version: link:../textdevDependencies:'@hypit/driver-node':specifier: workspace:*@@ -2163,6 +2193,28 @@ importers:'@google/genai':specifier: 1.52.0version: 1.52.0+ '@hypit/endpoint-kit':+ specifier: workspace:*+ version: link:../endpoint-kit+ '@hypit/gemini':+ specifier: workspace:*+ version: link:../gemini+ '@hypit/protocol':+ specifier: workspace:*+ version: link:../protocol+ '@hypit/runtime':+ specifier: workspace:*+ version: link:../runtime+ '@hypit/runtime-kit':+ specifier: workspace:*+ version: link:../runtime-kit+ '@hypit/text':+ specifier: workspace:*+ version: link:../text+ devDependencies:+ '@hypit/driver-node':+ specifier: workspace:*+ version: link:../driver-nodepackages/provider-whisperx-local:dependencies:@@ -2313,6 +2365,9 @@ importers:'@hypit/estimate':specifier: workspace:*version: link:../estimate+ '@hypit/gemini':+ specifier: workspace:*+ version: link:../gemini'@hypit/hyperframes':specifier: workspace:*version: link:../hyperframesdiff --git a/projects/hypihub-provider-smoke/build.svrun b/projects/hypihub-provider-smoke/build.svrunindex 2e8065aa..ba4cfb5e 100644--- a/projects/hypihub-provider-smoke/build.svrun+++ b/projects/hypihub-provider-smoke/build.svrun@@ -6,19 +6,27 @@<target output="gpt-edit.image"/><target output="nano-text.image"/><target output="nano-edit.image"/>- <target output="nano2-text.image"/>- <target output="nano2-edit.image"/><target output="seedream-text.image"/><target output="seedream-edit.image"/><target output="seedance-text.video"/><target output="seedance-image-audio.video"/>+ <target output="seedance-frame.video"/>+ <target output="seedance-video-reference.video"/><target output="seedance-standard-text.video"/><target output="seedance-fast-text.video"/><target output="seedance-v25-text.video"/><target output="minimax-text.video"/><target output="minimax-reference.video"/>+ <target output="minimax-frame.video"/>+ <target output="minimax-video-reference.video"/><target output="grok-text.video"/><target output="grok-image.video"/>+ <target output="gemini-flash-text.text"/>+ <target output="gemini-flash-image.text"/>+ <target output="gemini-flash-video.text"/>+ <target output="gemini-pro-text.text"/>+ <target output="gemini-pro-image.text"/>+ <target output="gemini-pro-video.text"/><target output="mimo-voice.audio"/><target output="mimo-preset.audio"/><target output="mimo-clone.audio"/>diff --git a/projects/hypihub-provider-smoke/gemini-media.svrun b/projects/hypihub-provider-smoke/gemini-media.svrunnew file mode 100644index 00000000..d62a3d1c--- /dev/null+++ b/projects/hypihub-provider-smoke/gemini-media.svrun@@ -0,0 +1,9 @@+<?svml using="@hypit/run-markup@1"?>++<svrun version="1">+ <author source="./main.svml"/>+ <target output="gemini-flash-image.text"/>+ <target output="gemini-flash-video.text"/>+ <target output="gemini-pro-image.text"/>+ <target output="gemini-pro-video.text"/>+</svrun>diff --git a/projects/hypihub-provider-smoke/gemini.svrun b/projects/hypihub-provider-smoke/gemini.svrunnew file mode 100644index 00000000..777e816b--- /dev/null+++ b/projects/hypihub-provider-smoke/gemini.svrun@@ -0,0 +1,11 @@+<?svml using="@hypit/run-markup@1"?>++<svrun version="1">+ <author source="./main.svml"/>+ <target output="gemini-flash-text.text"/>+ <target output="gemini-flash-image.text"/>+ <target output="gemini-flash-video.text"/>+ <target output="gemini-pro-text.text"/>+ <target output="gemini-pro-image.text"/>+ <target output="gemini-pro-video.text"/>+</svrun>diff --git a/projects/hypihub-provider-smoke/hypit.runtime.json b/projects/hypihub-provider-smoke/hypit.runtime.jsonindex 1f814f8a..31f36a9a 100644--- a/projects/hypihub-provider-smoke/hypit.runtime.json+++ b/projects/hypihub-provider-smoke/hypit.runtime.json@@ -3,17 +3,17 @@"runtime": {"use": "@hypit/runtime-local","config": {- "dataRoot": ".hypit/runtimes/local-v4",+ "dataRoot": ".hypit/runtimes/local","artifacts": { "use": "@hypit/artifact-store-fs", "config": { "path": "artifacts" } },"credentials": { "env": { "use": "@hypit/credential-store-env" } },"endpoints": {"hypihub.smoke": {"use": "@hypit/provider-hypihub",- "pool": "hypihub.smoke-v4",+ "pool": "hypihub.smoke","config": {"baseUrl": "https://hypit.ai/v1","apiKey": { "store": "env", "key": "HYPIHUB_API_KEY" },- "defaultConcurrency": 1,+ "defaultConcurrency": 8,"pollIntervalMs": 5000,"requestTimeoutMs": 120000,"audio": falsediff --git a/projects/hypihub-provider-smoke/images.svrun b/projects/hypihub-provider-smoke/images.svrunindex 6b4a7bdc..607b1992 100644--- a/projects/hypihub-provider-smoke/images.svrun+++ b/projects/hypihub-provider-smoke/images.svrun@@ -6,8 +6,6 @@<target output="gpt-edit.image"/><target output="nano-text.image"/><target output="nano-edit.image"/>- <target output="nano2-text.image"/>- <target output="nano2-edit.image"/><target output="seedream-text.image"/><target output="seedream-edit.image"/></svrun>diff --git a/projects/hypihub-provider-smoke/main.svml b/projects/hypihub-provider-smoke/main.svmlindex 20e8369b..7d212ef2 100644--- a/projects/hypihub-provider-smoke/main.svml+++ b/projects/hypihub-provider-smoke/main.svml@@ -9,6 +9,7 @@<import as="seedance" from="@hypit/seedance@1"/><import as="h3" from="@hypit/minimax-h3@1"/><import as="grok" from="@hypit/grok-imagine@1"/>+ <import as="gemini" from="@hypit/gemini@1"/><import as="mimo" from="@hypit/mimo-tts@1"/><import as="recipes" source="./recipes.svs"/>@@ -16,7 +17,12 @@<text:Value id="edit-prompt">Keep the paper boat unchanged and make the studio background warm orange, no text.</text:Value><text:Value id="motion-prompt">The paper boat glides slowly across the blue table in one continuous locked-camera shot.</text:Value><text:Value id="speech">The red paper boat moves across the blue table.</text:Value>+ <text:Value id="gemini-instruction">Answer with a single short factual sentence.</text:Value>+ <text:Value id="gemini-text-prompt">Reply with exactly: provider text works</text:Value>+ <text:Value id="gemini-image-prompt">Name the main object and its color in this image.</text:Value>+ <text:Value id="gemini-video-prompt">Briefly describe the visible motion in this video.</text:Value><asset:Image id="reference-image" src="./assets/reference.png"/>+ <asset:Video id="reference-video" src="./assets/reference.mp4"/><asset:Audio id="reference-audio" src="./assets/reference.wav"/><gpt:Image id="gpt-text" prompt={prompt} aspect-ratio="1:1" resolution="1K"/>@@ -43,6 +49,10 @@<seedance:Reference image={reference-image}/><seedance:Reference audio={reference-audio}/></seedance:ReferenceVideo>+ <seedance:FrameVideo id="seedance-frame" model="mini" prompt={motion-prompt} duration="5" resolution="480p" aspect-ratio="16:9" generate-audio="false" first-frame={reference-image}/>+ <seedance:ReferenceVideo id="seedance-video-reference" model="mini" prompt={motion-prompt} duration="5" resolution="480p" aspect-ratio="16:9" generate-audio="false">+ <seedance:Reference video={reference-video}/>+ </seedance:ReferenceVideo><seedance:TextVideo id="seedance-standard-text" model="standard" prompt={motion-prompt} duration="5" resolution="480p" aspect-ratio="16:9" generate-audio="false"/><seedance:TextVideo id="seedance-fast-text" model="fast" prompt={motion-prompt} duration="5" resolution="480p" aspect-ratio="16:9" generate-audio="false"/><seedance:TextVideo id="seedance-v25-text" model="2.5" prompt={motion-prompt} duration="5" resolution="720p" aspect-ratio="16:9" generate-audio="false"/>@@ -51,12 +61,31 @@<h3:ReferenceVideo id="minimax-reference" prompt={motion-prompt} duration="6" resolution="768P" aspect-ratio="16:9"><h3:Reference image={reference-image}/></h3:ReferenceVideo>+ <h3:FrameVideo id="minimax-frame" prompt={motion-prompt} duration="6" resolution="768P" first-frame={reference-image}/>+ <h3:ReferenceVideo id="minimax-video-reference" prompt={motion-prompt} duration="6" resolution="768P" aspect-ratio="16:9">+ <h3:Reference video={reference-video}/>+ </h3:ReferenceVideo><grok:PreviewVideo id="grok-text" prompt={motion-prompt} duration="6" aspect-ratio="16:9" resolution="480p"/><grok:Video id="grok-image" prompt={motion-prompt} duration="6" aspect-ratio="16:9" resolution="480p"><grok:Reference image={reference-image}/></grok:Video>+ <gemini:Generate id="gemini-flash-text" model="gemini-3.7-flash" instruction={gemini-instruction} prompt={gemini-text-prompt}/>+ <gemini:Generate id="gemini-flash-image" model="gemini-3.7-flash" instruction={gemini-instruction} prompt={gemini-image-prompt}>+ <gemini:Reference media={reference-image}/>+ </gemini:Generate>+ <gemini:Generate id="gemini-flash-video" model="gemini-3.7-flash" instruction={gemini-instruction} prompt={gemini-video-prompt}>+ <gemini:Reference media={reference-video}/>+ </gemini:Generate>+ <gemini:Generate id="gemini-pro-text" model="gemini-3.1-pro-preview" instruction={gemini-instruction} prompt={gemini-text-prompt}/>+ <gemini:Generate id="gemini-pro-image" model="gemini-3.1-pro-preview" instruction={gemini-instruction} prompt={gemini-image-prompt}>+ <gemini:Reference media={reference-image}/>+ </gemini:Generate>+ <gemini:Generate id="gemini-pro-video" model="gemini-3.1-pro-preview" instruction={gemini-instruction} prompt={gemini-video-prompt}>+ <gemini:Reference media={reference-video}/>+ </gemini:Generate>+<mimo:VoiceDesign id="mimo-voice" speech={speech}>A clear, warm narrator with a lightly playful delivery.</mimo:VoiceDesign><mimo:Preset id="mimo-preset" speech={speech} voice="Chloe"/><mimo:VoiceClone id="mimo-clone" speech={speech} sample={reference-audio}/>diff --git a/projects/hypihub-provider-smoke/unavailable.svrun b/projects/hypihub-provider-smoke/unavailable.svrunnew file mode 100644index 00000000..20e0c35e--- /dev/null+++ b/projects/hypihub-provider-smoke/unavailable.svrun@@ -0,0 +1,7 @@+<?svml using="@hypit/run-markup@1"?>++<svrun version="1">+ <author source="./main.svml"/>+ <target output="nano2-text.image"/>+ <target output="nano2-edit.image"/>+</svrun>diff --git a/projects/hypihub-provider-smoke/video-references.svrun b/projects/hypihub-provider-smoke/video-references.svrunnew file mode 100644index 00000000..4ad8ed32--- /dev/null+++ b/projects/hypihub-provider-smoke/video-references.svrun@@ -0,0 +1,9 @@+<?svml using="@hypit/run-markup@1"?>++<svrun version="1">+ <author source="./main.svml"/>+ <target output="seedance-frame.video"/>+ <target output="seedance-video-reference.video"/>+ <target output="minimax-frame.video"/>+ <target output="minimax-video-reference.video"/>+</svrun>diff --git a/projects/hypihub-provider-smoke/videos.svrun b/projects/hypihub-provider-smoke/videos.svrunindex a7b1afb1..e3850853 100644--- a/projects/hypihub-provider-smoke/videos.svrun+++ b/projects/hypihub-provider-smoke/videos.svrun@@ -4,11 +4,15 @@<author source="./main.svml"/><target output="seedance-text.video"/><target output="seedance-image-audio.video"/>+ <target output="seedance-frame.video"/>+ <target output="seedance-video-reference.video"/><target output="seedance-standard-text.video"/><target output="seedance-fast-text.video"/><target output="seedance-v25-text.video"/><target output="minimax-text.video"/><target output="minimax-reference.video"/>+ <target output="minimax-frame.video"/>+ <target output="minimax-video-reference.video"/><target output="grok-text.video"/><target output="grok-image.video"/></svrun>