Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

fix(provider-hypihub): align media upload transport

kashorin36fa5dae098b
diff --git a/packages/provider-hypihub/README.md b/packages/provider-hypihub/README.md
index 7c64cfb6..66532dc0 100644
--- a/packages/provider-hypihub/README.md
+++ b/packages/provider-hypihub/README.md
@@ -39,9 +39,17 @@ callers can still use the exported `createHypiHubGeminiGenerator`. Run `hypit au
an existing `/v1`/`/v1beta` base is accepted) only when choosing HypiHub. The Runtime Provider also
accepts the origin or either versioned base and normalizes it to `/v1`; missing or insufficient user
credentials should be resolved at [hypit.ai](https://hypit.ai). Referenced image, audio and video
-Artifacts are uploaded automatically through `POST /v1/files`, then their returned HTTPS capability
-URLs are used in image and video requests. One referenced Resource is uploaded once within one
-Runtime operation. Embedded callers may override that transport with `publicAssetUrl`.
+Resources are uploaded through a session from `POST /v1/files/uploads`, followed by the private
+regional multipart instructions returned by HypiHub. The Provider follows the server-selected part
+size and concurrency, retries only a failed part with a fresh signed URL, completes or cancels that
+one upload, and then passes the returned HTTPS URL to generation, Gemini or transcription. One
+Resource identity is uploaded once within one Runtime operation. Hypit keeps no upload catalog or
+cross-Build cache. Embedded callers may replace this transport with `publicAssetUrl`.
+
+The service currently requires whole-file and per-part SHA-256 values as fields of its signed upload
+protocol. They exist only while transferring bytes; Hypit never uses them as Resource identity,
+Result metadata, lookup keys or reuse evidence. Signed URLs and their query credentials are removed
+from surfaced upload errors.
The default remote alignment model is `victor-upmeet/whisperx`; `transcriptionModel` may select another
HypiHub model that exposes the `transcriptions` route. `hypit doctor` reads the authenticated model
@@ -54,3 +62,10 @@ HypiHub declares MiMo VoiceDesign together with every other capability it serves
one. When another selected Endpoint offers the same capability (a local WhisperX, a Vertex Gemini,
the official MiMo Provider), the Runtime Profile's `bindings` say which Endpoint serves it. Hypit does
not expose MiMo preset-voice or voice-cloning models.
+
+Execution policy remains local to this Provider. A profile may set `requestTimeoutMs`,
+`operationTimeoutMs`, `uploadPartTimeoutMs`, `uploadPartAttempts`, `downloadAttempts`,
+`geminiRateLimitAttempts` and `geminiRateLimitRetryDelayMs`; defaults are respectively 300 seconds,
+20 minutes, 5 minutes, 3 attempts, 3 attempts, 4 attempts and 2 seconds. `defaultConcurrency`
+controls this Endpoint's shared Runtime capacity. Multipart concurrency is different: HypiHub
+selects it for one upload session, and it does not create a Build queue or a second Runtime pool.
diff --git a/packages/provider-hypihub/src/activation.ts b/packages/provider-hypihub/src/activation.ts
index 129915a0..c9e82d03 100644
--- a/packages/provider-hypihub/src/activation.ts
+++ b/packages/provider-hypihub/src/activation.ts
@@ -15,7 +15,18 @@ const adapter = createRuntimeEndpointAdapterFacet({
if (context.pool === undefined) throw new Error("HypiHub Provider Pool is required");
const config = runtimeConfigObject(context.config, "HypiHub");
runtimeConfigExact(config, [
- "baseUrl", "apiKey", "defaultConcurrency", "pollIntervalMs", "requestTimeoutMs", "transcriptionModel",
+ "baseUrl",
+ "apiKey",
+ "defaultConcurrency",
+ "pollIntervalMs",
+ "requestTimeoutMs",
+ "operationTimeoutMs",
+ "uploadPartTimeoutMs",
+ "uploadPartAttempts",
+ "downloadAttempts",
+ "geminiRateLimitAttempts",
+ "geminiRateLimitRetryDelayMs",
+ "transcriptionModel",
], "HypiHub");
const baseUrl = runtimeConfigString(config.baseUrl, "HypiHub baseUrl");
if (baseUrl !== undefined) {
@@ -29,6 +40,12 @@ const adapter = createRuntimeEndpointAdapterFacet({
const defaultConcurrency = runtimeConfigPositiveInteger(config.defaultConcurrency, "HypiHub defaultConcurrency");
const pollIntervalMs = runtimeConfigPositiveInteger(config.pollIntervalMs, "HypiHub pollIntervalMs");
const requestTimeoutMs = runtimeConfigPositiveInteger(config.requestTimeoutMs, "HypiHub requestTimeoutMs");
+ const operationTimeoutMs = runtimeConfigPositiveInteger(config.operationTimeoutMs, "HypiHub operationTimeoutMs");
+ const uploadPartTimeoutMs = runtimeConfigPositiveInteger(config.uploadPartTimeoutMs, "HypiHub uploadPartTimeoutMs");
+ const uploadPartAttempts = runtimeConfigPositiveInteger(config.uploadPartAttempts, "HypiHub uploadPartAttempts");
+ const downloadAttempts = runtimeConfigPositiveInteger(config.downloadAttempts, "HypiHub downloadAttempts");
+ const geminiRateLimitAttempts = runtimeConfigPositiveInteger(config.geminiRateLimitAttempts, "HypiHub geminiRateLimitAttempts");
+ const geminiRateLimitRetryDelayMs = runtimeConfigPositiveInteger(config.geminiRateLimitRetryDelayMs, "HypiHub geminiRateLimitRetryDelayMs");
const transcriptionModel = runtimeConfigString(config.transcriptionModel, "HypiHub transcriptionModel");
const options = {
instance: context.instance,
@@ -38,6 +55,12 @@ const adapter = createRuntimeEndpointAdapterFacet({
...(defaultConcurrency === undefined ? {} : { defaultConcurrency }),
...(pollIntervalMs === undefined ? {} : { pollIntervalMs }),
...(requestTimeoutMs === undefined ? {} : { requestTimeoutMs }),
+ ...(operationTimeoutMs === undefined ? {} : { operationTimeoutMs }),
+ ...(uploadPartTimeoutMs === undefined ? {} : { uploadPartTimeoutMs }),
+ ...(uploadPartAttempts === undefined ? {} : { uploadPartAttempts }),
+ ...(downloadAttempts === undefined ? {} : { downloadAttempts }),
+ ...(geminiRateLimitAttempts === undefined ? {} : { geminiRateLimitAttempts }),
+ ...(geminiRateLimitRetryDelayMs === undefined ? {} : { geminiRateLimitRetryDelayMs }),
...(transcriptionModel === undefined ? {} : { transcriptionModel }),
};
return {
diff --git a/packages/provider-hypihub/src/gemini.ts b/packages/provider-hypihub/src/gemini.ts
index 4c415a7f..4aa76def 100644
--- a/packages/provider-hypihub/src/gemini.ts
+++ b/packages/provider-hypihub/src/gemini.ts
@@ -1,4 +1,5 @@
import type { GeminiInlinePart } from "@hypit/gemini";
+import { HypiHubUploader } from "./upload.js";
export type HypiHubGeminiPart = GeminiInlinePart
| { readonly fileData: { readonly mimeType?: string; readonly fileUri: string } };
@@ -8,6 +9,8 @@ export type HypiHubGeminiGeneratorOptions = {
readonly model?: string;
readonly baseUrl?: string;
readonly requestTimeoutMs?: number;
+ readonly uploadPartTimeoutMs?: number;
+ readonly uploadPartAttempts?: number;
readonly maxRateLimitRetries?: number;
readonly rateLimitRetryDelayMs?: number;
readonly fetch?: typeof globalThis.fetch;
@@ -38,7 +41,7 @@ function retryDelay(response: Response, fallback: number, attempt: number): numb
return fallback * (attempt + 1);
}
-function partValue(part: HypiHubGeminiPart, uploaded: Map<string, Promise<string>>,
+function partValue(part: HypiHubGeminiPart,
upload: (mimeType: string, data: string) => Promise<string>): Promise<Record<string, unknown>> {
if ("text" in part) return Promise.resolve({ text: part.text });
if ("fileData" in part) {
@@ -49,13 +52,8 @@ function partValue(part: HypiHubGeminiPart, uploaded: Map<string, Promise<string
if ("inlineData" in part) {
assert(typeof part.inlineData.mimeType === "string" && typeof part.inlineData.data === "string",
"HypiHub Gemini inline media part is incomplete");
- const key = `${part.inlineData.mimeType}\0${part.inlineData.data}`;
- let pending = uploaded.get(key);
- if (pending === undefined) {
- pending = upload(part.inlineData.mimeType, part.inlineData.data);
- uploaded.set(key, pending);
- }
- return pending.then((fileUri) => ({ fileData: { mimeType: part.inlineData.mimeType, fileUri } }));
+ return upload(part.inlineData.mimeType, part.inlineData.data)
+ .then((fileUri) => ({ fileData: { mimeType: part.inlineData.mimeType, fileUri } }));
}
throw new Error("HypiHub Gemini only supports text, inline media and file media parts");
}
@@ -78,35 +76,28 @@ export function createHypiHubGeminiGenerator(options: HypiHubGeminiGeneratorOpti
const rateLimitRetryDelayMs = positiveInteger(options.rateLimitRetryDelayMs ?? 2_000,
"HypiHub Gemini rateLimitRetryDelayMs");
const fetcher = options.fetch ?? globalThis.fetch;
+ const uploader = new HypiHubUploader({
+ baseUrl,
+ requestTimeoutMs: timeout,
+ uploadPartTimeoutMs: positiveInteger(options.uploadPartTimeoutMs ?? 5 * 60_000,
+ "HypiHub Gemini uploadPartTimeoutMs"),
+ uploadPartAttempts: positiveInteger(options.uploadPartAttempts ?? 3,
+ "HypiHub Gemini uploadPartAttempts"),
+ fetch: fetcher,
+ });
return async (input: HypiHubGeminiGenerateInput): Promise<string> => {
+ const upload = async (mimeType: string, encoded: string): Promise<string> => {
+ let bytes: Uint8Array;
+ try { bytes = new Uint8Array(Buffer.from(encoded, "base64")); }
+ catch (error) { throw new Error("HypiHub Gemini inline media is not valid base64", { cause: error }); }
+ assert(bytes.byteLength > 0, "HypiHub Gemini inline media is empty");
+ return await uploader.upload({ bytes, mediaType: mimeType }, apiKey);
+ };
+ const parts = await Promise.all(input.parts.map((part) => partValue(part, upload)));
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), timeout);
try {
- const uploaded = new Map<string, Promise<string>>();
- const upload = async (mimeType: string, encoded: string): Promise<string> => {
- let bytes: Uint8Array;
- try { bytes = new Uint8Array(Buffer.from(encoded, "base64")); }
- catch (error) { throw new Error("HypiHub Gemini inline media is not valid base64", { cause: error }); }
- assert(bytes.byteLength > 0, "HypiHub Gemini inline media is empty");
- const form = new FormData();
- const copy = new ArrayBuffer(bytes.byteLength);
- new Uint8Array(copy).set(bytes);
- form.append("file", new Blob([copy], { type: mimeType }), `gemini-reference.${extension(mimeType)}`);
- form.append("purpose", "reference");
- const uploadedResponse = await fetcher(`${baseUrl}/v1/files`, {
- method: "POST", signal: controller.signal,
- headers: { authorization: `Bearer ${apiKey}` }, body: form,
- });
- const uploadedText = await uploadedResponse.text();
- if (!uploadedResponse.ok) throw new Error(`HypiHub Gemini file upload returned HTTP ${uploadedResponse.status}: ${uploadedText.slice(0, 300)}`);
- let uploadedBody: unknown;
- try { uploadedBody = JSON.parse(uploadedText); } catch { throw new Error("HypiHub Gemini file upload returned invalid JSON"); }
- const url = (uploadedBody as { readonly url?: unknown }).url;
- assert(typeof url === "string" && /^https:\/\//iu.test(url), "HypiHub Gemini file upload returned no HTTPS URL");
- return url;
- };
- const parts = await Promise.all(input.parts.map((part) => partValue(part, uploaded, upload)));
let response: Response | undefined;
let text = "";
for (let attempt = 0; attempt <= maxRateLimitRetries; attempt += 1) {
@@ -145,12 +136,3 @@ export function createHypiHubGeminiGenerator(options: HypiHubGeminiGeneratorOpti
}
};
}
-
-function extension(mimeType: string): string {
- const value: Readonly<Record<string, string>> = {
- "image/jpeg": "jpg", "image/png": "png", "image/webp": "webp",
- "audio/mpeg": "mp3", "audio/wav": "wav", "audio/mp4": "m4a",
- "video/mp4": "mp4", "video/quicktime": "mov", "video/webm": "webm",
- };
- return value[mimeType.toLowerCase()] ?? "bin";
-}
diff --git a/packages/provider-hypihub/src/provider.ts b/packages/provider-hypihub/src/provider.ts
index 02bcfa39..0e88b503 100644
--- a/packages/provider-hypihub/src/provider.ts
+++ b/packages/provider-hypihub/src/provider.ts
@@ -16,6 +16,7 @@ import {
import type { WhisperXTranscriptResponse } from "@hypit/whisperx";
import { createHypiHubGeminiGenerator } from "./gemini.js";
import { hypiHubRouteForCapability, hypiHubRoutes } from "./routes.js";
+import { HypiHubUploader } from "./upload.js";
import type { RuntimeDoctorDiagnostic } from "@hypit/runtime-kit";
export const hypiHubProviderModuleRef = { name: "@hypit/provider-hypihub", version: "1" } as const;
@@ -28,10 +29,16 @@ export type CreateHypiHubProviderOptions = {
readonly defaultConcurrency?: number;
readonly pollIntervalMs?: number;
readonly requestTimeoutMs?: number;
+ readonly operationTimeoutMs?: number;
+ readonly uploadPartTimeoutMs?: number;
+ readonly uploadPartAttempts?: number;
+ readonly downloadAttempts?: number;
+ readonly geminiRateLimitAttempts?: number;
+ readonly geminiRateLimitRetryDelayMs?: number;
/** HypiHub model used for the Provider-neutral WhisperX alignment capability. */
readonly transcriptionModel?: string;
readonly fetch?: typeof globalThis.fetch;
- /** Overrides the default POST /v1/files upload for referenced artifacts. */
+ /** Overrides HypiHub's negotiated upload transport for referenced Resources. */
readonly publicAssetUrl?: (artifact: BlobRef, artifacts: ResourceStore) => Promise<string>;
};
@@ -85,31 +92,31 @@ async function verifyModelRoute(client: HypiHubClient, apiKey: string, model: st
`HypiHub model ${model} is not enabled for ${operation} with this API key`);
}
-function dataUrl(bytes: Uint8Array, mediaType: string): string {
- return `data:${mediaType};base64,${Buffer.from(bytes).toString("base64")}`;
-}
-
-async function resolveArtifactInline(artifacts: ResourceStore, artifact: BlobRef): Promise<string> {
- const bytes = await artifacts.get(artifact.resource);
- assert(bytes !== undefined, `HypiHub reference artifact ${artifact.resource} is unavailable`);
- return dataUrl(bytes, artifact.mediaType);
-}
-
-function mediaExtension(mediaType: string): string {
- const known: Readonly<Record<string, string>> = {
- "image/jpeg": "jpg", "image/png": "png", "image/webp": "webp",
- "video/mp4": "mp4", "video/quicktime": "mov",
- "audio/mpeg": "mp3", "audio/wav": "wav", "audio/x-wav": "wav", "audio/mp4": "m4a",
- };
- return known[mediaType] ?? "bin";
-}
-
class HypiHubClient {
readonly baseUrl: string;
readonly timeout: number;
+ readonly downloadAttempts: number;
readonly fetcher: typeof globalThis.fetch;
- constructor(options: { readonly baseUrl: string; readonly timeout: number; readonly fetcher: typeof globalThis.fetch }) {
- this.baseUrl = options.baseUrl.replace(/\/$/u, ""); this.timeout = options.timeout; this.fetcher = options.fetcher;
+ readonly uploader: HypiHubUploader;
+ constructor(options: {
+ readonly baseUrl: string;
+ readonly timeout: number;
+ readonly uploadPartTimeout: number;
+ readonly uploadPartAttempts: number;
+ readonly downloadAttempts: number;
+ readonly fetcher: typeof globalThis.fetch;
+ }) {
+ this.baseUrl = options.baseUrl.replace(/\/$/u, "");
+ this.timeout = options.timeout;
+ this.downloadAttempts = options.downloadAttempts;
+ this.fetcher = options.fetcher;
+ this.uploader = new HypiHubUploader({
+ baseUrl: this.baseUrl,
+ requestTimeoutMs: this.timeout,
+ uploadPartTimeoutMs: options.uploadPartTimeout,
+ uploadPartAttempts: options.uploadPartAttempts,
+ fetch: this.fetcher,
+ });
}
async json(path: string, apiKey: string, init: RequestInit = {}): Promise<Record<string, unknown>> {
const controller = new AbortController(); const timer = setTimeout(() => controller.abort(), this.timeout);
@@ -123,14 +130,17 @@ class HypiHubClient {
}
async download(url: string): Promise<{ readonly bytes: Uint8Array; readonly mediaType: string }> {
let lastError: unknown;
- for (let attempt = 0; attempt < 3; attempt += 1) {
+ for (let attempt = 0; attempt < this.downloadAttempts; attempt += 1) {
+ const controller = new AbortController();
+ const timer = setTimeout(() => controller.abort(), this.timeout);
try {
- const response = await this.fetcher(url);
+ const response = await this.fetcher(url, { signal: controller.signal });
if (!response.ok) throw new Error(`HypiHub asset returned HTTP ${response.status}`);
return { bytes: new Uint8Array(await response.arrayBuffer()), mediaType: response.headers.get("content-type")?.split(";", 1)[0] ?? "application/octet-stream" };
} catch (error) {
lastError = error;
- if (attempt < 2) await new Promise((resolve) => setTimeout(resolve, 500 * (attempt + 1)));
+ } finally {
+ clearTimeout(timer);
}
}
throw lastError instanceof Error ? lastError : new Error(String(lastError));
@@ -139,21 +149,30 @@ class HypiHubClient {
const bytes = await resources.get(artifact.resource);
assert(bytes !== undefined, `HypiHub reference artifact ${artifact.resource} is unavailable`);
assert(bytes.byteLength === artifact.size, `HypiHub reference artifact ${artifact.resource} size differs`);
- const form = new FormData();
- const copy = new ArrayBuffer(bytes.byteLength);
- new Uint8Array(copy).set(bytes);
- form.append("file", new Blob([copy], { type: artifact.mediaType }),
- `${artifact.resource}.${mediaExtension(artifact.mediaType)}`);
- form.append("purpose", "reference");
- const response = await this.json("/files", apiKey, { method: "POST", body: form });
- assert(typeof response.url === "string" && /^https:\/\//iu.test(response.url),
- "HypiHub file upload returned no HTTPS URL");
- return response.url;
+ return await this.uploader.upload({ bytes, mediaType: artifact.mediaType }, apiKey);
+ }
+ async transcribe(body: Record<string, unknown>, apiKey: string): Promise<Record<string, unknown>> {
+ return await this.json("/audio/transcriptions", apiKey, {
+ method: "POST",
+ headers: { "content-type": "application/json" },
+ body: JSON.stringify(body),
+ });
}
async binary(path: string, apiKey: string, body: Record<string, unknown>): Promise<{ readonly bytes: Uint8Array; readonly mediaType: string }> {
- const response = await this.fetcher(`${this.baseUrl}${path}`, { method: "POST", headers: { authorization: `Bearer ${apiKey}`, "content-type": "application/json" }, body: JSON.stringify(body) });
- if (!response.ok) throw new Error(`HypiHub returned HTTP ${response.status}: ${(await response.text()).slice(0, 300)}`);
- return { bytes: new Uint8Array(await response.arrayBuffer()), mediaType: response.headers.get("content-type")?.split(";", 1)[0] ?? "audio/wav" };
+ const controller = new AbortController();
+ const timer = setTimeout(() => controller.abort(), this.timeout);
+ try {
+ const response = await this.fetcher(`${this.baseUrl}${path}`, {
+ method: "POST",
+ headers: { authorization: `Bearer ${apiKey}`, "content-type": "application/json" },
+ body: JSON.stringify(body),
+ signal: controller.signal,
+ });
+ if (!response.ok) throw new Error(`HypiHub returned HTTP ${response.status}: ${(await response.text()).slice(0, 300)}`);
+ return { bytes: new Uint8Array(await response.arrayBuffer()), mediaType: response.headers.get("content-type")?.split(";", 1)[0] ?? "audio/wav" };
+ } finally {
+ clearTimeout(timer);
+ }
}
}
@@ -176,6 +195,9 @@ export async function diagnoseHypiHubProvider(
const client = new HypiHubClient({
baseUrl: apiBaseUrl(options.baseUrl ?? "https://hypit.ai/v1"),
timeout: options.requestTimeoutMs ?? 30_000,
+ uploadPartTimeout: options.uploadPartTimeoutMs ?? 5 * 60_000,
+ uploadPartAttempts: options.uploadPartAttempts ?? 3,
+ downloadAttempts: options.downloadAttempts ?? 3,
fetcher: options.fetch ?? globalThis.fetch,
});
const response = await client.json("/models", apiKey);
@@ -236,7 +258,7 @@ async function synthesizeAudio(client: HypiHubClient, context: EndpointInvocatio
const route = hypiHubRouteForCapability(context.need.capability);
assert(route !== undefined && route.media === "audio", "HypiHub does not implement this exact capability");
const compiled = await route.compile(context.need.constraints, async (artifact) => publicAssetUrl === undefined
- ? await resolveArtifactInline(context.resources, artifact)
+ ? await client.upload(artifact, context.resources, credential(context))
: await publicAssetUrl(artifact, context.resources));
await verifyModelRoute(client, credential(context), compiled.model, "audio_speech");
const audio = await client.binary("/audio/speech", credential(context), { model: compiled.model, ...(compiled.input as Record<string, unknown>) });
@@ -293,8 +315,34 @@ function endpoint(client: HypiHubClient, pollIntervalMs: number, maxOperationMs:
}
export function createHypiHubProvider(options: CreateHypiHubProviderOptions = {}) {
- const client = new HypiHubClient({ baseUrl: apiBaseUrl(options.baseUrl ?? "https://hypit.ai/v1"), timeout: options.requestTimeoutMs ?? 30_000, fetcher: options.fetch ?? globalThis.fetch });
- const asyncEndpoint = endpoint(client, options.pollIntervalMs ?? 10_000, 20 * 60_000, options.publicAssetUrl);
+ const requestTimeoutMs = options.requestTimeoutMs ?? 300_000;
+ const operationTimeoutMs = options.operationTimeoutMs ?? 20 * 60_000;
+ const uploadPartTimeoutMs = options.uploadPartTimeoutMs ?? 5 * 60_000;
+ const uploadPartAttempts = options.uploadPartAttempts ?? 3;
+ const downloadAttempts = options.downloadAttempts ?? 3;
+ const geminiRateLimitAttempts = options.geminiRateLimitAttempts ?? 4;
+ const geminiRateLimitRetryDelayMs = options.geminiRateLimitRetryDelayMs ?? 2_000;
+ for (const [name, value] of Object.entries({
+ requestTimeoutMs,
+ operationTimeoutMs,
+ uploadPartTimeoutMs,
+ uploadPartAttempts,
+ downloadAttempts,
+ geminiRateLimitAttempts,
+ geminiRateLimitRetryDelayMs,
+ })) {
+ assert(Number.isSafeInteger(value) && value > 0, `HypiHub ${name} must be a positive integer`);
+ }
+ assert(uploadPartAttempts <= 8, "HypiHub uploadPartAttempts must be within 1..8");
+ const client = new HypiHubClient({
+ baseUrl: apiBaseUrl(options.baseUrl ?? "https://hypit.ai/v1"),
+ timeout: requestTimeoutMs,
+ uploadPartTimeout: uploadPartTimeoutMs,
+ uploadPartAttempts,
+ downloadAttempts,
+ fetcher: options.fetch ?? globalThis.fetch,
+ });
+ const asyncEndpoint = endpoint(client, options.pollIntervalMs ?? 10_000, operationTimeoutMs, options.publicAssetUrl);
const audioEndpoint: ImmediateEndpointHandler = async (context) => {
try {
return await synthesizeAudio(client, context, options.publicAssetUrl);
@@ -308,16 +356,24 @@ export function createHypiHubProvider(options: CreateHypiHubProviderOptions = {}
apiKey: credential(context),
model: context.need.capability.name,
...(options.baseUrl === undefined ? {} : { baseUrl: options.baseUrl }),
- ...(options.requestTimeoutMs === undefined ? {} : { requestTimeoutMs: options.requestTimeoutMs }),
+ requestTimeoutMs,
+ uploadPartTimeoutMs,
+ uploadPartAttempts,
...(options.fetch === undefined ? {} : { fetch: options.fetch }),
- maxRateLimitRetries: 5,
- rateLimitRetryDelayMs: 3_000,
+ maxRateLimitRetries: geminiRateLimitAttempts - 1,
+ rateLimitRetryDelayMs: geminiRateLimitRetryDelayMs,
});
- const parts: Array<{ text: string } | { inlineData: { mimeType: string; data: string } }> = [{ text: request.prompt }];
+ const parts: Array<{ text: string } | { fileData: { mimeType: string; fileUri: string } }> = [{ text: request.prompt }];
+ const uploaded = new Map<string, Promise<string>>();
for (const item of request.media) {
- const bytes = await context.resources.get(item.artifact.resource);
- assert(bytes !== undefined, `HypiHub Gemini reference artifact ${item.artifact.resource} is unavailable`);
- parts.push({ inlineData: { mimeType: item.artifact.mediaType, data: Buffer.from(bytes).toString("base64") } });
+ let url = uploaded.get(item.artifact.resource);
+ if (url === undefined) {
+ url = options.publicAssetUrl === undefined
+ ? client.upload(item.artifact, context.resources, credential(context))
+ : options.publicAssetUrl(item.artifact, context.resources);
+ uploaded.set(item.artifact.resource, url);
+ }
+ parts.push({ fileData: { mimeType: item.artifact.mediaType, fileUri: await url } });
}
const value = await generate({ parts, instruction: request.instruction });
return { value: { kind: "inline", value: canonicalize(sealVisualObservation(value)) } };
@@ -328,20 +384,20 @@ export function createHypiHubProvider(options: CreateHypiHubProviderOptions = {}
const request = verifyWhisperXAlignmentRequest(context.need.constraints);
const bytes = await context.resources.get(request.audio.resource);
assert(bytes !== undefined && bytes.byteLength === request.audio.size,
- `WhisperX evidence Artifact ${request.audio.resource} is unavailable or has changed`);
+ `WhisperX evidence Resource ${request.audio.resource} is unavailable or has changed`);
assertWhisperXEvidenceWav(bytes, request.sampleFrames);
const apiKey = credential(context);
await verifyModelRoute(client, apiKey, transcriptionModel, "transcriptions");
- const copy = new ArrayBuffer(bytes.byteLength);
- new Uint8Array(copy).set(bytes);
- const form = new FormData();
- form.append("model", transcriptionModel);
- form.append("file", new Blob([copy], { type: request.audio.mediaType }), "alignment-evidence.wav");
- form.append("response_format", "verbose_json");
- form.append("language", request.language);
- form.append("timestamp_granularities[]", "segment");
- form.append("timestamp_granularities[]", "word");
- const response = await client.json("/audio/transcriptions", apiKey, { method: "POST", body: form });
+ const url = options.publicAssetUrl === undefined
+ ? await client.upload(request.audio, context.resources, apiKey)
+ : await options.publicAssetUrl(request.audio, context.resources);
+ const response = await client.transcribe({
+ model: transcriptionModel,
+ url,
+ response_format: "verbose_json",
+ language: request.language,
+ timestamp_granularities: ["segment", "word"],
+ }, apiKey);
const evidence = sealAlignedTranscriptEvidence({
passages: interpretWhisperXTranscript(response as WhisperXTranscriptResponse, request.sampleFrames),
});
diff --git a/packages/provider-hypihub/src/upload.ts b/packages/provider-hypihub/src/upload.ts
new file mode 100644
index 00000000..4cc4e0bc
--- /dev/null
+++ b/packages/provider-hypihub/src/upload.ts
@@ -0,0 +1,281 @@
+import { createHash } from "node:crypto";
+
+function assert(condition: unknown, message: string): asserts condition {
+ if (!condition) throw new Error(message);
+}
+
+function object(value: unknown, subject: string): Record<string, unknown> {
+ assert(value !== null && typeof value === "object" && !Array.isArray(value), `${subject} must be an object`);
+ return value as Record<string, unknown>;
+}
+
+function requiredString(value: unknown, subject: string): string {
+ assert(typeof value === "string" && value.length > 0, `${subject} is missing`);
+ return value;
+}
+
+function requiredInteger(value: unknown, subject: string): number {
+ assert(typeof value === "number" && Number.isSafeInteger(value) && value > 0, `${subject} is invalid`);
+ return value;
+}
+
+function extension(mediaType: string): string {
+ const known: Readonly<Record<string, string>> = {
+ "image/jpeg": "jpg", "image/png": "png", "image/webp": "webp",
+ "video/mp4": "mp4", "video/quicktime": "mov", "video/webm": "webm",
+ "audio/mpeg": "mp3", "audio/wav": "wav", "audio/x-wav": "wav", "audio/mp4": "m4a",
+ };
+ return known[mediaType.toLowerCase()] ?? "bin";
+}
+
+function apiBaseUrl(value: string): string {
+ const trimmed = value.trim().replace(/\/+$/u, "");
+ assert(trimmed.length > 0, "HypiHub upload base URL is empty");
+ return `${trimmed.replace(/\/(?:v1beta|v1)$/iu, "")}/v1`;
+}
+
+function assertHTTPS(value: string, subject: string): void {
+ let protocol = "";
+ try { protocol = new URL(value).protocol; }
+ catch { throw new Error(`${subject} is invalid`); }
+ assert(protocol === "https:", `${subject} must use HTTPS`);
+}
+
+function safeMessage(error: unknown): string {
+ const message = error instanceof Error ? error.message : String(error);
+ return message.replace(/https?:\/\/\S+/giu, "[redacted-url]").slice(0, 300);
+}
+
+export type HypiHubUploaderOptions = {
+ readonly baseUrl: string;
+ readonly requestTimeoutMs: number;
+ readonly uploadPartTimeoutMs: number;
+ readonly uploadPartAttempts: number;
+ readonly fetch: typeof globalThis.fetch;
+};
+
+export type HypiHubUploadInput = {
+ readonly bytes: Uint8Array;
+ readonly mediaType: string;
+ readonly filename?: string;
+ readonly purpose?: string;
+};
+
+type PartDeclaration = {
+ readonly part_number: number;
+ readonly bytes: number;
+ /** Required by the signed S3 transport; never used as a Hypit Resource identity. */
+ readonly checksum_sha256: string;
+};
+
+type CompletedPart = {
+ readonly part_number: number;
+ readonly etag: string;
+ readonly checksum_sha256: string;
+};
+
+/** Provider-private transport for HypiHub's negotiated regional multipart upload. */
+export class HypiHubUploader {
+ readonly #baseUrl: string;
+ readonly #requestTimeoutMs: number;
+ readonly #uploadPartTimeoutMs: number;
+ readonly #uploadPartAttempts: number;
+ readonly #fetch: typeof globalThis.fetch;
+
+ constructor(options: HypiHubUploaderOptions) {
+ this.#baseUrl = apiBaseUrl(options.baseUrl);
+ this.#requestTimeoutMs = requiredInteger(options.requestTimeoutMs, "HypiHub request timeout");
+ this.#uploadPartTimeoutMs = requiredInteger(options.uploadPartTimeoutMs, "HypiHub upload part timeout");
+ this.#uploadPartAttempts = requiredInteger(options.uploadPartAttempts, "HypiHub upload part attempts");
+ assert(this.#uploadPartAttempts <= 8, "HypiHub upload part attempts must be within 1..8");
+ this.#fetch = options.fetch;
+ }
+
+ async #json(path: string, apiKey: string, init: RequestInit = {}): Promise<Record<string, unknown>> {
+ const controller = new AbortController();
+ const timer = setTimeout(() => controller.abort(), this.#requestTimeoutMs);
+ try {
+ const response = await this.#fetch(`${this.#baseUrl}${path}`, {
+ ...init,
+ signal: controller.signal,
+ headers: { authorization: `Bearer ${apiKey}`, ...(init.headers ?? {}) },
+ });
+ const text = await response.text();
+ let body: unknown = {};
+ try { body = text.length === 0 ? {} : JSON.parse(text); }
+ catch { throw new Error(`HypiHub returned invalid JSON (${response.status})`); }
+ if (!response.ok) {
+ throw new Error(`HypiHub returned HTTP ${response.status}: ${safeMessage(text)}`);
+ }
+ return object(body, "HypiHub response");
+ } finally {
+ clearTimeout(timer);
+ }
+ }
+
+ async #signParts(
+ uploadId: string,
+ declarations: readonly PartDeclaration[],
+ apiKey: string,
+ ): Promise<Map<number, Record<string, unknown>>> {
+ const response = await this.#json(`/files/uploads/${encodeURIComponent(uploadId)}/parts`, apiKey, {
+ method: "POST",
+ headers: { "content-type": "application/json" },
+ body: JSON.stringify({ parts: declarations }),
+ });
+ assert(Array.isArray(response.parts), "HypiHub part signing response has no parts");
+ const signed = new Map<number, Record<string, unknown>>();
+ for (const raw of response.parts) {
+ const item = object(raw, "HypiHub signed upload part");
+ const partNumber = requiredInteger(item.part_number, "HypiHub signed upload part number");
+ assertHTTPS(requiredString(item.url, "HypiHub signed upload URL"), "HypiHub signed upload URL");
+ object(item.headers, "HypiHub signed upload headers");
+ signed.set(partNumber, item);
+ }
+ assert(signed.size === declarations.length, "HypiHub signed an incomplete part set");
+ return signed;
+ }
+
+ #signedHeaders(value: unknown, declaration: PartDeclaration): Record<string, string> {
+ const raw = object(value, "HypiHub signed upload headers");
+ const headers: Record<string, string> = {};
+ for (const [name, headerValue] of Object.entries(raw)) {
+ assert(typeof headerValue === "string", `HypiHub signed upload header ${name} is invalid`);
+ headers[name.toLowerCase()] = headerValue;
+ }
+ assert(headers["content-length"] === String(declaration.bytes),
+ "HypiHub signed a different part size");
+ assert(headers["x-amz-checksum-sha256"] === declaration.checksum_sha256,
+ "HypiHub signed a different part checksum");
+ return headers;
+ }
+
+ async #putPart(
+ uploadId: string,
+ declaration: PartDeclaration,
+ initial: Record<string, unknown>,
+ bytes: Uint8Array,
+ apiKey: string,
+ ): Promise<CompletedPart> {
+ let signed = initial;
+ for (let attempt = 1; attempt <= this.#uploadPartAttempts; attempt += 1) {
+ if (attempt > 1) {
+ signed = (await this.#signParts(uploadId, [declaration], apiKey)).get(declaration.part_number) ?? {};
+ }
+ const controller = new AbortController();
+ const timer = setTimeout(() => controller.abort(), this.#uploadPartTimeoutMs);
+ try {
+ const url = requiredString(signed.url, "HypiHub signed upload URL");
+ assertHTTPS(url, "HypiHub signed upload URL");
+ const body = new ArrayBuffer(bytes.byteLength);
+ new Uint8Array(body).set(bytes);
+ const response = await this.#fetch(url, {
+ method: "PUT",
+ headers: this.#signedHeaders(signed.headers, declaration),
+ body,
+ signal: controller.signal,
+ });
+ if (!response.ok) throw new Error(`S3 rejected upload part with HTTP ${response.status}`);
+ const etag = response.headers.get("etag");
+ assert(etag !== null && etag.length > 0,
+ `HypiHub upload part ${declaration.part_number} returned no ETag`);
+ const returnedChecksum = response.headers.get("x-amz-checksum-sha256");
+ assert(returnedChecksum === null || returnedChecksum === declaration.checksum_sha256,
+ `HypiHub upload part ${declaration.part_number} returned a different checksum`);
+ return {
+ part_number: declaration.part_number,
+ etag,
+ checksum_sha256: declaration.checksum_sha256,
+ };
+ } catch {
+ if (attempt === this.#uploadPartAttempts) {
+ throw new Error(`HypiHub upload part ${declaration.part_number} failed after ${attempt} attempts`);
+ }
+ } finally {
+ clearTimeout(timer);
+ }
+ }
+ throw new Error(`HypiHub upload part ${declaration.part_number} failed`);
+ }
+
+ async #uploadParts(
+ bytes: Uint8Array,
+ apiKey: string,
+ policy: Record<string, unknown>,
+ ): Promise<string> {
+ const uploadId = requiredString(policy.upload_id, "HypiHub upload id");
+ const partSize = requiredInteger(policy.part_size, "HypiHub upload part size");
+ const partCount = requiredInteger(policy.part_count, "HypiHub upload part count");
+ const concurrency = requiredInteger(policy.concurrency, "HypiHub upload concurrency");
+ assert(concurrency <= 8, "HypiHub upload concurrency exceeds 8");
+ assert(partCount <= 10_000, "HypiHub upload part count exceeds 10000");
+ assert(partCount === Math.ceil(bytes.byteLength / partSize),
+ "HypiHub upload part count differs from the file size");
+
+ const declarations = Array.from({ length: partCount }, (_, index): PartDeclaration => {
+ const part = bytes.subarray(index * partSize, Math.min((index + 1) * partSize, bytes.byteLength));
+ return {
+ part_number: index + 1,
+ bytes: part.byteLength,
+ checksum_sha256: createHash("sha256").update(part).digest("base64"),
+ };
+ });
+
+ try {
+ const signed = await this.#signParts(uploadId, declarations, apiKey);
+ const completed = new Array<CompletedPart>(partCount);
+ let cursor = 0;
+ const worker = async (): Promise<void> => {
+ while (cursor < partCount) {
+ const index = cursor;
+ cursor += 1;
+ const declaration = declarations[index];
+ assert(declaration !== undefined, "HypiHub upload part declaration is missing");
+ const capability = signed.get(declaration.part_number);
+ assert(capability !== undefined, `HypiHub upload part ${declaration.part_number} was not signed`);
+ const part = bytes.subarray(index * partSize, Math.min((index + 1) * partSize, bytes.byteLength));
+ completed[index] = await this.#putPart(uploadId, declaration, capability, part, apiKey);
+ }
+ };
+ await Promise.all(Array.from({ length: Math.min(concurrency, partCount) }, worker));
+ const response = await this.#json(`/files/uploads/${encodeURIComponent(uploadId)}/complete`, apiKey, {
+ method: "POST",
+ headers: { "content-type": "application/json" },
+ body: JSON.stringify({ parts: completed }),
+ });
+ const url = requiredString(response.url, "HypiHub upload URL");
+ assertHTTPS(url, "HypiHub upload URL");
+ return url;
+ } catch (error) {
+ try {
+ await this.#json(`/files/uploads/${encodeURIComponent(uploadId)}`, apiKey, { method: "DELETE" });
+ } catch {
+ // The upload already failed; cancellation is best-effort cleanup, not recovery.
+ }
+ // A signed transport URL may be present in a lower-level failure. Do not
+ // retain that failure as a public cause after the upload has been cancelled.
+ throw new Error(safeMessage(error));
+ }
+ }
+
+ async upload(input: HypiHubUploadInput, apiKey: string): Promise<string> {
+ assert(input.bytes.byteLength > 0, "HypiHub reference Resource is empty");
+ // The current HypiHub upload protocol asks for a whole-file transport checksum.
+ // It is not persisted by Hypit and never participates in Resource identity or reuse.
+ const transportChecksum = createHash("sha256").update(input.bytes).digest("hex");
+ const policy = await this.#json("/files/uploads", apiKey, {
+ method: "POST",
+ headers: { "content-type": "application/json" },
+ body: JSON.stringify({
+ filename: input.filename ?? `reference.${extension(input.mediaType)}`,
+ bytes: input.bytes.byteLength,
+ mime_type: input.mediaType,
+ purpose: input.purpose ?? "reference",
+ sha256: transportChecksum,
+ head_base64: Buffer.from(input.bytes.subarray(0, 512)).toString("base64"),
+ }),
+ });
+ assert(policy.upload_mode === "s3_multipart", "HypiHub returned an unknown upload mode");
+ return await this.#uploadParts(input.bytes, apiKey, policy);
+ }
+}
diff --git a/packages/provider-hypihub/test/gemini.test.ts b/packages/provider-hypihub/test/gemini.test.ts
index df3a90ce..4563ef75 100644
--- a/packages/provider-hypihub/test/gemini.test.ts
+++ b/packages/provider-hypihub/test/gemini.test.ts
@@ -13,11 +13,36 @@ test("HypiHub Gemini uses native wire format for text, image and video parts", a
baseUrl: "https://hypit.ai/v1",
fetch: async (input, init) => {
seenUrl = String(input);
- if (seenUrl.endsWith("/v1/files")) {
+ if (seenUrl.endsWith("/v1/files/uploads")) {
uploads += 1;
- assert.equal(init?.method, "POST");
- assert.ok(init?.body instanceof FormData);
- return Response.json({ url: `https://hypit.ai/files/ref-${uploads}.png` });
+ return Response.json({
+ upload_mode: "s3_multipart",
+ upload_id: `up_native_${uploads}`,
+ part_size: 16,
+ part_count: 1,
+ concurrency: 2,
+ });
+ }
+ const parts = /\/v1\/files\/uploads\/(up_native_\d+)\/parts$/u.exec(seenUrl);
+ if (parts) {
+ const body = JSON.parse(String(init?.body)) as {
+ readonly parts: readonly { readonly bytes: number; readonly checksum_sha256: string }[];
+ };
+ return Response.json({ parts: [{
+ part_number: 1,
+ url: `https://s3.example/${parts[1]}?signature=secret`,
+ headers: {
+ "content-length": String(body.parts[0]?.bytes),
+ "x-amz-checksum-sha256": body.parts[0]?.checksum_sha256,
+ },
+ }] });
+ }
+ if (seenUrl.startsWith("https://s3.example/up_native_")) {
+ return new Response(null, { headers: { etag: "part" } });
+ }
+ const completed = /\/v1\/files\/uploads\/(up_native_\d+)\/complete$/u.exec(seenUrl);
+ if (completed) {
+ return Response.json({ url: `https://hypit.ai/files/ref-${completed[1]}.png` });
}
seenBody = JSON.parse(String(init?.body)) as Record<string, unknown>;
return Response.json({ candidates: [{ content: { parts: [{ text: "OK" }] } }] });
@@ -33,8 +58,8 @@ test("HypiHub Gemini uses native wire format for text, image and video parts", a
assert.equal(uploads, 2);
assert.deepEqual(seenBody?.contents, [{ role: "user", parts: [
{ text: "inspect" },
- { fileData: { mimeType: "image/png", fileUri: "https://hypit.ai/files/ref-1.png" } },
- { fileData: { mimeType: "video/mp4", fileUri: "https://hypit.ai/files/ref-2.png" } },
+ { fileData: { mimeType: "image/png", fileUri: "https://hypit.ai/files/ref-up_native_1.png" } },
+ { fileData: { mimeType: "video/mp4", fileUri: "https://hypit.ai/files/ref-up_native_2.png" } },
] }]);
});
@@ -54,10 +79,35 @@ test("HypiHub Gemini retries upstream rate limits without reuploading files", as
apiKey: "test-key",
maxRateLimitRetries: 2,
rateLimitRetryDelayMs: 1,
- fetch: async (input) => {
+ fetch: async (input, init) => {
const url = String(input);
- if (url.endsWith("/v1/files")) {
+ if (url.endsWith("/v1/files/uploads")) {
uploads += 1;
+ return Response.json({
+ upload_mode: "s3_multipart",
+ upload_id: "up_retry",
+ part_size: 16,
+ part_count: 1,
+ concurrency: 1,
+ });
+ }
+ if (url.endsWith("/v1/files/uploads/up_retry/parts")) {
+ const body = JSON.parse(String(init?.body)) as {
+ readonly parts: readonly { readonly checksum_sha256: string }[];
+ };
+ return Response.json({ parts: [{
+ part_number: 1,
+ url: "https://s3.example/retry?signature=secret",
+ headers: {
+ "content-length": "5",
+ "x-amz-checksum-sha256": body.parts[0]?.checksum_sha256,
+ },
+ }] });
+ }
+ if (url.startsWith("https://s3.example/retry")) {
+ return new Response(null, { headers: { etag: "part" } });
+ }
+ if (url.endsWith("/v1/files/uploads/up_retry/complete")) {
return Response.json({ url: "https://hypit.ai/files/ref" });
}
generations += 1;
diff --git a/packages/provider-hypihub/test/provider.test.ts b/packages/provider-hypihub/test/provider.test.ts
index ad71d5c5..65136900 100644
--- a/packages/provider-hypihub/test/provider.test.ts
+++ b/packages/provider-hypihub/test/provider.test.ts
@@ -110,9 +110,10 @@ test("HypiHub declares VoiceDesign like everything else it can do; who serves it
assert.equal(resolved.status === "resolved" ? resolved.registration.id : undefined, "mimo.official");
});
-test("HypiHub uploads referenced Artifacts once, submits their HTTPS URLs, and persists the result", async () => {
+test("HypiHub uploads one referenced Resource once and submits its HTTPS URL", async () => {
const resources = new MemoryResourceStore();
- const reference = await resources.put(new Uint8Array([1, 2, 3]), "image/png");
+ const referenceBytes = new Uint8Array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]);
+ const reference = await resources.put(referenceBytes, "image/png");
const request = need(sealSeedanceRequest("seedance-2-mini", {
prompt: ["A presenter turns toward camera."],
referenceImage: [
@@ -123,18 +124,62 @@ test("HypiHub uploads referenced Artifacts once, submits their HTTPS URLs, and p
generateAudio: [false], webSearch: [false],
}) as unknown as CanonicalValue);
const calls: string[] = [];
+ const signedBatches: number[][] = [];
+ const partAttempts = new Map<number, number>();
const fakeFetch: typeof globalThis.fetch = async (input, init) => {
const url = String(input);
calls.push(url);
- if (url.endsWith("/v1/files")) {
+ if (url.endsWith("/v1/files/uploads")) {
assert.equal(init?.method, "POST");
assert.equal((init?.headers as Record<string, string>).authorization, "Bearer test-key");
- assert.ok(init?.body instanceof FormData);
- assert.equal(init.body.get("purpose"), "reference");
- const file = init.body.get("file");
- assert.ok(file instanceof File);
- assert.equal(file.type, "image/png");
- assert.deepEqual(new Uint8Array(await file.arrayBuffer()), new Uint8Array([1, 2, 3]));
+ const body = JSON.parse(String(init?.body)) as Record<string, unknown>;
+ assert.equal(body.bytes, referenceBytes.byteLength);
+ assert.equal(body.mime_type, "image/png");
+ assert.equal(body.filename, "reference.png");
+ return Response.json({
+ upload_mode: "s3_multipart",
+ upload_id: "up_reference",
+ part_size: 5,
+ part_count: 3,
+ concurrency: 2,
+ });
+ }
+ if (url.endsWith("/v1/files/uploads/up_reference/parts")) {
+ const body = JSON.parse(String(init?.body)) as {
+ readonly parts: readonly {
+ readonly part_number: number;
+ readonly bytes: number;
+ readonly checksum_sha256: string;
+ }[];
+ };
+ signedBatches.push(body.parts.map((part) => part.part_number));
+ return Response.json({ parts: body.parts.map((part) => ({
+ part_number: part.part_number,
+ url: `https://s3.example/reference-${part.part_number}?signature=secret`,
+ headers: {
+ "content-length": String(part.bytes),
+ "x-amz-checksum-sha256": part.checksum_sha256,
+ },
+ })) });
+ }
+ if (url.startsWith("https://s3.example/reference")) {
+ assert.equal(init?.method, "PUT");
+ const match = /reference-(\d+)/u.exec(url);
+ assert.ok(match);
+ const part = Number(match[1]);
+ const attempt = (partAttempts.get(part) ?? 0) + 1;
+ partAttempts.set(part, attempt);
+ if (part === 2 && attempt === 1) return new Response("retry", { status: 503 });
+ return new Response(null, {
+ headers: {
+ etag: `part-${part}`,
+ "x-amz-checksum-sha256": (init?.headers as Record<string, string>)["x-amz-checksum-sha256"] ?? "",
+ },
+ });
+ }
+ if (url.endsWith("/v1/files/uploads/up_reference/complete")) {
+ const body = JSON.parse(String(init?.body)) as { readonly parts: readonly unknown[] };
+ assert.equal(body.parts.length, 3);
return Response.json({ url: "https://hypit.ai/files/as_reference.png" });
}
if (url.endsWith("/v1/models/bytedance%2Fseedance-2-mini")) {
@@ -168,14 +213,90 @@ test("HypiHub uploads referenced Artifacts once, submits their HTTPS URLs, and p
};
const started = await endpoint.start(common);
assert.equal(started.status, "pending");
- assert.equal(calls.filter((url) => url.endsWith("/v1/files")).length, 1);
+ assert.equal(calls.filter((url) => url.endsWith("/v1/files/uploads")).length, 1);
+ assert.deepEqual(signedBatches, [[1, 2, 3], [2]]);
+ assert.deepEqual([...partAttempts.entries()].sort(), [[1, 1], [2, 2], [3, 1]]);
if (started.status !== "pending") return;
const completed = await endpoint.poll({ ...common, handle: started.handle });
assert.equal(completed.status, "completed");
- assert.equal(calls.length, 6);
});
-test("HypiHub fulfills Gemini through the Runtime endpoint and uploads every media Artifact", async () => {
+test("HypiHub stops before paid submission when a reference upload fails", async () => {
+ const resources = new MemoryResourceStore();
+ const reference = await resources.put(new Uint8Array([1, 2, 3]), "image/png");
+ const request = need(sealSeedanceRequest("seedance-2-mini", {
+ prompt: ["A presenter turns toward camera."],
+ referenceImage: [{ role: "image", artifact: reference }],
+ resolution: ["720p"], aspectRatio: ["16:9"], duration: [5],
+ generateAudio: [false], webSearch: [false],
+ }) as unknown as CanonicalValue);
+ let signingCalls = 0;
+ let uploadAttempts = 0;
+ let cancelled = false;
+ let paidSubmissions = 0;
+ const registry = new EndpointRegistry();
+ await createHypiHubProvider({
+ uploadPartAttempts: 2,
+ requestTimeoutMs: 1_000,
+ fetch: async (input, init) => {
+ const url = String(input);
+ if (url.endsWith("/v1/files/uploads")) {
+ return Response.json({
+ upload_mode: "s3_multipart",
+ upload_id: "up_failure",
+ part_size: 16,
+ part_count: 1,
+ concurrency: 1,
+ });
+ }
+ if (url.endsWith("/v1/files/uploads/up_failure/parts")) {
+ signingCalls += 1;
+ const body = JSON.parse(String(init?.body)) as {
+ readonly parts: readonly { readonly checksum_sha256: string }[];
+ };
+ return Response.json({ parts: [{
+ part_number: 1,
+ url: "https://private.s3.example/reference?signature=must-not-leak",
+ headers: {
+ "content-length": "3",
+ "x-amz-checksum-sha256": body.parts[0]?.checksum_sha256,
+ },
+ }] });
+ }
+ if (url.startsWith("https://private.s3.example/")) {
+ uploadAttempts += 1;
+ throw new Error(`network failure at ${url}`);
+ }
+ if (url.endsWith("/v1/files/uploads/up_failure") && init?.method === "DELETE") {
+ cancelled = true;
+ return Response.json({});
+ }
+ if (url.endsWith("/v1/videos")) {
+ paidSubmissions += 1;
+ return Response.json({ id: "must-not-exist", status: "queued" });
+ }
+ throw new Error(`Unexpected URL ${url}`);
+ },
+ }).install(registry);
+ const resolution = registry.resolve(request);
+ assert.equal(resolution.status, "resolved");
+ assert.equal(resolution.registration.kind, "asynchronous");
+ const outcome = await resolution.registration.endpoint.start({
+ command: { kind: "fulfill-need", id: "command:hypihub-upload-failure", need: request },
+ need: request,
+ resources,
+ credentials: { apiKey: { secret: "test-key" } },
+ operation: "operation:hypihub-upload-failure",
+ });
+ assert.equal(outcome.status, "failed");
+ assert.equal(signingCalls, 2);
+ assert.equal(uploadAttempts, 2);
+ assert.equal(cancelled, true);
+ assert.equal(paidSubmissions, 0);
+ assert.doesNotMatch(outcome.status === "failed" ? outcome.failure.message : "", /must-not-leak/u);
+});
+
+test("HypiHub fulfills Gemini through the Runtime endpoint and uploads every media Resource", async () => {
const resources = new MemoryResourceStore();
const image = await resources.put(new Uint8Array([1, 2, 3]), "image/png");
const video = await resources.put(new Uint8Array([4, 5, 6]), "video/mp4");
@@ -189,17 +310,42 @@ test("HypiHub fulfills Gemini through the Runtime endpoint and uploads every med
}) as unknown as CanonicalValue,
result: "record:hypihub-gemini",
};
- const uploads: string[] = [];
+ let uploadSessions = 0;
+ let completedUploads = 0;
let generationBody: Record<string, unknown> | undefined;
const registry = new EndpointRegistry();
await createHypiHubProvider({ fetch: async (input, init) => {
const url = String(input);
- if (url.endsWith("/v1/files")) {
- assert.ok(init?.body instanceof FormData);
- const file = init.body.get("file");
- assert.ok(file instanceof File);
- uploads.push(file.type);
- return Response.json({ url: `https://hypit.ai/files/${uploads.length}` });
+ if (url.endsWith("/v1/files/uploads")) {
+ uploadSessions += 1;
+ return Response.json({
+ upload_mode: "s3_multipart",
+ upload_id: `up_gemini_${uploadSessions}`,
+ part_size: 16,
+ part_count: 1,
+ concurrency: 2,
+ });
+ }
+ const parts = /\/v1\/files\/uploads\/(up_gemini_\d+)\/parts$/u.exec(url);
+ if (parts) {
+ const body = JSON.parse(String(init?.body)) as {
+ readonly parts: readonly { readonly checksum_sha256: string }[];
+ };
+ return Response.json({ parts: [{
+ part_number: 1,
+ url: `https://s3.example/${parts[1]}?signature=secret`,
+ headers: {
+ "content-length": "3",
+ "x-amz-checksum-sha256": body.parts[0]?.checksum_sha256,
+ },
+ }] });
+ }
+ if (url.startsWith("https://s3.example/up_gemini_")) {
+ return new Response(null, { headers: { etag: "part" } });
+ }
+ if (/\/v1\/files\/uploads\/up_gemini_\d+\/complete$/u.test(url)) {
+ completedUploads += 1;
+ return Response.json({ url: `https://hypit.ai/files/${completedUploads}` });
}
if (url.includes(":generateContent")) {
generationBody = JSON.parse(String(init?.body)) as Record<string, unknown>;
@@ -215,7 +361,8 @@ test("HypiHub fulfills Gemini through the Runtime endpoint and uploads every med
need: request, resources, credentials: { apiKey: { secret: "test-key" } },
});
assert.deepEqual(result.value, { kind: "inline", value: { text: "provider works" } });
- assert.deepEqual(uploads, ["image/png", "video/mp4"]);
+ assert.equal(uploadSessions, 2);
+ assert.equal(completedUploads, 2);
assert.deepEqual((generationBody?.contents as readonly unknown[]), [{ role: "user", parts: [
{ text: "Inspect both references." },
{ fileData: { mimeType: "image/png", fileUri: "https://hypit.ai/files/1" } },
@@ -244,17 +391,43 @@ test("HypiHub fulfills the Provider-neutral WhisperX alignment capability", asyn
if (url.endsWith("/v1/models/victor-upmeet%2Fwhisperx")) {
return Response.json({ endpoints: ["transcriptions"] });
}
+ if (url.endsWith("/v1/files/uploads")) {
+ return Response.json({
+ upload_mode: "s3_multipart",
+ upload_id: "up_whisperx",
+ part_size: 100_000,
+ part_count: 1,
+ concurrency: 1,
+ });
+ }
+ if (url.endsWith("/v1/files/uploads/up_whisperx/parts")) {
+ const body = JSON.parse(String(init?.body)) as {
+ readonly parts: readonly { readonly checksum_sha256: string }[];
+ };
+ return Response.json({ parts: [{
+ part_number: 1,
+ url: "https://s3.example/whisperx?signature=secret",
+ headers: {
+ "content-length": String(bytes.byteLength),
+ "x-amz-checksum-sha256": body.parts[0]?.checksum_sha256,
+ },
+ }] });
+ }
+ if (url.startsWith("https://s3.example/whisperx")) {
+ assert.deepEqual(new Uint8Array(await new Response(init?.body).arrayBuffer()), bytes);
+ return new Response(null, { headers: { etag: "whisperx" } });
+ }
+ if (url.endsWith("/v1/files/uploads/up_whisperx/complete")) {
+ return Response.json({ url: "https://hypit.ai/files/alignment-evidence.wav" });
+ }
if (url.endsWith("/v1/audio/transcriptions")) {
submitted = true;
- assert.ok(init?.body instanceof FormData);
- assert.equal(init.body.get("model"), "victor-upmeet/whisperx");
- assert.equal(init.body.get("language"), "en");
- assert.equal(init.body.get("response_format"), "verbose_json");
- assert.deepEqual(init.body.getAll("timestamp_granularities[]"), ["segment", "word"]);
- const file = init.body.get("file");
- assert.ok(file instanceof File);
- assert.equal(file.type, "audio/wav");
- assert.deepEqual(new Uint8Array(await file.arrayBuffer()), bytes);
+ const body = JSON.parse(String(init?.body)) as Record<string, unknown>;
+ assert.equal(body.model, "victor-upmeet/whisperx");
+ assert.equal(body.language, "en");
+ assert.equal(body.response_format, "verbose_json");
+ assert.equal(body.url, "https://hypit.ai/files/alignment-evidence.wav");
+ assert.deepEqual(body.timestamp_granularities, ["segment", "word"]);
return Response.json({
language: "en",
words: [