Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.
TypeScript agentic-aiaiai-agentsai-videocompilerdslffmpeggenerative-aillmmarkup-languagemonorepoplugin-systemprogramming-languagetext-to-videotypescriptvideovideo-automationvideo-clonevideo-editingvideo-generation
feat(provider): route whisperx alignment through hypihub
rponeb742d6dcb4fe
diff --git a/packages/provider-hypihub/README.md b/packages/provider-hypihub/README.mdindex e7ad763f..6d9b90b1 100644--- a/packages/provider-hypihub/README.md+++ b/packages/provider-hypihub/README.md@@ -41,5 +41,11 @@ Artifacts are uploaded automatically through `POST /v1/files`, then their returnURLs are used in image and video requests. Uploads are deduplicated by Artifact digest within oneRuntime operation. Embedded callers may override that transport with `publicAssetUrl`.+The provider also serves `@hypit/whisperx#whisperx-alignment` through HypiHub's synchronous+`/v1/audio/transcriptions` endpoint using `victor-upmeet/whisperx`. It always requests+`verbose_json` with both segment- and word-level timestamps, then converts HypiHub's OpenAI-shaped+seconds into Hypit's 16 kHz sample-domain `AlignedTranscriptEvidence`. Override the model with+`whisperxModel` only when the HypiHub catalog exposes a compatible replacement.+HypiHub exposes MiMo VoiceDesign by default. Set `audio: false` only when the user explicitly selectsanother VoiceDesign Provider. Hypit does not expose MiMo preset-voice or voice-cloning models.diff --git a/packages/provider-hypihub/package.json b/packages/provider-hypihub/package.jsonindex fc837de1..0c697ccb 100644--- a/packages/provider-hypihub/package.json+++ b/packages/provider-hypihub/package.json@@ -13,7 +13,9 @@"@hypit/protocol": "workspace:*","@hypit/runtime": "workspace:*","@hypit/runtime-kit": "workspace:*",- "@hypit/text": "workspace:*"+ "@hypit/text": "workspace:*",+ "@hypit/speech-evidence": "workspace:*",+ "@hypit/whisperx": "workspace:*"},"devDependencies": {"@hypit/driver-node": "workspace:*",diff --git a/packages/provider-hypihub/src/activation.ts b/packages/provider-hypihub/src/activation.tsindex c1daa94a..dfa1cc86 100644--- a/packages/provider-hypihub/src/activation.ts+++ b/packages/provider-hypihub/src/activation.ts@@ -15,7 +15,7 @@ const adapter = createRuntimeEndpointAdapterFacet({activate(context) {if (context.pool === undefined) throw new Error("HypiHub Provider Pool is required");const config = runtimeConfigObject(context.config, "HypiHub");- runtimeConfigExact(config, ["baseUrl", "apiKey", "defaultConcurrency", "pollIntervalMs", "requestTimeoutMs", "audio"], "HypiHub");+ runtimeConfigExact(config, ["baseUrl", "apiKey", "defaultConcurrency", "pollIntervalMs", "requestTimeoutMs", "audio", "whisperxModel"], "HypiHub");const baseUrl = runtimeConfigString(config.baseUrl, "HypiHub baseUrl");if (baseUrl !== undefined) {const url = new URL(baseUrl);@@ -29,6 +29,7 @@ const adapter = createRuntimeEndpointAdapterFacet({const pollIntervalMs = runtimeConfigPositiveInteger(config.pollIntervalMs, "HypiHub pollIntervalMs");const requestTimeoutMs = runtimeConfigPositiveInteger(config.requestTimeoutMs, "HypiHub requestTimeoutMs");const audio = runtimeConfigBoolean(config.audio, "HypiHub audio");+ const whisperxModel = runtimeConfigString(config.whisperxModel, "HypiHub whisperxModel");return {endpoint: createHypiHubProvider({instance: context.instance,@@ -39,6 +40,7 @@ const adapter = createRuntimeEndpointAdapterFacet({...(pollIntervalMs === undefined ? {} : { pollIntervalMs }),...(requestTimeoutMs === undefined ? {} : { requestTimeoutMs }),...(audio === undefined ? {} : { audio }),+ ...(whisperxModel === undefined ? {} : { whisperxModel }),}),};},diff --git a/packages/provider-hypihub/src/index.ts b/packages/provider-hypihub/src/index.tsindex 1e853459..b016ead7 100644--- a/packages/provider-hypihub/src/index.ts+++ b/packages/provider-hypihub/src/index.ts@@ -2,3 +2,5 @@ export { createHypiHubProvider, hypiHubProviderModuleRef } from "./provider.js";export type { CreateHypiHubProviderOptions } from "./provider.js";export { createHypiHubGeminiGenerator } from "./gemini.js";export type { HypiHubGeminiGenerateInput, HypiHubGeminiGeneratorOptions, HypiHubGeminiPart } from "./gemini.js";+export { interpretHypiHubWhisperXResponse } from "./whisperx.js";+export type { HypiHubWhisperXResponse } from "./whisperx.js";diff --git a/packages/provider-hypihub/src/provider.ts b/packages/provider-hypihub/src/provider.tsindex 34911277..34e9fe18 100644--- a/packages/provider-hypihub/src/provider.ts+++ b/packages/provider-hypihub/src/provider.ts@@ -6,10 +6,13 @@ import { canonicalize } from "@hypit/protocol";import type { BlobRef, CapabilityRef } from "@hypit/protocol";import { credentialRef } from "@hypit/runtime";import type { CredentialRef, ArtifactStore } from "@hypit/runtime";+import { speechEvidenceTypes } from "@hypit/speech-evidence";+import { whisperXCapabilities } from "@hypit/whisperx";import { sealText } from "@hypit/text";import { textTypes } from "@hypit/text";import { createHypiHubGeminiGenerator } from "./gemini.js";import { hypiHubRouteForCapability, hypiHubRoutes } from "./routes.js";+import { transcribeWithHypiHub, whisperXAlignmentRequest } from "./whisperx.js";export const hypiHubProviderModuleRef = { name: "@hypit/provider-hypihub", version: "1" } as const;@@ -23,6 +26,8 @@ export type CreateHypiHubProviderOptions = {readonly requestTimeoutMs?: number;/** Expose HypiHub VoiceDesign. Defaults to enabled; set false only for an explicit alternate Provider. */readonly audio?: boolean;+ /** HypiHub transcription model used for the WhisperX alignment capability. */+ readonly whisperxModel?: string;readonly fetch?: typeof globalThis.fetch;/** Overrides the default POST /v1/files upload for referenced artifacts. */readonly publicAssetUrl?: (artifact: BlobRef, artifacts: ArtifactStore) => Promise<string>;@@ -69,7 +74,7 @@ function jobId(value: Record<string, unknown>): string {return id;}-async function verifyModelRoute(client: HypiHubClient, apiKey: string, model: string, operation: "images" | "image_edits" | "videos" | "audio_speech"): Promise<void> {+async function verifyModelRoute(client: HypiHubClient, apiKey: string, model: string, operation: "images" | "image_edits" | "videos" | "audio_speech" | "transcriptions"): Promise<void> {const card = await client.json(`/models/${encodeURIComponent(model)}`, apiKey);const endpoints = card.endpoints;assert(Array.isArray(endpoints) && endpoints.includes(operation),@@ -141,6 +146,13 @@ class HypiHubClient {"HypiHub file upload returned no HTTPS URL");return response.url;}+ async transcribe(body: Record<string, unknown>, apiKey: string): Promise<Record<string, unknown>> {+ return this.json("/audio/transcriptions", apiKey, {+ method: "POST",+ headers: { "content-type": "application/json" },+ body: JSON.stringify(body),+ });+ }async binary(path: string, apiKey: string, body: Record<string, unknown>): Promise<{ readonly bytes: Uint8Array; readonly mediaType: string }> {const response = await this.fetcher(`${this.baseUrl}${path}`, { method: "POST", headers: { authorization: `Bearer ${apiKey}`, "content-type": "application/json" }, body: JSON.stringify(body) });if (!response.ok) throw new Error(`HypiHub returned HTTP ${response.status}: ${(await response.text()).slice(0, 300)}`);@@ -250,6 +262,24 @@ export function createHypiHubProvider(options: CreateHypiHubProviderOptions = {}const value = await generate({ parts, instruction: request.instruction });return { value: { kind: "inline", value: canonicalize(sealText(value)) } };};+ const whisperxEndpoint: ImmediateEndpointHandler = async (context) => {+ try {+ const request = whisperXAlignmentRequest(context.need.constraints);+ const apiKey = credential(context);+ const model = options.whisperxModel ?? "victor-upmeet/whisperx";+ await verifyModelRoute(client, apiKey, model, "transcriptions");+ const evidence = await transcribeWithHypiHub(+ client,+ request,+ context.artifacts,+ apiKey,+ model,+ );+ return { value: { kind: "inline", value: canonicalize(evidence) } };+ } catch (error) {+ throw new Error(guidedMessage(error), { cause: error });+ }+ };return defineEndpointPackage({module: hypiHubProviderModuleRef, facet: "gateway", instance: options.instance ?? "hypihub.default", pool: options.pool ?? options.instance ?? "hypihub.default",credentials: { apiKey: options.apiKey ?? credentialRef("os", "hypihub.oauth") }, credentialInputs: { apiKey: { label: "HypiHub login" } }, defaultConcurrency: options.defaultConcurrency ?? 3,@@ -266,6 +296,13 @@ export function createHypiHubProvider(options: CreateHypiHubProviderOptions = {}handler: geminiEndpoint,lane: "gemini",})),+ {+ capability: whisperXCapabilities.alignment,+ returns: speechEvidenceTypes.alignedTranscript,+ lifecycle: "immediate" as const,+ handler: whisperxEndpoint,+ lane: "whisperx",+ },],});}diff --git a/packages/provider-hypihub/src/whisperx.ts b/packages/provider-hypihub/src/whisperx.tsnew file mode 100644index 00000000..5d195215--- /dev/null+++ b/packages/provider-hypihub/src/whisperx.ts@@ -0,0 +1,111 @@+import { sealAlignedTranscriptEvidence, speechEvidenceTypes } from "@hypit/speech-evidence";+import type { AlignedTranscriptEvidence, SpeechTranscriptPassage } from "@hypit/speech-evidence";+import type { ArtifactStore } from "@hypit/runtime";+import { whisperXCapabilities } from "@hypit/whisperx";+import type { WhisperXAlignmentRequest } from "@hypit/whisperx";++type RawWord = { readonly word?: unknown; readonly text?: unknown; readonly start?: unknown; readonly end?: unknown; readonly score?: unknown };+type RawSegment = { readonly start?: unknown; readonly end?: unknown; readonly words?: unknown };+export type HypiHubWhisperXResponse = { readonly language?: unknown; readonly segments?: unknown; readonly words?: unknown };++function assert(condition: unknown, message: string): asserts condition {+ if (!condition) throw new Error(message);+}++function finite(value: unknown): value is number {+ return typeof value === "number" && Number.isFinite(value);+}++function sampleWindow(start: unknown, end: unknown, sampleFrames: number) {+ if (!finite(start) || !finite(end) || start < 0 || end < start) return undefined;+ const startSample = Math.round(start * 16_000);+ const endSampleExclusive = Math.round(end * 16_000);+ if (!Number.isSafeInteger(startSample) || !Number.isSafeInteger(endSampleExclusive)+ || startSample > sampleFrames || endSampleExclusive > sampleFrames) return undefined;+ return { startSample, endSampleExclusive };+}++function word(value: unknown) {+ assert(value !== null && typeof value === "object" && !Array.isArray(value), "HypiHub WhisperX word is invalid");+ const item = value as RawWord;+ const text = typeof item.word === "string" ? item.word.trim()+ : typeof item.text === "string" ? item.text.trim() : "";+ if (text.length === 0) return undefined;+ const score = finite(item.score) && item.score >= 0 && item.score <= 1 ? item.score : undefined;+ return {+ text,+ window: sampleWindow(item.start, item.end, Number.MAX_SAFE_INTEGER),+ ...(score === undefined ? {} : { score }),+ start: item.start,+ end: item.end,+ };+}++/** Map HypiHub's OpenAI-shaped verbose response into Hypit's sample-domain evidence. */+export function interpretHypiHubWhisperXResponse(+ response: HypiHubWhisperXResponse,+ sampleFrames: number,+): AlignedTranscriptEvidence {+ assert(Number.isSafeInteger(sampleFrames) && sampleFrames > 0, "WhisperX evidence sample count is invalid");+ assert(Array.isArray(response.segments), "HypiHub WhisperX response has no Segment array");+ const topWords = Array.isArray(response.words) ? response.words.map(word).filter((item): item is NonNullable<ReturnType<typeof word>> => item !== undefined) : [];+ const passages: SpeechTranscriptPassage[] = response.segments.map((value, index) => {+ assert(value !== null && typeof value === "object" && !Array.isArray(value), `HypiHub WhisperX Segment ${index} is invalid`);+ const segment = value as RawSegment;+ const passageWindow = sampleWindow(segment.start, segment.end, sampleFrames);+ const nested = Array.isArray(segment.words)+ ? segment.words.map(word).filter((item): item is NonNullable<ReturnType<typeof word>> => item !== undefined)+ : [];+ const candidates = nested.length > 0 ? nested : topWords.filter((item) => {+ if (!passageWindow || !item.window) return false;+ return item.window.endSampleExclusive > passageWindow.startSample+ && item.window.startSample < passageWindow.endSampleExclusive;+ });+ return {+ ...passageWindow,+ words: candidates.map((item) => ({+ text: item.text,+ ...(item.window === undefined ? {} : sampleWindow(item.start, item.end, sampleFrames)),+ ...(item.score === undefined ? {} : { score: item.score }),+ })),+ chars: [],+ };+ });+ assert(passages.some((passage) => passage.words.length > 0), "HypiHub WhisperX response has no word timestamps");+ return sealAlignedTranscriptEvidence({ passages });+}++export function whisperXAlignmentRequest(value: unknown): WhisperXAlignmentRequest {+ assert(value !== null && typeof value === "object" && !Array.isArray(value), "WhisperX alignment request must be an object");+ const request = value as WhisperXAlignmentRequest;+ assert(request.audio?.kind === "blob" && request.audio.mediaType === "audio/wav"+ && Number.isSafeInteger(request.sampleFrames) && request.sampleFrames > 0+ && (request.language === "en" || request.language === "zh" || request.language === "es"),+ "WhisperX alignment request is invalid");+ return request;+}++export async function transcribeWithHypiHub(+ client: {+ upload(artifact: WhisperXAlignmentRequest["audio"], artifacts: ArtifactStore, apiKey: string): Promise<string>;+ transcribe(body: Record<string, unknown>, apiKey: string): Promise<HypiHubWhisperXResponse>;+ },+ request: WhisperXAlignmentRequest,+ artifacts: ArtifactStore,+ apiKey: string,+ model: string,+): Promise<AlignedTranscriptEvidence> {+ const bytes = await artifacts.get(request.audio.digest);+ assert(bytes !== undefined && bytes.byteLength === request.audio.size,+ `HypiHub WhisperX evidence Artifact ${request.audio.digest} is unavailable or has changed`);+ const url = await client.upload(request.audio, artifacts, apiKey);+ return interpretHypiHubWhisperXResponse(await client.transcribe({+ model,+ url,+ language: request.language,+ response_format: "verbose_json",+ timestamp_granularities: ["word", "segment"],+ }, apiKey), request.sampleFrames);+}++export { speechEvidenceTypes, whisperXCapabilities };diff --git a/packages/reference-video-tools/README.md b/packages/reference-video-tools/README.mdindex 80d61e66..95cedb17 100644--- a/packages/reference-video-tools/README.md+++ b/packages/reference-video-tools/README.md@@ -92,13 +92,14 @@ rerun only the selected full-reference stages.It also produces `transcript`: the verbatim speech of the whole reference with a start and an end forevery single word. Placing an on-screen text reveal against the line that triggers it needs the timeof the word, not of the sentence around it, and that question comes up in every reconstruction. The-speech audio is extracted to `speech.wav` beside the shot media and measured by the local WhisperX-Provider in `@hypit/provider-whisperx-local`, which answers a loopback service on-`http://127.0.0.1:8765`; start the selected managed service with `hypit runtime up`.+speech audio is extracted to `speech.wav` beside the shot media and measured by the HypiHub WhisperX+Provider in `@hypit/provider-hypihub`, which calls the synchronous+`/v1/audio/transcriptions` route with word and segment timestamps. The selected HypiHub OAuth+credential must be available before preparation.The result reports `status`, `transcript_ref` and `word_count`, and the words themselves live in`transcript.json` as passages, each with a `words` array of `{ text, start_seconds, end_seconds, score }`.The transcript is deterministic local evidence rather than an observation: it is never written to the-observation cache and nothing about it is sent to Gemini. A machine with no WhisperX service running+observation cache and nothing about it is sent to Gemini. An unavailable HypiHub WhisperX endpointreports `status: "unavailable"` with the reason and prepares everything else.`observe_reference` observes every unfinished shot, or only the shots named by `--shot-id`. Each shotdiff --git a/packages/reference-video-tools/package.json b/packages/reference-video-tools/package.jsonindex 82c5b7ff..aece8a60 100644--- a/packages/reference-video-tools/package.json+++ b/packages/reference-video-tools/package.json@@ -28,7 +28,6 @@"@hypit/protocol": "workspace:*","@hypit/run-markup": "workspace:*","@hypit/runtime": "workspace:*",- "@hypit/provider-whisperx-local": "workspace:*","@hypit/provider-media-local": "workspace:*","@hypit/provider-hypihub": "workspace:*","@hypit/provider-vertex": "workspace:*",diff --git a/packages/reference-video-tools/src/transcript.ts b/packages/reference-video-tools/src/transcript.tsindex b9418575..360bb11a 100644--- a/packages/reference-video-tools/src/transcript.ts+++ b/packages/reference-video-tools/src/transcript.ts@@ -2,7 +2,9 @@ import { access } from "node:fs/promises";import { join } from "node:path";import { EndpointRegistry, MemoryArtifactStore } from "@hypit/driver-node";import type { Need } from "@hypit/protocol";-import { createLocalWhisperXProvider } from "@hypit/provider-whisperx-local";+import { createHypiHubProvider } from "@hypit/provider-hypihub";+import { OsCredentialStore } from "@hypit/credential-store-os";+import { credentialRef } from "@hypit/runtime";import { sealSpeechEvidenceAudio } from "@hypit/speech";import { speechEvidenceTypes } from "@hypit/speech-evidence";import type { AlignedTranscriptEvidence } from "@hypit/speech-evidence";@@ -13,26 +15,36 @@ import { assert, command, readBytes, round, writeJson } from "./media.js";import type { Transcript, TranscriptFile, TranscriptPassage, TranscriptWord } from "./types.js";/**- * Whether the local WhisperX service answers.+ * Whether the HypiHub WhisperX endpoint answers.** The route used to probe the service by hand, as three commands, before `prepare_reference`. The- * service answers `/health`, which is the same call the Provider makes before transcribing — made- * early enough to be reported before the minutes a first start spends loading the model. It reports- * rather than refuses: a machine without the service still prepares everything except the transcript,- * which `prepareTranscript` leaves recoverable.+ * The model-card probe is made early enough to report missing credentials or model routing before+ * preparation spends time extracting audio. It reports rather than refuses: an unavailable endpoint+ * still prepares everything except the transcript, which `prepareTranscript` leaves recoverable.*/-export async function whisperxHealth(baseUrl = "http://127.0.0.1:8765"): Promise<{ readonly ok: boolean; readonly reason: string }> {- const normalized = baseUrl.replace(/\/+$/u, "");+async function hypiHubApiKey(): Promise<string | undefined> {+ if (process.platform === "darwin" || process.platform === "win32") {+ const value = await new OsCredentialStore().resolve(credentialRef("os", "hypihub.oauth")).catch(() => undefined);+ const secret = value?.secret?.trim();+ if (secret) return secret;+ }+ return process.env.HYPIHUB_API_KEY?.trim() || undefined;+}++export async function whisperxHealth(baseUrl = process.env.HYPIHUB_BASE_URL?.trim() || "https://hypit.ai"): Promise<{ readonly ok: boolean; readonly reason: string }> {+ const normalized = `${baseUrl.replace(/\/+$/u, "").replace(/\/(?:v1beta|v1)$/iu, "")}/v1`;+ const apiKey = await hypiHubApiKey();+ if (!apiKey) return { ok: false, reason: "HypiHub WhisperX credentials are unavailable; run hypit auth login hypihub.default" };try {- const response = await fetch(`${normalized}/health`, { signal: AbortSignal.timeout(5_000) });+ const response = await fetch(`${normalized}/models/${encodeURIComponent("victor-upmeet/whisperx")}`, {+ headers: { authorization: `Bearer ${apiKey}` }, signal: AbortSignal.timeout(5_000),+ });if (!response.ok) return { ok: false, reason: `WhisperX answered ${response.status}` };- const body = await response.json().catch(() => undefined) as { readonly ok?: unknown } | null | undefined;- if (body === null || typeof body !== "object" || body.ok !== true) {- return { ok: false, reason: "WhisperX is up but not ready; a first start spends a few minutes loading the model" };- }+ const body = await response.json().catch(() => undefined) as { readonly endpoints?: unknown } | null | undefined;+ if (body === null || typeof body !== "object" || !Array.isArray(body.endpoints) || !body.endpoints.includes("transcriptions")) return { ok: false, reason: "HypiHub WhisperX model does not expose transcriptions" };return { ok: true, reason: "" };} catch {- return { ok: false, reason: `WhisperX is not reachable at ${normalized}` };+ return { ok: false, reason: `HypiHub WhisperX is not reachable at ${normalized}` };}}@@ -69,7 +81,7 @@ function reason(error: unknown): string {return error.cause === undefined ? error.message : `${error.message}: ${reason(error.cause)}`;}-/** Ask the local WhisperX Provider for the word times of one canonical evidence WAV. */+/** Ask the HypiHub WhisperX Provider for the word times of one canonical evidence WAV. */export async function transcribeSpeechAudio(audioPath: string,language: WhisperXLanguage,@@ -86,15 +98,19 @@ export async function transcribeSpeechAudio(result: "record:reference-video-transcript",};const registry = new EndpointRegistry();- await createLocalWhisperXProvider({ instance: "whisperx.local", pool: "whisperx.local" }).install(registry);+ await createHypiHubProvider({+ baseUrl: process.env.HYPIHUB_BASE_URL?.trim() || "https://hypit.ai",+ }).install(registry);const resolved = registry.resolve(need);- assert(resolved.status === "resolved", "the local WhisperX Provider does not offer word alignment");- assert(resolved.registration.kind === "immediate", "the local WhisperX Provider answers word alignment asynchronously");+ assert(resolved.status === "resolved", "the HypiHub Provider does not offer WhisperX word alignment");+ assert(resolved.registration.kind === "immediate", "the HypiHub Provider answers WhisperX alignment asynchronously");+ const apiKey = await hypiHubApiKey();+ assert(apiKey !== undefined, "HypiHub WhisperX credentials are unavailable; run hypit auth login hypihub.default");const fulfillment = await resolved.registration.handler({command: { kind: "fulfill-need", id: "command:reference-video-transcript", need },need,artifacts,- credentials: {},+ credentials: { apiKey: { secret: apiKey } },});assert(fulfillment.value.kind === "inline", "WhisperX returned alignment evidence by reference");const aligned = fulfillment.value.value as unknown as AlignedTranscriptEvidence;@@ -139,7 +155,7 @@ export async function prepareTranscript(if (redo || await access(audioPath).then(() => false, () => true)) await extractSpeechAudio(videoPath, audioPath);// A refused connection says only "fetch failed"; the reader still has to learn what to start.const passages = await transcribeSpeechAudio(audioPath, language)- .catch((error: unknown) => { throw new Error(`local WhisperX did not answer: ${reason(error)}`); });+ .catch((error: unknown) => { throw new Error(`HypiHub WhisperX did not answer: ${reason(error)}`); });const transcriptPath = join(root, "transcript.json");const file: TranscriptFile = { reference_id: reference, audio_ref: audioPath, passages };await writeJson(transcriptPath, file);diff --git a/packages/whisperx/README.md b/packages/whisperx/README.mdindex 45c2ec25..37621057 100644--- a/packages/whisperx/README.md+++ b/packages/whisperx/README.md@@ -1,8 +1,9 @@# @hypit/whisperxExplicit WhisperX model-family capability for the official speech program. Importing this package-selects WhisperX; Runtime registration only binds the resulting alignment Need to-a concrete WhisperX execution endpoint such as a trusted local worker or hosted service.+selects WhisperX; Runtime registration only binds the resulting alignment Need to a concrete+execution endpoint. The default Hypit Skill path uses the HypiHub-hosted WhisperX endpoint; the+trusted local worker remains an explicit fallback.The package contains no credentials, Python environment or queue. Providers translate the typedrequest directly into provider-neutral `AlignedTranscriptEvidence`. There is no vendor-shaped@@ -18,5 +19,6 @@ the returned evidence with the one Segment and emits one self-contained `SemantiThere is no whole-program WhisperX pass. Speech Track only receives already-semantic Takes and latertranslates their local frames when assembling the final ProgramSpace and complete semantic map.-`@hypit/provider-whisperx-local` is the first concrete adapter; it validates and stages those bytes-unchanged for a warm loopback service.+`@hypit/provider-hypihub` is the default concrete adapter; it uploads the canonical evidence audio+and requests verbose JSON with segment- and word-level timestamps. `@hypit/provider-whisperx-local`+remains available as an explicit local fallback.diff --git a/pnpm-lock.yaml b/pnpm-lock.yamlindex fda9fa02..1e38ea8d 100644--- a/pnpm-lock.yaml+++ b/pnpm-lock.yaml@@ -2032,9 +2032,15 @@ importers:'@hypit/runtime-kit':specifier: workspace:*version: link:../runtime-kit+ '@hypit/speech-evidence':+ specifier: workspace:*+ version: link:../speech-evidence'@hypit/text':specifier: workspace:*version: link:../text+ '@hypit/whisperx':+ specifier: workspace:*+ version: link:../whisperxdevDependencies:'@hypit/driver-node':specifier: workspace:*@@ -2449,9 +2455,6 @@ importers:'@hypit/provider-vertex':specifier: workspace:*version: link:../provider-vertex- '@hypit/provider-whisperx-local':- specifier: workspace:*- version: link:../provider-whisperx-local'@hypit/run-markup':specifier: workspace:*version: link:../run-markupdiff --git a/skills/hypit/SKILL.md b/skills/hypit/SKILL.mdindex ac5a035c..c1f80699 100644--- a/skills/hypit/SKILL.md+++ b/skills/hypit/SKILL.md@@ -69,8 +69,9 @@ Runtime Profile, resolved credentials for the capabilities the route will use, amanaged program installed and healthy. Credential setup is part of this gate: default to HypiHub OAuthwithout asking the author to choose, run the login yourself when the selected HypiHub OS credential ismissing, and continue only after it succeeds. Only when the author explicitly asks to use their own-provider key may you configure that less-recommended path instead. Run the required `hypit runtime up`-provisioning and health checks; WhisperX must report `Ready whisperx` (or an equivalent successful probe). Do this before+provider key may you configure that less-recommended path instead. Run the required Runtime provisioning+and health checks; the HypiHub WhisperX alignment model must be reachable through the selected Endpoint+and the profile must report alignment ready. Do this beforeinspecting media or examples, preparing a reference, choosing an observer, freezing a brief, readingvocabulary, writing Source, previewing, or building.diff --git a/skills/hypit/references/environment.md b/skills/hypit/references/environment.mdindex 90cd971d..5a893cc5 100644--- a/skills/hypit/references/environment.md+++ b/skills/hypit/references/environment.md@@ -28,9 +28,9 @@ This document must be read completely before any video-production route continuecomplete only when all of the following are true: a Hypit Distribution is selected; the projectdirectory and its package boundary are established; credentials for every capability the route willuse are resolved; the project's Runtime Profile is selected; `hypit runtime up` has finished installing-and starting every selected managed program; and every required health check passes. The local-WhisperX Endpoint is mandatory for video production and must report `Ready whisperx` (or an equivalent-successful probe).+and starting every selected managed program; and every required health check passes. The HypiHub+WhisperX alignment Endpoint is mandatory for video production and its configured transcription model+must be reachable through the selected HypiHub credential.Do not inspect media or examples, prepare or observe a reference, freeze a brief, inspect vocabulary,write Source, preview, or Build until this checklist is complete. Do not substitute another@@ -41,7 +41,8 @@ for signing in and does not enter a production route.## Credentials are part of environment setupResolve credentials immediately after selecting the Runtime Profile and before inspecting any media,-examples or references. HypiHub OAuth is the default for paid models and Gemini VLM. Check the selected+examples or references. HypiHub OAuth is the default for paid models, Gemini VLM and WhisperX+alignment. Check the selectedEndpoint:```text@@ -159,10 +160,10 @@ Reconstructing a video given as a link rather than as a file needs `uv` too: `yt`services/yt-dlp` and run from there, so the version is the repository's rather than the machine's.`prepare_reference` names what is missing when a link is passed without it.-When the Runtime Profile selects the local WhisperX Endpoint, `prepare_reference` probes the service's-health itself before transcribing, because a first start can spend several minutes loading the model.-A service that does not answer is reported in the command's result with a pointer to the repair rather-than silently producing an empty transcript.+When the Runtime Profile selects the HypiHub WhisperX Endpoint, the provider verifies that the+configured transcription model exposes the `transcriptions` route before submitting the alignment+request. A model that is unavailable is reported as a credential/model routing error rather than+silently producing an empty transcript.`host-setup.md` holds the host toolchain installations and repairs, and is read when a commandreports that a service or a binary is unavailable.diff --git a/skills/hypit/references/host-setup.md b/skills/hypit/references/host-setup.mdindex b962ab86..97e3367e 100644--- a/skills/hypit/references/host-setup.md+++ b/skills/hypit/references/host-setup.md@@ -78,7 +78,9 @@ do not improvise a Python `pip install` for this managed-program workflow.## Let Hypit install and run WhisperX-The project Runtime Profile must select the local WhisperX Endpoint. A minimal endpoint inside the+The project Runtime Profile may select the local WhisperX Endpoint only when the author explicitly+chooses the local fallback. The default Skill path uses HypiHub's WhisperX alignment Endpoint. A+minimal local endpoint inside theprofile's `runtime.config.endpoints` is:```json@@ -103,7 +105,7 @@ service on `127.0.0.1:8765`, and then starts the Worker. The first start may dowWhisper and language-alignment model weights, so it can take substantially longer than later starts.Do not run `uv sync` in an author project and do not install the `whisperx` Python package by hand.-`runtime up` printing `Ready whisperx` reports that the service answered its health probe.+`runtime up` printing `Ready whisperx` reports that the local service answered its health probe.`runtime up` probes each selected program. It creates that program's environment only when absent orunhealthy, then reuses it across projects and sessions. It never runs `uv sync` merely because a newdiff --git a/skills/hypit/references/original-authoring/route.md b/skills/hypit/references/original-authoring/route.mdindex d29882b5..268458aa 100644--- a/skills/hypit/references/original-authoring/route.md+++ b/skills/hypit/references/original-authoring/route.md@@ -29,10 +29,11 @@ component choices and package gaps are decided globally there before any variantThis route has a hard environment gate. At route entry read `../environment.md` completely and finishits environment steps (Distribution, project boundary, credentials, Runtime Profile, all selectedmanaged programs, and health checks) before entering the brief, example, vocabulary, Source, preview,-or Build steps. The local WhisperX Endpoint is one mandatory item and must report `Ready whisperx` (or-an equivalent successful health check). If no profile exists yet, create a minimal profile with local-media and WhisperX endpoints, select it, and provision it before continuing; extend that same profile-later. Do not proceed from a partial or unconfirmed environment, substitute another transcriber, or+or Build steps. The HypiHub WhisperX alignment Endpoint is one mandatory item and its configured model+must be reachable with the completed HypiHub OAuth credential. If no profile exists yet, create a+minimal profile with media, HypiHub and WhisperX alignment endpoints, select it, and provision it+before continuing; extend that same profile later. Do not proceed from a partial or unconfirmed+environment, substitute another transcriber, orcontinue because the user asks to bypass setup.## Checkpoint and recoverydiff --git a/skills/hypit/references/reconstruction/evidence.md b/skills/hypit/references/reconstruction/evidence.mdindex 42015fb1..4a65b5e6 100644--- a/skills/hypit/references/reconstruction/evidence.md+++ b/skills/hypit/references/reconstruction/evidence.md@@ -43,7 +43,7 @@ enough detail to draw from the words alone — which is what reconstructing a losince reference frames are never fed to generation.`transcript` is the verbatim speech of the whole reference with a start and an end for every single-word, measured locally by WhisperX. It is not an observation: no model wrote it, nothing about it+word, measured by the selected HypiHub WhisperX alignment Provider. It is not an observation: no model wrote it, nothing about itreaches an observer, and it does not go in the observation cache. It is identical on both observers,and on the `agent` observer it is the only exact record of the sound. Read it whenever a decisiondepends on when a word is said — placing an on-screen text reveal against the line that triggers it,@@ -54,11 +54,11 @@ to transcribe.`transcript` reports `status`, `transcript_ref` and `word_count`. Read the words from`transcript_ref`, a JSON file of passages, each with its own `words` array of-`{ text, start_seconds, end_seconds, score }`. A machine with no WhisperX service running reports+`{ text, start_seconds, end_seconds, score }`. If the HypiHub WhisperX model is unavailable, the tool reports`status: "unavailable"` with a `reason`. Under the Skill's hard environment gate, do not continue from-that result: run `hypit runtime up` for the project's selected Profile, confirm the service health, and-prepare the reference again only after it reports ready. `../host-setup.md` covers diagnosing that-service when starting it is not enough.+that result: confirm HypiHub OAuth and that the selected model exposes `transcriptions`, then prepare+the reference again. `../host-setup.md` covers the explicitly selected local fallback when it is not+enough to rely on the remote endpoint.When a completed preparation stage must be rerun, use one small `--redo` value:diff --git a/skills/hypit/references/reconstruction/observers.md b/skills/hypit/references/reconstruction/observers.mdindex ae46a946..1170a098 100644--- a/skills/hypit/references/reconstruction/observers.md+++ b/skills/hypit/references/reconstruction/observers.md@@ -113,7 +113,7 @@ You cannot hear the reference, and there is no second source of sound to fall baobservations are answered from the two things you do have. Answer them fully; do not leave them shortand do not put the gap to the author.-- **The transcript says when.** WhisperX measures every word locally on both paths, so the words and+- **The transcript says when.** The selected HypiHub WhisperX Provider measures every word on both paths, so the words andtheir timings are exact whichever observer reads the pictures. That settles whether anyone isspeaking at a given moment, and where a word sits. It does not set a take's duration: the take isgenerated and its length comes from `estimate:Speech`, as `../script-time.md` requires.diff --git a/skills/hypit/references/reconstruction/route.md b/skills/hypit/references/reconstruction/route.mdindex c0712ad2..8691ff74 100644--- a/skills/hypit/references/reconstruction/route.md+++ b/skills/hypit/references/reconstruction/route.md@@ -13,8 +13,9 @@ systems. Examples belong to original authoring and variant planning, not reconstThe complete environment is the hard gate. At route entry, read `../environment.md` completely, thenfinish its checklist through the environment steps: Distribution, project boundary, credentials,-Runtime Profile, all selected managed programs, and health checks. The local WhisperX Endpoint is-mandatory and must report `Ready whisperx` (or an equivalent successful probe). Do not enter the+Runtime Profile, all selected managed programs, and health checks. The HypiHub WhisperX alignment+Endpoint is mandatory and its configured model must be reachable with the completed HypiHub OAuth+credential. Do not enter theobservation or authoring steps while any environment item is missing, installing, unhealthy, orunconfirmed.@@ -172,7 +173,7 @@ for them again. HypiHub OAuth is the default; only configure an author-owned keyand this machine does not hold is named. If HypiHub OAuth is not configured, complete that login beforecontinuing; do not use `agent` to bypass the environment gate.-Create the project's `hypit.runtime.json` now with at least the local media and WhisperX endpoints,+Create the project's `hypit.runtime.json` now with at least the local media and HypiHub WhisperX endpoints,then select and start it before reference preparation:```bash@@ -224,9 +225,9 @@ hypit-reference-video-tools prepare_reference --video-path <path or link> --obse`--video-path` takes a link as readily as a path. The result names the link it fetched under`source_url`, which is the only record of which video was reconstructed once the bytes are on disk.-Verify the WhisperX service answered its health probe before running this — a first start can spend-several minutes loading the model. `../environment.md` says how; `../host-setup.md` covers the-failure branches.+Verify the HypiHub WhisperX model answers its model-card probe before running this. The selected+Runtime Profile and OAuth credential are the source of truth; `../environment.md` says how and+`../host-setup.md` covers the local fallback's failure branches.**Read now:** `evidence.md` — what the four whole-reference observations cover, what `transcript` isand how to recover it when it reports `status: "unavailable"`, and the `--redo` values.diff --git a/skills/hypit/references/revision/route.md b/skills/hypit/references/revision/route.mdindex 1ab22bea..ac9c9a48 100644--- a/skills/hypit/references/revision/route.md+++ b/skills/hypit/references/revision/route.md@@ -19,8 +19,9 @@ authoritative.Before any revision command or project inspection, read `../environment.md` completely and finish itsenvironment checklist. The selected Distribution, project boundary, credentials, Runtime Profile,-selected managed programs and their health checks must all be confirmed; the local WhisperX Endpoint-must report `Ready whisperx` (or an equivalent successful probe). Do not begin a revision from a+selected managed programs and their health checks must all be confirmed; the HypiHub WhisperX+alignment Endpoint and its configured model must be reachable with the completed HypiHub OAuth+credential. Do not begin a revision from apartial, installing, unhealthy or unconfirmed environment, even when the change itself appearsdeterministic or the user asks to bypass setup.diff --git a/skills/hypit/references/runtime.md b/skills/hypit/references/runtime.mdindex 4907e93d..29495ed4 100644--- a/skills/hypit/references/runtime.md+++ b/skills/hypit/references/runtime.md@@ -57,6 +57,11 @@ one conservative lane. Bind every paid capability HypiHub supports to this Endpoincluding MiMo VoiceDesign. Select KIE, Vertex, official MiMo or another BYOK Provider only when theauthor explicitly asks not to use HypiHub or explicitly selects that Provider.+Bind `@hypit/whisperx#whisperx-alignment` to a HypiHub Endpoint in the same Profile. It uses the same+`hypihub.oauth` credential and defaults to `victor-upmeet/whisperx`. Keep+`@hypit/provider-whisperx-local` only as an explicitly selected fallback; do not add it to the default+Skill-created Profile.+An Author Source importing `@hypit/gemini` remains Provider-neutral. Bind those capabilities to`@hypit/provider-hypihub` for HypiHub's upload-backed Gemini endpoint, or to`@hypit/provider-vertex` for direct Vertex. Never make this Build-time choice with@@ -66,8 +71,9 @@ observer, which runs outside a Build.Provider-specific input limits are also part of preflight. For the KIE GPT Image 2 route, do notsubmit `4:3`, `3:4` or `4:5`; the route rejects those aspect ratios before upload or paid submission.-The local Providers stay small: `media`, `whisperx` and `hyperframes` are bounded by this machine's-cores rather than by a remote queue, and raising them buys contention.+The local Providers stay small: `media` and `hyperframes` are bounded by this machine's cores rather+than by a remote queue. WhisperX alignment uses the HypiHub lane in the default Skill path; the local+WhisperX Provider remains available only when an author explicitly selects it.Set this while writing the Profile. Changing it later needs a Worker restart, and the restart rulebelow makes that costly once a Build is in flight.diff --git a/skills/hypit/references/variant-expansion/route.md b/skills/hypit/references/variant-expansion/route.mdindex 9a59875a..e14bf972 100644--- a/skills/hypit/references/variant-expansion/route.md+++ b/skills/hypit/references/variant-expansion/route.md@@ -12,8 +12,8 @@ approval; context recovery is never approval to repeat it.Before discovering or copying a batch, read `../environment.md` completely and finish its environmentchecklist. The selected Distribution, project boundary, credentials, Runtime Profile, selected managed-programs and health checks must be confirmed; the local WhisperX Endpoint must report `Ready whisperx`-(or an equivalent successful probe). Do not treat source-only delivery or an existing completed base+programs and health checks must be confirmed; the HypiHub WhisperX alignment Endpoint and its+configured model must be reachable with the completed HypiHub OAuth credential. Do not treat source-only delivery or an existing completed baseas a reason to skip this gate.## Recovery comes first