Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.
TypeScript agentic-aiaiai-agentsai-videocompilerdslffmpeggenerative-aillmmarkup-languagemonorepoplugin-systemprogramming-languagetext-to-videotypescriptvideovideo-automationvideo-clonevideo-editingvideo-generation
merge: align coarse-fine rebuild with origin/main
kashorin5ec6beee7a2a
diff --cc packages/provider-hypihub/test/provider.test.tsindex df232201,4633f609..c24eed85--- a/packages/provider-hypihub/test/provider.test.ts+++ b/packages/provider-hypihub/test/provider.test.ts@@@ -223,12 -118,12 +223,12 @@@ test("HypiHub uploads referenced Artifa});test("HypiHub fulfills Gemini through the Runtime endpoint and uploads every media Artifact", async () => {- const artifacts = new MemoryArtifactStore();- const image = await artifacts.put(new Uint8Array([1, 2, 3]), "image/png");- const video = await artifacts.put(new Uint8Array([4, 5, 6]), "video/mp4");+ const resources = new MemoryResourceStore();+ const image = await resources.put(new Uint8Array([1, 2, 3]), "image/png");+ const video = await resources.put(new Uint8Array([4, 5, 6]), "video/mp4");const request: Need = {id: "need:hypihub-gemini",- capability: geminiCapabilities["gemini-3.7-flash-openai"],+ capability: geminiCapabilities["gemini-3.1-pro"],returns: textTypes.text,constraints: sealGeminiRequest({instruction: "Answer briefly.", prompt: "Inspect both references.",diff --cc packages/reference-video-tools/README.mdindex 332d0d50,66c0f3dd..6df15e17--- a/packages/reference-video-tools/README.md+++ b/packages/reference-video-tools/README.md@@@ -21,64 -56,87 +21,76 @@@ new explicit observation; it is not sup```bashhypit-reference-video-tools list_svml_packages-hypit-reference-video-tools prepare_reference --video-path ./reference.mp4-hypit-reference-video-tools prepare_reference --video-path ./reference.mp4 --redo people-hypit-reference-video-tools prepare_reference --video-path ./reference.mp4 --redo transcript+hypit-reference-video-tools prepare_reference --video-path ./reference.mp4 --observer agenthypit-reference-video-tools observe_reference --reference-id <reference-id>-hypit-reference-video-tools observe_reference --reference-id <reference-id> --shot-id shot-007-hypit-reference-video-tools observe_reference --reference-id <reference-id> --shot-id shot-007 --question "How thick is the outline on the caption words?"-hypit-reference-video-tools observe_reference --reference-id <reference-id> --shot-id shot-007 --reobserve+hypit-reference-video-tools record_observation --reference-id <reference-id> --key <key> --text-file ./answer.mdhypit-reference-video-tools inspect_svml_vocabulary --package @hypit/media-track --tag Track-hypit-reference-video-tools compare_reconstruction --reference-id <reference-id> --shot-id shot-007 --image ./rendered.png-hypit-reference-video-tools variant_state --action discover --project-root ./projects/base-hypit-reference-video-tools variant_state --action start --project-root ./projects/base --output-root ./projects/base-variants/<batch-id> --count 100-hypit-reference-video-tools variant_init --project-root ./projects/base --output-root ./projects/base-variants/<batch-id> --slate ./slate.json-hypit-reference-video-tools variant_check --run ./projects/base-variants/<batch-id>/001-example/build.svrun+hypit-reference-video-tools validate_local_author_packages --run ./build.svrun+hypit-reference-video-tools validate_script_cues --run ./build.svrun+hypit-reference-video-tools preview_check ./build.svrun+hypit-reference-video-tools layout_check --run ./build.svrun+hypit-reference-video-tools render_element ./build.svrun --element title --segment intro --out ./title.mp4+hypit-reference-video-tools compare_reconstruction --reference-id <reference-id> --run ./build.svrun --segment intro --video ./title.mp4 --element title+hypit-reference-video-tools review_element --run ./build.svrun --element title --segment intro --video ./title.mp4 --intent-file ./title-intent.md+hypit-reference-video-tools reconstruction_check ./build.svrun --reference-id <reference-id>+hypit-reference-video-tools authoring_check ./build.svrun```-For automation, every command also accepts `--input '{"...":"..."}'` with the complete JSON input-object. This is useful when the calling agent already has a structured request.+Every command prints one JSON result. Commands also accept `--input '{...}'` when structured input is+more convenient. Batch forms read JSON arrays from files for observations, comparisons, renders,+reviews and layout decisions.++## Reference preparation-`list_svml_packages` reports every installed package that declares an activation, with the Surface-tags it registers. `inspect_svml_vocabulary` reads packages you can already name, and the Build CLI-is deliberately unable to scan a directory, so this is how a caller finds out what vocabulary exists-instead of working from memory.+`prepare_reference` accepts a local path or a supported video URL. It extracts bounded shot clips,+representative frames, a storyboard, speech audio and a word-timed transcript. The returned reference+id contains the source filename, byte length and modification time so it is inspectable without a+lookup table.-`prepare_reference` requires a local video path and stores derived media under the checkout's-gitignored `.hypit/reference-video-tools/` directory, which is where every other command reads it-from whichever directory it is run in. It produces four full-reference observations: `people_and_product`,-`voices`, `persistent_systems` — the on-screen text and graphic systems that continue or recur across-the whole video, each one's lifetime, and whether its appearance ever changes — and `places`: how-many locations the video was shot in, which camera positions appear in each, which parts of the video-use each one, and each position described in enough detail to draw from the words alone. Without `--redo`, completed stages are reused. Use `--redo media` to rebuild shot media and-clear derived observations, `--redo transcript` to measure the words again, or `--redo people`, `--redo voices`, `--redo systems`, `--redo places`, or `--redo all` to-rerun only the selected full-reference stages.+`--observer gemini` uses HypiHub or Vertex according to the configured provider. `--observer agent`+calls no visual provider; it returns observation tasks with local pictures and transcript context for+the calling agent to answer through `record_observation`.+ It also produces `transcript`: the verbatim speech of the whole reference with a start and an end for+ every single word. Placing an on-screen text reveal against the line that triggers it needs the time+ of the word, not of the sentence around it, and that question comes up in every reconstruction. The-speech audio is extracted to `speech.wav` beside the shot media and measured by the HypiHub WhisperX-Provider in `@hypit/provider-hypihub`, which calls the synchronous-`/v1/audio/transcriptions` route with word and segment timestamps. The selected HypiHub OAuth-credential must be available before preparation.++speech audio is extracted to `speech.wav` beside the shot media and measured through the++`@hypit/whisperx#whisperx-alignment` capability. This tool currently instantiates the local WhisperX++Provider directly; routing it through the selected Runtime Profile is pending.+ The result reports `status`, `transcript_ref` and `word_count`, and the words themselves live in+ `transcript.json` as passages, each with a `words` array of `{ text, start_seconds, end_seconds, score }`.+ The transcript is deterministic local evidence rather than an observation: it is never written to the-observation cache and nothing about it is sent to Gemini. An unavailable HypiHub WhisperX endpoint-reports `status: "unavailable"` with the reason and prepares everything else.--`observe_reference` observes every unfinished shot, or only the shots named by `--shot-id`. Each shot-produces three observations — `visual` (base picture, covering content, continuity, whether the frame-is a depicted scene or a flat designed field, and each framed element's inner picture separately from-its frame), `text_appearance` (typeface, weight, size, spacing, colour, stroke, shadow, glow,-per-word emphasis, position and entry of drawn text), and `audio`. The picture observation also-states whether the picture moves and how, separating camera movement from movement within it, since a-held still and a moving shot are reconstructed differently. Each cut adds one `continuity`-observation covering both whether the two shots are one continuous camera shot and whether an overlay-continues across the boundary. The preceding tail frame, audio tail and full-reference context are-attached automatically. Completed observations are reused; `--reobserve` is the only way to run a-completed one again.--Shots are never shorter than one second: a fragment below that is merged into the shot before it,-because clips that short carry no evidence and the model rejects them outright.--`HYPIT_REFERENCE_CONCURRENCY` and `HYPIT_REFERENCE_LAUNCH_GAP_MS` set how fast requests are issued,-defaulting to four at a time with a 1.5 second gap. They describe the quota behind the credentials-rather than anything about the video, so they are environment settings and not flags. Lower them if a-quota objects; the failure to expect is rate limiting, which backs off on its own before giving up.--`--question` is a separate narrow path: it requires exactly one `--shot-id`, answers only that-question from that shot's clip and frames in a single request, and neither reads nor writes the-observation cache. Cross-shot continuity uses the built-in boundary and three-shot window observations.--`compare_reconstruction` sends the named shot's reference frame and a rendered image as an-unlabelled pair and returns a description of their visible differences. It is never told which image-is which, what was built, or how. `--question` narrows it to one region of the picture and nothing-else. Results are not cached, because the rendered side changes on every iteration.--`unresolved` lists the keys of observations that failed. It is not a judgement about evidence-quality: a complete observation that says "unclear" is still complete.--Gemini uses `HYPIT_GEMINI_PROVIDER=auto|hypihub|vertex` to select its backend. The backends are-provided by `@hypit/provider-hypihub` and `@hypit/provider-vertex`. HypiHub uses-`HYPIHUB_API_KEY` and optional `HYPIHUB_BASE_URL` (default `https://hypit.ai`); Vertex uses the-existing `GOOGLE_CLOUD_PROJECT`, `GOOGLE_APPLICATION_CREDENTIALS_JSON` and optional-`GOOGLE_CLOUD_LOCATION`. The model defaults to `gemini-3.1-pro`. A missing or insufficient-user key should be resolved by obtaining a HypiHub key at https://hypit.ai, not by changing Source.++observation cache and nothing about it is sent to Gemini. An unavailable WhisperX Provider reports++`status: "unavailable"` with the reason and prepares everything else.+++Prepared stages are reused until the caller explicitly asks for `--redo ...` or `--reobserve`. This+is ordinary working data for one reference, not Build history.++## Inspection and review++`inspect_svml_vocabulary` reports the public Surfaces of packages the caller names.+`inspect_visual_schema` reports the actual Composition schema accepted from drawing Producers.+`validate_local_author_packages`, `validate_script_cues`, `preview_check`, `layout_check`,+`reconstruction_check` and `authoring_check` are independent reports. None approves or unlocks+another command.++`layout_check` always measures the current realized DOM. Its report contains advisory candidates such+as overflow, overlap and vertical offset. `layout_accept` attaches a human reason to a structural+finding id; it does not fingerprint the project or cache a previous measurement.++`render_element` draws one element directly from the Run using local preview media. A comparison reads+that render against the requested reference stretch. A review reads it against an explicit intent.+Their JSONL logs contain the actual question, paths and prose answer so later inspection does not+depend on a separate state database.++## Environment++Reference observation uses `HYPIT_GEMINI_PROVIDER=auto|hypihub|vertex`. HypiHub first reads the+`hypihub.oauth` session written by `hypit auth login` from the OS credential store, then falls back to+`HYPIHUB_API_KEY`; `HYPIHUB_BASE_URL` remains optional. Vertex reads `GOOGLE_CLOUD_PROJECT`,+`GOOGLE_APPLICATION_CREDENTIALS_JSON` and optional `GOOGLE_CLOUD_LOCATION`. Gemini defaults to- `gemini-3.7-flash-openai`.++`gemini-3.1-pro`.++Transcript preparation uses the selected local WhisperX service. Start services through the normal+Runtime configuration with `hypit runtime up`. `HYPIT_REFERENCE_CONCURRENCY` and+`HYPIT_REFERENCE_LAUNCH_GAP_MS` only control observation request pacing.diff --cc packages/reference-video-tools/package.jsonindex 82c5b7ff,aece8a60..5d375070--- a/packages/reference-video-tools/package.json+++ b/packages/reference-video-tools/package.json@@@ -31,7 -31,6 +31,7 @@@"@hypit/provider-media-local": "workspace:*","@hypit/provider-hypihub": "workspace:*","@hypit/provider-vertex": "workspace:*",++ "@hypit/provider-whisperx-local": "workspace:*","@hypit/script": "workspace:*","@hypit/speech": "workspace:*","@hypit/speech-evidence": "workspace:*",