Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

feat(hypit): add resumable variant expansion (#114)

rponeawa130faa5902bf
diff --git a/.agents/skills/hypit/SKILL.md b/.agents/skills/hypit/SKILL.md
index 7130b112..053edd00 100644
--- a/.agents/skills/hypit/SKILL.md
+++ b/.agents/skills/hypit/SKILL.md
@@ -1,6 +1,6 @@
---
name: hypit
-description: Produce a complete video program from a description — a ranking, a talking head, an explainer, a short — or reconstruct one from a reference video; author, check, preview, build, inspect and retrieve Hypit/SVML projects; configure runtimes and credentials; develop missing project-local author packages; and apply native video production playbooks. Use for video production with Hypit and SVML/SVS/SVRun authoring; this is not for developing the Hypit repository itself.
+description: Produce a complete video program from a description or reference video, revise a completed project, or batch-expand a validated project into independent variants; author, check, preview, build, inspect and retrieve Hypit/SVML projects; configure runtimes and credentials; develop missing project-local author packages; and apply native video production playbooks. Use for video production with Hypit and SVML/SVS/SVRun authoring; this is not for developing the Hypit repository itself.
---
# Hypit
@@ -101,6 +101,13 @@ invalid. Never commit `.env` or copy its secret values into route state, Source,
Run has no Studio session so the author can see the updated result. This routing rule also applies
after a paid Build has produced the full video: revise the accepted-material Run, never the rendered
artifact and never by resuming either creation route.
+- A request for many independent versions after reconstruction, original authoring or revision, or
+ from a validated existing project → read `references/variant-expansion/route.md`. The main agent
+ inspects examples, freezes format/Slate/component decisions, enumerates vocabulary, discloses the
+ fast/medium/new-package workload, and resolves package gaps before copying or dispatching variant
+ agents. Initial presenter/product/brand adaptation attached to a reconstruction remains inside the
+ reconstruction route; it is not Revision. Variant expansion defaults to mechanically checked
+ Source and performs no visual review or paid Build unless the author explicitly requests outputs.
- A syntax question about one element, or a source that already exists → read
`references/authoring.md`, then `references/quickstart.md` and the linked authoritative
docs/package READMEs. A whole video is `references/original-authoring/route.md`, not this file.
diff --git a/.agents/skills/hypit/references/original-authoring/route.md b/.agents/skills/hypit/references/original-authoring/route.md
index 1dbff70d..566f4102 100644
--- a/.agents/skills/hypit/references/original-authoring/route.md
+++ b/.agents/skills/hypit/references/original-authoring/route.md
@@ -22,6 +22,10 @@ observer.
This remains true after a paid Build has produced the full video: start `revision_state` against the
accepted-material Run, never edit the rendered file or resume the creation steps.
+If the same request also asks for many independent derivatives, finish this route through the final
+deterministic gate, then enter `../variant-expansion/route.md`. The base project's format, examples,
+component choices and package gaps are decided globally there before any variant agent starts.
+
## Checkpoint and recovery
Start the project snapshot after creating the project directory:
@@ -212,6 +216,10 @@ hypit-reference-video-tools authoring_check projects/<name>/build.svrun
any stretch of the Script has nothing drawing a full frame over it, and while any timed picture is
configured to empty its window.
+If the author requested batch variants, hand this checked Source to
+`../variant-expansion/route.md` now. Do not Build the base or any variant unless finished media was
+explicitly requested; the variant route owns its own aggregate and paid-build gates.
+
### 13. The Build
**Read now:** `../studio-confirmation.md`. Before the paid gate, call the preview-mock realizer and
diff --git a/.agents/skills/hypit/references/playbooks/craft/image-prompt-style.md b/.agents/skills/hypit/references/playbooks/craft/image-prompt-style.md
index 1c36242f..4797f60a 100644
--- a/.agents/skills/hypit/references/playbooks/craft/image-prompt-style.md
+++ b/.agents/skills/hypit/references/playbooks/craft/image-prompt-style.md
@@ -11,7 +11,9 @@
## Write every generation prompt in English
-Use four ordered parts:
+Write the final prompt as continuous natural-language prose, with no section headings, category
+labels, slash-separated field names, or key-value formatting. Cover these four concerns in order,
+but never print their names into the prompt:
1. **Reality contract:** open every photographic prompt with this sentence, verbatim, with only the
bracketed shot replaced. It is concatenated in front of the rest, not paraphrased — a described
@@ -27,7 +29,9 @@ Use four ordered parts:
asset such as a paper texture, a board or a panel is not a photograph, and this sentence would
damage it; write those prompts plainly.
2. **Visible story:** state the exact person/product, wardrobe, location, action, emotion, objects,
- object count, and initial physical state that must be visible.
+ object count, and initial physical state that must be visible. When a person is the main subject,
+ make their face and identity the most specific part of the shot description rather than treating
+ them as a generic figure inside a detailed set.
3. **Camera geometry:** state shot size, camera owner/placement, height, direction, lens relationship,
reflection logic, and which requested objects must fit inside the frame.
4. **Reference contract:** assign each reference a role and state which identity, product geometry,
@@ -59,16 +63,24 @@ The strongest prompts turn adjectives into observable evidence and keep the same
landmarks that must be included. Relational instructions such as “subject slightly right,
leaning left, looking toward an unseen person on the left” are more reliable than three isolated
adjectives.
-3. **Identity and appearance anchors.** Establish the recurring person or product with concrete,
- stable traits first (face shape, eyes, hair, skin, age presentation, distinctive marks), then
- add body silhouette, wardrobe, accessories, and styling. For a male-presenting subject whose
- physique matters, state a visible shoulder line — for example, “broad shoulders and a strong
- upper-body silhouette” — and choose a half-body or wider frame that can actually show it. When
- mixed Asian heritage is part of the intended identity, name the combination explicitly, such as
- “Japanese-American mixed-race man” or “Korean-American mixed-race man”; do not leave a
- reference-critical identity to the generic word “Asian.” Keep heritage as one stable identity
- anchor, never as a bundle of stereotyped physical claims. Prefer a few renderable facts over a
- stack of generic superlatives such as “extremely handsome.”
+3. **Identity and appearance anchors.** For any person-led image, make the identity description the
+ visual center of the prompt. Establish the recurring person with concrete, stable traits first:
+ age presentation; specifically intended ethnicity, nationality, or mixed heritage; face shape;
+ brow and eye shape and spacing; nose structure; lips, jaw, and chin; hairline, hair texture, and
+ hairstyle; skin tone and undertone; natural pores, fine lines, under-eye texture, small
+ asymmetries, and any distinctive marks. Then add body silhouette, wardrobe, accessories, and
+ styling. Use enough of these facts to make the person recognizable across generations instead of
+ writing a generic attractive face. For a male-presenting subject whose physique matters, state a
+ visible shoulder line — for example, “broad shoulders and a strong upper-body silhouette” — and
+ choose a half-body or wider frame that can actually show it. When mixed Asian heritage is part of
+ the intended identity, name the combination explicitly, such as “Japanese-American mixed-race
+ man” or “Korean-American mixed-race man”; do not leave a new, authored identity at the generic
+ word “Asian.” Keep heritage as one stable identity anchor, never as a bundle of stereotyped
+ physical claims. If a supplied reference owns an existing person's identity, do not guess a
+ sensitive background from appearance; describe observable facial traits and tell the model to
+ preserve that exact identity. Prefer specific, renderable human detail over generic superlatives
+ such as “extremely handsome,” “perfect face,” or “flawless skin,” which tend to produce synthetic,
+ interchangeable people.
4. **Action and physical relationships.** Give one readable starting action and posture. Say who the
subject is addressing, what supports their body, which hands are occupied, and how props contact
the scene. This explains a gaze, crossed legs, a microphone entering from frame left, or a person
@@ -88,23 +100,35 @@ The strongest prompts turn adjectives into observable evidence and keep the same
product shape, wardrobe, or UI. State what must remain stable and what the new shot is allowed to
change. Do not describe a reference-locked room a second time in prose as if it were a fresh set.
-For a prompt family, use this compact skeleton after the invariant prefix:
+For a prompt family, preserve this order after the invariant prefix, but write the result as flowing
+natural-language prose. The numbered parts above are an authoring checklist, never headings, labels,
+field names, slash-separated categories, or a key-value outline in the generated prompt. In
+particular, do not emit openings such as `Scene/purpose:`, `Composition/camera:`, or
+`Subject/identity/wardrobe:`. A finished prompt should read like a concise description of the image,
+for example:
```text
-Scene/purpose: …
-Composition/camera: …
-Subject/identity/wardrobe: …
-Action/posture/relationships: …
-Props/branding: …
-Lighting/material: …
-Acceptance constraints: …
-Reference roles: …
+Set the frame in a compact neighborhood podcast studio, with the acoustic wall and the edge of the
+desk still clearly visible. Compose it as a seated medium close-up from the guest's eye level, with
+the subject slightly right of center, leaning toward an unseen host on the left. He is a
+Japanese-American mixed-race man in his early thirties with a broad square face, softly hooded
+dark-brown eyes beneath straight brows, a medium-width nose, a defined jaw, and a slightly uneven
+smile. His warm light-olive skin shows natural pores, faint under-eye creases, and subtle cheek
+asymmetry; his short coarse black hair has a believable, mildly uneven hairline. He has broad
+shoulders and wears a faded charcoal work jacket over a white T-shirt. He rests one forearm on the
+desk and holds a single matte-black microphone near his mouth while listening with a restrained
+smile. Natural window light enters from frame left and keeps skin, cloth, painted wall, and desk
+textures visible without glossy polish or background blur. Keep both shoulder edges, the microphone,
+and its contact with his hand intact inside the frame. Use the first reference to preserve his exact
+identity and wardrobe, while allowing the new posture and camera distance described here.
```
-This is a writing order, not text to paste literally. Keep critical facts singular and unambiguous;
-use controlled repetition only for a fact that must survive across every view (identity, camera side,
-or lighting direction). Separate mutually exclusive poses, locations, or before/after states into
-different prompts rather than joining them with “or.”
+This example demonstrates the prose shape, not wording to paste literally. One or several short
+paragraphs are acceptable, but every sentence must continue the image description rather than name
+a prompt section. Keep critical facts singular and unambiguous; use controlled repetition only for a
+fact that must survive across every view (identity, camera side, or lighting direction). Separate
+mutually exclusive poses, locations, or before/after states into different prompts rather than
+joining them with “or.”
## Keep people and cameras physically possible
diff --git a/.agents/skills/hypit/references/reconstruction/route.md b/.agents/skills/hypit/references/reconstruction/route.md
index f1de45a2..1911c40c 100644
--- a/.agents/skills/hypit/references/reconstruction/route.md
+++ b/.agents/skills/hypit/references/reconstruction/route.md
@@ -60,6 +60,10 @@ observations describe the reference and are cached, and `compare_reconstruction`
element against it. Author the change in early and the evidence describes one video while the sources
describe another.
+If that same initial request also asks for many variants, step 23 still applies the requested
+presenter/product/brand adaptation inside reconstruction. After step 24 passes, hand the adapted base
+to `../variant-expansion/route.md`. Do not route the initial adaptation through Revision.
+
## Checkpoint and recovery
Start the project snapshot before the first route decision:
@@ -398,6 +402,10 @@ If the author requests a change after seeing the preview, leave this route and f
After any requested change (or when there was none), run `reconstruction_check` again and require
`passed: true`. This is the last source/graph gate before the paid handoff.
+If the initial request includes batch variants, enter `../variant-expansion/route.md` from this
+checked, adapted base now. The variant route defaults to source delivery; do not Build the base first
+unless the author explicitly requested it.
+
### 25. Studio confirmation and paid handoff
**Read now:** `../studio-confirmation.md`. Realize the preview-mock Run first, then start Studio with
diff --git a/.agents/skills/hypit/references/recovery.md b/.agents/skills/hypit/references/recovery.md
index e4443101..be070ffe 100644
--- a/.agents/skills/hypit/references/recovery.md
+++ b/.agents/skills/hypit/references/recovery.md
@@ -19,18 +19,38 @@ If the snapshot is missing, start one before continuing:
```bash
hypit-reference-video-tools route_state --action start \
- --project-root <project> --route reconstruction|description --run <run>
+ --project-root <project> --route reconstruction|description|variant|variant-package --run <run>
```
Use `--route description` for original-authoring and `--route reconstruction` for reference-video
work. A project has one active route; a route mismatch is an error rather than an invitation to merge
two histories.
+`variant_init` normally starts `variant`; the main agent starts `variant-package` explicitly for a
+staging project after a vocabulary gap has been proven.
+
+For batch variants, do not look for `.hypit/route-state.json` on the base and assume it names the
+batch. Read `variant-expansion/route.md`, then discover the sibling batch from the base locator:
+
+```bash
+hypit-reference-video-tools variant_state --action discover --project-root <base>
+hypit-reference-video-tools variant_state --action reconcile --project-root <base> --batch-id <id>
+```
+
+Read the returned canonical `<batch>/.hypit/variant-expansion-state.json`, reconcile every unfinished
+`variant-package` staging route and `variant` project route, inspect conflicts, and execute only the
+first unmet `next_action`. `variant_init` reuses matching initialized copies and refuses conflicting
+destinations. A component/package digest change, missing inspection evidence or file outside
+`allowed_changes` is a conflict, not permission to overwrite it.
For a completed project with a new natural-language change, use `.hypit/revision-state.json` and the
`revision_state` commands described in `revision.md`. Reconcile both the parent route snapshot and
the revision snapshot before editing. Revision does not call VLM/observer, render, or perform visual
review; it stops after the requested deterministic gates and leaves Studio untouched.
+After a completed revision, a request for many derivatives starts a new variant-expansion batch from
+the revised project. Reconcile the parent route and revision state before starting the batch; do not
+fold the batch into Revision.
+
## What the state means
The state is a small snapshot, not a transcript. It records the current route step, completed steps,
diff --git a/.agents/skills/hypit/references/revision.md b/.agents/skills/hypit/references/revision.md
index f515c679..ec51a0f5 100644
--- a/.agents/skills/hypit/references/revision.md
+++ b/.agents/skills/hypit/references/revision.md
@@ -58,3 +58,8 @@ and give the author the exact URL with the updated result.
Revision never edits or regenerates mock media. If a later, explicit Studio session previews the
updated Run, that session must continue using the existing SVRun-native preview-mock path; never
revive `make_placeholder`, hand-written SemanticTracks, or absolute-path mock Candidates.
+
+If the author asks for many independent derivatives after this revision is complete, reconcile and
+finish `revision_state`, then start `variant-expansion/route.md` from the revised project. The batch
+is not another revision step: its main agent rechecks examples, format/component plans, package gaps
+and per-variant scopes before copying or dispatching work.
diff --git a/.agents/skills/hypit/references/variant-expansion/route.md b/.agents/skills/hypit/references/variant-expansion/route.md
new file mode 100644
index 00000000..bde3c6cb
--- /dev/null
+++ b/.agents/skills/hypit/references/variant-expansion/route.md
@@ -0,0 +1,281 @@
+# Variant expansion route
+
+Use this route after a reconstruction, original-authoring or revision project has passed its final
+deterministic gate, or when the author supplies an existing project that passes the same gate. It
+derives exactly N independent source projects from one frozen base project. It does not reinterpret
+initial reconstruction customization as Revision: presenter, product and brand changes included in
+the original reconstruction request remain reconstruction step 23 work.
+
+The default deliverable is checked Source. Do not render, visually review or submit a Build unless
+the author explicitly asks for finished videos. A Build spends money and still requires a fresh cost
+approval; context recovery is never approval to repeat it.
+
+## Recovery comes first
+
+At the start of a new turn or after context compaction:
+
+1. Re-read `../../SKILL.md` and this route.
+2. Run `variant_state --action discover --project-root <base>`.
+3. Read the canonical state it reports.
+4. Inspect the parent route/revision digest, `git status`, recorded artifacts and `last_command`.
+5. Run `variant_state --action reconcile --project-root <base> [--batch-id <id>]`.
+6. Reconcile every unfinished `variant-package` staging route and every unfinished `variant` route.
+7. Stop on recorded conflicts; otherwise execute only the first unmet step named by `next_action`.
+
+The base locator is `<base>/.hypit/variant-expansions/<batch-id>.json`; the canonical batch state is
+`<batch>/.hypit/variant-expansion-state.json`. Never recover a sibling batch path from chat memory.
+
+## Batch state
+
+Start once, after the base project is selected:
+
+```bash
+hypit-reference-video-tools variant_state --action start \
+ --project-root <base> --output-root <batch-root> --run <base>/build.svrun \
+ --count <N> --delivery-mode source --request '<the complete request>'
+```
+
+Use `--delivery-mode build` only when the author explicitly requested finished variants. The durable
+batch stages are:
+
+```text
+baseline-validated → examples-inspected → format-plan-frozen → slate-drafted
+→ vocabulary-enumerated → component-plan-frozen → package-gaps-classified
+→ workload-disclosed → package-gaps-resolved → slate-frozen → projects-copied
+→ variants-dispatched → variants-complete → aggregate-checked
+→ [build-planned → build-approved → build-complete] → expansion-complete
+```
+
+Manual creative decisions are checkpointed explicitly. `reconcile` may confirm machine predicates,
+roll missing evidence back, and report digest or scope conflicts; it never invents a Format DNA,
+Slate, component choice, allowed change or cost approval.
+
+## 1. Validate and freeze the base
+
+The base must be a completed reconstruction, original-authoring or revision project, or an existing
+project that passes its corresponding deterministic checks. Record the parent route/revision digest.
+Do not Build merely to create a base for expansion.
+
+If the request began as “reconstruct this video with my product and make 100 variants,” finish the
+reference reconstruction and apply that product/person/brand adaptation inside reconstruction first.
+Variant expansion starts from that adapted, checked base. Revision is used only for a later natural-
+language change to an already completed project.
+
+## 2. Inspect examples and freeze format decisions
+
+**Read now:** `../brief-intake.md`, then inspect complete projects under the checkout's `examples/`
+directory when it exists. Recover intent, Hook, narrative development and the relationship between
+visuals, audio and text. README clone sets are useful evidence of the intended expansion pattern, but
+their Source, people, products, claims and assets are not templates.
+
+**Read now:** `../playbooks/index.md`. Select each variant's format from its intent. Read the matching
+`formats/*.md` file completely and the craft files its footer names. Keep the base format by default.
+A switch among ranking, street interview, podcast or another format is a high-change variant and must
+be explicit in the plan.
+
+Persist `format-plan.json`. Each entry records the variant id, example basis and digest, playbook path,
+and frozen Format DNA: Hook, narrative progression, speaker/role grammar, layout system, timing model,
+caption/overlay behavior, audio relationship and what must remain invariant. Checkpoint
+`examples-inspected` and `format-plan-frozen`; file existence alone does not make either decision.
+
+## 3. Draft the Slate
+
+Write `slate.json` with exactly N distinct, decision-complete entries. Each entry must provide at
+least:
+
+```json
+{
+ "slug": "short-unique-slug",
+ "brief": { "direction": "the complete variant decision" },
+ "component_class": "svml-only",
+ "allowed_changes": ["main.svml"],
+ "vocabulary": { "mode": "inherited", "packages": [] }
+}
+```
+
+`component_class` is one of `svml-only`, `existing-component`, `composed-components` or
+`new-package`. Use `vocabulary.mode: "inspect"` and list the assigned packages whenever a variant
+switches to a package the base did not use. A validated staging package can be injected once copied:
+
+```json
+"inject_packages": [
+ { "package_id": "my-component", "destination": "packages/my-component" }
+]
+```
+
+The package id must name a ready package frozen in batch state; the tool resolves its recorded
+`package_root` and refuses any digest mismatch. The destination must stay under the copied project's
+`packages/` directory. `allowed_changes` is authoritative. A `svml-only` variant must allow exactly
+`main.svml`. Add `.svs`, `build.svrun`, runtime or package paths only when the frozen decision actually
+changes Recipe/Style, Graph/Target/Candidate, capability or component implementation.
+
+Checkpoint `slate-drafted`, but do not freeze or dispatch it yet.
+
+## 4. Enumerate vocabulary and decide package work globally
+
+**Read now:** `../vocabulary.md`.
+
+Before any package or variant agent starts, the main agent runs:
+
+```bash
+hypit-reference-video-tools list_svml_packages
+hypit-reference-video-tools inspect_svml_vocabulary --package <candidate> [--package …]
+```
+
+Read every selected package's returned `readme_path`; never infer syntax from package source. Persist
+the complete listing/inspection as the batch vocabulary evidence and write `component-plan.json`.
+Classify every variant before dispatch:
+
+- base components, SVML-only changes;
+- an installed component the base does not currently use;
+- a deliberate composition of installed components;
+- a proven vocabulary gap requiring one project-local package.
+
+The component plan names exact Module, Tag, relevant Recipe properties, package README and inspection
+evidence. It also names which variants share each new package. Checkpoint `vocabulary-enumerated`,
+`component-plan-frozen` and `package-gaps-classified`.
+
+## 5. Disclose workload before agents start
+
+Tell the author, without pausing by default:
+
+- how many variants are fast `main.svml`-only work;
+- how many change SVS/Run or switch/compose installed components;
+- how many new packages are required, their purpose and affected variants;
+- that new package development materially increases completion time;
+- whether any requested delivery would invoke paid generation.
+
+Persist the same counts and message in `workload_disclosure`, then checkpoint `workload-disclosed`.
+Wait only for paid work, a material creative scope expansion, or a missing core decision.
+
+## 6. Resolve every package gap before ordinary variants
+
+Create one staging project per distinct gap and start:
+
+```bash
+hypit-reference-video-tools route_state --action start \
+ --project-root <staging> --route variant-package --run <staging>/build.svrun
+```
+
+The package agent must read `../vocabulary.md`, `../local-author-package.md`, the assigned format and
+craft documents, `../authoring.md` and `../preview.md`. Its route is:
+
+```text
+gap-confirmed → guidance-loaded → types-frozen → implemented
+→ vocabulary-inspected → package-validated → graph-checked → package-ready
+```
+
+It must repeat `list_svml_packages` and `inspect_svml_vocabulary` to confirm the gap, freeze Types and
+Manifest before implementation, then run Run-scoped inspection, `validate_local_author_packages` and
+`preview_check`. If package assets require paid generation, obtain cost approval first.
+
+When the route has passed through `graph-checked`, checkpoint the batch package entry with
+`status: "ready"` and name the package directory as `package_root` when it is below the staging
+project root. `variant_state` freezes that package tree's digest, writes package-digest evidence, and
+completes the staging route's `package-ready` step. The same digest is injected into every assigned
+variant; never develop the same component twice.
+
+No dependent variant may be copied or dispatched while its package route is incomplete. Once every
+gap is ready, checkpoint `package-gaps-resolved`, make all Slate decisions final, and checkpoint
+`slate-frozen`.
+
+## 7. Copy first
+
+Run:
+
+```bash
+hypit-reference-video-tools variant_init \
+ --project-root <base> --output-root <batch-root> --slate <slate.json>
+```
+
+The command creates `NNN-<slug>/` projects under the batch root. It uses copy-on-write when the
+filesystem supports it and falls back to ordinary copying. It preserves authored SVML, SVS, runtime,
+configuration, local packages, ordinary assets and the top-level `build.svrun`; it excludes Git and
+dependency state, old `.hypit`, secrets, logs/caches, non-canonical preview/mock/accepted-material
+Runs and generated result directories. Media are never excluded merely because of their extension.
+
+Historical `<build-record>` candidates and only the `<satisfy>` declarations selecting those
+candidates are removed from each copied canonical Run. Authored files, values, files, fragments,
+targets and unrelated satisfactions remain. The copied state records a digest for every retained
+file, writes the variant brief and `allowed_changes`, starts its `variant` route, and injects only the
+validated packages assigned to that variant. Re-running `variant_init` reuses matching initialized
+projects and refuses conflicting destinations.
+
+## 8. Dispatch variant agents in bounded waves
+
+The main agent supplies each variant agent only its copied project and fixed plan entries. The main
+agent, not the child, has already chosen examples, format, Format DNA, component strategy and scope.
+Checkpoint `variants-dispatched` only after the agents have actually been assigned; route-state files
+created by `variant_init` are not proof of dispatch.
+
+Every variant agent reads:
+
+1. its copied project, `.hypit/variant-brief.json` and `.hypit/allowed-changes.json`;
+2. its `format-plan.json` and `component-plan.json` entry;
+3. the assigned format playbook and every craft document linked by its footer;
+4. `../authoring.md`, `../script-time.md` and `../preview.md`;
+5. the assigned package README and inspection evidence.
+
+Checkpoint `guidance-loaded` after those reads. The agent does not rescan examples or replace the
+format/component decision.
+
+Vocabulary behavior is fixed:
+
+- unchanged Module/Tag/component combination and Recipe contract may use the inherited base evidence;
+- a package or component change requires `list_svml_packages`, Run-scoped
+ `inspect_svml_vocabulary`, and the returned README inside the copied project;
+- a claimed new gap must be returned to the main agent. The variant agent may not start package
+ development or silently invent a Tag, attribute or Recipe value.
+
+If the plan is inconsistent with installed declarations, record the conflict and stop that variant.
+Do not widen scope, change format or choose another component locally.
+
+## 9. Make the minimum change and check it
+
+The variant route is:
+
+```text
+baseline-copied → brief-frozen → change-scope-frozen → guidance-loaded
+→ vocabulary-verified → package-ready → script-checked → source-updated
+→ graph-checked → final-checked → variant-complete
+```
+
+Change only the files named by `allowed_changes`:
+
+- content, identity, product, Script and prompts in `main.svml`;
+- Recipe/Style changes in `.svs`;
+- Graph/Target/Candidate changes in `build.svrun`;
+- capability changes in runtime;
+- confirmed component changes in imports, project packages or dependencies.
+
+Do not reformat, rename or rewrite unaffected files. Then run the package/Cue gates, `hypit check`,
+`preview_check`, and:
+
+```bash
+hypit-reference-video-tools variant_check --run <variant>/build.svrun
+```
+
+`variant_check` is mechanical only. It verifies vocabulary evidence, package use, Cue lengths, source
+legality, graph tracing, frame-coverage/playback facts, generated-result leakage and the digest diff
+against `allowed_changes`. It reports Frame/Canvas boundary and layout geometry facts without making
+a visual judgement. It never renders or invokes a VLM.
+
+If the diff escapes `allowed_changes`, the route becomes blocked and the batch records
+`scope-expansion-required`. The main agent decides whether the requested brief genuinely requires the
+wider scope, atomically updates the batch variant record, baseline manifest, variant brief and
+`.hypit/allowed-changes.json`, records the decision/conflict resolution in batch state, and
+redispatches that variant. The three frozen scope copies must agree; a child never makes this change
+silently.
+
+## 10. Aggregate and optionally Build
+
+As variants finish, `variant_check` updates their batch records and writes
+`<batch>/.hypit/aggregate-check.json`. `variants-complete` requires every child route to be complete;
+`aggregate-checked` requires every mechanical report to pass. Interrupted large batches resume only
+unfinished routes.
+
+For source delivery, checkpoint `expansion-complete` after aggregate success. For finished-video
+delivery, prepare a Build plan and disclose the total cost, then checkpoint `build-planned`. Obtain
+explicit approval and checkpoint `build-approved` before the first paid submission. Persist each
+Build result and checkpoint `build-complete` only when all requested outputs are durable. Reconcile
+the state before any retry so a context boundary never repeats a paid Build.
diff --git a/packages/reference-video-tools/README.md b/packages/reference-video-tools/README.md
index 66025a66..0b01884d 100644
--- a/packages/reference-video-tools/README.md
+++ b/packages/reference-video-tools/README.md
@@ -2,8 +2,8 @@
CLI tools for reconstructing a reference video with Hypit.
-The package exposes reference-observation, vocabulary/gate, rendering, review, route-state and
-revision-state CLI subcommands, including `list_svml_packages`, `prepare_reference`,
+The package exposes reference-observation, vocabulary/gate, rendering, review, route-state,
+revision-state and variant-expansion CLI subcommands, including `list_svml_packages`, `prepare_reference`,
`observe_reference`, `record_observation`, `inspect_svml_vocabulary`, and `compare_reconstruction`.
Each command prints one JSON result to stdout. The final source files are authored by the calling
agent and checked with the installed `hypit check` command.
@@ -12,6 +12,12 @@ For a completed project change, `revision_state --action start|read|checkpoint|r
small atomic `.hypit/revision-state.json` snapshot. Revision edits Source/Recipe/Run and reruns
deterministic gates; it does not invoke a VLM/observer or perform visual review.
+For a validated source project that needs independent derivatives, `variant_state` stores the atomic
+batch snapshot and base-project locator, `variant_init` copies the authored project without generated
+results or secrets, and `variant_check` enforces the declared file scope plus the deterministic
+package, Cue, graph, coverage and playback gates. Variant checking performs no render or visual
+review. A paid Build remains a separate, explicitly approved action.
+
`--observer` on `prepare_reference` chooses who reads the reference, once per reference:
- `gemini` uploads the shot clips and the whole video to Vertex at a fixed temperature of `1.0`. It
@@ -41,6 +47,10 @@ hypit-reference-video-tools observe_reference --reference-id <reference-id> --sh
hypit-reference-video-tools observe_reference --reference-id <reference-id> --shot-id shot-007 --reobserve
hypit-reference-video-tools inspect_svml_vocabulary --package @hypit/media-track --tag Track
hypit-reference-video-tools compare_reconstruction --reference-id <reference-id> --shot-id shot-007 --image ./rendered.png
+hypit-reference-video-tools variant_state --action discover --project-root ./projects/base
+hypit-reference-video-tools variant_state --action start --project-root ./projects/base --output-root ./projects/base-variants/<batch-id> --count 100
+hypit-reference-video-tools variant_init --project-root ./projects/base --output-root ./projects/base-variants/<batch-id> --slate ./slate.json
+hypit-reference-video-tools variant_check --run ./projects/base-variants/<batch-id>/001-example/build.svrun
```
For automation, every command also accepts `--input '{"...":"..."}'` with the complete JSON input
diff --git a/packages/reference-video-tools/src/checks.ts b/packages/reference-video-tools/src/checks.ts
index 3baa3362..6263527b 100644
--- a/packages/reference-video-tools/src/checks.ts
+++ b/packages/reference-video-tools/src/checks.ts
@@ -274,6 +274,8 @@ export type AuthoringCheckInput = ReconstructionCheckInput & {
readonly mode?: AuthoringCheckMode;
};
+export type MechanicalAuthoringCheckInput = { readonly run: string };
+
/**
* Where a description-authored project's reviews are logged.
*
@@ -1651,3 +1653,36 @@ export async function authoringCheck(
}),
};
}
+
+/**
+ * Source-only delivery gate for variant expansion.
+ *
+ * It reuses the authoring check's graph-independent analysis but deliberately does not require a
+ * render, comparison, review log or VLM judgement. The returned geometry remains evidence for the
+ * caller; only deterministic failures (unresolved vocabulary, uncovered Script words and timed
+ * pictures that empty their windows) decide `passed` here.
+ */
+export async function mechanicalAuthoringCheck(
+ input: MechanicalAuthoringCheckInput,
+ roots: { readonly packageRoot: string },
+): Promise<Record<string, unknown>> {
+ const result = await authoringCheck({ run: input.run, mode: "description" }, roots);
+ const playback = Array.isArray(result.playback) ? result.playback : [];
+ const uncovered = Array.isArray(result.uncovered) ? result.uncovered : [];
+ const unresolved = result.unresolved_packages;
+ const summary = Array.isArray(result.summary) ? result.summary.map(String) : [];
+ const unreadable = summary.some((line) => line.startsWith("no aliased package import could be read")
+ || line.startsWith("no package the Source imports publishes a Surface that draws"));
+ const passed = playback.length === 0 && uncovered.length === 0 && unresolved === undefined && !unreadable;
+ return {
+ run: result.run,
+ passed,
+ summary,
+ playback,
+ uncovered,
+ out_of_bounds: result.out_of_bounds ?? [],
+ layout_geometry: result.layout_geometry,
+ ...(unresolved === undefined ? {} : { unresolved_packages: unresolved }),
+ note: "Mechanical variant gate only; no render, VLM comparison or visual review was performed.",
+ };
+}
diff --git a/packages/reference-video-tools/src/cli.ts b/packages/reference-video-tools/src/cli.ts
index 2e4784a3..b85ced57 100644
--- a/packages/reference-video-tools/src/cli.ts
+++ b/packages/reference-video-tools/src/cli.ts
@@ -2,7 +2,7 @@
import { readFile } from "node:fs/promises";
import { createReferenceVideoTools } from "./tools.js";
-import type { CompareReconstructionInput, ObserveReferenceInput, RecordObservationInput, RenderElementInput, ReviewElementInput, RouteStateCommandInput, RevisionStateCommandInput } from "./tools.js";
+import type { CompareReconstructionInput, ObserveReferenceInput, RecordObservationInput, RenderElementInput, ReviewElementInput, RouteStateCommandInput, RevisionStateCommandInput, VariantStateCommandInput } from "./tools.js";
/**
* A word range written on the command line as `from:to`, half-open.
@@ -28,7 +28,10 @@ function usage(): string {
" hypit-reference-video-tools list_svml_packages",
" hypit-reference-video-tools route_state --action start|read|checkpoint|reconcile --project-root <dir> [options]",
" hypit-reference-video-tools revision_state --action start|read|checkpoint|reconcile --project-root <dir> [options]",
- " start: --route reconstruction|description [--run <run>] [--reference-id <id>]",
+ " hypit-reference-video-tools variant_state --action discover|start|read|checkpoint|reconcile --project-root <dir> [options]",
+ " hypit-reference-video-tools variant_init --project-root <base> --output-root <batch> --slate <slate.json>",
+ " hypit-reference-video-tools variant_check --run <variant>/build.svrun [--runtime <hypit.runtime.json>]",
+ " route_state start: --route reconstruction|description|variant|variant-package [--run <run>] [--reference-id <id>]",
" checkpoint: --route <route> (--state <stage> | --step <n>) [--status in_progress|complete|blocked] [--next-action <text>] [--artifacts <json>]",
" read/reconcile: --project-root <dir>",
" hypit-reference-video-tools prepare_reference --video-path <path or link> [--observer gemini|agent] [--redo media|transcript|people|voices|systems|places|all]",
@@ -200,9 +203,9 @@ function usage(): string {
function routeStateUsage(): string {
return [
"Usage:",
- " hypit-reference-video-tools route_state --action start --project-root <dir> --route reconstruction|description [--run <run>] [--reference-id <id>]",
+ " hypit-reference-video-tools route_state --action start --project-root <dir> --route reconstruction|description|variant|variant-package [--run <run>] [--reference-id <id>]",
" hypit-reference-video-tools route_state --action read|reconcile --project-root <dir>",
- " hypit-reference-video-tools route_state --action checkpoint --project-root <dir> --route reconstruction|description (--state <stage> | --step <n>) [options]",
+ " hypit-reference-video-tools route_state --action checkpoint --project-root <dir> --route reconstruction|description|variant|variant-package (--state <stage> | --step <n>) [options]",
"",
"Use exactly one of --state and --step. --state is the durable stage name; --step is retained for compatibility.",
"",
@@ -312,7 +315,7 @@ async function main(): Promise<void> {
result = await tools.route_state(supplied as RouteStateCommandInput);
} else if (action === "start") {
const route = one(flags, "route");
- if (route !== "reconstruction" && route !== "description") throw new Error("--route must be reconstruction or description");
+ if (route !== "reconstruction" && route !== "description" && route !== "variant" && route !== "variant-package") throw new Error("--route must be reconstruction, description, variant or variant-package");
result = await tools.route_state({ action, project_root: projectRoot, route,
...(one(flags, "run") === undefined ? {} : { run: one(flags, "run") }),
...(one(flags, "reference-id") === undefined ? {} : { reference_id: one(flags, "reference-id") }) } as RouteStateCommandInput);
@@ -320,7 +323,7 @@ async function main(): Promise<void> {
result = await tools.route_state({ action, project_root: projectRoot });
} else {
const route = one(flags, "route");
- if (route !== "reconstruction" && route !== "description") throw new Error("--route must be reconstruction or description");
+ if (route !== "reconstruction" && route !== "description" && route !== "variant" && route !== "variant-package") throw new Error("--route must be reconstruction, description, variant or variant-package");
const stepFlag = one(flags, "step");
const stateFlag = one(flags, "state");
if ((stepFlag === undefined) === (stateFlag === undefined)) throw new Error("checkpoint requires exactly one of --state <stage> or --step <n>\n\n" + routeStateUsage());
@@ -392,6 +395,77 @@ async function main(): Promise<void> {
...(many(flags, "affected-source").length === 0 ? {} : { affected_source: many(flags, "affected-source") }),
} as RevisionStateCommandInput);
}
+ } else if (command === "variant_state") {
+ const action = typeof supplied?.action === "string" ? supplied.action : one(flags, "action");
+ if (action !== "discover" && action !== "start" && action !== "read" && action !== "checkpoint" && action !== "reconcile") {
+ throw new Error("--action must be discover, start, read, checkpoint or reconcile");
+ }
+ const projectRoot = typeof supplied?.project_root === "string" ? supplied.project_root : required(flags, "project-root");
+ if (supplied !== undefined) {
+ result = await tools.variant_state(supplied as VariantStateCommandInput);
+ } else if (action === "discover") {
+ result = await tools.variant_state({ action, project_root: projectRoot });
+ } else if (action === "start") {
+ const deliveryMode = one(flags, "delivery-mode");
+ if (deliveryMode !== undefined && deliveryMode !== "source" && deliveryMode !== "build") throw new Error("--delivery-mode must be source or build");
+ const countRaw = one(flags, "count");
+ const count = countRaw === undefined ? undefined : Number(countRaw);
+ if (count !== undefined && (!Number.isSafeInteger(count) || count < 1)) throw new Error("--count must be a positive integer");
+ const parentRoute = one(flags, "parent-route");
+ if (parentRoute !== undefined && parentRoute !== "reconstruction" && parentRoute !== "description") throw new Error("--parent-route must be reconstruction or description");
+ result = await tools.variant_state({
+ action, project_root: projectRoot, output_root: required(flags, "output-root"),
+ ...(one(flags, "run") === undefined ? {} : { run: one(flags, "run") }),
+ ...(one(flags, "request") === undefined ? {} : { request: one(flags, "request") }),
+ ...(count === undefined ? {} : { count }),
+ ...(deliveryMode === undefined ? {} : { delivery_mode: deliveryMode }),
+ ...(parentRoute === undefined ? {} : { parent_route: parentRoute }),
+ ...(one(flags, "parent-state-digest") === undefined ? {} : { parent_state_digest: one(flags, "parent-state-digest") }),
+ ...(one(flags, "revision-state-digest") === undefined ? {} : { revision_state_digest: one(flags, "revision-state-digest") }),
+ } as VariantStateCommandInput);
+ } else if (action === "read" || action === "reconcile") {
+ const outputRoot = one(flags, "output-root");
+ const batchId = one(flags, "batch-id");
+ result = await tools.variant_state({
+ action, project_root: projectRoot,
+ ...(outputRoot === undefined ? {} : { output_root: outputRoot }),
+ ...(batchId === undefined ? {} : { batch_id: batchId }),
+ });
+ } else {
+ const stepRaw = required(flags, "step");
+ const numericStep = Number(stepRaw);
+ const step: number | string = Number.isSafeInteger(numericStep) && numericStep >= 1 ? numericStep : stepRaw;
+ const status = one(flags, "status") ?? "complete";
+ if (status !== "in_progress" && status !== "complete" && status !== "blocked") throw new Error("--status must be in_progress, complete or blocked");
+ const parseObject = <T>(name: string): T | undefined => {
+ const raw = one(flags, name);
+ if (raw === undefined) return undefined;
+ try { return JSON.parse(raw) as T; } catch (error) { throw new Error(`--${name} is not valid JSON: ${error instanceof Error ? error.message : String(error)}`); }
+ };
+ result = await tools.variant_state({
+ action, project_root: projectRoot, step, status,
+ ...(one(flags, "output-root") === undefined ? {} : { output_root: one(flags, "output-root") }),
+ ...(one(flags, "batch-id") === undefined ? {} : { batch_id: one(flags, "batch-id") }),
+ ...(one(flags, "next-action") === undefined ? {} : { next_action: one(flags, "next-action") }),
+ ...(parseObject<Record<string, string>>("artifacts") === undefined ? {} : { artifacts: parseObject<Record<string, string>>("artifacts") }),
+ ...(parseObject<VariantStateCommandInput & object>("workload-disclosure") === undefined ? {} : { workload_disclosure: parseObject("workload-disclosure") }),
+ ...(parseObject<readonly unknown[]>("packages") === undefined ? {} : { packages: parseObject("packages") }),
+ ...(parseObject<readonly unknown[]>("variants") === undefined ? {} : { variants: parseObject("variants") }),
+ ...(one(flags, "decision") === undefined ? {} : { decision: one(flags, "decision") }),
+ ...(one(flags, "conflict") === undefined ? {} : { conflict: one(flags, "conflict") }),
+ ...(one(flags, "resolve-conflict") === undefined ? {} : { resolve_conflict: one(flags, "resolve-conflict") }),
+ ...(one(flags, "command") === undefined ? {} : { command: one(flags, "command") }),
+ ...(one(flags, "error") === undefined ? {} : { error: one(flags, "error") }),
+ } as VariantStateCommandInput);
+ }
+ } else if (command === "variant_init") {
+ result = await tools.variant_init((supplied ?? {
+ project_root: required(flags, "project-root"), output_root: required(flags, "output-root"), slate: required(flags, "slate"),
+ }) as { project_root: string; output_root: string; slate: string });
+ } else if (command === "variant_check") {
+ result = await tools.variant_check((supplied ?? {
+ run: required(flags, "run"), ...(one(flags, "runtime") === undefined ? {} : { runtime: one(flags, "runtime") }),
+ }) as { run: string; runtime?: string });
} else if (command === "prepare_reference") {
const observer = one(flags, "observer");
if (observer !== undefined && observer !== "gemini" && observer !== "agent") throw new Error("--observer must be gemini or agent");
diff --git a/packages/reference-video-tools/src/index.ts b/packages/reference-video-tools/src/index.ts
index 269fcc86..09bd3457 100644
--- a/packages/reference-video-tools/src/index.ts
+++ b/packages/reference-video-tools/src/index.ts
@@ -8,8 +8,11 @@ export type {
ValidateLocalAuthorPackagesInput,
RouteStateCommandInput,
RevisionStateCommandInput,
+ VariantStateCommandInput,
+ VariantInitInput,
+ VariantCheckInput,
} from "./tools.js";
-export type { ScriptCueCheckInput } from "./checks.js";
+export type { MechanicalAuthoringCheckInput, ScriptCueCheckInput } from "./checks.js";
export {
checkpointRouteState,
readRouteState,
@@ -19,7 +22,7 @@ export {
ROUTE_STATE_STEPS,
ROUTE_STATE_VERSION,
} from "./route-state.js";
-export type { DescriptionRouteStep, ReconstructionRouteStep, RouteCheckpointInput, RouteKind, RouteState, RouteStateInput, RouteStep } from "./route-state.js";
+export type { DescriptionRouteStep, ReconstructionRouteStep, RouteCheckpointInput, RouteKind, RouteState, RouteStateInput, RouteStep, VariantPackageRouteStep, VariantRouteStep } from "./route-state.js";
export {
checkpointRevisionState,
readRevisionState,
@@ -30,3 +33,34 @@ export {
REVISION_STEPS,
} from "./revision-state.js";
export type { RevisionCheckpointInput, RevisionState, RevisionStateInput, RevisionStatus, RevisionStep } from "./revision-state.js";
+export {
+ checkpointVariantExpansion,
+ discoverVariantExpansions,
+ readVariantExpansionState,
+ reconcileVariantExpansion,
+ startVariantExpansion,
+ variantExpansionLocatorPath,
+ variantExpansionStatePath,
+ VARIANT_EXPANSION_STATE_VERSION,
+ VARIANT_EXPANSION_STEPS,
+} from "./variant-state.js";
+export type {
+ StartVariantExpansionInput,
+ VariantExpansionCheckpointInput,
+ VariantExpansionLocator,
+ VariantExpansionPackage,
+ VariantExpansionState,
+ VariantExpansionStatus,
+ VariantExpansionStep,
+ VariantExpansionVariant,
+ VariantWorkloadDisclosure,
+} from "./variant-state.js";
+export {
+ findGeneratedLeakage,
+ initializeVariantProjects,
+ inspectSourceVocabularyUsage,
+ inspectVariantDiff,
+ removeGeneratedRunBindings,
+ snapshotProject,
+} from "./variant-project.js";
+export type { ApprovedVariantPackage, InitializedVariant, SlatePackageInjection, SlateVariant, VariantProjectManifest, VariantSlate } from "./variant-project.js";
diff --git a/packages/reference-video-tools/src/route-state.ts b/packages/reference-video-tools/src/route-state.ts
index a9c9d60d..9f2f5fc5 100644
--- a/packages/reference-video-tools/src/route-state.ts
+++ b/packages/reference-video-tools/src/route-state.ts
@@ -1,7 +1,7 @@
import { mkdir, readFile, rename, stat, writeFile } from "node:fs/promises";
import { dirname, join, resolve } from "node:path";
-export type RouteKind = "reconstruction" | "description";
+export type RouteKind = "reconstruction" | "description" | "variant" | "variant-package";
export type RouteStatus = "active" | "complete" | "blocked";
export type ReconstructionRouteStep =
| "environment" | "reference-prepared" | "reference-observed" | "vocabulary-checked"
@@ -13,7 +13,14 @@ export type DescriptionRouteStep =
| "script-checked" | "source-authored" | "graph-checked" | "review-planned"
| "preview-rendered" | "review-complete" | "repairs-complete" | "final-checked"
| "build-complete";
-export type RouteStep = ReconstructionRouteStep | DescriptionRouteStep;
+export type VariantRouteStep =
+ | "baseline-copied" | "brief-frozen" | "change-scope-frozen" | "guidance-loaded"
+ | "vocabulary-verified" | "package-ready" | "script-checked" | "source-updated"
+ | "graph-checked" | "final-checked" | "variant-complete";
+export type VariantPackageRouteStep =
+ | "gap-confirmed" | "guidance-loaded" | "types-frozen" | "implemented"
+ | "vocabulary-inspected" | "package-validated" | "graph-checked" | "package-ready";
+export type RouteStep = ReconstructionRouteStep | DescriptionRouteStep | VariantRouteStep | VariantPackageRouteStep;
export type RouteState = {
readonly version: 2;
@@ -62,6 +69,8 @@ const LEGACY_ROUTE_STATE_STEPS: Readonly<Record<RouteKind, readonly string[]>> =
"environment", "brief-frozen", "source-authored", "vocabulary-checked", "graph-checked",
"review-planned", "preview-rendered", "review-complete", "repairs-complete", "final-checked", "build-complete",
],
+ variant: [],
+ "variant-package": [],
};
export const ROUTE_STATE_STEPS: Readonly<Record<RouteKind, readonly string[]>> = {
@@ -74,6 +83,14 @@ export const ROUTE_STATE_STEPS: Readonly<Record<RouteKind, readonly string[]>> =
"environment", "brief-frozen", "vocabulary-checked", "package-ready", "script-checked", "source-authored",
"graph-checked", "review-planned", "preview-rendered", "review-complete", "repairs-complete", "final-checked", "build-complete",
],
+ variant: [
+ "baseline-copied", "brief-frozen", "change-scope-frozen", "guidance-loaded", "vocabulary-verified",
+ "package-ready", "script-checked", "source-updated", "graph-checked", "final-checked", "variant-complete",
+ ],
+ "variant-package": [
+ "gap-confirmed", "guidance-loaded", "types-frozen", "implemented", "vocabulary-inspected",
+ "package-validated", "graph-checked", "package-ready",
+ ],
};
export function routeStatePath(projectRoot: string): string {
@@ -84,7 +101,7 @@ function assertRouteState(value: unknown, path: string): asserts value is RouteS
if (value === null || typeof value !== "object") throw new Error(`route state at ${path} is not an object`);
const state = value as Partial<RouteState>;
if (state.version !== ROUTE_STATE_VERSION) throw new Error(`route state at ${path} has unsupported version`);
- if (state.route !== "reconstruction" && state.route !== "description") throw new Error(`route state at ${path} has an invalid route`);
+ if (state.route !== "reconstruction" && state.route !== "description" && state.route !== "variant" && state.route !== "variant-package") throw new Error(`route state at ${path} has an invalid route`);
if (typeof state.project_root !== "string" || state.project_root.length === 0) throw new Error(`route state at ${path} has no project_root`);
if (typeof state.status !== "string" || !["active", "complete", "blocked"].includes(state.status)) throw new Error(`route state at ${path} has an invalid status`);
const currentStep = state.current_step;
@@ -218,13 +235,18 @@ function assertPriorStagesComplete(route: RouteKind, step: number, completed: Re
/** Source cannot be accepted until vocabulary, package and Script gates are durable. */
function assertSourcePrerequisites(route: RouteKind, step: number, completed: ReadonlySet<number>): void {
- if (step !== stageNumber(route, "source-authored")) return;
- const required = ["vocabulary-checked", "package-ready", "script-checked"]
+ const sourceStage = route === "variant" ? "source-updated"
+ : route === "variant-package" ? "implemented" : "source-authored";
+ if (step !== stageNumber(route, sourceStage)) return;
+ const required = route === "variant"
+ ? ["vocabulary-verified", "package-ready", "script-checked"]
+ : route === "variant-package" ? [] : ["vocabulary-checked", "package-ready", "script-checked"];
+ const requiredSteps = required
.map((name) => stageNumber(route, name));
- const missing = required.filter((item) => !completed.has(item));
+ const missing = requiredSteps.filter((item) => !completed.has(item));
if (missing.length > 0) {
const names = missing.map((item) => ROUTE_STATE_STEPS[route][item - 1]).join(", ");
- throw new Error(`source-authored cannot be completed before ${names}`);
+ throw new Error(`${sourceStage} cannot be completed before ${names}`);
}
}
@@ -386,16 +408,29 @@ export async function reconcileRouteState(projectRoot: string): Promise<RouteSta
2: "brief", 3: "vocabulary", 4: "package_ready", 5: "script_cues", 6: "author_source", 7: "preview_check",
8: "review_plan", 9: "preview_render", 10: "review_log", 12: "final_check", 13: "build",
},
+ variant: {
+ 1: "baseline_manifest", 2: "variant_brief", 3: "allowed_changes", 4: "guidance",
+ 5: "vocabulary", 6: "package_ready", 7: "script_cues", 8: "author_source",
+ 9: "preview_check", 10: "variant_check", 11: "variant_check",
+ },
+ "variant-package": {
+ 1: "gap_plan", 2: "guidance", 3: "types", 4: "package_source", 5: "vocabulary",
+ 6: "package_ready", 7: "preview_check", 8: "package_digest",
+ },
};
for (const [step, key] of Object.entries(evidenceByStep[existing.route])) {
await reconcileStep(Number(step), key);
}
- const sourceStep = stageNumber(existing.route, "source-authored");
- const gateSteps = ["vocabulary-checked", "package-ready", "script-checked"]
- .map((name) => stageNumber(existing.route, name));
- if (completed.has(sourceStep) && gateSteps.some((step) => !completed.has(step))) {
- conflicts.push(`step ${sourceStep} (source-authored) was marked complete before vocabulary/package/Script gates`);
- completed.delete(sourceStep);
+ if (existing.route !== "variant-package") {
+ const sourceName = existing.route === "variant" ? "source-updated" : "source-authored";
+ const vocabularyName = existing.route === "variant" ? "vocabulary-verified" : "vocabulary-checked";
+ const sourceStep = stageNumber(existing.route, sourceName);
+ const gateSteps = [vocabularyName, "package-ready", "script-checked"]
+ .map((name) => stageNumber(existing.route, name));
+ if (completed.has(sourceStep) && gateSteps.some((step) => !completed.has(step))) {
+ conflicts.push(`step ${sourceStep} (${sourceName}) was marked complete before vocabulary/package/Script gates`);
+ completed.delete(sourceStep);
+ }
}
const completedSteps = [...completed].sort((a, b) => a - b);
const next = nextUncompleted(existing.route, completedSteps);
diff --git a/packages/reference-video-tools/src/tools.ts b/packages/reference-video-tools/src/tools.ts
index 7ab41b5f..54cf426d 100644
--- a/packages/reference-video-tools/src/tools.ts
+++ b/packages/reference-video-tools/src/tools.ts
@@ -26,7 +26,7 @@ import { loadStudioCompanionRegistry } from "@hypit/studio/src/companion-profile
import { openStudioArchive } from "@hypit/studio/src/archive.js";
import { loadStudioDomain } from "@hypit/studio/src/domain.js";
import { loadStudioRun } from "@hypit/studio/src/run.js";
-import { authoringCheck, previewCheck, reviewLogPath, scriptCueCheck } from "./checks.js";
+import { authoringCheck, mechanicalAuthoringCheck, previewCheck, reviewLogPath, scriptCueCheck } from "./checks.js";
import type { AuthoringCheckInput, PreviewCheckInput, ReconstructionCheckInput, ScriptCueCheckInput } from "./checks.js";
import {
checkpointRouteState,
@@ -46,6 +46,31 @@ import {
startRevisionState,
} from "./revision-state.js";
import type { RevisionState, RevisionStep } from "./revision-state.js";
+import {
+ checkpointVariantExpansion,
+ discoverVariantExpansions,
+ readVariantExpansionState,
+ reconcileVariantExpansion,
+ startVariantExpansion,
+ variantExpansionStatePath,
+ VARIANT_EXPANSION_STEPS,
+} from "./variant-state.js";
+import type {
+ VariantExpansionCheckpointInput,
+ VariantExpansionPackage,
+ VariantExpansionStep,
+ VariantExpansionVariant,
+ VariantWorkloadDisclosure,
+} from "./variant-state.js";
+import {
+ findGeneratedLeakage,
+ initializeVariantProjects,
+ inspectSourceVocabularyUsage,
+ inspectVariantDiff,
+ snapshotProject,
+ writeVariantJson,
+} from "./variant-project.js";
+import type { ApprovedVariantPackage, VariantProjectManifest } from "./variant-project.js";
import {
assert,
completelyStill,
@@ -206,6 +231,15 @@ export type RevisionStateCommandInput =
| ({ readonly action: "read" | "reconcile"; readonly project_root: string })
| ({ readonly action: "checkpoint"; readonly project_root: string; readonly step: number | RevisionStep; readonly status?: "in_progress" | "complete" | "blocked"; readonly run?: string; readonly parent_route?: "reconstruction" | "description"; readonly parent_state_digest?: string; readonly request?: string; readonly next_action?: string; readonly artifacts?: Readonly<Record<string, string>>; readonly decision?: string; readonly command?: string; readonly error?: string; readonly impact?: readonly string[]; readonly affected_source?: readonly string[] });
+export type VariantStateCommandInput =
+ | ({ readonly action: "discover"; readonly project_root: string })
+ | ({ readonly action: "start"; readonly project_root: string; readonly output_root: string; readonly run?: string; readonly request?: string; readonly count?: number; readonly delivery_mode?: "source" | "build"; readonly parent_route?: "reconstruction" | "description"; readonly parent_state_digest?: string; readonly revision_state_digest?: string })
+ | ({ readonly action: "read" | "reconcile"; readonly project_root: string; readonly output_root?: string; readonly batch_id?: string })
+ | ({ readonly action: "checkpoint"; readonly project_root: string; readonly output_root?: string; readonly batch_id?: string; readonly step: number | VariantExpansionStep; readonly status?: "in_progress" | "complete" | "blocked"; readonly next_action?: string; readonly artifacts?: Readonly<Record<string, string>>; readonly workload_disclosure?: VariantWorkloadDisclosure; readonly packages?: readonly VariantExpansionPackage[]; readonly variants?: readonly VariantExpansionVariant[]; readonly decision?: string; readonly conflict?: string; readonly resolve_conflict?: string; readonly command?: string; readonly error?: string });
+
+export type VariantInitInput = { readonly project_root: string; readonly output_root: string; readonly slate: string };
+export type VariantCheckInput = { readonly run: string; readonly runtime?: string };
+
export type { RenderElementInput, RenderPreviewsInput } from "./authoring.js";
export type { PreviewCheckInput, ReconstructionCheckInput } from "./checks.js";
@@ -235,6 +269,9 @@ export type ReferenceVideoTools = {
authoring_check(input: Omit<AuthoringCheckInput, "mode" | "reference_id">): Promise<Record<string, unknown>>;
route_state(input: RouteStateCommandInput): Promise<Record<string, unknown>>;
revision_state(input: RevisionStateCommandInput): Promise<Record<string, unknown>>;
+ variant_state(input: VariantStateCommandInput): Promise<Record<string, unknown>>;
+ variant_init(input: VariantInitInput): Promise<Record<string, unknown>>;
+ variant_check(input: VariantCheckInput): Promise<Record<string, unknown>>;
};
type ToolOptions = {
@@ -256,9 +293,19 @@ function revisionStateResult(state: RevisionState | undefined): Record<string, u
return state === undefined ? { state: null } : { state, path: revisionStatePath(state.project_root) };
}
-function routeStepFor(route: RouteKind, name: string): number {
- const index = ROUTE_STATE_STEPS[route].indexOf(name);
- if (index < 0) throw new Error(`route ${route} has no ${name} stage`);
+function variantStateResult(state: Awaited<ReturnType<typeof readVariantExpansionState>>): Record<string, unknown> {
+ return state === undefined ? { state: null } : { state, path: variantExpansionStatePath(state.output_root) };
+}
+
+function routeStepFor(route: RouteKind, name: string): number | undefined {
+ const aliases: Readonly<Record<string, string>> = route === "variant"
+ ? { "vocabulary-checked": "vocabulary-verified", "source-authored": "source-updated" }
+ : route === "variant-package"
+ ? { "vocabulary-checked": "vocabulary-inspected", "package-ready": "package-validated", "source-authored": "implemented" }
+ : {};
+ const stage = aliases[name] ?? name;
+ const index = ROUTE_STATE_STEPS[route].indexOf(stage);
+ if (index < 0) return undefined;
return index + 1;
}
@@ -317,14 +364,53 @@ async function runRevisionStateCommand(input: RevisionStateCommandInput): Promis
}));
}
+async function runVariantStateCommand(input: VariantStateCommandInput): Promise<Record<string, unknown>> {
+ if (input.action === "discover") {
+ const batches = await discoverVariantExpansions(input.project_root);
+ return { project_root: resolve(input.project_root), batches, active: batches.filter((batch) => batch.status !== "complete") };
+ }
+ if (input.action === "start") {
+ return variantStateResult(await startVariantExpansion({
+ projectRoot: input.project_root, outputRoot: input.output_root,
+ ...(input.run === undefined ? {} : { run: input.run }),
+ ...(input.request === undefined ? {} : { request: input.request }),
+ ...(input.count === undefined ? {} : { count: input.count }),
+ ...(input.delivery_mode === undefined ? {} : { deliveryMode: input.delivery_mode }),
+ ...(input.parent_route === undefined ? {} : { parentRoute: input.parent_route }),
+ ...(input.parent_state_digest === undefined ? {} : { parentStateDigest: input.parent_state_digest }),
+ ...(input.revision_state_digest === undefined ? {} : { revisionStateDigest: input.revision_state_digest }),
+ }));
+ }
+ if (input.action === "read") return variantStateResult(await readVariantExpansionState(input.project_root, input.output_root, input.batch_id));
+ if (input.action === "reconcile") return variantStateResult(await reconcileVariantExpansion(input.project_root, input.output_root, input.batch_id));
+ if (input.action !== "checkpoint") throw new Error(`unsupported variant state action ${(input as { action: string }).action}`);
+ const checkpoint: VariantExpansionCheckpointInput = {
+ projectRoot: input.project_root, step: input.step,
+ ...(input.output_root === undefined ? {} : { outputRoot: input.output_root }),
+ ...(input.batch_id === undefined ? {} : { batchId: input.batch_id }),
+ ...(input.status === undefined ? {} : { status: input.status }),
+ ...(input.next_action === undefined ? {} : { nextAction: input.next_action }),
+ ...(input.artifacts === undefined ? {} : { artifacts: input.artifacts }),
+ ...(input.workload_disclosure === undefined ? {} : { workloadDisclosure: input.workload_disclosure }),
+ ...(input.packages === undefined ? {} : { packages: input.packages }),
+ ...(input.variants === undefined ? {} : { variants: input.variants }),
+ ...(input.decision === undefined ? {} : { decision: input.decision }),
+ ...(input.conflict === undefined ? {} : { conflict: input.conflict }),
+ ...(input.resolve_conflict === undefined ? {} : { resolveConflict: input.resolve_conflict }),
+ ...(input.command === undefined ? {} : { command: input.command }),
+ ...(input.error === undefined ? {} : { error: input.error }),
+ };
+ return variantStateResult(await checkpointVariantExpansion(checkpoint));
+}
+
async function autoRouteCheckpoint(
runPath: string | undefined,
- step: number,
+ step: number | undefined,
artifacts: Readonly<Record<string, string>> = {},
nextAction?: string,
command?: string,
): Promise<void> {
- if (runPath === undefined) return;
+ if (runPath === undefined || step === undefined) return;
const projectRoot = dirname(resolve(runPath));
const state = await readRouteState(projectRoot);
if (state === undefined || state.status === "complete") return;
@@ -1782,7 +1868,7 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceV
const runPath = resolve(invokedFrom(), input.run);
const evidence = await persistRouteEvidence(runPath, "vocabulary.json", result);
const route = await readRouteState(dirname(runPath));
- const step = route?.route === "description" ? 3 : 4;
+ const step = routeStepFor(route?.route ?? "reconstruction", "vocabulary-checked");
await autoRouteCheckpoint(runPath, step, { vocabulary: evidence }, "validate_local_author_packages --run <build.svrun>");
}
return result;
@@ -1794,8 +1880,10 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceV
const evidence = await persistRouteEvidence(runPath, "package-ready.json", result);
if (result.passed === true) {
const route = await readRouteState(dirname(runPath));
- const step = route?.route === "description" ? 4 : 5;
- await autoRouteCheckpoint(runPath, step, { package_ready: evidence }, "validate_script_cues --run <build.svrun>");
+ const step = routeStepFor(route?.route ?? "reconstruction", "package-ready");
+ const next = route?.route === "variant-package" ? "run preview_check for the staging project"
+ : "validate_script_cues --run <build.svrun>";
+ await autoRouteCheckpoint(runPath, step, { package_ready: evidence }, next);
} else await autoRouteError(runPath, "validate_local_author_packages", (result.packages as unknown[] ?? []));
return result;
},
@@ -1806,7 +1894,7 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceV
const evidence = await persistRouteEvidence(runPath, "script-cues.json", result);
if (result.passed === true) {
const route = await readRouteState(dirname(runPath));
- const step = route?.route === "description" ? 5 : 6;
+ const step = routeStepFor(route?.route ?? "reconstruction", "script-checked");
await autoRouteCheckpoint(runPath, step, { script_cues: evidence }, "write main.svml, recipes.svs and build.svrun");
} else await autoRouteError(runPath, "validate_script_cues", result.violations ?? result.errors ?? "cue validation failed");
return result;
@@ -2412,9 +2500,12 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceV
const source = await authorSource(runPath).catch(() => undefined);
const route = await readRouteState(dirname(runPath));
const sourceStep = routeStepFor(route?.route ?? "reconstruction", "source-authored");
- await autoRouteCheckpoint(runPath, sourceStep, source === undefined ? {} : { author_source: source.path }, "run reconstruction_check or authoring_check");
+ const next = route?.route === "variant" ? "run variant_check --run <build.svrun>"
+ : route?.route === "variant-package" ? "freeze the package digest and checkpoint package-ready"
+ : "run reconstruction_check or authoring_check";
+ await autoRouteCheckpoint(runPath, sourceStep, source === undefined ? {} : { author_source: source.path }, next);
const routeAfterSource = await readRouteState(dirname(runPath));
- await autoRouteCheckpoint(runPath, routeStepFor(routeAfterSource?.route ?? "reconstruction", "graph-checked"), { preview_check: evidence }, "run reconstruction_check or authoring_check");
+ await autoRouteCheckpoint(runPath, routeStepFor(routeAfterSource?.route ?? "reconstruction", "graph-checked"), { preview_check: evidence }, next);
} else await autoRouteError(runPath, "preview_check", result.refused ?? result.problems ?? "preview check failed");
return result;
} catch (error) {
@@ -2484,6 +2575,226 @@ export function createReferenceVideoTools(options: ToolOptions = {}): ReferenceV
async revision_state(input): Promise<Record<string, unknown>> {
return await runRevisionStateCommand(input);
},
+
+ async variant_state(input): Promise<Record<string, unknown>> {
+ return await runVariantStateCommand(input);
+ },
+
+ async variant_init(input): Promise<Record<string, unknown>> {
+ const batch = await reconcileVariantExpansion(input.project_root, input.output_root);
+ if (batch === undefined) throw new Error("no variant expansion state; run variant_state --action start first");
+ if (batch.conflicts.length > 0) throw new Error(`variant_init is blocked by unresolved conflicts: ${batch.conflicts.join("; ")}`);
+ const required = ["workload-disclosed", "package-gaps-resolved", "slate-frozen"]
+ .map((name) => VARIANT_EXPANSION_STEPS.indexOf(name as VariantExpansionStep) + 1);
+ const missing = required.filter((step) => !batch.completed_steps.includes(step));
+ if (missing.length > 0) throw new Error(`variant_init is blocked until ${missing.map((step) => VARIANT_EXPANSION_STEPS[step - 1]).join(", ")}`);
+ const initialized = await initializeVariantProjects({
+ projectRoot: input.project_root, outputRoot: input.output_root, slatePath: input.slate,
+ expectedBaseDigest: batch.baseline_digest,
+ ...(batch.count === undefined ? {} : { expectedCount: batch.count }),
+ approvedPackages: batch.packages.flatMap((pack): ApprovedVariantPackage[] => pack.status === "ready" && pack.digest !== undefined
+ ? [{ id: pack.id, package_root: pack.package_root ?? pack.staging_root, digest: pack.digest }] : []),
+ });
+ const readPlan = async (key: "format_plan" | "component_plan"): Promise<unknown> => {
+ const path = batch.artifacts[key];
+ if (path === undefined) throw new Error(`batch state has no ${key} artifact`);
+ try { return JSON.parse(await readFile(path, "utf8")); }
+ catch (error) { throw new Error(`cannot read ${key} at ${path}: ${error instanceof Error ? error.message : String(error)}`); }
+ };
+ const [formatPlan, componentPlan] = await Promise.all([readPlan("format_plan"), readPlan("component_plan")]);
+ const selectedPlan = (plan: unknown, id: string): unknown => {
+ if (plan !== null && typeof plan === "object" && Array.isArray((plan as { variants?: unknown }).variants)) {
+ const entry = ((plan as { variants: unknown[] }).variants).find((candidate) => candidate !== null && typeof candidate === "object" && (candidate as { id?: unknown }).id === id);
+ if (entry !== undefined) return entry;
+ }
+ return plan;
+ };
+ const variants: VariantExpansionVariant[] = [];
+ for (const item of initialized.variants) {
+ const formatPlanPath = join(item.project_root, ".hypit", "format-plan.json");
+ const componentPlanPath = join(item.project_root, ".hypit", "component-plan.json");
+ await writeVariantJson(formatPlanPath, { variant_id: item.id, source: batch.artifacts.format_plan, plan: selectedPlan(formatPlan, item.id) });
+ await writeVariantJson(componentPlanPath, { variant_id: item.id, source: batch.artifacts.component_plan, plan: selectedPlan(componentPlan, item.id) });
+ await startRouteState({ projectRoot: item.project_root, route: "variant", run: item.run });
+ await checkpointRouteState({ projectRoot: item.project_root, route: "variant", step: 1, status: "complete", artifacts: { baseline_manifest: item.manifest } });
+ await checkpointRouteState({ projectRoot: item.project_root, route: "variant", step: 2, status: "complete", artifacts: { variant_brief: item.brief } });
+ await checkpointRouteState({ projectRoot: item.project_root, route: "variant", step: 3, status: "complete", artifacts: { allowed_changes: item.allowed_changes } });
+ if (item.vocabulary_mode === "inherited") {
+ await checkpointRouteState({ projectRoot: item.project_root, route: "variant", step: 5, status: "complete", artifacts: { vocabulary: join(item.project_root, ".hypit", "vocabulary.json") } });
+ }
+ variants.push({ ...item, route_state: routeStatePath(item.project_root), format_plan: formatPlanPath, component_plan: componentPlanPath });
+ }
+ const copyManifest = join(resolve(input.output_root), ".hypit", "variant-copy-manifest.json");
+ await writeVariantJson(copyManifest, {
+ version: 1, project_root: resolve(input.project_root), output_root: resolve(input.output_root),
+ base_digest: initialized.base_digest, slate: resolve(input.slate), variants, created_at: new Date().toISOString(),
+ });
+ const state = await checkpointVariantExpansion({
+ projectRoot: input.project_root, outputRoot: input.output_root, step: "projects-copied", status: "complete",
+ artifacts: { copy_manifest: copyManifest, slate: resolve(input.slate) }, variants,
+ command: `variant_init --project-root ${resolve(input.project_root)} --output-root ${resolve(input.output_root)} --slate ${resolve(input.slate)}`,
+ nextAction: "dispatch only the variant routes whose package prerequisites are ready",
+ });
+ return { passed: true, state, copy_manifest: copyManifest, variants };
+ },
+
+ async variant_check(input): Promise<Record<string, unknown>> {
+ const runPath = resolve(invokedFrom(), input.run);
+ const projectRoot = dirname(runPath);
+ const route = await readRouteState(projectRoot);
+ if (route?.route !== "variant") throw new Error(`${projectRoot} has no active variant route`);
+ const briefPath = join(projectRoot, ".hypit", "variant-brief.json");
+ const brief = JSON.parse(await readFile(briefPath, "utf8")) as {
+ id?: string;
+ batch_root?: string;
+ base_project_root?: string;
+ };
+ const manifestPath = join(projectRoot, ".hypit", "variant-baseline-manifest.json");
+ let manifest: VariantProjectManifest | undefined;
+ try { manifest = JSON.parse(await readFile(manifestPath, "utf8")) as VariantProjectManifest; }
+ catch { manifest = undefined; }
+ const batch = brief.batch_root === undefined || brief.base_project_root === undefined
+ ? undefined : await readVariantExpansionState(brief.base_project_root, brief.batch_root);
+ const expectedVariant = batch?.variants.find((variant) => resolve(variant.project_root) === projectRoot && (brief.id === undefined || variant.id === brief.id));
+ const stateBindingPassed = manifest !== undefined && expectedVariant !== undefined
+ && JSON.stringify(manifest.allowed_changes) === JSON.stringify(expectedVariant.allowed_change_paths)
+ && manifest.vocabulary_mode === expectedVariant.vocabulary_mode
+ && JSON.stringify(manifest.vocabulary_packages) === JSON.stringify(expectedVariant.vocabulary_packages)
+ && manifest.vocabulary_digest === expectedVariant.vocabulary_digest
+ && JSON.stringify(manifest.package_injections) === JSON.stringify(expectedVariant.package_injections);
+ const vocabularyPath = join(projectRoot, ".hypit", "vocabulary.json");
+ let vocabulary: Record<string, unknown> | undefined;
+ try { vocabulary = JSON.parse(await readFile(vocabularyPath, "utf8")) as Record<string, unknown>; }
+ catch { vocabulary = undefined; }
+ const requiredPackages = manifest?.vocabulary_packages ?? [];
+ const vocabularyMode = manifest?.vocabulary_mode ?? "inspect";
+ const evidencePackages = Array.isArray(vocabulary?.packages) ? vocabulary.packages.filter((item): item is string => typeof item === "string") : [];
+ const evidenceSurfaces = Array.isArray(vocabulary?.surfaces)
+ ? vocabulary.surfaces.filter((item): item is Record<string, unknown> => item !== null && typeof item === "object") : [];
+ const evidenceShapePassed = vocabulary !== undefined && Array.isArray(vocabulary.packages) && Array.isArray(vocabulary.surfaces);
+ const vocabularyBytes = await readFile(vocabularyPath).catch(() => undefined);
+ const vocabularyDigest = vocabularyBytes === undefined ? undefined : `sha256:${createHash("sha256").update(vocabularyBytes).digest("hex")}`;
+ const sourceUsage = await inspectSourceVocabularyUsage(runPath).catch(() => ({ imports: [] as readonly string[], tags: [] as readonly string[] }));
+ const baselineImports = manifest?.source_imports ?? [];
+ const addedImports = sourceUsage.imports.filter((name) => !baselineImports.includes(name));
+ const requiredImportedAndUsed = requiredPackages.every((name) => {
+ const imported = sourceUsage.imports.includes(name);
+ const tags = evidenceSurfaces.filter((surface) => surface.package_name === name)
+ .flatMap((surface) => typeof surface.tag === "string" ? [surface.tag] : []);
+ return imported && tags.length > 0 && tags.some((tag) => sourceUsage.tags.includes(tag));
+ });
+ const inheritedContractPassed = vocabularyMode === "inherited"
+ && manifest?.vocabulary_digest !== undefined && vocabularyDigest === manifest.vocabulary_digest
+ && JSON.stringify(sourceUsage.imports) === JSON.stringify(baselineImports)
+ && JSON.stringify(sourceUsage.tags) === JSON.stringify(manifest.source_tags);
+ const inspectedContractPassed = vocabularyMode === "inspect"
+ && requiredPackages.length > 0
+ && requiredPackages.every((name) => evidencePackages.includes(name))
+ && addedImports.every((name) => requiredPackages.includes(name))
+ && requiredImportedAndUsed;
+ const packageBindings = await Promise.all((manifest?.package_injections ?? []).map(async (injection) => ({
+ ...injection,
+ current_digest: await snapshotProject(join(projectRoot, injection.destination), { preserveTopLevelOutputs: true }).then((snapshot) => snapshot.digest, () => undefined),
+ })));
+ const packageBindingsPassed = packageBindings.every((binding) => binding.current_digest === binding.digest);
+ const vocabularyPassed = stateBindingPassed && evidenceShapePassed && packageBindingsPassed
+ && (inheritedContractPassed || inspectedContractPassed);
+ const packageGate = await validateLocalAuthorPackages({ run: runPath });
+ const cueGate = await scriptCueCheck({ run: runPath });
+ const graphGate = packageGate.passed === true && cueGate.passed === true
+ ? await previewCheck({ run: runPath, ...(input.runtime === undefined ? {} : { runtime: input.runtime }) }, { packageRoot })
+ .catch((error) => ({ run: runPath, sound: false, refused: error instanceof Error ? error.message : String(error) }))
+ : { run: runPath, sound: false, refused: "package or Script Cue gate failed" };
+ const mechanics = await mechanicalAuthoringCheck({ run: runPath }, { packageRoot })
+ .catch((error) => ({ run: runPath, passed: false, error: error instanceof Error ? error.message : String(error) }));
+ const minimalDiff = await inspectVariantDiff(projectRoot)
+ .catch((error) => ({ passed: false, error: error instanceof Error ? error.message : String(error) }));
+ const generatedLeakage = await findGeneratedLeakage(projectRoot, runPath)
+ .catch((error) => ({ passed: false, error: error instanceof Error ? error.message : String(error) }));
+ const result: Record<string, unknown> = {
+ run: runPath,
+ passed: vocabularyPassed && packageGate.passed === true && cueGate.passed === true
+ && graphGate.sound === true && mechanics.passed === true && minimalDiff.passed === true && generatedLeakage.passed === true,
+ vocabulary: {
+ passed: vocabularyPassed, mode: vocabularyMode, required_packages: requiredPackages, evidence: vocabularyPath,
+ evidence_digest: vocabularyDigest, baseline_digest: manifest?.vocabulary_digest,
+ state_binding: stateBindingPassed, evidence_shape: evidenceShapePassed,
+ baseline_imports: baselineImports, current_imports: sourceUsage.imports, added_imports: addedImports,
+ baseline_tags: manifest?.source_tags ?? [], current_tags: sourceUsage.tags,
+ required_packages_imported_and_used: requiredImportedAndUsed,
+ package_bindings: packageBindings,
+ },
+ package_gate: packageGate, script_cue_gate: cueGate, graph: graphGate, mechanics, minimal_diff: minimalDiff,
+ generated_result_leakage: generatedLeakage,
+ visual_review: "not performed",
+ };
+ const packageEvidence = await persistRouteEvidence(runPath, "package-ready.json", packageGate);
+ const cueEvidence = await persistRouteEvidence(runPath, "script-cues.json", cueGate);
+ const previewEvidence = await persistRouteEvidence(runPath, "preview-check.json", graphGate);
+ const finalEvidence = await persistRouteEvidence(runPath, "variant-check.json", result);
+ if (vocabularyPassed) await autoRouteCheckpoint(runPath, routeStepFor("variant", "vocabulary-checked"), { vocabulary: vocabularyPath });
+ if (packageGate.passed === true) await autoRouteCheckpoint(runPath, routeStepFor("variant", "package-ready"), { package_ready: packageEvidence });
+ if (cueGate.passed === true) await autoRouteCheckpoint(runPath, routeStepFor("variant", "script-checked"), { script_cues: cueEvidence });
+ if (graphGate.sound === true) {
+ const source = await authorSource(runPath).catch(() => undefined);
+ await autoRouteCheckpoint(runPath, routeStepFor("variant", "source-authored"), source === undefined ? {} : { author_source: source.path });
+ await autoRouteCheckpoint(runPath, routeStepFor("variant", "graph-checked"), { preview_check: previewEvidence });
+ }
+ if (result.passed === true) {
+ await autoRouteCheckpoint(runPath, routeStepFor("variant", "final-checked"), { variant_check: finalEvidence });
+ const readyToComplete = await readRouteState(projectRoot);
+ const completeStep = ROUTE_STATE_STEPS.variant.indexOf("variant-complete") + 1;
+ const allPrior = Array.from({ length: completeStep - 1 }, (_, index) => index + 1)
+ .every((step) => readyToComplete?.completed_steps.includes(step) === true);
+ if (allPrior) {
+ await checkpointRouteState({ projectRoot, route: "variant", step: "variant-complete", status: "complete", artifacts: { variant_check: finalEvidence } });
+ }
+ } else {
+ const outsideScope = Array.isArray((minimalDiff as { outside_allowed_changes?: unknown }).outside_allowed_changes)
+ && ((minimalDiff as { outside_allowed_changes: unknown[] }).outside_allowed_changes.length > 0);
+ if (outsideScope) {
+ await checkpointRouteState({ projectRoot, route: "variant", step: route.current_step, status: "blocked", run: runPath,
+ command: "variant_check", error: "scope-expansion-required" });
+ } else await autoRouteError(runPath, "variant_check", result);
+ }
+
+ result.route_state = await readRouteState(projectRoot);
+
+ if (brief.batch_root !== undefined && brief.base_project_root !== undefined) {
+ let batch = await reconcileVariantExpansion(brief.base_project_root, brief.batch_root);
+ if (batch !== undefined) {
+ const currentRoute = await readRouteState(projectRoot);
+ const outsideScope = Array.isArray((minimalDiff as { outside_allowed_changes?: unknown }).outside_allowed_changes)
+ && ((minimalDiff as { outside_allowed_changes: unknown[] }).outside_allowed_changes.length > 0);
+ const variants = batch.variants.map((variant) => variant.project_root === projectRoot
+ ? { ...variant, status: currentRoute?.status === "complete" ? "complete" as const
+ : outsideScope ? "scope-expansion-required" as const : result.passed === true ? "dispatched" as const : "failed" as const,
+ ...(result.passed === true ? {} : { error: outsideScope ? "scope-expansion-required" : "variant_check failed" }) }
+ : variant);
+ batch = await checkpointVariantExpansion({
+ projectRoot: brief.base_project_root, outputRoot: brief.batch_root,
+ step: batch.current_step <= VARIANT_EXPANSION_STEPS.length ? batch.current_step : VARIANT_EXPANSION_STEPS.length,
+ status: batch.status === "blocked" ? "blocked" : "in_progress", variants,
+ });
+ const reports = await Promise.all(variants.map(async (variant) => {
+ const path = join(variant.project_root, ".hypit", "variant-check.json");
+ let report: { passed?: unknown } | undefined;
+ try { report = JSON.parse(await readFile(path, "utf8")) as { passed?: unknown }; } catch { report = undefined; }
+ return { id: variant.id, project_root: variant.project_root, route_complete: (await readRouteState(variant.project_root))?.status === "complete", passed: report?.passed === true, report: path };
+ }));
+ const aggregate = { passed: reports.length > 0 && reports.every((item) => item.route_complete && item.passed), variants: reports, checked_at: new Date().toISOString() };
+ const aggregatePath = join(resolve(brief.batch_root), ".hypit", "aggregate-check.json");
+ await writeVariantJson(aggregatePath, aggregate);
+ if (reports.every((item) => item.route_complete)) {
+ await checkpointVariantExpansion({ projectRoot: brief.base_project_root, outputRoot: brief.batch_root, step: "variants-complete", status: "complete", variants });
+ }
+ if (aggregate.passed) {
+ await checkpointVariantExpansion({ projectRoot: brief.base_project_root, outputRoot: brief.batch_root, step: "aggregate-checked", status: "complete", artifacts: { aggregate_check: aggregatePath }, variants });
+ }
+ }
+ }
+ return result;
+ },
};
return tools;
}
diff --git a/packages/reference-video-tools/src/variant-project.ts b/packages/reference-video-tools/src/variant-project.ts
new file mode 100644
index 00000000..f967d026
--- /dev/null
+++ b/packages/reference-video-tools/src/variant-project.ts
@@ -0,0 +1,538 @@
+import { createHash } from "node:crypto";
+import { createReadStream } from "node:fs";
+import { constants, chmod, copyFile, lstat, mkdir, readFile, readdir, readlink, rename, symlink, writeFile } from "node:fs/promises";
+import { dirname, extname, isAbsolute, join, relative, resolve, sep } from "node:path";
+
+import { parseRunDocument } from "@hypit/run-markup";
+
+export type VariantManifestEntry = {
+ readonly path: string;
+ readonly kind: "file" | "symlink";
+ readonly digest: string;
+ readonly size: number;
+ readonly mode: number;
+};
+
+export type VariantProjectManifest = {
+ readonly version: 1;
+ readonly project_root: string;
+ readonly base_project_root: string;
+ readonly base_digest: string;
+ readonly slate_entry_digest: string;
+ readonly allowed_changes: readonly string[];
+ readonly vocabulary_mode: "inherited" | "inspect";
+ readonly vocabulary_packages: readonly string[];
+ readonly vocabulary_digest?: string;
+ readonly source_imports: readonly string[];
+ readonly source_tags: readonly string[];
+ readonly package_injections: readonly {
+ readonly package_id: string;
+ readonly destination: string;
+ readonly digest: string;
+ }[];
+ readonly files: readonly VariantManifestEntry[];
+ readonly digest: string;
+ readonly created_at: string;
+};
+
+export type SlatePackageInjection = { readonly package_id: string; readonly destination: string };
+export type SlateVariant = {
+ readonly id?: string;
+ readonly slug: string;
+ readonly brief?: unknown;
+ readonly component_class?: "svml-only" | "existing-component" | "composed-components" | "new-package";
+ readonly allowed_changes: readonly string[];
+ readonly vocabulary?: { readonly mode?: "inherited" | "inspect"; readonly packages?: readonly string[] };
+ readonly inject_packages?: readonly SlatePackageInjection[];
+ readonly [key: string]: unknown;
+};
+export type VariantSlate = { readonly variants: readonly SlateVariant[]; readonly [key: string]: unknown };
+
+export type InitializedVariant = {
+ readonly id: string;
+ readonly slug: string;
+ readonly project_root: string;
+ readonly run: string;
+ readonly manifest: string;
+ readonly brief: string;
+ readonly allowed_changes: string;
+ readonly allowed_change_paths: readonly string[];
+ readonly component_class: "svml-only" | "existing-component" | "composed-components" | "new-package";
+ readonly vocabulary_mode: "inherited" | "inspect";
+ readonly vocabulary_packages: readonly string[];
+ readonly vocabulary_digest?: string;
+ readonly package_injections: VariantProjectManifest["package_injections"];
+ readonly status: "ready";
+};
+
+const EXCLUDED_DIRECTORIES = new Set([".git", "node_modules", ".hypit", ".cache", ".turbo"]);
+const EXCLUDED_TOP_LEVEL_DIRECTORIES = new Set([
+ "coverage", "renders", "generated", "artifacts", "logs", "cache", "build-artifacts", "builds", "output", "outputs", "dist",
+]);
+const SECRET_FILE = /^(?:\.env(?:\..+)?|\.npmrc|\.yarnrc|credentials?(?:\..+)?\.json|secrets?(?:\..+)?\.json|service-account(?:\..+)?\.json)$/iu;
+
+function normalizedRelative(path: string): string {
+ return path.split(sep).join("/");
+}
+
+function exclusionReason(relativePath: string, directory: boolean, preserveTopLevelOutputs = false): string | undefined {
+ const normalized = normalizedRelative(relativePath);
+ const parts = normalized.split("/").filter(Boolean);
+ const name = parts.at(-1) ?? "";
+ if (parts.some((part) => EXCLUDED_DIRECTORIES.has(part))) return "generated or dependency directory";
+ const insideProjectPackage = parts[0] === "packages" && parts.length > 2;
+ if (!preserveTopLevelOutputs && !insideProjectPackage && parts.some((part) => EXCLUDED_TOP_LEVEL_DIRECTORIES.has(part))) {
+ return "generated output directory";
+ }
+ if (directory) return undefined;
+ if (SECRET_FILE.test(name) || /\.(?:pem|key)$/iu.test(name)) return "credential or secret file";
+ if (/\.(?:log|tmp|tsbuildinfo)$/iu.test(name)) return "log or cache file";
+ if (extname(name).toLowerCase() === ".svrun" && normalized !== "build.svrun") return "non-canonical preview/mock/accepted-material Run";
+ return undefined;
+}
+
+async function fileDigest(path: string): Promise<string> {
+ const hash = createHash("sha256");
+ await new Promise<void>((fulfil, reject) => {
+ const stream = createReadStream(path);
+ stream.on("data", (chunk) => hash.update(chunk));
+ stream.on("error", reject);
+ stream.on("end", fulfil);
+ });
+ return `sha256:${hash.digest("hex")}`;
+}
+
+function manifestDigest(entries: readonly VariantManifestEntry[]): string {
+ return `sha256:${createHash("sha256").update(JSON.stringify(entries)).digest("hex")}`;
+}
+
+function jsonDigest(value: unknown): string {
+ return `sha256:${createHash("sha256").update(JSON.stringify(value)).digest("hex")}`;
+}
+
+async function walkProject(root: string, preserveTopLevelOutputs = false): Promise<readonly VariantManifestEntry[]> {
+ const entries: VariantManifestEntry[] = [];
+ const visit = async (directory: string): Promise<void> => {
+ const children = await readdir(directory, { withFileTypes: true });
+ children.sort((left, right) => left.name.localeCompare(right.name));
+ for (const child of children) {
+ const path = join(directory, child.name);
+ const rel = normalizedRelative(relative(root, path));
+ if (exclusionReason(rel, child.isDirectory(), preserveTopLevelOutputs) !== undefined) continue;
+ const stats = await lstat(path);
+ if (stats.isDirectory()) {
+ await visit(path);
+ } else if (stats.isSymbolicLink()) {
+ const target = await readlink(path);
+ entries.push({ path: rel, kind: "symlink", digest: `sha256:${createHash("sha256").update(target).digest("hex")}`, size: target.length, mode: stats.mode & 0o777 });
+ } else if (stats.isFile()) {
+ entries.push({ path: rel, kind: "file", digest: await fileDigest(path), size: stats.size, mode: stats.mode & 0o777 });
+ }
+ }
+ };
+ await visit(root);
+ return entries.sort((left, right) => left.path.localeCompare(right.path));
+}
+
+export async function snapshotProject(projectRoot: string, options: { readonly preserveTopLevelOutputs?: boolean } = {}): Promise<{ readonly files: readonly VariantManifestEntry[]; readonly digest: string }> {
+ const files = await walkProject(resolve(projectRoot), options.preserveTopLevelOutputs === true);
+ return { files, digest: manifestDigest(files) };
+}
+
+async function copyOne(source: string, destination: string, mode: number): Promise<void> {
+ await mkdir(dirname(destination), { recursive: true });
+ try { await copyFile(source, destination, constants.COPYFILE_FICLONE); }
+ catch { await copyFile(source, destination); }
+ await chmod(destination, mode);
+}
+
+async function copySelectedTree(sourceRoot: string, destinationRoot: string, preserveTopLevelOutputs = false): Promise<void> {
+ const visit = async (sourceDirectory: string): Promise<void> => {
+ const children = await readdir(sourceDirectory, { withFileTypes: true });
+ children.sort((left, right) => left.name.localeCompare(right.name));
+ for (const child of children) {
+ const source = join(sourceDirectory, child.name);
+ const rel = normalizedRelative(relative(sourceRoot, source));
+ if (exclusionReason(rel, child.isDirectory(), preserveTopLevelOutputs) !== undefined) continue;
+ const destination = join(destinationRoot, rel);
+ const stats = await lstat(source);
+ if (stats.isDirectory()) {
+ await mkdir(destination, { recursive: true });
+ await visit(source);
+ } else if (stats.isSymbolicLink()) {
+ await mkdir(dirname(destination), { recursive: true });
+ await symlink(await readlink(source), destination);
+ } else if (stats.isFile()) {
+ await copyOne(source, destination, stats.mode & 0o777);
+ }
+ }
+ };
+ await mkdir(destinationRoot, { recursive: true });
+ await visit(sourceRoot);
+}
+
+function attribute(text: string, name: string): string | undefined {
+ return new RegExp(`\\b${name}="([^"]+)"`, "u").exec(text)?.[1];
+}
+
+export async function removeGeneratedRunBindings(runPath: string): Promise<{ readonly removed_build_records: readonly string[]; readonly removed_satisfactions: readonly string[] }> {
+ const original = await readFile(runPath, "utf8");
+ const runBody = (source: string): string => source.replace(/^\s*<\?svml\s+using="[^"]+"\?>\s*/u, "");
+ parseRunDocument(runPath, runBody(original));
+ const buildIds = new Set<string>();
+ for (const match of original.matchAll(/<build-record\b[^>]*\/\s*>/gsu)) {
+ const id = attribute(match[0], "id");
+ if (id !== undefined) buildIds.add(id);
+ }
+ if (buildIds.size === 0) return { removed_build_records: [], removed_satisfactions: [] };
+ const satisfactions: string[] = [];
+ let cleaned = original.replace(/<build-record\b[^>]*\/\s*>\s*/gsu, "");
+ cleaned = cleaned.replace(/<satisfy\b[^>]*\/\s*>\s*/gsu, (element) => {
+ const candidate = attribute(element, "candidate");
+ if (candidate === undefined || !buildIds.has(candidate)) return element;
+ satisfactions.push(candidate);
+ return "";
+ });
+ parseRunDocument(runPath, runBody(cleaned));
+ const temporary = `${runPath}.tmp-${process.pid}-${Date.now()}`;
+ await writeFile(temporary, cleaned, "utf8");
+ await rename(temporary, runPath);
+ return { removed_build_records: [...buildIds], removed_satisfactions: satisfactions };
+}
+
+function safeSlug(value: string): string {
+ const slug = value.normalize("NFKD").toLowerCase().replace(/[^a-z0-9]+/gu, "-").replace(/^-+|-+$/gu, "");
+ if (slug.length === 0) throw new Error(`variant slug ${JSON.stringify(value)} has no filesystem-safe characters`);
+ return slug.slice(0, 64);
+}
+
+function assertSlate(value: unknown, path: string): asserts value is VariantSlate {
+ if (value === null || typeof value !== "object" || !Array.isArray((value as { variants?: unknown }).variants)) throw new Error(`${path} must contain a variants array`);
+ const variants = (value as { variants: unknown[] }).variants;
+ if (variants.length === 0) throw new Error(`${path} must contain at least one variant`);
+ const slugs = new Set<string>();
+ const ids = new Set<string>();
+ for (const [index, item] of variants.entries()) {
+ if (item === null || typeof item !== "object") throw new Error(`${path} variants[${index}] must be an object`);
+ const variant = item as Partial<SlateVariant>;
+ if (typeof variant.slug !== "string" || variant.slug.trim().length === 0) throw new Error(`${path} variants[${index}] has no slug`);
+ const slug = safeSlug(variant.slug);
+ if (slugs.has(slug)) throw new Error(`${path} repeats variant slug ${slug}`);
+ slugs.add(slug);
+ if (variant.id !== undefined) {
+ if (typeof variant.id !== "string" || variant.id.trim().length === 0) throw new Error(`${path} variants[${index}] has an invalid id`);
+ if (ids.has(variant.id.trim())) throw new Error(`${path} repeats variant id ${variant.id.trim()}`);
+ ids.add(variant.id.trim());
+ }
+ if (!Array.isArray(variant.allowed_changes) || variant.allowed_changes.length === 0 || variant.allowed_changes.some((entry) => typeof entry !== "string" || entry.trim().length === 0)) {
+ throw new Error(`${path} variants[${index}] must declare non-empty allowed_changes`);
+ }
+ for (const allowed of variant.allowed_changes) {
+ const normalized = normalizedRelative(allowed).replace(/^\.\//u, "");
+ if (isAbsolute(allowed) || normalized === ".." || normalized.startsWith("../") || normalized === ".hypit" || normalized.startsWith(".hypit/")) {
+ throw new Error(`${path} variants[${index}] has unsafe allowed_changes entry ${allowed}`);
+ }
+ }
+ const componentClass = variant.component_class;
+ if (componentClass !== undefined && componentClass !== "svml-only" && componentClass !== "existing-component"
+ && componentClass !== "composed-components" && componentClass !== "new-package") {
+ throw new Error(`${path} variants[${index}] has invalid component_class`);
+ }
+ if ((componentClass ?? "svml-only") === "svml-only") {
+ const normalizedAllowed = variant.allowed_changes.map((entry) => normalizedRelative(entry).replace(/^\.\//u, ""));
+ if (normalizedAllowed.length !== 1 || normalizedAllowed[0] !== "main.svml") {
+ throw new Error(`${path} variants[${index}] is svml-only and must allow exactly main.svml`);
+ }
+ }
+ if (variant.vocabulary?.mode !== undefined && variant.vocabulary.mode !== "inherited" && variant.vocabulary.mode !== "inspect") {
+ throw new Error(`${path} variants[${index}] has invalid vocabulary.mode`);
+ }
+ if (variant.vocabulary?.packages !== undefined && (!Array.isArray(variant.vocabulary.packages)
+ || variant.vocabulary.packages.some((entry) => typeof entry !== "string" || entry.trim().length === 0))) {
+ throw new Error(`${path} variants[${index}] has invalid vocabulary.packages`);
+ }
+ if ((componentClass === "existing-component" || componentClass === "composed-components" || componentClass === "new-package")
+ && (variant.vocabulary?.mode !== "inspect" || (variant.vocabulary.packages?.length ?? 0) === 0)) {
+ throw new Error(`${path} variants[${index}] changes components and must inspect at least one assigned package`);
+ }
+ const injections = variant.inject_packages ?? [];
+ if (!Array.isArray(injections) || injections.some((entry) => entry === null || typeof entry !== "object"
+ || typeof entry.package_id !== "string" || entry.package_id.trim().length === 0
+ || typeof entry.destination !== "string" || entry.destination.trim().length === 0)) {
+ throw new Error(`${path} variants[${index}] has invalid inject_packages`);
+ }
+ if ((componentClass ?? "svml-only") === "new-package" && injections.length === 0) {
+ throw new Error(`${path} variants[${index}] is new-package but declares no package injection`);
+ }
+ if ((componentClass ?? "svml-only") !== "new-package" && injections.length > 0) {
+ throw new Error(`${path} variants[${index}] may inject packages only when component_class is new-package`);
+ }
+ const injectionIds = injections.map((entry) => entry.package_id.trim());
+ if (new Set(injectionIds).size !== injectionIds.length) throw new Error(`${path} variants[${index}] repeats an injected package id`);
+ const injectionDestinations = injections.map((entry) => normalizedRelative(entry.destination).replace(/^\.\//u, ""));
+ if (new Set(injectionDestinations).size !== injectionDestinations.length) throw new Error(`${path} variants[${index}] repeats an injection destination`);
+ for (const destination of injectionDestinations) {
+ if (isAbsolute(destination) || destination === "packages" || !destination.startsWith("packages/")
+ || destination.includes("/../") || destination.endsWith("/..")) {
+ throw new Error(`${path} variants[${index}] has unsafe package injection destination ${destination}`);
+ }
+ }
+ }
+}
+
+export async function writeVariantJson(path: string, value: unknown): Promise<void> {
+ await mkdir(dirname(path), { recursive: true });
+ const temporary = `${path}.tmp-${process.pid}-${Date.now()}`;
+ await writeFile(temporary, `${JSON.stringify(value, null, 2)}\n`, "utf8");
+ await rename(temporary, path);
+}
+
+function inside(root: string, path: string): boolean {
+ const rel = relative(root, path);
+ return rel.length === 0 || (!rel.startsWith(`..${sep}`) && rel !== ".." && !isAbsolute(rel));
+}
+
+export type ApprovedVariantPackage = {
+ readonly id: string;
+ readonly package_root: string;
+ readonly digest: string;
+};
+
+async function injectPackages(
+ projectRoot: string,
+ injections: readonly SlatePackageInjection[] | undefined,
+ approvedPackages: ReadonlyMap<string, ApprovedVariantPackage>,
+): Promise<VariantProjectManifest["package_injections"]> {
+ const injected: VariantProjectManifest["package_injections"][number][] = [];
+ for (const injection of injections ?? []) {
+ if (typeof injection?.package_id !== "string" || typeof injection.destination !== "string") throw new Error("inject_packages entries require package_id and destination");
+ const approved = approvedPackages.get(injection.package_id);
+ if (approved === undefined) throw new Error(`package injection ${injection.package_id} is not a ready frozen batch package`);
+ const source = resolve(approved.package_root);
+ const destination = resolve(projectRoot, injection.destination);
+ if (!inside(projectRoot, destination) || !normalizedRelative(relative(projectRoot, destination)).startsWith("packages/")) {
+ throw new Error(`package injection destination must stay under packages/: ${injection.destination}`);
+ }
+ const present = await lstat(destination).then(() => true, () => false);
+ if (present) throw new Error(`package injection would overwrite ${destination}`);
+ const sourceStats = await lstat(source).catch(() => undefined);
+ if (sourceStats?.isDirectory() !== true) throw new Error(`package injection source is not a directory: ${source}`);
+ const sourceDigest = (await snapshotProject(source, { preserveTopLevelOutputs: true })).digest;
+ if (sourceDigest !== approved.digest) throw new Error(`package injection ${injection.package_id} changed after it was frozen: expected ${approved.digest}, found ${sourceDigest}`);
+ await copySelectedTree(source, destination, true);
+ const destinationDigest = (await snapshotProject(destination, { preserveTopLevelOutputs: true })).digest;
+ if (destinationDigest !== approved.digest) throw new Error(`package injection ${injection.package_id} did not copy byte-for-byte into ${destination}`);
+ injected.push({ package_id: injection.package_id, destination: normalizedRelative(relative(projectRoot, destination)), digest: approved.digest });
+ }
+ return injected;
+}
+
+export async function inspectSourceVocabularyUsage(runPath: string): Promise<{ readonly imports: readonly string[]; readonly tags: readonly string[] }> {
+ const visited = new Set<string>();
+ const imports = new Set<string>();
+ const tags = new Set<string>();
+ const visit = async (path: string): Promise<void> => {
+ const absolute = resolve(path);
+ if (visited.has(absolute)) return;
+ visited.add(absolute);
+ const source = await readFile(absolute, "utf8");
+ for (const match of source.matchAll(/\bfrom="([^"]+)@\d+"/gu)) if (match[1]!.startsWith("@")) imports.add(match[1]!);
+ for (const match of source.matchAll(/<([A-Z][A-Za-z0-9_.:-]*)\b/gu)) tags.add(match[1]!);
+ for (const match of source.matchAll(/\bsource="([^"#]+)"/gu)) {
+ const child = match[1]!;
+ if (child.startsWith("@") || child.startsWith("http://") || child.startsWith("https://")) continue;
+ await visit(resolve(dirname(absolute), child));
+ }
+ };
+ await visit(runPath);
+ return { imports: [...imports].sort(), tags: [...tags].sort() };
+}
+
+export async function initializeVariantProjects(input: {
+ readonly projectRoot: string;
+ readonly outputRoot: string;
+ readonly slatePath: string;
+ readonly expectedBaseDigest?: string;
+ readonly expectedCount?: number;
+ readonly approvedPackages?: readonly ApprovedVariantPackage[];
+}): Promise<{ readonly slate: VariantSlate; readonly base_digest: string; readonly variants: readonly InitializedVariant[] }> {
+ const projectRoot = resolve(input.projectRoot);
+ const outputRoot = resolve(input.outputRoot);
+ if (inside(projectRoot, outputRoot)) throw new Error("variant output root must not be inside the base project");
+ const slatePath = resolve(input.slatePath);
+ let slateValue: unknown;
+ try { slateValue = JSON.parse(await readFile(slatePath, "utf8")); }
+ catch (error) { throw new Error(`cannot read slate ${slatePath}: ${error instanceof Error ? error.message : String(error)}`); }
+ assertSlate(slateValue, slatePath);
+ const slate = slateValue;
+ if (input.expectedCount !== undefined && slate.variants.length !== input.expectedCount) {
+ throw new Error(`slate has ${slate.variants.length} variants, expected ${input.expectedCount}`);
+ }
+ const approvedPackages = new Map((input.approvedPackages ?? []).map((pack) => [pack.id, pack]));
+ for (const variant of slate.variants) {
+ for (const injection of variant.inject_packages ?? []) {
+ if (!approvedPackages.has(injection.package_id)) throw new Error(`slate references package ${injection.package_id}, which is not ready and frozen in batch state`);
+ }
+ }
+ const base = await snapshotProject(projectRoot);
+ if (input.expectedBaseDigest !== undefined && input.expectedBaseDigest !== base.digest) {
+ throw new Error(`base project digest changed: expected ${input.expectedBaseDigest}, found ${base.digest}`);
+ }
+ await mkdir(outputRoot, { recursive: true });
+ const width = Math.max(3, String(slate.variants.length).length);
+ const inheritedVocabulary = join(projectRoot, ".hypit", "vocabulary.json");
+ const initialized: InitializedVariant[] = [];
+ const assignedIds = new Set<string>();
+ for (const [index, variant] of slate.variants.entries()) {
+ const number = String(index + 1).padStart(width, "0");
+ const slug = safeSlug(variant.slug);
+ const id = typeof variant.id === "string" && variant.id.trim().length > 0 ? variant.id.trim() : number;
+ const slateEntryDigest = jsonDigest(variant);
+ if (assignedIds.has(id)) throw new Error(`slate resolves more than one variant to id ${id}`);
+ assignedIds.add(id);
+ const variantRoot = join(outputRoot, `${number}-${slug}`);
+ const manifestPath = join(variantRoot, ".hypit", "variant-baseline-manifest.json");
+ const alreadyInitialized = await readFile(manifestPath, "utf8").then(() => true, () => false);
+ if (alreadyInitialized) {
+ let manifest: VariantProjectManifest;
+ let brief: { id?: unknown; slug?: unknown; slate_entry_digest?: unknown };
+ let allowed: { allowed_changes?: unknown };
+ try {
+ manifest = JSON.parse(await readFile(manifestPath, "utf8")) as VariantProjectManifest;
+ brief = JSON.parse(await readFile(join(variantRoot, ".hypit", "variant-brief.json"), "utf8")) as { id?: unknown; slug?: unknown; slate_entry_digest?: unknown };
+ allowed = JSON.parse(await readFile(join(variantRoot, ".hypit", "allowed-changes.json"), "utf8")) as { allowed_changes?: unknown };
+ } catch (error) {
+ throw new Error(`existing variant ${variantRoot} has unreadable initialization evidence: ${error instanceof Error ? error.message : String(error)}`);
+ }
+ const diff = await inspectVariantDiff(variantRoot);
+ if (manifest.base_digest !== base.digest || manifest.slate_entry_digest !== slateEntryDigest
+ || brief.id !== id || brief.slug !== slug || brief.slate_entry_digest !== slateEntryDigest
+ || JSON.stringify(allowed.allowed_changes) !== JSON.stringify(variant.allowed_changes)
+ || diff.passed !== true) {
+ throw new Error(`existing variant ${variantRoot} conflicts with the current base or slate`);
+ }
+ } else {
+ const occupied = await readdir(variantRoot).then((items) => items.length > 0, () => false);
+ if (occupied) throw new Error(`variant destination already exists without a manifest: ${variantRoot}`);
+ await copySelectedTree(projectRoot, variantRoot);
+ const run = join(variantRoot, "build.svrun");
+ const hasRun = await lstat(run).then((stats) => stats.isFile(), () => false);
+ if (!hasRun) throw new Error(`base project has no canonical top-level build.svrun`);
+ await removeGeneratedRunBindings(run);
+ const packageInjections = await injectPackages(variantRoot, variant.inject_packages, approvedPackages);
+ const vocabularyMode = variant.vocabulary?.mode ?? "inherited";
+ let vocabularyDigest: string | undefined;
+ if (vocabularyMode === "inherited") {
+ const evidence = await readFile(inheritedVocabulary).catch(() => undefined);
+ if (evidence === undefined) throw new Error(`variant ${id} inherits vocabulary but ${inheritedVocabulary} is missing`);
+ vocabularyDigest = `sha256:${createHash("sha256").update(evidence).digest("hex")}`;
+ await mkdir(join(variantRoot, ".hypit"), { recursive: true });
+ await writeFile(join(variantRoot, ".hypit", "vocabulary.json"), evidence);
+ }
+ const briefPath = join(variantRoot, ".hypit", "variant-brief.json");
+ const allowedPath = join(variantRoot, ".hypit", "allowed-changes.json");
+ await writeVariantJson(briefPath, { ...variant, id, slug, slate_entry_digest: slateEntryDigest, index: index + 1, batch_root: outputRoot, base_project_root: projectRoot });
+ await writeVariantJson(allowedPath, { allowed_changes: variant.allowed_changes });
+ const copied = await snapshotProject(variantRoot);
+ const sourceUsage = await inspectSourceVocabularyUsage(join(variantRoot, "build.svrun"));
+ const manifest: VariantProjectManifest = {
+ version: 1, project_root: variantRoot, base_project_root: projectRoot, base_digest: base.digest,
+ slate_entry_digest: slateEntryDigest, allowed_changes: variant.allowed_changes,
+ vocabulary_mode: vocabularyMode, vocabulary_packages: variant.vocabulary?.packages ?? [],
+ ...(vocabularyDigest === undefined ? {} : { vocabulary_digest: vocabularyDigest }),
+ source_imports: sourceUsage.imports, source_tags: sourceUsage.tags, package_injections: packageInjections,
+ files: copied.files, digest: copied.digest, created_at: new Date().toISOString(),
+ };
+ await writeVariantJson(manifestPath, manifest);
+ }
+ const componentClass = variant.component_class === "existing-component" || variant.component_class === "composed-components" || variant.component_class === "new-package"
+ ? variant.component_class : "svml-only";
+ const persistedManifest = JSON.parse(await readFile(manifestPath, "utf8")) as VariantProjectManifest;
+ initialized.push({
+ id, slug, project_root: variantRoot, run: join(variantRoot, "build.svrun"), manifest: manifestPath,
+ brief: join(variantRoot, ".hypit", "variant-brief.json"), allowed_changes: join(variantRoot, ".hypit", "allowed-changes.json"),
+ allowed_change_paths: persistedManifest.allowed_changes,
+ component_class: componentClass, vocabulary_mode: variant.vocabulary?.mode ?? "inherited",
+ vocabulary_packages: variant.vocabulary?.packages ?? [],
+ ...(persistedManifest.vocabulary_digest === undefined ? {} : { vocabulary_digest: persistedManifest.vocabulary_digest }),
+ package_injections: persistedManifest.package_injections,
+ status: "ready",
+ });
+ }
+ return { slate, base_digest: base.digest, variants: initialized };
+}
+
+function globExpression(pattern: string): RegExp {
+ const normalized = normalizedRelative(pattern).replace(/^\.\//u, "");
+ let source = "^";
+ for (let index = 0; index < normalized.length; index += 1) {
+ const character = normalized[index]!;
+ if (character === "*") {
+ if (normalized[index + 1] === "*") { source += ".*"; index += 1; }
+ else source += "[^/]*";
+ } else if (character === "?") source += "[^/]";
+ else source += character.replace(/[|\\{}()[\]^$+?.]/gu, "\\$&");
+ }
+ return new RegExp(`${source}$`, "u");
+}
+
+export async function inspectVariantDiff(projectRoot: string): Promise<Record<string, unknown>> {
+ const root = resolve(projectRoot);
+ const manifestPath = join(root, ".hypit", "variant-baseline-manifest.json");
+ let manifest: VariantProjectManifest;
+ try { manifest = JSON.parse(await readFile(manifestPath, "utf8")) as VariantProjectManifest; }
+ catch (error) { return { passed: false, manifest: manifestPath, errors: [`BASELINE_MANIFEST_UNREADABLE: ${error instanceof Error ? error.message : String(error)}`] }; }
+ if (manifest.version !== 1 || !Array.isArray(manifest.files) || !Array.isArray(manifest.allowed_changes)) {
+ return { passed: false, manifest: manifestPath, errors: ["BASELINE_MANIFEST_INVALID"] };
+ }
+ const allowedPath = join(root, ".hypit", "allowed-changes.json");
+ const allowedChanges = manifest.allowed_changes;
+ try {
+ const currentAllowed = JSON.parse(await readFile(allowedPath, "utf8")) as { allowed_changes?: unknown };
+ if (!Array.isArray(currentAllowed.allowed_changes) || currentAllowed.allowed_changes.some((item) => typeof item !== "string" || item.trim().length === 0)) {
+ return { passed: false, manifest: manifestPath, allowed_changes: allowedPath, errors: ["ALLOWED_CHANGES_INVALID"] };
+ }
+ if (JSON.stringify(currentAllowed.allowed_changes) !== JSON.stringify(manifest.allowed_changes)) {
+ return { passed: false, manifest: manifestPath, allowed_changes: allowedPath, errors: ["ALLOWED_CHANGES_CHANGED_OUTSIDE_BATCH_STATE"], outside_allowed_changes: [] };
+ }
+ } catch (error) {
+ return { passed: false, manifest: manifestPath, allowed_changes: allowedPath, errors: [`ALLOWED_CHANGES_UNREADABLE: ${error instanceof Error ? error.message : String(error)}`] };
+ }
+ const current = await snapshotProject(root);
+ const before = new Map(manifest.files.map((entry) => [entry.path, entry]));
+ const after = new Map(current.files.map((entry) => [entry.path, entry]));
+ const changed = [...new Set([...before.keys(), ...after.keys()])].filter((path) => {
+ const left = before.get(path); const right = after.get(path);
+ return left?.digest !== right?.digest || left?.kind !== right?.kind || left?.mode !== right?.mode;
+ }).sort();
+ const allowed = allowedChanges.map(globExpression);
+ const outside = changed.filter((path) => !allowed.some((pattern) => pattern.test(path)));
+ return {
+ passed: outside.length === 0,
+ manifest: manifestPath,
+ baseline_digest: manifest.digest,
+ current_digest: current.digest,
+ allowed_changes: allowedChanges,
+ changed,
+ outside_allowed_changes: outside,
+ };
+}
+
+export async function findGeneratedLeakage(projectRoot: string, runPath: string): Promise<Record<string, unknown>> {
+ const root = resolve(projectRoot);
+ const leaks: { path: string; reason: string }[] = [];
+ const visit = async (directory: string): Promise<void> => {
+ for (const child of await readdir(directory, { withFileTypes: true }).catch(() => [])) {
+ const path = join(directory, child.name);
+ const rel = normalizedRelative(relative(root, path));
+ if (rel === ".hypit" || rel.startsWith(".hypit/") || rel === "node_modules" || rel.startsWith("node_modules/") || rel === ".git" || rel.startsWith(".git/")) continue;
+ const reason = exclusionReason(rel, child.isDirectory());
+ if (reason !== undefined) { leaks.push({ path: rel, reason }); continue; }
+ if (child.isDirectory()) await visit(path);
+ }
+ };
+ await visit(root);
+ const run = await readFile(resolve(runPath), "utf8").catch(() => "");
+ const generatedCandidates = [...run.matchAll(/<build-record\b[^>]*\bid="([^"]+)"[^>]*\/\s*>/gsu)].map((match) => match[1]!);
+ return { passed: leaks.length === 0 && generatedCandidates.length === 0, leaks, generated_build_candidates: generatedCandidates };
+}
diff --git a/packages/reference-video-tools/src/variant-state.ts b/packages/reference-video-tools/src/variant-state.ts
new file mode 100644
index 00000000..b247b3b3
--- /dev/null
+++ b/packages/reference-video-tools/src/variant-state.ts
@@ -0,0 +1,502 @@
+import { createHash } from "node:crypto";
+import { mkdir, readFile, rename, stat, writeFile } from "node:fs/promises";
+import { basename, dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
+
+import { checkpointRouteState, readRouteState, reconcileRouteState, routeStatePath } from "./route-state.js";
+import { inspectVariantDiff, snapshotProject } from "./variant-project.js";
+
+export type VariantExpansionStatus = "active" | "complete" | "blocked";
+export type VariantExpansionStep =
+ | "baseline-validated" | "examples-inspected" | "format-plan-frozen" | "slate-drafted"
+ | "vocabulary-enumerated" | "component-plan-frozen" | "package-gaps-classified"
+ | "workload-disclosed" | "package-gaps-resolved" | "slate-frozen" | "projects-copied"
+ | "variants-dispatched" | "variants-complete" | "aggregate-checked" | "build-planned"
+ | "build-approved" | "build-complete" | "expansion-complete";
+
+export const VARIANT_EXPANSION_STATE_VERSION = 1 as const;
+export const VARIANT_EXPANSION_STEPS: readonly VariantExpansionStep[] = [
+ "baseline-validated", "examples-inspected", "format-plan-frozen", "slate-drafted",
+ "vocabulary-enumerated", "component-plan-frozen", "package-gaps-classified", "workload-disclosed",
+ "package-gaps-resolved", "slate-frozen", "projects-copied", "variants-dispatched",
+ "variants-complete", "aggregate-checked", "build-planned", "build-approved", "build-complete",
+ "expansion-complete",
+];
+
+export type VariantWorkloadDisclosure = {
+ readonly svml_only: number;
+ readonly medium: number;
+ readonly new_packages: number;
+ readonly package_purposes?: readonly string[];
+ readonly paid_generation: boolean;
+ readonly disclosed_at: string;
+};
+
+export type VariantExpansionVariant = {
+ readonly id: string;
+ readonly slug: string;
+ readonly project_root: string;
+ readonly run: string;
+ readonly route_state: string;
+ readonly manifest: string;
+ readonly brief: string;
+ readonly format_plan?: string;
+ readonly component_plan?: string;
+ readonly allowed_changes: string;
+ readonly allowed_change_paths: readonly string[];
+ readonly component_class: "svml-only" | "existing-component" | "composed-components" | "new-package";
+ readonly vocabulary_mode: "inherited" | "inspect";
+ readonly vocabulary_packages: readonly string[];
+ readonly vocabulary_digest?: string;
+ readonly package_injections: readonly { readonly package_id: string; readonly destination: string; readonly digest: string }[];
+ readonly status: "ready" | "dispatched" | "complete" | "failed" | "scope-expansion-required";
+ readonly error?: string;
+};
+
+export type VariantExpansionPackage = {
+ readonly id: string;
+ readonly purpose: string;
+ readonly staging_root: string;
+ readonly package_root?: string;
+ readonly route_state: string;
+ readonly digest?: string;
+ readonly status: "planned" | "active" | "ready" | "failed";
+ readonly error?: string;
+};
+
+export type VariantExpansionState = {
+ readonly version: 1;
+ readonly batch_id: string;
+ readonly project_root: string;
+ readonly output_root: string;
+ readonly run?: string;
+ readonly baseline_digest: string;
+ readonly parent_route?: "reconstruction" | "description";
+ readonly parent_state_digest?: string;
+ readonly revision_state_digest?: string;
+ readonly request?: string;
+ readonly count?: number;
+ readonly delivery_mode: "source" | "build";
+ readonly status: VariantExpansionStatus;
+ readonly current_step: number;
+ readonly completed_steps: readonly number[];
+ readonly in_progress: { readonly step: number; readonly started_at: string } | null;
+ readonly artifacts: Readonly<Record<string, string>>;
+ readonly workload_disclosure?: VariantWorkloadDisclosure;
+ readonly packages: readonly VariantExpansionPackage[];
+ readonly variants: readonly VariantExpansionVariant[];
+ readonly decisions: readonly string[];
+ readonly conflicts: readonly string[];
+ readonly next_action: string;
+ readonly last_command?: string;
+ readonly last_error?: string;
+ readonly created_at: string;
+ readonly updated_at: string;
+};
+
+export type VariantExpansionLocator = {
+ readonly version: 1;
+ readonly batch_id: string;
+ readonly project_root: string;
+ readonly output_root: string;
+ readonly state_path: string;
+ readonly baseline_digest: string;
+ readonly status: VariantExpansionStatus;
+ readonly updated_at: string;
+};
+
+export type StartVariantExpansionInput = {
+ readonly projectRoot: string;
+ readonly outputRoot: string;
+ readonly run?: string;
+ readonly request?: string;
+ readonly count?: number;
+ readonly deliveryMode?: "source" | "build";
+ readonly parentRoute?: "reconstruction" | "description";
+ readonly parentStateDigest?: string;
+ readonly revisionStateDigest?: string;
+};
+
+export type VariantExpansionCheckpointInput = {
+ readonly projectRoot: string;
+ readonly outputRoot?: string;
+ readonly batchId?: string;
+ readonly step: number | VariantExpansionStep;
+ readonly status?: "in_progress" | "complete" | "blocked";
+ readonly nextAction?: string;
+ readonly artifacts?: Readonly<Record<string, string>>;
+ readonly workloadDisclosure?: VariantWorkloadDisclosure;
+ readonly packages?: readonly VariantExpansionPackage[];
+ readonly variants?: readonly VariantExpansionVariant[];
+ readonly decision?: string;
+ readonly conflict?: string;
+ readonly resolveConflict?: string;
+ readonly command?: string;
+ readonly error?: string;
+};
+
+export function variantExpansionStatePath(outputRoot: string): string {
+ return join(resolve(outputRoot), ".hypit", "variant-expansion-state.json");
+}
+
+export function variantExpansionLocatorPath(projectRoot: string, batchId: string): string {
+ return join(resolve(projectRoot), ".hypit", "variant-expansions", `${batchId}.json`);
+}
+
+async function digestFile(path: string): Promise<string | undefined> {
+ const bytes = await readFile(path).catch(() => undefined);
+ return bytes === undefined ? undefined : `sha256:${createHash("sha256").update(bytes).digest("hex")}`;
+}
+
+async function atomicJson(path: string, value: unknown): Promise<void> {
+ await mkdir(dirname(path), { recursive: true });
+ const temporary = `${path}.tmp-${process.pid}-${Date.now()}`;
+ await writeFile(temporary, `${JSON.stringify(value, null, 2)}\n`, "utf8");
+ await rename(temporary, path);
+}
+
+function assertLocator(value: unknown, path: string): asserts value is VariantExpansionLocator {
+ if (value === null || typeof value !== "object") throw new Error(`variant locator at ${path} is not an object`);
+ const locator = value as Partial<VariantExpansionLocator>;
+ if (locator.version !== 1 || typeof locator.batch_id !== "string" || typeof locator.output_root !== "string" || typeof locator.state_path !== "string") {
+ throw new Error(`variant locator at ${path} is invalid`);
+ }
+}
+
+function assertState(value: unknown, path: string): asserts value is VariantExpansionState {
+ if (value === null || typeof value !== "object") throw new Error(`variant expansion state at ${path} is not an object`);
+ const state = value as Partial<VariantExpansionState>;
+ if (state.version !== 1) throw new Error(`variant expansion state at ${path} has unsupported version`);
+ if (typeof state.batch_id !== "string" || typeof state.project_root !== "string" || typeof state.output_root !== "string") throw new Error(`variant expansion state at ${path} has invalid roots`);
+ if (typeof state.baseline_digest !== "string" || (state.delivery_mode !== "source" && state.delivery_mode !== "build")) throw new Error(`variant expansion state at ${path} has invalid baseline metadata`);
+ if (state.status !== "active" && state.status !== "complete" && state.status !== "blocked") throw new Error(`variant expansion state at ${path} has invalid status`);
+ if (!Number.isSafeInteger(state.current_step) || state.current_step! < 1 || state.current_step! > VARIANT_EXPANSION_STEPS.length + 1) throw new Error(`variant expansion state at ${path} has invalid current_step`);
+ if (!Array.isArray(state.completed_steps) || state.completed_steps.some((step) => !Number.isSafeInteger(step) || step < 1 || step > VARIANT_EXPANSION_STEPS.length)) throw new Error(`variant expansion state at ${path} has invalid completed_steps`);
+ if (state.artifacts === null || typeof state.artifacts !== "object" || Array.isArray(state.artifacts)) throw new Error(`variant expansion state at ${path} has invalid artifacts`);
+ if (!Array.isArray(state.packages) || !Array.isArray(state.variants) || !Array.isArray(state.decisions) || !Array.isArray(state.conflicts)) throw new Error(`variant expansion state at ${path} has invalid collections`);
+ if (typeof state.next_action !== "string" || typeof state.created_at !== "string" || typeof state.updated_at !== "string") throw new Error(`variant expansion state at ${path} is missing recovery fields`);
+}
+
+async function readJsonPreserving(path: string, subject: string): Promise<unknown | undefined> {
+ const text = await readFile(path, "utf8").catch((error: unknown) => {
+ if ((error as NodeJS.ErrnoException).code === "ENOENT") return undefined;
+ throw error;
+ });
+ if (text === undefined) return undefined;
+ try { return JSON.parse(text); }
+ catch (error) { throw new Error(`${subject} at ${path} is invalid JSON; original file was preserved: ${error instanceof Error ? error.message : String(error)}`); }
+}
+
+async function writeLocator(state: VariantExpansionState): Promise<void> {
+ const locator: VariantExpansionLocator = {
+ version: 1, batch_id: state.batch_id, project_root: state.project_root, output_root: state.output_root,
+ state_path: variantExpansionStatePath(state.output_root), baseline_digest: state.baseline_digest,
+ status: state.status, updated_at: state.updated_at,
+ };
+ await atomicJson(variantExpansionLocatorPath(state.project_root, state.batch_id), locator);
+}
+
+async function writeState(state: VariantExpansionState): Promise<VariantExpansionState> {
+ await atomicJson(variantExpansionStatePath(state.output_root), state);
+ await writeLocator(state);
+ return state;
+}
+
+export async function discoverVariantExpansions(projectRoot: string): Promise<readonly VariantExpansionLocator[]> {
+ const root = join(resolve(projectRoot), ".hypit", "variant-expansions");
+ const files = await import("node:fs/promises").then(({ readdir }) => readdir(root)).catch((error: unknown) => {
+ if ((error as NodeJS.ErrnoException).code === "ENOENT") return [];
+ throw error;
+ });
+ const locators: VariantExpansionLocator[] = [];
+ for (const file of files.filter((name) => name.endsWith(".json")).sort()) {
+ const path = join(root, file);
+ const value = await readJsonPreserving(path, "variant locator");
+ assertLocator(value, path);
+ locators.push(value);
+ }
+ return locators.sort((left, right) => right.updated_at.localeCompare(left.updated_at));
+}
+
+async function selectedLocator(projectRoot: string, outputRoot?: string, batchId?: string): Promise<VariantExpansionLocator | undefined> {
+ if (outputRoot !== undefined) {
+ const path = variantExpansionStatePath(outputRoot);
+ const state = await readJsonPreserving(path, "variant expansion state");
+ if (state === undefined) return undefined;
+ assertState(state, path);
+ if (resolve(state.project_root) !== resolve(projectRoot)) throw new Error(`${path} belongs to ${state.project_root}, not ${resolve(projectRoot)}`);
+ return {
+ version: 1, batch_id: state.batch_id, project_root: state.project_root, output_root: state.output_root,
+ state_path: path, baseline_digest: state.baseline_digest, status: state.status, updated_at: state.updated_at,
+ };
+ }
+ const locators = await discoverVariantExpansions(projectRoot);
+ if (batchId !== undefined) return locators.find((item) => item.batch_id === batchId);
+ return locators.find((item) => item.status !== "complete") ?? locators[0];
+}
+
+export async function readVariantExpansionState(projectRoot: string, outputRoot?: string, batchId?: string): Promise<VariantExpansionState | undefined> {
+ const locator = await selectedLocator(projectRoot, outputRoot, batchId);
+ if (locator === undefined) return undefined;
+ const value = await readJsonPreserving(locator.state_path, "variant expansion state");
+ if (value === undefined) throw new Error(`variant locator ${variantExpansionLocatorPath(projectRoot, locator.batch_id)} points to missing state ${locator.state_path}`);
+ assertState(value, locator.state_path);
+ if (resolve(value.project_root) !== resolve(projectRoot)) throw new Error(`${locator.state_path} belongs to ${value.project_root}, not ${resolve(projectRoot)}`);
+ return value;
+}
+
+function nextUncompleted(completed: ReadonlySet<number>, deliveryMode: "source" | "build"): number {
+ const optional = deliveryMode === "source" ? new Set([15, 16, 17]) : new Set<number>();
+ for (let step = 1; step <= VARIANT_EXPANSION_STEPS.length; step += 1) {
+ if (!optional.has(step) && !completed.has(step)) return step;
+ }
+ return VARIANT_EXPANSION_STEPS.length + 1;
+}
+
+function stepNumber(step: number | VariantExpansionStep): number {
+ if (typeof step === "number") {
+ if (!Number.isSafeInteger(step) || step < 1 || step > VARIANT_EXPANSION_STEPS.length) throw new Error(`variant expansion step ${step} is outside the route`);
+ return step;
+ }
+ const index = VARIANT_EXPANSION_STEPS.indexOf(step);
+ if (index < 0) throw new Error(`unknown variant expansion step ${step}`);
+ return index + 1;
+}
+
+function requiredPriorSteps(step: number, deliveryMode: "source" | "build"): readonly number[] {
+ const optional = deliveryMode === "source" ? new Set([15, 16, 17]) : new Set<number>();
+ return Array.from({ length: step - 1 }, (_, index) => index + 1).filter((candidate) => !optional.has(candidate));
+}
+
+export async function startVariantExpansion(input: StartVariantExpansionInput): Promise<VariantExpansionState> {
+ const projectRoot = resolve(input.projectRoot);
+ const outputRoot = resolve(input.outputRoot);
+ const outputRelative = relative(projectRoot, outputRoot);
+ if (outputRelative.length === 0 || (!outputRelative.startsWith(`..${sep}`) && outputRelative !== ".." && !isAbsolute(outputRelative))) {
+ throw new Error("variant output root must not be inside the base project");
+ }
+ const batchId = basename(outputRoot);
+ if (batchId.length === 0 || batchId === "." || batchId === "..") throw new Error("variant output root must end in a batch id");
+ if (input.count !== undefined && (!Number.isSafeInteger(input.count) || input.count < 1)) throw new Error("variant count must be a positive integer");
+ const existing = await readVariantExpansionState(projectRoot, outputRoot);
+ if (existing !== undefined) return existing;
+ const locatorAtId = await readJsonPreserving(variantExpansionLocatorPath(projectRoot, batchId), "variant locator");
+ if (locatorAtId !== undefined) {
+ assertLocator(locatorAtId, variantExpansionLocatorPath(projectRoot, batchId));
+ if (resolve(locatorAtId.output_root) !== outputRoot) throw new Error(`batch id ${batchId} already points to ${locatorAtId.output_root}`);
+ }
+ const baseline = await snapshotProject(projectRoot);
+ const parentStatePath = join(projectRoot, ".hypit", "route-state.json");
+ const revisionStatePath = join(projectRoot, ".hypit", "revision-state.json");
+ const parentStateBytes = await readFile(parentStatePath).catch(() => undefined);
+ const parentStateDigest = input.parentStateDigest
+ ?? (parentStateBytes === undefined ? undefined : `sha256:${createHash("sha256").update(parentStateBytes).digest("hex")}`);
+ const revisionStateDigest = input.revisionStateDigest ?? await digestFile(revisionStatePath);
+ let parentRoute = input.parentRoute;
+ if (parentRoute === undefined && parentStateBytes !== undefined) {
+ try {
+ const parsed = JSON.parse(parentStateBytes.toString()) as { route?: unknown };
+ if (parsed.route === "reconstruction" || parsed.route === "description") parentRoute = parsed.route;
+ } catch { /* corrupt parent state is reported by reconcile rather than hidden here */ }
+ }
+ const now = new Date().toISOString();
+ return writeState({
+ version: 1, batch_id: batchId, project_root: projectRoot, output_root: outputRoot,
+ ...(input.run === undefined ? {} : { run: resolve(projectRoot, input.run) }),
+ baseline_digest: baseline.digest,
+ ...(parentRoute === undefined ? {} : { parent_route: parentRoute }),
+ ...(parentStateDigest === undefined ? {} : { parent_state_digest: parentStateDigest }),
+ ...(revisionStateDigest === undefined ? {} : { revision_state_digest: revisionStateDigest }),
+ ...(input.request === undefined ? {} : { request: input.request }),
+ ...(input.count === undefined ? {} : { count: input.count }),
+ delivery_mode: input.deliveryMode ?? "source", status: "active", current_step: 1, completed_steps: [],
+ in_progress: { step: 1, started_at: now }, artifacts: {}, packages: [], variants: [], decisions: [], conflicts: [],
+ next_action: `complete ${VARIANT_EXPANSION_STEPS[0]}`, created_at: now, updated_at: now,
+ });
+}
+
+export async function checkpointVariantExpansion(input: VariantExpansionCheckpointInput): Promise<VariantExpansionState> {
+ const existing = await readVariantExpansionState(input.projectRoot, input.outputRoot, input.batchId);
+ if (existing === undefined) throw new Error("no variant expansion state; run variant_state --action start first");
+ const step = stepNumber(input.step);
+ const status = input.status ?? "complete";
+ const completed = new Set(existing.completed_steps);
+ if (status === "complete" && typeof input.step === "string") {
+ const missing = requiredPriorSteps(step, existing.delivery_mode).filter((candidate) => !completed.has(candidate));
+ if (missing.length > 0) throw new Error(`cannot complete ${VARIANT_EXPANSION_STEPS[step - 1]} before ${missing.map((candidate) => VARIANT_EXPANSION_STEPS[candidate - 1]).join(", ")}`);
+ }
+ if (status === "complete" && step === 8 && input.workloadDisclosure === undefined && existing.workload_disclosure === undefined) {
+ throw new Error("workload-disclosed requires a persisted workload_disclosure");
+ }
+ if (input.workloadDisclosure !== undefined && existing.count !== undefined
+ && input.workloadDisclosure.svml_only + input.workloadDisclosure.medium !== existing.count) {
+ throw new Error(`workload disclosure accounts for ${input.workloadDisclosure.svml_only + input.workloadDisclosure.medium} variants, expected ${existing.count}`);
+ }
+ if (status === "complete") completed.add(step);
+ const next = status === "complete" ? nextUncompleted(completed, existing.delivery_mode) : step;
+ const now = new Date().toISOString();
+ const artifacts = input.artifacts === undefined ? existing.artifacts : {
+ ...existing.artifacts,
+ ...Object.fromEntries(Object.entries(input.artifacts).map(([key, value]) => [key, resolve(existing.project_root, value)])),
+ };
+ const decisions = input.decision === undefined || input.decision.trim().length === 0 || existing.decisions.includes(input.decision.trim())
+ ? existing.decisions : [...existing.decisions, input.decision.trim()];
+ const resolved = input.resolveConflict?.trim();
+ const retainedConflicts = resolved === undefined || resolved.length === 0
+ ? existing.conflicts : existing.conflicts.filter((message) => message !== resolved);
+ const conflicts = input.conflict === undefined || input.conflict.trim().length === 0 || retainedConflicts.includes(input.conflict.trim())
+ ? retainedConflicts : [...retainedConflicts, input.conflict.trim()];
+ if (input.packages !== undefined) {
+ const packageIds = input.packages.map((pack) => pack.id);
+ if (packageIds.some((id) => id.trim().length === 0) || new Set(packageIds).size !== packageIds.length) {
+ throw new Error("variant packages require distinct non-empty ids");
+ }
+ }
+ const packages: VariantExpansionPackage[] | undefined = input.packages === undefined ? undefined : await Promise.all(input.packages.map(async (pack) => {
+ if (pack.status !== "ready") return pack;
+ const route = await readRouteState(pack.staging_root);
+ if (route?.route !== "variant-package") throw new Error(`package ${pack.id} has no variant-package route at ${pack.staging_root}`);
+ const stagingRoot = resolve(pack.staging_root);
+ const packageRoot = resolve(pack.package_root ?? pack.staging_root);
+ const packageRelative = relative(stagingRoot, packageRoot);
+ if (packageRelative === ".." || packageRelative.startsWith(`..${sep}`) || isAbsolute(packageRelative)) {
+ throw new Error(`package ${pack.id} package_root must stay inside its staging project`);
+ }
+ const digest = (await snapshotProject(packageRoot, { preserveTopLevelOutputs: true })).digest;
+ const evidence = join(resolve(pack.staging_root), ".hypit", "package-digest.json");
+ await atomicJson(evidence, { package_id: pack.id, staging_root: stagingRoot, package_root: packageRoot, digest, frozen_at: now });
+ await checkpointRouteState({ projectRoot: pack.staging_root, route: "variant-package", step: "package-ready", status: "complete", artifacts: { package_digest: evidence } });
+ return { ...pack, staging_root: stagingRoot, package_root: packageRoot, route_state: routeStatePath(pack.staging_root), digest };
+ }));
+ return writeState({
+ ...existing,
+ status: status === "blocked" ? "blocked" : next > VARIANT_EXPANSION_STEPS.length ? "complete" : "active",
+ current_step: next, completed_steps: [...completed].sort((a, b) => a - b),
+ in_progress: next > VARIANT_EXPANSION_STEPS.length ? null : { step: next, started_at: now },
+ artifacts, decisions, conflicts,
+ ...(input.workloadDisclosure === undefined ? {} : { workload_disclosure: input.workloadDisclosure }),
+ ...(packages === undefined ? {} : { packages }),
+ ...(input.variants === undefined ? {} : { variants: input.variants }),
+ next_action: input.nextAction?.trim() || (next > VARIANT_EXPANSION_STEPS.length ? "variant expansion complete" : `complete ${VARIANT_EXPANSION_STEPS[next - 1]}`),
+ ...(input.command === undefined ? {} : { last_command: input.command }),
+ ...(input.error === undefined ? {} : { last_error: input.error }),
+ updated_at: now,
+ });
+}
+
+async function exists(path: string | undefined): Promise<boolean> {
+ return path !== undefined && await stat(path).then(() => true, () => false);
+}
+
+async function validJson(path: string | undefined, predicate: (value: unknown) => boolean = () => true): Promise<boolean> {
+ if (!(await exists(path)) || path === undefined) return false;
+ try { return predicate(JSON.parse(await readFile(path, "utf8"))); }
+ catch { return false; }
+}
+
+export async function reconcileVariantExpansion(projectRoot: string, outputRoot?: string, batchId?: string): Promise<VariantExpansionState | undefined> {
+ const existing = await readVariantExpansionState(projectRoot, outputRoot, batchId);
+ if (existing === undefined) return undefined;
+ const completed = new Set(existing.completed_steps);
+ const conflicts = existing.conflicts.filter((message) => !message.startsWith("[reconcile] "));
+ const addConflict = (message: string): void => { if (!conflicts.includes(message)) conflicts.push(message); };
+ const baseline = await snapshotProject(existing.project_root);
+ if (baseline.digest !== existing.baseline_digest) addConflict(`[reconcile] base project digest changed: expected ${existing.baseline_digest}, found ${baseline.digest}`);
+ const parentDigest = await digestFile(join(existing.project_root, ".hypit", "route-state.json"));
+ if (existing.parent_state_digest !== undefined && parentDigest !== existing.parent_state_digest) addConflict(`[reconcile] parent route state digest changed: expected ${existing.parent_state_digest}, found ${parentDigest ?? "missing"}`);
+ const revisionDigest = await digestFile(join(existing.project_root, ".hypit", "revision-state.json"));
+ if (existing.revision_state_digest !== undefined && revisionDigest !== existing.revision_state_digest) addConflict(`[reconcile] revision state digest changed: expected ${existing.revision_state_digest}, found ${revisionDigest ?? "missing"}`);
+
+ const packages: VariantExpansionPackage[] = [];
+ for (const pack of existing.packages) {
+ const route = await reconcileRouteState(pack.staging_root);
+ const currentDigest = await snapshotProject(pack.package_root ?? pack.staging_root, { preserveTopLevelOutputs: true }).then((snapshot) => snapshot.digest, () => undefined);
+ if (pack.digest !== undefined && currentDigest !== pack.digest) {
+ addConflict(`[reconcile] package ${pack.id} digest changed: expected ${pack.digest}, found ${currentDigest ?? "missing"}`);
+ }
+ const { error: _oldError, ...packWithoutError } = pack;
+ packages.push({ ...packWithoutError, status: route?.status === "complete" && pack.digest !== undefined && currentDigest === pack.digest ? "ready" : route?.status === "blocked" ? "failed" : "active",
+ ...(route === undefined ? { error: "route-state-missing" } : {}) });
+ }
+
+ const variants: VariantExpansionVariant[] = [];
+ for (const variant of existing.variants) {
+ const route = await reconcileRouteState(variant.project_root);
+ const diff = await inspectVariantDiff(variant.project_root);
+ const outside = Array.isArray(diff.outside_allowed_changes) ? diff.outside_allowed_changes : [];
+ if (outside.length > 0) addConflict(`[reconcile] variant ${variant.id} changed outside allowed_changes: ${outside.join(", ")}`);
+ if (diff.passed !== true && outside.length === 0) {
+ const errors = Array.isArray(diff.errors) ? diff.errors.map(String).join(", ") : "variant baseline evidence failed";
+ addConflict(`[reconcile] variant ${variant.id} scope evidence is invalid: ${errors}`);
+ }
+ const check = await validJson(join(variant.project_root, ".hypit", "variant-check.json"), (value) => (value as { passed?: unknown })?.passed === true);
+ const { error: _oldError, ...variantWithoutError } = variant;
+ variants.push({
+ ...variantWithoutError,
+ status: outside.length > 0 ? "scope-expansion-required"
+ : route?.status === "complete" && check ? "complete"
+ : route?.status === "blocked" ? "failed" : route === undefined ? "failed" : variant.status === "ready" ? "ready" : "dispatched",
+ ...(route === undefined ? { error: "route-state-missing" } : {}),
+ });
+ }
+
+ const artifact = (key: string): string | undefined => existing.artifacts[key];
+ const predicates = new Map<number, () => Promise<boolean>>([
+ [1, async () => await validJson(artifact("baseline_check"), (value) => (value as { passed?: unknown })?.passed === true)
+ || (await readRouteState(existing.project_root))?.status === "complete"],
+ [2, async () => await validJson(artifact("examples"))],
+ [3, async () => await validJson(artifact("format_plan"))],
+ [4, async () => await validJson(artifact("slate"), (value) => Array.isArray((value as { variants?: unknown })?.variants))],
+ [5, async () => await validJson(artifact("vocabulary"), (value) => Array.isArray((value as { packages?: unknown })?.packages) || Array.isArray((value as { surfaces?: unknown })?.surfaces))],
+ [6, async () => await validJson(artifact("component_plan"))],
+ [7, async () => await validJson(artifact("package_gaps"))],
+ [9, async () => {
+ const slatePath = artifact("slate");
+ if (slatePath === undefined) return false;
+ let slate: { variants?: readonly { component_class?: unknown; inject_packages?: readonly { package_id?: unknown }[] }[] };
+ try { slate = JSON.parse(await readFile(slatePath, "utf8")) as typeof slate; } catch { return false; }
+ const variantsInSlate = slate.variants ?? [];
+ if (variantsInSlate.some((variant) => variant.component_class === "new-package" && (variant.inject_packages?.length ?? 0) === 0)) return false;
+ const required = new Set(variantsInSlate.flatMap((variant) => variant.component_class === "new-package"
+ ? (variant.inject_packages ?? []).flatMap((injection) => typeof injection.package_id === "string" ? [injection.package_id] : [])
+ : []));
+ return packages.every((pack) => pack.status === "ready")
+ && [...required].every((id) => packages.some((pack) => pack.id === id && pack.status === "ready" && pack.digest !== undefined));
+ }],
+ [10, async () => await validJson(artifact("slate"), (value) => {
+ const variants = (value as { variants?: unknown[] })?.variants;
+ return Array.isArray(variants) && (existing.count === undefined || variants.length === existing.count);
+ })],
+ [11, async () => variants.length > 0 && (await Promise.all(variants.map(async (variant) => await exists(variant.manifest) && await exists(variant.project_root)))).every(Boolean)],
+ [12, async () => variants.length > 0 && (await Promise.all(variants.map(async (variant) => await exists(variant.route_state)))).every(Boolean)],
+ [13, async () => variants.length > 0 && variants.every((variant) => variant.status === "complete")],
+ [14, async () => await validJson(artifact("aggregate_check"), (value) => (value as { passed?: unknown })?.passed === true)],
+ [17, async () => completed.has(16) && await validJson(artifact("build"), (value) => (value as { passed?: unknown; status?: unknown })?.passed === true || (value as { status?: unknown })?.status === "complete")],
+ [18, async () => completed.has(14) && (existing.delivery_mode === "source" || completed.has(17))],
+ ]);
+ const machineSteps = new Set([1, 9, 11, 13, 14, 17, 18]);
+ for (const [step, predicate] of predicates) {
+ const satisfied = await predicate();
+ if (completed.has(step) && !satisfied) {
+ completed.delete(step);
+ addConflict(`[reconcile] step ${step} (${VARIANT_EXPANSION_STEPS[step - 1]}) was marked complete but its evidence is missing or failed`);
+ } else if (machineSteps.has(step) && satisfied) completed.add(step);
+ }
+ if (existing.workload_disclosure === undefined && completed.has(8)) {
+ completed.delete(8);
+ addConflict("[reconcile] workload-disclosed was marked complete without a persisted disclosure");
+ }
+ if (conflicts.length > 0) {
+ completed.delete(18);
+ }
+ const next = nextUncompleted(completed, existing.delivery_mode);
+ const now = new Date().toISOString();
+ return writeState({
+ ...existing,
+ status: next > VARIANT_EXPANSION_STEPS.length ? "complete" : existing.status === "blocked" ? "blocked" : "active",
+ current_step: next, completed_steps: [...completed].sort((a, b) => a - b),
+ in_progress: next > VARIANT_EXPANSION_STEPS.length ? null : { step: next, started_at: now },
+ conflicts, packages, variants,
+ next_action: next > VARIANT_EXPANSION_STEPS.length ? "variant expansion complete" : `complete ${VARIANT_EXPANSION_STEPS[next - 1]}`,
+ updated_at: now,
+ });
+}