Skip to content

Write HTML. Render video. Built for agents.

docs: show agents where Studio edits go and how tests build, and let docs skip captures (#4936)

Miguel Ángel5632356c093e
diff --git a/AGENTS.md b/AGENTS.md
index 9f2c424573..74a602b8c4 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -38,6 +38,22 @@ bun run build # Build all packages
bun run test # Run all tests
```
+Packages load each other through their `exports`. Under Node, core, parsers,
+lint and studio-server resolve to `dist/`, and player always loads its built
+bundle. Run `bun run build` once in a fresh worktree, before any package's
+tests or typecheck. After editing a package that another package imports, run
+that package's `build` again before the dependent package's tests. Core's
+`src/generated/` is git-ignored build output: if you see
+`Cannot find module '../generated/runtime-inline'`, run
+`bun run build:hyperframes-runtime`.
+
+One file: `bun run --cwd packages/<pkg> test <path>`. Each package's `test`
+script picks its runner (vitest in most, `bun test` in aws-lambda and
+gcp-cloud-run). Producer is the exception: a file that imports `bun:test` runs
+with `bun test <file>`, any other with `bunx vitest run <file>`. Root `scripts/*.test.mjs` run through `bun run test:scripts`,
+which is an explicit list in `package.json`. CI's test-reachability check fails
+any test file that no runner reaches.
+
### Linting & Formatting
Uses **oxlint** and **oxfmt** (not eslint, not prettier, not biome).
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 1a2ecfb251..326e4664f0 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -97,6 +97,8 @@ Reviewers judge rules 2 to 5 and 8. The `Comments` check (`scripts/check-comment
Only comment blocks holding a line your PR added can fail the citation and block rules, and the TODO and URL rules grade only the lines your PR added. Broken citations elsewhere in a file you touched are printed as warnings, and fixing one while you are there is welcome. To grade files by hand, pass their paths: `node scripts/check-comment-citations.mjs path/to/file.ts`.
+Run both checks before pushing: `node scripts/check-comment-citations.mjs` and `node scripts/comment-ratchet.mjs` grade your branch against its merge-base with `origin/main` (set `COMMENT_CHECK_BASE` to compare with another base). The pre-commit hooks run neither.
+
## Adding Registry Items (Blocks & Components)
The registry at `registry/` contains reusable items installable via `hyperframes add <name>`. Each item lives in its own directory under `registry/blocks/` or `registry/components/`.
@@ -198,14 +200,12 @@ All packages use **fixed versioning** — every release bumps all packages to th
### Stable releases
```bash
-bun run release:prepare 0.2.0 # drafts changelog if needed, then creates the release commit/tag after review
-git push origin main # push the release commit
-git push origin v0.2.0 # push the tag → triggers the publish workflow
+git switch -c release/v0.2.0 origin/main
+bun run release:prepare 0.2.0 # drafts release notes and stops for review; rerun after editing them
+git push origin release/v0.2.0 # push the branch, not the local tag
```
-> Push the **specific tag**, not `git push --tags` — the latter pushes every local tag and the whole push is rejected if any one already exists on the remote.
-
-The `release:prepare` script drafts missing release notes on the first run and stops for manual review. After the generated TODO summary is rewritten, rerun the same command; it delegates to `set-version`, which creates a `chore: release v<version>` commit and a `v<version>` git tag. Pushing the tag triggers CI to publish all packages to npm and create a GitHub Release.
+Open a PR from `release/v0.2.0` to `main`. When it merges, the publish workflow tags the merge commit, publishes every package to npm and creates the GitHub Release. Pushing a stable tag by hand publishes nothing: the workflow only accepts pre-release tags. The full procedure, including recovery from a failed publish, is in [docs/contributing/changelog-process.mdx](docs/contributing/changelog-process.mdx).
`set-version` also refuses to tag if a **higher** semver tag already exists (a stale higher tag would hijack tag-sorting installers like `npx skills`). Delete the stray tag (`git tag -d <tag> && git push origin :refs/tags/<tag>`) or, only if intentional, pass `--skip-monotonicity-check`.
diff --git a/packages/producer/README.md b/packages/producer/README.md
index a73baa223c..515ae7692f 100644
--- a/packages/producer/README.md
+++ b/packages/producer/README.md
@@ -184,6 +184,26 @@ The activity functions plus their result types are also re-exported from `@hyper
4. **Mix** — extracts `<audio>` elements and mixes them into the final video. For `png-sequence`, audio is written as an `audio.aac` sidecar.
5. **Finalize** — applies faststart for streaming-friendly MP4 (no-op for WebM, MOV, and `png-sequence`). For `hls`, stream-copies the encode into playlists and MPEG-TS segments instead.
+### Where each step lives
+
+`executeRenderJob` in `src/services/renderOrchestrator.ts` runs the stages in
+`src/services/render/stages/`, in this order:
+
+- `compileStage.ts`, `probeStage.ts`, `extractVideosStage.ts`, `audioStage.ts`
+- `src/services/render/captureCost.ts` sizes the worker count
+ (`runCaptureCalibration`, then `resolveRenderWorkerCount`)
+- `createCapturePlan` in `src/services/render/capturePlan.ts` picks the capture
+ kind: streaming, disk, segmented or HDR layered
+- capture: `captureStreamingStage.ts`, `captureSegmentedStage.ts` and
+ `captureHdrStage.ts` encode as they capture; disk capture (`captureStage.ts`)
+ encodes afterwards in `encodeStage.ts`
+- `assembleStage.ts`, except for PNG sequences and GIF, which are written
+ directly
+
+The job's `producerConfig` is used when present; otherwise `resolveConfig` in
+`packages/engine/src/config.ts` builds it. Capture, encoding and browser
+primitives are engine services (see `packages/engine/README.md`).
+
## Documentation
Full documentation: [hyperframes.heygen.com/packages/producer](https://hyperframes.heygen.com/packages/producer)
diff --git a/packages/studio/AGENTS.md b/packages/studio/AGENTS.md
index 9fa91cc02d..35ee67faab 100644
--- a/packages/studio/AGENTS.md
+++ b/packages/studio/AGENTS.md
@@ -21,6 +21,50 @@ Two consequences you will meet immediately:
chrome disagrees with the pixels underneath it, the bug is almost always in
the measurement, in `components/editor/domEditOverlayGeometry.ts`.
+## Where a canvas edit goes
+
+Paths are under `src/`; bare file names are in `src/components/editor/`. Each
+line names the owner; read it there.
+
+1. **Gesture.** `DomEditSelectionChrome.tsx` starts it, `domEditOverlayStartGesture.ts`
+ arms it, `useDomEditOverlayGestures.ts` (`onPointerUp`) hands it off. Nudge,
+ crop and inline text enter through `useDomEditNudge.ts`, `DomEditCropHandles.tsx`
+ and `useInlineTextEditing.tsx`. `hooks/useDomEditSession.ts` wires the handlers.
+2. **GSAP or plain.** The predicates `gsapWritesPosition`, `gsapWritesRotation`
+ and `gsapWritesBox` live in `hooks/gsapRuntimeKeyframes.ts`. The router is
+ `hooks/useGsapAwareEditing.ts`. Move and a plain rotation keep the route
+ chosen at press. A GSAP rotation checks ownership again at commit, and resize
+ decides at commit through `hooks/gsapResizeIntercept.ts`.
+3. **Writers.** Plain edits go through `hooks/elementOffsetStager.ts`,
+ `hooks/useDomGeometryCommits.ts` and `hooks/plainRotation.ts`, then
+ `hooks/useDomEditPositionPatchCommit.ts`. GSAP edits: `hooks/gsapRuntimeBridge.ts`
+ (the drag and rotate intercepts), `hooks/gsapDragCommit.ts`,
+ `hooks/gsapDragPositionCommit.ts` (keyframe at the playhead) and
+ `hooks/gsapWholePropertyOffsetCommit.ts`. Styles, text, attributes and groups:
+ `hooks/domStyleCommit.ts`, `hooks/useDomEditTextCommits.ts`,
+ `hooks/useDomEditAttributeCommits.ts`, `hooks/useGroupCommits.ts`.
+4. **Save.** There are three paths. A single DOM edit goes through
+ `hooks/useDomEditPersist.ts`. It first offers the edit to the SDK path
+ (`utils/sdkCutover.ts`, wired in `useDomEditSession.ts`), which writes the
+ file itself. Otherwise it posts studio-server's `file-mutations/patch-element`.
+ Atomic DOM batches go through `hooks/useDomEditCommits.ts` to
+ `patch-element-batches`. Script edits go through `hooks/useGsapScriptCommits.ts`,
+ to `gsap-mutations` or to the SDK path. The routes are in
+ `packages/studio-server/src/routes/files.ts`. The own-write token is in
+ `utils/studioFileVersion.ts`, and `hooks/useExternalFileChangeCoordinator.ts`
+ drops the echo.
+5. **Reload.** Each save path decides its own reload. After a `patch-element`
+ save, `useDomEditPersist.ts` reloads unless `skipRefresh` is set. For script
+ edits, `useGsapScriptCommits.ts` (`applyPreviewSync`) picks one of three: an
+ instant patch, `utils/gsapSoftReload.ts`, or a reload.
+ A reload bumps `refreshKey`, and `refreshPlayer` in
+ `player/hooks/useTimelinePlayer.ts` either swaps the scene or runs
+ `player/hooks/useShadowPreviewReload.ts`, which waits while a gesture or a
+ save is in flight.
+6. **Undo.** `hooks/usePersistentEditHistory.ts` records the edit.
+ `hooks/useEditHistoryActions.ts` steps through history.
+ `utils/gsapUndoRestore.ts` repaints the preview.
+
## Driving Studio for verification
A pixel-precise click inside the preview is not something an automated driver
@@ -74,6 +118,26 @@ properties (`rotate`, `scale`, `translate`) into computed style, and it has no
depends on real layout or real computed style, prove it in a browser and keep
the unit test on the pure function underneath.
+## The edit accuracy bench
+
+`tests/e2e/edit-accuracy/` performs real gestures in the built CLI's Studio and
+checks the saved file, the reloaded preview, undo and a producer frame. It
+needs the built CLI (`packages/cli/dist/cli.js`, from `bun run build`) and
+`chrome-headless-shell` (`npx hyperframes browser ensure`):
+
+```bash
+bun run --cwd packages/studio test:edit-accuracy -- --grid pr --filter '^resize-' --jobs 1
+```
+
+- Case ids come from `grid.mjs`; `--filter` is a regex on them. `--grid pr` is
+ the smaller slice, `full` is what CI runs.
+- Each run writes its results, a table and a candidate baseline to
+ `tests/e2e/evidence/edit-accuracy/<run>/` (git-ignored; `--out` moves it); a
+ failing case also gets its screens and saved files under `cases/<id>/`.
+- CI runs the full grid in shards and `ratchet.mjs` gates the result against the
+ base branch's `baseline.json`; its header states the rules. When cases newly
+ pass, commit the `baseline.json` from the `edit-accuracy-gate` artifact.
+
## Gates that will fail your PR
- **600 lines per file.** CI checks only non-test files your PR changed. A file
@@ -82,6 +146,10 @@ the unit test on the pure function underneath.
function, duplication, unused exports. Adding branches to an already-complex
function trips it; extract rather than nest.
- **oxlint and oxfmt**, not eslint or prettier.
+- **Before and After captures.** A PR that changes code under `packages/studio`
+ or `packages/player` needs `## Before` and `## After` sections in its
+ description, each with an image or video. The exemptions (a small change with
+ no visible effect, Markdown) are defined in `scripts/check-pr-captures.mjs`.
## Traps worth knowing
diff --git a/scripts/check-pr-captures.mjs b/scripts/check-pr-captures.mjs
index 452debd40b..9e6ab31420 100644
--- a/scripts/check-pr-captures.mjs
+++ b/scripts/check-pr-captures.mjs
@@ -1,5 +1,6 @@
#!/usr/bin/env node
// Fail a PR touching packages/studio or packages/player unless its body has Before and After sections with media.
+// Markdown under them is docs, not behaviour, so it is not watched.
// usage: node scripts/check-pr-captures.mjs --base origin/main --head <sha>; the body arrives in the env (see main).
import { execFileSync } from "node:child_process";
@@ -131,8 +132,9 @@ function parseRecord(record) {
return { path: rest.join("\t"), lines };
}
+const DOC_FILE = /\.mdx?$/i;
const isWatched = (path) =>
- path !== "" && WATCHED_PREFIXES.some((prefix) => path.startsWith(prefix));
+ path !== "" && WATCHED_PREFIXES.some((prefix) => path.startsWith(prefix)) && !DOC_FILE.test(path);
/** Parse `git diff --numstat -z --no-renames`, keeping watched paths. A binary file counts as a full budget. */
export const parseNumstat = (numstat) =>
diff --git a/scripts/check-pr-captures.test.mjs b/scripts/check-pr-captures.test.mjs
index 37b564e6a4..610c898c89 100644
--- a/scripts/check-pr-captures.test.mjs
+++ b/scripts/check-pr-captures.test.mjs
@@ -108,6 +108,14 @@ test("numstat keeps only watched paths and counts added plus deleted", () => {
]);
});
+test("markdown under a watched package needs no capture", () => {
+ const files = parseNumstat(
+ "60\t4\tpackages/studio/AGENTS.md\0" + "8\t0\tpackages/player/README.md\0",
+ );
+ assert.deepEqual(files, []);
+ assert.equal(evaluate({ body: "", files }).ok, true);
+});
+
test("a binary file spends the whole no-visible-change budget", () => {
const [file] = parseNumstat("-\t-\tpackages/studio/public/logo.png\0");
assert.equal(file.lines, 20);