Skip to content

Code-first video & image, rendered natively on WebGPU.

gemini.md skills

okanaslcf255e1c4c73
diff --git a/CLAUDE.md b/CLAUDE.md
new file mode 100644
index 00000000..2bc01a97
--- /dev/null
+++ b/CLAUDE.md
@@ -0,0 +1,76 @@
+# Framefields
+
+Code-first video engine on native WebGPU (Dawn) in Node, no browser. pnpm + turbo monorepo, Node >=22, ESM, Biome (tabs, double quotes), tsdown builds, vitest tests. Rendering needs a GPU. Our aim is to improve the engine harness: no cheating or workarounds.
+
+## Working rules
+- Be brave; we can change anything. Aim for engineering elegance, not just goal completion.
+- Debug to the root cause. If a workaround is the only option, stop and ask.
+- No `any`. Named exports unless a default is required. Top-of-file imports (no `await import` unless required). Pass the cyclomatic complexity lint.
+- pnpm only; never hand-edit `pnpm-lock.yaml`.
+- Never run bare `tsc` (it emits into source). Use the package script (`tsdown`, or `tsc --noEmit -p .`).
+- Render scripts end with `process.exit`.
+- Plain console output, no colors. Don't use the user's name. Final answers concise.
+- Keep this file current when structure or dev patterns change but do not bloat it.
+
+## Repo map
+- `packages/framefields`: public SDK. `Composition`, `Layer.*`, `LayerAnimation`, `Signal`, effects (`src/effects/classes.ts`), `FontManager`, `renderFrameGrid`. `src/index.ts` is the export surface. CLI in `src/cli/` (`init`, `add`, `list`, `fonts add|list`); `init` never prompts or overwrites, and its file templates are strings in `templates.ts` (a test parses them).
+- `packages/components`: the component registry `framefields add` copies into user projects (shadcn-style). `defineComponent`/`bind` contract in `src/core/contract.ts`, shared helpers in `src/core/` (keys, color, layout, surface, theme). Components import only `framefields` and `../core`. Registry items declare font needs (e.g. emoji); CLI fonts are pinned by URL, size and SHA-256 in `cli/fonts.ts`. Prop docs in `components.json` are generated (drift test). Tree-shake size budgets are tested.
+- `packages/compositions`: Yoga layout (`utils/layout/resolve-layout.ts`), program schema (`program/schema.ts`), lint (`program/lint.ts`), animation/evaluator.
+- `packages/webgpu-renderers`: WGSL shaders, 2D/3D pipelines, Slug text (`src/slug/`), audio DSP.
+- `packages/renderer`: headless Node renderer (Dawn, WebCodecs/mediabunny).
+- `packages/core`: shared types, `Effect` base, signals, `emojiPresentation` (the one UTS #51 rule used by layout, preload and lint).
+- Others: `renderers`, `media`, `node-sdk`, `server-utils`, `client-utils`, `tensor-webgpu`, `vision`, `tsconfig`.
+- `nodes/node-*`: one VFX/audio/layout node each (`src/shared/`, `src/renderers/`, `src/index.ts`). Only `node-compositor` has a `SKILL.md`; update it when Compositor behavior changes (guarded by `skillmd-check.test.ts`).
+- `apps/renderer-service`: HTTP render service (Docker, RunPod).
+- `examples/NN_*`: reference films. Study `19_framefields_film` (structure), `22_framefields_launch` (lyric sync), `28_physics_dancer` (physics, vision, GPU particles) before writing one.
+- `plugins/framefields/skills/framefields{,-compose,-effects,-render}`: agent skills, symlinked from `.agents/skills` and `.claude/skills`.
+- Package `exports` have a `development` condition pointing at `src/`; otherwise `dist/`, so `pnpm build` after changing a dependency.
+- Docs site is a separate repo: `/Users/okanaslankan/framefields-dev/docs` (Astro Starlight). `docs/lab` pins the published `framefields` version and builds the component demos; deploy with `pnpm run deploy`.
+
+## Commands
+- Build: `pnpm build`, or `pnpm --filter <name> build`.
+- Test: `pnpm --filter <name> test` (`packages/renderer`: `test:vitest`). Root `pnpm test` is slow.
+- Lint: `pnpm exec biome check --write <files>` (lefthook runs it on commit).
+- Example: `pnpm --filter @framefields/example-19-framefields-film assets|render|test|typecheck`.
+- Component gallery: `pnpm --filter framefields gallery`.
+- Skills: after engine API changes update the owning skill, then `pnpm run check:plugins`. `effects-catalog.md` is generated by `pnpm run sync:effects-catalog`; never hand-edit.
+- Releases: changesets (`.changeset/`).
+
+## Local-only files
+Gitignored, never commit: `assets/fonts`, `*.mp4/mp3/wav/png/jpg`, `*.onnx`, `scratch*/`, `examples/output/`, `.hermes/`, `.env*` (never print secrets). Put probes and logs under `scratch/` or `output/`.
+
+## Engine gotchas (fail silently)
+- Premultiplied-alpha blend uses `srcFactor: "one"`, never `"src-alpha"` (murky grey fades).
+- WGSL: every declared storage binding must be used, or Tint drops it from the layout.
+- Clamp scissor rects to non-negative integers within the surface.
+- Signals are class instances; never spread them (loses `.get(ctx)`). A `NaN` uniform blacks out the frame.
+- `CompositorProgramSchema` is `.strict()`: new node/program fields go in `program/schema.ts`, or `comp.toSpec()` fails.
+- Keyframe frames and effect frame options must be integers (`Math.round(...)`).
+- Master soundtrack layers to about -14 dBFS RMS, summed peak near 0.9.
+- Procedural SFX (`renderSfx`, `mixSfxInto`, `softLimit`): `atBar` is where the effect starts; a whoosh peaks 0.42s in, so start ~0.21 bars before the cut.
+- `withAnimation` keeps `animation` an enumerable data property so spreads keep tracks.
+- Keep `isAbsolutePositioned` symmetric between `buildYogaNode` and `readLayout`.
+- Compositor specs: audio is `config.layout` items with `kind: "media"`, `inputHandleId` (live handle ID, not label) and `volume`. Fonts go in top-level `"fonts"` with relative paths.
+- Fonts: every `Layer.text` font must be registered with `FontManager` and declared in `fonts`; no system fonts. Emoji need a color emoji font (`FontManager.registerEmojiFont`, or `framefields fonts add emoji`). Missing glyphs fall back per grapheme across registered families, except path text, which is single-font.
+- Path text (`pathOptions`) coordinates are local to the node's box; size the node to contain the path.
+- Shape paths support SVG arcs; a circle is two arcs and draws on with `trimEnd`.
+- Circular window: plate plus `Layer.shape("ellipse", { blendMode: "mask-in" })` in a box with `background: "transparent"`.
+- No per-effect helpers like `blurTrackedRegion`; extend `comp.section` / `Layer.section`.
+
+## Authoring API traps
+- `LayerAnimation` has no `moveX/moveY/scale/rotate/trimPath`; use `fromTo(prop, from, to, { start, end, ease })`. `fadeIn`/`fadeOut` are positional: `.fadeIn(0, 20, "power2.out")`.
+- Borders are `borderColor` + `borderWidth`; CSS `border` is ignored.
+- Setting only `x` or `y` makes a node absolute; use `position: "relative"` to nudge. A centered flex needs an explicit `width` (or `"fill"`).
+- Keyframes are relative to the node's own window (never add the parent's `startFrame`). Animating `y` overrides layout `y`: `fromTo("y", y + 40, y)`.
+- Eases: `none power1 power2 power3 sine circ expo back elastic bounce spring cubic hold` (no `power4`).
+- Don't nest an opaque container inside a `tilt3D`/`rotateX`/`perspective` container; make one card.
+- Every audio layer needs `durationFrames` (`audio.totalFrames`).
+
+## Video taste
+Follow `plugins/framefields/skills/framefields/SKILL.md`.
+- Tokens first: `theme.ts` with palette, type scale, spacing, radii, eases, one `BEAT`.
+- Native layout trees only; never fake UI, charts or screenshots with bitmaps.
+- One clock: captions/voiceover first, then BeatGrid, then reading pace (~2.4 words/s). Derive timing from generated audio; never guess or slice evenly.
+- Restrained hierarchy: one idea and focal point per shot, negative space, huge display type, small tracked-out labels.
+- Decisive motion: enter decelerating (`expo.out`, `power3.out`, `back.out(1.5)`, spring), exit fast (`power2/3.in`), never linear on position/scale/rotation, stagger siblings 2-4 frames.
+- Verify: typecheck -> schema parse -> `renderFrame` -> `renderFrameGrid` -> pixel probes -> `renderVideo` + `ffprobe`.