Code-first video & image, rendered natively on WebGPU.
gemini.md skills
okanaslcf255e1c4c73
diff --git a/CLAUDE.md b/CLAUDE.mdnew file mode 100644index 00000000..2bc01a97--- /dev/null+++ b/CLAUDE.md@@ -0,0 +1,76 @@+# Framefields++Code-first video engine on native WebGPU (Dawn) in Node, no browser. pnpm + turbo monorepo, Node >=22, ESM, Biome (tabs, double quotes), tsdown builds, vitest tests. Rendering needs a GPU. Our aim is to improve the engine harness: no cheating or workarounds.++## Working rules+- Be brave; we can change anything. Aim for engineering elegance, not just goal completion.+- Debug to the root cause. If a workaround is the only option, stop and ask.+- No `any`. Named exports unless a default is required. Top-of-file imports (no `await import` unless required). Pass the cyclomatic complexity lint.+- pnpm only; never hand-edit `pnpm-lock.yaml`.+- Never run bare `tsc` (it emits into source). Use the package script (`tsdown`, or `tsc --noEmit -p .`).+- Render scripts end with `process.exit`.+- Plain console output, no colors. Don't use the user's name. Final answers concise.+- Keep this file current when structure or dev patterns change but do not bloat it.++## Repo map+- `packages/framefields`: public SDK. `Composition`, `Layer.*`, `LayerAnimation`, `Signal`, effects (`src/effects/classes.ts`), `FontManager`, `renderFrameGrid`. `src/index.ts` is the export surface. CLI in `src/cli/` (`init`, `add`, `list`, `fonts add|list`); `init` never prompts or overwrites, and its file templates are strings in `templates.ts` (a test parses them).+- `packages/components`: the component registry `framefields add` copies into user projects (shadcn-style). `defineComponent`/`bind` contract in `src/core/contract.ts`, shared helpers in `src/core/` (keys, color, layout, surface, theme). Components import only `framefields` and `../core`. Registry items declare font needs (e.g. emoji); CLI fonts are pinned by URL, size and SHA-256 in `cli/fonts.ts`. Prop docs in `components.json` are generated (drift test). Tree-shake size budgets are tested.+- `packages/compositions`: Yoga layout (`utils/layout/resolve-layout.ts`), program schema (`program/schema.ts`), lint (`program/lint.ts`), animation/evaluator.+- `packages/webgpu-renderers`: WGSL shaders, 2D/3D pipelines, Slug text (`src/slug/`), audio DSP.+- `packages/renderer`: headless Node renderer (Dawn, WebCodecs/mediabunny).+- `packages/core`: shared types, `Effect` base, signals, `emojiPresentation` (the one UTS #51 rule used by layout, preload and lint).+- Others: `renderers`, `media`, `node-sdk`, `server-utils`, `client-utils`, `tensor-webgpu`, `vision`, `tsconfig`.+- `nodes/node-*`: one VFX/audio/layout node each (`src/shared/`, `src/renderers/`, `src/index.ts`). Only `node-compositor` has a `SKILL.md`; update it when Compositor behavior changes (guarded by `skillmd-check.test.ts`).+- `apps/renderer-service`: HTTP render service (Docker, RunPod).+- `examples/NN_*`: reference films. Study `19_framefields_film` (structure), `22_framefields_launch` (lyric sync), `28_physics_dancer` (physics, vision, GPU particles) before writing one.+- `plugins/framefields/skills/framefields{,-compose,-effects,-render}`: agent skills, symlinked from `.agents/skills` and `.claude/skills`.+- Package `exports` have a `development` condition pointing at `src/`; otherwise `dist/`, so `pnpm build` after changing a dependency.+- Docs site is a separate repo: `/Users/okanaslankan/framefields-dev/docs` (Astro Starlight). `docs/lab` pins the published `framefields` version and builds the component demos; deploy with `pnpm run deploy`.++## Commands+- Build: `pnpm build`, or `pnpm --filter <name> build`.+- Test: `pnpm --filter <name> test` (`packages/renderer`: `test:vitest`). Root `pnpm test` is slow.+- Lint: `pnpm exec biome check --write <files>` (lefthook runs it on commit).+- Example: `pnpm --filter @framefields/example-19-framefields-film assets|render|test|typecheck`.+- Component gallery: `pnpm --filter framefields gallery`.+- Skills: after engine API changes update the owning skill, then `pnpm run check:plugins`. `effects-catalog.md` is generated by `pnpm run sync:effects-catalog`; never hand-edit.+- Releases: changesets (`.changeset/`).++## Local-only files+Gitignored, never commit: `assets/fonts`, `*.mp4/mp3/wav/png/jpg`, `*.onnx`, `scratch*/`, `examples/output/`, `.hermes/`, `.env*` (never print secrets). Put probes and logs under `scratch/` or `output/`.++## Engine gotchas (fail silently)+- Premultiplied-alpha blend uses `srcFactor: "one"`, never `"src-alpha"` (murky grey fades).+- WGSL: every declared storage binding must be used, or Tint drops it from the layout.+- Clamp scissor rects to non-negative integers within the surface.+- Signals are class instances; never spread them (loses `.get(ctx)`). A `NaN` uniform blacks out the frame.+- `CompositorProgramSchema` is `.strict()`: new node/program fields go in `program/schema.ts`, or `comp.toSpec()` fails.+- Keyframe frames and effect frame options must be integers (`Math.round(...)`).+- Master soundtrack layers to about -14 dBFS RMS, summed peak near 0.9.+- Procedural SFX (`renderSfx`, `mixSfxInto`, `softLimit`): `atBar` is where the effect starts; a whoosh peaks 0.42s in, so start ~0.21 bars before the cut.+- `withAnimation` keeps `animation` an enumerable data property so spreads keep tracks.+- Keep `isAbsolutePositioned` symmetric between `buildYogaNode` and `readLayout`.+- Compositor specs: audio is `config.layout` items with `kind: "media"`, `inputHandleId` (live handle ID, not label) and `volume`. Fonts go in top-level `"fonts"` with relative paths.+- Fonts: every `Layer.text` font must be registered with `FontManager` and declared in `fonts`; no system fonts. Emoji need a color emoji font (`FontManager.registerEmojiFont`, or `framefields fonts add emoji`). Missing glyphs fall back per grapheme across registered families, except path text, which is single-font.+- Path text (`pathOptions`) coordinates are local to the node's box; size the node to contain the path.+- Shape paths support SVG arcs; a circle is two arcs and draws on with `trimEnd`.+- Circular window: plate plus `Layer.shape("ellipse", { blendMode: "mask-in" })` in a box with `background: "transparent"`.+- No per-effect helpers like `blurTrackedRegion`; extend `comp.section` / `Layer.section`.++## Authoring API traps+- `LayerAnimation` has no `moveX/moveY/scale/rotate/trimPath`; use `fromTo(prop, from, to, { start, end, ease })`. `fadeIn`/`fadeOut` are positional: `.fadeIn(0, 20, "power2.out")`.+- Borders are `borderColor` + `borderWidth`; CSS `border` is ignored.+- Setting only `x` or `y` makes a node absolute; use `position: "relative"` to nudge. A centered flex needs an explicit `width` (or `"fill"`).+- Keyframes are relative to the node's own window (never add the parent's `startFrame`). Animating `y` overrides layout `y`: `fromTo("y", y + 40, y)`.+- Eases: `none power1 power2 power3 sine circ expo back elastic bounce spring cubic hold` (no `power4`).+- Don't nest an opaque container inside a `tilt3D`/`rotateX`/`perspective` container; make one card.+- Every audio layer needs `durationFrames` (`audio.totalFrames`).++## Video taste+Follow `plugins/framefields/skills/framefields/SKILL.md`.+- Tokens first: `theme.ts` with palette, type scale, spacing, radii, eases, one `BEAT`.+- Native layout trees only; never fake UI, charts or screenshots with bitmaps.+- One clock: captions/voiceover first, then BeatGrid, then reading pace (~2.4 words/s). Derive timing from generated audio; never guess or slice evenly.+- Restrained hierarchy: one idea and focal point per shot, negative space, huge display type, small tracked-out labels.+- Decisive motion: enter decelerating (`expo.out`, `power3.out`, `back.out(1.5)`, spring), exit fast (`power2/3.in`), never linear on position/scale/rotation, stagger siblings 2-4 frames.+- Verify: typecheck -> schema parse -> `renderFrame` -> `renderFrameGrid` -> pixel probes -> `renderVideo` + `ffprobe`.