Skip to content

Code-first video & image, rendered natively on WebGPU.

4.3 KB
Raw Blame

trigger: always_on

General Information

Our aim is to improve the engine harness. Cheating or workarounds don't help us.

Working rules

  • We have power to do anything, be brave!
  • For the rendering scripts use process.exit so that it doesn't hang.
  • Find the root cause when debugging. If a workaround is the only option, stop and ask before proceeding.
  • Aim for an elegant solution: think engineering elegance, not only goal completion.
  • No any. Named exports unless a default is required. Top-of-file imports; no await import unless required.
  • Code must pass the cyclomatic complexity lint.
  • Use pnpm only. Never hand-edit pnpm-lock.yaml; use pnpm add/remove/install.
  • Never run bare tsc (it emits .d.ts/.js into source). Use the package's own script (tsdown, or tsc --noEmit -p .).
  • Do not run pnpm dev. Do not use a browser to test (needs Google auth); verify headlessly.
  • No fancy colors in console/log output.
  • Don't use the user's name in messages. Final answer: concise, no yapping.
  • Keep this file current: when structure or dev patterns change, update it.

Repo map

Framefields is a code-first video engine on native WebGPU (Dawn) in Node, with no browser. pnpm + turbo monorepo, Node >=22, ESM, Biome (tabs, double quotes), tsdown builds, vitest tests. Rendering needs a GPU.

  • packages/framefields: the public SDK. Composition, Layer.*, LayerAnimation, Signal, effect classes (src/effects/classes.ts), FontManager, renderFrameGrid (src/renderer/framegrid.ts). src/index.ts is large and is the export surface.
  • packages/compositions: layout (Yoga, utils/layout/resolve-layout.ts), program schema (program/schema.ts), animation and evaluator, to-virtual-media.
  • packages/webgpu-renderers: WGSL shaders, 2D/3D pipelines, Slug text, audio DSP.
  • packages/renderer: headless Node renderer (Dawn, WebCodecs/mediabunny, HeadlessMediaRenderer).
  • packages/core: shared types, Effect base, signals. Other packages: renderers, media, node-sdk (node authoring: shared/ + renderer/), server-utils, client-utils, tensor-webgpu, vision, tsconfig.
  • nodes/node-* (59): one VFX/audio/layout node each. Layout: src/shared/ (config and schema), src/renderers/ (WebGPU), src/index.ts. Only node-compositor has a SKILL.md; update it when Compositor behavior changes (packages/compositions/src/program/skillmd-check.test.ts guards it).
  • apps/renderer-service: HTTP render service (Docker, RunPod worker).
  • examples/NN_*: reference films/ads that import framefields via workspace. Study 19_framefields_film (best structure), 22_framefields_launch (lyric sync), and 28_physics_dancer (physics, live vision, GPU particles) before writing a new film.
  • plugins/framefields/skills/framefields{,-compose,-effects,-render,-vision,-cloud}: agent skills, symlinked from .agents/skills and .claude/skills.
  • examples/22_framefields_launch: reference for a lyric-driven launch film.
  • Package exports have a development condition pointing at src/. Otherwise consumers resolve dist/, so run pnpm build after changing a dependency.

Commands

  • Build: pnpm build (turbo, ^build first). Single package: pnpm --filter <name> build.
  • Test: pnpm --filter <name> test (vitest; packages/renderer uses test:vitest). Root pnpm test runs everything and is slow.
  • Lint and format: pnpm exec biome check --write <files> (lefthook runs it on commit).
  • Example: pnpm --filter @framefields/example-19-framefields-film assets|render|test|typecheck.
  • Agent skills: after changing engine API behavior, update the owning skill or reference; pnpm --filter framefields test fails if generated effects or the catalog are stale. plugins/framefields/skills/framefields-effects/references/effects-catalog.md is generated by pnpm run sync:effects-catalog; never hand-edit it.
  • Version bumps: use changesets (.changeset/).

Ignored, local-only files

Fonts (assets/fonts), *.mp4/mp3/wav/png/jpg, *.onnx, scratch*/, examples/output/, .hermes/ and .env* are gitignored. Never commit them or print secrets from .env. Write throwaway probes and logs under a scratch/ or output/ dir.

  • Every font in Layer.text must be registered with FontManager and declared in fonts. There are no system fonts headlessly.