Code-first video & image, rendered natively on WebGPU.
feat(forge): GLB writer, skeletons and humanoid preset for characters built in code
okanasl11fae062c425
diff --git a/.changeset/forge-glb-writer.md b/.changeset/forge-glb-writer.mdnew file mode 100644index 00000000..f90f2b38--- /dev/null+++ b/.changeset/forge-glb-writer.md@@ -0,0 +1,7 @@+---+"framefields": minor+---++`framefields/forge`: build rigged characters in code. `GltfDocument` writes GLB files with PBR materials and Khronos material extensions, skins, named morph targets and clips; `humanoidSkeleton` places a proportioned Mixamo-named skeleton that `pose`, `aim` and pose retargeting drive unchanged; `Skeleton` builds any rig with chains and mirroring; `boneGizmo` draws a rig as bones.++glTF base colors now load in the right color space: `baseColorFactor` is linear in glTF and was used as a display color, so tinted glTF materials rendered darker than authored.diff --git a/CLAUDE.md b/CLAUDE.mdindex f4a2f0d3..1b116dd1 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -20,6 +20,7 @@ Code-first video engine on native WebGPU (Dawn) in Node, no browser. pnpm + turb- `packages/renderer`: headless Node renderer (Dawn, WebCodecs/mediabunny).- `packages/vision-sam`: SAM 2.1 Tiny/Small and SAM 3 trackers (Apache-2.0 code; SAM 3 weights need runtime SAM License acceptance), private; `framefields` bundles it as a lazy chunk (`clip-vision.ts` imports it dynamically for `comp.vision(..., { sam2 | sam3 })`; `index.ts` re-exports types only) and exposes the trackers as `framefields/sam`. No weights ship; `packages/framefields/NOTICE.md` covers their licenses. Pins in `src/models/registry.ts`.- `packages/storage`: content-addressed assets on any S3-compatible bucket (R2 first): manifest format, SigV4 client (aws4fetch), verified downloads, and the locator the preview uses to redirect missing pinned files to their bucket. Shared by the CLI (`framefields/storage`), the preview and the render worker, which fetches pinned assets after LFS.+- `packages/forge`: rigged characters built in code (`framefields/forge`): `GltfDocument` GLB writer (materials + KHR extensions, skins, named morphs, clips), `Skeleton` / `humanoidSkeleton` (Mixamo names), `boneGizmo`. Pure TypeScript, no GPU; tests validate output with the Khronos validator and read it back through the engine loader. Plan: `specs/character-forge.md`.- `packages/core`: shared types, `Effect` base, signals, `emojiPresentation` (the one UTS #51 rule used by layout, preload and lint).- Others: `renderers`, `media`, `node-sdk`, `server-utils`, `client-utils`, `tensor-webgpu`, `vision`, `tsconfig`.- `nodes/node-*`: one VFX/audio/layout node each (`src/shared/`, `src/renderers/`, `src/index.ts`). Only `node-compositor` has a `SKILL.md`; update it when Compositor behavior changes (guarded by `skillmd-check.test.ts`). Effect renderers declare `reach` and `maxPixelScale` (node-sdk) when they can: without `maxPixelScale` a scaled-up leaf using the effect stays at layout resolution.diff --git a/examples/36_emberwake/CHARACTERS.md b/examples/36_emberwake/CHARACTERS.mdnew file mode 100644index 00000000..69ec1925--- /dev/null+++ b/examples/36_emberwake/CHARACTERS.md@@ -0,0 +1,145 @@+# Emberwake: characters++Every character is original. Each entry gives their look (for modeling and materials), personality and voice (for VO casting), their secret, their role in the trailer, and build notes for the engine (rig, constraints, materials).++Palette for the whole film: **ember** (#ff7a2e → #ffc46b, emissive), **frost** (#9fb8d6, #e9f1fa), **ash** (#6b6866, #2a2828), **amber** (#d99a2b, #f4c869), and **soul-pale** (#bfe8e0, emissive, the necromancer's color). Every faction owns one of them.++---++## Aldric Venn, the Last Lantern (human)++*Captain of the Lantern Watch, Calder. 54.*++**Look.** Broad, heavy and slow until he isn't. Grey stubble, a broken nose, and a burn scar up the left side of his neck that he keeps under a wool collar. Dented steel plate over a padded gambeson, the Watch's soot-black tabard with a hand-stitched lantern on it. An iron lantern hangs from his belt on a chain. His left gauntlet has a brass ember-chamber on the back, so he can light a lantern or a sword by closing his fist. He carries a longsword and a battered kite shield.++**Personality.** He is tired of all of it, but he does it anyway. Few words, dry humor, and he listens more than he speaks. He trusts nobody on this wall, himself least.++**Voice.** Low, gravelly and unhurried, worn down by forty winters of shouting into wind. He never raises it except once.++**Secret.** He held Hollowmere's gate shut the night the king burned it. He heard the people on the other side. He has never told anyone, and he still dreams it.++**In the trailer.** He lights the first lantern, gives the order "Light them. All of them," blocks a dead giant with his shield, and is the one who asks the elf the question.++**Build notes.** A humanoid rig on Mixamo names, so `pose`, `aim` and `retargetPose` work unchanged. Spring bones on the tabard and the belt lantern's chain. The ember gauntlet is emissive and drives a point light that casts shadows. Plate armor is metallic 0.9 / roughness 0.35, with `clearcoat` on the brass.++---++## Brunna Coalhand and Old Grudge (dwarf with a mech)++*Forge-mother of the Coalhand clan, Anvilmouth. 211, "young for it."*++**Look (Brunna).** Short, round and strong, with forearms like hams. Her copper-red beard is braided into two thick ropes with brass rings and clipped back over her shoulders out of the machinery's way. Soot goggles sit on her forehead. She wears a leather apron over chainmail, and her left hand is a mechanical prosthetic of bone and brass from a forge accident. She wears it with pride.++**Look (Old Grudge).** Four meters tall, hunched and heavy: an iron furnace on two thick legs, with long piston arms that end in a hammer-fist (right) and a three-fingered claw (left). The cockpit is an open cage in the chest. The emberbone heart is visible behind a grille of runes and glows like a coal. Exhaust stacks on its back breathe sparks. It has two round headlamp "eyes." Its hull is patched plate covered in clan runes, scratched tally marks and one painted daisy.++**Personality.** Loud, warm and practical. She trusts machines more than people and her mech more than both. She talks to Old Grudge like a stubborn old dog. Under the noise she has the Heartcut on her conscience.++**Voice.** Quick and bright, with a rolling burr, and she laughs in the middle of sentences. She shouts over machinery even when it is quiet.++**Secret.** Her drill broke into the Heartcut, and she felt the wall beat. She told the council, and the council told her to keep digging.++**In the trailer.** "Up, ya bucket." The mech wakes. It vaults off a cliff and punches a bone knight to pieces. She ends up flat on her back under a sword until a squirrel saves her ("…Was that a squirrel?"), then wrestles the bone dragon's jaw.++**Build notes.** Old Grudge is a rigged GLB (or built from `Layer3D` primitives with `GlbBuilder`): IK on both arms, `aim` on the head lamps, and `pose` on the pistons driven by physics impact signals so the mech recoils when it hits. The heart is emissive and blooms. The exhaust stacks emit `Layer.particles` sparks. Punches are `world.fracture` targets. Brunna rides in the cockpit as a child layer with `lookAt` aimed at the target, and her beard braids are spring bones.++---++## Ithrael Sunmourn (elf)++*Warden of the Long Sleep, the Amberwood. 912.*++**Look.** Tall, too still, and with an age that is hard to place. Pale-gold skin and white hair cut short at one side. She wears a crown of shed antlers threaded with living amber leaves that never fall. Her armor is layered bark lacquer, amber resin over dark wood, in shapes like closed petals, with a long moss-green cloak. She carries a recurve longbow of pale horn. Her arrows are amber light that crystallizes when fired. One gold tattoo runs down her right arm, a ring for every century.++**Personality.** Calm, courteous and distant. She has outlived everyone she ever loved, and she has decided that a city is a small price for a sleeping god. She isn't cruel; she is certain. The trailer should leave the audience unsure whether she is an ally.++**Voice.** Soft, precise and musical, with long pauses. She never uses contractions.++**Secret.** Nine hundred years ago she shot Kesh, the Ember-Mother's last hatchling, so the mother's grief would put her to sleep. It worked. She has kept the mountain asleep ever since.++**In the trailer.** Lit cold-blue: "I said I would not let her." She fires the amber arrow. In the last shot every head turns toward her, and she closes her eyes.++**Build notes.** The cloak and antler leaves are spring bones, and the leaves get scatter-style wind. Bark armor uses `clearcoat` over a dark base, and the amber inlays use `subsurface` with an amber `subsurfaceColor`. The arrow is an emissive capsule with camera motion blur and a `Layer.particles` trail. `lookAt` on the head for the stillness: she watches things and doesn't fidget.++---++## Whisk, of the Order of the Quiet Acorn (squirrel assassin)++*Red squirrel. Field rank: Hushtail. Age: "enough."*++**Look.** Thirty centimeters of red-copper fur with a cream belly, ear tufts, and an enormous tail that does most of her talking. She wears a tiny charcoal hooded cowl with two holes for the ear tufts, a leather bandolier of acorn-cap poison vials and a black silk scarf. She carries twin thorn daggers, real rose thorns set in silver. Her eyes are huge and black with a single catchlight. She looks exactly like a squirrel and kills like a nightmare.++**Personality.** Silent and professional, with a ritual bow after every kill. She never speaks; she communicates with tail flicks and one slow blink. She is terribly vain about her tail. Whenever there is a lull she is eating something.++**Voice.** None. Her sound is foley: a whisper of silk, the *tik* of claws on stone, and the crunch of an acorn.++**Secret.** She has a contract tonight, and it is on Ithrael. The Order wants the mother awake, because the heart-tree is dying.++**In the trailer.** She cleans her whiskers in the Amberwood and reveals a thorn dagger. She drops onto a bone knight's helm, gives two flicks, and bows while it collapses into bones. She leads the Hushtails' drop from the canopy. In the stinger, she offers an acorn to a dragon's eye.++**Build notes.** The tail is a chain of spring bones, the hero element, with high damping for a silky follow-through. `pose` drives the ear and whisker twitches from `Signal.sine` noise between beats. `lookAt` uses a tight `maxAngleDeg`, so the head snaps like a real squirrel's while the body stays still. Fur is `sheen` with `sheenColor` warm. The cowl is cloth sheen.++### The Hushtails (supporting)++- **Old Nib.** The Order's grandmaster, a grey squirrel with one eye and a monocle. He is never seen fighting, which is worse.+- **Bramble.** A huge, round fox squirrel who eats his targets' snacks first.+- **Tansy.** A flying squirrel and the Order's poisoner. She glides in on silk and is gone before the twitch.+- **The rest.** Seven Hushtails in matching cowls drop from the canopy on silk lines in formation (a crowd with its own phases).++---++## Saevra Duskwyn, the Pale Cantor (necromancer)++*Former healer of Hollowmere. 38.*++**Look.** Gaunt and beautiful in a frightening way. Her dark hair is grey at the roots and loose. Ash is smeared across her eyes like a mask, and her lips are stained soul-pale. She wears a healer's grey robe, burnt at the hem, under a mantle sewn from the scorched ribbons of Hollowmere's prayer-flags, each one stitched with a name. She carries a staff made of a city bell's clapper on a length of charred beam, and when she strikes it the dead stand. When she sings, pale light flows from her mouth like breath in the cold, and her throat glows soul-pale from the inside.++**Personality.** Gentle, and that is the horror of her. She speaks to the dead tenderly, like patients. She isn't mad; she is clear-eyed and grieving, and she believes she is owed a reckoning. Of everyone on the field, she is the one person the audience half wants to win.++**Voice.** Soft, close to the mic, almost singing. She is the trailer's narrator. Her one sung note is the film's signature sound: a pure tone that the score's choir answers.++**Secret.** She doesn't know who killed Kesh, and she assumes it was the crown. When she learns it was the elves, the whole war changes.++**In the trailer.** She narrates throughout. She walks among the ash statues of Hollowmere, sings the Hollow Choir awake, rides on Kesh's back over Lanternhold, and is the first one to look up when the mountain moves.++**Build notes.** The mantle ribbons are long spring-bone chains with wind. Her singing drives everything through signals: `Signal.*` envelopes on her song drive throat emission, the shockwave ring and the Choir's rise weights. Pale light is emissive `#bfe8e0` with bloom and volumetric fog lit from below. Her walk can be retargeted from a performer video (`person.retarget({ rig: "mixamo" })`) for an unsettling human gait.++### The Hollow Choir++The dead of Hollowmere: townsfolk in burnt clothes, crusted with ash and frost, their eyes glowing faint soul-pale. They don't snarl. They hum. Thousands of them, built as a skinned crowd scattered on the ash plain with clips "Rise," "Shamble," "March" and "Turn," phase-offset, every head `lookAt` toward Saevra. Some are armored bone knights from Hollowmere's garrison, and two are giants (dead ogres from the fighting pits).++---++## Kesh, the Hatchling (undead dragon)++*Vharra's last child. Dead for nine hundred years.*++**Look.** A young dragon's skeleton the size of a warship, blackened bone with frost in the joints. The wings are bare finger-bones strung with tattered pale membranes of song-light that glow and ripple. Embers still smolder deep in the ribcage, where a fire never quite went out. One of the ribs is snapped, with an ancient amber arrowhead still lodged in it. That is the clue: amber is elven. An attentive viewer can spot it in shot 13.++**Personality.** Confused, like a child. It cries rather than roars, a high, broken keen. It obeys Saevra's song, but it keeps turning its head toward the mountain.++**In the trailer.** It bursts up through the lake ice, dives over Lanternhold's wall, and fights Old Grudge jaw-to-fist.++**Build notes.** It needs a long-necked rig. Spine and tail are spring-bone chains. `aim` on the head runs toward the mountain on the beat (a pull against the song). The ice lake is a `world.fracture` slab with a `momentumTransfer` burst from below. The wing membranes are thin emissive planes with `iridescence`. Embers are particles inside the ribs.++---++## Vharra, the Ember-Mother (elder dragon)++*The mountain.*++**Look.** Mostly unseen. A ridge of black rock, a rib-arch in the snow, a warm stone that beats once. When she wakes, rock slides off obsidian scales lit from inside like cooling lava along every seam. Her head is the size of Lanternhold's keep, crowned with horns that people had taken for spires of rock. Her one eye is molten gold with a black slit, and it opens with the slow weight of a door the size of a cathedral.++**Voice.** Not a roar. A deep, layered, very slow voice heard as much in the ground as in the air: three pitched layers, with stones rattling underneath. She speaks only once.++**In the trailer.** She is felt throughout and seen only at the end: "Who… took… my child?"++**Build notes.** Her body is the terrain. The Spine is a ridged `Layer3D.terrain` whose scales are an SDF sculpture underneath. Waking is fractured rock slabs sliding off (fracture with a kinematic, animated stand-in), an emissive crack texture drawn in on a track, and camera shake. The eye is a sphere with an emissive iris, a slit pupil, and an eyelid that is a joint-posed shell. Scale cues: snow particles, tiny Lantern Watch soldiers on her nose, and `volumetricFog` god rays cutting past her horns.++---++## Supporting cast++- **King Osric III of Calder.** Seen once, in silhouette on a balcony as the lanterns go out. He doesn't come down.+- **The Lantern Watch.** Calder's soldiers. Steel caps, black tabards, and a lantern on every pole. They're a crowd with "Run," "Brace" and "Fall" clips, plus physics bodies when a wall section bursts.+- **The Steamguard.** The dwarven mech legion. Squat two-legged mechs smaller than Old Grudge, marching in step down the Anvilmouth causeway: a crowd on one clip with tiny phase offsets, so the march looks drilled rather than mechanical.+- **The Wardens of the Long Sleep.** Ithrael's elves. Silhouettes on golden branches, perfectly still.diff --git a/examples/36_emberwake/README.md b/examples/36_emberwake/README.mdnew file mode 100644index 00000000..80d74937--- /dev/null+++ b/examples/36_emberwake/README.md@@ -0,0 +1,32 @@+# Emberwake: a cinematic fantasy trailer++An original fantasy trailer rendered on framefields' native WebGPU 3D engine: a dragon-corpse mountain, a necromancer's choir of the dead, a human knight, a dwarf in a war-mech, an elf with a secret, a bone dragon, the Ember-Mother waking, and a squirrel assassin.++1920 × 1080 at 24 fps, letterboxed to 2.39:1, 2:04 (a 1:59 trailer plus a 5-second stinger), 32 shots.++- [STORY.md](STORY.md): the world, the factions and their secrets, the synopsis and the tone.+- [CHARACTERS.md](CHARACTERS.md): every character's look, personality, voice, secret, trailer role and build notes.+- [SCRIPT.md](SCRIPT.md): the shot-by-shot script with frame ranges, dialogue and VO, music and SFX cues, and the engine features each shot uses.++Everything here is original. The world takes its tone from big party RPGs, war MMO cinematics and grim monster-hunter sagas, but uses none of their characters, places or names.++## Status++Pre-production: story, characters and script. The package, assets and render code come next.++## Planned layout++Each act is a scene module, with shared rigs and set pieces, following example 35's `Episode` pattern:++```+src/+ theme.ts format, palette, fonts, frame ranges from SCRIPT.md+ timing.ts every shot's in/out frame and beat cue, placed on the music's measured beats+ rigs/ Aldric, Brunna + Old Grudge, Ithrael, Whisk, Saevra, Kesh, Vharra+ sets/ the Spine terrain, Lanternhold, Anvilmouth, the Amberwood, Hollowmere+ acts/ cold-open, world, rising, company, waking, title, stinger+ generate-audio.ts VO (gemini-3.8-flash-tts) with scribe-v2 timestamps+ generate-music.ts lyria-3.5 instrumental, tempo and drop measured+ generate-sfx.ts deterministic local SFX+ render.ts render <frames|grid|video> [act|film]+```diff --git a/examples/36_emberwake/SCRIPT.md b/examples/36_emberwake/SCRIPT.mdnew file mode 100644index 00000000..409c6c09--- /dev/null+++ b/examples/36_emberwake/SCRIPT.md@@ -0,0 +1,355 @@+# Emberwake: trailer script++**Format.** 1920 × 1080 at 24 fps, picture letterboxed to 2.39:1 (1920 × 804, 138 px bars). 2:04 in all, 2976 frames: a 1:59 trailer plus a 5-second stinger.++**Structure.** A cold open, then the world, the rising, the company, the waking, the title and the stinger. Saevra narrates. The score follows a classic trailer shape: a near-silent open, a pulse, a braam at 0:42, a driving build, a hard cut to silence at 1:42, and a final hit on the title.++Frame numbers are at 24 fps, and each shot runs from its first frame up to, but not including, its out frame. "VO" is voice-over and "O.S." is off screen. **Engine** notes name the framefields features each shot uses (see the matrix at the end).++---++## COLD OPEN (0:00–0:14)++### 1. Black, a heartbeat | 0:00–0:04 | frames 0–96++Black. Silence, then one heartbeat, so deep it is felt more than heard. On the beat a few embers pulse in the dark, dim, and are gone.++> **SAEVRA (V.O., close, almost sung):** Every grave is a promise…++*Engine:* `Layer.particles` embers with emission keyed to the beat (an audio signal).++### 2. The Spine, by night | 0:04–0:09 | frames 96–216++A slow aerial crane over an endless snow ridge under a cold moon. Arches of dark rock break the snow in a long row, like ribs, because they are. Far below, a chain of tiny orange lights: a dwarven mine-road. Snow drifts across the lens, and moonlight cuts god rays through low cloud.++> **SAEVRA (V.O.):** …that something will stay where you put it.++*Engine:* ridged `Layer3D.terrain` (frost and ash `colorByElevation`), rib arches as an SDF sculpture scattered along the ridge, `volumetricFog` lit by a shadow-casting moon, snow `Layer.particles` with wind, camera on a `LayerAnimation` crane path with motion blur.++### 3. The Heartcut | 0:09–0:14 | frames 216–336++Deep inside the mountain, a spinning dwarven drill head screams against rock. The wall bursts into pieces, and orange light floods the shaft. Behind the rock is not stone but a wall of black obsidian scales. A seam of molten light runs between two of them, and it **pulses**.++Hard cut to black. A second heartbeat, much louder.++*Engine:* the rock face is `world.fracture` (impact point at the drill tip, `explosionSpeed` low, debris under gravity), with motion blur on the debris. The scales are an SDF sculpture whose emissive seams follow a `Signal` heartbeat envelope, lit by point lights with `gi` color bleed onto the drill.++---++## THE WORLD (0:14–0:42)++### 4. Lanternhold at dusk | 0:14–0:19 | frames 336–456++A wide shot of a walled human city at the foot of the mountain. As the sky goes from violet to night, lanterns light along the walls in a wave that runs from the gate up to the keep, thousands of them.++> **SAEVRA (V.O.):** The kings of Calder built their walls on her feet…++*Engine:* the walls and keep are a kitbash of `Layer3D` primitives. Lanterns are instanced emissive spheres (`Layer3D.scatter` along the wall line) whose `emissiveIntensity` follows a per-instance delay, so the wave runs under bloom. The sky gradient is keyed.++### 5. Aldric's lantern | 0:19–0:23 | frames 456–552++Close on an iron lantern on a wall-pole, in wind. A gauntleted fist closes beside it; the ember-chamber on its back flares and the wick catches. Pull focus to ALDRIC behind it: weathered and still, the flame in his eyes. He looks out toward the dark lake.++*Engine:* the gauntlet's point light casts shadows. `pose` closes the fingers on keys and `lookAt` turns the head to the lake. The lantern chain is a spring bone. Depth of field comes from the camera.++### 6. Anvilmouth | 0:23–0:28 | frames 552–672++A forge-city carved into a colossal stone jaw, its teeth turned into towers, with furnace light pouring out of every window. Down a long causeway marches the Steamguard: two hundred squat mechs in perfect step, exhaust stacks breathing sparks in rhythm.++> **SAEVRA (V.O.):** …the dwarves carved her bones into furnaces…++*Engine:* the Steamguard is a skinned crowd (`Layer3D.scatter` with `crowd: { clips: ["March"] }`, small phase offsets). Sparks are GPU particles. Furnace light is emissive with bloom and `ssr` on the wet causeway. The camera tracks with the march, and motion blur shows on the swinging legs.++### 7. "Up, ya bucket." | 0:28–0:31 | frames 672–744++Inside Old Grudge's cockpit cage. BRUNNA pulls her goggles down and slams a brass lever. The emberbone heart behind the grille kicks from coal to blaze, the two headlamps ignite, and steam bursts from every joint.++> **BRUNNA:** Up, ya bucket.++The mech **rises** into frame, enormous.++*Engine:* the heart's emissive intensity is keyed and blooms. Steam comes from `Layer.vfx` "Smoke" and particles. The mech's rise is a clip ("Stand") with `pose` overrides on the pistons. Brunna's braids are spring bones that jolt with the lever.++### 8. The Amberwood | 0:31–0:36 | frames 744–864++Golden trees as tall as towers, their leaves glowing faintly amber in the dark, swaying. Elven silhouettes stand on the high branches, perfectly still. Slow push in on ITHRAEL, standing on a branch in her antler crown. A leaf falls past her face. She doesn't blink.++> **SAEVRA (V.O.):** …and the elves sang her lullabies… and kept their secrets.++*Engine:* trees are scattered meshes with `wind` and their leaves `subsurface` amber. Ithrael's cloak and the crown's leaves are spring bones. The falling leaf is a 3D physics body with air drag. `volumetricFog` amber light shafts. Her head `lookAt` follows the leaf for a beat and then returns.++### 9. A squirrel | 0:36–0:39 | frames 864–936++On a branch nearby, a red squirrel washes its whiskers. Pure nature documentary. Its head snaps toward camera. A catchlight in the huge black eye. The tail flicks once, and the movement uncovers a tiny silver thorn dagger strapped against its flank.++Cut before anyone can react.++*Engine:* WHISK's head snaps with a tight `lookAt` (`maxAngleDeg`, a hard ease), whisker twitches are `pose` on `Signal.sine` noise, the tail is a spring-bone chain, and the fur is `sheen`. A macro camera with shallow depth of field.++### 10. Hollowmere | 0:39–0:42 | frames 936–1008++An ash-grey ruin on a frozen lake. Among its broken houses stand ash statues: people frozen mid-step and mid-embrace, a mother holding a child. SAEVRA walks slowly between them, touching each one as she passes, prayer-ribbons trailing behind her.++> **SAEVRA (V.O.):** They called it mercy, when they burned my city.++She stops. Looks up into the lens.++> **SAEVRA:** I call it… a debt.++*Engine:* the statues are ash-material human models posed with `pose` (one rig, many held poses), with snow particles. Saevra's walk is retargeted from a performer video (`person.retarget`) or a walk clip. Ribbons are spring bones with wind. Cold blue `volumetricFog` at ground height.++**MUSIC: BRAAM on the cut.**++---++## THE RISING (0:42–1:12)++### 11. The note | 0:42–0:46 | frames 1008–1104++Close on Saevra. She opens her mouth and **sings**: one pure, high note. Pale light rises from her throat like breath in the cold. A ring of soul-pale light ripples out across the ash at her feet, and the ash statues crumble as it passes them.++*Engine:* her throat's emission and the ring's radius follow the sung note's envelope (an audio signal from the VO stem). The ring is an emissive torus scaled on a track. The statues fracture in the ring's path (each one's `fractureFrame` comes from its distance to her). Bloom.++### 12. The Hollow Choir | 0:46–0:51 | frames 1104–1224++A wide shot of the ash plain. The ground splits in a thousand places, and the dead of Hollowmere **rise** in unison, then turn their heads toward her at the same moment. They don't roar. They hum, and the hum becomes the choir in the score.++*Engine:* a 2,000-instance skinned crowd: "Rise" with seeded `animOffset` delays spreading outward from Saevra, then "Turn." The ground cracks are an emissive soul-pale texture drawn in along the ring. Camera crane up and back. Motion blur on the rising bodies.++### 13. Kesh | 0:51–0:55 | frames 1224–1320++The frozen lake. The ice bulges, cracks and **explodes** upward as a dragon skeleton bursts through it, blackened bone streaming water and ice. Its wings unfold: bare finger-bones strung with membranes of pale, rippling light. It throws back its head and *keens*, a high, broken, child's cry.++(A frame-perfect detail, hero-lit for half a second: an old amber arrowhead lodged in one of its ribs.)++*Engine:* the lake surface is a large `world.fracture` slab (`impactPoint` under Kesh, high `momentumTransfer`), with motion blur on the shards, SSR on the wet ice, and ember particles in the ribcage. Wings are posed with `pose` keys, and the spine and tail are spring bones. The keen's audio drives the membranes' emission.++### 14. The lanterns go out | 0:55–0:58 | frames 1320–1392++Lanternhold's wall from the lake side. The lanterns go out one by one, in a wave rushing toward camera. Last, high on a balcony of the keep, a crowned silhouette, KING OSRIC, steps back into the dark.++*Engine:* the same lantern instances as shot 4, with emission keyed out in reverse order. Bloom falls away as they die.++### 15. "Light them." | 0:58–1:01 | frames 1392–1464++Close on Aldric. Snow hits his face. The lanterns die along the wall behind him. He doesn't turn.++> **ALDRIC:** Light them. All of them.++Watch soldiers sprint past camera with torches.++*Engine:* the soldiers are a crowd on "Run" with camera motion blur. `lookAt` keeps Aldric's head fixed while his body stands braced against the wind.++### 16. The wall breaks | 1:01–1:05 | frames 1464–1560++From the battlements: the Choir hits the base of the wall like a tide. The wall section **bursts**, and stone blocks and soldiers tumble down into the mass of the dead.++*Engine:* the wall section is a `world.fracture` with its impact on the Choir's front line, soldiers as 3D physics bodies, motion blur, and dust from `Layer.vfx` "Dust" composited over the debris.++### 17. Overhead | 1:05–1:08 | frames 1560–1632++Looking straight up from the wall walk. Kesh dives over the battlements. A wing passes over camera so close it fills the frame, and its membranes light the snow pale for an instant.++Black.++*Engine:* an animated wing pass close to the lens with heavy camera motion blur (`shutter: 270`) and a fully occluded frame used as the cut.++### 18. The warden | 1:08–1:12 | frames 1632–1728++Ithrael on the wall, lit cold blue by Kesh's light, watching the dragon with no expression at all. Aldric is beside her.++> **ALDRIC:** You said she couldn't wake.+>+> **ITHRAEL:** *(a beat; she does not look at him)* I said I would not let her.++*Engine:* a two-shot with the key light from Kesh's emissive wings, `lookAt` on both heads (Ithrael's locked on the dragon, Aldric's on her). Depth of field.++---++## THE COMPANY (1:12–1:42)++The score drives. Cuts land on the beat.++### 19. Old Grudge | 1:12–1:15 | frames 1728–1800++Old Grudge vaults off a ledge into the battle. It lands, and the ground cracks under it. It swings the hammer-fist straight into a bone knight, which **explodes** into fragments.++> **BRUNNA:** *(laughing)* HA!++*Engine:* the jump is a physics body with a keyed arc, and the landing's impact signal drives the mech's `pose` recoil and a camera shake. The knight is a `world.fracture` hit by a kinematic fist (`momentumTransfer`). Motion blur.++### 20. The arrow | 1:15–1:17 | frames 1800–1848++Ithrael draws. The arrow forms out of amber light along the string. She releases, and the arrow streaks away across the frame.++*Engine:* her draw arm uses an IK `chain` to the string point. The arrow is an emissive capsule at 4,000 px/s with motion blur (the streak is the shot) and a particle trail.++### 21. The shield | 1:17–1:20 | frames 1848–1920++A dead giant swings a tree-trunk club down at Aldric. He takes it on his shield, sinking to one knee. The lantern at his belt **flares** white-hot, and the giant's arm catches fire.++*Engine:* the hit's impact signal drives `pose` on Aldric's spine and knees (a flinch weighted by the impact). The lantern's point light intensity is a spike. Fire from `Layer.vfx` "Fire."++### 22. "Was that a squirrel?" | 1:20–1:25 | frames 1920–2040++Brunna, thrown from the cockpit, flat on her back in the snow. A huge bone knight raises a two-handed sword over her.++Behind its helm, the smallest shadow.++WHISK drops onto the crown of the helm. *Tik. Tik.* Two flicks of a thorn dagger. A pause. Whisk steps off backward into the air.++The knight **falls apart**, every bone at once, into a heap.++Whisk lands, and bows.++> **BRUNNA:** *(staring)* …Was that a squirrel?+>+> **ALDRIC:** *(O.S., not looking)* Don't.++*Engine:* the knight is a `world.fracture` with zero `explosionSpeed`, so it collapses straight down under gravity. Whisk's bow is a `pose` held with `mode: "rest"`, and the tail spring bones overshoot and settle. The comedy timing (beat, pause, collapse) is set in frames in `timing.ts`. The music drops out for the pause.++### 23. The Hushtails | 1:25–1:28 | frames 2040–2112++Seven squirrels in matching charcoal cowls drop from the dark on silk lines in perfect formation. As they fall they throw acorn-cap bombs into the Choir, which burst into poison-green smoke.++*Engine:* a small crowd with phase-offset "Drop" clips and spring-bone tails. The acorns are 3D physics bodies. The bursts are `Layer.particles` with an emissive green.++### 24. Jaw to fist | 1:28–1:32 | frames 2112–2208++Kesh lands on the wall and lunges. Old Grudge catches its jaw in both hands, its feet skidding back on the stone, sparks pouring from the overloaded joints. The heart behind the grille burns white.++*Engine:* the mech's arms are an IK pair on the jaw bones' world positions, and its feet slide on a keyed track. Sparks are particles emitted at the arm joints. The heart's emission is keyed to bloom past white.++### 25. The cantor in the sky | 1:32–1:36 | frames 2208–2304++Saevra stands on Kesh's back as it climbs above the city, singing. Lightning tears the clouds. Below them, the whole mountain **shudders**: snow pours in avalanches down every ridge at once.++*Engine:* `Layer.vfx` "Lightning" lights the clouds, and its flashes drive a directional light's intensity. Avalanches are particles shed from the terrain's ridgelines, plus camera shake.++### 26. Eyes | 1:36–1:39 | frames 2304–2376++Five extreme close-ups of eyes, one per beat: Aldric (frames 2304–2319), Brunna (2319–2334), Ithrael (2334–2348), Whisk (2348–2362), and Saevra, whose irises flare pale (2362–2376).++*Engine:* five camera cuts in one scene, each with its own focus, catchlight from an area light, and `subsurface` skin.++### 27. The mountain opens | 1:39–1:42 | frames 2376–2448++A great slab of the mountainside **splits** and slides away. Beneath it, molten-orange light pours from a seam of black scales.++*Engine:* the slab is a `world.fracture` with a large `cells` count, kinematic and animated, so the pieces wake from the slab's motion. The emissive scales are lit with `gi` bouncing orange onto the snow.++---++## THE WAKING (1:42–1:54)++### 28. Silence | 1:42–1:44 | frames 2448–2496++**Hard cut to silence.** A wide shot of the whole battlefield, frozen. Snow hangs in the air. Every soldier, every dead thing, Kesh, all of them stop and turn toward the mountain.++*Engine:* time freezes: particles' `speed` and the crowds' `animSpeed` are keyed to 0.02, with motion blur off for one beat (camera `shutter` keyed to 0).++### 29. She rises | 1:44–1:49 | frames 2496–2616++The camera pulls back and back. The whole ridge of the Spine lifts. It is a neck. A head the size of the keep rises out of the mountain, avalanches of rock and snow sliding off its scales, and the light inside it glows through every seam. Lanternhold is a toy at her feet.++*Engine:* the head is an SDF sculpture with a terrain shell that fractures off on its kinematic rise (`world.fracture` over the animated layer). `volumetricFog` god rays cut between the horns. The scale comes from crowds of tiny soldiers and snow particles. A long camera pull with motion blur.++### 30. "Who took my child?" | 1:49–1:54 | frames 2616–2736++Extreme close-up. An eye the size of a cathedral door opens: molten gold, with a black slit that narrows to focus on the wall.++> **VHARRA (V.O., three layers, impossibly slow):** Who… took… my child?++On the wall, slowly, every head turns: Aldric's, Brunna's, Whisk's, Saevra's from Kesh's back, even the dead's. They all look at **Ithrael**.++Ithrael closes her eyes.++*Engine:* the eyelid shell is `pose`-keyed. The pupil's slit is a scale animation. Every head turn is a `lookAt` whose target is keyed to Ithrael at staggered frames, and the crowd's heads turn with it. Vharra's voice drives camera shake through an audio signal.++---++## TITLE (1:54–1:59)++### 31. EMBERWAKE | 1:54–1:59 | frames 2736–2856++Black. Embers drift in from the edges and gather into extruded, cold, dark-iron letters: **EMBERWAKE**. The letters **crack** along seams, and ember-light pours out of the cracks. The final hit of the score lands.++Under it, small: *The mountain remembers.*++*Engine:* `Layer3D.extrudedText` in a display font, fractured (`world.fracture` with a low `explosionSpeed` so the pieces only part along the seams), and an emissive interior color (`interiorColor` hot ember, bloom). Embers are `Layer.particles` with attractors on the letters. Motion blur.++---++## STINGER (1:59–2:04)++### 32. An offering | 1:59–2:04 | frames 2856–2976++Quiet. A warm stone ledge under falling snow. Whisk sits on it, cracking an acorn, tail curled around her.++The ledge **breathes**. It is a nostril. Whisk stops chewing.++Beside her, an eye the size of a door opens, molten gold, and fixes on her.++A long beat. Whisk slowly holds the acorn out toward it.++The pupil narrows.++Cut to black. A single tiny *crunch*.++*Engine:* the snout breathes through `pose` on a nostril joint (a slow sine), with warm `volumetricFog` exhaling. Whisk's freeze is held with `pose` `mode: "rest"` while only the tail tip keeps moving (spring bones). The eye reuses shot 30's.++---++## Audio++### Voice (AI Gateway, `google/gemini-3.8-flash-tts`)++| Who | Lines | Direction |+| --- | --- | --- |+| Saevra | 1, 2, 4, 6, 8, 10 | Close to the mic, soft, tender, almost sung, with no menace in it, which is what makes it frightening. Pauses where the script has ellipses. |+| Saevra (sung) | 11 | One sustained pure note, no words. Generated or synthesized, and pitched to the score's key. |+| Aldric | 15, 18, 22 | Low, gravel, tired. "Light them. All of them." is the one line he raises his voice on. |+| Brunna | 7, 19, 22 | Bright, quick, a rolling burr. |+| Ithrael | 18 | Soft, precise, musical, no contractions, a long pause before the line. |+| Vharra | 30 | Three layers of the same read, pitched -12, -19 and -24 semitones, very slow, with ground rumble under it. |++Word timestamps are verified with `elevenlabs/speech-to-text/scribe-v2`, and cuts are timed to the words, as in example 35.++### Music (`google/lyria-3.5`, instrumental)++| Time | Cue |+| --- | --- |+| 0:00–0:14 | Near silence, heartbeat subs, a low string drone. |+| 0:14–0:42 | A pulse enters, slow-building low strings, a lone solo voice (no lyrics). |+| 0:42 | **Braam.** Full choir enters on Saevra's note. |+| 0:42–1:12 | Percussion and choir building, staccato strings. |+| 1:12–1:42 | Full drive: taikos, brass ostinato, cuts on the beat. A dropout of about 0.8 s at shot 22's pause. |+| 1:42 | **Hard silence.** Only wind and a single heartbeat. |+| 1:49–1:54 | Sub rumble under Vharra's voice. |+| 1:54 | **Final hit** on the title crack, a long ringing tail. |+| 1:59–2:04 | Silence, falling snow, foley only. |++The generated track is measured for tempo and the drop (as in example 35's scene 7), and the braam and the cuts in shots 19–27 are placed on its beats.++### Sound effects (synthesized locally)++Heartbeats (sine subs), the drill whine, cracking rock and ice, wind, the lanterns igniting, mech servos and steam, bone collapses, the squirrel foley (*tik*, silk, crunch), Kesh's keen (pitched, granular), and the Ember-Mother's rumble.++---++## Engine feature matrix++| Feature | Shots |+| --- | --- |+| Terrain, SDF sculpture, scatter with wind | 2, 3, 6, 8, 25, 27, 29 |+| Skinned crowds | 6, 12, 15, 23, 28, 30 |+| Character posing (`pose`), `lookAt`, IK, spring bones | 5, 7, 8, 9, 10, 18, 20, 21, 22, 24, 30, 32 |+| Pose retargeting from vision | 10 (Saevra's walk), plus any performer-driven shot |+| Voronoi fracture (static and kinematic, animated) | 3, 11, 13, 16, 19, 22, 27, 29, 31 |+| 3D physics (Rapier) | 8, 16, 19, 23 |+| Per-object shutter motion blur | 2, 3, 6, 13, 15, 16, 17, 19, 20, 24, 29, 31 |+| Volumetric fog and god rays | 2, 8, 10, 29, 32 |+| SSR and diffuse GI | 3, 6, 13, 27 |+| GPU particles and procedural VFX | 1, 2, 6, 7, 13, 16, 20, 21, 23, 24, 25, 31 |+| Audio-driven signals | 1, 3, 11, 13, 30 |+| Extruded type | 31 |diff --git a/examples/36_emberwake/STORY.md b/examples/36_emberwake/STORY.mdnew file mode 100644index 00000000..c35321bf--- /dev/null+++ b/examples/36_emberwake/STORY.md@@ -0,0 +1,70 @@+# Emberwake: world and story++An original high-fantasy world in the spirit of the big party-and-war fantasies: a grim, morally grey frontier, a continent-scale war, and a company of misfits with secrets. Every name, place, character and event here is new; nothing is borrowed from an existing game, book or film.++**Logline.** A mountain range is the corpse of a dragon, and everyone has built a life on her bones. When a grieving necromancer sings the dead awake to settle a debt with the crown, a burnt-out knight, a dwarf in a war-mech, an elf with a nine-hundred-year-old secret and a squirrel assassin have one night to keep the mountain asleep. Then the mountain opens its eye.++**Tagline.** *The mountain remembers.*++## The world: the Spine of Vharra++A thousand years ago the Ember-Mother, **Vharra**, the last of the elder dragons, lay down across the north of the continent and did not get up. Snow covered her. Forests grew in her shadow. Over the centuries, people stopped calling the ridge her spine and started calling it *the Spine*. Most of them no longer know it was ever anything else.++Ribs the size of cathedrals still break through the snow on the high passes. The stone there is warm in winter.++Three peoples live off her body, and one remembers what she is.++| People | Where | What they take from her | What they believe |+| --- | --- | --- | --- |+| Humans of the **Kingdom of Calder** | **Lanternhold**, a walled city at the mountain's feet | Warmth, safe valleys, and *ember*, the glowing dust in her old wounds, which burns in their lanterns for a year and keeps the dead from rising | The dragon is a story for children. The lanterns are tradition. |+| Dwarves of **Anvilmouth** | A forge-city carved into what was her jaw | *Emberbone*: her bones still hold heat, and one shard runs a forge, or a war-mech, for a generation | Bone is ore. Ore is meant to be dug. |+| Elves of the **Amberwood** | A forest of golden trees on her flank, roots deep in her hide | Long life: the forest drinks her slow heartbeat | She is asleep, not dead. She must never wake, whatever it costs. |+| The dead of **Hollowmere** | An ash-grey ruin on the frozen lake below the Spine | Nothing. They were taken from. | Nothing yet. They are waiting for someone to sing. |++### Ember and the dead++In this world the dead do not stay down on their own. A body left in the cold for a winter will stand up in the spring. Ember stops it: burning ember keeps a grave quiet. This is why Calder's lanterns burn on every wall, and why the Lantern Watch, Calder's order of knights, burns ember at every crossroads and every grave.++Necromancy is the art of singing to the dead in the voice ember silences. Calder outlawed it three hundred years ago. In Calder, a necromancer is a *cantor*.++### What everyone is hiding++- **The Crown.** Twelve years ago a fever swept Hollowmere. King Osric III feared the dead would rise faster than the Watch could burn them, so he sealed the city's gates and burned it with everyone inside. The official story is "the fever took Hollowmere." Aldric Venn held the gate that night.+- **The dwarves.** Anvilmouth's deepest shaft, the Heartcut, has reached something warm that beats once an hour. The clan council voted to keep digging. Brunna's drill was first through the wall.+- **The elves.** Vharra did not lie down to die. She lay down to sleep, grieving her last hatchling, Kesh, who was lost the same year. Nobody knows what took Kesh. Ithrael knows: the elves killed Kesh, so the mother would fall into a grief deep enough to sleep, and stop burning their forests. Ithrael was the archer.+- **The squirrels.** The **Order of the Quiet Acorn** has worked as an assassins' guild for all four peoples for four hundred years. It belongs to none of them. Its one loyalty is the oldest tree in the Amberwood, which grows from Vharra's heart, and its creed is "the tree first." It has always known what the mountain is.++## Synopsis++Saevra Duskwyn was Hollowmere's healer. She was outside the walls, gathering winter herbs, the night the king burned the city. She came back to ash. She has spent twelve years learning to sing to the dead, and has learned that ember only quiets them; it doesn't put them to rest.++She wants a reckoning, and she has found the instrument for it. In the ice beneath Hollowmere lake lie the bones of a young dragon. She sings them awake: **Kesh**, Vharra's lost child, now a skeleton held together by song. If the child's voice calls, the mother will wake, and the Ember-Mother's first breath will burn Lanternhold and its king off her feet.++Saevra doesn't know who killed Kesh. She assumes it was humans. Everyone does.++On the night of the first snow, the dead of Hollowmere rise in their thousands, the Hollow Choir, and march on Lanternhold with Kesh flying above them. The lanterns go out one by one. Saevra's song does that.++Four unlikely defenders hold the wall:++- **Aldric Venn**, the last captain of the Lantern Watch. He is too old for this and carries the guilt of the Hollowmere gate.+- **Brunna Coalhand**, a dwarf forge-mother who came to Lanternhold to sell war-mechs and stayed to fight in one, **Old Grudge**, a mech with an emberbone heart.+- **Ithrael Sunmourn**, the elven warden sent "to help." Her real orders are to make sure Vharra doesn't wake, at any price, including Lanternhold.+- **Whisk**, a red squirrel of the Order of the Quiet Acorn, hired by someone to kill someone on the wall tonight. She won't say who.++The trailer ends on the moment the story turns. The Spine lifts, a head the size of a castle rises out of the mountain, and the Ember-Mother asks one question: *"Who took my child?"*++Every head on the battlefield, living and dead, turns toward the elf.++## The film beyond the trailer (for tone, not for the render)++- Whisk's contract is on Ithrael. The Order wants Vharra awake: the tree first, and the tree is dying because its heart is asleep.+- Saevra learns that the elves, not the crown, killed Kesh. She has to choose between her revenge and the truth, with a grieving dragon listening.+- Aldric's arc is confession. Brunna's is the Heartcut: she has been drilling into a living mother.+- There is no clean villain. Everyone took something from the mountain.++## Tone++- Grim, grounded and morally grey, like a monster-hunter saga: mud, frost, tired eyes, and nobody is purely good.+- Epic in scale, like a war-of-the-world MMO cinematic: armies, a dragon over a city, mechs marching, the score swelling.+- A party you fall for, like a modern tabletop RPG: banter under pressure, secrets, and one perfect comedy beat in the middle of the carnage.+- Squirrels are deadly and adorable at once. Play them dead straight; the comedy comes from everyone else's reactions.diff --git a/packages/forge/package.json b/packages/forge/package.jsonnew file mode 100644index 00000000..939d00b4--- /dev/null+++ b/packages/forge/package.json@@ -0,0 +1,38 @@+{+ "name": "@framefields/forge",+ "version": "2.14.0",+ "private": true,+ "description": "Build rigged, skinned glTF characters and props in code: a GLB writer with PBR material extensions, skins, morph targets and clips, and skeleton builders.",+ "type": "module",+ "sideEffects": false,+ "license": "Apache-2.0",+ "module": "./dist/index.mjs",+ "types": "./dist/index.d.mts",+ "exports": {+ ".": {+ "types": "./dist/index.d.mts",+ "development": "./src/index.ts",+ "import": "./dist/index.mjs",+ "default": "./dist/index.mjs"+ }+ },+ "files": [+ "dist/**"+ ],+ "scripts": {+ "build": "tsdown",+ "dev": "tsdown --watch",+ "typecheck": "tsc --noEmit -p .",+ "test": "vitest run",+ "clean": "rm -rf .turbo && rm -rf node_modules && rm -rf dist"+ },+ "devDependencies": {+ "@framefields/tsconfig": "workspace:*",+ "@framefields/webgpu-renderers": "workspace:*",+ "@types/node": "^24.1.0",+ "gltf-validator": "2.0.0-dev.3.10",+ "tsdown": "^0.19.0",+ "typescript": "^5.8.3",+ "vitest": "^4.1.5"+ }+}diff --git a/packages/forge/src/color.ts b/packages/forge/src/color.tsnew file mode 100644index 00000000..7aed3915--- /dev/null+++ b/packages/forge/src/color.ts@@ -0,0 +1,40 @@+/**+ * Colors as authors write them (`"#d99a2b"`, `"#d99a2bcc"`, or display+ * 0..1 RGB(A) tuples) turned into glTF's linear factors.+ */++/** A display (sRGB) color: hex, or 0..1 components. */+export type ColorInput =+ | string+ | readonly [number, number, number]+ | readonly [number, number, number, number];++const srgbToLinear = (c: number) =>+ c <= 0.04045 ? c / 12.92 : ((c + 0.055) / 1.055) ** 2.4;++function parseHex(hex: string): [number, number, number, number] {+ const h = hex.trim().replace(/^#/, "");+ const full =+ h.length === 3 || h.length === 4 ? [...h].map((c) => c + c).join("") : h;+ if (!/^[0-9a-fA-F]{6}([0-9a-fA-F]{2})?$/.test(full))+ throw new Error(`color: "${hex}" is not #rgb, #rgba, #rrggbb or #rrggbbaa`);+ const n = (i: number) => Number.parseInt(full.slice(i, i + 2), 16) / 255;+ return [n(0), n(2), n(4), full.length === 8 ? n(6) : 1];+}++/** Display RGBA, 0..1. */+export function displayRgba(c: ColorInput): [number, number, number, number] {+ if (typeof c === "string") return parseHex(c);+ return [c[0], c[1], c[2], c[3] ?? 1];+}++/** Linear RGB, 0..1, for glTF color factors. Alpha stays as written. */+export function linearRgba(c: ColorInput): [number, number, number, number] {+ const [r, g, b, a] = displayRgba(c);+ return [srgbToLinear(r), srgbToLinear(g), srgbToLinear(b), a];+}++export function linearRgb(c: ColorInput): [number, number, number] {+ const [r, g, b] = linearRgba(c);+ return [r, g, b];+}diff --git a/packages/forge/src/document.ts b/packages/forge/src/document.tsnew file mode 100644index 00000000..32ab1578--- /dev/null+++ b/packages/forge/src/document.ts@@ -0,0 +1,540 @@+/**+ * A glTF 2.0 document built in code and written as one GLB: meshes with+ * named morph targets, PBR materials with Khronos extensions, textures,+ * a node tree, skins from a `Skeleton`, and animation clips. Every call+ * checks its input, so a mistake fails where it is made, not as a blank+ * model in a render.+ *+ * ```ts+ * const doc = new GltfDocument();+ * const rig = doc.addSkeleton(humanoidSkeleton({ height: 186, heads: 8 }));+ * const skin = doc.addMaterial({ color: "#e8d2b0", roughness: 0.55, sheen: { color: "#ffd9c0" } });+ * const body = doc.addMesh({ name: "Body", primitives: [{ positions, normals, joints, weights, indices, material: skin }] });+ * doc.addNode({ name: "Ithrael", mesh: body, skin: rig.skin });+ * fs.writeFileSync("ithrael.glb", doc.toGlb());+ * ```+ */+import { GlbBuilder } from "./glb-builder.js";+import {+ type MaterialSpec,+ materialJson,+ type TextureRef,+} from "./material.js";+import type { Skeleton } from "./skeleton.js";++type Numbers = ArrayLike<number>;++export interface MorphTargetSpec {+ /** Written to `mesh.extras.targetNames` (the convention every DCC tool reads). */+ name: string;+ /** Per-vertex position deltas (3 per vertex). */+ positions: Numbers;+ /** Per-vertex normal deltas (3 per vertex). */+ normals?: Numbers;+}++export interface PrimitiveSpec {+ /** 3 per vertex. */+ positions: Numbers;+ /** 3 per vertex, unit length. */+ normals?: Numbers;+ /** 4 per vertex (xyz, handedness w). */+ tangents?: Numbers;+ /** 2 per vertex, glTF orientation (v down from the top of the image). */+ uvs?: Numbers;+ /** A second UV set (e.g. baked maps). */+ uvs1?: Numbers;+ /** Vertex color, linear RGBA, 4 per vertex. */+ colors?: Numbers;+ /** 4 joint indices per vertex, into the skin's joint list. */+ joints?: Numbers;+ /** 4 weights per vertex; normalized on write. */+ weights?: Numbers;+ /** Triangle (or line / point) indices; omit for an unindexed list. */+ indices?: Numbers;+ material?: MaterialRef;+ /** Default "triangles". */+ mode?: "triangles" | "lines" | "points";+ targets?: MorphTargetSpec[];+}++export interface MeshSpec {+ name?: string;+ primitives: PrimitiveSpec[];+ /** Default morph weights, one per target. */+ weights?: number[];+}++export interface NodeSpec {+ name?: string;+ mesh?: MeshRef;+ skin?: SkinRef;+ translation?: [number, number, number];+ /** Quaternion x, y, z, w. */+ rotation?: [number, number, number, number];+ scale?: [number, number, number];+ children?: NodeRef[];+ extras?: Record<string, unknown>;+}++export type AnimationPath = "translation" | "rotation" | "scale" | "weights";++export interface ChannelSpec {+ node: NodeRef;+ path: AnimationPath;+ /** Key times in seconds, increasing. */+ times: Numbers;+ /** 3 (translation, scale), 4 (rotation) or one per morph target (weights) per key. */+ values: Numbers;+ /** Default "linear". */+ interpolation?: "linear" | "step";+}++export interface AnimationSpec {+ name: string;+ channels: ChannelSpec[];+}++export interface TextureSpec {+ /** Encoded image bytes. */+ image: Uint8Array;+ mimeType: "image/png" | "image/jpeg" | "image/webp";+ wrap?: "repeat" | "clamp" | "mirror";+ filter?: "linear" | "nearest";+}++export interface MeshRef {+ readonly mesh: number;+ readonly targetNames: readonly string[];+ readonly maxJoint: number;+}+export interface MaterialRef {+ readonly material: number;+}+export interface NodeRef {+ readonly node: number;+}+export interface SkinRef {+ readonly skin: number;+ readonly jointCount: number;+}++export interface SkeletonRefs {+ skin: SkinRef;+ /** The root joints' nodes (usually one: Hips). */+ roots: NodeRef[];+ /** Each joint's node, by joint name. */+ joints: ReadonlyMap<string, NodeRef>;+ /** Joint name → index in the skin (for `PrimitiveSpec.joints`). */+ jointIndex: ReadonlyMap<string, number>;+}++const PATH_SIZE: Record<Exclude<AnimationPath, "weights">, number> = {+ translation: 3,+ rotation: 4,+ scale: 3,+};++const WRAP = { repeat: 10497, clamp: 33071, mirror: 33648 } as const;+const FILTER = {+ linear: { magFilter: 9729, minFilter: 9987 },+ nearest: { magFilter: 9728, minFilter: 9984 },+} as const;+const MODE = { points: 0, lines: 1, triangles: 4 } as const;++function floats(name: string, data: Numbers, size: number, count?: number) {+ if (data.length % size !== 0)+ throw new Error(+ `${name}: length ${data.length} is not a multiple of ${size}`,+ );+ if (count !== undefined && data.length / size !== count)+ throw new Error(+ `${name}: ${data.length / size} entries for ${count} vertices`,+ );+ const out = Float32Array.from(data);+ for (let i = 0; i < out.length; i++)+ if (!Number.isFinite(out[i]))+ throw new Error(`${name}: value ${i} is not finite`);+ return out;+}++/** Weights normalized to sum 1 per vertex (glTF requires it). */+function normalizedWeights(data: Numbers, count: number): Float32Array {+ const w = floats("weights", data, 4, count);+ for (let v = 0; v < count; v++) {+ const o = v * 4;+ let sum = 0;+ for (let i = 0; i < 4; i++) {+ if ((w[o + i] as number) < 0)+ throw new Error(`weights: vertex ${v} has a negative weight`);+ sum += w[o + i] as number;+ }+ if (sum <= 0) throw new Error(`weights: vertex ${v} has no weight`);+ for (let i = 0; i < 4; i++) w[o + i] = (w[o + i] as number) / sum;+ }+ return w;+}++export class GltfDocument {+ private readonly bin = new GlbBuilder();+ private readonly materials: Record<string, unknown>[] = [];+ private readonly meshes: Record<string, unknown>[] = [];+ private readonly meshRefs: MeshRef[] = [];+ private readonly nodes: Record<string, unknown>[] = [];+ private readonly parents = new Map<number, number>();+ private readonly skins: Record<string, unknown>[] = [];+ private readonly skinRefs: SkinRef[] = [];+ private readonly animations: Record<string, unknown>[] = [];+ private readonly textures: Record<string, unknown>[] = [];+ private readonly images: Record<string, unknown>[] = [];+ private readonly samplers: Record<string, unknown>[] = [];+ private readonly extensions = new Set<string>();++ addMaterial(spec: MaterialSpec): MaterialRef {+ const { json, extensions } = materialJson(spec);+ for (const e of extensions) this.extensions.add(e);+ for (const t of [+ spec.colorTexture,+ spec.metallicRoughnessTexture,+ spec.normalTexture,+ spec.occlusionTexture,+ spec.emissiveTexture,+ ])+ if (t) this.checkTexture(t);+ this.materials.push(json);+ return { material: this.materials.length - 1 };+ }++ addTexture(spec: TextureSpec): TextureRef {+ const bufferView = this.bin.image(spec.image);+ this.images.push({ bufferView, mimeType: spec.mimeType });+ const wrap = WRAP[spec.wrap ?? "repeat"];+ this.samplers.push({+ ...FILTER[spec.filter ?? "linear"],+ wrapS: wrap,+ wrapT: wrap,+ });+ this.textures.push({+ source: this.images.length - 1,+ sampler: this.samplers.length - 1,+ });+ return { texture: this.textures.length - 1 };+ }++ addMesh(spec: MeshSpec): MeshRef {+ if (!spec.primitives.length)+ throw new Error(`mesh ${spec.name ?? ""}: no primitives`);+ const names = spec.primitives[0]?.targets?.map((t) => t.name) ?? [];+ if (new Set(names).size !== names.length)+ throw new Error(`mesh ${spec.name ?? ""}: duplicate morph target names`);+ let maxJoint = -1;+ const primitives = spec.primitives.map((p, i) => {+ const label = `mesh ${spec.name ?? this.meshes.length} primitive ${i}`;+ const targets = p.targets?.map((t) => t.name) ?? [];+ if (targets.join("\0") !== names.join("\0"))+ throw new Error(+ `${label}: every primitive needs the same morph targets in the same order`,+ );+ const { json, maxJoint: mj } = this.primitive(p, label);+ maxJoint = Math.max(maxJoint, mj);+ return json;+ });+ if (spec.weights && spec.weights.length !== names.length)+ throw new Error(+ `mesh ${spec.name ?? ""}: ${spec.weights.length} default weights for ${names.length} targets`,+ );+ this.meshes.push({+ ...(spec.name ? { name: spec.name } : {}),+ primitives,+ ...(spec.weights ? { weights: spec.weights } : {}),+ ...(names.length ? { extras: { targetNames: names } } : {}),+ });+ const ref: MeshRef = {+ mesh: this.meshes.length - 1,+ targetNames: names,+ maxJoint,+ };+ this.meshRefs.push(ref);+ return ref;+ }++ private primitive(p: PrimitiveSpec, label: string) {+ const positions = floats(`${label} positions`, p.positions, 3);+ const count = positions.length / 3;+ if (!count) throw new Error(`${label}: no vertices`);+ const attributes: Record<string, number> = {+ POSITION: this.bin.accessor(positions, "VEC3", {+ bounds: true,+ target: "vertex",+ }),+ };+ const vertex = (data: Float32Array, type: "VEC2" | "VEC3" | "VEC4") =>+ this.bin.accessor(data, type, { target: "vertex" });+ if (p.normals)+ attributes.NORMAL = vertex(+ floats(`${label} normals`, p.normals, 3, count),+ "VEC3",+ );+ if (p.tangents)+ attributes.TANGENT = vertex(+ floats(`${label} tangents`, p.tangents, 4, count),+ "VEC4",+ );+ if (p.uvs)+ attributes.TEXCOORD_0 = vertex(+ floats(`${label} uvs`, p.uvs, 2, count),+ "VEC2",+ );+ if (p.uvs1)+ attributes.TEXCOORD_1 = vertex(+ floats(`${label} uvs1`, p.uvs1, 2, count),+ "VEC2",+ );+ if (p.colors)+ attributes.COLOR_0 = vertex(+ floats(`${label} colors`, p.colors, 4, count),+ "VEC4",+ );+ let maxJoint = -1;+ if (p.joints || p.weights) {+ if (!p.joints || !p.weights)+ throw new Error(`${label}: joints and weights go together`);+ const j = floats(`${label} joints`, p.joints, 4, count);+ for (const v of j) {+ if (!Number.isInteger(v) || v < 0)+ throw new Error(`${label}: joint index ${v} is not a whole number`);+ maxJoint = Math.max(maxJoint, v);+ }+ attributes.JOINTS_0 = this.bin.accessor(+ maxJoint < 256 ? Uint8Array.from(j) : Uint16Array.from(j),+ "VEC4",+ { target: "vertex" },+ );+ attributes.WEIGHTS_0 = vertex(+ normalizedWeights(p.weights, count),+ "VEC4",+ );+ }+ const json: Record<string, unknown> = { attributes };+ if (p.indices) {+ const idx = Array.from(p.indices);+ if (!idx.length) throw new Error(`${label}: empty indices`);+ for (const v of idx)+ if (!Number.isInteger(v) || v < 0 || v >= count)+ throw new Error(`${label}: index ${v} outside 0..${count - 1}`);+ json.indices = this.bin.accessor(+ count <= 65535 ? Uint16Array.from(idx) : Uint32Array.from(idx),+ "SCALAR",+ { target: "index" },+ );+ }+ if (p.material) {+ if (p.material.material >= this.materials.length)+ throw new Error(`${label}: unknown material`);+ json.material = p.material.material;+ }+ const mode = MODE[p.mode ?? "triangles"];+ if (mode !== 4) json.mode = mode;+ if (p.targets?.length) {+ json.targets = p.targets.map((t) => {+ const target: Record<string, number> = {+ POSITION: this.bin.accessor(+ floats(`${label} target ${t.name}`, t.positions, 3, count),+ "VEC3",+ { bounds: true, target: "vertex" },+ ),+ };+ if (t.normals)+ target.NORMAL = vertex(+ floats(`${label} target ${t.name} normals`, t.normals, 3, count),+ "VEC3",+ );+ return target;+ });+ }+ return { json, maxJoint };+ }++ addNode(spec: NodeSpec): NodeRef {+ const index = this.nodes.length;+ if (spec.skin && !spec.mesh)+ throw new Error(`node ${spec.name ?? index}: a skin needs a mesh`);+ if (spec.mesh && spec.skin && spec.mesh.maxJoint >= spec.skin.jointCount)+ throw new Error(+ `node ${spec.name ?? index}: mesh uses joint ${spec.mesh.maxJoint} but the skin has ${spec.skin.jointCount}`,+ );+ if (spec.mesh && !spec.skin && spec.mesh.maxJoint >= 0)+ throw new Error(+ `node ${spec.name ?? index}: skinned mesh without a skin`,+ );+ if (spec.rotation) {+ const [x, y, z, w] = spec.rotation;+ if (Math.abs(Math.hypot(x, y, z, w) - 1) > 1e-4)+ throw new Error(+ `node ${spec.name ?? index}: rotation is not a unit quaternion`,+ );+ }+ const children = spec.children?.map((c) => {+ if (this.parents.has(c.node))+ throw new Error(`node ${c.node} already has a parent`);+ this.parents.set(c.node, index);+ return c.node;+ });+ this.nodes.push({+ ...(spec.name ? { name: spec.name } : {}),+ ...(spec.mesh ? { mesh: spec.mesh.mesh } : {}),+ ...(spec.skin ? { skin: spec.skin.skin } : {}),+ ...(spec.translation ? { translation: spec.translation } : {}),+ ...(spec.rotation ? { rotation: spec.rotation } : {}),+ ...(spec.scale ? { scale: spec.scale } : {}),+ ...(children?.length ? { children } : {}),+ ...(spec.extras ? { extras: spec.extras } : {}),+ });+ return { node: index };+ }++ /**+ * The skeleton's joints as a node tree (each translated from its parent,+ * unrotated) and a skin whose inverse bind matrices undo each joint's+ * rest position. Place the tree with `parent`, or leave it at the root.+ */+ addSkeleton(+ skeleton: Skeleton,+ options: { parent?: NodeRef } = {},+ ): SkeletonRefs {+ const joints = new Map<string, NodeRef>();+ const jointIndex = new Map<string, number>();+ const list = skeleton.joints;+ if (!list.length) throw new Error("skeleton: no joints");+ // Children are listed after parents; create nodes leaf-first so+ // `children` refers to nodes that exist.+ const nodeOf = new Map<string, NodeRef>();+ for (let i = list.length - 1; i >= 0; i--) {+ const j = list[i] as (typeof list)[number];+ const children = skeleton+ .children(j.name)+ .map((c) => nodeOf.get(c.name) as NodeRef);+ nodeOf.set(+ j.name,+ this.addNode({+ name: j.name,+ translation: skeleton.local(j.name),+ ...(children.length ? { children } : {}),+ ...(j.extras ? { extras: j.extras } : {}),+ }),+ );+ }+ const ibm = new Float32Array(list.length * 16);+ list.forEach((j, i) => {+ joints.set(j.name, nodeOf.get(j.name) as NodeRef);+ jointIndex.set(j.name, i);+ const o = i * 16;+ ibm[o] = ibm[o + 5] = ibm[o + 10] = ibm[o + 15] = 1;+ ibm[o + 12] = 0 - j.at[0];+ ibm[o + 13] = 0 - j.at[1];+ ibm[o + 14] = 0 - j.at[2];+ });+ const roots = list+ .filter((j) => j.parent === undefined)+ .map((j) => nodeOf.get(j.name) as NodeRef);+ if (options.parent) {+ const parent = this.nodes[options.parent.node] as Record<string, unknown>;+ const kids = (parent.children as number[] | undefined) ?? [];+ for (const r of roots) {+ this.parents.set(r.node, options.parent.node);+ kids.push(r.node);+ }+ parent.children = kids;+ }+ this.skins.push({+ joints: list.map((j) => (nodeOf.get(j.name) as NodeRef).node),+ inverseBindMatrices: this.bin.accessor(ibm, "MAT4"),+ ...(roots.length === 1 ? { skeleton: (roots[0] as NodeRef).node } : {}),+ });+ const skin: SkinRef = {+ skin: this.skins.length - 1,+ jointCount: list.length,+ };+ this.skinRefs.push(skin);+ return { skin, roots, joints, jointIndex };+ }++ addAnimation(spec: AnimationSpec): void {+ if (!spec.channels.length)+ throw new Error(`animation ${spec.name}: no channels`);+ const samplers: Record<string, unknown>[] = [];+ const channels = spec.channels.map((c, i) => {+ const label = `animation ${spec.name} channel ${i}`;+ const node = this.nodes[c.node.node];+ if (!node) throw new Error(`${label}: unknown node`);+ const times = floats(`${label} times`, c.times, 1);+ if (!times.length) throw new Error(`${label}: no keys`);+ for (let k = 1; k < times.length; k++)+ if ((times[k] as number) <= (times[k - 1] as number))+ throw new Error(`${label}: times must increase`);+ if ((times[0] as number) < 0) throw new Error(`${label}: negative time`);+ const size =+ c.path === "weights"+ ? this.targetCount(node, label)+ : PATH_SIZE[c.path];+ const values = floats(`${label} values`, c.values, size, times.length);+ samplers.push({+ input: this.bin.accessor(times, "SCALAR", { bounds: true }),+ output: this.bin.accessor(+ values,+ c.path === "weights" ? "SCALAR" : size === 4 ? "VEC4" : "VEC3",+ ),+ ...(c.interpolation === "step" ? { interpolation: "STEP" } : {}),+ });+ return {+ sampler: samplers.length - 1,+ target: { node: c.node.node, path: c.path },+ };+ });+ this.animations.push({ name: spec.name, channels, samplers });+ }++ private targetCount(node: Record<string, unknown>, label: string): number {+ const mesh = node.mesh as number | undefined;+ const n =+ mesh === undefined ? 0 : (this.meshRefs[mesh]?.targetNames.length ?? 0);+ if (!n)+ throw new Error(`${label}: weights on a node without morph targets`);+ return n;+ }++ private checkTexture(t: TextureRef | { texture: TextureRef }) {+ const ref = typeof t.texture === "number" ? (t as TextureRef) : t.texture;+ if ((ref as TextureRef).texture >= this.textures.length)+ throw new Error("material: unknown texture");+ }++ /** The document's glTF JSON (without the binary chunk), for inspection. */+ json(): Record<string, unknown> {+ const roots = this.nodes+ .map((_, i) => i)+ .filter((i) => !this.parents.has(i));+ const out: Record<string, unknown> = {+ scene: 0,+ scenes: [{ nodes: roots }],+ nodes: this.nodes,+ };+ const put = (key: string, list: unknown[]) => {+ if (list.length) out[key] = list;+ };+ put("meshes", this.meshes);+ put("materials", this.materials);+ put("skins", this.skins);+ put("animations", this.animations);+ put("textures", this.textures);+ put("images", this.images);+ put("samplers", this.samplers);+ if (this.extensions.size) out.extensionsUsed = [...this.extensions].sort();+ return out;+ }++ /** The GLB file. */+ toGlb(): Uint8Array {+ return this.bin.build(this.json());+ }+}diff --git a/packages/forge/src/forge.test.ts b/packages/forge/src/forge.test.tsnew file mode 100644index 00000000..aea6fa98--- /dev/null+++ b/packages/forge/src/forge.test.ts@@ -0,0 +1,279 @@+/**+ * Forge output is standard glTF: the Khronos validator finds no errors or+ * warnings, and the engine's own loader reads back exactly what was written+ * (geometry, 32-bit indices, skins, named morph targets, materials, clips).+ */+import { parseGLB } from "@framefields/webgpu-renderers";+import { validateBytes } from "gltf-validator";+import { describe, expect, it } from "vitest";+import {+ boneGizmo,+ GltfDocument,+ humanoidSkeleton,+ linearRgba,+ Skeleton,+} from "./index.js";++async function validate(glb: Uint8Array) {+ const report = await validateBytes(glb, { maxIssues: 50 });+ const issues = report.issues.messages+ .filter((m: { severity: number }) => m.severity <= 1)+ .map(+ (m: { code: string; message: string; pointer?: string }) =>+ `${m.code} ${m.pointer ?? ""}: ${m.message}`,+ );+ return {+ errors: report.issues.numErrors,+ warnings: report.issues.numWarnings,+ issues,+ };+}++/** A quad strip of `rows` segments along +y, two vertices wide. */+function strip(rows: number, height: number) {+ const positions: number[] = [];+ const normals: number[] = [];+ const indices: number[] = [];+ for (let r = 0; r <= rows; r++) {+ const y = (r / rows) * height;+ positions.push(-5, y, 0, 5, y, 0);+ normals.push(0, 0, 1, 0, 0, 1);+ if (r < rows) {+ const a = r * 2;+ indices.push(a, a + 1, a + 3, a, a + 3, a + 2);+ }+ }+ return { positions, normals, indices };+}++describe("GltfDocument", () => {+ it("writes a skinned, morphing, animated character that validates and loads back", async () => {+ const skel = new Skeleton()+ .add("Root", { at: [0, 0, 0] })+ .add("Mid", { at: [0, 50, 0], parent: "Root" })+ .add("Tip", { at: [0, 100, 0], parent: "Mid" });+ const doc = new GltfDocument();+ const rig = doc.addSkeleton(skel);+ const s = strip(4, 100);+ const n = s.positions.length / 3;+ const joints: number[] = [];+ const weights: number[] = [];+ for (let v = 0; v < n; v++) {+ const y = s.positions[v * 3 + 1] as number;+ joints.push(y < 50 ? 0 : 1, y < 50 ? 1 : 2, 0, 0);+ weights.push(3, 1, 0, 0); // normalized on write+ }+ const bulge = s.positions.map((_, i) =>+ i % 3 === 0 ? s.positions[i]! * 0.5 : 0,+ );+ const material = doc.addMaterial({+ name: "Lacquer",+ color: "#d99a2b",+ roughness: 0.35,+ clearcoat: { factor: 1, roughness: 0.05 },+ sheen: { color: "#ffd9c0", roughness: 0.4 },+ doubleSided: true,+ });+ const mesh = doc.addMesh({+ name: "Strip",+ primitives: [+ {+ ...s,+ joints,+ weights,+ material,+ targets: [+ { name: "wide", positions: bulge },+ { name: "flat", positions: new Array(n * 3).fill(0) },+ ],+ },+ ],+ weights: [0, 0],+ });+ const body = doc.addNode({ name: "Body", mesh, skin: rig.skin });+ const t = [0, 1];+ doc.addAnimation({+ name: "Bend",+ channels: [+ {+ node: rig.joints.get("Mid")!,+ path: "rotation",+ times: t,+ values: [0, 0, 0, 1, 0, 0, Math.SQRT1_2, Math.SQRT1_2],+ },+ { node: body, path: "weights", times: t, values: [0, 0, 1, 0] },+ ],+ });+ const glb = doc.toGlb();++ const report = await validate(glb);+ expect(report.issues).toEqual([]);+ expect(report.errors).toBe(0);++ const model = parseGLB(glb);+ const m = model.meshes[0]!;+ expect(Array.from(m.positions)).toEqual(s.positions);+ expect(Array.from(m.indices)).toEqual(s.indices);+ expect(m.morphTargets).toHaveLength(2);+ expect(Array.from(m.morphTargets![0]!.positions!)).toEqual(bulge);+ expect(Array.from(m.jointWeights!.slice(0, 4))).toEqual([0.75, 0.25, 0, 0]);+ const skin = model.skins![0]!;+ expect(skin.joints.map((j) => model.nodes![j]!.name)).toEqual([+ "Root",+ "Mid",+ "Tip",+ ]);+ // Inverse bind matrices undo each joint's rest position.+ expect(+ Array.from(skin.inverseBindMatrices.slice(16 + 12, 16 + 15)),+ ).toEqual([0, -50, 0]);+ expect(model.nodes![skin.joints[2]!]!.translation).toEqual([0, 50, 0]);+ const clip = model.animations.find((a) => a.name === "Bend")!;+ expect(clip.duration).toBeCloseTo(1);+ expect(clip.tracks.some((k) => k.weights)).toBe(true);+ const mat = Object.values(model.materials)[0]!;+ expect(mat.roughness).toBeCloseTo(0.35);+ expect(mat.twoSided).toBe(true);+ // Factors are linear in the file; the loader hands back display color.+ const [r] = linearRgba("#d99a2b");+ expect(r).toBeLessThan(0.75);+ expect(mat.diffuseColor?.[0]).toBeCloseTo(0xd9 / 255, 2);+ const json = doc.json() as {+ meshes: { extras: { targetNames: string[] } }[];+ extensionsUsed: string[];+ };+ expect(json.meshes[0]!.extras.targetNames).toEqual(["wide", "flat"]);+ expect(json.extensionsUsed).toEqual([+ "KHR_materials_clearcoat",+ "KHR_materials_sheen",+ ]);+ });++ it("switches to 32-bit indices past 65,535 vertices", async () => {+ const s = strip(40000, 1000);+ const doc = new GltfDocument();+ doc.addNode({ mesh: doc.addMesh({ primitives: [s] }) });+ const glb = doc.toGlb();+ expect((await validate(glb)).errors).toBe(0);+ const m = parseGLB(glb).meshes[0]!;+ expect(m.indices).toBeInstanceOf(Uint32Array);+ expect(m.indices[m.indices.length - 1]).toBe(+ s.indices[s.indices.length - 1],+ );+ });++ it("writes every material extension in a valid form", async () => {+ const doc = new GltfDocument();+ const material = doc.addMaterial({+ color: "#ffffffcc",+ alphaMode: "mask",+ alphaCutoff: 0.4,+ emissive: "#ff7a2e",+ emissiveStrength: 6,+ specular: { factor: 0.6, color: "#fff4e0" },+ ior: 1.38,+ transmission: 0.3,+ volume: {+ thickness: 2,+ attenuationColor: "#d99a2b",+ attenuationDistance: 5,+ },+ iridescence: { factor: 0.8, ior: 1.4, thickness: [120, 420] },+ anisotropy: { strength: 0.6, rotation: 90 },+ });+ doc.addNode({+ mesh: doc.addMesh({ primitives: [{ ...strip(1, 10), material }] }),+ });+ const report = await validate(doc.toGlb());+ expect(report.issues).toEqual([]);+ expect(report.errors).toBe(0);+ });++ it("refuses inconsistent input where it is given", () => {+ const doc = new GltfDocument();+ expect(() =>+ doc.addMesh({+ primitives: [{ positions: [0, 0, 0, 1, 0, 0], normals: [0, 0, 1] }],+ }),+ ).toThrow(/1 entries for 2 vertices/);+ expect(() =>+ doc.addMesh({ primitives: [{ ...strip(1, 1), indices: [0, 1, 9] }] }),+ ).toThrow(/outside 0\.\.3/);+ const skinned = doc.addMesh({+ primitives: [+ {+ ...strip(1, 1),+ joints: new Array(16).fill(0).map((_, i) => (i % 4 ? 0 : 3)),+ weights: new Array(16).fill(0).map((_, i) => (i % 4 ? 0 : 1)),+ },+ ],+ });+ const rig = doc.addSkeleton(new Skeleton().add("A", { at: [0, 0, 0] }));+ expect(() => doc.addNode({ mesh: skinned, skin: rig.skin })).toThrow(+ /joint 3 but the skin has 1/,+ );+ expect(() => doc.addMaterial({ roughness: 2 })).toThrow(/outside 0\.\.1/);+ });+});++describe("Skeleton", () => {+ it("mirrors left joints to the right, under mirrored parents", () => {+ const s = new Skeleton()+ .add("Spine", { at: [0, 100, 0] })+ .add("LeftShoulder", { at: [5, 140, 0], parent: "Spine" })+ .add("LeftArm", { at: [20, 140, 0], parent: "LeftShoulder" })+ .mirror();+ expect(s.joint("RightArm").at).toEqual([-20, 140, 0]);+ expect(s.joint("RightArm").parent).toBe("RightShoulder");+ expect(s.joint("RightShoulder").parent).toBe("Spine");+ });++ it("builds a proportioned Mixamo humanoid", () => {+ const s = humanoidSkeleton({ height: 186, heads: 8 });+ for (const name of [+ "Hips",+ "Spine",+ "Spine2",+ "Neck",+ "Head",+ "LeftArm",+ "LeftForeArm",+ "RightHand",+ "RightHandIndex4",+ "LeftHandThumb4",+ "LeftUpLeg",+ "RightLeg",+ "RightToe_End",+ ])+ expect(s.has(name), name).toBe(true);+ expect(s.joint("HeadTop_End").at[1]).toBeCloseTo(186);+ expect(Math.min(...s.joints.map((j) => j.at[1]))).toBeGreaterThanOrEqual(0);+ // Symmetric, and arm spans scale with the figure.+ expect(s.joint("RightForeArm").at[0]).toBeCloseTo(+ -s.joint("LeftForeArm").at[0],+ );+ expect(s.length("LeftForeArm")).toBeCloseTo(0.186 * 186, 5);+ // A T-pose holds the arms level.+ const t = humanoidSkeleton({ armAngle: 0 });+ expect(t.joint("LeftHand").at[1]).toBeCloseTo(t.joint("LeftArm").at[1]);+ expect(+ humanoidSkeleton({ prefix: "mixamorig:" }).has("mixamorig:LeftArm"),+ ).toBe(true);+ });++ it("writes a skeleton with its bone gizmo as a valid skinned GLB", async () => {+ const skeleton = humanoidSkeleton();+ const doc = new GltfDocument();+ const rig = doc.addSkeleton(skeleton);+ const bones = doc.addMesh({+ name: "Bones",+ primitives: [boneGizmo(skeleton)],+ });+ doc.addNode({ name: "Gizmo", mesh: bones, skin: rig.skin });+ const glb = doc.toGlb();+ const report = await validate(glb);+ expect(report.issues).toEqual([]);+ const model = parseGLB(glb);+ expect(model.skins![0]!.joints).toHaveLength(skeleton.joints.length);+ });+});diff --git a/packages/forge/src/gizmo.ts b/packages/forge/src/gizmo.tsnew file mode 100644index 00000000..ea9137fc--- /dev/null+++ b/packages/forge/src/gizmo.ts@@ -0,0 +1,84 @@+/**+ * A skeleton drawn as octahedral bones, each rigidly skinned to its joint:+ * the classic DCC bone display, for checking a rig's rest pose and its+ * motion before there is a body.+ */+import type { PrimitiveSpec } from "./document.js";+import type { Skeleton, Vec3 } from "./skeleton.js";++const sub = (a: Vec3, b: Vec3): Vec3 => [a[0] - b[0], a[1] - b[1], a[2] - b[2]];+const cross = (a: Vec3, b: Vec3): Vec3 => [+ a[1] * b[2] - a[2] * b[1],+ a[2] * b[0] - a[0] * b[2],+ a[0] * b[1] - a[1] * b[0],+];+const norm = (v: Vec3): Vec3 => {+ const l = Math.hypot(v[0], v[1], v[2]) || 1;+ return [v[0] / l, v[1] / l, v[2] / l];+};++export interface BoneGizmoOptions {+ /** Bone width as a fraction of its length. Default 0.12. */+ width?: number;+}++/**+ * One octahedron per parent → child bone (flat-shaded), weighted fully to+ * the parent joint; skin indices are the skeleton's joint order, as+ * `GltfDocument.addSkeleton` writes them.+ */+export function boneGizmo(+ skeleton: Skeleton,+ options: BoneGizmoOptions = {},+): Required<+ Pick<PrimitiveSpec, "positions" | "normals" | "joints" | "weights">+> {+ const width = options.width ?? 0.12;+ const positions: number[] = [];+ const normals: number[] = [];+ const joints: number[] = [];+ const weights: number[] = [];+ for (const child of skeleton.joints) {+ if (child.parent === undefined) continue;+ const parent = skeleton.joint(child.parent);+ const axis = sub(child.at, parent.at);+ const len = Math.hypot(axis[0], axis[1], axis[2]);+ if (len < 1e-6) continue;+ const d = norm(axis);+ const helper: Vec3 = Math.abs(d[1]) < 0.9 ? [0, 1, 0] : [1, 0, 0];+ const u = norm(cross(d, helper));+ const v = cross(d, u);+ const r = len * width;+ const ring = [0, 1, 2, 3].map((k): Vec3 => {+ const a = (k * Math.PI) / 2;+ return [+ parent.at[0] ++ d[0] * len * 0.15 ++ (u[0] * Math.cos(a) + v[0] * Math.sin(a)) * r,+ parent.at[1] ++ d[1] * len * 0.15 ++ (u[1] * Math.cos(a) + v[1] * Math.sin(a)) * r,+ parent.at[2] ++ d[2] * len * 0.15 ++ (u[2] * Math.cos(a) + v[2] * Math.sin(a)) * r,+ ];+ });+ for (let k = 0; k < 4; k++) {+ const a = ring[k] as Vec3;+ const b = ring[(k + 1) % 4] as Vec3;+ for (const [p, q, s] of [+ [parent.at, b, a],+ [child.at, a, b],+ ] as [Vec3, Vec3, Vec3][]) {+ const n = norm(cross(sub(q, p), sub(s, p)));+ for (const pt of [p, q, s]) {+ positions.push(...pt);+ normals.push(...n);+ joints.push(parent.index, 0, 0, 0);+ weights.push(1, 0, 0, 0);+ }+ }+ }+ }+ return { positions, normals, joints, weights };+}diff --git a/packages/forge/src/glb-builder.ts b/packages/forge/src/glb-builder.tsnew file mode 100644index 00000000..b2051498--- /dev/null+++ b/packages/forge/src/glb-builder.ts@@ -0,0 +1,175 @@+/**+ * The binary layer of a GLB: typed arrays become 4-byte-aligned buffer+ * views and accessors in one BIN chunk, and `build` wraps a glTF JSON+ * document around them. `GltfDocument` is the typed layer on top; use this+ * directly only to hand-write unusual files.+ */++export type AccessorType = "SCALAR" | "VEC2" | "VEC3" | "VEC4" | "MAT4";++export type AccessorData =+ | Float32Array+ | Uint32Array+ | Uint16Array+ | Uint8Array+ | Int16Array+ | Int8Array;++export interface AccessorOptions {+ /** Integers read as 0..1 (unsigned) or -1..1 (signed). */+ normalized?: boolean;+ /** Write per-component `min` / `max` (required for POSITION). */+ bounds?: boolean;+ /** The buffer view's GPU target: vertex attributes or indices. */+ target?: "vertex" | "index";+}++const COMPONENTS: Record<AccessorType, number> = {+ SCALAR: 1,+ VEC2: 2,+ VEC3: 3,+ VEC4: 4,+ MAT4: 16,+};++function componentType(data: AccessorData): number {+ if (data instanceof Float32Array) return 5126;+ if (data instanceof Uint32Array) return 5125;+ if (data instanceof Uint16Array) return 5123;+ if (data instanceof Int16Array) return 5122;+ if (data instanceof Uint8Array) return 5121;+ return 5120;+}++/** Per-component min and max over an interleaved array. */+function bounds(data: AccessorData, size: number) {+ const min = new Array<number>(size).fill(Number.POSITIVE_INFINITY);+ const max = new Array<number>(size).fill(Number.NEGATIVE_INFINITY);+ for (let i = 0; i < data.length; i++) {+ const c = i % size;+ const v = data[i] as number;+ if (v < (min[c] as number)) min[c] = v;+ if (v > (max[c] as number)) max[c] = v;+ }+ return { min, max };+}++const align4 = (n: number) => (4 - (n % 4)) % 4;++/** Accumulates typed arrays into one binary buffer and the accessors that view them. */+export class GlbBuilder {+ private chunks: Uint8Array[] = [];+ private length = 0;+ readonly bufferViews: Record<string, unknown>[] = [];+ readonly accessors: Record<string, unknown>[] = [];++ private view(data: Uint8Array, target?: number): number {+ const pad = align4(this.length);+ if (pad) {+ this.chunks.push(new Uint8Array(pad));+ this.length += pad;+ }+ this.bufferViews.push({+ buffer: 0,+ byteOffset: this.length,+ byteLength: data.byteLength,+ ...(target ? { target } : {}),+ });+ this.chunks.push(data);+ this.length += data.byteLength;+ return this.bufferViews.length - 1;+ }++ /** An accessor over `data`, `type` components per element. Returns its index. */+ accessor(+ data: AccessorData,+ type: AccessorType,+ options: AccessorOptions = {},+ ): number {+ const size = COMPONENTS[type];+ if (data.length === 0 || data.length % size !== 0)+ throw new Error(+ `accessor: ${data.length} components is not a whole number of ${type}`,+ );+ const bytes = new Uint8Array(data.buffer, data.byteOffset, data.byteLength);+ const target =+ options.target === "vertex"+ ? 34962+ : options.target === "index"+ ? 34963+ : undefined;+ const accessor: Record<string, unknown> = {+ bufferView: this.view(bytes, target),+ componentType: componentType(data),+ count: data.length / size,+ type,+ };+ if (options.normalized) accessor.normalized = true;+ if (options.bounds) Object.assign(accessor, bounds(data, size));+ this.accessors.push(accessor);+ return this.accessors.length - 1;+ }++ /** Raw bytes (an image) as a buffer view. Returns its index. */+ image(bytes: Uint8Array): number {+ return this.view(bytes);+ }++ /** The GLB file: `json` plus the buffers, views and accessors collected here. */+ build(json: Record<string, unknown>): Uint8Array {+ const bin = concat(this.chunks, this.length);+ const doc = new TextEncoder().encode(+ JSON.stringify({+ asset: { version: "2.0", generator: "framefields forge" },+ ...json,+ ...(bin.byteLength+ ? {+ buffers: [{ byteLength: bin.byteLength }],+ bufferViews: this.bufferViews,+ accessors: this.accessors,+ }+ : {}),+ }),+ );+ const jsonChunk = padded(doc, 0x20);+ const binChunk = padded(bin, 0);+ const total =+ 12 ++ 8 ++ jsonChunk.byteLength ++ (bin.byteLength ? 8 + binChunk.byteLength : 0);+ const out = new Uint8Array(total);+ const dv = new DataView(out.buffer);+ dv.setUint32(0, 0x46546c67, true);+ dv.setUint32(4, 2, true);+ dv.setUint32(8, total, true);+ let at = 12;+ const chunk = (data: Uint8Array, type: number) => {+ dv.setUint32(at, data.byteLength, true);+ dv.setUint32(at + 4, type, true);+ out.set(data, at + 8);+ at += 8 + data.byteLength;+ };+ chunk(jsonChunk, 0x4e4f534a);+ if (bin.byteLength) chunk(binChunk, 0x004e4942);+ return out;+ }+}++function concat(chunks: Uint8Array[], length: number): Uint8Array {+ const out = new Uint8Array(length);+ let at = 0;+ for (const c of chunks) {+ out.set(c, at);+ at += c.byteLength;+ }+ return out;+}++function padded(data: Uint8Array, fill: number): Uint8Array {+ const pad = align4(data.byteLength);+ if (!pad) return data;+ const out = new Uint8Array(data.byteLength + pad).fill(fill);+ out.set(data);+ return out;+}diff --git a/packages/forge/src/gltf-validator.d.ts b/packages/forge/src/gltf-validator.d.tsnew file mode 100644index 00000000..5116ab64--- /dev/null+++ b/packages/forge/src/gltf-validator.d.ts@@ -0,0 +1,21 @@+/** The subset of the Khronos glTF validator's API the tests use. */+declare module "gltf-validator" {+ export interface ValidationMessage {+ code: string;+ message: string;+ /** 0 error, 1 warning, 2 info, 3 hint. */+ severity: number;+ pointer?: string;+ }+ export interface ValidationReport {+ issues: {+ numErrors: number;+ numWarnings: number;+ messages: ValidationMessage[];+ };+ }+ export function validateBytes(+ data: Uint8Array,+ options?: { maxIssues?: number; uri?: string },+ ): Promise<ValidationReport>;+}diff --git a/packages/forge/src/humanoid.ts b/packages/forge/src/humanoid.tsnew file mode 100644index 00000000..388ebe49--- /dev/null+++ b/packages/forge/src/humanoid.ts@@ -0,0 +1,134 @@+/**+ * A proportioned human skeleton on Mixamo joint names, so clips, `pose`,+ * `aim` and pose retargeting (`rig: "mixamo"`) drive it unchanged.+ * Proportions are classical figure-drawing ratios measured in head+ * heights; stylized characters change `heads` (8 is heroic, 6 is stocky)+ * and the width multipliers.+ */+import { Skeleton, type Vec3 } from "./skeleton.js";++export interface HumanoidOptions {+ /** Standing height, top of the head to the soles, model units. Default 180. */+ height?: number;+ /** Height in head heights: 7.5 natural, 8 heroic, 6 stocky. Default 7.5. */+ heads?: number;+ /** Multiplies the shoulder width. Default 1. */+ shoulderWidth?: number;+ /** Multiplies the hip width. Default 1. */+ hipWidth?: number;+ /** Arms' angle below horizontal at rest: 0 is a T-pose. Default 45 (an A-pose, which skins better). */+ armAngle?: number;+ /** Thumb and four fingers, three bones each. Default true. */+ fingers?: boolean;+ /** Toe joints. Default true. */+ toes?: boolean;+ /** Joint name prefix, e.g. "mixamorig:". Default none. */+ prefix?: string;+}++const add = (a: Vec3, b: Vec3, s = 1): Vec3 => [+ a[0] + b[0] * s,+ a[1] + b[1] * s,+ a[2] + b[2] * s,+];+const unit = (v: Vec3): Vec3 => {+ const l = Math.hypot(v[0], v[1], v[2]);+ return [v[0] / l, v[1] / l, v[2] / l];+};++/** Bone lengths of each finger as fractions of the hand, and its spread across the palm (fraction of height, +z is the thumb side). */+const FINGERS = [+ { name: "Index", spread: 0.012, lengths: [0.21, 0.13, 0.1] },+ { name: "Middle", spread: 0.004, lengths: [0.23, 0.15, 0.11] },+ { name: "Ring", spread: -0.004, lengths: [0.21, 0.14, 0.1] },+ { name: "Pinky", spread: -0.012, lengths: [0.17, 0.11, 0.09] },+] as const;++/**+ * A humanoid skeleton standing on y = 0, facing +z, its left at +x. Joint+ * names follow Mixamo: Hips, Spine, Spine1, Spine2, Neck, Head,+ * HeadTop_End; Left/Right Shoulder, Arm, ForeArm, Hand, HandThumb1–4,+ * HandIndex1–4 …; UpLeg, Leg, Foot, ToeBase, Toe_End.+ */+export function humanoidSkeleton(options: HumanoidOptions = {}): Skeleton {+ const H = options.height ?? 180;+ const heads = options.heads ?? 7.5;+ const sw = options.shoulderWidth ?? 1;+ const hw = options.hipWidth ?? 1;+ const p = options.prefix ?? "";+ const n = (name: string) => `${p}${name}`;+ const head = H / heads;+ const k = (f: number) => f * H;++ const s = new Skeleton();+ s.add(n("Hips"), { at: [0, k(0.53), 0] });+ s.add(n("Spine"), { at: [0, k(0.585), -k(0.004)], parent: n("Hips") });+ s.add(n("Spine1"), { at: [0, k(0.645), -k(0.008)], parent: n("Spine") });+ s.add(n("Spine2"), { at: [0, k(0.715), -k(0.008)], parent: n("Spine1") });+ s.add(n("Neck"), {+ at: [0, H - 1.25 * head, -k(0.006)],+ parent: n("Spine2"),+ });+ s.add(n("Head"), { at: [0, H - 0.78 * head, k(0.004)], parent: n("Neck") });+ s.add(n("HeadTop_End"), { at: [0, H, k(0.004)], parent: n("Head") });++ // Arms hang from the shoulder line, about 1.45 heads below the crown.+ const shoulderY = H - 1.45 * head;+ const angle = ((options.armAngle ?? 45) * Math.PI) / 180;+ const down: Vec3 = [Math.cos(angle), -Math.sin(angle), 0];+ const arm: Vec3 = [k(0.105) * sw, shoulderY, -k(0.006)];+ const elbow = add(arm, down, k(0.186));+ const wrist = add(elbow, down, k(0.146));+ const hand = k(0.108);+ s.add(n("LeftShoulder"), {+ at: [k(0.022) * sw, shoulderY + k(0.012), -k(0.006)],+ parent: n("Spine2"),+ });+ s.add(n("LeftArm"), { at: arm, parent: n("LeftShoulder") });+ s.add(n("LeftForeArm"), { at: elbow, parent: n("LeftArm") });+ s.add(n("LeftHand"), { at: wrist, parent: n("LeftForeArm") });++ if (options.fingers ?? true) {+ const knuckles = add(wrist, down, 0.45 * hand);+ for (const f of FINGERS) {+ let at = add(knuckles, [0, 0, 1], k(f.spread));+ const points: Vec3[] = [at];+ for (const len of f.lengths) {+ at = add(at, down, len * hand);+ points.push(at);+ }+ s.chain(n(`LeftHand${f.name}`), points, { parent: n("LeftHand") });+ }+ // The thumb leaves the palm forward and down, across the index.+ const thumbDir = unit(add(add(down, [0, 0, 1], 0.9), [0, -1, 0], 0.25));+ let at = add(add(wrist, down, 0.12 * hand), [0, 0, 1], k(0.012));+ const thumb: Vec3[] = [at];+ for (const len of [0.2, 0.17, 0.13]) {+ at = add(at, thumbDir, len * hand);+ thumb.push(at);+ }+ s.chain(n("LeftHandThumb"), thumb, { parent: n("LeftHand") });+ }++ const hipX = k(0.09) * hw;+ s.add(n("LeftUpLeg"), { at: [hipX, k(0.5), 0], parent: n("Hips") });+ s.add(n("LeftLeg"), {+ at: [hipX * 0.95, k(0.282), k(0.008)],+ parent: n("LeftUpLeg"),+ });+ s.add(n("LeftFoot"), {+ at: [hipX * 0.92, k(0.045), -k(0.01)],+ parent: n("LeftLeg"),+ });+ if (options.toes ?? true) {+ s.add(n("LeftToeBase"), {+ at: [hipX, k(0.012), k(0.095)],+ parent: n("LeftFoot"),+ });+ s.add(n("LeftToe_End"), {+ at: [hipX, k(0.012), k(0.145)],+ parent: n("LeftToeBase"),+ });+ }+ return s.mirror({ from: "Left", to: "Right" });+}diff --git a/packages/forge/src/index.ts b/packages/forge/src/index.tsnew file mode 100644index 00000000..f15b1765--- /dev/null+++ b/packages/forge/src/index.ts@@ -0,0 +1,47 @@+/**+ * `framefields/forge`: rigged characters and props built in code, written+ * as standard GLB files that `Layer.glb` (and every other glTF tool) loads.+ */+export {+ type ColorInput,+ displayRgba,+ linearRgb,+ linearRgba,+} from "./color.js";+export {+ type AnimationPath,+ type AnimationSpec,+ type ChannelSpec,+ GltfDocument,+ type MaterialRef,+ type MeshRef,+ type MeshSpec,+ type MorphTargetSpec,+ type NodeRef,+ type NodeSpec,+ type PrimitiveSpec,+ type SkeletonRefs,+ type SkinRef,+ type TextureSpec,+} from "./document.js";+export { type BoneGizmoOptions, boneGizmo } from "./gizmo.js";+export {+ type AccessorData,+ type AccessorOptions,+ type AccessorType,+ GlbBuilder,+} from "./glb-builder.js";+export { type HumanoidOptions, humanoidSkeleton } from "./humanoid.js";+export {+ type MaterialSpec,+ materialJson,+ type TextureRef,+ type TextureSlot,+} from "./material.js";+export {+ type Joint,+ type JointSpec,+ type MirrorOptions,+ Skeleton,+ type Vec3,+} from "./skeleton.js";diff --git a/packages/forge/src/material.ts b/packages/forge/src/material.tsnew file mode 100644index 00000000..c78f11b8--- /dev/null+++ b/packages/forge/src/material.ts@@ -0,0 +1,229 @@+/**+ * Materials in the words of a shading artist, written as glTF 2.0 PBR+ * metallic-roughness plus the Khronos extensions for clearcoat, sheen,+ * specular, IOR, transmission, volume, iridescence, anisotropy, emissive+ * strength and unlit.+ */+import { type ColorInput, linearRgb, linearRgba } from "./color.js";++/** A texture added to the document (`GltfDocument.addTexture`). */+export interface TextureRef {+ readonly texture: number;+}++export interface TextureSlot {+ texture: TextureRef;+ /** UV set. Default 0. */+ texCoord?: number;+}++export interface MaterialSpec {+ name?: string;+ /** Base color (display sRGB; alpha from #rrggbbaa or a 4th component). Default white. */+ color?: ColorInput;+ colorTexture?: TextureRef | TextureSlot;+ /** 0..1. Default 0 (glTF's default of 1 is rarely what a sculpt wants). */+ metallic?: number;+ /** 0..1. Default 0.5. */+ roughness?: number;+ /** Green = roughness, blue = metallic, as glTF packs them. */+ metallicRoughnessTexture?: TextureRef | TextureSlot;+ normalTexture?: TextureRef | (TextureSlot & { scale?: number });+ occlusionTexture?: TextureRef | (TextureSlot & { strength?: number });+ /** Emitted color (display sRGB). */+ emissive?: ColorInput;+ /** Multiplies `emissive` past 1 for bloom (KHR_materials_emissive_strength). */+ emissiveStrength?: number;+ emissiveTexture?: TextureRef | TextureSlot;+ /** Default "opaque". "mask" cuts out below `alphaCutoff` (hair cards, frayed hems). */+ alphaMode?: "opaque" | "mask" | "blend";+ /** Default 0.5. */+ alphaCutoff?: number;+ /** Render back faces (cloaks, leaves, hair cards). */+ doubleSided?: boolean;+ /** No lighting (KHR_materials_unlit). */+ unlit?: boolean;+ /** Lacquer layer: strength 0..1, or with its own roughness. */+ clearcoat?: number | { factor: number; roughness?: number };+ /** Cloth and fur retroreflection. */+ sheen?: { color: ColorInput; roughness?: number };+ /** Dielectric specular strength and tint. */+ specular?: { factor?: number; color?: ColorInput };+ /** Index of refraction. Default 1.5. */+ ior?: number;+ /** Light passing through thin surfaces, 0..1. */+ transmission?: number;+ /** Thick transmissive volume (eye lens, amber, ice). */+ volume?: {+ thickness: number;+ attenuationColor?: ColorInput;+ attenuationDistance?: number;+ };+ /** Thin-film interference: strength 0..1, or with film IOR and thickness range (nm). */+ iridescence?:+ | number+ | { factor: number; ior?: number; thickness?: [number, number] };+ /** Brushed / combed highlights: strength 0..1, rotation in degrees. */+ anisotropy?: { strength: number; rotation?: number };+ /** Free data for other tools (glTF `extras`). */+ extras?: Record<string, unknown>;+}++export interface MaterialJson {+ json: Record<string, unknown>;+ extensions: string[];+}++const slot = (t: TextureRef | TextureSlot): TextureSlot =>+ "texture" in t && typeof t.texture === "object"+ ? (t as TextureSlot)+ : { texture: t as TextureRef };++function textureInfo(+ t: TextureRef | TextureSlot,+ extra: Record<string, number> = {},+): Record<string, number> {+ const s = slot(t);+ return {+ index: s.texture.texture,+ ...(s.texCoord ? { texCoord: s.texCoord } : {}),+ ...extra,+ };+}++function check(name: string, v: number | undefined, lo = 0, hi = 1) {+ if (v === undefined) return;+ if (!Number.isFinite(v) || v < lo || v > hi)+ throw new Error(`material: ${name} ${v} is outside ${lo}..${hi}`);+}++/** The material's glTF JSON and the extensions it uses. */+export function materialJson(m: MaterialSpec): MaterialJson {+ check("metallic", m.metallic);+ check("roughness", m.roughness);+ check("alphaCutoff", m.alphaCutoff);+ check("transmission", m.transmission);+ check("ior", m.ior, 1, 4);+ check("emissiveStrength", m.emissiveStrength, 0, 1e6);++ const pbr: Record<string, unknown> = {+ baseColorFactor: linearRgba(m.color ?? "#ffffff"),+ metallicFactor: m.metallic ?? 0,+ roughnessFactor: m.roughness ?? 0.5,+ };+ if (m.colorTexture) pbr.baseColorTexture = textureInfo(m.colorTexture);+ if (m.metallicRoughnessTexture)+ pbr.metallicRoughnessTexture = textureInfo(m.metallicRoughnessTexture);++ const json: Record<string, unknown> = { pbrMetallicRoughness: pbr };+ if (m.name) json.name = m.name;+ if (m.normalTexture) {+ const scale =+ "scale" in m.normalTexture ? m.normalTexture.scale : undefined;+ json.normalTexture = textureInfo(+ m.normalTexture,+ scale === undefined ? {} : { scale },+ );+ }+ if (m.occlusionTexture) {+ const strength =+ "strength" in m.occlusionTexture+ ? m.occlusionTexture.strength+ : undefined;+ json.occlusionTexture = textureInfo(+ m.occlusionTexture,+ strength === undefined ? {} : { strength },+ );+ }+ if (m.emissive) json.emissiveFactor = linearRgb(m.emissive);+ if (m.emissiveTexture) {+ json.emissiveTexture = textureInfo(m.emissiveTexture);+ json.emissiveFactor ??= [1, 1, 1];+ }+ if (m.alphaMode && m.alphaMode !== "opaque") {+ json.alphaMode = m.alphaMode.toUpperCase();+ if (m.alphaMode === "mask" && m.alphaCutoff !== undefined)+ json.alphaCutoff = m.alphaCutoff;+ }+ if (m.doubleSided) json.doubleSided = true;+ if (m.extras) json.extras = m.extras;++ const ext: Record<string, Record<string, unknown>> = {};+ if (m.unlit) ext.KHR_materials_unlit = {};+ if (m.emissiveStrength !== undefined && m.emissiveStrength !== 1)+ ext.KHR_materials_emissive_strength = {+ emissiveStrength: m.emissiveStrength,+ };+ if (m.clearcoat !== undefined) {+ const c =+ typeof m.clearcoat === "number" ? { factor: m.clearcoat } : m.clearcoat;+ check("clearcoat", c.factor);+ check("clearcoat roughness", c.roughness);+ ext.KHR_materials_clearcoat = {+ clearcoatFactor: c.factor,+ ...(c.roughness === undefined+ ? {}+ : { clearcoatRoughnessFactor: c.roughness }),+ };+ }+ if (m.sheen) {+ check("sheen roughness", m.sheen.roughness);+ ext.KHR_materials_sheen = {+ sheenColorFactor: linearRgb(m.sheen.color),+ sheenRoughnessFactor: m.sheen.roughness ?? 0.5,+ };+ }+ if (m.specular) {+ check("specular", m.specular.factor);+ ext.KHR_materials_specular = {+ ...(m.specular.factor === undefined+ ? {}+ : { specularFactor: m.specular.factor }),+ ...(m.specular.color+ ? { specularColorFactor: linearRgb(m.specular.color) }+ : {}),+ };+ }+ if (m.ior !== undefined) ext.KHR_materials_ior = { ior: m.ior };+ if (m.transmission !== undefined)+ ext.KHR_materials_transmission = { transmissionFactor: m.transmission };+ if (m.volume) {+ ext.KHR_materials_volume = {+ thicknessFactor: m.volume.thickness,+ ...(m.volume.attenuationColor+ ? { attenuationColor: linearRgb(m.volume.attenuationColor) }+ : {}),+ ...(m.volume.attenuationDistance === undefined+ ? {}+ : { attenuationDistance: m.volume.attenuationDistance }),+ };+ }+ if (m.iridescence !== undefined) {+ const i =+ typeof m.iridescence === "number"+ ? { factor: m.iridescence }+ : m.iridescence;+ check("iridescence", i.factor);+ ext.KHR_materials_iridescence = {+ iridescenceFactor: i.factor,+ ...(i.ior === undefined ? {} : { iridescenceIor: i.ior }),+ ...(i.thickness+ ? {+ iridescenceThicknessMinimum: i.thickness[0],+ iridescenceThicknessMaximum: i.thickness[1],+ }+ : {}),+ };+ }+ if (m.anisotropy) {+ check("anisotropy", m.anisotropy.strength);+ ext.KHR_materials_anisotropy = {+ anisotropyStrength: m.anisotropy.strength,+ ...(m.anisotropy.rotation+ ? { anisotropyRotation: (m.anisotropy.rotation * Math.PI) / 180 }+ : {}),+ };+ }+ if (Object.keys(ext).length) json.extensions = ext;+ return { json, extensions: Object.keys(ext) };+}diff --git a/packages/forge/src/skeleton.ts b/packages/forge/src/skeleton.tsnew file mode 100644index 00000000..69b0dc70--- /dev/null+++ b/packages/forge/src/skeleton.ts@@ -0,0 +1,155 @@+/**+ * Skeletons for rigged characters: joints placed at their rest positions in+ * model space (glTF: +y up, the character faces +z, its left side at +x),+ * built bone by bone or from a proportioned preset, with left/right+ * mirroring. Joints rest unrotated, so a joint's local axes are the model's:+ * `pose: { LeftArm: { rotateZ: … } }` turns about the same axis on every+ * joint, and `aim` / retargeting find a bone's direction from its child.+ */++export type Vec3 = [number, number, number];++export interface JointSpec {+ name: string;+ /** Rest position in model space. */+ at: Vec3;+ parent?: string;+ /** Free data written to the joint's glTF node `extras`. */+ extras?: Record<string, unknown>;+}++export interface Joint extends JointSpec {+ readonly index: number;+}++export interface MirrorOptions {+ /** Name fragment of the joints to copy. Default "Left". */+ from?: string;+ /** What it becomes on the copies. Default "Right". */+ to?: string;+ /** Model axis to reflect across. Default "x" (left ↔ right). */+ axis?: "x" | "y" | "z";+}++const AXIS = { x: 0, y: 1, z: 2 } as const;++export class Skeleton {+ private readonly list: Joint[] = [];+ private readonly byName = new Map<string, Joint>();++ /** Adds a joint; a parent must already exist. */+ add(name: string, spec: Omit<JointSpec, "name">): this {+ if (this.byName.has(name))+ throw new Error(`skeleton: joint "${name}" already exists`);+ if (spec.parent !== undefined && !this.byName.has(spec.parent))+ throw new Error(+ `skeleton: joint "${name}" has unknown parent "${spec.parent}"`,+ );+ if (!spec.at.every(Number.isFinite))+ throw new Error(`skeleton: joint "${name}" has a non-finite position`);+ const joint: Joint = {+ name,+ at: [spec.at[0], spec.at[1], spec.at[2]],+ ...(spec.parent === undefined ? {} : { parent: spec.parent }),+ ...(spec.extras ? { extras: spec.extras } : {}),+ index: this.list.length,+ };+ this.list.push(joint);+ this.byName.set(name, joint);+ return this;+ }++ /**+ * A chain of joints through `points`, named `${base}1`, `${base}2`, …+ * (tails, braids, cloak columns, fingers). Returns the names, root first.+ */+ chain(+ base: string,+ points: readonly Vec3[],+ options: { parent?: string; start?: number } = {},+ ): string[] {+ const start = options.start ?? 1;+ let parent = options.parent;+ return points.map((at, i) => {+ const name = `${base}${start + i}`;+ this.add(name, { at, ...(parent === undefined ? {} : { parent }) });+ parent = name;+ return name;+ });+ }++ /**+ * Copies every joint whose name contains `from` (default "Left") across+ * the `axis` (default x), renamed with `to` (default "Right"). A copy's+ * parent is the mirrored parent where there is one, else the same parent+ * (`LeftShoulder` under `Spine2` becomes `RightShoulder` under `Spine2`).+ */+ mirror(options: MirrorOptions = {}): this {+ const from = options.from ?? "Left";+ const to = options.to ?? "Right";+ const axis = AXIS[options.axis ?? "x"];+ const sources = this.list.filter((j) => j.name.includes(from));+ if (!sources.length)+ throw new Error(`skeleton: no joint names contain "${from}" to mirror`);+ const rename = (n: string) => n.split(from).join(to);+ for (const j of sources) {+ const at: Vec3 = [j.at[0], j.at[1], j.at[2]];+ at[axis] = -at[axis];+ const parent =+ j.parent === undefined+ ? undefined+ : j.parent.includes(from)+ ? rename(j.parent)+ : j.parent;+ this.add(rename(j.name), {+ at,+ ...(parent === undefined ? {} : { parent }),+ ...(j.extras ? { extras: j.extras } : {}),+ });+ }+ return this;+ }++ /** Joints in order: every parent before its children. */+ get joints(): readonly Joint[] {+ return this.list;+ }++ has(name: string): boolean {+ return this.byName.has(name);+ }++ joint(name: string): Joint {+ const j = this.byName.get(name);+ if (!j) throw new Error(`skeleton: no joint "${name}"`);+ return j;+ }++ children(name: string): Joint[] {+ return this.list.filter((j) => j.parent === name);+ }++ /** The rest position relative to the parent (the glTF node translation). */+ local(name: string): Vec3 {+ const j = this.joint(name);+ if (j.parent === undefined) return [...j.at];+ const p = this.joint(j.parent).at;+ return [j.at[0] - p[0], j.at[1] - p[1], j.at[2] - p[2]];+ }++ /** Distance from a joint to its parent. */+ length(name: string): number {+ const [x, y, z] = this.local(name);+ return this.joint(name).parent === undefined ? 0 : Math.hypot(x, y, z);+ }++ /** Every joint moved by `offset` (e.g. to stand the feet on y = 0). */+ translate(offset: Vec3): this {+ for (const j of this.list) {+ j.at[0] += offset[0];+ j.at[1] += offset[1];+ j.at[2] += offset[2];+ }+ return this;+ }+}diff --git a/packages/forge/tsconfig.json b/packages/forge/tsconfig.jsonnew file mode 100644index 00000000..c00e3e04--- /dev/null+++ b/packages/forge/tsconfig.json@@ -0,0 +1,9 @@+{+ "extends": "@framefields/tsconfig/base.json",+ "compilerOptions": {+ "lib": ["ES2022", "ESNext"],+ "types": ["node"]+ },+ "include": ["src"],+ "exclude": ["dist", "node_modules"]+}diff --git a/packages/forge/tsdown.config.ts b/packages/forge/tsdown.config.tsnew file mode 100644index 00000000..94257a01--- /dev/null+++ b/packages/forge/tsdown.config.ts@@ -0,0 +1,10 @@+import { defineConfig } from "tsdown";++export default defineConfig({+ entry: ["src/index.ts"],+ format: ["esm"],+ dts: true,+ clean: true,+ sourcemap: true,+ treeshake: true,+});diff --git a/packages/framefields/package.json b/packages/framefields/package.jsonindex 700ae74d..b61d3a5d 100644--- a/packages/framefields/package.json+++ b/packages/framefields/package.json@@ -93,6 +93,12 @@"import": "./dist/storage/index.mjs","default": "./dist/storage/index.mjs"},+ "./forge": {+ "types": "./dist/forge/index.d.mts",+ "development": "./src/forge/index.ts",+ "import": "./dist/forge/index.mjs",+ "default": "./dist/forge/index.mjs"+ },"./sam": {"types": "./dist/sam/index.d.mts","development": "./src/sam/index.ts",@@ -198,6 +204,7 @@"@framefields/node-vision": "workspace:*","@framefields/renderer": "workspace:*","@framefields/renderers": "workspace:*",+ "@framefields/forge": "workspace:*","@framefields/storage": "workspace:*","@framefields/tensor-webgpu": "workspace:*","@framefields/tsconfig": "workspace:*",diff --git a/packages/framefields/src/character-render.test.ts b/packages/framefields/src/character-render.test.tsindex b12bdcbb..19eca272 100644--- a/packages/framefields/src/character-render.test.ts+++ b/packages/framefields/src/character-render.test.ts@@ -8,6 +8,7 @@import fs from "node:fs/promises";import os from "node:os";import path from "node:path";+import { GlbBuilder } from "@framefields/forge";import { afterAll, beforeAll, describe, expect, it } from "vitest";import {Composition,@@ -19,7 +20,6 @@ import {retargetPose,Signal,} from "./index.js";-import { GlbBuilder } from "./test-utils/glb.js";import { decodePngPixels } from "./test-utils/pixels.js";const SIZE = 400;diff --git a/packages/framefields/src/forge-render.test.ts b/packages/framefields/src/forge-render.test.tsnew file mode 100644index 00000000..7a4e27e5--- /dev/null+++ b/packages/framefields/src/forge-render.test.ts@@ -0,0 +1,127 @@+/**+ * Characters built with `framefields/forge` render and pose through the+ * compositor like any glTF: a Mixamo-named humanoid (drawn as its bone+ * gizmo) answers `pose`, `aim` and `retargetPose` with the `mixamo` preset.+ */+import fs from "node:fs/promises";+import os from "node:os";+import path from "node:path";+import { afterAll, beforeAll, describe, expect, it } from "vitest";+import { boneGizmo, GltfDocument, humanoidSkeleton } from "./forge/index.js";+import {+ Composition,+ Layer,+ type LayoutNode,+ type PoseSource,+ retargetPose,+} from "./index.js";+import { decodePngPixels } from "./test-utils/pixels.js";++const SIZE = 400;+const C = SIZE / 2;+const FEET = C + 150;++let dir = "";+let file = "";++beforeAll(async () => {+ dir = await fs.mkdtemp(path.join(os.tmpdir(), "framefields-forge-"));+ file = path.join(dir, "humanoid.glb");+ const skeleton = humanoidSkeleton({ height: 300 });+ const doc = new GltfDocument();+ const rig = doc.addSkeleton(skeleton);+ const white = doc.addMaterial({ color: "#ffffff", unlit: true });+ const bones = doc.addMesh({+ name: "Bones",+ primitives: [{ ...boneGizmo(skeleton, { width: 0.25 }), material: white }],+ });+ doc.addNode({ name: "Gizmo", mesh: bones, skin: rig.skin });+ await fs.writeFile(file, doc.toGlb());+});+afterAll(async () => {+ await fs.rm(dir, { recursive: true, force: true });+});++async function render(options: Record<string, unknown>) {+ const comp = new Composition({+ width: SIZE,+ height: SIZE,+ fps: 30,+ durationFrames: 30,+ backgroundColor: "#000000",+ });+ comp.add([+ Layer.camera({+ x: C,+ y: C,+ z: -C / Math.tan((25 * Math.PI) / 180),+ targetX: C,+ targetY: C,+ targetZ: 0,+ }),+ Layer.glb(file, {+ x: C,+ y: FEET,+ z: 0,+ rotateX: 180,+ ...options,+ }) as LayoutNode,+ ]);+ const { pixels } = await decodePngPixels(+ await comp.renderFrame({ frame: 0 }),+ );+ /** Lit pixels in a screen box. */+ return (x0: number, y0: number, x1: number, y1: number) => {+ let n = 0;+ for (let y = y0; y < y1; y++)+ for (let x = x0; x < x1; x++)+ if ((pixels[(y * SIZE + x) * 4] ?? 0) > 128) n++;+ return n;+ };+}++// The character's left arm hangs to screen right (it faces the camera).+// At rest the hand sits below the shoulder; raised, above it.+const above = [C + 60, 15, C + 130, 95] as const;+const below = [C + 60, 125, C + 130, 205] as const;++describe("forge characters", () => {+ it("stands in its A-pose rest", async () => {+ const lit = await render({});+ expect(lit(...below)).toBeGreaterThan(80);+ expect(lit(...above)).toBe(0);+ // Both arms: the right hangs to screen left.+ expect(lit(C - 130, 125, C - 60, 205)).toBeGreaterThan(80);+ });++ it("raises the arm by joint pose", async () => {+ const lit = await render({ pose: { LeftArm: { rotateZ: 90 } } });+ expect(lit(...above)).toBeGreaterThan(80);+ expect(lit(...below)).toBe(0);+ });++ it("follows a retargeted performer through the mixamo preset", async () => {+ // A performer (scene coordinates, y down) with the left arm straight up-and-out.+ const at: Record<string, [number, number]> = {+ leftShoulder: [C + 30, 110],+ leftElbow: [C + 70, 70],+ leftWrist: [C + 110, 30],+ };+ const source: PoseSource = {+ point: (name) => {+ const p = at[name];+ return p+ ? { x: p[0], y: p[1], visibility: 1 }+ : { x: 0, y: 0, visibility: 0 };+ },+ };+ const lit = await render({+ aim: retargetPose(source, {+ rig: "mixamo",+ bones: ["leftUpperArm", "leftLowerArm"],+ }),+ });+ expect(lit(...above)).toBeGreaterThan(80);+ expect(lit(...below)).toBe(0);+ });+});diff --git a/packages/framefields/src/forge/index.ts b/packages/framefields/src/forge/index.tsnew file mode 100644index 00000000..eaf3bff3--- /dev/null+++ b/packages/framefields/src/forge/index.ts@@ -0,0 +1,6 @@+/**+ * `framefields/forge`: rigged characters and props built in code (skeletons,+ * skinned meshes, named morph targets, PBR materials with Khronos+ * extensions, clips), written as standard GLB files for `Layer.glb`.+ */+export * from "@framefields/forge";diff --git a/packages/framefields/src/gltf-scene-render.test.ts b/packages/framefields/src/gltf-scene-render.test.tsindex c5d546aa..62884cb7 100644--- a/packages/framefields/src/gltf-scene-render.test.ts+++ b/packages/framefields/src/gltf-scene-render.test.ts@@ -7,10 +7,10 @@ import fs from "node:fs/promises";import { createRequire } from "node:module";import os from "node:os";import path from "node:path";+import { GlbBuilder } from "@framefields/forge";import { Canvas, loadImage } from "skia-canvas";import { afterAll, beforeAll, describe, expect, it } from "vitest";import { Composition, Layer } from "./index.js";-import { GlbBuilder } from "./test-utils/glb.js";const SIZE = 400;const BG = "#000000";diff --git a/packages/framefields/src/test-utils/glb.ts b/packages/framefields/src/test-utils/glb.tsdeleted file mode 100644index 7cfab1b5..00000000--- a/packages/framefields/src/test-utils/glb.ts+++ /dev/null@@ -1,82 +0,0 @@-/** Builds GLB files in tests: typed arrays become accessors in one binary chunk. */--/** Accumulates typed arrays into one binary buffer and the accessors that view them. */-export class GlbBuilder {- private chunks: Buffer[] = [];- private length = 0;- readonly bufferViews: object[] = [];- readonly accessors: object[] = [];-- private view(data: Buffer): number {- const pad = (4 - (this.length % 4)) % 4;- if (pad) {- this.chunks.push(Buffer.alloc(pad));- this.length += pad;- }- this.bufferViews.push({- buffer: 0,- byteOffset: this.length,- byteLength: data.length,- });- this.chunks.push(data);- this.length += data.length;- return this.bufferViews.length - 1;- }-- accessor(- data: Float32Array | Uint16Array,- type: "SCALAR" | "VEC2" | "VEC3" | "VEC4" | "MAT4",- ): number {- const bufferView = this.view(- Buffer.from(data.buffer, data.byteOffset, data.byteLength),- );- const size = { SCALAR: 1, VEC2: 2, VEC3: 3, VEC4: 4, MAT4: 16 }[type];- this.accessors.push({- bufferView,- componentType: data instanceof Float32Array ? 5126 : 5123,- count: data.length / size,- type,- });- return this.accessors.length - 1;- }-- image(png: Buffer): number {- return this.view(png);- }-- build(json: Record<string, unknown>): Buffer {- const bin = Buffer.concat(this.chunks);- const doc = Buffer.from(- JSON.stringify({- asset: { version: "2.0" },- ...json,- buffers: [{ byteLength: bin.length }],- bufferViews: this.bufferViews,- accessors: this.accessors,- }),- );- const docPadded = Buffer.concat([- doc,- Buffer.alloc((4 - (doc.length % 4)) % 4, 0x20),- ]);- const binPadded = Buffer.concat([- bin,- Buffer.alloc((4 - (bin.length % 4)) % 4),- ]);- const header = Buffer.alloc(12);- header.writeUInt32LE(0x46546c67, 0);- header.writeUInt32LE(2, 4);- header.writeUInt32LE(12 + 8 + docPadded.length + 8 + binPadded.length, 8);- const chunk = (data: Buffer, type: number) => {- const h = Buffer.alloc(8);- h.writeUInt32LE(data.length, 0);- h.writeUInt32LE(type, 4);- return Buffer.concat([h, data]);- };- return Buffer.concat([- header,- chunk(docPadded, 0x4e4f534a),- chunk(binPadded, 0x004e4942),- ]);- }-}diff --git a/packages/framefields/tsdown.config.ts b/packages/framefields/tsdown.config.tsindex cf27d889..80c81daf 100644--- a/packages/framefields/tsdown.config.ts+++ b/packages/framefields/tsdown.config.ts@@ -17,6 +17,7 @@ export default defineConfig({"src/sam/index.ts","src/cloud/index.ts","src/storage/index.ts",+ "src/forge/index.ts","src/cli.ts","src/cli/check-worker.ts","src/cli/local-worker.ts",diff --git a/packages/webgpu-renderers/src/loaders/gltf-loader.ts b/packages/webgpu-renderers/src/loaders/gltf-loader.tsindex e608a718..816027b2 100644--- a/packages/webgpu-renderers/src/loaders/gltf-loader.ts+++ b/packages/webgpu-renderers/src/loaders/gltf-loader.ts@@ -868,10 +868,11 @@ export class GltfLoader {basePath?: string,): Material3D {const pbr = mat.pbrMetallicRoughness;- const baseColor = pbr?.baseColorFactor ?? [1, 1, 1, 1];+ const [r, g, b, a] = pbr?.baseColorFactor ?? [1, 1, 1, 1];const material: Material3D = {name,- diffuseColor: baseColor,+ // baseColorFactor is linear; Material3D colors are display (sRGB), like COLOR_0 below.+ diffuseColor: [linearToSrgb(r), linearToSrgb(g), linearToSrgb(b), a],roughness: pbr?.roughnessFactor ?? 0.5,metallic: pbr?.metallicFactor ?? 0.0,shininess: 32.0 * (1.0 - (pbr?.roughnessFactor ?? 0.5)),diff --git a/packages/webgpu-renderers/src/loaders/types.ts b/packages/webgpu-renderers/src/loaders/types.tsindex 5841b5c8..16d8e6b9 100644--- a/packages/webgpu-renderers/src/loaders/types.ts+++ b/packages/webgpu-renderers/src/loaders/types.ts@@ -34,7 +34,7 @@ export interface Mesh3DData {export interface Material3D {name: string;- diffuseColor?: [number, number, number, number]; // RGBA+ diffuseColor?: [number, number, number, number]; // RGBA, display (sRGB) 0..1specularColor?: [number, number, number];ambientColor?: [number, number, number];shininess?: number;diff --git a/plugins/framefields/skills/framefields-compose/SKILL.md b/plugins/framefields/skills/framefields-compose/SKILL.mdindex 2ddfde16..82a9f189 100644--- a/plugins/framefields/skills/framefields-compose/SKILL.md+++ b/plugins/framefields/skills/framefields-compose/SKILL.md@@ -459,6 +459,19 @@ Order of evaluation: clips, `pose`, `aim`, `lookAt`, `ik` (so IK can still plantRules: a crowd mesh keeps its own `rotateX` / `scale` under each instance (y-up glTF: `rotateX: 180`). Spring bones follow the clip's motion and the model's current orientation, not the layer's keyframed travel. Per-instance `clip`, `animOffset`, `animSpeed` in `transforms` / `instanceAt` override the crowd's seeded choices.+**Building characters in code (`framefields/forge`)**: write rigged GLB files from TypeScript, then load them with `Layer.glb`. `humanoidSkeleton({ height, heads, shoulderWidth, hipWidth, armAngle, fingers, toes, prefix })` places Mixamo-named joints (feet on y = 0, facing +z, left at +x, A-pose by default) so `pose`, `aim` and `retarget({ rig: "mixamo" })` work unchanged; `new Skeleton().add(name, { at, parent }).chain(base, points).mirror()` builds any other rig. `GltfDocument` writes meshes (32-bit indices past 65,535 vertices, named morph targets in `extras.targetNames`), PBR materials with Khronos extensions (`clearcoat`, `sheen`, `specular`, `ior`, `transmission`, `volume`, `iridescence`, `anisotropy`, `emissiveStrength`, `unlit`, alpha `mask`), textures, skins and clips, and checks every input where it is given. Colors are display sRGB; the writer stores glTF's linear factors. `boneGizmo(skeleton)` draws the rig as octahedral bones for checking poses before there is a body.++```ts+import { boneGizmo, GltfDocument, humanoidSkeleton } from "framefields/forge";+const skeleton = humanoidSkeleton({ height: 186, heads: 8 });+const doc = new GltfDocument();+const rig = doc.addSkeleton(skeleton);+const gold = doc.addMaterial({ color: "#d99a2b", roughness: 0.35, clearcoat: { factor: 1, roughness: 0.05 } });+doc.addNode({ mesh: doc.addMesh({ primitives: [{ ...boneGizmo(skeleton), material: gold }] }), skin: rig.skin });+fs.writeFileSync("rig.glb", doc.toGlb());+Layer.glb("rig.glb", { rotateX: 180, pose: { LeftArm: { rotateZ: 90 } } });+```+### Procedural worlds: terrain, sculptures, scatterTerrain, SDF sculptures and scatters are ordinary model nodes: they take materials, lights, shadows, SSR/GI, physics, keyframes and a `texture` that can be any layer (an image path, `Layer.video`, a layout tree) drawn fresh each frame.diff --git a/pnpm-lock.yaml b/pnpm-lock.yamlindex 464569bf..cdabf0c4 100644--- a/pnpm-lock.yaml+++ b/pnpm-lock.yaml@@ -2908,6 +2908,30 @@ importers:specifier: ^4.1.11version: 4.1.11(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected](@noble/[email protected])([email protected]))([email protected](@types/[email protected])([email protected])([email protected])([email protected])([email protected])([email protected]))+ packages/forge:+ devDependencies:+ '@framefields/tsconfig':+ specifier: workspace:*+ version: link:../tsconfig+ '@framefields/webgpu-renderers':+ specifier: workspace:*+ version: link:../webgpu-renderers+ '@types/node':+ specifier: ^24.1.0+ version: 24.10.14+ gltf-validator:+ specifier: 2.0.0-dev.3.10+ version: 2.0.0-dev.3.10+ tsdown:+ specifier: ^0.19.0+ version: 0.19.0([email protected])([email protected])+ typescript:+ specifier: ^5.8.3+ version: 5.9.3+ vitest:+ specifier: ^4.1.11+ version: 4.1.11(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected](@noble/[email protected])([email protected]))([email protected](@types/[email protected])([email protected])([email protected])([email protected])([email protected])([email protected]))+packages/framefields:dependencies:'@huggingface/tokenizers':@@ -3001,6 +3025,9 @@ importers:'@framefields/core':specifier: workspace:*version: link:../core+ '@framefields/forge':+ specifier: workspace:*+ version: link:../forge'@framefields/node-audio-compressor':specifier: workspace:*version: link:../../nodes/node-audio-compressor@@ -5886,6 +5913,9 @@ packages:resolution: {integrity: sha512-jhIXaOzy1sb8IyocaruWSn1TjmnBVs8Ayhcy83rmxNJ8q2uWKCAj3CnJY+KpGSXCueAPc0i05kVvVKtP1t9S3g==}engines: {node: '>=10'}+ resolution: {integrity: sha512-odJ4k0tRkGXiDGn78yDBg+fBbAIvBnXxh3RwAta0emSxGtyagFE8B4xELB1oYe3S5RD8Ci3uZAsZaascH2LAEQ==}+resolution: {integrity: sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg==}engines: {node: '>= 0.4'}@@ -8543,7 +8573,7 @@ snapshots:obug: 2.2.1std-env: 4.2.0tinyrainbow: 3.1.0- vitest: 4.1.11(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected](@noble/[email protected])([email protected]))([email protected](@types/[email protected])([email protected])([email protected])([email protected])([email protected])([email protected]))+ vitest: 4.1.11(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected](@noble/[email protected])([email protected]))([email protected](@types/[email protected])([email protected])([email protected])([email protected])([email protected])([email protected]))'@vitest/[email protected]':dependencies:@@ -9047,6 +9077,8 @@ snapshots:merge2: 1.4.1slash: 3.0.0+ [email protected]: {}+diff --git a/specs/character-forge.md b/specs/character-forge.mdnew file mode 100644index 00000000..4d330455--- /dev/null+++ b/specs/character-forge.md@@ -0,0 +1,133 @@+# Character Forge: rigged characters built in code++Status: Phase 1 done (GLB writer, skeleton, humanoid preset, bone gizmo; `packages/forge`, exposed as `framefields/forge`). Phase 2 next. First character: **Ithrael Sunmourn** (`examples/36_emberwake/CHARACTERS.md`), an original hooded elven ranger with a long cloak, long hair, a sculpted face, real eyes, and a full set of expressions.++## Goal++`framefields/forge` builds production-quality, stylized characters from TypeScript and writes standard GLB files that every loader can read: a skeleton on Mixamo names, a skinned sculpted body, materials with several layers, cloth and hair chains, a face rig with eye joints and named blend shapes, and procedural clips. In a composition, the face is driven like the body: `face: { smile: Signal…, blink: auto }` next to `pose`, `aim`, `lookAt` and `springBones`.++Out of scope: photoreal humans and strand-based hair simulation. The target look is a premium stylized sculpt, like a painted miniature come to life.++## What exists and what's missing++| Area | Today | Missing |+| --- | --- | --- |+| GLB writing | `GlbBuilder` in `framefields/src/test-utils/glb.ts` | Public API, 32-bit indices, materials, skins, morph targets, animation writing |+| Sculpting | `sdf-geometry.ts`: six shapes, smooth booleans, Surface Nets at no more than 160 cells | Bone binding, ellipsoid / rounded cone / curve tube / lathe / sweep shapes, bend/twist/mirror/repeat operations, noise displacement, detail resolution, simplification to a budget |+| Skinning | Loader and renderer read skins | Generating weights |+| Morph targets | Loader and GPU morph pipeline (any count); clips animate the weights | `extras.targetNames`, per-target weights by name from a composition (`face`/`morphs`), deforming morphs with skinning |+| Materials | PBR plus emissive strength and unlit in glTF; substrate lobes only on `Layer3D` | Reading and writing `KHR_materials_clearcoat`, `sheen`, `specular`, `ior`, `transmission`, `volume`, `iridescence`, `anisotropy`, and a subsurface material for skin; alpha-tested hair cards |+| Eyes | None | Eye model: cornea with refraction and a wet highlight, iris parallax, dilating pupil, limbal ring, scleral tint |+| Hair | None | Hair cards on guide curves, alpha-tested hair shading (Kajiya-Kay or Marschner, two specular lobes), spring chains per lock |+| Cloth | Spring bones on given joints | Generated cloak and hood mesh with chains, collision against body capsules |+| Clips | glTF clips play | Writing clips: procedural idle/breathe/walk, keyframed acting, baking a retargeted video into a clip |++## Architecture++```+packages/forge (new workspace package, bundled into framefields as framefields/forge)+ src/+ glb-writer.ts public GLB writer: meshes, materials + KHR extensions, skins, morphs, clips, extras+ skeleton.ts Skeleton: joints with rest transforms, mirror(), naming presets (mixamo)+ sculpt/+ field.ts bound SDF: every shape belongs to a joint; evaluation returns distance + per-joint influence+ shapes.ts ellipsoid, roundCone, tube(curve, radii), lathe(profile), sweep(profile, path), extrude(outline)+ ops.ts bend, twist, mirror, repeatAlong(curve), displace(noise)+ mesh.ts narrow-band Surface Nets at hero resolution (sparse blocks), feature-preserving relax+ simplify.ts QEM simplification to a triangle budget; keeps UV seams, material borders and morph-relevant areas+ skin.ts weights from the field's per-joint influence, smoothed, top 4 normalized+ bake.ts vertex AO and cavity, curvature (edge wear), thickness (subsurface), normal map from the detailed field+ uv.ts per-material-island planar/cylindrical unwrap (face, iris, cards); triplanar elsewhere+ face/+ face-rig.ts face landmarks, eye/jaw/tongue joints, eyelid and brow regions+ expressions.ts FACS-style blend shapes made by moving the base mesh onto re-sculpted fields+ eyes.ts eyeball, cornea, iris and pupil meshes + eye material+ hair/+ guides.ts guide curves from a groom description (part, flow, length, clumping, curl)+ cards.ts card strips along guides with alpha texture; root-to-tip chains for spring bones+ texture.ts procedural strand alpha/ID/depth/root-tip atlas+ cloth/+ garment.ts cloak/hood/cape panels from outlines draped on the body capsule set; chains + collision capsules+ clips/+ procedural.ts breathe, idle sway, blink, saccades, walk/run from the gait solver+ keys.ts keyframed acting clips (joints + morph weights) → glTF animation+ bake.ts bake pose/aim/retarget output into a clip (for crowds)+ character.ts Character.build(spec) → GLB + report (triangles, joints, morphs, materials, watertight)+ qa.ts turntable, pose-extremes and expression contact sheets through the real renderer+```++Engine changes outside the package:++- **Loader** (`webgpu-renderers/src/loaders/gltf-loader.ts`): `extras.targetNames`, the KHR material extensions above mapped onto the existing substrate lobes, `alphaMode: MASK` hair with alpha-to-coverage, morphs combined with skinning (morph first, then skin).+- **Renderer**: an eye shading model (`KHR_materials_*` doesn't cover it; mark it with `extras.framefields.eye` so other viewers fall back to PBR); an anisotropic hair lobe for cards (`extras.framefields.hair`); subsurface skin from the existing subsurface lobe, with thickness from the baked vertex attribute.+- **Schema** (`character-schema.ts`): `face` (named expressions as `NumberTrack`s, including signals), `morphs` (raw target names), `eyes` (`target`, `saccades`, `blink`, `pupil`), and spring-chain `colliders` (capsules on joints).+- **Runtime** (`rig.ts`, `renderers/character.ts`): face evaluation order (clips → `morphs`/`face` → eyes look-at → blink); collision for spring chains.++## Face, eyes, and expressions++**Face rig joints.** `Head`, `Jaw`, `LeftEye`/`RightEye`, `LeftEyelidUpper`/`Lower` (also right), `Tongue`. The eyes rotate with real joints, so `lookAt` and saccades are joint rotations and the eyelids follow them as a corrective.++**Expression set** (blend shapes named after FACS action units, with friendly names on top):++| Group | Targets |+| --- | --- |+| Brows | browInnerUp, browDown.L/R, browOuterUp.L/R |+| Eyes | blink.L/R, squint.L/R, wide.L/R, eyeLookUp/Down (lid follow) |+| Nose and cheeks | sneer.L/R, cheekRaise.L/R, cheekPuff |+| Mouth | jawOpen, smile.L/R, frown.L/R, pucker, funnel, press, stretch.L/R, upperUp.L/R, lowerDown.L/R, dimple.L/R |+| Visemes | aa, E, ih, oh, ou, FV, MBP, L, TH, CH (derived from the units above) |++Each target is made by re-sculpting the face field (lids closed, mouth corners pulled up along the cheek, and so on) and moving the base mesh's vertices onto it along the gradient. The vertex layout stays identical, so the targets are valid blend shapes.++**Composition API:**++```ts+Layer.glb("ithrael.glb", {+ face: {+ expression: { contempt: 0.6, sad: [[0, 0], [48, 0.8]] }, // presets mix FACS units+ smile: Signal.sine({ min: 0, max: 0.2, period: 96 }), // any unit, any track+ speech: { audio: "vo/ithrael.wav", transcript }, // visemes from the word timestamps+ },+ eyes: { target: [960, 400, -300], saccades: true, blink: "auto", pupil: lightLevel },+});+```++Presets: neutral, contempt, cold, sad, angry, fear, surprise, smirk, determined, and a "stillness" preset of micro-motion (slow blink, tiny gaze shifts) for Ithrael.++Lip sync: visemes are derived from the scribe-v2 word timestamps through a grapheme-to-phoneme table. No external model is needed.++## Hair++A groom description in code: part line, flow field over the scalp, length map, clumping, curl and stray hairs. Guides grow over the scalp sculpt. Cards are laid along the clumped guides, with a procedural strand atlas texture. Each lock gets a short spring chain (3 to 5 joints) skinned to its cards and colliding with head, neck and shoulder capsules. Under the hood the cards sit on the hood's inner surface, and the loose front locks fall out of the hood.++Shading: alpha-to-coverage cutout, two shifted anisotropic specular lobes (a primary white one and a secondary tinted one), a root-to-tip darkening gradient, and a baked depth/ID texture for variation.++## Cloak and hood++The garment panels are 2D outlines with seams (hood, mantle, long cloak split into back panels for motion). They are draped on the body's capsule set by a short offline relaxation, which gives the rest shape. Each panel then becomes a grid mesh with spring chains down its length, at 6 to 10 columns. Collision capsules on the spine, thighs and upper arms stop the cloak from passing through the legs. The cloth material uses `sheen` and a frayed alpha-tested hem. The hood is weighted to `Head`, with an inner shadow from the baked AO.++## Ithrael's spec++An original design, distinct from any existing character (see CHARACTERS.md): pale-gold skin, short white hair cut close on one side with long locks on the other, an antler crown threaded with living amber leaves, a petal-shaped bark-lacquer armor of amber resin over dark wood, a moss-green hooded cloak, a recurve horn bow, a gold century-ring tattoo down the right arm, and amber eyes. Her acting is stillness: slow, deliberate movement, a gaze that holds, and small expressions (contempt, cold, a sad closing of the eyes).++Budgets: body 40–60k triangles, face area densest, hair 20–30k, cloak 8–12k, 70–90 joints including the face and chains, about 60 blend shapes.++## Phases++1. **GLB writer and skeleton.** Move `GlbBuilder` into `packages/forge` as a public writer: 32-bit indices, materials with KHR extensions, skins, morphs with `targetNames`, clips, extras. Add the skeleton with mirroring and Mixamo names. *Done when:* a round trip through our loader is exact, and the output passes the Khronos glTF validator.+2. **Bound sculpting and auto-skinning.** Add the new shapes and operations, hero-resolution narrow-band meshing, simplification, and weights from the field. *Done when:* a body bends at the elbows and knees without collapsing, checked on a pose-extremes contact sheet with a volume-loss test.+3. **Loader and material extensions.** Map the KHR materials onto substrate lobes, add alpha-to-coverage, and combine morphs with skinning. *Done when:* the Khronos sample models for clearcoat, sheen and alpha blending match their reference renders closely.+4. **Face.** Add the face rig, FACS targets made by projection, the `face`/`morphs`/`eyes` schema and runtime, and visemes from timestamps. *Done when:* the expression contact sheet reads clearly, blinks close fully with the eye looking up or down, and the lips land on the right words.+5. **Eyes.** Add the eye meshes and shading model (refraction, iris parallax, pupil, wet highlight). *Done when:* the catchlight follows the key light and the iris parallaxes under a camera orbit.+6. **Hair.** Add grooming, cards, the strand atlas, the anisotropic hair lobe, and lock chains with collision. *Done when:* a turntable shows no harsh card edges, and the hair settles after a head turn without passing through the head.+7. **Cloth.** Add the garment panels, draping, cloak chains and collision capsules. *Done when:* a walk clip shows the cloak swinging with no leg pass-through.+8. **Clips and QA.** Add procedural breathe/idle/blink/saccades, keyframed acting clips (draw, release, turn-and-look), bake-to-clip, and the turntable, pose and expression sheets. *Done when:* `examples/36_emberwake` renders Ithrael's test shot (shot 18: "I said I would not let her.") with lip sync, gaze and cloak in wind.++Each phase ships with tests (geometry invariants, a loader round trip, pixel probes through the real renderer), SKILL.md updates (`framefields-compose` Characters section) and `check:plugins`.++## Risks++- **Quality of the face sculpt.** It is the hardest part artistically. Mitigation: base the proportions on published anthropometric ratios, and iterate on the expression contact sheet at close-up camera distance before moving on.+- **Card hair alpha sorting.** Alpha-to-coverage with MSAA avoids sorting. If our pass doesn't run MSAA, use a dithered cutout with TAA-free temporal stability, verified frame to frame.+- **Spring chains have no self-collision between locks.** Clumped locks and capsule collision are enough at trailer distances.+- **Simplification versus morph quality.** Simplify the base before making the targets, and keep extra density in the face area.