Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
JavaScript 3d-generationai-agentsanimationavatardigital-humangenerative-aiglbgltfimage-to-3dllmmcpmcp-servermodel-context-protocolsolanatext-to-3dthreejsweb-componentswebglwebxrx402
feat(tty-avatar): render a live 3D avatar in the terminal with the tty, render, motion and avatar-stream packages
nirholasc9ee4b2097ee
diff --git a/docs/tty-avatar.md b/docs/tty-avatar.mdnew file mode 100644index 000000000..de2fb2eb6--- /dev/null+++ b/docs/tty-avatar.md@@ -0,0 +1,91 @@+# Terminal avatar (`@three-ws/tty-avatar`)++Every three.ws avatar and every agent's 3D body can run in a terminal. One command, no browser, no GPU:++```sh+npx @three-ws/tty-avatar 81a076b6-55ff-49a2-b007-1d88e7dce2aa+```++That draws the avatar in colour at 24 frames a second in whatever terminal you are in, swaying gently, and keeps it there until you press Ctrl-C. Point it at an agent instead (`agent:<id>`) and you get that agent's bound body. Point it at any `.glb` on disk or on the web and it draws that.++The reason it exists is the second half: wire it to Claude Code once and the avatar becomes your coding agent's face. It looks up and thinks while the agent reads your prompt, nods quickly while it edits and runs commands, shakes when a tool fails, pulses when the agent is waiting on you, and bounces when it finishes. The caption under it says what the agent is doing right now (`editing index.js`, `$ npm test`, `searching`).++## Try it++Pick any avatar in the [gallery](/gallery). Its page's **Embed** tab has a **Terminal** section with the exact command for that avatar. Or use an agent from [/agents](/agents) with `agent:` in front of its id.++```sh+# a three.ws avatar+npx @three-ws/tty-avatar 81a076b6-55ff-49a2-b007-1d88e7dce2aa++# an agent's body+npx @three-ws/tty-avatar agent:bd1b56b0-5494-47e2-ad78-04ef4c43ae5b++# the page URL works too+npx @three-ws/tty-avatar https://three.ws/avatars/81a076b6-55ff-49a2-b007-1d88e7dce2aa++# your own model+npx @three-ws/tty-avatar ./hero.glb --mode braille --zoom 1.3+```++Three glyph modes. `blocks` (default in a terminal) draws truecolor half-blocks, two colours per cell. `braille` draws 2x4 dots per cell for four times the vertical detail. `ascii` is a plain luminance ramp with no colour codes, and is what you get automatically when the output is piped, so `npx @three-ws/tty-avatar snapshot <id> > avatar.txt` produces something you can paste anywhere.++## Give your coding agent a face++```sh+npx @three-ws/tty-avatar install-hooks --write+```++That merges nine hook entries into `~/.claude/settings.json` (it leaves any hooks you already have alone, and running it twice does not duplicate them). Open two panes: run the viewer in one, Claude Code in the other.++| What Claude Code does | What the avatar does |+|---|---|+| Starts a session | bounces once, "session started" |+| Reads your prompt | tilts and looks up, "reading your prompt" |+| Reads, edits, runs, searches | quick working nods with the file or command as the caption |+| A tool fails | a short shake, "Bash failed" |+| Needs your permission or input | pulses, with the notification text, until you answer |+| Finishes a turn | bounces, "done", then settles |+| Compacts context | thinks, "compacting memory" |+| Ends the session | head down, asleep |++The hook command that gets installed is just `cat > ~/.three-ws/tty-avatar/event.json`. It starts no interpreter, so it costs a tool call nothing. The viewer polls that file five times a second and maps each payload to a mood itself. Any agent runtime that can write a Claude-Code-shaped JSON payload to that path gets the same behaviour, and any script can drive the mood directly:++```sh+npx @three-ws/tty-avatar mood happy --say "deploy landed" --ttl 5000+npx @three-ws/tty-avatar say "migrating the database"+```++`install-hooks` without `--write` prints the JSON so you can paste it yourself; `--json` prints just the `hooks` object for another tool to merge.++## Moods++`idle`, `spin`, `think`, `work`, `talk`, `happy`, `attention`, `error`, `sleep`. Each is procedural whole-body motion (sway, tilt, nod, bounce, shake), which is why it works on any GLB with no rig at all, props included. Switching moods cross-fades over half a second so a burst of hook events never makes the model pop. `tty-avatar moods` lists them.++## As a library++```js+import { resolveSource, parseGlb, TtyAvatar, snapshot } from '@three-ws/tty-avatar';++const { bytes, name } = await resolveSource('agent:bd1b56b0-5494-47e2-ad78-04ef4c43ae5b');+const mesh = await parseGlb(bytes);++const viewer = new TtyAvatar(mesh, { name, fps: 24 });+viewer.setMood('talk', { say: 'hello from the terminal', ttlMs: 4000 });+await viewer.start();+```++`snapshot(mesh, { mode, columns, rows, yaw, pitch, zoom, mood })` returns one frame as a string without touching the terminal, which is how the package tests render and how you would put a frame in a log or a README. The rasterizer (`createFrame`, `render`) and encoders (`encodeBlocks`, `encodeBraille`, `encodeAscii`) are exported separately for anything custom. The full export table is in the [package README](https://github.com/nirholas/three.ws/tree/main/packages/tty-avatar#library).++## How it draws++The terminal is the framebuffer. A character cell is split into subpixels (1x2 for half-blocks, 2x4 for braille), the mesh is projected with a perspective camera onto that grid, and a z-buffered scanline fill writes one lit colour per subpixel. Faces are shaded with a key light, a fill light, an ambient floor and a rim term so silhouettes stay legible on both dark and light terminal themes. Colour comes from each primitive's material factor and vertex colours; textures are not sampled because a cell is far coarser than a texel. A typical 20k-triangle three.ws avatar renders in under 5 ms per frame at 160x96 subpixels, so 24 fps costs a fraction of one core.++Id resolution uses the same public endpoints the web studio does: `/api/avatars/:id` for an avatar's `model_url`, `/api/agents/:id` for an agent's `avatar_model_url`. Models are decoded with `@gltf-transform`, so meshopt-compressed three.ws avatars and Draco files load as-is.++## Related++- [Avatar CLI](/docs/avatar-cli): manifests, validation and embed snippets from the shell.+- [Web component](/docs/web-component): the same avatar in a browser.+- [Share & embed](/docs/share-and-embed): every other way to put an avatar somewhere.+- Source and tests: [packages/tty-avatar](https://github.com/nirholas/three.ws/tree/main/packages/tty-avatar).diff --git a/packages/avatar-stream/LICENSE b/packages/avatar-stream/LICENSEnew file mode 100644index 000000000..dff21b7f4--- /dev/null+++ b/packages/avatar-stream/LICENSE@@ -0,0 +1,20 @@+Copyright 2026 nirholas++All rights reserved.++This software is proprietary and may not be used, copied, modified, distributed,+translated, or made available to any third party without the express written+permission of the copyright owner.++No rights are granted by implication, estoppel, or otherwise. Use of this+software is subject to the terms of a separate license agreement with the+copyright owner.++THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,+FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT.++IN NO EVENT SHALL THE COPYRIGHT OWNER BE LIABLE FOR ANY CLAIM, DAMAGES, OR+OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT, OR OTHERWISE,+ARISING FROM, OUT OF, OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER+DEALINGS IN THE SOFTWARE.diff --git a/packages/avatar-stream/bin/a3s.js b/packages/avatar-stream/bin/a3s.jsnew file mode 100755index 000000000..821de56e9--- /dev/null+++ b/packages/avatar-stream/bin/a3s.js@@ -0,0 +1,215 @@+#!/usr/bin/env node+/**+ * a3s - pack, inspect, verify, extract and benchmark A3S streams.+ *+ * Every command works on a plain file with no service running, which is the+ * point: an A3S stream is a static asset, so the tooling around it should be+ * shell-shaped rather than server-shaped.+ */++import { readFile, writeFile } from 'node:fs/promises';+import { basename } from 'node:path';+import { createHash } from 'node:crypto';++import { pack } from '../src/pack.js';+import { A3SStream } from '../src/reader.js';+import { decodePreamble, decodeHeader, RECOMMENDED_PREFIX_BYTES } from '../src/format.js';+import { reconstruct, triangleCount, triangleFingerprint } from '../src/reconstruct.js';++const KB = (n) => `${(n / 1024).toFixed(1)} KB`;+const PCT = (n, of) => `${((100 * n) / of).toFixed(1)}%`;++function parseArgs(argv) {+ const args = { _: [] };+ for (let i = 0; i < argv.length; i++) {+ const token = argv[i];+ if (token.startsWith('--')) {+ const [key, inline] = token.slice(2).split('=');+ const next = argv[i + 1];+ if (inline !== undefined) args[key] = inline;+ else if (next && !next.startsWith('-')) args[key] = argv[++i];+ else args[key] = true;+ } else if (token.startsWith('-') && token.length === 2) {+ args[token.slice(1)] = argv[++i];+ } else {+ args._.push(token);+ }+ }+ return args;+}++const USAGE = `a3s - progressive 3D over plain HTTP++Usage+ a3s pack <input.glb> [-o out.a3s] [--levels 0.03,0.1,0.3,1] [--base-texture 64] [--json]+ a3s inspect <file.a3s|url> [--json]+ a3s verify <file.a3s|url> [--deep] [--json]+ a3s extract <file.a3s|url> [-o preview.glb]+ a3s bench <input.glb> [--connection slow-3g|fast-3g|4g] [--json]++Commands+ pack Build an A3S stream from a GLB.+ inspect Print the layer table: bytes, triangles, and the Range each layer needs.+ verify Re-hash every layer. With --deep, replay the stream and prove the+ refined surface matches the source triangle for triangle.+ extract Write layer 0 out as a standalone, spec-valid GLB.+ bench Estimate time-to-first-frame against a classic GLB fetch.+`;++/** Load a stream from a path or a URL. */+async function openTarget(target, options = {}) {+ if (/^https?:\/\//.test(target)) return A3SStream.open(target, options);+ return A3SStream.open(new Uint8Array(await readFile(target)), options);+}++async function cmdPack(args) {+ const input = args._[0];+ if (!input) throw new Error('pack needs an input .glb');+ const output = args.o || args.output || input.replace(/\.glb$/i, '') + '.a3s';+ const levels = args.levels ? String(args.levels).split(',').map(Number) : undefined;+ const baseTextureSize = args['base-texture'] ? Number(args['base-texture']) : undefined;++ const source = await readFile(input);+ const started = Date.now();+ const { container, header, stats } = await pack(source, { levels, baseTextureSize, name: basename(input) });+ await writeFile(output, container);+ const elapsed = Date.now() - started;++ if (args.json) {+ console.log(JSON.stringify({ output, elapsedMs: elapsed, ...stats, header }, null, 2));+ return;+ }+ console.log(`packed ${basename(input)} -> ${output} in ${elapsed} ms\n`);+ console.log(` source ${KB(stats.sourceBytes)}`);+ console.log(` container ${KB(stats.containerBytes)}`);+ console.log(` base layer ${KB(stats.baseBytes)} (${PCT(stats.baseBytes, stats.sourceBytes)} of source, ${stats.baseTriangles} of ${stats.fullTriangles} triangles)\n`);+ printLayers(header);+}++function printLayers(header) {+ console.log(' level kind triangles bytes cumulative');+ let cumulative = 0;+ for (const layer of header.layers) {+ cumulative = layer.offset + layer.length;+ console.log(+ ` ${String(layer.level).padEnd(6)} ${layer.kind.padEnd(6)} ${String(layer.triangleCount).padStart(9)} ${KB(layer.length).padStart(10)} ${KB(cumulative).padStart(14)}`,+ );+ }+}++async function cmdInspect(args) {+ const target = args._[0];+ if (!target) throw new Error('inspect needs a .a3s file or URL');+ const stream = await openTarget(target);+ const { header, preamble } = stream;+ if (args.json) {+ console.log(JSON.stringify({ preamble, header }, null, 2));+ return;+ }+ console.log(`${header.source.name || target} (${header.version}, packed by ${header.generator})\n`);+ console.log(` source sha256 ${header.source.sha256}`);+ console.log(` source bytes ${KB(header.source.byteLength)}`);+ console.log(` geometry ${header.geometry.triangleCount} triangles, ${header.geometry.vertexCount} vertices, ${header.geometry.primitiveCount} primitives`);+ console.log(` first render ${KB(preamble.baseOffset + preamble.baseLength)} (one Range request)\n`);+ printLayers(header);+ console.log(`\n Range for layer 0: bytes=0-${preamble.baseOffset + preamble.baseLength - 1}`);+}++async function cmdVerify(args) {+ const target = args._[0];+ if (!target) throw new Error('verify needs a .a3s file or URL');+ const bytes = /^https?:\/\//.test(target) ? null : new Uint8Array(await readFile(target));+ const stream = await openTarget(target, { verify: true });+ const results = [];+ for (let level = 0; level < stream.layerCount; level++) {+ const { descriptor, payload } = await stream.layer(level);+ const actual = createHash('sha256').update(payload).digest('hex');+ results.push({ level, expected: descriptor.sha256, actual, ok: actual === descriptor.sha256 });+ }++ let deep = null;+ if (args.deep) {+ const { primitives } = await reconstruct(bytes || target, { verify: true });+ const rebuilt = triangleCount(primitives);+ deep = { triangles: rebuilt, expected: stream.header.geometry.triangleCount, ok: rebuilt === stream.header.geometry.triangleCount };+ }++ const ok = results.every((r) => r.ok) && (!deep || deep.ok);+ if (args.json) {+ console.log(JSON.stringify({ ok, layers: results, deep }, null, 2));+ } else {+ for (const r of results) console.log(` layer ${r.level} ${r.ok ? 'OK ' : 'BAD '} ${r.actual.slice(0, 16)}`);+ if (deep) console.log(` deep replay ${deep.ok ? 'OK' : 'MISMATCH'} ${deep.triangles} triangles reconstructed`);+ console.log(ok ? '\nverified' : '\nFAILED');+ }+ if (!ok) process.exitCode = 1;+}++async function cmdExtract(args) {+ const target = args._[0];+ if (!target) throw new Error('extract needs a .a3s file or URL');+ const output = args.o || args.output || 'preview.glb';+ const stream = await openTarget(target);+ await writeFile(output, stream.base);+ console.log(`wrote ${output} ${KB(stream.base.byteLength)} (${stream.header.layers[0].triangleCount} triangles, opens in any glTF viewer)`);+}++/** Downlink speeds in bytes per second, matching the profiles browsers ship. */+const CONNECTIONS = {+ 'slow-3g': 50 * 1024,+ 'fast-3g': 180 * 1024,+ '4g': 1200 * 1024,+};++async function cmdBench(args) {+ const input = args._[0];+ if (!input) throw new Error('bench needs an input .glb');+ const connection = args.connection || 'fast-3g';+ const bytesPerSecond = CONNECTIONS[connection];+ if (!bytesPerSecond) throw new Error(`unknown connection profile ${connection}; try ${Object.keys(CONNECTIONS).join(', ')}`);++ const source = await readFile(input);+ const { header, stats } = await pack(source, { name: basename(input) });+ const firstFrameBytes = header.layers[0].offset + header.layers[0].length;+ const classicMs = (stats.sourceBytes / bytesPerSecond) * 1000;+ const streamMs = (firstFrameBytes / bytesPerSecond) * 1000;++ const result = {+ input: basename(input),+ connection,+ classic: { bytes: stats.sourceBytes, firstFrameMs: Math.round(classicMs), triangles: stats.fullTriangles },+ stream: { bytes: firstFrameBytes, firstFrameMs: Math.round(streamMs), triangles: stats.baseTriangles },+ speedup: Number((classicMs / streamMs).toFixed(1)),+ };+ if (args.json) {+ console.log(JSON.stringify(result, null, 2));+ return;+ }+ console.log(`${result.input} on ${connection}\n`);+ console.log(` classic GLB ${KB(result.classic.bytes).padStart(10)} first frame at ${String(result.classic.firstFrameMs).padStart(6)} ms (${result.classic.triangles} triangles)`);+ console.log(` A3S stream ${KB(result.stream.bytes).padStart(10)} first frame at ${String(result.stream.firstFrameMs).padStart(6)} ms (${result.stream.triangles} triangles)`);+ console.log(`\n ${result.speedup}x faster to first frame, refining to full detail in the background.`);+}++const COMMANDS = { pack: cmdPack, inspect: cmdInspect, verify: cmdVerify, extract: cmdExtract, bench: cmdBench };++async function main() {+ const [command, ...rest] = process.argv.slice(2);+ if (!command || command === 'help' || command === '--help' || command === '-h') {+ console.log(USAGE);+ return;+ }+ const handler = COMMANDS[command];+ if (!handler) {+ console.error(`a3s: unknown command "${command}"\n`);+ console.log(USAGE);+ process.exitCode = 1;+ return;+ }+ await handler(parseArgs(rest));+}++main().catch((error) => {+ console.error(`a3s: ${error.message}`);+ process.exitCode = 1;+});diff --git a/packages/avatar-stream/package.json b/packages/avatar-stream/package.jsonnew file mode 100644index 000000000..f3bfa22c5--- /dev/null+++ b/packages/avatar-stream/package.json@@ -0,0 +1,82 @@+{+ "name": "@three-ws/avatar-stream",+ "version": "0.1.0",+ "description": "Progressive 3D over plain HTTP. Pack a GLB into a layered A3S stream whose first 50 KB is already a complete, skinned, spec-valid avatar, then refine it in place with byte-range requests. No custom server, no WebSocket, no new codec.",+ "type": "module",+ "main": "./src/index.js",+ "module": "./src/index.js",+ "exports": {+ ".": {+ "import": "./src/index.js",+ "default": "./src/index.js"+ },+ "./three": {+ "import": "./src/three.js",+ "default": "./src/three.js"+ },+ "./node": {+ "import": "./src/node.js",+ "default": "./src/node.js"+ },+ "./package.json": "./package.json"+ },+ "bin": {+ "a3s": "./bin/a3s.js"+ },+ "files": [+ "src",+ "bin",+ "README.md",+ "LICENSE"+ ],+ "scripts": {+ "test": "node --test test/*.test.js"+ },+ "keywords": [+ "3d",+ "glb",+ "gltf",+ "progressive",+ "streaming",+ "lod",+ "level-of-detail",+ "meshopt",+ "three.js",+ "avatar",+ "range-requests",+ "three.ws"+ ],+ "author": "nirholas <[email protected]>",+ "license": "Apache-2.0",+ "homepage": "https://three.ws/stream",+ "repository": {+ "type": "git",+ "url": "git+https://github.com/nirholas/three.ws.git",+ "directory": "packages/avatar-stream"+ },+ "bugs": {+ "url": "https://github.com/nirholas/three.ws/issues"+ },+ "engines": {+ "node": ">=18"+ },+ "peerDependencies": {+ "@gltf-transform/core": ">=4",+ "@gltf-transform/extensions": ">=4",+ "@gltf-transform/functions": ">=4",+ "draco3dgltf": ">=1.5",+ "meshoptimizer": ">=1.1",+ "sharp": ">=0.33"+ },+ "peerDependenciesMeta": {+ "@gltf-transform/core": { "optional": true },+ "@gltf-transform/extensions": { "optional": true },+ "@gltf-transform/functions": { "optional": true },+ "draco3dgltf": { "optional": true },+ "meshoptimizer": { "optional": true },+ "sharp": { "optional": true }+ },+ "publishConfig": {+ "access": "public"+ }+}diff --git a/packages/avatar-stream/src/format.js b/packages/avatar-stream/src/format.jsnew file mode 100644index 000000000..fb9a6905b--- /dev/null+++ b/packages/avatar-stream/src/format.js@@ -0,0 +1,207 @@+/**+ * A3S container format (`threews.a3s.v1`) - pure, isomorphic, zero-dependency.+ *+ * An A3S file is a single byte stream that holds one 3D asset at several levels+ * of detail, ordered coarsest-first, so that a *prefix* of the file is already a+ * complete renderable asset:+ *+ * [ 32-byte preamble ][ header JSON ][ layer 0: a valid GLB ][ patch 1 ] ... [ patch N ]+ *+ * The layout is deliberate. Layer 0 is a standalone, spec-valid GLB, so+ * `curl --range 0-<baseEnd> model.a3s | tail -c +<baseOffset+1> > preview.glb`+ * opens in any glTF tool on earth. Every later layer is a patch that adds newly+ * revealed vertices and swaps in a finer index buffer. Because the packer orders+ * the vertex buffer so each level's vertices are a prefix of the next level's,+ * a patch never rewrites bytes the client already holds: it only appends.+ *+ * Reading this module does not pull in a mesh library, a glTF parser, or Node+ * built-ins. It is the one piece both the packer and the browser reader share.+ */++/** Magic bytes at offset 0: "A3S1". */+export const MAGIC = 0x31533341; // 'A','3','S','1' read little-endian as u32+export const MAGIC_STRING = 'A3S1';++/** Fixed size of the preamble, in bytes. */+export const PREAMBLE_BYTES = 32;++/** Format version encoded in the preamble. */+export const FORMAT_VERSION = 1;++/** Version string carried in the header JSON. */+export const VERSION_TAG = 'threews.a3s.v1';++/**+ * How many leading bytes a client should request to be confident it has the+ * preamble, the header, and (for a typical avatar) the whole base layer in a+ * single round trip. Tuned against the platform's avatar corpus.+ */+export const RECOMMENDED_PREFIX_BYTES = 96 * 1024;++const textEncoder = new TextEncoder();+const textDecoder = new TextDecoder();++/** Round `n` up to the next multiple of 4, so every chunk stays 4-byte aligned. */+export function align4(n) {+ return (n + 3) & ~3;+}++function asUint8(bytes) {+ if (bytes instanceof Uint8Array) return bytes;+ if (ArrayBuffer.isView(bytes)) return new Uint8Array(bytes.buffer, bytes.byteOffset, bytes.byteLength);+ if (bytes instanceof ArrayBuffer) return new Uint8Array(bytes);+ throw new TypeError('expected ArrayBuffer or typed array');+}++/**+ * Read the 32-byte preamble. This is all a client needs to know where the+ * header and the base layer live, and it is the only fixed-offset structure in+ * the format.+ *+ * @param {ArrayBuffer|Uint8Array} bytes at least PREAMBLE_BYTES long, starting at file offset 0+ */+export function decodePreamble(bytes) {+ const u8 = asUint8(bytes);+ if (u8.byteLength < PREAMBLE_BYTES) {+ throw new Error(`a3s: need at least ${PREAMBLE_BYTES} bytes to read the preamble, got ${u8.byteLength}`);+ }+ const view = new DataView(u8.buffer, u8.byteOffset, u8.byteLength);+ const magic = view.getUint32(0, true);+ if (magic !== MAGIC) {+ throw new Error('a3s: bad magic, this is not an A3S stream');+ }+ const formatVersion = view.getUint32(4, true);+ if (formatVersion !== FORMAT_VERSION) {+ throw new Error(`a3s: unsupported format version ${formatVersion} (this build reads ${FORMAT_VERSION})`);+ }+ return {+ formatVersion,+ headerOffset: view.getUint32(8, true),+ headerLength: view.getUint32(12, true),+ baseOffset: view.getUint32(16, true),+ baseLength: view.getUint32(20, true),+ layerCount: view.getUint32(24, true),+ totalLength: view.getUint32(28, true),+ };+}++/** Serialise a preamble object back to its 32 bytes. */+export function encodePreamble(p) {+ const out = new Uint8Array(PREAMBLE_BYTES);+ const view = new DataView(out.buffer);+ view.setUint32(0, MAGIC, true);+ view.setUint32(4, FORMAT_VERSION, true);+ view.setUint32(8, p.headerOffset, true);+ view.setUint32(12, p.headerLength, true);+ view.setUint32(16, p.baseOffset, true);+ view.setUint32(20, p.baseLength, true);+ view.setUint32(24, p.layerCount, true);+ view.setUint32(28, p.totalLength, true);+ return out;+}++/**+ * Parse the header JSON out of a byte range that covers it.+ *+ * @param {ArrayBuffer|Uint8Array} bytes a buffer whose index 0 is file offset `baseByteOffset`+ * @param {object} preamble as returned by decodePreamble+ * @param {number} [baseByteOffset] file offset of `bytes[0]` (0 when you passed the file head)+ */+export function decodeHeader(bytes, preamble, baseByteOffset = 0) {+ const u8 = asUint8(bytes);+ const start = preamble.headerOffset - baseByteOffset;+ const end = start + preamble.headerLength;+ if (start < 0 || end > u8.byteLength) {+ throw new Error('a3s: header is outside the supplied byte range, fetch more of the file head');+ }+ const header = JSON.parse(textDecoder.decode(u8.subarray(start, end)));+ if (header.version !== VERSION_TAG) {+ throw new Error(`a3s: unsupported header version ${header.version}`);+ }+ return header;+}++/**+ * Build a complete container from its parts.+ *+ * `header` is written verbatim except for the `layers[].offset` / `length`+ * fields, which are computed here so the header can never disagree with the+ * bytes that follow it. That single-source-of-truth rule is what lets a reader+ * trust the header without re-scanning the payload.+ *+ * @param {object} args+ * @param {object} args.header header JSON minus the layer offsets+ * @param {Uint8Array} args.baseGlb layer 0, a complete valid GLB+ * @param {Uint8Array[]} args.patches layers 1..N, in order+ * @returns {Uint8Array}+ */+export function encodeContainer({ header, baseGlb, patches = [] }) {+ const base = asUint8(baseGlb);+ const patchBytes = patches.map(asUint8);++ // Two passes: the header's byte length depends on the offsets it carries, and+ // the offsets depend on the header's byte length. Encode once with placeholder+ // offsets to learn the length, pad the header to a stable size, then fill in+ // the real offsets. Padding beats iterating to a fixed point.+ const layers = [{ ...header.layers[0] }, ...header.layers.slice(1).map((l) => ({ ...l }))];+ const draft = { ...header, layers };+ for (const layer of layers) {+ layer.offset = 0;+ layer.length = 0;+ }+ const probe = textEncoder.encode(JSON.stringify(draft));+ // Reserve generous slack so the real offsets always fit in the same space.+ const headerCapacity = align4(probe.byteLength + 24 * layers.length + 64);++ const headerOffset = PREAMBLE_BYTES;+ let cursor = align4(headerOffset + headerCapacity);+ const baseOffset = cursor;+ layers[0].offset = baseOffset;+ layers[0].length = base.byteLength;+ cursor = align4(cursor + base.byteLength);+ patchBytes.forEach((p, i) => {+ layers[i + 1].offset = cursor;+ layers[i + 1].length = p.byteLength;+ cursor = align4(cursor + p.byteLength);+ });+ const totalLength = cursor;++ const finalHeader = textEncoder.encode(JSON.stringify(draft));+ if (finalHeader.byteLength > headerCapacity) {+ throw new Error('a3s: header outgrew its reserved capacity');+ }++ const out = new Uint8Array(totalLength);+ out.set(+ encodePreamble({+ headerOffset,+ headerLength: finalHeader.byteLength,+ baseOffset,+ baseLength: base.byteLength,+ layerCount: layers.length,+ totalLength,+ }),+ 0,+ );+ out.set(finalHeader, headerOffset);+ out.set(base, baseOffset);+ patchBytes.forEach((p, i) => out.set(p, layers[i + 1].offset));+ return out;+}++/**+ * The byte range a client should request to render layer `level`.+ * Layer 0's range starts at 0 because the client also needs preamble + header.+ */+export function rangeForLayer(header, preamble, level) {+ const layer = header.layers[level];+ if (!layer) throw new Error(`a3s: no layer ${level}`);+ if (level === 0) return { start: 0, end: preamble.baseOffset + preamble.baseLength - 1 };+ return { start: layer.offset, end: layer.offset + layer.length - 1 };+}++/** Format a `Range:` header value for a layer. */+export function rangeHeaderForLayer(header, preamble, level) {+ const { start, end } = rangeForLayer(header, preamble, level);+ return `bytes=${start}-${end}`;+}diff --git a/packages/avatar-stream/src/index.js b/packages/avatar-stream/src/index.jsnew file mode 100644index 000000000..a8b64d83e--- /dev/null+++ b/packages/avatar-stream/src/index.js@@ -0,0 +1,24 @@+/**+ * @three-ws/avatar-stream - progressive 3D over plain HTTP.+ *+ * This entry is isomorphic and dependency-free: the container format and the+ * streaming reader, nothing that assumes Node or a GPU. Import `./three` for the+ * renderer binding, or `./node` for the packer.+ */++export {+ MAGIC_STRING,+ PREAMBLE_BYTES,+ FORMAT_VERSION,+ VERSION_TAG,+ RECOMMENDED_PREFIX_BYTES,+ align4,+ decodePreamble,+ encodePreamble,+ decodeHeader,+ encodeContainer,+ rangeForLayer,+ rangeHeaderForLayer,+} from './format.js';++export { A3SStream, bytesSource, httpSource } from './reader.js';diff --git a/packages/avatar-stream/src/node.js b/packages/avatar-stream/src/node.jsnew file mode 100644index 000000000..4f6a5306a--- /dev/null+++ b/packages/avatar-stream/src/node.js@@ -0,0 +1,10 @@+/**+ * Node-only entry: packing and headless reconstruction.+ *+ * Kept separate from the isomorphic entry so a browser bundle never pulls in+ * sharp, draco, or a glTF writer it will not run.+ */++export { pack, getIO, buildLodChain, computeVertexOrder, verifyNesting, DEFAULT_LEVELS, DEFAULT_BASE_TEXTURE_SIZE } from './pack.js';+export { reconstruct, readBaseGeometry, triangleCount, triangleFingerprint } from './reconstruct.js';+export * from './index.js';diff --git a/packages/avatar-stream/src/pack.js b/packages/avatar-stream/src/pack.jsnew file mode 100644index 000000000..a2785b8d2--- /dev/null+++ b/packages/avatar-stream/src/pack.js@@ -0,0 +1,535 @@+/**+ * A3S packer - turns one GLB into a progressive A3S stream. Node only.+ *+ * The packer's whole job is to earn the format's central promise: that a prefix+ * of the file is a complete asset. It does that in four moves.+ *+ * 1. Build a level-of-detail chain by simplifying *successively*, finest to+ * coarsest. Each step collapses edges of the previous step's index buffer,+ * so the vertices surviving at level k are a strict subset of those at+ * level k+1. That nesting is not incidental, it is the property the whole+ * format rests on, and `verifyNesting` asserts it rather than assuming it.+ * 2. Rank every vertex by the coarsest level it survives into, then reorder+ * the vertex buffer by that rank. Now level k's vertices are exactly the+ * first V(k) entries, and a patch can append instead of rewrite.+ * 3. Emit level 0 as a standalone, spec-valid GLB carrying the full node+ * hierarchy and skin, so the coarse mesh is already skinned and posed.+ * 4. Emit each finer level as a patch: the newly revealed vertex bytes plus a+ * replacement index buffer.+ *+ * Skinning survives all of this for free: meshopt's simplifier only ever emits a+ * new index buffer over the original vertex array, so JOINTS_0 and WEIGHTS_0+ * ride along untouched. Morph targets are carried explicitly, reordered and+ * sliced in lockstep with the base attributes they deform.+ */++import { createHash } from 'node:crypto';+import { NodeIO } from '@gltf-transform/core';+import { ALL_EXTENSIONS } from '@gltf-transform/extensions';+import { cloneDocument, dequantize, prune, weldPrimitive } from '@gltf-transform/functions';+import { MeshoptSimplifier, MeshoptDecoder, MeshoptEncoder } from 'meshoptimizer';+import draco3d from 'draco3dgltf';+import sharp from 'sharp';++import { VERSION_TAG, encodeContainer, decodeHeader, decodePreamble, align4 } from './format.js';++/** Default level ratios, coarsest first. The trailing 1.0 level is the original mesh. */+export const DEFAULT_LEVELS = [0.03, 0.1, 0.3, 1.0];++/** Longest edge, in pixels, of the textures baked into the base layer. */+export const DEFAULT_BASE_TEXTURE_SIZE = 64;++/** glTF primitive mode for triangles; the only mode a mesh simplifier can reason about. */+const MODE_TRIANGLES = 4;++const sha256 = (bytes) => createHash('sha256').update(bytes).digest('hex');++let ioPromise = null;+/** Build (once) a NodeIO wired for every extension the platform's avatars use. */+export async function getIO() {+ if (!ioPromise) {+ ioPromise = (async () => {+ await MeshoptDecoder.ready;+ await MeshoptEncoder.ready;+ return new NodeIO().registerExtensions(ALL_EXTENSIONS).registerDependencies({+ 'draco3d.decoder': await draco3d.createDecoderModule(),+ 'draco3d.encoder': await draco3d.createEncoderModule(),+ 'meshopt.decoder': MeshoptDecoder,+ 'meshopt.encoder': MeshoptEncoder,+ });+ })();+ }+ return ioPromise;+}++/**+ * Assert that a chain of index buffers has strictly nested vertex sets, coarsest+ * first. A packer that skipped this could still produce a plausible-looking file+ * whose patches silently reference vertices the client was never sent.+ */+export function verifyNesting(chain) {+ const sets = chain.map((indices) => new Set(indices));+ for (let i = 0; i < sets.length - 1; i++) {+ for (const v of sets[i]) {+ if (!sets[i + 1].has(v)) {+ throw new Error(`a3s: level ${i} is not nested inside level ${i + 1}; cannot pack progressively`);+ }+ }+ }+ return sets;+}++/**+ * Build the coarse-to-fine index chain for one primitive.+ * @returns {Uint32Array[]} index buffers, coarsest first, finest (original) last+ */+export function buildLodChain(indices, positions, levels) {+ const original = indices instanceof Uint32Array ? indices : new Uint32Array(indices);+ const triangles = original.length / 3;+ // Walk fine -> coarse so each simplification consumes the previous result,+ // which is what makes the surviving vertex sets nest.+ const descending = levels.filter((r) => r < 1).sort((a, b) => b - a);+ const coarseToFine = [];+ let current = original;+ for (const ratio of descending) {+ const target = Math.max(1, Math.floor(triangles * ratio)) * 3;+ if (target >= current.length) continue;+ const [simplified] = MeshoptSimplifier.simplify(current, positions, 3, target, 1.0, ['LockBorder']);+ const next = simplified instanceof Uint32Array ? simplified : new Uint32Array(simplified);+ // A tiny primitive can collapse to nothing. An empty level would slice its+ // accessors to count 0, which is invalid glTF, so keep the finer buffer+ // rather than emitting a level that renders as a hole.+ if (next.length < 3) continue;+ current = next;+ coarseToFine.unshift(current);+ }+ // Pad so every primitive reports the same number of levels even when a small+ // mesh could not be decimated further; readers index layers positionally.+ while (coarseToFine.length < levels.length - 1) coarseToFine.unshift(coarseToFine[0] || original);+ coarseToFine.push(original);+ return coarseToFine;+}++/**+ * Rank vertices by the coarsest level they appear in, then produce the+ * old->new remap that groups each level's vertices into a contiguous prefix.+ */+export function computeVertexOrder(chain, vertexCount) {+ const RANK_UNUSED = chain.length; // vertices no level references sort last+ const rank = new Uint32Array(vertexCount).fill(RANK_UNUSED);+ for (let level = chain.length - 1; level >= 0; level--) {+ for (const v of chain[level]) rank[v] = level;+ }+ const sorted = Array.from({ length: vertexCount }, (_, i) => i).sort((a, b) => rank[a] - rank[b] || a - b);+ const remap = new Uint32Array(vertexCount);+ sorted.forEach((oldIndex, newIndex) => {+ remap[oldIndex] = newIndex;+ });+ const levelVertexCounts = [];+ let seen = 0;+ for (let level = 0; level < chain.length; level++) {+ while (seen < vertexCount && rank[sorted[seen]] <= level) seen++;+ levelVertexCounts.push(seen);+ }+ return { remap, order: Uint32Array.from(sorted), levelVertexCounts };+}++/** Reorder a flat typed array of `order.length` elements of `stride` components. */+function reorderArray(array, order, stride) {+ const out = new array.constructor(order.length * stride);+ for (let i = 0; i < order.length; i++) {+ const src = order[i] * stride;+ const dst = i * stride;+ for (let c = 0; c < stride; c++) out[dst + c] = array[src + c];+ }+ return out;+}++function indexArrayFor(vertexCount, indices) {+ return vertexCount <= 0xffff ? Uint16Array.from(indices) : Uint32Array.from(indices);+}++/**+ * Every per-vertex accessor a primitive owns, including morph target deltas.+ *+ * Accessors are routinely shared between primitives in a real GLB, so each one+ * is cloned into sole ownership first. Without that, reordering the vertices of+ * one primitive silently scrambles every sibling that pointed at the same+ * buffer, which surfaces much later as "all accessors of the same primitive must+ * have the same count".+ */+function claimVertexAccessors(prim) {+ const entries = [];+ for (const semantic of prim.listSemantics()) {+ const owned = prim.getAttribute(semantic).clone();+ prim.setAttribute(semantic, owned);+ entries.push({ key: semantic, accessor: owned });+ }+ prim.listTargets().forEach((target, t) => {+ for (const semantic of target.listSemantics()) {+ const owned = target.getAttribute(semantic).clone();+ target.setAttribute(semantic, owned);+ entries.push({ key: `targets/${t}/${semantic}`, accessor: owned });+ }+ });+ return entries;+}++async function resizeTexture(bytes, mimeType, maxSize) {+ const image = sharp(Buffer.from(bytes), { failOn: 'none' });+ const meta = await image.metadata();+ const longest = Math.max(meta.width || 0, meta.height || 0);+ if (!longest) return null;+ const pipeline = longest > maxSize ? image.resize({ width: maxSize, height: maxSize, fit: 'inside' }) : image;+ // Re-encode into the source container so the base layer stays loadable in+ // every glTF viewer, not just the ones that negotiated a newer image codec.+ let out;+ if (mimeType === 'image/jpeg') out = await pipeline.jpeg({ quality: 80 }).toBuffer();+ else if (mimeType === 'image/webp') out = await pipeline.webp({ quality: 80 }).toBuffer();+ else out = await pipeline.png({ compressionLevel: 9 }).toBuffer();+ const outMeta = await sharp(out).metadata();+ return { bytes: new Uint8Array(out), width: outMeta.width, height: outMeta.height };+}++/**+ * Pack a GLB into an A3S container.+ *+ * @param {Uint8Array|Buffer} glbBytes+ * @param {object} [options]+ * @param {number[]} [options.levels] LOD ratios, coarsest first, ending at 1.0+ * @param {number} [options.baseTextureSize] longest texture edge in the base layer+ * @param {string} [options.name] source name recorded in the header+ * @returns {Promise<{ container: Uint8Array, header: object, stats: object }>}+ */+export async function pack(glbBytes, options = {}) {+ const levels = options.levels || DEFAULT_LEVELS;+ const baseTextureSize = options.baseTextureSize || DEFAULT_BASE_TEXTURE_SIZE;+ const source = glbBytes instanceof Uint8Array ? glbBytes : new Uint8Array(glbBytes);+ const levelCount = levels.length;++ await MeshoptSimplifier.ready;+ const io = await getIO();+ const doc = await io.readBinary(new Uint8Array(source));+ // Positions arrive quantized on most platform avatars; the simplifier needs+ // real floats, and the base layer is small enough that the trade is cheap.+ await doc.transform(dequantize());++ const root = doc.getRoot();+ const primPlans = [];+ let skipped = 0;++ let primOrdinal = 0;+ for (const mesh of root.listMeshes()) {+ for (const prim of mesh.listPrimitives()) {+ const ordinal = primOrdinal++;+ const position = prim.getAttribute('POSITION');+ // Points, lines and strips have no triangles to collapse. They ride in+ // the base layer at full detail rather than being dropped.+ if (!position || prim.getMode() !== MODE_TRIANGLES) {+ skipped++;+ continue;+ }+ if (!prim.getIndices()) {+ // A non-indexed triangle soup shares no vertices between triangles, so+ // every edge reads as a border and the simplifier can collapse nothing.+ // Welding restores the shared topology the format needs.+ weldPrimitive(prim);+ }+ const vertexCount = prim.getAttribute('POSITION').getCount();+ let sourceIndices = prim.getIndices()?.getArray();+ if (!sourceIndices) {+ sourceIndices = new Uint32Array(vertexCount);+ for (let i = 0; i < vertexCount; i++) sourceIndices[i] = i;+ }+ if (sourceIndices.length < 3) {+ skipped++;+ continue;+ }++ const entries = claimVertexAccessors(prim);+ const positions = prim.getAttribute('POSITION').getArray();+ const chain = buildLodChain(new Uint32Array(sourceIndices), positions, levels);+ verifyNesting(chain);+ const { remap, order, levelVertexCounts } = computeVertexOrder(chain, vertexCount);++ const snapshot = {};+ for (const { key, accessor } of entries) {+ const stride = accessor.getElementSize();+ const reordered = reorderArray(accessor.getArray(), order, stride);+ accessor.setArray(reordered);+ snapshot[key] = {+ array: reordered,+ elementSize: stride,+ componentType: accessor.getComponentType(),+ type: accessor.getType(),+ normalized: accessor.getNormalized(),+ };+ }+ const remapped = chain.map((indices) => {+ const out = new Uint32Array(indices.length);+ for (let i = 0; i < indices.length; i++) out[i] = remap[indices[i]];+ return out;+ });++ // An indexed primitive is required from here on, since every level ships+ // its own index buffer.+ if (!prim.getIndices()) {+ prim.setIndices(doc.createAccessor().setType('SCALAR').setArray(new Uint32Array(sourceIndices)));+ } else {+ prim.setIndices(prim.getIndices().clone());+ }++ prim.setExtras({ ...(prim.getExtras() || {}), a3sPrim: ordinal });+ primPlans.push({ ordinal, prim, entries, snapshot, chain: remapped, levelVertexCounts, vertexCount });+ }+ }++ if (!primPlans.length) {+ throw new Error('a3s: no triangle primitives found; nothing to stream progressively');+ }++ // --- Texture pyramid ------------------------------------------------------+ // The base layer carries thumbnails so it stays tiny; the final layer carries+ // the originals. The format permits more steps; the packer emits two.+ const texturePlans = [];+ const textures = root.listTextures();+ for (let i = 0; i < textures.length; i++) {+ const texture = textures[i];+ const image = texture.getImage();+ const mimeType = texture.getMimeType();+ if (!image || !/^image\/(png|jpeg|webp)$/.test(mimeType || '')) continue;+ const small = await resizeTexture(image, mimeType, baseTextureSize);+ if (!small) continue;+ texturePlans.push({+ index: i,+ mimeType,+ full: new Uint8Array(image),+ small: small.bytes,+ slots: describeTextureSlots(root, texture),+ });+ texture.setImage(small.bytes);+ }++ // --- Layer 0: a standalone, valid GLB ------------------------------------+ for (const plan of primPlans) sliceToLevel(plan, 0);++ // Animation tracks are the heaviest thing in a rigged avatar: on a 65-joint+ // character they routinely outweigh the mesh several times over, and they are+ // the one part nobody can see in the first frame. The base layer keeps the+ // skeleton and sheds the clips, which ship as a geometry-free companion GLB+ // in the first patch. Clip tracks bind to nodes by name, so they drop onto+ // the base skeleton the moment they land.+ let animationGlb = null;+ const animationNames = root.listAnimations().map((a) => a.getName() || null);+ if (animationNames.length) {+ const animDoc = cloneDocument(doc);+ const animRoot = animDoc.getRoot();+ // A node may not keep a `skin` once its `mesh` is gone: that pairing is a+ // hard glTF constraint, and the companion carries clips, not geometry.+ for (const node of animRoot.listNodes()) {+ node.setMesh(null);+ node.setSkin(null);+ }+ for (const skin of animRoot.listSkins()) skin.dispose();+ for (const mesh of animRoot.listMeshes()) mesh.dispose();+ for (const material of animRoot.listMaterials()) material.dispose();+ for (const texture of animRoot.listTextures()) texture.dispose();+ disposeAccessorsExcept(animRoot, collectAnimationAccessors(animRoot));+ await animDoc.transform(prune({ keepAttributes: false, keepLeaves: true }));+ animationGlb = new Uint8Array(await io.writeBinary(animDoc));++ for (const animation of root.listAnimations()) {+ // Disposing an Animation orphans its sampler accessors rather than+ // deleting them, and they are the heaviest thing in the file.+ for (const sampler of animation.listSamplers()) {+ sampler.getInput()?.dispose();+ sampler.getOutput()?.dispose();+ }+ animation.dispose();+ }+ }+ await doc.transform(prune({ keepAttributes: false, keepLeaves: true }));+ const baseGlb = new Uint8Array(await io.writeBinary(doc));++ // --- Layers 1..N: patches -------------------------------------------------+ const patches = [];+ const layerDescriptors = [];+ for (let level = 1; level < levelCount; level++) {+ const chunks = [];+ let cursor = 0;+ const prims = [];+ for (const plan of primPlans) {+ const from = plan.levelVertexCounts[level - 1];+ const to = plan.levelVertexCounts[level];+ const attributes = {};+ for (const [key, meta] of Object.entries(plan.snapshot)) {+ const stride = meta.elementSize;+ const slice = meta.array.subarray(from * stride, to * stride);+ const bytes = new Uint8Array(slice.buffer, slice.byteOffset, slice.byteLength);+ attributes[key] = {+ offset: cursor,+ length: bytes.byteLength,+ componentType: meta.componentType,+ type: meta.type,+ normalized: meta.normalized,+ elementSize: stride,+ };+ chunks.push(bytes);+ cursor = align4(cursor + bytes.byteLength);+ }+ const indices = indexArrayFor(to, plan.chain[level]);+ const indexBytes = new Uint8Array(indices.buffer, indices.byteOffset, indices.byteLength);+ prims.push({+ prim: plan.ordinal,+ newVertexStart: from,+ newVertexCount: to - from,+ vertexCount: to,+ attributes,+ indices: {+ offset: cursor,+ length: indexBytes.byteLength,+ componentType: indices.BYTES_PER_ELEMENT === 2 ? 5123 : 5125,+ count: indices.length,+ },+ });+ chunks.push(indexBytes);+ cursor = align4(cursor + indexBytes.byteLength);+ }++ // The first patch carries the animation bundle: clips matter to a viewer+ // long before the finest triangles do.+ let animationChunk = null;+ if (level === 1 && animationGlb) {+ animationChunk = { offset: cursor, length: animationGlb.byteLength, clips: animationNames };+ chunks.push(animationGlb);+ cursor = align4(cursor + animationGlb.byteLength);+ }++ // The finest level also delivers the full-resolution textures.+ const texturePatches = [];+ if (level === levelCount - 1) {+ for (const plan of texturePlans) {+ texturePatches.push({+ texture: plan.index,+ offset: cursor,+ length: plan.full.byteLength,+ mimeType: plan.mimeType,+ slots: plan.slots,+ });+ chunks.push(plan.full);+ cursor = align4(cursor + plan.full.byteLength);+ }+ }++ const payload = new Uint8Array(cursor);+ let write = 0;+ for (const chunk of chunks) {+ payload.set(chunk, write);+ write = align4(write + chunk.byteLength);+ }+ patches.push(payload);+ layerDescriptors.push({+ level,+ kind: 'patch',+ sha256: sha256(payload),+ triangleCount: primPlans.reduce((n, p) => n + p.chain[level].length / 3, 0),+ vertexCount: primPlans.reduce((n, p) => n + p.levelVertexCounts[level], 0),+ prims,+ textures: texturePatches,+ animations: animationChunk,+ });+ }++ const header = {+ version: VERSION_TAG,+ generator: '@three-ws/avatar-stream',+ source: {+ name: options.name || null,+ sha256: sha256(source),+ byteLength: source.byteLength,+ },+ geometry: {+ vertexCount: primPlans.reduce((n, p) => n + p.vertexCount, 0),+ triangleCount: primPlans.reduce((n, p) => n + p.chain[levelCount - 1].length / 3, 0),+ primitiveCount: primPlans.length,+ passthroughPrimitiveCount: skipped,+ },+ levels,+ layers: [+ {+ level: 0,+ kind: 'base',+ sha256: sha256(baseGlb),+ triangleCount: primPlans.reduce((n, p) => n + p.chain[0].length / 3, 0),+ vertexCount: primPlans.reduce((n, p) => n + p.levelVertexCounts[0], 0),+ },+ ...layerDescriptors,+ ],+ };++ const container = encodeContainer({ header, baseGlb, patches });+ // Read the offsets straight back out of the encoded bytes, so what the caller+ // inspects is exactly what a client will parse, never a hopeful copy.+ const encodedHeader = decodeHeader(container, decodePreamble(container));+ return {+ container,+ header: encodedHeader,+ stats: {+ sourceBytes: source.byteLength,+ containerBytes: container.byteLength,+ baseBytes: baseGlb.byteLength,+ baseTriangles: header.layers[0].triangleCount,+ fullTriangles: header.geometry.triangleCount,+ layerCount: levelCount,+ },+ };+}++/** Every accessor an animation clip needs in order to play. */+function collectAnimationAccessors(root) {+ const keep = new Set();+ for (const animation of root.listAnimations()) {+ for (const sampler of animation.listSamplers()) {+ const input = sampler.getInput();+ const output = sampler.getOutput();+ if (input) keep.add(input);+ if (output) keep.add(output);+ }+ }+ return keep;+}++/** Drop every accessor the document owns that is not in `keep`. */+function disposeAccessorsExcept(root, keep) {+ for (const accessor of root.listAccessors()) {+ if (!keep.has(accessor)) accessor.dispose();+ }+}++/** Map a texture to the material slots that reference it, for client-side swap-in. */+function describeTextureSlots(root, texture) {+ const slots = [];+ for (const material of root.listMaterials()) {+ const pairs = [+ ['baseColorTexture', material.getBaseColorTexture()],+ ['metallicRoughnessTexture', material.getMetallicRoughnessTexture()],+ ['normalTexture', material.getNormalTexture()],+ ['occlusionTexture', material.getOcclusionTexture()],+ ['emissiveTexture', material.getEmissiveTexture()],+ ];+ for (const [slot, tex] of pairs) {+ if (tex === texture) slots.push({ material: material.getName() || null, slot });+ }+ }+ return slots;+}++/** Slice one primitive's accessors, morph targets included, down to a level. */+function sliceToLevel(plan, level) {+ const keep = plan.levelVertexCounts[level];+ for (const { key, accessor } of plan.entries) {+ const meta = plan.snapshot[key];+ accessor.setArray(meta.array.slice(0, keep * meta.elementSize));+ }+ plan.prim.getIndices().setArray(indexArrayFor(keep, plan.chain[level]));+}diff --git a/packages/avatar-stream/src/reader.js b/packages/avatar-stream/src/reader.jsnew file mode 100644index 000000000..2d5af3c46--- /dev/null+++ b/packages/avatar-stream/src/reader.js@@ -0,0 +1,167 @@+/**+ * A3S reader - isomorphic. Runs in a browser against a plain CDN URL, and in+ * Node against bytes you already hold.+ *+ * The reader exists to make the format's one round-trip promise real. Opening a+ * stream issues a single ranged GET for the head of the file, which in practice+ * carries the preamble, the header, and the entire base layer. From that one+ * response the caller already has a complete GLB to render. Everything after is+ * optional refinement the caller pulls when it wants it.+ *+ * No custom server is required anywhere in this path. `Range` is a 1997 HTTP+ * feature that every CDN, object store, and static host already implements.+ */++import { PREAMBLE_BYTES, RECOMMENDED_PREFIX_BYTES, decodeHeader, decodePreamble } from './format.js';++/** A byte source backed by an in-memory buffer. */+export function bytesSource(bytes) {+ const u8 = bytes instanceof Uint8Array ? bytes : new Uint8Array(bytes);+ return {+ size: u8.byteLength,+ async read(start, end) {+ return u8.subarray(start, Math.min(end + 1, u8.byteLength));+ },+ };+}++/**+ * A byte source backed by HTTP range requests.+ *+ * A server that ignores `Range` and returns 200 with the whole body is handled+ * rather than trusted: the full body is sliced locally so the reader still+ * works, just without the bandwidth saving.+ */+export function httpSource(url, { fetch: fetchImpl = globalThis.fetch, headers = {} } = {}) {+ if (typeof fetchImpl !== 'function') {+ throw new Error('a3s: no fetch implementation available');+ }+ return {+ size: null,+ async read(start, end) {+ const response = await fetchImpl(url, {+ headers: { ...headers, Range: `bytes=${start}-${end}` },+ });+ if (!response.ok) {+ throw new Error(`a3s: ${response.status} ${response.statusText} fetching ${url}`);+ }+ const buffer = new Uint8Array(await response.arrayBuffer());+ if (response.status === 206) return buffer;+ // 200: the host served the whole file, so slice out what was asked for.+ return buffer.subarray(start, Math.min(end + 1, buffer.byteLength));+ },+ };+}++/** Hash bytes with SubtleCrypto, which both modern browsers and Node expose. */+async function sha256Hex(bytes) {+ const subtle = globalThis.crypto?.subtle;+ if (!subtle) return null;+ const view = bytes.byteOffset === 0 && bytes.byteLength === bytes.buffer.byteLength ? bytes.buffer : bytes.slice().buffer;+ const digest = await subtle.digest('SHA-256', view);+ return Array.from(new Uint8Array(digest))+ .map((b) => b.toString(16).padStart(2, '0'))+ .join('');+}++export class A3SStream {+ constructor(source, preamble, header, base) {+ this.source = source;+ this.preamble = preamble;+ this.header = header;+ /** @type {Uint8Array} layer 0: a complete, spec-valid GLB */+ this.base = base;+ }++ /**+ * Open a stream and fetch its base layer in a single request.+ *+ * @param {string|Uint8Array|object} target URL, bytes, or a custom source+ * @param {object} [options]+ * @param {number} [options.prefixBytes] how much of the head to request+ * @param {boolean} [options.verify] check the base layer against its recorded hash+ */+ static async open(target, options = {}) {+ const source =+ typeof target === 'string'+ ? httpSource(target, options)+ : target instanceof Uint8Array || target instanceof ArrayBuffer+ ? bytesSource(target)+ : target;+ const prefixBytes = options.prefixBytes || RECOMMENDED_PREFIX_BYTES;++ let head = await source.read(0, prefixBytes - 1);+ if (head.byteLength < PREAMBLE_BYTES) {+ throw new Error('a3s: stream is shorter than a preamble');+ }+ const preamble = decodePreamble(head);++ // One more request only if the first guess undershot the head. For the+ // platform's avatar corpus this branch is not taken.+ const needed = preamble.baseOffset + preamble.baseLength;+ if (head.byteLength < needed) {+ head = await source.read(0, needed - 1);+ }+ const header = decodeHeader(head, preamble);+ const base = head.subarray(preamble.baseOffset, preamble.baseOffset + preamble.baseLength);++ if (options.verify) {+ const actual = await sha256Hex(base);+ const expected = header.layers[0].sha256;+ if (actual && expected && actual !== expected) {+ throw new Error('a3s: base layer failed integrity check');+ }+ }+ return new A3SStream(source, preamble, header, base);+ }++ /** Total number of layers, base included. */+ get layerCount() {+ return this.header.layers.length;+ }++ /** Bytes a client must read to render layer `level`, base included. */+ bytesThroughLayer(level) {+ const layer = this.header.layers[level];+ return layer.offset + layer.length;+ }++ /**+ * Fetch one refinement layer.+ * @param {number} level 1-based; level 0 is already in `.base`+ * @param {object} [options]+ * @param {boolean} [options.verify]+ * @returns {Promise<{ level: number, descriptor: object, payload: Uint8Array }>}+ */+ async layer(level, options = {}) {+ const descriptor = this.header.layers[level];+ if (!descriptor) throw new Error(`a3s: no layer ${level}`);+ if (level === 0) return { level, descriptor, payload: this.base };+ const payload = await this.source.read(descriptor.offset, descriptor.offset + descriptor.length - 1);+ if (options.verify) {+ const actual = await sha256Hex(payload);+ if (actual && descriptor.sha256 && actual !== descriptor.sha256) {+ throw new Error(`a3s: layer ${level} failed integrity check`);+ }+ }+ return { level, descriptor, payload };+ }++ /**+ * Iterate refinement layers coarse to fine, one request at a time, so the+ * caller can render between layers instead of waiting for all of them.+ */+ async *layers(options = {}) {+ for (let level = 1; level < this.layerCount; level++) {+ yield await this.layer(level, options);+ }+ }++ /**+ * Read a patch's chunk as raw bytes, given a descriptor entry that carries+ * `{ offset, length }` relative to the patch payload.+ */+ static chunk(payload, entry) {+ return payload.subarray(entry.offset, entry.offset + entry.length);+ }+}diff --git a/packages/avatar-stream/src/reconstruct.js b/packages/avatar-stream/src/reconstruct.jsnew file mode 100644index 000000000..2d49b0d4a--- /dev/null+++ b/packages/avatar-stream/src/reconstruct.js@@ -0,0 +1,127 @@+/**+ * Headless reconstruction - replays an A3S stream back into plain geometry.+ * Node only.+ *+ * The three.js binding is the path a browser takes, but a format nobody can+ * check outside a GPU context is a format nobody can trust. This module applies+ * the same patches with nothing but typed arrays, which is what lets the test+ * suite prove the round trip is lossless, and what lets `a3s verify --deep`+ * check a file on a server with no renderer anywhere near it.+ */++import { A3SStream } from './reader.js';+import { getIO } from './pack.js';++const COMPONENT_ARRAYS = {+ 5120: Int8Array,+ 5121: Uint8Array,+ 5122: Int16Array,+ 5123: Uint16Array,+ 5125: Uint32Array,+ 5126: Float32Array,+};++function typedArrayFrom(bytes, componentType) {+ const TypedArray = COMPONENT_ARRAYS[componentType];+ if (!TypedArray) throw new Error(`a3s: unknown component type ${componentType}`);+ const copy = bytes.slice();+ return new TypedArray(copy.buffer, copy.byteOffset, copy.byteLength / TypedArray.BYTES_PER_ELEMENT);+}++/**+ * Read the base layer's per-primitive geometry, keyed by the packer's ordinal.+ * @returns {Promise<Map<number, { attributes: Record<string, {array: ArrayLike, elementSize: number}>, indices: Uint32Array }>>}+ */+export async function readBaseGeometry(baseGlb) {+ const io = await getIO();+ const doc = await io.readBinary(new Uint8Array(baseGlb));+ const primitives = new Map();+ let fallbackOrdinal = 0;+ for (const mesh of doc.getRoot().listMeshes()) {+ for (const prim of mesh.listPrimitives()) {+ const ordinal = prim.getExtras()?.a3sPrim ?? fallbackOrdinal;+ fallbackOrdinal++;+ const attributes = {};+ for (const semantic of prim.listSemantics()) {+ const accessor = prim.getAttribute(semantic);+ attributes[semantic] = { array: accessor.getArray(), elementSize: accessor.getElementSize() };+ }+ prim.listTargets().forEach((target, t) => {+ for (const semantic of target.listSemantics()) {+ const accessor = target.getAttribute(semantic);+ attributes[`targets/${t}/${semantic}`] = { array: accessor.getArray(), elementSize: accessor.getElementSize() };+ }+ });+ const indices = prim.getIndices();+ primitives.set(ordinal, {+ attributes,+ indices: indices ? new Uint32Array(indices.getArray()) : new Uint32Array(0),+ });+ }+ }+ return primitives;+}++/**+ * Replay a stream up to `throughLevel` (default: every layer) and return the+ * resulting geometry per primitive.+ *+ * @param {string|Uint8Array|object} target anything A3SStream.open accepts+ * @param {object} [options]+ * @param {number} [options.throughLevel]+ * @param {boolean} [options.verify] check each layer's hash while replaying+ */+export async function reconstruct(target, options = {}) {+ const stream = await A3SStream.open(target, options);+ const primitives = await readBaseGeometry(stream.base);+ const last = options.throughLevel ?? stream.layerCount - 1;++ for (let level = 1; level <= last; level++) {+ const { descriptor, payload } = await stream.layer(level, options);+ for (const entry of descriptor.prims || []) {+ const primitive = primitives.get(entry.prim);+ if (!primitive) continue;+ for (const [key, meta] of Object.entries(entry.attributes)) {+ const incoming = typedArrayFrom(A3SStream.chunk(payload, meta), meta.componentType);+ const stride = meta.elementSize;+ const TypedArray = COMPONENT_ARRAYS[meta.componentType];+ const existing = primitive.attributes[key]?.array;+ const merged = new TypedArray((entry.newVertexStart + entry.newVertexCount) * stride);+ if (existing) merged.set(existing.subarray(0, Math.min(existing.length, entry.newVertexStart * stride)), 0);+ merged.set(incoming.subarray(0, entry.newVertexCount * stride), entry.newVertexStart * stride);+ primitive.attributes[key] = { array: merged, elementSize: stride };+ }+ primitive.indices = new Uint32Array(typedArrayFrom(A3SStream.chunk(payload, entry.indices), entry.indices.componentType));+ }+ }+ return { stream, primitives };+}++/** Total triangles across every reconstructed primitive. */+export function triangleCount(primitives) {+ let total = 0;+ for (const primitive of primitives.values()) total += primitive.indices.length / 3;+ return total;+}++/**+ * A deterministic fingerprint of a primitive's triangles that does not depend on+ * vertex ordering: the sorted list of rounded triangle centroids. Two meshes+ * with the same fingerprint describe the same surface, however their buffers are+ * arranged, which is exactly the invariant the packer's reordering must preserve.+ */+export function triangleFingerprint(positions, indices, precision = 4) {+ const scale = 10 ** precision;+ const centroids = new Array(indices.length / 3);+ for (let t = 0; t < indices.length; t += 3) {+ const a = indices[t] * 3;+ const b = indices[t + 1] * 3;+ const c = indices[t + 2] * 3;+ const x = Math.round(((positions[a] + positions[b] + positions[c]) / 3) * scale) / scale;+ const y = Math.round(((positions[a + 1] + positions[b + 1] + positions[c + 1]) / 3) * scale) / scale;+ const z = Math.round(((positions[a + 2] + positions[b + 2] + positions[c + 2]) / 3) * scale) / scale;+ centroids[t / 3] = `${x},${y},${z}`;+ }+ centroids.sort();+ return centroids;+}diff --git a/packages/avatar-stream/src/three.js b/packages/avatar-stream/src/three.jsnew file mode 100644index 000000000..56dd7efcc--- /dev/null+++ b/packages/avatar-stream/src/three.js@@ -0,0 +1,249 @@+/**+ * three.js binding for A3S - progressive refinement of a live scene.+ *+ * `three` and `GLTFLoader` are injected rather than imported, so this package+ * never pins a renderer version or ships a second copy of three into a host+ * app's bundle. Pass whatever the host already has.+ *+ * const player = await A3SPlayer.load('/api/avatar-stream?id=michelle', {+ * THREE, GLTFLoader,+ * onLayer: (l) => console.log('level', l.level, l.triangleCount, 'tris'),+ * });+ * scene.add(player.scene);+ * await player.refine();+ *+ * Refinement mutates the geometry already on screen: attributes grow in place+ * and the index buffer is swapped. Nothing is re-parsed, no second scene graph+ * is built, and the camera, pose, and animation state all survive untouched.+ */++import { A3SStream } from './reader.js';++const COMPONENT_ARRAYS = {+ 5120: Int8Array,+ 5121: Uint8Array,+ 5122: Int16Array,+ 5123: Uint16Array,+ 5125: Uint32Array,+ 5126: Float32Array,+};++/** glTF attribute semantic to three.js BufferGeometry attribute name. */+const SEMANTIC_TO_ATTRIBUTE = {+ POSITION: 'position',+ NORMAL: 'normal',+ TANGENT: 'tangent',+ TEXCOORD_0: 'uv',+ TEXCOORD_1: 'uv1',+ TEXCOORD_2: 'uv2',+ TEXCOORD_3: 'uv3',+ COLOR_0: 'color',+ JOINTS_0: 'skinIndex',+ WEIGHTS_0: 'skinWeight',+};++/** glTF material texture slot to the three.js material property it feeds. */+const SLOT_TO_PROPERTIES = {+ baseColorTexture: ['map'],+ metallicRoughnessTexture: ['roughnessMap', 'metalnessMap'],+ normalTexture: ['normalMap'],+ occlusionTexture: ['aoMap'],+ emissiveTexture: ['emissiveMap'],+};++const SRGB_SLOTS = new Set(['baseColorTexture', 'emissiveTexture']);++function typedArrayFrom(bytes, componentType) {+ const TypedArray = COMPONENT_ARRAYS[componentType];+ if (!TypedArray) throw new Error(`a3s: unknown component type ${componentType}`);+ // Copy rather than view: a payload sliced out of a larger response can start+ // at a byte offset the typed array constructor will not accept.+ const copy = bytes.slice();+ return new TypedArray(copy.buffer, copy.byteOffset, copy.byteLength / TypedArray.BYTES_PER_ELEMENT);+}++/** Index every streamable primitive in a loaded scene by its packer ordinal. */+export function indexPrimitives(root) {+ const byOrdinal = new Map();+ const ordered = [];+ root.traverse((object) => {+ if (!object.isMesh && !object.isSkinnedMesh) return;+ ordered.push(object);+ const ordinal = object.userData?.a3sPrim;+ if (typeof ordinal === 'number') byOrdinal.set(ordinal, object);+ });+ // The packer tags every primitive it touched via glTF `extras`. If a loader+ // dropped those, fall back to document order, which three.js preserves.+ if (!byOrdinal.size) ordered.forEach((mesh, i) => byOrdinal.set(i, mesh));+ return byOrdinal;+}++/**+ * Grow one geometry attribute so it covers `newVertexStart + newVertexCount`+ * vertices, keeping the bytes the client already holds.+ */+function growAttribute(THREE, geometry, name, meta, bytes, start, count) {+ const TypedArray = COMPONENT_ARRAYS[meta.componentType];+ const stride = meta.elementSize;+ const incoming = typedArrayFrom(bytes, meta.componentType);+ const total = (start + count) * stride;+ const existing = geometry.getAttribute(name);+ const merged = new TypedArray(total);+ if (existing) merged.set(existing.array.subarray(0, Math.min(existing.array.length, start * stride)), 0);+ merged.set(incoming.subarray(0, count * stride), start * stride);+ geometry.setAttribute(name, new THREE.BufferAttribute(merged, stride, meta.normalized));+}++function growMorphAttribute(THREE, geometry, targetIndex, semantic, meta, bytes, start, count) {+ const slot = { POSITION: 'position', NORMAL: 'normal', TANGENT: 'tangent' }[semantic];+ if (!slot) return;+ const stride = meta.elementSize;+ const TypedArray = COMPONENT_ARRAYS[meta.componentType];+ const incoming = typedArrayFrom(bytes, meta.componentType);+ const list = (geometry.morphAttributes[slot] ||= []);+ const existing = list[targetIndex];+ const merged = new TypedArray((start + count) * stride);+ if (existing) merged.set(existing.array.subarray(0, Math.min(existing.array.length, start * stride)), 0);+ merged.set(incoming.subarray(0, count * stride), start * stride);+ list[targetIndex] = new THREE.BufferAttribute(merged, stride, meta.normalized);+}++/** Apply one primitive's patch to its geometry. */+export function applyPrimitivePatch(THREE, mesh, entry, payload) {+ const geometry = mesh.geometry;+ for (const [key, meta] of Object.entries(entry.attributes)) {+ const bytes = A3SStream.chunk(payload, meta);+ const morph = key.match(/^targets\/(\d+)\/(.+)$/);+ if (morph) {+ growMorphAttribute(THREE, geometry, Number(morph[1]), morph[2], meta, bytes, entry.newVertexStart, entry.newVertexCount);+ continue;+ }+ const name = SEMANTIC_TO_ATTRIBUTE[key] || key.toLowerCase();+ growAttribute(THREE, geometry, name, meta, bytes, entry.newVertexStart, entry.newVertexCount);+ }+ const indexBytes = A3SStream.chunk(payload, entry.indices);+ geometry.setIndex(new THREE.BufferAttribute(typedArrayFrom(indexBytes, entry.indices.componentType), 1));+ geometry.computeBoundingSphere();+ geometry.computeBoundingBox();+}++/** Decode a texture patch and bind it to every material slot that referenced it. */+export async function applyTexturePatch(THREE, materialsByName, entry, payload) {+ const bytes = A3SStream.chunk(payload, entry);+ if (typeof createImageBitmap !== 'function') return;+ const bitmap = await createImageBitmap(new Blob([bytes.slice()], { type: entry.mimeType }));+ for (const { material: name, slot } of entry.slots) {+ const properties = SLOT_TO_PROPERTIES[slot];+ if (!properties) continue;+ for (const material of materialsByName.get(name) || []) {+ for (const property of properties) {+ const previous = material[property];+ if (!previous) continue;+ const texture = new THREE.Texture(bitmap);+ texture.flipY = false;+ texture.wrapS = previous.wrapS;+ texture.wrapT = previous.wrapT;+ texture.magFilter = previous.magFilter;+ texture.minFilter = previous.minFilter;+ texture.channel = previous.channel;+ if (SRGB_SLOTS.has(slot) && THREE.SRGBColorSpace) texture.colorSpace = THREE.SRGBColorSpace;+ texture.needsUpdate = true;+ material[property] = texture;+ material.needsUpdate = true;+ previous.dispose?.();+ }+ }+ }+}++function collectMaterials(root) {+ const byName = new Map();+ root.traverse((object) => {+ const materials = Array.isArray(object.material) ? object.material : object.material ? [object.material] : [];+ for (const material of materials) {+ const list = byName.get(material.name) || [];+ list.push(material);+ byName.set(material.name, list);+ }+ });+ return byName;+}++export class A3SPlayer {+ constructor({ THREE, GLTFLoader, stream, gltf, onLayer }) {+ this.THREE = THREE;+ this.GLTFLoader = GLTFLoader;+ this.stream = stream;+ this.gltf = gltf;+ this.scene = gltf.scene;+ /** Clips available so far. The base layer ships none; the first patch adds them. */+ this.animations = gltf.animations || [];+ this.onLayer = onLayer || null;+ this.level = 0;+ this.primitives = indexPrimitives(this.scene);+ this.materials = collectMaterials(this.scene);+ }++ /**+ * Open a stream and parse its base layer into a renderable scene.+ * Resolves as soon as the coarse avatar is ready to add to a scene.+ */+ static async load(target, options = {}) {+ const { THREE, GLTFLoader } = options;+ if (!THREE || !GLTFLoader) throw new Error('a3s: pass { THREE, GLTFLoader } from the host app');+ const stream = await A3SStream.open(target, options);+ const loader = new GLTFLoader();+ const gltf = await loader.parseAsync(stream.base.slice().buffer, '');+ const player = new A3SPlayer({ THREE, GLTFLoader, stream, gltf, onLayer: options.onLayer });+ player.onLayer?.({+ level: 0,+ triangleCount: stream.header.layers[0].triangleCount,+ bytes: stream.base.byteLength,+ clips: player.animations.length,+ });+ return player;+ }++ /** Total triangles once every layer has been applied. */+ get fullTriangleCount() {+ return this.stream.header.geometry.triangleCount;+ }++ /**+ * Pull and apply every remaining layer, coarse to fine, yielding to the+ * renderer between layers so the refinement is visible rather than a stall.+ */+ async refine(options = {}) {+ for await (const { level, descriptor, payload } of this.stream.layers(options)) {+ await this.applyLayer(descriptor, payload);+ this.level = level;+ this.onLayer?.({+ level,+ triangleCount: descriptor.triangleCount,+ bytes: payload.byteLength,+ clips: this.animations.length,+ });+ }+ return this;+ }++ /** Apply a single decoded layer to the live scene. */+ async applyLayer(descriptor, payload) {+ for (const entry of descriptor.prims || []) {+ const mesh = this.primitives.get(entry.prim);+ if (mesh) applyPrimitivePatch(this.THREE, mesh, entry, payload);+ }+ if (descriptor.animations) {+ const bytes = A3SStream.chunk(payload, descriptor.animations);+ const loader = new this.GLTFLoader();+ const companion = await loader.parseAsync(bytes.slice().buffer, '');+ // Clip tracks address nodes by name, so they bind straight onto the+ // skeleton the base layer already put on screen.+ this.animations = [...this.animations, ...companion.animations];+ this.gltf.animations = this.animations;+ }+ for (const entry of descriptor.textures || []) {+ await applyTexturePatch(this.THREE, this.materials, entry, payload);+ }+ }+}diff --git a/packages/avatar-stream/test/format.test.js b/packages/avatar-stream/test/format.test.jsnew file mode 100644index 000000000..134f7b269--- /dev/null+++ b/packages/avatar-stream/test/format.test.js@@ -0,0 +1,51 @@+import test from 'node:test';+import assert from 'node:assert/strict';++import { PREAMBLE_BYTES, decodePreamble, encodePreamble, encodeContainer, decodeHeader, rangeHeaderForLayer, VERSION_TAG, align4 } from '../src/format.js';++test('preamble round-trips through encode and decode', () => {+ const source = { headerOffset: 32, headerLength: 128, baseOffset: 160, baseLength: 2048, layerCount: 4, totalLength: 9000 };+ const decoded = decodePreamble(encodePreamble(source));+ for (const [key, value] of Object.entries(source)) assert.equal(decoded[key], value, key);+});++test('a non-A3S buffer is rejected rather than misread', () => {+ const bytes = new Uint8Array(PREAMBLE_BYTES);+ bytes.set([0x67, 0x6c, 0x54, 0x46], 0); // "glTF"+ assert.throws(() => decodePreamble(bytes), /not an A3S stream/);+});++test('a truncated head is rejected with a length hint', () => {+ assert.throws(() => decodePreamble(new Uint8Array(8)), /at least 32 bytes/);+});++test('align4 rounds up to the next word boundary', () => {+ assert.deepEqual([0, 1, 4, 5, 8].map(align4), [0, 4, 4, 8, 8]);+});++test('the encoder is the single source of truth for layer offsets', () => {+ const base = new Uint8Array(100).fill(1);+ const patch = new Uint8Array(50).fill(2);+ const header = {+ version: VERSION_TAG,+ generator: 'test',+ source: { name: 'x', sha256: 'a'.repeat(64), byteLength: 1 },+ geometry: { vertexCount: 1, triangleCount: 1, primitiveCount: 1 },+ levels: [0.5, 1],+ // Deliberately wrong: the encoder must overwrite these, not trust them.+ layers: [+ { level: 0, kind: 'base', offset: 999999, length: 1 },+ { level: 1, kind: 'patch', offset: 999999, length: 1 },+ ],+ };+ const container = encodeContainer({ header, baseGlb: base, patches: [patch] });+ const preamble = decodePreamble(container);+ const decoded = decodeHeader(container, preamble);++ assert.equal(decoded.layers[0].offset, preamble.baseOffset);+ assert.equal(decoded.layers[0].length, base.byteLength);+ assert.equal(decoded.layers[1].length, patch.byteLength);+ assert.deepEqual(container.subarray(decoded.layers[0].offset, decoded.layers[0].offset + 100), base);+ assert.deepEqual(container.subarray(decoded.layers[1].offset, decoded.layers[1].offset + 50), patch);+ assert.equal(rangeHeaderForLayer(decoded, preamble, 0), `bytes=0-${preamble.baseOffset + preamble.baseLength - 1}`);+});diff --git a/packages/motion/LICENSE b/packages/motion/LICENSEnew file mode 100644index 000000000..b5ae2a4e1--- /dev/null+++ b/packages/motion/LICENSE@@ -0,0 +1,201 @@+ Apache License+ Version 2.0, January 2004+ http://www.apache.org/licenses/++ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION++ 1. Definitions.++ "License" shall mean the terms and conditions for use, reproduction,+ and distribution as defined by Sections 1 through 9 of this document.++ "Licensor" shall mean the copyright owner or entity authorized by+ the copyright owner that is granting the License.++ "Legal Entity" shall mean the union of the acting entity and all+ other entities that control, are controlled by, or are under common+ control with that entity. For the purposes of this definition,+ "control" means (i) the power, direct or indirect, to cause the+ direction or management of such entity, whether by contract or+ otherwise, or (ii) ownership of fifty percent (50%) or more of the+ outstanding shares, or (iii) beneficial ownership of such entity.++ "You" (or "Your") shall mean an individual or Legal Entity+ exercising permissions granted by this License.++ "Source" form shall mean the preferred form for making modifications,+ including but not limited to software source code, documentation+ source, and configuration files.++ "Object" form shall mean any form resulting from mechanical+ transformation or translation of a Source form, including but+ not limited to compiled object code, generated documentation,+ and conversions to other media types.++ "Work" shall mean the work of authorship, whether in Source or+ Object form, made available under the License, as indicated by a+ copyright notice that is included in or attached to the work+ (an example is provided in the Appendix below).++ "Derivative Works" shall mean any work, whether in Source or Object+ form, that is based on (or derived from) the Work and for which the+ editorial revisions, annotations, elaborations, or other modifications+ represent, as a whole, an original work of authorship. For the purposes+ of this License, Derivative Works shall not include works that remain+ separable from, or merely link (or bind by name) to the interfaces of,+ the Work and Derivative Works thereof.++ "Contribution" shall mean any work of authorship, including+ the original version of the Work and any modifications or additions+ to that Work or Derivative Works thereof, that is intentionally+ submitted to Licensor for inclusion in the Work by the copyright owner+ or by an individual or Legal Entity authorized to submit on behalf of+ the copyright owner. For the purposes of this definition, "submitted"+ means any form of electronic, verbal, or written communication sent+ to the Licensor or its representatives, including but not limited to+ communication on electronic mailing lists, source code control systems,+ and issue tracking systems that are managed by, or on behalf of, the+ Licensor for the purpose of discussing and improving the Work, but+ excluding communication that is conspicuously marked or otherwise+ designated in writing by the copyright owner as "Not a Contribution."++ "Contributor" shall mean Licensor and any individual or Legal Entity+ on behalf of whom a Contribution has been received by Licensor and+ subsequently incorporated within the Work.++ 2. Grant of Copyright License. Subject to the terms and conditions of+ this License, each Contributor hereby grants to You a perpetual,+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable+ copyright license to reproduce, prepare Derivative Works of,+ publicly display, publicly perform, sublicense, and distribute the+ Work and such Derivative Works in Source or Object form.++ 3. Grant of Patent License. Subject to the terms and conditions of+ this License, each Contributor hereby grants to You a perpetual,+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable+ (except as stated in this section) patent license to make, have made,+ use, offer to sell, sell, import, and otherwise transfer the Work,+ where such license applies only to those patent claims licensable+ by such Contributor that are necessarily infringed by their+ Contribution(s) alone or by combination of their Contribution(s)+ with the Work to which such Contribution(s) was submitted. If You+ institute patent litigation against any entity (including a+ cross-claim or counterclaim in a lawsuit) alleging that the Work+ or a Contribution incorporated within the Work constitutes direct+ or contributory patent infringement, then any patent licenses+ granted to You under this License for that Work shall terminate+ as of the date such litigation is filed.++ 4. Redistribution. You may reproduce and distribute copies of the+ Work or Derivative Works thereof in any medium, with or without+ modifications, and in Source or Object form, provided that You+ meet the following conditions:++ (a) You must give any other recipients of the Work or+ Derivative Works a copy of this License; and++ (b) You must cause any modified files to carry prominent notices+ stating that You changed the files; and++ (c) You must retain, in the Source form of any Derivative Works+ that You distribute, all copyright, patent, trademark, and+ attribution notices from the Source form of the Work,+ excluding those notices that do not pertain to any part of+ the Derivative Works; and++ (d) If the Work includes a "NOTICE" text file as part of its+ distribution, then any Derivative Works that You distribute must+ include a readable copy of the attribution notices contained+ within such NOTICE file, excluding those notices that do not+ pertain to any part of the Derivative Works, in at least one+ of the following places: within a NOTICE text file distributed+ as part of the Derivative Works; within the Source form or+ documentation, if provided along with the Derivative Works; or,+ within a display generated by the Derivative Works, if and+ wherever such third-party notices normally appear. The contents+ of the NOTICE file are for informational purposes only and+ do not modify the License. You may add Your own attribution+ notices within Derivative Works that You distribute, alongside+ or as an addendum to the NOTICE text from the Work, provided+ that such additional attribution notices cannot be construed+ as modifying the License.++ You may add Your own copyright statement to Your modifications and+ may provide additional or different license terms and conditions+ for use, reproduction, or distribution of Your modifications, or+ for any such Derivative Works as a whole, provided Your use,+ reproduction, and distribution of the Work otherwise complies with+ the conditions stated in this License.++ 5. Submission of Contributions. Unless You explicitly state otherwise,+ any Contribution intentionally submitted for inclusion in the Work+ by You to the Licensor shall be under the terms and conditions of+ this License, without any additional terms or conditions.+ Notwithstanding the above, nothing herein shall supersede or modify+ the terms of any separate license agreement you may have executed+ with Licensor regarding such Contributions.++ 6. Trademarks. This License does not grant permission to use the trade+ names, trademarks, service marks, or product names of the Licensor,+ except as required for reasonable and customary use in describing the+ origin of the Work and reproducing the content of the NOTICE file.++ 7. Disclaimer of Warranty. Unless required by applicable law or+ agreed to in writing, Licensor provides the Work (and each+ Contributor provides its Contributions) on an "AS IS" BASIS,+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or+ implied, including, without limitation, any warranties or conditions+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A+ PARTICULAR PURPOSE. You are solely responsible for determining the+ appropriateness of using or redistributing the Work and assume any+ risks associated with Your exercise of permissions under this License.++ 8. Limitation of Liability. In no event and under no legal theory,+ whether in tort (including negligence), contract, or otherwise,+ unless required by applicable law (such as deliberate and grossly+ negligent acts) or agreed to in writing, shall any Contributor be+ liable to You for damages, including any direct, indirect, special,+ incidental, or consequential damages of any character arising as a+ result of this License or out of the use or inability to use the+ Work (including but not limited to damages for loss of goodwill,+ work stoppage, computer failure or malfunction, or any and all+ other commercial damages or losses), even if such Contributor+ has been advised of the possibility of such damages.++ 9. Accepting Warranty or Additional Liability. While redistributing+ the Work or Derivative Works thereof, You may choose to offer,+ and charge a fee for, acceptance of support, warranty, indemnity,+ or other liability obligations and/or rights consistent with this+ License. However, in accepting such obligations, You may act only+ on Your own behalf and on Your sole responsibility, not on behalf+ of any other Contributor, and only if You agree to indemnify,+ defend, and hold each Contributor harmless for any liability+ incurred by, or claims asserted against, such Contributor by reason+ of your accepting any such warranty or additional liability.++ END OF TERMS AND CONDITIONS++ APPENDIX: How to apply the Apache License to your work.++ To apply the Apache License to your work, attach the following+ boilerplate notice, with the fields enclosed by brackets "[]"+ replaced with your own identifying information. (Don't include+ the brackets!) The text should be enclosed in the appropriate+ comment syntax for the file format. We also recommend that a+ file or class name and description of purpose be included on the+ same "printed page" as the copyright notice for easier+ identification within third-party archives.++ Copyright 2026 nirholas++ Licensed under the Apache License, Version 2.0 (the "License");+ you may not use this file except in compliance with the License.+ You may obtain a copy of the License at++ http://www.apache.org/licenses/LICENSE-2.0++ Unless required by applicable law or agreed to in writing, software+ distributed under the License is distributed on an "AS IS" BASIS,+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+ See the License for the specific language governing permissions and+ limitations under the License.diff --git a/packages/motion/src/rig/anchors.js b/packages/motion/src/rig/anchors.jsnew file mode 100644index 000000000..dfca0ebe7--- /dev/null+++ b/packages/motion/src/rig/anchors.js@@ -0,0 +1,209 @@+// Anatomical anchors: the places on a body a motion can be written against.+//+// A score says "the right hand goes to the chest, a hand's width out and a+// little forward", never an XYZ. Two reasons that matters. It is the only form+// a language model can author reliably, and it survives the body moving: an+// anchor is measured once on the rest skeleton, then carried by whichever bone+// owns it, so "at the chin" is still at the chin after the torso has turned+// forty degrees and the hips have dropped into a crouch.++import { qConj, qRotate, vAdd, vScale, vSub } from './math.js';+import {+ BODY_FORWARD,+ BODY_LEFT,+ BODY_UP,+ GROUND_Y,+ SHOULDER_SPAN,+ hasBone,+ restPos,+ restWorld,+} from './skeleton.js';++const head = restPos('Head');+const neck = restPos('Neck');+const chest = restPos('Spine2');+const waist = restPos('Spine');+const hips = restPos('Hips');+const shoulderL = restPos('LeftArm');+const shoulderR = restPos('RightArm');+const chinY = neck[1] + (head[1] - neck[1]) * 0.55;+// Half the shoulder span: the lateral unit every side-aware anchor is placed in.+const halfSpan = SHOULDER_SPAN / 2;++const lateral = (metres) => vScale(BODY_LEFT, metres);+const ahead = (metres) => vScale(BODY_FORWARD, metres);+const above = (metres) => vScale(BODY_UP, metres);++/**+ * Every anchor, in model space on the reference rig. Side-aware anchors are an+ * object keyed by side; `anchorPoint` picks the one matching the acting limb.+ */+export const ANCHORS = Object.freeze({+ head,+ forehead: vAdd(head, above(0.07)),+ crown: vAdd(head, above(0.14)),+ temple: { Left: vAdd(head, vAdd(lateral(0.09), above(0.03))), Right: vAdd(head, vAdd(lateral(-0.09), above(0.03))) },+ nose: vAdd(head, vAdd(above(0.02), ahead(0.05))),+ mouth: vAdd([0, chinY + 0.04, head[2]], ahead(0.04)),+ chin: vAdd([0, chinY, head[2]], ahead(0.03)),+ neck,+ // Upper chest, where most gesturing lives. Measured up from the chest bone+ // toward the neck: the bone itself sits at mid-torso, and anchoring there+ // hangs every gesture at belly height.+ sternum: vAdd([0, chest[1] + (neck[1] - chest[1]) * 0.6, chest[2]], ahead(0.02)),+ chest: [0, chest[1], chest[2]],+ heart: vAdd([0, chest[1] + (neck[1] - chest[1]) * 0.4, chest[2]], lateral(0.06)),+ belly: [0, (chest[1] + hips[1]) / 2, waist[2]],+ waist: [0, waist[1], waist[2]],+ hips,+ shoulder: { Left: shoulderL, Right: shoulderR },+ hip: {+ Left: vAdd(hips, lateral(halfSpan * 0.55)),+ Right: vAdd(hips, lateral(-halfSpan * 0.55)),+ },+ /** Arms hanging: where a relaxed wrist sits beside the thigh. */+ side: {+ Left: vAdd(hips, vAdd(lateral(halfSpan * 0.92), vAdd(above(-0.12), ahead(0.02)))),+ Right: vAdd(hips, vAdd(lateral(-halfSpan * 0.92), vAdd(above(-0.12), ahead(0.02)))),+ },+ /** Straight out in front at chest height: the offering, pointing, handshake zone. */+ front: vAdd([0, chest[1] + 0.06, chest[2]], ahead(0.34)),+ /** Out to the side at shoulder height: the presenting, shrugging zone. */+ wide: {+ Left: vAdd(shoulderL, vAdd(lateral(0.3), ahead(0.08))),+ Right: vAdd(shoulderR, vAdd(lateral(-0.3), ahead(0.08))),+ },+ /** Arm raised: hailing, waving, celebrating. */+ overhead: {+ Left: vAdd(shoulderL, vAdd(above(0.42), lateral(0.06))),+ Right: vAdd(shoulderR, vAdd(above(0.42), lateral(-0.06))),+ },+ /** Behind the hip: reaching back, hands clasped behind. */+ behind: {+ Left: vAdd(hips, vAdd(lateral(halfSpan * 0.7), vAdd(above(-0.04), ahead(-0.22)))),+ Right: vAdd(hips, vAdd(lateral(-halfSpan * 0.7), vAdd(above(-0.04), ahead(-0.22)))),+ },+ /** Knee height in front: reaching down without the hips moving. */+ knee: {+ Left: vAdd(restPos('LeftLeg'), ahead(0.12)),+ Right: vAdd(restPos('RightLeg'), ahead(0.12)),+ },+ /** On the floor in front of the feet. */+ floor: vAdd([0, GROUND_Y + 0.05, 0], ahead(0.28)),+});++// Which bone carries each anchor, so the anchor follows the body. Anything not+// listed rides the chest, which is the right default for gesture space.+const ANCHOR_BONE = Object.freeze({+ head: 'Head',+ forehead: 'Head',+ crown: 'Head',+ temple: 'Head',+ nose: 'Head',+ mouth: 'Head',+ chin: 'Head',+ neck: 'Neck',+ sternum: 'Spine2',+ chest: 'Spine2',+ heart: 'Spine2',+ front: 'Spine2',+ wide: 'Spine2',+ overhead: 'Spine2',+ belly: 'Spine',+ waist: 'Spine',+ hips: 'Hips',+ hip: 'Hips',+ side: 'Hips',+ behind: 'Hips',+ knee: 'Hips',+ shoulder: 'Spine2',+ // The floor does not follow the body: that is what makes it the floor.+ floor: null,+});++/** Every anchor name a score may use. */+export const ANCHOR_NAMES = Object.freeze(Object.keys(ANCHORS).sort());++/**+ * Resolve an anchor plus a body-relative offset to a point on the REST+ * skeleton. Offsets are in metres: `out` moves away from the body's midline on+ * the acting side, `up` and `forward` do what they say.+ *+ * @param {string|number[]} anchor an ANCHORS key, or an explicit model-space point+ * @param {{ out?: number, up?: number, forward?: number, side?: 'Left'|'Right' }} [offset]+ * @returns {number[]} model-space point on the rest skeleton+ */+export function restAnchor(anchor, offset = {}) {+ const side = offset.side ?? 'Right';+ let base = Array.isArray(anchor) ? anchor : ANCHORS[anchor];+ if (base && !Array.isArray(base)) base = base[side];+ if (!base) throw new Error(`unknown motion anchor "${anchor}"`);+ const out = side === 'Left' ? (offset.out ?? 0) : -(offset.out ?? 0);+ return vAdd(+ base,+ vAdd(lateral(out), vAdd(above(offset.up ?? 0), ahead(offset.forward ?? 0))),+ );+}++/**+ * The same anchor, but placed on a POSED body: the rest point is expressed in+ * its carrier bone's frame and re-read through that bone's current rotation and+ * position. This is what keeps a hand target at the chin when the head turns.+ *+ * @param {import('./pose.js').Pose} pose+ * @param {string|number[]} anchor+ * @param {{ out?: number, up?: number, forward?: number, side?: 'Left'|'Right' }} [offset]+ * @returns {number[]} model-space point on the posed skeleton+ */+export function anchorPoint(pose, anchor, offset = {}) {+ const rest = restAnchor(anchor, offset);+ const bone = Array.isArray(anchor) ? null : ANCHOR_BONE[anchor];+ if (!bone || !hasBone(bone)) {+ // An unattached anchor still rides the root, so a body that walks forward+ // carries its gesture space with it, but a floor anchor stays on the floor.+ if (anchor === 'floor') return vAdd(rest, [pose.rootOffset[0], 0, pose.rootOffset[2]]);+ return vAdd(rest, pose.rootOffset);+ }+ const local = qRotate(qConj(restWorld(bone)), vSub(rest, restPos(bone)));+ return vAdd(pose.worldPos(bone), qRotate(pose.worldQuat(bone), local));+}++/**+ * A body-relative direction resolved against the POSED chest, so "palm forward"+ * means forward for the body rather than forward for the world.+ *+ * @param {import('./pose.js').Pose} pose+ * @param {string} name one of the DIRECTION keys+ * @param {'Left'|'Right'} [side]+ * @returns {number[]} unit world direction+ */+export function bodyDirection(pose, name, side = 'Right') {+ const local = DIRECTIONS[name];+ if (!local) throw new Error(`unknown motion direction "${name}"`);+ const bone = hasBone('Spine2') ? 'Spine2' : 'Hips';+ const mirrored = side === 'Right' && MIRRORED_DIRECTIONS.has(name)+ ? vScale(local, -1)+ : local;+ return qRotate(pose.worldQuat(bone), qRotate(qConj(restWorld(bone)), mirrored));+}++/** Named directions in the body's own frame, for palms, gaze, and pointing. */+export const DIRECTIONS = Object.freeze({+ forward: BODY_FORWARD,+ back: vScale(BODY_FORWARD, -1),+ up: BODY_UP,+ down: vScale(BODY_UP, -1),+ /** Across the body toward the other side. Mirrored per acting hand. */+ in: BODY_LEFT,+ /** Away from the body on the acting side. Mirrored per acting hand. */+ out: vScale(BODY_LEFT, -1),+ left: BODY_LEFT,+ right: vScale(BODY_LEFT, -1),+});++// `in` and `out` are relative to the acting hand; the rest are absolute for the+// body, so a score can say "palm left" and mean the body's left on either hand.+const MIRRORED_DIRECTIONS = new Set(['in', 'out']);++/** A hand's width on the reference rig: the unit small offsets are written in. */+export const HAND_WIDTH = SHOULDER_SPAN * 0.22;diff --git a/packages/motion/src/rig/ik.js b/packages/motion/src/rig/ik.jsnew file mode 100644index 000000000..164682393--- /dev/null+++ b/packages/motion/src/rig/ik.js@@ -0,0 +1,495 @@+// Inverse kinematics for a humanoid: two-bone limb solvers for the arms and+// legs, a distributed spine, a gaze chain, and hand shaping.+//+// Every solver takes world-space intent ("the wrist goes here", "the sole is+// flat on the floor there", "the chest leans 20 degrees") and writes local bone+// rotations. Nothing here reads the score format, so the same solvers serve the+// score compiler, an interactive editor, and a caller doing its own posing.+//+// The limb solvers share one shape on purpose: place the end joint on the+// circle of valid elbow/knee solutions, push that joint toward a pole so the+// bend goes the anatomically correct way, then let the twist fall out of the+// hand or foot orientation rather than leaving it in the wrist or ankle. That+// last step is the difference between a solved limb and a broken-looking one.++import {+ clamp,+ orientQuat,+ qAxisAngle,+ qConj,+ qMul,+ qRotate,+ vAdd,+ vCross,+ vDot,+ vLen,+ vNorm,+ vReject,+ vScale,+ vSub,+} from './math.js';+import {+ BODY_FORWARD,+ BODY_LEFT,+ BODY_UP,+ boneAxis,+ boneLength,+ hasBone,+ palmAxis,+ radialAxis,+ restLocal,+ restPalmWorld,+ restPos,+ restRadialWorld,+ restWorld,+} from './skeleton.js';++/**+ * Two-bone IK, shared by arms and legs.+ *+ * @param {import('./pose.js').Pose} pose mutated in place+ * @param {{ upper: string, lower: string, end: string }} chain bone names+ * @param {number[]} target world position for the end joint+ * @param {number[]} pole world direction the middle joint is pushed toward+ * @param {number} reach 0-1 cap on how straight the limb may lock+ * @returns {{ upper: number[], lower: number[], end: number[], reached: boolean }}+ * the solved joint positions, and whether the target was inside range+ */+export function solveTwoBone(pose, chain, target, pole, reach) {+ const root = pose.worldPos(chain.upper);+ const lu = boneLength(chain.upper);+ const ll = boneLength(chain.lower);++ let toTarget = vSub(target, root);+ let dist = vLen(toTarget);+ const maxReach = (lu + ll) * reach;+ const minReach = Math.abs(lu - ll) + 1e-3;+ const reached = dist <= maxReach && dist >= minReach;+ if (dist < 1e-6) {+ toTarget = [0, -1, 0];+ dist = minReach;+ }+ dist = clamp(dist, minReach, maxReach);+ const toDir = vNorm(toTarget);++ let polePerp = vReject(pole, toDir);+ if (vLen(polePerp) < 1e-6) polePerp = vReject([0, -1, 0], toDir);+ polePerp = vNorm(polePerp);++ // Law of cosines: how far along the root-to-target line the middle joint sits.+ const cosRoot = clamp((lu * lu + dist * dist - ll * ll) / (2 * lu * dist), -1, 1);+ const sinRoot = Math.sqrt(Math.max(0, 1 - cosRoot * cosRoot));+ const middle = vAdd(root, vAdd(vScale(toDir, lu * cosRoot), vScale(polePerp, lu * sinRoot)));+ const end = vAdd(root, vScale(toDir, dist));++ return {+ upper: root,+ lower: middle,+ end,+ reached,+ upperDir: vNorm(vSub(middle, root)),+ lowerDir: vNorm(vSub(end, middle)),+ polePerp,+ };+}++/**+ * Place a `side` wrist at a world point with a natural elbow, then orient the+ * hand.+ *+ * The elbow lands on the circle of valid solutions, pushed toward `pole`. The+ * default drops it down, back, and slightly out, where a relaxed arm's elbow+ * actually sits; a raised elbow is something a motion asks for, never something+ * the solver does on its own.+ *+ * Roll is anatomical: the upper arm twists so the elbow flexes in the plane the+ * pole defines, and the forearm carries the pronation the palm direction+ * implies, so the wrist itself only ever holds the small residual.+ *+ * @param {import('./pose.js').Pose} pose mutated in place+ * @param {'Left'|'Right'} side+ * @param {{ wrist: number[], fingers?: number[], palm?: number[], pole?: number[], reach?: number }} spec+ * @returns {import('./pose.js').Pose} the same pose+ */+export function solveArm(pose, side, spec) {+ const upper = `${side}Arm`;+ const lower = `${side}ForeArm`;+ const hand = `${side}Hand`;+ const outSign = side === 'Right' ? -1 : 1;+ const pole = spec.pole ?? [outSign * 0.42, -1, -0.3];++ const solved = solveTwoBone(+ pose,+ { upper, lower, end: hand },+ spec.wrist,+ pole,+ spec.reach ?? 0.985,+ );++ // The direction the forearm swings toward as the elbow closes: the anterior+ // side of the arm. Falls back to the pole when the arm is nearly straight.+ let flexDir = vReject(solved.lowerDir, solved.upperDir);+ if (vLen(flexDir) < 1e-4) flexDir = vScale(solved.polePerp, -1);+ flexDir = vNorm(flexDir);++ // On this rest pose (arms out, palms down) elbow flexion carries the hand+ // toward its radial side, so that rest direction is the upper arm's flexion+ // reference. Measured, not assumed.+ const upperRefLocal = vNorm(qRotate(qConj(restWorld(upper)), restRadialWorld(side)));+ pose.setWorldQuat(upper, orientQuat(boneAxis(upper), upperRefLocal, solved.upperDir, flexDir));++ const palmTarget = spec.palm ? vNorm(spec.palm) : null;+ const lowerRefLocal = vNorm(qRotate(qConj(restWorld(lower)), restPalmWorld(side)));+ pose.setWorldQuat(+ lower,+ orientQuat(boneAxis(lower), lowerRefLocal, solved.lowerDir, palmTarget ?? flexDir),+ );++ const fingersDir = spec.fingers ? vNorm(spec.fingers) : solved.lowerDir;+ if (palmTarget) {+ pose.setWorldQuat(hand, orientQuat(boneAxis(hand), palmAxis(hand), fingersDir, palmTarget));+ } else {+ pose.aim(hand, fingersDir);+ }+ return pose;+}++/**+ * The measured direction a flat foot points at rest: forward and sloping down+ * from the ankle to the ball, in model space.+ */+export const FOOT_FLAT_DIR = Object.freeze(+ vNorm(vSub(restPos('LeftToeBase'), restPos('LeftFoot'))),+);++// Turn the flat-foot geometry into the two world directions the foot bone is+// oriented by: where the toe points, and which way the sole faces.+function footFrame(spec) {+ if (spec.toe && spec.sole) return { toe: vNorm(spec.toe), sole: vNorm(spec.sole) };+ const yaw = qAxisAngle(BODY_UP, spec.heading ?? 0);+ const flat = qRotate(yaw, FOOT_FLAT_DIR);+ const tilt = PLANT_TILT[spec.plant ?? 'flat'] ?? 0;+ // Roll about the foot's own across-axis, so a raised heel pitches the foot+ // forward rather than swinging it sideways.+ let across = vCross(BODY_UP, flat);+ if (vLen(across) < 1e-6) across = BODY_LEFT;+ const roll = qAxisAngle(across, tilt);+ return {+ toe: vNorm(spec.toe ?? qRotate(roll, flat)),+ sole: vNorm(spec.sole ?? qRotate(roll, vScale(BODY_UP, -1))),+ };+}++/**+ * How far the foot rolls off flat, in degrees, for each way of standing on it.+ * Positive tips the toe down, which is what raising the heel does.+ */+export const PLANT_TILT = Object.freeze({ flat: 0, toe: 26, heel: -22, lift: 12 });++/**+ * Place a `side` ankle at a world point with the knee bending forward, then set+ * the foot down.+ *+ * The foot's resting direction already slopes from ankle to toe, so a flat foot+ * is that measured direction turned by `heading`, not a horizontal one: aiming+ * the foot along the floor plane instead lifts the toe off the ground by the+ * height of the ankle, which is the single most visible thing wrong with a+ * naively solved leg.+ *+ * @param {import('./pose.js').Pose} pose mutated in place+ * @param {'Left'|'Right'} side+ * @param {{+ * ankle: number[],+ * heading?: number,+ * plant?: 'flat'|'toe'|'heel'|'lift',+ * toe?: number[],+ * sole?: number[],+ * pole?: number[],+ * reach?: number,+ * }} spec `heading` turns the foot in degrees toward the body's left; `plant`+ * picks how it meets the floor. `toe`/`sole` override both with explicit world+ * directions when a caller is driving the foot itself.+ * @returns {import('./pose.js').Pose} the same pose+ */+export function solveLeg(pose, side, spec) {+ const upper = `${side}UpLeg`;+ const lower = `${side}Leg`;+ const foot = `${side}Foot`;+ // The knee goes forward and very slightly outward. A knee pole that is purely+ // forward makes a deep squat look pigeon-toed.+ const outSign = side === 'Right' ? -1 : 1;+ const pole = spec.pole ?? vAdd(BODY_FORWARD, vScale(BODY_LEFT, outSign * -0.18));++ const solved = solveTwoBone(+ pose,+ { upper, lower, end: foot },+ spec.ankle,+ pole,+ // Legs lock much straighter than arms: a standing pose should reproduce the+ // rig's own rest, and a visible knee bend at full extension reads as a limp.+ spec.reach ?? 0.9995,+ );++ let flexDir = vReject(solved.lowerDir, solved.upperDir);+ if (vLen(flexDir) < 1e-4) flexDir = vScale(solved.polePerp, -1);+ flexDir = vNorm(flexDir);++ // At rest the knee flexes toward the back of the leg, so the body's backward+ // direction is the thigh's flexion reference.+ const backLocal = vNorm(qRotate(qConj(restWorld(upper)), vScale(BODY_FORWARD, -1)));+ pose.setWorldQuat(upper, orientQuat(boneAxis(upper), backLocal, solved.upperDir, vScale(flexDir, -1)));++ const shinRefLocal = vNorm(qRotate(qConj(restWorld(lower)), vScale(BODY_FORWARD, -1)));+ pose.setWorldQuat(lower, orientQuat(boneAxis(lower), shinRefLocal, solved.lowerDir, vScale(flexDir, -1)));++ // The foot points down its toe; the sole normal resolves the roll. Both are+ // world directions, so a foot stays flat on the floor no matter what the leg+ // above it is doing, which is the whole point of solving the leg this way.+ const { toe: toeDir, sole: soleDir } = footFrame(spec);+ const footAxis = boneAxis(foot);+ const soleLocal = vNorm(qRotate(qConj(restWorld(foot)), vScale(BODY_UP, -1)));+ pose.setWorldQuat(foot, orientQuat(footAxis, soleLocal, toeDir, soleDir));+ return pose;+}++// ── spine, neck, and gaze ──────────────────────────────────────────────────++// A torso bend is shared out along the chain rather than slammed into one+// joint. Lower vertebrae take less: that is what makes a lean read as a spine+// and not as a hinge at the waist.+const SPINE_CHAIN = Object.freeze([+ { bone: 'Hips', share: 0.12 },+ { bone: 'Spine', share: 0.26 },+ { bone: 'Spine1', share: 0.3 },+ { bone: 'Spine2', share: 0.32 },+]);++/**+ * Bend, twist, and side-bend the torso, in degrees, distributed along the spine.+ *+ * @param {import('./pose.js').Pose} pose mutated in place+ * @param {{ lean?: number, twist?: number, sideBend?: number }} spec+ * `lean` is positive forward, `twist` positive toward the body's left,+ * `sideBend` positive leaning left.+ * @returns {import('./pose.js').Pose} the same pose+ */+export function solveSpine(pose, { lean = 0, twist = 0, sideBend = 0 } = {}) {+ if (!lean && !twist && !sideBend) return pose;+ // Rotating about the body's own axes rather than the world's keeps a twist+ // square to the shoulders even when the body has already turned.+ const leanAxis = BODY_LEFT;+ const twistAxis = BODY_UP;+ const bendAxis = BODY_FORWARD;+ for (const { bone, share } of SPINE_CHAIN) {+ if (!hasBone(bone)) continue;+ let world = pose.worldQuat(bone);+ if (lean) world = qMul(qAxisAngle(leanAxis, lean * share), world);+ if (twist) world = qMul(qAxisAngle(twistAxis, twist * share), world);+ if (sideBend) world = qMul(qAxisAngle(bendAxis, sideBend * share), world);+ pose.setWorldQuat(bone, world);+ }+ return pose;+}++/**+ * Turn the whole body about its vertical axis, from the root, so the legs and+ * arms below follow. Degrees, positive toward the body's left.+ */+export function solveTurn(pose, degrees) {+ if (!degrees) return pose;+ pose.setWorldQuat('Hips', qMul(qAxisAngle(BODY_UP, degrees), pose.worldQuat('Hips')));+ return pose;+}++// Neck takes the smaller half of a look: the head does most of it, which is how+// a person actually looks at something, and it keeps the chin off the chest.+const GAZE_CHAIN = Object.freeze([+ { bone: 'Neck', share: 0.38 },+ { bone: 'Head', share: 0.62 },+]);++/**+ * Look `yaw` degrees toward the body's left and `pitch` degrees up, relative to+ * wherever the chest ended up facing.+ *+ * @param {import('./pose.js').Pose} pose mutated in place+ * @param {{ yaw?: number, pitch?: number, roll?: number }} spec+ * @returns {import('./pose.js').Pose} the same pose+ */+export function solveGaze(pose, { yaw = 0, pitch = 0, roll = 0 } = {}) {+ if (!yaw && !pitch && !roll) return pose;+ // Yaw about the chest's own up, pitch about the chest's own left, so a look+ // stays level after the torso has leaned.+ const chest = hasBone('Spine2') ? 'Spine2' : 'Spine';+ const chestQ = pose.worldQuat(chest);+ const chestUp = vNorm(qRotate(chestQ, qRotate(qConj(restWorld(chest)), BODY_UP)));+ const chestLeft = vNorm(qRotate(chestQ, qRotate(qConj(restWorld(chest)), BODY_LEFT)));+ const chestForward = vNorm(vCross(chestUp, chestLeft));+ for (const { bone, share } of GAZE_CHAIN) {+ if (!hasBone(bone)) continue;+ let world = pose.worldQuat(bone);+ if (yaw) world = qMul(qAxisAngle(chestUp, yaw * share), world);+ if (pitch) world = qMul(qAxisAngle(chestLeft, pitch * share), world);+ if (roll) world = qMul(qAxisAngle(chestForward, roll * share), world);+ pose.setWorldQuat(bone, world);+ }+ return pose;+}++/**+ * Where the head is looking under this pose: a unit world direction. Used to+ * check a solved gaze, and by callers that want to aim something else at what+ * the avatar is looking at.+ */+export function gazeDirection(pose) {+ const head = hasBone('Head') ? 'Head' : 'Neck';+ return vNorm(qRotate(pose.worldQuat(head), qRotate(qConj(restWorld(head)), BODY_FORWARD)));+}++// ── hands ──────────────────────────────────────────────────────────────────++export const FINGERS = Object.freeze(['Index', 'Middle', 'Ring', 'Pinky']);++/** The three phalanx bones of one finger, root to tip. */+export function fingerBones(side, finger) {+ return [1, 2, 3].map((j) => `${side}Hand${finger}${j}`);+}++/**+ * Curl a hand. `curl` closes every finger 0 (flat) to 1 (fist); `thumb` closes+ * the thumb independently; `spread` fans the fingers apart. A hand shape is+ * three numbers rather than twenty joint angles, because a motion score is+ * written by a language model and twenty joint angles is not something to write.+ *+ * @param {import('./pose.js').Pose} pose mutated in place+ * @param {'Left'|'Right'} side+ * @param {{ curl?: number, thumb?: number, spread?: number, only?: string[] }} spec+ * `only` restricts the shaping to the named fingers, which is how a pointing+ * hand is built: curl everything, then straighten the index back out.+ * @returns {import('./pose.js').Pose} the same pose+ */+export function shapeHand(pose, side, { curl = 0, thumb = null, spread = 0, only = null } = {}) {+ const closed = clamp(curl, 0, 1);+ const thumbClosed = clamp(thumb ?? closed * 0.75, 0, 1);+ const fan = clamp(spread, -1, 1);++ for (const finger of FINGERS) {+ if (only && !only.includes(finger)) continue;+ const bones = fingerBones(side, finger);+ if (!hasBone(bones[0])) continue;+ // Knuckle, middle, tip: the middle joint closes hardest, which is what+ // gives a curled hand its shape instead of three even bends.+ const perJoint = [72, 88, 62];+ const splayDeg = fan * (FINGERS.indexOf(finger) - 1.5) * 6;+ bones.forEach((bone, j) => {+ if (!hasBone(bone)) return;+ const curlAxis = radialAxis(bone);+ const splayAxis = palmAxis(bone);+ let q = only ? restLocal(bone) : pose.getLocal(bone);+ q = qMul(q, qAxisAngle(curlAxis, side === 'Right' ? -perJoint[j] * closed : perJoint[j] * closed));+ if (j === 0 && splayDeg) q = qMul(q, qAxisAngle(splayAxis, splayDeg));+ pose.setLocal(bone, q);+ });+ }++ if (only) return pose;++ // The thumb opposes across the palm rather than curling in the finger plane.+ const thumbBones = [1, 2, 3].map((j) => `${side}HandThumb${j}`);+ const thumbDeg = [42, 52, 38];+ thumbBones.forEach((bone, j) => {+ if (!hasBone(bone)) return;+ const axis = palmAxis(bone);+ pose.setLocal(+ bone,+ qMul(pose.getLocal(bone), qAxisAngle(axis, side === 'Right' ? thumbDeg[j] * thumbClosed : -thumbDeg[j] * thumbClosed)),+ );+ });+ return pose;+}++// ── measurements ───────────────────────────────────────────────────────────++/**+ * The support polygon's centre under this pose: the midpoint of whatever feet+ * are carrying weight, on the floor. The balance check compares the centre of+ * mass against it.+ */+export function supportCentre(pose, weights = { Left: 0.5, Right: 0.5 }) {+ const total = (weights.Left ?? 0) + (weights.Right ?? 0);+ if (total < 1e-6) return [pose.worldPos('Hips')[0], 0, pose.worldPos('Hips')[2]];+ const left = pose.worldPos('LeftFoot');+ const right = pose.worldPos('RightFoot');+ return [+ (left[0] * (weights.Left ?? 0) + right[0] * (weights.Right ?? 0)) / total,+ 0,+ (left[2] * (weights.Left ?? 0) + right[2] * (weights.Right ?? 0)) / total,+ ];+}++// Segment masses as a fraction of body mass, from the standard anthropometric+// table (Winter, Biomechanics and Motor Control of Human Movement). Enough of+// the body to place the centre of mass honestly without modelling every bone.+const MASS_SEGMENTS = Object.freeze([+ { bone: 'Hips', mass: 0.142 },+ { bone: 'Spine1', mass: 0.216 },+ { bone: 'Spine2', mass: 0.139 },+ { bone: 'Head', mass: 0.081 },+ { bone: 'LeftArm', mass: 0.028 },+ { bone: 'RightArm', mass: 0.028 },+ { bone: 'LeftForeArm', mass: 0.022 },+ { bone: 'RightForeArm', mass: 0.022 },+ { bone: 'LeftUpLeg', mass: 0.1 },+ { bone: 'RightUpLeg', mass: 0.1 },+ { bone: 'LeftLeg', mass: 0.0465 },+ { bone: 'RightLeg', mass: 0.0465 },+ { bone: 'LeftFoot', mass: 0.0145 },+ { bone: 'RightFoot', mass: 0.0145 },+]);++/**+ * The pose's centre of mass in model space, mass-weighted over the segments+ * that carry most of a body's weight.+ */+export function centreOfMass(pose) {+ let total = 0;+ let acc = [0, 0, 0];+ for (const { bone, mass } of MASS_SEGMENTS) {+ if (!hasBone(bone)) continue;+ acc = vAdd(acc, vScale(pose.worldPos(bone), mass));+ total += mass;+ }+ return total > 0 ? vScale(acc, 1 / total) : [0, 0, 0];+}++/**+ * How far the centre of mass sits outside the support base, in metres on the+ * floor plane. Zero means balanced. The score compiler uses it to nudge the+ * hips back over the feet, which is what stops a deep lean from looking like a+ * fall.+ */+export function balanceError(pose, weights) {+ const com = centreOfMass(pose);+ const support = supportCentre(pose, weights);+ return vLen([com[0] - support[0], 0, com[2] - support[2]]);+}++/** The horizontal offset from the support base to the centre of mass. */+export function balanceOffset(pose, weights) {+ const com = centreOfMass(pose);+ const support = supportCentre(pose, weights);+ return [com[0] - support[0], 0, com[2] - support[2]];+}++/** Where the `side` wrist actually landed: the reachability check tests assert on. */+export function wristPosition(pose, side) {+ return pose.worldPos(`${side}Hand`);+}++/** Where the `side` ankle actually landed. */+export function anklePosition(pose, side) {+ return pose.worldPos(`${side}Foot`);+}++/** The rest (bind) position of a bone, re-exported so callers need one import. */+export { restPos, vDot };diff --git a/packages/motion/src/rig/math.js b/packages/motion/src/rig/math.jsnew file mode 100644index 000000000..fc532b72f--- /dev/null+++ b/packages/motion/src/rig/math.js@@ -0,0 +1,184 @@+// Vector and quaternion primitives, in the exact conventions the reference+// skeleton is measured in: right-handed, Y up, quaternions as [x, y, z, w],+// rotations composed parent-first (`qMul(parent, child)`).+//+// Plain arrays rather than classes on purpose. A pose is thousands of these per+// clip, they are compared in tests, serialized into clip JSON, and passed across+// a worker boundary, and an array is already all three of those things.++/** The rotation that does nothing. */+export const IDENTITY = Object.freeze([0, 0, 0, 1]);++export const v3 = (x, y, z) => [x, y, z];++export function vAdd(a, b) {+ return [a[0] + b[0], a[1] + b[1], a[2] + b[2]];+}++export function vSub(a, b) {+ return [a[0] - b[0], a[1] - b[1], a[2] - b[2]];+}++export function vScale(a, k) {+ return [a[0] * k, a[1] * k, a[2] * k];+}++export function vDot(a, b) {+ return a[0] * b[0] + a[1] * b[1] + a[2] * b[2];+}++export function vCross(a, b) {+ return [+ a[1] * b[2] - a[2] * b[1],+ a[2] * b[0] - a[0] * b[2],+ a[0] * b[1] - a[1] * b[0],+ ];+}++export function vLen(a) {+ return Math.hypot(a[0], a[1], a[2]);+}++export function vNorm(a) {+ const len = vLen(a);+ return len < 1e-9 ? [0, 0, 0] : [a[0] / len, a[1] / len, a[2] / len];+}++export function vLerp(a, b, t) {+ return [a[0] + (b[0] - a[0]) * t, a[1] + (b[1] - a[1]) * t, a[2] + (b[2] - a[2]) * t];+}++/** The component of `a` perpendicular to `axis`. */+export function vReject(a, axis) {+ const n = vNorm(axis);+ return vSub(a, vScale(n, vDot(a, n)));+}++/** Hamilton product: the rotation `a` followed by the rotation `b` in a's frame. */+export function qMul(a, b) {+ const [ax, ay, az, aw] = a;+ const [bx, by, bz, bw] = b;+ return [+ aw * bx + ax * bw + ay * bz - az * by,+ aw * by - ax * bz + ay * bw + az * bx,+ aw * bz + ax * by - ay * bx + az * bw,+ aw * bw - ax * bx - ay * by - az * bz,+ ];+}++export function qConj(q) {+ return [-q[0], -q[1], -q[2], q[3]];+}++export function qNorm(q) {+ const len = Math.hypot(q[0], q[1], q[2], q[3]);+ return len < 1e-9 ? [...IDENTITY] : [q[0] / len, q[1] / len, q[2] / len, q[3] / len];+}++export function qAxisAngle(axis, deg) {+ const n = vNorm(axis);+ const half = (deg * Math.PI) / 360;+ const s = Math.sin(half);+ return [n[0] * s, n[1] * s, n[2] * s, Math.cos(half)];+}++export function qRotate(q, v) {+ // v' = v + 2w(q⃗ × v) + 2(q⃗ × (q⃗ × v))+ const u = [q[0], q[1], q[2]];+ const t = vScale(vCross(u, v), 2);+ return vAdd(vAdd(v, vScale(t, q[3])), vCross(u, t));+}++/** The shortest rotation carrying direction `from` onto direction `to`. */+export function qBetween(from, to) {+ const a = vNorm(from);+ const b = vNorm(to);+ const dot = vDot(a, b);+ if (dot > 0.999999) return [...IDENTITY];+ if (dot < -0.999999) {+ // Opposed: any perpendicular axis is a valid 180-degree turn.+ let axis = vCross(a, Math.abs(a[0]) < 0.9 ? [1, 0, 0] : [0, 1, 0]);+ if (vLen(axis) < 1e-6) axis = vCross(a, [0, 0, 1]);+ return qAxisAngle(axis, 180);+ }+ const axis = vCross(a, b);+ return qNorm([axis[0], axis[1], axis[2], 1 + dot]);+}++export function qSlerp(a, b, t) {+ let [bx, by, bz, bw] = b;+ let cos = a[0] * bx + a[1] * by + a[2] * bz + a[3] * bw;+ if (cos < 0) {+ cos = -cos;+ bx = -bx;+ by = -by;+ bz = -bz;+ bw = -bw;+ }+ if (cos > 0.9995) {+ return qNorm([+ a[0] + (bx - a[0]) * t,+ a[1] + (by - a[1]) * t,+ a[2] + (bz - a[2]) * t,+ a[3] + (bw - a[3]) * t,+ ]);+ }+ const theta = Math.acos(cos);+ const sin = Math.sin(theta);+ const wa = Math.sin((1 - t) * theta) / sin;+ const wb = Math.sin(t * theta) / sin;+ return qNorm([a[0] * wa + bx * wb, a[1] * wa + by * wb, a[2] * wa + bz * wb, a[3] * wa + bw * wb]);+}++/** Quaternion from an orthonormal basis given as its three column vectors. */+export function qFromBasis(x, y, z) {+ const m00 = x[0], m10 = x[1], m20 = x[2];+ const m01 = y[0], m11 = y[1], m21 = y[2];+ const m02 = z[0], m12 = z[1], m22 = z[2];+ const trace = m00 + m11 + m22;+ if (trace > 0) {+ const s = 0.5 / Math.sqrt(trace + 1);+ return qNorm([(m21 - m12) * s, (m02 - m20) * s, (m10 - m01) * s, 0.25 / s]);+ }+ if (m00 > m11 && m00 > m22) {+ const s = 2 * Math.sqrt(1 + m00 - m11 - m22);+ return qNorm([0.25 * s, (m01 + m10) / s, (m02 + m20) / s, (m21 - m12) / s]);+ }+ if (m11 > m22) {+ const s = 2 * Math.sqrt(1 + m11 - m00 - m22);+ return qNorm([(m01 + m10) / s, 0.25 * s, (m12 + m21) / s, (m02 - m20) / s]);+ }+ const s = 2 * Math.sqrt(1 + m22 - m00 - m11);+ return qNorm([(m02 + m20) / s, (m12 + m21) / s, 0.25 * s, (m10 - m01) / s]);+}++/**+ * Orthonormal basis from a primary direction and a secondary reference. The+ * primary survives exactly; the reference only resolves the roll about it.+ */+export function basisFrom(primary, reference) {+ const a = vNorm(primary);+ let b = vReject(reference, a);+ if (vLen(b) < 1e-6) {+ // Reference parallel to the primary: any perpendicular keeps the frame+ // well-defined, and the caller's roll was undefined anyway.+ b = vReject(Math.abs(a[1]) < 0.9 ? [0, 1, 0] : [1, 0, 0], a);+ }+ b = vNorm(b);+ return [a, b, vCross(a, b)];+}++/**+ * The rotation carrying the local frame (`axisLocal`, `refLocal`) onto the world+ * frame (`axisWorld`, `refWorld`). The axis pair maps exactly; the reference+ * pair fixes the twist about it.+ */+export function orientQuat(axisLocal, refLocal, axisWorld, refWorld) {+ const [la, lb, lc] = basisFrom(axisLocal, refLocal);+ const [wa, wb, wc] = basisFrom(axisWorld, refWorld);+ return qNorm(qMul(qFromBasis(wa, wb, wc), qConj(qFromBasis(la, lb, lc))));+}++export const clamp = (n, lo, hi) => Math.min(hi, Math.max(lo, n));++export const DEG = Math.PI / 180;diff --git a/packages/motion/src/rig/pose.js b/packages/motion/src/rig/pose.jsnew file mode 100644index 000000000..d742e2f95--- /dev/null+++ b/packages/motion/src/rig/pose.js@@ -0,0 +1,185 @@+// A skeleton pose: per-bone LOCAL rotations plus a root translation, with+// forward kinematics so a caller can ask where a bone ended up and aim bones at+// world targets.+//+// Local rotations are ABSOLUTE, replacing the bind rotation exactly the way an+// AnimationClip quaternion track does, which is what lets a solved pose be+// written straight into a clip with no further conversion.+//+// The root translation is the piece a signing rig never needed and a walking,+// crouching, sitting body cannot do without: the hips move in world space, and+// every descendant follows, so lowering the hips over planted feet bends the+// knees through the leg IK instead of sinking the mesh through the floor.++import {+ IDENTITY,+ orientQuat,+ qAxisAngle,+ qBetween,+ qConj,+ qMul,+ qNorm,+ qRotate,+ qSlerp,+ vAdd,+ vNorm,+ vSub,+} from './math.js';+import {+ boneAxis,+ palmAxis,+ parentOf,+ restLocal,+ restPos,+ restWorld,+} from './skeleton.js';++export class Pose {+ /** @param {Pose|null} [from] copy-construct from another pose */+ constructor(from = null) {+ /** @type {Map<string, number[]>} local rotation per posed bone */+ this.local = new Map(from ? from.local : []);+ /** @type {Map<string, number>} blendshape weights, 0-1 by name */+ this.face = new Map(from ? from.face : []);+ /** @type {number[]} world translation applied to the root bone, in metres */+ this.rootOffset = from ? [...from.rootOffset] : [0, 0, 0];+ this._worldQ = new Map();+ this._worldP = new Map();+ }++ clone() {+ return new Pose(this);+ }++ _invalidate() {+ this._worldQ.clear();+ this._worldP.clear();+ }++ /** Local rotation of `bone`: posed if set, otherwise the rig's rest. */+ getLocal(bone) {+ return this.local.get(bone) ?? restLocal(bone);+ }++ /** Set `bone`'s local rotation directly. */+ setLocal(bone, q) {+ this.local.set(bone, qNorm(q));+ this._invalidate();+ return this;+ }++ /** Rotate `bone` by `deg` about a LOCAL axis, on top of its current rotation. */+ rotateLocal(bone, axis, deg) {+ return this.setLocal(bone, qMul(this.getLocal(bone), qAxisAngle(axis, deg)));+ }++ /** Move the root (and therefore the whole body) to this world offset. */+ setRootOffset(offset) {+ this.rootOffset = [offset[0], offset[1], offset[2]];+ this._invalidate();+ return this;+ }++ /** World (model-space) rotation of `bone` under this pose. */+ worldQuat(bone) {+ let q = this._worldQ.get(bone);+ if (q) return q;+ const parent = parentOf(bone);+ q = parent ? qMul(this.worldQuat(parent), this.getLocal(bone)) : this.getLocal(bone);+ this._worldQ.set(bone, q);+ return q;+ }++ /** World (model-space) position of `bone` under this pose. */+ worldPos(bone) {+ let p = this._worldP.get(bone);+ if (p) return p;+ const parent = parentOf(bone);+ if (!parent) {+ p = vAdd(restPos(bone), this.rootOffset);+ } else {+ // The bone's fixed offset from its parent, carried by the parent's posed+ // rotation. The rig's own bone lengths, never rescaled.+ const offsetLocal = qRotate(qConj(restWorld(parent)), vSub(restPos(bone), restPos(parent)));+ p = vAdd(this.worldPos(parent), qRotate(this.worldQuat(parent), offsetLocal));+ }+ this._worldP.set(bone, p);+ return p;+ }++ /** World direction `bone` points under this pose, toward its segment child. */+ worldDir(bone) {+ return vNorm(qRotate(this.worldQuat(bone), boneAxis(bone)));+ }++ /** Set `bone`'s WORLD rotation, storing the local that produces it. */+ setWorldQuat(bone, qWorld) {+ const parent = parentOf(bone);+ const local = parent ? qMul(qConj(this.worldQuat(parent)), qWorld) : qWorld;+ return this.setLocal(bone, local);+ }++ /**+ * Aim `bone` down `dirWorld`. With no `refWorld` the bone takes the shortest+ * path from where it is, which preserves whatever twist it already carries;+ * with one, the reference resolves the roll about the aim.+ */+ aim(bone, dirWorld, refWorld = null, refLocal = null) {+ if (!refWorld) {+ const current = this.worldDir(bone);+ return this.setWorldQuat(bone, qMul(qBetween(current, dirWorld), this.worldQuat(bone)));+ }+ return this.setWorldQuat(+ bone,+ orientQuat(boneAxis(bone), refLocal ?? palmAxis(bone), dirWorld, refWorld),+ );+ }++ /** Set blendshape weights by name, merged with whatever is already set. */+ setFace(weights) {+ for (const [shape, value] of Object.entries(weights ?? {})) {+ this.face.set(shape, Math.min(1, Math.max(0, value)));+ }+ return this;+ }++ /** Weight of one blendshape under this pose, 0 when unset. */+ getFace(shape) {+ return this.face.get(shape) ?? 0;+ }++ /** Plain `{bone: [x,y,z,w]}` of every posed bone: the clip track payload. */+ locals() {+ return Object.fromEntries(this.local);+ }++ /** True when nothing has been posed and the body is exactly at bind. */+ get isRest() {+ return this.local.size === 0 && this.face.size === 0+ && this.rootOffset[0] === 0 && this.rootOffset[1] === 0 && this.rootOffset[2] === 0;+ }+}++/** A fresh pose at the rig's bind position: the neutral every beat starts from. */+export function restPose() {+ return new Pose();+}++/** Per-bone slerp, per-shape lerp, and root lerp between two poses. */+export function blendPose(a, b, t) {+ const out = new Pose(a);+ const bones = new Set([...a.local.keys(), ...b.local.keys()]);+ for (const bone of bones) out.setLocal(bone, qSlerp(a.getLocal(bone), b.getLocal(bone), t));+ const shapes = new Set([...a.face.keys(), ...b.face.keys()]);+ for (const shape of shapes) {+ out.face.set(shape, a.getFace(shape) + (b.getFace(shape) - a.getFace(shape)) * t);+ }+ out.setRootOffset([+ a.rootOffset[0] + (b.rootOffset[0] - a.rootOffset[0]) * t,+ a.rootOffset[1] + (b.rootOffset[1] - a.rootOffset[1]) * t,+ a.rootOffset[2] + (b.rootOffset[2] - a.rootOffset[2]) * t,+ ]);+ return out;+}++export { IDENTITY };diff --git a/packages/motion/src/rig/skeleton-data.js b/packages/motion/src/rig/skeleton-data.jsnew file mode 100644index 000000000..11add1d5d--- /dev/null+++ b/packages/motion/src/rig/skeleton-data.js@@ -0,0 +1,243 @@+// GENERATED by scripts/build-canonical-rest.mjs from public/avatars/cz.glb.+// Do not edit by hand: re-run the generator, which writes this file and its+// twin at src/animation-canonical-rest.js from the same measurement pass.+//+// The reference skeleton every three.ws motion is authored against: the+// Avaturn rig the platform's ~3,000 clip library was baked on. Four tables+// describe it completely enough to run forward and inverse kinematics in plain+// JS with no renderer attached, which is what lets this package synthesize+// motion in a Node script, a Cloud Run handler, or a browser tab from the same+// code path.+//+// CANONICAL_REST local (bind) rotation per bone, [x,y,z,w]+// CANONICAL_REST_WORLD the same rotations composed to model space+// CANONICAL_REST_POSITION model-space bind position per bone, in metres+// CANONICAL_PARENT nearest canonical ancestor, null at the root+//+// tests/motion-skeleton-parity.test.js fails if this copy and the site copy+// ever disagree, so the published package can never drift from the rig the+// platform actually retargets onto.++export const CANONICAL_REST = Object.freeze({+ Head: [-0.11706881878098431, 9.438845082943482e-9, 3.72654613409334e-8, 0.993123804804428],+ Hips: [0.0010741148761599644, 6.537898515070053e-8, 7.022659074474357e-11, 0.9999994231384476],+ LeftArm: [-0.003089573263103852, 0.08018148459223169, -0.025041829737156785, 0.9964608827384491],+ LeftFoot: [0.5523737034239063, 0.0002518875693487157, 0.0036538113016644914, 0.8335885543726441],+ LeftForeArm: [-2.6826767953574183e-7, 1.2816740072681333e-7, 0.012369379808617674, 0.9999234962951223],+ LeftHand: [-0.0012910117424963744, 0.002537815819541244, 0.010116448936771993, 0.9999447735952517],+ LeftHandIndex1: [0.028803290204768833, 0.011362422123976328, 0.055965724298468224, 0.9979524555518633],+ LeftHandIndex2: [-0.005195237841915152, 0.014315447122751807, 0.00021581682553071412, 0.9998840087229335],+ LeftHandIndex3: [-0.0013151015878908295, -0.016941197343945148, -0.006458334440614796, 0.9998347644768737],+ LeftHandMiddle1: [0.027532617383276645, -0.031607746403852485, 0.016931179482824785, 0.9989775976007271],+ LeftHandMiddle2: [-0.001267981765488788, 0.0010468621884392817, -0.004466175725402764, 0.9999886747239598],+ LeftHandMiddle3: [-0.0015803533040544947, 0.0018352886876119797, 0.0038040706030556736, 0.9999898315711762],+ LeftHandPinky1: [0.009345046842530319, -0.07377447953394943, -0.04207784173776388, 0.9963430390695086],+ LeftHandPinky2: [-0.0019002180814112143, 0.0002770864318134059, 0.0004968185265866393, 0.9999980327809131],+ LeftHandPinky3: [-0.0014157518824255099, -0.005595319155644295, -0.004606740515194791, 0.9999727326242506],+ LeftHandRing1: [0.018207541076892743, -0.03436008716699126, -0.036855349348364555, 0.9985637451229231],+ LeftHandRing2: [-0.0006571286469542868, 0.015477225242721515, 0.01003471962236163, 0.9998296495318929],+ LeftHandRing3: [0.001815607007583071, -0.003099595768579846, 0.0072046170631637615, 0.999967594260012],+ LeftHandThumb1: [0.1900227602341495, -0.058826116539083484, 0.3478783626049766, 0.9161940209594684],+ LeftHandThumb2: [0.01801363066187901, -0.020968588375318502, -0.22063576333056306, 0.9749644544258214],+ LeftHandThumb3: [-0.014498565873099356, 0.03466160908487025, 0.011599642488048602, 0.9992266072970502],+ LeftLeg: [-0.022987686472704008, -0.0006608912898930547, 0.00007960712093047862, 0.9997355266049326],+ LeftShoulder: [0.5181706903502131, 0.5134387363832796, -0.48299308003518876, 0.4843526489955184],+ LeftToeBase: [0.23652217735689587, 0.0006644108661216591, -0.002051152932444189, 0.9716236982227819],+ LeftUpLeg: [-0.00004952055077835167, 0.0032567616062252223, 0.9999946186652781, -0.00039203576089550014],+ Neck: [0.2225107795455935, -2.3352293238692193e-7, -1.9647139012020554e-8, 0.974930229804142],+ RightArm: [-0.003089570938390283, -0.08018164882954634, 0.025041835809991725, 0.9964608693782484],+ RightFoot: [0.5523737112340288, -0.00025188760500252676, -0.0036538107149212758, 0.8335885491999105],+ RightForeArm: [-2.706717415294191e-7, 6.0675494372234465e-9, -0.012369382961452784, 0.9999234962561284],+ RightHand: [-0.0012910114923260908, -0.002537815924782367, -0.010116450486313036, 0.9999447735796301],+ RightHandIndex1: [0.028803284769926347, -0.01136242475457741, -0.05596571258489258, 0.9979524563349205],+ RightHandIndex2: [-0.005195229421057241, -0.014315442233259087, -0.00021582506177354608, 0.9998840088349297],+ RightHandIndex3: [-0.0013151045827228072, 0.016941319655899355, 0.006458327385746035, 0.9998347624459482],+ RightHandMiddle1: [0.027532662650122483, 0.03160778647114273, -0.01693120529165708, 0.9989775946478099],+ RightHandMiddle2: [-0.0012680273836480952, -0.0010469047250576836, 0.004466199390698661, 0.9999886745158897],+ RightHandMiddle3: [-0.0015803562094451136, -0.001835288773706009, -0.003804068167924464, 0.9999898315756854],+ RightHandPinky1: [0.009345030274151258, 0.07377446325595405, 0.04207784564870347, 0.9963430402658229],+ RightHandPinky2: [-0.0019002008217559384, -0.0002769620503994165, -0.0004968229018870381, 0.9999980328459891],+ RightHandPinky3: [-0.0014157497333212687, 0.0055951881780210025, 0.004606743315319843, 0.9999727333471813],+ RightHandRing1: [0.01820754441109714, 0.03436008722907556, 0.03685535429214536, 0.9985637448787448],+ RightHandRing2: [-0.0006571335802527884, -0.015477230302375519, -0.010034723368684675, 0.999829649412505],+ RightHandRing3: [0.0018156069598940002, 0.003099598646436032, -0.007204626788931167, 0.9999675941811231],+ RightHandThumb1: [0.19002276085311107, 0.05882618704962336, -0.3478783387552461, 0.9161940229953188],+ RightHandThumb2: [0.018013663274561078, 0.020968550782212388, 0.22063584956274163, 0.9749644350943654],+ RightHandThumb3: [-0.014498582331377526, -0.03466161961137741, -0.011599713293476643, 0.9992266058762234],+ RightLeg: [-0.022987685898989207, 0.0006609402907891899, -0.00007960698692017099, 0.9997355265857409],+ RightShoulder: [0.5181706006117709, -0.5134387953473804, 0.4829931766210871, 0.4843525890882128],+ RightToeBase: [0.23652222014073046, -0.0006644109035597298, 0.0020511526077556528, 0.9716236878087238],+ RightUpLeg: [0.000049438593286575395, 0.003256759408127677, 0.9999946186765544, 0.0003920356017771739],+ Spine: [-0.018332562694809717, -1.4544833124647894e-7, 2.6976261645093277e-10, 0.9998319444511763],+ Spine1: [-0.03815628571181615, 2.0010687323521798e-8, 5.34807122500512e-9, 0.9992717837809081],+ Spine2: [-0.056998031890724846, 1.3546308636882866e-7, -8.861325315468363e-10, 0.9983742907149431],+});++export const CANONICAL_REST_WORLD = Object.freeze({+ Head: [-0.005402766806732544, -1.5440620455356296e-7, 8.34223242528311e-10, 0.9999854049488961],+ Hips: [0.0010741148761599644, 6.537898515070053e-8, 7.022659074474357e-11, 0.9999994231384476],+ LeftArm: [0.4889318420664182, 0.5108287295941205, -0.5108290884863605, 0.48893078190839623],+ LeftFoot: [-0.00007112359875637768, 0.5349085331963127, 0.8448985998862468, -0.004383145577274271],+ LeftForeArm: [0.49521300584988104, 0.5047420653526237, -0.5047420379230214, 0.4952120815979495],+ LeftHand: [0.5009334720626235, 0.5016127781291206, -0.4977959877649929, 0.49964931327533474],+ LeftHandIndex1: [0.5480286232738694, 0.46388966319384123, -0.47756977349452234, 0.506357703809076],+ LeftHandIndex2: [0.5522711480640051, 0.4734474077351049, -0.46714980714345783, 0.5026083894226862],+ LeftHandIndex3: [0.5405471533897316, 0.46903549072233974, -0.4790521335412708, 0.5082553896043159],+ LeftHandMiddle1: [0.5069366564823887, 0.46312011824587895, -0.5184714785961669, 0.5096295836965867],+ LeftHandMiddle2: [0.5047591066222094, 0.46656986579426646, -0.5196237813564338, 0.5074661907650336],+ LeftHandMiddle3: [0.5066805225023663, 0.46639751835165716, -0.5160243364733309, 0.509379123507728],+ LeftHandPinky1: [0.4459394011607033, 0.4793433042957962, -0.5586434331057456, 0.5092009093395086],+ LeftHandPinky2: [0.4453638702749085, 0.48032344738196137, -0.5573549331147079, 0.5101920146245817],+ LeftHandPinky3: [0.4392981168466072, 0.4802964151504903, -0.5615019919023976, 0.5109286011992389],+ LeftHandRing1: [0.47371996160924823, 0.4931227775910745, -0.5418410288696247, 0.4886999364902444],+ LeftHandRing2: [0.48665266892609377, 0.49620490515614146, -0.5291888435031511, 0.48673303168821197],+ LeftHandRing3: [0.48945530931580683, 0.4902132045285772, -0.5280743092981476, 0.4911843263440915],+ LeftHandThumb1: [0.6991138207882592, 0.16132571576360077, -0.40704633814367386, 0.5652673372230799],+ LeftHandThumb2: [0.64766423234777, 0.2923511094450446, -0.5391393932239583, 0.4520957758831027],+ LeftHandThumb3: [0.6626872004096495, 0.3080994487427915, -0.5067905211225114, 0.45725679518685386],+ LeftLeg: [0.0006202111480182034, -0.020805293540463354, 0.9997832430476497, -0.00047118992773536965],+ LeftShoulder: [0.4605450937998788, 0.4559956590278878, -0.537559006794696, 0.5394409087622226],+ LeftToeBase: [-0.002764355526654178, 0.7195640056310856, 0.6944147145329627, -0.0028643286082629046],+ LeftUpLeg: [-0.00004987623525000119, 0.0021826506059020026, 0.9999975399461294, -0.00039198262713485737],+ Neck: [0.11170149382802401, -1.628868597421303e-7, -1.8309282630496044e-8, 0.9937418056399563],+ RightArm: [0.48893190585215524, -0.510828798312402, 0.5108290283825504, 0.48893071200502375],+ RightFoot: [0.00007112360405196955, 0.5349085398019146, 0.844898595705669, 0.004383145306035045],+ RightForeArm: [0.4952130023257164, -0.5047420673116668, 0.5047420415306929, 0.49521208229448166],+ RightHand: [0.5009334695181179, -0.5016127792807403, 0.49779599067075153, 0.49964931459617906],+ RightHandIndex1: [0.5480286139806538, -0.4638896746971551, 0.47756977858103167, 0.5063577013132781],+ RightHandIndex2: [0.5522711446711094, -0.4734474082706649, 0.4671498147721218, 0.5026083883587248],+ RightHandIndex3: [0.5405470934413514, -0.4690354263302402, 0.47905220274050364, 0.5082554503330304],+ RightHandMiddle1: [0.5069366680435416, -0.4631200623546733, 0.5184715098400317, 0.5096295939664325],+ RightHandMiddle2: [0.5047591061928465, -0.46656986723577604, 0.5196237819671778, 0.5074661920185282],+ RightHandMiddle3: [0.5066805195140975, -0.4663975225827095, 0.5160243369168757, 0.5093791249234989],+ RightHandPinky1: [0.44593939683725603, -0.4793433241517071, 0.5586434219640141, 0.5092009094272859],+ RightHandPinky2: [0.4453638073881448, -0.4803233923217868, 0.5573549834847424, 0.5101920690951159],+ RightHandPinky3: [0.4392981269307416, -0.4802964267888911, 0.5615019867791732, 0.5109285899785551],+ RightHandRing1: [0.473719957983008, -0.49312277921221886, 0.541841035805614, 0.4886999335659278],+ RightHandRing2: [0.48665266754747677, -0.4962049100848307, 0.5291888437972645, 0.4867330306200585],+ RightHandRing3: [0.4894553112113337, -0.49021320332208906, 0.5280743062058759, 0.4911843318554558],+ RightHandThumb1: [0.6991137732035737, -0.16132569450370315, 0.40704638897707807, 0.5652673641991798],+ RightHandThumb2: [0.6476641963757893, -0.2923511418798965, 0.5391394672107036, 0.45209571648691005],+ RightHandThumb3: [0.6626871862817486, -0.3080994479425299, 0.5067905521115058, 0.4572567801626834],+ RightLeg: [-0.0006202113471683521, -0.020805295090456907, 0.9997832430151556, 0.00047119017271936915],+ RightShoulder: [0.46054507521936333, -0.45599564278848104, 0.5375590232993736, 0.5394409245169026],+ RightToeBase: [0.0027643555231566678, 0.7195640416379069, 0.6944146772223956, 0.0028643286020973192],+ RightUpLeg: [0.000049925034443819034, 0.002182648459062353, 0.9999975399486299, 0.00039198198974856475],+ Spine: [-0.017258617754264162, -8.00818266828763e-8, 1.3823133775055108e-9, 0.9998510589648868],+ Spine1: [-0.055396652424023016, -5.997624651397324e-8, 3.32759956947242e-9, 0.9984644264570511],+ Spine2: [-0.11221710079276989, 7.513757486198981e-8, -8.485311473269302e-9, 0.9936837134066652],+});++// Positions and parentage. Together with the rotations above they close the+// skeleton: every bone knows where it starts, which way it points, and who+// carries it.++export const CANONICAL_REST_POSITION = Object.freeze({+ Head: [0, 1.644459, -0.02811],+ Hips: [0, 0.984089, 0.005022],+ LeftArm: [0.204124, 1.50826, -0.034115],+ LeftFoot: [0.095793, 0.088025, -0.010018],+ LeftForeArm: [0.458234, 1.508259, -0.045252],+ LeftHand: [0.725837, 1.508259, -0.050352],+ LeftHandIndex1: [0.815413, 1.499602, -0.024435],+ LeftHandIndex2: [0.847794, 1.497747, -0.020782],+ LeftHandIndex3: [0.872293, 1.496601, -0.017998],+ LeftHandMiddle1: [0.820067, 1.500491, -0.048839],+ LeftHandMiddle2: [0.854745, 1.498698, -0.047572],+ LeftHandMiddle3: [0.881683, 1.49736, -0.046832],+ LeftHandPinky1: [0.810112, 1.495569, -0.08743],+ LeftHandPinky2: [0.8359, 1.495002, -0.089537],+ LeftHandPinky3: [0.856018, 1.494638, -0.091171],+ LeftHandRing1: [0.81666, 1.499944, -0.069059],+ LeftHandRing2: [0.850313, 1.498728, -0.071468],+ LeftHandRing3: [0.87354, 1.497943, -0.072665],+ LeftHandThumb1: [0.761831, 1.487283, -0.024616],+ LeftHandThumb2: [0.783931, 1.477329, -0.003377],+ LeftHandThumb3: [0.806009, 1.466616, 0.003515],+ LeftLeg: [0.09541, 0.505427, 0.007362],+ LeftShoulder: [0.056656, 1.508576, -0.035092],+ LeftToeBase: [0.096919, 0.022281, 0.128921],+ LeftUpLeg: [0.095072, 0.936186, 0.005482],+ Neck: [0, 1.516499, -0.057245],+ RightArm: [-0.204124, 1.50826, -0.034115],+ RightFoot: [-0.095793, 0.088025, -0.010018],+ RightForeArm: [-0.458234, 1.508259, -0.045252],+ RightHand: [-0.725837, 1.508259, -0.050352],+ RightHandIndex1: [-0.815413, 1.499602, -0.024435],+ RightHandIndex2: [-0.847794, 1.497747, -0.020783],+ RightHandIndex3: [-0.872293, 1.496601, -0.017998],+ RightHandMiddle1: [-0.820067, 1.500491, -0.048839],+ RightHandMiddle2: [-0.854745, 1.498698, -0.047572],+ RightHandMiddle3: [-0.881683, 1.49736, -0.046832],+ RightHandPinky1: [-0.810112, 1.495569, -0.08743],+ RightHandPinky2: [-0.8359, 1.495002, -0.089537],+ RightHandPinky3: [-0.856018, 1.494638, -0.091171],+ RightHandRing1: [-0.81666, 1.499944, -0.069059],+ RightHandRing2: [-0.850313, 1.498729, -0.071468],+ RightHandRing3: [-0.87354, 1.497943, -0.072665],+ RightHandThumb1: [-0.761831, 1.487283, -0.024615],+ RightHandThumb2: [-0.783931, 1.477329, -0.003377],+ RightHandThumb3: [-0.806009, 1.466616, 0.003515],+ RightLeg: [-0.09541, 0.505427, 0.007362],+ RightShoulder: [-0.056656, 1.508576, -0.035092],+ RightToeBase: [-0.096919, 0.022281, 0.128921],+ RightUpLeg: [-0.095072, 0.936186, 0.005482],+ Spine: [0, 1.077013, 0.005222],+ Spine1: [0, 1.184494, 0.00151],+ Spine2: [0, 1.330917, -0.014788],+});++export const CANONICAL_PARENT = Object.freeze({+ Head: 'Neck',+ Hips: null,+ LeftArm: 'LeftShoulder',+ LeftFoot: 'LeftLeg',+ LeftForeArm: 'LeftArm',+ LeftHand: 'LeftForeArm',+ LeftHandIndex1: 'LeftHand',+ LeftHandIndex2: 'LeftHandIndex1',+ LeftHandIndex3: 'LeftHandIndex2',+ LeftHandMiddle1: 'LeftHand',+ LeftHandMiddle2: 'LeftHandMiddle1',+ LeftHandMiddle3: 'LeftHandMiddle2',+ LeftHandPinky1: 'LeftHand',+ LeftHandPinky2: 'LeftHandPinky1',+ LeftHandPinky3: 'LeftHandPinky2',+ LeftHandRing1: 'LeftHand',+ LeftHandRing2: 'LeftHandRing1',+ LeftHandRing3: 'LeftHandRing2',+ LeftHandThumb1: 'LeftHand',+ LeftHandThumb2: 'LeftHandThumb1',+ LeftHandThumb3: 'LeftHandThumb2',+ LeftLeg: 'LeftUpLeg',+ LeftShoulder: 'Spine2',+ LeftToeBase: 'LeftFoot',+ LeftUpLeg: 'Hips',+ Neck: 'Spine2',+ RightArm: 'RightShoulder',+ RightFoot: 'RightLeg',+ RightForeArm: 'RightArm',+ RightHand: 'RightForeArm',+ RightHandIndex1: 'RightHand',+ RightHandIndex2: 'RightHandIndex1',+ RightHandIndex3: 'RightHandIndex2',+ RightHandMiddle1: 'RightHand',+ RightHandMiddle2: 'RightHandMiddle1',+ RightHandMiddle3: 'RightHandMiddle2',+ RightHandPinky1: 'RightHand',+ RightHandPinky2: 'RightHandPinky1',+ RightHandPinky3: 'RightHandPinky2',+ RightHandRing1: 'RightHand',+ RightHandRing2: 'RightHandRing1',+ RightHandRing3: 'RightHandRing2',+ RightHandThumb1: 'RightHand',+ RightHandThumb2: 'RightHandThumb1',+ RightHandThumb3: 'RightHandThumb2',+ RightLeg: 'RightUpLeg',+ RightShoulder: 'Spine2',+ RightToeBase: 'RightFoot',+ RightUpLeg: 'Hips',+ Spine: 'Hips',+ Spine1: 'Spine',+ Spine2: 'Spine1',+});diff --git a/packages/motion/src/rig/skeleton.js b/packages/motion/src/rig/skeleton.jsnew file mode 100644index 000000000..b98973220--- /dev/null+++ b/packages/motion/src/rig/skeleton.js@@ -0,0 +1,198 @@+// The reference skeleton, read out of the measured bind pose.+//+// Everything above this file works in anatomy ("the wrist goes at the chin",+// "the hips drop to 60% of standing height"). Everything below it works in bone+// names and quaternions. This module is the translation: it knows which bone+// carries which, how long each segment is, and which way each one points when+// the body is at rest, all measured from the data rather than assumed, so the+// same code survives a re-measured rig.++import {+ CANONICAL_PARENT,+ CANONICAL_REST,+ CANONICAL_REST_POSITION,+ CANONICAL_REST_WORLD,+} from './skeleton-data.js';+import { IDENTITY, qConj, qRotate, vCross, vLen, vNorm, vReject, vScale, vSub } from './math.js';++export { CANONICAL_PARENT, CANONICAL_REST, CANONICAL_REST_POSITION, CANONICAL_REST_WORLD };++/** Nearest canonical ancestor of `bone`, or null at the root. */+export const parentOf = (bone) => CANONICAL_PARENT[bone] ?? null;++/** Local rest (bind) rotation of `bone`. */+export const restLocal = (bone) => CANONICAL_REST[bone] ?? [...IDENTITY];++/** World (model-space) rest rotation of `bone`. */+export const restWorld = (bone) => CANONICAL_REST_WORLD[bone] ?? [...IDENTITY];++/** World (model-space) rest position of `bone`, in metres. */+export const restPos = (bone) => CANONICAL_REST_POSITION[bone] ?? [0, 0, 0];++/** True when the reference rig carries this canonical bone. */+export const hasBone = (bone) => Object.hasOwn(CANONICAL_REST, bone);++/** Every canonical bone name, sorted, for callers that need the full set. */+export const CANONICAL_BONES = Object.freeze(Object.keys(CANONICAL_REST).sort());++/** The root of the skeleton: the one bone with no canonical parent. */+export const ROOT_BONE = CANONICAL_BONES.find((b) => parentOf(b) === null) ?? 'Hips';++const CHILDREN = (() => {+ const map = new Map();+ for (const [bone, parent] of Object.entries(CANONICAL_PARENT)) {+ if (!parent) continue;+ if (!map.has(parent)) map.set(parent, []);+ map.get(parent).push(bone);+ }+ for (const list of map.values()) list.sort();+ return map;+})();++/** Canonical children of `bone`, sorted, empty at a tip. */+export const childrenOf = (bone) => CHILDREN.get(bone) ?? [];++// A bone with several children needs the one that defines its segment: the hand+// points down the middle finger, the hips up the spine, the foot at the toe.+const PREFERRED_CHILD = {+ Hips: 'Spine',+ LeftHand: 'LeftHandMiddle1',+ RightHand: 'RightHandMiddle1',+ LeftFoot: 'LeftToeBase',+ RightFoot: 'RightToeBase',+};++function segmentChild(bone) {+ if (PREFERRED_CHILD[bone] && hasBone(PREFERRED_CHILD[bone])) return PREFERRED_CHILD[bone];+ return childrenOf(bone)[0] ?? null;+}++/** World direction `bone` points at rest: toward its segment child. */+export function restDirWorld(bone) {+ const child = segmentChild(bone);+ if (child) return vNorm(vSub(restPos(child), restPos(bone)));+ const parent = parentOf(bone);+ if (parent) return vNorm(vSub(restPos(bone), restPos(parent)));+ return [0, 1, 0];+}++const _axisCache = new Map();++/**+ * The bone's own axis: the LOCAL direction pointing at its child. Close to+ * [0,1,0] on most humanoid rigs, but measured rather than assumed, because the+ * hand and foot chains on a real rig are not.+ */+export function boneAxis(bone) {+ let axis = _axisCache.get(bone);+ if (!axis) {+ axis = vNorm(qRotate(qConj(restWorld(bone)), restDirWorld(bone)));+ _axisCache.set(bone, axis);+ }+ return axis;+}++const _lengthCache = new Map();++/** Length of `bone`'s segment in metres: the distance to its segment child. */+export function boneLength(bone) {+ let len = _lengthCache.get(bone);+ if (len === undefined) {+ const child = segmentChild(bone);+ len = child ? vLen(vSub(restPos(child), restPos(bone))) : 0;+ _lengthCache.set(bone, len);+ }+ return len;+}++// ── body frame ─────────────────────────────────────────────────────────────++// At the measured rest pose the arms are out to the sides with the palms facing+// down, so the palm normal is the hand frame's axis that points at the floor,+// and the thumb side is what remains. Both are recovered from the finger bones+// rather than hardcoded, and the downward test picks the sign, so a rig+// measured facing the other way still resolves correctly.+const HAND_FRAME = (() => {+ const frame = {};+ for (const side of ['Left', 'Right']) {+ const hand = `${side}Hand`;+ const index = `${side}HandIndex1`;+ const pinky = `${side}HandPinky1`;+ const fingers = restDirWorld(hand);+ const across = hasBone(index) && hasBone(pinky)+ ? vNorm(vSub(restPos(index), restPos(pinky)))+ : [0, 0, 1];+ let palm = vNorm(vCross(fingers, across));+ if (palm[1] > 0) palm = vScale(palm, -1);+ // The thumb sits on the index side of both hands, so the measured+ // index-to-pinky vector IS the thumb side once it is squared up against the+ // other two axes. Taking it from the data rather than from a cross product+ // keeps the frame correct on both hands without a handedness special case,+ // which is exactly where a sign error hides.+ const radial = vNorm(vReject(vReject(across, fingers), palm));+ frame[side] = { palm, radial, fingers };+ }+ return Object.freeze(frame);+})();++/** World palm normal of the `side` hand at rest. */+export const restPalmWorld = (side) => HAND_FRAME[side].palm;++/** World thumb-side direction of the `side` hand at rest. */+export const restRadialWorld = (side) => HAND_FRAME[side].radial;++const sideOf = (bone) => (bone.startsWith('Left') ? 'Left' : 'Right');++/** LOCAL palm normal of a hand or finger bone. */+export function palmAxis(bone) {+ const side = sideOf(bone);+ return vNorm(qRotate(qConj(restWorld(bone)), restPalmWorld(side)));+}++/** LOCAL thumb-side direction of a hand or finger bone. */+export function radialAxis(bone) {+ const side = sideOf(bone);+ return vNorm(qRotate(qConj(restWorld(bone)), restRadialWorld(side)));+}++/**+ * The body's own forward direction in model space: where the chest faces when+ * the skeleton is at rest. Read from the hand frame, so a rig measured facing+ * the other way still resolves correctly.+ */+export const BODY_FORWARD = Object.freeze(vNorm(restRadialWorld('Right')));++/** Model-space up. The rig is measured Y-up; stated once so nothing assumes it. */+export const BODY_UP = Object.freeze([0, 1, 0]);++/**+ * Model-space left. Asserted against the rig rather than assumed: LeftArm sits+ * on the +X side of the reference skeleton, and the opposite cross order would+ * silently name that side "right" and mirror every motion.+ */+export const BODY_LEFT = Object.freeze(vNorm(vCross(BODY_UP, BODY_FORWARD)));++/** Standing hip height in metres: the unit every vertical offset scales by. */+export const STANDING_HIP_HEIGHT = restPos('Hips')[1];++/** Shoulder-to-shoulder width in metres: the unit lateral offsets scale by. */+export const SHOULDER_SPAN = vLen(vSub(restPos('LeftArm'), restPos('RightArm')));++/** The floor, in model space. Every planted foot is placed relative to it. */+export const GROUND_Y = 0;++/**+ * Ankle height above the floor when the foot is flat, in metres. A planted foot+ * targets the ankle here rather than targeting the floor itself, which is what+ * keeps the sole on the ground instead of driving it through.+ */+export const ANKLE_REST_HEIGHT = restPos('LeftFoot')[1] - GROUND_Y;++/** Half the resting distance between the feet: the neutral stance width. */+export const STANCE_HALF_WIDTH = Math.abs(restPos('LeftFoot')[0]);++/** Full arm reach in metres, shoulder to wrist. */+export const ARM_REACH = boneLength('RightArm') + boneLength('RightForeArm');++/** Full leg length in metres, hip to ankle. */+export const LEG_LENGTH = boneLength('RightUpLeg') + boneLength('RightLeg');diff --git a/packages/motion/src/score.js b/packages/motion/src/score.jsnew file mode 100644index 000000000..c5d09e43c--- /dev/null+++ b/packages/motion/src/score.js@@ -0,0 +1,476 @@+// The Motion Score: the wire format between whatever decided what should happen+// and the solver that makes a body do it.+//+// A score is a list of beats. A beat is a body at one instant plus how long it+// takes to get there and how long it stays. Everything in it is anatomy and+// timing, nothing is a bone name or a quaternion, which is what lets a language+// model author one, a person edit one by hand, and a diff of two of them be+// readable.+//+// Validation is deliberately forgiving in one direction and strict in the+// other. Anything a caller can reasonably mean is accepted and normalized+// (a bare string where an object is allowed, degrees out of range, beats out of+// order); anything ambiguous is an error with the path that caused it, because+// a score that half-parses produces a body that half-moves, and that is worse+// than a rejection.++import {+ EASES,+ EFFORTS,+ ELBOW_POLES,+ EXPRESSIONS,+ GAZES,+ HAND_SHAPES,+ POSTURES,+ STANCES,+} from './vocabulary.js';+import { ANCHOR_NAMES, DIRECTIONS } from './rig/anchors.js';+import { PLANT_TILT } from './rig/ik.js';++/** The score format version this build reads and writes. */+export const MOTION_SCORE_VERSION = 1;++/** Hard bounds, so a malformed score cannot ask for an hour-long clip. */+export const LIMITS = Object.freeze({+ maxBeats: 64,+ maxDuration: 60,+ maxBeatSeconds: 12,+ minBeatSeconds: 0.02,+ maxNameLength: 120,+ maxOffset: 1.2,+ maxAngle: 180,+});++const SIDES = ['left', 'right'];+const PLANTS = Object.keys(PLANT_TILT);+const num = (v) => typeof v === 'number' && Number.isFinite(v);+const clamp = (v, lo, hi) => Math.min(hi, Math.max(lo, v));++class ScoreError extends Error {+ constructor(path, message) {+ super(`${path}: ${message}`);+ this.path = path;+ this.name = 'ScoreError';+ }+}++/**+ * Validate and normalize a score. Returns the normalized score, or throws a+ * ScoreError naming the exact path that failed.+ *+ * Normalizing means: every beat carries an absolute `at`, an `in`, a `hold` and+ * a resolved `effort`; every shorthand is expanded to its object form; every+ * angle and offset is inside its limit. After this, the solver never has to ask+ * whether a field is present.+ *+ * @param {object} input+ * @returns {object} the normalized score+ */+export function normalizeScore(input) {+ if (!input || typeof input !== 'object') throw new ScoreError('score', 'must be an object');+ const version = input.version ?? MOTION_SCORE_VERSION;+ if (version !== MOTION_SCORE_VERSION) {+ throw new ScoreError('score.version', `unsupported version ${version}, this build reads ${MOTION_SCORE_VERSION}`);+ }+ const name = typeof input.name === 'string' && input.name.trim()+ ? input.name.trim().slice(0, LIMITS.maxName ?? LIMITS.maxNameLength)+ : 'motion';+ const beatsIn = Array.isArray(input.beats) ? input.beats : null;+ if (!beatsIn || !beatsIn.length) throw new ScoreError('score.beats', 'must be a non-empty array');+ if (beatsIn.length > LIMITS.maxBeats) {+ throw new ScoreError('score.beats', `at most ${LIMITS.maxBeats} beats, got ${beatsIn.length}`);+ }++ const defaultEffort = resolveEffort(input.effort, 'score.effort') ?? { ...EFFORTS.neutral };+ let clock = 0;+ const beats = beatsIn.map((raw, i) => {+ const path = `score.beats[${i}]`;+ if (!raw || typeof raw !== 'object') throw new ScoreError(path, 'must be an object');+ const effort = resolveEffort(raw.effort, `${path}.effort`) ?? defaultEffort;+ // A sudden beat is quicker than a sustained one by default, so timing+ // follows effort unless the beat states its own.+ const travel = num(raw.in)+ ? clamp(raw.in, LIMITS.minBeatSeconds, LIMITS.maxBeatSeconds)+ : (i === 0 ? 0 : defaultTravel(effort));+ const hold = num(raw.hold) ? clamp(raw.hold, 0, LIMITS.maxBeatSeconds) : 0;+ const at = num(raw.at) ? Math.max(clock, raw.at) : clock + travel;+ clock = at + hold;+ if (clock > LIMITS.maxDuration) {+ throw new ScoreError(path, `score runs past the ${LIMITS.maxDuration}s limit`);+ }+ return {+ at,+ in: travel,+ hold,+ ease: pickEase(raw.ease, effort, path),+ effort,+ posture: pickPosture(raw.posture, path),+ root: normalizeRoot(raw.root, path),+ torso: normalizeTorso(raw.torso, path),+ gaze: normalizeGaze(raw.gaze, path),+ arms: normalizeArms(raw.arms, path),+ legs: normalizeLegs(raw.legs, path),+ face: normalizeFace(raw.face, path),+ label: typeof raw.label === 'string' ? raw.label.slice(0, LIMITS.maxNameLength) : null,+ };+ });++ return {+ version: MOTION_SCORE_VERSION,+ name,+ seed: typeof input.seed === 'string' && input.seed ? input.seed : name,+ loop: input.loop === true,+ effort: defaultEffort,+ beats,+ duration: beats[beats.length - 1].at + beats[beats.length - 1].hold,+ };+}++/**+ * Validate without throwing. Returns `{ ok, score, error }`, which is what an+ * HTTP handler and an LLM repair loop both actually want.+ * @param {object} input+ */+export function validateScore(input) {+ try {+ return { ok: true, score: normalizeScore(input), error: null };+ } catch (err) {+ return { ok: false, score: null, error: { path: err.path ?? 'score', message: err.message } };+ }+}++// A sudden effort covers ground fast. The curve is deliberately not linear in+// `time`: the interesting range for a gesture is the middle.+function defaultTravel(effort) {+ return 0.9 - 0.62 * effort.time ** 0.8;+}++function resolveEffort(spec, path) {+ if (spec == null) return null;+ if (typeof spec === 'string') {+ const preset = EFFORTS[spec];+ if (!preset) throw new ScoreError(path, `unknown effort "${spec}"`);+ return { ...preset };+ }+ if (typeof spec !== 'object') throw new ScoreError(path, 'must be a name or an object');+ const base = spec.preset ? EFFORTS[spec.preset] : EFFORTS.neutral;+ if (spec.preset && !base) throw new ScoreError(`${path}.preset`, `unknown effort "${spec.preset}"`);+ return {+ weight: clamp(num(spec.weight) ? spec.weight : base.weight, 0, 1),+ time: clamp(num(spec.time) ? spec.time : base.time, 0, 1),+ flow: clamp(num(spec.flow) ? spec.flow : base.flow, 0, 1),+ };+}++// With no ease named, effort picks one: sudden and bound snaps, sudden and free+// overshoots, sustained settles.+function pickEase(spec, effort, path) {+ if (spec == null) {+ if (effort.time > 0.75) return effort.flow > 0.6 ? 'overshoot' : 'snap';+ if (effort.time < 0.25) return 'settle';+ return 'smooth';+ }+ if (typeof spec !== 'string' || !EASES[spec]) {+ throw new ScoreError(`${path}.ease`, `unknown ease "${spec}"`);+ }+ return spec;+}++function pickPosture(spec, path) {+ if (spec == null) return null;+ if (typeof spec !== 'string' || !POSTURES[spec]) {+ throw new ScoreError(`${path}.posture`, `unknown posture "${spec}"`);+ }+ return spec;+}++function normalizeRoot(spec, path) {+ if (spec == null) return {};+ if (typeof spec !== 'object') throw new ScoreError(`${path}.root`, 'must be an object');+ const out = {};+ if (spec.height != null) out.height = boundedNumber(spec.height, 0.28, 1.15, `${path}.root.height`);+ if (spec.forward != null) out.forward = offset(spec.forward, `${path}.root.forward`);+ if (spec.side != null) out.side = offset(spec.side, `${path}.root.side`);+ if (spec.rise != null) out.rise = offset(spec.rise, `${path}.root.rise`);+ if (spec.turn != null) out.turn = angle(spec.turn, `${path}.root.turn`);+ return out;+}++function normalizeTorso(spec, path) {+ if (spec == null) return {};+ if (typeof spec !== 'object') throw new ScoreError(`${path}.torso`, 'must be an object');+ const out = {};+ for (const key of ['lean', 'twist', 'sideBend']) {+ if (spec[key] != null) out[key] = angle(spec[key], `${path}.torso.${key}`);+ }+ return out;+}++function normalizeGaze(spec, path) {+ if (spec == null) return null;+ if (typeof spec === 'string') {+ if (!GAZES[spec]) throw new ScoreError(`${path}.gaze`, `unknown gaze "${spec}"`);+ return { preset: spec };+ }+ if (typeof spec !== 'object') throw new ScoreError(`${path}.gaze`, 'must be a name or an object');+ const out = {};+ if (spec.preset != null) {+ if (!GAZES[spec.preset]) throw new ScoreError(`${path}.gaze.preset`, `unknown gaze "${spec.preset}"`);+ out.preset = spec.preset;+ }+ for (const key of ['yaw', 'pitch', 'roll']) {+ if (spec[key] != null) out[key] = angle(spec[key], `${path}.gaze.${key}`);+ }+ return out;+}++function normalizeArms(spec, path) {+ if (spec == null) return {};+ if (typeof spec !== 'object') throw new ScoreError(`${path}.arms`, 'must be an object');+ const out = {};+ for (const side of SIDES) {+ if (spec[side] == null) continue;+ out[side] = normalizeArm(spec[side], `${path}.arms.${side}`);+ }+ return out;+}++function normalizeArm(spec, path) {+ if (typeof spec === 'string') return { at: anchorName(spec, path) };+ if (typeof spec !== 'object') throw new ScoreError(path, 'must be an anchor name or an object');+ const out = {};+ if (spec.at != null) {+ if (Array.isArray(spec.at)) {+ if (spec.at.length !== 3 || !spec.at.every(num)) {+ throw new ScoreError(`${path}.at`, 'an explicit point must be three finite numbers');+ }+ out.at = [...spec.at];+ } else {+ out.at = anchorName(spec.at, `${path}.at`);+ }+ }+ for (const key of ['out', 'up', 'forward']) {+ if (spec[key] != null) out[key] = offset(spec[key], `${path}.${key}`);+ }+ if (spec.palm != null) out.palm = directionName(spec.palm, `${path}.palm`);+ if (spec.point != null) out.point = directionName(spec.point, `${path}.point`);+ if (spec.elbow != null) {+ if (typeof spec.elbow !== 'string' || !ELBOW_POLES[spec.elbow]) {+ throw new ScoreError(`${path}.elbow`, `unknown elbow "${spec.elbow}"`);+ }+ out.elbow = spec.elbow;+ }+ if (spec.hand != null) out.hand = normalizeHand(spec.hand, `${path}.hand`);+ return out;+}++function normalizeHand(spec, path) {+ if (typeof spec === 'string') {+ if (!HAND_SHAPES[spec]) throw new ScoreError(path, `unknown hand shape "${spec}"`);+ return { shape: spec };+ }+ if (typeof spec !== 'object') throw new ScoreError(path, 'must be a shape name or an object');+ const out = {};+ if (spec.shape != null) {+ if (!HAND_SHAPES[spec.shape]) throw new ScoreError(`${path}.shape`, `unknown hand shape "${spec.shape}"`);+ out.shape = spec.shape;+ }+ for (const key of ['curl', 'thumb']) {+ if (spec[key] != null) out[key] = boundedNumber(spec[key], 0, 1, `${path}.${key}`);+ }+ if (spec.spread != null) out.spread = boundedNumber(spec.spread, -1, 1, `${path}.spread`);+ return out;+}++function normalizeLegs(spec, path) {+ if (spec == null) return {};+ if (typeof spec === 'string') return { stance: stanceName(spec, `${path}.legs`) };+ if (typeof spec !== 'object') throw new ScoreError(`${path}.legs`, 'must be a stance name or an object');+ const out = {};+ if (spec.stance != null) out.stance = stanceName(spec.stance, `${path}.legs.stance`);+ for (const side of SIDES) {+ if (spec[side] == null) continue;+ out[side] = normalizeFoot(spec[side], `${path}.legs.${side}`);+ }+ return out;+}++function normalizeFoot(spec, path) {+ if (typeof spec !== 'object' || spec === null) throw new ScoreError(path, 'must be an object');+ const out = {};+ for (const key of ['out', 'forward', 'lift']) {+ if (spec[key] != null) out[key] = offset(spec[key], `${path}.${key}`);+ }+ if (spec.heading != null) out.heading = angle(spec.heading, `${path}.heading`);+ if (spec.plant != null) {+ if (typeof spec.plant !== 'string' || !PLANTS.includes(spec.plant)) {+ throw new ScoreError(`${path}.plant`, `unknown plant "${spec.plant}", expected one of ${PLANTS.join(', ')}`);+ }+ out.plant = spec.plant;+ }+ return out;+}++function normalizeFace(spec, path) {+ if (spec == null) return null;+ if (typeof spec === 'string') {+ if (!EXPRESSIONS[spec]) throw new ScoreError(`${path}.face`, `unknown expression "${spec}"`);+ return { [spec]: 1 };+ }+ if (typeof spec !== 'object') throw new ScoreError(`${path}.face`, 'must be an expression name or an object');+ const out = {};+ for (const [key, value] of Object.entries(spec)) {+ if (!num(value)) throw new ScoreError(`${path}.face.${key}`, 'must be a number');+ out[key] = clamp(value, 0, 1);+ }+ return out;+}++function anchorName(value, path) {+ if (typeof value !== 'string' || !ANCHOR_NAMES.includes(value)) {+ throw new ScoreError(path, `unknown anchor "${value}"`);+ }+ return value;+}++function directionName(value, path) {+ if (typeof value !== 'string' || !DIRECTIONS[value]) {+ throw new ScoreError(path, `unknown direction "${value}"`);+ }+ return value;+}++function stanceName(value, path) {+ if (typeof value !== 'string' || !STANCES[value]) {+ throw new ScoreError(path, `unknown stance "${value}"`);+ }+ return value;+}++function boundedNumber(value, lo, hi, path) {+ if (!num(value)) throw new ScoreError(path, 'must be a finite number');+ return clamp(value, lo, hi);+}++function offset(value, path) {+ return boundedNumber(value, -LIMITS.maxOffset, LIMITS.maxOffset, path);+}++function angle(value, path) {+ return boundedNumber(value, -LIMITS.maxAngle, LIMITS.maxAngle, path);+}++/**+ * The JSON Schema for a score, for tool-calling and for editors. Generated from+ * the same vocabulary the solver reads, so the enums a model is offered can+ * never drift from the ones the solver accepts.+ * @returns {object}+ */+export function scoreSchema() {+ const offsetProp = (description) => ({ type: 'number', minimum: -LIMITS.maxOffset, maximum: LIMITS.maxOffset, description });+ const angleProp = (description) => ({ type: 'number', minimum: -LIMITS.maxAngle, maximum: LIMITS.maxAngle, description });+ const armProp = {+ type: 'object',+ additionalProperties: false,+ properties: {+ at: { type: 'string', enum: [...ANCHOR_NAMES], description: 'anatomical place the wrist goes' },+ out: offsetProp('metres away from the midline, on this hand\'s side'),+ up: offsetProp('metres above the anchor'),+ forward: offsetProp('metres in front of the anchor'),+ palm: { type: 'string', enum: Object.keys(DIRECTIONS), description: 'where the palm faces' },+ point: { type: 'string', enum: Object.keys(DIRECTIONS), description: 'where the fingers point' },+ elbow: { type: 'string', enum: Object.keys(ELBOW_POLES), description: 'where the elbow is pushed' },+ hand: { type: 'string', enum: Object.keys(HAND_SHAPES), description: 'hand shape' },+ },+ };+ return {+ $schema: 'https://json-schema.org/draft/2020-12/schema',+ title: 'Motion Score',+ type: 'object',+ required: ['beats'],+ additionalProperties: false,+ properties: {+ version: { const: MOTION_SCORE_VERSION },+ name: { type: 'string', maxLength: LIMITS.maxNameLength },+ seed: { type: 'string', description: 'stable seed; the same seed and score always compile to the same clip' },+ loop: { type: 'boolean', description: 'true when the last beat should flow back into the first' },+ effort: { type: 'string', enum: Object.keys(EFFORTS), description: 'default performance quality for every beat' },+ beats: {+ type: 'array',+ minItems: 1,+ maxItems: LIMITS.maxBeats,+ items: {+ type: 'object',+ additionalProperties: false,+ properties: {+ label: { type: 'string', description: 'what this beat is, for a human reading the score' },+ in: { type: 'number', minimum: LIMITS.minBeatSeconds, maximum: LIMITS.maxBeatSeconds, description: 'seconds to travel into this beat' },+ hold: { type: 'number', minimum: 0, maximum: LIMITS.maxBeatSeconds, description: 'seconds to stay here once arrived' },+ ease: { type: 'string', enum: Object.keys(EASES) },+ effort: { type: 'string', enum: Object.keys(EFFORTS) },+ posture: { type: 'string', enum: Object.keys(POSTURES), description: 'whole-body shape this beat starts from' },+ root: {+ type: 'object',+ additionalProperties: false,+ properties: {+ height: { type: 'number', minimum: 0.28, maximum: 1.15, description: 'hip height as a fraction of standing' },+ forward: offsetProp('metres the body travels forward'),+ side: offsetProp('metres the body travels to its left'),+ rise: offsetProp('metres the whole body leaves the floor'),+ turn: angleProp('degrees the body turns toward its left'),+ },+ },+ torso: {+ type: 'object',+ additionalProperties: false,+ properties: {+ lean: angleProp('degrees forward, negative leans back'),+ twist: angleProp('degrees toward the body\'s left'),+ sideBend: angleProp('degrees leaning left, negative right'),+ },+ },+ gaze: { type: 'string', enum: Object.keys(GAZES) },+ arms: {+ type: 'object',+ additionalProperties: false,+ properties: { left: armProp, right: armProp },+ },+ legs: {+ type: 'object',+ additionalProperties: false,+ properties: {+ stance: { type: 'string', enum: Object.keys(STANCES) },+ left: { type: 'object', additionalProperties: false, properties: footSchema(offsetProp, angleProp) },+ right: { type: 'object', additionalProperties: false, properties: footSchema(offsetProp, angleProp) },+ },+ },+ face: { type: 'string', enum: Object.keys(EXPRESSIONS) },+ },+ },+ },+ },+ };+}++function footSchema(offsetProp, angleProp) {+ return {+ out: offsetProp('metres from the midline'),+ forward: offsetProp('metres in front of the hips'),+ lift: offsetProp('metres the ankle is raised off the floor'),+ heading: angleProp('degrees the foot turns toward the body\'s left'),+ plant: { type: 'string', enum: PLANTS, description: 'how the foot meets the floor' },+ };+}++/**+ * A one-line human summary of a score: how long it runs, how many beats, and+ * what it does. Used by the CLI, the API response, and anything logging one.+ * @param {object} score a normalized score+ * @returns {string}+ */+export function describeScore(score) {+ const beats = score.beats.length;+ const labels = score.beats.map((b) => b.label || b.posture).filter(Boolean);+ const shape = labels.length ? `: ${[...new Set(labels)].slice(0, 6).join(' > ')}` : '';+ return `${score.name}, ${score.duration.toFixed(2)}s, ${beats} beat${beats === 1 ? '' : 's'}${score.loop ? ', looping' : ''}${shape}`;+}diff --git a/packages/motion/src/solve.js b/packages/motion/src/solve.jsnew file mode 100644index 000000000..6b306d78a--- /dev/null+++ b/packages/motion/src/solve.js@@ -0,0 +1,350 @@+// Score to poses: the compiler.+//+// One beat becomes one solved skeleton, in a fixed order that matters:+//+// root where the hips are, which everything below hangs from+// turn the whole body's heading, so limbs solve in the turned frame+// torso lean, twist, side-bend, distributed along the spine+// legs ankles placed in WORLD space, so lowering the hips bends knees+// balance the hips nudged until the centre of mass sits over the feet+// arms wrists placed against anchors that ride the posed chest+// hands shape, after the wrist is where it belongs+// gaze last, so a look is relative to the chest it ended up on+// face expression weights+//+// Reversing any two of those produces a body that is subtly wrong in a way that+// is hard to see and impossible to fix downstream: solve the arms before the+// torso and every gesture sits where the chest used to be; solve the legs before+// the root and a crouch drives the feet through the floor.++import { anchorPoint, bodyDirection, restAnchor } from './rig/anchors.js';+import {+ ANKLE_REST_HEIGHT,+ BODY_FORWARD,+ BODY_LEFT,+ BODY_UP,+ STANDING_HIP_HEIGHT,+ boneLength,+ restPos,+} from './rig/skeleton.js';+import {+ FOOT_FLAT_DIR,+ PLANT_TILT,+ balanceOffset,+ shapeHand,+ solveArm,+ solveGaze,+ solveLeg,+ solveSpine,+ solveTurn,+} from './rig/ik.js';+import { Pose, restPose } from './rig/pose.js';+import { clamp, qAxisAngle, qRotate, vAdd, vCross, vDot, vLen, vNorm, vScale, vSub } from './rig/math.js';+import {+ ELBOW_POLES,+ EXPRESSIONS,+ GAZES,+ HAND_SHAPES,+ HAND_SHAPE_EXTENDED,+ POSTURES,+ STANCES,+} from './vocabulary.js';++/** How many times the balance pass runs. Two is enough; the third never moves. */+const BALANCE_PASSES = 3;++/** How much of the measured imbalance is corrected per pass. */+const BALANCE_GAIN = 0.75;++/** Balance closer than this is left alone: a body is never perfectly still. */+const BALANCE_TOLERANCE = 0.006;++/**+ * Solve one beat into a pose.+ *+ * @param {object} beat a normalized beat from `normalizeScore`+ * @returns {{ pose: import('./rig/pose.js').Pose, warnings: string[] }}+ */+export function solveBeat(beat) {+ const warnings = [];+ const posture = beat.posture ? POSTURES[beat.posture] : POSTURES.stand;+ const root = { height: posture.height, forward: 0, side: 0, rise: 0, turn: 0, ...beat.root };+ const torso = { ...posture.torso, ...beat.torso };+ const stance = STANCES[beat.legs.stance ?? posture.stance] ?? STANCES.neutral;+ const support = posture.support ?? 'feet';++ const pose = restPose();+ applyRoot(pose, root);+ solveTurn(pose, root.turn);+ solveSpine(pose, torso);++ const feet = footTargets(root, stance, beat.legs);+ placeFeet(pose, feet);++ if (support === 'feet') balanceOverFeet(pose, feet);++ for (const side of ['Left', 'Right']) {+ const spec = beat.arms[side.toLowerCase()];+ if (!spec) continue;+ const reached = placeArm(pose, side, spec);+ if (!reached) warnings.push(`the ${side.toLowerCase()} arm cannot reach ${describeArm(spec)}; it is extended toward it instead`);+ }++ for (const side of ['Left', 'Right']) {+ const spec = beat.arms[side.toLowerCase()];+ shapeFor(pose, side, spec?.hand);+ }++ applyGaze(pose, beat);+ if (beat.face) pose.setFace(expressionWeights(beat.face));++ return { pose, warnings };+}++/**+ * Solve every beat of a score.+ * @param {object} score a normalized score+ * @returns {{ keys: {time: number, pose: import('./rig/pose.js').Pose, beat: object}[], warnings: string[] }}+ */+export function solveScorePoses(score) {+ const warnings = [];+ const keys = score.beats.map((beat) => {+ const solved = solveBeat(beat);+ for (const w of solved.warnings) warnings.push(`beat at ${beat.at.toFixed(2)}s: ${w}`);+ return { time: beat.at, pose: solved.pose, beat };+ });+ return { keys, warnings };+}++// ── root ───────────────────────────────────────────────────────────────────++function applyRoot(pose, root) {+ const drop = STANDING_HIP_HEIGHT * (root.height - 1) + root.rise;+ const travel = vAdd(vScale(BODY_FORWARD, root.forward), vScale(BODY_LEFT, root.side));+ pose.setRootOffset([travel[0], drop + travel[1], travel[2]]);+}++// ── feet ───────────────────────────────────────────────────────────────────++const FOOT_LENGTH = boneLength('LeftFoot');++/**+ * Where each ankle goes, in world space.+ *+ * Feet are placed against the FLOOR and the body's travel, not against the+ * hips. That is the whole trick: the hips are free to drop, lean, and turn, and+ * the legs absorb it through IK, exactly as a real body does.+ */+function footTargets(root, stance, legs) {+ const heading = qAxisAngle(BODY_UP, root.turn);+ const planar = vAdd(+ vScale(BODY_FORWARD, root.forward),+ vScale(BODY_LEFT, root.side),+ );+ const out = {};+ for (const side of ['Left', 'Right']) {+ const key = side.toLowerCase();+ const spec = { ...(stance[key] ?? {}), ...(legs[key] ?? {}) };+ const outward = side === 'Left' ? (spec.out ?? 0.1) : -(spec.out ?? 0.1);+ const local = vAdd(vScale(BODY_LEFT, outward), vScale(BODY_FORWARD, spec.forward ?? 0));+ const ground = vAdd(planar, qRotate(heading, local));+ const plant = spec.plant ?? 'flat';+ const ankle = vAdd(+ [ground[0], 0, ground[2]],+ vScale(BODY_UP, ANKLE_REST_HEIGHT + (spec.lift ?? 0)),+ );+ out[side] = {+ ankle: vAdd(ankle, plantRise(plant, root.turn + (spec.heading ?? 0))),+ plant,+ heading: root.turn + (spec.heading ?? 0),+ bearing: spec.forward ?? 0,+ };+ }+ return out;+}++/**+ * How far the ankle rises when the foot rolls onto its toe or back onto its+ * heel. Derived by rotating the ankle-to-toe vector and keeping the contact+ * point still, which is what actually happens, rather than a tuned constant+ * that would be wrong for any other rig.+ */+function plantRise(plant, heading) {+ const tilt = PLANT_TILT[plant] ?? 0;+ if (!tilt) return [0, 0, 0];+ const yaw = qAxisAngle(BODY_UP, heading);+ const flat = qRotate(yaw, FOOT_FLAT_DIR);+ let across = vCross(BODY_UP, flat);+ if (vLen(across) < 1e-6) across = BODY_LEFT;+ const v = vScale(flat, FOOT_LENGTH);+ const rolled = qRotate(qAxisAngle(across, tilt), v);+ // A lifted foot is not in contact with anything, so it only hangs; it does+ // not push the ankle up.+ return plant === 'lift' ? [0, 0, 0] : vSub(v, rolled);+}++function placeFeet(pose, feet) {+ for (const side of ['Left', 'Right']) {+ const target = feet[side];+ solveLeg(pose, side, {+ ankle: target.ankle,+ heading: target.heading,+ plant: target.plant,+ });+ }+}++// ── balance ────────────────────────────────────────────────────────────────++/**+ * Nudge the hips horizontally until the centre of mass sits over the feet, then+ * put the legs back under the new hips.+ *+ * A lifted foot carries no weight, so the support base collapses onto the+ * standing foot and the body shifts over it, which is what makes a single-leg+ * beat read as standing on one leg instead of hovering.+ */+function balanceOverFeet(pose, feet) {+ const weights = {+ Left: feet.Left.plant === 'lift' ? 0 : 1,+ Right: feet.Right.plant === 'lift' ? 0 : 1,+ };+ if (!weights.Left && !weights.Right) return;+ for (let pass = 0; pass < BALANCE_PASSES; pass++) {+ const offset = balanceOffset(pose, weights);+ if (vLen(offset) < BALANCE_TOLERANCE) break;+ const corrected = vSub(pose.rootOffset, vScale(offset, BALANCE_GAIN));+ pose.setRootOffset(corrected);+ placeFeet(pose, feet);+ }+}++// ── arms ───────────────────────────────────────────────────────────────────++function placeArm(pose, side, spec) {+ const anchor = spec.at ?? 'side';+ const offset = {+ out: spec.out ?? 0,+ up: spec.up ?? 0,+ forward: spec.forward ?? 0,+ side,+ };+ const wrist = Array.isArray(anchor)+ ? vAdd(anchor, pose.rootOffset)+ : anchorPoint(pose, anchor, offset);++ const pole = spec.elbow ? poleDirection(pose, side, spec.elbow) : null;+ const palm = spec.palm ? bodyDirection(pose, spec.palm, side) : null;+ const fingers = spec.point ? bodyDirection(pose, spec.point, side) : null;++ const shoulder = pose.worldPos(`${side}Arm`);+ const distance = vLen(vSub(wrist, shoulder));+ const reachable = distance <= (boneLength(`${side}Arm`) + boneLength(`${side}ForeArm`)) * 0.99;++ solveArm(pose, side, {+ wrist,+ palm,+ fingers,+ ...(pole ? { pole } : {}),+ });+ return reachable;+}++function poleDirection(pose, side, name) {+ const spec = ELBOW_POLES[name];+ const outward = side === 'Left' ? spec.out : -spec.out;+ return vNorm(vAdd(+ vScale(BODY_LEFT, outward),+ vAdd(vScale(BODY_UP, spec.up), vScale(BODY_FORWARD, spec.forward)),+ ));+}++function describeArm(spec) {+ const at = Array.isArray(spec.at) ? 'that point' : `"${spec.at ?? 'side'}"`;+ return at;+}++function shapeFor(pose, side, hand) {+ if (!hand) return;+ const base = hand.shape ? HAND_SHAPES[hand.shape] : HAND_SHAPES.relaxed;+ const spec = {+ curl: hand.curl ?? base.curl,+ thumb: hand.thumb ?? base.thumb,+ spread: hand.spread ?? base.spread,+ };+ const extended = hand.shape ? HAND_SHAPE_EXTENDED[hand.shape] : null;+ if (extended?.length) {+ // A pointing hand is a fist with one finger out, which one curl value+ // cannot say. Curl everything, then straighten the named fingers.+ shapeHand(pose, side, spec);+ shapeHand(pose, side, { curl: 0, thumb: spec.thumb, spread: 0, only: extended });+ return;+ }+ shapeHand(pose, side, spec);+}++// ── gaze and face ──────────────────────────────────────────────────────────++function applyGaze(pose, beat) {+ const spec = beat.gaze;+ if (!spec) return;+ const preset = spec.preset ? GAZES[spec.preset] : { yaw: 0, pitch: 0 };+ let yaw = spec.yaw ?? preset.yaw ?? 0;+ let pitch = spec.pitch ?? preset.pitch ?? 0;+ if (spec.preset === 'hands') {+ // Look at whichever hand is doing the work, measured rather than guessed.+ const target = activeHand(pose, beat);+ if (target) {+ const aim = aimAngles(pose, target);+ yaw = spec.yaw ?? aim.yaw;+ pitch = spec.pitch ?? aim.pitch;+ }+ }+ solveGaze(pose, { yaw, pitch, roll: spec.roll ?? 0 });+}++// The acting hand is the one furthest from where a hand hangs at rest.+function activeHand(pose, beat) {+ let best = null;+ let bestDist = 0.12;+ for (const side of ['Left', 'Right']) {+ if (!beat.arms[side.toLowerCase()]) continue;+ const here = pose.worldPos(`${side}Hand`);+ const rest = vAdd(restAnchor('side', { side }), pose.rootOffset);+ const dist = vLen(vSub(here, rest));+ if (dist > bestDist) {+ bestDist = dist;+ best = here;+ }+ }+ return best;+}++// Yaw and pitch from the head to a world point, in the body's own frame.+function aimAngles(pose, point) {+ const to = vSub(point, pose.worldPos('Head'));+ const ahead = vDot(to, BODY_FORWARD);+ const aside = vDot(to, BODY_LEFT);+ const yaw = Math.atan2(aside, ahead) * (180 / Math.PI);+ const pitch = Math.atan2(vDot(to, BODY_UP), Math.hypot(ahead, aside)) * (180 / Math.PI);+ // A neck does not turn 180 degrees. Clamped to a look a body can hold.+ return { yaw: clamp(yaw, -72, 72), pitch: clamp(pitch, -48, 38) };+}++/** Expand an expression spec into ARKit blendshape weights. */+export function expressionWeights(spec) {+ const out = {};+ for (const [key, value] of Object.entries(spec)) {+ const preset = EXPRESSIONS[key];+ if (preset) {+ for (const [shape, weight] of Object.entries(preset)) {+ out[shape] = Math.max(out[shape] ?? 0, weight * value);+ }+ } else {+ out[key] = value;+ }+ }+ return out;+}++export { Pose };diff --git a/packages/motion/src/vocabulary.js b/packages/motion/src/vocabulary.jsnew file mode 100644index 000000000..0c7b85c92--- /dev/null+++ b/packages/motion/src/vocabulary.js@@ -0,0 +1,219 @@+// The words a motion score is written in.+//+// Everything here is a named shorthand for a set of numbers a solver can use.+// The point is not brevity: it is that a score author (a person, or a language+// model given the schema) can say `posture: "crouch"` and get a crouch with+// correct hip height, knee travel, foot spacing and torso counter-lean, instead+// of guessing eight numbers and producing something that reads as a fall.+//+// Every entry is a starting point, not a lock. A beat may override any field it+// names, so `{ posture: "crouch", root: { height: 0.5 } }` is a deeper crouch+// with everything else the posture already got right.++import { ARM_REACH, SHOULDER_SPAN, STANDING_HIP_HEIGHT } from './rig/skeleton.js';++/**+ * Whole-body postures. `height` is a fraction of standing hip height, `stance`+ * is the foot arrangement, and the torso values are the counter-lean that keeps+ * the shape balanced.+ *+ * `support` says what is carrying the body's weight, which decides whether the+ * balance pass runs. A standing body must keep its centre of mass over its+ * feet or it is falling; a seated or kneeling one is held up by the thing it is+ * sitting on, and forcing it over its feet would drag it off the chair.+ */+export const POSTURES = Object.freeze({+ stand: { height: 1, stance: 'neutral', torso: {} },+ /** Weight settled, feet a little apart: the default idle. */+ easy: { height: 0.995, stance: 'neutral', torso: { lean: 2 } },+ /** Athletic ready stance: knees soft, weight forward. */+ ready: { height: 0.93, stance: 'wide', torso: { lean: 10 } },+ /** Knees bent, hips back, chest up. */+ crouch: { height: 0.72, stance: 'wide', torso: { lean: 22 } },+ /** All the way down, heels threatening to lift. */+ squat: { height: 0.55, stance: 'wide', torso: { lean: 30 } },+ /** One knee down. */+ kneel: { support: 'ground', height: 0.52, stance: 'kneel', torso: { lean: 8 } },+ /** Seated on a surface at chair height, thighs level. */+ sit: { support: 'seat', height: 0.47, stance: 'sit', torso: { lean: 6 } },+ /** Seated and settled back. */+ slouch: { support: 'seat', height: 0.44, stance: 'sit', torso: { lean: -8 } },+ /** Weight forward over the front foot. */+ lunge: { height: 0.82, stance: 'split', torso: { lean: 14 } },+ /** Up on the balls of both feet. */+ tiptoe: { height: 1.06, stance: 'narrow', torso: { lean: 4 } },+ /** Leaning in, feet planted: interest, listening, confiding. */+ lean_in: { height: 0.985, stance: 'neutral', torso: { lean: 16 } },+ /** Leaning away: doubt, recoil, taking something in. */+ lean_back: { height: 0.99, stance: 'neutral', torso: { lean: -12 } },+ /** A bow from the waist. */+ bow: { height: 0.96, stance: 'narrow', torso: { lean: 42 } },+ /** Slumped: shoulders forward, head low, hips settled. */+ slump: { height: 0.94, stance: 'narrow', torso: { lean: 12 } },+ /** Drawn up: chest open, weight tall. */+ proud: { height: 1.01, stance: 'neutral', torso: { lean: -4 } },+});++export const POSTURE_NAMES = Object.freeze(Object.keys(POSTURES));++/**+ * Foot arrangements. Offsets are in metres from the body's centre line, in the+ * body's own frame: `out` is lateral, `forward` is toward where the body faces.+ * `lift` raises the ankle off the floor.+ */+export const STANCES = Object.freeze({+ neutral: { left: { out: 0.1, forward: 0 }, right: { out: 0.1, forward: 0 } },+ narrow: { left: { out: 0.055, forward: 0 }, right: { out: 0.055, forward: 0 } },+ wide: { left: { out: 0.19, forward: 0, heading: 9 }, right: { out: 0.19, forward: 0, heading: -9 } },+ /** One foot forward, one back: a step held, or a lunge. */+ split: { left: { out: 0.11, forward: 0.34 }, right: { out: 0.11, forward: -0.2, plant: 'toe' } },+ /** The mirror of split, so a walk can alternate. */+ split_right: { left: { out: 0.11, forward: -0.2, plant: 'toe' }, right: { out: 0.11, forward: 0.34 } },+ /** Front knee down, back foot on the toe. */+ kneel: { left: { out: 0.13, forward: 0.3 }, right: { out: 0.12, forward: -0.24, plant: 'toe' } },+ /** Sitting: both feet ahead of the hips, flat. */+ sit: { left: { out: 0.13, forward: 0.34 }, right: { out: 0.13, forward: 0.34 } },+ /** Both heels raised. */+ toes: { left: { out: 0.09, forward: 0, plant: 'toe' }, right: { out: 0.09, forward: 0, plant: 'toe' } },+ /** Weight on the left, right foot swinging free. */+ step_left: { left: { out: 0.09, forward: 0 }, right: { out: 0.11, forward: 0.22, lift: 0.07, plant: 'lift' } },+ step_right: { left: { out: 0.11, forward: 0.22, lift: 0.07, plant: 'lift' }, right: { out: 0.09, forward: 0 } },+});++export const STANCE_NAMES = Object.freeze(Object.keys(STANCES));++/**+ * Hand shapes, as the three numbers the hand solver takes. Enough to cover what+ * a gesture actually needs; anything finer belongs in a mocap clip, not a score.+ */+export const HAND_SHAPES = Object.freeze({+ open: { curl: 0, thumb: 0, spread: 0.35 },+ flat: { curl: 0, thumb: 0.1, spread: 0 },+ relaxed: { curl: 0.28, thumb: 0.3, spread: 0.1 },+ loose: { curl: 0.45, thumb: 0.4, spread: 0 },+ fist: { curl: 1, thumb: 0.85, spread: -0.2 },+ point: { curl: 0.9, thumb: 0.6, spread: 0 },+ pinch: { curl: 0.55, thumb: 0.95, spread: -0.1 },+ cup: { curl: 0.5, thumb: 0.2, spread: 0.25 },+ thumbs_up: { curl: 1, thumb: 0, spread: -0.2 },+});++export const HAND_SHAPE_NAMES = Object.freeze(Object.keys(HAND_SHAPES));++// The point shape has to leave the index finger straight, which no single curl+// value can express. The solver reads this exception and skips the named fingers.+export const HAND_SHAPE_EXTENDED = Object.freeze({+ point: ['Index'],+ thumbs_up: [],+});++/** Where an elbow is pushed, as a body-frame direction for the IK pole. */+export const ELBOW_POLES = Object.freeze({+ down: { out: 0.42, up: -1, forward: -0.3 },+ in: { out: -0.1, up: -1, forward: -0.15 },+ out: { out: 1, up: -0.35, forward: -0.2 },+ back: { out: 0.3, up: -0.5, forward: -1 },+ up: { out: 0.4, up: 0.8, forward: -0.2 },+ forward: { out: 0.3, up: -0.5, forward: 0.8 },+});++export const ELBOW_NAMES = Object.freeze(Object.keys(ELBOW_POLES));++/** Named gaze targets, in degrees of yaw (toward the body's left) and pitch. */+export const GAZES = Object.freeze({+ forward: { yaw: 0, pitch: 0 },+ down: { yaw: 0, pitch: -26 },+ up: { yaw: 0, pitch: 24 },+ left: { yaw: 34, pitch: 0 },+ right: { yaw: -34, pitch: 0 },+ away: { yaw: -22, pitch: 10 },+ /** Down and to one side: thinking, remembering, avoiding. */+ aside: { yaw: 20, pitch: -14 },+ /** Following the acting hand, resolved by the compiler. */+ hands: { yaw: 0, pitch: -18 },+});++export const GAZE_NAMES = Object.freeze(Object.keys(GAZES));++/**+ * Facial expressions as ARKit blendshape weights. The names match the shapes+ * the platform's avatars carry, and the retargeter drops any a given face is+ * missing rather than failing, so a score can always ask for an expression.+ */+export const EXPRESSIONS = Object.freeze({+ neutral: {},+ smile: { mouthSmileLeft: 0.45, mouthSmileRight: 0.45, cheekSquintLeft: 0.25, cheekSquintRight: 0.25 },+ grin: { mouthSmileLeft: 0.8, mouthSmileRight: 0.8, jawOpen: 0.16, cheekSquintLeft: 0.45, cheekSquintRight: 0.45 },+ sad: { mouthFrownLeft: 0.45, mouthFrownRight: 0.45, browInnerUp: 0.55 },+ angry: { browDownLeft: 0.8, browDownRight: 0.8, mouthPressLeft: 0.4, mouthPressRight: 0.4, noseSneerLeft: 0.25, noseSneerRight: 0.25 },+ surprised: { browInnerUp: 0.8, browOuterUpLeft: 0.7, browOuterUpRight: 0.7, jawOpen: 0.35, eyeWideLeft: 0.5, eyeWideRight: 0.5 },+ focused: { browDownLeft: 0.35, browDownRight: 0.35, eyeSquintLeft: 0.3, eyeSquintRight: 0.3 },+ doubt: { browOuterUpLeft: 0.55, browDownRight: 0.4, mouthPucker: 0.2 },+ wince: { eyeSquintLeft: 0.7, eyeSquintRight: 0.7, mouthPressLeft: 0.5, mouthPressRight: 0.5, browInnerUp: 0.4 },+ tired: { eyeBlinkLeft: 0.4, eyeBlinkRight: 0.4, browInnerUp: 0.3, jawOpen: 0.1 },+ speaking: { jawOpen: 0.22, mouthFunnel: 0.12 },+});++export const EXPRESSION_NAMES = Object.freeze(Object.keys(EXPRESSIONS));++/** Every blendshape any expression can touch, so a beat can zero the rest. */+export const EXPRESSION_SHAPES = Object.freeze([+ ...new Set(Object.values(EXPRESSIONS).flatMap((e) => Object.keys(e))),+].sort());++/**+ * Effort, in the sense a movement director means it: how a motion is performed+ * rather than what shape it makes. Three dials, each 0 to 1.+ *+ * weight 0 light, floating 1 heavy, committed+ * time 0 sustained, unhurried 1 sudden, urgent+ * flow 0 bound, controlled 1 free, released+ *+ * These are what make the same shapes read as a wave, a swat, or a salute. They+ * scale beat timing, pick the easing curve, and set how far a limb overshoots+ * its target before it settles.+ */+export const EFFORTS = Object.freeze({+ neutral: { weight: 0.5, time: 0.5, flow: 0.5 },+ gentle: { weight: 0.25, time: 0.25, flow: 0.65 },+ precise: { weight: 0.45, time: 0.6, flow: 0.15 },+ sharp: { weight: 0.6, time: 0.95, flow: 0.3 },+ heavy: { weight: 0.95, time: 0.35, flow: 0.25 },+ light: { weight: 0.1, time: 0.55, flow: 0.8 },+ sustained: { weight: 0.5, time: 0.08, flow: 0.6 },+ urgent: { weight: 0.7, time: 0.9, flow: 0.55 },+ weary: { weight: 0.85, time: 0.12, flow: 0.3 },+ playful: { weight: 0.3, time: 0.7, flow: 0.9 },+});++export const EFFORT_NAMES = Object.freeze(Object.keys(EFFORTS));++/** Easing curves a beat can be entered on. */+export const EASES = Object.freeze({+ linear: (u) => u,+ smooth: (u) => u * u * (3 - 2 * u),+ /** Fast out of the previous shape, soft into this one. */+ out: (u) => 1 - (1 - u) * (1 - u),+ /** Slow to leave, arriving at speed: a strike. */+ in: (u) => u * u,+ /** Almost no travel, then all of it: a snap. */+ snap: (u) => u * u * u * u,+ /** Overshoot and come back: weight arriving. */+ overshoot: (u) => {+ const s = 1.35;+ const v = u - 1;+ return v * v * ((s + 1) * v + s) + 1;+ },+ /** Settling: most of the way immediately, then a long tail. */+ settle: (u) => 1 - (1 - u) ** 3,+});++export const EASE_NAMES = Object.freeze(Object.keys(EASES));++/** Metric units a score's offsets are written in, exported so tools can label them. */+export const UNITS = Object.freeze({+ handWidth: SHOULDER_SPAN * 0.22,+ shoulderSpan: SHOULDER_SPAN,+ armReach: ARM_REACH,+ standingHipHeight: STANDING_HIP_HEIGHT,+});diff --git a/packages/render/LICENSE b/packages/render/LICENSEnew file mode 100644index 000000000..dff21b7f4--- /dev/null+++ b/packages/render/LICENSE@@ -0,0 +1,20 @@+Copyright 2026 nirholas++All rights reserved.++This software is proprietary and may not be used, copied, modified, distributed,+translated, or made available to any third party without the express written+permission of the copyright owner.++No rights are granted by implication, estoppel, or otherwise. Use of this+software is subject to the terms of a separate license agreement with the+copyright owner.++THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,+FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT.++IN NO EVENT SHALL THE COPYRIGHT OWNER BE LIABLE FOR ANY CLAIM, DAMAGES, OR+OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT, OR OTHERWISE,+ARISING FROM, OUT OF, OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER+DEALINGS IN THE SOFTWARE.diff --git a/packages/render/package.json b/packages/render/package.jsonnew file mode 100644index 000000000..d4d59990c--- /dev/null+++ b/packages/render/package.json@@ -0,0 +1,64 @@+{+ "name": "@three-ws/render",+ "version": "0.1.0",+ "description": "Render rigged, animated 3D avatars to PNG, GIF and truecolor terminal ANSI with no GPU, no WebGL and no headless browser. Pure JavaScript software rasterizer for glTF/GLB.",+ "keywords": [+ "3d",+ "glb",+ "gltf",+ "software-renderer",+ "rasterizer",+ "headless",+ "cpu-rendering",+ "ansi",+ "terminal",+ "avatar",+ "three.js",+ "skinning",+ "og-image"+ ],+ "author": "three.ws <[email protected]> (https://three.ws)",+ "license": "Apache-2.0",+ "homepage": "https://three.ws/tty",+ "repository": {+ "type": "git",+ "url": "git+https://github.com/nirholas/three.ws.git",+ "directory": "packages/render"+ },+ "bugs": {+ "url": "https://github.com/nirholas/three.ws/issues"+ },+ "type": "module",+ "main": "./src/index.js",+ "module": "./src/index.js",+ "exports": {+ ".": "./src/index.js",+ "./ansi": "./src/ansi.js",+ "./encode": "./src/encode.js",+ "./package.json": "./package.json"+ },+ "bin": {+ "three-ws-tty": "./bin/tty.js"+ },+ "files": [+ "src",+ "bin",+ "README.md",+ "LICENSE"+ ],+ "engines": {+ "node": ">=18"+ },+ "publishConfig": {+ "access": "public"+ },+ "dependencies": {+ "three": "^0.184.0"+ },+ "optionalDependencies": {+ "sharp": "^0.35.3"+ },+ "scripts": {+ "test": "vitest run tests"+ }+}diff --git a/packages/render/src/ansi.js b/packages/render/src/ansi.jsnew file mode 100644index 000000000..8178dccda--- /dev/null+++ b/packages/render/src/ansi.js@@ -0,0 +1,156 @@+/**+ * Terminal output.+ * ----------------+ * Three ladders, best first:+ * 1. Kitty graphics protocol, and the iTerm2 inline-image escape. Both put+ * the real PNG on the screen at full resolution.+ * 2. Truecolor half-blocks. Each cell holds two vertical pixels: the upper+ * half-block glyph U+2580 painted with the top pixel as foreground and the+ * bottom pixel as background. That doubles vertical resolution and is what+ * makes a terminal render read as an avatar rather than as mosaic art.+ * 3. The 256-colour cube, for terminals that never learned truecolor.+ *+ * Colour codes are emitted only when they change from the previous cell, which+ * roughly halves the bytes on a streamed animation.+ */++const ESC = '\x1b';+const UPPER_HALF = '▀';+const RESET = `${ESC}[0m`;++/** Nearest xterm-256 index for an 8-bit RGB triple. */+export function to256(r, g, b) {+ // Greys have their own 24-step ramp and quantize badly in the 6x6x6 cube.+ if (Math.abs(r - g) < 8 && Math.abs(g - b) < 8 && Math.abs(r - b) < 8) {+ if (r < 8) return 16;+ if (r > 248) return 231;+ return 232 + Math.round(((r - 8) / 247) * 23);+ }+ const q = (v) => Math.round((Math.max(0, Math.min(255, v)) / 255) * 5);+ return 16 + 36 * q(r) + 6 * q(g) + q(b);+}++function fgCode(r, g, b, truecolor) {+ return truecolor ? `${ESC}[38;2;${r};${g};${b}m` : `${ESC}[38;5;${to256(r, g, b)}m`;+}++function bgCode(r, g, b, truecolor) {+ return truecolor ? `${ESC}[48;2;${r};${g};${b}m` : `${ESC}[48;5;${to256(r, g, b)}m`;+}++function composite(data, offset, backdrop) {+ const a = data[offset + 3] / 255;+ if (a >= 0.999) return [data[offset], data[offset + 1], data[offset + 2], 1];+ if (a <= 0.002) return [0, 0, 0, 0];+ if (!backdrop) return [data[offset], data[offset + 1], data[offset + 2], a];+ return [+ Math.round(data[offset] * a + backdrop[0] * (1 - a)),+ Math.round(data[offset + 1] * a + backdrop[1] * (1 - a)),+ Math.round(data[offset + 2] * a + backdrop[2] * (1 - a)),+ 1,+ ];+}++/**+ * Turn an RGBA frame into half-block ANSI. The frame's height should be even;+ * an odd last row is drawn as a foreground-only cell.+ *+ * @param {{width:number,height:number,data:Uint8ClampedArray}} frame+ * @param {{truecolor?:boolean, backdrop?:number[]|null, indent?:number}} options+ */+export function toHalfBlocks(frame, { truecolor = true, backdrop = null, indent = 0 } = {}) {+ const { width, height, data } = frame;+ const pad = ' '.repeat(Math.max(0, indent));+ const lines = [];+ for (let y = 0; y < height; y += 2) {+ let line = pad;+ let lastFg = null;+ let lastBg = null;+ for (let x = 0; x < width; x++) {+ const top = composite(data, (y * width + x) * 4, backdrop);+ const bottom = y + 1 < height ? composite(data, ((y + 1) * width + x) * 4, backdrop) : [0, 0, 0, 0];++ if (top[3] === 0 && bottom[3] === 0) {+ if (lastFg !== null || lastBg !== null) {+ line += RESET;+ lastFg = null;+ lastBg = null;+ }+ line += ' ';+ continue;+ }++ const fgKey = top[3] === 0 ? 'none' : `${top[0]},${top[1]},${top[2]}`;+ const bgKey = bottom[3] === 0 ? 'none' : `${bottom[0]},${bottom[1]},${bottom[2]}`;++ if (bgKey === 'none' && lastBg !== 'none') {+ line += `${ESC}[49m`;+ lastBg = 'none';+ } else if (bgKey !== 'none' && bgKey !== lastBg) {+ line += bgCode(bottom[0], bottom[1], bottom[2], truecolor);+ lastBg = bgKey;+ }+ if (fgKey !== 'none' && fgKey !== lastFg) {+ line += fgCode(top[0], top[1], top[2], truecolor);+ lastFg = fgKey;+ }+ line += fgKey === 'none' ? ' ' : UPPER_HALF;+ }+ lines.push(`${line}${RESET}`);+ }+ return lines.join('\n');+}++/** Kitty graphics protocol payload for a PNG buffer. */+export function toKitty(png, { columns } = {}) {+ const base64 = Buffer.from(png).toString('base64');+ const chunks = [];+ for (let i = 0; i < base64.length; i += 4096) chunks.push(base64.slice(i, i + 4096));+ const size = columns ? `,c=${columns}` : '';+ return chunks+ .map((chunk, index) => {+ const more = index === chunks.length - 1 ? 0 : 1;+ const head = index === 0 ? `a=T,f=100${size},m=${more}` : `m=${more}`;+ return `${ESC}_G${head};${chunk}${ESC}\\`;+ })+ .join('');+}++/** iTerm2 inline image escape for a PNG buffer. */+export function toITerm(png, { columns } = {}) {+ const base64 = Buffer.from(png).toString('base64');+ const size = columns ? `;width=${columns}` : '';+ return `${ESC}]1337;File=inline=1${size};preserveAspectRatio=1:${base64}`;+}++/**+ * What this terminal can actually do. Honest about the difference between+ * "supports truecolor" and "claims 256 colours".+ */+export function detectTerminal(env = process.env, stream = process.stdout) {+ const term = env.TERM || '';+ const program = env.TERM_PROGRAM || '';+ const colorterm = env.COLORTERM || '';+ const kitty = Boolean(env.KITTY_WINDOW_ID) || term.includes('kitty') || program === 'ghostty' || program === 'WezTerm';+ const iterm = program === 'iTerm.app' || Boolean(env.ITERM_SESSION_ID);+ const truecolor =+ /truecolor|24bit/i.test(colorterm) || term.includes('24bit') || kitty || iterm || program === 'vscode';+ return {+ columns: stream?.columns || Number(env.COLUMNS) || 80,+ rows: stream?.rows || Number(env.LINES) || 24,+ truecolor: Boolean(truecolor),+ kitty,+ iterm,+ tty: Boolean(stream?.isTTY),+ };+}++export const CURSOR = {+ hide: `${ESC}[?25l`,+ show: `${ESC}[?25h`,+ home: `${ESC}[H`,+ clear: `${ESC}[2J${ESC}[H`,+ up: (n) => (n > 0 ? `${ESC}[${n}A` : ''),+ altScreen: `${ESC}[?1049h`,+ mainScreen: `${ESC}[?1049l`,+};diff --git a/packages/render/src/encode.js b/packages/render/src/encode.jsnew file mode 100644index 000000000..69d827542--- /dev/null+++ b/packages/render/src/encode.js@@ -0,0 +1,132 @@+/**+ * Image encoders with no native dependency.+ * -----------------------------------------+ * PNG for stills and APNG for animation, both written straight from RGBA with+ * node's zlib. APNG rather than GIF on purpose: GIF would quantize a rendered+ * avatar to 256 colours and hard-edge its alpha, while APNG keeps the full+ * 24-bit render and the soft contact shadow, and every current browser plays it.+ */++import zlib from 'node:zlib';++const CRC_TABLE = new Int32Array(256);+for (let n = 0; n < 256; n++) {+ let c = n;+ for (let k = 0; k < 8; k++) c = c & 1 ? 0xedb88320 ^ (c >>> 1) : c >>> 1;+ CRC_TABLE[n] = c;+}++function crc32(buf) {+ let c = -1;+ for (let i = 0; i < buf.length; i++) c = CRC_TABLE[(c ^ buf[i]) & 0xff] ^ (c >>> 8);+ return (c ^ -1) >>> 0;+}++function chunk(type, data) {+ const body = Buffer.concat([Buffer.from(type, 'ascii'), data]);+ const out = Buffer.allocUnsafe(body.length + 8);+ out.writeUInt32BE(data.length, 0);+ body.copy(out, 4);+ out.writeUInt32BE(crc32(body), out.length - 4);+ return out;+}++const SIGNATURE = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);++function ihdr(width, height) {+ const data = Buffer.alloc(13);+ data.writeUInt32BE(width, 0);+ data.writeUInt32BE(height, 4);+ data[8] = 8; // bit depth+ data[9] = 6; // truecolour with alpha+ return chunk('IHDR', data);+}++/** Filter scanlines with the Paeth predictor and deflate the result. */+function compressScanlines({ width, height, data }, level) {+ const stride = width * 4;+ const raw = Buffer.allocUnsafe((stride + 1) * height);+ let pos = 0;+ for (let y = 0; y < height; y++) {+ raw[pos++] = 4; // Paeth+ const row = y * stride;+ const prev = row - stride;+ for (let x = 0; x < stride; x++) {+ const value = data[row + x];+ const left = x >= 4 ? data[row + x - 4] : 0;+ const up = y > 0 ? data[prev + x] : 0;+ const upLeft = y > 0 && x >= 4 ? data[prev + x - 4] : 0;+ const p = left + up - upLeft;+ const pa = Math.abs(p - left);+ const pb = Math.abs(p - up);+ const pc = Math.abs(p - upLeft);+ const predictor = pa <= pb && pa <= pc ? left : pb <= pc ? up : upLeft;+ raw[pos++] = (value - predictor) & 0xff;+ }+ }+ return zlib.deflateSync(raw, { level });+}++/**+ * Encode one RGBA frame as PNG.+ * @param {{width:number,height:number,data:Uint8ClampedArray}} frame+ */+export function encodePng(frame, { level = 6 } = {}) {+ return Buffer.concat([+ SIGNATURE,+ ihdr(frame.width, frame.height),+ chunk('IDAT', compressScanlines(frame, level)),+ chunk('IEND', Buffer.alloc(0)),+ ]);+}++function fcTL(sequence, width, height, delayNum, delayDen) {+ const data = Buffer.alloc(26);+ data.writeUInt32BE(sequence, 0);+ data.writeUInt32BE(width, 4);+ data.writeUInt32BE(height, 8);+ data.writeUInt32BE(0, 12); // x offset+ data.writeUInt32BE(0, 16); // y offset+ data.writeUInt16BE(delayNum, 20);+ data.writeUInt16BE(delayDen, 22);+ data[24] = 0; // dispose: none+ data[25] = 0; // blend: source+ return chunk('fcTL', data);+}++/**+ * Encode a frame list as an animated PNG.+ * @param {Array<{width:number,height:number,data:Uint8ClampedArray}>} frames+ */+export function encodeApng(frames, { fps = 24, loops = 0, level = 6 } = {}) {+ if (!frames.length) throw new Error('encodeApng needs at least one frame');+ const { width, height } = frames[0];+ const delayDen = 1000;+ const delayNum = Math.max(1, Math.round(delayDen / fps));++ const parts = [SIGNATURE, ihdr(width, height)];+ const actl = Buffer.alloc(8);+ actl.writeUInt32BE(frames.length, 0);+ actl.writeUInt32BE(loops, 4);+ parts.push(chunk('acTL', actl));++ let sequence = 0;+ for (let i = 0; i < frames.length; i++) {+ const frame = frames[i];+ if (frame.width !== width || frame.height !== height) {+ throw new Error('every APNG frame must share the first frame size');+ }+ parts.push(fcTL(sequence++, width, height, delayNum, delayDen));+ const compressed = compressScanlines(frame, level);+ if (i === 0) {+ parts.push(chunk('IDAT', compressed));+ } else {+ const fdat = Buffer.allocUnsafe(compressed.length + 4);+ fdat.writeUInt32BE(sequence++, 0);+ compressed.copy(fdat, 4);+ parts.push(chunk('fdAT', fdat));+ }+ }+ parts.push(chunk('IEND', Buffer.alloc(0)));+ return Buffer.concat(parts);+}diff --git a/packages/render/src/env-shim.js b/packages/render/src/env-shim.jsnew file mode 100644index 000000000..1a1244161--- /dev/null+++ b/packages/render/src/env-shim.js@@ -0,0 +1,10 @@+/**+ * three.js' loader stack probes a handful of browser globals at module scope.+ * We removed every texture before parsing, so nothing here is ever used for+ * real work: these shims exist purely so the module graph evaluates in Node.+ * Imported for side effects, ahead of any three.js example module.+ */+if (typeof globalThis.self === 'undefined') globalThis.self = globalThis;+if (typeof globalThis.createImageBitmap === 'undefined') {+ globalThis.createImageBitmap = () => Promise.reject(new Error('createImageBitmap is not available in Node'));+}diff --git a/packages/render/src/glb.js b/packages/render/src/glb.jsnew file mode 100644index 000000000..45caae745--- /dev/null+++ b/packages/render/src/glb.js@@ -0,0 +1,217 @@+/**+ * GLB container surgery.+ * ----------------------+ * three.js' GLTFLoader is happy to parse geometry, skins and animation clips+ * in Node, but every texture it meets goes through ImageBitmapLoader, which+ * needs a DOM that does not exist here. Rather than shimming a browser we do+ * the honest thing: split the container ourselves, lift the encoded images out+ * of the binary chunk, rewrite the JSON so it declares no images at all, and+ * re-attach our own decoded pixels afterwards.+ *+ * The rewrite carries a marker (`extras.__twsMaterial`) on every material so+ * the re-attach step can map a three.js Material back to the glTF material+ * index it came from. GLTFLoader copies `extras` onto `Material.userData`,+ * which makes that mapping exact rather than name-based (names collide).+ */++const MAGIC_GLTF = 0x46546c67; // "glTF"+const CHUNK_JSON = 0x4e4f534a; // "JSON"+const CHUNK_BIN = 0x004e4942; // "BIN\0"++const textDecoder = new TextDecoder();+const textEncoder = new TextEncoder();++function pad4(n) {+ return (4 - (n % 4)) % 4;+}++/** Split a .glb ArrayBuffer into its JSON and BIN chunks. */+export function parseGlb(arrayBuffer) {+ const view = new DataView(arrayBuffer);+ if (arrayBuffer.byteLength < 12 || view.getUint32(0, true) !== MAGIC_GLTF) {+ throw new Error('not a GLB: missing glTF magic');+ }+ const version = view.getUint32(4, true);+ if (version !== 2) throw new Error(`unsupported GLB version ${version}`);+ const total = Math.min(view.getUint32(8, true), arrayBuffer.byteLength);++ let offset = 12;+ let json = null;+ let bin = null;+ while (offset + 8 <= total) {+ const length = view.getUint32(offset, true);+ const type = view.getUint32(offset + 4, true);+ const start = offset + 8;+ const end = Math.min(start + length, arrayBuffer.byteLength);+ if (type === CHUNK_JSON) json = JSON.parse(textDecoder.decode(new Uint8Array(arrayBuffer, start, end - start)));+ else if (type === CHUNK_BIN) bin = arrayBuffer.slice(start, end);+ offset = start + length + pad4(length);+ }+ if (!json) throw new Error('GLB has no JSON chunk');+ return { json, bin };+}++/** Re-pack a JSON + BIN pair into a .glb ArrayBuffer. */+export function packGlb(json, bin) {+ const jsonBytes = textEncoder.encode(JSON.stringify(json));+ const jsonPad = pad4(jsonBytes.length);+ const binBytes = bin ? new Uint8Array(bin) : null;+ const binPad = binBytes ? pad4(binBytes.length) : 0;++ let total = 12 + 8 + jsonBytes.length + jsonPad;+ if (binBytes) total += 8 + binBytes.length + binPad;++ const out = new ArrayBuffer(total);+ const view = new DataView(out);+ const bytes = new Uint8Array(out);++ view.setUint32(0, MAGIC_GLTF, true);+ view.setUint32(4, 2, true);+ view.setUint32(8, total, true);++ let offset = 12;+ view.setUint32(offset, jsonBytes.length + jsonPad, true);+ view.setUint32(offset + 4, CHUNK_JSON, true);+ bytes.set(jsonBytes, offset + 8);+ // glTF requires JSON chunks to be space-padded, not zero-padded.+ for (let i = 0; i < jsonPad; i++) bytes[offset + 8 + jsonBytes.length + i] = 0x20;+ offset += 8 + jsonBytes.length + jsonPad;++ if (binBytes) {+ view.setUint32(offset, binBytes.length + binPad, true);+ view.setUint32(offset + 4, CHUNK_BIN, true);+ bytes.set(binBytes, offset + 8);+ }+ return out;+}++function decodeDataUri(uri) {+ const comma = uri.indexOf(',');+ if (comma < 0) return null;+ const meta = uri.slice(5, comma);+ const isBase64 = /;base64$/i.test(meta);+ const mimeType = meta.replace(/;base64$/i, '') || 'application/octet-stream';+ const payload = uri.slice(comma + 1);+ const bytes = isBase64+ ? Uint8Array.from(Buffer.from(payload, 'base64'))+ : textEncoder.encode(decodeURIComponent(payload));+ return { mimeType, bytes };+}++/**+ * Pull every embedded image out of a glTF, keyed by its image index.+ * Images referenced by an external URI are skipped: the renderer degrades to+ * the material's base colour factor rather than issuing side-band fetches.+ */+export function extractImages(json, bin) {+ const images = new Map();+ const defs = json.images || [];+ for (let i = 0; i < defs.length; i++) {+ const def = defs[i];+ if (typeof def.uri === 'string') {+ if (def.uri.startsWith('data:')) {+ const decoded = decodeDataUri(def.uri);+ if (decoded) images.set(i, { mimeType: def.mimeType || decoded.mimeType, bytes: decoded.bytes });+ }+ continue;+ }+ if (typeof def.bufferView !== 'number' || !bin) continue;+ const bv = json.bufferViews?.[def.bufferView];+ if (!bv) continue;+ // Only buffer 0 lives in the BIN chunk of a self-contained GLB.+ if ((bv.buffer ?? 0) !== 0) continue;+ const start = bv.byteOffset || 0;+ const length = bv.byteLength || 0;+ if (start + length > bin.byteLength) continue;+ images.set(i, { mimeType: def.mimeType || 'image/png', bytes: new Uint8Array(bin, start, length).slice() });+ }+ return images;+}++const WRAP = { 33071: 'clamp', 33648: 'mirror', 10497: 'repeat' };++function textureBinding(json, ref) {+ if (!ref || typeof ref.index !== 'number') return null;+ const tex = json.textures?.[ref.index];+ if (!tex) return null;+ // KHR_texture_basisu points at a KTX2 payload we do not decode; the caller+ // falls back to the material factor, which is a real colour, not a stand-in.+ const source = tex.source ?? tex.extensions?.KHR_texture_basisu?.source;+ if (typeof source !== 'number') return null;+ const sampler = typeof tex.sampler === 'number' ? json.samplers?.[tex.sampler] : null;+ return {+ image: source,+ uv: ref.texCoord || 0,+ wrapS: WRAP[sampler?.wrapS] || 'repeat',+ wrapT: WRAP[sampler?.wrapT] || 'repeat',+ };+}++/**+ * Rewrite a glTF so it declares no images, recording what each material used.+ * Returns the stripped JSON plus one binding record per material index.+ */+export function stripImages(json) {+ const out = structuredClone(json);+ const bindings = new Map();+ const materials = out.materials || [];++ for (let i = 0; i < materials.length; i++) {+ const mat = materials[i];+ const pbr = mat.pbrMetallicRoughness || {};+ const binding = {+ baseColor: textureBinding(json, pbr.baseColorTexture),+ emissive: textureBinding(json, mat.emissiveTexture),+ baseColorFactor: pbr.baseColorFactor || [1, 1, 1, 1],+ metallicFactor: pbr.metallicFactor ?? 1,+ roughnessFactor: pbr.roughnessFactor ?? 1,+ emissiveFactor: mat.emissiveFactor || [0, 0, 0],+ emissiveStrength: mat.extensions?.KHR_materials_emissive_strength?.emissiveStrength ?? 1,+ alphaMode: mat.alphaMode || 'OPAQUE',+ alphaCutoff: mat.alphaCutoff ?? 0.5,+ doubleSided: mat.doubleSided === true,+ name: mat.name || `material_${i}`,+ };+ bindings.set(i, binding);++ mat.extras = { ...(mat.extras || {}), __twsMaterial: i };+ delete pbr.baseColorTexture;+ delete pbr.metallicRoughnessTexture;+ delete mat.normalTexture;+ delete mat.occlusionTexture;+ delete mat.emissiveTexture;+ if (mat.extensions) {+ delete mat.extensions.KHR_materials_pbrSpecularGlossiness;+ delete mat.extensions.KHR_materials_sheen;+ delete mat.extensions.KHR_materials_transmission;+ delete mat.extensions.KHR_materials_volume;+ delete mat.extensions.KHR_materials_specular;+ delete mat.extensions.KHR_materials_clearcoat;+ delete mat.extensions.KHR_materials_iridescence;+ if (Object.keys(mat.extensions).length === 0) delete mat.extensions;+ }+ }++ delete out.images;+ delete out.textures;+ delete out.samplers;++ const dropped = new Set([+ 'KHR_texture_basisu',+ 'KHR_texture_transform',+ 'EXT_texture_webp',+ 'KHR_materials_pbrSpecularGlossiness',+ 'KHR_materials_sheen',+ 'KHR_materials_transmission',+ 'KHR_materials_volume',+ 'KHR_materials_specular',+ 'KHR_materials_clearcoat',+ 'KHR_materials_iridescence',+ ]);+ const filterExt = (list) => (Array.isArray(list) ? list.filter((name) => !dropped.has(name)) : list);+ if (out.extensionsUsed) out.extensionsUsed = filterExt(out.extensionsUsed);+ if (out.extensionsRequired) out.extensionsRequired = filterExt(out.extensionsRequired);+ if (out.extensions) for (const name of dropped) delete out.extensions[name];++ return { json: out, bindings };+}diff --git a/packages/render/src/image.js b/packages/render/src/image.jsnew file mode 100644index 000000000..33076c75a--- /dev/null+++ b/packages/render/src/image.js@@ -0,0 +1,233 @@+/**+ * Texture decoding and mip pyramids.+ * ----------------------------------+ * PNG is decoded in-process with zlib (no native dependency), which covers+ * most glTF avatar textures. JPEG and everything else goes through sharp when+ * it is installed. A texture that cannot be decoded is reported as null and+ * the renderer falls back to the material's base colour factor.+ *+ * Every decoded texture carries a box-filtered mip pyramid. The rasterizer+ * picks a level per triangle from its UV-area to screen-area ratio, which is+ * what keeps a 2048px skin texture from boiling into noise on a 96px avatar.+ */++import zlib from 'node:zlib';++// Cache the promise, not a resolved flag: decodeImage is called concurrently+// for every texture in a model, and a flag would hand the losers of that race+// an undefined module while the winner is still awaiting the import.+let sharpPromise;++function loadSharp() {+ if (!sharpPromise) {+ sharpPromise = import('sharp').then(+ (mod) => mod.default || mod,+ () => null,+ );+ }+ return sharpPromise;+}++const PNG_MAGIC = [0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a];++function isPng(bytes) {+ if (bytes.length < 8) return false;+ for (let i = 0; i < 8; i++) if (bytes[i] !== PNG_MAGIC[i]) return false;+ return true;+}++function paethPredictor(a, b, c) {+ const p = a + b - c;+ const pa = Math.abs(p - a);+ const pb = Math.abs(p - b);+ const pc = Math.abs(p - c);+ if (pa <= pb && pa <= pc) return a;+ if (pb <= pc) return b;+ return c;+}++/** Minimal PNG decoder: 8-bit greyscale, RGB, palette and alpha variants. */+export function decodePng(bytes) {+ if (!isPng(bytes)) return null;+ const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);+ let offset = 8;+ let width = 0;+ let height = 0;+ let bitDepth = 8;+ let colorType = 6;+ let interlace = 0;+ let palette = null;+ let transparency = null;+ const idat = [];++ while (offset + 8 <= bytes.length) {+ const length = view.getUint32(offset);+ const type = String.fromCharCode(bytes[offset + 4], bytes[offset + 5], bytes[offset + 6], bytes[offset + 7]);+ const start = offset + 8;+ if (type === 'IHDR') {+ width = view.getUint32(start);+ height = view.getUint32(start + 4);+ bitDepth = bytes[start + 8];+ colorType = bytes[start + 9];+ interlace = bytes[start + 12];+ } else if (type === 'PLTE') {+ palette = bytes.subarray(start, start + length);+ } else if (type === 'tRNS') {+ transparency = bytes.subarray(start, start + length);+ } else if (type === 'IDAT') {+ idat.push(bytes.subarray(start, start + length));+ } else if (type === 'IEND') {+ break;+ }+ offset = start + length + 4;+ }++ if (!width || !height || bitDepth !== 8 || interlace !== 0) return null;++ const channels = { 0: 1, 2: 3, 3: 1, 4: 2, 6: 4 }[colorType];+ if (!channels) return null;+ if (colorType === 3 && !palette) return null;++ const raw = zlib.inflateSync(Buffer.concat(idat.map((c) => Buffer.from(c.buffer, c.byteOffset, c.byteLength))));+ const stride = width * channels;+ const pixels = new Uint8Array(height * stride);++ let pos = 0;+ for (let y = 0; y < height; y++) {+ const filter = raw[pos++];+ const rowStart = y * stride;+ const prevStart = rowStart - stride;+ for (let x = 0; x < stride; x++) {+ const value = raw[pos + x];+ const left = x >= channels ? pixels[rowStart + x - channels] : 0;+ const up = y > 0 ? pixels[prevStart + x] : 0;+ const upLeft = y > 0 && x >= channels ? pixels[prevStart + x - channels] : 0;+ let out;+ switch (filter) {+ case 0: out = value; break;+ case 1: out = value + left; break;+ case 2: out = value + up; break;+ case 3: out = value + ((left + up) >> 1); break;+ case 4: out = value + paethPredictor(left, up, upLeft); break;+ default: return null;+ }+ pixels[rowStart + x] = out & 0xff;+ }+ pos += stride;+ }++ const rgba = new Uint8Array(width * height * 4);+ for (let i = 0, n = width * height; i < n; i++) {+ const s = i * channels;+ const d = i * 4;+ if (colorType === 0) {+ rgba[d] = rgba[d + 1] = rgba[d + 2] = pixels[s];+ rgba[d + 3] = 255;+ } else if (colorType === 4) {+ rgba[d] = rgba[d + 1] = rgba[d + 2] = pixels[s];+ rgba[d + 3] = pixels[s + 1];+ } else if (colorType === 2) {+ rgba[d] = pixels[s];+ rgba[d + 1] = pixels[s + 1];+ rgba[d + 2] = pixels[s + 2];+ rgba[d + 3] = 255;+ } else if (colorType === 6) {+ rgba[d] = pixels[s];+ rgba[d + 1] = pixels[s + 1];+ rgba[d + 2] = pixels[s + 2];+ rgba[d + 3] = pixels[s + 3];+ } else {+ const idx = pixels[s];+ rgba[d] = palette[idx * 3];+ rgba[d + 1] = palette[idx * 3 + 1];+ rgba[d + 2] = palette[idx * 3 + 2];+ rgba[d + 3] = transparency && idx < transparency.length ? transparency[idx] : 255;+ }+ }+ return { width, height, data: rgba };+}++/**+ * Decode an encoded texture into RGBA. `maxSize` caps the working resolution:+ * a 4096px skin atlas costs 64 MB of RGBA and buys nothing at avatar scale.+ */+export async function decodeImage(bytes, mimeType, { maxSize = 1024 } = {}) {+ let decoded = null;+ if (isPng(bytes)) {+ try {+ decoded = decodePng(bytes);+ } catch {+ decoded = null;+ }+ }+ if (!decoded) {+ const sharp = await loadSharp();+ if (!sharp) return null;+ try {+ const pipeline = sharp(Buffer.from(bytes.buffer, bytes.byteOffset, bytes.byteLength), {+ failOn: 'none',+ limitInputPixels: 32768 * 32768,+ });+ const meta = await pipeline.metadata();+ const scale = Math.max(meta.width || 1, meta.height || 1) > maxSize;+ const out = await (scale ? pipeline.resize({ width: maxSize, height: maxSize, fit: 'inside' }) : pipeline)+ .ensureAlpha()+ .raw()+ .toBuffer({ resolveWithObject: true });+ decoded = {+ width: out.info.width,+ height: out.info.height,+ data: new Uint8Array(out.data.buffer, out.data.byteOffset, out.data.byteLength),+ };+ } catch {+ return null;+ }+ }+ if (!decoded) return null;+ if (Math.max(decoded.width, decoded.height) > maxSize) decoded = downscaleTo(decoded, maxSize);+ void mimeType;+ return decoded;+}++function downscaleTo(image, maxSize) {+ let current = image;+ while (Math.max(current.width, current.height) > maxSize && current.width > 1 && current.height > 1) {+ current = halveImage(current);+ }+ return current;+}++/** Box-filter an image down to half size in each axis (odd sizes clamp). */+export function halveImage({ width, height, data }) {+ const w = Math.max(1, width >> 1);+ const h = Math.max(1, height >> 1);+ const out = new Uint8Array(w * h * 4);+ for (let y = 0; y < h; y++) {+ const y0 = Math.min(height - 1, y * 2);+ const y1 = Math.min(height - 1, y * 2 + 1);+ for (let x = 0; x < w; x++) {+ const x0 = Math.min(width - 1, x * 2);+ const x1 = Math.min(width - 1, x * 2 + 1);+ const a = (y0 * width + x0) * 4;+ const b = (y0 * width + x1) * 4;+ const c = (y1 * width + x0) * 4;+ const d = (y1 * width + x1) * 4;+ const o = (y * w + x) * 4;+ for (let k = 0; k < 4; k++) {+ out[o + k] = (data[a + k] + data[b + k] + data[c + k] + data[d + k] + 2) >> 2;+ }+ }+ }+ return { width: w, height: h, data: out };+}++/** Build a mip chain down to 1x1. Level 0 is the source image. */+export function buildMipmaps(image) {+ const levels = [image];+ let current = image;+ while (current.width > 1 || current.height > 1) {+ current = halveImage(current);+ levels.push(current);+ }+ return levels;+}diff --git a/packages/render/src/load.js b/packages/render/src/load.jsnew file mode 100644index 000000000..5c031240f--- /dev/null+++ b/packages/render/src/load.js@@ -0,0 +1,111 @@+/**+ * Load a GLB into a renderable model, with no DOM and no GPU.+ * ----------------------------------------------------------+ * Textures are lifted out of the container before three.js sees the file (see+ * ./glb.js), decoded here, and re-attached as plain CPU mip pyramids keyed by+ * the glTF material index. Geometry, skins and animation clips come from the+ * real GLTFLoader, so anything the three.ws viewer can display, this can too:+ * meshopt compression, quantized attributes, morph-free humanoid rigs.+ */++import './env-shim.js';+import * as THREE from 'three';+import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader.js';+import { MeshoptDecoder } from 'three/examples/jsm/libs/meshopt_decoder.module.js';+import fs from 'node:fs/promises';+import { parseGlb, packGlb, extractImages, stripImages } from './glb.js';+import { decodeImage, buildMipmaps } from './image.js';++const DEFAULT_MAX_BYTES = 64 * 1024 * 1024;++async function readSource(source, { maxBytes, fetchImpl, timeoutMs }) {+ if (source instanceof ArrayBuffer) return source;+ if (ArrayBuffer.isView(source)) return source.buffer.slice(source.byteOffset, source.byteOffset + source.byteLength);+ if (typeof source !== 'string') throw new Error('source must be a URL, a file path, an ArrayBuffer or a TypedArray');++ if (/^https?:\/\//i.test(source)) {+ const doFetch = fetchImpl || globalThis.fetch;+ const controller = new AbortController();+ const timer = setTimeout(() => controller.abort(), timeoutMs);+ try {+ const res = await doFetch(source, { signal: controller.signal, redirect: 'follow' });+ if (!res.ok) throw new Error(`model fetch failed: ${res.status} ${res.statusText}`);+ const declared = Number(res.headers?.get?.('content-length') || 0);+ if (declared && declared > maxBytes) throw new Error(`model is ${declared} bytes, over the ${maxBytes} byte cap`);+ const buf = await res.arrayBuffer();+ if (buf.byteLength > maxBytes) throw new Error(`model is ${buf.byteLength} bytes, over the ${maxBytes} byte cap`);+ return buf;+ } finally {+ clearTimeout(timer);+ }+ }++ const file = await fs.readFile(source);+ if (file.byteLength > maxBytes) throw new Error(`model is ${file.byteLength} bytes, over the ${maxBytes} byte cap`);+ return file.buffer.slice(file.byteOffset, file.byteOffset + file.byteLength);+}++function assertSupported(json) {+ const required = json.extensionsRequired || [];+ const unsupported = required.filter((name) => name === 'KHR_draco_mesh_compression');+ if (unsupported.length) {+ throw new Error(+ `${unsupported.join(', ')} needs an external decoder this renderer does not ship. ` ++ 'Re-export the model with meshopt compression (gltfpack -cc) and it loads.',+ );+ }+}++async function decodeTextures(images, bindings, { maxTextureSize }) {+ const wanted = new Set();+ for (const binding of bindings.values()) {+ if (binding.baseColor) wanted.add(binding.baseColor.image);+ if (binding.emissive) wanted.add(binding.emissive.image);+ }+ const textures = new Map();+ await Promise.all(+ [...wanted].map(async (index) => {+ const encoded = images.get(index);+ if (!encoded) return;+ const decoded = await decodeImage(encoded.bytes, encoded.mimeType, { maxSize: maxTextureSize });+ if (decoded) textures.set(index, buildMipmaps(decoded));+ }),+ );+ return textures;+}++/**+ * @param {string|ArrayBuffer|ArrayBufferView} source URL, path, or raw GLB bytes.+ * @returns {Promise<{scene: THREE.Object3D, animations: THREE.AnimationClip[], bindings: Map, textures: Map}>}+ */+export async function loadModel(source, options = {}) {+ const {+ maxBytes = DEFAULT_MAX_BYTES,+ maxTextureSize = 1024,+ textures: wantTextures = true,+ fetchImpl,+ timeoutMs = 20_000,+ } = options;++ const raw = await readSource(source, { maxBytes, fetchImpl, timeoutMs });+ const { json, bin } = parseGlb(raw);+ assertSupported(json);++ const images = wantTextures ? extractImages(json, bin) : new Map();+ const { json: strippedJson, bindings } = stripImages(json);+ const stripped = packGlb(strippedJson, bin);++ const loader = new GLTFLoader();+ loader.setMeshoptDecoder(MeshoptDecoder);++ const gltf = await new Promise((resolve, reject) => {+ loader.parse(stripped, '', resolve, (err) => reject(err instanceof Error ? err : new Error(String(err))));+ });++ const textures = wantTextures ? await decodeTextures(images, bindings, { maxTextureSize }) : new Map();++ gltf.scene.updateMatrixWorld(true);+ return { scene: gltf.scene, animations: gltf.animations || [], bindings, textures };+}++export { THREE };diff --git a/packages/render/src/model.js b/packages/render/src/model.jsnew file mode 100644index 000000000..60c63c88f--- /dev/null+++ b/packages/render/src/model.js@@ -0,0 +1,302 @@+/**+ * The CPU side of a skinned avatar.+ * ---------------------------------+ * three.js does skinning on the GPU, so none of it happens for us. This module+ * evaluates animation clips with a real AnimationMixer, then blends the bone+ * matrices per vertex on the CPU and hands the rasterizer world-space+ * triangles. Normals ride the same blended matrix, so a walking avatar shades+ * correctly instead of shimmering the way recomputed face normals do.+ */++import './env-shim.js';+import * as THREE from 'three';+import { loadModel } from './load.js';+import { retargetClip } from './retarget.js';++const _matrix = new THREE.Matrix4();+const _bindInverse = new THREE.Matrix4();++function materialIndexOf(material) {+ const id = material?.userData?.__twsMaterial;+ return typeof id === 'number' ? id : -1;+}++function toLinear(c) {+ return c <= 0.04045 ? c / 12.92 : ((c + 0.055) / 1.055) ** 2.4;+}++export class AvatarModel {+ constructor({ scene, animations, bindings, textures }) {+ this.scene = scene;+ this.clips = animations || [];+ this.bindings = bindings;+ this.textures = textures;+ this.mixer = new THREE.AnimationMixer(scene);+ this.action = null;+ this.time = 0;+ this._meshes = [];+ this._collect();+ this.scene.updateMatrixWorld(true);+ this.bounds = this._computeBounds();+ }++ static async load(source, options = {}) {+ const parts = await loadModel(source, options);+ return new AvatarModel(parts);+ }++ get clipNames() {+ return this.clips.map((c) => c.name);+ }++ /** Bone names, normalized the way the retargeter sees them. */+ get boneNames() {+ const names = [];+ for (const mesh of this._meshes) {+ if (!mesh.skeleton) continue;+ for (const bone of mesh.skeleton.bones) if (!names.includes(bone.name)) names.push(bone.name);+ }+ return names;+ }++ /**+ * Bind external clips (from an animation-only GLB) onto this skeleton by+ * bone name, so a Mixamo walk drives a Ready Player Me body.+ */+ addClips(clips) {+ const skeleton = this._meshes.find((m) => m.skeleton)?.skeleton;+ if (!skeleton) return 0;+ let added = 0;+ for (const clip of clips) {+ const retargeted = retargetClip(clip, skeleton, this.scene);+ if (retargeted) {+ this.clips.push(retargeted);+ added++;+ }+ }+ return added;+ }++ /** Select a clip by name (case-insensitive substring) or index. */+ play(clip) {+ if (this.action) {+ this.action.stop();+ this.action = null;+ }+ if (clip === null || clip === undefined || this.clips.length === 0) return null;+ let found = null;+ if (typeof clip === 'number') found = this.clips[clip] || null;+ else {+ const needle = String(clip).toLowerCase();+ found =+ this.clips.find((c) => c.name.toLowerCase() === needle) ||+ this.clips.find((c) => c.name.toLowerCase().includes(needle)) ||+ null;+ }+ if (!found) return null;+ this.action = this.mixer.clipAction(found);+ this.action.reset();+ this.action.play();+ this.setTime(this.time);+ return found;+ }++ /** Absolute seek. The mixer only advances forward, so rewinds re-seek. */+ setTime(seconds) {+ if (!this.action) {+ this.time = seconds;+ this.scene.updateMatrixWorld(true);+ return;+ }+ const delta = seconds - this.time;+ this.time = seconds;+ if (delta >= 0) this.mixer.update(delta);+ else {+ this.mixer.setTime(0);+ this.mixer.update(Math.max(0, seconds));+ }+ this.scene.updateMatrixWorld(true);+ }++ get duration() {+ return this.action?.getClip()?.duration || 0;+ }++ _collect() {+ this.scene.updateMatrixWorld(true);+ this.scene.traverse((object) => {+ if (!object.isMesh || object.visible === false) return;+ const geometry = object.geometry;+ if (!geometry?.attributes?.position) return;+ const materials = Array.isArray(object.material) ? object.material : [object.material];+ const groups = geometry.groups?.length+ ? geometry.groups+ : [{ start: 0, count: geometry.index ? geometry.index.count : geometry.attributes.position.count, materialIndex: 0 }];+ this._meshes.push({+ object,+ geometry,+ skeleton: object.isSkinnedMesh ? object.skeleton : null,+ groups: groups.map((g) => ({ ...g, material: this._material(materials[g.materialIndex || 0]) })),+ world: new Float32Array(geometry.attributes.position.count * 3),+ normals: new Float32Array(geometry.attributes.position.count * 3),+ boneMatrices: null,+ });+ });+ }++ _material(material) {+ const index = materialIndexOf(material);+ const binding = this.bindings?.get(index);+ const factor = binding?.baseColorFactor || [1, 1, 1, 1];+ const emissiveFactor = binding?.emissiveFactor || [0, 0, 0];+ const strength = binding?.emissiveStrength ?? 1;+ const mips = binding?.baseColor ? this.textures.get(binding.baseColor.image) : null;+ return {+ name: binding?.name || material?.name || 'material',+ baseColorFactor: [toLinear(factor[0]), toLinear(factor[1]), toLinear(factor[2]), factor[3] ?? 1],+ emissive: [+ toLinear(emissiveFactor[0]) * strength,+ toLinear(emissiveFactor[1]) * strength,+ toLinear(emissiveFactor[2]) * strength,+ ],+ roughness: binding?.roughnessFactor ?? 0.8,+ metallic: binding?.metallicFactor ?? 0,+ alphaMode: binding?.alphaMode || 'OPAQUE',+ alphaCutoff: binding?.alphaCutoff ?? 0.5,+ doubleSided: binding?.doubleSided ?? false,+ mips: mips || null,+ wrapS: binding?.baseColor?.wrapS || 'repeat',+ wrapT: binding?.baseColor?.wrapT || 'repeat',+ hasVertexColor: false,+ };+ }++ /** Skin every mesh for the current pose into world-space buffers. */+ skin() {+ for (const entry of this._meshes) {+ const { object, geometry, skeleton } = entry;+ const position = geometry.attributes.position;+ const normal = geometry.attributes.normal;+ const count = position.count;+ const world = entry.world;+ const normals = entry.normals;++ if (!skeleton) {+ const m = object.matrixWorld.elements;+ for (let i = 0; i < count; i++) {+ const x = position.getX(i);+ const y = position.getY(i);+ const z = position.getZ(i);+ world[i * 3] = m[0] * x + m[4] * y + m[8] * z + m[12];+ world[i * 3 + 1] = m[1] * x + m[5] * y + m[9] * z + m[13];+ world[i * 3 + 2] = m[2] * x + m[6] * y + m[10] * z + m[14];+ if (normal) {+ const nx = normal.getX(i);+ const ny = normal.getY(i);+ const nz = normal.getZ(i);+ normals[i * 3] = m[0] * nx + m[4] * ny + m[8] * nz;+ normals[i * 3 + 1] = m[1] * nx + m[5] * ny + m[9] * nz;+ normals[i * 3 + 2] = m[2] * nx + m[6] * ny + m[10] * nz;+ }+ }+ continue;+ }++ const bones = skeleton.bones;+ const boneInverses = skeleton.boneInverses;+ if (!entry.boneMatrices || entry.boneMatrices.length !== bones.length * 16) {+ entry.boneMatrices = new Float32Array(bones.length * 16);+ }+ const bm = entry.boneMatrices;+ _bindInverse.copy(object.bindMatrixInverse);+ for (let b = 0; b < bones.length; b++) {+ // meshWorld * bindMatrixInverse * (boneWorld * boneInverse) * bindMatrix,+ // folded once per bone so the per-vertex loop is four weighted adds.+ _matrix.multiplyMatrices(bones[b].matrixWorld, boneInverses[b]);+ _matrix.premultiply(_bindInverse);+ _matrix.premultiply(object.matrixWorld);+ _matrix.multiply(object.bindMatrix);+ bm.set(_matrix.elements, b * 16);+ }++ const skinIndex = geometry.attributes.skinIndex;+ const skinWeight = geometry.attributes.skinWeight;+ for (let i = 0; i < count; i++) {+ const x = position.getX(i);+ const y = position.getY(i);+ const z = position.getZ(i);+ const nx = normal ? normal.getX(i) : 0;+ const ny = normal ? normal.getY(i) : 1;+ const nz = normal ? normal.getZ(i) : 0;++ let px = 0;+ let py = 0;+ let pz = 0;+ let vnx = 0;+ let vny = 0;+ let vnz = 0;+ let total = 0;+ for (let k = 0; k < 4; k++) {+ const weight = skinWeight.getComponent(i, k);+ if (weight === 0) continue;+ total += weight;+ const o = skinIndex.getComponent(i, k) * 16;+ px += weight * (bm[o] * x + bm[o + 4] * y + bm[o + 8] * z + bm[o + 12]);+ py += weight * (bm[o + 1] * x + bm[o + 5] * y + bm[o + 9] * z + bm[o + 13]);+ pz += weight * (bm[o + 2] * x + bm[o + 6] * y + bm[o + 10] * z + bm[o + 14]);+ vnx += weight * (bm[o] * nx + bm[o + 4] * ny + bm[o + 8] * nz);+ vny += weight * (bm[o + 1] * nx + bm[o + 5] * ny + bm[o + 9] * nz);+ vnz += weight * (bm[o + 2] * nx + bm[o + 6] * ny + bm[o + 10] * nz);+ }+ if (total > 0 && Math.abs(total - 1) > 1e-3) {+ const inv = 1 / total;+ px *= inv;+ py *= inv;+ pz *= inv;+ vnx *= inv;+ vny *= inv;+ vnz *= inv;+ }+ world[i * 3] = px;+ world[i * 3 + 1] = py;+ world[i * 3 + 2] = pz;+ normals[i * 3] = vnx;+ normals[i * 3 + 1] = vny;+ normals[i * 3 + 2] = vnz;+ }+ }+ return this._meshes;+ }++ get meshes() {+ return this._meshes;+ }++ _computeBounds() {+ this.skin();+ let minX = Infinity;+ let minY = Infinity;+ let minZ = Infinity;+ let maxX = -Infinity;+ let maxY = -Infinity;+ let maxZ = -Infinity;+ for (const entry of this._meshes) {+ const world = entry.world;+ for (let i = 0; i < world.length; i += 3) {+ if (world[i] < minX) minX = world[i];+ if (world[i] > maxX) maxX = world[i];+ if (world[i + 1] < minY) minY = world[i + 1];+ if (world[i + 1] > maxY) maxY = world[i + 1];+ if (world[i + 2] < minZ) minZ = world[i + 2];+ if (world[i + 2] > maxZ) maxZ = world[i + 2];+ }+ }+ if (!Number.isFinite(minX)) return { min: [0, 0, 0], max: [1, 1, 1], center: [0, 0.5, 0], size: [1, 1, 1] };+ return {+ min: [minX, minY, minZ],+ max: [maxX, maxY, maxZ],+ center: [(minX + maxX) / 2, (minY + maxY) / 2, (minZ + maxZ) / 2],+ size: [maxX - minX, maxY - minY, maxZ - minZ],+ };+ }+}diff --git a/packages/render/src/raster.js b/packages/render/src/raster.jsnew file mode 100644index 000000000..e6d4bce92--- /dev/null+++ b/packages/render/src/raster.js@@ -0,0 +1,350 @@+/**+ * A software rasterizer for skinned avatars.+ * ------------------------------------------+ * Scanline triangle fill with a z-buffer, perspective-correct varyings,+ * trilinear texture sampling off CPU mip pyramids, an opaque pass followed by+ * a depth-tested back-to-front transparent pass, a planar projected contact+ * shadow, and a supersampled resolve through an ACES-style tonemap.+ *+ * Everything works on plain typed arrays. There is no GPU, no WebGL context,+ * no canvas and no browser anywhere in this path.+ */++const EPSILON = 1e-6;+const NEAR_W = 1e-4;++const SRGB_TO_LINEAR = new Float32Array(256);+for (let i = 0; i < 256; i++) {+ const c = i / 255;+ SRGB_TO_LINEAR[i] = c <= 0.04045 ? c / 12.92 : ((c + 0.055) / 1.055) ** 2.4;+}++function linearToSrgb(c) {+ if (c <= 0) return 0;+ if (c >= 1) return 255;+ const v = c <= 0.0031308 ? c * 12.92 : 1.055 * c ** (1 / 2.4) - 0.055;+ return Math.max(0, Math.min(255, Math.round(v * 255)));+}++// Narkowicz's ACES fit. Keeps bright skin and emissive highlights from+// clipping to flat white the way a naive clamp does.+function tonemap(x) {+ const a = 2.51;+ const b = 0.03;+ const c = 2.43;+ const d = 0.59;+ const e = 0.14;+ return Math.max(0, Math.min(1, (x * (a * x + b)) / (x * (c * x + d) + e)));+}++function wrapCoord(value, size, mode) {+ if (mode === 'clamp') return Math.max(0, Math.min(size - 1, value));+ if (mode === 'mirror') {+ const period = size * 2;+ let v = ((value % period) + period) % period;+ if (v >= size) v = period - 1 - v;+ return v;+ }+ let v = value % size;+ if (v < 0) v += size;+ return v;+}++function sampleLevel(level, u, v, wrapS, wrapT, out) {+ const { width, height, data } = level;+ const x = u * width - 0.5;+ const y = v * height - 0.5;+ const x0 = Math.floor(x);+ const y0 = Math.floor(y);+ const fx = x - x0;+ const fy = y - y0;+ const xa = wrapCoord(x0, width, wrapS);+ const xb = wrapCoord(x0 + 1, width, wrapS);+ const ya = wrapCoord(y0, height, wrapT);+ const yb = wrapCoord(y0 + 1, height, wrapT);+ const i00 = (ya * width + xa) * 4;+ const i10 = (ya * width + xb) * 4;+ const i01 = (yb * width + xa) * 4;+ const i11 = (yb * width + xb) * 4;+ const w00 = (1 - fx) * (1 - fy);+ const w10 = fx * (1 - fy);+ const w01 = (1 - fx) * fy;+ const w11 = fx * fy;+ for (let k = 0; k < 3; k++) {+ out[k] =+ SRGB_TO_LINEAR[data[i00 + k]] * w00 ++ SRGB_TO_LINEAR[data[i10 + k]] * w10 ++ SRGB_TO_LINEAR[data[i01 + k]] * w01 ++ SRGB_TO_LINEAR[data[i11 + k]] * w11;+ }+ out[3] = (data[i00 + 3] * w00 + data[i10 + 3] * w10 + data[i01 + 3] * w01 + data[i11 + 3] * w11) / 255;+}++const _texA = new Float32Array(4);+const _texB = new Float32Array(4);++/** Trilinear sample across a mip pyramid. `lod` is a continuous mip level. */+export function sampleTexture(mips, lod, u, v, wrapS, wrapT, out) {+ const maxLevel = mips.length - 1;+ const clamped = Math.max(0, Math.min(maxLevel, lod));+ const base = Math.floor(clamped);+ const frac = clamped - base;+ sampleLevel(mips[base], u, v, wrapS, wrapT, _texA);+ if (frac <= 0.001 || base >= maxLevel) {+ out[0] = _texA[0];+ out[1] = _texA[1];+ out[2] = _texA[2];+ out[3] = _texA[3];+ return;+ }+ sampleLevel(mips[base + 1], u, v, wrapS, wrapT, _texB);+ for (let k = 0; k < 4; k++) out[k] = _texA[k] * (1 - frac) + _texB[k] * frac;+}++/** Colour, depth and coverage targets for one supersampled render. */+export class Framebuffer {+ constructor(width, height) {+ this.width = width;+ this.height = height;+ this.color = new Float32Array(width * height * 3);+ this.alpha = new Float32Array(width * height);+ this.depth = new Float32Array(width * height).fill(Infinity);+ this.shadow = new Float32Array(width * height);+ }+}++function lerpVertex(a, b, t, stride) {+ const out = new Float32Array(stride);+ for (let i = 0; i < stride; i++) out[i] = a[i] + (b[i] - a[i]) * t;+ return out;+}++// Vertex layout inside the clipper: clip xyzw, world xyz, normal xyz, uv, rgb.+const V_STRIDE = 15;++function clipNear(tri) {+ const out = [];+ for (let i = 0; i < 3; i++) {+ const current = tri[i];+ const next = tri[(i + 1) % 3];+ const cIn = current[3] > NEAR_W;+ const nIn = next[3] > NEAR_W;+ if (cIn) out.push(current);+ if (cIn !== nIn) {+ const t = (NEAR_W - current[3]) / (next[3] - current[3]);+ out.push(lerpVertex(current, next, t, V_STRIDE));+ }+ }+ if (out.length < 3) return [];+ if (out.length === 3) return [out];+ return [+ [out[0], out[1], out[2]],+ [out[0], out[2], out[3]],+ ];+}++/**+ * Draw one triangle. Vertices are packed float arrays in the V_STRIDE layout,+ * already in clip space:+ * 0..3 clip x, y, z, w+ * 4..6 world x, y, z+ * 7..9 world normal x, y, z+ * 10..11 uv+ * 12..14 vertex colour rgb+ */+function drawTriangle(fb, verts, material, ctx) {+ const { width, height } = fb;+ const needsClip = verts[0][3] <= NEAR_W || verts[1][3] <= NEAR_W || verts[2][3] <= NEAR_W;+ const pieces = needsClip ? clipNear(verts) : [verts];++ for (const piece of pieces) {+ const sx = new Float64Array(3);+ const sy = new Float64Array(3);+ const invW = new Float64Array(3);+ const depth = new Float64Array(3);+ for (let i = 0; i < 3; i++) {+ const v = piece[i];+ const iw = 1 / v[3];+ invW[i] = iw;+ sx[i] = (v[0] * iw * 0.5 + 0.5) * width;+ sy[i] = (0.5 - v[1] * iw * 0.5) * height;+ depth[i] = v[2] * iw;+ }++ const area = (sx[1] - sx[0]) * (sy[2] - sy[0]) - (sx[2] - sx[0]) * (sy[1] - sy[0]);+ if (!Number.isFinite(area) || Math.abs(area) < EPSILON) continue;++ // glTF front faces are counter-clockwise in a right-handed, Y-up NDC.+ // Screen space flips Y, so a front face lands here with a NEGATIVE+ // signed area under this cross product. Getting this backwards renders+ // the inside of the mesh: the silhouette still looks right and every+ // normal points away from the camera, which reads as a washed-out+ // rim-lit ghost rather than an obvious error.+ const facing = area < 0 ? 1 : -1;+ if (facing < 0 && !material.doubleSided) continue;++ const minX = Math.max(0, Math.floor(Math.min(sx[0], sx[1], sx[2])));+ const maxX = Math.min(width - 1, Math.ceil(Math.max(sx[0], sx[1], sx[2])));+ const minY = Math.max(0, Math.floor(Math.min(sy[0], sy[1], sy[2])));+ const maxY = Math.min(height - 1, Math.ceil(Math.max(sy[0], sy[1], sy[2])));+ if (minX > maxX || minY > maxY) continue;++ const invArea = 1 / area;++ // Texture level of detail from the UV-area to screen-area ratio. One+ // value per triangle: cheap, and stable enough that a 1k skin atlas+ // stops shimmering when the avatar turns.+ let lod = 0;+ const mips = material.mips;+ if (mips) {+ const u0 = piece[0][10];+ const v0 = piece[0][11];+ const uvArea = Math.abs(+ (piece[1][10] - u0) * (piece[2][11] - v0) - (piece[2][10] - u0) * (piece[1][11] - v0),+ );+ const texels = uvArea * mips[0].width * mips[0].height;+ const pixels = Math.abs(area);+ if (pixels > EPSILON && texels > EPSILON) lod = 0.5 * Math.log2(texels / pixels);+ }++ const color = fb.color;+ const alpha = fb.alpha;+ const depthBuffer = fb.depth;+ const { lights, cameraPos, ambient, texel } = ctx;+ const depthWrite = ctx.depthWrite;+ const blend = ctx.blend;+ const baseFactor = material.baseColorFactor;+ const emissive = material.emissive;+ const roughness = material.roughness;+ const shininess = 2 + 180 * (1 - roughness) * (1 - roughness);+ const specScale = 1 - roughness * 0.85;+ const alphaMode = material.alphaMode;+ const cutoff = material.alphaCutoff;++ for (let py = minY; py <= maxY; py++) {+ const pyc = py + 0.5;+ for (let px = minX; px <= maxX; px++) {+ const pxc = px + 0.5;+ const w0 = ((sx[1] - pxc) * (sy[2] - pyc) - (sx[2] - pxc) * (sy[1] - pyc)) * invArea;+ if (w0 < 0) continue;+ const w1 = ((sx[2] - pxc) * (sy[0] - pyc) - (sx[0] - pxc) * (sy[2] - pyc)) * invArea;+ if (w1 < 0) continue;+ const w2 = 1 - w0 - w1;+ if (w2 < 0) continue;++ const idx = py * width + px;+ const z = w0 * depth[0] + w1 * depth[1] + w2 * depth[2];+ if (z >= depthBuffer[idx]) continue;++ const iw = w0 * invW[0] + w1 * invW[1] + w2 * invW[2];+ if (iw <= 0) continue;+ const persp = 1 / iw;+ const pw0 = w0 * invW[0] * persp;+ const pw1 = w1 * invW[1] * persp;+ const pw2 = w2 * invW[2] * persp;++ let r = baseFactor[0];+ let g = baseFactor[1];+ let b = baseFactor[2];+ let a = baseFactor[3];++ if (mips) {+ const u = pw0 * piece[0][10] + pw1 * piece[1][10] + pw2 * piece[2][10];+ const v = pw0 * piece[0][11] + pw1 * piece[1][11] + pw2 * piece[2][11];+ sampleTexture(mips, lod, u, v, material.wrapS, material.wrapT, texel);+ r *= texel[0];+ g *= texel[1];+ b *= texel[2];+ a *= texel[3];+ }++ if (material.hasVertexColor) {+ r *= pw0 * piece[0][12] + pw1 * piece[1][12] + pw2 * piece[2][12];+ g *= pw0 * piece[0][13] + pw1 * piece[1][13] + pw2 * piece[2][13];+ b *= pw0 * piece[0][14] + pw1 * piece[1][14] + pw2 * piece[2][14];+ }++ if (alphaMode === 'MASK') {+ if (a < cutoff) continue;+ a = 1;+ } else if (alphaMode === 'OPAQUE') {+ a = 1;+ }+ if (a <= 0.003) continue;++ let nx = pw0 * piece[0][7] + pw1 * piece[1][7] + pw2 * piece[2][7];+ let ny = pw0 * piece[0][8] + pw1 * piece[1][8] + pw2 * piece[2][8];+ let nz = pw0 * piece[0][9] + pw1 * piece[1][9] + pw2 * piece[2][9];+ const nlen = Math.hypot(nx, ny, nz) || 1;+ const nscale = facing / nlen;+ nx *= nscale;+ ny *= nscale;+ nz *= nscale;++ const wx = pw0 * piece[0][4] + pw1 * piece[1][4] + pw2 * piece[2][4];+ const wy = pw0 * piece[0][5] + pw1 * piece[1][5] + pw2 * piece[2][5];+ const wz = pw0 * piece[0][6] + pw1 * piece[1][6] + pw2 * piece[2][6];++ let vx = cameraPos[0] - wx;+ let vy = cameraPos[1] - wy;+ let vz = cameraPos[2] - wz;+ const vlen = Math.hypot(vx, vy, vz) || 1;+ vx /= vlen;+ vy /= vlen;+ vz /= vlen;++ let lr = ambient[0];+ let lg = ambient[1];+ let lb = ambient[2];+ for (let li = 0; li < lights.length; li++) {+ const light = lights[li];+ const ndl = nx * light.dir[0] + ny * light.dir[1] + nz * light.dir[2];+ const diffuse = light.wrap ? Math.max(0, (ndl + light.wrap) / (1 + light.wrap)) : Math.max(0, ndl);+ if (diffuse <= 0) continue;+ lr += light.color[0] * diffuse;+ lg += light.color[1] * diffuse;+ lb += light.color[2] * diffuse;++ if (light.specular > 0) {+ let hx = light.dir[0] + vx;+ let hy = light.dir[1] + vy;+ let hz = light.dir[2] + vz;+ const hlen = Math.hypot(hx, hy, hz) || 1;+ hx /= hlen;+ hy /= hlen;+ hz /= hlen;+ const ndh = Math.max(0, nx * hx + ny * hy + nz * hz);+ const spec = ndh ** shininess * light.specular * specScale;+ lr += light.color[0] * spec;+ lg += light.color[1] * spec;+ lb += light.color[2] * spec;+ }+ }++ const ndv = Math.max(0, nx * vx + ny * vy + nz * vz);+ const rim = (1 - ndv) ** 3 * ctx.rimStrength;++ const outR = r * lr + emissive[0] + rim * ctx.rimColor[0];+ const outG = g * lg + emissive[1] + rim * ctx.rimColor[1];+ const outB = b * lb + emissive[2] + rim * ctx.rimColor[2];++ const ci = idx * 3;+ if (blend && a < 1) {+ const inv = 1 - a;+ color[ci] = color[ci] * inv + outR * a;+ color[ci + 1] = color[ci + 1] * inv + outG * a;+ color[ci + 2] = color[ci + 2] * inv + outB * a;+ alpha[idx] = alpha[idx] * inv + a;+ } else {+ color[ci] = outR;+ color[ci + 1] = outG;+ color[ci + 2] = outB;+ alpha[idx] = 1;+ }+ if (depthWrite) depthBuffer[idx] = z;+ }+ }+ }+}++export { drawTriangle, V_STRIDE, linearToSrgb, tonemap, SRGB_TO_LINEAR };diff --git a/packages/render/src/render.js b/packages/render/src/render.jsnew file mode 100644index 000000000..08074a089--- /dev/null+++ b/packages/render/src/render.js@@ -0,0 +1,457 @@+/**+ * The render loop: framing, lighting, passes, resolve.+ * ---------------------------------------------------+ * Frames are supersampled, drawn opaque-first with a depth-tested+ * back-to-front transparent pass, given a planar projected contact shadow, and+ * resolved through an ACES tonemap into straight-alpha RGBA. The default+ * framing matches the three.ws OG card (26 degrees of yaw, 1.22 margin) so a+ * CPU render and a browser render of the same avatar are the same picture.+ */++import './env-shim.js';+import * as THREE from 'three';+import { Framebuffer, drawTriangle, V_STRIDE, linearToSrgb, tonemap } from './raster.js';++const DEG = Math.PI / 180;++export const PRESETS = {+ studio: {+ ambient: [0.16, 0.17, 0.2],+ lights: [+ { dir: [-0.45, 0.72, 0.85], color: [1.05, 1.02, 0.98], specular: 0.35, wrap: 0 },+ { dir: [0.8, 0.25, 0.35], color: [0.3, 0.33, 0.42], specular: 0.08, wrap: 0.5 },+ { dir: [0.1, -0.4, -0.9], color: [0.14, 0.15, 0.2], specular: 0, wrap: 0.8 },+ ],+ rimStrength: 0.32,+ rimColor: [0.55, 0.72, 1],+ background: ['#11131c', '#05060a'],+ shadow: 0.55,+ },+ terminal: {+ ambient: [0.2, 0.21, 0.26],+ lights: [+ { dir: [-0.4, 0.7, 0.9], color: [1.15, 1.12, 1.05], specular: 0.3, wrap: 0 },+ { dir: [0.85, 0.2, 0.2], color: [0.28, 0.34, 0.5], specular: 0.05, wrap: 0.6 },+ ],+ rimStrength: 0.55,+ rimColor: [0.35, 0.85, 1],+ background: 'transparent',+ shadow: 0.4,+ },+ daylight: {+ ambient: [0.3, 0.32, 0.36],+ lights: [+ { dir: [-0.3, 0.85, 0.6], color: [1.1, 1.06, 0.96], specular: 0.4, wrap: 0 },+ { dir: [0.7, 0.1, -0.5], color: [0.3, 0.36, 0.45], specular: 0.05, wrap: 0.5 },+ ],+ rimStrength: 0.18,+ rimColor: [1, 0.96, 0.9],+ background: ['#eef2f8', '#cfd8e6'],+ shadow: 0.5,+ },+};++export const FOCUS = {+ full: { fit: 'all', offset: 0, margin: 1.22 },+ bust: { fit: 'upper', offset: 0, margin: 1.12 },+ head: { fit: 'head', offset: 0, margin: 1.05 },+};++function normalize(v) {+ const len = Math.hypot(v[0], v[1], v[2]) || 1;+ return [v[0] / len, v[1] / len, v[2] / len];+}++function parseColor(value) {+ if (typeof value !== 'string') return [0, 0, 0];+ const hex = value.replace('#', '').trim();+ const full = hex.length === 3 ? hex.split('').map((c) => c + c).join('') : hex;+ const n = Number.parseInt(full.slice(0, 6), 16);+ if (!Number.isFinite(n)) return [0, 0, 0];+ const srgb = [((n >> 16) & 255) / 255, ((n >> 8) & 255) / 255, (n & 255) / 255];+ return srgb.map((c) => (c <= 0.04045 ? c / 12.92 : ((c + 0.055) / 1.055) ** 2.4));+}++/** Frame a camera on the model according to the focus mode. */+export function frameCamera(bounds, { focus = 'full', yaw = 26, pitch = 6, fov = 30, aspect = 1, margin } = {}) {+ const mode = FOCUS[focus] || FOCUS.full;+ const [minX, minY, minZ] = bounds.min;+ const [maxX, maxY, maxZ] = bounds.max;+ const height = maxY - minY;++ let targetY = (minY + maxY) / 2;+ let fitHeight = height;+ let fitWidth = Math.max(maxX - minX, maxZ - minZ);+ if (mode.fit === 'head') {+ // The top ~13% of a humanoid is the head; framing off the bounding box+ // beats guessing a bone, and it works on non-humanoid bodies too.+ fitHeight = height * 0.18;+ targetY = maxY - fitHeight * 0.52;+ fitWidth = fitHeight * 0.9;+ } else if (mode.fit === 'upper') {+ fitHeight = height * 0.42;+ targetY = maxY - fitHeight * 0.5;+ fitWidth = Math.min(fitWidth, fitHeight * 1.1);+ }++ const useMargin = margin ?? mode.margin;+ const vFov = fov * DEG;+ const hFov = 2 * Math.atan(Math.tan(vFov / 2) * aspect);+ const distV = fitHeight / 2 / Math.tan(vFov / 2);+ const distH = fitWidth / 2 / Math.tan(hFov / 2);+ const distance = Math.max(distV, distH) * useMargin;++ const yr = yaw * DEG;+ const pr = pitch * DEG;+ const eye = [+ Math.sin(yr) * Math.cos(pr) * distance,+ targetY + Math.sin(pr) * distance,+ Math.cos(yr) * Math.cos(pr) * distance,+ ];+ const target = [0, targetY, 0];+ const centerX = (minX + maxX) / 2;+ const centerZ = (minZ + maxZ) / 2;+ eye[0] += centerX;+ eye[2] += centerZ;+ target[0] = centerX;+ target[2] = centerZ;++ const camera = new THREE.PerspectiveCamera(fov, aspect, Math.max(0.01, distance - fitHeight * 4), distance + fitHeight * 8);+ camera.position.set(eye[0], eye[1], eye[2]);+ camera.up.set(0, 1, 0);+ camera.lookAt(target[0], target[1], target[2]);+ camera.updateMatrixWorld(true);+ camera.updateProjectionMatrix();++ const viewProjection = new THREE.Matrix4().multiplyMatrices(camera.projectionMatrix, camera.matrixWorldInverse);+ return { camera, viewProjection, eye, target, distance };+}++function fillShadowTriangle(shadow, width, height, x0, y0, x1, y1, x2, y2) {+ const area = (x1 - x0) * (y2 - y0) - (x2 - x0) * (y1 - y0);+ if (!Number.isFinite(area) || Math.abs(area) < 1e-6) return;+ const invArea = 1 / area;+ const minX = Math.max(0, Math.floor(Math.min(x0, x1, x2)));+ const maxX = Math.min(width - 1, Math.ceil(Math.max(x0, x1, x2)));+ const minY = Math.max(0, Math.floor(Math.min(y0, y1, y2)));+ const maxY = Math.min(height - 1, Math.ceil(Math.max(y0, y1, y2)));+ for (let py = minY; py <= maxY; py++) {+ const pyc = py + 0.5;+ for (let px = minX; px <= maxX; px++) {+ const pxc = px + 0.5;+ const w0 = ((x1 - pxc) * (y2 - pyc) - (x2 - pxc) * (y1 - pyc)) * invArea;+ if (w0 < 0) continue;+ const w1 = ((x2 - pxc) * (y0 - pyc) - (x0 - pxc) * (y2 - pyc)) * invArea;+ if (w1 < 0) continue;+ if (1 - w0 - w1 < 0) continue;+ shadow[py * width + px] = 1;+ }+ }+}++function blur(buffer, width, height, radius) {+ if (radius < 1) return buffer;+ const temp = new Float32Array(buffer.length);+ const window = radius * 2 + 1;+ for (let y = 0; y < height; y++) {+ let sum = 0;+ for (let x = -radius; x <= radius; x++) sum += buffer[y * width + Math.max(0, Math.min(width - 1, x))];+ for (let x = 0; x < width; x++) {+ temp[y * width + x] = sum / window;+ const outIdx = Math.max(0, Math.min(width - 1, x - radius));+ const inIdx = Math.max(0, Math.min(width - 1, x + radius + 1));+ sum += buffer[y * width + inIdx] - buffer[y * width + outIdx];+ }+ }+ for (let x = 0; x < width; x++) {+ let sum = 0;+ for (let y = -radius; y <= radius; y++) sum += temp[Math.max(0, Math.min(height - 1, y)) * width + x];+ for (let y = 0; y < height; y++) {+ buffer[y * width + x] = sum / window;+ const outIdx = Math.max(0, Math.min(height - 1, y - radius));+ const inIdx = Math.max(0, Math.min(height - 1, y + radius + 1));+ sum += temp[inIdx * width + x] - temp[outIdx * width + x];+ }+ }+ return buffer;+}++function packVertex(out, world, normals, uv, colors, index, matrix) {+ const i3 = index * 3;+ const x = world[i3];+ const y = world[i3 + 1];+ const z = world[i3 + 2];+ out[0] = matrix[0] * x + matrix[4] * y + matrix[8] * z + matrix[12];+ out[1] = matrix[1] * x + matrix[5] * y + matrix[9] * z + matrix[13];+ out[2] = matrix[2] * x + matrix[6] * y + matrix[10] * z + matrix[14];+ out[3] = matrix[3] * x + matrix[7] * y + matrix[11] * z + matrix[15];+ out[4] = x;+ out[5] = y;+ out[6] = z;+ out[7] = normals[i3];+ out[8] = normals[i3 + 1];+ out[9] = normals[i3 + 2];+ if (uv) {+ out[10] = uv.getX(index);+ out[11] = uv.getY(index);+ } else {+ out[10] = 0;+ out[11] = 0;+ }+ if (colors) {+ out[12] = colors.getX(index);+ out[13] = colors.getY(index);+ out[14] = colors.getZ(index);+ } else {+ out[12] = 1;+ out[13] = 1;+ out[14] = 1;+ }+ return out;+}++/**+ * Render the model's current pose to straight-alpha RGBA.+ * @returns {{width:number,height:number,data:Uint8ClampedArray}}+ */+export function renderFrame(model, options = {}) {+ const {+ width = 512,+ height = 512,+ supersample = 2,+ preset = 'studio',+ yaw = 26,+ pitch = 6,+ fov = 30,+ focus = 'full',+ margin,+ background,+ shadow: shadowOpt,+ } = options;++ const style = { ...(PRESETS[preset] || PRESETS.studio) };+ const bg = background === undefined ? style.background : background;+ const shadowStrength = shadowOpt === undefined ? style.shadow : shadowOpt;++ const ss = Math.max(1, Math.min(4, Math.round(supersample)));+ const w = width * ss;+ const h = height * ss;+ const fb = new Framebuffer(w, h);++ const meshes = model.skin();+ const bounds = model.bounds;+ const { viewProjection, eye } = frameCamera(bounds, { focus, yaw, pitch, fov, aspect: width / height, margin });+ const vp = viewProjection.elements;++ const lights = style.lights.map((l) => ({ ...l, dir: normalize(l.dir) }));+ const ctx = {+ lights,+ ambient: style.ambient,+ cameraPos: eye,+ rimStrength: style.rimStrength,+ rimColor: style.rimColor,+ texel: new Float32Array(4),+ depthWrite: true,+ blend: false,+ };++ // Contact shadow: the same triangles flattened onto the floor plane and+ // pushed through the same camera, then blurred. No shadow map, no GPU.+ if (shadowStrength > 0) {+ const groundY = bounds.min[1];+ const lightDir = lights[0].dir;+ const sx = new Float64Array(3);+ const sy = new Float64Array(3);+ for (const entry of meshes) {+ const index = entry.geometry.index;+ const world = entry.world;+ for (const group of entry.groups) {+ if (group.material.alphaMode === 'BLEND') continue;+ const start = group.start || 0;+ const count = Math.min(group.count, (index ? index.count : world.length / 3) - start);+ for (let t = 0; t < count; t += 3) {+ let ok = true;+ for (let k = 0; k < 3; k++) {+ const vi = index ? index.getX(start + t + k) : start + t + k;+ const px = world[vi * 3];+ const py = world[vi * 3 + 1];+ const pz = world[vi * 3 + 2];+ const drop = lightDir[1] > 0.05 ? (py - groundY) / lightDir[1] : 0;+ const gx = px - lightDir[0] * drop;+ const gz = pz - lightDir[2] * drop;+ const cw = vp[3] * gx + vp[7] * groundY + vp[11] * gz + vp[15];+ if (cw <= 1e-4) {+ ok = false;+ break;+ }+ const cx = vp[0] * gx + vp[4] * groundY + vp[8] * gz + vp[12];+ const cy = vp[1] * gx + vp[5] * groundY + vp[9] * gz + vp[13];+ sx[k] = (cx / cw * 0.5 + 0.5) * w;+ sy[k] = (0.5 - cy / cw * 0.5) * h;+ }+ if (ok) fillShadowTriangle(fb.shadow, w, h, sx[0], sy[0], sx[1], sy[1], sx[2], sy[2]);+ }+ }+ }+ blur(fb.shadow, w, h, Math.max(1, Math.round(Math.min(w, h) * 0.012)));+ }++ // Background, with the shadow multiplied in underneath the model.+ const transparent = bg === 'transparent' || bg === null;+ let topColor = [0, 0, 0];+ let bottomColor = [0, 0, 0];+ if (!transparent) {+ if (Array.isArray(bg)) {+ topColor = parseColor(bg[0]);+ bottomColor = parseColor(bg[1] ?? bg[0]);+ } else {+ topColor = parseColor(bg);+ bottomColor = topColor;+ }+ }+ for (let y = 0; y < h; y++) {+ const t = h > 1 ? y / (h - 1) : 0;+ const r = topColor[0] + (bottomColor[0] - topColor[0]) * t;+ const g = topColor[1] + (bottomColor[1] - topColor[1]) * t;+ const b = topColor[2] + (bottomColor[2] - topColor[2]) * t;+ for (let x = 0; x < w; x++) {+ const idx = y * w + x;+ const occlusion = 1 - Math.min(1, fb.shadow[idx] * shadowStrength);+ const ci = idx * 3;+ fb.color[ci] = r * occlusion;+ fb.color[ci + 1] = g * occlusion;+ fb.color[ci + 2] = b * occlusion;+ fb.alpha[idx] = transparent ? Math.min(1, fb.shadow[idx] * shadowStrength) : 1;+ }+ }++ const opaque = [];+ const blended = [];+ for (const entry of meshes) {+ for (const group of entry.groups) {+ (group.material.alphaMode === 'BLEND' ? blended : opaque).push({ entry, group });+ }+ }++ const vertexA = new Float32Array(V_STRIDE);+ const vertexB = new Float32Array(V_STRIDE);+ const vertexC = new Float32Array(V_STRIDE);+ const tri = [vertexA, vertexB, vertexC];++ const drawGroups = (list) => {+ for (const { entry, group } of list) {+ const geometry = entry.geometry;+ const index = geometry.index;+ const uv = group.material.mips ? geometry.attributes.uv || geometry.attributes.uv1 : null;+ const colors = geometry.attributes.color || null;+ const material = { ...group.material, hasVertexColor: Boolean(colors) };+ const total = index ? index.count : geometry.attributes.position.count;+ const start = group.start || 0;+ const count = Math.min(group.count, total - start);+ for (let t = 0; t + 2 < count; t += 3) {+ const i0 = index ? index.getX(start + t) : start + t;+ const i1 = index ? index.getX(start + t + 1) : start + t + 1;+ const i2 = index ? index.getX(start + t + 2) : start + t + 2;+ packVertex(vertexA, entry.world, entry.normals, uv, colors, i0, vp);+ packVertex(vertexB, entry.world, entry.normals, uv, colors, i1, vp);+ packVertex(vertexC, entry.world, entry.normals, uv, colors, i2, vp);+ drawTriangle(fb, tri, material, ctx);+ }+ }+ };++ drawGroups(opaque);++ if (blended.length) {+ // Depth-tested, no depth write, far to near by group centroid.+ ctx.depthWrite = false;+ ctx.blend = true;+ blended.sort((a, b) => centroidDepth(b, vp) - centroidDepth(a, vp));+ drawGroups(blended);+ ctx.depthWrite = true;+ ctx.blend = false;+ }++ return resolve(fb, width, height, ss, transparent);+}++function centroidDepth({ entry }, vp) {+ const world = entry.world;+ if (!world.length) return 0;+ let x = 0;+ let y = 0;+ let z = 0;+ const step = Math.max(3, Math.floor(world.length / 96 / 3) * 3);+ let n = 0;+ for (let i = 0; i < world.length; i += step) {+ x += world[i];+ y += world[i + 1];+ z += world[i + 2];+ n++;+ }+ if (!n) return 0;+ x /= n;+ y /= n;+ z /= n;+ const cw = vp[3] * x + vp[7] * y + vp[11] * z + vp[15];+ const cz = vp[2] * x + vp[6] * y + vp[10] * z + vp[14];+ return cw > 1e-6 ? cz / cw : Infinity;+}++function resolve(fb, width, height, ss, transparent) {+ const data = new Uint8ClampedArray(width * height * 4);+ const samples = ss * ss;+ for (let y = 0; y < height; y++) {+ for (let x = 0; x < width; x++) {+ let r = 0;+ let g = 0;+ let b = 0;+ let a = 0;+ for (let sy = 0; sy < ss; sy++) {+ const row = (y * ss + sy) * fb.width;+ for (let sx = 0; sx < ss; sx++) {+ const idx = row + x * ss + sx;+ const alpha = fb.alpha[idx];+ const ci = idx * 3;+ r += fb.color[ci] * alpha;+ g += fb.color[ci + 1] * alpha;+ b += fb.color[ci + 2] * alpha;+ a += alpha;+ }+ }+ const o = (y * width + x) * 4;+ if (a <= 1e-6) {+ data[o] = 0;+ data[o + 1] = 0;+ data[o + 2] = 0;+ data[o + 3] = 0;+ continue;+ }+ // Straight alpha: un-premultiply after the box filter so edge pixels+ // keep their colour instead of fading toward black.+ const inv = 1 / a;+ data[o] = linearToSrgb(tonemap(r * inv));+ data[o + 1] = linearToSrgb(tonemap(g * inv));+ data[o + 2] = linearToSrgb(tonemap(b * inv));+ data[o + 3] = transparent ? Math.round(Math.min(1, a / samples) * 255) : 255;+ }+ }+ return { width, height, data };+}++/** Render an animated turntable or clip playback as a list of RGBA frames. */+export async function renderFrames(model, options = {}) {+ const { frames = 1, fps = 24, spin = 0, clip = null, startTime = 0, onFrame, yaw = 26 } = options;+ if (clip !== null) model.play(clip);+ const duration = model.duration;+ const out = [];+ for (let i = 0; i < frames; i++) {+ const t = startTime + i / fps;+ model.setTime(duration > 0 ? t % duration : t);+ const frameYaw = yaw + (spin ? (i / frames) * spin : 0);+ const frame = renderFrame(model, { ...options, yaw: frameYaw });+ out.push(frame);+ if (onFrame) await onFrame(frame, i, frames);+ }+ return out;+}diff --git a/packages/render/src/retarget.js b/packages/render/src/retarget.jsnew file mode 100644index 000000000..db1112618--- /dev/null+++ b/packages/render/src/retarget.js@@ -0,0 +1,171 @@+/**+ * Bone-name retargeting for animation-only GLBs.+ * ----------------------------------------------+ * A clip authored on a Mixamo rig and an avatar exported from Ready Player Me,+ * Avaturn, VRoid or Blender agree on the shape of a humanoid skeleton and+ * disagree on almost every bone name. This module normalizes both sides and+ * rebinds the clip's rotation tracks onto whatever skeleton it is handed, so+ * one walk cycle drives every body the platform can produce.+ *+ * Rotations transfer as-is. The root translation is rescaled by the hip-height+ * ratio, which is what stops a tall rig from skating and a short one from+ * sinking through the floor. Scale tracks are dropped: they are authoring+ * noise on every humanoid rig we have met.+ */++const PREFIXES = [/^mixamorig\d*[:_]?/i, /^armature[|:_]/i, /^root[|:]/i, /^bip\d*\s*/i];++const ALIASES = new Map(+ Object.entries({+ hips: 'hips',+ pelvis: 'hips',+ root: 'hips',+ spine: 'spine',+ spine1: 'spine1',+ spine2: 'spine2',+ chest: 'spine1',+ upperchest: 'spine2',+ neck: 'neck',+ head: 'head',+ headtop: 'headtop',+ leftshoulder: 'leftshoulder',+ lshoulder: 'leftshoulder',+ leftarm: 'leftarm',+ leftupperarm: 'leftarm',+ lupperarm: 'leftarm',+ leftforearm: 'leftforearm',+ leftlowerarm: 'leftforearm',+ llowerarm: 'leftforearm',+ lefthand: 'lefthand',+ lhand: 'lefthand',+ rightshoulder: 'rightshoulder',+ rshoulder: 'rightshoulder',+ rightarm: 'rightarm',+ rightupperarm: 'rightarm',+ rupperarm: 'rightarm',+ rightforearm: 'rightforearm',+ rightlowerarm: 'rightforearm',+ rlowerarm: 'rightforearm',+ righthand: 'righthand',+ rhand: 'righthand',+ leftupleg: 'leftupleg',+ leftupperleg: 'leftupleg',+ lupperleg: 'leftupleg',+ leftthigh: 'leftupleg',+ leftleg: 'leftleg',+ leftlowerleg: 'leftleg',+ llowerleg: 'leftleg',+ leftshin: 'leftleg',+ leftfoot: 'leftfoot',+ lfoot: 'leftfoot',+ lefttoebase: 'lefttoe',+ lefttoes: 'lefttoe',+ lefttoe: 'lefttoe',+ rightupleg: 'rightupleg',+ rightupperleg: 'rightupleg',+ rupperleg: 'rightupleg',+ rightthigh: 'rightupleg',+ rightleg: 'rightleg',+ rightlowerleg: 'rightleg',+ rlowerleg: 'rightleg',+ rightshin: 'rightleg',+ rightfoot: 'rightfoot',+ rfoot: 'rightfoot',+ righttoebase: 'righttoe',+ righttoes: 'righttoe',+ righttoe: 'righttoe',+ }),+);++/** Collapse a bone name to a rig-independent key. */+export function normalizeBoneName(name) {+ let out = String(name || '');+ for (const prefix of PREFIXES) out = out.replace(prefix, '');+ // camelCase side suffix, as in `shoulderL` / `handR`. Detected before the+ // lowercase pass, because after it the marker is indistinguishable from the+ // last letter of a word like `shoulder`.+ const camelSide = out.match(/^(.*[a-z0-9])(L|R)$/);+ if (camelSide) out = `${camelSide[2].toLowerCase()}_${camelSide[1]}`;+ out = out.toLowerCase();+ // VRM / VRoid encode the side in the prefix: J_Bip_L_UpperArm.+ const vrm = out.match(/^j_(?:bip|sec|adj|opt)_([clr])_(.+)$/);+ if (vrm) out = (vrm[1] === 'c' ? '' : `${vrm[1]}_`) + vrm[2];+ // Trailing or leading side markers: `Arm.L`, `arm_l`, `l_arm`, `armLeft`.+ out = out.replace(/[._-](l|left)$/i, '#left').replace(/[._-](r|right)$/i, '#right');+ out = out.replace(/^(l|left)[._-]/i, '#left').replace(/^(r|right)[._-]/i, '#right');+ out = out.replace(/[^a-z0-9#]/g, '');+ if (out.includes('#left')) out = `left${out.replace('#left', '')}`;+ else if (out.includes('#right')) out = `right${out.replace('#right', '')}`;+ return ALIASES.get(out) || out;+}++function boneMap(skeleton) {+ const map = new Map();+ for (const bone of skeleton.bones) {+ const key = normalizeBoneName(bone.name);+ if (!map.has(key)) map.set(key, bone);+ }+ return map;+}++function hipsHeight(skeleton) {+ const map = boneMap(skeleton);+ const hips = map.get('hips') || skeleton.bones[0];+ if (!hips) return 1;+ // World height above the skeleton root reads consistently across rigs that+ // disagree about where local space sits.+ const y = hips.getWorldPosition ? hips.getWorldPosition(hips.position.clone()).y : hips.position.y;+ return Math.abs(y) > 1e-4 ? Math.abs(y) : 1;+}++/**+ * Rebind one clip onto `skeleton`. Returns a new AnimationClip, or null when+ * fewer than four bones match (which means the rigs are genuinely unrelated+ * and playing the result would only produce a scramble).+ */+export function retargetClip(clip, skeleton, root) {+ const THREE = clip.constructor;+ const map = boneMap(skeleton);+ const tracks = [];+ let matched = 0;+ let sourceHipsY = 0;++ for (const track of clip.tracks) {+ const dot = track.name.lastIndexOf('.');+ if (dot < 0) continue;+ const nodeName = track.name.slice(0, dot);+ const property = track.name.slice(dot + 1);+ const key = normalizeBoneName(nodeName);+ const bone = map.get(key);+ if (!bone) continue;+ if (property === 'quaternion') {+ const next = track.clone();+ next.name = `${bone.name}.quaternion`;+ tracks.push(next);+ matched++;+ } else if (property === 'position' && key === 'hips') {+ sourceHipsY = Math.abs(track.values[1] || 0);+ const next = track.clone();+ next.name = `${bone.name}.position`;+ tracks.push(next);+ }+ }++ if (matched < 4) return null;++ const targetHipsY = hipsHeight(skeleton);+ if (sourceHipsY > 1e-4) {+ const scale = targetHipsY / sourceHipsY;+ if (Number.isFinite(scale) && scale > 0) {+ for (const track of tracks) {+ if (!track.name.endsWith('.position')) continue;+ for (let i = 0; i < track.values.length; i++) track.values[i] *= scale;+ }+ }+ }++ void root;+ void THREE;+ const Clip = clip.constructor;+ return new Clip(clip.name, clip.duration, tracks);+}diff --git a/packages/tty-3d/README.md b/packages/tty-3d/README.mdnew file mode 100644index 000000000..5c05cdceb--- /dev/null+++ b/packages/tty-3d/README.md@@ -0,0 +1,172 @@+# `@three-ws/tty-3d`++Render a rigged, animated 3D model in a terminal. GLB in, coloured text out.++No GPU, no browser, no display server, no WebGL. It is a software rasterizer: it+decodes the glTF, evaluates the animation, blends the joint matrices per vertex+on the CPU, and rasterizes triangles into a z-buffered framebuffer sized to your+terminal. That makes it work identically over SSH, in CI, inside a container with+no display, and on a serverless instance.++```bash+npx @three-ws/tty-3d ./avatar.glb+```++Or, with nothing installed at all:++```bash+curl three.ws/tty+```++## Why this exists++three.ws gives AI agents a body. AI agents increasingly live in a terminal. This+closes that gap: the same avatar that renders in WebGL on a web page renders in+the shell where the agent actually runs.++It is also the renderer core underneath [`@three-ws/tty-avatar`](../tty-avatar),+which adds moods, avatar-id resolution and Claude Code hooks on top of a+terminal renderer. This package owns the part that is purely 3D: skeletal+animation and rasterization.++## Install++```bash+npm install @three-ws/tty-3d+```++Requires Node 18 or newer.++## CLI++```bash+three-tty <model.glb | https://...> [options]+```++| Option | Meaning | Default |+| --- | --- | --- |+| `--width <n>` | Columns | terminal width |+| `--height <n>` | Rows. One row is two vertical pixels | terminal height minus 2 |+| `--fps <n>` | Frames per second | 24 |+| `--spin <n>` | Turntable speed, radians per second. `0` holds still | 0.9 |+| `--clip <name\|n>` | Which animation to play | the first one |+| `--no-anim` | Hold the rest pose | |+| `--once` | Draw one frame and exit | implied when piped |+| `--frames <n>` | Draw n frames and exit | |+| `--time <s>` | Where in the clip to start | 0 |+| `--pitch <r>` | Camera elevation, radians | 0.08 |+| `--zoom <n>` | 1 fits the model, higher moves closer | 1 |+| `--color <mode>` | `truecolor`, `ansi256`, or `mono` | detected |+| `--transparent` | Leave the background unpainted | |+| `--info` | Print what was loaded as JSON, then exit | |++Interactive: arrow keys or `a`/`d` orbit, `w`/`s` tilt, `+`/`-` zoom, space+pauses, `q` quits.++**Piping is safe.** When stdout is not a terminal, the default is a single frame,+not an infinite animation, so `three-tty model.glb > frame.txt` terminates.++```bash+# a still frame in plain ASCII, for a README or a commit hook+three-tty avatar.glb --once --width 72 --color mono > frame.txt++# eight seconds of animation, then exit+three-tty avatar.glb --frames 200 --fps 25+```++## Library++```js+import { loadModel, createRenderer } from '@three-ws/tty-3d';++const model = await loadModel('./avatar.glb'); // path, https URL, or bytes+const renderer = createRenderer(model, { width: 80, height: 40 });++process.stdout.write(renderer.frame(1.25)); // one string per frame+```++### `loadModel(source, options?)`++`source` is a filesystem path, an `http(s)` URL, or a `Uint8Array` / `ArrayBuffer`+of GLB bytes. Returns a model with `triangleCount`, `skinned`, `animations` and+`bounds`.++`EXT_meshopt_compression` is decoded unconditionally. Most real avatars ship with+it, and a reader that skips the decoder does not fail loudly, it reads garbage+vertex data.++### `createRenderer(model, options?)`++| Option | Type | Default |+| --- | --- | --- |+| `width`, `height` | number, in characters | 96 x 48 |+| `animation` | clip name, index, or `false` for the rest pose | first clip |+| `mode` | `'truecolor'` \| `'ansi256'` \| `'mono'` | `'truecolor'` |+| `background` | `[r, g, b]` in 0..1 | near black |+| `transparent` | leave uncovered cells as spaces | `false` |+| `spin` | turntable radians per second | 1 |+| `pitch`, `zoom` | camera elevation and distance | 0.08, 1 |+| `tint` | `[r, g, b]` multiplier | white |++Returns `{ frame(time), setOrbit({yaw, pitch, zoom}), orbit, width, height, framing }`.+`frame(time)` is pure and deterministic: the same time always renders the same+string.++### Other exports++`renderOnce(source, options)`, `describeModel(model)`, `selectAnimation(model, wanted)`,+`detectColorMode(env, stream)`, `toAnsi256(r, g, b)`, `framebufferToText(fb, opts)`,+`ColorMode`, and `ansi` (cursor control strings). The lower layers are reachable+at `@three-ws/tty-3d/model`, `/raster` and `/term` if you want the framebuffer+rather than the text.++## How it looks like graphics and not ASCII art++Each character cell is a `▀` (upper half block) with its own foreground and+background colour, so one cell carries **two** independently coloured pixels.+That doubles vertical resolution and, more importantly, makes pixels square:+terminal cells are roughly 1:2, so a half-block grid has a 1:1 pixel aspect and a+sphere comes out round instead of squashed.++Colour depth is detected conservatively. Guessing truecolor wrong sprays literal+escape sequences across the user's screen; guessing 256 wrong just looks flatter.+`NO_COLOR` is honoured, and `THREE_TTY_COLOR` forces a mode.++## Three things that are easy to get wrong++These are documented because each one shipped as a bug during development and is+now covered by a regression test.++1. **The skinned mesh node transform must be ignored.** glTF defines the skinning+ matrix as `inverse(meshNodeWorld) * jointWorld * inverseBind`, then transforms+ the result by `meshNodeWorld` again, so the two cancel. Applying the leading+ inverse without re-applying the outer transform cancels the skeleton's up-axis+ correction and lays the character on its back.+2. **A matrix multiply must not alias its own output.** `multiply(m, ibm, m)`+ reads `m`'s earlier columns while writing later ones. The result looks like a+ plausible matrix and renders as spikes radiating out of every joint.+3. **Unresolvable joints must keep their slot.** `JOINTS_0` indexes the joint list+ positionally, so filtering a hole out renumbers every joint after it and rigs+ vertices to the wrong bones.++## Framing++The camera radius is fixed from the widest frame of the clip while the centre is+recomputed per frame. Fitting the current frame alone makes the camera breathe as+the silhouette changes; unioning every frame is worse on anything with root+motion, because the union is a corridor as long as the character walks and fitting+that corridor shrinks the character to a speck.++## Tests++```bash+npx vitest run packages/tty-3d/tests+```++The fixtures are the Khronos sample models this repository already ships+(`cesium-man`, a skinned walk cycle with a Z-up skeleton root; `brainstem`, many+primitives and materials).++## License++Apache-2.0.diff --git a/pages/tty.html b/pages/tty.htmlnew file mode 100644index 000000000..43d537d52--- /dev/null+++ b/pages/tty.html@@ -0,0 +1,288 @@+<!DOCTYPE html>+<html lang="en">+<head>+ <script>/* three.ws theme boot — no-flash */(function(){try{var m=localStorage.getItem('twx_theme');var l=m==='auto'?(window.matchMedia&&window.matchMedia('(prefers-color-scheme: light)').matches):m==='light';document.documentElement.setAttribute('data-theme',l?'light':'dark');}catch(e){document.documentElement.setAttribute('data-theme','dark');}})();</script>+ <meta charset="utf-8" />+ <meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover" />+ <title>Your agent, in your terminal · three.ws</title>+ <meta name="description" content="curl three.ws/tty and a rigged 3D agent walks in your terminal. A real software rasterizer streams ANSI frames over HTTP: no GPU, no WebGL, no install. Plus @three-ws/tty-3d, the CLI and SDK that render any GLB in a terminal." />+ <link rel="canonical" href="https://three.ws/tty" />+ <link rel="icon" href="/favicon.ico" sizes="any" />+ <link rel="icon" type="image/svg+xml" href="/favicon.svg" />+ <link rel="apple-touch-icon" href="/apple-touch-icon.png" />+ <link rel="manifest" href="/manifest.webmanifest" />+ <link rel="stylesheet" href="/style.css" />+ <link rel="stylesheet" href="/nav.css" />+ <link rel="stylesheet" href="/mobile.css" />+ <meta property="og:type" content="website" />+ <meta property="og:title" content="Your agent, in your terminal · three.ws" />+ <meta property="og:description" content="curl three.ws/tty and a rigged 3D agent walks in your terminal. No GPU, no WebGL, no install." />+ <meta property="og:url" content="https://three.ws/tty" />+ <meta property="og:image" content="https://three.ws/api/page-og?s=dev&t=Your+agent%2C+in+your+terminal&d=curl+three.ws%2Ftty+and+a+rigged+3D+agent+walks+in+your+terminal.+A+software+rasterizer+streams+ANSI+frames+over+HTTP.&p=%2Ftty" />+ <meta name="twitter:card" content="summary_large_image" />+ <meta name="twitter:title" content="Your agent, in your terminal · three.ws" />+ <meta name="twitter:description" content="curl three.ws/tty and a rigged 3D agent walks in your terminal. No GPU, no WebGL, no install." />+ <meta name="twitter:image" content="https://three.ws/api/page-og?s=dev&t=Your+agent%2C+in+your+terminal&d=curl+three.ws%2Ftty+and+a+rigged+3D+agent+walks+in+your+terminal.+A+software+rasterizer+streams+ANSI+frames+over+HTTP.&p=%2Ftty" />+ <style>+ :root {+ --tty-bg: #07080c;+ --tty-panel: rgba(16, 18, 26, 0.9);+ --tty-border: rgba(255, 255, 255, 0.1);+ --tty-border-hi: rgba(255, 255, 255, 0.2);+ --tty-text: #e9ebf2;+ --tty-muted: #8a90a2;+ --tty-faint: #5c6273;+ --tty-accent: #7ee787;+ --tty-accent-2: #8b5cf6;+ --tty-screen: #05060a;+ }+ html, body { background: var(--tty-bg); color: var(--tty-text); margin: 0; }+ body { font-family: var(--font-body, 'Inter', system-ui, sans-serif); min-height: 100vh; }+ * { box-sizing: border-box; }++ .wrap { max-width: 1080px; margin: 0 auto; padding: 0 20px 96px; }+ .hero { padding: 72px 0 28px; }+ .eyebrow {+ display: inline-flex; align-items: center; gap: 8px;+ font-size: 12px; letter-spacing: 0.14em; text-transform: uppercase;+ color: var(--tty-accent); border: 1px solid rgba(126, 231, 135, 0.28);+ border-radius: 999px; padding: 6px 12px; margin-bottom: 20px;+ }+ h1 { font-size: clamp(30px, 5.4vw, 54px); line-height: 1.05; margin: 0 0 16px; letter-spacing: -0.02em; }+ .lede { font-size: clamp(16px, 2.1vw, 19px); color: var(--tty-muted); max-width: 62ch; line-height: 1.6; margin: 0 0 28px; }++ .cmd {+ display: flex; align-items: center; gap: 12px; flex-wrap: wrap;+ background: var(--tty-screen); border: 1px solid var(--tty-border-hi);+ border-radius: 12px; padding: 16px 18px; margin: 0 0 14px;+ font-family: var(--font-mono, ui-monospace, 'SF Mono', Menlo, monospace);+ font-size: clamp(14px, 2.4vw, 20px);+ }+ .cmd .prompt { color: var(--tty-accent); user-select: none; }+ .cmd code { flex: 1 1 auto; min-width: 0; overflow-x: auto; white-space: nowrap; color: var(--tty-text); }+ .copy {+ flex: 0 0 auto; background: rgba(255, 255, 255, 0.06); color: var(--tty-text);+ border: 1px solid var(--tty-border); border-radius: 8px; padding: 8px 14px;+ font: inherit; font-size: 13px; cursor: pointer;+ transition: background 140ms ease, border-color 140ms ease, transform 140ms ease;+ }+ .copy:hover { background: rgba(255, 255, 255, 0.12); border-color: var(--tty-border-hi); }+ .copy:active { transform: translateY(1px); }+ .copy:focus-visible { outline: 2px solid var(--tty-accent); outline-offset: 2px; }+ .copy[data-copied="true"] { color: var(--tty-accent); border-color: rgba(126, 231, 135, 0.5); }++ .screen-shell {+ background: var(--tty-screen); border: 1px solid var(--tty-border);+ border-radius: 14px; overflow: hidden; margin: 26px 0 12px;+ box-shadow: 0 24px 60px rgba(0, 0, 0, 0.5);+ }+ .screen-bar {+ display: flex; align-items: center; gap: 8px; padding: 10px 14px;+ border-bottom: 1px solid var(--tty-border); background: rgba(255, 255, 255, 0.02);+ }+ .dot { width: 11px; height: 11px; border-radius: 50%; background: #3a3f4d; }+ .screen-bar .title {+ margin-left: 8px; font-size: 12px; color: var(--tty-faint);+ font-family: var(--font-mono, ui-monospace, monospace);+ }+ .screen-bar .status { margin-left: auto; font-size: 12px; color: var(--tty-faint); }+ #screen {+ margin: 0; padding: 14px; overflow-x: auto; line-height: 1.0;+ font-family: var(--font-mono, ui-monospace, 'SF Mono', Menlo, monospace);+ font-size: 11px; letter-spacing: 0; white-space: pre; min-height: 340px;+ color: var(--tty-muted);+ }+ @media (max-width: 720px) { #screen { font-size: 7px; min-height: 240px; } }+ .screen-actions { display: flex; gap: 10px; flex-wrap: wrap; margin-bottom: 40px; }+ .btn {+ background: rgba(255, 255, 255, 0.05); color: var(--tty-text);+ border: 1px solid var(--tty-border); border-radius: 8px;+ padding: 9px 15px; font: inherit; font-size: 13px; cursor: pointer;+ transition: background 140ms ease, border-color 140ms ease, transform 140ms ease;+ }+ .btn:hover { background: rgba(255, 255, 255, 0.1); border-color: var(--tty-border-hi); }+ .btn:active { transform: translateY(1px); }+ .btn:focus-visible { outline: 2px solid var(--tty-accent); outline-offset: 2px; }+ .btn[aria-pressed="true"] { border-color: var(--tty-accent); color: var(--tty-accent); }++ h2 { font-size: clamp(20px, 3vw, 27px); margin: 52px 0 14px; letter-spacing: -0.01em; }+ h3 { font-size: 15px; margin: 26px 0 8px; color: var(--tty-text); }+ p { color: var(--tty-muted); line-height: 1.65; max-width: 68ch; }+ a { color: var(--tty-accent); }+ pre.block {+ background: var(--tty-screen); border: 1px solid var(--tty-border);+ border-radius: 10px; padding: 15px 17px; overflow-x: auto;+ font-family: var(--font-mono, ui-monospace, monospace); font-size: 13px;+ line-height: 1.6; color: #cdd3e1;+ }+ .grid { display: grid; gap: 14px; grid-template-columns: repeat(auto-fit, minmax(230px, 1fr)); margin: 18px 0; }+ .card {+ background: var(--tty-panel); border: 1px solid var(--tty-border);+ border-radius: 12px; padding: 16px 18px;+ transition: border-color 160ms ease, transform 160ms ease;+ }+ .card:hover { border-color: var(--tty-border-hi); transform: translateY(-2px); }+ .card h3 { margin: 0 0 6px; font-size: 14px; }+ .card p { margin: 0; font-size: 13.5px; }+ table { width: 100%; border-collapse: collapse; margin: 14px 0; font-size: 14px; }+ th, td { text-align: left; padding: 9px 12px; border-bottom: 1px solid var(--tty-border); }+ th { color: var(--tty-faint); font-weight: 600; font-size: 12px; text-transform: uppercase; letter-spacing: 0.06em; }+ td code { color: var(--tty-accent); font-family: var(--font-mono, ui-monospace, monospace); font-size: 13px; }+ .note {+ border-left: 2px solid var(--tty-accent-2); padding: 2px 0 2px 16px;+ margin: 18px 0; color: var(--tty-muted); font-size: 14.5px; line-height: 1.65;+ }+ footer { margin-top: 64px; padding-top: 26px; border-top: 1px solid var(--tty-border); color: var(--tty-faint); font-size: 13px; }+ footer a { margin-right: 18px; }+ @media (prefers-reduced-motion: reduce) { .card, .btn, .copy { transition: none; } }+ </style>+</head>+<body>+ <header class="site-header"><div id="nav-container"></div></header>+ <script src="/nav.js"></script>++ <main class="wrap">+ <section class="hero">+ <span class="eyebrow">no gpu · no webgl · no install</span>+ <h1>Your agent, in your terminal.</h1>+ <p class="lede">+ One command. A rigged 3D character loads, plays its animation, and turns on a+ turntable, drawn entirely out of coloured text. There is nothing to install and+ nothing to run: the frames are rasterized on our servers and streamed to your+ shell as they are drawn.+ </p>++ <div class="cmd">+ <span class="prompt" aria-hidden="true">$</span>+ <code id="curl-cmd">curl three.ws/tty</code>+ <button class="copy" type="button" data-copy-target="curl-cmd">Copy</button>+ </div>+ <p style="font-size:13.5px;margin-top:0">+ Works in any terminal that renders colour. Press Ctrl-C to stop it.+ </p>+ </section>++ <section aria-labelledby="demo-heading">+ <h2 id="demo-heading" style="margin-top:22px">This is the actual output</h2>+ <p>+ Not a video and not a recording. The panel below opens the same HTTP stream+ <code style="color:var(--tty-accent)">curl</code> would, and paints the escape+ sequences it receives.+ </p>+ <div class="screen-shell">+ <div class="screen-bar">+ <span class="dot" aria-hidden="true"></span>+ <span class="dot" aria-hidden="true"></span>+ <span class="dot" aria-hidden="true"></span>+ <span class="title">curl three.ws/tty</span>+ <span class="status" id="stream-status" role="status" aria-live="polite">idle</span>+ </div>+ <pre id="screen" tabindex="0" aria-label="Live terminal render of a 3D agent"></pre>+ </div>+ <div class="screen-actions">+ <button class="btn" type="button" id="btn-play">Run it</button>+ <button class="btn" type="button" id="btn-spin" aria-pressed="true">Turntable</button>+ <button class="btn" type="button" id="btn-color" aria-pressed="true">Colour</button>+ </div>+ </section>++ <section>+ <h2>Render any model, locally</h2>+ <p>+ The renderer is a published package. It takes a GLB, decodes it (including+ <code style="color:var(--tty-accent)">EXT_meshopt_compression</code>, which most+ real avatars ship with), evaluates the animation, applies linear blend skinning+ on the CPU, and rasterizes with a z-buffer. No GPU is involved at any point, so+ it runs the same over SSH, in CI, or in a container with no display.+ </p>+ <pre class="block"><code># a local file+npx @three-ws/tty-3d ./avatar.glb++# anything on the web+npx @three-ws/tty-3d https://three.ws/avatars/default.glb --spin 1.4++# a still frame, piped somewhere+npx @three-ws/tty-3d avatar.glb --once --width 80 --color mono > frame.txt</code></pre>+ <p style="font-size:13.5px">+ Arrow keys orbit, <code style="color:var(--tty-accent)">w</code>/<code style="color:var(--tty-accent)">s</code>+ tilt, <code style="color:var(--tty-accent)">+</code>/<code style="color:var(--tty-accent)">-</code> zoom,+ space pauses, <code style="color:var(--tty-accent)">q</code> quits.+ </p>++ <h2>Or as a library</h2>+ <pre class="block"><code>import { loadModel, createRenderer } from '@three-ws/tty-3d';++const model = await loadModel('./avatar.glb');+const renderer = createRenderer(model, { width: 80, height: 40 });++// One string per frame. Write it wherever you like.+process.stdout.write(renderer.frame(1.25));</code></pre>+ </section>++ <section>+ <h2>The hosted endpoint</h2>+ <p>+ <code style="color:var(--tty-accent)">GET /api/tty</code> streams frames as+ <code style="color:var(--tty-accent)">text/plain</code>, paced in real time, so the+ response itself is the animation. Every parameter is optional.+ </p>+ <table>+ <thead><tr><th>Parameter</th><th>Meaning</th><th>Default</th></tr></thead>+ <tbody>+ <tr><td><code>avatar</code></td><td>A public three.ws avatar id. Also the path form, <code>/tty/<id></code>.</td><td>the three.ws model</td></tr>+ <tr><td><code>src</code></td><td>Any public GLB URL.</td><td>none</td></tr>+ <tr><td><code>w</code>, <code>h</code></td><td>Columns and rows.</td><td>76 x 30</td></tr>+ <tr><td><code>frames</code></td><td>How many frames to send. <code>1</code> gives a still.</td><td>8 seconds</td></tr>+ <tr><td><code>fps</code></td><td>Frames per second, 4 to 30.</td><td>18</td></tr>+ <tr><td><code>t</code></td><td>Where in the clip to start, in seconds.</td><td>0</td></tr>+ <tr><td><code>spin</code></td><td>Turntable speed in radians per second. <code>0</code> holds still.</td><td>0.9</td></tr>+ <tr><td><code>clip</code></td><td>Animation name to play.</td><td>the first clip</td></tr>+ <tr><td><code>color</code></td><td><code>truecolor</code>, <code>ansi256</code>, or <code>mono</code>.</td><td>ansi256</td></tr>+ <tr><td><code>zoom</code>, <code>pitch</code></td><td>Camera distance and elevation.</td><td>1, 0.08</td></tr>+ </tbody>+ </table>+ <pre class="block"><code># your own avatar, wider, held still+curl 'three.ws/tty/<avatar-id>?w=110&spin=0'++# a single frame in plain ASCII, for a README or a commit hook+curl 'three.ws/tty?frames=1&color=mono&w=60'</code></pre>+ <p class="note">+ A browser asking for this URL is redirected here instead, because a browser would+ show the escape codes as literal garbage. Pass+ <code style="color:var(--tty-accent)">?raw=1</code> to see the bytes anyway.+ </p>+ </section>++ <section>+ <h2>How it works</h2>+ <div class="grid">+ <div class="card">+ <h3>Half blocks, not ASCII art</h3>+ <p>Each cell is a <code>▀</code> with its own foreground and background colour, so one character carries two pixels. That doubles vertical resolution and makes pixels square, which is why a sphere comes out round.</p>+ </div>+ <div class="card">+ <h3>Real skinning</h3>+ <p>Joint matrices are evaluated per frame and blended per vertex, so a rigged avatar actually moves its limbs. It is not a spinning static mesh.</p>+ </div>+ <div class="card">+ <h3>Framing that holds still</h3>+ <p>The camera radius is fixed from the widest frame of the clip and the centre tracks the pose, so a walk cycle with root motion neither breathes nor walks out of shot.</p>+ </div>+ <div class="card">+ <h3>Silhouette-first shading</h3>+ <p>A key light, a fill, and a rim term. At this resolution the outline carries the shape, so faces turning away are lifted rather than crushed to black.</p>+ </div>+ </div>+ </section>++ <footer>+ <a href="/docs/tty">Docs</a>+ <a href="/avatars">Avatars</a>+ <a href="/create">Create an agent</a>+ <a href="https://github.com/nirholas/three.ws/tree/main/packages/tty-3d">Source</a>+ </footer>+ </main>++ <script type="module" src="/src/tty-page.js"></script>+</body>+</html>diff --git a/src/tty-page.js b/src/tty-page.jsnew file mode 100644index 000000000..7e20ee1d0--- /dev/null+++ b/src/tty-page.js@@ -0,0 +1,249 @@+// /tty — the live panel that proves the claim on the page.+//+// It opens the same HTTP stream `curl three.ws/tty` opens and paints what comes+// back. Nothing is pre-rendered and nothing is faked: if the endpoint breaks,+// this panel breaks with it, which is the point of showing it.+//+// The parser handles exactly the subset of ANSI the renderer emits (SGR colour,+// cursor-up, cursor visibility). A general terminal emulator would be a large+// dependency to prove a small claim.++const ESC = '\x1b';++const screen = document.getElementById('screen');+const statusEl = document.getElementById('stream-status');+const playBtn = document.getElementById('btn-play');+const spinBtn = document.getElementById('btn-spin');+const colorBtn = document.getElementById('btn-color');++/** xterm-256 to rgb: 16 base colours, a 6x6x6 cube, then a 24-step grey ramp. */+const BASE_16 = [+ [0, 0, 0], [205, 49, 49], [13, 188, 121], [229, 229, 16],+ [36, 114, 200], [188, 63, 188], [17, 168, 205], [229, 229, 229],+ [102, 102, 102], [241, 76, 76], [35, 209, 139], [245, 245, 67],+ [59, 142, 234], [214, 112, 214], [41, 184, 219], [255, 255, 255],+];+const CUBE_STEPS = [0, 95, 135, 175, 215, 255];++function ansi256ToRgb(index) {+ if (index < 16) return BASE_16[index];+ if (index < 232) {+ const n = index - 16;+ return [CUBE_STEPS[Math.floor(n / 36) % 6], CUBE_STEPS[Math.floor(n / 6) % 6], CUBE_STEPS[n % 6]];+ }+ const v = 8 + (index - 232) * 10;+ return [v, v, v];+}++const rgbCss = ([r, g, b]) => `rgb(${r},${g},${b})`;++/**+ * Turn one frame of SGR-coloured text into HTML.+ *+ * Cells are coalesced into a span per colour run, matching how the renderer+ * emits escapes only on change. A span per character would be ~2000 nodes per+ * frame and would drop the frame rate on its own.+ */+function frameToHtml(text) {+ let html = '';+ let fg = null;+ let bg = null;+ let open = false;+ let buffer = '';++ const flush = () => {+ if (!buffer) return;+ const safe = buffer.replace(/&/g, '&').replace(/</g, '<').replace(/>/g, '>');+ if (fg || bg) {+ const style = `${fg ? `color:${rgbCss(fg)};` : ''}${bg ? `background:${rgbCss(bg)};` : ''}`;+ html += `<span style="${style}">${safe}</span>`;+ } else {+ html += safe;+ }+ buffer = '';+ };++ for (let i = 0; i < text.length; i += 1) {+ const ch = text[i];+ if (ch !== ESC) {+ buffer += ch;+ open = true;+ continue;+ }+ const match = /^\x1b\[([0-9;]*)m/.exec(text.slice(i));+ if (!match) {+ // Any other CSI sequence (cursor moves, visibility) is consumed and+ // ignored: the panel redraws whole frames, so cursor motion is noise.+ const skip = /^\x1b\[[0-9;?]*[A-Za-z]/.exec(text.slice(i));+ i += (skip ? skip[0].length : 1) - 1;+ continue;+ }+ flush();+ const params = match[1].split(';').filter((p) => p !== '');+ if (!params.length || params[0] === '0') { fg = null; bg = null; }+ for (let p = 0; p < params.length; p += 1) {+ if (params[p] === '38' && params[p + 1] === '5') { fg = ansi256ToRgb(Number(params[p + 2])); p += 2; }+ else if (params[p] === '48' && params[p + 1] === '5') { bg = ansi256ToRgb(Number(params[p + 2])); p += 2; }+ else if (params[p] === '38' && params[p + 1] === '2') { fg = [+params[p + 2], +params[p + 3], +params[p + 4]]; p += 4; }+ else if (params[p] === '48' && params[p + 1] === '2') { bg = [+params[p + 2], +params[p + 3], +params[p + 4]]; p += 4; }+ }+ i += match[0].length - 1;+ }+ flush();+ return open ? html : '';+}++const state = { spin: true, color: true, controller: null, running: false };++function setStatus(text) {+ if (statusEl) statusEl.textContent = text;+}++function rowsForViewport() {+ return window.innerWidth < 720 ? 26 : 30;+}++function colsForViewport() {+ if (!screen) return 76;+ // Measure one monospace character rather than assuming a ratio: font size is+ // set in CSS and changes at the mobile breakpoint.+ const probe = document.createElement('span');+ probe.textContent = '0'.repeat(50);+ probe.style.cssText = 'position:absolute;visibility:hidden;white-space:pre';+ screen.appendChild(probe);+ const charWidth = probe.getBoundingClientRect().width / 50;+ probe.remove();+ const usable = screen.clientWidth - 28;+ if (!charWidth || !usable) return 76;+ return Math.max(30, Math.min(140, Math.floor(usable / charWidth)));+}++function stop() {+ if (state.controller) state.controller.abort();+ state.controller = null;+ state.running = false;+ if (playBtn) playBtn.textContent = 'Run it';+ setStatus('stopped');+}++async function run() {+ stop();+ const controller = new AbortController();+ state.controller = controller;+ state.running = true;+ if (playBtn) playBtn.textContent = 'Stop';+ setStatus('connecting');++ const params = new URLSearchParams({+ w: String(colsForViewport()),+ h: String(rowsForViewport()),+ fps: '16',+ frames: '400',+ spin: state.spin ? '0.9' : '0',+ color: state.color ? 'ansi256' : 'mono',+ });+ // Landing here from /tty/<id> in a browser carries the avatar through, so the+ // panel shows the model the visitor actually asked for rather than the default.+ const asked = new URLSearchParams(location.search).get('avatar');+ if (asked) params.set('avatar', asked);++ try {+ const res = await fetch(`/api/tty?${params}`, { signal: controller.signal });+ if (!res.ok || !res.body) throw new Error(`stream unavailable (${res.status})`);+ setStatus('streaming');++ const reader = res.body.getReader();+ const decoder = new TextDecoder();+ let pending = '';++ for (;;) {+ const { done, value } = await reader.read();+ if (done) break;+ pending += decoder.decode(value, { stream: true });+ // Frames are separated by the cursor-up sequence the renderer writes+ // before each one. Keep only the last complete frame in the buffer: if+ // the tab was backgrounded and frames piled up, catching up beats+ // replaying a queue in fast-forward.+ const parts = pending.split(/\x1b\[\d+A/);+ if (parts.length > 1) {+ const latest = parts[parts.length - 2];+ pending = parts[parts.length - 1];+ const html = frameToHtml(latest);+ if (html) screen.innerHTML = html;+ }+ }+ setStatus('done');+ } catch (err) {+ if (err.name === 'AbortError') return;+ setStatus('unavailable');+ screen.textContent = `Could not reach the render stream: ${err.message}\n\nThe command still works from a terminal:\n\n $ curl three.ws/tty`;+ } finally {+ if (state.controller === controller) {+ state.controller = null;+ state.running = false;+ if (playBtn) playBtn.textContent = 'Run it';+ }+ }+}++if (playBtn) {+ playBtn.addEventListener('click', () => (state.running ? stop() : run()));+}+if (spinBtn) {+ spinBtn.addEventListener('click', () => {+ state.spin = !state.spin;+ spinBtn.setAttribute('aria-pressed', String(state.spin));+ if (state.running) run();+ });+}+if (colorBtn) {+ colorBtn.addEventListener('click', () => {+ state.color = !state.color;+ colorBtn.setAttribute('aria-pressed', String(state.color));+ if (state.running) run();+ });+}++for (const button of document.querySelectorAll('[data-copy-target]')) {+ button.addEventListener('click', async () => {+ const source = document.getElementById(button.dataset.copyTarget);+ if (!source) return;+ try {+ await navigator.clipboard.writeText(source.textContent.trim());+ button.textContent = 'Copied';+ button.dataset.copied = 'true';+ setTimeout(() => {+ button.textContent = 'Copy';+ delete button.dataset.copied;+ }, 1600);+ } catch {+ // Clipboard access can be denied outright. Selecting the text is a real+ // fallback, not a message telling the user to do it themselves.+ const range = document.createRange();+ range.selectNodeContents(source);+ const selection = window.getSelection();+ selection.removeAllRanges();+ selection.addRange(range);+ button.textContent = 'Select and copy';+ setTimeout(() => { button.textContent = 'Copy'; }, 2200);+ }+ });+}++// Stop the stream when the tab is hidden. A backgrounded tab holding an open+// render stream is pure server cost for frames nobody sees.+document.addEventListener('visibilitychange', () => {+ if (document.hidden && state.running) stop();+});++window.addEventListener('beforeunload', stop);++// Autoplay only where it is cheap and wanted: a real pointer, a wide viewport,+// and no reduced-motion preference.+const wantsMotion = !window.matchMedia('(prefers-reduced-motion: reduce)').matches;+if (wantsMotion && window.innerWidth >= 720) {+ run();+} else {+ setStatus('press run');+ screen.textContent = ' $ curl three.ws/tty';+}