Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
JavaScript 3d-generationai-agentsanimationavatardigital-humangenerative-aiglbgltfimage-to-3dllmmcpmcp-servermodel-context-protocolsolanatext-to-3dthreejsweb-componentswebglwebxrx402
docs(renderer): document the opt-in WebGPU path with measured parity and timings, add changelog, structure row and progress log
nirholasacbc40f51c17
diff --git a/STRUCTURE.md b/STRUCTURE.mdindex 8c53a273a..b5e2d2e66 100644--- a/STRUCTURE.md+++ b/STRUCTURE.md@@ -161,6 +161,7 @@ packages, a generated standalone GitHub mirror). See| Conversational refinement + remix economy | Iterate: [api/_mcp-studio/tools.js](api/_mcp-studio/tools.js) `refine_model` (free, on `/api/mcp-studio`) + [mcp-server/src/tools/refine-model.js](mcp-server/src/tools/refine-model.js) (paid stdio, $0.25); shared lineage core [mcp-server/src/tools/_lineage.js](mcp-server/src/tools/_lineage.js). Remix (paid track): feed [api/remix-feed.js](api/remix-feed.js) `→ /api/remix-feed` (free browse/publish/lineage), settlement [api/x402/remix-asset.js](api/x402/remix-asset.js) `→ /api/x402/remix-asset` + core [api/_lib/remix-settlement.js](api/_lib/remix-settlement.js) + split math [api/_lib/remix-royalty.js](api/_lib/remix-royalty.js). Docs [docs/remix.md](docs/remix.md), [docs/mcp-studio.md](docs/mcp-studio.md). | Live | Talk to a model to iterate it ("make it metallic") — real anchored re-generation with a revertable/branchable version lineage rendered as a viewer version strip; free on both the OpenAI 3D Studio and the paid MCP server. Creators publish a finished model as remixable (provenance + license + royalty on the existing `forge_creations` rows — no parallel store); another agent pays $0.25 USDC to remix it and a creator-set royalty (≤20%, capped) routes on-chain to the original creator via the audited `transferSolanaUSDC` rail. Split/caps/payout unit-tested ([tests/remix-settlement.test.js](tests/remix-settlement.test.js), [tests/remix-royalty.test.js](tests/remix-royalty.test.js), [tests/refine-lineage.test.js](tests/refine-lineage.test.js)). Free track stays crypto-free. Prompt 09. || Terminal renderer (3D in a shell, no GPU) | Renderer core [packages/tty-3d/](packages/tty-3d) (`@three-ws/tty-3d`, CLI `three-tty`), hosted stream [api/tty.js](api/tty.js) `→ /tty`, `/tty/:id`, `/api/tty`, page [pages/tty.html](pages/tty.html) + [src/tty-page.js](src/tty-page.js) `→ /tty`. Docs: [docs/tty.md](docs/tty.md). Tests: [packages/tty-3d/tests/tty-3d.test.js](packages/tty-3d/tests/tty-3d.test.js), [tests/api-tty.test.js](tests/api-tty.test.js) | Live | `curl three.ws/tty` and a rigged agent walks in your terminal, with nothing installed on either side. A pure-CPU software rasterizer (z-buffer, per-vertex linear blend skinning off the glTF animation samplers, meshopt decode on the way in) draws into a framebuffer the size of the caller's terminal and encodes it as `▀` half-blocks, so one cell carries two independently coloured pixels and pixels come out square. The server streams paced frames as `text/plain`, so the HTTP response IS the animation. Sibling of [packages/tty-avatar/](packages/tty-avatar), which adds moods, avatar-id resolution and Claude Code hooks on top of a terminal renderer but deliberately does not do skeletal animation; this package owns the 3D. || Restyle Studio (material/PBR editor + AI restyle + variants) | [pages/restyle.html](pages/restyle.html) `→ /restyle`, [src/restyle.js](src/restyle.js). PBR preset/variant library [packages/viewer-presets/src/materials.js](packages/viewer-presets/src/materials.js) (`@three-ws/viewer-presets`). AI restyle + persistence core [api/_lib/material-studio-store.js](api/_lib/material-studio-store.js), free hosted endpoint [api/material-studio.js](api/material-studio.js) `→ /api/material-studio`; paid agent twin [mcp-server/src/tools/restyle-material.js](mcp-server/src/tools/restyle-material.js) (`restyle_material`, $0.05 stdio MCP), thin core [mcp-server/src/tools/_material-core.js](mcp-server/src/tools/_material-core.js). Shares the `refine_model` lineage core ([mcp-server/src/tools/_lineage.js](mcp-server/src/tools/_lineage.js)) so a restyle/variant is just another kind of version, feeding prompt 09 remix. Docs: [docs/restyle.md](docs/restyle.md), [docs/mcp-tools.md](docs/mcp-tools.md). | Live | Re-skin any GLB without regenerating its mesh: live PBR editing (color/metalness/roughness/emissive/texture) with a one-click preset library (18 looks: chrome, gold, copper, brushed steel, gunmetal, matte, glossy, rubber, ceramic, glass, real glass, wood, stone, fabric, skin, car paint, neon, holographic), seeded reproducible colorway variants (mulberry32 PRNG, so the same base plus seed yields the same set), and free-text AI restyle ("make it chrome", "cyberpunk neon") via IBM Granite-proposed PBR factors applied + re-exported server-side. Every AI restyle / saved checkpoint mints a real, `gltf-validator`-checked, durably-stored GLB recorded in an immutable parent to child lineage: non-destructive in the durable sense, not just an in-browser Reset. Every restyle, variant, and HoloPart part regeneration is checked against its own source with [Model Diff](docs/model-diff.md)'s engine via [api/_lib/preserve-contract.js](api/_lib/preserve-contract.js) before it is ever persisted: topology, UVs, skeleton, skin and clips must come back unchanged (a part regenerate relaxes that to skeleton and clips, since the selected part's own shape is meant to change), or the operation throws instead of returning a URL. Each lineage step with a parent exposes a "Diff" link straight to the Model Diff page, pre-loaded with that exact before/after pair. Every server failure surfaces the API's own `error_description`, and a storage outage says so and points at Export GLB, which is wholly in-browser. Roadmap prompt 06. |+| Renderer factory (opt-in WebGPU) | [src/renderer-factory.js](src/renderer-factory.js), TSL view effects [src/restyle-effects.js](src/restyle-effects.js), bench [scripts/renderer-bench.mjs](scripts/renderer-bench.mjs) | Live (Restyle only) | WebGPU when the page opts in with `?renderer=webgpu` or a `renderer` attribute and an adapter initialises, classic WebGL2 otherwise. Wired into Restyle Studio; the main viewer stays on WebGL2 until its post-processing moves to TSL. Docs: [docs/webgpu-renderer.md](docs/webgpu-renderer.md). Tests: [tests/renderer-factory.test.js](tests/renderer-factory.test.js) || Embodiment: a living agent body in the chat | Hosted embed [pages/embodiment/embed.html](pages/embodiment/embed.html) `→ /embodiment/embed` (Vite input + vercel route, `frame-ancestors *`); engine [apps-sdk/embodiment/embodiment-stage.js](apps-sdk/embodiment/embodiment-stage.js) (`EmbodimentStage`) + [overlay.js](apps-sdk/embodiment/overlay.js); persona tools (free) [api/_mcp-studio/persona-tools.js](api/_mcp-studio/persona-tools.js) → `/api/mcp-studio` (`create_agent_persona`, `get_agent_persona`, `persona_say`; deliberately absent from the ChatGPT plugin surface `/api/mcp-chatgpt`, because their widget needs `frameDomains`) and (paid) [api/_mcp3d/tools/studio.js](api/_mcp3d/tools/studio.js); shared inline-body artifact [api/_lib/embodiment-artifact.js](api/_lib/embodiment-artifact.js); identity store [api/_lib/persona-store.js](api/_lib/persona-store.js) (Postgres `agent_personas` + fs fallback + durable R2 GLB); reload route [api/mcp3d/persona.js](api/mcp3d/persona.js) `→ /api/mcp3d/persona`; live chain-state feed [api/mcp3d/persona-identity.js](api/mcp3d/persona-identity.js) `→ /api/mcp3d/persona-identity` (polled only when the embed is opened with `?wallet=1`). Emotion/rig/lip-sync cores under [src/embodiment/](src/embodiment) + [src/voice/](src/voice). | Live | A generated avatar becomes a persistent, named body that renders inline in ChatGPT/Claude: it lip-syncs each reply, blends the matching expression + gesture, idles between turns, and reloads by `persona_id` in a fresh session (the id is the capability, no sign-in). Rides the universal canonicalize/retarget clip library; a rig without face morphs animates the jaw, a non-humanoid GLB falls back to a gentle idle (`decideRigMode`/`supportsCanonicalClips`), never a bind-pose T-pose. Zero crypto/token surface. Tests: [tests/persona-store.test.js](tests/persona-store.test.js), [tests/api/persona-resolve-route.test.js](tests/api/persona-resolve-route.test.js), [tests/api/persona-identity-route.test.js](tests/api/persona-identity-route.test.js), [tests/mcp-studio.test.js](tests/mcp-studio.test.js), [tests/embodiment-rig-mode.test.js](tests/embodiment-rig-mode.test.js). Evidence: [prompts/store-submissions/_generated/embodiment/](prompts/store-submissions/_generated/embodiment/). Store-submission prompt 07. || Claude.ai artifact embed + builder | Endpoint [api/artifact.js](api/artifact.js) `→ /api/artifact`, prebuilt viewer [public/artifact-viewer.bundle.js](public/artifact-viewer.bundle.js) (source [scripts/artifact-viewer/src.js](scripts/artifact-viewer/src.js), `npm run build:artifact-viewer`), builder page [public/artifact/index.html](public/artifact/index.html) `→ /artifact` + [README](public/artifact/README.md), contract [specs/CLAUDE_ARTIFACT.md](specs/CLAUDE_ARTIFACT.md), vendored sandbox CSP [public/claude-artifact-csp.txt](public/claude-artifact-csp.txt) (refresh: [scripts/refresh-claude-csp.mjs](scripts/refresh-claude-csp.mjs)). | Live | `GET /api/artifact?agent=<uuid>` returns ONE self-contained HTML document with three.js, GLTFLoader, the viewer and the agent GLB all inlined, because Claude's artifact sandbox CSP blocks every external script and fetch. Paste the URL into a Claude conversation and the agent renders inline. The builder page is the pre-flight: it lists real public agents that carry an embeddable avatar (`/api/agents/public?avatar=1`), renders the response in a sandboxed iframe under the same CSP Claude applies, and reports payload size, fetch time, frame render time and sandbox compliance before you paste. Size is graded against the ceiling the endpoint can actually emit (a 6 MB model base64-inlines to 8 MB, plus the 565 KB viewer), not against the raw model cap, so a large-but-accepted artifact is not flagged as over budget. `?model=<glb url>` is the viewer-only lane (allowlisted https origins, SSRF-guarded, 6 MB cap). Tests: [tests/api/artifact.test.js](tests/api/artifact.test.js) pins the generated HTML against the vendored Claude CSP. || Avatar Artifact: the standalone shareable viewer | Page [pages/avatar-artifact.html](pages/avatar-artifact.html) `→ /avatar-artifact`, scene + artifact loader [src/avatar-artifact.js](src/avatar-artifact.js), docs [docs/avatar-artifact.md](docs/avatar-artifact.md). | Live | One page, one avatar, no wrapper: the URL you paste into an `<iframe>` on someone else's site or send to a person who has never heard of three.ws. `?agent=<uuid>` renders that agent's avatar from the live public record (the name becomes the heading and the document title, an Open profile link appears), `?model=<glb url>` renders any GLB from the same allowlisted hosts `/api/artifact` enforces, and a bare link renders the house portrait: a procedural figure that tracks cursor, touch and arrow keys. Ships no third-party CDN (three.js is bundled, because an embed host never agreed to a cross-origin script) and suspends its render loop on a hidden tab. An avatar with no baked clips is driven by the shared retargeted `idle` from the universal clip library rather than left in a bind-pose T-pose. Every failure lane (no such agent, private avatar, no avatar yet, blocked host, dead model, no WebGL) has its own sentence and its own next move; the Copy embed snippet carries the artifact parameters so the iframe shows what you were looking at. |diff --git a/data/changelog.json b/data/changelog.jsonindex d1c7823e6..cff8a3f92 100644--- a/data/changelog.json+++ b/data/changelog.json@@ -1,6 +1,16 @@{"$comment": "Curated changelog entries for three.ws, the editorial layer of the public changelog. New PAGE launches are tracked automatically via the `added` field in data/pages.json; this file is for everything else holders care about: improvements, fixes, SDK releases, trading/payments capabilities, security work. scripts/build-page-index.mjs merges both sources into CHANGELOG.md, public/changelog.json, and public/changelog.xml. Newest first. Allowed tags: feature, improvement, fix, sdk, infra, docs, security, desktop (desktop entries become the desktop app's release notes). `link` is optional and must be a live three.ws path.","entries": [+ {+ "date": "2026-10-24",+ "title": "Opt-in WebGPU renderer with automatic WebGL2 fallback, starting in Restyle Studio",+ "summary": "three is now r186. Add ?renderer=webgpu to /restyle to render on WebGPU with a TSL rim light, or leave it off for the same WebGL2 picture as before. If the browser has no GPU adapter or WebGPU fails to start, the page falls back to WebGL2 on its own. A badge shows which backend is running, and exported GLBs are identical on both paths. Measured parity and timings are in the WebGPU renderer doc.",+ "tags": [+ "feature",+ "improvement"+ ],+ "link": "/docs/webgpu-renderer"+ },{"date": "2026-10-23","title": "Revoking a connected app now stops it everywhere",diff --git a/docs/webgpu-renderer.md b/docs/webgpu-renderer.mdnew file mode 100644index 000000000..0233f3232--- /dev/null+++ b/docs/webgpu-renderer.md@@ -0,0 +1,48 @@+# WebGPU renderer (opt-in)++three.ws ships on three r186. The classic WebGL2 renderer is still the default everywhere. WebGPU is strictly opt-in and always degrades to the same WebGL2 picture.++## Opting in++- Query string: `/restyle?renderer=webgpu`+- Element attribute (read by `pagePreference(el)`): `renderer="webgpu"`. The attribute wins over the query string, so an embedder can pin `renderer="webgl"`.++Only the exact value `webgpu` opts in. The factory is [src/renderer-factory.js](../src/renderer-factory.js): `createAutoRenderer(options, { prefer, fallback })` returns `{ renderer, backend, reason }` and stamps `data-renderer` on the canvas. It falls back to the guarded WebGL2 path when `navigator.gpu` is missing, `requestAdapter()` resolves null or throws, `WebGPURenderer.init()` fails, or the node renderer lands on its own GL backend. Callers branch on `backend`, never on what was requested.++## Where it is wired++[Restyle Studio](restyle.md) (`/restyle`) uses the factory. Its view-only Fresnel rim light lives in [src/restyle-effects.js](../src/restyle-effects.js):++- WebGL2: an `onBeforeCompile` GLSL patch on the glTF PBR materials.+- WebGPU: `MeshStandard/PhysicalNodeMaterial` with a TSL `emissiveNode` (`three/tsl`).++Node materials are not glTF-exportable, so on WebGPU every export goes through `withClassicMaterials`, which swaps the original classic materials back (carrying live edits) for the duration of the export. The exported GLB is identical on both paths and never contains the rim.++A renderer badge in the Restyle toolbar shows the backend in use. The Rim slider and color are in the same toolbar.++## Not ported, and why++- The main viewer and `<agent-3d>` still use `CinematicPipeline` (EffectComposer), which is WebGL-only. The factory and the `renderer` attribute parser are ready, but the viewer is not switched until its post-processing moves to TSL.+- `three/examples` `GaussianSplat` is WebGPU-only. The viewer splat stage keeps Spark, which also runs on WebGL2.+- `three-mesh-bvh/webgpu` (pinned 0.9.16) accelerates GPU-side queries. The viewer's picking is CPU raycasting, so it does not apply.++## Measuring++```+npm run dev # or any server on another port+node scripts/renderer-bench.mjs --base http://localhost:3000 --frames 200+```++The harness loads `/restyle?bench` three ways (default, `?renderer=webgpu`, `?renderer=webgpu` with `navigator.gpu` removed) and reports backend, console errors, offscreen render time per frame, pixel difference and silhouette overlap against the default, and the rim effect. It exits non-zero on any console error or wrong fallback. Parity and timing use an offscreen render target with readback because headless software WebGPU cannot present to a canvas.++### Results (2026-10-24, software adapter)++Headless Chromium with SwiftShader, 384x512 target, 200 frames, same model. This is a software rasterizer, so it compares the two code paths and is not GPU hardware data. Run the command above on a machine with a GPU for hardware numbers.++| Case | Backend | Render ms/frame (2 runs) | Mean diff vs default | Silhouette overlap | Console errors |+| --- | --- | --- | --- | --- | --- |+| default | WebGL2 | 9.2, 10.9 | n/a | n/a | 0 |+| `?renderer=webgpu` | WebGPU | 12.8, 17.0 | 0.01 / 255 | 1.0 | 0 |+| `?renderer=webgpu`, GPU denied | WebGL2 (fallback) | 9.4, 10.1 | 0 / 255 | 1.0 | 0 |++The rim effect shifts the frame by 1.18 / 255 on both paths and the two paths agree on it to 0.02 / 255.diff --git a/prompts/finish/_context/best3d-PROGRESS.md b/prompts/finish/_context/best3d-PROGRESS.mdindex 1bd2df205..19f442bd8 100644--- a/prompts/finish/_context/best3d-PROGRESS.md+++ b/prompts/finish/_context/best3d-PROGRESS.md@@ -149,3 +149,5 @@ Append a dated entry per closed order, with commit SHAs.- 2026-10-23: order 081 (splat parity) re-derived and PARTIALLY closed; the order file stays because only the live TripoSplat GPU step remains. Step 0 found Spark 2.3.1 with the LoD tree (`createLodSplats`, auto above 50k splats), the 25 camera hotspot cap, saved camera views, the camera timeline and in-browser video export already shipped (bd61fbd99, 3bd57493d, b72233fb8). Spark's `paged` option is deliberately not wired: it only streams `-lod-N` chunk files over HTTP and has no path for an in-memory buffer (documented in `src/splat-stage.js`). Gap found by driving the real Splat Viewer in Playwright with a real public .splat capture: video export returned an empty 110-byte WebM because the recorder sampled the canvas on a timer. Fixed (one `requestFrame` per rendered frame, two-frame tail flush, empty-output guard), with `tests/splat-hotspots.test.js` (11 tests: sampler, 25 cap, export flow). Re-run after the fix produced a 6.7 KB WebM that Chromium decodes (650x446). Caveat: the host was at load 60 to 120 on 32 cores with software GL, so only a 3000-splat prefix of the real capture was exported; the full 281k-splat file loaded and hotspots worked but rendered about 1 fps there.- Verified: `tests/splat-viewer-page.test.js`, `tests/splat-hotspots.test.js`, `tests/api/forge-splat.test.js` pass; `workers/model-triposplat/test_splat_prep.py` 6 passed; `workers/model-triposplat` README, STRUCTURE.md fleet row and changelog entry were already present (fulfils the triposplat README part of order 404); `check:rules` exit 0. `npm run build:pages` currently fails on another agent's changelog entry missing a tag, not this change.- Still open, owner: deploy the TripoSplat worker, set MODEL_TRIPOSPLAT_URL, run one real image through `/api/forge-splat` (row in production-100-OWNER-ACTIONS.md), then delete the order file.+- 2026-10-24: order 074 (WebGPU/TSL path) PARTIALLY closed; the order file stays. Shipped: three bumped to r186.1 and three-mesh-bvh 0.9.16 (ea835fe18, vendored the Forge text font r186 no longer ships); `src/renderer-factory.js` (WebGPU only on `?renderer=webgpu` or a `renderer` attribute AND a working adapter, else the shipped WebGL2 renderer); Restyle rim effect on TSL (`src/restyle-effects.js`) with classic materials swapped back for export (f0634f5a5). Measured with `scripts/renderer-bench.mjs` on a software SwiftShader adapter: WebGPU vs WebGL2 frame difference 0.01/255, silhouette overlap 1.0, 0 console errors, forced `navigator.gpu` removal falls back to WebGL2 with 0 errors; numbers in `docs/webgpu-renderer.md`. Headless WebGPU cannot present to a canvas, so parity is measured through offscreen readback.+ - Still open: the main viewer and `<agent-3d>` `renderer` attribute are not switched because `CinematicPipeline` (EffectComposer) is WebGL-only and needs a TSL post-processing port first. Native `GaussianSplat` is WebGPU-only so Spark stays until the viewer is on WebGPU. `three-mesh-bvh/webgpu` does not apply to the CPU raycast picking. Hardware-GPU timings need one run of the bench on a GPU machine.