Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

docs(structure): list the Modly GPU worker in the self-hosted 3D fleet row

nirholas3e06d8e24b0b
diff --git a/STRUCTURE.md b/STRUCTURE.md
index fd838384d..c4cf94e2a 100644
--- a/STRUCTURE.md
+++ b/STRUCTURE.md
@@ -162,7 +162,7 @@ bottom.
| Agent-native 3D + embeddable distribution | MCP tool [api/_mcp/tools/agents.js](api/_mcp/tools/agents.js) (`attach_avatar_to_agent`); shared oEmbed provider [api/agent-oembed.js](api/agent-oembed.js) `→ /api/oembed` + builder [api/_lib/embed.js](api/_lib/embed.js) (`forgeEmbedTarget`, shared by `get_embed_code` in [api/_mcp/tools/embed.js](api/_mcp/tools/embed.js)); Forge share page oEmbed discovery [api/forge-share.js](api/forge-share.js) `→ /forge/share/:id`; distribution snippet builders [src/forge-embed-snippets.js](src/forge-embed-snippets.js) + panel [src/forge-embed-panel.js](src/forge-embed-panel.js); autonomous demo [examples/agent-native-3d/](examples/agent-native-3d) | In-repo | Closes the loop: create → own → discover/remix → **use everywhere**. `attach_avatar_to_agent` gives a registered agent identity a persistent body (chains into `register_agent` for on-chain identity + `anchor_provenance` for a signed authenticity credential). Any Forge creation now unfurls via real oEmbed (paste `/forge/share/:id` into Notion/Discord/Slack) with the same `type:rich` iframe payload `get_embed_code` returns via MCP. The Forge "Embed this model" panel hands out five real distribution flavours from one GLB — iframe, `<model-viewer>`, `<agent-3d>`, a `@three-ws/page-agent` talking guide, and a `@three-ws/walk` companion — all built from one shared, pure snippet module so every surface stays byte-identical. `examples/agent-native-3d/run.mjs` drives the full agent-native chain live against `/api/mcp-studio` (mesh_forge → rig_mesh → create_agent_persona → persona_say → get_agent_persona → every distribution snippet) with zero mocks and a documented, logged fallback if the shared free-tier generation lane is rate-limited at run time. Spec: [specs/EMBED_SPEC.md](specs/EMBED_SPEC.md#share-routes-on-chain-agents). Docs: [docs/mcp.md#attach_avatar_to_agent](docs/mcp.md#attach_avatar_to_agent). Roadmap prompt 10 (closes the roadmap campaign). |
| Animation pipeline | [public/animations/](public/animations) + [scripts/build-animations.mjs](scripts/build-animations.mjs) | In-repo | Mixamo source FBX + GLB retargeting pipeline · see [docs/3d-asset-pipeline.md](docs/3d-asset-pipeline.md) |
| Procedural animation (runtime IK layers) | [src/procedural/](src/procedural) — analytic two-bone IK [two-bone-ik.js](src/procedural/two-bone-ik.js), gaze [look-at.js](src/procedural/look-at.js), terrain foot planting [foot-plant.js](src/procedural/foot-plant.js), drift-free re-application [pose-baseline.js](src/procedural/pose-baseline.js) | Live | Post-mixer layers that do what no baked clip can: the chest/neck/head chain turns toward a world point, and each foot lands on its own patch of uneven ground. Rig-agnostic via `canonicalNodeMapFromObject`; an unsupported rig reports `enabled === false` and no-ops. Wired into the corner companion ([walk-sdk/src/companion.js](walk-sdk/src/companion.js) `_updateGaze`), the walkaround world ([src/walk.js](src/walk.js) `rebuildFootPlant`), and agent avatars ([src/agent-avatar.js](src/agent-avatar.js) `_applyLookTarget` — which is what finally makes `setLookTarget()` and the `LOOK_AT` action move the rig). Ambient idle motion (breathing/blink/saccade/weight-shift) stays in [src/idle-animation.js](src/idle-animation.js). Docs: [docs/procedural-animation.md](docs/procedural-animation.md). Tests: [tests/procedural-anim.test.js](tests/procedural-anim.test.js), [tests/agent-avatar-look-target.test.js](tests/agent-avatar-look-target.test.js) |
-| Self-hosted 3D GPU fleet (the engines behind the Forge) | [workers/model-trellis2/](workers/model-trellis2) (TRELLIS.2, MIT, the default image-to-3D lane `trellis2`: PBR GLB with alpha, `resolution` 512/1024/1536; built and tested, deploy owner-gated), territory router [api/_lib/forge-territory.js](api/_lib/forge-territory.js) (EU, UK and KR never reach a Tencent-model lane; unknown country fails closed), [workers/model-trellis/](workers/model-trellis), [workers/model-hunyuan3d/](workers/model-hunyuan3d) (2.0 + 2.1 PBR + 2.1 RTX), [workers/model-triposg/](workers/model-triposg), [workers/model-triposr/](workers/model-triposr), [workers/model-text2motion/](workers/model-text2motion), [workers/rig/](workers/rig) (auto-rigging, deployed as `model-rig`), [workers/avatar-reconstruction/](workers/avatar-reconstruction), CPU stages [workers/rembg/](workers/rembg) · [workers/remesh/](workers/remesh) · [workers/segment/](workers/segment) · [workers/stylize/](workers/stylize), talking-avatar video [workers/longcat/](workers/longcat) (LongCat-Video-Avatar-1.5, gates the `/create/video` lane via `LONGCAT_WORKER_URL`; the one fleet member that is not a Cloud Run service, because its ~45 GB resident weight set needs an 80 GB GPU, so it is built and tested but undeployed), streaming video→3D [workers/model-video2scene/](workers/model-video2scene), selfie-pipeline CPU orchestrator [workers/avatar-pipeline-controller/](workers/avatar-pipeline-controller) | Live · Cloud Run GPU (us-central1, plus us-east4 where trellis/hunyuan/triposr hold standbys and `model-text2motion` runs as its production lane) | Every generation engine the Forge dispatches to, each with its own `cloudbuild.yaml`. The L4 quota is **per region** and both us-central1 and us-east4 are granted 3, so a warm pin in one region only starves lanes in that same region (increase to 16 filed 2026-07-16, still reconciling); the Hunyuan3D-2.1 PBR realism lane also ships an RTX PRO 6000 build (`model-hunyuan3d-21-rtx`, quota granted 1000) as the warm primary. `model-text2motion` (MDM, MIT) is the text→motion lane behind `GCP_TEXT2MOTION_URL`, the `text_to_animation` MCP tool, and `POST /api/forge-motion`: prompt in, a retargetable three.js `AnimationClip` out, applied with the same engine (`src/animation-retarget.js`) as the curated animation library. It is the one GPU service whose production URL points at us-east4, and its `cloudbuild.yaml` deploys both regions from one image so the standby never lags. Ops map: [docs/ops/gcp-credits-plan.md](docs/ops/gcp-credits-plan.md) |
+| Self-hosted 3D GPU fleet (the engines behind the Forge) | [workers/model-trellis2/](workers/model-trellis2) (TRELLIS.2, MIT, the default image-to-3D lane `trellis2`: PBR GLB with alpha, `resolution` 512/1024/1536; built and tested, deploy owner-gated), territory router [api/_lib/forge-territory.js](api/_lib/forge-territory.js) (EU, UK and KR never reach a Tencent-model lane; unknown country fails closed), [workers/model-trellis/](workers/model-trellis), [workers/model-hunyuan3d/](workers/model-hunyuan3d) (2.0 + 2.1 PBR + 2.1 RTX), [workers/model-triposg/](workers/model-triposg), [workers/modly/](workers/modly) (Modly v0.4.3 running its TripoSG and Hunyuan3D 2 Mini Turbo extensions as the untextured Forge lanes `modly` and `modly_hunyuan`, behind `MODEL_MODLY_URL`; weights staged, built and tested, deploy owner-gated), [workers/model-triposr/](workers/model-triposr), [workers/model-text2motion/](workers/model-text2motion), [workers/rig/](workers/rig) (auto-rigging, deployed as `model-rig`), [workers/avatar-reconstruction/](workers/avatar-reconstruction), CPU stages [workers/rembg/](workers/rembg) · [workers/remesh/](workers/remesh) · [workers/segment/](workers/segment) · [workers/stylize/](workers/stylize), talking-avatar video [workers/longcat/](workers/longcat) (LongCat-Video-Avatar-1.5, gates the `/create/video` lane via `LONGCAT_WORKER_URL`; the one fleet member that is not a Cloud Run service, because its ~45 GB resident weight set needs an 80 GB GPU, so it is built and tested but undeployed), streaming video→3D [workers/model-video2scene/](workers/model-video2scene), selfie-pipeline CPU orchestrator [workers/avatar-pipeline-controller/](workers/avatar-pipeline-controller) | Live · Cloud Run GPU (us-central1, plus us-east4 where trellis/hunyuan/triposr hold standbys and `model-text2motion` runs as its production lane) | Every generation engine the Forge dispatches to, each with its own `cloudbuild.yaml`. The L4 quota is **per region** and both us-central1 and us-east4 are granted 3, so a warm pin in one region only starves lanes in that same region (increase to 16 filed 2026-07-16, still reconciling); the Hunyuan3D-2.1 PBR realism lane also ships an RTX PRO 6000 build (`model-hunyuan3d-21-rtx`, quota granted 1000) as the warm primary. `model-text2motion` (MDM, MIT) is the text→motion lane behind `GCP_TEXT2MOTION_URL`, the `text_to_animation` MCP tool, and `POST /api/forge-motion`: prompt in, a retargetable three.js `AnimationClip` out, applied with the same engine (`src/animation-retarget.js`) as the curated animation library. It is the one GPU service whose production URL points at us-east4, and its `cloudbuild.yaml` deploys both regions from one image so the standby never lags. Ops map: [docs/ops/gcp-credits-plan.md](docs/ops/gcp-credits-plan.md) |
| Self-hosted Granite Guardian (governance lane behind watsonx) | [workers/granite-guardian/](workers/granite-guardian/README.md) (Cloud Run `granite-guardian`, europe-west4, vLLM on one L4), client lane in [api/_lib/granite-guardian.js](api/_lib/granite-guardian.js) | Live · Cloud Run GPU (europe-west4) | IBM's open-weight `ibm-granite/granite-guardian-3.3-8b` served by the stock vLLM OpenAI image, so `/api/guardian/assess`, the identity integrity check, and the Granite Twin/Oracle/Attest/IRL screens return real Granite Guardian verdicts on a deployment with no IBM Cloud account. watsonx.ai stays the lead lane when configured; every verdict names the `model` and `provider` that produced it. Placed in europe-west4 because that region's L4 grant is otherwise unused, so its warm pin never starves the 3D lanes. Tests: [tests/api-guardian.test.js](tests/api-guardian.test.js), [tests/api/identity-integrity.test.js](tests/api/identity-integrity.test.js) |
| Reconstruct completion backstop (abandoned selfie jobs) | [api/cron/reconstruct-sweep.js](api/cron/reconstruct-sweep.js), every 5 min via `vercel.json` crons → Cloud Scheduler | Live 2026-07-25 | A reconstruct job used to be advanced **only by the user's browser**: `/create/selfie` polls `regenerate-status`, and each poll runs the finalize stages. Close the tab and the worker still finished the GLB and stored it durably, but nothing ever collected it — 26 jobs were stranded that way, oldest from 2026-05-31, invisible to `db-retention` (which prunes only terminal rows). This sweep is the server-side driver: it picks up reconstruct jobs quiet >3 min and runs **the same shared stages** in [reconstruct-finalize.js](api/_lib/reconstruct-finalize.js), so an abandoned job still lands in the user's library. 30-day rescue window (the worker's GCS blobs + Firestore job state are durable, so a weeks-old job is still deliverable); past that, or when a job's provider is no longer pollable from platform credentials, rows fail out with an honest reason so the open set can't grow unbounded. BYOK (meshy/tripo) jobs stay browser-driven — a cron has no user context to resolve their keys. Sibling of [auto-rig-sweep.js](api/cron/auto-rig-sweep.js), which plays the same role for the rerig lane. Docs: [docs/avatar-reconstruction.md](docs/avatar-reconstruction.md#who-finishes-the-job) |
| Avatar fidelity evaluation (selfie → avatar, scored) | [workers/avatar-reconstruction/eval/](workers/avatar-reconstruction/eval) | In-repo · gates every reconstruction change | **Identity Shape Error (ISE)** — does the reconstructed avatar have *this person's* face shape? Texture-blind, pose- and scale-invariant, dimensionless: MediaPipe's 468 landmarks and the avatar's corresponding head vertices are both Umeyama-aligned onto the neutral canonical face over a stable identity subset (no eyes/lips, so a smile is not scored as identity), then compared. A pipeline cannot score better by resizing or tilting the head — only by matching shape, a property pinned in [eval/test_identity_eval.py](workers/avatar-reconstruction/eval/test_identity_eval.py) alongside monotonicity. [`run_eval.py`](workers/avatar-reconstruction/eval/run_eval.py) scores a whole reference set against a live worker and diffs two revisions, surfacing per-face regressions a mean would hide; [`tune_morph.py`](workers/avatar-reconstruction/eval/tune_morph.py) sweeps the morph parameters against ISE offline; [`robustness.py`](workers/avatar-reconstruction/eval/robustness.py) measures the opposing axis (does a blurred / turned / occluded / dim / low-res photo of the same person still yield the same head?), and the shipped `max_displacement_frac` is the knee of those two curves. The 40-face reference set is **synthesised** by [`make_refs.py`](workers/avatar-reconstruction/eval/make_refs.py) over a deterministic grid of the platform's diversity matrix — no real biometrics, no licence terms, rebuildable by any agent — gitignored and mirrored to `gs://three-ws-avatar-reconstructions/eval-refs/`. Docs: [eval/README.md](workers/avatar-reconstruction/eval/README.md), program plan [docs/avatar-fidelity-program.md](docs/avatar-fidelity-program.md) |