Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

docs(forge): document the Modly lanes in the engine grid, the worker runbook and the photo-to-3D chain

nirholas68573bec8680
diff --git a/.env.example b/.env.example
index fb54b32ab..f1b8cd498 100644
--- a/.env.example
+++ b/.env.example
@@ -1456,6 +1456,10 @@ MODEL_TRELLIS_URL=
# TRELLIS.2 worker (workers/model-trellis2), the default image-to-3D lane (forge backend
# `trellis2`). Unset skips the lane; the chain behind it serves the request.
MODEL_TRELLIS2_URL=
+# Modly worker (workers/modly), Modly's backend run headless on an L4. Serves the forge
+# backends `modly` (TripoSG) and `modly_hunyuan` (Hunyuan3D 2 Mini Turbo), both
+# untextured image-to-3D. Unset skips both lanes.
+MODEL_MODLY_URL=
# workers/model-hunyuan3d → self-hosted Hunyuan3D-2 image→3D
# (forge backend `hunyuan3d`, gcp provider mode `hunyuan`). NOT the same as
# GCP_RECONSTRUCTION_URL, whose avatar face pipeline rejects non-face images.
diff --git a/docs/forge-pipeline.md b/docs/forge-pipeline.md
index 4d1d5a0b7..0ce92e5cc 100644
--- a/docs/forge-pipeline.md
+++ b/docs/forge-pipeline.md
@@ -98,6 +98,8 @@ Self-hosted TRELLIS additionally maps tiers to sampler and texture budgets via `
| `trellis2` | Microsoft TRELLIS.2 (MIT, PBR with alpha) | Our Cloud Run L4 worker (`workers/model-trellis2/`) | image (native single-hop image to 3D) | free (our GPU) | `MODEL_TRELLIS2_URL` + `GCP_RECONSTRUCTION_KEY` |
| `trellis_selfhost` | Microsoft TRELLIS | Our Cloud Run L4 worker (`workers/model-trellis/`) | image (native single-hop image to 3D) | free (our GPU) | `MODEL_TRELLIS_URL` + `GCP_RECONSTRUCTION_KEY` |
| `hunyuan3d` | Tencent Hunyuan3D-2.1 | Our Cloud Run L4 worker (`workers/model-hunyuan3d/`) | image | free (our GPU) | `GCP_HUNYUAN3D_URL` + key |
+| `modly` | VAST AI TripoSG via Modly (MIT, untextured geometry) | Our Cloud Run L4 worker (`workers/modly/`) | image (single photo) | free (our GPU) | `MODEL_MODLY_URL` + `GCP_RECONSTRUCTION_KEY` |
+| `modly_hunyuan` | Tencent Hunyuan3D 2 Mini Turbo via Modly (untextured geometry) | Same worker as `modly` | image (single photo) | free (our GPU) | `MODEL_MODLY_URL` + `GCP_RECONSTRUCTION_KEY` |
| `triposg` | VAST AI TripoSG-scribble | Our Cloud Run L4 worker (`workers/model-triposg/`) | sketch only | free (our GPU) | `GCP_TRIPOSG_URL` + key |
| `trellis` | Microsoft TRELLIS | Replicate | image | paid platform, last resort | `REPLICATE_API_TOKEN` |
| `meshy` | Meshy 6 | vendor API | geometry, image | BYOK, credits | user key |
@@ -106,6 +108,8 @@ Self-hosted TRELLIS additionally maps tiers to sampler and texture budgets via `
| `stability` | Stable Fast 3D | vendor API | image, synchronous | BYOK | user key |
| `replicate_byok` | Replicate models | caller's Replicate account | image | BYOK | user key |
+The two Modly lanes run [Modly](https://github.com/lightningpixel/modly)'s backend headless on one worker (`workers/modly/README.md`): TripoSG and Hunyuan3D 2 Mini Turbo through Modly's own extensions, then a mesh repair pass and a decimate to the tier's face budget (`target_polycount`). They return geometry without textures, so they rank below every textured free lane and are meant as an explicit pick, or for retexture and stylize afterwards.
+
Two lanes deserve a warning. `hunyuan3d` must point at the general Hunyuan3D worker URL, never at the avatar-pipeline controller: the controller fronts a face pipeline and rejects general object images. And `nvidia` is text-only; every routing list that might carry a user photo filters it out.
## 4. Routing: free-first, subject-aware, health-tempered
@@ -115,7 +119,7 @@ Routing answers one question: given `(path, tier, userImages, prompt)`, which co
**Static preference order.** `freeLaneCandidates()` builds the ordered list both resolvers walk:
1. The tier's named free default (`FREE_DEFAULT_FOR_TIERS`): every image tier (draft, standard, high) names `trellis2`, the only default lane that serves every region. Hunyuan3D is no longer a named default; it is an explicit lane and a fallback outside the restricted territories (see Territory enforcement below). The unconfigured case matters: until `MODEL_TRELLIS2_URL` is set the lane is skipped and the chain behind it serves the request. The workers are our own GPUs; NVIDIA NIM is no longer a named default (its text-only, no-reference-image conditioning is the realism hole this table closes) but stays explicitly selectable and remains the final fallthrough in the per-path chain.
-2. The per-path free fallback chain (`FREE_FALLBACK_FOR_PATH`). For `image` that is `trellis2`, `trellis_selfhost`, `hunyuan3d`, `huggingface`, `nvidia`: our own GPU workers first (zero vendor cost), then external free lanes, with the text-only NIM lane trailing as a safety net.
+2. The per-path free fallback chain (`FREE_FALLBACK_FOR_PATH`). For `image` that is `trellis2`, `trellis_selfhost`, `hunyuan3d`, `huggingface`, `modly`, `modly_hunyuan`, `nvidia`: our own textured GPU workers first (zero vendor cost), then external free lanes, then the untextured Modly lanes, with the text-only NIM lane trailing as a safety net.
3. The paid platform default (`DEFAULT_BACKEND_FOR_PATH`: image → `trellis` on Replicate, geometry → `meshy`, sketch → `triposg`) only when nothing free is configured and healthy.
**Subject-aware reorder (high tier).** `classifyForgeSubject()` regex-classifies the prompt as hard-surface or organic. Hard-surface prompts (vehicles, machines, architecture) hoist `trellis_selfhost` above `hunyuan3d`; organic subjects (people, creatures) keep Hunyuan3D first. This exists because the two engines genuinely differ on those subject classes at high fidelity.
@@ -152,11 +156,11 @@ Async jobs get an opaque HMAC-signed token (`f1.<payload>.<sig>`, `api/_lib/forg
Every non-terminal poll also carries honest timing: `elapsed_seconds` (the age of the *job*, from `created_at`, not of the tab watching it), `eta_seconds` (the lane's typical total) and `eta_remaining_seconds`. The remaining figure is floored at 5, because the estimate is a typical duration rather than a deadline and a countdown that goes negative reads as "stuck" on a job that is merely slower than average. A `queued` poll on a scale-to-zero self-host lane additionally carries `cold_start` and `cold_start_seconds` (that lane's spin-up budget). `laneColdStart()` is module-scope for exactly this reason: it used to be a closure inside the submit handler, so only the submit response could say a worker was booting, and a client that never saw it (a coalesced job, a page resumed after a reload, a failover successor) was told "queued" with no explanation for a wait that is a container boot rather than a stall. `forgeStageNarration()` and `coldStartState()` in [src/shared/forge-frames.js](../src/shared/forge-frames.js) turn those fields into the same copy on every surface, off the payload and never off a local clock, and the flag clears on the first real `running` poll rather than on a timer. The in-flight job id is also written to `localStorage`, so an interrupted browser session resumes polling for up to 30 minutes; background completion persists the result to the user's gallery either way (see [Background generation](./forge-background-generation.md)).
-**Territory enforcement** (`api/_lib/forge-territory.js`). The Hunyuan3D license excludes the European Union, the United Kingdom and South Korea for the model and its output. The Forge resolves the requester's country on the server from the edge geo header (`clientCountry`), never from the request body, and never routes a restricted request to a Tencent-model lane. Those lanes are `hunyuan3d` and `huggingface` (the Spaces chain runs Hunyuan3D-2 and 2.1). An unknown or malformed country is treated as restricted, so a request with no geo signal fails closed to the MIT and non-Tencent lanes. The filter sits in every place a lane is chosen: auto routing and its health-aware variant, the HuggingFace failure hop, poll-time redispatch, the `retry_backends` suggestions, and the prompt-to-avatar plan. An explicit `backend=hunyuan3d` or `backend=huggingface` from a restricted country answers `403 region_restricted` and lists the open lanes in `retry_backends`. The public catalog is CDN-cached and country-agnostic, so it advertises the fail-closed defaults.
+**Territory enforcement** (`api/_lib/forge-territory.js`). The Hunyuan3D license excludes the European Union, the United Kingdom and South Korea for the model and its output. The Forge resolves the requester's country on the server from the edge geo header (`clientCountry`), never from the request body, and never routes a restricted request to a Tencent-model lane. Those lanes are `hunyuan3d`, `huggingface` (the Spaces chain runs Hunyuan3D-2 and 2.1) and `modly_hunyuan` (Hunyuan3D 2 Mini Turbo, under the Hunyuan3D 2.0 licence with the same exclusion). An unknown or malformed country is treated as restricted, so a request with no geo signal fails closed to the MIT and non-Tencent lanes. The filter sits in every place a lane is chosen: auto routing and its health-aware variant, the HuggingFace failure hop, poll-time redispatch, the `retry_backends` suggestions, and the prompt-to-avatar plan. An explicit `backend=hunyuan3d`, `backend=huggingface` or `backend=modly_hunyuan` from a restricted country answers `403 region_restricted` and lists the open lanes in `retry_backends`. The public catalog is CDN-cached and country-agnostic, so it advertises the fail-closed defaults.
**Resolution option.** `resolution` (512, 1024 or 1536) is a request option honoured by the `trellis2` lane and part of the dedup and cache key, so two resolutions of one prompt never share a result. It is rejected with a clear error when invalid, and a worker that cannot hold the requested size steps down and reports `resolution_served`.
-**Poll-time failover** (`api/_lib/forge-failover.js`) closes the nastier gap: a self-host worker can accept a task and fail it minutes later, in a poll handled by a different instance that no longer holds the request. On a failed poll the module recovers the original inputs (prompt plus reference image) from the creation row, resubmits to the next configured lane, and binds old handle to new handle in Redis (2 hour TTL). The client keeps polling the same job id and simply sees `status: running` with a new `backend`. Hops are capped by `MAX_FAILOVER_HOPS = 3` (one primary plus up to three backups). The redispatch order is `trellis2`, `trellis_selfhost`, `hunyuan3d`, `trellis`, then `triposg` as the final rung (Tencent lanes are skipped for a restricted or unknown territory). TripoSG runs its photo pipeline (gcp provider mode `triposg`) and returns untextured geometry, so it only serves a job once every textured lane has failed or is marked down; the client's lane note says the result will have no textures. It was added after the 2026-09-18 to 09-28 outage, when Hunyuan3D and TRELLIS self-host failed together, Replicate was unconfigured, and roughly 400 photo jobs a day ended in a hard failure while the TripoSG worker was healthy. TripoSG stays out of the `retry_backends` suggestions because its public lane is sketch-only. HuggingFace is excluded because its provider blocks through the whole generation (cannot be ridden from a poll), and NVIDIA is excluded because poll-time redispatch always reconstructs from a stored reference image, which NIM rejects. Both failover paths (this attended one and the unattended finalizer sweep) link the failed attempt to its successor row (`forge_creations.superseded_by`, migration `20260814200000_forge_failover_supersede.sql`), so the outcome ledger, the forge health sensor and `npm run forge:errors` count a recovered attempt as a recovery, not a loss.
+**Poll-time failover** (`api/_lib/forge-failover.js`) closes the nastier gap: a self-host worker can accept a task and fail it minutes later, in a poll handled by a different instance that no longer holds the request. On a failed poll the module recovers the original inputs (prompt plus reference image) from the creation row, resubmits to the next configured lane, and binds old handle to new handle in Redis (2 hour TTL). The client keeps polling the same job id and simply sees `status: running` with a new `backend`. Hops are capped by `MAX_FAILOVER_HOPS = 3` (one primary plus up to three backups). The redispatch order is `trellis2`, `trellis_selfhost`, `hunyuan3d`, `trellis`, `modly`, `modly_hunyuan`, then `triposg` as the final rung (Tencent lanes are skipped for a restricted or unknown territory). TripoSG runs its photo pipeline (gcp provider mode `triposg`) and returns untextured geometry, so it only serves a job once every textured lane has failed or is marked down; the client's lane note says the result will have no textures. It was added after the 2026-09-18 to 09-28 outage, when Hunyuan3D and TRELLIS self-host failed together, Replicate was unconfigured, and roughly 400 photo jobs a day ended in a hard failure while the TripoSG worker was healthy. TripoSG stays out of the `retry_backends` suggestions because its public lane is sketch-only; the two Modly lanes are image lanes and close that list. A Modly lane that rejects the submit itself (cold, restarting, throttled) is cooled and the same request retries on `trellis2`, then `trellis_selfhost`, inside the submit. HuggingFace is excluded because its provider blocks through the whole generation (cannot be ridden from a poll), and NVIDIA is excluded because poll-time redispatch always reconstructs from a stored reference image, which NIM rejects. Both failover paths (this attended one and the unattended finalizer sweep) link the failed attempt to its successor row (`forge_creations.superseded_by`, migration `20260814200000_forge_failover_supersede.sql`), so the outcome ledger, the forge health sensor and `npm run forge:errors` count a recovered attempt as a recovery, not a loss.
**Designed terminal failure.** When no successor is possible, the failed response carries `retryable: true` plus `retry_backends: [...]`, the ordered configured lanes a client can one-click resubmit to. A failure is never a bare string. The whole module is fail-open: no Redis disables automatic redispatch but keeps suggestions; no DB disables recovery but never turns a clean failure into a hang.
@@ -229,6 +233,7 @@ Workers are Python FastAPI services on Cloud Run (scale-to-zero L4 GPUs unless n
| Worker | Role |
| --- | --- |
| `workers/model-trellis2/` | Self-hosted TRELLIS.2, the default `trellis2` lane. `resolution` 512, 1024 or 1536, PBR GLB with alpha, meshopt packed. |
+| `workers/modly/` | Modly's backend headless, the `modly` (TripoSG) and `modly_hunyuan` (Hunyuan3D 2 Mini Turbo) lanes. Untextured GLB, repaired and decimated to the tier's face budget. |
| `workers/model-trellis/` | Self-hosted TRELLIS, the `trellis_selfhost` lane. Quality-tiered up to 4096px textures. |
| `workers/model-hunyuan3d/` | Hunyuan3D-2.1 single image to textured mesh, the `hunyuan3d` lane. |
| `workers/model-triposg/` | TripoSG 1.5B. `mode: image` for the avatar pool, `mode: scribble` for the sketch path. |
diff --git a/docs/image-to-3d.md b/docs/image-to-3d.md
index c1f14f839..e266a470f 100644
--- a/docs/image-to-3d.md
+++ b/docs/image-to-3d.md
@@ -20,6 +20,7 @@ Routing is free-first and health-aware, exactly as in Forge. Because the free NV
2. **TRELLIS (self-host)**, our own Cloud Run GPU worker, a native single-hop image-to-3D lane (image to TRELLIS to GLB), zero vendor cost.
3. **Hunyuan3D (self-host)**, our own high-poly reconstruction worker, strong on people and organic subjects. Its licence excludes the EU, the UK and South Korea, so requests from those countries (or an unknown country) skip it.
4. **Hunyuan3D / TRELLIS (free)** on Hugging Face Spaces, with automatic failover across Hunyuan3D 2.1, Hunyuan3D 2, TRELLIS, and TripoSR.
+5. **Modly TripoSG** and **Modly Hunyuan3D Mini Turbo (self-host)**, our Modly worker (`workers/modly/`). Both return clean, repaired geometry at the tier's face budget but no textures, so they sit below every textured lane. Pick one explicitly when you want shape only, for example to retexture or stylize afterwards. Mini Turbo follows the same territory rule as Hunyuan3D; Modly TripoSG serves every region. If the Modly worker is down when you submit, the same photo goes to TRELLIS.2, then TRELLIS.
If a self-host worker accepts your job and then fails it, the job moves to the next lane on its own and you keep watching the same progress screen. The last rung is **TripoSG (self-host)**: it only takes over when every textured lane is down, and it returns the shape without textures (the progress screen tells you when that happens), so an outage costs you color rather than the whole model.
diff --git a/docs/ops/forge-storage-failover.md b/docs/ops/forge-storage-failover.md
index e001e5018..e9e389dcc 100644
--- a/docs/ops/forge-storage-failover.md
+++ b/docs/ops/forge-storage-failover.md
@@ -57,7 +57,8 @@ because those blame the object and the fallback would be wrong.
Only ours. `backendAcceptsInlineViews` in
[api/_lib/forge-tiers.js](../../api/_lib/forge-tiers.js) is true exactly for the
-backends whose `provider` is `gcp` (`trellis_selfhost`, `hunyuan3d`, `triposg`),
+backends whose `provider` is `gcp` (`trellis2`, `trellis_selfhost`, `hunyuan3d`,
+`modly`, `modly_hunyuan`, `triposg`),
and it is derived from the registry rather than a hand-written list, so a backend
added later is classified by what it is.
diff --git a/docs/ops/gcp-model-workers.md b/docs/ops/gcp-model-workers.md
index 5d74a7836..a027fa34f 100644
--- a/docs/ops/gcp-model-workers.md
+++ b/docs/ops/gcp-model-workers.md
@@ -24,6 +24,8 @@ result_* }` — and authenticate with one shared bearer secret.
| `trellis2` | `workers/model-trellis2` | `model-trellis2` | `image` | Default image→3D lane. TRELLIS.2 (MIT): PBR GLB with alpha, `resolution` 512/1024/1536, meshopt packed. Weights staged by `workers/model-trellis2/stage_weights.sh`. |
| `trellis_selfhost` | `workers/model-trellis` | `model-trellis` | `image` | Native single-hop image→3D (Microsoft TRELLIS). Accepts user photos and the FLUX-synthesized view for text prompts. Textured GLB. |
| `hunyuan3d` | `workers/model-hunyuan3d`| `model-hunyuan3d`| `image` | High-poly image-conditioned reconstruction (Tencent Hunyuan3D). Poly-budget aware. |
+| `modly` | `workers/modly` | `model-modly` | `image` | TripoSG through Modly's headless backend (MIT). Single photo to untextured GLB, repaired and decimated to the tier's face budget. Weights staged by `workers/modly/stage_weights.sh`. |
+| `modly_hunyuan` | `workers/modly` | `model-modly` | `image` | Hunyuan3D 2 Mini Turbo on the same worker. Untextured. A Tencent lane: never routed to a restricted or unknown territory (`api/_lib/forge-territory.js`). |
| `triposg` | `workers/model-triposg` | `model-triposg` | `sketch` | Sketch→3D (TripoSG-scribble): a drawing + a prompt naming it → untextured geometry. |
All three are `provider: 'gcp'`, `free: true`. `model-trellis` pins one
@@ -89,6 +91,7 @@ faked.
| `MODEL_TRELLIS2_URL` | `trellis2` | Cloud Run URL of `model-trellis2`. Unset means the lane is skipped. |
| `MODEL_TRELLIS_URL` | `trellis_selfhost` | Cloud Run URL of `model-trellis`. |
| `GCP_HUNYUAN3D_URL` | `hunyuan3d` | Cloud Run URL of `model-hunyuan3d`. **Not** `GCP_RECONSTRUCTION_URL` — that is the avatar face pipeline, which rejects non-face images. |
+| `MODEL_MODLY_URL` | `modly`, `modly_hunyuan` | Cloud Run URL of `model-modly`. Unset means both Modly lanes are skipped. |
| `GCP_TRIPOSG_URL` | `sketch`, `triposg` | Cloud Run URL of `model-triposg`. Serves the sketch lane and the last-resort, untextured photo rung of the poll-time failover. |
| `GCP_UNIRIG_URL` | auto-rig (`rerig`) | Cloud Run URL of `model-rig` (the env name predates the engine swap that retired the old `unirig` service). Required for rigging: it is the only `rerig` backend, so without it the provider reports the mode unsupported rather than submitting anywhere. The provider speaks the worker's native schema (`mesh_gcs_url` in, `rigged_gcs_url` out) when this is set. |
| `GCP_RECONSTRUCTION_KEY`| all of the above | Shared bearer secret every worker checks (`avatar-reconstruction-key` in Secret Manager; `model-rig`'s `API_KEY` references the same secret). |