Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
model-triposr: fast image to 3D mesh (TripoSR)
The lightest of the mesh backends. TripoSR (VAST-AI / Stability AI, MIT) reconstructs a mesh from a single image in about 3 seconds of GPU work on a warm instance.
It bakes vertex colors from one input view rather than producing PBR materials, so fidelity trails the heavier backends model-triposg and model-trellis. Its job is to be the fast path and the fallback for when those backends are cold, saturated, or failing.
Measured against the deployed service on 2026-08-11, one 194 KB PNG in:
| Step | Time |
|---|---|
Scale-from-zero cold start until /health answers |
about 95 s |
/infer end to end, warm instance |
3.2 s, a 755 KB GLB (18,865 vertices, COLOR_0 baked) |
min-instances=0, so a request that arrives with no instance resident pays the
cold start on top of the inference. Callers that care about that keep the lane
warm (see keep-warm below) or dispatch to it as a fallback only.
Role in the pipeline
model-triposr is one of the mesh backends the
avatar-pipeline-controller can dispatch to. The
controller registers it from the MODEL_TRIPOSR_URL env var, weight-selects a
backend per job (MODEL_WEIGHTS), POSTs the input images to /infer, then polls
/tasks/:id for the resulting GLB before handing it to the rigging stage. The
worker itself is a single FastAPI process (main.py) that loads TripoSR onto the
GPU at startup and serves the task API below.
Today that controller is not deployed: there is no avatar-pipeline-controller
Cloud Run service, and the platform's GCP_RECONSTRUCTION_URL points at the
avatar-reconstruction worker instead. So nothing on
three.ws routes user jobs here right now; this service is reached directly at its
own URL with the shared bearer key, which is how the numbers above were measured.
Deploying the controller with MODEL_TRIPOSR_URL set is what re-opens the lane.
HTTP API
All routes require Authorization: Bearer $API_KEY except /health. Work is
asynchronous: /infer returns immediately with a task_id, and you poll
/tasks/:id until status is done or failed.
POST /infer → 202
curl -X POST "$SERVICE_URL/infer" \
-H "Authorization: Bearer $API_KEY" \
-H 'Content-Type: application/json' \
-d '{"images":["https://three.ws/avatars/thumbs/default.png"],"job_id":"abc123"}'
# → { "task_id": "…", "status": "queued", "model": "triposr" }
Request body:
| Field | Type | Notes |
|---|---|---|
images |
string[] (1–6) | https:// URLs or data:image/… URIs. Only the first is used. Remote URLs pass through the SSRF guard in worker_security.py (https-only; private/loopback/link-local/metadata IPs rejected after DNS; redirects re-validated per hop). |
body_type |
string | Optional, default "neutral". |
job_id |
string | Optional caller correlation id. |
GET /tasks/{task_id}
{
"task_id": "…",
"status": "done",
"model": "triposr",
"result_gcs_url": "https://storage.googleapis.com/three-ws-avatar-reconstructions/raw-meshes/triposr/<task_id>.glb",
"elapsed_ms": 9120
}
status moves queued → running → done, or → failed with a sanitized error
string. On success result_gcs_url points at the uploaded GLB
(raw-meshes/triposr/<task_id>.glb in the output bucket). Tasks are held in
process memory, so an id is only resolvable for the lifetime of the instance that
created it. Unknown ids return 404.
GET /health
{ "ok": true, "model": "triposr", "gpu_available": true, "gpu_name": "NVIDIA L4", "model_loaded": true }
Unauthenticated — this is the endpoint the controller's liveness probe hits.
Environment
| Var | Required | Default | Notes |
|---|---|---|---|
API_KEY |
yes | — | Shared bearer secret. In production from Secret Manager (avatar-reconstruction-key). |
GCS_BUCKET |
yes | — | Output bucket for meshes (three-ws-avatar-reconstructions). |
WEIGHTS_DIR |
no | /weights/triposr |
TripoSR weights. In production this is the mounted three-ws-model-weights GCS volume. |
MAX_CONCURRENT |
no | 2 |
Parallel inferences; the model is small enough to run two on one L4. |
HF_HOME |
production | (unset) | /weights/hf-cache. TripoSR's image tokenizer is facebook/dino-vitb16, which transformers otherwise pulls from huggingface.co while the model loads. Set with HF_HUB_OFFLINE below. |
HF_HUB_OFFLINE |
production | (unset) | 1. On 2026-07-28 the live pull was rate-limited on the shared Cloud Run egress IP and the revision crash looped; the files are staged in the weights bucket, so the worker reads them offline. |
U2NET_HOME |
no | /opt/u2net |
Where rembg looks for its matting model. Baked into the image at build time; unset it only if you have staged the model elsewhere. |
cloudbuild.yaml passes --set-env-vars, which replaces the whole env set
rather than merging. Every var this worker needs therefore has to be in that one
flag, HF_HOME and HF_HUB_OFFLINE included. To change a single var on the
running service without a rebuild, use gcloud run services update model-triposr --region us-central1 --update-env-vars KEY=value.
How it ships
Built by Cloud Build from cloudbuild.yaml and deployed as
the Cloud Run service model-triposr in us-central1: 1× nvidia-l4
GPU, 4 vCPU, 16 GiB, 900 s request timeout, min-instances=0, max-instances=2.
The three-ws-model-weights bucket is mounted at /weights as a Cloud Storage
volume; API_KEY comes from the avatar-reconstruction-key secret. GPU-backed
Cloud Run needs L4 quota in the region, so see the model-worker ops runbook,
docs/ops/gcp-model-workers.md.
A second deployment of the same image runs in us-east4 as a standby
(https://model-triposr-93741856042.us-east4.run.app). cloudbuild.yaml does
not manage it: it was deployed by hand from the us-central1 image, so a rebuild
updates us-central1 only and the standby has to be pointed at the new tag
explicitly with gcloud run deploy model-triposr --region us-east4 --image ....
Submit from the repo root — the build step declares dir: workers/model-triposr,
so the upload source is the whole repo:
gcloud builds submit --config workers/model-triposr/cloudbuild.yaml .
Or via the fleet helper (stages weights, then builds + deploys and writes the
controller's MODEL_TRIPOSR_URL):
HF_TOKEN=hf_xxx SERVICES="triposr" workers/deploy/stage-weights.sh
PROJECT_ID=<gcp-project> SERVICES="triposr" workers/deploy/deploy-all.sh
The container clones TripoSR at build time (it has no setup.py/pyproject, so
it is put on PYTHONPATH rather than pip-installed) and compiles its
torchmcubes CUDA extension for the L4 (TORCH_CUDA_ARCH_LIST=8.9+PTX), which is
why the build uses a CUDA devel base and a 3600 s Cloud Build timeout.
Run locally
Requires a CUDA GPU and local weights. TripoSR is cloned into the image and
compiles the torchmcubes CUDA extension at build time, so the reproducible path
is the image:
cd workers/model-triposr
docker build -t model-triposr .
docker run --rm --gpus all -p 8080:8080 \
-e API_KEY=dev-secret \
-e GCS_BUCKET=your-dev-bucket \
-e WEIGHTS_DIR=/weights/triposr \
-e GOOGLE_APPLICATION_CREDENTIALS=/gcp/sa.json \
-v /path/to/triposr:/weights/triposr:ro \
-v /path/to/sa.json:/gcp/sa.json:ro \
model-triposr
With the TripoSR environment already on PYTHONPATH:
# fetch weights once (see the header of main.py):
# huggingface-cli download stabilityai/TripoSR --local-dir /path/to/triposr
API_KEY=dev-secret GCS_BUCKET=your-dev-bucket WEIGHTS_DIR=/path/to/triposr \
uvicorn main:app --host 0.0.0.0 --port 8080
Tests
The pre-inference path (decode, matte to 3-channel RGB, output naming) and the auth + SSRF guards run without a GPU, a model, or network egress, so they are a plain unit suite. It needs only Pillow and httpx:
python3 workers/model-triposr/test_image_prep.py
# → 36 checks passed
The Dockerfile runs the same file as a build gate, so a regression in the matte step (which broke every job in 2026-07 by handing TripoSR a 4-channel tensor) fails the build instead of the next user's generation.
Keep-warm: this lane stays cold on purpose
Both deployments run cold (min-instances=0) and that is deliberate. L4 quota is
granted per region (3 concurrent instances in each of us-central1 and
us-east4), so anything held warm here is taken from a lane that has traffic.
The allowlist in api/cron/gpu-keepwarm.js
spells out the same rule for us-central1: the model-trellis and model-rig
floors already hold 2 of the 3, so pinning a third would cap the workhorse lane
at its floor. Two leftovers that broke that rule were undone on 2026-08-11:
- The
model-triposr-warmCloud Scheduler job pingedus-central1/healthevery 10 minutes in peak hours. With no instance resident, each ping booted an L4 for a 95 s cold start and let it be reclaimed minutes later: about 144 GPU boots a day for a lane that had served no job in 30 days. The job is now paused (gcloud scheduler jobs resume model-triposr-warm --location us-central1brings it back if the lane ever carries traffic). - The
us-east4standby ran atmin-instances=1, holding an L4 there around the clock. Withmodel-trellisalso pinned at 1, that left a single slot formodel-text2motion, which is the region's actual production lane. The standby is nowmin-instances=0.
Re-pin either one only together with a quota increase, and say which lane is giving up the slot.
Example: submit, poll, fetch
BASE=https://model-triposr-xxxxxxxx-uc.a.run.app
KEY=your-api-key
TASK=$(curl -s -X POST "$BASE/infer" \
-H "Authorization: Bearer $KEY" \
-H 'Content-Type: application/json' \
-d '{"images":["https://three.ws/avatars/thumbs/default.png"]}' \
| python3 -c 'import sys,json; print(json.load(sys.stdin)["task_id"])')
while :; do
STATE=$(curl -s "$BASE/tasks/$TASK" -H "Authorization: Bearer $KEY")
echo "$STATE"
echo "$STATE" | grep -q '"status": *"done"' && break
echo "$STATE" | grep -q '"status": *"failed"' && exit 1
sleep 3
done
curl -s "$BASE/tasks/$TASK" -H "Authorization: Bearer $KEY" \
| python3 -c 'import sys,json; print(json.load(sys.stdin)["result_gcs_url"])'
How three.ws calls it
TripoSR has no direct /forge lane of its own. It is dispatched through the
avatar-pipeline-controller, which registers it
from MODEL_TRIPOSR_URL (see workers/avatar-pipeline-controller/main.py)
and weight-selects it as a mesh backend per job. The controller is reached from
the platform via GCP_RECONSTRUCTION_URL in
api/_providers/gcp.js. This worker shares the
platform-side bearer secret GCP_RECONSTRUCTION_KEY, which must equal its API_KEY.
That is the wiring, not the current state: as noted under
Role in the pipeline, the controller is not deployed and
GCP_RECONSTRUCTION_URL currently resolves to the avatar-reconstruction
service, so no platform request reaches this worker today. Anything calling it
now does so directly with the same bearer key.