Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
model-trellis — single image → textured 3D mesh (TRELLIS)
FastAPI inference service that turns one image into a textured GLB using
Microsoft TRELLIS (TRELLIS-image-large,
MIT license). TRELLIS represents shape and appearance as structured latents,
decodes them to both a Gaussian appearance field and a mesh, then fuses the two
into a single textured GLB (postprocessing_utils.to_glb). Quality is tiered
(draft → max); the default bakes a 4096px texture and keeps most geometry,
and the max tier pushes the samplers to their ceiling and mattes the subject
first. It runs on one NVIDIA L4 (24 GB) on Cloud Run and is the self-hosted
TRELLIS mesh backend for the /forge image→3D lane.
Work is asynchronous: POST /infer returns 202 with a task_id, and the
caller polls GET /tasks/{id} until the mesh is ready. With GCS_BUCKET set,
the finished GLB lands at gs://$GCS_BUCKET/raw-meshes/trellis/{task_id}.glb
and is served back as a storage.googleapis.com URL. Without GCS_BUCKET, the
standalone container writes tasks and results atomically under OUTPUT_DIR and
serves the mesh from GET /results/{task_id}.glb.
Endpoints
POST /infer, GET /tasks/{id}, and GET /results/{task_id}.glb require
Authorization: Bearer $API_KEY.
GET /health and GET / are unauthenticated: they carry no secrets, and the
platform reads them to decide whether this lane can take work. Routing
(api/_lib/forge-lane-health.js,
api/_lib/forge-health.js) reads /health,
because a worker whose weight load failed is reachable but unusable; the
keep-warm cron (api/cron/gpu-keepwarm.js)
pings the root. Cloud Run's own startup probe is a TCP check on port 8080, not
an HTTP one.
POST /infer → 202
Request:
{
"images": ["https://three.ws/avatars/thumbs/default.png"],
"body_type": "neutral",
"job_id": "abc123",
"tier": "max",
"matte": true,
"rembg_model": "u2net_human_seg",
"quality": { "texture_size": 4096 },
"seed": 42
}
images— 1 to 6 entries, each adata:image/…;base64,…URI or anhttps://URL. A single image reconstructs one asset; multiple entries are FUSED as turnaround views (front/side/back) of the same subject viarun_multi_image.httpssources are pulled through the SSRF guard inworker_security.py(https-only; private, loopback, link-local, and cloud-metadata IPs rejected on every redirect hop). Each fetch is retried up to 3 times with exponential backoff on a timeout, a connection failure, or a retryable upstream status (408/425/429/5xx); a 404 or 403 fails immediately because it will never succeed. When a source is genuinely unusable the task'serrornames the reason and the host instead of an opaque error ref, so the caller can fix the URL.body_type— optional, default"neutral"(accepted, not used by TRELLIS).job_id— optional correlation string.tier— optional named quality preset:draft|standard|high|max(see table below). Omitted keeps the historical default (equivalent tohigh). Aqualitydict still overrides individual fields on top of a tier.matte— optional. Run the subject through the siblingrembg-servicebefore reconstruction so the background stops bleeding into the mesh. Defaults on fortier: "max", off otherwise. RequiresREMBG_SERVICE_URLto be set; if unset or the call fails, TRELLIS's own internal background removal still runs, so matting only ever improves the result, never gates it.rembg_model— optional model for the pre-matte:isnet-general-use(default),u2net,u2net_human_seg(best for people/portraits),silueta.quality— optional per-field override of the resolved tier (ss_steps,slat_steps,ss_cfg,slat_cfg,simplify,texture_size), each clamped to a safe L4 envelope.seed— optional deterministic sampling seed (default42).
Quality tiers
| tier | ss/slat steps | ss/slat cfg | simplify (tris removed) | texture | matte default | use |
|---|---|---|---|---|---|---|
draft |
12 / 12 | 7.5 / 3.0 | 0.95 | 1024 | off | fast previews / latency lane |
standard |
25 / 25 | 7.5 / 3.0 | 0.90 | 2048 | off | balanced |
high (default) |
40 / 40 | 7.5 / 3.0 | 0.75 | 4096 | off | production quality bar |
max |
50 / 50 | 8.5 / 4.5 | 0.50 | 4096 | on | maximum realism (photo → 3D) |
simplify is the fraction of triangles removed by to_glb's decimation, so
a lower value keeps more geometry: max keeps ~all of it. Steps drive both
diffusion stages (structure + structured-latent) and cost scales roughly
linearly with them; max runs both at the sampler ceiling. Cost is not the
constraint here (the L4 runs one job at a time against the platform's GCP credit
budget) — realism is.
Image → 3D from a real user photo
The MAX tier is built for turning a real photo of an object or a person into a
textured 3D asset. For a clean reconstruction, matte the subject first: matte: true sends the image to the sibling rembg-service, which returns an RGBA
cutout that TRELLIS uses directly as the subject mask (its alpha channel is
respected, bypassing a second internal rembg pass). For people, pass
rembg_model: "u2net_human_seg". Send a single clear, front-facing photo; add
side/back views of the same subject to the images array to fill in geometry
the front view cannot see.
Response:
{ "task_id": "3f2c…", "status": "queued" }
GET /tasks/{task_id}
Returns the live task record. status is one of queued → running → done,
or failed.
{
"task_id": "3f2c…",
"status": "done",
"model": "trellis-large",
"result_gcs_url": "https://storage.googleapis.com/three-ws-avatar-reconstructions/raw-meshes/trellis/3f2c….glb",
"elapsed_ms": 41230
}
In standalone local mode, a completed task returns result_url and
result_path instead of result_gcs_url:
{
"task_id": "3f2c…",
"status": "done",
"result_url": "/results/3f2c….glb",
"result_path": "/output/results/3f2c….glb"
}
On failure the record carries an error string. A bad image source reports the
actual reason (image source cdn.example.com unreachable after 3 attempts (ReadTimeout); check the URL is publicly readable) because the fault is in the
request; anything internal is reduced to a correlation-id ref with the full
traceback kept in the container log. A queued or running record that makes
no progress for 30 minutes is expired to failed so a client cannot poll
forever behind a dead runner. Unknown ids return 404.
GET /health
{
"ok": true,
"model": "trellis-image-large",
"gpu_available": true,
"gpu_name": "NVIDIA L4",
"pipeline_loaded": true,
"ready": true,
"load_error": null,
"load_attempts": 1,
"tiers": ["draft", "standard", "high", "max"],
"default_quality": { "ss_steps": 40, "slat_steps": 40, "ss_cfg": 7.5, "slat_cfg": 3.0, "simplify": 0.75, "texture_size": 4096 },
"rembg_matte": true,
"output_backend": "gcs",
"output_dir": null
}
The ~3 GB pipeline loads in the background after the port opens, so a cold
instance reports ready: false briefly; /infer tasks submitted during that
window wait for ready (up to 600 s) rather than failing.
A dead instance says so. The load is retried (MODEL_LOAD_ATTEMPTS) before
its error is treated as terminal, because the failures that reach it are
transient upstream ones rather than code faults. While retries remain,
load_error stays null and jobs keep waiting on ready. Once the budget is
spent, load_error latches and this endpoint answers 503 with
"ok": false, /infer refuses new work with 503, and the Cloud Run
liveness probe recycles the container.
That behaviour is the fix for the 2026-09-02 outage: a single transient
HTTP Error 403: rate limit exceeded during the load latched forever, /health
went on answering ok: true, /infer went on accepting jobs it could never
run, and min-instances=1 kept the dead instance resident. The default free
image lane failed 70 of 219 generations over 12 hours while every status surface
read green. Callers rely on the 503 from /infer: it is what trips the lane
failover in api/forge.js, which reroutes the request to another backend
instead of letting it die at poll time.
GET /
Unauthenticated service descriptor. It exists so a root ping stops recording a 404 in this service's log: there were 257 of those in the 24 h to 2026-08-11 against two real error events, which is how a genuine failure goes unnoticed.
{
"service": "model-trellis",
"model": "trellis-image-large",
"ready": true,
"endpoints": ["POST /infer", "GET /tasks/{task_id}", "GET /health"]
}
Environment
| Var | Required | Default | Purpose |
|---|---|---|---|
API_KEY |
yes | — | Shared bearer secret (Secret Manager avatar-reconstruction-key) |
GCS_BUCKET |
no | - | Cloud Storage bucket for production task state and output meshes. When unset, local storage is used. |
OUTPUT_DIR |
no | /output |
Mounted task and result volume used when GCS_BUCKET is unset |
WEIGHTS_DIR |
no | /weights/trellis-large |
Local path to TRELLIS weights (a mounted GCS volume in prod) |
MAX_CONCURRENT |
no | 1 |
In-flight inferences; one L4 fits exactly one |
ATTN_BACKEND |
no | xformers |
TRELLIS attention backend, read at import time |
SPCONV_ALGO |
no | native |
Sparse-conv algorithm, read at import time |
REMBG_SERVICE_URL |
no | — | Sibling rembg-service base URL for the matte pre-step. Unset disables matting (TRELLIS still removes the background internally). |
REMBG_MODEL |
no | isnet-general-use |
Default rembg model for the pre-matte |
REMBG_TIMEOUT_S |
no | 90 |
Max seconds to wait on the rembg-service round-trip before falling back to the un-matted image |
WEIGHTS_GCS_URI |
no | - | gs:// weight tree staged to local disk at startup with the storage client. Set in production; see the note below |
WEIGHTS_LOCAL_DIR |
no | /tmp/trellis-weights |
Where that staging lands, and what the pipeline then loads from |
IMAGE_FETCH_TIMEOUT_S |
no | 30 |
Per-attempt timeout for fetching a caller-supplied https:// image (3 attempts) |
MODEL_LOAD_ATTEMPTS |
no | 4 |
Model-load attempts before load_error latches and the instance reports itself dead |
MODEL_LOAD_RETRY_BASE_S |
no | 15 |
First load-retry backoff, doubling per attempt |
MODEL_LOAD_RETRY_CAP_S |
no | 120 |
Ceiling on that backoff |
DINOV2_LOCAL_DIR |
no | /opt/dinov2 |
Image-baked dinov2 checkout used for TRELLIS's image conditioner, so the load never calls GitHub |
TORCH_HOME |
no | /opt/torch |
Set in the image; holds the pre-baked dinov2_vitl14_reg checkpoint so torch.hub finds it cached |
Weights are not baked into the image — the three-ws-model-weights bucket
is mounted at /weights, so refreshing weights needs no rebuild.
At startup the loader stages that tree to local disk (WEIGHTS_GCS_URI →
WEIGHTS_LOCAL_DIR) with the storage client and loads from there. The GCS FUSE
mount serves the model's random-access reads over the network, and a cold load
routinely stalled on it (stalled read-req cancelled, context deadline exceeded), turning a ~50 s load into 15+ minutes or a hard timeout; a plain
sequential GET per object does not. If WEIGHTS_GCS_URI is unset, or staging
fails for any reason, the loader falls back to reading WEIGHTS_DIR off the
mount, so this can only add reliability. Already-staged objects whose size
matches are reused, so a same-instance reload does not re-pull 3 GB.
Pre-populate the bucket once with:
pip install huggingface_hub
huggingface-cli download microsoft/TRELLIS-image-large --local-dir /tmp/trellis-large
gsutil -m cp -r /tmp/trellis-large gs://three-ws-model-weights/trellis-large/
Standalone and NGC build
The production image compiles CUDA extensions for L4 (sm_89) by default. A
portable catalog build can supply a wider architecture list without changing
the Dockerfile:
docker build \
--build-arg 'TORCH_CUDA_ARCH_LIST=8.0;8.6;8.9;9.0+PTX' \
-t three-ws/trellis-mesh-server:latest \
workers/model-trellis
Run it with mounted weights and output volumes:
docker run --gpus all -p 8080:8080 \
-e API_KEY=choose-a-secret \
-v /path/to/trellis-image-large:/weights/trellis-large:ro \
-v /path/to/output:/output \
three-ws/trellis-mesh-server:latest
The local backend persists task JSON under /output/tasks and generated GLBs
under /output/results. It does not require Google credentials or make the
output directory public; downloads retain the worker's bearer authentication.
Tests
The request policy (quality tiers, the clamps that stop a caller value from
crashing the texture bake, the image-fetch retry) lives in
request_policy.py, deliberately free of torch and CUDA
so it can be proven anywhere:
cd workers/model-trellis
python3 -m pip install pytest httpx
python3 -m pytest test_request_policy.py test_storage_backend.py -q
test_app_contract.py covers the served surface
instead, and needs the image's dependency set (torch, FastAPI, the cloned
TRELLIS tree), so it runs inside the container rather than on your machine:
docker run --rm model-trellis python3 test_app_contract.py
It asserts the TRELLIS import chain resolves (FlexiCubes submodule included, the
packaging bug that only ever surfaced minutes into a doomed revision), the auth
boundary rejects a wrong bearer, / and /health answer, images arrays
outside 1 to 6 entries are refused, and the SSRF guard rejects cleartext,
loopback, and metadata-server sources. No GPU, no weights, no credentials: the
ASGI lifespan is not started.
Both suites run inside docker build (see the RUN gate in the Dockerfile),
so a broken tier table, a bad dependency resolve, or a hole in the auth boundary
fails the build instead of a user's generation. The GPU-bound half of the
service is covered by the platform-side integration tests in
tests/api/forge-trellis-selfhost.test.js.
Run locally
Requires a CUDA GPU and the TRELLIS weights on disk. Dependencies are installed
in layered order by the Dockerfile (torch → basic → xformers/spconv/kaolin →
compiled CUDA extensions → server → TRELLIS source cloned to /app/TRELLIS);
requirements.txt is only a pointer. The reproducible way to run it is the image:
cd workers/model-trellis
docker build -t model-trellis .
docker run --rm --gpus all -p 8080:8080 \
-e API_KEY=dev-secret \
-e GCS_BUCKET=your-dev-bucket \
-e WEIGHTS_DIR=/weights/trellis-large \
-e GOOGLE_APPLICATION_CREDENTIALS=/gcp/sa.json \
-v /path/to/trellis-large:/weights/trellis-large:ro \
-v /path/to/sa.json:/gcp/sa.json:ro \
model-trellis
On a box that already has the TRELLIS environment on PYTHONPATH you can run the
server directly:
API_KEY=dev-secret GCS_BUCKET=your-dev-bucket WEIGHTS_DIR=/path/to/trellis-large \
uvicorn main:app --host 0.0.0.0 --port 8080
GET /health reports gpu_available: false on a CPU-only box and inference will
fail there.
Deploy
Submit from the repo root — the build step declares dir: workers/model-trellis,
so the upload source is the whole repo:
gcloud builds submit --config workers/model-trellis/cloudbuild.yaml . \
--region us-central1 --project aerial-vehicle-466722-p5 \
--substitutions=SHORT_SHA=manual$(date +%s)
SHORT_SHA is only populated automatically for trigger-driven builds; a manual
submit must pass it, because the config tags and deploys
.../server:$SHORT_SHA. The config also pins the three-ws-build@ service
account (the project's default compute SA was deleted).
Or provision it alongside the rest of the fleet (idempotent; prints the URLs to
set on the three-ws-api service env). Valid service ids are exactly
hunyuan3d trellis triposr triposg rig:
PROJECT_ID=<gcp-project> SERVICES="hunyuan3d trellis triposg rig" \
workers/deploy/deploy-all.sh
Builds the image (CUDA extension compiles push the build past an hour — the
config sets timeout: 3600s) and deploys Cloud Run service model-trellis
in us-central1: 1× nvidia-l4 GPU, 8 vCPU, 32 GiB, 900 s request timeout,
min-instances=1 (one instance stays warm because a cold start pays a
multi-minute weight load), max-instances=3. Each instance serializes its own
GPU at MAX_CONCURRENT=1, so the instance count is the concurrent-generation
count. The weights bucket (three-ws-model-weights) is mounted at /weights
and API_KEY comes from the avatar-reconstruction-key secret.
Example — submit, poll, fetch
BASE=https://model-trellis-xxxxxxxx-uc.a.run.app
KEY=your-api-key
# 1. Submit
TASK=$(curl -s -X POST "$BASE/infer" \
-H "Authorization: Bearer $KEY" \
-H 'Content-Type: application/json' \
-d '{"images":["https://three.ws/avatars/thumbs/default.png"]}' \
| python3 -c 'import sys,json; print(json.load(sys.stdin)["task_id"])')
# 2. Poll until done
while :; do
STATE=$(curl -s "$BASE/tasks/$TASK" -H "Authorization: Bearer $KEY")
echo "$STATE"
echo "$STATE" | grep -q '"status": *"done"' && break
echo "$STATE" | grep -q '"status": *"failed"' && exit 1
sleep 5
done
# 3. Read the result URL
curl -s "$BASE/tasks/$TASK" -H "Authorization: Bearer $KEY" \
| python3 -c 'import sys,json; print(json.load(sys.stdin)["result_gcs_url"])'
How three.ws calls it
The platform points MODEL_TRELLIS_URL at this service. It is wired in
api/_providers/gcp.js as the trellis mode
(native single-image reconstruction, standard /infer + /tasks/:id shape) and
surfaced as the self-hosted TRELLIS /forge tier in
api/_lib/forge-tiers.js (requiresEnv: ['MODEL_TRELLIS_URL', 'GCP_RECONSTRUCTION_KEY']). The
avatar-pipeline-controller also lists it as a
mesh backend via its own MODEL_TRELLIS_URL. All workers share the platform-side
bearer secret GCP_RECONSTRUCTION_KEY, which must equal this service's API_KEY.