Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
model-hunyuan3d — single image → textured 3D mesh (Hunyuan3D)
FastAPI inference workers that turn one image into a textured GLB with
Tencent's Hunyuan3D. Two model generations run side by side as two separate
Cloud Run services, because their Python stacks are mutually incompatible
(torch 2.3 / cu121 + hy3dgen for 2.0 vs torch 2.5 / cu124 +
hy3dshape / hy3dpaint for 2.1):
| Service | Model | Entry point | Dockerfile | Output material |
|---|---|---|---|---|
model-hunyuan3d |
Hunyuan3D-2.0 | main.py |
Dockerfile |
single baked diffuse texture |
model-hunyuan3d-21 |
Hunyuan3D-2.1 | app21.py |
Dockerfile.hunyuan21 |
PBR: baseColor + metallicRoughness + normal |
model-hunyuan3d-21-rtx |
Hunyuan3D-2.1 | app21.py |
Dockerfile.hunyuan21rtx |
same PBR set, on RTX PRO 6000 (Blackwell) |
2.1 is the realism lane. It runs the hy3dshape shape DiT, then the
hy3dpaint PBR paint pass (multiview PBR diffusion → RealESRGAN super-res →
texture bake → inpaint), and exports a GLB with a true physically-based material
set. PBR maps are what make a render read as a real photographed object under
real lighting instead of a flat plastic toy — the single biggest realism lever
in the platform's own GPU fleet. 2.0 remains deployed and untouched as the
instant fallback lane: to roll back, repoint GCP_HUNYUAN3D_URL at the
model-hunyuan3d service URL — no rebuild.
All three services speak the same wire contract, so any URL drops straight
into the platform's GCP_HUNYUAN3D_URL env.
Which one is live right now: model-hunyuan3d-21-rtx. That is what
GCP_HUNYUAN3D_URL on the three-ws-api service points at, and it is the only
one of the three that runs warm (min-instances=1); the two L4 services scale
to zero and take a multi-minute weight load on their first request. Confirm
before you assume:
gcloud run services describe three-ws-api --region us-central1 \
--flatten="spec.template.spec.containers[].env[]" \
--format="value(spec.template.spec.containers.env.name, spec.template.spec.containers.env.value)" \
| grep HUNYUAN
# GCP_HUNYUAN3D_URL https://model-hunyuan3d-21-rtx-93741856042.us-central1.run.app
Why an RTX build exists. Two hard walls, found live on 2026-07-17:
- The L4 2.1 service cannot finish loading at all. app21 stages ~18 GiB
of weights into
/tmp, which on Cloud Run is memory-backed, then loads the ~14 GiB model on top; the L4 tier's 32 Gi memory ceiling SIGKILLs the instance every time (Container terminated on signal 9mid-load). The port binds and/healthanswers, but no 2.1 job can complete on an L4 with the current staging design. - L4 quota. All L4 services in us-central1 share one quota of 3 GPUs and the fleet is permanently at that ceiling; the RTX PRO 6000 quota in the same region is granted at 1000.
Dockerfile.hunyuan21rtx is the same app21.py on a CUDA 12.8 / torch 2.7.1
(cu128) stack, because Blackwell is compute capability 12.0 and the cu124
wheels in the L4 image ship no sm_120 kernels. Its extensions compile for both
8.9 and 12.0, so the image also boots on an L4 (though wall 1 above still
applies there). Deploy with
cloudbuild.hunyuan21rtx.yaml (BuildKit,
inline layer cache); platform minimums for the GPU type are 20 CPU / 80 Gi,
which also clears wall 1 with room to spare. The RTX service runs warm (min 1,
max 1: the quota preference reads granted 1000, but live deploy enforcement
allows exactly 1 RTX GPU in us-central1) as the 2.1 primary. Making the L4 build genuinely loadable (stage one
weight subtree at a time and delete each after its from_pretrained) is the
open follow-up if an L4 fallback for 2.1 is ever needed; until then the 2.0
lane is the fallback.
License
Hunyuan3D-2.1 is released under the TENCENT HUNYUAN NON-COMMERCIAL LICENSE
AGREEMENT (the full text ships in the weights bucket at
gs://three-ws-model-weights/hunyuan3d-2.1/LICENSE and in the cloned repo). This
worker performs self-hosted inference on our own GPU infrastructure, exactly
as the 2.0 lane already runs in production. The RealESRGAN super-res checkpoint
and DINOv2 reference encoder carry their own upstream licenses (BSD-3 and Apache
2.0 respectively). No weights or model code are redistributed by the platform;
only generated GLBs are returned to callers.
Territory rule (EU, UK and South Korea are excluded)
The agreement covers the worldwide territory excluding the European Union, the United Kingdom and South Korea (Section 1.l), and Section 5.c forbids using the model, or its output, outside that territory. The platform enforces this on the server, not in the client:
- The requester's country comes from the edge geo header (
clientCountryinapi/_lib/client-geo.js), never from a request body field. api/_lib/forge-territory.jsholds the restricted set (the 27 EU member states, GB and KR) andlaneAllowedInTerritory. An unknown or malformed country is treated as restricted (fail closed).- Both Tencent-model lanes are gated:
hunyuan3d(this worker) andhuggingface(a Spaces chain that runs Hunyuan3D-2 and 2.1). Auto routing, poll-time failover, the failover suggestions and the prompt-to-avatar plan all filter through it, and an explicitbackend=hunyuan3dorbackend=huggingfacerequest from a restricted country answers403 region_restrictedwith the lanes that remain open. - TRELLIS.2 (MIT,
workers/model-trellis2/) is the default image-to-3D lane everywhere, and the only one of the two families that serves the restricted regions.
The Notice file the agreement requires is in NOTICE.
Endpoints
POST /infer, POST /reconstruct, GET /tasks/{id}, GET /jobs/{id} require
Authorization: Bearer $API_KEY. GET / and GET /health are unauthenticated.
POST /infer → 202
{
"images": ["https://three.ws/avatars/thumbs/default.png"],
"tier": "high",
"job_id": "abc123"
}
images— 1 to 6 entries, each adata:image/…;base64,…URI or anhttps://URL. Only the first is used.httpssources go through the SSRF guard inworker_security.py(https-only; private, loopback, link-local, and cloud-metadata IPs rejected on every redirect hop).tier—draft|standard|high(defaulthigh). Selects the generation budget (see below).job_id— optional correlation string.
Response: { "task_id": "3f2c…", "status": "queued" }.
POST /reconstruct accepts the same body and additionally echoes job_id, for
the avatar controller's /reconstruct + /jobs/:id wire shape.
GET /tasks/{task_id}
status is one of queued → running → done, or failed.
{
"task_id": "3f2c…",
"status": "done",
"model": "hunyuan3d-2.1",
"result_gcs_url": "https://storage.googleapis.com/three-ws-avatar-reconstructions/raw-meshes/hunyuan3d/3f2c….glb",
"elapsed_ms": 224140
}
Task state is persisted to a durable tasks/{id}.json GCS blob on every
transition, so a POST /infer and a later GET /tasks/:id that land on
different Cloud Run instances still resolve the same record. Failures carry a
sanitized error string; unknown ids return 404.
GET /health
{ "ok": true, "model": "hunyuan3d-2.1", "gpu_available": true,
"gpu_name": "NVIDIA RTX PRO 6000 Blackwell Server Edition",
"pipeline_loaded": true, "ready": true, "load_error": null }
The pipeline loads in the background after the HTTP port opens, so the
instance answers /health immediately; ready flips to true once the shape +
paint models finish loading, and load_error carries a sanitized string if the
load failed.
These three fields are a wire contract, not diagnostics. The platform reads
them in both directions: api/_lib/forge-health.js
for the status dashboard and api/_lib/forge-lane-health.js
on the generation hot path. A populated load_error marks the lane down
(routing skips it); ready: false marks it up but never warm (a submit queues
until the weights land, so the caller widens its ETA). Renaming or dropping one
silently turns a dead worker back into a green backend. Both probes hit
/health for exactly this reason; before 2026-08-11 they hit the service root
and read the 404 as healthy.
GET /
{ "service": "model-hunyuan3d-21", "model": "hunyuan3d-2.1", "health": "/health",
"endpoints": ["POST /infer", "POST /reconstruct", "GET /tasks/{id}", "GET /jobs/{id}", "GET /health"] }
Service identity for a bare-root GET. It serves no inference; it exists so a
probe or an operator landing on the root gets pointed at /health instead of a
404. (Two platform probes did exactly that, at one 404 per minute per service,
until the readiness fix above.)
Quality tiers (2.1)
Quality over speed — GPU time is cheap against the platform's GCP credit budget. The budget splits across the shape DiT (inference steps + marching-cubes octree resolution) and the PBR paint pass (multiview count + per-view diffusion resolution). The paint texture atlas is fixed high at load (render 2048, texture 4096).
| Tier | shape steps | octree | paint views | paint resolution |
|---|---|---|---|---|
draft |
30 | 256 | 6 | 512 |
standard |
50 | 384 | 6 | 512 |
high (default) |
50 | 512 | 6 | 768 |
high holds the multiview count at 6 so the paint pass fits the L4's 24 GiB
VRAM alongside the shape DiT, DINOv2, and RealESRGAN, while pushing the shape
octree and per-view diffusion resolution up for sharper geometry and PBR maps.
Environment (2.1)
| Var | Required | Default | Purpose |
|---|---|---|---|
API_KEY |
yes | — | Shared bearer secret (Secret Manager avatar-reconstruction-key) |
GCS_BUCKET |
yes | — | Output-mesh bucket (three-ws-avatar-reconstructions) |
HUNYUAN3D_MODEL |
no | 2.1 |
Pins the image's lane |
WEIGHTS_GCS_URI |
no | — | gs:// prefix of the 2.1 weight tree to stage locally |
WEIGHTS_LOCAL_DIR |
no | /tmp/hunyuan3d-2.1 |
Local staging dir for the 2.1 weights |
WEIGHTS_DIR |
no | /weights/hunyuan3d-2.1 |
FUSE-mount fallback for the 2.1 weights |
DINO_GCS_URI |
no | — | gs:// prefix of facebook/dinov2-giant to stage locally |
DINO_LOCAL_DIR |
no | /tmp/dinov2-giant |
Local staging dir for DINOv2 |
DINO_DIR |
no | /weights/dinov2-giant |
FUSE-mount fallback for DINOv2 |
MAX_CONCURRENT |
no | 1 |
In-flight inferences; one L4 fits one |
U2NET_HOME |
no | ~/.u2net |
Where rembg looks for u2net.onnx; the deploy configs point it at the staged copy on the FUSE mount (/weights/u2net) |
Weight staging. The 2.1 shape DiT is a single 6.9 GiB .ckpt and the paint
UNet a single 3.7 GiB .bin. Reading files that large over the Cloud Storage
FUSE mount stalls indefinitely (the failure the 2.0 lane already hit and fixed),
so the worker streams the needed subtrees to local disk with the storage client
(a plain sequential GET, which does not stall) and loads from there. Staging is
gated on WEIGHTS_GCS_URI / DINO_GCS_URI and degrades to the FUSE mount on any
failure — it never crashes the loader.
Stage the weights (one-time)
The three-ws-model-weights bucket already holds both trees. To (re)populate:
pip install "huggingface_hub[cli]"
# 2.1 shape + PBR paint + VAE weights (~14 GiB)
hf download tencent/Hunyuan3D-2.1 --local-dir /tmp/hunyuan3d-2.1
gcloud storage rsync --recursive /tmp/hunyuan3d-2.1 gs://three-ws-model-weights/hunyuan3d-2.1
# DINOv2 reference encoder for the paint pass (~4.4 GiB)
hf download facebook/dinov2-giant --local-dir /tmp/dinov2-giant
gcloud storage rsync --recursive /tmp/dinov2-giant gs://three-ws-model-weights/dinov2-giant
The RealESRGAN super-res checkpoint is fetched into the image at build time
(hy3dpaint/ckpt/RealESRGAN_x4plus.pth).
rembg's u2net.onnx is staged too. Left alone, rembg downloads the 176 MB
matting model from GitHub on every cold boot, and one broken download is fatal:
the pipeline load fails and the instance answers every job with load_error
until it is replaced (production, 2026-09-18, after a restart). The deploy
configs set U2NET_HOME=/weights/u2net, so rembg finds the staged copy and never
touches the network. To (re)stage it:
curl -sSL -o /tmp/u2net.onnx https://github.com/danielgatis/rembg/releases/download/v0.0.0/u2net.onnx
md5sum /tmp/u2net.onnx # 60024c5c889badc19c04ad937298a77b, the hash rembg verifies
gcloud storage cp /tmp/u2net.onnx gs://three-ws-model-weights/u2net/u2net.onnx
Deploy
Submit from the repo root (the build step declares
dir: workers/model-hunyuan3d, so the upload source is the whole repo). Pass
SHORT_SHA explicitly — gcloud builds submit does not auto-populate it:
# 2.1 on RTX PRO 6000 (model-hunyuan3d-21-rtx): THE PRODUCTION LANE
gcloud builds submit --config workers/model-hunyuan3d/cloudbuild.hunyuan21rtx.yaml \
--substitutions=SHORT_SHA=$(git rev-parse --short HEAD) .
# 2.1 on L4 (model-hunyuan3d-21): cannot finish a job today, see wall 1 above
gcloud builds submit --config workers/model-hunyuan3d/cloudbuild.hunyuan21.yaml \
--substitutions=SHORT_SHA=21-$(git rev-parse --short HEAD) .
# 2.0 (model-hunyuan3d) — legacy fallback lane
gcloud builds submit --config workers/model-hunyuan3d/cloudbuild.yaml \
--substitutions=SHORT_SHA=$(git rev-parse --short HEAD) .
Every submit deploys as well as builds, so all three are owner-gated production deploys, not build checks.
The RTX deploy takes the lane down for one model load. Cloud Run enforces
the RTX PRO 6000 quota at one GPU, and the serving revision holds it, so a
normal rolling update can never start the new revision's instance (it waits for
a GPU until the 4 minute startup probe gives up, and the revision is left not
Ready while traffic stays on the old one). cloudbuild.hunyuan21rtx.yaml
therefore releases the GPU first with service-level manual scaling at 0,
deploys, and restores autoscaling (--scaling=auto --max=1) whether or not the
deploy succeeded. Expect about ten minutes without this lane while the new
instance loads its weights; forge fails over to the other image lanes
meanwhile. A config-only change (an env var, a probe) needs the same three
steps by hand:
gcloud run services update model-hunyuan3d-21-rtx --region us-central1 --scaling=0
gcloud run services update model-hunyuan3d-21-rtx --region us-central1 --update-env-vars KEY=VALUE
gcloud run services update model-hunyuan3d-21-rtx --region us-central1 --scaling=auto --max=1
The full incident is in docs/ops/production-log-triage.md.
What each config asks Cloud Run for, all in us-central1, all with no GPU zonal
redundancy, a 900 s request timeout, and the three-ws-build@ /
avatar-reconstruction-sa@ service accounts pinned (the project's default
compute SA was deleted, so an unpinned submit fails outright):
| Config | GPU | CPU / memory | min / max | Build timeout |
|---|---|---|---|---|
cloudbuild.hunyuan21rtx.yaml |
1× nvidia-rtx-pro-6000 |
20 / 80 Gi | 1 / 1 | 7200 s |
cloudbuild.hunyuan21.yaml |
1× nvidia-l4 |
8 / 32 Gi | 0 / 1 | 3600 s |
cloudbuild.yaml |
1× nvidia-l4 |
8 / 32 Gi | 0 / 1 | 3600 s |
The fourth service: a us-east4 standby
A second deployment of the 2.0 image runs in us-east4 as a standby
(https://model-hunyuan3d-93741856042.us-east4.run.app), alongside the
equivalent standbys for trellis and triposr (see the GPU-fleet row in
STRUCTURE.md). The L4 quota is granted per region,
so a second region is extra capacity that costs nothing while idle.
cloudbuild.yaml does not manage it. It was deployed by hand from the
us-central1 image, so a rebuild updates us-central1 only and the standby
keeps serving the old tag until it is pointed at the new one explicitly:
gcloud run deploy model-hunyuan3d --region us-east4 \
--image us-central1-docker.pkg.dev/aerial-vehicle-466722-p5/model-hunyuan3d/server:<tag>
Nothing routes to it today (GCP_HUNYUAN3D_URL is the RTX service), so it is a
capacity reserve, not a lane. Scale-to-zero, max-instances=1, same env and
same avatar-reconstruction-sa@ runtime SA as the us-central1 2.0 service.
Known benign log entry. A retiring instance of this standby can log
malloc_consolidate(): unaligned fastbin chunk detected followed by Container terminated on signal 11. That fires during interpreter teardown, AFTER uvicorn
has logged a clean Application shutdown complete, when the native CUDA and
mesh libraries unload in an order glibc dislikes. No request is in flight and no
task is affected; it is an exit-path artifact of a container that is already
going away, not a crash of a serving instance.
Only the RTX lane is kept warm. The L4 GPU allocation in this region is scarce and shared with the other model workers, so neither L4 Hunyuan3D service pins an instance: the RTX service is the primary and the 2.0 service is a fallback that pays its weight load on the first request after an idle period. The CUDA compiles (custom_rasterizer plus the mesh-processor pybind extension, and on the RTX image both the 8.9 and 12.0 CUDA archs) are what push the build timeouts far past the 10-minute default.
Wire it into the platform
The platform points GCP_HUNYUAN3D_URL at whichever service should serve
the Hunyuan3D /forge tier. After verifying the 2.1 service end to end:
gcloud run services update three-ws-api --region us-central1 \
--update-env-vars GCP_HUNYUAN3D_URL=https://model-hunyuan3d-21-XXXXXXX-uc.a.run.app
Always --update-env-vars, never --set-env-vars — the latter wipes every
other env var on the shared three-ws-api service. To fall back to 2.0, repoint
the same var at the model-hunyuan3d service URL. The tier is declared in
api/_lib/forge-tiers.js
(requiresEnv: ['GCP_HUNYUAN3D_URL', 'GCP_RECONSTRUCTION_KEY']), routed by
api/forge.js, health-probed by
api/_lib/forge-health.js and
api/_lib/forge-lane-health.js (both
against /health, see the readiness contract above), and the
avatar-pipeline-controller also selects it as a
mesh backend via MODEL_HUNYUAN3D_URL. All workers share the platform-side
bearer secret GCP_RECONSTRUCTION_KEY, which must equal each service's
API_KEY.
Tests
test_worker_api.py covers the 2.1 worker's whole
caller-facing surface with no GPU, no weights, and no network. It can do that
because app21.py defers its import torch into the loader thread, so the HTTP
app imports on a plain CPU box:
pip install pytest fastapi httpx pillow pydantic google-cloud-storage
python3 workers/model-hunyuan3d/test_worker_api.py # standalone runner
python3 -m pytest -q workers/model-hunyuan3d/test_worker_api.py # 9 tests
It pins the /health readiness contract the platform probes depend on, the
/ identity route, bearer auth on every inference and task route, the tier
budget table above, the SSRF refusals in the image decoder, and the durable
tasks/{id}.json round trip that lets a submit and a later poll land on
different instances and still agree.
The GPU inference itself is not covered here (it needs the built CUDA image and
real weights); exercise it with the submit-poll-fetch loop below against a
deployed service. The 2.0 worker (main.py) imports torch at module scope, so
it is only importable inside its own image.
Example — submit, poll, fetch
BASE=https://model-hunyuan3d-21-XXXXXXX-uc.a.run.app
KEY=your-api-key
TASK=$(curl -s -X POST "$BASE/infer" \
-H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
-d '{"images":["https://three.ws/avatars/thumbs/default.png"],"tier":"high"}' \
| python3 -c 'import sys,json; print(json.load(sys.stdin)["task_id"])')
while :; do
STATE=$(curl -s "$BASE/tasks/$TASK" -H "Authorization: Bearer $KEY")
echo "$STATE"
echo "$STATE" | grep -q '"status": *"done"' && break
echo "$STATE" | grep -q '"status": *"failed"' && exit 1
sleep 10
done