Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

feat(rig): add non-humanoid rigging via UniRig + SkinTokens

nirholasac2eca062eae
diff --git a/workers/rig-nonhuman/Dockerfile b/workers/rig-nonhuman/Dockerfile
new file mode 100644
index 000000000..ffad0388a
--- /dev/null
+++ b/workers/rig-nonhuman/Dockerfile
@@ -0,0 +1,70 @@
+# Non-humanoid rigging: UniRig (skeleton) + SkinTokens (skinning), both
+# VAST-AI-Research, both MIT, LICENSE files read 2026-10-09. Neither ships a
+# pip-installable package; both are subprocess CLIs (see pipeline.py), so
+# this image vendors both checkouts at a pinned commit.
+FROM nvidia/cuda:12.8.1-cudnn-runtime-ubuntu22.04
+
+ENV PYTHONDONTWRITEBYTECODE=1 \
+ PYTHONUNBUFFERED=1 \
+ PIP_NO_CACHE_DIR=1 \
+ DEBIAN_FRONTEND=noninteractive \
+ HF_HUB_DISABLE_TELEMETRY=1 \
+ DO_NOT_TRACK=1
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+ python3.11 python3.11-dev python3.11-venv python3-pip \
+ git curl build-essential \
+ libgl1 libglib2.0-0 \
+ && rm -rf /var/lib/apt/lists/* \
+ && ln -sf /usr/bin/python3.11 /usr/bin/python3 \
+ && ln -sf /usr/bin/python3.11 /usr/bin/python \
+ && python3 -m pip install --upgrade pip setuptools wheel
+
+# torch first, as its own cached layer, matching SkinTokens' documented
+# install (the newer and more exacting of the two dependency sets; UniRig
+# only requires torch>=2.3.1, so this satisfies both).
+RUN pip install torch==2.7.0 torchvision==0.22.0 torchaudio==2.7.0 \
+ --index-url https://download.pytorch.org/whl/cu128
+
+# UniRig (SIGGRAPH '25), pinned to the main HEAD read on 2026-10-11
+# (https://github.com/VAST-AI-Research/UniRig/commit/6793c6640ff01c8fb389f3993434124bb43d2933).
+ARG UNIRIG_COMMIT=6793c6640ff01c8fb389f3993434124bb43d2933
+RUN git clone https://github.com/VAST-AI-Research/UniRig.git /app/unirig \
+ && git -C /app/unirig checkout ${UNIRIG_COMMIT} \
+ && rm -rf /app/unirig/.git
+RUN pip install -r /app/unirig/requirements.txt \
+ && pip install spconv-cu120 \
+ && pip install torch_scatter torch_cluster \
+ -f https://data.pyg.org/whl/torch-2.7.0+cu128.html --no-cache-dir \
+ && pip install numpy==1.26.4
+
+# SkinTokens, pinned to the main HEAD read on 2026-10-11
+# (https://github.com/VAST-AI-Research/SkinTokens/commit/273b691d35989d71cd17ff2895fdc735097b92d1).
+# Successor to UniRig; predicts skin weights only.
+ARG SKINTOKENS_COMMIT=273b691d35989d71cd17ff2895fdc735097b92d1
+RUN git clone https://github.com/VAST-AI-Research/SkinTokens.git /app/skintokens \
+ && git -C /app/skintokens checkout ${SKINTOKENS_COMMIT} \
+ && rm -rf /app/skintokens/.git
+RUN pip install -r /app/skintokens/requirements.txt \
+ && pip install flash-attn --no-build-isolation
+
+WORKDIR /app
+COPY requirements.txt /app/requirements.txt
+RUN pip install -r /app/requirements.txt
+
+COPY worker_security.py gltf_meshopt.py pipeline.py main.py entrypoint.sh /app/
+COPY test_gltf_meshopt.py test_pipeline.py /app/
+RUN chmod +x /app/entrypoint.sh
+
+# Build-time guard: the chaining/error-handling logic needs no GPU and no
+# checkout, so it runs here and the image does not ship if it broke.
+RUN python3 /app/test_pipeline.py && python3 /app/test_gltf_meshopt.py
+
+# Checkpoints are staged in GCS (stage-assets.sh) and mounted at /weights by
+# Cloud Run; entrypoint.sh copies them into the literal relative paths UniRig
+# and SkinTokens fall back to when their own Hugging Face auto-fetch has no
+# egress (see stage-assets.sh for the exact layout).
+
+EXPOSE 8080
+
+CMD ["/app/entrypoint.sh"]
diff --git a/workers/rig-nonhuman/README.md b/workers/rig-nonhuman/README.md
new file mode 100644
index 000000000..0d3e6bad0
--- /dev/null
+++ b/workers/rig-nonhuman/README.md
@@ -0,0 +1,135 @@
+# model-rig-nonhuman: auto-rigging worker for quadrupeds, creatures, and props
+
+`workers/rig`'s engine (Make-It-Animatable) is trained on bipedal humanoids:
+it predicts a fixed 52-bone Mixamo skeleton and will not produce a usable rig
+for a four-legged creature, a winged mount, or an inanimate prop. This worker
+covers everything that worker's mesh classifier calls non-humanoid, using a
+different pair of engines that predict an arbitrary joint hierarchy instead
+of fitting one fixed template.
+
+## How it works
+
+1. **Skeleton**: [UniRig](https://github.com/VAST-AI-Research/UniRig) (MIT,
+ SIGGRAPH '25) autoregressively predicts a topologically valid joint
+ hierarchy for the input mesh (`launch/inference/generate_skeleton.sh`),
+ then grafts it onto the original GLB (`launch/inference/merge.sh`). No
+ skin weights yet.
+2. **Skinning**: [SkinTokens](https://github.com/VAST-AI-Research/SkinTokens)
+ (MIT), UniRig's successor, predicts ONLY the skin weights for that
+ skeleton-merged mesh (`demo.py --use_skeleton --use_transfer`), preserving
+ the original texture and scale and writing the final rigged GLB directly.
+ The work order specifies SkinTokens for this step rather than UniRig's own
+ `generate_skin.sh`: it is the newer, better-performing model for exactly
+ this sub-task (98-133% skinning-accuracy improvement per its README).
+
+Both are subprocess CLIs, not importable Python packages; `pipeline.py`
+chains them exactly per their own documented command contracts (verified
+against their live READMEs 2026-10-09/2026-10-11, not assumed from memory).
+`main.py` validates the input mesh the same way `workers/rig` does
+(duplicated here, not imported: each worker's Docker build context is
+isolated and cannot reach `../rig`).
+
+## Reached two ways
+
+Directly, or forwarded by `workers/rig`: that worker's `mesh_classify.py`
+runs a cheap geometric heuristic (height/footprint ratio + bilateral mirror
+symmetry) on every input mesh before dispatching, and forwards non-humanoid
+meshes here when `RIG_NONHUMAN_URL` is configured on that service. Either
+way the caller (or the forwarding worker, polling on the original caller's
+behalf) sees the identical API shape, so nothing downstream needs to know
+which engine actually rigged a given mesh.
+
+## API
+
+```
+POST /rig { mesh_gcs_url, job_id? }
+ -> 202 { task_id, status: "queued" } (Bearer API_KEY)
+GET /tasks/:id -> { task_id, status, rigged_gcs_url?, error?, elapsed_ms? }
+GET /health -> { ok, model, gpu_available, gpu_name, queued }
+```
+
+Every task runs under a hard timeout (`TASK_TIMEOUT_S`, default 900s: three
+chained model passes need more headroom than the single-pass humanoid
+worker's 420s). Task state is in-memory, same invariant as `workers/rig`:
+run with `min-instances = max-instances` so pollers always reach the
+instance that owns the task.
+
+## Deploy
+
+One-time asset staging (UniRig's skeleton checkpoint, SkinTokens' two
+checkpoints, and the Qwen3-0.6B config/tokenizer SkinTokens was fine-tuned
+from, under `gs://three-ws-model-weights/rig-nonhuman/`):
+
+```bash
+bash workers/rig-nonhuman/stage-assets.sh
+```
+
+Build + deploy (Cloud Build pins the `three-ws-build@` service account):
+
+```bash
+gcloud builds submit --config workers/rig-nonhuman/cloudbuild.yaml \
+ --substitutions=SHORT_SHA=manual$(date +%s) .
+```
+
+Service: `model-rig-nonhuman`, us-central1, 1x L4
+(`--no-gpu-zonal-redundancy`), one warm instance against the per-region L4
+quota in `docs/ops/gcp-credits-plan.md`. This is an additional GPU instance
+alongside `model-rig`, not a replacement; budget for it in that quota plan
+before deploying.
+
+Then point `workers/rig` at it:
+
+```bash
+gcloud run services update model-rig --region us-central1 \
+ --update-env-vars RIG_NONHUMAN_URL=$(gcloud run services describe \
+ model-rig-nonhuman --region us-central1 --format='value(status.url)')
+```
+
+`RIG_NONHUMAN_URL` is unset by default, so until that command runs, 100% of
+`model-rig` traffic stays on the pre-existing humanoid path: this service
+deploying does not change behavior on its own.
+
+## Local tests (no GPU needed)
+
+```bash
+cd workers/rig-nonhuman
+python3 test_pipeline.py # command contracts + subprocess chaining/errors
+python3 test_gltf_meshopt.py # the meshopt decode every input GLB passes through
+```
+
+`test_pipeline.py` checks the three command builders against UniRig's and
+SkinTokens' real CLI contracts, then exercises `run_pipeline`'s chaining and
+error handling by substituting tiny real scripts at the exact relative paths
+the commands invoke (a nonzero exit, a reported success with no output file,
+and a failure at the merge stage are each asserted to stop the chain with a
+`PipelineStepError` naming the failed step), so the logic that will run in
+production is exercised without a GPU, torch, or either upstream checkout.
+
+## Licensing
+
+UniRig and SkinTokens are both MIT (LICENSE files read 2026-10-09); Qwen3-0.6B
+(Apache 2.0) supplies only its config/tokenizer, not weights, per upstream's
+own `download.py`. No research-only license terms in this rigging path.
+
+## Environment
+
+| Var | Default | Meaning |
+|---|---|---|
+| `API_KEY` | (secret) | shared bearer secret (`avatar-reconstruction-key`) |
+| `GCS_BUCKET` | (required) | output bucket for rigged GLBs |
+| `UNIRIG_DIR` | `/app/unirig` | UniRig checkout |
+| `SKINTOKENS_DIR` | `/app/skintokens` | SkinTokens checkout |
+| `MAX_CONCURRENT` | `1` | parallel rig jobs per instance |
+| `TASK_TIMEOUT_S` | `900` | per-task hard timeout |
+| `WEIGHTS_ROOT` | `/weights/rig-nonhuman` | staged assets, read by `entrypoint.sh` |
+
+## Failure modes
+
+| Reported `error` | Cause | Fix |
+|---|---|---|
+| `input is not a binary glTF...` | `mesh_gcs_url` returned something that is not a `.glb` | fix the URL; rejected before touching the GPU |
+| `input GLB contains no triangles...` | a point cloud or curves-only export | rig a surface mesh |
+| `unirig-skeleton failed (exit ...)` / `unirig-merge failed (...)` / `skintokens-skin failed (...)` | the named stage's subprocess exited nonzero | the error carries a stderr tail; grep service logs for the full capture |
+| `... reported success but wrote no output file` | the named stage crashed without a nonzero exit | same as above, treat as that stage's failure |
+| `rigging timed out after 900s` | `TASK_TIMEOUT_S` elapsed | retry, or raise the timeout for very dense/complex meshes |
+| `internal error (ref <id>)` | anything unexpected | grep the service logs for the correlation id |
diff --git a/workers/rig-nonhuman/cloudbuild.yaml b/workers/rig-nonhuman/cloudbuild.yaml
new file mode 100644
index 000000000..dd067df56
--- /dev/null
+++ b/workers/rig-nonhuman/cloudbuild.yaml
@@ -0,0 +1,81 @@
+# Build + deploy the non-humanoid rig worker (UniRig skeleton + SkinTokens
+# skinning).
+# Manual submit (SHORT_SHA is only auto-set by triggers):
+# gcloud builds submit --config workers/rig-nonhuman/cloudbuild.yaml \
+# --substitutions=SHORT_SHA=manual$(date +%s) .
+serviceAccount: projects/aerial-vehicle-466722-p5/serviceAccounts/[email protected]
+
+substitutions:
+ _REGION: us-central1
+ _RUN_SA: avatar-reconstruction-sa@aerial-vehicle-466722-p5.iam.gserviceaccount.com
+ _SERVICE: model-rig-nonhuman
+ _REPO: model-rig-nonhuman
+ _GCS_BUCKET: three-ws-avatar-reconstructions
+ _WEIGHTS_BUCKET: three-ws-model-weights
+ # Task state lives in this service's memory, same invariant as workers/rig:
+ # min MUST equal max or GET /tasks/:id can 404 against the wrong instance.
+ _MIN_INSTANCES: "1"
+ _MAX_INSTANCES: "1"
+
+steps:
+ - id: build
+ name: gcr.io/cloud-builders/docker
+ dir: workers/rig-nonhuman
+ args:
+ - build
+ - --tag
+ - ${_REGION}-docker.pkg.dev/$PROJECT_ID/${_REPO}/server:$SHORT_SHA
+ - --tag
+ - ${_REGION}-docker.pkg.dev/$PROJECT_ID/${_REPO}/server:latest
+ - --cache-from
+ - ${_REGION}-docker.pkg.dev/$PROJECT_ID/${_REPO}/server:latest
+ - .
+
+ - id: push
+ name: gcr.io/cloud-builders/docker
+ args:
+ - push
+ - --all-tags
+ - ${_REGION}-docker.pkg.dev/$PROJECT_ID/${_REPO}/server
+
+ - id: deploy
+ name: gcr.io/google.com/cloudsdktool/cloud-sdk:slim
+ entrypoint: gcloud
+ args:
+ - run
+ - deploy
+ - ${_SERVICE}
+ - --image=${_REGION}-docker.pkg.dev/$PROJECT_ID/${_REPO}/server:$SHORT_SHA
+ - --region=${_REGION}
+ - --service-account=${_RUN_SA}
+ - --platform=managed
+ - --port=8080
+ - --gpu=1
+ - --gpu-type=nvidia-l4
+ - --no-gpu-zonal-redundancy
+ - --no-cpu-throttling
+ - --cpu=4
+ - --memory=16Gi
+ - --min-instances=${_MIN_INSTANCES}
+ - --max-instances=${_MAX_INSTANCES}
+ - --timeout=900
+ # Three chained autoregressive model loads (UniRig skeleton,
+ # UniRig merge, SkinTokens TokenRig + Qwen3-0.6B backbone) take a few
+ # minutes cold; the default 4-minute startup window is too tight.
+ - --startup-probe=tcpSocket.port=8080,initialDelaySeconds=0,periodSeconds=60,timeoutSeconds=60,failureThreshold=15
+ - --cpu-boost
+ - --allow-unauthenticated
+ - --add-volume=name=weights,type=cloud-storage,bucket=${_WEIGHTS_BUCKET}
+ - --add-volume-mount=volume=weights,mount-path=/weights
+ - --set-env-vars=GCS_BUCKET=${_GCS_BUCKET},MAX_CONCURRENT=1,TASK_TIMEOUT_S=900
+ - --set-secrets=API_KEY=avatar-reconstruction-key:latest
+
+images:
+ - ${_REGION}-docker.pkg.dev/$PROJECT_ID/${_REPO}/server:$SHORT_SHA
+ - ${_REGION}-docker.pkg.dev/$PROJECT_ID/${_REPO}/server:latest
+
+options:
+ machineType: E2_HIGHCPU_32
+ logging: CLOUD_LOGGING_ONLY
+
+timeout: 3600s
diff --git a/workers/rig-nonhuman/entrypoint.sh b/workers/rig-nonhuman/entrypoint.sh
new file mode 100755
index 000000000..69aef75e7
--- /dev/null
+++ b/workers/rig-nonhuman/entrypoint.sh
@@ -0,0 +1,33 @@
+#!/bin/sh
+# Wire staged assets into the UniRig and SkinTokens checkouts, then serve.
+#
+# Both engines resolve their checkpoints by trying Hugging Face Hub first and
+# falling back to the literal relative path on failure (UniRig's
+# src/inference/download.py, run.py:240); placing the files at that literal
+# path means inference needs no Hugging Face egress at runtime. Checkpoints
+# are COPIED from the FUSE-mounted bucket to local disk rather than
+# symlinked, same reasoning as workers/rig: torch.load over gcsfuse does slow
+# random reads, and a sequential cp of a few GB is fast and deterministic.
+set -eu
+
+WEIGHTS_ROOT="${WEIGHTS_ROOT:-/weights/rig-nonhuman}"
+UNIRIG_DIR="${UNIRIG_DIR:-/app/unirig}"
+SKINTOKENS_DIR="${SKINTOKENS_DIR:-/app/skintokens}"
+
+for pair in "unirig/experiments:$UNIRIG_DIR/experiments" \
+ "skintokens/experiments:$SKINTOKENS_DIR/experiments" \
+ "skintokens/models:$SKINTOKENS_DIR/models"; do
+ rel="${pair%%:*}"
+ dst="${pair#*:}"
+ src="$WEIGHTS_ROOT/$rel"
+ if [ ! -e "$src" ]; then
+ echo "fatal: expected staged assets at $src (run workers/rig-nonhuman/stage-assets.sh)" >&2
+ exit 1
+ fi
+ mkdir -p "$(dirname "$dst")"
+ rm -rf "$dst"
+ echo "staging $src -> $dst"
+ cp -r "$src" "$dst"
+done
+
+exec uvicorn main:app --host 0.0.0.0 --port 8080 --workers 1 --timeout-keep-alive 300
diff --git a/workers/rig-nonhuman/gltf_meshopt.py b/workers/rig-nonhuman/gltf_meshopt.py
new file mode 100644
index 000000000..10f7ba4ad
--- /dev/null
+++ b/workers/rig-nonhuman/gltf_meshopt.py
@@ -0,0 +1,107 @@
+"""EXT_meshopt_compression decoding for caller-supplied glTF input.
+
+Vendored byte-identical into every worker that loads a mesh a caller handed it,
+exactly like worker_security.py: each worker's Docker build context is its own
+directory, so `../` is unreachable. `npm run check:vendored` fails if the copies
+drift apart.
+
+Why it exists: trimesh has no decoder for EXT_meshopt_compression. A compressed
+asset stores its geometry in compressed buffer views and declares a second,
+empty "fallback" buffer that only exists once something decodes into it, so
+trimesh dies on the bufferView lookup with a bare `IndexError: list index out of
+range`. Meshopt is what gltfpack emits and what most three.ws avatars ship as,
+so this was a hard failure on our own assets (verified 2026-08-13:
+/avatars/michelle.glb failed every stylize filter in production). gltfpack is
+the same project's CLI and decodes what it encoded, so the fix is to transcode
+the asset to plain glTF before any reader touches it.
+
+The binary is pinned by release tag and checksum in each worker's Dockerfile.
+Set GLTFPACK_BIN to run the path against your own copy locally.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import os
+import subprocess
+import tempfile
+from pathlib import Path
+from typing import Optional
+
+log = logging.getLogger("gltf-meshopt")
+
+MESHOPT_EXTENSION = "EXT_meshopt_compression"
+GLTFPACK_BIN = os.environ.get("GLTFPACK_BIN", "gltfpack")
+GLTFPACK_TIMEOUT_S = int(os.environ.get("GLTFPACK_TIMEOUT_S", "120"))
+
+
+def gltf_document(data: bytes, suffix: str = ".glb") -> Optional[dict]:
+ """The glTF JSON of a .glb/.gltf payload, or None when it is neither."""
+ if suffix == ".gltf":
+ try:
+ return json.loads(data)
+ except ValueError:
+ return None
+ if suffix != ".glb" or len(data) < 20 or data[:4] != b"glTF":
+ return None
+ chunk_length = int.from_bytes(data[12:16], "little")
+ if data[16:20] != b"JSON":
+ return None
+ try:
+ return json.loads(data[20:20 + chunk_length])
+ except ValueError:
+ return None
+
+
+def uses_meshopt(document: Optional[dict]) -> bool:
+ """True when the asset needs a meshopt decode before any reader can use it.
+
+ Checked against both extension lists: an asset that only *uses* the
+ extension still stores its geometry in the compressed views, and its
+ fallback buffer stays empty until something decodes into it.
+ """
+ if not document:
+ return False
+ declared = set(document.get("extensionsRequired") or []) | set(document.get("extensionsUsed") or [])
+ return MESHOPT_EXTENSION in declared
+
+
+def transcode_meshopt(data: bytes, suffix: str = ".glb") -> bytes:
+ """Rewrite a meshopt-compressed asset as a plain GLB with gltfpack.
+
+ `-noq` keeps the source quantization instead of re-quantizing on the way
+ out, so downstream geometry is derived from what the author shipped.
+ """
+ with tempfile.TemporaryDirectory() as tmpdir:
+ src = Path(tmpdir) / f"input{suffix}"
+ dst = Path(tmpdir) / "decoded.glb"
+ src.write_bytes(data)
+ try:
+ proc = subprocess.run(
+ [GLTFPACK_BIN, "-i", str(src), "-o", str(dst), "-noq"],
+ capture_output=True,
+ timeout=GLTFPACK_TIMEOUT_S,
+ )
+ except FileNotFoundError as exc:
+ raise ValueError(
+ f"input uses {MESHOPT_EXTENSION} and the gltfpack decoder is not installed"
+ ) from exc
+ except subprocess.TimeoutExpired as exc:
+ raise ValueError(f"{MESHOPT_EXTENSION} decode timed out") from exc
+ if proc.returncode != 0 or not dst.exists():
+ detail = (proc.stderr or b"").decode("utf-8", "replace").strip().splitlines()
+ raise ValueError(f"{MESHOPT_EXTENSION} decode failed: {detail[-1][:200] if detail else 'no output'}")
+ return dst.read_bytes()
+
+
+def decode_if_meshopt(data: bytes, suffix: str = ".glb") -> tuple[bytes, str]:
+ """Return (data, suffix) ready for trimesh, decoding meshopt when present.
+
+ A payload that is not compressed glTF is handed back untouched, so this is
+ safe to call on every input path.
+ """
+ if not uses_meshopt(gltf_document(data, suffix)):
+ return data, suffix
+ log.info("input declares %s; decoding with gltfpack before load", MESHOPT_EXTENSION)
+ return transcode_meshopt(data, suffix), ".glb"
diff --git a/workers/rig-nonhuman/main.py b/workers/rig-nonhuman/main.py
new file mode 100644
index 000000000..0cff08b65
--- /dev/null
+++ b/workers/rig-nonhuman/main.py
@@ -0,0 +1,274 @@
+"""
+Non-humanoid auto-rigging service: skeletons and skin weights for quadrupeds,
+creatures, and props that `workers/rig`'s Make-It-Animatable engine (built
+for bipedal humanoids) is not trained for.
+
+Engines (both MIT, LICENSE files read 2026-10-09):
+ - UniRig (VAST-AI-Research/UniRig, SIGGRAPH '25) predicts the joint
+ hierarchy: launch/inference/generate_skeleton.sh, then grafts it onto the
+ original mesh with launch/inference/merge.sh.
+ - SkinTokens (VAST-AI-Research/SkinTokens), UniRig's successor, predicts
+ ONLY the skinning weights for that skeleton-merged mesh
+ (demo.py --use_skeleton --use_transfer), preserving the original
+ texture and scale.
+See pipeline.py for the exact command contract and chaining.
+
+This service is reached two ways: directly, or forwarded by workers/rig when
+its mesh_classify.py geometric heuristic calls a mesh non-humanoid. Either
+way the API contract matches workers/rig exactly, so a caller (or a
+forwarding worker) polls one shape regardless of which engine rigged the
+mesh:
+ POST /rig { mesh_gcs_url: str, job_id?: str }
+ -> 202 { task_id, status: "queued" }
+ GET /tasks/:id -> { task_id, status, rigged_gcs_url?, error?, elapsed_ms? }
+ GET /health -> { ok, model, gpu_available, gpu_name, queued }
+
+Design notes (mirrors workers/rig/main.py on purpose):
+ - Every task runs under a hard timeout (TASK_TIMEOUT_S): three chained
+ model forward passes can finish or fail, but never hang in "running"
+ forever.
+ - Task state is in-memory: run with min-instances = max-instances so
+ pollers always reach the instance that owns the task.
+ - Un-riggable input is rejected before the GPU is touched, same validation
+ as the humanoid worker (rig_glb.validate_input_mesh, vendored call site
+ duplicated here since this worker's Docker build context cannot reach
+ workers/rig/rig_glb.py).
+
+Environment variables:
+ API_KEY shared bearer secret (Secret Manager)
+ GCS_BUCKET output bucket for rigged GLBs
+ UNIRIG_DIR UniRig checkout (default /app/unirig)
+ SKINTOKENS_DIR SkinTokens checkout (default /app/skintokens)
+ MAX_CONCURRENT parallel rigging jobs (default 1)
+ TASK_TIMEOUT_S per-task hard timeout in seconds (default 1200)
+"""
+
+from __future__ import annotations
+
+import asyncio
+import io
+import logging
+import os
+import tempfile
+import time
+import uuid
+from contextlib import asynccontextmanager
+from pathlib import Path
+from typing import Optional
+
+import httpx
+import numpy as np
+import torch
+import trimesh
+from fastapi import BackgroundTasks, FastAPI, Header, HTTPException
+from google.cloud import storage
+from google.cloud.storage.retry import DEFAULT_RETRY
+from pydantic import BaseModel
+
+from gltf_meshopt import decode_if_meshopt
+from pipeline import PipelineStepError, run_pipeline
+from worker_security import (
+ UnsafeUrlError,
+ fetch_remote_bytes_async,
+ require_api_key,
+ safe_error,
+)
+
+logging.basicConfig(
+ level=logging.INFO,
+ format="%(asctime)s %(levelname)s %(name)s %(message)s",
+)
+log = logging.getLogger("rig-nonhuman")
+
+API_KEY = os.environ["API_KEY"]
+GCS_BUCKET = os.environ["GCS_BUCKET"]
+UNIRIG_DIR = Path(os.environ.get("UNIRIG_DIR", "/app/unirig"))
+SKINTOKENS_DIR = Path(os.environ.get("SKINTOKENS_DIR", "/app/skintokens"))
+MAX_CONCURRENT = int(os.environ.get("MAX_CONCURRENT", "1"))
+TASK_TIMEOUT_S = float(os.environ.get("TASK_TIMEOUT_S", "1200"))
+TASK_TTL_S = 3600.0
+
+_GLB_MAGIC = b"glTF"
+_GLB_HEADER_BYTES = 12
+
+_bucket: Optional[storage.Bucket] = None
+_sem: Optional[asyncio.Semaphore] = None
+_tasks: dict[str, dict] = {}
+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+ global _bucket, _sem
+ _bucket = storage.Client().bucket(GCS_BUCKET)
+ _sem = asyncio.Semaphore(MAX_CONCURRENT)
+ log.info(
+ "Service ready: max_concurrent=%d task_timeout=%.0fs unirig_dir=%s skintokens_dir=%s",
+ MAX_CONCURRENT, TASK_TIMEOUT_S, UNIRIG_DIR, SKINTOKENS_DIR,
+ )
+ yield
+
+
+app = FastAPI(title="rig-nonhuman", lifespan=lifespan)
+
+
+def _require_api_key(authorization: str) -> None:
+ try:
+ require_api_key(authorization, API_KEY)
+ except PermissionError as exc:
+ raise HTTPException(status_code=401, detail=str(exc)) from exc
+
+
+def _prune_tasks() -> None:
+ cutoff = time.time() - TASK_TTL_S
+ for tid in [t for t, v in _tasks.items() if v.get("created_at", 0) < cutoff]:
+ _tasks.pop(tid, None)
+
+
+class InvalidMeshError(ValueError):
+ """The caller's mesh cannot be rigged: wrong container, or no surface.
+ Mirrors workers/rig/rig_glb.validate_input_mesh (duplicated here: this
+ worker's Docker build context cannot reach that module's source file)."""
+
+
+def _validate_input_mesh(mesh_bytes: bytes) -> None:
+ if len(mesh_bytes) < _GLB_HEADER_BYTES or mesh_bytes[:4] != _GLB_MAGIC:
+ raise InvalidMeshError(
+ "input is not a binary glTF: expected a .glb starting with the "
+ "'glTF' magic header"
+ )
+ try:
+ mesh = trimesh.load(io.BytesIO(mesh_bytes), file_type="glb", force="mesh")
+ except Exception as exc:
+ log.warning("input GLB failed to parse: %s", exc, exc_info=True)
+ raise InvalidMeshError("input GLB could not be parsed as a mesh") from exc
+ faces = getattr(mesh, "faces", None)
+ if faces is None or len(faces) == 0:
+ raise InvalidMeshError(
+ "input GLB contains no triangles; rigging needs a surface mesh, "
+ "not a point cloud or a curves-only export"
+ )
+ if float(np.sum(mesh.area_faces)) <= 0.0:
+ raise InvalidMeshError(
+ "input GLB has zero surface area; every triangle is degenerate"
+ )
+
+
+def _rig_sync(mesh_bytes: bytes) -> bytes:
+ """The blocking rig pipeline; runs in a worker thread."""
+ _validate_input_mesh(mesh_bytes)
+ with tempfile.TemporaryDirectory() as tmp:
+ workdir = Path(tmp)
+ mesh_glb = workdir / "input.glb"
+ mesh_glb.write_bytes(mesh_bytes)
+ rigged_path = run_pipeline(mesh_glb, workdir, UNIRIG_DIR, SKINTOKENS_DIR)
+ return rigged_path.read_bytes()
+
+
+async def _run_task(task_id: str, mesh_gcs_url: str) -> None:
+ async with _sem:
+ _tasks[task_id]["status"] = "running"
+ loop = asyncio.get_event_loop()
+ t0 = time.time()
+ try:
+ async def _work() -> str:
+ async with httpx.AsyncClient(timeout=60, follow_redirects=False) as client:
+ try:
+ mesh_bytes = await fetch_remote_bytes_async(client, mesh_gcs_url)
+ except UnsafeUrlError as exc:
+ raise RuntimeError(f"refused to fetch mesh url: {exc}") from exc
+
+ mesh_bytes, _ = decode_if_meshopt(mesh_bytes, ".glb")
+ rigged = await loop.run_in_executor(None, _rig_sync, mesh_bytes)
+
+ blob_name = f"rigged-meshes/{task_id}.glb"
+ blob = _bucket.blob(blob_name)
+ await loop.run_in_executor(
+ None,
+ lambda: blob.upload_from_string(
+ rigged, content_type="model/gltf-binary", retry=DEFAULT_RETRY
+ ),
+ )
+ return f"https://storage.googleapis.com/{GCS_BUCKET}/{blob_name}"
+
+ gcs_url = await asyncio.wait_for(_work(), timeout=TASK_TIMEOUT_S)
+ elapsed = time.time() - t0
+ _tasks[task_id].update({
+ "status": "done",
+ "rigged_gcs_url": gcs_url,
+ "elapsed_ms": int(elapsed * 1000),
+ })
+ log.info("[%s] non-humanoid rigging done in %.1fs -> %s", task_id, elapsed, gcs_url)
+ except asyncio.TimeoutError:
+ _tasks[task_id].update({
+ "status": "failed",
+ "error": f"rigging timed out after {TASK_TIMEOUT_S:.0f}s",
+ "elapsed_ms": int((time.time() - t0) * 1000),
+ })
+ log.error("[%s] rigging timed out", task_id)
+ except InvalidMeshError as exc:
+ _tasks[task_id].update({
+ "status": "failed",
+ "error": str(exc),
+ "elapsed_ms": int((time.time() - t0) * 1000),
+ })
+ log.warning("[%s] rejected input mesh: %s", task_id, exc)
+ except PipelineStepError as exc:
+ # One upstream CLI step failed. Not the caller's fault (their
+ # mesh already passed validation), but still a specific,
+ # actionable reason rather than an opaque correlation id: it
+ # names which of the three stages broke.
+ _tasks[task_id].update({
+ "status": "failed",
+ "error": str(exc),
+ "elapsed_ms": int((time.time() - t0) * 1000),
+ })
+ log.error("[%s] pipeline step failed: %s", task_id, exc)
+ except Exception as exc:
+ _tasks[task_id].update({
+ "status": "failed",
+ "error": safe_error(exc, context=f"[{task_id}] non-humanoid rigging"),
+ "elapsed_ms": int((time.time() - t0) * 1000),
+ })
+
+
+class RigRequest(BaseModel):
+ mesh_gcs_url: str
+ job_id: str | None = None
+
+
[email protected]("/rig", status_code=202)
+async def rig(
+ body: RigRequest,
+ background_tasks: BackgroundTasks,
+ authorization: str = Header(...),
+) -> dict:
+ _require_api_key(authorization)
+ _prune_tasks()
+ task_id = str(uuid.uuid4())
+ _tasks[task_id] = {
+ "task_id": task_id,
+ "status": "queued",
+ "created_at": time.time(),
+ }
+ background_tasks.add_task(_run_task, task_id, body.mesh_gcs_url)
+ return {"task_id": task_id, "status": "queued"}
+
+
[email protected]("/tasks/{task_id}")
+async def get_task(task_id: str, authorization: str = Header(...)) -> dict:
+ _require_api_key(authorization)
+ task = _tasks.get(task_id)
+ if task is None:
+ raise HTTPException(status_code=404, detail="task not found")
+ return {k: v for k, v in task.items() if k != "created_at"}
+
+
+async def health() -> dict:
+ return {
+ "ok": True,
+ "model": "unirig+skintokens",
+ "gpu_available": torch.cuda.is_available(),
+ "gpu_name": torch.cuda.get_device_name(0) if torch.cuda.is_available() else None,
+ "queued": sum(1 for t in _tasks.values() if t.get("status") in ("queued", "running")),
+ }
diff --git a/workers/rig-nonhuman/pipeline.py b/workers/rig-nonhuman/pipeline.py
new file mode 100644
index 000000000..13bedb8f2
--- /dev/null
+++ b/workers/rig-nonhuman/pipeline.py
@@ -0,0 +1,118 @@
+"""
+Non-humanoid rigging pipeline: chains two VAST-AI-Research CLIs (both MIT,
+LICENSE files read 2026-10-09) over temp files, exactly per the command
+contract in their own READMEs. Neither project exposes a Python import API,
+so this worker calls them as subprocesses, the same way their own docs do.
+
+ 1. UniRig's generate_skeleton.sh predicts a joint hierarchy for an
+ arbitrary mesh (quadruped, creature, prop) and writes it as .fbx.
+ 2. UniRig's merge.sh grafts that skeleton onto the ORIGINAL .glb (adds
+ joint nodes; no skinning weights yet).
+ 3. SkinTokens' demo.py --use_skeleton --use_transfer takes the
+ skeleton-merged .glb and predicts ONLY the skinning weights, preserving
+ the original texture and scale, and writes a fully rigged .glb.
+
+Kept dependency-light (stdlib subprocess only) so the chaining/error-handling
+logic is unit-tested without a GPU or either upstream checkout; see
+test_pipeline.py, which substitutes tiny real scripts at the exact paths the
+command builders expect.
+"""
+
+from __future__ import annotations
+
+import logging
+import subprocess
+from pathlib import Path
+
+log = logging.getLogger("rig-nonhuman.pipeline")
+
+# Each stage is a multi-second-to-minute autoregressive model forward pass on
+# one mesh; generous but finite so a wedged subprocess cannot hang a task
+# forever. The caller still enforces an overall job timeout on top of this.
+SUBPROCESS_TIMEOUT_S = 600.0
+
+
+def build_skeleton_cmd(mesh_glb: Path, skeleton_fbx: Path) -> list[str]:
+ """UniRig: predict a skeleton for `mesh_glb`, write it to `skeleton_fbx`."""
+ return [
+ "bash", "launch/inference/generate_skeleton.sh",
+ "--input", str(mesh_glb),
+ "--output", str(skeleton_fbx),
+ ]
+
+
+def build_merge_cmd(skeleton_fbx: Path, mesh_glb: Path, merged_glb: Path) -> list[str]:
+ """UniRig: graft `skeleton_fbx` onto `mesh_glb`, write `merged_glb`."""
+ return [
+ "bash", "launch/inference/merge.sh",
+ "--source", str(skeleton_fbx),
+ "--target", str(mesh_glb),
+ "--output", str(merged_glb),
+ ]
+
+
+def build_skin_cmd(merged_glb: Path, rigged_glb: Path) -> list[str]:
+ """SkinTokens: predict skin weights for the already-skeletoned
+ `merged_glb`, preserving texture/scale, write the final `rigged_glb`."""
+ return [
+ "python3", "demo.py",
+ "--input", str(merged_glb),
+ "--output", str(rigged_glb),
+ "--use_skeleton",
+ "--use_transfer",
+ ]
+
+
+class PipelineStepError(RuntimeError):
+ """One of the three subprocess steps failed: non-zero exit, a timeout, or
+ a reported success that wrote no output file. The message carries the
+ step name and a stderr tail only; the caller logs the full capture
+ server-side (see main.py's use of worker_security.safe_error for what
+ ultimately reaches the HTTP caller)."""
+
+
+def _run(cmd: list[str], cwd: Path, step: str) -> subprocess.CompletedProcess:
+ log.info("[%s] %s (cwd=%s)", step, " ".join(cmd), cwd)
+ try:
+ return subprocess.run(
+ cmd, cwd=str(cwd), capture_output=True, text=True,
+ timeout=SUBPROCESS_TIMEOUT_S, check=False,
+ )
+ except subprocess.TimeoutExpired as exc:
+ raise PipelineStepError(
+ f"{step} timed out after {SUBPROCESS_TIMEOUT_S:.0f}s"
+ ) from exc
+
+
+def _run_step(cmd: list[str], cwd: Path, step: str, output: Path) -> None:
+ result = _run(cmd, cwd, step)
+ if result.returncode != 0:
+ tail = "\n".join(result.stderr.strip().splitlines()[-20:])
+ raise PipelineStepError(f"{step} failed (exit {result.returncode}): {tail}")
+ if not output.exists():
+ raise PipelineStepError(f"{step} reported success but wrote no output file")
+
+
+def run_pipeline(mesh_glb: Path, workdir: Path, unirig_dir: Path, skintokens_dir: Path) -> Path:
+ """Run the full skeleton -> merge -> skin chain over `mesh_glb`.
+
+ Returns the path to the final rigged .glb, written inside `workdir`.
+ `unirig_dir` and `skintokens_dir` are the two upstream checkouts' root
+ directories (their relative script paths are run with that cwd)."""
+ skeleton_fbx = workdir / "skeleton.fbx"
+ merged_glb = workdir / "merged.glb"
+ rigged_glb = workdir / "rigged.glb"
+
+ _run_step(
+ build_skeleton_cmd(mesh_glb, skeleton_fbx), unirig_dir,
+ "unirig-skeleton", skeleton_fbx,
+ )
+ _run_step(
+ build_merge_cmd(skeleton_fbx, mesh_glb, merged_glb), unirig_dir,
+ "unirig-merge", merged_glb,
+ )
+ _run_step(
+ build_skin_cmd(merged_glb, rigged_glb), skintokens_dir,
+ "skintokens-skin", rigged_glb,
+ )
+ return rigged_glb
diff --git a/workers/rig-nonhuman/requirements.txt b/workers/rig-nonhuman/requirements.txt
new file mode 100644
index 000000000..9326e0c0a
--- /dev/null
+++ b/workers/rig-nonhuman/requirements.txt
@@ -0,0 +1,13 @@
+# This worker's own FastAPI/GCS layer. torch/torchvision, spconv,
+# torch_scatter/torch_cluster, and flash-attn install in their own Dockerfile
+# layers BEFORE this file, per UniRig's and SkinTokens' own install docs
+# (neither ships a pip-installable package; both are subprocess CLIs).
+numpy==1.26.4
+trimesh>=4.4.0,<5
+
+fastapi>=0.115,<1
+uvicorn[standard]>=0.32,<1
+pydantic>=2.0.0
+httpx>=0.27.0
+
+google-cloud-storage==2.18.0
diff --git a/workers/rig-nonhuman/stage-assets.sh b/workers/rig-nonhuman/stage-assets.sh
new file mode 100755
index 000000000..0b96c16b1
--- /dev/null
+++ b/workers/rig-nonhuman/stage-assets.sh
@@ -0,0 +1,72 @@
+#!/usr/bin/env bash
+# Stage everything the non-humanoid rig worker needs into the model-weights
+# bucket. Run once (from any machine with gcloud + python3 + huggingface_hub)
+# before the first deploy, and again only when bumping UNIRIG_COMMIT or
+# SKINTOKENS_COMMIT in the Dockerfile.
+#
+# bash workers/rig-nonhuman/stage-assets.sh
+#
+# Layout produced under gs://three-ws-model-weights/rig-nonhuman/:
+# unirig/experiments/skeleton/articulation-xl_quantization_256/model.ckpt
+# UniRig's skeleton-prediction checkpoint (HF: VAST-AI/UniRig). This is
+# the only UniRig checkpoint staged: skinning runs through SkinTokens
+# instead of UniRig's own generate_skin.sh.
+# skintokens/experiments/skin_vae_2_10_32768/last.ckpt
+# SkinTokens' FSQ-CVAE skin-weight tokenizer (HF: VAST-AI/SkinTokens).
+# skintokens/experiments/articulation_xl_quantization_256_token_4/grpo_1400.ckpt
+# SkinTokens' TokenRig autoregressive rigging checkpoint, GRPO-refined
+# (the README's "recommended" model).
+# skintokens/models/Qwen3-0.6B/
+# Config + tokenizer only (no .bin/.safetensors) for the Qwen3-0.6B
+# backbone architecture SkinTokens was fine-tuned from; the actual
+# weights live inside the two checkpoints above, same as upstream's own
+# download.py --model.
+#
+# Both UniRig and SkinTokens resolve their checkpoint paths by first trying
+# huggingface_hub, then falling back to the literal relative path if that
+# fails (see UniRig's src/inference/download.py and run.py:240); entrypoint.sh
+# pre-places these files at those literal paths so inference needs no
+# Hugging Face egress at runtime.
+set -euo pipefail
+
+BUCKET="${BUCKET:-gs://three-ws-model-weights/rig-nonhuman}"
+WORK="$(mktemp -d)"
+trap 'rm -rf "$WORK"' EXIT
+
+echo "==> UniRig skeleton checkpoint (HF: VAST-AI/UniRig)"
+python3 -c "
+from huggingface_hub import hf_hub_download
+path = hf_hub_download(
+ repo_id='VAST-AI/UniRig',
+ filename='skeleton/articulation-xl_quantization_256/model.ckpt',
+ local_dir='$WORK/unirig/experiments',
+)
+print('downloaded to', path)
+"
+
+echo "==> SkinTokens checkpoints + Qwen3-0.6B config (HF: VAST-AI/SkinTokens, Qwen/Qwen3-0.6B)"
+python3 -c "
+from huggingface_hub import hf_hub_download, snapshot_download
+
+for name in (
+ 'experiments/skin_vae_2_10_32768/last.ckpt',
+ 'experiments/articulation_xl_quantization_256_token_4/grpo_1400.ckpt',
+):
+ path = hf_hub_download(repo_id='VAST-AI/SkinTokens', filename=name, local_dir='$WORK/skintokens')
+ print('downloaded to', path)
+
+path = snapshot_download(
+ repo_id='Qwen/Qwen3-0.6B',
+ local_dir='$WORK/skintokens/models/Qwen3-0.6B',
+ ignore_patterns=['*.bin', '*.safetensors'],
+)
+print('downloaded to', path)
+"
+
+echo "==> Upload to $BUCKET"
+gcloud storage cp -r "$WORK/unirig/experiments" "$BUCKET/unirig/experiments"
+gcloud storage cp -r "$WORK/skintokens/experiments" "$BUCKET/skintokens/experiments"
+gcloud storage cp -r "$WORK/skintokens/models" "$BUCKET/skintokens/models"
+
+echo "==> Staged. Verify:"
+gcloud storage ls -r "$BUCKET" | sed 's/^/ /'
diff --git a/workers/rig-nonhuman/test_gltf_meshopt.py b/workers/rig-nonhuman/test_gltf_meshopt.py
new file mode 100644
index 000000000..2d2e9d502
--- /dev/null
+++ b/workers/rig-nonhuman/test_gltf_meshopt.py
@@ -0,0 +1,141 @@
+"""Tests for gltf_meshopt.py, the shared EXT_meshopt_compression decoder.
+
+Vendored byte-identical into every worker that ships gltf_meshopt.py, exactly
+like the module itself, and run as a Docker build gate in each of them:
+
+ python3 test_gltf_meshopt.py
+
+The round-trip section needs the `gltfpack` binary (present in every worker
+image, pinned in the Dockerfile). Without it the suite still runs and pins the
+readable failure a caller gets instead, so a laptop run stays green.
+"""
+
+from __future__ import annotations
+
+import io
+import os
+import shutil
+import subprocess
+import sys
+import tempfile
+from pathlib import Path
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+
+import trimesh # noqa: E402
+
+import gltf_meshopt # noqa: E402
+
+PASS = 0
+
+
+def check(name: str, condition: bool, detail: str = "") -> None:
+ global PASS
+ if not condition:
+ print(f"FAIL {name} {detail}")
+ sys.exit(1)
+ PASS += 1
+ print(f"ok {name}")
+
+
+plain_glb = trimesh.Scene({"a": trimesh.creation.icosphere(subdivisions=2)}).export(file_type="glb")
+
+# ── document parsing ────────────────────────────────────────────────────────────
+
+doc = gltf_meshopt.gltf_document(plain_glb, ".glb")
+check("a glb's json chunk parses", isinstance(doc, dict) and "asset" in doc, str(type(doc)))
+check(
+ "a gltf json payload parses",
+ isinstance(gltf_meshopt.gltf_document(b'{"asset":{"version":"2.0"}}', ".gltf"), dict),
+)
+for payload, suffix, label in (
+ (b"ply\nformat ascii 1.0\n", ".ply", "a ply"),
+ (b"not json at all", ".gltf", "malformed gltf json"),
+ (b"glTF" + b"\x00" * 4, ".glb", "a truncated glb"),
+ (b"solid mesh\n", ".stl", "an stl"),
+):
+ check(f"{label} yields no gltf document", gltf_meshopt.gltf_document(payload, suffix) is None)
+
+# ── extension detection ─────────────────────────────────────────────────────────
+
+check("a plain glb needs no decode", not gltf_meshopt.uses_meshopt(doc))
+check("an empty document needs no decode", not gltf_meshopt.uses_meshopt(None))
+for declared in ("extensionsRequired", "extensionsUsed"):
+ check(
+ f"{declared} announcing {gltf_meshopt.MESHOPT_EXTENSION} needs a decode",
+ gltf_meshopt.uses_meshopt({declared: [gltf_meshopt.MESHOPT_EXTENSION]}),
+ )
+check(
+ "an unrelated extension needs no decode",
+ not gltf_meshopt.uses_meshopt({"extensionsUsed": ["KHR_materials_unlit"]}),
+)
+check(
+ "an uncompressed payload passes through decode_if_meshopt untouched",
+ gltf_meshopt.decode_if_meshopt(plain_glb, ".glb") == (plain_glb, ".glb"),
+)
+
+# ── the real decode ─────────────────────────────────────────────────────────────
+
+if shutil.which(gltf_meshopt.GLTFPACK_BIN):
+ with tempfile.TemporaryDirectory() as packdir:
+ raw = Path(packdir) / "raw.glb"
+ packed = Path(packdir) / "packed.glb"
+ raw.write_bytes(plain_glb)
+ subprocess.run(
+ [gltf_meshopt.GLTFPACK_BIN, "-i", str(raw), "-o", str(packed), "-cc"],
+ capture_output=True,
+ check=True,
+ timeout=gltf_meshopt.GLTFPACK_TIMEOUT_S,
+ )
+ packed_bytes = packed.read_bytes()
+
+ check(
+ "a gltfpack-compressed glb is detected",
+ gltf_meshopt.uses_meshopt(gltf_meshopt.gltf_document(packed_bytes, ".glb")),
+ )
+
+ # The bug this module exists for: trimesh dies on the compressed asset's
+ # fallback buffer, which is why every worker loading one failed.
+ try:
+ trimesh.load(io.BytesIO(packed_bytes), file_type="glb", force="mesh", process=False)
+ raw_load_failed = False
+ except Exception: # noqa: BLE001: the exact trimesh error is not the contract
+ raw_load_failed = True
+ check("trimesh alone cannot read a meshopt glb", raw_load_failed)
+
+ decoded, suffix = gltf_meshopt.decode_if_meshopt(packed_bytes, ".glb")
+ check("the decode returns a glb", suffix == ".glb" and decoded[:4] == b"glTF", suffix)
+ check("the decode is not a passthrough", decoded != packed_bytes)
+ check(
+ "the decoded asset no longer declares the extension",
+ not gltf_meshopt.uses_meshopt(gltf_meshopt.gltf_document(decoded, ".glb")),
+ )
+
+ mesh = trimesh.load(io.BytesIO(decoded), file_type="glb", force="mesh", process=False)
+ check(
+ "the decoded asset loads as real geometry",
+ len(mesh.faces) > 0 and len(mesh.vertices) > 0,
+ f"faces={len(mesh.faces)}",
+ )
+ source = trimesh.load(io.BytesIO(plain_glb), file_type="glb", force="mesh", process=False)
+ check(
+ "the decoded geometry matches the source face count",
+ len(mesh.faces) == len(source.faces),
+ f"{len(mesh.faces)} vs {len(source.faces)}",
+ )
+else:
+ original_bin = gltf_meshopt.GLTFPACK_BIN
+ try:
+ gltf_meshopt.GLTFPACK_BIN = "gltfpack-not-installed"
+ gltf_meshopt.transcode_meshopt(plain_glb, ".glb")
+ check("a missing gltfpack is a readable error", False, "no ValueError raised")
+ except ValueError as exc:
+ check(
+ "a missing gltfpack is a readable error",
+ gltf_meshopt.MESHOPT_EXTENSION in str(exc),
+ str(exc),
+ )
+ finally:
+ gltf_meshopt.GLTFPACK_BIN = original_bin
+
+print(f"\n{PASS} checks passed")
diff --git a/workers/rig-nonhuman/test_pipeline.py b/workers/rig-nonhuman/test_pipeline.py
new file mode 100644
index 000000000..c97acd343
--- /dev/null
+++ b/workers/rig-nonhuman/test_pipeline.py
@@ -0,0 +1,202 @@
+"""Unit tests for pipeline.py: command construction against the real UniRig
+and SkinTokens CLI contracts, and the subprocess-chaining/error-handling
+logic. No GPU and no upstream checkout: each test that exercises run_pipeline
+substitutes a tiny real script at the exact relative path the command
+builders invoke, so the chaining code actually executes a subprocess (just
+not the multi-gigabyte model behind it).
+
+No GPU, no torch, no GCP: `python3 test_pipeline.py`.
+"""
+
+from __future__ import annotations
+
+import shutil
+import stat
+import sys
+import tempfile
+from pathlib import Path
+
+from pipeline import (
+ PipelineStepError,
+ build_merge_cmd,
+ build_skeleton_cmd,
+ build_skin_cmd,
+ run_pipeline,
+)
+
+_FAILED = False
+
+
+def _check(cond: bool, label: str) -> None:
+ global _FAILED
+ status = "ok" if cond else "FAIL"
+ print(f" [{status}] {label}")
+ if not cond:
+ _FAILED = True
+
+
+def test_skeleton_cmd_matches_unirig_cli_contract():
+ tmp = Path("/tmp/x")
+ cmd = build_skeleton_cmd(tmp / "in.glb", tmp / "out.fbx")
+ _check(cmd == [
+ "bash", "launch/inference/generate_skeleton.sh",
+ "--input", str(tmp / "in.glb"),
+ "--output", str(tmp / "out.fbx"),
+ ], "skeleton cmd matches UniRig's generate_skeleton.sh contract")
+
+
+def test_merge_cmd_matches_unirig_cli_contract():
+ tmp = Path("/tmp/x")
+ cmd = build_merge_cmd(tmp / "skel.fbx", tmp / "in.glb", tmp / "merged.glb")
+ _check(cmd == [
+ "bash", "launch/inference/merge.sh",
+ "--source", str(tmp / "skel.fbx"),
+ "--target", str(tmp / "in.glb"),
+ "--output", str(tmp / "merged.glb"),
+ ], "merge cmd matches UniRig's merge.sh contract")
+
+
+def test_skin_cmd_matches_skintokens_cli_contract():
+ tmp = Path("/tmp/x")
+ cmd = build_skin_cmd(tmp / "merged.glb", tmp / "rigged.glb")
+ _check(cmd == [
+ "python3", "demo.py",
+ "--input", str(tmp / "merged.glb"),
+ "--output", str(tmp / "rigged.glb"),
+ "--use_skeleton",
+ "--use_transfer",
+ ], "skin cmd matches SkinTokens' demo.py --use_skeleton --use_transfer contract")
+
+
+def _fake_bash_script(path: Path, writes: Path | None = None, exit_code: int = 0) -> None:
+ path.parent.mkdir(parents=True, exist_ok=True)
+ body = "#!/bin/sh\n"
+ if writes is not None:
+ body += f"echo fake > '{writes}'\n"
+ body += f"exit {exit_code}\n"
+ path.write_text(body)
+ path.chmod(path.stat().st_mode | stat.S_IEXEC)
+
+
+def _fake_py_script(path: Path, writes: Path | None = None, exit_code: int = 0) -> None:
+ path.parent.mkdir(parents=True, exist_ok=True)
+ lines = ["import sys"]
+ if writes is not None:
+ lines.append(f"open(r'{writes}', 'w').write('fake')")
+ lines.append(f"sys.exit({exit_code})")
+ path.write_text("\n".join(lines) + "\n")
+
+
+def _stub_checkouts(root: Path) -> tuple[Path, Path]:
+ unirig_dir = root / "unirig"
+ skintokens_dir = root / "skintokens"
+ unirig_dir.mkdir()
+ skintokens_dir.mkdir()
+ return unirig_dir, skintokens_dir
+
+
+def test_run_pipeline_chains_all_three_steps():
+ root = Path(tempfile.mkdtemp())
+ try:
+ unirig_dir, skintokens_dir = _stub_checkouts(root)
+ workdir = root / "work"
+ workdir.mkdir()
+ mesh_glb = workdir / "mesh.glb"
+ mesh_glb.write_bytes(b"glTF-fake-mesh")
+
+ _fake_bash_script(
+ unirig_dir / "launch/inference/generate_skeleton.sh", workdir / "skeleton.fbx"
+ )
+ _fake_bash_script(unirig_dir / "launch/inference/merge.sh", workdir / "merged.glb")
+ _fake_py_script(skintokens_dir / "demo.py", workdir / "rigged.glb")
+
+ result = run_pipeline(mesh_glb, workdir, unirig_dir, skintokens_dir)
+
+ _check(result == workdir / "rigged.glb", "returns the final rigged.glb path")
+ _check(result.exists(), "final output file exists")
+ _check((workdir / "skeleton.fbx").exists(), "intermediate skeleton.fbx exists")
+ _check((workdir / "merged.glb").exists(), "intermediate merged.glb exists")
+ finally:
+ shutil.rmtree(root)
+
+
+def test_run_pipeline_raises_on_nonzero_exit():
+ root = Path(tempfile.mkdtemp())
+ try:
+ unirig_dir, skintokens_dir = _stub_checkouts(root)
+ workdir = root / "work"
+ workdir.mkdir()
+ mesh_glb = workdir / "mesh.glb"
+ mesh_glb.write_bytes(b"glTF-fake-mesh")
+
+ _fake_bash_script(
+ unirig_dir / "launch/inference/generate_skeleton.sh",
+ workdir / "skeleton.fbx",
+ exit_code=1,
+ )
+ try:
+ run_pipeline(mesh_glb, workdir, unirig_dir, skintokens_dir)
+ _check(False, "expected PipelineStepError for nonzero exit")
+ except PipelineStepError as exc:
+ _check("unirig-skeleton failed" in str(exc), f"error names the failed step: {exc}")
+ finally:
+ shutil.rmtree(root)
+
+
+def test_run_pipeline_raises_on_missing_output_despite_zero_exit():
+ root = Path(tempfile.mkdtemp())
+ try:
+ unirig_dir, skintokens_dir = _stub_checkouts(root)
+ workdir = root / "work"
+ workdir.mkdir()
+ mesh_glb = workdir / "mesh.glb"
+ mesh_glb.write_bytes(b"glTF-fake-mesh")
+
+ # Exits 0 but never writes skeleton.fbx: the no-output case a crashed
+ # model process can produce without a nonzero exit code.
+ _fake_bash_script(unirig_dir / "launch/inference/generate_skeleton.sh")
+ try:
+ run_pipeline(mesh_glb, workdir, unirig_dir, skintokens_dir)
+ _check(False, "expected PipelineStepError for missing output")
+ except PipelineStepError as exc:
+ _check("wrote no output file" in str(exc), f"missing-output case named: {exc}")
+ finally:
+ shutil.rmtree(root)
+
+
+def test_run_pipeline_stops_before_skinning_if_merge_fails():
+ root = Path(tempfile.mkdtemp())
+ try:
+ unirig_dir, skintokens_dir = _stub_checkouts(root)
+ workdir = root / "work"
+ workdir.mkdir()
+ mesh_glb = workdir / "mesh.glb"
+ mesh_glb.write_bytes(b"glTF-fake-mesh")
+
+ _fake_bash_script(
+ unirig_dir / "launch/inference/generate_skeleton.sh", workdir / "skeleton.fbx"
+ )
+ _fake_bash_script(unirig_dir / "launch/inference/merge.sh", exit_code=2)
+ # No demo.py stub at all: if the pipeline wrongly proceeded to
+ # skinning, this would fail with "No such file" instead of merge's.
+ try:
+ run_pipeline(mesh_glb, workdir, unirig_dir, skintokens_dir)
+ _check(False, "expected PipelineStepError for merge failure")
+ except PipelineStepError as exc:
+ _check("unirig-merge failed" in str(exc), f"stopped at merge, not skinning: {exc}")
+ finally:
+ shutil.rmtree(root)
+
+
+if __name__ == "__main__":
+ test_skeleton_cmd_matches_unirig_cli_contract()
+ test_merge_cmd_matches_unirig_cli_contract()
+ test_skin_cmd_matches_skintokens_cli_contract()
+ test_run_pipeline_chains_all_three_steps()
+ test_run_pipeline_raises_on_nonzero_exit()
+ test_run_pipeline_raises_on_missing_output_despite_zero_exit()
+ test_run_pipeline_stops_before_skinning_if_merge_fails()
+ if _FAILED:
+ print("\nFAILED")
+ sys.exit(1)
+ print("\nALL PASSED")
diff --git a/workers/rig-nonhuman/worker_security.py b/workers/rig-nonhuman/worker_security.py
new file mode 100644
index 000000000..802e1c32d
--- /dev/null
+++ b/workers/rig-nonhuman/worker_security.py
@@ -0,0 +1,320 @@
+"""
+Shared security helpers for the three.ws GPU/CPU workers.
+
+This module is intentionally dependency-light (stdlib only) so it can be
+COPYed verbatim into every worker's Docker build context. The same canonical
+file lives in each worker directory; keep them byte-identical when editing.
+
+It provides three primitives that every worker needs:
+
+ 1. ``require_api_key`` - constant-time bearer-token check (timing-safe).
+ 2. ``fetch_remote_bytes`` - SSRF-hardened HTTP(S) fetch of caller-supplied
+ URLs: https-only, DNS resolution with private/loopback/link-local/metadata
+ IP rejection, per-hop redirect re-validation, and a bounded response size.
+ 3. ``safe_error`` - turns an exception into an opaque, correlation-id-tagged
+ message safe to persist/return to callers, after logging the full
+ traceback server-side only.
+"""
+
+from __future__ import annotations
+
+import hmac
+import ipaddress
+import logging
+import asyncio
+import socket
+import time
+import uuid
+from typing import Iterable, Optional
+from urllib.parse import urlsplit
+
+import httpx
+
+log = logging.getLogger("worker_security")
+
+# ── auth ────────────────────────────────────────────────────────────────────────
+
+
+def require_api_key(authorization: Optional[str], api_key: str) -> None:
+ """Validate a ``Bearer <token>`` header in constant time.
+
+ Raises ``PermissionError`` on any failure so callers can translate it into
+ the framework's 401 (FastAPI workers map this to HTTPException(401)).
+ """
+ # Fail closed on an empty configured key: with api_key == "" a bare
+ # "Bearer " header would otherwise strip to "" and compare equal.
+ if not api_key:
+ raise PermissionError("worker api key not configured")
+ if not authorization or not authorization.startswith("Bearer "):
+ raise PermissionError("missing bearer token")
+ token = authorization[len("Bearer "):].strip()
+ if not token:
+ raise PermissionError("missing bearer token")
+ # Compare bytes: hmac.compare_digest is timing-safe, and on str operands it
+ # raises TypeError for non-ASCII input, which would surface as a 500.
+ if not hmac.compare_digest(token.encode("utf-8"), api_key.encode("utf-8")):
+ raise PermissionError("invalid api key")
+
+
+# ── SSRF-hardened fetch ─────────────────────────────────────────────────────────
+
+# Default cap on a fetched response. Avatar selfies / driving audio / raw meshes
+# are comfortably under this; anything larger is almost certainly abuse.
+DEFAULT_MAX_BYTES = 64 * 1024 * 1024 # 64 MiB
+_MAX_REDIRECTS = 4
+
+# Transient fetch failures are retried: a dropped connection, a timeout, or a
+# 429/5xx from the image host. The public R2 dev endpoint that serves every
+# forge reference image drops connections under load ("Server disconnected
+# without sending a response"), and before this retry each drop ended a
+# generation as an opaque internal error on TripoSG and Hunyuan3D (about 20 a
+# week measured 2026-10-09). Refusals (UnsafeUrlError), oversize bodies, and
+# other 4xx answers are final and never retried.
+_FETCH_ATTEMPTS = 3
+_FETCH_BACKOFF_SECS = (0.5, 1.5)
+
+# Hosts that enforce a User-Agent policy refuse httpx's default
+# "python-httpx/<version>": upload.wikimedia.org answers it with 403 Forbidden
+# and serves the same image to a descriptive agent (measured 2026-10-09, when
+# model-triposg failed every job whose reference photo was a Wikimedia link).
+# Callers may still pass their own User-Agent; it replaces this one.
+DEFAULT_USER_AGENT = "three.ws-worker/1.0 (+https://three.ws)"
+
+
+def _fetch_headers(headers: Optional[dict]) -> dict:
+ """Caller headers over the default User-Agent, matched case-insensitively."""
+ merged = dict(headers or {})
+ if not any(str(k).lower() == "user-agent" for k in merged):
+ merged["User-Agent"] = DEFAULT_USER_AGENT
+ return merged
+
+
+class UnsafeUrlError(ValueError):
+ """Raised when a URL targets a disallowed scheme or a private/internal IP."""
+
+
+def _is_blocked_ip(ip: ipaddress._BaseAddress) -> bool:
+ """Reject any address that could reach internal infrastructure.
+
+ Covers loopback, RFC1918 private ranges, link-local (incl. the cloud
+ metadata server at 169.254.169.254), unique-local IPv6 (fc00::/7),
+ multicast, reserved, unspecified, and any other non-global address (CGNAT
+ 100.64.0.0/10 included). IPv4-mapped IPv6 addresses
+ are unwrapped first so a mapped private v4 can't slip through.
+ """
+ if isinstance(ip, ipaddress.IPv6Address) and ip.ipv4_mapped is not None:
+ ip = ip.ipv4_mapped
+ return (
+ ip.is_private
+ or ip.is_loopback
+ or ip.is_link_local
+ or ip.is_multicast
+ or ip.is_reserved
+ or ip.is_unspecified
+ # Anything not globally routable, which also covers the CGNAT shared
+ # space 100.64.0.0/10 that is_private does not flag.
+ or not ip.is_global
+ )
+
+
+def _resolve_safe_host(host: str) -> None:
+ """DNS-resolve ``host`` and raise UnsafeUrlError if ANY address is internal.
+
+ Resolving and checking every returned address closes the gap where a name
+ resolves to both a public and a private address.
+ """
+ if not host:
+ raise UnsafeUrlError("missing host")
+ try:
+ infos = socket.getaddrinfo(host, None, proto=socket.IPPROTO_TCP)
+ except socket.gaierror as exc:
+ raise UnsafeUrlError(f"could not resolve host: {host}") from exc
+
+ addrs = {info[4][0] for info in infos}
+ if not addrs:
+ raise UnsafeUrlError(f"no addresses for host: {host}")
+ for addr in addrs:
+ ip = ipaddress.ip_address(addr)
+ if _is_blocked_ip(ip):
+ raise UnsafeUrlError(f"host resolves to a disallowed address: {host}")
+
+
+def assert_safe_url(url: str, *, allow_http: bool = False) -> str:
+ """Validate scheme + resolved IPs for a single URL. Returns it unchanged.
+
+ https is required by default; ``allow_http`` exists only for callers that
+ legitimately need cleartext (not used by the workers).
+ """
+ parts = urlsplit(url)
+ scheme = parts.scheme.lower()
+ allowed = ("https", "http") if allow_http else ("https",)
+ if scheme not in allowed:
+ raise UnsafeUrlError(f"scheme not allowed: {scheme or '(none)'}")
+ if not parts.hostname:
+ raise UnsafeUrlError("missing host")
+ _resolve_safe_host(parts.hostname)
+ return url
+
+
+def fetch_remote_bytes(
+ url: str,
+ *,
+ timeout: float = 30.0,
+ max_bytes: int = DEFAULT_MAX_BYTES,
+ allow_http: bool = False,
+ headers: Optional[dict] = None,
+) -> bytes:
+ """Fetch ``url`` safely and return its body bytes.
+
+ Hardening applied:
+ * https-only (unless ``allow_http``).
+ * every URL - including each redirect target - is DNS-resolved and
+ rejected if it maps to a private/loopback/link-local/metadata address.
+ * redirects are followed manually (max ``_MAX_REDIRECTS``) so each hop is
+ re-validated; httpx's own redirect following is disabled.
+ * the response body is streamed and aborted once it exceeds ``max_bytes``.
+ * transient failures (dropped connection, timeout, 429/5xx) are retried
+ with a short backoff; see ``_FETCH_ATTEMPTS``.
+
+ Raises ``UnsafeUrlError`` for disallowed targets and ``httpx.HTTPError`` /
+ ``ValueError`` for transport or size failures.
+ """
+ for attempt in range(_FETCH_ATTEMPTS - 1):
+ try:
+ return _fetch_once(
+ url, timeout=timeout, max_bytes=max_bytes,
+ allow_http=allow_http, headers=headers,
+ )
+ except httpx.HTTPError as exc:
+ if not _is_transient(exc):
+ raise
+ log.warning("transient fetch failure (attempt %d): %s", attempt + 1, type(exc).__name__)
+ time.sleep(_FETCH_BACKOFF_SECS[attempt])
+ return _fetch_once(
+ url, timeout=timeout, max_bytes=max_bytes,
+ allow_http=allow_http, headers=headers,
+ )
+
+
+def _is_transient(exc: httpx.HTTPError) -> bool:
+ if isinstance(exc, httpx.HTTPStatusError):
+ code = exc.response.status_code
+ return code == 429 or code >= 500
+ return isinstance(exc, httpx.TransportError)
+
+
+def _fetch_once(
+ url: str,
+ *,
+ timeout: float,
+ max_bytes: int,
+ allow_http: bool,
+ headers: Optional[dict],
+) -> bytes:
+ current = assert_safe_url(url, allow_http=allow_http)
+ with httpx.Client(follow_redirects=False, timeout=timeout) as client:
+ for _ in range(_MAX_REDIRECTS + 1):
+ with client.stream("GET", current, headers=_fetch_headers(headers)) as resp:
+ if resp.is_redirect:
+ location = resp.headers.get("location")
+ if not location:
+ raise UnsafeUrlError("redirect without Location header")
+ current = assert_safe_url(
+ str(resp.url.join(location)), allow_http=allow_http
+ )
+ continue
+ resp.raise_for_status()
+ declared = resp.headers.get("content-length")
+ if declared is not None and declared.isdigit() and int(declared) > max_bytes:
+ raise ValueError(f"response too large: {declared} bytes")
+ chunks: list[bytes] = []
+ total = 0
+ for chunk in resp.iter_bytes(65536):
+ total += len(chunk)
+ if total > max_bytes:
+ raise ValueError(f"response exceeded {max_bytes} bytes")
+ chunks.append(chunk)
+ return b"".join(chunks)
+ raise UnsafeUrlError(f"too many redirects (>{_MAX_REDIRECTS})")
+
+
+async def fetch_remote_bytes_async(
+ client: "httpx.AsyncClient",
+ url: str,
+ *,
+ max_bytes: int = DEFAULT_MAX_BYTES,
+ allow_http: bool = False,
+ headers: Optional[dict] = None,
+) -> bytes:
+ """Async variant of ``fetch_remote_bytes`` using a caller-provided client.
+
+ The provided ``client`` MUST be constructed with ``follow_redirects=False``
+ so this function can re-validate each redirect hop itself. Transient
+ failures are retried exactly as in the sync variant.
+ """
+ for attempt in range(_FETCH_ATTEMPTS - 1):
+ try:
+ return await _fetch_once_async(
+ client, url, max_bytes=max_bytes,
+ allow_http=allow_http, headers=headers,
+ )
+ except httpx.HTTPError as exc:
+ if not _is_transient(exc):
+ raise
+ log.warning("transient fetch failure (attempt %d): %s", attempt + 1, type(exc).__name__)
+ await asyncio.sleep(_FETCH_BACKOFF_SECS[attempt])
+ return await _fetch_once_async(
+ client, url, max_bytes=max_bytes,
+ allow_http=allow_http, headers=headers,
+ )
+
+
+async def _fetch_once_async(
+ client: "httpx.AsyncClient",
+ url: str,
+ *,
+ max_bytes: int,
+ allow_http: bool,
+ headers: Optional[dict],
+) -> bytes:
+ current = assert_safe_url(url, allow_http=allow_http)
+ for _ in range(_MAX_REDIRECTS + 1):
+ async with client.stream("GET", current, headers=_fetch_headers(headers)) as resp:
+ if resp.is_redirect:
+ location = resp.headers.get("location")
+ if not location:
+ raise UnsafeUrlError("redirect without Location header")
+ current = assert_safe_url(
+ str(resp.url.join(location)), allow_http=allow_http
+ )
+ continue
+ resp.raise_for_status()
+ declared = resp.headers.get("content-length")
+ if declared is not None and declared.isdigit() and int(declared) > max_bytes:
+ raise ValueError(f"response too large: {declared} bytes")
+ chunks: list[bytes] = []
+ total = 0
+ async for chunk in resp.aiter_bytes(65536):
+ total += len(chunk)
+ if total > max_bytes:
+ raise ValueError(f"response exceeded {max_bytes} bytes")
+ chunks.append(chunk)
+ return b"".join(chunks)
+ raise UnsafeUrlError(f"too many redirects (>{_MAX_REDIRECTS})")
+
+
+# ── opaque error handling ───────────────────────────────────────────────────────
+
+
+def safe_error(exc: BaseException, *, context: str = "") -> str:
+ """Log the full traceback server-side and return an opaque client message.
+
+ Never leak ``traceback.format_exc()`` (absolute paths, library versions,
+ internal hostnames, fragments of upstream responses) to callers. A short
+ correlation id is embedded in both the log line and the returned string so
+ operators can join a client-reported failure to the server log.
+ """
+ correlation_id = uuid.uuid4().hex[:12]
+ label = f"{context} " if context else ""
+ log.exception("%serror [correlation_id=%s]", label, correlation_id)
+ return f"internal error (ref {correlation_id})"
diff --git a/workers/rig/README.md b/workers/rig/README.md
index c2c71bfcc..0f7a5338f 100644
--- a/workers/rig/README.md
+++ b/workers/rig/README.md
@@ -33,6 +33,19 @@ its historical name.
powers emotions and lipsync (`src/voice/arkit-blendshapes.js`,
the embodiment stage) on generated avatars.
+### Non-humanoid meshes
+
+Make-It-Animatable is trained on bipedal humanoids. Before dispatching to it,
+`main.py` runs `mesh_classify.classify_mesh` (a cheap geometric heuristic:
+height/footprint aspect ratio plus bilateral mirror symmetry, no model) on
+the input mesh. A mesh it calls non-humanoid is forwarded to
+[`workers/rig-nonhuman`](../rig-nonhuman/) (UniRig + SkinTokens) when
+`RIG_NONHUMAN_URL` is configured; this worker then polls that service's
+`/tasks/:id` on the caller's behalf and surfaces the same `task_id`/poll
+contract regardless of which engine actually rigged the mesh. If
+`RIG_NONHUMAN_URL` is unset (the default), every mesh stays on this
+humanoid-only path, matching behavior before this branch existed.
+
## API
Identical to the previous rig worker so `api/_providers/gcp.js` needs no code
@@ -149,6 +162,9 @@ licenses, no Hunyuan territory carve-outs, in the rigging path.
| `MAX_CONCURRENT` | `1` | parallel rig jobs per instance |
| `TASK_TIMEOUT_S` | `420` | per-task hard timeout |
| `WEIGHTS_ROOT` | `/weights/make-it-animatable` | staged assets, read by `entrypoint.sh` |
+| `RIG_NONHUMAN_URL` | (unset) | `workers/rig-nonhuman` service URL; non-humanoid meshes forward here when set, stay on this worker when unset |
+| `RIG_NONHUMAN_API_KEY` | (secret) | bearer secret for the non-humanoid worker, if different from `API_KEY` |
+| `RIG_NONHUMAN_TIMEOUT_S` | `900` | hard timeout for a forwarded (non-humanoid) task |
`entrypoint.sh` copies the checkpoints and Mixamo templates from `WEIGHTS_ROOT`
(the FUSE-mounted weights bucket) onto local disk before serving, rather than
diff --git a/workers/rig/main.py b/workers/rig/main.py
index 4840fe5c4..1069403fb 100644
--- a/workers/rig/main.py
+++ b/workers/rig/main.py
@@ -8,6 +8,15 @@ original GLB bytes so materials/PBR textures are preserved exactly;
blendshapes.py transfers the ICT-FaceKit (MIT) ARKit-52 expression set onto
the head so emotions and lipsync work out of the box.
+Non-humanoid branch: mesh_classify.py runs a deterministic geometric check
+(tall + bilaterally symmetric, or not) before the GPU is touched. A humanoid
+mesh stays on the Make-It-Animatable path above. Everything else (quadrupeds,
+creatures, props) is forwarded, by GCS url, to the VAST-AI-Research
+SkinTokens/UniRig worker (workers/rig-nonhuman, MIT) and its task is polled
+to completion under this same task_id, so callers see one API contract no
+matter which engine rigged their mesh. If RIG_NONHUMAN_URL is unset, every
+mesh takes the humanoid path unchanged (the pre-existing behavior).
+
API contract (identical to the retired unirig worker, so the platform's
GCP_UNIRIG_URL wiring is a config-only cutover):
POST /rig { mesh_gcs_url: str, template?: str, blendshapes?: bool, job_id?: str }
@@ -34,6 +43,9 @@ Environment variables:
ARKIT_TEMPLATE baked template npz (default /app/assets/arkit_template.npz)
MAX_CONCURRENT parallel rigging jobs (default 1)
TASK_TIMEOUT_S per-task hard timeout in seconds (default 420)
+ RIG_NONHUMAN_URL base url of the SkinTokens/UniRig worker (unset = disabled)
+ RIG_NONHUMAN_API_KEY bearer secret for RIG_NONHUMAN_URL (Secret Manager)
+ RIG_NONHUMAN_TIMEOUT_S hard timeout for the forwarded job (default 900)
"""
from __future__ import annotations
@@ -59,6 +71,7 @@ import engine_mia
import rig_glb
from blendshapes import head_mask_from_weights, load_template, transfer_blendshapes
from gltf_meshopt import decode_if_meshopt
+from mesh_classify import HUMANOID, classify_mesh
from worker_security import (
UnsafeUrlError,
fetch_remote_bytes_async,
@@ -79,6 +92,10 @@ MAX_CONCURRENT = int(os.environ.get("MAX_CONCURRENT", "1"))
TASK_TIMEOUT_S = float(os.environ.get("TASK_TIMEOUT_S", "420"))
# Tasks older than this are pruned from the in-memory store.
TASK_TTL_S = 3600.0
+RIG_NONHUMAN_URL = os.environ.get("RIG_NONHUMAN_URL", "").rstrip("/")
+RIG_NONHUMAN_API_KEY = os.environ.get("RIG_NONHUMAN_API_KEY", "")
+RIG_NONHUMAN_TIMEOUT_S = float(os.environ.get("RIG_NONHUMAN_TIMEOUT_S", "900"))
+_NONHUMAN_POLL_INTERVAL_S = 2.0
_bucket: Optional[storage.Bucket] = None
_sem: Optional[asyncio.Semaphore] = None
@@ -150,44 +167,89 @@ def _rig_sync(mesh_bytes: bytes, want_blendshapes: bool) -> bytes:
)
+async def _forward_to_nonhuman(mesh_gcs_url: str, job_id: str) -> str:
+ """Forward a non-humanoid mesh to the SkinTokens/UniRig worker and block
+ until it finishes, polling the SAME /rig + /tasks/:id contract this
+ worker exposes, so the caller sees one API regardless of which engine
+ rigged their mesh."""
+ headers = {"Authorization": f"Bearer {RIG_NONHUMAN_API_KEY}"}
+ async with httpx.AsyncClient(timeout=60, follow_redirects=False) as client:
+ resp = await client.post(
+ f"{RIG_NONHUMAN_URL}/rig",
+ json={"mesh_gcs_url": mesh_gcs_url, "job_id": job_id},
+ headers=headers,
+ )
+ resp.raise_for_status()
+ sub_task_id = resp.json()["task_id"]
+
+ deadline = time.time() + RIG_NONHUMAN_TIMEOUT_S
+ while time.time() < deadline:
+ await asyncio.sleep(_NONHUMAN_POLL_INTERVAL_S)
+ poll = await client.get(
+ f"{RIG_NONHUMAN_URL}/tasks/{sub_task_id}", headers=headers
+ )
+ poll.raise_for_status()
+ state = poll.json()
+ if state["status"] == "done":
+ return state["rigged_gcs_url"]
+ if state["status"] == "failed":
+ raise RuntimeError(
+ state.get("error") or "non-humanoid rigging failed"
+ )
+ raise RuntimeError(
+ f"non-humanoid rigging timed out after {RIG_NONHUMAN_TIMEOUT_S:.0f}s"
+ )
+
+
async def _run_task(task_id: str, mesh_gcs_url: str, want_blendshapes: bool) -> None:
async with _sem:
_tasks[task_id]["status"] = "running"
loop = asyncio.get_event_loop()
t0 = time.time()
try:
- async def _work() -> str:
- async with httpx.AsyncClient(timeout=60, follow_redirects=False) as client:
- try:
- mesh_bytes = await fetch_remote_bytes_async(client, mesh_gcs_url)
- except UnsafeUrlError as exc:
- raise RuntimeError(f"refused to fetch mesh url: {exc}") from exc
-
- # A meshopt-compressed GLB is transcoded before anything reads
- # it: neither trimesh nor pygltflib can decode that extension,
- # and it is the format most three.ws avatars ship as.
- mesh_bytes, _ = decode_if_meshopt(mesh_bytes, ".glb")
-
- rigged = await loop.run_in_executor(
- None, _rig_sync, mesh_bytes, want_blendshapes
- )
-
- blob_name = f"rigged-meshes/{task_id}.glb"
- blob = _bucket.blob(blob_name)
- # upload_from_string defaults to DEFAULT_RETRY_IF_GENERATION_SPECIFIED,
- # which retries NOTHING when no generation precondition is set, so a
- # transient RemoteDisconnected on the resumable-upload handshake threw
- # away a finished rig. The blob name is a fresh uuid per task, so
- # re-sending can only overwrite this task's own object.
- await loop.run_in_executor(
- None,
- lambda: blob.upload_from_string(
- rigged, content_type="model/gltf-binary", retry=DEFAULT_RETRY
- ),
- )
- return f"https://storage.googleapis.com/{GCS_BUCKET}/{blob_name}"
-
- gcs_url = await asyncio.wait_for(_work(), timeout=TASK_TIMEOUT_S)
+ async with httpx.AsyncClient(timeout=60, follow_redirects=False) as client:
+ try:
+ mesh_bytes = await fetch_remote_bytes_async(client, mesh_gcs_url)
+ except UnsafeUrlError as exc:
+ raise RuntimeError(f"refused to fetch mesh url: {exc}") from exc
+
+ # A meshopt-compressed GLB is transcoded before anything reads
+ # it: neither trimesh nor pygltflib can decode that extension,
+ # and it is the format most three.ws avatars ship as.
+ mesh_bytes, _ = decode_if_meshopt(mesh_bytes, ".glb")
+ rig_glb.validate_input_mesh(mesh_bytes)
+ classification = classify_mesh(rig_glb.load_mesh_vertices(mesh_bytes))
+
+ if classification != HUMANOID and RIG_NONHUMAN_URL:
+ timeout_s = RIG_NONHUMAN_TIMEOUT_S
+ work = _forward_to_nonhuman(mesh_gcs_url, task_id)
+ else:
+ timeout_s = TASK_TIMEOUT_S
+
+ async def _humanoid_work() -> str:
+ rigged = await loop.run_in_executor(
+ None, _rig_sync, mesh_bytes, want_blendshapes
+ )
+ blob_name = f"rigged-meshes/{task_id}.glb"
+ blob = _bucket.blob(blob_name)
+ # upload_from_string defaults to
+ # DEFAULT_RETRY_IF_GENERATION_SPECIFIED, which retries
+ # NOTHING when no generation precondition is set, so a
+ # transient RemoteDisconnected on the resumable-upload
+ # handshake threw away a finished rig. The blob name is a
+ # fresh uuid per task, so re-sending can only overwrite
+ # this task's own object.
+ await loop.run_in_executor(
+ None,
+ lambda: blob.upload_from_string(
+ rigged, content_type="model/gltf-binary", retry=DEFAULT_RETRY
+ ),
+ )
+ return f"https://storage.googleapis.com/{GCS_BUCKET}/{blob_name}"
+
+ work = _humanoid_work()
+
+ gcs_url = await asyncio.wait_for(work, timeout=timeout_s)
elapsed = time.time() - t0
_tasks[task_id].update({
"status": "done",
@@ -198,7 +260,7 @@ async def _run_task(task_id: str, mesh_gcs_url: str, want_blendshapes: bool) ->
except asyncio.TimeoutError:
_tasks[task_id].update({
"status": "failed",
- "error": f"rigging timed out after {TASK_TIMEOUT_S:.0f}s",
+ "error": f"rigging timed out after {timeout_s:.0f}s",
"elapsed_ms": int((time.time() - t0) * 1000),
})
log.error("[%s] rigging timed out", task_id)
@@ -265,4 +327,5 @@ async def health() -> dict:
"gpu_name": torch.cuda.get_device_name(0) if torch.cuda.is_available() else None,
"model_loaded": engine_mia.is_ready(),
"queued": sum(1 for t in _tasks.values() if t.get("status") in ("queued", "running")),
+ "nonhuman_worker_configured": bool(RIG_NONHUMAN_URL),
}
diff --git a/workers/rig/mesh_classify.py b/workers/rig/mesh_classify.py
new file mode 100644
index 000000000..7e0d304d8
--- /dev/null
+++ b/workers/rig/mesh_classify.py
@@ -0,0 +1,77 @@
+"""
+Humanoid vs. non-humanoid mesh classification: a deterministic geometric
+heuristic, not a learned model, so it needs no weights and runs on CPU
+before any GPU engine is touched.
+
+A standing humanoid mesh is tall relative to its footprint (height clearly
+exceeds width and depth) and left-right symmetric about its vertical
+mid-plane (arms and legs mirror). A quadruped, prop, or creature mesh
+typically fails one or both: a wolf's footprint (length) exceeds its height,
+and a chair is often asymmetric. Neither trait alone is reliable, so both
+are required for the humanoid verdict.
+
+Kept dependency-light (numpy / scipy only) so it is unit-tested with
+synthetic point sets; see test_mesh_classify.py. The caller loads the mesh
+(see rig_glb.load_mesh_vertices) and passes only the vertex array in.
+"""
+
+from __future__ import annotations
+
+import numpy as np
+from scipy.spatial import cKDTree
+
+HUMANOID = "humanoid"
+OTHER = "other"
+
+# bbox height / max(width, depth): a standing human clears this easily;
+# a quadruped's footprint (length) typically exceeds its shoulder height.
+_HEIGHT_TO_FOOTPRINT_MIN = 1.3
+# Mean nearest-neighbor distance between the mesh and its own mirror image,
+# normalized by the bbox diagonal. Below this, the mesh mirrors itself
+# closely enough to call it bilaterally symmetric.
+_SYMMETRY_TOLERANCE = 0.06
+# Mirror-distance query is O(V log V); this many vertices is plenty to
+# estimate symmetry and keeps classification well under a second.
+_SAMPLE_CAP = 4000
+
+
+def classify_mesh(vertices: np.ndarray) -> str:
+ """Classify a mesh's vertex positions (V,3), Y-up, as HUMANOID or OTHER.
+
+ Degenerate input (too few vertices, wrong shape) is classified OTHER:
+ the caller's existing humanoid path is the one with more to lose from a
+ false positive (a non-humanoid mesh misrouted through Make-It-Animatable
+ just produces a bad rig the pipeline already handles), so an uncertain
+ classification prefers the branch that cannot misroute a real human.
+ """
+ v = np.asarray(vertices, dtype=np.float64)
+ if v.ndim != 2 or v.shape[1] != 3 or v.shape[0] < 4:
+ return OTHER
+
+ mins, maxs = v.min(axis=0), v.max(axis=0)
+ extent = maxs - mins
+ height, width, depth = float(extent[1]), float(extent[0]), float(extent[2])
+ footprint = max(width, depth, 1e-6)
+
+ if height / footprint < _HEIGHT_TO_FOOTPRINT_MIN:
+ return OTHER
+
+ rng = np.random.default_rng(0)
+ if v.shape[0] > _SAMPLE_CAP:
+ idx = rng.choice(v.shape[0], _SAMPLE_CAP, replace=False)
+ sample = v[idx]
+ else:
+ sample = v
+
+ center_x = (mins[0] + maxs[0]) / 2.0
+ mirrored = sample.copy()
+ mirrored[:, 0] = 2 * center_x - mirrored[:, 0]
+
+ tree = cKDTree(sample)
+ dist, _ = tree.query(mirrored, k=1)
+ diag = float(np.linalg.norm(extent)) or 1.0
+ symmetry_error = float(np.mean(dist)) / diag
+
+ if symmetry_error > _SYMMETRY_TOLERANCE:
+ return OTHER
+ return HUMANOID
diff --git a/workers/rig/rig_glb.py b/workers/rig/rig_glb.py
index 9c8752059..81250873e 100644
--- a/workers/rig/rig_glb.py
+++ b/workers/rig/rig_glb.py
@@ -306,6 +306,14 @@ def validate_input_mesh(mesh_bytes: bytes) -> None:
)
+def load_mesh_vertices(mesh_bytes: bytes) -> np.ndarray:
+ """Load a validated GLB's vertex positions, for mesh classification ahead
+ of the humanoid/non-humanoid routing decision. Parses the same way
+ validate_input_mesh does, so anything that passed validation loads here."""
+ mesh = trimesh.load(io.BytesIO(mesh_bytes), file_type="glb", force="mesh")
+ return np.asarray(mesh.vertices, dtype=np.float64)
+
+
def build_rigged_glb(mesh_bytes, mesh, joints, parents, weights,
blendshape_data=None, joint_names=None):
"""Author a fully rigged GLB from the raw mesh and the model's predictions.
diff --git a/workers/rig/test_mesh_classify.py b/workers/rig/test_mesh_classify.py
new file mode 100644
index 000000000..b4bebeb49
--- /dev/null
+++ b/workers/rig/test_mesh_classify.py
@@ -0,0 +1,105 @@
+"""Unit tests for mesh_classify.classify_mesh: synthetic point clouds, no GPU,
+no network. A standing-humanoid-shaped cloud, a quadruped-shaped cloud, and an
+asymmetric prop should land on opposite sides of the heuristic.
+
+No GPU, no torch, no GCP: `python3 test_mesh_classify.py`.
+"""
+
+from __future__ import annotations
+
+import sys
+
+import numpy as np
+
+from mesh_classify import HUMANOID, OTHER, classify_mesh
+
+
+def _standing_humanoid_cloud(n: int = 2000) -> np.ndarray:
+ """A tall, left-right symmetric capsule-ish cloud: torso + two
+ mirrored arms + two mirrored legs, roughly a standing human's silhouette."""
+ rng = np.random.default_rng(1)
+ torso = rng.normal(0, 1, (n // 2, 3)) * [0.25, 0.9, 0.15]
+ torso[:, 1] += 1.0
+ leg_x = rng.choice([-0.15, 0.15], n // 4)
+ legs = np.stack([
+ leg_x + rng.normal(0, 0.05, n // 4),
+ rng.uniform(-0.9, 0.3, n // 4),
+ rng.normal(0, 0.05, n // 4),
+ ], axis=1)
+ arm_x = rng.choice([-0.4, 0.4], n // 4)
+ arms = np.stack([
+ arm_x + rng.normal(0, 0.05, n // 4),
+ rng.uniform(0.6, 1.1, n // 4),
+ rng.normal(0, 0.05, n // 4),
+ ], axis=1)
+ return np.concatenate([torso, legs, arms], axis=0)
+
+
+def _quadruped_cloud(n: int = 2000) -> np.ndarray:
+ """A long, low, symmetric cloud: body length and leg span both exceed
+ shoulder height, like a wolf or a horse standing on four legs."""
+ rng = np.random.default_rng(2)
+ body = rng.normal(0, 1, (n // 2, 3)) * [1.4, 0.35, 0.3]
+ body[:, 1] += 0.5
+ leg_z = rng.choice([-0.25, 0.25], n // 2)
+ leg_x = rng.uniform(-1.1, 1.1, n // 2)
+ legs = np.stack([
+ leg_x,
+ rng.uniform(0.0, 0.5, n // 2),
+ leg_z + rng.normal(0, 0.04, n // 2),
+ ], axis=1)
+ return np.concatenate([body, legs], axis=0)
+
+
+def _asymmetric_prop_cloud(n: int = 2000) -> np.ndarray:
+ """Tall enough to pass the height test, but an L-bracket: a tall post on
+ one side and a low shelf spanning the footprint, with no mirror-image
+ counterpart on the other side, like a leaning prop or a shelf unit."""
+ rng = np.random.default_rng(3)
+ post = np.stack([
+ np.full(n // 2, 1.0) + rng.normal(0, 0.03, n // 2),
+ rng.uniform(0.3, 3.0, n // 2),
+ rng.normal(0, 0.03, n // 2),
+ ], axis=1)
+ shelf = np.stack([
+ rng.uniform(0.0, 1.2, n // 2),
+ rng.uniform(0.0, 0.3, n // 2),
+ rng.normal(0, 0.03, n // 2),
+ ], axis=1)
+ return np.concatenate([post, shelf], axis=0)
+
+
+def test_standing_humanoid_classifies_humanoid():
+ assert classify_mesh(_standing_humanoid_cloud()) == HUMANOID
+
+
+def test_quadruped_classifies_other():
+ assert classify_mesh(_quadruped_cloud()) == OTHER
+
+
+def test_asymmetric_prop_classifies_other():
+ assert classify_mesh(_asymmetric_prop_cloud()) == OTHER
+
+
+def test_degenerate_input_classifies_other():
+ assert classify_mesh(np.zeros((0, 3))) == OTHER
+ assert classify_mesh(np.zeros((2, 3))) == OTHER
+ assert classify_mesh(np.zeros((10, 2))) == OTHER
+
+
+def test_classification_is_deterministic():
+ cloud = _standing_humanoid_cloud()
+ assert classify_mesh(cloud) == classify_mesh(cloud)
+
+
+if __name__ == "__main__":
+ try:
+ test_standing_humanoid_classifies_humanoid()
+ test_quadruped_classifies_other()
+ test_asymmetric_prop_classifies_other()
+ test_degenerate_input_classifies_other()
+ test_classification_is_deterministic()
+ except AssertionError:
+ print("\nFAILED")
+ sys.exit(1)
+ print("\nALL PASSED")