Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
Likeness evaluation
Does the avatar look like the person in the photos? This is the harness that answers it with a number instead of an opinion.
Phase 1 of the roadmap is verified on a likeness score: users complete a capture and mint an agent of themselves "with >=4/5 likeness score". Until this landed, nothing produced that number for a real generation. The platform could assert likeness; it could not measure it.
What was already measured, and why it was not enough
Two quality instruments existed before this one, and neither answers the identity question:
| Instrument | Asks | Blind to |
|---|---|---|
Realism quality bench (api/_lib/quality-bench.js) |
Is the generated asset photorealistic, coherent, on-brief? | Whose face it is. A beautiful render of a stranger scores perfectly. |
Identity Shape Error (eval/identity_eval.py) |
Does the head have this person's face shape? | Texture, by design. Offline, against a synthetic reference set, and stores nothing per generation. |
The gap between them is the whole product claim. A pipeline that paints a selfie onto a template head can score well on realism and poorly on shape while still reading, to the person looking at it, as them or as not them. That perception is what a face-recognition embedding measures.
The metric
For one finished reconstruction:
- The avatar is rendered head-framed at three yaws: head-on (0°),
three-quarter (35°), and profile (65°), using the platform's own
headshotscene preset so the framing matches what a user actually sees on a thumbnail or share card. - Each render and each input capture is passed through a face detector (YuNet) and an identity embedder (SFace), producing a 128-dimensional L2-normalised vector per face.
- Each view is scored as the cosine similarity between its render vector and the best-matching capture vector.
- The headline likeness score is the head-on view's cosine, mapped to 1-5.
profile is 65° rather than a true 90° on purpose. A full side-on view presents
at most three of the five landmarks the alignment step needs, so no model of
this family can measure it. 65° is the steepest turn that still yields a
measurable face, and it is well past the point where a frontal-only texture
transfer stops covering the head.
Why 1-5 means what it means
The mapping is anchored to the embedding model's own calibration, not to taste:
| Cosine | Score | Meaning |
|---|---|---|
| 0.000 | 1.0 | No relationship between the faces |
| 0.363 | 3.0 | OpenCV's published SFace decision boundary: the exact point where the model stops calling this the same person |
| 0.682 | 4.0 | The roadmap gate |
| 1.000 | 5.0 | Identical embedding |
So 4/5 is a cosine of 0.68, roughly twice the same-identity threshold: not "arguably the same person" but "confidently the same person". Between the anchors the mapping is linear.
What the harness refuses to do
A missing input is reported as a status, never scored as a zero, because these are three different findings that a single low number would hide:
captures_unusable: no input photo contained a findable face. A capture-quality problem; nothing can be concluded about the avatar.render_unusable: no render contained a findable face. The reconstruction did not produce something recognisable as a head.okwith a low score: the avatar has a face, and it is not this person's.
budget_exhausted is a fourth: a sweep cut short by its time budget covers
fewer views than a complete one, so its score is withheld rather than compared
against runs that were not truncated.
Secondary readings
- turn falloff: head-on cosine minus the worst view's. A pipeline that only holds up frontally is visible here and in no single-view number.
- capture cohesion: mean pairwise cosine among the input photos. A selfie set whose own photos only agree at 0.5 cannot support a claim about the avatar at 0.6.
- captures embedded / total: how much of the input the measurement rests on.
The models, and why these two
Both are pulled from OpenCV Zoo at a pinned commit and verified against a hardcoded SHA-256 before they are loaded.
| Role | Model | License | Size |
|---|---|---|---|
| Detection + 5 landmarks | YuNet (face_detection_yunet_2023mar) |
MIT | 233 KB |
| Identity embedding | SFace (face_recognition_sface_2021dec) |
Apache-2.0 | 39 MB |
Both licenses are commercially clean, which is the same bar the reconstruction
worker holds ("no non-commercial 3DMM": no FLAME, BFM or SMPL weights anywhere
in the pipeline). The widely-cited alternative, InsightFace's buffalo_l
ArcFace pack, is materially stronger on benchmarks and is released for
non-commercial research use only, so it is not available to us at any
quality.
They run on onnxruntime-web's
WASM backend (MIT) rather than onnxruntime-node. The node binding's installer
downloads CUDA provider packages from NuGet during npm install and fails
outright on a machine with no GPU tree, which would make the repository
uninstallable. WASM has no native install step and behaves identically on a
laptop and on Cloud Run.
Weights are not vendored into git. They are fetched once from the pinned
URL, hash-verified, and cached under FACE_MODEL_CACHE_DIR (default: the system
temp directory).
opencv_zoo stores its weights in Git LFS, so
raw.githubusercontent.comserves a 130-byte pointer file rather than the model. The fetch usesgithub.com/<repo>/raw/<commit>/…, which resolves LFS. The SHA-256 check is what caught the difference.
Running it
Weekly, automatically
/api/cron/likeness-eval runs Mondays at 03:47 UTC, 24 minutes after the
realism bench, on the same cadence. It takes finished reconstructions the
current scorer version has not measured, scores them, and files the results. It
bounds itself to a 280-second budget and skips any subject it cannot finish, so
it never collects a Cloud Run 504 the way an unbounded sweep does. A sweep whose
mean falls below the 4.0 gate logs at warn level:
gcloud logging read 'resource.type="cloud_run_revision" textPayload:"likeness-eval"' --freshness=7d
On demand
# Score everything the current scorer version has not seen (needs DATABASE_URL)
node --env-file=.env scripts/likeness-eval.mjs --backfill
# Run 10 reconstructions through the real production pipeline and score them
node --env-file=.env scripts/likeness-eval.mjs --live=10 --out=reports/likeness.json
--live signs in with the QA account and, for each subject, renders a headshot
of a distinct avatar from the public library, submits that portrait as the
reconstruction photo, polls the job to completion, and scores the finished
avatar against the exact bytes it was built from. The subjects are therefore
synthesised, never real people's photos, for the same reason the
reconstruction worker synthesises its reference set.
A live subject is a real generation record, because live mode submits a real
reconstruction. Not filing it would not keep it out of the numbers, only defer
them: the next sweep would find an unscored creation and count it toward the
gate rate like any user's avatar. So it is filed deliberately, under the status
benchmark, which the distribution's status = 'ok' filter excludes from every
aggregate while the outcome breakdown and the recent table still show it. The
work queue keys on a row existing at all rather than on its status, so the
subject is never re-picked either. Backfill subjects keep their real status.
Both modes print a cross-subject control: every avatar is also scored against every other subject's captures. A likeness number is unfalsifiable without it, because a scorer that returned 0.8 for any two faces would look identical to a working one on the matched pairs alone. The separation between the two distributions is the evidence the instrument works.
The first measurement (2026-08-14, threews.likeness.sface.v1)
The eleven reconstructions that existed in production when the harness landed, scored in one backfill:
| Subjects | 11 (10 scored, 1 no_captures) |
| Likeness score | min 1.51, mean 2.19, max 3.04 |
| At or above the 4/5 gate | 0 of 10 |
| Recognised as the same person (cosine >= 0.363) | 2 of 10 |
| Matched cosine | min 0.093, median 0.218, mean 0.217, max 0.376 |
| Cross-subject control | min -0.024, median 0.176, mean 0.181, max 0.549 |
| Separation (matched mean minus control mean) | 0.035 |
| Views that produced no detectable face | 3 of 30, all at 65 degrees |
| Cost | ~80 s per subject |
Read the separation line first. A matched pair (an avatar against its own source photo) scores barely above a mismatched one, and the control's best mismatch (0.549) beats every matched pair in the set. Some reconstructions resemble a different subject's photo more than their own.
That is a finding about the pipeline, not about the instrument. The embedder separates faces strongly when the identity is actually present: scored across a 35 degree turn on eleven studio-composed avatars, the same avatar matched itself at mean cosine 0.776 while different avatars sat at 0.443, a separation of 0.33 rather than 0.035. The production reconstruction lane transfers a face texture onto a fixed template head, and by its own measurement only about 10% of the head's texels come from the photograph (avatar fidelity program). A near-noise-floor identity reading is the number that shape of pipeline should produce, and it is consistent with the geometry-side ISE result.
This is the baseline Phase 1 has to move. It is quoted here so the next run has
something to be compared against, and it is only comparable within
threews.likeness.sface.v1.
The board reads one subject higher than this table, and that discrepancy is what
produced the benchmark status described above. Verifying --live submitted a
real reconstruction, which registered a generation record like any other, and
the next backfill scored it at 2.32 as if a user had made it. Live subjects are
now filed as benchmark so they cannot reach the gate rate; the one that landed
before the fix is left where it is rather than quietly edited out of a published
baseline. The table above stays the dated snapshot it was.
Reading the results
/likeness-bench (internal, admin or x-ops-secret) renders the distribution:
the fraction clearing the 4/5 gate, the histogram, mean turn falloff, the
outcome breakdown, and the most recent individual scores. The same data is
available as JSON:
curl -H "x-ops-secret: $OPS_SECRET" 'https://three.ws/api/likeness-bench?days=30&limit=50'
Scores are keyed on the generation record (forge_creations.id) and stamped
with scorer_version. A score is only comparable to another score taken by
the same scorer version. When the model, the alignment template or the view
angles change, the version changes with them, old rows stay valid, and
where scorer_version <> $current is the backfill query.
What is deliberately not stored
No capture URLs, no crops, and no embeddings. A face embedding is biometric data; this table holds the measurement, not the person. The stored report carries per-view statuses and cosines, and records a rejected capture by index and reason only.
Related
- Selfie to avatar: the capture flow being measured
- Avatar fidelity program: the tracks this metric scores
- Realism quality bench: the sibling harness for photorealism
- Reconstruction fidelity evaluation: the geometry-side metric