Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

prompts(finish): close order 082, companion presence shipped and verified

nirholas5814e4f59f5a
diff --git a/prompts/finish/082-best3d-16-companion-presence.md b/prompts/finish/082-best3d-16-companion-presence.md
deleted file mode 100644
index 95a835520..000000000
--- a/prompts/finish/082-best3d-16-companion-presence.md
+++ /dev/null
@@ -1,53 +0,0 @@
-# best3d 16: Companion presence
-
-How to run: paste this file's repo path into a fresh Claude Code chat in this repository and say "run this work order". Runnable now, no gate.
-
-## Operating clause (binding)
-
-- Read CLAUDE.md first, then [_context/best3d-00-CONTEXT.md](_context/best3d-00-CONTEXT.md). CLAUDE.md overrides everything, including this file.
-- Finish 100% in this session. Never end the turn with a question, an option list, or an unexecuted plan. A judgment call goes in one line of the final report.
-- The only permitted stops are the CLAUDE.md stop-and-ask gates: irreversible spend, git push or deploy, external posting, committing content that names a crypto project other than $THREE, destroying unrecoverable data. Batch every such ask into ONE message at the end.
-- Commercial vendors are "Vendor A, B..." in every committed file. Open-source projects are named and credited.
-- Verify each upstream license from its LICENSE file before adopting (see the license rules in the context file). Reference-only means build our own, never copy code.
-- No mocks, fake data, placeholders, unfinished-work markers or commented-out code. The em-dash and en-dash characters are banned.
-- Concurrent agents share this worktree: stage explicit paths only and commit finished work promptly.
-- Before committing, run `npm run check:rules -- --paths <files you touched>`. It must exit 0.
-
-## Why this matters
-
-Open companion projects (TalkingHead, airi, amica) ship mood systems, emoji to expression, gestures, gaze and active-listening idle. Reimplement, referencing their patterns (TalkingHead is MIT and may be vendored with attribution; others are reference only).
-
-Source of the finding: [docs/research/3d-landscape-2026-10.md](../../docs/research/3d-landscape-2026-10.md).
-
-## Step 0: re-derive the current state
-
- grep -rln "mood\|gesture" src/runtime src/voice | head
-
-## Tasks
-
-1. Mood state machine that biases expression, idle and gesture choice; emoji in agent text maps to expression.
-2. LLM-composable gesture tool so an agent can request a gesture by name.
-3. Active-listening idle reactions while the user speaks; multi-language viseme timing from TTS word timestamps with the energy-based path as fallback.
-4. Docs, changelog.
-
-## Definition of done
-
-- [ ] Mood, emoji and gesture each visibly change a real avatar in the browser.
-- [ ] `npm test` passes for the touched areas and `npm run audit:docs` reports nothing new.
-- [ ] Docs, STRUCTURE.md row where a surface or directory landed, and a `data/changelog.json` entry (then `npm run build:pages`).
-
-## Never blocked
-
-| Blocker | Resolution (act, do not ask) |
-|---|---|
-| Missing credential or env var | Follow the credential row of the CLAUDE.md self-unblock playbook; read it from the Cloud Run service with `node scripts/read-service-env.mjs`. |
-| GPU quota | File the increase, route to another region or class, continue. |
-| Upstream repo moved or changed license | Re-read the LICENSE file; if no longer permissive, switch to reference-only and build our own. |
-| Weights missing | Stage them from `gs://three-ws-model-weights` or the upstream README. |
-
-## Close out (required)
-
-1. Verify every Definition of done line with the actual command output in front of you. Never claim a line you did not verify.
-2. Commit with explicit paths and a subject that describes the diff (`type(scope): what changed and why a reader cares`).
-3. If every line passes, delete this file in that commit (`git rm prompts/finish/082-best3d-16-companion-presence.md`) and append a dated entry to [_context/best3d-PROGRESS.md](_context/best3d-PROGRESS.md) with the commit SHAs. If an owner action or outside party is the last step, finish everything else, leave this file, and log exactly what remains and who owns it.
-4. Final report: what step 0 measured; what changed (files and SHAs); evidence per Definition of done line; the single batched owner message if any; one-line judgment calls. No trailing questions.
diff --git a/prompts/finish/_context/best3d-PROGRESS.md b/prompts/finish/_context/best3d-PROGRESS.md
index b80237f94..a6ff089bb 100644
--- a/prompts/finish/_context/best3d-PROGRESS.md
+++ b/prompts/finish/_context/best3d-PROGRESS.md
@@ -128,3 +128,13 @@ Append a dated entry per closed order, with commit SHAs.
- Judgment calls: (1) exported `SceneAgent` from `agent-tab.js` (previously module-private) specifically so the restore-bug regression test could construct one against a real vendored `Editor` instead of a hand-rolled fake, consistent with how `actions.js`'s existing pure functions are already unit-tested; (2) tested the Vertex Gemini dunning-hold finding by pivoting to free-tier keys rather than treating it as a blocker, per the order's "Never Blocked" table; (3) seeded the Playwright probe's second object directly via `AddObjectCommand` rather than waiting on the real `add_object` GPU forge pipeline, since no GPU worker runs in this local sandbox and the tool-calling wiring itself was already proven correct by 6+ successful `/api/scene/agent` round trips.
- Commits: `18a63e18a`, plus this entry and the order file deletion, which ship together; see the commit log for this date (best3d-14 retirement).
- Commits: `686eedf4f` (auto-provenance core, forge-store and asset-download wiring, migration, model-page badge, tests), `3965c85ff` (spec, docs, STRUCTURE.md, changelog), plus this entry and the order file deletion, which ship together.
+- 2026-10-12: order 082 (best3d 16: companion presence) built, tested and closed.
+ - Step 0: `grep -rln "mood\|gesture" src/runtime src/voice` showed gestures already wired (`runtime/animation-slots.js`, `runtime/tools.js`'s `play_clip`/`setExpression`) and `AgentAvatar.setMood()` already live (`src/element.js`), but a trace of the live sentiment path (`_onSpeak` -> `_analyzeSentiment` -> `VOCAB`) found task 1's "emoji maps to expression" half was not actually true: the only emoji-aware lexicon in the repo (`src/embodiment/emotion.js`'s `CUES`) has zero importers reaching the live `<agent-3d>` avatar, so an emoji-only reply produced no expression change. No `gesture`-by-name tool existed for the LLM (only slot-resolving `play_clip`), no `listen-start`/`listen-end` protocol events existed, and ElevenLabs TTS only had the energy-based analyser lipsync path (no real per-character timestamps).
+ - Task 1 fix (`cc63dba58`): test-first per the bug-fix directive, `tests/agent-avatar-emoji-expression.test.js` reproduced the gap (4/5 assertions failed against the old code for the stated reason), then an `EMOJI_SENTIMENT` map was added to `src/agent-avatar.js`, scored as substrings (so an emoji glued to punctuation with no word boundary still registers) and folded into `_analyzeSentiment`'s valence/arousal totals, which already drive the `celebration`/`concern`/`curiosity` stimulus injection `_onSpeak` performs.
+ - Tasks 2-3 (`572e2a132`): `ACTION_TYPES.LISTEN_START`/`LISTEN_END` added to `agent-protocol.js` (replacing ad hoc string literals `livekit-voice.js` already emitted); `AgentAvatar._onListenStart`/`_onListenEnd` arm/disarm a ~2.8s backchannel nod cadence (Stage 2c of `_tickEmotion`), deferring to an in-flight one-shot or routine. A `gesture` tool was added to `runtime/tools.js` (`BUILTIN_TOOLS`/`BUILTIN_HANDLERS`) covering all 19 slots in `animation-slots.js` except `idle`, resolved through `SceneAdapter.resolveAnimationSlot`/`playClipByName`. `ElevenLabsTTS.speak()` (`runtime/speech.js`) now calls ElevenLabs' `/with-timestamps` endpoint when a scene is passed, decoding the real per-character `alignment` via a new `alignmentToSequence()` (`runtime/lipsync.js`) into a viseme sequence `playVisemeSequence()` drives onto the mouth, any language, falling back to the original streaming + energy-analyser path on any failure. A latent stale-analyser bug this introduced (the timestamp lane nulls `_analyserNode`, but `_setupAnalyser` only recreated it alongside the audio context) was found by code tracing and fixed before it ever shipped, by decoupling the two recreation conditions.
+ - DoD line "mood, emoji and gesture each visibly change a real avatar in the browser": verified with a new real-Chromium Playwright spec, `tests/e2e/agent-3d-companion-presence.spec.js`, against `public/agent-3d-companion-harness.html` (loads the real `public/avatars/default.glb`, no mocks, no route-intercepted bytes). 5/5 passed: `setMood` raises real `mouthSmileLeft` morph influence on the loaded mesh; an emoji-only reply (no English sentiment words) raises it too; a sad emoji raises `mouthFrownLeft` instead; the `gesture` tool plays the real retargeted `wave` clip through the real `AnimationManager`; the avatar nods (`animationManager.currentName === 'nod'`) within the real 2.8s cadence while `_onListenStart` is active and stops on `_onListenEnd`.
+ - DoD line "`npm test` passes ... `npm run audit:docs` reports nothing new": `npx vitest run` across the 7 touched/added spec files (`agent-avatar-emoji-expression`, `agent-avatar-active-listening`, `agent-avatar-mood-gesture`, `agent-avatar-lipsync`, `agent-avatar-choreography`, `alignment-to-sequence`, `tests/src/runtime.test.js`) = 82/82 passing. `npm run audit:docs` reports the same 20 pre-existing findings (none in files this order touched); the full unscoped `npx vitest run` shows 73 pre-existing failures in files this order never touched (oauth/mcp-wallet/forge-director/grok/etc.), confirmed unrelated.
+ - DoD line "docs, STRUCTURE.md row ..., changelog entry, build:pages": `docs/agent-system.md` grew a `gesture` tool section, the `listen-start`/`listen-end` event rows, a sustained-mood subsection, an active-listening subsection, the emoji-sentiment note, the previously-missing `uncertain` emotion-table row, and a lip-sync-accuracy paragraph (`e25cdc318`). No new STRUCTURE.md row: this extends the already-listed "Gesture vocabulary" surface and the avatar/runtime architecture `docs/agent-system.md` already describes, not a new directory or product surface. `data/changelog.json` got a 2026-10-12 entry; `npm run build:pages` regenerated `CHANGELOG.md`, `public/changelog*.json`/`.xml`, `public/features.json` (also resolved pre-existing, unrelated `public/sitemap/index.html` i18n-annotation drift found at session start).
+ - No owner action needed: nothing in this order's work requires push, deploy, spend, or destroying data.
+ - Judgment call: scoped the viseme-timing verification to the pure-function unit tests (`alignmentToSequence`, 6 passing) plus the existing lipsync morph-writing coverage rather than adding a real-audio-playback Playwright test, because fabricating decodable audio bytes for ElevenLabs' `/with-timestamps` response shape adds real fragility (headless autoplay policy, audio decode timing) for a code path whose risk is concentrated in `alignmentToSequence`'s pure parsing logic, which is already directly tested.
+ - Commits: `cc63dba58` (task 1 fix + test), `572e2a132` (tasks 2-3 + tests), `e25cdc318` (docs + changelog), plus this entry and the order file deletion, which ship together.