Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

feat(home-scene): stand the visitor's own agent in their house

nirholasea3345a13194
diff --git a/docs/home-scene.md b/docs/home-scene.md
index 2b0f59a06..f0dff496e 100644
--- a/docs/home-scene.md
+++ b/docs/home-scene.md
@@ -62,6 +62,26 @@ climate and security rollups a real room gets, and the page offers a link
straight to Home Assistant's own Areas screen. The house is fully usable before
anyone has filed a thing.
+### Whose body stands in the house
+
+Yours. The figure in the scene is your own agent, wearing the same avatar the
+[walk world](../STRUCTURE.md), `/play` and the
+[voice satellite](home-satellite.md) show, resolved from the one canonical
+"my agent" record the whole platform reads. Switch agents in another tab and the
+body in the house changes with it, without a reload.
+
+It is resolved after the first frame rather than before it, so the house never
+waits on that round trip, and three ordinary situations end at the platform's
+default body instead: you have not made an agent yet, your agent has no avatar
+yet, or your avatar is private, which by design publishes no model URL for any
+page to read. A model that fails to load at all leaves a lit capsule standing in
+the room. The house is fully usable in every one of those cases; it just loses
+the face.
+
+The body is driven by the canonical clip library, so any rig that animates
+elsewhere on three.ws animates here, and one that cannot be skeleton-driven
+falls back rather than freezing in a T-pose.
+
## Reactivity
The page holds one Server-Sent Events subscription to
diff --git a/src/home/scene-render.js b/src/home/scene-render.js
index af4193602..1d2825eaa 100644
--- a/src/home/scene-render.js
+++ b/src/home/scene-render.js
@@ -201,6 +201,9 @@ export function createHomeScene(container, options = {}) {
const spare = { color: new Color(), target: new Color(), v: new Vector3(), toward: new Vector3(), forward: new Vector3() };
let model = null;
+ // The body the agent wears. Null until the visitor's own agent resolves, and
+ // the default body stands in meanwhile rather than nobody standing there.
+ let avatarUrl = options.avatarUrl || null;
let stale = false;
let staleAmount = 0;
let acting = null;
@@ -859,7 +862,7 @@ export function createHomeScene(container, options = {}) {
function placeAgent(stand) {
if (!stand) return;
if (!agent) {
- agent = createAgentBody(scene, renderer);
+ agent = createAgentBody(scene, renderer, avatarUrl);
}
agent.moveTo(stand.x, stand.y, stand.z, stand.facing);
}
@@ -937,6 +940,17 @@ export function createHomeScene(container, options = {}) {
stale = Boolean(next);
},
/** Highlight the room an action is happening in, for a short beat. */
+ /**
+ * Put the visitor's own agent in the house, or swap it for another one.
+ * Resolving that record is a round trip the house must never wait on, so
+ * it arrives here after the first frame rather than gating it.
+ */
+ setAvatarUrl(next) {
+ const url = next || null;
+ if (url === avatarUrl) return;
+ avatarUrl = url;
+ agent?.setAvatarUrl(url);
+ },
setActing(next, holdMs = 2600) {
acting = next;
actingUntil = performance.now() + holdMs;
@@ -974,6 +988,14 @@ export function createHomeScene(container, options = {}) {
drawCalls: renderer.info.render.calls,
geometries: renderer.info.memory.geometries,
textures: renderer.info.memory.textures,
+ // Reported, not inferred. `prefers-reduced-motion` turns off both
+ // the camera's coasting and the damping that walks a light to its
+ // new brightness, and neither is visible from outside the closure:
+ // a test could only watch pixels and guess. These two are the
+ // setting as the renderer actually applied it, which is the thing
+ // worth asserting, and they cost nothing to read.
+ reduceMotion,
+ damping: controls.enableDamping,
};
},
dispose() {
@@ -994,20 +1016,32 @@ export function createHomeScene(container, options = {}) {
};
}
+/** The body every house falls back to when the visitor has no avatar of their own. */
+const DEFAULT_BODY = '/avatars/default.glb';
+
/**
* The agent's body, standing in the house.
*
- * The platform's default rigged body driven by the canonical clip library, the
- * same pair `/walk`, `/play` and Docs World use, so a rig that animates
+ * It is the VISITOR'S OWN agent whenever they have one: the same avatar the
+ * walk world, `/play` and the voice satellite show, resolved through the
+ * canonical active-agent record and swapped live when they switch agents in
+ * another tab. Somebody who has not made one yet, or whose avatar is private
+ * and therefore has no readable model URL, gets the platform's default body
+ * rather than an empty room.
+ *
+ * Either way it is driven by the canonical clip library, so a rig that animates
* anywhere on three.ws animates here. A rig that cannot be skeleton-driven, or
* a load that fails outright, falls back to a lit capsule: the house stays
* fully usable, it just loses the face.
*/
-function createAgentBody(scene, renderer) {
+function createAgentBody(scene, renderer, avatarUrl = null) {
const root = new Group();
scene.add(root);
let anim = null;
+ let body = null;
let disposed = false;
+ /** Guards against a slow first avatar landing after a faster second one. */
+ let loadToken = 0;
const target = new Vector3();
let bob = 0;
let gesture = 0;
@@ -1024,34 +1058,76 @@ function createAgentBody(scene, renderer) {
placeholder.position.y = 0.75;
root.add(placeholder);
- (async () => {
+ /** Drop the body currently standing here, with its GPU resources. */
+ function clearBody() {
+ anim?.dispose?.();
+ anim = null;
+ if (!body) return;
+ root.remove(body);
+ body.traverse((node) => {
+ if (!node.isMesh) return;
+ node.geometry?.dispose?.();
+ for (const material of Array.isArray(node.material) ? node.material : [node.material]) {
+ if (!material) continue;
+ for (const value of Object.values(material)) {
+ if (value?.isTexture) value.dispose();
+ }
+ material.dispose?.();
+ }
+ });
+ body = null;
+ }
+
+ /**
+ * Load one avatar. Falls back to the platform body once when a personal one
+ * cannot be fetched or decoded, which is the ordinary case for an avatar
+ * whose owner made it private after pointing a wall display at this house.
+ */
+ async function load(url) {
+ const token = (loadToken += 1);
+ const wanted = url || DEFAULT_BODY;
try {
const [{ gltfLoader }, { AnimationManager }] = await Promise.all([
import('../loaders/gltf.js'),
import('../animation-manager.js'),
]);
- const gltf = await new Promise((resolve, reject) => gltfLoader(renderer).load('/avatars/default.glb', resolve, undefined, reject));
- if (disposed) return;
+ const gltf = await new Promise((resolve, reject) => gltfLoader(renderer).load(wanted, resolve, undefined, reject));
+ if (disposed || token !== loadToken) return;
+ clearBody();
const model = gltf.scene;
model.traverse((node) => {
if (node.isMesh) node.frustumCulled = false;
});
root.add(model);
+ body = model;
placeholder.visible = false;
anim = new AnimationManager();
- anim.attach(model, { avatarUrl: '/avatars/default.glb' });
+ anim.attach(model, { avatarUrl: wanted });
anim.relaxUndrivenArms?.();
const defs = await fetch('/animations/manifest.json').then((r) => (r.ok ? r.json() : []));
+ if (disposed || token !== loadToken) return;
if (Array.isArray(defs) && defs.length && anim.supportsCanonicalClips?.() !== false) {
anim.setAnimationDefs(defs);
if (await anim.ensureLoaded('idle')) await anim.crossfadeTo('idle', 0);
}
} catch {
- // The capsule is already standing; the house does not need the body.
+ if (disposed || token !== loadToken) return;
+ // One retry, on the body we ship and serve ourselves. Beyond that the
+ // capsule is already standing and the house does not need the body.
+ if (wanted !== DEFAULT_BODY) await load(DEFAULT_BODY);
}
- })();
+ }
+
+ load(avatarUrl);
return {
+ /**
+ * Show a different avatar without rebuilding the house. Called when the
+ * visitor switches agents, in this tab or another one.
+ */
+ setAvatarUrl(next) {
+ load(next);
+ },
moveTo(x, y, z, facing) {
target.set(x, y, z);
root.position.copy(target);
@@ -1074,7 +1150,7 @@ function createAgentBody(scene, renderer) {
},
dispose() {
disposed = true;
- anim?.dispose?.();
+ clearBody();
scene.remove(root);
placeholder?.geometry.dispose();
placeholder?.material.dispose();
diff --git a/src/home/scene.js b/src/home/scene.js
index 8b18baa81..54da0e73d 100644
--- a/src/home/scene.js
+++ b/src/home/scene.js
@@ -15,7 +15,7 @@
import { HomeApiError, callService, getHome, getLayout, grantEntity, openStream } from './api.js';
import { buildSceneModel } from './scene-model.js';
import { createHomeFallback, webglAvailable } from './scene-fallback.js';
-import { formatNumber, locale, plural, t } from './i18n-home.js';
+import { formatNumber, locale, plural, relativeAge, t } from './i18n-home.js';
const VIEW_KEY = 'three:home:view';
/** A device that cannot hold this for a few seconds is sent to the 2D house. */
@@ -69,6 +69,12 @@ const state = {
// first painted frame that carries it. Reported on window for the
// measurement pass and for the e2e spec.
latency: { last: null, samples: [] },
+ /**
+ * The GLB the agent standing in the house wears: the visitor's own agent,
+ * resolved after the first frame. Null until it lands, and null forever for
+ * somebody who has not made one, in which case the default body stands there.
+ */
+ avatarUrl: null,
overlay: null,
fpsSince: 0,
// True once a person picked a view, by clicking the toggle or by asking for
@@ -132,10 +138,41 @@ async function boot() {
el.reconnect.addEventListener('click', () => reconnect());
document.addEventListener('keydown', onKeydown);
holdScreenAwake();
+ watchAgentBody();
await load();
}
+/**
+ * Whose body stands in the house.
+ *
+ * Every other three.ws surface that draws the visitor's agent (the walk world,
+ * `/play`, the voice satellite) reads one canonical record, and the house has
+ * to agree with them: a person who has given their agent a face should see THAT
+ * face in their kitchen, not a stranger's. Resolved off the critical path and
+ * applied whenever it lands, so the house never waits on it, and re-applied
+ * when they switch agents, including from another tab.
+ *
+ * Nothing here is fatal. A signed-out visitor, an agent with no avatar, and an
+ * avatar its owner made private (which leaves no readable model URL by design)
+ * all end at the same place: the platform's default body, already standing.
+ */
+async function watchAgentBody() {
+ try {
+ const { getActiveAgent, onActiveAgentChange } = await import('../agents/active-agent.js');
+ const apply = (agent) => {
+ const url = agent?.avatar_model_url || null;
+ if (url === state.avatarUrl) return;
+ state.avatarUrl = url;
+ state.renderer?.setAvatarUrl?.(url);
+ };
+ onActiveAgentChange(apply);
+ apply(await getActiveAgent());
+ } catch {
+ // The default body is already in the room.
+ }
+}
+
/**
* The decision, and why it is this one.
*
@@ -481,6 +518,7 @@ async function mount3d() {
const { createHomeScene } = await import('./scene-render.js');
if (state.view !== '3d') return;
state.renderer = createHomeScene(el.stage, {
+ avatarUrl: state.avatarUrl,
onSelect: (entityId, object) => {
state.selected = entityId ? { entityId, object } : null;
for (const button of el.rooms.querySelectorAll('.hs-room-device')) {
@@ -1211,23 +1249,6 @@ function renderAge() {
}
}
-/**
- * "2 minutes ago", in the reader's language, through Intl.RelativeTimeFormat.
- *
- * Not three catalog keys with an English `s` suffix bolted on: that spelling of
- * a plural is wrong in most of the 84 locales this ships in, and several need
- * forms English has no word for. The platform already knows all of them.
- */
-function relativeAge(ms) {
- const s = Math.max(0, Math.round(ms / 1000));
- const [value, unit] = s < 60 ? [s, 'second'] : s < 3600 ? [Math.round(s / 60), 'minute'] : [Math.round(s / 3600), 'hour'];
- try {
- return new Intl.RelativeTimeFormat(locale(), { numeric: 'always' }).format(-value, unit);
- } catch {
- return `${value} ${unit}${value === 1 ? '' : 's'} ago`;
- }
-}
-
function pushLog(entry) {
state.log.unshift({ ...entry, at: new Date() });
// Bounded on purpose: a wall display left open overnight must not grow one