Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
JavaScript 3d-generationai-agentsanimationavatardigital-humangenerative-aiglbgltfimage-to-3dllmmcpmcp-servermodel-context-protocolsolanatext-to-3dthreejsweb-componentswebglwebxrx402
docs(structure): map the floorplan editor and its layout API onto the 3D home row
nirholas934ae7aea097
diff --git a/STRUCTURE.md b/STRUCTURE.mdindex 77bac9604..aef18a259 100644--- a/STRUCTURE.md+++ b/STRUCTURE.md@@ -167,7 +167,7 @@ bottom.| ASL recognition (webcam signing → text) | worker [workers/model-asl-recognition/](workers/model-asl-recognition), API [api/asl-recognition.js](api/asl-recognition.js) (`GCP_ASL_RECOGNITION_URL`) | Live | The reverse direction of [sign speech](docs/sign-language.md): the browser extracts MediaPipe landmarks from the webcam (video never leaves the device) and the worker transcribes continuous ASL fingerspelling to text on CPU via LiteRT, with a decode-confidence score and an LLM cleanup pass on the API. Model: Kaggle 2023 ASLFR 1st-place (Apache-2.0 code + weights, FSboard CC BY 4.0 corpus, the license-clean chain most ASL models lack). Cloud Run gen2 + `patchelf --clear-execstack` on the LiteRT .so. Next: word-level signs (PopSign retrain). Tests: [test_decode.py](workers/model-asl-recognition/test_decode.py) (schema + decode, no weights) and [test_model_smoke.py](workers/model-asl-recognition/test_model_smoke.py) (real weights end to end, run inside the built image). || Smart home: the connect flow and household management | Page [pages/smart-home.html](pages/smart-home.html) `-> /smart-home` + controller [src/home/connect.js](src/home/connect.js), manage view [src/home/manage.js](src/home/manage.js), styles [src/home/home.css](src/home/home.css). Consumes [api/home/index.js](api/home/index.js) (`GET`/`POST /api/home`), [api/home/[id].js](api/home/[id].js), `/api/home/:id/grants` and `/api/home/:id/log`. Reachability rules shared with the server through the [@three-ws/home-bridge](packages/home-bridge) `./url` subpath export | Live | The front door of the Home lane: how a stranger gets from "I have a Home Assistant somewhere" to "my agent is connected", and every way that goes wrong. Twelve designed states rather than a happy path plus an error toast: signed out, empty, inline validation, the private-host refusal, verifying, connected, a rejected token, unreachable, degraded, disconnected, many homes, and the plan ceiling. Two rules hold the surface together. **The token goes to the server once and never comes back**: the field is `type=password` with a reveal toggle, it is never written to storage, never put in a URL, never logged and never echoed by the API, and it is gone from the DOM the moment a connect succeeds. **Reachability is decided in the browser, before the network**: a `192.168.x.x` or `.local` address is refused with zero network calls using the same `normalizeBaseUrl`/`isPrivateHost` the server validates with, so the two can never disagree and nobody waits out a timeout to be told something we knew at the first keystroke. Every string that came from a house (a label, an entity id, an area name) is rendered as `textContent`, because those are attacker-influenceable and there is a physical actuator on the other end. Docs: [docs/smart-home.md](docs/smart-home.md). Tests: [tests/e2e/home-connect.spec.js](tests/e2e/home-connect.spec.js) || Smart home: plans, entitlements and quotas (`/smart-home/plan`) | Page [pages/smart-home-plan.html](pages/smart-home-plan.html) `-> /smart-home/plan` + controller [src/home/plan.js](src/home/plan.js), styles [src/home/plan.css](src/home/plan.css). Resolver [api/_lib/home/entitlements.js](api/_lib/home/entitlements.js), counters [api/_lib/home/usage.js](api/_lib/home/usage.js), endpoint [api/home/plan.js](api/home/plan.js). Enforced at [api/home/index.js](api/home/index.js) (acquisition), [api/home/[id]/stream.js](api/home/[id]/stream.js) (streams), [api/home/[id]/members.js](api/home/[id]/members.js) (seats), [api/home/[id]/call.js](api/home/[id]/call.js) (paused homes), [api/_lib/home/privacy.js](api/_lib/home/privacy.js) (retention ceiling). Cost measurement [scripts/measure-home-entitlement-cost.mjs](scripts/measure-home-entitlement-cost.mjs). Migration `api/_lib/migrations/20260903160000_home_plan_overrides.sql` | Live | What a plan covers for connected houses, and the two things no plan may ever refuse. **A limit never blocks a safety action:** over quota, downgraded or paused, locking a door, closing a garage or valve and arming an alarm always execute, checked before any counter is read and classified from the domain and service alone so the exemption never depends on a healthy socket. **Nothing about the gate is sold:** confirmations, the action log's integrity and every role safety property are on every tier and there is deliberately no dimension for any of them. Seven metered dimensions (homes, seats, streams, voice minutes, agent turns, log retention, relay connections), each carrying the measured cost that justifies it. **No fourth entitlement system:** limits resolve from `account-tier.js` badges merged by MAX (so holding $THREE never costs a Pro their Pro limits) and are scaled by the `three-tier.js` holder multiplier, never by thresholds of our own. **One counter:** an agent turn is the same priced `usage_events` chat row an invoice reads, tagged with `home_id`, not a second tally that could disagree. **A downgrade pauses, it never disconnects:** excess homes get `deactivated_at` with a written reason, the credential and the action log survive, `revoked_at` stays null, and the user swaps which houses are live. Enterprise limits are a `home_plan_overrides` row with the reason on it, not a tier. Every number is env-overridable (`HOME_LIMIT_<TIER>_<DIMENSION>`), so applying an owner-approved price is a config change. Docs: [docs/home-plans.md](docs/home-plans.md). Tests: [tests/home-entitlements.test.js](tests/home-entitlements.test.js) |-| Smart home: the live 3D home (`/smart-home/:id`) | Page [pages/home-scene.html](pages/home-scene.html) `-> /smart-home/:id` and `-> /home/:id`, styles [public/home-scene.css](public/home-scene.css). Pure layout [src/home/scene-model.js](src/home/scene-model.js), WebGL renderer [src/home/scene-render.js](src/home/scene-render.js), the 2D house [src/home/scene-fallback.js](src/home/scene-fallback.js), page controller [src/home/scene.js](src/home/scene.js), browser half of the API contract [src/home/api.js](src/home/api.js). Consumes `GET /api/home/:id`, the SSE stream [api/home/[id]/stream.js](api/home/%5Bid%5D/stream.js), [api/home/[id]/call.js](api/home/%5Bid%5D/call.js) and `/api/home/:id/grants` | Live | A live 3D model of the user's house, driven entirely by their own Home Assistant, with the agent's body standing in it. **Home Assistant stores no geometry**, so everything spatial is derived from its four flat registries by a PURE module with no Three.js in it: floors stack by level, rooms pack per floor in name order, and a saved floorplan overrides the packing through one `layout` option. That split is why the layout of a real recorded house is asserted in a unit test without a GPU. **A room is lit the way the room is lit**: colour and intensity from the lights that are actually on, genuinely dark when they are all off, and the outline around its footprint goes amber the moment anything in it is unlocked or open. **A room with sixty entities is not sixty objects**: objects are ranked (lights, locks, covers, climate, alarms, media, fans, cameras, the rest), capped per surface, and the remainder summarized; numeric sensors become readouts, never objects. A device Home Assistant cannot reach is drawn as a ghost with a marker, never omitted. **The house never empties**: a dropped connection greys the scene, keeps every room, and says how old it is. Stale (the platform is retrying) and disconnected (nobody is) are different pills, and only the second offers a button. **The 2D view is not a consolation prize**: same model, real rows and real buttons, reads and operates the whole house, and it is where a browser with no WebGL or a device measured below 18 fps is routed automatically, with the page saying so. A guarded action answers 409 and the question is rendered on the object it would move, with an optional standing allowance for that one entity. Measured: 86 ms median from a real Home Assistant service call to the SSE frame, 10 to 23 ms from that frame to the painted one, 0.1 to 0.3 ms of per-frame work, 108 draw calls, and a flat heap across ten minutes of live updates. Docs: [docs/home-scene.md](docs/home-scene.md). Tests: [tests/home-scene-model.test.js](tests/home-scene-model.test.js), [tests/e2e/home-scene.spec.js](tests/e2e/home-scene.spec.js) |+| Smart home: the live 3D home (`/smart-home/:id`) | Page [pages/home-scene.html](pages/home-scene.html) `-> /smart-home/:id` and `-> /home/:id`, styles [public/home-scene.css](public/home-scene.css). Pure layout [src/home/scene-model.js](src/home/scene-model.js), WebGL renderer [src/home/scene-render.js](src/home/scene-render.js), the 2D house [src/home/scene-fallback.js](src/home/scene-fallback.js), page controller [src/home/scene.js](src/home/scene.js), the top-down floorplan editor the scene follows [src/home/floorplan.js](src/home/floorplan.js) with its validation and storage [api/_lib/home/layout.js](api/_lib/home/layout.js) behind `GET`/`PUT`/`DELETE /api/home/:id/layout` [api/home/[id]/layout.js](api/home/%5Bid%5D/layout.js), browser half of the API contract [src/home/api.js](src/home/api.js). Consumes `GET /api/home/:id`, the SSE stream [api/home/[id]/stream.js](api/home/%5Bid%5D/stream.js), [api/home/[id]/call.js](api/home/%5Bid%5D/call.js) and `/api/home/:id/grants` | Live | A live 3D model of the user's house, driven entirely by their own Home Assistant, with the agent's body standing in it. **Home Assistant stores no geometry**, so everything spatial is derived from its four flat registries by a PURE module with no Three.js in it: floors stack by level, rooms pack per floor in name order, and a saved floorplan overrides the packing through one `layout` option. That split is why the layout of a real recorded house is asserted in a unit test without a GPU. **A room is lit the way the room is lit**: colour and intensity from the lights that are actually on, genuinely dark when they are all off, and the outline around its footprint goes amber the moment anything in it is unlocked or open. **A room with sixty entities is not sixty objects**: objects are ranked (lights, locks, covers, climate, alarms, media, fans, cameras, the rest), capped per surface, and the remainder summarized; numeric sensors become readouts, never objects. A device Home Assistant cannot reach is drawn as a ghost with a marker, never omitted. **The house never empties**: a dropped connection greys the scene, keeps every room, and says how old it is. Stale (the platform is retrying) and disconnected (nobody is) are different pills, and only the second offers a button. **The 2D view is not a consolation prize**: same model, real rows and real buttons, reads and operates the whole house, and it is where a browser with no WebGL or a device measured below 18 fps is routed automatically, with the page saying so. A guarded action answers 409 and the question is rendered on the object it would move, with an optional standing allowance for that one entity. Measured: 86 ms median from a real Home Assistant service call to the SSE frame, 10 to 23 ms from that frame to the painted one, 0.1 to 0.3 ms of per-frame work, 108 draw calls, and a flat heap across ten minutes of live updates. Docs: [docs/home-scene.md](docs/home-scene.md). Tests: [tests/home-scene-model.test.js](tests/home-scene-model.test.js), [tests/e2e/home-scene.spec.js](tests/e2e/home-scene.spec.js) || three.ws Home privacy, retention, export and deletion | Inventory + purge + export + deletion [api/_lib/home/privacy.js](api/_lib/home/privacy.js), disclosure copy [api/_lib/home/disclosure.js](api/_lib/home/disclosure.js), endpoint [api/home/privacy.js](api/home/privacy.js) `-> /api/home/privacy`, schema [api/_lib/migrations/20260903180000_home_privacy_retention.sql](api/_lib/migrations/20260903180000_home_privacy_retention.sql), purge section E of [api/cron/db-retention.js](api/cron/db-retention.js) | Live | The single place that knows everything the Home lane stores. `INVENTORY` is not documentation of the code, it IS the code: the endpoint renders it, the export walks it, and [tests/home-privacy.test.js](tests/home-privacy.test.js) re-derives the set of `home_*` tables from the migration files and fails the build if one of them has no row, which is what stops an undocumented data class from ever landing. The load-bearing promise is that entity names and states are never persisted (the room graph lives in the bridge runtime's memory and dies with the instance); a second test fails any migration that creates a table shaped like a state history. The action log's window is per home and the owner's, 90 days by default, adjustable down to a day and up to ten years with a written reason the schema also enforces, applied immediately on change and swept by the platform's existing retention cron rather than a cron of its own. Deletion goes further than the cascades: memberships and grants on other people's homes, the departed user's id inside another household's action log, invitations addressed to their email, and the lane's rows in the 365-day `audit_log`. Docs: [docs/home-privacy.md](docs/home-privacy.md). Tests: [tests/home-privacy.test.js](tests/home-privacy.test.js) || Avatar thumbnail coverage | [api/_lib/avatar-thumbs.js](api/_lib/avatar-thumbs.js) + crons [api/cron/avatar-thumbnail-backfill.js](api/cron/avatar-thumbnail-backfill.js) (absent thumbnails, `*/5`) and [api/cron/avatar-thumbnail-render.js](api/cron/avatar-thumbnail-render.js) (stale marketplace thumbnails, `*/10`) | Live | The single owner of `avatars.thumbnail_key`. Enforces the invariant that a thumbnail key is persisted **only after** its R2 object is confirmed to exist — a key pointing at a missing object makes R2 return 404 `text/plain`, which Chrome blocks as `ERR_BLOCKED_BY_ORB` inside an `<img>`. Fills coverage cheapest-first: zero-copy adoption of `forge_creations.preview_key`, then a headless-chromium GLB→PNG render to `thumb/<avatarId>.png` ([api/_lib/render-glb.js](api/_lib/render-glb.js)). Bounded retries via the `avatar_thumbnail_backfill` claim ledger ([migration](api/_lib/migrations/20260709120000_avatar_thumbnail_backfill.sql)), with `isBrowserInfrastructureError()` separating an OOM-killed browser from a genuinely broken model so a crash never retires healthy avatars. Bulk runner: [scripts/backfill-avatar-thumbnails.mjs](scripts/backfill-avatar-thumbnails.mjs) (`--status`, `--adopt-only`, `--loop`, `--reset-infra`). Docs: [docs/avatar-thumbnails.md](docs/avatar-thumbnails.md). Tests: [tests/avatar-thumbs.test.js](tests/avatar-thumbs.test.js), [tests/home-thumbnail-orb.test.js](tests/home-thumbnail-orb.test.js) || Forge model categorization | Classifier [api/_lib/forge-classify.js](api/_lib/forge-classify.js), wired at generation in [api/_lib/forge-store.js](api/_lib/forge-store.js) (`createCreation`), backfill [scripts/backfill-forge-categories.mjs](scripts/backfill-forge-categories.mjs), coloured badges in [src/forge-showcase.js](src/forge-showcase.js) + `/forge-studio` twin | Live | Assigns every forge model a `model_category` (avatar/accessory/item/scene/creature/vehicle/other) from its prompt with a deterministic, dependency-free keyword classifier — most-specific-first so "a knight riding a horse" is an avatar, not a creature. Nothing set the category before, so ~99.8% of the corpus was `other` and the category badges, filters, and Forge-Off category dimension were dead; the classifier now runs inline on every generation (all lanes) and reclassifies ~85% of the 9.8k-row back catalogue (dry-run verified). Coloured category pills render across the showcase/gallery/community feed. Tests: [tests/forge-classify.test.js](tests/forge-classify.test.js). |