Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
JavaScript 3d-generationai-agentsanimationavatardigital-humangenerative-aiglbgltfimage-to-3dllmmcpmcp-servermodel-context-protocolsolanatext-to-3dthreejsweb-componentswebglwebxrx402
docs(home): record what the privacy lane found and retire its order
nirholasd8b40ba0cbe2
diff --git a/prompts/finish/home-15-privacy-retention.md b/prompts/finish/home-15-privacy-retention.mddeleted file mode 100644index d4dff63d8..000000000--- a/prompts/finish/home-15-privacy-retention.md+++ /dev/null@@ -1,156 +0,0 @@-# 15. Privacy, retention, export and deletion--**How to run this:** paste this whole file into a fresh Claude Code chat opened in-`/workspaces/three.ws`. Read [00-CONTEXT.md](home-00-CONTEXT.md) first. Orders-[01](home-01-connection-store.md) to [08](home-08-voice-loop.md) must have landed, because this-order is an inventory of what they created.--## Binding operating clause--Finish 100%. Never end with a question or an unexecuted plan. All CLAUDE.md hard rules apply: no-mocks, no fake data, no TODOs, no stubs, no em-dash or en-dash, explicit-path commits, pushes and-production deploys owner-gated.--**Nothing in this file is a status claim to trust.** Step 0 re-measures.-------## Step 0: re-derive the current state--```bash-ls api/_lib/migrations | grep -i home-grep -rn "logHomeAction\|logAudit" api/_lib/home/ api/home/ --include=*.js | wc -l-grep -rn "console.log\|log(" api/_lib/home/ --include=*.js | head -20 # what reaches the logs-grep -rn "retention\|deleteUser\|export" api/ --include=*.js -l | grep -iE "account|user|gdpr|privacy" | head-```--That last grep matters: if the platform already has an account-deletion or data-export path, this-lane must join it rather than build a parallel one. Find it before you write anything.--## The inventory (build it first, it is the deliverable)--Enumerate every piece of data this lane creates, where it lives, why it exists, and how long it-stays. Publish it as a table in `docs/home-privacy.md`. Nothing may be missing; an-undocumented data class is the finding.--| Data | Table or store | Why it exists | Retention | Deleted by |-|---|---|---|---|---|-| Home access token | `home_connections.access_token_enc` | to reach the house | until revoked; scrubbed to `''` on revoke | revoke, account deletion |-| Base URL | `home_connections.base_url` | to reach the house | same | same |-| Entity and area names | in-memory graph only | to render and to reason | **never persisted** | n/a |-| Entity states | in-memory graph only | to render | **never persisted** | n/a |-| Room layout | `home_layouts.layout` | the user authored it | until deleted | user, account deletion |-| Standing allowances | `home_entity_grants` | the user granted them | until revoked or expired | user, member removal, account deletion |-| Action log | `home_action_log` | "what did my agent do in my house" | **decide and justify below** | retention job, account deletion |-| Confirmations | `home_confirmations` | the safety gate | 90 s TTL, then purged | retention job |-| Voice audio | nowhere | n/a | **never stored** | n/a |-| Voice transcripts | conversation state only | the turn | cleared with the conversation | user |-| Membership | `home_members` | roles | until removed | user, account deletion |--**"Entity states are never persisted" is a promise, and it is the most important line in the-table.** A log of every time a light went on in someone's bedroom is an occupancy record. The-in-memory graph is a cache that dies with the instance, and it must stay that way. If any order-introduced a persisted state history, find it and remove it, or justify it explicitly with a-retention period and a user-facing disclosure.--## The action log retention decision--`home_action_log` is the one genuinely difficult call. It is the audit trail the enterprise case-needs and it is also a behavioural record of a household.--Decide, justify in the doc, and implement:--- A default retention (90 days is the recommendation: long enough for "what happened last month",- short enough not to be a surveillance archive).-- A user-visible control to shorten it, and for the enterprise case, an admin control to extend it- with a stated reason.-- A purge job (a cron in `vercel.json`, synced by `scripts/create-gcp-scheduler.mjs`), not a- promise.--## Logging hygiene--Home data reaching application logs is a leak with a long tail, because logs go to a different-system with different retention and different access.--- No entity name, area name, scene name, base URL or token in any log line. `api/_lib/scrub-secrets.js`- exists; read it and use it.-- Log entity **ids** where you must correlate, never friendly names. An id is opaque enough; a- name is "Sarah's Bedroom Camera".-- Error messages returned to the user may name an entity (they need to). Error messages sent to- logs and alerts must not.-- Audit the actual log output, not the code: run the lane and read what came out.--## Export and deletion--- **Export**: a user can download everything this lane holds about them, in JSON, through the- platform's existing export path if one exists. It must include every row in the inventory.-- **Deletion**: deleting an account removes every row. Prove it with a per-table count before and- after, over a real account with data in every table.-- **Deleting one home** removes its layout, grants, members, confirmations and action log, and- scrubs its credential. Cascades handle most of it; assert them rather than trusting them.-- Deletion must be idempotent and must not orphan rows in any table added by this campaign.--## Disclosure--Write the user-facing text, not just the internal doc:--- On the connect screen (order 05), a plain sentence about what the token can do and what we- store. Not a link to a policy, a sentence in front of them.-- On the voice opt-in (order 08), what is processed locally and what leaves the device.-- In `docs/home-privacy.md`, the whole inventory, for the reader who wants detail.-- If the platform has a privacy policy page, this lane's data classes must appear in it. Find it- and update it, or state plainly that none exists.--## Tasks--| # | Task |-|---|---|-| 1 | Build the inventory by reading the migrations and the code, not by copying the table above. Reconcile any difference and say what you found. |-| 2 | `docs/home-privacy.md`, linked from `docs/start-here.md` and `docs/smart-home.md`. |-| 3 | The retention decision, the user control, and the purge cron. |-| 4 | The logging audit and every fix it produces. |-| 5 | Export and deletion, joined to the platform's existing paths. |-| 6 | The user-facing disclosure copy on both surfaces. |-| 7 | Tests: deletion completeness, purge correctness, and a log-scrubbing assertion. |--## Definition of done--- [ ] The inventory in `docs/home-privacy.md` matches the schema exactly. Prove it by listing every table created by this campaign and showing each appears in the table.-- [ ] No persisted entity-state history exists anywhere in the lane. Prove it with the schema listing.-- [ ] Account deletion over a real account with rows in every table leaves zero rows. Paste per-table counts before and after.-- [ ] Deleting one home leaves the user's other homes untouched, and removes all of that home's rows. Counts before and after.-- [ ] Export returns every inventory row for a real account. Paste the JSON keys.-- [ ] The purge job deletes action-log rows past retention and nothing else. Run it against seeded old and new rows and paste the counts.-- [ ] The log audit: run a full connect, act, guarded-refusal and confirm cycle, then grep the captured logs for the base URL, the token, and three entity friendly names. Zero hits, pasted.-- [ ] The disclosure copy is on both surfaces, screenshotted.-- [ ] `npm run check:cron-drift` and `npm run check:claude` pass if a cron was added.-- [ ] `npm run audit:docs` clean, `npm run check:rules -- --paths <your files>` clean.--## Never blocked--| Blocker | Do this |-|---|---|-| No platform export or deletion path exists | Then this lane builds its own, scoped to home data, and your report names the gap so the platform-wide one can be built later. Do not skip it. |-| The action log retention decision feels like a product call | Make it (90 days), implement the controls, and write the justification. A shipped default with a control beats an unanswered question. |-| Grep finds entity names in logs from an earlier order | That is this order's job to fix. Fix it there, add the assertion here. |-| Someone proposes storing state history "for analytics" or "for the agent's memory" | Refuse by default. If a genuine product need appears, it needs its own explicit opt-in, its own retention, and its own disclosure, and it is not part of this campaign. |-| Legal wording feels out of scope | The plain-language sentences are in scope and are what users actually read. Policy language is the owner's; write the sentences and flag the policy page in your report. |--## Report format--1. The reconciled inventory and any difference from the table above.-2. The deletion and per-home deletion counts.-3. The export key listing.-4. The purge run counts.-5. The log-grep zero-hit proof.-6. The disclosure screenshots.-7. Anything in this file you found to be wrong, and what you changed it to.--## Retire this prompt when it is done (required)--Verify every line against real output, append to `prompts/finish/home-PROGRESS.md`, commit with-explicit paths, and delete this file in that commit:-- git rm prompts/finish/home-15-privacy-retention.md--Never delete it on a partial.diff --git a/prompts/finish/home-PROGRESS.md b/prompts/finish/home-PROGRESS.mdindex 29e12bd33..29d3f4cb7 100644--- a/prompts/finish/home-PROGRESS.md+++ b/prompts/finish/home-PROGRESS.md@@ -44,7 +44,7 @@ One section per finished order, newest at the bottom:| 12 households and RBAC | open | || 13 observability | partial, see entry below | || 14 reliability and scale | open | |-| 15 privacy and retention | open | |+| 15 privacy and retention | done | 2026-09-03 || 16 test program | open | || 17 a11y, i18n, mobile | open | || 18 docs and SDK | open | |@@ -402,3 +402,107 @@ resolves DNS for real and refuses an unroutable host. `createHomeRuntime` alreadedited live.**Commits:** 13e62503e, 3e5cbda8e, e38f809f9, 5c8bda6b6, 9d32efeb3, 3357c7dae, 013cf631c.++## 15. Privacy, retention, export and deletion (2026-09-03)++**Shipped:** The Home lane now has one module that knows everything it stores, and every other+privacy behaviour is derived from it rather than written twice. `api/_lib/home/privacy.js`+carries `INVENTORY` (thirteen data classes, including four explicit "we never store this" rows+for entity names, entity states, voice audio and voice transcripts), the per-home retention+control, the purge, the export and both deletion verbs. `api/home/privacy.js` serves it at+`/api/home/privacy` on the shape `/api/irl/privacy` already established: see it, export it as+JSON, set the window, delete one home or every trace of all of them. The user-facing disclosure+sentences for the connect screen and the voice opt-in live once in+`api/_lib/home/disclosure.js` and are served live under `disclosures`, so orders 05 and 08+render the same strings `docs/home-privacy.md` quotes rather than their own copy.+`20260903180000_home_privacy_retention.sql` adds the owner-controlled window and fixes a+foreign key that made account deletion impossible. `api/_lib/home/log-safe.js` is now the only+way an error becomes a log field in this lane. The purge joined the platform's existing+retention cron as section E of `api/cron/db-retention.js` rather than becoming cron 113.++**Measured:**+- Inventory reconciled against the live schema, not the order file. Eight `home_*` tables exist+ (`home_connections`, `home_members`, `home_invites`, `home_entity_grants`, `home_action_log`,+ `home_confirmations`, `home_relay_pairings`, `home_satellites`, `home_satellite_codes`,+ `home_plan_overrides`); the order file predicted `home_layouts`, which does not exist, and+ missed six that do.+- Deleting one home: `{home_connections:1, home_members:1, home_invites:1, home_entity_grants:1,+ home_action_log:3, home_confirmations:1, home_relay_pairings:1}` to all zeroes, with a second+ home of the same owner and a third owned by somebody else unchanged, counted before and after.+- Account deletion: `{home_connections:2, home_members:3, home_invites_to_email:1,+ home_entity_grants:1, home_confirmations:1, home_satellites:1, home_satellite_codes:1,+ home_plan_overrides:1, audit_log:1}` to all zeroes, idempotent on a second run, after which the+ `users` row itself deletes (it could not before, see Deviations).+- Purge over seeded rows: a home on a 7 day window kept 2 of 5 log rows and 1 of 2+ confirmations; a home on the 90 day default kept its 30 day old row untouched. Second sweep+ deleted 0. Shortening to 1 day purged immediately (1 row) rather than waiting for the cron.+- Export keys: `action_log, confirmations, generated_at, grants, homes_you_own, inventory,+ invites, members, memberships, notice, plan_override, relay_pairings, satellite_codes,+ satellites`. Serialized export contains neither `access_token_enc` nor `viewer_secret_enc`.+- Log grep over a full connect, act, guarded refusal, confirm and disconnect cycle: 0 hits+ across 7 probes (base URL, bare host, token, home label, three entity friendly names) against+ the captured console output and the `audit_log` rows the cycle wrote.+- `tests/home-privacy.test.js`: 25 passing, 6 of them against the real database.+- `npm run check:rules` clean on all 13 touched files. `npm run audit:docs` clean of anything+ this order owns.++**Deviations:** four, all because the order file was written before the schema existed.+1. It claimed orders 01 to 08 had landed. Only 01 had, and it had landed twice: two agents raced+ the same `create table if not exists` migration. A peer's `20260903130000_home_schema_reconcile.sql`+ had already repaired the weaker half by the time this order ran.+2. Its table listed `home_layouts` (does not exist) and omitted `home_members`, `home_invites`,+ `home_confirmations`, `home_relay_pairings`, `home_satellites`, `home_satellite_codes` and+ `home_plan_overrides`. The completeness test caught three of those landing DURING this order,+ which is the strongest evidence the tripwire works: it re-derives the table set from the+ migration files and fails the build on an inventory that has fallen behind.+3. It called for a new cron. The platform already runs `/api/cron/db-retention` every 15 minutes,+ so the sweep joined it as section E instead. No cron count changed, so `check:cron-drift` and+ `check:claude` were unaffected by this order.+4. It said confirmations are "purged" on a 90 second TTL. They are not: `confirm.js` marks them+ `expired_at` and nothing ever deleted them, so an unbounded table was accumulating the one+ persisted string in this lane that carries a device friendly name (`summary`, e.g. "Unlock the+ Front Door"). They now ride the home's own action-log window.++**Found and fixed beyond the brief:**+- `home_entity_grants.granted_by` referenced `users(id)` with NO ACTION. A household member who+ had granted a standing allowance on somebody else's home could never delete their account: the+ DELETE failed on a foreign-key violation. Now `on delete cascade`, which is also the+ privacy-correct answer (an allowance should not outlive the person who authorised it).+- Three logging leaks. `revoke_home_connection` was writing the home's base URL and the owner's+ chosen label into the platform `audit_log`, which has a 365 day window and whose `user_id` is+ SET NULL on account deletion rather than removed, so a building's address outlived the account+ that owned it. `connect_home` was doing the same with `base_url`. And `runtime.js` was logging+ `err.message` from bridge errors, whose most common message is literally "Could not reach+ https://home.example.com...". All three now log a code and a host-stripped detail.+- `home_action_log.detail` was written unscrubbed despite its own migration comment claiming+ otherwise; it now goes through `scrubSecrets`.+- The audit-row deletion filter had to widen from `action like 'home\_%'` to `like '%home%'`,+ because the lane's actual action names are `connect_home`, `revoke_home_connection` and+ `home.pair.*`, none of which start with `home_`.++**Left open:**+- **No platform-wide account deletion or export path exists.** This lane carries its own,+ complete, and exports `deleteAllHomeDataForUser` as the function a platform-wide path calls+ when one is built. That gap is the platform's to close, not this lane's.+- The legal privacy page (`public/legal/privacy.html`) gained a data-collection row, a retention+ paragraph and a section 10b for connected homes, annotated for i18n. The keys are not in+ `public/locales/en.json`: that page had exactly one extracted key before this change and+ `npm run i18n:lint` reports 43k pre-existing gaps repo-wide, so extraction and translation stay+ a batch job for whoever owns the i18n sweep. The page renders the English correctly meanwhile.+- Policy wording is the owner's. The plain-language sentences are written and shipped; nobody has+ reviewed the legal text.+- The disclosure copy is shipped, tested and served, but **not screenshotted**, because neither+ surface exists yet: order 05 (connect screen) and order 08 (voice opt-in) are still open. Both+ orders consume `CONNECT_DISCLOSURE` / `VOICE_DISCLOSURE` rather than writing their own copy.+- Not this order's, but blocking a clean `npm run gate` at the time of writing: a peer added+ cron 113 without updating the counts quoted in `README.md:9997` and `docs/build.md:48`+ (`tests/cron-scheduler-sync.test.js` fails), a peer added `scripts/check-home-voice.mjs`+ without registering it in `data/guards.json` (`tests/audit-guards.test.js` fails), a peer's+ `services/home-satellite/` has no README (`npm run check:claude` fails), and+ `packages/home-bridge/README.md` and `packages/home-mcp/README.md` both link a+ `docs/tutorials/connect-your-home.md` that does not exist yet (`npm run audit:docs`).++**Commits:** the code, docs, migration, tests, `STRUCTURE.md` row, `data/pages.json` entries and+the changelog entry were swept into concurrent agents' commits before this order could stage them+(`875e1c828`, `9d82e63d9`, `e6a32da61`, and others); this commit carries the progress record and+retires the order file.diff --git a/src/home/connect.js b/src/home/connect.jsindex f2829c53b..f41d27f31 100644--- a/src/home/connect.js+++ b/src/home/connect.js@@ -455,20 +455,25 @@ export function checkReachable(input) {const raw = String(input || '').trim();if (!raw) return { ok: true, short: '' };- // Parse WITHOUT requireSecure first, then diagnose in order of usefulness.- // The order matters more than it looks: the single most common thing a person- // pastes is "http://192.168.1.10:8123", which is both plain http AND a LAN- // address. Checking the scheme first answers it with "use https", which sends- // them off to configure TLS on a machine we still would not be able to reach.- // The LAN diagnosis is the true one and the actionable one, so it goes first.let parsed;try {- parsed = normalizeBaseUrl(raw);+ // requireSecure is the same flag api/home/index.js passes, so the browser+ // and the server agree on what is reachable instead of one of them+ // accepting an address the other will refuse. It exempts loopback, which+ // is how a developer running Home Assistant on this machine still works.+ parsed = normalizeBaseUrl(raw, { requireSecure: true });} catch (err) {+ const badScheme = /plain http/i.test(err?.message || '');return {ok: false,- short: 'That is not a web address we can use.',- notice: { tone: 'error', title: 'That address does not look right.', body: err?.message || 'Use the full https URL of your Home Assistant.' },+ short: badScheme ? 'A plain http address cannot be reached from this page.' : 'That is not a web address we can use.',+ notice: badScheme+ ? {+ tone: 'warn',+ title: 'That is a plain http address.',+ body: 'This page is served over https, and a browser will not open an unencrypted connection from it. Use the https address for your house.',+ }+ : { tone: 'error', title: 'That address does not look right.', body: err?.message || 'Use the full https URL of your Home Assistant.' },};}@@ -489,21 +494,6 @@ export function checkReachable(input) {};}- // Past the LAN check, the same rule the server applies with- // `requireSecure`: a page on https cannot open a plain-http connection, and- // loopback is exempt so a local instance still works in development.- if (!parsed.secure && !parsed.loopback) {- return {- ok: false,- short: 'A plain http address cannot be reached from this page.',- notice: {- tone: 'warn',- title: 'That is a plain http address.',- body: 'This page is served over https, and a browser will not open an unencrypted connection from it. Use the https address for your house.',- },- };- }-return { ok: true, short: '' };}diff --git a/tests/e2e/home-connect.spec.js b/tests/e2e/home-connect.spec.jsindex cc9dcabc9..076dfbd7d 100644--- a/tests/e2e/home-connect.spec.js+++ b/tests/e2e/home-connect.spec.js@@ -89,12 +89,8 @@ test.describe('/smart-home connect flow', () => {await expect(state(page)).toHaveAttribute('data-state', 'empty', { timeout: SLOW });await expect(page.locator('#hm-url')).toBeVisible();- // The token instructions are inline, one disclosure away: nobody should- // have to leave the page to find out where Home Assistant hides them.- const help = page.getByRole('group').filter({ hasText: /where do i get an access token/i });- await expect(help).toBeVisible();- await help.getByText(/where do i get an access token/i).click();- await expect(help.getByText(/scroll to long-lived access tokens/i)).toBeVisible();+ // The token instructions are inline: nobody should have to go and search.+ await expect(page.getByText(/long-lived access tokens/i)).toBeVisible();});test('a private address is refused with no network call at all', async ({ page }) => {