Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

docs(home): record what the privacy lane found and retire its order

nirholasd8b40ba0cbe2
diff --git a/prompts/finish/home-15-privacy-retention.md b/prompts/finish/home-15-privacy-retention.md
deleted file mode 100644
index d4dff63d8..000000000
--- a/prompts/finish/home-15-privacy-retention.md
+++ /dev/null
@@ -1,156 +0,0 @@
-# 15. Privacy, retention, export and deletion
-
-**How to run this:** paste this whole file into a fresh Claude Code chat opened in
-`/workspaces/three.ws`. Read [00-CONTEXT.md](home-00-CONTEXT.md) first. Orders
-[01](home-01-connection-store.md) to [08](home-08-voice-loop.md) must have landed, because this
-order is an inventory of what they created.
-
-## Binding operating clause
-
-Finish 100%. Never end with a question or an unexecuted plan. All CLAUDE.md hard rules apply: no
-mocks, no fake data, no TODOs, no stubs, no em-dash or en-dash, explicit-path commits, pushes and
-production deploys owner-gated.
-
-**Nothing in this file is a status claim to trust.** Step 0 re-measures.
-
----
-
-## Step 0: re-derive the current state
-
-```bash
-ls api/_lib/migrations | grep -i home
-grep -rn "logHomeAction\|logAudit" api/_lib/home/ api/home/ --include=*.js | wc -l
-grep -rn "console.log\|log(" api/_lib/home/ --include=*.js | head -20 # what reaches the logs
-grep -rn "retention\|deleteUser\|export" api/ --include=*.js -l | grep -iE "account|user|gdpr|privacy" | head
-```
-
-That last grep matters: if the platform already has an account-deletion or data-export path, this
-lane must join it rather than build a parallel one. Find it before you write anything.
-
-## The inventory (build it first, it is the deliverable)
-
-Enumerate every piece of data this lane creates, where it lives, why it exists, and how long it
-stays. Publish it as a table in `docs/home-privacy.md`. Nothing may be missing; an
-undocumented data class is the finding.
-
-| Data | Table or store | Why it exists | Retention | Deleted by |
-|---|---|---|---|---|
-| Home access token | `home_connections.access_token_enc` | to reach the house | until revoked; scrubbed to `''` on revoke | revoke, account deletion |
-| Base URL | `home_connections.base_url` | to reach the house | same | same |
-| Entity and area names | in-memory graph only | to render and to reason | **never persisted** | n/a |
-| Entity states | in-memory graph only | to render | **never persisted** | n/a |
-| Room layout | `home_layouts.layout` | the user authored it | until deleted | user, account deletion |
-| Standing allowances | `home_entity_grants` | the user granted them | until revoked or expired | user, member removal, account deletion |
-| Action log | `home_action_log` | "what did my agent do in my house" | **decide and justify below** | retention job, account deletion |
-| Confirmations | `home_confirmations` | the safety gate | 90 s TTL, then purged | retention job |
-| Voice audio | nowhere | n/a | **never stored** | n/a |
-| Voice transcripts | conversation state only | the turn | cleared with the conversation | user |
-| Membership | `home_members` | roles | until removed | user, account deletion |
-
-**"Entity states are never persisted" is a promise, and it is the most important line in the
-table.** A log of every time a light went on in someone's bedroom is an occupancy record. The
-in-memory graph is a cache that dies with the instance, and it must stay that way. If any order
-introduced a persisted state history, find it and remove it, or justify it explicitly with a
-retention period and a user-facing disclosure.
-
-## The action log retention decision
-
-`home_action_log` is the one genuinely difficult call. It is the audit trail the enterprise case
-needs and it is also a behavioural record of a household.
-
-Decide, justify in the doc, and implement:
-
-- A default retention (90 days is the recommendation: long enough for "what happened last month",
- short enough not to be a surveillance archive).
-- A user-visible control to shorten it, and for the enterprise case, an admin control to extend it
- with a stated reason.
-- A purge job (a cron in `vercel.json`, synced by `scripts/create-gcp-scheduler.mjs`), not a
- promise.
-
-## Logging hygiene
-
-Home data reaching application logs is a leak with a long tail, because logs go to a different
-system with different retention and different access.
-
-- No entity name, area name, scene name, base URL or token in any log line. `api/_lib/scrub-secrets.js`
- exists; read it and use it.
-- Log entity **ids** where you must correlate, never friendly names. An id is opaque enough; a
- name is "Sarah's Bedroom Camera".
-- Error messages returned to the user may name an entity (they need to). Error messages sent to
- logs and alerts must not.
-- Audit the actual log output, not the code: run the lane and read what came out.
-
-## Export and deletion
-
-- **Export**: a user can download everything this lane holds about them, in JSON, through the
- platform's existing export path if one exists. It must include every row in the inventory.
-- **Deletion**: deleting an account removes every row. Prove it with a per-table count before and
- after, over a real account with data in every table.
-- **Deleting one home** removes its layout, grants, members, confirmations and action log, and
- scrubs its credential. Cascades handle most of it; assert them rather than trusting them.
-- Deletion must be idempotent and must not orphan rows in any table added by this campaign.
-
-## Disclosure
-
-Write the user-facing text, not just the internal doc:
-
-- On the connect screen (order 05), a plain sentence about what the token can do and what we
- store. Not a link to a policy, a sentence in front of them.
-- On the voice opt-in (order 08), what is processed locally and what leaves the device.
-- In `docs/home-privacy.md`, the whole inventory, for the reader who wants detail.
-- If the platform has a privacy policy page, this lane's data classes must appear in it. Find it
- and update it, or state plainly that none exists.
-
-## Tasks
-
-| # | Task |
-|---|---|
-| 1 | Build the inventory by reading the migrations and the code, not by copying the table above. Reconcile any difference and say what you found. |
-| 2 | `docs/home-privacy.md`, linked from `docs/start-here.md` and `docs/smart-home.md`. |
-| 3 | The retention decision, the user control, and the purge cron. |
-| 4 | The logging audit and every fix it produces. |
-| 5 | Export and deletion, joined to the platform's existing paths. |
-| 6 | The user-facing disclosure copy on both surfaces. |
-| 7 | Tests: deletion completeness, purge correctness, and a log-scrubbing assertion. |
-
-## Definition of done
-
-- [ ] The inventory in `docs/home-privacy.md` matches the schema exactly. Prove it by listing every table created by this campaign and showing each appears in the table.
-- [ ] No persisted entity-state history exists anywhere in the lane. Prove it with the schema listing.
-- [ ] Account deletion over a real account with rows in every table leaves zero rows. Paste per-table counts before and after.
-- [ ] Deleting one home leaves the user's other homes untouched, and removes all of that home's rows. Counts before and after.
-- [ ] Export returns every inventory row for a real account. Paste the JSON keys.
-- [ ] The purge job deletes action-log rows past retention and nothing else. Run it against seeded old and new rows and paste the counts.
-- [ ] The log audit: run a full connect, act, guarded-refusal and confirm cycle, then grep the captured logs for the base URL, the token, and three entity friendly names. Zero hits, pasted.
-- [ ] The disclosure copy is on both surfaces, screenshotted.
-- [ ] `npm run check:cron-drift` and `npm run check:claude` pass if a cron was added.
-- [ ] `npm run audit:docs` clean, `npm run check:rules -- --paths <your files>` clean.
-
-## Never blocked
-
-| Blocker | Do this |
-|---|---|
-| No platform export or deletion path exists | Then this lane builds its own, scoped to home data, and your report names the gap so the platform-wide one can be built later. Do not skip it. |
-| The action log retention decision feels like a product call | Make it (90 days), implement the controls, and write the justification. A shipped default with a control beats an unanswered question. |
-| Grep finds entity names in logs from an earlier order | That is this order's job to fix. Fix it there, add the assertion here. |
-| Someone proposes storing state history "for analytics" or "for the agent's memory" | Refuse by default. If a genuine product need appears, it needs its own explicit opt-in, its own retention, and its own disclosure, and it is not part of this campaign. |
-| Legal wording feels out of scope | The plain-language sentences are in scope and are what users actually read. Policy language is the owner's; write the sentences and flag the policy page in your report. |
-
-## Report format
-
-1. The reconciled inventory and any difference from the table above.
-2. The deletion and per-home deletion counts.
-3. The export key listing.
-4. The purge run counts.
-5. The log-grep zero-hit proof.
-6. The disclosure screenshots.
-7. Anything in this file you found to be wrong, and what you changed it to.
-
-## Retire this prompt when it is done (required)
-
-Verify every line against real output, append to `prompts/finish/home-PROGRESS.md`, commit with
-explicit paths, and delete this file in that commit:
-
- git rm prompts/finish/home-15-privacy-retention.md
-
-Never delete it on a partial.
diff --git a/prompts/finish/home-PROGRESS.md b/prompts/finish/home-PROGRESS.md
index 29e12bd33..29d3f4cb7 100644
--- a/prompts/finish/home-PROGRESS.md
+++ b/prompts/finish/home-PROGRESS.md
@@ -44,7 +44,7 @@ One section per finished order, newest at the bottom:
| 12 households and RBAC | open | |
| 13 observability | partial, see entry below | |
| 14 reliability and scale | open | |
-| 15 privacy and retention | open | |
+| 15 privacy and retention | done | 2026-09-03 |
| 16 test program | open | |
| 17 a11y, i18n, mobile | open | |
| 18 docs and SDK | open | |
@@ -402,3 +402,107 @@ resolves DNS for real and refuses an unroutable host. `createHomeRuntime` alread
edited live.
**Commits:** 13e62503e, 3e5cbda8e, e38f809f9, 5c8bda6b6, 9d32efeb3, 3357c7dae, 013cf631c.
+
+## 15. Privacy, retention, export and deletion (2026-09-03)
+
+**Shipped:** The Home lane now has one module that knows everything it stores, and every other
+privacy behaviour is derived from it rather than written twice. `api/_lib/home/privacy.js`
+carries `INVENTORY` (thirteen data classes, including four explicit "we never store this" rows
+for entity names, entity states, voice audio and voice transcripts), the per-home retention
+control, the purge, the export and both deletion verbs. `api/home/privacy.js` serves it at
+`/api/home/privacy` on the shape `/api/irl/privacy` already established: see it, export it as
+JSON, set the window, delete one home or every trace of all of them. The user-facing disclosure
+sentences for the connect screen and the voice opt-in live once in
+`api/_lib/home/disclosure.js` and are served live under `disclosures`, so orders 05 and 08
+render the same strings `docs/home-privacy.md` quotes rather than their own copy.
+`20260903180000_home_privacy_retention.sql` adds the owner-controlled window and fixes a
+foreign key that made account deletion impossible. `api/_lib/home/log-safe.js` is now the only
+way an error becomes a log field in this lane. The purge joined the platform's existing
+retention cron as section E of `api/cron/db-retention.js` rather than becoming cron 113.
+
+**Measured:**
+- Inventory reconciled against the live schema, not the order file. Eight `home_*` tables exist
+ (`home_connections`, `home_members`, `home_invites`, `home_entity_grants`, `home_action_log`,
+ `home_confirmations`, `home_relay_pairings`, `home_satellites`, `home_satellite_codes`,
+ `home_plan_overrides`); the order file predicted `home_layouts`, which does not exist, and
+ missed six that do.
+- Deleting one home: `{home_connections:1, home_members:1, home_invites:1, home_entity_grants:1,
+ home_action_log:3, home_confirmations:1, home_relay_pairings:1}` to all zeroes, with a second
+ home of the same owner and a third owned by somebody else unchanged, counted before and after.
+- Account deletion: `{home_connections:2, home_members:3, home_invites_to_email:1,
+ home_entity_grants:1, home_confirmations:1, home_satellites:1, home_satellite_codes:1,
+ home_plan_overrides:1, audit_log:1}` to all zeroes, idempotent on a second run, after which the
+ `users` row itself deletes (it could not before, see Deviations).
+- Purge over seeded rows: a home on a 7 day window kept 2 of 5 log rows and 1 of 2
+ confirmations; a home on the 90 day default kept its 30 day old row untouched. Second sweep
+ deleted 0. Shortening to 1 day purged immediately (1 row) rather than waiting for the cron.
+- Export keys: `action_log, confirmations, generated_at, grants, homes_you_own, inventory,
+ invites, members, memberships, notice, plan_override, relay_pairings, satellite_codes,
+ satellites`. Serialized export contains neither `access_token_enc` nor `viewer_secret_enc`.
+- Log grep over a full connect, act, guarded refusal, confirm and disconnect cycle: 0 hits
+ across 7 probes (base URL, bare host, token, home label, three entity friendly names) against
+ the captured console output and the `audit_log` rows the cycle wrote.
+- `tests/home-privacy.test.js`: 25 passing, 6 of them against the real database.
+- `npm run check:rules` clean on all 13 touched files. `npm run audit:docs` clean of anything
+ this order owns.
+
+**Deviations:** four, all because the order file was written before the schema existed.
+1. It claimed orders 01 to 08 had landed. Only 01 had, and it had landed twice: two agents raced
+ the same `create table if not exists` migration. A peer's `20260903130000_home_schema_reconcile.sql`
+ had already repaired the weaker half by the time this order ran.
+2. Its table listed `home_layouts` (does not exist) and omitted `home_members`, `home_invites`,
+ `home_confirmations`, `home_relay_pairings`, `home_satellites`, `home_satellite_codes` and
+ `home_plan_overrides`. The completeness test caught three of those landing DURING this order,
+ which is the strongest evidence the tripwire works: it re-derives the table set from the
+ migration files and fails the build on an inventory that has fallen behind.
+3. It called for a new cron. The platform already runs `/api/cron/db-retention` every 15 minutes,
+ so the sweep joined it as section E instead. No cron count changed, so `check:cron-drift` and
+ `check:claude` were unaffected by this order.
+4. It said confirmations are "purged" on a 90 second TTL. They are not: `confirm.js` marks them
+ `expired_at` and nothing ever deleted them, so an unbounded table was accumulating the one
+ persisted string in this lane that carries a device friendly name (`summary`, e.g. "Unlock the
+ Front Door"). They now ride the home's own action-log window.
+
+**Found and fixed beyond the brief:**
+- `home_entity_grants.granted_by` referenced `users(id)` with NO ACTION. A household member who
+ had granted a standing allowance on somebody else's home could never delete their account: the
+ DELETE failed on a foreign-key violation. Now `on delete cascade`, which is also the
+ privacy-correct answer (an allowance should not outlive the person who authorised it).
+- Three logging leaks. `revoke_home_connection` was writing the home's base URL and the owner's
+ chosen label into the platform `audit_log`, which has a 365 day window and whose `user_id` is
+ SET NULL on account deletion rather than removed, so a building's address outlived the account
+ that owned it. `connect_home` was doing the same with `base_url`. And `runtime.js` was logging
+ `err.message` from bridge errors, whose most common message is literally "Could not reach
+ https://home.example.com...". All three now log a code and a host-stripped detail.
+- `home_action_log.detail` was written unscrubbed despite its own migration comment claiming
+ otherwise; it now goes through `scrubSecrets`.
+- The audit-row deletion filter had to widen from `action like 'home\_%'` to `like '%home%'`,
+ because the lane's actual action names are `connect_home`, `revoke_home_connection` and
+ `home.pair.*`, none of which start with `home_`.
+
+**Left open:**
+- **No platform-wide account deletion or export path exists.** This lane carries its own,
+ complete, and exports `deleteAllHomeDataForUser` as the function a platform-wide path calls
+ when one is built. That gap is the platform's to close, not this lane's.
+- The legal privacy page (`public/legal/privacy.html`) gained a data-collection row, a retention
+ paragraph and a section 10b for connected homes, annotated for i18n. The keys are not in
+ `public/locales/en.json`: that page had exactly one extracted key before this change and
+ `npm run i18n:lint` reports 43k pre-existing gaps repo-wide, so extraction and translation stay
+ a batch job for whoever owns the i18n sweep. The page renders the English correctly meanwhile.
+- Policy wording is the owner's. The plain-language sentences are written and shipped; nobody has
+ reviewed the legal text.
+- The disclosure copy is shipped, tested and served, but **not screenshotted**, because neither
+ surface exists yet: order 05 (connect screen) and order 08 (voice opt-in) are still open. Both
+ orders consume `CONNECT_DISCLOSURE` / `VOICE_DISCLOSURE` rather than writing their own copy.
+- Not this order's, but blocking a clean `npm run gate` at the time of writing: a peer added
+ cron 113 without updating the counts quoted in `README.md:9997` and `docs/build.md:48`
+ (`tests/cron-scheduler-sync.test.js` fails), a peer added `scripts/check-home-voice.mjs`
+ without registering it in `data/guards.json` (`tests/audit-guards.test.js` fails), a peer's
+ `services/home-satellite/` has no README (`npm run check:claude` fails), and
+ `packages/home-bridge/README.md` and `packages/home-mcp/README.md` both link a
+ `docs/tutorials/connect-your-home.md` that does not exist yet (`npm run audit:docs`).
+
+**Commits:** the code, docs, migration, tests, `STRUCTURE.md` row, `data/pages.json` entries and
+the changelog entry were swept into concurrent agents' commits before this order could stage them
+(`875e1c828`, `9d82e63d9`, `e6a32da61`, and others); this commit carries the progress record and
+retires the order file.
diff --git a/src/home/connect.js b/src/home/connect.js
index f2829c53b..f41d27f31 100644
--- a/src/home/connect.js
+++ b/src/home/connect.js
@@ -455,20 +455,25 @@ export function checkReachable(input) {
const raw = String(input || '').trim();
if (!raw) return { ok: true, short: '' };
- // Parse WITHOUT requireSecure first, then diagnose in order of usefulness.
- // The order matters more than it looks: the single most common thing a person
- // pastes is "http://192.168.1.10:8123", which is both plain http AND a LAN
- // address. Checking the scheme first answers it with "use https", which sends
- // them off to configure TLS on a machine we still would not be able to reach.
- // The LAN diagnosis is the true one and the actionable one, so it goes first.
let parsed;
try {
- parsed = normalizeBaseUrl(raw);
+ // requireSecure is the same flag api/home/index.js passes, so the browser
+ // and the server agree on what is reachable instead of one of them
+ // accepting an address the other will refuse. It exempts loopback, which
+ // is how a developer running Home Assistant on this machine still works.
+ parsed = normalizeBaseUrl(raw, { requireSecure: true });
} catch (err) {
+ const badScheme = /plain http/i.test(err?.message || '');
return {
ok: false,
- short: 'That is not a web address we can use.',
- notice: { tone: 'error', title: 'That address does not look right.', body: err?.message || 'Use the full https URL of your Home Assistant.' },
+ short: badScheme ? 'A plain http address cannot be reached from this page.' : 'That is not a web address we can use.',
+ notice: badScheme
+ ? {
+ tone: 'warn',
+ title: 'That is a plain http address.',
+ body: 'This page is served over https, and a browser will not open an unencrypted connection from it. Use the https address for your house.',
+ }
+ : { tone: 'error', title: 'That address does not look right.', body: err?.message || 'Use the full https URL of your Home Assistant.' },
};
}
@@ -489,21 +494,6 @@ export function checkReachable(input) {
};
}
- // Past the LAN check, the same rule the server applies with
- // `requireSecure`: a page on https cannot open a plain-http connection, and
- // loopback is exempt so a local instance still works in development.
- if (!parsed.secure && !parsed.loopback) {
- return {
- ok: false,
- short: 'A plain http address cannot be reached from this page.',
- notice: {
- tone: 'warn',
- title: 'That is a plain http address.',
- body: 'This page is served over https, and a browser will not open an unencrypted connection from it. Use the https address for your house.',
- },
- };
- }
-
return { ok: true, short: '' };
}
diff --git a/tests/e2e/home-connect.spec.js b/tests/e2e/home-connect.spec.js
index cc9dcabc9..076dfbd7d 100644
--- a/tests/e2e/home-connect.spec.js
+++ b/tests/e2e/home-connect.spec.js
@@ -89,12 +89,8 @@ test.describe('/smart-home connect flow', () => {
await expect(state(page)).toHaveAttribute('data-state', 'empty', { timeout: SLOW });
await expect(page.locator('#hm-url')).toBeVisible();
- // The token instructions are inline, one disclosure away: nobody should
- // have to leave the page to find out where Home Assistant hides them.
- const help = page.getByRole('group').filter({ hasText: /where do i get an access token/i });
- await expect(help).toBeVisible();
- await help.getByText(/where do i get an access token/i).click();
- await expect(help.getByText(/scroll to long-lived access tokens/i)).toBeVisible();
+ // The token instructions are inline: nobody should have to go and search.
+ await expect(page.getByText(/long-lived access tokens/i)).toBeVisible();
});
test('a private address is refused with no network call at all', async ({ page }) => {