Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

feat(drive): put the agent in the car, voice-first, on the surface CarPlay actually allows

nirholasd378eeaef52d
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c269085f7..cc4ef1195 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,6 +4,12 @@
Public history for [three.ws](https://three.ws), newest first. New pages come from `added` dates in data/pages.json; everything else is curated in data/changelog.json. Also available as [JSON](https://three.ws/changelog.json) and [RSS](https://three.ws/changelog.xml), live at [three.ws/changelog](https://three.ws/changelog).
+## 2026-09-03
+
+- **Docs · three.ws Drive** (`/docs/carplay`): How a three.ws agent rides in the car: the CarPlay entitlement categories measured, why Apple's voice-based conversational category is the one that fits and why it grants no drawing surface, the open-source in-car projects worth adopting and the copyleft ones that are not, and how the /drive surface, the native CarPlay scene, and an open head unit share one implementation.
+- **three.ws Drive** (`/drive`): Your 3D agent in the car: voice-first, glanceable, and built to the rules CarPlay and Android Auto enforce. Runs on a head unit, a phone in the cradle, or inside the native car shell.
+- **three.ws Drive: your agent, in the car**: There is a new page at /drive built for a car screen instead of a desk. Your agent is on it full size, you hold one big button and talk, and it answers out loud in its own voice with its face moving. It shows you one line of what it said rather than a wall of chat, because you are driving. It starts dark, because a bright panel at night is glare, and it will switch to a day palette on one press or one sentence. The keyboard stays out of the way while you are moving: if your browser can tell how fast you are going, text entry locks itself the moment you pull away and comes back when you have properly stopped. A few things you say never leave the car at all, so "repeat that", "stop talking", "louder" and "night mode" happen instantly, while "stop at the next charger" still goes to your agent like any other question. It works today on a head unit browser, on a phone in a cradle, and on Android Automotive. The Apple CarPlay version is built and waiting on Apple's approval for voice assistants in the car. One thing this fixed for everyone: an avatar used to answer in silence if you were not signed in, and now it always has a voice. (`/drive`) `[feature, improvement]`
+
## 2026-09-02
- **Docs · Materialize: turn a generation into a real object** (`/docs/materialize`): How to print any three.ws model for real: the printability report, preparation and repair, materials and how pricing is computed, USDC checkout on Solana, the x402 lane an AI agent uses to order manufacturing, on-chain certificates of authenticity, and the fabrication content policy.
diff --git a/STRUCTURE.md b/STRUCTURE.md
index 15dd861f0..3c04f42fc 100644
--- a/STRUCTURE.md
+++ b/STRUCTURE.md
@@ -59,6 +59,7 @@ bottom.
| Herald: messages delivered in person | SDK [herald-sdk/](herald-sdk) `→ /herald.js` (published as `@three-ws/herald`), rail [api/herald/announce.js](api/herald/announce.js) + [api/herald/stream.js](api/herald/stream.js) + [api/_lib/herald.js](api/_lib/herald.js), playground [pages/herald.html](pages/herald.html) `→ /herald` + [src/herald-page.js](src/herald-page.js), CLI [herald-sdk/bin/herald.mjs](herald-sdk/bin/herald.mjs), MCP [packages/herald-mcp/](packages/herald-mcp) | Live | A 3D character walks onto the page and says what happened, instead of a badge nobody notices. The library is the interrupt budget: importance scoring with pluggable scorers, a floor, quiet hours, a rate limit, dedupe with a TTL, freshness, batch collapse, focus awareness, and a stable reason for every message it refuses (`duplicate`, `rate-limited`, `quiet-hours`, `below-importance-floor`). Delivery is the walk companion ([walk-sdk](walk-sdk) `control.announce`) with an accessible `aria-live` card fallback for anything that cannot render 3D, and opt-in gesture-gated voice. **The rail is the part nobody expects:** `POST /api/herald/announce` with an API key carrying the `herald:announce` scope reaches the browser tab the caller has open over SSE, so `herald watch -- npm test` makes your avatar tell you the build finished, and an AI agent can do the same through the MCP server. An announcement is always delivered to the caller's own sessions (no recipient field, so no key can interrupt anyone else); the queue is Redis-backed, drained not broadcast, and expires in five minutes. Docs: [docs/herald.md](docs/herald.md), [docs/api-reference.md](docs/api-reference.md). Tests: [tests/herald-rules.test.js](tests/herald-rules.test.js), [tests/herald-runtime.test.js](tests/herald-runtime.test.js), [tests/herald-rail.test.js](tests/herald-rail.test.js), [tests/herald-announce-endpoint.test.js](tests/herald-announce-endpoint.test.js) |
| Notification bell & preferences | Event vocabulary [api/_lib/feed.js](api/_lib/feed.js) (`USER_EVENT_TYPES` → `publishUserEvent()` → `user_notifications`), endpoints [api/notifications/](api/notifications), preference matrix [api/_lib/notify-prefs.js](api/_lib/notify-prefs.js), bell client [src/notifications.js](src/notifications.js), avatar announcements [src/notification-herald.js](src/notification-herald.js), settings panel [src/dashboard-next/pages/settings.js](src/dashboard-next/pages/settings.js) | Live | In-app inbox covering remixes ([api/x402/remix-asset.js](api/x402/remix-asset.js)), DMs ([api/friends/messages.js](api/friends/messages.js)), coin bonding-curve graduations (pump cron in [api/cron/[name].js](api/cron/%5Bname%5D.js)), follows, sales, purchases, IRL, market, and account events. Preferences are a category × channel matrix (`in_app` always on; push/email/telegram/avatar are per-category opt-outs) edited from `/dashboard/settings`. **The `avatar` channel delivers in person:** for a fresh, unread notification in an opted-in category the corner companion walks on screen, gestures, and says the line with a link straight to it, summoning itself for that one message and retreating if it was off ([walk-sdk](walk-sdk) `control.announce`). Rules (15-minute freshness, once per browser, two per batch, visible tab only, device-level mute) live in [src/notification-herald.js](src/notification-herald.js). Docs: [docs/notifications.md](docs/notifications.md), [docs/social-layer.md](docs/social-layer.md). Tests: [tests/notification-herald.test.js](tests/notification-herald.test.js), [tests/notify-prefs.test.js](tests/notify-prefs.test.js) |
| Personal companion (`/companion`) | Page [pages/companion.html](pages/companion.html) `→ /companion` + [src/companion/](src/companion) (page, source cards, voice), API [api/companion/](api/companion) (settings, sources, contacts, events, `ingest`, `stream`, `poll`), engine [api/_lib/companion/](api/_lib/companion) (store, encrypted BYOK configs, triage, lanes for Telegram / iCal / IMAP), cron [api/cron/companion-poll.js](api/cron/companion-poll.js), migration [api/_lib/migrations/20260828140000_companion_harness.sql](api/_lib/migrations/20260828140000_companion_harness.sql) | Live | Turns a person's own inbox, chats, calendar and phone into deliveries a 3D character makes **in person**. BYOK per source (a Telegram bot the user made, a private iCal URL, IMAP with an app password) plus a universal bridge token anything can POST to. Every message is scored 0-100 by the shared deterministic rules ([@three-ws/companion/triage](packages/companion-sdk/src/triage-rules.js), the same module the CLI runs locally) and optionally refined by the user's own Anthropic key; above their threshold and outside quiet hours it is delivered through `companion_delivery` notifications, so the bell, Web Push, and the avatar channel all carry it. A contact's message is spoken by **that contact's avatar and voice** (`control.announce({ avatar })` in [walk-sdk](walk-sdk)). A delivery whose lane can carry an answer (Telegram, through the user's own bot) offers a reply box in place, and the reply is stored with the message. Live SSE at `/api/companion/stream` feeds the desktop app and any third-party body. Docs: [docs/companion.md](docs/companion.md). Tests: [tests/companion-triage.test.js](tests/companion-triage.test.js), [tests/companion-calendar.test.js](tests/companion-calendar.test.js), [packages/companion-sdk/tests/](packages/companion-sdk/tests) |
+| three.ws Drive: the agent in the car (`/drive`) | Page [pages/drive.html](pages/drive.html) `→ /drive` + [src/drive/](src/drive) (page controller, surface presets, native bridge, local commands, motion gate); native CarPlay scene [ios/native/App/App/CarPlaySceneDelegate.swift](ios/native/App/App/CarPlaySceneDelegate.swift) + [DriveLink.swift](ios/native/App/App/DriveLink.swift) | Live (web) / built, entitlement pending (CarPlay) | The existing talk loop (`/api/asr` → `/api/chat` → `/api/tts` → lipsync on a real rigged avatar) rebuilt for a car panel: one glanceable two-line reply instead of a transcript, exactly four controls, night by default, and a keyboard that hides itself while the wheels turn. Speed comes from `navigator.geolocation`'s real ground speed, so moving above walking pace locks text entry back and a sustained stop is what unlocks it. Control phrases ("repeat that", "stop talking", "louder", "night mode") are answered locally with no round trip, matched against the whole utterance so "stop at the next charger" still reaches the agent. Hands free ends a turn from the real captured mic level on the Riva lane. **CarPlay is a second scene, not a second product:** Apple's voice-based conversational category (iOS 26.4) grants templates and an audio session and no drawing surface, so the car screen renders four rows plus `CPVoiceControlTemplate` while this page runs on the phone, joined by the versioned `threeWsDrive` WebKit channel. Docs: [docs/carplay.md](docs/carplay.md), [ios/docs/CARPLAY.md](ios/docs/CARPLAY.md). Tests: [tests/drive-surface.test.js](tests/drive-surface.test.js) |
| Knock: a priced door to a person (`/knock`) | Hub [pages/knock.html](pages/knock.html) `→ /knock` + [src/knock/page.js](src/knock/page.js), public door [pages/knock-door.html](pages/knock-door.html) `→ /knock/<handle>` + [src/knock/door.js](src/knock/door.js), API [api/knock/](api/knock) (door, directory, send, reply, settings, inbox), paid lane [api/x402/knock.js](api/x402/knock.js), engine [api/_lib/knock/](api/_lib/knock) (pure policy, store, delivery, receipt tokens), migration [api/_lib/migrations/20260828210000_knock_doors.sql](api/_lib/migrations/20260828210000_knock_doors.sql), SDK + CLI [packages/knock-sdk/](packages/knock-sdk) (`@three-ws/knock`), MCP [packages/knock-mcp/](packages/knock-mcp) (`@three-ws/knock-mcp`) | Live | Every account can publish a door at `/knock/<username>` and set what one message from a stranger costs. Paying it buys exactly one message through, and the delivery is **in person**: the knock becomes a `companion_events` row whose importance is derived from the amount paid, so the recipient's corner avatar walks on wherever they are and says who is at the door and what they paid. **The USDC settles straight to the recipient** (`payTo` on the 402 challenge is their own address, and a priced door cannot be opened without one), so there is no platform wallet in the path and no cut. Solana leads; Base is advertised only when the owner sets an address for it. **Every refusal happens before the 402** (shut door, daily cap, block list, length limits) and again before settlement, so a knock that was never going to land is never a knock somebody paid for. Only the subject line is spoken; an untrusted body never uses the recipient's avatar as a speaker. The sender gets a derived HMAC receipt URL and reads the reply through it with no account, which is why answering someone never hands out an address. Free doors take a plain POST on a tight per-IP bucket. Spec: [specs/KNOCK_PROTOCOL.md](specs/KNOCK_PROTOCOL.md). Docs: [docs/knock.md](docs/knock.md). Tests: [tests/knock-policy.test.js](tests/knock-policy.test.js), [packages/knock-sdk/test/](packages/knock-sdk/test), [packages/knock-mcp/test/](packages/knock-mcp/test) |
| Visitor feedback loop | Recorder [@three-ws/witness](packages/witness), capture + panel [src/feedback-companion.js](src/feedback-companion.js) (mounted by [src/walk-companion.js](src/walk-companion.js)), intake [api/feedback/report.js](api/feedback/report.js), queue API [api/feedback/index.js](api/feedback/index.js), scoring [api/_lib/feedback/triage.js](api/_lib/feedback/triage.js), store [api/_lib/feedback/store.js](api/_lib/feedback/store.js), cron [api/cron/feedback-triage.js](api/cron/feedback-triage.js), review page [pages/feedback.html](pages/feedback.html) `→ /feedback` + [src/feedback-page.js](src/feedback-page.js) | Live | The return channel on the corner companion: the visitor tells the avatar what broke and the report carries the route, build SHA, viewport, and the last console errors and failed requests the browser captured, so a vague sentence lands as an actionable row. The companion offers proactively at the one moment worth asking (right after the page actually threw), once per route per session, never in a background tab. **Feedback is data, never an instruction:** the model that reads untrusted visitor text writes six scalars to that report's own row and has no path to the repo, a deploy, or any other write; the admin-only `/feedback` queue groups reports into one row per problem and a person decides. **A report arrives as a runnable failing test:** the recorded session compiles into a Playwright spec asserting the failure is GONE (red until fixed, green after), reachable from the queue's Copy test button, `GET /api/feedback/repro`, `npm run feedback:repro`, and the `get_feedback_repro` MCP tool. Docs: [docs/feedback.md](docs/feedback.md), [docs/witness.md](docs/witness.md). Tests: [tests/feedback-triage.test.js](tests/feedback-triage.test.js) |
| Session-to-test recorder | [packages/witness/](packages/witness) `→ @three-ws/witness`: recorder [recorder.js](packages/witness/src/recorder.js), selector synthesis [selector.js](packages/witness/src/selector.js), capture-time privacy [redact.js](packages/witness/src/redact.js), spec compiler [compile.js](packages/witness/src/compile.js); compiled repros land in [tests/repros/](tests/repros) with [their own Playwright config](tests/repros/playwright.config.mjs) | Published as `@three-ws/witness` (Apache-2.0, zero deps) | Turns a real user session into a runnable failing test. Records INTENT (about 2 KB) rather than DOM mutations (megabytes), so it can be always-on, which is the point: the trace already exists when somebody decides to complain. Every element gets the strongest handle that is unique in the live document (test id > id > aria-label > role+name > text > name > placeholder > class > structural path), framework-generated ids and utility classes refused, and each trace carries a replay confidence equal to its WEAKEST link. The compiled spec asserts the failure is gone, never that it happened, so it locks in the fix rather than the bug; a session with no machine-visible failure emits `test.fail()` and says which assertion a human must add rather than faking green. Privacy is enforced at capture: typed values are never held (only a length and a shape), password/hidden/email/tel/date inputs and `data-witness="off"` subtrees record nothing, URLs lose their credentials before storage. Docs: [docs/witness.md](docs/witness.md). Tests: [packages/witness/test/](packages/witness/test) (36, `npm run test:witness`) |
@@ -238,7 +239,7 @@ bottom.
| BNB Chain live block race | [pages/bnb-latency.html](pages/bnb-latency.html) + [src/bnb-latency.js](src/bnb-latency.js) `→ /bnb-latency`, pure helpers [src/bnb-latency-helpers.js](src/bnb-latency-helpers.js), live multi-chain API [api/bnb/latency.js](api/bnb/latency.js) over [api/_lib/bnb/latency-lanes.js](api/_lib/bnb/latency-lanes.js) (reuses `probeBlockTime` from [api/_lib/bnb/chains.js](api/_lib/bnb/chains.js) for BNB, adds Base/Ethereum/Solana lanes) | In-repo · shipped | Standalone, shareable proof page: four lanes (BNB Chain, Base, Ethereum, Solana) polled every 5s, each showing a freshly measured average block/slot time and a rolling sparkline — no hardcoded numbers, a down RPC shows "reconnecting" on just that lane. Headline strip shows BNB's live average plus its honest measured-vs-measured speedup over Base/Ethereum. Linked from the [/bnb](pages/bnb.html) hub's "real-time on-chain world" card, which auto-lights-up now that this route resolves. |
| BNB Chain payments (MPP, x402↔MPP bridge, gasless) | Accept [api/_lib/bnb/mpp-server.js](api/_lib/bnb/mpp-server.js) (`mppRequirements`/`mppChallenge`/`mppVerify`/`mppSettle`), pay [api/_lib/bnb/mpp-buyer.js](api/_lib/bnb/mpp-buyer.js) (`mppFetch`, re-exported from [api/_lib/x402-buyer-fetch.js](api/_lib/x402-buyer-fetch.js)), gasless rail [api/_lib/bnb/megafuel.js](api/_lib/bnb/megafuel.js), pilot dual-protocol endpoint [api/x402/three-intel.js](api/x402/three-intel.js) | In-repo · shipped, settlement needs merchant creds | three.ws speaks MPP (BNB Chain's Machine Payments Protocol) additively alongside x402: MPP's `b402` layer is x402 v2 byte-for-byte (same `X-PAYMENT`/`X-PAYMENT-RESPONSE`, same EIP-3009 credential), so a dual-protocol endpoint just advertises an extra BNB (`eip155:56`/`97`) accept entry and routes BNB-network payments to the b402 facilitator while Solana/Base traffic stays on the untouched x402 path. Verify (off-chain, free, replay-guarded via Redis `SET NX PX`) always runs before settle; without `B402_*` merchant credentials, settle fails closed with `mpp_not_configured` (503) rather than fabricating a receipt. Guide: [docs/bnb-payments.md](docs/bnb-payments.md). Wire contract: [specs/x402-mpp-bridge.md](specs/x402-mpp-bridge.md). Tests: [tests/bnb-mpp-server.test.js](tests/bnb-mpp-server.test.js), [tests/bnb-mpp-buyer.test.js](tests/bnb-mpp-buyer.test.js). |
| Seeker / Saga dApp Store app | [solana-mobile/](solana-mobile): TWA wrapper [solana-mobile/twa/](solana-mobile/twa), Publisher Portal listing copy [solana-mobile/publish/](solana-mobile/publish), MWA wallet + Seeker detection [solana-mobile/src/](solana-mobile/src) | Live · v1.0.0 on the Solana dApp Store (submitted 2026-08-28, approved 2026-09-01) | Everything needed to package three.ws as a Solana Mobile app and publish it to the on-chain dApp Store. Targets Seeker first; compatible with Saga Gen 1 and Gen 2. Docs: [solana-mobile/README.md](solana-mobile/README.md) |
-| iOS App Store app | [ios/](ios): Capacitor shell + config [ios/capacitor.config.ts](ios/capacitor.config.ts), Xcode project [ios/native/App/](ios/native/App), web-side native bridge [ios/src/native-bridge.js](ios/src/native-bridge.js), offline shell [ios/shell/](ios/shell), listing copy [ios/publish/](ios/publish), app association handler in [api/wk.js](api/wk.js) | In-repo · project builds and the web half ships on every deploy · awaiting an Apple Developer organization account | three.ws as a native iOS app (`ws.three.app`). A Capacitor 8 shell whose WKWebView runs the live `https://three.ws`, so the app is never a version behind the site; what makes it an app is the native layer around it. `ios/src/native-bridge.js` ships with the SITE, not the .ipa (Capacitor injects its bridge into the remote page, so `window.Capacitor` exists on three.ws only inside the app): it supplies the `navigator.share` WKWebView does not have, routes off-site links to an in-app Safari sheet instead of stranding the app's only WebView, handles universal links and `threews://` wallet returns, holds the launch screen until the first real frame, and marks `html.ios-app`. The `three-ws-ios-native-bridge` plugin in [vite.config.js](vite.config.js) copies it into `dist/` content-hashed and injects it into every built page except the third-party embed entries (671 pages, `/viewer` and `/ar` included, which a `transformIndexHtml` could not reach because they are copied verbatim from `public/`). `GET /.well-known/apple-app-site-association` is served from the API rather than a static file because Apple demands an exact path, a JSON content type and no redirect; it answers `503 not_configured` until `APPLE_TEAM_ID` is set rather than publishing an association that would fail silently on device. Docs: [ios/README.md](ios/README.md), [docs/ios-app.md](docs/ios-app.md), App Review posture [ios/docs/REVIEW-RISK.md](ios/docs/REVIEW-RISK.md), submission [ios/docs/SUBMISSION.md](ios/docs/SUBMISSION.md). |
+| iOS App Store app | [ios/](ios): Capacitor shell + config [ios/capacitor.config.ts](ios/capacitor.config.ts), Xcode project [ios/native/App/](ios/native/App), web-side native bridge [ios/src/native-bridge.js](ios/src/native-bridge.js), offline shell [ios/shell/](ios/shell), listing copy [ios/publish/](ios/publish), app association handler in [api/wk.js](api/wk.js) | In-repo · project builds and the web half ships on every deploy · awaiting an Apple Developer organization account | three.ws as a native iOS app (`ws.three.app`). A Capacitor 8 shell whose WKWebView runs the live `https://three.ws`, so the app is never a version behind the site; what makes it an app is the native layer around it. `ios/src/native-bridge.js` ships with the SITE, not the .ipa (Capacitor injects its bridge into the remote page, so `window.Capacitor` exists on three.ws only inside the app): it supplies the `navigator.share` WKWebView does not have, routes off-site links to an in-app Safari sheet instead of stranding the app's only WebView, handles universal links and `threews://` wallet returns, holds the launch screen until the first real frame, and marks `html.ios-app`. The `three-ws-ios-native-bridge` plugin in [vite.config.js](vite.config.js) copies it into `dist/` content-hashed and injects it into every built page except the third-party embed entries (671 pages, `/viewer` and `/ar` included, which a `transformIndexHtml` could not reach because they are copied verbatim from `public/`). `GET /.well-known/apple-app-site-association` is served from the API rather than a static file because Apple demands an exact path, a JSON content type and no redirect; it answers `503 not_configured` until `APPLE_TEAM_ID` is set rather than publishing an association that would fail silently on device. The app also carries the **CarPlay** scene for three.ws Drive: `CarPlaySceneDelegate` renders the car screen's templates while `DriveLink` bridges them to `/drive` in the WebView, gated behind the `com.apple.developer.carplay-voice-based-conversation` entitlement, which is inert until Apple grants it. Docs: [ios/README.md](ios/README.md), [docs/ios-app.md](docs/ios-app.md), App Review posture [ios/docs/REVIEW-RISK.md](ios/docs/REVIEW-RISK.md), submission [ios/docs/SUBMISSION.md](ios/docs/SUBMISSION.md), CarPlay [ios/docs/CARPLAY.md](ios/docs/CARPLAY.md). |
| Walk Avatar browser extension | [extensions/walk-avatar/](extensions/walk-avatar) — MV3 `manifest.json`, `background.js` | In-repo · Chrome Web Store release checklist staged ([RELEASE.md](extensions/walk-avatar/RELEASE.md)) | Walks your three.ws avatar across any site and reads pages aloud — a browsing companion built on the same rig as the [walk-sdk](walk-sdk). Ships its own store-required [PRIVACY.md](extensions/walk-avatar/PRIVACY.md), [PERMISSIONS.md](extensions/walk-avatar/PERMISSIONS.md), and [TERMS.md](extensions/walk-avatar/TERMS.md). |
| Checkout companion browser extension | [extensions/checkout-companion/](extensions/checkout-companion) (MV3: [extract.js](extensions/checkout-companion/extract.js), [content.js](extensions/checkout-companion/content.js), [panel.js](extensions/checkout-companion/panel.js), [background.js](extensions/checkout-companion/background.js)), analysis [api/_lib/companion/checkout.js](api/_lib/companion/checkout.js), endpoint [api/companion/checkout.js](api/companion/checkout.js) | In-repo | The companion at the moment money moves: on a checkout page it reports the total against the price you were shown, money the page did not itemise, fees named and summed, whether the charge repeats, and the cancellation terms. **Arithmetic is never the model's job** (integer minor units, computed server-side) and **no finding is ever a legal verdict** (`sanitizeModelFindings()` drops any that claims an amount or calls something illegal). With no model reachable the arithmetic findings still stand. The extension **never reads an input**: `collectText`/`collectAmounts` refuse form fields, iframes and editable regions structurally, and redaction runs in the browser before anything is sent; only the hostname travels and nothing about the page is stored. Docs: [docs/checkout-companion.md](docs/checkout-companion.md). Tests: [tests/companion-checkout.test.js](tests/companion-checkout.test.js), [tests/checkout-extract.test.js](tests/checkout-extract.test.js) |
| Desktop companion app | [apps/desktop/](apps/desktop). Electron main [src/main.js](apps/desktop/src/main.js), preload [src/preload.cjs](apps/desktop/src/preload.cjs), stage [src/renderer/](apps/desktop/src/renderer) | In-repo · packaged with electron-builder (`npm run dist`), not yet published | A transparent, click-through, always-on-top character that strolls across the bottom of the desktop and walks over to deliver what cleared the user's bar. Subscribes to `/api/companion/stream` with the same bridge token the CLI stores, and drives the published [walk embed](pages/walk-embed.html) over its postMessage contract, so it carries no 3D code of its own. Not an npm workspace: it installs and builds on its own (see [apps/desktop/README.md](apps/desktop/README.md)). |
diff --git a/data/changelog.json b/data/changelog.json
index bb140c861..c275a3e6f 100644
--- a/data/changelog.json
+++ b/data/changelog.json
@@ -25835,6 +25835,16 @@
"improvement"
],
"link": "/materialize"
+ },
+ {
+ "date": "2026-09-03",
+ "title": "three.ws Drive: your agent, in the car",
+ "summary": "There is a new page at /drive built for a car screen instead of a desk. Your agent is on it full size, you hold one big button and talk, and it answers out loud in its own voice with its face moving. It shows you one line of what it said rather than a wall of chat, because you are driving. It starts dark, because a bright panel at night is glare, and it will switch to a day palette on one press or one sentence. The keyboard stays out of the way while you are moving: if your browser can tell how fast you are going, text entry locks itself the moment you pull away and comes back when you have properly stopped. A few things you say never leave the car at all, so \"repeat that\", \"stop talking\", \"louder\" and \"night mode\" happen instantly, while \"stop at the next charger\" still goes to your agent like any other question. It works today on a head unit browser, on a phone in a cradle, and on Android Automotive. The Apple CarPlay version is built and waiting on Apple's approval for voice assistants in the car. One thing this fixed for everyone: an avatar used to answer in silence if you were not signed in, and now it always has a voice.",
+ "tags": [
+ "feature",
+ "improvement"
+ ],
+ "link": "/drive"
}
]
}
diff --git a/data/pages.json b/data/pages.json
index 6ce6e21b7..f192f7583 100644
--- a/data/pages.json
+++ b/data/pages.json
@@ -152,6 +152,15 @@
"added": "2026-07-01",
"showcase": true
},
+ {
+ "path": "/drive",
+ "title": "three.ws Drive",
+ "description": "Your 3D agent in the car: voice-first, glanceable, and built to the rules CarPlay and Android Auto enforce. Runs on a head unit, a phone in the cradle, or inside the native car shell.",
+ "priority": 0.7,
+ "changefreq": "weekly",
+ "added": "2026-09-03",
+ "showcase": true
+ },
{
"path": "/agenc/embodied",
"title": "AgenC · Embodied",
@@ -2621,6 +2630,14 @@
"changefreq": "monthly",
"added": "2026-09-02"
},
+ {
+ "path": "/docs/carplay",
+ "title": "Docs \u00b7 three.ws Drive",
+ "description": "How a three.ws agent rides in the car: the CarPlay entitlement categories measured, why Apple's voice-based conversational category is the one that fits and why it grants no drawing surface, the open-source in-car projects worth adopting and the copyleft ones that are not, and how the /drive surface, the native CarPlay scene, and an open head unit share one implementation.",
+ "priority": 0.6,
+ "changefreq": "monthly",
+ "added": "2026-09-03"
+ },
{
"path": "/docs/tty-avatar",
"title": "Docs · Terminal avatar",
diff --git a/docs/carplay.md b/docs/carplay.md
new file mode 100644
index 000000000..2a3c3be10
--- /dev/null
+++ b/docs/carplay.md
@@ -0,0 +1,247 @@
+# three.ws Drive: the agent in the car
+
+**Status:** research complete, architecture decided, the shared car surface (`/drive`) is
+built and verified in a browser, and the native CarPlay scene is written and wired into the
+iOS app. The one step that cannot be finished from here is Apple's entitlement grant.
+**Date:** 2026-09-03.
+**Question asked:** can a three.ws 3D agent be the assistant on an Apple CarPlay screen, and
+how much of it already exists in open source?
+
+**Short answer:** the door opened five months ago and almost nobody has walked through it.
+In iOS 26.4 Apple added a CarPlay app category for **voice-based conversational apps**, with
+its own entitlement, aimed squarely at assistants like this one. iOS 27 then let the Voice
+Control template present as an overlay on any other template, including a live map. That is
+the unlock. What Apple does **not** give a conversational app is a drawing surface, so the
+honest architecture is not "render the avatar on the car screen": it is a voice-first agent
+whose control surface is Apple's templates and whose face lives everywhere else, on the
+phone in the cradle, on a head unit browser, on the passenger display, and on the open head
+units where nothing is gated at all.
+
+---
+
+## 1. Why this is a better fit than it looks
+
+The [smart home plan](smart-home.md) argued that a home is the first place where a body, a
+voice, memory and identity all pay off at once. A car is the second, and it is a better
+first shipment, for a reason that has nothing to do with ambition: **in a car, voice is not
+one interface among several. It is the only one that is legal.**
+
+Every constraint that makes an in-car product hard is a constraint three.ws already builds
+against:
+
+- **You cannot ask the driver to read.** three.ws agents already speak, with a real voice
+ and real lipsync, through `/api/tts` and the existing talk loop.
+- **You cannot ask the driver to look for long.** A glanceable presence that conveys state
+ through a face and a voice, rather than a list, is the whole product.
+- **The agent has to be someone.** A generic assistant is interchangeable. An agent with a
+ name, a face, a voice, memory, and an on-chain identity is the one a person actually wants
+ riding along, and it is the one they can bring from the browser to the car to the house
+ without re-teaching it anything.
+
+The car is also where the platform's other surfaces stop being separate products. An agent
+that can already reach a home through [`@three-ws/home-bridge`](../packages/home-bridge) is
+the same agent that can be told "we are ten minutes out" on the way back.
+
+---
+
+## 2. The landscape, measured
+
+### What Apple actually allows (this is the part that decides the design)
+
+CarPlay apps are not free-form. An app declares one category, requests the matching
+entitlement, and gets a fixed vocabulary of templates that iOS renders on the car screen.
+The app is the controller and the model; iOS is the view.
+
+| Category | What it gets | Fit |
+|---|---|---|
+| Navigation (`carplay-maps`) | A `UIWindow` to draw a map into, plus map templates | The only category with a real render surface, but the base view must be used **exclusively** to draw a map, with no panels or overlays of our own. A 3D agent in it is not a navigation app. |
+| Audio, Communication, EV charging, Parking, Quick ordering | Lists, grids, now-playing | Voice only through SiriKit, no conversation |
+| Driving task | Lists and grids, depth 2 (depth 3 on iOS 26.4+) | Explicitly must not duplicate navigation, and cannot use the voice template |
+| **Voice-based conversation** (`carplay-voice-based-conversation`) | The Voice Control template, an audio session, depth 3 | **This one.** Added in iOS 26.4 for conversational assistants. |
+
+The rules that come with the voice category shape the product, so they are worth stating
+plainly rather than discovering during review: voice must be the primary modality on launch,
+the app cannot be the system-wide assistant, there is **no wake word**, it must be launched
+by hand from the CarPlay home screen, and the audio session may be held open only while
+voice is actually in use.
+
+iOS 27 loosened the visual half: the Voice Control template can present as an overlay over
+another template instead of taking the whole screen, and the template was opened to every
+category. An app holding `carplay-maps` can now keep a map rendering underneath a live
+assistant. That is a door worth walking through later; it is not the first shipment.
+
+### Android's side, for symmetry
+
+Android draws the same line in a different place. The Car App Library
+(`androidx.car.app`, Apache-2.0) is template-based too, and the host renders the UI. But its
+`NavigationTemplate` hands an app a real `Surface` through `SurfaceCallback`, which can be
+drawn into with Canvas or with the VirtualDisplay/Presentation APIs. MapLibre publishes a
+working sample of exactly that. So the "3D agent rendered on the car screen" idea is
+technically reachable on Android Auto through the navigation category, and blocked on
+CarPlay by the exclusive-map rule. Android Automotive OS, which runs the app on the head
+unit itself, is looser still.
+
+### The open-source inventory
+
+| Project | License | What it gives us | Verdict |
+|---|---|---|---|
+| [rhysmorgan134/node-CarPlay](https://github.com/rhysmorgan134/node-CarPlay) | MIT | Talks to a Carlinkit CPC200 USB dongle over WebUSB or Node USB bindings and forwards the CarPlay H.264 video and PCM audio, with mic and touch going back | **Adopt** for the head unit lane. It is the only practical way to render a real CarPlay session inside our own JS surface. |
+| [birkir/react-native-carplay](https://github.com/birkir/react-native-carplay) | MIT | One template API across CarPlay and Android Auto for React Native apps | **Read, do not adopt.** Its template mapping is the best reference implementation in the open, but the three.ws app is Capacitor, not React Native, and the scene delegate we need is 200 lines of Swift. |
+| [androidx.car.app](https://developer.android.com/training/cars/apps/library) | Apache-2.0 | The Android Auto and AAOS template host, including the navigation `Surface` | **Adopt** when the Android leg lands. |
+| [maplibre/MapLibre-Android-Auto-Sample](https://github.com/maplibre/MapLibre-Android-Auto-Sample) | BSD-2 | A worked example of rendering into the Android Auto navigation surface | **Reference.** This is the shape a 3D agent on an Android car screen would take. |
+| [f1xpl/openauto](https://github.com/f1xpl/openauto) | GPL-3.0 | Android Auto head unit emulation on a Raspberry Pi | **Reject.** Copyleft we cannot link into a commercial product, and effectively unmaintained. |
+
+Everything below the protocol layer we already have and should not rebuild: speech in
+(`/api/asr`, NVIDIA Riva), speech out (`/api/tts`, NVIDIA Magpie), the model chain
+(`/api/chat`), lipsync, the retargeted animation library, and the avatar itself.
+
+---
+
+## 3. The decision
+
+**Three lanes, one surface.** Everything that is actually the product lives in one page,
+`/drive`, and each lane is a different way of putting that page in front of a driver. That
+is what keeps the car from forking into three half-products.
+
+### Lane A: `/drive`, the car surface (built)
+
+A voice-first page sized and behaved for a car panel. It is the whole product on a head unit
+browser, on a phone in a cradle, and on Android Automotive, with no approval from anyone.
+See [Section 4](#4-what-drive-actually-does).
+
+### Lane B: native CarPlay (built, waiting on Apple)
+
+A `CPTemplateApplicationSceneDelegate` in the existing iOS app. The car screen shows the
+four controls and the Voice Control template; the phone runs `/drive`; a WebKit message
+channel joins them. Written, wired, and documented in
+[`ios/docs/CARPLAY.md`](../ios/docs/CARPLAY.md). It cannot ship until Apple grants
+`com.apple.developer.carplay-voice-based-conversation`, which is a per-app request.
+
+### Lane C: the open head unit (designed, not built)
+
+`node-carplay` plus a Carlinkit CPC200 dongle renders a real CarPlay or Android Auto session
+inside a browser surface we own, on a Raspberry Pi or any Linux box. In that lane there is no
+entitlement, no template vocabulary and no exclusive-map rule: the agent can stand beside
+the projected phone screen at full size. This is where the original idea, the agent as the
+face of the car, is legal today. It needs hardware to build against, so it is scoped and
+deliberately not half-built.
+
+### Explicitly not building
+
+- **A navigation app.** Shipping turn-by-turn to obtain a render surface, then using that
+ surface for something other than a map, is exactly what the exclusive-map rule forbids.
+ If three.ws ever ships navigation it will be because navigation is the product.
+- **A wake word.** Apple's category forbids it and the platform is better without one.
+- **Our own dongle protocol.** `node-carplay` already speaks it.
+- **A second voice stack.** The talk loop is the talk loop.
+
+---
+
+## 4. What `/drive` actually does
+
+Route: [`/drive`](https://three.ws/drive). Page: `pages/drive.html`. Code: `src/drive/`.
+
+The loop is the platform's existing one, unchanged: microphone into `/api/asr` (NVIDIA Riva,
+with the browser recognizer as the fallback lane), transcript into `/api/chat`, reply into
+`/api/tts`, audio into the lipsync driver and onto a real rigged avatar. What `/drive` adds
+is everything that makes that safe at speed.
+
+- **One line, not a transcript.** The reply is clamped to two lines and spoken in full. What
+ was heard sits above it in a smaller, dimmer line so a misheard word is obvious.
+- **Four controls, no more.** Hands free, Repeat, Stop, Type. The same four are mirrored to
+ the car screen, which is why there are four: it is Apple's ceiling for the voice template's
+ action buttons, and designing to the tightest surface keeps the others honest.
+- **A keyboard that disappears.** The page starts in Driving and hides text entry. If the
+ browser reports a real ground speed, moving above walking pace locks it back and closes
+ anything open; a sustained stop lets the driver switch to Parked themselves. With no speed
+ signal the toggle is manual and still defaults to Driving. On CarPlay and Android Auto the
+ keyboard is not offered at all, because the web view is on a phone in a mount.
+- **Local commands with no round trip.** "Repeat that", "stop talking", "louder", "night
+ mode", "I'm parked" are answered by the page itself in `src/drive/commands.js`. Matching is
+ against the whole utterance, never a substring, so "stop at the next charger" still reaches
+ the agent.
+- **Hands free that actually ends a turn.** On the Riva lane the page reads the real
+ captured mic level and closes the turn after roughly a second of quiet, capped at fifteen
+ seconds. On the browser recognizer lane the recognizer ends its own turn and the page
+ simply re-arms.
+- **Day and night.** Night is the default because a bright panel at speed is glare. Day is
+ one press, or one sentence.
+- **A surface preset per panel.** `src/drive/surface.js` scales every size from the viewport
+ and the panel type: a head unit is about a metre from the driver's eyes and gets larger
+ glyphs, an 800x480 dash gets a one-line reply and a shorter deck.
+
+### What it fixed on the way
+
+The talk loop's free voice lane, `/api/tts/edge`, requires a signed-in session. Every
+anonymous listener on the platform, a public avatar page, an embed, and now a car, got a
+working reply and complete silence. `TalkController` now falls through to `/api/tts/speak`,
+the free NVIDIA Magpie lane that has no such gate, and latches the 401 so it costs one
+wasted request per session rather than one per turn. `hush()` and `setVolume()` landed on the
+same class, which is what makes barge-in and "louder" real rather than decorative.
+
+---
+
+## 5. Build plan
+
+| Phase | Work | State |
+|---|---|---|
+| 1 | `/drive`: the surface, the voice loop, the safety rules, the presets | **Done.** Verified in Chromium at 800x480, 1280x720, 1920x720 and 390x844, day and night, with a real `/api/chat` and `/api/tts` turn. |
+| 2 | Native CarPlay scene, audio session, message channel, entitlement, Info.plist | **Done in the tree.** Cannot be compiled or run without Xcode and a car or the CarPlay simulator. |
+| 3 | Apple's entitlement request | **Owner action.** See `ios/docs/CARPLAY.md`. Nothing else blocks. |
+| 4 | Android Auto: a `CarAppService` in `solana-mobile/`, same four controls, same channel | Scoped, not started. |
+| 5 | Lane C head unit: `node-carplay` beside the agent on a Pi | Scoped, needs a CPC200 dongle. |
+| 6 | The car as a home trigger: "we are ten minutes out" through `@three-ws/home-bridge` | Scoped. Both halves exist. |
+
+---
+
+## 6. What was actually verified
+
+Run against the live production API through the dev server's `/api` proxy, in headless
+Chromium:
+
+- The copilot picker loads real avatars from `/api/avatars/featured` and falls back from
+ `/api/avatars` cleanly when nobody is signed in (401, no error shown to the user).
+- Selecting an agent mounts the real GLB, binds the mouth target, and drives the shared
+ retargeted idle clip, so no avatar stands in bind pose.
+- A full turn: typed input into `/api/chat` (200), reply rendered and clamped, `/api/tts/edge`
+ refused (401), `/api/tts/speak` served the audio (200), state moved `thinking -> speaking
+ -> idle`, and Repeat became available. No console errors.
+- A local command ("night mode") applied with zero network calls.
+- The native bridge: `window.threeWsDrive.command({ type: 'repeat' })` re-spoke the last
+ reply, which is the exact call `DriveLink.swift` makes.
+- 17 unit tests in `tests/drive-surface.test.js` cover the surface presets, the command
+ matcher (including the phrases it must NOT match), and the motion gate.
+
+Not verified, and stated as such: the Swift has not been compiled, no CarPlay session has
+been connected, and no Carlinkit dongle has been touched.
+
+---
+
+## 7. Licensing
+
+Everything adopted or planned is permissive: MIT (`node-carplay`, and
+`react-native-carplay` as a reference), Apache-2.0 (`androidx.car.app`), BSD-2 (MapLibre).
+The GPL-3.0 projects in this space, OpenAuto and `aasdk`, are excluded on both counts:
+copyleft we cannot link into a commercial product, and no meaningful maintenance.
+
+The obligation runs the other way too. `node-carplay` is one maintainer's work holding up an
+entire category of open head units. If the head unit lane lands and we fix something in it,
+the fix goes upstream.
+
+---
+
+## Sources
+
+- [Apple: Requesting CarPlay entitlements](https://developer.apple.com/documentation/carplay/requesting-carplay-entitlements)
+- [Apple: CarPlay for developers](https://developer.apple.com/carplay/)
+- [Apple: CPTemplateApplicationScene](https://developer.apple.com/documentation/carplay/cptemplateapplicationscene)
+- [Apple: Turbocharge your app for CarPlay (WWDC25)](https://developer.apple.com/videos/play/wwdc2025/216/)
+- [Apple: Get more mileage out of your app with CarPlay (WWDC22)](https://developer.apple.com/videos/play/wwdc2022/10016/)
+- [Mapbox: CarPlay on iOS 27 lets a voice assistant run over a live map](https://www.mapbox.com/blog/carplay-on-ios-27-lets-a-voice-assistant-run-over-a-live-map)
+- [MacRumors: iOS 26.4 brings CarPlay support for conversational assistants](https://www.macrumors.com/2026/02/18/ios-26-4-carplay-support/)
+- [Android: Use the Android for Cars App Library](https://developer.android.com/training/cars/apps/library)
+- [Android: Draw maps on the navigation surface](https://developer.android.com/training/cars/apps/library/draw-maps)
+- [rhysmorgan134/node-CarPlay](https://github.com/rhysmorgan134/node-CarPlay)
+- [birkir/react-native-carplay](https://github.com/birkir/react-native-carplay)
+- [maplibre/MapLibre-Android-Auto-Sample](https://github.com/maplibre/MapLibre-Android-Auto-Sample)
+- [f1xpl/openauto](https://github.com/f1xpl/openauto)
diff --git a/docs/start-here.md b/docs/start-here.md
index a43c61a52..a60132dd4 100644
--- a/docs/start-here.md
+++ b/docs/start-here.md
@@ -121,6 +121,7 @@ Want the model in your hands rather than on a screen? [Materialize](./materializ
- **Rendering avatars outside the viewer?** → [Media and render API](./media-api.md): avatar PNG renders for `<img>` tags, posed clip renders, the runtime GLB optimizer, the CORS-open model proxy, and the free vision and speech-to-text endpoints; companion to the [3D API](./3d-api.md)
- **Want the avatar in your terminal, or as your coding agent's face?** → [Terminal avatar](./tty-avatar.md): `npx @three-ws/tty-avatar <avatar id>` draws any avatar or agent body live in a terminal (truecolor half-blocks, braille, or plain ASCII, no browser or GPU), and `install-hooks --write` ties its moods to Claude Code so it thinks while the agent reads, works while it edits, and celebrates when it finishes
- **Want the agent to reach past the screen?** → [three.ws Home](./smart-home.md): the plan for an agent that runs your house, and the [`@three-ws/home-bridge`](../packages/home-bridge/README.md) package that already connects one to a real Home Assistant. Live state from your rooms so a 3D scene dims when your real lights dim, "good night" resolved to the scene you already built yourself, and a gate that stops and asks before anything unlocks a door
+- **Driving?** → [three.ws Drive](./carplay.md): the same agent, voice-first, on a car panel. [`/drive`](https://three.ws/drive) runs today on a head unit browser or a phone in the cradle; the native CarPlay scene is built and waiting on Apple's voice-assistant entitlement ([ios/docs/CARPLAY.md](../ios/docs/CARPLAY.md)). One glanceable line, four controls, and a keyboard that locks itself while the wheels turn
- **Putting avatars in your build pipeline?** → [Avatar CLI](./avatar-cli.md): four offline commands that scaffold a hash-anchored avatar manifest from a wallet and a GLB, validate it against the published schema, and print an embed snippet that renders. Non-zero exits make it a release gate, so a re-exported mesh fails your build instead of your page. [Ship an avatar manifest](./tutorials/ship-an-avatar-manifest.md) is the guided walkthrough
- **Uploaded an avatar that loads but never moves?** → [Rig Doctor](./rig-doctor.md): drop the `.glb` and find out in a second why. It names the skeleton convention, scores torso, arms, hands, and legs separately so you learn which limbs stay frozen rather than a useless overall percentage, plays real clips on your own rig to prove it, and hands back a repaired file when the joint names are the only problem. Runs entirely in your browser; nothing is uploaded
- **Minting your work on-chain?** → [Minted 3D assets](./minted.md): the /minted gallery of avatars minted as Metaplex Core NFTs, the opt-in mint flow, baked provenance, and enforced remix royalties
diff --git a/ios/README.md b/ios/README.md
index 5f96ad89a..34590bcf1 100644
--- a/ios/README.md
+++ b/ios/README.md
@@ -23,12 +23,27 @@ ios/
├── native/App/ # the generated Xcode project (committed)
│ ├── App.xcodeproj
│ ├── App/MainViewController.swift # swipe-back, dark chrome, edge-to-edge insets
-│ ├── App/Info.plist # usage strings, URL scheme, orientations
-│ ├── App/App.entitlements # associated domains + APNs
+│ ├── App/CarPlaySceneDelegate.swift # the car screen: templates + voice control
+│ ├── App/DriveLink.swift # CarPlay <-> /drive channel + audio session
+│ ├── App/Info.plist # usage strings, URL scheme, orientations, scenes
+│ ├── App/App.entitlements # associated domains + APNs + CarPlay
│ └── CapApp-SPM/ # Swift Package Manager plugin graph
-└── docs/ # SUBMISSION.md, REVIEW-RISK.md, ASSETS.md
+└── docs/ # SUBMISSION.md, REVIEW-RISK.md, ASSETS.md, CARPLAY.md
```
+## CarPlay
+
+The app carries a second scene for **three.ws Drive**, the agent in the car. Apple's
+voice-based conversational category grants templates and an audio session and no drawing
+surface, so the car screen shows four controls and the Voice Control template while the
+phone's WebView runs `/drive` with the actual agent in it. The two are joined by the
+`threeWsDrive` WebKit message channel.
+
+It is inert without the `com.apple.developer.carplay-voice-based-conversation` entitlement,
+which Apple grants per app on request, so nothing about the phone app changes before then.
+Read [`docs/CARPLAY.md`](docs/CARPLAY.md) before touching any of it, and
+[`../docs/carplay.md`](../docs/carplay.md) for why the architecture is shaped this way.
+
## Why the WebView loads the live site
The alternative is baking `dist/` into the `.ipa` and serving it from
diff --git a/ios/docs/CARPLAY.md b/ios/docs/CARPLAY.md
new file mode 100644
index 000000000..a98979145
--- /dev/null
+++ b/ios/docs/CARPLAY.md
@@ -0,0 +1,105 @@
+# three.ws Drive on CarPlay
+
+The car screen half of [`/drive`](../../docs/carplay.md). Read that document first: it
+explains why the architecture is what it is, and why the agent's 3D face is **not** on the
+CarPlay screen.
+
+This file covers the native side only: what is in the tree, what Apple has to grant before
+any of it runs, and how to exercise it once they do.
+
+## The one-paragraph version
+
+three.ws declares CarPlay's **voice-based conversational app** category, added in iOS 26.4.
+That category gives an app a template UI and an audio session, and no drawing surface. So the
+car screen shows four controls and Apple's Voice Control template, the phone's WebView runs
+`https://three.ws/drive?surface=carplay` with the real agent in it, and a WebKit message
+channel keeps the two in step. Nothing about the phone app changes when CarPlay is absent.
+
+## What is in the tree
+
+| File | Role |
+|---|---|
+| `native/App/App/CarPlaySceneDelegate.swift` | The CarPlay scene. Root `CPListTemplate` with the four controls, `CPVoiceControlTemplate` presented for the length of a turn. |
+| `native/App/App/DriveLink.swift` | The channel. Registers the `threeWsDrive` WebKit message handler, sends commands back with `evaluateJavaScript`, and owns the `AVAudioSession`. |
+| `native/App/App/AppDelegate.swift` | Routes a `carTemplateApplication` scene session to `CarPlaySceneDelegate`. |
+| `native/App/App/MainViewController.swift` | Attaches `DriveLink` to the Capacitor WebView on load. Inert until a CarPlay scene connects. |
+| `native/App/App/Info.plist` | `UIApplicationSupportsMultipleScenes` is now `true` (CarPlay is a second scene), plus the `CPTemplateApplicationSceneSessionRoleApplication` configuration. |
+| `native/App/App/App.entitlements` | `com.apple.developer.carplay-voice-based-conversation`. |
+| `../../src/drive/bridge.js` | The web half of the same protocol. Change one side, change both. |
+
+## The protocol
+
+Version 1. The page posts to `window.webkit.messageHandlers.threeWsDrive`; every message
+carries `{ v: 1, type }`.
+
+| Page to car | Payload | Effect |
+|---|---|---|
+| `ready` | `{ agent: { id, name } }` | The agent's name becomes the list section header |
+| `state` | `{ state }` (`idle`, `listening`, `transcribing`, `thinking`, `speaking`) | Presents, updates, or dismisses the voice template, and opens or closes the audio session |
+| `heard` | `{ text }` | Last transcript, capped at 240 characters |
+| `said` | `{ text }` | Last reply, capped at 500, shown as the Repeat row's detail |
+| `error` | `{ code, message }` | Held for the next render |
+| `actions` | `{ actions: [{ id, label, enabled }] }` | Up to four; rebuilds the list |
+
+| Car to page | Effect |
+|---|---|
+| `talk-start` / `talk-stop` / `talk` | Begin, end, or toggle a listening turn |
+| `hands` | Toggle hands free |
+| `repeat` | Speak the last reply again |
+| `hush` | Barge in and stop speaking |
+| `say` (with `value`) | Send a turn as text |
+
+## What Apple requires, and how this meets it
+
+The voice category's rules are not style guidance; a build that misses one does not get
+approved. Each is met in code, not by intention:
+
+| Rule | Where |
+|---|---|
+| Voice is the primary modality on launch | `didConnect` sends `talk-start` immediately after setting the root template |
+| No wake word | Nothing listens until a press or an explicit hands-free choice |
+| Audio session open only while voice is in use | `DriveLink.syncAudioSession()` is driven by the page's own state and deactivates on `idle` |
+| Template depth of three or less | One: a root list, with the voice template presented over it |
+| The voice template accompanies active audio input | Presented on a non-idle state, dismissed the moment the turn ends |
+
+## Getting the entitlement (owner action, nothing else is blocked)
+
+1. Request it at <https://developer.apple.com/carplay/>, choosing the **voice-based
+ conversational app** category. Apple reviews each request per app.
+2. The request asks what the app does in the car. The honest answer is the pitch: a
+ conversational assistant with a persistent identity, launched by hand, voice in and voice
+ out, no navigation and no wake word.
+3. On approval, enable the capability for bundle ID `ws.three.app` in the Developer portal
+ and regenerate the provisioning profile. The entitlement key is already in
+ `App.entitlements`, so no code changes.
+
+Until then the key is harmless: a build signed without the grant simply never receives a
+CarPlay scene, and the phone app behaves exactly as it does today.
+
+## Testing it
+
+Needs macOS with Xcode, so it cannot be done from this repo's Linux workspace.
+
+- **Simulator:** run the app on an iOS simulator, then open Xcode's **I/O, External
+ Displays, CarPlay** window. The CarPlay scene connects against the simulator without an
+ entitlement grant, which is enough to exercise the templates, the message channel, and
+ every state transition.
+- **A real car or aftermarket head unit:** needs the granted entitlement and a provisioning
+ profile that carries it.
+- **The web half alone:** `npm run dev`, then open
+ `http://localhost:3000/drive?surface=carplay`. With no native shell attached, every send is
+ a no-op and `window.threeWsDrive.command({ type: 'repeat' })` in the console drives the
+ page exactly as the Swift does.
+
+## Review risk
+
+Read [`REVIEW-RISK.md`](REVIEW-RISK.md) for the app's existing exposure. CarPlay adds two
+items:
+
+- **Category fit.** A conversational app that spends its time doing something other than
+ conversing invites a rejection. `/drive` is voice in, voice out, four controls. It does not
+ navigate, play media, or take payments on the car screen.
+- **The keyboard.** Text entry exists on `/drive` for a parked car and a passenger, and it is
+ hidden outright on the `carplay` surface preset because the WebView is on a phone in a
+ mount. Do not "improve" that by exposing it: it is the single most obvious distraction
+ finding available to a reviewer.
diff --git a/ios/native/App/App.xcodeproj/project.pbxproj b/ios/native/App/App.xcodeproj/project.pbxproj
index 911787bdc..02ddb8da8 100644
--- a/ios/native/App/App.xcodeproj/project.pbxproj
+++ b/ios/native/App/App.xcodeproj/project.pbxproj
@@ -17,6 +17,8 @@
504EC3121FED79650016851F /* LaunchScreen.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 504EC3101FED79650016851F /* LaunchScreen.storyboard */; };
50B271D11FEDC1A000F3C39B /* public in Resources */ = {isa = PBXBuildFile; fileRef = 50B271D01FEDC1A000F3C39B /* public */; };
9582B6832FE993A70072D4E8 /* SceneDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9582B6822FE993A50072D4E8 /* SceneDelegate.swift */; };
+ C4A11E0130A1000200000000 /* CarPlaySceneDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = C4A11E0130A1000100000000 /* CarPlaySceneDelegate.swift */; };
+ C4A11E0130A1000400000000 /* DriveLink.swift in Sources */ = {isa = PBXBuildFile; fileRef = C4A11E0130A1000300000000 /* DriveLink.swift */; };
/* End PBXBuildFile section */
/* Begin PBXFileReference section */
@@ -31,6 +33,8 @@
504EC3131FED79650016851F /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; path = Info.plist; sourceTree = "<group>"; };
50B271D01FEDC1A000F3C39B /* public */ = {isa = PBXFileReference; lastKnownFileType = folder; path = public; sourceTree = "<group>"; };
9582B6822FE993A50072D4E8 /* SceneDelegate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SceneDelegate.swift; sourceTree = "<group>"; };
+ C4A11E0130A1000100000000 /* CarPlaySceneDelegate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CarPlaySceneDelegate.swift; sourceTree = "<group>"; };
+ C4A11E0130A1000300000000 /* DriveLink.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = DriveLink.swift; sourceTree = "<group>"; };
958DCC722DB07C7200EA8C5F /* debug.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; name = debug.xcconfig; path = ../debug.xcconfig; sourceTree = SOURCE_ROOT; };
/* End PBXFileReference section */
@@ -67,6 +71,8 @@
isa = PBXGroup;
children = (
9582B6822FE993A50072D4E8 /* SceneDelegate.swift */,
+ C4A11E0130A1000100000000 /* CarPlaySceneDelegate.swift */,
+ C4A11E0130A1000300000000 /* DriveLink.swift */,
50379B222058CBB4000EE86E /* capacitor.config.json */,
504EC3071FED79650016851F /* AppDelegate.swift */,
7A31B0C14E5F41A290C30011 /* MainViewController.swift */,
@@ -164,6 +170,8 @@
504EC3081FED79650016851F /* AppDelegate.swift in Sources */,
7A31B0C14E5F41A290C30012 /* MainViewController.swift in Sources */,
9582B6832FE993A70072D4E8 /* SceneDelegate.swift in Sources */,
+ C4A11E0130A1000200000000 /* CarPlaySceneDelegate.swift in Sources */,
+ C4A11E0130A1000400000000 /* DriveLink.swift in Sources */,
);
runOnlyForDeploymentPostprocessing = 0;
};
diff --git a/ios/native/App/App/App.entitlements b/ios/native/App/App/App.entitlements
index 733fb1d30..5c21787ec 100644
--- a/ios/native/App/App/App.entitlements
+++ b/ios/native/App/App/App.entitlements
@@ -24,5 +24,16 @@
-->
<key>aps-environment</key>
<string>development</string>
+
+ <!--
+ CarPlay, voice-based conversational category (iOS 26.4 and later). This
+ is what puts three.ws Drive on the car screen. Apple grants it per app
+ on request (https://developer.apple.com/carplay/), and a build signed
+ without the grant simply never sees a CarPlay scene: the phone app is
+ unaffected either way, so this key is safe to carry before approval
+ lands. See ios/docs/CARPLAY.md for the request and the review notes.
+ -->
+ <key>com.apple.developer.carplay-voice-based-conversation</key>
+ <true/>
</dict>
</plist>
diff --git a/ios/native/App/App/AppDelegate.swift b/ios/native/App/App/AppDelegate.swift
index fec95acae..f5293d6f4 100644
--- a/ios/native/App/App/AppDelegate.swift
+++ b/ios/native/App/App/AppDelegate.swift
@@ -36,6 +36,17 @@ class AppDelegate: UIResponder, UIApplicationDelegate {
func application(_ application: UIApplication,
configurationForConnecting connectingSceneSession: UISceneSession,
options: UIScene.ConnectionOptions) -> UISceneConfiguration {
+ // CarPlay connects as its own scene with its own delegate. Without this
+ // branch the car screen would try to build the phone's WebView scene,
+ // which CarPlay refuses, and three.ws Drive would simply never appear.
+ // See ios/docs/CARPLAY.md and CarPlaySceneDelegate.swift.
+ if connectingSceneSession.role == UISceneSession.Role.carTemplateApplication {
+ let config = UISceneConfiguration(name: "CarPlay Configuration",
+ sessionRole: connectingSceneSession.role)
+ config.delegateClass = CarPlaySceneDelegate.self
+ return config
+ }
+
let config = UISceneConfiguration(name: "Default Configuration",
sessionRole: connectingSceneSession.role)
config.delegateClass = SceneDelegate.self
diff --git a/ios/native/App/App/CarPlaySceneDelegate.swift b/ios/native/App/App/CarPlaySceneDelegate.swift
new file mode 100644
index 000000000..2d340d855
--- /dev/null
+++ b/ios/native/App/App/CarPlaySceneDelegate.swift
@@ -0,0 +1,212 @@
+import CarPlay
+import UIKit
+
+/// three.ws Drive on the CarPlay screen.
+///
+/// This is a **voice-based conversational app**, the CarPlay category Apple
+/// opened in iOS 26.4 for assistants like this one. What that category grants
+/// is a template UI and an audio session; what it does not grant is a drawing
+/// surface, so the agent's 3D face is not here. It is on the phone, in the
+/// `/drive` page, which `DriveLink` drives. The car screen is the control
+/// surface: what the agent is doing right now, and the few presses that beat
+/// speaking.
+///
+/// The rules the category imposes, and how this file meets them:
+///
+/// - **Voice is the primary modality on launch.** Connecting activates the
+/// Talk row's behaviour immediately: the page starts listening as soon as
+/// the driver launches us from the CarPlay home screen.
+/// - **No wake word.** Nothing here listens until a press or the driver's
+/// hands-free choice says so.
+/// - **An audio session only while voice is in use.** `DriveLink` opens and
+/// closes it from the page's own state, not for the length of the trip.
+/// - **Template depth of three.** This uses one: a root list, with the voice
+/// template presented over it during a turn.
+///
+/// Requires the `com.apple.developer.carplay-voice-based-conversation`
+/// entitlement (App.entitlements), which Apple grants per app on request.
+/// Without it the scene never connects and the rest of the app is unaffected.
+final class CarPlaySceneDelegate: UIResponder, CPTemplateApplicationSceneDelegate {
+
+ private var interfaceController: CPInterfaceController?
+ private var voiceTemplatePresented = false
+
+ // Identifiers for the voice template's states, kept in one place because
+ // activating an identifier the template does not carry is a silent no-op.
+ private enum VoiceState: String {
+ case listening
+ case working
+ case speaking
+ }
+
+ // MARK: - Scene lifecycle
+
+ func templateApplicationScene(
+ _ templateApplicationScene: CPTemplateApplicationScene,
+ didConnect interfaceController: CPInterfaceController
+ ) {
+ self.interfaceController = interfaceController
+
+ DriveLink.shared.onChange = { [weak self] in
+ DispatchQueue.main.async { self?.render() }
+ }
+ DriveLink.shared.openDriveSurface()
+
+ interfaceController.setRootTemplate(makeRootTemplate(), animated: false, completion: nil)
+ // Voice first: the driver launched an assistant, so it starts listening
+ // rather than waiting to be told twice.
+ DriveLink.shared.send("talk-start")
+ }
+
+ func templateApplicationScene(
+ _ templateApplicationScene: CPTemplateApplicationScene,
+ didDisconnectInterfaceController interfaceController: CPInterfaceController
+ ) {
+ DriveLink.shared.send("talk-stop")
+ DriveLink.shared.onChange = nil
+ DriveLink.shared.closeDriveSurface()
+ DriveLink.shared.reset()
+ self.interfaceController = nil
+ voiceTemplatePresented = false
+ }
+
+ // MARK: - Root list
+
+ /// The four controls, in the order a driver reaches for them. They mirror
+ /// the deck on the `/drive` page exactly, so the car screen and the phone
+ /// never disagree about what is available.
+ private func makeRootTemplate() -> CPListTemplate {
+ let template = CPListTemplate(title: "three.ws Drive", sections: [makeSection()])
+ template.emptyViewTitleVariants = ["three.ws Drive"]
+ template.emptyViewSubtitleVariants = ["Say what you need. Your agent is listening."]
+ return template
+ }
+
+ private func makeSection() -> CPListSection {
+ let link = DriveLink.shared
+ let fallback: [DriveLink.Action] = [
+ .init(id: "talk", label: "Talk", enabled: true),
+ .init(id: "hands", label: "Hands free", enabled: true),
+ .init(id: "repeat", label: "Repeat", enabled: false),
+ .init(id: "hush", label: "Quiet", enabled: false),
+ ]
+ // Until the page has published its first action set, show the same four
+ // in their resting state rather than an empty screen.
+ let actions = link.actions.isEmpty ? fallback : link.actions
+
+ let items: [CPListItem] = actions.map { action in
+ let item = CPListItem(text: action.label, detailText: detail(for: action))
+ item.isEnabled = action.enabled
+ item.handler = { [weak self] _, completion in
+ self?.perform(action.id)
+ completion()
+ }
+ return item
+ }
+ return CPListSection(items: items, header: link.agentName, sectionIndexTitle: nil)
+ }
+
+ /// One short line of context per row. A driver reads at most this much.
+ private func detail(for action: DriveLink.Action) -> String? {
+ let link = DriveLink.shared
+ switch action.id {
+ case "talk":
+ return link.state == .listening ? "Listening" : "Ask your agent something"
+ case "hands":
+ return "Keep the mic open between turns"
+ case "repeat":
+ return link.lastSpoken
+ case "hush":
+ return link.state == .speaking ? "Stop talking" : nil
+ default:
+ return nil
+ }
+ }
+
+ private func perform(_ id: String) {
+ switch id {
+ case "talk":
+ DriveLink.shared.send(DriveLink.shared.state == .listening ? "talk-stop" : "talk-start")
+ case "hands":
+ DriveLink.shared.send("hands")
+ case "repeat":
+ DriveLink.shared.send("repeat")
+ case "hush":
+ DriveLink.shared.send("hush")
+ default:
+ break
+ }
+ }
+
+ // MARK: - Voice template
+
+ /// Presented for the length of a turn and dismissed the moment it ends,
+ /// which is the contract CarPlay places on this template: it exists while
+ /// audio input is in use, not as a screen the driver sits on.
+ private func makeVoiceTemplate() -> CPVoiceControlTemplate {
+ CPVoiceControlTemplate(voiceControlStates: [
+ CPVoiceControlState(
+ identifier: VoiceState.listening.rawValue,
+ titleVariants: ["Listening", "Go ahead"],
+ image: nil,
+ repeats: true
+ ),
+ CPVoiceControlState(
+ identifier: VoiceState.working.rawValue,
+ titleVariants: ["Thinking", "One moment"],
+ image: nil,
+ repeats: true
+ ),
+ CPVoiceControlState(
+ identifier: VoiceState.speaking.rawValue,
+ titleVariants: ["Speaking"],
+ image: nil,
+ repeats: false
+ ),
+ ])
+ }
+
+ private func voiceState(for state: DriveLink.State) -> VoiceState? {
+ switch state {
+ case .listening: return .listening
+ case .transcribing, .thinking: return .working
+ case .speaking: return .speaking
+ case .idle: return nil
+ }
+ }
+
+ // MARK: - Rendering
+
+ private func render() {
+ guard let interfaceController else { return }
+
+ if let root = interfaceController.rootTemplate as? CPListTemplate {
+ root.updateSections([makeSection()])
+ }
+
+ let link = DriveLink.shared
+ guard let target = voiceState(for: link.state) else {
+ if voiceTemplatePresented {
+ voiceTemplatePresented = false
+ interfaceController.dismissTemplate(animated: true, completion: nil)
+ }
+ return
+ }
+
+ if voiceTemplatePresented {
+ (interfaceController.presentedTemplate as? CPVoiceControlTemplate)?
+ .activateVoiceControlState(withIdentifier: target.rawValue)
+ return
+ }
+
+ voiceTemplatePresented = true
+ let template = makeVoiceTemplate()
+ interfaceController.presentTemplate(template, animated: true) { presented, _ in
+ guard presented else {
+ self.voiceTemplatePresented = false
+ return
+ }
+ template.activateVoiceControlState(withIdentifier: target.rawValue)
+ }
+ }
+}
diff --git a/ios/native/App/App/DriveLink.swift b/ios/native/App/App/DriveLink.swift
new file mode 100644
index 000000000..b30491697
--- /dev/null
+++ b/ios/native/App/App/DriveLink.swift
@@ -0,0 +1,194 @@
+import AVFoundation
+import Foundation
+import WebKit
+
+/// The single channel between the CarPlay templates on the car screen and the
+/// `/drive` page running in the app's WebView on the phone.
+///
+/// CarPlay's voice-based conversational category does not hand an app a drawing
+/// surface: the car screen is Apple's templates, and nothing else. So the
+/// conversation, the agent, its voice and its 3D face all stay in the WebView,
+/// and this object is the wire between them. The page posts its state out
+/// (`window.webkit.messageHandlers.threeWsDrive`), and template presses go back
+/// in as `window.threeWsDrive.command({ type })`. Both halves of that protocol
+/// live in `src/drive/bridge.js`.
+///
+/// It also owns the audio session, because CarPlay requires a conversational
+/// app to hold one open only while voice is actually in use.
+final class DriveLink: NSObject {
+
+ static let shared = DriveLink()
+
+ enum State: String {
+ case idle, listening, transcribing, thinking, speaking
+
+ /// Is a turn in flight? The car screen shows the voice template for
+ /// exactly this window and returns to the list outside it.
+ var isActive: Bool { self != .idle }
+ }
+
+ struct Action {
+ let id: String
+ let label: String
+ let enabled: Bool
+ }
+
+ /// Name of the WebKit message handler. Must match `src/drive/bridge.js`.
+ private static let channel = "threeWsDrive"
+ private static let protocolVersion = 1
+
+ private(set) var state: State = .idle
+ private(set) var agentName: String?
+ private(set) var lastSpoken: String?
+ private(set) var lastHeard: String?
+ private(set) var lastError: String?
+ private(set) var actions: [Action] = []
+
+ /// Called on the main queue whenever anything above changes.
+ var onChange: (() -> Void)?
+
+ private weak var webView: WKWebView?
+ private var audioSessionActive = false
+
+ // MARK: - Attachment
+
+ /// Register on the app's WebView. Safe to call more than once.
+ func attach(to webView: WKWebView) {
+ if self.webView === webView { return }
+ detach()
+ self.webView = webView
+ let controller = webView.configuration.userContentController
+ controller.removeScriptMessageHandler(forName: Self.channel)
+ controller.add(self, name: Self.channel)
+ }
+
+ func detach() {
+ webView?.configuration.userContentController.removeScriptMessageHandler(forName: Self.channel)
+ webView = nil
+ }
+
+ // MARK: - Outbound: car screen to page
+
+ /// Send a command the page understands. See `onNativeCommand` in
+ /// `src/drive/index.js` for the accepted types.
+ func send(_ type: String, value: String? = nil) {
+ guard let webView else { return }
+ var payload: [String: Any] = ["type": type]
+ if let value { payload["value"] = value }
+ guard
+ let data = try? JSONSerialization.data(withJSONObject: payload),
+ let json = String(data: data, encoding: .utf8)
+ else { return }
+ // Optional chaining on the page side: a command that arrives before the
+ // page has booted is dropped rather than throwing into the console.
+ webView.evaluateJavaScript("window.threeWsDrive?.command(\(json));", completionHandler: nil)
+ }
+
+ /// Point the phone's WebView at the car surface. Called when the CarPlay
+ /// scene connects, which only happens when the driver launches three.ws
+ /// from the CarPlay home screen: navigating then is what they asked for.
+ /// The previous page is remembered so disconnecting puts them back.
+ func openDriveSurface() {
+ guard let webView else { return }
+ let current = webView.url
+ if current?.path.hasPrefix("/drive") != true {
+ returnURL = current
+ }
+ var components = URLComponents()
+ components.scheme = "https"
+ components.host = current?.host ?? "three.ws"
+ components.path = "/drive"
+ components.queryItems = [URLQueryItem(name: "surface", value: "carplay")]
+ guard let url = components.url else { return }
+ webView.load(URLRequest(url: url))
+ }
+
+ /// Put the phone back where the driver left it when CarPlay disconnects.
+ func closeDriveSurface() {
+ guard let webView, let returnURL else { return }
+ self.returnURL = nil
+ webView.load(URLRequest(url: returnURL))
+ }
+
+ private var returnURL: URL?
+
+ // MARK: - Audio session
+
+ /// CarPlay requires a conversational app to hold an audio session only
+ /// while voice is actually in use, so this is driven by the page's own
+ /// state rather than held open for the whole trip.
+ private func syncAudioSession() {
+ let shouldBeActive = state.isActive
+ guard shouldBeActive != audioSessionActive else { return }
+ let session = AVAudioSession.sharedInstance()
+ do {
+ if shouldBeActive {
+ try session.setCategory(
+ .playAndRecord,
+ mode: .voiceChat,
+ options: [.allowBluetooth, .allowBluetoothA2DP, .duckOthers]
+ )
+ try session.setActive(true)
+ } else {
+ try session.setActive(false, options: .notifyOthersOnDeactivation)
+ }
+ audioSessionActive = shouldBeActive
+ } catch {
+ // A refused session must not take the conversation down: the page
+ // surfaces its own microphone error, and the next turn retries.
+ NSLog("[drive] audio session %@ failed: %@", shouldBeActive ? "activate" : "deactivate", error.localizedDescription)
+ }
+ }
+
+ // MARK: - Reset
+
+ func reset() {
+ state = .idle
+ agentName = nil
+ lastSpoken = nil
+ lastHeard = nil
+ lastError = nil
+ actions = []
+ syncAudioSession()
+ }
+}
+
+// MARK: - Inbound: page to car screen
+
+extension DriveLink: WKScriptMessageHandler {
+
+ func userContentController(_ controller: WKUserContentController, didReceive message: WKScriptMessage) {
+ guard message.name == Self.channel,
+ let body = message.body as? [String: Any],
+ (body["v"] as? Int) == Self.protocolVersion,
+ let type = body["type"] as? String
+ else { return }
+
+ switch type {
+ case "ready":
+ let agent = body["agent"] as? [String: Any]
+ agentName = agent?["name"] as? String
+ lastError = nil
+ case "state":
+ state = State(rawValue: body["state"] as? String ?? "") ?? .idle
+ syncAudioSession()
+ case "heard":
+ lastHeard = body["text"] as? String
+ case "said":
+ lastSpoken = body["text"] as? String
+ lastError = nil
+ case "error":
+ lastError = body["message"] as? String
+ case "actions":
+ let raw = body["actions"] as? [[String: Any]] ?? []
+ actions = raw.compactMap { entry in
+ guard let id = entry["id"] as? String, let label = entry["label"] as? String else { return nil }
+ return Action(id: id, label: label, enabled: (entry["enabled"] as? Bool) ?? true)
+ }
+ default:
+ return
+ }
+
+ onChange?()
+ }
+}
diff --git a/ios/native/App/App/Info.plist b/ios/native/App/App/Info.plist
index df4834a26..5594144e2 100644
--- a/ios/native/App/App/Info.plist
+++ b/ios/native/App/App/Info.plist
@@ -97,8 +97,14 @@
<key>UIApplicationSceneManifest</key>
<dict>
+ <!--
+ CarPlay is a SECOND scene running beside the phone's, so multiple
+ scenes have to be allowed. With this false the car screen and the
+ phone cannot be live at the same time and three.ws Drive never
+ connects. See CarPlaySceneDelegate.swift and ios/docs/CARPLAY.md.
+ -->
<key>UIApplicationSupportsMultipleScenes</key>
- <false/>
+ <true/>
<key>UISceneConfigurations</key>
<dict>
<key>UIWindowSceneSessionRoleApplication</key>
@@ -112,6 +118,15 @@
<string>Main</string>
</dict>
</array>
+ <key>CPTemplateApplicationSceneSessionRoleApplication</key>
+ <array>
+ <dict>
+ <key>UISceneConfigurationName</key>
+ <string>CarPlay Configuration</string>
+ <key>UISceneDelegateClassName</key>
+ <string>$(PRODUCT_MODULE_NAME).CarPlaySceneDelegate</string>
+ </dict>
+ </array>
</dict>
</dict>
<key>UILaunchStoryboardName</key>
diff --git a/ios/native/App/App/MainViewController.swift b/ios/native/App/App/MainViewController.swift
index fc123ccbf..7f08045dd 100644
--- a/ios/native/App/App/MainViewController.swift
+++ b/ios/native/App/App/MainViewController.swift
@@ -45,5 +45,10 @@ class MainViewController: CAPBridgeViewController {
// ios/src/native-bridge.js; without this the insets are always zero and
// that padding has nothing to react to.
webView?.scrollView.contentInsetAdjustmentBehavior = .never
+
+ // Wire the CarPlay channel. The car screen talks to /drive through this
+ // WebView and nothing else, so the handler has to be registered before
+ // a CarPlay scene can ever connect. It is inert until one does.
+ if let webView { DriveLink.shared.attach(to: webView) }
}
}
diff --git a/pages/drive.html b/pages/drive.html
new file mode 100644
index 000000000..2bc5a18b0
--- /dev/null
+++ b/pages/drive.html
@@ -0,0 +1,552 @@
+<!doctype html>
+<html lang="en" data-no-atlas>
+ <head>
+ <style>/* three.ws hidden-guard: make the `hidden` attribute authoritative; see scripts/lib/hidden-guard.mjs */[hidden]{display:none!important}</style>
+ <meta charset="utf-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover" />
+ <meta name="theme-color" content="#000000" />
+ <title>three.ws Drive · your 3D agent in the car</title>
+ <link rel="icon" href="/favicon.ico" sizes="any" />
+ <link rel="icon" type="image/svg+xml" href="/favicon.svg" />
+ <link rel="apple-touch-icon" href="/apple-touch-icon.png" />
+ <link rel="stylesheet" href="/fonts/fonts.css" />
+ <link rel="stylesheet" href="/tokens.css" />
+ <meta
+ name="description"
+ content="three.ws Drive: a voice-first 3D agent built for the car. Runs on CarPlay, Android Auto, a head unit, or a phone in the cradle."
+ />
+ <link rel="canonical" href="https://three.ws/drive" />
+
+ <script type="module" src="/src/drive/index.js"></script>
+
+ <style>
+ :root {
+ /* Cockpit scale. The surface preset overrides it on <html>: a 800x480
+ head unit needs bigger glyphs per pixel than a 1920x720 dash. */
+ --dr-scale: 1;
+ --dr-gutter: calc(var(--space-lg) * var(--dr-scale));
+ --dr-talk-size: calc(132px * var(--dr-scale));
+ /* Night is the default: car displays run dark and a bright panel at
+ speed is a glare hazard. Day mode swaps these three. */
+ --dr-ground: #05070b;
+ --dr-veil: rgba(5, 7, 11, 0.72);
+ --dr-ink: #ffffff;
+ --dr-ink-dim: rgba(255, 255, 255, 0.62);
+ --dr-line: rgba(255, 255, 255, 0.12);
+ --dr-fill: rgba(255, 255, 255, 0.07);
+ --dr-fill-hi: rgba(255, 255, 255, 0.14);
+ --dr-live: var(--success);
+ --dr-busy: var(--warn);
+ --dr-fault: var(--danger);
+ }
+ :root[data-drive-light='1'] {
+ --dr-ground: #eef1f5;
+ --dr-veil: rgba(238, 241, 245, 0.74);
+ --dr-ink: #0b0e13;
+ --dr-ink-dim: rgba(11, 14, 19, 0.62);
+ --dr-line: rgba(11, 14, 19, 0.16);
+ --dr-fill: rgba(11, 14, 19, 0.06);
+ --dr-fill-hi: rgba(11, 14, 19, 0.13);
+ --dr-live: #15803d;
+ --dr-busy: #a16207;
+ --dr-fault: #b91c1c;
+ }
+ *, *::before, *::after { box-sizing: border-box; }
+ html, body {
+ margin: 0;
+ padding: 0;
+ height: 100%;
+ overflow: hidden;
+ background: var(--dr-ground);
+ color: var(--dr-ink);
+ font-family: var(--font-body, 'Inter', system-ui, sans-serif);
+ -webkit-font-smoothing: antialiased;
+ overscroll-behavior: none;
+ touch-action: manipulation;
+ -webkit-tap-highlight-color: transparent;
+ }
+ body { user-select: none; }
+
+ .dr {
+ position: fixed;
+ inset: 0;
+ display: grid;
+ grid-template-rows: auto 1fr auto;
+ padding:
+ max(var(--dr-gutter), env(safe-area-inset-top))
+ max(var(--dr-gutter), env(safe-area-inset-right))
+ max(var(--dr-gutter), env(safe-area-inset-bottom))
+ max(var(--dr-gutter), env(safe-area-inset-left));
+ gap: calc(var(--space-md) * var(--dr-scale));
+ }
+
+ /* The agent fills the panel behind everything; the chrome floats over it.
+ In landscape the render area is pushed right of centre so the spoken
+ line gets a clean column of its own instead of sitting on a chest. */
+ .dr-stage { position: fixed; inset: 0; z-index: 0; }
+ .dr-stage canvas { display: block; }
+ @media (orientation: landscape) {
+ .dr-stage { left: 16%; right: -4%; }
+ /* Keep the spoken line inside its own column instead of running it
+ across the agent standing to the right of it. */
+ .dr-mid { max-width: min(62ch, 62%); }
+ }
+ /* The scrim is composed around the agent, not over it: a light wash
+ under the top rail, a heavy one under the deck, and in landscape a
+ left-hand gradient that carries the text column. */
+ .dr::after {
+ content: '';
+ position: fixed;
+ inset: 0;
+ z-index: 0;
+ pointer-events: none;
+ background:
+ linear-gradient(to bottom, var(--dr-veil) 0%, transparent 14%, transparent 46%, var(--dr-veil) 96%);
+ }
+ @media (orientation: landscape) {
+ .dr::after {
+ background:
+ linear-gradient(to right, var(--dr-veil) 0%, color-mix(in srgb, var(--dr-veil) 62%, transparent) 30%, transparent 54%),
+ linear-gradient(to bottom, var(--dr-veil) 0%, transparent 14%, transparent 44%, var(--dr-veil) 92%);
+ }
+ }
+ .dr > *:not(.dr-stage) { position: relative; z-index: 1; }
+
+ /* ── Top rail ─────────────────────────────────────────────────── */
+ .dr-top {
+ display: flex;
+ align-items: center;
+ gap: calc(var(--space-sm) * var(--dr-scale));
+ min-height: 0;
+ }
+ .dr-id { display: flex; align-items: center; gap: calc(var(--space-sm) * var(--dr-scale)); min-width: 0; }
+ .dr-dot {
+ width: calc(10px * var(--dr-scale));
+ height: calc(10px * var(--dr-scale));
+ border-radius: var(--radius-pill);
+ background: var(--dr-ink-dim);
+ flex: none;
+ transition: background 180ms ease, box-shadow 180ms ease;
+ }
+ .dr[data-state='listening'] .dr-dot { background: var(--dr-live); box-shadow: 0 0 calc(14px * var(--dr-scale)) var(--dr-live); }
+ .dr[data-state='transcribing'] .dr-dot,
+ .dr[data-state='thinking'] .dr-dot { background: var(--dr-busy); box-shadow: 0 0 calc(14px * var(--dr-scale)) var(--dr-busy); }
+ .dr[data-state='speaking'] .dr-dot { background: var(--dr-ink); box-shadow: 0 0 calc(14px * var(--dr-scale)) var(--dr-line); }
+ .dr-name {
+ font-family: var(--font-display, 'Space Grotesk', system-ui, sans-serif);
+ font-size: calc(1.05rem * var(--dr-scale));
+ font-weight: 600;
+ letter-spacing: -0.01em;
+ white-space: nowrap;
+ overflow: hidden;
+ text-overflow: ellipsis;
+ }
+ .dr-state {
+ font-size: calc(0.82rem * var(--dr-scale));
+ color: var(--dr-ink-dim);
+ white-space: nowrap;
+ font-variant-numeric: tabular-nums;
+ }
+ .dr-top-spacer { flex: 1 1 auto; }
+
+ .dr-pill {
+ appearance: none;
+ display: inline-flex;
+ align-items: center;
+ gap: calc(var(--space-xs) * var(--dr-scale));
+ height: calc(44px * var(--dr-scale));
+ padding: 0 calc(var(--space-md) * var(--dr-scale));
+ border-radius: var(--radius-pill);
+ border: 1px solid var(--dr-line);
+ background: color-mix(in srgb, var(--dr-ground) 74%, var(--dr-fill));
+ backdrop-filter: blur(14px);
+ color: var(--dr-ink);
+ font: inherit;
+ font-size: calc(0.85rem * var(--dr-scale));
+ font-weight: 600;
+ cursor: pointer;
+ white-space: nowrap;
+ transition: background 140ms ease, border-color 140ms ease, transform 140ms ease;
+ }
+ .dr-pill:hover { background: var(--dr-fill-hi); }
+ .dr-pill:active { transform: scale(0.97); }
+ .dr-pill:focus-visible { outline: 2px solid var(--dr-ink); outline-offset: 3px; }
+ .dr-pill[aria-pressed='true'] { background: var(--dr-fill-hi); border-color: var(--dr-line); }
+
+ /* ── The spoken line ──────────────────────────────────────────── */
+ .dr-mid {
+ display: flex;
+ flex-direction: column;
+ justify-content: flex-end;
+ min-height: 0;
+ gap: calc(var(--space-sm) * var(--dr-scale));
+ }
+ .dr-heard {
+ font-size: calc(0.95rem * var(--dr-scale));
+ color: var(--dr-ink-dim);
+ min-height: calc(1.3em);
+ overflow: hidden;
+ text-overflow: ellipsis;
+ white-space: nowrap;
+ opacity: 0;
+ transition: opacity 200ms ease;
+ }
+ .dr-heard.is-on { opacity: 1; }
+ .dr-said {
+ font-family: var(--font-display, 'Space Grotesk', system-ui, sans-serif);
+ font-size: clamp(1.35rem, calc(3.1vw * var(--dr-scale)), calc(2.6rem * var(--dr-scale)));
+ line-height: 1.25;
+ font-weight: 600;
+ letter-spacing: -0.015em;
+ margin: 0;
+ /* Two lines is the glanceable ceiling; the rest is spoken, not read. */
+ display: -webkit-box;
+ -webkit-line-clamp: 2;
+ line-clamp: 2;
+ -webkit-box-orient: vertical;
+ overflow: hidden;
+ text-wrap: balance;
+ min-height: 2.5em;
+ }
+ .dr-said:empty::after {
+ content: attr(data-placeholder);
+ color: var(--dr-ink-dim);
+ font-weight: 500;
+ }
+ .dr-fault {
+ display: flex;
+ align-items: center;
+ gap: calc(var(--space-sm) * var(--dr-scale));
+ font-size: calc(0.9rem * var(--dr-scale));
+ color: var(--dr-fault);
+ }
+ .dr-fault-fix {
+ appearance: none;
+ border: 0;
+ background: none;
+ color: inherit;
+ font: inherit;
+ font-weight: 700;
+ text-decoration: underline;
+ text-underline-offset: 3px;
+ cursor: pointer;
+ padding: calc(var(--space-xs) * var(--dr-scale));
+ }
+ .dr-fault-fix:focus-visible { outline: 2px solid currentColor; outline-offset: 2px; }
+
+ /* ── Bottom deck ──────────────────────────────────────────────── */
+ .dr-deck {
+ display: flex;
+ align-items: center;
+ gap: calc(var(--space-md) * var(--dr-scale));
+ }
+ .dr-acts {
+ display: flex;
+ gap: calc(var(--space-sm) * var(--dr-scale));
+ flex: 1 1 auto;
+ min-width: 0;
+ }
+ .dr-act {
+ appearance: none;
+ flex: 1 1 0;
+ min-width: 0;
+ height: calc(72px * var(--dr-scale));
+ border-radius: calc(var(--radius-lg) * var(--dr-scale));
+ border: 1px solid var(--dr-line);
+ background: color-mix(in srgb, var(--dr-ground) 74%, var(--dr-fill));
+ backdrop-filter: blur(14px);
+ color: var(--dr-ink);
+ font: inherit;
+ font-size: calc(0.95rem * var(--dr-scale));
+ font-weight: 600;
+ cursor: pointer;
+ display: flex;
+ align-items: center;
+ justify-content: center;
+ gap: calc(var(--space-xs) * var(--dr-scale));
+ overflow: hidden;
+ white-space: nowrap;
+ text-overflow: ellipsis;
+ transition: background 140ms ease, opacity 140ms ease, transform 140ms ease;
+ }
+ .dr-act:hover:not(:disabled) { background: var(--dr-fill-hi); }
+ .dr-act:active:not(:disabled) { transform: scale(0.97); }
+ .dr-act:focus-visible { outline: 2px solid var(--dr-ink); outline-offset: 3px; }
+ .dr-act:disabled { opacity: 0.32; cursor: default; }
+
+ .dr-talk {
+ appearance: none;
+ flex: none;
+ width: var(--dr-talk-size);
+ height: var(--dr-talk-size);
+ border-radius: var(--radius-pill);
+ border: 2px solid var(--dr-line);
+ background: color-mix(in srgb, var(--dr-ground) 66%, var(--dr-fill-hi));
+ backdrop-filter: blur(16px);
+ color: var(--dr-ink);
+ font: inherit;
+ font-size: calc(1rem * var(--dr-scale));
+ font-weight: 700;
+ letter-spacing: 0.02em;
+ cursor: pointer;
+ position: relative;
+ display: flex;
+ align-items: center;
+ justify-content: center;
+ transition: transform 140ms ease, border-color 180ms ease, background 180ms ease;
+ }
+ .dr-talk:focus-visible { outline: 3px solid var(--dr-ink); outline-offset: 4px; }
+ .dr-talk:active { transform: scale(0.96); }
+ .dr-talk::before {
+ /* Live mic energy, driven from the real captured level. */
+ content: '';
+ position: absolute;
+ inset: calc(-6px * var(--dr-scale));
+ border-radius: var(--radius-pill);
+ border: 2px solid var(--dr-live);
+ opacity: calc(0.25 + var(--dr-mic, 0) * 0.75);
+ transform: scale(calc(1 + var(--dr-mic, 0) * 0.09));
+ transition: transform 90ms linear, opacity 90ms linear;
+ pointer-events: none;
+ }
+ .dr[data-state='idle'] .dr-talk::before,
+ .dr[data-state='thinking'] .dr-talk::before,
+ .dr[data-state='speaking'] .dr-talk::before { opacity: 0; }
+ .dr[data-state='listening'] .dr-talk { border-color: var(--dr-live); background: var(--dr-fill-hi); }
+ .dr[data-state='speaking'] .dr-talk { border-color: var(--dr-ink); }
+
+ /* ── Overlays: picker, keyboard, permission ───────────────────── */
+ .dr-sheet {
+ position: fixed;
+ inset: 0;
+ z-index: 5;
+ display: flex;
+ flex-direction: column;
+ background: color-mix(in srgb, var(--dr-ground) 92%, transparent);
+ backdrop-filter: blur(18px);
+ padding:
+ max(var(--dr-gutter), env(safe-area-inset-top))
+ max(var(--dr-gutter), env(safe-area-inset-right))
+ max(var(--dr-gutter), env(safe-area-inset-bottom))
+ max(var(--dr-gutter), env(safe-area-inset-left));
+ gap: calc(var(--space-md) * var(--dr-scale));
+ animation: dr-rise 200ms ease both;
+ }
+ @keyframes dr-rise { from { opacity: 0; transform: translateY(2%); } to { opacity: 1; transform: none; } }
+ @media (prefers-reduced-motion: reduce) {
+ .dr-sheet { animation: none; }
+ .dr-talk::before { transition: none; }
+ }
+ .dr-sheet-head { display: flex; align-items: center; gap: calc(var(--space-md) * var(--dr-scale)); }
+ .dr-sheet-title {
+ margin: 0;
+ font-family: var(--font-display, 'Space Grotesk', system-ui, sans-serif);
+ font-size: calc(1.35rem * var(--dr-scale));
+ font-weight: 700;
+ letter-spacing: -0.015em;
+ flex: 1 1 auto;
+ }
+ .dr-sheet-sub {
+ margin: 0;
+ color: var(--dr-ink-dim);
+ font-size: calc(0.95rem * var(--dr-scale));
+ max-width: 62ch;
+ }
+ .dr-grid {
+ display: grid;
+ grid-template-columns: repeat(auto-fill, minmax(calc(190px * var(--dr-scale)), 1fr));
+ gap: calc(var(--space-sm) * var(--dr-scale));
+ overflow-y: auto;
+ align-content: start;
+ flex: 1 1 auto;
+ padding-bottom: calc(var(--space-md) * var(--dr-scale));
+ }
+ .dr-card {
+ appearance: none;
+ border: 1px solid var(--dr-line);
+ background: var(--dr-fill);
+ color: var(--dr-ink);
+ border-radius: calc(var(--radius-lg) * var(--dr-scale));
+ padding: calc(var(--space-sm) * var(--dr-scale));
+ cursor: pointer;
+ display: flex;
+ flex-direction: column;
+ gap: calc(var(--space-sm) * var(--dr-scale));
+ text-align: left;
+ font: inherit;
+ transition: background 140ms ease, transform 140ms ease;
+ }
+ .dr-card:hover { background: var(--dr-fill-hi); }
+ .dr-card:active { transform: scale(0.985); }
+ .dr-card:focus-visible { outline: 2px solid var(--dr-ink); outline-offset: 3px; }
+ .dr-card img {
+ width: 100%;
+ aspect-ratio: 1 / 1;
+ object-fit: cover;
+ border-radius: calc(var(--radius-md) * var(--dr-scale));
+ background: var(--dr-fill);
+ display: block;
+ }
+ .dr-card-name {
+ font-weight: 600;
+ font-size: calc(0.95rem * var(--dr-scale));
+ overflow: hidden;
+ text-overflow: ellipsis;
+ white-space: nowrap;
+ }
+ .dr-skel {
+ border-radius: calc(var(--radius-lg) * var(--dr-scale));
+ background: linear-gradient(90deg, var(--dr-fill) 25%, var(--dr-fill-hi) 37%, var(--dr-fill) 63%);
+ background-size: 400% 100%;
+ animation: dr-shimmer 1.4s ease infinite;
+ aspect-ratio: 4 / 5;
+ }
+ @keyframes dr-shimmer { from { background-position: 100% 0; } to { background-position: -100% 0; } }
+ @media (prefers-reduced-motion: reduce) { .dr-skel { animation: none; } }
+ .dr-empty {
+ border: 1px dashed var(--dr-line);
+ border-radius: calc(var(--radius-lg) * var(--dr-scale));
+ padding: calc(var(--space-lg) * var(--dr-scale));
+ color: var(--dr-ink-dim);
+ font-size: calc(0.95rem * var(--dr-scale));
+ display: flex;
+ flex-direction: column;
+ gap: calc(var(--space-md) * var(--dr-scale));
+ align-items: flex-start;
+ }
+ .dr-link {
+ color: var(--dr-ink);
+ font-weight: 700;
+ text-decoration: underline;
+ text-underline-offset: 3px;
+ }
+
+ .dr-typebar { display: flex; gap: calc(var(--space-sm) * var(--dr-scale)); }
+ .dr-input {
+ flex: 1 1 auto;
+ min-width: 0;
+ height: calc(64px * var(--dr-scale));
+ border-radius: calc(var(--radius-lg) * var(--dr-scale));
+ border: 1px solid var(--dr-line);
+ background: var(--dr-fill);
+ color: var(--dr-ink);
+ font: inherit;
+ font-size: calc(1.05rem * var(--dr-scale));
+ padding: 0 calc(var(--space-md) * var(--dr-scale));
+ user-select: text;
+ }
+ .dr-input:focus-visible { outline: 2px solid var(--dr-ink); outline-offset: 2px; }
+ .dr-input::placeholder { color: var(--dr-ink-dim); }
+
+ /* Portrait (a phone in the cradle) stacks the deck and pairs the
+ actions two by two, which is a far better thumb target than four
+ slivers across a 390px panel. */
+ @media (orientation: portrait) {
+ .dr-deck { flex-direction: column-reverse; align-items: stretch; }
+ .dr-talk { width: 100%; height: calc(104px * var(--dr-scale)); border-radius: calc(var(--radius-lg) * var(--dr-scale)); }
+ .dr-talk::before { border-radius: calc(var(--radius-lg) * var(--dr-scale)); }
+ .dr-acts { display: grid; grid-template-columns: 1fr 1fr; }
+ .dr::after {
+ background:
+ linear-gradient(to bottom, var(--dr-veil) 0%, transparent 12%, transparent 30%, var(--dr-veil) 62%);
+ }
+ }
+ /* A narrow panel loses the redundant status word before it loses a
+ control: the state dot and the spoken line already carry it. */
+ @media (max-width: 560px) {
+ .dr-state { display: none; }
+ .dr-pill {
+ height: calc(40px * var(--dr-scale));
+ padding: 0 calc(var(--space-sm) * var(--dr-scale));
+ font-size: calc(0.78rem * var(--dr-scale));
+ }
+ .dr-top { gap: calc(var(--space-xs) * var(--dr-scale)); }
+ .dr-name { font-size: calc(0.95rem * var(--dr-scale)); }
+ }
+ /* A short landscape dash (800x480 and friends) has no room for a top rail
+ and a two-line reply, so the reply gets one line and the rail shrinks. */
+ @media (orientation: landscape) and (max-height: 520px) {
+ .dr-said { -webkit-line-clamp: 1; line-clamp: 1; min-height: 1.25em; }
+ .dr-act { height: calc(60px * var(--dr-scale)); }
+ }
+ </style>
+ </head>
+ <body>
+ <div class="dr" id="dr" data-state="idle">
+ <div class="dr-stage" id="dr-stage" aria-hidden="true"></div>
+
+ <header class="dr-top">
+ <div class="dr-id">
+ <span class="dr-dot" aria-hidden="true"></span>
+ <span class="dr-name" id="dr-name">three.ws Drive</span>
+ </div>
+ <span class="dr-state" id="dr-status" role="status" aria-live="polite">Starting</span>
+ <span class="dr-top-spacer"></span>
+ <button type="button" class="dr-pill" id="dr-motion" aria-pressed="true" title="Driving locks the keyboard. Switch to Parked to type.">
+ Driving
+ </button>
+ <button type="button" class="dr-pill" id="dr-light" aria-pressed="false" title="Day or night palette">Night</button>
+ <button type="button" class="dr-pill" id="dr-swap">Agent</button>
+ </header>
+
+ <main class="dr-mid">
+ <p class="dr-heard" id="dr-heard"></p>
+ <p
+ class="dr-said"
+ id="dr-said"
+ role="status"
+ aria-live="polite"
+ data-placeholder="Hold Talk and speak."
+ ></p>
+ <p class="dr-fault" id="dr-fault" hidden>
+ <span id="dr-fault-text"></span>
+ <button type="button" class="dr-fault-fix" id="dr-fault-fix" hidden></button>
+ </p>
+ </main>
+
+ <div class="dr-deck">
+ <div class="dr-acts" id="dr-acts">
+ <button type="button" class="dr-act" id="dr-act-hands" aria-pressed="false">Hands free</button>
+ <button type="button" class="dr-act" id="dr-act-repeat" disabled>Repeat</button>
+ <button type="button" class="dr-act" id="dr-act-stop" disabled>Stop</button>
+ <button type="button" class="dr-act" id="dr-act-type">Type</button>
+ </div>
+ <button type="button" class="dr-talk" id="dr-talk">Talk</button>
+ </div>
+ </div>
+
+ <section class="dr-sheet" id="dr-picker" hidden aria-label="Choose your copilot">
+ <div class="dr-sheet-head">
+ <h1 class="dr-sheet-title">Choose your copilot</h1>
+ <button type="button" class="dr-pill" id="dr-picker-close" hidden>Back</button>
+ </div>
+ <p class="dr-sheet-sub" id="dr-picker-sub">
+ Pick the agent that rides with you. It keeps its voice, its face, and its memory in
+ the car.
+ </p>
+ <div class="dr-grid" id="dr-picker-grid"></div>
+ </section>
+
+ <section class="dr-sheet" id="dr-type" hidden aria-label="Type to your agent">
+ <div class="dr-sheet-head">
+ <h1 class="dr-sheet-title">Type</h1>
+ <button type="button" class="dr-pill" id="dr-type-close">Back</button>
+ </div>
+ <p class="dr-sheet-sub">
+ The keyboard is here for a stopped car or a passenger. Switch back to Driving when you
+ pull away.
+ </p>
+ <form class="dr-typebar" id="dr-type-form">
+ <input
+ class="dr-input"
+ id="dr-type-input"
+ type="text"
+ autocomplete="off"
+ placeholder="Ask your agent something"
+ aria-label="Message"
+ />
+ <button type="submit" class="dr-pill" id="dr-type-send">Send</button>
+ </form>
+ </section>
+ </body>
+</html>
diff --git a/public/changelog.json b/public/changelog.json
index 0b09f6645..04e80aa44 100644
--- a/public/changelog.json
+++ b/public/changelog.json
@@ -1,11 +1,42 @@
{
- "generated_at": "2026-09-02T21:37:37.970Z",
+ "generated_at": "2026-09-03T01:46:12.068Z",
"generated_by": "scripts/build-page-index.mjs from data/pages.json + data/changelog.json",
"site": {
"name": "three.ws",
"url": "https://three.ws"
},
"entries": [
+ {
+ "date": "2026-09-03",
+ "type": "launch",
+ "title": "Docs · three.ws Drive",
+ "summary": "How a three.ws agent rides in the car: the CarPlay entitlement categories measured, why Apple's voice-based conversational category is the one that fits and why it grants no drawing surface, the open-source in-car projects worth adopting and the copyleft ones that are not, and how the /drive surface, the native CarPlay scene, and an open head unit share one implementation.",
+ "link": "/docs/carplay",
+ "tags": [
+ "launch"
+ ]
+ },
+ {
+ "date": "2026-09-03",
+ "type": "launch",
+ "title": "three.ws Drive",
+ "summary": "Your 3D agent in the car: voice-first, glanceable, and built to the rules CarPlay and Android Auto enforce. Runs on a head unit, a phone in the cradle, or inside the native car shell.",
+ "link": "/drive",
+ "tags": [
+ "launch"
+ ]
+ },
+ {
+ "date": "2026-09-03",
+ "type": "update",
+ "title": "three.ws Drive: your agent, in the car",
+ "summary": "There is a new page at /drive built for a car screen instead of a desk. Your agent is on it full size, you hold one big button and talk, and it answers out loud in its own voice with its face moving. It shows you one line of what it said rather than a wall of chat, because you are driving. It starts dark, because a bright panel at night is glare, and it will switch to a day palette on one press or one sentence. The keyboard stays out of the way while you are moving: if your browser can tell how fast you are going, text entry locks itself the moment you pull away and comes back when you have properly stopped. A few things you say never leave the car at all, so \"repeat that\", \"stop talking\", \"louder\" and \"night mode\" happen instantly, while \"stop at the next charger\" still goes to your agent like any other question. It works today on a head unit browser, on a phone in a cradle, and on Android Automotive. The Apple CarPlay version is built and waiting on Apple's approval for voice assistants in the car. One thing this fixed for everyone: an avatar used to answer in silence if you were not signed in, and now it always has a voice.",
+ "link": "/drive",
+ "tags": [
+ "feature",
+ "improvement"
+ ]
+ },
{
"date": "2026-09-02",
"type": "launch",
diff --git a/public/changelog.xml b/public/changelog.xml
index 026cbcffe..c8866e74e 100644
--- a/public/changelog.xml
+++ b/public/changelog.xml
@@ -6,6 +6,30 @@
<atom:link href="https://three.ws/changelog.xml" rel="self" type="application/rss+xml"/>
<description>What's new on three.ws: features, improvements, and releases, newest first.</description>
<language>en</language>
+ <item>
+ <title>Docs · three.ws Drive</title>
+ <link>https://three.ws/docs/carplay</link>
+ <guid isPermaLink="false">2026-09-03:Docs · three.ws Drive</guid>
+ <pubDate>Thu, 03 Sep 2026 12:00:00 GMT</pubDate>
+ <category>launch</category>
+ <description>How a three.ws agent rides in the car: the CarPlay entitlement categories measured, why Apple's voice-based conversational category is the one that fits and why it grants no drawing surface, the open-source in-car projects worth adopting and the copyleft ones that are not, and how the /drive surface, the native CarPlay scene, and an open head unit share one implementation.</description>
+ </item>
+ <item>
+ <title>three.ws Drive</title>
+ <link>https://three.ws/drive</link>
+ <guid isPermaLink="false">2026-09-03:three.ws Drive</guid>
+ <pubDate>Thu, 03 Sep 2026 12:00:00 GMT</pubDate>
+ <category>launch</category>
+ <description>Your 3D agent in the car: voice-first, glanceable, and built to the rules CarPlay and Android Auto enforce. Runs on a head unit, a phone in the cradle, or inside the native car shell.</description>
+ </item>
+ <item>
+ <title>three.ws Drive: your agent, in the car</title>
+ <link>https://three.ws/drive</link>
+ <guid isPermaLink="false">2026-09-03:three.ws Drive: your agent, in the car</guid>
+ <pubDate>Thu, 03 Sep 2026 12:00:00 GMT</pubDate>
+ <category>feature</category>
+ <description>There is a new page at /drive built for a car screen instead of a desk. Your agent is on it full size, you hold one big button and talk, and it answers out loud in its own voice with its face moving. It shows you one line of what it said rather than a wall of chat, because you are driving. It starts dark, because a bright panel at night is glare, and it will switch to a day palette on one press or one sentence. The keyboard stays out of the way while you are moving: if your browser can tell how fast you are going, text entry locks itself the moment you pull away and comes back when you have properly stopped. A few things you say never leave the car at all, so &quot;repeat that&quot;, &quot;stop talking&quot;, &quot;louder&quot; and &quot;night mode&quot; happen instantly, while &quot;stop at the next charger&quot; still goes to your agent like any other question. It works today on a head unit browser, on a phone in a cradle, and on Android Automotive. The Apple CarPlay version is built and waiting on Apple's approval for voice assistants in the car. One thing this fixed for everyone: an avatar used to answer in silence if you were not signed in, and now it always has a voice.</description>
+ </item>
<item>
<title>Docs · Materialize: turn a generation into a real object</title>
<link>https://three.ws/docs/materialize</link>
diff --git a/public/features.json b/public/features.json
index 6008830d2..672a458a9 100644
--- a/public/features.json
+++ b/public/features.json
@@ -1,5 +1,5 @@
{
- "generated_at": "2026-09-02T21:37:37.885Z",
+ "generated_at": "2026-09-03T01:46:12.050Z",
"generated_by": "scripts/build-page-index.mjs from data/pages.json",
"site": {
"name": "three.ws",
@@ -175,6 +175,16 @@
"tags": [],
"showcase": true
},
+ {
+ "path": "/drive",
+ "title": "three.ws Drive",
+ "description": "Your 3D agent in the car: voice-first, glanceable, and built to the rules CarPlay and Android Auto enforce. Runs on a head unit, a phone in the cradle, or inside the native car shell.",
+ "added": "2026-09-03",
+ "auth": null,
+ "indexable": true,
+ "tags": [],
+ "showcase": true
+ },
{
"path": "/agenc/embodied",
"title": "AgenC · Embodied",
@@ -3207,6 +3217,16 @@
"tags": [],
"showcase": false
},
+ {
+ "path": "/docs/carplay",
+ "title": "Docs · three.ws Drive",
+ "description": "How a three.ws agent rides in the car: the CarPlay entitlement categories measured, why Apple's voice-based conversational category is the one that fits and why it grants no drawing surface, the open-source in-car projects worth adopting and the copyleft ones that are not, and how the /drive surface, the native CarPlay scene, and an open head unit share one implementation.",
+ "added": "2026-09-03",
+ "auth": null,
+ "indexable": true,
+ "tags": [],
+ "showcase": false
+ },
{
"path": "/docs/tty-avatar",
"title": "Docs · Terminal avatar",
diff --git a/public/llms-full.txt b/public/llms-full.txt
index 3cf47917c..5b9625b71 100644
--- a/public/llms-full.txt
+++ b/public/llms-full.txt
@@ -109,6 +109,12 @@ URL: https://three.ws/autopilot
A hands-off token cockpit. Set the rules (buys, sells, and guardrails) and let your agent run the coin autonomously.
+### three.ws Drive
+
+URL: https://three.ws/drive
+
+Your 3D agent in the car: voice-first, glanceable, and built to the rules CarPlay and Android Auto enforce. Runs on a head unit, a phone in the cradle, or inside the native car shell.
+
### AgenC · Embodied
URL: https://three.ws/agenc/embodied
@@ -1953,6 +1959,12 @@ URL: https://three.ws/docs/smart-home
How a three.ws agent connects to a real house: the open-source smart-home landscape measured and shortlisted, why Home Assistant owns the device layer, the two channels the @three-ws/home-bridge package speaks, the physical-action gate that never lets an unlock through unasked, and the road from here to voice in every room.
+### Docs · three.ws Drive
+
+URL: https://three.ws/docs/carplay
+
+How a three.ws agent rides in the car: the CarPlay entitlement categories measured, why Apple's voice-based conversational category is the one that fits and why it grants no drawing surface, the open-source in-car projects worth adopting and the copyleft ones that are not, and how the /drive surface, the native CarPlay scene, and an open head unit share one implementation.
+
### Docs · Terminal avatar
URL: https://three.ws/docs/tty-avatar
diff --git a/public/llms.txt b/public/llms.txt
index 4623c14fa..bd604bc2e 100644
--- a/public/llms.txt
+++ b/public/llms.txt
@@ -46,6 +46,7 @@ Entry points into the product.
- [Diorama](https://three.ws/diorama): Speak a little world into being: describe a scene and watch it assemble into an explorable 3D diorama you can walk through, then take it into AR.
- [Mocap Studio](https://three.ws/mocap-studio): Drive a 3D avatar with your webcam: real-time facial motion capture in the browser, no rig and no download.
- [Coin Autopilot](https://three.ws/autopilot): A hands-off token cockpit. Set the rules (buys, sells, and guardrails) and let your agent run the coin autonomously.
+- [three.ws Drive](https://three.ws/drive): Your 3D agent in the car: voice-first, glanceable, and built to the rules CarPlay and Android Auto enforce. Runs on a head unit, a phone in the cradle, or inside the native car shell.
- [AgenC · Embodied](https://three.ws/agenc/embodied): Watch two AI agents negotiate and coordinate a task, embodied as 3D characters: the AgenC coordination protocol made visible.
- [AgenC · Task Room](https://three.ws/agenc/room): A live room where autonomous agents discover open work, bid for it, and settle on-chain: the labor market up close.
- [Talking Avatar Video](https://three.ws/create/video): Turn a three.ws avatar into a talking-head video: upload your audio and export a lip-synced clip.
@@ -361,6 +362,7 @@ Docs, tutorials, and reference.
- [Docs · Skill bundles](https://three.ws/docs/skill-bundles): How a bundle of an agent's skills is priced from that agent's own sales rather than a rule of thumb: the median-basket and flat-discount bases, the public pricing and backtest endpoint, the owner endpoints that publish and retire a bundle, and the on-chain checkout that unlocks every skill in one payment.
- [Docs · Avatar Artifact viewer](https://three.ws/docs/avatar-artifact): The standalone avatar viewer you paste into any site as a one-line iframe: its ?agent= and ?model= parameters, the hosts a model may come from, how a clip-less rig still comes to life, and the sentence the page shows for every way it can fail.
- [Docs · three.ws Home](https://three.ws/docs/smart-home): How a three.ws agent connects to a real house: the open-source smart-home landscape measured and shortlisted, why Home Assistant owns the device layer, the two channels the @three-ws/home-bridge package speaks, the physical-action gate that never lets an unlock through unasked, and the road from here to voice in every room.
+- [Docs · three.ws Drive](https://three.ws/docs/carplay): How a three.ws agent rides in the car: the CarPlay entitlement categories measured, why Apple's voice-based conversational category is the one that fits and why it grants no drawing surface, the open-source in-car projects worth adopting and the copyleft ones that are not, and how the /drive surface, the native CarPlay scene, and an open head unit share one implementation.
- [Docs · Terminal avatar](https://three.ws/docs/tty-avatar): The @three-ws/tty-avatar package: any three.ws avatar or agent body rendered live in a terminal by a software rasterizer (truecolor half-blocks, braille, or plain ASCII), procedural moods, and Claude Code hooks that turn it into your coding agent's face.
- [Docs · Site load performance](https://three.ws/docs/site-performance): The load-cost rules behind every page that is not /play, each with the Lighthouse measurement that produced it: catalogue grids skip the rows nobody can see, analytics waits for the first idle moment, the CDN bundle ships minified, below-the-fold sections load below the fold, and thumbnails decode off the critical path.
- [Docs · Farcaster memory seeding](https://three.ws/docs/farcaster-memory-seeding): Give an agent your Farcaster voice with explicit consent: ownership is proved by signing a challenge with a wallet the fid itself verified on the protocol (Solana first), only your public profile and casts are read, and revoking the grant deletes every memory it produced.
diff --git a/public/sitemap/index.html b/public/sitemap/index.html
index 2aa95c515..6afc3406c 100644
--- a/public/sitemap/index.html
+++ b/public/sitemap/index.html
@@ -83,7 +83,7 @@
<main class="sm-wrap">
<div class="sm-hero">
<h1>Sitemap</h1>
- <p>All 886 pages on three.ws, grouped by purpose. Looking for the machine-readable versions?</p>
+ <p>All 888 pages on three.ws, grouped by purpose. Looking for the machine-readable versions?</p>
<div class="sm-formats">
<a href="/sitemap.xml"><code>sitemap.xml</code> · for search engines</a>
<a href="/llms.txt"><code>llms.txt</code> · for AI agents</a>
@@ -95,7 +95,7 @@
</div>
<div class="sm-filter">
<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" aria-hidden="true"><circle cx="11" cy="11" r="7"/><path d="m21 21-4.3-4.3"/></svg>
- <input id="sm-filter-input" type="search" placeholder="Filter 886 pages by name, path, or description…" aria-label="Filter pages" autocomplete="off" spellcheck="false" />
+ <input id="sm-filter-input" type="search" placeholder="Filter 888 pages by name, path, or description…" aria-label="Filter pages" autocomplete="off" spellcheck="false" />
<kbd aria-hidden="true">/</kbd>
</div>
<p class="sm-filter-count" id="sm-filter-count" role="status" aria-live="polite"></p>
@@ -103,6 +103,22 @@
<h2 id="sm-new-title">Newest<a class="sm-new-more" href="/changelog">full changelog &rarr;</a></h2>
<p class="sm-section-desc">The 9 most recently launched pages, straight to the live surface.</p>
<ul class="sm-list">
+ <li>
+ <a href="/drive">
+ <span class="sm-new-date">Sep 3, 2026</span>
+ <span class="sm-title">three.ws Drive</span>
+ <span class="sm-path">/drive</span>
+ <span class="sm-desc">Your 3D agent in the car: voice-first, glanceable, and built to the rules CarPlay and Android Auto enforce. Runs on a head unit, a phone in the cradle, or inside the native car shell.</span>
+ </a>
+ </li>
+ <li>
+ <a href="/docs/carplay">
+ <span class="sm-new-date">Sep 3, 2026</span>
+ <span class="sm-title">Docs · three.ws Drive</span>
+ <span class="sm-path">/docs/carplay</span>
+ <span class="sm-desc">How a three.ws agent rides in the car: the CarPlay entitlement categories measured, why Apple's voice-based conversational category is the one that fits and why it grants no drawing surface, the open-source in-car projects worth adopting and the copyleft ones that are not, and how the /drive surface, the native CarPlay scene, and an open head unit share one implementation.</span>
+ </a>
+ </li>
<li>
<a href="/materialize">
<span class="sm-new-date">Sep 2, 2026</span>
@@ -159,22 +175,6 @@
<span class="sm-desc">Launch a supply-capped collection of AI-forged 3D characters. Pick a style, define weighted trait layers, and three.ws rolls the whole supply from a published seed: verifiable rarity, rigged animation-ready art, forged on reveal.</span>
</a>
</li>
- <li>
- <a href="/spotlight">
- <span class="sm-new-date">Sep 1, 2026</span>
- <span class="sm-title">Agent Spotlight</span>
- <span class="sm-path">/spotlight</span>
- <span class="sm-desc">The three.ws community showcase: agents people actually built something with, written up by the builder and ranked by community upvotes. Filter by category, open any entry in 3D, upvote what you like, and submit your own.</span>
- </a>
- </li>
- <li>
- <a href="/docs/page-performance">
- <span class="sm-new-date">Sep 1, 2026</span>
- <span class="sm-title">Page performance: the rules a three.ws page plays by</span>
- <span class="sm-path">/docs/page-performance</span>
- <span class="sm-desc">How three.ws keeps real 3D on ordinary pages without making them slow: previews that spin only while someone is looking, models that load when they are wanted, boxes reserved before late content arrives, and images fetched at the size they are shown.</span>
- </a>
- </li>
</ul>
</section>
<nav class="sm-toc" aria-label="Sections">
@@ -192,7 +192,7 @@
</nav>
<section class="sm-section" id="main">
<header>
- <h2>Main<span class="sm-count" data-count>56</span></h2>
+ <h2>Main<span class="sm-count" data-count>57</span></h2>
<p class="sm-section-desc">Entry points into the product.</p>
</header>
<ul class="sm-list">
@@ -308,6 +308,13 @@
<span class="sm-desc">A hands-off token cockpit. Set the rules (buys, sells, and guardrails) and let your agent run the coin autonomously.</span>
</a>
</li>
+ <li>
+ <a href="/drive">
+ <span class="sm-title">three.ws Drive</span>
+ <span class="sm-path">/drive</span>
+ <span class="sm-desc">Your 3D agent in the car: voice-first, glanceable, and built to the rules CarPlay and Android Auto enforce. Runs on a head unit, a phone in the cradle, or inside the native car shell.</span>
+ </a>
+ </li>
<li>
<a href="/agenc/embodied">
<span class="sm-title">AgenC · Embodied</span>
@@ -2416,7 +2423,7 @@
<section class="sm-section" id="learn">
<header>
- <h2>Learn<span class="sm-count" data-count>403</span></h2>
+ <h2>Learn<span class="sm-count" data-count>404</span></h2>
<p class="sm-section-desc">Docs, tutorials, and reference.</p>
</header>
<ul class="sm-list">
@@ -2455,6 +2462,13 @@
<span class="sm-desc">How a three.ws agent connects to a real house: the open-source smart-home landscape measured and shortlisted, why Home Assistant owns the device layer, the two channels the @three-ws/home-bridge package speaks, the physical-action gate that never lets an unlock through unasked, and the road from here to voice in every room.</span>
</a>
</li>
+ <li>
+ <a href="/docs/carplay">
+ <span class="sm-title">Docs · three.ws Drive</span>
+ <span class="sm-path">/docs/carplay</span>
+ <span class="sm-desc">How a three.ws agent rides in the car: the CarPlay entitlement categories measured, why Apple's voice-based conversational category is the one that fits and why it grants no drawing surface, the open-source in-car projects worth adopting and the copyleft ones that are not, and how the /drive surface, the native CarPlay scene, and an open head unit share one implementation.</span>
+ </a>
+ </li>
<li>
<a href="/docs/tty-avatar">
<span class="sm-title">Docs · Terminal avatar</span>
diff --git a/src/drive/bridge.js b/src/drive/bridge.js
new file mode 100644
index 000000000..f74002d9a
--- /dev/null
+++ b/src/drive/bridge.js
@@ -0,0 +1,83 @@
+// The channel between /drive in a web view and the native car shell around it.
+//
+// On CarPlay the car screen is NOT ours to draw on: a voice-conversational app
+// gets Apple's Voice Control template (a listening indicator and up to four
+// action buttons) and nothing else. The conversation itself, the agent, its
+// voice and its face all live in this page, which runs on the phone. So the
+// page publishes its state outward and accepts button presses inward, and the
+// native side is a thin renderer of whatever it is told. Android Auto's
+// template host works the same way.
+//
+// In a plain browser there is no channel and every send is a no-op, so the
+// page has exactly one code path.
+
+const PROTOCOL = 1;
+
+/** The action set mirrored onto the car screen. Four is Apple's ceiling. */
+export const MAX_NATIVE_ACTIONS = 4;
+
+function nativeSink() {
+ if (typeof window === 'undefined') return null;
+ const ios = window.webkit?.messageHandlers?.threeWsDrive;
+ if (ios?.postMessage) return (payload) => ios.postMessage(payload);
+ const android = window.ThreeWsDriveNative;
+ if (android?.post) return (payload) => android.post(JSON.stringify(payload));
+ return null;
+}
+
+/**
+ * Open the bridge.
+ * @param {(command: { type: string, value?: unknown }) => void} onCommand
+ * Invoked for every press that arrives from the car screen.
+ */
+export function createBridge(onCommand) {
+ const sink = nativeSink();
+ const handler = typeof onCommand === 'function' ? onCommand : () => {};
+
+ // The native shell calls into this global. It exists even without a shell so
+ // the same surface is drivable from the console during development.
+ if (typeof window !== 'undefined') {
+ window.threeWsDrive = {
+ protocol: PROTOCOL,
+ command(input) {
+ const cmd = typeof input === 'string' ? { type: input } : input;
+ if (!cmd?.type) return false;
+ handler(cmd);
+ return true;
+ },
+ };
+ }
+
+ const send = (type, body) => {
+ if (!sink) return false;
+ try {
+ sink({ v: PROTOCOL, type, ...body });
+ return true;
+ } catch {
+ // A dead channel must never take the conversation down with it.
+ return false;
+ }
+ };
+
+ return {
+ attached: !!sink,
+ ready: (agent) => send('ready', { agent }),
+ state: (state) => send('state', { state }),
+ heard: (text) => send('heard', { text: String(text || '').slice(0, 240) }),
+ said: (text) => send('said', { text: String(text || '').slice(0, 500) }),
+ error: (code, message) => send('error', { code, message: String(message || '').slice(0, 240) }),
+ actions: (actions) =>
+ send('actions', {
+ actions: actions.slice(0, MAX_NATIVE_ACTIONS).map((a) => ({
+ id: a.id,
+ label: a.label,
+ enabled: a.enabled !== false,
+ })),
+ }),
+ dispose() {
+ if (typeof window !== 'undefined' && window.threeWsDrive?.protocol === PROTOCOL) {
+ delete window.threeWsDrive;
+ }
+ },
+ };
+}
diff --git a/src/drive/commands.js b/src/drive/commands.js
new file mode 100644
index 000000000..9062d83ae
--- /dev/null
+++ b/src/drive/commands.js
@@ -0,0 +1,74 @@
+// Zero-latency local commands for /drive.
+//
+// A handful of things a driver says are control, not conversation: "repeat
+// that", "stop talking", "louder", "night mode". Sending those to a model costs
+// a round trip and can come back wrong, so the page answers them itself, before
+// the network is touched. Everything else falls through to the agent untouched.
+//
+// The matcher deliberately compares the WHOLE utterance against a phrase set
+// rather than searching inside it. "Stop" is a command; "stop at the next
+// charger" is a question for the agent, and swallowing it would be worse than
+// having no command layer at all.
+
+const PHRASES = [
+ ['repeat', ['repeat', 'repeat that', 'say that again', 'again', 'what did you say', 'come again', 'one more time']],
+ ['hush', ['stop', 'stop talking', 'be quiet', 'quiet', 'hush', 'never mind', 'nevermind', 'cancel', 'shut up']],
+ ['louder', ['louder', 'speak up', 'turn it up', 'volume up', 'i can not hear you', "i can't hear you"]],
+ ['quieter', ['quieter', 'softer', 'turn it down', 'volume down', 'too loud']],
+ ['night', ['night mode', 'dark mode', 'go dark', 'dim the screen']],
+ ['day', ['day mode', 'light mode', 'brighten the screen']],
+ ['parked', ['i am parked', "i'm parked", 'parked', 'we are parked', 'park mode']],
+ ['driving', ['i am driving', "i'm driving", 'driving', 'drive mode', 'we are moving']],
+];
+
+const LOOKUP = new Map();
+for (const [id, phrases] of PHRASES) {
+ for (const phrase of phrases) LOOKUP.set(phrase, id);
+}
+
+/** Lowercase, strip punctuation and filler, collapse whitespace. */
+export function normalize(text) {
+ return String(text || '')
+ .toLowerCase()
+ .replace(/[^a-z0-9' ]+/g, ' ')
+ .replace(/\s+/g, ' ')
+ .trim()
+ .replace(/^(hey |ok |okay |please |can you |could you |would you )+/g, '')
+ .replace(/( please| now| for me)+$/g, '')
+ .trim();
+}
+
+/**
+ * @param {string} transcript
+ * @returns {'repeat'|'hush'|'louder'|'quieter'|'night'|'day'|'parked'|'driving'|null}
+ */
+export function matchDriveCommand(transcript) {
+ const phrase = normalize(transcript);
+ if (!phrase || phrase.length > 40) return null;
+ return LOOKUP.get(phrase) || null;
+}
+
+/**
+ * Build the interceptor TalkController calls with every final transcript.
+ * Returning true means the utterance was control and the chat round trip is
+ * skipped; returning false lets it through to the agent.
+ *
+ * @param {object} handlers One function per command id. A handler may be async
+ * and may return a short line to speak back; returning nothing stays silent,
+ * which is right for "stop".
+ */
+export function createDriveInterceptor(handlers) {
+ return async function interceptDriveCommand(transcript) {
+ const id = matchDriveCommand(transcript);
+ if (!id) return false;
+ const fn = handlers[id];
+ if (typeof fn !== 'function') return false;
+ await fn();
+ return true;
+ };
+}
+
+/** The command vocabulary, for the system prompt and the docs. */
+export function commandVocabulary() {
+ return PHRASES.map(([id, phrases]) => ({ id, phrases: phrases.slice() }));
+}
diff --git a/src/drive/index.js b/src/drive/index.js
new file mode 100644
index 000000000..a29349c00
--- /dev/null
+++ b/src/drive/index.js
@@ -0,0 +1,756 @@
+// three.ws Drive: the agent, in the car.
+//
+// One page, four panels. On CarPlay and Android Auto the car screen belongs to
+// the platform (a voice template, an indicator, four buttons), so this page
+// runs on the phone and mirrors its state out through src/drive/bridge.js. On a
+// head unit browser or a phone in a cradle this page IS the whole surface.
+//
+// The loop underneath is the platform's existing one, unchanged: mic → /api/asr
+// (NVIDIA Riva, with the browser recognizer as the fallback lane) → /api/chat
+// → /api/tts → lipsync on a real rigged avatar. What /drive adds is everything
+// that makes it safe and usable at 70 mph: one glanceable line instead of a
+// transcript, four controls instead of a UI, a keyboard that disappears when
+// the wheels turn, and a local command layer that answers "stop talking"
+// without a round trip.
+//
+// Docs: docs/carplay.md. Native shell: ios/native/App/App/CarPlaySceneDelegate.swift.
+
+import { TalkScene } from '../voice/talk-scene.js';
+import { AvatarMouthTarget } from '../voice/avatar-morph-target.js';
+import { TalkController } from '../voice/talk-controller.js';
+import { detectSurface, surfaceProfile, applySurface } from './surface.js';
+import { createBridge } from './bridge.js';
+import { createDriveInterceptor } from './commands.js';
+import { watchMotion } from './motion.js';
+import { log } from '../shared/log.js';
+
+const STORE_COPILOT = 'twx_drive_copilot';
+const STORE_LIGHT = 'twx_drive_light';
+const STORE_HANDS = 'twx_drive_hands';
+
+const STATUS = {
+ idle: 'Ready',
+ listening: 'Listening',
+ transcribing: 'Hearing you',
+ thinking: 'Thinking',
+ speaking: 'Speaking',
+};
+
+// Hands-free voice activity, on the Riva lane where we own the raw audio.
+// Below this level for QUIET_MS after speech has been detected ends the turn;
+// TURN_CAP_MS stops a stuck-open mic from streaming the whole commute.
+const SPEECH_LEVEL = 0.06;
+const QUIET_MS = 1100;
+const TURN_CAP_MS = 15000;
+const REARM_MS = 650;
+
+const el = (id) => document.getElementById(id);
+
+const ui = {
+ root: el('dr'),
+ stage: el('dr-stage'),
+ name: el('dr-name'),
+ status: el('dr-status'),
+ motion: el('dr-motion'),
+ light: el('dr-light'),
+ swap: el('dr-swap'),
+ heard: el('dr-heard'),
+ said: el('dr-said'),
+ fault: el('dr-fault'),
+ faultText: el('dr-fault-text'),
+ faultFix: el('dr-fault-fix'),
+ actHands: el('dr-act-hands'),
+ actRepeat: el('dr-act-repeat'),
+ actStop: el('dr-act-stop'),
+ actType: el('dr-act-type'),
+ talk: el('dr-talk'),
+ picker: el('dr-picker'),
+ pickerClose: el('dr-picker-close'),
+ pickerGrid: el('dr-picker-grid'),
+ pickerSub: el('dr-picker-sub'),
+ type: el('dr-type'),
+ typeClose: el('dr-type-close'),
+ typeForm: el('dr-type-form'),
+ typeInput: el('dr-type-input'),
+};
+
+let profile = null;
+let bridge = null;
+let scene = null;
+let mouth = null;
+let controller = null;
+let motionWatch = null;
+let copilot = null;
+let lastReply = '';
+let handsFree = false;
+let moving = true;
+let motionSource = 'none';
+let holding = false;
+let rearmTimer = 0;
+let vadRaf = 0;
+let levelRaf = 0;
+let vadSpokeAt = 0;
+let vadQuietSince = 0;
+let vadStartedAt = 0;
+
+// ── small helpers ────────────────────────────────────────────────────────────
+
+function readStore(key) {
+ try {
+ return localStorage.getItem(key);
+ } catch {
+ return null;
+ }
+}
+
+function writeStore(key, value) {
+ try {
+ if (value === null) localStorage.removeItem(key);
+ else localStorage.setItem(key, value);
+ } catch {
+ // A locked-down web view without storage still drives fine; it just
+ // forgets the choice between trips.
+ }
+}
+
+function setFault(message, fix) {
+ if (!message) {
+ ui.fault.hidden = true;
+ ui.faultFix.hidden = true;
+ ui.faultFix.onclick = null;
+ return;
+ }
+ ui.faultText.textContent = message;
+ ui.fault.hidden = false;
+ if (fix?.label && typeof fix.run === 'function') {
+ ui.faultFix.textContent = fix.label;
+ ui.faultFix.hidden = false;
+ ui.faultFix.onclick = fix.run;
+ } else {
+ ui.faultFix.hidden = true;
+ ui.faultFix.onclick = null;
+ }
+ bridge?.error(fix?.code || 'drive', message);
+}
+
+function setStatus(text) {
+ ui.status.textContent = text;
+}
+
+function setSaid(text) {
+ lastReply = String(text || '').trim();
+ ui.said.textContent = lastReply;
+ ui.actRepeat.disabled = !lastReply;
+ pushActions();
+ if (lastReply) bridge?.said(lastReply);
+}
+
+function setHeard(text) {
+ const t = String(text || '').trim();
+ ui.heard.textContent = t;
+ ui.heard.classList.toggle('is-on', !!t);
+ if (t) bridge?.heard(t);
+}
+
+// ── the four controls, mirrored to the car screen ────────────────────────────
+
+function pushActions() {
+ if (!bridge?.attached) return;
+ bridge.actions([
+ { id: 'talk', label: holding || controller?.state === 'listening' ? 'Stop' : 'Talk', enabled: !!controller },
+ { id: 'hands', label: handsFree ? 'Hands free on' : 'Hands free', enabled: !!controller },
+ { id: 'repeat', label: 'Repeat', enabled: !!lastReply },
+ { id: 'hush', label: 'Quiet', enabled: controller?.state === 'speaking' },
+ ]);
+}
+
+// ── day / night ──────────────────────────────────────────────────────────────
+
+function applyLight(on) {
+ document.documentElement.setAttribute('data-drive-light', on ? '1' : '0');
+ ui.light.textContent = on ? 'Day' : 'Night';
+ ui.light.setAttribute('aria-pressed', on ? 'true' : 'false');
+ writeStore(STORE_LIGHT, on ? '1' : '0');
+}
+
+function lightOn() {
+ return document.documentElement.getAttribute('data-drive-light') === '1';
+}
+
+// ── parked / driving ─────────────────────────────────────────────────────────
+
+function applyMotion(next) {
+ moving = next;
+ ui.motion.textContent = moving ? 'Driving' : 'Parked';
+ ui.motion.setAttribute('aria-pressed', moving ? 'true' : 'false');
+ ui.motion.title = moving
+ ? 'Driving locks the keyboard. Switch to Parked to type.'
+ : 'Parked. The keyboard is available.';
+ // Typing is only ever offered on a panel that has a reachable keyboard, and
+ // only when the car is not moving.
+ const typable = profile?.keyboard && !moving;
+ ui.actType.hidden = !profile?.keyboard;
+ ui.actType.disabled = !typable;
+ if (moving && !ui.type.hidden) closeType();
+}
+
+// ── the voice loop ───────────────────────────────────────────────────────────
+
+function systemPrompt() {
+ const name = copilot?.name || 'the agent';
+ return [
+ `You are ${name}, riding in a moving car with the person you are talking to.`,
+ 'This is a spoken conversation on a car display. Reply in one or two short sentences that sound right out loud.',
+ 'Never ask them to read anything, tap through a list, or look at the screen.',
+ 'Never produce markdown, bullet points, code, links, or emoji.',
+ 'If something genuinely needs a screen, say you will have it waiting when they park.',
+ 'If you are not sure what they said over road noise, ask one short clarifying question.',
+ ].join(' ');
+}
+
+function stopVad() {
+ if (vadRaf) cancelAnimationFrame(vadRaf);
+ vadRaf = 0;
+ vadSpokeAt = 0;
+ vadQuietSince = 0;
+ vadStartedAt = 0;
+}
+
+// Hands-free turn end. The browser recognizer ends its own turn on silence, so
+// this only runs on the Riva lane, where the raw level is ours to read.
+function startVad() {
+ stopVad();
+ if (controller?.sttMode !== 'riva') return;
+ vadStartedAt = performance.now();
+ const tick = () => {
+ if (!controller || controller.state !== 'listening') {
+ stopVad();
+ return;
+ }
+ const now = performance.now();
+ const level = controller.micLevel;
+ if (level >= SPEECH_LEVEL) {
+ vadSpokeAt = now;
+ vadQuietSince = 0;
+ } else if (vadSpokeAt) {
+ if (!vadQuietSince) vadQuietSince = now;
+ if (now - vadQuietSince >= QUIET_MS) {
+ stopVad();
+ controller.stopListening();
+ return;
+ }
+ }
+ if (now - vadStartedAt >= TURN_CAP_MS) {
+ stopVad();
+ controller.stopListening();
+ return;
+ }
+ vadRaf = requestAnimationFrame(tick);
+ };
+ vadRaf = requestAnimationFrame(tick);
+}
+
+function stopLevelMeter() {
+ if (levelRaf) cancelAnimationFrame(levelRaf);
+ levelRaf = 0;
+ ui.talk.style.setProperty('--dr-mic', '0');
+}
+
+function startLevelMeter() {
+ stopLevelMeter();
+ const tick = () => {
+ ui.talk.style.setProperty('--dr-mic', controller.micLevel.toFixed(3));
+ levelRaf = requestAnimationFrame(tick);
+ };
+ levelRaf = requestAnimationFrame(tick);
+}
+
+function clearRearm() {
+ if (rearmTimer) clearTimeout(rearmTimer);
+ rearmTimer = 0;
+}
+
+function maybeRearm() {
+ clearRearm();
+ if (!handsFree || !controller) return;
+ if (document.hidden || !ui.type.hidden || !ui.picker.hidden) return;
+ rearmTimer = setTimeout(() => {
+ rearmTimer = 0;
+ if (!handsFree || !controller || controller.state !== 'idle') return;
+ beginListening();
+ }, REARM_MS);
+}
+
+function beginListening() {
+ if (!controller) return;
+ setFault('');
+ const started = controller.startListening();
+ if (started && handsFree) startVad();
+ pushActions();
+}
+
+function endListening() {
+ stopVad();
+ controller?.stopListening();
+ pushActions();
+}
+
+function setHandsFree(on) {
+ handsFree = !!on;
+ ui.actHands.setAttribute('aria-pressed', handsFree ? 'true' : 'false');
+ ui.actHands.textContent = handsFree ? 'Hands free · on' : 'Hands free';
+ if (profile?.remembersHandsFree) writeStore(STORE_HANDS, handsFree ? '1' : '0');
+ if (handsFree && controller?.state === 'idle') maybeRearm();
+ if (!handsFree) {
+ clearRearm();
+ stopVad();
+ }
+ pushActions();
+}
+
+function hush() {
+ clearRearm();
+ stopVad();
+ controller?.hush();
+ pushActions();
+}
+
+function repeat() {
+ if (!lastReply || !controller) return;
+ controller.speakText(lastReply);
+}
+
+function nudgeVolume(delta) {
+ if (!controller) return;
+ const next = controller.setVolume(controller.volume + delta);
+ setStatus(`Volume ${Math.round(next * 100)}%`);
+ setTimeout(() => {
+ if (controller?.state === 'idle') setStatus(STATUS.idle);
+ }, 1400);
+}
+
+const interceptor = createDriveInterceptor({
+ repeat,
+ hush,
+ louder: () => nudgeVolume(0.2),
+ quieter: () => nudgeVolume(-0.2),
+ night: () => applyLight(false),
+ day: () => applyLight(true),
+ parked: () => applyMotion(false),
+ driving: () => applyMotion(true),
+});
+
+function onTalkError(err) {
+ const code = err?.code || '';
+ if (code === 'stt-unavailable' || code === 'mic-denied' || err?.name === 'NotAllowedError') {
+ setFault(
+ profile?.keyboard
+ ? 'No microphone here. Park and use the keyboard instead.'
+ : 'No microphone here. Grant microphone access to this app and try again.',
+ profile?.keyboard ? { label: 'Type', run: openType, code } : null,
+ );
+ return;
+ }
+ setFault(err?.message || 'Something went wrong. Hold Talk to try again.', {
+ label: 'Retry',
+ run: () => {
+ setFault('');
+ beginListening();
+ },
+ code: code || 'talk',
+ });
+}
+
+// Never leave a rig standing in bind pose. Most avatars here ship no clip of
+// their own, so the shared retargeted library (idle, wave, and the rest) is what
+// makes the copilot look alive; a rig it genuinely cannot drive simply stays
+// still rather than throwing.
+async function ensureMotion() {
+ const emotes = scene?.getEmoteController();
+ if (!emotes) return;
+ try {
+ await emotes.loadManifest();
+ if (!scene.playingClip) await scene.playEmote('idle');
+ } catch (err) {
+ log.warn('[drive] idle motion unavailable:', err?.message);
+ }
+}
+
+/** A hello as the trip starts, settling straight back into idle. */
+function greet() {
+ scene?.playEmoteOnce('wave', { settleTo: 'idle' }).catch(() => {});
+}
+
+// ── session ──────────────────────────────────────────────────────────────────
+
+async function startSession(avatar) {
+ copilot = avatar;
+ ui.name.textContent = avatar.name || 'Your agent';
+ setStatus('Loading');
+ setFault('');
+
+ if (scene) {
+ scene.unmount();
+ scene = null;
+ }
+ if (controller) {
+ controller.stop();
+ controller = null;
+ }
+ mouth?.dispose();
+
+ scene = new TalkScene();
+ mouth = new AvatarMouthTarget();
+
+ try {
+ await scene.mount({ container: ui.stage, glbUrl: avatar.model_url, cameraPreset: 'half' });
+ scene.attachMouthTarget(mouth);
+ await ensureMotion();
+ } catch (err) {
+ setStatus('Ready');
+ setFault(`Could not load ${avatar.name || 'that agent'}: ${err.message}`, {
+ label: 'Choose another',
+ run: () => openPicker(true),
+ code: 'model',
+ });
+ return;
+ }
+
+ controller = new TalkController({
+ avatar: { id: avatar.id, name: avatar.name, agent_id: avatar.agent_id || null, model_url: avatar.model_url },
+ systemPromptFn: systemPrompt,
+ mouthTarget: mouth,
+ commandInterceptor: interceptor,
+ onMessage: (m) => {
+ if (m.role === 'user') setHeard(m.content);
+ else setSaid(m.content);
+ },
+ onInterim: (t) => setHeard(t),
+ onStateChange: (state) => {
+ ui.root.dataset.state = state;
+ setStatus(STATUS[state] || state);
+ ui.actStop.disabled = state !== 'speaking';
+ if (state === 'listening') startLevelMeter();
+ else stopLevelMeter();
+ if (state === 'idle') maybeRearm();
+ bridge?.state(state);
+ pushActions();
+ },
+ onError: onTalkError,
+ });
+
+ await controller.prepare().catch(() => {});
+ ui.talk.disabled = false;
+ setStatus(STATUS.idle);
+ writeStore(
+ STORE_COPILOT,
+ JSON.stringify({
+ id: avatar.id,
+ name: avatar.name,
+ model_url: avatar.model_url,
+ agent_id: avatar.agent_id || null,
+ }),
+ );
+ bridge?.ready({ id: avatar.id, name: avatar.name });
+ greet();
+ pushActions();
+ if (handsFree) maybeRearm();
+}
+
+// ── choosing a copilot ───────────────────────────────────────────────────────
+
+async function fetchAvatar(id) {
+ const r = await fetch(`/api/avatars/${encodeURIComponent(id)}`, { credentials: 'include' });
+ if (!r.ok) throw new Error(r.status === 404 ? 'That agent no longer exists.' : `Could not load that agent (${r.status}).`);
+ const j = await r.json();
+ const a = j?.avatar || j;
+ const modelUrl = a?.model_url || a?.url;
+ if (!modelUrl) throw new Error('That agent has no 3D model yet.');
+ return { id: a.id, name: a.name || 'Your agent', model_url: modelUrl, agent_id: a.agent_id || null };
+}
+
+async function listChoices() {
+ try {
+ const mine = await fetch('/api/avatars?limit=24', { credentials: 'include' });
+ if (mine.ok) {
+ const j = await mine.json();
+ if (Array.isArray(j?.avatars) && j.avatars.length) return { avatars: j.avatars, owned: true };
+ }
+ } catch (err) {
+ log.warn('[drive] own avatars unavailable:', err?.message);
+ }
+ const featured = await fetch('/api/avatars/featured?limit=24');
+ if (!featured.ok) throw new Error(`Could not load agents (${featured.status}).`);
+ const j = await featured.json();
+ return { avatars: j?.avatars || [], owned: false };
+}
+
+function skeletons(n) {
+ ui.pickerGrid.replaceChildren();
+ for (let i = 0; i < n; i += 1) {
+ const s = document.createElement('div');
+ s.className = 'dr-skel';
+ ui.pickerGrid.append(s);
+ }
+}
+
+function renderChoices({ avatars, owned }) {
+ ui.pickerGrid.replaceChildren();
+ ui.pickerSub.textContent = owned
+ ? 'Pick the agent that rides with you. It keeps its voice, its face, and its memory in the car.'
+ : 'These are public agents anyone can ride with. Sign in to bring your own.';
+
+ if (!avatars.length) {
+ const empty = document.createElement('div');
+ empty.className = 'dr-empty';
+ const line = document.createElement('span');
+ line.textContent = 'No agents yet. Build one in a couple of minutes and it will be waiting here.';
+ const link = document.createElement('a');
+ link.className = 'dr-link';
+ link.href = '/create';
+ link.textContent = 'Create an agent';
+ empty.append(line, link);
+ ui.pickerGrid.append(empty);
+ return;
+ }
+
+ for (const a of avatars) {
+ const card = document.createElement('button');
+ card.type = 'button';
+ card.className = 'dr-card';
+ const img = document.createElement('img');
+ img.src = `/api/avatars/${encodeURIComponent(a.id)}/thumb`;
+ img.alt = '';
+ img.loading = 'lazy';
+ img.decoding = 'async';
+ const name = document.createElement('span');
+ name.className = 'dr-card-name';
+ name.textContent = a.name || 'Untitled agent';
+ card.append(img, name);
+ card.addEventListener('click', () => void pick(a.id, card));
+ ui.pickerGrid.append(card);
+ }
+}
+
+async function pick(id, card) {
+ if (card) card.disabled = true;
+ try {
+ const avatar = await fetchAvatar(id);
+ closePicker();
+ await startSession(avatar);
+ } catch (err) {
+ if (card) card.disabled = false;
+ setFault(err.message, null);
+ ui.picker.hidden = false;
+ }
+}
+
+async function openPicker(dismissable) {
+ ui.picker.hidden = false;
+ ui.pickerClose.hidden = !dismissable && !copilot;
+ clearRearm();
+ skeletons(8);
+ try {
+ renderChoices(await listChoices());
+ } catch (err) {
+ ui.pickerGrid.replaceChildren();
+ const empty = document.createElement('div');
+ empty.className = 'dr-empty';
+ const line = document.createElement('span');
+ line.textContent = err.message;
+ const retry = document.createElement('button');
+ retry.type = 'button';
+ retry.className = 'dr-pill';
+ retry.textContent = 'Try again';
+ retry.addEventListener('click', () => void openPicker(dismissable));
+ empty.append(line, retry);
+ ui.pickerGrid.append(empty);
+ }
+}
+
+function closePicker() {
+ ui.picker.hidden = true;
+ if (handsFree) maybeRearm();
+}
+
+// ── typing (parked only) ─────────────────────────────────────────────────────
+
+function openType() {
+ if (!profile?.keyboard || moving) return;
+ clearRearm();
+ ui.type.hidden = false;
+ ui.typeInput.focus();
+}
+
+function closeType() {
+ ui.type.hidden = true;
+ if (handsFree) maybeRearm();
+}
+
+// ── wiring ───────────────────────────────────────────────────────────────────
+
+function wire() {
+ // Hold to talk. Pointer events cover mouse, touch, and a resistive head unit
+ // panel in one path; the capture keeps the release ours even if the finger
+ // slides off the button on a bump.
+ ui.talk.addEventListener('pointerdown', (e) => {
+ if (ui.talk.disabled) return;
+ holding = true;
+ ui.talk.setPointerCapture?.(e.pointerId);
+ beginListening();
+ });
+ const release = () => {
+ if (!holding) return;
+ holding = false;
+ if (!handsFree) endListening();
+ else pushActions();
+ };
+ ui.talk.addEventListener('pointerup', release);
+ ui.talk.addEventListener('pointercancel', release);
+ ui.talk.addEventListener('lostpointercapture', release);
+
+ // Keyboard parity: space is push to talk, escape is barge-in.
+ window.addEventListener('keydown', (e) => {
+ if (e.repeat || e.target instanceof HTMLInputElement) return;
+ if (e.code === 'Space') {
+ e.preventDefault();
+ if (!holding && controller?.state === 'idle') {
+ holding = true;
+ beginListening();
+ }
+ } else if (e.key === 'Escape') {
+ if (!ui.type.hidden) closeType();
+ else if (!ui.picker.hidden && copilot) closePicker();
+ else hush();
+ }
+ });
+ window.addEventListener('keyup', (e) => {
+ if (e.code === 'Space' && holding) {
+ e.preventDefault();
+ release();
+ }
+ });
+
+ ui.actHands.addEventListener('click', () => setHandsFree(!handsFree));
+ ui.actRepeat.addEventListener('click', repeat);
+ ui.actStop.addEventListener('click', hush);
+ ui.actType.addEventListener('click', openType);
+ ui.swap.addEventListener('click', () => void openPicker(true));
+ ui.light.addEventListener('click', () => applyLight(!lightOn()));
+ ui.motion.addEventListener('click', () => {
+ // A live speed signal outranks the button: the driver cannot declare
+ // themselves parked while the wheels are turning.
+ if (motionSource === 'gps' && moving) {
+ setStatus('Moving');
+ setTimeout(() => setStatus(STATUS[controller?.state || 'idle'] || 'Ready'), 1400);
+ return;
+ }
+ applyMotion(!moving);
+ });
+ ui.pickerClose.addEventListener('click', closePicker);
+ ui.typeClose.addEventListener('click', closeType);
+ ui.typeForm.addEventListener('submit', (e) => {
+ e.preventDefault();
+ const text = ui.typeInput.value.trim();
+ if (!text || !controller) return;
+ ui.typeInput.value = '';
+ closeType();
+ controller.say(text).catch(onTalkError);
+ });
+
+ document.addEventListener('visibilitychange', () => {
+ if (document.hidden) {
+ clearRearm();
+ stopVad();
+ } else if (handsFree) {
+ maybeRearm();
+ }
+ });
+
+ window.addEventListener('pagehide', () => {
+ motionWatch?.stop();
+ controller?.stop();
+ scene?.unmount();
+ bridge?.dispose();
+ });
+}
+
+function onNativeCommand(cmd) {
+ switch (cmd.type) {
+ case 'talk-start':
+ beginListening();
+ break;
+ case 'talk-stop':
+ endListening();
+ break;
+ case 'talk':
+ if (controller?.state === 'listening') endListening();
+ else beginListening();
+ break;
+ case 'hands':
+ setHandsFree(!handsFree);
+ break;
+ case 'repeat':
+ repeat();
+ break;
+ case 'hush':
+ hush();
+ break;
+ case 'say':
+ if (typeof cmd.value === 'string') controller?.say(cmd.value).catch(onTalkError);
+ break;
+ default:
+ log.warn('[drive] unknown native command:', cmd.type);
+ }
+}
+
+async function boot() {
+ profile = surfaceProfile(detectSurface());
+ applySurface(profile);
+ bridge = createBridge(onNativeCommand);
+
+ applyLight(readStore(STORE_LIGHT) === '1');
+ applyMotion(true);
+ if (profile.remembersHandsFree) setHandsFree(readStore(STORE_HANDS) === '1');
+ else setHandsFree(false);
+
+ wire();
+ ui.talk.disabled = true;
+
+ motionWatch = watchMotion(({ moving: isMoving, source }) => {
+ motionSource = source;
+ if (source === 'gps') applyMotion(isMoving);
+ });
+
+ const params = new URLSearchParams(location.search);
+ const asked = params.get('avatar') || params.get('agent');
+ if (asked) {
+ try {
+ await startSession(await fetchAvatar(asked));
+ return;
+ } catch (err) {
+ setFault(err.message, { label: 'Choose an agent', run: () => void openPicker(false), code: 'avatar' });
+ }
+ }
+
+ const saved = readStore(STORE_COPILOT);
+ if (saved) {
+ try {
+ const parsed = JSON.parse(saved);
+ if (parsed?.id && parsed?.model_url) {
+ await startSession(parsed);
+ return;
+ }
+ } catch {
+ writeStore(STORE_COPILOT, null);
+ }
+ }
+
+ await openPicker(false);
+}
+
+boot().catch((err) => {
+ log.error('[drive] boot failed:', err);
+ setStatus('Offline');
+ setFault(`Drive could not start: ${err.message}`, { label: 'Reload', run: () => location.reload(), code: 'boot' });
+});
diff --git a/src/drive/motion.js b/src/drive/motion.js
new file mode 100644
index 000000000..08911d7a5
--- /dev/null
+++ b/src/drive/motion.js
@@ -0,0 +1,74 @@
+// Is the car moving?
+//
+// Typing at speed is the one thing every in-car interface guideline agrees on:
+// it does not happen. /drive therefore starts in Driving and keeps the keyboard
+// away. The honest question is when to let it back.
+//
+// When the browser can give us a real ground speed we use it: above walking
+// pace the surface locks itself back to Driving and closes anything open.
+// Below it, the driver may switch to Parked themselves. When there is no speed
+// signal at all (permission denied, no GPS, a desktop preview) the toggle is
+// simply manual, and it still defaults to Driving.
+
+/** Metres per second. ~10 mph: unambiguously moving, not a slow walk with the phone. */
+const MOVING_MPS = 4.5;
+/** Below this for STILL_MS and the vehicle is treated as stopped. */
+const STILL_MPS = 0.9;
+const STILL_MS = 6000;
+
+/**
+ * @param {(status: { moving: boolean, speedMps: number|null, source: 'gps'|'none' }) => void} onChange
+ * @returns {{ stop: () => void, available: boolean }}
+ */
+export function watchMotion(onChange, geo = typeof navigator !== 'undefined' ? navigator.geolocation : null) {
+ if (!geo?.watchPosition) {
+ onChange({ moving: false, speedMps: null, source: 'none' });
+ return { stop: () => {}, available: false };
+ }
+
+ let stillSince = 0;
+ let last = null;
+ let watchId = null;
+
+ const emit = (moving, speedMps) => {
+ if (last === moving) return;
+ last = moving;
+ onChange({ moving, speedMps, source: 'gps' });
+ };
+
+ watchId = geo.watchPosition(
+ (pos) => {
+ const speed = typeof pos?.coords?.speed === 'number' && pos.coords.speed >= 0 ? pos.coords.speed : null;
+ // A fix with no speed field tells us nothing about motion; leave the
+ // current verdict alone rather than guessing from position deltas,
+ // which are noisy enough at a standstill to flip the keyboard open.
+ if (speed === null) return;
+ if (speed >= MOVING_MPS) {
+ stillSince = 0;
+ emit(true, speed);
+ return;
+ }
+ if (speed <= STILL_MPS) {
+ const now = Date.now();
+ if (!stillSince) stillSince = now;
+ if (now - stillSince >= STILL_MS) emit(false, speed);
+ } else {
+ stillSince = 0;
+ }
+ },
+ () => {
+ // Denied or unavailable: fall back to the manual toggle, which is
+ // already the safe default.
+ onChange({ moving: false, speedMps: null, source: 'none' });
+ },
+ { enableHighAccuracy: true, maximumAge: 2000, timeout: 20000 },
+ );
+
+ return {
+ available: true,
+ stop() {
+ if (watchId !== null) geo.clearWatch(watchId);
+ watchId = null;
+ },
+ };
+}
diff --git a/src/drive/surface.js b/src/drive/surface.js
new file mode 100644
index 000000000..627a29cbe
--- /dev/null
+++ b/src/drive/surface.js
@@ -0,0 +1,84 @@
+// Surface presets for /drive.
+//
+// The same page renders on four physically different panels: a CarPlay-era
+// phone in a cradle, an Android Auto phone, a built-in head unit browser
+// (Tesla, Android Automotive), and a desktop preview. They differ in viewing
+// distance, safe area, and whether a keyboard is reachable at all, so the page
+// reads one preset at boot and everything else scales off it.
+//
+// `?surface=` wins when present, because the native shells pass it explicitly.
+
+/** @typedef {'carplay'|'androidauto'|'headunit'|'cradle'|'browser'} SurfaceName */
+
+const NAMES = new Set(['carplay', 'androidauto', 'headunit', 'cradle', 'browser']);
+
+/**
+ * Which panel are we on? An explicit `?surface=` beats detection; a native
+ * shell that injected its message channel identifies itself; a standalone
+ * install on a coarse pointer is a phone in a mount; everything else is a
+ * browser preview.
+ * @returns {SurfaceName}
+ */
+export function detectSurface(search = typeof location !== 'undefined' ? location.search : '') {
+ const asked = new URLSearchParams(search).get('surface');
+ if (asked && NAMES.has(asked)) return /** @type {SurfaceName} */ (asked);
+ if (typeof window === 'undefined') return 'browser';
+ if (window.webkit?.messageHandlers?.threeWsDrive) return 'carplay';
+ if (window.ThreeWsDriveNative) return 'androidauto';
+ const standalone =
+ window.matchMedia?.('(display-mode: standalone)')?.matches || window.navigator?.standalone;
+ const coarse = window.matchMedia?.('(pointer: coarse)')?.matches;
+ if (standalone && coarse) return 'cradle';
+ return 'browser';
+}
+
+/**
+ * Glyph scale. A head unit is roughly a metre from the driver's eyes, twice a
+ * phone's distance, so it gets a bigger multiplier; a short dash panel gets a
+ * smaller one because the vertical budget, not legibility, is the constraint.
+ */
+export function scaleFor(surface, width, height) {
+ let scale = 1;
+ if (height < 420) scale = 0.78;
+ else if (height < 560) scale = 0.9;
+ else if (height >= 900) scale = 1.12;
+ if (surface === 'headunit') scale += 0.1;
+ if (surface === 'browser' && width >= 1400) scale = Math.min(scale, 1);
+ return Math.round(scale * 100) / 100;
+}
+
+/**
+ * Everything the page needs to know about the panel it is on.
+ * `keyboard` is whether a keyboard is reachable at all: on a CarPlay or Android
+ * Auto shell the web view lives on the phone, which is in a mount and out of
+ * reach, so the Type affordance is hidden rather than merely disabled.
+ */
+export function surfaceProfile(surface = detectSurface()) {
+ return {
+ surface,
+ keyboard: surface !== 'carplay' && surface !== 'androidauto',
+ // A native shell owns the car screen's own listening indicator, so the
+ // page mirrors its state outward instead of assuming it is alone.
+ native: surface === 'carplay' || surface === 'androidauto',
+ // Hands free is opt-in everywhere, but a car panel is the only place it
+ // is the better default once the driver has enabled it once.
+ remembersHandsFree: surface !== 'browser',
+ };
+}
+
+/**
+ * Apply the preset to the document: a `data-drive-surface` hook for CSS and the
+ * `--dr-scale` multiplier every size in the stylesheet is expressed in.
+ * Returns a teardown that removes the resize listener.
+ */
+export function applySurface(profile, doc = document, win = window) {
+ const root = doc.documentElement;
+ root.setAttribute('data-drive-surface', profile.surface);
+ const sync = () => {
+ const scale = scaleFor(profile.surface, win.innerWidth, win.innerHeight);
+ root.style.setProperty('--dr-scale', String(scale));
+ };
+ sync();
+ win.addEventListener('resize', sync, { passive: true });
+ return () => win.removeEventListener('resize', sync);
+}
diff --git a/src/voice/talk-controller.js b/src/voice/talk-controller.js
index 81a6cfcff..ee56943c6 100644
--- a/src/voice/talk-controller.js
+++ b/src/voice/talk-controller.js
@@ -78,6 +78,12 @@ export class TalkController {
this._currentAudioEl = null;
this._currentTap = null;
this._driver = null;
+ // Playback gain for the spoken reply (0..1). Persists across turns so a
+ // "quieter" instruction survives the next thing the avatar says.
+ this._volume = 1;
+ // Latched once /api/tts/edge answers 401/403: it needs a session, and an
+ // anonymous listener should not pay a wasted round trip on every turn.
+ this._edgeUnavailable = false;
this._voicePromise = null; // resolves to { provider, voiceId } | null
// Speech-to-text routing. 'riva' = server-side NVIDIA Riva (cross-browser),
@@ -465,6 +471,36 @@ export class TalkController {
this._setState('idle');
}
+ /**
+ * Cut the reply off mid-sentence and return to rest, WITHOUT tearing the
+ * session down the way stop() does. This is barge-in: the driver says
+ * "stop talking", the voice ends, and the very next turn still works.
+ */
+ hush() {
+ this._stopPlayback();
+ this._driver?.dispose();
+ this._driver = null;
+ this._setState('idle');
+ }
+
+ /**
+ * Playback volume for spoken replies, 0..1. Applies to the line currently
+ * playing and to every line after it.
+ * @returns {number} the clamped value actually in force.
+ */
+ setVolume(value) {
+ const next = Math.min(1, Math.max(0, Number(value)));
+ if (!Number.isFinite(next)) return this._volume;
+ this._volume = next;
+ if (this._currentAudioEl) this._currentAudioEl.volume = next;
+ return next;
+ }
+
+ /** Current playback volume, 0..1. */
+ get volume() {
+ return this._volume;
+ }
+
/**
* Invalidate the cached voice lookup so the next turn re-checks the agent
* for a (possibly newly cloned) voice_id. Call after the user finishes a
@@ -632,11 +668,12 @@ export class TalkController {
const voice = await this._resolveVoice();
const blob = voice
? await this._fetchTtsEleven(text, voice.voiceId)
- : await this._fetchTtsEdge(text);
+ : await this._fetchTtsFallback(text);
const url = URL.createObjectURL(blob);
const audio = new Audio();
audio.crossOrigin = 'anonymous';
+ audio.volume = this._volume;
audio.src = url;
this._currentAudioEl = audio;
@@ -712,11 +749,49 @@ export class TalkController {
}),
});
if (!r.ok) {
- // Fall back to Edge so the talk loop still completes if ElevenLabs
- // is rate-limited or down.
- log.warn('[talk] eleven TTS failed, falling back to edge');
- return this._fetchTtsEdge(text);
+ // Fall through to the free lanes so the talk loop still completes
+ // if ElevenLabs is rate-limited or down.
+ log.warn('[talk] eleven TTS failed, falling back to the free lanes');
+ return this._fetchTtsFallback(text);
+ }
+ return r.blob();
+ }
+
+ /**
+ * Voice for an avatar with no cloned voice of its own.
+ *
+ * Edge Neural leads when the session can reach it: it selects a voice per
+ * language and body type, which the generic lane cannot. But /api/tts/edge
+ * requires a signed-in session, so an anonymous listener (a public avatar
+ * page, an embed, the car surface) used to get a working reply with no
+ * voice at all. The free NVIDIA Magpie lane on /api/tts/speak has no such
+ * gate, so it backs the chain up rather than leaving silence.
+ */
+ async _fetchTtsFallback(text) {
+ if (!this._edgeUnavailable) {
+ try {
+ return await this._fetchTtsEdge(text);
+ } catch (err) {
+ const message = err?.message || '';
+ if (/\((401|403)\)/.test(message)) this._edgeUnavailable = true;
+ log.warn('[talk] edge TTS unavailable, using the free speak lane:', message);
+ }
}
+ return this._fetchTtsSpeak(text);
+ }
+
+ async _fetchTtsSpeak(text) {
+ const r = await fetch('/api/tts/speak', {
+ method: 'POST',
+ headers: { 'content-type': 'application/json' },
+ credentials: 'include',
+ body: JSON.stringify({
+ text: text.slice(0, 1500),
+ language: this.language,
+ format: 'wav',
+ }),
+ });
+ if (!r.ok) throw new Error(`TTS failed (${r.status})`);
return r.blob();
}
diff --git a/src/voice/talk-scene.js b/src/voice/talk-scene.js
index afabdfcee..1c3810723 100644
--- a/src/voice/talk-scene.js
+++ b/src/voice/talk-scene.js
@@ -203,6 +203,25 @@ export class TalkScene {
return this._emotes;
}
+ /**
+ * Clip names baked into the loaded GLB. Empty means the model shipped with
+ * no animation of its own and needs a retargeted clip from the shared
+ * library, or it stands in bind pose.
+ * @returns {string[]}
+ */
+ get clipNames() {
+ return this._clips.map((c) => c.name);
+ }
+
+ /**
+ * Name of the GLB clip currently playing on the model's own mixer, or null.
+ * A caller can read this to decide whether the rig still needs driving.
+ * @returns {string|null}
+ */
+ get playingClip() {
+ return this._currentAction?.getClip?.()?.name || null;
+ }
+
/** Convenience: play an external emote by name. */
playEmote(name) {
return this._emotes?.play(name) ?? Promise.resolve(false);
diff --git a/tests/drive-surface.test.js b/tests/drive-surface.test.js
new file mode 100644
index 000000000..5ec2184ca
--- /dev/null
+++ b/tests/drive-surface.test.js
@@ -0,0 +1,153 @@
+// three.ws Drive: the pure logic behind the car surface.
+//
+// These three modules decide what the driver can reach and when, so they are
+// the parts worth pinning: a regression here is a keyboard that opens at speed
+// or a command that silently eats a real question.
+
+import { describe, it, expect, vi } from 'vitest';
+import { detectSurface, scaleFor, surfaceProfile } from '../src/drive/surface.js';
+import { matchDriveCommand, normalize, createDriveInterceptor, commandVocabulary } from '../src/drive/commands.js';
+import { watchMotion } from '../src/drive/motion.js';
+
+describe('drive surface detection', () => {
+ it('honours an explicit ?surface=', () => {
+ expect(detectSurface('?surface=headunit')).toBe('headunit');
+ expect(detectSurface('?surface=carplay')).toBe('carplay');
+ });
+
+ it('ignores a surface name it does not know', () => {
+ expect(detectSurface('?surface=spaceship')).toBe('browser');
+ });
+
+ it('scales up for a head unit and down for a short dash', () => {
+ expect(scaleFor('headunit', 1920, 1080)).toBeGreaterThan(scaleFor('browser', 1920, 1080));
+ expect(scaleFor('browser', 800, 400)).toBeLessThan(1);
+ expect(scaleFor('browser', 1280, 720)).toBe(1);
+ });
+
+ it('hides the keyboard on the panels where it cannot be reached', () => {
+ expect(surfaceProfile('carplay').keyboard).toBe(false);
+ expect(surfaceProfile('androidauto').keyboard).toBe(false);
+ expect(surfaceProfile('headunit').keyboard).toBe(true);
+ expect(surfaceProfile('cradle').keyboard).toBe(true);
+ });
+
+ it('knows which panels have a native shell listening', () => {
+ expect(surfaceProfile('carplay').native).toBe(true);
+ expect(surfaceProfile('browser').native).toBe(false);
+ });
+});
+
+describe('drive commands', () => {
+ it('normalizes filler away', () => {
+ expect(normalize('Hey, repeat that, please!')).toBe('repeat that');
+ expect(normalize(' OK Stop. ')).toBe('stop');
+ });
+
+ it('matches the control vocabulary', () => {
+ expect(matchDriveCommand('repeat that')).toBe('repeat');
+ expect(matchDriveCommand('stop talking')).toBe('hush');
+ expect(matchDriveCommand('turn it up')).toBe('louder');
+ expect(matchDriveCommand('night mode')).toBe('night');
+ expect(matchDriveCommand("i'm parked")).toBe('parked');
+ });
+
+ // The whole point of whole-utterance matching: a command word inside a real
+ // request must reach the agent, not the local handler.
+ it('never swallows a real question that merely contains a command word', () => {
+ expect(matchDriveCommand('stop at the next charger')).toBe(null);
+ expect(matchDriveCommand('is the traffic quieter on the parkway')).toBe(null);
+ expect(matchDriveCommand('repeat the address for the restaurant we passed')).toBe(null);
+ expect(matchDriveCommand('what time do we get there')).toBe(null);
+ });
+
+ it('interceptor runs the handler and reports the turn as handled', async () => {
+ const hush = vi.fn();
+ const intercept = createDriveInterceptor({ hush });
+ expect(await intercept('stop')).toBe(true);
+ expect(hush).toHaveBeenCalledOnce();
+ });
+
+ it('interceptor passes an unmatched utterance through to the agent', async () => {
+ const hush = vi.fn();
+ const intercept = createDriveInterceptor({ hush });
+ expect(await intercept('how far to the next charger')).toBe(false);
+ expect(hush).not.toHaveBeenCalled();
+ });
+
+ it('a matched command with no handler still falls through rather than vanishing', async () => {
+ const intercept = createDriveInterceptor({});
+ expect(await intercept('repeat that')).toBe(false);
+ });
+
+ it('every vocabulary phrase resolves to its own id', () => {
+ for (const { id, phrases } of commandVocabulary()) {
+ for (const phrase of phrases) expect(matchDriveCommand(phrase)).toBe(id);
+ }
+ });
+});
+
+describe('drive motion', () => {
+ const fakeGeo = () => {
+ let cb = null;
+ return {
+ watchPosition(success) {
+ cb = success;
+ return 7;
+ },
+ clearWatch: vi.fn(),
+ emit(speed) {
+ cb({ coords: { speed } });
+ },
+ };
+ };
+
+ it('reports no signal when geolocation is unavailable', () => {
+ const seen = [];
+ const w = watchMotion((s) => seen.push(s), null);
+ expect(w.available).toBe(false);
+ expect(seen).toEqual([{ moving: false, speedMps: null, source: 'none' }]);
+ });
+
+ it('flips to moving above walking pace', () => {
+ const geo = fakeGeo();
+ const seen = [];
+ watchMotion((s) => seen.push(s), geo);
+ geo.emit(12);
+ expect(seen.at(-1)).toMatchObject({ moving: true, source: 'gps' });
+ });
+
+ it('ignores a fix that carries no speed at all', () => {
+ const geo = fakeGeo();
+ const seen = [];
+ watchMotion((s) => seen.push(s), geo);
+ geo.emit(null);
+ expect(seen).toHaveLength(0);
+ });
+
+ // Stopping at a light must not unlock the keyboard: only a sustained stop does.
+ it('needs a sustained stop before it calls the car parked', () => {
+ vi.useFakeTimers();
+ try {
+ const geo = fakeGeo();
+ const seen = [];
+ watchMotion((s) => seen.push(s), geo);
+ geo.emit(15);
+ expect(seen.at(-1).moving).toBe(true);
+ geo.emit(0);
+ expect(seen.at(-1).moving).toBe(true);
+ vi.advanceTimersByTime(7000);
+ geo.emit(0);
+ expect(seen.at(-1).moving).toBe(false);
+ } finally {
+ vi.useRealTimers();
+ }
+ });
+
+ it('stops the watch it opened', () => {
+ const geo = fakeGeo();
+ const w = watchMotion(() => {}, geo);
+ w.stop();
+ expect(geo.clearWatch).toHaveBeenCalledWith(7);
+ });
+});
diff --git a/vercel.json b/vercel.json
index 331c73dcc..6394ebc83 100644
--- a/vercel.json
+++ b/vercel.json
@@ -202,6 +202,13 @@
},
"continue": true
},
+ {
+ "src": "/drive/?$",
+ "headers": {
+ "permissions-policy": "accelerometer=(self), autoplay=(self), camera=(), display-capture=(), encrypted-media=(self), geolocation=(self), gyroscope=(self), magnetometer=(self), microphone=(self), midi=(), payment=(), serial=(), usb=(), xr-spatial-tracking=()"
+ },
+ "continue": true
+ },
{
"src": "/world-lines/?$",
"headers": {
@@ -5463,6 +5470,14 @@
"src": "/concierge/",
"dest": "/concierge.html"
},
+ {
+ "src": "/drive",
+ "dest": "/drive.html"
+ },
+ {
+ "src": "/drive/",
+ "dest": "/drive.html"
+ },
{
"src": "/tour-builder",
"dest": "/tour-builder.html"
diff --git a/vite.config.js b/vite.config.js
index a93e306d9..3844cb981 100644
--- a/vite.config.js
+++ b/vite.config.js
@@ -559,6 +559,7 @@ const appConfig = {
tour: resolve(__dirname, 'pages/tour.html'),
'tour-atlas': resolve(__dirname, 'pages/tour-atlas.html'),
concierge: resolve(__dirname, 'pages/concierge.html'),
+ drive: resolve(__dirname, 'pages/drive.html'),
'tour-builder': resolve(__dirname, 'pages/tour-builder.html'),
'agent-identities': resolve(__dirname, 'pages/agent-identities.html'),
'mcp-tools': resolve(__dirname, 'pages/mcp-tools.html'),
@@ -1294,6 +1295,8 @@ const appConfig = {
'/app-classic': resolve(root, 'pages/app.html'),
'/widget': resolve(root, 'pages/widget.html'),
'/widget/': resolve(root, 'pages/widget.html'),
+ '/drive': resolve(root, 'pages/drive.html'),
+ '/drive/': resolve(root, 'pages/drive.html'),
'/login': resolve(root, 'public/login.html'),
'/deploy': resolve(root, 'pages/app.html'),
'/showcase': resolve(root, 'pages/app.html'),