Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
JavaScript 3d-generationai-agentsanimationavatardigital-humangenerative-aiglbgltfimage-to-3dllmmcpmcp-servermodel-context-protocolsolanatext-to-3dthreejsweb-componentswebglwebxrx402
docs(roadmap): add Phase 5, native widgets on the home screen, starting with the shipped Android TWA
nirholas578fd492bc19
diff --git a/README.md b/README.mdindex e0f45ae1c..f0b1a1f7c 100644--- a/README.md+++ b/README.md@@ -10643,7 +10643,7 @@ This is the direction three.ws is heading: **photo → avatar → agent → onch## Roadmap-three.ws ships in four phases. Each phase closes a specific gap between the current platform and the end-state vision: **anyone can mint a 3D agent of themselves, own it onchain, and embed it anywhere on the internet.**+three.ws ships in five phases. Each phase closes a specific gap between the current platform and the end-state vision: **anyone can mint a 3D agent of themselves, own it onchain, and embed it anywhere on the internet.**| Phase | Theme | Status || ----- | -------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |@@ -10652,6 +10652,7 @@ three.ws ships in four phases. Each phase closes a specific gap between the curr| **2** | Agent personalization + voice cloning | 🟡 In progress — voice clone, persona, memory seeds shipped behind `/demos`; main-flow integration next || **3** | Onchain economy (agent tokens, reputation markets, royalties) | 🟡 In progress: bonding-curve sim, EAS-reputation viewer, 0xsplits + EAS SDKs landed; per-call skill royalties accrue on paid skill calls (`royalty_ledger`); contracts + audits next || **4** | Open inference network (decentralized GPU layer) | 🟡 Live core: open node-operator client (CPU + CUDA images) + `/api/nodes` job queue with signed, server-recomputed receipts shipped; Livepeer federation behind a flag |+| **5** | Native widgets (agent on your home screen and desktop) | 🔵 Planned: Android app widget inside the shipped `ws.three.app` TWA first, then Windows 11, then macOS and iOS |---@@ -10761,6 +10762,35 @@ The path a capture actually takes, end to end. Each hop is a real service; nothi---+### Phase 5: Native Widgets++**Goal:** your agent lives on the home screen, not only in a browser tab. A glanceable widget that shows the agent you own, one live number about it, and a single tap back into the app.++This is a different product from the embeddable web widgets at [three.ws/widgets](https://three.ws/widgets), which put a 3D agent on someone else's web page. Phase 5 is about the operating system surfaces: the Android home screen, the Windows 11 widgets board, and the macOS and iOS widget galleries.++**Why it is next.** The Android shell already exists. three.ws ships as a signed Trusted Web Activity (`ws.three.app`) for the Solana dApp Store, Digital Asset Links verify against the live release key, and deep links, shortcuts and the share sheet already open the app. A widget is the piece that makes the app worth keeping installed between visits. The render side exists too: `POST /api/render/avatar-clip` already produces a posed, camera-framed PNG of any avatar from the headless renderer, which is exactly the image a widget needs, because no widget runtime on any of these platforms can execute WebGL.++**Order of delivery**++| # | Surface | Shell needed | Notes |+| - | ------- | ------------ | ----- |+| 1 | Android home screen | Already shipped (`ws.three.app`) | Native `AppWidgetProvider` overlay added to the Bubblewrap project, refreshed by `WorkManager`, tapping deep links into the TWA |+| 2 | Windows 11 widgets board | None | The installed PWA declares a `widgets` manifest member and an Adaptive Card template; the service worker answers `widgetinstall` and `widgetclick` |+| 3 | macOS | Small SwiftUI host app | WidgetKit extension plus a menu bar companion; needs an Apple Developer account |+| 4 | iOS | App Store wrapper | Same WidgetKit extension target as macOS; gated on shipping the iOS app itself |++**Deliverables**++- A cacheable widget data endpoint: one signed, per-account URL returning the agent card as a PNG (Android, macOS, iOS) and as JSON for the Windows Adaptive Card, built on the existing render pipeline rather than a second renderer+- Android app widget in three sizes, refreshed on a battery-aware schedule, degrading to the last cached card when the device is offline+- Windows 11 widget through the PWA manifest, so it installs with no separate store submission+- A native shell for Apple platforms, which is also the prerequisite for an iOS build of three.ws+- Widget content that is worth a home screen slot: the agent, its recent activity, and a direct action (create, chat, open my agents)++**Verification:** a widget installed from the Android app updates without opening the app, survives a reboot and airplane mode, and returns the user into the right screen on tap. Work order: [prompts/roadmap/native-widgets.md](prompts/roadmap/native-widgets.md).++---+### What we need| Resource | Used for | Phase |@@ -10772,6 +10802,7 @@ The path a capture actually takes, end to end. Each hop is a real service; nothi| **Indexer infrastructure** | Multi-chain crawl + reputation aggregation | 3 || **Node operator credits** | Bootstrap the open inference network | 4 || **Engineering headcount** | Capture pipeline, contracts, indexer, ops | 1–4 |+| **Apple Developer account** | macOS and iOS widget host app, App Store review | 5 |Phases 1 and 2 unblock the consumer story — _anyone gets an agent of themselves_. Phases 3 and 4 unblock the onchain story — _those agents are real economic actors that don't depend on any one company to keep running_. Both are required for the vision; neither is funded yet.diff --git a/docs/seeker-app.md b/docs/seeker-app.mdindex b742e6acc..548eee323 100644--- a/docs/seeker-app.md+++ b/docs/seeker-app.md@@ -38,3 +38,14 @@ The handoff is one-shot and expires after ten minutes, so a stale share can neve## Testing without a SeekerThe APK can be exercised in a stock Android emulator (Pixel 7, Android 14 image). Everything except the Seed Vault itself works there: install, launch, shortcuts, the share intent, deep-link verification state. The recipe is in [solana-mobile/README.md](../solana-mobile/README.md#emulator-qa). For the wallet flows, `tests/solana-mobile-*.test.js` drive the MWA wrapper against a fake transport, and the Solana Mobile reference wallet can be sideloaded into the emulator for an end-to-end sign-in.++## What is next: a home screen widget++The app currently earns its place on the home screen only while you are using it. The next+piece is an Android app widget: your agent, one live number about it, and a tap that lands you+back in the right screen, refreshed by the system without opening the app. The shell for it+already exists (this APK) and so does the image source (`POST /api/render/avatar-clip`, the+same headless renderer behind the OG cards), because no Android widget can run WebGL. Windows+11 follows through the installed PWA's manifest, then macOS and iOS through a shared WidgetKit+extension. The scoped work order is [prompts/roadmap/native-widgets.md](../prompts/roadmap/native-widgets.md),+and it is Phase 5 of the [roadmap](../README.md#roadmap).diff --git a/prompts/roadmap/00-README.md b/prompts/roadmap/00-README.mdindex 898068c3f..20dc9e373 100644--- a/prompts/roadmap/00-README.md+++ b/prompts/roadmap/00-README.md@@ -75,6 +75,7 @@ retired; they remain readable in git history| [creation-consolidation.md](creation-consolidation.md) | The last three creation-surface merges | Open. Two redirects are blocked behind real capability gaps, and closing those gaps is the work. || [avatar-parametric-editor.md](avatar-parametric-editor.md) | Parametric avatar editor, phase 2 onward | Open. Phase 1 (122-slider parametric base) shipped 2026-07-22. || [developer-resources-repos.md](developer-resources-repos.md) | The public examples satellite repo and its one-way export | Open. npm packages are published, so the blocker that stalled it is gone. |+| [native-widgets.md](native-widgets.md) | Native widgets on the Android, Windows, macOS and iOS home screens | Open. The signed `ws.three.app` TWA and the server-side render endpoints already exist, so the shell and the image source are not the work. |### Strategy and reference (read, do not execute)diff --git a/prompts/roadmap/native-widgets.md b/prompts/roadmap/native-widgets.mdnew file mode 100644index 000000000..38b7d3a50--- /dev/null+++ b/prompts/roadmap/native-widgets.md@@ -0,0 +1,141 @@+# RM-WIDGETS: Native widgets, the agent on the home screen and the desktop++**How to run this:** paste this whole file into a fresh Claude Code chat opened in+`/workspaces/three.ws`, or say "execute `prompts/roadmap/native-widgets.md`".+It is complete on its own. Also read `prompts/roadmap/00-README.md` and `CLAUDE.md`.++## Binding operating clause++1. Finish 100%. Never end with a question, a plan you did not execute, or "should I proceed?".+2. Additive only. `ws.three.app` is a signed, published-track Android artifact and the PWA+ manifest is live for every installed user. Nothing here may change how the existing TWA+ boots, how the share target behaves, or how the service worker serves the offline shell.+3. CLAUDE.md hard rules: no mocks, no fake data, no TODO comments, no commented-out code, no+ em-dash or en-dash characters. Stage explicit paths only.+4. Every widget renders real data for a real signed-in account or it renders the signed-out+ card. There is no sample agent and no placeholder avatar.++## What this is, and what it is not++**Not this:** the embeddable web widgets at `/widgets` and `/studio` (`docs/widgets.md`,+`docs/widget-studio.md`). Those put a live WebGL agent inside someone else's web page.++**This:** operating system widget surfaces. The Android home screen, the Windows 11 widgets+board, and the macOS and iOS widget galleries. None of them can run WebGL, none of them can run+our viewer, and all of them refresh on a schedule the OS controls rather than on a page load.+So every one of them is fed the same thing: a small, cacheable, server-rendered card.++## What already exists (verified 2026-08-28, re-verify before acting)++- **The Android app is real and signed.** `ws.three.app`, a Trusted Web Activity built by+ `solana-mobile/scripts/build-apk.sh`, versionCode 1, targetSdk 36, 4.1 MB. Digital Asset+ Links verify against the live release key: `https://three.ws/.well-known/assetlinks.json`+ and Google's `statements:list` both return `98:0A:1A:AB:...:11:13:D7`. Deep links,+ long-press shortcuts (Create / Discover / My agents) and the Android share sheet target are+ registered and were exercised on a Pixel 7 emulator. Packaging and release: `solana-mobile/README.md`.+- **A real server-side renderer.** `POST /api/render/avatar-clip` returns a posed,+ camera-orbited PNG of any GLB (pose preset, orbit in degrees, background color, up to+ 2048 px) through the same headless chromium pipeline the OG cards use;+ `POST /api/render/glb` is the plain variant. `api/_lib/render-clip.js` is the core.+ There is no reason to build a second renderer for widgets.+- **Avatar thumbnails already have a storage discipline.** `docs/avatar-thumbnails.md`: a+ `thumbnail_key` is only persisted after the object behind it is confirmed to exist in R2.+ Widget cards obey the same rule, for the same reason.+- **The PWA is installable and its service worker is extensible.** `vite.config.js` builds the+ web manifest and a workbox `generateSW` service worker that already `importScripts`+ `/push-sw.js` and `/share-target-sw.js`. A widget lane is another script in that list, not a+ switch to `injectManifest`.++## Task 1: one widget card endpoint, two encodings++Build `api/widget/card.js` served at `GET /api/widget/card`. This is the single source every+platform reads.++- **Auth:** an opaque, revocable, per-account widget token, not the session cookie. A home+ screen widget outlives a session and a Windows widget is fetched by the OS, not the browser.+ Mint it at `POST /api/widget/token` for a signed-in account, store the hash, and let the+ account revoke it from the dashboard. Treat the token as a bearer credential in a query+ parameter: scope it to read-only widget data and nothing else.+- **Response, `?format=png`:** the card as an image, sized by `?size=small|medium|large`.+ Composed from the account's primary agent: the avatar render from `render-clip`, the agent+ name, and one live number. Cache it in R2 under the same confirm-before-persist rule as+ thumbnails, and serve `cache-control: public, max-age=900`.+- **Response, `?format=json`:** the same fields as data, for the Windows Adaptive Card and for+ any future surface. This is the contract; write it into `specs/` because native shells that+ ship through app stores cannot be redeployed in step with the server.+- **Signed out, and empty states:** an account with no agent gets a card that says how to make+ one, not a blank frame. A revoked token gets a card that says to re-link, never a 401 image+ the OS renders as a grey box.+- **The one live number** is the first real decision in this work order. It has to be worth a+ home screen slot and it has to be true: pick from the agent's recent activity, its earnings,+ or its reputation, take whichever the platform can compute cheaply and correctly for every+ account, and record the choice and the alternatives in your report.++## Task 2: Android home screen widget++The TWA project is regenerated from `twa/twa-manifest.json` by `bubblewrap update` on every+build, so anything native must be applied as an overlay AFTER that step and BEFORE+`bubblewrap build`. That ordering is the whole trick.++1. Add `solana-mobile/android-overlay/` holding the native sources: an `AppWidgetProvider`,+ its `RemoteViews` layouts in three sizes, the widget info XML, and the manifest fragment.+2. Teach `solana-mobile/scripts/build-apk.sh` to copy the overlay into the generated project+ and merge the provider declaration into `AndroidManifest.xml` between `update` and `build`.+ A missing overlay must fail the build loudly, never silently ship a widget-less APK that+ the listing claims has one.+3. Refresh with `WorkManager` on a battery-aware periodic schedule, not `AlarmManager`. Fetch+ `/api/widget/card?format=png`, write it to internal storage, and update the `RemoteViews`.+ No network means the last cached card stays on screen with its timestamp, never a spinner+ and never a broken image.+4. Tapping the widget opens the TWA at the deep link for that agent. Tapping a quick action+ opens `/create` or `/my-agents`. Both already resolve inside the app because Digital Asset+ Links verify.+5. Bump `appVersionCode` in `twa/twa-manifest.json`, rebuild, and verify on the Pixel 7+ emulator recipe in `solana-mobile/README.md`: add the widget from the picker, confirm it+ refreshes without opening the app, force stop the app and confirm the widget survives,+ reboot the emulator and confirm it comes back, then toggle airplane mode and confirm the+ cached card holds.+6. Update `solana-mobile/docs/CHECKLIST.md` and `docs/seeker-app.md`, and add the widget to+ `publish/listing/new-in-version.txt` only once it is verified on a device.++## Task 3: Windows 11 widget, through the PWA++Windows 11 renders widgets for installed PWAs from the `widgets` member of the web manifest,+driven by Adaptive Card templates and service worker events. No store submission and no native+shell, which is why it ships second and not fourth.++1. Add the `widgets` member to the manifest block in `vite.config.js`, next to `share_target`.+2. Add `public/widget-sw.js` and register it in the existing `workbox.importScripts` list.+ Handle `widgetinstall`, `widgetresume`, `widgetuninstall` and `widgetclick`: on install and+ resume, fetch `/api/widget/card?format=json` and call `widgets.updateByInstanceId`; on+ click, open the matching route.+3. Ship the Adaptive Card template and its data binding as real files under `public/widgets/`,+ version them, and keep the template's fields identical to the JSON contract from Task 1.+4. Verify on Windows 11 with the PWA installed from Edge: the widget appears in the widgets+ board picker, populates with the real account's data, refreshes, and its click targets land+ on the right routes.++## Task 4: Apple platforms, macOS first++macOS and iOS share one WidgetKit extension, and both need a host app, which is the reason this+is last. macOS first because it can be distributed outside the App Store while the iOS build+waits on review.++1. A minimal SwiftUI host app that signs in and stores the widget token in the keychain, plus a+ WidgetKit extension with a `TimelineProvider` that fetches `/api/widget/card?format=png`.+2. A menu bar companion on macOS, since that is the surface people actually keep.+3. The iOS build is the same extension target against the same host app, submitted once an+ Apple Developer account exists. State plainly in your report if that account is the blocker.++## Definition of done++- `GET /api/widget/card` returns a real card for a real account in both encodings, documented+ in `docs/api-reference.md`, with the JSON contract in `specs/`.+- The Android widget is in a signed APK, verified on the emulator against all five checks in+ Task 2 step 5.+- The Windows widget is live in the manifest and verified in the widgets board.+- `docs/native-widgets.md` exists and explains, for a reader with no context, what the widget+ shows, how to add it on each platform, and how to revoke its token.+- `STRUCTURE.md` has a row for the widget surface, `data/pages.json` has any new public route,+ and `data/changelog.json` has an entry per release.+- `npm run gate` and `npm run check:rules -- --paths <your files>` are green.