Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

feat(companion): answer a delivery without leaving it, and say what went wrong in words

nirholasdb5ac068cc0e
diff --git a/STRUCTURE.md b/STRUCTURE.md
index 7b87fdc5c..88465781b 100644
--- a/STRUCTURE.md
+++ b/STRUCTURE.md
@@ -58,7 +58,7 @@ bottom.
| Follow graph | [api/users/[username]/follow.js](api/users/%5Busername%5D/follow.js) + [api/users/[username]/follows.js](api/users/%5Busername%5D/follows.js), table `user_follows`, buttons on [pages/profile.html](pages/profile.html) | Live | `GET` returns `{following, followed_by, followers_count, following_count}`; `POST`/`DELETE` are idempotent (session + CSRF, self-follow 400) and return the same envelope so one round-trip updates the button and the counts. A genuinely new edge (`ON CONFLICT DO NOTHING`) publishes a `follow` user event to the notification bell exactly once. Feeds the activity feed's following scope, search ranking, and the `followers` leaderboard metric. Tests: [tests/api/users-follow.test.js](tests/api/users-follow.test.js). Docs: [docs/social-layer.md](docs/social-layer.md) |
| Herald: messages delivered in person | SDK [herald-sdk/](herald-sdk) `→ /herald.js` (published as `@three-ws/herald`), rail [api/herald/announce.js](api/herald/announce.js) + [api/herald/stream.js](api/herald/stream.js) + [api/_lib/herald.js](api/_lib/herald.js), playground [pages/herald.html](pages/herald.html) `→ /herald` + [src/herald-page.js](src/herald-page.js), CLI [herald-sdk/bin/herald.mjs](herald-sdk/bin/herald.mjs), MCP [packages/herald-mcp/](packages/herald-mcp) | Live | A 3D character walks onto the page and says what happened, instead of a badge nobody notices. The library is the interrupt budget: importance scoring with pluggable scorers, a floor, quiet hours, a rate limit, dedupe with a TTL, freshness, batch collapse, focus awareness, and a stable reason for every message it refuses (`duplicate`, `rate-limited`, `quiet-hours`, `below-importance-floor`). Delivery is the walk companion ([walk-sdk](walk-sdk) `control.announce`) with an accessible `aria-live` card fallback for anything that cannot render 3D, and opt-in gesture-gated voice. **The rail is the part nobody expects:** `POST /api/herald/announce` with an API key carrying the `herald:announce` scope reaches the browser tab the caller has open over SSE, so `herald watch -- npm test` makes your avatar tell you the build finished, and an AI agent can do the same through the MCP server. An announcement is always delivered to the caller's own sessions (no recipient field, so no key can interrupt anyone else); the queue is Redis-backed, drained not broadcast, and expires in five minutes. Docs: [docs/herald.md](docs/herald.md), [docs/api-reference.md](docs/api-reference.md). Tests: [tests/herald-rules.test.js](tests/herald-rules.test.js), [tests/herald-runtime.test.js](tests/herald-runtime.test.js), [tests/herald-rail.test.js](tests/herald-rail.test.js), [tests/herald-announce-endpoint.test.js](tests/herald-announce-endpoint.test.js) |
| Notification bell & preferences | Event vocabulary [api/_lib/feed.js](api/_lib/feed.js) (`USER_EVENT_TYPES` → `publishUserEvent()` → `user_notifications`), endpoints [api/notifications/](api/notifications), preference matrix [api/_lib/notify-prefs.js](api/_lib/notify-prefs.js), bell client [src/notifications.js](src/notifications.js), avatar announcements [src/notification-herald.js](src/notification-herald.js), settings panel [src/dashboard-next/pages/settings.js](src/dashboard-next/pages/settings.js) | Live | In-app inbox covering remixes ([api/x402/remix-asset.js](api/x402/remix-asset.js)), DMs ([api/friends/messages.js](api/friends/messages.js)), coin bonding-curve graduations (pump cron in [api/cron/[name].js](api/cron/%5Bname%5D.js)), follows, sales, purchases, IRL, market, and account events. Preferences are a category × channel matrix (`in_app` always on; push/email/telegram/avatar are per-category opt-outs) edited from `/dashboard/settings`. **The `avatar` channel delivers in person:** for a fresh, unread notification in an opted-in category the corner companion walks on screen, gestures, and says the line with a link straight to it, summoning itself for that one message and retreating if it was off ([walk-sdk](walk-sdk) `control.announce`). Rules (15-minute freshness, once per browser, two per batch, visible tab only, device-level mute) live in [src/notification-herald.js](src/notification-herald.js). Docs: [docs/notifications.md](docs/notifications.md), [docs/social-layer.md](docs/social-layer.md). Tests: [tests/notification-herald.test.js](tests/notification-herald.test.js), [tests/notify-prefs.test.js](tests/notify-prefs.test.js) |
-| Personal companion (`/companion`) | Page [pages/companion.html](pages/companion.html) `→ /companion` + [src/companion/](src/companion) (page, source cards, voice), API [api/companion/](api/companion) (settings, sources, contacts, events, `ingest`, `stream`, `poll`), engine [api/_lib/companion/](api/_lib/companion) (store, encrypted BYOK configs, triage, lanes for Telegram / iCal / IMAP), cron [api/cron/companion-poll.js](api/cron/companion-poll.js), migration [api/_lib/migrations/20260828140000_companion_harness.sql](api/_lib/migrations/20260828140000_companion_harness.sql) | Live | Turns a person's own inbox, chats, calendar and phone into deliveries a 3D character makes **in person**. BYOK per source (a Telegram bot the user made, a private iCal URL, IMAP with an app password) plus a universal bridge token anything can POST to. Every message is scored 0-100 by the shared deterministic rules ([@three-ws/companion/triage](packages/companion-sdk/src/triage-rules.js), the same module the CLI runs locally) and optionally refined by the user's own Anthropic key; above their threshold and outside quiet hours it is delivered through `companion_delivery` notifications, so the bell, Web Push, and the avatar channel all carry it. A contact's message is spoken by **that contact's avatar and voice** (`control.announce({ avatar })` in [walk-sdk](walk-sdk)). Live SSE at `/api/companion/stream` feeds the desktop app and any third-party body. Docs: [docs/companion.md](docs/companion.md). Tests: [tests/companion-triage.test.js](tests/companion-triage.test.js), [tests/companion-calendar.test.js](tests/companion-calendar.test.js), [packages/companion-sdk/tests/](packages/companion-sdk/tests) |
+| Personal companion (`/companion`) | Page [pages/companion.html](pages/companion.html) `→ /companion` + [src/companion/](src/companion) (page, source cards, voice), API [api/companion/](api/companion) (settings, sources, contacts, events, `ingest`, `stream`, `poll`), engine [api/_lib/companion/](api/_lib/companion) (store, encrypted BYOK configs, triage, lanes for Telegram / iCal / IMAP), cron [api/cron/companion-poll.js](api/cron/companion-poll.js), migration [api/_lib/migrations/20260828140000_companion_harness.sql](api/_lib/migrations/20260828140000_companion_harness.sql) | Live | Turns a person's own inbox, chats, calendar and phone into deliveries a 3D character makes **in person**. BYOK per source (a Telegram bot the user made, a private iCal URL, IMAP with an app password) plus a universal bridge token anything can POST to. Every message is scored 0-100 by the shared deterministic rules ([@three-ws/companion/triage](packages/companion-sdk/src/triage-rules.js), the same module the CLI runs locally) and optionally refined by the user's own Anthropic key; above their threshold and outside quiet hours it is delivered through `companion_delivery` notifications, so the bell, Web Push, and the avatar channel all carry it. A contact's message is spoken by **that contact's avatar and voice** (`control.announce({ avatar })` in [walk-sdk](walk-sdk)). A delivery whose lane can carry an answer (Telegram, through the user's own bot) offers a reply box in place, and the reply is stored with the message. Live SSE at `/api/companion/stream` feeds the desktop app and any third-party body. Docs: [docs/companion.md](docs/companion.md). Tests: [tests/companion-triage.test.js](tests/companion-triage.test.js), [tests/companion-calendar.test.js](tests/companion-calendar.test.js), [packages/companion-sdk/tests/](packages/companion-sdk/tests) |
| Knock: a priced door to a person (`/knock`) | Hub [pages/knock.html](pages/knock.html) `→ /knock` + [src/knock/page.js](src/knock/page.js), public door [pages/knock-door.html](pages/knock-door.html) `→ /knock/<handle>` + [src/knock/door.js](src/knock/door.js), API [api/knock/](api/knock) (door, directory, send, reply, settings, inbox), paid lane [api/x402/knock.js](api/x402/knock.js), engine [api/_lib/knock/](api/_lib/knock) (pure policy, store, delivery, receipt tokens), migration [api/_lib/migrations/20260828210000_knock_doors.sql](api/_lib/migrations/20260828210000_knock_doors.sql), SDK + CLI [packages/knock-sdk/](packages/knock-sdk) (`@three-ws/knock`), MCP [packages/knock-mcp/](packages/knock-mcp) (`@three-ws/knock-mcp`) | Live | Every account can publish a door at `/knock/<username>` and set what one message from a stranger costs. Paying it buys exactly one message through, and the delivery is **in person**: the knock becomes a `companion_events` row whose importance is derived from the amount paid, so the recipient's corner avatar walks on wherever they are and says who is at the door and what they paid. **The USDC settles straight to the recipient** (`payTo` on the 402 challenge is their own address, and a priced door cannot be opened without one), so there is no platform wallet in the path and no cut. Solana leads; Base is advertised only when the owner sets an address for it. **Every refusal happens before the 402** (shut door, daily cap, block list, length limits) and again before settlement, so a knock that was never going to land is never a knock somebody paid for. Only the subject line is spoken; an untrusted body never uses the recipient's avatar as a speaker. The sender gets a derived HMAC receipt URL and reads the reply through it with no account, which is why answering someone never hands out an address. Free doors take a plain POST on a tight per-IP bucket. Spec: [specs/KNOCK_PROTOCOL.md](specs/KNOCK_PROTOCOL.md). Docs: [docs/knock.md](docs/knock.md). Tests: [tests/knock-policy.test.js](tests/knock-policy.test.js), [packages/knock-sdk/test/](packages/knock-sdk/test), [packages/knock-mcp/test/](packages/knock-mcp/test) |
| Visitor feedback loop | Recorder [@three-ws/witness](packages/witness), capture + panel [src/feedback-companion.js](src/feedback-companion.js) (mounted by [src/walk-companion.js](src/walk-companion.js)), intake [api/feedback/report.js](api/feedback/report.js), queue API [api/feedback/index.js](api/feedback/index.js), scoring [api/_lib/feedback/triage.js](api/_lib/feedback/triage.js), store [api/_lib/feedback/store.js](api/_lib/feedback/store.js), cron [api/cron/feedback-triage.js](api/cron/feedback-triage.js), review page [pages/feedback.html](pages/feedback.html) `→ /feedback` + [src/feedback-page.js](src/feedback-page.js) | Live | The return channel on the corner companion: the visitor tells the avatar what broke and the report carries the route, build SHA, viewport, and the last console errors and failed requests the browser captured, so a vague sentence lands as an actionable row. The companion offers proactively at the one moment worth asking (right after the page actually threw), once per route per session, never in a background tab. **Feedback is data, never an instruction:** the model that reads untrusted visitor text writes six scalars to that report's own row and has no path to the repo, a deploy, or any other write; the admin-only `/feedback` queue groups reports into one row per problem and a person decides. **A report arrives as a runnable failing test:** the recorded session compiles into a Playwright spec asserting the failure is GONE (red until fixed, green after), reachable from the queue's Copy test button, `GET /api/feedback/repro`, `npm run feedback:repro`, and the `get_feedback_repro` MCP tool. Docs: [docs/feedback.md](docs/feedback.md), [docs/witness.md](docs/witness.md). Tests: [tests/feedback-triage.test.js](tests/feedback-triage.test.js) |
| Session-to-test recorder | [packages/witness/](packages/witness) `→ @three-ws/witness`: recorder [recorder.js](packages/witness/src/recorder.js), selector synthesis [selector.js](packages/witness/src/selector.js), capture-time privacy [redact.js](packages/witness/src/redact.js), spec compiler [compile.js](packages/witness/src/compile.js); compiled repros land in [tests/repros/](tests/repros) with [their own Playwright config](tests/repros/playwright.config.mjs) | Published as `@three-ws/witness` (Apache-2.0, zero deps) | Turns a real user session into a runnable failing test. Records INTENT (about 2 KB) rather than DOM mutations (megabytes), so it can be always-on, which is the point: the trace already exists when somebody decides to complain. Every element gets the strongest handle that is unique in the live document (test id > id > aria-label > role+name > text > name > placeholder > class > structural path), framework-generated ids and utility classes refused, and each trace carries a replay confidence equal to its WEAKEST link. The compiled spec asserts the failure is gone, never that it happened, so it locks in the fix rather than the bug; a session with no machine-visible failure emits `test.fail()` and says which assertion a human must add rather than faking green. Privacy is enforced at capture: typed values are never held (only a length and a shape), password/hidden/email/tel/date inputs and `data-witness="off"` subtrees record nothing, URLs lose their credentials before storage. Docs: [docs/witness.md](docs/witness.md). Tests: [packages/witness/test/](packages/witness/test) (36, `npm run test:witness`) |
diff --git a/api/_lib/companion/lanes/telegram.js b/api/_lib/companion/lanes/telegram.js
index ca1815d8a..fd44d19cd 100644
--- a/api/_lib/companion/lanes/telegram.js
+++ b/api/_lib/companion/lanes/telegram.js
@@ -67,6 +67,23 @@ function textOf(message) {
return '';
}
+/**
+ * Send an answer back into the conversation the message came from.
+ *
+ * @param {object} config the source's stored config (bot token)
+ * @param {object} replyTo the event's stored reply_to
+ * @param {string} text what the person typed
+ */
+export async function replyTelegram(config, replyTo, text) {
+ if (!replyTo?.chat_id) throw new Error('this message has nothing to reply to');
+ const sent = await callBot(config.bot_token, 'sendMessage', {
+ chat_id: replyTo.chat_id,
+ text,
+ ...(replyTo.message_id ? { reply_parameters: { message_id: replyTo.message_id, allow_sending_without_reply: true } } : {}),
+ });
+ return { message_id: sent.message_id, chat: replyTo.chat_title || null };
+}
+
/**
* @returns {{ items: Array, cursor: object }}
*/
@@ -98,6 +115,12 @@ export async function pollTelegram({ config, cursor = {} }) {
body,
url: message.chat?.username ? `https://t.me/${message.chat.username}` : null,
occurs_at: message.date ? new Date(message.date * 1000).toISOString() : null,
+ // Everything a reply needs to land in the same conversation, quoting
+ // the message it answers. Telegram is the one lane here that can
+ // carry an answer back without a second credential.
+ reply_to: message.chat?.id
+ ? { chat_id: message.chat.id, message_id: message.message_id, chat_title: message.chat.title || name }
+ : null,
});
}
diff --git a/api/_lib/companion/poll.js b/api/_lib/companion/poll.js
index e9750d091..dff5c022f 100644
--- a/api/_lib/companion/poll.js
+++ b/api/_lib/companion/poll.js
@@ -21,12 +21,15 @@ import {
dueSources,
} from './store.js';
import { decryptConfig } from './crypto.js';
-import { pollTelegram, verifyTelegram } from './lanes/telegram.js';
+import { pollTelegram, verifyTelegram, replyTelegram } from './lanes/telegram.js';
import { pollCalendar, verifyCalendar } from './lanes/calendar.js';
import { pollEmail, verifyEmail } from './lanes/email.js';
+// A lane can poll, verify a credential, and (where the protocol allows it)
+// carry an answer back. Only Telegram can reply today: an ICS feed is read only,
+// and answering mail would need SMTP credentials this feature never asks for.
const LANES = {
- telegram: { poll: pollTelegram, verify: verifyTelegram },
+ telegram: { poll: pollTelegram, verify: verifyTelegram, reply: replyTelegram },
calendar: { poll: pollCalendar, verify: verifyCalendar },
email: { poll: pollEmail, verify: verifyEmail },
};
@@ -77,6 +80,7 @@ export async function ingestItem(settings, raw, { anthropicKey = null } = {}) {
title: raw.title,
body: raw.body ?? null,
url: raw.url ?? null,
+ reply_to: raw.reply_to ?? null,
importance: verdict.importance,
reason: verdict.reason,
spoken_line: verdict.line,
diff --git a/api/_lib/companion/store.js b/api/_lib/companion/store.js
index dcf813e8e..162a3ceba 100644
--- a/api/_lib/companion/store.js
+++ b/api/_lib/companion/store.js
@@ -232,15 +232,17 @@ export async function insertEvent(userId, event) {
const [row] = await sql`
insert into companion_events
(user_id, source_id, source_kind, external_id, contact_id, sender, sender_id,
- title, body, url, importance, reason, spoken_line, triage_engine, occurs_at)
+ title, body, url, importance, reason, spoken_line, triage_engine, occurs_at, reply_to)
values (${userId}, ${event.source_id ?? null}, ${event.source_kind}, ${event.external_id},
${event.contact_id ?? null}, ${event.sender ?? null}, ${event.sender_id ?? null},
${event.title}, ${event.body ?? null}, ${event.url ?? null},
${event.importance}, ${event.reason ?? null}, ${event.spoken_line ?? null},
- ${event.triage_engine ?? 'rules'}, ${event.occurs_at ?? null})
+ ${event.triage_engine ?? 'rules'}, ${event.occurs_at ?? null},
+ ${event.reply_to ? JSON.stringify(event.reply_to) : null})
on conflict (user_id, source_kind, external_id) do nothing
returning id, source_kind, external_id, contact_id, sender, sender_id, title, body, url,
- importance, reason, spoken_line, triage_engine, occurs_at, delivered_at, created_at
+ importance, reason, spoken_line, triage_engine, occurs_at, delivered_at, created_at,
+ (reply_to is not null) as can_reply
`;
return row || null;
}
@@ -252,6 +254,7 @@ export async function listEvents(userId, { limit = 30, before = null, pendingOnl
select e.id, e.source_kind, e.sender, e.sender_id, e.title, e.body, e.url,
e.importance, e.reason, e.spoken_line, e.triage_engine, e.occurs_at,
e.delivered_at, e.dismissed_at, e.created_at,
+ (e.reply_to is not null) as can_reply, e.replied_at, e.reply_text,
c.display_name as contact_name, c.avatar_glb_url as contact_avatar_glb_url,
c.avatar_image_url as contact_avatar_image_url, c.voice as contact_voice
from companion_events e
@@ -274,3 +277,30 @@ export async function markEvent(userId, id, { delivered = false, dismissed = fal
`;
return row || null;
}
+
+// One event with everything a reply needs: the lane, the source's decrypted
+// credentials, and the routing the lane stored when the message arrived.
+export async function getReplyTarget(userId, eventId) {
+ const [row] = await sql`
+ select e.id, e.source_kind, e.reply_to, e.sender, e.replied_at,
+ s.id as source_id, s.kind as source_kind_actual, s.config_encrypted, s.enabled
+ from companion_events e
+ left join companion_sources s on s.id = e.source_id
+ where e.id = ${eventId} and e.user_id = ${userId}
+ `;
+ if (!row) return null;
+ return {
+ ...row,
+ config: row.config_encrypted ? await decryptConfig(row.config_encrypted) : null,
+ };
+}
+
+export async function recordReply(userId, eventId, text) {
+ const [row] = await sql`
+ update companion_events
+ set replied_at = now(), reply_text = ${text}
+ where id = ${eventId} and user_id = ${userId}
+ returning id, replied_at, reply_text
+ `;
+ return row || null;
+}
diff --git a/api/_lib/migrations/20260828233000_companion_replies.sql b/api/_lib/migrations/20260828233000_companion_replies.sql
new file mode 100644
index 000000000..3d5c16756
--- /dev/null
+++ b/api/_lib/migrations/20260828233000_companion_replies.sql
@@ -0,0 +1,21 @@
+begin;
+
+-- Replying from the delivery.
+--
+-- The companion brings you a message; the natural next thing a person does is
+-- answer it, and making them go find the app to do that is where the illusion
+-- of a personal assistant breaks. `reply_to` carries whatever the lane needs to
+-- route an answer back (for Telegram: the chat id and the message id it is a
+-- reply to), stored per event because a source can span many chats.
+--
+-- Nullable on purpose: a calendar reminder and a phone-bridge notification have
+-- nothing to reply to, and the UI reads exactly this column to decide whether to
+-- offer the box at all.
+alter table companion_events add column if not exists reply_to jsonb;
+
+-- What the user actually sent back, so a delivery reads as a conversation
+-- rather than a receipt: the feed shows the reply under the message.
+alter table companion_events add column if not exists replied_at timestamptz;
+alter table companion_events add column if not exists reply_text text;
+
+commit;
diff --git a/api/companion/events/[id]/reply.js b/api/companion/events/[id]/reply.js
new file mode 100644
index 000000000..cd111d5ad
--- /dev/null
+++ b/api/companion/events/[id]/reply.js
@@ -0,0 +1,68 @@
+// POST /api/companion/events/:id/reply { text }
+//
+// The other half of a delivery. The companion brings you a message; answering it
+// should not mean going to find the app. This routes an answer back through the
+// same connection the message arrived on, quoting the original so it lands as a
+// reply rather than a stray line in a chat.
+//
+// Only lanes whose protocol can carry an answer offer this (today: Telegram,
+// through the user's own bot). A calendar feed is read only, and answering mail
+// would need SMTP credentials this feature deliberately never asks for, so both
+// are refused with a message that says which lane could not do it.
+
+import { z } from 'zod';
+import { getRequestUser } from '../../../_lib/auth.js';
+import { cors, json, method, wrap, error, readJson, rateLimited } from '../../../_lib/http.js';
+import { requireCsrf } from '../../../_lib/csrf.js';
+import { limits } from '../../../_lib/rate-limit.js';
+import { parse } from '../../../_lib/validate.js';
+import { getReplyTarget, recordReply } from '../../../_lib/companion/store.js';
+import { laneFor } from '../../../_lib/companion/poll.js';
+
+const replyBody = z.object({
+ text: z.string().min(1).max(4000),
+});
+
+export default wrap(async (req, res) => {
+ if (cors(req, res, { methods: 'POST,OPTIONS', credentials: true })) return;
+ if (!method(req, res, ['POST'])) return;
+
+ const user = await getRequestUser(req, res);
+ if (!user) return error(res, 401, 'unauthorized', 'sign in required');
+ if (!(await requireCsrf(req, res, user.id))) return;
+
+ // A reply opens a connection to the user's own provider, so it shares the
+ // bucket with "check now" rather than the cheap read bucket.
+ const rl = await limits.companionPoll(user.id);
+ if (!rl.success) return rateLimited(res, rl);
+
+ const id = String(req.query?.id || '').trim();
+ if (!/^[0-9a-f-]{36}$/i.test(id)) return error(res, 400, 'validation_error', 'invalid event id');
+
+ const { text } = parse(replyBody, await readJson(req));
+
+ const target = await getReplyTarget(user.id, id);
+ if (!target) return error(res, 404, 'not_found', 'no such message');
+ if (!target.reply_to) {
+ return error(res, 400, 'not_repliable', `a ${target.source_kind} message has nothing to reply to`);
+ }
+ if (!target.config) {
+ return error(res, 409, 'source_disconnected', 'the connection this message came from has been disconnected');
+ }
+
+ const lane = laneFor(target.source_kind_actual || target.source_kind);
+ if (!lane?.reply) {
+ return error(res, 400, 'not_repliable', `the ${target.source_kind} lane cannot send a reply`);
+ }
+
+ try {
+ const sent = await lane.reply(target.config, target.reply_to, text);
+ const row = await recordReply(user.id, id, text);
+ return json(res, 200, { sent: true, to: sent.chat || target.sender || null, event: row });
+ } catch (err) {
+ // The provider's own words are the useful ones here ("bot was blocked by
+ // the user", "chat not found"), so they are passed through rather than
+ // flattened into a generic failure.
+ return error(res, 502, 'reply_failed', String(err?.message || err).slice(0, 300));
+ }
+});
diff --git a/api/companion/stream.js b/api/companion/stream.js
index c693f7e72..47a4e66a2 100644
--- a/api/companion/stream.js
+++ b/api/companion/stream.js
@@ -115,6 +115,7 @@ export default async function handleCompanionStream(req, res) {
const rows = await sql`
select e.id, e.source_kind, e.sender, e.sender_id, e.title, e.body, e.url,
e.importance, e.reason, e.spoken_line, e.occurs_at, e.created_at,
+ (e.reply_to is not null) as can_reply,
c.display_name as contact_name, c.avatar_glb_url as contact_avatar_glb_url,
c.avatar_image_url as contact_avatar_image_url, c.voice as contact_voice
from companion_events e
diff --git a/data/changelog.json b/data/changelog.json
index ea5956ce1..99747765f 100644
--- a/data/changelog.json
+++ b/data/changelog.json
@@ -1,6 +1,16 @@
{
"$comment": "Curated changelog entries for three.ws, the editorial layer of the public changelog. New PAGE launches are tracked automatically via the `added` field in data/pages.json; this file is for everything else holders care about: improvements, fixes, SDK releases, trading/payments capabilities, security work. scripts/build-page-index.mjs merges both sources into CHANGELOG.md, public/changelog.json, and public/changelog.xml. Newest first. Allowed tags: feature, improvement, fix, sdk, infra, docs, security. `link` is optional and must be a live three.ws path.",
"entries": [
+ {
+ "date": "2026-08-28",
+ "title": "Embedded avatars load again on other people's websites",
+ "summary": "Anyone who dropped a three.ws avatar onto their own site with the <agent-3d> tag and pointed it straight at a model file got an error panel instead of the character: the embed code referred to something that was not there and gave up at the last step, right before it would have started the idle animation. It now loads and plays as documented. Two related repairs came with it: the compression decoders three.ws hosts can finally be used by other sites (they were being blocked, so every embed quietly fell back to a third-party CDN and logged errors doing it), and the code that runs on other people's pages is now checked for this exact class of mistake before it can ship again.",
+ "tags": [
+ "fix",
+ "sdk"
+ ],
+ "link": "/docs/web-component"
+ },
{
"date": "2026-08-28",
"title": "Portal: paste any web address and walk it in 3D",
@@ -128,7 +138,7 @@
{
"date": "2026-08-28",
"title": "Your notifications now have a body: meet your companion",
- "summary": "Your phone buzzes forty times a day and two of those mattered. three.ws/companion is the other half of that problem. Connect what you already use, on your own keys: a Telegram bot you create, the private calendar link Google, Apple and Outlook all publish, your inbox over IMAP with an app password, or nothing at all but a token your phone, your Mac, your build script or an AI agent can post to. Everything that arrives is scored out of 100 against the things people actually want interrupting for (a saved contact asking you something, a one-time code, a meeting starting in ten minutes, a payment that failed) and the rest is kept quietly in a feed with the reason it stayed quiet. What clears your bar is delivered in person: a 3D character walks on, and if it is from someone you gave a face to, it arrives wearing their avatar and speaking in their voice. It reaches you on three.ws, as a push on your phone, and on your desktop, where a new companion app puts a character that walks across your screen and comes over to tell you. Quiet hours are absolute, the credentials are encrypted, and one button revokes every device. Developers get the whole thing as @three-ws/companion: a client, the same triage rules the server runs (so you can score your mail locally and never send the rest), a drop-in 3D stage for any page, a CLI, and an MCP server so an agent that finds something urgent can stand in front of you and say it.",
+ "summary": "Your phone buzzes forty times a day and two of those mattered. three.ws/companion is the other half of that problem. Connect what you already use, on your own keys: a Telegram bot you create, the private calendar link Google, Apple and Outlook all publish, your inbox over IMAP with an app password, or nothing at all but a token your phone, your Mac, your build script or an AI agent can post to. Everything that arrives is scored out of 100 against the things people actually want interrupting for (a saved contact asking you something, a one-time code, a meeting starting in ten minutes, a payment that failed) and the rest is kept quietly in a feed with the reason it stayed quiet. What clears your bar is delivered in person: a 3D character walks on, and if it is from someone you gave a face to, it arrives wearing their avatar and speaking in their voice. It reaches you on three.ws, as a push on your phone, and on your desktop, where a new companion app puts a character that walks across your screen and comes over to tell you. You can answer straight from the message: a Telegram delivery carries a reply box that sends back into the same chat, so it is a conversation and not a receipt. Quiet hours are absolute, the credentials are encrypted, and one button revokes every device. Developers get the whole thing as @three-ws/companion: a client, the same triage rules the server runs (so you can score your mail locally and never send the rest), a drop-in 3D stage for any page, a CLI, and an MCP server so an agent that finds something urgent can stand in front of you and say it.",
"tags": [
"feature",
"sdk",
diff --git a/docs/companion.md b/docs/companion.md
index 5c6c72599..3ef9b63c0 100644
--- a/docs/companion.md
+++ b/docs/companion.md
@@ -164,6 +164,23 @@ Above your threshold and outside quiet hours, it is **spoken**. Below it, it is
**kept**, with the score and the reason, in the feed on
[/companion](https://three.ws/companion).
+### Answering it
+
+The companion brings you a message; going to find the app to answer it is where
+the illusion breaks. Any delivery whose lane can carry an answer gets a **Reply**
+box in place, under the message, in the feed:
+
+- **Telegram** replies through your own bot, into the same chat, quoting the
+ message it answers. Enter sends, Shift+Enter is a newline.
+- A calendar reminder and a phone notification have nothing to reply to, so they
+ do not offer the box at all rather than offering one that fails.
+- Answering mail would need SMTP credentials this feature deliberately never
+ asks for, so email is read only.
+
+Programmatically: `POST /api/companion/events/:id/reply { text }`, or
+`companion.reply(id, text)` in the SDK. The reply is stored with the message, so
+the feed reads as a conversation rather than a receipt.
+
---
## Who gets a face
@@ -278,6 +295,7 @@ Everything the page does, you can do. Session cookie or bridge token.
| `/api/companion/stream` | GET | Live deliveries as Server-Sent Events. |
| `/api/companion/events` | GET | The feed, with scores and reasons. |
| `/api/companion/events/:id` | PATCH | Mark delivered or dismissed. |
+| `/api/companion/events/:id/reply` | POST | Answer through the connection it arrived on. |
| `/api/companion/sources` | GET, POST | List or connect a source (verified before saving). |
| `/api/companion/sources/:id` | PATCH, DELETE, POST | Rename, pause, disconnect, or check now. |
| `/api/companion/contacts` | GET, POST | The people who get a face. |
@@ -297,6 +315,7 @@ A delivery on the stream carries everything a body needs to perform it:
"avatar_glb_url": "https://three.ws/api/avatars/…/glb",
"voice": "nova",
"source_kind": "telegram",
+ "can_reply": true,
"created_at": "2026-08-28T12:04:11.221Z"
}
```
diff --git a/packages/companion-sdk/README.md b/packages/companion-sdk/README.md
index 5e6aaba51..073c13dad 100644
--- a/packages/companion-sdk/README.md
+++ b/packages/companion-sdk/README.md
@@ -107,7 +107,20 @@ The stream reconnects on its own and resumes from the last delivery it saw, so a
laptop that slept wakes up and catches what it missed (bounded server side to
the last few hours, because a monologue is not a delivery).
-## 3. Put a body on a page
+## 3. Answer it
+
+A delivery whose lane can carry an answer comes back with `can_reply: true`:
+
+```js
+await companion.reply(delivery.id, 'on my way down');
+```
+
+It goes back through the same connection the message arrived on, into the same
+conversation, quoting the message it answers. Today that is Telegram, through
+the user's own bot; a calendar reminder and a phone notification have nothing to
+reply to and are refused with `not_repliable` rather than failing silently.
+
+## 4. Put a body on a page
```js
import { createCompanionClient, createCompanionStage } from '@three-ws/companion';
@@ -135,7 +148,7 @@ stage.deliver({
Inside a page that is already signed in to three.ws, omit the token entirely and
the browser's session cookie is used instead.
-## 4. Judge a message locally, with no key
+## 5. Judge a message locally, with no key
The triage rules are the same code the server runs. Import them when you want
the judgement without sending anything anywhere:
@@ -162,7 +175,7 @@ is, and whatever priority the sending device already assigned.
This is what makes a **local privacy mode** possible: score on your own machine,
and send only the one line that earned an interruption.
-## 5. The command line
+## 6. The command line
```bash
npx @three-ws/companion login --token cmp_…
@@ -227,7 +240,7 @@ Needs `@modelcontextprotocol/sdk` installed alongside this package.
| `fetch` | platform | Override, for tests or a proxy. |
| `retryMs` | `3000` | Stream reconnect delay. |
-Returns `{ send, list, markDelivered, dismiss, contacts, checkNow, stream }`.
+Returns `{ send, list, markDelivered, dismiss, reply, contacts, checkNow, stream }`.
Every method rejects with a `CompanionError` carrying `.status` and `.code`.
### `createCompanionStage(options)`
diff --git a/packages/companion-sdk/src/client.js b/packages/companion-sdk/src/client.js
index c36f2111b..967012955 100644
--- a/packages/companion-sdk/src/client.js
+++ b/packages/companion-sdk/src/client.js
@@ -79,9 +79,12 @@ export function createCompanionClient({
}
}
if (!res.ok) {
- throw new CompanionError(data?.message || data?.error || `request failed (${res.status})`, {
+ // three.ws answers errors as { error: <machine code>,
+ // error_description: <sentence> }. The sentence is what a person or a
+ // log reader needs; the code is what a program branches on.
+ throw new CompanionError(data?.error_description || data?.message || `request failed (${res.status})`, {
status: res.status,
- code: data?.code || null,
+ code: data?.error || data?.code || null,
});
}
return data;
@@ -117,6 +120,19 @@ export function createCompanionClient({
});
},
+ /**
+ * Answer a delivery through the connection it arrived on. Only messages
+ * that carry `can_reply` can take one (today: Telegram, through the
+ * user's own bot); anything else is refused with `not_repliable`.
+ */
+ reply(id, text) {
+ if (!String(text || '').trim()) throw new CompanionError('a reply needs some text');
+ return call(`/api/companion/events/${encodeURIComponent(id)}/reply`, {
+ method: 'POST',
+ body: { text },
+ });
+ },
+
dismiss(id) {
return call(`/api/companion/events/${encodeURIComponent(id)}`, {
method: 'PATCH',
diff --git a/packages/companion-sdk/tests/client.test.js b/packages/companion-sdk/tests/client.test.js
index 1a9ad7aaa..fec234f99 100644
--- a/packages/companion-sdk/tests/client.test.js
+++ b/packages/companion-sdk/tests/client.test.js
@@ -44,9 +44,10 @@ describe('createCompanionClient', () => {
expect(() => client.send({ body: 'no title' })).toThrow(CompanionError);
});
- it('surfaces the server error message and status', async () => {
+ it('surfaces the human sentence, not the machine code, and keeps the status', async () => {
const fetchImpl = vi.fn().mockResolvedValue(
- jsonResponse({ code: 'unauthorized', message: 'unknown bridge token' }, { status: 401 }),
+ // The real envelope three.ws sends (api/_lib/http.js error()).
+ jsonResponse({ error: 'unauthorized', error_description: 'unknown bridge token' }, { status: 401 }),
);
const client = createCompanionClient({ token: 'bad', fetch: fetchImpl });
await expect(client.send({ title: 'hi' })).rejects.toMatchObject({
diff --git a/pages/companion.html b/pages/companion.html
index a3a6fa2dd..e219d9fd1 100644
--- a/pages/companion.html
+++ b/pages/companion.html
@@ -134,6 +134,9 @@
.event-line{font-size:14px;line-height:1.55;}
.event-sub{font-size:12px;color:var(--muted);margin-top:5px;display:flex;gap:10px;align-items:center;flex-wrap:wrap;}
.event-actions{display:flex;gap:6px;flex-shrink:0;align-items:flex-start;}
+ .event-reply{margin-top:8px;font-size:13px;color:var(--muted);border-left:2px solid var(--blue);padding-left:10px;line-height:1.5;}
+ .reply-box{margin-top:12px;display:flex;flex-direction:column;gap:8px;}
+ .reply-box textarea{resize:vertical;min-height:52px;}
/* States */
.empty{text-align:center;padding:56px 20px;color:var(--muted);}
.empty h3{font-size:17px;font-weight:600;color:var(--text);margin-bottom:8px;}
diff --git a/scripts/companion-e2e.mjs b/scripts/companion-e2e.mjs
index 72e0c01fb..07ab44548 100644
--- a/scripts/companion-e2e.mjs
+++ b/scripts/companion-e2e.mjs
@@ -157,7 +157,24 @@ try {
});
check('an unauthenticated source connect is refused', badSource.status === 401);
- // 9. The SSE stream authenticates with the bridge token and replays.
+ // 9. Replying. The endpoint needs a session (the E2E has none, so it is
+ // checked for the refusal), and the routing that decides whether a reply is
+ // even possible is checked directly against the stored row.
+ const bridgeEvent = feed.find((e) => e.source_kind === 'bridge');
+ const replyUnauthed = await call('../api/companion/events/[id]/reply.js', {
+ method: 'POST',
+ url: `/api/companion/events/${bridgeEvent.id}/reply`,
+ query: { id: bridgeEvent.id },
+ body: { text: 'this should not send' },
+ });
+ check('replying needs a session', replyUnauthed.status === 401);
+
+ const { getReplyTarget } = await import('../api/_lib/companion/store.js');
+ const target = await getReplyTarget(user.id, bridgeEvent.id);
+ check('a phone notification has nothing to reply to', target !== null && target.reply_to === null);
+ check('the feed reports which messages can be replied to', feed.every((e) => e.can_reply === false));
+
+ // 10. The SSE stream authenticates with the bridge token and replays.
const streamMod = await import('../api/companion/stream.js');
const res = mockRes();
const req = mockReq({
diff --git a/src/companion/api.js b/src/companion/api.js
index 5d70b469e..fcb07cdc0 100644
--- a/src/companion/api.js
+++ b/src/companion/api.js
@@ -25,9 +25,13 @@ async function request(path, { method = 'GET', body = null, allowAnonymous = fal
}
}
if (!res.ok) {
- const err = new Error(data?.message || data?.error || `request failed (${res.status})`);
+ // The platform's error envelope is { error: <machine code>,
+ // error_description: <sentence for a human> } (api/_lib/http.js). Reading
+ // `error` as the message is how a user ends up looking at
+ // "source_disconnected" instead of being told what to do about it.
+ const err = new Error(data?.error_description || data?.message || `request failed (${res.status})`);
err.status = res.status;
- err.code = data?.code || null;
+ err.code = data?.error || data?.code || null;
throw err;
}
return data;
@@ -55,6 +59,7 @@ export const companionApi = {
if (loudOnly) params.set('min_importance', String(threshold));
return request(`/api/companion/events?${params.toString()}`);
},
+ reply: (id, text) => request(`/api/companion/events/${encodeURIComponent(id)}/reply`, { method: 'POST', body: { text } }),
markEvent: (id, patch) => request(`/api/companion/events/${encodeURIComponent(id)}`, { method: 'PATCH', body: patch }),
avatars: () => request('/api/avatars?limit=100'),
diff --git a/src/companion/page.js b/src/companion/page.js
index b147ace57..f0069341c 100644
--- a/src/companion/page.js
+++ b/src/companion/page.js
@@ -198,6 +198,7 @@ function renderFeed() {
<div class="event-icon">${SOURCE_ICON[event.source_kind] || '📨'}</div>
<div class="event-main">
<div class="event-line">${esc(event.spoken_line || event.title)}</div>
+ ${event.replied_at ? `<div class="event-reply">You replied: ${esc(event.reply_text || '')}</div>` : ''}
<div class="event-sub">
<span>${esc(event.contact_name || event.sender || 'Unknown')}</span>
<span class="meter" title="${esc(event.reason || '')}"><span class="${importanceClass(event.importance)}" style="width:${Math.max(3, event.importance)}%"></span></span>
@@ -207,6 +208,7 @@ function renderFeed() {
</div>
</div>
<div class="event-actions">
+ ${event.can_reply ? `<button type="button" class="btn btn-sm" data-feed-action="reply" data-id="${esc(event.id)}">Reply</button>` : ''}
<button type="button" class="btn btn-sm" data-feed-action="stage" data-id="${esc(event.id)}">Replay</button>
${event.dismissed_at ? '' : `<button type="button" class="btn btn-sm" data-feed-action="dismiss" data-id="${esc(event.id)}">Dismiss</button>`}
</div>
@@ -215,6 +217,60 @@ function renderFeed() {
el('feed-more').hidden = state.events.length < FEED_PAGE;
}
+// Replying to a delivery. Only messages whose lane can carry an answer offer
+// this, and the composer opens in place under the message rather than in a
+// modal: an answer to "are you coming?" is one line, and a dialog for one line
+// is the reason people go to the app instead.
+function openReplyBox(event, row) {
+ const existing = row.querySelector('.reply-box');
+ if (existing) {
+ existing.querySelector('textarea').focus();
+ return;
+ }
+ const box = document.createElement('form');
+ box.className = 'reply-box';
+ box.innerHTML = `
+ <textarea rows="2" maxlength="4000" placeholder="Reply to ${esc(event.contact_name || event.sender || 'them')}…" aria-label="Your reply"></textarea>
+ <div class="hero-actions">
+ <button type="submit" class="btn btn-sm btn-primary">Send</button>
+ <button type="button" class="btn btn-sm" data-cancel>Cancel</button>
+ <span class="hint" data-status></span>
+ </div>
+ `;
+ row.querySelector('.event-main').appendChild(box);
+ const textarea = box.querySelector('textarea');
+ const status = box.querySelector('[data-status]');
+ textarea.focus();
+
+ box.querySelector('[data-cancel]').addEventListener('click', () => box.remove());
+ // Enter sends, Shift+Enter is a newline: the shape every messaging app uses.
+ textarea.addEventListener('keydown', (e) => {
+ if (e.key === 'Enter' && !e.shiftKey) {
+ e.preventDefault();
+ box.requestSubmit();
+ }
+ });
+
+ box.addEventListener('submit', async (e) => {
+ e.preventDefault();
+ const text = textarea.value.trim();
+ if (!text) return;
+ const send = box.querySelector('button[type=submit]');
+ send.disabled = true;
+ status.textContent = 'Sending…';
+ try {
+ const result = await companionApi.reply(event.id, text);
+ event.replied_at = new Date().toISOString();
+ event.reply_text = text;
+ toast(`Sent${result.to ? ` to ${result.to}` : ''}.`, { variant: 'success' });
+ renderFeed();
+ } catch (err) {
+ send.disabled = false;
+ status.textContent = err.message;
+ }
+ });
+}
+
async function loadFeed({ append = false } = {}) {
const data = await companionApi.events({
limit: FEED_PAGE,
@@ -571,6 +627,9 @@ function wireEvents() {
if (!button) return;
const event = state.events.find((row) => row.id === button.dataset.id);
if (!event) return;
+ if (button.dataset.feedAction === 'reply') {
+ return openReplyBox(event, button.closest('.event'));
+ }
if (button.dataset.feedAction === 'stage') {
state.stageEvent = event;
renderStage();
diff --git a/tests/companion-telegram.test.js b/tests/companion-telegram.test.js
new file mode 100644
index 000000000..7ce87e51c
--- /dev/null
+++ b/tests/companion-telegram.test.js
@@ -0,0 +1,116 @@
+// The Telegram lane (api/_lib/companion/lanes/telegram.js): what it reads out of
+// an update, how it advances its cursor, and how a reply is routed back.
+//
+// The Bot API is stubbed at the fetch boundary, so these are real assertions
+// about the request we send and the shape we hand the triage pass, without
+// talking to Telegram.
+
+import { describe, it, expect, vi, afterEach } from 'vitest';
+import { pollTelegram, replyTelegram, verifyTelegram } from '../api/_lib/companion/lanes/telegram.js';
+
+function stubBot(handler) {
+ const calls = [];
+ vi.stubGlobal('fetch', async (url, init) => {
+ const method = String(url).split('/').pop();
+ const body = init?.body ? JSON.parse(init.body) : {};
+ calls.push({ method, body });
+ const result = handler(method, body);
+ return {
+ ok: result.ok !== false,
+ status: result.status || 200,
+ json: async () => (result.ok === false
+ ? { ok: false, description: result.description }
+ : { ok: true, result: result.result }),
+ };
+ });
+ return calls;
+}
+
+afterEach(() => vi.unstubAllGlobals());
+
+const message = (over = {}) => ({
+ message_id: 501,
+ date: 1787040000,
+ from: { id: 42, first_name: 'Sarah', last_name: 'K', username: 'sarah_k' },
+ chat: { id: -1001, title: 'Home' },
+ text: 'I am downstairs and cannot find your door',
+ ...over,
+});
+
+describe('pollTelegram', () => {
+ it('turns an update into a lane item with every identity the matcher can use', async () => {
+ stubBot(() => ({ result: [{ update_id: 900, message: message() }] }));
+ const { items, cursor } = await pollTelegram({ config: { bot_token: 't' }, cursor: {} });
+
+ expect(items).toHaveLength(1);
+ const [item] = items;
+ expect(item.external_id).toBe('tg:900');
+ expect(item.sender).toBe('Sarah K');
+ expect(item.sender_id).toBe('@sarah_k');
+ expect(item.identity_candidates).toEqual(['@sarah_k', '42', 'Home']);
+ expect(item.title).toBe('I am downstairs and cannot find your door');
+ expect(item.occurs_at).toBe(new Date(1787040000 * 1000).toISOString());
+ // Everything a reply needs, captured while the message is in hand.
+ expect(item.reply_to).toEqual({ chat_id: -1001, message_id: 501, chat_title: 'Home' });
+ // The cursor acknowledges the batch: the next poll starts past it.
+ expect(cursor.offset).toBe(901);
+ });
+
+ it('describes a message that carries no text so it is still worth hearing', async () => {
+ stubBot(() => ({
+ result: [
+ { update_id: 1, message: message({ text: undefined, voice: { duration: 3 } }) },
+ { update_id: 2, message: message({ text: undefined, document: { file_name: 'lease.pdf' } }) },
+ { update_id: 3, message: message({ text: undefined, location: { latitude: 1, longitude: 2 } }) },
+ ],
+ }));
+ const { items } = await pollTelegram({ config: { bot_token: 't' }, cursor: {} });
+ expect(items.map((i) => i.body)).toEqual([
+ 'sent a voice message',
+ 'sent a file: lease.pdf',
+ 'shared a location',
+ ]);
+ });
+
+ it('sends the stored offset and holds it when a poll comes back empty', async () => {
+ const calls = stubBot(() => ({ result: [] }));
+ const { items, cursor } = await pollTelegram({ config: { bot_token: 't' }, cursor: { offset: 77 } });
+ expect(items).toEqual([]);
+ expect(calls[0].body.offset).toBe(77);
+ expect(cursor.offset).toBe(77);
+ });
+
+ it('surfaces the provider\'s own words when the token is wrong', async () => {
+ stubBot(() => ({ ok: false, status: 401, description: 'Unauthorized' }));
+ await expect(verifyTelegram({ bot_token: 'bad' })).rejects.toThrow(/telegram: Unauthorized/);
+ });
+});
+
+describe('replyTelegram', () => {
+ it('answers in the same chat, quoting the message it replies to', async () => {
+ const calls = stubBot(() => ({ result: { message_id: 777 } }));
+ const sent = await replyTelegram(
+ { bot_token: 't' },
+ { chat_id: -1001, message_id: 501, chat_title: 'Home' },
+ 'on my way down',
+ );
+ expect(calls[0].method).toBe('sendMessage');
+ expect(calls[0].body).toEqual({
+ chat_id: -1001,
+ text: 'on my way down',
+ reply_parameters: { message_id: 501, allow_sending_without_reply: true },
+ });
+ expect(sent).toEqual({ message_id: 777, chat: 'Home' });
+ });
+
+ it('still sends when the original message id is gone', async () => {
+ const calls = stubBot(() => ({ result: { message_id: 778 } }));
+ await replyTelegram({ bot_token: 't' }, { chat_id: 5 }, 'hello');
+ expect(calls[0].body.reply_parameters).toBeUndefined();
+ expect(calls[0].body.chat_id).toBe(5);
+ });
+
+ it('refuses a message that carries no route back', async () => {
+ await expect(replyTelegram({ bot_token: 't' }, null, 'hi')).rejects.toThrow(/nothing to reply to/);
+ });
+});