An agentic browser harness: an AI side panel & orb in your own logged-in Chrome, Edge, Arc, Brave or even Firefox run by the agent/subscription you already use (Claude Code, Codex, Antigravity). Free, open source, no API key. Also an MCP server.
TypeScript agentic-browseragentic-browsingagy-cliaiai-agentai-browser-extensionai-toolsbrave-extensionbrowser-agentbrowser-automationbrowser-extensionbrowser-usebrowserskillchrome-extensionclaude-codecodexfirefox-addonmcp-servernanobrowserweb-automation
docs: record the session store, reply rendering, and cancellation rules
Shahriarc33bdebdf18a
diff --git a/CLAUDE.md b/CLAUDE.mdindex 8317373..93e588d 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -85,7 +85,7 @@ Call path: side panel / bridge → `invokeForHarness` → `browser.tabs.sendMess- **Auth is two independent gates.** At the HTTP upgrade the daemon classifies the peer by handshake `Origin` (extension scheme → extension path; `http(s)` → 403; absent → control client, lockfile token required). That only proves the peer is *an* extension; the `hello` frame must then carry a one-time `pairingToken` or a session key bound to that same origin. Credentials ride in the frame because the browser `WebSocket` API cannot set request headers, and a URL query would leak into logs.- **The extension is off by default.** `connectDaemon()` returns without dialing unless `storage.local` holds a session key, so an unpaired browser never contacts the daemon. Pairing state lives in `~/.voicelink/auth.json` (`0600`) — deliberately *not* the lockfile, which is deleted on shutdown.- Verified: Chrome does send `Origin: chrome-extension://<id>` from an MV3 worker.-- **The wire protocol is v5** (`SOCKET_PROTOCOL_VERSION`): on top of the v1 request/reply frames, the extension may send `instruct`/`cancel`/`decision`/`reset`/`analyzeFile`/`saveSkill`/`deleteSkill`/`deleteSiteMap`/`activateSiteMap`/`discardSiteMap` unprompted, and the daemon streams `run` and `siteMapDraft` frames back. Any new extension-initiated frame must also be listed in `EXTENSION_REQUEST_FRAMES` — `isExtensionRequest()` keys on it, and a frame missing from that list is silently dropped by `ExtensionLink.receive` rather than erroring. `assertToolNamesRoundTrip` aside, protocol bumps require rebuilding both sides *and* reloading the extension — an older worker is refused at `hello`.+- **The wire protocol is v6** (`SOCKET_PROTOCOL_VERSION`): on top of the v1 request/reply frames, the extension may send `instruct`/`cancel`/`decision`/`reset`/`analyzeFile`/`saveSkill`/`deleteSkill`/`deleteSiteMap`/`activateSiteMap`/`discardSiteMap`/`nameSession` unprompted, and the daemon streams `run` and `siteMapDraft` frames back. Any new extension-initiated frame must also be listed in `EXTENSION_REQUEST_FRAMES` — `isExtensionRequest()` keys on it, and a frame missing from that list is silently dropped by `ExtensionLink.receive` rather than erroring. `assertToolNamesRoundTrip` aside, protocol bumps require rebuilding both sides *and* reloading the extension — an older worker is refused at `hello`.- **Navigation is special-cased** in `lib/bridge/invoke.ts`. It tries the in-page action first (pushes history) and falls back to `browser.tabs.update` only when `sendMessage` fails with "Receiving end does not exist" — a pre-delivery error proving the action never ran. Any other failure means the page already unloaded and is navigating, so re-issuing would navigate twice.- **Service-worker lifetime.** Daemon pings every 20s reset Chrome's 30s idle timer; a 1-minute `alarms` tick is the only thing that can revive the worker once it is torn down.@@ -97,7 +97,7 @@ Call path: side panel/popup → `runtime.connect` port (`lib/bridge/run-port.ts`- **Run identity is the safety boundary.** The spawn carries `VOICELINK_AGENT_RUN=<runId>`; `RemoteBridge` attaches it to every invoke, and the daemon routes tagged invokes through `AgentSession.invokeForRun` — which emits the timeline `tool`/`toolResult` events, applies the approval gate (`requireApproval` in `~/.voicelink/config.json`, default `page.submitForm`), and refuses invokes whose run is no longer active (a cancelled agent cannot keep driving).- **Skills are markdown system prompts**, read from three directories on every run, each shadowing the last by name: `mcp/skills/*.md` (bundled, in git) → `~/.voicelink/skills/` (hand-written) → `~/voicelink/skills/` (uploaded from the extension; machine-managed, so the daemon only ever writes here). Front-matter: `name`, `description`, `triggers: [...]`, `default: true`, `category`, `domains: [...]`. The format's parser *and* serializer live in `lib/skills/format.ts`, shared by both ends — it must stay import-free so it loads in a worker and in bare Node.-- **Category decides base vs overlay.** A `general` skill is the **base** prompt and competes on trigger hits (`@name` wins, else most hits, else `default: true`). A `category: site-exploration` skill is an **overlay**: never a base, forced `isDefault: false`, and applied on top of the base whenever the active tab's host matches its `domains` (suffix on a dot boundary, so `acme.com` covers `www.acme.com`). `@name` on one pins it regardless of host. The host comes from `instruct`'s `context.url` — URL only; a page controls its own title, so nothing else about the page reaches the system prompt. `buildSystemPrompt(base, overlays)` composes them under a frozen precedence line.+- **Category decides base vs overlay.** A `general` skill is the **base** prompt and competes on trigger hits (`@name` wins, else most hits, else `default: true`). A `category: site-exploration` skill is an **overlay**: never a base, forced `isDefault: false`, and applied on top of the base whenever the active tab's host matches its `domains` (suffix on a dot boundary, so `acme.com` covers `www.acme.com`). `@name` on one pins it regardless of host. The host comes from `instruct`'s `context.url` — URL only; a page controls its own title, so nothing about the page beyond its address reaches the system prompt. (`RunContext` also carries the attached-file list, which the user chose rather than a page.) `buildSystemPrompt(base, overlays, extras)` composes them under a frozen precedence line; `extras` holds the two daemon-assembled blocks, `fetched` and `attachments`, each with its own untrusted-data preamble.- **The prompt is one argv element.** `--append-system-prompt` takes the whole thing, and Linux caps a single argument at 128 KB — hence the 32 KB body and 64 KB prompt caps in `format.ts`/`prompt.ts`. Over the budget, whole overlays are dropped with a log line rather than truncated mid-sentence.### Site mapping (`mcp/src/agent/{sitemap,mapping,site-map-store}.ts`)@@ -111,8 +111,11 @@ Call path: side panel/popup → `runtime.connect` port (`lib/bridge/run-port.ts`- **Quarantine is a filesystem fact.** A map in flight lives under `<skillsDir>/.staging/<uuid>/`, and `readDir` skips dot-prefixed entries — so an unreviewed map is not merely unrouted, it is never opened. Activation is a directory rename. `requireApproval` could not serve as the gate: [run-timeline.tsx](components/run-timeline.tsx) renders the action name and a clipped summary, never `input`.- **Skills now load in two forms**: `<name>/SKILL.md` (what a map generates, since it needs somewhere for screenshots) and flat `<name>.md`; a flat file wins a collision and the collision is logged. `deleteSkill` removes only the flat file — a map has its own `deleteSiteMap` op, so removing a same-named upload cannot take a map's screenshots with it.- **Uploads: the extension's storage is the library, the disk is a projection.** `lib/bridge/skill-store.ts` holds the records (`storage.local`, index and bodies split like `file-store.ts`); `saveSkill` sends *structured fields*, never finished markdown, so the daemon composes the front matter and an upload can never set a key it was not offered — `serializeSkillFile` flattens every value to one line, which is what stops a newline in a description from smuggling in `default: true`. On reconnect the worker re-pushes only records that are not yet `saved`. `voicelink-mcp skills` lists what the router can actually see.-- **Conversation continuity is Claude Code's own session store**: first run passes `--session-id`, follow-ups `--resume`; the side panel's clear button sends `reset`, which just forgets the id.+- **Conversation continuity is Claude Code's own session store, and the *extension* owns the id.** First run passes `--session-id`, follow-ups `--resume`. `AgentSession` still keeps a copy, but only as the fallback for callers that send none — because `dispose()` nulls it on every socket supersede and disconnect (`daemon.ts:197`/`:201`), so a daemon restart or a replaced socket used to lose continuity silently while the panel went on showing the transcript. The authority is `RunContext.claudeSessionId`, read from the saved session record in `storage.local`, which is what makes a conversation resumable days later. Two rules hold it together: the extension may only ever send an id the daemon *confirmed* via a `session` run event (an id Claude Code never created is not resumable, and sending it would fail every run from then on), and `established === false` after a resume attempt means precisely "Claude Code no longer has that session", so the daemon answers `{ kind: 'session', claudeSessionId: null }` and the extension drops it instead of retrying forever. The id is regex-checked before it becomes an argv element. Mapping runs opt out of all of this, as they always did. The clear button still sends `reset`, which forgets the daemon's fallback copy.- **Run events** (`RunEvent` in `lib/actions/protocol.ts`) stream daemon → extension as `run` frames; the background worker fans them out to extension pages over a long-lived `runtime.connect` port (streamed text arrives token by token — `storage.session` would be wrong here). If no page is watching, the run is cancelled after a grace period.+- **Stop always answers, because a wedged Stop button has no other escape.** The panel's `running` comes only from the worker's `active` message, and the worker used to clear `activeRunId` only on a `done`/`error` frame — so any run the daemon had already forgotten (socket replaced, `AgentSession` disposed, error emitted to a dead link) left a Stop button that posted a `cancel` nobody would ever answer. Three things close that: `stopRun()` ends the run locally if `cancelRun` finds no open socket or the daemon does not confirm within `CANCEL_CONFIRM_MS`, a fresh `welcome` reconciles (accepting a connection disposes the previous session, so any run this worker still believes in is over), and `AgentSession.cancel` now emits `done` for a run id it does not recognise instead of returning silently. `instruct` likewise refuses out loud rather than dropping the command, since the page marked itself working the moment it sent it. Matching rule on the receiving side: a `done`/`error` only ends the run whose id it carries — `RUN_IN_PROGRESS` names the *rejected* instruction, and treating it as the end would hand back the composer mid-run.+- **Cancelling drops `AgentSession.active` immediately, rather than waiting for `start()`'s `finally`.** That `finally` does not run until the child process closes, and a child outlives its abort — SIGTERM, then SIGKILL 5s later, longer if it is wedged. While `active` still names the run, `invokeForRun` keeps matching it and honouring its tool calls, i.e. a cancelled agent going on driving the tab. Releasing the slot at cancel time is also what lets the next instruction through without waiting out the teardown. Two related traps in the same path: an `AbortSignal` that has *already* aborted never fires a newly added listener, so `spawnClaude` checks `signal.aborted` and kills at once (otherwise a run cancelled during an earlier `await` spawned a child nothing could kill); and `start()` re-checks `run.abort.signal.aborted` after `prepareMapping`, because `fetchSiteIndex` deliberately never throws, so a cancel during its network phase arrives as no signal at all.+- **Attached files reach the agent as prompt, not as a tool.** Nothing in a run can open a file — `page.listFiles` names them and `page.attachFile` uploads one — so `run-port.ts` reads the file index and sends it in `RunContext.files`, and `filesBlock()` renders it into the system prompt under `FILES_INTRO`. The notes come from the summarizer (`agent/analyze.ts` asks one `claude -p` for both the panel's `summary` and a longer `digest`, since the file is only on disk for that call). The list is the user's choice and trustworthy; the notes restate document content, so they are framed untrusted exactly like page text, and `FILES_INTRO` states the capability limit so the agent says "not in the notes" rather than claiming to have read the file. Capped twice — per-file in the extension, per-block in the daemon — and skipped entirely for mapping runs.- `voicelink-mcp logs` is the debugging surface: every run start/finish, routed skill, and tool call lands in `~/.voicelink/daemon.log`.### The local intent funnel (`lib/intent/`, `lib/bridge/fast-path.ts`)@@ -142,6 +145,15 @@ One module per action: `lib/actions/page/<name>.ts` exporting `defineAction({ na## AI integration points-- **Chat responses:** real — the side panel's composer sends instructions through the agent harness (see above); replies and tool activity stream back into `RunTimeline`. Quick commands never get that far: the local intent funnel answers them in the browser (see above).+- **Chat responses:** real — the side panel's composer sends instructions through the agent harness (see above); replies and tool activity stream back into `RunTimeline`. Quick commands never get that far: the local intent funnel answers them in the browser (see above). The panel's header has the only two view controls: the history button swaps `RunTimeline` for `SessionList` in the same `ScrollArea`, and the reset button beside it closes the current conversation out and starts a fresh one.+- **Reply rendering** is `components/markdown.tsx`, hand-rolled rather than a markdown dependency for two reasons: it renders text a model wrote while reading untrusted pages, so it must not be *able* to emit HTML (every node is a React element, no `dangerouslySetInnerHTML`, hrefs pass a scheme allowlist, an image becomes a link rather than a remote fetch), and it re-runs on every streamed token, so it parses in one pass and lets half-written syntax render as literal text until the rest arrives. Four traps worth knowing if you extend it. The inline rules are shared sticky regexes, so read `lastIndex` *before* calling the node builder — it recurses through the same objects, and reading it after is an infinite loop. A link's label is parsed with linking rules switched off, and anything that recurses must pass `inLink` on, or `[**https://a**](https://b)` nests an anchor whose inner href wins the click. Every quantifier in the link patterns is bounded and `[`/`!` are skipped unless a `)` is within `MAX_LINK_SPAN`: unbounded runs before a required `)` made each bracket in a line of unterminated `](` a full-length backtrack, measured at 29s of blocked main thread, on a parser that re-runs per streamed token. And block parsing recurses per nesting level while rendering recurses again, so `MAX_BLOCK_DEPTH` caps it — one `>` is one stack frame, and an overflow inside render unmounts the panel. Only assistant text is rendered; the user's own words stay as typed.+- **Long replies wrap** because `ScrollArea`'s viewport carries `[&>div]:block!`. Radix wraps children in an inline-styled `display: table` div to measure them, and a table box grows to fit its content — so percentage widths inside resolve against the grown width and text scrolls sideways instead of wrapping. Bubbles need `min-w-0` alongside `max-w-[85%]` for the same reason: a flex item's default `min-width: auto` is its content's min-content width, which one long URL makes wider than the panel.- **Voice input:** real, via the browser's Web Speech API (`webkitSpeechRecognition` — Chrome streams the audio to Google to transcribe; no bundled model). `lib/bridge/use-speech.ts` wraps recognition (mic-permission prompt, continuous mode, silence-driven auto-restart, feature detection); `lib/bridge/use-voice-composer.ts` layers hands-free dictation on top — finalized phrases append to an editable buffer and auto-send after a short silence, with any of {more speech, typing, cancel} calling off the pending send. The **side panel** listens by default (persisted `voicelink:voiceEnabled`, toggled by the mic button); the **popup** is press-to-talk, since a popup dies on blur and can't hold continuous listening. Both surfaces fall back to typing when recognition is unavailable or the mic is blocked. STT swap-out point is `use-speech.ts` alone.- **Page context:** the side panel's paperclip button calls the `page.getPageInfo` action on the active tab and splices the result into the prompt.+- **Attached files:** the upload button stores a file in `storage.local` (`lib/bridge/file-store.ts`) and the daemon summarizes it once. Every later instruction carries the library in `RunContext.files`, which is the only way an agent learns a file exists — see the agent-harness section for why it is prompt rather than a tool.+- **Saved conversations** (`lib/bridge/session-store.ts`, `components/session-list.tsx`) are the third store on the `voicelink:sessions` index + `voicelink:session:<id>` body pattern, and the first one that enforces its own caps (`MAX_SESSIONS`, `MAX_ITEMS`) — sessions accumulate with no user action to prompt pruning. The **side panel** writes them, debounced, from `useRun({ persist: true })`; `text` events arrive per token, so an undebounced write would be one `storage.local.set` per token. Every run end forces a flush, which bounds the loss to one debounce window and only when the panel closes mid-stream. The popup deliberately does *not* persist: it fires one instruction and closes, so it would only ever mint half a session and the panel would mint another.+ - **Two writers share each record**, which is why `putSession` merges rather than replaces: the panel owns `SessionFields`, the worker owns `title`/`titledAtTurn`/`namingAt`. A wholesale replace would erase the title on the panel's next keystroke.+ - **Names come from a one-shot `claude -p`** (`mcp/src/agent/title.ts`, dispatched beside `analyzeFile` in `daemon.ts` since it is stateless), regenerated at the `TITLE_TURNS` thresholds. It is handed **the user's own messages only** — an assistant turn restates whatever page the agent was reading, so feeding one in would let a site name a conversation in the user's history. The name is flattened and clamped on both ends, and rendered as text, never through `Markdown`. `namingAt` is a timestamp rather than a flag because Chrome can stop the worker mid-request: a flag would sit true forever and that conversation would never be named again.+ - **`runClaudeJson` is shared** by the namer and the summarizer (`runner.ts`) rather than copied — the stdout shape, the ENOENT wording and the abort handling must not drift. It denies every tool by default; the summarizer asks `Read` back, the namer asks for nothing.+ - **`RunItem` ids are uuids, not a counter.** Rows now outlive the page that made them, so a restored transcript and the run that follows it are two id producers at once, and a counter would have the second reusing the first's — patching, or replacing, a row from the conversation just reopened.+ - Restoring navigates the tab back to the session's last URL through the ordinary bridge `invoke`, so it inherits the in-page-then-tabs-API fallback in `invoke.ts`. It is refused while a run is in flight: swapping the transcript out from under one would leave its remaining events appending to — and saved into — the conversation just reopened. It deliberately never sends `reset`, which would forget the very conversation being restored.