An agentic browser harness: an AI side panel & orb in your own logged-in Chrome, Edge, Arc, Brave or even Firefox run by the agent/subscription you already use (Claude Code, Codex, Antigravity). Free, open source, no API key. Also an MCP server.
TypeScript agentic-browseragentic-browsingagy-cliaiai-agentai-browser-extensionai-toolsbrave-extensionbrowser-agentbrowser-automationbrowser-extensionbrowser-usebrowserskillchrome-extensionclaude-codecodexfirefox-addonmcp-servernanobrowserweb-automation
docs(hands-free): the panel closing everywhere on detach, the whole script in an approval, the agent's pointer passing through the mic, and a recognizer that stopped without writing speech off
Shahriarc6211be3b8bc
diff --git a/docs/guide/features/hands-free.md b/docs/guide/features/hands-free.mdindex faed787..884474c 100644--- a/docs/guide/features/hands-free.md+++ b/docs/guide/features/hands-free.md@@ -42,7 +42,8 @@ When a run finishes, its answer streams in above the mic the same way, and fadesThe mic listens for **the tab in front of you**, and only while the browser window is focused. Switchtabs and it follows; switch to another app and it stops until you come back. It also stops while the-agent is working, and while a side panel is open anywhere.+agent is working. The browser lets only one thing listen at a time, so going hands-free closes the+side panel in every window, and opening the panel anywhere ends hands-free.## The menu@@ -84,8 +85,10 @@ buttons out in that turned order; captions and permission requests move to whichThe mic turns orange and says what the agent is asking to do. Press it for the request — the same**Allow**, **Deny** and **Always on ‹host›** the panel offers, or the code itself when the agent-wrote some. If the request comes from a tab you are not looking at, a card appears on the page you-are on, or a system notification when the browser is in the background; clicking it takes you there.+wrote some — all of it, scrolling if it is long, since **Allow** runs every line. If the request+comes from a tab you are not looking at, a card appears on the page you are on, or a system+notification when the browser is in the background; clicking it takes you there. Each request is+announced once.## A-Eye never lands on the mic@@ -93,6 +96,10 @@ The mic, like the rail and Browsentic's cards, is marked as Browsentic's own. A-agent's — never outlines it and never picks it; a click on the mic while A-Eye is up does nothing.When you start A-Eye from the mic's menu, the mic steps aside until you have picked.+The agent's pointer passes through it too. A button under the mic — a cookie banner's **Accept**, a+sticky footer — is still one the agent can click or drag to, and an agent's click there never lands+on the mic's own stop.+## Which browsersBrowsentic offers hands-free only where it can actually hear you:diff --git a/docs/guide/troubleshooting.md b/docs/guide/troubleshooting.mdindex da6fb94..44c008b 100644--- a/docs/guide/troubleshooting.md+++ b/docs/guide/troubleshooting.md@@ -29,6 +29,7 @@ Those three answer most questions. The daemon log also lives at `~/.browsentic/d| "This browser has no speech service" | Brave and some Chromium builds ship speech recognition without a transcription service behind it | Type instead, or use Chrome or Edge for dictation || No detach button in the panel's header, and no `/hands-free` | Hands-free is offered only where speech can be transcribed — never in Firefox or Brave, and not in a browser whose speech service has failed before it ever worked | Use Chrome or Edge. If the service was only out of reach, dictate in the panel once: the first word it transcribes brings the button back || "Hands-free is not available here", and the mic left the page | The speech service failed before it had ever transcribed a word in this browser | As above. Open Browsentic from the toolbar to carry on by typing |+| The hands-free mic says speech recognition stopped | The recognizer stopped for a reason other than a missing service — a language the service does not take, a moment offline, or the page it listens from could not start | Press the mic to try again. Unlike "no speech service", this never hides hands-free || The hands-free mic turns amber and says another page took the microphone | Chrome runs one speech recognizer at a time, and another page or the panel started one | Press the mic to take it back || Holding Control does nothing | Hold to talk is off; the key went to a frame embedded in the page (some editors); another key or a click joined it; or the mic is muted | Turn **Hold to talk** on from the mic's menu, click the page outside the embedded editor, and hold left Control on its own || `EXTENSION_OFFLINE` | Browser closed, or not paired | Open the browser; `browsentic sessions` to check pairing |diff --git a/docs/internals/extension.md b/docs/internals/extension.mdindex 7a1bb55..02f7e56 100644--- a/docs/internals/extension.md+++ b/docs/internals/extension.md@@ -210,11 +210,16 @@ content script would run on the page's origin — a prompt per site, and none attab already granted. The document exists exactly while the orb should listen: creating it starts themic, closing it stops it, so it needs no command channel of its own. It reports a `DictationPhase`and each interim and final transcript; the background keeps the phase in `browsentic/dictation` and-relays transcripts to the one tab listening.+relays transcripts to the one tab listening. The phase is only ever what the page itself reported —+the background writes none ahead of it, so the orb never shows a microphone the page has not opened —+except `failed` for a document that could not be created, which then waits for a press on the orb+rather than being retried on every repaint. Only `network` is `no-service`; any other recognizer+error is `failed`, a retry and no verdict on the browser.**Chrome runs one recognizer at a time**, and a second one aborts the first. That is why the panel-and hands-free never overlap — the panel's `panelOpened` ends hands-free, and the orb does not listen-while any panel's run port is connected — and why an `aborted` the document did not ask for is read+and hands-free never overlap — the panel's `panelOpened` ends hands-free, starting hands-free sends+every connected panel `close`, and the orb does not listen while any panel's run port is connected —+and why an `aborted` the document did not ask for is readas another page taking the mic. The document then stops as `yielded` rather than restarting into atug-of-war; a press on the orb takes the mic back.@@ -234,23 +239,35 @@ itself is the pure `capableOf()` so it can be tested at all.**The microphone listens for one tab**: the one in front of a focused window, whose conversation hasno run and no approval waiting, while the link is up and the user has not muted it. Every other orbis painted `paused`. `paint()` computes each tab's `OrbView`, sends only the ones that changed since-that tab was last told, and records which tabs answered — a tab with no orb is never aimed at. Like-the rail it is a cache the worker can lose: a revived worker repaints everything.+that tab was last told, and records which tabs answered — a tab with no orb is never aimed at. The+tab in front is told afresh until it answers, and gets the content script injected if it has none,+as every open tab does after the extension reloads. The tab the mic is aimed at is asked again once+it finishes loading, so a navigation to a page the content script cannot run in lets go of the mic.+Like the rail it is a cache the worker can lose: a revived worker repaints everything. Only the parts+of `browsentic/tabSessions` an orb shows — which tabs, the run, the approval — and the daemon link's+state trigger a repaint; a run rewrites the rest on every step.**The orb owns only what dies with the page**: what has been said since the last send, the countdownring (`AUTO_SEND_MS`, the composer's 1.6 s), the A-Eye focus, the live-code toggle and the filesattached since the last send. An instruction goes through the same `instruct` the panel's run porttakes, via `runCommand()`, so it lands in the tab's own conversation and takes the fast path as atyped one would. A finished turn's last reply or error comes back as a `say` caption, through-`onTurnSettled()`.+`onTurnSettled()`, which also fires for a turn that never reached the agent — answered on the fast+path, refused as `SESSION_LIMIT` or `RUN_IN_PROGRESS`, or with no daemon attached — since no panel is+there to show those. An attach answers only once the file belongs to the conversation, via+`attachFile()`.**Hold to talk swaps the dictation page's mode, not the orb's.** `browsentic/pushToTalk` in`storage.local` is a preference, so it is remembered. With it on, the background opens-`dictation.html?hold`, which holds no microphone until a `talk` command arrives — relayed from the-orb only for the tab being listened for — and on `talk: false` calls `stop()` rather than `abort()`,+`dictation.html?hold`, which holds no microphone until a `talk` command arrives — `talk: true`+relayed only from the tab being listened for, `talk: false` from any, since the key can come up after+the mic moved on — and on `talk: false` calls `stop()` rather than `abort()`,so Chrome finishes the words already spoken; anything still interim is reported as final, then thepage says `held`. The orb sends on that `held`, or after 2.5 s, whichever comes first. A page left-in the other mode is closed and opened afresh, which is how the toggle takes effect.+in the other mode is closed and opened afresh, which is how the toggle takes effect, and so is a hold+page whenever the mic moves to another tab, so words held down in one never land in another. The+page checks it is still wanted after the permission query, so a release during that query cannot+leave a recognizer running, and an orb removed mid-hold releases the key on its way out.The key is `event.code === 'ControlLeft'` (`HOLD_KEY_CODE`) — a physical position, so layout-proof,present on every platform, and inert on its own everywhere. Because Control is a modifier, a hold@@ -262,16 +279,24 @@ removed tears its listeners down on the next event instead of answering beside i**Approvals are the orb's to show.** The view carries the waiting approval, the orb turns ember, anda press opens the same Allow / Deny / Always choice as the timeline, placed on whichever side of the-orb it fits and pointing at it. An approval in a tab nobody is looking at raises a toast on the page-that is, or an OS notification when the browser is in the background.+orb it fits and pointing at it. It carries the agent's code whole, scrollable, never cut short,+since Allow runs all of it. An approval in a tab nobody is looking at raises a toast on the page that+is, or an OS notification when the browser is in the background — once, with the announced ids kept+in `browsentic/approvalsAnnounced` so a revived worker does not raise them again.**A-Eye skips every overlay.** The rail, the toast and the orb carry `data-browsentic-overlay`(`src/lib/overlay.ts`). A closed shadow root retargets a hit to its host, so the lens sees the hostelement under the pointer; it draws no box there and a click there picks nothing, whether the pickis the user's or the agent's. An orb that starts a pick also hides until the pick is over.+**So does the agent's pointer.** `elementAt()` in `pointer.ts` reads `elementsFromPoint()` past any+overlay, so `assertUncovered` never blames the mic for covering a target and a drag never drops onto+it. A real pointer still lands on whatever is on top, so `trusted-input.ts` marks every overlay host+`inert` for the length of a trusted click or drag — `inert` reaches through the closed shadow root —+and the gesture reaches the page beneath instead of pressing the mic's own stop.+State: `browsentic/handsFree` in `storage.session` (on while present, with `muted` and `since`),-`browsentic/dictation` in `storage.session`, and in `storage.local` `browsentic/speechService`,+`browsentic/dictation` and `browsentic/approvalsAnnounced` in `storage.session`, and in `storage.local` `browsentic/speechService`,`browsentic/pushToTalk` and `browsentic/orbPosition` — the orb's centre as viewport fractions, so one drag places it on every tab. Hands-free is session stateon purpose: a restarted browser comes back with the panel, not a live microphone.diff --git a/docs/internals/state.md b/docs/internals/state.mdindex e1fe158..68c30b8 100644--- a/docs/internals/state.md+++ b/docs/internals/state.md@@ -58,8 +58,9 @@ leaves history. The daemon holds a copy only while the analyst reads it, in theworkspace at `0600`, and deletes it when the report is in.**Tab sessions** live in `browser.storage.session` under `browsentic/tabSessions`, so they are gone-when the browser closes. So does **hands-free mode** (`browsentic/handsFree`, and the listening-state under `browsentic/dictation`) — a restarted browser never comes back with a microphone on.+when the browser closes. So does **hands-free mode** (`browsentic/handsFree`, the listening+state under `browsentic/dictation`, and the approvals it has already announced under+`browsentic/approvalsAnnounced`) — a restarted browser never comes back with a microphone on.Only where the orb was dragged to (`browsentic/orbPosition`), whether hold-to-talk is on(`browsentic/pushToTalk`) and whether this browser's speech service has ever worked(`browsentic/speechService`) are kept in `storage.local`. So do **diagnostics buffers** (`browsentic/diagnostics`), monitors and