An agentic browser harness: an AI side panel & orb in your own logged-in Chrome, Edge, Arc, Brave or even Firefox run by the agent/subscription you already use (Claude Code, Codex, Antigravity). Free, open source, no API key. Also an MCP server.
Inside the extension
The extension is Browsentic's browser-side component: it runs every page action, hosts the side panel and the settings page, draws the rail, the hands-free orb and the action cues into pages, and keeps each run in its own tab. This page covers where an invoke frame runs and how each of those parts works.

Background vs content script
Not every capability can run in the page, so the background service worker keeps some for itself:
| Handled entirely in the background | Why |
|---|---|
listFiles, attachFile, listRecordings, readRecording |
The data lives in extension storage |
startMonitor, monitorStatus, awaitMonitor, stopMonitor |
Monitors outlive any single page |
startDiagnostics, readConsole, readNetwork, stopDiagnostics |
The debugger attaches to a tab, and the buffers outlive any single page |
openTab, switchTab, closeTab, screenshot, navigate |
Need the tabs/scripting APIs |
| Everything else | Forwarded to the content script |
Nine of those need Chrome's debugger API, which Firefox has no counterpart for: trustedClick,
the two captcha tools, the four diagnostics tools, injectCode and runCode. Each is marked
chromiumOnly in its module, and describeActions('firefox') leaves them out, so a Firefox build
never lists them. The agent there is offered a shorter list, and the daemon knows that list by its
hash and reports it as in sync. The actions themselves stay registered on both builds: a stale
skill that names one is answered with UNSUPPORTED and a hint, not UNKNOWN_ACTION. The
build-time branch lives in one place, describeOwnActions() in src/lib/bridge/own-actions.ts;
the registry itself never reads import.meta.env, because the daemon bundles it under Node.
The microphone is granted from a tab
Chromium anchors a permission prompt to a tab. A side panel and a popup have none, so
getUserMedia there is refused without asking, and no manifest permission grants the microphone to
an extension. use-speech.ts therefore reads navigator.permissions first. On prompt it never
calls getUserMedia (repeated silent refusals would earn the origin a temporary block) and offers
Allow microphone instead, which opens the unlisted mic-permission.html entrypoint in a tab,
where the prompt can appear. The grant belongs to the extension's origin, so the panel, the launcher
and hands-free mode's offscreen page all see the PermissionStatus change and start listening
without a reload. Firefox prompts from its sidebar on its own and skips the check.
The Firefox build and its gecko block
yarn build:firefox produces a Manifest V2 extension from the same source, and its manifest carries
a browser_specific_settings.gecko block the Chrome build has no use for:
id: [email protected]: addons.mozilla.org signs nothing without a permanent id, and the first signing bound this one to the project's AMO account for good. Changing it would make every installed copy a different add-on.update_url: theupdates.jsonunder the latest GitHub release. Every signed build carries this URL, so moving it means every older install stops updating; the release job publishes the file next to each signed.xpi.data_collection_permissions: websiteContent: Firefox shows this at install. It is the honest declaration: what the agent reads on a page leaves the browser for the daemon and reaches the model behind whichever agent CLI the user runs.strict_min_version: 140.0: the first Firefox that understands the data-collection key.
Six permissions are filtered out of that build (sidePanel, debugger, offscreen,
userScripts, topSites and favicon): the Firefox Manifest V2 build has no use for most of them,
and AMO's validator flags each name it does not know. topSites comes back as an optional
permission, which Browsentic Home asks for (see Browsentic Home). The Firefox sidebar is sidebar_action, which WXT derives from the same
entrypoint, and the nine tools that need the debugger are left off the list a Firefox build offers
(see Background vs content script).
Self-healing injection
The forwarding call is invokeInTab(). A tab that loaded before the extension did has no content
script, so tabs.sendMessage fails with Receiving end does not exist. The extension then injects
the content script via browser.scripting.executeScript and:
- for the four idempotent reads (
getPageInfo,extractText,waitForElement,navigate) retries immediately; - for anything that changes the page returns
TAB_UNREACHABLEwith instructions to re-snapshot first, because the caller's selectors were computed against a page it has not actually seen.
onInstalled also sweeps every open, non-discarded tab and injects there, so a fresh install does
not leave a browser full of unreachable tabs.
Pages that refuse content scripts entirely (chrome://, the Web Store, the new-tab page) stay
TAB_UNREACHABLE permanently. page_navigate still works there: on any tab that is not an http(s)
page it goes straight through the tabs API, which is why it is the documented escape hatch.
The toolbar button
On Chromium, serveLaunchers() in launchers.ts calls
openPanelFromToolbar() from side-panel.ts, which sets
sidePanel.setPanelBehavior({ openPanelOnActionClick: true }), so a click on the toolbar button
opens the side panel and a second click closes it. On Firefox, browserAction.onClicked calls
sidebarAction.toggle(). Neither has a popup in between.
Where sidePanel.setPanelBehavior is missing or rejects (a Chromium browser without a side panel),
the button falls back to action.setPopup({ popup: 'launcher.html' }). That launcher,
entrypoints/launcher/, is a small popup with the status pill, a mic, a text field, Open side
panel, Open hands-free, and the same DaemonLink and AgentPicker the connection sheet shows.
The Firefox build leaves it out.
WXT writes the manifest's action key only when a popup/ entrypoint exists, so wxt.config.ts
declares action: { default_title: 'Browsentic' } itself. The shortcut bound to the button
(_execute_action, or _execute_browser_action on Firefox) is listed as Same as the toolbar
button.
The panel's tabs
PanelNav owns the tab strip: Chat, History, Skills, Recordings, Schedules.
Adding one takes an entry in PANEL_TABS (which PanelTab is derived from), an entry in TABS and
RAIL_TABS, and a branch in the side panel's body; there is no router. A stored tab that is no
longer in PANEL_TABS (settings, from before it moved out) opens on Chat.
The strip shows as many labels as fit. A ResizeObserver measures the width the labelled row
needs and steps through three fits: every label, then only the open tab's, then icons alone with a
dot where the count chip was. A width is only knowable while it is on screen, so each fit records
its own and steps one rung down; stepping back up waits for the width it already learned, which
keeps the strip from flapping between two fits at one panel width. In practice five labels want
~485 px and one wants ~225 px, so a side panel at its usual size lands on the middle fit.
The settings page
Settings are not a panel tab. They are the extension's options page, entrypoints/options/, which
WXT turns into options_ui with open_in_tab on both browsers. So it needs no permission, Chrome
lists it as Options on the toolbar icon's menu, and Firefox as Preferences in about:addons.
The panel's header, the launcher's header and the connection sheet open it with
runtime.openOptionsPage(), which focuses a settings tab already open instead of adding another.
The page is a sidebar of seven sections (Extension, Profile, Guardrails, Blocked sites, Agent, Connection, About),
with the open one in the URL hash, so a reload or a link lands on it; an unknown hash,
such as the old #appearance, lands on Extension. Agent and Connection are the same
AgentPicker and DaemonLink the connection sheet and the launcher show; those keep their copies,
so a blocked run is still one click from its fix.
Extension is the theme, then what belongs to this browser alone, in storage.local: the new
tab page (browsentic/newTab, see Browsentic Home), the
right-click items (browsentic/contextMenu), the keyboard shortcuts, and hold to talk
(browsentic/pushToTalk, the switch the orb's menu writes too). The shortcuts are the manifest's
commands, declared from shortcuts.ts with suggested keys. The
page lists them with commands.getAll(), re-read whenever it comes back into view, because the
browser owns the keys and only its own page changes them: chrome://extensions/shortcuts
(edge:// on Edge), or commands.openShortcutSettings() on Firefox.
Blocked sites is the one section the daemon never sees. Its list is browsentic/blockedSites in
storage.local, written only by useBlockedSites() on this page, and no socket frame carries it in
either direction: its purpose is to bind the agent, so it cannot live on the agent's side of the
socket. It is enforced in the extension, in three layers:
- The gate at the top of
invokeForHarness(invoke.ts) checks the target tab'surlandpendingUrl, every frame on the focused path, and where the action would send the browser (navigate,captureDownload). It is written as a list of exemptions (actions that touch no tab, andopenTab/switchTab/closeTab, which check the tab they pick in tabs.ts), so an action added later is gated without anyone remembering to. It also refuses any tab that is not an http(s) page, which keeps the debugger-backed actions off the extension's own pages. After the action, the tab is checked again, and a result from a tab that landed somewhere blocked is replaced by the refusal. - The content script re-checks its own frame's
location.hrefbefore dispatching (host.ts). - Everything that does not pass through the gate checks for itself through site-guard.ts: the scheduled-task tab, saved tools and their user scripts, A-Eye, the recorder, monitors and diagnostics (which also end when their tab moves onto a blocked site, or the list changes under them), recordings offered to the agent, and the run start, which drops the tab's URL and the picked element.
The list is read from storage on every check (a woken worker's memory is not the truth), and a value that is not a list refuses every web page, not none. The pattern grammar is in blocked-sites.ts, one RegExp per pattern so that a user script can carry the ones for its own origin.
About reads, writes and sends nothing. Its links and the bug report's URL come from
src/lib/about.ts, which the Windows app shares; the Mac app keeps the same
in About.swift. The versions are the manifest's, the build target (import.meta.env.FIREFOX), the
daemon's from DaemonState, SOCKET_PROTOCOL_VERSION, the active agent, the browser's release
(userAgentData.brands, or runtime.getBrowserInfo() on Firefox) and runtime.getPlatformInfo().
Report a bug opens the bug_report.yml issue form with those in its environment and agent
fields; nothing leaves the browser until the reader submits it on GitHub.
The theme and Guardrails are shared with the Mac app, and the daemon keeps both in
~/.browsentic/config.json:
- Two bridge ops,
preferencesandsetPreference, forward to the socket frames of the same name. The answer,preferencesInfo, is also pushed on connect and after every change to the file, whoever made it, and lands inDaemonState.preferences. The guardrail section renders that, so a row flipped in the Mac app moves here without a reload. - The theme stays in
storage.localtoo, because every surface paints from it before any socket exists.servePreferences()in preferences.ts keeps the two in step:- A push writes the daemon's theme locally.
- A local pick is sent up, unless it only echoes the push.
- A pick made while the daemon could not be told sets
browsentic/theme.unsynced, and wins at the next connect. So does a pick made before the browser was ever paired, when config.json names no theme yet.
Themes
A theme is one block of raw tokens in globals.css: grounds, inks, lines, the six named colours,
and four dials (--wash, --grain, --neon, --glow) that the utilities read instead of naming a
colour themselves. daylight sets --grain and --neon to nothing because film grain and a neon
halo only exist on black. The semantic tokens the shadcn primitives read are mapped once,
afterwards, from whichever block won, so a theme is that block and no more.
The blocks hang off [data-theme] instead of :root, and the attribute is scoped, not global.
Put it on a swatch and that subtree renders in the theme it advertises, which is how the picker
draws four live previews with no second copy of the palette. Only raw tokens re-resolve that way
(bg-ground, bg-brand); the semantic ones (bg-background, border-border) are computed at
:root and inherit, so a scoped preview must not use them.
browser.storage.local under browsentic/theme is the truth, read by useTheme(). It answers a
tick after the page has painted, though, which on a light theme means a dark flash every time the
panel opens. So mountTheme() applies a localStorage mirror of the same id synchronously before
React mounts, and lets storage correct it a moment later. index.html carries
data-theme="ember" and an inline ground so the frame before the stylesheet is neither white nor
unpainted; applyTheme clears that inline colour, since an inline colour outranks every rule and
would otherwise pin <html> to a dark ground under a light theme.
Browsentic Home: the new tab page
entrypoints/newtab/ is the browser's new tab page; WXT writes it as chrome_url_overrides.newtab
in both builds. It is a full-tab surface over the panel's own pieces: the Composer at
size="hero" (no A-Eye, since there is no page to point at, and no scheduling, since the Schedules
tab lives in the panel), an agent chip that opens AgentPicker, the status pill over DaemonLink
and AgentPicker, frequent sites, and recent conversations. See New tab
for what the user sees.
Its conversation is its own tab's. useOwnTab() reads the page's tab with tabs.getCurrent(),
and useRun(own) pins the run port to that tab instead of following the tab in front, so a Home tab
in the background still shows, and sends to, its own session. The browser hides a new tab page's
address from the extension, so a run from Home starts with no URL, and with no URL the run's host
scope starts open, as it does on a blank tab.
Sending opens the side panel first. sidePanel.open() needs a live gesture, a click stays live
for about five seconds, and a run reaches its first navigation well after that. So Home opens the
panel in the key press or click that sends, then posts instruct. When the agent navigates the tab,
the panel is already showing that tab's conversation, which the navigation does not touch: the
session belongs to the tab. Two things were checked against Chrome 153 before settling on this:
open() refuses outside a gesture, and setOptions({ tabId, enabled: false }) removes only a
tab-specific panel, so the window-wide panel cannot be hidden on Home alone and stays open there if
it already was.
navigate leaves a page that is not a web page through the tabs API. routeInTab sends it
straight to tabs.update when the tab's URL is not http(s), without messaging the page first. An
extension page in a tab receives tabs.sendMessage as soon as it has a runtime.onMessage
listener, and its undefined answer would read as a navigation that happened.
Frequent sites come from topSites.get(), one per host, minus the hosts the user hid, eight at
most (frequent-sites.ts). Chrome serves their icons through
the _favicon endpoint (favicon permission); Firefox returns data URLs. In Chrome, <all_urls>
already covers the warnings for topSites and favicon, so adding them did not hold the update for
re-approval. Firefox would show topSites as a new warning and hold the update until it was
accepted, so the Firefox build lists it under optional_permissions, and Home asks for it from a
Show frequent sites click. The suggestions under an idle composer are tasks matched to those
hosts in suggestions.ts, padded with general ones to three.
Switching Home off cannot give the slot back: only the manifest overrides the new tab page. The
page hands the tab on instead, through tabs.update, since a page may not open a chrome:// address
itself: Google Chrome goes to chrome://new-tab-page/, and every other browser, whose own page is out
of an extension's reach, gets a plain page of frequent sites. The switch is browsentic/newTab in
storage.local, mirrored to localStorage as browsentic/newTab.paint so a tab with Home off never
paints it before handing on.
Home's mic is per page. The panel's voice switch is a stored preference; Home's starts off on every tab, so a new tab never starts listening because the panel was left listening.
Chrome asks once whether to keep the new tab page, on Windows, Mac and ChromeOS. Its Change
back disables the whole extension (DISABLE_USER_ACTION), not only the page, and Keep it is
remembered for good. The guide tells users to keep it and switch Home off in settings instead.
Minimizing: the rail lives in the page
Nothing can resize a side panel. chrome.sidePanel offers open, close, setOptions and
getLayout, and PanelLayout carries only side; the width is the user's, dragged and remembered
by Chrome. Collapsing the panel into itself would only leave an empty column, so minimizing
closes the panel and draws a 44 px rail into the page, a surface the extension can size.
src/lib/rail/events.ts |
The channel, the RailView the background computes, the tab list with its Lucide paths copied out, and one palette per theme spelled in oklch |
src/lib/rail/host.ts |
exposeRail() in the content script: builds the rail in a closed shadow root on documentElement |
src/lib/bridge/rail.ts |
serveRail() and syncRail() in the background: what to paint, and when |
src/lib/bridge/panel-view.ts |
browsentic/panelCollapsed and browsentic/panelTab, read by use-panel-view.ts in the panel |
The content script carries no React and no icon package, which is why the paths and colours are
copied, not imported: src/extension/components/ would drag the whole panel bundle onto every page.
For the same reason, the rail is the one place a theme is spelled twice. It has no stylesheet to
read globals.css from, so RAIL_PALETTES and RAIL_TONES carry an entry per [data-theme]
block, RailView carries the id, and describeRail() reads browsentic/theme alongside everything
else. The same storage.local.onChanged listener that repaints the rail on a tab change repaints
it on a theme change.
Both records are Record<ThemeId, …>, so a new id that forgets them fails yarn compile instead
of stranding an ember rail on every page. The two halves the compiler cannot check are the
[data-theme] block itself and the entry in THEMES: a theme missing either is one the picker
offers and the stylesheet ignores, or the reverse.
A closed shadow root on documentElement is load-bearing. It keeps the rail out of
body.innerText (so extractText never returns it), out of the page's querySelectorAll, and out
of any page stylesheet. The host element has no layout footprint; the rail inside it is
position: fixed, centred on the panel's own side, inset from the edge so it never covers the
page's scrollbar.
The click is the gesture. sidePanel.open() needs user activation, and the only activation the
panel will ever get is the click on the rail, forwarded from the content script. serveRail()
spends it before any await, in a single statement with no storage read in front of it. It
deliberately does not clear the collapsed flag: the panel clears it on mount with the
panelOpened bridge op, so a refused open() leaves the rail on screen instead of leaving the user
with nothing.
syncRail() broadcasts to every tab instead of tracking which tabs carry a rail. That is a
service-worker constraint: a Set of painted tabs comes back empty when the worker is revived,
which would strand a rail on a page with no way to clear it. The first sync of a worker's life
always broadcasts; only a repeated clear is skipped. A tab that has just finished loading is
repainted on its own instead of triggering a broadcast.
A rail only exists while the panel is minimized on purpose. The side panel holds a run port
open for as long as it lives, so the background knows the moment the last panel is gone. If the
collapsed flag is not set at that moment (the panel was closed with the browser's own button, not
the collapse button), clearStrandedRail() forgets what it thinks tabs are showing and forces the
hide onto all of them, catching any tab an earlier broadcast missed. The content script heals its
own side of the same problem: on injection it removes any rail element left by a previous extension
life, and on a back/forward-cache restore it drops the cached rail and asks the background for the
current state with the rail channel's sync op.
The same panel-presence signal drives the context menu, which
launchers.ts repaints from scratch (removeAll, then one
create per item, one paint at a time) whenever the panel opens or closes, hands-free starts or
ends, the settings page switches an item, or the speech service is found missing. Open Browsentic
reads Close Browsentic while a panel is open, and a click on Close shuts it: Firefox's
background closes the sidebar inside the gesture, and Chromium panels are told over the run port to
close themselves. Open Browsentic (Hands Free) exists only where handsFreeSupported() says
speech works, reads Close Browsentic (Hands Free) while the orb is up, and toggles
browsentic/handsFree, exactly as the panel's detach button and the launcher's Open hands-free
do. The toggle-side-panel and toggle-hands-free shortcuts go through the same two toggles. The
panel opens before anything is awaited, because both a menu click and a shortcut hand over a
gesture the first await would spend.
Pages that refuse content scripts (chrome://, the Web Store, the new-tab page) get no rail. That
is the same TAB_UNREACHABLE set as everywhere else, and it is why the toolbar icon and the
Open/Close Browsentic context-menu item remain the guaranteed way back in.
Hands-free: the orb lives in the page
Detaching is the rail's sibling: the panel closes, and a microphone orb the user talks to is drawn
into the page instead. The panel cannot be shrunk, so the fold the user sees (the panel collapsing
into a mic that drops away) is DetachVeil playing for 560 ms before closeSidePanel(), then the
page's orb rising once the page has widened. Firefox closes its sidebar only inside the click's
gesture, so it skips the fold.
src/lib/handsfree/events.ts |
The channel, OrbView, the orb's requests, the dictation reports, and the Lucide paths |
src/lib/handsfree/host.ts + styles.ts |
exposeHandsFree(): the orb, its arc menu, caption, countdown ring and approval card, in a closed shadow root |
src/lib/handsfree/geometry.ts |
Pure layout: which way the arc opens from the edges the orb is against, and which side a caption or card fits on |
src/lib/bridge/hands-free.ts |
serveHandsFree(): when the microphone listens and for which tab, what every orb shows, and the orb's requests |
src/extension/entrypoints/dictation/ |
The offscreen page speech recognition runs in, Chromium only |
Speech needs a document, and the panel is gone. The background has no DOM, and recognition in a
content script would run on the page's origin: a prompt per site, and none at all where the page's
Permissions-Policy forbids the microphone. So recognition runs in an offscreen document
(dictation.html, reason USER_MEDIA) on the extension's own origin, which the mic-permission
tab already granted. The document exists exactly while the orb should listen: creating it starts the
mic and closing it stops it, so it needs no command channel of its own. It reports a
DictationPhase and each interim and final transcript; the background keeps the phase in
browsentic/dictation and relays transcripts to the one tab listening. The phase is only ever what
the page itself reported (the background writes none ahead of it, so the orb never shows a
microphone the page has not opened), except failed for a document that could not be created,
which then waits for a press on the orb instead of being retried on every repaint. Only network
is no-service; any other recognizer error is failed, a retry and no verdict on the browser.
Chrome runs one recognizer at a time, and a second one aborts the first. That is why the panel
and hands-free never overlap (the panel's panelOpened ends hands-free, starting hands-free sends
every connected panel close, and the orb does not listen while any panel's run port is
connected), and why an aborted the document did not ask for is read as another page taking the
mic. The document then stops as yielded instead of restarting into a tug-of-war; a press on the
orb takes the mic back.
Hands-free only exists where speech does. speech-support.ts decides it in two layers.
speechCapable() covers what is knowable without listening: a Firefox build (no offscreen
documents, no recognizer), no offscreen API, no recognizer constructor, or a brand known to ship
the recognizer without a service behind it (Brave, which fails every attempt with network). The
service itself settles the rest: the panel's dictation and the offscreen page record
browsentic/speechService in storage.local, works on the first transcript and missing on a
network failure, and works always wins, since a service that answered once is only out of reach.
The panel hides the detach button and /hands-free behind useHandsFreeSupported(), which starts
hidden so nothing flashes; paint() ends hands-free on the spot if it finds itself anywhere
unsupported, and a missing found mid-session raises a toast saying why the mic left. Vitest
serves import.meta.env.FIREFOX as the string "false", which is truthy, so the decision itself
is the pure capableOf() to make it testable at all.
The microphone listens for one tab: the one in front of a focused window, whose conversation has
no run and no approval waiting, while the link is up and the user has not muted it. Every other orb
is painted paused. paint() computes each tab's OrbView, sends only the ones that changed since
that tab was last told, and records which tabs answered; a tab with no orb is never aimed at. The
tab in front is told afresh until it answers, and gets the content script injected if it has none,
as every open tab does after the extension reloads. The tab the mic is aimed at is asked again once
it finishes loading, so a navigation to a page the content script cannot run in lets go of the mic.
Like the rail, this is a cache the worker can lose: a revived worker repaints everything. Only the
daemon link's state and the parts of browsentic/tabSessions an orb shows (which tabs, the run,
the approval) trigger a repaint; a run rewrites the rest on every step.
The orb owns only what dies with the page: what has been said since the last send, the countdown
ring (AUTO_SEND_MS, the composer's 1.6 s), the A-Eye focus, the live-code toggle and the files
attached since the last send. An instruction goes through the same instruct the panel's run port
takes, via runCommand(), so it lands in the tab's own conversation and takes the fast path as a
typed one would. A finished turn's last reply or error comes back as a say caption through
onTurnSettled(), which also fires for a turn that never reached the agent (answered on the fast
path, refused as SESSION_LIMIT or RUN_IN_PROGRESS, or with no daemon attached), since no panel
is there to show those. An attach answers only once the file belongs to the conversation, via
attachFile().
Hold to talk swaps the dictation page's mode, not the orb's. browsentic/pushToTalk in
storage.local is a preference, so it is remembered. With it on, the background opens
dictation.html?hold, which holds no microphone until a talk command arrives (talk: true
relayed only from the tab being listened for, talk: false from any, since the key can come up
after the mic moved on). On talk: false it calls stop(), not abort(), so Chrome finishes the
words already spoken; anything still interim is reported as final, then the page says held. The
orb sends on that held or after 2.5 s, whichever comes first. A page left in the other mode is
closed and opened afresh, which is how the toggle takes effect, and so is a hold page whenever the
mic moves to another tab, so words held down in one never land in another. The page checks it is
still wanted after the permission query, so a release during that query cannot leave a recognizer
running, and an orb removed mid-hold releases the key on its way out.
The key is event.code === 'ControlLeft' (HOLD_KEY_CODE): a physical position, so layout-proof,
present on every platform, and inert on its own everywhere. Because Control is a modifier, a hold
engages only after 250 ms with nothing else pressed, and any other key, a pointer press, a wheel, a
window blur or the tab hiding cancels it and discards what was heard. Only trusted events count;
otherwise a page that synthesizes a keydown could open the user's microphone. The listeners are on
the top page's window, since the content script runs in no other frame; an orb whose host the page
removed tears its listeners down on the next event instead of answering beside its replacement.
The orb shows approvals. The view carries the waiting approval, the orb turns ember, and a
press opens the same Allow / Deny / Always choice as the timeline, placed on whichever side of the
orb it fits and pointing at it. It carries the agent's code whole, scrollable and never cut short,
since Allow runs all of it. An approval in a tab nobody is looking at raises a toast on the page
that is in view, or an OS notification when the browser is in the background. It does so once:
the announced ids are kept in browsentic/approvalsAnnounced so a revived worker does not raise
them again.
A-Eye skips every overlay. The rail, the toast and the orb carry data-browsentic-overlay
(src/lib/overlay.ts). A closed shadow root retargets a hit to its host, so the lens sees the host
element under the pointer; it draws no box there and a click there picks nothing, whether the pick
is the user's or the agent's. An orb that starts a pick also hides until the pick is over.
So does the agent's pointer. elementAt() in pointer.ts reads elementsFromPoint() past any
overlay, so assertUncovered never blames the mic for covering a target and a drag never drops onto
it. A real pointer still lands on whatever is on top, so trusted-input.ts marks every overlay host
inert for the length of a trusted click or drag (inert reaches through the closed shadow root),
and the gesture reaches the page beneath instead of pressing the mic's own stop.
State: browsentic/handsFree in storage.session (on while present, with muted and since),
browsentic/dictation and browsentic/approvalsAnnounced in storage.session, and in storage.local browsentic/speechService,
browsentic/pushToTalk and browsentic/orbPosition (the orb's centre as viewport fractions, so one drag places it on every tab). Hands-free is session state
on purpose: a restarted browser comes back with the panel, not a live microphone.
Action cues: a ring in whichever frame the agent acts in
Every agent action is ringed on the page; see Action cues. The
background drives it, because invokeForHarness is the one place that sees each action together
with its tab: the content script's dispatch knows no run, and also receives internal sub-calls (a
screenshot is a plan plus a scroll per tile) that are not the agent's actions.
One table decides what an action looks like. cueFor() in src/lib/cues/plan.ts maps each
registry action to an element cue (a target, a point, or the focused element), a page cue (the
viewport's edge) or none, and a test fails if a registry action has no entry, so a new action has
to choose. It is built from the sealed input, before releaseForAction, and copies only the
target fields, points, a named key and a navigation's host, never a value, a text or a query.
cued() wraps the tab-bound half of dispatch (src/lib/bridge/action-cues.ts). It sends
show to the focused frame for an element cue or the top frame for a page cue, waits at most
CUE_LEAD_MS so the ring is painted before the action fires, runs the action, then sends settle
with its outcome. A frame that does not answer (frozen, alerting, or without a content script)
costs the action that wait and nothing else, and a cue never fails an action. Blocked sites are
refused by guardTarget before dispatch runs, so they are never ringed. The switch is
browsentic/actionCues in storage.local, off unless it is true.
The page gets one element and nothing more. exposeCues() runs in every frame (before the
top-frame return in content.ts), and the first cue in a document mounts #browsentic-cues on
documentElement with its attributes already set. That is one childList mutation for the page's
observers and none after it, because every ring, caption and fade is inside a closed shadow root.
Nothing in it takes a pointer, so a trusted click's inert pass (markOverlaysInert) leaves it out,
and it never focuses or scrolls. Its one listener is pageshow, to clear a ring a back/forward
restore brings back. While a cue is on screen a requestAnimationFrame loop follows the element's
box; a target that has not appeared yet (waitForElement) is looked for again every 250 ms.
It sits in the top layer. The host is a popover="manual", shown while a cue is up and hidden
when none is, and raised again when a modal dialog, another popover or a fullscreen element has
appeared, so a ring on a cookie banner's button sits above the banner. Showing and hiding it fires
the popover's toggle events on the host, which a page listening in the capture phase can hear, and
its ::backdrop is switched off from inside the shadow root so a page's own ::backdrop rule never
dims the screen.
A capture quenches first. screenshot, findCaptcha, solveCaptcha and pickElement carry
quench in the table, so cued() clears every frame still lit on that tab. The background
remembers which frames are lit until a few seconds after the cue settles, longer than a hidden
tab's throttled timers take to fade it. The host answers after two frames, once the page has
repainted without it, or at once in a hidden tab. A tab with nothing lit is not messaged at all.
The phone tab
Driving a phone (see Android phones) adds two unlisted entrypoints and a Port:
| Piece | What it is |
|---|---|
entrypoints/phone/ → phone.html |
The phone tab: a phone frame the page draws itself, the screencast on a canvas, and a bar with back, forward, reload, the address and the phone's tabs. It holds no state of its own beyond what is on screen |
entrypoints/phone-page.ts → phone-page.js |
The page bundle evaluated in an isolated world on the phone: the same action code desktop tabs run, under __browsenticPhone, with no extension API in it (page-api.test.ts fails the build on any). It is never injected into a desktop page |
browsentic/phone Port |
Between the phone tab and the background (phone-mirror.ts): screencast frames and page state down, the user's taps, scrolls, keys and the bar's commands up |
The phone session itself is browsentic/phone in storage.session, written only through
phone.ts, and the side panel and the launcher read it with usePhoneSession. The panel
recognises the phone tab by its id, not its URL: without the tabs permission an extension page's
URL is hidden from tabs.query. The Android toggle is hidden on Firefox, and the Firefox build
leaves out both entrypoints. No new permission was needed.
Tab scoping
A panel conversation is bound to its tabs: the tab it started in, the tabs the user added, and the tabs its runs reached.
The background keeps a registry of tab sessions in browser.storage.session under
browsentic/tabSessions. Each entry maps a sessionId to:
- its main tab, and
tabIds, every tab it spans, userTabIds, the tabs the user put there (the main tab, picker and right-click adds),openedTabIds, the tabs its runs or its task opened,- the tab its next action should land on (
currentTabId), - the live title of its main tab,
- the run currently going in it, if any.
Every frame the daemon sends for an agent run carries that run's runId, and the extension resolves
it to the owning session's current tab. So a run keeps working in its own tab while the user browses
elsewhere, and two sessions in two tabs act independently.
| Situation | Behaviour |
|---|---|
The user adds tabs (addTabs on the run port, or the right-click item) |
Joined to userTabIds; a tab held by an idle session moves, and a session left with none is filed to History. A running, phone or task session keeps its tabs |
| The user sends from one of the session's tabs while it is idle | That tab becomes currentTabId (focusTab) |
A run opens a tab with page.openTab |
Adopted into the same session and recorded in openedTabIds |
page.switchTab onto a tab no session holds |
Adopted, but in neither list |
page.switchTab onto a tab another session owns |
Refused with TAB_IN_USE |
| Every tab of a session is gone | Its actions fail with SESSION_TAB_CLOSED |
| A ninth session is opened | SESSION_LIMIT (the cap is 8) |
Calls with no run behind them (an external MCP client, the local fast path) still target the active tab of the current window.
A session on the phone tab carries phone: { serial } and is anchored on the
phone's page, never on phone.html. Its runs take the phone branch in invokeForHarness, before
guardTarget and every other desktop check, because the tab they resolve to is an extension page
that must never be navigated, reloaded or injected into.
A site-mapping run keeps its own older pin, threading a literal tabId and failing with
MAPPING_TAB_CHANGED if that tab goes away.
When a session spans more than one tab, startTurn sends the daemon its roster in context.tabs
(protocol 26 and later): this window's tabs in tab-strip order, each with its id, sealed title and
host, the current one marked, and chosen on the user's tabs. Tabs on a blocked site are left out
entirely. The daemon lists them in the prompt and lets page.switchTab reach them without asking
(see guardrails.md).
Lifecycle
Closing the main tab hands the session to its first heir that is still open, outside a closing
window: the current tab if the user chose it, then the user's other tabs, then tabs a run reached
but did not open. The session takes the heir's title and gets a note; a run working in the closed tab
is cancelled. With no heir, and always for a task's or the phone's session, closing the main tab ends
the session: the run is cancelled, the transcript is flushed to history, and the entry leaves the
registry. Closing any other tab only takes it out. A finished scheduled task closes openedTabIds
only, so a tab the user added stays open.
Closing the side panel does not: the tab is the anchor. While a run is going, its tab carries a dot on the toolbar badge and on its favicon.
Scheduled runs
The daemon owns the schedule and the clock (src/daemon/schedules/); the extension only runs what it
is handed. A due task arrives as a runTask frame, answered like an invoke. startTaskRun in
run-port.ts opens a background tab and tags its session with the task. An instruction then goes
through startTurn like the Send button's; a recording replays through invokeForHarness, step by
step, handing a failed step to the agent. When the run settles, finishTask reports taskDone, files
the transcript under browsentic:taskRuns instead of History, shows the notice and closes the tab.
The task list reaches the panel the same way the agent state does: the daemon pushes taskList on
connect and after every change, and the extension caches it in browsentic/tasks so the tab still
renders while the daemon is down.
An approval raised by a scheduled run is drawn as a toast on the page in front, not only in the panel.
Its buttons answer through answerApproval, the panel's own path. Only the tab the card was drawn
on can answer, only with isTrusted clicks, and none in its first 800 ms. The page can still
restyle the card's host element and lure a click onto it, so the card offers Allow and Deny
only; Always on ‹site› stays in the panel.
nativeMessaging is what lets a browser start a daemon that is down. When no port answers,
giveUp calls wakeDaemon, which asks the browser to run the com.browsentic.daemon host that
browsentic setup registered. That host runs ensureDaemon and exits. On macOS a browser runs its
native hosts quarantined, and a daemon the host spawned would pass that on to every file it and its
agents write, so Gatekeeper would refuse any code they unpack. So whenever a probe file the caller
writes comes out quarantined, ensureDaemon starts the daemon as a launchd job instead of as its own
child, and waits for one that launchd is still running (src/daemon/quarantine.ts). browsentic stop leaves
~/.browsentic/stopped behind, and while it is there the host starts nothing; the next daemon to
start, by any other path, removes it.
Saved tools that run on every visit
A saved tool keeps its code in storage.local under browsentic/savedTools, and autoRun on its
record says whether it should run by itself. That flag is the truth. auto-run.ts mirrors it into
Chrome's userScripts API, one registration per tool with the id browsentic-tool-<id>, in the
MAIN world at document_idle. The mirror is brought back into line whenever the list changes
(storage.local.onChanged), whenever the worker starts, and on the one-minute browsentic/autoRun
alarm. It compares code and match patterns and re-registers only what differs.
The debugger path that / uses would be wrong here. It shows a bar on every attach, fails with
DevTools open, and would have to be driven on each navigation from the worker. A user script is
injected by the browser itself, is exempt from the page's CSP, and outlives a restart. The cost is
Chrome's own per-extension Allow User Scripts switch (Developer mode before Chrome 138), which no
API can flip. Until it is on, chrome.userScripts is undefined, and autoRunReady() turns that
into the autoRunReady flag on the run port's tools message. Turning it on raises no event and
does not restart the extension; the API just appears in the running worker. So while the panel
shows a tool waiting on it, the panel sends listTools every two seconds, which reconciles first.
The alarm covers a closed panel.
The registration matches the whole host, on any port. Scope is decided inside the page by the rule
/ applies: the exact origin, then the slug of the first path segment. That keeps it right on
single-page sites. There, arriving at /watch is a history entry and not a load, so the script also
listens to the Navigation API's currententrychange. It runs the entry point on each arrival into
scope, not on each move within it, once load has fired and the DOM has gone 400 ms without a
mutation (3 s at most). The approved code is wrapped the way the installer wraps it, and it is
evaluated once per document.
Updates wait for the panel
A browser installs a store update only once the extension is idle, and a live Bridge link means it
never is: a worker that stops reconnects within a second. So a downloaded update used to sit, held
by the browser, until the next browser start. Chrome records this in Secure Preferences as
idle_install_info with delay_install_reason: 2 (wait for idle).
extension-update.ts holds the update for the panel instead. runtime.onUpdateAvailable gives the
held version, which goes to storage.session under browsentic/waitingUpdate. The panel shows the
pill above the prompt while that version is newer than the one running, and while no conversation
has a run going and nothing is being recorded, since a reload ends them all.
Update calls runtime.reload(), which installs the held update at once (Chromium's
ExtensionRegistrar finishes a delayed install on reload). A reload closes the side panel and
every extension page, and clears storage.session.
A Bridge newer than the extension, read from the welcome, means a release is out, so the background
asks the store now with runtime.requestUpdateCheck() rather than waiting for the browser's own
check. Chrome allows one such check every five hours (its backoff starts at the update frequency),
so browsentic/updateCheckedFor spends it once per Bridge version. The call resolves as soon as the
store names a version, before the download, so the pill waits for onUpdateAvailable rather than
trusting that answer. A Bridge-only release finds nothing and shows nothing.
Firefox applies an update at once when nothing listens to onUpdateAvailable, mid-run included.
The listener makes it wait for the pill too. Firefox has no requestUpdateCheck, so it finds an
update only on its own daily check.
An unpacked copy has no store. Its files are read from the folder the Bridge rewrites, so on a
newer Bridge the worker fetches /manifest.json, which reports the version on disk, not the one
loaded. If that is newer, it is held the same way, and Update reloads the extension from the
folder. browsentic update and the apps restart the Bridge before they rewrite the folder, so the
welcome often comes too early; the panel reads the manifest again whenever it opens or gets focus.
Notices while the browser is behind
A finished monitor, scheduled task or reminder first asks noticeOnDevice() in device-notices.ts.
While no browser window is focused and un-minimized, it sends the notice to the Bridge for the
Browsentic app to show as the computer's own notification; a card on a page nobody is looking at
helps no one. When the app takes it, the notice's tab is remembered in storage.session under
browsentic/noticeTabs (the newest 30), and a click in the app brings that tab to the front,
restoring a minimized window first in Chromium; Firefox's focus restores it as it was, where its
state: "normal" would also undo a maximize. When
it does not (the browser in front, no app, notifications off, a Bridge before protocol 24), the
caller goes on to its card or the browser's own notification, as before.
Pairing by itself
auto-pair.ts pairs the extension with Browsentic Bridge through the native host, with no code.
native-pair.ts sends the host a pair op and reads the answer; the grant it hands back is redeemed
by redeemGrant() in socket.ts, through the same handshake a typed code uses, on the grant's own
port first. Transport has the Bridge's half.
When it tries. Only while there is no usable session key: on onInstalled (install and update),
onStartup, the worker's start and the one-minute alarm, whenever a page that shows the link opens
(useAutoPair() sends the autoPair bridge op), and on Connect or Reconnect. One attempt
runs at a time, and each launches a process, so isDue() spaces them by what the last one found:
every minute for the first ten minutes with no Bridge, then every fifteen; hourly for a Bridge too
old or blocked by policy; ten minutes while it is held; 1, 2, 4… up to 30 minutes after failures;
never unasked once this browser was revoked or denied, or after Disconnect. A click always tries.
While ask mode waits on a person and a page is open, the page holds the ask open 25 s at a time,
so it connects the moment someone allows it.
What it shows. DaemonState.bridge says whether the host was found and in what shape (ready,
old, missing, forbidden, policy, stopped, failed, off), read from the browsers' own
error strings; anything else is failed, never a confident wrong cause. DaemonState.pairing
carries the mode, a wait and its approver, a block, or Disconnect. PairingStatus renders each
with its own message, and the code form one click away (leading when nothing automatic can work).
The welcome page. entrypoints/welcome/ opens once, on install: the three pieces (the extension,
the Bridge, an agent), the one-line setup command (with Copy) and the install guide while the Bridge
is missing, and once connected the
agent picker and Open the side panel. The window id is read ahead of the click, because opening
the panel needs the gesture and an await spends it.
Keys. A session key is replaced only by a new one the Bridge proved it issued. When the Bridge
says it holds no session for the key (unauthorized.code), the key is marked stale: unknown pairs
again at once, revoked waits for Reconnect.
Next
Agent runs →: Path B, where an instruction becomes a spawned CLI.