An agentic browser harness: an AI side panel & orb in your own logged-in Chrome, Edge, Arc, Brave or even Firefox run by the agent/subscription you already use (Claude Code, Codex, Antigravity). Free, open source, no API key. Also an MCP server.
TypeScript agentic-browseragentic-browsingagy-cliaiai-agentai-browser-extensionai-toolsbrave-extensionbrowser-agentbrowser-automationbrowser-extensionbrowser-usebrowserskillchrome-extensionclaude-codecodexfirefox-addonmcp-servernanobrowserweb-automation
fix(voice): grant the microphone from a tab, since a side panel and a popup can never be asked
Shahriaree655eea10c0
diff --git a/docs/guide/features/conversations.md b/docs/guide/features/conversations.mdindex cee78f8..43280bf 100644--- a/docs/guide/features/conversations.md+++ b/docs/guide/features/conversations.md@@ -18,6 +18,10 @@ Speech uses the browser's built-in recognition. Nothing is bundled or downloadedmeans audio is streamed to Google for transcription. Type instead if that matters to you; see[Limits](../limits.md#speech-goes-to-google).+The first time you turn the mic on, the panel shows **Allow microphone**. It opens a Browsentic tab+where the browser can ask — a side panel and a popup have nowhere to show that prompt. Choose+**Allow** once; the tab closes itself and dictation starts.+Replies stream back token by token. Follow-ups continue the same conversation, so **"now click thesecond one"** works.diff --git a/docs/guide/troubleshooting.md b/docs/guide/troubleshooting.mdindex 84c5072..e8947dd 100644--- a/docs/guide/troubleshooting.md+++ b/docs/guide/troubleshooting.md@@ -24,6 +24,9 @@ Those three answer most questions. The daemon log also lives at `~/.browsentic/d| "That pairing code is wrong or expired" | Codes are single-use and last 10 minutes | `browsentic pair` for a fresh one. A failed attempt does not burn the outstanding code || "No Browsentic daemon is running" | Nothing on 8765–8767 | `browsentic status`; check `browsentic logs` || `browsentic: command not found` | The global npm prefix is not on `PATH` | `npm prefix -g`, then add its `bin` directory |+| "Browsentic has not been given the microphone yet" | A browser cannot show its microphone prompt inside a side panel or a popup, so a fresh install has never been asked | Press **Allow microphone** and choose **Allow** in the tab that opens |+| "Microphone access is blocked" | The microphone was refused for the extension | Press **Allow microphone**, then click the icon at the left of that tab's address bar and set **Microphone** to **Allow** |+| "This browser has no speech service" | Brave and some Chromium builds ship speech recognition without a transcription service behind it | Type instead, or use Chrome or Edge for dictation || `EXTENSION_OFFLINE` | Browser closed, or not paired | Open the browser; `browsentic sessions` to check pairing || "Load unpacked" cannot see `~/browsentic` | A Flatpak or Snap browser, sandboxed away from your home directory. Snap Chromium is the Ubuntu default | Grant it: `flatpak override --user --filesystem=~/browsentic com.google.Chrome`. Or install somewhere the sandbox can read: `browsentic setup --dir ~/snap/chromium/common/browsentic-extension` || The folder picker does not show `~/browsentic` | It is there; some pickers open elsewhere by default | macOS: press ⇧⌘G and paste the path. Linux: Ctrl+L |diff --git a/docs/internals/extension.md b/docs/internals/extension.mdindex 5cd8acb..f367122 100644--- a/docs/internals/extension.md+++ b/docs/internals/extension.md@@ -18,6 +18,17 @@ Not every capability can run in the page. The background service worker splits t| `openTab`, `switchTab`, `closeTab`, `screenshot`, `navigate` | Need the `tabs`/`scripting` APIs || Everything else | Forwarded to the content script |+## The microphone is granted from a tab++Chromium anchors a permission prompt to a tab. A side panel and a popup have none, so+`getUserMedia` there is refused without ever asking, and no manifest permission grants the+microphone to an extension. `use-speech.ts` therefore reads `navigator.permissions` first: on+`prompt` it never calls `getUserMedia` — repeated silent refusals would earn the origin a temporary+block — and offers **Allow microphone** instead. That opens the unlisted `mic-permission.html`+entrypoint in a tab, where the prompt can appear. The grant belongs to the extension's origin, so the+panel and the popup both see the `PermissionStatus` change and start listening without a reload.+Firefox prompts from its sidebar on its own and skips the check.+## Self-healing injectionThe forwarding call is `invokeInTab()`. A tab that loaded *before* the extension did has no contentdiff --git a/src/extension/components/composer.tsx b/src/extension/components/composer.tsxindex 0792c72..696dbcc 100644--- a/src/extension/components/composer.tsx+++ b/src/extension/components/composer.tsx@@ -363,9 +363,18 @@ function FocusChip({function VoiceStatus({ voice, voiceEnabled }: { voice: Voice; voiceEnabled: boolean }) {if (voiceEnabled && voice.error) {return (- <p className="mb-2 rounded-lg border border-amber/40 bg-amber/10 px-2.5 py-1.5 text-[11px] text-amber">- {voice.error}- </p>+ <div className="mb-2 flex items-center gap-2 rounded-lg border border-amber/40 bg-amber/10 px-2.5 py-1.5 text-[11px] text-amber">+ <span className="flex-1">{voice.error}</span>+ {voice.needsGrant && (+ <button+ type="button"+ onClick={voice.grant}+ className="shrink-0 rounded-full border border-amber/50 px-2 py-0.5 transition-colors hover:bg-amber/15"+ >+ Allow microphone+ </button>+ )}+ </div>);}diff --git a/src/extension/entrypoints/mic-permission/App.tsx b/src/extension/entrypoints/mic-permission/App.tsxnew file mode 100644index 0000000..0a7b915--- /dev/null+++ b/src/extension/entrypoints/mic-permission/App.tsx@@ -0,0 +1,83 @@+import { useCallback, useEffect, useState } from 'react';+import { browser } from 'wxt/browser';+import { Mic, MicOff } from 'lucide-react';+import { Wordmark } from '@/extension/components/brand';+import { Button } from '@/extension/components/ui/button';++type Phase = 'asking' | 'granted' | 'refused' | 'no-microphone';++const CLOSE_AFTER_MS = 1500;++const COPY: Record<Phase, { title: string; body: string }> = {+ asking: {+ title: 'Allow the microphone',+ body: 'Your browser is asking above. Choose Allow, and dictation works in the side panel and the popup from then on.',+ },+ granted: {+ title: 'The microphone is on',+ body: 'This tab closes itself. Go back to Browsentic and talk.',+ },+ refused: {+ title: 'The microphone is blocked',+ body: 'Click the icon at the left of the address bar, set Microphone to Allow, then try again. Typing keeps working either way.',+ },+ 'no-microphone': {+ title: 'No microphone was found',+ body: 'Plug one in or check your system sound settings, then try again.',+ },+};++async function closeThisTab() {+ const tab = await browser.tabs.getCurrent();+ if (tab?.id != null) await browser.tabs.remove(tab.id);+}++export default function App() {+ const [phase, setPhase] = useState<Phase>('asking');++ const ask = useCallback(async () => {+ setPhase('asking');+ try {+ const stream = await navigator.mediaDevices.getUserMedia({ audio: true });+ stream.getTracks().forEach((track) => track.stop());+ setPhase('granted');+ } catch (cause) {+ setPhase(cause instanceof DOMException && cause.name === 'NotFoundError' ? 'no-microphone' : 'refused');+ }+ }, []);++ useEffect(() => void ask(), [ask]);++ useEffect(() => {+ if (phase !== 'granted') return;+ const timer = setTimeout(() => void closeThisTab(), CLOSE_AFTER_MS);+ return () => clearTimeout(timer);+ }, [phase]);++ const { title, body } = COPY[phase];+ const failed = phase === 'refused' || phase === 'no-microphone';++ return (+ <div className="dot-grid flex min-h-screen items-center justify-center p-6">+ <main className="panel-card flex w-full max-w-sm flex-col items-center gap-4 p-8 text-center">+ <Wordmark />+ <span+ className={+ failed+ ? 'flex size-16 items-center justify-center rounded-full border border-amber/50 bg-amber/10 text-amber'+ : 'glow-brand flex size-16 items-center justify-center rounded-full border border-brand/50 bg-brand/10 text-brand'+ }+ >+ {failed ? <MicOff className="size-7" /> : <Mic className="size-7" />}+ </span>+ <h1 className="text-base font-medium">{title}</h1>+ <p className="text-xs leading-relaxed text-ink-dim">{body}</p>+ {failed && (+ <Button size="sm" onClick={() => void ask()}>+ Try again+ </Button>+ )}+ </main>+ </div>+ );+}diff --git a/src/extension/entrypoints/mic-permission/index.html b/src/extension/entrypoints/mic-permission/index.htmlnew file mode 100644index 0000000..e0fd958--- /dev/null+++ b/src/extension/entrypoints/mic-permission/index.html@@ -0,0 +1,13 @@+<!doctype html>+<html lang="en" data-theme="ember" style="background-color: #1a1512">+ <head>+ <meta charset="UTF-8" />+ <meta name="color-scheme" content="dark" />+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />+ <title>Browsentic — microphone</title>+ </head>+ <body>+ <div id="root"></div>+ <script type="module" src="./main.tsx"></script>+ </body>+</html>diff --git a/src/extension/entrypoints/mic-permission/main.tsx b/src/extension/entrypoints/mic-permission/main.tsxnew file mode 100644index 0000000..0026177--- /dev/null+++ b/src/extension/entrypoints/mic-permission/main.tsx@@ -0,0 +1,13 @@+import React from 'react';+import ReactDOM from 'react-dom/client';+import App from './App';+import { mountTheme } from '@/lib/bridge/theme';+import '@/extension/assets/globals.css';++mountTheme();++ReactDOM.createRoot(document.getElementById('root')!).render(+ <React.StrictMode>+ <App />+ </React.StrictMode>,+);diff --git a/src/extension/entrypoints/popup/App.tsx b/src/extension/entrypoints/popup/App.tsxindex 031629b..a652ab5 100644--- a/src/extension/entrypoints/popup/App.tsx+++ b/src/extension/entrypoints/popup/App.tsx@@ -53,6 +53,10 @@ export default function App() {function toggleMic() {if (!voice.supported) return;+ if (micOn && voice.needsGrant) {+ voice.grant();+ return;+ }if (micOn && voice.error) {voice.retry();return;@@ -62,7 +66,9 @@ export default function App() {const hint = status.blocker? BLOCKED[status.blocker]- : voice.error+ : voice.needsGrant+ ? `${voice.error} Tap to allow it.`+ : voice.error? voice.error: micOn && voice.listening? voice.interim || 'Listening… speak, then pause'diff --git a/src/lib/bridge/mic-permission.ts b/src/lib/bridge/mic-permission.tsnew file mode 100644index 0000000..443911b--- /dev/null+++ b/src/lib/bridge/mic-permission.ts@@ -0,0 +1,43 @@+import { browser } from 'wxt/browser';++export type MicPermission = 'granted' | 'prompt' | 'denied';++const MIC_PERMISSION_PAGE = '/mic-permission.html';++function micStatus(): Promise<PermissionStatus> {+ return navigator.permissions.query({ name: 'microphone' as PermissionName });+}++export async function micPermission(): Promise<MicPermission> {+ if (import.meta.env.FIREFOX) return 'granted';+ try {+ return (await micStatus()).state;+ } catch {+ return 'granted';+ }+}++export function watchMicPermission(onChange: (state: MicPermission) => void): () => void {+ if (import.meta.env.FIREFOX) return () => {};+ let status: PermissionStatus | null = null;+ let live = true;+ const listener = () => {+ if (status) onChange(status.state);+ };+ void micStatus()+ .then((found) => {+ if (!live) return;+ status = found;+ status.addEventListener('change', listener);+ })+ .catch(() => {});+ return () => {+ live = false;+ status?.removeEventListener('change', listener);+ };+}++export function openMicPermissionPage() {+ const url = (browser.runtime.getURL as (path: string) => string)(MIC_PERMISSION_PAGE);+ return browser.tabs.create({ url });+}diff --git a/src/lib/bridge/use-speech.ts b/src/lib/bridge/use-speech.tsindex e7d0296..52f7769 100644--- a/src/lib/bridge/use-speech.ts+++ b/src/lib/bridge/use-speech.ts@@ -1,5 +1,6 @@import { useCallback, useEffect, useMemo, useRef, useState } from 'react';import { browser } from 'wxt/browser';+import { micPermission, openMicPermissionPage, watchMicPermission } from './mic-permission';interface SpeechAlternativeLike {readonly transcript: string;@@ -48,6 +49,8 @@ function recognitionCtor(): SpeechRecognitionConstructor | null {const RESTART_DELAY_MS = 400;const MIC_BLOCKED = 'Microphone access is blocked. Allow it for this extension, or type instead.';+const NO_SPEECH_SERVICE = 'This browser has no speech service to transcribe with. Type instead.';+const MIC_NOT_GRANTED = 'Browsentic has not been given the microphone yet.';export const VOICE_PREF_KEY = 'browsentic:voiceEnabled';@@ -88,6 +91,8 @@ export interface Speech {supported: boolean;listening: boolean;error: string | null;+ needsGrant: boolean;+ grant: () => void;start: () => void;stop: () => void;}@@ -95,6 +100,7 @@ export interface Speech {export function useSpeech({ onFinal, onInterim }: UseSpeechOptions): Speech {const [listening, setListening] = useState(false);const [error, setError] = useState<string | null>(null);+ const [needsGrant, setNeedsGrant] = useState(false);const supported = useMemo(() => recognitionCtor() !== null, []);const handlers = useRef({ onFinal, onInterim });@@ -102,6 +108,7 @@ export function useSpeech({ onFinal, onInterim }: UseSpeechOptions): Speech {const recognition = useRef<SpeechRecognitionLike | null>(null);const wanted = useRef(false);+ const blocked = useRef(false);const restartTimer = useRef<ReturnType<typeof setTimeout> | null>(null);const micGrant = useRef<Promise<void> | null>(null);const beginRef = useRef<() => void>(() => {});@@ -127,11 +134,20 @@ export function useSpeech({ onFinal, onInterim }: UseSpeechOptions): Speech {const begin = useCallback(async () => {const Ctor = recognitionCtor();if (!Ctor || recognition.current) return;+ if ((await micPermission()) === 'prompt') {+ setListening(false);+ setNeedsGrant(true);+ setError(MIC_NOT_GRANTED);+ return;+ }+ setNeedsGrant(false);try {await ensureMic();} catch {wanted.current = false;+ blocked.current = true;setListening(false);+ setNeedsGrant(true);setError(MIC_BLOCKED);return;}@@ -163,7 +179,9 @@ export function useSpeech({ onFinal, onInterim }: UseSpeechOptions): Speech {if (event.error === 'aborted' || event.error === 'no-speech') return;if (event.error === 'not-allowed' || event.error === 'service-not-allowed') {wanted.current = false;+ blocked.current = true;micGrant.current = null;+ setNeedsGrant(true);setError(MIC_BLOCKED);return;}@@ -172,6 +190,11 @@ export function useSpeech({ onFinal, onInterim }: UseSpeechOptions): Speech {setError('No microphone was found.');return;}+ if (event.error === 'network') {+ wanted.current = false;+ setError(NO_SPEECH_SERVICE);+ return;+ }setError(event.error);};speech.onend = () => {@@ -201,6 +224,7 @@ export function useSpeech({ onFinal, onInterim }: UseSpeechOptions): Speech {const stop = useCallback(() => {wanted.current = false;+ blocked.current = false;if (restartTimer.current) clearTimeout(restartTimer.current);const speech = recognition.current;recognition.current = null;@@ -215,5 +239,26 @@ export function useSpeech({ onFinal, onInterim }: UseSpeechOptions): Speech {useEffect(() => () => stop(), [stop]);- return { supported, listening, error, start, stop };+ useEffect(+ () =>+ watchMicPermission((state) => {+ if (state === 'prompt') return;+ micGrant.current = null;+ if (state === 'denied') {+ setNeedsGrant(true);+ setError(MIC_BLOCKED);+ return;+ }+ setNeedsGrant(false);+ setError(null);+ if (blocked.current) wanted.current = true;+ blocked.current = false;+ if (wanted.current) beginRef.current();+ }),+ [],+ );++ const grant = useCallback(() => void openMicPermissionPage(), []);++ return { supported, listening, error, needsGrant, grant, start, stop };}diff --git a/src/lib/bridge/use-voice-composer.ts b/src/lib/bridge/use-voice-composer.tsindex 0f8c4eb..d6ab235 100644--- a/src/lib/bridge/use-voice-composer.ts+++ b/src/lib/bridge/use-voice-composer.ts@@ -10,6 +10,8 @@ export interface VoiceComposer {listening: boolean;supported: boolean;error: string | null;+ needsGrant: boolean;+ grant: () => void;pendingSend: boolean;autoSendMs: number;cancelPending: () => void;@@ -99,6 +101,8 @@ export function useVoiceComposer(opts: {listening: speech.listening,supported: speech.supported,error: speech.error,+ needsGrant: speech.needsGrant,+ grant: speech.grant,pendingSend,autoSendMs,cancelPending,diff --git a/wxt.config.ts b/wxt.config.tsindex 0edd7c6..977951a 100644--- a/wxt.config.ts+++ b/wxt.config.ts@@ -20,16 +20,12 @@ export default defineConfig({vite: () => ({plugins: [tailwindcss()],}),- manifest: ({ browser }) => ({+ manifest: () => ({name: 'Browsentic',description: 'Reimagine browsing as agentic — driven by the AI agent you already run, in your own logged-in browser.',permissions: ['storage', 'unlimitedStorage', 'activeTab', 'sidePanel', 'contextMenus', 'alarms', 'scripting','notifications', 'debugger', 'downloads',- // Chrome grants the microphone at install time or not at all: a permission bubble has nowhere to- // anchor in a popup or a side panel, so getUserMedia there is refused without ever asking. Firefox- // has no such permission and prompts from the sidebar on its own.- ...(browser === 'firefox' ? [] : ['audioCapture']),],host_permissions: ['<all_urls>'],}),