Skip to content

Self-improving AI Employees for everyone

fix(workspace): let browser and phone views fill the normal dock

trohitg09686e7c68aa
diff --git a/client/src/components/browser/BrowserWorkspace.tsx b/client/src/components/browser/BrowserWorkspace.tsx
index 7fd302bf..9368e271 100644
--- a/client/src/components/browser/BrowserWorkspace.tsx
+++ b/client/src/components/browser/BrowserWorkspace.tsx
@@ -30,7 +30,7 @@ export default function BrowserWorkspace({ workflowId, nodes, visible = true }:
return (
<div className="flex min-h-0 min-w-0 flex-1 flex-col gap-2">
{nodes.length > 1 && (
- <select aria-label="Browser node" className={inputClass} value={nodeId} onChange={(e) => setSelected(e.target.value)}>
+ <select aria-label="Browser node" className={`${inputClass} mx-2 mt-2 shrink-0`} value={nodeId} onChange={(e) => setSelected(e.target.value)}>
{nodes.map((node) => <option key={node.node_id} value={node.node_id}>{node.label || 'Browser'}</option>)}
</select>
)}
@@ -314,28 +314,28 @@ function BrowserSessionView({ workflowId, nodeId, visible }: { workflowId: strin
const statusLabel = phase === 'connecting' ? 'Connecting…' : phase === 'idle' ? 'Browser is stopped' : phase === 'error' ? 'Disconnected' : control ? 'You have control' : state.challenge_required ? 'Paused for site verification' : state.state === 'agent' ? 'Employee is browsing' : state.state === 'awaiting_user' ? 'Your help is needed' : state.controller === 'other' ? 'Another viewer has control' : 'Live browser';
return (
- <FullView label="Browser">
- <div className="flex flex-wrap items-center justify-between gap-2 text-xs">
- <span role="status" className="text-fg-muted">{installing ? 'Installing browser…' : starting ? 'Starting browser…' : statusLabel}</span>
+ <FullView label="Browser" toolbar={<>
+ <span role="status" className="min-w-0 flex-1 truncate text-xs text-fg-muted" title={statusLabel}>{installing ? 'Installing browser…' : starting ? 'Starting browser…' : statusLabel}</span>
{phase === 'live' && <Button size="sm" variant="outline" disabled={takingControl} onClick={() => {
setError('');
if (control) send({ type: 'control_release' });
else { setTakingControl(true); send({ type: 'control_request', ...(state.controller === 'other' ? { force: true } : {}) }); }
}}>{control ? (state.challenge_required ? 'Hand back and resume' : 'Hand back') : takingControl ? 'Taking control…' : state.controller === 'other' ? 'Take over here' : 'Take control'}</Button>}
- </div>
- {state.request && <div className="rounded border border-border-default p-2 text-sm text-fg-default">{state.request.message || state.request.reason || 'The employee needs your help.'}</div>}
- {error && <div role="alert" className="flex items-start gap-2 text-xs text-destructive"><span className="min-w-0 flex-1 break-words">{error}</span><button aria-label="Dismiss browser error" onClick={() => setError('')}>×</button></div>}
+ </>} controls={<>
<form className="flex items-center gap-1" onSubmit={(e) => { e.preventDefault(); if (control) navigate('goto'); }}>
<Button type="button" size="icon-sm" variant="ghost" aria-label="Back" disabled={!control} onClick={() => navigate('back')}><ArrowLeft className="size-4" /></Button>
<Button type="button" size="icon-sm" variant="ghost" aria-label="Forward" disabled={!control} onClick={() => navigate('forward')}><ArrowRight className="size-4" /></Button>
<Button type="button" size="icon-sm" variant="ghost" aria-label="Reload page" disabled={!control} onClick={() => navigate('reload')}><RotateCw className="size-4" /></Button>
<input aria-label="Browser address" className={`${inputClass} flex-1`} value={address} disabled={!control} placeholder="Enter a website" onChange={(e) => setAddress(e.target.value)} />
</form>
- {tabs.length > 1 && <select aria-label="Browser tab" className={inputClass} disabled={!control} value={tabs.find((tab) => tab.active)?.target_id || ''} onChange={(e) => send({ type: 'tab', action: 'activate', target_id: e.target.value })}>
+ {tabs.length > 1 && <select aria-label="Browser tab" className={`${inputClass} w-full`} disabled={!control} value={tabs.find((tab) => tab.active)?.target_id || ''} onChange={(e) => send({ type: 'tab', action: 'activate', target_id: e.target.value })}>
{tabs.map((tab) => <option key={tab.target_id} value={tab.target_id}>{tab.title || tab.url || 'New tab'}</option>)}
</select>}
+ </>}>
+ {state.request && <div className="max-h-[25%] shrink-0 overflow-y-auto border-b border-border-default p-2 text-sm text-fg-default">{state.request.message || state.request.reason || 'The employee needs your help.'}</div>}
+ {error && <div role="alert" className="flex max-h-[25%] shrink-0 items-start gap-2 overflow-y-auto p-2 text-xs text-destructive"><span className="min-w-0 flex-1 break-words">{error}</span><button aria-label="Dismiss browser error" onClick={() => setError('')}>×</button></div>}
<div ref={surfaceRef} role="group" aria-label="Live browser page" tabIndex={control ? 0 : -1}
- className="relative flex min-h-40 flex-1 items-center justify-center overflow-hidden rounded border border-border-default bg-bg-app outline-none focus-visible:ring-2 focus-visible:ring-ring"
+ className="relative flex min-h-0 min-w-0 flex-1 items-center justify-center overflow-hidden bg-bg-app outline-none focus-visible:ring-2 focus-visible:ring-inset focus-visible:ring-ring"
style={{ touchAction: control ? 'none' : 'auto' }}
onPointerDown={(e) => pointer(e, 'down')} onPointerMove={(e) => pointer(e, 'move')} onPointerUp={(e) => pointer(e, 'up')} onPointerCancel={releaseControl}
onContextMenu={(e) => { if (control) e.preventDefault(); }} onKeyDown={(e) => key(e, 'down')} onKeyUp={(e) => key(e, 'up')}
@@ -348,7 +348,7 @@ function BrowserSessionView({ workflowId, nodeId, visible }: { workflowId: strin
{phase === 'error' && <Button variant="outline" onClick={() => setAttempt((n) => n + 1)}>Reconnect</Button>}
</div>}
</div>
- {dialog && <div role="dialog" aria-label="Browser page dialog" className="flex flex-col gap-2 rounded border border-border-default bg-bg-panel p-3 text-sm">
+ {dialog && <div role="dialog" aria-label="Browser page dialog" className="flex max-h-[40%] shrink-0 flex-col gap-2 overflow-y-auto border-t border-border-default bg-bg-panel p-3 text-sm">
<p className="break-words text-fg-default">{dialog.message}</p>
{dialog.kind === 'prompt' && <input aria-label="Dialog response" className={inputClass} value={promptText} onChange={(e) => setPromptText(e.target.value)} disabled={!control} />}
<div className="flex gap-2">{[true, false].map((accept) => <Button key={String(accept)} size="sm" variant="outline" disabled={!control} onClick={() => { send({ type: 'dialog_reply', accept, prompt_text: promptText }); setDialog(null); }}>{accept ? 'OK' : 'Cancel'}</Button>)}</div>
diff --git a/client/src/components/browser/__tests__/BrowserWorkspace.test.tsx b/client/src/components/browser/__tests__/BrowserWorkspace.test.tsx
index 9b6ef34d..f6230920 100644
--- a/client/src/components/browser/__tests__/BrowserWorkspace.test.tsx
+++ b/client/src/components/browser/__tests__/BrowserWorkspace.test.tsx
@@ -81,6 +81,7 @@ describe('BrowserWorkspace session lifecycle', () => {
const { unmount } = render(<BrowserWorkspace workflowId="wf" nodes={nodes} />);
const socket = MockSocket.instances[0];
act(() => { socket.open(); socket.message({ type: 'state', state: 'agent', controller: null }); });
+ fireEvent.click(screen.getByRole('button', { name: 'Browser controls' }));
expect(screen.getByRole('textbox', { name: 'Browser address' })).toBeDisabled();
fireEvent.click(screen.getByRole('button', { name: 'Take control' }));
expect(socket.sent.at(-1)).toEqual({ type: 'control_request' });
diff --git a/client/src/components/mobile/MobileWorkspace.tsx b/client/src/components/mobile/MobileWorkspace.tsx
index 9805fe1c..1f8e67b4 100644
--- a/client/src/components/mobile/MobileWorkspace.tsx
+++ b/client/src/components/mobile/MobileWorkspace.tsx
@@ -18,8 +18,8 @@ export default function MobileWorkspace({ workflowId, nodes, visible = true }: {
const nodeId = nodes.some((node) => node.node_id === selected) ? selected : nodes[0]?.node_id;
if (!workflowId || workflowId === 'new') return <Empty message="Save this workflow before opening its mobile workspace." />;
if (!nodeId) return <Empty message="Add a Mobile Agent or Android tool to this workflow to use its phone here." />;
- return <div className="flex min-h-0 flex-1 flex-col gap-2">
- {nodes.length > 1 && <select className={fieldClass} aria-label="Mobile agent" value={nodeId} onChange={(event) => setSelected(event.target.value)}>
+ return <div className="flex min-h-0 min-w-0 flex-1 flex-col gap-2">
+ {nodes.length > 1 && <select className={`${fieldClass} mx-2 mt-2 shrink-0`} aria-label="Mobile agent" value={nodeId} onChange={(event) => setSelected(event.target.value)}>
{nodes.map((node) => <option key={node.node_id} value={node.node_id}>{node.label}</option>)}
</select>}
<MobileSession key={`${workflowId}:${nodeId}`} workflowId={workflowId} nodeId={nodeId} visible={visible} />
@@ -182,56 +182,27 @@ function MobileSession({ workflowId, nodeId, visible }: { workflowId: string; no
const installed = doctor?.adb && doctor?.emulator && doctor?.image && doctor?.engine && doctor?.video;
const setupActive = typeof status?.setup === 'string' && status.setup.startsWith('installing_');
const setupFailed = status?.setup === 'error' || status?.setup === 'interrupted' || !!status?.setup_error;
- return <FullView label="Phone"><div className="flex min-h-0 flex-1 flex-col gap-2 overflow-y-auto group-data-[full-view=true]/fullview:overflow-hidden">
- <div className="flex flex-wrap items-center gap-2">
- <Smartphone aria-hidden className="size-4" />
- <span className="flex-1 text-sm font-medium">Android phone</span>
- <span role="status" className="text-xs text-fg-muted">{setupActive ? 'Setting up' : status?.starting ? 'Starting phone…' : busy ? `${busy}…` : running ? (held ? 'You’re using the phone' : status.active?.status === 'running' ? 'AI is working' : status.control_state === 'recovering' ? 'Reconnecting…' : 'Ready') : status ? 'Phone is off' : 'Connecting…'}</span>
- <Button size="icon-sm" variant="ghost" aria-label="Refresh mobile status" onClick={() => void perform('refresh', async () => { setDoctor(await mobileRequest(`${path}/doctor`)); })}><RotateCw /></Button>
- </div>
- {connectionError && <p role="status" className="m-0 text-sm text-fg-muted">{connectionError}</p>}
- {error && <p role="alert" className="m-0 rounded border border-destructive/30 bg-destructive/10 p-2 text-sm text-destructive"><span>{error}</span><button type="button" aria-label="Dismiss message" className="ml-3 underline" onClick={() => setError('')}>Dismiss</button></p>}
- {setupFailed && <p role="alert" className="m-0 text-sm text-destructive">{status?.setup_error || (status?.setup === 'interrupted' ? 'Setup was interrupted. Retry to finish preparing the phone.' : 'Setup failed. Review the details below and retry.')}</p>}
- {doctor?.supported === false && <p className="m-0 text-sm text-fg-muted">This host cannot run the mobile runtime. Android setup currently requires Windows x64 with virtualization available.</p>}
- {((doctor && !installed && !running) || setupActive || setupFailed) && <section aria-label="Mobile setup" className="space-y-2 rounded border border-border-default bg-bg-panel p-3">
- <h3 className="m-0 text-sm font-semibold">Set up your Android phone</h3>
- <p className="m-0 text-xs text-fg-muted">One-time setup downloads everything your phone needs. This may take several minutes. Your apps and sign-ins are saved.</p>
- <details><summary className="cursor-pointer text-xs text-fg-muted">Download details</summary><ul className="m-0 list-none space-y-1 p-0 text-xs text-fg-muted">{(['adb', 'emulator', 'image', 'engine', 'video'] as const).map((key) => <li key={key}>{key === 'adb' ? 'Android tools' : key === 'engine' ? 'Agent runtime' : key === 'video' ? 'Live video' : key === 'image' ? 'Android image' : 'Emulator'}: {doctor?.[key] ? 'Ready' : 'Needed'}</li>)}</ul></details>
- <label className="flex items-start gap-2 text-xs"><input type="checkbox" checked={accepted} onChange={(event) => setAccepted(event.target.checked)} aria-label="Accept Android SDK license terms" />
- <span>I accept the <a className="underline" href="https://developer.android.com/studio/terms" target="_blank" rel="noreferrer">Android SDK license terms</a> and want to download the required components.</span></label>
- <Button size="sm" disabled={!accepted || !!busy || setupActive || doctor?.supported === false} onClick={() => void perform('setup', async () => {
- await mobileRequest(`${path}/setup`, { licenses_accepted: true }); setDoctor(await mobileRequest(`${path}/doctor`));
- })}>{setupFailed ? 'Retry setup' : 'Set up phone'}</Button>
- {(setupActive || setupFailed || status?.setup_progress) && <SetupProgress status={status} ticking={visible && pageVisible} />}
- </section>}
- {status?.setup === 'ready' && installed && !setupFailed && status.setup_progress && <details aria-label="Completed mobile setup" className="group-data-[full-view=true]/fullview:hidden rounded border border-border-default p-3 text-xs text-fg-muted"><summary className="cursor-pointer">Setup finished · View details</summary>
- <SetupProgress status={status} ticking={visible && pageVisible} />
- </details>}
- {!running && !status?.starting && installed && <p className="m-0 text-sm text-fg-muted">Your phone is ready to turn on. Click Start phone to use it.</p>}
- <div className="flex flex-wrap gap-2">
- <Button size="sm" variant="outline" disabled={!!busy || setupActive || status?.starting || !status || doctor?.supported === false || (!running && !installed)} onClick={() => void perform(running ? 'stopping' : 'starting', async () => { await mobileRequest(`${path}/${running ? 'stop' : 'start'}`, {}); setEpoch(null); })}>
- {running ? <Square className="size-3.5" /> : <Play className="size-3.5" />}{running ? 'Stop phone' : 'Start phone'}
- </Button>
+ const powerControl = (
+ <Button size="sm" variant="outline" disabled={!!busy || setupActive || status?.starting || !status || doctor?.supported === false || (!running && !installed)} onClick={() => void perform(running ? 'stopping' : 'starting', async () => { await mobileRequest(`${path}/${running ? 'stop' : 'start'}`, {}); setEpoch(null); })}>
+ {running ? <Square className="size-3.5" /> : <Play className="size-3.5" />}{running ? 'Stop phone' : 'Start phone'}
+ </Button>
+ );
+ return <FullView label="Phone" toolbar={<>
+ <Smartphone aria-hidden className="size-4 shrink-0 text-fg-muted" />
+ <span role="status" className="min-w-0 flex-1 truncate text-xs text-fg-muted">{setupActive ? 'Setting up' : status?.starting ? 'Starting phone…' : busy ? `${busy}…` : running ? (held ? 'You’re using the phone' : status.active?.status === 'running' ? 'AI is working' : status.control_state === 'recovering' ? 'Reconnecting…' : 'Ready') : status ? 'Phone is off' : 'Connecting…'}</span>
+ {!running && powerControl}
{running && (held ? <>
<Button size="sm" onClick={() => void perform('releasing', async () => { await mobileRequest(`${path}/release`, { viewer_id: viewerId, epoch, resume: true }); leaseEpoch.current = null; setEpoch(null); })} disabled={!!busy}>Let AI continue</Button>
<Button size="sm" variant="outline" onClick={() => void perform('releasing', async () => { await mobileRequest(`${path}/release`, { viewer_id: viewerId, epoch, resume: false }); leaseEpoch.current = null; setEpoch(null); })} disabled={!!busy}>Finish using phone</Button>
</> : <Button size="sm" variant="outline" disabled={!!busy} onClick={() => void perform('taking control', async () => { const claim = await mobileRequest<{ epoch: number; owner: string }>(`${path}/takeover`, { viewer_id: viewerId });
if (!active.current || !viewActive.current) { await mobileRequest(`${path}/release`, { viewer_id: viewerId, epoch: claim.epoch, resume: false }); return; }
leaseEpoch.current = claim.epoch; setEpoch(claim.epoch); setLeaseOwner(claim.owner); })}>Use phone</Button>)}
- </div>
- {running && <>
- <p className="m-0 text-xs text-fg-muted">{held ? 'Tap or swipe the screen, just like a phone. Finish when you want the AI to use it.' : 'Watch your phone here. Choose Use phone to tap and type yourself.'}</p>
- <div ref={surface} className={`relative flex h-[min(60vh,680px)] min-h-80 shrink-0 group-data-[full-view=true]/fullview:h-auto group-data-[full-view=true]/fullview:min-h-0 group-data-[full-view=true]/fullview:flex-1 touch-none items-center justify-center overflow-hidden rounded border border-border-default bg-slate-950 ${held ? 'cursor-crosshair' : ''}`} onPointerDown={down} onPointerUp={up} onPointerCancel={() => { pointer.current = null; }} aria-label={held ? 'Phone screen: tap or drag to interact' : 'Phone screen, view only'}>
- <canvas ref={canvas} className="absolute h-full w-full object-contain" />
- {!live && !videoError && <p className="pointer-events-none z-10 text-sm text-fg-muted">Connecting live view…</p>}
+ </>} controls={<>
+ <div className="flex flex-wrap items-center gap-2">
+ {running && powerControl}
+ <Button size="icon-sm" variant="ghost" aria-label="Refresh mobile status" onClick={() => void perform('refresh', async () => { setDoctor(await mobileRequest(`${path}/doctor`)); })}><RotateCw /></Button>
</div>
- {videoError && <div className="flex items-center gap-2"><p role="alert" className="m-0 flex-1 text-xs text-fg-muted">{videoError}</p><Button size="sm" variant="outline" onClick={() => setVideoAttempt((value) => value + 1)}>Reconnect</Button></div>}
- {held && <div className="flex flex-wrap gap-1">
- <Button size="icon-sm" variant="outline" aria-label="Phone Back" disabled={!!busy} onClick={() => void perform('input', () => input('key', { key: 'back' }))}><ArrowLeft /></Button>
- <Button size="icon-sm" variant="outline" aria-label="Phone Home" disabled={!!busy} onClick={() => void perform('input', () => input('key', { key: 'home' }))}><Home /></Button>
- <input className={`${fieldClass} flex-1`} aria-label="Text to type on phone" value={text} onChange={(event) => setText(event.target.value)} placeholder="Type on phone" />
- <Button size="sm" variant="outline" disabled={!text || !!busy} onClick={() => void perform('input', async () => { await input('text', { text }); setText(''); })}>Type</Button>
- </div>}
+ <p className="m-0 text-xs text-fg-muted">{held ? 'Tap or swipe the screen, just like a phone. Finish when you want the AI to use it.' : 'Watch your phone here. Choose Use phone to tap and type yourself.'}</p>
{held && <label className="flex items-center gap-2 text-xs text-fg-muted">Install an app file (.apk, up to 256 MB)
<input type="file" accept=".apk" aria-label="Install APK" disabled={!!busy} className="min-w-0 flex-1 text-xs" onChange={(event) => {
const file = event.target.files?.[0]; event.target.value = '';
@@ -244,25 +215,65 @@ function MobileSession({ workflowId, nodeId, visible }: { workflowId: string; no
});
}} />
</label>}
- </>}
- <form className="mt-auto flex shrink-0 flex-col group-data-[full-view=true]/fullview:hidden gap-2 border-t border-border-default pt-3" onSubmit={(event) => { event.preventDefault(); void submit(); }}>
- <label htmlFor={`task-${nodeId}`} className="text-sm font-medium">Ask AI to use the phone</label>
- <textarea id={`task-${nodeId}`} className={`${fieldClass} resize-y`} rows={2} maxLength={12000} value={prompt} onChange={(event) => setPrompt(event.target.value)} placeholder="For example: Open Settings and turn on dark mode" />
- <p className="m-0 text-xs text-fg-muted">Uses your global AI model unless this phone has a model override. Only one person or AI can use the phone at a time.</p>
- <div className="flex items-center gap-2"><Button size="sm" type="submit" disabled={!running || !prompt.trim() || !!busy || hasTask}>Run task</Button>
- {hasTask && <Button size="sm" type="button" variant="outline" disabled={!!busy} onClick={() => void perform('cancelling', async () => { setTask(await mobileRequest(`${path}/tasks/${encodeURIComponent(submission!)}/cancel`, {})); })}>Cancel task</Button>}
- {task && <span role="status" className="text-xs text-fg-muted">{({queued: 'Waiting for the phone', running: 'Working on your request', completed: 'Done', failed: 'Could not finish', cancelled: 'Cancelled', awaiting_user: 'Waiting for you'})[taskState(task)] || 'Request received'}</span>}
- </div>
- {task?.error != null && <p role="alert" className="m-0 break-words text-sm text-destructive">{describe(task.error)}</p>}
- {task?.result != null && <pre className="m-0 max-h-48 overflow-auto whitespace-pre-wrap rounded bg-bg-panel p-2 text-xs">{describe(task.result)}</pre>}
- </form>
- {doctor?.acceleration && <details className="group-data-[full-view=true]/fullview:hidden rounded border border-border-default px-3 py-2 text-xs text-fg-muted">
- <summary className="cursor-pointer font-medium">Help &amp; diagnostics</summary>
- <p className="mb-0 mt-2">If the phone will not start, share the details below when asking for help.</p><pre className="mb-0 mt-2 whitespace-pre-wrap break-words">{doctor.acceleration}</pre>
- <p className="mb-0 mt-2">If acceleration is unavailable on Windows, enable virtualization in your computer’s firmware and Windows Hypervisor Platform in Windows Features, then restart Windows and refresh mobile status.</p>
- {!!status?.diagnostics?.length && <details className="mt-2"><summary className="cursor-pointer">Recent phone activity</summary><pre className="max-h-40 overflow-auto whitespace-pre-wrap">{status.diagnostics.map((item) => `${item.at} ${item.level} ${item.event}${item.operation ? ` (${item.operation})` : ''}${item.error_type ? `: ${item.error_type}` : ''}${item.code ? ` [${item.code}]` : ''}`).join('\n')}</pre></details>}
- </details>}
- </div></FullView>;
+ {status?.setup === 'ready' && installed && !setupFailed && status.setup_progress && <details aria-label="Completed mobile setup" className="rounded border border-border-default p-3 text-xs text-fg-muted"><summary className="cursor-pointer">Setup finished · View details</summary>
+ <SetupProgress status={status} ticking={visible && pageVisible} />
+ </details>}
+ {doctor?.acceleration && <details className="rounded border border-border-default px-3 py-2 text-xs text-fg-muted">
+ <summary className="cursor-pointer font-medium">Help &amp; diagnostics</summary>
+ <p className="mb-0 mt-2">If the phone will not start, share the details below when asking for help.</p><pre className="mb-0 mt-2 whitespace-pre-wrap break-words">{doctor.acceleration}</pre>
+ <p className="mb-0 mt-2">If acceleration is unavailable on Windows, enable virtualization in your computer’s firmware and Windows Hypervisor Platform in Windows Features, then restart Windows and refresh mobile status.</p>
+ {!!status?.diagnostics?.length && <details className="mt-2"><summary className="cursor-pointer">Recent phone activity</summary><pre className="max-h-40 overflow-auto whitespace-pre-wrap">{status.diagnostics.map((item) => `${item.at} ${item.level} ${item.event}${item.operation ? ` (${item.operation})` : ''}${item.error_type ? `: ${item.error_type}` : ''}${item.code ? ` [${item.code}]` : ''}`).join('\n')}</pre></details>}
+ </details>}
+ </>}>
+ <div className="flex min-h-0 min-w-0 flex-1 flex-col overflow-hidden">
+ {(connectionError || error || setupFailed || doctor?.supported === false || !running || setupActive) && <div className={`space-y-2 overflow-y-auto p-2 ${running ? 'max-h-[40%] shrink-0' : 'min-h-0 flex-1'}`}>
+ {connectionError && <p role="status" className="m-0 text-sm text-fg-muted">{connectionError}</p>}
+ {error && <p role="alert" className="m-0 rounded border border-destructive/30 bg-destructive/10 p-2 text-sm text-destructive"><span>{error}</span><button type="button" aria-label="Dismiss message" className="ml-3 underline" onClick={() => setError('')}>Dismiss</button></p>}
+ {setupFailed && <p role="alert" className="m-0 text-sm text-destructive">{status?.setup_error || (status?.setup === 'interrupted' ? 'Setup was interrupted. Retry to finish preparing the phone.' : 'Setup failed. Review the details below and retry.')}</p>}
+ {doctor?.supported === false && <p className="m-0 text-sm text-fg-muted">This host cannot run the mobile runtime. Android setup currently requires Windows x64 with virtualization available.</p>}
+ {((doctor && !installed && !running) || setupActive || setupFailed) && <section aria-label="Mobile setup" className="space-y-2 rounded border border-border-default bg-bg-panel p-3">
+ <h3 className="m-0 text-sm font-semibold">Set up your Android phone</h3>
+ <p className="m-0 text-xs text-fg-muted">One-time setup downloads everything your phone needs. This may take several minutes. Your apps and sign-ins are saved.</p>
+ <details><summary className="cursor-pointer text-xs text-fg-muted">Download details</summary><ul className="m-0 list-none space-y-1 p-0 text-xs text-fg-muted">{(['adb', 'emulator', 'image', 'engine', 'video'] as const).map((key) => <li key={key}>{key === 'adb' ? 'Android tools' : key === 'engine' ? 'Agent runtime' : key === 'video' ? 'Live video' : key === 'image' ? 'Android image' : 'Emulator'}: {doctor?.[key] ? 'Ready' : 'Needed'}</li>)}</ul></details>
+ <label className="flex items-start gap-2 text-xs"><input type="checkbox" checked={accepted} onChange={(event) => setAccepted(event.target.checked)} aria-label="Accept Android SDK license terms" />
+ <span>I accept the <a className="underline" href="https://developer.android.com/studio/terms" target="_blank" rel="noreferrer">Android SDK license terms</a> and want to download the required components.</span></label>
+ <Button size="sm" disabled={!accepted || !!busy || setupActive || doctor?.supported === false} onClick={() => void perform('setup', async () => {
+ await mobileRequest(`${path}/setup`, { licenses_accepted: true }); setDoctor(await mobileRequest(`${path}/doctor`));
+ })}>{setupFailed ? 'Retry setup' : 'Set up phone'}</Button>
+ {(setupActive || setupFailed || status?.setup_progress) && <SetupProgress status={status} ticking={visible && pageVisible} />}
+ </section>}
+ {!running && !status?.starting && installed && <p className="m-0 text-sm text-fg-muted">Your phone is ready to turn on. Click Start phone to use it.</p>}
+ </div>}
+ {running && <>
+ <div ref={surface} className={`relative flex min-h-0 min-w-0 flex-1 touch-none items-center justify-center overflow-hidden bg-slate-950 ${held ? 'cursor-crosshair' : ''}`} onPointerDown={down} onPointerUp={up} onPointerCancel={() => { pointer.current = null; }} aria-label={held ? 'Phone screen: tap or drag to interact' : 'Phone screen, view only'}>
+ <canvas ref={canvas} className="absolute inset-0 h-full w-full object-contain" />
+ {!live && !videoError && <p className="pointer-events-none z-10 text-sm text-fg-muted">Connecting live view…</p>}
+ </div>
+ {videoError && <div className="flex shrink-0 items-center gap-2 p-2"><p role="alert" className="m-0 flex-1 text-xs text-fg-muted">{videoError}</p><Button size="sm" variant="outline" onClick={() => setVideoAttempt((value) => value + 1)}>Reconnect</Button></div>}
+ {held && <div className="flex shrink-0 flex-wrap gap-1 border-t border-border-default p-2">
+ <Button size="icon-sm" variant="outline" aria-label="Phone Back" disabled={!!busy} onClick={() => void perform('input', () => input('key', { key: 'back' }))}><ArrowLeft /></Button>
+ <Button size="icon-sm" variant="outline" aria-label="Phone Home" disabled={!!busy} onClick={() => void perform('input', () => input('key', { key: 'home' }))}><Home /></Button>
+ <input className={`${fieldClass} flex-1`} aria-label="Text to type on phone" value={text} onChange={(event) => setText(event.target.value)} placeholder="Type on phone" />
+ <Button size="sm" variant="outline" disabled={!text || !!busy} onClick={() => void perform('input', async () => { await input('text', { text }); setText(''); })}>Type</Button>
+ </div>}
+ </>}
+ <details aria-label="Phone AI task" className="max-h-[45%] shrink-0 overflow-y-auto border-t border-border-default group-data-[full-view=true]/fullview:hidden">
+ <summary className="cursor-pointer px-2 py-2 text-xs font-medium outline-none focus-visible:ring-2 focus-visible:ring-inset focus-visible:ring-ring">Ask AI to use the phone
+ {task && <span role="status" className="ml-2 font-normal text-fg-muted">{({queued: 'Waiting for the phone', running: 'Working on your request', completed: 'Done', failed: 'Could not finish', cancelled: 'Cancelled', awaiting_user: 'Waiting for you'})[taskState(task)] || 'Request received'}</span>}
+ </summary>
+ <form className="flex flex-col gap-2 p-2" onSubmit={(event) => { event.preventDefault(); void submit(); }}>
+ <label htmlFor={`task-${nodeId}`} className="sr-only">Ask AI to use the phone</label>
+ <textarea id={`task-${nodeId}`} className={`${fieldClass} resize-none`} rows={2} maxLength={12000} value={prompt} onChange={(event) => setPrompt(event.target.value)} placeholder="For example: Open Settings and turn on dark mode" />
+ <p className="m-0 text-xs text-fg-muted">Uses your global AI model unless this phone has a model override. Only one person or AI can use the phone at a time.</p>
+ <div className="flex items-center gap-2"><Button size="sm" type="submit" disabled={!running || !prompt.trim() || !!busy || hasTask}>Run task</Button>
+ {hasTask && <Button size="sm" type="button" variant="outline" disabled={!!busy} onClick={() => void perform('cancelling', async () => { setTask(await mobileRequest(`${path}/tasks/${encodeURIComponent(submission!)}/cancel`, {})); })}>Cancel task</Button>}
+ </div>
+ {task?.error != null && <p role="alert" className="m-0 break-words text-sm text-destructive">{describe(task.error)}</p>}
+ {task?.result != null && <pre className="m-0 max-h-48 overflow-auto whitespace-pre-wrap rounded bg-bg-panel p-2 text-xs">{describe(task.result)}</pre>}
+ </form>
+ </details>
+ </div>
+ </FullView>;
}
function SetupProgress({ status, ticking }: { status: MobileStatus | null; ticking: boolean }) {
diff --git a/client/src/components/mobile/__tests__/MobileWorkspace.test.tsx b/client/src/components/mobile/__tests__/MobileWorkspace.test.tsx
index 2bf81806..191534d9 100644
--- a/client/src/components/mobile/__tests__/MobileWorkspace.test.tsx
+++ b/client/src/components/mobile/__tests__/MobileWorkspace.test.tsx
@@ -4,7 +4,8 @@ import userEvent from '@testing-library/user-event';
import MobileWorkspace from '../MobileWorkspace';
import { mobilePoint, mobileRequest } from '../api';
-vi.mock('../video', () => ({ connectMobileVideo: () => () => {} }));
+const { connectVideo } = vi.hoisted(() => ({ connectVideo: vi.fn(() => () => {}) }));
+vi.mock('../video', () => ({ connectMobileVideo: connectVideo }));
const fetchMock = vi.fn();
const nodes = [{ node_id: 'flow:mobile_agent:1', label: 'Phone assistant' }];
const ready = { supported: true, adb: true, emulator: true, image: true, engine: true, video: true, acceleration: 'WHPX is installed and usable.' };
@@ -13,6 +14,7 @@ const json = (body: unknown, status = 200) => new Response(JSON.stringify(body),
beforeEach(() => {
sessionStorage.clear();
+ connectVideo.mockClear();
snapshot = { running: false, control_state: 'idle', controller: null, epoch: 0, geometry: { width: 1080, height: 1920, rotation: 0 } };
vi.stubGlobal('fetch', fetchMock);
fetchMock.mockReset().mockImplementation(async (url: string) => json(url.endsWith('/doctor') ? ready : snapshot));
@@ -20,6 +22,25 @@ beforeEach(() => {
afterEach(() => { cleanup(); vi.unstubAllGlobals(); });
describe('Mobile Workspace', () => {
+ it('keeps the phone connected and preserves a task draft when secondary panels collapse', async () => {
+ snapshot.running = true;
+ render(<MobileWorkspace workflowId="flow" nodes={nodes} />);
+ await screen.findByText('Ready');
+ await waitFor(() => expect(connectVideo).toHaveBeenCalledOnce());
+ const canvas = screen.getByLabelText('Phone screen, view only').querySelector('canvas');
+ const summary = screen.getByText('Ask AI to use the phone', { selector: 'summary' });
+ expect(screen.getByRole('textbox', { name: 'Ask AI to use the phone' })).not.toBeVisible();
+ await userEvent.click(summary);
+ await userEvent.type(screen.getByRole('textbox', { name: 'Ask AI to use the phone' }), 'Open Settings');
+ await userEvent.click(summary);
+ await userEvent.click(screen.getByRole('button', { name: 'Phone controls' }));
+ expect(screen.getByRole('button', { name: 'Stop phone' })).toBeVisible();
+ await userEvent.click(screen.getByRole('button', { name: 'Phone controls' }));
+ await userEvent.click(summary);
+ expect(screen.getByRole('textbox', { name: 'Ask AI to use the phone' })).toHaveValue('Open Settings');
+ expect(screen.getByLabelText('Phone screen, view only').querySelector('canvas')).toBe(canvas);
+ expect(connectVideo).toHaveBeenCalledOnce();
+ });
it('clears a failed status check when the phone reconnects', async () => {
let polls = 0;
snapshot.running = true;
@@ -37,6 +58,7 @@ describe('Mobile Workspace', () => {
it('explains how to start and disables AI tasks while the phone is off', async () => {
render(<MobileWorkspace workflowId="flow" nodes={nodes} />);
await screen.findByText('Phone is off');
+ await userEvent.click(screen.getByText('Ask AI to use the phone', { selector: 'summary' }));
await userEvent.type(screen.getByRole('textbox', { name: 'Ask AI to use the phone' }), 'Open Settings');
expect(screen.getByRole('button', { name: 'Run task' })).toBeDisabled();
});
@@ -52,6 +74,7 @@ describe('Mobile Workspace', () => {
render(<MobileWorkspace workflowId="flow" nodes={nodes} />);
await screen.findByRole('button', { name: 'Start phone' });
await waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(2));
+ await userEvent.click(screen.getByRole('button', { name: 'Phone controls' }));
expect(fetchMock.mock.calls.every(([, options]) => options.method === 'GET')).toBe(true);
expect(screen.getByText('Help & diagnostics')).toBeInTheDocument();
expect(screen.getByText('WHPX is installed and usable.')).toBeInTheDocument();
@@ -117,6 +140,7 @@ describe('Mobile Workspace', () => {
return json(url.endsWith('/doctor') ? ready : snapshot);
});
render(<MobileWorkspace workflowId="home-flow" nodes={nodes} />);
+ await userEvent.click(screen.getByText('Ask AI to use the phone', { selector: 'summary' }));
await userEvent.type(screen.getByRole('textbox', { name: 'Ask AI to use the phone' }), 'Open settings');
await userEvent.click(screen.getByRole('button', { name: 'Run task' }));
await screen.findByText('Connection dropped');
@@ -145,6 +169,7 @@ describe('Mobile Workspace', () => {
await waitFor(() => expect(fetchMock.mock.calls.some(([url]) => url.endsWith('/input'))).toBe(true));
const request = fetchMock.mock.calls.find(([url]) => url.endsWith('/input'))!;
expect(JSON.parse(request[1].body)).toMatchObject({ epoch: 7, operation: 'key', parameters: { key: 'home' } });
+ await userEvent.click(screen.getByRole('button', { name: 'Phone controls' }));
expect(screen.getByLabelText('Install APK')).toBeInTheDocument();
fireEvent.change(screen.getByLabelText('Install APK'), { target: { files: [new File(['bad'], 'not-an-app.txt')] } });
expect(screen.getByText('Choose an APK file no larger than 256 MB.')).toBeInTheDocument();
diff --git a/client/src/components/parameterPanel/MiddleSection.tsx b/client/src/components/parameterPanel/MiddleSection.tsx
index 7c1be080..69caf904 100644
--- a/client/src/components/parameterPanel/MiddleSection.tsx
+++ b/client/src/components/parameterPanel/MiddleSection.tsx
@@ -620,7 +620,7 @@ const MiddleSection: React.FC<MiddleSectionProps> = ({
>
{/* The node's live browser, above its settings. */}
{isBrowserNode && (
- <div className="mb-4 h-90 overflow-hidden rounded-md border border-border-default">
+ <div className="mb-4 flex h-90 min-h-0 flex-col overflow-hidden rounded-md border border-border-default">
<BrowserWorkspace workflowId={currentWorkflow?.id} nodes={[{ node_id: nodeId, label: 'Browser' }]} />
</div>
)}
diff --git a/client/src/components/workspace/FullView.tsx b/client/src/components/workspace/FullView.tsx
index 9aa72014..5ef332ac 100644
--- a/client/src/components/workspace/FullView.tsx
+++ b/client/src/components/workspace/FullView.tsx
@@ -1,13 +1,15 @@
-import { useEffect, useRef, useState, type ReactNode } from 'react';
-import { Maximize2, Minimize2 } from 'lucide-react';
+import { useEffect, useId, useRef, useState, type ReactNode } from 'react';
+import { Maximize2, Minimize2, SlidersHorizontal } from 'lucide-react';
import { Button } from '@/components/ui/button';
/** Fullscreen the existing viewer; never remount its canvas or connection. */
-export function FullView({ label, children }: { label: string; children: ReactNode }) {
+export function FullView({ label, toolbar, controls, children }: { label: string; toolbar?: ReactNode; controls?: ReactNode; children: ReactNode }) {
const root = useRef<HTMLDivElement>(null);
const button = useRef<HTMLButtonElement>(null);
const [full, setFull] = useState(false);
const [error, setError] = useState('');
+ const [controlsOpen, setControlsOpen] = useState(false);
+ const controlsId = useId();
useEffect(() => {
const element = root.current;
let wasFull = false;
@@ -33,14 +35,18 @@ export function FullView({ label, children }: { label: string; children: ReactNo
setError('Could not open full view. Try again or use the Workspace expand button.');
}
};
- return <div ref={root} data-full-view={full} className={`group/fullview flex min-h-0 min-w-0 flex-1 flex-col gap-2 bg-bg-app ${full ? 'h-screen w-screen overflow-hidden p-3' : ''}`}>
- <div className="flex shrink-0 items-center justify-end gap-2">
- {full && <span className="mr-auto text-sm font-medium">{label} <span className="text-xs font-normal text-fg-muted">· Esc to return</span></span>}
+ return <div ref={root} data-full-view={full} className={`group/fullview flex min-h-0 min-w-0 flex-1 flex-col overflow-hidden bg-bg-app ${full ? 'h-screen w-screen' : ''}`}>
+ <div className="flex shrink-0 flex-wrap items-center justify-end gap-1.5 border-b border-border-default px-2 py-1.5">
+ {toolbar ?? (full && <span className="mr-auto text-sm font-medium">{label} <span className="text-xs font-normal text-fg-muted">· Esc to return</span></span>)}
+ {controls && <Button size="icon-sm" variant="ghost" aria-label={`${label} controls`} title={controlsOpen ? 'Hide controls' : 'Show controls'} aria-expanded={controlsOpen} aria-controls={controlsId} onClick={() => setControlsOpen((open) => !open)}>
+ <SlidersHorizontal className="size-4" />
+ </Button>}
<Button ref={button} size="sm" variant="outline" aria-label={full ? `Exit ${label.toLowerCase()} full view` : `${label} full view`} aria-pressed={full} onClick={() => void toggle()}>
{full ? <Minimize2 className="size-4" /> : <Maximize2 className="size-4" />}{full ? 'Exit full view' : 'Full view'}
</Button>
</div>
- {error && <p role="alert" className="m-0 text-xs text-destructive">{error}</p>}
+ {error && <p role="alert" className="m-0 shrink-0 px-2 py-1 text-xs text-destructive">{error}</p>}
+ {controls && <div id={controlsId} hidden={!controlsOpen} className="max-h-[40%] shrink-0 space-y-2 overflow-y-auto border-b border-border-default p-2">{controls}</div>}
{children}
</div>;
}
diff --git a/client/src/components/workspace/WorkspaceTabs.tsx b/client/src/components/workspace/WorkspaceTabs.tsx
index 16fd6121..94fd4bb7 100644
--- a/client/src/components/workspace/WorkspaceTabs.tsx
+++ b/client/src/components/workspace/WorkspaceTabs.tsx
@@ -14,7 +14,7 @@ export function WorkspaceTabs({ tab, onTabChange, browser, board, android }: {
}) {
return (
<TabsPrimitive.Root value={tab} onValueChange={(value) => onTabChange(value as WorkspaceTab)} className="flex min-h-0 min-w-0 flex-1 flex-col">
- <TabsPrimitive.List aria-label="Workspace views" className="flex h-10 shrink-0 gap-0.5 overflow-x-auto border-b border-border-default px-2">
+ <TabsPrimitive.List aria-label="Workspace views" className="flex h-10 shrink-0 gap-0.5 overflow-x-auto overflow-y-hidden border-b border-border-default px-2">
{([
['browser', 'Browser', Globe],
['board', 'Canvas', PanelsTopLeft],
@@ -28,7 +28,7 @@ export function WorkspaceTabs({ tab, onTabChange, browser, board, android }: {
{([
['browser', browser], ['board', board], ['android', android ?? <AndroidWorkspace />],
] as const).map(([value, content]) => (
- <TabsPrimitive.Content key={value} value={value} className="flex min-h-0 min-w-0 flex-1 flex-col overflow-hidden bg-bg-app p-3 outline-none">
+ <TabsPrimitive.Content key={value} value={value} className={`flex min-h-0 min-w-0 flex-1 flex-col overflow-hidden bg-bg-app outline-none ${value === 'board' ? 'p-3' : ''}`}>
{content}
</TabsPrimitive.Content>
))}
diff --git a/client/src/components/workspace/__tests__/FullView.test.tsx b/client/src/components/workspace/__tests__/FullView.test.tsx
index adf1b9e9..aa24f8b6 100644
--- a/client/src/components/workspace/__tests__/FullView.test.tsx
+++ b/client/src/components/workspace/__tests__/FullView.test.tsx
@@ -40,3 +40,22 @@ it('explains when fullscreen is unavailable without hiding the viewer', async ()
expect(screen.getByRole('alert')).toHaveTextContent('Full view is unavailable');
expect(screen.getByTestId('viewer')).toBeInTheDocument();
});
+
+it('collapses secondary controls without losing their state or remounting the viewer', () => {
+ const mount = vi.fn();
+ function Viewer() { useEffect(() => { mount(); }, []); return <canvas data-testid="viewer" />; }
+ render(<FullView label="Browser" toolbar={<span role="status">Live browser</span>} controls={<input aria-label="Address" defaultValue="example.com" />}><Viewer /></FullView>);
+ const toggle = screen.getByRole('button', { name: 'Browser controls' });
+ const viewer = screen.getByTestId('viewer');
+ expect(toggle).toHaveAttribute('aria-expanded', 'false');
+ expect(screen.queryByRole('textbox', { name: 'Address' })).toBeNull();
+ fireEvent.click(toggle);
+ fireEvent.change(screen.getByRole('textbox', { name: 'Address' }), { target: { value: 'example.org' } });
+ fireEvent.click(toggle);
+ expect(screen.getByRole('status')).toBeVisible();
+ expect(screen.getByRole('button', { name: 'Browser full view' })).toBeVisible();
+ fireEvent.click(toggle);
+ expect(screen.getByRole('textbox', { name: 'Address' })).toHaveValue('example.org');
+ expect(screen.getByTestId('viewer')).toBe(viewer);
+ expect(mount).toHaveBeenCalledTimes(1);
+});
diff --git a/docs-internal/browser_workspace.md b/docs-internal/browser_workspace.md
index 43781cfe..20c5e4ec 100644
--- a/docs-internal/browser_workspace.md
+++ b/docs-internal/browser_workspace.md
@@ -23,13 +23,21 @@ the same viewer above the node's settings.
## Full view
-Click **Full view** above the browser to fill the screen. Navigation, address,
+The normal Browser and Mobile views fill the remaining Workspace height and
+width, without an outer gutter or a dedicated Full view button row. The compact
+toolbar keeps status and takeover visible. Its **Browser controls** toggle
+expands navigation, address and tab selection without reconnecting the stream.
+Mobile has a **Phone controls** toggle and a collapsible AI task panel; the phone
+image keeps its original proportions, so a wide panel can still have side bars.
+
+Click **Full view** in the toolbar to fill the screen. Navigation, address,
control and exit buttons remain outside the page image. **Exit full view** or
**Esc** returns to the previous layout. The same mounted canvas and stream stay
active, so entering or leaving does not reopen the session. The shared
`components/workspace/FullView.tsx` boundary handles native fullscreen changes
and restores focus to its button on exit. Mobile uses the same boundary, hiding
-its task form and technical panels while enlarged. Full view does not grant
+its task form while enlarged; secondary controls stay available through the
+toolbar toggle. Full view does not grant
manual control; use the existing control button separately. If fullscreen is
unavailable, the UI explains how to use the Workspace expand button instead.
diff --git a/docs/mobile-workspace.md b/docs/mobile-workspace.md
index 8ce9c06b..006be9ed 100644
--- a/docs/mobile-workspace.md
+++ b/docs/mobile-workspace.md
@@ -34,7 +34,9 @@ Both nodes use the same phone. Adding another node does not create another emula
## Everyday controls
-Choose **Full view** above the phone to fill the screen. The live phone expands to the available height; the task form, completed setup history and technical help are hidden. Phone controls remain outside the picture so they do not cover Android buttons. Use **Exit full view** or **Esc** to return. Entering full view keeps the current connection and does not restart the phone or grant manual control. Browser Workspace has the same full-view control, retaining its address bar and navigation outside the page. If the browser does not support fullscreen, use the existing Workspace expand button instead.
+The live phone fills the available Workspace height in normal view. Its proportions are preserved, so wide panels can have side bars. Expand **Ask AI to use the phone** to enter a task; collapsing it keeps your draft. The **Phone controls** toolbar toggle exposes Stop phone, refresh, APK installation while controlling the phone, completed setup history, and help. Status and takeover stay visible.
+
+Choose **Full view** in the toolbar to fill the screen. The task panel is hidden, and secondary controls remain available through the toolbar toggle. Controls stay outside the picture so they do not cover Android buttons. Use **Exit full view** or **Esc** to return. Entering full view keeps the current connection and does not restart the phone or grant manual control. Browser Workspace uses the same layout, with address and navigation under **Browser controls**. If the browser does not support fullscreen, use the existing Workspace expand button instead.
| Label | Meaning |
| --- | --- |