an open-source alternative to the dots, bots and muses of the world, run an office of claude code/codex like agents on your laptop, sandboxes or anywhere, uses your existing subscriptions
v0.3.5 Feature Research — Voice Control · Settings · Markdown Preview · Git Visualization (+ QM survey)
Deep-research pass run 2026-08-06 against the v0.3.4 integration branch, by five parallel research agents (code-grounded + web-verified). No code changes were made. File:line references are against
integrate/v0.3.4.
0. Cross-cutting findings
yc-software/qmis real and relevant: YC's open-sourced (MIT, 2026-07-31, ~11.6k★) "multiplayer agent harness for work" — TypeScript/Fastify/Postgres server + Lit web UI, pluggable harnesses (Claude Code, Codex, Pi, OpenCode), durable sandboxes, crons/watches, scoped skills, three-tier security postures. It has NO voice surface and NO git/diff UI — Munder Difflin is ahead of it on both. Adopt patterns, not modules (its stack doesn't transplant into Electron/React).- QM adoption shortlist (tiered):
- Security postures: one Strict/Auto/Dangerous dial + a non-negotiable command deny-floor (no recursive deletes / destructive SQL) enforced in every posture → maps to a Settings "Autonomy" preset + a hive-level command filter.
- Crons + watches: condition-triggered agent runs (CI red, file changed, queue depth) alongside schedules; steal the cron humanizer UX.
- Skills manager UI: per-agent/project skills viewable/editable/promotable to all agents.
- Durability by default: queues/tasks/audit must survive restart (SQLite, not memory).
- Streaming-markdown pipeline: streaming render + dedupe + lazy hljs/KaTeX (feeds feature 3).
- Deep-linkable UI state · conversation forking (pairs with worktrees) · per-scope resource sidebar · single-source versioned JSONC config with a doctor validator (fixes our 3-file HarnessConfig mirror pain) · untrusted-ingress prompt-injection screening (Browser/Slack).
- Security finding (fix regardless):
config:updateIPC (src/main/index.ts:2475) accepts an arbitrary unvalidatedPartial<HarnessConfig>; config carries secrets (groqApiKey, slackBotToken, slackSigningSecret, webhookSecret) and dangerous keys (harnessHome). Any voice/settings tooling must go through a main-side per-key allowlist — and the raw handler deserves validation on its own.
1. Realtime Michael — deep context + full control (est. 6.5–8.5 days, 7 PRs)
Current state
gpt-realtime-2over WebRTC (@openai/agents-realtime0.11.8), ephemeral token minted main-side (src/main/realtime.ts:50-106). Persona ~950 tokens (renderer session.ts:78-106).- Context = ONE connect-time snapshot (~150-250 tokens: agent counts, doing/blocked counts, last 6 log kinds) — never refreshed; only completion pushes + greeting inject mid-session.
- 11 read tools (fleet/tasks/board/cost/schedules/config-allowlist/memory/activity/messages,
via IPC;
hive:agentDirectoryat index.ts:2885 is already the consolidated per-agent join). - 14 action verbs with a solid main-side safety spine (realtimeActions.ts): soft vs destructive
tiers, single pending slot + 120s TTL, distinct-token confirm, god/mass-target hard denylist,
michael-voiceattribution + god cross-notify.
Gaps
- Snapshot goes stale; no push on status/task/spawn changes; live store state (
Agent.action, queue depth) unused; every factual answer costs a tool round trip. - No voice path for: resume (pause is one-way by voice — sharpest gap), auto-delivery,
/clearcontext, restart/model change, settings, archive, delete task, create schedule, tool gating, queue read/clear. /clearhas NO IPC — it's a renderer-store queued message (store.ts:220,677) delivered by the idle-gated flush loop (useHive.ts) which zeroes the gauge on delivery.restartWithModelis component-local in CommandCenterPanel — not callable elsewhere.
Realtime API facts (verified 2026-08, developers.openai.com)
- GA: gpt-realtime-2.1 / 2.1-mini (128k context, 60-min sessions, reasoning-capable —
use
reasoning.effort: lowfor voice). App pins gpt-realtime-2 (still fine; consider bump). session.updateCAN swap instructions/tools mid-session (voice locks after first audio; instructions+tools ≤ 16,384 tokens) — but it busts the prompt cache. Caching is automatic exact-prefix (≥1,024 tokens); cached audio input is ~99% off ($0.40 vs $32 /1M). Therefore: stable prefix + append-only conversation items, never mutate instructions.conversation.item.createinjects messages mid-stream; SDK:sendMessage(msg, {}, {triggerResponse:false})silent injection,updateAgent(), out-of-bandresponse.create {conversation:"none"}. Parallel tool calls: NOT supported (sequential). No hard tool-count cap — the token budget is the limit; ~30 crisp tools is fine. Mask tool latency with spoken preambles; return compact flagged JSON, not prose.
Recommended architecture
- Snapshot at connect + append-only deltas (never touch instructions): compact floor table
(one line per agent: name|status|action|ctx%|breaker|inbox; ~400-600 tokens) injected as the
FIRST conversation item; a main-side FloorDeltaWatcher (sibling of
RealtimeCompletionWatcher) pushes coalesced deltas (status flips, task moves, spawn/archive,
breaker trips, ctx 70/90%) via the existing sanitized sendMessage path — debounced ≥10-15s,
≤150 tokens each. Add
get_floor_statereadinguseStore.getState()(zero IPC) returning flagged JSON. - Extend the verb spine, keep narrow tools (no free-text query_hive in v1):
- Soft:
resume_agent,set_auto_delivery,gate_tool,delete_task,get/clear_queue. - Destructive (echo-back + distinct token):
clear_context(main confirms → rendererenqueueMessage('/clear')so ALL existing delivery gates apply),restart_agent(extract restartWithModel into a shared renderer service first),archive_agent,create_schedule,update_setting. update_settingpolicy table (main-side): forbidden = harnessHome + all secret keys + providerBaseUrls + integrations; soft = notifications/themes/freeflow/idle-disconnect/ strongKeepalive; confirm = autoMode/defaultModel/god engine/worker caps/breaker/slack/ webhook/maxTurns. Validate type+range; echo old→new in the confirm.
- Soft:
- Latency budget: persona+30 tools
5k tokens cached (free); snapshot once; deltas ~1-2k/hr; most "what's happening" answers need ZERO round trips; detail = 1 round trip masked by a preamble line added to the persona.
PR sequence
1 snapshot+get_floor_state (0.5-1d) · 2 FloorDeltaWatcher (1-1.5d) · 3 soft verbs (1-1.5d) · 4 clear_context + queue tools (0.5-1d) · 5 update_setting policy table (1d) · 6 restart_agent service extraction (1.5-2d, highest risk) · 7 polish/model bump (0.5d).
2. Settings redesign (est. 3–4 days, 4 phases)
Problems found (full inventory in agent report; SettingsModal is 1,428 lines)
- "General" is six unrelated domains; dead read-only rows (Auto mode / Semantic memory / Command are displayed but not editable — autoMode has NO editor anywhere post-onboarding).
- Onboarding-trapped settings:
strongKeepalive,audience, open-at-login — set once, unreachable forever. defaultModel— the most consequential setting — has no UI at all; ditto maxTurns, 4 of 6 circuit-breaker fields (incl. hardStop, a safety decision).- Duplication: auto-compact toggle in General AND Schedules; semanticMemory shown in General, toggled in MemoryPanel; theme forks into localStorage + terminalTheme.
- Secrets scattered across 5 entry UIs; OpenAI key in "AI Engines" silently gates Talk in "Integrations".
- Default-drift bugs in config.ts: knowledgeGraph
enabled:truecontradicts its own "Default OFF" comments; SettingsModal seeds freeflowEnabled?? falsewhile default is true (UI shows OFF when ON); multiWindow code=true vs comment="OFF by default"; legacycostCapUsdstill enforced — delete it.
Proposed IA — 6 tabs, simple-by-default + per-tab Advanced disclosure + header search
- General — home folder + reset row (Danger Zone absorbed), appearance (ONE theme control; office-theme picker absorbs tvShowOffices), notifications, keep-awake, open-at-login, explain-simply (audience), auto-update. Adv: multiWindow.
- Agents & Models — default agent model (NEW UI), Michael's engine (shared component with Command Center), BYOK keys with "what each key unlocks" note. Adv: per-CLI base URLs + default models, defaultCommand, maxTurns.
- Autonomy & Budgets (QM-style) — Ask-first/Autonomous preset (autoMode), full circuit breaker as one unit (give the 4 hidden fields UI), floor budget + velocity. Adv: worker fleet knobs. Delete costCapUsd.
- Connections — MCP tiers, Slack, Webhook, REST registry; setup guides become collapsible flows; ports under Advanced.
- Voice — Free Flow (toggle + Groq key) + Realtime Michael (key status w/ jump-link, devices, idle disconnect, cost HUD).
- Memory & Knowledge — semantic memory + embedding model, Knowledge Graph. Adv: reflection enabled+interval (leave 4 threshold fields config-only).
Phases: extract SettingRow/Section primitives + split the modal (0.5d) → new nav + move blocks +
missing editors, all via existing updateConfig (1d) → Voice tab + collapsible setup + cleanup
(1d) → search + QA (0.5-1d). Watch the 3-file HarnessConfig mirror on every field change; the
15-state rehydration useEffect is the main regression risk.
3. Markdown preview — IDE + terminal click (est. 3–5 days core, 3 PRs + polish)
Current state
- IDE (IdePanel.tsx): tabs
{key, rel, mode: 'edit'|'diff'};languageForPathalready maps markdown; Monaco fully self-hosted (CSP precedent). No way to open the IDE "at a file". - Reusable overlay exists:
fullscreenFilePathstore field →FullscreenFileEditor(z 280 — already above fullscreen terminal 250, below IDE 290), resolves ABSOLUTE paths incl. outside-workspace, renders CodeMirror (lang-markdown already wired). - No terminal link plumbing at all — no WebLinksAddon, no registerLinkProvider; xterm 5.5's
ILinkProviderAPI is available; agents in store carry cwd+ptyId for relative resolution. - CSP blocks remote images (a feature for untrusted md).
Recommendation: react-markdown@10 + remark-gfm@4 (~45-55 kB min+gz)
Safe by construction for untrusted agent output — renders to React elements, no HTML sink,
no rehype-raw, default urlTransform drops javascript:. QM uses marked+DOMPurify with a fused
postprocess-sanitize hook (plugins/web-ui/src/markdown-sanitize.ts) — validated fallback
pattern, but react-markdown fits a React app better (link interception, tokens theming,
position-based scroll sync). Phase 2: rehype-highlight; mermaid only as a lazy bundled chunk
with securityLevel:'strict'. Reject @uiw/react-markdown-preview (enables rehype-raw, forces
GitHub look, 3.6 MB).
Workflows
- Shared:
markdown/MarkdownPreview.tsx— CTH-token styled (light+dark), every anchor intercepted: http(s)/mailto →app:openExternal(add scheme allowlist), relative .md → open-in-context, all else inert. - A. IDE: per-markdown-tab
view: 'code'|'split'|'preview'(default split, persisted); three-segment toggle in the EditorBar (only for md); Cmd+Shift+V cycles; preview reads the live edit buffer viauseDeferredValue— save flow untouched; md links open sibling tabs. - B. Terminal: custom
ILinkProviderregistered once inacquireTerminal()(covers all three terminal surfaces automatically); regex for absolute/~/relative *.md (+ tolerant of quotes/backticks/:line); resolve vs owning agent's cwd; verify existence via ONE newfs:statIPC (mirror in preload + index.d.ts — 3-file gotcha); Cmd/Ctrl+click →setFullscreenFile(abs, {view:'preview'})→ FullscreenFileEditor gains an edit|preview toggle (Obsidian-style; preview default from terminal); "open in IDE" escalation via a newideInitialFilestore field consumed on IDE mount. FilesTab's narrow editor gets the same toggle. - Security: no HTML sink; anchors never navigate the window; terminal-derived paths flow through safeJoin + 2 MB cap + binary sniff, opened only on explicit modifier-click.
PRs: 1 MarkdownPreview + IDE split (1-2d) · 2 overlay + FilesTab toggles (0.5-1d) · 3 link provider + fs:stat + tests (1-2d) · 4 polish: scroll sync, highlight, mermaid (1-2d).
4. Git visualization (est. 6–8 days, 4 PRs)
Striking finding: ~40% is already built and UNMOUNTED
GitTab.tsx— complete branch/status/log panel with commit graph — orphaned (nothing imports it since the tab slim-down).git/CommitGraph.tsx— themed wrapper over thecommit-graphnpm dep already in package.json — display-only today (onCommitClick/getDiff props unused).git/graph.ts— hand-written DAG lane layout — also unused (documented fallback).git.tsalready returns full DAG topology (getLogemits hash+parents+refs) and contains the exact dirty/ahead safety logic a safe checkout needs (worktreeHasUnintegratedWork).- Missing entirely: per-commit diffs, branch compare, checkout, graph interactivity.
Library verdict
[email protected] (DoltHub; active, React 18, permissive, 323 kB) is best-in-class and takes
exactly our data shape; keep graph.ts as the custom-SVG fallback if its popups fight the
theme. @gitgraph/* is dead; vscode Git Graph's license PROHIBITS reuse (approach only);
isomorphic-git unnecessary (we shell out); no diff library needed — Monaco DiffEditor does it.
QM has no git UI to borrow.
Architecture
- Data (PR1):
getLogGraph(+--topo-order,--skippagination, run at mainRepoRoot so all agent/ worktree branches appear) ·getCommitFiles(diff-tree -z -M --name-status, handle R/C renames) ·getFileAtRev(generalizeshow HEAD:to any rev; keep 2 MB cap + binary sniff; per-commit diff = sha^ vs sha into MonacoDiff) ·compareRefs(three-dot default w/ two-dot toggle + left-right counts + merge-base) ·listWorktrees· guardedcheckout. Strict rev regex +--separation (renderer-supplied refs). New IPC beside index.ts:2570; update renderer type mirrors. - UI (PR2/3): IDE left rail becomes CHANGES · HISTORY · COMPARE sub-tabs. HISTORY:
mount CommitGraph with onCommitClick → file list (reuse CHANGES row styling) → Monaco diff
tabs keyed
commit::sha::path; infinite scroll past 200. COMPARE: base/head dropdowns + swap + ahead/behind counts + dot-mode toggle → file list → diff tabs. Checkout via context menu on branch chips/commit rows; detached-HEAD affordance exists in GitTab. Re-mount slim GitTab as a 4th sidebar tab (one-line mount) for at-a-glance status. - Checkout safety (main-process, non-negotiable): clean tree only (no auto-stash v1); owning agent must be idle — and gate on ANY agent whose cwd resolves to the same root (the god-at-mainRepoRoot sneaky case); branch-held-by-worktree errors mapped to "checked out by agent X"; fail-safe on any query error.
- Risks: big repos (--all --topo-order walk; 8s runGit timeout is the backstop; --skip re-walks — fine to ~50k commits); do NOT add graph queries to the 4s polling loops (refresh on tab activation/explicit/status change); commit-graph popup styling may need the global.css override block extended.
PRs: 1 data layer (1-1.5d) · 2 HISTORY (2-3d) · 3 COMPARE + checkout UX (2d) · 4 polish + GitTab remount (1d).
5. Suggested build order (if all four are greenlit)
- Markdown PR1-2 (fast, visible, zero risk) → Git PR1 + mount HISTORY (the unmounted 40% makes this the best value-per-day in the codebase).
- Voice PR1-2 (context wins — biggest UX delta for talk mode) → Voice PR3-5 (control).
- Settings phases 1-3 (independent; can interleave; fixes real config bugs).
- Long tail: Git COMPARE/checkout, Voice restart_agent, markdown polish, Settings search.
- Cross-cutting from QM (separate track): autonomy postures + command deny-floor, watches, skills manager, config single-sourcing.
Grand total if everything ships: ~19–26 dev-days.