an open-source alternative to the dots, bots and muses of the world, run an office of claude code/codex like agents on your laptop, sandboxes or anywhere, uses your existing subscriptions
HTML agent-harnessagent-orchestrationagentsai-agentsautonomous-agentsclaude-codecodexcoding-agentsdesktop-appdeveloper-toolselectrongemini-cliharnessharness-engineeringlocal-firstmemorymulti-agentopencodeorchestrationtypescript
feat(telemetry): instrument the activation funnel (launch → first agent)
chaitanyagiri5e2e40b6993e
diff --git a/TELEMETRY.md b/TELEMETRY.mdindex 96e9aa74..3f6014ec 100644--- a/TELEMETRY.md+++ b/TELEMETRY.md@@ -1,8 +1,9 @@# TelemetryMunder Difflin collects a small set of **anonymous** usage events so we can-understand adoption (how many people launch the app, which features get used)-and make the product better. This document is the complete, authoritative+understand adoption (how many people launch the app, whether they get a first+agent running, which features get used) and make the product better. This+document is the complete, authoritativecontract: **if an event or property is not listed here, the app does not sendit.** The implementation lives in [`src/main/analytics.ts`](src/main/analytics.ts)and enforces this list as a hard allowlist — the code and this file are kept in@@ -26,6 +27,11 @@ The events:| `app_launched` | — | Each app start || `update_applied` | `from_version`, `to_version` — version strings, or `unknown` for an install older than this event; `via` — one of `auto` (the app's own updater installed it), `manual`, `unknown` | Once, on the first start after the app's version changes || `agent_spawned` | `provider` (CLI engine name, e.g. `claude`, `codex`) | An agent terminal is spawned |+| `onboarding_completed` | `provider` (the CLI engine chosen in the setup wizard) | Once, when onboarding finishes |+| `agent_spawn_attempted` | `provider` | Every time an agent spawn is requested, so it can be compared against `agent_spawned` |+| `agent_spawn_failed` | `provider`; `reason` is one of `cli_missing`, `cwd_missing`, `already_running`, `spawn_error` | A requested spawn did not start an agent |+| `agent_install_started` | `provider`; `rung` is one of `npm`, `node-then-npm`, `native` | The engine CLI was missing, so the bundled auto-installer began |+| `agent_install_finished` | `provider`; `rung` as above; `outcome` is one of `agent_launched`, `install_failed` | The auto-installer exited || `feature_used` | `feature` — one of `slack_trigger`, `webhook_trigger`, `hire_install`, `voice_dictation` | At most once per feature per app session || `session_ended` | `duration_bucket` — one of `<5m`, `5-30m`, `30m-2h`, `2-8h`, `8h+` | On quit (coarse bucket, never raw duration) |diff --git a/docs/privacy.html b/docs/privacy.htmlindex 58f8286f..4e020339 100644--- a/docs/privacy.html+++ b/docs/privacy.html@@ -220,7 +220,13 @@<tbody><tr><td>first_run</td><td>—</td><td>Once, the first time the app ever starts</td></tr><tr><td>app_launched</td><td>—</td><td>Each app start</td></tr>+ <tr><td>update_applied</td><td><code>from_version</code>, <code>to_version</code> (version strings, or <code>unknown</code> for an install older than this event); <code>via</code> is one of <code>auto</code>, <code>manual</code>, <code>unknown</code></td><td>Once, on the first start after the app's version changes</td></tr><tr><td>agent_spawned</td><td><code>provider</code> — the CLI engine name, e.g. <code>claude</code>, <code>codex</code></td><td>An agent terminal is spawned</td></tr>+ <tr><td>onboarding_completed</td><td><code>provider</code> (the CLI engine chosen in the setup wizard)</td><td>Once, when onboarding finishes</td></tr>+ <tr><td>agent_spawn_attempted</td><td><code>provider</code></td><td>Every time an agent spawn is requested, so it can be compared against <code>agent_spawned</code></td></tr>+ <tr><td>agent_spawn_failed</td><td><code>provider</code>; <code>reason</code> is one of <code>cli_missing</code>, <code>cwd_missing</code>, <code>already_running</code>, <code>spawn_error</code></td><td>A requested spawn did not start an agent</td></tr>+ <tr><td>agent_install_started</td><td><code>provider</code>; <code>rung</code> is one of <code>npm</code>, <code>node-then-npm</code>, <code>native</code></td><td>The engine CLI was missing, so the bundled auto-installer began</td></tr>+ <tr><td>agent_install_finished</td><td><code>provider</code>; <code>rung</code> as above; <code>outcome</code> is one of <code>agent_launched</code>, <code>install_failed</code></td><td>The auto-installer exited</td></tr><tr><td>feature_used</td><td><code>feature</code> — one of <code>slack_trigger</code>, <code>webhook_trigger</code>, <code>hire_install</code>, <code>voice_dictation</code></td><td>At most once per feature per session</td></tr><tr><td>session_ended</td><td><code>duration_bucket</code> — one of <code><5m</code>, <code>5-30m</code>, <code>30m-2h</code>, <code>2-8h</code>, <code>8h+</code></td><td>On quit — a coarse bucket, never a raw duration</td></tr></tbody>diff --git a/src/main/analytics.ts b/src/main/analytics.tsindex 18431b9e..3d55466a 100644--- a/src/main/analytics.ts+++ b/src/main/analytics.ts@@ -58,6 +58,29 @@ const EVENTS: Record<string, ReadonlySet<string>> = {update_applied: new Set<string>(['from_version', 'to_version', 'via']),/** An agent PTY spawned. `provider` is the CLI engine name only. */agent_spawned: new Set<string>(['provider']),+ /** ── The activation funnel (v0.4.7): app_launched → onboarding_completed →+ * agent_spawn_attempted → {agent_spawned | agent_spawn_failed |+ * agent_install_started → agent_install_finished}. Every added property is a+ * closed enum or a closed CLI name — nothing free-form, same allowlist rule. */+ /** Onboarding wizard finished (the install crossed onboardingComplete+ * false→true). `provider` is the engine chosen, a closed CLI name. The top of+ * the funnel: what share of installs finish setup, and which engine they pick. */+ onboarding_completed: new Set<string>(['provider']),+ /** A spawn was REQUESTED — every path through spawnAgentCore. Mirrors+ * agent_spawned; (attempted − spawned) is the activation fallout. */+ agent_spawn_attempted: new Set<string>(['provider']),+ /** A spawn did NOT produce a running agent. `reason` is a fixed enum (see+ * SpawnFailReason): `cli_missing` (engine absent and no auto-installer, manual+ * only), `cwd_missing`, `already_running`, `spawn_error`. */+ agent_spawn_failed: new Set<string>(['provider', 'reason']),+ /** The engine CLI was absent, so the auto-installer PTY started. `rung` is a+ * fixed enum (see InstallRung): `npm`, `node-then-npm`, `native`. */+ agent_install_started: new Set<string>(['provider', 'rung']),+ /** The auto-installer PTY exited. `outcome` is a fixed enum (see+ * InstallOutcome): `agent_launched` (clean exit, the agent is relaunching) or+ * `install_failed` (non-zero exit — e.g. an installer that cannot complete+ * unattended). This is the signal that a first agent never actually started. */+ agent_install_finished: new Set<string>(['provider', 'rung', 'outcome']),/** Coarse feature adoption; `feature` is a fixed enum (see FEATURES), fired* at most once per feature per app session. */feature_used: new Set<string>(['feature']),@@ -72,6 +95,18 @@ export type AnalyticsFeature =| 'hire_install'| 'voice_dictation';+/** The only values `agent_spawn_failed.reason` may take. A closed enum so the+ * "why didn't the agent start" split can never carry a free-form message. */+export type SpawnFailReason = 'cli_missing' | 'cwd_missing' | 'already_running' | 'spawn_error';++/** The only values the install events' `rung` may take. Mirrors+ * cliInstall.InstallRungKind minus `manual` — a manual rung spawns no installer,+ * so it never reaches agent_install_started; it is an agent_spawn_failed:cli_missing. */+export type InstallRung = 'npm' | 'node-then-npm' | 'native';++/** The only values `agent_install_finished.outcome` may take. */+export type InstallOutcome = 'agent_launched' | 'install_failed';+export interface AnalyticsInitOptions {/** Directory for the install-id file (userData). Created if missing. */stateDir: string;diff --git a/src/main/index.ts b/src/main/index.tsindex 5944af45..f4f4ab37 100644--- a/src/main/index.ts+++ b/src/main/index.ts@@ -56,6 +56,7 @@ import type { TaskCard, InboxMessage } from './realtimeCompletionWatcher';import { TelemetryCollector } from './telemetry';import { CostLedgerTotals } from './costLifetime';import { analytics } from './analytics';+import type { SpawnFailReason } from './analytics';import { IntegrationBroker } from './integrationBroker';import * as integrations from './integrations';import { validateBaseUrl, buildAuthHeaders, resolveUpstreamUrl, secretRefFor, INTEGRATION_TEMPLATES } from '../shared/integrations';@@ -228,7 +229,7 @@ const ptyToAgent = new Map<string, string>();* in this PTY; when it exits cleanly the exit handler re-runs the SAME spawn (with* install disabled) so the freshly-installed CLI launches in the SAME pty/window —* no user click. Cleared the moment it's consumed, so it can never loop installs. */-const pendingInstallRelaunch = new Map<string, { opts: AgentSpawnOptions; owner: Electron.WebContents | null; bin: string }>();+const pendingInstallRelaunch = new Map<string, { opts: AgentSpawnOptions; owner: Electron.WebContents | null; bin: string; rung: string }>();const hive = new HiveManager(() => readConfig().harnessHome,(channel, payload) => {@@ -579,7 +580,11 @@ ptyManager.setExitHandler((id, exitCode) => {const pending = pendingInstallRelaunch.get(id);if (pending) {pendingInstallRelaunch.delete(id);+ // Activation funnel: did the auto-installer actually complete? A non-zero exit+ // is the Linux-installer-cannot-finish-unattended signal that used to be silent.+ const provider = pending.opts.provider ?? inferAgentProvider(pending.opts.command, undefined);if (exitCode === 0) {+ analytics.track('agent_install_finished', { provider, rung: pending.rung, outcome: 'agent_launched' });// Re-arm the renderer's pooled terminal (clear the "process exited" line +// re-enable input) so the freshly-spawned CLI paints onto a clean, typeable// grid, then re-run the normal spawn — which now finds the installed binary.@@ -589,6 +594,7 @@ ptyManager.setExitHandler((id, exitCode) => {return; // an install PTY has no agent/worktree to tear down}// Non-zero exit = install failed; leave its honest manual-fix message on screen.+ analytics.track('agent_install_finished', { provider, rung: pending.rung, outcome: 'install_failed' });}teardownPty(id);});@@ -2499,6 +2505,16 @@ function findCodexHomeForSession(sessionId: string, siblingsRoot: string): strin* ephemeral-worker watcher. */type AgentSpawnOptions = SpawnOptions & { hive?: AgentMeta; isolate?: boolean; resume?: boolean; requireResume?: boolean; resumeSessionId?: string; provider?: AgentProvider; noAutoInstall?: boolean };+/** Map a `ptyManager.spawn` failure string to the closed `agent_spawn_failed.reason`+ * enum (analytics.ts). The two known strings come from PtyManager.spawn; anything+ * else is a generic `spawn_error`. The raw message never leaves the machine — only+ * the enum value does, per TELEMETRY.md. */+function spawnFailReason(error?: string): SpawnFailReason {+ if (error?.startsWith('cwd does not exist')) return 'cwd_missing';+ if (error?.includes('already exists')) return 'already_running';+ return 'spawn_error';+}+ipcMain.handle('pty:spawn', async (evt, opts: AgentSpawnOptions) => {if (!opts || typeof opts.id !== 'string' || typeof opts.cwd !== 'string' || typeof opts.command !== 'string') {return { ok: false, error: 'invalid SpawnOptions' };@@ -2534,6 +2550,11 @@ async function spawnAgentCore(opts: AgentSpawnOptions, owner: Electron.WebContenconst claudeProvider = isClaudeProvider(provider);opts.provider = provider;if (opts.hive) opts.hive = { ...opts.hive, provider };+ // Activation-funnel entry (v0.4.7): every spawn REQUEST, so (attempted − spawned)+ // measures the fallout the whole rebuild exists to see. Gated on !noAutoInstall so+ // the missing-CLI relaunch (the only re-entry, index.ts install-exit handler) does+ // NOT double-count a single user attempt — it is the SAME attempt continuing.+ if (!opts.noAutoInstall) analytics.track('agent_spawn_attempted', { provider });// ── Missing engine CLI → run its installer visibly (pre-spawn) ───────────────// If the agent's engine binary (claude/codex/…) isn't installed, spawning it// just dies with "— process exited (code 1) —" and the user has no idea why.@@ -2587,7 +2608,18 @@ async function spawnAgentCore(opts: AgentSpawnOptions, owner: Electron.WebConten// binary and die with the bare "process exited (code 1)" this whole path exists// to replace.if (res.ok && rung.command) {- pendingInstallRelaunch.set(opts.id, { opts, owner, bin });+ pendingInstallRelaunch.set(opts.id, { opts, owner, bin, rung: rung.kind });+ // The auto-installer PTY is running; agent_install_finished on its exit says+ // whether it actually produced an agent (rung is non-manual here by construction).+ analytics.track('agent_install_started', { provider, rung: rung.kind });+ } else if (res.ok) {+ // Manual rung: the PTY only printed a hint (no installer to run, no relaunch+ // armed), so no agent will start. This is the Mode 2 case that used to send+ // NOTHING — an absent engine with no unattended install path.+ analytics.track('agent_spawn_failed', { provider, reason: 'cli_missing' });+ } else {+ // The install PTY itself failed to spawn (cwd gone, id clash, throw).+ analytics.track('agent_spawn_failed', { provider, reason: spawnFailReason(res.error) });}syncKeepAwake();return res;@@ -2887,6 +2919,7 @@ async function spawnAgentCore(opts: AgentSpawnOptions, owner: Electron.WebConten}const res = ptyManager.spawn(opts, owner);if (res.ok) analytics.track('agent_spawned', { provider });+ else analytics.track('agent_spawn_failed', { provider, reason: spawnFailReason(res.error) });syncKeepAwake(); // arm the power-save blocker while ≥1 agent PTY is alive (#18)// Hand the resolved worktree path back to the renderer so it can persist it on// the agent (only set when isolation actually provisioned a worktree above).@@ -3062,9 +3095,16 @@ ipcMain.handle('config:update', (_evt, patch: Partial<HarnessConfig>) => {// relaunch, so bootstrap here on the null → set transition. Gated on the// transition so ordinary config writes never re-enter it.const hiveWasEnabled = hive.enabled();+ const wasOnboarded = readConfig().onboardingComplete;const next = writeConfig(patch);// Live opt-in/out from Settings → Privacy (TELEMETRY.md).if (typeof patch?.telemetryEnabled === 'boolean') analytics.setEnabled(patch.telemetryEnabled);+ // Activation funnel (v0.4.7): onboarding just finished (false → true) — the top of+ // the launch → first-agent funnel. `provider` is the engine chosen in the wizard.+ // Fired here (main), not in the renderer, so it rides the same allowlist as the rest.+ if (!wasOnboarded && next.onboardingComplete) {+ analytics.track('onboarding_completed', { provider: next.godProvider ?? 'claude' });+ }// Keep the hive's mirror of the spawn gate current. The queue itself reads// config per tick so it gates immediately; this is for the PROMPT, which is// built per spawn, so flipping the toggle reaches god the next time he starts.diff --git a/test/telemetry-funnel.test.cjs b/test/telemetry-funnel.test.cjsnew file mode 100644index 00000000..66cd9c65--- /dev/null+++ b/test/telemetry-funnel.test.cjs@@ -0,0 +1,107 @@+'use strict';++// Activation-funnel telemetry (v0.4.7): the 5 new events must pass the analytics+// allowlist, enforce their closed enums (drop any unknown key), stay anonymous, and+// be wired at the four spawn-funnel sites in index.ts. index.ts imports electron and+// cannot load under plain node, so its wiring is asserted against the source text —+// the same pattern the rest of the suite uses for main-process wiring.++const test = require('node:test');+const assert = require('node:assert/strict');+const fs = require('node:fs');+const os = require('node:os');+const path = require('node:path');+const loadTs = require('./load-ts.cjs');++// Stub the build-time key + PostHog BEFORE analytics.ts loads, exactly like+// update-applied.test.cjs, so the class runs end to end with no network or key.+globalThis.__POSTHOG_KEY__ = 'test-key';+globalThis.__POSTHOG_HOST__ = 'https://example.invalid';+delete process.env.DO_NOT_TRACK;++const captured = [];+class FakePostHog {+ capture(payload) { captured.push(payload); }+ async shutdown() {}+}+const posthogPath = require.resolve('posthog-node');+require.cache[posthogPath] = {+ id: posthogPath, filename: posthogPath, loaded: true, exports: { PostHog: FakePostHog }+};++const { Analytics } = loadTs('src/main/analytics.ts');++function freshDir() {+ return fs.mkdtempSync(path.join(os.tmpdir(), 'md-telemetry-funnel-'));+}++function bootedAnalytics() {+ const a = new Analytics();+ a.init({ stateDir: freshDir(), appVersion: '0.4.7', enabled: true });+ captured.length = 0; // drop the first_run / app_launched from init+ return a;+}++// ── the allowlist accepts the new events and enforces their enums ─────────────++test('all five funnel events are on the allowlist and carry their properties', () => {+ const a = bootedAnalytics();+ a.track('onboarding_completed', { provider: 'gemini' });+ a.track('agent_spawn_attempted', { provider: 'claude' });+ a.track('agent_spawn_failed', { provider: 'codex', reason: 'cli_missing' });+ a.track('agent_install_started', { provider: 'claude', rung: 'npm' });+ a.track('agent_install_finished', { provider: 'claude', rung: 'npm', outcome: 'install_failed' });++ const byEvent = Object.fromEntries(captured.map((c) => [c.event, c.properties]));+ assert.equal(byEvent.onboarding_completed.provider, 'gemini');+ assert.equal(byEvent.agent_spawn_attempted.provider, 'claude');+ assert.equal(byEvent.agent_spawn_failed.reason, 'cli_missing');+ assert.equal(byEvent.agent_install_started.rung, 'npm');+ assert.equal(byEvent.agent_install_finished.outcome, 'install_failed');+ // Common props still stamped on the new events.+ assert.equal(byEvent.agent_spawn_attempted.app_version, '0.4.7');+});++test('an unknown property on a new event is dropped, never sent', () => {+ const a = bootedAnalytics();+ // A caller that tries to smuggle a free-form value must not leak it.+ a.track('agent_spawn_failed', { provider: 'claude', reason: 'spawn_error', cwd: '/Users/someone/secret-repo' });+ const props = captured.find((c) => c.event === 'agent_spawn_failed').properties;+ assert.equal(props.reason, 'spawn_error');+ assert.equal(props.cwd, undefined, 'the un-allowlisted key must be dropped');+});++test('the new events stay anonymous by construction', () => {+ const a = bootedAnalytics();+ a.track('agent_install_finished', { provider: 'claude', rung: 'native', outcome: 'agent_launched' });+ const props = captured.find((c) => c.event === 'agent_install_finished').properties;+ assert.equal(props.$process_person_profile, false);+});++// ── index.ts wires the funnel at the four spawn sites ─────────────────────────++const main = fs.readFileSync(path.resolve(__dirname, '..', 'src/main/index.ts'), 'utf8');++test('agent_spawn_attempted fires once per attempt, gated against the install relaunch', () => {+ assert.match(main, /if \(!opts\.noAutoInstall\) analytics\.track\('agent_spawn_attempted', \{ provider \}\);/);+});++test('the real spawn reports both success and failure', () => {+ assert.match(main, /if \(res\.ok\) analytics\.track\('agent_spawned', \{ provider \}\);/);+ assert.match(main, /else analytics\.track\('agent_spawn_failed', \{ provider, reason: spawnFailReason\(res\.error\) \}\);/);+});++test('the missing-CLI branch distinguishes installer-running, Mode 2, and spawn error', () => {+ assert.match(main, /analytics\.track\('agent_install_started', \{ provider, rung: rung\.kind \}\);/);+ assert.match(main, /analytics\.track\('agent_spawn_failed', \{ provider, reason: 'cli_missing' \}\);/);+});++test('the install-PTY exit reports whether the auto-installer completed', () => {+ assert.match(main, /analytics\.track\('agent_install_finished', \{ provider, rung: pending\.rung, outcome: 'agent_launched' \}\);/);+ assert.match(main, /analytics\.track\('agent_install_finished', \{ provider, rung: pending\.rung, outcome: 'install_failed' \}\);/);+});++test('onboarding completion is reported on the false to true transition', () => {+ assert.match(main, /if \(!wasOnboarded && next\.onboardingComplete\) \{/);+ assert.match(main, /analytics\.track\('onboarding_completed', \{ provider: next\.godProvider \?\? 'claude' \}\);/);+});