Skip to content

an open-source alternative to the dots, bots and muses of the world, run an office of claude code/codex like agents on your laptop, sandboxes or anywhere, uses your existing subscriptions

feat(telemetry): instrument the activation funnel (launch → first agent)

chaitanyagiri5e2e40b6993e
diff --git a/TELEMETRY.md b/TELEMETRY.md
index 96e9aa74..3f6014ec 100644
--- a/TELEMETRY.md
+++ b/TELEMETRY.md
@@ -1,8 +1,9 @@
# Telemetry
Munder Difflin collects a small set of **anonymous** usage events so we can
-understand adoption (how many people launch the app, which features get used)
-and make the product better. This document is the complete, authoritative
+understand adoption (how many people launch the app, whether they get a first
+agent running, which features get used) and make the product better. This
+document is the complete, authoritative
contract: **if an event or property is not listed here, the app does not send
it.** The implementation lives in [`src/main/analytics.ts`](src/main/analytics.ts)
and enforces this list as a hard allowlist — the code and this file are kept in
@@ -26,6 +27,11 @@ The events:
| `app_launched` | — | Each app start |
| `update_applied` | `from_version`, `to_version` — version strings, or `unknown` for an install older than this event; `via` — one of `auto` (the app's own updater installed it), `manual`, `unknown` | Once, on the first start after the app's version changes |
| `agent_spawned` | `provider` (CLI engine name, e.g. `claude`, `codex`) | An agent terminal is spawned |
+| `onboarding_completed` | `provider` (the CLI engine chosen in the setup wizard) | Once, when onboarding finishes |
+| `agent_spawn_attempted` | `provider` | Every time an agent spawn is requested, so it can be compared against `agent_spawned` |
+| `agent_spawn_failed` | `provider`; `reason` is one of `cli_missing`, `cwd_missing`, `already_running`, `spawn_error` | A requested spawn did not start an agent |
+| `agent_install_started` | `provider`; `rung` is one of `npm`, `node-then-npm`, `native` | The engine CLI was missing, so the bundled auto-installer began |
+| `agent_install_finished` | `provider`; `rung` as above; `outcome` is one of `agent_launched`, `install_failed` | The auto-installer exited |
| `feature_used` | `feature` — one of `slack_trigger`, `webhook_trigger`, `hire_install`, `voice_dictation` | At most once per feature per app session |
| `session_ended` | `duration_bucket` — one of `<5m`, `5-30m`, `30m-2h`, `2-8h`, `8h+` | On quit (coarse bucket, never raw duration) |
diff --git a/docs/privacy.html b/docs/privacy.html
index 58f8286f..4e020339 100644
--- a/docs/privacy.html
+++ b/docs/privacy.html
@@ -220,7 +220,13 @@
<tbody>
<tr><td>first_run</td><td>—</td><td>Once, the first time the app ever starts</td></tr>
<tr><td>app_launched</td><td>—</td><td>Each app start</td></tr>
+ <tr><td>update_applied</td><td><code>from_version</code>, <code>to_version</code> (version strings, or <code>unknown</code> for an install older than this event); <code>via</code> is one of <code>auto</code>, <code>manual</code>, <code>unknown</code></td><td>Once, on the first start after the app's version changes</td></tr>
<tr><td>agent_spawned</td><td><code>provider</code> — the CLI engine name, e.g. <code>claude</code>, <code>codex</code></td><td>An agent terminal is spawned</td></tr>
+ <tr><td>onboarding_completed</td><td><code>provider</code> (the CLI engine chosen in the setup wizard)</td><td>Once, when onboarding finishes</td></tr>
+ <tr><td>agent_spawn_attempted</td><td><code>provider</code></td><td>Every time an agent spawn is requested, so it can be compared against <code>agent_spawned</code></td></tr>
+ <tr><td>agent_spawn_failed</td><td><code>provider</code>; <code>reason</code> is one of <code>cli_missing</code>, <code>cwd_missing</code>, <code>already_running</code>, <code>spawn_error</code></td><td>A requested spawn did not start an agent</td></tr>
+ <tr><td>agent_install_started</td><td><code>provider</code>; <code>rung</code> is one of <code>npm</code>, <code>node-then-npm</code>, <code>native</code></td><td>The engine CLI was missing, so the bundled auto-installer began</td></tr>
+ <tr><td>agent_install_finished</td><td><code>provider</code>; <code>rung</code> as above; <code>outcome</code> is one of <code>agent_launched</code>, <code>install_failed</code></td><td>The auto-installer exited</td></tr>
<tr><td>feature_used</td><td><code>feature</code> — one of <code>slack_trigger</code>, <code>webhook_trigger</code>, <code>hire_install</code>, <code>voice_dictation</code></td><td>At most once per feature per session</td></tr>
<tr><td>session_ended</td><td><code>duration_bucket</code> — one of <code>&lt;5m</code>, <code>5-30m</code>, <code>30m-2h</code>, <code>2-8h</code>, <code>8h+</code></td><td>On quit — a coarse bucket, never a raw duration</td></tr>
</tbody>
diff --git a/src/main/analytics.ts b/src/main/analytics.ts
index 18431b9e..3d55466a 100644
--- a/src/main/analytics.ts
+++ b/src/main/analytics.ts
@@ -58,6 +58,29 @@ const EVENTS: Record<string, ReadonlySet<string>> = {
update_applied: new Set<string>(['from_version', 'to_version', 'via']),
/** An agent PTY spawned. `provider` is the CLI engine name only. */
agent_spawned: new Set<string>(['provider']),
+ /** ── The activation funnel (v0.4.7): app_launched → onboarding_completed →
+ * agent_spawn_attempted → {agent_spawned | agent_spawn_failed |
+ * agent_install_started → agent_install_finished}. Every added property is a
+ * closed enum or a closed CLI name — nothing free-form, same allowlist rule. */
+ /** Onboarding wizard finished (the install crossed onboardingComplete
+ * false→true). `provider` is the engine chosen, a closed CLI name. The top of
+ * the funnel: what share of installs finish setup, and which engine they pick. */
+ onboarding_completed: new Set<string>(['provider']),
+ /** A spawn was REQUESTED — every path through spawnAgentCore. Mirrors
+ * agent_spawned; (attempted − spawned) is the activation fallout. */
+ agent_spawn_attempted: new Set<string>(['provider']),
+ /** A spawn did NOT produce a running agent. `reason` is a fixed enum (see
+ * SpawnFailReason): `cli_missing` (engine absent and no auto-installer, manual
+ * only), `cwd_missing`, `already_running`, `spawn_error`. */
+ agent_spawn_failed: new Set<string>(['provider', 'reason']),
+ /** The engine CLI was absent, so the auto-installer PTY started. `rung` is a
+ * fixed enum (see InstallRung): `npm`, `node-then-npm`, `native`. */
+ agent_install_started: new Set<string>(['provider', 'rung']),
+ /** The auto-installer PTY exited. `outcome` is a fixed enum (see
+ * InstallOutcome): `agent_launched` (clean exit, the agent is relaunching) or
+ * `install_failed` (non-zero exit — e.g. an installer that cannot complete
+ * unattended). This is the signal that a first agent never actually started. */
+ agent_install_finished: new Set<string>(['provider', 'rung', 'outcome']),
/** Coarse feature adoption; `feature` is a fixed enum (see FEATURES), fired
* at most once per feature per app session. */
feature_used: new Set<string>(['feature']),
@@ -72,6 +95,18 @@ export type AnalyticsFeature =
| 'hire_install'
| 'voice_dictation';
+/** The only values `agent_spawn_failed.reason` may take. A closed enum so the
+ * "why didn't the agent start" split can never carry a free-form message. */
+export type SpawnFailReason = 'cli_missing' | 'cwd_missing' | 'already_running' | 'spawn_error';
+
+/** The only values the install events' `rung` may take. Mirrors
+ * cliInstall.InstallRungKind minus `manual` — a manual rung spawns no installer,
+ * so it never reaches agent_install_started; it is an agent_spawn_failed:cli_missing. */
+export type InstallRung = 'npm' | 'node-then-npm' | 'native';
+
+/** The only values `agent_install_finished.outcome` may take. */
+export type InstallOutcome = 'agent_launched' | 'install_failed';
+
export interface AnalyticsInitOptions {
/** Directory for the install-id file (userData). Created if missing. */
stateDir: string;
diff --git a/src/main/index.ts b/src/main/index.ts
index 5944af45..f4f4ab37 100644
--- a/src/main/index.ts
+++ b/src/main/index.ts
@@ -56,6 +56,7 @@ import type { TaskCard, InboxMessage } from './realtimeCompletionWatcher';
import { TelemetryCollector } from './telemetry';
import { CostLedgerTotals } from './costLifetime';
import { analytics } from './analytics';
+import type { SpawnFailReason } from './analytics';
import { IntegrationBroker } from './integrationBroker';
import * as integrations from './integrations';
import { validateBaseUrl, buildAuthHeaders, resolveUpstreamUrl, secretRefFor, INTEGRATION_TEMPLATES } from '../shared/integrations';
@@ -228,7 +229,7 @@ const ptyToAgent = new Map<string, string>();
* in this PTY; when it exits cleanly the exit handler re-runs the SAME spawn (with
* install disabled) so the freshly-installed CLI launches in the SAME pty/window —
* no user click. Cleared the moment it's consumed, so it can never loop installs. */
-const pendingInstallRelaunch = new Map<string, { opts: AgentSpawnOptions; owner: Electron.WebContents | null; bin: string }>();
+const pendingInstallRelaunch = new Map<string, { opts: AgentSpawnOptions; owner: Electron.WebContents | null; bin: string; rung: string }>();
const hive = new HiveManager(
() => readConfig().harnessHome,
(channel, payload) => {
@@ -579,7 +580,11 @@ ptyManager.setExitHandler((id, exitCode) => {
const pending = pendingInstallRelaunch.get(id);
if (pending) {
pendingInstallRelaunch.delete(id);
+ // Activation funnel: did the auto-installer actually complete? A non-zero exit
+ // is the Linux-installer-cannot-finish-unattended signal that used to be silent.
+ const provider = pending.opts.provider ?? inferAgentProvider(pending.opts.command, undefined);
if (exitCode === 0) {
+ analytics.track('agent_install_finished', { provider, rung: pending.rung, outcome: 'agent_launched' });
// Re-arm the renderer's pooled terminal (clear the "process exited" line +
// re-enable input) so the freshly-spawned CLI paints onto a clean, typeable
// grid, then re-run the normal spawn — which now finds the installed binary.
@@ -589,6 +594,7 @@ ptyManager.setExitHandler((id, exitCode) => {
return; // an install PTY has no agent/worktree to tear down
}
// Non-zero exit = install failed; leave its honest manual-fix message on screen.
+ analytics.track('agent_install_finished', { provider, rung: pending.rung, outcome: 'install_failed' });
}
teardownPty(id);
});
@@ -2499,6 +2505,16 @@ function findCodexHomeForSession(sessionId: string, siblingsRoot: string): strin
* ephemeral-worker watcher. */
type AgentSpawnOptions = SpawnOptions & { hive?: AgentMeta; isolate?: boolean; resume?: boolean; requireResume?: boolean; resumeSessionId?: string; provider?: AgentProvider; noAutoInstall?: boolean };
+/** Map a `ptyManager.spawn` failure string to the closed `agent_spawn_failed.reason`
+ * enum (analytics.ts). The two known strings come from PtyManager.spawn; anything
+ * else is a generic `spawn_error`. The raw message never leaves the machine — only
+ * the enum value does, per TELEMETRY.md. */
+function spawnFailReason(error?: string): SpawnFailReason {
+ if (error?.startsWith('cwd does not exist')) return 'cwd_missing';
+ if (error?.includes('already exists')) return 'already_running';
+ return 'spawn_error';
+}
+
ipcMain.handle('pty:spawn', async (evt, opts: AgentSpawnOptions) => {
if (!opts || typeof opts.id !== 'string' || typeof opts.cwd !== 'string' || typeof opts.command !== 'string') {
return { ok: false, error: 'invalid SpawnOptions' };
@@ -2534,6 +2550,11 @@ async function spawnAgentCore(opts: AgentSpawnOptions, owner: Electron.WebConten
const claudeProvider = isClaudeProvider(provider);
opts.provider = provider;
if (opts.hive) opts.hive = { ...opts.hive, provider };
+ // Activation-funnel entry (v0.4.7): every spawn REQUEST, so (attempted − spawned)
+ // measures the fallout the whole rebuild exists to see. Gated on !noAutoInstall so
+ // the missing-CLI relaunch (the only re-entry, index.ts install-exit handler) does
+ // NOT double-count a single user attempt — it is the SAME attempt continuing.
+ if (!opts.noAutoInstall) analytics.track('agent_spawn_attempted', { provider });
// ── Missing engine CLI → run its installer visibly (pre-spawn) ───────────────
// If the agent's engine binary (claude/codex/…) isn't installed, spawning it
// just dies with "— process exited (code 1) —" and the user has no idea why.
@@ -2587,7 +2608,18 @@ async function spawnAgentCore(opts: AgentSpawnOptions, owner: Electron.WebConten
// binary and die with the bare "process exited (code 1)" this whole path exists
// to replace.
if (res.ok && rung.command) {
- pendingInstallRelaunch.set(opts.id, { opts, owner, bin });
+ pendingInstallRelaunch.set(opts.id, { opts, owner, bin, rung: rung.kind });
+ // The auto-installer PTY is running; agent_install_finished on its exit says
+ // whether it actually produced an agent (rung is non-manual here by construction).
+ analytics.track('agent_install_started', { provider, rung: rung.kind });
+ } else if (res.ok) {
+ // Manual rung: the PTY only printed a hint (no installer to run, no relaunch
+ // armed), so no agent will start. This is the Mode 2 case that used to send
+ // NOTHING — an absent engine with no unattended install path.
+ analytics.track('agent_spawn_failed', { provider, reason: 'cli_missing' });
+ } else {
+ // The install PTY itself failed to spawn (cwd gone, id clash, throw).
+ analytics.track('agent_spawn_failed', { provider, reason: spawnFailReason(res.error) });
}
syncKeepAwake();
return res;
@@ -2887,6 +2919,7 @@ async function spawnAgentCore(opts: AgentSpawnOptions, owner: Electron.WebConten
}
const res = ptyManager.spawn(opts, owner);
if (res.ok) analytics.track('agent_spawned', { provider });
+ else analytics.track('agent_spawn_failed', { provider, reason: spawnFailReason(res.error) });
syncKeepAwake(); // arm the power-save blocker while ≥1 agent PTY is alive (#18)
// Hand the resolved worktree path back to the renderer so it can persist it on
// the agent (only set when isolation actually provisioned a worktree above).
@@ -3062,9 +3095,16 @@ ipcMain.handle('config:update', (_evt, patch: Partial<HarnessConfig>) => {
// relaunch, so bootstrap here on the null → set transition. Gated on the
// transition so ordinary config writes never re-enter it.
const hiveWasEnabled = hive.enabled();
+ const wasOnboarded = readConfig().onboardingComplete;
const next = writeConfig(patch);
// Live opt-in/out from Settings → Privacy (TELEMETRY.md).
if (typeof patch?.telemetryEnabled === 'boolean') analytics.setEnabled(patch.telemetryEnabled);
+ // Activation funnel (v0.4.7): onboarding just finished (false → true) — the top of
+ // the launch → first-agent funnel. `provider` is the engine chosen in the wizard.
+ // Fired here (main), not in the renderer, so it rides the same allowlist as the rest.
+ if (!wasOnboarded && next.onboardingComplete) {
+ analytics.track('onboarding_completed', { provider: next.godProvider ?? 'claude' });
+ }
// Keep the hive's mirror of the spawn gate current. The queue itself reads
// config per tick so it gates immediately; this is for the PROMPT, which is
// built per spawn, so flipping the toggle reaches god the next time he starts.
diff --git a/test/telemetry-funnel.test.cjs b/test/telemetry-funnel.test.cjs
new file mode 100644
index 00000000..66cd9c65
--- /dev/null
+++ b/test/telemetry-funnel.test.cjs
@@ -0,0 +1,107 @@
+'use strict';
+
+// Activation-funnel telemetry (v0.4.7): the 5 new events must pass the analytics
+// allowlist, enforce their closed enums (drop any unknown key), stay anonymous, and
+// be wired at the four spawn-funnel sites in index.ts. index.ts imports electron and
+// cannot load under plain node, so its wiring is asserted against the source text —
+// the same pattern the rest of the suite uses for main-process wiring.
+
+const test = require('node:test');
+const assert = require('node:assert/strict');
+const fs = require('node:fs');
+const os = require('node:os');
+const path = require('node:path');
+const loadTs = require('./load-ts.cjs');
+
+// Stub the build-time key + PostHog BEFORE analytics.ts loads, exactly like
+// update-applied.test.cjs, so the class runs end to end with no network or key.
+globalThis.__POSTHOG_KEY__ = 'test-key';
+globalThis.__POSTHOG_HOST__ = 'https://example.invalid';
+delete process.env.DO_NOT_TRACK;
+
+const captured = [];
+class FakePostHog {
+ capture(payload) { captured.push(payload); }
+ async shutdown() {}
+}
+const posthogPath = require.resolve('posthog-node');
+require.cache[posthogPath] = {
+ id: posthogPath, filename: posthogPath, loaded: true, exports: { PostHog: FakePostHog }
+};
+
+const { Analytics } = loadTs('src/main/analytics.ts');
+
+function freshDir() {
+ return fs.mkdtempSync(path.join(os.tmpdir(), 'md-telemetry-funnel-'));
+}
+
+function bootedAnalytics() {
+ const a = new Analytics();
+ a.init({ stateDir: freshDir(), appVersion: '0.4.7', enabled: true });
+ captured.length = 0; // drop the first_run / app_launched from init
+ return a;
+}
+
+// ── the allowlist accepts the new events and enforces their enums ─────────────
+
+test('all five funnel events are on the allowlist and carry their properties', () => {
+ const a = bootedAnalytics();
+ a.track('onboarding_completed', { provider: 'gemini' });
+ a.track('agent_spawn_attempted', { provider: 'claude' });
+ a.track('agent_spawn_failed', { provider: 'codex', reason: 'cli_missing' });
+ a.track('agent_install_started', { provider: 'claude', rung: 'npm' });
+ a.track('agent_install_finished', { provider: 'claude', rung: 'npm', outcome: 'install_failed' });
+
+ const byEvent = Object.fromEntries(captured.map((c) => [c.event, c.properties]));
+ assert.equal(byEvent.onboarding_completed.provider, 'gemini');
+ assert.equal(byEvent.agent_spawn_attempted.provider, 'claude');
+ assert.equal(byEvent.agent_spawn_failed.reason, 'cli_missing');
+ assert.equal(byEvent.agent_install_started.rung, 'npm');
+ assert.equal(byEvent.agent_install_finished.outcome, 'install_failed');
+ // Common props still stamped on the new events.
+ assert.equal(byEvent.agent_spawn_attempted.app_version, '0.4.7');
+});
+
+test('an unknown property on a new event is dropped, never sent', () => {
+ const a = bootedAnalytics();
+ // A caller that tries to smuggle a free-form value must not leak it.
+ a.track('agent_spawn_failed', { provider: 'claude', reason: 'spawn_error', cwd: '/Users/someone/secret-repo' });
+ const props = captured.find((c) => c.event === 'agent_spawn_failed').properties;
+ assert.equal(props.reason, 'spawn_error');
+ assert.equal(props.cwd, undefined, 'the un-allowlisted key must be dropped');
+});
+
+test('the new events stay anonymous by construction', () => {
+ const a = bootedAnalytics();
+ a.track('agent_install_finished', { provider: 'claude', rung: 'native', outcome: 'agent_launched' });
+ const props = captured.find((c) => c.event === 'agent_install_finished').properties;
+ assert.equal(props.$process_person_profile, false);
+});
+
+// ── index.ts wires the funnel at the four spawn sites ─────────────────────────
+
+const main = fs.readFileSync(path.resolve(__dirname, '..', 'src/main/index.ts'), 'utf8');
+
+test('agent_spawn_attempted fires once per attempt, gated against the install relaunch', () => {
+ assert.match(main, /if \(!opts\.noAutoInstall\) analytics\.track\('agent_spawn_attempted', \{ provider \}\);/);
+});
+
+test('the real spawn reports both success and failure', () => {
+ assert.match(main, /if \(res\.ok\) analytics\.track\('agent_spawned', \{ provider \}\);/);
+ assert.match(main, /else analytics\.track\('agent_spawn_failed', \{ provider, reason: spawnFailReason\(res\.error\) \}\);/);
+});
+
+test('the missing-CLI branch distinguishes installer-running, Mode 2, and spawn error', () => {
+ assert.match(main, /analytics\.track\('agent_install_started', \{ provider, rung: rung\.kind \}\);/);
+ assert.match(main, /analytics\.track\('agent_spawn_failed', \{ provider, reason: 'cli_missing' \}\);/);
+});
+
+test('the install-PTY exit reports whether the auto-installer completed', () => {
+ assert.match(main, /analytics\.track\('agent_install_finished', \{ provider, rung: pending\.rung, outcome: 'agent_launched' \}\);/);
+ assert.match(main, /analytics\.track\('agent_install_finished', \{ provider, rung: pending\.rung, outcome: 'install_failed' \}\);/);
+});
+
+test('onboarding completion is reported on the false to true transition', () => {
+ assert.match(main, /if \(!wasOnboarded && next\.onboardingComplete\) \{/);
+ assert.match(main, /analytics\.track\('onboarding_completed', \{ provider: next\.godProvider \?\? 'claude' \}\);/);
+});