Skip to content

Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.

Usage telemetry: tell people from bots, see managed-agent sessions, track activation and Switch Cloud, count per period (#1373)

Christian McDermott65e352123916
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1964102c8..c2dfe3847 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -216,6 +216,39 @@ version of their own to them without also giving them a release of their own.
compares it only when someone signs in.
#### Changed
+- **Usage telemetry tells people from bots.** Slack, Discord and Mattermost
+ posts from an integration (an app, a bot account, a webhook that isn't
+ Switch's own) were counted as people. The bridge now marks them, and the
+ daily snapshot reports `person_identity_active_*`, `bot_identity_active_*`
+ and `account_active_*` beside the old `chat_identity_active_*` and
+ `user_active_*`, which are still sent for one more stable release. Room and
+ agent message events report such posts as `sender_kind = bot`. An app's post
+ on Discord or Mattermost also no longer answers a request card or wakes an
+ agent following the conversation, as already on Slack.
+- **Usage telemetry follows Switch Cloud.** Every event now carries
+ `deployment_kind` (`switch_cloud`, `managed_by_console` or `self_hosted`),
+ so Cloud can be charted on its own or left out. New events cover its funnel:
+ `account_created` (sign-up, a first single sign-on sign-in, or an admin),
+ `signup_refused` with the reason, `workspace_created`, and a cloud machine
+ being requested, becoming ready (with how long a new one took), failing
+ (with the reason), being woken, and being stopped. Machine events are sent
+ only once their change is committed.
+- **Usage telemetry counts each finished day, week and month.** The daily
+ snapshot's `*_7d` counts are a trailing week, so a chart adding them up
+ over a week counted a room once for every day it was active. A new
+ `usage_period` event is sent once per finished UTC day, ISO week and
+ calendar month, with distinct active rooms, people and accounts for exactly
+ that period, dated to its middle. Summed at the matching interval it is the
+ real figure. A server that was off sends the periods it missed, up to 35
+ days back, when it next runs.
+- **Agent telemetry says connection where it meant one.**
+ `agent_connected`, `agent_disconnected` and the `first_agent_connected`
+ milestone are sent alongside `agent_session_started`, `agent_session_ended`
+ and `first_session_started`, which measured an agent coming online, not a
+ coding-agent session. The old names stay for one more stable release. The
+ new `first_agent_session` milestone is the first session a host reports. A
+ deployment that already passed either milestone claims it at startup
+ without reporting it.
- **Workspaces belong to organizations. Upgrade by replacing every pod, and
downgrade the database before rolling back.** Every existing workspace gets
an organization of its own, and its owners become the organization's admins.
@@ -2398,6 +2431,20 @@ version of their own to them without also giving them a release of their own.
removed.
#### Changed
+- **Usage data follows an install to a working agent.** Three new
+ `activation_milestone` steps (`server_ready`, `agent_ready`,
+ `first_agent_session`) are each sent once per install, and only by installs
+ made on this version or later, with `path` saying whether the install
+ started switch-buddy or skipped it. Sessions of agents on the agent
+ controller Console runs on this computer, which never reached
+ `session_started`, are sent as `managed_session_started`. Updates send
+ `update_download_started` and, on the first launch after one,
+ `update_installed` with the version it came from. Adding a server by
+ address, starting a server Console runs, and creating a room now say why
+ they failed where they said `error`, and Switch Cloud is its own
+ `server_kind`. An install once signed in with a staff account stays
+ labelled internal after it signs out. `agent_reset`, which was listed but
+ never sent, is gone from the list. `docs/TELEMETRY.md` has every field.
- **Managed agents and machines update when the server says they changed.**
Against a server that pushes change notices, Console reads those lists
again when told to, and otherwise only every five minutes instead of every
@@ -4457,6 +4504,11 @@ tags; see RELEASING.md.
### [Unreleased]
#### Added
+- **Tells Switch Console when a session started.** Once Switch has taken a
+ session host's start report, the controller Console runs on its own
+ computer passes the session's provider and how it started to Console over
+ its IPC channel, for Console's usage data. A controller with no parent
+ listening, on an SSH host or run as a service, sends nothing.
- **Waits its turn when Switch is busy opening connections.** An open or an
attach refused as `busy` comes with when to try again, the controller's
place in the server's queue, and the controller now waits that long
diff --git a/console/AGENTS.md b/console/AGENTS.md
index 6cb0a0dc2..cf4db9b01 100644
--- a/console/AGENTS.md
+++ b/console/AGENTS.md
@@ -507,6 +507,10 @@ pnpm run lint
from there. What a session host does tell is the Switch server it already works for:
that a new session started and how (`host/session-start.ts`), as ordinary traffic to
that server. Whether that leaves the server as telemetry is the server's own setting.
+ The agent controller this app runs on its own computer still reports nothing itself:
+ once Switch has taken a session's start it tells its parent over the IPC channel
+ (`host/session-started.ts`), and this app reports `managed_session_started` under its
+ own gate. A controller on any other machine has no parent listening; keep it that way.
- **Redaction is split by destination, and both halves must be preserved:**
- **Secrets** (tokens, keys, JWTs, PEM blocks, URL credentials) are redacted on the
write path by `redactSecrets()` and must never reach disk.
diff --git a/console/apps/switch-console-desktop/src/main/core/embedded-controller/controller-supervisor.test.ts b/console/apps/switch-console-desktop/src/main/core/embedded-controller/controller-supervisor.test.ts
index 66d23e398..0c0cf5048 100644
--- a/console/apps/switch-console-desktop/src/main/core/embedded-controller/controller-supervisor.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/embedded-controller/controller-supervisor.test.ts
@@ -1,6 +1,8 @@
import {
isProviderRecheckRequest,
PROVIDER_RECHECKED_KIND,
+ SESSION_STARTED_KIND,
+ type SessionStartedNotice,
STOP_MESSAGE,
} from '@switch-console/agent-providers';
import { describe, expect, it } from 'vitest';
@@ -19,16 +21,18 @@ function harness(launch: () => Promise<ControllerLaunch> = async () => LAUNCH) {
const phases: EmbeddedControllerPhase[] = [];
const finals: string[] = [];
const lines: [ControllerLogLevel, string][] = [];
+ const starts: SessionStartedNotice[] = [];
const supervisor = new ControllerSupervisor({
spawn,
launch,
onPhase: (phase) => phases.push(phase),
onFinal: (exit) => finals.push(exit),
onLine: (level, line) => lines.push([level, line]),
+ onSessionStarted: (notice) => starts.push(notice),
now: () => NOW,
backoff: { initialMs: 10, maxMs: 40, stableMs: 10_000 },
});
- return { supervisor, calls, phases, finals, lines };
+ return { supervisor, calls, phases, finals, lines, starts };
}
const LAUNCH: ControllerLaunch = {
@@ -73,6 +77,24 @@ describe('ControllerSupervisor', () => {
await supervisor.stop(1_000);
});
+ it('passes on the session starts the controller tells it of, and nothing else it sends', async () => {
+ const { supervisor, calls, starts } = harness();
+ supervisor.start();
+ await waitFor(() => calls.length === 1, 'the spawn');
+ const child = calls[0]!.child;
+ const notice: SessionStartedNotice = {
+ kind: SESSION_STARTED_KIND,
+ sessionId: 'session-1',
+ provider: 'claude',
+ startSource: 'room',
+ };
+ child.emit('message', notice);
+ child.emit('message', { ...notice, startSource: 'somewhere' });
+ child.emit('message', { kind: PROVIDER_RECHECKED_KIND, id: 'x', error: null });
+ expect(starts).toEqual([notice]);
+ await supervisor.stop(1_000);
+ });
+
it('restarts a controller that exits on its own, backing off up to the cap', async () => {
const { supervisor, calls, phases } = harness();
supervisor.start();
diff --git a/console/apps/switch-console-desktop/src/main/core/embedded-controller/controller-supervisor.ts b/console/apps/switch-console-desktop/src/main/core/embedded-controller/controller-supervisor.ts
index a851d820c..222620411 100644
--- a/console/apps/switch-console-desktop/src/main/core/embedded-controller/controller-supervisor.ts
+++ b/console/apps/switch-console-desktop/src/main/core/embedded-controller/controller-supervisor.ts
@@ -3,8 +3,10 @@ import { createInterface } from 'node:readline';
import type { Readable, Writable } from 'node:stream';
import {
isProviderRecheckReply,
+ isSessionStartedNotice,
PROVIDER_RECHECK_KIND,
type ProviderRecheckRequest,
+ type SessionStartedNotice,
STOP_MESSAGE,
} from '@switch-console/agent-providers';
import type { EmbeddedControllerPhase } from '@shared/core/embedded-controller/embedded-controller';
@@ -81,6 +83,8 @@ export type SupervisorDeps = {
onFinal: (exit: ControllerFinalExit) => void;
/** One line the controller wrote, at the level it wrote it at. */
onLine: (level: ControllerLogLevel, line: string) => void;
+ /** A session of one of its agents started, and Switch took the report. */
+ onSessionStarted: (notice: SessionStartedNotice) => void;
now: () => number;
backoff: ControllerBackoff;
};
@@ -227,6 +231,9 @@ export class ControllerSupervisor {
};
child.once('error', (error) => finish(null, null, error));
child.once('exit', (code, signal) => finish(code, signal, null));
+ child.on('message', (message) => {
+ if (isSessionStartedNotice(message)) this.deps.onSessionStarted(message);
+ });
this.read(child.stdout, 'stdout');
this.read(child.stderr, 'stderr');
// A child that died before reading closes the pipe under us; its exit says why.
diff --git a/console/apps/switch-console-desktop/src/main/core/embedded-controller/embedded-controller-service.test.ts b/console/apps/switch-console-desktop/src/main/core/embedded-controller/embedded-controller-service.test.ts
index 0ededdc31..626aa009e 100644
--- a/console/apps/switch-console-desktop/src/main/core/embedded-controller/embedded-controller-service.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/embedded-controller/embedded-controller-service.test.ts
@@ -85,6 +85,7 @@ function service(overrides: Partial<EmbeddedControllerDeps> = {}): EmbeddedContr
emit: (event) => events.push(event),
log: { info: () => {}, warn: () => {}, error: () => {} },
controllerLine: () => {},
+ sessionStarted: () => {},
now: () => NOW,
backoff: { initialMs: 10, maxMs: 40, stableMs: 10_000 },
revokeGraceMs: 200,
diff --git a/console/apps/switch-console-desktop/src/main/core/embedded-controller/embedded-controller-service.ts b/console/apps/switch-console-desktop/src/main/core/embedded-controller/embedded-controller-service.ts
index 8d512f4a5..4bd9e0c1d 100644
--- a/console/apps/switch-console-desktop/src/main/core/embedded-controller/embedded-controller-service.ts
+++ b/console/apps/switch-console-desktop/src/main/core/embedded-controller/embedded-controller-service.ts
@@ -1,3 +1,4 @@
+import type { SessionStartedNotice } from '@switch-console/agent-providers';
import { MovedAgentsHereError } from '@shared/core/agent-migration/agent-migration';
import type {
EmbeddedControllerEnrollment,
@@ -80,6 +81,8 @@ export type EmbeddedControllerDeps = {
log: ServiceLog;
/** Where the controller's own log lines go. */
controllerLine: (serverId: string, level: ControllerLogLevel, line: string) => void;
+ /** A session of an agent on this computer started (`reportManagedSessionStarted`). */
+ sessionStarted: (notice: SessionStartedNotice) => void;
now: () => number;
backoff: ControllerBackoff;
/** How long a revoked controller is given to stop its agents and exit by itself. */
@@ -463,6 +466,7 @@ export class EmbeddedControllerService {
onPhase: (phase) => this.setRunnerPhase(serverId, phase),
onFinal: (exit) => void this.onFinal(serverId, exit),
onLine: (level, line) => this.deps.controllerLine(serverId, level, line),
+ onSessionStarted: (notice) => this.deps.sessionStarted(notice),
now: this.deps.now,
backoff: this.deps.backoff,
}),
diff --git a/console/apps/switch-console-desktop/src/main/core/embedded-controller/embedded-controllers.ts b/console/apps/switch-console-desktop/src/main/core/embedded-controller/embedded-controllers.ts
index 8ee64fb7b..93e55cf17 100644
--- a/console/apps/switch-console-desktop/src/main/core/embedded-controller/embedded-controllers.ts
+++ b/console/apps/switch-console-desktop/src/main/core/embedded-controller/embedded-controllers.ts
@@ -12,6 +12,7 @@ import {
import { getAgentById } from '@main/core/agents/getAgentById';
import { encryptedAppSecretsStore } from '@main/core/secrets/encrypted-app-secrets-store';
import { getServer } from '@main/core/switch-servers/servers-store';
+import { reportManagedSessionStarted } from '@main/core/telemetry/managed-session';
import { events } from '@main/lib/events';
import { log } from '@main/lib/logger';
import { embeddedControllerStateChannel } from '@shared/events/embeddedControllerEvents';
@@ -139,6 +140,7 @@ export const embeddedControllerService = new EmbeddedControllerService({
error: (message, fields) => log.error(message, { event: 'embedded_controller', ...fields }),
},
controllerLine: (serverId, level, line) => controllerLog[level](line, { serverId }),
+ sessionStarted: reportManagedSessionStarted,
now: Date.now,
backoff: DEFAULT_BACKOFF,
revokeGraceMs: 20_000,
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.test.ts
index 9285c05d9..bf962103d 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.test.ts
@@ -286,7 +286,11 @@ it('surfaces a failed backup with its error and refuses sessions until a retry s
it('records an upgrade that never turned healthy as failed', async () => {
m.start.mockImplementation(
- upgradingStart({ kind: 'error', message: 'The server did not become healthy in time.' })
+ upgradingStart({
+ kind: 'error',
+ message: 'The server did not become healthy in time.',
+ failure: 'health_timeout',
+ })
);
const service = new LocalServerService();
await boot(service);
@@ -351,7 +355,9 @@ it('a Start click during the boot upgrade joins it instead of starting twice', a
});
it('keeps an owed upgrade pending across a stop', async () => {
- m.start.mockImplementationOnce(upgradingStart({ kind: 'error', message: 'unhealthy' }));
+ m.start.mockImplementationOnce(
+ upgradingStart({ kind: 'error', message: 'unhealthy', failure: 'health_timeout' })
+ );
const service = new LocalServerService();
await boot(service);
await service.stop();
@@ -391,7 +397,11 @@ it('records what happens to the stack: an update, a start, a stop, a failed star
m.record.mockClear();
await service.stop();
- m.start.mockResolvedValue({ kind: 'error', message: 'bind: address already in use' });
+ m.start.mockResolvedValue({
+ kind: 'error',
+ message: 'bind: address already in use',
+ failure: 'unexpected',
+ });
await service.start();
expect(m.record.mock.calls).toEqual([
['stopped', null],
@@ -451,7 +461,11 @@ it('reports a stop that errored with the stack still up, or unreadable, as a fai
});
// The next start says nothing about the stop any more.
- m.start.mockResolvedValue({ kind: 'error', message: 'bind: address already in use' });
+ m.start.mockResolvedValue({
+ kind: 'error',
+ message: 'bind: address already in use',
+ failure: 'unexpected',
+ });
await service.start();
expect(service.getStatus()).toMatchObject({ phase: 'error', failedTo: null });
});
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.ts
index dfc8bf9ed..941096155 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.ts
@@ -565,7 +565,11 @@ export class LocalServerService {
private async runStart(activate: boolean): Promise<StartLocalServerResult> {
if (this.busy) {
- return { kind: 'error', message: 'A local-server operation is already in progress.' };
+ return {
+ kind: 'error',
+ message: 'A local-server operation is already in progress.',
+ failure: 'operation_in_progress',
+ };
}
this.busy = true;
this.startAbort = new AbortController();
@@ -707,7 +711,7 @@ export class LocalServerService {
this.failUpgrade(message);
await recordLocalActivity('start-failed', message);
reportManagedServerStartThrew('local');
- return { kind: 'error', message };
+ return { kind: 'error', message, failure: 'unexpected' };
} finally {
host.dispose();
this.busy = false;
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/native/native-stack.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/native/native-stack.ts
index 0c8086e37..be354cf27 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/native/native-stack.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/native/native-stack.ts
@@ -311,6 +311,7 @@ export async function startNativeStack(opts: NativeStartOptions): Promise<Native
message:
'The local server cannot run as root: PostgreSQL refuses to. Start Switch Console as ' +
'a normal user, then set up the server.',
+ failure: 'running_as_root',
},
bringUpMattermost: null,
};
@@ -445,7 +446,11 @@ export async function startNativeStack(opts: NativeStartOptions): Promise<Native
? ''
: `: ${reason.length > room ? `${reason.slice(0, room - 1)}…` : reason}`;
return {
- result: { kind: 'error', message: `${head}${shown}${tail}` },
+ result: {
+ kind: 'error',
+ message: `${head}${shown}${tail}`,
+ failure: alive ? 'health_timeout' : 'core_exited',
+ },
bringUpMattermost: null,
};
}
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-shared-host.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-shared-host.test.ts
index 37d5665bb..373d986f1 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-shared-host.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-shared-host.test.ts
@@ -324,6 +324,7 @@ describe('starting a shared stack', () => {
expect(await startStack(startOptions(host))).toEqual({
kind: 'error',
message: 'The server did not become healthy in time.',
+ failure: 'health_timeout',
});
expect(ensureManagedServerMock).not.toHaveBeenCalled();
});
@@ -475,6 +476,7 @@ describe('starting a shared stack', () => {
expect(await startStack(startOptions(host))).toEqual({
kind: 'error',
message: 'unshared vm-1 /home/alice/.switchdash/switch-server',
+ failure: 'start_refused',
});
expect(writeFile).not.toHaveBeenCalled();
expect(publishEnvMock).not.toHaveBeenCalled();
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.ts
index e251f9fa9..ca31cbdf6 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.ts
@@ -546,7 +546,7 @@ export async function startStack(opts: StartStackOptions): Promise<StartLocalSer
log.error(`managed-switch-server: refusing to start the stack on ${host.label}`, {
reason: plan.message,
});
- return { kind: 'error', message: plan.message };
+ return { kind: 'error', message: plan.message, failure: 'start_refused' };
}
// Before anything reads the working dir: another account may have changed
@@ -652,7 +652,11 @@ export async function startStack(opts: StartStackOptions): Promise<StartLocalSer
onMessage('Waiting for the server to become healthy…');
if ((await unansweredAddress(settings.ports, signal)) !== null) {
- return { kind: 'error', message: 'The server did not become healthy in time.' };
+ return {
+ kind: 'error',
+ message: 'The server did not become healthy in time.',
+ failure: 'health_timeout',
+ };
}
const serverId = await registerAndSignIn(ref, serverName, settings, activate, onMessage);
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.ts
index 50a3cb80f..50b78c818 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.ts
@@ -767,7 +767,11 @@ export class RemoteServerService {
action: 'starting' | 'updating'
): Promise<StartRemoteServerResult> {
if (this.busy.has(sshHost)) {
- return { kind: 'error', message: `An operation is already in progress for ${sshHost}.` };
+ return {
+ kind: 'error',
+ message: `An operation is already in progress for ${sshHost}.`,
+ failure: 'operation_in_progress',
+ };
}
hostReachabilityService.requireReachable(sshHost);
this.busy.add(sshHost);
@@ -873,7 +877,7 @@ export class RemoteServerService {
this.setStatus(sshHost, { phase: 'error', error: message });
this.failUpgrade(sshHost, message);
reportManagedServerStartThrew('remote');
- return { kind: 'error', message };
+ return { kind: 'error', message, failure: 'unexpected' };
} finally {
this.busy.delete(sshHost);
this.startAborts.delete(sshHost);
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.upgrade.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.upgrade.test.ts
index 7c3c4e46a..0f0cb1067 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.upgrade.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.upgrade.test.ts
@@ -318,7 +318,11 @@ it('upgrades a host that was unreachable at boot once it comes back, holding ses
it('surfaces a failed upgrade with its error and retries when the host is reconciled again', async () => {
m.start.mockImplementationOnce(
- upgradingStart({ kind: 'error', message: 'The server did not become healthy in time.' })
+ upgradingStart({
+ kind: 'error',
+ message: 'The server did not become healthy in time.',
+ failure: 'health_timeout',
+ })
);
const service = new RemoteServerService();
const upgraded = vi.fn();
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/controller.test.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/controller.test.ts
index fe0fdfd4f..402649fef 100644
--- a/console/apps/switch-console-desktop/src/main/core/switch-servers/controller.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/controller.test.ts
@@ -32,6 +32,10 @@ const followServerApiUrl = vi.hoisted(() => vi.fn());
const listCloudMachines = vi.hoisted(() => vi.fn());
const ensureCloudMachine = vi.hoisted(() => vi.fn());
const serverFlags = vi.hoisted(() => ({ hosted_agents: true }));
+const { DuplicateServerUrlError, NotTheServerAddressError } = vi.hoisted(() => ({
+ DuplicateServerUrlError: class extends Error {},
+ NotTheServerAddressError: class extends Error {},
+}));
// Stub the modules the controller imports that would otherwise pull electron /
// ssh / agent side effects at load.
@@ -113,10 +117,12 @@ vi.mock('./gateway-client', () => ({
vi.mock('./server-address', () => ({
assertServerAddress,
assertServesDashboard,
+ NotTheServerAddressError,
}));
vi.mock('./servers-store', () => ({
getServer,
addServer,
+ DuplicateServerUrlError,
findServerByUrl,
findServerByWebAddress,
deleteSessionCookie: vi.fn(),
@@ -730,6 +736,7 @@ describe('adding a server by URL', () => {
expect(trackEvent).toHaveBeenCalledExactlyOnceWith('server_added', {
server_kind: 'external',
outcome: 'success',
+ failure_reason: 'none',
});
});
@@ -745,7 +752,9 @@ describe('adding a server by URL', () => {
});
it('adds nothing, and reports the failure, when the address is a dashboard’s', async () => {
- assertServerAddress.mockRejectedValueOnce(new Error('answered with a web page'));
+ assertServerAddress.mockRejectedValueOnce(
+ new NotTheServerAddressError('answered with a web page')
+ );
await expect(switchServersController.addServer(params)).rejects.toThrow(
'answered with a web page'
@@ -754,6 +763,18 @@ describe('adding a server by URL', () => {
expect(trackEvent).toHaveBeenCalledExactlyOnceWith('server_added', {
server_kind: 'external',
outcome: 'failure',
+ failure_reason: 'not_the_server_address',
+ });
+ });
+
+ it('reports an address another server already has as already added', async () => {
+ addServer.mockRejectedValue(new DuplicateServerUrlError('already the address of S'));
+
+ await expect(switchServersController.addServer(params)).rejects.toThrow('already');
+ expect(trackEvent).toHaveBeenCalledExactlyOnceWith('server_added', {
+ server_kind: 'external',
+ outcome: 'failure',
+ failure_reason: 'already_added',
});
});
@@ -765,6 +786,7 @@ describe('adding a server by URL', () => {
expect(trackEvent).toHaveBeenCalledWith('server_added', {
server_kind: 'external',
outcome: 'failure',
+ failure_reason: 'error',
});
});
});
@@ -797,8 +819,9 @@ describe('connecting to Switch Cloud', () => {
});
expect(assertServerAddress).not.toHaveBeenCalled();
expect(trackEvent).toHaveBeenCalledWith('server_added', {
- server_kind: 'external',
+ server_kind: 'cloud',
outcome: 'success',
+ failure_reason: 'none',
});
});
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/controller.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/controller.ts
index 235c3b8f9..25ef87eb4 100644
--- a/console/apps/switch-console-desktop/src/main/core/switch-servers/controller.ts
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/controller.ts
@@ -6,7 +6,11 @@ import { resetChatStream } from '@main/core/chats/chat-streams';
import { embeddedControllerService } from '@main/core/embedded-controller/embedded-controllers';
import { featureFlagsService } from '@main/core/feature-flags/feature-flags';
import { isManagedServerRunning } from '@main/core/managed-switch-server/managed-server-status';
-import type { TelemetryAuthMethod, TelemetrySignInFailure } from '@main/core/telemetry/events';
+import type {
+ TelemetryAuthMethod,
+ TelemetryServerAddFailure,
+ TelemetrySignInFailure,
+} from '@main/core/telemetry/events';
import { trackEvent } from '@main/core/telemetry/telemetry-service';
import { reconcileServerWorkspaces } from '@main/core/workspaces/reconcile-workspaces';
import {
@@ -109,9 +113,14 @@ import {
} from './github-browser-flow';
import { deleteManagedClaudeCredential } from './managed-claude-credential';
import { hostUnreachable, requireReachableServer, requireServer } from './require-server';
-import { assertServerAddress, assertServesDashboard } from './server-address';
+import {
+ assertServerAddress,
+ assertServesDashboard,
+ NotTheServerAddressError,
+} from './server-address';
import {
addServer,
+ DuplicateServerUrlError,
deleteSessionCookie,
findServerByUrl,
findServerByWebAddress,
@@ -224,6 +233,13 @@ function reportSignIn(
});
}
+/** Why registering an address failed, as a reportable code. */
+function serverAddFailureOf(error: unknown): TelemetryServerAddFailure {
+ if (error instanceof DuplicateServerUrlError) return 'already_added';
+ if (error instanceof NotTheServerAddressError) return 'not_the_server_address';
+ return 'error';
+}
+
/**
* Register Switch Cloud as a server, or hand back the one already registered.
*
@@ -239,10 +255,14 @@ async function registerSwitchCloud({ url }: SwitchCloudEndpoint): Promise<Switch
try {
server = await addServer({ name: SWITCH_CLOUD_NAME, url });
} catch (error) {
- trackEvent('server_added', { server_kind: 'external', outcome: 'failure' });
+ trackEvent('server_added', {
+ server_kind: 'cloud',
+ outcome: 'failure',
+ failure_reason: serverAddFailureOf(error),
+ });
throw error;
}
- trackEvent('server_added', { server_kind: 'external', outcome: 'success' });
+ trackEvent('server_added', { server_kind: 'cloud', outcome: 'success', failure_reason: 'none' });
return server;
}
@@ -314,10 +334,18 @@ export const switchServersController = createRPCController({
await assertServerAddress(params.url);
server = await addServer(params);
} catch (error) {
- trackEvent('server_added', { server_kind: 'external', outcome: 'failure' });
+ trackEvent('server_added', {
+ server_kind: 'external',
+ outcome: 'failure',
+ failure_reason: serverAddFailureOf(error),
+ });
throw error;
}
- trackEvent('server_added', { server_kind: 'external', outcome: 'success' });
+ trackEvent('server_added', {
+ server_kind: 'external',
+ outcome: 'success',
+ failure_reason: 'none',
+ });
return server;
},
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.db.test.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.db.test.ts
index 9f858911d..e11b8de19 100644
--- a/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.db.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.db.test.ts
@@ -308,6 +308,7 @@ describe('servers-store: rename & delete', () => {
expect(telemetryMocks.trackEvent).toHaveBeenCalledWith('server_added', {
server_kind: 'local',
outcome: 'success',
+ failure_reason: 'none',
});
});
@@ -325,6 +326,7 @@ describe('servers-store: rename & delete', () => {
expect(telemetryMocks.trackEvent).toHaveBeenCalledWith('server_added', {
server_kind: 'remote_managed',
outcome: 'success',
+ failure_reason: 'none',
});
});
@@ -376,6 +378,22 @@ describe('servers-store: rename & delete', () => {
});
});
+ it('reports Switch Cloud as its own kind', async () => {
+ vi.stubEnv('SWITCH_CLOUD_URL', 'https://cloud.example.com');
+ await fixture.db.insert(switchServers).values({
+ id: 'srv-cloud',
+ name: 'Switch Cloud',
+ url: 'https://cloud.example.com/',
+ });
+
+ await removeServer('srv-cloud');
+ vi.unstubAllEnvs();
+
+ expect(telemetryMocks.trackEvent).toHaveBeenCalledWith('server_removed', {
+ server_kind: 'cloud',
+ });
+ });
+
it('reports a managed server by the kind it was managed as', async () => {
await fixture.db.insert(switchServers).values({
id: 'srv-rm',
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.ts
index 68756648d..c641d84dc 100644
--- a/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.ts
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.ts
@@ -1,7 +1,7 @@
import { randomUUID } from 'node:crypto';
import { and, desc, eq, isNull, ne, or, sql } from 'drizzle-orm';
import { encryptedAppSecretsStore } from '@main/core/secrets/encrypted-app-secrets-store';
-import type { TelemetryEventMap } from '@main/core/telemetry/events';
+import type { TelemetryServerKind } from '@main/core/telemetry/events';
import { forgetSessionAccount, recordSessionAccount } from '@main/core/telemetry/internal-account';
import { trackEvent } from '@main/core/telemetry/telemetry-service';
import { forgetServerSession } from '@main/core/workspaces/workspace-session';
@@ -29,6 +29,7 @@ import {
} from '@shared/core/switch-servers/switch-servers';
import { workspaceUnavailability } from '@shared/core/workspaces/workspaces';
import { deleteManagedClaudeCredential } from './managed-claude-credential';
+import { switchCloudEndpoint } from './switch-cloud';
// Keyed to the server, not the workspace: one gateway session cookie covers
// every workspace on a server.
@@ -259,6 +260,7 @@ export async function ensureManagedServer(
trackEvent('server_added', {
server_kind: ref.kind === 'remote' ? 'remote_managed' : 'local',
outcome: 'success',
+ failure_reason: 'none',
});
}
return server;
@@ -435,11 +437,25 @@ export async function removeServer(id: string): Promise<void> {
}
/** The reported kind of a server, in the same terms `server_added` uses. */
-export function serverKindOf(
- server: SwitchServer
-): TelemetryEventMap['server_added']['server_kind'] {
- if (!server.managed) return 'external';
- return server.managementKind === 'remote' ? 'remote_managed' : 'local';
+export function serverKindOf(server: SwitchServer): TelemetryServerKind {
+ if (server.managed) return server.managementKind === 'remote' ? 'remote_managed' : 'local';
+ return isSwitchCloud(server.url) ? 'cloud' : 'external';
+}
+
+/**
+ * Whether `url` is Switch Cloud's, as this build or run names it. A build that
+ * names no Cloud, or names one it cannot parse, has no Cloud server to count:
+ * the misconfiguration surfaces where the Cloud is offered, not here, where it
+ * would fail whatever is being reported on.
+ */
+function isSwitchCloud(url: string): boolean {
+ let cloud: ReturnType<typeof switchCloudEndpoint>;
+ try {
+ cloud = switchCloudEndpoint();
+ } catch {
+ return false;
+ }
+ return cloud !== null && normaliseUrl(cloud.url) === normaliseUrl(url);
}
/**
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/activation.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/activation.test.ts
new file mode 100644
index 000000000..573199e8b
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/activation.test.ts
@@ -0,0 +1,237 @@
+import { beforeEach, describe, expect, it, vi } from 'vitest';
+import type { TelemetryEvent, TelemetryInstallKind } from './events';
+
+/**
+ * The activation milestones: which sent events mark them, that each is
+ * reported once for the life of an install, and only by an install that began
+ * on a version reporting them.
+ *
+ * Each test loads the module afresh, as a new launch would, over records that
+ * outlive it the way the database does.
+ */
+
+const { records, launch, unwritable } = vi.hoisted(() => ({
+ records: new Map<string, unknown>(),
+ launch: { kind: 'new' as TelemetryInstallKind | 'unknown' },
+ unwritable: { on: false },
+}));
+
+vi.mock('@main/db/kv', () => ({
+ KV: class {
+ async get(key: string) {
+ return records.get(key) ?? null;
+ }
+ async setOrThrow(key: string, value: unknown) {
+ if (unwritable.on) throw new Error('kv: write failed');
+ records.set(key, value);
+ }
+ },
+}));
+vi.mock('./launch-history', () => ({ launchInstallKind: async () => launch.kind }));
+vi.mock('./telemetry-service', () => ({
+ trackEvent: vi.fn(),
+ telemetryService: { setSentListener: vi.fn() },
+}));
+vi.mock('@main/lib/logger', () => ({
+ log: { info: vi.fn(), warn: vi.fn(), error: vi.fn(), debug: vi.fn() },
+}));
+
+/** A launch of the app: the module as it loads, with nothing in memory. */
+async function launchApp() {
+ vi.resetModules();
+ vi.clearAllMocks();
+ const activation = await import('./activation');
+ const { trackEvent } = await import('./telemetry-service');
+ const { log } = await import('@main/lib/logger');
+ return { ...activation, trackEvent: vi.mocked(trackEvent), log: vi.mocked(log) };
+}
+
+const settle = () => new Promise((resolve) => setTimeout(resolve, 0));
+
+const SIGNED_IN: TelemetryEvent = {
+ name: 'server_sign_in',
+ properties: {
+ auth_method: 'password',
+ server_kind: 'external',
+ outcome: 'success',
+ failure_reason: 'none',
+ },
+};
+const AGENT_CREATED: TelemetryEvent = {
+ name: 'agent_created',
+ properties: {
+ agent_type: 'claude',
+ location: 'local',
+ outcome: 'success',
+ failure_reason: 'none',
+ entry_point: 'onboarding',
+ },
+};
+const MANAGED_SESSION: TelemetryEvent = {
+ name: 'managed_session_started',
+ properties: { agent_type: 'claude', start_source: 'room' },
+};
+
+beforeEach(() => {
+ records.clear();
+ launch.kind = 'new';
+ unwritable.on = false;
+});
+
+describe('which events mark a milestone', () => {
+ it('names the milestone each successful step marks, and nothing for the rest', async () => {
+ const { milestoneOf } = await launchApp();
+ const failedSignIn: TelemetryEvent = {
+ name: 'server_sign_in',
+ properties: { ...SIGNED_IN.properties, outcome: 'failure', failure_reason: 'failed' },
+ } as TelemetryEvent;
+ const managedStart = (action: 'start' | 'stop', outcome: 'success' | 'failure') =>
+ ({
+ name: 'managed_server_action',
+ properties: {
+ action,
+ target: 'local',
+ outcome,
+ failure_reason: outcome === 'success' ? 'none' : 'error',
+ docker_available: 'available',
+ },
+ }) as TelemetryEvent;
+
+ expect(milestoneOf(SIGNED_IN)).toBe('server_ready');
+ expect(milestoneOf(failedSignIn)).toBeNull();
+ expect(milestoneOf(managedStart('start', 'success'))).toBe('server_ready');
+ expect(milestoneOf(managedStart('start', 'failure'))).toBeNull();
+ expect(milestoneOf(managedStart('stop', 'success'))).toBeNull();
+ expect(milestoneOf(AGENT_CREATED)).toBe('agent_ready');
+ expect(
+ milestoneOf({
+ name: 'session_started',
+ properties: {
+ agent_type: 'codex',
+ location: 'remote',
+ outcome: 'success',
+ failure_reason: 'none',
+ entry_point: 'session_list',
+ start_source: 'user',
+ has_initial_prompt: false,
+ connected_to_room: false,
+ },
+ })
+ ).toBe('first_agent_session');
+ expect(milestoneOf(MANAGED_SESSION)).toBe('first_agent_session');
+ expect(milestoneOf({ name: 'app_launched', properties: { install_kind: 'new' } })).toBeNull();
+ });
+});
+
+describe('a new install', () => {
+ it('reports each milestone once, even across launches', async () => {
+ const first = await launchApp();
+ first.noteSentEvent(SIGNED_IN);
+ first.noteSentEvent(SIGNED_IN);
+ first.noteSentEvent(AGENT_CREATED);
+ await settle();
+ expect(first.trackEvent.mock.calls).toEqual([
+ ['activation_milestone', { milestone: 'server_ready', path: 'unknown' }],
+ ['activation_milestone', { milestone: 'agent_ready', path: 'unknown' }],
+ ]);
+
+ launch.kind = 'same';
+ const second = await launchApp();
+ second.noteSentEvent(SIGNED_IN);
+ second.noteSentEvent(MANAGED_SESSION);
+ await settle();
+ expect(second.trackEvent.mock.calls).toEqual([
+ ['activation_milestone', { milestone: 'first_agent_session', path: 'unknown' }],
+ ]);
+ });
+
+ it('carries the way it went at Meet switch-buddy, the first answer standing', async () => {
+ const app = await launchApp();
+ app.noteSentEvent(SIGNED_IN);
+ await settle();
+ app.noteBuddyStep('skip');
+ app.noteBuddyStep('start');
+ app.noteSentEvent(AGENT_CREATED);
+ await settle();
+ expect(app.trackEvent.mock.calls).toEqual([
+ ['activation_milestone', { milestone: 'server_ready', path: 'unknown' }],
+ ['activation_milestone', { milestone: 'agent_ready', path: 'manual' }],
+ ]);
+
+ launch.kind = 'same';
+ const later = await launchApp();
+ later.noteBuddyStep('start');
+ later.noteSentEvent(MANAGED_SESSION);
+ await settle();
+ expect(later.trackEvent).toHaveBeenCalledWith('activation_milestone', {
+ milestone: 'first_agent_session',
+ path: 'manual',
+ });
+ });
+
+ it('knows the buddy was started even when its agent is created before that is stored', async () => {
+ const app = await launchApp();
+ app.noteBuddyStep('start');
+ app.noteSentEvent(AGENT_CREATED);
+ await settle();
+ expect(app.trackEvent).toHaveBeenCalledWith('activation_milestone', {
+ milestone: 'agent_ready',
+ path: 'buddy',
+ });
+ });
+
+ it('is decided on its first launch, before it reaches anything', async () => {
+ const first = await launchApp();
+ await first.activationTracked();
+
+ launch.kind = 'same';
+ const second = await launchApp();
+ second.noteSentEvent(AGENT_CREATED);
+ await settle();
+ expect(second.trackEvent).toHaveBeenCalledWith('activation_milestone', {
+ milestone: 'agent_ready',
+ path: 'unknown',
+ });
+ });
+});
+
+describe('an install that existed before', () => {
+ it('reports no milestone, on this launch or any later one', async () => {
+ launch.kind = 'updated';
+ const first = await launchApp();
+ first.noteSentEvent(SIGNED_IN);
+ await settle();
+ expect(first.trackEvent).not.toHaveBeenCalled();
+
+ launch.kind = 'same';
+ const second = await launchApp();
+ second.noteSentEvent(AGENT_CREATED);
+ second.noteSentEvent(MANAGED_SESSION);
+ await settle();
+ expect(second.trackEvent).not.toHaveBeenCalled();
+ });
+});
+
+describe('when the records fail', () => {
+ it('decides nothing on a launch that could not be recorded', async () => {
+ launch.kind = 'unknown';
+ const app = await launchApp();
+ app.noteSentEvent(SIGNED_IN);
+ await settle();
+ expect(app.trackEvent).not.toHaveBeenCalled();
+ expect(records.has('tracked')).toBe(false);
+ });
+
+ it('sends nothing it could not record, and says so', async () => {
+ const app = await launchApp();
+ await app.activationTracked();
+ unwritable.on = true;
+ app.noteSentEvent(SIGNED_IN);
+ await settle();
+ expect(app.trackEvent).not.toHaveBeenCalled();
+ expect(app.log.warn).toHaveBeenCalledWith(
+ 'telemetry: could not record an activation milestone, so it is not sent',
+ expect.objectContaining({ milestone: 'server_ready' })
+ );
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/activation.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/activation.ts
new file mode 100644
index 000000000..6551be63f
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/activation.ts
@@ -0,0 +1,163 @@
+import { KV } from '@main/db/kv';
+import { log } from '@main/lib/logger';
+import type {
+ TelemetryActivationMilestone,
+ TelemetryActivationPath,
+ TelemetryEvent,
+ TelemetryEventMap,
+} from './events';
+import { launchInstallKind } from './launch-history';
+import { telemetryService, trackEvent } from './telemetry-service';
+
+/**
+ * The steps from installing the app to an agent doing work, as
+ * `activation_milestone`, each reported once for the life of an install, the
+ * first time it is reached:
+ *
+ * - `server_ready`: signed in to a server, or a server the app runs started.
+ * - `agent_ready`: an agent created, however it was made.
+ * - `first_agent_session`: a session of one of its agents started, in the app
+ * (`session_started`) or on the agent controller it runs on this computer
+ * (`managed_session_started`).
+ *
+ * Each is derived from the event that marks it, once that event has been sent,
+ * rather than reported at each place it can happen: a new way to sign in or
+ * make an agent counts towards activation by reporting its own event, with
+ * nothing more to wire. Following a sent event also keeps the claim behind the
+ * consent gate, and a milestone whose event did not get through is reached
+ * again the next time it happens.
+ *
+ * Only an install that began on a version reporting milestones reports them.
+ * One that already existed passed some of them before anything recorded it,
+ * and taking its next sign-in or session for its first would count every
+ * existing install as newly activated.
+ */
+
+type ActivationRecords = {
+ /** Whether this install reports milestones; decided once, on its first launch with them. */
+ tracked: boolean;
+ path: Exclude<TelemetryActivationPath, 'unknown'>;
+} & Record<TelemetryActivationMilestone, boolean>;
+
+const store = new KV<ActivationRecords>('telemetry-activation');
+
+/** The milestone an event marks, if any. */
+export function milestoneOf(event: TelemetryEvent): TelemetryActivationMilestone | null {
+ switch (event.name) {
+ case 'server_sign_in':
+ return event.properties.outcome === 'success' ? 'server_ready' : null;
+ case 'managed_server_action':
+ return event.properties.action === 'start' && event.properties.outcome === 'success'
+ ? 'server_ready'
+ : null;
+ case 'agent_created':
+ return event.properties.outcome === 'success' ? 'agent_ready' : null;
+ case 'session_started':
+ return event.properties.outcome === 'success' ? 'first_agent_session' : null;
+ case 'managed_session_started':
+ return 'first_agent_session';
+ default:
+ return null;
+ }
+}
+
+let tracking: Promise<boolean> | null = null;
+
+/**
+ * Whether this install reports milestones, deciding it on the first launch
+ * that asks. Boot asks as soon as the launch is recorded, so a new install that
+ * quits before reaching any milestone is still decided on its first launch.
+ * A launch that could not be recorded decides nothing, and the next one does.
+ */
+export function activationTracked(): Promise<boolean> {
+ tracking ??= (async () => {
+ const decided = await store.get('tracked');
+ if (decided !== null) return decided;
+ const kind = await launchInstallKind();
+ if (kind === 'unknown') return false;
+ const tracked = kind === 'new';
+ await store.setOrThrow('tracked', tracked);
+ return tracked;
+ })();
+ return tracking;
+}
+
+/**
+ * The way the install went at Meet switch-buddy, kept in memory as well as
+ * stored: starting the buddy and creating it are reported moments apart, and
+ * the create's `agent_ready` must not read the record before it is written.
+ */
+let routeThisRun: ActivationRecords['path'] | null = null;
+
+/**
+ * Note which way the install went at Meet switch-buddy, from a `buddy_step`
+ * the interface reported. The first answer stands: someone who skipped the
+ * buddy and set it up later went the manual way.
+ */
+export function noteBuddyStep(action: TelemetryEventMap['buddy_step']['action']): void {
+ const route = action === 'start' ? 'buddy' : action === 'skip' ? 'manual' : null;
+ if (route === null || routeThisRun !== null) return;
+ routeThisRun = route;
+ void (async () => {
+ const stored = await store.get('path');
+ if (stored !== null) {
+ routeThisRun = stored;
+ return;
+ }
+ await store.setOrThrow('path', route);
+ })().catch((error: unknown) => {
+ log.warn('telemetry: could not record the way through onboarding', { error });
+ });
+}
+
+async function pathNow(): Promise<TelemetryActivationPath> {
+ return (await store.get('path')) ?? routeThisRun ?? 'unknown';
+}
+
+const claimedThisRun = new Set<TelemetryActivationMilestone>();
+
+/**
+ * Whether this is the install's first time reaching the milestone, claiming it
+ * if so. The in-memory guard comes first: the events that mark a milestone
+ * arrive in bursts (several sessions restoring at once), and the stored record
+ * does not land in time to stop the second. The write is `setOrThrow`, for the
+ * reason `claimOnce` in `./renderer-events` gives: a record that did not save
+ * is a report that may be missed, rather than one that is certainly repeated.
+ */
+async function claim(milestone: TelemetryActivationMilestone): Promise<boolean> {
+ if (claimedThisRun.has(milestone)) return false;
+ claimedThisRun.add(milestone);
+ if (!(await activationTracked())) return false;
+ if ((await store.get(milestone)) === true) return false;
+ await store.setOrThrow(milestone, true);
+ return true;
+}
+
+/** Report the milestone this sent event marks, if it is the install's first. */
+export function noteSentEvent(event: TelemetryEvent): void {
+ const milestone = milestoneOf(event);
+ if (milestone === null) return;
+ void (async () => {
+ if (!(await claim(milestone))) return;
+ trackEvent('activation_milestone', { milestone, path: await pathNow() });
+ })().catch((error: unknown) => {
+ log.warn('telemetry: could not record an activation milestone, so it is not sent', {
+ milestone,
+ error,
+ });
+ });
+}
+
+/**
+ * Derive milestones from every event sent from now on, and decide whether this
+ * install reports them. Boot calls it after starting the launch's recording,
+ * which the decision waits on.
+ */
+export function startActivationTracking(): void {
+ telemetryService.setSentListener(noteSentEvent);
+ void activationTracked().catch((error: unknown) => {
+ log.warn('telemetry: could not decide whether this install reports activation milestones', {
+ error,
+ });
+ });
+}
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/app-version.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/app-version.test.ts
new file mode 100644
index 000000000..8da24a46d
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/app-version.test.ts
@@ -0,0 +1,23 @@
+import { describe, expect, it } from 'vitest';
+import { appVersionOf } from './app-version';
+
+describe('an app version as a reportable value', () => {
+ it('passes a stable and a canary version through', () => {
+ expect(appVersionOf('0.40.1')).toBe('0.40.1');
+ expect(appVersionOf('0.41.0-canary.3')).toBe('0.41.0-canary.3');
+ expect(appVersionOf('1.2.0-canary.rc.1')).toBe('1.2.0-canary.rc.1');
+ });
+
+ it('refuses anything that is not a version, so no free text reaches a payload', () => {
+ for (const raw of [
+ '',
+ 'latest',
+ '0.40',
+ '/Users/ada/app',
+ '0.40.1 extra',
+ '0.40.1-' + 'x'.repeat(40),
+ ]) {
+ expect(appVersionOf(raw)).toBe('unknown');
+ }
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/app-version.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/app-version.ts
new file mode 100644
index 000000000..c76f8f340
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/app-version.ts
@@ -0,0 +1,20 @@
+import type { TelemetryAppVersion } from './events';
+
+/**
+ * The shape of a version this app reports for itself: semver, with the
+ * prerelease part a canary build carries (`0.41.0-canary.3`). Short, and with
+ * nothing in it that could be a name or a path.
+ */
+const APP_VERSION = /^\d{1,4}\.\d{1,4}\.\d{1,4}(?:-[0-9A-Za-z.]{1,24})?$/;
+
+/**
+ * A version as a reportable value, or `unknown` when it does not look like one.
+ *
+ * The version an event already carries comes from the running build. One read
+ * back from local storage, as the last launch's is, has only been written by this
+ * app, but it is still checked rather than trusted: nothing may reach a payload as
+ * free text.
+ */
+export function appVersionOf(raw: string): TelemetryAppVersion | 'unknown' {
+ return APP_VERSION.test(raw) ? (raw as TelemetryAppVersion) : 'unknown';
+}
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/events.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/events.ts
index 5fe4c3dc9..707d51dd9 100644
--- a/console/apps/switch-console-desktop/src/main/core/telemetry/events.ts
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/events.ts
@@ -1,4 +1,5 @@
import type { InstallMethod } from '@switch-console/core/deps';
+import type { ManagedStartFailure } from '@shared/core/managed-switch-server/managed-switch-server';
import type { AgentProviderId } from '@shared/core/providers/agent-provider-registry';
import type { HostSetupStepKind } from '@shared/core/remote-hosts/setup';
import type { SearchStatus } from '@shared/core/search';
@@ -98,14 +99,6 @@ export type TelemetryAgentRemoveFailure =
| 'gateway_network'
| 'error';
-/** Why resetting a remote agent failed. Also named rather than derived. */
-export type TelemetryAgentResetFailure =
- | 'none'
- | 'agent_not_found'
- | 'not_remote'
- | 'connect'
- | 'error';
-
/**
* What removed an agent.
*
@@ -163,6 +156,14 @@ export type TelemetryCliFailure =
declare const durationMsBrand: unique symbol;
export type TelemetryDurationMs = number & { readonly [durationMsBrand]: true };
+/**
+ * An app version, as the app reports its own: semver only, checked where it is
+ * minted (`appVersionOf`). Branded so only that check produces one, the way a
+ * platform key below is: a string type here would let free text through.
+ */
+declare const appVersionBrand: unique symbol;
+export type TelemetryAppVersion = string & { readonly [appVersionBrand]: true };
+
declare const platformKeyBrand: unique symbol;
/**
@@ -209,12 +210,23 @@ export type TelemetryIdentityClaimOutcome = 'success' | 'pending' | 'failure';
*/
export type TelemetryIdentityLinkKind = 'new' | 'existing' | 'unknown';
+/**
+ * Why a room create failed. `unreachable` is the server's host known to be
+ * down before asking; `network` a request that never reached the server;
+ * `forbidden`, `not_found`, `conflict` and `server_error` the server's own
+ * answer (403, 404, 409, 5xx); `error` anything with no answer to read.
+ */
export type TelemetryRoomCreateFailure =
| 'none'
| 'unauthenticated'
| 'bridge_unavailable'
| 'invalid'
| 'unreachable'
+ | 'network'
+ | 'forbidden'
+ | 'not_found'
+ | 'conflict'
+ | 'server_error'
| 'error';
/** How a room came to be made: the Create room dialog, or a template run. */
@@ -245,12 +257,18 @@ export type TelemetryHostSetupAction = 'install' | 'update' | 'skip';
export type TelemetryManagedServerAction = 'start' | 'stop' | 'reset';
+/**
+ * Why a managed server's start, stop or reset failed. The start's own causes
+ * are named where the code can tell them apart (`ManagedStartFailure`); `error`
+ * is everything else, including anything that threw.
+ */
export type TelemetryManagedServerFailure =
| 'none'
| 'docker_not_installed'
| 'docker_daemon_down'
| 'version_downgrade'
| 'matrix_migration_failed'
+ | Exclude<ManagedStartFailure, 'unexpected'>
| 'error';
/**
@@ -289,8 +307,44 @@ export type TelemetrySettingKey = AppSettingsKeyName;
/** How a search ended. The search's own union, reused rather than restated. */
export type TelemetrySearchStatus = SearchStatus;
-/** The kinds of server this install can hold. Named once, used by five events. */
-export type TelemetryServerKind = 'local' | 'remote_managed' | 'external';
+/**
+ * The kinds of server this install can hold. Named once, used by five events.
+ * `cloud` is Switch Cloud, the server this build offers by name; `external` is
+ * any other server someone registered by address.
+ */
+export type TelemetryServerKind = 'local' | 'remote_managed' | 'external' | 'cloud';
+
+/**
+ * Why registering a server by address failed: the address is already another
+ * server's, or it answered with a page rather than the server's API (the
+ * dashboard's address pasted in place of the server's).
+ */
+export type TelemetryServerAddFailure =
+ | 'none'
+ | 'already_added'
+ | 'not_the_server_address'
+ | 'error';
+
+/**
+ * How a session of an agent on this computer's agent controller started, as
+ * its session host told Switch (`host/session-start.ts`): a person in Console,
+ * the agent being addressed in a room, or the automation API. `unknown` is a
+ * launcher that did not say.
+ */
+export type TelemetryManagedSessionStartSource = 'user' | 'room' | 'automation' | 'unknown';
+
+/**
+ * The steps from installing the app to an agent doing work, each reported
+ * once for the life of an install (`./activation`).
+ */
+export type TelemetryActivationMilestone = 'server_ready' | 'agent_ready' | 'first_agent_session';
+
+/**
+ * Which way the install went at Meet switch-buddy: `buddy` started the buddy,
+ * `manual` skipped it. `unknown` until one of the two happens, which is after
+ * a server is ready, so `server_ready` normally carries it.
+ */
+export type TelemetryActivationPath = 'buddy' | 'manual' | 'unknown';
/**
* A setup step's kind, reused from the remote-host model rather than restated.
@@ -364,9 +418,13 @@ export type TelemetryEventMap = {
* counting those here as well would double-count the same wall.
*/
server_added: {
- /** `external` is a server the user registered by URL: Switch Console does not run it. */
- server_kind: 'local' | 'remote_managed' | 'external';
+ /**
+ * `external` is a server the user registered by URL: Switch Console does not
+ * run it. `cloud` is Switch Cloud, registered from its own button.
+ */
+ server_kind: TelemetryServerKind;
outcome: TelemetryOutcome;
+ failure_reason: TelemetryServerAddFailure;
};
/**
* The app checked for an update. `trigger` separates a check someone asked for
@@ -376,6 +434,13 @@ export type TelemetryEventMap = {
trigger: TelemetryUpdateTrigger;
result: 'available' | 'up_to_date' | 'failed';
};
+ /**
+ * Download was pressed for an available update. Downloads only ever start on
+ * that press (`autoDownload` is off), so this is the press, and
+ * `update_downloaded` is how it ended. Nothing to carry: which update is the
+ * event's own app version and the check before it.
+ */
+ update_download_started: Record<never, never>;
update_downloaded: {
outcome: TelemetryOutcome;
};
@@ -385,11 +450,25 @@ export type TelemetryEventMap = {
* Deliberately named for what it can actually observe. A successful install
* quits the app, so there is no moment afterwards in which to report one; what
* this counts is the point the update was handed to the installer. Whether it
- * worked is answered by the next `app_launched` and its version.
+ * worked is answered by `update_installed` on the next launch.
*/
update_install_started: {
outcome: TelemetryOutcome;
};
+ /**
+ * The first launch on a new version after an update: the end of the update.
+ * The new version is the event's own app version, which every event carries.
+ *
+ * `from_version` is the version the previous launch ran; `unknown` for an
+ * install from before launches were recorded, or a stored value that is not a
+ * version. `trigger` is `restart_button` when the previous run handed the update
+ * to the installer from Restart to update, and `other` when the version changed
+ * without that: installed when the app quit, or by hand.
+ */
+ update_installed: {
+ from_version: TelemetryAppVersion | 'unknown';
+ trigger: 'restart_button' | 'other';
+ };
/** A messaging platform was connected to a server. */
bridge_connected: {
bridge_platform: TelemetryBridgePlatform;
@@ -434,16 +513,6 @@ export type TelemetryEventMap = {
outcome: TelemetryOutcome;
failure_reason: TelemetryAgentRemoveFailure;
};
- /**
- * A remote agent was reset — every one of its panes on the host killed and the
- * agent brought back up. Remote-only, and a real SSH operation, so unlike most
- * of this catalogue it fails for reasons outside the app.
- */
- agent_reset: {
- agent_type: TelemetryAgentType;
- outcome: TelemetryOutcome;
- failure_reason: TelemetryAgentResetFailure;
- };
/**
* A server was removed from this install.
*
@@ -453,7 +522,7 @@ export type TelemetryEventMap = {
* so a count here would always be zero, which reads as an answer.
*/
server_removed: {
- server_kind: 'local' | 'remote_managed' | 'external';
+ server_kind: TelemetryServerKind;
};
/**
* A session whose setup had not completed was provisioned again.
@@ -467,6 +536,25 @@ export type TelemetryEventMap = {
trigger: TelemetrySessionProvisionTrigger;
outcome: TelemetryOutcome;
};
+ /**
+ * A session of an agent on the agent controller this app runs on this
+ * computer started, and its Switch server took the report. These agents
+ * have no session of their own in the app, so `session_started` never sees
+ * them; a session on another machine's controller is not reported at all.
+ */
+ managed_session_started: {
+ agent_type: TelemetryAgentType;
+ start_source: TelemetryManagedSessionStartSource;
+ };
+ /**
+ * A step towards an agent doing work, the first time this install reaches
+ * it. Derived from the events that mark each one, never sent twice, and only
+ * by an install that began on a version that reports them (`./activation`).
+ */
+ activation_milestone: {
+ milestone: TelemetryActivationMilestone;
+ path: TelemetryActivationPath;
+ };
/**
* A remote session's terminal was attached, or could not be.
*
@@ -630,6 +718,11 @@ export type TelemetryEventMap = {
export type TelemetryEventName = keyof TelemetryEventMap;
+/** One event and what it carries, as a union a `switch` on `name` narrows. */
+export type TelemetryEvent = {
+ [K in TelemetryEventName]: { name: K; properties: TelemetryEventMap[K] };
+}[TelemetryEventName];
+
/**
* The properties each event may carry, as data rather than as types alone.
*
@@ -652,10 +745,12 @@ export const TELEMETRY_EVENT_PROPERTIES = {
'connected_to_room',
],
session_ended: ['agent_type', 'location', 'outcome'],
- server_added: ['server_kind', 'outcome'],
+ server_added: ['server_kind', 'outcome', 'failure_reason'],
update_checked: ['trigger', 'result'],
+ update_download_started: [],
update_downloaded: ['outcome'],
update_install_started: ['outcome'],
+ update_installed: ['from_version', 'trigger'],
bridge_connected: ['bridge_platform', 'outcome', 'failure_reason'],
bridge_disconnected: ['bridge_platform', 'outcome'],
bridge_identity_claimed: ['bridge_platform', 'outcome'],
@@ -668,7 +763,6 @@ export const TELEMETRY_EVENT_PROPERTIES = {
'outcome',
'failure_reason',
],
- agent_reset: ['agent_type', 'outcome', 'failure_reason'],
server_removed: ['server_kind'],
session_provision_retried: ['agent_type', 'location', 'trigger', 'outcome'],
setting_changed: ['setting_key'],
@@ -712,6 +806,8 @@ export const TELEMETRY_EVENT_PROPERTIES = {
renderer_crashed: [],
telemetry_consent_changed: ['source', 'install_kind'],
session_attached: ['agent_type', 'outcome'],
+ managed_session_started: ['agent_type', 'start_source'],
+ activation_milestone: ['milestone', 'path'],
} as const satisfies { [K in TelemetryEventName]: readonly (keyof TelemetryEventMap[K])[] };
/**
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.test.ts
index b7da2f7aa..ccdb6b882 100644
--- a/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.test.ts
@@ -12,14 +12,7 @@ vi.mock('@main/db/kv', () => ({
},
}));
-const {
- currentInternalFlag,
- emailFromJwt,
- forgetSessionAccount,
- internalFrom,
- isInternalEmail,
- recordSessionAccount,
-} = await import('./internal-account');
+const { emailFromJwt, internalFrom, isInternalEmail } = await import('./internal-account');
function jwtWith(payload: Record<string, unknown>): string {
const part = (value: object) => Buffer.from(JSON.stringify(value)).toString('base64url');
@@ -73,27 +66,45 @@ describe('reading the account from a session', () => {
});
describe('the answer kept beside each session', () => {
+ // A fresh module per test: the answers live in the module's store, and one
+ // test's staff sign-in would otherwise make every later one internal.
+ async function fresh() {
+ vi.resetModules();
+ return import('./internal-account');
+ }
+
it('is recorded when a session is stored, and forgotten when it goes', async () => {
- expect(await currentInternalFlag()).toBe('unknown');
+ const accounts = await fresh();
+ expect(await accounts.currentInternalFlag()).toBe('unknown');
- await recordSessionAccount('outside', jwtWith({ email: '[email protected]' }));
- expect(await currentInternalFlag()).toBe('false');
+ await accounts.recordSessionAccount('outside', jwtWith({ email: '[email protected]' }));
+ expect(await accounts.currentInternalFlag()).toBe('false');
- await recordSessionAccount('staff', jwtWith({ email: '[email protected]' }));
- expect(await currentInternalFlag()).toBe('true');
+ await accounts.forgetSessionAccount('outside');
+ expect(await accounts.currentInternalFlag()).toBe('unknown');
+ });
- await forgetSessionAccount('staff');
- expect(await currentInternalFlag()).toBe('false');
+ it('follows a server signed in again as someone else', async () => {
+ const accounts = await fresh();
+ await accounts.recordSessionAccount('server', jwtWith({ email: '[email protected]' }));
+ await accounts.recordSessionAccount('server', jwtWith({ email: '[email protected]' }));
- await forgetSessionAccount('outside');
- expect(await currentInternalFlag()).toBe('unknown');
+ expect(await accounts.currentInternalFlag()).toBe('false');
});
- it('follows a server signed in again as someone else', async () => {
- await recordSessionAccount('server', jwtWith({ email: '[email protected]' }));
- await recordSessionAccount('server', jwtWith({ email: '[email protected]' }));
+ it('stays internal once a staff account has signed in, after that session goes', async () => {
+ // Staff who sign out, or whose session lapses, are still staff using this
+ // installation; dropping back would count them as a customer.
+ const accounts = await fresh();
+ await accounts.recordSessionAccount('outside', jwtWith({ email: '[email protected]' }));
+ await accounts.recordSessionAccount('staff', jwtWith({ email: '[email protected]' }));
+ expect(await accounts.currentInternalFlag()).toBe('true');
+
+ await accounts.forgetSessionAccount('staff');
+ expect(await accounts.currentInternalFlag()).toBe('true');
- expect(await currentInternalFlag()).toBe('false');
- await forgetSessionAccount('server');
+ await accounts.recordSessionAccount('staff', jwtWith({ email: '[email protected]' }));
+ await accounts.forgetSessionAccount('outside');
+ expect(await accounts.currentInternalFlag()).toBe('true');
});
});
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.ts
index e8544720a..d0ec198dc 100644
--- a/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.ts
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.ts
@@ -59,7 +59,9 @@ export function emailFromJwt(jwt: string): string | null {
* briefly locked would sign people out of servers they were not even using.
* So it is worked out once, from the token in hand, when a session is stored.
*/
-const store = new KV<{ accounts: Record<string, TelemetryInternal> }>('telemetry-accounts');
+const store = new KV<{ accounts: Record<string, TelemetryInternal>; ever_internal: boolean }>(
+ 'telemetry-accounts'
+);
let writes: Promise<void> = Promise.resolve();
@@ -72,11 +74,17 @@ function update(change: (accounts: Record<string, TelemetryInternal>) => void):
return writes;
}
-/** Record the account a server's session was issued to. */
+/**
+ * Record the account a server's session was issued to.
+ *
+ * A staff account is also remembered for the installation, beyond its session:
+ * see `currentInternalFlag`.
+ */
export function recordSessionAccount(serverId: string, jwt: string): Promise<void> {
+ const internal = internalFrom([emailFromJwt(jwt)]);
return update((accounts) => {
- accounts[serverId] = internalFrom([emailFromJwt(jwt)]);
- });
+ accounts[serverId] = internal;
+ }).then(() => (internal === 'true' ? store.set('ever_internal', true) : undefined));
}
/** Forget a server's account when its session goes. */
@@ -90,8 +98,16 @@ export function forgetSessionAccount(serverId: string): Promise<void> {
* Across every server this installation holds a session for. A session stored
* before this was recorded reads as unknown until it is next renewed or
* signed in again.
+ *
+ * Once a staff account has signed in here, the installation stays internal
+ * after that session goes: staff who sign out, or whose session lapses, are
+ * still staff using it, and dropping back to `unknown` would count their usage
+ * as a customer's in every chart that excludes staff. Nothing can mark the
+ * events sent before that first sign-in, which is why dashboards also exclude
+ * installs that ever reported `true`.
*/
export async function currentInternalFlag(): Promise<TelemetryInternal> {
+ if ((await store.get('ever_internal')) === true) return 'true';
const answers = Object.values((await store.get('accounts')) ?? {});
if (answers.includes('true')) return 'true';
return answers.includes('false') ? 'false' : 'unknown';
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.test.ts
index 18bc82631..36027cde9 100644
--- a/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.test.ts
@@ -9,6 +9,7 @@ const h = vi.hoisted(() => ({
readFails: false,
isCanary: false,
databaseExisted: true,
+ handedOver: false,
}));
vi.mock('./telemetry-service', () => ({ trackEvent: h.trackEvent }));
@@ -24,6 +25,9 @@ vi.mock('@main/db/client', () => ({
},
}));
vi.mock('@main/lib/logger', () => ({ log: { warn: h.warn } }));
+vi.mock('@main/core/updates/pending-install', () => ({
+ pendingInstall: { take: async () => h.handedOver },
+}));
vi.mock('@main/db/kv', () => ({
KV: class {
private values = new Map<string, unknown>();
@@ -69,8 +73,8 @@ describe('which kind of launch this is', () => {
describe('canary and stable, which share one database', () => {
it('do not read a switch between them as an upgrade', async () => {
- const launch = (channel: 'canary' | 'stable', version: string) =>
- recordLaunch({ version, channel, databaseExisted: true });
+ const launch = async (channel: 'canary' | 'stable', version: string) =>
+ (await recordLaunch({ version, channel, databaseExisted: true })).kind;
expect(await launch('stable', '1.4.0')).toBe('updated');
expect(await launch('canary', '1.5.0-canary.2')).toBe('updated');
@@ -90,6 +94,7 @@ describe('reporting a launch at boot', () => {
h.trackEvent.mockClear();
h.warn.mockClear();
h.readFails = false;
+ h.handedOver = false;
launches = await import('./launch-history');
});
@@ -109,6 +114,50 @@ describe('reporting a launch at boot', () => {
]);
});
+ const launchAt = (version: string) => async () => ({
+ version,
+ channel: 'stable' as const,
+ databaseExisted: true,
+ });
+
+ it('reports an upgrade as installed, with the version it came from', async () => {
+ h.handedOver = true;
+
+ await launches.reportLaunch(launchAt('0.40.1'));
+ await launches.reportLaunch(launchAt('0.41.0'));
+
+ expect(h.trackEvent.mock.calls).toEqual([
+ // A database with no launch record predates the tracking: an upgrade from
+ // a version nobody wrote down.
+ ['app_launched', { install_kind: 'updated' }],
+ ['update_installed', { from_version: 'unknown', trigger: 'restart_button' }],
+ ['app_launched', { install_kind: 'updated' }],
+ ['update_installed', { from_version: '0.40.1', trigger: 'restart_button' }],
+ ]);
+ });
+
+ it('says other for an update nobody pressed Restart for', async () => {
+ await launches.reportLaunch(launchAt('0.40.1'));
+ h.trackEvent.mockClear();
+
+ await launches.reportLaunch(launchAt('0.41.0-canary.2'));
+
+ expect(h.trackEvent.mock.calls).toEqual([
+ ['app_launched', { install_kind: 'updated' }],
+ ['update_installed', { from_version: '0.40.1', trigger: 'other' }],
+ ]);
+ });
+
+ it('reports no install on a new install or a relaunch', async () => {
+ await launches.reportLaunch(firstLaunch);
+ await launches.reportLaunch(firstLaunch);
+
+ expect(h.trackEvent.mock.calls.map(([event]) => event)).toEqual([
+ 'app_launched',
+ 'app_launched',
+ ]);
+ });
+
it('never rejects when the launch cannot be recorded, so boot is not held up', async () => {
// Boot starts this without waiting on it; a rejection would be unhandled,
// and an awaited one would stop everything after it, the window included.
@@ -141,6 +190,7 @@ describe('asking for this launch’s kind', () => {
vi.resetModules();
h.warn.mockClear();
h.readFails = false;
+ h.handedOver = false;
launches = await import('./launch-history');
});
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.ts
index d8f58130e..3c1f5877a 100644
--- a/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.ts
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.ts
@@ -1,7 +1,9 @@
import { resolveAppVersion } from '@main/core/app/utils';
+import { pendingInstall } from '@main/core/updates/pending-install';
import { KV } from '@main/db/kv';
import { log } from '@main/lib/logger';
import { IS_CANARY } from '@shared/app-identity';
+import { appVersionOf } from './app-version';
import type { TelemetryInstallKind } from './events';
import { trackEvent } from './telemetry-service';
@@ -47,6 +49,9 @@ type LaunchInputs = {
databaseExisted: boolean;
};
+/** Which kind this launch is, and the version the previous launch ran. */
+export type LaunchRecord = { kind: TelemetryInstallKind; lastVersion: string | null };
+
/**
* Record this launch and say which kind it is. Recorded on every launch,
* whether or not usage is shared: it is local, and the next launch has to know
@@ -56,12 +61,12 @@ export async function recordLaunch({
version,
channel,
databaseExisted,
-}: LaunchInputs): Promise<TelemetryInstallKind> {
+}: LaunchInputs): Promise<LaunchRecord> {
const key = `lastLaunchedVersion.${channel}`;
const lastVersion = (await store.get(key)) ?? null;
const kind = installKindFor({ lastVersion, version, databaseExisted });
await store.set(key, version);
- return kind;
+ return { kind, lastVersion };
}
/** What recording this launch needs, read when boot starts it. */
@@ -75,18 +80,37 @@ export async function readThisLaunch(): Promise<LaunchInputs> {
}
/**
- * Record this launch and report it as `app_launched`. Never rejects, so boot
- * can start it without waiting on it: nothing telemetry does may stop the app
- * opening. A launch that cannot be recorded is logged and goes unreported,
- * rather than reported with a kind nobody worked out.
+ * Record this launch and report it as `app_launched`, and an upgrade as
+ * `update_installed` as well. Never rejects, so boot can start it without
+ * waiting on it: nothing telemetry does may stop the app opening. A launch that
+ * cannot be recorded is logged and goes unreported, rather than reported with a
+ * kind nobody worked out.
*/
export async function reportLaunch(read: () => Promise<LaunchInputs>): Promise<void> {
const recorded = (async () => recordLaunch(await read()))();
- recording = recorded.catch(() => null);
+ recording = recorded.then(
+ (record) => record.kind,
+ () => null
+ );
+ let record: LaunchRecord;
try {
- trackEvent('app_launched', { install_kind: await recorded });
+ record = await recorded;
} catch (error) {
log.warn('telemetry: could not record this launch, so app_launched is not sent', { error });
+ return;
+ }
+ trackEvent('app_launched', { install_kind: record.kind });
+ if (record.kind !== 'updated') return;
+ try {
+ const handedOver = await pendingInstall.take();
+ trackEvent('update_installed', {
+ from_version: record.lastVersion === null ? 'unknown' : appVersionOf(record.lastVersion),
+ trigger: handedOver ? 'restart_button' : 'other',
+ });
+ } catch (error) {
+ log.warn('telemetry: could not read how this update was installed, so it is not sent', {
+ error,
+ });
}
}
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/managed-server.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/managed-server.ts
index 249910993..71743247e 100644
--- a/console/apps/switch-console-desktop/src/main/core/telemetry/managed-server.ts
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/managed-server.ts
@@ -21,7 +21,9 @@ function startFailureReason(result: StartLocalServerResult): TelemetryManagedSer
case 'matrix-migration-failed':
return 'matrix_migration_failed';
case 'error':
- return 'error';
+ // `unexpected` is something that threw: `error`, as a start that threw
+ // past the result is reported.
+ return result.failure === 'unexpected' ? 'error' : result.failure;
}
}
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/managed-session.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/managed-session.test.ts
new file mode 100644
index 000000000..d27eb6d84
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/managed-session.test.ts
@@ -0,0 +1,37 @@
+import { SESSION_STARTED_KIND, type SessionStartedNotice } from '@switch-console/agent-providers';
+import { beforeEach, describe, expect, it, vi } from 'vitest';
+
+vi.mock('./telemetry-service', () => ({ trackEvent: vi.fn() }));
+
+const { reportManagedSessionStarted } = await import('./managed-session');
+const { trackEvent } = await import('./telemetry-service');
+
+function notice(sessionId: string, provider = 'claude'): SessionStartedNotice {
+ return { kind: SESSION_STARTED_KIND, sessionId, provider, startSource: 'room' };
+}
+
+beforeEach(() => vi.clearAllMocks());
+
+describe('sessions of agents on this computer’s controller', () => {
+ it('reports a start with the agent’s type and how it started, and never the session', () => {
+ reportManagedSessionStarted(notice('session-1'));
+ expect(trackEvent).toHaveBeenCalledWith('managed_session_started', {
+ agent_type: 'claude',
+ start_source: 'room',
+ });
+ });
+
+ it('counts a start reported twice once', () => {
+ reportManagedSessionStarted(notice('session-2'));
+ reportManagedSessionStarted(notice('session-2'));
+ expect(trackEvent).toHaveBeenCalledTimes(1);
+ });
+
+ it('reports a provider it does not know as unknown', () => {
+ reportManagedSessionStarted(notice('session-3', 'some-new-cli'));
+ expect(trackEvent).toHaveBeenCalledWith('managed_session_started', {
+ agent_type: 'unknown',
+ start_source: 'room',
+ });
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/managed-session.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/managed-session.ts
new file mode 100644
index 000000000..e5ce68b27
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/managed-session.ts
@@ -0,0 +1,30 @@
+import type { SessionStartedNotice } from '@switch-console/agent-providers';
+import { agentTypeOf } from './agent-type';
+import { trackEvent } from './telemetry-service';
+
+/** Enough to recognise a start reported twice, which follows the first closely. */
+const REMEMBERED_SESSIONS = 256;
+
+const reported = new Set<string>();
+
+/**
+ * Report a session the agent controller on this computer started, once its
+ * Switch server has taken the report.
+ *
+ * A session host stopped before it heard Switch answer reports the same start
+ * again when it next runs. Within a run that is told apart by the session's id,
+ * which goes no further than this set; across a restart of the app it is
+ * counted twice, which is rare enough to leave.
+ */
+export function reportManagedSessionStarted(notice: SessionStartedNotice): void {
+ if (reported.has(notice.sessionId)) return;
+ reported.add(notice.sessionId);
+ if (reported.size > REMEMBERED_SESSIONS) {
+ const oldest = reported.values().next();
+ if (!oldest.done) reported.delete(oldest.value);
+ }
+ trackEvent('managed_session_started', {
+ agent_type: agentTypeOf(notice.provider),
+ start_source: notice.startSource,
+ });
+}
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/renderer-events.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/renderer-events.ts
index 0ec2364c3..bb0fc8371 100644
--- a/console/apps/switch-console-desktop/src/main/core/telemetry/renderer-events.ts
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/renderer-events.ts
@@ -5,6 +5,7 @@ import { ALL_COMMAND_DEFS, type CommandId } from '@shared/commands';
import { ONBOARDING_STEP_IDS } from '@shared/core/onboarding/checklist';
import { ADD_SERVER_CHOICES, ADD_SERVER_STEPS } from '@shared/core/switch-servers/add-server-steps';
import { VIEW_IDS } from '@shared/core/views/view-ids';
+import { noteBuddyStep } from './activation';
import type { TelemetryEventMap, TelemetryEventName } from './events';
import { telemetryService, trackEvent } from './telemetry-service';
@@ -246,6 +247,8 @@ export function trackFromRenderer(request: RendererTelemetryRequest): void {
}
const properties = parsed.data as TelemetryEventMap[RendererTelemetryEvent];
+ if (request.name === 'buddy_step')
+ noteBuddyStep((properties as TelemetryEventMap['buddy_step']).action);
if (!ONCE_PER_INSTALL.has(request.name)) {
trackEvent(request.name, properties);
return;
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.ts
index 53c36540c..c3688914e 100644
--- a/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.ts
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.ts
@@ -3,7 +3,7 @@ import { resolveAppVersion } from '@main/core/app/utils';
import { log } from '@main/lib/logger';
import { resolveTelemetryConfig, type TelemetryDisabledReason } from './config';
import { isTelemetryAllowed } from './consent';
-import type { TelemetryEventMap, TelemetryEventName } from './events';
+import type { TelemetryEvent, TelemetryEventMap, TelemetryEventName } from './events';
import { getInstallId } from './install-id';
import type { TelemetryInternal } from './internal-account';
import { buildOtlpPayload, postTelemetryEvent, TelemetrySendError } from './relay-client';
@@ -23,11 +23,22 @@ class TelemetryService {
* because it reads the servers store, which reports through this service.
*/
private internalSource: () => Promise<TelemetryInternal> = async () => 'unknown';
+ /**
+ * Told of each event once it has been sent, which is how the activation
+ * milestones follow the events that mark them (`./activation`). Registered
+ * at boot rather than imported, for the same reason as the internal source:
+ * it reports through this service.
+ */
+ private sentListener: (event: TelemetryEvent) => void = () => {};
setInternalSource(source: () => Promise<TelemetryInternal>): void {
this.internalSource = source;
}
+ setSentListener(listener: (event: TelemetryEvent) => void): void {
+ this.sentListener = listener;
+ }
+
/**
* Send one event, if this build can send and the user has said yes.
*
@@ -71,6 +82,7 @@ class TelemetryService {
});
await postTelemetryEvent(payload, resolution.config);
+ this.sentListener({ name, properties } as TelemetryEvent);
}
/**
diff --git a/console/apps/switch-console-desktop/src/main/core/updates/pending-install.test.ts b/console/apps/switch-console-desktop/src/main/core/updates/pending-install.test.ts
new file mode 100644
index 000000000..973042c16
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/updates/pending-install.test.ts
@@ -0,0 +1,50 @@
+import { beforeEach, describe, expect, it, vi } from 'vitest';
+import type * as PendingInstall from './pending-install';
+
+const h = vi.hoisted(() => ({ values: new Map<string, unknown>(), reads: 0 }));
+
+vi.mock('@main/db/kv', () => ({
+ KV: class {
+ async get(key: string) {
+ h.reads += 1;
+ return h.values.get(key) ?? null;
+ }
+ async set(key: string, value: unknown) {
+ h.values.set(key, value);
+ }
+ async del(key: string) {
+ h.values.delete(key);
+ }
+ },
+}));
+
+describe('the handover marker', () => {
+ let marker: typeof PendingInstall;
+
+ beforeEach(async () => {
+ vi.resetModules();
+ h.values.clear();
+ h.reads = 0;
+ marker = await import('./pending-install');
+ });
+
+ it('gives every caller in a run the same answer, and clears the record once', async () => {
+ h.values.set('handover', true);
+
+ const [first, second] = await Promise.all([
+ marker.pendingInstall.take(),
+ marker.pendingInstall.take(),
+ ]);
+
+ // The update service and the launch report both ask; whichever is second
+ // must not read the record the first one cleared and answer no.
+ expect([first, second]).toEqual([true, true]);
+ expect(await marker.pendingInstall.take()).toBe(true);
+ expect(h.values.has('handover')).toBe(false);
+ expect(h.reads).toBe(1);
+ });
+
+ it('answers no when the previous run handed nothing over', async () => {
+ expect(await marker.pendingInstall.take()).toBe(false);
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/updates/pending-install.ts b/console/apps/switch-console-desktop/src/main/core/updates/pending-install.ts
index 355b82cac..e46f48899 100644
--- a/console/apps/switch-console-desktop/src/main/core/updates/pending-install.ts
+++ b/console/apps/switch-console-desktop/src/main/core/updates/pending-install.ts
@@ -16,6 +16,9 @@ const store = new KV<Record<string, boolean>>('updates:pending-install');
const KEY = 'handover';
+/** This run's answer, once anything has asked. */
+let taken: Promise<boolean> | null = null;
+
export const pendingInstall = {
set: (): Promise<void> => store.set(KEY, true),
clear: (): Promise<void> => store.del(KEY),
@@ -25,10 +28,17 @@ export const pendingInstall = {
* launch is only ever counted once. Cleared before the caller reports, so a
* report that fails to send loses one event rather than repeating it on every
* launch from here on.
+ *
+ * The same answer for every caller in this run: the update service reports
+ * the handover and the launch report says which kind of install it was, and
+ * whichever asks first must not leave the other reading a cleared record.
*/
- async take(): Promise<boolean> {
- if ((await store.get(KEY)) !== true) return false;
- await store.del(KEY);
- return true;
+ take(): Promise<boolean> {
+ taken ??= (async () => {
+ if ((await store.get(KEY)) !== true) return false;
+ await store.del(KEY);
+ return true;
+ })();
+ return taken;
},
};
diff --git a/console/apps/switch-console-desktop/src/main/core/updates/update-service.test.ts b/console/apps/switch-console-desktop/src/main/core/updates/update-service.test.ts
index 9928a33c5..e86b18011 100644
--- a/console/apps/switch-console-desktop/src/main/core/updates/update-service.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/updates/update-service.test.ts
@@ -140,6 +140,39 @@ describe('what a check reports', () => {
});
});
+describe('what a download reports', () => {
+ beforeEach(() => {
+ h.handlers.get('update-available')?.({ version: '1.3.0' });
+ h.trackEvent.mockClear();
+ });
+
+ it('reports the press, then how the download ended', async () => {
+ h.downloadUpdate.mockResolvedValueOnce(undefined);
+
+ await updateService.downloadUpdate();
+
+ expect(
+ h.trackEvent.mock.calls.filter(([name]) => String(name).startsWith('update_download'))
+ ).toEqual([
+ ['update_download_started', {}],
+ ['update_downloaded', { outcome: 'success' }],
+ ]);
+ });
+
+ it('reports the press even when the download then fails', async () => {
+ h.downloadUpdate.mockRejectedValueOnce(new Error('connection reset'));
+
+ await expect(updateService.downloadUpdate()).rejects.toThrow('connection reset');
+
+ expect(
+ h.trackEvent.mock.calls.filter(([name]) => String(name).startsWith('update_download'))
+ ).toEqual([
+ ['update_download_started', {}],
+ ['update_downloaded', { outcome: 'failure' }],
+ ]);
+ });
+});
+
describe('what one install attempt reports', () => {
beforeEach(() => {
// Through the service's own listeners: "downloaded" is not reachable any
diff --git a/console/apps/switch-console-desktop/src/main/core/updates/update-service.ts b/console/apps/switch-console-desktop/src/main/core/updates/update-service.ts
index dbad7f8d4..e80873a9b 100644
--- a/console/apps/switch-console-desktop/src/main/core/updates/update-service.ts
+++ b/console/apps/switch-console-desktop/src/main/core/updates/update-service.ts
@@ -340,6 +340,7 @@ class UpdateService implements IInitializable, IDisposable {
this.updateState.status = 'downloading';
events.emit(updateDownloadingEvent, { version: this.updateState.availableVersion });
+ trackEvent('update_download_started', {});
try {
if (this.fake) await this.fake.download();
diff --git a/console/apps/switch-console-desktop/src/main/core/workspaces/controller.ts b/console/apps/switch-console-desktop/src/main/core/workspaces/controller.ts
index 5fd5abe29..86e65109a 100644
--- a/console/apps/switch-console-desktop/src/main/core/workspaces/controller.ts
+++ b/console/apps/switch-console-desktop/src/main/core/workspaces/controller.ts
@@ -211,11 +211,25 @@ function roomCreateFailureReason(result: CreateRoomResult): TelemetryRoomCreateF
return 'bridge_unavailable';
case 'invalid':
return 'invalid';
- default:
- return 'error';
+ case 'error':
+ // The one generic result: the request never reached the server.
+ return 'network';
}
}
+/**
+ * A room create that threw, by the server's answer: the failures with no
+ * result of their own. `error` is one with no answer to read.
+ */
+function roomCreateErrorReason(error: unknown): TelemetryRoomCreateFailure {
+ if (!(error instanceof GatewayError) || error.kind !== 'http') return 'error';
+ if (error.status === 403) return 'forbidden';
+ if (error.status === 404) return 'not_found';
+ if (error.status === 409) return 'conflict';
+ if (error.status !== undefined && error.status >= 500) return 'server_error';
+ return 'error';
+}
+
/**
* Which platform a bridge id is on.
*
@@ -746,7 +760,7 @@ export const workspacesController = createRPCController({
})
);
} catch (error) {
- report('error');
+ report(roomCreateErrorReason(error));
throw error;
}
diff --git a/console/apps/switch-console-desktop/src/main/index.ts b/console/apps/switch-console-desktop/src/main/index.ts
index 59cc32e8d..6ed5d87e0 100644
--- a/console/apps/switch-console-desktop/src/main/index.ts
+++ b/console/apps/switch-console-desktop/src/main/index.ts
@@ -41,6 +41,7 @@ import { appSettingsService } from './core/settings/settings-service';
import { sshConnectionManager } from './core/ssh/lifecycle/production-ssh-connection-manager';
import { autoSessionWatcher } from './core/switch-rooms/auto-session-watcher';
import { retireDashboardFallbacks } from './core/switch-servers/server-address';
+import { startActivationTracking } from './core/telemetry/activation';
import { currentInternalFlag } from './core/telemetry/internal-account';
import { readThisLaunch, reportLaunch } from './core/telemetry/launch-history';
import { registerTelemetryListeners } from './core/telemetry/telemetry-listeners';
@@ -166,6 +167,7 @@ void app.whenReady().then(async () => {
registerTelemetryListeners();
telemetryService.setInternalSource(currentInternalFlag);
void reportLaunch(readThisLaunch);
+ startActivationTracking();
// Kept off the boot path: this can open an SSH/SFTP connection per remote
// agent, so awaiting it here delayed the window opening. Session relaunch below
diff --git a/console/apps/switch-console-desktop/src/shared/core/managed-switch-server/managed-switch-server.ts b/console/apps/switch-console-desktop/src/shared/core/managed-switch-server/managed-switch-server.ts
index 97cd3131a..6fe7ac298 100644
--- a/console/apps/switch-console-desktop/src/shared/core/managed-switch-server/managed-switch-server.ts
+++ b/console/apps/switch-console-desktop/src/shared/core/managed-switch-server/managed-switch-server.ts
@@ -287,7 +287,23 @@ export type StartLocalServerResult =
// history was copied out of it, and the copy failed. The stack is untouched
// and still on `deployed`; retrying is safe and resumes where it stopped.
| { kind: 'matrix-migration-failed'; deployed: string; expected: string; detail: string }
- | { kind: 'error'; message: string };
+ | { kind: 'error'; message: string; failure: ManagedStartFailure };
+
+/**
+ * Which way a start failed that the result has no kind of its own for, as a
+ * code rather than its message: `start_refused` is the start's own check turning
+ * it down before anything ran, `health_timeout` a server that started but never
+ * answered, `core_exited` one that stopped during startup, `running_as_root` the
+ * app run as root (the local database refuses it), `operation_in_progress`
+ * another start or stop already running, and `unexpected` anything that threw.
+ */
+export type ManagedStartFailure =
+ | 'start_refused'
+ | 'health_timeout'
+ | 'core_exited'
+ | 'running_as_root'
+ | 'operation_in_progress'
+ | 'unexpected';
/** Outcome of starting a remote stack: a start's, or `cancelled` when the user
* stopped waiting for another Console's hold on the stack, before anything
diff --git a/console/packages/agent-controller/src/controller.test.ts b/console/packages/agent-controller/src/controller.test.ts
index 21f26214f..cd012a680 100644
--- a/console/packages/agent-controller/src/controller.test.ts
+++ b/console/packages/agent-controller/src/controller.test.ts
@@ -81,6 +81,7 @@ function deps(server = core.url): ControllerDeps {
rechecks: new ProviderRechecks(),
refreshPath: async () => {},
providerSeed: [],
+ sessionStarted: () => {},
tools: async () => [],
fetch,
openWebSocket: core.openWebSocket,
diff --git a/console/packages/agent-controller/src/controller.ts b/console/packages/agent-controller/src/controller.ts
index 6c4eca296..87dbf46d5 100644
--- a/console/packages/agent-controller/src/controller.ts
+++ b/console/packages/agent-controller/src/controller.ts
@@ -1,4 +1,8 @@
-import type { OpenAgentStream, ProviderReadiness } from '@switch-console/agent-providers';
+import type {
+ OpenAgentStream,
+ ProviderReadiness,
+ SessionStartedNotice,
+} from '@switch-console/agent-providers';
import { AgentHub } from './agent-hub';
import {
AccessTokens,
@@ -102,6 +106,11 @@ export type ControllerDeps = {
refreshPath: () => Promise<void>;
/** Provider checks made before this controller started, taken as its own (`ProviderStatuses.seed`). */
providerSeed: ProviderSeed;
+ /**
+ * A session of an agent here started, and Switch took its host's report:
+ * passed to whoever runs this controller (`SessionStartedNotice`).
+ */
+ sessionStarted: (started: Omit<SessionStartedNotice, 'kind'>) => void;
/** `git`, `gh` and vendor tools, as sessions here would be set up with them (`toolStatuses`). */
tools: () => Promise<ToolStatus[]>;
fetch: Fetch;
@@ -324,6 +333,11 @@ export async function runController(
},
log,
calls,
+ sessionStarted: ({ agentId, sessionId, startSource }) => {
+ const entry = assignment?.agents.find((held) => held.agent_id === agentId);
+ if (!entry) return;
+ deps.sessionStarted({ sessionId, provider: entry.definition.provider, startSource });
+ },
}),
});
/** The agents whose saved cursor the hub has been given. */
diff --git a/console/packages/agent-controller/src/main.ts b/console/packages/agent-controller/src/main.ts
index e774d6956..db3a723a8 100644
--- a/console/packages/agent-controller/src/main.ts
+++ b/console/packages/agent-controller/src/main.ts
@@ -10,6 +10,8 @@ import {
isStopMessage,
PROVIDER_RECHECKED_KIND,
type ProviderRecheckReply,
+ SESSION_STARTED_KIND,
+ type SessionStartedNotice,
} from '@switch-console/agent-providers';
import packageJson from '../package.json' with { type: 'json' };
import {
@@ -657,6 +659,19 @@ async function runCommand(args: string[]): Promise<number> {
rechecks,
refreshPath,
providerSeed,
+ // Only Switch Console's own controller has a parent listening on IPC.
+ sessionStarted: (started) => {
+ if (!process.send || !process.connected) return;
+ const notice: SessionStartedNotice = { kind: SESSION_STARTED_KIND, ...started };
+ // With a callback, a channel that closed meanwhile is reported here
+ // rather than as an `error` event on the process, which nothing handles.
+ process.send(notice, undefined, undefined, (error) => {
+ if (error)
+ log.warn('Could not tell Switch Console a session started', {
+ error: errorMessage(error),
+ });
+ });
+ },
tools: () => toolStatuses(process.env.PATH, process.platform, cliToolsBase()),
fetch,
openWebSocket: nodeWebSocket,
diff --git a/console/packages/agent-controller/src/relay-forward.ts b/console/packages/agent-controller/src/relay-forward.ts
index d334240f1..650a4ce4e 100644
--- a/console/packages/agent-controller/src/relay-forward.ts
+++ b/console/packages/agent-controller/src/relay-forward.ts
@@ -7,6 +7,7 @@ import {
} from 'node:http';
import { request as httpsRequest } from 'node:https';
import { pipeline } from 'node:stream/promises';
+import { type OwedSessionStart, reportedStartSource } from '@switch-console/agent-providers';
import { isRevoked, PROTOCOL_HEADER } from './api';
import { errorMessage, type Logger } from './log';
import type { Forwarder } from './relay';
@@ -53,6 +54,8 @@ const ERROR_BODY_BYTES = 64 * 1024;
/** An agent operation, `POST /agents/{agent}/ops/{operation}`, which the socket can carry. */
const OPERATION = /^\/agents\/([^/?]+)\/ops\/([^/?]+)$/;
+/** A session host's report that its session started (`host/session-start.ts`). */
+const SESSION_STARTED = /^\/agent-sessions\/([^/?]+)\/started$/;
export type UpstreamAuth = {
/** The controller's current access token. */
@@ -73,6 +76,12 @@ export class UpstreamForwarder implements Forwarder {
log: Logger;
/** Carries the agents' operations on the controller's socket, when it takes them. */
calls?: SocketCalls;
+ /** A session host's start report that Switch took. */
+ sessionStarted: (started: {
+ agentId: string;
+ sessionId: string;
+ startSource: OwedSessionStart;
+ }) => void;
}
) {}
@@ -157,7 +166,10 @@ export class UpstreamForwarder implements Forwarder {
res.end(text);
return;
}
- res.writeHead(upstream.statusCode ?? 502, responseHeaders(upstream.headers, true));
+ const status = upstream.statusCode ?? 502;
+ if (method === 'POST' && status >= 200 && status < 300 && buffered !== null)
+ this.noticeStart(target, buffered);
+ res.writeHead(status, responseHeaders(upstream.headers, true));
try {
await pipeline(upstream, res);
} catch (error) {
@@ -170,6 +182,32 @@ export class UpstreamForwarder implements Forwarder {
}
}
+ /** Passes on a session start Switch took, when this was the host's report of one. */
+ private noticeStart(target: { agentId: string; path: string }, body: Buffer): void {
+ const route = SESSION_STARTED.exec(new URL(target.path, 'http://relay.invalid').pathname);
+ if (!route) return;
+ let sessionId: string;
+ let startSource: OwedSessionStart | null;
+ try {
+ sessionId = decodeURIComponent(route[1]!);
+ startSource = reportedStartSource(
+ (JSON.parse(body.toString('utf8')) as { start_source?: unknown }).start_source
+ );
+ } catch {
+ return;
+ }
+ if (startSource === null) return;
+ try {
+ this.deps.sessionStarted({ agentId: target.agentId, sessionId, startSource });
+ } catch (error) {
+ // Passing the start on is a courtesy to whoever runs this controller;
+ // the report itself reached Switch, and its answer still goes back.
+ this.deps.log.warn('Could not pass on a session start', {
+ error: errorMessage(error),
+ });
+ }
+ }
+
/**
* An operation sent as a call on the controller's socket rather than as an
* HTTP request (`SocketCalls`), answered with the status and body the HTTP
diff --git a/console/packages/agent-controller/src/relay.test.ts b/console/packages/agent-controller/src/relay.test.ts
index 1f035f6f8..58654999c 100644
--- a/console/packages/agent-controller/src/relay.test.ts
+++ b/console/packages/agent-controller/src/relay.test.ts
@@ -93,6 +93,7 @@ let relay: LocalRelay;
let calls: SocketCalls;
let token: string;
let revokedCalls: number;
+let startsNoticed: { agentId: string; sessionId: string; startSource: string }[];
let cursorsSaved: [string, number][];
const stops: (() => void)[] = [];
@@ -118,6 +119,7 @@ function newRelay(): LocalRelay {
},
log: silentLogger,
calls,
+ sessionStarted: (started) => startsNoticed.push(started),
}),
});
}
@@ -127,6 +129,7 @@ beforeEach(async () => {
await core.start();
core.setAssignment({ revision: 1, agents: [assigned(AGENT), assigned('agent-2')] });
revokedCalls = 0;
+ startsNoticed = [];
cursorsSaved = [];
calls = new SocketCalls();
hub = new AgentHub({
@@ -681,6 +684,22 @@ describe('forwarding to Switch', () => {
expect(sent).toEqual([]);
});
+ it('passes on a session start once Switch has taken it, and nothing else', async () => {
+ await post('/agent-sessions/session-1/started', { start_source: 'room' });
+ await post('/agent-sessions/session-2/activity', { start_source: 'room' });
+ await post('/agent-sessions/session-3/started', { start_source: 'made-up' });
+ core.scripted.push({
+ method: 'POST',
+ path: '/agent-sessions/session-4/started',
+ status: 503,
+ body: { detail: 'not now' },
+ });
+ await post('/agent-sessions/session-4/started', { start_source: 'user' });
+ expect(startsNoticed).toEqual([
+ { agentId: AGENT, sessionId: 'session-1', startSource: 'room' },
+ ]);
+ });
+
it('answers 502 when the socket closes before Switch answers the call', async () => {
calls.open(() => setImmediate(() => calls.close('The controller socket closed.')), 4);
const response = await post(`/agents/${AGENT}/ops/post_message`, { body: 'lost' });
diff --git a/console/packages/agent-controller/src/systemd-runtime.e2e.test.ts b/console/packages/agent-controller/src/systemd-runtime.e2e.test.ts
index c752ca0df..9aa6d351a 100644
--- a/console/packages/agent-controller/src/systemd-runtime.e2e.test.ts
+++ b/console/packages/agent-controller/src/systemd-runtime.e2e.test.ts
@@ -138,6 +138,7 @@ describe.skipIf(!dataDir)('agents as users of their own, under systemd', () => {
rechecks: new ProviderRechecks(),
refreshPath: async () => {},
providerSeed: [],
+ sessionStarted: () => {},
tools: async () => [],
fetch,
openWebSocket: core.openWebSocket,
diff --git a/console/packages/agent-providers/src/host/session-started.ts b/console/packages/agent-providers/src/host/session-started.ts
new file mode 100644
index 000000000..a7ea3b457
--- /dev/null
+++ b/console/packages/agent-providers/src/host/session-started.ts
@@ -0,0 +1,40 @@
+import { hostStartSources, type OwedSessionStart } from './session-start';
+
+/**
+ * The agent controller telling the process that runs it, over its IPC
+ * channel, that a session of one of its agents has started and Switch has
+ * taken the report (`host/session-start.ts`).
+ *
+ * Switch Console listens on the controller it runs on its own computer, and
+ * counts the session in its usage data under its own consent gate. Nothing
+ * else hears it: a controller on an SSH host or run headless has no parent
+ * listening, so its sessions reach only the Switch server they work for.
+ */
+export const SESSION_STARTED_KIND = 'switch-session-started';
+
+export type SessionStartedNotice = {
+ kind: typeof SESSION_STARTED_KIND;
+ /** Lets the listener count a start the host reported twice once. Never sent on. */
+ sessionId: string;
+ /** The agent's provider, as its definition names it. */
+ provider: string;
+ startSource: OwedSessionStart;
+};
+
+const START_SOURCES: readonly string[] = [...hostStartSources, 'unknown'];
+
+export function isSessionStartedNotice(raw: unknown): raw is SessionStartedNotice {
+ if (typeof raw !== 'object' || raw === null) return false;
+ const notice = raw as Partial<Record<keyof SessionStartedNotice, unknown>>;
+ return (
+ notice.kind === SESSION_STARTED_KIND &&
+ typeof notice.sessionId === 'string' &&
+ typeof notice.provider === 'string' &&
+ reportedStartSource(notice.startSource) !== null
+ );
+}
+
+/** The start source a session host reported, or null when it is not one. */
+export function reportedStartSource(raw: unknown): OwedSessionStart | null {
+ return typeof raw === 'string' && START_SOURCES.includes(raw) ? (raw as OwedSessionStart) : null;
+}
diff --git a/console/packages/agent-providers/src/index.ts b/console/packages/agent-providers/src/index.ts
index d37d84df5..96b75b09d 100644
--- a/console/packages/agent-providers/src/index.ts
+++ b/console/packages/agent-providers/src/index.ts
@@ -84,7 +84,17 @@ export {
type ProviderRecheckReply,
type ProviderRecheckRequest,
} from './host/provider-recheck';
-export { hostStartSources, type HostStartSource } from './host/session-start';
+export {
+ hostStartSources,
+ type HostStartSource,
+ type OwedSessionStart,
+} from './host/session-start';
+export {
+ isSessionStartedNotice,
+ reportedStartSource,
+ SESSION_STARTED_KIND,
+ type SessionStartedNotice,
+} from './host/session-started';
export {
type AgentEventStream,
type AgentStreamDeps,
diff --git a/core/switch_core/bridges/agent/api/session_reporter.py b/core/switch_core/bridges/agent/api/session_reporter.py
index ddb3feffc..65b0e76ca 100644
--- a/core/switch_core/bridges/agent/api/session_reporter.py
+++ b/core/switch_core/bridges/agent/api/session_reporter.py
@@ -1,4 +1,6 @@
-"""Reporting an agent session, which is not the same as a connection.
+"""Reporting an agent coming online and going offline (`agent_connected` /
+`agent_disconnected`), which is not the same as a connection, and not a
+coding-agent session either: hosts report those (`telemetry/session_start.py`).
Two ordinary things make an agent hold a succession of short connections while
being continuously present: a stream rejected at the room claim and retried,
@@ -24,6 +26,7 @@ from switch_core.bridges.agent.protocol.agent_connections import (
)
from switch_core.db.models import Agent
from switch_core.telemetry import TelemetryService, emit_safely
+from switch_core.telemetry.catalogue import PropertyValue
from switch_core.telemetry.snapshot import normalise_known_agent_type
logger = logging.getLogger(__name__)
@@ -81,13 +84,19 @@ class SessionReporter:
runtime = normalise_known_agent_type(agent.metadata_)
self._sessions[conn.agent_id] = _Session(time.monotonic(), runtime)
+ # Both names until the old ones are retired: `agent_session_*` and
+ # `first_session_started` are connections, which the new names say.
emit_safely(
self._telemetry, "agent_session_started", {"known_agent_type": runtime}
)
+ emit_safely(self._telemetry, "agent_connected", {"known_agent_type": runtime})
if self._telemetry is not None:
await self._telemetry.emit_milestone(
"first_session_started", known_agent_type=runtime
)
+ await self._telemetry.emit_milestone(
+ "first_agent_connected", known_agent_type=runtime
+ )
def on_close(self, conn: AgentConnection) -> None:
"""The registry's close listener, for every path that closes."""
@@ -131,14 +140,12 @@ class SessionReporter:
session = self._sessions.pop(agent_id, None)
if session is None:
return
- emit_safely(
- self._telemetry,
- "agent_session_ended",
- {
- "duration_seconds": max(time.monotonic() - session.started_at, 0.0),
- "reason": reason,
- },
- )
+ properties: dict[str, PropertyValue] = {
+ "duration_seconds": max(time.monotonic() - session.started_at, 0.0),
+ "reason": reason,
+ }
+ emit_safely(self._telemetry, "agent_session_ended", properties)
+ emit_safely(self._telemetry, "agent_disconnected", properties)
async def aclose(self) -> None:
"""Cancel any pending ends at shutdown, without reporting them: every
diff --git a/core/switch_core/bridges/agent/operations/my_context.py b/core/switch_core/bridges/agent/operations/my_context.py
index 8252a0b07..a0927a826 100644
--- a/core/switch_core/bridges/agent/operations/my_context.py
+++ b/core/switch_core/bridges/agent/operations/my_context.py
@@ -36,7 +36,6 @@ from switch_core.db.models import (
)
from switch_core.db.session_scope import tenant_session
from switch_core.db.stores.service_connection_store import ServiceConnectionStore
-from switch_core.feature_flags import SWITCH_CLOUD
from switch_core.version import switch_core_version
_CONNECTIONS = ServiceConnectionStore()
@@ -209,7 +208,7 @@ async def get_my_context() -> dict[str, Any]:
"members": members,
},
"server": {
- "kind": _server_kind(protocol.config),
+ "kind": protocol.config.deployment_kind,
"version": switch_core_version(),
"features": sorted(
key for key, enabled in protocol.config.feature_flags.items() if enabled
@@ -263,14 +262,6 @@ async def get_my_context() -> dict[str, Any]:
}
-def _server_kind(config: Any) -> str:
- if config.feature_flags[SWITCH_CLOUD]:
- return "switch_cloud"
- if config.managed_by_console:
- return "managed_by_console"
- return "self_hosted"
-
-
def _how_to_add(protocol: Any) -> list[dict[str, Any]]:
service = protocol.install_service
official = set(service.platforms()) if service is not None else set()
diff --git a/core/switch_core/bridges/collaboration/collaboration_core.py b/core/switch_core/bridges/collaboration/collaboration_core.py
index 7d33e16bd..9747a747d 100644
--- a/core/switch_core/bridges/collaboration/collaboration_core.py
+++ b/core/switch_core/bridges/collaboration/collaboration_core.py
@@ -1306,6 +1306,9 @@ class CollaborationCore:
else None
),
metered=True,
+ extra_content=(
+ {SENDER_IS_APP_MARKER: True} if msg.sender_is_app else None
+ ),
)
if event_id is None:
logger.error(
diff --git a/core/switch_core/bridges/collaboration/discord/adapter.py b/core/switch_core/bridges/collaboration/discord/adapter.py
index d7ef0ed4d..1e7529d93 100644
--- a/core/switch_core/bridges/collaboration/discord/adapter.py
+++ b/core/switch_core/bridges/collaboration/discord/adapter.py
@@ -2994,6 +2994,10 @@ class DiscordAdapter(PlatformAdapter):
attachments=attachments,
attachment_failures=attachment_failures,
self_mention_token=str(bot_user_id) if self_mention else None,
+ # Another bot, or a webhook that is not one of ours (ours were
+ # dropped above): an integration posting, not a person.
+ sender_is_app=bool(getattr(author, "bot", False))
+ or getattr(message, "webhook_id", None) is not None,
)
)
diff --git a/core/switch_core/bridges/collaboration/mattermost/adapter.py b/core/switch_core/bridges/collaboration/mattermost/adapter.py
index 4ed8afcda..145090d2f 100644
--- a/core/switch_core/bridges/collaboration/mattermost/adapter.py
+++ b/core/switch_core/bridges/collaboration/mattermost/adapter.py
@@ -2804,16 +2804,26 @@ class MattermostAdapter(PlatformAdapter):
raise RuntimeError("Cannot handle posted event: adapter not started")
ws_loop = asyncio.get_event_loop()
+ user_is_bot = False
try:
user = await ws_loop.run_in_executor(
None, self._admin_driver.users.get_user, user_id
)
username: str = user.get("username", user_id)
+ user_is_bot = bool(user.get("is_bot", False))
except Exception:
logger.exception("Failed to resolve Mattermost user %s", user_id)
username = user_id
props = post.get("props") or {}
- if str(props.get("from_webhook", "")).lower() == "true":
+ from_webhook = str(props.get("from_webhook", "")).lower() == "true"
+ # An integration rather than a person: an incoming webhook, or a bot
+ # account other than the bridge's own (those never reach here).
+ sender_is_app = (
+ from_webhook
+ or user_is_bot
+ or str(props.get("from_bot", "")).lower() == "true"
+ )
+ if from_webhook:
# An incoming webhook's post (or a custom slash command's reply)
# carries the user id of whoever set the integration up, though
# anyone holding its URL may have sent it. It is the integration
@@ -2873,6 +2883,7 @@ class MattermostAdapter(PlatformAdapter):
channel_name=channel_name,
attachments=attachments,
attachment_failures=attachment_failures,
+ sender_is_app=sender_is_app,
)
coro = self._on_message(inbound)
self._dispatch(coro, loop) # type: ignore[arg-type]
diff --git a/core/switch_core/bridges/collaboration/models.py b/core/switch_core/bridges/collaboration/models.py
index 281d078cf..704c3c9d3 100644
--- a/core/switch_core/bridges/collaboration/models.py
+++ b/core/switch_core/bridges/collaboration/models.py
@@ -131,11 +131,13 @@ class InboundMessage(BaseModel):
# tagged. Lets the bridge guide users who tag the app instead of an agent.
self_mention_token: str | None = None
# The platform reported this post as coming from an app rather than a
- # person (a Slack workflow, a third-party integration). Such a post is
- # relayed like any other, but it cannot answer a request: a decision is
- # attributed to whoever made it, and an app made none. Only Slack reports
- # it today; a platform that cannot tell an app from a person leaves this
- # False and its cards accept the answer.
+ # person (a Slack workflow, a third-party integration, another Switch
+ # server's agent). Such a post is relayed like any other, but it cannot
+ # answer a request: a decision is attributed to whoever made it, and an app
+ # made none. Usage telemetry leaves it out of every count of people for the
+ # same reason. Slack, Discord and Mattermost report it. Telegram never
+ # delivers another bot's posts, and Teams does not mark them, so both leave
+ # this False and their cards accept the answer.
sender_is_app: bool = False
diff --git a/core/switch_core/clients/actor.py b/core/switch_core/clients/actor.py
index 472600ed6..48acd9208 100644
--- a/core/switch_core/clients/actor.py
+++ b/core/switch_core/clients/actor.py
@@ -281,6 +281,7 @@ class Actor[ConfigT: ClientConfig]:
caption: str | None = None,
thread_root_id: str | None = None,
group: dict[str, object] | None = None,
+ extra_content: dict[str, object] | None = None,
) -> str | None:
"""Send an m.image / m.file event pointing at an uploaded media URI.
@@ -309,6 +310,7 @@ class Actor[ConfigT: ClientConfig]:
caption=caption,
thread_root_id=thread_root_id,
group=group,
+ extra_content=extra_content,
)
except TransportError as exc:
logger.error("Failed to send media to %s: %s", room_id, exc)
diff --git a/core/switch_core/clients/admin_messages.py b/core/switch_core/clients/admin_messages.py
index af7fca396..fcb06bc49 100644
--- a/core/switch_core/clients/admin_messages.py
+++ b/core/switch_core/clients/admin_messages.py
@@ -43,9 +43,10 @@ ADMIN_MARKER = "com.switch.admin"
PLATFORM_MARKER = "com.switch.platform"
# Set by the collaboration bridge on a message the platform reported as posted
-# by an app (a workflow, an integration) rather than a person. Only the bridge
-# writes it, and only Slack reports it today. Following the conversation
-# (`delivery/conversation.py`) never delivers such a message.
+# by an app (a workflow, an integration, another Switch server's agent) rather
+# than a person. Only the bridge writes it; Slack, Discord and Mattermost report
+# it. Following the conversation (`delivery/conversation.py`) never delivers
+# such a message, and usage telemetry counts its sender as a bot, not a person.
SENDER_IS_APP_MARKER = "com.switch.sender_is_app"
diff --git a/core/switch_core/config.py b/core/switch_core/config.py
index 79acba652..1a89b0ca1 100644
--- a/core/switch_core/config.py
+++ b/core/switch_core/config.py
@@ -24,10 +24,13 @@ from switch_core.feature_flags import (
KNOWN_FEATURE_FLAGS,
ORGANIZATION_CONNECTOR_APPS,
SERVICE_CONNECTIONS,
+ SWITCH_CLOUD,
)
from switch_core.keys import Keyring
from switch_core.outbound import OutboundPolicy
+DeploymentKind = Literal["switch_cloud", "managed_by_console", "self_hosted"]
+
# A Postgres time value: a bare count of milliseconds, or a count with a unit.
_PG_INTERVAL_RE = re.compile(r"^\d+\s*(us|ms|s|min|h|d)?$")
@@ -1314,6 +1317,16 @@ class SwitchConfig(BaseSettings):
enabled = self._enabled_feature_flags()
return {key: key in enabled for key in sorted(KNOWN_FEATURE_FLAGS)}
+ @property
+ def deployment_kind(self) -> DeploymentKind:
+ """Who runs this server: Switch Cloud (its feature flag), Switch Console
+ for the person using it (`managed_by_console`), or anyone else."""
+ if self.feature_flags[SWITCH_CLOUD]:
+ return "switch_cloud"
+ if self.managed_by_console:
+ return "managed_by_console"
+ return "self_hosted"
+
@property
def agent_management_enabled(self) -> bool:
return self.feature_flags[AGENT_MANAGEMENT]
diff --git a/core/switch_core/db/stores/user_store.py b/core/switch_core/db/stores/user_store.py
index 441563c24..4b7d0e169 100644
--- a/core/switch_core/db/stores/user_store.py
+++ b/core/switch_core/db/stores/user_store.py
@@ -3,6 +3,7 @@ from __future__ import annotations
import logging
from collections.abc import Collection
from datetime import timedelta
+from typing import NamedTuple
from sqlalchemy import case, func, select
from sqlalchemy.dialects.postgresql import insert
@@ -49,6 +50,14 @@ def email_taken(err: IntegrityError) -> bool:
return getattr(cause, "constraint_name", None) == _EMAIL_CONSTRAINT
+class OidcUser(NamedTuple):
+ """The account an OIDC identity resolved to, and whether resolving it
+ made the account (a first sign-in) rather than finding or linking one."""
+
+ user: User
+ created: bool
+
+
class OidcIdentityConflictError(Exception):
"""An unverified OIDC login's email already belongs to a different account.
@@ -274,9 +283,9 @@ class UserStore:
sub: str,
email_verified: bool,
join_tenant: bool,
- ) -> User:
+ ) -> OidcUser:
"""Resolve an OIDC identity to a gateway user, provisioning on first
- login (JIT).
+ login (JIT). `created` says whether this call made the account.
`join_tenant` decides whether an account this creates, or links, is
given a membership in the tenant bound to the caller's context. With
@@ -315,7 +324,7 @@ class UserStore:
for attempt in range(1, _MAX_ATTEMPTS + 1):
user = await self.get_by_oidc_identity(session, iss=iss, sub=sub)
if user is not None:
- return user
+ return OidcUser(user, created=False)
existing = await self.get_by_email(session, email)
if existing is not None:
@@ -336,7 +345,7 @@ class UserStore:
except IntegrityError:
self._raise_if_exhausted(attempt)
continue
- return existing
+ return OidcUser(existing, created=False)
user = User(name=name, email=email.lower(), role="user", password_hash=None)
try:
@@ -356,7 +365,7 @@ class UserStore:
except IntegrityError:
self._raise_if_exhausted(attempt)
continue
- return user
+ return OidcUser(user, created=True)
raise AssertionError("unreachable: the loop above always returns or raises")
diff --git a/core/switch_core/delivery/recipient.py b/core/switch_core/delivery/recipient.py
index e05c9c000..87dc3bc11 100644
--- a/core/switch_core/delivery/recipient.py
+++ b/core/switch_core/delivery/recipient.py
@@ -54,6 +54,7 @@ from switch_core.budgets import BudgetExceeded, BudgetGuard
from switch_core.clients.admin_messages import (
AUTO_REPLY_FLAG,
PLATFORM_MARKER,
+ SENDER_IS_APP_MARKER,
platform_on_behalf_of,
platform_replies_in_channel,
)
@@ -1418,7 +1419,7 @@ class AgentRecipient:
tenant_session(self.session_factory, self.tenant_id) as session,
):
machine = await wake_controller_machine(
- session, controller_id, datetime.now(UTC)
+ session, controller_id, datetime.now(UTC), "message"
)
state = None if machine is None else CloudMachineState.of(machine)
await session.commit()
@@ -1710,6 +1711,7 @@ class AgentRecipient:
room_id=meta.room_id,
sender_transport_user_id=event.sender,
from_platform=PLATFORM_MARKER in event.content,
+ from_app=event.content.get(SENDER_IS_APP_MARKER) is True,
agent_metadata=self.agent.metadata_,
agent_live=agent_live,
has_attachment=has_attachment,
diff --git a/core/switch_core/gateway/auth_routes.py b/core/switch_core/gateway/auth_routes.py
index 45a675296..f55ba5abd 100644
--- a/core/switch_core/gateway/auth_routes.py
+++ b/core/switch_core/gateway/auth_routes.py
@@ -50,6 +50,7 @@ from switch_core.gateway.auth import (
verify_password_off_loop,
)
from switch_core.gateway.dependencies import (
+ current_telemetry,
get_agent_store,
get_api_key_store,
get_bridge_store,
@@ -91,6 +92,7 @@ from switch_core.gateway.schemas import (
)
from switch_core.providers.hosted import HostedControllerSettings
from switch_core.room_service import RoomService
+from switch_core.telemetry import emit_safely
from switch_core.tenant_context import tenant_scope
from switch_core.version import server_declaration
@@ -216,6 +218,7 @@ async def _refuse_signup(
config.gateway_signup_max_per_hour,
)
await session.rollback()
+ emit_safely(current_telemetry(), "signup_refused", {"reason": "rate_limited"})
raise HTTPException(
status_code=429,
detail="Too many sign-ups on this server in the last hour. Try again later.",
@@ -223,6 +226,7 @@ async def _refuse_signup(
)
if await user_store.get_by_email(session, email) is not None:
await session.rollback()
+ emit_safely(current_telemetry(), "signup_refused", {"reason": "email_taken"})
raise HTTPException(status_code=409, detail="Email already registered")
@@ -246,6 +250,7 @@ async def signup(
fails the sign-up: the response says why instead.
"""
if not config.gateway_signup_open:
+ emit_safely(current_telemetry(), "signup_refused", {"reason": "disabled"})
raise HTTPException(
status_code=403, detail="Sign-up is disabled on this server"
)
@@ -275,11 +280,15 @@ async def signup(
if not email_taken(err):
raise
await session.rollback()
+ emit_safely(
+ current_telemetry(), "signup_refused", {"reason": "email_taken"}
+ )
raise HTTPException(
status_code=409, detail="Email already registered"
) from None
await session.commit()
logger.info("Signed up user: %s (%s)", user.email, user.id)
+ emit_safely(current_telemetry(), "account_created", {"method": "signup"})
set_session_cookie(
response,
@@ -628,6 +637,7 @@ async def create_user(
await session.commit()
logger.info("Created user: %s (%s)", user.email, user.id)
+ emit_safely(current_telemetry(), "account_created", {"method": "admin"})
return UserResponse(
id=user.id,
name=user.name,
diff --git a/core/switch_core/gateway/cloud_controllers.py b/core/switch_core/gateway/cloud_controllers.py
index a8c7ad374..438a28d5c 100644
--- a/core/switch_core/gateway/cloud_controllers.py
+++ b/core/switch_core/gateway/cloud_controllers.py
@@ -16,7 +16,7 @@ not import (`management/wiring.py` installs it here with
from __future__ import annotations
from datetime import UTC, datetime, timedelta
-from typing import Any, Protocol
+from typing import Any, Literal, Protocol
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
@@ -33,7 +33,9 @@ from switch_core.db.stores.hosted_machine_store import (
bump_revision,
idle_sleeping,
)
+from switch_core.gateway.dependencies import current_telemetry
from switch_core.keys import Keyring
+from switch_core.telemetry import emit_after_commit
# How recent the machine's last status report must be for its silence to count
# as idle rather than as a machine that stopped reporting.
@@ -212,15 +214,36 @@ async def record_controller_status(
machine.error = None
machine.error_code = None
machine.updated_at = now
+ # Revision 1 is the machine as first claimed: anything since (a wake, a
+ # restart, a workspace joining) bumped it. Only the first is timed,
+ # since nothing records when a later start began.
+ new_machine = machine.revision == 1
+ emit_after_commit(
+ session,
+ current_telemetry(),
+ "cloud_machine_ready",
+ {
+ "new_machine": new_machine,
+ "provisioning_seconds": (
+ max((now - machine.created_at).total_seconds(), 0.0)
+ if new_machine
+ else -1
+ ),
+ },
+ )
async def wake_controller_machine(
- session: AsyncSession, controller_id: str, now: datetime
+ session: AsyncSession,
+ controller_id: str,
+ now: datetime,
+ trigger: Literal["message", "agent_placed"],
) -> CloudMachine | None:
"""The cloud machine running `controller_id`, started again if it was put
to sleep when idle, and kept awake otherwise. A machine its owner stopped
is left stopped. None for a controller that runs on no cloud machine. The
- caller holds no other machine lock, and commits."""
+ caller holds no other machine lock, and commits; `trigger` is what woke
+ it, for telemetry."""
found = await machine_of_controller(session, controller_id)
if found is None:
return None
@@ -228,6 +251,12 @@ async def wake_controller_machine(
store = CloudMachineStore()
if idle_sleeping(machine):
store.start(machine, now)
+ emit_after_commit(
+ session,
+ current_telemetry(),
+ "cloud_machine_woken",
+ {"trigger": trigger, "stopped_by": "idle"},
+ )
elif machine.desired_state == "running":
machine.active_at = now
return machine
diff --git a/core/switch_core/gateway/hosted_controller.py b/core/switch_core/gateway/hosted_controller.py
index 47a415438..bd8ca1a4e 100644
--- a/core/switch_core/gateway/hosted_controller.py
+++ b/core/switch_core/gateway/hosted_controller.py
@@ -30,9 +30,14 @@ from switch_core.gateway.cloud_controllers import (
enrollment_code,
live_controller,
)
-from switch_core.gateway.dependencies import get_config, get_session_factory
+from switch_core.gateway.dependencies import (
+ current_telemetry,
+ get_config,
+ get_session_factory,
+)
from switch_core.gateway.hosted_machines import controller_settings
from switch_core.providers.hosted import HostedControllerSettings
+from switch_core.telemetry import emit_after_commit
from switch_core.tenant_context import no_tenant, tenant_scope
logger = logging.getLogger(__name__)
@@ -143,16 +148,19 @@ async def _sweep(
machine.error_code = "machine_connect_timeout"
machine.error = "The cloud machine was not scheduled within 10 minutes. Retry it in Switch Console, or contact your administrator if it still cannot start."
machine.updated_at = now
+ _report_failure(session, "not_scheduled")
elif _connect_timed_out(machine, now):
machine.state = "error"
machine.error_code = "machine_connect_timeout"
machine.error = "The cloud machine started but did not connect to Switch within 10 minutes. Retry it in Switch Console, or ask your administrator to check the machine's startup logs."
machine.updated_at = now
+ _report_failure(session, "never_connected")
elif _start_timed_out(machine, now):
machine.state = "error"
machine.error_code = "machine_connect_timeout"
machine.error = "The cloud machine did not start within 10 minutes. Retry it in Switch Console, or contact your administrator if it still cannot start."
machine.updated_at = now
+ _report_failure(session, "never_started")
elif machine.state in {"retained", "error"} and retention_expired(machine, now):
machine.desired_state = "deleted"
bump_revision(machine, now)
@@ -174,6 +182,34 @@ async def _sweep(
machine, workspaces, retention_days=retention_days, now=now
):
store.stop(machine, "idle", now)
+ emit_after_commit(
+ session,
+ current_telemetry(),
+ "cloud_machine_stopped",
+ {"reason": "idle"},
+ )
+
+
+FailureReason = Literal[
+ "not_scheduled",
+ "never_started",
+ "never_connected",
+ "owner_left",
+ "needs_attention",
+ "reported",
+]
+
+
+def _observed_failure(error_code: str | None) -> FailureReason:
+ """An error the machine reported: `needs_attention` for the one code an
+ observation can carry, `reported` for any other."""
+ return "needs_attention" if error_code == "machine_needs_attention" else "reported"
+
+
+def _report_failure(session: AsyncSession, reason: FailureReason) -> None:
+ emit_after_commit(
+ session, current_telemetry(), "cloud_machine_failed", {"reason": reason}
+ )
@router.get("/machines")
@@ -285,6 +321,8 @@ async def prepare(
},
}
if not controllers:
+ if machine.state != "error":
+ _report_failure(session, "owner_left")
machine.state = "error"
machine.error = "The cloud machine's owner is no longer a member of any workspace it serves."
machine.updated_at = now
@@ -350,6 +388,8 @@ async def observe(
elif (
body.state == "error" or body.error is not None
) and machine.state not in STALE_ERROR_IGNORED_STATES:
+ if machine.state != "error":
+ _report_failure(session, _observed_failure(body.error_code))
machine.state = "error"
machine.error = body.error or OBSERVED_ERROR
machine.error_code = body.error_code
@@ -373,6 +413,8 @@ async def observe(
if machine.state != "ready":
machine.state = "provisioning"
elif body.state == "error":
+ if machine.state != "error":
+ _report_failure(session, _observed_failure(body.error_code))
machine.state = "error"
machine.error = body.error or OBSERVED_ERROR
machine.error_code = body.error_code
diff --git a/core/switch_core/gateway/hosted_machines.py b/core/switch_core/gateway/hosted_machines.py
index bd72abaf5..8432e5e50 100644
--- a/core/switch_core/gateway/hosted_machines.py
+++ b/core/switch_core/gateway/hosted_machines.py
@@ -26,11 +26,13 @@ from switch_core.db.stores.hosted_machine_store import (
from switch_core.gateway.auth import get_current_user
from switch_core.gateway.cloud_controllers import workspace_reported
from switch_core.gateway.dependencies import (
+ current_telemetry,
get_config,
get_session,
get_session_factory,
)
from switch_core.providers.hosted import HostedControllerSettings
+from switch_core.telemetry import emit_after_commit
router = APIRouter(prefix="/hosted-machines")
@@ -157,6 +159,7 @@ async def ensure_machine(
machines = CloudMachineStore()
await lock_claims(session)
existing = await machines.live_for_owner(session, owner_id)
+ workspace = None
if existing is not None:
existing = await machines.locked(session, existing.id)
workspace = (
@@ -171,13 +174,36 @@ async def ensure_machine(
)
):
return existing, workspace
- return await machines.claim(
+ change = _claim_change(existing, workspace)
+ claimed = await machines.claim(
session,
factory,
owner_id=owner_id,
capacity=config.hosted_launch_capacity,
now=datetime.now(UTC),
)
+ if change is not None:
+ emit_after_commit(
+ session, current_telemetry(), "cloud_machine_requested", {"change": change}
+ )
+ return claimed
+
+
+def _claim_change(
+ existing: CloudMachine | None, workspace: MachineWorkspace | None
+) -> str | None:
+ """What claiming the owner's machine will change, as `claim` decides it,
+ or None when it only marks a running machine active. The first that
+ applies, when a claim does two of these at once."""
+ if existing is None:
+ return "new"
+ if workspace is None:
+ return "joined_workspace"
+ if existing.desired_state == "retained":
+ return "unretained"
+ if existing.desired_state == "stopped":
+ return "restarted"
+ return None
async def _owned(
@@ -275,8 +301,22 @@ async def lifecycle(
raise HTTPException(409, f"machine is {machine.desired_state}")
now = datetime.now(UTC)
if body.action == "stop":
+ if machine.desired_state != "stopped":
+ emit_after_commit(
+ session,
+ current_telemetry(),
+ "cloud_machine_stopped",
+ {"reason": "owner"},
+ )
machines.stop(machine, "owner", now)
elif body.action == "start":
+ if machine.desired_state == "stopped":
+ emit_after_commit(
+ session,
+ current_telemetry(),
+ "cloud_machine_woken",
+ {"trigger": "owner", "stopped_by": machine.stop_reason or "unknown"},
+ )
machines.start(machine, now)
else:
if machine.state != "error":
diff --git a/core/switch_core/gateway/oidc_routes.py b/core/switch_core/gateway/oidc_routes.py
index 5308596a5..2ba8e3acd 100644
--- a/core/switch_core/gateway/oidc_routes.py
+++ b/core/switch_core/gateway/oidc_routes.py
@@ -27,11 +27,13 @@ from switch_core.db.stores.user_store import (
)
from switch_core.gateway.auth import initial_tenant_claim, set_session_cookie
from switch_core.gateway.dependencies import (
+ current_telemetry,
get_config,
get_session_factory,
get_system_session,
get_user_store,
)
+from switch_core.telemetry import emit_safely
from switch_core.tenant_context import tenant_scope
logger = logging.getLogger(__name__)
@@ -329,7 +331,7 @@ async def oidc_callback(
)
with scope:
try:
- user = await user_store.get_or_create_oidc_user(
+ user, created = await user_store.get_or_create_oidc_user(
session,
iss=iss,
email=email,
@@ -353,6 +355,8 @@ async def oidc_callback(
headers={"Retry-After": "1"},
) from exc
await session.commit()
+ if created:
+ emit_safely(current_telemetry(), "account_created", {"method": "oidc"})
# Verify-at-login only: we don't persist the IdP tokens.
handoff = request.session.pop(_HANDOFF_SESSION_KEY, None)
diff --git a/core/switch_core/gateway/workspace_creation.py b/core/switch_core/gateway/workspace_creation.py
index f9a123e06..4c3eb2de1 100644
--- a/core/switch_core/gateway/workspace_creation.py
+++ b/core/switch_core/gateway/workspace_creation.py
@@ -44,11 +44,13 @@ from switch_core.gateway.auth import (
set_session_cookie,
workspace_creation_refusal,
)
+from switch_core.gateway.dependencies import current_telemetry
from switch_core.gateway.schemas import (
OrganizationCreateResponse,
OrganizationSummary,
TenantMembershipResponse,
)
+from switch_core.telemetry import emit_safely
logger = logging.getLogger(__name__)
@@ -416,6 +418,11 @@ async def _add_creator(
creator_id,
)
raise
+ emit_safely(
+ current_telemetry(),
+ "workspace_created",
+ {"new_organization": new_organization is not None},
+ )
async def _discard_organization(
diff --git a/core/switch_core/main.py b/core/switch_core/main.py
index 81d1cb531..8358c5d8c 100644
--- a/core/switch_core/main.py
+++ b/core/switch_core/main.py
@@ -1615,6 +1615,9 @@ async def run(config: SwitchConfig, process_id: str) -> None:
"Switch is running on http://%s:%d", config.server_host, config.server_port
)
+ # Before the server takes a connection or a session report, either of
+ # which can report a milestone this settles.
+ await telemetry.settle_superseded_milestones()
telemetry.emit("deployment_started", tenant_count=len(tenant_ids))
# The funnel's first step, and only for a deployment with an install date.
await telemetry.emit_milestone("deployment_installed")
diff --git a/core/switch_core/management/service.py b/core/switch_core/management/service.py
index 4ab24dc9a..544915590 100644
--- a/core/switch_core/management/service.py
+++ b/core/switch_core/management/service.py
@@ -1224,7 +1224,9 @@ class ManagementService:
if check_placement:
# Placing an agent on a sleeping Switch cloud machine wakes it.
machine = (
- await wake_controller_machine(session, controller.id, self.now())
+ await wake_controller_machine(
+ session, controller.id, self.now(), "agent_placed"
+ )
if controller.kind == "ec2"
else None
)
diff --git a/core/switch_core/telemetry/__init__.py b/core/switch_core/telemetry/__init__.py
index 489fb6be1..4f03e2995 100644
--- a/core/switch_core/telemetry/__init__.py
+++ b/core/switch_core/telemetry/__init__.py
@@ -6,19 +6,25 @@ switches it on. The events are declared in
:mod:`switch_core.telemetry.catalogue` and explained in
``docs/old/telemetry-events.md``.
-Call sites want two things from this package and nothing else: `emit_safely`,
-to report an event from the middle of real work without a catalogue mistake
-being able to break it, and the `TelemetryService` that gets handed around.
+Call sites want two things from this package and nothing else: `emit_safely`
+(or `emit_after_commit`, for a change a caller further up commits), to report
+an event from the middle of real work without a catalogue mistake being able
+to break it, and the `TelemetryService` that gets handed around.
Everything else here is wiring.
"""
from switch_core.telemetry.catalogue import TelemetryCatalogueError
-from switch_core.telemetry.service import TelemetryService, emit_safely
+from switch_core.telemetry.service import (
+ TelemetryService,
+ emit_after_commit,
+ emit_safely,
+)
from switch_core.telemetry.setup import build_telemetry
__all__ = [
"TelemetryCatalogueError",
"TelemetryService",
"build_telemetry",
+ "emit_after_commit",
"emit_safely",
]
diff --git a/core/switch_core/telemetry/catalogue.py b/core/switch_core/telemetry/catalogue.py
index c72786820..da574fae0 100644
--- a/core/switch_core/telemetry/catalogue.py
+++ b/core/switch_core/telemetry/catalogue.py
@@ -22,6 +22,12 @@ from switch_core import messaging_platforms
# against the same relay.
EVENT_NAME_PREFIX = "switch_core"
+# Added to every event by the service, not declared by any: who runs the
+# deployment (`SwitchConfig.deployment_kind`), so Switch Cloud's traffic can be
+# told from the servers people run themselves on any chart. Reserved, so no
+# event can declare a property of its own under the name.
+DEPLOYMENT_KIND_PROPERTY = "deployment_kind"
+
class PropertyType:
"""Base for the three kinds of value a property may take."""
@@ -147,10 +153,13 @@ LOOKED_UP_AGENT_TYPE = one_of(*KNOWN_AGENT_TYPE.values, "unknown")
ACTOR_KIND = one_of("user", "agent", "system")
-# Who said a message. `platform` is Switch itself speaking — a template's
-# kickoff, or a request it carries on a person's behalf — and `unknown` a
-# sender whose client could not be looked up.
-SENDER_KIND = one_of("user", "agent", "platform", "unknown")
+# Who said a message. `user` is a person on a messaging platform; `bot` is an
+# integration posting there (a Slack app, a Discord bot or webhook, a
+# Mattermost bot account, another Switch server's agent), which until this
+# value existed was counted as `user`. `platform` is Switch itself speaking — a
+# template's kickoff, or a request it carries on a person's behalf — and
+# `unknown` a sender whose client could not be looked up.
+SENDER_KIND = one_of("user", "bot", "agent", "platform", "unknown")
OUTCOME = one_of("success", "failure")
@@ -199,13 +208,28 @@ _SNAPSHOT_COUNTS = (
"user_count",
"user_internal_count",
# Switch accounts, not chat identities — see `chat_identity_*` for those.
+ # Superseded by `account_active_*`, which is the same count; kept for one
+ # stable release so charts can move across.
"user_active_1d",
"user_active_7d",
"user_internal_active_1d",
"user_internal_active_7d",
"chat_identity_count",
+ # Superseded by `person_identity_active_*` plus `bot_identity_active_*`:
+ # this counts an integration's identity as a person. Kept for one stable
+ # release.
"chat_identity_active_1d",
"chat_identity_active_7d",
+ # Who used a room with an agent, split three ways. People: chat identities
+ # that never posted as an app, linked to a Switch account or not (the
+ # headline). Bots: identities whose platform marked their posts as an
+ # app's. Accounts: Switch accounts whose linked identity spoke (the floor).
+ "person_identity_active_1d",
+ "person_identity_active_7d",
+ "bot_identity_active_1d",
+ "bot_identity_active_7d",
+ "account_active_1d",
+ "account_active_7d",
# `room_count` is the headline: rooms a *person* made. The other two keep
# agent scratch rooms and adopted channels out of it.
"room_count",
@@ -271,7 +295,7 @@ _SINCE_INSTALL: Mapping[str, PropertyType] = {"seconds_since_install": NUMBER}
CATALOGUE: Mapping[str, Mapping[str, PropertyType]] = {
# ── The daily snapshot ───────────────────────────────────────────────────
# Counted locally, where the ids are known; only totals are reported.
- # `duration_ms` is how long the pass took to collect. It is roughly fifteen
+ # `duration_ms` is how long the pass took to collect. It is roughly twenty
# queries per tenant, on a schedule, against the same database serving
# rooms — "what does this cost at scale" is an open question in
# `docs/old/telemetry-events.md`, and this is the only place it can be
@@ -280,6 +304,47 @@ CATALOGUE: Mapping[str, Mapping[str, PropertyType]] = {
**{name: NUMBER for name in _SNAPSHOT_COUNTS},
"duration_ms": NUMBER,
},
+ # One per finished UTC day, ISO week and calendar month, dated to its
+ # middle: distinct use over exactly that period, so the events of one
+ # period kind add up across deployments where the snapshot's trailing
+ # `*_7d` counts, summed over days, count each room up to seven times.
+ "usage_period": {
+ "period": one_of("day", "week", "month"),
+ "room_active": NUMBER,
+ "person_identity_active": NUMBER,
+ "account_active": NUMBER,
+ "tenant_failed_count": NUMBER,
+ },
+ # ── Accounts, workspaces and cloud machines ─────────────────────────────
+ # Switch Cloud's funnel, though any server that has them sends them:
+ # `deployment_kind`, on every event, says which kind of server it was.
+ "account_created": {"method": one_of("signup", "oidc", "admin")},
+ "signup_refused": {"reason": one_of("disabled", "rate_limited", "email_taken")},
+ "workspace_created": {"new_organization": BOOLEAN},
+ # One per claim that changed something, naming the first of these that
+ # applies: a retained machine joined from another workspace is
+ # `joined_workspace`, though the claim runs it again as well.
+ "cloud_machine_requested": {
+ "change": one_of("new", "joined_workspace", "unretained", "restarted"),
+ },
+ # `provisioning_seconds` is from the claim to the first controller report,
+ # for a new machine; -1 for a later start, which nothing times.
+ "cloud_machine_ready": {"new_machine": BOOLEAN, "provisioning_seconds": NUMBER},
+ "cloud_machine_failed": {
+ "reason": one_of(
+ "not_scheduled",
+ "never_started",
+ "never_connected",
+ "owner_left",
+ "needs_attention",
+ "reported",
+ ),
+ },
+ "cloud_machine_woken": {
+ "trigger": one_of("message", "agent_placed", "owner"),
+ "stopped_by": one_of("idle", "owner", "unknown"),
+ },
+ "cloud_machine_stopped": {"reason": one_of("idle", "owner")},
# ── Milestones ───────────────────────────────────────────────────────────
# At most once per deployment, and only for one installed after this
# shipped. Together they are the activation funnel.
@@ -301,7 +366,20 @@ CATALOGUE: Mapping[str, Mapping[str, PropertyType]] = {
**_SINCE_INSTALL,
"known_agent_type": KNOWN_AGENT_TYPE,
},
+ # An agent's first live connection, which is all this ever measured.
+ # Superseded by `first_agent_connected`, the same moment under an honest
+ # name; kept for one stable release.
"first_session_started": {**_SINCE_INSTALL, "known_agent_type": KNOWN_AGENT_TYPE},
+ "first_agent_connected": {**_SINCE_INSTALL, "known_agent_type": KNOWN_AGENT_TYPE},
+ # The first coding-agent session a host reported (`session_started`): an
+ # agent doing work, not just coming online. A deployment that had already
+ # done either before this existed claims it without reporting, so it never
+ # announces a "first" months in (`settle_superseded_milestones`).
+ "first_agent_session": {
+ **_SINCE_INSTALL,
+ "known_agent_type": KNOWN_AGENT_TYPE,
+ "start_source": one_of("user", "room", "automation", "unknown"),
+ },
# ── Lifecycle ────────────────────────────────────────────────────────────
# The version is a resource attribute, so this is the upgrade curve.
"deployment_started": {"tenant_count": NUMBER},
@@ -366,7 +444,10 @@ CATALOGUE: Mapping[str, Mapping[str, PropertyType]] = {
# An agent coming online: its first live connection. No "start source": the
# server sees an authenticated connection whether a person launched the
# session or Console spawned it. `session_started` is the one that says.
+ # Superseded by `agent_connected`, the same event under a name that says
+ # what it is; kept for one stable release.
"agent_session_started": {"known_agent_type": KNOWN_AGENT_TYPE},
+ "agent_connected": {"known_agent_type": KNOWN_AGENT_TYPE},
# A coding-agent session starting, as its host reports it once, when the
# session is new. `start_source` is what the launcher stamped on it: `user`
# a person starting one in Console, `room` the agent being addressed in a
@@ -379,12 +460,20 @@ CATALOGUE: Mapping[str, Mapping[str, PropertyType]] = {
},
# No runtime: the connection registry is the only thing that knows a
# session ended and it holds none. Starts carry it.
+ # Superseded by `agent_disconnected`, the same event renamed; kept for one
+ # stable release.
"agent_session_ended": {
"duration_seconds": NUMBER,
"reason": one_of(
"normal", "heartbeat_lapsed", "replaced", "room_claimed", "error"
),
},
+ "agent_disconnected": {
+ "duration_seconds": NUMBER,
+ "reason": one_of(
+ "normal", "heartbeat_lapsed", "replaced", "room_claimed", "error"
+ ),
+ },
"connector_added": {
"bridge_platform": BRIDGE_PLATFORM,
"seconds_since_install": NUMBER,
diff --git a/core/switch_core/telemetry/deployment.py b/core/switch_core/telemetry/deployment.py
index d4b7c5cf2..43d818d26 100644
--- a/core/switch_core/telemetry/deployment.py
+++ b/core/switch_core/telemetry/deployment.py
@@ -16,7 +16,7 @@ import logging
from datetime import UTC, datetime
from typing import Literal
-from sqlalchemy import delete, select
+from sqlalchemy import delete, or_, select
from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
@@ -24,6 +24,28 @@ from switch_core.db.models import DeploymentIdentity, TelemetryMilestone
logger = logging.getLogger(__name__)
+# The prefix of the claims `session_started` takes per session
+# (`telemetry/session_start.py`), here because the milestones below read them.
+SESSION_START_CLAIM_PREFIX = "session_started:"
+
+# Milestones added after deployments were already running, each with the
+# claims that show a deployment passed it before it existed: exact milestone
+# names, or a prefix ending in `*`. `settle_superseded_milestones` claims them
+# without reporting, so a deployment that did this long ago is never reported
+# as doing it for the first time now.
+# Claimed beside each superseded milestone once it has been settled, so it is
+# settled only once (`settle_superseded_milestones`).
+SETTLED_PREFIX = "settled:"
+
+SUPERSEDED_MILESTONES: dict[str, tuple[str, ...]] = {
+ # The same moment as `first_session_started`, under the name that says it
+ # is a connection.
+ "first_agent_connected": ("first_session_started",),
+ # A person using a room with an agent in it, or a session reported in the
+ # last week (those claims are kept that long), means sessions have run.
+ "first_agent_session": ("first_room_active", f"{SESSION_START_CLAIM_PREFIX}*"),
+}
+
class DeploymentIdentityMissingError(RuntimeError):
"""The singleton identity row is absent.
@@ -149,3 +171,78 @@ async def claim_milestone(
"Could not record the %s telemetry milestone; not reporting it.", name
)
return False
+
+
+async def settle_superseded_milestones(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> list[str]:
+ """Claim, without reporting, each `SUPERSEDED_MILESTONES` entry this
+ deployment had already passed when they were added. Returns the names it
+ claimed.
+
+ Once per deployment, at the first boot that has them, before anything can
+ report one: on a new deployment there is no evidence yet, so each is
+ reported normally the first time it happens. Only once, because evidence
+ written later is the milestone happening, not history: with several
+ processes, one booting while another takes the deployment's first session
+ would otherwise read that session's claim as history and claim the
+ milestone out from under its report. A failure claims nothing and is
+ logged, leaving the milestone to be reported when it next happens — late
+ rather than lost.
+ """
+ settled: list[str] = []
+ try:
+ async with session_factory() as session:
+ for milestone, evidence in SUPERSEDED_MILESTONES.items():
+ marker = f"{SETTLED_PREFIX}{milestone}"
+ if (
+ await session.scalar(
+ pg_insert(TelemetryMilestone)
+ .values(name=marker)
+ .on_conflict_do_nothing(index_elements=["name"])
+ .returning(TelemetryMilestone.name)
+ )
+ is None
+ ):
+ continue
+ exact = [name for name in evidence if not name.endswith("*")]
+ prefixes = [name[:-1] for name in evidence if name.endswith("*")]
+ found = await session.scalar(
+ select(TelemetryMilestone.name)
+ .where(
+ or_(
+ TelemetryMilestone.name.in_(exact),
+ *(
+ TelemetryMilestone.name.startswith(
+ prefix, autoescape=True
+ )
+ for prefix in prefixes
+ ),
+ )
+ )
+ .limit(1)
+ )
+ if found is None:
+ continue
+ claimed = await session.scalar(
+ pg_insert(TelemetryMilestone)
+ .values(name=milestone)
+ .on_conflict_do_nothing(index_elements=["name"])
+ .returning(TelemetryMilestone.name)
+ )
+ if claimed is not None:
+ settled.append(milestone)
+ await session.commit()
+ except Exception:
+ logger.exception(
+ "Could not settle telemetry milestones superseded by newer ones; "
+ "each will be reported when it next happens."
+ )
+ return []
+ if settled:
+ logger.info(
+ "Telemetry milestones already passed before they existed, claimed "
+ "without reporting: %s",
+ ", ".join(settled),
+ )
+ return settled
diff --git a/core/switch_core/telemetry/messages.py b/core/switch_core/telemetry/messages.py
index fe54a6d63..d64920dc6 100644
--- a/core/switch_core/telemetry/messages.py
+++ b/core/switch_core/telemetry/messages.py
@@ -119,6 +119,7 @@ class _AgentAddressed:
room_id: str
sender_transport_user_id: str
from_platform: bool
+ from_app: bool
known_agent_type: str
agent_live: bool
has_attachment: bool
@@ -220,6 +221,7 @@ class MessageTelemetry:
room_id: str,
sender_transport_user_id: str,
from_platform: bool,
+ from_app: bool,
agent_metadata: dict | None,
agent_live: bool,
has_attachment: bool,
@@ -237,6 +239,7 @@ class MessageTelemetry:
room_id=room_id,
sender_transport_user_id=sender_transport_user_id,
from_platform=from_platform,
+ from_app=from_app,
known_agent_type=normalise_known_agent_type(agent_metadata),
agent_live=agent_live,
has_attachment=has_attachment,
@@ -314,7 +317,11 @@ class MessageTelemetry:
room = await self._room_facts(message.tenant_id, message.room_id)
self._telemetry.emit(
"room_message_sent",
- sender_kind=_ROLE_KIND.get(message.sender_role, "unknown"),
+ sender_kind=(
+ "bot"
+ if message.sender_is_app
+ else _ROLE_KIND.get(message.sender_role, "unknown")
+ ),
bridge_platform=room.bridge_platform,
channel_type=room.channel_type,
room_user_count=room.user_count,
@@ -338,11 +345,14 @@ class MessageTelemetry:
async def _report_addressed(self, item: _AgentAddressed) -> None:
room = await self._room_facts(item.tenant_id, item.room_id)
- sender_kind = (
- "platform"
- if item.from_platform
- else await self._sender_kind(item.tenant_id, item.sender_transport_user_id)
- )
+ if item.from_platform:
+ sender_kind = "platform"
+ elif item.from_app:
+ sender_kind = "bot"
+ else:
+ sender_kind = await self._sender_kind(
+ item.tenant_id, item.sender_transport_user_id
+ )
self._telemetry.emit(
"agent_message_received",
sender_kind=sender_kind,
diff --git a/core/switch_core/telemetry/periods.py b/core/switch_core/telemetry/periods.py
new file mode 100644
index 000000000..86d241234
--- /dev/null
+++ b/core/switch_core/telemetry/periods.py
@@ -0,0 +1,90 @@
+"""The calendar periods usage is counted over, for `usage_period`.
+
+The snapshot's `*_7d` counts are trailing windows sent every day, so a chart
+that adds them up over a week counts each room up to seven times. A finished
+period is counted once: one event per deployment per UTC day, ISO week and
+calendar month, each carrying distinct counts for exactly that period, so the
+sum across deployments at the matching interval is the real figure.
+
+Each is dated to the middle of its period rather than to when it was sent. A
+laptop that was off sends the periods it missed when it next runs, days late,
+and an event dated "now" would land in the wrong bucket; the middle is inside
+the same day, week and month in any timezone a chart is drawn in, where the
+start would fall on the previous day west of UTC.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from datetime import datetime, timedelta
+
+from switch_core.telemetry.snapshot import as_utc
+
+# How far back a pass reports periods that ended while nothing was sending.
+# Bounds the queries a server back from a long time off runs at once, and how
+# old what it reports can be. Long enough to cover the month before for a
+# laptop that was off for most of the month after.
+BACKFILL = timedelta(days=35)
+
+_DAY = timedelta(days=1)
+_WEEK = timedelta(days=7)
+
+
+@dataclass(frozen=True)
+class Period:
+ kind: str
+ start: datetime
+ end: datetime
+
+ @property
+ def midpoint(self) -> datetime:
+ return self.start + (self.end - self.start) / 2
+
+
+def _day_start(moment: datetime) -> datetime:
+ return moment.replace(hour=0, minute=0, second=0, microsecond=0)
+
+
+def _month_after(start: datetime) -> datetime:
+ return start.replace(
+ year=start.year + start.month // 12, month=start.month % 12 + 1
+ )
+
+
+def _periods_of(kind: str, first_start: datetime, before: datetime) -> list[Period]:
+ """Every period of `kind` from the one starting at `first_start` to the
+ last that ends by `before`."""
+ periods: list[Period] = []
+ start = first_start
+ while True:
+ end = (
+ _month_after(start)
+ if kind == "month"
+ else start + (_DAY if kind == "day" else _WEEK)
+ )
+ if end > before:
+ return periods
+ periods.append(Period(kind=kind, start=start, end=end))
+ start = end
+
+
+def periods_ending_between(since: datetime, now: datetime) -> list[Period]:
+ """Every day, week and month that ended after `since` and by `now`, and
+ within `BACKFILL` of `now`, oldest first within each kind.
+
+ Successive passes hand in successive windows (the snapshot watermark), so
+ each period's end falls in exactly one of them and it is reported once.
+ """
+ now = as_utc(now)
+ floor = max(as_utc(since), now - BACKFILL)
+ # Start early enough that the first period ending after `floor` is in the
+ # walk; those ending at or before it are dropped below.
+ first_day = _day_start(floor)
+ first_week = first_day - timedelta(days=first_day.weekday())
+ first_month = first_day.replace(day=1)
+ walked = (
+ _periods_of("day", first_day, now)
+ + _periods_of("week", first_week, now)
+ + _periods_of("month", first_month, now)
+ )
+ return [period for period in walked if period.end > floor]
diff --git a/core/switch_core/telemetry/reporter.py b/core/switch_core/telemetry/reporter.py
index 8db62e051..3227b8f38 100644
--- a/core/switch_core/telemetry/reporter.py
+++ b/core/switch_core/telemetry/reporter.py
@@ -10,7 +10,8 @@ same minute would otherwise both read "due" and both send the day's snapshot.
The first pass sends the snapshot but not the room-activation events: with no
watermark, "first interaction since then" means every room that ever went
-active, arriving at once and all dated today.
+active, arriving at once and all dated today. Nor the finished periods
+(`periods.py`): with no watermark there is no telling which were sent before.
"""
from __future__ import annotations
@@ -26,10 +27,12 @@ from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from switch_core.db.models import TelemetrySnapshotWatermark
from switch_core.telemetry.deployment import claim_milestone
+from switch_core.telemetry.periods import periods_ending_between
from switch_core.telemetry.service import TelemetryService
from switch_core.telemetry.session_start import prune_session_start_claims
from switch_core.telemetry.snapshot import (
as_utc,
+ collect_period_counts,
collect_usage,
newly_active_rooms,
summarise,
@@ -166,6 +169,35 @@ class SnapshotReporter:
"after it."
)
+ if since is not None:
+ await self._report_periods(since=since, now=now)
+
+ async def _report_periods(self, *, since: datetime, now: datetime) -> None:
+ """One `usage_period` for every day, week and month that ended in this
+ pass's window. Contained like the rooms above, for the same reason: a
+ raise would hand the watermark back and send the snapshot again. A
+ period that fails is not reported; the next window starts after it."""
+ for period in periods_ending_between(since, now):
+ try:
+ counts = await collect_period_counts(
+ self._session_factory, start=period.start, end=period.end
+ )
+ self._telemetry.emit_at(
+ "usage_period",
+ period.midpoint,
+ period=period.kind,
+ room_active=counts.room_active,
+ person_identity_active=counts.person_identity_active,
+ account_active=counts.account_active,
+ tenant_failed_count=counts.tenant_failed_count,
+ )
+ except Exception:
+ logger.exception(
+ "Usage period: the %s from %s could not be reported.",
+ period.kind,
+ period.start.date().isoformat(),
+ )
+
async def _report_first_room_active(self, active: list) -> None:
"""The activation milestone: the first room a person actually used.
diff --git a/core/switch_core/telemetry/service.py b/core/switch_core/telemetry/service.py
index 24d59e1c2..451152e9d 100644
--- a/core/switch_core/telemetry/service.py
+++ b/core/switch_core/telemetry/service.py
@@ -15,16 +15,30 @@ import asyncio
import logging
import time
from collections.abc import Mapping
-from datetime import datetime
+from datetime import UTC, datetime, timedelta
+from sqlalchemy import event as sa_event
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
-
-from switch_core.telemetry.catalogue import PropertyValue, validate, wire_name
-from switch_core.telemetry.deployment import claim_milestone, seconds_since_install
+from sqlalchemy.orm import Session, SessionTransaction
+
+from switch_core.config import DeploymentKind
+from switch_core.telemetry.catalogue import (
+ DEPLOYMENT_KIND_PROPERTY,
+ PropertyValue,
+ validate,
+ wire_name,
+)
+from switch_core.telemetry.deployment import (
+ claim_milestone,
+ seconds_since_install,
+ settle_superseded_milestones,
+)
from switch_core.telemetry.sink import TelemetryRecord, TelemetrySink
logger = logging.getLogger(__name__)
+_EPOCH = datetime(1970, 1, 1, tzinfo=UTC)
+
class TelemetryService:
"""Validates, tags and dispatches events.
@@ -43,11 +57,13 @@ class TelemetryService:
version: str | None,
telemetry_environment: str,
telemetry_internal: bool,
+ deployment_kind: DeploymentKind,
session_factory: async_sessionmaker[AsyncSession] | None = None,
installed_at: datetime | None = None,
) -> None:
self._sink = sink
self._enabled = enabled
+ self._deployment_kind = deployment_kind
# Only the milestone path needs these. Optional so the ordinary
# `emit` is usable from a test that has no database at all.
self._session_factory = session_factory
@@ -78,6 +94,17 @@ class TelemetryService:
def emit(self, event: str, **properties: PropertyValue) -> None:
"""Report one event. Never blocks; raises only on a malformed event."""
+ self._emit(event, time.time_ns(), properties)
+
+ def emit_at(self, event: str, at: datetime, **properties: PropertyValue) -> None:
+ """`emit`, dated `at` rather than now: for an event about a time that
+ has passed, which a chart must place where it happened."""
+ microseconds = (at - _EPOCH) // timedelta(microseconds=1)
+ self._emit(event, microseconds * 1_000, properties)
+
+ def _emit(
+ self, event: str, timestamp_ns: int, properties: dict[str, PropertyValue]
+ ) -> None:
# Validated even when disabled, so a mistake in a rarely-taken branch
# is caught by whichever test exercises it rather than lying dormant
# until the day a deployment switches telemetry on.
@@ -87,12 +114,24 @@ class TelemetryService:
self._dispatch(
TelemetryRecord(
name=wire_name(event),
- properties=dict(properties),
+ properties={
+ **properties,
+ DEPLOYMENT_KIND_PROPERTY: self._deployment_kind,
+ },
resource=dict(self._resource),
- timestamp_ns=time.time_ns(),
+ timestamp_ns=timestamp_ns,
)
)
+ async def settle_superseded_milestones(self) -> None:
+ """Claim without reporting the milestones this deployment passed
+ before they existed (`deployment.SUPERSEDED_MILESTONES`). Once, at boot,
+ before anything can report one. Nothing while telemetry is off, for the
+ reason `emit_milestone` gives."""
+ if not self._enabled or self._session_factory is None:
+ return
+ await settle_superseded_milestones(self._session_factory)
+
async def emit_milestone(self, event: str, **properties: PropertyValue) -> None:
"""Report a once-ever activation milestone, if it has not been reported.
@@ -173,3 +212,41 @@ def emit_safely(
"bug in the event, not a relay problem.",
event,
)
+
+
+def emit_after_commit(
+ session: AsyncSession,
+ telemetry: TelemetryService | None,
+ event: str,
+ properties: Mapping[str, PropertyValue],
+) -> None:
+ """`emit_safely` once `session` commits, and never if it rolls back.
+
+ For a state change made where the transaction is committed by a caller
+ further up, or not at all when a later step refuses: reporting it at the
+ point it is decided would count a change that never happened.
+ """
+ if telemetry is None:
+ return
+ sync = session.sync_session
+ committed = False
+ decided = False
+
+ def on_commit(_session: Session) -> None:
+ nonlocal committed
+ committed = True
+
+ def on_end(_session: Session, transaction: SessionTransaction) -> None:
+ nonlocal decided
+ # A savepoint ending, committed or rolled back, decides nothing: only
+ # the outermost transaction does. The listeners stay registered, inert
+ # once decided: removing one while the session dispatches its events
+ # is refused, and they go with the session.
+ if decided or transaction.parent is not None:
+ return
+ decided = True
+ if committed:
+ emit_safely(telemetry, event, properties)
+
+ sa_event.listen(sync, "after_commit", on_commit)
+ sa_event.listen(sync, "after_transaction_end", on_end)
diff --git a/core/switch_core/telemetry/session_start.py b/core/switch_core/telemetry/session_start.py
index b1df17942..8314d0261 100644
--- a/core/switch_core/telemetry/session_start.py
+++ b/core/switch_core/telemetry/session_start.py
@@ -19,7 +19,11 @@ from typing import Literal
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from switch_core.db.models import Agent
-from switch_core.telemetry.deployment import claim_milestone, prune_claims
+from switch_core.telemetry.deployment import (
+ SESSION_START_CLAIM_PREFIX,
+ claim_milestone,
+ prune_claims,
+)
from switch_core.telemetry.service import TelemetryService, emit_safely
from switch_core.telemetry.snapshot import normalise_known_agent_type
@@ -30,7 +34,7 @@ StartSource = Literal["user", "room", "automation", "unknown"]
# The claims share `telemetry_milestones` with the once-ever milestones and the
# per-connector claims, which are never pruned; the prefix is what tells them
# apart.
-CLAIM_PREFIX = "session_started:"
+CLAIM_PREFIX = SESSION_START_CLAIM_PREFIX
# How long a claim is kept. It exists only so a retried report counts once, and
# a host retries within seconds while Switch answers, or the next time a
@@ -123,13 +127,16 @@ async def report_session_started(
session_factory, f"{CLAIM_PREFIX}{agent.id}:{session_id}"
):
return False
+ known_agent_type = normalise_known_agent_type(agent.metadata_)
emit_safely(
telemetry,
"session_started",
- {
- "start_source": start_source,
- "known_agent_type": normalise_known_agent_type(agent.metadata_),
- },
+ {"start_source": start_source, "known_agent_type": known_agent_type},
+ )
+ await telemetry.emit_milestone(
+ "first_agent_session",
+ known_agent_type=known_agent_type,
+ start_source=start_source,
)
return True
diff --git a/core/switch_core/telemetry/setup.py b/core/switch_core/telemetry/setup.py
index 18f855b2c..88e4b54ec 100644
--- a/core/switch_core/telemetry/setup.py
+++ b/core/switch_core/telemetry/setup.py
@@ -145,6 +145,7 @@ def _service(
version=version,
telemetry_environment=config.telemetry_environment,
telemetry_internal=config.telemetry_internal,
+ deployment_kind=config.deployment_kind,
session_factory=session_factory,
installed_at=installed_at,
)
diff --git a/core/switch_core/telemetry/snapshot.py b/core/switch_core/telemetry/snapshot.py
index d0b900555..44bf9b683 100644
--- a/core/switch_core/telemetry/snapshot.py
+++ b/core/switch_core/telemetry/snapshot.py
@@ -38,7 +38,7 @@ from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from sqlalchemy.orm import aliased
from switch_core import messaging_platforms
-from switch_core.clients.admin_messages import AUTO_REPLY_FLAG
+from switch_core.clients.admin_messages import AUTO_REPLY_FLAG, SENDER_IS_APP_MARKER
from switch_core.db.models import (
Agent,
ApiKey,
@@ -100,7 +100,7 @@ class UsageCounts:
# wire: without it, a partial pass is every count dropping at once with
# nothing to attribute it to.
tenant_failed_count: int = 0
- # Wall time to collect the whole pass. Roughly fifteen queries per tenant
+ # Wall time to collect the whole pass. Roughly twenty queries per tenant
# against the database that is also serving rooms, so what it costs at
# scale is a real question — and a background task is invisible to the
# HTTP request histogram that times everything else.
@@ -130,6 +130,20 @@ class UsageCounts:
chat_identity_count: int = 0
chat_identity_active_1d: int = 0
chat_identity_active_7d: int = 0
+ # The same chat identities split by who is behind them. An identity whose
+ # posts the platform marked as an app's (a Slack app, a Discord bot or
+ # webhook, a Mattermost bot account: alerts, CI, another Switch server's
+ # agents) is a bot. Every other identity is a person, linked to a Switch
+ # account or not. `chat_identity_active_*` counted both, which is why these
+ # are new names rather than a new meaning for the old ones.
+ person_identity_active_1d: int = 0
+ person_identity_active_7d: int = 0
+ bot_identity_active_1d: int = 0
+ bot_identity_active_7d: int = 0
+ # `user_active_*` under the name it should have had: Switch accounts, from
+ # the same ids. Bots never link an account, so it needs no bot rule.
+ account_active_1d: int = 0
+ account_active_7d: int = 0
# The other tiers, as chat identities like `chat_identity_active_*`:
# anyone who said anything in any room, and anyone Switch has seen in a
# room — see `_activity_tiers` for what that can and cannot see.
@@ -188,6 +202,12 @@ class UsageCounts:
"chat_identity_count": self.chat_identity_count,
"chat_identity_active_1d": self.chat_identity_active_1d,
"chat_identity_active_7d": self.chat_identity_active_7d,
+ "person_identity_active_1d": self.person_identity_active_1d,
+ "person_identity_active_7d": self.person_identity_active_7d,
+ "bot_identity_active_1d": self.bot_identity_active_1d,
+ "bot_identity_active_7d": self.bot_identity_active_7d,
+ "account_active_1d": self.account_active_1d,
+ "account_active_7d": self.account_active_7d,
"chat_identity_posted_1d": self.chat_identity_posted_1d,
"chat_identity_posted_7d": self.chat_identity_posted_7d,
"chat_identity_in_room_count": self.chat_identity_in_room_count,
@@ -342,21 +362,27 @@ def _human_message_conditions(tenant_id: str) -> tuple[Any, ...]:
)
-def _human_interaction(tenant_id: str, since: datetime) -> Select[tuple[str]]:
- """Room ids a human used since `since`. See `_human_activity_conditions`."""
+def _human_interaction(
+ tenant_id: str, since: datetime, until: datetime
+) -> Select[tuple[str]]:
+ """Room ids a human used in `[since, until)`. See
+ `_human_activity_conditions`."""
return (
select(distinct(Message.room_id))
.join(Client, Client.id == Message.sender_client_id)
.where(
Message.tenant_id == tenant_id,
Message.sent_at >= since,
+ Message.sent_at < until,
*_human_activity_conditions(tenant_id),
)
)
-def _active_accounts(tenant_id: str, since: datetime) -> Select[tuple[str]]:
- """Distinct Switch accounts that used a room since `since`: an account
+def _active_accounts(
+ tenant_id: str, since: datetime, until: datetime
+) -> Select[tuple[str]]:
+ """Distinct Switch accounts that used a room in `[since, until)`: an account
is active when a chat account it has claimed spoke. A chat account several
people claim makes each of them active. See `_human_activity_conditions`."""
return (
@@ -379,13 +405,16 @@ def _active_accounts(tenant_id: str, since: datetime) -> Select[tuple[str]]:
.where(
Message.tenant_id == tenant_id,
Message.sent_at >= since,
+ Message.sent_at < until,
*_human_activity_conditions(tenant_id),
)
)
-def _active_humans(tenant_id: str, since: datetime) -> Select[tuple[str | None]]:
- """Distinct human clients who used a room since `since`. See
+def _active_humans(
+ tenant_id: str, since: datetime, until: datetime
+) -> Select[tuple[str | None]]:
+ """Distinct human clients who used a room in `[since, until)`. See
`_human_activity_conditions`."""
return (
select(distinct(Message.sender_client_id))
@@ -393,11 +422,58 @@ def _active_humans(tenant_id: str, since: datetime) -> Select[tuple[str | None]]
.where(
Message.tenant_id == tenant_id,
Message.sent_at >= since,
+ Message.sent_at < until,
*_human_activity_conditions(tenant_id),
)
)
+def _active_bots(
+ tenant_id: str, since: datetime, until: datetime
+) -> Select[tuple[str | None]]:
+ """The identities among `_active_humans` that posted as an app in
+ `[since, until)`: a post the bridge marked `SENDER_IS_APP_MARKER` because
+ its platform said an integration, not a person, sent it."""
+ return (
+ select(distinct(Message.sender_client_id))
+ .join(Client, Client.id == Message.sender_client_id)
+ .where(
+ Message.tenant_id == tenant_id,
+ Message.sent_at >= since,
+ Message.sent_at < until,
+ *_human_activity_conditions(tenant_id),
+ Message.content.has_key(SENDER_IS_APP_MARKER),
+ )
+ )
+
+
+def _active_people(
+ tenant_id: str, since: datetime, until: datetime
+) -> Select[tuple[str | None]]:
+ """`_active_humans` less every identity that posted as an app in the same
+ window, not just its marked posts.
+
+ By identity rather than by post, because a bot's posts from before its
+ bridge marked them carry no marker: once it has posted once with one, the
+ rest of its window is a bot's too, rather than a person who is counted
+ until the old posts age out.
+ """
+ return (
+ select(distinct(Message.sender_client_id))
+ .join(Client, Client.id == Message.sender_client_id)
+ .where(
+ Message.tenant_id == tenant_id,
+ Message.sent_at >= since,
+ Message.sent_at < until,
+ *_human_activity_conditions(tenant_id),
+ ~Message.content.has_key(SENDER_IS_APP_MARKER),
+ Message.sender_client_id.not_in(
+ _active_bots(tenant_id, since, until).scalar_subquery()
+ ),
+ )
+ )
+
+
def _activity_tiers(
tenant_id: str, day_ago: datetime, week_ago: datetime
) -> Select[tuple[int, int, int]]:
@@ -485,10 +561,10 @@ async def collect_tenant_counts(
week_ago = now - _WEEK
counts.active_user_ids_1d |= set(
- (await session.execute(_active_accounts(tenant_id, day_ago))).scalars()
+ (await session.execute(_active_accounts(tenant_id, day_ago, now))).scalars()
)
counts.active_user_ids_7d |= set(
- (await session.execute(_active_accounts(tenant_id, week_ago))).scalars()
+ (await session.execute(_active_accounts(tenant_id, week_ago, now))).scalars()
)
counts.chat_identity_count += await _scalar(
session,
@@ -497,10 +573,22 @@ async def collect_tenant_counts(
.where(Client.tenant_id == tenant_id, Client.type == HUMAN_CLIENT_TYPE),
)
counts.chat_identity_active_1d += await _count(
- session, _active_humans(tenant_id, day_ago)
+ session, _active_humans(tenant_id, day_ago, now)
)
counts.chat_identity_active_7d += await _count(
- session, _active_humans(tenant_id, week_ago)
+ session, _active_humans(tenant_id, week_ago, now)
+ )
+ counts.person_identity_active_1d += await _count(
+ session, _active_people(tenant_id, day_ago, now)
+ )
+ counts.person_identity_active_7d += await _count(
+ session, _active_people(tenant_id, week_ago, now)
+ )
+ counts.bot_identity_active_1d += await _count(
+ session, _active_bots(tenant_id, day_ago, now)
+ )
+ counts.bot_identity_active_7d += await _count(
+ session, _active_bots(tenant_id, week_ago, now)
)
posted_1d, posted_7d, in_room = (
await session.execute(_activity_tiers(tenant_id, day_ago, week_ago))
@@ -509,10 +597,10 @@ async def collect_tenant_counts(
counts.chat_identity_posted_7d += posted_7d
counts.chat_identity_in_room_count += in_room
counts.room_active_1d += await _count(
- session, _human_interaction(tenant_id, day_ago)
+ session, _human_interaction(tenant_id, day_ago, now)
)
counts.room_active_7d += await _count(
- session, _human_interaction(tenant_id, week_ago)
+ session, _human_interaction(tenant_id, week_ago, now)
)
# A room predating the stamp reads as user-created, the conservative
@@ -839,6 +927,8 @@ def _merge_tenant_counts(total: UsageCounts, tenant: UsageCounts) -> None:
"user_internal_count",
"user_active_1d",
"user_active_7d",
+ "account_active_1d",
+ "account_active_7d",
"user_internal_active_1d",
"user_internal_active_7d",
"active_user_ids_1d",
@@ -935,10 +1025,75 @@ async def collect_usage(
)
counts.user_active_1d = len(counts.active_user_ids_1d)
counts.user_active_7d = len(counts.active_user_ids_7d)
+ counts.account_active_1d = counts.user_active_1d
+ counts.account_active_7d = counts.user_active_7d
counts.duration_ms = round((time.monotonic() - started) * 1000)
return counts
+@dataclass
+class PeriodCounts:
+ """Distinct use over one finished period (`periods.Period`), by the same
+ definitions as the snapshot's `room_active_*`, `person_identity_active_*`
+ and `account_active_*`, so a period and a trailing window agree on what
+ counts."""
+
+ room_active: int = 0
+ person_identity_active: int = 0
+ account_ids: set[str] = field(default_factory=set)
+ tenant_failed_count: int = 0
+
+ @property
+ def account_active(self) -> int:
+ return len(self.account_ids)
+
+
+async def collect_period_counts(
+ session_factory: async_sessionmaker[AsyncSession],
+ *,
+ start: datetime,
+ end: datetime,
+) -> PeriodCounts:
+ """Every tenant's distinct use in `[start, end)`, summed like
+ `collect_usage`: one tenant's failure leaves nothing of it behind and is
+ disclosed in `tenant_failed_count`. Accounts are a set across tenants, so
+ one account active in two counts once."""
+ counts = PeriodCounts()
+ tenant_ids = await all_tenant_ids(session_factory)
+ failed_tenants: list[str] = []
+ for tenant_id in tenant_ids:
+ try:
+ async with tenant_session(session_factory, tenant_id) as session:
+ rooms = await _count(session, _human_interaction(tenant_id, start, end))
+ people = await _count(session, _active_people(tenant_id, start, end))
+ accounts = set(
+ (
+ await session.execute(_active_accounts(tenant_id, start, end))
+ ).scalars()
+ )
+ except Exception:
+ logger.exception(
+ "Usage period: failed to collect counts for tenant %s", tenant_id
+ )
+ failed_tenants.append(tenant_id)
+ continue
+ counts.room_active += rooms
+ counts.person_identity_active += people
+ counts.account_ids |= accounts
+ if failed_tenants:
+ logger.error(
+ "Usage period %s to %s: excluded %d of %d tenant(s): %s. Its counts "
+ "read low, and tenant_failed_count on the event says so.",
+ start.isoformat(),
+ end.isoformat(),
+ len(failed_tenants),
+ len(tenant_ids),
+ ", ".join(failed_tenants),
+ )
+ counts.tenant_failed_count = len(failed_tenants)
+ return counts
+
+
async def newly_active_rooms(
session_factory: async_sessionmaker[AsyncSession],
*,
diff --git a/core/switch_core/transport/content.py b/core/switch_core/transport/content.py
index 615096eba..cacbc261f 100644
--- a/core/switch_core/transport/content.py
+++ b/core/switch_core/transport/content.py
@@ -69,6 +69,7 @@ def media_content(
caption: str | None = None,
thread_root_id: str | None = None,
group: dict[str, object] | None = None,
+ extra_content: dict[str, object] | None = None,
) -> dict[str, object]:
# When a caption is given it becomes the event body, with the real filename
# carried separately per the rich-media-caption convention.
@@ -79,6 +80,10 @@ def media_content(
"info": {"mimetype": mimetype, "size": size},
"sender_name": sender_name,
}
+ # The same caller-supplied markers a text message can carry, so a file
+ # says who posted it as faithfully as the text beside it does.
+ if extra_content:
+ content.update(extra_content)
if caption:
content["filename"] = filename
if group is not None:
diff --git a/core/switch_core/transport/observer.py b/core/switch_core/transport/observer.py
index f7f2aefe7..f85aa340d 100644
--- a/core/switch_core/transport/observer.py
+++ b/core/switch_core/transport/observer.py
@@ -25,6 +25,9 @@ class ParticipantMessage:
sender_client_id: str
#: The writer's `ActorRole`: `human`, `agent`, `system` or `bridge`.
sender_role: str
+ #: The bridge marked the post as an integration's, not a person's
+ #: (`SENDER_IS_APP_MARKER`). Only ever true for a `human` writer.
+ sender_is_app: bool
has_attachment: bool
in_thread: bool
diff --git a/core/switch_core/transport/port.py b/core/switch_core/transport/port.py
index 25daf2e15..8eaaba17f 100644
--- a/core/switch_core/transport/port.py
+++ b/core/switch_core/transport/port.py
@@ -144,12 +144,14 @@ class MessageTransport(Protocol):
caption: str | None = None,
thread_root_id: str | None = None,
group: dict[str, object] | None = None,
+ extra_content: dict[str, object] | None = None,
) -> SendResult:
"""Post an event pointing at an already-uploaded `uri`.
`group` marks this as one part of a multi-attachment message —
`{"id": ..., "index": i, "total": n}` — for transports that cannot
- carry several files on one event. `metered` is as for `send_message`.
+ carry several files on one event. `metered` and `extra_content` are as
+ for `send_message`.
"""
...
diff --git a/core/switch_core/transport/postgres.py b/core/switch_core/transport/postgres.py
index 1f8b8ba0e..bd53b4763 100644
--- a/core/switch_core/transport/postgres.py
+++ b/core/switch_core/transport/postgres.py
@@ -49,6 +49,7 @@ from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any, cast
from switch_core.attachments import parse_attachment_group
+from switch_core.clients.admin_messages import SENDER_IS_APP_MARKER
from switch_core.db.models import (
ClientRoom,
MediaBlob,
@@ -1177,6 +1178,7 @@ class PostgresTransport:
caption: str | None = None,
thread_root_id: str | None = None,
group: dict[str, object] | None = None,
+ extra_content: dict[str, object] | None = None,
) -> SendResult:
content = media_content(
uri,
@@ -1188,6 +1190,7 @@ class PostgresTransport:
caption=caption,
thread_root_id=thread_root_id,
group=group,
+ extra_content=extra_content,
)
return await self._send(
room_id, "m.room.message", content, sender_name, metered=metered
@@ -1316,6 +1319,7 @@ class PostgresTransport:
room_id=room_id,
sender_client_id=self.client_id,
sender_role=self._actor_role,
+ sender_is_app=content.get(SENDER_IS_APP_MARKER) is True,
has_attachment=kind == "media",
in_thread=thread_root_of(content) is not None,
)
diff --git a/core/tests/benchmarks/server.py b/core/tests/benchmarks/server.py
index 71a2d75f2..062abdef7 100644
--- a/core/tests/benchmarks/server.py
+++ b/core/tests/benchmarks/server.py
@@ -511,6 +511,7 @@ async def _serve(
version=None,
telemetry_environment="dev",
telemetry_internal=True,
+ deployment_kind="self_hosted",
),
session_factory=session_factory,
)
diff --git a/core/tests/integration/conftest.py b/core/tests/integration/conftest.py
index 04756fab7..a53ea12ed 100644
--- a/core/tests/integration/conftest.py
+++ b/core/tests/integration/conftest.py
@@ -623,6 +623,7 @@ async def harness(
version=None,
telemetry_environment="dev",
telemetry_internal=True,
+ deployment_kind="self_hosted",
),
session_factory=session_factory,
)
diff --git a/core/tests/switch_core/bridges/agent/api/test_events_endpoint_dispatch.py b/core/tests/switch_core/bridges/agent/api/test_events_endpoint_dispatch.py
index 8ca5b9055..eb6efc0d1 100644
--- a/core/tests/switch_core/bridges/agent/api/test_events_endpoint_dispatch.py
+++ b/core/tests/switch_core/bridges/agent/api/test_events_endpoint_dispatch.py
@@ -684,6 +684,7 @@ class TestOpeningAStreamReportsASession:
version="1.0.0",
telemetry_environment="prod",
telemetry_internal=False,
+ deployment_kind="self_hosted",
)
protocol = _Protocol(db_buffer)
protocol.sessions = SessionReporter(service, protocol.connections)
@@ -698,7 +699,10 @@ class TestOpeningAStreamReportsASession:
await _call(protocol, accept="text/event-stream", connection_id="c1")
await service.aclose()
- assert [r.name for r in sink.sent] == ["switch_core.agent_session_started"]
+ assert [r.name for r in sink.sent] == [
+ "switch_core.agent_session_started",
+ "switch_core.agent_connected",
+ ]
async def test_a_reattach_to_the_same_connection_is_not_another_start(
self, db_buffer: EventBuffer
@@ -709,7 +713,10 @@ class TestOpeningAStreamReportsASession:
await _call(protocol, accept="text/event-stream", connection_id="c1")
await service.aclose()
- assert [r.name for r in sink.sent] == ["switch_core.agent_session_started"]
+ assert [r.name for r in sink.sent] == [
+ "switch_core.agent_session_started",
+ "switch_core.agent_connected",
+ ]
async def test_a_refused_open_then_a_retry_is_one_start(
self, db_buffer: EventBuffer
@@ -734,7 +741,10 @@ class TestOpeningAStreamReportsASession:
)
await service.aclose()
- assert [r.name for r in sink.sent] == ["switch_core.agent_session_started"]
+ assert [r.name for r in sink.sent] == [
+ "switch_core.agent_session_started",
+ "switch_core.agent_connected",
+ ]
async def test_an_open_that_failed_after_registering_is_counted_on_retry(
self, db_buffer: EventBuffer
@@ -761,7 +771,10 @@ class TestOpeningAStreamReportsASession:
)
await service.aclose()
- assert [r.name for r in sink.sent] == ["switch_core.agent_session_started"]
+ assert [r.name for r in sink.sent] == [
+ "switch_core.agent_session_started",
+ "switch_core.agent_connected",
+ ]
async def test_closing_the_only_connection_ends_the_session(
self, monkeypatch: pytest.MonkeyPatch, db_buffer: EventBuffer
@@ -778,5 +791,7 @@ class TestOpeningAStreamReportsASession:
assert [r.name for r in sink.sent] == [
"switch_core.agent_session_started",
+ "switch_core.agent_connected",
"switch_core.agent_session_ended",
+ "switch_core.agent_disconnected",
]
diff --git a/core/tests/switch_core/bridges/collaboration/test_bridge_inbound_media.py b/core/tests/switch_core/bridges/collaboration/test_bridge_inbound_media.py
index fea444154..62fc0174a 100644
--- a/core/tests/switch_core/bridges/collaboration/test_bridge_inbound_media.py
+++ b/core/tests/switch_core/bridges/collaboration/test_bridge_inbound_media.py
@@ -39,6 +39,7 @@ class _FakeHumanActor:
caption=None,
thread_root_id=None,
group=None,
+ extra_content=None,
): # noqa: ANN001, ANN201
self.media.append(
{
@@ -51,6 +52,7 @@ class _FakeHumanActor:
"caption": caption,
"thread_root_id": thread_root_id,
"group": group,
+ "extra_content": extra_content,
}
)
return f"$evt-{len(self.media) - 1}"
@@ -299,3 +301,36 @@ async def test_a_post_from_a_person_carries_no_app_marker() -> None:
await CollaborationCore._handle_inbound_message(bridge, _msg(content="hello"))
assert bridge.human_actor.messages[0]["extra_content"] is None
+
+
+async def test_files_an_app_posts_are_marked_too() -> None:
+ # An alerting app that attaches a screenshot is still an app: every file of
+ # its post carries the marker, not only a text message would.
+ bridge = _fake_bridge()
+
+ await CollaborationCore._handle_inbound_message(
+ bridge,
+ _msg(
+ content="dashboard snapshot",
+ attachments=[
+ _attachment("graph.png", "image/png"),
+ _attachment("alert.json", "application/json"),
+ ],
+ sender_is_app=True,
+ ),
+ )
+
+ assert [m["extra_content"] for m in bridge.human_actor.media] == [
+ {SENDER_IS_APP_MARKER: True},
+ {SENDER_IS_APP_MARKER: True},
+ ]
+
+
+async def test_files_a_person_posts_carry_no_app_marker() -> None:
+ bridge = _fake_bridge()
+
+ await CollaborationCore._handle_inbound_message(
+ bridge, _msg(attachments=[_attachment("cat.png", "image/png")])
+ )
+
+ assert [m["extra_content"] for m in bridge.human_actor.media] == [None]
diff --git a/core/tests/switch_core/bridges/collaboration/test_discord_adapter.py b/core/tests/switch_core/bridges/collaboration/test_discord_adapter.py
index 7687b5541..045f6d169 100644
--- a/core/tests/switch_core/bridges/collaboration/test_discord_adapter.py
+++ b/core/tests/switch_core/bridges/collaboration/test_discord_adapter.py
@@ -213,9 +213,10 @@ class _FakeClient:
class _Author:
- def __init__(self, user_id: int, name: str) -> None:
+ def __init__(self, user_id: int, name: str, *, bot: bool = False) -> None:
self.id = user_id
self.name = name
+ self.bot = bot
def _gateway_message(
@@ -322,6 +323,25 @@ def test_own_webhook_message_dropped_but_foreign_webhook_bridged() -> None:
assert len(captured) == 1
assert captured[0].message_ref == f"{CHANNEL_ID}:2"
+ # Someone else's webhook is an integration posting, not a person.
+ assert captured[0].sender_is_app is True
+
+
+def test_another_bot_is_bridged_as_an_app_and_a_person_is_not() -> None:
+ adapter = _adapter()
+ captured = _capture_messages(adapter)
+ channel = _FakeChannel()
+
+ _run(
+ adapter._handle_message(
+ _gateway_message(
+ channel=channel, author=_Author(55, "ci-bot", bot=True), message_id=1
+ )
+ )
+ )
+ _run(adapter._handle_message(_gateway_message(channel=channel, message_id=2)))
+
+ assert [m.sender_is_app for m in captured] == [True, False]
def test_system_messages_skipped() -> None:
diff --git a/core/tests/switch_core/bridges/collaboration/test_mattermost_webhook_posts.py b/core/tests/switch_core/bridges/collaboration/test_mattermost_webhook_posts.py
index 64ac963ca..c30def038 100644
--- a/core/tests/switch_core/bridges/collaboration/test_mattermost_webhook_posts.py
+++ b/core/tests/switch_core/bridges/collaboration/test_mattermost_webhook_posts.py
@@ -20,10 +20,13 @@ from switch_core.bridges.collaboration.mattermost.adapter import (
from switch_core.bridges.collaboration.models import InboundCommand, InboundMessage
_CREATOR = "creator-user-id"
+_BOT_ACCOUNT = "jira-bot-user-id"
class _Users:
- def get_user(self, user_id: str) -> dict[str, str]:
+ def get_user(self, user_id: str) -> dict[str, object]:
+ if user_id == _BOT_ACCOUNT:
+ return {"id": _BOT_ACCOUNT, "username": "jira", "is_bot": True}
if user_id != _CREATOR:
raise AssertionError(f"looked up {user_id!r}, not a Mattermost user")
return {"id": _CREATOR, "username": "ada"}
@@ -84,6 +87,7 @@ def test_a_persons_own_post_is_theirs() -> None:
[message] = received
assert isinstance(message, InboundMessage)
assert (message.sender_id, message.sender_name) == (_CREATOR, "ada")
+ assert message.sender_is_app is False
def test_a_webhook_post_goes_under_a_stand_in_not_its_creator() -> None:
@@ -98,6 +102,8 @@ def test_a_webhook_post_goes_under_a_stand_in_not_its_creator() -> None:
assert message.sender_id == f"webhook:{_CREATOR}"
# Named after the person who set it up, not the name the post claims.
assert message.sender_name == "ada's webhook"
+ # And an integration posting, for the cards and the people counts alike.
+ assert message.sender_is_app is True
def test_a_command_from_a_webhook_is_not_the_creators() -> None:
@@ -124,3 +130,24 @@ def test_the_stand_in_is_never_mentioned() -> None:
assert loop.run_until_complete(adapter._mention(f"webhook:{_CREATOR}")) is None
finally:
loop.close()
+
+
+def test_a_bot_accounts_post_is_an_apps() -> None:
+ adapter, received = _adapter()
+
+ _deliver(adapter, {**_post("PROJ-1 moved to Done"), "user_id": _BOT_ACCOUNT})
+
+ [message] = received
+ assert isinstance(message, InboundMessage)
+ assert message.sender_id == _BOT_ACCOUNT
+ assert message.sender_is_app is True
+
+
+def test_a_post_mattermost_marks_from_a_bot_is_an_apps() -> None:
+ adapter, received = _adapter()
+
+ _deliver(adapter, _post("nightly report", from_bot="true"))
+
+ [message] = received
+ assert isinstance(message, InboundMessage)
+ assert message.sender_is_app is True
diff --git a/core/tests/switch_core/db/stores/test_user_store_oidc.py b/core/tests/switch_core/db/stores/test_user_store_oidc.py
index d8e970ead..7edd29a39 100644
--- a/core/tests/switch_core/db/stores/test_user_store_oidc.py
+++ b/core/tests/switch_core/db/stores/test_user_store_oidc.py
@@ -23,7 +23,7 @@ class TestGetOrCreateOidcUser:
) -> None:
store = UserStore()
async with session_factory() as session:
- user = await store.get_or_create_oidc_user(
+ user, created = await store.get_or_create_oidc_user(
session,
iss=_ISS,
@@ -33,6 +33,7 @@ class TestGetOrCreateOidcUser:
join_tenant=True,
)
await session.commit()
+ assert created is True
assert user.role == "user"
assert user.password_hash is None
assert user.metadata_ is None
@@ -40,6 +41,17 @@ class TestGetOrCreateOidcUser:
again = await store.get_by_oidc_identity(session, iss=_ISS, sub="okta|9")
assert again is not None and again.id == user.id
+ returning, created_again = await store.get_or_create_oidc_user(
+ session,
+ iss=_ISS,
+ name="New",
+ sub="okta|9",
+ email_verified=True,
+ join_tenant=True,
+ )
+ assert (returning.id, created_again) == (user.id, False)
+
async def test_new_user_email_is_stored_lowercase(
self, session_factory: async_sessionmaker[AsyncSession]
) -> None:
@@ -48,7 +60,7 @@ class TestGetOrCreateOidcUser:
# case-insensitive and would have caught a collision either way.
store = UserStore()
async with session_factory() as session:
- user = await store.get_or_create_oidc_user(
+ user, _ = await store.get_or_create_oidc_user(
session,
iss=_ISS,
@@ -65,7 +77,7 @@ class TestGetOrCreateOidcUser:
) -> None:
store = UserStore()
async with session_factory() as session:
- first = await store.get_or_create_oidc_user(
+ first, _ = await store.get_or_create_oidc_user(
session,
iss=_ISS,
@@ -78,7 +90,7 @@ class TestGetOrCreateOidcUser:
first.role = "admin"
await session.commit()
- again = await store.get_or_create_oidc_user(
+ again, _ = await store.get_or_create_oidc_user(
session,
iss=_ISS,
@@ -107,7 +119,7 @@ class TestGetOrCreateOidcUser:
await store.create(session, existing)
await session.commit()
- linked = await store.get_or_create_oidc_user(
+ linked, created = await store.get_or_create_oidc_user(
session,
iss=_ISS,
@@ -119,6 +131,8 @@ class TestGetOrCreateOidcUser:
await session.commit()
assert linked.id == existing.id
+ # Linking an account someone already had is not a new account.
+ assert created is False
# Password login must not be weakened by the link.
assert linked.password_hash == "bcrypt-hash"
@@ -143,7 +157,7 @@ class TestGetOrCreateOidcUser:
await store.create(session, existing)
await session.commit()
- linked = await store.get_or_create_oidc_user(
+ linked, _ = await store.get_or_create_oidc_user(
session,
iss=_ISS,
@@ -203,7 +217,7 @@ class TestGetOrCreateOidcUser:
# both resolve back to the same account, never to each other.
store = UserStore()
async with session_factory() as session:
- first = await store.get_or_create_oidc_user(
+ first, _ = await store.get_or_create_oidc_user(
session,
iss=_ISS,
@@ -214,7 +228,7 @@ class TestGetOrCreateOidcUser:
)
await session.commit()
- second = await store.get_or_create_oidc_user(
+ second, _ = await store.get_or_create_oidc_user(
session,
iss=_ISS,
@@ -268,7 +282,7 @@ class TestGetOrCreateOidcUser:
# the same subject can never move the identity to someone else's row.
store = UserStore()
async with session_factory() as session:
- owner = await store.get_or_create_oidc_user(
+ owner, _ = await store.get_or_create_oidc_user(
session,
iss=_ISS,
@@ -288,7 +302,7 @@ class TestGetOrCreateOidcUser:
await store.create(session, other)
await session.commit()
- again = await store.get_or_create_oidc_user(
+ again, _ = await store.get_or_create_oidc_user(
session,
iss=_ISS,
@@ -317,7 +331,7 @@ class TestGetOrCreateOidcUser:
session.add(OidcIdentity(user_id=legacy_user.id, iss=None, sub="okta|7"))
await session.commit()
- got = await store.get_or_create_oidc_user(
+ got, _ = await store.get_or_create_oidc_user(
session,
iss=_ISS,
@@ -355,7 +369,7 @@ class TestGetOrCreateOidcUser:
session.add(OidcIdentity(user_id=legacy_admin.id, iss=None, sub="okta|8"))
await session.commit()
- got = await store.get_or_create_oidc_user(
+ got, _ = await store.get_or_create_oidc_user(
session,
iss=_ISS,
@@ -378,7 +392,7 @@ class TestGetOrCreateOidcUser:
logging.WARNING, logger="switch_core.db.stores.user_store"
):
async with session_factory() as session:
- user = await store.get_or_create_oidc_user(
+ user, _ = await store.get_or_create_oidc_user(
session,
iss=_ISS,
@@ -483,7 +497,7 @@ class TestGetOrCreateOidcUser:
# row instead of surfacing that as a 500.
store = UserStore()
async with session_factory() as winner_session:
- winner = await store.get_or_create_oidc_user(
+ winner, _ = await store.get_or_create_oidc_user(
winner_session,
iss=_ISS,
@@ -508,7 +522,7 @@ class TestGetOrCreateOidcUser:
monkeypatch.setattr(store, "get_by_oidc_identity", lookup_that_misses_once)
- resolved = await store.get_or_create_oidc_user(
+ resolved, _ = await store.get_or_create_oidc_user(
session,
iss=_ISS,
@@ -529,7 +543,7 @@ class TestGetOrCreateOidcUser:
# the winner's row instead of surfacing that as a 500.
store = UserStore()
async with session_factory() as winner_session:
- winner = await store.get_or_create_oidc_user(
+ winner, _ = await store.get_or_create_oidc_user(
winner_session,
iss=_ISS,
@@ -552,7 +566,7 @@ class TestGetOrCreateOidcUser:
monkeypatch.setattr(store, "get_by_email", lookup_that_misses_once)
- resolved = await store.get_or_create_oidc_user(
+ resolved, _ = await store.get_or_create_oidc_user(
session,
iss=_ISS,
@@ -586,7 +600,7 @@ class TestProvisioningWithoutJoiningATenant:
) -> None:
store = UserStore()
async with session_factory() as session:
- user = await store.get_or_create_oidc_user(
+ user, _ = await store.get_or_create_oidc_user(
session,
iss=_ISS,
@@ -612,7 +626,7 @@ class TestProvisioningWithoutJoiningATenant:
session.add(existing)
await session.commit()
- user = await store.get_or_create_oidc_user(
+ user, _ = await store.get_or_create_oidc_user(
session,
iss=_ISS,
diff --git a/core/tests/switch_core/delivery/test_attachment_groups.py b/core/tests/switch_core/delivery/test_attachment_groups.py
index c5771b960..0213d6bd7 100644
--- a/core/tests/switch_core/delivery/test_attachment_groups.py
+++ b/core/tests/switch_core/delivery/test_attachment_groups.py
@@ -528,6 +528,7 @@ async def test_a_grouped_post_addressed_to_the_agent_is_reported_once(
"room_id": scene.room.id,
"sender_transport_user_id": scene.alice.transport_user_id,
"from_platform": False,
+ "from_app": False,
"agent_metadata": None,
"agent_live": True,
"has_attachment": True,
diff --git a/core/tests/switch_core/delivery/test_reports_addressed.py b/core/tests/switch_core/delivery/test_reports_addressed.py
index e9a4f888e..ad63f4bb8 100644
--- a/core/tests/switch_core/delivery/test_reports_addressed.py
+++ b/core/tests/switch_core/delivery/test_reports_addressed.py
@@ -92,6 +92,7 @@ async def test_an_addressed_message_is_reported_once(scene: _Scene) -> None:
"room_id": scene.room.id,
"sender_transport_user_id": scene.alice.transport_user_id,
"from_platform": False,
+ "from_app": False,
"agent_metadata": None,
"agent_live": True,
"has_attachment": False,
diff --git a/core/tests/switch_core/gateway/test_cloud_controller_machines.py b/core/tests/switch_core/gateway/test_cloud_controller_machines.py
index df32e5205..787b95199 100644
--- a/core/tests/switch_core/gateway/test_cloud_controller_machines.py
+++ b/core/tests/switch_core/gateway/test_cloud_controller_machines.py
@@ -39,6 +39,7 @@ from switch_core.db.stores.hosted_machine_store import (
workspace_on,
workspaces_on,
)
+from switch_core.gateway import cloud_controllers, hosted_controller, hosted_machines
from switch_core.gateway.auth import get_current_user
from switch_core.gateway.cloud_controllers import (
set_cloud_enrollment,
@@ -67,6 +68,7 @@ from tests.switch_core.management.harness import (
platform,
report_status,
)
+from tests.telemetry_records import RecordingSink, recording_telemetry
TOKEN = "SYNTHETIC-CLOUD-CONTROLLER-CREDENTIAL-0123456789"
HEADERS = {"Authorization": f"Bearer {TOKEN}"}
@@ -235,6 +237,15 @@ async def _connected(
return controller
+def reported(monkeypatch: pytest.MonkeyPatch) -> RecordingSink:
+ """What the cloud machine routes and the controller's reports send."""
+ service, sink = recording_telemetry()
+ for module in (cloud_controllers, hosted_controller, hosted_machines):
+ monkeypatch.setattr(module, "current_telemetry", lambda: service)
+ return sink
+
+
class TestControllerMachines:
async def test_a_machine_enrolls_its_controller_with_a_code_core_hands_over(
self, harness: Harness
@@ -370,7 +381,7 @@ class TestControllerMachines:
]
async def test_its_status_reports_make_it_ready_and_its_agents_are_the_machines(
- self, harness: Harness
+ self, harness: Harness, reported: RecordingSink
) -> None:
owner = await add_member(harness.session_factory, "ada")
machine = await _claim(harness.session_factory, owner)
@@ -408,6 +419,12 @@ class TestControllerMachines:
assert ready.state == "ready"
assert ready.heartbeat is not None
assert ready.heartbeat["disk"]["total_bytes"] > 0
+ (became_ready,) = reported.events("cloud_machine_ready")
+ assert became_ready["new_machine"] is True
+ assert isinstance(became_ready["provisioning_seconds"], float)
+ assert became_ready["provisioning_seconds"] >= 0
+ await report_status(client, controller, 3) # type: ignore[arg-type]
+ assert len(reported.events("cloud_machine_ready")) == 1
created = await create_managed_agent(
client,
@@ -549,7 +566,7 @@ class TestControllerMachines:
assert after.desired_state == "running"
async def test_an_idle_machine_with_agents_is_put_to_sleep(
- self, harness: Harness
+ self, harness: Harness, reported: RecordingSink
) -> None:
owner = await add_member(harness.session_factory, "ada")
machine = await _claim(harness.session_factory, owner)
@@ -581,9 +598,10 @@ class TestControllerMachines:
assert listed.status_code == 200, listed.text
after = await _machine(harness.session_factory, machine.id)
assert (after.desired_state, after.stop_reason) == ("stopped", "idle")
+ assert reported.events("cloud_machine_stopped") == [{"reason": "idle"}]
async def test_a_message_or_a_placement_wakes_a_sleeping_machine(
- self, harness: Harness
+ self, harness: Harness, reported: RecordingSink
) -> None:
owner = await add_member(harness.session_factory, "ada")
machine = await _claim(harness.session_factory, owner)
@@ -603,7 +621,7 @@ class TestControllerMachines:
before = await _machine(harness.session_factory, machine.id)
async with harness.session_factory() as session:
woken = await wake_controller_machine(
- session, enrolled["controller_id"], datetime.now(UTC)
+ session, enrolled["controller_id"], datetime.now(UTC), "message"
)
await session.commit()
assert woken is not None
@@ -619,13 +637,47 @@ class TestControllerMachines:
)
async with harness.session_factory() as session:
await wake_controller_machine(
- session, enrolled["controller_id"], datetime.now(UTC)
+ session, enrolled["controller_id"], datetime.now(UTC), "message"
)
await session.commit()
# A machine its owner stopped stays stopped.
assert (await _machine(harness.session_factory, machine.id)).desired_state == (
"stopped"
)
+ assert reported.events("cloud_machine_woken") == [
+ {"trigger": "message", "stopped_by": "idle"}
+ ]
+
+ async def test_placing_an_agent_on_a_sleeping_machine_wakes_it(
+ self, harness: Harness, reported: RecordingSink
+ ) -> None:
+ owner = await add_member(harness.session_factory, "ada")
+ machine = await _claim(harness.session_factory, owner)
+ async with _hosted_app(harness.session_factory, owner) as hosted:
+ code = (await _prepare(hosted, machine.id))["controllers"][0][
+ "enrollment_code"
+ ]
+ async with harness.client() as client:
+ enrolled = await _enroll(client, code)
+ await _connected(client, harness, owner, enrolled)
+ await _update(
+ harness.session_factory,
+ machine.id,
+ desired_state="stopped",
+ stop_reason="idle",
+ )
+ created = await create_managed_agent(
+ client,
+ owner,
+ name="reviewer",
+ controller_id=enrolled["controller_id"],
+ )
+ assert created.status_code == 201, created.text
+ after = await _machine(harness.session_factory, machine.id)
+ assert (after.desired_state, after.stop_reason) == ("running", None)
+ assert reported.events("cloud_machine_woken") == [
+ {"trigger": "agent_placed", "stopped_by": "idle"}
+ ]
async def test_a_controller_machine_needs_agent_management(
diff --git a/core/tests/switch_core/gateway/test_hosted_controller.py b/core/tests/switch_core/gateway/test_hosted_controller.py
index 10b11f4db..cccaad7b6 100644
--- a/core/tests/switch_core/gateway/test_hosted_controller.py
+++ b/core/tests/switch_core/gateway/test_hosted_controller.py
@@ -30,6 +30,7 @@ from switch_core.db.stores.hosted_machine_store import (
lock_claims,
workspace_on,
)
+from switch_core.gateway import hosted_controller
from switch_core.gateway.auth import get_current_user
from switch_core.gateway.cloud_controllers import (
controller_machine_idle,
@@ -55,6 +56,7 @@ from tests.switch_core.hosted_machine_helpers import (
place_managed_agent,
seed_machine,
)
+from tests.telemetry_records import recording_telemetry
TEST_KEYRING = Keyring.parse("test:" + "x" * 40, legacy_secret=None)
@@ -1384,3 +1386,76 @@ async def test_a_stopped_machine_enrolls_its_revoked_controller_at_its_next_star
assert await enroll_revoked_again(session, here, datetime.now(UTC)) is not None
await session.commit()
assert (await machine_of(app.factory, app.machine_id)).revision == before
+
+
+def reported(monkeypatch):
+ """What the operator controller's routes report."""
+ service, sink = recording_telemetry()
+ monkeypatch.setattr(hosted_controller, "current_telemetry", lambda: service)
+ return sink
+
+
+ ("values", "reason"),
+ [
+ ({"updated_at_minutes_ago": 11}, "not_scheduled"),
+ (
+ {"state": "provisioning", "running_observed_at_minutes_ago": 11},
+ "never_connected",
+ ),
+ ({"state": "provisioning", "updated_at_minutes_ago": 11}, "never_started"),
+ ],
+)
+async def test_each_timeout_is_reported_with_its_own_reason(
+ controller_app, reported, values, reason
+):
+ app = controller_app
+ now = datetime.now(UTC)
+ changes: dict[str, object] = {}
+ if "state" in values:
+ changes["state"] = values["state"]
+ if "updated_at_minutes_ago" in values:
+ changes["updated_at"] = now - timedelta(
+ minutes=values["updated_at_minutes_ago"]
+ )
+ if "running_observed_at_minutes_ago" in values:
+ changes["running_observed_at"] = now - timedelta(
+ minutes=values["running_observed_at_minutes_ago"]
+ )
+ await update_machine(app.factory, app.machine_id, **changes)
+
+ await list_machines(app.client)
+ await list_machines(app.client)
+
+ assert reported.events("cloud_machine_failed") == [{"reason": reason}]
+
+
+async def test_a_departed_owner_is_reported_once(controller_app, reported):
+ app = controller_app
+ async with app.factory() as session:
+ await session.delete(
+ await session.get(TenantMember, (require_tenant_id(), app.owner_id))
+ )
+ await session.commit()
+
+ assert (await prepare(app.client, app.machine_id)).status_code == 409
+ assert (await prepare(app.client, app.machine_id)).status_code == 409
+
+ assert reported.events("cloud_machine_failed") == [{"reason": "owner_left"}]
+
+
+async def test_an_observed_error_is_reported_by_its_code_once(controller_app, reported):
+ app = controller_app
+ await update_machine(app.factory, app.machine_id, state="provisioning")
+ for _ in range(2):
+ await observe(
+ app.client,
+ app.machine_id,
+ state="error",
+ revision=1,
+ error="The instance failed its status checks.",
+ error_code="machine_needs_attention",
+ )
+
+ assert reported.events("cloud_machine_failed") == [{"reason": "needs_attention"}]
diff --git a/core/tests/switch_core/gateway/test_organization_routes.py b/core/tests/switch_core/gateway/test_organization_routes.py
index a62240df1..6ab920633 100644
--- a/core/tests/switch_core/gateway/test_organization_routes.py
+++ b/core/tests/switch_core/gateway/test_organization_routes.py
@@ -11,6 +11,7 @@ from datetime import timedelta
from types import SimpleNamespace
import httpx
+import pytest
from fastapi import FastAPI
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
@@ -40,10 +41,12 @@ from switch_core.db.stores.tenant_store import TenantStore
from switch_core.db.stores.user_store import UserStore
from switch_core.db.stores.workspace_service_store import WorkspaceServiceStore
from switch_core.gateway import dependencies as gw_deps
+from switch_core.gateway import workspace_creation
from switch_core.gateway.auth import create_jwt, decode_jwt
from switch_core.gateway.organizations import router as organizations_router
from switch_core.gateway.tenants import router as tenants_router
from switch_core.keys import Keyring
+from tests.telemetry_records import recording_telemetry
_KEYRING = Keyring.parse(
"test:" + "organization-routes-test-secret-" + "0" * 32, legacy_secret=None
@@ -587,6 +590,33 @@ async def _workspaces_created(
class TestCreatingAnOrganization:
+ async def test_a_new_organization_and_a_workspace_in_one_are_reported(
+ self,
+ session_factory: async_sessionmaker[AsyncSession],
+ monkeypatch: pytest.MonkeyPatch,
+ ) -> None:
+ service, sink = recording_telemetry()
+ monkeypatch.setattr(workspace_creation, "current_telemetry", lambda: service)
+ ada = await _user(session_factory, "user")
+
+ async with _client(_app(session_factory), ada, None) as client:
+ created = await client.post(
+ "/organizations", json={"name": "Acme", "workspace_name": "Research"}
+ )
+ assert created.status_code == 201, created.text
+ async with _client(
+ _app(session_factory), ada, created.json()["workspace"]["id"]
+ ) as client:
+ another = await client.post(
+ "/organizations/current/workspaces", json={"name": "Design"}
+ )
+ assert another.status_code == 201, another.text
+
+ assert sink.events("workspace_created") == [
+ {"new_organization": True},
+ {"new_organization": False},
+ ]
+
async def test_anyone_signed_in_creates_one_with_its_first_workspace(
self, session_factory: async_sessionmaker[AsyncSession]
) -> None:
diff --git a/core/tests/switch_core/gateway/test_signup.py b/core/tests/switch_core/gateway/test_signup.py
index a4315e671..ff36d6f68 100644
--- a/core/tests/switch_core/gateway/test_signup.py
+++ b/core/tests/switch_core/gateway/test_signup.py
@@ -22,6 +22,7 @@ from switch_core.db.stores.hosted_machine_store import (
workspace_on,
)
from switch_core.db.stores.user_store import UserStore
+from switch_core.gateway import auth_routes, hosted_machines
from switch_core.gateway.auth import decode_jwt, get_current_user, verify_password
from switch_core.gateway.auth_routes import MACHINE_OWNER_STOPPED, _prewarm
from switch_core.gateway.auth_routes import router as auth_router
@@ -36,6 +37,7 @@ from switch_core.gateway.hosted_machines import MACHINES_DISABLED
from switch_core.gateway.hosted_machines import router as machine_router
from switch_core.keys import Keyring
from switch_core.tenant_context import tenant_scope
+from tests.telemetry_records import recording_telemetry
TEST_KEYRING = Keyring.parse("test:" + "x" * 40, legacy_secret=None)
@@ -530,3 +532,96 @@ async def test_concurrent_signups_cannot_overrun_the_cap(signup_app):
)
assert sorted(r.status_code for r in responses) == [201, 429]
assert await _user_count(app) == 1
+
+
+def reported(monkeypatch):
+ """What sign-up and the cloud machine routes report."""
+ service, sink = recording_telemetry()
+ monkeypatch.setattr(auth_routes, "current_telemetry", lambda: service)
+ monkeypatch.setattr(hosted_machines, "current_telemetry", lambda: service)
+ return sink
+
+
+async def test_a_sign_up_reports_the_account_and_the_new_machine(signup_app, reported):
+ assert (await _signup(signup_app)).status_code == 201
+
+ assert reported.events("account_created") == [{"method": "signup"}]
+ assert reported.events("cloud_machine_requested") == [{"change": "new"}]
+
+
+async def test_a_claim_that_changes_nothing_is_not_reported(signup_app, reported):
+ await _signup(signup_app)
+ assert (await _ensure(signup_app, "[email protected]")).status_code == 200
+
+ assert reported.events("cloud_machine_requested") == [{"change": "new"}]
+
+
+async def test_a_claim_that_starts_a_sleeping_machine_says_so(signup_app, reported):
+ app = signup_app
+ await _signup(app)
+ (machine,) = await _machines(app)
+ with tenant_scope(TENANT_ZERO_ID):
+ async with app.factory() as session:
+ row = await session.get(CloudMachine, machine.id)
+ assert row is not None
+ row.desired_state = "stopped"
+ row.stop_reason = "idle"
+ await session.commit()
+
+ assert (await _ensure(app, "[email protected]")).status_code == 200
+
+ assert reported.events("cloud_machine_requested") == [
+ {"change": "new"},
+ {"change": "restarted"},
+ ]
+
+
+async def test_refused_sign_ups_say_why(signup_app, reported):
+ app = signup_app
+ assert (await _signup(app)).status_code == 201
+ assert (await _signup(app)).status_code == 409
+ app.config.gateway_signup_open = False
+ assert (await _signup(app, email="[email protected]")).status_code == 403
+
+ assert reported.events("signup_refused") == [
+ {"reason": "email_taken"},
+ {"reason": "disabled"},
+ ]
+ assert reported.events("account_created") == [{"method": "signup"}]
+
+
+async def test_a_sign_up_over_the_hourly_cap_is_reported_as_rate_limited(
+ signup_app, reported
+):
+ signup_app.config.gateway_signup_max_per_hour = 1
+ assert (await _signup(signup_app, email="[email protected]")).status_code == 201
+ assert (await _signup(signup_app, email="[email protected]")).status_code == 429
+
+ assert reported.events("signup_refused") == [{"reason": "rate_limited"}]
+
+
+async def test_the_owner_stopping_and_starting_their_machine_is_reported(
+ signup_app, reported
+):
+ app = signup_app
+ await _signup(app)
+ app.identity["user"] = await _user(app, "[email protected]")
+ (machine,) = await _machines(app)
+
+ with tenant_scope(TENANT_ZERO_ID):
+ stopped = await app.client.post(
+ f"/hosted-machines/{machine.id}/lifecycle",
+ json={"action": "stop", "revision": machine.revision},
+ )
+ assert stopped.status_code == 200, stopped.text
+ started = await app.client.post(
+ f"/hosted-machines/{machine.id}/lifecycle",
+ json={"action": "start", "revision": stopped.json()["machine"]["revision"]},
+ )
+ assert started.status_code == 200, started.text
+
+ assert reported.events("cloud_machine_stopped") == [{"reason": "owner"}]
+ assert reported.events("cloud_machine_woken") == [
+ {"trigger": "owner", "stopped_by": "owner"}
+ ]
diff --git a/core/tests/switch_core/management/test_my_context.py b/core/tests/switch_core/management/test_my_context.py
index 122b93568..547ef7c69 100644
--- a/core/tests/switch_core/management/test_my_context.py
+++ b/core/tests/switch_core/management/test_my_context.py
@@ -79,6 +79,13 @@ def _config(
key: key == SWITCH_CLOUD and cloud for key in KNOWN_FEATURE_FLAGS
},
managed_by_console=managed_by_console,
+ deployment_kind=(
+ "switch_cloud"
+ if cloud
+ else "managed_by_console"
+ if managed_by_console
+ else "self_hosted"
+ ),
)
diff --git a/core/tests/switch_core/session_activity/test_session_started_route.py b/core/tests/switch_core/session_activity/test_session_started_route.py
index cbe78c73a..26c825d76 100644
--- a/core/tests/switch_core/session_activity/test_session_started_route.py
+++ b/core/tests/switch_core/session_activity/test_session_started_route.py
@@ -30,6 +30,7 @@ from switch_core.telemetry.session_start import (
default_session_start_limiter,
)
from switch_core.telemetry.sink import TelemetryRecord
+from tests.telemetry_records import declared
# What every launcher mints: a random v4 from Console, or a v5-shaped hash from
# the room watcher.
@@ -55,6 +56,7 @@ def _telemetry(sink: _RecordingSink, *, enabled: bool) -> TelemetryService:
version=None,
telemetry_environment="prod",
telemetry_internal=False,
+ deployment_kind="self_hosted",
)
@@ -88,9 +90,7 @@ async def _post(app: FastAPI, session_id: str, body: object) -> httpx.Response:
def _started(sink: _RecordingSink) -> list[dict[str, object]]:
- return [
- dict(r.properties) for r in sink.sent if r.name == "switch_core.session_started"
- ]
+ return [declared(r) for r in sink.sent if r.name == "switch_core.session_started"]
async def test_a_new_session_is_reported_with_how_it_started(session_factory) -> None:
diff --git a/core/tests/switch_core/telemetry/test_bridge_connect_reporting.py b/core/tests/switch_core/telemetry/test_bridge_connect_reporting.py
index bfebd621e..008e7b8b0 100644
--- a/core/tests/switch_core/telemetry/test_bridge_connect_reporting.py
+++ b/core/tests/switch_core/telemetry/test_bridge_connect_reporting.py
@@ -52,6 +52,7 @@ from tests.switch_core.bridges.collaboration.test_lifecycle_tenant_binding impor
_StubAdapter,
_StubConfig,
)
+from tests.telemetry_records import declared
class _RecordingSink:
@@ -80,6 +81,7 @@ def _service_with_telemetry(
version=None,
telemetry_environment="prod",
telemetry_internal=False,
+ deployment_kind="self_hosted",
session_factory=session_factory,
installed_at=installed_at,
)
@@ -128,7 +130,7 @@ async def _bridge_connected_events(sink: _RecordingSink) -> list[dict[str, Any]]
for record in sink.sent:
if record.name != "switch_core.bridge_connected":
continue
- properties = dict(record.properties)
+ properties = declared(record)
duration = properties.pop("duration_ms")
assert isinstance(duration, int | float) and not isinstance(duration, bool)
assert duration >= 0, "a connect cannot take less than no time"
diff --git a/core/tests/switch_core/telemetry/test_connector_setup_time.py b/core/tests/switch_core/telemetry/test_connector_setup_time.py
index bbe033441..00fa0599a 100644
--- a/core/tests/switch_core/telemetry/test_connector_setup_time.py
+++ b/core/tests/switch_core/telemetry/test_connector_setup_time.py
@@ -36,6 +36,7 @@ def _switched_off(telemetry: TelemetryService) -> TelemetryService:
version=None,
telemetry_environment="prod",
telemetry_internal=False,
+ deployment_kind="self_hosted",
session_factory=telemetry._session_factory,
installed_at=INSTALLED,
)
diff --git a/core/tests/switch_core/telemetry/test_deployment_and_reporter.py b/core/tests/switch_core/telemetry/test_deployment_and_reporter.py
index 20c3be2c8..937bdf2c1 100644
--- a/core/tests/switch_core/telemetry/test_deployment_and_reporter.py
+++ b/core/tests/switch_core/telemetry/test_deployment_and_reporter.py
@@ -175,6 +175,7 @@ class TestEmitMilestone:
version="1.0.0",
telemetry_environment="prod",
telemetry_internal=False,
+ deployment_kind="self_hosted",
session_factory=session_factory,
installed_at=installed_at,
)
@@ -236,6 +237,7 @@ class TestTheSnapshotSchedule:
version="1.0.0",
telemetry_environment="prod",
telemetry_internal=False,
+ deployment_kind="self_hosted",
session_factory=session_factory,
installed_at=datetime.now(UTC) - timedelta(days=1),
)
diff --git a/core/tests/switch_core/telemetry/test_emit_after_commit.py b/core/tests/switch_core/telemetry/test_emit_after_commit.py
new file mode 100644
index 000000000..c8b5bdcfb
--- /dev/null
+++ b/core/tests/switch_core/telemetry/test_emit_after_commit.py
@@ -0,0 +1,67 @@
+"""An event about a change is sent once the change is committed, never for
+one that rolled back.
+
+Against real Postgres, because commit and rollback are what is being tested.
+"""
+
+from __future__ import annotations
+
+from sqlalchemy import text
+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
+
+from switch_core.telemetry import emit_after_commit
+from tests.telemetry_records import recording_telemetry
+
+STOPPED = {"reason": "idle"}
+
+
+async def test_it_is_sent_when_the_transaction_commits(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
+ service, sink = recording_telemetry()
+ async with session_factory() as session:
+ await session.execute(text("SELECT 1"))
+ emit_after_commit(session, service, "cloud_machine_stopped", STOPPED)
+ assert sink.sent == []
+ await session.commit()
+ await service.aclose()
+
+ assert sink.events("cloud_machine_stopped") == [STOPPED]
+
+
+async def test_a_rolled_back_change_is_never_sent_not_even_by_a_later_commit(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
+ service, sink = recording_telemetry()
+ async with session_factory() as session:
+ await session.execute(text("SELECT 1"))
+ emit_after_commit(session, service, "cloud_machine_stopped", STOPPED)
+ await session.rollback()
+ await session.execute(text("SELECT 1"))
+ await session.commit()
+ await service.aclose()
+
+ assert sink.sent == []
+
+
+async def test_a_savepoint_rolling_back_does_not_cancel_it(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
+ service, sink = recording_telemetry()
+ async with session_factory() as session:
+ await session.execute(text("SELECT 1"))
+ emit_after_commit(session, service, "cloud_machine_stopped", STOPPED)
+ nested = await session.begin_nested()
+ await nested.rollback()
+ await session.commit()
+ await service.aclose()
+
+ assert sink.events("cloud_machine_stopped") == [STOPPED]
+
+
+async def test_nothing_is_wired_where_there_is_no_service(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
+ async with session_factory() as session:
+ emit_after_commit(session, None, "cloud_machine_stopped", STOPPED)
+ await session.commit()
diff --git a/core/tests/switch_core/telemetry/test_emitters_are_live.py b/core/tests/switch_core/telemetry/test_emitters_are_live.py
index 798fcaee3..6551f55a1 100644
--- a/core/tests/switch_core/telemetry/test_emitters_are_live.py
+++ b/core/tests/switch_core/telemetry/test_emitters_are_live.py
@@ -41,6 +41,7 @@ def _telemetry() -> TelemetryService:
version="1.0.0",
telemetry_environment="prod",
telemetry_internal=False,
+ deployment_kind="self_hosted",
)
diff --git a/core/tests/switch_core/telemetry/test_every_event_is_emitted.py b/core/tests/switch_core/telemetry/test_every_event_is_emitted.py
index 34baa97ed..732356d49 100644
--- a/core/tests/switch_core/telemetry/test_every_event_is_emitted.py
+++ b/core/tests/switch_core/telemetry/test_every_event_is_emitted.py
@@ -29,11 +29,12 @@ def _emitted_names() -> set[str]:
Matched on the call rather than on the bare string, so a name that appears
only in a comment, a docstring or a reason map does not count as an
- emitter. The three shapes in the tree are `telemetry.emit("x", ...)`,
- `await telemetry.emit_milestone("x", ...)` and
- `emit_safely(target, "x", {...})`.
+ emitter. The shapes in the tree are `telemetry.emit("x", ...)`,
+ `telemetry.emit_at("x", at, ...)`, `await telemetry.emit_milestone("x",
+ ...)`, `emit_safely(target, "x", {...})` and
+ `emit_after_commit(session, target, "x", {...})`.
"""
- emitters = {"emit", "emit_milestone", "emit_safely"}
+ emitters = {"emit", "emit_at", "emit_milestone", "emit_safely", "emit_after_commit"}
found: set[str] = set()
for path in _PACKAGE_ROOT.rglob("*.py"):
if path == _CATALOGUE:
@@ -52,9 +53,10 @@ def _emitted_names() -> set[str]:
)
if name not in emitters:
continue
- # `emit_safely` takes the service first, so scan the leading
- # arguments rather than assuming a position.
- for arg in node.args[:2]:
+ # `emit_safely` takes the service first, and `emit_after_commit`
+ # the session before it, so scan the leading arguments rather than
+ # assuming a position.
+ for arg in node.args[:3]:
if isinstance(arg, ast.Constant) and isinstance(arg.value, str):
found.add(arg.value)
return found
diff --git a/core/tests/switch_core/telemetry/test_first_room_active.py b/core/tests/switch_core/telemetry/test_first_room_active.py
index 733ea8964..8a67941ca 100644
--- a/core/tests/switch_core/telemetry/test_first_room_active.py
+++ b/core/tests/switch_core/telemetry/test_first_room_active.py
@@ -67,6 +67,7 @@ def _reporter(
version="1.0.0",
telemetry_environment="prod",
telemetry_internal=False,
+ deployment_kind="self_hosted",
session_factory=session_factory,
installed_at=INSTALLED,
)
diff --git a/core/tests/switch_core/telemetry/test_merge_fixes.py b/core/tests/switch_core/telemetry/test_merge_fixes.py
index 6a90e99d0..e4447f054 100644
--- a/core/tests/switch_core/telemetry/test_merge_fixes.py
+++ b/core/tests/switch_core/telemetry/test_merge_fixes.py
@@ -142,6 +142,7 @@ class TestASessionIsTheAgentNotTheConnection:
version="1.0.0",
telemetry_environment="prod",
telemetry_internal=False,
+ deployment_kind="self_hosted",
)
reporter = SessionReporter(service, registry)
registry.set_close_listener(reporter.on_close)
@@ -215,7 +216,7 @@ class TestASessionIsTheAgentNotTheConnection:
await asyncio.sleep(0.01) # well inside the grace period
await _settle()
- assert self._names(sink) == ["agent_session_started"]
+ assert self._names(sink) == ["agent_session_started", "agent_connected"]
async def test_the_session_ends_once_the_agent_stays_away(
self, monkeypatch: pytest.MonkeyPatch
@@ -231,8 +232,14 @@ class TestASessionIsTheAgentNotTheConnection:
await asyncio.sleep(0.15)
await _settle()
- assert self._names(sink) == ["agent_session_started", "agent_session_ended"]
- assert sink.sent[1].properties["reason"] == "heartbeat_lapsed"
+ assert self._names(sink) == [
+ "agent_session_started",
+ "agent_connected",
+ "agent_session_ended",
+ "agent_disconnected",
+ ]
+ assert sink.sent[2].properties["reason"] == "heartbeat_lapsed"
+ assert sink.sent[3].properties == sink.sent[2].properties
async def test_the_duration_spans_the_whole_session(
self, monkeypatch: pytest.MonkeyPatch
@@ -266,7 +273,7 @@ class TestASessionIsTheAgentNotTheConnection:
await self._stream(reporter, registry, cid)
await _settle()
- assert self._names(sink) == ["agent_session_started"]
+ assert self._names(sink) == ["agent_session_started", "agent_connected"]
async def test_closing_one_of_several_connections_ends_nothing(
self, monkeypatch: pytest.MonkeyPatch
@@ -284,7 +291,7 @@ class TestASessionIsTheAgentNotTheConnection:
await asyncio.sleep(0.15)
await _settle()
- assert self._names(sink) == ["agent_session_started"]
+ assert self._names(sink) == ["agent_session_started", "agent_connected"]
async def test_two_agents_are_two_sessions(self) -> None:
registry = AgentConnectionRegistry()
@@ -298,7 +305,9 @@ class TestASessionIsTheAgentNotTheConnection:
assert self._names(sink) == [
"agent_session_started",
+ "agent_connected",
"agent_session_started",
+ "agent_connected",
]
async def test_shutdown_reports_no_ends(self) -> None:
@@ -313,7 +322,7 @@ class TestASessionIsTheAgentNotTheConnection:
await reporter.aclose()
await _settle()
- assert self._names(sink) == ["agent_session_started"]
+ assert self._names(sink) == ["agent_session_started", "agent_connected"]
def test_a_listener_that_raises_does_not_break_the_close(self) -> None:
"""The registry's contract — the connection is closed and handed back —
diff --git a/core/tests/switch_core/telemetry/test_message_events.py b/core/tests/switch_core/telemetry/test_message_events.py
index 05cb84edb..ed1130a90 100644
--- a/core/tests/switch_core/telemetry/test_message_events.py
+++ b/core/tests/switch_core/telemetry/test_message_events.py
@@ -9,6 +9,7 @@ there would be a wrong chart with no error anywhere.
from __future__ import annotations
import asyncio
+import dataclasses
import time
import uuid
from types import SimpleNamespace
@@ -32,6 +33,7 @@ from switch_core.telemetry.service import TelemetryService
from switch_core.telemetry.sink import TelemetryRecord
from switch_core.transport.observer import ParticipantMessage
from tests.session_counting import CountingSessionFactory
+from tests.telemetry_records import declared
TENANT_ZERO = "00000000-0000-0000-0000-000000000000"
@@ -55,6 +57,7 @@ def _service(sink: _RecordingSink, *, enabled: bool = True) -> TelemetryService:
version="1.0.0",
telemetry_environment="prod",
telemetry_internal=False,
+ deployment_kind="self_hosted",
)
@@ -163,6 +166,7 @@ def _said(world: _World, sender_client_id: str, role: str) -> ParticipantMessage
room_id=world.room.id,
sender_client_id=sender_client_id,
sender_role=role,
+ sender_is_app=False,
has_attachment=False,
in_thread=False,
)
@@ -176,6 +180,7 @@ def _said_in(
room_id=room_id,
sender_client_id=sender_client_id,
sender_role=sender_role,
+ sender_is_app=False,
has_attachment=False,
in_thread=False,
)
@@ -222,7 +227,7 @@ async def _reported(
await messages._queue.join()
await messages.aclose()
await service.aclose()
- return [(record.name, record.properties) for record in sink.sent]
+ return [(record.name, declared(record)) for record in sink.sent]
class TestRoomMessageSent:
@@ -251,6 +256,25 @@ class TestRoomMessageSent:
)
]
+ async def test_an_integration_posting_is_a_bot_not_a_user(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ """The bridge writes an app's post through a human actor like anyone
+ else's; its mark is what keeps it out of the people count."""
+ world = await _world(session_factory)
+ sink = _RecordingSink()
+ service = _service(sink)
+ messages = MessageTelemetry(telemetry=service, session_factory=session_factory)
+
+ messages.observe(
+ dataclasses.replace(
+ _said(world, world.humans[0].id, "human"), sender_is_app=True
+ )
+ )
+
+ reported = await _reported(messages, service, sink)
+ assert [p["sender_kind"] for _, p in reported] == ["bot"]
+
async def test_an_internal_room_has_no_platform(
self, session_factory: async_sessionmaker[AsyncSession]
) -> None:
@@ -269,6 +293,7 @@ class TestRoomMessageSent:
room_id=room.id,
sender_client_id=human.id,
sender_role="human",
+ sender_is_app=False,
has_attachment=True,
in_thread=True,
)
@@ -295,6 +320,7 @@ class TestRoomMessageSent:
room_id=str(uuid.uuid4()),
sender_client_id="whoever",
sender_role="human",
+ sender_is_app=False,
has_attachment=False,
in_thread=False,
)
@@ -366,6 +392,7 @@ class TestAgentMessageReceived:
room_id=world.room.id,
sender_transport_user_id=world.humans[0].transport_user_id,
from_platform=False,
+ from_app=False,
agent_metadata={"known_agent_type": "claude-code"},
agent_live=True,
has_attachment=False,
@@ -406,6 +433,7 @@ class TestAgentMessageReceived:
room_id=world.room.id,
sender_transport_user_id=sender,
from_platform=from_platform,
+ from_app=False,
agent_metadata=None,
agent_live=False,
has_attachment=False,
@@ -419,6 +447,28 @@ class TestAgentMessageReceived:
]
assert {p["known_agent_type"] for _, p in reported} == {"none"}
+ async def test_an_integration_asking_an_agent_is_a_bot(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ world = await _world(session_factory)
+ sink = _RecordingSink()
+ service = _service(sink)
+ messages = MessageTelemetry(telemetry=service, session_factory=session_factory)
+
+ messages.agent_addressed(
+ tenant_id=TENANT_ZERO,
+ room_id=world.room.id,
+ sender_transport_user_id=world.humans[0].transport_user_id,
+ from_platform=False,
+ from_app=True,
+ agent_metadata=None,
+ agent_live=True,
+ has_attachment=False,
+ )
+
+ reported = await _reported(messages, service, sink)
+ assert [p["sender_kind"] for _, p in reported] == ["bot"]
+
class TestFailingLookups:
async def test_a_room_whose_lookup_failed_is_not_retried_and_warns_once(
@@ -485,6 +535,7 @@ class TestFailingLookups:
room_id="elsewhere",
sender_client_id="someone",
sender_role="human",
+ sender_is_app=False,
has_attachment=False,
in_thread=False,
)
@@ -575,6 +626,7 @@ class TestCaching:
room_id=world.room.id,
sender_transport_user_id=world.humans[0].transport_user_id,
from_platform=False,
+ from_app=False,
agent_metadata=None,
agent_live=True,
has_attachment=False,
@@ -663,6 +715,7 @@ class TestCaching:
room_id=world.room.id,
sender_transport_user_id=late_sender,
from_platform=False,
+ from_app=False,
agent_metadata=None,
agent_live=True,
has_attachment=False,
@@ -721,6 +774,7 @@ class TestTheWholeWorkspace:
room_id=other.id,
sender_transport_user_id=world.humans[1].transport_user_id,
from_platform=False,
+ from_app=False,
agent_metadata=None,
agent_live=True,
has_attachment=False,
@@ -893,6 +947,7 @@ class TestFailingSenderAndRuntimeLookups:
room_id="room",
sender_transport_user_id=sender,
from_platform=False,
+ from_app=False,
agent_metadata=None,
agent_live=True,
has_attachment=False,
@@ -1041,6 +1096,7 @@ class TestFailingReports:
room_id=world.room.id,
sender_transport_user_id="@nobody-we-know:test",
from_platform=False,
+ from_app=False,
agent_metadata=None,
agent_live=True,
has_attachment=False,
@@ -1071,6 +1127,7 @@ class TestShutdown:
room_id="room",
sender_client_id="someone",
sender_role="human",
+ sender_is_app=False,
has_attachment=False,
in_thread=False,
)
@@ -1151,6 +1208,7 @@ class TestShutdown:
room_id="room",
sender_client_id="someone",
sender_role="human",
+ sender_is_app=False,
has_attachment=False,
in_thread=False,
)
@@ -1197,6 +1255,7 @@ class TestOff:
room_id=world.room.id,
sender_transport_user_id=world.humans[0].transport_user_id,
from_platform=False,
+ from_app=False,
agent_metadata=None,
agent_live=False,
has_attachment=False,
diff --git a/core/tests/switch_core/telemetry/test_preconfigured_connectors.py b/core/tests/switch_core/telemetry/test_preconfigured_connectors.py
index bc5f28e05..c0d92fb91 100644
--- a/core/tests/switch_core/telemetry/test_preconfigured_connectors.py
+++ b/core/tests/switch_core/telemetry/test_preconfigured_connectors.py
@@ -39,6 +39,7 @@ from tests.switch_core.telemetry.test_bridge_connect_reporting import (
_RecordingSink,
_service_with_telemetry,
)
+from tests.telemetry_records import declared
INSTALLED = datetime.now(UTC) - timedelta(minutes=5)
@@ -60,9 +61,7 @@ async def _events(
) -> list[dict[str, Any]]:
await service._telemetry.aclose() # type: ignore[union-attr]
return [
- dict(record.properties)
- for record in sink.sent
- if record.name == f"switch_core.{name}"
+ declared(record) for record in sink.sent if record.name == f"switch_core.{name}"
]
@@ -156,7 +155,7 @@ class TestTheConnectorAPersonAdds:
assert await _events(service, sink, "connector_added") == []
[milestone] = [
- dict(r.properties)
+ declared(r)
for r in sink.sent
if r.name == "switch_core.first_connector_added"
]
@@ -269,6 +268,7 @@ class TestRegisteringRecordsTheFlag:
version=None,
telemetry_environment="prod",
telemetry_internal=False,
+ deployment_kind="self_hosted",
)
service._session_factory = session_factory
service._client_lifecycle.create_client = AsyncMock( # type: ignore[method-assign]
diff --git a/core/tests/switch_core/telemetry/test_relay_attribute_cap.py b/core/tests/switch_core/telemetry/test_relay_attribute_cap.py
index bdb6b23b5..718a28ffd 100644
--- a/core/tests/switch_core/telemetry/test_relay_attribute_cap.py
+++ b/core/tests/switch_core/telemetry/test_relay_attribute_cap.py
@@ -81,6 +81,7 @@ async def _attributes_on_the_wire(
version="1.0.0",
telemetry_environment="prod",
telemetry_internal=False,
+ deployment_kind="self_hosted",
)
service.emit(event, **{key: value(kind) for key, kind in event_spec(event).items()})
await service.aclose()
diff --git a/core/tests/switch_core/telemetry/test_service_and_sink.py b/core/tests/switch_core/telemetry/test_service_and_sink.py
index 2f28b277c..580571896 100644
--- a/core/tests/switch_core/telemetry/test_service_and_sink.py
+++ b/core/tests/switch_core/telemetry/test_service_and_sink.py
@@ -4,13 +4,18 @@ from __future__ import annotations
import asyncio
import json
+from datetime import UTC, datetime
import httpx
import pytest
from switch_core.observability.otlp import OtlpClient
from switch_core.telemetry import sink as sink_module
-from switch_core.telemetry.catalogue import TelemetryCatalogueError
+from switch_core.telemetry.catalogue import (
+ CATALOGUE,
+ DEPLOYMENT_KIND_PROPERTY,
+ TelemetryCatalogueError,
+)
from switch_core.telemetry.service import TelemetryService, emit_safely
from switch_core.telemetry.sink import NullSink, OtlpRelaySink, TelemetryRecord
@@ -52,6 +57,7 @@ def _service(sink: object, *, enabled: bool = True) -> TelemetryService:
version="1.2.3",
telemetry_environment="prod",
telemetry_internal=False,
+ deployment_kind="self_hosted",
)
@@ -114,6 +120,7 @@ class TestTagging:
version="1.2.3",
telemetry_environment=environment,
telemetry_internal=False,
+ deployment_kind="self_hosted",
)
service.emit("deployment_started", tenant_count=1)
@@ -134,6 +141,7 @@ class TestTagging:
version="1.2.3",
telemetry_environment="prod",
telemetry_internal=internal,
+ deployment_kind="self_hosted",
)
service.emit("deployment_started", tenant_count=1)
@@ -141,6 +149,48 @@ class TestTagging:
assert sink.sent[0].resource["flint_internal"] == sent
+ @pytest.mark.parametrize(
+ "kind", ["switch_cloud", "managed_by_console", "self_hosted"]
+ )
+ async def test_every_event_says_who_runs_the_deployment(self, kind: str) -> None:
+ """An event property rather than a resource attribute: the relay
+ passes event properties to Amplitude, so Switch Cloud's traffic can be
+ filtered out of, or into, any chart."""
+ sink = _RecordingSink()
+ service = TelemetryService(
+ sink=sink, # type: ignore[arg-type]
+ enabled=True,
+ client_id="deployment-uuid",
+ version="1.2.3",
+ telemetry_environment="prod",
+ telemetry_internal=False,
+ deployment_kind=kind, # type: ignore[arg-type]
+ )
+
+ service.emit("deployment_started", tenant_count=1)
+ service.emit_at(
+ "usage_period",
+ datetime(2026, 11, 2, 12, tzinfo=UTC),
+ period="day",
+ room_active=0,
+ person_identity_active=0,
+ account_active=0,
+ tenant_failed_count=0,
+ )
+ await service.aclose()
+
+ assert [record.properties["deployment_kind"] for record in sink.sent] == [
+ kind,
+ kind,
+ ]
+
+ def test_no_event_declares_the_property_the_service_adds(self) -> None:
+ assert not [
+ event
+ for event, spec in CATALOGUE.items()
+ if DEPLOYMENT_KIND_PROPERTY in spec
+ ]
+
async def test_an_unknown_version_is_omitted_rather_than_empty(self) -> None:
"""An absent attribute reads as "not known"; an empty string reads as a
real version named nothing."""
@@ -152,6 +202,7 @@ class TestTagging:
version=None,
telemetry_environment="prod",
telemetry_internal=False,
+ deployment_kind="self_hosted",
)
service.emit("deployment_started", tenant_count=1)
diff --git a/core/tests/switch_core/telemetry/test_snapshot_people_and_bots.py b/core/tests/switch_core/telemetry/test_snapshot_people_and_bots.py
new file mode 100644
index 000000000..3dafc486c
--- /dev/null
+++ b/core/tests/switch_core/telemetry/test_snapshot_people_and_bots.py
@@ -0,0 +1,207 @@
+"""People, bots and accounts are counted apart.
+
+A messaging platform relays an integration's posts — an alerting app, CI,
+another Switch server's agent — through the same kind of chat identity a person
+gets. `chat_identity_active_*` counted both; `person_identity_active_*` and
+`bot_identity_active_*` split them on the mark the bridge puts on an app's post,
+and `account_active_*` is `user_active_*` under the name it keeps.
+"""
+
+from __future__ import annotations
+
+import uuid
+from datetime import UTC, datetime, timedelta
+
+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
+
+from switch_core.clients.admin_messages import SENDER_IS_APP_MARKER
+from switch_core.db.models import (
+ TENANT_ZERO_ID,
+ Client,
+ ClientRoom,
+ CollaborationBridge,
+ ExternalUser,
+ ExternalUserClaim,
+ Message,
+ Room,
+ User,
+)
+from switch_core.db.session_scope import tenant_session
+from switch_core.telemetry.snapshot import collect_usage
+
+
+async def _identity_posts(
+ session_factory: async_sessionmaker[AsyncSession],
+ *,
+ posts: list[tuple[bool, timedelta]],
+ claimed_by: str | None = None,
+ agent_in_room: bool = True,
+) -> None:
+ """One chat identity posts in a room of its own. Each post is `(as_app,
+ age)`: whether the bridge marked it as an app's, and how long ago."""
+ suffix = uuid.uuid4().hex[:8]
+ async with tenant_session(session_factory, TENANT_ZERO_ID) as session:
+ bridge_client = Client(
+ transport_user_id=f"@bridge-{suffix}:test",
+ display_name="bridge",
+ type="bridge",
+ )
+ identity = Client(
+ transport_user_id=f"@identity-{suffix}:test",
+ display_name="identity",
+ type="user",
+ )
+ agent = Client(
+ transport_user_id=f"@agent-{suffix}:test",
+ display_name="agent",
+ type="agent",
+ )
+ room = Room(
+ transport_room_id=f"!{suffix}:test",
+ name=f"room-{suffix}",
+ description="",
+ metadata_={"created_by_kind": "user"},
+ )
+ session.add_all([bridge_client, identity, agent, room])
+ await session.flush()
+ bridge = CollaborationBridge(
+ type="slack",
+ display_name="Slack",
+ client_id=bridge_client.id,
+ status="active",
+ )
+ session.add(bridge)
+ await session.flush()
+ chat_account = ExternalUser(
+ bridge_id=bridge.id,
+ external_user_id=f"U{suffix}",
+ external_username=f"identity-{suffix}",
+ client_id=identity.id,
+ )
+ session.add(chat_account)
+ if agent_in_room:
+ session.add(ClientRoom(client_id=agent.id, room_id=room.id))
+ await session.flush()
+ if claimed_by is not None:
+ session.add(
+ ExternalUserClaim(external_user_id=chat_account.id, user_id=claimed_by)
+ )
+ for seq, (as_app, age) in enumerate(posts, start=1):
+ content: dict[str, object] = {"body": "hello"}
+ if as_app:
+ content[SENDER_IS_APP_MARKER] = True
+ session.add(
+ Message(
+ room_id=room.id,
+ seq=seq,
+ transport_event_id=f"$evt-{suffix}-{seq}",
+ sender_id=identity.transport_user_id,
+ sender_client_id=identity.id,
+ event_type="m.room.message",
+ msgtype="m.text",
+ body="hello",
+ content=content,
+ sent_at=datetime.now(UTC) - age,
+ )
+ )
+ await session.commit()
+
+
+async def _account(session_factory: async_sessionmaker[AsyncSession]) -> str:
+ async with session_factory() as session:
+ user = User(
+ name="someone", email=f"{uuid.uuid4().hex[:6]}@example.com", role="user"
+ )
+ session.add(user)
+ await session.commit()
+ return user.id
+
+
+HOUR = timedelta(hours=1)
+THREE_DAYS = timedelta(days=3)
+
+
+async def test_a_person_and_an_integration_are_one_of_each(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
+ await _identity_posts(session_factory, posts=[(False, HOUR)])
+ await _identity_posts(session_factory, posts=[(True, HOUR)])
+
+ counts = await collect_usage(session_factory)
+
+ assert counts.chat_identity_active_1d == 2
+ assert counts.person_identity_active_1d == 1
+ assert counts.bot_identity_active_1d == 1
+ assert counts.person_identity_active_7d == 1
+ assert counts.bot_identity_active_7d == 1
+
+
+async def test_an_integration_with_unmarked_older_posts_is_still_a_bot(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
+ """Posts relayed before its bridge marked app posts carry no mark. One marked
+ post in the window makes the whole identity a bot, not a person as well."""
+ await _identity_posts(session_factory, posts=[(False, THREE_DAYS), (True, HOUR)])
+
+ counts = await collect_usage(session_factory)
+
+ assert counts.person_identity_active_7d == 0
+ assert counts.bot_identity_active_7d == 1
+ assert counts.chat_identity_active_7d == 1
+
+
+async def test_the_day_and_the_week_are_judged_on_their_own_posts(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
+ """An identity whose only marked post is days old is a bot for the week and,
+ with an unmarked post today, a person for the day: each window decides from
+ what it can see."""
+ await _identity_posts(session_factory, posts=[(True, THREE_DAYS), (False, HOUR)])
+
+ counts = await collect_usage(session_factory)
+
+ assert counts.person_identity_active_1d == 1
+ assert counts.bot_identity_active_1d == 0
+ assert counts.person_identity_active_7d == 0
+ assert counts.bot_identity_active_7d == 1
+
+
+async def test_neither_counts_without_an_agent_in_the_room(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
+ await _identity_posts(session_factory, posts=[(False, HOUR)], agent_in_room=False)
+ await _identity_posts(session_factory, posts=[(True, HOUR)], agent_in_room=False)
+
+ counts = await collect_usage(session_factory)
+
+ assert counts.person_identity_active_7d == 0
+ assert counts.bot_identity_active_7d == 0
+
+
+async def test_account_active_is_user_active_under_its_new_name(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
+ someone = await _account(session_factory)
+ await _identity_posts(session_factory, posts=[(False, HOUR)], claimed_by=someone)
+ await _identity_posts(session_factory, posts=[(False, THREE_DAYS)])
+
+ counts = await collect_usage(session_factory)
+
+ assert counts.account_active_1d == counts.user_active_1d == 1
+ assert counts.account_active_7d == counts.user_active_7d == 1
+ assert counts.person_identity_active_7d == 2
+
+
+async def test_the_snapshot_event_carries_all_three(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
+ await _identity_posts(session_factory, posts=[(False, HOUR)])
+ await _identity_posts(session_factory, posts=[(True, HOUR)])
+
+ properties = (await collect_usage(session_factory)).as_event_properties(
+ session_live_count=0
+ )
+
+ assert properties["person_identity_active_7d"] == 1
+ assert properties["bot_identity_active_7d"] == 1
+ assert properties["account_active_7d"] == 0
diff --git a/core/tests/switch_core/telemetry/test_superseded_milestones.py b/core/tests/switch_core/telemetry/test_superseded_milestones.py
new file mode 100644
index 000000000..4a6902441
--- /dev/null
+++ b/core/tests/switch_core/telemetry/test_superseded_milestones.py
@@ -0,0 +1,262 @@
+"""Milestones added after deployments were already running.
+
+`first_agent_connected` renames what `first_session_started` always measured,
+and `first_agent_session` is the first real coding-agent session. A deployment
+that passed either before it existed must not report it now, months in, as if
+it were its first: at boot it claims them without reporting. A new deployment
+has nothing to settle and reports each the first time it happens.
+
+Against real Postgres, because the claims are rows and the guard is the
+primary key.
+"""
+
+from __future__ import annotations
+
+import uuid
+from datetime import UTC, datetime, timedelta
+from types import SimpleNamespace
+
+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
+
+from switch_core.bridges.agent.api.session_reporter import SessionReporter
+from switch_core.bridges.agent.protocol.agent_connections import (
+ AgentConnectionRegistry,
+ ClientDeclaration,
+)
+from switch_core.bridges.agent.protocol.positions import OWN_EVENTS
+from switch_core.db.models import Agent
+from switch_core.telemetry.deployment import (
+ SESSION_START_CLAIM_PREFIX,
+ claim_milestone,
+ milestone_claimed,
+ settle_superseded_milestones,
+)
+from switch_core.telemetry.service import TelemetryService
+from switch_core.telemetry.session_start import (
+ default_session_start_limiter,
+ report_session_started,
+)
+from switch_core.telemetry.sink import TelemetryRecord
+
+
+class _RecordingSink:
+ def __init__(self) -> None:
+ self.sent: list[TelemetryRecord] = []
+
+ def send(self, record: TelemetryRecord) -> None:
+ self.sent.append(record)
+
+ async def aclose(self) -> None:
+ return None
+
+
+def _service(
+ sink: _RecordingSink,
+ session_factory: async_sessionmaker[AsyncSession],
+ *,
+ enabled: bool = True,
+) -> TelemetryService:
+ return TelemetryService(
+ sink=sink, # type: ignore[arg-type]
+ enabled=enabled,
+ client_id="deployment-uuid",
+ version="1.0.0",
+ telemetry_environment="prod",
+ telemetry_internal=False,
+ deployment_kind="self_hosted",
+ session_factory=session_factory,
+ installed_at=datetime.now(UTC) - timedelta(days=1),
+ )
+
+
+def _agent() -> Agent:
+ """The reporter reads only the agent's id and runtime, so it need not be a
+ stored row."""
+ return Agent(id=str(uuid.uuid4()), name="agent", metadata_={})
+
+
+class TestSettling:
+ async def test_a_new_deployment_has_nothing_to_settle(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ assert await settle_superseded_milestones(session_factory) == []
+ assert await milestone_claimed(session_factory, "first_agent_connected") == (
+ "false"
+ )
+ assert await milestone_claimed(session_factory, "first_agent_session") == (
+ "false"
+ )
+
+ async def test_a_deployment_that_came_online_before_settles_the_new_name(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ await claim_milestone(session_factory, "first_session_started")
+
+ assert await settle_superseded_milestones(session_factory) == [
+ "first_agent_connected"
+ ]
+ assert await milestone_claimed(session_factory, "first_agent_connected") == (
+ "true"
+ )
+
+ async def test_a_room_used_with_an_agent_means_sessions_already_ran(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ await claim_milestone(session_factory, "first_room_active")
+
+ assert await settle_superseded_milestones(session_factory) == [
+ "first_agent_session"
+ ]
+
+ async def test_a_session_reported_this_week_means_sessions_already_ran(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ await claim_milestone(
+ session_factory, f"{SESSION_START_CLAIM_PREFIX}agent-1:session-1"
+ )
+
+ assert await settle_superseded_milestones(session_factory) == [
+ "first_agent_session"
+ ]
+
+ async def test_settling_twice_claims_nothing_the_second_time(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ await claim_milestone(session_factory, "first_session_started")
+ await claim_milestone(session_factory, "first_room_active")
+
+ assert sorted(await settle_superseded_milestones(session_factory)) == [
+ "first_agent_connected",
+ "first_agent_session",
+ ]
+ assert await settle_superseded_milestones(session_factory) == []
+
+ async def test_evidence_written_after_the_first_settling_is_not_history(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ """A process booting while another takes the deployment's first
+ session must not read that session's claim as history and claim the
+ milestone before its report does."""
+ assert await settle_superseded_milestones(session_factory) == []
+ await claim_milestone(
+ session_factory, f"{SESSION_START_CLAIM_PREFIX}agent-1:session-1"
+ )
+
+ assert await settle_superseded_milestones(session_factory) == []
+ assert await milestone_claimed(session_factory, "first_agent_session") == (
+ "false"
+ )
+
+ async def test_a_settled_milestone_is_never_reported(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ await claim_milestone(session_factory, "first_session_started")
+ sink = _RecordingSink()
+ service = _service(sink, session_factory)
+
+ await service.settle_superseded_milestones()
+ await service.emit_milestone("first_agent_connected", known_agent_type="none")
+ await service.aclose()
+
+ assert sink.sent == []
+
+ async def test_nothing_is_settled_while_telemetry_is_off(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ """For the reason `emit_milestone` claims nothing while off: switching
+ it on later must find the right claims, not ones taken in the dark."""
+ await claim_milestone(session_factory, "first_session_started")
+ off = _service(_RecordingSink(), session_factory, enabled=False)
+
+ await off.settle_superseded_milestones()
+
+ assert await milestone_claimed(session_factory, "first_agent_connected") == (
+ "false"
+ )
+
+
+class TestTheFirstRealSession:
+ async def test_the_first_reported_session_is_the_milestone_and_only_once(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ agent = _agent()
+ sink = _RecordingSink()
+ service = _service(sink, session_factory)
+ limiter = default_session_start_limiter()
+
+ for session_id in (str(uuid.uuid4()), str(uuid.uuid4())):
+ await report_session_started(
+ service, session_factory, limiter, agent, session_id, "room"
+ )
+ await service.aclose()
+
+ names = [record.name for record in sink.sent]
+ assert names == [
+ "switch_core.session_started",
+ "switch_core.first_agent_session",
+ "switch_core.session_started",
+ ]
+ milestone = sink.sent[1]
+ assert milestone.properties["start_source"] == "room"
+ assert "seconds_since_install" in milestone.properties
+
+ async def test_a_deployment_that_already_had_sessions_does_not_report_one(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ await claim_milestone(session_factory, "first_room_active")
+ agent = _agent()
+ sink = _RecordingSink()
+ service = _service(sink, session_factory)
+
+ await service.settle_superseded_milestones()
+ await report_session_started(
+ service,
+ session_factory,
+ default_session_start_limiter(),
+ agent,
+ str(uuid.uuid4()),
+ "user",
+ )
+ await service.aclose()
+
+ assert [record.name for record in sink.sent] == ["switch_core.session_started"]
+
+
+class TestComingOnline:
+ async def test_the_first_connection_reports_both_names_once(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ """Until `first_session_started` is retired, a new deployment reports the
+ moment under both names, and each only once."""
+ registry = AgentConnectionRegistry()
+ sink = _RecordingSink()
+ service = _service(sink, session_factory)
+ reporter = SessionReporter(service, registry)
+ registry.set_close_listener(reporter.on_close)
+
+ for agent_id in ("agent-1", "agent-2"):
+ connection_id = uuid.uuid4().hex
+ registry.open(
+ agent_id=agent_id,
+ connection_id=connection_id,
+ scope="all",
+ delivery_filter="all",
+ spawn_capable=False,
+ position={OWN_EVENTS: 0},
+ declaration=ClientDeclaration(),
+ expected_generation=None,
+ )
+ connection = registry.get(connection_id)
+ assert connection is not None
+ await reporter.started(
+ SimpleNamespace(metadata_={"known_agent_type": "codex"}), # type: ignore[arg-type]
+ connection,
+ )
+ await service.aclose()
+
+ milestones = [
+ record.name.removeprefix("switch_core.")
+ for record in sink.sent
+ if record.name.removeprefix("switch_core.").startswith("first_")
+ ]
+ assert milestones == ["first_session_started", "first_agent_connected"]
diff --git a/core/tests/switch_core/telemetry/test_usage_periods.py b/core/tests/switch_core/telemetry/test_usage_periods.py
new file mode 100644
index 000000000..e37ac3320
--- /dev/null
+++ b/core/tests/switch_core/telemetry/test_usage_periods.py
@@ -0,0 +1,278 @@
+"""Usage counted per finished day, week and month (`usage_period`).
+
+The snapshot's trailing `*_7d` counts add up wrong over a week: each room is
+counted on every day it was in the window. A period is counted once, over
+exactly its own span, and each pass reports the periods that ended in its
+window, so successive passes report every period once.
+
+The counting is against real Postgres, because it is SQL; the period
+arithmetic needs none.
+"""
+
+from __future__ import annotations
+
+import uuid
+from datetime import UTC, datetime, timedelta
+
+from sqlalchemy import delete
+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
+
+from switch_core.clients.admin_messages import SENDER_IS_APP_MARKER
+from switch_core.db.models import (
+ TENANT_ZERO_ID,
+ Client,
+ ClientRoom,
+ Message,
+ Room,
+ TelemetrySnapshotWatermark,
+)
+from switch_core.db.session_scope import tenant_session
+from switch_core.telemetry.periods import BACKFILL, Period, periods_ending_between
+from switch_core.telemetry.reporter import SnapshotReporter
+from switch_core.telemetry.service import TelemetryService
+from switch_core.telemetry.sink import TelemetryRecord
+from switch_core.telemetry.snapshot import collect_period_counts
+
+
+def _at(text: str) -> datetime:
+ return datetime.fromisoformat(text).replace(tzinfo=UTC)
+
+
+def _spans(periods: list[Period]) -> list[tuple[str, str, str]]:
+ return [
+ (period.kind, period.start.date().isoformat(), period.end.date().isoformat())
+ for period in periods
+ ]
+
+
+class TestWhichPeriodsAPassReports:
+ def test_an_ordinary_day_is_reported_alone(self) -> None:
+ # Wednesday to Thursday: no week or month ends in between.
+ periods = periods_ending_between(
+ _at("2026-11-04T14:00:00"), _at("2026-11-05T14:00:00")
+ )
+
+ assert _spans(periods) == [("day", "2026-11-04", "2026-11-05")]
+
+ def test_a_window_over_a_monday_and_the_first_reports_the_week_and_month(
+ self,
+ ) -> None:
+ periods = periods_ending_between(
+ _at("2026-11-29T10:00:00"), _at("2026-12-01T10:00:00")
+ )
+
+ assert _spans(periods) == [
+ ("day", "2026-11-29", "2026-11-30"),
+ ("day", "2026-11-30", "2026-12-01"),
+ ("week", "2026-11-23", "2026-11-30"),
+ ("month", "2026-11-01", "2026-12-01"),
+ ]
+
+ def test_successive_passes_report_every_period_once(self) -> None:
+ """Passes are a day apart give or take, as the watermark keeps them,
+ across a year boundary and a short month."""
+ start = _at("2026-12-20T09:13:00")
+ moments = [start]
+ for step in range(80):
+ moments.append(moments[-1] + timedelta(hours=24, minutes=7 * (step % 5)))
+
+ reported: list[Period] = []
+ for since, now in zip(moments, moments[1:], strict=False):
+ reported.extend(periods_ending_between(since, now))
+
+ assert len(reported) == len(set(reported))
+ days = [period for period in reported if period.kind == "day"]
+ assert all(
+ later.start == earlier.end
+ for earlier, later in zip(days, days[1:], strict=False)
+ )
+ weeks = [period for period in reported if period.kind == "week"]
+ assert all(week.start.weekday() == 0 for week in weeks)
+ assert all(
+ later.start == earlier.end
+ for earlier, later in zip(weeks, weeks[1:], strict=False)
+ )
+ assert _spans([period for period in reported if period.kind == "month"]) == [
+ ("month", "2026-12-01", "2027-01-01"),
+ ("month", "2027-01-01", "2027-02-01"),
+ ("month", "2027-02-01", "2027-03-01"),
+ ]
+
+ def test_a_long_gap_reports_only_what_ended_within_the_backfill(self) -> None:
+ now = _at("2026-11-03T12:00:00")
+
+ periods = periods_ending_between(now - timedelta(days=200), now)
+
+ assert periods
+ assert min(period.end for period in periods) > now - BACKFILL
+ assert ("month", "2026-10-01", "2026-11-01") in _spans(periods)
+
+ def test_each_is_dated_to_its_middle(self) -> None:
+ day, week, month = periods_ending_between(
+ _at("2026-11-29T10:00:00"), _at("2026-12-01T10:00:00")
+ )[1:]
+
+ assert day.midpoint == _at("2026-11-30T12:00:00")
+ # Thursday noon, inside the same week whether weeks start on Sunday or
+ # Monday, in any timezone.
+ assert week.midpoint == _at("2026-11-26T12:00:00")
+ assert month.midpoint == _at("2026-11-16T00:00:00")
+
+
+async def _room_with_posts(
+ session_factory: async_sessionmaker[AsyncSession],
+ *,
+ posts: list[tuple[datetime, bool]],
+) -> None:
+ """A room with an agent in it, where one chat identity posts at each
+ `(sent_at, as_app)`."""
+ suffix = uuid.uuid4().hex[:8]
+ async with tenant_session(session_factory, TENANT_ZERO_ID) as session:
+ identity = Client(
+ transport_user_id=f"@identity-{suffix}:test",
+ display_name="identity",
+ type="user",
+ )
+ agent = Client(
+ transport_user_id=f"@agent-{suffix}:test",
+ display_name="agent",
+ type="agent",
+ )
+ room = Room(
+ transport_room_id=f"!{suffix}:test",
+ name=f"room-{suffix}",
+ description="",
+ metadata_={"created_by_kind": "user"},
+ )
+ session.add_all([identity, agent, room])
+ await session.flush()
+ session.add(ClientRoom(client_id=agent.id, room_id=room.id))
+ for seq, (sent_at, as_app) in enumerate(posts, start=1):
+ content: dict[str, object] = {"body": "hello"}
+ if as_app:
+ content[SENDER_IS_APP_MARKER] = True
+ session.add(
+ Message(
+ room_id=room.id,
+ seq=seq,
+ transport_event_id=f"$evt-{suffix}-{seq}",
+ sender_id=identity.transport_user_id,
+ sender_client_id=identity.id,
+ event_type="m.room.message",
+ msgtype="m.text",
+ body="hello",
+ content=content,
+ sent_at=sent_at,
+ )
+ )
+ await session.commit()
+
+
+class TestCountingAPeriod:
+ async def test_only_what_happened_inside_the_period_counts(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ start, end = _at("2026-11-02T00:00:00"), _at("2026-11-09T00:00:00")
+ await _room_with_posts(session_factory, posts=[(start, False)])
+ await _room_with_posts(
+ session_factory, posts=[(end - timedelta(microseconds=1), False)]
+ )
+ await _room_with_posts(
+ session_factory, posts=[(start - timedelta(seconds=1), False)]
+ )
+ await _room_with_posts(session_factory, posts=[(end, False)])
+
+ counts = await collect_period_counts(session_factory, start=start, end=end)
+
+ assert counts.room_active == 2
+ assert counts.person_identity_active == 2
+ assert counts.tenant_failed_count == 0
+
+ async def test_a_bot_uses_a_room_but_is_not_a_person(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ start, end = _at("2026-11-02T00:00:00"), _at("2026-11-03T00:00:00")
+ await _room_with_posts(
+ session_factory, posts=[(start + timedelta(hours=1), True)]
+ )
+
+ counts = await collect_period_counts(session_factory, start=start, end=end)
+
+ assert counts.room_active == 1
+ assert counts.person_identity_active == 0
+
+
+class _RecordingSink:
+ def __init__(self) -> None:
+ self.sent: list[TelemetryRecord] = []
+
+ def send(self, record: TelemetryRecord) -> None:
+ self.sent.append(record)
+
+ async def aclose(self) -> None:
+ return None
+
+
+def _reporter(
+ sink: _RecordingSink, session_factory: async_sessionmaker[AsyncSession]
+) -> SnapshotReporter:
+ service = TelemetryService(
+ sink=sink, # type: ignore[arg-type]
+ enabled=True,
+ client_id="deployment-uuid",
+ version="1.0.0",
+ telemetry_environment="prod",
+ telemetry_internal=False,
+ deployment_kind="self_hosted",
+ session_factory=session_factory,
+ installed_at=datetime.now(UTC) - timedelta(days=30),
+ )
+ return SnapshotReporter(
+ telemetry=service,
+ session_factory=session_factory,
+ interval_hours=24.0,
+ installed_at=service.installed_at,
+ live_session_count=lambda: 0,
+ )
+
+
+def _periods_sent(sink: _RecordingSink) -> list[TelemetryRecord]:
+ return [record for record in sink.sent if record.name == "switch_core.usage_period"]
+
+
+class TestThePassReportsThem:
+ async def test_each_finished_period_is_sent_dated_to_its_middle(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ await _room_with_posts(
+ session_factory,
+ posts=[(_at("2026-11-30T09:00:00"), False)],
+ )
+ sink = _RecordingSink()
+
+ await _reporter(sink, session_factory).run_once(
+ now=_at("2026-12-01T10:00:00"), since=_at("2026-11-30T10:00:00")
+ )
+
+ sent = _periods_sent(sink)
+ assert [record.properties["period"] for record in sent] == ["day", "month"]
+ day, month = sent
+ assert day.timestamp_ns == int(_at("2026-11-30T12:00:00").timestamp()) * 10**9
+ assert day.properties["room_active"] == 1
+ assert day.properties["person_identity_active"] == 1
+ assert month.timestamp_ns == int(_at("2026-11-16T00:00:00").timestamp()) * 10**9
+ assert month.properties["room_active"] == 1
+
+ async def test_the_first_pass_sends_none(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ """With no watermark there is no telling which periods were sent."""
+ async with session_factory() as session:
+ await session.execute(delete(TelemetrySnapshotWatermark))
+ await session.commit()
+ sink = _RecordingSink()
+
+ assert await _reporter(sink, session_factory).run_once_if_due() is True
+
+ assert "switch_core.usage_snapshot" in [record.name for record in sink.sent]
+ assert _periods_sent(sink) == []
diff --git a/core/tests/switch_core/test_config_feature_flags.py b/core/tests/switch_core/test_config_feature_flags.py
index 2082572ff..7199c8572 100644
--- a/core/tests/switch_core/test_config_feature_flags.py
+++ b/core/tests/switch_core/test_config_feature_flags.py
@@ -43,3 +43,21 @@ def test_an_unknown_flag_is_a_startup_error(monkeypatch: pytest.MonkeyPatch) ->
monkeypatch.setenv("FEATURE_FLAGS_ENABLED", "ecosystem.show_ownres")
with pytest.raises(ValidationError, match="ecosystem.show_ownres"):
SwitchConfig(**_BASE_KWARGS) # type: ignore[arg-type]
+
+
+ ("flags", "managed_by_console", "kind"),
+ [
+ ("", False, "self_hosted"),
+ ("", True, "managed_by_console"),
+ ("switch_cloud", False, "switch_cloud"),
+ # The Cloud flag decides even if the managed setting is stray.
+ ("switch_cloud", True, "switch_cloud"),
+ ],
+)
+def test_the_deployment_kind_follows_the_cloud_flag_then_console(
+ monkeypatch: pytest.MonkeyPatch, flags: str, managed_by_console: bool, kind: str
+) -> None:
+ monkeypatch.setenv("FEATURE_FLAGS_ENABLED", flags)
+ config = SwitchConfig(**_BASE_KWARGS, managed_by_console=managed_by_console) # type: ignore[arg-type]
+ assert config.deployment_kind == kind
diff --git a/core/tests/switch_core/transport/fake.py b/core/tests/switch_core/transport/fake.py
index d385e09df..e338c7eab 100644
--- a/core/tests/switch_core/transport/fake.py
+++ b/core/tests/switch_core/transport/fake.py
@@ -147,6 +147,7 @@ class FakeTransport:
caption: str | None = None,
thread_root_id: str | None = None,
group: dict[str, object] | None = None,
+ extra_content: dict[str, object] | None = None,
) -> SendResult:
if self._fail_send:
raise TransportError(self._fail_send)
@@ -163,6 +164,7 @@ class FakeTransport:
"caption": caption,
"thread_root_id": thread_root_id,
"group": group,
+ "extra_content": extra_content,
}
)
content: dict[str, object] = {
@@ -172,6 +174,8 @@ class FakeTransport:
"info": {"mimetype": mimetype, "size": size},
"sender_name": sender_name,
}
+ if extra_content:
+ content.update(extra_content)
if caption:
content["filename"] = filename
if group is not None:
diff --git a/core/tests/switch_core/transport/test_postgres_transport.py b/core/tests/switch_core/transport/test_postgres_transport.py
index ab61a627e..228d0ca0a 100644
--- a/core/tests/switch_core/transport/test_postgres_transport.py
+++ b/core/tests/switch_core/transport/test_postgres_transport.py
@@ -24,6 +24,7 @@ from sqlalchemy import delete, select
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
+from switch_core.clients.admin_messages import SENDER_IS_APP_MARKER
from switch_core.db.models import (
TENANT_ZERO_ID,
Client,
@@ -410,11 +411,52 @@ class TestTheParticipantMessageObserver:
room_id=room_id,
sender_client_id=client_id,
sender_role="human",
+ sender_is_app=False,
has_attachment=False,
in_thread=False,
)
]
+ async def test_an_app_post_and_its_files_say_so(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ """The bridge's mark rides on the message and on a file sent alone, so
+ an integration's post is told from a person's whichever it is."""
+ async with session_factory() as session:
+ room_id, transport_room_id, client_id, user_id = await _make_room(session)
+ await session.commit()
+ recorder = _Recorder()
+ transport = _transport(
+ session_factory,
+ client_id=client_id,
+ user_id=user_id,
+ actor_role="human",
+ message_observer=recorder,
+ )
+ marker = {SENDER_IS_APP_MARKER: True}
+
+ await transport.send_message(
+ transport_room_id,
+ "deploy finished",
+ sender_name="ci",
+ metered=True,
+ extra_content=marker,
+ )
+ await transport.send_media(
+ transport_room_id,
+ "switch-media://log",
+ "build.log",
+ "text/plain",
+ 3,
+ sender_name="ci",
+ metered=True,
+ msgtype="m.file",
+ extra_content=marker,
+ )
+
+ assert [m.sender_is_app for m in recorder.seen] == [True, True]
+ assert [m.has_attachment for m in recorder.seen] == [False, True]
+
async def test_a_reply_in_a_thread_says_so(
self, session_factory: async_sessionmaker[AsyncSession]
) -> None:
diff --git a/core/tests/telemetry_records.py b/core/tests/telemetry_records.py
new file mode 100644
index 000000000..51412f6d7
--- /dev/null
+++ b/core/tests/telemetry_records.py
@@ -0,0 +1,54 @@
+"""Reading what a test's telemetry sink was handed."""
+
+from __future__ import annotations
+
+from switch_core.telemetry.catalogue import DEPLOYMENT_KIND_PROPERTY, PropertyValue
+from switch_core.telemetry.service import TelemetryService
+from switch_core.telemetry.sink import TelemetryRecord
+
+
+def declared(record: TelemetryRecord) -> dict[str, PropertyValue]:
+ """The properties the event itself declared, without the one the service
+ adds to every event (`DEPLOYMENT_KIND_PROPERTY`), so a test can compare an
+ event's properties exactly."""
+ return {
+ name: value
+ for name, value in record.properties.items()
+ if name != DEPLOYMENT_KIND_PROPERTY
+ }
+
+
+class RecordingSink:
+ """A sink that keeps what it is handed."""
+
+ def __init__(self) -> None:
+ self.sent: list[TelemetryRecord] = []
+
+ def send(self, record: TelemetryRecord) -> None:
+ self.sent.append(record)
+
+ async def aclose(self) -> None:
+ return None
+
+ def events(self, name: str) -> list[dict[str, PropertyValue]]:
+ """The declared properties of every `name` event sent, in order."""
+ return [
+ declared(record)
+ for record in self.sent
+ if record.name == f"switch_core.{name}"
+ ]
+
+
+def recording_telemetry() -> tuple[TelemetryService, RecordingSink]:
+ """An enabled telemetry service over a `RecordingSink`."""
+ sink = RecordingSink()
+ service = TelemetryService(
+ sink=sink,
+ enabled=True,
+ client_id="deployment-uuid",
+ version="1.0.0",
+ telemetry_environment="prod",
+ telemetry_internal=False,
+ deployment_kind="switch_cloud",
+ )
+ return service, sink
diff --git a/docs/TELEMETRY.md b/docs/TELEMETRY.md
index d9149d96d..8cbd9cc83 100644
--- a/docs/TELEMETRY.md
+++ b/docs/TELEMETRY.md
@@ -75,7 +75,7 @@ machine, in what repository, or what the prompt said.
| `service.version` | app version | `0.9.14`, `1.0.2` |
| `build` | release channel | `dev`, `canary`, `stable` |
| `flint_env` | which analytics project the event is filed under | `prod` (a released stable build), `staging` (a released canary), `dev` (a packaged build that is not a release), `local` (running from source) |
-| `flint_internal` | whether an account this app is signed in to a Switch server with has a SandboxAQ email address — **only the yes/no is sent, never the address** | `true`, `false`, `unknown` (no account, or only the one a local server signs in with) |
+| `flint_internal` | whether an account this app is signed in to a Switch server with has a SandboxAQ email address — **only the yes/no is sent, never the address**. Once an installation has been signed in with one, it stays `true`, so staff usage stays labelled after that account signs out | `true`, `false`, `unknown` (no account, or only the one a local server signs in with) |
| `os.type` | OS family | `darwin`, `windows`, `linux`, `other` |
| `os.version` | OS release string | `23.6.0`, `10.0.22631`, `6.1.0-53-cloud-amd64` |
| `flint.client_id` | random install UUID | `3f2a9c41-8d7e-4b16-9a55-c0e1d2f47b83` |
@@ -92,7 +92,7 @@ id, no IP field, no Switch identity.
| `outcome` | `success`, `failure` (on `session_ended`: `normal`, `failed`; on `bridge_identity_claimed`, also `pending`) |
| `agent_type` | the AI provider, from a fixed registry: `claude`, `codex`, `gemini`, `cursor`, `copilot`, `opencode`, `grok`, `devin`, `qwen`, `droid`, `amp`, `goose`, `cline`, `continue`, `mistral`, `kiro`, `junie`, … and `unknown`. **Never an agent's name.** |
| `location` | `local`, `remote`, `unknown`. **Never a path, directory or project name.** |
-| `server_kind` | `local`, `remote_managed`, `external`. **Never a server name or URL.** |
+| `server_kind` | `local`, `remote_managed`, `external`, `cloud` (Switch Cloud, the server this build offers by name). **Never a server name or URL.** |
| `bridge_platform` | A platform key the server lists as one of its registered platforms (`slack`, `mattermost`, `discord`, `teams`, `telegram`, and any platform added since), else `other`, or `unknown` when it cannot be read. Always a short lowercase key. **Never a workspace or channel name.** |
| `entry_point` | `command_palette`, `sidebar`, `server_page`, `onboarding`, `agent_page`, `session_list`, `room_row`, `link`, `template`, `switch_buddy`, `unknown` |
| `target` | `local`, `remote`, `unknown` |
@@ -106,8 +106,10 @@ id, no IP field, no Switch identity.
| `app_launched` | `install_kind`: `new` (the installation's very first launch) / `updated` (the first launch after its version changed) / `same` |
| `renderer_crashed` | *(no fields)* |
| `update_checked` | `trigger`: `user` / `startup` / `scheduled` · `result`: `available` / `up_to_date` / `failed` |
+| `update_download_started` | *(no fields)* — Download was pressed for an available update |
| `update_downloaded` | `outcome`: `success` / `failure` |
| `update_install_started` | `outcome`: `success` / `failure` |
+| `update_installed` | `from_version`: `0.40.1`, or `unknown` when the previous launch did not record one · `trigger`: `restart_button` (the app restarted itself to install it) / `other` (installed on quit, or by hand) — sent on the first launch after the version changed |
| `telemetry_consent_changed` | `source`: `first_run` / `settings` · `install_kind`: as on `app_launched`, or `unknown` if the launch could not be recorded |
| `setting_changed` | `setting_key`, one of exactly 12: `localLocation`, `sessions`, `defaultAgent`, `notifications`, `theme`, `openIn`, `interface`, `browserPreview`, `browser`, `changesViewMode`, `onboarding`, `telemetry`. **The new value is never sent** — we learn that someone changed their theme, not to what. |
| `search_performed` | `status`: `ok` / `recents` / `query-too-short` / `failed` · `result_count`: `0`, `3`, `17`. **The query is never sent.** |
@@ -133,18 +135,37 @@ id, no IP field, no Switch identity.
|---|---|
| `agent_created` | `agent_type`: `codex` · `location`: `remote` · `outcome`: `failure` · `failure_reason`: `none` / `unauthenticated` / `name_conflict` / `credentials_conflict` / `already_configured` / `invalid_name` / `directory_unusable` / `not_configured` / `agent_not_on_server` / `refused` / `invalid` / `error` · `entry_point`: `sidebar` |
| `agent_removed` | `agent_type` · `location` · `delete_in_switch`: `true` · `trigger`: `user` / `server_teardown` · `outcome` · `failure_reason`: `none` / `not_linked_to_switch` / `gateway_unauthorized` / `gateway_http` / `gateway_network` / `error` |
-| `agent_reset` | `agent_type` · `outcome` · `failure_reason`: `none` / `agent_not_found` / `not_remote` / `connect` / `error` |
| `agent_cli_action` | `agent_type` · `target`: `local` / `remote` · `install_method`: `homebrew` / `npm` / `winget` / `powershell` / `apt` / `curl` / `pip` / `cargo` / `installer-macos` / `installer-windows` / `installer-linux` / `other` / `unspecified` · `action`: `install` / `update` / `uninstall` · `outcome` · `failure_reason`: `none` / `unknown_dependency` / `no_install_command` / `no_update_strategy` / `no_uninstall_strategy` / `no_uninstall_command` / `permission_denied` / `command_failed` / `process_open_failed` / `not_detected_after_install` / `not_detected_after_update` / `still_present` / `error` · `duration_ms`: `8421` |
| `session_started` | `agent_type`: `claude` · `location`: `local` · `outcome`: `success` · `failure_reason`: `none` / `agent_not_found` / `already_exists` / `spawn_failed` · `entry_point`: `command_palette` · `start_source`: `user` / `auto` / `adopted` / `unknown` · `has_initial_prompt`: `true` (**a boolean — never the prompt**) · `connected_to_room`: `false` |
| `session_ended` | `agent_type` · `location` · `outcome`: `normal` / `failed` |
| `session_attached` | `agent_type` · `outcome` |
| `session_provision_retried` | `agent_type` · `location` · `trigger`: `auto` / `retry_button` · `outcome` |
+| `managed_session_started` | `agent_type`: `claude` · `start_source`: `user` / `room` / `automation` / `unknown` — a session of an agent on the agent controller this app runs on this computer started |
+| `activation_milestone` | `milestone`: `server_ready` / `agent_ready` / `first_agent_session` · `path`: `buddy` / `manual` / `unknown` — a step towards an agent doing work, the first time this installation reaches it |
Note the shape of `failure_reason` everywhere: a short enumerated code such as
`permission_denied` or `docker_daemon_down`. It is **never** an exception message,
a stack trace, or a command's stderr — those are mapped to `error` if they don't
match a known code.
+`managed_session_started` counts sessions this app has no session of its own
+for: those of the agents it runs on the agent controller on this computer. That
+controller tells the app, over the channel between the two processes on this
+machine, once the agent's Switch server has taken the session's start, and the
+app reports it under the same consent setting as everything else. The session's
+id is used only to count a start told twice once, and goes no further. A
+controller on another machine — an SSH host, or one run on its own — sends
+nothing here, because it cannot ask this setting.
+
+`activation_milestone` marks three steps the first time an installation reaches
+each: `server_ready` (signed in to a server, or a server this app runs started),
+`agent_ready` (an agent created) and `first_agent_session` (a session of one of
+its agents started, here or on the controller above). Each is sent once for the
+life of the installation, and only by one that was installed on a version that
+sends them. `path` says which way it went at Meet switch-buddy: `buddy` if the
+buddy was started, `manual` if it was skipped, `unknown` before either; it is
+recorded on this machine and sent only on these events.
+
`duration_ms` on `agent_cli_action` is how long the operation took, in whole
milliseconds, measured on a monotonic clock around the operation itself. Like
`agent_count` and `result_count` it is a number rather than a value from a fixed
@@ -156,11 +177,11 @@ machine from another.
| Event | Fields, with example values |
|---|---|
-| `server_added` | `server_kind`: `remote_managed` · `outcome`: `success` |
+| `server_added` | `server_kind`: `remote_managed` · `outcome`: `success` · `failure_reason`: `none` / `already_added` (another server already has the address) / `not_the_server_address` (it answered with a web page, usually a dashboard's address) / `error` |
| `server_removed` | `server_kind`: `external` |
| `server_sign_in` | `auth_method`: `password` / `oidc` / `signup` · `server_kind` · `outcome` · `failure_reason`: `none` / `invalid_credentials` / `cancelled` / `failed` / `unreachable` / `email_taken` / `invalid` / `disabled` / `rate_limited` |
| `server_sign_out` | `server_kind`: `local` |
-| `managed_server_action` | `action`: `start` / `stop` / `reset` · `target`: `local` / `remote` · `outcome` · `failure_reason`: `none` / `docker_not_installed` / `docker_daemon_down` / `version_downgrade` / `matrix_migration_failed` / `error` · `docker_available`: `available` / `unavailable` / `unknown` |
+| `managed_server_action` | `action`: `start` / `stop` / `reset` · `target`: `local` / `remote` · `outcome` · `failure_reason`: `none` / `docker_not_installed` / `docker_daemon_down` / `version_downgrade` / `matrix_migration_failed` / `start_refused` (the stack already on the host cannot be started from here, such as another account's) / `health_timeout` (it started but did not become healthy in time) / `core_exited` (the server process exited while starting) / `running_as_root` / `operation_in_progress` (another start, stop or reset was running) / `error` · `docker_available`: `available` / `unavailable` / `unknown` |
No server name, URL, hostname or username appears in any of these. A failed
sign-in records `invalid_credentials` — not the username tried, not the server.
@@ -173,7 +194,7 @@ sign-in records `invalid_credentials` — not the username tried, not the server
| `bridge_disconnected` | `bridge`: `collaboration` · `bridge_platform`: `mattermost` · `outcome` |
| `bridge_identity_claimed` | `bridge_platform` · `outcome`: `success` (linked) / `pending` (waiting for a one-time code to be posted from the account, which is what confirms the link) / `failure` |
| `bridge_identity_confirmed` | `bridge_platform` · `link_kind`: `new` (a link that was waiting on its code) / `existing` (a link made before links had to be confirmed, which worked all along) / `unknown` — sent when the code is posted and the link confirmed, after asking the server that it is |
-| `room_created` | `server_kind`: `local` · `bridge_platform`: `slack` · `agent_count`: `3` · `has_instructions`: `true` · `outcome` · `failure_reason`: `none` / `unauthenticated` / `bridge_unavailable` / `invalid` / `unreachable` / `error` · `source`: `dialog` / `template` |
+| `room_created` | `server_kind`: `local` · `bridge_platform`: `slack` · `agent_count`: `3` · `has_instructions`: `true` · `outcome` · `failure_reason`: `none` / `unauthenticated` / `bridge_unavailable` / `invalid` / `unreachable` / `network` / `forbidden` / `not_found` / `conflict` / `server_error` / `error` · `source`: `dialog` / `template` |
| `room_deleted` | `server_kind` · `outcome` |
| `room_agents_added` | `agent_count`: `2` · `direction`: `agents_to_room` / `room_to_agents` |
diff --git a/docs/old/telemetry-events.md b/docs/old/telemetry-events.md
index 92fd1d9a3..9581a1f8b 100644
--- a/docs/old/telemetry-events.md
+++ b/docs/old/telemetry-events.md
@@ -222,6 +222,17 @@ On every event:
| `flint_env` | `prod` \| `staging` \| `dev` \| `local`, from `TELEMETRY_ENVIRONMENT` |
| `flint_internal` | `true` \| `false`, from `TELEMETRY_INTERNAL` |
+And one event property on every event, added by the service rather than
+declared by any event (no event may declare it):
+
+| Property | Value |
+|---|---|
+| `deployment_kind` | `switch_cloud` when the `switch_cloud` feature flag is on; else `managed_by_console` when Switch Console runs the server (`MANAGED_BY_CONSOLE`); else `self_hosted`. The same answer `get_my_context` gives an agent |
+
+A property rather than a resource attribute because the relay passes event
+properties to Amplitude and promotes only a fixed few resource attributes, so
+this is what lets a chart take Switch Cloud's traffic out, or look at it alone.
+
The rule against free text holds here too. `service.name` is a constant rather
than the `SERVICE_NAME` setting, and the `ENVIRONMENT` setting is not sent as
`deployment.environment`: both are free text an operator sets for their own log
@@ -235,6 +246,10 @@ the relay receives for it.
**A daily snapshot**, one per deployment, carrying the counts that describe the
installation. This answers "how much".
+**Period counts**, one per deployment per finished day, week and month, each
+counting distinct rooms and people over exactly that period. This answers "how
+many this week" in a way that adds up across deployments.
+
**Milestone events**, emitted at most once per deployment, each carrying the
seconds elapsed since install. This answers "how fast to value".
@@ -341,16 +356,22 @@ never invisible.
|---|---|---|
| `tenant_count` | number | tenants this pass actually counted — see `tenant_failed_count` |
| `tenant_failed_count` | number | tenants whose queries raised and were stepped over. One tenant's failure no longer takes the whole pass down, so the pair is what makes a partial pass self-describing: without it, every count dropping at once is indistinguishable from a deployment losing its users |
-| `duration_ms` | number | wall time to collect the pass, on a monotonic clock. Roughly fifteen queries per tenant against the database that is also serving rooms, and a background task is invisible to `switch.http.request.duration` — so this is the only place "what does the snapshot cost at scale" can be answered from |
+| `duration_ms` | number | wall time to collect the pass, on a monotonic clock. Roughly twenty queries per tenant against the database that is also serving rooms, and a background task is invisible to `switch.http.request.duration` — so this is the only place "what does the snapshot cost at scale" can be answered from |
| `user_count` | number | user accounts that exist |
| `user_internal_count` | number | of those, accounts on the company's own email domains — see "Internal usage" |
-| `user_active_1d` | number | distinct Switch **accounts** that used a room with an agent in 24h: an account counts when a chat account it has claimed spoke. One person on two platforms is one account |
-| `user_active_7d` | number | same over 7 days |
+| `user_active_1d` | number | distinct Switch **accounts** that used a room with an agent in 24h: an account counts when a chat account it has claimed spoke. One person on two platforms is one account. Superseded by `account_active_1d`, the same figure; sent for one more stable release |
+| `user_active_7d` | number | same over 7 days. Superseded by `account_active_7d` |
| `user_internal_active_1d` | number | of `user_active_1d`, accounts on the company's own email domains |
| `user_internal_active_7d` | number | same over 7 days |
| `chat_identity_count` | number | chat identities — Slack, Mattermost and other platform accounts — that exist, claimed by an account or not |
-| `chat_identity_active_1d` | number | distinct chat identities that used a room with an agent in 24h. Most belong to no Switch account and one person may have several, which is why this runs above `user_active_1d`. Servers released before this change reported this figure as `user_active_1d` |
-| `chat_identity_active_7d` | number | same over 7 days |
+| `chat_identity_active_1d` | number | distinct chat identities that used a room with an agent in 24h. Most belong to no Switch account and one person may have several, which is why this runs above `user_active_1d`. Servers released before this change reported this figure as `user_active_1d`. Counts integrations as well as people; superseded by `person_identity_active_1d` plus `bot_identity_active_1d`, and sent for one more stable release |
+| `chat_identity_active_7d` | number | same over 7 days. Superseded likewise |
+| `person_identity_active_1d` | number | **the headline count of people** — distinct chat identities that used a room with an agent in 24h and never posted as an app in that window, linked to a Switch account or not. One person on two platforms is two identities. See "People, bots and accounts" |
+| `person_identity_active_7d` | number | same over 7 days |
+| `bot_identity_active_1d` | number | distinct chat identities that posted as an app in a room with an agent in 24h: an integration such as an alerting or CI app, or another Switch server's agent. See "People, bots and accounts" |
+| `bot_identity_active_7d` | number | same over 7 days |
+| `account_active_1d` | number | distinct Switch **accounts** that used a room with an agent in 24h, the same figure as `user_active_1d` under the name it keeps. The floor of the people count: an account exists only for someone who linked one |
+| `account_active_7d` | number | same over 7 days |
| `chat_identity_posted_1d` | number | distinct chat identities that said anything in any room in 24h, agent or no agent |
| `chat_identity_posted_7d` | number | same over 7 days |
| `chat_identity_in_room_count` | number | distinct chat identities Switch has seen in a room: a member of at least one unarchived room of any origin, or anyone who said anything in the last 7 days. Not a channel's whole audience — see "Tiers of activity" |
@@ -394,6 +415,22 @@ more easily than it charts a nested object. They count **configured** bridges,
whether or not each is currently connected — which is up is process state, and
`bridge_connected` / `bridge_disconnected` are how that is reported.
+**People, bots and accounts.** A messaging platform relays posts from more
+than people: an alerting or CI integration, a workflow, and another Switch
+server's agents sharing the channel all post there. Switch relays them into
+the room like any post, through the same kind of chat identity a person gets,
+so `chat_identity_active_*` counted them as people. Where the platform says a
+post came from an app (a Slack app, a Discord bot or a webhook that is not
+Switch's own, a Mattermost bot account or incoming webhook), the bridge marks
+it, and that identity counts in `bot_identity_active_*` rather than
+`person_identity_active_*` for the whole window. Telegram never delivers
+another bot's posts and Teams does not mark them, so nothing there is counted
+as a bot. A post relayed before its bridge marked app posts carries no mark,
+so for the first week after upgrading, an integration that has not posted
+since can still count as a person. `account_active_*` is the floor:
+integrations never link a Switch account, and most people never link one
+either.
+
**What counts as a message.** Every durable event is a row in the message log:
arrivals, tool- and LLM-call reports and task transitions sit beside the
conversation. The message counts, the turn counts and every "interacted" figure
@@ -445,6 +482,66 @@ near-complete, not exact. That is fine for "how much, and is it growing", and it
should be said plainly wherever the number is presented rather than discovered
later.
+### `usage_period` — once per finished day, week and month
+
+The snapshot's `*_7d` counts are a trailing week, sent every day. A chart that
+adds them up at a weekly interval counts a room once for every day it was in
+the window, up to seven times. A daily interval avoids that, but a server that
+is off on a given day, as laptops often are, sends nothing for it. `usage_period` is counted over a fixed
+calendar period instead, and sent once per period, so at the matching interval
+the sum across deployments is the real figure.
+
+| Property | Type | Meaning |
+|---|---|---|
+| `period` | `day` / `week` / `month` | a UTC day, an ISO week (Monday to Monday, UTC), or a calendar month |
+| `room_active` | number | distinct rooms a person used with an agent in the period: `room_active_*`'s definition |
+| `person_identity_active` | number | distinct people who did, bots excluded: `person_identity_active_*`'s definition |
+| `account_active` | number | distinct Switch accounts among them: `account_active_*`'s definition, deduplicated across tenants |
+| `tenant_failed_count` | number | tenants whose counts could not be read, left out of the three above, as on the snapshot |
+
+**Chart it at its own interval.** Filter on one `period`, set the interval to
+match, and sum each property: weekly active rooms is the sum of
+`room_active` over `usage_period` where `period = week`, at a weekly interval.
+At any other interval the sum is wrong again — weeks summed into a month count a
+room once per week — so each interval has its own chart. Across deployments the
+sum counts a person who uses two servers twice, as every per-deployment figure
+does.
+
+**Dated to the middle of the period**, not to when it was sent: noon for a day,
+Thursday noon for a week, the middle of the month. A server that was off sends
+what it missed when it next runs, and an event dated "now" would land in a later
+bucket than the period it describes. The middle is inside the same day, week and
+month whichever day a chart starts its weeks on and in any timezone it is drawn
+in. Datadog's log intake refuses records dated more than 18 hours back, so its
+copy of the relay's events has few of these; Amplitude takes them.
+
+**Sent with the snapshot**, for every period that ended since the previous
+snapshot, so each is sent once. Not on a deployment's first pass, which has no
+previous snapshot to measure from. A server back after a long time off sends
+only the periods that ended in the last 35 days, which still covers the month
+before for a laptop that was off for most of the month after. Like
+`room_became_active`, a server whose telemetry was switched off and on again
+sends the periods that ended in between, within those 35 days.
+
+### Accounts, workspaces and cloud machines
+
+Switch Cloud's funnel: someone signs up, makes a workspace, and gets a cloud
+machine for their agents. Any server that does these sends the events, so read
+them with `deployment_kind` (on every event) to look at Cloud alone. Each is
+sent once the change is committed: a machine change made in a transaction that
+then rolls back is never reported.
+
+| Event | Properties | Sent when |
+|---|---|---|
+| `account_created` | `method`: `signup` (the sign-up page) / `oidc` (a first single sign-on sign-in) / `admin` (an admin created it) | a new account exists. A single sign-on sign-in that links an existing account is not one |
+| `signup_refused` | `reason`: `disabled` / `rate_limited` (the hourly cap) / `email_taken` | the sign-up page was refused |
+| `workspace_created` | `new_organization` (boolean): an organization was created with it | a workspace and its owner's membership exist |
+| `cloud_machine_requested` | `change`: `new` / `joined_workspace` (the owner's machine now serves another workspace) / `unretained` (a machine kept after its agents left, running again) / `restarted` (a stopped machine started by a claim) | a claim changed something; a claim on a machine already running is not reported. One event per claim, naming the first of these that applies, so a retained machine joined from another workspace is `joined_workspace` though it also runs again |
+| `cloud_machine_ready` | `new_machine` (boolean) · `provisioning_seconds`: from the claim to the first controller report for a new machine, `-1` for a later start | the machine's controller reported at its current revision |
+| `cloud_machine_failed` | `reason`: `not_scheduled` / `never_started` / `never_connected` (each after 10 minutes) / `owner_left` (its owner is in none of its workspaces) / `needs_attention` / `reported` (any other error the machine reported) | the machine went into error; not again while it stays there |
+| `cloud_machine_woken` | `trigger`: `message` (its agent was addressed) / `agent_placed` / `owner` · `stopped_by`: `idle` / `owner` / `unknown` | a stopped machine was started again |
+| `cloud_machine_stopped` | `reason`: `idle` / `owner` | a running machine was told to stop |
+
### Milestone events — at most once per deployment
Each carries `seconds_since_install` (number). Together with
@@ -459,7 +556,18 @@ deployments installed after this ships — see
| `first_room_created` | the first **user-created** room is created | `channel_type`, `bridge_platform` |
| `first_room_active` | that room sees its first interaction | `bridge_platform`, `seconds_since_room_created` |
| `first_agent_registered` | the first agent registers | `known_agent_type` |
-| `first_session_started` | the first agent session opens | `known_agent_type` |
+| `first_session_started` | the first agent comes online: its first live connection. Superseded by `first_agent_connected`, the same moment; sent for one more stable release | `known_agent_type` |
+| `first_agent_connected` | the first agent comes online | `known_agent_type` |
+| `first_agent_session` | a host reports the first coding-agent session (`session_started`): an agent doing work | `known_agent_type`, `start_source` |
+
+`first_agent_connected` and `first_agent_session` were added after deployments
+were already running. One that had already passed them claims them without
+reporting, once, at the first boot of a version that has them, so it is never
+reported as doing for the first time what it did long ago: `first_agent_connected` once `first_session_started` was claimed,
+and `first_agent_session` once `first_room_active` was claimed or a session was
+reported in the last 7 days. A deployment whose only sessions were started
+from Switch Console, more than a week before upgrading, with no person ever
+using a room with an agent, reports `first_agent_session` late.
The two room milestones track user-created rooms only, for the reason given
under [Definitions](#definitions): a room an agent made for itself is not the
@@ -594,6 +702,11 @@ that as `false` would misfile a lookup failure as a failed setup.
| `registration_path` | `bootstrap` \| `personal_key` \| `gateway` \| `other` |
| `has_parent` | boolean — a subagent rather than a top-level agent |
+**`agent_connected`** — `known_agent_type`, and **`agent_disconnected`** —
+the properties of `agent_session_ended` below. The same two events as
+`agent_session_started` and `agent_session_ended`, under names that say what
+they measure. Both pairs are sent for one more stable release.
+
**`agent_session_started`** — `known_agent_type`. An agent *coming online*: its
first live connection, not a session a person began. Sessions share their
agent's one connection, held by Switch Console or a remote host's sidecar, so
@@ -686,7 +799,7 @@ the person said.
| Property | Type |
|---|---|
-| `sender_kind` | `user` \| `agent` — only people and agents are metered, so `platform` does not occur here |
+| `sender_kind` | `user` \| `bot` \| `agent` — only participants are metered, so `platform` does not occur here. `bot` is an integration's post (see "People, bots and accounts"); before that value existed those were `user` |
| `bridge_platform` | platform, `none` for an internal room, `unknown` if the room could not be read |
| `channel_type` | channel type, or `unknown` if the room could not be read |
| `room_user_count` | number — human members of the room, `-1` if it could not be read |
@@ -716,7 +829,7 @@ the requests an agent could act on as they arrived.
| Property | Type |
|---|---|
-| `sender_kind` | `user` \| `agent` \| `platform` \| `unknown` |
+| `sender_kind` | `user` \| `bot` \| `agent` \| `platform` \| `unknown` — `bot` as for `room_message_sent` |
| `known_agent_type` | the receiving agent's runtime |
| `bridge_platform` | platform |
| `channel_type` | channel type |
@@ -796,8 +909,10 @@ it today; the room events read the type from a row they already hold.
four built-in types; a user-defined type's slug is free text, so it reports as
`other`.
-`sender_kind`: `user` | `agent` | `platform` | `unknown`. `platform` is Switch
-itself speaking; `unknown` a sender whose client could not be looked up.
+`sender_kind`: `user` | `bot` | `agent` | `platform` | `unknown`. `user` is a
+person on a messaging platform and `bot` an integration posting there (see
+"People, bots and accounts"); `platform` is Switch itself speaking; `unknown` a
+sender whose client could not be looked up.
`outcome`: `success` | `failure`. `failure_reason` is an enumerated code per
event, `none` on success — never an exception message.
@@ -992,15 +1107,21 @@ Two things to set up first, or a third of the list cannot fire at all:
**Sessions** — `bridges/agent/api/handlers.py`, and the connection registry
-- `agent_session_started` — an agent opening its event stream. A supervisor
- reattaching to a connection it already had correctly emits nothing.
-- `first_session_started` — the same, once ever.
-- `agent_session_ended` — kill an agent and wait for the heartbeat sweep
- (a few seconds). Reported through a listener on the registry, so every path
- that closes a connection reports, not just the sweep.
+- `agent_session_started` and `agent_connected` — an agent opening its event
+ stream; both are sent. A supervisor reattaching to a connection it already
+ had correctly emits nothing.
+- `first_session_started` and `first_agent_connected` — the same, once ever.
+ On a deployment that claimed `first_session_started` before this release,
+ `first_agent_connected` is claimed at boot and never sent.
+- `agent_session_ended` and `agent_disconnected` — kill an agent and wait for
+ the heartbeat sweep (a few seconds). Reported through a listener on the
+ registry, so every path that closes a connection reports, not just the sweep.
- `session_started` — `POST /agent-sessions/{id}/started` from a session host
(`bridges/agent/api/activity_routes.py`), with a UUID id. A repeat for the
same agent and session emits nothing, across restarts, for 7 days.
+- `first_agent_session` — the first `session_started` a deployment sends, once
+ ever; claimed at boot without sending on a deployment that already had
+ sessions (see the milestone table).
**Messages** — `telemetry/messages.py`, fed by `transport/postgres.py` and
`delivery/recipient.py`
@@ -1089,7 +1210,7 @@ one event watched arriving downstream, which no local check can substitute for.
## Open questions
-- **Query cost at scale.** The snapshot is roughly fifteen queries per tenant,
+- **Query cost at scale.** The snapshot is roughly twenty queries per tenant,
including two seven-day `DISTINCT` scans over `messages` and a window function
over the message table. Nobody has run it against a production-sized dataset.
That is the open question most worth answering before this is switched on for