Skip to content

Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.

[CHOO-2394] telemetry: pin typed properties and the relay's silent drops, and say which Amplitude project each event belongs in (#593)

Christian McDermott3231a957a9f4
diff --git a/.env.example b/.env.example
index f91551fd6..c1ba5ac7d 100644
--- a/.env.example
+++ b/.env.example
@@ -123,6 +123,16 @@ DB_NAME=switch
# OTLP_TIMEOUT_SECONDS=10
# OTLP_EXPORT_INTERVAL_SECONDS=60
+# ── Product telemetry ────────────────────────────────────────────────────────
+# Anonymous usage counts, off unless switched on (docs/old/telemetry-events.md).
+# TELEMETRY_ENABLED=false
+#
+# Which Amplitude project this server's usage lands in: prod, staging, dev or
+# local. Unset means prod, which is right for every customer deployment. This
+# file is for a developer's machine, so it says local, keeping a local run's
+# usage out of the production numbers if telemetry is ever switched on.
+TELEMETRY_ENVIRONMENT=local
+
# ── Client identity ──────────────────────────────────────────────────────────
# The server half of every member's `@localpart:server` id. Nothing is
# contacted at it; the ids are stable public handles. (MATRIX_SERVER_NAME, the
diff --git a/.github/workflows/switch-console-release.yml b/.github/workflows/switch-console-release.yml
index 06d844848..59f6ac2fa 100644
--- a/.github/workflows/switch-console-release.yml
+++ b/.github/workflows/switch-console-release.yml
@@ -68,6 +68,7 @@ jobs:
manifest: ${{ steps.resolve.outputs.manifest }}
artifact_prefix: ${{ steps.resolve.outputs.artifact_prefix }}
vite_build: ${{ steps.resolve.outputs.vite_build }}
+ vite_release: ${{ steps.resolve.outputs.vite_release }}
steps:
- name: Checkout
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
@@ -124,6 +125,13 @@ jobs:
version="$pkg_version"
;;
esac
+ # A tagged build is a published release; a workflow_dispatch test build
+ # is not. The app reports its usage to production's Amplitude project
+ # (or staging's, for a canary) only when this says it is a release.
+ case "$GITHUB_REF" in
+ refs/tags/*) vite_release=1 ;;
+ *) vite_release= ;;
+ esac
if [ "$channel" = canary ]; then
builder_config=electron-builder.canary.config.ts
manifest=canary
@@ -142,6 +150,7 @@ jobs:
echo "manifest=$manifest"
echo "artifact_prefix=$artifact_prefix"
echo "vite_build=$vite_build"
+ echo "vite_release=$vite_release"
} | tee -a "$GITHUB_OUTPUT"
# Creates the (assetless) Release up front so the platform jobs can run
@@ -273,6 +282,7 @@ jobs:
# Root recursive build (topological): packages/** then the desktop app.
env:
VITE_BUILD: ${{ needs.resolve.outputs.vite_build }}
+ VITE_RELEASE: ${{ needs.resolve.outputs.vite_release }}
# Only canary knows where Switch Cloud is; a build without it offers no Cloud.
MAIN_VITE_SWITCH_CLOUD_URL: ${{ needs.resolve.outputs.channel == 'canary' && vars.SWITCH_CLOUD_URL || '' }}
run: pnpm run build
@@ -566,6 +576,7 @@ jobs:
- name: Build workspace packages + app
env:
VITE_BUILD: ${{ needs.resolve.outputs.vite_build }}
+ VITE_RELEASE: ${{ needs.resolve.outputs.vite_release }}
# Only canary knows where Switch Cloud is; a build without it offers no Cloud.
MAIN_VITE_SWITCH_CLOUD_URL: ${{ needs.resolve.outputs.channel == 'canary' && vars.SWITCH_CLOUD_URL || '' }}
run: pnpm run build
@@ -677,6 +688,7 @@ jobs:
- name: Build workspace packages + app
env:
VITE_BUILD: ${{ needs.resolve.outputs.vite_build }}
+ VITE_RELEASE: ${{ needs.resolve.outputs.vite_release }}
# Only canary knows where Switch Cloud is; a build without it offers no Cloud.
MAIN_VITE_SWITCH_CLOUD_URL: ${{ needs.resolve.outputs.channel == 'canary' && vars.SWITCH_CLOUD_URL || '' }}
run: pnpm run build
diff --git a/console/AGENTS.md b/console/AGENTS.md
index 98b857c4d..600b59202 100644
--- a/console/AGENTS.md
+++ b/console/AGENTS.md
@@ -439,8 +439,10 @@ pnpm run lint
enough — excess-property checking does not apply through a spread — so the runtime
filter is what makes "nothing free-text can reach a payload" true rather than intended.
Permitted: which of the catalogued things happened, agent type, local-vs-remote,
- success-vs-failure, how long an operation took, app version, operating system, and the
- random install id. A duration is a number rather than a value from a fixed set,
+ success-vs-failure, how long an operation took, app version, operating system, the
+ random install id, which kind of launch it was (first, after an update, or neither),
+ and whether a signed-in account is on a company email domain — the yes/no only, read
+ from the domain, never the address. A duration is a number rather than a value from a fixed set,
so it is held to `TelemetryDurationMs`: measured on a monotonic clock, whole
milliseconds, and never a span that could encode something else. That is a branded
type, not an alias for `number`, and `startTimer()` is the only thing that mints one —
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.test.ts
index 7db87948f..3f90a2f9b 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.test.ts
@@ -31,6 +31,8 @@ describe('buildEnvFile', () => {
ports: { gateway: 51000, api: 51001, mattermost: 51002, postgres: 51003 },
secrets,
telemetryEnabled: false,
+ telemetryEnvironment: 'prod',
+ telemetryInternal: false,
});
const vars = Object.fromEntries(
env
@@ -157,10 +159,49 @@ describe('buildEnvFile', () => {
ports: { gateway: 51000, api: 51001, mattermost: 51002, postgres: 51003 },
secrets,
telemetryEnabled: true,
+ telemetryEnvironment: 'prod',
+ telemetryInternal: false,
});
expect(sharing).toContain('TELEMETRY_ENABLED=true');
});
+ it('tells the server which Amplitude project its usage belongs in', () => {
+ // The relay files each event under the project its environment names, so a
+ // server a development build runs must say so as plainly as the app does.
+ const fromDevBuild = buildEnvFile({
+ version: '1.2.3',
+ registry: 'ghcr.io',
+ namespace: 'sandbox-quantum',
+ ports: { gateway: 51000, api: 51001, mattermost: 51002, postgres: 51003 },
+ secrets,
+ telemetryEnabled: true,
+ telemetryEnvironment: 'local',
+ telemetryInternal: false,
+ });
+
+ expect(fromDevBuild).toContain('TELEMETRY_ENVIRONMENT=local');
+ expect(vars.TELEMETRY_ENVIRONMENT).toBe('prod');
+ // Written but never forwarded would reach no container.
+ expect(composeYaml).toMatch(/^\s+TELEMETRY_ENVIRONMENT:\s*$/m);
+ });
+
+ it('tells the server whether the person it runs for is staff', () => {
+ const forStaff = buildEnvFile({
+ version: '1.2.3',
+ registry: 'ghcr.io',
+ namespace: 'sandbox-quantum',
+ ports: { gateway: 51000, api: 51001, mattermost: 51002, postgres: 51003 },
+ secrets,
+ telemetryEnabled: true,
+ telemetryEnvironment: 'prod',
+ telemetryInternal: true,
+ });
+
+ expect(forStaff).toContain('TELEMETRY_INTERNAL=true');
+ expect(vars.TELEMETRY_INTERNAL).toBe('false');
+ expect(composeYaml).toMatch(/^\s+TELEMETRY_INTERNAL:\s*$/m);
+ });
+
it('is the only thing the compose file needs to forward the gate', () => {
// A value in the `.env` that the `switch` service does not name never
// reaches the container, which is the failure mode this pairing exists to
@@ -186,6 +227,8 @@ describe('readStackEnv', () => {
ports,
secrets,
telemetryEnabled: true,
+ telemetryEnvironment: 'prod',
+ telemetryInternal: false,
});
it('reads back exactly what buildEnvFile wrote', () => {
@@ -298,6 +341,8 @@ describe('keysDisagreeing', () => {
ports,
secrets,
telemetryEnabled: true,
+ telemetryEnvironment: 'prod',
+ telemetryInternal: false,
});
it('finds nothing wrong with a copy of the same settings', () => {
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.ts
index 1bd28cba1..7da8ed79d 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.ts
@@ -1,3 +1,4 @@
+import type { TelemetryFlintEnv } from '../telemetry/config';
import {
LOCAL_SERVER_ADMIN_EMAIL,
LOCAL_SERVER_BIND_ADDR,
@@ -23,6 +24,13 @@ export type LocalServerEnvParams = {
* and nothing else — not a separate server-side preference (CHOO-2890).
*/
telemetryEnabled: boolean;
+ /**
+ * The Amplitude project the server's usage lands in: the Console build's
+ * own, so a development build's server is as much `local` as the app.
+ */
+ telemetryEnvironment: TelemetryFlintEnv;
+ /** Whether the person this server runs for is staff, as Console tells it. */
+ telemetryInternal: boolean;
};
/** The variable the server reads its telemetry gate from. Named here because
@@ -36,7 +44,16 @@ export const TELEMETRY_ENABLED_KEY = 'TELEMETRY_ENABLED';
* app's pin. Kept as a pure function so the exact output is unit-testable.
*/
export function buildEnvFile(params: LocalServerEnvParams): string {
- const { version, registry, namespace, ports, secrets, telemetryEnabled } = params;
+ const {
+ version,
+ registry,
+ namespace,
+ ports,
+ secrets,
+ telemetryEnabled,
+ telemetryEnvironment,
+ telemetryInternal,
+ } = params;
return [
'# Generated by Switch Console local-server mode — do not edit by hand.',
'# Secrets originate in the app encrypted store; this file is derived.',
@@ -87,6 +104,8 @@ export function buildEnvFile(params: LocalServerEnvParams): string {
// because the compose file passes this key through only when the `.env`
// carries it — so "off" has to be said, not left unsaid.
`${TELEMETRY_ENABLED_KEY}=${telemetryEnabled ? 'true' : 'false'}`,
+ `TELEMETRY_ENVIRONMENT=${telemetryEnvironment}`,
+ `TELEMETRY_INTERNAL=${telemetryInternal ? 'true' : 'false'}`,
'',
'MATTERMOST_ADMIN_USER=admin',
`MATTERMOST_ADMIN_PASSWORD=${secrets.mattermostAdminPassword}`,
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-matrix-migration.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-matrix-migration.test.ts
index b8c8fd5d5..8d90868aa 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-matrix-migration.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-matrix-migration.test.ts
@@ -59,6 +59,9 @@ vi.mock('./telemetry-consent', () => ({
telemetryConsent: () => Promise.resolve(false),
readDeployedTelemetry: vi.fn(),
}));
+vi.mock('@main/core/telemetry/internal-account', () => ({
+ currentInternalFlag: async () => 'unknown',
+}));
vi.mock('@main/core/switch-servers/servers-store', () => ({
assertManagedServerUrlFree: () => Promise.resolve(),
ensureManagedServer: () => Promise.resolve({ id: 'srv-1' }),
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-shared-host.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-shared-host.test.ts
index 748bcb941..c9fa83b98 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-shared-host.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-shared-host.test.ts
@@ -89,6 +89,9 @@ vi.mock('./ports', () => ({
const telemetryConsentMock = vi.hoisted(() => vi.fn(() => Promise.resolve(false)));
vi.mock('./telemetry-consent', () => ({ telemetryConsent: telemetryConsentMock }));
const assertManagedServerUrlFreeMock = vi.hoisted(() => vi.fn(() => Promise.resolve()));
+vi.mock('@main/core/telemetry/internal-account', () => ({
+ currentInternalFlag: async () => 'unknown',
+}));
vi.mock('@main/core/switch-servers/servers-store', () => ({
assertManagedServerUrlFree: assertManagedServerUrlFreeMock,
ensureManagedServer: ensureManagedServerMock,
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.test.ts
index 278caaccf..9adb2861c 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.test.ts
@@ -17,6 +17,9 @@ const logError = vi.hoisted(() => vi.fn());
const logWarn = vi.hoisted(() => vi.fn());
const buildEnvFileMock = vi.hoisted(() => vi.fn((_params: unknown) => 'SWITCH_VERSION=0.11.0\n'));
const telemetryConsentMock = vi.hoisted(() => vi.fn(() => Promise.resolve(false)));
+const currentInternalFlagMock = vi.hoisted(() =>
+ vi.fn((): Promise<'true' | 'false' | 'unknown'> => Promise.resolve('unknown'))
+);
// Pin the build's expected version so the guard's arithmetic is not coupled to
// whatever the real pin happens to be.
@@ -65,6 +68,9 @@ vi.mock('./telemetry-consent', () => ({
readDeployedTelemetry: vi.fn(),
}));
const setActiveServerIdMock = vi.hoisted(() => vi.fn());
+vi.mock('@main/core/telemetry/internal-account', () => ({
+ currentInternalFlag: currentInternalFlagMock,
+}));
vi.mock('@main/core/switch-servers/servers-store', () => ({
assertManagedServerUrlFree: () => Promise.resolve(),
ensureManagedServer: () => Promise.resolve({ id: 'srv-1' }),
@@ -83,6 +89,7 @@ vi.mock('./managed-upgrade', () => ({
}));
const { startStack } = await import('./pipeline');
+const { currentFlintEnv } = await import('../telemetry/config');
const { ENV_FILE_NAME } = await import('./constants');
function options() {
@@ -118,6 +125,7 @@ beforeEach(() => {
vi.clearAllMocks();
prepareUpgradeMock.mockResolvedValue(null);
telemetryConsentMock.mockResolvedValue(false);
+ currentInternalFlagMock.mockResolvedValue('unknown');
buildEnvFileMock.mockReturnValue('SWITCH_VERSION=0.11.0\n');
});
@@ -161,6 +169,32 @@ describe('startStack version guard', () => {
expect(composeUpMock).toHaveBeenCalledOnce();
});
+ it('tells the server which project its usage belongs in, and whether its person is staff', async () => {
+ readDeployedVersionMock.mockResolvedValue({ kind: 'absent' });
+ currentInternalFlagMock.mockResolvedValue('true');
+ const { opts } = options();
+
+ await startStack(opts);
+
+ expect(buildEnvFileMock).toHaveBeenCalledWith(
+ expect.objectContaining({
+ telemetryEnvironment: currentFlintEnv(),
+ telemetryInternal: true,
+ })
+ );
+ });
+
+ it('does not call an unknown account staff', async () => {
+ readDeployedVersionMock.mockResolvedValue({ kind: 'absent' });
+ const { opts } = options();
+
+ await startStack(opts);
+
+ expect(buildEnvFileMock).toHaveBeenCalledWith(
+ expect.objectContaining({ telemetryInternal: false })
+ );
+ });
+
it('starts a fresh host with nothing deployed', async () => {
readDeployedVersionMock.mockResolvedValue({ kind: 'absent' });
const { opts } = options();
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.ts
index c9490ae1f..8166c9928 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.ts
@@ -14,6 +14,8 @@ import {
type StartLocalServerResult,
} from '@shared/core/managed-switch-server/managed-switch-server';
import type { ManagedServerRef } from '@shared/core/switch-servers/switch-servers';
+import { currentFlintEnv } from '../telemetry/config';
+import { currentInternalFlag } from '../telemetry/internal-account';
import { bundledComposeYaml } from './bundled-compose';
import { checkoutBuildOverrideYaml } from './checkout-build';
import { composeDown, composeUp } from './compose';
@@ -565,6 +567,8 @@ export async function startStack(opts: StartStackOptions): Promise<StartLocalSer
ports: settings.ports,
secrets: settings.secrets,
telemetryEnabled,
+ telemetryEnvironment: currentFlintEnv(),
+ telemetryInternal: (await currentInternalFlag()) === 'true',
});
await host.writeFile(ENV_FILE_NAME, env, 0o600);
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/resources/standalone-docker-compose.pinned.yml b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/resources/standalone-docker-compose.pinned.yml
index b283d4d84..1a7d73bcc 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/resources/standalone-docker-compose.pinned.yml
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/resources/standalone-docker-compose.pinned.yml
@@ -275,6 +275,12 @@ services:
# brought up by hand, an operator sets it here the same way.
TELEMETRY_ENABLED:
TELEMETRY_ENDPOINT:
+ # Which Amplitude project the usage lands in. Unset is prod; Switch
+ # Console writes its own build's environment here for a server it runs.
+ TELEMETRY_ENVIRONMENT:
+ # Whether this is one of the company's own deployments. Switch Console
+ # writes its own answer here for a server it runs.
+ TELEMETRY_INTERNAL:
TELEMETRY_SNAPSHOT_INTERVAL_HOURS:
depends_on:
postgres:
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-state.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-state.test.ts
index 281e991d9..0c1c69b83 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-state.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-state.test.ts
@@ -51,6 +51,8 @@ function envFor(overrides: Partial<LocalServerSecrets> = {}, version = '0.27.0')
ports,
secrets: { ...secrets, ...overrides },
telemetryEnabled: false,
+ telemetryEnvironment: 'prod',
+ telemetryInternal: false,
});
}
diff --git a/console/apps/switch-console-desktop/src/main/core/settings/controller.telemetry.test.ts b/console/apps/switch-console-desktop/src/main/core/settings/controller.telemetry.test.ts
index 21d562cc9..374dfd939 100644
--- a/console/apps/switch-console-desktop/src/main/core/settings/controller.telemetry.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/settings/controller.telemetry.test.ts
@@ -16,6 +16,7 @@ const { h } = vi.hoisted(() => ({
}));
vi.mock('@main/core/telemetry/telemetry-service', () => ({ trackEvent: h.trackEvent }));
+vi.mock('@main/core/telemetry/launch-history', () => ({ currentInstallKind: () => 'new' }));
vi.mock('./settings-service', () => ({
appSettingsService: {
get: h.get,
@@ -87,6 +88,7 @@ describe('agreeing to share usage data', () => {
expect(h.trackEvent).toHaveBeenCalledWith('telemetry_consent_changed', {
source: 'first_run',
+ install_kind: 'new',
});
});
@@ -95,7 +97,10 @@ describe('agreeing to share usage data', () => {
await appSettingsController.update('telemetry', { enabled: true, askedAt: 1 } as never);
- expect(h.trackEvent).toHaveBeenCalledWith('telemetry_consent_changed', { source: 'settings' });
+ expect(h.trackEvent).toHaveBeenCalledWith('telemetry_consent_changed', {
+ source: 'settings',
+ install_kind: 'new',
+ });
});
it('says nothing when someone declines', async () => {
diff --git a/console/apps/switch-console-desktop/src/main/core/settings/controller.ts b/console/apps/switch-console-desktop/src/main/core/settings/controller.ts
index a8ba13462..b84c9fb8e 100644
--- a/console/apps/switch-console-desktop/src/main/core/settings/controller.ts
+++ b/console/apps/switch-console-desktop/src/main/core/settings/controller.ts
@@ -1,4 +1,5 @@
import type { TelemetrySettingKey } from '@main/core/telemetry/events';
+import { currentInstallKind } from '@main/core/telemetry/launch-history';
import { trackEvent } from '@main/core/telemetry/telemetry-service';
import type { TelemetrySettings } from '@shared/core/app-settings';
import { createRPCController } from '@shared/lib/ipc/rpc';
@@ -23,6 +24,7 @@ function reportConsent(previous: TelemetrySettings, next: TelemetrySettings): vo
trackEvent('telemetry_consent_changed', {
source: previous.askedAt === null ? 'first_run' : 'settings',
+ install_kind: currentInstallKind(),
});
}
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.ts
index 6a0ca46dd..b311047bf 100644
--- a/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.ts
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.ts
@@ -2,6 +2,7 @@ import { randomUUID } from 'node:crypto';
import { and, desc, eq, isNull, ne, or, sql } from 'drizzle-orm';
import { encryptedAppSecretsStore } from '@main/core/secrets/encrypted-app-secrets-store';
import type { TelemetryEventMap } from '@main/core/telemetry/events';
+import { forgetSessionAccount, recordSessionAccount } from '@main/core/telemetry/internal-account';
import { trackEvent } from '@main/core/telemetry/telemetry-service';
import { forgetServerSession } from '@main/core/workspaces/workspace-session';
import {
@@ -15,6 +16,7 @@ import {
} from '@main/core/workspaces/workspaces-store';
import { db } from '@main/db/client';
import { type SwitchServerRow, switchServers } from '@main/db/schema';
+import { log } from '@main/lib/logger';
import {
urlOrigin,
type AddServerParams,
@@ -375,8 +377,14 @@ export async function getSessionCookie(serverId: string): Promise<string | null>
export async function setSessionCookie(serverId: string, jwt: string): Promise<void> {
await encryptedAppSecretsStore.setSecret(cookieSecretKey(serverId), jwt);
+ await recordSessionAccount(serverId, jwt).catch((error: unknown) => {
+ log.warn('switch-servers: could not record whether the account is internal', { error });
+ });
}
export async function deleteSessionCookie(serverId: string): Promise<void> {
await encryptedAppSecretsStore.deleteSecret(cookieSecretKey(serverId));
+ await forgetSessionAccount(serverId).catch((error: unknown) => {
+ log.warn('switch-servers: could not forget whether the account was internal', { error });
+ });
}
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/catalogue.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/catalogue.test.ts
index ac0975a66..a692e4781 100644
--- a/console/apps/switch-console-desktop/src/main/core/telemetry/catalogue.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/catalogue.test.ts
@@ -26,6 +26,8 @@ const CONTEXT: TelemetryContext = {
osType: 'darwin',
osVersion: '24.3.0',
build: 'stable',
+ flintEnv: 'prod',
+ internal: 'false',
timeMs: 1_700_000_000_000,
};
@@ -109,6 +111,34 @@ describe('the catalogue as a whole', () => {
);
});
+ it.each(EVENT_NAMES)('sends %s well under the attribute count the relay drops at', (name) => {
+ // The relay drops any record with more than 128 attributes, and still
+ // answers 200. Held here, over every event, with room to spare.
+ const payload = buildOtlpPayload(name, sampleEvent(name) as never, CONTEXT);
+
+ expect(recordOf(payload).attributes.length).toBeLessThanOrEqual(100);
+ });
+
+ it.each(EVENT_NAMES)('sends %s well under the event size the relay drops at', (name) => {
+ // The relay drops any Amplitude event whose JSON is over 32 KiB, and still
+ // answers 200. Every string here is longer than any value the catalogue
+ // allows, and the OTLP attributes wrap each value in more JSON than the
+ // Amplitude event does, so this over-counts.
+ const largest = Object.fromEntries(
+ Object.entries(sampleEvent(name)).map(([key, value]) => [
+ key,
+ typeof value === 'string'
+ ? 'x'.repeat(256)
+ : typeof value === 'number'
+ ? -123456789012345.67
+ : false,
+ ])
+ );
+ const payload = buildOtlpPayload(name, largest as never, CONTEXT);
+
+ expect(JSON.stringify(recordOf(payload).attributes).length).toBeLessThanOrEqual(16384);
+ });
+
it.each(EVENT_NAMES)('refuses to send %s with a property missing', (name) => {
const properties = sampleEvent(name);
const declared = TELEMETRY_EVENT_PROPERTIES[name];
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/config.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/config.test.ts
index 28f08ac43..4ce817536 100644
--- a/console/apps/switch-console-desktop/src/main/core/telemetry/config.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/config.test.ts
@@ -1,13 +1,18 @@
import { describe, expect, it } from 'vitest';
import {
+ flintEnvFor,
resolveTelemetryEnvironment,
TELEMETRY_RELAY_ENDPOINT,
type TelemetryBuildChannel,
type TelemetryEnvironment,
} from './config';
-function resolve(build: TelemetryBuildChannel, env: TelemetryEnvironment['env'] = {}) {
- return resolveTelemetryEnvironment({ build, env });
+function resolve(
+ build: TelemetryBuildChannel,
+ env: TelemetryEnvironment['env'] = {},
+ release = build !== 'dev'
+) {
+ return resolveTelemetryEnvironment({ build, release, env });
}
describe('a dev run', () => {
@@ -18,7 +23,7 @@ describe('a dev run', () => {
it('reports to the relay once opted in', () => {
expect(resolve('dev', { SWITCHDASH_TELEMETRY_DEV: '1' })).toEqual({
enabled: true,
- config: { endpoint: TELEMETRY_RELAY_ENDPOINT, build: 'dev' },
+ config: { endpoint: TELEMETRY_RELAY_ENDPOINT, build: 'dev', flintEnv: 'local' },
});
});
@@ -47,7 +52,11 @@ describe.each(['stable', 'canary'] as const)('a %s build', (build) => {
// this replaced, there is no build that is inert for want of a key.
expect(resolve(build)).toEqual({
enabled: true,
- config: { endpoint: TELEMETRY_RELAY_ENDPOINT, build },
+ config: {
+ endpoint: TELEMETRY_RELAY_ENDPOINT,
+ build,
+ flintEnv: build === 'canary' ? 'staging' : 'prod',
+ },
});
});
@@ -68,3 +77,24 @@ describe.each(['stable', 'canary'] as const)('a %s build', (build) => {
expect(resolution.enabled && resolution.config.build).toBe(build);
});
});
+
+describe('the Amplitude project a build reports to', () => {
+ // The relay files each event under the project its flint_env names, so only a
+ // released stable build's events are production's.
+ it.each([
+ ['stable', true, 'prod'],
+ ['canary', true, 'staging'],
+ ['stable', false, 'dev'],
+ ['canary', false, 'dev'],
+ ['dev', false, 'local'],
+ ['dev', true, 'local'],
+ ] as const)('is %s, released %s → %s', (build, release, environment) => {
+ expect(flintEnvFor({ build, release })).toBe(environment);
+ });
+
+ it('is carried in the resolved config, so every event of a build agrees', () => {
+ const resolution = resolve('stable', {}, false);
+
+ expect(resolution.enabled && resolution.config.flintEnv).toBe('dev');
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/config.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/config.ts
index b812640db..bf237de34 100644
--- a/console/apps/switch-console-desktop/src/main/core/telemetry/config.ts
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/config.ts
@@ -1,4 +1,4 @@
-import { IS_CANARY } from '@shared/app-identity';
+import { IS_CANARY, IS_RELEASE_BUILD } from '@shared/app-identity';
type ImportMetaWithEnv = ImportMeta & { env?: { DEV?: boolean } };
@@ -29,6 +29,7 @@ export type TelemetryBuildChannel = 'dev' | 'canary' | 'stable';
export type TelemetryConfig = {
endpoint: string;
build: TelemetryBuildChannel;
+ flintEnv: TelemetryFlintEnv;
};
/**
@@ -41,6 +42,38 @@ export type TelemetryResolution =
| { enabled: true; config: TelemetryConfig }
| { enabled: false; reason: TelemetryDisabledReason };
+/**
+ * The Amplitude project an event belongs in, sent as the `flint_env` resource
+ * attribute. The relay keeps one project per environment and files each event
+ * under the one this names, so a developer's app and a pre-release never land
+ * in production's numbers. `build` stays on every event besides, for slicing
+ * within a project.
+ */
+export type TelemetryFlintEnv = 'prod' | 'staging' | 'dev' | 'local';
+
+/**
+ * Running from source is `local`. A packaged build that is not a release — a
+ * workflow_dispatch test build, or one packaged on a laptop — is `dev`, whatever
+ * channel it was built as. Only a released canary is `staging`, and only a
+ * released stable build is `prod`.
+ */
+export function flintEnvFor({
+ build,
+ release,
+}: {
+ build: TelemetryBuildChannel;
+ release: boolean;
+}): TelemetryFlintEnv {
+ if (build === 'dev') return 'local';
+ if (!release) return 'dev';
+ return build === 'canary' ? 'staging' : 'prod';
+}
+
+/** The project for the build that is actually running. */
+export function currentFlintEnv(): TelemetryFlintEnv {
+ return flintEnvFor({ build: telemetryBuildChannel(), release: IS_RELEASE_BUILD });
+}
+
export function telemetryBuildChannel(): TelemetryBuildChannel {
if (viteEnv?.DEV === true) return 'dev';
return IS_CANARY ? 'canary' : 'stable';
@@ -49,6 +82,8 @@ export function telemetryBuildChannel(): TelemetryBuildChannel {
/** Everything the decision depends on, so that it depends on nothing else. */
export type TelemetryEnvironment = {
build: TelemetryBuildChannel;
+ /** Whether the release workflow stamped this build as a published release. */
+ release: boolean;
env: NodeJS.ProcessEnv;
};
@@ -69,6 +104,7 @@ export type TelemetryEnvironment = {
*/
export function resolveTelemetryEnvironment({
build,
+ release,
env,
}: TelemetryEnvironment): TelemetryResolution {
const isDev = build === 'dev';
@@ -77,10 +113,21 @@ export function resolveTelemetryEnvironment({
}
const override = isDev ? env.SWITCHDASH_TELEMETRY_ENDPOINT?.trim() : undefined;
- return { enabled: true, config: { endpoint: override || TELEMETRY_RELAY_ENDPOINT, build } };
+ return {
+ enabled: true,
+ config: {
+ endpoint: override || TELEMETRY_RELAY_ENDPOINT,
+ build,
+ flintEnv: flintEnvFor({ build, release }),
+ },
+ };
}
/** The same decision for the build that is actually running. */
export function resolveTelemetryConfig(): TelemetryResolution {
- return resolveTelemetryEnvironment({ build: telemetryBuildChannel(), env: process.env });
+ return resolveTelemetryEnvironment({
+ build: telemetryBuildChannel(),
+ release: IS_RELEASE_BUILD,
+ env: process.env,
+ });
}
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/events.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/events.ts
index d0c047a5c..670e52cab 100644
--- a/console/apps/switch-console-desktop/src/main/core/telemetry/events.ts
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/events.ts
@@ -18,6 +18,13 @@ import type {
*/
export type TelemetryLocationKind = 'local' | 'remote' | 'unknown';
+/**
+ * Which kind of launch an event belongs to: `new` on an installation's very
+ * first launch, `updated` on the first launch after its version changed, and
+ * `same` otherwise. Counting `new` counts new installs; `updated`, upgrades.
+ */
+export type TelemetryInstallKind = 'new' | 'updated' | 'same';
+
/**
* Which CLI agent this concerns.
*
@@ -288,16 +295,11 @@ export type TelemetryHostSetupStepKind = HostSetupStepKind | 'unknown';
* and it is far easier to add an event than to take one away once dashboards
* depend on it.
*/
-/**
- * An event with no properties of its own. `Record<never, never>` rather than
- * `Record<string, never>`, whose `keyof` is `string` — which claims every
- * property name, including the ambient ones the emitter adds.
- */
-type NoProperties = Record<never, never>;
-
export type TelemetryEventMap = {
- /** The app started. Its interesting fields are the ambient ones. */
- app_launched: NoProperties;
+ /** The app started, and which kind of launch it was. */
+ app_launched: {
+ install_kind: TelemetryInstallKind;
+ };
agent_created: {
agent_type: TelemetryAgentType;
location: TelemetryLocationKind;
@@ -579,6 +581,8 @@ export type TelemetryEventMap = {
*/
telemetry_consent_changed: {
source: 'first_run' | 'settings';
+ /** This launch's kind, so a new install agreeing at first run is one filter. */
+ install_kind: TelemetryInstallKind;
};
};
@@ -593,7 +597,7 @@ export type TelemetryEventName = keyof TelemetryEventMap;
* checking does not apply — cannot ride along into a payload unnoticed.
*/
export const TELEMETRY_EVENT_PROPERTIES = {
- app_launched: [],
+ app_launched: ['install_kind'],
agent_created: ['agent_type', 'location', 'outcome', 'failure_reason', 'entry_point'],
session_started: [
'agent_type',
@@ -659,7 +663,7 @@ export const TELEMETRY_EVENT_PROPERTIES = {
onboarding_completed: [],
add_server_step: ['step', 'choice', 'first_run'],
renderer_crashed: [],
- telemetry_consent_changed: ['source'],
+ telemetry_consent_changed: ['source', 'install_kind'],
session_attached: ['agent_type', 'outcome'],
} as const satisfies { [K in TelemetryEventName]: readonly (keyof TelemetryEventMap[K])[] };
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.test.ts
new file mode 100644
index 000000000..b7da2f7aa
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.test.ts
@@ -0,0 +1,99 @@
+import { describe, expect, it, vi } from 'vitest';
+
+vi.mock('@main/db/kv', () => ({
+ KV: class {
+ private values = new Map<string, unknown>();
+ async get(key: string) {
+ return this.values.get(key) ?? null;
+ }
+ async set(key: string, value: unknown) {
+ this.values.set(key, structuredClone(value));
+ }
+ },
+}));
+
+const {
+ currentInternalFlag,
+ emailFromJwt,
+ forgetSessionAccount,
+ internalFrom,
+ isInternalEmail,
+ recordSessionAccount,
+} = await import('./internal-account');
+
+function jwtWith(payload: Record<string, unknown>): string {
+ const part = (value: object) => Buffer.from(JSON.stringify(value)).toString('base64url');
+ return `${part({ alg: 'HS256' })}.${part(payload)}.signature`;
+}
+
+describe('whether an account is staff', () => {
+ it.each([
+ ])('counts %s', (email) => {
+ expect(isInternalEmail(email)).toBe(true);
+ });
+
+ 'does not count %s',
+ (email) => {
+ expect(isInternalEmail(email)).toBe(false);
+ }
+ );
+});
+
+describe('whether the person is staff', () => {
+ it('is true when any account is', () => {
+ expect(internalFrom(['[email protected]', '[email protected]'])).toBe('true');
+ });
+
+ it('is false when only outside accounts are signed in', () => {
+ expect(internalFrom(['[email protected]', null])).toBe('false');
+ });
+
+ it('is unknown when no account says, including the one a local server signs in with', () => {
+ expect(internalFrom([])).toBe('unknown');
+ expect(internalFrom([null, '[email protected]'])).toBe('unknown');
+ });
+});
+
+describe('reading the account from a session', () => {
+ it('reads the email claim', () => {
+ expect(emailFromJwt(jwtWith({ sub: 'u1', email: '[email protected]' }))).toBe(
+ );
+ });
+
+ it('reads nothing from a malformed token or one without the claim', () => {
+ expect(emailFromJwt('not-a-jwt')).toBeNull();
+ expect(emailFromJwt(jwtWith({ sub: 'u1' }))).toBeNull();
+ });
+});
+
+describe('the answer kept beside each session', () => {
+ it('is recorded when a session is stored, and forgotten when it goes', async () => {
+ expect(await currentInternalFlag()).toBe('unknown');
+
+ await recordSessionAccount('outside', jwtWith({ email: '[email protected]' }));
+ expect(await currentInternalFlag()).toBe('false');
+
+ await recordSessionAccount('staff', jwtWith({ email: '[email protected]' }));
+ expect(await currentInternalFlag()).toBe('true');
+
+ await forgetSessionAccount('staff');
+ expect(await currentInternalFlag()).toBe('false');
+
+ await forgetSessionAccount('outside');
+ expect(await currentInternalFlag()).toBe('unknown');
+ });
+
+ it('follows a server signed in again as someone else', async () => {
+ await recordSessionAccount('server', jwtWith({ email: '[email protected]' }));
+ await recordSessionAccount('server', jwtWith({ email: '[email protected]' }));
+
+ expect(await currentInternalFlag()).toBe('false');
+ await forgetSessionAccount('server');
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.ts
new file mode 100644
index 000000000..e8544720a
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.ts
@@ -0,0 +1,98 @@
+import { LOCAL_SERVER_ADMIN_EMAIL } from '@main/core/managed-switch-server/constants';
+import { KV } from '@main/db/kv';
+
+/**
+ * Email domains whose accounts are the company's own staff. Their usage is
+ * internal: real, but not adoption, and product needs to tell the two apart.
+ */
+export const INTERNAL_EMAIL_DOMAINS: readonly string[] = ['sandboxaq.com', 'sandboxquantum.com'];
+
+/**
+ * Whether the person using this installation is staff, as the `flint_internal`
+ * resource attribute: `true`, `false`, or `unknown` when no account says. Only
+ * this travels; the address it was read from never leaves the machine.
+ */
+export type TelemetryInternal = 'true' | 'false' | 'unknown';
+
+export function isInternalEmail(email: string): boolean {
+ const domain = email.slice(email.lastIndexOf('@') + 1).toLowerCase();
+ return INTERNAL_EMAIL_DOMAINS.some((d) => domain === d || domain.endsWith(`.${d}`));
+}
+
+/**
+ * From the accounts this installation is signed in with. Any staff account
+ * makes it internal; otherwise any real account makes it external. The
+ * account a server Console runs for the user is a fixed local address that
+ * says nothing about the person, so it does not count.
+ */
+export function internalFrom(emails: readonly (string | null)[]): TelemetryInternal {
+ const real = emails.filter(
+ (email): email is string =>
+ email !== null && email.includes('@') && email.toLowerCase() !== LOCAL_SERVER_ADMIN_EMAIL
+ );
+ if (real.some(isInternalEmail)) return 'true';
+ return real.length > 0 ? 'false' : 'unknown';
+}
+
+/**
+ * The `email` claim of a gateway session, read without verifying it. That is
+ * enough here: the token was issued to this app by the server, and the result
+ * only labels usage data.
+ */
+export function emailFromJwt(jwt: string): string | null {
+ const parts = jwt.split('.');
+ if (parts.length !== 3) return null;
+ try {
+ const payload = JSON.parse(Buffer.from(parts[1], 'base64url').toString('utf8')) as {
+ email?: unknown;
+ };
+ return typeof payload.email === 'string' ? payload.email : null;
+ } catch {
+ return null;
+ }
+}
+
+/**
+ * Each server's answer, kept beside its session rather than read from it.
+ * Working it out from the stored session would decrypt every server's cookie
+ * on every event, and a failed decrypt deletes the cookie: a keychain that is
+ * briefly locked would sign people out of servers they were not even using.
+ * So it is worked out once, from the token in hand, when a session is stored.
+ */
+const store = new KV<{ accounts: Record<string, TelemetryInternal> }>('telemetry-accounts');
+
+let writes: Promise<void> = Promise.resolve();
+
+function update(change: (accounts: Record<string, TelemetryInternal>) => void): Promise<void> {
+ writes = writes.then(async () => {
+ const accounts = (await store.get('accounts')) ?? {};
+ change(accounts);
+ await store.set('accounts', accounts);
+ });
+ return writes;
+}
+
+/** Record the account a server's session was issued to. */
+export function recordSessionAccount(serverId: string, jwt: string): Promise<void> {
+ return update((accounts) => {
+ accounts[serverId] = internalFrom([emailFromJwt(jwt)]);
+ });
+}
+
+/** Forget a server's account when its session goes. */
+export function forgetSessionAccount(serverId: string): Promise<void> {
+ return update((accounts) => {
+ delete accounts[serverId];
+ });
+}
+
+/**
+ * Across every server this installation holds a session for. A session stored
+ * before this was recorded reads as unknown until it is next renewed or
+ * signed in again.
+ */
+export async function currentInternalFlag(): Promise<TelemetryInternal> {
+ const answers = Object.values((await store.get('accounts')) ?? {});
+ if (answers.includes('true')) return 'true';
+ return answers.includes('false') ? 'false' : 'unknown';
+}
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.test.ts
new file mode 100644
index 000000000..bc04e4cde
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.test.ts
@@ -0,0 +1,56 @@
+import { describe, expect, it, vi } from 'vitest';
+
+vi.mock('@main/db/kv', () => ({
+ KV: class {
+ private values = new Map<string, unknown>();
+ async get(key: string) {
+ return this.values.get(key) ?? null;
+ }
+ async set(key: string, value: unknown) {
+ this.values.set(key, structuredClone(value));
+ }
+ },
+}));
+
+const { installKindFor, recordLaunch } = await import('./launch-history');
+
+describe('which kind of launch this is', () => {
+ it('is new on an installation’s very first launch, so counting it counts installs', () => {
+ expect(installKindFor({ lastVersion: null, version: '1.2.0', databaseExisted: false })).toBe(
+ 'new'
+ );
+ });
+
+ it('is an upgrade on the first launch at a different version', () => {
+ expect(installKindFor({ lastVersion: '1.1.0', version: '1.2.0', databaseExisted: true })).toBe(
+ 'updated'
+ );
+ });
+
+ it('is the same on a relaunch', () => {
+ expect(installKindFor({ lastVersion: '1.2.0', version: '1.2.0', databaseExisted: true })).toBe(
+ 'same'
+ );
+ });
+
+ it('is an upgrade, never new, for an installation that predates this tracking', () => {
+ // It has a database but no record: it was installed at some earlier version,
+ // and this is its first launch at one that records.
+ expect(installKindFor({ lastVersion: null, version: '1.2.0', databaseExisted: true })).toBe(
+ 'updated'
+ );
+ });
+});
+
+describe('canary and stable, which share one database', () => {
+ it('do not read a switch between them as an upgrade', async () => {
+ const launch = (channel: 'canary' | 'stable', version: string) =>
+ recordLaunch({ version, channel, databaseExisted: true });
+
+ expect(await launch('stable', '1.4.0')).toBe('updated');
+ expect(await launch('canary', '1.5.0-canary.2')).toBe('updated');
+ expect(await launch('stable', '1.4.0')).toBe('same');
+ expect(await launch('canary', '1.5.0-canary.2')).toBe('same');
+ expect(await launch('canary', '1.5.0-canary.3')).toBe('updated');
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.ts
new file mode 100644
index 000000000..97d2b8e20
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.ts
@@ -0,0 +1,69 @@
+import { KV } from '@main/db/kv';
+import { log } from '@main/lib/logger';
+import type { TelemetryInstallKind } from './events';
+
+/**
+ * Which kind of launch this is, from what the last launch recorded.
+ *
+ * `new` is the first launch an installation ever makes, so counting it counts
+ * new installs; `updated` is the first launch on a different version than the
+ * last one, an upgrade; `same` is everything else. An installation with a
+ * database but no record predates this tracking, and its first launch with it
+ * is on a version it was not installed at, so it reads as `updated`.
+ */
+export function installKindFor({
+ lastVersion,
+ version,
+ databaseExisted,
+}: {
+ lastVersion: string | null;
+ version: string;
+ databaseExisted: boolean;
+}): TelemetryInstallKind {
+ if (lastVersion === null) return databaseExisted ? 'updated' : 'new';
+ return lastVersion === version ? 'same' : 'updated';
+}
+
+/**
+ * One record per channel: canary and stable share a database, and comparing
+ * one's version with the other's would read every switch between them as an
+ * upgrade.
+ */
+const store = new KV<Record<string, string>>('telemetry-launches');
+
+let current: TelemetryInstallKind | null = null;
+
+/**
+ * Record this launch and say which kind it is. Recorded on every launch,
+ * whether or not usage is shared: it is local, and the next launch has to know
+ * this one happened to tell an upgrade from a relaunch.
+ */
+export async function recordLaunch({
+ version,
+ channel,
+ databaseExisted,
+}: {
+ version: string;
+ channel: 'canary' | 'stable';
+ databaseExisted: boolean;
+}): Promise<TelemetryInstallKind> {
+ const key = `lastLaunchedVersion.${channel}`;
+ const lastVersion = (await store.get(key)) ?? null;
+ current = installKindFor({ lastVersion, version, databaseExisted });
+ await store.set(key, version);
+ return current;
+}
+
+/**
+ * This launch's kind. Boot records it before any window opens, so nothing a
+ * person does can ask first; if something does, it is said in the log rather
+ * than passed off as a real answer, and reads as `same`, the kind that claims
+ * nothing.
+ */
+export function currentInstallKind(): TelemetryInstallKind {
+ if (current === null) {
+ log.warn('telemetry: install kind read before this launch was recorded');
+ return 'same';
+ }
+ return current;
+}
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/relay-client.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/relay-client.test.ts
index 784acf261..de1e8c2d3 100644
--- a/console/apps/switch-console-desktop/src/main/core/telemetry/relay-client.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/relay-client.test.ts
@@ -1,4 +1,3 @@
-import { durationMs } from '@tooling/utils/telemetry-duration';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import type { TelemetryConfig } from './config';
import {
@@ -14,12 +13,15 @@ const CONTEXT: TelemetryContext = {
osType: 'darwin',
osVersion: '24.3.0',
build: 'stable',
+ flintEnv: 'prod',
+ internal: 'false',
timeMs: 1_700_000_000_000,
};
const CONFIG: TelemetryConfig = {
endpoint: 'https://telemetry.example/v1/logs',
build: 'stable',
+ flintEnv: 'prod',
};
type Attribute = { key: string; value: { stringValue: string } };
@@ -63,18 +65,20 @@ const SESSION_STARTED = {
describe('the record that gets built', () => {
it('carries the client id the relay requires, as a resource attribute', () => {
// Without it the relay drops the whole payload — and still answers 200.
- const payload = buildOtlpPayload('app_launched', {}, CONTEXT);
+ const payload = buildOtlpPayload('app_launched', { install_kind: 'same' }, CONTEXT);
expect(resourceAttributes(payload)['flint.client_id']).toBe(CONTEXT.clientId);
});
it('says which app and version it is, for both destinations', () => {
- const payload = buildOtlpPayload('app_launched', {}, CONTEXT);
+ const payload = buildOtlpPayload('app_launched', { install_kind: 'same' }, CONTEXT);
expect(resourceAttributes(payload)).toEqual({
'service.name': 'switch-console',
'service.version': '1.2.3',
'flint.client_id': CONTEXT.clientId,
+ flint_env: 'prod',
+ flint_internal: 'false',
'os.type': 'darwin',
'os.version': '24.3.0',
});
@@ -90,7 +94,7 @@ describe('the record that gets built', () => {
// The attribute alone gets a record past the relay's "is it named?" filter
// and then dropped by an exporter that only reads the field — silently, and
// with a 200 at every step. Both, or nothing arrives and nothing says so.
- const payload = buildOtlpPayload('app_launched', {}, CONTEXT);
+ const payload = buildOtlpPayload('app_launched', { install_kind: 'same' }, CONTEXT);
expect(logRecord(payload).eventName).toBe('switch_console.app_launched');
});
@@ -128,19 +132,20 @@ describe('the record that gets built', () => {
expect(JSON.stringify(payload)).not.toContain('secret-project');
});
- it('sends an event with no properties of its own as just its name and build', () => {
- const payload = buildOtlpPayload('app_launched', {}, CONTEXT);
+ it('sends an event as its name, its build and its own properties, and nothing else', () => {
+ const payload = buildOtlpPayload('app_launched', { install_kind: 'new' }, CONTEXT);
expect(logAttributes(payload)).toEqual({
'event.name': 'switch_console.app_launched',
build: 'stable',
+ install_kind: 'new',
});
});
it('carries a body, so the event is not a blank line in the log sink', () => {
// The relay forwards the same record to Datadog, which reads the body as
// the message. It repeats the name rather than adding anything new.
- const payload = buildOtlpPayload('app_launched', {}, CONTEXT);
+ const payload = buildOtlpPayload('app_launched', { install_kind: 'same' }, CONTEXT);
expect(logRecord(payload).body).toEqual({ stringValue: 'switch_console.app_launched' });
});
@@ -164,7 +169,36 @@ describe('the record that gets built', () => {
expect(rawLogAttributes(payload).connected_to_room).toEqual({ boolValue: false });
});
- it('carries a count as a number, so it can be summed at the far end', () => {
+ it.each(['true', 'false', 'unknown'] as const)(
+ 'says whether the person is staff (%s), and nothing about who',
+ (internal) => {
+ const payload = buildOtlpPayload(
+ 'app_launched',
+ { install_kind: 'same' },
+ { ...CONTEXT, internal }
+ );
+
+ expect(resourceAttributes(payload).flint_internal).toBe(internal);
+ expect(JSON.stringify(payload)).not.toContain('@');
+ }
+ );
+
+ it.each(['prod', 'staging', 'dev', 'local'] as const)(
+ 'says the event belongs in the %s Amplitude project',
+ (flintEnv) => {
+ // The relay picks the project from this, beside the client id it already
+ // requires on the resource.
+ const payload = buildOtlpPayload(
+ 'app_launched',
+ { install_kind: 'same' },
+ { ...CONTEXT, flintEnv }
+ );
+
+ expect(resourceAttributes(payload).flint_env).toBe(flintEnv);
+ }
+ );
+
+ it('carries a yes as a boolean, not as the word', () => {
const payload = buildOtlpPayload(
'session_started',
{ ...SESSION_STARTED, has_initial_prompt: true },
@@ -172,43 +206,48 @@ describe('the record that gets built', () => {
);
expect(rawLogAttributes(payload).has_initial_prompt).toEqual({ boolValue: true });
- expect(rawLogAttributes(payload).entry_point).toEqual({ stringValue: 'sidebar' });
});
- it('refuses a count that is not a real number rather than sending null', () => {
- // NaN and Infinity both serialise to `null` in JSON, which at the far end
- // is indistinguishable from a property that was never sent.
- const broken = { ...SESSION_STARTED, has_initial_prompt: Number.NaN } as never;
+ it('carries a count as a number, so it can be summed at the far end', () => {
+ // A count sent as text arrives as a category: it can be grouped by, never
+ // summed or averaged.
+ const payload = buildOtlpPayload(
+ 'search_performed',
+ { status: 'ok', result_count: 3 },
+ CONTEXT
+ );
- expect(() => buildOtlpPayload('session_started', broken, CONTEXT)).toThrow(/non-finite/);
+ expect(rawLogAttributes(payload).result_count).toEqual({ doubleValue: 3 });
});
+ it('carries a count of zero as a number, not as a property that was never sent', () => {
+ const payload = buildOtlpPayload(
+ 'search_performed',
+ { status: 'ok', result_count: 0 },
+ CONTEXT
+ );
+
+ expect(rawLogAttributes(payload).result_count).toEqual({ doubleValue: 0 });
+ });
+
+ it.each([Number.NaN, Number.POSITIVE_INFINITY, Number.NEGATIVE_INFINITY])(
+ 'refuses a count of %s rather than sending null',
+ (count) => {
+ // NaN and Infinity both serialise to `null` in JSON, which at the far end
+ // is indistinguishable from a property that was never sent.
+ const broken = { status: 'ok', result_count: count } as const;
+
+ expect(() => buildOtlpPayload('search_performed', broken, CONTEXT)).toThrow(/non-finite/);
+ }
+ );
+
it('stamps the time in nanoseconds, which is what OTLP counts in', () => {
- const payload = buildOtlpPayload('app_launched', {}, CONTEXT);
+ const payload = buildOtlpPayload('app_launched', { install_kind: 'same' }, CONTEXT);
const record = logRecord(payload);
expect(record.timeUnixNano).toBe('1700000000000000000');
expect(record.observedTimeUnixNano).toBe('1700000000000000000');
});
-
- it('stays far under the attribute count the relay drops a record for', () => {
- // The guard drops any record with more than 128 attributes.
- const payload = buildOtlpPayload(
- 'agent_cli_action',
- {
- agent_type: 'claude',
- target: 'remote',
- install_method: 'unspecified',
- action: 'install',
- outcome: 'success',
- failure_reason: 'none',
- duration_ms: durationMs(4200),
- },
- CONTEXT
- );
-
- expect(Object.keys(logAttributes(payload)).length).toBeLessThan(16);
- });
});
describe('sending it', () => {
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/relay-client.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/relay-client.ts
index 8dcaeced1..f7553ae6c 100644
--- a/console/apps/switch-console-desktop/src/main/core/telemetry/relay-client.ts
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/relay-client.ts
@@ -1,9 +1,10 @@
-import type { TelemetryConfig } from './config';
+import type { TelemetryConfig, TelemetryFlintEnv } from './config';
import {
TELEMETRY_EVENT_PROPERTIES,
type TelemetryEventMap,
type TelemetryEventName,
} from './events';
+import type { TelemetryInternal } from './internal-account';
const SEND_TIMEOUT_MS = 10_000;
@@ -69,6 +70,10 @@ export type TelemetryContext = {
osType: string;
osVersion: string;
build: TelemetryConfig['build'];
+ /** The Amplitude project the event belongs in, which the relay routes on. */
+ flintEnv: TelemetryFlintEnv;
+ /** Whether the person is staff, from their account's email domain. */
+ internal: TelemetryInternal;
timeMs: number;
};
@@ -163,6 +168,8 @@ export function buildOtlpPayload<K extends TelemetryEventName>(
'service.name': SERVICE_NAME,
'service.version': context.appVersion,
'flint.client_id': context.clientId,
+ flint_env: context.flintEnv,
+ flint_internal: context.internal,
'os.type': context.osType,
'os.version': context.osVersion,
}),
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/release-stamp.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/release-stamp.test.ts
new file mode 100644
index 000000000..837dfe110
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/release-stamp.test.ts
@@ -0,0 +1,37 @@
+import { readFileSync } from 'node:fs';
+import { dirname, resolve } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { describe, expect, it } from 'vitest';
+
+/**
+ * A packaged build reports its usage to production's Amplitude project only
+ * when the release workflow stamped it `VITE_RELEASE=1`; anything else reports
+ * to dev's. So a release whose build step lost the stamp would file every real
+ * user under dev, with nothing anywhere reporting a fault. This holds the
+ * workflow to stamping every build step, and only on a tag.
+ */
+const workflow = readFileSync(
+ resolve(
+ dirname(fileURLToPath(import.meta.url)),
+ '../../../../../../../.github/workflows/switch-console-release.yml'
+ ),
+ 'utf8'
+);
+
+describe('the release workflow', () => {
+ it('stamps every build step that sets the channel', () => {
+ const buildSteps = workflow.match(/^\s+VITE_BUILD: .*$/gm) ?? [];
+ const stamps = workflow.match(
+ /^\s+VITE_RELEASE: \$\{\{ needs\.resolve\.outputs\.vite_release \}\}$/gm
+ );
+
+ expect(buildSteps.length).toBeGreaterThan(0);
+ expect(stamps?.length).toBe(buildSteps.length);
+ });
+
+ it('marks a build a release on a tag and on nothing else', () => {
+ expect(workflow).toMatch(/refs\/tags\/\*\) vite_release=1 ;;/);
+ expect(workflow).toMatch(/\*\) vite_release= ;;/);
+ expect(workflow).toMatch(/echo "vite_release=\$vite_release"/);
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.test.ts
index 6c4a601b5..de3f18210 100644
--- a/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.test.ts
@@ -80,7 +80,7 @@ describe('the consent gate', () => {
it('makes no request at all when consent is off', async () => {
vi.mocked(isTelemetryAllowed).mockResolvedValue(false);
- await telemetryService.track('app_launched', {});
+ await telemetryService.track('app_launched', { install_kind: 'same' });
expect(fetchMock).not.toHaveBeenCalled();
});
@@ -88,8 +88,8 @@ describe('the consent gate', () => {
it('asks the gate again for every event, so revoking consent stops the next one', async () => {
vi.mocked(isTelemetryAllowed).mockResolvedValueOnce(true).mockResolvedValueOnce(false);
- await telemetryService.track('app_launched', {});
- await telemetryService.track('app_launched', {});
+ await telemetryService.track('app_launched', { install_kind: 'same' });
+ await telemetryService.track('app_launched', { install_kind: 'same' });
expect(isTelemetryAllowed).toHaveBeenCalledTimes(2);
expect(fetchMock).toHaveBeenCalledTimes(1);
@@ -130,7 +130,7 @@ describe('a build that cannot send', () => {
vi.mocked(isTelemetryAllowed).mockResolvedValue(true);
const service = await freshService();
- await service.track('app_launched', {});
+ await service.track('app_launched', { install_kind: 'same' });
expect(fetchMock).not.toHaveBeenCalled();
expect(isTelemetryAllowed).not.toHaveBeenCalled();
@@ -144,9 +144,9 @@ describe('a build that cannot send', () => {
delete process.env.SWITCHDASH_TELEMETRY_DEV;
const service = await freshService();
- await service.track('app_launched', {});
- await service.track('app_launched', {});
- await service.track('app_launched', {});
+ await service.track('app_launched', { install_kind: 'same' });
+ await service.track('app_launched', { install_kind: 'same' });
+ await service.track('app_launched', { install_kind: 'same' });
expect(log.info).toHaveBeenCalledTimes(1);
});
@@ -217,7 +217,7 @@ describe('the payload', () => {
const now = vi.spyOn(Date, 'now');
now.mockReturnValueOnce(1_000).mockReturnValue(9_000);
- await telemetryService.track('app_launched', {});
+ await telemetryService.track('app_launched', { install_kind: 'same' });
expect(sentRecord().timeUnixNano).toBe('1000000000');
now.mockRestore();
@@ -255,7 +255,7 @@ describe('the payload', () => {
process.platform as string
];
- await telemetryService.track('app_launched', {});
+ await telemetryService.track('app_launched', { install_kind: 'same' });
expect(sentResource()['os.type']).toBe(expected ?? 'other');
expect(sentResource()['os.version']).toBe(release());
@@ -273,6 +273,8 @@ describe('the payload', () => {
expect(Object.keys(sentResource()).sort()).toEqual([
'flint.client_id',
+ 'flint_env',
+ 'flint_internal',
'os.type',
'os.version',
'service.name',
@@ -296,7 +298,7 @@ describe('failure', () => {
it('never reaches the caller, and is logged with a code', async () => {
fetchMock.mockResolvedValue({ ok: false, status: 503 } as Response);
- expect(() => trackEvent('app_launched', {})).not.toThrow();
+ expect(() => trackEvent('app_launched', { install_kind: 'same' })).not.toThrow();
await vi.waitFor(() =>
expect(log.warn).toHaveBeenCalledWith(
'telemetry: event not sent',
@@ -308,7 +310,7 @@ describe('failure', () => {
it('reports a network failure as such rather than as a send', async () => {
fetchMock.mockRejectedValue(new Error('getaddrinfo ENOTFOUND'));
- trackEvent('app_launched', {});
+ trackEvent('app_launched', { install_kind: 'same' });
await vi.waitFor(() =>
expect(log.warn).toHaveBeenCalledWith(
@@ -321,7 +323,7 @@ describe('failure', () => {
it('does not retry', async () => {
fetchMock.mockResolvedValue({ ok: false, status: 500 } as Response);
- await telemetryService.track('app_launched', {}).catch(() => {});
+ await telemetryService.track('app_launched', { install_kind: 'same' }).catch(() => {});
expect(fetchMock).toHaveBeenCalledTimes(1);
});
diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.ts
index 6ea614b54..bc2b25cb8 100644
--- a/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.ts
+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.ts
@@ -5,6 +5,7 @@ import { resolveTelemetryConfig, type TelemetryDisabledReason } from './config';
import { isTelemetryAllowed } from './consent';
import type { TelemetryEventMap, TelemetryEventName } from './events';
import { getInstallId } from './install-id';
+import type { TelemetryInternal } from './internal-account';
import { buildOtlpPayload, postTelemetryEvent, TelemetrySendError } from './relay-client';
/** OpenTelemetry's names for what `process.platform` calls darwin/win32/linux. */
@@ -17,6 +18,15 @@ const OS_TYPES: Record<string, string> = {
class TelemetryService {
private reportedDisabled = new Set<TelemetryDisabledReason>();
private appVersion: Promise<string> | null = null;
+ /**
+ * Whether the person is staff. Registered at boot rather than imported,
+ * because it reads the servers store, which reports through this service.
+ */
+ private internalSource: () => Promise<TelemetryInternal> = async () => 'unknown';
+
+ setInternalSource(source: () => Promise<TelemetryInternal>): void {
+ this.internalSource = source;
+ }
/**
* Send one event, if this build can send and the user has said yes.
@@ -48,6 +58,8 @@ class TelemetryService {
osType: OS_TYPES[process.platform] ?? 'other',
osVersion: release(),
build: resolution.config.build,
+ flintEnv: resolution.config.flintEnv,
+ internal: await this.resolveInternal(),
timeMs,
});
@@ -68,6 +80,17 @@ class TelemetryService {
return resolveTelemetryConfig().enabled && (await isTelemetryAllowed());
}
+ /** `unknown` when it cannot be read: that is the honest answer, and telemetry
+ * never gets to fail a user's action. */
+ private async resolveInternal(): Promise<TelemetryInternal> {
+ try {
+ return await this.internalSource();
+ } catch (error) {
+ log.warn('telemetry: could not tell whether the account is internal', { error });
+ return 'unknown';
+ }
+ }
+
private resolveVersion(): Promise<string> {
this.appVersion ??= resolveAppVersion();
return this.appVersion;
diff --git a/console/apps/switch-console-desktop/src/main/db/client.ts b/console/apps/switch-console-desktop/src/main/db/client.ts
index d11a58206..bb67af0fb 100644
--- a/console/apps/switch-console-desktop/src/main/db/client.ts
+++ b/console/apps/switch-console-desktop/src/main/db/client.ts
@@ -1,3 +1,4 @@
+import { existsSync } from 'node:fs';
import Database from 'better-sqlite3';
import { drizzle } from 'drizzle-orm/better-sqlite3';
import { resolveDatabasePath } from './path';
@@ -8,7 +9,16 @@ export type DrizzleTx = Parameters<AppDb['transaction']>[0] extends (tx: infer T
? T
: never;
-export const sqlite = new Database(resolveDatabasePath());
+const databasePath = resolveDatabasePath();
+
+/**
+ * Whether this installation already had a database before the app opened it:
+ * a relaunch or an upgrade, rather than the first launch ever. Read before
+ * opening, which creates the file.
+ */
+export const databaseExistedAtStart = existsSync(databasePath);
+
+export const sqlite = new Database(databasePath);
sqlite.pragma('journal_mode = WAL');
sqlite.pragma('busy_timeout = 5000');
diff --git a/console/apps/switch-console-desktop/src/main/index.ts b/console/apps/switch-console-desktop/src/main/index.ts
index 8c95c3dd2..9aafd68d7 100644
--- a/console/apps/switch-console-desktop/src/main/index.ts
+++ b/console/apps/switch-console-desktop/src/main/index.ts
@@ -4,7 +4,7 @@ import { setTimeout as delay } from 'node:timers/promises';
import { config as dotenvConfig } from 'dotenv';
import { app, BrowserWindow, dialog, ipcMain, powerMonitor } from 'electron';
import dockIcon from '@/assets/images/switch-console/icon-dock.png?asset';
-import { PRODUCT_NAME } from '@shared/app-identity';
+import { IS_CANARY, PRODUCT_NAME } from '@shared/app-identity';
import { registerRPCRouter } from '@shared/lib/ipc/rpc';
import { flushPendingDeeplink, setupDeeplinks } from './app/deeplinks';
import { setupApplicationMenu } from './app/menu';
@@ -15,6 +15,7 @@ import { setAgentStorageMigrationReady } from './core/agents/agent-storage-migra
import { migrateAgentStorage } from './core/agents/migrate-agent-storage';
import { initializeRemoteDiscovery, initializeRemoteWatchers } from './core/agents/remote-watcher';
import { appService } from './core/app/service';
+import { resolveAppVersion } from './core/app/utils';
import { controlService } from './core/control-api/control-service';
import { localDependencyManager } from './core/dependencies/dependency-managers';
import { embeddedControllerService } from './core/embedded-controller/embedded-controllers';
@@ -33,8 +34,10 @@ import { searchService } from './core/search/search-service';
import { appSettingsService } from './core/settings/settings-service';
import { sshConnectionManager } from './core/ssh/lifecycle/production-ssh-connection-manager';
import { autoSessionWatcher } from './core/switch-rooms/auto-session-watcher';
+import { currentInternalFlag } from './core/telemetry/internal-account';
+import { recordLaunch } from './core/telemetry/launch-history';
import { registerTelemetryListeners } from './core/telemetry/telemetry-listeners';
-import { trackEvent } from './core/telemetry/telemetry-service';
+import { telemetryService, trackEvent } from './core/telemetry/telemetry-service';
import { updateService } from './core/updates/update-service';
import { viewStateService } from './core/view-state/view-state-service';
import { reconcileAllWorkspaces } from './core/workspaces/reconcile-workspaces';
@@ -144,7 +147,15 @@ void app.whenReady().then(async () => {
// After the settings store, which owns the consent gate every event asks
// before it is sent, and never before the database it is read from.
registerTelemetryListeners();
- trackEvent('app_launched', {});
+ telemetryService.setInternalSource(currentInternalFlag);
+ trackEvent('app_launched', {
+ install_kind: await recordLaunch({
+ version: await resolveAppVersion(),
+ channel: IS_CANARY ? 'canary' : 'stable',
+ // Imported here, not at the top: the client opens the database on import.
+ databaseExisted: (await import('./db/client')).databaseExistedAtStart,
+ }),
+ });
// Kept off the boot path: this can open an SSH/SFTP connection per remote
// agent, so awaiting it here delayed the window opening. Session relaunch below
diff --git a/console/apps/switch-console-desktop/src/shared/app-identity.ts b/console/apps/switch-console-desktop/src/shared/app-identity.ts
index a21619829..84cc7e710 100644
--- a/console/apps/switch-console-desktop/src/shared/app-identity.ts
+++ b/console/apps/switch-console-desktop/src/shared/app-identity.ts
@@ -1,8 +1,14 @@
-type ImportMetaWithEnv = ImportMeta & { env?: { DEV?: boolean; VITE_BUILD?: string } };
+type ImportMetaWithEnv = ImportMeta & {
+ env?: { DEV?: boolean; VITE_BUILD?: string; VITE_RELEASE?: string };
+};
const env = (import.meta as ImportMetaWithEnv).env;
const isDev = env?.DEV === true;
const isCanary = env?.VITE_BUILD === 'canary';
+// Stamped by the release workflow on a tagged build, and only there: a
+// workflow_dispatch test build, or one packaged on a laptop, is not a release
+// however it is otherwise configured.
+const isRelease = env?.VITE_RELEASE === '1';
// What a user reads, what the app calls itself to the OS, and where data lives
// are three different names here, and only the first one moved.
@@ -38,6 +44,7 @@ export const USER_DATA_DIR_NAME = isDev ? 'switchdash-dev' : 'switchdash';
export const UPDATE_CHANNEL = isCanary ? 'v1-canary' : 'v1-stable';
export const ARTIFACT_PREFIX = isCanary ? 'switch-console-canary' : 'switch-console';
export const IS_CANARY = isCanary;
+export const IS_RELEASE_BUILD = isRelease;
// GitHub repo the desktop app publishes releases to / reads auto-updates from.
// The repo is public, so the feed is read unauthenticated. Mirrored in
diff --git a/core/switch_core/bridges/collaboration/lifecycle_service.py b/core/switch_core/bridges/collaboration/lifecycle_service.py
index e8c4e5256..606ad3b04 100644
--- a/core/switch_core/bridges/collaboration/lifecycle_service.py
+++ b/core/switch_core/bridges/collaboration/lifecycle_service.py
@@ -1272,17 +1272,21 @@ class CollaborationBridgeLifecycleService:
},
)
- # `enabled`, not just `is not None`: off is a real service with a
- # discarding sink, so testing for None alone spends the claim below on
- # a deployment reporting nothing.
- if self._telemetry is None or not self._telemetry.enabled:
- return
# The first successful connect for this bridge, ever. Claimed against
# the bridge id so restarting a working bridge does not re-report a
- # setup that happened months ago.
- if not await claim_milestone(
+ # setup that happened months ago — and claimed whether or not telemetry
+ # is on, unlike the milestones. `connector_added` measures setup time,
+ # configuration saved to first connect; a first connect nobody was told
+ # about is setup that went unmeasured, and reporting it the day
+ # telemetry is switched on would report the connector's whole age.
+ first_connect = await claim_milestone(
self._session_factory, f"connector_added:{bridge_id}"
- ):
+ )
+ # `enabled`, not just `is not None`: off is a real service with a
+ # discarding sink.
+ if self._telemetry is None or not self._telemetry.enabled:
+ return
+ if not first_connect:
# This bridge has reported, but the deployment-wide milestone may
# not have. `emit_milestone` is itself once-ever.
if not preconfigured:
diff --git a/core/switch_core/config.py b/core/switch_core/config.py
index 6ea03c4fe..3bd15fe50 100644
--- a/core/switch_core/config.py
+++ b/core/switch_core/config.py
@@ -339,6 +339,22 @@ class SwitchConfig(BaseSettings):
# to how long this process has been up.
telemetry_snapshot_interval_hours: float = 24.0
+ # Which Amplitude project this deployment's usage lands in, sent as
+ # `flint_env`. The relay keeps one project per environment and files an
+ # event naming none, or `prod`, under production — which is what every
+ # customer's deployment is, and why it is the default. Our own non-customer
+ # servers say so here: `dev` for the development deployment, `local` for a
+ # developer's machine (`just init-env` writes it), so their usage never
+ # reads as adoption. Any other value is refused at startup, because the
+ # relay drops an event naming an environment it has no project for.
+ telemetry_environment: Literal["prod", "staging", "dev", "local"] = "prod"
+
+ # Whether this deployment is one of the company's own, sent as
+ # `flint_internal` so staff usage can be told from adoption. A customer's
+ # deployment never sets it. Within a deployment, staff accounts are counted
+ # separately by email domain whether or not this is set.
+ telemetry_internal: bool = False
+
server_host: str = "0.0.0.0"
server_port: int = 8000
diff --git a/core/switch_core/migrations/versions/2575637e78d4_claim_connector_added_for_existing_connectors.py b/core/switch_core/migrations/versions/2575637e78d4_claim_connector_added_for_existing_connectors.py
new file mode 100644
index 000000000..13de526a2
--- /dev/null
+++ b/core/switch_core/migrations/versions/2575637e78d4_claim_connector_added_for_existing_connectors.py
@@ -0,0 +1,45 @@
+"""claim connector_added for every connector that already exists
+
+Revision ID: 2575637e78d4
+Revises: e4a7c1d93b25
+
+`connector_added` reports a connector's setup time — configuration saved to
+first connect — once, on that first connect, guarded by a claim in
+`telemetry_milestones`. The claim used to be taken only while telemetry was
+on, so a connector that existed before telemetry started reporting claimed it
+on its first connect afterwards and reported its whole age as setup time:
+millions of seconds.
+
+The claim is now taken whether telemetry is on or not. A connector that exists
+at this migration has, as far as anything can tell, already made its first
+connect unobserved, so its claim is taken here and it never reports
+`connector_added`. One that never managed to connect is the exception it
+costs: if it connects for the first time later, that goes unreported too.
+"""
+
+from collections.abc import Sequence
+
+from alembic import op
+
+revision: str = "2575637e78d4"
+down_revision: str | Sequence[str] | None = "e4a7c1d93b25"
+branch_labels: str | Sequence[str] | None = None
+depends_on: str | Sequence[str] | None = None
+
+
+def upgrade() -> None:
+ # Runs as the schema owner, which row-level security does not restrict, so
+ # every tenant's connectors are seen.
+ op.execute(
+ """
+ INSERT INTO telemetry_milestones (name)
+ SELECT 'connector_added:' || id FROM collaboration_bridges
+ ON CONFLICT (name) DO NOTHING
+ """
+ )
+
+
+def downgrade() -> None:
+ # The claims taken here cannot be told apart from ones a real report took,
+ # and keeping them only keeps those connectors quiet.
+ pass
diff --git a/core/switch_core/telemetry/catalogue.py b/core/switch_core/telemetry/catalogue.py
index 22ffcabd3..8b2f4a544 100644
--- a/core/switch_core/telemetry/catalogue.py
+++ b/core/switch_core/telemetry/catalogue.py
@@ -150,8 +150,13 @@ _SNAPSHOT_COUNTS = (
"tenant_count",
"tenant_failed_count",
"user_count",
+ "user_internal_count",
+ # Switch accounts, not chat identities — see `chat_identity_*` for those.
"user_active_1d",
"user_active_7d",
+ "chat_identity_count",
+ "chat_identity_active_1d",
+ "chat_identity_active_7d",
# `room_count` is the headline: rooms a *person* made. The other two keep
# agent scratch rooms and adopted channels out of it.
"room_count",
diff --git a/core/switch_core/telemetry/internal.py b/core/switch_core/telemetry/internal.py
new file mode 100644
index 000000000..30a20c863
--- /dev/null
+++ b/core/switch_core/telemetry/internal.py
@@ -0,0 +1,33 @@
+"""Whose usage is the company's own.
+
+Staff use the product too, and their usage is real but is not adoption, so the
+numbers have to be able to leave it out. Two signals, neither an identifier:
+a deployment says it is one of the company's own (`TELEMETRY_INTERNAL`), and
+the daily snapshot counts the accounts on the company's email domains. The
+same domains mark a staff member in Switch Console.
+"""
+
+from __future__ import annotations
+
+from sqlalchemy import ColumnElement, func, or_
+from sqlalchemy.orm import InstrumentedAttribute
+
+INTERNAL_EMAIL_DOMAINS: tuple[str, ...] = ("sandboxaq.com", "sandboxquantum.com")
+
+
+def internal_email_condition(
+ email: ColumnElement[str] | InstrumentedAttribute[str],
+) -> ColumnElement[bool]:
+ """Whether `email` is on one of `INTERNAL_EMAIL_DOMAINS` or a subdomain of
+ one, as SQL, so accounts are counted without loading their addresses."""
+ lowered = func.lower(email)
+ return or_(
+ *(
+ pattern
+ for internal in INTERNAL_EMAIL_DOMAINS
+ for pattern in (
+ lowered.like(f"%@{internal}"),
+ lowered.like(f"%@%.{internal}"),
+ )
+ )
+ )
diff --git a/core/switch_core/telemetry/service.py b/core/switch_core/telemetry/service.py
index 1391177b9..2fcc125fc 100644
--- a/core/switch_core/telemetry/service.py
+++ b/core/switch_core/telemetry/service.py
@@ -43,6 +43,8 @@ class TelemetryService:
service_name: str,
version: str | None,
environment: str | None,
+ telemetry_environment: str,
+ telemetry_internal: bool,
session_factory: async_sessionmaker[AsyncSession] | None = None,
installed_at: datetime | None = None,
) -> None:
@@ -55,6 +57,13 @@ class TelemetryService:
self._resource = {
"service.name": service_name,
"flint.client_id": client_id,
+ # The relay picks the Amplitude project from this. Sent even as
+ # `prod`, so an event says where it belongs rather than relying on
+ # what the relay does with silence.
+ "flint_env": telemetry_environment,
+ # A string, as Switch Console sends it, where `unknown` is a third
+ # answer; a deployment always knows which it is.
+ "flint_internal": "true" if telemetry_internal else "false",
}
# Omitted rather than sent empty: an absent attribute reads as "not
# configured", where `""` reads as a real environment named nothing.
diff --git a/core/switch_core/telemetry/setup.py b/core/switch_core/telemetry/setup.py
index 0ca9b6bff..3b065c8f6 100644
--- a/core/switch_core/telemetry/setup.py
+++ b/core/switch_core/telemetry/setup.py
@@ -134,6 +134,8 @@ def _service(
service_name=config.service_name,
version=version,
environment=config.environment,
+ telemetry_environment=config.telemetry_environment,
+ telemetry_internal=config.telemetry_internal,
session_factory=session_factory,
installed_at=installed_at,
)
diff --git a/core/switch_core/telemetry/sink.py b/core/switch_core/telemetry/sink.py
index 562cc4d60..85daaf10e 100644
--- a/core/switch_core/telemetry/sink.py
+++ b/core/switch_core/telemetry/sink.py
@@ -10,6 +10,7 @@ lets a reporting problem reach the caller.
from __future__ import annotations
import logging
+from collections.abc import Mapping
from dataclasses import dataclass
from typing import Any, Protocol
@@ -19,6 +20,7 @@ from switch_core.observability.otlp import (
OtlpResource,
OtlpSendError,
build_logs_payload,
+ otlp_attributes,
)
from switch_core.telemetry.catalogue import PropertyValue
@@ -98,6 +100,7 @@ class OtlpRelaySink:
_resource_from(record),
)
_add_event_name(payload, record.name)
+ _add_relay_context(payload, record.resource)
try:
await self._client.post("logs", payload)
@@ -130,6 +133,17 @@ def _resource_from(record: TelemetryRecord) -> OtlpResource:
)
+def _add_relay_context(payload: dict[str, Any], resource: Mapping[str, str]) -> None:
+ """Add `flint_env` and `flint_internal` to the resource, which the shared
+ encoder's resource has no fields for: it describes the process to an
+ operator's own collector, where they mean nothing. The relay sends each
+ event to the Amplitude project `flint_env` names, and drops one it cannot
+ place; `flint_internal` tells staff usage from adoption."""
+ context = {key: resource[key] for key in ("flint_env", "flint_internal")}
+ for resource_log in payload["resourceLogs"]:
+ resource_log["resource"]["attributes"].extend(otlp_attributes(context))
+
+
def _add_event_name(payload: dict[str, Any], name: str) -> None:
"""Set the record's own `eventName` field, which `build_logs_payload` does
not: an operational log line has no event name."""
diff --git a/core/switch_core/telemetry/snapshot.py b/core/switch_core/telemetry/snapshot.py
index bbd9f9be2..7b57f7b77 100644
--- a/core/switch_core/telemetry/snapshot.py
+++ b/core/switch_core/telemetry/snapshot.py
@@ -44,6 +44,8 @@ from switch_core.db.models import (
ClientRoom,
CollaborationBridge,
Document,
+ ExternalUser,
+ ExternalUserClaim,
Message,
MessageAttachment,
Package,
@@ -56,6 +58,7 @@ from switch_core.db.models import (
)
from switch_core.db.session_scope import tenant_session
from switch_core.db.tenant_lookup import all_tenant_ids
+from switch_core.telemetry.internal import internal_email_condition
logger = logging.getLogger(__name__)
@@ -104,8 +107,26 @@ class UsageCounts:
# HTTP request histogram that times everything else.
duration_ms: int = 0
user_count: int = 0
+ # Accounts on the company's own email domains, so staff can be told from
+ # customers inside one deployment. Counted once for the deployment, like
+ # `user_count`.
+ user_internal_count: int = 0
+ # Switch accounts that used a room with an agent: an account counts when a
+ # chat account it has claimed spoke. Counted from the ids below once every
+ # tenant is in, since one account can be active in several tenants and a
+ # per-tenant count would add it once for each.
user_active_1d: int = 0
user_active_7d: int = 0
+ active_user_ids_1d: set[str] = field(default_factory=set)
+ active_user_ids_7d: set[str] = field(default_factory=set)
+ # Chat identities — a Slack, Mattermost or other platform account — that
+ # used a room with an agent, whether or not anyone has claimed them. One
+ # person on two platforms is two identities, and most identities belong to
+ # no Switch account, which is why this runs above `user_active_*`.
+ # `chat_identity_count` is the whole population they are drawn from.
+ chat_identity_count: int = 0
+ chat_identity_active_1d: int = 0
+ chat_identity_active_7d: int = 0
room_count: int = 0
room_agent_created_count: int = 0
room_system_created_count: int = 0
@@ -147,8 +168,12 @@ class UsageCounts:
"tenant_failed_count": self.tenant_failed_count,
"duration_ms": self.duration_ms,
"user_count": self.user_count,
+ "user_internal_count": self.user_internal_count,
"user_active_1d": self.user_active_1d,
"user_active_7d": self.user_active_7d,
+ "chat_identity_count": self.chat_identity_count,
+ "chat_identity_active_1d": self.chat_identity_active_1d,
+ "chat_identity_active_7d": self.chat_identity_active_7d,
"room_count": self.room_count,
"room_agent_created_count": self.room_agent_created_count,
"room_system_created_count": self.room_system_created_count,
@@ -313,6 +338,35 @@ def _human_interaction(tenant_id: str, since: datetime) -> Select[tuple[str]]:
)
+def _active_accounts(tenant_id: str, since: datetime) -> Select[tuple[str]]:
+ """Distinct Switch accounts that used a room since `since`: an account
+ is active when a chat account it has claimed spoke. A chat account several
+ people claim makes each of them active. See `_human_activity_conditions`."""
+ return (
+ select(distinct(ExternalUserClaim.user_id))
+ .select_from(Message)
+ .join(Client, Client.id == Message.sender_client_id)
+ .join(
+ ExternalUser,
+ and_(
+ ExternalUser.client_id == Client.id, ExternalUser.tenant_id == tenant_id
+ ),
+ )
+ .join(
+ ExternalUserClaim,
+ and_(
+ ExternalUserClaim.external_user_id == ExternalUser.id,
+ ExternalUserClaim.tenant_id == tenant_id,
+ ),
+ )
+ .where(
+ Message.tenant_id == tenant_id,
+ Message.sent_at >= since,
+ *_human_activity_conditions(tenant_id),
+ )
+ )
+
+
def _active_humans(tenant_id: str, since: datetime) -> Select[tuple[str | None]]:
"""Distinct human clients who used a room since `since`. See
`_human_activity_conditions`."""
@@ -353,8 +407,24 @@ async def collect_tenant_counts(
day_ago = now - _DAY
week_ago = now - _WEEK
- counts.user_active_1d += await _count(session, _active_humans(tenant_id, day_ago))
- counts.user_active_7d += await _count(session, _active_humans(tenant_id, week_ago))
+ counts.active_user_ids_1d |= set(
+ (await session.execute(_active_accounts(tenant_id, day_ago))).scalars()
+ )
+ counts.active_user_ids_7d |= set(
+ (await session.execute(_active_accounts(tenant_id, week_ago))).scalars()
+ )
+ counts.chat_identity_count += await _scalar(
+ session,
+ select(func.count())
+ .select_from(Client)
+ .where(Client.tenant_id == tenant_id, Client.type == HUMAN_CLIENT_TYPE),
+ )
+ counts.chat_identity_active_1d += await _count(
+ session, _active_humans(tenant_id, day_ago)
+ )
+ counts.chat_identity_active_7d += await _count(
+ session, _active_humans(tenant_id, week_ago)
+ )
counts.room_active_1d += await _count(
session, _human_interaction(tenant_id, day_ago)
)
@@ -683,6 +753,11 @@ def _merge_tenant_counts(total: UsageCounts, tenant: UsageCounts) -> None:
"tenant_failed_count",
"duration_ms",
"user_count",
+ "user_internal_count",
+ "user_active_1d",
+ "user_active_7d",
+ "active_user_ids_1d",
+ "active_user_ids_7d",
"room_users_max",
"connector_counts",
}
@@ -691,6 +766,8 @@ def _merge_tenant_counts(total: UsageCounts, tenant: UsageCounts) -> None:
continue
setattr(total, f.name, getattr(total, f.name) + getattr(tenant, f.name))
total.room_users_max = max(total.room_users_max, tenant.room_users_max)
+ total.active_user_ids_1d |= tenant.active_user_ids_1d
+ total.active_user_ids_7d |= tenant.active_user_ids_7d
for platform, count in tenant.connector_counts.items():
total.connector_counts[platform] += count
@@ -747,6 +824,14 @@ async def collect_usage(
counts.user_count = await _scalar(
session, select(func.count()).select_from(User)
)
+ counts.user_internal_count = await _scalar(
+ session,
+ select(func.count())
+ .select_from(User)
+ .where(internal_email_condition(User.email)),
+ )
+ counts.user_active_1d = len(counts.active_user_ids_1d)
+ counts.user_active_7d = len(counts.active_user_ids_7d)
counts.duration_ms = round((time.monotonic() - started) * 1000)
return counts
diff --git a/core/tests/switch_core/bridges/agent/api/test_events_endpoint_dispatch.py b/core/tests/switch_core/bridges/agent/api/test_events_endpoint_dispatch.py
index dfedf3c7b..23e7f5937 100644
--- a/core/tests/switch_core/bridges/agent/api/test_events_endpoint_dispatch.py
+++ b/core/tests/switch_core/bridges/agent/api/test_events_endpoint_dispatch.py
@@ -465,6 +465,8 @@ class TestOpeningAStreamReportsASession:
service_name="switch-core",
version="1.0.0",
environment=None,
+ telemetry_environment="prod",
+ telemetry_internal=False,
)
protocol = _Protocol()
protocol.sessions = SessionReporter(service, protocol.connections)
diff --git a/core/tests/switch_core/session_activity/test_session_started_route.py b/core/tests/switch_core/session_activity/test_session_started_route.py
index c0134df7b..ff55dfbd1 100644
--- a/core/tests/switch_core/session_activity/test_session_started_route.py
+++ b/core/tests/switch_core/session_activity/test_session_started_route.py
@@ -55,6 +55,8 @@ def _telemetry(sink: _RecordingSink, *, enabled: bool) -> TelemetryService:
service_name="switch-core",
version=None,
environment=None,
+ telemetry_environment="prod",
+ telemetry_internal=False,
)
diff --git a/core/tests/switch_core/telemetry/test_bridge_connect_reporting.py b/core/tests/switch_core/telemetry/test_bridge_connect_reporting.py
index 5edb4df3e..9003eeef1 100644
--- a/core/tests/switch_core/telemetry/test_bridge_connect_reporting.py
+++ b/core/tests/switch_core/telemetry/test_bridge_connect_reporting.py
@@ -79,6 +79,8 @@ def _service_with_telemetry(
service_name="switch-core",
version=None,
environment=None,
+ telemetry_environment="prod",
+ telemetry_internal=False,
session_factory=session_factory,
installed_at=installed_at,
)
diff --git a/core/tests/switch_core/telemetry/test_catalogue.py b/core/tests/switch_core/telemetry/test_catalogue.py
index f55a65f66..15a1354de 100644
--- a/core/tests/switch_core/telemetry/test_catalogue.py
+++ b/core/tests/switch_core/telemetry/test_catalogue.py
@@ -143,10 +143,13 @@ class TestValuesAreClosed:
with pytest.raises(TelemetryCatalogueError, match="expected a number"):
validate("usage_snapshot", payload) # type: ignore[arg-type]
- def test_a_non_finite_count_is_refused(self) -> None:
- """A mean over zero rooms must not arrive as NaN."""
+ @pytest.mark.parametrize("value", [float("nan"), float("inf"), float("-inf")])
+ def test_a_non_finite_count_is_refused(self, value: float) -> None:
+ """A mean over zero rooms must not arrive as NaN. JSON has no NaN or
+ infinity, so none of them can reach Amplitude as a number whatever the
+ relay does with it."""
payload = _valid_payload("usage_snapshot")
- payload["room_users_mean"] = float("nan")
+ payload["room_users_mean"] = value
with pytest.raises(TelemetryCatalogueError, match="finite"):
validate("usage_snapshot", payload) # type: ignore[arg-type]
diff --git a/core/tests/switch_core/telemetry/test_connector_setup_time.py b/core/tests/switch_core/telemetry/test_connector_setup_time.py
new file mode 100644
index 000000000..f0206ad5f
--- /dev/null
+++ b/core/tests/switch_core/telemetry/test_connector_setup_time.py
@@ -0,0 +1,74 @@
+"""`connector_added` reports setup time only when setup was actually seen.
+
+`seconds_since_configured` is configuration saved to first connect. A first
+connect made while telemetry was off went unmeasured, and reporting it on the
+next connect after telemetry is switched on reports the connector's whole age
+— millions of seconds — as setup time. So the first connect is recorded
+whether telemetry is on or not, and only a connector whose first connect
+happened while reporting reports it.
+"""
+
+from __future__ import annotations
+
+import uuid
+from datetime import UTC, datetime, timedelta
+
+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
+
+from switch_core.telemetry.service import TelemetryService
+from tests.switch_core.telemetry.test_bridge_connect_reporting import (
+ _service_with_telemetry,
+)
+from tests.switch_core.telemetry.test_preconfigured_connectors import (
+ INSTALLED,
+ _events,
+ _running,
+)
+
+CONFIGURED_LONG_AGO = datetime.now(UTC) - timedelta(days=60)
+
+
+def _switched_off(telemetry: TelemetryService) -> TelemetryService:
+ return TelemetryService(
+ sink=telemetry._sink,
+ enabled=False,
+ client_id="deployment-uuid",
+ service_name="switch-core",
+ version=None,
+ environment=None,
+ telemetry_environment="prod",
+ telemetry_internal=False,
+ session_factory=telemetry._session_factory,
+ installed_at=INSTALLED,
+ )
+
+
+async def test_a_first_connect_while_telemetry_was_off_is_never_reported(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
+ service, sink = _service_with_telemetry(session_factory, installed_at=INSTALLED)
+ reporting = service._telemetry
+ assert reporting is not None
+ bridge = f"slack-{uuid.uuid4().hex[:8]}"
+ _running(service, bridge, preconfigured=False)
+
+ service._telemetry = _switched_off(reporting)
+ await service._report_connector_up(bridge, "slack", CONFIGURED_LONG_AGO)
+ service._telemetry = reporting
+ await service._report_connector_up(bridge, "slack", CONFIGURED_LONG_AGO)
+
+ assert await _events(service, sink, "connector_added") == []
+
+
+async def test_a_first_connect_while_reporting_reports_its_setup_time(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
+ service, sink = _service_with_telemetry(session_factory, installed_at=INSTALLED)
+ bridge = f"slack-{uuid.uuid4().hex[:8]}"
+ _running(service, bridge, preconfigured=False)
+ configured = datetime.now(UTC) - timedelta(seconds=90)
+
+ await service._report_connector_up(bridge, "slack", configured)
+
+ [added] = await _events(service, sink, "connector_added")
+ assert 90 <= added["seconds_since_configured"] < 600
diff --git a/core/tests/switch_core/telemetry/test_deployment_and_reporter.py b/core/tests/switch_core/telemetry/test_deployment_and_reporter.py
index 9de91b70a..e85d01689 100644
--- a/core/tests/switch_core/telemetry/test_deployment_and_reporter.py
+++ b/core/tests/switch_core/telemetry/test_deployment_and_reporter.py
@@ -174,6 +174,8 @@ class TestEmitMilestone:
service_name="switch-core",
version="1.0.0",
environment=None,
+ telemetry_environment="prod",
+ telemetry_internal=False,
session_factory=session_factory,
installed_at=installed_at,
)
@@ -235,6 +237,8 @@ class TestTheSnapshotSchedule:
service_name="switch-core",
version="1.0.0",
environment=None,
+ telemetry_environment="prod",
+ telemetry_internal=False,
session_factory=session_factory,
installed_at=datetime.now(UTC) - timedelta(days=1),
)
diff --git a/core/tests/switch_core/telemetry/test_emitters_are_live.py b/core/tests/switch_core/telemetry/test_emitters_are_live.py
index 47e5f1312..46f00b0f3 100644
--- a/core/tests/switch_core/telemetry/test_emitters_are_live.py
+++ b/core/tests/switch_core/telemetry/test_emitters_are_live.py
@@ -39,6 +39,8 @@ def _telemetry() -> TelemetryService:
service_name="switch-core",
version="1.0.0",
environment=None,
+ telemetry_environment="prod",
+ telemetry_internal=False,
)
diff --git a/core/tests/switch_core/telemetry/test_environment_reaches_the_relay.py b/core/tests/switch_core/telemetry/test_environment_reaches_the_relay.py
new file mode 100644
index 000000000..c1761266a
--- /dev/null
+++ b/core/tests/switch_core/telemetry/test_environment_reaches_the_relay.py
@@ -0,0 +1,82 @@
+"""A deployment's TELEMETRY_ENVIRONMENT reaches the relay as `flint_env`.
+
+The relay files each event under the Amplitude project that attribute names. A
+server that read the setting and then sent something else would land its
+usage in the wrong project with nothing anywhere reporting a fault, so this
+follows the value from configuration to the request body.
+"""
+
+from __future__ import annotations
+
+import json
+import uuid
+from typing import Any
+
+import httpx
+import pytest
+from sqlalchemy import delete
+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
+
+from switch_core.config import SwitchConfig
+from switch_core.db.models import DeploymentIdentity
+from switch_core.telemetry import setup as telemetry_setup
+
+
+def _config(environment: str) -> SwitchConfig:
+ return SwitchConfig( # type: ignore[call-arg]
+ db_host="h",
+ db_port="5432",
+ db_user="u",
+ db_password="p",
+ db_name="d",
+ matrix_server_name="test",
+ agent_registration_token="t",
+ jwt_secret_key="k",
+ gateway_admin_email="[email protected]",
+ gateway_admin_password="pw",
+ secret_keys="test:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
+ telemetry_enabled=True,
+ telemetry_endpoint="https://relay.example",
+ telemetry_environment=environment,
+ telemetry_internal=False,
+ )
+
+
[email protected]("environment", ["prod", "staging", "dev", "local"])
+async def test_the_configured_environment_is_sent_as_flint_env(
+ environment: str,
+ session_factory: async_sessionmaker[AsyncSession],
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ async with session_factory() as session:
+ await session.execute(delete(DeploymentIdentity))
+ session.add(DeploymentIdentity(id=1, client_id=str(uuid.uuid4())))
+ await session.commit()
+ bodies: list[dict[str, Any]] = []
+
+ def _handle(request: httpx.Request) -> httpx.Response:
+ bodies.append(json.loads(request.content))
+ return httpx.Response(200)
+
+ transport = httpx.MockTransport(_handle)
+ real_client = httpx.AsyncClient
+ monkeypatch.setattr(
+ telemetry_setup.httpx,
+ "AsyncClient",
+ lambda **kwargs: real_client(transport=transport, **kwargs),
+ )
+
+ service, _, http = await telemetry_setup.build_telemetry(
+ _config(environment), session_factory, "1.0.0"
+ )
+ service.emit("deployment_started", tenant_count=1)
+ await service.aclose()
+ assert http is not None
+ await http.aclose()
+
+ [body] = bodies
+ resource = {
+ a["key"]: a["value"] for a in body["resourceLogs"][0]["resource"]["attributes"]
+ }
+ assert resource["flint_env"] == {"stringValue": environment}
+ assert resource["flint_internal"] == {"stringValue": "false"}
diff --git a/core/tests/switch_core/telemetry/test_first_room_active.py b/core/tests/switch_core/telemetry/test_first_room_active.py
index 44bf95810..5fa1af07c 100644
--- a/core/tests/switch_core/telemetry/test_first_room_active.py
+++ b/core/tests/switch_core/telemetry/test_first_room_active.py
@@ -67,6 +67,8 @@ def _reporter(
service_name="switch-core",
version="1.0.0",
environment=None,
+ telemetry_environment="prod",
+ telemetry_internal=False,
session_factory=session_factory,
installed_at=INSTALLED,
)
diff --git a/core/tests/switch_core/telemetry/test_merge_fixes.py b/core/tests/switch_core/telemetry/test_merge_fixes.py
index 32ed86c6b..40295e17e 100644
--- a/core/tests/switch_core/telemetry/test_merge_fixes.py
+++ b/core/tests/switch_core/telemetry/test_merge_fixes.py
@@ -141,6 +141,8 @@ class TestASessionIsTheAgentNotTheConnection:
service_name="switch-core",
version="1.0.0",
environment=None,
+ telemetry_environment="prod",
+ telemetry_internal=False,
)
reporter = SessionReporter(service, registry)
registry.set_close_listener(reporter.on_close)
diff --git a/core/tests/switch_core/telemetry/test_preconfigured_connectors.py b/core/tests/switch_core/telemetry/test_preconfigured_connectors.py
index 0f64ff0ed..201591263 100644
--- a/core/tests/switch_core/telemetry/test_preconfigured_connectors.py
+++ b/core/tests/switch_core/telemetry/test_preconfigured_connectors.py
@@ -269,6 +269,8 @@ class TestRegisteringRecordsTheFlag:
service_name="switch-core",
version=None,
environment=None,
+ telemetry_environment="prod",
+ telemetry_internal=False,
)
service._session_factory = session_factory
service._client_lifecycle.create_client = AsyncMock( # type: ignore[method-assign]
diff --git a/core/tests/switch_core/telemetry/test_relay_attribute_cap.py b/core/tests/switch_core/telemetry/test_relay_attribute_cap.py
new file mode 100644
index 000000000..6d9561f15
--- /dev/null
+++ b/core/tests/switch_core/telemetry/test_relay_attribute_cap.py
@@ -0,0 +1,119 @@
+"""Every event fits through the relay's limits, with room to spare.
+
+The relay drops any log record carrying more than 128 attributes, and any
+event whose JSON is over 32 KiB never reaches Amplitude. It answers 200 either
+way. An event that grew past either would vanish from the dashboard with
+nothing anywhere reporting a fault. They are the silent drops at the relay that
+Switch itself controls, so the margins are held here, beside the catalogue.
+"""
+
+from __future__ import annotations
+
+import json
+from collections.abc import Callable
+from typing import Any
+
+import httpx
+import pytest
+
+from switch_core.observability.otlp import OtlpClient
+from switch_core.telemetry.catalogue import BOOLEAN, CATALOGUE, NUMBER, PropertyType
+from switch_core.telemetry.service import TelemetryService
+from switch_core.telemetry.sink import OtlpRelaySink, TelemetryRecord
+
+# `filter/guard` in the relay's collector config: `Len(attributes) > 128`.
+RELAY_ATTRIBUTE_CAP = 128
+# Headroom, so an event growing toward the cap is caught in review rather than
+# after its records have started disappearing.
+MAX_ATTRIBUTES = 100
+# `max_event_bytes` on the relay's Amplitude exporter: a larger event is dropped.
+RELAY_EVENT_BYTES = 32768
+# Half of it. Measured on the OTLP attributes, which wrap each value in more
+# JSON than the Amplitude event does, so this over-counts.
+MAX_EVENT_BYTES = RELAY_EVENT_BYTES // 2
+
+
+class _Capture:
+ def __init__(self) -> None:
+ self.sent: list[TelemetryRecord] = []
+
+ async def send(self, record: TelemetryRecord) -> None:
+ self.sent.append(record)
+
+ async def aclose(self) -> None:
+ return None
+
+
+def _example(kind: PropertyType) -> Any:
+ """A value the catalogue accepts for a property of this kind."""
+ if kind is NUMBER:
+ return 1
+ if kind is BOOLEAN:
+ return True
+ return sorted(kind.values)[0] # type: ignore[attr-defined]
+
+
+def _largest(kind: PropertyType) -> Any:
+ """The longest value the catalogue accepts for a property of this kind."""
+ if kind is NUMBER:
+ return -123456789012345.67
+ if kind is BOOLEAN:
+ return False
+ return max(kind.values, key=len) # type: ignore[attr-defined]
+
+
+async def _attributes_on_the_wire(
+ event: str, value: Callable[[PropertyType], Any] = _example
+) -> list[dict[str, Any]]:
+ """The log record's attributes exactly as the relay would receive them:
+ validated by the service, encoded by the sink."""
+ capture = _Capture()
+ service = TelemetryService(
+ sink=capture, # type: ignore[arg-type]
+ enabled=True,
+ client_id="11111111-1111-1111-1111-111111111111",
+ service_name="switch-core",
+ version="1.0.0",
+ environment=None,
+ telemetry_environment="prod",
+ telemetry_internal=False,
+ )
+ service.emit(event, **{key: value(kind) for key, kind in CATALOGUE[event].items()})
+ await service.aclose()
+
+ body: dict[str, Any] = {}
+
+ def _handle(request: httpx.Request) -> httpx.Response:
+ body.update(json.loads(request.content))
+ return httpx.Response(200)
+
+ http = httpx.AsyncClient(transport=httpx.MockTransport(_handle))
+ [record] = capture.sent
+ await OtlpRelaySink(client=OtlpClient("https://relay.example", 5, {}, http)).send(
+ record
+ )
+ await http.aclose()
+ return body["resourceLogs"][0]["scopeLogs"][0]["logRecords"][0]["attributes"]
+
+
[email protected]("event", sorted(CATALOGUE))
+async def test_every_event_fits_through_the_relay(event: str) -> None:
+ attributes = await _attributes_on_the_wire(event)
+
+ assert len(attributes) <= MAX_ATTRIBUTES, (
+ f"{event} carries {len(attributes)} attributes on the wire. The relay "
+ f"drops any record over {RELAY_ATTRIBUTE_CAP} and still answers 200, "
+ f"so this event is close to vanishing without a trace. Split it."
+ )
+
+
[email protected]("event", sorted(CATALOGUE))
+async def test_every_event_stays_well_under_the_relays_event_size(event: str) -> None:
+ attributes = await _attributes_on_the_wire(event, _largest)
+ size = len(json.dumps(attributes))
+
+ assert size <= MAX_EVENT_BYTES, (
+ f"{event} is {size} bytes of attributes at its largest. The relay drops "
+ f"any Amplitude event over {RELAY_EVENT_BYTES} bytes and still answers "
+ f"200, so this event is close to vanishing from Amplitude. Split it."
+ )
diff --git a/core/tests/switch_core/telemetry/test_service_and_sink.py b/core/tests/switch_core/telemetry/test_service_and_sink.py
index 0ab2e90b5..172845554 100644
--- a/core/tests/switch_core/telemetry/test_service_and_sink.py
+++ b/core/tests/switch_core/telemetry/test_service_and_sink.py
@@ -34,6 +34,8 @@ def _service(sink: object, *, enabled: bool = True) -> TelemetryService:
service_name="switch-core",
version="1.2.3",
environment="pilot",
+ telemetry_environment="prod",
+ telemetry_internal=False,
)
@@ -78,10 +80,57 @@ class TestTagging:
assert sink.sent[0].resource == {
"service.name": "switch-core",
"flint.client_id": "deployment-uuid",
+ "flint_env": "prod",
+ "flint_internal": "false",
"service.version": "1.2.3",
"deployment.environment": "pilot",
}
+ @pytest.mark.parametrize("environment", ["prod", "staging", "dev", "local"])
+ async def test_every_event_says_which_amplitude_project_it_belongs_in(
+ self, environment: str
+ ) -> None:
+ """The relay sends each event to the Amplitude project its `flint_env`
+ names, so a development server's usage never lands in production's."""
+ sink = _RecordingSink()
+ service = TelemetryService(
+ sink=sink, # type: ignore[arg-type]
+ enabled=True,
+ client_id="deployment-uuid",
+ service_name="switch-core",
+ version="1.2.3",
+ environment=None,
+ telemetry_environment=environment,
+ telemetry_internal=False,
+ )
+
+ service.emit("deployment_started", tenant_count=1)
+ await service.aclose()
+
+ assert sink.sent[0].resource["flint_env"] == environment
+
+ @pytest.mark.parametrize(("internal", "sent"), [(True, "true"), (False, "false")])
+ async def test_every_event_says_whether_the_deployment_is_the_companys_own(
+ self, internal: bool, sent: str
+ ) -> None:
+ """So staff usage can be told from adoption without any identifier."""
+ sink = _RecordingSink()
+ service = TelemetryService(
+ sink=sink, # type: ignore[arg-type]
+ enabled=True,
+ client_id="deployment-uuid",
+ service_name="switch-core",
+ version="1.2.3",
+ environment=None,
+ telemetry_environment="prod",
+ telemetry_internal=internal,
+ )
+
+ service.emit("deployment_started", tenant_count=1)
+ await service.aclose()
+
+ assert sink.sent[0].resource["flint_internal"] == sent
+
async def test_an_unset_environment_is_omitted_rather_than_empty(self) -> None:
"""An absent attribute reads as "not configured"; an empty string reads
as a real environment named nothing."""
@@ -93,6 +142,8 @@ class TestTagging:
service_name="switch-core",
version=None,
environment=None,
+ telemetry_environment="prod",
+ telemetry_internal=False,
)
service.emit("deployment_started", tenant_count=1)
@@ -154,6 +205,8 @@ class TestTheWireFormat:
resource={
"service.name": "switch-core",
"flint.client_id": "deployment-uuid",
+ "flint_env": "dev",
+ "flint_internal": "false",
},
timestamp_ns=1_700_000_000_000_000_000,
)
@@ -187,6 +240,39 @@ class TestTheWireFormat:
assert attributes["from_template"] == {"boolValue": True}
+ async def test_a_false_and_a_zero_are_sent_rather_than_dropped(self) -> None:
+ """Falsy values are where an encoder that tests `if value:` loses them,
+ and a missing property reads downstream exactly like one never sent."""
+ body: dict = {}
+
+ def _handle(request: httpx.Request) -> httpx.Response:
+ body.update(json.loads(request.content))
+ return httpx.Response(200)
+
+ http = httpx.AsyncClient(transport=httpx.MockTransport(_handle))
+ sink = OtlpRelaySink(client=OtlpClient("https://relay.example", 5, {}, http))
+ await sink.send(
+ TelemetryRecord(
+ name="switch_core.connector_added",
+ properties={
+ "is_preconfigured": False,
+ "failed_attempts_before_success": 0,
+ },
+ resource={
+ "service.name": "switch-core",
+ "flint.client_id": "deployment-uuid",
+ "flint_env": "prod",
+ "flint_internal": "false",
+ },
+ timestamp_ns=1_700_000_000_000_000_000,
+ )
+ )
+ await http.aclose()
+ attributes = {a["key"]: a["value"] for a in self._record(body)["attributes"]}
+
+ assert attributes["is_preconfigured"] == {"boolValue": False}
+ assert attributes["failed_attempts_before_success"] == {"doubleValue": 0.0}
+
async def test_the_body_carries_the_name_so_the_log_line_is_not_blank(
self,
) -> None:
@@ -203,6 +289,9 @@ class TestTheWireFormat:
assert resource["flint.client_id"] == {"stringValue": "deployment-uuid"}
assert resource["service.name"] == {"stringValue": "switch-core"}
+ # The relay sends each event to the Amplitude project this names.
+ assert resource["flint_env"] == {"stringValue": "dev"}
+ assert resource["flint_internal"] == {"stringValue": "false"}
async def test_it_posts_to_the_logs_signal(self) -> None:
"""Not `/v1/metrics`: the relay routes on log records, and a metric
@@ -290,7 +379,12 @@ def _a_record() -> TelemetryRecord:
return TelemetryRecord(
name="switch_core.deployment_started",
properties={"tenant_count": 1},
- resource={"service.name": "switch-core", "flint.client_id": "deployment-uuid"},
+ resource={
+ "service.name": "switch-core",
+ "flint.client_id": "deployment-uuid",
+ "flint_env": "dev",
+ "flint_internal": "false",
+ },
timestamp_ns=1,
)
diff --git a/core/tests/switch_core/telemetry/test_snapshot.py b/core/tests/switch_core/telemetry/test_snapshot.py
index 5ec349c51..2391a3d04 100644
--- a/core/tests/switch_core/telemetry/test_snapshot.py
+++ b/core/tests/switch_core/telemetry/test_snapshot.py
@@ -233,7 +233,7 @@ class TestWhatCountsAsActive:
counts = await _counts(session)
assert counts.room_active_1d == 1
- assert counts.user_active_1d == 1
+ assert counts.chat_identity_active_1d == 1
async def test_two_agents_talking_is_not_activity(
self, session_factory: async_sessionmaker[AsyncSession]
@@ -252,7 +252,7 @@ class TestWhatCountsAsActive:
counts = await _counts(session)
assert counts.room_active_1d == 0
- assert counts.user_active_1d == 0
+ assert counts.chat_identity_active_1d == 0
async def test_a_human_in_a_room_with_no_agent_is_not_activity(
self, session_factory: async_sessionmaker[AsyncSession]
@@ -283,7 +283,7 @@ class TestWhatCountsAsActive:
counts = await _counts(session)
- assert counts.user_active_1d == 0
+ assert counts.chat_identity_active_1d == 0
assert counts.room_active_1d == 0
async def test_the_day_and_week_windows_differ(
@@ -320,7 +320,7 @@ class TestWhatCountsAsActive:
counts = await _counts(session)
assert counts.room_active_1d == 0
- assert counts.user_active_1d == 0
+ assert counts.chat_identity_active_1d == 0
class TestRoomsAreCountedByWhoMadeThem:
@@ -516,8 +516,8 @@ class TestOnlyWhatSomeoneSaidCounts:
ever_active = await room_had_human_activity(session, TENANT_ZERO, room.id)
assert counts.message_from_human_1d == 0
- assert counts.user_active_1d == 0
- assert counts.user_active_7d == 0
+ assert counts.chat_identity_active_1d == 0
+ assert counts.chat_identity_active_7d == 0
assert counts.room_active_1d == 0
assert counts.room_active_7d == 0
assert ever_active is False
@@ -543,7 +543,7 @@ class TestOnlyWhatSomeoneSaidCounts:
counts = await _counts(session)
assert counts.message_from_human_1d == 1
- assert counts.user_active_1d == 1
+ assert counts.chat_identity_active_1d == 1
assert counts.room_active_1d == 1
async def test_the_notice_switch_posts_for_an_agent_is_not_the_agent_talking(
@@ -929,7 +929,7 @@ class TestTheFourPathsAgreeOnActivity:
)
assert counts.room_active_1d == 0
- assert counts.user_active_1d == 0
+ assert counts.chat_identity_active_1d == 0
assert was_ever_active is False
assert newly_active == []
@@ -956,7 +956,7 @@ class TestTheFourPathsAgreeOnActivity:
)
assert counts.room_active_1d == 1
- assert counts.user_active_1d == 1
+ assert counts.chat_identity_active_1d == 1
assert was_ever_active is True
assert len(newly_active) == 1
@@ -991,7 +991,7 @@ class TestTheFourPathsAgreeOnActivity:
)
assert counts.room_active_1d == 1
- assert counts.user_active_1d == 1
+ assert counts.chat_identity_active_1d == 1
assert was_ever_active is True
assert len(newly_active) == 1
assert newly_active[0].first_active_at == NOW - timedelta(minutes=90)
diff --git a/core/tests/switch_core/telemetry/test_snapshot_accounts.py b/core/tests/switch_core/telemetry/test_snapshot_accounts.py
new file mode 100644
index 000000000..f24afa548
--- /dev/null
+++ b/core/tests/switch_core/telemetry/test_snapshot_accounts.py
@@ -0,0 +1,250 @@
+"""Active users are Switch accounts; chat identities are counted beside them.
+
+A person reaches Switch through a chat account — Slack, Mattermost, and so on
+— and an account is active when a chat account it has claimed speaks in a room
+with an agent. Unclaimed chat accounts, and one person's second platform, are
+what made "active" run above "registered"; they are still counted, as
+`chat_identity_*`, so nothing that number showed is lost.
+"""
+
+from __future__ import annotations
+
+import uuid
+from datetime import UTC, datetime, timedelta
+
+import pytest
+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
+
+from switch_core.db.models import (
+ TENANT_ZERO_ID,
+ Client,
+ ClientRoom,
+ CollaborationBridge,
+ ExternalUser,
+ ExternalUserClaim,
+ Message,
+ Room,
+ Tenant,
+ User,
+)
+from switch_core.db.session_scope import tenant_session
+from switch_core.telemetry import snapshot as snapshot_module
+from switch_core.telemetry.snapshot import UsageCounts, collect_usage
+
+OTHER_TENANT = "22222222-2222-2222-2222-222222222222"
+
+
+async def _account(
+ session_factory: async_sessionmaker[AsyncSession], email: str
+) -> str:
+ async with session_factory() as session:
+ user = User(name=email, email=email, role="user")
+ session.add(user)
+ await session.commit()
+ return user.id
+
+
+async def _other_tenant(session_factory: async_sessionmaker[AsyncSession]) -> None:
+ async with session_factory() as session:
+ session.add(
+ Tenant(id=OTHER_TENANT, name="other", slug=f"o-{uuid.uuid4().hex[:8]}")
+ )
+ await session.commit()
+
+
+async def _chat_account_speaks(
+ session_factory: async_sessionmaker[AsyncSession],
+ *,
+ claimed_by: list[str],
+ tenant_id: str = TENANT_ZERO_ID,
+ spoke_at: datetime | None = None,
+) -> None:
+ """A chat account, claimed by `claimed_by`, says something in a room with
+ an agent."""
+ suffix = uuid.uuid4().hex[:8]
+ async with tenant_session(session_factory, tenant_id) as session:
+ bridge_client = Client(
+ transport_user_id=f"@bridge-{suffix}:test",
+ display_name="bridge",
+ type="bridge",
+ )
+ person = Client(
+ transport_user_id=f"@person-{suffix}:test",
+ display_name="person",
+ type="user",
+ )
+ agent = Client(
+ transport_user_id=f"@agent-{suffix}:test",
+ display_name="agent",
+ type="agent",
+ )
+ room = Room(
+ transport_room_id=f"!{suffix}:test",
+ name=f"room-{suffix}",
+ description="",
+ metadata_={"created_by_kind": "user"},
+ )
+ session.add_all([bridge_client, person, agent, room])
+ await session.flush()
+ bridge = CollaborationBridge(
+ type="slack",
+ display_name="Slack",
+ client_id=bridge_client.id,
+ status="active",
+ )
+ session.add(bridge)
+ await session.flush()
+ chat_account = ExternalUser(
+ bridge_id=bridge.id,
+ external_user_id=f"U{suffix}",
+ external_username=f"person-{suffix}",
+ client_id=person.id,
+ )
+ session.add(chat_account)
+ session.add(ClientRoom(client_id=agent.id, room_id=room.id))
+ await session.flush()
+ for user_id in claimed_by:
+ session.add(
+ ExternalUserClaim(external_user_id=chat_account.id, user_id=user_id)
+ )
+ session.add(
+ Message(
+ room_id=room.id,
+ seq=1,
+ transport_event_id=f"$evt-{suffix}",
+ sender_id=person.transport_user_id,
+ sender_client_id=person.id,
+ event_type="m.room.message",
+ msgtype="m.text",
+ body="hello",
+ content={"body": "hello"},
+ sent_at=spoke_at or datetime.now(UTC) - timedelta(hours=1),
+ )
+ )
+ await session.commit()
+
+
+async def test_a_claimed_chat_account_makes_its_switch_account_active(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
+ someone = await _account(session_factory, f"{uuid.uuid4().hex[:6]}@example.com")
+ await _chat_account_speaks(session_factory, claimed_by=[someone])
+ await _chat_account_speaks(session_factory, claimed_by=[])
+
+ counts = await collect_usage(session_factory)
+
+ assert counts.user_active_1d == 1
+ assert counts.user_active_7d == 1
+ assert counts.chat_identity_active_1d == 2
+ assert counts.chat_identity_count == 2
+
+
+async def test_one_person_on_two_platforms_is_one_active_user(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
+ someone = await _account(session_factory, f"{uuid.uuid4().hex[:6]}@example.com")
+ await _chat_account_speaks(session_factory, claimed_by=[someone])
+ await _chat_account_speaks(session_factory, claimed_by=[someone])
+
+ counts = await collect_usage(session_factory)
+
+ assert counts.user_active_1d == 1
+ assert counts.chat_identity_active_1d == 2
+
+
+async def test_one_account_active_in_two_tenants_is_one_active_user(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
+ """Accounts are deployment-wide and tenants are not, so a per-tenant count
+ summed across them would add this person once for each."""
+ await _other_tenant(session_factory)
+ someone = await _account(session_factory, f"{uuid.uuid4().hex[:6]}@example.com")
+ await _chat_account_speaks(session_factory, claimed_by=[someone])
+ await _chat_account_speaks(
+ session_factory, claimed_by=[someone], tenant_id=OTHER_TENANT
+ )
+
+ counts = await collect_usage(session_factory)
+
+ assert counts.user_active_1d == 1
+ assert counts.chat_identity_active_1d == 2
+
+
+async def test_a_chat_account_two_people_claim_makes_both_active(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
+ first = await _account(session_factory, f"{uuid.uuid4().hex[:6]}@example.com")
+ second = await _account(session_factory, f"{uuid.uuid4().hex[:6]}@example.com")
+ await _chat_account_speaks(session_factory, claimed_by=[first, second])
+
+ counts = await collect_usage(session_factory)
+
+ assert counts.user_active_1d == 2
+
+
+async def test_activity_older_than_a_day_counts_for_the_week_only(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
+ someone = await _account(session_factory, f"{uuid.uuid4().hex[:6]}@example.com")
+ await _chat_account_speaks(
+ session_factory,
+ claimed_by=[someone],
+ spoke_at=datetime.now(UTC) - timedelta(days=3),
+ )
+
+ counts = await collect_usage(session_factory)
+
+ assert counts.user_active_1d == 0
+ assert counts.user_active_7d == 1
+
+
+async def test_staff_accounts_are_counted_apart(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
+ for email in (
+ ):
+ await _account(session_factory, f"{uuid.uuid4().hex[:4]}.{email}")
+
+ counts = await collect_usage(session_factory)
+
+ assert counts.user_count == 5
+ assert counts.user_internal_count == 3
+
+
+async def test_a_tenant_that_fails_after_collecting_its_accounts_adds_none_of_them(
+ session_factory: async_sessionmaker[AsyncSession],
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """A failing tenant contributes nothing, including account ids it had
+ already collected before the failure, so its people are not counted from a
+ pass that otherwise excluded it."""
+ await _other_tenant(session_factory)
+ healthy = await _account(session_factory, f"{uuid.uuid4().hex[:6]}@example.com")
+ failing = await _account(session_factory, f"{uuid.uuid4().hex[:6]}@example.com")
+ await _chat_account_speaks(session_factory, claimed_by=[healthy])
+ await _chat_account_speaks(
+ session_factory, claimed_by=[failing], tenant_id=OTHER_TENANT
+ )
+ collect = snapshot_module.collect_tenant_counts
+
+ async def fails_after_collecting(
+ session: AsyncSession, tenant_id: str, counts: UsageCounts, now: datetime
+ ) -> None:
+ await collect(session, tenant_id, counts, now)
+ if tenant_id == OTHER_TENANT:
+ raise RuntimeError("simulated failure after the accounts were read")
+
+ monkeypatch.setattr(
+ snapshot_module, "collect_tenant_counts", fails_after_collecting
+ )
+
+ counts = await collect_usage(session_factory)
+
+ assert counts.tenant_failed_count == 1
+ assert counts.user_active_1d == 1
+ assert counts.chat_identity_active_1d == 1
diff --git a/core/tests/switch_core/telemetry/test_snapshot_multi_tenant.py b/core/tests/switch_core/telemetry/test_snapshot_multi_tenant.py
index 6eafa3ae8..1b34e6e70 100644
--- a/core/tests/switch_core/telemetry/test_snapshot_multi_tenant.py
+++ b/core/tests/switch_core/telemetry/test_snapshot_multi_tenant.py
@@ -145,7 +145,7 @@ class TestCountsDoNotMultiplyByTenant:
# Two, not four. Without the tenant predicate each pass saw both rooms.
assert counts.room_count == 2
assert counts.room_active_1d == 2
- assert counts.user_active_1d == 2
+ assert counts.chat_identity_active_1d == 2
assert counts.message_count_1d == 2
assert counts.room_membership_total == 2
@@ -164,7 +164,7 @@ class TestCountsDoNotMultiplyByTenant:
assert counts.tenant_count == 2
assert counts.room_count == 1
assert counts.room_active_7d == 1
- assert counts.user_active_7d == 1
+ assert counts.chat_identity_active_7d == 1
assert counts.message_count_1d == 1
diff --git a/core/tests/switch_core/test_config_telemetry_environment.py b/core/tests/switch_core/test_config_telemetry_environment.py
new file mode 100644
index 000000000..2d37db1a2
--- /dev/null
+++ b/core/tests/switch_core/test_config_telemetry_environment.py
@@ -0,0 +1,66 @@
+"""TELEMETRY_ENVIRONMENT picks the Amplitude project a deployment's usage lands in.
+
+The relay files an event under the project its `flint_env` names, and drops one
+naming an environment it has no project for — answering 200 either way. So the
+setting is held to the four the relay knows, and an unknown value stops the
+server at startup rather than quietly losing its usage.
+"""
+
+import pytest
+from pydantic import ValidationError
+
+from switch_core.config import SwitchConfig
+
+_BASE_KWARGS = dict(
+ db_host="db",
+ db_port="5432",
+ db_user="postgres",
+ db_password="pw",
+ db_name="switch",
+ matrix_server_name="switch.local",
+ agent_registration_token="token",
+ jwt_secret_key="jwt",
+ gateway_admin_email="[email protected]",
+ gateway_admin_password="pw",
+ secret_keys="test:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
+)
+
+
+def test_a_deployment_that_says_nothing_is_production(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """Every customer's deployment is production, and none of them sets it."""
+ monkeypatch.delenv("TELEMETRY_ENVIRONMENT", raising=False)
+
+ assert SwitchConfig(**_BASE_KWARGS).telemetry_environment == "prod" # type: ignore[arg-type]
+
+
[email protected]("environment", ["prod", "staging", "dev", "local"])
+def test_each_environment_the_relay_knows_is_accepted(
+ environment: str, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ monkeypatch.setenv("TELEMETRY_ENVIRONMENT", environment)
+
+ assert SwitchConfig(**_BASE_KWARGS).telemetry_environment == environment # type: ignore[arg-type]
+
+
[email protected]("environment", ["qa", "production", "development", "Dev"])
+def test_an_environment_the_relay_does_not_know_is_refused(
+ environment: str, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ monkeypatch.setenv("TELEMETRY_ENVIRONMENT", environment)
+
+ with pytest.raises(ValidationError, match="telemetry_environment"):
+ SwitchConfig(**_BASE_KWARGS) # type: ignore[arg-type]
+
+
+def test_a_deployment_is_not_the_companys_own_unless_it_says_so(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ monkeypatch.delenv("TELEMETRY_INTERNAL", raising=False)
+
+ assert SwitchConfig(**_BASE_KWARGS).telemetry_internal is False # type: ignore[arg-type]
+
+ monkeypatch.setenv("TELEMETRY_INTERNAL", "true")
+
+ assert SwitchConfig(**_BASE_KWARGS).telemetry_internal is True # type: ignore[arg-type]
diff --git a/core/tests/switch_core/test_migration_claim_existing_connectors.py b/core/tests/switch_core/test_migration_claim_existing_connectors.py
new file mode 100644
index 000000000..42a599a9e
--- /dev/null
+++ b/core/tests/switch_core/test_migration_claim_existing_connectors.py
@@ -0,0 +1,98 @@
+"""Connectors that already exist never report `connector_added`.
+
+`connector_added` reports setup time, configuration saved to first connect.
+A connector that existed before its first connect was tracked would report its
+whole age instead, so the `2575637e78d4` migration takes the claim for every
+connector already in the database. This seeds connectors across two tenants
+and a claim one of them had already spent, runs the migration, and checks
+every connector is claimed exactly once and nothing else is touched.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+import pytest
+from sqlalchemy import text
+from sqlalchemy.ext.asyncio import create_async_engine
+
+from tests.switch_core.test_migration_oidc_identities_backfill import _upgrade_to
+
+_DB = "migration_claim_existing_connectors"
+_PRE_MIGRATION_REVISION = "e4a7c1d93b25"
+_MIGRATION_UNDER_TEST = "2575637e78d4"
+
+
+async def migration_url(postgres_url: str) -> Any:
+ admin = create_async_engine(postgres_url, isolation_level="AUTOCOMMIT")
+ async with admin.connect() as connection:
+ await connection.execute(text(f'DROP DATABASE IF EXISTS "{_DB}"'))
+ await connection.execute(text(f'CREATE DATABASE "{_DB}"'))
+ await admin.dispose()
+
+ base, _, _ = postgres_url.rpartition("/")
+ try:
+ yield f"{base}/{_DB}"
+ finally:
+ admin = create_async_engine(postgres_url, isolation_level="AUTOCOMMIT")
+ async with admin.connect() as connection:
+ await connection.execute(text(f'DROP DATABASE IF EXISTS "{_DB}"'))
+ await admin.dispose()
+
+
+async def test_every_existing_connector_is_claimed_once(migration_url: str) -> None:
+ engine = create_async_engine(migration_url)
+ try:
+ async with engine.begin() as connection:
+ await connection.run_sync(_upgrade_to(_PRE_MIGRATION_REVISION))
+
+ async with engine.begin() as connection:
+ # The connectors' tenants and clients are beside the point here, so
+ # their foreign keys are not checked while seeding.
+ await connection.execute(
+ text("SET LOCAL session_replication_role = replica")
+ )
+ await connection.execute(
+ text(
+ """
+ INSERT INTO collaboration_bridges
+ (id, type, display_name, client_id, status, tenant_id)
+ VALUES
+ ('slack-a', 'slack', 'Slack', 'client-a', 'active', 'tenant-a'),
+ ('mm-a', 'mattermost', 'Mattermost', 'client-a', 'active', 'tenant-a'),
+ ('teams-b', 'teams', 'Teams', 'client-b', 'active', 'tenant-b')
+ """
+ )
+ )
+ await connection.execute(
+ text(
+ """
+ INSERT INTO telemetry_milestones (name)
+ VALUES ('connector_added:slack-a'), ('first_connector_added')
+ """
+ )
+ )
+
+ async with engine.begin() as connection:
+ await connection.run_sync(_upgrade_to(_MIGRATION_UNDER_TEST))
+
+ async with engine.connect() as connection:
+ claims = (
+ (
+ await connection.execute(
+ text("SELECT name FROM telemetry_milestones ORDER BY name")
+ )
+ )
+ .scalars()
+ .all()
+ )
+ finally:
+ await engine.dispose()
+
+ assert claims == [
+ "connector_added:mm-a",
+ "connector_added:slack-a",
+ "connector_added:teams-b",
+ "first_connector_added",
+ ]
diff --git a/deploy/local/standalone-docker-compose.yml b/deploy/local/standalone-docker-compose.yml
index de2cfd6bb..3acfcd385 100644
--- a/deploy/local/standalone-docker-compose.yml
+++ b/deploy/local/standalone-docker-compose.yml
@@ -268,6 +268,12 @@ services:
# brought up by hand, an operator sets it here the same way.
TELEMETRY_ENABLED:
TELEMETRY_ENDPOINT:
+ # Which Amplitude project the usage lands in. Unset is prod; Switch
+ # Console writes its own build's environment here for a server it runs.
+ TELEMETRY_ENVIRONMENT:
+ # Whether this is one of the company's own deployments. Switch Console
+ # writes its own answer here for a server it runs.
+ TELEMETRY_INTERNAL:
TELEMETRY_SNAPSHOT_INTERVAL_HOURS:
depends_on:
postgres:
diff --git a/deploy/remote/helm/switch/templates/_helpers.tpl b/deploy/remote/helm/switch/templates/_helpers.tpl
index b0509a837..5a9dd88ca 100644
--- a/deploy/remote/helm/switch/templates/_helpers.tpl
+++ b/deploy/remote/helm/switch/templates/_helpers.tpl
@@ -794,6 +794,10 @@ this one. Drop it once the oldest supported image reads ID_SERVER_NAME. */}}
value: {{ .Values.switchCore.telemetry.endpoint | quote }}
- name: TELEMETRY_SNAPSHOT_INTERVAL_HOURS
value: {{ .Values.switchCore.telemetry.snapshotIntervalHours | quote }}
+- name: TELEMETRY_ENVIRONMENT
+ value: {{ .Values.switchCore.telemetry.environment | quote }}
+- name: TELEMETRY_INTERNAL
+ value: {{ .Values.switchCore.telemetry.internal | quote }}
{{- end }}
# switch-core sits behind the cluster/ALB and enforces its own
# BearerAuthMiddleware, so fastmcp's browser-oriented DNS-rebinding
diff --git a/deploy/remote/helm/switch/values.yaml b/deploy/remote/helm/switch/values.yaml
index a4771e6b8..85e027875 100644
--- a/deploy/remote/helm/switch/values.yaml
+++ b/deploy/remote/helm/switch/values.yaml
@@ -233,6 +233,13 @@ switchCore:
endpoint: https://telemetry.flintai.dev
# How often the usage snapshot is collected and sent.
snapshotIntervalHours: 24
+ # Which Amplitude project the usage lands in: prod, staging, dev or local.
+ # Leave it at prod unless this is one of Flint's own non-production
+ # deployments; anything else is refused at startup.
+ environment: prod
+ # Whether this is one of Flint's own deployments, so staff usage can be
+ # told from adoption. Leave it false on any other deployment.
+ internal: false
# Public base URL the gateway is served on (used for links / OIDC redirects).
frontendBaseUrl: ""
diff --git a/docs/TELEMETRY.md b/docs/TELEMETRY.md
index e4d630345..1d235b691 100644
--- a/docs/TELEMETRY.md
+++ b/docs/TELEMETRY.md
@@ -36,6 +36,8 @@ abbreviated:
"service.name": "switch-console",
"service.version": "0.9.14",
"flint.client_id": "3f2a9c41-8d7e-4b16-9a55-c0e1d2f47b83",
+ "flint_env": "prod",
+ "flint_internal": "false",
"os.type": "darwin",
"os.version": "23.6.0"
},
@@ -72,6 +74,8 @@ machine, in what repository, or what the prompt said.
| `service.name` | constant | `switch-console` |
| `service.version` | app version | `0.9.14`, `1.0.2` |
| `build` | release channel | `dev`, `canary`, `stable` |
+| `flint_env` | which analytics project the event is filed under | `prod` (a released stable build), `staging` (a released canary), `dev` (a packaged build that is not a release), `local` (running from source) |
+| `flint_internal` | whether an account this app is signed in to a Switch server with has a SandboxAQ email address — **only the yes/no is sent, never the address** | `true`, `false`, `unknown` (no account, or only the one a local server signs in with) |
| `os.type` | OS family | `darwin`, `windows`, `linux`, `other` |
| `os.version` | OS release string | `23.6.0`, `10.0.22631`, `6.1.0-53-cloud-amd64` |
| `flint.client_id` | random install UUID | `3f2a9c41-8d7e-4b16-9a55-c0e1d2f47b83` |
@@ -99,12 +103,12 @@ id, no IP field, no Switch identity.
| Event | Fields, with example values |
|---|---|
-| `app_launched` | *(no fields)* |
+| `app_launched` | `install_kind`: `new` (the installation's very first launch) / `updated` (the first launch after its version changed) / `same` |
| `renderer_crashed` | *(no fields)* |
| `update_checked` | `trigger`: `user` / `startup` / `scheduled` · `result`: `available` / `up_to_date` / `failed` |
| `update_downloaded` | `outcome`: `success` / `failure` |
| `update_install_started` | `outcome`: `success` / `failure` |
-| `telemetry_consent_changed` | `source`: `first_run` / `settings` |
+| `telemetry_consent_changed` | `source`: `first_run` / `settings` · `install_kind`: as on `app_launched` |
| `setting_changed` | `setting_key`, one of exactly 15: `theme`, `notifications`, `terminal`, `defaultAgent`, `sessions`, `location`, `localLocation`, `openIn`, `interface`, `browser`, `browserPreview`, `changesViewMode`, `remote`, `onboarding`, `telemetry`. **The new value is never sent** — we learn that someone changed their theme, not to what. |
| `search_performed` | `status`: `ok` / `recents` / `query-too-short` / `failed` · `result_count`: `0`, `3`, `17`. **The query is never sent.** |
diff --git a/docs/old/telemetry-events.md b/docs/old/telemetry-events.md
index a429097a0..d9c9b229d 100644
--- a/docs/old/telemetry-events.md
+++ b/docs/old/telemetry-events.md
@@ -57,6 +57,83 @@ choices are load-bearing rather than incidental:
- **A name prefix per product**, because one Amplitude project holds several.
The Console sends `switch_console.<event>`; the server sends
`switch_core.<event>`.
+- **Numbers go as `doubleValue`, booleans as `boolValue`.** The relay delivers
+ both typed to Amplitude and Datadog — zero and `false` included — so a count
+ can be summed and a flag filtered on at the far end. A number sent as text
+ arrives as a category instead. The relay's own test suite pins this end to
+ end; the tests beside each encoder pin the half Switch controls.
+- **Non-finite numbers are refused before sending.** JSON has no NaN or
+ infinity, so neither could arrive as a number whatever the relay did with it.
+- **At most 100 attributes per record.** The relay drops any record over 128
+ and still answers 200. The margin is held by a test over every event, on both
+ the server and the Console.
+- **At most 16 KiB per event.** The relay drops an event over 32 KiB before it
+ reaches Amplitude, because Amplitude refuses a request over 1 MB. It answers
+ 200 and counts the drop. A test over every event, at its largest possible
+ values, holds the margin on both clients; the largest today is about 3 KiB.
+
+### Which Amplitude project
+
+The relay keeps one Amplitude project per environment and files each event
+under the one its `flint_env` resource attribute names: `prod`, `staging`,
+`dev` or `local`. An event naming none goes to production, and one naming
+anything else is dropped and counted. So every event says where it belongs,
+even `prod`:
+
+- **The server** sends `TELEMETRY_ENVIRONMENT`, which defaults to `prod`
+ because every customer's deployment is production. Flint's own development
+ deployment sets `dev`; `.env.example`, which `just init-env` copies for a
+ developer's machine, sets `local`. Any other value stops the server at
+ startup. It is set by the Helm chart's `switchCore.telemetry.environment`,
+ and forwarded by the standalone compose file.
+- **Switch Console** sends one per build: `prod` from a released stable build,
+ `staging` from a released canary, `dev` from a packaged build that is not a
+ release (a test build, or one packaged on a laptop), and `local` when run
+ from source. The release workflow stamps tagged builds `VITE_RELEASE=1`, and
+ a test holds every build step to it. A server Console runs for the user is
+ given Console's.
+
+`flint_env` rides on the resource, beside the client id. It describes where
+the reporting process runs, not anything inside the deployment, so it is not
+an identifier in the sense of the rule above.
+
+### Internal usage
+
+Staff use the product too, and their usage is real but is not adoption. Three
+signals tell it apart, none of them an identifier:
+
+- **`flint_internal`** on the resource: `"true"` or `"false"` from the server,
+ set by `TELEMETRY_INTERNAL` (Helm `switchCore.telemetry.internal`) on the
+ company's own deployments and never on a customer's. Switch Console sends
+ `"true"` when any account it is signed in to a Switch server with is on
+ `sandboxaq.com` or `sandboxquantum.com`, `"false"` when only other accounts
+ are, and `"unknown"` when none is; only that answer leaves the machine. A
+ server Console runs for the user is given Console's answer.
+- **`user_internal_count`** in the daily snapshot: accounts on those domains,
+ so a deployment that serves both staff and customers can be split.
+- Each deployment's `flint.client_id`, for the company's own deployments by
+ name.
+
+`tenant_id` is deliberately not one of them: no identifier for anything inside
+a deployment is ever sent, and a tenant is inside one.
+
+### What the relay's 200 means
+
+The relay answers `200` once it has accepted a payload, before it forwards
+anything: it batches, and at response time it does not yet know whether
+Amplitude will take the events. So a `200` means *received*, never *delivered*,
+and a sender cannot learn from the response that an event was lost. That is
+deliberate. An error status would be either a `400` the sender can do nothing
+about or a `500` that makes it retry the same payload forever.
+
+Making loss visible is therefore the relay's job, not the sender's: counting
+what it receives, what its own checks drop, and what Amplitude and Datadog
+refuse, and alerting on those counts. The relay's own repository documents the
+metrics, the alerts and how they are switched on, and the gaps that remain.
+
+What that leaves to Switch is not sending anything the relay would drop: the
+event name in both places, finite numbers, the attribute and size margins, and a
+valid client id.
## The rule: abstracted counts, never specifics
@@ -245,8 +322,12 @@ never invisible.
| `tenant_failed_count` | number | tenants whose queries raised and were stepped over. One tenant's failure no longer takes the whole pass down, so the pair is what makes a partial pass self-describing: without it, every count dropping at once is indistinguishable from a deployment losing its users |
| `duration_ms` | number | wall time to collect the pass, on a monotonic clock. Roughly fifteen queries per tenant against the database that is also serving rooms, and a background task is invisible to `switch.http.request.duration` — so this is the only place "what does the snapshot cost at scale" can be answered from |
| `user_count` | number | user accounts that exist |
-| `user_active_1d` | number | distinct humans who interacted in 24h |
+| `user_internal_count` | number | of those, accounts on the company's own email domains — see "Internal usage" |
+| `user_active_1d` | number | distinct Switch **accounts** that used a room with an agent in 24h: an account counts when a chat account it has claimed spoke. One person on two platforms is one account |
| `user_active_7d` | number | same over 7 days |
+| `chat_identity_count` | number | chat identities — Slack, Mattermost and other platform accounts — that exist, claimed by an account or not |
+| `chat_identity_active_1d` | number | distinct chat identities that used a room with an agent in 24h. Most belong to no Switch account and one person may have several, which is why this runs above `user_active_1d`. Servers released before this change reported this figure as `user_active_1d` |
+| `chat_identity_active_7d` | number | same over 7 days |
| `room_count` | number | **the headline figure** — unarchived rooms a *person* created |
| `room_agent_created_count` | number | unarchived rooms an agent created for itself |
| `room_system_created_count` | number | unarchived channels Switch adopted after being invited to them on a platform |
@@ -373,11 +454,14 @@ onboarding. A bundled connector registered before the flag existed is marked
on the setup step's next run; if it first connects before that, its one
`connector_added` reports `false`.
-**`seconds_since_configured` is not setup time for an old connector.** The
-event fires on the first connect *seen with telemetry on*. A connector that
-already existed when telemetry was switched on reports its whole age on its
-first connect afterwards. Deployments that predate telemetry send `-1` in
-`seconds_since_install`, so filtering to `>= 0` drops most of them.
+**`connector_added` fires only for a first connect that was seen.** A
+connector's first connect is recorded whether telemetry is on or not, so one
+that first connected while telemetry was off never reports `connector_added`:
+its setup went unmeasured, and reporting it later would report its whole age.
+Connectors that already existed when this tracking arrived are recorded by a
+migration and never report either. Servers released before it reported an old
+connector's age instead, often millions of seconds; filtering to
+`seconds_since_install >= 0` drops most of those.
`seconds_since_configured` and `failed_attempts_before_success` are what answer
"is one platform too hard". Elapsed time from install mostly measures when