Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.
Python agent-orchestrationaiai-agentschatopsdiscordlangchainmcpmicrosoft-teamsmulti-agentopen-sourceopenaiself-hostedslack
[CHOO-2394] telemetry: pin typed properties and the relay's silent drops, and say which Amplitude project each event belongs in (#593)
Christian McDermott3231a957a9f4
diff --git a/.env.example b/.env.exampleindex f91551fd6..c1ba5ac7d 100644--- a/.env.example+++ b/.env.example@@ -123,6 +123,16 @@ DB_NAME=switch# OTLP_TIMEOUT_SECONDS=10# OTLP_EXPORT_INTERVAL_SECONDS=60+# ── Product telemetry ────────────────────────────────────────────────────────+# Anonymous usage counts, off unless switched on (docs/old/telemetry-events.md).+# TELEMETRY_ENABLED=false+#+# Which Amplitude project this server's usage lands in: prod, staging, dev or+# local. Unset means prod, which is right for every customer deployment. This+# file is for a developer's machine, so it says local, keeping a local run's+# usage out of the production numbers if telemetry is ever switched on.+TELEMETRY_ENVIRONMENT=local+# ── Client identity ──────────────────────────────────────────────────────────# The server half of every member's `@localpart:server` id. Nothing is# contacted at it; the ids are stable public handles. (MATRIX_SERVER_NAME, thediff --git a/.github/workflows/switch-console-release.yml b/.github/workflows/switch-console-release.ymlindex 06d844848..59f6ac2fa 100644--- a/.github/workflows/switch-console-release.yml+++ b/.github/workflows/switch-console-release.yml@@ -68,6 +68,7 @@ jobs:manifest: ${{ steps.resolve.outputs.manifest }}artifact_prefix: ${{ steps.resolve.outputs.artifact_prefix }}vite_build: ${{ steps.resolve.outputs.vite_build }}+ vite_release: ${{ steps.resolve.outputs.vite_release }}steps:- name: Checkoutuses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0@@ -124,6 +125,13 @@ jobs:version="$pkg_version";;esac+ # A tagged build is a published release; a workflow_dispatch test build+ # is not. The app reports its usage to production's Amplitude project+ # (or staging's, for a canary) only when this says it is a release.+ case "$GITHUB_REF" in+ refs/tags/*) vite_release=1 ;;+ *) vite_release= ;;+ esacif [ "$channel" = canary ]; thenbuilder_config=electron-builder.canary.config.tsmanifest=canary@@ -142,6 +150,7 @@ jobs:echo "manifest=$manifest"echo "artifact_prefix=$artifact_prefix"echo "vite_build=$vite_build"+ echo "vite_release=$vite_release"} | tee -a "$GITHUB_OUTPUT"# Creates the (assetless) Release up front so the platform jobs can run@@ -273,6 +282,7 @@ jobs:# Root recursive build (topological): packages/** then the desktop app.env:VITE_BUILD: ${{ needs.resolve.outputs.vite_build }}+ VITE_RELEASE: ${{ needs.resolve.outputs.vite_release }}# Only canary knows where Switch Cloud is; a build without it offers no Cloud.MAIN_VITE_SWITCH_CLOUD_URL: ${{ needs.resolve.outputs.channel == 'canary' && vars.SWITCH_CLOUD_URL || '' }}run: pnpm run build@@ -566,6 +576,7 @@ jobs:- name: Build workspace packages + appenv:VITE_BUILD: ${{ needs.resolve.outputs.vite_build }}+ VITE_RELEASE: ${{ needs.resolve.outputs.vite_release }}# Only canary knows where Switch Cloud is; a build without it offers no Cloud.MAIN_VITE_SWITCH_CLOUD_URL: ${{ needs.resolve.outputs.channel == 'canary' && vars.SWITCH_CLOUD_URL || '' }}run: pnpm run build@@ -677,6 +688,7 @@ jobs:- name: Build workspace packages + appenv:VITE_BUILD: ${{ needs.resolve.outputs.vite_build }}+ VITE_RELEASE: ${{ needs.resolve.outputs.vite_release }}# Only canary knows where Switch Cloud is; a build without it offers no Cloud.MAIN_VITE_SWITCH_CLOUD_URL: ${{ needs.resolve.outputs.channel == 'canary' && vars.SWITCH_CLOUD_URL || '' }}run: pnpm run builddiff --git a/console/AGENTS.md b/console/AGENTS.mdindex 98b857c4d..600b59202 100644--- a/console/AGENTS.md+++ b/console/AGENTS.md@@ -439,8 +439,10 @@ pnpm run lintenough — excess-property checking does not apply through a spread — so the runtimefilter is what makes "nothing free-text can reach a payload" true rather than intended.Permitted: which of the catalogued things happened, agent type, local-vs-remote,- success-vs-failure, how long an operation took, app version, operating system, and the- random install id. A duration is a number rather than a value from a fixed set,+ success-vs-failure, how long an operation took, app version, operating system, the+ random install id, which kind of launch it was (first, after an update, or neither),+ and whether a signed-in account is on a company email domain — the yes/no only, read+ from the domain, never the address. A duration is a number rather than a value from a fixed set,so it is held to `TelemetryDurationMs`: measured on a monotonic clock, wholemilliseconds, and never a span that could encode something else. That is a brandedtype, not an alias for `number`, and `startTimer()` is the only thing that mints one —diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.test.tsindex 7db87948f..3f90a2f9b 100644--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.test.ts+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.test.ts@@ -31,6 +31,8 @@ describe('buildEnvFile', () => {ports: { gateway: 51000, api: 51001, mattermost: 51002, postgres: 51003 },secrets,telemetryEnabled: false,+ telemetryEnvironment: 'prod',+ telemetryInternal: false,});const vars = Object.fromEntries(env@@ -157,10 +159,49 @@ describe('buildEnvFile', () => {ports: { gateway: 51000, api: 51001, mattermost: 51002, postgres: 51003 },secrets,telemetryEnabled: true,+ telemetryEnvironment: 'prod',+ telemetryInternal: false,});expect(sharing).toContain('TELEMETRY_ENABLED=true');});+ it('tells the server which Amplitude project its usage belongs in', () => {+ // The relay files each event under the project its environment names, so a+ // server a development build runs must say so as plainly as the app does.+ const fromDevBuild = buildEnvFile({+ version: '1.2.3',+ registry: 'ghcr.io',+ namespace: 'sandbox-quantum',+ ports: { gateway: 51000, api: 51001, mattermost: 51002, postgres: 51003 },+ secrets,+ telemetryEnabled: true,+ telemetryEnvironment: 'local',+ telemetryInternal: false,+ });++ expect(fromDevBuild).toContain('TELEMETRY_ENVIRONMENT=local');+ expect(vars.TELEMETRY_ENVIRONMENT).toBe('prod');+ // Written but never forwarded would reach no container.+ expect(composeYaml).toMatch(/^\s+TELEMETRY_ENVIRONMENT:\s*$/m);+ });++ it('tells the server whether the person it runs for is staff', () => {+ const forStaff = buildEnvFile({+ version: '1.2.3',+ registry: 'ghcr.io',+ namespace: 'sandbox-quantum',+ ports: { gateway: 51000, api: 51001, mattermost: 51002, postgres: 51003 },+ secrets,+ telemetryEnabled: true,+ telemetryEnvironment: 'prod',+ telemetryInternal: true,+ });++ expect(forStaff).toContain('TELEMETRY_INTERNAL=true');+ expect(vars.TELEMETRY_INTERNAL).toBe('false');+ expect(composeYaml).toMatch(/^\s+TELEMETRY_INTERNAL:\s*$/m);+ });+it('is the only thing the compose file needs to forward the gate', () => {// A value in the `.env` that the `switch` service does not name never// reaches the container, which is the failure mode this pairing exists to@@ -186,6 +227,8 @@ describe('readStackEnv', () => {ports,secrets,telemetryEnabled: true,+ telemetryEnvironment: 'prod',+ telemetryInternal: false,});it('reads back exactly what buildEnvFile wrote', () => {@@ -298,6 +341,8 @@ describe('keysDisagreeing', () => {ports,secrets,telemetryEnabled: true,+ telemetryEnvironment: 'prod',+ telemetryInternal: false,});it('finds nothing wrong with a copy of the same settings', () => {diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.tsindex 1bd28cba1..7da8ed79d 100644--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.ts+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.ts@@ -1,3 +1,4 @@+import type { TelemetryFlintEnv } from '../telemetry/config';import {LOCAL_SERVER_ADMIN_EMAIL,LOCAL_SERVER_BIND_ADDR,@@ -23,6 +24,13 @@ export type LocalServerEnvParams = {* and nothing else — not a separate server-side preference (CHOO-2890).*/telemetryEnabled: boolean;+ /**+ * The Amplitude project the server's usage lands in: the Console build's+ * own, so a development build's server is as much `local` as the app.+ */+ telemetryEnvironment: TelemetryFlintEnv;+ /** Whether the person this server runs for is staff, as Console tells it. */+ telemetryInternal: boolean;};/** The variable the server reads its telemetry gate from. Named here because@@ -36,7 +44,16 @@ export const TELEMETRY_ENABLED_KEY = 'TELEMETRY_ENABLED';* app's pin. Kept as a pure function so the exact output is unit-testable.*/export function buildEnvFile(params: LocalServerEnvParams): string {- const { version, registry, namespace, ports, secrets, telemetryEnabled } = params;+ const {+ version,+ registry,+ namespace,+ ports,+ secrets,+ telemetryEnabled,+ telemetryEnvironment,+ telemetryInternal,+ } = params;return ['# Generated by Switch Console local-server mode — do not edit by hand.','# Secrets originate in the app encrypted store; this file is derived.',@@ -87,6 +104,8 @@ export function buildEnvFile(params: LocalServerEnvParams): string {// because the compose file passes this key through only when the `.env`// carries it — so "off" has to be said, not left unsaid.`${TELEMETRY_ENABLED_KEY}=${telemetryEnabled ? 'true' : 'false'}`,+ `TELEMETRY_ENVIRONMENT=${telemetryEnvironment}`,+ `TELEMETRY_INTERNAL=${telemetryInternal ? 'true' : 'false'}`,'','MATTERMOST_ADMIN_USER=admin',`MATTERMOST_ADMIN_PASSWORD=${secrets.mattermostAdminPassword}`,diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-matrix-migration.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-matrix-migration.test.tsindex b8c8fd5d5..8d90868aa 100644--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-matrix-migration.test.ts+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-matrix-migration.test.ts@@ -59,6 +59,9 @@ vi.mock('./telemetry-consent', () => ({telemetryConsent: () => Promise.resolve(false),readDeployedTelemetry: vi.fn(),}));+vi.mock('@main/core/telemetry/internal-account', () => ({+ currentInternalFlag: async () => 'unknown',+}));vi.mock('@main/core/switch-servers/servers-store', () => ({assertManagedServerUrlFree: () => Promise.resolve(),ensureManagedServer: () => Promise.resolve({ id: 'srv-1' }),diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-shared-host.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-shared-host.test.tsindex 748bcb941..c9fa83b98 100644--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-shared-host.test.ts+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-shared-host.test.ts@@ -89,6 +89,9 @@ vi.mock('./ports', () => ({const telemetryConsentMock = vi.hoisted(() => vi.fn(() => Promise.resolve(false)));vi.mock('./telemetry-consent', () => ({ telemetryConsent: telemetryConsentMock }));const assertManagedServerUrlFreeMock = vi.hoisted(() => vi.fn(() => Promise.resolve()));+vi.mock('@main/core/telemetry/internal-account', () => ({+ currentInternalFlag: async () => 'unknown',+}));vi.mock('@main/core/switch-servers/servers-store', () => ({assertManagedServerUrlFree: assertManagedServerUrlFreeMock,ensureManagedServer: ensureManagedServerMock,diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.test.tsindex 278caaccf..9adb2861c 100644--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.test.ts+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.test.ts@@ -17,6 +17,9 @@ const logError = vi.hoisted(() => vi.fn());const logWarn = vi.hoisted(() => vi.fn());const buildEnvFileMock = vi.hoisted(() => vi.fn((_params: unknown) => 'SWITCH_VERSION=0.11.0\n'));const telemetryConsentMock = vi.hoisted(() => vi.fn(() => Promise.resolve(false)));+const currentInternalFlagMock = vi.hoisted(() =>+ vi.fn((): Promise<'true' | 'false' | 'unknown'> => Promise.resolve('unknown'))+);// Pin the build's expected version so the guard's arithmetic is not coupled to// whatever the real pin happens to be.@@ -65,6 +68,9 @@ vi.mock('./telemetry-consent', () => ({readDeployedTelemetry: vi.fn(),}));const setActiveServerIdMock = vi.hoisted(() => vi.fn());+vi.mock('@main/core/telemetry/internal-account', () => ({+ currentInternalFlag: currentInternalFlagMock,+}));vi.mock('@main/core/switch-servers/servers-store', () => ({assertManagedServerUrlFree: () => Promise.resolve(),ensureManagedServer: () => Promise.resolve({ id: 'srv-1' }),@@ -83,6 +89,7 @@ vi.mock('./managed-upgrade', () => ({}));const { startStack } = await import('./pipeline');+const { currentFlintEnv } = await import('../telemetry/config');const { ENV_FILE_NAME } = await import('./constants');function options() {@@ -118,6 +125,7 @@ beforeEach(() => {vi.clearAllMocks();prepareUpgradeMock.mockResolvedValue(null);telemetryConsentMock.mockResolvedValue(false);+ currentInternalFlagMock.mockResolvedValue('unknown');buildEnvFileMock.mockReturnValue('SWITCH_VERSION=0.11.0\n');});@@ -161,6 +169,32 @@ describe('startStack version guard', () => {expect(composeUpMock).toHaveBeenCalledOnce();});+ it('tells the server which project its usage belongs in, and whether its person is staff', async () => {+ readDeployedVersionMock.mockResolvedValue({ kind: 'absent' });+ currentInternalFlagMock.mockResolvedValue('true');+ const { opts } = options();++ await startStack(opts);++ expect(buildEnvFileMock).toHaveBeenCalledWith(+ expect.objectContaining({+ telemetryEnvironment: currentFlintEnv(),+ telemetryInternal: true,+ })+ );+ });++ it('does not call an unknown account staff', async () => {+ readDeployedVersionMock.mockResolvedValue({ kind: 'absent' });+ const { opts } = options();++ await startStack(opts);++ expect(buildEnvFileMock).toHaveBeenCalledWith(+ expect.objectContaining({ telemetryInternal: false })+ );+ });+it('starts a fresh host with nothing deployed', async () => {readDeployedVersionMock.mockResolvedValue({ kind: 'absent' });const { opts } = options();diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.tsindex c9490ae1f..8166c9928 100644--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.ts+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.ts@@ -14,6 +14,8 @@ import {type StartLocalServerResult,} from '@shared/core/managed-switch-server/managed-switch-server';import type { ManagedServerRef } from '@shared/core/switch-servers/switch-servers';+import { currentFlintEnv } from '../telemetry/config';+import { currentInternalFlag } from '../telemetry/internal-account';import { bundledComposeYaml } from './bundled-compose';import { checkoutBuildOverrideYaml } from './checkout-build';import { composeDown, composeUp } from './compose';@@ -565,6 +567,8 @@ export async function startStack(opts: StartStackOptions): Promise<StartLocalSerports: settings.ports,secrets: settings.secrets,telemetryEnabled,+ telemetryEnvironment: currentFlintEnv(),+ telemetryInternal: (await currentInternalFlag()) === 'true',});await host.writeFile(ENV_FILE_NAME, env, 0o600);diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/resources/standalone-docker-compose.pinned.yml b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/resources/standalone-docker-compose.pinned.ymlindex b283d4d84..1a7d73bcc 100644--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/resources/standalone-docker-compose.pinned.yml+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/resources/standalone-docker-compose.pinned.yml@@ -275,6 +275,12 @@ services:# brought up by hand, an operator sets it here the same way.TELEMETRY_ENABLED:TELEMETRY_ENDPOINT:+ # Which Amplitude project the usage lands in. Unset is prod; Switch+ # Console writes its own build's environment here for a server it runs.+ TELEMETRY_ENVIRONMENT:+ # Whether this is one of the company's own deployments. Switch Console+ # writes its own answer here for a server it runs.+ TELEMETRY_INTERNAL:TELEMETRY_SNAPSHOT_INTERVAL_HOURS:depends_on:postgres:diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-state.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-state.test.tsindex 281e991d9..0c1c69b83 100644--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-state.test.ts+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-state.test.ts@@ -51,6 +51,8 @@ function envFor(overrides: Partial<LocalServerSecrets> = {}, version = '0.27.0')ports,secrets: { ...secrets, ...overrides },telemetryEnabled: false,+ telemetryEnvironment: 'prod',+ telemetryInternal: false,});}diff --git a/console/apps/switch-console-desktop/src/main/core/settings/controller.telemetry.test.ts b/console/apps/switch-console-desktop/src/main/core/settings/controller.telemetry.test.tsindex 21d562cc9..374dfd939 100644--- a/console/apps/switch-console-desktop/src/main/core/settings/controller.telemetry.test.ts+++ b/console/apps/switch-console-desktop/src/main/core/settings/controller.telemetry.test.ts@@ -16,6 +16,7 @@ const { h } = vi.hoisted(() => ({}));vi.mock('@main/core/telemetry/telemetry-service', () => ({ trackEvent: h.trackEvent }));+vi.mock('@main/core/telemetry/launch-history', () => ({ currentInstallKind: () => 'new' }));vi.mock('./settings-service', () => ({appSettingsService: {get: h.get,@@ -87,6 +88,7 @@ describe('agreeing to share usage data', () => {expect(h.trackEvent).toHaveBeenCalledWith('telemetry_consent_changed', {source: 'first_run',+ install_kind: 'new',});});@@ -95,7 +97,10 @@ describe('agreeing to share usage data', () => {await appSettingsController.update('telemetry', { enabled: true, askedAt: 1 } as never);- expect(h.trackEvent).toHaveBeenCalledWith('telemetry_consent_changed', { source: 'settings' });+ expect(h.trackEvent).toHaveBeenCalledWith('telemetry_consent_changed', {+ source: 'settings',+ install_kind: 'new',+ });});it('says nothing when someone declines', async () => {diff --git a/console/apps/switch-console-desktop/src/main/core/settings/controller.ts b/console/apps/switch-console-desktop/src/main/core/settings/controller.tsindex a8ba13462..b84c9fb8e 100644--- a/console/apps/switch-console-desktop/src/main/core/settings/controller.ts+++ b/console/apps/switch-console-desktop/src/main/core/settings/controller.ts@@ -1,4 +1,5 @@import type { TelemetrySettingKey } from '@main/core/telemetry/events';+import { currentInstallKind } from '@main/core/telemetry/launch-history';import { trackEvent } from '@main/core/telemetry/telemetry-service';import type { TelemetrySettings } from '@shared/core/app-settings';import { createRPCController } from '@shared/lib/ipc/rpc';@@ -23,6 +24,7 @@ function reportConsent(previous: TelemetrySettings, next: TelemetrySettings): votrackEvent('telemetry_consent_changed', {source: previous.askedAt === null ? 'first_run' : 'settings',+ install_kind: currentInstallKind(),});}diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.tsindex 6a0ca46dd..b311047bf 100644--- a/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.ts+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.ts@@ -2,6 +2,7 @@ import { randomUUID } from 'node:crypto';import { and, desc, eq, isNull, ne, or, sql } from 'drizzle-orm';import { encryptedAppSecretsStore } from '@main/core/secrets/encrypted-app-secrets-store';import type { TelemetryEventMap } from '@main/core/telemetry/events';+import { forgetSessionAccount, recordSessionAccount } from '@main/core/telemetry/internal-account';import { trackEvent } from '@main/core/telemetry/telemetry-service';import { forgetServerSession } from '@main/core/workspaces/workspace-session';import {@@ -15,6 +16,7 @@ import {} from '@main/core/workspaces/workspaces-store';import { db } from '@main/db/client';import { type SwitchServerRow, switchServers } from '@main/db/schema';+import { log } from '@main/lib/logger';import {urlOrigin,type AddServerParams,@@ -375,8 +377,14 @@ export async function getSessionCookie(serverId: string): Promise<string | null>export async function setSessionCookie(serverId: string, jwt: string): Promise<void> {await encryptedAppSecretsStore.setSecret(cookieSecretKey(serverId), jwt);+ await recordSessionAccount(serverId, jwt).catch((error: unknown) => {+ log.warn('switch-servers: could not record whether the account is internal', { error });+ });}export async function deleteSessionCookie(serverId: string): Promise<void> {await encryptedAppSecretsStore.deleteSecret(cookieSecretKey(serverId));+ await forgetSessionAccount(serverId).catch((error: unknown) => {+ log.warn('switch-servers: could not forget whether the account was internal', { error });+ });}diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/catalogue.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/catalogue.test.tsindex ac0975a66..a692e4781 100644--- a/console/apps/switch-console-desktop/src/main/core/telemetry/catalogue.test.ts+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/catalogue.test.ts@@ -26,6 +26,8 @@ const CONTEXT: TelemetryContext = {osType: 'darwin',osVersion: '24.3.0',build: 'stable',+ flintEnv: 'prod',+ internal: 'false',timeMs: 1_700_000_000_000,};@@ -109,6 +111,34 @@ describe('the catalogue as a whole', () => {);});+ it.each(EVENT_NAMES)('sends %s well under the attribute count the relay drops at', (name) => {+ // The relay drops any record with more than 128 attributes, and still+ // answers 200. Held here, over every event, with room to spare.+ const payload = buildOtlpPayload(name, sampleEvent(name) as never, CONTEXT);++ expect(recordOf(payload).attributes.length).toBeLessThanOrEqual(100);+ });++ it.each(EVENT_NAMES)('sends %s well under the event size the relay drops at', (name) => {+ // The relay drops any Amplitude event whose JSON is over 32 KiB, and still+ // answers 200. Every string here is longer than any value the catalogue+ // allows, and the OTLP attributes wrap each value in more JSON than the+ // Amplitude event does, so this over-counts.+ const largest = Object.fromEntries(+ Object.entries(sampleEvent(name)).map(([key, value]) => [+ key,+ typeof value === 'string'+ ? 'x'.repeat(256)+ : typeof value === 'number'+ ? -123456789012345.67+ : false,+ ])+ );+ const payload = buildOtlpPayload(name, largest as never, CONTEXT);++ expect(JSON.stringify(recordOf(payload).attributes).length).toBeLessThanOrEqual(16384);+ });+it.each(EVENT_NAMES)('refuses to send %s with a property missing', (name) => {const properties = sampleEvent(name);const declared = TELEMETRY_EVENT_PROPERTIES[name];diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/config.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/config.test.tsindex 28f08ac43..4ce817536 100644--- a/console/apps/switch-console-desktop/src/main/core/telemetry/config.test.ts+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/config.test.ts@@ -1,13 +1,18 @@import { describe, expect, it } from 'vitest';import {+ flintEnvFor,resolveTelemetryEnvironment,TELEMETRY_RELAY_ENDPOINT,type TelemetryBuildChannel,type TelemetryEnvironment,} from './config';-function resolve(build: TelemetryBuildChannel, env: TelemetryEnvironment['env'] = {}) {- return resolveTelemetryEnvironment({ build, env });+function resolve(+ build: TelemetryBuildChannel,+ env: TelemetryEnvironment['env'] = {},+ release = build !== 'dev'+) {+ return resolveTelemetryEnvironment({ build, release, env });}describe('a dev run', () => {@@ -18,7 +23,7 @@ describe('a dev run', () => {it('reports to the relay once opted in', () => {expect(resolve('dev', { SWITCHDASH_TELEMETRY_DEV: '1' })).toEqual({enabled: true,- config: { endpoint: TELEMETRY_RELAY_ENDPOINT, build: 'dev' },+ config: { endpoint: TELEMETRY_RELAY_ENDPOINT, build: 'dev', flintEnv: 'local' },});});@@ -47,7 +52,11 @@ describe.each(['stable', 'canary'] as const)('a %s build', (build) => {// this replaced, there is no build that is inert for want of a key.expect(resolve(build)).toEqual({enabled: true,- config: { endpoint: TELEMETRY_RELAY_ENDPOINT, build },+ config: {+ endpoint: TELEMETRY_RELAY_ENDPOINT,+ build,+ flintEnv: build === 'canary' ? 'staging' : 'prod',+ },});});@@ -68,3 +77,24 @@ describe.each(['stable', 'canary'] as const)('a %s build', (build) => {expect(resolution.enabled && resolution.config.build).toBe(build);});});++describe('the Amplitude project a build reports to', () => {+ // The relay files each event under the project its flint_env names, so only a+ // released stable build's events are production's.+ it.each([+ ['stable', true, 'prod'],+ ['canary', true, 'staging'],+ ['stable', false, 'dev'],+ ['canary', false, 'dev'],+ ['dev', false, 'local'],+ ['dev', true, 'local'],+ ] as const)('is %s, released %s → %s', (build, release, environment) => {+ expect(flintEnvFor({ build, release })).toBe(environment);+ });++ it('is carried in the resolved config, so every event of a build agrees', () => {+ const resolution = resolve('stable', {}, false);++ expect(resolution.enabled && resolution.config.flintEnv).toBe('dev');+ });+});diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/config.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/config.tsindex b812640db..bf237de34 100644--- a/console/apps/switch-console-desktop/src/main/core/telemetry/config.ts+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/config.ts@@ -1,4 +1,4 @@-import { IS_CANARY } from '@shared/app-identity';+import { IS_CANARY, IS_RELEASE_BUILD } from '@shared/app-identity';type ImportMetaWithEnv = ImportMeta & { env?: { DEV?: boolean } };@@ -29,6 +29,7 @@ export type TelemetryBuildChannel = 'dev' | 'canary' | 'stable';export type TelemetryConfig = {endpoint: string;build: TelemetryBuildChannel;+ flintEnv: TelemetryFlintEnv;};/**@@ -41,6 +42,38 @@ export type TelemetryResolution =| { enabled: true; config: TelemetryConfig }| { enabled: false; reason: TelemetryDisabledReason };+/**+ * The Amplitude project an event belongs in, sent as the `flint_env` resource+ * attribute. The relay keeps one project per environment and files each event+ * under the one this names, so a developer's app and a pre-release never land+ * in production's numbers. `build` stays on every event besides, for slicing+ * within a project.+ */+export type TelemetryFlintEnv = 'prod' | 'staging' | 'dev' | 'local';++/**+ * Running from source is `local`. A packaged build that is not a release — a+ * workflow_dispatch test build, or one packaged on a laptop — is `dev`, whatever+ * channel it was built as. Only a released canary is `staging`, and only a+ * released stable build is `prod`.+ */+export function flintEnvFor({+ build,+ release,+}: {+ build: TelemetryBuildChannel;+ release: boolean;+}): TelemetryFlintEnv {+ if (build === 'dev') return 'local';+ if (!release) return 'dev';+ return build === 'canary' ? 'staging' : 'prod';+}++/** The project for the build that is actually running. */+export function currentFlintEnv(): TelemetryFlintEnv {+ return flintEnvFor({ build: telemetryBuildChannel(), release: IS_RELEASE_BUILD });+}+export function telemetryBuildChannel(): TelemetryBuildChannel {if (viteEnv?.DEV === true) return 'dev';return IS_CANARY ? 'canary' : 'stable';@@ -49,6 +82,8 @@ export function telemetryBuildChannel(): TelemetryBuildChannel {/** Everything the decision depends on, so that it depends on nothing else. */export type TelemetryEnvironment = {build: TelemetryBuildChannel;+ /** Whether the release workflow stamped this build as a published release. */+ release: boolean;env: NodeJS.ProcessEnv;};@@ -69,6 +104,7 @@ export type TelemetryEnvironment = {*/export function resolveTelemetryEnvironment({build,+ release,env,}: TelemetryEnvironment): TelemetryResolution {const isDev = build === 'dev';@@ -77,10 +113,21 @@ export function resolveTelemetryEnvironment({}const override = isDev ? env.SWITCHDASH_TELEMETRY_ENDPOINT?.trim() : undefined;- return { enabled: true, config: { endpoint: override || TELEMETRY_RELAY_ENDPOINT, build } };+ return {+ enabled: true,+ config: {+ endpoint: override || TELEMETRY_RELAY_ENDPOINT,+ build,+ flintEnv: flintEnvFor({ build, release }),+ },+ };}/** The same decision for the build that is actually running. */export function resolveTelemetryConfig(): TelemetryResolution {- return resolveTelemetryEnvironment({ build: telemetryBuildChannel(), env: process.env });+ return resolveTelemetryEnvironment({+ build: telemetryBuildChannel(),+ release: IS_RELEASE_BUILD,+ env: process.env,+ });}diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/events.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/events.tsindex d0c047a5c..670e52cab 100644--- a/console/apps/switch-console-desktop/src/main/core/telemetry/events.ts+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/events.ts@@ -18,6 +18,13 @@ import type {*/export type TelemetryLocationKind = 'local' | 'remote' | 'unknown';+/**+ * Which kind of launch an event belongs to: `new` on an installation's very+ * first launch, `updated` on the first launch after its version changed, and+ * `same` otherwise. Counting `new` counts new installs; `updated`, upgrades.+ */+export type TelemetryInstallKind = 'new' | 'updated' | 'same';+/*** Which CLI agent this concerns.*@@ -288,16 +295,11 @@ export type TelemetryHostSetupStepKind = HostSetupStepKind | 'unknown';* and it is far easier to add an event than to take one away once dashboards* depend on it.*/-/**- * An event with no properties of its own. `Record<never, never>` rather than- * `Record<string, never>`, whose `keyof` is `string` — which claims every- * property name, including the ambient ones the emitter adds.- */-type NoProperties = Record<never, never>;-export type TelemetryEventMap = {- /** The app started. Its interesting fields are the ambient ones. */- app_launched: NoProperties;+ /** The app started, and which kind of launch it was. */+ app_launched: {+ install_kind: TelemetryInstallKind;+ };agent_created: {agent_type: TelemetryAgentType;location: TelemetryLocationKind;@@ -579,6 +581,8 @@ export type TelemetryEventMap = {*/telemetry_consent_changed: {source: 'first_run' | 'settings';+ /** This launch's kind, so a new install agreeing at first run is one filter. */+ install_kind: TelemetryInstallKind;};};@@ -593,7 +597,7 @@ export type TelemetryEventName = keyof TelemetryEventMap;* checking does not apply — cannot ride along into a payload unnoticed.*/export const TELEMETRY_EVENT_PROPERTIES = {- app_launched: [],+ app_launched: ['install_kind'],agent_created: ['agent_type', 'location', 'outcome', 'failure_reason', 'entry_point'],session_started: ['agent_type',@@ -659,7 +663,7 @@ export const TELEMETRY_EVENT_PROPERTIES = {onboarding_completed: [],add_server_step: ['step', 'choice', 'first_run'],renderer_crashed: [],- telemetry_consent_changed: ['source'],+ telemetry_consent_changed: ['source', 'install_kind'],session_attached: ['agent_type', 'outcome'],} as const satisfies { [K in TelemetryEventName]: readonly (keyof TelemetryEventMap[K])[] };diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.test.tsnew file mode 100644index 000000000..b7da2f7aa--- /dev/null+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.test.ts@@ -0,0 +1,99 @@+import { describe, expect, it, vi } from 'vitest';++vi.mock('@main/db/kv', () => ({+ KV: class {+ private values = new Map<string, unknown>();+ async get(key: string) {+ return this.values.get(key) ?? null;+ }+ async set(key: string, value: unknown) {+ this.values.set(key, structuredClone(value));+ }+ },+}));++const {+ currentInternalFlag,+ emailFromJwt,+ forgetSessionAccount,+ internalFrom,+ isInternalEmail,+ recordSessionAccount,+} = await import('./internal-account');++function jwtWith(payload: Record<string, unknown>): string {+ const part = (value: object) => Buffer.from(JSON.stringify(value)).toString('base64url');+ return `${part({ alg: 'HS256' })}.${part(payload)}.signature`;+}++describe('whether an account is staff', () => {+ it.each([+ '[email protected]',+ '[email protected]',+ '[email protected]',+ '[email protected]',+ ])('counts %s', (email) => {+ expect(isInternalEmail(email)).toBe(true);+ });++ 'does not count %s',+ (email) => {+ expect(isInternalEmail(email)).toBe(false);+ }+ );+});++describe('whether the person is staff', () => {+ it('is true when any account is', () => {+ expect(internalFrom(['[email protected]', '[email protected]'])).toBe('true');+ });++ it('is false when only outside accounts are signed in', () => {+ expect(internalFrom(['[email protected]', null])).toBe('false');+ });++ it('is unknown when no account says, including the one a local server signs in with', () => {+ expect(internalFrom([])).toBe('unknown');+ expect(internalFrom([null, '[email protected]'])).toBe('unknown');+ });+});++describe('reading the account from a session', () => {+ it('reads the email claim', () => {+ expect(emailFromJwt(jwtWith({ sub: 'u1', email: '[email protected]' }))).toBe(+ );+ });++ it('reads nothing from a malformed token or one without the claim', () => {+ expect(emailFromJwt('not-a-jwt')).toBeNull();+ expect(emailFromJwt(jwtWith({ sub: 'u1' }))).toBeNull();+ });+});++describe('the answer kept beside each session', () => {+ it('is recorded when a session is stored, and forgotten when it goes', async () => {+ expect(await currentInternalFlag()).toBe('unknown');++ await recordSessionAccount('outside', jwtWith({ email: '[email protected]' }));+ expect(await currentInternalFlag()).toBe('false');++ await recordSessionAccount('staff', jwtWith({ email: '[email protected]' }));+ expect(await currentInternalFlag()).toBe('true');++ await forgetSessionAccount('staff');+ expect(await currentInternalFlag()).toBe('false');++ await forgetSessionAccount('outside');+ expect(await currentInternalFlag()).toBe('unknown');+ });++ it('follows a server signed in again as someone else', async () => {+ await recordSessionAccount('server', jwtWith({ email: '[email protected]' }));+ await recordSessionAccount('server', jwtWith({ email: '[email protected]' }));++ expect(await currentInternalFlag()).toBe('false');+ await forgetSessionAccount('server');+ });+});diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.tsnew file mode 100644index 000000000..e8544720a--- /dev/null+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/internal-account.ts@@ -0,0 +1,98 @@+import { LOCAL_SERVER_ADMIN_EMAIL } from '@main/core/managed-switch-server/constants';+import { KV } from '@main/db/kv';++/**+ * Email domains whose accounts are the company's own staff. Their usage is+ * internal: real, but not adoption, and product needs to tell the two apart.+ */+export const INTERNAL_EMAIL_DOMAINS: readonly string[] = ['sandboxaq.com', 'sandboxquantum.com'];++/**+ * Whether the person using this installation is staff, as the `flint_internal`+ * resource attribute: `true`, `false`, or `unknown` when no account says. Only+ * this travels; the address it was read from never leaves the machine.+ */+export type TelemetryInternal = 'true' | 'false' | 'unknown';++export function isInternalEmail(email: string): boolean {+ const domain = email.slice(email.lastIndexOf('@') + 1).toLowerCase();+ return INTERNAL_EMAIL_DOMAINS.some((d) => domain === d || domain.endsWith(`.${d}`));+}++/**+ * From the accounts this installation is signed in with. Any staff account+ * makes it internal; otherwise any real account makes it external. The+ * account a server Console runs for the user is a fixed local address that+ * says nothing about the person, so it does not count.+ */+export function internalFrom(emails: readonly (string | null)[]): TelemetryInternal {+ const real = emails.filter(+ (email): email is string =>+ email !== null && email.includes('@') && email.toLowerCase() !== LOCAL_SERVER_ADMIN_EMAIL+ );+ if (real.some(isInternalEmail)) return 'true';+ return real.length > 0 ? 'false' : 'unknown';+}++/**+ * The `email` claim of a gateway session, read without verifying it. That is+ * enough here: the token was issued to this app by the server, and the result+ * only labels usage data.+ */+export function emailFromJwt(jwt: string): string | null {+ const parts = jwt.split('.');+ if (parts.length !== 3) return null;+ try {+ const payload = JSON.parse(Buffer.from(parts[1], 'base64url').toString('utf8')) as {+ email?: unknown;+ };+ return typeof payload.email === 'string' ? payload.email : null;+ } catch {+ return null;+ }+}++/**+ * Each server's answer, kept beside its session rather than read from it.+ * Working it out from the stored session would decrypt every server's cookie+ * on every event, and a failed decrypt deletes the cookie: a keychain that is+ * briefly locked would sign people out of servers they were not even using.+ * So it is worked out once, from the token in hand, when a session is stored.+ */+const store = new KV<{ accounts: Record<string, TelemetryInternal> }>('telemetry-accounts');++let writes: Promise<void> = Promise.resolve();++function update(change: (accounts: Record<string, TelemetryInternal>) => void): Promise<void> {+ writes = writes.then(async () => {+ const accounts = (await store.get('accounts')) ?? {};+ change(accounts);+ await store.set('accounts', accounts);+ });+ return writes;+}++/** Record the account a server's session was issued to. */+export function recordSessionAccount(serverId: string, jwt: string): Promise<void> {+ return update((accounts) => {+ accounts[serverId] = internalFrom([emailFromJwt(jwt)]);+ });+}++/** Forget a server's account when its session goes. */+export function forgetSessionAccount(serverId: string): Promise<void> {+ return update((accounts) => {+ delete accounts[serverId];+ });+}++/**+ * Across every server this installation holds a session for. A session stored+ * before this was recorded reads as unknown until it is next renewed or+ * signed in again.+ */+export async function currentInternalFlag(): Promise<TelemetryInternal> {+ const answers = Object.values((await store.get('accounts')) ?? {});+ if (answers.includes('true')) return 'true';+ return answers.includes('false') ? 'false' : 'unknown';+}diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.test.tsnew file mode 100644index 000000000..bc04e4cde--- /dev/null+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.test.ts@@ -0,0 +1,56 @@+import { describe, expect, it, vi } from 'vitest';++vi.mock('@main/db/kv', () => ({+ KV: class {+ private values = new Map<string, unknown>();+ async get(key: string) {+ return this.values.get(key) ?? null;+ }+ async set(key: string, value: unknown) {+ this.values.set(key, structuredClone(value));+ }+ },+}));++const { installKindFor, recordLaunch } = await import('./launch-history');++describe('which kind of launch this is', () => {+ it('is new on an installation’s very first launch, so counting it counts installs', () => {+ expect(installKindFor({ lastVersion: null, version: '1.2.0', databaseExisted: false })).toBe(+ 'new'+ );+ });++ it('is an upgrade on the first launch at a different version', () => {+ expect(installKindFor({ lastVersion: '1.1.0', version: '1.2.0', databaseExisted: true })).toBe(+ 'updated'+ );+ });++ it('is the same on a relaunch', () => {+ expect(installKindFor({ lastVersion: '1.2.0', version: '1.2.0', databaseExisted: true })).toBe(+ 'same'+ );+ });++ it('is an upgrade, never new, for an installation that predates this tracking', () => {+ // It has a database but no record: it was installed at some earlier version,+ // and this is its first launch at one that records.+ expect(installKindFor({ lastVersion: null, version: '1.2.0', databaseExisted: true })).toBe(+ 'updated'+ );+ });+});++describe('canary and stable, which share one database', () => {+ it('do not read a switch between them as an upgrade', async () => {+ const launch = (channel: 'canary' | 'stable', version: string) =>+ recordLaunch({ version, channel, databaseExisted: true });++ expect(await launch('stable', '1.4.0')).toBe('updated');+ expect(await launch('canary', '1.5.0-canary.2')).toBe('updated');+ expect(await launch('stable', '1.4.0')).toBe('same');+ expect(await launch('canary', '1.5.0-canary.2')).toBe('same');+ expect(await launch('canary', '1.5.0-canary.3')).toBe('updated');+ });+});diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.tsnew file mode 100644index 000000000..97d2b8e20--- /dev/null+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/launch-history.ts@@ -0,0 +1,69 @@+import { KV } from '@main/db/kv';+import { log } from '@main/lib/logger';+import type { TelemetryInstallKind } from './events';++/**+ * Which kind of launch this is, from what the last launch recorded.+ *+ * `new` is the first launch an installation ever makes, so counting it counts+ * new installs; `updated` is the first launch on a different version than the+ * last one, an upgrade; `same` is everything else. An installation with a+ * database but no record predates this tracking, and its first launch with it+ * is on a version it was not installed at, so it reads as `updated`.+ */+export function installKindFor({+ lastVersion,+ version,+ databaseExisted,+}: {+ lastVersion: string | null;+ version: string;+ databaseExisted: boolean;+}): TelemetryInstallKind {+ if (lastVersion === null) return databaseExisted ? 'updated' : 'new';+ return lastVersion === version ? 'same' : 'updated';+}++/**+ * One record per channel: canary and stable share a database, and comparing+ * one's version with the other's would read every switch between them as an+ * upgrade.+ */+const store = new KV<Record<string, string>>('telemetry-launches');++let current: TelemetryInstallKind | null = null;++/**+ * Record this launch and say which kind it is. Recorded on every launch,+ * whether or not usage is shared: it is local, and the next launch has to know+ * this one happened to tell an upgrade from a relaunch.+ */+export async function recordLaunch({+ version,+ channel,+ databaseExisted,+}: {+ version: string;+ channel: 'canary' | 'stable';+ databaseExisted: boolean;+}): Promise<TelemetryInstallKind> {+ const key = `lastLaunchedVersion.${channel}`;+ const lastVersion = (await store.get(key)) ?? null;+ current = installKindFor({ lastVersion, version, databaseExisted });+ await store.set(key, version);+ return current;+}++/**+ * This launch's kind. Boot records it before any window opens, so nothing a+ * person does can ask first; if something does, it is said in the log rather+ * than passed off as a real answer, and reads as `same`, the kind that claims+ * nothing.+ */+export function currentInstallKind(): TelemetryInstallKind {+ if (current === null) {+ log.warn('telemetry: install kind read before this launch was recorded');+ return 'same';+ }+ return current;+}diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/relay-client.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/relay-client.test.tsindex 784acf261..de1e8c2d3 100644--- a/console/apps/switch-console-desktop/src/main/core/telemetry/relay-client.test.ts+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/relay-client.test.ts@@ -1,4 +1,3 @@-import { durationMs } from '@tooling/utils/telemetry-duration';import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';import type { TelemetryConfig } from './config';import {@@ -14,12 +13,15 @@ const CONTEXT: TelemetryContext = {osType: 'darwin',osVersion: '24.3.0',build: 'stable',+ flintEnv: 'prod',+ internal: 'false',timeMs: 1_700_000_000_000,};const CONFIG: TelemetryConfig = {endpoint: 'https://telemetry.example/v1/logs',build: 'stable',+ flintEnv: 'prod',};type Attribute = { key: string; value: { stringValue: string } };@@ -63,18 +65,20 @@ const SESSION_STARTED = {describe('the record that gets built', () => {it('carries the client id the relay requires, as a resource attribute', () => {// Without it the relay drops the whole payload — and still answers 200.- const payload = buildOtlpPayload('app_launched', {}, CONTEXT);+ const payload = buildOtlpPayload('app_launched', { install_kind: 'same' }, CONTEXT);expect(resourceAttributes(payload)['flint.client_id']).toBe(CONTEXT.clientId);});it('says which app and version it is, for both destinations', () => {- const payload = buildOtlpPayload('app_launched', {}, CONTEXT);+ const payload = buildOtlpPayload('app_launched', { install_kind: 'same' }, CONTEXT);expect(resourceAttributes(payload)).toEqual({'service.name': 'switch-console','service.version': '1.2.3','flint.client_id': CONTEXT.clientId,+ flint_env: 'prod',+ flint_internal: 'false','os.type': 'darwin','os.version': '24.3.0',});@@ -90,7 +94,7 @@ describe('the record that gets built', () => {// The attribute alone gets a record past the relay's "is it named?" filter// and then dropped by an exporter that only reads the field — silently, and// with a 200 at every step. Both, or nothing arrives and nothing says so.- const payload = buildOtlpPayload('app_launched', {}, CONTEXT);+ const payload = buildOtlpPayload('app_launched', { install_kind: 'same' }, CONTEXT);expect(logRecord(payload).eventName).toBe('switch_console.app_launched');});@@ -128,19 +132,20 @@ describe('the record that gets built', () => {expect(JSON.stringify(payload)).not.toContain('secret-project');});- it('sends an event with no properties of its own as just its name and build', () => {- const payload = buildOtlpPayload('app_launched', {}, CONTEXT);+ it('sends an event as its name, its build and its own properties, and nothing else', () => {+ const payload = buildOtlpPayload('app_launched', { install_kind: 'new' }, CONTEXT);expect(logAttributes(payload)).toEqual({'event.name': 'switch_console.app_launched',build: 'stable',+ install_kind: 'new',});});it('carries a body, so the event is not a blank line in the log sink', () => {// The relay forwards the same record to Datadog, which reads the body as// the message. It repeats the name rather than adding anything new.- const payload = buildOtlpPayload('app_launched', {}, CONTEXT);+ const payload = buildOtlpPayload('app_launched', { install_kind: 'same' }, CONTEXT);expect(logRecord(payload).body).toEqual({ stringValue: 'switch_console.app_launched' });});@@ -164,7 +169,36 @@ describe('the record that gets built', () => {expect(rawLogAttributes(payload).connected_to_room).toEqual({ boolValue: false });});- it('carries a count as a number, so it can be summed at the far end', () => {+ it.each(['true', 'false', 'unknown'] as const)(+ 'says whether the person is staff (%s), and nothing about who',+ (internal) => {+ const payload = buildOtlpPayload(+ 'app_launched',+ { install_kind: 'same' },+ { ...CONTEXT, internal }+ );++ expect(resourceAttributes(payload).flint_internal).toBe(internal);+ expect(JSON.stringify(payload)).not.toContain('@');+ }+ );++ it.each(['prod', 'staging', 'dev', 'local'] as const)(+ 'says the event belongs in the %s Amplitude project',+ (flintEnv) => {+ // The relay picks the project from this, beside the client id it already+ // requires on the resource.+ const payload = buildOtlpPayload(+ 'app_launched',+ { install_kind: 'same' },+ { ...CONTEXT, flintEnv }+ );++ expect(resourceAttributes(payload).flint_env).toBe(flintEnv);+ }+ );++ it('carries a yes as a boolean, not as the word', () => {const payload = buildOtlpPayload('session_started',{ ...SESSION_STARTED, has_initial_prompt: true },@@ -172,43 +206,48 @@ describe('the record that gets built', () => {);expect(rawLogAttributes(payload).has_initial_prompt).toEqual({ boolValue: true });- expect(rawLogAttributes(payload).entry_point).toEqual({ stringValue: 'sidebar' });});- it('refuses a count that is not a real number rather than sending null', () => {- // NaN and Infinity both serialise to `null` in JSON, which at the far end- // is indistinguishable from a property that was never sent.- const broken = { ...SESSION_STARTED, has_initial_prompt: Number.NaN } as never;+ it('carries a count as a number, so it can be summed at the far end', () => {+ // A count sent as text arrives as a category: it can be grouped by, never+ // summed or averaged.+ const payload = buildOtlpPayload(+ 'search_performed',+ { status: 'ok', result_count: 3 },+ CONTEXT+ );- expect(() => buildOtlpPayload('session_started', broken, CONTEXT)).toThrow(/non-finite/);+ expect(rawLogAttributes(payload).result_count).toEqual({ doubleValue: 3 });});+ it('carries a count of zero as a number, not as a property that was never sent', () => {+ const payload = buildOtlpPayload(+ 'search_performed',+ { status: 'ok', result_count: 0 },+ CONTEXT+ );++ expect(rawLogAttributes(payload).result_count).toEqual({ doubleValue: 0 });+ });++ it.each([Number.NaN, Number.POSITIVE_INFINITY, Number.NEGATIVE_INFINITY])(+ 'refuses a count of %s rather than sending null',+ (count) => {+ // NaN and Infinity both serialise to `null` in JSON, which at the far end+ // is indistinguishable from a property that was never sent.+ const broken = { status: 'ok', result_count: count } as const;++ expect(() => buildOtlpPayload('search_performed', broken, CONTEXT)).toThrow(/non-finite/);+ }+ );+it('stamps the time in nanoseconds, which is what OTLP counts in', () => {- const payload = buildOtlpPayload('app_launched', {}, CONTEXT);+ const payload = buildOtlpPayload('app_launched', { install_kind: 'same' }, CONTEXT);const record = logRecord(payload);expect(record.timeUnixNano).toBe('1700000000000000000');expect(record.observedTimeUnixNano).toBe('1700000000000000000');});-- it('stays far under the attribute count the relay drops a record for', () => {- // The guard drops any record with more than 128 attributes.- const payload = buildOtlpPayload(- 'agent_cli_action',- {- agent_type: 'claude',- target: 'remote',- install_method: 'unspecified',- action: 'install',- outcome: 'success',- failure_reason: 'none',- duration_ms: durationMs(4200),- },- CONTEXT- );-- expect(Object.keys(logAttributes(payload)).length).toBeLessThan(16);- });});describe('sending it', () => {diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/relay-client.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/relay-client.tsindex 8dcaeced1..f7553ae6c 100644--- a/console/apps/switch-console-desktop/src/main/core/telemetry/relay-client.ts+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/relay-client.ts@@ -1,9 +1,10 @@-import type { TelemetryConfig } from './config';+import type { TelemetryConfig, TelemetryFlintEnv } from './config';import {TELEMETRY_EVENT_PROPERTIES,type TelemetryEventMap,type TelemetryEventName,} from './events';+import type { TelemetryInternal } from './internal-account';const SEND_TIMEOUT_MS = 10_000;@@ -69,6 +70,10 @@ export type TelemetryContext = {osType: string;osVersion: string;build: TelemetryConfig['build'];+ /** The Amplitude project the event belongs in, which the relay routes on. */+ flintEnv: TelemetryFlintEnv;+ /** Whether the person is staff, from their account's email domain. */+ internal: TelemetryInternal;timeMs: number;};@@ -163,6 +168,8 @@ export function buildOtlpPayload<K extends TelemetryEventName>('service.name': SERVICE_NAME,'service.version': context.appVersion,'flint.client_id': context.clientId,+ flint_env: context.flintEnv,+ flint_internal: context.internal,'os.type': context.osType,'os.version': context.osVersion,}),diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/release-stamp.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/release-stamp.test.tsnew file mode 100644index 000000000..837dfe110--- /dev/null+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/release-stamp.test.ts@@ -0,0 +1,37 @@+import { readFileSync } from 'node:fs';+import { dirname, resolve } from 'node:path';+import { fileURLToPath } from 'node:url';+import { describe, expect, it } from 'vitest';++/**+ * A packaged build reports its usage to production's Amplitude project only+ * when the release workflow stamped it `VITE_RELEASE=1`; anything else reports+ * to dev's. So a release whose build step lost the stamp would file every real+ * user under dev, with nothing anywhere reporting a fault. This holds the+ * workflow to stamping every build step, and only on a tag.+ */+const workflow = readFileSync(+ resolve(+ dirname(fileURLToPath(import.meta.url)),+ '../../../../../../../.github/workflows/switch-console-release.yml'+ ),+ 'utf8'+);++describe('the release workflow', () => {+ it('stamps every build step that sets the channel', () => {+ const buildSteps = workflow.match(/^\s+VITE_BUILD: .*$/gm) ?? [];+ const stamps = workflow.match(+ /^\s+VITE_RELEASE: \$\{\{ needs\.resolve\.outputs\.vite_release \}\}$/gm+ );++ expect(buildSteps.length).toBeGreaterThan(0);+ expect(stamps?.length).toBe(buildSteps.length);+ });++ it('marks a build a release on a tag and on nothing else', () => {+ expect(workflow).toMatch(/refs\/tags\/\*\) vite_release=1 ;;/);+ expect(workflow).toMatch(/\*\) vite_release= ;;/);+ expect(workflow).toMatch(/echo "vite_release=\$vite_release"/);+ });+});diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.test.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.test.tsindex 6c4a601b5..de3f18210 100644--- a/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.test.ts+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.test.ts@@ -80,7 +80,7 @@ describe('the consent gate', () => {it('makes no request at all when consent is off', async () => {vi.mocked(isTelemetryAllowed).mockResolvedValue(false);- await telemetryService.track('app_launched', {});+ await telemetryService.track('app_launched', { install_kind: 'same' });expect(fetchMock).not.toHaveBeenCalled();});@@ -88,8 +88,8 @@ describe('the consent gate', () => {it('asks the gate again for every event, so revoking consent stops the next one', async () => {vi.mocked(isTelemetryAllowed).mockResolvedValueOnce(true).mockResolvedValueOnce(false);- await telemetryService.track('app_launched', {});- await telemetryService.track('app_launched', {});+ await telemetryService.track('app_launched', { install_kind: 'same' });+ await telemetryService.track('app_launched', { install_kind: 'same' });expect(isTelemetryAllowed).toHaveBeenCalledTimes(2);expect(fetchMock).toHaveBeenCalledTimes(1);@@ -130,7 +130,7 @@ describe('a build that cannot send', () => {vi.mocked(isTelemetryAllowed).mockResolvedValue(true);const service = await freshService();- await service.track('app_launched', {});+ await service.track('app_launched', { install_kind: 'same' });expect(fetchMock).not.toHaveBeenCalled();expect(isTelemetryAllowed).not.toHaveBeenCalled();@@ -144,9 +144,9 @@ describe('a build that cannot send', () => {delete process.env.SWITCHDASH_TELEMETRY_DEV;const service = await freshService();- await service.track('app_launched', {});- await service.track('app_launched', {});- await service.track('app_launched', {});+ await service.track('app_launched', { install_kind: 'same' });+ await service.track('app_launched', { install_kind: 'same' });+ await service.track('app_launched', { install_kind: 'same' });expect(log.info).toHaveBeenCalledTimes(1);});@@ -217,7 +217,7 @@ describe('the payload', () => {const now = vi.spyOn(Date, 'now');now.mockReturnValueOnce(1_000).mockReturnValue(9_000);- await telemetryService.track('app_launched', {});+ await telemetryService.track('app_launched', { install_kind: 'same' });expect(sentRecord().timeUnixNano).toBe('1000000000');now.mockRestore();@@ -255,7 +255,7 @@ describe('the payload', () => {process.platform as string];- await telemetryService.track('app_launched', {});+ await telemetryService.track('app_launched', { install_kind: 'same' });expect(sentResource()['os.type']).toBe(expected ?? 'other');expect(sentResource()['os.version']).toBe(release());@@ -273,6 +273,8 @@ describe('the payload', () => {expect(Object.keys(sentResource()).sort()).toEqual(['flint.client_id',+ 'flint_env',+ 'flint_internal','os.type','os.version','service.name',@@ -296,7 +298,7 @@ describe('failure', () => {it('never reaches the caller, and is logged with a code', async () => {fetchMock.mockResolvedValue({ ok: false, status: 503 } as Response);- expect(() => trackEvent('app_launched', {})).not.toThrow();+ expect(() => trackEvent('app_launched', { install_kind: 'same' })).not.toThrow();await vi.waitFor(() =>expect(log.warn).toHaveBeenCalledWith('telemetry: event not sent',@@ -308,7 +310,7 @@ describe('failure', () => {it('reports a network failure as such rather than as a send', async () => {fetchMock.mockRejectedValue(new Error('getaddrinfo ENOTFOUND'));- trackEvent('app_launched', {});+ trackEvent('app_launched', { install_kind: 'same' });await vi.waitFor(() =>expect(log.warn).toHaveBeenCalledWith(@@ -321,7 +323,7 @@ describe('failure', () => {it('does not retry', async () => {fetchMock.mockResolvedValue({ ok: false, status: 500 } as Response);- await telemetryService.track('app_launched', {}).catch(() => {});+ await telemetryService.track('app_launched', { install_kind: 'same' }).catch(() => {});expect(fetchMock).toHaveBeenCalledTimes(1);});diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.tsindex 6ea614b54..bc2b25cb8 100644--- a/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.ts+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/telemetry-service.ts@@ -5,6 +5,7 @@ import { resolveTelemetryConfig, type TelemetryDisabledReason } from './config';import { isTelemetryAllowed } from './consent';import type { TelemetryEventMap, TelemetryEventName } from './events';import { getInstallId } from './install-id';+import type { TelemetryInternal } from './internal-account';import { buildOtlpPayload, postTelemetryEvent, TelemetrySendError } from './relay-client';/** OpenTelemetry's names for what `process.platform` calls darwin/win32/linux. */@@ -17,6 +18,15 @@ const OS_TYPES: Record<string, string> = {class TelemetryService {private reportedDisabled = new Set<TelemetryDisabledReason>();private appVersion: Promise<string> | null = null;+ /**+ * Whether the person is staff. Registered at boot rather than imported,+ * because it reads the servers store, which reports through this service.+ */+ private internalSource: () => Promise<TelemetryInternal> = async () => 'unknown';++ setInternalSource(source: () => Promise<TelemetryInternal>): void {+ this.internalSource = source;+ }/*** Send one event, if this build can send and the user has said yes.@@ -48,6 +58,8 @@ class TelemetryService {osType: OS_TYPES[process.platform] ?? 'other',osVersion: release(),build: resolution.config.build,+ flintEnv: resolution.config.flintEnv,+ internal: await this.resolveInternal(),timeMs,});@@ -68,6 +80,17 @@ class TelemetryService {return resolveTelemetryConfig().enabled && (await isTelemetryAllowed());}+ /** `unknown` when it cannot be read: that is the honest answer, and telemetry+ * never gets to fail a user's action. */+ private async resolveInternal(): Promise<TelemetryInternal> {+ try {+ return await this.internalSource();+ } catch (error) {+ log.warn('telemetry: could not tell whether the account is internal', { error });+ return 'unknown';+ }+ }+private resolveVersion(): Promise<string> {this.appVersion ??= resolveAppVersion();return this.appVersion;diff --git a/console/apps/switch-console-desktop/src/main/db/client.ts b/console/apps/switch-console-desktop/src/main/db/client.tsindex d11a58206..bb67af0fb 100644--- a/console/apps/switch-console-desktop/src/main/db/client.ts+++ b/console/apps/switch-console-desktop/src/main/db/client.ts@@ -1,3 +1,4 @@+import { existsSync } from 'node:fs';import Database from 'better-sqlite3';import { drizzle } from 'drizzle-orm/better-sqlite3';import { resolveDatabasePath } from './path';@@ -8,7 +9,16 @@ export type DrizzleTx = Parameters<AppDb['transaction']>[0] extends (tx: infer T? T: never;-export const sqlite = new Database(resolveDatabasePath());+const databasePath = resolveDatabasePath();++/**+ * Whether this installation already had a database before the app opened it:+ * a relaunch or an upgrade, rather than the first launch ever. Read before+ * opening, which creates the file.+ */+export const databaseExistedAtStart = existsSync(databasePath);++export const sqlite = new Database(databasePath);sqlite.pragma('journal_mode = WAL');sqlite.pragma('busy_timeout = 5000');diff --git a/console/apps/switch-console-desktop/src/main/index.ts b/console/apps/switch-console-desktop/src/main/index.tsindex 8c95c3dd2..9aafd68d7 100644--- a/console/apps/switch-console-desktop/src/main/index.ts+++ b/console/apps/switch-console-desktop/src/main/index.ts@@ -4,7 +4,7 @@ import { setTimeout as delay } from 'node:timers/promises';import { config as dotenvConfig } from 'dotenv';import { app, BrowserWindow, dialog, ipcMain, powerMonitor } from 'electron';import dockIcon from '@/assets/images/switch-console/icon-dock.png?asset';-import { PRODUCT_NAME } from '@shared/app-identity';+import { IS_CANARY, PRODUCT_NAME } from '@shared/app-identity';import { registerRPCRouter } from '@shared/lib/ipc/rpc';import { flushPendingDeeplink, setupDeeplinks } from './app/deeplinks';import { setupApplicationMenu } from './app/menu';@@ -15,6 +15,7 @@ import { setAgentStorageMigrationReady } from './core/agents/agent-storage-migraimport { migrateAgentStorage } from './core/agents/migrate-agent-storage';import { initializeRemoteDiscovery, initializeRemoteWatchers } from './core/agents/remote-watcher';import { appService } from './core/app/service';+import { resolveAppVersion } from './core/app/utils';import { controlService } from './core/control-api/control-service';import { localDependencyManager } from './core/dependencies/dependency-managers';import { embeddedControllerService } from './core/embedded-controller/embedded-controllers';@@ -33,8 +34,10 @@ import { searchService } from './core/search/search-service';import { appSettingsService } from './core/settings/settings-service';import { sshConnectionManager } from './core/ssh/lifecycle/production-ssh-connection-manager';import { autoSessionWatcher } from './core/switch-rooms/auto-session-watcher';+import { currentInternalFlag } from './core/telemetry/internal-account';+import { recordLaunch } from './core/telemetry/launch-history';import { registerTelemetryListeners } from './core/telemetry/telemetry-listeners';-import { trackEvent } from './core/telemetry/telemetry-service';+import { telemetryService, trackEvent } from './core/telemetry/telemetry-service';import { updateService } from './core/updates/update-service';import { viewStateService } from './core/view-state/view-state-service';import { reconcileAllWorkspaces } from './core/workspaces/reconcile-workspaces';@@ -144,7 +147,15 @@ void app.whenReady().then(async () => {// After the settings store, which owns the consent gate every event asks// before it is sent, and never before the database it is read from.registerTelemetryListeners();- trackEvent('app_launched', {});+ telemetryService.setInternalSource(currentInternalFlag);+ trackEvent('app_launched', {+ install_kind: await recordLaunch({+ version: await resolveAppVersion(),+ channel: IS_CANARY ? 'canary' : 'stable',+ // Imported here, not at the top: the client opens the database on import.+ databaseExisted: (await import('./db/client')).databaseExistedAtStart,+ }),+ });// Kept off the boot path: this can open an SSH/SFTP connection per remote// agent, so awaiting it here delayed the window opening. Session relaunch belowdiff --git a/console/apps/switch-console-desktop/src/shared/app-identity.ts b/console/apps/switch-console-desktop/src/shared/app-identity.tsindex a21619829..84cc7e710 100644--- a/console/apps/switch-console-desktop/src/shared/app-identity.ts+++ b/console/apps/switch-console-desktop/src/shared/app-identity.ts@@ -1,8 +1,14 @@-type ImportMetaWithEnv = ImportMeta & { env?: { DEV?: boolean; VITE_BUILD?: string } };+type ImportMetaWithEnv = ImportMeta & {+ env?: { DEV?: boolean; VITE_BUILD?: string; VITE_RELEASE?: string };+};const env = (import.meta as ImportMetaWithEnv).env;const isDev = env?.DEV === true;const isCanary = env?.VITE_BUILD === 'canary';+// Stamped by the release workflow on a tagged build, and only there: a+// workflow_dispatch test build, or one packaged on a laptop, is not a release+// however it is otherwise configured.+const isRelease = env?.VITE_RELEASE === '1';// What a user reads, what the app calls itself to the OS, and where data lives// are three different names here, and only the first one moved.@@ -38,6 +44,7 @@ export const USER_DATA_DIR_NAME = isDev ? 'switchdash-dev' : 'switchdash';export const UPDATE_CHANNEL = isCanary ? 'v1-canary' : 'v1-stable';export const ARTIFACT_PREFIX = isCanary ? 'switch-console-canary' : 'switch-console';export const IS_CANARY = isCanary;+export const IS_RELEASE_BUILD = isRelease;// GitHub repo the desktop app publishes releases to / reads auto-updates from.// The repo is public, so the feed is read unauthenticated. Mirrored indiff --git a/core/switch_core/bridges/collaboration/lifecycle_service.py b/core/switch_core/bridges/collaboration/lifecycle_service.pyindex e8c4e5256..606ad3b04 100644--- a/core/switch_core/bridges/collaboration/lifecycle_service.py+++ b/core/switch_core/bridges/collaboration/lifecycle_service.py@@ -1272,17 +1272,21 @@ class CollaborationBridgeLifecycleService:},)- # `enabled`, not just `is not None`: off is a real service with a- # discarding sink, so testing for None alone spends the claim below on- # a deployment reporting nothing.- if self._telemetry is None or not self._telemetry.enabled:- return# The first successful connect for this bridge, ever. Claimed against# the bridge id so restarting a working bridge does not re-report a- # setup that happened months ago.- if not await claim_milestone(+ # setup that happened months ago — and claimed whether or not telemetry+ # is on, unlike the milestones. `connector_added` measures setup time,+ # configuration saved to first connect; a first connect nobody was told+ # about is setup that went unmeasured, and reporting it the day+ # telemetry is switched on would report the connector's whole age.+ first_connect = await claim_milestone(self._session_factory, f"connector_added:{bridge_id}"- ):+ )+ # `enabled`, not just `is not None`: off is a real service with a+ # discarding sink.+ if self._telemetry is None or not self._telemetry.enabled:+ return+ if not first_connect:# This bridge has reported, but the deployment-wide milestone may# not have. `emit_milestone` is itself once-ever.if not preconfigured:diff --git a/core/switch_core/config.py b/core/switch_core/config.pyindex 6ea03c4fe..3bd15fe50 100644--- a/core/switch_core/config.py+++ b/core/switch_core/config.py@@ -339,6 +339,22 @@ class SwitchConfig(BaseSettings):# to how long this process has been up.telemetry_snapshot_interval_hours: float = 24.0+ # Which Amplitude project this deployment's usage lands in, sent as+ # `flint_env`. The relay keeps one project per environment and files an+ # event naming none, or `prod`, under production — which is what every+ # customer's deployment is, and why it is the default. Our own non-customer+ # servers say so here: `dev` for the development deployment, `local` for a+ # developer's machine (`just init-env` writes it), so their usage never+ # reads as adoption. Any other value is refused at startup, because the+ # relay drops an event naming an environment it has no project for.+ telemetry_environment: Literal["prod", "staging", "dev", "local"] = "prod"++ # Whether this deployment is one of the company's own, sent as+ # `flint_internal` so staff usage can be told from adoption. A customer's+ # deployment never sets it. Within a deployment, staff accounts are counted+ # separately by email domain whether or not this is set.+ telemetry_internal: bool = False+server_host: str = "0.0.0.0"server_port: int = 8000diff --git a/core/switch_core/migrations/versions/2575637e78d4_claim_connector_added_for_existing_connectors.py b/core/switch_core/migrations/versions/2575637e78d4_claim_connector_added_for_existing_connectors.pynew file mode 100644index 000000000..13de526a2--- /dev/null+++ b/core/switch_core/migrations/versions/2575637e78d4_claim_connector_added_for_existing_connectors.py@@ -0,0 +1,45 @@+"""claim connector_added for every connector that already exists++Revision ID: 2575637e78d4+Revises: e4a7c1d93b25++`connector_added` reports a connector's setup time — configuration saved to+first connect — once, on that first connect, guarded by a claim in+`telemetry_milestones`. The claim used to be taken only while telemetry was+on, so a connector that existed before telemetry started reporting claimed it+on its first connect afterwards and reported its whole age as setup time:+millions of seconds.++The claim is now taken whether telemetry is on or not. A connector that exists+at this migration has, as far as anything can tell, already made its first+connect unobserved, so its claim is taken here and it never reports+`connector_added`. One that never managed to connect is the exception it+costs: if it connects for the first time later, that goes unreported too.+"""++from collections.abc import Sequence++from alembic import op++revision: str = "2575637e78d4"+down_revision: str | Sequence[str] | None = "e4a7c1d93b25"+branch_labels: str | Sequence[str] | None = None+depends_on: str | Sequence[str] | None = None+++def upgrade() -> None:+ # Runs as the schema owner, which row-level security does not restrict, so+ # every tenant's connectors are seen.+ op.execute(+ """+ INSERT INTO telemetry_milestones (name)+ SELECT 'connector_added:' || id FROM collaboration_bridges+ ON CONFLICT (name) DO NOTHING+ """+ )+++def downgrade() -> None:+ # The claims taken here cannot be told apart from ones a real report took,+ # and keeping them only keeps those connectors quiet.+ passdiff --git a/core/switch_core/telemetry/catalogue.py b/core/switch_core/telemetry/catalogue.pyindex 22ffcabd3..8b2f4a544 100644--- a/core/switch_core/telemetry/catalogue.py+++ b/core/switch_core/telemetry/catalogue.py@@ -150,8 +150,13 @@ _SNAPSHOT_COUNTS = ("tenant_count","tenant_failed_count","user_count",+ "user_internal_count",+ # Switch accounts, not chat identities — see `chat_identity_*` for those."user_active_1d","user_active_7d",+ "chat_identity_count",+ "chat_identity_active_1d",+ "chat_identity_active_7d",# `room_count` is the headline: rooms a *person* made. The other two keep# agent scratch rooms and adopted channels out of it."room_count",diff --git a/core/switch_core/telemetry/internal.py b/core/switch_core/telemetry/internal.pynew file mode 100644index 000000000..30a20c863--- /dev/null+++ b/core/switch_core/telemetry/internal.py@@ -0,0 +1,33 @@+"""Whose usage is the company's own.++Staff use the product too, and their usage is real but is not adoption, so the+numbers have to be able to leave it out. Two signals, neither an identifier:+a deployment says it is one of the company's own (`TELEMETRY_INTERNAL`), and+the daily snapshot counts the accounts on the company's email domains. The+same domains mark a staff member in Switch Console.+"""++from __future__ import annotations++from sqlalchemy import ColumnElement, func, or_+from sqlalchemy.orm import InstrumentedAttribute++INTERNAL_EMAIL_DOMAINS: tuple[str, ...] = ("sandboxaq.com", "sandboxquantum.com")+++def internal_email_condition(+ email: ColumnElement[str] | InstrumentedAttribute[str],+) -> ColumnElement[bool]:+ """Whether `email` is on one of `INTERNAL_EMAIL_DOMAINS` or a subdomain of+ one, as SQL, so accounts are counted without loading their addresses."""+ lowered = func.lower(email)+ return or_(+ *(+ pattern+ for internal in INTERNAL_EMAIL_DOMAINS+ for pattern in (+ lowered.like(f"%@{internal}"),+ lowered.like(f"%@%.{internal}"),+ )+ )+ )diff --git a/core/switch_core/telemetry/service.py b/core/switch_core/telemetry/service.pyindex 1391177b9..2fcc125fc 100644--- a/core/switch_core/telemetry/service.py+++ b/core/switch_core/telemetry/service.py@@ -43,6 +43,8 @@ class TelemetryService:service_name: str,version: str | None,environment: str | None,+ telemetry_environment: str,+ telemetry_internal: bool,session_factory: async_sessionmaker[AsyncSession] | None = None,installed_at: datetime | None = None,) -> None:@@ -55,6 +57,13 @@ class TelemetryService:self._resource = {"service.name": service_name,"flint.client_id": client_id,+ # The relay picks the Amplitude project from this. Sent even as+ # `prod`, so an event says where it belongs rather than relying on+ # what the relay does with silence.+ "flint_env": telemetry_environment,+ # A string, as Switch Console sends it, where `unknown` is a third+ # answer; a deployment always knows which it is.+ "flint_internal": "true" if telemetry_internal else "false",}# Omitted rather than sent empty: an absent attribute reads as "not# configured", where `""` reads as a real environment named nothing.diff --git a/core/switch_core/telemetry/setup.py b/core/switch_core/telemetry/setup.pyindex 0ca9b6bff..3b065c8f6 100644--- a/core/switch_core/telemetry/setup.py+++ b/core/switch_core/telemetry/setup.py@@ -134,6 +134,8 @@ def _service(service_name=config.service_name,version=version,environment=config.environment,+ telemetry_environment=config.telemetry_environment,+ telemetry_internal=config.telemetry_internal,session_factory=session_factory,installed_at=installed_at,)diff --git a/core/switch_core/telemetry/sink.py b/core/switch_core/telemetry/sink.pyindex 562cc4d60..85daaf10e 100644--- a/core/switch_core/telemetry/sink.py+++ b/core/switch_core/telemetry/sink.py@@ -10,6 +10,7 @@ lets a reporting problem reach the caller.from __future__ import annotationsimport logging+from collections.abc import Mappingfrom dataclasses import dataclassfrom typing import Any, Protocol@@ -19,6 +20,7 @@ from switch_core.observability.otlp import (OtlpResource,OtlpSendError,build_logs_payload,+ otlp_attributes,)from switch_core.telemetry.catalogue import PropertyValue@@ -98,6 +100,7 @@ class OtlpRelaySink:_resource_from(record),)_add_event_name(payload, record.name)+ _add_relay_context(payload, record.resource)try:await self._client.post("logs", payload)@@ -130,6 +133,17 @@ def _resource_from(record: TelemetryRecord) -> OtlpResource:)+def _add_relay_context(payload: dict[str, Any], resource: Mapping[str, str]) -> None:+ """Add `flint_env` and `flint_internal` to the resource, which the shared+ encoder's resource has no fields for: it describes the process to an+ operator's own collector, where they mean nothing. The relay sends each+ event to the Amplitude project `flint_env` names, and drops one it cannot+ place; `flint_internal` tells staff usage from adoption."""+ context = {key: resource[key] for key in ("flint_env", "flint_internal")}+ for resource_log in payload["resourceLogs"]:+ resource_log["resource"]["attributes"].extend(otlp_attributes(context))++def _add_event_name(payload: dict[str, Any], name: str) -> None:"""Set the record's own `eventName` field, which `build_logs_payload` doesnot: an operational log line has no event name."""diff --git a/core/switch_core/telemetry/snapshot.py b/core/switch_core/telemetry/snapshot.pyindex bbd9f9be2..7b57f7b77 100644--- a/core/switch_core/telemetry/snapshot.py+++ b/core/switch_core/telemetry/snapshot.py@@ -44,6 +44,8 @@ from switch_core.db.models import (ClientRoom,CollaborationBridge,Document,+ ExternalUser,+ ExternalUserClaim,Message,MessageAttachment,Package,@@ -56,6 +58,7 @@ from switch_core.db.models import ()from switch_core.db.session_scope import tenant_sessionfrom switch_core.db.tenant_lookup import all_tenant_ids+from switch_core.telemetry.internal import internal_email_conditionlogger = logging.getLogger(__name__)@@ -104,8 +107,26 @@ class UsageCounts:# HTTP request histogram that times everything else.duration_ms: int = 0user_count: int = 0+ # Accounts on the company's own email domains, so staff can be told from+ # customers inside one deployment. Counted once for the deployment, like+ # `user_count`.+ user_internal_count: int = 0+ # Switch accounts that used a room with an agent: an account counts when a+ # chat account it has claimed spoke. Counted from the ids below once every+ # tenant is in, since one account can be active in several tenants and a+ # per-tenant count would add it once for each.user_active_1d: int = 0user_active_7d: int = 0+ active_user_ids_1d: set[str] = field(default_factory=set)+ active_user_ids_7d: set[str] = field(default_factory=set)+ # Chat identities — a Slack, Mattermost or other platform account — that+ # used a room with an agent, whether or not anyone has claimed them. One+ # person on two platforms is two identities, and most identities belong to+ # no Switch account, which is why this runs above `user_active_*`.+ # `chat_identity_count` is the whole population they are drawn from.+ chat_identity_count: int = 0+ chat_identity_active_1d: int = 0+ chat_identity_active_7d: int = 0room_count: int = 0room_agent_created_count: int = 0room_system_created_count: int = 0@@ -147,8 +168,12 @@ class UsageCounts:"tenant_failed_count": self.tenant_failed_count,"duration_ms": self.duration_ms,"user_count": self.user_count,+ "user_internal_count": self.user_internal_count,"user_active_1d": self.user_active_1d,"user_active_7d": self.user_active_7d,+ "chat_identity_count": self.chat_identity_count,+ "chat_identity_active_1d": self.chat_identity_active_1d,+ "chat_identity_active_7d": self.chat_identity_active_7d,"room_count": self.room_count,"room_agent_created_count": self.room_agent_created_count,"room_system_created_count": self.room_system_created_count,@@ -313,6 +338,35 @@ def _human_interaction(tenant_id: str, since: datetime) -> Select[tuple[str]]:)+def _active_accounts(tenant_id: str, since: datetime) -> Select[tuple[str]]:+ """Distinct Switch accounts that used a room since `since`: an account+ is active when a chat account it has claimed spoke. A chat account several+ people claim makes each of them active. See `_human_activity_conditions`."""+ return (+ select(distinct(ExternalUserClaim.user_id))+ .select_from(Message)+ .join(Client, Client.id == Message.sender_client_id)+ .join(+ ExternalUser,+ and_(+ ExternalUser.client_id == Client.id, ExternalUser.tenant_id == tenant_id+ ),+ )+ .join(+ ExternalUserClaim,+ and_(+ ExternalUserClaim.external_user_id == ExternalUser.id,+ ExternalUserClaim.tenant_id == tenant_id,+ ),+ )+ .where(+ Message.tenant_id == tenant_id,+ Message.sent_at >= since,+ *_human_activity_conditions(tenant_id),+ )+ )++def _active_humans(tenant_id: str, since: datetime) -> Select[tuple[str | None]]:"""Distinct human clients who used a room since `since`. See`_human_activity_conditions`."""@@ -353,8 +407,24 @@ async def collect_tenant_counts(day_ago = now - _DAYweek_ago = now - _WEEK- counts.user_active_1d += await _count(session, _active_humans(tenant_id, day_ago))- counts.user_active_7d += await _count(session, _active_humans(tenant_id, week_ago))+ counts.active_user_ids_1d |= set(+ (await session.execute(_active_accounts(tenant_id, day_ago))).scalars()+ )+ counts.active_user_ids_7d |= set(+ (await session.execute(_active_accounts(tenant_id, week_ago))).scalars()+ )+ counts.chat_identity_count += await _scalar(+ session,+ select(func.count())+ .select_from(Client)+ .where(Client.tenant_id == tenant_id, Client.type == HUMAN_CLIENT_TYPE),+ )+ counts.chat_identity_active_1d += await _count(+ session, _active_humans(tenant_id, day_ago)+ )+ counts.chat_identity_active_7d += await _count(+ session, _active_humans(tenant_id, week_ago)+ )counts.room_active_1d += await _count(session, _human_interaction(tenant_id, day_ago))@@ -683,6 +753,11 @@ def _merge_tenant_counts(total: UsageCounts, tenant: UsageCounts) -> None:"tenant_failed_count","duration_ms","user_count",+ "user_internal_count",+ "user_active_1d",+ "user_active_7d",+ "active_user_ids_1d",+ "active_user_ids_7d","room_users_max","connector_counts",}@@ -691,6 +766,8 @@ def _merge_tenant_counts(total: UsageCounts, tenant: UsageCounts) -> None:continuesetattr(total, f.name, getattr(total, f.name) + getattr(tenant, f.name))total.room_users_max = max(total.room_users_max, tenant.room_users_max)+ total.active_user_ids_1d |= tenant.active_user_ids_1d+ total.active_user_ids_7d |= tenant.active_user_ids_7dfor platform, count in tenant.connector_counts.items():total.connector_counts[platform] += count@@ -747,6 +824,14 @@ async def collect_usage(counts.user_count = await _scalar(session, select(func.count()).select_from(User))+ counts.user_internal_count = await _scalar(+ session,+ select(func.count())+ .select_from(User)+ .where(internal_email_condition(User.email)),+ )+ counts.user_active_1d = len(counts.active_user_ids_1d)+ counts.user_active_7d = len(counts.active_user_ids_7d)counts.duration_ms = round((time.monotonic() - started) * 1000)return countsdiff --git a/core/tests/switch_core/bridges/agent/api/test_events_endpoint_dispatch.py b/core/tests/switch_core/bridges/agent/api/test_events_endpoint_dispatch.pyindex dfedf3c7b..23e7f5937 100644--- a/core/tests/switch_core/bridges/agent/api/test_events_endpoint_dispatch.py+++ b/core/tests/switch_core/bridges/agent/api/test_events_endpoint_dispatch.py@@ -465,6 +465,8 @@ class TestOpeningAStreamReportsASession:service_name="switch-core",version="1.0.0",environment=None,+ telemetry_environment="prod",+ telemetry_internal=False,)protocol = _Protocol()protocol.sessions = SessionReporter(service, protocol.connections)diff --git a/core/tests/switch_core/session_activity/test_session_started_route.py b/core/tests/switch_core/session_activity/test_session_started_route.pyindex c0134df7b..ff55dfbd1 100644--- a/core/tests/switch_core/session_activity/test_session_started_route.py+++ b/core/tests/switch_core/session_activity/test_session_started_route.py@@ -55,6 +55,8 @@ def _telemetry(sink: _RecordingSink, *, enabled: bool) -> TelemetryService:service_name="switch-core",version=None,environment=None,+ telemetry_environment="prod",+ telemetry_internal=False,)diff --git a/core/tests/switch_core/telemetry/test_bridge_connect_reporting.py b/core/tests/switch_core/telemetry/test_bridge_connect_reporting.pyindex 5edb4df3e..9003eeef1 100644--- a/core/tests/switch_core/telemetry/test_bridge_connect_reporting.py+++ b/core/tests/switch_core/telemetry/test_bridge_connect_reporting.py@@ -79,6 +79,8 @@ def _service_with_telemetry(service_name="switch-core",version=None,environment=None,+ telemetry_environment="prod",+ telemetry_internal=False,session_factory=session_factory,installed_at=installed_at,)diff --git a/core/tests/switch_core/telemetry/test_catalogue.py b/core/tests/switch_core/telemetry/test_catalogue.pyindex f55a65f66..15a1354de 100644--- a/core/tests/switch_core/telemetry/test_catalogue.py+++ b/core/tests/switch_core/telemetry/test_catalogue.py@@ -143,10 +143,13 @@ class TestValuesAreClosed:with pytest.raises(TelemetryCatalogueError, match="expected a number"):validate("usage_snapshot", payload) # type: ignore[arg-type]- def test_a_non_finite_count_is_refused(self) -> None:- """A mean over zero rooms must not arrive as NaN."""+ @pytest.mark.parametrize("value", [float("nan"), float("inf"), float("-inf")])+ def test_a_non_finite_count_is_refused(self, value: float) -> None:+ """A mean over zero rooms must not arrive as NaN. JSON has no NaN or+ infinity, so none of them can reach Amplitude as a number whatever the+ relay does with it."""payload = _valid_payload("usage_snapshot")- payload["room_users_mean"] = float("nan")+ payload["room_users_mean"] = valuewith pytest.raises(TelemetryCatalogueError, match="finite"):validate("usage_snapshot", payload) # type: ignore[arg-type]diff --git a/core/tests/switch_core/telemetry/test_connector_setup_time.py b/core/tests/switch_core/telemetry/test_connector_setup_time.pynew file mode 100644index 000000000..f0206ad5f--- /dev/null+++ b/core/tests/switch_core/telemetry/test_connector_setup_time.py@@ -0,0 +1,74 @@+"""`connector_added` reports setup time only when setup was actually seen.++`seconds_since_configured` is configuration saved to first connect. A first+connect made while telemetry was off went unmeasured, and reporting it on the+next connect after telemetry is switched on reports the connector's whole age+— millions of seconds — as setup time. So the first connect is recorded+whether telemetry is on or not, and only a connector whose first connect+happened while reporting reports it.+"""++from __future__ import annotations++import uuid+from datetime import UTC, datetime, timedelta++from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker++from switch_core.telemetry.service import TelemetryService+from tests.switch_core.telemetry.test_bridge_connect_reporting import (+ _service_with_telemetry,+)+from tests.switch_core.telemetry.test_preconfigured_connectors import (+ INSTALLED,+ _events,+ _running,+)++CONFIGURED_LONG_AGO = datetime.now(UTC) - timedelta(days=60)+++def _switched_off(telemetry: TelemetryService) -> TelemetryService:+ return TelemetryService(+ sink=telemetry._sink,+ enabled=False,+ client_id="deployment-uuid",+ service_name="switch-core",+ version=None,+ environment=None,+ telemetry_environment="prod",+ telemetry_internal=False,+ session_factory=telemetry._session_factory,+ installed_at=INSTALLED,+ )+++async def test_a_first_connect_while_telemetry_was_off_is_never_reported(+ session_factory: async_sessionmaker[AsyncSession],+) -> None:+ service, sink = _service_with_telemetry(session_factory, installed_at=INSTALLED)+ reporting = service._telemetry+ assert reporting is not None+ bridge = f"slack-{uuid.uuid4().hex[:8]}"+ _running(service, bridge, preconfigured=False)++ service._telemetry = _switched_off(reporting)+ await service._report_connector_up(bridge, "slack", CONFIGURED_LONG_AGO)+ service._telemetry = reporting+ await service._report_connector_up(bridge, "slack", CONFIGURED_LONG_AGO)++ assert await _events(service, sink, "connector_added") == []+++async def test_a_first_connect_while_reporting_reports_its_setup_time(+ session_factory: async_sessionmaker[AsyncSession],+) -> None:+ service, sink = _service_with_telemetry(session_factory, installed_at=INSTALLED)+ bridge = f"slack-{uuid.uuid4().hex[:8]}"+ _running(service, bridge, preconfigured=False)+ configured = datetime.now(UTC) - timedelta(seconds=90)++ await service._report_connector_up(bridge, "slack", configured)++ [added] = await _events(service, sink, "connector_added")+ assert 90 <= added["seconds_since_configured"] < 600diff --git a/core/tests/switch_core/telemetry/test_deployment_and_reporter.py b/core/tests/switch_core/telemetry/test_deployment_and_reporter.pyindex 9de91b70a..e85d01689 100644--- a/core/tests/switch_core/telemetry/test_deployment_and_reporter.py+++ b/core/tests/switch_core/telemetry/test_deployment_and_reporter.py@@ -174,6 +174,8 @@ class TestEmitMilestone:service_name="switch-core",version="1.0.0",environment=None,+ telemetry_environment="prod",+ telemetry_internal=False,session_factory=session_factory,installed_at=installed_at,)@@ -235,6 +237,8 @@ class TestTheSnapshotSchedule:service_name="switch-core",version="1.0.0",environment=None,+ telemetry_environment="prod",+ telemetry_internal=False,session_factory=session_factory,installed_at=datetime.now(UTC) - timedelta(days=1),)diff --git a/core/tests/switch_core/telemetry/test_emitters_are_live.py b/core/tests/switch_core/telemetry/test_emitters_are_live.pyindex 47e5f1312..46f00b0f3 100644--- a/core/tests/switch_core/telemetry/test_emitters_are_live.py+++ b/core/tests/switch_core/telemetry/test_emitters_are_live.py@@ -39,6 +39,8 @@ def _telemetry() -> TelemetryService:service_name="switch-core",version="1.0.0",environment=None,+ telemetry_environment="prod",+ telemetry_internal=False,)diff --git a/core/tests/switch_core/telemetry/test_environment_reaches_the_relay.py b/core/tests/switch_core/telemetry/test_environment_reaches_the_relay.pynew file mode 100644index 000000000..c1761266a--- /dev/null+++ b/core/tests/switch_core/telemetry/test_environment_reaches_the_relay.py@@ -0,0 +1,82 @@+"""A deployment's TELEMETRY_ENVIRONMENT reaches the relay as `flint_env`.++The relay files each event under the Amplitude project that attribute names. A+server that read the setting and then sent something else would land its+usage in the wrong project with nothing anywhere reporting a fault, so this+follows the value from configuration to the request body.+"""++from __future__ import annotations++import json+import uuid+from typing import Any++import httpx+import pytest+from sqlalchemy import delete+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker++from switch_core.config import SwitchConfig+from switch_core.db.models import DeploymentIdentity+from switch_core.telemetry import setup as telemetry_setup+++def _config(environment: str) -> SwitchConfig:+ return SwitchConfig( # type: ignore[call-arg]+ db_host="h",+ db_port="5432",+ db_user="u",+ db_password="p",+ db_name="d",+ matrix_server_name="test",+ agent_registration_token="t",+ jwt_secret_key="k",+ gateway_admin_email="[email protected]",+ gateway_admin_password="pw",+ secret_keys="test:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",+ telemetry_enabled=True,+ telemetry_endpoint="https://relay.example",+ telemetry_environment=environment,+ telemetry_internal=False,+ )++[email protected]("environment", ["prod", "staging", "dev", "local"])+async def test_the_configured_environment_is_sent_as_flint_env(+ environment: str,+ session_factory: async_sessionmaker[AsyncSession],+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ async with session_factory() as session:+ await session.execute(delete(DeploymentIdentity))+ session.add(DeploymentIdentity(id=1, client_id=str(uuid.uuid4())))+ await session.commit()+ bodies: list[dict[str, Any]] = []++ def _handle(request: httpx.Request) -> httpx.Response:+ bodies.append(json.loads(request.content))+ return httpx.Response(200)++ transport = httpx.MockTransport(_handle)+ real_client = httpx.AsyncClient+ monkeypatch.setattr(+ telemetry_setup.httpx,+ "AsyncClient",+ lambda **kwargs: real_client(transport=transport, **kwargs),+ )++ service, _, http = await telemetry_setup.build_telemetry(+ _config(environment), session_factory, "1.0.0"+ )+ service.emit("deployment_started", tenant_count=1)+ await service.aclose()+ assert http is not None+ await http.aclose()++ [body] = bodies+ resource = {+ a["key"]: a["value"] for a in body["resourceLogs"][0]["resource"]["attributes"]+ }+ assert resource["flint_env"] == {"stringValue": environment}+ assert resource["flint_internal"] == {"stringValue": "false"}diff --git a/core/tests/switch_core/telemetry/test_first_room_active.py b/core/tests/switch_core/telemetry/test_first_room_active.pyindex 44bf95810..5fa1af07c 100644--- a/core/tests/switch_core/telemetry/test_first_room_active.py+++ b/core/tests/switch_core/telemetry/test_first_room_active.py@@ -67,6 +67,8 @@ def _reporter(service_name="switch-core",version="1.0.0",environment=None,+ telemetry_environment="prod",+ telemetry_internal=False,session_factory=session_factory,installed_at=INSTALLED,)diff --git a/core/tests/switch_core/telemetry/test_merge_fixes.py b/core/tests/switch_core/telemetry/test_merge_fixes.pyindex 32ed86c6b..40295e17e 100644--- a/core/tests/switch_core/telemetry/test_merge_fixes.py+++ b/core/tests/switch_core/telemetry/test_merge_fixes.py@@ -141,6 +141,8 @@ class TestASessionIsTheAgentNotTheConnection:service_name="switch-core",version="1.0.0",environment=None,+ telemetry_environment="prod",+ telemetry_internal=False,)reporter = SessionReporter(service, registry)registry.set_close_listener(reporter.on_close)diff --git a/core/tests/switch_core/telemetry/test_preconfigured_connectors.py b/core/tests/switch_core/telemetry/test_preconfigured_connectors.pyindex 0f64ff0ed..201591263 100644--- a/core/tests/switch_core/telemetry/test_preconfigured_connectors.py+++ b/core/tests/switch_core/telemetry/test_preconfigured_connectors.py@@ -269,6 +269,8 @@ class TestRegisteringRecordsTheFlag:service_name="switch-core",version=None,environment=None,+ telemetry_environment="prod",+ telemetry_internal=False,)service._session_factory = session_factoryservice._client_lifecycle.create_client = AsyncMock( # type: ignore[method-assign]diff --git a/core/tests/switch_core/telemetry/test_relay_attribute_cap.py b/core/tests/switch_core/telemetry/test_relay_attribute_cap.pynew file mode 100644index 000000000..6d9561f15--- /dev/null+++ b/core/tests/switch_core/telemetry/test_relay_attribute_cap.py@@ -0,0 +1,119 @@+"""Every event fits through the relay's limits, with room to spare.++The relay drops any log record carrying more than 128 attributes, and any+event whose JSON is over 32 KiB never reaches Amplitude. It answers 200 either+way. An event that grew past either would vanish from the dashboard with+nothing anywhere reporting a fault. They are the silent drops at the relay that+Switch itself controls, so the margins are held here, beside the catalogue.+"""++from __future__ import annotations++import json+from collections.abc import Callable+from typing import Any++import httpx+import pytest++from switch_core.observability.otlp import OtlpClient+from switch_core.telemetry.catalogue import BOOLEAN, CATALOGUE, NUMBER, PropertyType+from switch_core.telemetry.service import TelemetryService+from switch_core.telemetry.sink import OtlpRelaySink, TelemetryRecord++# `filter/guard` in the relay's collector config: `Len(attributes) > 128`.+RELAY_ATTRIBUTE_CAP = 128+# Headroom, so an event growing toward the cap is caught in review rather than+# after its records have started disappearing.+MAX_ATTRIBUTES = 100+# `max_event_bytes` on the relay's Amplitude exporter: a larger event is dropped.+RELAY_EVENT_BYTES = 32768+# Half of it. Measured on the OTLP attributes, which wrap each value in more+# JSON than the Amplitude event does, so this over-counts.+MAX_EVENT_BYTES = RELAY_EVENT_BYTES // 2+++class _Capture:+ def __init__(self) -> None:+ self.sent: list[TelemetryRecord] = []++ async def send(self, record: TelemetryRecord) -> None:+ self.sent.append(record)++ async def aclose(self) -> None:+ return None+++def _example(kind: PropertyType) -> Any:+ """A value the catalogue accepts for a property of this kind."""+ if kind is NUMBER:+ return 1+ if kind is BOOLEAN:+ return True+ return sorted(kind.values)[0] # type: ignore[attr-defined]+++def _largest(kind: PropertyType) -> Any:+ """The longest value the catalogue accepts for a property of this kind."""+ if kind is NUMBER:+ return -123456789012345.67+ if kind is BOOLEAN:+ return False+ return max(kind.values, key=len) # type: ignore[attr-defined]+++async def _attributes_on_the_wire(+ event: str, value: Callable[[PropertyType], Any] = _example+) -> list[dict[str, Any]]:+ """The log record's attributes exactly as the relay would receive them:+ validated by the service, encoded by the sink."""+ capture = _Capture()+ service = TelemetryService(+ sink=capture, # type: ignore[arg-type]+ enabled=True,+ client_id="11111111-1111-1111-1111-111111111111",+ service_name="switch-core",+ version="1.0.0",+ environment=None,+ telemetry_environment="prod",+ telemetry_internal=False,+ )+ service.emit(event, **{key: value(kind) for key, kind in CATALOGUE[event].items()})+ await service.aclose()++ body: dict[str, Any] = {}++ def _handle(request: httpx.Request) -> httpx.Response:+ body.update(json.loads(request.content))+ return httpx.Response(200)++ http = httpx.AsyncClient(transport=httpx.MockTransport(_handle))+ [record] = capture.sent+ await OtlpRelaySink(client=OtlpClient("https://relay.example", 5, {}, http)).send(+ record+ )+ await http.aclose()+ return body["resourceLogs"][0]["scopeLogs"][0]["logRecords"][0]["attributes"]++[email protected]("event", sorted(CATALOGUE))+async def test_every_event_fits_through_the_relay(event: str) -> None:+ attributes = await _attributes_on_the_wire(event)++ assert len(attributes) <= MAX_ATTRIBUTES, (+ f"{event} carries {len(attributes)} attributes on the wire. The relay "+ f"drops any record over {RELAY_ATTRIBUTE_CAP} and still answers 200, "+ f"so this event is close to vanishing without a trace. Split it."+ )++[email protected]("event", sorted(CATALOGUE))+async def test_every_event_stays_well_under_the_relays_event_size(event: str) -> None:+ attributes = await _attributes_on_the_wire(event, _largest)+ size = len(json.dumps(attributes))++ assert size <= MAX_EVENT_BYTES, (+ f"{event} is {size} bytes of attributes at its largest. The relay drops "+ f"any Amplitude event over {RELAY_EVENT_BYTES} bytes and still answers "+ f"200, so this event is close to vanishing from Amplitude. Split it."+ )diff --git a/core/tests/switch_core/telemetry/test_service_and_sink.py b/core/tests/switch_core/telemetry/test_service_and_sink.pyindex 0ab2e90b5..172845554 100644--- a/core/tests/switch_core/telemetry/test_service_and_sink.py+++ b/core/tests/switch_core/telemetry/test_service_and_sink.py@@ -34,6 +34,8 @@ def _service(sink: object, *, enabled: bool = True) -> TelemetryService:service_name="switch-core",version="1.2.3",environment="pilot",+ telemetry_environment="prod",+ telemetry_internal=False,)@@ -78,10 +80,57 @@ class TestTagging:assert sink.sent[0].resource == {"service.name": "switch-core","flint.client_id": "deployment-uuid",+ "flint_env": "prod",+ "flint_internal": "false","service.version": "1.2.3","deployment.environment": "pilot",}+ @pytest.mark.parametrize("environment", ["prod", "staging", "dev", "local"])+ async def test_every_event_says_which_amplitude_project_it_belongs_in(+ self, environment: str+ ) -> None:+ """The relay sends each event to the Amplitude project its `flint_env`+ names, so a development server's usage never lands in production's."""+ sink = _RecordingSink()+ service = TelemetryService(+ sink=sink, # type: ignore[arg-type]+ enabled=True,+ client_id="deployment-uuid",+ service_name="switch-core",+ version="1.2.3",+ environment=None,+ telemetry_environment=environment,+ telemetry_internal=False,+ )++ service.emit("deployment_started", tenant_count=1)+ await service.aclose()++ assert sink.sent[0].resource["flint_env"] == environment++ @pytest.mark.parametrize(("internal", "sent"), [(True, "true"), (False, "false")])+ async def test_every_event_says_whether_the_deployment_is_the_companys_own(+ self, internal: bool, sent: str+ ) -> None:+ """So staff usage can be told from adoption without any identifier."""+ sink = _RecordingSink()+ service = TelemetryService(+ sink=sink, # type: ignore[arg-type]+ enabled=True,+ client_id="deployment-uuid",+ service_name="switch-core",+ version="1.2.3",+ environment=None,+ telemetry_environment="prod",+ telemetry_internal=internal,+ )++ service.emit("deployment_started", tenant_count=1)+ await service.aclose()++ assert sink.sent[0].resource["flint_internal"] == sent+async def test_an_unset_environment_is_omitted_rather_than_empty(self) -> None:"""An absent attribute reads as "not configured"; an empty string readsas a real environment named nothing."""@@ -93,6 +142,8 @@ class TestTagging:service_name="switch-core",version=None,environment=None,+ telemetry_environment="prod",+ telemetry_internal=False,)service.emit("deployment_started", tenant_count=1)@@ -154,6 +205,8 @@ class TestTheWireFormat:resource={"service.name": "switch-core","flint.client_id": "deployment-uuid",+ "flint_env": "dev",+ "flint_internal": "false",},timestamp_ns=1_700_000_000_000_000_000,)@@ -187,6 +240,39 @@ class TestTheWireFormat:assert attributes["from_template"] == {"boolValue": True}+ async def test_a_false_and_a_zero_are_sent_rather_than_dropped(self) -> None:+ """Falsy values are where an encoder that tests `if value:` loses them,+ and a missing property reads downstream exactly like one never sent."""+ body: dict = {}++ def _handle(request: httpx.Request) -> httpx.Response:+ body.update(json.loads(request.content))+ return httpx.Response(200)++ http = httpx.AsyncClient(transport=httpx.MockTransport(_handle))+ sink = OtlpRelaySink(client=OtlpClient("https://relay.example", 5, {}, http))+ await sink.send(+ TelemetryRecord(+ name="switch_core.connector_added",+ properties={+ "is_preconfigured": False,+ "failed_attempts_before_success": 0,+ },+ resource={+ "service.name": "switch-core",+ "flint.client_id": "deployment-uuid",+ "flint_env": "prod",+ "flint_internal": "false",+ },+ timestamp_ns=1_700_000_000_000_000_000,+ )+ )+ await http.aclose()+ attributes = {a["key"]: a["value"] for a in self._record(body)["attributes"]}++ assert attributes["is_preconfigured"] == {"boolValue": False}+ assert attributes["failed_attempts_before_success"] == {"doubleValue": 0.0}+async def test_the_body_carries_the_name_so_the_log_line_is_not_blank(self,) -> None:@@ -203,6 +289,9 @@ class TestTheWireFormat:assert resource["flint.client_id"] == {"stringValue": "deployment-uuid"}assert resource["service.name"] == {"stringValue": "switch-core"}+ # The relay sends each event to the Amplitude project this names.+ assert resource["flint_env"] == {"stringValue": "dev"}+ assert resource["flint_internal"] == {"stringValue": "false"}async def test_it_posts_to_the_logs_signal(self) -> None:"""Not `/v1/metrics`: the relay routes on log records, and a metric@@ -290,7 +379,12 @@ def _a_record() -> TelemetryRecord:return TelemetryRecord(name="switch_core.deployment_started",properties={"tenant_count": 1},- resource={"service.name": "switch-core", "flint.client_id": "deployment-uuid"},+ resource={+ "service.name": "switch-core",+ "flint.client_id": "deployment-uuid",+ "flint_env": "dev",+ "flint_internal": "false",+ },timestamp_ns=1,)diff --git a/core/tests/switch_core/telemetry/test_snapshot.py b/core/tests/switch_core/telemetry/test_snapshot.pyindex 5ec349c51..2391a3d04 100644--- a/core/tests/switch_core/telemetry/test_snapshot.py+++ b/core/tests/switch_core/telemetry/test_snapshot.py@@ -233,7 +233,7 @@ class TestWhatCountsAsActive:counts = await _counts(session)assert counts.room_active_1d == 1- assert counts.user_active_1d == 1+ assert counts.chat_identity_active_1d == 1async def test_two_agents_talking_is_not_activity(self, session_factory: async_sessionmaker[AsyncSession]@@ -252,7 +252,7 @@ class TestWhatCountsAsActive:counts = await _counts(session)assert counts.room_active_1d == 0- assert counts.user_active_1d == 0+ assert counts.chat_identity_active_1d == 0async def test_a_human_in_a_room_with_no_agent_is_not_activity(self, session_factory: async_sessionmaker[AsyncSession]@@ -283,7 +283,7 @@ class TestWhatCountsAsActive:counts = await _counts(session)- assert counts.user_active_1d == 0+ assert counts.chat_identity_active_1d == 0assert counts.room_active_1d == 0async def test_the_day_and_week_windows_differ(@@ -320,7 +320,7 @@ class TestWhatCountsAsActive:counts = await _counts(session)assert counts.room_active_1d == 0- assert counts.user_active_1d == 0+ assert counts.chat_identity_active_1d == 0class TestRoomsAreCountedByWhoMadeThem:@@ -516,8 +516,8 @@ class TestOnlyWhatSomeoneSaidCounts:ever_active = await room_had_human_activity(session, TENANT_ZERO, room.id)assert counts.message_from_human_1d == 0- assert counts.user_active_1d == 0- assert counts.user_active_7d == 0+ assert counts.chat_identity_active_1d == 0+ assert counts.chat_identity_active_7d == 0assert counts.room_active_1d == 0assert counts.room_active_7d == 0assert ever_active is False@@ -543,7 +543,7 @@ class TestOnlyWhatSomeoneSaidCounts:counts = await _counts(session)assert counts.message_from_human_1d == 1- assert counts.user_active_1d == 1+ assert counts.chat_identity_active_1d == 1assert counts.room_active_1d == 1async def test_the_notice_switch_posts_for_an_agent_is_not_the_agent_talking(@@ -929,7 +929,7 @@ class TestTheFourPathsAgreeOnActivity:)assert counts.room_active_1d == 0- assert counts.user_active_1d == 0+ assert counts.chat_identity_active_1d == 0assert was_ever_active is Falseassert newly_active == []@@ -956,7 +956,7 @@ class TestTheFourPathsAgreeOnActivity:)assert counts.room_active_1d == 1- assert counts.user_active_1d == 1+ assert counts.chat_identity_active_1d == 1assert was_ever_active is Trueassert len(newly_active) == 1@@ -991,7 +991,7 @@ class TestTheFourPathsAgreeOnActivity:)assert counts.room_active_1d == 1- assert counts.user_active_1d == 1+ assert counts.chat_identity_active_1d == 1assert was_ever_active is Trueassert len(newly_active) == 1assert newly_active[0].first_active_at == NOW - timedelta(minutes=90)diff --git a/core/tests/switch_core/telemetry/test_snapshot_accounts.py b/core/tests/switch_core/telemetry/test_snapshot_accounts.pynew file mode 100644index 000000000..f24afa548--- /dev/null+++ b/core/tests/switch_core/telemetry/test_snapshot_accounts.py@@ -0,0 +1,250 @@+"""Active users are Switch accounts; chat identities are counted beside them.++A person reaches Switch through a chat account — Slack, Mattermost, and so on+— and an account is active when a chat account it has claimed speaks in a room+with an agent. Unclaimed chat accounts, and one person's second platform, are+what made "active" run above "registered"; they are still counted, as+`chat_identity_*`, so nothing that number showed is lost.+"""++from __future__ import annotations++import uuid+from datetime import UTC, datetime, timedelta++import pytest+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker++from switch_core.db.models import (+ TENANT_ZERO_ID,+ Client,+ ClientRoom,+ CollaborationBridge,+ ExternalUser,+ ExternalUserClaim,+ Message,+ Room,+ Tenant,+ User,+)+from switch_core.db.session_scope import tenant_session+from switch_core.telemetry import snapshot as snapshot_module+from switch_core.telemetry.snapshot import UsageCounts, collect_usage++OTHER_TENANT = "22222222-2222-2222-2222-222222222222"+++async def _account(+ session_factory: async_sessionmaker[AsyncSession], email: str+) -> str:+ async with session_factory() as session:+ user = User(name=email, email=email, role="user")+ session.add(user)+ await session.commit()+ return user.id+++async def _other_tenant(session_factory: async_sessionmaker[AsyncSession]) -> None:+ async with session_factory() as session:+ session.add(+ Tenant(id=OTHER_TENANT, name="other", slug=f"o-{uuid.uuid4().hex[:8]}")+ )+ await session.commit()+++async def _chat_account_speaks(+ session_factory: async_sessionmaker[AsyncSession],+ *,+ claimed_by: list[str],+ tenant_id: str = TENANT_ZERO_ID,+ spoke_at: datetime | None = None,+) -> None:+ """A chat account, claimed by `claimed_by`, says something in a room with+ an agent."""+ suffix = uuid.uuid4().hex[:8]+ async with tenant_session(session_factory, tenant_id) as session:+ bridge_client = Client(+ transport_user_id=f"@bridge-{suffix}:test",+ display_name="bridge",+ type="bridge",+ )+ person = Client(+ transport_user_id=f"@person-{suffix}:test",+ display_name="person",+ type="user",+ )+ agent = Client(+ transport_user_id=f"@agent-{suffix}:test",+ display_name="agent",+ type="agent",+ )+ room = Room(+ transport_room_id=f"!{suffix}:test",+ name=f"room-{suffix}",+ description="",+ metadata_={"created_by_kind": "user"},+ )+ session.add_all([bridge_client, person, agent, room])+ await session.flush()+ bridge = CollaborationBridge(+ type="slack",+ display_name="Slack",+ client_id=bridge_client.id,+ status="active",+ )+ session.add(bridge)+ await session.flush()+ chat_account = ExternalUser(+ bridge_id=bridge.id,+ external_user_id=f"U{suffix}",+ external_username=f"person-{suffix}",+ client_id=person.id,+ )+ session.add(chat_account)+ session.add(ClientRoom(client_id=agent.id, room_id=room.id))+ await session.flush()+ for user_id in claimed_by:+ session.add(+ ExternalUserClaim(external_user_id=chat_account.id, user_id=user_id)+ )+ session.add(+ Message(+ room_id=room.id,+ seq=1,+ transport_event_id=f"$evt-{suffix}",+ sender_id=person.transport_user_id,+ sender_client_id=person.id,+ event_type="m.room.message",+ msgtype="m.text",+ body="hello",+ content={"body": "hello"},+ sent_at=spoke_at or datetime.now(UTC) - timedelta(hours=1),+ )+ )+ await session.commit()+++async def test_a_claimed_chat_account_makes_its_switch_account_active(+ session_factory: async_sessionmaker[AsyncSession],+) -> None:+ someone = await _account(session_factory, f"{uuid.uuid4().hex[:6]}@example.com")+ await _chat_account_speaks(session_factory, claimed_by=[someone])+ await _chat_account_speaks(session_factory, claimed_by=[])++ counts = await collect_usage(session_factory)++ assert counts.user_active_1d == 1+ assert counts.user_active_7d == 1+ assert counts.chat_identity_active_1d == 2+ assert counts.chat_identity_count == 2+++async def test_one_person_on_two_platforms_is_one_active_user(+ session_factory: async_sessionmaker[AsyncSession],+) -> None:+ someone = await _account(session_factory, f"{uuid.uuid4().hex[:6]}@example.com")+ await _chat_account_speaks(session_factory, claimed_by=[someone])+ await _chat_account_speaks(session_factory, claimed_by=[someone])++ counts = await collect_usage(session_factory)++ assert counts.user_active_1d == 1+ assert counts.chat_identity_active_1d == 2+++async def test_one_account_active_in_two_tenants_is_one_active_user(+ session_factory: async_sessionmaker[AsyncSession],+) -> None:+ """Accounts are deployment-wide and tenants are not, so a per-tenant count+ summed across them would add this person once for each."""+ await _other_tenant(session_factory)+ someone = await _account(session_factory, f"{uuid.uuid4().hex[:6]}@example.com")+ await _chat_account_speaks(session_factory, claimed_by=[someone])+ await _chat_account_speaks(+ session_factory, claimed_by=[someone], tenant_id=OTHER_TENANT+ )++ counts = await collect_usage(session_factory)++ assert counts.user_active_1d == 1+ assert counts.chat_identity_active_1d == 2+++async def test_a_chat_account_two_people_claim_makes_both_active(+ session_factory: async_sessionmaker[AsyncSession],+) -> None:+ first = await _account(session_factory, f"{uuid.uuid4().hex[:6]}@example.com")+ second = await _account(session_factory, f"{uuid.uuid4().hex[:6]}@example.com")+ await _chat_account_speaks(session_factory, claimed_by=[first, second])++ counts = await collect_usage(session_factory)++ assert counts.user_active_1d == 2+++async def test_activity_older_than_a_day_counts_for_the_week_only(+ session_factory: async_sessionmaker[AsyncSession],+) -> None:+ someone = await _account(session_factory, f"{uuid.uuid4().hex[:6]}@example.com")+ await _chat_account_speaks(+ session_factory,+ claimed_by=[someone],+ spoke_at=datetime.now(UTC) - timedelta(days=3),+ )++ counts = await collect_usage(session_factory)++ assert counts.user_active_1d == 0+ assert counts.user_active_7d == 1+++async def test_staff_accounts_are_counted_apart(+ session_factory: async_sessionmaker[AsyncSession],+) -> None:+ for email in (+ "[email protected]",+ "[email protected]",+ "[email protected]",+ "[email protected]",+ "[email protected]",+ ):+ await _account(session_factory, f"{uuid.uuid4().hex[:4]}.{email}")++ counts = await collect_usage(session_factory)++ assert counts.user_count == 5+ assert counts.user_internal_count == 3+++async def test_a_tenant_that_fails_after_collecting_its_accounts_adds_none_of_them(+ session_factory: async_sessionmaker[AsyncSession],+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ """A failing tenant contributes nothing, including account ids it had+ already collected before the failure, so its people are not counted from a+ pass that otherwise excluded it."""+ await _other_tenant(session_factory)+ healthy = await _account(session_factory, f"{uuid.uuid4().hex[:6]}@example.com")+ failing = await _account(session_factory, f"{uuid.uuid4().hex[:6]}@example.com")+ await _chat_account_speaks(session_factory, claimed_by=[healthy])+ await _chat_account_speaks(+ session_factory, claimed_by=[failing], tenant_id=OTHER_TENANT+ )+ collect = snapshot_module.collect_tenant_counts++ async def fails_after_collecting(+ session: AsyncSession, tenant_id: str, counts: UsageCounts, now: datetime+ ) -> None:+ await collect(session, tenant_id, counts, now)+ if tenant_id == OTHER_TENANT:+ raise RuntimeError("simulated failure after the accounts were read")++ monkeypatch.setattr(+ snapshot_module, "collect_tenant_counts", fails_after_collecting+ )++ counts = await collect_usage(session_factory)++ assert counts.tenant_failed_count == 1+ assert counts.user_active_1d == 1+ assert counts.chat_identity_active_1d == 1diff --git a/core/tests/switch_core/telemetry/test_snapshot_multi_tenant.py b/core/tests/switch_core/telemetry/test_snapshot_multi_tenant.pyindex 6eafa3ae8..1b34e6e70 100644--- a/core/tests/switch_core/telemetry/test_snapshot_multi_tenant.py+++ b/core/tests/switch_core/telemetry/test_snapshot_multi_tenant.py@@ -145,7 +145,7 @@ class TestCountsDoNotMultiplyByTenant:# Two, not four. Without the tenant predicate each pass saw both rooms.assert counts.room_count == 2assert counts.room_active_1d == 2- assert counts.user_active_1d == 2+ assert counts.chat_identity_active_1d == 2assert counts.message_count_1d == 2assert counts.room_membership_total == 2@@ -164,7 +164,7 @@ class TestCountsDoNotMultiplyByTenant:assert counts.tenant_count == 2assert counts.room_count == 1assert counts.room_active_7d == 1- assert counts.user_active_7d == 1+ assert counts.chat_identity_active_7d == 1assert counts.message_count_1d == 1diff --git a/core/tests/switch_core/test_config_telemetry_environment.py b/core/tests/switch_core/test_config_telemetry_environment.pynew file mode 100644index 000000000..2d37db1a2--- /dev/null+++ b/core/tests/switch_core/test_config_telemetry_environment.py@@ -0,0 +1,66 @@+"""TELEMETRY_ENVIRONMENT picks the Amplitude project a deployment's usage lands in.++The relay files an event under the project its `flint_env` names, and drops one+naming an environment it has no project for — answering 200 either way. So the+setting is held to the four the relay knows, and an unknown value stops the+server at startup rather than quietly losing its usage.+"""++import pytest+from pydantic import ValidationError++from switch_core.config import SwitchConfig++_BASE_KWARGS = dict(+ db_host="db",+ db_port="5432",+ db_user="postgres",+ db_password="pw",+ db_name="switch",+ matrix_server_name="switch.local",+ agent_registration_token="token",+ jwt_secret_key="jwt",+ gateway_admin_email="[email protected]",+ gateway_admin_password="pw",+ secret_keys="test:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",+)+++def test_a_deployment_that_says_nothing_is_production(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ """Every customer's deployment is production, and none of them sets it."""+ monkeypatch.delenv("TELEMETRY_ENVIRONMENT", raising=False)++ assert SwitchConfig(**_BASE_KWARGS).telemetry_environment == "prod" # type: ignore[arg-type]++[email protected]("environment", ["prod", "staging", "dev", "local"])+def test_each_environment_the_relay_knows_is_accepted(+ environment: str, monkeypatch: pytest.MonkeyPatch+) -> None:+ monkeypatch.setenv("TELEMETRY_ENVIRONMENT", environment)++ assert SwitchConfig(**_BASE_KWARGS).telemetry_environment == environment # type: ignore[arg-type]++[email protected]("environment", ["qa", "production", "development", "Dev"])+def test_an_environment_the_relay_does_not_know_is_refused(+ environment: str, monkeypatch: pytest.MonkeyPatch+) -> None:+ monkeypatch.setenv("TELEMETRY_ENVIRONMENT", environment)++ with pytest.raises(ValidationError, match="telemetry_environment"):+ SwitchConfig(**_BASE_KWARGS) # type: ignore[arg-type]+++def test_a_deployment_is_not_the_companys_own_unless_it_says_so(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ monkeypatch.delenv("TELEMETRY_INTERNAL", raising=False)++ assert SwitchConfig(**_BASE_KWARGS).telemetry_internal is False # type: ignore[arg-type]++ monkeypatch.setenv("TELEMETRY_INTERNAL", "true")++ assert SwitchConfig(**_BASE_KWARGS).telemetry_internal is True # type: ignore[arg-type]diff --git a/core/tests/switch_core/test_migration_claim_existing_connectors.py b/core/tests/switch_core/test_migration_claim_existing_connectors.pynew file mode 100644index 000000000..42a599a9e--- /dev/null+++ b/core/tests/switch_core/test_migration_claim_existing_connectors.py@@ -0,0 +1,98 @@+"""Connectors that already exist never report `connector_added`.++`connector_added` reports setup time, configuration saved to first connect.+A connector that existed before its first connect was tracked would report its+whole age instead, so the `2575637e78d4` migration takes the claim for every+connector already in the database. This seeds connectors across two tenants+and a claim one of them had already spent, runs the migration, and checks+every connector is claimed exactly once and nothing else is touched.+"""++from __future__ import annotations++from typing import Any++import pytest+from sqlalchemy import text+from sqlalchemy.ext.asyncio import create_async_engine++from tests.switch_core.test_migration_oidc_identities_backfill import _upgrade_to++_DB = "migration_claim_existing_connectors"+_PRE_MIGRATION_REVISION = "e4a7c1d93b25"+_MIGRATION_UNDER_TEST = "2575637e78d4"+++async def migration_url(postgres_url: str) -> Any:+ admin = create_async_engine(postgres_url, isolation_level="AUTOCOMMIT")+ async with admin.connect() as connection:+ await connection.execute(text(f'DROP DATABASE IF EXISTS "{_DB}"'))+ await connection.execute(text(f'CREATE DATABASE "{_DB}"'))+ await admin.dispose()++ base, _, _ = postgres_url.rpartition("/")+ try:+ yield f"{base}/{_DB}"+ finally:+ admin = create_async_engine(postgres_url, isolation_level="AUTOCOMMIT")+ async with admin.connect() as connection:+ await connection.execute(text(f'DROP DATABASE IF EXISTS "{_DB}"'))+ await admin.dispose()+++async def test_every_existing_connector_is_claimed_once(migration_url: str) -> None:+ engine = create_async_engine(migration_url)+ try:+ async with engine.begin() as connection:+ await connection.run_sync(_upgrade_to(_PRE_MIGRATION_REVISION))++ async with engine.begin() as connection:+ # The connectors' tenants and clients are beside the point here, so+ # their foreign keys are not checked while seeding.+ await connection.execute(+ text("SET LOCAL session_replication_role = replica")+ )+ await connection.execute(+ text(+ """+ INSERT INTO collaboration_bridges+ (id, type, display_name, client_id, status, tenant_id)+ VALUES+ ('slack-a', 'slack', 'Slack', 'client-a', 'active', 'tenant-a'),+ ('mm-a', 'mattermost', 'Mattermost', 'client-a', 'active', 'tenant-a'),+ ('teams-b', 'teams', 'Teams', 'client-b', 'active', 'tenant-b')+ """+ )+ )+ await connection.execute(+ text(+ """+ INSERT INTO telemetry_milestones (name)+ VALUES ('connector_added:slack-a'), ('first_connector_added')+ """+ )+ )++ async with engine.begin() as connection:+ await connection.run_sync(_upgrade_to(_MIGRATION_UNDER_TEST))++ async with engine.connect() as connection:+ claims = (+ (+ await connection.execute(+ text("SELECT name FROM telemetry_milestones ORDER BY name")+ )+ )+ .scalars()+ .all()+ )+ finally:+ await engine.dispose()++ assert claims == [+ "connector_added:mm-a",+ "connector_added:slack-a",+ "connector_added:teams-b",+ "first_connector_added",+ ]diff --git a/deploy/local/standalone-docker-compose.yml b/deploy/local/standalone-docker-compose.ymlindex de2cfd6bb..3acfcd385 100644--- a/deploy/local/standalone-docker-compose.yml+++ b/deploy/local/standalone-docker-compose.yml@@ -268,6 +268,12 @@ services:# brought up by hand, an operator sets it here the same way.TELEMETRY_ENABLED:TELEMETRY_ENDPOINT:+ # Which Amplitude project the usage lands in. Unset is prod; Switch+ # Console writes its own build's environment here for a server it runs.+ TELEMETRY_ENVIRONMENT:+ # Whether this is one of the company's own deployments. Switch Console+ # writes its own answer here for a server it runs.+ TELEMETRY_INTERNAL:TELEMETRY_SNAPSHOT_INTERVAL_HOURS:depends_on:postgres:diff --git a/deploy/remote/helm/switch/templates/_helpers.tpl b/deploy/remote/helm/switch/templates/_helpers.tplindex b0509a837..5a9dd88ca 100644--- a/deploy/remote/helm/switch/templates/_helpers.tpl+++ b/deploy/remote/helm/switch/templates/_helpers.tpl@@ -794,6 +794,10 @@ this one. Drop it once the oldest supported image reads ID_SERVER_NAME. */}}value: {{ .Values.switchCore.telemetry.endpoint | quote }}- name: TELEMETRY_SNAPSHOT_INTERVAL_HOURSvalue: {{ .Values.switchCore.telemetry.snapshotIntervalHours | quote }}+- name: TELEMETRY_ENVIRONMENT+ value: {{ .Values.switchCore.telemetry.environment | quote }}+- name: TELEMETRY_INTERNAL+ value: {{ .Values.switchCore.telemetry.internal | quote }}{{- end }}# switch-core sits behind the cluster/ALB and enforces its own# BearerAuthMiddleware, so fastmcp's browser-oriented DNS-rebindingdiff --git a/deploy/remote/helm/switch/values.yaml b/deploy/remote/helm/switch/values.yamlindex a4771e6b8..85e027875 100644--- a/deploy/remote/helm/switch/values.yaml+++ b/deploy/remote/helm/switch/values.yaml@@ -233,6 +233,13 @@ switchCore:endpoint: https://telemetry.flintai.dev# How often the usage snapshot is collected and sent.snapshotIntervalHours: 24+ # Which Amplitude project the usage lands in: prod, staging, dev or local.+ # Leave it at prod unless this is one of Flint's own non-production+ # deployments; anything else is refused at startup.+ environment: prod+ # Whether this is one of Flint's own deployments, so staff usage can be+ # told from adoption. Leave it false on any other deployment.+ internal: false# Public base URL the gateway is served on (used for links / OIDC redirects).frontendBaseUrl: ""diff --git a/docs/TELEMETRY.md b/docs/TELEMETRY.mdindex e4d630345..1d235b691 100644--- a/docs/TELEMETRY.md+++ b/docs/TELEMETRY.md@@ -36,6 +36,8 @@ abbreviated:"service.name": "switch-console","service.version": "0.9.14","flint.client_id": "3f2a9c41-8d7e-4b16-9a55-c0e1d2f47b83",+ "flint_env": "prod",+ "flint_internal": "false","os.type": "darwin","os.version": "23.6.0"},@@ -72,6 +74,8 @@ machine, in what repository, or what the prompt said.| `service.name` | constant | `switch-console` || `service.version` | app version | `0.9.14`, `1.0.2` || `build` | release channel | `dev`, `canary`, `stable` |+| `flint_env` | which analytics project the event is filed under | `prod` (a released stable build), `staging` (a released canary), `dev` (a packaged build that is not a release), `local` (running from source) |+| `flint_internal` | whether an account this app is signed in to a Switch server with has a SandboxAQ email address — **only the yes/no is sent, never the address** | `true`, `false`, `unknown` (no account, or only the one a local server signs in with) || `os.type` | OS family | `darwin`, `windows`, `linux`, `other` || `os.version` | OS release string | `23.6.0`, `10.0.22631`, `6.1.0-53-cloud-amd64` || `flint.client_id` | random install UUID | `3f2a9c41-8d7e-4b16-9a55-c0e1d2f47b83` |@@ -99,12 +103,12 @@ id, no IP field, no Switch identity.| Event | Fields, with example values ||---|---|-| `app_launched` | *(no fields)* |+| `app_launched` | `install_kind`: `new` (the installation's very first launch) / `updated` (the first launch after its version changed) / `same` || `renderer_crashed` | *(no fields)* || `update_checked` | `trigger`: `user` / `startup` / `scheduled` · `result`: `available` / `up_to_date` / `failed` || `update_downloaded` | `outcome`: `success` / `failure` || `update_install_started` | `outcome`: `success` / `failure` |-| `telemetry_consent_changed` | `source`: `first_run` / `settings` |+| `telemetry_consent_changed` | `source`: `first_run` / `settings` · `install_kind`: as on `app_launched` || `setting_changed` | `setting_key`, one of exactly 15: `theme`, `notifications`, `terminal`, `defaultAgent`, `sessions`, `location`, `localLocation`, `openIn`, `interface`, `browser`, `browserPreview`, `changesViewMode`, `remote`, `onboarding`, `telemetry`. **The new value is never sent** — we learn that someone changed their theme, not to what. || `search_performed` | `status`: `ok` / `recents` / `query-too-short` / `failed` · `result_count`: `0`, `3`, `17`. **The query is never sent.** |diff --git a/docs/old/telemetry-events.md b/docs/old/telemetry-events.mdindex a429097a0..d9c9b229d 100644--- a/docs/old/telemetry-events.md+++ b/docs/old/telemetry-events.md@@ -57,6 +57,83 @@ choices are load-bearing rather than incidental:- **A name prefix per product**, because one Amplitude project holds several.The Console sends `switch_console.<event>`; the server sends`switch_core.<event>`.+- **Numbers go as `doubleValue`, booleans as `boolValue`.** The relay delivers+ both typed to Amplitude and Datadog — zero and `false` included — so a count+ can be summed and a flag filtered on at the far end. A number sent as text+ arrives as a category instead. The relay's own test suite pins this end to+ end; the tests beside each encoder pin the half Switch controls.+- **Non-finite numbers are refused before sending.** JSON has no NaN or+ infinity, so neither could arrive as a number whatever the relay did with it.+- **At most 100 attributes per record.** The relay drops any record over 128+ and still answers 200. The margin is held by a test over every event, on both+ the server and the Console.+- **At most 16 KiB per event.** The relay drops an event over 32 KiB before it+ reaches Amplitude, because Amplitude refuses a request over 1 MB. It answers+ 200 and counts the drop. A test over every event, at its largest possible+ values, holds the margin on both clients; the largest today is about 3 KiB.++### Which Amplitude project++The relay keeps one Amplitude project per environment and files each event+under the one its `flint_env` resource attribute names: `prod`, `staging`,+`dev` or `local`. An event naming none goes to production, and one naming+anything else is dropped and counted. So every event says where it belongs,+even `prod`:++- **The server** sends `TELEMETRY_ENVIRONMENT`, which defaults to `prod`+ because every customer's deployment is production. Flint's own development+ deployment sets `dev`; `.env.example`, which `just init-env` copies for a+ developer's machine, sets `local`. Any other value stops the server at+ startup. It is set by the Helm chart's `switchCore.telemetry.environment`,+ and forwarded by the standalone compose file.+- **Switch Console** sends one per build: `prod` from a released stable build,+ `staging` from a released canary, `dev` from a packaged build that is not a+ release (a test build, or one packaged on a laptop), and `local` when run+ from source. The release workflow stamps tagged builds `VITE_RELEASE=1`, and+ a test holds every build step to it. A server Console runs for the user is+ given Console's.++`flint_env` rides on the resource, beside the client id. It describes where+the reporting process runs, not anything inside the deployment, so it is not+an identifier in the sense of the rule above.++### Internal usage++Staff use the product too, and their usage is real but is not adoption. Three+signals tell it apart, none of them an identifier:++- **`flint_internal`** on the resource: `"true"` or `"false"` from the server,+ set by `TELEMETRY_INTERNAL` (Helm `switchCore.telemetry.internal`) on the+ company's own deployments and never on a customer's. Switch Console sends+ `"true"` when any account it is signed in to a Switch server with is on+ `sandboxaq.com` or `sandboxquantum.com`, `"false"` when only other accounts+ are, and `"unknown"` when none is; only that answer leaves the machine. A+ server Console runs for the user is given Console's answer.+- **`user_internal_count`** in the daily snapshot: accounts on those domains,+ so a deployment that serves both staff and customers can be split.+- Each deployment's `flint.client_id`, for the company's own deployments by+ name.++`tenant_id` is deliberately not one of them: no identifier for anything inside+a deployment is ever sent, and a tenant is inside one.++### What the relay's 200 means++The relay answers `200` once it has accepted a payload, before it forwards+anything: it batches, and at response time it does not yet know whether+Amplitude will take the events. So a `200` means *received*, never *delivered*,+and a sender cannot learn from the response that an event was lost. That is+deliberate. An error status would be either a `400` the sender can do nothing+about or a `500` that makes it retry the same payload forever.++Making loss visible is therefore the relay's job, not the sender's: counting+what it receives, what its own checks drop, and what Amplitude and Datadog+refuse, and alerting on those counts. The relay's own repository documents the+metrics, the alerts and how they are switched on, and the gaps that remain.++What that leaves to Switch is not sending anything the relay would drop: the+event name in both places, finite numbers, the attribute and size margins, and a+valid client id.## The rule: abstracted counts, never specifics@@ -245,8 +322,12 @@ never invisible.| `tenant_failed_count` | number | tenants whose queries raised and were stepped over. One tenant's failure no longer takes the whole pass down, so the pair is what makes a partial pass self-describing: without it, every count dropping at once is indistinguishable from a deployment losing its users || `duration_ms` | number | wall time to collect the pass, on a monotonic clock. Roughly fifteen queries per tenant against the database that is also serving rooms, and a background task is invisible to `switch.http.request.duration` — so this is the only place "what does the snapshot cost at scale" can be answered from || `user_count` | number | user accounts that exist |-| `user_active_1d` | number | distinct humans who interacted in 24h |+| `user_internal_count` | number | of those, accounts on the company's own email domains — see "Internal usage" |+| `user_active_1d` | number | distinct Switch **accounts** that used a room with an agent in 24h: an account counts when a chat account it has claimed spoke. One person on two platforms is one account || `user_active_7d` | number | same over 7 days |+| `chat_identity_count` | number | chat identities — Slack, Mattermost and other platform accounts — that exist, claimed by an account or not |+| `chat_identity_active_1d` | number | distinct chat identities that used a room with an agent in 24h. Most belong to no Switch account and one person may have several, which is why this runs above `user_active_1d`. Servers released before this change reported this figure as `user_active_1d` |+| `chat_identity_active_7d` | number | same over 7 days || `room_count` | number | **the headline figure** — unarchived rooms a *person* created || `room_agent_created_count` | number | unarchived rooms an agent created for itself || `room_system_created_count` | number | unarchived channels Switch adopted after being invited to them on a platform |@@ -373,11 +454,14 @@ onboarding. A bundled connector registered before the flag existed is markedon the setup step's next run; if it first connects before that, its one`connector_added` reports `false`.-**`seconds_since_configured` is not setup time for an old connector.** The-event fires on the first connect *seen with telemetry on*. A connector that-already existed when telemetry was switched on reports its whole age on its-first connect afterwards. Deployments that predate telemetry send `-1` in-`seconds_since_install`, so filtering to `>= 0` drops most of them.+**`connector_added` fires only for a first connect that was seen.** A+connector's first connect is recorded whether telemetry is on or not, so one+that first connected while telemetry was off never reports `connector_added`:+its setup went unmeasured, and reporting it later would report its whole age.+Connectors that already existed when this tracking arrived are recorded by a+migration and never report either. Servers released before it reported an old+connector's age instead, often millions of seconds; filtering to+`seconds_since_install >= 0` drops most of those.`seconds_since_configured` and `failed_attempts_before_success` are what answer"is one platform too hard". Elapsed time from install mostly measures when