Skip to content

The open-source app everyone uses to manage agents at work

Add telemetry proposal extractor (#9544)

Nicky Leach90f85a7d11c5
diff --git a/doc/TELEMETRY_WORKFLOW.md b/doc/TELEMETRY_WORKFLOW.md
new file mode 100644
index 0000000000..734442291a
--- /dev/null
+++ b/doc/TELEMETRY_WORKFLOW.md
@@ -0,0 +1,59 @@
+# Telemetry Workflow
+
+Paperclip first-party telemetry is schema-led for stable events and proposal-led for new product instrumentation.
+
+Stable events must be present in `packages/shared/src/telemetry/generated/paperclip-telemetry.ts` before normal client code emits them. Proposed events may be added ahead of schema registration only with an `@ts-expect-error` proposal marker on the `client.track()` event-name argument.
+
+## Proposed Events
+
+A proposed event is a normal `client.track()` call whose event name is not yet in `PaperclipEventName`. The runtime client swallows unregistered first-party event names before queueing, state initialization, or network flush, so proposed events do not leave the process until the generated telemetry schema adopts the event name.
+
+Use this marker shape when possible:
+
+```ts
+import type { TelemetryClient } from "./client.js";
+
+export function trackYourFeatureActionPerformed(
+ client: TelemetryClient,
+ dims: {
+ action_source: "toolbar" | "menu" | (string & {});
+ item_count: number;
+ },
+): void {
+ client.track(
+ // @ts-expect-error -- proposed-telemetry(https://github.com/paperclipai/paperclip/issues/123): measure feature action completion
+ "your_feature.action_performed",
+ dims,
+ );
+}
+```
+
+The multi-line shape is recommended because TypeScript places the unregistered-name error on the event-name line. When the schema later registers the event, that error disappears and TypeScript raises TS2578 for the now-unused directive, which tells the adopter to remove the marker in the same change that syncs the generated schema.
+
+The suffix format is:
+
+```text
+-- proposed-telemetry(<issue>): <rationale>
+```
+
+`<issue>` should be a public `https://github.com/paperclipai/paperclip/issues/123` URL. The rationale should be a short product reason for collecting the event. Missing issue or rationale text is tolerated at the call site and flagged by `scripts/extract-proposed-events.mjs`; it is not an OSS CI failure.
+
+These formatting conventions are documentation-only. Do not add repo-wide bans for `@ts-expect-error`, casts, or single-line calls as part of this workflow.
+
+## Extracting Proposals
+
+Run the extractor from the repo root:
+
+```sh
+node scripts/extract-proposed-events.mjs --ref $(git rev-parse HEAD)
+```
+
+The extractor scans `packages/shared/src/telemetry/events.ts` for `@ts-expect-error` directives attached to `<identifier>.track()` event-name arguments inside telemetry wrapper functions, including function declarations and variable-assigned arrow/function expressions. It emits `proposed-telemetry-extractor.v2` JSON with event names, primitive dimension names/types from the wrapper `dims` parameter type, rationale fields and missing-field flags, plus repo-relative file/line/column provenance.
+
+Extractor provenance is deliberately repo-relative. Absolute paths, `..` segments, Windows drive-letter paths, and backslash-separated paths are rejected so developer host paths cannot enter proposal inventory records.
+
+## Adoption
+
+When a proposed event is approved and registered in the telemetry backend, sync the regenerated telemetry artifact into the OSS repo. The event name is then part of `PaperclipEventName`, so the proposal marker should fail with TS2578. Remove the marker and keep the wrapper payload aligned with the registered dimensions in the same change.
+
+Old clients that do not yet have the synced schema continue to swallow the proposed event. Clients with the synced schema emit it through the normal stable telemetry path.
diff --git a/packages/shared/src/telemetry/README.md b/packages/shared/src/telemetry/README.md
index b81ca4eebc..ae4aad4e11 100644
--- a/packages/shared/src/telemetry/README.md
+++ b/packages/shared/src/telemetry/README.md
@@ -87,10 +87,16 @@ and let the receiving layer canonicalize it.
## Adding Or Changing Telemetry
Client code is responsible for emitting approved telemetry events at the right
-place in the product. It is not responsible for deciding which new events should
-exist. Do not introduce ad hoc event names, dimensions, or enum domains in client
-code; they must exist in the generated telemetry contract before emitters use
-them.
+place in the product. Stable event names, dimensions, and enum domains must come
+from the generated telemetry contract before normal emitters use them.
+
+For product work that needs to propose a new first-party event before schema
+registration, use the proposal marker workflow in `doc/TELEMETRY_WORKFLOW.md`.
+Those proposed calls stay on `client.track()`, carry an `@ts-expect-error`
+marker on the event-name argument, and are swallowed at runtime until the
+generated schema registers the event name.
+
+For stable event work:
1. Start from `generated/paperclip-telemetry.ts`. The generated types are what
reviewers use to verify event names, dimensions, optionality, value types,
diff --git a/packages/shared/src/telemetry/client.test.ts b/packages/shared/src/telemetry/client.test.ts
new file mode 100644
index 0000000000..87da22ba41
--- /dev/null
+++ b/packages/shared/src/telemetry/client.test.ts
@@ -0,0 +1,103 @@
+import { afterEach, describe, expect, it, vi } from "vitest";
+import { TelemetryClient } from "./client.js";
+import type { TelemetryConfig, TelemetryState } from "./types.js";
+
+const TEST_STATE: TelemetryState = {
+ installId: "test-install",
+ salt: "test-salt",
+ createdAt: "2026-01-01T00:00:00Z",
+ firstSeenVersion: "0.0.0",
+};
+
+function makeClient(stateFactory = vi.fn(() => TEST_STATE), config?: Partial<TelemetryConfig>) {
+ return {
+ client: new TelemetryClient(
+ { enabled: true, endpoint: "http://localhost:9999/ingest", ...config },
+ stateFactory,
+ "0.0.0-test",
+ ),
+ stateFactory,
+ };
+}
+
+function sentBody() {
+ const requestInit = vi.mocked(fetch).mock.calls.at(-1)?.[1] as RequestInit | undefined;
+ return JSON.parse(String(requestInit?.body ?? "{}"));
+}
+
+describe("TelemetryClient runtime event gate", () => {
+ afterEach(() => {
+ vi.restoreAllMocks();
+ });
+
+ it("swallows proposed first-party events before they touch state or the queue", async () => {
+ vi.stubGlobal("fetch", vi.fn().mockResolvedValue({ ok: true }));
+ const { client, stateFactory } = makeClient();
+
+ client.track(
+ // @ts-expect-error -- proposed-telemetry(PAP-2411): fixture proposal not in generated schema
+ "skill_studio.skill_created",
+ { sharing_scope: "team" },
+ );
+
+ await client.flush();
+
+ expect(stateFactory).not.toHaveBeenCalled();
+ expect(fetch).not.toHaveBeenCalled();
+ });
+
+ it("uses own-property membership so prototype event names are swallowed", async () => {
+ vi.stubGlobal("fetch", vi.fn().mockResolvedValue({ ok: true }));
+ const { client, stateFactory } = makeClient();
+
+ // @ts-expect-error constructor is grammar-valid but not a registered Paperclip event.
+ client.track("constructor", {});
+ // @ts-expect-error toString is grammar-valid but not a registered Paperclip event.
+ client.track("toString", {});
+
+ await client.flush();
+
+ expect(stateFactory).not.toHaveBeenCalled();
+ expect(fetch).not.toHaveBeenCalled();
+ });
+
+ it("keeps registered event batches unchanged", async () => {
+ vi.stubGlobal("fetch", vi.fn().mockResolvedValue({ ok: true }));
+ const { client, stateFactory } = makeClient();
+
+ client.track("install.started", {});
+ await client.flush();
+
+ expect(stateFactory).toHaveBeenCalledTimes(1);
+ expect(fetch).toHaveBeenCalledTimes(1);
+ expect(sentBody()).toMatchObject({
+ app: "paperclip",
+ schemaVersion: "1",
+ installId: "test-install",
+ version: "0.0.0-test",
+ events: [
+ {
+ name: "install.started",
+ dimensions: {},
+ },
+ ],
+ });
+ expect(sentBody().events[0]?.occurredAt).toEqual(expect.any(String));
+ });
+
+ it("does not change trackDynamic plugin emission", async () => {
+ vi.stubGlobal("fetch", vi.fn().mockResolvedValue({ ok: true }));
+ const { client } = makeClient();
+
+ client.trackDynamic("plugin.linear.sync_completed", { status: "ok" });
+ await client.flush();
+
+ expect(fetch).toHaveBeenCalledTimes(1);
+ expect(sentBody().events).toEqual([
+ expect.objectContaining({
+ name: "plugin.linear.sync_completed",
+ dimensions: { status: "ok" },
+ }),
+ ]);
+ });
+});
diff --git a/packages/shared/src/telemetry/client.ts b/packages/shared/src/telemetry/client.ts
index 178e5f0556..6af4e7f375 100644
--- a/packages/shared/src/telemetry/client.ts
+++ b/packages/shared/src/telemetry/client.ts
@@ -7,6 +7,7 @@ import type {
TelemetryEventName,
TelemetryState,
} from "./types.js";
+import { PAPERCLIP_EVENTS } from "./generated/paperclip-telemetry.js";
const DEFAULT_ENDPOINTS = [
"https://telemetry.paperclip.ing/ingest",
@@ -39,6 +40,7 @@ export class TelemetryClient {
* backend event schema.
*/
track<K extends TelemetryEventName>(eventName: K, ...args: TrackArgs<K>): void {
+ if (!Object.hasOwn(PAPERCLIP_EVENTS, eventName)) return;
const [dimensions] = args;
this.enqueue(eventName, dimensions);
}
diff --git a/scripts/extract-proposed-events.mjs b/scripts/extract-proposed-events.mjs
new file mode 100644
index 0000000000..ce29ace3fc
--- /dev/null
+++ b/scripts/extract-proposed-events.mjs
@@ -0,0 +1,358 @@
+#!/usr/bin/env node
+import { readFileSync } from "node:fs";
+import { isAbsolute, relative, resolve, sep } from "node:path";
+import { fileURLToPath } from "node:url";
+import ts from "typescript";
+
+export const PROPOSED_TELEMETRY_SCHEMA_VERSION = "proposed-telemetry-extractor.v2";
+
+const DEFAULT_EVENTS_FILE = "packages/shared/src/telemetry/events.ts";
+const EVENT_NAME_PATTERN = /^[a-z0-9][a-z0-9._:-]{1,63}$/;
+const ISSUE_PATTERN = /^(PAP-\d+|https:\/\/github\.com\/paperclipai\/paperclip\/issues\/\d+)$/;
+
+export function assertRepoRelativePath(value) {
+ if (typeof value !== "string" || value.length === 0) {
+ throw new Error("provenance.file must be a non-empty repo-relative path");
+ }
+ if (value.startsWith("/") || isAbsolute(value)) {
+ throw new Error(`provenance.file must be repo-relative: ${value}`);
+ }
+ if (/^[A-Za-z]:/.test(value)) {
+ throw new Error(`provenance.file must not use a drive-letter path: ${value}`);
+ }
+ if (value.includes("\\")) {
+ throw new Error(`provenance.file must use forward slashes: ${value}`);
+ }
+ const parts = value.split("/");
+ if (parts.some((part) => part === "" || part === "." || part === "..")) {
+ throw new Error(`provenance.file contains an unsafe path segment: ${value}`);
+ }
+ if (!/^[A-Za-z0-9._-]+(\/[A-Za-z0-9._-]+)*$/.test(value)) {
+ throw new Error(`provenance.file contains unsupported characters: ${value}`);
+ }
+ return value;
+}
+
+export function toRepoRelativePath(repoRoot, filePath) {
+ const absoluteRoot = resolve(repoRoot);
+ const absoluteFile = resolve(filePath);
+ const repoRelative = relative(absoluteRoot, absoluteFile).split(sep).join("/");
+ if (repoRelative === "" || repoRelative === ".." || repoRelative.startsWith("../")) {
+ throw new Error(`events file must be inside repo root: ${filePath}`);
+ }
+ return assertRepoRelativePath(repoRelative);
+}
+
+export function parseProposedTelemetryDirective(commentText) {
+ const normalized = normalizeComment(commentText);
+ if (!normalized.includes("@ts-expect-error")) return null;
+
+ const marker = normalized.match(/@ts-expect-error\b(?:\s*--\s*)?(?:proposed-telemetry\(([^)]+)\):\s*(.*))?/s);
+ if (!marker) return null;
+
+ const issue = marker[1]?.trim() || null;
+ const text = marker[2]?.trim() || null;
+
+ if (issue && !ISSUE_PATTERN.test(issue)) {
+ throw new Error(
+ `proposed telemetry rationale issue must be PAP-<digits> or a paperclipai/paperclip GitHub issue URL: ${issue}`,
+ );
+ }
+
+ return {
+ issue,
+ text,
+ missingIssue: issue === null,
+ missingRationale: text === null,
+ };
+}
+
+export function extractProposedEvents(options = {}) {
+ const repoRoot = resolve(options.repoRoot ?? process.cwd());
+ const eventsFile = resolve(repoRoot, options.eventsFile ?? DEFAULT_EVENTS_FILE);
+ const provenanceFile = toRepoRelativePath(repoRoot, eventsFile);
+ const sourceText = readFileSync(eventsFile, "utf8");
+ const sourceFile = ts.createSourceFile(eventsFile, sourceText, ts.ScriptTarget.Latest, true, ts.ScriptKind.TS);
+ const proposals = new Map();
+
+ function recordProposal(wrapper, wrapperName, eventNameNode, directive) {
+ const name = eventNameNode.text;
+ assertEventName(name, "event name");
+ const position = sourceFile.getLineAndCharacterOfPosition(eventNameNode.getStart(sourceFile));
+ const dimensions = extractWrapperDimensions(wrapper, sourceFile, wrapperName);
+
+ let proposal = proposals.get(name);
+ if (!proposal) {
+ proposal = {
+ name,
+ dimensions: new Map(),
+ rationale: { issue: null, text: null },
+ provenance: [],
+ };
+ proposals.set(name, proposal);
+ }
+
+ mergeRationale(proposal.rationale, directive, name);
+ for (const dimension of dimensions) {
+ const existing = proposal.dimensions.get(dimension.name);
+ if (existing && existing.type !== dimension.type) {
+ throw new Error(
+ `conflicting inferred types for dimension ${dimension.name} on proposed event ${name}: ${existing.type} vs ${dimension.type}`,
+ );
+ }
+ proposal.dimensions.set(dimension.name, dimension);
+ }
+ proposal.provenance.push({
+ file: provenanceFile,
+ line: position.line + 1,
+ column: position.character,
+ wrapper: wrapperName,
+ });
+ }
+
+ function visitWrapper(wrapper, wrapperName = wrapper.name?.text ?? "<anonymous>") {
+ if (!wrapper.body) return;
+
+ function visit(node) {
+ if (ts.isCallExpression(node) && isIdentifierTrackCall(node)) {
+ const eventNameNode = node.arguments[0];
+ if (eventNameNode && ts.isStringLiteral(eventNameNode)) {
+ const directive = findDirectiveForNode(sourceText, eventNameNode);
+ if (directive) recordProposal(wrapper, wrapperName, eventNameNode, directive);
+ }
+ }
+ ts.forEachChild(node, visit);
+ }
+
+ visit(wrapper.body);
+ }
+
+ function visit(node) {
+ if (ts.isFunctionDeclaration(node)) {
+ visitWrapper(node);
+ } else if (ts.isVariableDeclaration(node)) {
+ visitVariableWrapper(node);
+ }
+ ts.forEachChild(node, visit);
+ }
+
+ function visitVariableWrapper(node) {
+ if (!ts.isIdentifier(node.name)) return;
+ const initializer = node.initializer;
+ if (!initializer || (!ts.isArrowFunction(initializer) && !ts.isFunctionExpression(initializer))) return;
+ visitWrapper(initializer, node.name.text);
+ }
+
+ visit(sourceFile);
+
+ return {
+ schemaVersion: PROPOSED_TELEMETRY_SCHEMA_VERSION,
+ source: buildSource(options),
+ proposals: [...proposals.values()]
+ .map(formatProposal)
+ .sort((a, b) => a.name.localeCompare(b.name)),
+ };
+}
+
+function buildSource(options) {
+ const source = {
+ repo: options.repo ?? "paperclipai/paperclip",
+ ref: options.ref ?? process.env.GITHUB_SHA ?? process.env.PAPERCLIP_WORKSPACE_REPO_REF ?? "unknown",
+ };
+ const baseRef = options.baseRef ?? process.env.GITHUB_BASE_REF;
+ if (baseRef) source.baseRef = baseRef;
+ return source;
+}
+
+function isIdentifierTrackCall(node) {
+ const callee = node.expression;
+ return (
+ ts.isPropertyAccessExpression(callee) &&
+ callee.name.text === "track" &&
+ ts.isIdentifier(callee.expression)
+ );
+}
+
+function findDirectiveForNode(sourceText, node) {
+ const ranges = ts.getLeadingCommentRanges(sourceText, node.getFullStart()) ?? [];
+ for (const range of ranges) {
+ const directive = parseProposedTelemetryDirective(sourceText.slice(range.pos, range.end));
+ if (directive) return directive;
+ }
+ return null;
+}
+
+function normalizeComment(commentText) {
+ return commentText
+ .replace(/^\s*\/\//, "")
+ .replace(/^\s*\/\*/, "")
+ .replace(/\*\/\s*$/, "")
+ .split("\n")
+ .map((line) => line.replace(/^\s*\*\s?/, "").trim())
+ .join(" ")
+ .trim();
+}
+
+function extractWrapperDimensions(wrapper, sourceFile, wrapperName = wrapper.name?.text ?? "<anonymous>") {
+ const dimsParam = wrapper.parameters.find(
+ (parameter) => ts.isIdentifier(parameter.name) && parameter.name.text === "dims",
+ );
+ if (!dimsParam) return [];
+ if (!dimsParam.type) {
+ throw new Error(`wrapper ${wrapperName} has an untyped dims parameter`);
+ }
+ const typeNode = unwrapTypeNode(dimsParam.type);
+ if (!ts.isTypeLiteralNode(typeNode)) {
+ throw new Error(`wrapper ${wrapperName} dims parameter must be a type literal`);
+ }
+
+ return typeNode.members.map((member) => extractDimension(member, sourceFile));
+}
+
+function extractDimension(member, sourceFile) {
+ if (!ts.isPropertySignature(member) || !member.type) {
+ throw new Error("dims parameter type may contain only typed property signatures");
+ }
+ const name = propertyNameText(member.name);
+ assertEventName(name, "dimension name");
+ return {
+ name,
+ type: classifyTypeNode(member.type, sourceFile),
+ };
+}
+
+function propertyNameText(name) {
+ if (ts.isIdentifier(name) || ts.isStringLiteral(name) || ts.isNumericLiteral(name)) {
+ return name.text;
+ }
+ throw new Error("dimension names must be literal identifiers or string literals");
+}
+
+function classifyTypeNode(typeNode, sourceFile) {
+ const node = unwrapTypeNode(typeNode);
+
+ if (node.kind === ts.SyntaxKind.StringKeyword) return "string";
+ if (node.kind === ts.SyntaxKind.NumberKeyword) return "number";
+ if (node.kind === ts.SyntaxKind.BooleanKeyword) return "boolean";
+
+ if (ts.isLiteralTypeNode(node)) {
+ const literal = node.literal;
+ if (ts.isStringLiteral(literal) || literal.kind === ts.SyntaxKind.NoSubstitutionTemplateLiteral) return "string";
+ if (ts.isNumericLiteral(literal)) return "number";
+ if (literal.kind === ts.SyntaxKind.TrueKeyword || literal.kind === ts.SyntaxKind.FalseKeyword) return "boolean";
+ }
+
+ if (ts.isTypeReferenceNode(node) && ts.isIdentifier(node.typeName) && node.typeName.text === "RawDimension") {
+ const inner = node.typeArguments?.[0];
+ if (!inner) throw new Error(`RawDimension is missing a type argument at ${node.getText(sourceFile)}`);
+ return classifyTypeNode(inner, sourceFile);
+ }
+
+ if (ts.isUnionTypeNode(node)) {
+ const primitiveTypes = new Set();
+ for (const member of node.types) {
+ const unwrapped = unwrapTypeNode(member);
+ if (unwrapped.kind === ts.SyntaxKind.UndefinedKeyword) continue;
+ if (unwrapped.kind === ts.SyntaxKind.NullKeyword) continue;
+ if (ts.isLiteralTypeNode(unwrapped) && unwrapped.literal.kind === ts.SyntaxKind.NullKeyword) continue;
+ primitiveTypes.add(classifyTypeNode(unwrapped, sourceFile));
+ }
+ if (primitiveTypes.size !== 1) {
+ throw new Error(`dimension union must resolve to one primitive type: ${node.getText(sourceFile)}`);
+ }
+ return [...primitiveTypes][0];
+ }
+
+ throw new Error(`unsupported dimension type: ${node.getText(sourceFile)}`);
+}
+
+function unwrapTypeNode(typeNode) {
+ let node = typeNode;
+ while (ts.isParenthesizedTypeNode(node)) {
+ node = node.type;
+ }
+ return node;
+}
+
+function assertEventName(value, label) {
+ if (!EVENT_NAME_PATTERN.test(value)) {
+ throw new Error(`${label} must match ${EVENT_NAME_PATTERN}: ${value}`);
+ }
+}
+
+function mergeRationale(target, incoming, eventName) {
+ mergeRationaleField(target, incoming, "issue", eventName);
+ mergeRationaleField(target, incoming, "text", eventName);
+}
+
+function mergeRationaleField(target, incoming, key, eventName) {
+ const value = incoming[key];
+ if (!value) return;
+ if (target[key] && target[key] !== value) {
+ throw new Error(`conflicting rationale ${key} for proposed event ${eventName}`);
+ }
+ target[key] = value;
+}
+
+function formatProposal(proposal) {
+ const provenance = proposal.provenance
+ .map((item) => ({
+ file: assertRepoRelativePath(item.file),
+ line: item.line,
+ column: item.column,
+ }))
+ .sort((a, b) => a.file.localeCompare(b.file) || a.line - b.line || a.column - b.column);
+
+ return {
+ name: proposal.name,
+ dimensions: [...proposal.dimensions.values()].sort((a, b) => a.name.localeCompare(b.name)),
+ rationale: {
+ issue: proposal.rationale.issue,
+ text: proposal.rationale.text,
+ missingIssue: proposal.rationale.issue === null,
+ missingRationale: proposal.rationale.text === null,
+ },
+ provenance,
+ };
+}
+
+function parseArgs(argv) {
+ const options = {};
+ for (let index = 0; index < argv.length; index += 1) {
+ const arg = argv[index];
+ if (arg === "--help" || arg === "-h") {
+ options.help = true;
+ continue;
+ }
+ const next = argv[index + 1];
+ if (!next || next.startsWith("--")) {
+ throw new Error(`${arg} requires a value`);
+ }
+ index += 1;
+ if (arg === "--repo-root") options.repoRoot = next;
+ else if (arg === "--events-file") options.eventsFile = next;
+ else if (arg === "--repo") options.repo = next;
+ else if (arg === "--ref") options.ref = next;
+ else if (arg === "--base-ref") options.baseRef = next;
+ else throw new Error(`unknown option: ${arg}`);
+ }
+ return options;
+}
+
+function printHelp() {
+ process.stdout.write(`Usage: node scripts/extract-proposed-events.mjs [options]\n\nOptions:\n --repo-root <path> Repository root. Defaults to cwd.\n --events-file <path> events.ts path, absolute or repo-relative.\n --repo <slug> Source repository slug. Defaults to paperclipai/paperclip.\n --ref <ref> Source ref/SHA for the extractor envelope.\n --base-ref <ref> Optional base ref for diff-oriented inventory jobs.\n`);
+}
+
+if (process.argv[1] === fileURLToPath(import.meta.url)) {
+ try {
+ const options = parseArgs(process.argv.slice(2));
+ if (options.help) {
+ printHelp();
+ process.exit(0);
+ }
+ process.stdout.write(`${JSON.stringify(extractProposedEvents(options), null, 2)}\n`);
+ } catch (error) {
+ process.stderr.write(`${error instanceof Error ? error.message : String(error)}\n`);
+ process.exit(1);
+ }
+}
diff --git a/scripts/extract-proposed-events.test.mjs b/scripts/extract-proposed-events.test.mjs
new file mode 100644
index 0000000000..6f3181ff9c
--- /dev/null
+++ b/scripts/extract-proposed-events.test.mjs
@@ -0,0 +1,261 @@
+import assert from "node:assert/strict";
+import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import test from "node:test";
+import ts from "typescript";
+
+import {
+ PROPOSED_TELEMETRY_SCHEMA_VERSION,
+ assertRepoRelativePath,
+ extractProposedEvents,
+ toRepoRelativePath,
+} from "./extract-proposed-events.mjs";
+
+function withFixtureRepo(source, callback) {
+ const repoRoot = mkdtempSync(join(tmpdir(), "paperclip-proposed-events-"));
+ const eventsFile = join(repoRoot, "packages", "shared", "src", "telemetry", "events.ts");
+ mkdirSync(join(repoRoot, "packages", "shared", "src", "telemetry"), { recursive: true });
+ writeFileSync(eventsFile, source);
+ try {
+ return callback({ repoRoot, eventsFile });
+ } finally {
+ rmSync(repoRoot, { recursive: true, force: true });
+ }
+}
+
+const fixtureSource = `
+import type { TelemetryClient } from "./client.js";
+
+type RawDimension<T extends string | undefined> = T | (string & {});
+
+export function trackSkillStudioCreated(
+ client: TelemetryClient,
+ dims: {
+ sharing_scope: RawDimension<"team" | "private">;
+ category_count: number;
+ launched_from_template: boolean;
+ },
+): void {
+ client.track(
+ // @ts-expect-error -- proposed-telemetry(PAP-2411): measure Skill Studio create completion
+ "skill_studio.skill_created",
+ dims,
+ );
+}
+
+export function trackSkillStudioOpened(
+ client: TelemetryClient,
+ dims: {
+ surface: "modal" | "page";
+ },
+): void {
+ client.track(
+ // @ts-expect-error
+ "skill_studio.opened",
+ dims,
+ );
+}
+
+export function trackInstallStarted(client: TelemetryClient): void {
+ client.track("install.started", {});
+}
+`;
+
+test("extractor emits deterministic proposed-telemetry-extractor.v2 records", () => {
+ const output = withFixtureRepo(fixtureSource, ({ repoRoot, eventsFile }) =>
+ extractProposedEvents({ repoRoot, eventsFile, ref: "fixture-sha", baseRef: "master" }),
+ );
+
+ assert.equal(output.schemaVersion, PROPOSED_TELEMETRY_SCHEMA_VERSION);
+ assert.deepEqual(output.source, {
+ repo: "paperclipai/paperclip",
+ ref: "fixture-sha",
+ baseRef: "master",
+ });
+ assert.deepEqual(
+ output.proposals.map((proposal) => proposal.name),
+ ["skill_studio.opened", "skill_studio.skill_created"],
+ );
+
+ const created = output.proposals.find((proposal) => proposal.name === "skill_studio.skill_created");
+ assert.deepEqual(created.dimensions, [
+ { name: "category_count", type: "number" },
+ { name: "launched_from_template", type: "boolean" },
+ { name: "sharing_scope", type: "string" },
+ ]);
+ assert.deepEqual(created.rationale, {
+ issue: "PAP-2411",
+ text: "measure Skill Studio create completion",
+ missingIssue: false,
+ missingRationale: false,
+ });
+ assert.equal(created.provenance.length, 1);
+ assert.equal(created.provenance[0].file, "packages/shared/src/telemetry/events.ts");
+ assert.equal(typeof created.provenance[0].line, "number");
+ assert.equal(typeof created.provenance[0].column, "number");
+});
+
+test("extractor flags a missing proposed-telemetry suffix without hard-failing", () => {
+ const output = withFixtureRepo(fixtureSource, ({ repoRoot, eventsFile }) =>
+ extractProposedEvents({ repoRoot, eventsFile, ref: "fixture-sha" }),
+ );
+
+ const opened = output.proposals.find((proposal) => proposal.name === "skill_studio.opened");
+ assert.deepEqual(opened.rationale, {
+ issue: null,
+ text: null,
+ missingIssue: true,
+ missingRationale: true,
+ });
+ assert.deepEqual(opened.dimensions, [{ name: "surface", type: "string" }]);
+});
+
+test("extractor scans TelemetryClient wrappers whose receiver is not named client", () => {
+ const output = withFixtureRepo(
+ `type TelemetryClient = { track(name: string, dims: unknown): void };
+
+export function trackWorkspaceOpened(
+ telemetry: TelemetryClient,
+ dims: { surface: string },
+): void {
+ telemetry.track(
+ // @ts-expect-error -- proposed-telemetry(PAP-2463): exercise alternate telemetry client parameter names
+ "workspace.opened",
+ dims,
+ );
+}
+`,
+ ({ repoRoot, eventsFile }) => extractProposedEvents({ repoRoot, eventsFile, ref: "fixture-sha" }),
+ );
+
+ assert.deepEqual(
+ output.proposals.map((proposal) => proposal.name),
+ ["workspace.opened"],
+ );
+ assert.deepEqual(output.proposals[0].dimensions, [{ name: "surface", type: "string" }]);
+});
+
+test("extractor scans variable-assigned wrappers and ignores nullable union members", () => {
+ const output = withFixtureRepo(
+ `type TelemetryClient = { track(name: string, dims: unknown): void };
+
+export const trackWorkspaceArrow = (
+ telemetry: TelemetryClient,
+ dims: { surface: string | null | undefined },
+): void => {
+ telemetry.track(
+ // @ts-expect-error -- proposed-telemetry(PAP-2463): exercise arrow wrapper extraction
+ "workspace.arrow_opened",
+ dims,
+ );
+};
+
+export const trackWorkspaceFunctionExpression = function (
+ tc: TelemetryClient,
+ dims: { accepted: true | false | null },
+): void {
+ tc.track(
+ // @ts-expect-error -- proposed-telemetry(PAP-2463): exercise function-expression wrapper extraction
+ "workspace.function_expression_opened",
+ dims,
+ );
+};
+`,
+ ({ repoRoot, eventsFile }) => extractProposedEvents({ repoRoot, eventsFile, ref: "fixture-sha" }),
+ );
+
+ assert.deepEqual(
+ output.proposals.map((proposal) => proposal.name),
+ ["workspace.arrow_opened", "workspace.function_expression_opened"],
+ );
+ assert.deepEqual(output.proposals[0].dimensions, [{ name: "surface", type: "string" }]);
+ assert.deepEqual(output.proposals[1].dimensions, [{ name: "accepted", type: "boolean" }]);
+});
+
+test("extractor rejects invalid rationale issue references when present", () => {
+ assert.throws(
+ () =>
+ withFixtureRepo(
+ `export function trackBad(client, dims: { source: string }): void {\n client.track(\n // @ts-expect-error -- proposed-telemetry(PROJ-1): bad issue ref\n "skill_studio.bad_issue",\n dims,\n );\n}\n`,
+ ({ repoRoot, eventsFile }) => extractProposedEvents({ repoRoot, eventsFile, ref: "fixture-sha" }),
+ ),
+ /rationale issue must be PAP-<digits>/,
+ );
+});
+
+test("provenance paths are repo-relative and reject dev-host path shapes", () => {
+ assert.equal(
+ assertRepoRelativePath("packages/shared/src/telemetry/events.ts"),
+ "packages/shared/src/telemetry/events.ts",
+ );
+ assert.throws(() => assertRepoRelativePath("/tmp/events.ts"), /repo-relative/);
+ assert.throws(() => assertRepoRelativePath("../events.ts"), /unsafe path segment/);
+ assert.throws(() => assertRepoRelativePath("C:/repo/events.ts"), /drive-letter/);
+ assert.throws(() => assertRepoRelativePath("packages\\shared\\events.ts"), /forward slashes/);
+
+ const repoRoot = mkdtempSync(join(tmpdir(), "paperclip-provenance-root-"));
+ try {
+ assert.throws(() => toRepoRelativePath(repoRoot, join(repoRoot, "..", "events.ts")), /inside repo root/);
+ } finally {
+ rmSync(repoRoot, { recursive: true, force: true });
+ }
+});
+
+function diagnosticsFor(sourceText) {
+ const repoRoot = mkdtempSync(join(tmpdir(), "paperclip-ts2578-"));
+ const fileName = join(repoRoot, "fixture.ts");
+ writeFileSync(fileName, sourceText);
+ try {
+ const program = ts.createProgram([fileName], {
+ strict: true,
+ noEmit: true,
+ skipLibCheck: true,
+ target: ts.ScriptTarget.ES2023,
+ module: ts.ModuleKind.NodeNext,
+ moduleResolution: ts.ModuleResolutionKind.NodeNext,
+ types: [],
+ });
+ return ts
+ .getPreEmitDiagnostics(program)
+ .filter((diagnostic) => diagnostic.file?.fileName === fileName)
+ .map((diagnostic) => diagnostic.code);
+ } finally {
+ rmSync(repoRoot, { recursive: true, force: true });
+ }
+}
+
+function tsMechanicsFixture(eventUnion, mapEntry) {
+ return `
+type TelemetryEventName = ${eventUnion};
+interface EventDimensionsMap {
+ "install.started": {};
+ ${mapEntry}
+}
+type TelemetryEventDimensions<K extends TelemetryEventName> = EventDimensionsMap[K];
+type TrackArgs<K extends TelemetryEventName> = keyof TelemetryEventDimensions<K> extends never
+ ? [dimensions?: TelemetryEventDimensions<K>]
+ : [dimensions: TelemetryEventDimensions<K>];
+declare const client: {
+ track<K extends TelemetryEventName>(eventName: K, ...args: TrackArgs<K>): void;
+};
+client.track(
+ // @ts-expect-error -- proposed-telemetry(PAP-2411): TS2578 expiry fixture
+ "skill_studio.skill_created",
+ { sharing_scope: "team" },
+);
+`;
+}
+
+test("TS2578 expires the directive once a fixture event is registered", () => {
+ const unregistered = diagnosticsFor(tsMechanicsFixture('"install.started"', ""));
+ assert.deepEqual(unregistered, []);
+
+ const registered = diagnosticsFor(
+ tsMechanicsFixture(
+ '"install.started" | "skill_studio.skill_created"',
+ '"skill_studio.skill_created": { sharing_scope: string };',
+ ),
+ );
+ assert.ok(registered.includes(2578), `expected TS2578, got ${registered.join(", ")}`);
+});