Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

feat(metrics): send the source partition with each metrics row to clickhouse (#129624)

Jimmy Zhue734a71d0098
diff --git a/nodejs/src/ingestion/pipelines/metrics/metrics-steps.test.ts b/nodejs/src/ingestion/pipelines/metrics/metrics-steps.test.ts
index edd901b1ada7..f71d37d7ff4c 100644
--- a/nodejs/src/ingestion/pipelines/metrics/metrics-steps.test.ts
+++ b/nodejs/src/ingestion/pipelines/metrics/metrics-steps.test.ts
@@ -244,6 +244,8 @@ describe('metrics ingestion steps', () => {
record_count: '3',
'retention-days': String(DEFAULT_METRICS_RETENTION_DAYS),
batch_uuid: 'b1',
+ source_topic: 'test-topic',
+ source_partition: '5',
},
})
expect(recordMetricsIngested).toHaveBeenCalledWith(7, 300, 3)
diff --git a/nodejs/src/ingestion/pipelines/metrics/produce-metrics-step.ts b/nodejs/src/ingestion/pipelines/metrics/produce-metrics-step.ts
index b36a61fd7669..259ce483037e 100644
--- a/nodejs/src/ingestion/pipelines/metrics/produce-metrics-step.ts
+++ b/nodejs/src/ingestion/pipelines/metrics/produce-metrics-step.ts
@@ -49,6 +49,10 @@ export function createProduceMetricsStep<T extends ProduceMetricsInput>(
...input.kafkaHeaders,
...headers,
'retention-days': DEFAULT_METRICS_RETENTION_DAYS.toString(),
+ // ClickHouse counts each row toward the partition it was read from too, so the
+ // freshness checkpoint sees a lagging partition on this topic.
+ source_topic: input.message.topic,
+ source_partition: input.message.partition.toString(),
},
}),
PRODUCE_RETRY