:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
common-kafka-consumer
Kafka consumption primitives shared by stateful services.
Modules
config:ConsumerConfigBuilder, the rdkafkaClientConfigbuilder with PostHog consumer defaults.types:OffsetandPartition.accumulator:Group, one routing key's messages from one poll on one partition, andAccumulator, the demux that builds a poll's groups.charge:Charge, a message's cost (events and bytes) against consumer admission budgets.partition_offset_ledger:PartitionOffsetLedger, offset accounting scoped to one partition assignment: completion in any order, an observable contiguous frontier, and explicit consumption at commit points.topic_offset_ledger:TopicOffsetLedger, one partition ledger per assigned topic-partition, keyed by generation so work from a previous assignment cannot land on the current one.
Metrics
TopicOffsetLedger publishes these itself, so every consumer built on the crate reports the same series.
| Metric | Labels | Meaning |
|---|---|---|
kafka_consumer_ledger_stale_slices_total (counter) |
stage |
Charges and settlements dropped because their partition was reassigned while they were in flight; a few around a rebalance are expected. |
kafka_consumer_ledger_errors_total (counter) |
stage, kind |
Contract violations in the accounting; must stay 0. A violation resets that partition's ledger and the consumer keeps running. |