:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
PostHog metrics agent
A small deployable image that scrapes the Prometheus /metrics endpoints you already expose, or pulls metrics from Google Cloud Monitoring, and forwards them to PostHog as OTLP metrics.
Use it when you want PostHog Metrics without touching application code: no PostHog SDK, no exporter changes.
Under the hood it is the OpenTelemetry Collector (contrib distribution, pinned) with a PostHog-rendered config:
prometheus and/or googlecloudmonitoring receiver → memory_limiter + batch → otlphttp exporter pointed at PostHog ingestion.
Exemplars survive the trip: counters and histograms scraped with OpenMetrics exemplars (trace_id/span_id) become clickable trace links in the PostHog Metrics UI.
Quickstart (Docker)
docker run -d --name posthog-metrics-agent \
-e POSTHOG_API_KEY=<your project API key> \
-e POSTHOG_HOST=https://us.i.posthog.com \
-e SCRAPE_TARGETS=your-app:9090,your-worker:9091 \
posthog/metrics-agent:latest
EU cloud: set POSTHOG_HOST=https://eu.i.posthog.com.
Environment variables
| Variable | Required | Default | Meaning |
|---|---|---|---|
POSTHOG_API_KEY |
yes | — | Project API key, sent as Authorization: Bearer |
POSTHOG_HOST |
no | https://us.i.posthog.com |
PostHog ingestion origin |
SCRAPE_TARGETS |
yes* | — | Comma-separated host:port list to scrape |
SCRAPE_INTERVAL |
no | 15s |
Scrape interval |
SCRAPE_METRICS_PATH |
no | /metrics |
Metrics path on the targets |
SCRAPE_JOB_NAME |
no | posthog-metrics-agent |
Prometheus job name; becomes service_name on every metric in PostHog |
GCP_PROJECT_ID |
no | unset | GCP project to pull Cloud Monitoring metrics from; setting it enables the source |
GCP_METRICS |
yes** | — | Comma-separated metric types to pull |
GCP_METRIC_FILTERS |
yes** | — | Semicolon-separated metric descriptor filters |
GCP_COLLECTION_INTERVAL |
no | 60s |
Pull interval. The collector rejects values under 60s |
GCP_SERVICE_NAME |
no | google-cloud-monitoring |
service_name set on every pulled metric in PostHog |
GOOGLE_APPLICATION_CREDENTIALS |
no | unset | Path to a service account JSON key. Omit on GKE Workload Identity / GCE |
POSTHOG_DEBUG |
no | unset | 1/true: also log exported batches to the container's stdout |
POSTHOG_INGEST_PATH |
no | /i/v1/metrics |
Advanced: override the ingest route (used by tests) |
SHARD_COUNT |
no | 1 |
Size of an agent fleet; above 1 each instance scrapes only its share |
SHARD_INDEX |
no | from hostname ordinal | This instance's index in 0..SHARD_COUNT-1 (see Scaling out) |
PERSIST_QUEUE |
no | unset | 1/true: buffer undeliverable batches to disk so restarts lose nothing |
QUEUE_DIR |
no | /var/lib/posthog-agent |
Where the persistent queue is stored |
* not required when you mount your own scrape configs (see Escape hatches) or set GCP_PROJECT_ID.
** with GCP_PROJECT_ID: at least one of GCP_METRICS, GCP_METRIC_FILTERS, or a mounted gcp_metrics_list.yaml. Filters are separated by ;, not ,, because filter expressions can contain commas (e.g. one_of("a", "b")).
Pull metrics from Google Cloud Monitoring
Google Cloud Monitoring cannot push OTLP by itself, so the agent polls it: when GCP_PROJECT_ID is set, the agent calls the Cloud Monitoring timeSeries.list API for the metric types you list and forwards the points to PostHog, in the same pipeline as any Prometheus scrape.
Use each path for what it is for:
- Prometheus scrape — anything that exposes a
/metricsendpoint (your apps, exporters). - Direct OTLP — application code you control (PostHog SDK
posthog.metrics, or an OpenTelemetry exporter pointed at PostHog). - Cloud Monitoring pull — metrics only Google has: Cloud SQL, GKE, Cloud Run, load balancers, Pub/Sub, and the other managed services.
# The service account needs roles/monitoring.viewer (monitoring.timeSeries.list).
# Keep a dedicated copy of the key readable only by the agent's uid 10001
# (install -m 0600 -o 10001 sa.json /run/posthog-gcp/sa.json). Do not chmod 644
# the original key: that makes the private key readable by every local user.
docker run -d --name posthog-gcp-agent \
-e POSTHOG_API_KEY=<your project API key> \
-e GCP_PROJECT_ID=<gcp-project-id> \
-e GCP_METRICS=compute.googleapis.com/instance/cpu/utilization,cloudsql.googleapis.com/database/cpu/utilization \
-v /run/posthog-gcp/sa.json:/etc/gcp/sa.json:ro \
-e GOOGLE_APPLICATION_CREDENTIALS=/etc/gcp/sa.json \
posthog/metrics-agent:latest
For a prefix of metric types instead of an explicit list, use a filter:
-e GCP_METRIC_FILTERS='metric.type = starts_with("compute.googleapis.com/instance/cpu/")'
Both sources in one agent is fine — add SCRAPE_TARGETS as usual and the two receivers share the pipeline.
On GKE, skip the key file and use Workload Identity instead. Create a GCP service account with roles/monitoring.viewer, let the agent's Kubernetes service account use it, and annotate:
gcloud iam service-accounts add-iam-policy-binding agent@<gcp-project-id>.iam.gserviceaccount.com \
--role roles/iam.workloadIdentityUser \
--member "serviceAccount:<gcp-project-id>.svc.id.goog[<namespace>/<k8s-service-account>]"
# helm values
scrape:
annotationDiscovery: false
gcp:
projectId: <gcp-project-id>
metrics:
- compute.googleapis.com/instance/cpu/utilization
serviceAccount:
annotations:
iam.gke.io/gcp-service-account: agent@<gcp-project-id>.iam.gserviceaccount.com
(Outside Workload Identity, put the JSON key in a Secret and set gcp.credentialsSecret.name; the chart mounts it and sets GOOGLE_APPLICATION_CREDENTIALS for you.)
Notes:
GCP_COLLECTION_INTERVALdefaults to60s, the collector's minimum (Cloud Sampling period for most metrics; lower values are rejected). Eachmetrics_listentry costs onetimeSeries.listAPI call per interval — prefer ametric_descriptor_filterprefix over a long name list, and raise the interval for large lists (Cloud Monitoring API quota applies).- Not compatible with sharding (
SHARD_COUNT> 1 / Helmshards> 1): every shard would pull the same series. Run a separate single-instance agent (or Helm release) for Cloud Monitoring. - The pulled metrics get
service_name = google-cloud-monitoring(override withGCP_SERVICE_NAME) so they group cleanly in the Metrics UI. - The upstream receiver is alpha: metric naming and resource attributes can change across collector versions, and on restart there is no checkpoint, so a gap of up to one pull interval is possible.
Scaling out (sharding)
A single agent scrapes every target itself.
For a large target set, run a fleet: set SHARD_COUNT to the fleet size and each instance keeps only the targets whose address hashes to its SHARD_INDEX, so the fleet partitions the work with no coordination, no double-counting, and no gaps.
SHARD_INDEX defaults to the trailing ordinal of the container hostname (agent-0 → 0), so a Kubernetes StatefulSet gets correct indices for free; set it explicitly for other orchestrators.
This is the same target-sharding model vmagent uses.
The Helm chart wires this up automatically — set shards: <n> and it runs a StatefulSet fleet.
Durability
By default the export queue is in memory: if PostHog is briefly unreachable the agent retries, but a restart during that window drops whatever was buffered.
Set PERSIST_QUEUE=1 (and give the container a durable volume at QUEUE_DIR) to back the queue with disk, so samples scraped during an outage survive restarts and deliver on recovery.
The Helm chart's persistence.enabled=true provisions the volume for you.
Escape hatches
Checked in this order:
- Full config override: mount a complete collector config at
/etc/posthog/config.yaml. It is used verbatim (${env:POSTHOG_API_KEY}-style references still resolve). This is how the Helm chart drives the image. - Custom scrape configs: mount a YAML list of Prometheus
scrape_configsat/etc/posthog/scrape_configs.yamlto replace the env-generated job while keeping the PostHog exporter wiring. Tip: addscrape_protocols: [OpenMetricsText1.0.0, OpenMetricsText0.0.1, PrometheusText0.0.4]to each job so exemplars keep flowing. - Custom Cloud Monitoring metric list: mount a raw
metrics_listYAML list at/etc/posthog/gcp_metrics_list.yamlto control exactly which metric types and filters thegooglecloudmonitoringreceiver pulls (requiresGCP_PROJECT_ID). - Otherwise the scrape job is rendered from
SCRAPE_TARGETSand the Cloud Monitoring source fromGCP_PROJECT_ID+GCP_METRICS/GCP_METRIC_FILTERS.
Exemplars (metric ↔ trace linking)
- The agent scrapes with OpenMetrics negotiation enabled, which is the only Prometheus exposition format that carries exemplars. Exemplar
trace_id/span_idlabels are converted to OTLP exemplars and linked to traces in PostHog automatically. - Exemplars exist only on counters and histogram buckets; gauges never have them.
- The agent preserves exemplars, it cannot create them: your application's Prometheus client must be configured to attach them (for example prom-client's
enableExemplars, or a span-context supplier in Micrometer). If your endpoint only serves classic Prometheus text, metrics still flow but without trace links.
Notes and limits
- One
service_nameper scrape job: Prometheusjob_namemaps toservice_namein PostHog and target labels cannot override it. Run one agent (or one mounted scrape job) per logical service if you need distinct service names. - Don't run bare replicas of one agent — they all scrape the same targets (and pull the same Cloud Monitoring series) and double-count. To scale scraping, use a sharded fleet (see Scaling out) so each instance takes a disjoint slice. Cloud Monitoring is not shardable: run one puller.
- Metrics are rate limited server side per project; keep label cardinality sane (avoid user IDs, request IDs and the like as label values).
- Health endpoint for probes:
:13133. - The agent exposes its own metrics (scrape success, queue depth, points sent/dropped) at
:8888/metrics— point your monitoring at it, or scrape it with the agent itself. - The collector version is pinned in the
Dockerfile; bump it together withtests/integration/docker-compose.yml.
Development
# Config rendering golden tests (pure sh, no docker):
tests/render/run.sh
# Helm chart render + behavior tests (needs helm):
tests/helm/run.sh
# Integration smoke test (builds the image; asserts exemplars survive scrape -> OTLP,
# and validates the rendered configs — including the GCP source — with the real collector):
tests/integration/run.sh
# Durability: scrape through a simulated outage, hard-kill, assert nothing lost:
tests/durability/run.sh
# Fleet sharding at scale (synthetic target farm; completeness + disjointness):
tests/scale/run.sh
# Real-Kubernetes E2E on kind (restricted-PSS admission, discovery, sharding, exemplars):
tests/kind/run.sh
End-to-end against the local dev stack (requires hogli start with capture-logs and the metrics ingestion consumer running):
docker build -t posthog-metrics-agent:dev .
docker run --rm \
-e POSTHOG_API_KEY=phc_local \
-e POSTHOG_HOST=http://host.docker.internal:4320 \
-e SCRAPE_TARGETS=host.docker.internal:6738 \
-e SCRAPE_METRICS_PATH=/_metrics \
-e SCRAPE_JOB_NAME=agent-e2e \
posthog-metrics-agent:dev
Then observe rows arriving:
SELECT service_name, metric_name, count()
FROM posthog.metrics
WHERE service_name = 'agent-e2e'
GROUP BY 1, 2
The Google Cloud Monitoring source is covered by the render goldens and the offline collector validate step in the integration suite; a live pull needs real GCP credentials and cannot run in CI. To verify it end to end, run the agent with a service account (see above) and check for rows with service_name = 'google-cloud-monitoring' in the same query.