Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

22.2 KB
Raw Blame

Choosing an AI gateway

✅ Default to PostHog/ai-gateway for new callers and features.

⚠️ Use services/llm-gateway temporarily only when an active caller is blocked by a gap below.

Migration policy

services/llm-gateway is under an unofficial code freeze. Everyone should move to the Go gateway unless they can name a contract it does not support.

A PR that changes the Python gateway must:

  1. Name the active caller and its use case.
  2. Name the exact Go gateway parity gap.
  3. Explain why the change cannot wait for or land in the Go gateway.
  4. Limit the Python change to the blocked caller's needs.
  5. Update this document when it discovers or closes a gap.

Bug, security, and reliability fixes for blocked callers are valid reasons. Convenience, an existing Python client, or an unfamiliar Go API are not parity gaps.

When a gap closes, new work uses the Go gateway and affected callers should migrate. Do not add the same feature to both gateways by default.

Choose by use case

✅ Use the Go gateway

Use case Why it fits
Customer LLM traffic The caller uses a project secret or OAuth credential, the team wallet should pay, and standard OpenAI or Anthropic APIs are enough.
Server-to-server PostHog call A phs_ credential, team wallet billing, and informational event properties provide enough policy and attribution. Internal spend can use a PostHog-owned team wallet.
Capped delegated run A standard credential can mint a short-lived phe_ token with pinned attribution and a per-token spend cap.
Stock SDK proxy The caller needs OpenAI Chat Completions or Responses, Anthropic Messages or token counting, streaming, idempotency, or the model catalog.
Gateway-managed routing The caller accepts operator-managed provider plans and health-aware selection across OpenAI, Anthropic, Azure OpenAI, Bedrock, or configured Modal, Fireworks, and Baseten hosts.

Existing Django callers should use build_openai_client, build_async_openai_client, build_anthropic_client, or build_async_anthropic_client from posthog/llm/gateway_client.py. The builders forward product attribution and caller-selected metadata while keeping a temporary Python fallback during rollout.

⛔ Stay on the Python gateway for now

Use case Blocking contract
Cloud agents, onboarding, and similar first-party products Go can pin product and user attribution in a scoped token, but it does not apply Django's OAuth application allowlists, project access checks, plans, or billing policy. PostHog Desktop and PostHog Code mint scoped tokens on the Go gateway; the posthog_code product is retired here and answers 403 with reason product_retired. An asserted product remains caller-controlled and is not an authorization boundary.
Django plan or quota enforcement Go supports rolling attribution budgets, a hard team cap, and per-token caps. Python still owns plan checks, quota buckets, free-model policy, and requests that debit no wallet.
Agent runs a customer can start on demand Go can cap one run with a scoped token, but the Django integration only mints tokens for server-authorized internal products. User-started runs still need Python's provenance-based policy. The wizard is the exception, and the worked example: Django runs the blocklist, the rollout switch, the program pin, the mint throttle, and the posture-tiered cap at mint, then pins the model, effort, product, and org into the token. The wizard product is retired here, so a wizard call on this gateway answers 403 with an upgrade message.
OpenRouter or Cloudflare Workers AI These provider paths are not available in Go.
OpenAI audio transcription Go has no transcription endpoint.
OpenAI models through Anthropic Messages Go does not provide this reverse translation.
Python product usage status The Python product usage and quota API is different from Go request usage and wallet APIs.

The posthog_code product is retired on this gateway. PostHog Desktop and PostHog Code mint scoped tokens on the Go gateway, and a posthog_code call here answers 403 with reason product_retired and a message that names the token refusals that send a client here. The desktop access gate and the Code-only per-user spend caps stay in the code, but no request reaches them. The retirement also closes the gap where the posthog_code_credits quota check failed open: Django's quota endpoint requires project:read, a Code OAuth token carries only llm_gateway:read, so every check answered 403 and the gateway let the request through.

Open-weight models added to the Python gateway under the freeze stay there for the same reason. zai-org/glm-5.3-flash is Baseten-exclusive and reachable only by the review_hog product. Its caller is ReviewHog, which uses a shared gateway key and the product model pin. No open-weight model is behind a per-model access flag now; the mechanism stays for the next rollout. The Go catalog serves GLM 5.2 only, and Baseten on Go still depends on the provider deployment check below. gpt-6-astra is also Python-only because the Go catalog does not serve it, while cloud agents require the Python product policy above. claude-opus-5-5 and gpt-6-sol stay Python-only for the same reason: the Go catalog does not serve them. The Go catalog serves gpt-6-luna, and the slack_app_routing route allows it too, so the Slack reply suggestions keep working in an environment that has only the Python gateway.

gpt-6.1-sol uses the Python product policies for Slack tasks and background agents. These callers still require the OAuth application restrictions and product billing policies above, even when the Go catalog discovers the model.

🔎 Verify before switching

These are compatibility checks, not automatic blockers:

  • Model: it appears in GET /v1/models and supports the requested API shape and capabilities.
  • Credential: Go accepts phs_, pha_, and gateway-minted phe_ credentials. Confirm the resolved team, scope, revocation state, attribution, and cap.
  • Billing: identify which team owns the Go credential and should pay, then confirm its wallet is funded for the expected usage. Internal workloads can debit a PostHog-owned team wallet so spend stays attributable without charging a customer.
  • Attribution: X-PostHog-Product, X-PostHog-Distinct-Id, X-PostHog-Trace-Id, and X-PostHog-Properties provide enough event context. Go does not derive the event distinct ID from OpenAI user, Anthropic metadata.user_id, or the OAuth user. Caller-supplied attribution is not trusted policy.
  • Session attribution: X-PostHog-Session-Id provides the required gateway-owned $ai_session_id.
  • Provider deployment: Fireworks and Baseten host support exists in Go, but the target environment must declare the required host and credentials. Baseten also requires approved-subprocessor status before deployment.
  • Provider behavior: operator routing policy, health-aware provider ordering, or strict X-PostHog-Provider pinning matches the caller's fallback requirements.
  • Wire behavior: request fields, streaming chunks, errors, timeouts, and retries match what the caller handles.
  • Metadata: Python per-key property and feature flag headers are converted to the Go JSON properties header.

Parity map

Contract Go gateway today Python-only behavior that can block migration
Authentication phs_ project secrets and pha_ OAuth credentials require llm_gateway:read. A pha_ credential is also rejected when its user has is_email_verified = false; NULL (legacy accounts) and true pass. Standard credentials can mint capped and revocable phe_ tokens. phx_ personal keys (rejected once their expires_at passes and on the same is_email_verified = false rule, without the email-verification-disabled incident flag bypass that Django honors), wildcard OAuth scope handling, product-specific API key or OAuth application rules, and Desktop's request-selected project validated against OAuth scope and live organization membership.
Trusted first-party identity Scoped tokens pin product, user, and obo attribution at mint. Standard credential headers remain caller-controlled and do not authorize a product. Product routes select allowlisted OAuth apps, server credential requirements, model policy, and Django product policy. Django also decides which callers may receive a product-scoped credential.
Attribution Distinct ID comes from X-PostHog-Distinct-Id or falls back to a namespaced team identity. Product, user, and obo attribution land on events and ledger entries with provenance. OAuth user identity, ACL-resolved OAuth team attribution, staff-owned personal-key customer-team overrides, body-derived end-user identity, gateway-owned $ai_billable, per-key properties, and feature flag headers.
Billing and limits Wallet reservation and settlement, soft attribution budgets, a hard team cap, per-token caps, idempotency, balance and usage reads, and front-line rate limits. Plans, quota buckets, product-specific free-model policy, unbilled products, product usage status, and Django provenance rules that decide which cap applies.
OpenAI APIs Chat Completions, Responses, bare Responses alias, and normalized router chat. Audio transcription and broader LiteLLM translation.
Anthropic APIs Messages and token counting, including Bedrock-hosted models. OpenAI models exposed through the Anthropic shape and Python-specific Bedrock opt-in behavior.
Providers OpenAI, Anthropic, Azure OpenAI, Bedrock, and configured Modal, Fireworks, and Baseten hosts. OpenRouter and Cloudflare Workers AI.
Models Gateway-owned catalog, canonical IDs and aliases, capability checks, router categories, and OpenRouter-shaped pricing. Broader LiteLLM model acceptance, Python product allowlists, per-model access flags with caller-filtered listings, and product-scoped model pricing.
Routing and failure behavior Operator-managed provider plans, health-aware ordering, circuit breakers, hosted-provider failover, and strict provider pinning. Caller opt-in Bedrock fallback and provider-specific Python routing.
Event metadata One X-PostHog-Properties JSON object plus dedicated product, user, obo, distinct ID, trace ID, and provider headers. X-POSTHOG-PROPERTY-* and X-POSTHOG-FLAG-* headers.
Session attribution X-PostHog-Session-Id is recorded as the gateway-owned $ai_session_id. The per-key property header can also emit $ai_session_id.
Request validation Not verified for a bare-string or unknown-type content part in a Chat Completions user message. The Chat Completions route answers 400 invalid_request_error before choosing a provider when a user message carries a content part outside litellm's allowed types. Callers on this gateway send these, and litellm otherwise raises an error with no status code that the handler reports as 500.

Migration checklist

Run /migrating-llm-gateway-callers to inventory and convert a caller.

  1. Describe the caller and identify which identity controls its access and spend.
  2. Check every relevant contract above against the real request and response.
  3. Start with a shared Go-capable builder where one exists.
  4. Test success, streaming if used, provider errors, attribution, and billing.
  5. Keep the Python fallback only for staged rollout or a named blocker. Record the rollout plan or blocker in the PR.

Refreshing this document

Run /auditing-llm-gateway-parity after either gateway changes auth, attribution, billing, endpoints, providers, models, routing, or event metadata. The skill audits implementation sources in both repositories and updates this file without migrating callers.

Last verified on 2026-09-08 against:

  • PostHog/posthog working tree compared with master at 5d1154d4f2d93613a5d9875e9c11d66778958a79
  • PostHog/ai-gateway main at 7f61cfe690a855d465e362297ff76b98d6526314

References