Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
feature/Analytics-raw-user-id · L-260923-09ed58 (#1235)
Thomas Hebrard8e2a73fc5755
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 369431bf2..ea91f8a49 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -1,5 +1,11 @@# Changelog+## [Unreleased]++### Changed++- **Pipelex's own telemetry stream receives the run's `user_id` as it is**: the Gateway stream sent a one-way digest of the caller taken inside the Gateway-key hash, and carried a `deployment` group instead of the run's own `analytics_groups`. It now resolves identity like the operator's identified stream — the run's `user_id` is the PostHog `distinct_id` and its `analytics_groups` ride the groups facet — so a host's events and the runtime's land on the same person and the same organization. The `NAMESPACED` identity policy and the `deployment` group type are removed. Anything that names no caller still reports under the Gateway-key hash.+## [v0.63.0] - 2026-09-23### Addeddiff --git a/docs/features/telemetry.md b/docs/features/telemetry.mdindex c6daaf0c3..2132f3efb 100644--- a/docs/features/telemetry.md+++ b/docs/features/telemetry.md@@ -25,7 +25,7 @@ Event tracking and AI span tracing with fine-grained privacy controls. Choose be## Per-run attribution-A span or an event produced during a run is attributed to that run's own caller, not to the process that served it — which is what lets one deployment run many people's methods and still see each person's generations on their own timeline. The identity comes from the run: `user_id` is who started it, and the optional `analytics_groups` mapping says which of your entities it belongs to. Both are supplied where the run is, they ride the job through every nested pipe and across a process boundary, and they reach PostHog in the fields it reserves for identity — never as an event property or a span name. Anything that names no caller of its own reports under the `user_id` you configured: an event outside any run, and a run on your own machine, whose caller is the literal `local` rather than a person a backend could tell apart — though its groups still ride, because which entities a run belongs to is a separate question from who started it. All of that describes the destinations you control; Pipelex's own Gateway stream receives a one-way digest of the caller inside your key's hash and none of your groups. See [Telemetry Setup](../setup/telemetry.md) for how to supply the groups.+A span or an event produced during a run is attributed to that run's own caller, not to the process that served it — which is what lets one deployment run many people's methods and still see each person's generations on their own timeline. The identity comes from the run: `user_id` is who started it, and the optional `analytics_groups` mapping says which of your entities it belongs to. Both are supplied where the run is, they ride the job through every nested pipe and across a process boundary, and they reach PostHog in the fields it reserves for identity — never as an event property or a span name. Anything that names no caller of its own reports under the `user_id` you configured: an event outside any run, and a run on your own machine, whose caller is the literal `local` rather than a person a backend could tell apart — though its groups still ride, because which entities a run belongs to is a separate question from who started it. Pipelex's own Gateway stream is attributed the same way: the run's `user_id` as it is, and its groups. See [Telemetry Setup](../setup/telemetry.md) for how to supply the groups.## Gateway Telemetrydiff --git a/docs/setup/telemetry.md b/docs/setup/telemetry.mdindex 70e2e09a0..590a70d71 100644--- a/docs/setup/telemetry.md+++ b/docs/setup/telemetry.md@@ -12,7 +12,7 @@ Pipelex supports two independent telemetry streams that serve different purposesWhen you use **Pipelex Gateway** as your inference backend, identified telemetry is **automatically enabled**. This telemetry is tied to your Gateway API key (hashed for security) and operates independently from your `telemetry.toml` settings.-A run that names a caller of its own is still distinguished on this stream, but never by a value you supplied: the caller's `user_id` is folded one way into your key's hash, so the value you spelled never reaches us, and the same caller name at another deployment is a different person here. Your key's own hash travels beside the fold, so your usage stays countable as yours. Read the fold as a pseudonym rather than as a promise of anonymity: it cannot be reversed, but it is a digest and not a secret, so it is not proof against somebody who already holds a candidate id and wants to check it. Your `analytics_groups` are not forwarded to this stream at all — they are your own vocabulary about your own customers, and they stay on the destinations you control.+A run that names a caller of its own is attributed to that caller on this stream too: the run's `user_id` is sent as the `distinct_id` exactly as you supplied it, and its `analytics_groups` ride the capture as groups. Anything that names no caller reports under your key's hash.**What we collect:**@@ -27,7 +27,6 @@ A run that names a caller of its own is still distinguished on this stream, but- Your prompts or completions- Your pipe codes or output class names- File contents or business data-- A run's `user_id` as you spell it, or its `analytics_groups` in any formThis telemetry allows us to:@@ -115,7 +114,7 @@ The run-level metadata every run carries (`RunMetadata`) has an `analytics_groupanalytics_groups = {"organization": "org_acme"}```-On **your own** PostHog stream, every span and every event the run produces is then captured under that run's `user_id`, with the groups attached through PostHog's own groups facet — so a generation made for one of your customers appears on that customer's timeline and inside their organization, instead of under one identity per deployment. The same values reach every OpenTelemetry exporter as the span attributes `pipelex.run.user_id` and `pipelex.run.analytics_groups`, and Langfuse receives the user id in the field it reserves for it. Pipelex's own Gateway stream is the one exception, and it is described above: it receives a one-way digest of the caller and none of your groups.+On **your own** PostHog stream, every span and every event the run produces is then captured under that run's `user_id`, with the groups attached through PostHog's own groups facet — so a generation made for one of your customers appears on that customer's timeline and inside their organization, instead of under one identity per deployment. The same values reach every OpenTelemetry exporter as the span attributes `pipelex.run.user_id` and `pipelex.run.analytics_groups`, and Langfuse receives the user id in the field it reserves for it. Pipelex's own Gateway stream receives the same `user_id` and groups, as described above.Anything that names no caller of its own keeps reporting under the `user_id` you configured in `telemetry.toml`, which is what that setting now means: the identity of everything that is not one caller's run. That covers an event outside any run — a CLI command listing your pipes — and a run whose caller is not a distinguishable person either: a run on your own machine is attributed to the literal `local`, the same string on every machine, so Pipelex declines it as an identity and uses your configured id instead. The same goes for `single-tenant`, which pipelex-api states for every run when it is deployed with no users. Per-run attribution is for a host that passes a real `user_id` per run.diff --git a/pipelex/system/analytics_groups.py b/pipelex/system/analytics_groups.pyindex 51d5c6eb2..10111aa26 100644--- a/pipelex/system/analytics_groups.py+++ b/pipelex/system/analytics_groups.py@@ -10,11 +10,9 @@ mapping of group *type* to group *key* that the runtime carries on`RunMetadata`, unread, for a telemetry consumer that understands groups toforward. Carrying it unread is all this module does, but it is read downstream:a run's mapping rides its spans as `pipelex.run.analytics_groups` and becomes-PostHog's own groups facet on the operator's stream, so whatever is put here is-a live group key in that project and a span attribute on every OpenTelemetry-export. Pipelex's own stream is the exception — one shared project across every-deployment, where a host's group vocabulary would collide with another's, so-the mapping never reaches it.+PostHog's own groups facet on the operator's stream and on Pipelex's own, so+whatever is put here is a live group key in those projects and a span attribute+on every OpenTelemetry export.The hosted Pipelex platform fills it with `{"organization": "<org_id>"}`, butnothing here knows or checks that: no key is privileged, and a deployment thatdiff --git a/pipelex/system/telemetry/otel_factory.py b/pipelex/system/telemetry/otel_factory.pyindex 79cfbb8e9..2e40e745b 100644--- a/pipelex/system/telemetry/otel_factory.py+++ b/pipelex/system/telemetry/otel_factory.py@@ -293,13 +293,12 @@ class OtelFactory:# Add Pipelex PostHog Exporter if client is provided (mandatory for gateway)if pipelex_posthog_client:- # The Pipelex stream is one shared project for every deployment, so a- # span's own user reaches it only as a digest taken inside the- # gateway-key hash, and the run's groups do not reach it at all.+ # A span's own user is its `distinct_id` and the run's groups ride+ # the capture, as on the operator's stream.pipelex_posthog_exporter = PostHogSpanExporter(posthog_client=pipelex_posthog_client,fallback_distinct_id=pipelex_distinct_id,- run_identity_policy=RunIdentityPolicy.NAMESPACED,+ run_identity_policy=RunIdentityPolicy.DIRECT,redaction_config=pipelex_gateway_redaction_config,)provider.add_span_processor(OTelBatchSpanProcessor(pipelex_posthog_exporter))diff --git a/pipelex/system/telemetry/posthog_span_exporter.py b/pipelex/system/telemetry/posthog_span_exporter.pyindex a03d43223..d5d4527fd 100644--- a/pipelex/system/telemetry/posthog_span_exporter.py+++ b/pipelex/system/telemetry/posthog_span_exporter.py@@ -32,8 +32,8 @@ class PostHogSpanExporter(SpanExporter):and attributes each span to the run that produced it — see:mod:`pipelex.system.telemetry.telemetry_identity`. One class serves bothstreams: the operator's exporter is built with their configured id and the- policy their mode implies, Pipelex's with the gateway-key hash and the- namespacing policy a shared project requires.+ policy their mode implies, Pipelex's with the gateway-key hash as its+ fallback and the direct policy."""def __init__(@@ -52,8 +52,8 @@ class PostHogSpanExporter(SpanExporter):this used to be the identity for every span on the stream.run_identity_policy: What this stream may do with a span's own user.`NONE` on an operator's stream in `anonymous` mode, where the- operator chose not to identify people at all; `NAMESPACED` on- Pipelex's shared project.+ operator chose not to identify people at all; `DIRECT`+ otherwise, Pipelex's stream included.redaction_config: What this stream is allowed to see."""self.posthog_client = posthog_clientdiff --git a/pipelex/system/telemetry/telemetry_identity.py b/pipelex/system/telemetry/telemetry_identity.pyindex b54214079..c6bf34884 100644--- a/pipelex/system/telemetry/telemetry_identity.py+++ b/pipelex/system/telemetry/telemetry_identity.py@@ -13,19 +13,13 @@ needs.**One resolution, applied everywhere.** The exporter resolves per span, thetracker per event, and both go through `TelemetryIdentity` so they cannot drift.What a stream may do with a run's identity is its `RunIdentityPolicy`, and there-are three — the enum below says which stream carries which, and why:+are two — the enum below says which stream carries which, and why:1. `NONE` — the capture is anonymous whatever the run says, and the caller marksit so. No user, no fallback, no groups.2. `DIRECT` — the run's own `user_id` is the `distinct_id`, spelled as theoperator's own backend already knows it, and the run's groups ride thecapture.-3. `NAMESPACED` — the run's `user_id` is folded one way into the stream's own- id. The capture stays per-run, but it is never a raw caller value and can- never collide with another deployment's. The run's own groups do not travel;- the stream's own id travels in their place, as the deployment group, so a- capture that names a caller is still counted under the deployment that- produced it.**A run that names nobody** reports under the stream's configured fallback, andso does an event that belongs to no run at all — a CLI command, a dry-run sweep.@@ -52,7 +46,6 @@ from pipelex.system.storage_scope import DRY_RUN_USER_ID, LOCAL_USER_ID, SINGLE_from pipelex.system.telemetry.otel_constants import LangfuseSpanAttr, PipelexSpanAttrfrom pipelex.system.telemetry.telemetry_config import PostHogModefrom pipelex.tools.log.log import log-from pipelex.tools.misc.hash_utils import hash_sha256from pipelex.tools.misc.json_utils import pure_json_strif TYPE_CHECKING:@@ -68,8 +61,7 @@ if TYPE_CHECKING:# user model states, are true statements rather than identities a backend can# tell apart. `LOCAL_USER_ID` is the literal "local", the same# string on every machine on earth: honoured as a `distinct_id` it would collapse-# every Pipelex user's local runs onto one person, and on the Pipelex stream it-# would replace a per-deployment identity with a single global one. `DRY_RUN_USER_ID`+# every Pipelex user's local runs onto one person. `DRY_RUN_USER_ID`# says outright that there is no caller. `SINGLE_TENANT_USER_ID` is what# pipelex-api states for every run on a deployment configured with no users, and# is as global as `"local"`.@@ -81,49 +73,19 @@ if TYPE_CHECKING:_NON_DISTINGUISHING_RUN_USER_IDS = frozenset({LOCAL_USER_ID, DRY_RUN_USER_ID, SINGLE_TENANT_USER_ID})-# The group type Pipelex's own stream carries its deployment under.-#-# `NAMESPACED` folds the gateway-key hash into the `distinct_id`, which is what-# makes that id per-caller and unable to collide across deployments — and which-# consumes the hash, because the fold is one-way. Nothing else on the capture-# would carry it back: the identity is deliberately kept out of every property,-# and the Pipelex project is one shared project that cannot partition itself by-# deployment. So the hash travels as a group, the one facet PostHog can still-# aggregate on after the fact, and the stream stays what it is documented to be-# — tied to the Gateway API key that produced it.-PIPELEX_DEPLOYMENT_GROUP_TYPE = "deployment"--class RunIdentityPolicy(StrEnum):"""What one stream may do with a run's identity.- Two streams, three policies, because they do not report to the same kind of- audience.-- The operator's own stream reports into the operator's own PostHog project. A- caller's `user_id` there is a value they already hold and a group key is- their own vocabulary, so `DIRECT` sends both as they are — and `anonymous`- mode turns the whole thing off with `NONE`, because an operator who chose to- identify nobody chose it for their users too.-- Pipelex's stream is a SHARED project: every deployment of this version- reports into the one key the remote config names. A raw `user_id` there is- wrong twice over — two customers whose callers are both `user-42` would- merge into a single person, and a caller id that is frequently an e-mail- address would leave the deployment that holds it. The host's group keys are- its own business vocabulary and PostHog's group types are project-global, so- they do not belong there either. Hence `NAMESPACED`: Pipelex still tells two- callers of one deployment apart, but only through a one-way digest taken- inside that deployment's gateway-key hash, and the host's own groups stay- home. The deployment itself does travel, as the one group this stream sets:- the fold consumes the gateway-key hash, and without a facet carrying it back- Pipelex could no longer count a deployment's usage under the key that paid- for it.+ A run's `user_id` is the host's own identifier for its caller, and its+ groups are the host's own entities, so a stream that identifies people+ sends both as they are: `DIRECT`. Pipelex's stream always does. The+ operator's stream does unless the operator set `anonymous` mode, which turns+ the whole thing off with `NONE`, because an operator who chose to identify+ nobody chose it for their users too."""NONE = "none"DIRECT = "direct"- NAMESPACED = "namespaced"@classmethoddef make_for_operator_stream(cls, *, posthog_mode: PostHogMode) -> "RunIdentityPolicy":@@ -137,32 +99,6 @@ class RunIdentityPolicy(StrEnum):return cls.DIRECT if posthog_mode.is_identified else cls.NONE-def _make_namespaced_distinct_id(*, namespace: str, run_user_id: str) -> str:- """Fold a run's user into one deployment's namespace, one way.-- The result tells two callers of the same deployment apart, and can never- equal another deployment's id for a caller of the same name, because the- namespace is inside the digest rather than beside it. It is not reversible,- so a caller id that happens to be an e-mail address never leaves the- deployment that holds it as a value anyone can read.-- It is a pseudonym and not a secret, and the difference is worth stating: the- namespace is itself sent in the clear, as the `distinct_id` of every capture- that names nobody, so a party who already holds a candidate caller id can- fold it the same way and see whether the result is present. Making a guess- useless would take a key that never reaches the stream — the raw gateway key- rather than its hash — which is a wider change than this function.-- Args:- namespace: The stream's own id for this deployment — the gateway-key hash.- run_user_id: The run's user, already known to distinguish a caller.-- Returns:- A 16-character digest, the same shape as the namespace itself.- """- return hash_sha256(data=f"{namespace}:{run_user_id}", length=16)--def make_run_identity_span_attributes(*, run_metadata: "RunMetadata", is_langfuse_enabled: bool) -> dict[str, str]:"""Build the span attributes that carry a run's identity to every exporter.@@ -211,8 +147,7 @@ class TelemetryIdentity(BaseModel):# The entities the capture belongs to, forwarded through the backend's own# groups facet. Empty on an anonymous capture, which has no person for a- # group to qualify. On a `NAMESPACED` stream it holds exactly one entry — the- # deployment — and never a host's own vocabulary.+ # group to qualify.groups: dict[str, str] = Field(default_factory=dict)@property@@ -234,30 +169,12 @@ class TelemetryIdentity(BaseModel):fallback_distinct_id: str | None,run_identity_policy: RunIdentityPolicy,) -> "TelemetryIdentity":- """The one resolution, stated once — see this module's docstring for the three cases."""+ """The one resolution, stated once — see this module's docstring for the two cases."""if run_identity_policy is RunIdentityPolicy.NONE:return cls.make_anonymous()distinguishing_user_id = run_user_id if run_user_id and run_user_id not in _NON_DISTINGUISHING_RUN_USER_IDS else None- if run_identity_policy is RunIdentityPolicy.NAMESPACED:- if not fallback_distinct_id:- # No namespace to fold into means no id that is safe to send on a- # shared project, so the capture goes out with none rather than- # under a raw caller value.- return cls.make_anonymous()- # The deployment travels as a group whether or not the run names a- # caller, so one query counts a deployment's whole usage — the runs- # that name somebody, under their digests, and the runs that name- # nobody, under the hash itself.- deployment_groups = {PIPELEX_DEPLOYMENT_GROUP_TYPE: fallback_distinct_id}- if distinguishing_user_id:- return cls(- distinct_id=_make_namespaced_distinct_id(namespace=fallback_distinct_id, run_user_id=distinguishing_user_id),- groups=deployment_groups,- )- return cls(distinct_id=fallback_distinct_id, groups=deployment_groups)-resolved_distinct_id = distinguishing_user_id or fallback_distinct_idif not resolved_distinct_id:return cls.make_anonymous()diff --git a/pipelex/system/telemetry/telemetry_manager.py b/pipelex/system/telemetry/telemetry_manager.pyindex 857d3465a..3d895c7bf 100644--- a/pipelex/system/telemetry/telemetry_manager.py+++ b/pipelex/system/telemetry/telemetry_manager.py@@ -153,7 +153,7 @@ class TelemetryManager(TelemetryManagerAbstract):pipelex_identity=TelemetryIdentity.make_from_run_metadata(run_metadata=None,fallback_distinct_id=self._pipelex_distinct_id,- run_identity_policy=RunIdentityPolicy.NAMESPACED,+ run_identity_policy=RunIdentityPolicy.DIRECT,),)@@ -491,12 +491,11 @@ class TelemetryManager(TelemetryManagerAbstract):log.verbose(f"Tracked anonymous event '{event_name}' with properties: {capture_properties}")def _track_to_pipelex(self, event_name: str, *, properties: dict[str, Any], run_metadata: "RunMetadata | None" = None):- """Track event to Pipelex's PostHog (always identified, always namespaced).+ """Track event to Pipelex's PostHog (always identified, always direct).- The stream has no mode to turn identification off, but it is a shared- project: a run's own user reaches it only as a digest taken inside the- gateway-key hash, and the hash itself is what an event with no run- reports under. The run's groups do not travel here at all.+ The stream has no mode to turn identification off. A run's own+ `user_id` is the `distinct_id` and its groups ride the capture; an event+ that names nobody reports under the stream's fallback."""if not self.pipelex_posthog_client or not self._pipelex_distinct_id:log.error("Could not track event to Pipelex telemetry because pipelex_posthog_client or _pipelex_distinct_id is not set")@@ -504,7 +503,7 @@ class TelemetryManager(TelemetryManagerAbstract):identity = TelemetryIdentity.make_from_run_metadata(run_metadata=run_metadata,fallback_distinct_id=self._pipelex_distinct_id,- run_identity_policy=RunIdentityPolicy.NAMESPACED,+ run_identity_policy=RunIdentityPolicy.DIRECT,)self.pipelex_posthog_client.capture(event_name,@@ -677,7 +676,7 @@ class TelemetryManager(TelemetryManagerAbstract):pipelex_identity = TelemetryIdentity.make_from_run_metadata(run_metadata=run_metadata,fallback_distinct_id=self._pipelex_distinct_id,- run_identity_policy=RunIdentityPolicy.NAMESPACED,+ run_identity_policy=RunIdentityPolicy.DIRECT,)if pipelex_identity.distinct_id:self.pipelex_posthog_client.capture(diff --git a/tests/unit/pipelex/system/telemetry/test_exception_capture_identity.py b/tests/unit/pipelex/system/telemetry/test_exception_capture_identity.pyindex c5b49da72..6cfffef5b 100644--- a/tests/unit/pipelex/system/telemetry/test_exception_capture_identity.py+++ b/tests/unit/pipelex/system/telemetry/test_exception_capture_identity.py@@ -27,7 +27,6 @@ from pytest_mock import MockerFixturefrom pipelex.system.telemetry.exception_capture import DualClientExceptionCapturefrom pipelex.system.telemetry.otel_constants import PostHogAttrfrom pipelex.system.telemetry.telemetry_config import PostHogConfig, PostHogMode, TelemetryConfig-from pipelex.system.telemetry.telemetry_identity import PIPELEX_DEPLOYMENT_GROUP_TYPEfrom pipelex.system.telemetry.telemetry_manager import TelemetryManager@@ -100,8 +99,8 @@ class TestExceptionCaptureIdentity:assert custom_client.capture_exception.call_args.kwargs["distinct_id"] == "configured-id"- def test_the_pipelex_stream_captures_an_exception_under_its_deployment(self, mocker: MockerFixture) -> None:- """No run means no fold, so the gateway hash is both the person and the group."""+ def test_the_pipelex_stream_captures_an_exception_under_its_fallback(self, mocker: MockerFixture) -> None:+ """No run means no caller and no groups, so the stream's fallback is the person."""capture, _, pipelex_client = _make_capture(mocker=mocker,mode=PostHogMode.IDENTIFIED,@@ -113,7 +112,7 @@ class TestExceptionCaptureIdentity:capture_kwargs = pipelex_client.capture_exception.call_args.kwargsassert capture_kwargs["distinct_id"] == "gateway-hash"- assert capture_kwargs["groups"] == {PIPELEX_DEPLOYMENT_GROUP_TYPE: "gateway-hash"}+ assert capture_kwargs["groups"] is Nonedef test_the_pipelex_stream_is_unaffected_by_the_operators_anonymous_mode(self, mocker: MockerFixture) -> None:"""The two streams answer to different settings, and anonymity on one is not anonymity on the other."""diff --git a/tests/unit/pipelex/system/telemetry/test_exception_capture_sdk_contract.py b/tests/unit/pipelex/system/telemetry/test_exception_capture_sdk_contract.pyindex 7585d48e0..afbb5d810 100644--- a/tests/unit/pipelex/system/telemetry/test_exception_capture_sdk_contract.py+++ b/tests/unit/pipelex/system/telemetry/test_exception_capture_sdk_contract.py@@ -108,8 +108,7 @@ class TestExceptionCaptureSdkContract:PostHog stamps, and the stamp is still on it when the second streamasks. Left in place, the stream that goes second — always Pipelex's, ona machine that also reports to an operator's own project — returns- before building an event, and the deployment group it resolved is- recorded nowhere.+ before building an event, and the crash is recorded nowhere on it."""custom_client, custom_capture = _make_stubbed_client(mocker=mocker, api_key="phc_custom")pipelex_client, pipelex_capture = _make_stubbed_client(mocker=mocker, api_key="phc_pipelex")diff --git a/tests/unit/pipelex/system/telemetry/test_posthog_span_exporter.py b/tests/unit/pipelex/system/telemetry/test_posthog_span_exporter.pyindex 8a57ad7fa..70e2e8818 100644--- a/tests/unit/pipelex/system/telemetry/test_posthog_span_exporter.py+++ b/tests/unit/pipelex/system/telemetry/test_posthog_span_exporter.py@@ -19,8 +19,7 @@ from pytest_mock import MockerFixturefrom pipelex.system.telemetry.otel_constants import PipelexSpanAttr, PostHogAttr, PostHogEvent, SpanCategoryfrom pipelex.system.telemetry.posthog_span_exporter import PostHogSpanExporterfrom pipelex.system.telemetry.telemetry_config import TelemetryRedactionConfig-from pipelex.system.telemetry.telemetry_identity import PIPELEX_DEPLOYMENT_GROUP_TYPE, RunIdentityPolicy-from pipelex.tools.misc.hash_utils import hash_sha256+from pipelex.system.telemetry.telemetry_identity import RunIdentityPolicy_FULL_CAPTURE = TelemetryRedactionConfig(redact_content=False,@@ -167,34 +166,6 @@ class TestPostHogSpanExporter:assert "groups" not in capture_kwargsassert capture_kwargs["properties"][PostHogAttr.PROCESS_PERSON_PROFILE] is False- def test_a_namespaced_stream_digests_the_user_and_keeps_only_its_own_group(self, mocker: MockerFixture) -> None:- """Pipelex's own exporter writes into one shared project across every deployment.-- The host's group keys are its own business vocabulary and PostHog group- types are project-global, so none of them travel. The deployment does:- the digest consumes the gateway hash, and the group facet is what carries- it back.- """- client = mocker.MagicMock(spec=Posthog)- exporter = _make_exporter(client=client, fallback_distinct_id="deployment-hash", run_identity_policy=RunIdentityPolicy.NAMESPACED)-- exporter.export(- [- _make_span(- span_category=SpanCategory.PIPE,- attributes={- PipelexSpanAttr.RUN_USER_ID: "user-42",- PipelexSpanAttr.RUN_ANALYTICS_GROUPS: '{"tenant": "acme-corp"}',- },- )- ]- )-- capture_kwargs = client.capture.call_args.kwargs- assert capture_kwargs["distinct_id"] == hash_sha256(data="deployment-hash:user-42", length=16)- assert capture_kwargs["groups"] == {PIPELEX_DEPLOYMENT_GROUP_TYPE: "deployment-hash"}- assert "acme-corp" not in repr(capture_kwargs)-def test_neither_the_user_nor_the_groups_appear_in_the_properties(self, mocker: MockerFixture) -> None:client = mocker.MagicMock(spec=Posthog)exporter = _make_exporter(client=client, fallback_distinct_id=None, run_identity_policy=RunIdentityPolicy.DIRECT)diff --git a/tests/unit/pipelex/system/telemetry/test_telemetry_identity.py b/tests/unit/pipelex/system/telemetry/test_telemetry_identity.pyindex 6ec34fee0..4de91cfde 100644--- a/tests/unit/pipelex/system/telemetry/test_telemetry_identity.py+++ b/tests/unit/pipelex/system/telemetry/test_telemetry_identity.py@@ -7,13 +7,8 @@ contract, and it is a matrix of `RunIdentityPolicy` against what the run names:- `NONE` captures anonymously whatever the run says, which is what an operatorasking to identify nobody asked for.- `DIRECT` takes the run's own user, or the stream's fallback when the run names- nobody, and carries the run's groups either way.-- `NAMESPACED` takes a one-way digest of the run's user inside the stream's own- id, so a shared project can tell two callers of one deployment apart without- ever receiving a raw caller value. It carries none of the host's groups, and- exactly one of its own: the deployment, which the digest would otherwise- consume, and without which a capture could no longer be counted under the- gateway key that produced it.+ nobody, and carries the run's groups either way. Pipelex's own stream is+ always `DIRECT`.The span-attribute half is the round trip: what a span site writes is what theexporter reads back, and a span that carries nothing resolves to the fallback —@@ -27,12 +22,10 @@ from pipelex.system.storage_scope import DRY_RUN_USER_ID, LOCAL_USER_ID, SINGLE_from pipelex.system.telemetry.otel_constants import LangfuseSpanAttr, PipelexSpanAttrfrom pipelex.system.telemetry.telemetry_config import PostHogModefrom pipelex.system.telemetry.telemetry_identity import (- PIPELEX_DEPLOYMENT_GROUP_TYPE,RunIdentityPolicy,TelemetryIdentity,make_run_identity_span_attributes,)-from pipelex.tools.misc.hash_utils import hash_sha256def _run_metadata(*, user_id: str = "user-42", analytics_groups: dict[str, str] | None = None) -> RunMetadata:@@ -157,159 +150,6 @@ class TestTelemetryIdentity:def test_an_operator_stream_reads_its_policy_off_the_mode(self, posthog_mode: PostHogMode, expected_policy: RunIdentityPolicy) -> None:assert RunIdentityPolicy.make_for_operator_stream(posthog_mode=posthog_mode) is expected_policy- # ------------------------------------------------------------- NAMESPACED-- def test_a_namespaced_stream_sends_a_digest_and_never_the_raw_user(self) -> None:- """The Pipelex stream is one shared project, so a raw caller value must never reach it."""- identity = TelemetryIdentity.make_from_run_metadata(- run_metadata=_run_metadata(user_id="[email protected]"),- fallback_distinct_id="gateway-hash",- run_identity_policy=RunIdentityPolicy.NAMESPACED,- )-- assert identity.distinct_id == hash_sha256(data="gateway-hash:[email protected]", length=16)- assert identity.distinct_id != "[email protected]"- assert identity.distinct_id != "gateway-hash"- assert "alice" not in str(identity.distinct_id)-- def test_two_deployments_never_collide_on_the_same_caller_name(self) -> None:- """The whole point: `user-42` at one customer is not `user-42` at another."""- one = TelemetryIdentity.make_from_run_metadata(- run_metadata=_run_metadata(),- fallback_distinct_id="gateway-hash-one",- run_identity_policy=RunIdentityPolicy.NAMESPACED,- )- other = TelemetryIdentity.make_from_run_metadata(- run_metadata=_run_metadata(),- fallback_distinct_id="gateway-hash-other",- run_identity_policy=RunIdentityPolicy.NAMESPACED,- )-- assert one.distinct_id != other.distinct_id-- def test_two_callers_of_one_deployment_stay_apart(self) -> None:- """Namespacing must not cost the per-run attribution it exists to make safe."""- one = TelemetryIdentity.make_from_run_metadata(- run_metadata=_run_metadata(user_id="user-42"),- fallback_distinct_id="gateway-hash",- run_identity_policy=RunIdentityPolicy.NAMESPACED,- )- other = TelemetryIdentity.make_from_run_metadata(- run_metadata=_run_metadata(user_id="user-43"),- fallback_distinct_id="gateway-hash",- run_identity_policy=RunIdentityPolicy.NAMESPACED,- )-- assert one.distinct_id != other.distinct_id-- def test_the_same_caller_resolves_to_the_same_digest_every_time(self) -> None:- """A digest that moved between two captures would make every run a new person."""- first = TelemetryIdentity.make_from_run_metadata(- run_metadata=_run_metadata(),- fallback_distinct_id="gateway-hash",- run_identity_policy=RunIdentityPolicy.NAMESPACED,- )- second = TelemetryIdentity.make_from_span_attributes(- attributes={PipelexSpanAttr.RUN_USER_ID: "user-42"},- fallback_distinct_id="gateway-hash",- run_identity_policy=RunIdentityPolicy.NAMESPACED,- )-- assert first.distinct_id == second.distinct_id-- def test_a_namespaced_stream_never_forwards_the_runs_groups(self) -> None:- """Group keys are the host's own business vocabulary and PostHog group types are project-global."""- identity = TelemetryIdentity.make_from_run_metadata(- run_metadata=_run_metadata(analytics_groups={"tenant": "acme-corp", "plan_tier": "enterprise"}),- fallback_distinct_id="gateway-hash",- run_identity_policy=RunIdentityPolicy.NAMESPACED,- )-- assert "tenant" not in identity.groups- assert "plan_tier" not in identity.groups- assert identity.groups == {PIPELEX_DEPLOYMENT_GROUP_TYPE: "gateway-hash"}-- def test_a_namespaced_stream_keeps_the_deployment_countable_under_a_caller_digest(self) -> None:- """The fold consumes the gateway hash, so the group facet is the only thing that carries it back.-- Without this the shared project can tell two callers apart and can no- longer say whose key paid for either, which is what the stream is for.- """- identity = TelemetryIdentity.make_from_run_metadata(- run_metadata=_run_metadata(),- fallback_distinct_id="gateway-hash",- run_identity_policy=RunIdentityPolicy.NAMESPACED,- )-- assert identity.distinct_id != "gateway-hash"- assert identity.distinct_id != "user-42"- assert identity.groups == {PIPELEX_DEPLOYMENT_GROUP_TYPE: "gateway-hash"}-- def test_one_deployment_counts_the_same_whether_a_run_names_a_caller_or_not(self) -> None:- """Two runs, two different persons, one group — which is what makes the rollup a single query."""- named = TelemetryIdentity.make_from_run_metadata(- run_metadata=_run_metadata(),- fallback_distinct_id="gateway-hash",- run_identity_policy=RunIdentityPolicy.NAMESPACED,- )- nameless = TelemetryIdentity.make_from_run_metadata(- run_metadata=_run_metadata(user_id=LOCAL_USER_ID),- fallback_distinct_id="gateway-hash",- run_identity_policy=RunIdentityPolicy.NAMESPACED,- )-- assert named.distinct_id != nameless.distinct_id- assert named.groups == nameless.groups == {PIPELEX_DEPLOYMENT_GROUP_TYPE: "gateway-hash"}-- def test_two_deployments_never_share_a_deployment_group(self) -> None:- first = TelemetryIdentity.make_from_run_metadata(- run_metadata=_run_metadata(),- fallback_distinct_id="gateway-hash-a",- run_identity_policy=RunIdentityPolicy.NAMESPACED,- )- second = TelemetryIdentity.make_from_run_metadata(- run_metadata=_run_metadata(),- fallback_distinct_id="gateway-hash-b",- run_identity_policy=RunIdentityPolicy.NAMESPACED,- )-- assert first.groups != second.groups-- def test_a_namespaced_span_never_forwards_the_spans_groups(self) -> None:- identity = TelemetryIdentity.make_from_span_attributes(- attributes={- PipelexSpanAttr.RUN_USER_ID: "user-42",- PipelexSpanAttr.RUN_ANALYTICS_GROUPS: '{"tenant": "acme-corp"}',- },- fallback_distinct_id="gateway-hash",- run_identity_policy=RunIdentityPolicy.NAMESPACED,- )-- assert "tenant" not in identity.groups- assert identity.groups == {PIPELEX_DEPLOYMENT_GROUP_TYPE: "gateway-hash"}-- def test_a_namespaced_stream_reports_the_namespace_itself_when_the_run_names_nobody(self) -> None:- """Which is the deployment-level identity everything reported under before per-run attribution."""- identity = TelemetryIdentity.make_from_run_metadata(- run_metadata=None,- fallback_distinct_id="gateway-hash",- run_identity_policy=RunIdentityPolicy.NAMESPACED,- )-- assert identity.distinct_id == "gateway-hash"- assert identity.groups == {PIPELEX_DEPLOYMENT_GROUP_TYPE: "gateway-hash"}-- def test_a_namespaced_stream_with_no_namespace_is_anonymous_not_raw(self) -> None:- """No namespace to fold into means no id that is safe to send on a shared project."""- identity = TelemetryIdentity.make_from_run_metadata(- run_metadata=_run_metadata(),- fallback_distinct_id=None,- run_identity_policy=RunIdentityPolicy.NAMESPACED,- )-- assert identity.is_anonymous is True- assert identity.distinct_id != "user-42"-# ------------------------------------------ user ids that distinguish nobody@pytest.mark.parametrize("placeholder", [LOCAL_USER_ID, DRY_RUN_USER_ID, SINGLE_TENANT_USER_ID])@@ -352,7 +192,7 @@ class TestTelemetryIdentity:identity = TelemetryIdentity.make_from_span_attributes(attributes={PipelexSpanAttr.RUN_USER_ID: placeholder},fallback_distinct_id="gateway-hash",- run_identity_policy=RunIdentityPolicy.NAMESPACED,+ run_identity_policy=RunIdentityPolicy.DIRECT,)assert identity.distinct_id == "gateway-hash"diff --git a/tests/unit/pipelex/system/telemetry/test_telemetry_manager_identity.py b/tests/unit/pipelex/system/telemetry/test_telemetry_manager_identity.pyindex 0867555d2..de5ce2796 100644--- a/tests/unit/pipelex/system/telemetry/test_telemetry_manager_identity.py+++ b/tests/unit/pipelex/system/telemetry/test_telemetry_manager_identity.py@@ -22,9 +22,7 @@ from pipelex.system.job_metadata import RunMetadatafrom pipelex.system.telemetry.events import EventNamefrom pipelex.system.telemetry.otel_constants import PostHogAttrfrom pipelex.system.telemetry.telemetry_config import PostHogConfig, PostHogMode, PostHogTracingConfig, TelemetryConfig-from pipelex.system.telemetry.telemetry_identity import PIPELEX_DEPLOYMENT_GROUP_TYPEfrom pipelex.system.telemetry.telemetry_manager import TelemetryManager-from pipelex.tools.misc.hash_utils import hash_sha256def _run_metadata(*, user_id: str = "user-42", analytics_groups: dict[str, str] | None = None) -> RunMetadata:@@ -107,13 +105,8 @@ class TestTelemetryManagerIdentity:custom_client.capture.assert_not_called()- def test_the_pipelex_stream_digests_the_run_inside_the_gateway_hash(self, mocker: MockerFixture) -> None:- """One shared project serves every deployment, so a raw caller value must never reach it.-- Two customers whose callers are both `user-42` would otherwise merge into- a single person there, and a caller id that is an e-mail address would- leave the deployment holding it.- """+ def test_the_pipelex_stream_sends_the_runs_user_and_groups_as_they_are(self, mocker: MockerFixture) -> None:+ """The run's `user_id` is the `distinct_id` and its groups ride the capture, unchanged."""manager, _, pipelex_client = _make_manager(mocker=mocker,mode=PostHogMode.OFF,@@ -128,24 +121,8 @@ class TestTelemetryManagerIdentity:)capture_kwargs = pipelex_client.capture.call_args.kwargs- assert capture_kwargs["distinct_id"] == hash_sha256(data="gateway-hash:user-42", length=16)- assert capture_kwargs["groups"] == {PIPELEX_DEPLOYMENT_GROUP_TYPE: "gateway-hash"}- assert "org_acme" not in repr(capture_kwargs)-- def test_two_deployments_never_merge_on_the_same_caller_name(self, mocker: MockerFixture) -> None:- captured_ids: list[str] = []- for gateway_hash in ("gateway-hash-one", "gateway-hash-other"):- manager, _, pipelex_client = _make_manager(- mocker=mocker,- mode=PostHogMode.OFF,- configured_user_id=None,- pipelex_distinct_id=gateway_hash,- )- assert pipelex_client is not None- manager.track_event(EventName.PIPE_RUN, run_metadata=_run_metadata())- captured_ids.append(pipelex_client.capture.call_args.kwargs["distinct_id"])-- assert captured_ids[0] != captured_ids[1]+ assert capture_kwargs["distinct_id"] == "user-42"+ assert capture_kwargs["groups"] == {"organization": "org_acme"}def test_an_event_with_no_run_falls_back_to_the_gateway_hash(self, mocker: MockerFixture) -> None:manager, _, pipelex_client = _make_manager(@@ -231,7 +208,7 @@ class TestTelemetryManagerIdentity:assert "distinct_id" not in capture_kwargsassert capture_kwargs["properties"][PostHogAttr.PROCESS_PERSON_PROFILE] is False- def test_the_pipelex_stream_trace_start_is_namespaced_like_its_spans(self, mocker: MockerFixture) -> None:+ def test_the_pipelex_stream_trace_start_is_attributed_like_its_spans(self, mocker: MockerFixture) -> None:"""The trace's first event has to land on the same person its spans will."""manager, _, pipelex_client = _make_manager(mocker=mocker,@@ -249,8 +226,8 @@ class TestTelemetryManagerIdentity:)capture_kwargs = pipelex_client.capture.call_args.kwargs- assert capture_kwargs["distinct_id"] == hash_sha256(data="gateway-hash:user-42", length=16)- assert capture_kwargs["groups"] == {PIPELEX_DEPLOYMENT_GROUP_TYPE: "gateway-hash"}+ assert capture_kwargs["distinct_id"] == "user-42"+ assert capture_kwargs["groups"] == {"organization": "org_acme"}def test_the_identity_never_becomes_a_property(self, mocker: MockerFixture) -> None:manager, custom_client, _ = _make_manager(mocker=mocker, mode=PostHogMode.IDENTIFIED, configured_user_id="configured-id")