🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
Topics PoC
Turn selected v4 traces into facet summaries, embeddings, discovered topic maps,
and assignments to an existing map. Open /project/<projectId>/topics on the
current Langfuse instance (locally, normally http://localhost:3000).
Facets and selection rules
Topics reuses the evaluator and evaluation-rule tables:
evaluatorswith typeFACETstore facet identity, name and description.isBuiltInmarks the immutable Intent, Outcome and Issues presets. Custom facets can have their own instructions.evaluator_versionsstores immutable prompts keyed by evaluator and numeric version. Saving an unchanged custom prompt does not create a version.evaluation_rulesstores saved names and observation filters;evaluation_rule_evaluator_assignmentsattaches facets. Editing a rule does not create prompt versions. Random/latest ordering and sample limits belong to each manual processing request, not the saved rule.- At trigger, the request freezes the rule ID, resolved trace IDs, selected prompt versions and runtime summary/embedding configuration. A hash identifies the original request, including its selection criteria. Retries never re-evaluate a rule.
Postgres uses normal Prisma migrations; topic_clustering_runs is the only new
Topics table. Configure Topics upserts the per-project idle time in
tracing_readiness_settings; ingestion uses it to schedule trace readiness.
Idle time is per project, not per rule: see
tracing_readiness_settings in worker/src/features/traceBatching/README.md.
Setup
Use the normal local Postgres, ClickHouse, Redis, web, and worker stack. Apply the
repository's Postgres and ClickHouse migrations and regenerate/build shared before
starting the worker. Set LANGFUSE_TOPICS_ENABLED=true on both
web and worker. It defaults to false: Topics routes, effective session flags,
and queues are disabled. Trace and project deletion always clean Topics data.
Set LANGFUSE_TOPICS_ENABLED_PROJECT_IDS=project-a,project-b on both services to
allow processing for those project IDs. An unset or empty list admits no projects;
IDs are comma-separated and whitespace is trimmed. Clear the list to pause
processing while retaining Topics access.
Topics environment variables and the trace-batch-to-Topics integration are
internal PoC controls, not a supported self-hosting configuration surface.
They are omitted from .env.prod.example.
LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES is a separate general flag and does not
enable Topics; use the dedicated Topics controls for this experiment.
UI visibility and read/configuration access additionally use the langfuseTopics
feature flag and project permissions. Trigger/retry and both worker processors
check deployment enablement and the allowlist; rejected queue jobs fail without retrying or running pipeline
work. Apply changes by restarting web and worker; this does not cancel work that
is already running. Web and worker use the same databases and Redis. Topics does
not write to object storage or require a shared filesystem; ingestion storage is unchanged.
Build the numerical addon from the repository root (the normal worker build and dev commands also build it):
pnpm --filter @langfuse/native run build
Topics runs on the project's own LLM connections (the same connections evals
and the playground use). Each project stores one topic_configs row with
three slots, each a connection plus a model ID: facet summaries, embeddings (plus
dimensions), and topic clustering (the model that names and describes topics). Set them on the Topics page under Models.
Processing is unavailable until all three are set. Any connection whose save-time
test call succeeds can serve summaries and clustering, including Anthropic.
Embeddings need OpenAI (including OpenAI-compatible base URLs), Azure OpenAI,
Amazon Bedrock, Google AI Studio or Vertex AI; Anthropic has no embeddings API,
so its embedding test call fails. Summary and clustering calls normally go through
generateLLMText; embeddings go through embedLLMText. Both retry 429 and 5xx
responses twice with backoff. Cached Bedrock GPT-5.6+ summaries use a single
InvokeModel request with explicit cache breakpoints. On Cloud, Bedrock and Vertex
connections need explicit credentials, except operator-provisioned Bedrock
connections in the runtime-credential allowlist; self-hosted deployments can use
the default credential chain.
Saving the settings makes one real call per configured slot (structured output
for summaries and clustering, one embedding whose length must equal the
configured dimensions) and rejects the save if any call fails. Saves record the
user in created_by / updated_by and write topicConfig (and, for the rule,
job) audit log entries.
The embedding model and dimensions cannot change once the project has embedded
summaries, so existing vectors stay comparable. Summary and clustering model changes apply
to new work only; an execution frozen on another summary or embedding model
fails and must be started again. Deleting a connection that a slot uses turns
automatic processing off and records why. Usage is retained for any model, but
cost estimates are only available for the model IDs priced in
models.ts.
Summary and naming calls emit one generation each to the configured internal
AI-features project (LANGFUSE_AI_FEATURES_PROJECT_ID) only when the source
project is listed in LANGFUSE_TOPICS_TRACING_ENABLED_PROJECT_IDS and its
organization enables AI features and AI analytics. The tracing allowlist is
comma-separated, trims whitespace, and defaults to empty (tracing off). It is
independent of the processing allowlist; projects can run Topics without tracing.
Multi-facet summaries share one generation. Both connection-based SDK calls and
cached Bedrock InvokeModel calls record inputs, outputs, models, usage, and errors;
embeddings are untraced.
Trace costs use the destination project's model catalog. Unmatched connection
model IDs retain their usage but have no trace cost, independently of Topics'
own estimates in models.ts.
Traces use langfuse-topics. Topics excludes all langfuse-prefixed environments
and llm-as-a-judge through the shared eval exclusion predicate, preventing loops
from these internal traces. Manual selection checks matching root observations
within the requested time range; workers also check the source environment they
load. If an OpenRouter connection broadcasts requests back
to a Topics-enabled project, use a dedicated Topics API key excluded from that
Broadcast destination. OpenRouter's separate public traces do not retain the
reserved internal environment and can otherwise trigger more Topics calls.
Numerical fitting uses the worker's existing Node runtime and compiled
@langfuse/native addon, with no extra runtime or service.
Summary and assignment records use trace_id as their source when present;
session_id can also record that trace's parent session. With an empty trace_id,
session_id identifies the processed session. At least one ID is required.
The current pipeline processes traces only. unit_start_time (unitStartTime in
TypeScript) is the first observation's start time for the processed source. Summaries set
trigger_type to automatic (trace-batch) or manual (manual processing); rows written before that split hold manual_poc. Each clustering attempt records its own started_at. Failed updates start a fresh
attempt while keeping the previous published map available.
Summaries snapshot source environment and trace name; assignments copy that snapshot, including empty environment strings. Reprocessing refreshes metadata even when text is reused; session results have no trace name.
For Bedrock summary models, use us.openai.gpt-6-luna for US routing or
global.openai.gpt-6-luna for global routing; GPT-6 Luna has no EU geographic
profile. Both support Bedrock Converse tool-based structured output; OpenAI
models on Bedrock are called with reasoning effort none.
Summary cost estimates include both GPT-6 Luna profiles and the existing
us.openai.gpt-5.6-luna profile. See the
GPT-6 Luna model card
for supported source regions and pricing. The
recommended Bedrock clustering model is us.openai.gpt-5.6-terra. The AWS model cards for GPT-5.6 Luna
and GPT-5.6 Terra
currently list us. geographic inference profiles in commercial Regions, but
no eu. profiles; these profiles route within the US geography, including when
invoked from eu-west-1.
On Bedrock, Cohere Embed v4 uses eu.cohere.embed-v4:0 for EU routing or
us.cohere.embed-v4:0 for US routing. Avoid the global profile when geographic
residency matters. Every provider is asked for clustering vectors of the
configured size (Cohere clustering input type with truncation disabled, Google
CLUSTERING task type, OpenAI dimensions). Fixed-size models such as text-embedding-ada-002 reject
the size option; a request rejected with HTTP 400 is retried once without it, and
the vector must still have the configured length. Default: 1,024
dimensions; any whole number from 64 to 4,096 is accepted, and a provider that
returns a vector of another length fails the call. Embeddings are batched per trace: each trace's
applicable facets are embedded in one call, in trace-batch and in embedding jobs
(which run up to LANGFUSE_TOPICS_TRACE_CONCURRENCY traces at once). The call's
usage is recorded on the trace's first row; one failure fails that trace's call,
and a retry embeds the traces that are left. Saving the models requires the
embedding model to embed two texts in one call.
Embedding settings are independent of immutable facet versions; changing them
with stored-summary reuse enabled regenerates vectors without summarization inference. This PoC does
not use aiEmbed or change ingestion. It accepts traces already stored in v4
events; legacy-only traces are unsupported.
Drain existing work before deploying model changes; old-model staged jobs cannot resume. Old execution history remains readable. After changing the summary model, start a new Process traces execution; stored-summary reuse cannot reuse results from a different summary model. Existing topic names remain until their definitions change during Update topics. If a provider's content filter blocks naming, the cluster is published with an unnamed placeholder and a warning; its assignments are preserved and other groups and facets continue. Update topics retries placeholder names. Other naming failures still interrupt the update.
Trace-batch connection
After the trace-batch job assembles a transcript, it summarizes that same
transcript for all current facets of an allowlisted project in one model call:
the system prompt lists every facet under a key, asks for each to be answered
independently with short evidence notes, and returns one entry per facet. The
call's usage is recorded on the first facet's row. It does not load the trace
from ClickHouse again. Projects outside
LANGFUSE_TOPICS_ENABLED_PROJECT_IDS are skipped. A finished facet version is
skipped on retry. A failing trace does not stop the batch: every trace is
processed, up to LANGFUSE_TOPICS_TRACE_CONCURRENCY traces at once (default
20; manual processing uses the same limit per queue job), and outcomes are counted once per job (langfuse.topics.trace_outcomes
by outcome and reason). Failed traces are not retried, because a retry would
read the shared batch from ClickHouse again. Only a failed batch read fails the
job. Summaries and embeddings use the same project models as manual
processing. While the read starts, one Postgres query loads the enabled model
settings of every allowlisted project in the batch; facets load once per
project. Projects whose settings are off or incomplete are skipped, and new
traces that arrive meanwhile are not queued. An authentication or billing error
(HTTP 401, 402, 403) turns the project's automatic processing off with the
failing slot as the reason, and the rest of the batch skips that project.
After the write, the batch queues its stored complete summaries on
topics-automation, one job per project and 100 traces; a retried batch reuses
its job IDs. A failed enqueue is logged and does not fail the batch. The job
rereads those summaries and assigns the ones not yet assigned to their facet's
compatible serving map with origin online; BullMQ retries a failed job three
times. Each Topics queue has its own QUEUE_CONSUMER_<QUEUE>_QUEUE_IS_ENABLED
switch (TOPICS, TOPICS_UPDATE, TOPICS_EMBEDDING, TOPICS_AUTOMATION); set
them to false on fleets that only run trace batches.
A facet version without any map starts one automatic Update topics execution
once a recent window contains at least 100 compatible summaries, targeting 80%
of the worker's row and byte limits to leave room for late summaries. The row
target stays at least 100 when the hard limit permits it. Automatic processing
must be on with the embedding model those summaries used. It first checks seven
days, then searches for a smaller recent window using database counts and byte estimates. Metadata-only
counts narrow row-oversized windows before estimating text payloads; byte
estimates are projected before sorting. Searches are throttled per facet and
embedding configuration for 60 seconds. Redis deduplicates deferred checks to
one pending check per facet/model, including a successor to a running check. The entire
selected window uses the same clustering and naming pipeline as a manual update.
It appears in History under the system:topics actor. Active discoveries schedule
a follow-up before dispatch, so a lost queue job or capacity failure is checked
even without another incoming batch. Each capacity failure can create one new
request after the cooldown, starting the search at half the failed window's
duration. Requests keep their selected windows unchanged. If no admissible
window exists, discovery waits for another summary batch; other failures are
resumed from History. An existing map or fit in progress for the version, even an
incompatible one, blocks discovery; replace it with Update topics. Summaries
stored while the first map is being fitted stay unassigned until the next
Update topics.
Required for a local run, in addition to Postgres, ClickHouse, and Redis:
| Variable | Role |
|---|---|
NEXT_PUBLIC_LANGFUSE_CLOUD_REGION |
Must be set or the batch job is discarded. Local dev uses DEV. |
LANGFUSE_TRACE_BATCH_INGESTION_ENABLED |
Track accepted writes. Default off. |
LANGFUSE_TRACE_BATCH_DISPATCHER_ENABLED |
Enqueue idle traces. Default off. |
QUEUE_CONSUMER_TRACE_BATCH_QUEUE_IS_ENABLED |
Register the batch worker. Default off. |
LANGFUSE_TRACE_BATCH_READ_ENABLED |
Allow the ClickHouse read. Default off. |
LANGFUSE_TRACE_BATCH_IDLE_MS |
Idle time before a trace is ready. Unset is 2 minutes on DEV and 10 minutes otherwise. |
LANGFUSE_TOPICS_ENABLED_PROJECT_IDS |
Same allowlist on web and worker. Unset defaults to the demo project. These projects bypass trace-batch sampling at ingestion, so every trace is summarized automatically; set LANGFUSE_TRACE_BATCH_SAMPLING_RATE=0 to run the flow for them only. |
Run the experiment
- Initialize facets and inspect their instructions.
Intent,Sentiment,Outcome, andIssuesare built-in presets with immutable instructions. Create a custom facet for different instructions; a facet is not a list of topic classes. - Choose Process traces and select traces through filters or pasted IDs. The request freezes the selection and selected facet versions. It generates summaries and embeddings, then assigns only this batch to the current compatible map. With no compatible map, completed summaries are Awaiting topics, not outliers. Processing never clusters or renames topics.
- Choose Update topics to fit a map from the latest completed compatible summaries whose source started in the last seven days. Initial discovery and later updates use this same operation. It does not load traces, summarize, or generate embeddings.
- Inspect the resulting summaries, names, representative examples, and outliers. Non-applicable and insufficient-input results remain separate from outliers.
- Process subsequent trace batches as they arrive. Update topics manually when ready to incorporate new evidence into the clustering and names.
Minimum traces for clustering applies only to topic updates. It defaults to 100 applicable summaries per facet, with HDBSCAN minimum cluster size 15 and minimum samples 5. Exploratory mode defaults to 10 / 3 / 2. The minimum can be set to at least 3; changing it does not change cluster-size or density settings. This minimum does not establish cluster quality.
Use the repository seed CLI for synthetic local trace data (pnpm run seed -- list). There is no automatic fixture insertion in this feature.
Current topics and explicit updates
There is no total trace-count cap; explicit lookups use bounded internal batches. Each processing request pins the current compatible published map per facet before summarization, including the absence of a map. Retries keep that choice. New requests can use a map completed while an earlier request was running. The serving map is the completed run with the highest facet version, then newest creation time and ID. A newer configured version without a completed map does not hide the previous map. Pending, running, failed and skipped runs are ineligible. A valid all-outlier map is completed even though it contains no topics.
Current summaries and membership use the last seven days of source start times. Each update freezes that window in its execution input and run configuration; retries keep the same boundaries. Published map cohorts and continuity reads use their run's stored window, even after the current window advances.
Within its window, an update selects the latest summary per trace, then keeps only complete results matching the selected facet version and embedding configuration. The attempt uses that in-memory population for fitting; a failed update selects it again on retry. Non-applicable or incompatible newer summaries never revive older eligible results. A change to embedding dimensions must first be processed explicitly; topic updates do not silently re-embed historical data. The explicit update always attempts a fit subject to its minimum count. There is no conditional-refit heuristic or periodic scheduler in this PoC.
Continuity uses at least 80% reciprocal overlap of trace identities, 10 anchors (3 exploratory), and 50% old-topic coverage. Compatible centroids must be within cosine distance 0.15. Material split/merge branches start new identities. These thresholds require quality calibration; they are not universal guarantees.
Summaries and assignments partition by the month of unit_start_time, with
project, source-start minute and facet version leading their sorting keys.
Summaries replace the same minute/facet/source using processed_at; assignments
retain map and origin in the sorting-key suffix and replace using assigned_at.
Run and origin remain distinct so unpublished attempts and online assignments
cannot remove a published map's initial membership or coordinates. Assignment
time is neither a partition key nor a sorting key.
Payload reads have an explicit source-time window. A metadata-only projection ordered by project, facet and source validates candidates against their latest facet version and processing time, including versions outside that window. Late observations can move a source into another minute or month; its earlier row remains stored but cannot reappear as current or enter clustering/counts. The projection rebuilds during replacement merges and lightweight deletes. Manual processing loads the complete trace first and reuses stored summaries only at that exact source start; a changed start generates a new summary. Accepted Redis payloads still resume without source or ClickHouse result reads.
Assignments contain classification results only. A topic ID means assigned;
an empty topic ID means outlier. An empty map ID supports future ad-hoc
classification. Awaiting-map, non-applicable and insufficient-input states come
from summaries, with no placeholder assignment rows. summary_processed_at
records which summary an assignment classified, so reprocessing can invalidate
old membership. Current membership considers published-map and ad-hoc assignments;
topic filtering follows latest selection. Execution pages show current results,
not exact historical batch membership. A late older job may win by timestamp.
Algorithm and queue recovery
The worker loads each trace lazily, shares one in-memory transcript across facets, and releases it before the next trace. Accepted summary references resume without fetching source data; source snapshots, transcripts and model request bodies are not persisted.
Processing and fitting use separate topics and topics-update queues with one
coordinator slot each. topics-embedding has two slots and receives reference-only
batches of up to 100 traces across facets. This internal batch size is not a
selected-trace cap.
- Summarization stages accepted results in Redis with a fixed deadline:
LANGFUSE_TOPICS_REDIS_TTL_SECONDSdefaults to three hours. Retries never extend it. - Embedding saves its vector with the staged summary before the combined ClickHouse insert. A completed embedding job acknowledges that insert; no unfinished summary rows reach ClickHouse.
- The coordinator releases its slot while its saved phase is
embedding. It polls the embedding job for its batch ID in Redis without database work. - After assignment inserts succeed, save terminal BullMQ state before deleting payloads. Cleanup failures leave them to expire. Repeated writes keep their identity. Inserts are bounded by 10,000 rows / 8 MiB and await async acknowledgement.
Normal retries use frozen inputs and accepted references, not summary/assignment reads. Missing/expired payloads fail explicitly. A new execution with Reuse stored summaries can recover persisted results. Redis loss or a crash between a provider response and saving it can repeat inference.
Stored-summary reuse is opt-in and requires the same facet version, summary model,
transcript producer (transcript_id) and version; it does not check content freshness. Compatible
vectors are reused; changed dimensions regenerate only embeddings. Results receive
the new execution's staging ownership,
replace previous summaries, record reuse timestamps and count only new usage.
This is not a cross-execution deduplication cache.
Manual processing and trace-batch share one summary request
(summarizeTraceFacets): the facets of a trace that still need a summary go into
one call on the same projected transcript. loadTopicTranscript uses the shared
orderObservations and assembleTranscript producer without a cap, like
trace-batch. The model receives the projected transcript: threads with
conversation history and current-turn messages and normalized parts, without
observation provenance. A null or empty transcript produces insufficient_input
without a summarization or embedding call.
The shared producer defines observation eligibility, tool association and replay
deduplication. It includes generations and matched tool responses. Root-only
non-generation traces, unmatched tool observations and observation-level status
messages are outside that conversation contract. Topics does not add fallbacks,
coverage fields, custom media handling or a separate transcript projection.
See packages/shared/src/server/transcript/README.md for the producer's semantics
and limitations.
If transcript plus instructions/schema exceeds the execution's input allowance,
the worker fails before calling the provider; it does not
silently change the evidence. Stored summaries record
transcript_id (trace-batch; older manual rows used poc with a capped transcript)
and transcript_version (shared-transcript-v3), plus the models used. Historical summaries from other producer versions are regenerated.
Accepted Redis payloads retain their recorded version when the same execution
resumes. Transcripts are regenerated from current observations; original source
snapshots and content hashes are not retained.
The summary prompt and schema write the summary before deciding applicability. Check one trace after changing either prompt or schema before spending on a batch. Replaying an accepted summary or embedding does not require its source trace. If a retry needs missing facets, it summarizes them together from the current transcript.
UMAP and HDBSCAN propose density clusters. Original-space unit centroids and cosine radii form the serving classifier. A radius uses the 95th percentile of leave-one-out member distances, capped by the fifth percentile of the nearest rival cluster's distances. The closest topic must pass its radius; there is no fallback to a farther topic. These are provisional heuristics, particularly uncertain for small or rare populations.
UMAP uses min(15, max(3, floor(n/3))) neighbors, bounded below the cohort size.
The pinned Rust backend is holomap 0.3.0 plus hdbscan-rs 0.6.1. Unit-normalized
embeddings feed seeded cosine UMAP (random initialization, seed 42), separately
into at most 10 dimensions for Euclidean HDBSCAN/EOM and 2 dimensions for display.
The Rust adapter adds one to minSamples because the library includes the point
itself in its neighbor count. Rust owns the fixed reduction and cluster-selection
defaults; the worker passes only minimum population, cluster size, and samples.
The numerical backend version is recorded per run.
Numerical results are held in memory until their assignments and coordinates are
saved in ClickHouse. An unpublished attempt is rerun from scratch after failure.
The synchronous native fit runs in a credential-free Node child process with a
120-second kill deadline and output limits proportional to cohort size. Discovery
admission defaults to 20,000 summaries and a 256 MiB input budget per facet,
controlled by LANGFUSE_TOPICS_MAX_CLUSTERING_ROWS and
LANGFUSE_TOPICS_MAX_CLUSTERING_BYTES. A metadata count rejects excessive rows
or embedding payload before loading vectors; the streamed read also enforces
rows and bytes to cover large texts and changes after the count. Exceeding a
limit fails the update with a smaller-window message. Automatic discovery selects
a fitting window before queuing; manual updates retain their requested window.
Neither samples the clustering cohort. These are initial capacity limits, not a guarantee about
container RSS: parent/child serialization, native fitting, and concurrent queue
jobs need additional headroom. Exact neighbor search has quadratic cost; admitted
cohorts can still exceed the deadline. Map coordinates are repeatable on the same platform
and build, but are not a compatibility contract across backends or architectures.
Synthetic cohorts do not establish topic quality; validate representative data
before expanding deployment.
The same classifier determines initial and later memberships. Naming sees its
effective populations: all full member summaries that fit the naming request,
plus up to three nearby contrasts. Each group uses one seeded random traversal
of its members, retaining whole summaries that fit; this is not a sample of
the numerical fitting population. Variable-length summaries make this a random
order with a fitting budget, not an equal-probability fixed-size sample. The
selection is repeatable for the same run/group and reused by the conflict retry.
Its additional memory is bounded by the selected prompt; no per-trace cache
survives the request. Vector calculations reuse per-pass scratch space and yield
between batches so queue timers can run. Each group gets one naming call with no
tool loop. If the name is
already taken in the run, one topics-naming-conflict call sees the conflicting
topic and is asked for a distinguishing name; a name that still repeats is kept,
since topics are keyed by ID. Member evidence references are validated before
accepting output.
The provider schema keeps evidence IDs as strings, avoiding the API's enum-size
limit for large clusters; local validation still requires genuine member IDs. This checks structural grounding, not factual
or semantic correctness; inspect the examples to judge usefulness.
No informative clusters yields a terminal no_topics result and publishes an
empty map with explicit outlier assignments for the selected cohort. HDBSCAN's single root cluster is disabled, so
a single overall population is not forced into a topic; validating that case
needs a future coherence policy. Maps are
published only after their initial assignments and coordinates are readable in
ClickHouse. Topic definitions and initial coordinates remain fixed while later
assignments can extend a map's live membership. Displayed summaries use current
text. Updates match final memberships to the previous published map for that facet
version. Changed embedding spaces use membership evidence without comparing
centroids. Continuing topics retain their stable topic IDs. When the embedding
space, centroid and radius are unchanged, the existing definition is reused
without another naming call. Geometry comparisons allow bounded Float64 roundoff;
reused definitions keep their stored centroid and radius for naming and assignment.
Changed definitions receive new topic version IDs. Material
splits/merges receive new IDs with predecessor lineage in topic metadata.
Cost and recovery
The Topics page renders saved 2D coordinates from the map's initial ClickHouse assignments. This view uses the same scale on both axes, with topic filtering, summary selection and trace links. It does not recompute clusters or charge a model. Later assigned traces remain unpositioned until a new discovery map; the numerical worker does not retain a UMAP transform.
The summary prompt distinguishes the requested task from whether that task succeeded. Each call receives the shared transcript and facet instruction and returns summary text plus applicability status. No citations or block IDs are requested from the model. A non-applicable result containing a summary is rejected, not silently repaired. Real model quality checks remain necessary; mocked tests cannot establish summary accuracy.
Postgres stores one batch_actions row per manual request for either operation,
and one clustering-run row per real attempt. Update admission creates the first
pending run for each selected facet/version and saves its reference atomically.
BatchAction owns lifecycle, error and aggregate progress; run config contains
embedding compatibility and numerical settings. Retry counts for completed maps
come from initial assignments, excluding later online classifications.
These contain settings and aggregate progress, not per-trace records or paid outputs.
BullMQ jobs carry bounded batches of up to 100 trace IDs, accepted summary references,
and temporary retry state. ClickHouse stores current summary text, embeddings,
classifications, and map coordinates. Completed summaries can await a map without
an assignment row.
ClickHouse stores immutable topic names, descriptions and prototypes, including
their original creation run. Each Postgres run stores the exact topic-version
IDs used by its classifier; missing definitions prevent loading that map.
Naming calls use short member labels; accepted evidence references store typed
facet-version and trace-or-session fields under the definition's project.
A trace reference omits its parent session. Those references identify the current
source result, not a historical summary snapshot.
Summary embeddings are stored as Array(BFloat16) (migration 0054_update_topic_tables). JSON inserts truncate each element to BFloat16 and reads return plain numbers, so vectors loaded from storage differ slightly from the model output that online assignment classified. Classifier inputs are normalized and compared by cosine distance, which this rounding barely moves. Topic reuse is stricter: sameTopicGeometry only tolerates Float64 roundoff, so centroids recomputed from summaries written as Float32 before the BFloat16 migration never match, and the first update of such a map names every topic again and creates new topic versions.
Definitions use RowBinary inserts with Float64 geometry to preserve classifier precision and
ReplacingMergeTree(created_at) partitioned by creation month and ordered by
project, creation date and version ID to deduplicate identical retry writes.
Definition lookups use creation-time bounds that include reused definitions:
an old definition may still be in use by the current map. There is no age-based
expiry. Transcripts stay in memory.
There are no Topics object-storage manifests, numerical checkpoints or shared-disk files.
Provider usage and calculated model costs use the events-style
provided_usage_details, usage_details, provided_cost_details and cost_details
maps. Summary and embedding keys are prefixed by stage; effective maps include
combined totals.
Manual processing summaries carry their execution ID in metadata. Run estimates
sum retained summary and embedding costs with available pricing. They exclude
topic naming and failed provider calls, and can change after reprocessing or deletion.
Priced model IDs: global Bedrock text rates per million input/output tokens are $0.20/$1.20 for
Luna
and $2/$12 for
Terra.
Naming requests above 272,000 input tokens use $4/$18 for the whole request.
Embedding costs use Bedrock-reported input tokens at $0.12/million. Missing usage
is logged and omitted from usage/cost maps, never estimated with an OpenAI tokenizer.
Model prices and token budgets live in models.ts and the shared
Topics contracts. Summary input defaults to 120,000 tokens. Oversized naming
input is limited to 900,000 estimated tokens and 16 MiB of serialized input.
The token cap is an application ceiling, not a guarantee that every configured
model supports that context. Selection reserves bounded space for a conflict
retry; contrast and conflicting examples are omitted whole when they exceed
their ancillary budgets. Member text is never truncated, and a group with no
fitting member fails before inference. Definition metadata records the full
population and the selected count separately.
Provider calls retry 429 and 5xx responses twice. Embedding queue jobs retry transient failures
up to three attempts with exponential backoff; authentication and invalid
input/output failures stop immediately. A manual resume can retry a failed batch
while its Redis payload still exists; each manual resume resets the three-attempt
retry budget.
An interrupted update starts a new attempt from compatible summaries in its frozen time window, then fits again and names only new or changed definitions. Partially named failed attempts cannot overwrite the previous published map. After publication, an acknowledgement retry recognizes the completed attempt without fitting again. Skipped attempts also resume without recomputation, using the previously saved cohort count to distinguish no applicable summaries from insufficient data.
Automatic processing starts only a facet version's first discovery; there is no refit schedule yet. The planned six-hour scheduler should invoke the same update path as the manual button. Large per-project fitting and distributed stream processing remain follow-up work; this is not a 100-million-traces/day throughput validation. The incoming trace path uses bounded batches; naming and discovery limits apply to individual project/facet updates. No process-wide summary or embedding cache is used.
Trace deletion removes stored facet summaries (including embeddings), assignments and map coordinates through the existing batched ClickHouse deletion path. It does not cancel in-flight Topics jobs, which can write results after deletion. Topic-name/centroid refresh remains separate lifecycle work. Project deletion removes summaries, assignments and topic definitions through the batch project cleaner.
Datadog metrics
metrics.ts owns langfuse.topics.executions, .results,
.stage_duration_ms and .errors. Tags are fixed categories without tenant IDs
or customer content. Counters measure attempts and work, not unique persisted
rows: resumes can count again and accepted results can precede persistence
failures. Nested catches count the same error once per attempt; completed replays
emit no attempt/result metrics. Model durations exclude cache hits.
Metrics are best effort, not an exactly-once ledger or heartbeat. Queue backlog,
waiting time and BullMQ outcomes use langfuse.queue.topics.*,
langfuse.queue.topics_update.* and langfuse.queue.topics_embedding.*.
Queue .depth includes type:delayed. Topics .pending_head_age_ms samples one
entry from each waiting, paused, delayed and active state without loading job
payloads. Retries can re-enter behind newer jobs, so this is head age, not the
oldest creation time anywhere in the backlog.
.errors reasons include input_too_large (transcript above the summary input
limit), trace_too_large (trace above the transcript load limits) and
definition_readback (a new topic definition did not read back identically; the
error names the differing fields, never their values).
Summary .results carry source:model|rule and facet (the built-in facet's
name, or custom): the model decides applicable, not_applicable or
insufficient_input; local rules decide insufficient_input (no transcript
content), invalid_output and input_too_large.
Every model call counts once in .llm_calls{stage,adapter,status}: ok,
400, 401, 403, 429, 4xx or 5xx for any other code (the exact code is
on the span), or the failure reason when there is no HTTP status. One
embedding call can fan out into several provider requests (single-text Bedrock
models, the retry without dimensions), and AI SDK retries are not counted
separately. Each successful call adds its egress to the shared
langfuse.export.serialized_bytes{integration:topics}, measured exactly like
LLM-as-a-judge (llmRequestEgressBytes: serialized provider messages plus the
output JSON Schema, uncompressed); responses (mostly embedding vectors) and
failed or retried requests are not included.
Datadog spans and logs
Spans carry tenant and request detail that metrics must not:
Worker.run *:langfuse.project.idfor every queue; Topics jobs add the execution ID, operation, status and phase. Trace-batch jobs addlangfuse.topics.traces.<outcome>totals.topics-trace-summary: one per summarized trace (both paths), with facet result counts underlangfuse.topics.facets.*.topics-facet-update: one per facet of an update, with its outcome. Children:topics.clustering(vectors, dimensions, status),topics.naming(one per group) andtopics.assignment(also used when manual processing assigns). A failed facet marks its enclosing span as an error, although the run handles it.topics.llm.summary|embedding|naming: one per provider request, with model, adapter and prompt bytes; on failure, the HTTP status and, when the provider answered, bounded response metadata (langfuse.topics.response.*: finish reason, raw finish reason from a fixed list, text length, text and reasoning part counts, output tokens and the output-token cap; never the response text).
A failed provider request logs one Topics model request failed warning with the
same fields. Provider error strings, codes and parameters can echo request or
response content, so only the HTTP status is kept. A failed facet logs Topics facet failed: a warning for provider or
configuration errors, an error otherwise.
langfuse.topics.tokens{stage:summary|embedding|naming,direction:input|output}
counts available provider-reported usage immediately after each model response,
before local validation or persistence. Embeddings have input tokens only.
Cached results and persistence retries do not add tokens; new provider calls do.
Missing or invalid usage increments .token_usage_missing with the same tags;
configured token budgets are never reported as measured usage. Responses rejected
inside the provider SDK may not expose usage, so these counters are not a bill.
Clustering consumes vectors rather than tokens: .clustering_vectors counts
attempted input vectors and .clustering_cohort_size records each fit's size,
both tagged by dimensions. Existing stage durations measure elapsed time, not CPU.
The queue metrics runner also reports .staged_results (unexpired retained
payloads), .staged_expiry_remaining_ms (earliest remaining Redis TTL, zero when
empty), and .staged_expired (expiry counter). Alert on low remaining TTL only
while staged results are present. The shared staging helper maintains a best-effort
Redis index of original deadlines; embedding completion retains tracking until
assignment acknowledgement removes the payload. Retries never renew its deadline.
Expiry means a retained payload expired, not necessarily lost work: cleanup may
have failed after durable persistence. .staging_tracking_errors{operation:track|release}
exposes index failures without rejecting accepted results.
Telemetry commands have a one-second wait deadline so Redis retries cannot stall
the paid-result handoff or completion. Timed-out commands may finish later; the
index and expiry counters remain best effort.
Collection runs independently of consumer progress while Topics and
LANGFUSE_QUEUE_METRICS_ENABLED are enabled. Global snapshot gauges must not be
summed across worker reporters: use max for counts/age and min for remaining TTL.
Expired index entries are removed and counted atomically in batches of at most
1,000, preventing duplicate expiry counts across reporters. Entries older than
one day past expiry may be discarded during indexing; the index itself expires
one day after its latest deadline. Existing payloads become visible only when
staged through the instrumented helper; deployment does not scan/backfill Redis.
Offline verification
pnpm --filter worker run test features/topics
pnpm --filter @langfuse/native exec cargo test --features napi/dyn-symbols topics
Pipeline tests mock provider calls and storage. numeric.native.test.ts runs
real UMAP/HDBSCAN through the worker's child process on more than 1,000 synthetic
vectors and checks serving prototypes, cold-start, identical-input and invalid
vector behavior. Rust tests cover deterministic fitting and validation. Neither
command calls a paid model.
storage.integration.test.ts checks exact definition roundtrips against real
ClickHouse, including Float64 values affected by decimal parsing. Apply the normal
Postgres and ClickHouse migrations before running it.
Default facet extraction
Intent names the goals of the whole run, earliest first, even when execution
fails; follow-up checks belong to the goal they serve. Sentiment labels the end
user's attitude toward the interaction (Positive, Negative, Mixed,
Neutral) and what it was directed at; runs without end-user text are not
applicable. Outcome labels where the run ended (Completed, Partial,
Unconfirmed, Needs input, Not completed), judged on results rather than the
assistant's claims. Issues describes the mistake that did the most damage to the
result in one sentence (the mistake, the kind of step, and its consequence),
without a category label, and checks four points before reporting no issue; a
problem quoted for analysis is not itself an agent defect. Each preset prompt
defines its facet's semantics; custom facets use their own versioned instructions.
The shared extraction wrapper asks for one English sentence of at most 30 words (a second only for a material distinction), preserves meaningful distinctions, and omits incidental identifiers, source references and narration. Applicability is separate from task success: absent signals and insufficient evidence retain their distinct statuses and empty summaries, so they do not become embedded topics. With one summary per trace/facet, the Issues default prioritizes the principal problem rather than claiming to enumerate every independent issue.