πͺ’ Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
TypeScript analyticsautogenevaluationlangchainlarge-language-modelsllama-indexllmllm-evaluationllm-observabilityllmopsmonitoringobservabilityopen-sourceopenaiplaygroundprompt-engineeringprompt-managementself-hostedycombinator
perf(trace): lay out large trace graphs off the main thread (#16054)
Nikita Kabardin484cc2ba4ffd
diff --git a/.agents/skills/seed-test-data/SKILL.md b/.agents/skills/seed-test-data/SKILL.mdindex 0e0b1ccb26..c0c68f70a0 100644--- a/.agents/skills/seed-test-data/SKILL.md+++ b/.agents/skills/seed-test-data/SKILL.md@@ -32,6 +32,7 @@ this.| The same tree readable in the v4 events UI | add `--v4` (writes `events_full`; `events_core` fills via MV) || Async parents whose subtree outlives their own span (subtree wall-clock duration badge) | add `--async-parents` to `trace-tree` (root + hub end immediately while children keep running) || A realistic agent flow over a timeline (graph view + scrubbable timeline) | `pnpm run seed -- agent-timeline --turns 6 --v4` (LangGraph refine loop plannerβretrieverβgeneratorβcriticβloop, staggered in time; add `--timing-only` for the pure timing fallback) |+| A trace that is large as a GRAPH (many distinct node names + connections, the trace-graph layout stress) | `pnpm run seed -- agent-graph --v4` (~1,350 distinct connections from 350 observations; `--nodes 120 --steps 100 --parallel 8` crosses the layout ceiling, `--nodes 80 --steps 30 --parallel 4` is small-but-dense) || A demo-grade, real-looking agent trace (videos, screenshots, docs) | `pnpm run seed -- support-agent --v4 --id-prefix <hex>` (one fixed, fully handcrafted support-copilot refund run: guardrails, parallel context fan-out, 3-turn ReAct loop with real payloads/costs; deterministic β reseed with a FRESH prefix for a clean take; the prefix is the trace id, so a hex prefix reads like production) || A plain trace with no agentic types (collapsed-by-default graph panel) | add `--plain` to `trace-tree` (SPAN/GENERATION/EVENT only) || A trace with MORE observations than the detail view loads (v4 caps the tree at 10k, `startTime ASC` β the chronological tail is missing) | `pnpm run seed -- trace-tree --observations 12000 --stride-ms 10 --v4` (`--stride-ms` starts each observation index Γ N ms in, so start times are unique and strictly increasing: observation index < 10000 loads, `-obs-10000` and up fall past the cap. Without it thousands of rows share one millisecond and the boundary is arbitrary. Re-runs that CHANGE timing flags need a fresh `--id-prefix`: `start_time` is an events ORDER BY key, so both versions persist in the same trace) |diff --git a/.agents/skills/sentry-instrumentation/SKILL.md b/.agents/skills/sentry-instrumentation/SKILL.mdindex 29ff70810b..a16bfd7aed 100644--- a/.agents/skills/sentry-instrumentation/SKILL.md+++ b/.agents/skills/sentry-instrumentation/SKILL.md@@ -20,11 +20,11 @@ should reach Sentry β do not `captureException` (or `console.error`) reflexiveRun the decision tree, then say in one line what you chose and why (in the planor PR description). The three outcomes:-| The failure isβ¦ | Do | Never |-|---|---|---|-| an **expected user-facing state** β a missing/forbidden resource, expired session, invalid user input, a malformed URL in user content | render the UX (error page / toast / plain text) | capture β it is the product working as designed |-| a **transport / offline / infra** failure β fetch failed, a 5xx on a poll, an LB blip | let the UI degrade; the server owns this signal | capture client-side β it is an amplified, lower-fidelity copy of a server truth |-| **our code failed** β an invariant broke, a parse threw, a worker failed to load | `captureException` a **real `Error`** with an `area` tag via the shared helpers | pass a raw string/object/`Event` |+| The failure isβ¦ | Do | Never |+| -------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |+| an **expected user-facing state** β a missing/forbidden resource, expired session, invalid user input, a malformed URL in user content | render the UX (error page / toast / plain text) | capture β it is the product working as designed |+| a **transport / offline / infra** failure β fetch failed, a 5xx on a poll, an LB blip | let the UI degrade; the server owns this signal | capture client-side β it is an amplified, lower-fidelity copy of a server truth |+| **our code failed** β an invariant broke, a parse threw, a worker failed to load | `captureException` a **real `Error`** with an `area` tag via the shared helpers | pass a raw string/object/`Event` |**`console.error` is a capture API here.** `instrumentation-client.ts` enables`captureConsoleIntegration({ levels: ["error"] })`, so **every `console.error`@@ -41,9 +41,12 @@ properly (below).β turns any caught value into a legible `Error` (real `Error`s pass throughwith their stack; everything else is synthesized), tags `area`, and logs via`console.warn` so the console integration does not double-capture.- - [`reportParserWorkerError(hook, event)`](../../../web/src/hooks/parserWorkerError.ts)- β extracts the real fields from a Worker `ErrorEvent` (message/filename/lineno)- instead of stringifying it to `[object ErrorEvent]`.+ - [`reportWorkerLoadError({area, source, event, extra?})`](../../../web/src/utils/reportWorkerLoadError.ts)+ β extracts the real fields from a Worker `ErrorEvent`+ (message/filename/lineno/colno) instead of stringifying it to+ `[object ErrorEvent]`. Every worker's `onerror` reports through this one;+ [`reportParserWorkerError(hook, event)`](../../../web/src/hooks/parserWorkerError.ts)+ is the JSON-parser worker's thin wrapper over it.- **Filter predicates belong in[`web/src/utils/sentryFilters.ts`](../../../web/src/utils/sentryFilters.ts)**(documented, unit-tested), called from `beforeSend` in@@ -59,7 +62,7 @@ properly (below).chokepoint every query/mutation error flows through β the place to dropexpected codes and tag the rest (the seam-classification lever, PR #15243).- **Tag `area`, keep the message static.** `captureException(err, { tags: { area },- extra })`. Fingerprints group on the message, so put variable IDs in `extra`,+extra })`. Fingerprints group on the message, so put variable IDs in `extra`,never in the message string.## The rules (each earned the hard way β cited to workstream PRs)@@ -77,7 +80,7 @@ properly (below).2. **Capture a REAL `Error`, never a string / object / `SyntheticEvent` /`ErrorEvent`.** Those collapse to opaque `[object Object]` / `[object- ErrorEvent]` fingerprints with no stack. β The 5EX cluster (opaque non-Error+ErrorEvent]` fingerprints with no stack. β The 5EX cluster (opaque non-Errorcaptures, PR #15175) and the parse-worker `[object ErrorEvent]` family (PR#15173) were exactly this. Route unknowns through `captureUnknownError` /`reportParserWorkerError`.@@ -139,8 +142,8 @@ properly (below).test does NOT prove the console error is gone. Reproduce it in a browseragainst the running app (Playwright), do an A/B, and confirm the eventdisappears. β PR #15245 was verified this way β its native `<a>` fired 0- where the prior `<Link>` fired the error Γ12. Unit tests lock the *contract*; the- browser proves the *noise removal*.+ where the prior `<Link>` fired the error Γ12. Unit tests lock the _contract_; the+ browser proves the _noise removal_.## Workflowdiff --git a/packages/shared/scripts/seeder/AGENTS.md b/packages/shared/scripts/seeder/AGENTS.mdindex cbd33e46cd..9796c362cb 100644--- a/packages/shared/scripts/seeder/AGENTS.md+++ b/packages/shared/scripts/seeder/AGENTS.md@@ -14,6 +14,7 @@ pnpm run seed -- trace-tree --observations 12 --v4 --tags "Zebra,apple,Γrger"pnpm run seed -- trace-tree --observations 12000 --stride-ms 10 --v4 # more observations than the detail view loads; --stride-ms makes the startTime-ordered cap boundary exact (index < 10000 loads)pnpm run seed -- deep-chain --v4 # 1401 sequential generations in ONE parent chain (depth = count; LFE-10959 layout stress)pnpm run seed -- agent-timeline --turns 6 --v4 # realistic agent flow-with-loop over a timeline (graph view)+pnpm run seed -- agent-graph --v4 # graph-DENSE trace: ~1,350 distinct node-pair connections from 350 observations (trace-graph layout stress)pnpm run seed -- support-agent --v4 --id-prefix <hex> # demo-grade handcrafted support-copilot run (videos/screenshots)pnpm run seed -- long-session --traces 300 --observations-per-trace 8pnpm run seed -- session-shapes --shape all # chat / coding-agent / mixed / media v4 sessionsdiff --git a/packages/shared/scripts/seeder/scenarios/agent-graph.ts b/packages/shared/scripts/seeder/scenarios/agent-graph.tsnew file mode 100644index 0000000000..d15992b44c--- /dev/null+++ b/packages/shared/scripts/seeder/scenarios/agent-graph.ts@@ -0,0 +1,377 @@+import {+ createTrace,+ createObservation,+ createTracesCh,+ createObservationsCh,+ createEventsCh,+ ObservationRecordInsertType,+} from "../../../src/server";+import { ObservationType } from "../../../src/domain";+import { observationToEvent, traceToEvent } from "./event-mirror";+import { buildPayload, generationUsageCost } from "./payload";+import { jitter, Rng, utcDayStartMs } from "./rng";+import {+ chunk,+ ScenarioContext,+ ScenarioDefinition,+ SeedError,+ SeedSummary,+} from "./types";+import { countRows, traceLink } from "./verify";++/**+ * One trace that is large as a GRAPH rather than as a tree: many distinct+ * `langgraph_node` names, and many distinct name-pair connections between them.+ *+ * This is a different axis from trace-tree/deep-chain, where thousands of+ * observations still collapse into a handful of graph nodes. The aggregated graph+ * connects every node of one super-step to every node of the next, so a step+ * carrying `--parallel` branches contributes parallelΒ² connections per step+ * boundary β the shape that made a reported trace hand ELK ~1,400 distinct edges+ * (~23k before dedupe) and freeze the tab for minutes.+ *+ * Defaults land ~1,350 distinct connections from 350 observations. Scale+ * `--nodes`/`--parallel` up to cross the layout ceiling (`--nodes 120 --steps 100+ * --parallel 8` β 5,100 connections β the "too large to lay out" notice), or make+ * a smaller graph pathologically dense (`--nodes 80 --steps 30 --parallel 4`+ * β 450 connections over 80 nodes, tens of seconds of layout).+ */++const LANGGRAPH_NODE_TAG = "langgraph_node";+const LANGGRAPH_STEP_TAG = "langgraph_step";++// Node-name vocabulary: `${ROLE}_${index}` keeps every name distinct while+// reading like a real agent graph.+const ROLES = [+ "planner",+ "router",+ "retriever",+ "ranker",+ "generator",+ "critic",+ "tool_caller",+ "summarizer",+];++const TYPES: ObservationType[] = [+ "AGENT",+ "RETRIEVER",+ "GENERATION",+ "EVALUATOR",+ "TOOL",+];++const nodeName = (index: number) => `${ROLES[index % ROLES.length]}_${index}`;++/**+ * Node indices visited in super-step `step`. Hash-spread over the vocabulary+ * (stateless in (seed, step, slot)) rather than a rotating stride: a stride+ * repeats its step-to-step differences and the distinct-pair count saturates in+ * the low hundreds, while a spread keeps minting new name pairs.+ */+function stepNodes(+ seed: number,+ step: number,+ nodes: number,+ parallel: number,+): number[] {+ const picked: number[] = [];+ for (let slot = 0; picked.length < Math.min(parallel, nodes); slot++) {+ if (slot > parallel * 4) break; // give up rather than loop on a tiny vocabulary+ const index = jitter(seed, step * 1_009 + slot, nodes - 1);+ if (!picked.includes(index)) picked.push(index);+ }+ return picked;+}++/** Distinct (from,to) name pairs the aggregated graph will derive β the number+ * the layout budget is measured against. */+function countDistinctEdges(+ seed: number,+ steps: number,+ nodes: number,+ parallel: number,+): number {+ const pairs = new Set<string>();+ for (let step = 0; step < steps - 1; step++) {+ for (const from of stepNodes(seed, step, nodes, parallel)) {+ for (const to of stepNodes(seed, step + 1, nodes, parallel)) {+ if (from !== to) pairs.add(`${from}>${to}`);+ }+ }+ }+ // Plus the terminal edges into the synthetic __end__ node.+ return pairs.size + stepNodes(seed, steps - 1, nodes, parallel).length;+}++const run = async (+ ctx: ScenarioContext,+ params: Record<string, string | number | boolean>,+): Promise<SeedSummary> => {+ const startedAt = Date.now();+ const nodes = params["nodes"] as number;+ const steps = params["steps"] as number;+ const parallel = params["parallel"] as number;+ const withV4 = params["v4"] as boolean;++ if (nodes < 2 || steps < 2 || parallel < 1) {+ throw new SeedError(+ `--nodes and --steps must be >= 2 and --parallel >= 1, got ${nodes}/${steps}/${parallel}`,+ "try the defaults: --nodes 60 --steps 60 --parallel 5",+ );+ }++ const rng = new Rng(ctx.seed);+ const traceId = `${ctx.idPrefix}-trace`;+ const traceTimestamp = utcDayStartMs();+ const distinctEdges = countDistinctEdges(ctx.seed, steps, nodes, parallel);++ type PlannedObs = {+ index: number;+ node: string;+ step: number;+ type: ObservationType;+ startOffset: number;+ endOffset: number;+ };++ const planned: PlannedObs[] = [];+ const STEP_MS = 220; // wall-clock advance per super-step+ for (let step = 0; step < steps; step++) {+ for (const nodeIndex of stepNodes(ctx.seed, step, nodes, parallel)) {+ const index = planned.length;+ // Branches inside a step overlap (that IS the parallelism); steps never do.+ const start = step * STEP_MS + jitter(ctx.seed, index, 20);+ planned.push({+ index,+ node: nodeName(nodeIndex),+ step: step + 1, // langgraph_step is 1-based+ type: TYPES[nodeIndex % TYPES.length],+ startOffset: start,+ endOffset: start + 80 + jitter(ctx.seed, index + 1, 100),+ });+ }+ }++ const counts: Record<string, number> = {+ traces: 1,+ observations: planned.length,+ events: withV4 ? planned.length + 1 : 0,+ // Distinct names actually visited, NOT the --nodes vocabulary: the hash spread+ // does not guarantee full coverage (steps x parallel picks over `nodes` slots),+ // so a small run touches fewer names than the flag asks for.+ graphNodes: new Set(planned.map((p) => p.node)).size,+ graphConnections: distinctEdges,+ };++ if (ctx.dryRun) {+ return {+ scenario: "agent-graph",+ target: "clickhouse",+ params,+ projectId: ctx.projectId,+ environment: ctx.environment,+ traceIds: [traceId],+ sessionIds: [],+ counts,+ verified: {},+ links: [traceLink(ctx, traceId, traceTimestamp)],+ dryRun: true,+ durationMs: Date.now() - startedAt,+ };+ }++ const trace = createTrace({+ id: traceId,+ project_id: ctx.projectId,+ environment: ctx.environment,+ name: `seed-agent-graph (${nodes} nodes, ${distinctEdges} connections)`,+ timestamp: traceTimestamp,+ user_id: `user-${ctx.idPrefix}`,+ session_id: null,+ release: "seed-1.0.0",+ version: "seed-agent-graph-v1",+ tags: ["seed", "agent-graph"],+ public: false,+ bookmarked: false,+ metadata: { scenario: "agent-graph", seed: String(ctx.seed) },+ input: JSON.stringify({ task: "resolve a multi-hop research question" }),+ output: JSON.stringify({ answer: "done", steps }),+ created_at: Date.now(),+ updated_at: Date.now(),+ event_ts: Date.now(),+ });++ const observations: ObservationRecordInsertType[] = planned.map((p) => {+ const start = traceTimestamp + p.startOffset;+ const end = traceTimestamp + p.endOffset;+ const isGeneration = p.type === "GENERATION";+ const usageInput = rng.int(200, 3000);+ const usageOutput = rng.int(50, 1200);++ return createObservation({+ id: `${ctx.idPrefix}-obs-${p.index}`,+ trace_id: traceId,+ project_id: ctx.projectId,+ environment: ctx.environment,+ type: p.type,+ parent_observation_id: null, // every node span sits directly under the trace+ name: p.node,+ start_time: start,+ end_time: end,+ completion_start_time: isGeneration ? start + 20 : null,+ level: "DEFAULT",+ status_message: null,+ version: null,+ input: isGeneration+ ? JSON.stringify({+ messages: [+ { role: "user", content: buildPayload("text", 200, rng) },+ ],+ })+ : JSON.stringify({ node: p.node, step: p.step }),+ output: isGeneration ? buildPayload("text", 150, rng) : null,+ metadata: {+ scenario: "agent-graph",+ [LANGGRAPH_NODE_TAG]: p.node,+ [LANGGRAPH_STEP_TAG]: String(p.step),+ },+ provided_model_name: isGeneration ? "gpt-5.4-mini" : null,+ internal_model_id: null,+ model_parameters: isGeneration+ ? JSON.stringify({ temperature: 0.2, max_tokens: 1024 })+ : "{}",+ ...(isGeneration+ ? generationUsageCost(usageInput, usageOutput)+ : {+ provided_usage_details: {},+ usage_details: {},+ provided_cost_details: {},+ cost_details: {},+ total_cost: null,+ }),+ prompt_id: null,+ prompt_name: null,+ prompt_version: null,+ created_at: Date.now(),+ updated_at: Date.now(),+ event_ts: Date.now(),+ });+ });++ const events = withV4+ ? [+ traceToEvent(trace),+ ...observations.map((o) => observationToEvent(o, trace)),+ ]+ : [];+ counts.events = events.length;++ ctx.log(+ `writing 1 trace, ${observations.length} observations across ${steps} steps β ~${distinctEdges} distinct graph connections${withV4 ? `, ${events.length} events` : ""}`,+ );+ await createTracesCh([trace]);+ for (const batch of chunk(observations, 1000)) {+ await createObservationsCh(batch);+ }+ for (const batch of chunk(events, 500)) {+ await createEventsCh(batch);+ }++ const verified: Record<string, number> = {+ traces: await countRows(+ "traces",+ `project_id = {projectId: String} AND id = {traceId: String}`,+ { projectId: ctx.projectId, traceId },+ "uniqExact(id)",+ ),+ observations: await countRows(+ "observations",+ `project_id = {projectId: String} AND trace_id = {traceId: String}`,+ { projectId: ctx.projectId, traceId },+ "uniqExact(id)",+ ),+ // The distinct node names ARE the point of this scenario β verify them, not+ // just the row count.+ langgraphNodes: await countRows(+ "observations",+ `project_id = {projectId: String} AND trace_id = {traceId: String} AND metadata[{nodeTag: String}] != ''`,+ { projectId: ctx.projectId, traceId, nodeTag: LANGGRAPH_NODE_TAG },+ `uniqExact(metadata[{nodeTag: String}])`,+ ),+ };+ if (withV4) {+ verified.events = await countRows(+ "events_full",+ `project_id = {projectId: String} AND trace_id = {traceId: String}`,+ { projectId: ctx.projectId, traceId },+ "uniqExact(span_id)",+ );+ }++ if (verified.traces < 1) {+ throw new SeedError(+ `Readback mismatch: trace ${traceId} not found after insert`,+ );+ }+ if (verified.observations < observations.length) {+ throw new SeedError(+ `Readback mismatch: expected ${observations.length} observations, found ${verified.observations}`,+ );+ }+ if (withV4 && verified.events < events.length) {+ throw new SeedError(+ `Readback mismatch: expected ${events.length} events_full rows, found ${verified.events}`,+ );+ }++ return {+ scenario: "agent-graph",+ target: "clickhouse",+ params,+ projectId: ctx.projectId,+ environment: ctx.environment,+ traceIds: [traceId],+ sessionIds: [],+ counts,+ verified,+ links: [traceLink(ctx, traceId, traceTimestamp)],+ dryRun: false,+ durationMs: Date.now() - startedAt,+ };+};++export const agentGraphScenario: ScenarioDefinition = {+ name: "agent-graph",+ description:+ "One trace that is large as a GRAPH, not as a tree: many distinct langgraph_node names with parallel branches per super-step, so the aggregated graph gets thousands of distinct node-pair connections from a few hundred observations (the shape that froze the trace graph). Defaults β 1,350 connections; scale --nodes/--parallel to cross the layout ceiling, or use --nodes 80 --steps 30 --parallel 4 for a small-but-dense graph.",+ supportsV4: true,+ flags: [+ {+ flag: "nodes",+ type: "number",+ default: 60,+ description: "distinct graph node names (the vocabulary)",+ },+ {+ flag: "steps",+ type: "number",+ default: 70,+ description: "super-steps in the run (observations = steps Γ parallel)",+ },+ {+ flag: "parallel",+ type: "number",+ default: 5,+ description:+ "branches per super-step; connections per step boundary are parallelΒ²",+ },+ {+ flag: "v4",+ type: "boolean",+ default: false,+ description: "also mirror into v4 events_full/events_core",+ },+ ],+ run,+};diff --git a/packages/shared/scripts/seeder/scenarios/index.ts b/packages/shared/scripts/seeder/scenarios/index.tsindex 31be24f77b..8e2a78b727 100644--- a/packages/shared/scripts/seeder/scenarios/index.ts+++ b/packages/shared/scripts/seeder/scenarios/index.ts@@ -1,3 +1,4 @@+import { agentGraphScenario } from "./agent-graph";import { agentTimelineScenario } from "./agent-timeline";import { annotationQueueScenario } from "./annotation-queue";import { customModelsScenario } from "./custom-models";@@ -17,6 +18,7 @@ import { ScenarioDefinition } from "./types";export const scenarios: Record<string, ScenarioDefinition> = {"trace-tree": traceTreeScenario,"agent-timeline": agentTimelineScenario,+ "agent-graph": agentGraphScenario,"deep-chain": deepChainScenario,"long-session": longSessionScenario,"many-traces": manyTracesScenario,diff --git a/web/src/features/trace-graph-view/README.md b/web/src/features/trace-graph-view/README.mdindex 45bfa08f8c..3eb212b4b7 100644--- a/web/src/features/trace-graph-view/README.md+++ b/web/src/features/trace-graph-view/README.md@@ -18,7 +18,9 @@ agentGraphData (tRPC getAgentGraphData)minus EVENTs β framework metadata is ignored);edges from the instrumented hierarchy +happened-before sibling ordering (fork/join)- β layout/elkLayout.computeGraphLayout async ELK (lazy import);+ β layout/graphLayoutWorkerClient.requestGraphLayout+ layout/elkLayout prepare (dedupe, size, count ceiling) on the main+ thread β run ELK in workers/elk-layout.worker.tslayout/measureNode estimates node boxes (labels, counter reserve)β components/ElkGraphRenderer draws + gesturescomponents/GraphNode view-only node (memo)@@ -48,10 +50,14 @@ downstream is mode-agnostic. The selected mode is a trace view preferencefolder owns only the projection observation-ids β node names(`components/TraceGraphView.tsx`) and the glow rendering (`GraphNode`).- **Pure layout math**: `layout/*` has no React imports and is unit-tested- (`layout/*.clienttest.ts`).+ (`layout/*.clienttest.ts`). `elkLayout.ts` stays free of worker plumbing so it+ runs unchanged on either thread; `graphLayoutWorkerClient.ts` owns the worker+ singleton, request ids, cancellation and the wall-clock deadline.+- **Layout thread**: ELK runs in `web/src/workers/elk-layout.worker.ts`, so a+ slow layout never blocks the trace view. ELK is uninterruptible even there β+ cancelling a stale layout means terminating the worker.## Next migration slices-- Move ELK layout into a Web Worker (seam: `layout/elkLayout.ts#getElk`).- Node virtualization for very large graphs (only render nodes intersectingthe viewport β the renderer's world/viewport split is already shaped for it).diff --git a/web/src/features/trace-graph-view/components/ElkGraphRenderer.tsx b/web/src/features/trace-graph-view/components/ElkGraphRenderer.tsxindex 5f13038148..a7a4196ea4 100644--- a/web/src/features/trace-graph-view/components/ElkGraphRenderer.tsx+++ b/web/src/features/trace-graph-view/components/ElkGraphRenderer.tsx@@ -16,12 +16,13 @@ import {import { ZoomIn, ZoomOut, Maximize } from "lucide-react";import { Button } from "@/src/components/ui/button";+import { reportError } from "@/src/utils/reportError";import { type GraphCanvasData, type GraphNodeData } from "../types";import {- computeGraphLayout,type GraphLayout,type GraphLayoutDirection,} from "../layout/elkLayout";+import { requestGraphLayout } from "../layout/graphLayoutWorkerClient";import { GraphNode } from "./GraphNode";type ElkGraphRendererProps = {@@ -56,6 +57,7 @@ const ZOOM_STEP = 1.4;// Below this scale labels are unreadable noise β show only node shape + icon.const LABEL_HIDE_SCALE = 0.5;const CLICK_MOVE_THRESHOLD = 4; // px; beyond this a pointerup is a drag, not a click+const SLOW_LAYOUT_HINT_MS = 4_000;function toPath(points: { x: number; y: number }[]): string {return points.map((p, i) => `${i === 0 ? "M" : "L"} ${p.x} ${p.y}`).join(" ");@@ -117,6 +119,10 @@ export const ElkGraphRenderer: React.FC<ElkGraphRendererProps> = ({const [layout, setLayout] = useState<GraphLayout | null>(null);const [layoutError, setLayoutError] = useState(false);const [layoutAttempt, setLayoutAttempt] = useState(0);+ // A layout past this point is a big graph (most finish in well under a second):+ // say so, and say the rest of the view is still usable β the whole point of+ // laying out off the main thread.+ const [slowLayout, setSlowLayout] = useState(false);const [size, setSize] = useState({ width: 0, height: 0 });// Discrete zoom derivation: labels hide below LABEL_HIDE_SCALE.const [compact, setCompact] = useState(false);@@ -149,26 +155,46 @@ export const ElkGraphRenderer: React.FC<ElkGraphRendererProps> = ({return map;}, [nodeToObservationsMap, currentObservationIndices]);- // Compute layout via ELK whenever the graph changes (or a retry is asked).+ // Lay the graph out in the layout worker whenever the graph changes (or a retry+ // is asked). The two guards do different jobs: `abort` stops the worker's+ // now-pointless run, `cancelled` keeps a late result from landing on a newer+ // graph.useEffect(() => {let cancelled = false;+ const controller = new AbortController();setLayout(null);setLayoutError(false);setFitted(false);+ setSlowLayout(false);+ const slowTimer = setTimeout(+ () => setSlowLayout(true),+ SLOW_LAYOUT_HINT_MS,+ );// A new graph gets a fresh fit; stale hover highlighting drops too.overrideRef.current = null;setHoveredId(null);- computeGraphLayout(graph, nodeToObservationsMap, layoutDirection)+ requestGraphLayout(+ graph,+ nodeToObservationsMap,+ layoutDirection,+ controller.signal,+ ).then((result) => {+ clearTimeout(slowTimer); // a landed layout is not a slow oneif (!cancelled) setLayout(result);}).catch((error) => {- console.error("Graph layout failed:", error);- // Guarded so a superseded effect's rejection can't stomp newer state.- if (!cancelled) setLayoutError(true);+ clearTimeout(slowTimer);+ if (cancelled) return; // superseded β the rejection IS the cancellation+ // No warnMessage: the console line must carry ELK's own reason (e.g.+ // "too much recursion"), which is the whole diagnostic.+ reportError(error, { area: "trace-graph-layout" });+ setLayoutError(true);});return () => {cancelled = true;+ clearTimeout(slowTimer);+ controller.abort();};}, [graph, nodeToObservationsMap, layoutDirection, layoutAttempt]);@@ -340,13 +366,22 @@ export const ElkGraphRenderer: React.FC<ElkGraphRendererProps> = ({onClick={handleBackgroundClick}>{!layout && !layoutError && (- <div className="text-muted-foreground absolute inset-0 flex items-center justify-center text-sm">- Laying out graphβ¦+ <div className="text-muted-foreground absolute inset-0 flex flex-col items-center justify-center gap-1 px-4 text-center text-sm">+ <span>Laying out graphβ¦</span>+ {slowLayout && (+ <span>+ This is a large graph β the tree and timeline stay usable while it+ finishes.+ </span>+ )}</div>)}{layoutError && (- <div className="text-muted-foreground absolute inset-0 flex flex-col items-center justify-center gap-2 text-sm">- <span>Could not lay out the graph.</span>+ <div className="text-muted-foreground absolute inset-0 flex flex-col items-center justify-center gap-2 px-4 text-center text-sm">+ <span>+ Could not lay out the graph. Try the tree or timeline view to+ explore this trace.+ </span><Buttonvariant="outline"size="sm"@@ -360,8 +395,9 @@ export const ElkGraphRenderer: React.FC<ElkGraphRendererProps> = ({</div>)}{layout?.tooLarge && (- // Budget exceeded: ELK was skipped rather than freeze the tab. No retry- // β it would just wedge again. Point the user at the tree/timeline.+ // No layout: over the count ceiling, or the worker ran past its deadline+ // and was killed. No retry β it would end the same way. Point the user at+ // the tree/timeline instead.<div className="text-muted-foreground absolute inset-0 flex flex-col items-center justify-center gap-1 px-4 text-center text-sm"><span>This graph is too large to lay out@@ -372,21 +408,23 @@ export const ElkGraphRenderer: React.FC<ElkGraphRendererProps> = ({</span><span>Try the{" "}- {onShowExpanded ? (- <button- type="button"- onClick={(e) => {- e.stopPropagation(); // don't treat as a canvas deselect- onShowExpanded();- }}- className="text-primary underline underline-offset-2 hover:opacity-80"- >- expanded graph- </button>- ) : (- "expanded graph"+ {/* The expanded graph is an alternative only from another view. */}+ {onShowExpanded && (+ <>+ <button+ type="button"+ onClick={(e) => {+ e.stopPropagation(); // don't treat as a canvas deselect+ onShowExpanded();+ }}+ className="text-primary underline underline-offset-2 hover:opacity-80"+ >+ expanded graph+ </button>+ ,{" "}+ </>)}- , tree, or timeline view to explore this trace.+ tree or timeline view to explore this trace.</span></div>)}diff --git a/web/src/features/trace-graph-view/layout/elkLayout.clienttest.ts b/web/src/features/trace-graph-view/layout/elkLayout.clienttest.tsindex f235418956..249524bf94 100644--- a/web/src/features/trace-graph-view/layout/elkLayout.clienttest.ts+++ b/web/src/features/trace-graph-view/layout/elkLayout.clienttest.ts@@ -104,9 +104,9 @@ describe("dedupeEdges", () => {});describe("computeGraphLayout layout budget", () => {- // A dense cyclic aggregated graph freezes ELK (measured: 100 nodes/800 edges- // β 177s on the main thread, a real trace fed ~1,400 edges froze >110s). ELK- // is synchronous, so the only safe fix is to refuse the layout up front.+ // A dense cyclic aggregated graph grinds for minutes (measured: 1,000+ // nodes/3,000 edges β 27s, 2,000/6,000 β 107s). ELK is uninterruptible even in+ // the layout worker, so past the ceiling the layout is refused up front.// `edgeCount` distinct directed edges among the first `nodeCount` node ids// (no self-loops) β a dense graph, but edge endpoints stay within the nodes.diff --git a/web/src/features/trace-graph-view/layout/elkLayout.ts b/web/src/features/trace-graph-view/layout/elkLayout.tsindex 4d3150a724..58d901b35c 100644--- a/web/src/features/trace-graph-view/layout/elkLayout.ts+++ b/web/src/features/trace-graph-view/layout/elkLayout.ts@@ -2,6 +2,7 @@ import type { ELK, ElkNode } from "elkjs";import {type GraphCanvasData,+ type GraphNodeData,LANGFUSE_START_NODE_NAME,LANGFUSE_END_NODE_NAME,LANGGRAPH_START_NODE_NAME,@@ -32,9 +33,10 @@ export interface GraphLayout {width: number;height: number;/**- * Set when the graph exceeded the layout budget: ELK was NOT run (it would- * freeze the tab), `nodes`/`edges` are empty, and the renderer shows a- * "too large to lay out" notice instead. See MAX_GRAPH_LAYOUT_* below.+ * Set when the graph got no layout: either past the count ceiling+ * (MAX_GRAPH_LAYOUT_*, so ELK never ran) or past the worker's wall-clock+ * deadline. `nodes`/`edges` are empty and the renderer shows a "too large to+ * lay out" notice.*/tooLarge?: boolean;/** Distinct node / deduped-edge counts β surfaced in the "too large" notice. */@@ -44,6 +46,20 @@ export interface GraphLayout {export type GraphLayoutDirection = "DOWN" | "RIGHT";+/**+ * Everything ELK needs, and nothing else β this crosses the worker boundary, and+ * `postMessage`'s structured clone is synchronous main-thread work. Edges are+ * deduped and the observation map is reduced to per-node counter widths BEFORE+ * the post, so we never clone the raw multigraph or the full id lists.+ */+export interface GraphLayoutRequest {+ nodes: GraphNodeData[];+ edges: GraphCanvasData["edges"];+ /** [nodeId, extra label chars] β see buildCounterReserve. */+ counterReserve: [string, number][];+ direction: GraphLayoutDirection;+}+/*** ELK "layered" options for a deterministic DAG. Orthogonal routing + merged* edges keep dense traces from turning into a hairball. Direction is@@ -69,36 +85,58 @@ const LAYOUT_OPTIONS: Record<string, string> = {* costs tens of seconds well before that. Above this bound the graph keeps* the RIGHT direction but skips wrapping (a plain ribbon lays out in* milliseconds at any size).+ *+ * Layout now runs in a Worker, whose stack is SMALLER than the main thread's β+ * measured, a deep aggregated graph overflows around 500 layers there β so this+ * bound stays conservative rather than being relaxed.*/const MAX_WRAP_NODES = 300;/**- * Layout budget for the aggregated (DOWN) graph. Aggregation collapses repeated- * step names into one node, which turns a large trace into a small-but-DENSE,- * often CYCLIC multigraph β the pathological input for ELK's layered algorithm- * (cycle breaking + crossing minimization + orthogonal routing all blow up- * super-linearly with edge density). Measured with the app's exact layout- * options on dense cyclic graphs: ~40 nodes/200 edges β 1.4s, 50/250 β 5.8s,- * 60/300 β 10s, 80/600 β 70s, 100/800 β 177s β and a real reported trace fed- * ELK 1,422 distinct edges and froze the tab for >110s (indefinite wedge /- * "too much recursion" on small-stack browsers). elkjs runs synchronously on- * the main thread, so once started it can't be interrupted or caught β the ONLY- * safe fix is to not start it. Above the budget we skip layout and the renderer- * shows a "too large" notice.+ * Count ceiling for the aggregated (DOWN) graph β a memory and sanity bound, NOT+ * a time budget. Layout cost tracks graph SHAPE far more than size: measured with+ * these exact options at identical counts, 60 nodes / 300 edges lay out in ~0.35s+ * when the edges concentrate on a few nodes and ~16s when they are spread+ * uniformly (aggregation collapses repeated step names, which is what turns a+ * large trace into a small-but-dense, often cyclic multigraph β the pathological+ * input for the layered algorithm). No count can predict that, so the real budget+ * is the wall-clock GRAPH_LAYOUT_DEADLINE_MS enforced by the layout worker's+ * client; ELK stays uninterruptible even in a worker, hence a ceiling as well.*- * Sized to sit well below the danger zone (worst-case dense layout at the cap- * stays a few seconds, never minutes) while leaving large headroom over real- * aggregated graphs, which have far fewer distinct name-pairs. SPARSE/acyclic- * graphs of any size are cheap (3,840 acyclic edges lay out in ~0.6s), but the- * aggregated view rarely reaches this many distinct nodes/edges without also- * being dense β exactly the case we must protect against.+ * Sized from the sparse case, which is what real aggregated graphs at this scale+ * look like: 2,500 nodes / 2,499 edges lay out in ~0.45s using ~88MB. Edges are+ * the expensive dimension (dense 1,000 / 3,000 β 27s), so they get the tighter+ * number; dense graphs inside the ceiling hit the deadline instead and get the+ * "too large" notice β a graph the previous 250-edge budget refused outright now+ * usually renders.** The expanded (RIGHT) path is exempt: it unrolls loops into an ACYCLIC DAG and* is already bounded upstream (MAX_EXPANDED_EDGES + the MAX_WRAP_NODES wrapping* cap), so it lays out in ~2.5s even at its own limits.*/-export const MAX_GRAPH_LAYOUT_EDGES = 250;-export const MAX_GRAPH_LAYOUT_NODES = 500;+export const MAX_GRAPH_LAYOUT_EDGES = 2_000;+export const MAX_GRAPH_LAYOUT_NODES = 2_500;++/**+ * Ceiling for a layout that must run on the CALLING thread β no Worker support, or+ * the worker script failed to load. A wall-clock deadline is structurally+ * impossible there (synchronous ELK cannot be interrupted), so counts are the only+ * protection left, and these are deliberately the PRE-worker numbers: that path+ * behaves exactly as the app did before layout moved off the main thread, RIGHT+ * exemption included. Without this, the raised ceiling above would let a dense+ * graph freeze the tab for minutes on the fallback path.+ */+export const MAX_MAIN_THREAD_LAYOUT_EDGES = 250;+export const MAX_MAIN_THREAD_LAYOUT_NODES = 500;++/** True when a request is too big to lay out on the calling thread. */+export function exceedsMainThreadBudget(request: GraphLayoutRequest): boolean {+ return (+ request.direction === "DOWN" &&+ (request.nodes.length > MAX_MAIN_THREAD_LAYOUT_NODES ||+ request.edges.length > MAX_MAIN_THREAD_LAYOUT_EDGES)+ );+}/*** Distinct, self-loop-free edges keyed by (from, to). The aggregated graph@@ -124,14 +162,11 @@ export function dedupeEdges(return out;}-/** Map our {nodes, edges} into an ELK graph. Edges are pre-deduped by the caller. */-function buildElkGraph(- graph: GraphCanvasData,- dedupedEdges: GraphCanvasData["edges"],- counterReserve: Map<string, number>,- direction: GraphLayoutDirection,-): ElkNode {- const children = graph.nodes.map((node) => {+/** Map a layout request into an ELK graph. Edges are pre-deduped by the caller. */+function buildElkGraph(request: GraphLayoutRequest): ElkNode {+ const { nodes, edges: dedupedEdges, direction } = request;+ const counterReserve = new Map(request.counterReserve);+ const children = nodes.map((node) => {const { width, height } = measureNode(node,counterReserve.get(node.id) ?? 0,@@ -177,7 +212,7 @@ function buildElkGraph(// rows near the panel's aspect ratio, so fit-zoom stays readable// instead of shrinking a 1ΓN ribbon to nothing (bounded β see// MAX_WRAP_NODES).- ...(direction === "RIGHT" && graph.nodes.length <= MAX_WRAP_NODES+ ...(direction === "RIGHT" && nodes.length <= MAX_WRAP_NODES? {"elk.layered.wrapping.strategy": "MULTI_EDGE","elk.aspectRatio": "1.6",@@ -192,9 +227,9 @@ function buildElkGraph(let elkInstance: Promise<ELK> | null = null;/**- * Lazy-load ELK (~1MB) only when a graph is rendered, and reuse the instance.- * Runs on the main thread for now; this is the seam to move into a Web Worker- * for very large graphs without touching callers.+ * Lazy-load ELK (~1MB) and reuse the instance. Only the main-thread fallback+ * path uses this β the worker imports elkjs directly (see+ * `workers/elk-layout.worker.ts`).*/function getElk(): Promise<ELK> {if (!elkInstance) {@@ -217,62 +252,90 @@ function getElk(): Promise<ELK> {*/function buildCounterReserve(nodeToObservationsMap: Record<string, string[]>,-): Map<string, number> {- const reserve = new Map<string, number>();+): [string, number][] {+ const reserve: [string, number][] = [];for (const [id, observations] of Object.entries(nodeToObservationsMap)) {if (observations.length > 1) {- reserve.set(+ reserve.push([id,` (${observations.length}/${observations.length})`.length,- );+ ]);}}return reserve;}-/** Compute a deterministic hierarchical layout for the graph via ELK. */-export async function computeGraphLayout(+/**+ * Decided without running ELK: either the finished layout (empty graph, or one+ * refused by the budget) or the request to hand to ELK β on the worker thread if+ * one is available. All of this is O(nodes + edges) and safe on the main thread.+ */+export type PreparedGraphLayout =+ | { kind: "layout"; layout: GraphLayout }+ | { kind: "request"; request: GraphLayoutRequest };++export function prepareGraphLayout(graph: GraphCanvasData,nodeToObservationsMap: Record<string, string[]> = {},direction: GraphLayoutDirection = "DOWN",-): Promise<GraphLayout> {+): PreparedGraphLayout {if (graph.nodes.length === 0) {- return { nodes: [], edges: [], width: 0, height: 0 };+ return {+ kind: "layout",+ layout: { nodes: [], edges: [], width: 0, height: 0 },+ };}const dedupedEdges = dedupeEdges(graph.edges);// Budget gate (aggregated DOWN layout only β see MAX_GRAPH_LAYOUT_* and the- // RIGHT-path exemption). elkjs is synchronous and uninterruptible, so a graph- // past the budget is refused BEFORE the layout call β the only way to avoid a- // multi-minute main-thread freeze / stack overflow.+ // RIGHT-path exemption). Past the budget the layout is refused up front rather+ // than started: ELK is uninterruptible even in a worker, and at these+ // densities it would run for minutes and blow through memory.if (direction === "DOWN" &&(graph.nodes.length > MAX_GRAPH_LAYOUT_NODES ||dedupedEdges.length > MAX_GRAPH_LAYOUT_EDGES)) {return {- nodes: [],- edges: [],- width: 0,- height: 0,- tooLarge: true,- nodeCount: graph.nodes.length,- edgeCount: dedupedEdges.length,+ kind: "layout",+ layout: {+ nodes: [],+ edges: [],+ width: 0,+ height: 0,+ tooLarge: true,+ nodeCount: graph.nodes.length,+ edgeCount: dedupedEdges.length,+ },};}- const elk = await getElk();- const counterReserve = buildCounterReserve(nodeToObservationsMap);- // Defensive guard around the synchronous elkjs call: a graph that slips past- // the budget could still throw (e.g. RangeError "too much recursion"). Rethrow- // so the renderer surfaces a recoverable error state instead of an unhandled- // rejection.+ return {+ kind: "request",+ request: {+ nodes: graph.nodes,+ edges: dedupedEdges,+ counterReserve: buildCounterReserve(nodeToObservationsMap),+ direction,+ },+ };+}++/**+ * Run ELK on a prepared request. Pure: no DOM, no bundler-specific imports β it+ * runs identically in the layout worker and on the main thread.+ */+export async function runGraphLayout(+ elk: ELK,+ request: GraphLayoutRequest,+): Promise<GraphLayout> {+ // Defensive guard: elkjs can still throw on a graph inside the budget (e.g.+ // RangeError "too much recursion"). Rethrow a real Error so the caller β the+ // worker's message handler or the renderer β surfaces a recoverable state.let result: ElkNode;try {- result = await elk.layout(- buildElkGraph(graph, dedupedEdges, counterReserve, direction),- );+ result = await elk.layout(buildElkGraph(request));} catch (error) {throw error instanceof Error ? error : new Error(String(error));}@@ -310,3 +373,27 @@ export async function computeGraphLayout(height: result.height ?? 0,};}++/**+ * Lay a prepared request out on the CALLING thread β the fallback for browsers+ * without Web Workers, and for a worker that failed to load.+ */+export async function layoutGraphOnThisThread(+ request: GraphLayoutRequest,+): Promise<GraphLayout> {+ return runGraphLayout(await getElk(), request);+}++/**+ * Whole layout pipeline on the calling thread. The renderer goes through the+ * worker client instead; this is what the layout tests exercise.+ */+export async function computeGraphLayout(+ graph: GraphCanvasData,+ nodeToObservationsMap: Record<string, string[]> = {},+ direction: GraphLayoutDirection = "DOWN",+): Promise<GraphLayout> {+ const prepared = prepareGraphLayout(graph, nodeToObservationsMap, direction);+ if (prepared.kind === "layout") return prepared.layout;+ return layoutGraphOnThisThread(prepared.request);+}diff --git a/web/src/features/trace-graph-view/layout/graphLayoutWorkerClient.clienttest.ts b/web/src/features/trace-graph-view/layout/graphLayoutWorkerClient.clienttest.tsnew file mode 100644index 0000000000..59d4c74dda--- /dev/null+++ b/web/src/features/trace-graph-view/layout/graphLayoutWorkerClient.clienttest.ts@@ -0,0 +1,177 @@+import { type GraphCanvasData, type GraphNodeData } from "../types";+import { MAX_MAIN_THREAD_LAYOUT_NODES } from "./elkLayout";++const node = (id: string): GraphNodeData => ({ id, label: id, type: "AGENT" });++const graph: GraphCanvasData = {+ nodes: [node("a"), node("b")],+ edges: [+ { from: "a", to: "b" },+ { from: "a", to: "b" }, // deduped before the post+ ],+};++/** An ELK answer for `graph`, as elkjs's worker would post it back. */+const laidOut = (width: number) => ({+ id: "root",+ width,+ height: 128,+ children: [+ { id: "a", x: 0, y: 0, width: 96, height: 34 },+ { id: "b", x: 0, y: 94, width: 96, height: 34 },+ ],+ edges: [+ {+ id: "edge-0",+ sources: ["a"],+ targets: ["b"],+ sections: [{ startPoint: { x: 48, y: 34 }, endPoint: { x: 48, y: 94 } }],+ },+ ],+});++type ElkMessage = {+ id: number;+ cmd: string;+ graph?: { edges?: unknown[]; layoutOptions?: Record<string, string> };+};++/** Stands in for the bundled elkjs worker: same message protocol, no layout. */+class FakeWorker {+ static instances: FakeWorker[] = [];+ onmessage: ((event: MessageEvent) => void) | null = null;+ onerror: ((event: ErrorEvent) => void) | null = null;+ posted: ElkMessage[] = [];+ terminated = false;++ constructor() {+ FakeWorker.instances.push(this);+ }++ postMessage(message: ElkMessage) {+ this.posted.push(message);+ }++ terminate() {+ this.terminated = true;+ }++ get layoutRequests() {+ return this.posted.filter((m) => m.cmd === "layout");+ }++ answer(message: ElkMessage, data: unknown) {+ this.onmessage?.({ data: { id: message.id, data } } as MessageEvent);+ }+}++// The client keeps the worker in module state, so every test needs its own copy.+async function loadClient() {+ vi.resetModules();+ return import("./graphLayoutWorkerClient");+}++/** elk-api delivers worker answers through a 0ms timeout. */+const flush = () => new Promise((resolve) => setTimeout(resolve, 0));++describe("requestGraphLayout", () => {+ beforeEach(() => {+ FakeWorker.instances = [];+ vi.stubGlobal("Worker", FakeWorker);+ });++ afterEach(() => {+ vi.useRealTimers();+ vi.unstubAllGlobals();+ });++ it("resolves with the layout the worker sends back", async () => {+ const client = await loadClient();+ const pending = client.requestGraphLayout(graph, {}, "DOWN");+ const worker = FakeWorker.instances[0];+ await flush();++ const [request] = worker.layoutRequests;+ // Duplicate edges are deduped before the post β the payload stays minimal.+ expect(request.graph?.edges).toHaveLength(1);+ expect(request.graph?.layoutOptions?.["elk.direction"]).toBe("DOWN");++ worker.answer(request, laidOut(200));+ await expect(pending).resolves.toMatchObject({+ width: 200,+ edges: [{ source: "a", target: "b" }],+ });+ });++ it("drops a cancelled request's result instead of landing it on the next graph", async () => {+ const client = await loadClient();+ const controller = new AbortController();+ const stale = client+ .requestGraphLayout(graph, {}, "DOWN", controller.signal)+ .catch((error: Error) => error);+ const staleWorker = FakeWorker.instances[0];+ await flush();+ const [staleRequest] = staleWorker.layoutRequests;++ controller.abort();+ expect(await stale).toBeInstanceOf(client.GraphLayoutCancelledError);+ // Cancelling always kills the worker, however young the run is: ELK cannot be+ // interrupted, so anything left running would hold the one worker thread with+ // no deadline and starve the request that replaced it.+ expect(staleWorker.terminated).toBe(true);++ const fresh = client.requestGraphLayout(graph, {}, "RIGHT");+ const freshWorker = FakeWorker.instances[1];+ await flush();++ // A late answer from the killed worker must not become the current layout.+ staleWorker.answer(staleRequest, laidOut(1));+ freshWorker.answer(freshWorker.layoutRequests[0], laidOut(2));+ await expect(fresh).resolves.toMatchObject({ width: 2 });+ });++ it("gives up with the too-large notice when a layout blows the deadline", async () => {+ vi.useFakeTimers();+ const client = await loadClient();+ const pending = client.requestGraphLayout(graph, {}, "DOWN");+ const worker = FakeWorker.instances[0];++ vi.advanceTimersByTime(client.GRAPH_LAYOUT_DEADLINE_MS);++ await expect(pending).resolves.toMatchObject({+ tooLarge: true,+ nodeCount: 2,+ edgeCount: 1,+ });+ // ELK cannot be interrupted β the worker has to die for the layout to stop.+ expect(worker.terminated).toBe(true);+ });++ it("lays out on the calling thread when the browser has no Worker", async () => {+ vi.stubGlobal("Worker", undefined);+ const client = await loadClient();++ const layout = await client.requestGraphLayout(graph, {}, "DOWN");+ expect(layout.tooLarge).toBeFalsy();+ expect(layout.nodes).toHaveLength(2);+ });++ it("refuses a graph the main thread cannot afford when there is no Worker", async () => {+ // No deadline can apply to synchronous ELK, so the pre-worker counts are the+ // only protection on that path β refuse instead of freezing the tab.+ vi.stubGlobal("Worker", undefined);+ const client = await loadClient();+ const wide: GraphCanvasData = {+ nodes: Array.from({ length: MAX_MAIN_THREAD_LAYOUT_NODES + 1 }, (_, i) =>+ node(`n${i}`),+ ),+ edges: [{ from: "n0", to: "n1" }],+ };++ const started = Date.now();+ const layout = await client.requestGraphLayout(wide, {}, "DOWN");+ expect(Date.now() - started).toBeLessThan(1000); // ELK never ran+ expect(layout.tooLarge).toBe(true);+ expect(layout.nodeCount).toBe(MAX_MAIN_THREAD_LAYOUT_NODES + 1);+ });+});diff --git a/web/src/features/trace-graph-view/layout/graphLayoutWorkerClient.ts b/web/src/features/trace-graph-view/layout/graphLayoutWorkerClient.tsnew file mode 100644index 0000000000..7513b52a5b--- /dev/null+++ b/web/src/features/trace-graph-view/layout/graphLayoutWorkerClient.ts@@ -0,0 +1,220 @@+import ELKConstructor from "elkjs/lib/elk-api";+import type { ELK } from "elkjs";++import { reportError } from "@/src/utils/reportError";+import { reportWorkerLoadError } from "@/src/utils/reportWorkerLoadError";++import { type GraphCanvasData } from "../types";+import {+ exceedsMainThreadBudget,+ layoutGraphOnThisThread,+ prepareGraphLayout,+ runGraphLayout,+ type GraphLayout,+ type GraphLayoutDirection,+ type GraphLayoutRequest,+} from "./elkLayout";++/**+ * Wall-clock ceiling for one layout, and the real layout budget: cost depends far+ * more on graph SHAPE than on size (measured with the app's layout options, same+ * counts: a graph whose edges concentrate on a few nodes lays out 60 nodes / 300+ * edges in ~0.35s, while a uniformly dense one of the same counts takes ~16s), so+ * no node/edge count can predict it. ELK can't be interrupted even in a worker, so+ * the deadline is enforced by terminating the worker and showing the "too large"+ * notice. See MAX_GRAPH_LAYOUT_* for the count ceiling that stays.+ */+export const GRAPH_LAYOUT_DEADLINE_MS = 60_000;++type PendingLayout = {+ id: string;+ request: GraphLayoutRequest;+ timer: ReturnType<typeof setTimeout>;+ resolve: (layout: GraphLayout) => void;+ reject: (error: Error) => void;+};++/** ELK plus the worker it runs in β we own the worker so we can terminate it. */+type LayoutWorker = { elk: ELK; worker: Worker };++const pending = new Map<string, PendingLayout>();+let active: LayoutWorker | null = null;+/** Set when the worker script itself won't load β degrade to the main thread. */+let workerUnavailable = false;+let requestCounter = 0;++export class GraphLayoutCancelledError extends Error {+ constructor() {+ super("Graph layout cancelled");+ this.name = "GraphLayoutCancelledError";+ }+}++/** The "too large to lay out" answer, carrying the counts the notice shows. */+function refusedLayout(request: GraphLayoutRequest): GraphLayout {+ return {+ nodes: [],+ edges: [],+ width: 0,+ height: 0,+ tooLarge: true,+ nodeCount: request.nodes.length,+ edgeCount: request.edges.length,+ };+}++/**+ * The no-worker path. Refuse what the main thread cannot afford β no deadline can+ * apply to synchronous ELK β and otherwise lay out inline.+ */+function layoutWithoutWorker(+ request: GraphLayoutRequest,+): Promise<GraphLayout> {+ if (exceedsMainThreadBudget(request)) {+ return Promise.resolve(refusedLayout(request));+ }+ return layoutGraphOnThisThread(request);+}++function handleWorkerError(event: ErrorEvent) {+ // `onerror` means the worker SCRIPT failed β a stale chunk after a deploy is+ // the dominant cause. (A layout that throws rejects its own promise instead.)+ // Retire the worker and lay out on the main thread: slow beats no graph at all.+ workerUnavailable = true;+ reportWorkerLoadError({+ area: "graph-layout-worker",+ source: "ELK layout",+ event,+ });+ restartWorker();+}++function getLayoutWorker(): LayoutWorker | null {+ if (workerUnavailable) return null;+ if (typeof window === "undefined" || !window.Worker) return null;+ if (active) return active;+ try {+ const worker = new Worker(+ new URL("@/src/workers/elk-layout.worker.ts", import.meta.url),+ );+ worker.onerror = handleWorkerError;+ // elk-api only sets `onmessage` on the worker we hand it, so the instance+ // above stays ours to terminate.+ const elk = new ELKConstructor({ workerFactory: () => worker });+ active = { elk, worker };+ } catch (error) {+ workerUnavailable = true;+ active = null;+ reportError(error, {+ area: "graph-layout-worker",+ warnMessage: "could not start the ELK layout worker",+ });+ return null;+ }+ return active;+}++/** Run one request on `target`, tracked so it can be cancelled or timed out. */+function dispatch(entry: PendingLayout, target: LayoutWorker) {+ pending.set(entry.id, entry);+ runGraphLayout(target.elk, entry.request).then(+ (layout) => settle(entry.id, (found) => found.resolve(layout)),+ (error: unknown) =>+ settle(entry.id, (found) =>+ found.reject(error instanceof Error ? error : new Error(String(error))),+ ),+ );+}++/**+ * Answer a request β unless it is no longer pending, which is the staleness+ * guard: the worker outlives individual callers, so a result that arrives after+ * its cancellation or deadline must never land on a newer graph.+ */+function settle(id: string, finish: (entry: PendingLayout) => void) {+ const entry = pending.get(id);+ if (!entry) return;+ pending.delete(id);+ clearTimeout(entry.timer);+ finish(entry);+}++/**+ * Terminate the worker β the only way to stop an in-flight ELK run β and re-issue+ * whatever else was queued behind it on a fresh one (it was waiting on a thread+ * that no longer exists).+ */+function restartWorker() {+ const carried = [...pending.values()];+ pending.clear();+ active?.worker.terminate();+ active = null;+ if (carried.length === 0) return;+ const target = getLayoutWorker();+ for (const entry of carried) {+ if (target) dispatch(entry, target);+ else layoutWithoutWorker(entry.request).then(entry.resolve, entry.reject);+ }+}++function cancel(id: string) {+ const entry = pending.get(id);+ if (!entry) return; // already answered+ pending.delete(id);+ clearTimeout(entry.timer);+ // ALWAYS terminate, however young the run is. Letting a just-started layout+ // finish looks cheaper β it saves a worker startup β but its duration is not+ // knowable at cancel time, and dropping the entry also drops its deadline: a+ // 60s layout cancelled after 100ms would keep the single worker thread busy+ // with no deadline at all, and the request that replaced it would sit behind+ // the zombie until ITS deadline reported a false "too large".+ restartWorker();+ entry.reject(new GraphLayoutCancelledError());+}++function onDeadline(id: string) {+ const entry = pending.get(id);+ if (!entry) return;+ pending.delete(id);+ restartWorker();+ entry.resolve(refusedLayout(entry.request));+}++/**+ * Lay the graph out in the ELK worker, so the rest of the trace view (tree, tabs,+ * navigation) stays interactive while ELK grinds.+ *+ * Abort via `signal` when the request goes stale (trace, view mode or direction+ * changed): the promise rejects with {@link GraphLayoutCancelledError} and a late+ * result can never land on the newer graph.+ */+export function requestGraphLayout(+ graph: GraphCanvasData,+ nodeToObservationsMap: Record<string, string[]> = {},+ direction: GraphLayoutDirection = "DOWN",+ signal?: AbortSignal,+): Promise<GraphLayout> {+ // Dedupe, size and gate before touching the worker: all O(nodes + edges), and+ // it keeps the posted payload to the deduped graph β postMessage's structured+ // clone is main-thread work too.+ const prepared = prepareGraphLayout(graph, nodeToObservationsMap, direction);+ if (prepared.kind === "layout") return Promise.resolve(prepared.layout);++ const target = getLayoutWorker();+ if (!target) return layoutWithoutWorker(prepared.request);++ return new Promise<GraphLayout>((resolve, reject) => {+ const id = String(++requestCounter);+ dispatch(+ {+ id,+ request: prepared.request,+ timer: setTimeout(() => onDeadline(id), GRAPH_LAYOUT_DEADLINE_MS),+ resolve,+ reject,+ },+ target,+ );+ signal?.addEventListener("abort", () => cancel(id), { once: true });+ });+}diff --git a/web/src/hooks/parserWorkerError.ts b/web/src/hooks/parserWorkerError.tsindex b6657a71e0..7719a91332 100644--- a/web/src/hooks/parserWorkerError.ts+++ b/web/src/hooks/parserWorkerError.ts@@ -1,52 +1,24 @@-import { reportError } from "@/src/utils/reportError";+import { reportWorkerLoadError } from "@/src/utils/reportWorkerLoadError";/**- * Turn a JSON-parser Web Worker `onerror` ErrorEvent into a legible,- * Sentry-reportable error.+ * Report a JSON-parser Web Worker `onerror` for the hook that owns it.*- * `Worker.onerror` fires when the worker *script* itself fails to load or throws- * during init β the dominant cause is a stale-deploy chunk 404. The previous- * handlers did `console.error("... Worker error:", event)` with the raw- * ErrorEvent; Sentry's captureConsoleIntegration stringifies that to- * "[object ErrorEvent]", discarding message/filename/lineno and making the- * resulting issues undiagnosable.+ * The extraction itself is shared β see {@link reportWorkerLoadError} for why a+ * raw ErrorEvent must never be logged. This only pins the parser's `area` tag and+ * the `workerHook` context key.*- * This extracts the real fields and reports one proper Error with structured- * context through the {@link reportError} seam, which tags `area` and logs the- * companion line via `console.warn` (NOT `console.error`, so- * captureConsoleIntegration β which captures the "error" level β does not- * double-capture a second, opaque event for the same failure).- *- * Note: in-worker parse failures are a separate, already-legible path β they- * post a "Parse error" string back over the message channel and are handled in- * the message callback, not here.+ * Note: in-worker parse failures are a separate, already-legible path β they post+ * a "Parse error" string back over the message channel and are handled in the+ * message callback, not here.*/export function reportParserWorkerError(hookName: string,event: ErrorEvent,): void {- const details = `${event.message || "unknown"} @ ${event.filename || "?"}:${event.lineno ?? "?"}:${event.colno ?? "?"}`;-- // Prefer the real Error when the worker threw during init; otherwise- // synthesize one from the ErrorEvent's fields so the message is legible.- const realError =- event.error instanceof Error- ? event.error- : new Error(`[${hookName}] worker failed to load: ${details}`);-- reportError(realError, {+ reportWorkerLoadError({area: "io-parse-worker",- extra: {- workerHook: hookName,- message: event.message,- filename: event.filename,- lineno: event.lineno,- colno: event.colno,- },- // Companion console line: carry hookName + location details exactly once- // in both branches β the pass-through Error's message lacks them, the- // synthesized one already embeds them. Console-only; the captured event- // is the Error above.- warnMessage: `${hookName} worker failed to load: ${details}`,+ source: hookName,+ event,+ extra: { workerHook: hookName },});}diff --git a/web/src/utils/reportWorkerLoadError.ts b/web/src/utils/reportWorkerLoadError.tsnew file mode 100644index 0000000000..58731301ec--- /dev/null+++ b/web/src/utils/reportWorkerLoadError.ts@@ -0,0 +1,58 @@+import { reportError } from "@/src/utils/reportError";++/**+ * Turn a Web Worker `onerror` ErrorEvent into one legible, Sentry-reportable+ * error.+ *+ * `Worker.onerror` fires when the worker *script* itself fails to load or throws+ * during init β the dominant cause is a stale-deploy chunk 404. Handlers used to+ * do `console.error("... Worker error:", event)` with the raw ErrorEvent; Sentry's+ * captureConsoleIntegration stringifies that to "[object ErrorEvent]", discarding+ * message/filename/lineno and making the resulting issues undiagnosable.+ *+ * This extracts the real fields and reports one proper Error with structured+ * context through the {@link reportError} seam, which tags `area` and logs the+ * companion line via `console.warn` (NOT `console.error`, so+ * captureConsoleIntegration β which captures the "error" level β does not+ * double-capture a second, opaque event for the same failure).+ *+ * One implementation for every worker: each caller supplies its `area` tag, the+ * `source` name the message should read as, and any extra context.+ */+export function reportWorkerLoadError({+ area,+ source,+ event,+ extra,+}: {+ area: string;+ /** What failed, as it should read in the message: a hook or worker name. */+ source: string;+ event: ErrorEvent;+ extra?: Record<string, unknown>;+}): void {+ const details = `${event.message || "unknown"} @ ${event.filename || "?"}:${event.lineno ?? "?"}:${event.colno ?? "?"}`;++ // Prefer the real Error when the worker threw during init; otherwise+ // synthesize one from the ErrorEvent's fields so the message is legible.+ const realError =+ event.error instanceof Error+ ? event.error+ : new Error(`[${source}] worker failed to load: ${details}`);++ reportError(realError, {+ area,+ extra: {+ ...extra,+ message: event.message,+ filename: event.filename,+ lineno: event.lineno,+ colno: event.colno,+ },+ // Companion console line: carry source + location details exactly once in+ // both branches β the pass-through Error's message lacks them, the+ // synthesized one already embeds them. Console-only; the captured event is+ // the Error above.+ warnMessage: `${source} worker failed to load: ${details}`,+ });+}diff --git a/web/src/workers/elk-layout.worker.ts b/web/src/workers/elk-layout.worker.tsnew file mode 100644index 0000000000..dd4dc2212e--- /dev/null+++ b/web/src/workers/elk-layout.worker.ts@@ -0,0 +1,12 @@+/**+ * Worker entry for trace-graph ELK layout.+ *+ * elkjs ships its own worker build: loaded without a `document` (i.e. inside a+ * Worker) `elk-worker.min.js` installs its own `self.onmessage` and speaks+ * elk-api's message protocol, so hosting it IS the whole worker β the main thread+ * talks to it through `new ELK({ workerFactory })`.+ *+ * `elk.bundled.js` is NOT usable here: it takes the same branch, then exports no+ * in-process worker class, so constructing ELK inside a worker throws.+ */+import "elkjs/lib/elk-worker.min.js";