🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
TypeScript analyticsautogenevaluationlangchainlarge-language-modelsllama-indexllmllm-evaluationllm-observabilityllmopsmonitoringobservabilityopen-sourceopenaiplaygroundprompt-engineeringprompt-managementself-hostedycombinator
fix(sessions): render Langfuse media tokens inline in the session view (LFE-14815, LFE-9577) (#15882)
Nikita Kabardin6cc71f544b50
diff --git a/.agents/skills/seed-test-data/SKILL.md b/.agents/skills/seed-test-data/SKILL.mdindex f5097b71ec..ea9ce4ba2e 100644--- a/.agents/skills/seed-test-data/SKILL.md+++ b/.agents/skills/seed-test-data/SKILL.md@@ -26,28 +26,29 @@ this.## Need → command-| I need... | Command |-| --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |-| A very complex observation tree (v3) | `pnpm run seed -- trace-tree --observations 5000 --depth 12 --breadth 500` |-| The same tree readable in the v4 events UI | add `--v4` (writes `events_full`; `events_core` fills via MV) |-| Async parents whose subtree outlives their own span (subtree wall-clock duration badge) | add `--async-parents` to `trace-tree` (root + hub end immediately while children keep running) |-| A realistic agent flow over a timeline (graph view + scrubbable timeline) | `pnpm run seed -- agent-timeline --turns 6 --v4` (LangGraph refine loop planner→retriever→generator→critic→loop, staggered in time; add `--timing-only` for the pure timing fallback) |-| A demo-grade, real-looking agent trace (videos, screenshots, docs) | `pnpm run seed -- support-agent --v4 --id-prefix <hex>` (one fixed, fully handcrafted support-copilot refund run: guardrails, parallel context fan-out, 3-turn ReAct loop with real payloads/costs; deterministic — reseed with a FRESH prefix for a clean take; the prefix is the trace id, so a hex prefix reads like production) |-| A plain trace with no agentic types (collapsed-by-default graph panel) | add `--plain` to `trace-tree` (SPAN/GENERATION/EVENT only) |-| An extremely DEEP single-chain trace (tree depth = observation count; layout stress) | `pnpm run seed -- deep-chain --v4` (1401 sequential generations, each the sole child of the previous — the mis-parented-instrumentation shape from LFE-10959 that collapses tree/timeline layouts; `--observations N` to change depth) |-| A super tough session (v3 legacy session view) | `pnpm run seed -- long-session --traces 300 --observations-per-trace 8` |-| Diverse v4 session shapes (chat / coding-agent / mixed) for the session-detail view | `pnpm run seed -- session-shapes --shape all` (the `agent` shape has I/O on AGENT/TOOL with no GENERATION — pre-LFE-10520 the "first generation" default rendered empty cards for it; the current "All observations with I/O" default renders it correctly; v4 on by default) |-| Many traces for list/filter performance | `pnpm run seed -- many-traces --count 100000 --days 14` |-| Long-window v4 traffic with cost/latency/token OUTLIERS (outlier chart strip, LFE-14451) | `pnpm run seed -- outlier-traffic --days 90` (diurnal base load + deterministic spikes + hour-long ×8-latency incidents; root AGENT + GENERATION carrying cost + TOOL per trace; v4 on by default) |-| Scores with spaces in the name (filter/grammar testing) | `pnpm run seed -- scored-traces --traces 24 --v4` |-| Lots of scores on every node (dense score badges, tree-row overflow testing) | add `--scores-per-node 12` to `trace-tree` (N distinct scores per observation; try `--depth 2 --breadth 44` for many tall sibling rows) |-| Extra trace tags, incl. mixed case/accents (tag filter ordering) | add `--tags "Zebra,apple,Ärger"` to `trace-tree` (comma-separated, appended to the scenario's own tags) |-| Varied human-annotation queues (annotate UI / keyboard testing) | `pnpm run seed -- annotation-queue --core-items 12` (creates a "core types" queue covering every score-field render path + an "edge cases" queue with archived/stale/partial scores and observation/session/deleted/completed items) |-| Huge/malformed/unicode payloads | `pnpm run seed -- trace-tree --payload-bytes 1000000 --payload-style malformed` (styles: json, text, malformed, unicode, bignum, base64) |-| Big integers beyond 2^53-1 (number-precision testing) | `pnpm run seed -- trace-tree --observations 1 --payload-style bignum` |-| Huge base64 data-URI in ChatML IO (multimodal crash shape, LFE-10152) | `pnpm run seed -- trace-tree --observations 30 --payload-bytes 20000000 --payload-style base64 --v4` (one unbroken multi-MB base64 token in trace + root-observation IO; max 50 MB) |-| See all scenarios and flags | `pnpm run seed -- list --json` |-| Predict without writing | add `--dry-run` |+| I need... | Command |+| ------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |+| A very complex observation tree (v3) | `pnpm run seed -- trace-tree --observations 5000 --depth 12 --breadth 500` |+| The same tree readable in the v4 events UI | add `--v4` (writes `events_full`; `events_core` fills via MV) |+| Async parents whose subtree outlives their own span (subtree wall-clock duration badge) | add `--async-parents` to `trace-tree` (root + hub end immediately while children keep running) |+| A realistic agent flow over a timeline (graph view + scrubbable timeline) | `pnpm run seed -- agent-timeline --turns 6 --v4` (LangGraph refine loop planner→retriever→generator→critic→loop, staggered in time; add `--timing-only` for the pure timing fallback) |+| A demo-grade, real-looking agent trace (videos, screenshots, docs) | `pnpm run seed -- support-agent --v4 --id-prefix <hex>` (one fixed, fully handcrafted support-copilot refund run: guardrails, parallel context fan-out, 3-turn ReAct loop with real payloads/costs; deterministic — reseed with a FRESH prefix for a clean take; the prefix is the trace id, so a hex prefix reads like production) |+| A plain trace with no agentic types (collapsed-by-default graph panel) | add `--plain` to `trace-tree` (SPAN/GENERATION/EVENT only) |+| An extremely DEEP single-chain trace (tree depth = observation count; layout stress) | `pnpm run seed -- deep-chain --v4` (1401 sequential generations, each the sole child of the previous — the mis-parented-instrumentation shape from LFE-10959 that collapses tree/timeline layouts; `--observations N` to change depth) |+| A super tough session (v3 legacy session view) | `pnpm run seed -- long-session --traces 300 --observations-per-trace 8` |+| Diverse v4 session shapes (chat / coding-agent / mixed / media) for the session-detail view | `pnpm run seed -- session-shapes --shape all` (the `agent` shape has I/O on AGENT/TOOL with no GENERATION — pre-LFE-10520 the "first generation" default rendered empty cards for it; the current "All observations with I/O" default renders it correctly; v4 on by default) |+| Session messages carrying Langfuse media references (inline image, several refs in one message, bare refs in a content array, link-only payload) | `pnpm run seed -- session-shapes --shape media` — uploads the image/audio/pdf fixtures to MinIO and links them to the observation, so the inline chip and the "Media" strip both resolve (LFE-14815, LFE-9577) |+| Many traces for list/filter performance | `pnpm run seed -- many-traces --count 100000 --days 14` |+| Long-window v4 traffic with cost/latency/token OUTLIERS (outlier chart strip, LFE-14451) | `pnpm run seed -- outlier-traffic --days 90` (diurnal base load + deterministic spikes + hour-long ×8-latency incidents; root AGENT + GENERATION carrying cost + TOOL per trace; v4 on by default) |+| Scores with spaces in the name (filter/grammar testing) | `pnpm run seed -- scored-traces --traces 24 --v4` |+| Lots of scores on every node (dense score badges, tree-row overflow testing) | add `--scores-per-node 12` to `trace-tree` (N distinct scores per observation; try `--depth 2 --breadth 44` for many tall sibling rows) |+| Extra trace tags, incl. mixed case/accents (tag filter ordering) | add `--tags "Zebra,apple,Ärger"` to `trace-tree` (comma-separated, appended to the scenario's own tags) |+| Varied human-annotation queues (annotate UI / keyboard testing) | `pnpm run seed -- annotation-queue --core-items 12` (creates a "core types" queue covering every score-field render path + an "edge cases" queue with archived/stale/partial scores and observation/session/deleted/completed items) |+| Huge/malformed/unicode payloads | `pnpm run seed -- trace-tree --payload-bytes 1000000 --payload-style malformed` (styles: json, text, malformed, unicode, bignum, base64) |+| Big integers beyond 2^53-1 (number-precision testing) | `pnpm run seed -- trace-tree --observations 1 --payload-style bignum` |+| Huge base64 data-URI in ChatML IO (multimodal crash shape, LFE-10152) | `pnpm run seed -- trace-tree --observations 30 --payload-bytes 20000000 --payload-style base64 --v4` (one unbroken multi-MB base64 token in trace + root-observation IO; max 50 MB) |+| See all scenarios and flags | `pnpm run seed -- list --json` |+| Predict without writing | add `--dry-run` |## Contractdiff --git a/packages/shared/scripts/seeder/AGENTS.md b/packages/shared/scripts/seeder/AGENTS.mdindex a16c35632d..32999ab1fb 100644--- a/packages/shared/scripts/seeder/AGENTS.md+++ b/packages/shared/scripts/seeder/AGENTS.md@@ -15,7 +15,8 @@ pnpm run seed -- deep-chain --v4 # 1401 sequential generations in ONE parent chpnpm run seed -- agent-timeline --turns 6 --v4 # realistic agent flow-with-loop over a timeline (graph view)pnpm run seed -- support-agent --v4 --id-prefix <hex> # demo-grade handcrafted support-copilot run (videos/screenshots)pnpm run seed -- long-session --traces 300 --observations-per-trace 8-pnpm run seed -- session-shapes --shape all # chat / coding-agent / mixed v4 sessions+pnpm run seed -- session-shapes --shape all # chat / coding-agent / mixed / media v4 sessions+pnpm run seed -- session-shapes --shape media # messages carrying @@@langfuseMedia:...@@@ refs (needs MinIO)pnpm run seed -- many-traces --count 100000 --days 14pnpm run seed -- outlier-traffic --days 90 # diurnal v4 traffic w/ cost/latency/token outliers (outlier chart strip)pnpm run seed -- scored-traces --traces 24 --v4 # scores w/ spaces in the namediff --git a/packages/shared/scripts/seeder/scenarios/session-shapes.ts b/packages/shared/scripts/seeder/scenarios/session-shapes.tsindex 2284c07410..9ca83ac0e3 100644--- a/packages/shared/scripts/seeder/scenarios/session-shapes.ts+++ b/packages/shared/scripts/seeder/scenarios/session-shapes.ts@@ -12,6 +12,11 @@ import {ScoreRecordInsertType,TraceRecordInsertType,} from "../../../src/server";+import {+ ensureSeedMediaUploaded,+ linkSeedMediaToObservation,+ type SeedMediaFixture,+} from "../seed-media";import { observationToEvent, traceToEvent } from "./event-mirror";import { jitter, Rng, utcDayStartMs } from "./rng";import {@@ -39,14 +44,18 @@ import { countRows, sessionLink } from "./verify";* - agent a coding/agent session: I/O on the root AGENT + TOOL children,* NO GENERATION — the shape the old default rendered empty.* - mixed alternating chat/agent turns.+ * - media messages carrying `@@@langfuseMedia:…@@@` references (LFE-14815):+ * an inline image, several references in one message, and a+ * link-only payload the chat renderer cannot inline. Uploads its+ * assets and links them to the observation.** Each shape is written as its own session (id `<prefix>-<shape>`) so a single* `--shape all` run hands back one session link per shape.*/-type Shape = "chat" | "agent" | "mixed";+type Shape = "chat" | "agent" | "mixed" | "media";-const SHAPES: readonly Shape[] = ["chat", "agent", "mixed"];+const SHAPES: readonly Shape[] = ["chat", "agent", "mixed", "media"];const CHAT_TURNS: ReadonlyArray<{ user: string; assistant: string }> = [{@@ -137,6 +146,78 @@ const buildChatMessages = (turnIdx: number): string => {return JSON.stringify({ messages });};+/**+ * The media-bearing message shapes behind LFE-14815 / LFE-13602 / LFE-9577:+ *+ * 0 inline-image one `image_url` part holding a `@@@langfuseMedia:…@@@`+ * reference — the customer's shape; the image must render+ * inline, and NOT also in the "Media" strip (deduped).+ * 1 multi-ref two references in one message (image + audio).+ * 2 linked-only a non-ChatML payload the chat renderer cannot inline, with+ * the media linked to the observation — the "Media" strip is+ * the only surface, so it proves session/trace-detail parity.+ * 3 bare-refs content is an ARRAY of bare reference strings, with no+ * `{type, …}` wrapper at all (LFE-9577).+ * 4 collapsed-prompt a system prompt long enough to collapse by default that+ * also carries an image part. Collapsing hides long TEXT, so+ * the attachment must still render — the strip dedupes it as+ * "rendered inline", so a collapsed render that skipped it+ * would hide the asset in both places.+ */+type MediaVariant =+ | "inline-image"+ | "multi-ref"+ | "linked-only"+ | "bare-refs"+ | "collapsed-prompt";++const MEDIA_VARIANTS: readonly MediaVariant[] = [+ "inline-image",+ "multi-ref",+ "linked-only",+ "bare-refs",+ "collapsed-prompt",+];++/** Comfortably past the ~250-char / 4-line default collapse threshold. */+const LONG_SYSTEM_PROMPT = [+ "You are an HVAC troubleshooting assistant for residential air-conditioning systems.",+ "Always inspect any attached media before answering, and describe what you actually see.",+ "Prefer the cheapest safe diagnostic first: condensate drain, then coil, then blower, then wiring.",+ "Never instruct the homeowner to open an electrical panel; escalate to a technician instead.",+ "Close every answer with one concrete next step the homeowner can take today.",+].join("\n");++type MediaFixtures = Record<"image" | "audio" | "pdf", SeedMediaFixture>;++const MEDIA_PROMPTS: Record<MediaVariant, { user: string; assistant: string }> =+ {+ "inline-image": {+ user: "My AC has rust in it. I've attached an image of the air handler — what am I looking at?",+ assistant:+ "That rust is most likely from moisture sitting in the air handler — usually a condensate drain problem rather than the thermostat wiring. Want me to walk you through the quick checks?",+ },+ "multi-ref": {+ user: "Here's a photo of the unit and a recording of the noise it makes on startup.",+ assistant:+ "Thanks — the photo shows surface rust below the coil, and the startup noise is consistent with a failing blower bearing. I'd book a technician for the blower.",+ },+ "linked-only": {+ user: "Filing the attachment against the work order.",+ assistant: "Attachment stored against work order #4821.",+ },+ "bare-refs": {+ user: "Attaching the unit photo and the inspection report.",+ assistant:+ "Received both. The report confirms the drain pan was replaced last year, so the rust is recent — I'd check the condensate line first.",+ },+ "collapsed-prompt": {+ user: "Photo of the coil as requested.",+ assistant:+ "Thanks — the coil looks iced at the inlet, which points at low airflow rather than a refrigerant leak. Next step: replace the filter and run the fan for an hour.",+ },+ };+const microPrice = (tokens: number, rate: number) => tokens * rate;/**@@ -434,16 +515,214 @@ const buildAgentTrace = (return { trace, observations };};+/**+ * Builds one media-bearing trace: a root AGENT plus a GENERATION whose input+ * carries a Langfuse media reference. Also reports the observation/field pairs+ * to link in `observation_media` so the "Media" strip resolves.+ */+const buildMediaTrace = (+ ctx: ScenarioContext,+ rng: Rng,+ sessionId: string,+ traceId: string,+ turnIdx: number,+ timestamp: number,+ fixtures: MediaFixtures,+): {+ trace: TraceRecordInsertType;+ observations: ObservationRecordInsertType[];+ mediaLinks: { observationId: string; mediaId: string }[];+} => {+ const variant = MEDIA_VARIANTS[turnIdx % MEDIA_VARIANTS.length];+ const prompt = MEDIA_PROMPTS[variant];+ const genId = `${traceId}-o1`;++ const imagePart = {+ type: "image_url",+ image_url: { url: fixtures.image.referenceString },+ };++ const userContent =+ variant === "bare-refs"+ ? [fixtures.image.referenceString, fixtures.pdf.referenceString]+ : variant === "collapsed-prompt"+ ? prompt.user+ : variant === "inline-image"+ ? [{ type: "text", text: prompt.user }, imagePart]+ : [+ { type: "text", text: prompt.user },+ imagePart,+ {+ type: "input_audio",+ input_audio: { data: fixtures.audio.referenceString },+ },+ ];++ // collapsed-prompt hangs the attachment off the long SYSTEM message, which is+ // the message the UI collapses by default.+ const systemContent =+ variant === "collapsed-prompt"+ ? [{ type: "text", text: LONG_SYSTEM_PROMPT }, imagePart]+ : "You are an HVAC troubleshooting assistant. Inspect attached media before answering.";++ // linked-only stays deliberately non-ChatML: the chat renderer cannot inline+ // it, so only the media strip can surface the asset.+ const genInput =+ variant === "linked-only"+ ? JSON.stringify({+ work_order: "4821",+ note: prompt.user,+ attachment: {+ mediaId: fixtures.image.mediaId,+ contentType: fixtures.image.contentType,+ referenceString: fixtures.image.referenceString,+ },+ })+ : JSON.stringify({+ messages: [+ { role: "system", content: systemContent },+ { role: "user", content: userContent },+ ],+ });++ const mediaLinks = [+ { observationId: genId, mediaId: fixtures.image.mediaId },+ ];+ if (variant === "multi-ref") {+ mediaLinks.push({ observationId: genId, mediaId: fixtures.audio.mediaId });+ }+ if (variant === "bare-refs") {+ mediaLinks.push({ observationId: genId, mediaId: fixtures.pdf.mediaId });+ }++ const trace = createTrace({+ id: traceId,+ project_id: ctx.projectId,+ environment: ctx.environment,+ session_id: sessionId,+ timestamp,+ name: "hvac-troubleshooting-agent",+ user_id: `user-${ctx.idPrefix}`,+ release: "v2.0.0",+ version: "v2.0.0",+ tags: ["seed", "session-shapes", "media"],+ public: false,+ bookmarked: false,+ metadata: {+ scenario: "session-shapes",+ shape: "media",+ variant,+ turn: String(turnIdx),+ },+ input: prompt.user,+ output: prompt.assistant,+ created_at: Date.now(),+ updated_at: Date.now(),+ event_ts: Date.now(),+ });++ const rootStart = timestamp + jitter(ctx.seed, turnIdx * 17, 60);+ const genStart = rootStart + 40 + jitter(ctx.seed, turnIdx * 17 + 1, 80);+ const genEnd = genStart + rng.int(900, 3200);+ const usageInput = rng.int(400, 2200);+ const usageOutput = rng.int(80, 600);+ const inputCost = microPrice(usageInput, 2e-6);+ const outputCost = microPrice(usageOutput, 6e-6);++ const observations: ObservationRecordInsertType[] = [+ createObservation({+ id: `${traceId}-o0`,+ trace_id: traceId,+ project_id: ctx.projectId,+ environment: ctx.environment,+ type: "AGENT",+ parent_observation_id: null,+ name: "troubleshooting-agent",+ start_time: rootStart,+ end_time: genEnd + 20,+ completion_start_time: null,+ level: "DEFAULT",+ status_message: null,+ version: null,+ input: null,+ output: null,+ metadata: { scenario: "session-shapes", shape: "media", variant },+ provided_model_name: null,+ internal_model_id: null,+ model_parameters: "{}",+ provided_usage_details: {},+ usage_details: {},+ provided_cost_details: {},+ cost_details: {},+ total_cost: null,+ prompt_id: null,+ prompt_name: null,+ prompt_version: null,+ created_at: Date.now(),+ updated_at: Date.now(),+ event_ts: Date.now(),+ }),+ createObservation({+ id: genId,+ trace_id: traceId,+ project_id: ctx.projectId,+ environment: ctx.environment,+ type: "GENERATION",+ parent_observation_id: `${traceId}-o0`,+ name: "gpt-5.4-vision-completion",+ start_time: genStart,+ end_time: genEnd,+ completion_start_time: genStart + rng.int(90, 320),+ level: "DEFAULT",+ status_message: null,+ version: null,+ input: genInput,+ output: prompt.assistant,+ metadata: { scenario: "session-shapes", shape: "media", variant },+ provided_model_name: "gpt-5.4",+ internal_model_id: null,+ model_parameters: JSON.stringify({ temperature: 0.2 }),+ provided_usage_details: {+ input: usageInput,+ output: usageOutput,+ total: usageInput + usageOutput,+ },+ usage_details: {+ input: usageInput,+ output: usageOutput,+ total: usageInput + usageOutput,+ },+ provided_cost_details: { input: inputCost, output: outputCost },+ cost_details: {+ input: inputCost,+ output: outputCost,+ total: inputCost + outputCost,+ },+ total_cost: inputCost + outputCost,+ prompt_id: null,+ prompt_name: null,+ prompt_version: null,+ created_at: Date.now(),+ updated_at: Date.now(),+ event_ts: Date.now(),+ }),+ ];++ return { trace, observations, mediaLinks };+};+const buildSession = (ctx: ScenarioContext,rng: Rng,shape: Shape,turns: number,+ mediaFixtures?: MediaFixtures,): {sessionId: string;sessionStart: number;traces: TraceRecordInsertType[];observations: ObservationRecordInsertType[];+ mediaLinks: { traceId: string; observationId: string; mediaId: string }[];} => {const sessionId = `${ctx.idPrefix}-${shape}`;// Anchor each session in a recent, distinct hour so UI time windows show@@ -454,16 +733,45 @@ const buildSession = (const traces: TraceRecordInsertType[] = [];const observations: ObservationRecordInsertType[] = [];+ const mediaLinks: {+ traceId: string;+ observationId: string;+ mediaId: string;+ }[] = [];for (let t = 0; t < turns; t++) {const traceId = `${ctx.idPrefix}-${shape}-t${t}`;const timestamp =sessionStart + Math.floor(t * stepMs) + jitter(ctx.seed, t, 400);- // mixed alternates chat/agent turns; chat/agent are uniform.+ // mixed alternates chat/agent turns; chat/agent/media are uniform.const turnShape: Exclude<Shape, "mixed"> =shape === "mixed" ? (t % 2 === 0 ? "chat" : "agent") : shape;+ if (turnShape === "media") {+ if (!mediaFixtures) {+ throw new SeedError(+ "media shape requires seeded media fixtures",+ "this is a bug in the scenario — fixtures are resolved before buildSession",+ );+ }+ const built = buildMediaTrace(+ ctx,+ rng,+ sessionId,+ traceId,+ t,+ timestamp,+ mediaFixtures,+ );+ traces.push(built.trace);+ observations.push(...built.observations);+ mediaLinks.push(+ ...built.mediaLinks.map((link) => ({ ...link, traceId })),+ );+ continue;+ }+const built =turnShape === "chat"? buildChatTrace(ctx, rng, sessionId, traceId, t, timestamp)@@ -473,7 +781,7 @@ const buildSession = (observations.push(...built.observations);}- return { sessionId, sessionStart, traces, observations };+ return { sessionId, sessionStart, traces, observations, mediaLinks };};const run = async (@@ -498,11 +806,38 @@ const run = async ();}- const shapesToSeed: Shape[] =+ let shapesToSeed: Shape[] =shapeParam === "all" ? [...SHAPES] : [shapeParam as Shape];const rng = new Rng(ctx.seed);+ // Uploaded up front: the reference strings must be embedded in the payloads+ // the builders produce, and a media shape with unresolvable assets would+ // silently look like the bug it exists to disprove. Only the media shape+ // needs storage — the others must still seed without it.+ let mediaFixtures: MediaFixtures | undefined;+ if (shapesToSeed.includes("media") && !ctx.dryRun) {+ const [image, audio, pdf] = await Promise.all([+ ensureSeedMediaUploaded(ctx.projectId, "image"),+ ensureSeedMediaUploaded(ctx.projectId, "audio"),+ ensureSeedMediaUploaded(ctx.projectId, "pdf"),+ ]);++ if (image && audio && pdf) {+ mediaFixtures = { image, audio, pdf };+ } else if (shapeParam === "media") {+ throw new SeedError(+ "could not seed media assets for the media shape",+ "check LANGFUSE_S3_MEDIA_UPLOAD_BUCKET and that MinIO is running (pnpm run seed -- doctor)",+ );+ } else {+ ctx.log(+ "skipping the media shape: media assets could not be uploaded (check LANGFUSE_S3_MEDIA_UPLOAD_BUCKET / MinIO)",+ );+ shapesToSeed = shapesToSeed.filter((shape) => shape !== "media");+ }+ }+if (ctx.dryRun) {return {scenario: "session-shapes",@@ -533,12 +868,17 @@ const run = async (const links: string[] = [];for (const shape of shapesToSeed) {- const { sessionId, sessionStart, traces, observations } = buildSession(- ctx,- rng,- shape,- turns,- );+ const { sessionId, sessionStart, traces, observations, mediaLinks } =+ buildSession(ctx, rng, shape, turns, mediaFixtures);+ for (const link of mediaLinks) {+ await linkSeedMediaToObservation({+ projectId: ctx.projectId,+ traceId: link.traceId,+ observationId: link.observationId,+ mediaId: link.mediaId,+ field: "input",+ });+ }sessionIds.push(sessionId);links.push(sessionLink(ctx, sessionId));allTraces.push(...traces);@@ -665,14 +1005,14 @@ const run = async (export const sessionShapesScenario: ScenarioDefinition = {name: "session-shapes",description:- "Diverse v4 session shapes for the session-detail view: a clean multi-turn CHAT session (renders as chat), a coding/AGENT session whose I/O lives on AGENT/TOOL observations with NO GENERATION (the default 'first generation' preset yields empty cards — LFE-10520), and a MIXED session. Creates the Postgres trace_sessions rows.",+ "Diverse v4 session shapes for the session-detail view: a clean multi-turn CHAT session (renders as chat), a coding/AGENT session whose I/O lives on AGENT/TOOL observations with NO GENERATION (the default 'first generation' preset yields empty cards — LFE-10520), a MIXED session, and a MEDIA session whose messages carry @@@langfuseMedia:...@@@ references (inline image, multiple references in one message, and a link-only payload — LFE-14815). Creates the Postgres trace_sessions rows; the media shape uploads its assets to MinIO.",supportsV4: true,flags: [{flag: "shape",type: "string",default: "all",- description: "session shape: chat | agent | mixed | all",+ description: "session shape: chat | agent | mixed | media | all",},{flag: "turns",diff --git a/packages/shared/scripts/seeder/seed-media.ts b/packages/shared/scripts/seeder/seed-media.tsindex 7f064ab72e..49c0b055ec 100644--- a/packages/shared/scripts/seeder/seed-media.ts+++ b/packages/shared/scripts/seeder/seed-media.ts@@ -134,20 +134,65 @@ function getBucketPath(}/**- * Upload a media file to storage and create database records+ * Uploads a fixture and creates its `media` row, idempotently. Returns the+ * fixture so callers can embed `referenceString` in a payload — that alone is+ * enough for the UI to resolve the asset (`media.getById`); a trace/observation+ * link only adds the separate "Media" strip.+ */+export async function ensureSeedMediaUploaded(+ projectId: string,+ mediaKey: SeedMediaKey,+): Promise<SeedMediaFixture | null> {+ const fixture = getSeedMediaFixture(mediaKey);+ if (!fixture) return null;++ const mediaId = await uploadAndCreateMediaRecord(+ projectId,+ MEDIA_FILES[mediaKey],+ );++ return mediaId ? fixture : null;+}++/**+ * Links an already-uploaded fixture to one observation's field, mirroring what+ * ingestion writes. Drives the "Media" strip on observation surfaces.+ */+export async function linkSeedMediaToObservation({+ projectId,+ traceId,+ observationId,+ mediaId,+ field,+}: {+ projectId: string;+ traceId: string;+ observationId: string;+ mediaId: string;+ field: "input" | "output" | "metadata";+}): Promise<void> {+ await prisma.$queryRaw`+ INSERT INTO "observation_media" ("id", "project_id", "trace_id", "observation_id", "media_id", "field")+ VALUES (${crypto.randomUUID()}, ${projectId}, ${traceId}, ${observationId}, ${mediaId}, ${field})+ ON CONFLICT DO NOTHING;+ `;+}++/**+ * Upload a media file to storage and create its `media` row. Returns the+ * media id it derived, so callers link rows without re-hashing the file, or+ * null when storage or the fixture file is unavailable.*/async function uploadAndCreateMediaRecord(projectId: string,- traceId: string,- field: "input" | "output" | "metadata",mediaFile: MediaFile,-): Promise<void> {+): Promise<string | null> {// Check if bucket is configuredif (!env.LANGFUSE_S3_MEDIA_UPLOAD_BUCKET) {logger.warn("[seed-media] LANGFUSE_S3_MEDIA_UPLOAD_BUCKET not configured, skipping media seeding",);- return;+ return null;}// Check if file exists@@ -155,7 +200,7 @@ async function uploadAndCreateMediaRecord(logger.warn(`[seed-media] Test file not found: ${mediaFile.filePath}, skipping`,);- return;+ return null;}const fileBytes = fs.readFileSync(mediaFile.filePath);@@ -178,16 +223,9 @@ async function uploadAndCreateMediaRecord(if (existingMedia && existingMedia.uploadHttpStatus === 200) {logger.debug(- `[seed-media] Media already exists for ${mediaFile.name}, creating TraceMedia link only`,+ `[seed-media] Media already exists for ${mediaFile.name}, reusing it`,);-- // Just create the TraceMedia link- await prisma.$queryRaw`- INSERT INTO "trace_media" ("id", "project_id", "trace_id", "media_id", "field")- VALUES (${crypto.randomUUID()}, ${projectId}, ${traceId}, ${existingMedia.id}, ${field})- ON CONFLICT DO NOTHING;- `;- return;+ return mediaId;}// Upload to storage@@ -212,7 +250,7 @@ async function uploadAndCreateMediaRecord(logger.debug(`[seed-media] Uploaded ${mediaFile.name} to ${bucketPath}`);} catch (error) {logger.error(`[seed-media] Failed to upload ${mediaFile.name}:`, error);- return;+ return null;}// Create Media record@@ -249,16 +287,25 @@ async function uploadAndCreateMediaRecord("upload_http_status" = ${200}`;- // Create TraceMedia link+ logger.info(`[seed-media] Created media record for ${mediaFile.name}`);+ return mediaId;+}++/** Links an uploaded fixture to a trace's field (the `dx` seed path). */+async function linkSeedMediaToTrace(+ projectId: string,+ traceId: string,+ field: "input" | "output" | "metadata",+ mediaFile: MediaFile,+): Promise<void> {+ const mediaId = await uploadAndCreateMediaRecord(projectId, mediaFile);+ if (!mediaId) return;+await prisma.$queryRaw`INSERT INTO "trace_media" ("id", "project_id", "trace_id", "media_id", "field")VALUES (${crypto.randomUUID()}, ${projectId}, ${traceId}, ${mediaId}, ${field})ON CONFLICT DO NOTHING;`;-- logger.info(- `[seed-media] Created media record for ${mediaFile.name} -> ${field}`,- );}/**@@ -281,28 +328,13 @@ export async function seedMediaTraces(projectId: string): Promise<void> {// Trace 1: Image only (input)const trace1Id = MEDIA_TEST_TRACE_IDS.imageOnly;- await uploadAndCreateMediaRecord(- projectId,- trace1Id,- "input",- MEDIA_FILES.image,- );+ await linkSeedMediaToTrace(projectId, trace1Id, "input", MEDIA_FILES.image);// Trace 2: All media typesconst trace2Id = MEDIA_TEST_TRACE_IDS.allTypes;- await uploadAndCreateMediaRecord(- projectId,- trace2Id,- "input",- MEDIA_FILES.image,- );- await uploadAndCreateMediaRecord(- projectId,- trace2Id,- "output",- MEDIA_FILES.pdf,- );- await uploadAndCreateMediaRecord(+ await linkSeedMediaToTrace(projectId, trace2Id, "input", MEDIA_FILES.image);+ await linkSeedMediaToTrace(projectId, trace2Id, "output", MEDIA_FILES.pdf);+ await linkSeedMediaToTrace(projectId,trace2Id,"metadata",@@ -311,19 +343,9 @@ export async function seedMediaTraces(projectId: string): Promise<void> {// Trace 3: All media types with ChatML format (pretty-rendered)const trace3Id = MEDIA_TEST_TRACE_IDS.allTypesChatML;- await uploadAndCreateMediaRecord(- projectId,- trace3Id,- "input",- MEDIA_FILES.image,- );- await uploadAndCreateMediaRecord(- projectId,- trace3Id,- "output",- MEDIA_FILES.pdf,- );- await uploadAndCreateMediaRecord(+ await linkSeedMediaToTrace(projectId, trace3Id, "input", MEDIA_FILES.image);+ await linkSeedMediaToTrace(projectId, trace3Id, "output", MEDIA_FILES.pdf);+ await linkSeedMediaToTrace(projectId,trace3Id,"metadata",diff --git a/packages/shared/src/utils/IORepresentation/chatML/types.ts b/packages/shared/src/utils/IORepresentation/chatML/types.tsindex 22089500e4..6176682bf0 100644--- a/packages/shared/src/utils/IORepresentation/chatML/types.ts+++ b/packages/shared/src/utils/IORepresentation/chatML/types.ts@@ -36,6 +36,10 @@ export type ParsedMediaReferenceType = z.infer<typeof ParsedMediaReferenceSchema>;+// Matches the format parsed by MediaReferenceStringSchema. Non-greedy, so+// consecutive references in one string are separate matches.+export const MEDIA_REFERENCE_PATTERN = /@@@langfuseMedia:.+?@@@/g;+/*** Schema that parses Langfuse media reference magic strings.* Format: @@@langfuseMedia:type=image/jpeg|id=<uuid>|source=base64@@@@@ -163,7 +167,27 @@ const OpenAIImageContentPart = z.object({export type OpenAIImageContentPartType = z.infer<typeof OpenAIImageContentPart>;/**- * Array of OpenAI content parts (text, image, audio).+ * A bare `@@@langfuseMedia:…@@@` reference used as a whole content part.+ *+ * Validates through the canonical parser but deliberately does NOT transform:+ * a transforming member of this union hands the renderer a shape its own part+ * guards reject, which is how inline media silently stopped rendering+ * (LFE-14815). The raw string must survive.+ */+export const MediaReferencePartSchema = z.string().refine((value) => {+ // Exactly ONE whole reference: the parser's `^…(.*)…$` is greedy, so two+ // concatenated tags would parse as one, silently mixing tag 1's type with+ // tag 2's id. Splitting first mirrors getStandaloneMediaReferenceStrings.+ const matches = value.match(MEDIA_REFERENCE_PATTERN) ?? [];+ return (+ matches.length === 1 &&+ matches[0] === value &&+ MediaReferenceStringSchema.safeParse(value).success+ );+});++/**+ * Array of OpenAI content parts (text, image, audio, bare media reference).* Used when message content is structured with multiple parts.*/export const OpenAIContentParts = z.array(@@ -171,9 +195,15 @@ export const OpenAIContentParts = z.array(OpenAITextContentPart,OpenAIImageContentPart,OpenAIInputAudioContentPart,+ MediaReferencePartSchema,]),);+export const isMediaReferencePart = (+ content: unknown,+): content is z.infer<typeof MediaReferencePartSchema> =>+ MediaReferencePartSchema.safeParse(content).success;+/*** OpenAI output audio schema.* Represents audio generated by the model.diff --git a/packages/shared/src/utils/mediaReferences.test.ts b/packages/shared/src/utils/mediaReferences.test.tsindex ed30a33199..79a746906e 100644--- a/packages/shared/src/utils/mediaReferences.test.ts+++ b/packages/shared/src/utils/mediaReferences.test.ts@@ -1,5 +1,6 @@import { describe, expect, it } from "vitest";+import { isMediaReferencePart } from "./IORepresentation/chatML/types";import { findMediaReferences } from "./mediaReferences";const imageRef =@@ -81,3 +82,15 @@ describe("findMediaReferences", () => {expect(findMediaReferences(imageRef)).toMatchObject([{ jsonPath: "$" }]);});});++describe("isMediaReferencePart", () => {+ it("accepts one whole reference and rejects concatenated or padded ones", () => {+ expect(isMediaReferencePart(imageRef)).toBe(true);+ // The parser's magic-string regex is greedy, so two tags in one string+ // would otherwise parse as one — mixing tag 1's type with tag 2's id.+ expect(isMediaReferencePart(`${imageRef} ${audioRef}`)).toBe(false);+ expect(isMediaReferencePart(`${imageRef}${audioRef}`)).toBe(false);+ expect(isMediaReferencePart(`${imageRef} trailing text`)).toBe(false);+ expect(isMediaReferencePart("not a reference")).toBe(false);+ });+});diff --git a/packages/shared/src/utils/mediaReferences.ts b/packages/shared/src/utils/mediaReferences.tsindex fa449619ba..5e0381b4b4 100644--- a/packages/shared/src/utils/mediaReferences.ts+++ b/packages/shared/src/utils/mediaReferences.ts@@ -2,9 +2,6 @@ import { JSONPath } from "jsonpath-plus";import { MediaReferenceStringSchema } from "./IORepresentation/chatML/types";-// Matches the format parsed by MediaReferenceStringSchema.-export const MEDIA_REFERENCE_PATTERN = /@@@langfuseMedia:.+?@@@/g;-/*** Scans a JSON value for `@@@langfuseMedia:...@@@` reference strings and* returns each occurrence with the JSONPath of the containing string.diff --git a/web/src/components/session/SessionObservationIO.clienttest.tsx b/web/src/components/session/SessionObservationIO.clienttest.tsxindex 1a616f3e92..85ba3deda9 100644--- a/web/src/components/session/SessionObservationIO.clienttest.tsx+++ b/web/src/components/session/SessionObservationIO.clienttest.tsx@@ -9,6 +9,9 @@ import { fireEvent, render, screen, waitFor } from "@testing-library/react";const fullIOQuery = vi.fn();const downloadJsonFile = vi.fn();const capture = vi.fn();+const useMediaMock = vi.fn(+ (_args: Record<string, unknown>): { data: unknown } => ({ data: undefined }),+);vi.mock("@/src/utils/api", () => ({api: {@@ -29,7 +32,13 @@ vi.mock("@/src/features/posthog-analytics/usePostHogClientCapture", () => ({// The component imports IOPreview through the traces feature surface, so the// mock has to intercept the surface — mocking the module behind it does nothing.vi.mock("@/src/features/traces", () => ({- IOPreview: () => <div data-testid="io-preview" />,+ IOPreview: ({ media }: { media?: { mediaId: string }[] }) => (+ <div+ data-testid="io-preview"+ data-media-ids={(media ?? []).map((m) => m.mediaId).join(",")}+ />+ ),+ useMedia: (args: Record<string, unknown>) => useMediaMock(args),}));vi.mock("@/src/components/session/actions/downloadSessionAsJson", () => ({@@ -162,6 +171,31 @@ describe("SessionObservationIO", () => {expect(screen.getByText(/"key":"value"/)).toBeInTheDocument();});+ // LFE-14815: trace detail resolved observation media, the session view did+ // not, so a media-bearing message had no thumbnail here.+ it("forwards observation media to IOPreview, and only looks it up when referenced", () => {+ useMediaMock.mockReturnValue({ data: [{ mediaId: "media-1" }] });+ renderComponent({+ ...baseObservation,+ input: `{"content":"@@@langfuseMedia:type=image/png|id=media-1|source=base64_data_uri@@@"}`,+ } as SessionTraceObservation);++ expect(screen.getByTestId("io-preview")).toHaveAttribute(+ "data-media-ids",+ "media-1",+ );+ expect(useMediaMock).toHaveBeenCalledWith(+ expect.objectContaining({ observationId: "obs-1", enabled: true }),+ );++ useMediaMock.mockClear();+ renderComponent(baseObservation);++ expect(useMediaMock).toHaveBeenCalledWith(+ expect.objectContaining({ enabled: false }),+ );+ });+it("keeps IOPreview but points to the trace view when only metadata was truncated", () => {const { onOpenInTraceView } = renderComponent({...baseObservation,diff --git a/web/src/components/session/SessionObservationIO.tsx b/web/src/components/session/SessionObservationIO.tsxindex 7b28ae538f..75c24e7623 100644--- a/web/src/components/session/SessionObservationIO.tsx+++ b/web/src/components/session/SessionObservationIO.tsx@@ -3,6 +3,7 @@ import { Download, ExternalLinkIcon, Loader2 } from "lucide-react";import { Button } from "@/src/components/ui/button";import {IOPreview,+ useMedia,type ChatMLParserResult,type IOPreviewContentMode,type ViewMode,@@ -20,6 +21,19 @@ export type SessionTraceObservation =/** Display cap of a preview section — matches the server's preview head. */const PREVIEW_DISPLAY_CHARS = 4_000;+const MEDIA_REFERENCE_PREFIX = "@@@langfuseMedia:";++/**+ * A card renders up to 50 observations, so the media-link lookup is gated on+ * the payload actually carrying a reference. This path ships I/O as raw+ * strings (see `sessions.observationsForTraceFromEvents`).+ */+const referencesMedia = (observation: SessionTraceObservation): boolean =>+ [observation.input, observation.output, observation.metadata].some(+ (value) =>+ typeof value === "string" && value.includes(MEDIA_REFERENCE_PREFIX),+ );+/*** One field of an over-limit observation: a bounded, non-interactive preview* head. Never grows with payload size — the display is capped even when the@@ -111,6 +125,15 @@ export const SessionObservationIO = ({observation.inputTruncated || observation.outputTruncated,);+ // Same media source as trace detail, so a media-bearing message renders+ // identically on both surfaces (LFE-14815).+ const observationMedia = useMedia({+ projectId,+ traceId,+ observationId: observation.id,+ enabled: !isIOTruncated && referencesMedia(observation),+ });+const onDownload = async () => {capture("session_detail:truncated_observation_download_click");setIsDownloading(true);@@ -160,6 +183,7 @@ export const SessionObservationIO = ({metadata={observation.metadata ?? undefined}observationName={observation.name ?? undefined}hideIfNull+ media={observationMedia.data}projectId={projectId}traceId={traceId}observationId={observation.id}diff --git a/web/src/components/ui/MarkdownJsonView.tsx b/web/src/components/ui/MarkdownJsonView.tsxindex 44c52d4751..d4d4307676 100644--- a/web/src/components/ui/MarkdownJsonView.tsx+++ b/web/src/components/ui/MarkdownJsonView.tsx@@ -62,20 +62,22 @@ export function MarkdownJsonViewHeader({);}-const isSupportedMarkdownFormat = (+/**+ * Whether `content` renders through MarkdownView (inline text/image/audio+ * parts) rather than as a JSON table. Narrows to the schema's INPUT type:+ * `MediaReferenceStringSchema` transforms `@@@langfuseMedia:…@@@` into a parsed+ * object, and the renderer's own part guards re-validate the untransformed+ * shape — so the raw content is what must reach it (LFE-14815).+ *+ * Exported so callers deduping the media strip against what rendered inline+ * ask the same question the renderer answered.+ */+export const canRenderContentAsMarkdown = (content: unknown,- contentValidation: z.ZodSafeParseResult<z.infer<typeof OpenAIContentSchema>>,-): content is z.infer<typeof OpenAIContentSchema> => {- if (!contentValidation.success) return false;-+): content is z.input<typeof OpenAIContentSchema> =>+ OpenAIContentSchema.safeParse(content).success &&// Don't render if markdown content is huge- const contentSize = JSON.stringify(content || {}).length;- if (contentSize > MARKDOWN_RENDER_CHARACTER_LIMIT) {- return false;- }-- return true;-};+ JSON.stringify(content || {}).length <= MARKDOWN_RENDER_CHARACTER_LIMIT;// MarkdownJsonView will render markdown if `isMarkdownEnabled` (global context) is true and the content is valid markdown// otherwise, if content is valid markdown will render JSON with switch to enable markdown globally@@ -103,23 +105,18 @@ export function MarkdownJsonView({the title can carry a message `name` instead of the role). */isSystemPrompt?: boolean;}) {- const validatedOpenAIContent = useMemo(- () => OpenAIContentSchema.safeParse(content),+ // Boxed so a renderable `null` content stays distinguishable from+ // "not renderable as markdown", without re-widening the narrowed type.+ const markdownContent = useMemo(+ () => (canRenderContentAsMarkdown(content) ? { value: content } : null),[content],);- const canEnableMarkdown = isSupportedMarkdownFormat(- content,- validatedOpenAIContent,- );-return (<>- {canEnableMarkdown ? (+ {markdownContent ? (<MarkdownView- markdown={- validatedOpenAIContent.success ? validatedOpenAIContent.data : null- }+ markdown={markdownContent.value}title={title}titleIcon={titleIcon}audio={audio}diff --git a/web/src/components/ui/MarkdownViewer.tsx b/web/src/components/ui/MarkdownViewer.tsxindex aead4af116..4d5cda2872 100644--- a/web/src/components/ui/MarkdownViewer.tsx+++ b/web/src/components/ui/MarkdownViewer.tsx@@ -28,6 +28,7 @@ import {type OpenAIOutputAudioType,isOpenAITextContentPart,isOpenAIImageContentPart,+ isMediaReferencePart,} from "@langfuse/shared";import { type z } from "zod";import { ResizableImage } from "@/src/components/ui/resizable-image";@@ -462,11 +463,13 @@ function MarkdownRenderer({}}const parseOpenAIContentParts = (- content: z.infer<typeof OpenAIContentParts> | null,+ content: z.input<typeof OpenAIContentParts> | null,): string => {return (content ?? []).map((item) => {- if (item.type === "text") {+ if (typeof item === "string") {+ return item;+ } else if (item.type === "text") {return item.text;} else if (item.type === "image_url") {return ``;@@ -488,7 +491,9 @@ export function MarkdownView({afterHeader,isSystemPrompt,}: {- markdown: string | z.infer<typeof OpenAIContentSchema>;+ /** The UNPARSED content shape — see `canRenderContentAsMarkdown`. Media+ reference strings must still be strings when they reach the part guards. */+ markdown: string | z.input<typeof OpenAIContentSchema>;title?: string;titleIcon?: React.ReactNode;audio?: OpenAIOutputAudioType;@@ -607,57 +612,22 @@ export function MarkdownView({</>)) : (- // content parts (multi-modal); collapsed = preview of the joined text+ // content parts (multi-modal); collapsing hides long TEXT only —+ // attachments are not text, so media parts render either way. That+ // also keeps the shared media strip's dedup honest: it assumes any+ // inline-renderable media did render (LFE-14815).<>{isCollapsed ? (- <MarkdownRenderer markdown={truncatedContent} theme={theme} />+ <>+ <MarkdownRenderer markdown={truncatedContent} theme={theme} />+ {(markdown ?? []).map((content, index) =>+ isOpenAITextContentPart(content)+ ? null+ : renderContentPart(content, index),+ )}+ </>) : (- (markdown ?? []).map((content, index) => {- if (isOpenAITextContentPart(content)) {- return (- <MarkdownRenderer- key={index}- markdown={content.text}- theme={theme}- />- );- }-- if (isOpenAIImageContentPart(content)) {- const imageUrl = content.image_url.url;- const safeImageUrl =- typeof imageUrl === "string" &&- OpenAIUrlImageUrl.safeParse(imageUrl).success- ? getSafeImageUrl(imageUrl)- : null;-- return safeImageUrl ? (- <div key={index}>- <ResizableImage src={safeImageUrl} />- </div>- ) : MediaReferenceStringSchema.safeParse(imageUrl).success ? (- <LangfuseMediaView mediaReferenceString={imageUrl} />- ) : (- <div className="grid grid-cols-[auto_1fr] items-center gap-2">- <span title="<Base64 data URI>" className="h-4 w-4">- <ImageOff className="h-4 w-4" />- </span>- <span- className="truncate text-sm"- title={imageUrl.toString()}- >- {imageUrl.toString()}- </span>- </div>- );- }-- return content.type === "input_audio" ? (- <LangfuseMediaView- mediaReferenceString={content.input_audio.data}- />- ) : null;- })+ (markdown ?? []).map(renderContentPart))}{collapseToggle}</>@@ -692,4 +662,56 @@ export function MarkdownView({)}</div>);++ function renderContentPart(+ content: NonNullable<z.input<typeof OpenAIContentParts>>[number],+ index: number,+ ) {+ // A bare reference string is a whole part (LFE-9577).+ if (isMediaReferencePart(content)) {+ return <LangfuseMediaView key={index} mediaReferenceString={content} />;+ }++ if (isOpenAITextContentPart(content)) {+ return (+ <MarkdownRenderer key={index} markdown={content.text} theme={theme} />+ );+ }++ if (isOpenAIImageContentPart(content)) {+ const imageUrl = content.image_url.url;+ const safeImageUrl =+ typeof imageUrl === "string" &&+ OpenAIUrlImageUrl.safeParse(imageUrl).success+ ? getSafeImageUrl(imageUrl)+ : null;++ return safeImageUrl ? (+ <div key={index}>+ <ResizableImage src={safeImageUrl} />+ </div>+ ) : MediaReferenceStringSchema.safeParse(imageUrl).success ? (+ <LangfuseMediaView key={index} mediaReferenceString={imageUrl} />+ ) : (+ <div+ key={index}+ className="grid grid-cols-[auto_1fr] items-center gap-2"+ >+ <span title="<Base64 data URI>" className="h-4 w-4">+ <ImageOff className="h-4 w-4" />+ </span>+ <span className="truncate text-sm" title={imageUrl.toString()}>+ {imageUrl.toString()}+ </span>+ </div>+ );+ }++ return content.type === "input_audio" ? (+ <LangfuseMediaView+ key={index}+ mediaReferenceString={content.input_audio.data}+ />+ ) : null;+ }}diff --git a/web/src/components/ui/markdown-media.utils.ts b/web/src/components/ui/markdown-media.utils.tsindex a2a0ddf47c..f1d634f1cf 100644--- a/web/src/components/ui/markdown-media.utils.ts+++ b/web/src/components/ui/markdown-media.utils.ts@@ -2,6 +2,7 @@ import {MEDIA_REFERENCE_PATTERN,MediaReferenceStringSchema,isOpenAIImageContentPart,+ isMediaReferencePart,type OpenAIContentSchema,type OpenAIOutputAudioType,type ParsedMediaReferenceType,@@ -40,7 +41,7 @@ export const getRenderedInlineMediaIds = ({markdown,audio,}: {- markdown: string | z.infer<typeof OpenAIContentSchema>;+ markdown: string | z.input<typeof OpenAIContentSchema>;audio?: OpenAIOutputAudioType;}): Set<string> => {const mediaIds = new Set<string>();@@ -54,7 +55,12 @@ export const getRenderedInlineMediaIds = ({});} else {(markdown ?? []).forEach((content) => {- if (isOpenAIImageContentPart(content)) {+ if (isMediaReferencePart(content)) {+ const mediaId = getMediaReferenceId(content);+ if (mediaId) {+ mediaIds.add(mediaId);+ }+ } else if (isOpenAIImageContentPart(content)) {const mediaId = getMediaReferenceId(content.image_url.url);if (mediaId) {mediaIds.add(mediaId);diff --git a/web/src/features/traces/components/ChatMessageList.tsx b/web/src/features/traces/components/ChatMessageList.tsxindex 594f3d64c2..17fa0e56de 100644--- a/web/src/features/traces/components/ChatMessageList.tsx+++ b/web/src/features/traces/components/ChatMessageList.tsx@@ -5,6 +5,7 @@ import {filterAlreadyRenderedMedia,getRenderedInlineMediaIds,} from "@/src/components/ui/markdown-media.utils";+import { canRenderContentAsMarkdown } from "@/src/components/ui/MarkdownJsonView";import {ChatMessage,type ViewMode,@@ -97,10 +98,10 @@ export function ChatMessageList({visibleMessages.forEach(({ message }) => {const content = message.content;- // Queue previews reliably render standalone media tags in strings and- // output audio inline, but OpenAI content-part images still fall back to- // the shared media strip on this surface.- if (typeof content === "string") {+ // Only content that ChatMessage renders through MarkdownView shows its+ // media inline; anything falling back to a JSON table still needs the+ // shared strip, so ask the renderer's own predicate.+ if (typeof content === "string" || canRenderContentAsMarkdown(content)) {getRenderedInlineMediaIds({markdown: content,audio: message.audio,diff --git a/web/src/features/traces/components/IOPreview/components/ChatMessage.clienttest.tsx b/web/src/features/traces/components/IOPreview/components/ChatMessage.clienttest.tsxindex e1d57a4680..9c7a0c44ee 100644--- a/web/src/features/traces/components/IOPreview/components/ChatMessage.clienttest.tsx+++ b/web/src/features/traces/components/IOPreview/components/ChatMessage.clienttest.tsx@@ -7,6 +7,35 @@ const { copyTextToClipboard } = vi.hoisted(() => ({vi.mock("@/src/utils/clipboard", () => ({ copyTextToClipboard }));+// ChatMessage mounts both render paths (one is display:none), so the JSON+// path's media tag resolves too — stub its project/URL lookups.+vi.mock("@/src/hooks/useProjectIdFromURL", () => ({ default: () => "p1" }));+vi.mock("@/src/components/ui/media/useResolvedMedia", () => ({+ useResolvedMedia: () => ({ status: "idle", url: undefined }),+}));++// Stands in for the real media renderer (it needs a session + signed-URL+// query); the contract under test is which reference reaches it.+vi.mock("@/src/components/ui/LangfuseMediaView", () => ({+ LangfuseMediaView: ({+ mediaReferenceString,+ }: {+ mediaReferenceString?: unknown;+ }) => (+ <div+ data-testid="langfuse-media"+ data-media-ref={+ typeof mediaReferenceString === "string"+ ? mediaReferenceString+ : (mediaReferenceString as { referenceString?: string })+ ?.referenceString+ }+ />+ ),+}));++import { ChatMlArraySchema } from "@langfuse/shared";+import { ChatMessage } from "./ChatMessage";import { type ChatMlMessage } from "../../../fns/chatMessageUtils";import { MarkdownContextProvider } from "@/src/features/theming/useMarkdownContext";@@ -305,3 +334,88 @@ describe("ChatMessage copy", () => {expect(copied).not.toContain("\\u3053");});});++describe("ChatMessage media content parts", () => {+ const mediaId = "abc123def456ghi789jkl0";+ const referenceString = `@@@langfuseMedia:type=image/png|id=${mediaId}|source=base64_data_uri@@@`;++ beforeEach(() => {+ vi.stubGlobal("localStorage", createMemoryStorage());+ vi.stubGlobal("sessionStorage", createMemoryStorage());+ });++ // LFE-14815 / LFE-13602: OpenAIContentSchema's url union parses the media+ // reference into an object, so a renderer re-validating the untransformed+ // schema drops the part silently.+ it("renders an image_url part carrying a langfuse media reference", () => {+ renderChatMessage({+ role: "user",+ content: [+ { type: "text", text: "here is the thermostat" },+ { type: "image_url", image_url: { url: referenceString } },+ ],+ } as unknown as ChatMlMessage);++ expect(screen.getByText("here is the thermostat")).toBeInTheDocument();+ expect(screen.getByTestId("langfuse-media")).toHaveAttribute(+ "data-media-ref",+ referenceString,+ );+ });++ // LFE-9577: a single reference string renders inline, and so do several in+ // one string — but an ARRAY of bare references failed OpenAIContentParts+ // (which only accepted `{type, …}` objects) and dropped to a JSON table.+ it("renders an array of bare media reference strings inline", () => {+ const second = referenceString.replace(mediaId, "zzz987yxw654vut321srq0");+ renderChatMessage({+ role: "user",+ content: [referenceString, second],+ } as unknown as ChatMlMessage);++ expect(+ screen+ .getAllByTestId("langfuse-media")+ .map((el) => el.getAttribute("data-media-ref")),+ ).toEqual([referenceString, second]);+ });++ // The ChatML transform materializes all 10 schema keys, so a message that+ // normalized to `{}` used to table out one `undefined` row per key.+ it("renders nothing for a parsed message that carries no data", () => {+ const parsed = ChatMlArraySchema.parse([{}]);+ const { container } = renderChatMessage(parsed[0] as ChatMlMessage);++ expect(container).toBeEmptyDOMElement();+ });++ // Same materialized keys, but this shape has one real field so it still+ // renders through the whole-message fallback table.+ it("omits unset ChatML keys from the fallback table", () => {+ const parsed = ChatMlArraySchema.parse([+ { role: "user", name: "Agent", content: "" },+ ]);+ renderChatMessage(parsed[0] as ChatMlMessage);++ expect(screen.queryByText("undefined")).not.toBeInTheDocument();+ expect(screen.queryByText("tool_call_id")).not.toBeInTheDocument();+ });++ // The media strip dedupes against what rendered inline, so a collapsed+ // system prompt must not swallow its attachments.+ it("keeps media parts visible while a long system prompt is collapsed", () => {+ renderChatMessage({+ role: "system",+ content: [+ { type: "text", text: longSystemPrompt },+ { type: "image_url", image_url: { url: referenceString } },+ ],+ } as unknown as ChatMlMessage);++ expect(expandButton()).toBeInTheDocument();+ expect(screen.getByTestId("langfuse-media")).toHaveAttribute(+ "data-media-ref",+ referenceString,+ );+ });+});diff --git a/web/src/features/traces/components/IOPreview/components/ChatMessage.tsx b/web/src/features/traces/components/IOPreview/components/ChatMessage.tsxindex af678529a0..04741ecd97 100644--- a/web/src/features/traces/components/IOPreview/components/ChatMessage.tsx+++ b/web/src/features/traces/components/IOPreview/components/ChatMessage.tsx@@ -20,6 +20,7 @@ import {parseToolCallsFromMessage,hasThinkingContent,hasRedactedThinkingContent,+ withoutUnsetFields,} from "../../../fns/chatMessageUtils";import { ThinkingBlock, RedactedThinkingBlock } from "../../ThinkingBlock";import { type IOPreviewContentMode } from "../IOPreview";@@ -236,7 +237,7 @@ export function ChatMessage({<div className="hover:bg-muted transition-colors"><PrettyJsonViewtitle={title || (isOutputMessage ? "Output" : "Input")}- json={message}+ json={withoutUnsetFields(message)}currentView={currentView}/></div>diff --git a/web/src/features/traces/fns/chatMessageUtils.ts b/web/src/features/traces/fns/chatMessageUtils.tsindex cc41dc86a3..2cd6f58cbf 100644--- a/web/src/features/traces/fns/chatMessageUtils.ts+++ b/web/src/features/traces/fns/chatMessageUtils.ts@@ -22,12 +22,29 @@ export function hasRenderableContent(message: ChatMlMessage): boolean {/*** Check if message has additional data beyond role and content.+ *+ * Values, not keys: the ChatML transform materializes every schema key, so a+ * key-presence check is true for even an empty message — which rendered as a+ * table of `undefined` rows (LFE-14815).*/export function hasAdditionalData(message: ChatMlMessage): boolean {- const messageKeys = Object.keys(message).filter(- (key) => key !== "role" && key !== "content",+ return Object.entries(message).some(+ ([key, value]) =>+ key !== "role" && key !== "content" && value !== undefined,+ );+}++/**+ * The message without its unset keys — the transform above materializes all ten+ * whether or not they were sent, and a table of the raw object renders one+ * `undefined` row each.+ */+export function withoutUnsetFields(+ message: ChatMlMessage,+): Record<string, unknown> {+ return Object.fromEntries(+ Object.entries(message).filter(([, value]) => value !== undefined),);- return messageKeys.length > 0;}/**diff --git a/web/src/features/traces/hooks/useMedia.ts b/web/src/features/traces/hooks/useMedia.tsindex d12a90b681..9c2f4837f3 100644--- a/web/src/features/traces/hooks/useMedia.ts+++ b/web/src/features/traces/hooks/useMedia.ts@@ -4,6 +4,8 @@ export type UseMediaParams = {projectId: string;traceId: string;observationId?: string;+ /** Skip the lookup on surfaces that render many observations at once. */+ enabled?: boolean;};/**@@ -17,6 +19,7 @@ export function useMedia({projectId,traceId,observationId,+ enabled = true,}: UseMediaParams) {return api.media.getByTraceOrObservationId.useQuery({@@ -25,6 +28,7 @@ export function useMedia({observationId,},{+ enabled,refetchOnWindowFocus: false,refetchOnMount: false,refetchOnReconnect: false,diff --git a/web/src/features/traces/index.ts b/web/src/features/traces/index.tsindex 49fe35b0ea..c6c864075a 100644--- a/web/src/features/traces/index.ts+++ b/web/src/features/traces/index.ts@@ -35,6 +35,9 @@ export {useChatMLParser,type ChatMLParserResult,} from "@/src/features/traces/hooks/useChatMLParser";+// The session view resolves observation media the same way trace detail does,+// so a media-bearing message renders identically on both surfaces (LFE-14815).+export { useMedia } from "@/src/features/traces/hooks/useMedia";// The JSON viewer's comment-range contract. It cannot be promoted to// `src/fns` — it depends on the viewer's own path and row types — so the