Skip to content

πŸͺ’ Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.

feat(transcript): implement message comparison and continuity features (#19021)

marliessophie3d1934a5d63d
diff --git a/packages/shared/src/server/transcript/README.md b/packages/shared/src/server/transcript/README.md
index c9c55f1ccc..089fc5ae6e 100644
--- a/packages/shared/src/server/transcript/README.md
+++ b/packages/shared/src/server/transcript/README.md
@@ -91,8 +91,8 @@ It must be a safe integer of at least 4, the serialized length of `null`.
Omitting it preserves the complete transcript and existing assembly behavior.
Limiting happens after normalization, matching and turn splitting. Text and
-unsigned textual reasoning without provider metadata can be shortened with an
-ellipsis; identifiers, tool arguments/results, media, signed reasoning and other structured fields
+unsigned textual enrichment without provider metadata can be shortened with an
+ellipsis; identifiers, tool arguments/results, media, signed enrichment and other structured fields
stay intact. Messages that cannot fit even with shortened text are omitted.
When too many messages remain, the beginning and end are retained, preserving
their order and history/current-turn partition. Contributor references are
@@ -171,35 +171,34 @@ registration and replayed-result suppression happen inside message appending.
### 1. Select a thread
-Compare generation messages by role and non-reasoning parts. Continue a thread
-when all its non-system messages with non-reasoning content appear in the incoming
+Compare generation messages by role and identity parts. Continue a thread
+when all its non-system messages with identity parts appear in the incoming
input, regardless of message order. At least one such message must match: system
-messages and reasoning alone cannot establish continuity. Supplemental tool
+messages and enrichment alone cannot establish continuity. Attached tool
responses are not required. Otherwise create a new thread. Matching checks
presence, not occurrence counts.
When several threads match, the most recently created matching thread wins.
Instrumentation should capture the full input and output of each model call,
-including reasoning. Thread matching tolerates missing or changed reasoning and
+including enrichment. Thread matching tolerates missing or changed enrichment and
provider annotations; it still requires matching text, tool calls, and arguments.
### 2. Append messages
-A message is identified by stable JSON of **role + non-reasoning parts**. Object-property
-order is ignored; array order matters. `senderName`, `source`, `finishReason`
-and observation provenance are excluded. Parts exclude `providerMetadata` and
-tool calls' raw `toolType` from identity; both remain in the returned messages.
-All other non-reasoning part fields, including tool arguments with those names,
-are included.
+Messages compare **role + allowlisted part fields** (see `comparison.ts`).
+Provider annotations (`providerMetadata`, `providerExecuted`) and provenance are
+ignored. Tool arguments and JSON results are compared in full. Object-property
+order is ignored; array order matters.
+Prepared messages reuse `identityKey` and `enrichmentKey` during selection and replay.
- **Inputs:** append only occurrences beyond the number already shown in the
thread, so replayed history disappears but additional identical copies survive (eg user responds "Thank you" twice).
- **Outputs:** always append, then count them so subsequent inputs do not repeat them.
-- **Reasoning:** compare all reasoning parts in a message as one ordered group,
- counted by role and content within the thread. Preserve reasoning in new messages;
+- **Enrichment:** compare all enrichment parts in a message as one ordered group,
+ counted by role and content within the thread. Preserve enrichment in new messages;
on replay, retain only additional occurrences of the whole group. A changed group
is retained in full, without deduplicating its individual parts. Newly seen
- reasoning in a repeated message is appended on
+ enrichment in a repeated message is appended on
its own with the input observation's provenance; it does not duplicate the
accompanying text or calls or rewrite an earlier output. The same history/current
turn split applies to these messages.
@@ -228,7 +227,7 @@ are included.
`[B, C, A, New]` adds only `New`. Existing display order stays `[A, B, C]`.
- Additional identical input occurrences and always-visible outputs.
- New system instructions within a continuing conversation.
-- Missing or changed reasoning in otherwise matching history.
+- Missing or changed enrichment in otherwise matching history.
- Multiple threads when their histories distinguish them.
## Current limitations
@@ -239,7 +238,7 @@ are included.
- **Identical conversations:** unrelated conversations with matching history can join.
- **Whole-message matching:** equivalent content split into different messages
or parts may not match. Reordering parts within a message also changes identity.
- Splitting or regrouping reasoning can retain repeated reasoning content.
+ Splitting or regrouping enrichment can retain repeated enrichment content.
Registered tool responses are the exception: replay is matched by call ID
within the thread (and occurrence for reused IDs), even when grouped with other parts.
- **Name matching is best-effort:** same-name parallel executions can start in a
@@ -273,7 +272,12 @@ transcript/
β”œβ”€β”€ ordering.test.ts ordering rules
β”œβ”€β”€ transcript.ts assembleTranscript
β”œβ”€β”€ limit.ts optional hard limit on serialized JSON
-β”œβ”€β”€ threads.ts thread selection and message deduplication
+β”œβ”€β”€ comparison.ts messageKey
+β”œβ”€β”€ comparison.test.ts focused equality regressions
+β”œβ”€β”€ messages.ts message preparation and part classification
+β”œβ”€β”€ continuity.ts contributesToContinuity
+β”œβ”€β”€ replay.ts createMessageDeduplicator
+β”œβ”€β”€ threads.ts thread selection and message appending
β”œβ”€β”€ tool-calls.ts tool matching and response association
β”œβ”€β”€ types.ts Transcript, Thread, Turn, ThreadMessage
└── fixtures/
diff --git a/packages/shared/src/server/transcript/comparison.test.ts b/packages/shared/src/server/transcript/comparison.test.ts
new file mode 100644
index 0000000000..e0e6e4ad50
--- /dev/null
+++ b/packages/shared/src/server/transcript/comparison.test.ts
@@ -0,0 +1,69 @@
+import { describe, expect, it } from "vitest";
+import type {
+ NormalizedMessage,
+ ToolCallPart,
+} from "../../utils/normalized-io";
+import { messageKey } from "./comparison";
+
+describe("messageKey", () => {
+ it("matches a tool call replayed with execution and provider annotations", () => {
+ const call: ToolCallPart = {
+ type: "tool-call",
+ toolCallId: "call-search",
+ toolName: "search",
+ input: { query: "Langfuse", limit: 2 },
+ };
+ const original: NormalizedMessage = {
+ role: "assistant",
+ source: "output",
+ parts: [call],
+ };
+
+ for (const providerExecuted of [true, false]) {
+ const replay: NormalizedMessage = {
+ role: "assistant",
+ source: "input",
+ parts: [
+ {
+ ...call,
+ input: { limit: 2, query: "Langfuse" },
+ toolType: "function",
+ providerExecuted,
+ providerMetadata: { provider: { id: "executed-search" } },
+ },
+ ],
+ };
+ expect(messageKey(replay)).toBe(messageKey(original));
+ }
+ });
+
+ it("distinguishes changed tool arguments even when they are named like annotations", () => {
+ const call: ToolCallPart = {
+ type: "tool-call",
+ toolCallId: "call-search",
+ toolName: "search",
+ input: {
+ providerExecuted: false,
+ providerMetadata: { query: "Langfuse" },
+ },
+ };
+ const original: NormalizedMessage = {
+ role: "assistant",
+ source: "output",
+ parts: [call],
+ };
+
+ for (const input of [
+ { providerExecuted: true, providerMetadata: { query: "Langfuse" } },
+ { providerExecuted: false, providerMetadata: { query: "Other results" } },
+ ]) {
+ expect(
+ messageKey({
+ ...original,
+ source: "input",
+ parts: [{ ...call, input }],
+ }),
+ ).not.toBe(messageKey(original));
+ }
+ });
+});
diff --git a/packages/shared/src/server/transcript/comparison.ts b/packages/shared/src/server/transcript/comparison.ts
new file mode 100644
index 0000000000..7a0b08a58f
--- /dev/null
+++ b/packages/shared/src/server/transcript/comparison.ts
@@ -0,0 +1,66 @@
+import type {
+ NormalizedMessage,
+ NormalizedMessagePart,
+} from "../../utils/normalized-io";
+
+/** Identify selected message parts independently of provider annotations and provenance. */
+export function messageKey(message: NormalizedMessage): string {
+ return canonicalJson([message.role, message.parts.map(comparablePart)]);
+}
+
+function comparablePart(part: NormalizedMessagePart): Record<string, unknown> {
+ switch (part.type) {
+ case "text":
+ return { type: part.type, text: part.text, refusal: part.refusal };
+ case "tool-call":
+ return {
+ type: part.type,
+ toolCallId: part.toolCallId,
+ toolName: part.toolName,
+ input: part.input,
+ invalid: part.invalid,
+ };
+ case "tool-result":
+ return {
+ type: part.type,
+ toolCallId: part.toolCallId,
+ toolName: part.toolName,
+ output: {
+ type: part.output.type,
+ value:
+ part.output.type === "content"
+ ? part.output.value.map(comparablePart)
+ : part.output.value,
+ },
+ isError: part.isError,
+ };
+ case "file":
+ return {
+ type: part.type,
+ content: part.content,
+ mediaType: part.mediaType,
+ filename: part.filename,
+ };
+ case "data":
+ return { type: part.type, name: part.name, value: part.value };
+ case "custom":
+ return { type: part.type, kind: part.kind, value: part.value };
+ case "reasoning":
+ return { type: part.type, content: part.content };
+ }
+}
+
+const isRecord = (value: unknown): value is Record<string, unknown> =>
+ value !== null && typeof value === "object" && !Array.isArray(value);
+
+function canonicalJson(value: unknown): string {
+ return JSON.stringify(value, (_key, value: unknown) =>
+ isRecord(value)
+ ? Object.fromEntries(
+ Object.keys(value)
+ .sort()
+ .map((key) => [key, value[key]]),
+ )
+ : value,
+ );
+}
diff --git a/packages/shared/src/server/transcript/continuity.ts b/packages/shared/src/server/transcript/continuity.ts
new file mode 100644
index 0000000000..3951ca4173
--- /dev/null
+++ b/packages/shared/src/server/transcript/continuity.ts
@@ -0,0 +1,6 @@
+import type { PreparedMessage } from "./messages";
+
+/** System instructions and enrichment alone cannot establish continuity. */
+export function contributesToContinuity(message: PreparedMessage) {
+ return message.role !== "system" && message.identityKey !== undefined;
+}
diff --git a/packages/shared/src/server/transcript/messages.ts b/packages/shared/src/server/transcript/messages.ts
new file mode 100644
index 0000000000..3cefbe6dc3
--- /dev/null
+++ b/packages/shared/src/server/transcript/messages.ts
@@ -0,0 +1,34 @@
+import { partition } from "lodash";
+import type { NormalizedMessage } from "../../utils/normalized-io";
+import { messageKey } from "./comparison";
+
+export type MessageKeys = {
+ /** Identifies parts used to establish continuity and detect replay. */
+ identityKey?: string;
+ /** Identifies additional parts preserved and deduplicated independently. */
+ enrichmentKey?: string;
+};
+
+export type PreparedMessage = NormalizedMessage & MessageKeys;
+
+export function prepareMessage(message: NormalizedMessage): PreparedMessage {
+ const [enrichmentParts, identityParts] = partition(
+ message.parts,
+ isEnrichmentPart,
+ );
+ return {
+ ...message,
+ identityKey:
+ identityParts.length > 0
+ ? messageKey({ ...message, parts: identityParts })
+ : undefined,
+ enrichmentKey:
+ enrichmentParts.length > 0
+ ? messageKey({ ...message, parts: enrichmentParts })
+ : undefined,
+ };
+}
+
+export const isEnrichmentPart = (part: NormalizedMessage["parts"][number]) =>
+ part.type === "reasoning" ||
+ (part.type === "file" && part.reasoning === true);
diff --git a/packages/shared/src/server/transcript/replay.ts b/packages/shared/src/server/transcript/replay.ts
new file mode 100644
index 0000000000..a4bbe40d58
--- /dev/null
+++ b/packages/shared/src/server/transcript/replay.ts
@@ -0,0 +1,58 @@
+import { isEnrichmentPart, type PreparedMessage } from "./messages";
+
+type Occurrences = { shown: number; seen: number };
+
+/** Match each input occurrence to one already-shown occurrence within a generation. */
+export function createMessageDeduplicator(messages: PreparedMessage[]) {
+ const identities = new Map<string, Occurrences>();
+ const enrichments = new Map<string, Occurrences>();
+
+ const countsFor = (
+ counts: Map<string, Occurrences>,
+ key: string | undefined,
+ ) => {
+ if (key === undefined) return undefined;
+ const existing = counts.get(key);
+ if (existing) return existing;
+ const entry = { shown: 0, seen: 0 };
+ counts.set(key, entry);
+ return entry;
+ };
+
+ // Identity and enrichment parts may be replayed together or separately.
+ for (const message of messages) {
+ const identity = countsFor(identities, message.identityKey);
+ const enrichment = countsFor(enrichments, message.enrichmentKey);
+ if (identity) identity.shown++;
+ if (enrichment) enrichment.shown++;
+ }
+
+ return (message: PreparedMessage) => {
+ const whole = {
+ parts: message.parts,
+ identityKey: message.identityKey,
+ enrichmentKey: message.enrichmentKey,
+ isEnrichment: false,
+ };
+ if (message.source === "output") return whole;
+
+ const identity = countsFor(identities, message.identityKey);
+ const enrichment = countsFor(enrichments, message.enrichmentKey);
+ const repeatsIdentity = !identity || ++identity.seen <= identity.shown;
+ const repeatsEnrichment =
+ !enrichment || ++enrichment.seen <= enrichment.shown;
+
+ // Count retained enrichment before checking later input occurrences.
+ if (enrichment && (!repeatsIdentity || !repeatsEnrichment))
+ enrichment.shown++;
+ if (!repeatsIdentity) return whole;
+ if (!repeatsEnrichment)
+ return {
+ ...whole,
+ parts: message.parts.filter(isEnrichmentPart),
+ identityKey: undefined,
+ isEnrichment: message.identityKey !== undefined,
+ };
+ return undefined;
+ };
+}
diff --git a/packages/shared/src/server/transcript/threads.ts b/packages/shared/src/server/transcript/threads.ts
index 16b58cf325..7a503f0afb 100644
--- a/packages/shared/src/server/transcript/threads.ts
+++ b/packages/shared/src/server/transcript/threads.ts
@@ -1,146 +1,90 @@
-import { partition } from "lodash";
import type { NormalizedMessage } from "../../utils/normalized-io";
import type { Thread, ThreadMessage } from "./types";
import type { OrderedObservation } from "./ordering";
import type { ToolCallRegistry } from "./tool-calls";
+import { contributesToContinuity } from "./continuity";
+import {
+ prepareMessage,
+ type MessageKeys,
+ type PreparedMessage,
+} from "./messages";
+import { createMessageDeduplicator } from "./replay";
export type TranscriptObservation = OrderedObservation & { traceId: string };
-export type KeyedMessage = {
- message: NormalizedMessage;
- key: string;
-};
-
type Contributor = Pick<
TranscriptObservation,
"id" | "traceId" | "type" | "nestingLevel"
>;
-/** Thread message with the key it was deduplicated by; tool responses have none. */
-export type EmittedMessage = ThreadMessage & { key?: string };
-
export type ThreadState = {
- /** Every message in display order; `splitTurn` derives the public shape. */
- messages: EmittedMessage[];
+ /** Display order; attached tool responses carry no comparison keys. */
+ messages: (ThreadMessage & MessageKeys)[];
/** Observations in the order they first contributed a message. */
contributors: Contributor[];
- /** Added reasoning from replayed content must not establish a turn boundary. */
- replaySupplements: WeakSet<EmittedMessage>;
+ /** Enrichment added to replayed identity parts must not establish a turn boundary. */
+ replayEnrichments: WeakSet<ThreadMessage>;
};
export const createThread = (): ThreadState => ({
messages: [],
contributors: [],
- replaySupplements: new WeakSet(),
+ replayEnrichments: new WeakSet(),
});
-const isRecord = (value: unknown): value is Record<string, unknown> =>
- value !== null && typeof value === "object" && !Array.isArray(value);
-
-const isReasoningPart = (part: NormalizedMessage["parts"][number]) =>
- part.type === "reasoning" ||
- (part.type === "file" && part.reasoning === true);
-
-/**
- * Identity of a message: role and all parts, including reasoning. Provider
- * annotations, property order, and provenance do not define identity.
- */
-export function messageKey(message: NormalizedMessage): string {
- const parts = message.parts.map(
- ({ providerMetadata: _metadata, ...part }) => {
- if (part.type === "tool-call") {
- const { toolType: _toolType, ...call } = part;
- return call;
- }
- return part;
- },
- );
- return JSON.stringify([message.role, parts], (_key, value: unknown) =>
- isRecord(value)
- ? Object.fromEntries(
- Object.keys(value)
- .sort()
- .map((key) => [key, value[key]]),
- )
- : value,
- );
-}
-
-/** Reasoning can change without breaking conversational continuity. */
-function continuityKey(message: NormalizedMessage, fullKey?: string): string {
- if (fullKey !== undefined && !message.parts.some(isReasoningPart))
- return fullKey;
- return messageKey({
- ...message,
- parts: message.parts.filter((part) => !isReasoningPart(part)),
- });
-}
-
-function contributesToThreadContinuity(message: NormalizedMessage) {
- return (
- message.role !== "system" &&
- message.parts.some((part) => !isReasoningPart(part))
- );
-}
-
/** Find the newest thread whose continuity-defining messages all appear in the input. */
-export function findThread(threads: ThreadState[], input: KeyedMessage[]) {
- const inputKeys = new Set(
- input
- .filter(({ message }) => contributesToThreadContinuity(message))
- .map(({ message, key }) => continuityKey(message, key)),
+export function findThread(threads: ThreadState[], input: PreparedMessage[]) {
+ const incoming = new Set(
+ input.filter(contributesToContinuity).map((message) => message.identityKey),
);
return threads.findLast(({ messages }) => {
- const defining = messages.filter(
- (message) =>
- message.key !== undefined && contributesToThreadContinuity(message),
- );
+ const defining = messages.filter(contributesToContinuity);
return (
defining.length > 0 &&
- defining.every((message) =>
- inputKeys.has(continuityKey(message, message.key)),
- )
+ defining.every((message) => incoming.has(message.identityKey))
);
});
}
/**
* Append a generation's messages. The n-th input occurrence of a key repeats
- * the n-th message the thread already shows with that key. If only reasoning
- * differs, retain the new reasoning at its arrival position. Other inputs and
+ * the n-th message the thread already shows with that key. If only enrichment
+ * differs, retain the new enrichment at its arrival position. Other inputs and
* all outputs are emitted. Registered tool results attach to their calls.
*/
export function append(
thread: ThreadState,
observation: TranscriptObservation,
- input: KeyedMessage[],
- output: KeyedMessage[],
+ input: PreparedMessage[],
+ output: PreparedMessage[],
toolCalls: ToolCallRegistry,
) {
const deduplicate = createMessageDeduplicator(thread.messages);
const calls = toolCalls.forGeneration(thread, observation, input, output);
- for (const { message, key: fullKey } of [...input, ...output]) {
+ for (const message of [...input, ...output]) {
const results: NormalizedMessage["parts"] = [];
const parts: NormalizedMessage["parts"] = [];
for (const part of message.parts) {
(calls.claims(part) ? results : parts).push(part);
}
- if (parts.length) {
- const key = results.length ? messageKey({ ...message, parts }) : fullKey;
- const retained = deduplicate({ message: { ...message, parts }, key });
+ if (parts.length > 0) {
+ const remaining =
+ results.length > 0 ? prepareMessage({ ...message, parts }) : message;
+ const retained = deduplicate(remaining);
if (retained) {
- const emitted: EmittedMessage = {
+ const emitted: ThreadMessage & MessageKeys = {
...message,
parts: retained.parts,
- key: retained.key,
+ identityKey: retained.identityKey,
+ enrichmentKey: retained.enrichmentKey,
observationId: observation.id,
traceId: observation.traceId,
startTime: observation.startTime,
endTime: observation.endTime,
};
thread.messages.push(emitted);
- if (retained.isSupplement) thread.replaySupplements.add(emitted);
+ if (retained.isEnrichment) thread.replayEnrichments.add(emitted);
addContributor(thread, observation);
calls.register(emitted);
}
@@ -149,64 +93,6 @@ export function append(
}
}
-/** Match each input occurrence to one already-shown occurrence. */
-function createMessageDeduplicator(messages: EmittedMessage[]) {
- const shown = new Map<string, number>();
- const seen = new Map<string, number>();
-
- const isDuplicate = (key: string) => {
- const occurrence = (seen.get(key) ?? 0) + 1;
- seen.set(key, occurrence);
- return occurrence <= (shown.get(key) ?? 0);
- };
-
- // An answer and its reasoning may later be replayed together or separately.
- const recordContentAndReasoningSeparately = (message: EmittedMessage) => {
- if (message.key === undefined) return;
-
- const [reasoning, content] = partition(message.parts, isReasoningPart);
- for (const parts of [content, reasoning]) {
- if (!parts.length) continue;
-
- const key =
- parts.length === message.parts.length
- ? message.key
- : messageKey({ ...message, parts });
- shown.set(key, (shown.get(key) ?? 0) + 1);
- }
- };
-
- for (const message of messages) recordContentAndReasoningSeparately(message);
-
- return ({ message, key }: KeyedMessage) => {
- const whole = { parts: message.parts, key, isSupplement: false };
- if (message.source === "output") return whole;
-
- const [reasoning, content] = partition(message.parts, isReasoningPart);
- const reasoningKey = content.length
- ? messageKey({ ...message, parts: reasoning })
- : key;
- const repeatsContent =
- !content.length ||
- isDuplicate(reasoning.length ? continuityKey(message) : key);
- const repeatsReasoning = !reasoning.length || isDuplicate(reasoningKey);
-
- // New messages retain their reasoning; count it before checking later replays.
- if (reasoning.length && (!repeatsContent || !repeatsReasoning))
- shown.set(reasoningKey, (shown.get(reasoningKey) ?? 0) + 1);
- if (!repeatsContent) return whole;
-
- if (!repeatsReasoning)
- return {
- parts: reasoning,
- key: reasoningKey,
- isSupplement: content.length > 0,
- };
-
- return undefined;
- };
-}
-
export function addContributor(
thread: ThreadState,
observation: TranscriptObservation,
@@ -230,7 +116,7 @@ export function addContributor(
export function splitTurn({
messages,
contributors,
- replaySupplements,
+ replayEnrichments,
}: ThreadState): Thread {
// Earlier traces are history; the last contributing trace is the current turn.
const currentTraceId = contributors.at(-1)?.traceId;
@@ -246,12 +132,12 @@ export function splitTurn({
replayed.findLastIndex(
(message) =>
(message.role === "assistant" || message.role === "tool") &&
- !replaySupplements.has(message),
+ !replayEnrichments.has(message),
) +
1;
while (
turnStart < messages.length &&
- replaySupplements.has(messages[turnStart]!)
+ replayEnrichments.has(messages[turnStart]!)
)
turnStart++;
const current = messages.slice(turnStart);
@@ -267,11 +153,24 @@ export function splitTurn({
conversationHistory: messages
.slice(0, turnStart)
.map(
- ({ key: _key, observationId, traceId, startTime, endTime, ...rest }) =>
- rest,
+ ({
+ identityKey: _identityKey,
+ enrichmentKey: _enrichmentKey,
+ observationId,
+ traceId,
+ startTime,
+ endTime,
+ ...rest
+ }) => rest,
),
currentTurn: {
- messages: current.map(({ key: _key, ...message }) => message),
+ messages: current.map(
+ ({
+ identityKey: _identityKey,
+ enrichmentKey: _enrichmentKey,
+ ...message
+ }) => message,
+ ),
nestingLevel: firstGeneration?.nestingLevel ?? 0,
observations: currentContributors.map(({ id, traceId }) => ({
id,
diff --git a/packages/shared/src/server/transcript/tool-calls.ts b/packages/shared/src/server/transcript/tool-calls.ts
index 0cc84f07e5..81b4e7e6df 100644
--- a/packages/shared/src/server/transcript/tool-calls.ts
+++ b/packages/shared/src/server/transcript/tool-calls.ts
@@ -7,8 +7,6 @@ import type {
import type { ThreadMessage } from "./types";
import {
addContributor,
- type EmittedMessage,
- type KeyedMessage,
type ThreadState,
type TranscriptObservation,
} from "./threads";
@@ -95,15 +93,15 @@ export function createToolCallRegistry() {
function forGeneration(
thread: ThreadState,
observation: TranscriptObservation,
- input: KeyedMessage[],
- output: KeyedMessage[],
+ input: NormalizedMessage[],
+ output: NormalizedMessage[],
) {
// Input results of a resolved call; output results resolve on attach.
const claimed = new Map<Part, Call | undefined>();
// Reused IDs need complete, ordered call history to disambiguate results.
const totals = new Map<string, number>();
- for (const { message } of input) {
+ for (const message of input) {
for (const part of message.parts) {
if (part.type === "tool-call" && part.toolCallId)
totals.set(part.toolCallId, (totals.get(part.toolCallId) ?? 0) + 1);
@@ -111,7 +109,7 @@ export function createToolCallRegistry() {
}
const occurrences = new Map<string, number>();
const threadCalls = callsByThread.get(thread);
- for (const { message } of input) {
+ for (const message of input) {
for (const part of message.parts) {
if (part.type === "tool-call" && part.toolCallId)
occurrences.set(
@@ -140,7 +138,7 @@ export function createToolCallRegistry() {
// Output results attach to calls registered earlier, including by a
// preceding part of the same generation's output.
const outputCallIds = new Set<string>();
- for (const { message } of output) {
+ for (const message of output) {
for (const part of message.parts) {
if (part.type === "tool-call" && part.toolCallId)
outputCallIds.add(part.toolCallId);
@@ -157,7 +155,7 @@ export function createToolCallRegistry() {
return {
claims: (part: Part) => claimed.has(part),
/** Register an emitted output's tool calls as anchors for their responses. */
- register(message: EmittedMessage) {
+ register(message: ThreadMessage) {
if (message.source !== "output") return;
for (const part of message.parts) {
@@ -180,9 +178,9 @@ export function createToolCallRegistry() {
function attachToolOutput(
observation: TranscriptObservation,
- output: KeyedMessage[],
+ output: NormalizedMessage[],
) {
- const parts = output.flatMap(({ message }) => message.parts);
+ const parts = output.flatMap((message) => message.parts);
if (!parts.length) return;
let hasId = false;
diff --git a/packages/shared/src/server/transcript/transcript.ts b/packages/shared/src/server/transcript/transcript.ts
index 4231642e13..c9bb00172b 100644
--- a/packages/shared/src/server/transcript/transcript.ts
+++ b/packages/shared/src/server/transcript/transcript.ts
@@ -7,11 +7,11 @@ import {
append,
createThread,
findThread,
- messageKey,
splitTurn,
type ThreadState,
} from "./threads";
import { createToolCallRegistry } from "./tool-calls";
+import { prepareMessage } from "./messages";
/** Check if an observation is relevant for the transcript. */
const isRelevantObservation = (
@@ -33,8 +33,8 @@ function normalize(observation: OrderedObservation) {
},
});
return partition(
- messages.map((message) => ({ message, key: messageKey(message) })),
- ({ message }) => message.source === "input",
+ messages.map(prepareMessage),
+ (message) => message.source === "input",
);
}
@@ -42,8 +42,8 @@ function normalize(observation: OrderedObservation) {
* Assemble threads from observations: normalize their I/O, reconcile replayed
* history, and retain first-seen provenance. The caller supplies the
* observations in transcript order, see `orderObservations`; they are consumed
- * as given. Optional timings separate normalization (including initial message
- * keys) from remaining assembly work, excluding ordering and optional truncation.
+ * as given. Optional timings separate normalization and key preparation from
+ * remaining assembly work, excluding ordering and optional truncation.
*/
export function assembleTranscript(
orderedObservations: OrderedObservation[],