Skip to content

A self-improving RLM agent for coding workflows and long-running autonomous tasks.

Rust

feat(coding-agent): per-request provider timing diagnostics (PI_REQUEST_TIMING) (#2462)

Seth Karten690e23d8c8d7
diff --git a/packages/coding-agent/.changes/request-timing.md b/packages/coding-agent/.changes/request-timing.md
new file mode 100644
index 0000000000..635427b04b
--- /dev/null
+++ b/packages/coding-agent/.changes/request-timing.md
@@ -0,0 +1 @@
+- New request timing diagnostics: `PI_REQUEST_TIMING=1` (or settings `requestTiming: true`) logs each provider request's phase timeline - prompt-built, request-sent (with body bytes), first-byte, first-token, stream-done - to `~/.prime/agent/logs/agent.jsonl` under `coding-agent.request-timing`, so a long `Waiting` state can be attributed to client-side build, upload, provider prefill, or a prompt-cache miss (summary usage shows cacheRead/cacheWrite). Zero overhead when disabled.
diff --git a/packages/coding-agent/docs/development.md b/packages/coding-agent/docs/development.md
index 22631cb0a3..5e57ecd54b 100644
--- a/packages/coding-agent/docs/development.md
+++ b/packages/coding-agent/docs/development.md
@@ -68,6 +68,33 @@ Do not resolve packaged assets directly from `__dirname`.
The hidden `/debug` command writes `~/.prime/agent/prime-agent-debug.log` with rendered TUI lines, their visible widths, and the current agent messages. Daemon, worker, client, and provider diagnostic logs live under `~/.prime/agent/logs/`.
+### Request timing
+
+When the UI sits in `Waiting` for a long time before the first model output appears, request timing shows what that wait is made of. Enable it with `PI_REQUEST_TIMING=1` in the environment (restart the daemon so worker processes inherit it) or `"requestTiming": true` in settings.json (applies to sessions opened after the change). Each agent-loop provider request then logs its phase timeline to `~/.prime/agent/logs/agent.jsonl` under the component `coding-agent.request-timing`. One-shot completion calls outside the agent loop (compaction, branch-summary, and refinement requests) use a separate path and are not logged:
+
+```bash
+grep '"coding-agent.request-timing"' ~/.prime/agent/logs/agent.jsonl | tail -5
+```
+
+Phases per request, in order. Each entry carries the gap it closed (`phaseMs`) and the elapsed time since turn dispatch (`totalMs`); `prompt-built` fires before the request has model or session fields, so correlate it with the later entries by `requestSeq`. If a provider never reports a phase (for example `request-sent` when no payload hook runs), that delta is simply omitted:
+
+| Phase | Meaning | A long gap here means |
+|-------|---------|----------------------|
+| `prompt-built` | Turn dispatched, prompt message array built | Client-side prompt build is slow |
+| `request-sent` | Payload handed to the provider client | Client-side request build (auth, params, extension payload hooks, the `requestBytes` serialization) is slow |
+| `first-byte` | HTTP response headers received (`requestBytes`: serialized request body size in UTF-8 bytes) | Upload of the request body plus provider TTFB (prefill, prompt-cache miss, queueing) |
+| `first-token` | First streamed content block (what clears `Waiting`) | Stream parse delay; usually near zero |
+| `stream-done` | Terminal event, with the summary below | Slow full stream |
+
+The final entry (`msg: "request timing summary"`) carries every delta it measured: `phases.dispatchToPromptBuiltMs`, `promptBuiltToRequestSentMs`, `requestSentToFirstByteMs`, `firstByteToFirstTokenMs`, `firstTokenToStreamDoneMs`, plus `contextEntries`, `requestBytes`, `sessionId`, the final `usage`, and an `outcome` (`done`, `aborted` for an early stop such as a user cancel, or `failed` when the provider failed before or during the stream).
+
+Reading it for a slow large-context turn:
+
+- Large `promptBuiltToRequestSentMs` — the client is slow. Both phases before the request leaves are client-side.
+- Large `requestSentToFirstByteMs` with a normal TTFT on the inference dashboard — the time is spent on the wire or inside inference before the dashboard's TTFT timer starts: uploading a multi-MB request body on a slow uplink, provider queueing, or prefilling an uncached prompt. `requestBytes` shows how much had to be uploaded.
+- Prompt-cache miss check — the summary's `usage.cacheRead` near 0 with `cacheWrite` close to the full prompt size means the provider re-prefilled the whole context instead of reading its prompt cache. `sessionId` is the cache key the client sends for OpenAI-family providers (session-affinity headers and `prompt_cache_key`); Anthropic instead caches the message prefix, so watch `cacheRead`/`cacheWrite` there.
+- Every retry re-issues the turn and gets a fresh request sequence number (`requestSeq`), so a silent retry loop shows up as several request timelines plus the usual auto-retry status.
+
Useful service commands:
```bash
diff --git a/packages/coding-agent/docs/settings.md b/packages/coding-agent/docs/settings.md
index e3f468c003..620f4c74ee 100644
--- a/packages/coding-agent/docs/settings.md
+++ b/packages/coding-agent/docs/settings.md
@@ -280,6 +280,12 @@ available, authenticated model, the bounded wait runs instead.
}
```
+### Diagnostics
+
+| Setting | Type | Default | Description |
+|---------|------|---------|-------------|
+| `requestTiming` | boolean | `false` | Log per-request provider timing phases to the diagnostic log (see [Development: Request timing](development.md#request-timing)); `PI_REQUEST_TIMING=1` also enables it |
+
### Message Delivery
| Setting | Type | Default | Description |
diff --git a/packages/coding-agent/src/core/request-timing.ts b/packages/coding-agent/src/core/request-timing.ts
new file mode 100644
index 0000000000..d0544928d1
--- /dev/null
+++ b/packages/coding-agent/src/core/request-timing.ts
@@ -0,0 +1,459 @@
+/**
+ * Per-request phase timing for provider requests.
+ *
+ * Answers "what is the agent waiting for" when the TUI sits in `Waiting`
+ * before the first reasoning/text token appears: the wait is split into
+ * client-side phases (turn dispatch -> prompt-built -> request-sent) and
+ * wire/server phases (request-sent -> first-byte covers request body
+ * serialization, upload, and provider TTFB). A long request-sent ->
+ * first-byte gap with a normal server-side TTFT points at slow upload or
+ * provider prefill/prompt-cache miss rather than client work, and the
+ * stream-done summary carries the final usage so a cache miss shows as
+ * cacheRead ~ 0 with cacheWrite ~ the full prompt.
+ *
+ * Enable with `PI_REQUEST_TIMING=1` (env, inherited by daemon workers) or
+ * `"requestTiming": true` in settings.json. Entries go to the shared JSONL
+ * diagnostic log (~/.prime/agent/logs/agent.jsonl) under the component
+ * "coding-agent.request-timing".
+ *
+ * Zero overhead when disabled: the wrappers pass straight through with no
+ * timestamps, no payload serialization, and no log entries.
+ */
+
+import type { AgentMessage, StreamFn } from "@earendil-works/pi-agent-core";
+import { type AssistantMessageEventStream, getLogger, type Message } from "@earendil-works/pi-ai";
+
+const log = getLogger("coding-agent.request-timing");
+
+const REQUEST_TIMING_ENV = "PI_REQUEST_TIMING";
+
+/** Whether request timing is on. Evaluated per request so the flag can change without a restart. */
+type RequestTimingEnabled = () => boolean;
+
+/** Truthy follows the PI_OFFLINE/PI_TIMING convention: 1/true/yes. */
+function truthyEnvFlag(value: string | undefined): boolean {
+ if (!value) return false;
+ const normalized = value.toLowerCase();
+ return normalized === "1" || normalized === "true" || normalized === "yes";
+}
+
+/**
+ * Request timing is on when either the settings flag or the env override is
+ * set. Both checks are cheap property reads on the disabled path.
+ */
+export function isRequestTimingEnabled(settingsFlag: boolean): boolean {
+ return settingsFlag || truthyEnvFlag(process.env[REQUEST_TIMING_ENV]);
+}
+
+/** Correlation state recorded while the agent loop builds the request. */
+interface PromptBuildTiming {
+ /** Turn dispatch: streamAssistantResponse entry (first seam of the turn). */
+ dispatchedAt: number;
+ /** After convertToLlm: the LLM message array is built. */
+ promptBuiltAt: number;
+ /** LLM message count of the built prompt. */
+ contextEntries: number;
+}
+
+/** Dispatch timestamps keyed by the transformContext result array. */
+const dispatchedAtByContext = new WeakMap<object, number>();
+/** Built-prompt timing keyed by the convertToLlm output array (fresh per turn). */
+const promptBuildByLlmMessages = new WeakMap<object, PromptBuildTiming>();
+/** Per-request sequence numbers keyed by the LLM messages array. */
+const requestSeqByLlmMessages = new WeakMap<object, number>();
+let moduleRequestSeq = 0;
+
+/**
+ * Record the turn dispatch time for the array that continues into prompt
+ * build. The agent context snapshot array is reused across turns of one
+ * run, so the latest turn overwrites the previous entry; concurrent agents
+ * use distinct arrays and cannot collide.
+ */
+function markRequestTimingDispatch(messages: object[], dispatchedAt: number): void {
+ dispatchedAtByContext.set(messages, dispatchedAt);
+}
+
+function takeRequestTimingDispatch(messages: object[]): number | undefined {
+ return dispatchedAtByContext.get(messages);
+}
+
+/**
+ * Instruments the agent-loop transformContext seam: the entry timestamp is
+ * the request's "send-received" moment (turn dispatch).
+ */
+export function instrumentTransformContext(
+ enabled: RequestTimingEnabled,
+ transform: (messages: AgentMessage[], signal?: AbortSignal) => Promise<AgentMessage[]>,
+): (messages: AgentMessage[], signal?: AbortSignal) => Promise<AgentMessage[]> {
+ return async (messages: AgentMessage[], signal?: AbortSignal) => {
+ if (!enabled()) {
+ return transform(messages, signal);
+ }
+ const startedAt = performance.now();
+ const result = await transform(messages, signal);
+ markRequestTimingDispatch(result, startedAt);
+ return result;
+ };
+}
+
+/**
+ * Instruments the agent-loop convertToLlm seam: records the prompt-built
+ * phase (with the LLM message count) keyed by the output array the provider
+ * request consumes, and emits the prompt-built phase entry.
+ */
+export function instrumentConvertToLlm(
+ enabled: RequestTimingEnabled,
+ convert: (messages: AgentMessage[]) => Message[],
+): (messages: AgentMessage[]) => Message[] {
+ return (messages: AgentMessage[]) => {
+ if (!enabled()) {
+ return convert(messages);
+ }
+ const output = convert(messages);
+ const startedAt = takeRequestTimingDispatch(messages) ?? performance.now();
+ const built = performance.now();
+ const timing: PromptBuildTiming = {
+ dispatchedAt: startedAt,
+ promptBuiltAt: built,
+ contextEntries: output.length,
+ };
+ promptBuildByLlmMessages.set(output, timing);
+ log.info("request timing", {
+ phase: "prompt-built",
+ requestSeq: nextRequestSeq(output),
+ phaseMs: roundMs(built - startedAt),
+ totalMs: roundMs(built - startedAt),
+ contextEntries: output.length,
+ });
+ return output;
+ };
+}
+
+function nextRequestSeq(llmMessages: object): number {
+ let seq = requestSeqByLlmMessages.get(llmMessages);
+ if (seq === undefined) {
+ seq = ++moduleRequestSeq;
+ requestSeqByLlmMessages.set(llmMessages, seq);
+ }
+ return seq;
+}
+
+/** Provider request identity fields shared by every phase entry. */
+interface RequestTimingRequestInfo {
+ model: string;
+ provider?: string;
+ api?: string;
+ sessionId?: string;
+}
+
+/**
+ * Mutable phase clock for one provider request. Created at the streamFn
+ * seam; phase transitions are logged as they happen so a hung request shows
+ * the last completed phase in the live log.
+ */
+class RequestTiming {
+ readonly requestSeq: number;
+ private readonly info: RequestTimingRequestInfo;
+ private readonly dispatchedAt: number | undefined;
+ private readonly promptBuiltAt: number | undefined;
+ private readonly contextEntries: number | undefined;
+ private readonly streamFnEnteredAt: number;
+ private requestSentAt: number | undefined;
+ private requestBytes: number | undefined;
+ private firstByteAt: number | undefined;
+ private firstTokenAt: number | undefined;
+ private streamDoneAt: number | undefined;
+ private stopReason: string | undefined;
+ private errorMessage: string | undefined;
+ private usage: { input: number; output: number; cacheRead: number; cacheWrite: number } | undefined;
+ private summaryEmitted = false;
+
+ constructor(llmMessages: object, info: RequestTimingRequestInfo, promptBuild: PromptBuildTiming | undefined) {
+ this.requestSeq = nextRequestSeq(llmMessages);
+ this.info = info;
+ this.dispatchedAt = promptBuild?.dispatchedAt;
+ this.promptBuiltAt = promptBuild?.promptBuiltAt;
+ this.contextEntries = promptBuild?.contextEntries;
+ this.streamFnEnteredAt = performance.now();
+ }
+
+ /** request-sent: payload handed to the provider client. */
+ markRequestSent(): void {
+ if (this.requestSentAt !== undefined) return;
+ const now = performance.now();
+ this.requestSentAt = now;
+ const from = this.promptBuiltAt ?? this.streamFnEnteredAt;
+ this.emit("request-sent", {
+ phaseMs: roundMs(now - from),
+ totalMs: this.totalFromDispatch(now),
+ contextEntries: this.contextEntries,
+ });
+ }
+
+ /** Serialized request body size, measured before request-sent so its cost lands in the client-side phase. */
+ recordRequestBytes(requestBytes: number | undefined): void {
+ this.requestBytes = requestBytes;
+ }
+
+ /** first-byte: HTTP response headers received (provider TTFB complete). */
+ markFirstByte(): void {
+ if (this.firstByteAt !== undefined) return;
+ const now = performance.now();
+ this.firstByteAt = now;
+ const from = this.requestSentAt ?? this.promptBuiltAt ?? this.streamFnEnteredAt;
+ this.emit("first-byte", {
+ phaseMs: roundMs(now - from),
+ // Without a request-sent timestamp (provider never called onPayload) the
+ // delta spans from prompt-built, not just the wire wait.
+ ...(this.requestSentAt === undefined ? { phaseFrom: "prompt-built" } : {}),
+ totalMs: this.totalFromDispatch(now),
+ requestBytes: this.requestBytes,
+ });
+ }
+
+ /** first-content-token: first streamed content block (thinking/text/toolcall), which clears the TUI Waiting state. */
+ markFirstToken(): void {
+ if (this.firstTokenAt !== undefined) return;
+ const now = performance.now();
+ this.firstTokenAt = now;
+ const from = this.firstByteAt ?? this.requestSentAt ?? this.streamFnEnteredAt;
+ this.emit("first-token", {
+ phaseMs: roundMs(now - from),
+ totalMs: this.totalFromDispatch(now),
+ });
+ }
+
+ /** stream-done: terminal event observed on the provider stream. */
+ markStreamDone(stopReason: string, errorMessage?: string): void {
+ this.stopReason = stopReason;
+ this.errorMessage = errorMessage;
+ this.streamDoneAt = performance.now();
+ }
+
+ /** Final usage from the completed assistant message; cache read/write answers prompt-cache misses. */
+ markUsage(usage: { input: number; output: number; cacheRead: number; cacheWrite: number }): void {
+ this.usage = usage;
+ }
+
+ /** Emit the summary with every known phase delta. Safe to call once; later calls are ignored. */
+ emitSummary(outcome: "done" | "aborted" | "failed"): void {
+ if (this.summaryEmitted) return;
+ this.summaryEmitted = true;
+ const doneAt = this.streamDoneAt ?? performance.now();
+ const phases: Record<string, number | undefined> = {};
+ if (this.dispatchedAt !== undefined && this.promptBuiltAt !== undefined) {
+ phases.dispatchToPromptBuiltMs = roundMs(this.promptBuiltAt - this.dispatchedAt);
+ }
+ const requestStart = this.promptBuiltAt ?? this.streamFnEnteredAt;
+ if (this.requestSentAt !== undefined) {
+ phases.promptBuiltToRequestSentMs = roundMs(this.requestSentAt - requestStart);
+ }
+ if (this.requestSentAt !== undefined && this.firstByteAt !== undefined) {
+ phases.requestSentToFirstByteMs = roundMs(this.firstByteAt - this.requestSentAt);
+ }
+ const firstTokenFrom = this.firstByteAt ?? this.requestSentAt;
+ if (this.firstTokenAt !== undefined && firstTokenFrom !== undefined) {
+ phases.firstByteToFirstTokenMs = roundMs(this.firstTokenAt - firstTokenFrom);
+ }
+ if (this.firstTokenAt !== undefined) {
+ phases.firstTokenToStreamDoneMs = roundMs(doneAt - this.firstTokenAt);
+ }
+ log.info("request timing summary", {
+ phase: "stream-done",
+ requestSeq: this.requestSeq,
+ outcome,
+ ...this.identity(),
+ contextEntries: this.contextEntries,
+ requestBytes: this.requestBytes,
+ phases,
+ totalMs: this.totalFromDispatch(doneAt) ?? roundMs(doneAt - this.streamFnEnteredAt),
+ stopReason: this.stopReason,
+ errorMessage: this.errorMessage,
+ usage: this.usage,
+ });
+ }
+
+ private emit(phase: string, fields: Record<string, unknown>): void {
+ log.info("request timing", {
+ phase,
+ requestSeq: this.requestSeq,
+ ...this.identity(),
+ ...fields,
+ });
+ }
+
+ private identity(): Record<string, unknown> {
+ return {
+ model: this.info.model,
+ ...(this.info.provider ? { provider: this.info.provider } : {}),
+ ...(this.info.api ? { api: this.info.api } : {}),
+ ...(this.info.sessionId ? { sessionId: this.info.sessionId } : {}),
+ };
+ }
+
+ private totalFromDispatch(at: number): number | undefined {
+ return this.dispatchedAt !== undefined ? roundMs(at - this.dispatchedAt) : undefined;
+ }
+}
+
+/** Event types carrying the first visible model output; these clear the TUI Waiting state. */
+const FIRST_TOKEN_EVENT_TYPES = new Set([
+ "thinking_start",
+ "thinking_delta",
+ "text_start",
+ "text_delta",
+ "toolcall_start",
+ "toolcall_delta",
+]);
+
+function isRequestTimingFirstTokenEvent(type: string): boolean {
+ return FIRST_TOKEN_EVENT_TYPES.has(type);
+}
+
+/** Serialized request body size; providers serialize for the wire anyway, but only measure when timing is on. */
+function measureRequestBytes(payload: unknown): number | undefined {
+ try {
+ const serialized = JSON.stringify(payload);
+ // UTF-8 bytes as sent on the wire; .length would count UTF-16 code units and underreport non-ASCII prompts.
+ return serialized === undefined ? undefined : Buffer.byteLength(serialized, "utf-8");
+ } catch {
+ return undefined;
+ }
+}
+
+/**
+ * Instruments the streamFn seam: creates the per-request clock, chains the
+ * onPayload/onResponse hooks (request-sent / first-byte), wraps the event
+ * stream (first-token, stream-done), and emits a failed summary when auth
+ * rejects the request before it is sent.
+ */
+export function instrumentStreamFn(enabled: RequestTimingEnabled, streamFn: StreamFn): StreamFn {
+ return async (model, context, options) => {
+ if (!enabled()) {
+ return streamFn(model, context, options);
+ }
+ const llmMessages: object[] = context.messages ?? [];
+ const timing = new RequestTiming(
+ llmMessages,
+ {
+ model: model.id,
+ provider: model.provider,
+ api: model.api,
+ sessionId: options?.sessionId,
+ },
+ takePromptBuild(llmMessages),
+ );
+ try {
+ const stream = await streamFn(model, context, {
+ ...options,
+ onPayload: async (payload, payloadModel) => {
+ const next = await options?.onPayload?.(payload, payloadModel);
+ // Measured before request-sent: the provider awaits this hook before it
+ // opens the HTTP request, so the serialization cost belongs to the
+ // client-side build delta, not to request-sent -> first-byte.
+ timing.recordRequestBytes(measureRequestBytes(next ?? payload));
+ timing.markRequestSent();
+ return next;
+ },
+ onResponse: async (response, responseModel) => {
+ timing.markFirstByte();
+ await options?.onResponse?.(response, responseModel);
+ },
+ });
+ return wrapRequestTimingEventStream(stream, timing, options?.signal);
+ } catch (error) {
+ timing.emitSummary("failed");
+ throw error;
+ }
+ };
+}
+
+function takePromptBuild(llmMessages: object[]): PromptBuildTiming | undefined {
+ const timing = promptBuildByLlmMessages.get(llmMessages);
+ if (timing) {
+ promptBuildByLlmMessages.delete(llmMessages);
+ }
+ return timing;
+}
+
+/**
+ * Wrap a provider event stream so first-byte (fallback via the start event),
+ * first-content-token, and the terminal event are timed. The summary is
+ * emitted on the terminal event, or as aborted when iteration stops early.
+ */
+function wrapRequestTimingEventStream(
+ stream: AssistantMessageEventStream,
+ timing: RequestTiming,
+ signal?: AbortSignal,
+): AssistantMessageEventStream {
+ // Delegates to the provider stream so result/push/end (and instanceof checks)
+ // keep working; only iteration is overridden to observe phases.
+ const wrapped = Object.create(stream);
+ Object.defineProperty(wrapped, Symbol.asyncIterator, {
+ value: async function* () {
+ try {
+ try {
+ for await (const event of stream) {
+ switch (event?.type) {
+ case "start":
+ // Providers push start after response headers; used only when onResponse did not fire.
+ timing.markFirstByte();
+ break;
+ case "done":
+ timing.markStreamDone(event.message.stopReason, event.message.errorMessage);
+ timing.markUsage({
+ input: event.message.usage.input,
+ output: event.message.usage.output,
+ cacheRead: event.message.usage.cacheRead,
+ cacheWrite: event.message.usage.cacheWrite,
+ });
+ timing.emitSummary("done");
+ break;
+ case "error":
+ timing.markStreamDone(event.error.stopReason, event.error.errorMessage);
+ timing.markUsage({
+ input: event.error.usage.input,
+ output: event.error.usage.output,
+ cacheRead: event.error.usage.cacheRead,
+ cacheWrite: event.error.usage.cacheWrite,
+ });
+ // A terminal provider error is a failed (or aborted) request, not a completed one.
+ timing.emitSummary(event.error.stopReason === "aborted" ? "aborted" : "failed");
+ break;
+ default:
+ if (event && isRequestTimingFirstTokenEvent(event.type)) {
+ timing.markFirstToken();
+ }
+ break;
+ }
+ yield event;
+ }
+ } catch (error) {
+ // A provider-side crash is a failed request, not a user abort; an
+ // abort-driven rejection still reports as aborted. The error is
+ // re-thrown so the consumer still sees the failure.
+ const errorMessage = error instanceof Error ? error.message : String(error);
+ if (signal?.aborted) {
+ timing.markStreamDone("aborted", errorMessage);
+ timing.emitSummary("aborted");
+ } else {
+ timing.markStreamDone("error", errorMessage);
+ timing.emitSummary("failed");
+ }
+ throw error;
+ }
+ } finally {
+ // Early termination (abort, hung stream) still reports what was measured.
+ timing.emitSummary("aborted");
+ }
+ },
+ enumerable: true,
+ writable: true,
+ configurable: true,
+ });
+ return wrapped as unknown as AssistantMessageEventStream;
+}
+
+function roundMs(delta: number): number {
+ return Math.round(delta * 10) / 10;
+}
diff --git a/packages/coding-agent/src/core/sdk.ts b/packages/coding-agent/src/core/sdk.ts
index 4f1f7fd7ed..761113fd99 100644
--- a/packages/coding-agent/src/core/sdk.ts
+++ b/packages/coding-agent/src/core/sdk.ts
@@ -13,6 +13,12 @@ import { McpManager } from "./mcp/mcp-manager.js";
import { convertToLlm } from "./messages.js";
import { ModelRegistry } from "./model-registry.js";
import { findInitialModel, findSessionModelWithReadinessWait } from "./model-resolver.js";
+import {
+ instrumentConvertToLlm,
+ instrumentStreamFn,
+ instrumentTransformContext,
+ isRequestTimingEnabled,
+} from "./request-timing.js";
import type { ResourceLoader } from "./resource-loader.js";
import { DefaultResourceLoader } from "./resource-loader.js";
import { getDefaultSessionDir, SessionManager } from "./session-manager.js";
@@ -247,6 +253,8 @@ export async function createAgentSession(options: CreateAgentSessionOptions = {}
let agent: Agent;
+ const requestTimingEnabled = (): boolean => isRequestTimingEnabled(settingsManager.getRequestTiming());
+
const convertToLlmWithBlockImages = (messages: AgentMessage[]): Message[] => {
const converted = convertToLlm(messages);
if (!settingsManager.getBlockImages()) {
@@ -290,8 +298,8 @@ export async function createAgentSession(options: CreateAgentSessionOptions = {}
serviceTier,
tools: [],
},
- convertToLlm: convertToLlmWithBlockImages,
- streamFn: async (model, context, options) => {
+ convertToLlm: instrumentConvertToLlm(requestTimingEnabled, convertToLlmWithBlockImages),
+ streamFn: instrumentStreamFn(requestTimingEnabled, async (model, context, options) => {
const auth = await modelRegistry.getApiKeyAndHeaders(model, options?.headers);
if (!auth.ok) {
throw new Error(auth.error);
@@ -304,7 +312,7 @@ export async function createAgentSession(options: CreateAgentSessionOptions = {}
timeoutMs: options?.timeoutMs ?? providerRetrySettings.timeoutMs,
headers: auth.headers,
});
- },
+ }),
onPayload: async (payload, _model) => {
const runner = extensionRunnerRef.current;
if (!runner?.hasHandlers("before_provider_request")) {
@@ -324,11 +332,11 @@ export async function createAgentSession(options: CreateAgentSessionOptions = {}
});
},
sessionId: sessionManager.getSessionId(),
- transformContext: async (messages) => {
+ transformContext: instrumentTransformContext(requestTimingEnabled, async (messages) => {
const runner = extensionRunnerRef.current;
if (!runner) return messages;
return runner.emitContext(messages);
- },
+ }),
steeringMode: settingsManager.getSteeringMode(),
followUpMode: settingsManager.getFollowUpMode(),
transport: settingsManager.getTransport(),
diff --git a/packages/coding-agent/src/core/settings-manager.ts b/packages/coding-agent/src/core/settings-manager.ts
index ce045cea66..e8ef970dae 100644
--- a/packages/coding-agent/src/core/settings-manager.ts
+++ b/packages/coding-agent/src/core/settings-manager.ts
@@ -257,6 +257,8 @@ export interface Settings {
markdown?: MarkdownSettings;
warnings?: WarningSettings;
sessionDir?: string; // Custom session storage directory (same format as --session-dir CLI flag)
+ /** Log per-request provider timing phases to the diagnostic log. Default: false */
+ requestTiming?: boolean;
}
export interface AgentTracesSettings {
@@ -1368,6 +1370,10 @@ export class SettingsManager {
return this.settings.images?.blockImages ?? false;
}
+ getRequestTiming(): boolean {
+ return this.settings.requestTiming ?? false;
+ }
+
setBlockImages(blocked: boolean): void {
if (!this.globalSettings.images) {
this.globalSettings.images = {};
diff --git a/packages/coding-agent/test/request-timing.test.ts b/packages/coding-agent/test/request-timing.test.ts
new file mode 100644
index 0000000000..56c0bbc7e1
--- /dev/null
+++ b/packages/coding-agent/test/request-timing.test.ts
@@ -0,0 +1,276 @@
+import { mkdirSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import type { AgentMessage, StreamFn } from "@earendil-works/pi-agent-core";
+import {
+ type AssistantMessage,
+ type AssistantMessageEventStream,
+ type Context,
+ createAssistantMessageEventStream,
+ fauxAssistantMessage,
+ type LogEntry,
+ type Message,
+ type Model,
+ registerFauxProvider,
+ setLogSink,
+} from "@earendil-works/pi-ai";
+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
+import type { AgentSession } from "../src/core/agent-session.js";
+import { AuthStorage } from "../src/core/auth-storage.js";
+import { ModelRegistry } from "../src/core/model-registry.js";
+import { instrumentConvertToLlm, instrumentStreamFn, instrumentTransformContext } from "../src/core/request-timing.js";
+import { createAgentSession } from "../src/core/sdk.js";
+import { SessionManager } from "../src/core/session-manager.js";
+import { SettingsManager } from "../src/core/settings-manager.js";
+
+const model = {
+ id: "bench/bench-model",
+ api: "openai-completions",
+ provider: "bench",
+ baseUrl: "https://bench.test/v1",
+ reasoning: true,
+ input: ["text"],
+ contextWindow: 1_000_000,
+ maxTokens: 128_000,
+ cost: { input: 1, output: 2, cacheRead: 0.1, cacheWrite: 1.2 },
+} as unknown as Model<"openai-completions">;
+const PAYLOAD = { messages: [{ role: "user", content: "hello \u{1F680}" }] };
+
+function finalMessage(): AssistantMessage {
+ return {
+ role: "assistant",
+ content: [{ type: "thinking", thinking: "hm" }],
+ api: model.api,
+ provider: model.provider,
+ model: model.id,
+ usage: {
+ input: 800_000,
+ output: 12,
+ cacheRead: 790_000,
+ cacheWrite: 0,
+ totalTokens: 800_012,
+ cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
+ },
+ stopReason: "stop",
+ timestamp: 0,
+ };
+}
+
+function createGate() {
+ let open!: () => void;
+ const promise = new Promise<void>((resolve) => {
+ open = resolve;
+ });
+ return { promise, open };
+}
+
+function scriptedProvider(
+ gates: Record<"response" | "firstToken" | "done", { promise: Promise<void> }>,
+ onResponse: boolean,
+): StreamFn {
+ return async (_model, _context, options) => {
+ const stream = createAssistantMessageEventStream();
+ await options?.onPayload?.(PAYLOAD, model);
+ void (async () => {
+ await gates.response.promise;
+ if (onResponse) await options?.onResponse?.({ status: 200, headers: {} }, model);
+ const partial = { ...finalMessage() };
+ stream.push({ type: "start", partial });
+ await gates.firstToken.promise;
+ stream.push({ type: "thinking_start", contentIndex: 0, partial });
+ await gates.done.promise;
+ stream.push({ type: "done", reason: "stop", message: finalMessage() });
+ })();
+ return stream;
+ };
+}
+
+async function drain(stream: AssistantMessageEventStream): Promise<void> {
+ for await (const _event of stream) {
+ // Consume everything the provider emits.
+ }
+}
+
+async function runTimedRequest(streamFn: StreamFn): Promise<AssistantMessageEventStream> {
+ const enabled = () => true;
+ const transform = instrumentTransformContext(enabled, async (messages) => messages);
+ const convert = instrumentConvertToLlm(enabled, (messages) => messages as unknown as Message[]);
+ const input = [{ role: "user", content: "hello" } as unknown as AgentMessage];
+ const llmMessages = convert(await transform(input));
+ return instrumentStreamFn(enabled, streamFn)(model, {
+ systemPrompt: "",
+ messages: llmMessages,
+ sessionId: "sess-timing",
+ } as never);
+}
+
+describe("request timing", () => {
+ let entries: LogEntry[];
+ const originalEnv = process.env.PI_REQUEST_TIMING;
+ beforeEach(() => {
+ entries = [];
+ setLogSink((entry) => entries.push(entry));
+ delete process.env.PI_REQUEST_TIMING;
+ });
+ afterEach(() => {
+ setLogSink(undefined);
+ process.env.PI_REQUEST_TIMING = originalEnv;
+ });
+ const timingEntries = () =>
+ entries.filter((entry) => entry.component === "coding-agent.request-timing") as Array<Record<string, any>>;
+
+ it.each([
+ ["first-byte from onResponse", true],
+ ["first-byte from the start event when onResponse is omitted", false],
+ ])("%s", async (_name, onResponse: boolean) => {
+ vi.useFakeTimers();
+ try {
+ const gates = { response: createGate(), firstToken: createGate(), done: createGate() };
+ const stream = await runTimedRequest(scriptedProvider(gates, onResponse));
+ // Attach the rejection handler before advancing timers per the race-test convention.
+ const consumed = drain(stream);
+ consumed.catch(() => undefined);
+ await vi.advanceTimersByTimeAsync(25);
+ gates.response.open();
+ await vi.advanceTimersByTimeAsync(0); // flush at t=25
+ await vi.advanceTimersByTimeAsync(10);
+ gates.firstToken.open();
+ await vi.advanceTimersByTimeAsync(0); // flush at t=35
+ await vi.advanceTimersByTimeAsync(5);
+ gates.done.open();
+ await consumed;
+ } finally {
+ vi.useRealTimers();
+ }
+ const timing = timingEntries();
+ expect(timing.map((entry) => entry.phase).join(",")).toBe(
+ "prompt-built,request-sent,first-byte,first-token,stream-done",
+ );
+ expect(new Set(timing.map((entry) => entry.requestSeq)).size).toBe(1);
+ expect(timing.at(-1)).toMatchObject({
+ outcome: "done",
+ contextEntries: 1,
+ requestBytes: Buffer.byteLength(JSON.stringify(PAYLOAD)),
+ usage: { input: 800_000, output: 12, cacheRead: 790_000, cacheWrite: 0 },
+ phases: {
+ dispatchToPromptBuiltMs: 0,
+ promptBuiltToRequestSentMs: 0,
+ requestSentToFirstByteMs: 25,
+ firstByteToFirstTokenMs: 10,
+ firstTokenToStreamDoneMs: 5,
+ },
+ totalMs: 40,
+ });
+ });
+
+ it("measures the payload exactly once when enabled and never when disabled", async () => {
+ const seenOptions: unknown[] = [];
+ const probes: number[] = [];
+ const baseStreamFn: StreamFn = async (_model, _context, options) => {
+ seenOptions.push(options);
+ const probe = {
+ toJSON: () => {
+ probes.push(1);
+ return PAYLOAD;
+ },
+ };
+ await options?.onPayload?.(probe, model);
+ const stream = createAssistantMessageEventStream();
+ stream.push({ type: "done", reason: "stop", message: finalMessage() });
+ return stream;
+ };
+ const plainOptions = { sessionId: "sess-off" };
+ await drain(await instrumentStreamFn(() => false, baseStreamFn)(model, {} as Context, plainOptions));
+ expect(seenOptions[0]).toBe(plainOptions);
+ expect(probes).toHaveLength(0);
+ expect(timingEntries()).toHaveLength(0);
+ // Enabled: serialized once; the size lands on later entries, not request-sent.
+ await drain(await instrumentStreamFn(() => true, baseStreamFn)(model, {} as Context, {}));
+ expect(probes).toHaveLength(1);
+ const enabled = timingEntries();
+ expect(enabled.find((entry) => entry.phase === "request-sent")!.requestBytes).toBeUndefined();
+ expect(enabled.at(-1)!.requestBytes).toBe(Buffer.byteLength(JSON.stringify(PAYLOAD)));
+ });
+
+ it("reports provider failures as failed instead of losing the timeline", async () => {
+ const partial = { ...finalMessage(), content: [] };
+ const crashing = async () =>
+ ({
+ async *[Symbol.asyncIterator]() {
+ yield { type: "start", partial } as never;
+ throw new Error("socket hang up");
+ },
+ }) as never;
+ const crashed = drain(await instrumentStreamFn(() => true, crashing)(model, {} as Context, {}));
+ await expect(crashed).rejects.toThrow("socket hang up");
+ expect(timingEntries().at(-1)).toMatchObject({
+ phase: "stream-done",
+ outcome: "failed",
+ errorMessage: "socket hang up",
+ });
+
+ const errorEvent = async () => {
+ const stream = createAssistantMessageEventStream();
+ queueMicrotask(() =>
+ stream.push({
+ type: "error",
+ reason: "error",
+ error: { ...finalMessage(), stopReason: "error", errorMessage: "provider exploded" },
+ }),
+ );
+ return stream;
+ };
+ await drain(await instrumentStreamFn(() => true, errorEvent)(model, {} as Context, {}));
+ expect(timingEntries().at(-1)).toMatchObject({
+ phase: "stream-done",
+ outcome: "failed",
+ stopReason: "error",
+ errorMessage: "provider exploded",
+ });
+ });
+
+ it("pins the sdk wiring: faux sessions emit the timeline only when the flag is on", async () => {
+ // Isolate the ambient agent dir so the harness digest renders empty state.
+ const dir = join(tmpdir(), `pi-request-timing-${Date.now()}-${Math.random().toString(36).slice(2)}`);
+ mkdirSync(dir, { recursive: true });
+ const ambientAgentDir = process.env.PRIME_AGENT_CODING_AGENT_DIR;
+ process.env.PRIME_AGENT_CODING_AGENT_DIR = dir;
+ const created: AgentSession[] = [];
+ try {
+ // Flag on: faux never calls onPayload, so request-sent is absent and unmeasured phases are omitted.
+ await promptOnFauxSession(dir, true, created);
+ expect(
+ timingEntries()
+ .map((entry) => entry.phase)
+ .join(","),
+ ).toBe("prompt-built,first-byte,first-token,stream-done");
+ expect(timingEntries().at(-1)).toMatchObject({
+ outcome: "done",
+ sessionId: created[0].sessionManager.getSessionId(),
+ });
+ } finally {
+ for (const session of created) session.dispose();
+ rmSync(dir, { recursive: true, force: true });
+ process.env.PRIME_AGENT_CODING_AGENT_DIR = ambientAgentDir;
+ }
+ });
+});
+
+async function promptOnFauxSession(dir: string, requestTiming: boolean, created: AgentSession[]) {
+ const faux = registerFauxProvider();
+ faux.setResponses([fauxAssistantMessage("ok")]);
+ const authStorage = AuthStorage.inMemory();
+ authStorage.setRuntimeApiKey(faux.getModel().provider, "faux-key");
+ const { session } = await createAgentSession({
+ cwd: dir,
+ agentDir: dir,
+ model: faux.getModel(),
+ authStorage,
+ modelRegistry: ModelRegistry.create(authStorage, join(dir, "models.json")),
+ sessionManager: SessionManager.inMemory(dir),
+ settingsManager: SettingsManager.inMemory({ requestTiming }),
+ });
+ created.push(session);
+ await session.prompt("hello");
+ faux.unregister();
+}