A self-improving RLM agent for coding workflows and long-running autonomous tasks.
Rust
feat(coding-agent): per-request provider timing diagnostics (PI_REQUEST_TIMING) (#2462)
Seth Karten690e23d8c8d7
diff --git a/packages/coding-agent/.changes/request-timing.md b/packages/coding-agent/.changes/request-timing.mdnew file mode 100644index 0000000000..635427b04b--- /dev/null+++ b/packages/coding-agent/.changes/request-timing.md@@ -0,0 +1 @@+- New request timing diagnostics: `PI_REQUEST_TIMING=1` (or settings `requestTiming: true`) logs each provider request's phase timeline - prompt-built, request-sent (with body bytes), first-byte, first-token, stream-done - to `~/.prime/agent/logs/agent.jsonl` under `coding-agent.request-timing`, so a long `Waiting` state can be attributed to client-side build, upload, provider prefill, or a prompt-cache miss (summary usage shows cacheRead/cacheWrite). Zero overhead when disabled.diff --git a/packages/coding-agent/docs/development.md b/packages/coding-agent/docs/development.mdindex 22631cb0a3..5e57ecd54b 100644--- a/packages/coding-agent/docs/development.md+++ b/packages/coding-agent/docs/development.md@@ -68,6 +68,33 @@ Do not resolve packaged assets directly from `__dirname`.The hidden `/debug` command writes `~/.prime/agent/prime-agent-debug.log` with rendered TUI lines, their visible widths, and the current agent messages. Daemon, worker, client, and provider diagnostic logs live under `~/.prime/agent/logs/`.+### Request timing++When the UI sits in `Waiting` for a long time before the first model output appears, request timing shows what that wait is made of. Enable it with `PI_REQUEST_TIMING=1` in the environment (restart the daemon so worker processes inherit it) or `"requestTiming": true` in settings.json (applies to sessions opened after the change). Each agent-loop provider request then logs its phase timeline to `~/.prime/agent/logs/agent.jsonl` under the component `coding-agent.request-timing`. One-shot completion calls outside the agent loop (compaction, branch-summary, and refinement requests) use a separate path and are not logged:++```bash+grep '"coding-agent.request-timing"' ~/.prime/agent/logs/agent.jsonl | tail -5+```++Phases per request, in order. Each entry carries the gap it closed (`phaseMs`) and the elapsed time since turn dispatch (`totalMs`); `prompt-built` fires before the request has model or session fields, so correlate it with the later entries by `requestSeq`. If a provider never reports a phase (for example `request-sent` when no payload hook runs), that delta is simply omitted:++| Phase | Meaning | A long gap here means |+|-------|---------|----------------------|+| `prompt-built` | Turn dispatched, prompt message array built | Client-side prompt build is slow |+| `request-sent` | Payload handed to the provider client | Client-side request build (auth, params, extension payload hooks, the `requestBytes` serialization) is slow |+| `first-byte` | HTTP response headers received (`requestBytes`: serialized request body size in UTF-8 bytes) | Upload of the request body plus provider TTFB (prefill, prompt-cache miss, queueing) |+| `first-token` | First streamed content block (what clears `Waiting`) | Stream parse delay; usually near zero |+| `stream-done` | Terminal event, with the summary below | Slow full stream |++The final entry (`msg: "request timing summary"`) carries every delta it measured: `phases.dispatchToPromptBuiltMs`, `promptBuiltToRequestSentMs`, `requestSentToFirstByteMs`, `firstByteToFirstTokenMs`, `firstTokenToStreamDoneMs`, plus `contextEntries`, `requestBytes`, `sessionId`, the final `usage`, and an `outcome` (`done`, `aborted` for an early stop such as a user cancel, or `failed` when the provider failed before or during the stream).++Reading it for a slow large-context turn:++- Large `promptBuiltToRequestSentMs` — the client is slow. Both phases before the request leaves are client-side.+- Large `requestSentToFirstByteMs` with a normal TTFT on the inference dashboard — the time is spent on the wire or inside inference before the dashboard's TTFT timer starts: uploading a multi-MB request body on a slow uplink, provider queueing, or prefilling an uncached prompt. `requestBytes` shows how much had to be uploaded.+- Prompt-cache miss check — the summary's `usage.cacheRead` near 0 with `cacheWrite` close to the full prompt size means the provider re-prefilled the whole context instead of reading its prompt cache. `sessionId` is the cache key the client sends for OpenAI-family providers (session-affinity headers and `prompt_cache_key`); Anthropic instead caches the message prefix, so watch `cacheRead`/`cacheWrite` there.+- Every retry re-issues the turn and gets a fresh request sequence number (`requestSeq`), so a silent retry loop shows up as several request timelines plus the usual auto-retry status.+Useful service commands:```bashdiff --git a/packages/coding-agent/docs/settings.md b/packages/coding-agent/docs/settings.mdindex e3f468c003..620f4c74ee 100644--- a/packages/coding-agent/docs/settings.md+++ b/packages/coding-agent/docs/settings.md@@ -280,6 +280,12 @@ available, authenticated model, the bounded wait runs instead.}```+### Diagnostics++| Setting | Type | Default | Description |+|---------|------|---------|-------------|+| `requestTiming` | boolean | `false` | Log per-request provider timing phases to the diagnostic log (see [Development: Request timing](development.md#request-timing)); `PI_REQUEST_TIMING=1` also enables it |+### Message Delivery| Setting | Type | Default | Description |diff --git a/packages/coding-agent/src/core/request-timing.ts b/packages/coding-agent/src/core/request-timing.tsnew file mode 100644index 0000000000..d0544928d1--- /dev/null+++ b/packages/coding-agent/src/core/request-timing.ts@@ -0,0 +1,459 @@+/**+ * Per-request phase timing for provider requests.+ *+ * Answers "what is the agent waiting for" when the TUI sits in `Waiting`+ * before the first reasoning/text token appears: the wait is split into+ * client-side phases (turn dispatch -> prompt-built -> request-sent) and+ * wire/server phases (request-sent -> first-byte covers request body+ * serialization, upload, and provider TTFB). A long request-sent ->+ * first-byte gap with a normal server-side TTFT points at slow upload or+ * provider prefill/prompt-cache miss rather than client work, and the+ * stream-done summary carries the final usage so a cache miss shows as+ * cacheRead ~ 0 with cacheWrite ~ the full prompt.+ *+ * Enable with `PI_REQUEST_TIMING=1` (env, inherited by daemon workers) or+ * `"requestTiming": true` in settings.json. Entries go to the shared JSONL+ * diagnostic log (~/.prime/agent/logs/agent.jsonl) under the component+ * "coding-agent.request-timing".+ *+ * Zero overhead when disabled: the wrappers pass straight through with no+ * timestamps, no payload serialization, and no log entries.+ */++import type { AgentMessage, StreamFn } from "@earendil-works/pi-agent-core";+import { type AssistantMessageEventStream, getLogger, type Message } from "@earendil-works/pi-ai";++const log = getLogger("coding-agent.request-timing");++const REQUEST_TIMING_ENV = "PI_REQUEST_TIMING";++/** Whether request timing is on. Evaluated per request so the flag can change without a restart. */+type RequestTimingEnabled = () => boolean;++/** Truthy follows the PI_OFFLINE/PI_TIMING convention: 1/true/yes. */+function truthyEnvFlag(value: string | undefined): boolean {+ if (!value) return false;+ const normalized = value.toLowerCase();+ return normalized === "1" || normalized === "true" || normalized === "yes";+}++/**+ * Request timing is on when either the settings flag or the env override is+ * set. Both checks are cheap property reads on the disabled path.+ */+export function isRequestTimingEnabled(settingsFlag: boolean): boolean {+ return settingsFlag || truthyEnvFlag(process.env[REQUEST_TIMING_ENV]);+}++/** Correlation state recorded while the agent loop builds the request. */+interface PromptBuildTiming {+ /** Turn dispatch: streamAssistantResponse entry (first seam of the turn). */+ dispatchedAt: number;+ /** After convertToLlm: the LLM message array is built. */+ promptBuiltAt: number;+ /** LLM message count of the built prompt. */+ contextEntries: number;+}++/** Dispatch timestamps keyed by the transformContext result array. */+const dispatchedAtByContext = new WeakMap<object, number>();+/** Built-prompt timing keyed by the convertToLlm output array (fresh per turn). */+const promptBuildByLlmMessages = new WeakMap<object, PromptBuildTiming>();+/** Per-request sequence numbers keyed by the LLM messages array. */+const requestSeqByLlmMessages = new WeakMap<object, number>();+let moduleRequestSeq = 0;++/**+ * Record the turn dispatch time for the array that continues into prompt+ * build. The agent context snapshot array is reused across turns of one+ * run, so the latest turn overwrites the previous entry; concurrent agents+ * use distinct arrays and cannot collide.+ */+function markRequestTimingDispatch(messages: object[], dispatchedAt: number): void {+ dispatchedAtByContext.set(messages, dispatchedAt);+}++function takeRequestTimingDispatch(messages: object[]): number | undefined {+ return dispatchedAtByContext.get(messages);+}++/**+ * Instruments the agent-loop transformContext seam: the entry timestamp is+ * the request's "send-received" moment (turn dispatch).+ */+export function instrumentTransformContext(+ enabled: RequestTimingEnabled,+ transform: (messages: AgentMessage[], signal?: AbortSignal) => Promise<AgentMessage[]>,+): (messages: AgentMessage[], signal?: AbortSignal) => Promise<AgentMessage[]> {+ return async (messages: AgentMessage[], signal?: AbortSignal) => {+ if (!enabled()) {+ return transform(messages, signal);+ }+ const startedAt = performance.now();+ const result = await transform(messages, signal);+ markRequestTimingDispatch(result, startedAt);+ return result;+ };+}++/**+ * Instruments the agent-loop convertToLlm seam: records the prompt-built+ * phase (with the LLM message count) keyed by the output array the provider+ * request consumes, and emits the prompt-built phase entry.+ */+export function instrumentConvertToLlm(+ enabled: RequestTimingEnabled,+ convert: (messages: AgentMessage[]) => Message[],+): (messages: AgentMessage[]) => Message[] {+ return (messages: AgentMessage[]) => {+ if (!enabled()) {+ return convert(messages);+ }+ const output = convert(messages);+ const startedAt = takeRequestTimingDispatch(messages) ?? performance.now();+ const built = performance.now();+ const timing: PromptBuildTiming = {+ dispatchedAt: startedAt,+ promptBuiltAt: built,+ contextEntries: output.length,+ };+ promptBuildByLlmMessages.set(output, timing);+ log.info("request timing", {+ phase: "prompt-built",+ requestSeq: nextRequestSeq(output),+ phaseMs: roundMs(built - startedAt),+ totalMs: roundMs(built - startedAt),+ contextEntries: output.length,+ });+ return output;+ };+}++function nextRequestSeq(llmMessages: object): number {+ let seq = requestSeqByLlmMessages.get(llmMessages);+ if (seq === undefined) {+ seq = ++moduleRequestSeq;+ requestSeqByLlmMessages.set(llmMessages, seq);+ }+ return seq;+}++/** Provider request identity fields shared by every phase entry. */+interface RequestTimingRequestInfo {+ model: string;+ provider?: string;+ api?: string;+ sessionId?: string;+}++/**+ * Mutable phase clock for one provider request. Created at the streamFn+ * seam; phase transitions are logged as they happen so a hung request shows+ * the last completed phase in the live log.+ */+class RequestTiming {+ readonly requestSeq: number;+ private readonly info: RequestTimingRequestInfo;+ private readonly dispatchedAt: number | undefined;+ private readonly promptBuiltAt: number | undefined;+ private readonly contextEntries: number | undefined;+ private readonly streamFnEnteredAt: number;+ private requestSentAt: number | undefined;+ private requestBytes: number | undefined;+ private firstByteAt: number | undefined;+ private firstTokenAt: number | undefined;+ private streamDoneAt: number | undefined;+ private stopReason: string | undefined;+ private errorMessage: string | undefined;+ private usage: { input: number; output: number; cacheRead: number; cacheWrite: number } | undefined;+ private summaryEmitted = false;++ constructor(llmMessages: object, info: RequestTimingRequestInfo, promptBuild: PromptBuildTiming | undefined) {+ this.requestSeq = nextRequestSeq(llmMessages);+ this.info = info;+ this.dispatchedAt = promptBuild?.dispatchedAt;+ this.promptBuiltAt = promptBuild?.promptBuiltAt;+ this.contextEntries = promptBuild?.contextEntries;+ this.streamFnEnteredAt = performance.now();+ }++ /** request-sent: payload handed to the provider client. */+ markRequestSent(): void {+ if (this.requestSentAt !== undefined) return;+ const now = performance.now();+ this.requestSentAt = now;+ const from = this.promptBuiltAt ?? this.streamFnEnteredAt;+ this.emit("request-sent", {+ phaseMs: roundMs(now - from),+ totalMs: this.totalFromDispatch(now),+ contextEntries: this.contextEntries,+ });+ }++ /** Serialized request body size, measured before request-sent so its cost lands in the client-side phase. */+ recordRequestBytes(requestBytes: number | undefined): void {+ this.requestBytes = requestBytes;+ }++ /** first-byte: HTTP response headers received (provider TTFB complete). */+ markFirstByte(): void {+ if (this.firstByteAt !== undefined) return;+ const now = performance.now();+ this.firstByteAt = now;+ const from = this.requestSentAt ?? this.promptBuiltAt ?? this.streamFnEnteredAt;+ this.emit("first-byte", {+ phaseMs: roundMs(now - from),+ // Without a request-sent timestamp (provider never called onPayload) the+ // delta spans from prompt-built, not just the wire wait.+ ...(this.requestSentAt === undefined ? { phaseFrom: "prompt-built" } : {}),+ totalMs: this.totalFromDispatch(now),+ requestBytes: this.requestBytes,+ });+ }++ /** first-content-token: first streamed content block (thinking/text/toolcall), which clears the TUI Waiting state. */+ markFirstToken(): void {+ if (this.firstTokenAt !== undefined) return;+ const now = performance.now();+ this.firstTokenAt = now;+ const from = this.firstByteAt ?? this.requestSentAt ?? this.streamFnEnteredAt;+ this.emit("first-token", {+ phaseMs: roundMs(now - from),+ totalMs: this.totalFromDispatch(now),+ });+ }++ /** stream-done: terminal event observed on the provider stream. */+ markStreamDone(stopReason: string, errorMessage?: string): void {+ this.stopReason = stopReason;+ this.errorMessage = errorMessage;+ this.streamDoneAt = performance.now();+ }++ /** Final usage from the completed assistant message; cache read/write answers prompt-cache misses. */+ markUsage(usage: { input: number; output: number; cacheRead: number; cacheWrite: number }): void {+ this.usage = usage;+ }++ /** Emit the summary with every known phase delta. Safe to call once; later calls are ignored. */+ emitSummary(outcome: "done" | "aborted" | "failed"): void {+ if (this.summaryEmitted) return;+ this.summaryEmitted = true;+ const doneAt = this.streamDoneAt ?? performance.now();+ const phases: Record<string, number | undefined> = {};+ if (this.dispatchedAt !== undefined && this.promptBuiltAt !== undefined) {+ phases.dispatchToPromptBuiltMs = roundMs(this.promptBuiltAt - this.dispatchedAt);+ }+ const requestStart = this.promptBuiltAt ?? this.streamFnEnteredAt;+ if (this.requestSentAt !== undefined) {+ phases.promptBuiltToRequestSentMs = roundMs(this.requestSentAt - requestStart);+ }+ if (this.requestSentAt !== undefined && this.firstByteAt !== undefined) {+ phases.requestSentToFirstByteMs = roundMs(this.firstByteAt - this.requestSentAt);+ }+ const firstTokenFrom = this.firstByteAt ?? this.requestSentAt;+ if (this.firstTokenAt !== undefined && firstTokenFrom !== undefined) {+ phases.firstByteToFirstTokenMs = roundMs(this.firstTokenAt - firstTokenFrom);+ }+ if (this.firstTokenAt !== undefined) {+ phases.firstTokenToStreamDoneMs = roundMs(doneAt - this.firstTokenAt);+ }+ log.info("request timing summary", {+ phase: "stream-done",+ requestSeq: this.requestSeq,+ outcome,+ ...this.identity(),+ contextEntries: this.contextEntries,+ requestBytes: this.requestBytes,+ phases,+ totalMs: this.totalFromDispatch(doneAt) ?? roundMs(doneAt - this.streamFnEnteredAt),+ stopReason: this.stopReason,+ errorMessage: this.errorMessage,+ usage: this.usage,+ });+ }++ private emit(phase: string, fields: Record<string, unknown>): void {+ log.info("request timing", {+ phase,+ requestSeq: this.requestSeq,+ ...this.identity(),+ ...fields,+ });+ }++ private identity(): Record<string, unknown> {+ return {+ model: this.info.model,+ ...(this.info.provider ? { provider: this.info.provider } : {}),+ ...(this.info.api ? { api: this.info.api } : {}),+ ...(this.info.sessionId ? { sessionId: this.info.sessionId } : {}),+ };+ }++ private totalFromDispatch(at: number): number | undefined {+ return this.dispatchedAt !== undefined ? roundMs(at - this.dispatchedAt) : undefined;+ }+}++/** Event types carrying the first visible model output; these clear the TUI Waiting state. */+const FIRST_TOKEN_EVENT_TYPES = new Set([+ "thinking_start",+ "thinking_delta",+ "text_start",+ "text_delta",+ "toolcall_start",+ "toolcall_delta",+]);++function isRequestTimingFirstTokenEvent(type: string): boolean {+ return FIRST_TOKEN_EVENT_TYPES.has(type);+}++/** Serialized request body size; providers serialize for the wire anyway, but only measure when timing is on. */+function measureRequestBytes(payload: unknown): number | undefined {+ try {+ const serialized = JSON.stringify(payload);+ // UTF-8 bytes as sent on the wire; .length would count UTF-16 code units and underreport non-ASCII prompts.+ return serialized === undefined ? undefined : Buffer.byteLength(serialized, "utf-8");+ } catch {+ return undefined;+ }+}++/**+ * Instruments the streamFn seam: creates the per-request clock, chains the+ * onPayload/onResponse hooks (request-sent / first-byte), wraps the event+ * stream (first-token, stream-done), and emits a failed summary when auth+ * rejects the request before it is sent.+ */+export function instrumentStreamFn(enabled: RequestTimingEnabled, streamFn: StreamFn): StreamFn {+ return async (model, context, options) => {+ if (!enabled()) {+ return streamFn(model, context, options);+ }+ const llmMessages: object[] = context.messages ?? [];+ const timing = new RequestTiming(+ llmMessages,+ {+ model: model.id,+ provider: model.provider,+ api: model.api,+ sessionId: options?.sessionId,+ },+ takePromptBuild(llmMessages),+ );+ try {+ const stream = await streamFn(model, context, {+ ...options,+ onPayload: async (payload, payloadModel) => {+ const next = await options?.onPayload?.(payload, payloadModel);+ // Measured before request-sent: the provider awaits this hook before it+ // opens the HTTP request, so the serialization cost belongs to the+ // client-side build delta, not to request-sent -> first-byte.+ timing.recordRequestBytes(measureRequestBytes(next ?? payload));+ timing.markRequestSent();+ return next;+ },+ onResponse: async (response, responseModel) => {+ timing.markFirstByte();+ await options?.onResponse?.(response, responseModel);+ },+ });+ return wrapRequestTimingEventStream(stream, timing, options?.signal);+ } catch (error) {+ timing.emitSummary("failed");+ throw error;+ }+ };+}++function takePromptBuild(llmMessages: object[]): PromptBuildTiming | undefined {+ const timing = promptBuildByLlmMessages.get(llmMessages);+ if (timing) {+ promptBuildByLlmMessages.delete(llmMessages);+ }+ return timing;+}++/**+ * Wrap a provider event stream so first-byte (fallback via the start event),+ * first-content-token, and the terminal event are timed. The summary is+ * emitted on the terminal event, or as aborted when iteration stops early.+ */+function wrapRequestTimingEventStream(+ stream: AssistantMessageEventStream,+ timing: RequestTiming,+ signal?: AbortSignal,+): AssistantMessageEventStream {+ // Delegates to the provider stream so result/push/end (and instanceof checks)+ // keep working; only iteration is overridden to observe phases.+ const wrapped = Object.create(stream);+ Object.defineProperty(wrapped, Symbol.asyncIterator, {+ value: async function* () {+ try {+ try {+ for await (const event of stream) {+ switch (event?.type) {+ case "start":+ // Providers push start after response headers; used only when onResponse did not fire.+ timing.markFirstByte();+ break;+ case "done":+ timing.markStreamDone(event.message.stopReason, event.message.errorMessage);+ timing.markUsage({+ input: event.message.usage.input,+ output: event.message.usage.output,+ cacheRead: event.message.usage.cacheRead,+ cacheWrite: event.message.usage.cacheWrite,+ });+ timing.emitSummary("done");+ break;+ case "error":+ timing.markStreamDone(event.error.stopReason, event.error.errorMessage);+ timing.markUsage({+ input: event.error.usage.input,+ output: event.error.usage.output,+ cacheRead: event.error.usage.cacheRead,+ cacheWrite: event.error.usage.cacheWrite,+ });+ // A terminal provider error is a failed (or aborted) request, not a completed one.+ timing.emitSummary(event.error.stopReason === "aborted" ? "aborted" : "failed");+ break;+ default:+ if (event && isRequestTimingFirstTokenEvent(event.type)) {+ timing.markFirstToken();+ }+ break;+ }+ yield event;+ }+ } catch (error) {+ // A provider-side crash is a failed request, not a user abort; an+ // abort-driven rejection still reports as aborted. The error is+ // re-thrown so the consumer still sees the failure.+ const errorMessage = error instanceof Error ? error.message : String(error);+ if (signal?.aborted) {+ timing.markStreamDone("aborted", errorMessage);+ timing.emitSummary("aborted");+ } else {+ timing.markStreamDone("error", errorMessage);+ timing.emitSummary("failed");+ }+ throw error;+ }+ } finally {+ // Early termination (abort, hung stream) still reports what was measured.+ timing.emitSummary("aborted");+ }+ },+ enumerable: true,+ writable: true,+ configurable: true,+ });+ return wrapped as unknown as AssistantMessageEventStream;+}++function roundMs(delta: number): number {+ return Math.round(delta * 10) / 10;+}diff --git a/packages/coding-agent/src/core/sdk.ts b/packages/coding-agent/src/core/sdk.tsindex 4f1f7fd7ed..761113fd99 100644--- a/packages/coding-agent/src/core/sdk.ts+++ b/packages/coding-agent/src/core/sdk.ts@@ -13,6 +13,12 @@ import { McpManager } from "./mcp/mcp-manager.js";import { convertToLlm } from "./messages.js";import { ModelRegistry } from "./model-registry.js";import { findInitialModel, findSessionModelWithReadinessWait } from "./model-resolver.js";+import {+ instrumentConvertToLlm,+ instrumentStreamFn,+ instrumentTransformContext,+ isRequestTimingEnabled,+} from "./request-timing.js";import type { ResourceLoader } from "./resource-loader.js";import { DefaultResourceLoader } from "./resource-loader.js";import { getDefaultSessionDir, SessionManager } from "./session-manager.js";@@ -247,6 +253,8 @@ export async function createAgentSession(options: CreateAgentSessionOptions = {}let agent: Agent;+ const requestTimingEnabled = (): boolean => isRequestTimingEnabled(settingsManager.getRequestTiming());+const convertToLlmWithBlockImages = (messages: AgentMessage[]): Message[] => {const converted = convertToLlm(messages);if (!settingsManager.getBlockImages()) {@@ -290,8 +298,8 @@ export async function createAgentSession(options: CreateAgentSessionOptions = {}serviceTier,tools: [],},- convertToLlm: convertToLlmWithBlockImages,- streamFn: async (model, context, options) => {+ convertToLlm: instrumentConvertToLlm(requestTimingEnabled, convertToLlmWithBlockImages),+ streamFn: instrumentStreamFn(requestTimingEnabled, async (model, context, options) => {const auth = await modelRegistry.getApiKeyAndHeaders(model, options?.headers);if (!auth.ok) {throw new Error(auth.error);@@ -304,7 +312,7 @@ export async function createAgentSession(options: CreateAgentSessionOptions = {}timeoutMs: options?.timeoutMs ?? providerRetrySettings.timeoutMs,headers: auth.headers,});- },+ }),onPayload: async (payload, _model) => {const runner = extensionRunnerRef.current;if (!runner?.hasHandlers("before_provider_request")) {@@ -324,11 +332,11 @@ export async function createAgentSession(options: CreateAgentSessionOptions = {}});},sessionId: sessionManager.getSessionId(),- transformContext: async (messages) => {+ transformContext: instrumentTransformContext(requestTimingEnabled, async (messages) => {const runner = extensionRunnerRef.current;if (!runner) return messages;return runner.emitContext(messages);- },+ }),steeringMode: settingsManager.getSteeringMode(),followUpMode: settingsManager.getFollowUpMode(),transport: settingsManager.getTransport(),diff --git a/packages/coding-agent/src/core/settings-manager.ts b/packages/coding-agent/src/core/settings-manager.tsindex ce045cea66..e8ef970dae 100644--- a/packages/coding-agent/src/core/settings-manager.ts+++ b/packages/coding-agent/src/core/settings-manager.ts@@ -257,6 +257,8 @@ export interface Settings {markdown?: MarkdownSettings;warnings?: WarningSettings;sessionDir?: string; // Custom session storage directory (same format as --session-dir CLI flag)+ /** Log per-request provider timing phases to the diagnostic log. Default: false */+ requestTiming?: boolean;}export interface AgentTracesSettings {@@ -1368,6 +1370,10 @@ export class SettingsManager {return this.settings.images?.blockImages ?? false;}+ getRequestTiming(): boolean {+ return this.settings.requestTiming ?? false;+ }+setBlockImages(blocked: boolean): void {if (!this.globalSettings.images) {this.globalSettings.images = {};diff --git a/packages/coding-agent/test/request-timing.test.ts b/packages/coding-agent/test/request-timing.test.tsnew file mode 100644index 0000000000..56c0bbc7e1--- /dev/null+++ b/packages/coding-agent/test/request-timing.test.ts@@ -0,0 +1,276 @@+import { mkdirSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";+import type { AgentMessage, StreamFn } from "@earendil-works/pi-agent-core";+import {+ type AssistantMessage,+ type AssistantMessageEventStream,+ type Context,+ createAssistantMessageEventStream,+ fauxAssistantMessage,+ type LogEntry,+ type Message,+ type Model,+ registerFauxProvider,+ setLogSink,+} from "@earendil-works/pi-ai";+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";+import type { AgentSession } from "../src/core/agent-session.js";+import { AuthStorage } from "../src/core/auth-storage.js";+import { ModelRegistry } from "../src/core/model-registry.js";+import { instrumentConvertToLlm, instrumentStreamFn, instrumentTransformContext } from "../src/core/request-timing.js";+import { createAgentSession } from "../src/core/sdk.js";+import { SessionManager } from "../src/core/session-manager.js";+import { SettingsManager } from "../src/core/settings-manager.js";++const model = {+ id: "bench/bench-model",+ api: "openai-completions",+ provider: "bench",+ baseUrl: "https://bench.test/v1",+ reasoning: true,+ input: ["text"],+ contextWindow: 1_000_000,+ maxTokens: 128_000,+ cost: { input: 1, output: 2, cacheRead: 0.1, cacheWrite: 1.2 },+} as unknown as Model<"openai-completions">;+const PAYLOAD = { messages: [{ role: "user", content: "hello \u{1F680}" }] };++function finalMessage(): AssistantMessage {+ return {+ role: "assistant",+ content: [{ type: "thinking", thinking: "hm" }],+ api: model.api,+ provider: model.provider,+ model: model.id,+ usage: {+ input: 800_000,+ output: 12,+ cacheRead: 790_000,+ cacheWrite: 0,+ totalTokens: 800_012,+ cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },+ },+ stopReason: "stop",+ timestamp: 0,+ };+}++function createGate() {+ let open!: () => void;+ const promise = new Promise<void>((resolve) => {+ open = resolve;+ });+ return { promise, open };+}++function scriptedProvider(+ gates: Record<"response" | "firstToken" | "done", { promise: Promise<void> }>,+ onResponse: boolean,+): StreamFn {+ return async (_model, _context, options) => {+ const stream = createAssistantMessageEventStream();+ await options?.onPayload?.(PAYLOAD, model);+ void (async () => {+ await gates.response.promise;+ if (onResponse) await options?.onResponse?.({ status: 200, headers: {} }, model);+ const partial = { ...finalMessage() };+ stream.push({ type: "start", partial });+ await gates.firstToken.promise;+ stream.push({ type: "thinking_start", contentIndex: 0, partial });+ await gates.done.promise;+ stream.push({ type: "done", reason: "stop", message: finalMessage() });+ })();+ return stream;+ };+}++async function drain(stream: AssistantMessageEventStream): Promise<void> {+ for await (const _event of stream) {+ // Consume everything the provider emits.+ }+}++async function runTimedRequest(streamFn: StreamFn): Promise<AssistantMessageEventStream> {+ const enabled = () => true;+ const transform = instrumentTransformContext(enabled, async (messages) => messages);+ const convert = instrumentConvertToLlm(enabled, (messages) => messages as unknown as Message[]);+ const input = [{ role: "user", content: "hello" } as unknown as AgentMessage];+ const llmMessages = convert(await transform(input));+ return instrumentStreamFn(enabled, streamFn)(model, {+ systemPrompt: "",+ messages: llmMessages,+ sessionId: "sess-timing",+ } as never);+}++describe("request timing", () => {+ let entries: LogEntry[];+ const originalEnv = process.env.PI_REQUEST_TIMING;+ beforeEach(() => {+ entries = [];+ setLogSink((entry) => entries.push(entry));+ delete process.env.PI_REQUEST_TIMING;+ });+ afterEach(() => {+ setLogSink(undefined);+ process.env.PI_REQUEST_TIMING = originalEnv;+ });+ const timingEntries = () =>+ entries.filter((entry) => entry.component === "coding-agent.request-timing") as Array<Record<string, any>>;++ it.each([+ ["first-byte from onResponse", true],+ ["first-byte from the start event when onResponse is omitted", false],+ ])("%s", async (_name, onResponse: boolean) => {+ vi.useFakeTimers();+ try {+ const gates = { response: createGate(), firstToken: createGate(), done: createGate() };+ const stream = await runTimedRequest(scriptedProvider(gates, onResponse));+ // Attach the rejection handler before advancing timers per the race-test convention.+ const consumed = drain(stream);+ consumed.catch(() => undefined);+ await vi.advanceTimersByTimeAsync(25);+ gates.response.open();+ await vi.advanceTimersByTimeAsync(0); // flush at t=25+ await vi.advanceTimersByTimeAsync(10);+ gates.firstToken.open();+ await vi.advanceTimersByTimeAsync(0); // flush at t=35+ await vi.advanceTimersByTimeAsync(5);+ gates.done.open();+ await consumed;+ } finally {+ vi.useRealTimers();+ }+ const timing = timingEntries();+ expect(timing.map((entry) => entry.phase).join(",")).toBe(+ "prompt-built,request-sent,first-byte,first-token,stream-done",+ );+ expect(new Set(timing.map((entry) => entry.requestSeq)).size).toBe(1);+ expect(timing.at(-1)).toMatchObject({+ outcome: "done",+ contextEntries: 1,+ requestBytes: Buffer.byteLength(JSON.stringify(PAYLOAD)),+ usage: { input: 800_000, output: 12, cacheRead: 790_000, cacheWrite: 0 },+ phases: {+ dispatchToPromptBuiltMs: 0,+ promptBuiltToRequestSentMs: 0,+ requestSentToFirstByteMs: 25,+ firstByteToFirstTokenMs: 10,+ firstTokenToStreamDoneMs: 5,+ },+ totalMs: 40,+ });+ });++ it("measures the payload exactly once when enabled and never when disabled", async () => {+ const seenOptions: unknown[] = [];+ const probes: number[] = [];+ const baseStreamFn: StreamFn = async (_model, _context, options) => {+ seenOptions.push(options);+ const probe = {+ toJSON: () => {+ probes.push(1);+ return PAYLOAD;+ },+ };+ await options?.onPayload?.(probe, model);+ const stream = createAssistantMessageEventStream();+ stream.push({ type: "done", reason: "stop", message: finalMessage() });+ return stream;+ };+ const plainOptions = { sessionId: "sess-off" };+ await drain(await instrumentStreamFn(() => false, baseStreamFn)(model, {} as Context, plainOptions));+ expect(seenOptions[0]).toBe(plainOptions);+ expect(probes).toHaveLength(0);+ expect(timingEntries()).toHaveLength(0);+ // Enabled: serialized once; the size lands on later entries, not request-sent.+ await drain(await instrumentStreamFn(() => true, baseStreamFn)(model, {} as Context, {}));+ expect(probes).toHaveLength(1);+ const enabled = timingEntries();+ expect(enabled.find((entry) => entry.phase === "request-sent")!.requestBytes).toBeUndefined();+ expect(enabled.at(-1)!.requestBytes).toBe(Buffer.byteLength(JSON.stringify(PAYLOAD)));+ });++ it("reports provider failures as failed instead of losing the timeline", async () => {+ const partial = { ...finalMessage(), content: [] };+ const crashing = async () =>+ ({+ async *[Symbol.asyncIterator]() {+ yield { type: "start", partial } as never;+ throw new Error("socket hang up");+ },+ }) as never;+ const crashed = drain(await instrumentStreamFn(() => true, crashing)(model, {} as Context, {}));+ await expect(crashed).rejects.toThrow("socket hang up");+ expect(timingEntries().at(-1)).toMatchObject({+ phase: "stream-done",+ outcome: "failed",+ errorMessage: "socket hang up",+ });++ const errorEvent = async () => {+ const stream = createAssistantMessageEventStream();+ queueMicrotask(() =>+ stream.push({+ type: "error",+ reason: "error",+ error: { ...finalMessage(), stopReason: "error", errorMessage: "provider exploded" },+ }),+ );+ return stream;+ };+ await drain(await instrumentStreamFn(() => true, errorEvent)(model, {} as Context, {}));+ expect(timingEntries().at(-1)).toMatchObject({+ phase: "stream-done",+ outcome: "failed",+ stopReason: "error",+ errorMessage: "provider exploded",+ });+ });++ it("pins the sdk wiring: faux sessions emit the timeline only when the flag is on", async () => {+ // Isolate the ambient agent dir so the harness digest renders empty state.+ const dir = join(tmpdir(), `pi-request-timing-${Date.now()}-${Math.random().toString(36).slice(2)}`);+ mkdirSync(dir, { recursive: true });+ const ambientAgentDir = process.env.PRIME_AGENT_CODING_AGENT_DIR;+ process.env.PRIME_AGENT_CODING_AGENT_DIR = dir;+ const created: AgentSession[] = [];+ try {+ // Flag on: faux never calls onPayload, so request-sent is absent and unmeasured phases are omitted.+ await promptOnFauxSession(dir, true, created);+ expect(+ timingEntries()+ .map((entry) => entry.phase)+ .join(","),+ ).toBe("prompt-built,first-byte,first-token,stream-done");+ expect(timingEntries().at(-1)).toMatchObject({+ outcome: "done",+ sessionId: created[0].sessionManager.getSessionId(),+ });+ } finally {+ for (const session of created) session.dispose();+ rmSync(dir, { recursive: true, force: true });+ process.env.PRIME_AGENT_CODING_AGENT_DIR = ambientAgentDir;+ }+ });+});++async function promptOnFauxSession(dir: string, requestTiming: boolean, created: AgentSession[]) {+ const faux = registerFauxProvider();+ faux.setResponses([fauxAssistantMessage("ok")]);+ const authStorage = AuthStorage.inMemory();+ authStorage.setRuntimeApiKey(faux.getModel().provider, "faux-key");+ const { session } = await createAgentSession({+ cwd: dir,+ agentDir: dir,+ model: faux.getModel(),+ authStorage,+ modelRegistry: ModelRegistry.create(authStorage, join(dir, "models.json")),+ sessionManager: SessionManager.inMemory(dir),+ settingsManager: SettingsManager.inMemory({ requestTiming }),+ });+ created.push(session);+ await session.prompt("hello");+ faux.unregister();+}