Skip to content

OpenAI's Codex Security CLI and TypeScript SDK for finding, validating, and fixing security vulnerabilities. npm: https://www.npmjs.com/package/@openai/codex-security

fix(deps): upgrade Codex and preserve oversized MCP recovery (#1879)

dependabot[bot]00b0ac57dc5d
diff --git a/evals/deep-reducer/README.md b/evals/deep-reducer/README.md
index f1e7f6f3c..221d05d5d 100644
--- a/evals/deep-reducer/README.md
+++ b/evals/deep-reducer/README.md
@@ -2,7 +2,7 @@
This optional model eval runs one reducer against synthetic persisted worker findings and a previous aggregate. It uses the pinned Codex CLI, its external code-mode host, the production reducer prompt, and the production artifact tool handlers. It does not launch a repository scan.
-The fixture has a 32 MiB individual semantic field containing quotes, backslashes, newlines, and Unicode. Its old duplicated MCP response is about 114 MiB. The first input read deliberately returns that old response; all subsequent calls use the production paged implementation. The IPC error comes from the real transport, not a mocked error message.
+The fixture has a 32 MiB individual semantic field containing quotes, backslashes, newlines, and Unicode. Its old duplicated MCP response is about 114 MiB. The first input read deliberately returns that old response; all subsequent calls use the production paged implementation. The production stdio transport reports an explicit response-size error before Codex's native transport would discard the response and leave the request waiting.
The grader requires a smaller byte budget on the retry with the same cursor and reference, byte-bounded subsequent pages, one successful result submission, every source accounted for exactly once, complete original payloads matching their SHA-256 hashes, and preservation of the previous finding identity and synthesized history.
@@ -19,7 +19,9 @@ The command prints the path to an ignored directory under `evals/deep-reducer/re
## Deterministic regression test
-The same fixture, real transport, and grading helpers live under `plugins/codex-security/mcp-app/tests/support/reducer-paging/`. The normal MCP test suite exercises them with scripted model responses from a local HTTP provider, a private temporary Codex home, and no model credentials or external model requests. It additionally asserts the exact transport diagnostic and checks that the grader rejects missing input pages and collapsed independent findings.
+The same fixture, real transport, and grading helpers live under `plugins/codex-security/mcp-app/tests/support/reducer-paging/`. The normal MCP test suite exercises both the production stdio response-size error and the native 64 MiB code-mode IPC error. The latter uses a local streamable HTTP MCP server so the oversized response reaches the later IPC boundary. Both cases require the same complete evidence preservation and recovery checks; neither simulates the error.
+
+The tests use scripted model responses from a local HTTP provider, a private temporary Codex home, and no model credentials or external model requests. They additionally assert the actual transport diagnostics and check that the grader rejects missing input pages and collapsed independent findings.
Run that test directly from the repository root:
diff --git a/evals/triage-finding/package.json b/evals/triage-finding/package.json
index 6b058adaa..05701ab15 100644
--- a/evals/triage-finding/package.json
+++ b/evals/triage-finding/package.json
@@ -30,7 +30,7 @@
"pf:view": "PROMPTFOO_CONFIG_DIR=.promptfoo PROMPTFOO_DISABLE_WAL_MODE=true node --experimental-strip-types scripts/run-promptfoo.mts view"
},
"dependencies": {
- "@openai/codex-sdk": "0.163.0",
+ "@openai/codex-sdk": "0.164.0",
"@opencode-ai/sdk": "1.19.2",
"promptfoo": "0.124.1"
},
diff --git a/evals/triage-finding/pnpm-lock.yaml b/evals/triage-finding/pnpm-lock.yaml
index cb3468d3a..9cfd88aaa 100644
--- a/evals/triage-finding/pnpm-lock.yaml
+++ b/evals/triage-finding/pnpm-lock.yaml
@@ -5,8 +5,8 @@ settings:
excludeLinksFromLockfile: false
overrides:
- '@openai/codex': 0.163.0
- '@openai/codex-sdk': 0.163.0
+ '@openai/codex': 0.164.0
+ '@openai/codex-sdk': 0.164.0
'@opencode-ai/sdk': 1.19.2
[email protected]>axios: 1.20.0
[email protected]>basic-ftp: 6.2.1
@@ -17,8 +17,8 @@ importers:
.:
dependencies:
'@openai/codex-sdk':
- specifier: 0.163.0
- version: 0.163.0
+ specifier: 0.164.0
+ version: 0.164.0
'@opencode-ai/sdk':
specifier: 1.19.2
version: 1.19.2
@@ -1081,47 +1081,47 @@ packages:
peerDependencies:
zod: ^4.0.0
- '@openai/[email protected]':
- resolution: {integrity: sha512-o10d8dqxFjhzoWp75MUrhaN8C6QrXPQqUJGwVnySj++ztAnrA322UuFRzHZvSa903lFgjjaimuf5rkqVsj0RHg==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-FyhTWM4Xa8hXpWhkr9MOvROVhBqTeD+s/XCzVLzqSQXTSnGSNkdBWjv0IzZdOO29BX4LwtMjul6ogB8tOME5og==}
engines: {node: '>=18'}
- '@openai/[email protected]':
- resolution: {integrity: sha512-HNd7vR9oTK6M2ikxFnvNDCdYUgAsyo9VVnd5KnEd2ON0GYI++7MiX9XNPUkhtdgGKFPsWS+CEppQ1UU63uVhUw==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-TWj7UGyPRe3Pj/A3ddNAxs1te3a/W/xAd+J0LHrrFTmsSfahGMOLF/vn88D3QCIgAT3B1rm1y/Z3NYE0rx6cZQ==}
engines: {node: '>=16'}
hasBin: true
- '@openai/[email protected]':
- resolution: {integrity: sha512-i7Dzk2a5aGmuGM9aNZgaUVTojZ/4bc5lRfosz++Y7X2BGhDkz+cySgbD+NYS1pOxfm0VEbLi9PZ3pAulI4kchw==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-pDizgOUAtq3/k3AbUFHbaU19WpYKCja+oFpNSTLKrDQj4lTUNSDuTPJip4uQVn2/1EipQdpQrGMOQBVuI1znAA==}
engines: {node: '>=16'}
cpu: [arm64]
os: [darwin]
- '@openai/[email protected]':
- resolution: {integrity: sha512-Ioj7h+B1ME9ZU1PH1ATfvFV5DiototEpFFaSSbg3kfDifB9djhl5fcJ++g5mpcL/+uVbcnReei+RiSXEx8o+kw==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-lIe+udnguPGzBezkgbkj2Eaz0YqOhNXT7jja136Fj0sXfI+UKNS2icmaK14GcoRAgeVEpDTtC1ftamWBPmD1zg==}
engines: {node: '>=16'}
cpu: [x64]
os: [darwin]
- '@openai/[email protected]':
- resolution: {integrity: sha512-IH2/R+4OarjGvsAf1ll3fdyLwrPgYLl3uzp6u9CyD9sj2Lw+ViDSe3QBZVHdcHJs0eIiZFg6Wprcuy4CRZaUGw==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-LL5iTQjyDL4YRX44EWqhIfEQ5uGnmgxkQBJSxjZSeJ8L1Og83pdlzUJW/PRJ76oaVwOVgh6HXwuPEiBNEHU4lg==}
engines: {node: '>=16'}
cpu: [arm64]
os: [linux]
- '@openai/[email protected]':
- resolution: {integrity: sha512-uwRaOsrYogLQhcxAM/GoeWrw0QIWEMRo7c9gM8sKAviVkZQHlu9kRLJ5PkufbV+ABGjyZgIwiFdJ0LCJ0N6e8w==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-XLAoaWsGOT2khzl+P4FYDPTkzAr9W0VFXFaVkqlFdkM0soa1gH/4S0wSD57Ek3HHwkirAEGz2/Mwehseh+l5Mg==}
engines: {node: '>=16'}
cpu: [x64]
os: [linux]
- '@openai/[email protected]':
- resolution: {integrity: sha512-QnLn3gR2dExcdTThE/HAo5BrUq0/C8zDGQ0UZL3hNzfqt9ZklT6jGTyt8JhDyNZX4uSB4HhcJtbcOWucZGC0VQ==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-Zkzt8HNmB/vOnlDo5D7IOlrt5rHaxYQB7iGbNP7bHcXZtrk6gaD7UriwaieGYJpcf8DqAT/0tqzY4xbs8ZwbmQ==}
engines: {node: '>=16'}
cpu: [arm64]
os: [win32]
- '@openai/[email protected]':
- resolution: {integrity: sha512-QQGQ/q9dCJ5C5jqhgFrwycpWG3szIyFNThMiKl6hYtu6N/eM3Qd7StesgnCFAfzVED8IkE3N3oH71szzcludqw==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-urWPhwh9YavuoAPIkmVYKBTtd5OtGVL4Ki0Fb28dhK/Mz6Pt4LSy+gbRrZFz5+RXrw9ubvXuXDCcd8beYcWhDg==}
engines: {node: '>=16'}
cpu: [x64]
os: [win32]
@@ -4764,35 +4764,35 @@ snapshots:
- ws
optional: true
- '@openai/[email protected]':
+ '@openai/[email protected]':
dependencies:
- '@openai/codex': 0.163.0
+ '@openai/codex': 0.164.0
- '@openai/[email protected]':
+ '@openai/[email protected]':
optionalDependencies:
- '@openai/codex-darwin-arm64': '@openai/[email protected]'
- '@openai/codex-darwin-x64': '@openai/[email protected]'
- '@openai/codex-linux-arm64': '@openai/[email protected]'
- '@openai/codex-linux-x64': '@openai/[email protected]'
- '@openai/codex-win32-arm64': '@openai/[email protected]'
- '@openai/codex-win32-x64': '@openai/[email protected]'
+ '@openai/codex-darwin-arm64': '@openai/[email protected]'
+ '@openai/codex-darwin-x64': '@openai/[email protected]'
+ '@openai/codex-linux-arm64': '@openai/[email protected]'
+ '@openai/codex-linux-x64': '@openai/[email protected]'
+ '@openai/codex-win32-arm64': '@openai/[email protected]'
+ '@openai/codex-win32-x64': '@openai/[email protected]'
- '@openai/[email protected]':
+ '@openai/[email protected]':
optional: true
- '@openai/[email protected]':
+ '@openai/[email protected]':
optional: true
- '@openai/[email protected]':
+ '@openai/[email protected]':
optional: true
- '@openai/[email protected]':
+ '@openai/[email protected]':
optional: true
- '@openai/[email protected]':
+ '@openai/[email protected]':
optional: true
- '@openai/[email protected]':
+ '@openai/[email protected]':
optional: true
'@opencode-ai/[email protected]':
@@ -6807,7 +6807,7 @@ snapshots:
'@modelcontextprotocol/sdk': 1.32.1(@cfworker/[email protected])([email protected])
'@modelcontextprotocol/server-filesystem': 2026.8.31(@cfworker/[email protected])([email protected])
'@openai/agents': 0.11.8(@aws-sdk/[email protected])(@cfworker/[email protected])(@smithy/[email protected])(@smithy/[email protected])([email protected])([email protected])
- '@openai/codex-sdk': 0.163.0
+ '@openai/codex-sdk': 0.164.0
'@opencode-ai/sdk': 1.19.2
'@slack/web-api': 8.1.1
'@smithy/node-http-handler': 4.12.1
diff --git a/plugins/codex-security/mcp-app/main.ts b/plugins/codex-security/mcp-app/main.ts
index 87c750b1b..924200389 100644
--- a/plugins/codex-security/mcp-app/main.ts
+++ b/plugins/codex-security/mcp-app/main.ts
@@ -1,7 +1,7 @@
-import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import { createCodexSecurityArtifactWriterServer } from "./artifact-writer-main.js";
import { createCodexSecurityServer } from "./server.js";
import { runCliDeepScan } from "./deep-scan-main.js";
+import { CodexSecurityStdioTransport } from "./src/stdio-transport.js";
async function main(): Promise<void> {
if (process.argv.includes("--deep-scan-engine")) {
@@ -13,7 +13,7 @@ async function main(): Promise<void> {
? await createCodexSecurityArtifactWriterServer()
: createCodexSecurityServer();
await server.connect(
- new StdioServerTransport(process.stdin, process.stdout, {
+ new CodexSecurityStdioTransport(process.stdin, process.stdout, {
// Local scan checkpoints can exceed the SDK's default 10 MiB buffer.
maxBufferSize: Infinity,
}),
diff --git a/plugins/codex-security/mcp-app/package.json b/plugins/codex-security/mcp-app/package.json
index 342d58051..8ce3af9d0 100644
--- a/plugins/codex-security/mcp-app/package.json
+++ b/plugins/codex-security/mcp-app/package.json
@@ -11,7 +11,7 @@
},
"dependencies": {
"@modelcontextprotocol/sdk": "^1.32.1",
- "@openai/codex-sdk": "0.163.0",
+ "@openai/codex-sdk": "0.164.0",
"ajv": "8.20.0",
"smol-toml": "1.9.1",
"zod": "^4.6.7"
diff --git a/plugins/codex-security/mcp-app/pnpm-lock.yaml b/plugins/codex-security/mcp-app/pnpm-lock.yaml
index 0902fbb78..cc3651658 100644
--- a/plugins/codex-security/mcp-app/pnpm-lock.yaml
+++ b/plugins/codex-security/mcp-app/pnpm-lock.yaml
@@ -12,8 +12,8 @@ importers:
specifier: ^1.32.1
version: 1.32.1([email protected])
'@openai/codex-sdk':
- specifier: 0.163.0
- version: 0.163.0
+ specifier: 0.164.0
+ version: 0.164.0
ajv:
specifier: 8.20.0
version: 8.20.0
@@ -211,47 +211,47 @@ packages:
'@cfworker/json-schema':
optional: true
- '@openai/[email protected]':
- resolution: {integrity: sha512-o10d8dqxFjhzoWp75MUrhaN8C6QrXPQqUJGwVnySj++ztAnrA322UuFRzHZvSa903lFgjjaimuf5rkqVsj0RHg==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-FyhTWM4Xa8hXpWhkr9MOvROVhBqTeD+s/XCzVLzqSQXTSnGSNkdBWjv0IzZdOO29BX4LwtMjul6ogB8tOME5og==}
engines: {node: '>=18'}
- '@openai/[email protected]':
- resolution: {integrity: sha512-HNd7vR9oTK6M2ikxFnvNDCdYUgAsyo9VVnd5KnEd2ON0GYI++7MiX9XNPUkhtdgGKFPsWS+CEppQ1UU63uVhUw==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-TWj7UGyPRe3Pj/A3ddNAxs1te3a/W/xAd+J0LHrrFTmsSfahGMOLF/vn88D3QCIgAT3B1rm1y/Z3NYE0rx6cZQ==}
engines: {node: '>=16'}
hasBin: true
- '@openai/[email protected]':
- resolution: {integrity: sha512-i7Dzk2a5aGmuGM9aNZgaUVTojZ/4bc5lRfosz++Y7X2BGhDkz+cySgbD+NYS1pOxfm0VEbLi9PZ3pAulI4kchw==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-pDizgOUAtq3/k3AbUFHbaU19WpYKCja+oFpNSTLKrDQj4lTUNSDuTPJip4uQVn2/1EipQdpQrGMOQBVuI1znAA==}
engines: {node: '>=16'}
cpu: [arm64]
os: [darwin]
- '@openai/[email protected]':
- resolution: {integrity: sha512-Ioj7h+B1ME9ZU1PH1ATfvFV5DiototEpFFaSSbg3kfDifB9djhl5fcJ++g5mpcL/+uVbcnReei+RiSXEx8o+kw==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-lIe+udnguPGzBezkgbkj2Eaz0YqOhNXT7jja136Fj0sXfI+UKNS2icmaK14GcoRAgeVEpDTtC1ftamWBPmD1zg==}
engines: {node: '>=16'}
cpu: [x64]
os: [darwin]
- '@openai/[email protected]':
- resolution: {integrity: sha512-IH2/R+4OarjGvsAf1ll3fdyLwrPgYLl3uzp6u9CyD9sj2Lw+ViDSe3QBZVHdcHJs0eIiZFg6Wprcuy4CRZaUGw==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-LL5iTQjyDL4YRX44EWqhIfEQ5uGnmgxkQBJSxjZSeJ8L1Og83pdlzUJW/PRJ76oaVwOVgh6HXwuPEiBNEHU4lg==}
engines: {node: '>=16'}
cpu: [arm64]
os: [linux]
- '@openai/[email protected]':
- resolution: {integrity: sha512-uwRaOsrYogLQhcxAM/GoeWrw0QIWEMRo7c9gM8sKAviVkZQHlu9kRLJ5PkufbV+ABGjyZgIwiFdJ0LCJ0N6e8w==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-XLAoaWsGOT2khzl+P4FYDPTkzAr9W0VFXFaVkqlFdkM0soa1gH/4S0wSD57Ek3HHwkirAEGz2/Mwehseh+l5Mg==}
engines: {node: '>=16'}
cpu: [x64]
os: [linux]
- '@openai/[email protected]':
- resolution: {integrity: sha512-QnLn3gR2dExcdTThE/HAo5BrUq0/C8zDGQ0UZL3hNzfqt9ZklT6jGTyt8JhDyNZX4uSB4HhcJtbcOWucZGC0VQ==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-Zkzt8HNmB/vOnlDo5D7IOlrt5rHaxYQB7iGbNP7bHcXZtrk6gaD7UriwaieGYJpcf8DqAT/0tqzY4xbs8ZwbmQ==}
engines: {node: '>=16'}
cpu: [arm64]
os: [win32]
- '@openai/[email protected]':
- resolution: {integrity: sha512-QQGQ/q9dCJ5C5jqhgFrwycpWG3szIyFNThMiKl6hYtu6N/eM3Qd7StesgnCFAfzVED8IkE3N3oH71szzcludqw==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-urWPhwh9YavuoAPIkmVYKBTtd5OtGVL4Ki0Fb28dhK/Mz6Pt4LSy+gbRrZFz5+RXrw9ubvXuXDCcd8beYcWhDg==}
engines: {node: '>=16'}
cpu: [x64]
os: [win32]
@@ -861,35 +861,35 @@ snapshots:
transitivePeerDependencies:
- supports-color
- '@openai/[email protected]':
+ '@openai/[email protected]':
dependencies:
- '@openai/codex': 0.163.0
+ '@openai/codex': 0.164.0
- '@openai/[email protected]':
+ '@openai/[email protected]':
optionalDependencies:
- '@openai/codex-darwin-arm64': '@openai/[email protected]'
- '@openai/codex-darwin-x64': '@openai/[email protected]'
- '@openai/codex-linux-arm64': '@openai/[email protected]'
- '@openai/codex-linux-x64': '@openai/[email protected]'
- '@openai/codex-win32-arm64': '@openai/[email protected]'
- '@openai/codex-win32-x64': '@openai/[email protected]'
-
- '@openai/[email protected]':
+ '@openai/codex-darwin-arm64': '@openai/[email protected]'
+ '@openai/codex-darwin-x64': '@openai/[email protected]'
+ '@openai/codex-linux-arm64': '@openai/[email protected]'
+ '@openai/codex-linux-x64': '@openai/[email protected]'
+ '@openai/codex-win32-arm64': '@openai/[email protected]'
+ '@openai/codex-win32-x64': '@openai/[email protected]'
+
+ '@openai/[email protected]':
optional: true
- '@openai/[email protected]':
+ '@openai/[email protected]':
optional: true
- '@openai/[email protected]':
+ '@openai/[email protected]':
optional: true
- '@openai/[email protected]':
+ '@openai/[email protected]':
optional: true
- '@openai/[email protected]':
+ '@openai/[email protected]':
optional: true
- '@openai/[email protected]':
+ '@openai/[email protected]':
optional: true
diff --git a/plugins/codex-security/mcp-app/src/stdio-transport.ts b/plugins/codex-security/mcp-app/src/stdio-transport.ts
new file mode 100644
index 000000000..cece2e585
--- /dev/null
+++ b/plugins/codex-security/mcp-app/src/stdio-transport.ts
@@ -0,0 +1,36 @@
+import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
+import { serializeMessage } from "@modelcontextprotocol/sdk/shared/stdio.js";
+import {
+ ErrorCode,
+ type JSONRPCMessage,
+} from "@modelcontextprotocol/sdk/types.js";
+
+// The native client's rmcp transport drops lines above 16 MiB, including the LF,
+// without completing the request. Return a correlated error before that happens.
+const NATIVE_STDIO_LINE_LIMIT_BYTES = 16 * 1024 * 1024;
+
+export class CodexSecurityStdioTransport extends StdioServerTransport {
+ override async send(message: JSONRPCMessage): Promise<void> {
+ if (
+ !("result" in message || "error" in message) ||
+ message.id === undefined
+ ) {
+ return super.send(message);
+ }
+
+ const bytes = Buffer.byteLength(serializeMessage(message), "utf8");
+ if (bytes <= NATIVE_STDIO_LINE_LIMIT_BYTES) return super.send(message);
+
+ return super.send({
+ jsonrpc: "2.0",
+ id: message.id,
+ error: {
+ code: ErrorCode.InternalError,
+ message:
+ `MCP response size ${bytes} bytes exceeds the native stdio line limit of ${NATIVE_STDIO_LINE_LIMIT_BYTES} bytes. ` +
+ "For paged reads, reduce the requested page byte budget and retry the same page. " +
+ "The operation may already have completed; check its result before repeating a write.",
+ },
+ });
+ }
+}
diff --git a/plugins/codex-security/mcp-app/tests/support/reducer-paging/controlled-code-mode.ts b/plugins/codex-security/mcp-app/tests/support/reducer-paging/controlled-code-mode.ts
index 58ceb79a8..28140470a 100644
--- a/plugins/codex-security/mcp-app/tests/support/reducer-paging/controlled-code-mode.ts
+++ b/plugins/codex-security/mcp-app/tests/support/reducer-paging/controlled-code-mode.ts
@@ -44,7 +44,7 @@ export async function runControlledCodeMode({
prompt,
}: {
code: string;
- mcpServer: { command: string; args: string[] };
+ mcpServer: { command: string; args: string[] } | { url: string };
workingDirectory: string;
prompt: string;
}) {
@@ -129,8 +129,12 @@ export async function runControlledCodeMode({
"features.code_mode.enabled=true",
"features.code_mode_host.enabled=true",
"features.code_mode_host.disable_in_process_fallback=true",
- `mcp_servers.cs_artifacts.command=${JSON.stringify(mcpServer.command)}`,
- `mcp_servers.cs_artifacts.args=${JSON.stringify(mcpServer.args)}`,
+ ...("url" in mcpServer
+ ? [`mcp_servers.cs_artifacts.url=${JSON.stringify(mcpServer.url)}`]
+ : [
+ `mcp_servers.cs_artifacts.command=${JSON.stringify(mcpServer.command)}`,
+ `mcp_servers.cs_artifacts.args=${JSON.stringify(mcpServer.args)}`,
+ ]),
];
// Pass only process-launch essentials; never inherit the caller's model auth.
const env: NodeJS.ProcessEnv = {
diff --git a/plugins/codex-security/mcp-app/tests/support/reducer-paging/deep-reducer-paging-server.ts b/plugins/codex-security/mcp-app/tests/support/reducer-paging/deep-reducer-paging-server.ts
index 54a584d36..d13d33186 100644
--- a/plugins/codex-security/mcp-app/tests/support/reducer-paging/deep-reducer-paging-server.ts
+++ b/plugins/codex-security/mcp-app/tests/support/reducer-paging/deep-reducer-paging-server.ts
@@ -2,13 +2,21 @@ import { readJson } from "../json.ts";
import type { TextContent } from "@modelcontextprotocol/sdk/types.js";
import type { ZodRawShapeCompat } from "@modelcontextprotocol/sdk/server/zod-compat.js";
import { appendFile } from "node:fs/promises";
+import { randomUUID } from "node:crypto";
+import { createServer } from "node:http";
+import type { AddressInfo } from "node:net";
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
-import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
+import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js";
+import { CodexSecurityStdioTransport } from "../../../src/stdio-transport.js";
import { getCodexSecurityDeepReducerInputs } from "../../../src/artifact-deep-reducer.js";
import { registerCompactWorkerArtifactTools } from "../../../src/server/compact-artifact-tools.js";
async function main() {
- const { context, tracePath } = await readJson(process.argv[2]);
+ const {
+ context,
+ tracePath,
+ transport: protocol,
+ } = await readJson(process.argv[2]);
const server = new McpServer({
name: "deep-reducer-paging-eval",
version: "1",
@@ -21,7 +29,7 @@ async function main() {
let sequence = 0;
// Exercise the production tool registration and handlers. Only the first read
- // returns the old duplicated result, forcing a real code-mode transport error.
+ // returns the old duplicated result, exercising response-size error recovery.
registerCompactWorkerArtifactTools(
{
registerTool(...[name, definition, handler]: Parameters<RegisterTool>) {
@@ -82,7 +90,31 @@ async function main() {
context,
);
- await server.connect(new StdioServerTransport());
+ if (protocol === "http") {
+ // Legacy HTTP lets the oversized result reach the real code-mode IPC limit.
+ // Production stdio separately exercises its earlier response-size error.
+ const transport = new StreamableHTTPServerTransport({
+ sessionIdGenerator: randomUUID,
+ enableJsonResponse: true,
+ });
+ await server.connect(transport);
+ const http = createServer((request, response) => {
+ void transport.handleRequest(request, response);
+ });
+ await new Promise<void>((resolve, reject) => {
+ http.once("error", reject);
+ http.listen(0, "127.0.0.1", resolve);
+ });
+ process.send!({
+ url: `http://127.0.0.1:${(http.address() as AddressInfo).port}/mcp`,
+ });
+ } else {
+ await server.connect(
+ new CodexSecurityStdioTransport(process.stdin, process.stdout, {
+ maxBufferSize: Infinity,
+ }),
+ );
+ }
async function trace(event: Record<string, unknown>) {
await appendFile(tracePath, JSON.stringify(event) + "\n");
diff --git a/plugins/codex-security/mcp-app/tests/support/reducer-paging/deep-reducer-paging.ts b/plugins/codex-security/mcp-app/tests/support/reducer-paging/deep-reducer-paging.ts
index 879431970..82a4fad25 100644
--- a/plugins/codex-security/mcp-app/tests/support/reducer-paging/deep-reducer-paging.ts
+++ b/plugins/codex-security/mcp-app/tests/support/reducer-paging/deep-reducer-paging.ts
@@ -10,6 +10,8 @@ export interface ReducerTraceEvent {
nextCursor?: string;
}
import assert from "node:assert/strict";
+import { spawn } from "node:child_process";
+import { once } from "node:events";
import { readFile, writeFile } from "node:fs/promises";
import path from "node:path";
import { Codex } from "@openai/codex-sdk";
@@ -25,6 +27,7 @@ interface ReducerPagingEvalOptions {
root: string;
mode?: "deterministic" | "model";
model?: string;
+ transport?: "stdio" | "http";
}
/** Run one reducer through real code-mode IPC and the production artifact tools. */
@@ -47,6 +50,7 @@ async function runReducer({
root,
mode = "deterministic",
model,
+ transport: protocol = "stdio",
}: ReducerPagingEvalOptions) {
assert.ok(mode === "deterministic" || mode === "model");
const fixture = await createReducerPagingFixture(root);
@@ -54,7 +58,11 @@ async function runReducer({
const serverConfigPath = path.join(root, "server-config.json");
const serverPath = path.join(root, "artifact-server.cjs");
await writeFile(tracePath, "");
- await writeJson(serverConfigPath, { context: fixture.context, tracePath });
+ await writeJson(serverConfigPath, {
+ context: fixture.context,
+ tracePath,
+ transport: protocol,
+ });
const [, { renderDedupPrompt }] = await Promise.all([
build({
entryPoints: [
@@ -79,68 +87,108 @@ async function runReducer({
reducerLabel: "paging-eval",
claimedWorkerIds,
});
- const mcpServer = {
+ let mcpServer:
+ | { command: string; args: string[]; required: true }
+ | { url: string; required: true } = {
command: process.execPath,
args: [serverPath, serverConfigPath],
required: true,
};
- let transport;
- let usage;
- if (mode === "deterministic") {
- transport = await runControlledCodeMode({
- code: controlledReducerCode,
- mcpServer,
- workingDirectory: fixture.context.repoRoot,
- prompt,
- });
- assert.equal(transport.exitCode, 0, transport.stderr);
- assert.deepEqual(transport.serverErrors, []);
- const output = JSON.stringify(transport.toolOutputs);
- assert.match(
- output,
- /code-mode delegate response exceeds the IPC frame limit: code-mode IPC frame length [0-9]+ exceeds 67108864 bytes/,
- "the real transport must reject the legacy payload",
- );
- assert.match(output, /reductionRecorded/);
- } else {
- const codex = new Codex({
- config: {
- mcp_servers: { cs_artifacts: mcpServer },
- features: {
- code_mode: { enabled: true },
- code_mode_host: { enabled: true, disable_in_process_fallback: true },
+ const httpChild =
+ protocol === "http"
+ ? spawn(process.execPath, [serverPath, serverConfigPath], {
+ stdio: ["ignore", "ignore", "inherit", "ipc"],
+ })
+ : undefined;
+ const httpClosed = httpChild ? once(httpChild, "close") : undefined;
+ try {
+ if (httpChild) {
+ const [message] = await Promise.race([
+ once(httpChild, "message"),
+ httpClosed!.then(([code, signal]) => {
+ throw new Error(
+ `HTTP MCP fixture exited before ready (${signal ?? code}).`,
+ );
+ }),
+ ]);
+ assert.equal(typeof message.url, "string");
+ mcpServer = { url: message.url, required: true };
+ }
+ let transport;
+ let usage;
+ if (mode === "deterministic") {
+ transport = await runControlledCodeMode({
+ code: controlledReducerCode,
+ mcpServer,
+ workingDirectory: fixture.context.repoRoot,
+ prompt,
+ });
+ assert.equal(transport.exitCode, 0, transport.stderr);
+ assert.deepEqual(transport.serverErrors, []);
+ const output = JSON.stringify(transport.toolOutputs);
+ assert.match(
+ output,
+ protocol === "http"
+ ? /code-mode delegate response exceeds the IPC frame limit: code-mode IPC frame length [0-9]+ exceeds 67108864 bytes/
+ : /MCP response.*exceeds.*16777216/,
+ "the selected transport must report its actual response-size error",
+ );
+ assert.match(output, /reductionRecorded/);
+ } else {
+ const codex = new Codex({
+ config: {
+ mcp_servers: { cs_artifacts: mcpServer },
+ features: {
+ code_mode: { enabled: true },
+ code_mode_host: {
+ enabled: true,
+ disable_in_process_fallback: true,
+ },
+ },
},
- },
- });
- const thread = codex.startThread({
+ });
+ const thread = codex.startThread({
+ ...(model ? { model } : {}),
+ workingDirectory: fixture.context.repoRoot,
+ skipGitRepoCheck: true,
+ approvalPolicy: "never",
+ sandboxMode: "read-only",
+ });
+ const result = await thread.run(prompt);
+ usage = result.usage;
+ await writeFile(
+ path.join(root, "model-response.txt"),
+ result.finalResponse,
+ );
+ }
+ const trace = (await readFile(tracePath, "utf8"))
+ .trim()
+ .split("\n")
+ .filter(Boolean)
+ .map((line) => JSON.parse(line));
+ const report = {
+ mode,
+ transport: protocol,
...(model ? { model } : {}),
- workingDirectory: fixture.context.repoRoot,
- skipGitRepoCheck: true,
- approvalPolicy: "never",
- sandboxMode: "read-only",
- });
- const result = await thread.run(prompt);
- usage = result.usage;
- await writeFile(
- path.join(root, "model-response.txt"),
- result.finalResponse,
- );
+ ...fixture.measurements,
+ ...gradeReducerPagingTrace(
+ trace,
+ fixture.measurements.ipcFrameLimitBytes,
+ ),
+ ...(transport
+ ? {
+ realIpcErrorObserved: protocol === "http",
+ stdioResponseErrorObserved: protocol === "stdio",
+ }
+ : {}),
+ ...(await gradeReducerPagingResult(fixture)),
+ ...(usage ? { usage } : {}),
+ };
+ return { report, fixture };
+ } finally {
+ httpChild?.kill();
+ if (httpClosed) await httpClosed;
}
- const trace = (await readFile(tracePath, "utf8"))
- .trim()
- .split("\n")
- .filter(Boolean)
- .map((line) => JSON.parse(line));
- const report = {
- mode,
- ...(model ? { model } : {}),
- ...fixture.measurements,
- ...gradeReducerPagingTrace(trace, fixture.measurements.ipcFrameLimitBytes),
- ...(transport ? { realIpcErrorObserved: true } : {}),
- ...(await gradeReducerPagingResult(fixture)),
- ...(usage ? { usage } : {}),
- };
- return { report, fixture };
}
/** Grade the captured I/O independently of model wording or another model run. */
@@ -241,6 +289,7 @@ async function readDocument(findingRef) {
let response;
try {
response = await tools.mcp__cs_artifacts__get_codex_security_deep_reducer_inputs(input);
+ if (response.isError) throw new Error(response.content[0].text);
} catch (error) {
text(String(error));
maxBytes = Math.floor(maxBytes / 2);
diff --git a/plugins/codex-security/mcp-app/tests/test_deep_reducer_paging_eval.ts b/plugins/codex-security/mcp-app/tests/test_deep_reducer_paging_eval.ts
index caaac04a3..40f002119 100644
--- a/plugins/codex-security/mcp-app/tests/test_deep_reducer_paging_eval.ts
+++ b/plugins/codex-security/mcp-app/tests/test_deep_reducer_paging_eval.ts
@@ -10,63 +10,67 @@ import {
} from "./support/reducer-paging/deep-reducer-paging.ts";
import { gradeReducerPagingResult } from "./support/reducer-paging/deep-reducer-paging-fixture.ts";
-test("a reducer recovers from the real IPC frame limit and records all sources", async (t) => {
- const root = await mkdtemp(path.join(tmpdir(), "deep-reducer-ipc-eval-"));
- t.after(() => rm(root, { recursive: true, force: true }));
- const { report, fixture } = await runReducerPagingEval({ root });
- assert.equal(report.realIpcErrorObserved, true);
- assert.ok(report.actualOversizedResponseBytes! > report.ipcFrameLimitBytes);
- assert.ok(report.recoveryBudget < report.firstBudget);
- assert.ok(report.successfulPages > 1);
- assert.equal(report.referenceReads, 2);
- assert.equal(report.accountedSourceCount, 3);
- assert.equal(report.preservedOriginalCount, 3);
- assert.equal(report.previousIdentityPreserved, true);
- assert.equal(report.synthesizedHistoryPreserved, true);
+for (const transport of ["stdio", "http"] as const) {
+ test(`a reducer recovers from the ${transport === "stdio" ? "production stdio response limit" : "real IPC frame limit"} and records all sources`, async (t) => {
+ const root = await mkdtemp(path.join(tmpdir(), "deep-reducer-ipc-eval-"));
+ t.after(() => rm(root, { recursive: true, force: true }));
+ const { report, fixture } = await runReducerPagingEval({ root, transport });
+ assert.equal(report.realIpcErrorObserved, transport === "http");
+ assert.equal(report.stdioResponseErrorObserved, transport === "stdio");
+ assert.ok(report.actualOversizedResponseBytes! > report.ipcFrameLimitBytes);
+ assert.ok(report.recoveryBudget < report.firstBudget);
+ assert.ok(report.successfulPages > 1);
+ assert.equal(report.referenceReads, 2);
+ assert.equal(report.accountedSourceCount, 3);
+ assert.equal(report.preservedOriginalCount, 3);
+ assert.equal(report.previousIdentityPreserved, true);
+ assert.equal(report.synthesizedHistoryPreserved, true);
- const trace = await readJsonLines(path.join(root, "tool-trace.jsonl"));
- const rootPages = trace.filter(
- (event) =>
- event.event === "request" &&
- event.tool === "get_codex_security_deep_reducer_inputs" &&
- event.input.findingRef === undefined,
- );
- const missingPage = rootPages[2].id;
- assert.throws(
- () =>
- gradeReducerPagingTrace(
- trace.filter(
- (event) => !(event.event === "response" && event.id === missingPage),
+ const trace = await readJsonLines(path.join(root, "tool-trace.jsonl"));
+ const rootPages = trace.filter(
+ (event) =>
+ event.event === "request" &&
+ event.tool === "get_codex_security_deep_reducer_inputs" &&
+ event.input.findingRef === undefined,
+ );
+ const missingPage = rootPages[2].id;
+ assert.throws(
+ () =>
+ gradeReducerPagingTrace(
+ trace.filter(
+ (event) =>
+ !(event.event === "response" && event.id === missingPage),
+ ),
+ report.ipcFrameLimitBytes,
),
- report.ipcFrameLimitBytes,
- ),
- /read every assigned-input page/,
- );
+ /read every assigned-input page/,
+ );
- // Keeping every original is insufficient if distinct issues are collapsed.
- const result: {
- findings: {
- provenance: { sourceFindingIds: string[]; sourceFindings: unknown[] };
- }[];
- } = await readJson(fixture.resultPath);
- const previous = result.findings.find((finding) =>
- finding.provenance.sourceFindingIds.includes(
- fixture.expected.previousSourceId,
- ),
- );
- previous!.provenance.sourceFindingIds = result.findings.flatMap(
- (finding) => finding.provenance.sourceFindingIds,
- );
- previous!.provenance.sourceFindings = result.findings.flatMap(
- (finding) => finding.provenance.sourceFindings,
- );
- result.findings = [previous!];
- await writeFile(fixture.resultPath, JSON.stringify(result));
- await assert.rejects(
- gradeReducerPagingResult(fixture),
- /distinct|independent/,
- );
-});
+ // Keeping every original is insufficient if distinct issues are collapsed.
+ const result: {
+ findings: {
+ provenance: { sourceFindingIds: string[]; sourceFindings: unknown[] };
+ }[];
+ } = await readJson(fixture.resultPath);
+ const previous = result.findings.find((finding) =>
+ finding.provenance.sourceFindingIds.includes(
+ fixture.expected.previousSourceId,
+ ),
+ );
+ previous!.provenance.sourceFindingIds = result.findings.flatMap(
+ (finding) => finding.provenance.sourceFindingIds,
+ );
+ previous!.provenance.sourceFindings = result.findings.flatMap(
+ (finding) => finding.provenance.sourceFindings,
+ );
+ result.findings = [previous!];
+ await writeFile(fixture.resultPath, JSON.stringify(result));
+ await assert.rejects(
+ gradeReducerPagingResult(fixture),
+ /distinct|independent/,
+ );
+ });
+}
for (const failure of ["synthetic model failure", "empty trace"]) {
test(`retains a diagnostic report for ${failure}`, async (t) => {
diff --git a/plugins/codex-security/mcp-app/tests/test_stdio_transport.ts b/plugins/codex-security/mcp-app/tests/test_stdio_transport.ts
new file mode 100644
index 000000000..543bf6bfd
--- /dev/null
+++ b/plugins/codex-security/mcp-app/tests/test_stdio_transport.ts
@@ -0,0 +1,116 @@
+import assert from "node:assert/strict";
+import { PassThrough, Writable } from "node:stream";
+import test from "node:test";
+import { serializeMessage } from "@modelcontextprotocol/sdk/shared/stdio.js";
+import {
+ ErrorCode,
+ type JSONRPCMessage,
+} from "@modelcontextprotocol/sdk/types.js";
+import { CodexSecurityStdioTransport } from "../src/stdio-transport.ts";
+
+const nativeLineLimitBytes = 16 * 1024 * 1024;
+
+function captureTransport() {
+ const lines: string[] = [];
+ const stdout = new Writable({
+ write(chunk: Buffer, _encoding, callback) {
+ lines.push(chunk.toString("utf8"));
+ callback();
+ },
+ });
+ return {
+ transport: new CodexSecurityStdioTransport(new PassThrough(), stdout),
+ lines,
+ };
+}
+
+test("stdio preserves ordinary result and error responses exactly", async () => {
+ const { transport, lines } = captureTransport();
+ const messages: JSONRPCMessage[] = [
+ { jsonrpc: "2.0", id: 0, result: { text: 'A quote: " and a newline:\n' } },
+ {
+ jsonrpc: "2.0",
+ id: "",
+ error: { code: ErrorCode.InvalidParams, message: "Invalid page cursor" },
+ },
+ ];
+ for (const message of messages) await transport.send(message);
+ assert.deepEqual(lines, messages.map(serializeMessage));
+});
+
+test("stdio counts UTF-8 JSON escaping and the LF at the native line boundary", async () => {
+ for (const bytes of [nativeLineLimitBytes, nativeLineLimitBytes + 1]) {
+ const { transport, lines } = captureTransport();
+ const message = {
+ jsonrpc: "2.0" as const,
+ id: "boundary",
+ result: { text: 'é😀\n"' },
+ };
+ message.result.text += "a".repeat(
+ bytes - Buffer.byteLength(serializeMessage(message)),
+ );
+ const serialized = serializeMessage(message);
+ assert.equal(Buffer.byteLength(serialized), bytes);
+ assert.ok(serialized.length < bytes);
+
+ await transport.send(message);
+ assert.equal(lines.length, 1);
+ if (bytes === nativeLineLimitBytes) {
+ assert.equal(lines[0], serialized);
+ } else {
+ const response = JSON.parse(lines[0]!);
+ assert.equal(response.id, message.id);
+ assert.equal(response.error.code, ErrorCode.InternalError);
+ assert.match(response.error.message, /16777217 bytes.*16777216 bytes/);
+ assert.match(
+ response.error.message,
+ /reduce the requested page byte budget/,
+ );
+ assert.match(
+ response.error.message,
+ /operation may already have completed/,
+ );
+ }
+ }
+});
+
+test("an oversized error keeps its request ID and the next response succeeds", async () => {
+ const { transport, lines } = captureTransport();
+ await transport.send({
+ jsonrpc: "2.0",
+ id: 0,
+ error: {
+ code: ErrorCode.InternalError,
+ message: "Large diagnostic",
+ data: "a".repeat(nativeLineLimitBytes),
+ },
+ });
+ const next: JSONRPCMessage = { jsonrpc: "2.0", id: 1, result: { ok: true } };
+ await transport.send(next);
+
+ const response = JSON.parse(lines[0]!);
+ assert.equal(response.id, 0);
+ assert.equal(response.error.code, ErrorCode.InternalError);
+ assert.equal(response.error.data, undefined);
+ assert.ok(Buffer.byteLength(lines[0]!) <= nativeLineLimitBytes);
+ assert.equal(lines[1], serializeMessage(next));
+});
+
+test("stdio leaves outgoing requests and notifications unchanged", async () => {
+ const { transport, lines } = captureTransport();
+ const messages: JSONRPCMessage[] = [
+ {
+ jsonrpc: "2.0",
+ id: "server-request",
+ method: "sampling/createMessage",
+ params: { text: "a".repeat(nativeLineLimitBytes) },
+ },
+ {
+ jsonrpc: "2.0",
+ method: "notifications/message",
+ params: { data: "a".repeat(nativeLineLimitBytes) },
+ },
+ ];
+ for (const message of messages) await transport.send(message);
+ assert.deepEqual(lines, messages.map(serializeMessage));
+});
diff --git a/plugins/codex-security/scripts/workbench_saved_results.py b/plugins/codex-security/scripts/workbench_saved_results.py
index ffe954672..cd709d050 100644
--- a/plugins/codex-security/scripts/workbench_saved_results.py
+++ b/plugins/codex-security/scripts/workbench_saved_results.py
@@ -668,8 +668,27 @@ def _retained_findings(finding: dict[str, Any]) -> Iterator[dict[str, Any]]:
)
-def _deferred_rows(coverage: dict[str, Any]) -> list[Any]:
- rows = coverage.get("deferred", [])
+def _append_previous_finding(
+ item: dict[str, Any], finding: dict[str, Any], *, clone: bool = False
+) -> None:
+ if not isinstance(item.get("previousFindings"), list):
+ item["previousFindings"] = []
+ history = item["previousFindings"]
+ if not any(
+ isinstance(previous, dict)
+ and _finding_key(previous) == _finding_key(finding)
+ and _finding_content(previous) == _finding_content(finding)
+ for previous in history
+ ):
+ history.append(copy.deepcopy(finding) if clone else finding)
+
+
+def _has_candidate_metadata(row: dict[str, Any]) -> bool:
+ return any(key in row for key in ("candidateId", "candidate", "finding"))
+
+
+def _coverage_rows(coverage: dict[str, Any], field: str) -> list[Any]:
+ rows = coverage.get(field, [])
return rows if isinstance(rows, list) else []
@@ -681,7 +700,7 @@ def _resolved_deferred_rows(draft: dict[str, Any], schema: dict[str, Any]) -> li
try:
# Invalid closure metadata cannot discard the evidence it names.
_validate_schema_node(rows, schema, "coverage.resolvedDeferred")
- _validate_resolved_deferred({**coverage, "deferred": _deferred_rows(coverage)})
+ _validate_resolved_deferred({**coverage, "deferred": _coverage_rows(coverage, "deferred")})
except ContractError:
return []
return rows
@@ -696,15 +715,14 @@ def _merge_tied_parent_observations(
merged["findings"].append(copy.deepcopy(finding))
coverage = merged["coverage"]
for field in ("surfaces", "explicitExclusions", "deferred", "openQuestions"):
- rows = previous["coverage"].get(field, [])
output = coverage.setdefault(field, [])
- if isinstance(rows, list) and isinstance(output, list):
- for row in rows:
+ if isinstance(output, list):
+ for row in _coverage_rows(previous["coverage"], field):
if row not in output:
output.append(copy.deepcopy(row))
pending_ids = {
identity
- for row in _deferred_rows(coverage)
+ for row in _coverage_rows(coverage, "deferred")
if isinstance(row, dict)
for identity in (row.get("id"), row.get("candidateId"))
if isinstance(identity, str)
@@ -734,9 +752,7 @@ def _deferred_candidate_id(
identity = row.get("candidateId") or row.get("id")
if not isinstance(identity, str):
return None
- if (owner, identity) in ambiguous_deferred and not any(
- key in row for key in ("candidateId", "candidate", "finding")
- ):
+ if (owner, identity) in ambiguous_deferred and not _has_candidate_metadata(row):
return None
return identity
@@ -781,8 +797,7 @@ def _generic_surface_updates(
saved_surfaces: dict[tuple[str | None, str], list[tuple[str, dict[str, Any]]]] = {}
for relative, draft, owner in sources:
- rows = draft["coverage"].get("surfaces", [])
- for row in rows if isinstance(rows, list) else []:
+ for row in _coverage_rows(draft["coverage"], "surfaces"):
if isinstance(row, dict) and isinstance(row.get("id"), str):
saved_surfaces.setdefault((owner, row["id"]), []).append((relative, row))
replaced: set[int] = set()
@@ -806,8 +821,7 @@ def _generic_surface_updates(
if source_order[relative] == source_order[latest_source]
)
for relative, draft, owner in sources:
- rows = draft["coverage"].get("surfaces", [])
- for row in rows if isinstance(rows, list) else []:
+ for row in _coverage_rows(draft["coverage"], "surfaces"):
if (
isinstance(row, dict)
and "id" not in row
@@ -844,8 +858,7 @@ def _generic_surface_updates(
}
if not closed_ids and not reopened_ids and receipt_root is None:
continue
- current = draft["coverage"].get("surfaces", [])
- for surface in current if isinstance(current, list) else []:
+ for surface in _coverage_rows(draft["coverage"], "surfaces"):
if not isinstance(surface, dict):
continue
reopening = surface.get("disposition") == "needs_follow_up"
@@ -867,9 +880,7 @@ def _generic_surface_updates(
by_source = dict(matches)
if any(row != by_source[saved_path] for saved_path, row in matches):
continue
- if any(
- "candidateId" in row or "candidate" in row or "finding" in row for _, row in matches
- ):
+ if any(_has_candidate_metadata(row) for _, row in matches):
continue
if any(
row is not surface
@@ -886,7 +897,7 @@ def _generic_surface_updates(
or (
isinstance(row.get("id"), str)
and (owner, row["id"]) in ambiguous_deferred
- and not any(key in row for key in ("candidateId", "candidate", "finding"))
+ and not _has_candidate_metadata(row)
)
)
and linked(row, identity)
@@ -1393,16 +1404,13 @@ def merge_saved_results(
for field in ("deferred", "surfaces"):
named_rows: dict[str | None, list[dict[str, Any]]] = {}
for _, draft, owner in all_sources:
- rows = draft["coverage"].get(field, [])
named_rows.setdefault(owner, []).extend(
row
- for row in (rows if isinstance(rows, list) else [])
+ for row in _coverage_rows(draft["coverage"], field)
if isinstance(row, dict) and isinstance(row.get("id"), str)
)
for _, draft, owner in all_sources:
- rows = draft["coverage"].get(field, [])
- if not isinstance(rows, list):
- continue
+ rows = _coverage_rows(draft["coverage"], field)
reserved = {
row["id"]
for row in rows
@@ -1411,9 +1419,7 @@ def merge_saved_results(
for row in rows:
if not isinstance(row, dict) or "id" in row:
continue
- if field == "deferred" and any(
- key in row for key in ("candidateId", "candidate", "finding")
- ):
+ if field == "deferred" and _has_candidate_metadata(row):
continue
content = {"receiptRefs": [], **row} if field == "surfaces" else row
identity = next(
@@ -1555,7 +1561,7 @@ def merge_saved_results(
{
id(item)
for field in ("surfaces", "explicitExclusions", "deferred")
- for item in (coverage.get(field) if isinstance(coverage.get(field), list) else [])
+ for item in _coverage_rows(coverage, field)
}
if parent_is_canonical
else set()
@@ -1589,7 +1595,8 @@ def merge_saved_results(
source_order["parent"] = (0, parent_modified)
deferred_rows = {
- relative: _deferred_rows(draft["coverage"]) for relative, draft, _ in all_sources
+ relative: _coverage_rows(draft["coverage"], "deferred")
+ for relative, draft, _ in all_sources
}
# A legacy source can contain independent tasks with the same explicit ID.
# Later rewrites cannot make an ID-only closure identify one of those tasks.
@@ -1599,15 +1606,14 @@ def merge_saved_results(
candidate_aliases = {
identity
for row in deferred_rows[relative]
- if isinstance(row, dict)
- and any(key in row for key in ("candidateId", "candidate", "finding"))
+ if isinstance(row, dict) and _has_candidate_metadata(row)
for identity in (row.get("id"), row.get("candidateId"))
if isinstance(identity, str)
}
for row in deferred_rows[relative]:
if not isinstance(row, dict) or not isinstance(identity := row.get("id"), str):
continue
- if any(key in row for key in ("candidateId", "candidate", "finding")):
+ if _has_candidate_metadata(row):
continue
if identity in candidate_aliases or (identity in by_id and row != by_id[identity]):
ambiguous_deferred.add((owner, identity))
@@ -1688,7 +1694,7 @@ def merge_saved_results(
isinstance(row, dict)
and isinstance(identity := row.get("id"), str)
and (owner, identity) in ambiguous_deferred
- and not any(key in row for key in ("candidateId", "candidate", "finding"))
+ and not _has_candidate_metadata(row)
and (owner, row) not in reopened_rows
):
reopened_rows.append((owner, row))
@@ -1726,8 +1732,7 @@ def merge_saved_results(
):
outcomes.append((relative, owner, candidate_id, "reported"))
for field in ("surfaces", "explicitExclusions"):
- items = draft["coverage"].get(field, [])
- for item in items if isinstance(items, list) else []:
+ for item in _coverage_rows(draft["coverage"], field):
if (
isinstance(item, dict)
and isinstance(item.get("candidateId"), str)
@@ -1875,10 +1880,7 @@ def merge_saved_results(
if (
selected_coverage_superseded
and field != "deferred"
- and not (
- isinstance(item, dict)
- and any(key in item for key in ("candidateId", "candidate", "finding"))
- )
+ and not (isinstance(item, dict) and _has_candidate_metadata(item))
):
return False
# A selected outcome must retain its evidence even if its result write failed.
@@ -1913,7 +1915,7 @@ def merge_saved_results(
for owner, row in reopened_rows
if isinstance(row.get("id"), str)
and (owner, row["id"]) in ambiguous_deferred
- and not any(key in row for key in ("candidateId", "candidate", "finding"))
+ and not _has_candidate_metadata(row)
for identity in [
row["id"],
*(row.get("surfaceIds", []) if isinstance(row.get("surfaceIds", []), list) else []),
@@ -1921,15 +1923,22 @@ def merge_saved_results(
if isinstance(identity, str)
}
for _, draft, owner in all_sources:
- surfaces = draft["coverage"].get("surfaces", [])
by_id: dict[str, dict[str, Any]] = {}
- for surface in surfaces if isinstance(surfaces, list) else []:
+ for surface in _coverage_rows(draft["coverage"], "surfaces"):
if not isinstance(surface, dict) or not isinstance(identity := surface.get("id"), str):
continue
if identity in by_id and surface != by_id[identity]:
ambiguous_surface_ids.add((owner, identity))
by_id[identity] = surface
+ def ambiguous_pending_surface(row: dict[str, Any], owner: str | None) -> bool:
+ return (
+ isinstance(row.get("id"), str)
+ and (owner, row["id"]) in ambiguous_surface_ids
+ and row.get("disposition") == "needs_follow_up"
+ and not _has_candidate_metadata(row)
+ )
+
replaced_deferred = {
id(row)
for relative, _, owner in all_sources
@@ -1963,18 +1972,11 @@ def merge_saved_results(
prospective_tasks.append(
(owner, _project_worker_coverage_row(row, owner, "deferred"))
)
- rows = draft["coverage"].get("surfaces", [])
- for row in rows if isinstance(rows, list) else []:
+ for row in _coverage_rows(draft["coverage"], "surfaces"):
if owner is None or not isinstance(row, dict):
continue
emitted = coverage_row_is_retained("surfaces", row, owner, selection)
- ambiguous = (
- isinstance(row.get("id"), str)
- and (owner, row["id"]) in ambiguous_surface_ids
- and row.get("disposition") == "needs_follow_up"
- and not any(key in row for key in ("candidateId", "candidate", "finding"))
- )
- if not emitted and not ambiguous:
+ if not emitted and not ambiguous_pending_surface(row, owner):
continue
projected = _project_worker_coverage_row(row, owner, "surfaces")
if isinstance(row.get("id"), str):
@@ -2057,14 +2059,10 @@ def merge_saved_results(
if isinstance(pending, list):
append_coverage_row(pending, item, owner, "deferred")
for _, draft, owner in all_sources:
- surfaces = draft["coverage"].get("surfaces", [])
- for surface in surfaces if isinstance(surfaces, list) else []:
+ for surface in _coverage_rows(draft["coverage"], "surfaces"):
if (
isinstance(surface, dict)
- and isinstance(surface.get("id"), str)
- and (owner, surface["id"]) in ambiguous_surface_ids
- and surface.get("disposition") == "needs_follow_up"
- and not any(key in surface for key in ("candidateId", "candidate", "finding"))
+ and ambiguous_pending_surface(surface, owner)
and isinstance(coverage.get("surfaces"), list)
):
append_coverage_row(
@@ -2154,19 +2152,9 @@ def merge_saved_results(
"rejected",
"not_applicable",
}:
- surfaces = coverage.get("surfaces")
- for item in surfaces if isinstance(surfaces, list) else []:
+ for item in _coverage_rows(coverage, "surfaces"):
if isinstance(item, dict) and item.get("candidateId") == candidate_id:
- if not isinstance(item.get("previousFindings"), list):
- item["previousFindings"] = []
- history = item["previousFindings"]
- if not any(
- isinstance(previous, dict)
- and _finding_key(previous) == _finding_key(finding)
- and _finding_content(previous) == _finding_content(finding)
- for previous in history
- ):
- history.append(finding)
+ _append_previous_finding(item, finding)
continue
for key in ("findingId", "occurrenceId", "fingerprints"):
finding.pop(key, None)
@@ -2293,17 +2281,8 @@ def merge_saved_results(
and (history_findings := rejected_history.get((worker_id, item["candidateId"])))
):
item = copy.deepcopy(item)
- if not isinstance(item.get("previousFindings"), list):
- item["previousFindings"] = []
- history = item["previousFindings"]
for finding in history_findings:
- if not any(
- isinstance(previous, dict)
- and _finding_key(previous) == _finding_key(finding)
- and _finding_content(previous) == _finding_content(finding)
- for previous in history
- ):
- history.append(copy.deepcopy(finding))
+ _append_previous_finding(item, finding, clone=True)
if field == "surfaces" and isinstance(item, dict):
item = {**item, "receiptRefs": item.get("receiptRefs", [])}
if isinstance(item, dict) and "id" not in item:
@@ -2345,11 +2324,8 @@ def merge_saved_results(
# Candidate metadata can give an anonymous row the same preliminary ID as
# another surface, so only the final surface ID can be used for its links.
retained_surface_links: dict[int, tuple[dict[str, Any], dict[str, dict[str, Any]]]] = {}
- output_tasks = coverage.get("deferred", [])
- task_candidates = [
- (worker_row_owners.get(id(row)), row)
- for row in (output_tasks if isinstance(output_tasks, list) else [])
- ]
+ output_tasks = _coverage_rows(coverage, "deferred")
+ task_candidates = [(worker_row_owners.get(id(row)), row) for row in output_tasks]
for task, published_id, surface, _, task_source in surface_links:
retained_surface = linked_surface(surface, linked_surface_outputs, worker_surface_outputs)
if retained_surface is None or not isinstance(retained_surface.get("id"), str):
@@ -2375,8 +2351,7 @@ def merge_saved_results(
named_surface_outputs = {
identity for outputs in worker_surface_outputs.values() for identity in outputs
}
- surfaces = coverage.get("surfaces", [])
- for surface in surfaces if isinstance(surfaces, list) else []:
+ for surface in _coverage_rows(coverage, "surfaces"):
if not isinstance(surface, dict):
continue
owner, identity = worker_row_owners.get(id(surface)), surface.get("id")
@@ -2390,7 +2365,7 @@ def merge_saved_results(
and id(existing) not in named_surface_outputs
):
worker_surfaces[key] = surface
- for task in output_tasks if isinstance(output_tasks, list) else []:
+ for task in output_tasks:
if (
not isinstance(task, dict)
or (owner := worker_row_owners.get(id(task))) is None
@@ -2407,12 +2382,12 @@ def merge_saved_results(
for field in ("surfaces", "explicitExclusions", "deferred"):
used: set[str] = set()
items = coverage.setdefault(field, [])
+ if not isinstance(items, list):
+ continue
reserved = {
- row["id"]
- for row in (items if isinstance(items, list) else [])
- if isinstance(row, dict) and isinstance(row.get("id"), str)
+ row["id"] for row in items if isinstance(row, dict) and isinstance(row.get("id"), str)
}
- for item in items if isinstance(items, list) else []:
+ for item in items:
if not isinstance(item, dict):
continue
if id(item) in canonical_rows:
diff --git a/sdk/typescript/package.json b/sdk/typescript/package.json
index 54cf470ef..61775d4d7 100644
--- a/sdk/typescript/package.json
+++ b/sdk/typescript/package.json
@@ -84,8 +84,8 @@
"@inquirer/prompts": "8.7.3",
"@linear/sdk": "98.0.0",
"@octokit/core": "7.0.8",
- "@openai/codex": "0.163.0",
- "@openai/codex-sdk": "0.163.0",
+ "@openai/codex": "0.164.0",
+ "@openai/codex-sdk": "0.164.0",
"ajv": "8.20.0",
"fflate": "0.8.3",
"incur": "0.7.1",
diff --git a/sdk/typescript/pnpm-lock.yaml b/sdk/typescript/pnpm-lock.yaml
index c23f4565b..5a8127a47 100644
--- a/sdk/typescript/pnpm-lock.yaml
+++ b/sdk/typescript/pnpm-lock.yaml
@@ -23,11 +23,11 @@ importers:
specifier: 7.0.8
version: 7.0.8
'@openai/codex':
- specifier: 0.163.0
- version: 0.163.0
+ specifier: 0.164.0
+ version: 0.164.0
'@openai/codex-sdk':
- specifier: 0.163.0
- version: 0.163.0
+ specifier: 0.164.0
+ version: 0.164.0
ajv:
specifier: 8.20.0
version: 8.20.0
@@ -745,47 +745,47 @@ packages:
'@octokit/[email protected]':
resolution: {integrity: sha512-l6bAF43PNxkJp6g+W4PjoUSSkxHomXw2nOum5CTftJz1NlV3vu93NImgOYtLf6CbBUb5j+fiuzW0PPQ5JTSvZA==}
- '@openai/[email protected]':
- resolution: {integrity: sha512-o10d8dqxFjhzoWp75MUrhaN8C6QrXPQqUJGwVnySj++ztAnrA322UuFRzHZvSa903lFgjjaimuf5rkqVsj0RHg==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-FyhTWM4Xa8hXpWhkr9MOvROVhBqTeD+s/XCzVLzqSQXTSnGSNkdBWjv0IzZdOO29BX4LwtMjul6ogB8tOME5og==}
engines: {node: '>=18'}
- '@openai/[email protected]':
- resolution: {integrity: sha512-HNd7vR9oTK6M2ikxFnvNDCdYUgAsyo9VVnd5KnEd2ON0GYI++7MiX9XNPUkhtdgGKFPsWS+CEppQ1UU63uVhUw==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-TWj7UGyPRe3Pj/A3ddNAxs1te3a/W/xAd+J0LHrrFTmsSfahGMOLF/vn88D3QCIgAT3B1rm1y/Z3NYE0rx6cZQ==}
engines: {node: '>=16'}
hasBin: true
- '@openai/[email protected]':
- resolution: {integrity: sha512-i7Dzk2a5aGmuGM9aNZgaUVTojZ/4bc5lRfosz++Y7X2BGhDkz+cySgbD+NYS1pOxfm0VEbLi9PZ3pAulI4kchw==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-pDizgOUAtq3/k3AbUFHbaU19WpYKCja+oFpNSTLKrDQj4lTUNSDuTPJip4uQVn2/1EipQdpQrGMOQBVuI1znAA==}
engines: {node: '>=16'}
cpu: [arm64]
os: [darwin]
- '@openai/[email protected]':
- resolution: {integrity: sha512-Ioj7h+B1ME9ZU1PH1ATfvFV5DiototEpFFaSSbg3kfDifB9djhl5fcJ++g5mpcL/+uVbcnReei+RiSXEx8o+kw==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-lIe+udnguPGzBezkgbkj2Eaz0YqOhNXT7jja136Fj0sXfI+UKNS2icmaK14GcoRAgeVEpDTtC1ftamWBPmD1zg==}
engines: {node: '>=16'}
cpu: [x64]
os: [darwin]
- '@openai/[email protected]':
- resolution: {integrity: sha512-IH2/R+4OarjGvsAf1ll3fdyLwrPgYLl3uzp6u9CyD9sj2Lw+ViDSe3QBZVHdcHJs0eIiZFg6Wprcuy4CRZaUGw==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-LL5iTQjyDL4YRX44EWqhIfEQ5uGnmgxkQBJSxjZSeJ8L1Og83pdlzUJW/PRJ76oaVwOVgh6HXwuPEiBNEHU4lg==}
engines: {node: '>=16'}
cpu: [arm64]
os: [linux]
- '@openai/[email protected]':
- resolution: {integrity: sha512-uwRaOsrYogLQhcxAM/GoeWrw0QIWEMRo7c9gM8sKAviVkZQHlu9kRLJ5PkufbV+ABGjyZgIwiFdJ0LCJ0N6e8w==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-XLAoaWsGOT2khzl+P4FYDPTkzAr9W0VFXFaVkqlFdkM0soa1gH/4S0wSD57Ek3HHwkirAEGz2/Mwehseh+l5Mg==}
engines: {node: '>=16'}
cpu: [x64]
os: [linux]
- '@openai/[email protected]':
- resolution: {integrity: sha512-QnLn3gR2dExcdTThE/HAo5BrUq0/C8zDGQ0UZL3hNzfqt9ZklT6jGTyt8JhDyNZX4uSB4HhcJtbcOWucZGC0VQ==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-Zkzt8HNmB/vOnlDo5D7IOlrt5rHaxYQB7iGbNP7bHcXZtrk6gaD7UriwaieGYJpcf8DqAT/0tqzY4xbs8ZwbmQ==}
engines: {node: '>=16'}
cpu: [arm64]
os: [win32]
- '@openai/[email protected]':
- resolution: {integrity: sha512-QQGQ/q9dCJ5C5jqhgFrwycpWG3szIyFNThMiKl6hYtu6N/eM3Qd7StesgnCFAfzVED8IkE3N3oH71szzcludqw==}
+ '@openai/[email protected]':
+ resolution: {integrity: sha512-urWPhwh9YavuoAPIkmVYKBTtd5OtGVL4Ki0Fb28dhK/Mz6Pt4LSy+gbRrZFz5+RXrw9ubvXuXDCcd8beYcWhDg==}
engines: {node: '>=16'}
cpu: [x64]
os: [win32]
@@ -2209,35 +2209,35 @@ snapshots:
dependencies:
'@octokit/openapi-types': 29.0.1
- '@openai/[email protected]':
+ '@openai/[email protected]':
dependencies:
- '@openai/codex': 0.163.0
+ '@openai/codex': 0.164.0
- '@openai/[email protected]':
+ '@openai/[email protected]':
optionalDependencies:
- '@openai/codex-darwin-arm64': '@openai/[email protected]'
- '@openai/codex-darwin-x64': '@openai/[email protected]'
- '@openai/codex-linux-arm64': '@openai/[email protected]'
- '@openai/codex-linux-x64': '@openai/[email protected]'
- '@openai/codex-win32-arm64': '@openai/[email protected]'
- '@openai/codex-win32-x64': '@openai/[email protected]'
+ '@openai/codex-darwin-arm64': '@openai/[email protected]'
+ '@openai/codex-darwin-x64': '@openai/[email protected]'
+ '@openai/codex-linux-arm64': '@openai/[email protected]'
+ '@openai/codex-linux-x64': '@openai/[email protected]'
+ '@openai/codex-win32-arm64': '@openai/[email protected]'
+ '@openai/codex-win32-x64': '@openai/[email protected]'
- '@openai/[email protected]':
+ '@openai/[email protected]':
optional: true
- '@openai/[email protected]':
+ '@openai/[email protected]':
optional: true
- '@openai/[email protected]':
+ '@openai/[email protected]':
optional: true
- '@openai/[email protected]':
+ '@openai/[email protected]':
optional: true
- '@openai/[email protected]':
+ '@openai/[email protected]':
optional: true
- '@openai/[email protected]':
+ '@openai/[email protected]':
optional: true
'@scalar/[email protected]': {}