Skip to content

OpenAI's Codex Security CLI and TypeScript SDK for finding, validating, and fixing security vulnerabilities. npm: https://www.npmjs.com/package/@openai/codex-security

fix(deep-scan): preserve pager state in reducer retry guidance (#1589)

Michael D'Angeloe9743676b77b
diff --git a/plugins/codex-security/mcp-app/src/deep-scan/worker-runner.ts b/plugins/codex-security/mcp-app/src/deep-scan/worker-runner.ts
index 98cf800ca..cfa691ba7 100644
--- a/plugins/codex-security/mcp-app/src/deep-scan/worker-runner.ts
+++ b/plugins/codex-security/mcp-app/src/deep-scan/worker-runner.ts
@@ -685,6 +685,7 @@ function reducerInputRecoveryInstructions(): string[] {
return [
"If input retrieval exceeded a response or IPC size limit, call get_codex_security_deep_reducer_inputs({ cursor?, maxBytes, findingRef? }) with a smaller maxBytes budget (for example, halve the failed request's budget), preserving its cursor and findingRef to retry the same page.",
"Do not repeat an oversized request unchanged. If inputs have not been read yet, paginate all assigned findings and the previous aggregate, assembling each json fragment in code and following nextCursor until absent; inspect assembled data selectively and fetch details by findingRef.",
+ "Persist fragments, cursor: nextCursor ?? null, and completion together in one serializable store value after each successful page. Never store an undefined cursor or append a page already committed before a later cell failure. Check MCP isError before parsing its text as a page.",
"Preserve pages already read and account for every finding before submitting the aggregate.",
];
}
diff --git a/plugins/codex-security/mcp-app/src/server/compact-artifact-tools.ts b/plugins/codex-security/mcp-app/src/server/compact-artifact-tools.ts
index ff9ab1499..81ab81bbb 100644
--- a/plugins/codex-security/mcp-app/src/server/compact-artifact-tools.ts
+++ b/plugins/codex-security/mcp-app/src/server/compact-artifact-tools.ts
@@ -288,7 +288,8 @@ export function registerCompactWorkerArtifactTools(
title: "Get Codex Security Deep Reducer Inputs",
description:
"Read a byte-budgeted JSON fragment of the assigned findings and previous aggregate. " +
- "Continue with nextCursor; concatenate json fragments before parsing. " +
+ "Continue with nextCursor until absent; concatenate json fragments before parsing. " +
+ "When using code-mode store, save fragments, cursor:nextCursor??null, and done:!nextCursor together in one state object; undefined cannot be stored. Reuse completed state without fetching again. " +
"Use findingRef source:<sourceFindingId> or previous:N to read full retained provenance.",
inputSchema: deepReducerInputsInputSchema,
annotations: readingAnnotations,
diff --git a/plugins/codex-security/mcp-app/templates/deep-scan/dedup.md b/plugins/codex-security/mcp-app/templates/deep-scan/dedup.md
index a062071c2..1ac9ecea8 100644
--- a/plugins/codex-security/mcp-app/templates/deep-scan/dedup.md
+++ b/plugins/codex-security/mcp-app/templates/deep-scan/dedup.md
@@ -10,6 +10,35 @@ Read the assigned, already-validated Standard findings and the previous aggregat
Each tool response has one text content block containing `{ json, nextCursor? }`. Parse that block to obtain the page. Its `json` is a fragment, not necessarily a complete JSON value. Follow `nextCursor` until absent, concatenate the fragments inside code mode, and only then parse the assembled document as `{ discoveries, previous }`. Inspect the findings and context selectively; do not print or return the entire assembled document from code mode. Do not concatenate pages from different `findingRef` requests.
+Keep fragments, cursor and completion together in one serializable `store` value. An absent terminal cursor must become `null`; `store` cannot save `undefined`. Commit the state only after a successful page, so a retry resumes at the next unread page rather than appending a completed tail again. In code mode, define this helper and call it with your selected byte budget (and optional `findingRef`); inspect only the needed fields of its returned document:
+
+```javascript
+async function readReducerDocument(maxBytes, findingRef = null) {
+ const key = "deep-reducer-pages:" + (findingRef ?? "assigned");
+ let state = load(key) ?? { json: "", cursor: null, done: false };
+ while (!state.done) {
+ const response =
+ await tools.mcp__cs_artifacts__get_codex_security_deep_reducer_inputs({
+ maxBytes,
+ ...(state.cursor === null ? {} : { cursor: state.cursor }),
+ ...(findingRef === null ? {} : { findingRef }),
+ });
+ const body = response.content.find((item) => item.type === "text").text;
+ if (response.isError) throw new Error(body);
+ const page = JSON.parse(body);
+ state = {
+ json: state.json + page.json,
+ cursor: page.nextCursor ?? null,
+ done: page.nextCursor == null,
+ };
+ store(key, state);
+ }
+ return JSON.parse(state.json);
+}
+```
+
+Use the exact exposed tool name if the host uses a different MCP namespace. Code mode has only its documented globals; use `load`/`store` for state between cells and ordinary JSON operations for JSON data, rather than assuming Node.js globals or `structuredClone` are available.
+
The input view omits the retained `provenance.sourceFindings`, `previousFindings`, and `originalCandidates` bodies. The host preserves those bodies when saving the aggregate. Fetch full details when needed with the same paged tool and `findingRef`: use `source:<sourceFindingId>` for an original source, or `previous:N` for the previous aggregate's zero-based finding index, including its synthesized history. Never copy those retained bodies into your submission; submit the semantic finding and its source refs.
If a response exceeds the IPC frame limit, retry the failed page with a smaller `maxBytes` (halve the budget used for that request), keeping its cursor and findingRef. Do not retry an unchanged oversized request. Preserve already-read fragments and continue from the failed page. If the budget is too small to hold even one character and the page envelope, increase it as directed by the tool's diagnostic.
diff --git a/plugins/codex-security/mcp-app/tests/test_deep_scan_templates.ts b/plugins/codex-security/mcp-app/tests/test_deep_scan_templates.ts
index 8e22123d1..295c953fd 100644
--- a/plugins/codex-security/mcp-app/tests/test_deep_scan_templates.ts
+++ b/plugins/codex-security/mcp-app/tests/test_deep_scan_templates.ts
@@ -73,6 +73,91 @@ assert.deepEqual(previousReduction, {
claimedWorkerIds: [],
});
+// Execute the documented pager across a failed page and fresh code-mode cells.
+// Its stored terminal state must never replay a successfully committed tail.
+const pagerExample = dedup.match(/```javascript\n([\s\S]*?)\n```/);
+assert.ok(pagerExample);
+const values = new Map<string, unknown>();
+const requests: { maxBytes: number; cursor?: string; findingRef?: string }[] =
+ [];
+const assigned = { discoveries: [{ workerId: "worker-001" }], previous: null };
+const fullFinding = { title: "Original source finding" };
+let failNextAssignedTail = true;
+const tools = {
+ async mcp__cs_artifacts__get_codex_security_deep_reducer_inputs(input: {
+ maxBytes: number;
+ cursor?: string;
+ findingRef?: string;
+ }) {
+ requests.push(input);
+ if (input.cursor && !input.findingRef && failNextAssignedTail) {
+ failNextAssignedTail = false;
+ return {
+ isError: true,
+ content: [
+ { type: "text", text: "Retry this page with a smaller byte budget." },
+ ],
+ };
+ }
+ const json = JSON.stringify(input.findingRef ? fullFinding : assigned);
+ const offset = Number(input.cursor ?? 0);
+ const end = Math.min(offset + 20, json.length);
+ return {
+ content: [
+ {
+ type: "text",
+ text: JSON.stringify({
+ json: json.slice(offset, end),
+ ...(end < json.length ? { nextCursor: String(end) } : {}),
+ }),
+ },
+ ],
+ };
+ },
+};
+function freshPager() {
+ return new Function(
+ "tools",
+ "load",
+ "store",
+ `${pagerExample![1]}\nreturn readReducerDocument;`,
+ )(
+ tools,
+ (key: string) => values.get(key),
+ (key: string, value: unknown) => {
+ const persisted = JSON.parse(JSON.stringify(value));
+ assert.deepEqual(
+ persisted,
+ value,
+ "stored paging state must not contain undefined fields",
+ );
+ values.set(key, persisted);
+ },
+ );
+}
+await assert.rejects(freshPager()(1000), /smaller byte budget/);
+assert.deepEqual(values.get("deep-reducer-pages:assigned"), {
+ json: JSON.stringify(assigned).slice(0, 20),
+ cursor: "20",
+ done: false,
+});
+assert.deepEqual(await freshPager()(500), assigned);
+assert.equal(requests[1].cursor, requests[2].cursor, "retry the unread page");
+assert.ok(requests[2].maxBytes < requests[1].maxBytes);
+const afterCompletion = requests.length;
+assert.deepEqual(await freshPager()(500), assigned);
+assert.equal(
+ requests.length,
+ afterCompletion,
+ "a new cell must not append the terminal page again",
+);
+assert.deepEqual(await freshPager()(500, "source:worker-001:0"), fullFinding);
+assert.deepEqual(
+ await freshPager()(500),
+ assigned,
+ "findingRef reads use separate paging state",
+);
+
function firstJsonBlock(prompt: string) {
const match = prompt.match(/```json\n([\s\S]*?)\n```/);
assert.ok(match, "prompt should contain a JSON context block");