Skip to content

OpenAI's Codex Security CLI and TypeScript SDK for finding, validating, and fixing security vulnerabilities. npm: https://www.npmjs.com/package/@openai/codex-security

fix(deep-scan): report completed audits and batch consolidation (#2305)

Michael D'Angelo366093f77b03
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index e3e773561..8a58479fb 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -29,8 +29,8 @@ maintained in the SDK.
`pnpm --dir sdk/typescript run types` checks freshness.
Model-based evaluations live in [`evals/`](evals/README.md), outside the plugin
-source. Deterministic triage checks and the MCP reducer IPC regression remain
-part of normal CI.
+source. CI runs deterministic triage checks and the Deep Scan iteration and
+publication tests in the MCP suite.
See the [CI guide](docs/ci.md) for check selection, platform/version coverage,
required status checks, and test-runner qualification.
diff --git a/docs/project-configuration.md b/docs/project-configuration.md
index 4df29d821..294c57363 100644
--- a/docs/project-configuration.md
+++ b/docs/project-configuration.md
@@ -79,8 +79,8 @@ node sdk/typescript/bin/codex-security.mjs scan . -c docs/examples/codex-securit
## Desktop and conversational Deep Scans
When the Codex host supplies the launching turn’s speed, Deep Scan uses it for
-discovery and reducer workers, including resumed workers. Selecting Default
-overrides a faster tier in the worker’s Codex config. On hosts that omit the
+Standard scan workers, resumed workers, and finding comparisons. Selecting
+Default overrides a faster tier in the worker’s Codex config. On hosts that omit the
turn’s speed, each worker uses the Codex configuration it loads. To keep a scan
at Default on those hosts, leave `service_tier = "default"` in the worker’s
config for the duration of the scan, including any resumes. Other scans that
diff --git a/evals/README.md b/evals/README.md
index f8372424b..d6605e639 100644
--- a/evals/README.md
+++ b/evals/README.md
@@ -18,5 +18,5 @@ being embedded in its source tree or shipped npm runtime.
for individual skill files and bundles, including cross-file behavior.
Model runs are opt-in. CI runs the deterministic triage, secret-discovery,
-time-budget, and workflow-routing helper checks and the real-IPC reducer
-regression through the normal MCP test suite.
+time-budget, and workflow-routing helper checks. The MCP suite covers Deep Scan
+iterations, recovery, and publication.
diff --git a/plugins/codex-security/mcp-app/server.ts b/plugins/codex-security/mcp-app/server.ts
index 88f5aed47..e704b3203 100644
--- a/plugins/codex-security/mcp-app/server.ts
+++ b/plugins/codex-security/mcp-app/server.ts
@@ -1027,12 +1027,24 @@ export function createCodexSecurityServer(): McpServer {
),
]);
// Full findings and workspace history can exceed the host's tool-result limit.
- const { progress, reportAvailable } = completion.scan as {
- progress: { status: string };
- reportAvailable: boolean;
- };
+ const { progress, reportAvailable, usage, cost } =
+ completion.scan as {
+ progress: { status: string };
+ reportAvailable: boolean;
+ usage?: JsonObject;
+ cost?: { model: string; estimatedUsd: number };
+ };
+ // Rate-limit snapshots can grow independently of the measured scan totals.
+ const { accountRateLimits: _limits, ...scanUsage } = usage ?? {};
return deepScanTerminalResult(run, {
- scan: { progress: { status: progress.status }, reportAvailable },
+ scan: {
+ progress: { status: progress.status },
+ reportAvailable,
+ ...(usage && { usage: scanUsage }),
+ ...(cost && {
+ cost: { model: cost.model, estimatedUsd: cost.estimatedUsd },
+ }),
+ },
});
} catch (error) {
logDeepScanEvent({
@@ -2332,14 +2344,20 @@ function deepScanTerminalResult(
) {
if (run.status === "succeeded") {
if (!run.manifestPath) return undefined;
+ const outcome =
+ run.terminalReason === "capped" && run.completionSequence === 0
+ ? "The discovery deadline expired before any Standard scan completed. Report the coverage limitation."
+ : "Report the retained findings from completed Standard scans.";
const instructions = run.sdkOwnsCompletion
- ? `Deep Scan ${run.scanId} aggregated its validated findings under ${run.scanDir}. End this turn with the returned scanId and manifestPath. The SDK owns finalization and report generation after the turn ends.`
- : `Deep Scan ${run.scanId} completed. The coordinator validated and sealed scan-manifest.json and findings.json and generated report.md under ${run.scanDir}. Independent Standard scans have already performed validation and attack-path analysis. Use the returned authoritative scanId, scanDir, and manifestPath. Do not call complete_codex_security_scan again, rerun worker phases, list candidates, submit another draft, or start a replacement scan. Report the completed results and link the generated report.`;
+ ? `Deep Scan ${run.scanId} stopped. ${outcome} End this turn with the returned scanId and manifestPath. The SDK owns finalization and report generation after the turn ends.`
+ : `Deep Scan ${run.scanId} completed. ${outcome} The coordinator sealed scan-manifest.json and findings.json and generated report.md under ${run.scanDir}. Use the returned authoritative scanId, scanDir, and manifestPath. Do not call complete_codex_security_scan again, rerun worker phases, list candidates, submit another draft, or start a replacement scan. Link the generated report.`;
return scanActionResult(
{
scanId: run.scanId,
scanDir: run.scanDir,
manifestPath: run.manifestPath,
+ completedAudits: run.completionSequence,
+ terminalReason: run.terminalReason,
...(completion === undefined
? {}
: { completion, reportPath: resolve(run.scanDir, "report.md") }),
diff --git a/plugins/codex-security/mcp-app/src/deep-scan/response.ts b/plugins/codex-security/mcp-app/src/deep-scan/response.ts
index 72291e074..a69b659a1 100644
--- a/plugins/codex-security/mcp-app/src/deep-scan/response.ts
+++ b/plugins/codex-security/mcp-app/src/deep-scan/response.ts
@@ -94,6 +94,7 @@ export const runSchema = z
.optional(),
scanDir: requiredString,
dispatchedCount: nonNegativeInteger,
+ completionSequence: nonNegativeInteger.optional(),
noNewStreak: nonNegativeInteger,
consecutiveErrors: nonNegativeInteger
.nullish()
diff --git a/plugins/codex-security/mcp-app/tests/test_deep_scan_publication_integration.ts b/plugins/codex-security/mcp-app/tests/test_deep_scan_publication_integration.ts
index 9355d9a53..865c9b1be 100644
--- a/plugins/codex-security/mcp-app/tests/test_deep_scan_publication_integration.ts
+++ b/plugins/codex-security/mcp-app/tests/test_deep_scan_publication_integration.ts
@@ -19,6 +19,7 @@ import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js"
import { buildServer } from "./build-server.ts";
import { importModule } from "./import-module.ts";
import { readJson } from "./support/json.ts";
+import { readOnlyParentSandboxState } from "./sandbox-state.ts";
import { finding as sourceFinding } from "./scan-draft-fixture.ts";
import type { RunArtifactWorkbench } from "../src/artifact-context.ts";
import type {
@@ -604,6 +605,21 @@ for (const claimed of [false, true]) {
);
assert.deepEqual(completed.checkpoints, interrupted.checkpoints);
assert.equal((await store.get(run.scanId, owner)).terminalReason, "capped");
+ const joined = await fixture.call(
+ "start_codex_security_deep_scan",
+ { scanId: run.scanId },
+ {
+ "codex/sandbox-state-meta": readOnlyParentSandboxState(run.targetPath),
+ },
+ );
+ assertSuccess(joined);
+ const receipt = joined.structuredContent as Record<string, unknown>;
+ assert.equal(receipt.completedAudits, 0);
+ assert.equal(receipt.terminalReason, "capped");
+ assert.match(
+ String(receipt.instructions),
+ /deadline.*before any Standard scan completed/i,
+ );
});
}
@@ -758,12 +774,13 @@ async function createFixture(t: TestContext, paths: FixturePaths = {}) {
async function call(
name: string,
args: Record<string, unknown> | ScanDraftInput,
+ meta = {},
) {
await (connected ??= client.connect(transport));
return client.callTool({
name,
arguments: { ...args },
- _meta: { "openai/threadId": owner },
+ _meta: { "openai/threadId": owner, ...meta },
});
}
return {
@@ -776,6 +793,7 @@ async function createFixture(t: TestContext, paths: FixturePaths = {}) {
now = new Date(Date.parse(instant) + offset * 1_000).toISOString();
return now;
},
+ call,
recordDraft: (draft: ScanDraftInput | Record<string, unknown>) =>
call("record_codex_security_scan_draft", draft),
completeScan: () =>
diff --git a/plugins/codex-security/mcp-app/tests/test_deep_scan_stdio_lifecycle.ts b/plugins/codex-security/mcp-app/tests/test_deep_scan_stdio_lifecycle.ts
index b7871f8b0..9d42586a3 100644
--- a/plugins/codex-security/mcp-app/tests/test_deep_scan_stdio_lifecycle.ts
+++ b/plugins/codex-security/mcp-app/tests/test_deep_scan_stdio_lifecycle.ts
@@ -950,12 +950,26 @@ service_tier = "ultrafast"
/Do not call complete_codex_security_scan again/,
);
const completion = resumed.structuredContent.completion;
+ const sealed = await runWorkbench(environment, [
+ "get-scan",
+ "--scan-id",
+ resumedScanId,
+ ]);
+ const { accountRateLimits: _limits, ...usage } = sealed.scan.usage;
+ assert.equal(usage.coverage, "unavailable");
+ assert.ok(usage.warnings.length);
assert.deepEqual(completion, {
- scan: { progress: { status: "complete" }, reportAvailable: true },
+ scan: {
+ progress: { status: "complete" },
+ reportAvailable: true,
+ usage,
+ },
});
assert.deepEqual(resumed.structuredContent, {
scanId: resumedScanId,
scanDir: resumedScan.scanDir,
+ completedAudits: 2,
+ terminalReason: "saturated",
manifestPath,
reportPath: path.join(resumedScan.scanDir, "report.md"),
instructions,
diff --git a/plugins/codex-security/scripts/deep_scan_workbench.py b/plugins/codex-security/scripts/deep_scan_workbench.py
index a73b8ac96..e2fb2e18d 100644
--- a/plugins/codex-security/scripts/deep_scan_workbench.py
+++ b/plugins/codex-security/scripts/deep_scan_workbench.py
@@ -382,21 +382,20 @@ def independent_review_progress(
).fetchone()
if run is None:
return None
- active = connection.execute(
+ active, buffered = connection.execute(
"""
- SELECT COUNT(*)
+ SELECT COUNT(*) FILTER (WHERE status IN ('queued', 'running')),
+ COUNT(*) FILTER (WHERE status = 'succeeded' AND merge_state = 'buffered')
FROM deep_scan_workers
- WHERE scan_id = ?
- AND kind = 'discovery'
- AND status IN ('queued', 'running')
+ WHERE scan_id = ? AND kind = 'discovery'
""",
(scan_id,),
- ).fetchone()[0]
+ ).fetchone()
return {
"active": int(active),
"completed": int(run["completion_sequence"]),
"maximum": int(run["max_discovery_runs"]),
- "consolidating": run["phase"] in {"reducing", "terminal"},
+ "consolidating": run["phase"] in {"reducing", "terminal"} or (active == 0 and buffered > 0),
"updatedAt": str(run["updated_at"]),
}
diff --git a/plugins/codex-security/skills/deep-security-scan/SKILL.md b/plugins/codex-security/skills/deep-security-scan/SKILL.md
index 6d4daecbe..36a923dec 100644
--- a/plugins/codex-security/skills/deep-security-scan/SKILL.md
+++ b/plugins/codex-security/skills/deep-security-scan/SKILL.md
@@ -7,8 +7,8 @@ description: Run repeated Standard security scans of a repository or scoped path
The host runs Standard audits and merges their findings. Workers in the same scan can exchange investigation leads and verify each other’s evidence. Later passes receive the titles and locations already reported. The host stops when consecutive audits add nothing new or the configured scan or time limit is reached. Its final report includes every accepted audit.
-1. Resolve the authorized target and scope. For a scoped directory use that directory as `targetPath` and `scope: "."`. For a continuation, call `get_codex_security_scan_context` with its `scanId` and supplied `handoffClaimToken`. Use the saved identity, scope and mode. Follow `../../references/scan-prologue.md` for source access and context updates.
+1. Resolve the authorized target and scope. For a scoped directory use that directory as `targetPath` and `scope: "."`. For a continuation, call `get_codex_security_scan_context` with its `scanId` and supplied `handoffClaimToken`. Use its saved identity and scope. If its mode is Standard or diff, continue with the corresponding scan skill. Follow `../../references/scan-prologue.md` for source access and context updates.
2. For a new target, call `inspect_codex_security_setup({ targetPath, scope: ".", mode: "deep", projectConfigPath? })`. Show the effective settings and their sources. `max_discovery_runs` caps the total number of Standard scans, including when several run together. Pass a project file only when the user selected it. Existing scans use their saved configuration.
3. For a newly launched native scan, inspect `otherRunningDeepScans` once. If another Deep scan is active, show its target, phase and human-friendly start time. Explain the additional CPU, memory and token use. Offer **Cancel (Recommended)** and **Continue** using `request_user_input`, then `request_codex_security_user_input` or plain chat if unavailable. Wait for an explicit choice. A dismissed dialog is not a cancellation. Cancel only the new scan if requested.
4. Follow `../../references/access-advisory.md`, then call `start_codex_security_deep_scan` with the existing scan ID and claim token, or with the new target, scope, exact user context and selected project file. Await the tool. If the tool runner returns a running cell ID, wait on that cell until it returns the scan result. If the connection closes, rejoin with the same arguments. Report a terminal failure with its saved results. For a policy refusal, identify the worker from its saved error when available. On explicit cancellation call `cancel_codex_security_scan`.
-5. For an SDK-owned scan, return the host's scan ID and manifest path so the SDK can finalize it. When the tool returns `completion`, link its generated `report.md` and canonical artifacts and report measured usage per `../../references/final-report.md`. Distinguish a completed audit with zero findings from a deadline reached before any audit completed.
+5. For an SDK-owned scan, return the host's scan ID and manifest path so the SDK can finalize it. When the tool returns `completion`, link its generated `report.md` and canonical artifacts. Use `completion.scan.usage` and any separate cost estimate to report usage per `../../references/final-report.md`. Distinguish a completed audit with zero findings from a deadline reached before any audit completed.
diff --git a/plugins/codex-security/tests/test_workbench_deep_scan.py b/plugins/codex-security/tests/test_workbench_deep_scan.py
index 28ea88db9..9d87ab61c 100644
--- a/plugins/codex-security/tests/test_workbench_deep_scan.py
+++ b/plugins/codex-security/tests/test_workbench_deep_scan.py
@@ -643,7 +643,7 @@ def test_paused_discovery_resumes_after_restart_with_original_handoff_claim(
"completed": 1,
"active": 0,
"maximum": 2,
- "consolidating": False,
+ "consolidating": True,
}
assert (paused["reportAvailable"], paused["findingCount"], paused["artifacts"]) == (
False,
@@ -1102,8 +1102,12 @@ def test_scan_progress_projects_active_and_completed_independent_reviews(
"active": 0,
"completed": 1,
"maximum": 40,
- "consolidating": False,
+ "consolidating": True,
}
+ deep.record_iteration(
+ name="dedup-0001", input_worker_ids=[first_worker_id], new_findings_count=0
+ )
+ assert independent_reviews()["consolidating"] is False
@pytest.mark.parametrize(
diff --git a/sdk/typescript/README.md b/sdk/typescript/README.md
index 9208ec4a1..2fa7af84f 100644
--- a/sdk/typescript/README.md
+++ b/sdk/typescript/README.md
@@ -4,8 +4,8 @@ Run Codex Security from TypeScript or the command line to find, validate, and fi
security vulnerabilities in your code. The package includes the Codex runtime,
security plugin, and TypeScript declarations. It uses ES modules.
-- Scan repositories, selected paths, or Git changes. Deep scans run parallel
- discovery workers on repositories and selected paths.
+- Scan repositories, selected paths, or Git changes. Deep scans repeat Standard
+ scans of the selected scope and merge their findings after each batch.
- Validate findings, generate patches, and verify existing fixes.
- Draft security policies and retain threat models.
- Review saved findings, identify duplicates, classify severity, and suggest owners.
@@ -392,9 +392,9 @@ const result = await security.run("/path/to/repository", {
});
```
-`workers` controls concurrent discovery workers; `subagents` controls delegation
-within each worker. Time and run limits apply to discovery. After discovery
-stops, the scan combines and returns completed findings. See
+`workers` controls concurrent Standard scans and `subagents` controls delegation
+within each scan. Time and run limits apply to discovery. The coordinator merges
+completed findings before the next batch starts. See
[deep scan settings](docs/cli.md#configure-deep-scans) for defaults and saved settings.
Deep discovery retains its enforced deadline, with a default and maximum of
5760 minutes. `maxTimeHours` remains a deprecated Deep-only alias in hours.