Skip to content

OpenAI's Codex Security CLI and TypeScript SDK for finding, validating, and fixing security vulnerabilities. npm: https://www.npmjs.com/package/@openai/codex-security

fix(dedupe): preserve undecided findings after model refusals (#960)

Milan Pavlík8cc1dcfd5e2d
diff --git a/sdk/typescript/README.md b/sdk/typescript/README.md
index 28b74074c..b846992df 100644
--- a/sdk/typescript/README.md
+++ b/sdk/typescript/README.md
@@ -1964,7 +1964,7 @@ console.log(receipt.repositoryId, receipt.findingIds);
Publish the scan with `--to custom` (or import it through the bulk API with its
`repositoryId`) before deduplicating. The
workflow reads a completed saved scan, queries candidates by finding ID, and
-runs Luna and Sol in the calling SDK/CLI process. Once all reviews succeed,
+runs Luna and Sol in the calling SDK/CLI process. Once all reviews finish,
it posts accepted groups to the service. It does not re-upload findings or
change scan artifacts.
@@ -1985,7 +1985,14 @@ has finished and none voted `DISTINCT`. It can run while unrelated Luna
screenings continue. Results are combined in input order so completion timing
does not change the groups.
-If a job fails after its retries, queued jobs stop and already running jobs
+An explicit model refusal keeps the affected pairs separate and allows unrelated
+reviews to continue. It is recorded as `NO_DECISION`, not a reviewed `DISTINCT`
+verdict. A screening refusal applies to every anchor/candidate pair in that
+screening; a pair-review refusal applies only to its assigned pair. Neither pair
+can be merged indirectly through other findings. Recognized policy errors and
+explicit refusal responses are not retried or sent to another model.
+
+If another kind of job fails after its retries, queued jobs stop and already running jobs
finish before the command reports the failure. No groups are posted from an
incomplete review. To retain completed reviews across runs, use a
`--workflow-id` as described below.
@@ -2049,6 +2056,19 @@ has the highest reported severity; ties use finding ID. Results do not delete,
merge, or change stored finding documents. Accepted groups are saved as durable
associations in the service before `deduplicationStatus` becomes `completed`.
+When any review is refused, the result instead has
+`deduplicationStatus: "completed_with_refusals"` and a `refusals` array. Each
+entry contains `decision: "NO_DECISION"`, `stage`, `model`, `findingIds`, and
+`reason`. For screening, the first finding ID is the anchor and the remaining
+IDs are its candidates. The CLI logs each refusal to stderr and exits
+successfully after saving the accepted groups. SDK callers can inspect and log
+the same structured entries. Findings retained because of a refusal are not
+confirmed unique. Successful runs without refusals keep their existing output
+shape. No command, flag, or default changes are required.
+
+This behavior applies to post-scan Luna/Sol deduplication. Deep Scan's internal
+reducer uses a separate workflow.
+
### Stored duplicate groups
`POST /v1/dedupe-groups` accepts a batch of explicitly reviewed member sets:
@@ -2150,10 +2170,16 @@ exponential backoff with jitter; HTTP retries honor `Retry-After`. Waiting to re
occupies the job's concurrency slot.
Cancellation, authentication or configuration errors, permanent HTTP errors, and
-required-source-access blockers are not retried. Exhausted retries fail deduplication;
+required-source-access blockers are not retried. Model refusals continue with
+`NO_DECISION` as described above. Other exhausted retries fail deduplication;
invalid or unfinished reviews are not cached. Completed checkpoints remain
available when the workflow resumes.
+Refusals are not saved as validated review checkpoints. A completed workflow
+retains its final `completed_with_refusals` result, including all refusal entries,
+and returns it on subsequent calls. To attempt those reviews again after
+resolving the refusal, run dedupe without that workflow ID or with a new one.
+
Checkpoints bind to the exact original records and ordering, approved source path,
Git revision and current file contents (including ignored files), repository scope,
model and reasoning settings, Codex configuration and version, and prompt/contract version. Changed
diff --git a/sdk/typescript/scripts/check-package.mjs b/sdk/typescript/scripts/check-package.mjs
index 04d8c56e2..69963955a 100644
--- a/sdk/typescript/scripts/check-package.mjs
+++ b/sdk/typescript/scripts/check-package.mjs
@@ -225,6 +225,7 @@ const distFiles = new Set(
"server/api",
"deduplication/codex-review",
"deduplication/checkpointed-review",
+ "deduplication/refusal",
"deduplication/retry",
"deduplication/deduplication",
"finding-retrieval",
diff --git a/sdk/typescript/src/cli.ts b/sdk/typescript/src/cli.ts
index 4957f7cbe..496ecd068 100644
--- a/sdk/typescript/src/cli.ts
+++ b/sdk/typescript/src/cli.ts
@@ -3918,7 +3918,18 @@ export async function main(
scanId: z.string(),
uniqueFindingIds: z.array(z.string()),
duplicateGroups: z.array(z.array(z.string())),
- deduplicationStatus: z.literal("completed"),
+ deduplicationStatus: z.enum(["completed", "completed_with_refusals"]),
+ refusals: z
+ .array(
+ z.object({
+ decision: z.literal("NO_DECISION"),
+ stage: z.enum(["screening", "pair-review"]),
+ model: z.string(),
+ findingIds: z.array(z.string()),
+ reason: z.string(),
+ }),
+ )
+ .optional(),
})
.optional(),
async run({ options }) {
@@ -3938,7 +3949,7 @@ export async function main(
throw new CodexSecurityError(
"Deduplication requires --scan or --workflow-id.",
);
- return await (
+ const result = await (
dependencies.deduplicateScan ?? deduplicateScanInternal
)(
scanId,
@@ -3957,6 +3968,16 @@ export async function main(
runWorkbench: dependencies.runWorkbench,
},
);
+ for (const refusal of result.refusals ?? []) {
+ try {
+ errorOutput.write(
+ `codex-security: ${refusal.stage} refused by ${refusal.model} for ${refusal.findingIds.join(", ")}: ${refusal.reason} No decision was made; affected pairs were kept separate.\n`,
+ );
+ } catch {
+ // Optional diagnostics must not discard the completed result.
+ }
+ }
+ return result;
} catch (error) {
const signal = controller.signal.reason;
errorOutput.write(
diff --git a/sdk/typescript/src/deduplication/codex-review.ts b/sdk/typescript/src/deduplication/codex-review.ts
index 8409b7fbe..4e3295b31 100644
--- a/sdk/typescript/src/deduplication/codex-review.ts
+++ b/sdk/typescript/src/deduplication/codex-review.ts
@@ -40,6 +40,7 @@ import {
sourceReviewInstructions,
} from "./deduplication-prompts.js";
import { retryDelay, waitForRetry } from "./retry.js";
+import { isReviewRefusal } from "./refusal.js";
const reviewErrorSchema = z
.object({ reason: z.string().trim().min(1) })
@@ -93,6 +94,7 @@ interface Message {
tool?: string;
namespace?: string | null;
arguments?: unknown;
+ item?: { type: string; text?: string };
};
}
@@ -346,12 +348,14 @@ export class CodexReviewRunner {
let turnId: string | undefined;
let accepted: T | undefined;
let validationFailure: string | undefined;
+ let finalResponse: string | undefined;
let turns = 0;
const startTurn = (prompt: string) => {
this.signal?.throwIfAborted();
turns++;
turnId = undefined;
validationFailure = undefined;
+ finalResponse = undefined;
send({
id: 3 + state.attempts,
method: "turn/start",
@@ -433,9 +437,11 @@ export class CodexReviewRunner {
});
if (reportedFailure !== undefined)
throw new ReviewAttemptError(
- "model",
+ isReviewRefusal(reportedFailure) ? "refusal" : "model",
`Required review check could not be completed: ${reportedFailure}`,
- "A required review check could not be completed.",
+ isReviewRefusal(reportedFailure)
+ ? "The model refused the deduplication review."
+ : "A required review check could not be completed.",
);
} else {
send({
@@ -444,11 +450,18 @@ export class CodexReviewRunner {
});
}
} else if (message.error !== undefined) {
+ const refused = isReviewRefusal(
+ message.error.message,
+ message.error.data?.codexErrorInfo,
+ );
throw new ReviewAttemptError(
- "transport",
+ refused ? "refusal" : "transport",
message.error?.message ?? "Codex rejected the review request",
- "Codex rejected the review request.",
- transientCodexError(message.error.data?.codexErrorInfo),
+ refused
+ ? "The model refused the deduplication review."
+ : "Codex rejected the review request.",
+ !refused &&
+ transientCodexError(message.error.data?.codexErrorInfo),
);
} else if (message.id === 1) {
send({ method: "initialized" });
@@ -478,6 +491,13 @@ export class CodexReviewRunner {
params.threadId === threadId
) {
turnId = params?.turn?.id;
+ } else if (
+ message.method === "item/completed" &&
+ params?.threadId === threadId &&
+ params?.turnId === turnId &&
+ params?.item?.type === "agentMessage"
+ ) {
+ finalResponse = params.item.text;
} else if (
message.method === "turn/completed" &&
params !== undefined &&
@@ -487,15 +507,30 @@ export class CodexReviewRunner {
params.turn?.id === turnId
) {
if (params.turn.status !== "completed") {
- throw new ReviewAttemptError(
- "model",
+ const reason =
params.turn.error?.message ??
- `Codex review turn ${params.turn.status}`,
- "Codex review turn failed.",
- transientCodexError(params.turn.error?.codexErrorInfo),
+ `Codex review turn ${params.turn.status}`;
+ const refused = isReviewRefusal(
+ reason,
+ params.turn.error?.codexErrorInfo,
+ );
+ throw new ReviewAttemptError(
+ refused ? "refusal" : "model",
+ reason,
+ refused
+ ? "The model refused the deduplication review."
+ : "Codex review turn failed.",
+ !refused &&
+ transientCodexError(params.turn.error?.codexErrorInfo),
);
}
if (accepted === undefined) {
+ if (finalResponse && isReviewRefusal(finalResponse))
+ throw new ReviewAttemptError(
+ "refusal",
+ finalResponse,
+ "The model refused the deduplication review.",
+ );
if (turns === 1) {
state.attempts++;
startTurn(
diff --git a/sdk/typescript/src/deduplication/deduplication.ts b/sdk/typescript/src/deduplication/deduplication.ts
index 13ada8599..da952e4ea 100644
--- a/sdk/typescript/src/deduplication/deduplication.ts
+++ b/sdk/typescript/src/deduplication/deduplication.ts
@@ -1,10 +1,15 @@
import type { Finding } from "../models.js";
import type { FindingNeighborhood } from "../finding-retrieval.js";
-import { CodexSecurityError } from "../errors.js";
+import {
+ CodexSecurityError,
+ DeduplicationReviewError,
+ type DeduplicationReviewStage,
+} from "../errors.js";
import {
pairKey,
screeningPairSlot,
type DeduplicationReviewer,
+ type ScreeningResult,
} from "./deduplication-reviewer.js";
export const DEFAULT_DEDUPE_CONCURRENCY = 8;
@@ -61,10 +66,21 @@ async function runQueued(
if (failure !== undefined) throw failure.error;
}
+export interface DeduplicationRefusal {
+ decision: "NO_DECISION";
+ stage: DeduplicationReviewStage;
+ model: string;
+ /** For screening, the first finding is the anchor and the rest are its candidates. */
+ findingIds: string[];
+ reason: string;
+}
+
export interface DeduplicationResult {
uniqueFindingIds: string[];
duplicateGroups: string[][];
- deduplicationStatus: "completed";
+ deduplicationStatus: "completed" | "completed_with_refusals";
+ /** Refused reviews are kept separate without claiming a DISTINCT verdict. */
+ refusals?: DeduplicationRefusal[];
}
const severityOrder: Record<Finding["severity"]["level"], number> = {
@@ -242,6 +258,23 @@ export class FindingDeduplicator {
const concurrency = deduplicationConcurrency(this.concurrency);
const ids = [...new Set(findingIds)];
const findings = new Map<string, Finding>();
+ const refusals = new Map<string, DeduplicationRefusal>();
+ const recordRefusal = (error: unknown, ids: string[]): void => {
+ this.signal?.throwIfAborted();
+ if (
+ !(error instanceof DeduplicationReviewError) ||
+ error.metadata.category !== "refusal"
+ )
+ throw error;
+ const { stage, model, reason } = error.metadata;
+ refusals.set(JSON.stringify([stage, ...ids]), {
+ decision: "NO_DECISION",
+ stage,
+ model,
+ findingIds: ids,
+ reason,
+ });
+ };
const neighborhoods = new Array<Finding[]>(ids.length);
await runQueued(
ids.map((id, index) => async () => {
@@ -288,7 +321,15 @@ export class FindingDeduplicator {
const pending = neighborhoods
.filter((neighborhood) => neighborhood.length > 1)
.map((neighborhood) => async () => {
- const screening = await this.reviewer.screen(neighborhood);
+ let screening: ScreeningResult | undefined;
+ try {
+ screening = await this.reviewer.screen(neighborhood);
+ } catch (error) {
+ recordRefusal(
+ error,
+ neighborhood.map((finding) => finding.findingId),
+ );
+ }
this.signal?.throwIfAborted();
for (let index = 0; index < neighborhood.length - 1; index++) {
const key = pairKey([
@@ -297,20 +338,25 @@ export class FindingDeduplicator {
]);
const state = pairs.get(key)!;
if (
+ screening === undefined ||
screening.decisions[screeningPairSlot(index)]!.decision ===
- "DISTINCT"
+ "DISTINCT"
)
state.rejected = true;
state.remaining--;
- // Wait for every screening of this pair: a later DISTINCT veto must
+ // Wait for every screening: a later DISTINCT verdict or refusal must
// prevent verification, including a verification that could fail.
if (state.remaining === 0 && !state.rejected) {
ready.push(async () => {
- state.decision = (
- await this.reviewer.reviewPair(
- state.ids.map((id) => findings.get(id)!),
- )
- ).decision;
+ try {
+ state.decision = (
+ await this.reviewer.reviewPair(
+ state.ids.map((id) => findings.get(id)!),
+ )
+ ).decision;
+ } catch (error) {
+ recordRefusal(error, state.ids);
+ }
});
}
}
@@ -400,7 +446,17 @@ export class FindingDeduplicator {
return {
uniqueFindingIds: [...new Set(ids.map((id) => canonical.get(id) ?? id))],
duplicateGroups,
- deduplicationStatus: "completed",
+ deduplicationStatus:
+ refusals.size > 0 ? "completed_with_refusals" : "completed",
+ ...(refusals.size > 0
+ ? {
+ refusals: [...refusals.entries()]
+ .sort(([left], [right]) =>
+ left < right ? -1 : left > right ? 1 : 0,
+ )
+ .map(([, refusal]) => refusal),
+ }
+ : {}),
};
}
}
diff --git a/sdk/typescript/src/deduplication/refusal.ts b/sdk/typescript/src/deduplication/refusal.ts
new file mode 100644
index 000000000..f1560a343
--- /dev/null
+++ b/sdk/typescript/src/deduplication/refusal.ts
@@ -0,0 +1,20 @@
+/** Recognize policy blocks and explicit refusal responses, not ordinary review failures. */
+export function isReviewRefusal(
+ message: string,
+ codexErrorInfo?: unknown,
+): boolean {
+ if (
+ codexErrorInfo === "cyberPolicy" ||
+ codexErrorInfo === "misalignmentPolicyViolation"
+ )
+ return true;
+ return [
+ /\bflagged for possible cybersecurity risk\b/iu,
+ /\bflagged for potentially high-risk cyber activity\b/iu,
+ /\bcyber[_\s-]?policy\b/iu,
+ /\b(?:cybersecurity|cyber|content|safety)[ _-]*policy[ _-]*(?:violation|refusal|refused)\b/iu,
+ /\b(?:refusal|refused)\b[^\n]*\b(?:cybersecurity|cyber|safety policy)\b/iu,
+ /\b(?:cybersecurity|cyber|safety policy)\b[^\n]*\b(?:refusal|refused)\b/iu,
+ /^(?:I(?:['’]m| am) sorry[,.:]?\s*(?:but\s+)?|Sorry[,.:]?\s*)?I(?:\s+(?:cannot|can['’]t|won['’]t|am unable to)|['’]m unable to)\s+(?:help|assist|comply)\b/iu,
+ ].some((pattern) => pattern.test(message.trim()));
+}
diff --git a/sdk/typescript/src/errors.ts b/sdk/typescript/src/errors.ts
index 7125bed35..2b59b192d 100644
--- a/sdk/typescript/src/errors.ts
+++ b/sdk/typescript/src/errors.ts
@@ -33,7 +33,7 @@ export class CodexSecurityError extends Error {
export type DeduplicationReviewStage = "screening" | "pair-review";
export type DeduplicationReviewFailureCategory =
- "validation" | "no-submission" | "model" | "transport";
+ "validation" | "no-submission" | "model" | "transport" | "refusal";
export interface DeduplicationReviewFailureMetadata {
stage: DeduplicationReviewStage;
diff --git a/sdk/typescript/src/index.ts b/sdk/typescript/src/index.ts
index c62ad02c6..e1d3cc42a 100644
--- a/sdk/typescript/src/index.ts
+++ b/sdk/typescript/src/index.ts
@@ -129,6 +129,7 @@ export type {
DeduplicateScanOptions,
DeduplicateScanResult,
} from "./deduplication/scan.js";
+export type { DeduplicationRefusal } from "./deduplication/deduplication.js";
export { importGitHubCodeScanningAlerts } from "./github.js";
export type {
GitHubCodeScanningImportOptions,
diff --git a/sdk/typescript/tests-ts/cli-dedupe.test.ts b/sdk/typescript/tests-ts/cli-dedupe.test.ts
index 4e2717f9d..8832e2bad 100644
--- a/sdk/typescript/tests-ts/cli-dedupe.test.ts
+++ b/sdk/typescript/tests-ts/cli-dedupe.test.ts
@@ -11,6 +11,43 @@ const args = [
"--json",
];
+test.each([false, true])(
+ "dedupe reports refusals and succeeds even when diagnostic output fails: %j",
+ async (brokenLog) => {
+ const deps = dependencies();
+ const result = {
+ scanId: "scan-example",
+ uniqueFindingIds: ["finding-one", "finding-two"],
+ duplicateGroups: [],
+ deduplicationStatus: "completed_with_refusals" as const,
+ refusals: [
+ {
+ decision: "NO_DECISION" as const,
+ stage: "pair-review" as const,
+ model: "gpt-5.6-sol",
+ findingIds: ["finding-one", "finding-two"],
+ reason: "The model refused the deduplication review.",
+ },
+ ],
+ };
+ deps.deduplicateScan = async () => result;
+ const stdout = capture();
+ const stderr = capture();
+ if (brokenLog)
+ stderr.stream.write = () => {
+ throw new Error("Synthetic logging failure");
+ };
+ expect(await main(args, stdout.stream, stderr.stream, deps)).toBe(0);
+ expect(JSON.parse(stdout.text())).toEqual(result);
+ if (!brokenLog) {
+ expect(stderr.text()).toContain("pair-review refused by gpt-5.6-sol");
+ expect(stderr.text()).toContain("finding-one, finding-two");
+ expect(stderr.text()).toContain("No decision was made");
+ expect(stderr.text()).toContain("kept separate");
+ }
+ },
+);
+
test("dedupe resolves a workflow's pinned scan and passes the workflow ID to the SDK", async () => {
const deps = dependencies();
deps.runWorkbench = async (args, input) => {
diff --git a/sdk/typescript/tests-ts/codex-review.test.ts b/sdk/typescript/tests-ts/codex-review.test.ts
index a51e5cbc4..93b74aa93 100644
--- a/sdk/typescript/tests-ts/codex-review.test.ts
+++ b/sdk/typescript/tests-ts/codex-review.test.ts
@@ -21,12 +21,20 @@ import { resolveCodexCommand } from "../src/runtime.js";
import { environmentEntry } from "../src/scan-comparison.js";
import { runTestInSubprocess } from "./support/test-subprocess.js";
import { retryDelay, waitForRetry } from "../src/deduplication/retry.js";
+import { isReviewRefusal } from "../src/deduplication/refusal.js";
const fixture = fileURLToPath(
new URL("fixtures/codex-review.mjs", import.meta.url),
);
const failureReasons: Record<string, string> = {
+ "policy-turn-code": "Request blocked.",
+ "policy-request-code": "Request blocked.",
+ "policy-turn": "Request flagged for possible cybersecurity risk.",
+ "policy-request": "Request rejected: cyber_policy.",
+ "refusal-text": "I'm sorry, but I can't assist with that request.",
+ "policy-reported-error":
+ "Required review check could not be completed: Request refused due to cybersecurity policy violation.",
"text-only": "Codex did not submit a validated review",
"failed-turn": "Rate limit exceeded",
"server-error": "Provider temporarily unavailable",
@@ -67,6 +75,8 @@ const recoveredScenarios: Record<string, string> = {
"recover-no-submission": "text-only",
};
const modelFailures = new Set([
+ "policy-turn-code",
+ "policy-turn",
"failed-turn",
"server-error",
"connection-error",
@@ -257,15 +267,23 @@ for (const {
});
const reportsBlocker =
scenario.startsWith("required-source-error") ||
+ scenario === "policy-reported-error" ||
scenario === "invalid-review-error";
- const reviewRunner = reportsBlocker
- ? new CheckpointedReviewRunner(
- new FindingWorkflow("blocked-review", process.env, checkpoints.run),
- runner,
- checkpoints.source,
- { allRepositories: true },
- )
- : runner;
+ const refused =
+ scenario.startsWith("policy-") || scenario === "refusal-text";
+ const reviewRunner =
+ reportsBlocker || refused
+ ? new CheckpointedReviewRunner(
+ new FindingWorkflow(
+ "blocked-review",
+ process.env,
+ checkpoints.run,
+ ),
+ runner,
+ checkpoints.source,
+ { allRepositories: true },
+ )
+ : runner;
const result = reviewRunner.run({
stage: "pair-review",
model: "gpt-5.6-sol",
@@ -342,8 +360,9 @@ for (const {
expect(reviewFailure.metadata).toEqual({
stage: "pair-review",
model: "gpt-5.6-sol",
- category:
- scenario === "invalid-submission"
+ category: refused
+ ? "refusal"
+ : scenario === "invalid-submission"
? "validation"
: scenario === "text-only"
? "no-submission"
@@ -358,8 +377,9 @@ for (const {
].includes(scenario)
? 2
: 1) * sessions,
- reason:
- scenario === "credential-error"
+ reason: refused
+ ? "The model refused the deduplication review."
+ : scenario === "credential-error"
? "[redacted]"
: scenario === "invalid-submission"
? "The submitted review failed semantic validation."
@@ -385,7 +405,8 @@ for (const {
? sessions
: 0,
);
- if (reportsBlocker) expect(checkpoints.saved).toHaveLength(0);
+ if (reportsBlocker || refused)
+ expect(checkpoints.saved).toHaveLength(0);
}
expect(starts).toBe(sessions);
expect(delays).toEqual(
@@ -453,7 +474,12 @@ for (const {
"required-source-error-after-text",
].includes(scenario)
? 2
- : ["request-error", "credential-error"].includes(scenario)
+ : [
+ "request-error",
+ "credential-error",
+ "policy-request",
+ "policy-request-code",
+ ].includes(scenario)
? 0
: 1) * sessions,
);
@@ -476,6 +502,37 @@ for (const {
});
}
+test.each([
+ "cyber_policy",
+ "Request blocked by a safety policy violation.",
+ "Request flagged for potentially high-risk cyber activity.",
+ "I cannot help with this request.",
+ "I’m sorry, but I can’t assist with your request.",
+ "I can't help analyze or merge these vulnerability reports.",
+ "I'm unable to assist with this comparison.",
+])("recognizes explicit review refusals: %s", (message) => {
+ expect(isReviewRefusal(message)).toBe(true);
+});
+
+test.each(["cyberPolicy", "misalignmentPolicyViolation"])(
+ "recognizes structured policy refusal codes independently of message wording: %s",
+ (code) => {
+ expect(isReviewRefusal("Request blocked.", code)).toBe(true);
+ expect(isReviewRefusal("Request blocked.", "unauthorized")).toBe(false);
+ },
+);
+
+test.each([
+ "Rate limit exceeded",
+ "Authentication required",
+ "Required source revision could not be read.",
+ "I cannot complete the review because the source is unavailable.",
+ "Connection refused",
+ "Here is the review JSON.",
+])("does not turn other failures into refused reviews: %s", (message) => {
+ expect(isReviewRefusal(message)).toBe(false);
+});
+
test("empty credential paths use default directories without denying cwd", async () => {
if (
runTestInSubprocess(
diff --git a/sdk/typescript/tests-ts/finding-deduplication.test.ts b/sdk/typescript/tests-ts/finding-deduplication.test.ts
index 6652a739c..9a5d958aa 100644
--- a/sdk/typescript/tests-ts/finding-deduplication.test.ts
+++ b/sdk/typescript/tests-ts/finding-deduplication.test.ts
@@ -19,7 +19,7 @@ import {
type DuplicateDecision,
type ScreeningResult,
} from "../src/deduplication/deduplication-reviewer.js";
-import { CodexSecurityError } from "../src/errors.js";
+import { CodexSecurityError, DeduplicationReviewError } from "../src/errors.js";
import { FindingsClient } from "../src/findings-client.js";
import { deduplicateScanDirectory } from "../src/index.js";
import {
@@ -870,6 +870,66 @@ test("empty and isolated imports avoid models, while review failures propagate",
await expect(service.run([first.findingId])).rejects.toBe(failure);
});
+test.each(["screening", "pair-review"] as const)(
+ "a %s refusal keeps affected pairs separate without stopping other reviews",
+ async (stage) => {
+ const findings = [entry(1), entry(2), entry(3)];
+ const ids = findings.map((finding) => finding.findingId);
+ const refusedPair = pairKey([ids[0]!, ids[2]!]);
+ const nominations = new Set([
+ pairKey(ids.slice(0, 2)),
+ pairKey(ids.slice(1)),
+ refusedPair,
+ ]);
+ const failure = new DeduplicationReviewError({
+ stage,
+ model: stage === "screening" ? "gpt-5.6-luna" : "gpt-5.6-sol",
+ category: "refusal",
+ attempts: 1,
+ reason: "The model refused the deduplication review.",
+ });
+ const reviewed: string[] = [];
+ const result = await new FindingDeduplicator(candidates(findings), {
+ async screen(values) {
+ if (stage === "screening" && values[0]!.findingId === ids[0])
+ throw failure;
+ return screening(values, nominations);
+ },
+ async reviewPair(values) {
+ const key = pairKey(values.map((finding) => finding.findingId));
+ reviewed.push(key);
+ if (stage === "pair-review" && key === refusedPair) throw failure;
+ return same(values);
+ },
+ }).run(ids);
+
+ expect(result.deduplicationStatus).toBe("completed_with_refusals");
+ expect(result.refusals).toEqual([
+ {
+ decision: "NO_DECISION",
+ stage,
+ model: failure.metadata.model,
+ reason: failure.metadata.reason,
+ findingIds: stage === "screening" ? ids : [ids[2]!, ids[0]!],
+ },
+ ]);
+ expect(result.duplicateGroups.some((group) => group.length === 2)).toBe(
+ true,
+ );
+ expect(result.uniqueFindingIds).toHaveLength(2);
+ // A refused pair must not be reconnected through a third finding.
+ expect(
+ result.duplicateGroups.some(
+ (group) => group.includes(ids[0]!) && group.includes(ids[2]!),
+ ),
+ ).toBe(false);
+ if (stage === "screening") {
+ expect(result.uniqueFindingIds).toContain(ids[0]!);
+ expect(reviewed).toEqual([pairKey(ids.slice(1))]);
+ }
+ },
+);
+
test("validates exact screening slots without model-owned finding identity", () => {
const findings = [entry(1), entry(2), entry(3)];
const ids = findings.map((finding) => finding.findingId);
@@ -1316,7 +1376,7 @@ test("writes accepted groups only after all reviews and fails on review or write
if (process.platform !== "win32") await chmod(directory, 0o700);
const findings = [document.findings[0]!, entry(2), entry(3)];
const ids = findings.map((finding) => finding.findingId);
- for (const failure of ["none", "write", "review"]) {
+ for (const failure of ["none", "write", "review", "refusal"]) {
const phases: string[] = [];
const controller = new AbortController();
const result = deduplicateScanInternal(
@@ -1341,7 +1401,11 @@ test("writes accepted groups only after all reviews and fails on review or write
"http://synthetic.test/api/v1/dedupe-groups",
);
expect(JSON.parse(options.body as string)).toEqual({
- groups: [[...ids].sort()],
+ groups: [
+ failure === "refusal"
+ ? ids.slice(0, 2).sort()
+ : [...ids].sort(),
+ ],
});
return Response.json([], {
status: failure === "write" ? 409 : 201,
@@ -1367,6 +1431,14 @@ test("writes accepted groups only after all reviews and fails on review or write
},
async reviewPair(values) {
phases.push("pair");
+ if (failure === "refusal" && values[1]!.findingId === ids[2])
+ throw new DeduplicationReviewError({
+ stage: "pair-review",
+ model: "gpt-5.6-sol",
+ category: "refusal",
+ attempts: 1,
+ reason: "The model refused the deduplication review.",
+ });
if (failure === "review" && values[1]!.findingId === ids[2])
throw new CodexSecurityError(
"Required source revision could not be read.",
@@ -1378,6 +1450,18 @@ test("writes accepted groups only after all reviews and fails on review or write
);
if (failure === "none") {
expect((await result).duplicateGroups).toEqual([[...ids].sort()]);
+ } else if (failure === "refusal") {
+ expect(await result).toMatchObject({
+ deduplicationStatus: "completed_with_refusals",
+ duplicateGroups: [ids.slice(0, 2).sort()],
+ refusals: [
+ {
+ decision: "NO_DECISION",
+ stage: "pair-review",
+ findingIds: [ids[0], ids[2]],
+ },
+ ],
+ });
} else if (failure === "review") {
await expect(result).rejects.toThrow(
"Required source revision could not be read.",
diff --git a/sdk/typescript/tests-ts/finding-workflow-integration.test.ts b/sdk/typescript/tests-ts/finding-workflow-integration.test.ts
index d57fedbb1..e9732e19a 100644
--- a/sdk/typescript/tests-ts/finding-workflow-integration.test.ts
+++ b/sdk/typescript/tests-ts/finding-workflow-integration.test.ts
@@ -22,6 +22,7 @@ import {
type DuplicateDecision,
} from "../src/deduplication/deduplication-reviewer.js";
import { PLUGIN_ROOT } from "./plugin-root.js";
+import { DeduplicationReviewError } from "../src/errors.js";
import { workflowFixture } from "./support/workflow-fixture.js";
const fixtures: Array<Awaited<ReturnType<typeof workflowFixture>>> = [];
@@ -92,6 +93,80 @@ async function fixtureWithFindings(count: number) {
return value;
}
+test("completed workflows preserve refusal outcomes without caching a false verdict", async () => {
+ const { environment, document, history } = await fixtureWithFindings(2);
+ const options = {
+ workflowId: "refused-review",
+ findingsUrl: "http://synthetic.test",
+ };
+ let reviews = 0;
+ const savedStages: string[] = [];
+ const dependencies = {
+ environment,
+ runWorkbench: async (args: readonly string[], input?: string) => {
+ const payload = input ? JSON.parse(input) : {};
+ if (payload.action === "save-review")
+ savedStages.push(payload.binding.stage);
+ return await history(args, input);
+ },
+ reviewRunner: {
+ async run<T>(review: CodexReview<T>): Promise<T> {
+ reviews++;
+ if (review.stage === "pair-review")
+ throw new DeduplicationReviewError({
+ stage: review.stage,
+ model: review.model,
+ category: "refusal",
+ attempts: 1,
+ reason: "The model refused the deduplication review.",
+ });
+ return review.validate({
+ decisions: {
+ "pair-1": {
+ decision: "SAME",
+ rationale: "One shared correction may cover both findings.",
+ },
+ },
+ });
+ },
+ },
+ fetch: async (url: URL) => {
+ if (url.pathname.endsWith("/bulk/findings"))
+ return Response.json(
+ document.findings.map((finding) => finding.findingId),
+ );
+ const id = decodeURIComponent(url.pathname.split("/").at(-2)!);
+ return Response.json({
+ finding: document.findings.find((finding) => finding.findingId === id),
+ potentialDuplicates: document.findings.filter(
+ (finding) => finding.findingId !== id,
+ ),
+ });
+ },
+ };
+ const result = await deduplicateScanInternal(
+ document.scanId,
+ options,
+ dependencies,
+ );
+ expect(result).toMatchObject({
+ deduplicationStatus: "completed_with_refusals",
+ duplicateGroups: [],
+ uniqueFindingIds: document.findings.map((finding) => finding.findingId),
+ refusals: [{ decision: "NO_DECISION", stage: "pair-review" }],
+ });
+ expect(reviews).toBe(3);
+ expect(savedStages).toEqual(["screening", "screening"]);
+ expect(
+ (await new FindingWorkflow(options.workflowId, environment).get())?.stages
+ .dedupe,
+ ).toEqual({ status: "completed", result });
+ expect(
+ await deduplicateScanInternal(document.scanId, options, dependencies),
+ ).toEqual(result);
+ expect(reviews).toBe(3);
+});
+
async function restoreLegacyWorkflow(
environment: NodeJS.ProcessEnv,
state: object,
diff --git a/sdk/typescript/tests-ts/fixtures/codex-review.mjs b/sdk/typescript/tests-ts/fixtures/codex-review.mjs
index e8032f879..e21fb4508 100644
--- a/sdk/typescript/tests-ts/fixtures/codex-review.mjs
+++ b/sdk/typescript/tests-ts/fixtures/codex-review.mjs
@@ -4,6 +4,14 @@ import { createInterface } from "node:readline";
const [scenario, transcript, checkout] = process.argv.slice(2);
const turnFailures = {
+ "policy-turn-code": {
+ message: "Request blocked.",
+ codexErrorInfo: "cyberPolicy",
+ },
+ "policy-turn": {
+ message: "Request flagged for possible cybersecurity risk.",
+ codexErrorInfo: { httpConnectionFailed: { httpStatusCode: 503 } },
+ },
"failed-turn": {
message: "Rate limit exceeded",
codexErrorInfo: "usageLimitExceeded",
@@ -65,16 +73,30 @@ for await (const line of createInterface({ input: process.stdin })) {
assert.equal(message.params.apiKey, "synthetic-review-key");
send({ id: message.id, result: { type: "apiKey" } });
} else if (message.method === "thread/start") {
- if (["request-error", "credential-error"].includes(scenario)) {
+ if (
+ [
+ "request-error",
+ "credential-error",
+ "policy-request",
+ "policy-request-code",
+ ].includes(scenario)
+ ) {
send({
id: message.id,
error: {
code: -32000,
message:
- scenario === "credential-error"
- ? "Authentication failed: Bearer synthetic-review-key"
- : "Authentication required",
- data: "Synthetic private response data",
+ scenario === "policy-request-code"
+ ? "Request blocked."
+ : scenario === "policy-request"
+ ? "Request rejected: cyber_policy."
+ : scenario === "credential-error"
+ ? "Authentication failed: Bearer synthetic-review-key"
+ : "Authentication required",
+ data:
+ scenario === "policy-request-code"
+ ? { codexErrorInfo: "cyberPolicy" }
+ : "Synthetic private response data",
},
});
continue;
@@ -145,6 +167,7 @@ for await (const line of createInterface({ input: process.stdin })) {
submit("invalid", { decision: "UNKNOWN" });
} else if (
scenario === "text-only" ||
+ scenario === "refusal-text" ||
([
"text-only-correction",
"cancel-continuation",
@@ -157,7 +180,13 @@ for await (const line of createInterface({ input: process.stdin })) {
params: {
threadId: "review-thread",
turnId,
- item: { type: "agentMessage", text: '{"decision":"SAME"}' },
+ item: {
+ type: "agentMessage",
+ text:
+ scenario === "refusal-text"
+ ? "I'm sorry, but I can't assist with that request."
+ : '{"decision":"SAME"}',
+ },
},
});
complete();
@@ -175,12 +204,20 @@ for await (const line of createInterface({ input: process.stdin })) {
submit("invalid", { decision: "UNKNOWN" });
} else if (scenario === "invalid-review-error") {
submit("invalid-error", { reason: " " }, { tool: "submit_error" });
- } else if (scenario.startsWith("required-source-error")) {
+ } else if (
+ scenario.startsWith("required-source-error") ||
+ scenario === "policy-reported-error"
+ ) {
if (scenario === "required-source-error-after-verdict")
submit("pending-verdict", { decision: "SAME" });
submit(
"blocked",
- { reason: "Required source revision could not be read." },
+ {
+ reason:
+ scenario === "policy-reported-error"
+ ? "Request refused due to cybersecurity policy violation."
+ : "Required source revision could not be read.",
+ },
{ tool: "submit_error" },
);
} else if (scenario === "incomplete-content") {