Skip to content

OpenAI's Codex Security CLI and TypeScript SDK for finding, validating, and fixing security vulnerabilities. npm: https://www.npmjs.com/package/@openai/codex-security

fix(imports): preserve AWS code scanning evidence and retries (#2347)

Michael D'Angelo7bfb94d67e5e
diff --git a/docs/scanner-imports/README.md b/docs/scanner-imports/README.md
index d030cbebe..dc2b92673 100644
--- a/docs/scanner-imports/README.md
+++ b/docs/scanner-imports/README.md
@@ -35,11 +35,13 @@ Ask it to preview the report with your configured model.
Check your version with `codex-security --version`. Support for AWS Continuum,
Black Duck, SonarQube Cloud, Veracode, the Semgrep Platform SDK example, and
Cloud private inbox imports came after [version 0.8.2](https://github.com/openai/codex-security/blob/npm-v0.8.2/sdk/typescript/docs/cli.md#import-vendor-findings-locally).
+For AWS inbox imports, use a CLI and workspace that support v2 finding import
+jobs. The dry-run preview shows `privateImport` when that route is available.
| Guide | Supported input |
| ----------------------------------------------------------- | ------------------------------------------------------------------------------- |
| [Amazon Inspector](inspector.md) | Inspector2 `ListFindings` JSON for one Code Security repository |
-| [AWS Continuum](continuum.md) | Code finding summaries or a detailed finding verified for one repository |
+| [AWS Continuum](continuum.md) | Summaries or selected code finding details for one verified repository |
| [Black Duck SCA](blackduck.md) | BOM component vulnerabilities for one project version |
| [Checkmarx SAST](checkmarx.md) | Checkmarx One CLI JSON or self-managed CxSAST XML for one scan |
| [Endor Labs](endor.md) | Finding JSON from `endorctl`, or SARIF with finding UUIDs |
@@ -81,10 +83,10 @@ codex-security findings imported /path/to/project --format json
For a Cloud import, check the receipt for the destination shown in the preview.
-| What the Cloud preview shows | How to verify the import |
-| ------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
-| Private inbox (`privateImport`) | Check the job status, errors, exclusions, and unprocessed count. Open a finding in your inbox to assess it before sharing. |
-| Repository publisher (environments in `requests`) | Check each receipt, the verified and unacknowledged counts, failures, and exclusions. |
+| What the Cloud preview shows | How to verify the import |
+| ------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Private inbox (`privateImport`) | Check that `privateJob.status` is `completed` and its counts account for the selected findings, with zero errors or unprocessed rows. Review exclusions. |
+| Repository publisher (environments in `requests`) | Check each receipt, the verified and unacknowledged counts, failures, and exclusions. |
If `publish findings` stops and returns `savedSubmission`, [resume from that checkpoint](schema-guide.md#retry-a-cloud-submission).
diff --git a/docs/scanner-imports/continuum.md b/docs/scanner-imports/continuum.md
index 659c0088a..ce84e2d3d 100644
--- a/docs/scanner-imports/continuum.md
+++ b/docs/scanner-imports/continuum.md
@@ -1,9 +1,8 @@
# Import AWS Continuum code findings
-Use `GetCodeFinding` JSON to investigate one finding with its locations,
-description, mitigation, and AWS validation evidence when available. Use
-`ListCodeFindings` JSON for summaries from an environment dedicated to one
-repository.
+Use `GetCodeFinding` JSON to import selected findings with their locations,
+mitigation, and AWS validation evidence. Use `ListCodeFindings` JSON for
+summaries from an environment dedicated to one repository.
## Quick start
@@ -13,8 +12,9 @@ repository supplied the scanned code, including the archive for an S3 source.
For a finding from another scanner, you can also check its repository in that
scanner. [AWS Continuum code sources](https://docs.aws.amazon.com/continuum/latest/userguide/cv-getting-started.html)
-**Saved report.** Set `report` to a native detail response or a complete list
-response with its `items` array, then skip to the preview.
+**Saved report.** Set `report` to a native detail response, an array of detail
+responses, or a complete list response with its `items` array. Then skip to
+the preview.
```sh
report=/private/path/continuum.json
@@ -38,6 +38,17 @@ out=$(mktemp -d) &&
report="$out/continuum.json"
```
+**Selected findings.** Fetch one detail response per finding ID. Verify each
+belongs to the same environment and Git repository. To publish them together,
+combine the saved objects into a JSON array with `jq`.
+
+```sh
+report=
+out=$(mktemp -d) &&
+ jq -s '.' first-detail.json second-detail.json > "$out/selected.json" &&
+ report="$out/selected.json"
+```
+
**All summaries.** Verify that the environment contains findings from just
this repository. Start on the first page and let the AWS CLI fetch every
page. [AWS CLI list-code-findings](https://docs.aws.amazon.com/cli/latest/reference/continuum/list-code-findings.html)
@@ -65,17 +76,38 @@ codex-security publish findings "$report" --to local --repository . \
Check the repository, finding ARNs, paths, and exclusions. In a
multi-repository environment, a customer-supplied path can begin with a
-repository name. The reader keeps that prefix. If the preview matches,
-[save locally or publish to Cloud and verify the result](README.md#save-or-publish-and-verify).
-For Cloud, the deployment must accept the `continuum` provider.
+repository name. The reader keeps that prefix. To save the findings in your
+local workbench, remove `--dry-run` and confirm.
+
+### Publish to your Cloud inbox
+
+Sign in to a workspace with the GitHub repository connected and private
+Continuum imports enabled. If you need locations, mitigation, or validation
+evidence, publish the detailed `get-code-finding` export first. A later upload
+with changed evidence for the same ARN can return `source_order_unknown`.
+Preview the file and source key using your connected repository URL.
+
+```sh
+repo=https://github.com/example/project
+codex-security publish findings "$report" --to cloud --repository "$repo" \
+ --provider continuum --source-repository-url "$repo" \
+ --source-key "continuum/$ENVIRONMENT_ID/example/project" \
+ --dry-run --format json
+```
+
+Confirm the account, repository, expected finding ARNs and `privateImport`
+in the preview. Remove `--dry-run`, rerun and confirm to upload. Check the
+receipt, then open a finding in your inbox and compare its source ID with the AWS ARN. [Verify the import](README.md#verify-the-import)
## Detail, summaries, and repeat imports
-Both formats use the finding ARN. With the same source key, importing a
-detail response after its summary updates the existing finding. A later
-summary replaces the richer evidence, so use `get-code-finding` for updates
-when you need those details.
+Both formats use the finding ARN. In the local workbench, importing a detail
+response under the same source key updates the existing summary. A later
+summary replaces the richer local evidence. In a private Cloud inbox,
+retrying identical evidence under the same source key preserves the finding.
-For both scanner-generated and customer-supplied findings,
-`evidence.source_data.finding` keeps the original AWS fields, including any
-Continuum status and validation results.
+The importer uses `statusUpdatedAt` for the source update time when available.
+Dependency findings expose affected package names and concrete versions.
+Cloud can show up to 100 structured packages per finding. The full list,
+including version ranges, remains in `evidence.source_data.finding` alongside
+the AWS status and validation results. [AWS detail fields](https://docs.aws.amazon.com/continuum/latest/APIReference/API_GetCodeFinding.html)
diff --git a/docs/scanner-imports/inspector.md b/docs/scanner-imports/inspector.md
index f527b7889..7e75fd4af 100644
--- a/docs/scanner-imports/inspector.md
+++ b/docs/scanner-imports/inspector.md
@@ -20,6 +20,10 @@ report=/private/path/inspector.json
### Export repository findings
+Check the scan triggers if you see pull request findings but the export is
+empty. Inspector shows results from pull-request-only or merge-request-only
+configurations in the source control platform. [Inspector scan configurations](https://docs.aws.amazon.com/inspector/latest/user/code-security-assessments-create-configuration.html)
+
With `inspector2:ListFindings` permission, replace the account, Region, and
full repository resource ID below with values from Inspector coverage. Project
names can match several integrations.
@@ -65,17 +69,44 @@ codex-security publish findings "$report" --to local --repository . \
Check the repository and finding ARNs. For a selection, verify that each
requested ARN appears once, with no other ARNs or exclusions. If the selected
rows lack repository evidence, export those ARNs again with the missing fields.
-Then [save locally or publish to Cloud and verify the result](README.md#save-or-publish-and-verify).
-For Cloud retries, reuse the Git URL and any project ARN from the preview.
+To save the selected findings in your local workbench, remove `--dry-run` and confirm.
+
+### Publish to your Cloud inbox
+
+Sign in to a workspace with the GitHub repository connected and private
+Inspector imports enabled. Preview the same file and source key, using your
+connected repository URL. Include the verified project ARN if the local
+preview required it.
+
+```sh
+repo=https://github.com/example/project
+codex-security publish findings "$report" --to cloud --repository "$repo" \
+ --provider inspector --source-repository-url "$repo" \
+ --inspector-project-arn "$VERIFIED_PROJECT_ARN" \
+ --source-key inspector/account-alias/us-east-1/example/project \
+ --dry-run --format json
+```
+
+Confirm the account, repository, expected finding ARNs and `privateImport`
+in the preview. Remove `--dry-run`, rerun and confirm to upload. Check the
+receipt, then open a finding in your inbox and compare its source ID with the
+AWS ARN. Retry with the same report, source key, Git URL, and any project ARN.
+A private import with changed evidence for the same ARN can return
+`source_order_unknown`. [Verify the import](README.md#verify-the-import)
## Identity and repository mapping
-Within a source key, reimporting an ARN updates the existing finding. Local
-findings omitted from later exports stay in the workbench. Package findings
-retain the advisory and dependency versions. Code findings retain detector
-details and reported repository-relative locations. Inspector status, severity, and
-remediation are source evidence. New closed or suppressed findings start as
-`not_assessed`.
+In the local workbench, importing changed evidence for the same source key
+and ARN updates the existing finding. Findings omitted from later exports
+stay in the workbench. Inspector status, severity, and remediation are source
+evidence. New closed or suppressed findings start as `not_assessed`.
+
+Code findings expose detector information, CWEs, and the reported source line
+range. When those fields exceed Cloud limits, you can read them in
+`evidence.source_data.finding`. Package findings use the primary vulnerability ID as their advisory
+and keep dependency versions. Related vulnerability IDs remain available in
+`evidence.source_data.finding.packageVulnerabilityDetails.relatedVulnerabilities`.
+[AWS code findings](https://docs.aws.amazon.com/inspector/v2/APIReference/API_CodeVulnerabilityDetails.html) · [AWS package findings](https://docs.aws.amazon.com/inspector/v2/APIReference/API_PackageVulnerabilityDetails.html)
Custom GitHub hosts need a native GitHub provider hint for Cloud. For local
imports from self-managed GitLab, use the verified HTTPS clone URL if SSH
diff --git a/docs/scanner-imports/schema-guide.md b/docs/scanner-imports/schema-guide.md
index 3d700943e..da13da9ae 100644
--- a/docs/scanner-imports/schema-guide.md
+++ b/docs/scanner-imports/schema-guide.md
@@ -26,7 +26,7 @@ changes.
| Report | What identifies one record |
| ------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Amazon Inspector](inspector.md), [Endor Labs](endor.md), [SonarQube Cloud](sonar.md) | Inspector finding ARN, Endor Finding UUID or Sonar issue key. Endor native JSON and SARIF use the same UUID. |
-| [AWS Continuum](continuum.md) | Continuum finding ARN. A list summary and a detailed response for that ARN update the same record. |
+| [AWS Continuum](continuum.md) | Continuum finding ARN. A list summary and a detailed response for that ARN identify the same finding. |
| [Checkmarx SAST](checkmarx.md) | Checkmarx One uses the similarity ID and result hash. CxSAST scopes the similarity ID and ordered flow to the project and query. |
| [Wiz](wiz.md) | Package occurrence ID, or a SAST, repository-secret or IaC occurrence ID prefixed with its family. Normalized JSONL uses the supplied ID. |
| [GitHub Dependabot](github-dependabot.md), [code scanning](github-code-scanning.md) | Host, repository, alert family and alert number. Code scanning also includes the selected ref. Two alerts citing one GHSA remain separate. |
diff --git a/sdk/typescript/src/continuum-findings.ts b/sdk/typescript/src/continuum-findings.ts
index 2c9839943..a2ee150e1 100644
--- a/sdk/typescript/src/continuum-findings.ts
+++ b/sdk/typescript/src/continuum-findings.ts
@@ -1,13 +1,16 @@
import { CodexSecurityError } from "./errors.js";
import type { ExternalFinding, ExternalFindings } from "./external-findings.js";
import {
+ DEFAULT_CLOUD_STRING_BYTES,
type ExternalImportTarget,
- validateExternalEvidence,
+ validateEnrichedExternalEvidence,
} from "./external-import-contract.js";
-import type { Provider } from "./external-import-models.js";
+import type { ImportedPackage, Provider } from "./external-import-models.js";
import { isRecord } from "./record.js";
import {
array,
+ awsTimestampSeconds,
+ concreteVersion,
normalizeRecords,
readScannerJson,
record,
@@ -146,16 +149,8 @@ function codeLocations(
return { locations, code: { end_line: endLine, snippet } };
}
-export function normalizeContinuumFindings(
- input: unknown,
- sourceRepositoryUrl: string,
- target: ExternalImportTarget = "cloud",
-): ExternalFindings {
- const repository = declaredScannerRepositoryUrl(
- sourceRepositoryUrl,
- "Continuum",
- target,
- );
+function reportRows(input: unknown, previous: boolean) {
+ if (Array.isArray(input) && !previous) return { rows: input, summary: false };
const document = record(input, "Continuum report");
if (
Object.hasOwn(document, "nextToken") ||
@@ -169,9 +164,72 @@ export function normalizeContinuumFindings(
"Expected a Continuum ListCodeFindings or GetCodeFinding response.",
);
const summary = "items" in document;
- const rows = summary
- ? array(document["items"], "Continuum items")
- : [document];
+ return {
+ rows: summary ? array(document["items"], "Continuum items") : [document],
+ summary,
+ };
+}
+
+function statusUpdatedAt(value: unknown) {
+ const seconds = Math.floor(awsTimestampSeconds(value));
+ return Number.isSafeInteger(seconds) && seconds >= 0 ? seconds : null;
+}
+
+const versionEcosystems = new Map([
+ ["gradle", "maven"],
+ ["pnpm", "npm"],
+ ["yarn", "npm"],
+]);
+
+function affectedPackages(
+ items: unknown[],
+ target: ExternalImportTarget,
+): ImportedPackage[] {
+ const unique = new Map<string, ImportedPackage>();
+ const field = (value: unknown) => {
+ const result = text(value);
+ return result &&
+ !result.includes("\0") &&
+ (target === "local" ||
+ Buffer.byteLength(result) <= DEFAULT_CLOUD_STRING_BYTES)
+ ? result
+ : null;
+ };
+ for (const item of items) {
+ if (!isRecord(item)) continue;
+ const name = field(item["name"]);
+ if (!name) continue;
+ const ecosystem = field(item["packageManager"]);
+ const manager = ecosystem?.toLowerCase();
+ const syntax = manager ? (versionEcosystems.get(manager) ?? manager) : null;
+ const version = (value: unknown) =>
+ concreteVersion(field(value), syntax, false);
+ const fixed = version(item["fixedInVersion"]);
+ const pkg = {
+ name,
+ ecosystem,
+ installed_version: version(item["version"]),
+ fixed_versions: fixed ? [fixed] : [],
+ };
+ unique.set(JSON.stringify(pkg), pkg);
+ // The Cloud contract accepts 100 structured packages. The native row keeps them all.
+ if (target === "cloud" && unique.size === 100) break;
+ }
+ return [...unique.values()];
+}
+
+function normalize(
+ input: unknown,
+ sourceRepositoryUrl: string,
+ target: ExternalImportTarget,
+ previous: boolean,
+): ExternalFindings {
+ const repository = declaredScannerRepositoryUrl(
+ sourceRepositoryUrl,
+ "Continuum",
+ target,
+ );
+ const { rows, summary } = reportRows(input, previous);
const scopes = new Set<string>();
const output = normalizeRecords(
rows,
@@ -203,13 +261,10 @@ export function normalizeContinuumFindings(
const advisoryIds = [
...new Set(strings(all.map((item) => item["cveId"]))),
];
- const packageFinding =
- advisoryIds.length > 0 ||
- all.some(
- (v) =>
- Array.isArray(v["affectedPackages"]) &&
- v["affectedPackages"].length > 0,
- );
+ const packages = all.flatMap((item) =>
+ Array.isArray(item["affectedPackages"]) ? item["affectedPackages"] : [],
+ );
+ const packageFinding = advisoryIds.length > 0 || packages.length > 0;
const posture =
row["ocsfClass"] === "APPLICATION_SECURITY_POSTURE_FINDING" ||
row["posture"] != null;
@@ -219,38 +274,52 @@ export function normalizeContinuumFindings(
!packageFinding &&
(code.snippet !== null ||
locations.some((location) => location.line !== null));
- return validateExternalEvidence(
- {
- title: requiredText(row["title"], "Continuum title"),
- severity:
- severity === "fatal"
- ? "critical"
- : severity === "other"
- ? "unknown"
- : severity,
- description: text(row["detailedDescription"]) ?? text(row["summary"]),
- locations,
- advisory_ids: advisoryIds,
- source_scan_id: text(row["scanRunId"]),
- ...(codeFinding && {
- details: {
- kind: "sast",
- repository: { url: repository },
- scanner_origin: source,
- scanner_verdict: text(validation["verdict"]),
- source_status: status,
- weakness_ids: strings(row["cweIds"]),
- remediation_instructions: text(row["mitigation"]),
- code,
- },
- }),
- source_data: {
- format,
- sourceRepositoryUrl: repository,
- finding: structuredClone(row),
+ const evidence = {
+ title: requiredText(row["title"], "Continuum title"),
+ severity:
+ severity === "fatal"
+ ? "critical"
+ : severity === "other"
+ ? "unknown"
+ : severity,
+ description: text(row["detailedDescription"]) ?? text(row["summary"]),
+ locations,
+ advisory_ids: advisoryIds,
+ source_scan_id: text(row["scanRunId"]),
+ ...(codeFinding && {
+ details: {
+ kind: "sast",
+ repository: { url: repository },
+ scanner_origin: source,
+ scanner_verdict: text(validation["verdict"]),
+ source_status: status,
+ weakness_ids: strings(row["cweIds"]),
+ remediation_instructions: text(row["mitigation"]),
+ code,
},
+ }),
+ source_data: {
+ format,
+ sourceRepositoryUrl: repository,
+ finding: structuredClone(row),
},
+ };
+ const updatedAt = previous
+ ? null
+ : statusUpdatedAt(row["statusUpdatedAt"]);
+ // Keep the status timestamp when structured packages exceed the Cloud budget.
+ return validateEnrichedExternalEvidence(
+ evidence,
target,
+ previous
+ ? undefined
+ : {
+ packages: affectedPackages(packages, target),
+ source_updated_at: updatedAt,
+ },
+ packages.length && updatedAt !== null
+ ? { source_updated_at: updatedAt }
+ : undefined,
);
},
(value) => {
@@ -265,6 +334,23 @@ export function normalizeContinuumFindings(
return output;
}
+export function normalizeContinuumFindings(
+ input: unknown,
+ sourceRepositoryUrl: string,
+ target: ExternalImportTarget = "cloud",
+): ExternalFindings {
+ return normalize(input, sourceRepositoryUrl, target, false);
+}
+
+/** @internal Reconstruct the original evidence for an interrupted Cloud request. */
+export function normalizePreviousContinuumFindings(
+ input: unknown,
+ sourceRepositoryUrl: string,
+ target: ExternalImportTarget = "cloud",
+): ExternalFindings {
+ return normalize(input, sourceRepositoryUrl, target, true);
+}
+
export async function readContinuumFindings(
path: string,
sourceRepositoryUrl: string,
diff --git a/sdk/typescript/src/external-findings-publish.ts b/sdk/typescript/src/external-findings-publish.ts
index 08b3d6f4a..7a1299164 100644
--- a/sdk/typescript/src/external-findings-publish.ts
+++ b/sdk/typescript/src/external-findings-publish.ts
@@ -23,6 +23,8 @@ import {
selectPrivateReport,
type PrivateImportFormat,
readPrivateCheckpoint,
+ parsePrivateCheckpoint,
+ type PrivateImportContext,
unsuccessfulJob,
type PrivateImportJob,
} from "./private-finding-import.js";
@@ -676,6 +678,7 @@ export async function prepareExternalPublication(
input =
parsed.publication ??
normalizedPublicationInput(parsed.findings, provider);
+ input.previousAws = parsed.previousFindings;
}
return preparePublicationInput(
input,
@@ -897,6 +900,7 @@ function hostedRepositoryMatcher(
}
interface PublicationSelection extends ExternalPublicationInput {
+ previousAws?: () => ExternalFindings;
original?: {
parsed: ExternalFindings;
identity: unknown;
@@ -1423,7 +1427,32 @@ async function preparePublicationInput(
const catalogKey = trivyReportUrls
? (url: string) => cloudSourceKey(url, "github")
: cloudRepositoryUrlKey;
- function checkSelection(input: PublicationSelection) {
+ const state = externalPublicationStateDirectory(environment);
+ let previousAws: ExternalFindings | undefined;
+ async function previousAwsIfSaved(
+ input: PublicationSelection,
+ currentKey?: string,
+ ) {
+ if (!input.previousAws || previousAws) return previousAws;
+ // An empty state directory (or just this current receipt) needs only the
+ // current reader for its preview.
+ const entries = await readdir(state).catch(
+ (error: NodeJS.ErrnoException) => {
+ if (error.code === "ENOENT") return [];
+ throw error;
+ },
+ );
+ if (
+ entries.some(
+ (name) =>
+ /^[a-f0-9]{64}\.(?:pending|private-import)\.json$/u.test(name) &&
+ name.split(".", 1)[0] !== currentKey,
+ )
+ )
+ previousAws = input.previousAws();
+ return previousAws;
+ }
+ async function checkSelection(input: PublicationSelection) {
const parsed = input.current;
checkTrivyReportProvider(options.provider, parsed.report);
for (const finding of parsed.findings)
@@ -1435,11 +1464,12 @@ async function preparePublicationInput(
!input.potentialIds?.length &&
!input.uncertainFindingIds?.length &&
!input.unprovenOccurrenceCount &&
- !input.unprovenIdPrefixes?.length
+ !input.unprovenIdPrefixes?.length &&
+ !(await previousAwsIfSaved(input))?.findings.length
)
throw new NoPublishableFindingsError(parsed);
}
- if (typeof supplied !== "function") checkSelection(supplied);
+ if (typeof supplied !== "function") await checkSelection(supplied);
progress({ phase: "discovering", completed: 0 });
const apiBaseUrl = cloudBaseUrl(environment);
const cloudApiUrl = `${apiBaseUrl}/external`;
@@ -1449,6 +1479,10 @@ async function preparePublicationInput(
dependencies.credentials ?? (() => readCloudCredentials(environment))
)();
let privateProviders: Set<string> | undefined;
+ const jobVersionHeader = "Codex-Security-Private-Import-Job-Version";
+ let jobApiVersion: 2 | undefined;
+ const aws =
+ options.provider === "inspector" || options.provider === "continuum";
async function request(
endpoint: string,
body?: FindingImportRequest,
@@ -1456,6 +1490,7 @@ async function preparePublicationInput(
dependencies.signal?.throwIfAborted();
// Allow upload and response transport around the server's publication budget.
const timeout = AbortSignal.timeout(body ? 60_000 : 30_000);
+ const repositoryCatalog = endpoint.startsWith("/repositories?");
const response = await (dependencies.fetch ?? globalThis.fetch)(
`${cloudApiUrl}${endpoint}`,
{
@@ -1464,6 +1499,7 @@ async function preparePublicationInput(
Authorization: `Bearer ${credentials.access_token}`,
"ChatGPT-Account-ID": credentials.account_id,
Accept: "application/json",
+ ...(repositoryCatalog ? { [jobVersionHeader]: "2" } : {}),
...(body ? { "Content-Type": "application/json" } : {}),
},
...(body ? { body: JSON.stringify(body) } : {}),
@@ -1483,7 +1519,9 @@ async function preparePublicationInput(
`Cloud import failed (HTTP ${response.status}). ${payload?.error?.message ?? response.statusText}${retryAfter ? ` Retry-After: ${retryAfter}.` : ""}`,
);
}
- if (endpoint.startsWith("/repositories?")) {
+ if (repositoryCatalog) {
+ jobApiVersion =
+ aws && response.headers.get(jobVersionHeader) === "2" ? 2 : undefined;
const advertised = response.headers.get(
"Codex-Security-Private-Import-Providers",
);
@@ -1534,13 +1572,12 @@ async function preparePublicationInput(
const destination = await discoverDestination();
const deferred = typeof supplied === "function";
const resolved = deferred ? await supplied(destination) : supplied;
- if (deferred) checkSelection(resolved);
+ if (deferred) await checkSelection(resolved);
const report: ExternalFindings & Partial<SnykIssuesFindings> =
resolved.current;
const source = { provider: options.provider, source_key: options.sourceKey };
const importPrefix = source.provider === "sonar" ? "/v2" : "";
const sourceReportsPath = importPrefix + sourcePath(destination);
- const state = externalPublicationStateDirectory(environment);
const checkpointIdentity = [
// Keep existing production checkpoints readable; other deployments must
// never share a saved request or receipt with the production default.
@@ -1591,6 +1628,84 @@ async function preparePublicationInput(
if (capturedPending !== undefined) break;
}
}
+ // Changed AWS projections get new keys, but both transports must replay the
+ // exact bytes already approved by the earlier reader when a checkpoint exists.
+ const privateClient = privateImportClient(
+ apiBaseUrl,
+ credentials,
+ dependencies.fetch ?? globalThis.fetch,
+ dependencies.signal,
+ );
+ let previewContext: PrivateImportContext | undefined;
+ const currentAwsKey = resolved.previousAws
+ ? scopeKey(resolved.currentIdentity)
+ : undefined;
+ let previousAwsKey: string | undefined;
+ let alternateKey: string | undefined;
+ let resumingPreviousAws = false;
+ function previousCandidateKey(previous: ExternalFindings | undefined) {
+ if (!previous?.findings.length) return;
+ // The file reader already validated plain JSON evidence for Cloud. Only a
+ // matching checkpoint needs the full normalized publication snapshot.
+ previousAwsKey ??= scopeKey(publicationIdentity(previous));
+ return previousAwsKey !== currentAwsKey ? previousAwsKey : undefined;
+ }
+ async function hasPrivateCheckpoint(
+ key: string,
+ context: () => Promise<PrivateImportContext>,
+ ) {
+ const saved = await readOptionalFile(
+ join(state, key + ".private-import.json"),
+ );
+ if (saved === undefined) return false;
+ const checkpoint = parsePrivateCheckpoint(saved);
+ // A completed import cannot acknowledge findings newly admitted by the
+ // current reader. Pending uploads and jobs always retain their original bytes.
+ if (
+ key === previousAwsKey &&
+ previousAws &&
+ checkpoint.receipt?.status === "completed" &&
+ !sameFindingSelection(previousAws.findings, resolved.current.findings)
+ )
+ return false;
+ return checkpoint.contextVersion === (await context()).context_version;
+ }
+ async function assertNoAlternateCheckpoint(
+ context = () => privateClient.context(),
+ ) {
+ const otherKey = alternateKey;
+ if (!otherKey) return;
+ if (
+ (await readOptionalFile(join(state, otherKey + ".pending.json"))) !==
+ undefined ||
+ (await hasPrivateCheckpoint(otherKey, context))
+ )
+ throw new CodexSecurityError(
+ resumingPreviousAws
+ ? "Both AWS reader versions have saved publications for this report. Resume either one using its original normalized findings."
+ : "Another publication prepared this input. Run the command again to review and resume the saved request.",
+ );
+ }
+ if (currentAwsKey)
+ alternateKey = previousCandidateKey(
+ await previousAwsIfSaved(resolved, currentAwsKey),
+ );
+ if (alternateKey && previousAws) {
+ capturedPending = await readOptionalFile(
+ join(state, alternateKey + ".pending.json"),
+ );
+ const context = async () =>
+ (previewContext ??= await privateClient.context());
+ if (
+ capturedPending !== undefined ||
+ (await hasPrivateCheckpoint(alternateKey, context))
+ ) {
+ projection = normalizedPublicationInput(previousAws, options.provider);
+ resumingPreviousAws = true;
+ alternateKey = currentAwsKey;
+ await assertNoAlternateCheckpoint(context);
+ }
+ }
const input = projection;
let {
current: parsed,
@@ -2590,6 +2705,17 @@ async function preparePublicationInput(
}
const lockKeys = [key, fileKey, stagingKey];
+ if (alternateKey) lockKeys.push(alternateKey);
+ async function underPublishLocks<T>(operation: () => Promise<T>) {
+ // An older publisher may already hold its lock before its checkpoint exists.
+ // Derive that key for actual writes, even after a checkpoint-free preview.
+ if (resolved.previousAws && !previousAws) {
+ previousAws = resolved.previousAws();
+ alternateKey = previousCandidateKey(previousAws);
+ if (alternateKey) lockKeys.push(alternateKey);
+ }
+ return underLocks(lockKeys, operation);
+ }
if (selected?.historical) {
const observed = selected;
selected = await underLocks(lockKeys, async () => {
@@ -2905,6 +3031,10 @@ async function preparePublicationInput(
// Existing repository submissions keep their original request bodies and receipts.
const privatePath = join(state, `${key}.private-import.json`);
const privateSaved = await readOptionalFile(privatePath);
+ if (resumingPreviousAws && !saved && privateSaved === undefined)
+ throw new CodexSecurityError(
+ "The saved publication changed. Run the command again to preview this report.",
+ );
const usePrivateImport =
!saved &&
(privateProviders?.has(source.provider) || privateSaved !== undefined);
@@ -2953,13 +3083,8 @@ async function preparePublicationInput(
}
if (usePrivateImport) {
- const client = privateImportClient(
- apiBaseUrl,
- credentials,
- dependencies.fetch ?? globalThis.fetch,
- dependencies.signal,
- );
- const context = await client.context();
+ const client = privateClient;
+ const context = previewContext ?? (await client.context());
if (source.source_key.trim() !== source.source_key)
throw new CodexSecurityError(
"A private source key cannot have leading or trailing whitespace.",
@@ -2989,15 +3114,13 @@ async function preparePublicationInput(
: readPrivateCheckpoint(data, binding);
}
const previous = checkpointForContext(privateSaved);
- function requireAccess(
- value: typeof context,
- savedFile: boolean,
- savedJob: boolean,
- ) {
+ const canStartJob = !aws || privateProviders?.has(source.provider) === true;
+ function requireAccess(value: typeof context, saved: typeof previous) {
if (
!value.can_read ||
- (!savedJob &&
- ((!savedFile && !value.can_upload) ||
+ (!saved && !canStartJob) ||
+ (!saved?.jobId &&
+ ((!saved?.fileId && !value.can_upload) ||
!value.can_create_import_jobs ||
!value.formats.includes(report.format)))
)
@@ -3005,9 +3128,10 @@ async function preparePublicationInput(
"Private findings import is unavailable for this account or report format.",
);
}
- requireAccess(context, !!previous?.fileId, !!previous?.jobId);
+ requireAccess(context, previous);
const retryOfJobId =
unsuccessfulJob(previous?.receipt) &&
+ canStartJob &&
context.can_upload &&
context.can_create_import_jobs &&
context.formats.includes(report.format)
@@ -3035,33 +3159,43 @@ async function preparePublicationInput(
return {
preview: structuredClone(preview),
publish: () =>
- underLocks(lockKeys, async () => {
+ underPublishLocks(async () => {
const current = await client.context();
if (current.context_version !== binding.contextVersion)
throw new CodexSecurityError(
"The private inbox changed. Run the command again to review the new destination.",
);
+ await assertNoAlternateCheckpoint(async () => current);
const stored = checkpointForContext(
await readOptionalFile(privatePath),
);
+ if (resumingPreviousAws && !stored)
+ throw new CodexSecurityError(
+ "The saved private import changed. Run the command again to preview this report.",
+ );
// Only a preview of this terminal job can start a new attempt.
const retrying =
retryOfJobId !== undefined && stored?.receipt?.id === retryOfJobId;
const active = retrying ? undefined : stored;
if (resumed && !active) checkRepositories(parsed);
- requireAccess(current, !!active?.fileId, !!active?.jobId);
+ requireAccess(current, active);
if (await selectedCheckpoint())
throw new CodexSecurityError(
"A repository submission is pending. Review its saved request before continuing.",
);
- const checkpoint = active ?? {
- ...binding,
- uploadRequestId: randomUUID(),
- jobRequestId: randomUUID(),
- fileId: undefined,
- jobId: undefined,
- receipt: undefined,
- previousReceipt: retrying ? stored?.receipt : undefined,
+ // Both routes share request IDs. A server upgrade can promote an
+ // existing AWS job, and its saved route also survives gate rollback.
+ const checkpoint = {
+ ...(active ?? {
+ ...binding,
+ uploadRequestId: randomUUID(),
+ jobRequestId: randomUUID(),
+ fileId: undefined,
+ jobId: undefined,
+ receipt: undefined,
+ previousReceipt: retrying ? stored?.receipt : undefined,
+ }),
+ jobApiVersion: stored?.jobApiVersion ?? jobApiVersion,
};
const result: ExternalPublicationResult = {
cloudApiUrl: preview.cloudApiUrl,
@@ -3103,7 +3237,8 @@ async function preparePublicationInput(
const savedReceipt =
checkpoint.receipt ?? checkpoint.previousReceipt;
if (savedReceipt) acknowledge(savedReceipt);
- if (!active) await writeAtomicJson(privatePath, checkpoint);
+ if (!active || active.jobApiVersion !== checkpoint.jobApiVersion)
+ await writeAtomicJson(privatePath, checkpoint);
if (!checkpoint.fileId) {
progress({ phase: "uploading", completed: 0 });
checkpoint.fileId = await client.upload({
@@ -3124,6 +3259,7 @@ async function preparePublicationInput(
source,
repository: binding.repository,
jobId: checkpoint.jobId,
+ jobApiVersion: checkpoint.jobApiVersion,
},
async (id) => {
checkpoint.jobId = id;
@@ -3360,8 +3496,9 @@ async function preparePublicationInput(
);
}
let acquired = false;
- return await underLocks(lockKeys, async () => {
+ return await underPublishLocks(async () => {
acquired = true;
+ await assertNoAlternateCheckpoint();
// Re-read under the lock to keep checkpoints from another publisher.
const selected = await selectedCheckpoint(
saved ? fileKey : undefined,
diff --git a/sdk/typescript/src/external-findings-read.ts b/sdk/typescript/src/external-findings-read.ts
index e993f0eb0..b67de2866 100644
--- a/sdk/typescript/src/external-findings-read.ts
+++ b/sdk/typescript/src/external-findings-read.ts
@@ -31,9 +31,15 @@ import { readGitHubFindings } from "./github-findings.js";
import { readSonarFindings } from "./sonar-findings.js";
import { readVeracodeFindings } from "./veracode-findings.js";
import { readBlackDuckFindings } from "./blackduck-findings.js";
-import { readContinuumFindings } from "./continuum-findings.js";
+import {
+ normalizeContinuumFindings,
+ normalizePreviousContinuumFindings,
+} from "./continuum-findings.js";
import { readCheckmarxFindings } from "./checkmarx-findings.js";
-import { readInspectorFindings } from "./inspector-findings.js";
+import {
+ normalizeInspectorFindings,
+ normalizePreviousInspectorFindings,
+} from "./inspector-findings.js";
import { isRecord } from "./record.js";
import {
readLocalWizFindings,
@@ -76,6 +82,7 @@ export interface ExternalFindingsReaderOptions {
/** @internal Shared parser output, including immutable Cloud retry projections. */
export interface ReadExternalFindings {
findings: ExternalFindings;
+ previousFindings?: () => ExternalFindings;
/** Preserve previous Wiz projections when resuming immutable Cloud requests. */
publication?: ExternalPublicationInput;
sourceRepositories?: ExternalSourceRepository[];
@@ -223,23 +230,44 @@ export async function readExternalFindings(
}
if (provider === "github")
return { findings: await readGitHubFindings(path, target) };
+ if (provider === "inspector" || provider === "continuum") {
+ const url = options.sourceRepositoryUrl ?? "";
+ const mapping =
+ options.inspectorProjectMap && new Map(options.inspectorProjectMap);
+ const input = await readScannerJson(path);
+ const normalize =
+ provider === "inspector"
+ ? normalizeInspectorFindings
+ : normalizeContinuumFindings;
+ const previous =
+ provider === "inspector"
+ ? normalizePreviousInspectorFindings
+ : normalizePreviousContinuumFindings;
+ // Both projections describe the same captured file, including rows admitted
+ // by just one reader. Detail arrays are new to Continuum.
+ const wasSupported = provider === "inspector" || !Array.isArray(input);
+ return {
+ findings: normalize(input, url, target, mapping),
+ previousFindings:
+ target === "cloud" && wasSupported
+ ? () => previous(input, url, target, mapping)
+ : undefined,
+ sourceRepositories: [{ url }],
+ };
+ }
if (
- provider === "inspector" ||
- provider === "continuum" ||
provider === "checkmarx" ||
provider === "blackduck" ||
provider === "veracode"
) {
const url = options.sourceRepositoryUrl ?? "";
const read = {
- inspector: readInspectorFindings,
- continuum: readContinuumFindings,
checkmarx: readCheckmarxFindings,
blackduck: readBlackDuckFindings,
veracode: readVeracodeFindings,
}[provider];
return {
- findings: await read(path, url, target, options.inspectorProjectMap),
+ findings: await read(path, url, target),
sourceRepositories: [{ url }],
};
}
diff --git a/sdk/typescript/src/external-import-contract.ts b/sdk/typescript/src/external-import-contract.ts
index 0c94a05b8..f20cf8d93 100644
--- a/sdk/typescript/src/external-import-contract.ts
+++ b/sdk/typescript/src/external-import-contract.ts
@@ -375,6 +375,23 @@ export function previousAdmission(value: object) {
return previousAdmissions.get(value);
}
+/** Keep a valid native record when its optional structured fields exceed the target's bounds. */
+export function validateEnrichedExternalEvidence(
+ evidence: Record<string, unknown>,
+ target: ExternalImportTarget,
+ ...projections: (Record<string, unknown> | undefined)[]
+): ExternalFindingEvidence {
+ for (const extra of projections) {
+ if (!extra) continue;
+ try {
+ return validateExternalEvidence({ ...evidence, ...extra }, target);
+ } catch {
+ // Try the next projection. The native record must pass the same validation.
+ }
+ }
+ return validateExternalEvidence(evidence, target);
+}
+
export function validateExternalEvidence(
input: unknown,
target: ExternalImportTarget = "cloud",
diff --git a/sdk/typescript/src/inspector-findings.ts b/sdk/typescript/src/inspector-findings.ts
index efda61b11..9775193ce 100644
--- a/sdk/typescript/src/inspector-findings.ts
+++ b/sdk/typescript/src/inspector-findings.ts
@@ -1,6 +1,5 @@
import { posix } from "node:path";
import { z } from "zod";
-import validSemver from "semver/functions/valid.js";
import type {
ExternalFinding,
ExternalFindings,
@@ -9,7 +8,7 @@ import type {
import {
cloudRepositoryUrlKey,
repositoryUrlKey,
- validateExternalEvidence,
+ validateEnrichedExternalEvidence,
type ExternalImportTarget,
} from "./external-import-contract.js";
import { isGitHubCloudHost } from "./git-remote.js";
@@ -17,11 +16,14 @@ import { CodexSecurityError, errorMessage } from "./errors.js";
import type { Provider } from "./external-import-models.js";
import { isRecord } from "./record.js";
import {
+ awsTimestampSeconds,
+ concreteVersion,
normalizeRecords,
readScannerJson,
record,
repositoryFilePath,
requiredText,
+ strings,
text,
} from "./scanner-import.js";
@@ -330,6 +332,7 @@ const repositoryFinding = z.object({
fileName: optionalValue,
filePath: optionalValue,
startLine: optionalValue,
+ endLine: optionalValue,
}),
sourceLambdaLayerArn: absent,
})
@@ -445,15 +448,7 @@ export function inspectorReportRepository(
}
function timestamp(value: unknown): number {
- const seconds =
- typeof value === "number"
- ? value
- : typeof value === "string" &&
- /^\d{4}-\d\d-\d\dT\d\d:\d\d:\d\d(?:\.\d+)?(?:Z|[+-]\d\d:\d\d)$/u.test(
- value,
- )
- ? Date.parse(value) / 1000
- : NaN;
+ const seconds = awsTimestampSeconds(value);
if (!Number.isFinite(seconds))
throw new Error("Invalid Inspector timestamp.");
return Math.floor(seconds);
@@ -518,46 +513,12 @@ function manifestPath(path: string | null): string | null {
: null;
}
-function concreteVersion(
- value: unknown,
- ecosystem: string | null,
- osPackage: boolean,
-): string | null {
- const version = text(value);
- if (!version) return null;
- // Debian versions use internal tildes for prereleases and backports.
- if (osPackage && version.includes("~"))
- return /^(?:\d+:)?\d[a-z\d.+~]*(?:-[a-z\d.+~]+)*$/iu.test(version)
- ? version
- : null;
- if (validSemver(version)) return version;
- if (ecosystem === "npm" || ecosystem === "cargo") return null;
- if (ecosystem === "java" || ecosystem === "maven") {
- if (
- version === "LATEST" ||
- version === "RELEASE" ||
- version.startsWith("latest.")
- )
- return null;
- return /^[a-z\d][\w.+-]*$/iu.test(version) && !version.endsWith("+")
- ? version
- : null;
- }
- if (!ecosystem && version.endsWith("+")) return null;
- const publicVersion =
- ecosystem === "pypi" ? version.split("+", 1)[0]! : version;
- return /\d/u.test(version) &&
- /^(?:\d+:)?[a-z\d][\w.+!-]*$/iu.test(version) &&
- !/(?:^|\.)x(?:\.|$)/iu.test(publicVersion)
- ? version
- : null;
-}
-
function evidence(
{ native: finding, pkg, code }: ReturnType<typeof selectedFinding>,
id: string,
source: string,
target: ExternalImportTarget,
+ previousProjection: boolean,
) {
const {
description,
@@ -596,53 +557,67 @@ function evidence(
const locations: { path: string; line: number | null }[] = [...paths].map(
(path) => ({ path, line: null }),
);
+ let endLine: number | null = null;
if (code) {
- const { fileName, filePath, startLine } = code.filePath;
+ const { fileName, filePath, startLine, endLine: end } = code.filePath;
const path = repositoryFilePath(filePath);
- if (path && posix.basename(path) === fileName)
- locations.push({
- path,
- line:
- typeof startLine === "number" &&
- Number.isSafeInteger(startLine) &&
- startLine > 0
- ? startLine
- : null,
- });
- }
- return validateExternalEvidence(
- {
- title:
- text(finding["title"]) ??
- pkg?.vulnerabilityId ??
- code?.detectorName ??
- id,
- description: requiredText(description, "Inspector description"),
- severity: ["critical", "high", "medium", "low", "informational"].includes(
- level,
+ if (path && posix.basename(path) === fileName) {
+ const line =
+ typeof startLine === "number" &&
+ Number.isSafeInteger(startLine) &&
+ startLine > 0
+ ? startLine
+ : null;
+ locations.push({ path, line });
+ if (
+ line !== null &&
+ typeof end === "number" &&
+ Number.isSafeInteger(end) &&
+ end >= line
)
- ? level
- : "unknown",
- url: text(pkg?.sourceUrl),
- advisory_ids: pkg
- ? [
- ...new Set([
- pkg.vulnerabilityId,
- ...(pkg.relatedVulnerabilities ?? []),
- ]),
- ]
- : [],
- packages,
- locations,
- source_updated_at: updatedAt === undefined ? null : timestamp(updatedAt),
- source_data: {
- format,
- sourceRepositoryUrl: source,
- finding: structuredClone(finding),
- },
+ endLine = end;
+ }
+ }
+ const advisoryIds = pkg ? [pkg.vulnerabilityId] : [];
+ const base = {
+ title:
+ text(finding["title"]) ??
+ pkg?.vulnerabilityId ??
+ code?.detectorName ??
+ id,
+ description: requiredText(description, "Inspector description"),
+ severity: ["critical", "high", "medium", "low", "informational"].includes(
+ level,
+ )
+ ? level
+ : "unknown",
+ url: text(pkg?.sourceUrl),
+ advisory_ids:
+ previousProjection && pkg?.relatedVulnerabilities
+ ? [...new Set(advisoryIds.concat(pkg.relatedVulnerabilities))]
+ : advisoryIds,
+ packages,
+ locations,
+ source_updated_at: updatedAt === undefined ? null : timestamp(updatedAt),
+ source_data: {
+ format,
+ sourceRepositoryUrl: source,
+ finding: structuredClone(finding),
},
- target,
- );
+ };
+ const enrichment =
+ code && !previousProjection
+ ? {
+ details: {
+ kind: "sast",
+ repository: { url: source },
+ rule: { id: code.detectorId, name: code.detectorName },
+ weakness_ids: strings(code.cwes),
+ code: endLine === null ? null : { end_line: endLine },
+ },
+ }
+ : undefined;
+ return validateEnrichedExternalEvidence(base, target, enrichment);
}
export function normalizeInspectorFindings(
@@ -650,6 +625,26 @@ export function normalizeInspectorFindings(
sourceRepositoryUrl: string,
target: ExternalImportTarget = "cloud",
projectMap?: ReadonlyMap<string, string>,
+): ExternalFindings {
+ return normalizeReport(input, sourceRepositoryUrl, target, projectMap, false);
+}
+
+/** @internal Reconstruct the prior evidence when matching a pending publication. */
+export function normalizePreviousInspectorFindings(
+ input: unknown,
+ sourceRepositoryUrl: string,
+ target: ExternalImportTarget = "cloud",
+ projectMap?: ReadonlyMap<string, string>,
+): ExternalFindings {
+ return normalizeReport(input, sourceRepositoryUrl, target, projectMap, true);
+}
+
+function normalizeReport(
+ input: unknown,
+ sourceRepositoryUrl: string,
+ target: ExternalImportTarget,
+ projectMap: ReadonlyMap<string, string> | undefined,
+ previousProjection: boolean,
): ExternalFindings {
const source = repositoryUrl(sourceRepositoryUrl);
if (
@@ -697,7 +692,14 @@ export function normalizeInspectorFindings(
}
return normalizeRecords(
rows,
- ({ finding }, id) => evidence(selectedFinding(finding), id, source, target),
+ ({ finding }, id) =>
+ evidence(
+ selectedFinding(finding),
+ id,
+ source,
+ target,
+ previousProjection,
+ ),
({ finding }) => identity(record(finding, "Inspector finding")).arn,
target,
{ filter: checkRetainedScope },
diff --git a/sdk/typescript/src/private-finding-import.ts b/sdk/typescript/src/private-finding-import.ts
index 96f79c20c..9c56628c6 100644
--- a/sdk/typescript/src/private-finding-import.ts
+++ b/sdk/typescript/src/private-finding-import.ts
@@ -278,6 +278,8 @@ const checkpointSchema = z.object({
repository: z.object({ id: z.string(), repo_connector_id: z.string() }),
uploadRequestId: z.uuid(),
jobRequestId: z.uuid(),
+ // Earlier checkpoints used the unversioned job route.
+ jobApiVersion: z.literal(2).optional(),
fileId: fileId.optional(),
jobId: jobSchema.shape.id.optional(),
receipt: jobSchema
@@ -290,6 +292,10 @@ export function unsuccessfulJob(job: PrivateImportJob | undefined) {
return job?.status === "failed" || job?.status === "cancelled";
}
+export function parsePrivateCheckpoint(serialized: string) {
+ return checkpointSchema.parse(JSON.parse(serialized));
+}
+
export function readPrivateCheckpoint(
serialized: string,
expected: {
@@ -302,10 +308,11 @@ export function readPrivateCheckpoint(
repository: { id: string; repo_connector_id: string };
},
) {
- const checkpoint = checkpointSchema.parse(JSON.parse(serialized));
+ const checkpoint = parsePrivateCheckpoint(serialized);
const {
uploadRequestId,
jobRequestId,
+ jobApiVersion,
fileId,
jobId,
receipt,
@@ -366,7 +373,7 @@ export function privateImportClient(
) {
async function request(path: string, body?: FormData | string) {
const timeout = AbortSignal.timeout(body === undefined ? 30_000 : 60_000);
- const response = await fetch(`${base}/finding-imports/${path}`, {
+ const response = await fetch(`${base}${path}`, {
method: body === undefined ? "GET" : "POST",
body,
redirect: "error",
@@ -398,7 +405,9 @@ export function privateImportClient(
}
return {
async context() {
- return contextSchema.parse(await (await request("context")).json());
+ return contextSchema.parse(
+ await (await request("/finding-imports/context")).json(),
+ );
},
async upload(input: {
requestId: string;
@@ -410,7 +419,9 @@ export function privateImportClient(
let response: Response | undefined;
if (input.recovering) {
try {
- response = await request(`file-requests/${input.requestId}`);
+ response = await request(
+ `/finding-imports/file-requests/${input.requestId}`,
+ );
} catch (error) {
if (
!(error instanceof PrivateImportRequestError) ||
@@ -430,7 +441,7 @@ export function privateImportClient(
new Blob([input.bytes], { type: "application/json" }),
"findings.json",
);
- response = await request("files", form);
+ response = await request("/finding-imports/files", form);
}
const file = fileSchema.parse(await response.json());
if (
@@ -451,14 +462,19 @@ export function privateImportClient(
source: ImportSource;
repository: { id: string; repo_connector_id: string };
jobId?: string;
+ jobApiVersion?: 2;
},
accepted: (id: string) => Promise<void>,
) {
+ const jobs =
+ input.jobApiVersion === 2
+ ? "/v2/finding-imports/jobs"
+ : "/finding-imports/jobs";
let id = input.jobId;
let response = id
- ? await request(`jobs/${id}`)
+ ? await request(`${jobs}/${id}`)
: await request(
- "jobs",
+ jobs,
JSON.stringify({
request_id: input.requestId,
context_version: input.contextVersion,
@@ -514,7 +530,7 @@ export function privateImportClient(
undefined,
{ signal },
);
- response = await request(`jobs/${id}`);
+ response = await request(`${jobs}/${id}`);
}
},
};
diff --git a/sdk/typescript/src/scanner-import.ts b/sdk/typescript/src/scanner-import.ts
index f3b5a90c0..f96421714 100644
--- a/sdk/typescript/src/scanner-import.ts
+++ b/sdk/typescript/src/scanner-import.ts
@@ -3,6 +3,7 @@ import { readFile, type FileHandle } from "node:fs/promises";
import { posix } from "node:path";
import { promisify } from "node:util";
import { gunzip } from "node:zlib";
+import validSemver from "semver/functions/valid.js";
import { CodexSecurityError, errorMessage } from "./errors.js";
import type { ExternalFinding, ExternalFindings } from "./external-findings.js";
import { SCAN_SEVERITIES } from "./scan-settings.js";
@@ -147,6 +148,52 @@ export function text(value: unknown): string | null {
return typeof value === "string" && value.trim() ? value : null;
}
+/** AWS APIs use epoch seconds and AWS CLI exports use timezone-qualified ISO dates. */
+export function awsTimestampSeconds(value: unknown): number {
+ if (typeof value === "number") return value;
+ if (
+ typeof value === "string" &&
+ /^\d{4}-\d\d-\d\dT\d\d:\d\d:\d\d(?:\.\d+)?(?:Z|[+-]\d\d:\d\d)$/u.test(value)
+ )
+ return Date.parse(value) / 1000;
+ return NaN;
+}
+
+export function concreteVersion(
+ value: unknown,
+ ecosystem: string | null,
+ osPackage: boolean,
+): string | null {
+ const version = text(value);
+ if (!version) return null;
+ // Debian versions use internal tildes for prereleases and backports.
+ if (osPackage && version.includes("~"))
+ return /^(?:\d+:)?\d[a-z\d.+~]*(?:-[a-z\d.+~]+)*$/iu.test(version)
+ ? version
+ : null;
+ if (validSemver(version)) return version;
+ if (ecosystem === "npm" || ecosystem === "cargo") return null;
+ if (ecosystem === "java" || ecosystem === "maven") {
+ if (
+ version === "LATEST" ||
+ version === "RELEASE" ||
+ version.startsWith("latest.")
+ )
+ return null;
+ return /^[a-z\d][\w.+-]*$/iu.test(version) && !version.endsWith("+")
+ ? version
+ : null;
+ }
+ if (!ecosystem && version.endsWith("+")) return null;
+ const publicVersion =
+ ecosystem === "pypi" ? version.split("+", 1)[0]! : version;
+ return /\d/u.test(version) &&
+ /^(?:\d+:)?[a-z\d][\w.+!-]*$/iu.test(version) &&
+ !/(?:^|\.)x(?:\.|$)/iu.test(publicVersion)
+ ? version
+ : null;
+}
+
export function requiredText(value: unknown, label: string): string {
const result = text(value);
if (!result) throw new Error(`${label} must be nonempty text.`);
diff --git a/sdk/typescript/tests-ts/continuum-findings.test.ts b/sdk/typescript/tests-ts/continuum-findings.test.ts
index 06814b60e..f823477ef 100644
--- a/sdk/typescript/tests-ts/continuum-findings.test.ts
+++ b/sdk/typescript/tests-ts/continuum-findings.test.ts
@@ -2,7 +2,10 @@ import { expect, test } from "bun:test";
import { writeFile } from "node:fs/promises";
import { gzipSync } from "node:zlib";
import { prepareNormalizedLocalImport } from "../src/external-findings-local.js";
-import { normalizeContinuumFindings } from "../src/continuum-findings.js";
+import {
+ normalizeContinuumFindings,
+ normalizePreviousContinuumFindings,
+} from "../src/continuum-findings.js";
import {
createAndReplayFinding,
localReportFixture,
@@ -28,6 +31,7 @@ test("a Continuum list and a detailed finding share the AWS occurrence ARN", ()
expect(list.findings[0]!.evidence).toMatchObject({
title: first.title,
severity: "critical",
+ source_updated_at: Date.parse(first.statusUpdatedAt) / 1000,
locations: [{ path: first.filePath, line: null }],
source_data: { sourceRepositoryUrl: continuumRepository, finding: first },
});
@@ -43,6 +47,7 @@ test("a Continuum list and a detailed finding share the AWS occurrence ARN", ()
source_finding_id: first.findingArn,
evidence: {
severity: "critical",
+ source_updated_at: Date.parse(detail.statusUpdatedAt) / 1000,
description: detail.detailedDescription,
locations: [
{ path: first.filePath, line: 3 },
@@ -63,14 +68,46 @@ test("a Continuum list and a detailed finding share the AWS occurrence ARN", ()
});
});
+test("a selected array retains the same evidence as separate GetCodeFinding responses", () => {
+ const first = continuumDetail();
+ const second = {
+ ...first,
+ ...continuumSummary("two"),
+ statusUpdatedAt: 1_767_312_123.5,
+ };
+ const selected = normalize([first, second]);
+ expect(selected.excluded).toEqual([]);
+ expect(selected.findings).toEqual([
+ ...normalize(first).findings,
+ ...normalize(second).findings,
+ ]);
+ expect(selected.findings[1]!.evidence.source_updated_at).toBe(1_767_312_123);
+ expect(() => normalize([first, continuumSummary("two", "other")])).toThrow(
+ /same environment/,
+ );
+ expect(normalize([first, first]).excluded).toHaveLength(2);
+});
+
test("preserves dependency and posture evidence without classifying it as a code flaw", () => {
const summary = continuumSummary();
+ const pkg = {
+ name: "example",
+ version: "1.0.0",
+ packageManager: "npm",
+ fixedInVersion: "1.0.1",
+ };
const dependency = {
...summary,
ocsfClass: "VULNERABILITY_FINDING",
cveId: "CVE-2099-0001",
- affectedPackages: [
- { name: "example", version: "1.0.0", fixedInVersion: "1.0.1" },
+ affectedPackages: [pkg],
+ vulnerabilities: [
+ {
+ affectedPackages: [
+ pkg,
+ { name: "other", packageManager: "npm", version: "^2.0.0" },
+ ],
+ },
],
};
const posture = {
@@ -85,9 +122,108 @@ test("preserves dependency and posture evidence without classifying it as a code
expect(evidence.details).toBeUndefined();
expect(evidence.source_data?.["finding"]).toEqual(finding);
}
- expect(normalize(dependency).findings[0]!.evidence.advisory_ids).toEqual([
- dependency.cveId,
+ expect(normalize(dependency).findings[0]!.evidence).toMatchObject({
+ advisory_ids: [dependency.cveId],
+ packages: [
+ {
+ name: "example",
+ ecosystem: "npm",
+ installed_version: "1.0.0",
+ fixed_versions: ["1.0.1"],
+ },
+ {
+ name: "other",
+ ecosystem: "npm",
+ installed_version: null,
+ fixed_versions: [],
+ },
+ ],
+ });
+});
+
+test("keeps package-manager selectors in the native finding", () => {
+ const finding = {
+ ...continuumSummary(),
+ affectedPackages: [
+ { name: "example-java", packageManager: "Gradle", version: "1.+" },
+ { name: "example-js", packageManager: "pnpm", version: "1" },
+ ],
+ };
+ const [row] = normalize(finding).findings;
+ expect(row!.evidence.packages?.map((p) => p.installed_version)).toEqual([
+ null,
+ null,
]);
+ expect(row!.evidence.source_data?.["finding"]).toEqual(finding);
+});
+
+test("retains status timestamps without inventing an update time from other AWS fields", () => {
+ const summary = continuumSummary();
+ for (const statusUpdatedAt of [undefined, "unknown", "2026-01-02T00:01:00"]) {
+ const { findings, excluded } = normalize({ ...summary, statusUpdatedAt });
+ expect(excluded).toEqual([]);
+ expect(findings[0]!.evidence.source_updated_at).toBeNull();
+ }
+ const offset = normalize({
+ ...summary,
+ statusUpdatedAt: "2026-01-02T01:01:00.999+01:00",
+ });
+ expect(offset.findings[0]!.evidence.source_updated_at).toBe(
+ Date.parse(summary.statusUpdatedAt) / 1000,
+ );
+});
+
+test("Cloud projects packages within its limits and retains all AWS package evidence", () => {
+ const valid = Array.from({ length: 101 }, (_, i) => ({
+ name: `package-${i}`,
+ }));
+ const finding = {
+ ...continuumSummary(),
+ affectedPackages: [{ name: "界".repeat(200) }, ...valid],
+ };
+ const cloud = normalize(finding);
+ expect(cloud.excluded).toEqual([]);
+ expect(cloud.findings[0]!.evidence.packages?.map((pkg) => pkg.name)).toEqual(
+ valid.slice(0, 100).map((pkg) => pkg.name),
+ );
+ expect(cloud.findings[0]!.evidence.source_data?.["finding"]).toEqual(finding);
+ const local = normalizeContinuumFindings(
+ finding,
+ continuumRepository,
+ "local",
+ );
+ expect(local.findings[0]!.evidence.packages).toHaveLength(102);
+});
+
+test("a Cloud finding that fits with native packages alone still imports", () => {
+ const finding = {
+ ...continuumSummary(),
+ affectedPackages: [
+ { name: "example", version: "1.0.0", packageManager: "npm" },
+ ],
+ padding: "",
+ };
+ const previous = normalizePreviousContinuumFindings(
+ finding,
+ continuumRepository,
+ );
+ const bytes = Buffer.byteLength(
+ JSON.stringify(previous.findings[0]!.evidence),
+ );
+ finding.padding = "x".repeat(256 * 1024 - bytes);
+ const result = normalize(finding);
+ expect(result.excluded).toEqual([]);
+ expect(result.findings[0]!.evidence.packages).toEqual([]);
+ expect(result.findings[0]!.evidence.source_data?.["finding"]).toEqual(
+ finding,
+ );
+ finding.padding = finding.padding.slice(10);
+ const withTimestamp = normalize(finding);
+ expect(withTimestamp.findings[0]!.evidence).toMatchObject({
+ packages: [],
+ source_updated_at: Date.parse(finding.statusUpdatedAt) / 1000,
+ source_data: { finding },
+ });
});
test("unsafe paths stay in native evidence without becoming checkout locations", () => {
diff --git a/sdk/typescript/tests-ts/external-findings-publish.test.ts b/sdk/typescript/tests-ts/external-findings-publish.test.ts
index 3bb644b80..26bf51209 100644
--- a/sdk/typescript/tests-ts/external-findings-publish.test.ts
+++ b/sdk/typescript/tests-ts/external-findings-publish.test.ts
@@ -102,7 +102,11 @@ import {
} from "./support/github-import-fixtures.js";
import { createCliTest } from "./support/cli-run.js";
import { normalizeTrivyFindings } from "../src/trivy-findings.js";
-import { normalizeInspectorFindings } from "../src/inspector-findings.js";
+import {
+ normalizeInspectorFindings,
+ normalizePreviousInspectorFindings,
+} from "../src/inspector-findings.js";
+import { normalizePreviousContinuumFindings } from "../src/continuum-findings.js";
import {
trivyReport,
trivyRepositoryHeader,
@@ -110,6 +114,7 @@ import {
import { prepareNormalizedLocalImport } from "../src/external-findings-local.js";
import { localFindingsFixture } from "./external-findings-local-helpers.js";
import {
+ inspectorCodeFinding,
inspectorReport,
inspectorPackageFinding,
inspectorRepositoryUrl,
@@ -4535,6 +4540,121 @@ test.each(["inspector", "sonar"] as const)(
},
);
+function inspectorReportWithRelatedCves() {
+ const pkg = inspectorPackageFinding();
+ // The previous reader excluded this row because it projected 101 advisory IDs.
+ pkg.packageVulnerabilityDetails.relatedVulnerabilities = Array.from(
+ { length: 100 },
+ (_, i) => `CVE-2099-${3000 + i}`,
+ );
+ return { findings: [inspectorCodeFinding(), pkg] };
+}
+
+test.each([
+ {
+ provider: "inspector",
+ raw: inspectorReportWithRelatedCves(),
+ normalizePrevious: normalizePreviousInspectorFindings,
+ sourceRepositoryUrl: inspectorRepositoryUrl,
+ previousDigest:
+ "4e90748318e5952a1fa90ff9c25153396340a097db6112f27bdf89a3a092dade",
+ expectedCount: 2,
+ },
+ {
+ provider: "continuum",
+ raw: continuumDetail(),
+ normalizePrevious: normalizePreviousContinuumFindings,
+ sourceRepositoryUrl: continuumRepository,
+ previousDigest:
+ "22307dedb51cefecf56d11fd68f45324dceb5226bfddfbd48c635a462bb68fb5",
+ expectedCount: 1,
+ },
+] as const)(
+ "a raw $provider retry preserves the repository request from the previous reader",
+ async ({
+ provider,
+ raw,
+ normalizePrevious,
+ sourceRepositoryUrl,
+ previousDigest,
+ expectedCount,
+ }) => {
+ const f = await fixture(raw, sourceRepositoryUrl);
+ const settings = { ...options, provider, sourceRepositoryUrl };
+ const prepare = () => prepareExternalPublication(f.file, settings, f.deps);
+
+ // Round-trip the previous SDK's output through JSON as a saved caller artifact.
+ const previous = JSON.parse(
+ JSON.stringify(normalizePrevious(raw, sourceRepositoryUrl)),
+ );
+ expect(hash("sha256", canonicalJson(previous.findings))).toBe(
+ previousDigest,
+ );
+ const original = await prepareNormalizedExternalPublication(
+ previous,
+ settings,
+ f.deps,
+ );
+ const pending = await f.loseResponse(original);
+ const request = f.posts[0]!;
+
+ const retry = await prepare();
+ expect(retry.preview.resumed).toBe(true);
+ expect(retry.preview.requests).toEqual(original.preview.requests);
+ expect(await f.loseResponse(retry)).toBe(pending);
+ expect((await (await prepare()).publish()).verified).toBe(1);
+ expect(f.posts).toEqual([request, request, request]);
+
+ const fresh = await prepare();
+ expect(fresh.preview.resumed).toBe(false);
+ expect(fresh.preview.findings).toHaveLength(expectedCount);
+ expect(fresh.preview.findings).not.toEqual(original.preview.findings);
+ },
+);
+
+test("a fresh AWS publication waits for the previous reader to save its request", async () => {
+ const raw = continuumDetail();
+ const f = await fixture(raw, continuumRepository);
+ const settings = {
+ ...options,
+ provider: "continuum" as const,
+ sourceRepositoryUrl: continuumRepository,
+ };
+ const previous = await prepareNormalizedExternalPublication(
+ normalizePreviousContinuumFindings(raw, continuumRepository),
+ settings,
+ f.deps,
+ );
+ const fresh = await prepareExternalPublication(f.file, settings, f.deps);
+ const observer = observeLockContention();
+ const rename = fs.rename;
+ let competitor: Promise<unknown> | undefined;
+ let outcome: string | undefined;
+ const install = spyOn(fs, "rename").mockImplementationOnce(
+ async (...args) => {
+ // The earlier publisher holds its lock but has not installed its checkpoint yet.
+ competitor = fresh.publish().catch((error: unknown) => error);
+ outcome = await Promise.race([
+ observer.contended.then(() => "locked"),
+ competitor.then(() => "finished"),
+ ]);
+ return rename(...args);
+ },
+ );
+ try {
+ f.state.throttle = true;
+ await expect(previous.publish()).rejects.toThrow("Slow down");
+ expect(outcome).toBe("locked");
+ expect(String(await competitor)).toContain(
+ "Another publication prepared this input",
+ );
+ expect(f.posts).toHaveLength(1);
+ } finally {
+ install.mockRestore();
+ observer.restore();
+ }
+});
+
const savedBlackDuckEvidence = {
data: { format: "blackduck-bom-6" },
requiredProvider: "blackduck",
@@ -11147,6 +11267,29 @@ test("the GitHub reader resumes a released boxed score whose Buffer fit the Clou
});
test.each([
+ {
+ provider: "github",
+ format: "security.findings.v1",
+ raw: [importedAlert()],
+ },
+ {
+ provider: "inspector",
+ format: "security.findings.v1",
+ raw: inspectorReportWithRelatedCves(),
+ sourceRepositoryUrl: inspectorRepositoryUrl,
+ previousReader: normalizePreviousInspectorFindings,
+ previousSha256:
+ "91b5a4d309daee520029a3d54e809ad84d8b47ee97df5fd54d56323fc246f640",
+ },
+ {
+ provider: "continuum",
+ format: "security.findings.v1",
+ raw: continuumDetail(),
+ sourceRepositoryUrl: continuumRepository,
+ previousReader: normalizePreviousContinuumFindings,
+ previousSha256:
+ "98f888f8b748bb54a209b19de53af40867f5f129049a99da0387e697895aa320",
+ },
{ provider: "snyk", format: "security.findings.v1", raw: snykSarif() },
{
provider: "endor",
@@ -11168,16 +11311,15 @@ test.each([
},
] as const)(
"a signed-in $provider private import resumes lost upload and job responses",
- async ({ provider, format, raw }) => {
+ async (scenario) => {
+ const { provider, format, raw } = scenario;
+ const sourceRepositoryUrl =
+ "sourceRepositoryUrl" in scenario
+ ? scenario.sourceRepositoryUrl
+ : undefined;
const f = await fixture(raw);
const originalBytes = await readFile(f.file, "utf8");
- const settings = { ...options, provider };
- const repositoryPreview = await prepareExternalPublication(
- f.file,
- settings,
- f.deps,
- );
- f.state.environmentId = null;
+ const settings = { ...options, provider, sourceRepositoryUrl };
let contextVersion = "generation-a";
let formats = [
"security.findings.v1",
@@ -11188,12 +11330,21 @@ test.each([
let loseResponses = true;
let unavailable = false;
let terminal: "completed" | "failed" | "cancelled" = "completed";
- let advertised = true;
+ let advertised = false;
+ const v1Jobs = "/finding-imports/jobs";
+ const v2Jobs = "/v2" + v1Jobs;
+ let supportsJobV2 = provider !== "inspector";
+ let expectedJobsPath = provider === "continuum" ? v2Jobs : v1Jobs;
let canCreate = true;
let fileRequest: string | undefined;
let fileHash: string | undefined;
let fileBytes = 0;
+ let reportCount = 1;
+ let unorderedUpdates = 0;
+ const uploadedReports: { id: string; bytes: string }[] = [];
let jobRequest: string | undefined;
+ let loseNextJobResponse = true;
+ const jobBodies: string[] = [];
const fileId = "aifile_" + "a".repeat(32);
let jobId = "import_" + "b".repeat(32);
const calls: string[] = [];
@@ -11219,13 +11370,13 @@ test.each([
],
status,
counts: {
- total: 1,
- discovered: 1,
- created: status === "completed" ? 1 : 0,
+ total: reportCount,
+ discovered: reportCount,
+ created: status === "completed" ? reportCount - unorderedUpdates : 0,
updated: 0,
unchanged: 0,
- error: 0,
- unprocessed: status === "completed" ? 0 : 1,
+ error: status === "completed" ? unorderedUpdates : 0,
+ unprocessed: status === "completed" ? 0 : reportCount,
},
poll_after_seconds: null,
error: null,
@@ -11244,15 +11395,18 @@ test.each([
calls.push(`${init.method} ${path}`);
if (path === "/external/repositories") {
const response = await f.deps.fetch(url, init);
- return new Response(response.body, {
- headers: advertised
- ? {
- "Codex-Security-Private-Import-Providers":
- "snyk,github,endor,wiz",
- }
- : {},
- });
+ const headers = new Headers();
+ if (advertised)
+ headers.set(
+ "Codex-Security-Private-Import-Providers",
+ "snyk,github,endor,wiz,inspector,continuum",
+ );
+ const version = "Codex-Security-Private-Import-Job-Version";
+ expect(new Headers(init.headers).get(version)).toBe("2");
+ if (supportsJobV2) headers.set(version, "2");
+ return new Response(response.body, { headers });
}
+ if (path.startsWith("/external/")) return f.deps.fetch(url, init);
if (path === "/finding-imports/context")
return Response.json({
object: "security.finding_import_context",
@@ -11277,19 +11431,42 @@ test.each([
expect(doc.format).toBe(expectedFormat);
expect(doc.findings[0].source_finding_id).toBeTruthy();
expect(doc.findings[0].evidence.source_data).toBeTruthy();
+ reportCount = doc.findings.length;
+ if (provider === "inspector" || provider === "continuum") {
+ const native = provider === "inspector" ? raw.findings : [raw];
+ const expected = loseResponses ? native.slice(0, 1) : native;
+ expect(doc.findings).toHaveLength(expected.length);
+ for (const row of expected) {
+ const finding = doc.findings.find(
+ (item: { source_finding_id: string }) =>
+ item.source_finding_id === row.findingArn,
+ );
+ expect(finding?.evidence.source_data.finding).toEqual(row);
+ }
+ }
} else {
expect(bytes).toBe(originalBytes);
}
+ uploadedReports.push({ id: fileRequest, bytes });
fileHash = hash("sha256", bytes);
fileBytes = Buffer.byteLength(bytes);
- if (loseResponses) throw new Error("lost upload response");
+ if (loseResponses && uploadedReports.length === 1)
+ throw new Error("lost upload response");
return Response.json(fileMetadata());
}
- if (path === `/finding-imports/file-requests/${fileRequest}`)
+ if (path === `/finding-imports/file-requests/${fileRequest}`) {
+ if ("previousSha256" in scenario && uploadedReports.length === 1)
+ return new Response(null, { status: 404 });
return Response.json(fileMetadata());
- if (path === "/finding-imports/jobs") {
+ }
+ if (path === expectedJobsPath) {
+ jobBodies.push(String(init.body));
const body = JSON.parse(String(init.body));
expect(body.request_id).not.toBe(fileRequest);
+ expect(body.source).toEqual({
+ provider,
+ source_key: settings.sourceKey,
+ });
expect(body.input).toEqual({
type: "file",
file_id: fileId,
@@ -11311,16 +11488,17 @@ test.each([
: "import_" + "c".repeat(32);
return Response.json(job(terminal));
}
- if (!jobRequest) {
- jobRequest = body.request_id;
+ if (jobRequest) expect(body.request_id).toBe(jobRequest);
+ jobRequest = body.request_id;
+ if (loseNextJobResponse) {
+ loseNextJobResponse = false;
throw new Error("lost job response");
}
- expect(body.request_id).toBe(jobRequest);
return Response.json(job("queued"), {
headers: { "Retry-After": "0" },
});
}
- if (path === `/finding-imports/jobs/${jobId}`)
+ if (path === `${expectedJobsPath}/${jobId}`)
return unavailable
? new Response(null, { status: 503 })
: Response.json(job(terminal));
@@ -11328,11 +11506,16 @@ test.each([
},
};
const prepare = () => prepareExternalPublication(f.file, settings, deps);
+ const repositoryPreview = await prepare();
+ f.state.environmentId = null;
+ advertised = true;
const original = await prepare();
expect(original.preview.privateImport?.sha256).toBeTruthy();
expect(original.preview.privateImport?.format).toBe(format);
const cli = createCliTest(main);
const dryRun = [...f.command, "--dry-run"];
+ if (sourceRepositoryUrl)
+ dryRun.push("--source-repository-url", sourceRepositoryUrl);
dryRun[dryRun.indexOf("--provider") + 1] = provider;
dryRun[dryRun.indexOf("--format") + 1] = "toon";
expect(
@@ -11347,32 +11530,114 @@ test.each([
canCreate = false;
await expect(prepare()).rejects.toThrow("unavailable");
canCreate = true;
- await expect((await prepare()).publish()).rejects.toThrow(
- "lost upload response",
- );
+ // Stage the older AWS reader's serialized evidence to reproduce an upgrade
+ // after upload. A fresh context below must use the current reader again.
+ const staged =
+ scenario.previousReader && scenario.sourceRepositoryUrl
+ ? await prepareNormalizedExternalPublication(
+ JSON.parse(
+ JSON.stringify(
+ scenario.previousReader(raw, scenario.sourceRepositoryUrl),
+ ),
+ ),
+ settings,
+ deps,
+ )
+ : await prepare();
+ const interrupted = await staged.publish().catch((error: unknown) => error);
+ if (!(interrupted instanceof ExternalPublicationError))
+ throw new Error("Expected an interrupted private upload", {
+ cause: interrupted,
+ });
+ expect(interrupted.message).toContain("lost upload response");
+ const saved = JSON.parse(
+ await readFile(interrupted.result.savedSubmission!, "utf8"),
+ );
+ expect(saved.jobApiVersion).toBe(provider === "continuum" ? 2 : undefined);
+ expect(fileHash).toBe(saved.sha256);
+ expect(fileRequest).toBe(saved.uploadRequestId);
+ if ("previousSha256" in scenario) {
+ expect(fileHash).toBe(scenario.previousSha256);
+ expect(fileHash).not.toBe(original.preview.privateImport?.sha256);
+ }
await expect(repositoryPreview.publish()).rejects.toThrow(
- "A private import already owns this selection",
+ "previousSha256" in scenario
+ ? "Another publication prepared this input"
+ : "A private import already owns this selection",
);
expect(f.posts).toHaveLength(0);
advertised = false;
- await expect((await prepare()).publish()).rejects.toThrow(
- "lost job response",
- );
+ const resumed = await prepare();
+ expect(resumed.preview.resumed).toBe(true);
+ expect(resumed.preview.privateImport?.sha256).toBe(saved.sha256);
+ await expect(resumed.publish()).rejects.toThrow("lost job response");
+ expect(jobRequest).toBe(saved.jobRequestId);
+ expect(calls.at(-1)).toBe(`POST ${expectedJobsPath}`);
+ if (provider === "inspector") {
+ // The upgraded server accepts the same job request only on v2.
+ supportsJobV2 = true;
+ expectedJobsPath = v2Jobs;
+ loseNextJobResponse = true;
+ await expect((await prepare()).publish()).rejects.toThrow(
+ "lost job response",
+ );
+ expect(calls.at(-1)).toBe(`POST ${v2Jobs}`);
+ expect(jobBodies).toHaveLength(2);
+ expect(jobBodies[1]).toBe(jobBodies[0]);
+ const promoted = JSON.parse(
+ await readFile(interrupted.result.savedSubmission!, "utf8"),
+ );
+ expect(promoted).toMatchObject({
+ jobApiVersion: 2,
+ jobRequestId: saved.jobRequestId,
+ uploadRequestId: saved.uploadRequestId,
+ sha256: saved.sha256,
+ fileId,
+ });
+ }
+ // A saved v2 job can still be retrieved if the catalogue acknowledgement disappears.
+ supportsJobV2 = false;
const completed = await (await prepare()).publish();
expect(completed.privateJob?.counts.created).toBe(1);
+ expect(calls.at(-1)).toBe(`GET ${expectedJobsPath}/${jobId}`);
expect(completed.receipts).toEqual([]);
expect(completed.verified).toBe(0);
- expect(
- calls.filter((call) => call === "POST /finding-imports/files"),
- ).toHaveLength(1);
+ if ("previousSha256" in scenario) {
+ expect(uploadedReports).toHaveLength(2);
+ expect(uploadedReports[1]).toEqual(uploadedReports[0]);
+ } else {
+ expect(uploadedReports).toHaveLength(1);
+ }
+ if (provider === "inspector") {
+ // A completed old job cannot account for the newly admitted package finding.
+ advertised = true;
+ supportsJobV2 = true;
+ loseResponses = false;
+ const next = await prepare();
+ expect(next.preview.resumed).toBe(false);
+ expect(next.preview.privateImport?.sha256).toBe(
+ original.preview.privateImport?.sha256,
+ );
+ // The code finding's changed evidence has no baseline. Only the new ARN
+ // can be created in this workspace.
+ unorderedUpdates = 1;
+ const nextResult = await next.publish();
+ expect(nextResult.privateJob?.id).not.toBe(completed.privateJob?.id);
+ expect(nextResult.privateJob?.counts).toMatchObject({
+ created: 1,
+ error: 1,
+ });
+ }
canCreate = false;
+ advertised = false;
+ supportsJobV2 = true;
expect((await (await prepare()).publish()).privateJob?.id).toBe(jobId);
- expect(calls.at(-1)).toBe(`GET /finding-imports/jobs/${jobId}`);
+ expect(calls.at(-1)).toBe(`GET ${expectedJobsPath}/${jobId}`);
unavailable = true;
const acknowledged = {
result: {
status: "interrupted",
- counts: { created: 1 },
+ counts: { created: reportCount - unorderedUpdates },
unacknowledged: 0,
privateJob: { id: jobId },
},
@@ -11386,15 +11651,42 @@ test.each([
formats = ["security.findings.v1"];
expect((await prepare()).preview.privateImport?.format).toBe(format);
contextVersion = "generation-c";
+ expectedJobsPath =
+ provider === "inspector" || provider === "continuum" ? v2Jobs : v1Jobs;
+ unorderedUpdates = 0;
+ advertised = true;
canCreate = true;
loseResponses = false;
expectedFormat = "security.findings.v1";
- terminal = provider === "wiz" ? "cancelled" : "failed";
+ terminal =
+ provider === "wiz" || provider === "continuum" ? "cancelled" : "failed";
const fresh = await prepare();
expect(fresh.preview.resumed).toBe(false);
expect(fresh.preview.privateImport?.format).toBe(expectedFormat);
+ if ("previousSha256" in scenario)
+ expect(fresh.preview.privateImport?.sha256).toBe(
+ original.preview.privateImport?.sha256,
+ );
expect((await fresh.publish()).privateJob?.status).toBe(terminal);
+ expect(fileHash).toBe(fresh.preview.privateImport?.sha256);
const failedJob = jobId;
+ if (expectedJobsPath === v2Jobs) {
+ // Losing AWS write access still allows the failed or cancelled job to be read.
+ advertised = false;
+ const before = calls.length;
+ const readOnly = await prepare();
+ expect(readOnly.preview.privateImport?.retryOfJobId).toBeUndefined();
+ expect((await readOnly.publish()).privateJob).toMatchObject({
+ id: failedJob,
+ status: terminal,
+ });
+ expect(calls.at(-1)).toBe(`GET ${v2Jobs}/${failedJob}`);
+ expect(
+ calls.slice(before).filter((call) => call.startsWith("POST")),
+ ).toEqual([]);
+ advertised = true;
+ supportsJobV2 = false; // The saved version also applies to its replacement attempt.
+ }
const retry = await prepare();
const secondPreview = await prepare();
expect(retry.preview.resumed).toBe(false);
@@ -11415,7 +11707,7 @@ test.each([
await expect(retry.publish()).rejects.toMatchObject({
result: {
status: "interrupted",
- counts: { created: 1 },
+ counts: { created: reportCount },
unacknowledged: 0,
privateJob: { status: "completed" },
},
diff --git a/sdk/typescript/tests-ts/inspector-findings.test.ts b/sdk/typescript/tests-ts/inspector-findings.test.ts
index a659ac2c5..863553e65 100644
--- a/sdk/typescript/tests-ts/inspector-findings.test.ts
+++ b/sdk/typescript/tests-ts/inspector-findings.test.ts
@@ -8,9 +8,11 @@ import type { ExternalImportTarget } from "../src/external-import-contract.js";
import {
inspectorReportRepository,
normalizeInspectorFindings,
+ normalizePreviousInspectorFindings,
readInspectorFindings,
} from "../src/inspector-findings.js";
import {
+ inspectorCodeFinding as codeFinding,
inspectorIntegrationArn,
inspectorPackageFinding,
inspectorProjectId,
@@ -27,28 +29,6 @@ function normalize(
return normalizeInspectorFindings({ findings }, source, target, projectMap);
}
-function codeFinding(
- filePath = "src/example.ts",
- fileName = "example.ts",
- startLine = 8,
-) {
- const { packageVulnerabilityDetails: _packages, ...common } =
- inspectorPackageFinding();
- return {
- ...common,
- type: "CODE_VULNERABILITY",
- findingArn: common.findingArn.replace(/a{32}$/u, "b".repeat(32)),
- title: "Synthetic code finding",
- codeVulnerabilityDetails: {
- detectorId: "synthetic-detector",
- detectorName: "Synthetic detector",
- cwes: ["CWE-79"],
- detectorTags: ["example-tag"],
- filePath: { fileName, filePath, startLine, endLine: 12 },
- },
- };
-}
-
function projectFinding(
repoChanges: Record<string, unknown>,
resourceChanges: Record<string, unknown> = {},
@@ -74,7 +54,7 @@ test("preserves the native package finding and projects only reported dependency
title: native.title,
severity: "high",
description: native.description,
- advisory_ids: ["CVE-2099-1234", "GHSA-2345-6789-cfgh"],
+ advisory_ids: ["CVE-2099-1234"],
url: native.packageVulnerabilityDetails.sourceUrl,
packages: [
{
@@ -111,6 +91,30 @@ test("preserves the native package finding and projects only reported dependency
});
});
+test("related vulnerabilities stay in native evidence even when they exceed the Cloud advisory ID limit", () => {
+ const native = inspectorPackageFinding();
+ native.packageVulnerabilityDetails.relatedVulnerabilities = Array.from(
+ { length: 100 },
+ (_, i) => `CVE-2099-${4000 + i}`,
+ );
+ const report = { findings: [native] };
+ const { findings, excluded } = normalizeInspectorFindings(
+ report,
+ inspectorRepositoryUrl,
+ );
+ expect(excluded).toEqual([]);
+ expect(findings[0]!.evidence.advisory_ids).toEqual([
+ native.packageVulnerabilityDetails.vulnerabilityId,
+ ]);
+ expect(findings[0]!.evidence.source_data?.["finding"]).toEqual(native);
+ const previous = normalizePreviousInspectorFindings(
+ report,
+ inspectorRepositoryUrl,
+ );
+ expect(previous.findings).toHaveLength(0);
+ expect(previous.excluded[0]?.reason).toContain("advisory_ids");
+});
+
test.each([
["NPM", "1.2.3+build.x.1", "2.x", "1.2.3+build.x.1", []],
["YARN", "workspace:1.2.3", "1.2.3-rc.x.1", null, ["1.2.3-rc.x.1"]],
@@ -214,6 +218,29 @@ test("projects only repository-relative code locations with a matching filename"
const evidence = (finding: ReturnType<typeof codeFinding>) =>
normalize([finding]).findings[0]!.evidence;
const code = codeFinding();
+ expect(evidence(code).details).toMatchObject({
+ kind: "sast",
+ repository: { url: inspectorRepositoryUrl },
+ rule: {
+ id: code.codeVulnerabilityDetails.detectorId,
+ name: code.codeVulnerabilityDetails.detectorName,
+ },
+ weakness_ids: code.codeVulnerabilityDetails.cwes,
+ code: { end_line: 12 },
+ });
+ expect(
+ evidence(codeFinding("src/other.ts")).details?.code?.end_line ?? null,
+ ).toBeNull();
+ expect(
+ evidence(codeFinding("src/example.ts", "example.ts", 13)).details?.code
+ ?.end_line ?? null,
+ ).toBeNull();
+ expect(
+ normalizePreviousInspectorFindings(
+ { findings: [code] },
+ inspectorRepositoryUrl,
+ ).findings[0]!.evidence.details,
+ ).toBeUndefined();
expect(evidence(code).locations).toEqual([
{ path: "src/example.ts", line: 8 },
]);
@@ -235,6 +262,37 @@ test("projects only repository-relative code locations with a matching filename"
).toEqual([{ path: "src/example.ts", line: null }]);
});
+test("oversized optional SAST fields preserve the complete Inspector finding", () => {
+ const native = codeFinding();
+ native.codeVulnerabilityDetails.detectorId = "a".repeat(513);
+ const { findings, excluded } = normalize([native]);
+ expect(excluded).toEqual([]);
+ expect(findings[0]!.evidence.source_data?.["finding"]).toEqual(native);
+ expect(findings[0]!.evidence.locations).toEqual([
+ { path: "src/example.ts", line: 8 },
+ ]);
+ expect(
+ normalize([native], inspectorRepositoryUrl, "local").findings[0]!.evidence
+ .details?.rule?.id,
+ ).toBe(native.codeVulnerabilityDetails.detectorId);
+});
+
+test("SAST details do not displace native evidence at the Cloud byte limit", () => {
+ const native = { ...codeFinding(), padding: "" };
+ const previous = normalizePreviousInspectorFindings(
+ { findings: [native] },
+ inspectorRepositoryUrl,
+ );
+ native.padding = " ".repeat(
+ 262_144 - Buffer.byteLength(JSON.stringify(previous.findings[0]!.evidence)),
+ );
+ expect(
+ normalize([native]).findings[0]!.evidence.source_data?.["finding"],
+ ).toEqual(native);
+ native.padding += " ";
+ expect(normalize([native]).excluded[0]?.reason).toContain("256 KiB");
+});
+
test("accepts Unix and timezone-qualified ISO timestamps", () => {
const native = inspectorPackageFinding();
const epoch = Date.parse(native.updatedAt) / 1000;
diff --git a/sdk/typescript/tests-ts/support/inspector-import-fixtures.ts b/sdk/typescript/tests-ts/support/inspector-import-fixtures.ts
index 5f44c8ec5..b22becb53 100644
--- a/sdk/typescript/tests-ts/support/inspector-import-fixtures.ts
+++ b/sdk/typescript/tests-ts/support/inspector-import-fixtures.ts
@@ -51,6 +51,28 @@ export function inspectorPackageFinding() {
};
}
+export function inspectorCodeFinding(
+ filePath = "src/example.ts",
+ fileName = "example.ts",
+ startLine = 8,
+) {
+ const { packageVulnerabilityDetails: _packages, ...common } =
+ inspectorPackageFinding();
+ return {
+ ...common,
+ type: "CODE_VULNERABILITY",
+ findingArn: common.findingArn.replace(/a{32}$/u, "b".repeat(32)),
+ title: "Synthetic code finding",
+ codeVulnerabilityDetails: {
+ detectorId: "synthetic-detector",
+ detectorName: "Synthetic detector",
+ cwes: ["CWE-79"],
+ detectorTags: ["example-tag"],
+ filePath: { fileName, filePath, startLine, endLine: 12 },
+ },
+ };
+}
+
export function inspectorReport() {
return { findings: [inspectorPackageFinding()] };
}