Skip to content

OpenAI's Codex Security CLI and TypeScript SDK for finding, validating, and fixing security vulnerabilities. npm: https://www.npmjs.com/package/@openai/codex-security

fix(imports): verify scanner evidence and route workspace findings (#2353)

Michael D'Angelo765a5de4b1ab
diff --git a/docs/scanner-imports/README.md b/docs/scanner-imports/README.md
index 50133e2fd..d41374d03 100644
--- a/docs/scanner-imports/README.md
+++ b/docs/scanner-imports/README.md
@@ -34,7 +34,7 @@ Ask it to preview the report with your configured model.
With imports enabled, run `codex-security publish findings --help` to check the
providers in your installed CLI. The guides cover local and Cloud imports
-unless an input is marked local only. Cloud must support the selected provider. For AWS inbox imports, use a CLI and workspace that support v2 finding import jobs. Check for `privateImport` in the dry-run preview.
+unless an input is marked local only.
| Guide | Supported input |
| ----------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
@@ -74,6 +74,14 @@ Set `--to cloud` and pass the connected repository URL or Cloud ID to
`--repository`. Preview again with `--dry-run`. Check the account, endpoint,
destination, findings, and exclusions. Remove `--dry-run` and confirm. [Cloud setup](../../sdk/typescript/docs/cli.md#preview-and-publish)
+Add `--format json` to see which destination your workspace offers.
+When the preview includes `privateImport`, confirming sends the findings to
+your private inbox with the selected repository and connector. GitHub code scanning,
+Dependabot and advisories use the GitHub provider. OpenGrep uses the Semgrep
+provider. For Amazon Inspector or AWS Continuum, the CLI and workspace must
+support v2 import jobs. For Black Duck, Checkmarx, SonarQube Cloud, Trivy or
+Veracode, they must support v3 to use the private inbox.
+
### Verify the import
For a local import, list the saved findings using the checkout path from the preview.
@@ -89,6 +97,10 @@ For a Cloud import, check the receipt for the destination shown in the preview.
| Private inbox (`privateImport`) | Check that `privateJob.status` is `completed` and its counts account for the selected findings, with zero errors or unprocessed rows. Review exclusions. |
| Repository publisher (environments in `requests`) | Check each receipt, the verified and unacknowledged counts, failures, and exclusions. |
+If the repository publisher requires a reported severity, the preview excludes
+findings whose severity the scanner left unknown. You can keep those findings
+in a local import or a private inbox when available.
+
If `publish findings` stops and returns `savedSubmission`, [resume from that checkpoint](schema-guide.md#retry-a-cloud-submission).
## Troubleshoot Semgrep and Snyk collection
diff --git a/docs/scanner-imports/blackduck.md b/docs/scanner-imports/blackduck.md
index 15973154e..1b60b094a 100644
--- a/docs/scanner-imports/blackduck.md
+++ b/docs/scanner-imports/blackduck.md
@@ -42,7 +42,10 @@ later exports of the same project version.
The project version, component version, origin, advisory source and advisory
name identify each occurrence. Package URLs and origin coordinates identify
-the package. `advisory_ids` keeps `vulnerabilityName` and any distinct
+the package. For a GitHub dependency, a `pkg:github/owner/repo@tag` package URL
+supplies the dependency's owner, repository name and installed tag or commit.
+`--source-repository-url` identifies the repository Black Duck scanned.
+`advisory_ids` keeps `vulnerabilityName` and any distinct
`relatedVulnerability` verbatim, including Black Duck resource URLs.
Look in the original row for CWE, matched-files links and remediation
diff --git a/docs/scanner-imports/endor.md b/docs/scanner-imports/endor.md
index 47fca559f..ef7412f3d 100644
--- a/docs/scanner-imports/endor.md
+++ b/docs/scanner-imports/endor.md
@@ -6,7 +6,18 @@ that Endor reports. You can also import self-contained `endorctl` SARIF 2.1.0
with finding UUIDs. A later SARIF import for the same UUID, destination
and source key can replace the stored commit, package and
advisory details, and leave severity `unknown`. Use native JSON for updates
-when you need those fields.
+when you need those fields. If the Cloud repository publisher requires a
+reported severity, the preview lists unscored findings under exclusions. Local
+imports and the private inbox can retain findings with an unreported severity.
+
+Native JSON keeps vulnerability and malware advisory IDs and aliases. When
+Endor supplies only the fully qualified dependency name, the import preserves
+that name, including its ecosystem and version. A branch scan uses its REF
+context ID. The source version's ref can also identify a tag or commit. If
+optional package or advisory details exceed Cloud's field or finding size
+limits, new private imports use `security.findings.v1` to upload the reviewed
+fields and original Endor record together. The service must support that
+format for these findings. Retrying a saved import reuses its original upload.
## Quick start
diff --git a/docs/scanner-imports/github-code-scanning.md b/docs/scanner-imports/github-code-scanning.md
index e47a3b6f5..dbc5b3cef 100644
--- a/docs/scanner-imports/github-code-scanning.md
+++ b/docs/scanner-imports/github-code-scanning.md
@@ -89,6 +89,8 @@ console.log(prepared.preview);
```
After reviewing the Cloud preview, call `await prepared.publish()` to confirm.
+If you supply normalized evidence that keeps the collector record in
+`source_data`, imports check its original repository against the destination.
For local storage, call `normalizeGitHubCodeScanningAlerts(alerts, "local")` on
the original alerts to retain evidence that exceeds Cloud limits. Pass the result
diff --git a/docs/scanner-imports/github-security-advisories.md b/docs/scanner-imports/github-security-advisories.md
index a56276d20..c826c56f5 100644
--- a/docs/scanner-imports/github-security-advisories.md
+++ b/docs/scanner-imports/github-security-advisories.md
@@ -90,4 +90,6 @@ pass `normalizeGitHubAdvisories(advisories, "cloud")` to
`importGitHubAdvisories()` defaults to `{ source: "repository", state: "published" }`; use
`{ source: "global", advisoryIds: ["GHSA-2345-6789-cfgh"] }` for selected database
records. Both forms require `repository: "OWNER/REPO"` as source or target context.
+When normalized evidence retains a collector record in `source_data`, imports
+check a repository disclosure's original repository against the destination.
Supply `githubHost` and `githubToken` to override the GitHub CLI login and host.
diff --git a/docs/scanner-imports/schema-guide.md b/docs/scanner-imports/schema-guide.md
index 9f8e203dd..97e57bb27 100644
--- a/docs/scanner-imports/schema-guide.md
+++ b/docs/scanner-imports/schema-guide.md
@@ -81,10 +81,19 @@ repository path from the preview. Each record includes
unprocessed findings. Open a finding in the inbox to assess it and publish
the assessment when you're ready to share it.
+Reimporting identical evidence for the same occurrence leaves the saved
+observation in place. If the job reports `source_order_unknown`, Cloud
+received different evidence for an existing finding and needs its current
+observation version before replacing it. Check the previous observation and
+the new evidence before retrying. A newer scanner timestamp alone cannot
+authorize that update.
+
**Cloud repository publisher.** Each item includes a `source_finding_id` and an `expected_version`.
Check the per-item result for conflicts. The receipt's source-report,
observation and canonical-finding IDs identify Cloud records. Read the source
-report for the imported evidence.
+report for the imported evidence. If this destination requires a known
+severity, findings whose scanner reported no severity appear in the preview's
+exclusions. The private inbox also accepts unknown severity.
The repository publisher allows up to 256 KiB and 20 levels of nesting per finding. The publisher
batches up to 100 items and 5 MiB per request. Local file imports do not apply
diff --git a/docs/scanner-imports/semgrep.md b/docs/scanner-imports/semgrep.md
index f3bb3b8f7..933cd1500 100644
--- a/docs/scanner-imports/semgrep.md
+++ b/docs/scanner-imports/semgrep.md
@@ -64,9 +64,12 @@ creates a new record. [Imported evidence and identity](schema-guide.md)
For CLI/CI JSON and SARIF, use SAST-only rules. The reader excludes results
it recognizes as Supply Chain or Secrets, but SARIF can omit product metadata.
Scan errors or an unsuccessful SARIF invocation stop the import. Direct artifact URIs and native
-`%SRCROOT%` paths can supply locations. Indexed artifacts and unresolved bases
+`%SRCROOT%` paths can supply locations. The reader preserves the scanner's path
+spelling, including backslashes in POSIX filenames and Windows paths.
+Indexed artifacts and unresolved bases
stay in the raw evidence and need a fingerprint for identity. Keep the original
-report for run-wide inventories and timing.
+report for run-wide inventories, matching explanations and timing. Individual
+findings keep their own match data.
If you call the JSON or SARIF normalizer from the SDK, pass `sourceRepositoryUrl`.
When calling `prepareNormalizedLocalImport` directly, pass the original SARIF
@@ -286,6 +289,7 @@ The SDK preserves large positive IDs exactly. Key order and equivalent numeric
spellings (such as `2` and `2.0`) leave the stored record unchanged.
The preview excludes findings that lack both a native rule name and a message.
+It verifies the selected project's repository even when every finding is excluded.
Only repository-relative files become locations. URI-like and directory-only
paths stay in the native evidence.
diff --git a/docs/scanner-imports/snyk.md b/docs/scanner-imports/snyk.md
index 95ba121aa..61c950b5f 100644
--- a/docs/scanner-imports/snyk.md
+++ b/docs/scanner-imports/snyk.md
@@ -223,6 +223,8 @@ plugin scan's `issueCount`, can update a stored snapshot of the same
occurrence. [Imported evidence and identity](schema-guide.md)
Known source Git URLs must match a comparable local remote when available,
-or the selected Cloud repository. For Snyk Code, use the
+or the selected Cloud repository. Plugin and SARIF reports can identify a
+connected GitHub repository by its HTTPS or standard Git SSH clone URL.
+For Snyk Code, use the
[monitored project collector](#existing-issues-from-snyk). Container, IaC and
SBOM results fall outside these import routes.
diff --git a/docs/scanner-imports/trivy.md b/docs/scanner-imports/trivy.md
index 2e917139a..32c29a8ed 100644
--- a/docs/scanner-imports/trivy.md
+++ b/docs/scanner-imports/trivy.md
@@ -27,6 +27,7 @@ Use credential-free repository URLs. If the importer reports ambiguous or confli
## Read the evidence and repeat an import
- Each finding retains its native vulnerability, exact `FixedVersion` expression and, when available, the matched package and declaration lines. Ambiguous archive paths stay in the raw evidence.
+- A package can appear many times in the same lockfile. Cloud projects the first 100 locations per finding. The full list of declaration ranges is in `source_data.package.Locations`.
- Trivy can scan uncommitted files. Its reported commit stays in `source_data.report.Metadata.Commit`. Imported findings start as `not_assessed`, including when Trivy reports an upstream fix.
- Changes to severity, installed version or declaration line update the same occurrence. A different manifest or package path identifies a separate occurrence. Duplicate identities are excluded as a group. Omitted findings stay open.
- With the SDK, use `readTrivyFindings` for a file or `normalizeTrivyFindings` for parsed JSON. When selecting rows from a repository-bound report, retain its native header and each row’s `source_data.report` or `details.repository.url`, and import with provider `trivy`. Empty and fully excluded reports also retain their native header.
diff --git a/docs/scanner-imports/veracode.md b/docs/scanner-imports/veracode.md
index e6042ff88..ad94249f2 100644
--- a/docs/scanner-imports/veracode.md
+++ b/docs/scanner-imports/veracode.md
@@ -45,9 +45,14 @@ from the same application and context.
The API origin, application, context and native `issue_id` identify a policy
flaw. Build number, source line, CWE and Pipeline flaw ID are separate fields.
The original row keeps the numeric severity, CWE, description, status,
-mitigation and resolution. Veracode reports the static analysis path, line
-and module. Check the path against the checkout yourself. The import supplies
-no column or Git commit.
+mitigation and resolution. The reader accepts the numeric CWE and category name
+defined in the Findings API schema, as well as expanded CWE and category objects
+in HAL exports. [Findings API schema](https://api.swaggerhub.com/apis/Veracode/veracode-findings_api_specification/2.1)
+
+If you export with a numeric category filter, the rows must include category
+IDs so the reader can verify the selection. Veracode reports the static analysis
+path, line and module. Check the path against the checkout yourself. The import
+supplies no column or Git commit.
A Veracode resolution records upstream status. Assess the fix separately.
The workbench keeps flaws omitted from later exports.
diff --git a/docs/scanner-imports/wiz.md b/docs/scanner-imports/wiz.md
index 966b830d9..ab699ce9d 100644
--- a/docs/scanner-imports/wiz.md
+++ b/docs/scanner-imports/wiz.md
@@ -64,6 +64,11 @@ collection. JSONL can hold one package record per line. Preserve `id`, `name`,
available package, image and repository metadata. Use the Wiz finding ID for
identity. Many package occurrences can share a CVE.
+For GitHub Enterprise branch assets, keep `vulnerableAsset.cloudProviderURL`
+and either `nativeType: "github#repositoryBranch"` or both
+`type: "REPOSITORY_BRANCH"` and `cloudPlatform: "GitHub"`. The reader uses
+the repository in that URL to check the import destination.
+
Detailed package CSV requires `ID`, `Name`, `DetailedName`, `AssetID`, `WizURL`,
`CloudPlatform` and `CloudProviderURL`. The platform must be GitHub and the
provider URL must identify a repository branch. That repository must match a
diff --git a/sdk/typescript/src/blackduck-findings.ts b/sdk/typescript/src/blackduck-findings.ts
index 5ee3d98d3..65d28f84b 100644
--- a/sdk/typescript/src/blackduck-findings.ts
+++ b/sdk/typescript/src/blackduck-findings.ts
@@ -2,7 +2,7 @@ import { CodexSecurityError } from "./errors.js";
import type { Provider } from "./external-import-models.js";
import {
type ExternalImportTarget,
- validateExternalEvidence,
+ validateEnrichedExternalEvidence,
} from "./external-import-contract.js";
import type { ExternalFinding, ExternalFindings } from "./external-findings.js";
import { canonicalJson, isRecord } from "./record.js";
@@ -294,6 +294,8 @@ const nameVersionForges = new Map([
function packageUrlName(ecosystem: string, parts: string[]) {
if (parts.some((part) => !part || part.includes("/"))) return;
+ if (ecosystem === "github" && parts.length === 2)
+ return parts.join("/").toLowerCase();
if (ecosystem === "maven" || ecosystem === "composer")
return parts.length === 2
? parts.join(ecosystem === "maven" ? ":" : "/")
@@ -333,6 +335,7 @@ function originPackage(row: Record<string, unknown>) {
function blackDuckPackages(row: Record<string, unknown>) {
const origin = originPackage(row);
+ const fallback = origin ? [origin] : [];
const purl =
/^pkg:([a-z][a-z0-9.+-]*)\/((?:@[^/@?#]+\/)?[^@?#]+)(?:@([^@?#]+))?(?:[?#].*)?$/iu.exec(
text(row["packageUrl"]) ?? "",
@@ -346,29 +349,32 @@ function blackDuckPackages(row: Record<string, unknown>) {
);
if (name) {
if (!purl[3] && origin?.ecosystem === ecosystem && origin.name === name)
- return [origin];
- return [
- {
- name,
- ecosystem,
- installed_version: purl[3]
- ? decodeURIComponent(purl[3])
- : text(row["componentVersionName"]),
- },
- ];
+ return { packages: fallback };
+ const pkg = {
+ name,
+ ecosystem,
+ installed_version: purl[3]
+ ? decodeURIComponent(purl[3])
+ : text(row["componentVersionName"]),
+ };
+ // Earlier versions used the origin fallback for owner/repository PURLs.
+ return ecosystem === "github" && name.includes("/")
+ ? { packages: fallback, enrichment: { packages: [pkg] } }
+ : { packages: [pkg] };
}
} catch {
// A malformed escape does not invalidate the native origin coordinate.
}
}
- return origin ? [origin] : [];
+ return { packages: fallback };
}
-export function normalizeBlackDuckFindings(
+function normalizeReport(
input: unknown,
sourceRepositoryUrl: string,
- target: ExternalImportTarget = "cloud",
+ target: ExternalImportTarget,
+ previous: boolean,
): ExternalFindings {
const repository = declaredScannerRepositoryUrl(
sourceRepositoryUrl,
@@ -413,14 +419,15 @@ export function normalizeBlackDuckFindings(
);
const severity = text(vulnerability["severity"])?.toLowerCase();
const related = text(vulnerability["relatedVulnerability"]);
- return validateExternalEvidence(
+ const { packages, enrichment } = blackDuckPackages(row);
+ return validateEnrichedExternalEvidence(
{
title: `${advisory} in ${name}`,
description: text(vulnerability["description"]),
severity: FailureSeveritySchema.safeParse(severity).data ?? "unknown",
advisory_ids:
related && related !== advisory ? [advisory, related] : [advisory],
- packages: blackDuckPackages(row),
+ packages,
source_data: {
format,
sourceRepositoryUrl: repository,
@@ -429,6 +436,7 @@ export function normalizeBlackDuckFindings(
},
},
target,
+ previous ? undefined : enrichment,
);
},
(entry) => occurrenceId(entry, projectVersion),
@@ -452,6 +460,34 @@ export function normalizeBlackDuckFindings(
return result;
}
+export function normalizeBlackDuckFindings(
+ input: unknown,
+ sourceRepositoryUrl: string,
+ target: ExternalImportTarget = "cloud",
+): ExternalFindings {
+ return normalizeReport(input, sourceRepositoryUrl, target, false);
+}
+
+/** @internal Earlier readers used the native origin fallback for GitHub package URLs. */
+export function needsPreviousBlackDuckProjection({
+ findings,
+}: ExternalFindings) {
+ return findings.some(({ evidence }) =>
+ evidence.packages?.some(
+ (pkg) => pkg.ecosystem === "github" && pkg.name.includes("/"),
+ ),
+ );
+}
+
+/** @internal Reconstruct the original evidence for an interrupted Cloud request. */
+export function normalizePreviousBlackDuckFindings(
+ input: unknown,
+ sourceRepositoryUrl: string,
+ target: ExternalImportTarget = "cloud",
+): ExternalFindings {
+ return normalizeReport(input, sourceRepositoryUrl, target, true);
+}
+
export async function readBlackDuckFindings(
path: string,
sourceRepositoryUrl: string,
diff --git a/sdk/typescript/src/endor-findings.ts b/sdk/typescript/src/endor-findings.ts
index c1c638161..8ed1bd540 100644
--- a/sdk/typescript/src/endor-findings.ts
+++ b/sdk/typescript/src/endor-findings.ts
@@ -1,5 +1,9 @@
-import type { ExternalImportTarget } from "./external-import-contract.js";
-import { validateExternalEvidence } from "./external-import-contract.js";
+import { markPreviousCloudProjection } from "./reader-selection.js";
+import {
+ type ExternalImportTarget,
+ validateEnrichedExternalEvidence,
+ validateExternalEvidence,
+} from "./external-import-contract.js";
import type { ExternalFindings } from "./external-findings.js";
import { asRecord } from "./record.js";
import {
@@ -100,6 +104,29 @@ function endorListObjects(root: Record<string, unknown>): unknown[] {
export function normalizeEndorFindings(
input: unknown,
target: ExternalImportTarget = "cloud",
+): ExternalFindings {
+ return normalizeReport(input, target, false);
+}
+
+const reducedReports = new WeakSet<ExternalFindings>();
+
+/** A native upload would ask the worker to reconstruct fields that Cloud could not accept. */
+export function requiresStructuredEndorReport(findings: ExternalFindings) {
+ return reducedReports.has(findings);
+}
+
+/** @internal Reproduce the selection from before malware and fully qualified package mapping. */
+export function normalizePreviousEndorFindings(
+ input: unknown,
+ target: ExternalImportTarget = "cloud",
+): ExternalFindings {
+ return normalizeReport(input, target, true);
+}
+
+function normalizeReport(
+ input: unknown,
+ target: ExternalImportTarget,
+ previous: boolean,
): ExternalFindings {
const root = Array.isArray(input) ? null : record(input, "Endor report");
if (root?.["version"] === "2.1.0" && root["runs"] !== undefined)
@@ -110,14 +137,19 @@ export function normalizeEndorFindings(
: Array.isArray(input)
? input
: [input];
- return normalizeRecords(
+ let changed = false;
+ let reduced = false;
+ const result = normalizeRecords(
values,
(value) => {
const finding = record(value, "Endor finding");
const meta = record(finding["meta"], "Endor meta");
const spec = record(finding["spec"], "Endor spec");
requiredText(spec["project_uuid"], "Endor project_uuid");
- const name = text(spec["target_dependency_name"]);
+ const bareName = text(spec["target_dependency_name"]);
+ const name =
+ bareName ??
+ (previous ? null : text(spec["target_dependency_package_name"]));
const paths = [
...new Set(
strings(spec["dependency_file_paths"])
@@ -168,6 +200,15 @@ export function normalizeEndorFindings(
const aliases = strings(
record(vulnerability["spec"] ?? {}, "Vulnerability spec")["aliases"],
);
+ const vulnerabilityIds = [
+ ...new Set([...(advisory ? [advisory] : []), ...aliases]),
+ ];
+ const malware = previous ? undefined : asRecord(metadata["malware"]);
+ const advisories = new Set([
+ ...vulnerabilityIds,
+ ...strings([asRecord(malware?.["meta"])?.["name"]]),
+ ...strings(asRecord(malware?.["spec"])?.["aliases"]),
+ ]);
const level = text(spec["level"])
?.replace(/^FINDING_LEVEL_/u, "")
.toLowerCase();
@@ -190,7 +231,12 @@ export function normalizeEndorFindings(
: [];
const context = record(finding["context"] ?? {}, "Endor context");
const updated = text(meta["update_time"]);
- return validateExternalEvidence(
+ const malwareIds =
+ advisories.size > vulnerabilityIds.length
+ ? { advisory_ids: [...advisories] }
+ : undefined;
+ const fallbackPackage = name !== bareName ? { packages } : undefined;
+ const evidence = validateEnrichedExternalEvidence(
{
title: requiredText(meta["name"], "Endor finding title"),
description:
@@ -212,15 +258,24 @@ export function normalizeEndorFindings(
),
source_updated_at:
updated === null ? null : Math.floor(Date.parse(updated) / 1000),
- advisory_ids: [
- ...new Set([...(advisory ? [advisory] : []), ...aliases]),
- ],
- packages,
+ advisory_ids: vulnerabilityIds,
+ packages: bareName ? packages : [],
locations,
source_data: structuredClone(finding),
},
target,
+ malwareIds && fallbackPackage && { ...malwareIds, ...fallbackPackage },
+ malwareIds,
+ fallbackPackage,
);
+ const acceptedIds = evidence.advisory_ids!.length;
+ const acceptedPackages = evidence.packages!.length;
+ changed ||=
+ acceptedIds > vulnerabilityIds.length ||
+ (!bareName && acceptedPackages > 0);
+ reduced ||=
+ acceptedIds < advisories.size || acceptedPackages < packages.length;
+ return evidence;
},
(value) =>
requiredText(
@@ -229,6 +284,11 @@ export function normalizeEndorFindings(
),
target,
);
+ if (target === "cloud") {
+ if (changed) markPreviousCloudProjection(result);
+ if (reduced) reducedReports.add(result);
+ }
+ return result;
}
/** Endor's documented SARIF producer uses properties.finding-uuid, not its rule name. */
diff --git a/sdk/typescript/src/external-findings-local.ts b/sdk/typescript/src/external-findings-local.ts
index 777a01756..762279e57 100644
--- a/sdk/typescript/src/external-findings-local.ts
+++ b/sdk/typescript/src/external-findings-local.ts
@@ -82,6 +82,7 @@ import { declaredScannerRepositoryUrl } from "./scanner-repository-url.js";
import { sonarRepositoryUrls } from "./sonar-findings.js";
import { isRecord } from "./record.js";
import { dependabotRepository } from "./github-dependabot-findings.js";
+import { retainedGitHubRepository } from "./github-findings.js";
import {
type SnykIssuesFindings,
snykRestImportRepository,
@@ -995,6 +996,13 @@ export async function prepareLocalImport(
if (semgrep.migration) Object.assign(finding.evidence, semgrep.migration);
}
const { evidence } = finding;
+ if (options.provider === "github") {
+ const source = retainedGitHubRepository(
+ finding,
+ (url) => identify(url).key,
+ );
+ if (source) sources.set(source, { url: source, platform: "github" });
+ }
const guardian = guardianFindingSourceData(finding);
const url = evidence.details?.repository?.url;
if (guardian) {
diff --git a/sdk/typescript/src/external-findings-publish.ts b/sdk/typescript/src/external-findings-publish.ts
index ad34f9020..5ab28f2c2 100644
--- a/sdk/typescript/src/external-findings-publish.ts
+++ b/sdk/typescript/src/external-findings-publish.ts
@@ -20,6 +20,7 @@ import {
} from "node:util/types";
import {
privateImportClient,
+ privateJobVersions,
selectPrivateReport,
type PrivateImportFormat,
readPrivateCheckpoint,
@@ -52,6 +53,7 @@ import {
readExternalFindings,
requireCloudReader,
semgrepSarifRepositories,
+ snykFileRepositories,
type CloudFindingsReader,
} from "./external-findings-read.js";
import {
@@ -61,10 +63,16 @@ import {
} from "./record.js";
import { notify } from "./value.js";
import { wizCsvRepositoryUrl } from "./wiz-csv.js";
-import { wizFindingRepository } from "./wiz-findings.js";
-import { previousReaderSelection } from "./reader-selection.js";
+import { isWizPackageFinding, wizFindingRepository } from "./wiz-findings.js";
+import { getOpenGrepSourcePositions } from "./opengrep-findings.js";
+import {
+ previousReaderSelection,
+ hasPreviousCloudProjection,
+} from "./reader-selection.js";
import { dependabotRepository } from "./github-dependabot-findings.js";
+import { retainedGitHubRepository } from "./github-findings.js";
import {
+ normalizePreviousSemgrepFindings,
normalizeSemgrepFindings,
readSemgrepReport,
} from "./semgrep-findings.js";
@@ -100,7 +108,10 @@ import {
} from "./hosted-git-repository.js";
import { trivyRemoteKey, trivyRepositoryUrl } from "./trivy-findings.js";
import { hostedRepositoryUrl } from "./hosted-scanner-http.js";
-import { endorRepositoryUrls } from "./endor-findings.js";
+import {
+ endorRepositoryUrls,
+ requiresStructuredEndorReport,
+} from "./endor-findings.js";
import {
blackDuckReportRepositories,
projectedBlackDuckRepositoryUrl,
@@ -227,6 +238,7 @@ interface SavedSubmission {
requests: FindingImportRequest[];
receipts?: FindingImportReceipt[];
inputProjection?: {
+ repositorySeverities?: string[];
retainedHostedSelection?: true;
semgrepPartialScopeDigest?: string;
identityFormat?: 2;
@@ -296,6 +308,103 @@ function jsonDigest(value: unknown, serialize = wireJson): string {
return hash("sha256", serialize(value));
}
+function matchesRepositoryEvidence(
+ expected: ExternalFindings["findings"][number]["evidence"],
+ stored: ExternalFindings["findings"][number]["evidence"],
+): boolean {
+ const storedWire = wireJson(stored);
+ if (wireJson(expected) === storedWire) return true;
+ // A repository can store a literal first-component colon with an explicit
+ // relative prefix. Match only that spelling in typed locations and their manifests.
+ const qualified = new Map<string, string>();
+ const locations = expected.locations?.map((location, index) => {
+ const { path } = location;
+ const actual = stored.locations?.[index]?.path;
+ if (actual !== `./${path}` || !/^[^/:\\@]+:[^/\\]/u.test(path))
+ return location;
+ qualified.set(path, actual);
+ return { ...location, path: actual };
+ });
+ if (!qualified.size) return false;
+ const packages = expected.packages?.map((item) => {
+ const path = item.manifest_path && qualified.get(item.manifest_path);
+ return path ? { ...item, manifest_path: path } : item;
+ });
+ return wireJson({ ...expected, locations, packages }) === storedWire;
+}
+
+function isCompleteUnscoredOpenGrepReport(report: ExternalFindings) {
+ let rows = report.excluded.length;
+ for (const { evidence } of report.findings) {
+ const traces = evidence.source_data?.["results"];
+ if (
+ evidence.details?.scanner_origin !== "OpenGrep" ||
+ evidence.severity !== "unknown" ||
+ !Array.isArray(traces)
+ )
+ return false;
+ rows += traces.length;
+ }
+ return rows === report.read;
+}
+
+function repositoryFindings(
+ report: ExternalFindings,
+ severities: readonly string[] | undefined,
+) {
+ return severities
+ ? report.findings.filter(({ evidence }) =>
+ severities.includes(evidence.severity),
+ )
+ : report.findings;
+}
+
+function repositorySelection(
+ report: ExternalFindings,
+ severities: readonly string[] | undefined,
+ sourcePositions: PublicationSelection["sourcePositions"],
+): ExternalFindings {
+ const findings = repositoryFindings(report, severities);
+ if (findings.length === report.findings.length) return report;
+ const excluded = [...report.excluded];
+ const skipped = new Set(excluded.map(({ position }) => position));
+ function exclude(position: number, id: string | null, severity: string) {
+ excluded.push({
+ position,
+ source_finding_id: id,
+ reason: `The repository publisher does not accept severity ${JSON.stringify(severity)}. A local import or an available private inbox can retain this finding.`,
+ });
+ }
+ // Serialized OpenGrep selections retain every trace but lose the reader's
+ // position map. All these traces have unknown severity, so the remaining
+ // source rows are provably excluded even when their individual IDs are lost.
+ if (!sourcePositions && isCompleteUnscoredOpenGrepReport(report)) {
+ const id =
+ report.findings.length === 1
+ ? report.findings[0]!.source_finding_id
+ : null;
+ for (let row = 1; row <= report.read; row++)
+ if (!skipped.has(row)) exclude(row, id, "unknown");
+ } else {
+ const retained = new Set(findings);
+ let position = 0;
+ for (const finding of report.findings) {
+ do {
+ position++;
+ } while (skipped.has(position));
+ if (retained.has(finding)) continue;
+ const { source_finding_id, evidence } = finding;
+ for (const row of sourcePositions?.get(source_finding_id) ?? [position])
+ exclude(row, source_finding_id, evidence.severity);
+ }
+ }
+ return {
+ ...report,
+ findings,
+ excluded: excluded.sort((a, b) => a.position - b.position),
+ };
+}
+
function sameSubmission(
left: SavedSubmission,
right: SavedSubmission,
@@ -616,6 +725,7 @@ export async function prepareExternalPublication(
);
const provider = EXTERNAL_FINDINGS_PROVIDERS[settings.provider];
let input: PreparePublicationInput;
+ let fileRepositories: (() => readonly ExternalSourceRepository[]) | undefined;
if (settings.provider === "semgrep" || settings.provider === "snyk") {
const semgrep = settings.provider === "semgrep";
const report = await (semgrep
@@ -627,23 +737,26 @@ export async function prepareExternalPublication(
url,
semgrep ? platform : undefined,
);
+ const semgrepOptions = {
+ sourceRepositoryUrl: settings.sourceRepositoryUrl,
+ sourceRepositories: references,
+ };
const findings = semgrep
- ? normalizeSemgrepFindings(
- report,
- {
- sourceRepositoryUrl: settings.sourceRepositoryUrl,
- sourceRepositories: references,
- },
- "cloud",
- )
+ ? normalizeSemgrepFindings(report, semgrepOptions, "cloud")
: await normalizeSnykFile(report, "cloud", references);
- return normalizedPublicationInput(findings, provider);
+ const selection = normalizedPublicationInput(findings, provider);
+ if (semgrep && hasPreviousCloudProjection(findings))
+ selection.readPreviousFindings = () =>
+ normalizePreviousSemgrepFindings(report, semgrepOptions);
+ return selection;
};
const hosted =
isRecord(report) &&
(semgrep
? isSemgrepPlatformFormat(report["format"])
: isSnykRestFormat(report["format"]));
+ if (!semgrep && !hosted)
+ fileRepositories = () => snykFileRepositories(report);
input = hosted
? async (destination) => {
try {
@@ -685,7 +798,13 @@ export async function prepareExternalPublication(
input =
parsed.publication ??
normalizedPublicationInput(parsed.findings, provider);
- input.previousAws = parsed.previousFindings;
+ input.readPreviousFindings = parsed.previousFindings;
+ if (provider === "endor") {
+ fileRepositories = () => parsed.sourceRepositories ?? [];
+ input.requiresStructuredReport = requiresStructuredEndorReport(
+ parsed.findings,
+ );
+ }
}
return preparePublicationInput(
input,
@@ -694,6 +813,7 @@ export async function prepareExternalPublication(
readerSources,
settings.sourceRepositoryUrl,
path,
+ fileRepositories,
);
}
@@ -908,7 +1028,9 @@ function hostedRepositoryMatcher(
}
interface PublicationSelection extends ExternalPublicationInput {
- previousAws?: () => ExternalFindings;
+ requiresStructuredReport?: boolean;
+ sourcePositions?: ReadonlyMap<string, readonly number[]>;
+ readPreviousFindings?: () => ExternalFindings;
original?: {
parsed: ExternalFindings;
identity: unknown;
@@ -1031,10 +1153,18 @@ function previousNormalizedFindings(
return snapshot;
}
-function publicationIdentity(parsed: ExternalFindings) {
- return parsed.report === undefined
- ? parsed.findings
- : { findings: parsed.findings, report: reportForJson(parsed.report) };
+function publicationIdentity(
+ parsed: ExternalFindings,
+ provider?: NormalizedExternalPublicationOptions["provider"],
+) {
+ // For Trivy, the independently checked header binds the repository. Files
+ // and normalized rows use the same checkpoint when their findings agree.
+ if (
+ parsed.report === undefined ||
+ (provider === "trivy" && isTrivyReportHeader(parsed.report))
+ )
+ return parsed.findings;
+ return { findings: parsed.findings, report: reportForJson(parsed.report) };
}
function validatePublicationFindings(
@@ -1213,19 +1343,13 @@ function normalizedPublicationInput(
current = { read: original.parsed.read, findings: [], excluded: [] };
}
rejectLocalOnlyFindings(current);
- // A Trivy header provides repository provenance for its findings. Either entrypoint
- // can resume the same immutable request when the independently checked rows agree.
- const identity =
- provider === "trivy" &&
- current.report &&
- isTrivyReportHeader(current.report)
- ? current.findings
- : publicationIdentity(current);
+ const identity = publicationIdentity(current, provider);
return {
current,
legacy: current,
currentIdentity: identity,
legacyIdentity: identity,
+ sourcePositions: getOpenGrepSourcePositions(ownData(input, "findings")),
original,
freshError,
unprovenIdPrefixes: reader?.unprovenIdPrefixes,
@@ -1407,6 +1531,7 @@ async function preparePublicationInput(
readerSources: readonly ExternalSourceRepository[],
declaredUrl?: string,
nativePath?: string,
+ fileRepositories?: () => readonly ExternalSourceRepository[],
): Promise<PreparedExternalPublication> {
const { inspectorProjectMap } = options;
const progress = (event: ExternalPublicationProgress): void =>
@@ -1436,12 +1561,13 @@ async function preparePublicationInput(
? (url: string) => cloudSourceKey(url, "github")
: cloudRepositoryUrlKey;
const state = externalPublicationStateDirectory(environment);
- let previousAws: ExternalFindings | undefined;
- async function previousAwsIfSaved(
+ let previousFindings: ExternalFindings | undefined;
+ async function previousFindingsIfSaved(
input: PublicationSelection,
currentKey?: string,
) {
- if (!input.previousAws || previousAws) return previousAws;
+ if (!input.readPreviousFindings || previousFindings)
+ return previousFindings;
// An empty state directory (or just this current receipt) needs only the
// current reader for its preview.
const entries = await readdir(state).catch(
@@ -1457,8 +1583,8 @@ async function preparePublicationInput(
name.split(".", 1)[0] !== currentKey,
)
)
- previousAws = input.previousAws();
- return previousAws;
+ previousFindings = input.readPreviousFindings();
+ return previousFindings;
}
async function checkSelection(input: PublicationSelection) {
const parsed = input.current;
@@ -1473,7 +1599,7 @@ async function preparePublicationInput(
!input.uncertainFindingIds?.length &&
!input.unprovenOccurrenceCount &&
!input.unprovenIdPrefixes?.length &&
- !(await previousAwsIfSaved(input))?.findings.length
+ !(await previousFindingsIfSaved(input))?.findings.length
)
throw new NoPublishableFindingsError(parsed);
}
@@ -1486,11 +1612,14 @@ async function preparePublicationInput(
const credentials = await (
dependencies.credentials ?? (() => readCloudCredentials(environment))
)();
- let privateProviders: Set<string> | undefined;
+ let canStartJob = false;
+ let repositorySeverities: string[] | undefined;
const jobVersionHeader = "Codex-Security-Private-Import-Job-Version";
- let jobApiVersion: 2 | undefined;
- const aws =
- options.provider === "inspector" || options.provider === "continuum";
+ const providerJobVersion = privateJobVersions[options.provider];
+ // Older services recognize an exact v2 request for AWS. Only ask for v3
+ // when publishing a provider introduced by that contract.
+ const requestedJobVersion = String(providerJobVersion ?? 2);
+ let jobApiVersion: typeof providerJobVersion;
async function request(
endpoint: string,
body?: FindingImportRequest,
@@ -1507,7 +1636,9 @@ async function preparePublicationInput(
Authorization: `Bearer ${credentials.access_token}`,
"ChatGPT-Account-ID": credentials.account_id,
Accept: "application/json",
- ...(repositoryCatalog ? { [jobVersionHeader]: "2" } : {}),
+ ...(repositoryCatalog
+ ? { [jobVersionHeader]: requestedJobVersion }
+ : {}),
...(body ? { "Content-Type": "application/json" } : {}),
},
...(body ? { body: JSON.stringify(body) } : {}),
@@ -1528,15 +1659,22 @@ async function preparePublicationInput(
);
}
if (repositoryCatalog) {
- jobApiVersion =
- aws && response.headers.get(jobVersionHeader) === "2" ? 2 : undefined;
- const advertised = response.headers.get(
- "Codex-Security-Private-Import-Providers",
+ const advertised = (header: string) =>
+ response.headers
+ .get(header)
+ ?.split(",")
+ .map((item) => item.trim());
+ repositorySeverities = advertised(
+ "Codex-Security-Repository-Import-Severities",
);
- if (advertised !== null)
- privateProviders = new Set(
- advertised.split(",").map((provider) => provider.trim()),
- );
+ canStartJob =
+ advertised("Codex-Security-Private-Import-Providers")?.includes(
+ options.provider,
+ ) ?? false;
+ jobApiVersion =
+ response.headers.get(jobVersionHeader) === requestedJobVersion
+ ? providerJobVersion
+ : undefined;
}
return response.json();
}
@@ -1636,7 +1774,7 @@ async function preparePublicationInput(
if (capturedPending !== undefined) break;
}
}
- // Changed AWS projections get new keys, but both transports must replay the
+ // Changed reader projections get new keys, but both transports must replay the
// exact bytes already approved by the earlier reader when a checkpoint exists.
const privateClient = privateImportClient(
apiBaseUrl,
@@ -1645,72 +1783,72 @@ async function preparePublicationInput(
dependencies.signal,
);
let previewContext: PrivateImportContext | undefined;
- const currentAwsKey = resolved.previousAws
+ const currentReaderKey = resolved.readPreviousFindings
? scopeKey(resolved.currentIdentity)
: undefined;
- let previousAwsKey: string | undefined;
let alternateKey: string | undefined;
- let resumingPreviousAws = false;
+ let resumingPreviousReader = false;
function previousCandidateKey(previous: ExternalFindings | undefined) {
if (!previous?.findings.length) return;
// The file reader already validated plain JSON evidence for Cloud. Only a
// matching checkpoint needs the full normalized publication snapshot.
- previousAwsKey ??= scopeKey(publicationIdentity(previous));
- return previousAwsKey !== currentAwsKey ? previousAwsKey : undefined;
+ const previousKey = scopeKey(
+ publicationIdentity(previous, options.provider),
+ );
+ return previousKey !== currentReaderKey ? previousKey : undefined;
}
- async function hasPrivateCheckpoint(
+ async function alternateCheckpoint(
key: string,
context: () => Promise<PrivateImportContext>,
- ) {
+ ): Promise<{ pending: string | undefined } | undefined> {
+ const pending = await readOptionalFile(join(state, key + ".pending.json"));
+ if (pending !== undefined) return { pending };
const saved = await readOptionalFile(
join(state, key + ".private-import.json"),
);
- if (saved === undefined) return false;
+ if (saved === undefined) return;
const checkpoint = parsePrivateCheckpoint(saved);
// A completed import cannot acknowledge findings newly admitted by the
// current reader. Pending uploads and jobs always retain their original bytes.
if (
- key === previousAwsKey &&
- previousAws &&
+ !resumingPreviousReader &&
+ previousFindings &&
checkpoint.receipt?.status === "completed" &&
- !sameFindingSelection(previousAws.findings, resolved.current.findings)
+ !sameFindingSelection(
+ previousFindings.findings,
+ resolved.current.findings,
+ )
)
- return false;
- return checkpoint.contextVersion === (await context()).context_version;
+ return;
+ if (checkpoint.contextVersion === (await context()).context_version)
+ return { pending };
}
async function assertNoAlternateCheckpoint(
context = () => privateClient.context(),
) {
- const otherKey = alternateKey;
- if (!otherKey) return;
- if (
- (await readOptionalFile(join(state, otherKey + ".pending.json"))) !==
- undefined ||
- (await hasPrivateCheckpoint(otherKey, context))
- )
+ if (alternateKey && (await alternateCheckpoint(alternateKey, context)))
throw new CodexSecurityError(
- resumingPreviousAws
- ? "Both AWS reader versions have saved publications for this report. Resume either one using its original normalized findings."
+ resumingPreviousReader
+ ? "Both reader versions have saved publications for this report. Resume either one using its original normalized findings."
: "Another publication prepared this input. Run the command again to review and resume the saved request.",
);
}
- if (currentAwsKey)
+ if (currentReaderKey)
alternateKey = previousCandidateKey(
- await previousAwsIfSaved(resolved, currentAwsKey),
- );
- if (alternateKey && previousAws) {
- capturedPending = await readOptionalFile(
- join(state, alternateKey + ".pending.json"),
+ await previousFindingsIfSaved(resolved, currentReaderKey),
);
+ if (alternateKey && previousFindings) {
const context = async () =>
(previewContext ??= await privateClient.context());
- if (
- capturedPending !== undefined ||
- (await hasPrivateCheckpoint(alternateKey, context))
- ) {
- projection = normalizedPublicationInput(previousAws, options.provider);
- resumingPreviousAws = true;
- alternateKey = currentAwsKey;
+ const checkpoint = await alternateCheckpoint(alternateKey, context);
+ capturedPending = checkpoint?.pending;
+ if (checkpoint) {
+ projection = normalizedPublicationInput(
+ previousFindings,
+ options.provider,
+ );
+ resumingPreviousReader = true;
+ alternateKey = currentReaderKey;
await assertNoAlternateCheckpoint(context);
}
}
@@ -2055,8 +2193,11 @@ async function preparePublicationInput(
const content = readSaved(record);
const saved = submissionFindings(content.requests);
const wire = wireJson(saved);
- const expected = legacy ? legacyParsed.findings : parsed.findings;
const projection = content.inputProjection;
+ const expected = repositoryFindings(
+ legacy ? legacyParsed : parsed,
+ projection?.repositorySeverities,
+ );
if (
projection &&
(!sameFindingSelection(saved, expected) ||
@@ -2717,9 +2858,9 @@ async function preparePublicationInput(
async function underPublishLocks<T>(operation: () => Promise<T>) {
// An older publisher may already hold its lock before its checkpoint exists.
// Derive that key for actual writes, even after a checkpoint-free preview.
- if (resolved.previousAws && !previousAws) {
- previousAws = resolved.previousAws();
- alternateKey = previousCandidateKey(previousAws);
+ if (resolved.readPreviousFindings && !previousFindings) {
+ previousFindings = resolved.readPreviousFindings();
+ alternateKey = previousCandidateKey(previousFindings);
if (alternateKey) lockKeys.push(alternateKey);
}
return underLocks(lockKeys, operation);
@@ -2762,6 +2903,72 @@ async function preparePublicationInput(
"Supply the verified Inspector project binding again to resume.",
);
+ // Existing repository submissions keep their original request bodies and receipts.
+ const privatePath = join(state, `${key}.private-import.json`);
+ const privateSaved = await readOptionalFile(privatePath);
+ if (resumingPreviousReader && !saved && privateSaved === undefined)
+ throw new CodexSecurityError(
+ "The saved publication changed. Run the command again to preview this report.",
+ );
+ const usePrivateImport =
+ !saved && (canStartJob || privateSaved !== undefined);
+ async function preparePrivateAttempt() {
+ const context = previewContext ?? (await privateClient.context());
+ if (source.source_key.trim() !== source.source_key)
+ throw new CodexSecurityError(
+ "A private source key cannot have leading or trailing whitespace.",
+ );
+ const report = await selectPrivateReport(
+ parsed,
+ source.provider,
+ nativePath,
+ context,
+ privateSaved,
+ resolved.requiresStructuredReport,
+ );
+ const binding = {
+ format: report.format,
+ apiBaseUrl,
+ accountId: credentials.account_id,
+ source,
+ repository: {
+ id: destination.id,
+ repo_connector_id: destination.repo_connector_id,
+ },
+ sha256: report.digest,
+ contextVersion: context.context_version,
+ };
+ return { context, report, binding };
+ }
+ function acceptsJob(
+ context: PrivateImportContext,
+ format: PrivateImportFormat,
+ ) {
+ return context.can_create_import_jobs && context.formats.includes(format);
+ }
+ // Use an earlier catalog URL only after the account, connector, repository,
+ // inbox context and exact upload digest all match the immutable checkpoint.
+ const existingPrivate =
+ usePrivateImport && privateSaved !== undefined
+ ? await preparePrivateAttempt()
+ : undefined;
+ const previous =
+ existingPrivate &&
+ readPrivateCheckpoint(privateSaved, existingPrivate.binding);
+ const retryOfJobId =
+ unsuccessfulJob(previous?.receipt) &&
+ canStartJob &&
+ existingPrivate?.context.can_upload &&
+ acceptsJob(existingPrivate.context, existingPrivate.report.format)
+ ? previous?.jobId
+ : undefined;
+ const resumed = previous !== undefined && retryOfJobId === undefined;
+ const privateApproval = resumed ? previous : undefined;
+ const privateUrl = privateApproval?.approvedRepositoryUrl;
+ const approvedPlatform =
+ saved?.approvedRepositoryPlatform ??
+ privateApproval?.approvedRepositoryPlatform;
+
// The findings-only checkpoint key lets projected and native callers resume one immutable
// request. Supplemental report claims are outside that digest and must still match its source.
const recheckOriginal =
@@ -2828,7 +3035,8 @@ async function preparePublicationInput(
}
// The released request retained its typed finding URL, but only the authenticated
// catalog or a previously recorded approval can bind a new source declaration.
- const approvedUrl = saved?.approvedRepositoryUrl ?? destination.url;
+ const approvedUrl =
+ saved?.approvedRepositoryUrl ?? privateUrl ?? destination.url;
if (
hosted &&
readerSources.some(({ platform, url }) => {
@@ -2846,10 +3054,10 @@ async function preparePublicationInput(
throw new CodexSecurityError(
"The reader's source repository does not match the selected Cloud repository.",
);
- await recheckFileRepository?.(approvedUrl, saved?.approvedRepositoryPlatform);
+ await recheckFileRepository?.(approvedUrl, approvedPlatform);
const hostedSource =
hosted && (!saved || declaredSource)
- ? sourceFor(approvedUrl, saved?.approvedRepositoryPlatform)
+ ? sourceFor(approvedUrl, approvedPlatform)
: undefined;
const matchesHostedRepository = hostedSource?.matches;
function requireApprovedSource(url: string, platform?: "other") {
@@ -2865,7 +3073,8 @@ async function preparePublicationInput(
requireApprovedSource(approvedUrl);
if (legacyTypedUrl) requireApprovedSource(legacyTypedUrl);
- const trivyTarget = saved?.trivyRepositoryUrl ?? destination.url;
+ const trivyTarget =
+ saved?.trivyRepositoryUrl ?? privateUrl ?? destination.url;
function requireTrivyRemote(url: string, id?: string) {
if (trivyRemoteKey(url, trivyTarget) !== catalogKey(trivyTarget)) {
const label =
@@ -2884,9 +3093,10 @@ async function preparePublicationInput(
function checkRepositories(
{ findings, report }: Pick<ExternalFindings, "findings" | "report">,
- historicalCheckmarx = Boolean(saved),
+ historical = Boolean(saved),
) {
- checkTrivyReportProvider(options.provider, report);
+ const { provider } = options;
+ checkTrivyReportProvider(provider, report);
// A catalog casing or spelling change keeps the original source comparison target.
const approved = hosted ? saved?.approvedRepositoryUrl : undefined;
const sameCatalog =
@@ -2897,7 +3107,7 @@ async function preparePublicationInput(
? trivyTarget
: sameCatalog
? approved
- : destination.url;
+ : (privateUrl ?? destination.url);
const key = sameCatalog
? (url: string) =>
cloudSourceKey(
@@ -2919,13 +3129,25 @@ async function preparePublicationInput(
`Finding ${JSON.stringify(id)} belongs to ${JSON.stringify(url)}, which does not match the selected Cloud repository ${JSON.stringify(destination.url)}. Verify the source repository mapping before publishing.`,
);
}
+ // An immutable retry may refer to an earlier repository name. Additional
+ // file declarations outside its saved findings authorize no new writes.
+ if (!historical)
+ for (const { url } of fileRepositories?.() ?? [])
+ requireDestination(
+ "Source report",
+ provider === "snyk"
+ ? normalizeHostedGitRepositoryUrl(url, destination.url, true)
+ : url,
+ );
// Historical checkpoints already fixed the provider and request bytes.
// Recheck any projected source against the destination discovered for this retry.
- const replay =
- saved && options.provider !== "blackduck" && !trivyReportUrls;
- if (!saved || options.provider === "sonar" || trivyReportUrls) {
+ const checkNative = (...providers: (typeof provider)[]) =>
+ !historical ||
+ trivyReportUrls !== undefined ||
+ providers.includes(provider);
+ if (checkNative("sonar")) {
for (const url of sonarRepositoryUrls(
- options.provider,
+ provider,
findings,
report,
"cloud",
@@ -2933,45 +3155,45 @@ async function preparePublicationInput(
requireDestination("Sonar source", url);
}
const checkmarx = checkmarxSelection(
- options.provider,
+ provider,
findings,
report,
"cloud",
- historicalCheckmarx,
+ historical,
);
requireDestination("Checkmarx source", checkmarx.reportUrl);
- const repositories = replay
- ? undefined
- : blackDuckReportRepositories(options.provider, findings, "cloud");
+ const blackDuck = checkNative("blackduck")
+ ? blackDuckReportRepositories(provider, findings, "cloud")
+ : undefined;
const checkmarxRepository = checkmarx.repositories();
- for (const [provider, repositories] of [
+ for (const [scanner, repositories] of [
["veracode", veracodeReportRepositories],
["continuum", continuumReportRepositories],
] as const) {
- if (saved && options.provider !== provider && !trivyReportUrls) continue;
- for (const [id, source] of repositories(
- options.provider,
- findings,
- "cloud",
- ))
+ if (!checkNative(scanner)) continue;
+ for (const [id, source] of repositories(provider, findings, "cloud"))
requireDestination(id, source.url);
}
- if (
- !replay ||
- options.provider === "inspector" ||
- options.provider === "veracode"
- ) {
+ if (checkNative("blackduck", "inspector", "veracode")) {
const binding = inspectorReportRepository(
- options.provider,
+ provider,
findings,
"cloud",
inspectorProjectMap,
);
if (binding) requireDestination(binding.findingId, binding.url);
}
+ const checkOnlyProjected = !checkNative(
+ "blackduck",
+ "inspector",
+ "veracode",
+ "checkmarx",
+ "continuum",
+ "sonar",
+ );
for (const finding of findings) {
const { source_finding_id: id, evidence } = finding;
- checkRetainedTrivyProvider(options.provider, finding);
+ checkRetainedTrivyProvider(provider, finding);
if (trivyReportUrls) {
const remote = trivyRepositoryUrl(
retainedTrivyReport(finding),
@@ -2979,45 +3201,43 @@ async function preparePublicationInput(
);
if (remote) requireTrivyRemote(remote, id);
}
- const blackDuckUrl = repositories
- ? repositories.get(id)?.url
+ const blackDuckUrl = blackDuck
+ ? blackDuck.get(id)?.url
: projectedBlackDuckRepositoryUrl(finding, "cloud");
const hasProjection =
!!blackDuckUrl || checkmarx.projections.has(finding);
- if (
- replay &&
- !hasProjection &&
- options.provider !== "inspector" &&
- options.provider !== "veracode" &&
- options.provider !== "checkmarx" &&
- options.provider !== "continuum" &&
- options.provider !== "sonar"
- )
- continue;
- const urls = [
+ if (checkOnlyProjected && !hasProjection) continue;
+ let urls = [
blackDuckUrl,
checkmarxRepository(finding)?.url,
evidence.details?.repository?.url,
- options.provider === "wiz"
- ? hasProjection
- ? wizFindingRepository(evidence, "cloud")?.url
- : wizCsvRepositoryUrl(evidence)
- : undefined,
- ...(options.provider === "snyk"
- ? snykRepositoryUrls(evidence.source_data, {
- cloudRepositoryUrl: destination.url,
- })
- : []),
- ...(options.provider === "endor"
- ? endorRepositoryUrls(
- evidence.source_data,
- Boolean(replay && !blackDuckUrl),
- )
- : []),
- options.provider === "github"
- ? dependabotRepository(finding)
- : undefined,
];
+ if (provider === "wiz") {
+ const native =
+ hasProjection ||
+ (!historical && isWizPackageFinding(evidence.source_data));
+ urls.push(
+ native
+ ? wizFindingRepository(evidence, "cloud")?.url
+ : wizCsvRepositoryUrl(evidence),
+ );
+ } else if (provider === "snyk") {
+ urls = urls.concat(
+ snykRepositoryUrls(evidence.source_data, {
+ cloudRepositoryUrl: destination.url,
+ }),
+ );
+ } else if (provider === "endor") {
+ urls = urls.concat(
+ endorRepositoryUrls(
+ evidence.source_data,
+ !blackDuck && !blackDuckUrl,
+ ),
+ );
+ } else if (provider === "github") {
+ if (!historical) urls.push(retainedGitHubRepository(finding));
+ urls.push(dependabotRepository(finding));
+ }
for (const url of urls) requireDestination(id, url);
}
}
@@ -3036,16 +3256,6 @@ async function preparePublicationInput(
);
}
}
- // Existing repository submissions keep their original request bodies and receipts.
- const privatePath = join(state, `${key}.private-import.json`);
- const privateSaved = await readOptionalFile(privatePath);
- if (resumingPreviousAws && !saved && privateSaved === undefined)
- throw new CodexSecurityError(
- "The saved publication changed. Run the command again to preview this report.",
- );
- const usePrivateImport =
- !saved &&
- (privateProviders?.has(source.provider) || privateSaved !== undefined);
if (!usePrivateImport) checkRepositories(parsed);
// A retry sends its saved projection, which may differ from today's reader.
if (saved) {
@@ -3092,61 +3302,22 @@ async function preparePublicationInput(
if (usePrivateImport) {
const client = privateClient;
- const context = previewContext ?? (await client.context());
- if (source.source_key.trim() !== source.source_key)
- throw new CodexSecurityError(
- "A private source key cannot have leading or trailing whitespace.",
- );
- const report = await selectPrivateReport(
- parsed,
- source.provider,
- nativePath,
- context,
- privateSaved,
- );
- const binding = {
- format: report.format,
- apiBaseUrl,
- accountId: credentials.account_id,
- source,
- repository: {
- id: destination.id,
- repo_connector_id: destination.repo_connector_id,
- },
- sha256: report.digest,
- contextVersion: context.context_version,
- };
- function checkpointForContext(data: string | undefined) {
- return data === undefined
- ? undefined
- : readPrivateCheckpoint(data, binding);
- }
- const previous = checkpointForContext(privateSaved);
- const canStartJob = !aws || privateProviders?.has(source.provider) === true;
+ const { context, report, binding } =
+ existingPrivate ?? (await preparePrivateAttempt());
function requireAccess(value: typeof context, saved: typeof previous) {
if (
!value.can_read ||
(!saved && !canStartJob) ||
(!saved?.jobId &&
((!saved?.fileId && !value.can_upload) ||
- !value.can_create_import_jobs ||
- !value.formats.includes(report.format)))
+ !acceptsJob(value, report.format)))
)
throw new CodexSecurityError(
"Private findings import is unavailable for this account or report format.",
);
}
- requireAccess(context, previous);
- const retryOfJobId =
- unsuccessfulJob(previous?.receipt) &&
- canStartJob &&
- context.can_upload &&
- context.can_create_import_jobs &&
- context.formats.includes(report.format)
- ? previous?.jobId
- : undefined;
- const resumed = previous !== undefined && retryOfJobId === undefined;
checkRepositories(parsed, resumed);
+ requireAccess(context, previous);
const privateImport = {
format: report.format,
contextVersion: binding.contextVersion,
@@ -3174,10 +3345,15 @@ async function preparePublicationInput(
"The private inbox changed. Run the command again to review the new destination.",
);
await assertNoAlternateCheckpoint(async () => current);
- const stored = checkpointForContext(
+ const stored = readPrivateCheckpoint(
await readOptionalFile(privatePath),
+ binding,
);
- if (resumingPreviousAws && !stored)
+ if (
+ !stored &&
+ (resumingPreviousReader ||
+ (privateUrl && privateUrl !== destination.url))
+ )
throw new CodexSecurityError(
"The saved private import changed. Run the command again to preview this report.",
);
@@ -3192,10 +3368,13 @@ async function preparePublicationInput(
"A repository submission is pending. Review its saved request before continuing.",
);
// Both routes share request IDs. A server upgrade can promote an
- // existing AWS job, and its saved route also survives gate rollback.
+ // existing job, and its saved route also survives gate rollback.
const checkpoint = {
...(active ?? {
...binding,
+ approvedRepositoryUrl: destination.url,
+ approvedRepositoryPlatform:
+ hostedSource?.github === false ? ("other" as const) : undefined,
uploadRequestId: randomUUID(),
jobRequestId: randomUUID(),
fileId: undefined,
@@ -3290,16 +3469,22 @@ async function preparePublicationInput(
};
}
+ // The original selection and request bytes govern a retry even if discovery changes.
+ const severities = saved
+ ? saved.inputProjection?.repositorySeverities
+ : repositorySeverities;
+ const direct = repositorySelection(parsed, severities, input.sourcePositions);
+ if (!direct.findings.length) throw new NoPublishableFindingsError(direct);
let requests = saved?.requests;
if (!requests) {
let preparedCount = 0;
progress({
phase: "preparing",
completed: 0,
- total: parsed.findings.length,
+ total: direct.findings.length,
});
const previousReports = await readInBatches(
- parsed.findings,
+ direct.findings,
async (finding): Promise<SourceReportSummary | undefined> => {
const query = new URLSearchParams({
provider: source.provider,
@@ -3329,7 +3514,7 @@ async function preparePublicationInput(
progress({
phase: "preparing",
completed: ++preparedCount,
- total: parsed.findings.length,
+ total: direct.findings.length,
});
return previous;
},
@@ -3337,7 +3522,7 @@ async function preparePublicationInput(
requests = [];
let current: FindingImportRequest | undefined;
let currentBytes = 0;
- for (const [index, finding] of parsed.findings.entries()) {
+ for (const [index, finding] of direct.findings.entries()) {
const previous = previousReports[index];
const environmentId =
previous?.environment_id ?? destination.import_environment_id;
@@ -3399,6 +3584,7 @@ async function preparePublicationInput(
legacyApproval: saved?.legacyApproval,
wireKey: saved?.wireKey ?? (fileKey === key ? undefined : key),
inputProjection: saved?.inputProjection ?? {
+ ...(direct !== parsed && { repositorySeverities: severities }),
identityFormat: saved ? undefined : 2,
...(scopedSemgrepReport && { retainedHostedSelection: true }),
...(bindSemgrepScope && {
@@ -3421,7 +3607,7 @@ async function preparePublicationInput(
submission.approvedRepositoryPlatform = "other";
if (inspectorProjectMap) submission.inspectorProjectMapRequired = true;
const preview: ExternalPublicationPreview = {
- ...parsed,
+ ...direct,
findings: submissionFindings(requests),
cloudApiUrl,
cloudUrl: cloudFindingsUrl(environment, destination.id),
@@ -3461,18 +3647,18 @@ async function preparePublicationInput(
cloudApiUrl,
cloudUrl: preview.cloudUrl,
status:
- parsed.excluded.length || counts.error ? "partial" : "complete",
+ direct.excluded.length || counts.error ? "partial" : "complete",
read: parsed.read,
...(parsed.report === undefined
? {}
: snapshotNormalizedFindings({ report: parsed.report })),
- ready: parsed.findings.length,
- excluded: parsed.excluded,
+ ready: direct.findings.length,
+ excluded: direct.excluded,
receipts: acknowledged,
counts,
failures,
unacknowledged:
- parsed.findings.length -
+ direct.findings.length -
Object.values(counts).reduce((sum, count) => sum + count, 0),
verified,
};
@@ -3601,7 +3787,7 @@ async function preparePublicationInput(
progress({
phase: "uploading",
completed: 0,
- total: parsed.findings.length,
+ total: direct.findings.length,
});
let acknowledgedCount = 0;
for (const [batchIndex, batch] of requests.entries()) {
@@ -3615,7 +3801,7 @@ async function preparePublicationInput(
progress({
phase: "uploading",
completed: acknowledgedCount,
- total: parsed.findings.length,
+ total: direct.findings.length,
});
if (batchIndex === receipts.length) {
receipts.push(receipt);
@@ -3660,7 +3846,10 @@ async function preparePublicationInput(
report.version < item.version! ||
(report.version === item.version &&
(report.observation_id !== item.observation_id ||
- wireJson(report.evidence) !== wireJson(expected.evidence))) ||
+ !matchesRepositoryEvidence(
+ expected.evidence,
+ report.evidence,
+ ))) ||
report.source_finding_id !== expected.source_finding_id ||
report.repo_id !== destination.id ||
report.repo_connector_id !== destination.repo_connector_id ||
diff --git a/sdk/typescript/src/external-findings-read.ts b/sdk/typescript/src/external-findings-read.ts
index 38b5ac666..9ae6ea919 100644
--- a/sdk/typescript/src/external-findings-read.ts
+++ b/sdk/typescript/src/external-findings-read.ts
@@ -4,7 +4,6 @@ import type {
ExternalPublicationInput,
ExternalSourceRepository,
} from "./external-findings.js";
-import { hasNoExternalRecords } from "./external-findings.js";
import { readScannerJson, sarifRunRepositories } from "./scanner-import.js";
import type { Provider } from "./external-import-models.js";
import { CodexSecurityError } from "./errors.js";
@@ -23,16 +22,28 @@ import {
readSemgrepReport,
} from "./semgrep-findings.js";
import { semgrepPlatformReportRepository } from "./semgrep-platform-findings.js";
-import { readTrivyFindings } from "./trivy-findings.js";
+import {
+ normalizeTrivyFindings,
+ needsPreviousTrivyProjection,
+ normalizePreviousTrivyFindings,
+} from "./trivy-findings.js";
import { readOpenGrepFindings } from "./opengrep-findings.js";
import {
normalizeEndorFindings,
+ normalizePreviousEndorFindings,
endorRepositoryUrls,
} from "./endor-findings.js";
import { readGitHubFindings } from "./github-findings.js";
import { readSonarFindings } from "./sonar-findings.js";
-import { readVeracodeFindings } from "./veracode-findings.js";
-import { readBlackDuckFindings } from "./blackduck-findings.js";
+import {
+ normalizeVeracodeFindings,
+ normalizePreviousVeracodeFindings,
+} from "./veracode-findings.js";
+import {
+ needsPreviousBlackDuckProjection,
+ normalizeBlackDuckFindings,
+ normalizePreviousBlackDuckFindings,
+} from "./blackduck-findings.js";
import {
normalizeContinuumFindings,
normalizePreviousContinuumFindings,
@@ -42,6 +53,7 @@ import {
normalizeInspectorFindings,
normalizePreviousInspectorFindings,
} from "./inspector-findings.js";
+import { hasPreviousCloudProjection } from "./reader-selection.js";
import { isRecord } from "./record.js";
import {
readLocalWizFindings,
@@ -181,6 +193,13 @@ function sarifFileRepositories(
return input["runs"].flatMap((run) => sarifRunRepositories(run, producer));
}
+/** Repository claims from every project or SARIF run, even those with no findings. */
+export function snykFileRepositories(input: unknown) {
+ return (
+ sarifFileRepositories(input, "Snyk") ?? emptySnykReportRepositories(input)
+ );
+}
+
/** @internal Read vendor evidence for the local and Cloud import entrypoints. */
export async function readExternalFindings(
path: string,
@@ -189,55 +208,69 @@ export async function readExternalFindings(
references: readonly ExternalSourceRepository[] = [],
): Promise<ReadExternalFindings> {
const { provider } = options;
- if (provider === "tenable") {
- if (target === "cloud") requireCloudReader(options);
+ const url = options.sourceRepositoryUrl ?? "";
+ // Both projections describe the same captured rows, including exclusions.
+ function withReplay(
+ findings: ExternalFindings,
+ couldDiffer: (findings: ExternalFindings) => boolean,
+ previous: () => ExternalFindings,
+ ): ReadExternalFindings {
return {
- findings: await readTenableFindings(
- path,
- options.sourceRepositoryUrl ?? "",
- ),
+ findings,
+ previousFindings:
+ target === "cloud" && couldDiffer(findings) ? previous : undefined,
};
}
- if (provider === "trivy")
- return {
- findings: await readTrivyFindings(path, target, options.trivyScanRoot),
- };
+ if (provider === "tenable") {
+ if (target === "cloud") requireCloudReader(options);
+ return { findings: await readTenableFindings(path, url) };
+ }
+ if (provider === "trivy") {
+ const args = [
+ await readScannerJson(path),
+ target,
+ options.trivyScanRoot,
+ ] as const;
+ return withReplay(
+ normalizeTrivyFindings(...args),
+ needsPreviousTrivyProjection,
+ () => normalizePreviousTrivyFindings(...args),
+ );
+ }
if (provider === "sonar") {
- const url = options.sourceRepositoryUrl;
- const findings = await readSonarFindings(path, url ?? "", target);
+ const findings = await readSonarFindings(path, url, target);
const binding = findings.report?.["repositoryUrl"];
return {
findings,
sourceRepositories: typeof binding === "string" ? [{ url: binding }] : [],
};
}
- if (provider === "opengrep") {
- const sourceRepositoryUrl = options.sourceRepositoryUrl ?? "";
+ if (provider === "opengrep" || provider === "checkmarx") {
+ const findings =
+ provider === "opengrep"
+ ? await readOpenGrepFindings(path, { sourceRepositoryUrl: url }, target)
+ : await readCheckmarxFindings(path, url, target);
+ return { findings, sourceRepositories: [{ url }] };
+ }
+ if (provider === "snyk") {
+ const input = await readSnykReport(path);
+ const findings = await normalizeSnykFile(input, target, references);
+ return { findings, sourceRepositories: snykFileRepositories(input) };
+ }
+ if (provider === "endor") {
+ const input = await readScannerJson(path);
+ const findings = normalizeEndorFindings(input, target);
return {
- findings: await readOpenGrepFindings(
- path,
- { sourceRepositoryUrl },
- target,
+ ...withReplay(findings, hasPreviousCloudProjection, () =>
+ normalizePreviousEndorFindings(input, target),
),
- sourceRepositories: [{ url: sourceRepositoryUrl }],
+ sourceRepositories:
+ sarifFileRepositories(input, "Endor") ??
+ findings.findings.flatMap((row) =>
+ localFindingRepositories(provider, row),
+ ),
};
}
- if (provider === "snyk" || provider === "endor") {
- const snyk = provider === "snyk";
- const input = await (snyk ? readSnykReport : readScannerJson)(path);
- const findings = snyk
- ? await normalizeSnykFile(input, target, references)
- : normalizeEndorFindings(input, target);
- const header = sarifFileRepositories(input, snyk ? "Snyk" : "Endor");
- const sourceRepositories =
- header ??
- (snyk && hasNoExternalRecords(findings)
- ? emptySnykReportRepositories(input)
- : findings.findings.flatMap((row) =>
- localFindingRepositories(provider, row),
- ));
- return { findings, sourceRepositories };
- }
if (provider === "semgrep") {
const input = await readSemgrepReport(path);
const findings = normalizeSemgrepFindings(
@@ -256,59 +289,50 @@ export async function readExternalFindings(
}
if (provider === "github")
return { findings: await readGitHubFindings(path, target) };
- if (provider === "inspector" || provider === "continuum") {
- const url = options.sourceRepositoryUrl ?? "";
- const mapping =
- options.inspectorProjectMap && new Map(options.inspectorProjectMap);
- const input = await readScannerJson(path);
- const normalize =
- provider === "inspector"
- ? normalizeInspectorFindings
- : normalizeContinuumFindings;
- const previous =
- provider === "inspector"
- ? normalizePreviousInspectorFindings
- : normalizePreviousContinuumFindings;
- // Both projections describe the same captured file, including rows admitted
- // by just one reader. Detail arrays are new to Continuum.
- const wasSupported = provider === "inspector" || !Array.isArray(input);
- return {
- findings: normalize(input, url, target, mapping),
- previousFindings:
- target === "cloud" && wasSupported
- ? () => previous(input, url, target, mapping)
- : undefined,
- sourceRepositories: [{ url }],
- };
- }
- if (
- provider === "checkmarx" ||
- provider === "blackduck" ||
- provider === "veracode"
- ) {
- const url = options.sourceRepositoryUrl ?? "";
- const read = {
- checkmarx: readCheckmarxFindings,
- blackduck: readBlackDuckFindings,
- veracode: readVeracodeFindings,
- }[provider];
+ if (provider === "wiz") {
+ const wiz =
+ target === "local"
+ ? await readLocalWizFindings(path)
+ : await readVendorFindingsForPublication(path, target);
+ const sourceRepositories = wiz.current.findings.flatMap((finding) =>
+ localFindingRepositories("wiz", finding),
+ );
return {
- findings: await read(path, url, target),
- sourceRepositories: [{ url }],
+ findings: wiz.current,
+ ...("legacy" in wiz && { publication: wiz }),
+ sourceRepositories: [...wiz.sourceRepositories, ...sourceRepositories],
};
}
- // A new reader must choose its parser before reaching the Wiz replay path.
- provider satisfies "wiz";
- const wiz =
- target === "local"
- ? await readLocalWizFindings(path)
- : await readVendorFindingsForPublication(path, target);
- const sourceRepositories = wiz.current.findings.flatMap((finding) =>
- localFindingRepositories("wiz", finding),
- );
+ const mapping =
+ options.inspectorProjectMap && new Map(options.inspectorProjectMap);
+ const input = await readScannerJson(path);
+ const [normalize, previous, couldDiffer] = (
+ {
+ inspector: [
+ normalizeInspectorFindings,
+ normalizePreviousInspectorFindings,
+ () => true,
+ ],
+ continuum: [
+ normalizeContinuumFindings,
+ normalizePreviousContinuumFindings,
+ () => !Array.isArray(input),
+ ],
+ blackduck: [
+ normalizeBlackDuckFindings,
+ normalizePreviousBlackDuckFindings,
+ needsPreviousBlackDuckProjection,
+ ],
+ veracode: [
+ normalizeVeracodeFindings,
+ normalizePreviousVeracodeFindings,
+ hasPreviousCloudProjection,
+ ],
+ } as const
+ )[provider];
+ const args = [input, url, target, mapping] as const;
return {
- findings: wiz.current,
- ...("legacy" in wiz && { publication: wiz }),
- sourceRepositories: [...wiz.sourceRepositories, ...sourceRepositories],
+ ...withReplay(normalize(...args), couldDiffer, () => previous(...args)),
+ sourceRepositories: [{ url }],
};
}
diff --git a/sdk/typescript/src/github-findings.ts b/sdk/typescript/src/github-findings.ts
index ced61f8d1..5433c76a3 100644
--- a/sdk/typescript/src/github-findings.ts
+++ b/sdk/typescript/src/github-findings.ts
@@ -5,6 +5,7 @@ import { isBoxedPrimitive, isDate, isProxy } from "node:util/types";
import { GHSA_ID, githubRepositoryParts } from "./github-identifiers.js";
import { CodexSecurityError, errorMessage } from "./errors.js";
import {
+ cloudRepositoryUrlKey,
CloudEvidenceDepthError,
exportedGitHubRecord,
previousGitHubExclusion,
@@ -78,11 +79,8 @@ function codeScanningPrefix(
return `${host}/${name}/${alert}/`;
}
-function codeScanningFinding(record: Record<string, unknown>): {
- source_finding_id: string;
- evidence: unknown;
-} {
- const repository = repositoryName(record["repository"]);
+function codeScanningSource(record: Record<string, unknown>) {
+ const name = repositoryName(record["repository"]);
const alert = asRecord(record["alert"]);
const number = record["number"];
if (
@@ -99,11 +97,20 @@ function codeScanningFinding(record: Record<string, unknown>): {
if (!url || url !== alert["html_url"])
throw new Error("GitHub alert URL must match the original alert.");
const link = sourceUrl(url);
- const prefix = codeScanningPrefix(link, repository, number);
+ const prefix = codeScanningPrefix(link, name, number);
if (!prefix)
throw new Error(
"GitHub alert URL does not match the selected repository and alert.",
);
+ const repository = { url: `${link.origin}/${name}`, name };
+ return { alert, number, prefix, url, repository };
+}
+
+function codeScanningFinding(record: Record<string, unknown>): {
+ source_finding_id: string;
+ evidence: unknown;
+} {
+ const { alert, number, prefix, url, repository } = codeScanningSource(record);
const rule = asRecord(alert["rule"]);
const instance = asRecord(alert["most_recent_instance"]);
const ref = text(instance?.["ref"]);
@@ -148,10 +155,7 @@ function codeScanningFinding(record: Record<string, unknown>): {
source_data: record,
details: {
kind: "sast",
- repository: {
- url: `${link.origin}/${repository}`,
- name: repository,
- },
+ repository,
// Dismissal covers every branch; detection state belongs to the selected ref.
source_status:
alert["state"] === "dismissed"
@@ -176,11 +180,8 @@ function advisorySourceId(
return `${host}/${selection === "repository" ? repository.toLowerCase() : "global"}/security-advisory/${id.toUpperCase()}`;
}
-function advisoryFinding(record: Record<string, unknown>): {
- source_finding_id: string;
- evidence: unknown;
-} {
- const repository = repositoryName(record["repository"]);
+function advisorySource(record: Record<string, unknown>) {
+ const name = repositoryName(record["repository"]);
const advisory = asRecord(record["advisory"]);
const id = text(record["ghsaId"]);
const selection = record["selection"];
@@ -205,7 +206,7 @@ function advisoryFinding(record: Record<string, unknown>): {
selection === "global" && parsedUrl.host === "github.com";
const expectedPath =
selection === "repository"
- ? `/${repository}/security/advisories/${id}`
+ ? `/${name}/security/advisories/${id}`
: `/advisories/${id}`;
if (
!host ||
@@ -215,6 +216,16 @@ function advisoryFinding(record: Record<string, unknown>): {
throw new Error(
"GitHub advisory URL does not match the selected source and GHSA ID.",
);
+ const repository = { url: `${origin}/${name}`, name };
+ return { advisory, id, selection, url, host, repository } as const;
+}
+
+function advisoryFinding(record: Record<string, unknown>): {
+ source_finding_id: string;
+ evidence: unknown;
+} {
+ const { advisory, id, selection, url, host, repository } =
+ advisorySource(record);
const vulnerabilities =
advisory["vulnerabilities"] === null ? [] : advisory["vulnerabilities"];
if (!Array.isArray(vulnerabilities))
@@ -250,7 +261,7 @@ function advisoryFinding(record: Record<string, unknown>): {
});
const cwes = Array.isArray(advisory["cwes"]) ? advisory["cwes"] : [];
return {
- source_finding_id: advisorySourceId(host, selection, repository, id),
+ source_finding_id: advisorySourceId(host, selection, repository.name, id),
evidence: {
title: text(advisory["summary"]) ?? id,
severity: severity(advisory["severity"]),
@@ -263,14 +274,7 @@ function advisoryFinding(record: Record<string, unknown>): {
details: {
kind: "advisory",
advisory: { source: selection },
- ...(selection === "repository"
- ? {
- repository: {
- url: `${origin}/${repository}`,
- name: repository,
- },
- }
- : {}),
+ ...(selection === "repository" ? { repository } : {}),
source_status: text(advisory["withdrawn_at"])
? "withdrawn"
: (text(advisory["state"]) ??
@@ -284,6 +288,45 @@ function advisoryFinding(record: Record<string, unknown>): {
};
}
+/** Keep the repository from a retained collector export when SDK callers supply their own projection. */
+export function retainedGitHubRepository(
+ { evidence }: ExternalFinding,
+ repositoryKey = cloudRepositoryUrlKey,
+): string | undefined {
+ const data = evidence.source_data;
+ if (!data?.["url"] || !data["repository"]) return;
+ let source;
+ if (
+ data["source"] === "github-code-scanning" &&
+ data["number"] !== undefined &&
+ asRecord(data["alert"])?.["html_url"]
+ ) {
+ source = codeScanningSource(data);
+ } else if (
+ data["source"] === "github-security-advisory" &&
+ data["ghsaId"] &&
+ data["githubHost"] &&
+ data["selection"] &&
+ asRecord(data["advisory"])?.["html_url"]
+ ) {
+ source = advisorySource(data);
+ if (source.selection === "global") {
+ if (evidence.details?.repository)
+ throw new CodexSecurityError(
+ "Global advisory evidence cannot claim a source repository.",
+ );
+ return;
+ }
+ } else return;
+ const native = source.repository.url;
+ const projected = evidence.details?.repository?.url;
+ if (projected && repositoryKey(projected) !== repositoryKey(native))
+ throw new CodexSecurityError(
+ "The normalized repository does not match the original GitHub finding.",
+ );
+ return native;
+}
+
function mapRecord(input: unknown) {
const record = asRecord(input);
if (!record) throw new Error("GitHub finding must be an object.");
diff --git a/sdk/typescript/src/native-json-findings.ts b/sdk/typescript/src/native-json-findings.ts
index 8db1e9f39..2223e7b05 100644
--- a/sdk/typescript/src/native-json-findings.ts
+++ b/sdk/typescript/src/native-json-findings.ts
@@ -1,3 +1,4 @@
+import { posix } from "node:path";
import {
type ExternalImportTarget,
validateExternalEvidence,
@@ -59,10 +60,23 @@ export function position(value: unknown, label: string): number {
return value;
}
+// Validate both POSIX filenames and Windows separators before a reader emits a path.
+export function nativeResultPath(
+ raw: string,
+ relative = relativePath(raw),
+): string | null {
+ if (relative === null) return null;
+ const path = posix.normalize(raw);
+ return relativePath(posix.normalize(path.replaceAll("\\", "/")))
+ ? path
+ : null;
+}
+
/** Parse the native code-result fields shared by Semgrep and OpenGrep. */
export function parseNativeJsonResult(
value: unknown,
origin: "Semgrep" | "OpenGrep",
+ projectPath: (raw: string) => string | null,
) {
const result = record(value, `${origin} result`);
const extra = record(result["extra"], `${origin} extra`);
@@ -100,7 +114,7 @@ export function parseNativeJsonResult(
metadata,
rule,
rawPath,
- path: relativePath(rawPath),
+ path: projectPath(rawPath),
line,
column,
endLine,
diff --git a/sdk/typescript/src/opengrep-findings.ts b/sdk/typescript/src/opengrep-findings.ts
index c0de28b01..9aae0d040 100644
--- a/sdk/typescript/src/opengrep-findings.ts
+++ b/sdk/typescript/src/opengrep-findings.ts
@@ -1,4 +1,3 @@
-import { posix } from "node:path";
import {
previousAdmission,
validateExternalEvidenceJson,
@@ -10,6 +9,7 @@ import { errorMessage } from "./errors.js";
import type { ExternalFindings } from "./external-findings.js";
import {
messageText,
+ nativeResultPath,
parseNativeJsonResult,
projectNativeJsonResult,
sourceRepositoryUrl,
@@ -19,7 +19,6 @@ import {
array,
readScannerJson,
record,
- relativePath,
requiredText,
stableId,
} from "./scanner-import.js";
@@ -29,6 +28,16 @@ export interface OpenGrepImportOptions {
sourceRepositoryUrl: string;
}
+// Occurrences are sorted by ID and can retain traces from nonadjacent input rows.
+const sourcePositions = new WeakMap<
+ ExternalFindings["findings"],
+ ReadonlyMap<string, readonly number[]>
+>();
+
+export function getOpenGrepSourcePositions(findings: unknown) {
+ return Array.isArray(findings) ? sourcePositions.get(findings) : undefined;
+}
+
/** Import native scanner evidence without assigning a Codex assessment. */
export function normalizeOpenGrepFindings(
input: unknown,
@@ -168,15 +177,7 @@ export function normalizeOpenGrepFindings(
for (const [index, value] of values.entries()) {
let id: string | null = null;
try {
- const parsed = parseNativeJsonResult(value, "OpenGrep");
- if (parsed.path !== null) {
- // Validate the emitted spelling across platforms, keeping literal POSIX backslashes.
- const path = posix.normalize(parsed.rawPath);
- parsed.path =
- relativePath(posix.normalize(path.replaceAll("\\", "/"))) === null
- ? null
- : path;
- }
+ const parsed = parseNativeJsonResult(value, "OpenGrep", nativeResultPath);
// Native fingerprint counters change when another analysis pass adds matches.
// A source move creates a new occurrence; this is not cross-revision matching.
id = stableId("opengrep-location", [
@@ -195,13 +196,17 @@ export function normalizeOpenGrepFindings(
exclude(index + 1, id, error);
}
}
+ const positions = new Map<string, number[]>();
for (const [id, observations] of [...groups].sort(([left], [right]) =>
left < right ? -1 : left > right ? 1 : 0,
)) {
if (observations.length === 1) {
- selection.append(observations[0]!.position, id, () =>
- projectGroup(observations),
- );
+ const { position } = observations[0]!;
+ selection.append(position, id, () => {
+ const evidence = projectGroup(observations);
+ positions.set(id, [position]);
+ return evidence;
+ });
continue;
}
const ordered = observations
@@ -245,12 +250,17 @@ export function normalizeOpenGrepFindings(
}
if (previousAdmission(evidence) === null) selection.uncertainIds.add(id);
output.findings.push({ source_finding_id: id, evidence });
+ positions.set(
+ id,
+ retained.map(({ position }) => position),
+ );
} catch (error) {
for (const observation of retained)
exclude(observation.position, id, error);
}
}
output.excluded.sort((left, right) => left.position - right.position);
+ sourcePositions.set(output.findings, positions);
return selection.finish();
}
diff --git a/sdk/typescript/src/private-finding-import.ts b/sdk/typescript/src/private-finding-import.ts
index 9c56628c6..1326e2e7d 100644
--- a/sdk/typescript/src/private-finding-import.ts
+++ b/sdk/typescript/src/private-finding-import.ts
@@ -15,6 +15,19 @@ const privateFormat = z.enum([
"wiz.vulnerability_json.v1",
]);
export type PrivateImportFormat = z.infer<typeof privateFormat>;
+const jobVersion = z.union([z.literal(2), z.literal(3)]);
+export type PrivateJobVersion = z.infer<typeof jobVersion>;
+export const privateJobVersions: Partial<
+ Record<ImportSource["provider"], PrivateJobVersion>
+> = {
+ inspector: 2,
+ continuum: 2,
+ blackduck: 3,
+ checkmarx: 3,
+ sonar: 3,
+ trivy: 3,
+ veracode: 3,
+};
const fileId = z.string().regex(/^aifile_[0-9a-f]{32}$/u);
const contextSchema = z.object({
object: z.literal("security.finding_import_context"),
@@ -173,6 +186,7 @@ export async function selectPrivateReport(
path: string | undefined,
context: PrivateImportContext,
saved: string | undefined,
+ requiresStructuredReport = false,
): Promise<ReturnType<typeof privateReport>> {
const checkpoint =
saved === undefined ? undefined : checkpointSchema.parse(JSON.parse(saved));
@@ -180,6 +194,13 @@ export async function selectPrivateReport(
checkpoint?.contextVersion === context.context_version
? checkpoint.format
: undefined;
+ if (requiresStructuredReport && !originalFormat) {
+ if (!context.formats.includes(structuredFormat))
+ throw new CodexSecurityError(
+ `This Cloud service must support ${structuredFormat} to publish the reviewed Endor fields.`,
+ );
+ return privateReport(selection.findings, context.limits);
+ }
const format =
provider === "endor"
? "endor.findings_json.v1"
@@ -279,7 +300,9 @@ const checkpointSchema = z.object({
uploadRequestId: z.uuid(),
jobRequestId: z.uuid(),
// Earlier checkpoints used the unversioned job route.
- jobApiVersion: z.literal(2).optional(),
+ jobApiVersion: jobVersion.optional(),
+ approvedRepositoryUrl: z.string().optional(),
+ approvedRepositoryPlatform: z.literal("other").optional(),
fileId: fileId.optional(),
jobId: jobSchema.shape.id.optional(),
receipt: jobSchema
@@ -297,7 +320,7 @@ export function parsePrivateCheckpoint(serialized: string) {
}
export function readPrivateCheckpoint(
- serialized: string,
+ serialized: string | undefined,
expected: {
format: PrivateImportFormat;
apiBaseUrl: string;
@@ -308,11 +331,14 @@ export function readPrivateCheckpoint(
repository: { id: string; repo_connector_id: string };
},
) {
+ if (serialized === undefined) return;
const checkpoint = parsePrivateCheckpoint(serialized);
const {
uploadRequestId,
jobRequestId,
jobApiVersion,
+ approvedRepositoryUrl,
+ approvedRepositoryPlatform,
fileId,
jobId,
receipt,
@@ -462,14 +488,12 @@ export function privateImportClient(
source: ImportSource;
repository: { id: string; repo_connector_id: string };
jobId?: string;
- jobApiVersion?: 2;
+ jobApiVersion?: PrivateJobVersion;
},
accepted: (id: string) => Promise<void>,
) {
- const jobs =
- input.jobApiVersion === 2
- ? "/v2/finding-imports/jobs"
- : "/finding-imports/jobs";
+ const prefix = input.jobApiVersion ? `/v${input.jobApiVersion}` : "";
+ const jobs = prefix + "/finding-imports/jobs";
let id = input.jobId;
let response = id
? await request(`${jobs}/${id}`)
diff --git a/sdk/typescript/src/reader-selection.ts b/sdk/typescript/src/reader-selection.ts
index 43b3e9521..dbf7550a0 100644
--- a/sdk/typescript/src/reader-selection.ts
+++ b/sdk/typescript/src/reader-selection.ts
@@ -4,6 +4,19 @@ import { isBoxedPrimitive, isProxy } from "node:util/types";
import type { ExternalFindings } from "./external-findings.js";
import { canonicalJson, cloneWithoutGetters } from "./record.js";
+const changedCloudProjections = new WeakSet<ExternalFindings>();
+
+// Readers mark selections whose original Cloud request can have different evidence.
+export function markPreviousCloudProjection(findings: ExternalFindings): void {
+ changedCloudProjections.add(findings);
+}
+
+export function hasPreviousCloudProjection(
+ findings: ExternalFindings,
+): boolean {
+ return changedCloudProjections.has(findings);
+}
+
export interface LegacyReaderSelection {
sourceFindingIds: string[];
snapshot?: ExternalFindings;
diff --git a/sdk/typescript/src/semgrep-findings.ts b/sdk/typescript/src/semgrep-findings.ts
index adad3579b..9deb7bec2 100644
--- a/sdk/typescript/src/semgrep-findings.ts
+++ b/sdk/typescript/src/semgrep-findings.ts
@@ -1,3 +1,4 @@
+import { markPreviousCloudProjection } from "./reader-selection.js";
import {
normalizeSemgrepPlatformFindings,
isSemgrepPlatformFormat,
@@ -21,6 +22,7 @@ import type {
} from "./external-findings.js";
import {
messageText,
+ nativeResultPath,
parseNativeJsonResult,
position,
projectNativeJsonResult,
@@ -196,9 +198,50 @@ export function normalizeSemgrepFindings(
{ ...options, sourceRepositoryUrl: options?.sourceRepositoryUrl ?? "" },
target,
);
+ return normalizeSemgrepJson(document, options, target);
+}
+
+function nativeProjection(target: ExternalImportTarget, previous: boolean) {
+ let changed = false;
+ return {
+ location(raw: string) {
+ const oldPath = relativePath(raw);
+ if (previous) return oldPath;
+ // Semgrep retains its raw spelling; OpenGrep groups by the normalized path.
+ const path = nativeResultPath(raw, oldPath) === null ? null : raw;
+ if (path !== oldPath) changed = true;
+ return path;
+ },
+ finish(findings: ExternalFindings, omittedExplanations = false) {
+ if (target === "cloud" && (changed || omittedExplanations))
+ markPreviousCloudProjection(findings);
+ return findings;
+ },
+ };
+}
+
+/** @internal Rebuild the native JSON or SARIF that an earlier reader approved. */
+export function normalizePreviousSemgrepFindings(
+ input: unknown,
+ options: Partial<SemgrepImportOptions>,
+): ExternalFindings {
+ const document = record(input, "Semgrep report");
+ return document["version"] === "2.1.0" && document["runs"] !== undefined
+ ? normalizeSarif(document, options, "cloud", true)
+ : normalizeSemgrepJson(document, options, "cloud", true);
+}
+
+function normalizeSemgrepJson(
+ document: Record<string, unknown>,
+ options: Partial<SemgrepImportOptions>,
+ target: ExternalImportTarget,
+ previous = false,
+): ExternalFindings {
const repositoryUrl = sourceRepository(options);
requiredText(document["version"], "Semgrep version");
const { results, paths: _paths, time: _time, ...context } = document;
+ const omitExplanations = !previous && Object.hasOwn(context, "explanations");
+ if (omitExplanations) delete context["explanations"];
const rows = array(results, "Semgrep results");
if (!rows.length && document["errors"] === undefined)
throw new Error("An empty Semgrep report must include its errors array.");
@@ -206,19 +249,24 @@ export function normalizeSemgrepFindings(
throw new Error(
"The Semgrep report contains scan errors; supply a successful report before importing findings.",
);
- return normalizeRecords(
+ const projection = nativeProjection(target, previous);
+ const findings = normalizeRecords(
rows,
(value) =>
- projectNativeJsonResult(parseNativeJsonResult(value, "Semgrep"), {
- origin: "Semgrep",
- repositoryUrl,
- severity,
- reportContext: context,
- target,
- }),
+ projectNativeJsonResult(
+ parseNativeJsonResult(value, "Semgrep", projection.location),
+ {
+ origin: "Semgrep",
+ repositoryUrl,
+ severity,
+ reportContext: context,
+ target,
+ },
+ ),
jsonIdentity,
target,
);
+ return projection.finish(findings, omitExplanations);
}
export function isSemgrepSarifDriver(name: string): boolean {
@@ -229,6 +277,15 @@ export function normalizeSemgrepSarif(
input: unknown,
options: SemgrepImportOptions,
target: ExternalImportTarget = "cloud",
+): ExternalFindings {
+ return normalizeSarif(input, options, target);
+}
+
+function normalizeSarif(
+ input: unknown,
+ options: Partial<SemgrepImportOptions>,
+ target: ExternalImportTarget,
+ previous = false,
): ExternalFindings {
const repositoryUrl = sourceRepository(options);
const repositoryKey = cloudRepositoryUrlKey(repositoryUrl);
@@ -253,7 +310,8 @@ export function normalizeSemgrepSarif(
throw new Error("Expected a Semgrep SARIF producer.");
return sarifRunResults(context, "Semgrep");
});
- return normalizeRecords(
+ const projection = nativeProjection(target, previous);
+ const findings = normalizeRecords(
entries,
({ result: value, context }) => {
const result = record(value, "Semgrep result");
@@ -306,7 +364,7 @@ export function normalizeSemgrepSarif(
typeof artifact["uri"] === "string" &&
/^[a-z][a-z\d+.-]*:/iu.test(artifact["uri"]);
const path =
- rawPath === null || absoluteUri ? null : relativePath(rawPath);
+ rawPath === null || absoluteUri ? null : projection.location(rawPath);
return validateExternalEvidence(
{
title: ruleId,
@@ -346,6 +404,7 @@ export function normalizeSemgrepSarif(
({ result, context }) => sarifIdentity(result, context.runContext),
target,
);
+ return projection.finish(findings);
}
/** @internal Read the original file before resolving repository provenance. */
diff --git a/sdk/typescript/src/semgrep-guardian-findings.ts b/sdk/typescript/src/semgrep-guardian-findings.ts
index 2adef9e03..d92a569f2 100644
--- a/sdk/typescript/src/semgrep-guardian-findings.ts
+++ b/sdk/typescript/src/semgrep-guardian-findings.ts
@@ -43,14 +43,12 @@ export function guardianFindingSourceData({
const data = evidence.source_data ?? {};
const generated = guardianFindingId.test(source_finding_id);
if (data["format"] !== guardianSelectionFormat && !generated) return;
- if (generated) {
- const row = record(data["finding"], "Original Guardian finding");
- requireGuardianRowScope(data, row);
- if (source_finding_id !== guardianId(data, row["id"]))
- throw new CodexSecurityError(
- "The normalized finding ID does not match the original Guardian finding.",
- );
- }
+ const row = record(data["finding"], "Original Guardian finding");
+ requireGuardianRowScope(data, row);
+ if (source_finding_id !== guardianId(data, row["id"]))
+ throw new CodexSecurityError(
+ "The normalized finding ID does not match the original Guardian finding.",
+ );
return data;
}
diff --git a/sdk/typescript/src/semgrep-platform-findings.ts b/sdk/typescript/src/semgrep-platform-findings.ts
index 551165813..874f46bbb 100644
--- a/sdk/typescript/src/semgrep-platform-findings.ts
+++ b/sdk/typescript/src/semgrep-platform-findings.ts
@@ -787,7 +787,7 @@ export function normalizeSelectedSemgrepPlatformFindings(
options: SemgrepSelectionInput & {
sourceRepositoryUrl: string;
},
-): ExternalFindings {
+) {
const url = hostedRepositoryUrl(options.sourceRepositoryUrl, "Semgrep");
const { scope, rows } = selectedRows(
options,
@@ -806,11 +806,14 @@ export function normalizeSelectedSemgrepPlatformFindings(
},
hostedRepositoryUrl,
);
- return normalizePlatformRows(
- rows.map((finding) => ({ finding, selectable: true })),
- scope,
- "local",
- );
+ return {
+ ...normalizePlatformRows(
+ rows.map((finding) => ({ finding, selectable: true })),
+ scope,
+ "local",
+ ),
+ sourceRepositoryUrl: scope.project.url,
+ };
}
function sourceId(
diff --git a/sdk/typescript/src/snyk-findings.ts b/sdk/typescript/src/snyk-findings.ts
index ee940e47d..6bf287a29 100644
--- a/sdk/typescript/src/snyk-findings.ts
+++ b/sdk/typescript/src/snyk-findings.ts
@@ -11,6 +11,7 @@ import {
SNYK_REST_ISSUES_V1,
} from "./snyk-rest-v1.js";
import type { SnykIssuesFindings } from "./snyk-rest-findings.js";
+import { normalizeHostedGitRepositoryUrl } from "./hosted-git-repository.js";
import { isRecord } from "./record.js";
import {
array,
@@ -196,39 +197,12 @@ function normalizeGitRepositoryUrl(
repository: string,
cloudRepositoryUrl?: string,
): string {
- // Cloud's selected catalog repository supplies connected GitHub host context;
- // imported evidence alone cannot establish an Enterprise host's SSH aliases.
- const destination = cloudRepositoryUrl
- ? new URL(cloudRepositoryUrl)
- : undefined;
- const connectedHost =
- destination?.protocol === "https:"
- ? destination.hostname.toLowerCase()
- : undefined;
- const connectedTenant = /^([^.]+)\.ghe\.com$/u.exec(connectedHost ?? "")?.[1];
- const isGitHubSshUser = (username: string, hostname: string) =>
- (username === "git" &&
- (hostname === "github.com" || hostname === connectedHost)) ||
- (username === connectedTenant && hostname === connectedHost);
- const scp = /^([^@/:]+)@([^/:]+):(.+)$/u.exec(repository);
- if (scp && isGitHubSshUser(scp[1]!, scp[2]!.toLowerCase()))
- return `https://${scp[2]}/${scp[3]}`;
- // Compare Git's literal SSH userinfo before URL strips tabs or newlines.
- const ssh = /^ssh:\/\/([^/]+)@/u.exec(repository);
- if (ssh) {
- // Git retains these bytes, but URL parsing would erase them from the alias.
- if (/[\t\r\n]/u.test(repository)) return repository;
- const url = new URL(repository);
- const hostname = url.hostname.toLowerCase();
- const githubSsh =
- isGitHubSshUser(ssh[1]!, hostname) &&
- (url.port === "" || url.port === "22");
- const githubSshOverHttps =
- ssh[1] === "git" && hostname === "ssh.github.com" && url.port === "443";
- if (githubSshOverHttps) return `https://github.com${url.pathname}`;
- if (githubSsh) return `https://${hostname}${url.pathname}`;
- }
- return repository;
+ // Without a selected Cloud repository, only the public GitHub host is established.
+ return normalizeHostedGitRepositoryUrl(
+ repository,
+ cloudRepositoryUrl ?? "https://github.com",
+ cloudRepositoryUrl !== undefined,
+ );
}
function snykJsonProjectRepository(
diff --git a/sdk/typescript/src/trivy-findings.ts b/sdk/typescript/src/trivy-findings.ts
index 63fb9859a..899ca99ad 100644
--- a/sdk/typescript/src/trivy-findings.ts
+++ b/sdk/typescript/src/trivy-findings.ts
@@ -25,6 +25,7 @@ type NativeRecord = Record<string, unknown>;
const urlScheme = /^[a-z][a-z\d+.-]*:\/\//iu;
const javaArchiveAncestor = /\.(?:jar|war|ear|par)\//iu;
+const cloudLocationLimit = 100;
const curlUrlPrefix = /^(?:(?:https?|ftps?)::)?(?:https?|ftps?):\/{1,3}/iu;
const unsafeRepositoryUrl =
@@ -378,6 +379,32 @@ export function normalizeTrivyFindings(
input: unknown,
target: ExternalImportTarget = "cloud",
scanRoot?: string,
+): ExternalFindings {
+ const locationLimit = target === "cloud" ? cloudLocationLimit : Infinity;
+ return normalizeReport(input, target, scanRoot, locationLimit);
+}
+
+/** @internal Only a finding at the Cloud location limit can have a different prior selection. */
+export function needsPreviousTrivyProjection({ findings }: ExternalFindings) {
+ return findings.some(
+ ({ evidence }) => evidence.locations?.length === cloudLocationLimit,
+ );
+}
+
+/** @internal Reconstruct the original selection for an interrupted Cloud request. */
+export function normalizePreviousTrivyFindings(
+ input: unknown,
+ target: ExternalImportTarget = "cloud",
+ scanRoot?: string,
+): ExternalFindings {
+ return normalizeReport(input, target, scanRoot, Infinity);
+}
+
+function normalizeReport(
+ input: unknown,
+ target: ExternalImportTarget,
+ scanRoot: string | undefined,
+ locationLimit: number,
): ExternalFindings {
const document = record(input, "Trivy report");
if (document["SchemaVersion"] !== 2)
@@ -471,6 +498,8 @@ export function normalizeTrivyFindings(
if (targetPath && targetPath !== declaration)
locations.push({ path: targetPath, line: null });
if (declaration) {
+ // The scan target already consumes a slot when it differs from this declaration.
+ const lineLimit = locationLimit - locations.length;
const lines = new Set<number>();
for (const range of array(
pkg?.["Locations"] ?? [],
@@ -480,7 +509,8 @@ export function normalizeTrivyFindings(
if (
typeof start === "number" &&
Number.isSafeInteger(start) &&
- start > 0
+ start > 0 &&
+ lines.size < lineLimit
)
lines.add(start);
}
diff --git a/sdk/typescript/src/veracode-findings.ts b/sdk/typescript/src/veracode-findings.ts
index 09a361c14..ab25a3f9e 100644
--- a/sdk/typescript/src/veracode-findings.ts
+++ b/sdk/typescript/src/veracode-findings.ts
@@ -1,4 +1,5 @@
import { z } from "zod";
+import { markPreviousCloudProjection } from "./reader-selection.js";
import { CodexSecurityError } from "./errors.js";
import type { Provider } from "./external-import-models.js";
import {
@@ -362,7 +363,29 @@ function readPage(input: unknown) {
return { rows, scope, query, filter, number, size, total, pages };
}
-function rowSelectionValidator(query: URLSearchParams) {
+function findingDetails(row: Record<string, unknown>) {
+ return record(row["finding_details"], "Veracode finding_details");
+}
+
+// The Findings API schema uses a numeric CWE and a category name. HAL exports
+// can expand either field to a descriptor containing its ID and name.
+function expandedDetails(
+ details: Record<string, unknown>,
+): Record<string, unknown> {
+ const { cwe, finding_category: category } = details;
+ if (typeof cwe !== "number" && typeof category !== "string") return details;
+ return {
+ ...details,
+ cwe: typeof cwe === "number" ? { id: cwe } : cwe,
+ finding_category:
+ typeof category === "string" ? { name: category } : category,
+ };
+}
+
+function rowSelectionValidator(
+ query: URLSearchParams,
+ expand: typeof expandedDetails,
+) {
const staticOnly = query.has("scan_type");
const violates = query.get("violates_policy");
const newOnly = query.get("new") === "true";
@@ -385,6 +408,7 @@ function rowSelectionValidator(query: URLSearchParams) {
});
return (row: Record<string, unknown>) => {
+ let expandedFilter = false;
if (
newOnly &&
record(row["finding_status"], "Veracode finding_status")["new"] !== true
@@ -399,14 +423,14 @@ function rowSelectionValidator(query: URLSearchParams) {
"Veracode finding contradicts the include_annot selection.",
);
if (idFilters.length || scoreFilters.length) {
- const details = record(
- row["finding_details"],
- "Veracode finding_details",
- );
+ const native = findingDetails(row);
+ const details = expand(native);
for (const { field, ids } of idFilters) {
if (row["scan_type"] !== "STATIC") continue;
const entry = record(details[field], `Veracode ${field}`);
const id = integer(entry["id"], `${field} ID`);
+ // Older readers required HAL descriptors for filtered IDs.
+ expandedFilter ||= details !== native && native[field] !== entry;
if (!ids.has(String(id)))
throw new Error(
`Veracode finding contradicts the ${field} selection.`,
@@ -435,10 +459,11 @@ function rowSelectionValidator(query: URLSearchParams) {
throw new Error(
"Veracode finding contradicts the violates_policy selection.",
);
+ return expandedFilter;
};
}
-function readReport(input: unknown) {
+function readReport(input: unknown, expand: typeof expandedDetails) {
const pages = (Array.isArray(input) ? input : [input])
.map(readPage)
.sort((left, right) => left.number - right.number);
@@ -459,14 +484,15 @@ function readReport(input: unknown) {
"Supply every Veracode page exactly once with stable totals, selection, origin and application context.",
);
const { scope } = first;
- const checkSelection = rowSelectionValidator(first.query);
+ const checkSelection = rowSelectionValidator(first.query, expand);
+ let usedExpandedFilter = false;
const seen = new Set<string>();
const rows: Record<string, unknown>[] = [];
for (const page of pages) {
for (const value of page.rows) {
const row = record(value, "Veracode finding");
requireApplication(row, scope);
- checkSelection(row);
+ if (checkSelection(row)) usedExpandedFilter = true;
const issue = row["issue_id"];
if (
typeof row["scan_type"] === "string" &&
@@ -482,27 +508,27 @@ function readReport(input: unknown) {
rows.push(row);
}
}
- return { scope, rows };
+ return { scope, rows, usedExpandedFilter };
}
-export function normalizeVeracodeFindings(
+function normalizeReport(
input: unknown,
sourceRepositoryUrl: string,
- target: ExternalImportTarget = "cloud",
+ target: ExternalImportTarget,
+ expand: typeof expandedDetails,
): ExternalFindings {
const repository = declaredScannerRepositoryUrl(
sourceRepositoryUrl,
"Veracode",
target,
);
- const { scope, rows } = readReport(input);
- return normalizeRecords(
+ const { scope, rows, usedExpandedFilter } = readReport(input, expand);
+ let admittedScalar = false;
+ const findings = normalizeRecords(
rows,
(row) => {
- const details = record(
- row["finding_details"],
- "Veracode finding_details",
- );
+ const native = findingDetails(row);
+ const details = expand(native);
const status = record(row["finding_status"], "Veracode finding_status");
const cwe = record(details["cwe"], "Veracode CWE");
const weakness = `CWE-${integer(cwe["id"], "CWE ID", 1)}`;
@@ -514,7 +540,7 @@ export function normalizeVeracodeFindings(
severityByScore[integer(details["severity"], "severity")];
if (!severity)
throw new Error("Veracode severity must be on the native 0–5 scale.");
- return validateExternalEvidence(
+ const evidence = validateExternalEvidence(
{
title: text(category?.["name"]) ?? text(cwe["name"]) ?? weakness,
severity,
@@ -536,10 +562,37 @@ export function normalizeVeracodeFindings(
},
target,
);
+ if (details !== native) admittedScalar = true;
+ return evidence;
},
(row) => occurrenceId(row, scope),
target,
);
+ if (target === "cloud" && admittedScalar && !usedExpandedFilter)
+ markPreviousCloudProjection(findings);
+ return findings;
+}
+
+export function normalizeVeracodeFindings(
+ input: unknown,
+ sourceRepositoryUrl: string,
+ target: ExternalImportTarget = "cloud",
+): ExternalFindings {
+ return normalizeReport(input, sourceRepositoryUrl, target, expandedDetails);
+}
+
+/** Reconstruct a saved publication made when only HAL descriptors were admitted. */
+export function normalizePreviousVeracodeFindings(
+ input: unknown,
+ sourceRepositoryUrl: string,
+ target: ExternalImportTarget = "cloud",
+): ExternalFindings {
+ return normalizeReport(
+ input,
+ sourceRepositoryUrl,
+ target,
+ (details) => details,
+ );
}
export async function readVeracodeFindings(
diff --git a/sdk/typescript/src/wiz-findings.ts b/sdk/typescript/src/wiz-findings.ts
index 006fc3eaf..02f5c33f9 100644
--- a/sdk/typescript/src/wiz-findings.ts
+++ b/sdk/typescript/src/wiz-findings.ts
@@ -29,6 +29,20 @@ function sourceFindingId(value: unknown, target: ExternalImportTarget): string {
return id;
}
+/** Distinguish original Wiz package records from opaque SDK asset metadata. */
+export function isWizPackageFinding(
+ data: VendorFinding["evidence"]["source_data"],
+): boolean {
+ return Boolean(
+ data &&
+ data["detectionMethod"] !== "EXTERNAL_NETWORK_SCAN" &&
+ text(data["id"]) &&
+ text(data["name"]) &&
+ text(data["detailedName"]) &&
+ text(asRecord(data["vulnerableAsset"])?.["id"]),
+ );
+}
+
/** Keep a Wiz vulnerability occurrence intact; advisory names are not identities. */
function wizFinding(
record: Record<string, unknown>,
@@ -468,7 +482,9 @@ export function wizFindingRepository(
if (!url) return;
const github =
asset["nativeType"] === "github#repositoryBranch" ||
- (asset["type"] === "REPOSITORY_BRANCH" && isGitHubCloudHost(url.hostname));
+ (asset["type"] === "REPOSITORY_BRANCH" &&
+ (text(asset["cloudPlatform"])?.toLowerCase() === "github" ||
+ isGitHubCloudHost(url.hostname)));
if (!github)
return asset["type"] === "REPOSITORY_BRANCH"
? { url: asset["cloudProviderURL"] }
diff --git a/sdk/typescript/tests-ts/blackduck-findings.test.ts b/sdk/typescript/tests-ts/blackduck-findings.test.ts
index e1e178f66..a31d473c9 100644
--- a/sdk/typescript/tests-ts/blackduck-findings.test.ts
+++ b/sdk/typescript/tests-ts/blackduck-findings.test.ts
@@ -4,7 +4,9 @@ import { join } from "node:path";
import { gzipSync } from "node:zlib";
import {
blackDuckReportRepositories,
+ needsPreviousBlackDuckProjection,
normalizeBlackDuckFindings,
+ normalizePreviousBlackDuckFindings,
readBlackDuckFindings,
} from "../src/blackduck-findings.js";
import {
@@ -138,6 +140,43 @@ test("replay keeps native occurrence identity when package coordinates and advis
expect(finding.evidence.source_data?.["row"]).toEqual(opaque);
});
+test("keeps GitHub package coordinates separate from the repository that was scanned", () => {
+ const row = {
+ ...blackDuckBomRow(),
+ componentVersionOriginName: "github",
+ componentVersionOriginId: "LibraryOrg/Library/v1.0",
+ packageUrl: "pkg:github/LibraryOrg/Library@release%2Fv1.0",
+ };
+ const previous = normalizePreviousBlackDuckFindings([row], repository)
+ .findings[0]!;
+ expect(previous.evidence.packages).toEqual([]);
+ for (const target of ["local", "cloud"] as const) {
+ const current = normalizeBlackDuckFindings([row], repository, target);
+ expect(needsPreviousBlackDuckProjection(current)).toBe(true);
+ expect(current.excluded).toEqual([]);
+ expect(current.findings[0]).toMatchObject({
+ source_finding_id: previous.source_finding_id,
+ evidence: {
+ packages: [
+ {
+ ecosystem: "github",
+ name: "libraryorg/library",
+ installed_version: "release/v1.0",
+ },
+ ],
+ source_data: { sourceRepositoryUrl: repository, row },
+ },
+ });
+ }
+ row.packageUrl = "pkg:github/LibraryOrg/Library@" + "v".repeat(513);
+ const bounded = normalizeBlackDuckFindings([row], repository);
+ expect(needsPreviousBlackDuckProjection(bounded)).toBe(false);
+ expect(bounded).toEqual(
+ normalizePreviousBlackDuckFindings([row], repository),
+ );
+ expect(bounded.findings[0]!.evidence.source_data?.["row"]).toEqual(row);
+});
+
test("uses the full generic package path and keeps the origin fallback for an invalid namespace", () => {
const row = blackDuckBomRow();
const packages = (packageUrl: string) =>
diff --git a/sdk/typescript/tests-ts/endor-findings.test.ts b/sdk/typescript/tests-ts/endor-findings.test.ts
index 2dc2ed1b1..971dfd856 100644
--- a/sdk/typescript/tests-ts/endor-findings.test.ts
+++ b/sdk/typescript/tests-ts/endor-findings.test.ts
@@ -1,7 +1,10 @@
+import { hasPreviousCloudProjection } from "../src/reader-selection.js";
import { expect, test } from "bun:test";
import {
normalizeEndorFindings,
normalizeEndorSarif,
+ normalizePreviousEndorFindings,
+ requiresStructuredEndorReport,
} from "../src/endor-findings.js";
import { endorFinding, endorSarif } from "./support/endor-import-fixtures.js";
@@ -34,6 +37,102 @@ test("Endor JSON preserves the complete source, UUID, severity, aliases and repo
});
});
+test.each([
+ { oversizedPackage: false, oversizedAliases: false },
+ { oversizedPackage: false, oversizedAliases: true },
+ { oversizedPackage: true, oversizedAliases: false },
+ { oversizedPackage: true, oversizedAliases: true },
+])(
+ "Endor retains the source and projects the package and malware IDs that fit %p",
+ ({ oversizedPackage, oversizedAliases }) => {
+ const base = endorFinding();
+ const { target_dependency_name: _, ...spec } = base.spec;
+ const { vulnerability } = spec.finding_metadata;
+ const originalIds = [
+ vulnerability.meta.name,
+ ...vulnerability.spec.aliases,
+ ];
+ const name = oversizedPackage ? "x".repeat(513) : "npm://[email protected]";
+ const aliases = oversizedAliases
+ ? Array.from({ length: 100 }, (_, i) => `MAL-alias-${i}`)
+ : ["MAL-alias"];
+ const malwareId = "MAL-2099-0001";
+ const input = {
+ ...base,
+ spec: {
+ ...spec,
+ target_dependency_package_name: name,
+ finding_metadata: {
+ vulnerability,
+ malware: {
+ meta: { name: malwareId },
+ spec: { aliases: [originalIds[0], ...aliases] },
+ },
+ },
+ },
+ };
+ const previous = normalizePreviousEndorFindings(input);
+ expect(previous.findings[0]).toMatchObject({
+ source_finding_id: input.uuid,
+ evidence: { packages: [], advisory_ids: originalIds, source_data: input },
+ });
+ expect(hasPreviousCloudProjection(previous)).toBe(false);
+
+ const current = normalizeEndorFindings(input);
+ expect(current.excluded).toEqual([]);
+ expect(current.findings[0]).toMatchObject({
+ source_finding_id: input.uuid,
+ evidence: {
+ advisory_ids: oversizedAliases
+ ? originalIds
+ : [...originalIds, malwareId, ...aliases],
+ source_data: input,
+ },
+ });
+ expect(current.findings[0]!.evidence.packages!.map((p) => p.name)).toEqual(
+ oversizedPackage ? [] : spec.dependency_file_paths.map(() => name),
+ );
+ expect(hasPreviousCloudProjection(current)).toBe(
+ !oversizedPackage || !oversizedAliases,
+ );
+ expect(requiresStructuredEndorReport(current)).toBe(
+ oversizedPackage || oversizedAliases,
+ );
+ if (oversizedPackage && oversizedAliases) expect(current).toEqual(previous);
+ },
+);
+
+test("Endor keeps its original projection for duplicate or malformed malware metadata", () => {
+ const base = endorFinding();
+ const { vulnerability } = base.spec.finding_metadata;
+ for (const finding_metadata of [
+ { vulnerability, malware: vulnerability },
+ { malware: "unavailable" },
+ { malware: { meta: [], spec: "unknown" } },
+ ]) {
+ const input = {
+ ...base,
+ spec: {
+ ...base.spec,
+ target_dependency_package_name: "npm://[email protected]",
+ finding_metadata,
+ },
+ };
+ const current = normalizeEndorFindings(input);
+ expect(current.findings[0]!.evidence).toMatchObject({
+ advisory_ids: finding_metadata.vulnerability
+ ? [vulnerability.meta.name, ...vulnerability.spec.aliases]
+ : [],
+ source_data: input,
+ });
+ expect(current.findings[0]!.evidence.packages![0]!.name).toBe(
+ base.spec.target_dependency_name,
+ );
+ expect(current).toEqual(normalizePreviousEndorFindings(input));
+ expect(hasPreviousCloudProjection(current)).toBe(false);
+ }
+});
+
test("arrays and ListFindingsResponse use the same identities and empty lists are empty", () => {
const input = endorFinding();
expect(normalizeEndorFindings([input])).toEqual(
diff --git a/sdk/typescript/tests-ts/external-findings-publish.test.ts b/sdk/typescript/tests-ts/external-findings-publish.test.ts
index f0d482c29..e3b9c6e6f 100644
--- a/sdk/typescript/tests-ts/external-findings-publish.test.ts
+++ b/sdk/typescript/tests-ts/external-findings-publish.test.ts
@@ -22,7 +22,10 @@ import {
} from "./support/semgrep-import-fixtures.js";
import { endorFinding, endorSarif } from "./support/endor-import-fixtures.js";
import { sonarGitTestEnvironment } from "./support/sonar-import-fixtures.js";
-import { normalizeEndorFindings } from "../src/endor-findings.js";
+import {
+ normalizeEndorFindings,
+ normalizePreviousEndorFindings,
+} from "../src/endor-findings.js";
import { createServer, type ServerResponse } from "node:http";
import { execFile } from "node:child_process";
import { hash } from "node:crypto";
@@ -60,7 +63,10 @@ import {
selectSemgrepPlatformFindings,
semgrepPlatformFormat,
} from "../src/semgrep-platform-findings.js";
-import { normalizeSemgrepFindings } from "../src/semgrep-findings.js";
+import {
+ normalizePreviousSemgrepFindings,
+ normalizeSemgrepFindings,
+} from "../src/semgrep-findings.js";
import { collectSemgrepPlatformFindings } from "../src/semgrep-platform-collector.js";
import { normalizeSnykIssues } from "../src/snyk-rest-findings.js";
import releasedSnykCheckpoints from "./fixtures/snyk-released-checkpoints.json" with { type: "json" };
@@ -81,6 +87,7 @@ import {
selectPrivateReport,
} from "../src/private-finding-import.js";
import { readVendorFindings } from "../src/wiz-findings.js";
+import { normalizeOpenGrepFindings } from "../src/opengrep-findings.js";
import {
cloudRepositoryUrlKey,
validateExternalEvidence,
@@ -101,7 +108,10 @@ import {
importedAdvisory,
} from "./support/github-import-fixtures.js";
import { createCliTest } from "./support/cli-run.js";
-import { normalizeTrivyFindings } from "../src/trivy-findings.js";
+import {
+ normalizePreviousTrivyFindings,
+ normalizeTrivyFindings,
+} from "../src/trivy-findings.js";
import {
normalizeInspectorFindings,
normalizePreviousInspectorFindings,
@@ -109,6 +119,7 @@ import {
import { normalizePreviousContinuumFindings } from "../src/continuum-findings.js";
import {
trivyReport,
+ trivyReportWithManyLocations,
trivyRepositoryHeader,
} from "./support/trivy-import-fixtures.js";
import { prepareNormalizedLocalImport } from "../src/external-findings-local.js";
@@ -123,9 +134,16 @@ import {
continuumDetail,
continuumRepository,
} from "./support/continuum-import-fixtures.js";
-import { normalizeBlackDuckFindings } from "../src/blackduck-findings.js";
-import { normalizeVeracodeFindings } from "../src/veracode-findings.js";
import {
+ normalizeBlackDuckFindings,
+ normalizePreviousBlackDuckFindings,
+} from "../src/blackduck-findings.js";
+import {
+ normalizePreviousVeracodeFindings,
+ normalizeVeracodeFindings,
+} from "../src/veracode-findings.js";
+import {
+ veracodeMixedPolicyPage,
veracodePolicyPage,
veracodePolicyRow,
} from "./support/veracode-import-fixtures.js";
@@ -381,6 +399,15 @@ test("Wiz's gzip JSON report retains finding identity, evidence and repository l
source_data: report,
},
});
+ f.state.repositoryUrl = otherRepository;
+ await expect(
+ prepareExternalPublication(f.file, options, f.deps),
+ ).rejects.toThrow("does not match the selected Cloud repository");
+ await expect(
+ prepareNormalizedExternalPublication(parsed, options, f.deps),
+ ).rejects.toThrow("does not match the selected Cloud repository");
+ expect(f.posts).toHaveLength(0);
+ f.state.repositoryUrl = exampleRepository;
const result = await (
await prepareExternalPublication(f.file, options, f.deps)
).publish();
@@ -655,6 +682,8 @@ async function fixture(
brokenReadback: false,
throttle: false,
catalogReads: 0,
+ catalogHeaders: {} as Record<string, string>,
+ sourceLookups: [] as string[],
postBudget: Infinity,
finalError: false,
};
@@ -678,6 +707,8 @@ async function fixture(
const url = new URL(incoming.url!, "http://localhost");
if (url.pathname.endsWith("/repositories")) {
state.catalogReads++;
+ for (const [name, value] of Object.entries(state.catalogHeaders))
+ response.setHeader(name, value);
return json(response, {
object: "page",
data: [destination()],
@@ -686,7 +717,9 @@ async function fixture(
});
}
if (url.pathname.endsWith("/source_reports")) {
- const report = reports.get(url.searchParams.get("source_finding_id")!);
+ const id = url.searchParams.get("source_finding_id")!;
+ state.sourceLookups.push(id);
+ const report = reports.get(id);
let summary;
if (report) {
const { evidence, last_import: _lastImport, ...rest } = report;
@@ -886,6 +919,16 @@ async function fixture(
deps,
cliDeps,
command,
+ async writeCheckpoint(filename: string, checkpoint: unknown) {
+ const directory = join(
+ environment.CODEX_SECURITY_STATE_DIR,
+ "external-finding-publications",
+ );
+ await mkdir(directory, { recursive: true, mode: 0o700 });
+ const path = join(directory, filename);
+ await writeFile(path, JSON.stringify(checkpoint), { mode: 0o600 });
+ return path;
+ },
async loseResponse(prepared: PreparedExternalPublication) {
state.loseResponse = true;
const failure: unknown = await prepared
@@ -901,6 +944,130 @@ async function fixture(
};
}
+test.each([true, false])(
+ "repository severity exclusions keep the approved requests on retry (initial advertisement %s)",
+ async (advertised) => {
+ const f = await fixture([
+ normalized("unreported", "unknown"),
+ normalized("invalid", "unsupported"),
+ normalized("reported", "high"),
+ ]);
+ const header = "Codex-Security-Repository-Import-Severities";
+ const severities = "critical,high,medium,low,informational";
+ if (advertised) f.state.catalogHeaders[header] = severities;
+ const prepare = () => prepareExternalPublication(f.file, options, f.deps);
+ const first = await prepare();
+ const selected = advertised ? ["reported"] : ["unreported", "reported"];
+ expect(first.preview.findings.map((f) => f.source_finding_id)).toEqual(
+ selected,
+ );
+ expect(f.state.sourceLookups.sort()).toEqual([...selected].sort());
+ expect(first.preview.excluded.map(({ position }) => position)).toEqual(
+ advertised ? [1, 2] : [2],
+ );
+ if (advertised)
+ expect(first.preview.excluded[0]?.reason).toContain('severity "unknown"');
+ await f.loseResponse(first);
+ f.state.catalogHeaders = advertised ? {} : { [header]: severities };
+ const retry = await prepare();
+ expect(retry.preview.resumed).toBe(true);
+ expect(retry.preview.requests).toEqual(first.preview.requests);
+ expect(retry.preview.excluded).toEqual(first.preview.excluded);
+ const result = await retry.publish();
+ expect(result).toMatchObject({
+ status: "partial",
+ read: 3,
+ ready: selected.length,
+ verified: selected.length,
+ unacknowledged: 0,
+ });
+ expect(f.posts).toEqual([f.posts[0]!, f.posts[0]!]);
+ },
+);
+
+test.each([
+ "equivalent",
+ "other path",
+ "raw change",
+ "manifest change",
+ "URL",
+] as const)(
+ "direct readback compares qualified literal scanner paths without changing the approved request (%s)",
+ async (mode) => {
+ const path =
+ mode === "URL" ? "https://example.test/file.json" : "schema:prod.json";
+ const input = normalized();
+ const f = await fixture([
+ {
+ ...input,
+ evidence: {
+ ...input.evidence,
+ locations: [{ path, line: 5 }],
+ packages: [{ name: "example", manifest_path: path }],
+ source_data: { original_path: path },
+ },
+ },
+ ]);
+ const prepare = () =>
+ prepareExternalPublication(f.file, options, {
+ ...f.deps,
+ fetch: async (url, init) => {
+ const response = await f.deps.fetch(url, init);
+ if (!new URL(url).pathname.includes("/source_reports/"))
+ return response;
+ const report = await response.json();
+ report.evidence.locations[0].path =
+ mode === "other path" ? "./different:file.json" : `./${path}`;
+ report.evidence.packages[0].manifest_path =
+ mode === "manifest change" ? "different.json" : `./${path}`;
+ if (mode === "raw change")
+ report.evidence.source_data.original_path = `./${path}`;
+ return Response.json(report);
+ },
+ });
+ const first = await prepare();
+ await f.loseResponse(first);
+ const resumed = await prepare();
+ expect(resumed.preview.requests).toEqual(first.preview.requests);
+ const result = resumed.publish();
+ if (mode === "equivalent") expect((await result).verified).toBe(1);
+ else await expect(result).rejects.toThrow("Cloud readback did not match");
+ expect(f.posts[1]).toBe(f.posts[0]);
+ expect(JSON.parse(f.posts[0]!).items[0].evidence).toMatchObject({
+ locations: [{ path }],
+ packages: [{ manifest_path: path }],
+ source_data: { original_path: path },
+ });
+ },
+);
+
+test("repository severity restrictions exclude unsupported rows before lookups; private imports retain them", async () => {
+ const { f, prepareFile } = await privateUploadFixture(
+ [normalized("unreported", "unknown")],
+ "wiz",
+ );
+ f.state.catalogHeaders = {
+ "Codex-Security-Repository-Import-Severities": "high",
+ };
+ await expect(prepareFile()).rejects.toMatchObject({
+ selection: {
+ read: 1,
+ findings: [],
+ excluded: [{ position: 1, source_finding_id: "unreported" }],
+ },
+ });
+ expect(f.state.catalogReads).toBe(1);
+ expect(f.state.sourceLookups).toEqual([]);
+ f.state.catalogHeaders["Codex-Security-Private-Import-Providers"] = "wiz";
+ const { preview } = await prepareFile();
+ expect(preview.privateImport).toBeDefined();
+ expect(preview.findings.map(({ evidence }) => evidence.severity)).toEqual([
+ "unknown",
+ ]);
+ expect(preview.excluded).toEqual([]);
+ expect(f.posts).toHaveLength(0);
+});
+
test("a zero-record report needs no Cloud account, destination lookup, confirmation, or submission", async () => {
const f = await fixture([]);
await rm(f.environment.CODEX_HOME, { recursive: true });
@@ -1153,9 +1320,8 @@ function selectedSemgrepReport(
});
}
-test("a projected Snyk issue resumes its approved private repository after a rename", async () => {
+function privateSnykProject() {
const url = "https://git.example.test/Team/OldProject";
- const renamed = "https://git.example.test/Team/NewProject";
const capture = snykRestFixture("open-source", { sourceRepositoryUrl: url });
const project = JSON.parse(capture.projectJson);
project.attributes.origin = "cli";
@@ -1165,6 +1331,12 @@ test("a projected Snyk issue resumes its approved private repository after a ren
const report = normalizeSnykIssues(capture, "cloud", [
{ url, platform: "github" },
]);
+ return { url, capture, report };
+}
+
+test("a projected Snyk issue resumes its approved private repository after a rename", async () => {
+ const { url, capture, report } = privateSnykProject();
+ const renamed = "https://git.example.test/Team/NewProject";
const row = report.findings[0]!;
row.evidence.source_data = {
issueJson: row.evidence.source_data!["issueJson"],
@@ -1183,6 +1355,88 @@ test("a projected Snyk issue resumes its approved private repository after a ren
expect(fresh.posts).toHaveLength(0);
});
+async function privateUploadFixture(
+ raw: unknown,
+ provider: "endor" | "snyk" | "wiz",
+ repositoryUrl?: string,
+) {
+ const f = await fixture(raw, repositoryUrl);
+ const settings = { ...options, provider };
+ f.state.catalogHeaders["Codex-Security-Private-Import-Providers"] = provider;
+ const inbox = {
+ version: "original-inbox",
+ formats: ["security.findings.v1"],
+ recoverUpload: false,
+ };
+ const uploads: { id: string; bytes: string }[] = [];
+ const jobs: unknown[] = [];
+ const fileId = "aifile_" + "a".repeat(32);
+ const deps = {
+ ...f.deps,
+ fetch: async (url: string, init: RequestInit) => {
+ const { pathname } = new URL(url);
+ if (pathname.endsWith("/finding-imports/context"))
+ return privateInboxContext(inbox.version, inbox.formats);
+ if (pathname.includes("/finding-imports/file-requests/")) {
+ const prior = uploads.find((upload) =>
+ pathname.endsWith("/" + upload.id),
+ );
+ return inbox.recoverUpload && prior
+ ? Response.json(privateFileMetadata(fileId, prior.bytes))
+ : new Response(null, { status: 404 });
+ }
+ if (pathname.endsWith("/finding-imports/jobs")) {
+ expect(init.method).toBe("POST");
+ const job = JSON.parse(String(init.body));
+ jobs.push(job);
+ return Response.json(
+ privateJobReceipt(job, "import_" + "b".repeat(32)),
+ );
+ }
+ if (pathname.endsWith("/finding-imports/files")) {
+ const form = init.body as FormData;
+ uploads.push({
+ id: String(form.get("request_id")),
+ bytes: await (form.get("file") as Blob).text(),
+ });
+ throw new Error("Interrupted private upload");
+ }
+ return f.deps.fetch(url, init);
+ },
+ };
+ const prepareFile = () => prepareExternalPublication(f.file, settings, deps);
+ return { f, settings, deps, inbox, uploads, jobs, fileId, prepareFile };
+}
+
+test("a Snyk private upload resumes after a rename and still checks supplemental SDK repository claims", async () => {
+ const { url, capture, report } = privateSnykProject();
+ const { f, settings, deps, inbox, uploads, prepareFile } =
+ await privateUploadFixture(capture, "snyk", url);
+ const original = await prepareFile();
+ await expect(original.publish()).rejects.toThrow(
+ "Interrupted private upload",
+ );
+ f.state.repositoryUrl = "https://git.example.test/Team/NewProject";
+ const conflicting = { ...report, sourceRepositoryUrl: f.state.repositoryUrl };
+ await expect(
+ prepareNormalizedExternalPublication(conflicting, settings, deps),
+ ).rejects.toThrow(/repository|project/iu);
+ expect((await prepareFile()).preview.resumed).toBe(true);
+ const retry = await prepareNormalizedExternalPublication(
+ report,
+ settings,
+ deps,
+ );
+ expect(retry.preview.resumed).toBe(true);
+ expect(retry.preview.privateImport).toEqual(original.preview.privateImport);
+ await expect(retry.publish()).rejects.toThrow("Interrupted private upload");
+ expect(uploads).toHaveLength(2);
+ expect(uploads[1]).toEqual(uploads[0]);
+
+ inbox.version = "new-inbox";
+ await expect(prepareFile()).rejects.toThrow(/repository|project/iu);
+});
+
test("a self-contained Snyk issue keeps its Cloud identity with or without a report header", async () => {
const capture = snykRestFixture("open-source");
const report = normalizeSnykIssues(capture);
@@ -2585,14 +2839,7 @@ test.each(["open-source", "code"] as const)(
// The released SDK approved these findings without retaining a native issue.
const { file, checkpoint } = releasedSnykCheckpoints[product];
- const directory = join(
- f.environment.CODEX_SECURITY_STATE_DIR,
- "external-finding-publications",
- );
- await mkdir(directory, { recursive: true, mode: 0o700 });
- await writeFile(join(directory, file), JSON.stringify(checkpoint), {
- mode: 0o600,
- });
+ await f.writeCheckpoint(file, checkpoint);
const first = await f.prepareNormalized(bare);
expect(first.preview.resumed).toBe(true);
await f.loseResponse(first);
@@ -3600,16 +3847,15 @@ test.each([
},
);
-test("Sonar replays the selected evidence locally and to Cloud without signing in to Sonar again", async () => {
- const repo = "https://github.com/example/project";
- const source = {
+function sonarCapture(repositoryUrl: string) {
+ return {
format: "sonar-issues/v1" as const,
server: "https://sonarcloud.io",
organization: "example",
project: "example_project",
branch: "main",
projectBase: "",
- repositoryUrl: repo,
+ repositoryUrl,
pages: [
{
paging: { pageIndex: 1, pageSize: 100, total: 1 },
@@ -3638,6 +3884,11 @@ test("Sonar replays the selected evidence locally and to Cloud without signing i
},
],
};
+}
+
+test("Sonar replays the selected evidence locally and to Cloud without signing in to Sonar again", async () => {
+ const repo = "https://github.com/example/project";
+ const source = sonarCapture(repo);
const f = await fixture(source, repo);
const checkout = join(f.root, "checkout");
const git = promisify(execFile);
@@ -4235,13 +4486,17 @@ async function inspectorPublicationFixture(name = inspectorRepositoryUrl) {
return { f, input, project, map, prepare, loseResponse };
}
-test("Checkmarx publishes a native SAST finding only to its declared Cloud repository", async () => {
- const row = {
+function checkmarxSastRow() {
+ return {
type: "sast",
similarityId: "17",
severity: "MEDIUM",
data: { resultHash: "c97c", queryName: "Example_Query" },
};
+}
+
+test("Checkmarx publishes a native SAST finding only to its declared Cloud repository", async () => {
+ const row = checkmarxSastRow();
const f = await fixture({ scanID: "scan-a", totalCount: 1, results: [row] });
const selected = {
...options,
@@ -4550,7 +4805,16 @@ function inspectorReportWithRelatedCves() {
return { findings: [inspectorCodeFinding(), pkg] };
}
-test.each([
+const exampleRepository = "https://github.com/example/project";
+const githubOrigin = {
+ ...blackDuckBomRow(),
+ componentVersionOriginName: "github",
+ componentVersionOriginId: "example/library/v1",
+ packageUrl: "pkg:github/Example/Library@v1",
+};
+const nativeSemgrep = semgrepJson();
+
+const awsReaderUpgrades = [
{
provider: "inspector",
raw: inspectorReportWithRelatedCves(),
@@ -4559,6 +4823,8 @@ test.each([
previousDigest:
"4e90748318e5952a1fa90ff9c25153396340a097db6112f27bdf89a3a092dade",
expectedCount: 2,
+ previousSha256:
+ "91b5a4d309daee520029a3d54e809ad84d8b47ee97df5fd54d56323fc246f640",
},
{
provider: "continuum",
@@ -4568,6 +4834,59 @@ test.each([
previousDigest:
"22307dedb51cefecf56d11fd68f45324dceb5226bfddfbd48c635a462bb68fb5",
expectedCount: 1,
+ previousSha256:
+ "98f888f8b748bb54a209b19de53af40867f5f129049a99da0387e697895aa320",
+ },
+] as const;
+
+test.each([
+ ...awsReaderUpgrades,
+ {
+ provider: "blackduck",
+ raw: blackDuckBomReport([githubOrigin]),
+ normalizePrevious: normalizePreviousBlackDuckFindings,
+ sourceRepositoryUrl: exampleRepository,
+ previousDigest:
+ "ade8411242d3cac1fb490a86159d1b4a3b57a7d7f18bee64cadc971e3433185e",
+ expectedCount: 1,
+ },
+ {
+ provider: "veracode",
+ raw: veracodeMixedPolicyPage(),
+ normalizePrevious: normalizePreviousVeracodeFindings,
+ sourceRepositoryUrl: exampleRepository,
+ previousDigest:
+ "0ca5d464a797a1a17e18a69f9b4cad172a458be2955c8a24048df78e2798b8b6",
+ expectedCount: 2,
+ },
+ {
+ provider: "semgrep",
+ raw: {
+ ...nativeSemgrep,
+ explanations: [
+ {
+ op: "And",
+ matches: [{ ...nativeSemgrep.results[0]!, path: "src/other.ts" }],
+ children: [],
+ },
+ ],
+ },
+ normalizePrevious: (raw: unknown, sourceRepositoryUrl: string) =>
+ normalizePreviousSemgrepFindings(raw, { sourceRepositoryUrl }),
+ sourceRepositoryUrl: exampleRepository,
+ previousDigest:
+ "2265f4b64025fe061d950e8b20eba222563001a781b645846646000fbbcdc4a0",
+ expectedCount: 1,
+ },
+ {
+ provider: "trivy",
+ raw: trivyReportWithManyLocations(),
+ normalizePrevious: (raw: unknown) =>
+ normalizePreviousTrivyFindings(raw, "cloud", "."),
+ sourceRepositoryUrl: exampleRepository,
+ previousDigest:
+ "b4158738af7b92798b6f2b77d8f07343a9422cfb0ffafe4db6410a2bfe7d3b1e",
+ expectedCount: 2,
},
] as const)(
"a raw $provider retry preserves the repository request from the previous reader",
@@ -4580,7 +4899,13 @@ test.each([
expectedCount,
}) => {
const f = await fixture(raw, sourceRepositoryUrl);
- const settings = { ...options, provider, sourceRepositoryUrl };
+ const settings = {
+ ...options,
+ provider,
+ ...(provider === "trivy"
+ ? { trivyScanRoot: "." }
+ : { sourceRepositoryUrl }),
+ };
const prepare = () => prepareExternalPublication(f.file, settings, f.deps);
// Round-trip the previous SDK's output through JSON as a saved caller artifact.
@@ -4609,9 +4934,26 @@ test.each([
expect(fresh.preview.resumed).toBe(false);
expect(fresh.preview.findings).toHaveLength(expectedCount);
expect(fresh.preview.findings).not.toEqual(original.preview.findings);
+ expect((await fresh.publish()).verified).toBe(expectedCount);
},
);
+test("publishes a filtered numeric Veracode CWE that the previous reader rejected", async () => {
+ const raw = veracodeMixedPolicyPage();
+ raw._links.self.href += "&cwe=79";
+ const f = await fixture(raw, exampleRepository);
+ const settings = {
+ ...options,
+ provider: "veracode" as const,
+ sourceRepositoryUrl: exampleRepository,
+ };
+ expect(() =>
+ normalizePreviousVeracodeFindings(raw, exampleRepository),
+ ).toThrow("Veracode cwe must be an object");
+ const prepared = await prepareExternalPublication(f.file, settings, f.deps);
+ expect((await prepared.publish()).verified).toBe(2);
+});
+
test("a fresh AWS publication waits for the previous reader to save its request", async () => {
const raw = continuumDetail();
const f = await fixture(raw, continuumRepository);
@@ -4667,7 +5009,6 @@ const savedDependabotData = {
};
const checkmarxFormat = { format: "cxsast-xml" };
-const exampleRepository = "https://github.com/example/project";
const otherRepository = "https://github.com/example/other";
const savedEndorData = (repositoryUri: string) => ({
...checkmarxFormat,
@@ -6984,13 +7325,18 @@ test("readback retry preserves acknowledged findings and scanner diagnostics", a
expect(f.posts).toHaveLength(1);
});
-test("repository selection is sent on every discovery page", async () => {
- const f = await fixture();
+test("each discovery page requests the repository and refreshes its private import capabilities", async () => {
+ const f = await fixture(blackDuckBomReport());
const repository = `${f.destination().url}.git`;
const queries: URLSearchParams[] = [];
const prepared = await prepareExternalPublication(
f.file,
- { ...options, repository },
+ {
+ ...options,
+ repository,
+ provider: "blackduck",
+ sourceRepositoryUrl: f.destination().url,
+ },
{
...f.deps,
fetch: async (input, init) => {
@@ -6999,15 +7345,26 @@ test("repository selection is sent on every discovery page", async () => {
return f.deps.fetch(input, init);
queries.push(url.searchParams);
const first = queries.length === 1;
- return Response.json({
- object: "page",
- data: first ? [] : [f.destination()],
- has_more: first,
- next: first ? "next-page" : null,
- });
+ return Response.json(
+ {
+ object: "page",
+ data: first ? [] : [f.destination()],
+ has_more: first,
+ next: first ? "next-page" : null,
+ },
+ {
+ headers: first
+ ? {
+ "Codex-Security-Private-Import-Job-Version": "3",
+ "Codex-Security-Private-Import-Providers": "blackduck",
+ }
+ : {},
+ },
+ );
},
},
);
+ expect(prepared.preview.privateImport).toBeUndefined();
expect(prepared.preview.destination.id).toBe(options.repository);
expect(queries.map((query) => query.get("repository"))).toEqual([
repository,
@@ -7389,6 +7746,44 @@ test("GitHub publication previews, imports, re-imports, updates and resumes sele
expect(f.reports.size).toBe(2);
});
+test.each([
+ ["code scanning", () => normalizeGitHubCodeScanningAlerts([importedAlert()])],
+ [
+ "repository advisory",
+ () => normalizeGitHubAdvisories([importedAdvisory()]),
+ ],
+] as const)(
+ "the retained GitHub %s export binds its SDK projection to the original repository",
+ async (_, make) => {
+ const input = make();
+ const evidence = input.findings[0]!.evidence;
+ const f = await fixture(undefined, otherRepository);
+ const prepare = () =>
+ prepareNormalizedExternalPublication(
+ input,
+ { ...options, provider: "github" },
+ f.deps,
+ );
+ evidence.details!.repository!.url = otherRepository;
+ await expect(prepare()).rejects.toThrow(
+ "does not match the original GitHub finding",
+ );
+ delete evidence.details;
+ await expect(prepare()).rejects.toThrow(
+ "does not match the selected Cloud repository",
+ );
+ expect(f.posts).toHaveLength(0);
+
+ f.state.repositoryUrl = exampleRepository;
+ await f.loseResponse(await prepare());
+ f.state.repositoryUrl = otherRepository;
+ const retry = await prepare();
+ expect(retry.preview.resumed).toBe(true);
+ expect((await retry.publish()).verified).toBe(1);
+ expect(f.posts[1]).toBe(f.posts[0]);
+ },
+);
+
test("GitHub imports reject a different Cloud destination before uploading", async () => {
const imported = importedAlert("refs/heads/main", "git.example.test");
const f = await fixture([imported]);
@@ -9257,6 +9652,50 @@ for (const [provider, report, normalize, dryRunTitle] of [
expect(f.posts).toHaveLength(0);
});
+ test(`${provider} checks empty SARIF runs before a fresh submission`, async () => {
+ const source = report();
+ const run = {
+ ...source.runs[0]!,
+ results: [],
+ versionControlProvenance: [
+ {
+ repositoryUri:
+ provider === "snyk"
+ ? "git+ssh://[email protected]/example/project.git"
+ : exampleRepository,
+ },
+ ],
+ };
+ const input = { ...source, runs: [...source.runs, run] };
+ const f = await fixture(input);
+ const selected = { ...options, provider };
+ const prepare = () => prepareExternalPublication(f.file, selected, f.deps);
+ expect((await prepare()).preview.findings).toHaveLength(1);
+
+ run.versionControlProvenance[0]!.repositoryUri = otherRepository;
+ await writeFile(f.file, JSON.stringify(input));
+ await expect(prepare()).rejects.toThrow(
+ "does not match the selected Cloud repository",
+ );
+ expect(f.posts).toHaveLength(0);
+
+ // An SDK selection carries only its result-bearing run. Existing approved
+ // requests must send those exact bytes, even if the native file has other runs.
+ const sdk = await prepareNormalizedExternalPublication(
+ normalize(input),
+ selected,
+ f.deps,
+ );
+ await f.loseResponse(sdk);
+ const retry = await prepare();
+ expect(retry.preview.resumed).toBe(true);
+ expect((await retry.publish()).verified).toBe(1);
+ expect(f.posts).toEqual([f.posts[0]!, f.posts[0]!]);
+ await expect(prepare()).rejects.toThrow(
+ "does not match the selected Cloud repository",
+ );
+ });
+
test.each([
{ uris: ["https://github.com/other/project"] },
{
@@ -9297,6 +9736,25 @@ for (const [provider, report, normalize, dryRunTitle] of [
);
}
+test("Snyk checks empty projects alongside findings in a CLI JSON export", async () => {
+ const project = { ...snykJson(), path: exampleRepository };
+ const other = {
+ ...project,
+ ok: true,
+ vulnerabilities: [],
+ path: otherRepository,
+ };
+ const f = await fixture([project, other]);
+ await expect(
+ prepareExternalPublication(
+ f.file,
+ { ...options, provider: "snyk" },
+ f.deps,
+ ),
+ ).rejects.toThrow("does not match the selected Cloud repository");
+ expect(f.posts).toHaveLength(0);
+});
+
test("native Snyk remote tests bind file and normalized imports to the reported GitHub repository", async () => {
const input = snykJson();
input.path = "https://github.com/other/project#release/v1";
@@ -9884,6 +10342,11 @@ test.each([
gitHubProject,
] as const,
["plugin", "unknown", gitHubProject] as const,
+ [
+ "plugin",
+ "git+ssh://[email protected]/example/project.git",
+ gitHubProject,
+ ] as const,
])(
"Snyk %s export imports and replays with repository remote %s",
async (format, repository, destination) => {
@@ -10060,6 +10523,11 @@ test.each([
gitHubProject,
] as const,
["SARIF", "unknown", gitHubProject] as const,
+ [
+ "plugin",
+ "ssh://[email protected]/example/project.git?other",
+ gitHubProject,
+ ] as const,
])(
"Snyk %s repository claim %s cannot target %s",
async (format, repository, destination) => {
@@ -10246,6 +10714,66 @@ test("OpenGrep publication retains grouped evidence through Cloud readback and r
);
});
+test("repository severity exclusions identify every original OpenGrep trace", async () => {
+ const report = opengrepTraceJson();
+ const [first, second] = report.results;
+ const other = { ...first!, path: "src/other.ts" };
+ const input = {
+ ...report,
+ errors: [],
+ results: [first, other, null, second],
+ };
+ const f = await fixture(input);
+ const settings = {
+ ...options,
+ provider: "opengrep" as const,
+ sourceRepositoryUrl: "https://github.com/example/project",
+ };
+ const parsed = normalizeOpenGrepFindings(input, settings);
+ const ids = new Map(
+ parsed.findings.map(({ source_finding_id, evidence }) => [
+ evidence.locations![0]!.path,
+ source_finding_id,
+ ]),
+ );
+ f.state.catalogHeaders["Codex-Security-Repository-Import-Severities"] =
+ "high";
+ const prepare = (input: ExternalFindings) =>
+ prepareNormalizedExternalPublication(
+ input,
+ { ...settings, provider: "semgrep" },
+ f.deps,
+ );
+ const originalIds = [
+ ids.get(first!.path)!,
+ ids.get(other.path)!,
+ null,
+ ids.get(first!.path)!,
+ ];
+ for (const [load, expectedIds] of [
+ [() => prepareExternalPublication(f.file, settings, f.deps), originalIds],
+ [() => prepare({ ...parsed }), originalIds],
+ [() => prepare(structuredClone(parsed)), [null, null, null, null]],
+ [
+ () => prepare(JSON.parse(JSON.stringify(parsed))),
+ [null, null, null, null],
+ ],
+ ] as const) {
+ const failure = await load().catch((error: unknown) => error);
+ expect(failure).toBeInstanceOf(NoPublishableFindingsError);
+ const { selection, summary } = failure as NoPublishableFindingsError;
+ expect(summary).toContain("4 read, 4 excluded");
+ expect(selection.excluded.map(({ position }) => position)).toEqual([
+ 1, 2, 3, 4,
+ ]);
+ expect(
+ selection.excluded.map(({ source_finding_id }) => source_finding_id),
+ ).toEqual([...expectedIds]);
+ expect(selection.excluded[2]).toEqual(parsed.excluded[0]);
+ }
+ expect(f.posts).toHaveLength(0);
+});
+
test("empty OpenGrep Cloud imports retain scanner diagnostics without uploading", async () => {
const input = opengrepJson();
input.results = [];
@@ -11268,31 +11796,212 @@ test("the GitHub reader resumes a released boxed score whose Buffer fit the Clou
expect(f.posts.map((body) => hash("sha256", body))).toEqual([bodyHash]);
});
+const sonarReport = sonarCapture(exampleRepository);
+const trivy = trivyReport();
+const endorSource = endorFinding();
+const boundEndorSarif = endorSarif();
+Object.assign(boundEndorSarif.runs[0]!, {
+ versionControlProvenance: [{ repositoryUri: exampleRepository }],
+});
+const endorVulnerability = endorSource.spec.finding_metadata.vulnerability;
+test.each(["package", "malware aliases"] as const)(
+ "the Endor private upload retains the reviewed %s when its other enrichment exceeds Cloud limits",
+ async (retained) => {
+ const keepPackage = retained === "package";
+ const packageName = keepPackage ? "alternate-package" : "x".repeat(513);
+ const aliases = keepPackage
+ ? Array.from({ length: 99 }, (_, i) => `MAL-${i}`)
+ : ["MAL-example"];
+ const raw = {
+ ...endorSource,
+ spec: {
+ ...endorSource.spec,
+ target_dependency_name: null,
+ target_dependency_package_name: packageName,
+ finding_metadata: {
+ vulnerability: endorVulnerability,
+ malware: { spec: { aliases } },
+ },
+ },
+ };
+ const { inbox, uploads, prepareFile } = await privateUploadFixture(
+ raw,
+ "endor",
+ );
+ inbox.formats = ["endor.findings_json.v1"];
+ await expect(prepareFile()).rejects.toThrow("security.findings.v1");
+ inbox.formats.push("security.findings.v1");
+ const prepared = await prepareFile();
+ expect(prepared.preview.privateImport?.format).toBe("security.findings.v1");
+ await expect(prepared.publish()).rejects.toThrow(
+ "Interrupted private upload",
+ );
+ const { findings, format } = JSON.parse(uploads[0]!.bytes);
+ expect(format).toBe("security.findings.v1");
+ expect(findings).toHaveLength(1);
+ const { evidence } = findings[0];
+ expect(evidence.source_data).toEqual(raw);
+ expect(evidence.advisory_ids).toEqual([
+ endorVulnerability.meta.name,
+ ...endorVulnerability.spec.aliases,
+ ...(keepPackage ? [] : aliases),
+ ]);
+ expect(evidence.packages ?? []).toEqual(
+ keepPackage
+ ? endorSource.spec.dependency_file_paths.map((path) => ({
+ name: packageName,
+ ecosystem: "npm",
+ installed_version: endorSource.spec.target_dependency_version,
+ manifest_path: path,
+ fixed_versions: [],
+ }))
+ : [],
+ );
+ },
+);
+
+test("an interrupted native Endor checkpoint retains its original bytes and IDs until its inbox changes", async () => {
+ const raw = {
+ ...endorSource,
+ extra: "x".repeat(190 * 1024),
+ spec: {
+ ...endorSource.spec,
+ target_dependency_name: null,
+ target_dependency_package_name: "x".repeat(495),
+ dependency_file_paths: Array.from(
+ { length: 100 },
+ (_, i) => `modules/${i}/yarn.lock`,
+ ),
+ },
+ };
+ const { f, inbox, uploads, prepareFile, jobs, fileId, settings } =
+ await privateUploadFixture(raw, "endor");
+ f.state.catalogHeaders = {};
+ const repositoryPreview = await prepareFile();
+ f.state.catalogHeaders["Codex-Security-Private-Import-Providers"] = "endor";
+ const fresh = await prepareFile();
+ expect(fresh.preview.privateImport?.format).toBe("security.findings.v1");
+ expect(fresh.preview.findings[0]!.evidence).toMatchObject({
+ packages: [],
+ source_data: raw,
+ });
+ inbox.formats = ["endor.findings_json.v1"];
+ await expect(prepareFile()).rejects.toThrow("security.findings.v1");
+ // This released checkpoint predates structured-only uploads for reduced Endor fields.
+ const key =
+ "52629feec0c5f197e2bce1779b908306b6b01d232112b359451da0da111c6ccd";
+ const checkpoint = {
+ format: "endor.findings_json.v1",
+ apiBaseUrl: "https://chatgpt.com/backend-api/aardvark",
+ accountId: "synthetic-account",
+ contextVersion: inbox.version,
+ source: { provider: "endor", source_key: settings.sourceKey },
+ repository: { id: options.repository, repo_connector_id: "github" },
+ sha256: "2aba52fd27bf9b6b2fdf6cb84b701a61690266a29924fb6d6a25e9a4c1574341",
+ uploadRequestId: "200ce23c-1576-4525-8b58-50c43812bce8",
+ jobRequestId: "9bceaf8a-8ef4-4bbc-99a7-78fc0d6dcbcf",
+ };
+ await f.writeCheckpoint(key + ".private-import.json", checkpoint);
+ const retry = await prepareFile();
+ expect(retry.preview.resumed).toBe(true);
+ expect(retry.preview.privateImport).toMatchObject({
+ format: checkpoint.format,
+ sha256: checkpoint.sha256,
+ });
+ expect(checkpoint.sha256).not.toBe(fresh.preview.privateImport?.sha256);
+ await expect(retry.publish()).rejects.toThrow("Interrupted private upload");
+ expect(uploads).toEqual([
+ { id: checkpoint.uploadRequestId, bytes: JSON.stringify(raw) },
+ ]);
+ await expect(repositoryPreview.publish()).rejects.toThrow(
+ "A private import already owns this selection",
+ );
+ inbox.recoverUpload = true;
+ expect((await (await prepareFile()).publish()).privateJob?.status).toBe(
+ "completed",
+ );
+ expect(jobs).toEqual([
+ {
+ request_id: checkpoint.jobRequestId,
+ context_version: inbox.version,
+ ...privateJobFields("endor", fileId, checkpoint.format),
+ },
+ ]);
+ inbox.version = "new-inbox";
+ inbox.formats.push("security.findings.v1");
+ const next = await prepareFile();
+ expect(next.preview.resumed).toBe(false);
+ expect(next.preview.privateImport?.sha256).toBe(
+ fresh.preview.privateImport?.sha256,
+ );
+ inbox.recoverUpload = false;
+ await expect(next.publish()).rejects.toThrow("Interrupted private upload");
+ const uploaded = JSON.parse(uploads.at(-1)!.bytes);
+ expect(uploaded.format).toBe("security.findings.v1");
+ expect(uploaded.findings).toHaveLength(1);
+ expect(uploaded.findings[0].evidence).toMatchObject({
+ packages: [],
+ source_data: raw,
+ });
+});
+
+const versionThreeReports = [
+ ["blackduck", blackDuckBomReport(), { row: blackDuckBomRow() }],
+ [
+ "checkmarx",
+ { scanID: "scan-a", totalCount: 1, results: [checkmarxSastRow()] },
+ { row: checkmarxSastRow() },
+ ],
+ ["sonar", sonarReport, { issue: sonarReport.pages[0]!.issues[0]! }],
+ ["trivy", trivy, { vulnerability: trivy.Results[0]!.Vulnerabilities[0]! }],
+ ["veracode", veracodePolicyPage(), { row: veracodePolicyRow() }],
+] as const;
+
test.each([
+ ...versionThreeReports.map(([provider, raw, retained]) => ({
+ provider,
+ raw,
+ retained,
+ sourceRepositoryUrl: provider === "trivy" ? undefined : exampleRepository,
+ format: "security.findings.v1" as const,
+ jobVersion: 3 as const,
+ })),
{
provider: "github",
format: "security.findings.v1",
raw: [importedAlert()],
},
+ ...awsReaderUpgrades.map((upgrade) => ({
+ ...upgrade,
+ format: "security.findings.v1" as const,
+ jobVersion: 2 as const,
+ })),
+ { provider: "snyk", format: "security.findings.v1", raw: snykSarif() },
{
- provider: "inspector",
+ provider: "endor",
format: "security.findings.v1",
- raw: inspectorReportWithRelatedCves(),
- sourceRepositoryUrl: inspectorRepositoryUrl,
- previousReader: normalizePreviousInspectorFindings,
+ raw: [
+ {
+ uuid: "malware-example",
+ meta: { name: "Malicious package" },
+ spec: {
+ project_uuid: "project",
+ target_dependency_package_name: "npm://[email protected]",
+ finding_metadata: { malware: { meta: { name: "MAL-2099-0001" } } },
+ },
+ },
+ ],
+ normalizePrevious: (report: unknown) =>
+ normalizePreviousEndorFindings(report),
previousSha256:
- "91b5a4d309daee520029a3d54e809ad84d8b47ee97df5fd54d56323fc246f640",
+ "a6a869d6ac1ac02acf44849bbd55d571e350ff9ef3c8ca8562891eb8cdd64708",
},
{
- provider: "continuum",
+ provider: "endor",
format: "security.findings.v1",
- raw: continuumDetail(),
- sourceRepositoryUrl: continuumRepository,
- previousReader: normalizePreviousContinuumFindings,
- previousSha256:
- "98f888f8b748bb54a209b19de53af40867f5f129049a99da0387e697895aa320",
+ raw: boundEndorSarif,
+ rename: true,
},
- { provider: "snyk", format: "security.findings.v1", raw: snykSarif() },
{
provider: "endor",
format: "endor.findings_json.v1",
@@ -11321,7 +12030,12 @@ test.each([
: undefined;
const f = await fixture(raw);
const originalBytes = await readFile(f.file, "utf8");
- const settings = { ...options, provider, sourceRepositoryUrl };
+ const settings = {
+ ...options,
+ provider,
+ sourceRepositoryUrl,
+ ...(provider === "trivy" && { trivyScanRoot: "." }),
+ };
let contextVersion = "generation-a";
let formats = [
"security.findings.v1",
@@ -11334,13 +12048,16 @@ test.each([
let terminal: "completed" | "failed" | "cancelled" = "completed";
let advertised = false;
const v1Jobs = "/finding-imports/jobs";
- const v2Jobs = "/v2" + v1Jobs;
- let supportsJobV2 = provider !== "inspector";
- let expectedJobsPath = provider === "continuum" ? v2Jobs : v1Jobs;
+ const jobVersion =
+ "jobVersion" in scenario ? scenario.jobVersion : undefined;
+ const requestedVersion = String(jobVersion ?? 2);
+ const versionedJobs = jobVersion ? `/v${jobVersion}${v1Jobs}` : v1Jobs;
+ const upgrade = provider === "inspector" || provider === "checkmarx";
+ // An older acknowledgement also keeps the provider on its unversioned route.
+ let acknowledgedVersion = upgrade ? undefined : requestedVersion;
+ if (provider === "checkmarx") acknowledgedVersion = "2";
+ let expectedJobsPath = upgrade ? v1Jobs : versionedJobs;
let canCreate = true;
- let fileRequest: string | undefined;
- let fileHash: string | undefined;
- let fileBytes = 0;
let reportCount = 1;
let unorderedUpdates = 0;
const uploadedReports: { id: string; bytes: string }[] = [];
@@ -11350,39 +12067,17 @@ test.each([
const fileId = "aifile_" + "a".repeat(32);
let jobId = "import_" + "b".repeat(32);
const calls: string[] = [];
- const fileMetadata = () => ({
- object: "security.finding_import_file",
- id: fileId,
- purpose: "report",
- sha256: fileHash,
- bytes: fileBytes,
- });
- const job = (status: string) => ({
- object: "security.finding_import_job",
- id: jobId,
- request_id: jobRequest,
- source: { provider, source_key: settings.sourceKey },
- input: { type: "file", file_id: fileId, format: expectedFormat },
- default_targets: [
- {
- type: "repository",
- repo_id: options.repository,
- repo_connector_id: "github",
- },
- ],
- status,
- counts: {
- total: reportCount,
- discovered: reportCount,
- created: status === "completed" ? reportCount - unorderedUpdates : 0,
- updated: 0,
- unchanged: 0,
- error: status === "completed" ? unorderedUpdates : 0,
- unprocessed: status === "completed" ? 0 : reportCount,
- },
- poll_after_seconds: null,
- error: null,
- });
+ const fileMetadata = () =>
+ privateFileMetadata(fileId, uploadedReports.at(-1)!.bytes);
+ const jobFields = () => privateJobFields(provider, fileId, expectedFormat);
+ const job = (status: string) =>
+ privateJobReceipt(
+ { ...jobFields(), request_id: jobRequest },
+ jobId,
+ status,
+ reportCount,
+ unorderedUpdates,
+ );
const deps = {
...f.deps,
fetch: async (url: string, init: RequestInit) => {
@@ -11399,34 +12094,20 @@ test.each([
const response = await f.deps.fetch(url, init);
const headers = new Headers();
if (advertised)
- headers.set(
- "Codex-Security-Private-Import-Providers",
- "snyk,github,endor,wiz,inspector,continuum",
- );
+ headers.set("Codex-Security-Private-Import-Providers", provider);
const version = "Codex-Security-Private-Import-Job-Version";
- expect(new Headers(init.headers).get(version)).toBe("2");
- if (supportsJobV2) headers.set(version, "2");
+ expect(new Headers(init.headers).get(version)).toBe(requestedVersion);
+ if (acknowledgedVersion) headers.set(version, acknowledgedVersion);
return new Response(response.body, { headers });
}
if (path.startsWith("/external/")) return f.deps.fetch(url, init);
if (path === "/finding-imports/context")
- return Response.json({
- object: "security.finding_import_context",
- context_version: contextVersion,
- can_upload: canCreate,
- can_create_import_jobs: canCreate,
- can_read: true,
- formats,
- limits: {
- max_direct_file_bytes: 32 * 1024 * 1024,
- max_record_bytes: 1024 * 1024,
- },
- });
+ return privateInboxContext(contextVersion, formats, canCreate);
if (path === "/finding-imports/files") {
const form = init.body as FormData;
expect(form.get("context_version")).toBe(contextVersion);
expect(form.get("purpose")).toBe("report");
- fileRequest = String(form.get("request_id"));
+ const id = String(form.get("request_id"));
const bytes = await (form.get("file") as Blob).text();
if (expectedFormat === "security.findings.v1") {
const doc = JSON.parse(bytes);
@@ -11434,6 +12115,12 @@ test.each([
expect(doc.findings[0].source_finding_id).toBeTruthy();
expect(doc.findings[0].evidence.source_data).toBeTruthy();
reportCount = doc.findings.length;
+ if ("retained" in scenario) {
+ expect(doc.findings).toHaveLength(1);
+ expect(doc.findings[0].evidence.source_data).toMatchObject(
+ scenario.retained,
+ );
+ }
if (provider === "inspector" || provider === "continuum") {
const native = provider === "inspector" ? raw.findings : [raw];
const expected = loseResponses ? native.slice(0, 1) : native;
@@ -11449,13 +12136,12 @@ test.each([
} else {
expect(bytes).toBe(originalBytes);
}
- uploadedReports.push({ id: fileRequest, bytes });
- fileHash = hash("sha256", bytes);
- fileBytes = Buffer.byteLength(bytes);
+ uploadedReports.push({ id, bytes });
if (loseResponses && uploadedReports.length === 1)
throw new Error("lost upload response");
return Response.json(fileMetadata());
}
+ const fileRequest = uploadedReports.at(-1)?.id;
if (path === `/finding-imports/file-requests/${fileRequest}`) {
if ("previousSha256" in scenario && uploadedReports.length === 1)
return new Response(null, { status: 404 });
@@ -11465,22 +12151,11 @@ test.each([
jobBodies.push(String(init.body));
const body = JSON.parse(String(init.body));
expect(body.request_id).not.toBe(fileRequest);
- expect(body.source).toEqual({
- provider,
- source_key: settings.sourceKey,
- });
- expect(body.input).toEqual({
- type: "file",
- file_id: fileId,
- format: expectedFormat,
+ expect(body).toEqual({
+ request_id: body.request_id,
+ context_version: contextVersion,
+ ...jobFields(),
});
- expect(body.default_targets).toEqual([
- {
- type: "repository",
- repo_id: options.repository,
- repo_connector_id: "github",
- },
- ]);
if (!loseResponses) {
expect(body.request_id).not.toBe(jobRequest);
jobRequest = body.request_id;
@@ -11516,6 +12191,7 @@ test.each([
expect(original.preview.privateImport?.format).toBe(format);
const cli = createCliTest(main);
const dryRun = [...f.command, "--dry-run"];
+ if (provider === "trivy") dryRun.push("--trivy-scan-root", ".");
if (sourceRepositoryUrl)
dryRun.push("--source-repository-url", sourceRepositoryUrl);
dryRun[dryRun.indexOf("--provider") + 1] = provider;
@@ -11526,20 +12202,24 @@ test.each([
expect(cli.stderr.text()).toContain(
`Report: ${format} (${original.preview.privateImport?.bytes} bytes)`,
);
+ if (format === "endor.findings_json.v1") {
+ const supported = formats;
+ formats = [format];
+ expect((await prepare()).preview.privateImport?.format).toBe(format);
+ formats = supported;
+ }
expect(calls.filter((call) => call.startsWith("POST"))).toEqual([]);
contextVersion = "generation-b";
await expect(original.publish()).rejects.toThrow("private inbox changed");
canCreate = false;
await expect(prepare()).rejects.toThrow("unavailable");
canCreate = true;
- // Stage the older AWS reader's serialized evidence to reproduce an upgrade
- // after upload. A fresh context below must use the current reader again.
const staged =
- scenario.previousReader && scenario.sourceRepositoryUrl
+ "normalizePrevious" in scenario && scenario.normalizePrevious
? await prepareNormalizedExternalPublication(
JSON.parse(
JSON.stringify(
- scenario.previousReader(raw, scenario.sourceRepositoryUrl),
+ scenario.normalizePrevious(raw, sourceRepositoryUrl ?? ""),
),
),
settings,
@@ -11555,12 +12235,14 @@ test.each([
const saved = JSON.parse(
await readFile(interrupted.result.savedSubmission!, "utf8"),
);
- expect(saved.jobApiVersion).toBe(provider === "continuum" ? 2 : undefined);
- expect(fileHash).toBe(saved.sha256);
- expect(fileRequest).toBe(saved.uploadRequestId);
+ expect(saved).toMatchObject({
+ sha256: fileMetadata().sha256,
+ uploadRequestId: uploadedReports.at(-1)!.id,
+ });
+ expect(saved.jobApiVersion).toBe(upgrade ? undefined : jobVersion);
if ("previousSha256" in scenario) {
- expect(fileHash).toBe(scenario.previousSha256);
- expect(fileHash).not.toBe(original.preview.privateImport?.sha256);
+ expect(saved.sha256).toBe(scenario.previousSha256);
+ expect(saved.sha256).not.toBe(original.preview.privateImport?.sha256);
}
await expect(repositoryPreview.publish()).rejects.toThrow(
"previousSha256" in scenario
@@ -11568,6 +12250,21 @@ test.each([
: "A private import already owns this selection",
);
expect(f.posts).toHaveLength(0);
+ if ("rename" in scenario) {
+ f.state.repositoryUrl = "https://github.com/example/renamed";
+ expect(saved.approvedRepositoryUrl).toBe(exampleRepository);
+ // Earlier checkpoints bound the immutable report and connector ID only.
+ const { approvedRepositoryUrl, ...legacy } = saved;
+ const path = interrupted.result.savedSubmission!;
+ const pending = await prepare();
+ await rm(path);
+ await expect(pending.publish()).rejects.toThrow(
+ "saved private import changed",
+ );
+ await writeFile(path, JSON.stringify(legacy), { mode: 0o600 });
+ expect((await prepare()).preview.resumed).toBe(true);
+ await writeFile(path, JSON.stringify(saved));
+ }
advertised = false;
const resumed = await prepare();
expect(resumed.preview.resumed).toBe(true);
@@ -11575,30 +12272,30 @@ test.each([
await expect(resumed.publish()).rejects.toThrow("lost job response");
expect(jobRequest).toBe(saved.jobRequestId);
expect(calls.at(-1)).toBe(`POST ${expectedJobsPath}`);
- if (provider === "inspector") {
- // The upgraded server accepts the same job request only on v2.
- supportsJobV2 = true;
- expectedJobsPath = v2Jobs;
+ if (upgrade) {
+ // The upgraded server accepts the same approved request on its new route.
+ acknowledgedVersion = requestedVersion;
+ expectedJobsPath = versionedJobs;
loseNextJobResponse = true;
await expect((await prepare()).publish()).rejects.toThrow(
"lost job response",
);
- expect(calls.at(-1)).toBe(`POST ${v2Jobs}`);
+ expect(calls.at(-1)).toBe(`POST ${versionedJobs}`);
expect(jobBodies).toHaveLength(2);
expect(jobBodies[1]).toBe(jobBodies[0]);
const promoted = JSON.parse(
await readFile(interrupted.result.savedSubmission!, "utf8"),
);
expect(promoted).toMatchObject({
- jobApiVersion: 2,
+ jobApiVersion: jobVersion,
jobRequestId: saved.jobRequestId,
uploadRequestId: saved.uploadRequestId,
sha256: saved.sha256,
fileId,
});
}
- // A saved v2 job can still be retrieved if the catalogue acknowledgement disappears.
- supportsJobV2 = false;
+ // Saved jobs retain their route if the catalogue acknowledgement disappears.
+ acknowledgedVersion = undefined;
const completed = await (await prepare()).publish();
expect(completed.privateJob?.counts.created).toBe(1);
expect(calls.at(-1)).toBe(`GET ${expectedJobsPath}/${jobId}`);
@@ -11613,7 +12310,7 @@ test.each([
if (provider === "inspector") {
// A completed old job cannot account for the newly admitted package finding.
advertised = true;
- supportsJobV2 = true;
+ acknowledgedVersion = requestedVersion;
loseResponses = false;
const next = await prepare();
expect(next.preview.resumed).toBe(false);
@@ -11632,7 +12329,7 @@ test.each([
}
canCreate = false;
advertised = false;
- supportsJobV2 = true;
+ acknowledgedVersion = requestedVersion;
expect((await (await prepare()).publish()).privateJob?.id).toBe(jobId);
expect(calls.at(-1)).toBe(`GET ${expectedJobsPath}/${jobId}`);
unavailable = true;
@@ -11651,10 +12348,11 @@ test.each([
// The same context retains its native wire representation even if discovery changes.
formats = ["security.findings.v1"];
- expect((await prepare()).preview.privateImport?.format).toBe(format);
+ expect((await prepare()).preview.privateImport?.format).toBe(
+ expectedFormat,
+ );
contextVersion = "generation-c";
- expectedJobsPath =
- provider === "inspector" || provider === "continuum" ? v2Jobs : v1Jobs;
+ expectedJobsPath = versionedJobs;
unorderedUpdates = 0;
advertised = true;
canCreate = true;
@@ -11662,6 +12360,10 @@ test.each([
expectedFormat = "security.findings.v1";
terminal =
provider === "wiz" || provider === "continuum" ? "cancelled" : "failed";
+ if ("rename" in scenario) {
+ await expect(prepare()).rejects.toThrow("does not match");
+ f.state.repositoryUrl = exampleRepository;
+ }
const fresh = await prepare();
expect(fresh.preview.resumed).toBe(false);
expect(fresh.preview.privateImport?.format).toBe(expectedFormat);
@@ -11670,29 +12372,40 @@ test.each([
original.preview.privateImport?.sha256,
);
expect((await fresh.publish()).privateJob?.status).toBe(terminal);
- expect(fileHash).toBe(fresh.preview.privateImport?.sha256);
+ expect(fresh.preview.privateImport?.sha256).toBe(fileMetadata().sha256);
const failedJob = jobId;
- if (expectedJobsPath === v2Jobs) {
- // Losing AWS write access still allows the failed or cancelled job to be read.
- advertised = false;
- const before = calls.length;
- const readOnly = await prepare();
- expect(readOnly.preview.privateImport?.retryOfJobId).toBeUndefined();
- expect((await readOnly.publish()).privateJob).toMatchObject({
- id: failedJob,
- status: terminal,
- });
- expect(calls.at(-1)).toBe(`GET ${v2Jobs}/${failedJob}`);
- expect(
- calls.slice(before).filter((call) => call.startsWith("POST")),
- ).toEqual([]);
- advertised = true;
- supportsJobV2 = false; // The saved version also applies to its replacement attempt.
+ // The saved receipt remains readable after this provider drops out of discovery.
+ advertised = false;
+ acknowledgedVersion = undefined;
+ const before = calls.length;
+ const readOnly = await prepare();
+ expect(readOnly.preview.privateImport?.retryOfJobId).toBeUndefined();
+ expect((await readOnly.publish()).privateJob).toMatchObject({
+ id: failedJob,
+ status: terminal,
+ });
+ expect(calls.at(-1)).toBe(`GET ${expectedJobsPath}/${failedJob}`);
+ contextVersion = "different-inbox";
+ await expect(prepare()).rejects.toThrow(
+ "Private findings import is unavailable",
+ );
+ expect(
+ calls.slice(before).filter((call) => call.startsWith("POST")),
+ ).toEqual([]);
+ contextVersion = "generation-c";
+ advertised = true;
+ acknowledgedVersion = undefined; // The saved version also applies to its replacement attempt.
+ if ("rename" in scenario) {
+ f.state.repositoryUrl = "https://github.com/example/renamed";
+ await expect(prepare()).rejects.toThrow("does not match");
+ f.state.repositoryUrl = exampleRepository;
}
const retry = await prepare();
const secondPreview = await prepare();
- expect(retry.preview.resumed).toBe(false);
- expect(retry.preview.privateImport?.retryOfJobId).toBe(failedJob);
+ expect(retry.preview).toMatchObject({
+ resumed: false,
+ privateImport: { retryOfJobId: failedJob },
+ });
// A terminal response still acknowledges the report if saving its receipt fails.
terminal = "completed";
@@ -11719,25 +12432,80 @@ test.each([
}
// A second preview of the failed job replays the already-started replacement.
loseResponses = true;
- const uploads = calls.filter(
- (call) => call === "POST /finding-imports/files",
- ).length;
+ const priorUploads = uploadedReports.length;
expect((await secondPreview.publish()).privateJob?.id).not.toBe(failedJob);
- expect(
- calls.filter((call) => call === "POST /finding-imports/files"),
- ).toHaveLength(uploads);
+ expect(uploadedReports).toHaveLength(priorUploads);
},
);
-function privateInboxContext(context_version = "generation-a") {
+function privateJobFields(provider: string, fileId: string, format: string) {
+ return {
+ source: { provider, source_key: options.sourceKey },
+ input: { type: "file", file_id: fileId, format },
+ default_targets: [
+ {
+ type: "repository",
+ repo_id: options.repository,
+ repo_connector_id: "github",
+ },
+ ],
+ };
+}
+
+function privateFileMetadata(id: string, bytes: string) {
+ return {
+ object: "security.finding_import_file",
+ id,
+ purpose: "report",
+ sha256: hash("sha256", bytes),
+ bytes: Buffer.byteLength(bytes),
+ };
+}
+
+function privateJobReceipt(
+ request: object,
+ id: string,
+ status = "completed",
+ total = 1,
+ errors = 0,
+) {
+ const processed = status === "completed" ? total : 0;
+ const error = status === "completed" ? errors : 0;
+ return {
+ ...request,
+ object: "security.finding_import_job",
+ id,
+ status,
+ counts: {
+ total,
+ discovered: total,
+ created: processed - error,
+ updated: 0,
+ unchanged: 0,
+ error,
+ unprocessed: total - processed,
+ },
+ poll_after_seconds: null,
+ error: null,
+ };
+}
+
+function privateInboxContext(
+ context_version = "generation-a",
+ formats = ["security.findings.v1"],
+ canCreate = true,
+) {
return Response.json({
object: "security.finding_import_context",
context_version,
can_read: true,
- can_upload: true,
- can_create_import_jobs: true,
- formats: ["security.findings.v1"],
- limits: { max_direct_file_bytes: 32768, max_record_bytes: 8192 },
+ can_upload: canCreate,
+ can_create_import_jobs: canCreate,
+ formats,
+ limits: {
+ max_direct_file_bytes: 32 * 1024 * 1024,
+ max_record_bytes: 1024 * 1024,
+ },
});
}
@@ -11763,7 +12531,8 @@ test.each([
])(
"only a verified private Wiz checkpoint replays pre-Checkmarx evidence ($data)",
async ({ data, report, key, sha256 }) => {
- const f = await fixture();
+ const { f, deps, inbox, jobs } = await privateUploadFixture([], "wiz");
+ f.state.catalogHeaders = {}; // Recovery must also work after discovery stops advertising Wiz.
const selection = sdkSelection(
[sdkRow(normalized().source_finding_id, data)],
report,
@@ -11785,51 +12554,22 @@ test.each([
jobRequestId: "a942a3d4-ad70-4b4d-aaca-4e1e2b70cb20",
fileId: "aifile_" + "a".repeat(32),
};
- const directory = join(f.root, "state", "external-finding-publications");
- const path = join(directory, key + ".private-import.json");
- await mkdir(directory, { recursive: true, mode: 0o700 });
- await writeFile(path, JSON.stringify(checkpoint), { mode: 0o600 });
- let contextVersion = checkpoint.contextVersion;
- const requests: unknown[] = [];
+ const path = await f.writeCheckpoint(
+ key + ".private-import.json",
+ checkpoint,
+ );
const prepare = () =>
prepareNormalizedExternalPublication(selection, options, {
+ ...deps,
environment: {
...f.environment,
CODEX_SECURITY_CLOUD_BASE_URL: undefined,
CODEX_SECURITY_CLOUD_WEB_URL: undefined,
},
- fetch: async (url, init) => {
- const pathname = new URL(url).pathname;
- if (pathname.endsWith("/repositories"))
- return f.deps.fetch(url, init);
- if (pathname.endsWith("/finding-imports/context"))
- return privateInboxContext(contextVersion);
- expect(pathname).toEndWith("/finding-imports/jobs");
- expect(init.method).toBe("POST");
- const job = JSON.parse(String(init.body));
- requests.push(job);
- return Response.json({
- ...job,
- object: "security.finding_import_job",
- id: "import_" + "b".repeat(32),
- status: "completed",
- counts: {
- total: 1,
- discovered: 1,
- created: 1,
- updated: 0,
- unchanged: 0,
- error: 0,
- unprocessed: 0,
- },
- poll_after_seconds: null,
- error: null,
- });
- },
});
- contextVersion = "different-inbox";
+ inbox.version = "different-inbox";
await expect(prepare()).rejects.toThrow("requires the checkmarx provider");
- contextVersion = checkpoint.contextVersion;
+ inbox.version = checkpoint.contextVersion;
f.state.repositoryUrl = otherRepository;
await expect(prepare()).rejects.toThrow(/belongs to|does not match/);
f.state.repositoryUrl = exampleRepository;
@@ -11837,23 +12577,11 @@ test.each([
expect(resumed.preview.resumed).toBe(true);
expect(resumed.preview.privateImport?.sha256).toBe(checkpoint.sha256);
expect((await resumed.publish()).counts.created).toBe(1);
- expect(requests).toEqual([
+ expect(jobs).toEqual([
{
request_id: checkpoint.jobRequestId,
- context_version: checkpoint.contextVersion,
- source,
- input: {
- type: "file",
- file_id: checkpoint.fileId,
- format: checkpoint.format,
- },
- default_targets: [
- {
- type: "repository",
- repo_id: repository.id,
- repo_connector_id: repository.repo_connector_id,
- },
- ],
+ context_version: inbox.version,
+ ...privateJobFields("wiz", checkpoint.fileId, checkpoint.format),
},
]);
await rm(path);
@@ -11861,30 +12589,18 @@ test.each([
"requires the checkmarx provider",
);
await expect(prepare()).rejects.toThrow("requires the checkmarx provider");
- expect(requests).toHaveLength(1);
+ expect(jobs).toHaveLength(1);
},
);
test("a private preview recognizes a later repository checkpoint under its historical filename", async () => {
- const f = await fixture();
+ const { f, deps, uploads, jobs } = await privateUploadFixture([], "wiz");
const privatePreview = await prepareNormalizedExternalPublication(
scoredFindings(2.5),
options,
- {
- ...f.deps,
- fetch: async (url, init) => {
- expect(init.method).toBe("GET");
- if (new URL(url).pathname.endsWith("/repositories")) {
- const response = await f.deps.fetch(url, init);
- return new Response(response.body, {
- headers: { "Codex-Security-Private-Import-Providers": "wiz" },
- });
- }
- expect(new URL(url).pathname).toEndWith("/finding-imports/context");
- return privateInboxContext();
- },
- },
+ deps,
);
+ f.state.catalogHeaders = {};
const repository = await prepareNormalizedExternalPublication(
scoredFindings(new Number(2.5)),
options,
@@ -11895,6 +12611,8 @@ test("a private preview recognizes a later repository checkpoint under its histo
await expect(privatePreview.publish()).rejects.toThrow(
"A repository submission is pending",
);
+ expect(uploads).toHaveLength(0);
+ expect(jobs).toHaveLength(0);
expect(f.posts).toHaveLength(1);
});
diff --git a/sdk/typescript/tests-ts/github-findings-local.test.ts b/sdk/typescript/tests-ts/github-findings-local.test.ts
index 9c58e622e..33d66f3bb 100644
--- a/sdk/typescript/tests-ts/github-findings-local.test.ts
+++ b/sdk/typescript/tests-ts/github-findings-local.test.ts
@@ -1,4 +1,8 @@
import { expect, test } from "bun:test";
+import {
+ normalizeGitHubAdvisories,
+ normalizeGitHubCodeScanningAlerts,
+} from "../src/github-findings.js";
import {
createAndReplayFinding,
localReportFixture,
@@ -65,19 +69,82 @@ test.each(["code-scanning", "repository-advisory", "global-advisory"] as const)(
);
test.each(["code-scanning", "repository-advisory"] as const)(
- "GitHub %s refuses to attach another repository's evidence locally",
+ "GitHub %s checks repository scope for raw and previously normalized findings",
async (kind) => {
+ const wrongRepository = "https://github.com/example/other";
await using fixture = await localReportFixture(
"github.json",
{ provider: "github", sourceKey: "github/example/project" },
- { remote: "https://github.com/example/other" },
+ { remote: wrongRepository },
);
- await fixture.writeJson([githubSource(kind)]);
+ const source = githubSource(kind);
+ await fixture.writeJson([source]);
await expect(fixture.prepare()).rejects.toThrow(/repository/iu);
expect(await fixture.read()).toEqual([]);
+
+ const input =
+ source.source === "github-code-scanning"
+ ? normalizeGitHubCodeScanningAlerts([source], "local")
+ : normalizeGitHubAdvisories([source], "local");
+ const row = input.findings[0]!;
+ const details = row.evidence.details!;
+ details.repository!.url = wrongRepository;
+ await expect(fixture.prepareNormalized(input)).rejects.toThrow(
+ "does not match the original GitHub finding",
+ );
+ delete row.evidence.details;
+ await expect(fixture.prepareNormalized(input)).rejects.toThrow(
+ /configured remote/iu,
+ );
+ fixture.git(
+ "remote",
+ "set-url",
+ "origin",
+ "[email protected]:Example/Project.git",
+ );
+ expect(
+ (await fixture.prepareNormalized(input)).preview.findings,
+ ).toHaveLength(1);
+ row.evidence.details = details;
+ details.repository!.url = "https://github.com:8443/example/project";
+ await expect(fixture.prepareNormalized(input)).rejects.toThrow(
+ "does not match the original GitHub finding",
+ );
+ details.repository!.url = "https://github.com:443/example/project";
+ row.source_finding_id = "existing-sdk-occurrence";
+ const saved = await (await fixture.prepareNormalized(input)).import();
+ expect(saved.counts.created).toBe(1);
+ expect((await fixture.read())[0]?.sourceFindingId).toBe(
+ row.source_finding_id,
+ );
},
);
+test("a retained global advisory may be investigated in another repository", async () => {
+ const remote = "https://github.com/example/other";
+ await using fixture = await localReportFixture(
+ "github.json",
+ { provider: "github", sourceKey: "github/global-advisories" },
+ { remote },
+ );
+ const input = normalizeGitHubAdvisories(
+ [importedAdvisory("global")],
+ "local",
+ );
+ const evidence = input.findings[0]!.evidence;
+ expect(
+ (await fixture.prepareNormalized(input)).preview.findings,
+ ).toHaveLength(1);
+ evidence.details = {
+ kind: "advisory",
+ advisory: { source: "repository" },
+ repository: { url: remote },
+ };
+ await expect(fixture.prepareNormalized(input)).rejects.toThrow(
+ "Global advisory evidence cannot claim a source repository",
+ );
+});
+
test.each(["code-scanning", "repository-advisory"] as const)(
"GitHub %s matches custom Enterprise repository names without changing source evidence",
async (kind) => {
diff --git a/sdk/typescript/tests-ts/semgrep-findings.test.ts b/sdk/typescript/tests-ts/semgrep-findings.test.ts
index 0fa0c80b6..9df939973 100644
--- a/sdk/typescript/tests-ts/semgrep-findings.test.ts
+++ b/sdk/typescript/tests-ts/semgrep-findings.test.ts
@@ -452,6 +452,38 @@ test("native SARIF keeps CE percent paths distinct after reordering", () => {
}
});
+test("native JSON and SARIF retain the scanner's path spelling on both platforms", () => {
+ const json = semgrepJson();
+ const sarif = semgrepSarif();
+ const paths = [
+ "src/example.ts",
+ "src\\example.ts",
+ ".\\C:\\outside.ts",
+ ] as const;
+ json.results = paths.map((path, index) => {
+ const result = structuredClone(json.results[0]!);
+ result.path = path;
+ result.extra.fingerprint += index;
+ return result;
+ });
+ const run = sarif.runs[0]!;
+ run.results = paths.map((path, index) => {
+ const result = structuredClone(run.results[0]!);
+ result.locations[0]!.physicalLocation.artifactLocation.uri = path;
+ result.fingerprints["matchBasedId/v1"] += index;
+ return result;
+ });
+ for (const input of [json, sarif]) {
+ const output = normalizeSemgrepFindings(input);
+ expect(output.excluded).toEqual([]);
+ expect(output.findings.map((row) => row.evidence.locations)).toEqual([
+ [{ path: paths[0], line: 3 }],
+ [{ path: paths[1], line: 3 }],
+ [],
+ ]);
+ }
+});
+
test.each([
"%2e%2e/outside.ts",
"%2Foutside.ts",
@@ -853,6 +885,25 @@ test("large scan inventories and timing tables do not exclude a small finding",
expect(input.paths.scanned).toHaveLength(10000);
});
+test("native JSON keeps other files' matching explanations outside each finding", () => {
+ const report = semgrepJson();
+ const expected = normalizeSemgrepFindings(report);
+ const explanation = {
+ op: "And",
+ matches: [{ ...report.results[0], path: "src/other.ts" }],
+ children: [],
+ };
+ // Either run-wide inventory would include another file's match. The larger
+ // one would also exceed the per-finding Cloud budget.
+ for (const explanations of [[explanation], Array(1000).fill(explanation)]) {
+ const input = { ...report, explanations };
+ for (const target of ["cloud", "local"] as const) {
+ expect(normalizeJson(input, options, target)).toEqual(expected);
+ }
+ expect(input.explanations).toBe(explanations);
+ }
+});
+
test.each([undefined, "sast"])(
"accepts the documented Code NO_VALIDATOR marker with product %p",
(product) => {
diff --git a/sdk/typescript/tests-ts/semgrep-guardian-findings.test.ts b/sdk/typescript/tests-ts/semgrep-guardian-findings.test.ts
index 69700f3e4..3e574d32f 100644
--- a/sdk/typescript/tests-ts/semgrep-guardian-findings.test.ts
+++ b/sdk/typescript/tests-ts/semgrep-guardian-findings.test.ts
@@ -242,6 +242,12 @@ test("normalized Guardian imports verify the original finding against the select
);
const normalized = normalizeSemgrepFindings(capture, {}, "local");
const original = normalized.findings[0]!;
+ await expect(
+ fixture.prepareNormalized({
+ ...normalized,
+ findings: [{ ...original, source_finding_id: "another-source-id" }],
+ }),
+ ).rejects.toThrow(/original Guardian finding/);
for (const changed of [
{ ...row, repository: { name: "other/service" } },
{ ...row, ref: "refs/heads/other" },
diff --git a/sdk/typescript/tests-ts/semgrep-platform-selection.test.ts b/sdk/typescript/tests-ts/semgrep-platform-selection.test.ts
index a04acc63a..94e1aaa11 100644
--- a/sdk/typescript/tests-ts/semgrep-platform-selection.test.ts
+++ b/sdk/typescript/tests-ts/semgrep-platform-selection.test.ts
@@ -112,6 +112,30 @@ test("reimporting two selected IDs from a non-default branch leaves both SQLite
);
});
+test("a fully excluded SDK selection still verifies the checkout against the selected project", async () => {
+ const selected = normalizeSelectedSemgrepPlatformFindings({
+ ...source,
+ findingIds: ["8"],
+ findingsResponses: [
+ JSON.stringify({ findings: [{ id: 8, ref, repository: { url } }] }),
+ ],
+ });
+ expect(selected.findings).toHaveLength(0);
+ expect(selected.excluded).toHaveLength(1);
+ await using fixture = await localFindingsFixture({ remote: url + "-other" });
+ await expect(
+ prepareNormalizedLocalImport(
+ selected,
+ {
+ provider: "semgrep",
+ sourceKey: "demo/topic",
+ repository: fixture.repository,
+ },
+ { environment: fixture.environment },
+ ),
+ ).rejects.toThrow("configured remote");
+});
+
test("each selected finding must match the independently named project, ref, and SAST product", () => {
const first = page(ids[0]!);
const second = page(ids[1]!);
diff --git a/sdk/typescript/tests-ts/snyk-findings-local.test.ts b/sdk/typescript/tests-ts/snyk-findings-local.test.ts
index 3425a8317..12bd4e095 100644
--- a/sdk/typescript/tests-ts/snyk-findings-local.test.ts
+++ b/sdk/typescript/tests-ts/snyk-findings-local.test.ts
@@ -107,6 +107,25 @@ test.each(["plugin", "JSON", "SARIF"] as const)(
},
);
+test("Snyk checks clean projects alongside vulnerable projects against the local checkout", async () => {
+ const remote = "https://github.com/example/project";
+ await using fixture = await localReportFixture(
+ "snyk.json",
+ { provider: "snyk", sourceKey: "snyk-sca" },
+ { remote },
+ );
+ const project = { ...snykJson(), path: remote };
+ const clean = { ...project, ok: true, vulnerabilities: [] };
+ await fixture.writeJson([
+ project,
+ { ...clean, path: "https://github.com/example/other" },
+ ]);
+ await expect(fixture.prepare()).rejects.toThrow(/does not match.*remote/iu);
+ expect(await fixture.read()).toEqual([]);
+ await fixture.writeJson([project, clean]);
+ expect((await (await fixture.prepare()).import()).counts.created).toBe(1);
+});
+
test.each([2222, 2223])(
"Snyk plugin repository matching preserves the SSH port (checkout port %i)",
async (port) => {
diff --git a/sdk/typescript/tests-ts/support/trivy-import-fixtures.ts b/sdk/typescript/tests-ts/support/trivy-import-fixtures.ts
index 9e4f016db..27555d361 100644
--- a/sdk/typescript/tests-ts/support/trivy-import-fixtures.ts
+++ b/sdk/typescript/tests-ts/support/trivy-import-fixtures.ts
@@ -67,3 +67,15 @@ export function trivyReport() {
],
};
}
+
+export function trivyReportWithManyLocations() {
+ const report = trivyReport();
+ const repeated = structuredClone(report.Results[0]!);
+ repeated.Target = "workspace/package-lock.json";
+ repeated.Packages[0]!.Locations = Array.from({ length: 101 }, (_, i) => ({
+ StartLine: i * 5 + 1,
+ EndLine: i * 5 + 4,
+ }));
+ report.Results.push(repeated);
+ return report;
+}
diff --git a/sdk/typescript/tests-ts/support/veracode-import-fixtures.ts b/sdk/typescript/tests-ts/support/veracode-import-fixtures.ts
index 37ca04631..a935108b3 100644
--- a/sdk/typescript/tests-ts/support/veracode-import-fixtures.ts
+++ b/sdk/typescript/tests-ts/support/veracode-import-fixtures.ts
@@ -47,3 +47,20 @@ export function veracodePolicyPage(
page: { size, number, total_elements: total, total_pages: pages },
};
}
+
+export function veracodeMixedPolicyPage() {
+ const first = veracodePolicyRow(1);
+ const second = veracodePolicyRow(2);
+ const scalar = {
+ ...second,
+ finding_details: {
+ ...second.finding_details,
+ cwe: second.finding_details.cwe.id,
+ finding_category: second.finding_details.finding_category.name,
+ },
+ };
+ return {
+ ...veracodePolicyPage([first, second]),
+ _embedded: { findings: [first, scalar] },
+ };
+}
diff --git a/sdk/typescript/tests-ts/trivy-findings.test.ts b/sdk/typescript/tests-ts/trivy-findings.test.ts
index 726ad6d14..1d9e3d6d6 100644
--- a/sdk/typescript/tests-ts/trivy-findings.test.ts
+++ b/sdk/typescript/tests-ts/trivy-findings.test.ts
@@ -4,11 +4,16 @@ import { tmpdir } from "node:os";
import { join } from "node:path";
import { gzipSync } from "node:zlib";
import {
+ needsPreviousTrivyProjection,
+ normalizePreviousTrivyFindings,
normalizeTrivyFindings,
readTrivyFindings,
trivyRepositoryUrl,
} from "../src/trivy-findings.js";
-import { trivyReport } from "./support/trivy-import-fixtures.js";
+import {
+ trivyReport,
+ trivyReportWithManyLocations,
+} from "./support/trivy-import-fixtures.js";
function normalizeAtRoot(input: unknown) {
return normalizeTrivyFindings(input, "cloud", ".");
@@ -572,6 +577,32 @@ test("declaration ranges come from an unambiguous package at the reported path",
expect(run().source_data?.["package"]).toBeUndefined();
});
+test("Cloud keeps a repeated lockfile dependency with all native declaration ranges", () => {
+ const report = trivyReportWithManyLocations();
+ const ranges = report.Results[1]!.Packages[0]!.Locations;
+ const current = normalizeAtRoot(report);
+ const local = normalizeTrivyFindings(report, "local", ".");
+ const previous = normalizePreviousTrivyFindings(report, "cloud", ".");
+ const [ordinary, repeated] = current.findings;
+ expect(current.excluded).toEqual([]);
+ expect(previous.findings).toEqual([ordinary!]);
+ expect(previous.excluded).toMatchObject([
+ { position: 2, source_finding_id: repeated!.source_finding_id },
+ ]);
+ expect(repeated!.source_finding_id).toBe(
+ local.findings[1]!.source_finding_id,
+ );
+ expect(local.findings[1]!.evidence.locations).toHaveLength(101);
+ expect(repeated!.evidence.locations).toEqual(
+ local.findings[1]!.evidence.locations!.slice(0, 100),
+ );
+ expect(repeated!.evidence.source_data?.["package"]).toMatchObject({
+ Locations: ranges,
+ });
+ expect(needsPreviousTrivyProjection(current)).toBe(true);
+ expect(needsPreviousTrivyProjection(previous)).toBe(false);
+});
+
test("only ID-less detections use a unique name/version match from inventory", () => {
const result = trivyReport().Results[0]!;
const pkg = result.Packages[0]!;
diff --git a/sdk/typescript/tests-ts/veracode-findings.test.ts b/sdk/typescript/tests-ts/veracode-findings.test.ts
index 3b011f2d7..53bf84148 100644
--- a/sdk/typescript/tests-ts/veracode-findings.test.ts
+++ b/sdk/typescript/tests-ts/veracode-findings.test.ts
@@ -1,14 +1,17 @@
+import { hasPreviousCloudProjection } from "../src/reader-selection.js";
import { expect, test } from "bun:test";
import { mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { gzipSync } from "node:zlib";
import {
+ normalizePreviousVeracodeFindings,
normalizeVeracodeFindings,
readVeracodeFindings,
veracodeReportRepositories,
} from "../src/veracode-findings.js";
import {
+ veracodeMixedPolicyPage,
veracodePolicyPage,
veracodePolicyRow,
} from "./support/veracode-import-fixtures.js";
@@ -68,6 +71,40 @@ test("keeps only finding-level evidence, not a generated checkout location or a
expect(evidence.source_data?.["row"]).toEqual(original);
});
+test("accepts schema and expanded CWE and category fields under one policy flaw identity", () => {
+ const page = veracodePolicyPage([veracodePolicyRow(1), veracodePolicyRow(2)]);
+ const mixed = veracodeMixedPolicyPage();
+ const scalar = mixed._embedded.findings[1]!;
+ for (const target of ["local", "cloud"] as const) {
+ const expanded = normalize(page, target);
+ const result = normalize(mixed, target);
+ expect(hasPreviousCloudProjection(expanded)).toBe(false);
+ expect(hasPreviousCloudProjection(result)).toBe(target === "cloud");
+ expect(result.excluded).toEqual([]);
+ // Only the native row spelling should distinguish the two projections.
+ expanded.findings[1]!.evidence.source_data!["row"] = scalar;
+ expect(result.findings).toEqual(expanded.findings);
+ const previous = normalizePreviousVeracodeFindings(
+ mixed,
+ repository,
+ target,
+ );
+ expect(previous.findings).toEqual([expanded.findings[0]!]);
+ expect(previous.excluded).toMatchObject([
+ {
+ position: 2,
+ source_finding_id: expanded.findings[1]!.source_finding_id,
+ },
+ ]);
+ }
+ mixed._links.self.href += "&cwe=79";
+ const filtered = normalize(mixed);
+ expect(filtered.findings).toHaveLength(2);
+ expect(hasPreviousCloudProjection(filtered)).toBe(false);
+ mixed._links.self.href += "&finding_category=4";
+ expect(() => normalize(mixed)).toThrow(/finding_category ID/);
+});
+
test("canonical severity covers the six native levels and malformed levels are not guessed", () => {
const rows = [0, 1, 2, 3, 4, 5, 6, -1, 1.5].map((severity, index) => {
const row = veracodePolicyRow(index + 1);
diff --git a/sdk/typescript/tests-ts/wiz-findings-local.test.ts b/sdk/typescript/tests-ts/wiz-findings-local.test.ts
index 29e5ca1d9..8833da589 100644
--- a/sdk/typescript/tests-ts/wiz-findings-local.test.ts
+++ b/sdk/typescript/tests-ts/wiz-findings-local.test.ts
@@ -36,6 +36,35 @@ function csvReport(source: Record<string, string>): string {
.join("\r\n");
}
+test("Wiz package exports can identify GitHub Enterprise branches by cloudPlatform alone", async () => {
+ await using fixture = await localReportFixture(
+ "wiz.json",
+ { provider: "wiz", sourceKey: "example-project" },
+ { remote: "[email protected]:example/project.git" },
+ );
+ const asset = {
+ id: "branch",
+ type: "REPOSITORY_BRANCH",
+ cloudPlatform: "GitHub",
+ cloudProviderURL:
+ "https://git.example.test/Example/Project/tree/feature/ref",
+ };
+ const finding = {
+ id: "package-occurrence",
+ name: "CVE-2099-0001",
+ detailedName: "example-package",
+ severity: "HIGH",
+ vulnerableAsset: asset,
+ };
+ await writeFile(fixture.file, JSON.stringify(finding));
+ const imported = await (await fixture.prepare()).import();
+ expect(imported.counts.created).toBe(1);
+ expect(imported.findings[0]!.evidence.source_data).toEqual(finding);
+ asset.cloudProviderURL = "https://git.example.test/example/other/tree/main";
+ await writeFile(fixture.file, JSON.stringify(finding));
+ await expect(fixture.prepare()).rejects.toThrow("does not match");
+});
+
test.each(["CSV", "CSV gzip", "normalized JSONL", "normalized JSONL gzip"])(
"Wiz %s matches GitHub Enterprise repository names without changing evidence",
async (format) => {