OpenAI's Codex Security CLI and TypeScript SDK for finding, validating, and fixing security vulnerabilities. npm: https://www.npmjs.com/package/@openai/codex-security
TypeScript ai-securityapplication-securityclicode-scanningcodexcodex-securitycybersecuritydevsecopsnodejsnpmopenaisdksecuritytypescriptvulnerability-scanning
feat(imports): normalize selected Semgrep Code findings for local import (#2164)
Michael D'Angelo80821889d0b5
diff --git a/docs/scanner-imports/README.md b/docs/scanner-imports/README.mdindex 4a1369bf4..d9c554af5 100644--- a/docs/scanner-imports/README.md+++ b/docs/scanner-imports/README.md@@ -13,7 +13,7 @@ guide shows how to preview the findings before you save or publish them.| [GitHub Dependabot](github-dependabot.md) | Dependency alerts exported from one repository || [GitHub Security Advisories](../../sdk/typescript/docs/cli.md#import-ghsa-advisories-separately) | Selected repository or global advisories || [OpenGrep](opengrep.md) | Native scan JSON, optionally gzip-compressed |-| [Semgrep Code](semgrep.md) | CLI or CI JSON, or a self-contained SARIF report |+| [Semgrep Code](semgrep.md) | CLI/CI JSON, self-contained SARIF, or selected Platform Web API responses (SDK) || [Snyk Open Source](snyk.md) | CLI JSON, a Codex plugin export, or SARIF || [SonarQube Cloud](sonar.md) | Selected security issues from the signed-in Sonar CLI or saved issue-search pages || [Veracode Static Analysis](veracode.md) | Findings API pages for one application's policy flaws |diff --git a/docs/scanner-imports/schema-guide.md b/docs/scanner-imports/schema-guide.mdindex a2dcf1f03..6378ea25a 100644--- a/docs/scanner-imports/schema-guide.md+++ b/docs/scanner-imports/schema-guide.md@@ -24,7 +24,8 @@ changes.| [GitHub advisories](../../sdk/typescript/docs/cli.md#import-ghsa-advisories-separately) | Host, repository or global scope, and GHSA ID. This identifies a disclosure. Check installed packages separately. || [Snyk Open Source](snyk.md) | Native CLI JSON distinguishes manifest, package manager, advisory, package name and version, and dependency path. Plugin and SARIF reports use different identities. || [Black Duck SCA](blackduck.md), [Veracode Static Analysis](veracode.md) | Black Duck combines project version, component version, origin and advisory identity. Veracode scopes a native issue ID to its API origin, application and context. |-| [Semgrep Code](semgrep.md), [OpenGrep](opengrep.md) | Semgrep prefers a fingerprint and falls back to rule, path and range. OpenGrep groups traces by rule, path and range. Both store provider `semgrep`. Use a separate OpenGrep source key. |+| [Semgrep CLI or CI reports](semgrep.md), [OpenGrep](opengrep.md) | Semgrep prefers a fingerprint and falls back to rule, path and range. OpenGrep groups traces by rule, path and range. Both store provider `semgrep`. Use a separate OpenGrep source key. |+| [Semgrep Platform SAST](semgrep.md#import-selected-semgrep-platform-sast-findings-with-the-sdk) | Deployment, project ID, SAST product, full Git ref and native finding ID. The same occurrence keeps its identity when its file moves. |Repeat local imports replace changed evidence for the same identity. Identicalevidence is unchanged, and records omitted from a later report remain stored.diff --git a/docs/scanner-imports/semgrep.md b/docs/scanner-imports/semgrep.mdindex 27752cea1..a9b82f73b 100644--- a/docs/scanner-imports/semgrep.md+++ b/docs/scanner-imports/semgrep.md@@ -3,6 +3,8 @@Import a saved Semgrep Code CLI or CI report into the local workbench or Cloud.Native JSON preserves original severities and snippets when available. The readeralso accepts self-contained Semgrep SARIF 2.1.0. Either format can be gzipped.+To import selected existing findings from the Platform API, use the+[SDK example](#import-selected-semgrep-platform-sast-findings-with-the-sdk).## Quick start@@ -100,10 +102,53 @@ directly, pass the original repository references in its third argument's`sourceRepositories`. Use `{ url, platform: "github" }` for a verified customGitHub host.-### Existing Platform findings+### Import selected Semgrep Platform SAST findings with the SDK++Request `issue_type=sast` and keep the project lookup and findings pages as original+[Semgrep Web API](https://docs.semgrep.dev/semgrep-appsec-platform/semgrep-api) response text. Pass each page separately. When a page contributes a selected ID, the+SDK checks its rows for conflicting product evidence. If a page contains only fields+shared by several products, its product depends on the original request filter.+[Guardian for Codex](https://docs.semgrep.dev/semgrep-guardian/ide-setup/codex) and+[Semgrep MCP](https://github.com/semgrep/semgrep/blob/fe209763a4a2ccfcdf6434e6c332edc24a2f586e/cli/src/semgrep/mcp/server.py) return different records.+Pass a fully qualified ref, such as `refs/heads/topic` or `refs/tags/topic`, exactly as it appears in the findings.+Supply the HTTP(S) repository URL you independently associate with the target checkout.+The native `project.url` and any selected `finding.repository.url` must match this spelling. Legacy findings can+omit the project ID. An exact URL then identifies the repository even if the+display name has changed. If multiple Semgrep projects share that URL, select+the IDs from the intended project: the URL alone can also match a sibling project+on the same ref. A conflicting project ID is always rejected. A finding without a+URL must carry the matching project ID and a name equal to the namespace-qualified,+decoded path in the operator-supplied URL.+The local importer checks the checkout using its existing repository rules.++```ts+const selected = normalizeSelectedSemgrepPlatformFindings({+ deployment,+ ref,+ sourceRepositoryUrl,+ projectResponse,+ findingsResponses,+ findingIds,+});+const prepared = await prepareNormalizedLocalImport(+ selected,+ { provider: "semgrep", sourceKey, repository: checkout },+ { sourceRepositories: [{ url: sourceRepositoryUrl }] },+);+if (prepared.preview.excluded.length) {+ console.error(prepared.preview.excluded);+} else {+ await prepared.import();+}+```++Keep the same source key on replay. Finding identity includes the deployment, project ID,+SAST product, full ref, and native finding ID. Pass selected finding IDs as positive+decimal strings without leading zeros. Semgrep emits 64-bit finding IDs. The adapter+also accepts larger positive integer IDs in saved responses without rounding. The+stored evidence includes the selected finding and the project's ID, name and URL.+A record without a native rule name or message is excluded during preview.-This file reader accepts scan reports. Semgrep Guardian, MCP and the Platform-findings API return different records. For this workflow, use a Code scan-artifact. [Guardian for Codex](https://docs.semgrep.dev/semgrep-guardian/ide-setup/codex),-[Semgrep MCP implementation](https://github.com/semgrep/semgrep/blob/fe209763a4a2ccfcdf6434e6c332edc24a2f586e/cli/src/semgrep/mcp/server.py)-and [Platform findings API](https://docs.semgrep.dev/semgrep-appsec-platform/semgrep-api)+Changes to key order or equivalent numeric spellings such as `2` and `2.0` leave+the stored record unchanged. Only repository-relative files become locations. Names+with URI-like prefixes and directory-only paths remain in the native evidence.diff --git a/sdk/typescript/README.md b/sdk/typescript/README.mdindex 57d6c79a7..0261db320 100644--- a/sdk/typescript/README.md+++ b/sdk/typescript/README.md@@ -674,6 +674,8 @@ Omit `issueKeys` to collect the project's open security issues. An empty list isIf you know the source repository, pass its Git URL in `options.sourceRepositoryUrl`to check the Sonar binding before fetching issues.Pure normalizers and file readers remain available.+Use `normalizeSelectedSemgrepPlatformFindings` for selected SAST findings from saved+Semgrep Web API responses. See the [Semgrep SDK example](https://github.com/openai/codex-security/blob/main/docs/scanner-imports/semgrep.md#import-selected-semgrep-platform-sast-findings-with-the-sdk).This does not affect scan publishing or the existing GitHub alert collector.The SDK also exposes `prepareExternalPublication(file, options)` for supporteddiff --git a/sdk/typescript/scripts/package-dist-files.mjs b/sdk/typescript/scripts/package-dist-files.mjsindex 70f5b8f65..57c085db5 100644--- a/sdk/typescript/scripts/package-dist-files.mjs+++ b/sdk/typescript/scripts/package-dist-files.mjs@@ -32,6 +32,7 @@ const distModules = ["snyk-findings","scanner-repository-url","semgrep-findings",+ "semgrep-platform-findings","sonar-findings","sonar-collector","sonar-import",diff --git a/sdk/typescript/src/index.ts b/sdk/typescript/src/index.tsindex cc810740a..2a63c1572 100644--- a/sdk/typescript/src/index.ts+++ b/sdk/typescript/src/index.ts@@ -354,6 +354,7 @@ export {normalizeSnykSarif,readSnykFindings,} from "./snyk-findings.js";+export { normalizeSelectedSemgrepPlatformFindings } from "./semgrep-platform-findings.js";export {normalizeSemgrepFindings,normalizeSemgrepSarif,diff --git a/sdk/typescript/src/inspector-findings.ts b/sdk/typescript/src/inspector-findings.tsindex 476db7fad..b46a31c5a 100644--- a/sdk/typescript/src/inspector-findings.ts+++ b/sdk/typescript/src/inspector-findings.ts@@ -20,7 +20,7 @@ import {normalizeRecords,readScannerJson,record,- relativePath,+ repositoryFilePath,requiredText,text,} from "./scanner-import.js";@@ -444,17 +444,6 @@ export function inspectorReportRepository(};}-function repoFile(value: unknown): string | null {- const relative = relativePath(value);- if (!relative) return null;- const path = posix.normalize(relative);- return path === "." ||- path.endsWith("/") ||- /^(?:[a-z][a-z\d+.-]*:|~[^/]*(?:\/|$)|[^/]+@[^/]+:)/iu.test(path)- ? null- : path;-}-function timestamp(value: unknown): number {const seconds =typeof value === "number"@@ -587,7 +576,7 @@ function evidence(const level = requiredText(severity, "Inspector severity").toLowerCase();const paths = new Set<string>();const packages = (pkg?.vulnerablePackages ?? []).map((item) => {- const path = repoFile(item.filePath);+ const path = repositoryFilePath(item.filePath);if (path) paths.add(path);const ecosystem = ecosystems.get(text(item.packageManager) ?? "") ?? null;const version = (value: unknown) =>@@ -609,7 +598,7 @@ function evidence();if (code) {const { fileName, filePath, startLine } = code.filePath;- const path = repoFile(filePath);+ const path = repositoryFilePath(filePath);if (path && posix.basename(path) === fileName)locations.push({path,diff --git a/sdk/typescript/src/scanner-import.ts b/sdk/typescript/src/scanner-import.tsindex ce214decc..d9f92f425 100644--- a/sdk/typescript/src/scanner-import.ts+++ b/sdk/typescript/src/scanner-import.ts@@ -1,5 +1,6 @@import { hash } from "node:crypto";import { readFile, type FileHandle } from "node:fs/promises";+import { posix } from "node:path";import { promisify } from "node:util";import { gunzip } from "node:zlib";import { CodexSecurityError, errorMessage } from "./errors.js";@@ -45,6 +46,18 @@ export function relativePath(value: unknown): string | null {: null;}+export function repositoryFilePath(value: unknown): string | null {+ const relative = relativePath(value);+ if (!relative) return null;+ const path = posix.normalize(relative);+ // Anchor the SCP branch at the first eligible @ to avoid rescanning long filenames.+ return path === "." ||+ path.endsWith("/") ||+ /^(?:[a-z][a-z\d+.-]*:|~[^/]*(?:\/|$)|[^/][^/@]*@[^/]+:)/iu.test(path)+ ? null+ : path;+}+export function stableId(prefix: string, fields: unknown[]): string {return `${prefix}:${hash("sha256", JSON.stringify(fields))}`;}diff --git a/sdk/typescript/src/semgrep-platform-findings.ts b/sdk/typescript/src/semgrep-platform-findings.tsnew file mode 100644index 000000000..bff161c84--- /dev/null+++ b/sdk/typescript/src/semgrep-platform-findings.ts@@ -0,0 +1,322 @@+import { validateExternalEvidence } from "./external-import-contract.js";+import type { ExternalFindings } from "./external-findings.js";+import { isRecord } from "./record.js";+import { position, weaknessIds } from "./native-json-findings.js";+import {+ array,+ normalizeRecords,+ record,+ relativePath,+ repositoryFilePath,+ requiredText,+ stableId,+ text,+} from "./scanner-import.js";+import { declaredScannerRepositoryUrl } from "./scanner-repository-url.js";++// These v1 Findings API response fields belong only to AI detections or Supply Chain.+const nonSastFields = [+ "ai_impact",+ "epss_score",+ "fix_recommendations",+ "found_dependency",+ "is_malicious",+ "reachability",+ "reachable_condition",+ "usage",+ "vulnerability_identifier",+];++function identifiesNonSastProduct(value: unknown): boolean {+ if (!isRecord(value)) return false;+ return (+ nonSastFields.some((field) => value[field] != null) ||+ [value["issue_type"], value["product"]].some(+ (product) =>+ typeof product === "string" &&+ /^(?:sca|ai_sast|secrets)$/iu.test(product),+ )+ );+}++// Node 22 supports JSON source text and raw values. The SDK targets the ES2022 type library.+const wireJson = JSON as JSON & {+ rawJSON(source: string): { rawJSON: string };+ isRawJSON(value: unknown): value is { rawJSON: string };+};++// A retained JSON number is represented as an object in JavaScript.+function responseRecord(value: unknown, label: string) {+ return record(wireJson.isRawJSON(value) ? undefined : value, label);+}++// Normalize equivalent JSON numbers exactly. Keep very large exponents compact.+function canonicalNumber(token: string): string {+ const [, sign = "", integer, fraction = "", exponent = "0"] =+ /^(-?)(\d+)(?:\.(\d+))?(?:e([+-]?\d+))?$/iu.exec(token)!;+ const significant = (integer + fraction).replace(/^0+/u, "");+ if (!significant) return "0";+ let end = significant.length;+ while (significant[end - 1] === "0") end--;+ const digits = significant.slice(0, end);+ const point = BigInt(exponent) + BigInt(significant.length - fraction.length);+ // Ordinary evidence follows JSON.stringify’s fixed/scientific notation thresholds.+ if (point > -6n && point <= 21n) {+ const split = Number(point);+ if (split <= 0) return sign + "0." + "0".repeat(-split) + digits;+ if (split >= digits.length) return sign + digits.padEnd(split, "0");+ return sign + digits.slice(0, split) + "." + digits.slice(split);+ }+ const trailing = digits.slice(1);+ const scientific = point - 1n;+ return (+ sign ++ digits.charAt(0) ++ (trailing ? "." + trailing : "") ++ "e" ++ (scientific >= 0n ? "+" : "") ++ scientific+ );+}++function sourceLine(value: unknown): number | null {+ if (value == null) return null;+ if (!wireJson.isRawJSON(value)) return position(value, "Semgrep source line");+ const exact = canonicalNumber(value.rawJSON);+ return position(+ /^\d+$/u.test(exact) ? Number(exact) : undefined,+ "Semgrep source line",+ );+}++/** Keep original number tokens, including adjacent int64 IDs above Number.MAX_SAFE_INTEGER. */+function parseSemgrepResponse(json: string): Record<string, unknown> {+ return responseRecord(+ JSON.parse(+ json.replace(/^\uFEFF/u, ""),+ (_key: string, value: unknown, context?: { source?: string }) =>+ typeof value === "number" ? wireJson.rawJSON(context!.source!) : value,+ ),+ "Semgrep API response",+ );+}++function platformId(value: unknown): string | undefined {+ let token: string;+ if (wireJson.isRawJSON(value)) token = value.rawJSON;+ else if (typeof value === "string" && /^[1-9][0-9]*$/u.test(value))+ token = value;+ else return undefined;+ const id = canonicalNumber(token);+ if (/^[1-9][0-9]*$/u.test(id)) return id;+ // A large scientific value is an integer only when every significant digit lies left of its decimal point.+ const scientific = /^[1-9](?:\.(\d+))?e\+(\d+)$/u.exec(id);+ return scientific && BigInt(scientific[2]!) >= (scientific[1]?.length ?? 0)+ ? id+ : undefined;+}++function requiredPlatformId(value: unknown): string {+ const id = platformId(value);+ if (!id) throw new Error("Expected an exact, positive Semgrep Platform ID.");+ return id;+}++/** Select SAST findings from operator-supplied Web API responses for local SDK import. */+export function normalizeSelectedSemgrepPlatformFindings(options: {+ deployment: string;+ ref: string;+ /** Independently supplied by the operator for the selected local checkout. */+ sourceRepositoryUrl: string;+ projectResponse: string;+ findingsResponses: Iterable<string>;+ findingIds: readonly string[];+}): ExternalFindings {+ const native = responseRecord(+ parseSemgrepResponse(options.projectResponse)["project"],+ "Semgrep project",+ );+ const scope = {+ deployment: requiredText(options.deployment, "Semgrep deployment"),+ project: {+ id: requiredPlatformId(native["id"]),+ name: requiredText(native["name"], "Semgrep project name"),+ url: declaredScannerRepositoryUrl(+ options.sourceRepositoryUrl,+ "Semgrep",+ "local",+ ),+ },+ ref: requiredText(options.ref, "Semgrep ref"),+ product: "sast",+ };+ // git-check-ref-format distinguishes literal branch/tag names from revision expressions.+ const refParts = scope.ref.split("/");+ if (+ refParts[0] !== "refs" ||+ refParts.length < 2 ||+ refParts.some(+ (part) => !part || part.startsWith(".") || part.endsWith(".lock"),+ ) ||+ scope.ref.endsWith(".") ||+ scope.ref.includes("..") ||+ scope.ref.includes("@{") ||+ /[\u0000-\u0020\u007f~^:?*\[\\]/u.test(scope.ref)+ )+ throw new Error(+ "Use a fully qualified Semgrep ref such as refs/heads/topic.",+ );+ if (native["url"] !== scope.project.url)+ throw new Error(+ "Selected Semgrep project does not match the operator's source repository URL.",+ );+ const wanted = new Set(+ options.findingIds.map((id) =>+ requiredPlatformId(requiredText(id, "Selected Semgrep finding ID")),+ ),+ );+ if (!wanted.size) throw new Error("Select at least one Semgrep finding ID.");+ const selected = new Map<string, Record<string, unknown>>();+ const pages =+ typeof options.findingsResponses === "string"+ ? [options.findingsResponses]+ : options.findingsResponses;+ const projectPath = decodeURIComponent(+ new URL(scope.project.url).pathname,+ ).replace(/^\/|\/$/gu, "");+ for (const response of pages) {+ const findings = array(+ parseSemgrepResponse(response)["findings"],+ "Semgrep findings",+ );+ const previousCount = selected.size;+ for (const finding of findings) {+ if (!isRecord(finding)) continue;+ const id = platformId(finding["id"]);+ if (!id || !wanted.has(id)) continue;+ if (selected.has(id))+ throw new Error(`Duplicate selected Semgrep finding ${id}.`);+ const repo = responseRecord(+ finding["repository"],+ "Semgrep finding repository",+ );+ const repositoryId =+ repo["id"] == null ? undefined : requiredPlatformId(repo["id"]);+ const url = repo["url"];+ const name = repo["name"];+ const matchesQualifiedPath =+ name === projectPath && projectPath.includes("/");+ if (+ finding["ref"] !== scope.ref ||+ (repositoryId !== undefined && repositoryId !== scope.project.id) ||+ (url != null+ ? url !== scope.project.url+ : repositoryId === undefined || !matchesQualifiedPath)+ )+ throw new Error(+ "Selected Semgrep finding does not belong to the selected project and ref.",+ );+ if (+ [finding["issue_type"], finding["product"]].some(+ (product) =>+ product != null &&+ (typeof product !== "string" ||+ !["sast", "code"].includes(product.toLowerCase())),+ )+ )+ throw new Error("Select Semgrep Code (SAST) findings.");+ selected.set(id, finding);+ }+ // The v1 findings endpoint returns a single product per response page.+ if (+ selected.size !== previousCount &&+ findings.some(identifiesNonSastProduct)+ )+ throw new Error("Select Semgrep Code (SAST) findings.");+ }+ const rows = Array.from(wanted, (id) => {+ const found = selected.get(id);+ if (!found) throw new Error(`Selected Semgrep finding ${id} is missing.`);+ return found;+ });+ // Older Platform imports hash safe integers as numbers, and larger IDs as strings.+ const exact = (id: string) =>+ Number.isSafeInteger(Number(id)) ? Number(id) : id;+ return normalizeRecords(+ rows,+ (finding) => {+ const rule = responseRecord(finding["rule"] ?? {}, "Semgrep rule");+ const name = text(rule["name"]) ?? text(finding["rule_name"]);+ const severity = text(finding["severity"])?.toLowerCase() ?? "";+ const description =+ text(rule["message"]) ?? text(finding["rule_message"]);+ const location = responseRecord(+ finding["location"] ?? {},+ "Semgrep location",+ );+ const rawPath = text(location["file_path"]);+ const relative = relativePath(rawPath);+ const path =+ relative && !relative.endsWith("/.") && repositoryFilePath(relative)+ ? rawPath+ : null;+ const startLine = path ? sourceLine(location["line"]) : null;+ return validateExternalEvidence(+ {+ title: requiredText(name ?? description, "Semgrep finding title"),+ description,+ severity: ["low", "medium", "high", "critical"].includes(severity)+ ? severity+ : "unknown",+ locations: path ? [{ path, line: startLine }] : [],+ branch: scope.ref,+ details: {+ kind: "sast",+ repository: { url: scope.project.url },+ scanner_origin: "Semgrep",+ rule: name ? { id: name } : null,+ source_status: text(finding["status"]),+ weakness_ids: weaknessIds(rule["cwe_names"]),+ code: {+ end_line:+ startLine !== null ? sourceLine(location["end_line"]) : null,+ },+ },+ source_data: {+ format: "semgrep-platform-selection/v1",+ ...scope,+ project: {+ ...scope.project,+ id: text(native["id"]) ?? scope.project.id,+ },+ semgrepPlatformFindingJson: JSON.stringify(+ finding,+ (_key, value: unknown) => {+ if (wireJson.isRawJSON(value))+ return wireJson.rawJSON(canonicalNumber(value.rawJSON));+ // Native object key order can vary across otherwise identical reads.+ if (isRecord(value))+ return Object.fromEntries(+ Object.keys(value)+ .sort()+ .map((key) => [key, value[key]]),+ );+ return value;+ },+ ),+ },+ },+ "local",+ );+ },+ (finding) =>+ stableId("semgrep-platform", [+ scope.deployment,+ exact(scope.project.id),+ scope.product,+ scope.ref,+ exact(requiredPlatformId(finding["id"])),+ ]),+ "local",+ );+}diff --git a/sdk/typescript/tests-ts/request-metadata.test.ts b/sdk/typescript/tests-ts/request-metadata.test.tsindex 4e834bbc1..c54f446cf 100644--- a/sdk/typescript/tests-ts/request-metadata.test.ts+++ b/sdk/typescript/tests-ts/request-metadata.test.ts@@ -1,5 +1,5 @@import { writeFile } from "node:fs/promises";-import { join } from "node:path";+import { dirname, join } from "node:path";import { pathToFileURL } from "node:url";import { afterEach, expect, test } from "bun:test";import { parse } from "smol-toml";@@ -114,7 +114,8 @@ test("native table overrides preserve inherited literal metadata and opt-outs",);const config = await readDeepScanRuntimeConfig({codexPath: resolveCodexCommand({}).command,- cwd: home,+ // Native descendants can hold their working directory open on Windows.+ cwd: dirname(home),configOverrides: [context.configOverride],env: {PATH: process.env["PATH"],diff --git a/sdk/typescript/tests-ts/semgrep-platform-selection.test.ts b/sdk/typescript/tests-ts/semgrep-platform-selection.test.tsnew file mode 100644index 000000000..a001eb345--- /dev/null+++ b/sdk/typescript/tests-ts/semgrep-platform-selection.test.ts@@ -0,0 +1,387 @@+import { expect, test } from "bun:test";+import { execFileSync } from "node:child_process";+import {+ listLocalExternalFindings,+ prepareNormalizedLocalImport,+} from "../src/external-findings-local.js";+import { normalizeSelectedSemgrepPlatformFindings } from "../src/semgrep-platform-findings.js";+import { localFindingsFixture } from "./external-findings-local-helpers.js";++const url = "https://github.com/example/project";+const ref = "refs/heads/topic";+const ids = ["9007199254740992", "9007199254740993"];+const source = {+ deployment: "demo",+ ref,+ sourceRepositoryUrl: url,+ projectResponse: JSON.stringify({+ project: { id: 77, name: "example/project", url },+ }),+ findingIds: ids,+};+const page = (id: string, branch = ref, repo = url) =>+ `{"findings":[{"id":${id},"ref":"${branch}","repository":{"url":"${repo}"},"rule":{"name":"example.rule"},"severity":"medium","location":{"file_path":"src/app.ts","line":2},"score":1.00000000000000001,"units":9007199254740993,"magnitude":1e100000}]}`;++test("reimporting two selected IDs from a non-default branch leaves both SQLite records unchanged", async () => {+ await using fixture = await localFindingsFixture({ remote: url });+ execFileSync(+ "git",+ ["-C", fixture.repository, "switch", "--quiet", "--orphan", "topic"],+ {+ env: fixture.environment,+ },+ );+ expect(+ execFileSync("git", ["-C", fixture.repository, "symbolic-ref", "HEAD"], {+ encoding: "utf8",+ env: fixture.environment,+ }).trim(),+ ).toBe(ref);+ const options = {+ provider: "semgrep" as const,+ sourceKey: "demo/topic",+ repository: fixture.repository,+ };+ const dependencies = {+ environment: fixture.environment,+ sourceRepositories: [{ url }],+ };+ const prepare = (findingsResponses: string[]) =>+ prepareNormalizedLocalImport(+ normalizeSelectedSemgrepPlatformFindings({+ ...source,+ findingsResponses,+ }),+ options,+ dependencies,+ );+ const pages = [page("3"), ...ids.map((id) => page(id))];+ const first = await prepare(pages);+ expect(first.preview.excluded).toEqual([]);+ expect((await first.import()).counts).toEqual({+ created: 2,+ updated: 0,+ unchanged: 0,+ });+ const original = await listLocalExternalFindings(options, dependencies);+ expect(original).toHaveLength(2);+ expect(new Set(original.map((row) => row.sourceFindingId)).size).toBe(2);+ for (const row of original) {+ expect(row.evidence.branch).toBe(ref);+ const raw = row.evidence.source_data?.["semgrepPlatformFindingJson"];+ expect(raw).toContain('"score":1.00000000000000001');+ expect(raw).toContain('"units":9007199254740993');+ expect(raw).toMatch(/"magnitude":1e\+?100000/u);+ }+ const replay = await prepare(+ pages+ .reverse()+ .map((p) =>+ p+ .replace('"severity":"medium","location":', '"location":')+ .replace('"line":2', '"line":2.0')+ .replace(+ ',"score":1.00000000000000001',+ ',"severity":"medium","score":1.000000000000000010',+ )+ .replace('"units":9007199254740993', '"units":90071992547409930e-1')+ .replace('"magnitude":1e100000', '"magnitude":10e99999'),+ ),+ );+ expect((await replay.import()).counts).toEqual({+ created: 0,+ updated: 0,+ unchanged: 2,+ });+ const malformed = await prepare([+ page(ids[0]!).replace('"rule":{"name":"example.rule"}', '"rule":42'),+ page(ids[1]!).replace(+ '"location":{"file_path":"src/app.ts","line":2}',+ '"location":42',+ ),+ ]);+ expect(malformed.preview.excluded).toHaveLength(2);+ expect((await malformed.import()).counts).toEqual({+ created: 0,+ updated: 0,+ unchanged: 0,+ });+ const identityOnly = await prepare(+ ids.map((id) =>+ JSON.stringify({+ findings: [{ id, ref, repository: { url } }],+ }),+ ),+ );+ expect(identityOnly.preview.excluded).toHaveLength(2);+ expect((await identityOnly.import()).counts).toEqual({+ created: 0,+ updated: 0,+ unchanged: 0,+ });+ expect(await listLocalExternalFindings(options, dependencies)).toEqual(+ original,+ );+});++test("each selected finding must match the independently named project, ref, and SAST product", () => {+ const first = page(ids[0]!);+ const second = page(ids[1]!);+ const select = (pages: string[]) =>+ normalizeSelectedSemgrepPlatformFindings({+ ...source,+ findingsResponses: pages,+ });+ const unrelated = '{"findings":[{"rule_name":"no-id"},{"id":2.5}]}';+ expect(select([first, unrelated, second]).findings).toHaveLength(2);+ const findingsPage = (rows: unknown[]) => JSON.stringify({ findings: rows });+ const selectedRow = {+ id: ids[1],+ ref,+ repository: { url },+ rule_name: "example.rule",+ };+ for (const other of [+ { ai_impact: "Synthetic impact" },+ { is_malicious: false },+ { issue_type: "ai_sast" },+ ]) {+ const sibling = { ...selectedRow, id: "99", ...other };+ expect(() => select([first, findingsPage([selectedRow, sibling])])).toThrow(+ "SAST",+ );+ expect(() => select([first, findingsPage([sibling, selectedRow])])).toThrow(+ "SAST",+ );+ expect(+ select([first, findingsPage([sibling]), second]).findings,+ ).toHaveLength(2);+ }+ const unknown = select([+ first,+ second.replace('"severity":"medium"', '"severity":"WARNING"'),+ ]).findings[1]!;+ expect(unknown.evidence.severity).toBe("unknown");+ expect(+ unknown.evidence.source_data?.["semgrepPlatformFindingJson"],+ ).toContain('"severity":"WARNING"');+ expect(() => select([first, unrelated])).toThrow(ids[1]);+ expect(() => select([first, second, second])).toThrow("Duplicate");+ expect(() => select([first, page(ids[1]!, "refs/heads/main")])).toThrow(+ "project and ref",+ );+ expect(() => select([first, page(ids[1]!, ref, url + ".git")])).toThrow(+ "project and ref",+ );+ for (const fields of [+ { ai_impact: "Synthetic impact" },+ { epss_score: { score: 0 } },+ { fix_recommendations: [{ package: "example", version: "2.0.0" }] },+ { found_dependency: { package: "example" } },+ { is_malicious: false },+ { reachability: "reachable" },+ { reachable_condition: "imported" },+ { usage: { location: { file_path: "src/app.ts" } } },+ { vulnerability_identifier: "CVE-2099-0001" },+ ]) {+ const finding = { ...selectedRow, ...fields };+ expect(() =>+ select([first, JSON.stringify({ findings: [finding] })]),+ ).toThrow("SAST");+ }+ for (const product of ['"sca"', '["sast"]']) {+ expect(() =>+ select([+ first,+ second.replace('"id":', '"product":' + product + ',"id":'),+ ]),+ ).toThrow("SAST");+ }+ expect(() =>+ normalizeSelectedSemgrepPlatformFindings({+ ...source,+ sourceRepositoryUrl: url + "-other",+ findingsResponses: [first, second],+ }),+ ).toThrow("operator");+});++test("the same native ID from distinct project IDs or refs has distinct occurrence IDs", () => {+ const occurrence = (projectId: number, branch: string) =>+ normalizeSelectedSemgrepPlatformFindings({+ ...source,+ projectResponse: JSON.stringify({+ project: { id: projectId, name: "example/project", url },+ }),+ ref: branch,+ findingIds: ["8"],+ findingsResponses: page("8", branch),+ }).findings[0]!.source_finding_id;+ expect(+ new Set([+ occurrence(77, ref),+ occurrence(78, ref),+ occurrence(77, "refs/tags/topic"),+ ]).size,+ ).toBe(3);+ for (const invalid of [+ "topic",+ "refs/heads/main~1",+ "refs/heads/.hidden",+ "refs/heads/a.lock",+ ]) {+ expect(() => occurrence(77, invalid)).toThrow(/ref/i);+ }+});++test("equivalent numeric IDs replay as integers while textual IDs stay exact", () => {+ const select = (token: string, wanted = "8", project = "77") =>+ normalizeSelectedSemgrepPlatformFindings({+ ...source,+ projectResponse: source.projectResponse.replace(+ '"id":77',+ '"id":' + project,+ ),+ findingIds: [wanted],+ findingsResponses: page(token),+ }).findings[0]!;+ const original = select("8");+ expect(select("8.0", "8", "77.0")).toEqual(original);+ expect(select("8e0", "8", "77e0")).toEqual(original);+ for (const token of ['"8.0"', '"8e0"', "8.00000000000000001"])+ expect(() => select(token)).toThrow("missing");+ expect(() => select("8", JSON.parse("[8.00000000000000001]")[0])).toThrow(+ "must be nonempty text",+ );+ const int64Max = "9223372036854775807";+ expect(select("9.223372036854775807e18", int64Max)).toEqual(+ select(int64Max, int64Max),+ );+ const large = "12345678901234567890123";+ expect(select(large + ".0", large)).toEqual(select(large, large));+ expect(select(large + "e0", large)).toEqual(select(large, large));+ expect(+ select(large, large).evidence.source_data?.["semgrepPlatformFindingJson"],+ ).toContain('"id":1.2345678901234567890123e+22,');+ const textualProject = select("8", "8", JSON.stringify(large));+ expect(textualProject.evidence.source_data?.["project"]).toMatchObject({+ id: large,+ });+ expect(textualProject.source_finding_id).toBe(+ select("8", "8", large).source_finding_id,+ );+ const power = "1000000000000000000000";+ expect(select("1e21", power)).toEqual(select(power, power));+ expect(() => select("1.0000000000000000000001e21", power)).toThrow("missing");+});++test("a URL-less row needs the matching project ID and full name, while an exact URL identifies the path of a short-named project", () => {+ function select(+ projectName: string,+ repository: Record<string, unknown>,+ projectUrl = url,+ ) {+ return normalizeSelectedSemgrepPlatformFindings({+ ...source,+ sourceRepositoryUrl: projectUrl,+ projectResponse: JSON.stringify({+ project: { id: 77, name: projectName, url: projectUrl },+ }),+ findingIds: ["8"],+ findingsResponses: [+ JSON.stringify({+ findings: [{ id: 8, ref, repository, rule_name: "example.rule" }],+ }),+ ],+ });+ }+ const name = "example/project";+ expect(() =>+ select("old-group/project", { id: 77, name: "old-group/project" }),+ ).toThrow("project and ref");+ expect(select(name, { id: 77, name }).findings).toHaveLength(1);+ expect(select("project", { id: 77, name }).findings).toHaveLength(1);+ expect(() => select("project", { id: 77, name: "project" })).toThrow(+ "project and ref",+ );+ for (const repository of [+ { name },+ { id: 78, name },+ { id: 77, name: "project" },+ ]) {+ expect(() => select(name, repository)).toThrow("project and ref");+ }+ expect(select("project", { url, name }).findings).toHaveLength(1);+ expect(select("old-group/project", { url, name }).findings).toHaveLength(1);+ expect(+ select("group/old-name", { url, name: "project" }).findings,+ ).toHaveLength(1);+ const encoded = "https://github.com/example/caf%C3%A9";+ expect(+ select("café", { id: 77, name: "example/café" }, encoded).findings,+ ).toHaveLength(1);+ const literal = "https://scm.example.test/team/repo.git";+ expect(() =>+ select("team/repo.git", { id: 77, name: "team/repo" }, literal),+ ).toThrow("project and ref");+ expect(+ select("team/repo.git", { id: 77, name: "team/repo.git" }, literal)+ .findings,+ ).toHaveLength(1);+ const cloneUrl = url + ".git";+ expect(+ select("project", { url: cloneUrl, name }, cloneUrl).findings,+ ).toHaveLength(1);+ const historicalProject = "old-group/project/api";+ const sibling = { url, name: "old-group/project/cli" };+ expect(select(historicalProject, sibling).findings).toHaveLength(1);+ expect(() => select(historicalProject, { ...sibling, id: 78 })).toThrow(+ "project and ref",+ );+});++test("a fractional source position cannot round to an integer during projection", () => {+ for (const position of [+ '"line":2.00000000000000001',+ '"line":2,"end_line":2.00000000000000001',+ ]) {+ const result = normalizeSelectedSemgrepPlatformFindings({+ ...source,+ findingIds: ["8"],+ findingsResponses: [page("8").replace('"line":2', position)],+ });+ expect(result.findings).toHaveLength(0);+ expect(result.excluded).toHaveLength(1);+ }+});++test("only repository-relative files are projected and original paths remain in evidence", () => {+ const evidenceFor = (path: string) =>+ normalizeSelectedSemgrepPlatformFindings({+ ...source,+ findingIds: ["8"],+ findingsResponses: page("8").replace(+ '"src/app.ts"',+ JSON.stringify(path),+ ),+ }).findings[0]!.evidence;+ for (const path of ["src/vendor://app.ts", "src/a\\b.ts", "src/a/b.ts"]) {+ expect(evidenceFor(path).locations).toEqual([{ path, line: 2 }]);+ }+ for (const path of [+ "file:src/app.ts",+ "data:text/plain,hello",+ "generated:/api.ts",+ "./C:/work/app.ts",+ "git@host:team/repo/app.ts",+ "src\\..\\outside.ts",+ ".",+ "src/",+ ]) {+ const evidence = evidenceFor(path);+ expect(evidence.locations).toEqual([]);+ expect(evidence.source_data?.["semgrepPlatformFindingJson"]).toContain(+ JSON.stringify(path),+ );+ }+});