Skip to content

OpenAI's Codex Security CLI and TypeScript SDK for finding, validating, and fixing security vulnerabilities. npm: https://www.npmjs.com/package/@openai/codex-security

feat(imports): normalize selected Semgrep Code findings for local import (#2164)

Michael D'Angelo80821889d0b5
diff --git a/docs/scanner-imports/README.md b/docs/scanner-imports/README.md
index 4a1369bf4..d9c554af5 100644
--- a/docs/scanner-imports/README.md
+++ b/docs/scanner-imports/README.md
@@ -13,7 +13,7 @@ guide shows how to preview the findings before you save or publish them.
| [GitHub Dependabot](github-dependabot.md) | Dependency alerts exported from one repository |
| [GitHub Security Advisories](../../sdk/typescript/docs/cli.md#import-ghsa-advisories-separately) | Selected repository or global advisories |
| [OpenGrep](opengrep.md) | Native scan JSON, optionally gzip-compressed |
-| [Semgrep Code](semgrep.md) | CLI or CI JSON, or a self-contained SARIF report |
+| [Semgrep Code](semgrep.md) | CLI/CI JSON, self-contained SARIF, or selected Platform Web API responses (SDK) |
| [Snyk Open Source](snyk.md) | CLI JSON, a Codex plugin export, or SARIF |
| [SonarQube Cloud](sonar.md) | Selected security issues from the signed-in Sonar CLI or saved issue-search pages |
| [Veracode Static Analysis](veracode.md) | Findings API pages for one application's policy flaws |
diff --git a/docs/scanner-imports/schema-guide.md b/docs/scanner-imports/schema-guide.md
index a2dcf1f03..6378ea25a 100644
--- a/docs/scanner-imports/schema-guide.md
+++ b/docs/scanner-imports/schema-guide.md
@@ -24,7 +24,8 @@ changes.
| [GitHub advisories](../../sdk/typescript/docs/cli.md#import-ghsa-advisories-separately) | Host, repository or global scope, and GHSA ID. This identifies a disclosure. Check installed packages separately. |
| [Snyk Open Source](snyk.md) | Native CLI JSON distinguishes manifest, package manager, advisory, package name and version, and dependency path. Plugin and SARIF reports use different identities. |
| [Black Duck SCA](blackduck.md), [Veracode Static Analysis](veracode.md) | Black Duck combines project version, component version, origin and advisory identity. Veracode scopes a native issue ID to its API origin, application and context. |
-| [Semgrep Code](semgrep.md), [OpenGrep](opengrep.md) | Semgrep prefers a fingerprint and falls back to rule, path and range. OpenGrep groups traces by rule, path and range. Both store provider `semgrep`. Use a separate OpenGrep source key. |
+| [Semgrep CLI or CI reports](semgrep.md), [OpenGrep](opengrep.md) | Semgrep prefers a fingerprint and falls back to rule, path and range. OpenGrep groups traces by rule, path and range. Both store provider `semgrep`. Use a separate OpenGrep source key. |
+| [Semgrep Platform SAST](semgrep.md#import-selected-semgrep-platform-sast-findings-with-the-sdk) | Deployment, project ID, SAST product, full Git ref and native finding ID. The same occurrence keeps its identity when its file moves. |
Repeat local imports replace changed evidence for the same identity. Identical
evidence is unchanged, and records omitted from a later report remain stored.
diff --git a/docs/scanner-imports/semgrep.md b/docs/scanner-imports/semgrep.md
index 27752cea1..a9b82f73b 100644
--- a/docs/scanner-imports/semgrep.md
+++ b/docs/scanner-imports/semgrep.md
@@ -3,6 +3,8 @@
Import a saved Semgrep Code CLI or CI report into the local workbench or Cloud.
Native JSON preserves original severities and snippets when available. The reader
also accepts self-contained Semgrep SARIF 2.1.0. Either format can be gzipped.
+To import selected existing findings from the Platform API, use the
+[SDK example](#import-selected-semgrep-platform-sast-findings-with-the-sdk).
## Quick start
@@ -100,10 +102,53 @@ directly, pass the original repository references in its third argument's
`sourceRepositories`. Use `{ url, platform: "github" }` for a verified custom
GitHub host.
-### Existing Platform findings
+### Import selected Semgrep Platform SAST findings with the SDK
+
+Request `issue_type=sast` and keep the project lookup and findings pages as original
+[Semgrep Web API](https://docs.semgrep.dev/semgrep-appsec-platform/semgrep-api) response text. Pass each page separately. When a page contributes a selected ID, the
+SDK checks its rows for conflicting product evidence. If a page contains only fields
+shared by several products, its product depends on the original request filter.
+[Guardian for Codex](https://docs.semgrep.dev/semgrep-guardian/ide-setup/codex) and
+[Semgrep MCP](https://github.com/semgrep/semgrep/blob/fe209763a4a2ccfcdf6434e6c332edc24a2f586e/cli/src/semgrep/mcp/server.py) return different records.
+Pass a fully qualified ref, such as `refs/heads/topic` or `refs/tags/topic`, exactly as it appears in the findings.
+Supply the HTTP(S) repository URL you independently associate with the target checkout.
+The native `project.url` and any selected `finding.repository.url` must match this spelling. Legacy findings can
+omit the project ID. An exact URL then identifies the repository even if the
+display name has changed. If multiple Semgrep projects share that URL, select
+the IDs from the intended project: the URL alone can also match a sibling project
+on the same ref. A conflicting project ID is always rejected. A finding without a
+URL must carry the matching project ID and a name equal to the namespace-qualified,
+decoded path in the operator-supplied URL.
+The local importer checks the checkout using its existing repository rules.
+
+```ts
+const selected = normalizeSelectedSemgrepPlatformFindings({
+ deployment,
+ ref,
+ sourceRepositoryUrl,
+ projectResponse,
+ findingsResponses,
+ findingIds,
+});
+const prepared = await prepareNormalizedLocalImport(
+ selected,
+ { provider: "semgrep", sourceKey, repository: checkout },
+ { sourceRepositories: [{ url: sourceRepositoryUrl }] },
+);
+if (prepared.preview.excluded.length) {
+ console.error(prepared.preview.excluded);
+} else {
+ await prepared.import();
+}
+```
+
+Keep the same source key on replay. Finding identity includes the deployment, project ID,
+SAST product, full ref, and native finding ID. Pass selected finding IDs as positive
+decimal strings without leading zeros. Semgrep emits 64-bit finding IDs. The adapter
+also accepts larger positive integer IDs in saved responses without rounding. The
+stored evidence includes the selected finding and the project's ID, name and URL.
+A record without a native rule name or message is excluded during preview.
-This file reader accepts scan reports. Semgrep Guardian, MCP and the Platform
-findings API return different records. For this workflow, use a Code scan
-artifact. [Guardian for Codex](https://docs.semgrep.dev/semgrep-guardian/ide-setup/codex),
-[Semgrep MCP implementation](https://github.com/semgrep/semgrep/blob/fe209763a4a2ccfcdf6434e6c332edc24a2f586e/cli/src/semgrep/mcp/server.py)
-and [Platform findings API](https://docs.semgrep.dev/semgrep-appsec-platform/semgrep-api)
+Changes to key order or equivalent numeric spellings such as `2` and `2.0` leave
+the stored record unchanged. Only repository-relative files become locations. Names
+with URI-like prefixes and directory-only paths remain in the native evidence.
diff --git a/sdk/typescript/README.md b/sdk/typescript/README.md
index 57d6c79a7..0261db320 100644
--- a/sdk/typescript/README.md
+++ b/sdk/typescript/README.md
@@ -674,6 +674,8 @@ Omit `issueKeys` to collect the project's open security issues. An empty list is
If you know the source repository, pass its Git URL in `options.sourceRepositoryUrl`
to check the Sonar binding before fetching issues.
Pure normalizers and file readers remain available.
+Use `normalizeSelectedSemgrepPlatformFindings` for selected SAST findings from saved
+Semgrep Web API responses. See the [Semgrep SDK example](https://github.com/openai/codex-security/blob/main/docs/scanner-imports/semgrep.md#import-selected-semgrep-platform-sast-findings-with-the-sdk).
This does not affect scan publishing or the existing GitHub alert collector.
The SDK also exposes `prepareExternalPublication(file, options)` for supported
diff --git a/sdk/typescript/scripts/package-dist-files.mjs b/sdk/typescript/scripts/package-dist-files.mjs
index 70f5b8f65..57c085db5 100644
--- a/sdk/typescript/scripts/package-dist-files.mjs
+++ b/sdk/typescript/scripts/package-dist-files.mjs
@@ -32,6 +32,7 @@ const distModules = [
"snyk-findings",
"scanner-repository-url",
"semgrep-findings",
+ "semgrep-platform-findings",
"sonar-findings",
"sonar-collector",
"sonar-import",
diff --git a/sdk/typescript/src/index.ts b/sdk/typescript/src/index.ts
index cc810740a..2a63c1572 100644
--- a/sdk/typescript/src/index.ts
+++ b/sdk/typescript/src/index.ts
@@ -354,6 +354,7 @@ export {
normalizeSnykSarif,
readSnykFindings,
} from "./snyk-findings.js";
+export { normalizeSelectedSemgrepPlatformFindings } from "./semgrep-platform-findings.js";
export {
normalizeSemgrepFindings,
normalizeSemgrepSarif,
diff --git a/sdk/typescript/src/inspector-findings.ts b/sdk/typescript/src/inspector-findings.ts
index 476db7fad..b46a31c5a 100644
--- a/sdk/typescript/src/inspector-findings.ts
+++ b/sdk/typescript/src/inspector-findings.ts
@@ -20,7 +20,7 @@ import {
normalizeRecords,
readScannerJson,
record,
- relativePath,
+ repositoryFilePath,
requiredText,
text,
} from "./scanner-import.js";
@@ -444,17 +444,6 @@ export function inspectorReportRepository(
};
}
-function repoFile(value: unknown): string | null {
- const relative = relativePath(value);
- if (!relative) return null;
- const path = posix.normalize(relative);
- return path === "." ||
- path.endsWith("/") ||
- /^(?:[a-z][a-z\d+.-]*:|~[^/]*(?:\/|$)|[^/]+@[^/]+:)/iu.test(path)
- ? null
- : path;
-}
-
function timestamp(value: unknown): number {
const seconds =
typeof value === "number"
@@ -587,7 +576,7 @@ function evidence(
const level = requiredText(severity, "Inspector severity").toLowerCase();
const paths = new Set<string>();
const packages = (pkg?.vulnerablePackages ?? []).map((item) => {
- const path = repoFile(item.filePath);
+ const path = repositoryFilePath(item.filePath);
if (path) paths.add(path);
const ecosystem = ecosystems.get(text(item.packageManager) ?? "") ?? null;
const version = (value: unknown) =>
@@ -609,7 +598,7 @@ function evidence(
);
if (code) {
const { fileName, filePath, startLine } = code.filePath;
- const path = repoFile(filePath);
+ const path = repositoryFilePath(filePath);
if (path && posix.basename(path) === fileName)
locations.push({
path,
diff --git a/sdk/typescript/src/scanner-import.ts b/sdk/typescript/src/scanner-import.ts
index ce214decc..d9f92f425 100644
--- a/sdk/typescript/src/scanner-import.ts
+++ b/sdk/typescript/src/scanner-import.ts
@@ -1,5 +1,6 @@
import { hash } from "node:crypto";
import { readFile, type FileHandle } from "node:fs/promises";
+import { posix } from "node:path";
import { promisify } from "node:util";
import { gunzip } from "node:zlib";
import { CodexSecurityError, errorMessage } from "./errors.js";
@@ -45,6 +46,18 @@ export function relativePath(value: unknown): string | null {
: null;
}
+export function repositoryFilePath(value: unknown): string | null {
+ const relative = relativePath(value);
+ if (!relative) return null;
+ const path = posix.normalize(relative);
+ // Anchor the SCP branch at the first eligible @ to avoid rescanning long filenames.
+ return path === "." ||
+ path.endsWith("/") ||
+ /^(?:[a-z][a-z\d+.-]*:|~[^/]*(?:\/|$)|[^/][^/@]*@[^/]+:)/iu.test(path)
+ ? null
+ : path;
+}
+
export function stableId(prefix: string, fields: unknown[]): string {
return `${prefix}:${hash("sha256", JSON.stringify(fields))}`;
}
diff --git a/sdk/typescript/src/semgrep-platform-findings.ts b/sdk/typescript/src/semgrep-platform-findings.ts
new file mode 100644
index 000000000..bff161c84
--- /dev/null
+++ b/sdk/typescript/src/semgrep-platform-findings.ts
@@ -0,0 +1,322 @@
+import { validateExternalEvidence } from "./external-import-contract.js";
+import type { ExternalFindings } from "./external-findings.js";
+import { isRecord } from "./record.js";
+import { position, weaknessIds } from "./native-json-findings.js";
+import {
+ array,
+ normalizeRecords,
+ record,
+ relativePath,
+ repositoryFilePath,
+ requiredText,
+ stableId,
+ text,
+} from "./scanner-import.js";
+import { declaredScannerRepositoryUrl } from "./scanner-repository-url.js";
+
+// These v1 Findings API response fields belong only to AI detections or Supply Chain.
+const nonSastFields = [
+ "ai_impact",
+ "epss_score",
+ "fix_recommendations",
+ "found_dependency",
+ "is_malicious",
+ "reachability",
+ "reachable_condition",
+ "usage",
+ "vulnerability_identifier",
+];
+
+function identifiesNonSastProduct(value: unknown): boolean {
+ if (!isRecord(value)) return false;
+ return (
+ nonSastFields.some((field) => value[field] != null) ||
+ [value["issue_type"], value["product"]].some(
+ (product) =>
+ typeof product === "string" &&
+ /^(?:sca|ai_sast|secrets)$/iu.test(product),
+ )
+ );
+}
+
+// Node 22 supports JSON source text and raw values. The SDK targets the ES2022 type library.
+const wireJson = JSON as JSON & {
+ rawJSON(source: string): { rawJSON: string };
+ isRawJSON(value: unknown): value is { rawJSON: string };
+};
+
+// A retained JSON number is represented as an object in JavaScript.
+function responseRecord(value: unknown, label: string) {
+ return record(wireJson.isRawJSON(value) ? undefined : value, label);
+}
+
+// Normalize equivalent JSON numbers exactly. Keep very large exponents compact.
+function canonicalNumber(token: string): string {
+ const [, sign = "", integer, fraction = "", exponent = "0"] =
+ /^(-?)(\d+)(?:\.(\d+))?(?:e([+-]?\d+))?$/iu.exec(token)!;
+ const significant = (integer + fraction).replace(/^0+/u, "");
+ if (!significant) return "0";
+ let end = significant.length;
+ while (significant[end - 1] === "0") end--;
+ const digits = significant.slice(0, end);
+ const point = BigInt(exponent) + BigInt(significant.length - fraction.length);
+ // Ordinary evidence follows JSON.stringify’s fixed/scientific notation thresholds.
+ if (point > -6n && point <= 21n) {
+ const split = Number(point);
+ if (split <= 0) return sign + "0." + "0".repeat(-split) + digits;
+ if (split >= digits.length) return sign + digits.padEnd(split, "0");
+ return sign + digits.slice(0, split) + "." + digits.slice(split);
+ }
+ const trailing = digits.slice(1);
+ const scientific = point - 1n;
+ return (
+ sign +
+ digits.charAt(0) +
+ (trailing ? "." + trailing : "") +
+ "e" +
+ (scientific >= 0n ? "+" : "") +
+ scientific
+ );
+}
+
+function sourceLine(value: unknown): number | null {
+ if (value == null) return null;
+ if (!wireJson.isRawJSON(value)) return position(value, "Semgrep source line");
+ const exact = canonicalNumber(value.rawJSON);
+ return position(
+ /^\d+$/u.test(exact) ? Number(exact) : undefined,
+ "Semgrep source line",
+ );
+}
+
+/** Keep original number tokens, including adjacent int64 IDs above Number.MAX_SAFE_INTEGER. */
+function parseSemgrepResponse(json: string): Record<string, unknown> {
+ return responseRecord(
+ JSON.parse(
+ json.replace(/^\uFEFF/u, ""),
+ (_key: string, value: unknown, context?: { source?: string }) =>
+ typeof value === "number" ? wireJson.rawJSON(context!.source!) : value,
+ ),
+ "Semgrep API response",
+ );
+}
+
+function platformId(value: unknown): string | undefined {
+ let token: string;
+ if (wireJson.isRawJSON(value)) token = value.rawJSON;
+ else if (typeof value === "string" && /^[1-9][0-9]*$/u.test(value))
+ token = value;
+ else return undefined;
+ const id = canonicalNumber(token);
+ if (/^[1-9][0-9]*$/u.test(id)) return id;
+ // A large scientific value is an integer only when every significant digit lies left of its decimal point.
+ const scientific = /^[1-9](?:\.(\d+))?e\+(\d+)$/u.exec(id);
+ return scientific && BigInt(scientific[2]!) >= (scientific[1]?.length ?? 0)
+ ? id
+ : undefined;
+}
+
+function requiredPlatformId(value: unknown): string {
+ const id = platformId(value);
+ if (!id) throw new Error("Expected an exact, positive Semgrep Platform ID.");
+ return id;
+}
+
+/** Select SAST findings from operator-supplied Web API responses for local SDK import. */
+export function normalizeSelectedSemgrepPlatformFindings(options: {
+ deployment: string;
+ ref: string;
+ /** Independently supplied by the operator for the selected local checkout. */
+ sourceRepositoryUrl: string;
+ projectResponse: string;
+ findingsResponses: Iterable<string>;
+ findingIds: readonly string[];
+}): ExternalFindings {
+ const native = responseRecord(
+ parseSemgrepResponse(options.projectResponse)["project"],
+ "Semgrep project",
+ );
+ const scope = {
+ deployment: requiredText(options.deployment, "Semgrep deployment"),
+ project: {
+ id: requiredPlatformId(native["id"]),
+ name: requiredText(native["name"], "Semgrep project name"),
+ url: declaredScannerRepositoryUrl(
+ options.sourceRepositoryUrl,
+ "Semgrep",
+ "local",
+ ),
+ },
+ ref: requiredText(options.ref, "Semgrep ref"),
+ product: "sast",
+ };
+ // git-check-ref-format distinguishes literal branch/tag names from revision expressions.
+ const refParts = scope.ref.split("/");
+ if (
+ refParts[0] !== "refs" ||
+ refParts.length < 2 ||
+ refParts.some(
+ (part) => !part || part.startsWith(".") || part.endsWith(".lock"),
+ ) ||
+ scope.ref.endsWith(".") ||
+ scope.ref.includes("..") ||
+ scope.ref.includes("@{") ||
+ /[\u0000-\u0020\u007f~^:?*\[\\]/u.test(scope.ref)
+ )
+ throw new Error(
+ "Use a fully qualified Semgrep ref such as refs/heads/topic.",
+ );
+ if (native["url"] !== scope.project.url)
+ throw new Error(
+ "Selected Semgrep project does not match the operator's source repository URL.",
+ );
+ const wanted = new Set(
+ options.findingIds.map((id) =>
+ requiredPlatformId(requiredText(id, "Selected Semgrep finding ID")),
+ ),
+ );
+ if (!wanted.size) throw new Error("Select at least one Semgrep finding ID.");
+ const selected = new Map<string, Record<string, unknown>>();
+ const pages =
+ typeof options.findingsResponses === "string"
+ ? [options.findingsResponses]
+ : options.findingsResponses;
+ const projectPath = decodeURIComponent(
+ new URL(scope.project.url).pathname,
+ ).replace(/^\/|\/$/gu, "");
+ for (const response of pages) {
+ const findings = array(
+ parseSemgrepResponse(response)["findings"],
+ "Semgrep findings",
+ );
+ const previousCount = selected.size;
+ for (const finding of findings) {
+ if (!isRecord(finding)) continue;
+ const id = platformId(finding["id"]);
+ if (!id || !wanted.has(id)) continue;
+ if (selected.has(id))
+ throw new Error(`Duplicate selected Semgrep finding ${id}.`);
+ const repo = responseRecord(
+ finding["repository"],
+ "Semgrep finding repository",
+ );
+ const repositoryId =
+ repo["id"] == null ? undefined : requiredPlatformId(repo["id"]);
+ const url = repo["url"];
+ const name = repo["name"];
+ const matchesQualifiedPath =
+ name === projectPath && projectPath.includes("/");
+ if (
+ finding["ref"] !== scope.ref ||
+ (repositoryId !== undefined && repositoryId !== scope.project.id) ||
+ (url != null
+ ? url !== scope.project.url
+ : repositoryId === undefined || !matchesQualifiedPath)
+ )
+ throw new Error(
+ "Selected Semgrep finding does not belong to the selected project and ref.",
+ );
+ if (
+ [finding["issue_type"], finding["product"]].some(
+ (product) =>
+ product != null &&
+ (typeof product !== "string" ||
+ !["sast", "code"].includes(product.toLowerCase())),
+ )
+ )
+ throw new Error("Select Semgrep Code (SAST) findings.");
+ selected.set(id, finding);
+ }
+ // The v1 findings endpoint returns a single product per response page.
+ if (
+ selected.size !== previousCount &&
+ findings.some(identifiesNonSastProduct)
+ )
+ throw new Error("Select Semgrep Code (SAST) findings.");
+ }
+ const rows = Array.from(wanted, (id) => {
+ const found = selected.get(id);
+ if (!found) throw new Error(`Selected Semgrep finding ${id} is missing.`);
+ return found;
+ });
+ // Older Platform imports hash safe integers as numbers, and larger IDs as strings.
+ const exact = (id: string) =>
+ Number.isSafeInteger(Number(id)) ? Number(id) : id;
+ return normalizeRecords(
+ rows,
+ (finding) => {
+ const rule = responseRecord(finding["rule"] ?? {}, "Semgrep rule");
+ const name = text(rule["name"]) ?? text(finding["rule_name"]);
+ const severity = text(finding["severity"])?.toLowerCase() ?? "";
+ const description =
+ text(rule["message"]) ?? text(finding["rule_message"]);
+ const location = responseRecord(
+ finding["location"] ?? {},
+ "Semgrep location",
+ );
+ const rawPath = text(location["file_path"]);
+ const relative = relativePath(rawPath);
+ const path =
+ relative && !relative.endsWith("/.") && repositoryFilePath(relative)
+ ? rawPath
+ : null;
+ const startLine = path ? sourceLine(location["line"]) : null;
+ return validateExternalEvidence(
+ {
+ title: requiredText(name ?? description, "Semgrep finding title"),
+ description,
+ severity: ["low", "medium", "high", "critical"].includes(severity)
+ ? severity
+ : "unknown",
+ locations: path ? [{ path, line: startLine }] : [],
+ branch: scope.ref,
+ details: {
+ kind: "sast",
+ repository: { url: scope.project.url },
+ scanner_origin: "Semgrep",
+ rule: name ? { id: name } : null,
+ source_status: text(finding["status"]),
+ weakness_ids: weaknessIds(rule["cwe_names"]),
+ code: {
+ end_line:
+ startLine !== null ? sourceLine(location["end_line"]) : null,
+ },
+ },
+ source_data: {
+ format: "semgrep-platform-selection/v1",
+ ...scope,
+ project: {
+ ...scope.project,
+ id: text(native["id"]) ?? scope.project.id,
+ },
+ semgrepPlatformFindingJson: JSON.stringify(
+ finding,
+ (_key, value: unknown) => {
+ if (wireJson.isRawJSON(value))
+ return wireJson.rawJSON(canonicalNumber(value.rawJSON));
+ // Native object key order can vary across otherwise identical reads.
+ if (isRecord(value))
+ return Object.fromEntries(
+ Object.keys(value)
+ .sort()
+ .map((key) => [key, value[key]]),
+ );
+ return value;
+ },
+ ),
+ },
+ },
+ "local",
+ );
+ },
+ (finding) =>
+ stableId("semgrep-platform", [
+ scope.deployment,
+ exact(scope.project.id),
+ scope.product,
+ scope.ref,
+ exact(requiredPlatformId(finding["id"])),
+ ]),
+ "local",
+ );
+}
diff --git a/sdk/typescript/tests-ts/request-metadata.test.ts b/sdk/typescript/tests-ts/request-metadata.test.ts
index 4e834bbc1..c54f446cf 100644
--- a/sdk/typescript/tests-ts/request-metadata.test.ts
+++ b/sdk/typescript/tests-ts/request-metadata.test.ts
@@ -1,5 +1,5 @@
import { writeFile } from "node:fs/promises";
-import { join } from "node:path";
+import { dirname, join } from "node:path";
import { pathToFileURL } from "node:url";
import { afterEach, expect, test } from "bun:test";
import { parse } from "smol-toml";
@@ -114,7 +114,8 @@ test("native table overrides preserve inherited literal metadata and opt-outs",
);
const config = await readDeepScanRuntimeConfig({
codexPath: resolveCodexCommand({}).command,
- cwd: home,
+ // Native descendants can hold their working directory open on Windows.
+ cwd: dirname(home),
configOverrides: [context.configOverride],
env: {
PATH: process.env["PATH"],
diff --git a/sdk/typescript/tests-ts/semgrep-platform-selection.test.ts b/sdk/typescript/tests-ts/semgrep-platform-selection.test.ts
new file mode 100644
index 000000000..a001eb345
--- /dev/null
+++ b/sdk/typescript/tests-ts/semgrep-platform-selection.test.ts
@@ -0,0 +1,387 @@
+import { expect, test } from "bun:test";
+import { execFileSync } from "node:child_process";
+import {
+ listLocalExternalFindings,
+ prepareNormalizedLocalImport,
+} from "../src/external-findings-local.js";
+import { normalizeSelectedSemgrepPlatformFindings } from "../src/semgrep-platform-findings.js";
+import { localFindingsFixture } from "./external-findings-local-helpers.js";
+
+const url = "https://github.com/example/project";
+const ref = "refs/heads/topic";
+const ids = ["9007199254740992", "9007199254740993"];
+const source = {
+ deployment: "demo",
+ ref,
+ sourceRepositoryUrl: url,
+ projectResponse: JSON.stringify({
+ project: { id: 77, name: "example/project", url },
+ }),
+ findingIds: ids,
+};
+const page = (id: string, branch = ref, repo = url) =>
+ `{"findings":[{"id":${id},"ref":"${branch}","repository":{"url":"${repo}"},"rule":{"name":"example.rule"},"severity":"medium","location":{"file_path":"src/app.ts","line":2},"score":1.00000000000000001,"units":9007199254740993,"magnitude":1e100000}]}`;
+
+test("reimporting two selected IDs from a non-default branch leaves both SQLite records unchanged", async () => {
+ await using fixture = await localFindingsFixture({ remote: url });
+ execFileSync(
+ "git",
+ ["-C", fixture.repository, "switch", "--quiet", "--orphan", "topic"],
+ {
+ env: fixture.environment,
+ },
+ );
+ expect(
+ execFileSync("git", ["-C", fixture.repository, "symbolic-ref", "HEAD"], {
+ encoding: "utf8",
+ env: fixture.environment,
+ }).trim(),
+ ).toBe(ref);
+ const options = {
+ provider: "semgrep" as const,
+ sourceKey: "demo/topic",
+ repository: fixture.repository,
+ };
+ const dependencies = {
+ environment: fixture.environment,
+ sourceRepositories: [{ url }],
+ };
+ const prepare = (findingsResponses: string[]) =>
+ prepareNormalizedLocalImport(
+ normalizeSelectedSemgrepPlatformFindings({
+ ...source,
+ findingsResponses,
+ }),
+ options,
+ dependencies,
+ );
+ const pages = [page("3"), ...ids.map((id) => page(id))];
+ const first = await prepare(pages);
+ expect(first.preview.excluded).toEqual([]);
+ expect((await first.import()).counts).toEqual({
+ created: 2,
+ updated: 0,
+ unchanged: 0,
+ });
+ const original = await listLocalExternalFindings(options, dependencies);
+ expect(original).toHaveLength(2);
+ expect(new Set(original.map((row) => row.sourceFindingId)).size).toBe(2);
+ for (const row of original) {
+ expect(row.evidence.branch).toBe(ref);
+ const raw = row.evidence.source_data?.["semgrepPlatformFindingJson"];
+ expect(raw).toContain('"score":1.00000000000000001');
+ expect(raw).toContain('"units":9007199254740993');
+ expect(raw).toMatch(/"magnitude":1e\+?100000/u);
+ }
+ const replay = await prepare(
+ pages
+ .reverse()
+ .map((p) =>
+ p
+ .replace('"severity":"medium","location":', '"location":')
+ .replace('"line":2', '"line":2.0')
+ .replace(
+ ',"score":1.00000000000000001',
+ ',"severity":"medium","score":1.000000000000000010',
+ )
+ .replace('"units":9007199254740993', '"units":90071992547409930e-1')
+ .replace('"magnitude":1e100000', '"magnitude":10e99999'),
+ ),
+ );
+ expect((await replay.import()).counts).toEqual({
+ created: 0,
+ updated: 0,
+ unchanged: 2,
+ });
+ const malformed = await prepare([
+ page(ids[0]!).replace('"rule":{"name":"example.rule"}', '"rule":42'),
+ page(ids[1]!).replace(
+ '"location":{"file_path":"src/app.ts","line":2}',
+ '"location":42',
+ ),
+ ]);
+ expect(malformed.preview.excluded).toHaveLength(2);
+ expect((await malformed.import()).counts).toEqual({
+ created: 0,
+ updated: 0,
+ unchanged: 0,
+ });
+ const identityOnly = await prepare(
+ ids.map((id) =>
+ JSON.stringify({
+ findings: [{ id, ref, repository: { url } }],
+ }),
+ ),
+ );
+ expect(identityOnly.preview.excluded).toHaveLength(2);
+ expect((await identityOnly.import()).counts).toEqual({
+ created: 0,
+ updated: 0,
+ unchanged: 0,
+ });
+ expect(await listLocalExternalFindings(options, dependencies)).toEqual(
+ original,
+ );
+});
+
+test("each selected finding must match the independently named project, ref, and SAST product", () => {
+ const first = page(ids[0]!);
+ const second = page(ids[1]!);
+ const select = (pages: string[]) =>
+ normalizeSelectedSemgrepPlatformFindings({
+ ...source,
+ findingsResponses: pages,
+ });
+ const unrelated = '{"findings":[{"rule_name":"no-id"},{"id":2.5}]}';
+ expect(select([first, unrelated, second]).findings).toHaveLength(2);
+ const findingsPage = (rows: unknown[]) => JSON.stringify({ findings: rows });
+ const selectedRow = {
+ id: ids[1],
+ ref,
+ repository: { url },
+ rule_name: "example.rule",
+ };
+ for (const other of [
+ { ai_impact: "Synthetic impact" },
+ { is_malicious: false },
+ { issue_type: "ai_sast" },
+ ]) {
+ const sibling = { ...selectedRow, id: "99", ...other };
+ expect(() => select([first, findingsPage([selectedRow, sibling])])).toThrow(
+ "SAST",
+ );
+ expect(() => select([first, findingsPage([sibling, selectedRow])])).toThrow(
+ "SAST",
+ );
+ expect(
+ select([first, findingsPage([sibling]), second]).findings,
+ ).toHaveLength(2);
+ }
+ const unknown = select([
+ first,
+ second.replace('"severity":"medium"', '"severity":"WARNING"'),
+ ]).findings[1]!;
+ expect(unknown.evidence.severity).toBe("unknown");
+ expect(
+ unknown.evidence.source_data?.["semgrepPlatformFindingJson"],
+ ).toContain('"severity":"WARNING"');
+ expect(() => select([first, unrelated])).toThrow(ids[1]);
+ expect(() => select([first, second, second])).toThrow("Duplicate");
+ expect(() => select([first, page(ids[1]!, "refs/heads/main")])).toThrow(
+ "project and ref",
+ );
+ expect(() => select([first, page(ids[1]!, ref, url + ".git")])).toThrow(
+ "project and ref",
+ );
+ for (const fields of [
+ { ai_impact: "Synthetic impact" },
+ { epss_score: { score: 0 } },
+ { fix_recommendations: [{ package: "example", version: "2.0.0" }] },
+ { found_dependency: { package: "example" } },
+ { is_malicious: false },
+ { reachability: "reachable" },
+ { reachable_condition: "imported" },
+ { usage: { location: { file_path: "src/app.ts" } } },
+ { vulnerability_identifier: "CVE-2099-0001" },
+ ]) {
+ const finding = { ...selectedRow, ...fields };
+ expect(() =>
+ select([first, JSON.stringify({ findings: [finding] })]),
+ ).toThrow("SAST");
+ }
+ for (const product of ['"sca"', '["sast"]']) {
+ expect(() =>
+ select([
+ first,
+ second.replace('"id":', '"product":' + product + ',"id":'),
+ ]),
+ ).toThrow("SAST");
+ }
+ expect(() =>
+ normalizeSelectedSemgrepPlatformFindings({
+ ...source,
+ sourceRepositoryUrl: url + "-other",
+ findingsResponses: [first, second],
+ }),
+ ).toThrow("operator");
+});
+
+test("the same native ID from distinct project IDs or refs has distinct occurrence IDs", () => {
+ const occurrence = (projectId: number, branch: string) =>
+ normalizeSelectedSemgrepPlatformFindings({
+ ...source,
+ projectResponse: JSON.stringify({
+ project: { id: projectId, name: "example/project", url },
+ }),
+ ref: branch,
+ findingIds: ["8"],
+ findingsResponses: page("8", branch),
+ }).findings[0]!.source_finding_id;
+ expect(
+ new Set([
+ occurrence(77, ref),
+ occurrence(78, ref),
+ occurrence(77, "refs/tags/topic"),
+ ]).size,
+ ).toBe(3);
+ for (const invalid of [
+ "topic",
+ "refs/heads/main~1",
+ "refs/heads/.hidden",
+ "refs/heads/a.lock",
+ ]) {
+ expect(() => occurrence(77, invalid)).toThrow(/ref/i);
+ }
+});
+
+test("equivalent numeric IDs replay as integers while textual IDs stay exact", () => {
+ const select = (token: string, wanted = "8", project = "77") =>
+ normalizeSelectedSemgrepPlatformFindings({
+ ...source,
+ projectResponse: source.projectResponse.replace(
+ '"id":77',
+ '"id":' + project,
+ ),
+ findingIds: [wanted],
+ findingsResponses: page(token),
+ }).findings[0]!;
+ const original = select("8");
+ expect(select("8.0", "8", "77.0")).toEqual(original);
+ expect(select("8e0", "8", "77e0")).toEqual(original);
+ for (const token of ['"8.0"', '"8e0"', "8.00000000000000001"])
+ expect(() => select(token)).toThrow("missing");
+ expect(() => select("8", JSON.parse("[8.00000000000000001]")[0])).toThrow(
+ "must be nonempty text",
+ );
+ const int64Max = "9223372036854775807";
+ expect(select("9.223372036854775807e18", int64Max)).toEqual(
+ select(int64Max, int64Max),
+ );
+ const large = "12345678901234567890123";
+ expect(select(large + ".0", large)).toEqual(select(large, large));
+ expect(select(large + "e0", large)).toEqual(select(large, large));
+ expect(
+ select(large, large).evidence.source_data?.["semgrepPlatformFindingJson"],
+ ).toContain('"id":1.2345678901234567890123e+22,');
+ const textualProject = select("8", "8", JSON.stringify(large));
+ expect(textualProject.evidence.source_data?.["project"]).toMatchObject({
+ id: large,
+ });
+ expect(textualProject.source_finding_id).toBe(
+ select("8", "8", large).source_finding_id,
+ );
+ const power = "1000000000000000000000";
+ expect(select("1e21", power)).toEqual(select(power, power));
+ expect(() => select("1.0000000000000000000001e21", power)).toThrow("missing");
+});
+
+test("a URL-less row needs the matching project ID and full name, while an exact URL identifies the path of a short-named project", () => {
+ function select(
+ projectName: string,
+ repository: Record<string, unknown>,
+ projectUrl = url,
+ ) {
+ return normalizeSelectedSemgrepPlatformFindings({
+ ...source,
+ sourceRepositoryUrl: projectUrl,
+ projectResponse: JSON.stringify({
+ project: { id: 77, name: projectName, url: projectUrl },
+ }),
+ findingIds: ["8"],
+ findingsResponses: [
+ JSON.stringify({
+ findings: [{ id: 8, ref, repository, rule_name: "example.rule" }],
+ }),
+ ],
+ });
+ }
+ const name = "example/project";
+ expect(() =>
+ select("old-group/project", { id: 77, name: "old-group/project" }),
+ ).toThrow("project and ref");
+ expect(select(name, { id: 77, name }).findings).toHaveLength(1);
+ expect(select("project", { id: 77, name }).findings).toHaveLength(1);
+ expect(() => select("project", { id: 77, name: "project" })).toThrow(
+ "project and ref",
+ );
+ for (const repository of [
+ { name },
+ { id: 78, name },
+ { id: 77, name: "project" },
+ ]) {
+ expect(() => select(name, repository)).toThrow("project and ref");
+ }
+ expect(select("project", { url, name }).findings).toHaveLength(1);
+ expect(select("old-group/project", { url, name }).findings).toHaveLength(1);
+ expect(
+ select("group/old-name", { url, name: "project" }).findings,
+ ).toHaveLength(1);
+ const encoded = "https://github.com/example/caf%C3%A9";
+ expect(
+ select("café", { id: 77, name: "example/café" }, encoded).findings,
+ ).toHaveLength(1);
+ const literal = "https://scm.example.test/team/repo.git";
+ expect(() =>
+ select("team/repo.git", { id: 77, name: "team/repo" }, literal),
+ ).toThrow("project and ref");
+ expect(
+ select("team/repo.git", { id: 77, name: "team/repo.git" }, literal)
+ .findings,
+ ).toHaveLength(1);
+ const cloneUrl = url + ".git";
+ expect(
+ select("project", { url: cloneUrl, name }, cloneUrl).findings,
+ ).toHaveLength(1);
+ const historicalProject = "old-group/project/api";
+ const sibling = { url, name: "old-group/project/cli" };
+ expect(select(historicalProject, sibling).findings).toHaveLength(1);
+ expect(() => select(historicalProject, { ...sibling, id: 78 })).toThrow(
+ "project and ref",
+ );
+});
+
+test("a fractional source position cannot round to an integer during projection", () => {
+ for (const position of [
+ '"line":2.00000000000000001',
+ '"line":2,"end_line":2.00000000000000001',
+ ]) {
+ const result = normalizeSelectedSemgrepPlatformFindings({
+ ...source,
+ findingIds: ["8"],
+ findingsResponses: [page("8").replace('"line":2', position)],
+ });
+ expect(result.findings).toHaveLength(0);
+ expect(result.excluded).toHaveLength(1);
+ }
+});
+
+test("only repository-relative files are projected and original paths remain in evidence", () => {
+ const evidenceFor = (path: string) =>
+ normalizeSelectedSemgrepPlatformFindings({
+ ...source,
+ findingIds: ["8"],
+ findingsResponses: page("8").replace(
+ '"src/app.ts"',
+ JSON.stringify(path),
+ ),
+ }).findings[0]!.evidence;
+ for (const path of ["src/vendor://app.ts", "src/a\\b.ts", "src/a/b.ts"]) {
+ expect(evidenceFor(path).locations).toEqual([{ path, line: 2 }]);
+ }
+ for (const path of [
+ "file:src/app.ts",
+ "data:text/plain,hello",
+ "generated:/api.ts",
+ "./C:/work/app.ts",
+ "git@host:team/repo/app.ts",
+ "src\\..\\outside.ts",
+ ".",
+ "src/",
+ ]) {
+ const evidence = evidenceFor(path);
+ expect(evidence.locations).toEqual([]);
+ expect(evidence.source_data?.["semgrepPlatformFindingJson"]).toContain(
+ JSON.stringify(path),
+ );
+ }
+});