OpenAI's Codex Security CLI and TypeScript SDK for finding, validating, and fixing security vulnerabilities. npm: https://www.npmjs.com/package/@openai/codex-security
TypeScript ai-securityapplication-securityclicode-scanningcodexcodex-securitycybersecuritydevsecopsnodejsnpmopenaisdksecuritytypescriptvulnerability-scanning
feat(semgrep): import Code findings (#1690)
Michael D'Angelo00bfe9d4ecf2
diff --git a/docs/scanner-imports/semgrep.md b/docs/scanner-imports/semgrep.mdnew file mode 100644index 000000000..8f98ae958--- /dev/null+++ b/docs/scanner-imports/semgrep.md@@ -0,0 +1,123 @@+# Import Semgrep Code findings++Prefer native **Semgrep Code/SAST CLI or CI JSON** for this reader. It preserves+the scanner's original severity and source evidence. Self-contained SARIF 2.1.0+is a compatibility path. Both formats may be plain or gzip-compressed.++This reader does not accept Platform API, Guardian or MCP response envelopes,+or a general Supply Chain or Secrets feed. Those products and acquisition+contracts require separate mappings; installing a plugin does not change the+formats accepted here.++To produce JSON locally, run the following from the source repository root with+your selected SAST rules. This runs a new scan; it does not retrieve earlier+findings from Semgrep AppSec Platform.++```sh+semgrep scan --config /path/to/sast-rules.yaml --json-output semgrep.json .+codex-security publish findings semgrep.json --to local \+ --provider semgrep --source-key ORGANIZATION/PROJECT/BRANCH \+ --source-repository-url https://github.com/example/project \+ --repository . --dry-run --format json+```++Existing CI jobs can add `--json-output semgrep.json` to their SAST scan and pass+that artifact to the publisher. With authenticated `semgrep ci`, select `--code`;+its output follows the job's policy and PR/baseline scope and may contain only new+blocking findings. It is not a complete inventory of Platform findings.+[CLI output and scope reference](https://docs.semgrep.dev/cli-reference).++Review the findings, source repository and exclusions, then repeat the command+without `--dry-run` and confirm the import, or add `--yes` after review. Local+imports require no Cloud login and store unassessed evidence in the workbench+SQLite database. Read it with `codex-security findings imported . --format json`.+Repeated imports with the same source identity update the same records; omitted+records remain stored. See the movement behavior under Evidence and identity.+They do not create a completed scan or automatically assess or patch findings.++For Cloud, use the same report and source selection with a Cloud destination:++```sh+codex-security publish findings semgrep.json --to cloud \+ --provider semgrep --source-key ORGANIZATION/PROJECT/BRANCH \+ --source-repository-url https://github.com/example/project \+ --repository https://github.com/example/project --dry-run --format json+```++Keep the same source key for one organization, project and branch across runs.+`--source-repository-url` declares where the report came from; the importer checks+it against configured local Git remotes or the Cloud destination. Cloud requires+an authorized repository and support for the `semgrep` provider. Local SARIF+imports also check every original `versionControlProvenance.repositoryUri` on+accepted findings. Unknown Git hosts keep case-sensitive repository paths.++## Existing Platform findings++Semgrep's official+[Guardian Codex plugin](https://docs.semgrep.dev/semgrep-guardian/ide-setup/codex)+can select and inspect existing findings. Its output is not accepted by this+reader. A Platform importer needs a mapping qualified against the hosted tools'+actual responses, repository/ref filters and pagination using an authorized test+tenant. The separate local `semgrep mcp` server also exposes `semgrep_findings`,+but its current tool+returns a limited list without continuation metadata. It cannot establish a+complete export. [Official MCP implementation](https://github.com/semgrep/semgrep/blob/fe209763a4a2ccfcdf6434e6c332edc24a2f586e/cli/src/semgrep/mcp/server.py).++For an automated acquisition path, Semgrep's+[findings API](https://docs.semgrep.dev/semgrep-appsec-platform/semgrep-api) supports+pagination with a Web API token. Its Platform finding objects+need their own mapping into the shared publisher; they are not CLI `results`+objects. Do not rename an API/MCP response to `semgrep.json` or reconstruct a scan+report from a summary. No manual website export is required for either path.++## Evidence and identity++JSON preserves original severity, rule, CWE IDs, source range, snippets when+available and ignored state. SARIF retains its selected result and rule, including+trace and fix evidence. Its Critical and High ratings both become diagnostic+`error`, displayed here as High; prefer JSON when that distinction matters.+Original evidence stays in `source_data`, and source state stays separate from a+Codex assessment.++Both local and Cloud file imports currently apply the shared Cloud evidence+bounds, including a maximum JSON depth of 20. Deeply nested native taint traces+can exceed that bound and are excluded with a reason in the preview. The reader+preserves accepted source evidence without flattening or truncating traces.++Keep the supplied JSON `extra.fingerprint` or SARIF `matchBasedId/v1`+fingerprint. If a later report supplies the same fingerprint after the code+moves, importing it under the same repository and source key updates that record+with the new range; the raw fingerprint stays in `source_data`.++Community Edition may omit the fingerprint or emit `requires login`. The fallback+identity uses the rule, literal path and range. Moving such a finding creates a+second record, and the earlier record remains stored. The importer does not+infer that they are the same finding, merge them or delete the earlier record.+Read back both with `codex-security findings imported . --format json` before+selecting evidence for assessment. A current checkout does not establish the+report's revision or branch.++The adapter accepts SAST only. Select SAST rules or a Code-only CI scan, since+SARIF may omit enough product metadata to distinguish Secrets. Identifiable+Supply Chain and Secrets records are excluded. Reports with scan errors,+unsuccessful SARIF invocations or scan-error notifications are rejected. The+preview reports invalid findings and duplicate source identities before import.++SARIF supports direct artifact URIs and Semgrep's native `%SRCROOT%` marker without+a base definition. Native paths retain literal percent characters; direct URIs+are decoded once. Indexed artifacts and unresolved bases remain in original+evidence without mapped repository locations. CE records with these references+are excluded because they lack a usable identity. JSON scan-wide path inventories+and timing tables are omitted from individual findings. SARIF artifact inventories+are omitted when they are unrelated to the selected finding; required inventories+retain their original entries and indexes. Empty external-property reference+containers are accepted, while actual external file references are rejected.++SDK callers pass `{ sourceRepositoryUrl }` to `normalizeSemgrepFindings`,+`normalizeSemgrepSarif`, or `readSemgrepFindings`. When passing that normalized+output directly to `prepareNormalizedLocalImport`, supply the original SARIF+repository references in the third argument's `sourceRepositories`; the file+importer forwards them automatically. For an independently known GitHub custom+host, use `{ url, platform: "github" }` for its references. The Semgrep file reader+accepts native reports, not normalized JSON/JSONL exports.+See the [native output fields](https://docs.semgrep.dev/semgrep-appsec-platform/json-and-sarif).diff --git a/plugins/codex-security/skills/publish-findings/SKILL.md b/plugins/codex-security/skills/publish-findings/SKILL.mdindex 89c6de398..6a89b36fa 100644--- a/plugins/codex-security/skills/publish-findings/SKILL.md+++ b/plugins/codex-security/skills/publish-findings/SKILL.md@@ -1,6 +1,6 @@---name: publish-findings-description: Import selected Snyk Open Source findings or Wiz package vulnerabilities, SAST, repository secrets, and IaC findings locally or to Codex Security Cloud after previewing the destination and evidence. Prefers native JSON; also supports Snyk plugin SCA bundles and Open Source SARIF, Wiz normalized JSONL and repository package-vulnerability CSV, and gzip; does not import workload secrets or cloud configuration, run a scan, assess findings, or synchronize triage decisions.+description: Import selected Snyk Open Source findings, Semgrep Code CLI/CI reports, or Wiz package vulnerabilities, SAST, repository secrets, and IaC findings locally or to Codex Security Cloud after previewing the destination and evidence. Prefers native JSON; also supports Snyk plugin SCA bundles, Snyk Open Source and Semgrep Code SARIF, Wiz normalized JSONL and repository package-vulnerability CSV, and gzip; does not import workload secrets or cloud configuration, run a scan, assess findings, or synchronize triage decisions.---# Publish external findings@@ -55,6 +55,18 @@ Use one format consistently per repository, branch and scan target, with a stablRetain package versions, reported fixed versions, provenance and original evidence without claiming a fix was applied. Known repository claims must match the selected Cloud repository or, when available, a configured local Git remote. Resolve an unknown repository with the user; never infer that an export belongs to the current checkout. Follow [local imports](#local-imports) for preview, confirmation and readback or [Cloud publication](#review-and-publish-to-cloud) for upload authorization. All three Snyk formats support either destination.+## Semgrep Code++Prefer native Code/SAST CLI or CI JSON for this reader; use self-contained SARIF 2.1.0 as a compatibility path. JSON retains the original severity better than SARIF. The reader does not accept Platform API, Guardian or MCP response envelopes, or a general Supply Chain or Secrets feed. Select SAST rules or Code-only CI output; identifiable non-SAST records are excluded, but SARIF may omit the product metadata needed to distinguish Secrets.++For a user-requested new scan, use the installed Semgrep CLI from the source repository root with the selected SAST rules: `semgrep scan --config /path/to/sast-rules.yaml --json-output /path/to/semgrep.json .`. Existing CI jobs can produce the same artifact using `--json-output`; authenticated `semgrep ci --code` follows the job's policy and baseline scope. Explain that this generates fresh scan findings, and do not substitute it for a request to retrieve existing Platform findings.++For existing findings, the official [Semgrep Guardian Codex plugin](https://docs.semgrep.dev/semgrep-guardian/ide-setup/codex) can help select and inspect them. Importing its output needs a separate qualified response mapping and complete pagination for the selected repository/ref. The local `semgrep_findings` MCP tool and Platform REST API also use contracts distinct from CLI reports. Do not reshape a summary into a scan report or claim that a limited tool response is a complete export.++Import the native report with `--provider semgrep`, a stable organization/project/branch `--source-key`, and `--source-repository-url` identifying the actual source repository. Use `--to local --repository /path/to/project` for local storage or `--to cloud --repository URL_OR_ID` for Cloud. The source URL is checked against configured local Git remotes or the Cloud destination. Preview with `--dry-run --format json` before confirming the import.++Keep supplied fingerprints. Reimporting the same fingerprint after code moves updates the existing source record with the new range. If the fingerprint is absent or `requires login`, the fallback uses rule, literal path and range: movement creates a second retained record, and the earlier record remains stored. Do not infer a merge or deletion. Read the persisted evidence with `codex-security findings imported /path/to/project --format json` before assessment. Source rule, CWE, snippets, traces and status are vendor evidence, not Codex assessments. Do not infer a report revision from the current checkout. Resolve scan errors and review all exclusions before uploading; do not remove error metadata to force acceptance.+## Review and publish to CloudWhen publishing to staging or another Cloud deployment, set `CODEX_SECURITY_CLOUD_BASE_URL` to its API root before both preview and upload. If the Cloud page or user supplies its app URL, also set `CODEX_SECURITY_CLOUD_WEB_URL` to that full URL so the command can return a repository link. The UI setting does not change API routing or authentication. Use that deployment's login and an isolated state directory for tests. Verify `cloudApiUrl` and any `cloudUrl` in the preview against the intended deployment; the former is the API endpoint and the latter is the UI link. Saved retries are separated by deployment. The native scan setting `CODEX_SECURITY_CLOUD_PUBLISH_URL` does not route vendor imports. Never fall back to production when staging is unavailable.diff --git a/sdk/typescript/README.md b/sdk/typescript/README.mdindex 658615480..a88a739ee 100644--- a/sdk/typescript/README.md+++ b/sdk/typescript/README.md@@ -702,6 +702,8 @@ reports carry this platform in their repository inventory. If you export only thnormalized finding rows to JSONL, supply that metadata separately; the originalfinding evidence does not include the surrounding inventory.+See [Semgrep Code imports](https://github.com/openai/codex-security/blob/main/docs/scanner-imports/semgrep.md) for native CLI/CI JSON, SARIF compatibility, and local or Cloud imports. Platform API and Guardian/MCP responses need a separate mapping.+## Findings storage and deduplicationSaved-scan deduplication uses local SQLite directly:diff --git a/sdk/typescript/scripts/package-dist-files.mjs b/sdk/typescript/scripts/package-dist-files.mjsindex 832595b50..93f335eda 100644--- a/sdk/typescript/scripts/package-dist-files.mjs+++ b/sdk/typescript/scripts/package-dist-files.mjs@@ -29,6 +29,7 @@ const distModules = ["external-findings-read","scanner-import","snyk-findings",+ "semgrep-findings","external-findings-publish","external-import-contract","external-import-models",diff --git a/sdk/typescript/src/cli.ts b/sdk/typescript/src/cli.tsindex 0b68cf016..7334ba6e3 100644--- a/sdk/typescript/src/cli.ts+++ b/sdk/typescript/src/cli.ts@@ -3531,15 +3531,15 @@ export async function main(});publication.command("findings", {description:- "Preview and import selected Wiz or Snyk findings locally or to Cloud.",- hint: "For Snyk Open Source SCA vulnerabilities only, prefer native snyk test JSON or an official plugin snyk-findings-export/v1 bundle; Open Source SARIF 2.1.0 is also supported. Snyk licenses, Code, Container, IaC and platform backlogs are unsupported. For Wiz, prefer a native package vulnerability JSON report or named SAST/repository-secret/IaC JSON collections with repository metadata. Normalized JSONL and repository package-vulnerability CSV (SCA only, GitHub repositories) are also supported. Gzip is supported for each format; Wiz CSV does not cover SAST or cloud posture. Other Wiz CSV profiles, scan events, workload secrets, and cloud configuration exports are not supported. --to local stores unassessed evidence for a local repository path without Cloud login. --full-output requires --format json or --format jsonl.\nExample: codex-security publish findings selected-wiz.json --to local --repository ./project --provider wiz --source-key TENANT_ID/vulnerability-finding --dry-run --format json",+ "Preview and import selected Wiz, Snyk, or Semgrep Code findings locally or to Cloud.",+ hint: "For Snyk Open Source SCA vulnerabilities only, prefer native snyk test JSON or an official plugin snyk-findings-export/v1 bundle; Open Source SARIF 2.1.0 is also supported. Snyk licenses, Code, Container, IaC and platform backlogs are unsupported. For Semgrep Code, supply native JSON or SARIF and --source-repository-url. For Wiz, prefer a native package vulnerability JSON report or named SAST/repository-secret/IaC JSON collections with repository metadata. Normalized JSONL and repository package-vulnerability CSV (SCA only, GitHub repositories) are also supported. Gzip is supported for each format; Wiz CSV does not cover SAST or cloud posture. Other Wiz CSV profiles, scan events, workload secrets, and cloud configuration exports are not supported. --to local stores unassessed evidence for a local repository path without Cloud login. --full-output requires --format json or --format jsonl.\nExample: codex-security publish findings selected-wiz.json --to local --repository ./project --provider wiz --source-key TENANT_ID/vulnerability-finding --dry-run --format json",destructive: true,mcp: false,args: z.object({file: z.string().describe(- "Snyk native test JSON, plugin SCA bundle or Open Source SARIF; native Wiz JSON, normalized JSONL, or repository package-vulnerability CSV.",+ "Snyk native test JSON, plugin SCA bundle or Open Source SARIF; Semgrep Code JSON/SARIF; native Wiz JSON, normalized JSONL, or repository package-vulnerability CSV.",),}),options: z.object({@@ -3548,8 +3548,13 @@ export async function main("Local repository path for --to local; authorized repository ID or URL for --to cloud.",),provider: z- .enum(["wiz", "snyk"])+ .enum(["wiz", "snyk", "semgrep"]).describe("Vendor that produced these findings."),+ sourceRepositoryUrl: optionValue("--source-repository-url")+ .optional()+ .describe(+ "Required for Semgrep: source repository URL, checked against the Cloud destination or configured local Git remotes.",+ ),sourceKey: optionValue("--source-key").describe("Stable vendor tenant and finding namespace; not a project selection filter.",),diff --git a/sdk/typescript/src/external-findings-import.ts b/sdk/typescript/src/external-findings-import.tsindex 8fe8cc8ad..dd4391256 100644--- a/sdk/typescript/src/external-findings-import.ts+++ b/sdk/typescript/src/external-findings-import.ts@@ -1,4 +1,8 @@import type { ExternalPublicationOptions } from "./external-findings-publish.js";+import {+ cloudRepositoryUrlKey,+ repositoryUrlKey,+} from "./external-import-contract.js";import { readExternalFindings } from "./external-findings-read.js";import {prepareNormalizedLocalImport,@@ -17,11 +21,37 @@ export async function prepareLocalExternalImport(() => readExternalFindings(path, options),dependencies.signal,);+ let sourceRepositories = parsed.sourceRepositories ?? [];+ if (options.provider === "semgrep") {+ // The Cloud reader allows custom GitHub casing. Local provenance must also+ // agree with the caller's source URL when the checkout has no remote.+ const source = options.sourceRepositoryUrl!;+ const knownGitHub = dependencies.sourceRepositories?.some(+ ({ url, platform }) =>+ platform === "github" &&+ cloudRepositoryUrlKey(url) === cloudRepositoryUrlKey(source),+ );+ const key = knownGitHub ? cloudRepositoryUrlKey : repositoryUrlKey;+ for (const reference of sourceRepositories) {+ if (key(reference.url) !== key(source))+ throw new Error(+ "SARIF source repository does not match the declared source repository URL.",+ );+ }+ if (knownGitHub)+ sourceRepositories = [+ { url: source, platform: "github" },+ ...sourceRepositories.map((reference) => ({+ ...reference,+ platform: "github" as const,+ })),+ ];+ }return prepareNormalizedLocalImport(parsed.findings, options, {...dependencies,sourceRepositories: [...(dependencies.sourceRepositories ?? []),- ...(parsed.sourceRepositories ?? []),+ ...sourceRepositories,],});}diff --git a/sdk/typescript/src/external-findings-publish.ts b/sdk/typescript/src/external-findings-publish.tsindex fdf5aae6f..1efa09017 100644--- a/sdk/typescript/src/external-findings-publish.ts+++ b/sdk/typescript/src/external-findings-publish.ts@@ -62,6 +62,7 @@ export interface NormalizedExternalPublicationOptions {/** File readers accept the vendor formats installed in this SDK. */export interface ExternalPublicationOptions extends NormalizedExternalPublicationOptions {provider: ExternalFindingsReaderOptions["provider"];+ sourceRepositoryUrl?: string;}/** Optional transport, account, cancellation, and progress hooks. */diff --git a/sdk/typescript/src/external-findings-read.ts b/sdk/typescript/src/external-findings-read.tsindex 9233237e8..3b64918bb 100644--- a/sdk/typescript/src/external-findings-read.ts+++ b/sdk/typescript/src/external-findings-read.ts@@ -5,13 +5,15 @@ import type {} from "./external-findings.js";import { wizCsvRepositoryUrl } from "./wiz-csv.js";import { readSnykFindings, snykRepositoryUrls } from "./snyk-findings.js";+import { readSemgrepFindings } from "./semgrep-findings.js";import { isGitHubCloudHost } from "./git-remote.js";import { isRecord } from "./record.js";import { readVendorFindingsForPublication } from "./wiz-findings.js";/** File readers available in this SDK, shared by local and Cloud imports. */export interface ExternalFindingsReaderOptions {- provider: "wiz" | "snyk";+ provider: "wiz" | "snyk" | "semgrep";+ sourceRepositoryUrl?: string;}/** @internal Shared parser output, including immutable Cloud retry projections. */@@ -38,6 +40,27 @@ export async function readExternalFindings(),};}+ if (options.provider === "semgrep") {+ const findings = await readSemgrepFindings(path, {+ sourceRepositoryUrl: options.sourceRepositoryUrl ?? "",+ });+ const sourceRepositories = findings.findings.flatMap(({ evidence }) => {+ const context = evidence.source_data?.["runContext"];+ const references = isRecord(context)+ ? context["versionControlProvenance"]+ : undefined;+ return Array.isArray(references)+ ? references.flatMap((reference) =>+ isRecord(reference) &&+ typeof reference["repositoryUri"] === "string" &&+ reference["repositoryUri"].trim()+ ? [{ url: reference["repositoryUri"] }]+ : [],+ )+ : [];+ });+ return { findings, sourceRepositories };+ }const publication = await readVendorFindingsForPublication(path);// Raw package exports keep repository-branch URLs in retained source data.const sourceRepositories = publication.current.findings.flatMap(diff --git a/sdk/typescript/src/external-import-contract.ts b/sdk/typescript/src/external-import-contract.tsindex 8df3bcc24..4cb105895 100644--- a/sdk/typescript/src/external-import-contract.ts+++ b/sdk/typescript/src/external-import-contract.ts@@ -34,7 +34,9 @@ const STRING_BYTE_LIMITS: Readonly<Record<string, number>> = {* Keep protocols and explicit ports distinct, matching the Cloud URL contract. */export function repositoryUrlKey(value: string): string {const normalized = value.replace(/\/$/u, "").replace(/\.git$/u, "");- const parts = /^(https?):\/\/([^/?#]+)([^?#]*)(.*)$/iu.exec(normalized);+ const parts = /^(https?):\/\/([^/?#]+)((?:\/[^?#]*)?)((?:[?#].*)?)$/iu.exec(+ normalized,+ );if (!parts) return normalized;const authority = parts[2]!.toLowerCase();const hostname = authority.replace(/:\d+$/u, "");diff --git a/sdk/typescript/src/index.ts b/sdk/typescript/src/index.tsindex afe56e6cb..ca670f3ae 100644--- a/sdk/typescript/src/index.ts+++ b/sdk/typescript/src/index.ts@@ -318,3 +318,9 @@ export {normalizeSnykSarif,readSnykFindings,} from "./snyk-findings.js";+export {+ normalizeSemgrepFindings,+ normalizeSemgrepSarif,+ readSemgrepFindings,+ type SemgrepImportOptions,+} from "./semgrep-findings.js";diff --git a/sdk/typescript/src/semgrep-findings.ts b/sdk/typescript/src/semgrep-findings.tsnew file mode 100644index 000000000..205428a3a--- /dev/null+++ b/sdk/typescript/src/semgrep-findings.ts@@ -0,0 +1,466 @@+import {+ cloudRepositoryUrlKey,+ validateExternalEvidence,+} from "./external-import-contract.js";+import type { ExternalFindings } from "./external-findings.js";+import {+ array,+ hasSarifExternalPropertyReferences,+ normalizeRecords,+ readScannerJson,+ record,+ relativePath,+ requiredText,+ sarifRunContextForResult,+ stableId,+ strings,+ text,+} from "./scanner-import.js";++export interface SemgrepImportOptions {+ /** Source repository declared by the caller; never inferred from the destination. */+ sourceRepositoryUrl: string;+}++function sourceRepository(options: SemgrepImportOptions): string {+ const value = requiredText(+ options?.sourceRepositoryUrl,+ "Semgrep sourceRepositoryUrl (--source-repository-url)",+ );+ const url = new URL(value);+ if (+ !["http:", "https:"].includes(url.protocol) ||+ url.username ||+ url.password+ )+ throw new Error(+ "Use an HTTP(S) source repository URL without credentials.",+ );+ return value;+}++function ungatedText(value: unknown): string | null {+ const supplied = text(value);+ // The current CLI emits this placeholder for gated Community Edition fields.+ return supplied === "requires login" ? null : supplied;+}++function messageText(value: unknown): string | null {+ if (typeof value !== "string")+ throw new Error("Expected a Semgrep message string.");+ return text(value);+}++function severity(value: unknown): string {+ const levels: Record<string, string> = {+ CRITICAL: "critical",+ HIGH: "high",+ MEDIUM: "medium",+ LOW: "low",+ ERROR: "high",+ WARNING: "medium",+ INFO: "low",+ };+ return typeof value === "string" ? (levels[value] ?? "unknown") : "unknown";+}++function weaknessIds(value: unknown): string[] {+ const labels = typeof value === "string" ? [value] : strings(value);+ return [+ ...new Set(+ labels.flatMap((label) => {+ const match = /^(CWE-\d+)(?::|$)/u.exec(label);+ return match ? [match[1]!] : [];+ }),+ ),+ ];+}++function position(value: unknown, label: string): number {+ if (typeof value !== "number" || !Number.isSafeInteger(value) || value < 1)+ throw new Error(`${label} must be a positive integer.`);+ return value;+}++function identity(+ rule: string,+ path: string,+ line: number,+ column: number,+ endLine: number,+ endColumn: number,+): string {+ // CE omits fingerprints and source snippets. Its fallback is stable only while the range is stable.+ return stableId("semgrep-location", [+ rule,+ path,+ line,+ column,+ endLine,+ endColumn,+ ]);+}++function jsonIdentity(value: unknown): string {+ const result = record(value, "Semgrep result");+ const extra = record(result["extra"], "Semgrep extra");+ const supplied = ungatedText(extra["fingerprint"]);+ if (supplied) return stableId("semgrep", [supplied]);+ const start = record(result["start"], "Semgrep start");+ const end = record(result["end"], "Semgrep end");+ return identity(+ requiredText(result["check_id"], "Semgrep check_id"),+ requiredText(result["path"], "Semgrep path"),+ position(start["line"], "Start line"),+ position(start["col"], "Start column"),+ position(end["line"], "End line"),+ position(end["col"], "End column"),+ );+}++function sarifArtifactUri(+ artifact: Record<string, unknown>,+ runContext: Record<string, unknown>,+): string | null {+ const base = artifact["uriBaseId"];+ // Native Semgrep uses %SRCROOT% without defining a base. Keep other references raw.+ if (+ artifact["index"] !== undefined ||+ (base !== undefined &&+ (base !== "%SRCROOT%" ||+ record(runContext["originalUriBaseIds"] ?? {}, "SARIF URI bases")[+ base+ ] !== undefined))+ )+ return null;+ const uri = requiredText(artifact["uri"], "Semgrep artifact URI");+ // Semgrep's native formatter copies filesystem paths without URI encoding.+ return base === "%SRCROOT%" ? uri : decodeURIComponent(uri);+}++function sarifIdentity(+ value: unknown,+ runContext: Record<string, unknown>,+): string {+ const result = record(value, "Semgrep result");+ const supplied = ungatedText(+ record(result["fingerprints"] ?? {}, "Semgrep fingerprints")[+ "matchBasedId/v1"+ ],+ );+ if (supplied) return stableId("semgrep", [supplied]);+ const locations = array(result["locations"] ?? [], "Semgrep locations");+ if (locations.length !== 1)+ throw new Error("Expected Semgrep's single primary result location.");+ const physical = record(+ record(locations[0], "Location")["physicalLocation"],+ "Physical location",+ );+ const uri = sarifArtifactUri(+ record(physical["artifactLocation"], "Artifact location"),+ runContext,+ );+ if (uri === null)+ throw new Error(+ "Cannot derive a Semgrep CE identity from an indexed or unresolved base-relative artifact URI.",+ );+ const region = record(physical["region"], "Semgrep region");+ const line = position(region["startLine"], "Start line");+ const column = position(region["startColumn"] ?? 1, "Start column");+ return identity(+ requiredText(result["ruleId"], "Semgrep ruleId"),+ uri,+ line,+ column,+ position(region["endLine"] ?? line, "End line"),+ position(region["endColumn"] ?? column, "End column"),+ );+}++export function normalizeSemgrepFindings(+ input: unknown,+ options: SemgrepImportOptions,+): ExternalFindings {+ const document = record(input, "Semgrep report");+ if (document["version"] === "2.1.0" && document["runs"] !== undefined)+ return normalizeSemgrepSarif(document, options);+ const repositoryUrl = sourceRepository(options);+ requiredText(document["version"], "Semgrep version");+ if (array(document["errors"] ?? [], "Semgrep errors").length)+ throw new Error(+ "The Semgrep report contains scan errors; supply a successful report before importing findings.",+ );+ const { results, paths: _paths, time: _time, ...context } = document;+ return normalizeRecords(+ array(results, "Semgrep results"),+ (value, sourceFindingId) => {+ const result = record(value, "Semgrep result");+ const extra = record(result["extra"], "Semgrep extra");+ const metadata = record(extra["metadata"] ?? {}, "Semgrep metadata");+ const product = metadata["product"];+ const validation = extra["validation_state"];+ if (+ (product != null && product !== "sast") ||+ extra["sca_info"] != null ||+ extra["historical_info"] != null ||+ (validation != null && validation !== "NO_VALIDATOR")+ )+ throw new Error(+ "This adapter supports Semgrep SAST; Supply Chain and Secrets reports need separate qualification.",+ );+ const rule = requiredText(result["check_id"], "Semgrep check_id");+ const rawPath = requiredText(result["path"], "Semgrep path");+ const start = record(result["start"], "Semgrep start");+ const end = record(result["end"], "Semgrep end");+ const line = position(start["line"], "Start line");+ const column = position(start["col"], "Start column");+ const endLine = position(end["line"], "End line");+ const endColumn = position(end["col"], "End column");+ if (endLine < line || (endLine === line && endColumn < column))+ throw new Error("Semgrep source range ends before it starts.");+ const path = relativePath(rawPath);+ return {+ source_finding_id: sourceFindingId,+ evidence: validateExternalEvidence({+ title: rule,+ description: messageText(extra["message"]),+ severity: severity(extra["severity"]),+ locations: path ? [{ path, line }] : [],+ details: {+ kind: "sast",+ repository: { url: repositoryUrl },+ scanner_origin: "Semgrep",+ rule: { id: rule },+ weakness_ids: weaknessIds(metadata["cwe"]),+ code: {+ end_line: path ? endLine : null,+ snippet: ungatedText(extra["lines"]),+ },+ source_status:+ typeof extra["is_ignored"] === "boolean"+ ? extra["is_ignored"]+ ? "ignored"+ : "open"+ : null,+ },+ source_data: structuredClone({ report: context, result }),+ }),+ };+ },+ jsonIdentity,+ );+}++export function normalizeSemgrepSarif(+ input: unknown,+ options: SemgrepImportOptions,+): ExternalFindings {+ const repositoryUrl = sourceRepository(options);+ const document = record(input, "Semgrep SARIF");+ if (document["version"] !== "2.1.0")+ throw new Error("Expected Semgrep SARIF 2.1.0.");+ const entries = array(document["runs"], "SARIF runs").flatMap((value) => {+ const run = record(value, "SARIF run");+ for (const entry of array(+ run["versionControlProvenance"] ?? [],+ "Version control provenance",+ )) {+ const declared = text(+ record(entry, "Repository provenance")["repositoryUri"],+ );+ if (+ declared &&+ cloudRepositoryUrlKey(declared) !== cloudRepositoryUrlKey(repositoryUrl)+ )+ throw new Error(+ "SARIF source repository does not match the declared source repository URL.",+ );+ }+ const tool = record(run["tool"], "SARIF tool");+ const driver = record(tool["driver"], "SARIF driver");+ if (+ !["semgrep", "semgrep oss", "semgrep pro"].includes(+ requiredText(driver["name"], "SARIF producer").toLowerCase(),+ )+ )+ throw new Error("Expected a Semgrep SARIF producer.");+ if (+ hasSarifExternalPropertyReferences(run["externalPropertyFileReferences"])+ )+ throw new Error("Use a self-contained Semgrep SARIF report.");+ for (const invocation of array(+ run["invocations"] ?? [],+ "Semgrep invocations",+ )) {+ const details = record(invocation, "Semgrep invocation");+ if (details["executionSuccessful"] === false)+ throw new Error("Semgrep SARIF contains an unsuccessful invocation.");+ if (+ array(+ details["toolExecutionNotifications"] ?? [],+ "Semgrep scan notifications",+ ).length+ )+ throw new Error("Semgrep SARIF contains scan error notifications.");+ }+ const rules = array(driver["rules"] ?? [], "Semgrep rules").map((r) =>+ record(r, "Semgrep rule"),+ );+ const rulesById = new Map<+ unknown,+ { rule: Record<string, unknown>; index: number } | null+ >();+ for (const [index, rule] of rules.entries()) {+ const id = rule["id"];+ rulesById.set(id, rulesById.has(id) ? null : { rule, index });+ }+ const { runs: _runs, ...documentContext } = document;+ const { results: _results, tool: _tool, ...runContext } = run;+ const { rules: _rules, ...driverContext } = driver;+ const { driver: _driver, ...toolContext } = tool;+ return array(run["results"] ?? [], "Semgrep results").map((result) => ({+ result,+ rulesById,+ hasRules: rules.length > 0,+ documentContext,+ runContext,+ driverContext,+ toolContext,+ }));+ });+ return normalizeRecords(+ entries,+ ({ result: value, rulesById, hasRules, ...context }, sourceFindingId) => {+ const result = record(value, "Semgrep result");+ const properties = record(+ result["properties"] ?? {},+ "Semgrep result properties",+ );+ if (+ properties["exposure"] != null ||+ properties["dependencyPaths"] != null+ )+ throw new Error(+ "This adapter supports Semgrep SAST; Supply Chain reports need separate qualification.",+ );+ const ruleId = requiredText(result["ruleId"], "Semgrep ruleId");+ const selectedRule = rulesById.get(ruleId);+ if (selectedRule === null || (hasRules && selectedRule === undefined))+ throw new Error(+ "Semgrep result must reference exactly one rule in its run.",+ );+ const rule = selectedRule?.rule;+ const ruleIndex = selectedRule?.index;+ if (+ result["ruleIndex"] !== undefined &&+ result["ruleIndex"] !== ruleIndex+ )+ throw new Error("Semgrep ruleIndex and ruleId disagree.");+ if (result["rule"] !== undefined) {+ const ref = record(result["rule"], "Semgrep rule reference");+ if (+ ref["toolComponent"] !== undefined ||+ (ref["id"] !== undefined && ref["id"] !== ruleId) ||+ (ref["index"] !== undefined && ref["index"] !== ruleIndex) ||+ (ref["guid"] !== undefined && ref["guid"] !== rule?.["guid"])+ )+ throw new Error("Inconsistent or external Semgrep rule reference.");+ }+ const locations = array(result["locations"] ?? [], "Semgrep locations");+ if (locations.length !== 1)+ throw new Error("Expected Semgrep's single primary result location.");+ const physical = record(+ record(locations[0], "Location")["physicalLocation"],+ "Physical location",+ );+ const artifact = record(+ physical["artifactLocation"],+ "Artifact location",+ );+ const rawPath = sarifArtifactUri(artifact, context.runContext);+ const region = record(physical["region"], "Semgrep region");+ const line = position(region["startLine"], "Start line");+ const column = position(region["startColumn"] ?? 1, "Start column");+ const endLine = position(region["endLine"] ?? line, "End line");+ const endColumn = position(region["endColumn"] ?? column, "End column");+ if (endLine < line || (endLine === line && endColumn < column))+ throw new Error("Semgrep source range ends before it starts.");+ const message = record(result["message"], "Semgrep message");+ const configuration = record(+ rule?.["defaultConfiguration"] ?? {},+ "Rule configuration",+ );+ const level = text(result["level"]) ?? text(configuration["level"]);+ // SARIF collapses severity categories; retain its diagnostic level and all original metadata.+ const mappedSeverity =+ level === "error"+ ? "high"+ : level === "warning"+ ? "medium"+ : level === "note"+ ? "low"+ : "unknown";+ // Native %SRCROOT% values are filesystem paths; direct URIs can name other targets.+ const absoluteUri =+ artifact["uriBaseId"] !== "%SRCROOT%" &&+ typeof artifact["uri"] === "string" &&+ /^[a-z][a-z\d+.-]*:/iu.test(artifact["uri"]);+ const path =+ rawPath === null || absoluteUri ? null : relativePath(rawPath);+ return {+ source_finding_id: sourceFindingId,+ evidence: validateExternalEvidence({+ title: ruleId,+ description:+ text(message["markdown"]) ?? messageText(message["text"]),+ severity: mappedSeverity,+ locations: path ? [{ path, line }] : [],+ details: {+ kind: "sast",+ repository: { url: repositoryUrl },+ scanner_origin: "Semgrep",+ rule: { id: ruleId },+ weakness_ids: weaknessIds(+ record(rule?.["properties"] ?? {}, "Rule properties")["tags"],+ ),+ code: {+ end_line: path ? endLine : null,+ snippet: ungatedText(+ record(region["snippet"] ?? {}, "Snippet")["text"],+ ),+ },+ source_status:+ Array.isArray(result["suppressions"]) &&+ result["suppressions"].some((value) => {+ const status = record(value, "Suppression")["status"];+ return status === undefined || status === "accepted";+ })+ ? "suppressed"+ : null,+ },+ source_data: structuredClone({+ ...context,+ runContext: sarifRunContextForResult(+ context.runContext,+ result,+ rule,+ context.driverContext,+ context.toolContext,+ context.documentContext,+ ),+ ...(rule ? { rule } : {}),+ result,+ }),+ }),+ };+ },+ ({ result, runContext }) => sarifIdentity(result, runContext),+ );+}++export async function readSemgrepFindings(+ path: string,+ options: SemgrepImportOptions,+): Promise<ExternalFindings> {+ return normalizeSemgrepFindings(await readScannerJson(path), options);+}diff --git a/sdk/typescript/tests-ts/external-findings-publish.test.ts b/sdk/typescript/tests-ts/external-findings-publish.test.tsindex f47a7f7d9..8a2043f29 100644--- a/sdk/typescript/tests-ts/external-findings-publish.test.ts+++ b/sdk/typescript/tests-ts/external-findings-publish.test.ts@@ -3280,3 +3280,190 @@ test("native Snyk JSON publishes selected dependency occurrences and replays the).publish();expect(second.counts.unchanged).toBe(1);});++test("Semgrep publication preserves identity across JSON and SARIF with explicit repository provenance", async () => {+ const { semgrepJson, semgrepSarif } =+ await import("./support/semgrep-import-fixtures.js");+ const f = await fixture(semgrepJson());+ const settings = {+ ...options,+ provider: "semgrep" as const,+ sourceKey: "organization/project/main",+ sourceRepositoryUrl: "https://github.com/example/project",+ };+ const first = await (+ await prepareExternalPublication(f.file, settings, f.deps)+ ).publish();+ expect(first.counts.created).toBe(1);+ expect(first.verified).toBe(1);+ const original = structuredClone([...f.reports.values()][0]!);+ expect(original.source.provider).toBe("semgrep");+ expect(original.evidence.details?.kind).toBe("sast");+ expect(+ (+ await (+ await prepareExternalPublication(f.file, settings, f.deps)+ ).publish()+ ).counts.unchanged,+ ).toBe(1);+ await writeFile(f.file, JSON.stringify(semgrepSarif()));+ const changed = await (+ await prepareExternalPublication(f.file, settings, f.deps)+ ).publish();+ expect(changed.counts.updated).toBe(1);+ const updated = [...f.reports.values()][0]!;+ expect(updated.id).toBe(original.id);+ expect(updated.canonical_finding_id).toBe(original.canonical_finding_id);+ expect(updated.evidence.source_data).toMatchObject({+ result: semgrepSarif().runs[0]!.results[0],+ });+});++test("Semgrep native percent paths retain CE records through JSON-to-SARIF publication", async () => {+ const { semgrepJson, semgrepSarif } =+ await import("./support/semgrep-import-fixtures.js");+ const paths = [+ "src/my file.ts",+ "src/my%20file.ts",+ "src/100%.ts",+ "src/100%25.ts",+ ];+ const json: any = semgrepJson();+ const sarif: any = semgrepSarif();+ delete json.results[0].extra.fingerprint;+ delete sarif.runs[0].results[0].fingerprints;+ json.results = paths.map((path) => ({ ...json.results[0], path }));+ const run = sarif.runs[0];+ run.results = paths.map((path) => {+ const result = structuredClone(run.results[0]);+ result.locations[0].physicalLocation.artifactLocation.uri = path;+ return result;+ });+ const f = await fixture(json);+ const settings = {+ ...options,+ provider: "semgrep" as const,+ sourceKey: "organization/project/main",+ sourceRepositoryUrl: "https://github.com/example/project",+ };+ const first = await (+ await prepareExternalPublication(f.file, settings, f.deps)+ ).publish();+ expect(first.counts.created).toBe(paths.length);+ expect(first.verified).toBe(paths.length);+ const originalIds = [...f.reports.values()]+ .map((report) => [report.id, report.canonical_finding_id])+ .sort();+ await writeFile(f.file, JSON.stringify(sarif));+ const changed = await (+ await prepareExternalPublication(f.file, settings, f.deps)+ ).publish();+ expect(changed.counts.created).toBe(0);+ expect(changed.counts.updated).toBe(paths.length);+ expect(changed.verified).toBe(paths.length);+ expect(+ [...f.reports.values()]+ .map((report) => [report.id, report.canonical_finding_id])+ .sort(),+ ).toEqual(originalIds);+ expect(+ [...f.reports.values()]+ .map((report) => report.evidence.locations?.[0]?.path)+ .sort(),+ ).toEqual(paths.toSorted());+ expect(+ (+ await (+ await prepareExternalPublication(f.file, settings, f.deps)+ ).publish()+ ).counts.unchanged,+ ).toBe(paths.length);+});++test("Semgrep publication retains unsupported SARIF bases without repository locations", async () => {+ const { semgrepSarif } = await import("./support/semgrep-import-fixtures.js");+ const input: any = semgrepSarif();+ const run = input.runs[0];+ run.results[0].locations[0].physicalLocation.artifactLocation.uriBaseId =+ "ROOT";+ run.originalUriBaseIds = { ROOT: { uri: "file:///outside/" } };+ const f = await fixture(input);+ const settings = {+ ...options,+ provider: "semgrep" as const,+ sourceKey: "organization/project/main",+ sourceRepositoryUrl: "https://github.com/example/project",+ };+ const first = await (+ await prepareExternalPublication(f.file, settings, f.deps)+ ).publish();+ expect(first.counts.created).toBe(1);+ expect(first.verified).toBe(1);+ const original = structuredClone([...f.reports.values()][0]!);+ expect(original.evidence.locations).toEqual([]);+ expect(original.evidence.source_data).toMatchObject({+ result: run.results[0],+ runContext: { originalUriBaseIds: run.originalUriBaseIds },+ });+ expect(JSON.parse(f.posts[0]!).items[0].evidence.locations).toEqual([]);++ run.originalUriBaseIds.ROOT.uri = "subproject/";+ await writeFile(f.file, JSON.stringify(input));+ const changed = await (+ await prepareExternalPublication(f.file, settings, f.deps)+ ).publish();+ expect(changed.counts.updated).toBe(1);+ const updated = [...f.reports.values()][0]!;+ expect(updated.id).toBe(original.id);+ expect(updated.canonical_finding_id).toBe(original.canonical_finding_id);+ expect(updated.evidence.locations).toEqual([]);+ expect(updated.evidence.source_data).toMatchObject({+ runContext: { originalUriBaseIds: run.originalUriBaseIds },+ });+ expect(+ (+ await (+ await prepareExternalPublication(f.file, settings, f.deps)+ ).publish()+ ).counts.unchanged,+ ).toBe(1);+});++test("Semgrep refuses missing source identity and a conflicting Cloud destination without uploads", async () => {+ const { semgrepJson } = await import("./support/semgrep-import-fixtures.js");+ const f = await fixture(semgrepJson());+ await expect(+ prepareExternalPublication(+ f.file,+ { ...options, provider: "semgrep" },+ f.deps,+ ),+ ).rejects.toThrow("sourceRepositoryUrl");+ await expect(+ prepareExternalPublication(+ f.file,+ {+ ...options,+ provider: "semgrep",+ sourceRepositoryUrl: "https://github.com/other/project",+ },+ f.deps,+ ),+ ).rejects.toThrow(/repository/i);+ expect(f.posts).toHaveLength(0);+});++test("Semgrep CLI dry-run preserves explicit source repository and does not upload", async () => {+ const { semgrepJson } = await import("./support/semgrep-import-fixtures.js");+ const f = await fixture(semgrepJson());+ const command = [+ ...f.command,+ "--dry-run",+ "--source-repository-url",+ "https://github.com/example/project",+ ];+ command[command.indexOf("--provider") + 1] = "semgrep";+ const cli = createCliTest(main);+ expect(await cli.runCli(command, f.cliDeps)).toBe(0);+ expect(f.posts).toHaveLength(0);+});diff --git a/sdk/typescript/tests-ts/semgrep-findings-local.test.ts b/sdk/typescript/tests-ts/semgrep-findings-local.test.tsnew file mode 100644index 000000000..c3559c9cd--- /dev/null+++ b/sdk/typescript/tests-ts/semgrep-findings-local.test.ts@@ -0,0 +1,564 @@+import { access, readFile, writeFile } from "node:fs/promises";+import { gzipSync } from "node:zlib";+import { join } from "node:path";+import { expect, test } from "bun:test";+import { prepareLocalExternalImport } from "../src/external-findings-import.js";+import {+ listLocalExternalFindings,+ prepareNormalizedLocalImport,+} from "../src/external-findings-local.js";+import {+ normalizeSemgrepSarif,+ readSemgrepFindings,+} from "../src/semgrep-findings.js";+import { localFindingsFixture } from "./external-findings-local-helpers.js";+import {+ semgrepJson,+ semgrepSarif,+} from "./support/semgrep-import-fixtures.js";++const repositoryUrl = "https://github.com/example/project";+const formats = ["JSON", "SARIF"] as const;++function report(+ format: (typeof formats)[number],+ message = "Synthetic finding",+ sparse = false,+) {+ if (format === "JSON") {+ const input = semgrepJson();+ const result = input.results[0]!;+ result.extra.message = message;+ if (sparse) {+ result.path = "../outside.ts";+ result.extra.severity = "EXPERIMENT";+ result.extra.lines = "requires login";+ }+ return input;+ }+ const input = semgrepSarif();+ const run = input.runs[0]!;+ const result = run.results[0]!;+ result.message.text = message;+ if (sparse) {+ result.locations[0]!.physicalLocation.artifactLocation.uri =+ "../outside.ts";+ run.tool.driver.rules[0]!.defaultConfiguration.level = "none";+ }+ return input;+}++function reportAtLine(+ format: (typeof formats)[number],+ line: number,+ fingerprint: string,+) {+ if (format === "JSON") {+ const input = semgrepJson();+ const result = input.results[0]!;+ result.start.line = line;+ result.end.line = line;+ result.extra.fingerprint = fingerprint;+ return input;+ }+ const input = semgrepSarif();+ const result = input.runs[0]!.results[0]!;+ const region = result.locations[0]!.physicalLocation.region;+ region.startLine = line;+ region.endLine = line;+ result.fingerprints["matchBasedId/v1"] = fingerprint;+ return input;+}++test("Semgrep file reader rejects malformed SARIF repository provenance", async () => {+ const fixture = await localFindingsFixture();+ try {+ const file = join(fixture.root, "semgrep.sarif");+ const input = semgrepSarif();+ input.runs = input.runs.map((run) => ({+ ...run,+ versionControlProvenance: [{ repositoryUri: 'http://""?\nX' }],+ }));+ await writeFile(file, JSON.stringify(input));+ await expect(+ readSemgrepFindings(file, { sourceRepositoryUrl: repositoryUrl }),+ ).rejects.toThrow(+ "SARIF source repository does not match the declared source repository URL.",+ );+ } finally {+ await fixture.dispose();+ }+});++for (const suppliedFingerprint of [true, false]) {+ test.each(formats)(+ `Semgrep %s movement ${suppliedFingerprint ? "updates the supplied fingerprint" : "retains both fallback records"} in SQLite`,+ async (format) => {+ const fixture = await localFindingsFixture({ remote: repositoryUrl });+ try {+ const file = join(fixture.root, "semgrep.json");+ const options = {+ repository: fixture.repository,+ provider: "semgrep" as const,+ sourceKey: "example-project:main",+ sourceRepositoryUrl: repositoryUrl,+ };+ const dependencies = { environment: fixture.environment };+ const fingerprint = suppliedFingerprint+ ? "synthetic-fingerprint"+ : "requires login";+ async function importLine(line: number) {+ await writeFile(+ file,+ JSON.stringify(reportAtLine(format, line, fingerprint)),+ );+ return (+ await prepareLocalExternalImport(file, options, dependencies)+ ).import();+ }+ expect((await importLine(3)).counts).toEqual({+ created: 1,+ updated: 0,+ unchanged: 0,+ });+ const [original] = await listLocalExternalFindings(+ options,+ dependencies,+ );+ expect((await importLine(9)).counts).toEqual(+ suppliedFingerprint+ ? { created: 0, updated: 1, unchanged: 0 }+ : { created: 1, updated: 0, unchanged: 0 },+ );+ const records = await listLocalExternalFindings(options, dependencies);+ const moved = records.find(+ (record) => record.evidence.locations?.[0]?.line === 9,+ );+ expect(moved).toBeDefined();+ expect(moved!.evidence.locations).toEqual([+ { path: "src/example.ts", line: 9 },+ ]);+ expect(moved!.evidence.source_data?.["result"]).toMatchObject(+ format === "JSON"+ ? { extra: { fingerprint } }+ : { fingerprints: { "matchBasedId/v1": fingerprint } },+ );+ expect(moved!.assessment).toEqual({ state: "not_assessed" });+ if (suppliedFingerprint) {+ expect(records).toHaveLength(1);+ expect(moved).toMatchObject({+ id: original!.id,+ sourceFindingId: original!.sourceFindingId,+ createdAt: original!.createdAt,+ version: 2,+ });+ } else {+ expect(records).toHaveLength(2);+ expect(records.find((record) => record.id === original!.id)).toEqual(+ original,+ );+ expect(moved!.sourceFindingId).not.toBe(original!.sourceFindingId);+ expect(moved!.version).toBe(1);+ }+ expect((await importLine(9)).counts).toEqual({+ created: 0,+ updated: 0,+ unchanged: 1,+ });+ expect(await listLocalExternalFindings(options, dependencies)).toEqual(+ records,+ );+ } finally {+ await fixture.dispose();+ }+ },+ );+}++test.each(formats)(+ "Semgrep %s imports into SQLite and updates the same source finding",+ async (format) => {+ const fixture = await localFindingsFixture({ remote: repositoryUrl });+ try {+ const file = join(fixture.root, "semgrep.json");+ const options = {+ repository: fixture.repository,+ provider: "semgrep" as const,+ sourceKey: "example-project:main",+ sourceRepositoryUrl: repositoryUrl,+ };+ const dependencies = { environment: fixture.environment };+ await writeFile(file, JSON.stringify(report(format)));+ const prepared = await prepareLocalExternalImport(+ file,+ options,+ dependencies,+ );+ expect(prepared.preview.findings).toHaveLength(1);+ expect(prepared.preview.excluded).toEqual([]);+ const imported = await prepared.import();+ expect(imported.counts).toEqual({ created: 1, updated: 0, unchanged: 0 });++ const [stored] = await listLocalExternalFindings(options, dependencies);+ expect(stored).toMatchObject({+ provider: "semgrep",+ sourceKey: options.sourceKey,+ sourceFindingId: prepared.preview.findings[0]!.source_finding_id,+ evidence: prepared.preview.findings[0]!.evidence,+ assessment: { state: "not_assessed" },+ version: 1,+ });+ expect(stored!.evidence.locations).toEqual([+ { path: "src/example.ts", line: 3 },+ ]);+ expect(stored!.evidence).not.toHaveProperty("confidence");++ const repeated = await prepareLocalExternalImport(+ file,+ options,+ dependencies,+ );+ expect((await repeated.import()).counts).toEqual({+ created: 0,+ updated: 0,+ unchanged: 1,+ });+ expect(await listLocalExternalFindings(options, dependencies)).toEqual([+ stored!,+ ]);++ await writeFile(+ file,+ JSON.stringify(report(format, "Updated scanner evidence")),+ );+ const changed = await prepareLocalExternalImport(+ file,+ options,+ dependencies,+ );+ expect((await changed.import()).counts).toEqual({+ created: 0,+ updated: 1,+ unchanged: 0,+ });+ const records = await listLocalExternalFindings(options, dependencies);+ expect(records).toHaveLength(1);+ expect(records[0]).toMatchObject({+ id: stored!.id,+ sourceFindingId: stored!.sourceFindingId,+ evidence: changed.preview.findings[0]!.evidence,+ assessment: { state: "not_assessed" },+ version: 2,+ createdAt: stored!.createdAt,+ });+ expect(records[0]!.evidence.description).toBe("Updated scanner evidence");+ } finally {+ await fixture.dispose();+ }+ },+);++test.each(formats)(+ "Semgrep %s preserves sparse evidence without inventing a repository location or assessment",+ async (format) => {+ const fixture = await localFindingsFixture({ remote: repositoryUrl });+ try {+ const file = join(fixture.root, "semgrep.json");+ await writeFile(file, JSON.stringify(report(format, "", true)));+ const options = {+ repository: fixture.repository,+ provider: "semgrep" as const,+ sourceKey: "example-project:main",+ sourceRepositoryUrl: repositoryUrl,+ };+ const dependencies = { environment: fixture.environment };+ const prepared = await prepareLocalExternalImport(+ file,+ options,+ dependencies,+ );+ expect((await prepared.import()).counts.created).toBe(1);+ const [stored] = await listLocalExternalFindings(options, dependencies);+ expect(stored!.evidence).toEqual(prepared.preview.findings[0]!.evidence);+ expect(stored!.evidence).toMatchObject({+ description: null,+ severity: "unknown",+ locations: [],+ code_revision: null,+ details: { code: { end_line: null, snippet: null } },+ });+ expect(stored!.evidence.source_data).toHaveProperty("result");+ expect(stored!.evidence).not.toHaveProperty("confidence");+ expect(stored!.evidence).not.toHaveProperty("remediation");+ expect(stored!.assessment).toEqual({ state: "not_assessed" });+ } finally {+ await fixture.dispose();+ }+ },+);++test.each(formats)(+ "Semgrep %s rejects a source repository that conflicts with the selected checkout",+ async (format) => {+ const fixture = await localFindingsFixture({ remote: repositoryUrl });+ try {+ const file = join(fixture.root, "semgrep.json");+ await writeFile(file, JSON.stringify(report(format)));+ const dependencies = { environment: fixture.environment };+ await expect(+ prepareLocalExternalImport(+ file,+ {+ repository: fixture.repository,+ provider: "semgrep",+ sourceKey: "example-project:main",+ sourceRepositoryUrl: "https://github.com/other/project",+ },+ dependencies,+ ),+ ).rejects.toThrow("does not match");+ expect(+ await listLocalExternalFindings(+ { repository: fixture.repository },+ dependencies,+ ),+ ).toEqual([]);+ } finally {+ await fixture.dispose();+ }+ },+);++function sarifWithRepositories(urls: string[]) {+ const input = semgrepSarif();+ return {+ ...input,+ runs: [+ {+ ...input.runs[0]!,+ versionControlProvenance: urls.map((repositoryUri) => ({+ repositoryUri,+ })),+ },+ ],+ };+}++for (const format of ["SARIF", "SARIF gzip"]) {+ test.each([+ ["unknown host", "git.example.test", "with remote"],+ ["GitHub", "github.com", "with remote"],+ ["known GitHub custom host", "git.example.test", "with remote"],+ ["GitHub", "github.com", "with uppercase remote"],+ ["known GitHub custom host", "git.example.test", "with uppercase remote"],+ ["unknown host", "git.example.test", "without remote"],+ ["GitHub", "github.com", "without remote"],+ ["known GitHub custom host", "git.example.test", "without remote"],+ ])(+ `Semgrep ${format} preserves original repository references on %s (%s, %s)`,+ async (platform, host, checkout) => {+ const sourceRepositoryUrl = `https://${host}/team/project`;+ const originalUrl = `https://${host}/Team/Project`;+ const fixture = await localFindingsFixture(+ checkout === "without remote"+ ? {}+ : {+ remote: `git@${host}:${checkout === "with uppercase remote" ? "Team/Project" : "team/project"}.git`,+ },+ );+ try {+ const file = join(fixture.root, "semgrep-report");+ const options = {+ repository: fixture.repository,+ provider: "semgrep" as const,+ sourceKey: "example-project:main",+ sourceRepositoryUrl,+ };+ const dependencies = {+ environment: fixture.environment,+ ...(platform === "known GitHub custom host"+ ? {+ sourceRepositories: [+ { url: originalUrl, platform: "github" as const },+ ],+ }+ : {}),+ };+ const database = join(fixture.stateDirectory, "workbench.sqlite3");+ async function writeReport(url: string) {+ // Keep both refs: checking only the first would miss the case-distinct one.+ const input = sarifWithRepositories([sourceRepositoryUrl, url]);+ const contents = JSON.stringify(input);+ await writeFile(+ file,+ format.endsWith("gzip") ? gzipSync(contents) : contents,+ );+ return input;+ }+ await writeReport(originalUrl);+ if (platform === "unknown host") {+ await expect(+ prepareLocalExternalImport(file, options, dependencies),+ ).rejects.toThrow("does not match");+ await expect(access(database)).rejects.toThrow();+ await writeReport(sourceRepositoryUrl);+ }+ const parsed = await readSemgrepFindings(file, { sourceRepositoryUrl });+ const prepared = await prepareLocalExternalImport(+ file,+ options,+ dependencies,+ );+ expect(prepared.preview.findings).toEqual(parsed.findings);+ expect((await prepared.import()).counts).toEqual({+ created: 1,+ updated: 0,+ unchanged: 0,+ });+ const stored = await listLocalExternalFindings(options, dependencies);+ expect(stored[0]!.evidence).toEqual(parsed.findings[0]!.evidence);+ expect(stored[0]!.evidence.details?.repository?.url).toBe(+ sourceRepositoryUrl,+ );+ expect(stored[0]!.evidence.source_data?.["runContext"]).toMatchObject({+ versionControlProvenance: [+ { repositoryUri: sourceRepositoryUrl },+ {+ repositoryUri:+ platform === "unknown host" ? sourceRepositoryUrl : originalUrl,+ },+ ],+ });+ expect(+ (+ await (+ await prepareLocalExternalImport(file, options, dependencies)+ ).import()+ ).counts,+ ).toEqual({ created: 0, updated: 0, unchanged: 1 });+ const before = await readFile(database);+ await writeReport(+ platform === "unknown host"+ ? originalUrl+ : `https://${host}/other/project`,+ );+ await expect(+ prepareLocalExternalImport(file, options, dependencies),+ ).rejects.toThrow("does not match");+ expect(await readFile(database)).toEqual(before);+ expect(await listLocalExternalFindings(options, dependencies)).toEqual(+ stored,+ );+ } finally {+ await fixture.dispose();+ }+ },+ );+}++test("normalized Semgrep SDK imports retain original SARIF repository metadata through the explicit dependency", async () => {+ const sourceRepositoryUrl = "https://git.example.test/team/project";+ const originalUrl = "https://git.example.test/Team/Project";+ const fixture = await localFindingsFixture({ remote: sourceRepositoryUrl });+ try {+ const options = {+ repository: fixture.repository,+ provider: "semgrep" as const,+ sourceKey: "example-project:main",+ };+ const parsed = normalizeSemgrepSarif(sarifWithRepositories([originalUrl]), {+ sourceRepositoryUrl,+ });+ const saved = JSON.parse(JSON.stringify(parsed));+ await expect(+ prepareNormalizedLocalImport(saved, options, {+ environment: fixture.environment,+ sourceRepositories: [{ url: originalUrl }],+ }),+ ).rejects.toThrow("does not match");+ await expect(+ access(join(fixture.stateDirectory, "workbench.sqlite3")),+ ).rejects.toThrow();+ const dependencies = {+ environment: fixture.environment,+ sourceRepositories: [{ url: originalUrl, platform: "github" as const }],+ };+ const prepared = await prepareNormalizedLocalImport(+ saved,+ options,+ dependencies,+ );+ expect((await prepared.import()).counts.created).toBe(1);+ expect(+ (await listLocalExternalFindings(options, dependencies))[0]!.evidence,+ ).toEqual(parsed.findings[0]!.evidence);+ } finally {+ await fixture.dispose();+ }+});++test.each(["JSON", "JSON gzip"])(+ "Semgrep %s uses the caller source URL without treating rule metadata as repository provenance",+ async (format) => {+ const sourceRepositoryUrl = "https://git.example.test/Team/Project";+ const fixture = await localFindingsFixture({+ remote: "[email protected]:Team/Project.git",+ });+ try {+ const input = semgrepJson();+ const first = input.results[0]!;+ const report = {+ ...input,+ results: [+ {+ ...first,+ extra: {+ ...first.extra,+ metadata: {+ ...first.extra.metadata,+ source: "https://rules.example.test/Other/Repository",+ },+ },+ },+ ],+ };+ const file = join(fixture.root, "semgrep-report");+ const contents = JSON.stringify(report);+ await writeFile(+ file,+ format.endsWith("gzip") ? gzipSync(contents) : contents,+ );+ const options = {+ repository: fixture.repository,+ provider: "semgrep" as const,+ sourceKey: "example-project:main",+ sourceRepositoryUrl,+ };+ const dependencies = { environment: fixture.environment };+ const parsed = await readSemgrepFindings(file, { sourceRepositoryUrl });+ const prepared = await prepareLocalExternalImport(+ file,+ options,+ dependencies,+ );+ expect((await prepared.import()).counts.created).toBe(1);+ const stored = await listLocalExternalFindings(options, dependencies);+ expect(stored[0]!.evidence).toEqual(parsed.findings[0]!.evidence);+ await expect(+ prepareLocalExternalImport(+ file,+ {+ ...options,+ sourceRepositoryUrl: "https://git.example.test/team/project",+ },+ dependencies,+ ),+ ).rejects.toThrow("does not match");+ expect(await listLocalExternalFindings(options, dependencies)).toEqual(+ stored,+ );+ } finally {+ await fixture.dispose();+ }+ },+);diff --git a/sdk/typescript/tests-ts/semgrep-findings.test.ts b/sdk/typescript/tests-ts/semgrep-findings.test.tsnew file mode 100644index 000000000..8e970550a--- /dev/null+++ b/sdk/typescript/tests-ts/semgrep-findings.test.ts@@ -0,0 +1,922 @@+import { expect, test } from "bun:test";+import {+ normalizeSemgrepFindings as normalizeJson,+ normalizeSemgrepSarif as normalizeSarif,+} from "../src/semgrep-findings.js";+import {+ semgrepJson,+ semgrepSarif,+} from "./support/semgrep-import-fixtures.js";++test("Semgrep JSON preserves source severity, findings, source assertions and report metadata", () => {+ const input = semgrepJson();+ const original = structuredClone(input);+ const output = normalizeSemgrepFindings(input);+ expect(input).toEqual(original);+ expect(output.excluded).toEqual([]);+ expect(output.findings[0]!.evidence).toMatchObject({+ title: "example.synthetic-rule",+ severity: "critical",+ locations: [{ path: "src/example.ts", line: 3 }],+ source_data: {+ result: original.results[0],+ report: { version: original.version, errors: [] },+ },+ });+ expect(output.findings[0]!.evidence).not.toHaveProperty("assessment");+});++test("platform fingerprints survive line, severity and rule text updates", () => {+ const input = semgrepJson();+ const first = normalizeSemgrepFindings(input).findings[0]!;+ input.results[0]!.start.line = 9;+ input.results[0]!.end.line = 9;+ input.results[0]!.extra.severity = "LOW";+ input.results[0]!.extra.message = "Updated source message";+ const second = normalizeSemgrepFindings(input).findings[0]!;+ expect(second.source_finding_id).toBe(first.source_finding_id);+ expect(second.evidence).not.toEqual(first.evidence);+ expect(+ normalizeSemgrepSarif(semgrepSarif()).findings[0]!.source_finding_id,+ ).toBe(first.source_finding_id);+});++test("Community Edition fallback is stable for identical ranges and distinct for moved findings", () => {+ const input: any = semgrepJson();+ delete input.results[0].extra.fingerprint;+ const first = normalizeSemgrepFindings(input).findings[0]!;+ input.results[0].extra.message = "New message";+ expect(normalizeSemgrepFindings(input).findings[0]!.source_finding_id).toBe(+ first.source_finding_id,+ );+ input.results[0].start.line++;+ input.results[0].end.line++;+ expect(+ normalizeSemgrepFindings(input).findings[0]!.source_finding_id,+ ).not.toBe(first.source_finding_id);+});++test.each([+ ["ERROR", "high"],+ ["WARNING", "medium"],+ ["INFO", "low"],+ ["CRITICAL", "critical"],+ ["HIGH", "high"],+ ["MEDIUM", "medium"],+ ["LOW", "low"],+ ["EXPERIMENT", "unknown"],+] as const)("maps severity %s to %s", (inputLevel, expected) => {+ const input = semgrepJson();+ input.results[0]!.extra.severity = inputLevel!;+ expect(normalizeSemgrepFindings(input).findings[0]!.evidence.severity).toBe(+ expected,+ );+});++test("SARIF retains complete selected rule, suppressions and dataflow", () => {+ const input = semgrepSarif();+ const output = normalizeSemgrepFindings(input);+ expect(output.excluded).toEqual([]);+ expect(output.findings[0]!.evidence).toMatchObject({+ severity: "high",+ source_data: {+ result: input.runs[0]!.results[0],+ rule: input.runs[0]!.tool.driver.rules[0],+ },+ });+});++test.each([+ [[], null],+ [[undefined], "suppressed"],+ [["accepted"], "suppressed"],+ [["rejected"], null],+ [["underReview"], null],+ [["rejected", "underReview"], null],+ [["rejected", "accepted"], "suppressed"],+] as const)(+ "SARIF suppression statuses %p preserve the source decision",+ (statuses, expected) => {+ const input: any = semgrepSarif();+ const originalId =+ normalizeSemgrepFindings(input).findings[0]!.source_finding_id;+ input.runs[0].results[0].suppressions = statuses.map((status) => ({+ kind: "inSource",+ ...(status === undefined ? {} : { status }),+ }));+ const original = structuredClone(input);+ const result = normalizeSemgrepFindings(input);+ expect(result.excluded).toEqual([]);+ expect(result.findings[0]!.source_finding_id).toBe(originalId);+ expect(result.findings[0]!.evidence.details).toMatchObject({+ source_status: expected,+ });+ expect(result.findings[0]!.evidence.source_data).toMatchObject({+ result: original.runs[0].results[0],+ });+ expect(result.findings[0]!.evidence).not.toHaveProperty("assessment");+ expect(input).toEqual(original);+ },+);++test("failed scans and unrelated SARIF producers fail before publishing partial results", () => {+ const input: any = semgrepJson();+ input.errors = [{ message: "Scan incomplete" }];+ expect(() => normalizeSemgrepFindings(input)).toThrow("scan errors");+ const sarif = semgrepSarif();+ sarif.runs[0]!.invocations[0]!.executionSuccessful = false;+ expect(() => normalizeSemgrepFindings(sarif)).toThrow("unsuccessful");+ sarif.runs[0]!.tool.driver.name = "other";+ expect(() => normalizeSemgrepFindings(sarif)).toThrow(+ "Semgrep SARIF producer",+ );+});++test.each([+ ["empty object", {}],+ ["empty reference arrays", { artifacts: [], results: [], invocations: [] }],+ ["metadata only", { properties: { description: "Synthetic metadata" } }],+] as const)(+ "accepts self-contained SARIF external references: %s",+ (_name, refs) => {+ const input: any = semgrepSarif();+ input.runs[0].externalPropertyFileReferences = refs;+ const original = structuredClone(input);+ const output = normalizeSemgrepSarif(input);+ expect(output.excluded).toEqual([]);+ expect(output.findings).toHaveLength(1);+ expect(output.findings[0]!.evidence.source_data).toMatchObject({+ runContext: { externalPropertyFileReferences: refs },+ result: original.runs[0].results[0],+ });+ expect(input).toEqual(original);+ },+);++test.each([+ ["artifact file", { artifacts: [{ location: { uri: "external.sarif" } }] }],+ [+ "result GUID",+ { results: [{ guid: "00000000-0000-4000-8000-000000000001" }] },+ ],+ ["driver file", { driver: { location: { uri: "external.sarif" } } }],+] as const)("rejects SARIF external references: %s", (_name, refs) => {+ const input: any = semgrepSarif();+ input.runs[0].externalPropertyFileReferences = refs;+ expect(() => normalizeSemgrepSarif(input)).toThrow("self-contained");+});++test("a small SARIF finding does not copy an unrelated run artifact inventory", () => {+ const input: any = semgrepSarif();+ input.runs[0].artifacts = Array.from({ length: 5000 }, (_, index) => ({+ location: { uri: `unrelated/source-${index}.ts`, uriBaseId: "%SRCROOT%" },+ length: 100,+ sourceLanguage: "typescript",+ }));+ const original = structuredClone(input);+ const output = normalizeSemgrepSarif(input);+ expect(output.excluded).toEqual([]);+ expect(output.findings).toHaveLength(1);+ expect(output.findings[0]!.evidence.source_data).toMatchObject({+ result: original.runs[0].results[0],+ rule: original.runs[0].tool.driver.rules[0],+ });+ expect(+ output.findings[0]!.evidence.source_data?.["runContext"],+ ).not.toHaveProperty("artifacts");+ expect(input).toEqual(original);+});++test.each(["primary URI", "trace index", "tool index", "invocation index"])(+ "SARIF preserves artifact inventory needed by the %s",+ (reference) => {+ const input: any = semgrepSarif();+ const run = input.runs[0];+ run.artifacts = [+ { location: { uri: "unrelated.ts", uriBaseId: "%SRCROOT%" } },+ {+ location: { uri: "src/example.ts", uriBaseId: "%SRCROOT%" },+ contents: { text: "Synthetic source context" },+ },+ ];+ if (reference !== "primary URI") {+ run.artifacts[1].location.uri = "trace/source.ts";+ if (reference === "trace index")+ run.results[0].codeFlows = [+ {+ threadFlows: [+ {+ locations: [+ {+ location: {+ physicalLocation: { artifactLocation: { index: 1 } },+ },+ },+ ],+ },+ ],+ },+ ];+ else if (reference === "tool index")+ run.tool.driver.locations = [{ index: 1 }];+ else run.invocations[0].workingDirectory = { index: 1 };+ }+ const original = structuredClone(input);+ const output = normalizeSemgrepSarif(input);+ expect(output.excluded).toEqual([]);+ expect(output.findings).toHaveLength(1);+ expect(output.findings[0]!.evidence.source_data).toMatchObject({+ result: original.runs[0].results[0],+ runContext: { artifacts: original.runs[0].artifacts },+ });+ expect(input).toEqual(original);+ },+);++test("all duplicate fingerprints are excluded even when one result is invalid", () => {+ const input = semgrepJson();+ input.results.push(structuredClone(input.results[0]!));+ input.results[1]!.extra.message = "x".repeat(65537);+ const output = normalizeSemgrepFindings(input);+ expect(output.findings).toEqual([]);+ expect(output.excluded).toHaveLength(2);+ const sarif: any = semgrepSarif();+ sarif.runs[0].results.push(structuredClone(sarif.runs[0].results[0]));+ delete sarif.runs[0].results[1].message;+ expect(normalizeSemgrepFindings(sarif).findings).toEqual([]);+});++test("unknown paths stay raw and invalid ranges are excluded", () => {+ const input = semgrepJson();+ input.results[0]!.path = "../outside.ts";+ expect(+ normalizeSemgrepFindings(input).findings[0]!.evidence.locations,+ ).toEqual([]);+ input.results[0]!.start.line = 0;+ expect(normalizeSemgrepFindings(input).findings).toEqual([]);+});++test("500 distinct fingerprints survive reordering", () => {+ const input = semgrepJson();+ const original = input.results[0]!;+ input.results = Array.from({ length: 500 }, (_, i) => ({+ ...structuredClone(original),+ extra: { ...original.extra, fingerprint: `synthetic-${i}` },+ }));+ const first = normalizeSemgrepFindings(input);+ input.results.reverse();+ const second = normalizeSemgrepFindings(input);+ expect(first.findings).toHaveLength(500);+ expect(second.excluded).toEqual([]);+ expect(first.findings.map((f) => f.source_finding_id).sort()).toEqual(+ second.findings.map((f) => f.source_finding_id).sort(),+ );+});++const options = { sourceRepositoryUrl: "https://github.com/example/project" };+const normalizeSemgrepFindings = (input: unknown) =>+ normalizeJson(input, options);+const normalizeSemgrepSarif = (input: unknown) =>+ normalizeSarif(input, options);++test("requires source identity and rejects conflicting SARIF repository provenance", () => {+ expect(() =>+ normalizeJson(semgrepJson(), { sourceRepositoryUrl: "" }),+ ).toThrow("sourceRepositoryUrl");+ const input: any = semgrepSarif();+ input.runs[0].versionControlProvenance = [+ { repositoryUri: "https://github.com/other/project" },+ ];+ expect(() => normalizeSemgrepFindings(input)).toThrow("does not match");+});++test.each([true, false])(+ "direct SARIF URIs decode once and match JSON identity (fingerprint=%p)",+ (fingerprint) => {+ for (const path of ["src/my file.ts", "src/my%20file.ts"]) {+ const json: any = semgrepJson();+ const sarif: any = semgrepSarif();+ json.results[0].path = path;+ const artifact =+ sarif.runs[0].results[0].locations[0].physicalLocation.artifactLocation;+ delete artifact.uriBaseId;+ artifact.uri = path.split("/").map(encodeURIComponent).join("/");+ if (!fingerprint) {+ delete json.results[0].extra.fingerprint;+ delete sarif.runs[0].results[0].fingerprints;+ }+ const original = structuredClone(sarif);+ const finding = normalizeSemgrepSarif(sarif).findings[0]!;+ expect(finding.evidence.locations).toEqual([{ path, line: 3 }]);+ expect(finding.source_finding_id).toBe(+ normalizeSemgrepFindings(json).findings[0]!.source_finding_id,+ );+ expect(finding.evidence.source_data).toMatchObject({+ result: original.runs[0].results[0],+ });+ expect(sarif).toEqual(original);+ }+ },+);++test.each([true, false])(+ "native SARIF preserves literal percent paths (fingerprint=%p)",+ (fingerprint) => {+ for (const path of [+ "src/my file.ts",+ "src/my%20file.ts",+ "src/100%.ts",+ "src/%2Ffile.ts",+ "src/literal%2e%2e.ts",+ "src/caf%C3%A9.ts",+ "src/café.ts",+ ]) {+ const json: any = semgrepJson();+ const sarif: any = semgrepSarif();+ json.results[0].path = path;+ sarif.runs[0].results[0].locations[0].physicalLocation.artifactLocation.uri =+ path;+ if (!fingerprint) {+ delete json.results[0].extra.fingerprint;+ delete sarif.runs[0].results[0].fingerprints;+ }+ const original = structuredClone(sarif);+ const output = normalizeSemgrepSarif(sarif);+ expect(output.excluded).toEqual([]);+ expect(output.findings).toHaveLength(1);+ const finding = output.findings[0]!;+ expect(finding.evidence.locations).toEqual([{ path, line: 3 }]);+ expect(finding.source_finding_id).toBe(+ normalizeSemgrepFindings(json).findings[0]!.source_finding_id,+ );+ expect(finding.evidence.source_data).toMatchObject({+ result: original.runs[0].results[0],+ });+ expect(sarif).toEqual(original);+ }+ },+);++test("native SARIF keeps CE percent paths distinct after reordering", () => {+ const paths = [+ "src/my file.ts",+ "src/my%20file.ts",+ "src/100%.ts",+ "src/100%25.ts",+ ];+ const json: any = semgrepJson();+ const sarif: any = semgrepSarif();+ delete json.results[0].extra.fingerprint;+ delete sarif.runs[0].results[0].fingerprints;+ json.results = paths.map((path) => ({ ...json.results[0], path }));+ const run = sarif.runs[0];+ run.results = paths.map((path) => {+ const result = structuredClone(run.results[0]);+ result.locations[0].physicalLocation.artifactLocation.uri = path;+ return result;+ });+ const expected = normalizeSemgrepFindings(json).findings;+ expect(expected).toHaveLength(paths.length);+ for (const results of [run.results, run.results.toReversed()]) {+ run.results = results;+ const output = normalizeSemgrepSarif(sarif);+ expect(output.excluded).toEqual([]);+ expect(output.findings).toHaveLength(paths.length);+ expect(output.findings.map((x) => x.source_finding_id).sort()).toEqual(+ expected.map((x) => x.source_finding_id).sort(),+ );+ expect(+ output.findings.map((x) => x.evidence.locations?.[0]?.path).sort(),+ ).toEqual(paths.toSorted());+ }+});++test.each([+ "%2e%2e/outside.ts",+ "%2Foutside.ts",+ "C%3A%5Coutside.ts",+ "src/%00.ts",+])(+ "SARIF retains encoded unsafe URI %s without a repository location",+ (uri) => {+ const input: any = semgrepSarif();+ const artifact =+ input.runs[0].results[0].locations[0].physicalLocation.artifactLocation;+ delete artifact.uriBaseId;+ artifact.uri = uri;+ const output = normalizeSemgrepSarif(input);+ expect(output.excluded).toEqual([]);+ expect(output.findings[0]!.evidence.locations).toEqual([]);+ expect(output.findings[0]!.evidence.details?.code?.end_line).toBeNull();+ expect(output.findings[0]!.evidence.source_data).toMatchObject({+ result: input.runs[0]!.results[0],+ });+ },+);++test("SARIF matches custom GitHub Enterprise repository casing without merging ports or schemes", () => {+ const input: any = semgrepSarif();+ input.runs[0].versionControlProvenance = [+ { repositoryUri: "https://git.example.test/Example/Project.git" },+ ];+ expect(+ normalizeSarif(input, {+ sourceRepositoryUrl: "https://GIT.EXAMPLE.TEST/example/project/",+ }).findings,+ ).toHaveLength(1);+ for (const sourceRepositoryUrl of [+ "http://git.example.test/example/project",+ "https://git.example.test:443/example/project",+ "https://git.example.test/other/project",+ ]) {+ expect(() => normalizeSarif(input, { sourceRepositoryUrl })).toThrow(+ "does not match",+ );+ }+});++test("SARIF repository binding preserves query, fragment and malformed suffixes", () => {+ const input: any = semgrepSarif();+ const repositoryUri = "HTTPS://GITHUB.COM/Example/Project?ref=Main#Finding";+ input.runs[0].versionControlProvenance = [{ repositoryUri }];+ const sourceRepositoryUrl =+ "https://github.com/example/project?ref=Main#Finding";+ const output = normalizeSarif(input, { sourceRepositoryUrl });+ expect(output.findings).toHaveLength(1);+ expect(output.findings[0]!.evidence.details?.repository?.url).toBe(+ sourceRepositoryUrl,+ );+ expect(output.findings[0]!.evidence.source_data).toMatchObject({+ runContext: { versionControlProvenance: [{ repositoryUri }] },+ });+ for (const suffix of [+ "",+ "?ref=Other#Finding",+ "?ref=Main#Other",+ ...["\n", "\r", "\u2028", "\u2029"].map(+ (newline) => `?ref=${newline}Main#Finding`,+ ),+ ]) {+ input.runs[0].versionControlProvenance[0].repositoryUri = `https://github.com/example/project${suffix}`;+ expect(() => normalizeSarif(input, { sourceRepositoryUrl })).toThrow(+ "does not match",+ );+ }+});++test.each([+ [+ "external base",+ { uri: "src/example.ts", uriBaseId: "ROOT" },+ { ROOT: { uri: "file:///outside/" } },+ ],+ [+ "subdirectory base",+ { uri: "src/example.ts", uriBaseId: "PACKAGE" },+ { PACKAGE: { uri: "project-a/", uriBaseId: "%SRCROOT%" } },+ ],+ [+ "redefined source root",+ { uri: "src/example.ts", uriBaseId: "%SRCROOT%" },+ { "%SRCROOT%": { uri: "project-a/", uriBaseId: "ROOT" } },+ ],+ [+ "explicit source-root URI",+ { uri: "src/example.ts", uriBaseId: "%SRCROOT%" },+ { "%SRCROOT%": { uri: "file:///repo/subproject/" } },+ ],+ ["indexed URI", { uri: "src/example.ts", index: 0 }, {}],+ ["index only", { index: 0 }, {}],+] as const)(+ "SARIF retains unsupported %s without inventing a path or CE identity",+ (_label, artifact, bases) => {+ const input: any = semgrepSarif();+ const expectedId =+ normalizeSemgrepSarif(input).findings[0]!.source_finding_id;+ const run = input.runs[0];+ run.originalUriBaseIds = structuredClone(bases);+ run.artifacts = [{ location: { uri: "nested/actual.ts" } }];+ run.results[0].locations[0].physicalLocation.artifactLocation =+ structuredClone(artifact);+ const original = structuredClone(input);+ const output = normalizeSemgrepSarif(input);+ expect(output.excluded).toEqual([]);+ expect(output.findings).toHaveLength(1);+ expect(output.findings[0]!.source_finding_id).toBe(expectedId);+ expect(output.findings[0]!.evidence.locations).toEqual([]);+ expect(output.findings[0]!.evidence.details?.code?.end_line).toBeNull();+ expect(output.findings[0]!.evidence.source_data).toMatchObject({+ result: original.runs[0].results[0],+ runContext: {+ originalUriBaseIds: bases,+ artifacts: original.runs[0].artifacts,+ },+ });+ expect(input).toEqual(original);++ delete run.results[0].fingerprints;+ const ce = normalizeSemgrepSarif(input);+ expect(ce.findings).toEqual([]);+ expect(ce.excluded).toHaveLength(1);+ expect(ce.excluded[0]!.source_finding_id).toBeNull();+ expect(ce.excluded[0]!.reason).toContain("indexed or unresolved");+ },+);++test("SARIF unresolved bases cannot collide with a valid CE location", () => {+ const input: any = semgrepSarif();+ const run = input.runs[0];+ delete run.results[0].fingerprints;+ const expected = normalizeSemgrepSarif(input).findings[0]!;+ const first = structuredClone(run.results[0]);+ const second = structuredClone(run.results[0]);+ first.locations[0].physicalLocation.artifactLocation.uriBaseId = "A";+ second.locations[0].physicalLocation.artifactLocation.uriBaseId = "B";+ run.originalUriBaseIds = {+ A: { uri: "project-a/", uriBaseId: "%SRCROOT%" },+ B: { uri: "project-b/", uriBaseId: "%SRCROOT%" },+ };+ run.results.unshift(first, second);+ for (const results of [run.results, run.results.toReversed()]) {+ run.results = results;+ const output = normalizeSemgrepSarif(input);+ expect(output.findings).toHaveLength(1);+ expect(output.findings[0]!.source_finding_id).toBe(+ expected.source_finding_id,+ );+ expect(output.findings[0]!.evidence.locations).toEqual(+ expected.evidence.locations,+ );+ expect(output.excluded).toHaveLength(2);+ for (const excluded of output.excluded) {+ expect(excluded.source_finding_id).toBeNull();+ expect(excluded.reason).toContain("indexed or unresolved");+ }+ }+});++test.each([undefined, "%SRCROOT%"])(+ "SARIF native root %s preserves CE identity across JSON and SARIF",+ (base) => {+ const json: any = semgrepJson();+ const input: any = semgrepSarif();+ delete json.results[0].extra.fingerprint;+ delete input.runs[0].results[0].fingerprints;+ input.runs[0].originalUriBaseIds = { OTHER: { uri: "file:///outside/" } };+ const artifact =+ input.runs[0].results[0].locations[0].physicalLocation.artifactLocation;+ if (base === undefined) delete artifact.uriBaseId;+ else artifact.uriBaseId = base;+ const output = normalizeSemgrepSarif(input);+ expect(output.excluded).toEqual([]);+ expect(output.findings[0]!.evidence.locations).toEqual([+ { path: "src/example.ts", line: 3 },+ ]);+ expect(output.findings[0]!.source_finding_id).toBe(+ normalizeSemgrepFindings(json).findings[0]!.source_finding_id,+ );+ },+);++test("gated CE fingerprint placeholders use independent location identities", () => {+ const input: any = semgrepSarif();+ delete input.runs[0].tool.driver.rules;+ const first = input.runs[0]!.results[0]!;+ first.fingerprints["matchBasedId/v1"] = "requires login";+ const second = structuredClone(first);+ second.locations[0]!.physicalLocation.region.startLine = 6;+ second.locations[0]!.physicalLocation.region.endLine = 6;+ input.runs[0]!.results.push(second);+ const output = normalizeSemgrepFindings(input);+ expect(output.findings).toHaveLength(2);+ expect(output.excluded).toEqual([]);+ expect(output.findings[0]!.source_finding_id).not.toBe(+ output.findings[1]!.source_finding_id,+ );+});++test("does not relabel Supply Chain or Secrets findings as SAST", () => {+ for (const extra of [+ { sca_info: { reachable: true } },+ { validation_state: "CONFIRMED_VALID" },+ ]) {+ const input: any = semgrepJson();+ Object.assign(input.results[0].extra, extra);+ const output = normalizeSemgrepFindings(input);+ expect(output.findings).toEqual([]);+ expect(output.excluded[0]!.reason).toContain("separate qualification");+ }+});++test.each([undefined, "NO_VALIDATOR"])(+ "excludes Secrets history without product metadata (validation=%p)",+ (validation) => {+ const input: any = semgrepJson();+ const extra = input.results[0].extra;+ extra.validation_state = validation;+ extra.historical_info = {+ git_commit: "a".repeat(40),+ git_blob: "b".repeat(40),+ git_commit_timestamp: "2026-01-01T00:00:00Z",+ };+ const original = structuredClone(input);+ const output = normalizeSemgrepFindings(input);+ expect(output.findings).toEqual([]);+ expect(output.excluded).toHaveLength(1);+ expect(output.excluded[0]!.reason).toContain("supports Semgrep SAST");+ expect(input).toEqual(original);++ extra.historical_info = null;+ const sast = normalizeSemgrepFindings(input);+ expect(sast.excluded).toEqual([]);+ expect(sast.findings).toHaveLength(1);+ expect(sast.findings[0]!.evidence.source_data?.["result"]).toEqual(+ input.results[0],+ );+ },+);++test.each(["Semgrep PRO", "semgrep"])(+ "accepts the native %s SARIF producer name",+ (producer) => {+ const input = semgrepSarif();+ input.runs[0]!.tool.driver.name = producer;+ const output = normalizeSemgrepSarif(input);+ expect(output.findings).toHaveLength(1);+ expect(output.excluded).toEqual([]);+ },+);++test.each(["omitted", "empty"])(+ "accepts CE results with %s rule descriptors",+ (descriptors) => {+ const input: any = semgrepSarif();+ const run = input.runs[0];+ if (descriptors === "omitted") delete run.tool.driver.rules;+ else run.tool.driver.rules = [];+ run.results[0].fingerprints["matchBasedId/v1"] = "requires login";+ run.results[0].level = "warning";+ const known = normalizeSemgrepSarif(input);+ expect(known.excluded).toEqual([]);+ expect(known.findings).toHaveLength(1);+ expect(known.findings[0]!.evidence.severity).toBe("medium");+ expect(known.findings[0]!.evidence.source_data).not.toHaveProperty("rule");+ delete run.results[0].level;+ const unknown = normalizeSemgrepSarif(input);+ expect(unknown.excluded).toEqual([]);+ expect(unknown.findings[0]!.evidence.severity).toBe("unknown");+ expect(unknown.findings[0]!.source_finding_id).toBe(+ known.findings[0]!.source_finding_id,+ );+ },+);++test("SARIF selects the exact rule ID and preserves its original index and GUID", () => {+ const input: any = semgrepSarif();+ const run = input.runs[0];+ const rule = {+ ...run.tool.driver.rules[0],+ id: " example.synthetic-rule ",+ guid: "synthetic-rule-guid",+ };+ run.tool.driver.rules.push(rule);+ Object.assign(run.results[0], {+ ruleId: rule.id,+ ruleIndex: 1,+ rule: { id: rule.id, index: 1, guid: rule.guid },+ });+ const output = normalizeSemgrepSarif(input);+ expect(output.excluded).toEqual([]);+ expect(output.findings).toHaveLength(1);+ expect(output.findings[0]!.evidence.details?.rule?.id).toBe(rule.id);+ expect(output.findings[0]!.evidence.source_data?.["rule"]).toEqual(rule);+});++test.each(["missing", "duplicate", "wrong index", "wrong GUID"])(+ "SARIF excludes a result with a %s rule reference",+ (kind) => {+ const input: any = semgrepSarif();+ const run = input.runs[0];+ const rule = run.tool.driver.rules[0];+ if (kind === "missing") rule.id = "another-rule";+ else if (kind === "duplicate")+ run.tool.driver.rules.push(structuredClone(rule));+ else if (kind === "wrong index") run.results[0].ruleIndex = 1;+ else {+ rule.guid = "synthetic-rule-guid";+ run.results[0].rule = { guid: "another-rule-guid" };+ }+ const output = normalizeSemgrepSarif(input);+ expect(output.findings).toEqual([]);+ expect(output.excluded).toHaveLength(1);+ },+);++test.each([+ { ruleIndex: 0 },+ { ruleIndex: -1 },+ { rule: { index: 0 } },+ { rule: { guid: "synthetic-rule-guid" } },+])(+ "rejects unresolved references when CE descriptors are absent (%p)",+ (reference) => {+ const input: any = semgrepSarif();+ delete input.runs[0].tool.driver.rules;+ Object.assign(input.runs[0].results[0], reference);+ const output = normalizeSemgrepSarif(input);+ expect(output.findings).toEqual([]);+ expect(output.excluded).toHaveLength(1);+ },+);++test.each([+ ["secrets", undefined],+ ["secrets", null],+ ["sca", undefined],+ ["sca", null],+ ["secrets", "NO_VALIDATOR"],+ ["sca", "NO_VALIDATOR"],+] as const)(+ "excludes explicit %s product without validator metadata (%p)",+ (product, validation) => {+ const input: any = semgrepJson();+ input.results[0].extra.metadata = { product };+ input.results[0].extra.validation_state = validation;+ const output = normalizeSemgrepFindings(input);+ expect(output.findings).toEqual([]);+ expect(output.excluded[0]!.reason).toContain("supports Semgrep SAST");+ },+);++test("accepts an explicit SAST product marker", () => {+ const input: any = semgrepJson();+ input.results[0].extra.metadata = { product: "sast" };+ expect(normalizeSemgrepFindings(input).findings).toHaveLength(1);+});++test.each(["error", "warning", "note", undefined])(+ "rejects native SARIF scan notifications even with executionSuccessful=true (%s)",+ (level) => {+ const input: any = semgrepSarif();+ input.runs[0].invocations = [+ {+ executionSuccessful: true,+ toolExecutionNotifications: [+ { level, message: { text: "A source file could not be analyzed" } },+ ],+ },+ ];+ expect(() => normalizeSemgrepSarif(input)).toThrow(+ "scan error notifications",+ );+ },+);++test("large scan inventories and timing tables do not exclude a small finding", () => {+ const input: any = semgrepJson();+ const paths = Array.from(+ { length: 10000 },+ (_, index) => `path-inventory-entry-${index}/ordinary-source-file.ts`,+ );+ input.paths = { scanned: paths };+ input.time = { targets: paths.map((path) => ({ path, parse_time: 0.001 })) };+ const output = normalizeSemgrepFindings(input);+ expect(output.excluded).toEqual([]);+ expect(output.findings).toHaveLength(1);+ expect(output.findings[0]!.evidence.source_data).toMatchObject({+ result: input.results[0],+ });+ expect(+ JSON.stringify(output.findings[0]!.evidence.source_data),+ ).not.toContain("path-inventory-entry-");+ expect(input.paths.scanned).toHaveLength(10000);+});++test.each([undefined, "sast"])(+ "accepts the documented Code NO_VALIDATOR marker with product %p",+ (product) => {+ const input: any = semgrepJson();+ if (product) input.results[0].extra.metadata = { product };+ else delete input.results[0].extra.metadata;+ input.results[0].extra.validation_state = "NO_VALIDATOR";+ const output = normalizeSemgrepFindings(input);+ expect(output.findings).toHaveLength(1);+ expect(output.excluded).toEqual([]);+ expect(output.findings[0]!.evidence.source_data).toMatchObject({+ result: { extra: { validation_state: "NO_VALIDATOR" } },+ });+ },+);++test.each(["json", "sarif"])(+ "preserves %s findings with empty native messages",+ (format) => {+ const input: any = format === "json" ? semgrepJson() : semgrepSarif();+ if (format === "json") input.results[0].extra.message = "";+ else input.runs[0].results[0].message = { text: "" };+ const output = normalizeSemgrepFindings(input);+ expect(output.excluded).toEqual([]);+ expect(output.findings).toHaveLength(1);+ expect(output.findings[0]!.evidence.description).toBeNull();+ expect(output.findings[0]!.evidence.source_data).toMatchObject({+ result:+ format === "json"+ ? { extra: { message: "" } }+ : { message: { text: "" } },+ });+ },+);++test.each(["json", "sarif"])(+ "still excludes %s findings with missing or non-string messages",+ (format) => {+ for (const message of [undefined, null, 42]) {+ const input: any = format === "json" ? semgrepJson() : semgrepSarif();+ if (format === "json") input.results[0].extra.message = message;+ else input.runs[0].results[0].message = { text: message };+ const output = normalizeSemgrepFindings(input);+ expect(output.findings).toEqual([]);+ expect(output.excluded).toHaveLength(1);+ }+ },+);++test.each(["scalar", "array"])(+ "JSON and SARIF expose the same CWE IDs from native %s metadata",+ (shape) => {+ const json: any = semgrepJson();+ const sarif: any = semgrepSarif();+ const label = "CWE-79: Improper Neutralization of Input";+ json.results[0].extra.metadata.cwe =+ shape === "scalar" ? label : [label, label];+ sarif.runs[0].tool.driver.rules[0].properties = {+ tags: [label, "security", "OWASP-A03:2021", label],+ };+ for (const input of [json, sarif]) {+ const original = structuredClone(input);+ const output = normalizeSemgrepFindings(input);+ expect(output.excluded).toEqual([]);+ expect(output.findings[0]!.evidence.details?.weakness_ids).toEqual([+ "CWE-79",+ ]);+ expect(input).toEqual(original);+ }+ expect(+ normalizeSemgrepFindings(json).findings[0]!.evidence.source_data,+ ).toMatchObject({+ result: {+ extra: { metadata: { cwe: json.results[0].extra.metadata.cwe } },+ },+ });+ expect(+ normalizeSemgrepFindings(sarif).findings[0]!.evidence.source_data,+ ).toMatchObject({+ rule: {+ properties: {+ tags: sarif.runs[0].tool.driver.rules[0].properties.tags,+ },+ },+ });+ },+);++test("literal and URI-encoded colons preserve native repository paths and CE identity", () => {+ const json: any = semgrepJson();+ const sarif: any = semgrepSarif();+ const path = "src/example:variant.ts";+ json.results[0].path = path;+ delete json.results[0].extra.fingerprint;+ delete sarif.runs[0].results[0].fingerprints;+ const artifact =+ sarif.runs[0].results[0].locations[0].physicalLocation.artifactLocation;+ artifact.uri = path;+ const native = normalizeSemgrepSarif(sarif).findings[0]!;+ delete artifact.uriBaseId;+ artifact.uri = "src/example%3Avariant.ts";+ const direct = normalizeSemgrepSarif(sarif).findings[0]!;+ const expected = normalizeSemgrepFindings(json).findings[0]!;+ for (const finding of [expected, native, direct]) {+ expect(finding.evidence.locations).toEqual([{ path, line: 3 }]);+ expect(finding.source_finding_id).toBe(expected.source_finding_id);+ }+});++test.each(["file:/outside/example.ts", "https://example.test/source.ts"])(+ "SARIF URI %s stays in source evidence without a repository location",+ (uri) => {+ const input: any = semgrepSarif();+ const artifact =+ input.runs[0].results[0].locations[0].physicalLocation.artifactLocation;+ delete artifact.uriBaseId;+ artifact.uri = uri;+ for (const fingerprinted of [true, false]) {+ if (!fingerprinted) delete input.runs[0].results[0].fingerprints;+ const output = normalizeSemgrepSarif(input);+ expect(output.excluded).toEqual([]);+ expect(output.findings[0]!.evidence.locations).toEqual([]);+ expect(output.findings[0]!.evidence.source_data).toMatchObject({+ result: input.runs[0].results[0],+ });+ }+ },+);diff --git a/sdk/typescript/tests-ts/support/semgrep-import-fixtures.ts b/sdk/typescript/tests-ts/support/semgrep-import-fixtures.tsnew file mode 100644index 000000000..772a2b038--- /dev/null+++ b/sdk/typescript/tests-ts/support/semgrep-import-fixtures.ts@@ -0,0 +1,75 @@+// Synthetic scanner output; no Semgrep rule implementations or real findings.+export function semgrepJson() {+ return {+ version: "1.100.0",+ errors: [],+ paths: { scanned: ["src/example.ts"] },+ results: [+ {+ check_id: "example.synthetic-rule",+ path: "src/example.ts",+ start: { line: 3, col: 1, offset: 20 },+ end: { line: 3, col: 9, offset: 28 },+ extra: {+ message: "Synthetic finding",+ severity: "CRITICAL",+ fingerprint: "synthetic-fingerprint",+ is_ignored: true,+ lines: "example()",+ metadata: {+ cwe: ["CWE-000"],+ references: ["https://example.invalid/rule"],+ },+ },+ },+ ],+ };+}++export function semgrepSarif() {+ return {+ version: "2.1.0",+ runs: [+ {+ invocations: [{ executionSuccessful: true }],+ tool: {+ driver: {+ name: "Semgrep OSS",+ rules: [+ {+ id: "example.synthetic-rule",+ defaultConfiguration: { level: "error" },+ help: { text: "Synthetic rule help" },+ },+ ],+ },+ },+ results: [+ {+ ruleId: "example.synthetic-rule",+ fingerprints: { "matchBasedId/v1": "synthetic-fingerprint" },+ message: { text: "Synthetic finding" },+ suppressions: [{ kind: "inSource" }],+ codeFlows: [{ message: { text: "Synthetic dataflow" } }],+ locations: [+ {+ physicalLocation: {+ artifactLocation: {+ uri: "src/example.ts",+ uriBaseId: "%SRCROOT%",+ },+ region: {+ startLine: 3,+ startColumn: 1,+ endLine: 3,+ endColumn: 9,+ },+ },+ },+ ],+ },+ ],+ },+ ],+ };+}