OpenAI's Codex Security CLI and TypeScript SDK for finding, validating, and fixing security vulnerabilities. npm: https://www.npmjs.com/package/@openai/codex-security
TypeScript ai-securityapplication-securityclicode-scanningcodexcodex-securitycybersecuritydevsecopsnodejsnpmopenaisdksecuritytypescriptvulnerability-scanning
docs: add an Invoice Desk scan and fix walkthrough (#1701)
Michael D'Angelo92b1fa936418
diff --git a/examples/README.md b/examples/README.mdindex 0709ef772..5a434bce9 100644--- a/examples/README.md+++ b/examples/README.md@@ -22,12 +22,13 @@ Examples and templates for using Codex Security:validation script against a deliberately vulnerable API using synthetic data.Follow the demo's setup instructions, and do not deploy the example app.-- [Invoice Desk sample](invoice-desk/README.md): a Node.js invoice app with ten- seeded finding scenarios, synthetic data, and no dependencies or external- services. Its [CI workflow](invoice-desk/README.md#ci-and-openai-scans) runs- behavior tests on PRs to `main` and queues OpenAI scans with environment- approval. Sample guides and expected findings stay outside the scan input.- Results include finding metrics and downloadable reports.+- [Invoice Desk walkthrough](invoice-desk/walkthrough.md): scan a standalone+ synthetic invoice app, inspect saved findings and coverage, explicitly request+ a patch, and verify the original boundary. The app has ten seeded scenarios+ and no dependencies or external services. Its+ [CI workflow](invoice-desk/README.md#ci-and-openai-scans) runs behavior tests and+ automatic OpenAI scans. Guides and expected findings stay outside the scan+ input; scan results include finding metrics and downloadable reports.- [Azure Pipelines with Amazon Bedrock](azure-pipelines/README.md): centrally runmanual full or committed-diff scans against Azure Repos, with OIDC credentials,@@ -37,9 +38,9 @@ Examples and templates for using Codex Security:This top-level `examples/` directory is available in the repository only; it isnot included in the `@openai/codex-security` npm package. The package is built-from [`sdk/typescript`](../sdk/typescript/package.json), whose `files` list-includes only the CLI launcher, compiled SDK, bundled plugin, license, and-package README. The package check rejects files outside its expected contents.+from [`sdk/typescript`](../sdk/typescript/package.json); its `files` list is the+source of truth for included files. The package check rejects files outside its+expected contents.The bundled plugin's own example artifacts are separate and remain part of thepackage.diff --git a/examples/invoice-desk/README.md b/examples/invoice-desk/README.mdindex 0857fd037..77666a94a 100644--- a/examples/invoice-desk/README.md+++ b/examples/invoice-desk/README.md@@ -11,6 +11,10 @@ the nested [QA guide](qa/README.md) explain the sample and its expected findingsThe [threat model](SECURITY.md) describes the app's assets, trust boundaries, andintended security properties. It also stays outside the standalone scanner input.+To scan the sample and verify a patch, follow the+[scan, inspect, patch, and verify walkthrough](walkthrough.md). It uses a standalone+copy of the app and keeps illustrative evidence separate from your actual results.+## StartFrom the repository root:diff --git a/examples/invoice-desk/walkthrough.md b/examples/invoice-desk/walkthrough.mdnew file mode 100644index 000000000..362030610--- /dev/null+++ b/examples/invoice-desk/walkthrough.md@@ -0,0 +1,186 @@+# Invoice Desk: scan, inspect, patch, and verify++Use the synthetic Invoice Desk app to follow one finding from a source scan to+an explicit patch and a separate verification. This guide supplies commands and+an illustrative finding; it is **not a recorded model run**. Finding counts,+severity, evidence, and coverage must come from your own saved results.++The app is deliberately insecure. Use only its synthetic data on loopback, and+keep the [QA guide](qa/README.md) outside the scan input. Ten seeded scenarios do+not guarantee ten findings. The behavior tests establish the sample's behavior,+not scanner recall.++## 1. Prepare a standalone copy++Install and authenticate the [Codex Security CLI](../../sdk/typescript/docs/cli.md#authentication).+The app itself needs Node.js 22.13 or later and has no dependencies. Follow the+CLI's own [runtime requirements](../../sdk/typescript/README.md#install).+Scanning, patching, and verification use model inference.++Run these Bash commands from the repository root:++```bash+invoice_source="$PWD"+invoice_run="$(mktemp -d "${TMPDIR:-/tmp}/invoice-desk-workflow.XXXXXX")"+cp -R "$invoice_source/examples/invoice-desk/app" "$invoice_run/app"+git -C "$invoice_source" rev-parse HEAD > "$invoice_run/source-revision.txt"+codex-security --version > "$invoice_run/cli-version.txt"+cd "$invoice_run/app"+git init -q+git add .+git -c user.name="Invoice Desk demo" -c user.email="[email protected]" \+ commit -qm "Baseline synthetic Invoice Desk app"+git rev-parse HEAD > "$invoice_run/app-revision.txt"+```++The new Git repository has no remote. It provides a baseline for reviewing the+patch. Results will be siblings of `app/`, outside its worktree. The copied app+contains no sample guide, threat model, QA harness, answer key, or prior report.+Keep the scanner's working directory here and do not supply the original checkout+as additional context. You do not need to start the HTTP server for a source scan.++## 2. Run and retain the scan++```bash+codex-security scan . --mode standard --effort high --format json \+ --output-dir "$invoice_run/scan" > "$invoice_run/scan-result.json"+```++This uses your configured provider and model. To select a model explicitly, add+`--model MODEL` with a model available through that provider. Retain the effective+settings from the saved scan below, alongside the CLI version and source revision.+Use a new output directory for another run.++With these options, exit `0` means the scan completed, including when it reports+findings. Exit `2` means execution or required output failed; inspect diagnostics+and retained artifacts instead of treating it as a clean result. If you add+`--fail-on-severity`, exit `1` means findings met that threshold.++After completion, read `scan.id` from `$invoice_run/scan/scan-manifest.json`. Replace `SCAN_ID`+in the following commands with that actual value:++```bash+codex-security scans show SCAN_ID --json > "$invoice_run/saved-scan.json"+codex-security scans logs SCAN_ID+```++`scans show` records the saved model, reasoning effort, mode, and replay recipe.+The manifest identifies the target revision and scope; it does not define model+configuration fields. Always use the explicit scan ID when navigating results so+a later failed or active run is not confused with an earlier completed scan.++## 3. Read evidence and coverage together++Start with `$invoice_run/scan/report.md`, then inspect the structured artifacts.+Paths in this table are relative to `$invoice_run`:++| Artifact | What to inspect |+| ------------------------- | ---------------------------------------------------------------------------------------------------------- |+| `scan/scan-manifest.json` | `scan.id`, `scan.status`, target identity, revision, and requested scope. |+| `scan/findings.json` | The actual finding's `occurrenceId`, source evidence, root cause, attack path, confidence, and validation. |+| `scan/coverage.json` | Assessed `surfaces`, their dispositions, `explicitExclusions`, `deferred` work, and any `openQuestions`. |+| `saved-scan.json` | Saved scan settings and recipe, for reproducing or comparing the run. |++When present, follow `rootCause.evidenceRefs` into `codeEvidence`; otherwise,+inspect the finding's recorded source locations and evidence. Separate what the+source proves from what runtime validation actually exercised. Missing runtime validation is a proof gap, not evidence that+the issue is safe. An empty finding list also says nothing about excluded or+deferred surfaces.++### Illustrative example: a workspace boundary++Read this example **after independent discovery**. It describes the seeded+cross-workspace invoice read; it is not a claim that your scan found it.++| Part of the finding | Source-supported interpretation |+| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |+| Attacker-controlled input | An authenticated member supplies an invoice ID to `GET /api/invoices/:id`. |+| Broken boundary | The GET route calls `getInvoice(store, id)`, which looks up the invoice without checking the member's workspace. |+| Impact | A North Studio member can read a Cedar Workshop invoice by ID. |+| Existing control | `ownedInvoice(store, user, id)` checks workspace ownership for other invoice operations; the list endpoint also filters by workspace. |+| Verification target | Cross-workspace reads must fail, while the member can still read an invoice in their own workspace. |++Inspect the [HTTP route](app/server.mjs) and [invoice helpers](app/invoices.mjs).+The separate [QA harness](qa/app.test.mjs) reproduces this behavior as ID-002 and+checks legitimate controls. Its scenario tests intentionally assert that seeded+weaknesses remain present; they are not post-fix acceptance tests.++If your scan reports this root cause, use its actual `occurrenceId` from this+run's `findings.json`. Do not substitute the QA label `ID-002`, a title, or an ID+from an unrelated scan. If it does not report the issue, record a miss or+uncertainty when evaluating recall. A later patch from a supplied description+would test remediation, not retroactively count as scanner discovery.++## 4. Explicitly request one patch++Continue only after choosing a source-supported finding from the saved scan.+Set `invoice_occurrence` to its actual ID and preserve that finding before patching:++```bash+invoice_occurrence="occ_REPLACE_WITH_ACTUAL_ID"+node --input-type=module - "$invoice_run/scan/findings.json" \+ "$invoice_occurrence" "$invoice_run/finding.json" <<'JS'+import { readFileSync, writeFileSync } from "node:fs";+const [source, id, output] = process.argv.slice(2);+const finding = JSON.parse(readFileSync(source, "utf8")).findings.find(+ (item) => item.occurrenceId === id,+);+if (!finding) throw new Error("Select an occurrence from this scan.");+writeFileSync(output, JSON.stringify(finding, null, 2) + "\n");+JS+```++Now explicitly authorize changes to the standalone app associated with that finding:++```bash+codex-security patch "$invoice_occurrence" --json > "$invoice_run/patch-result.json"+git status --short+git diff+```++`patch` fixes and verifies the selected finding. Review the returned status and+verification evidence, the tracked diff, and any new files reported by `git status`.+For the example above, look for enforcement at the invoice-read boundary, a+regression check for the other workspace, and a legitimate same-workspace control.+A change in finding count or a modified file alone does not prove remediation.++The saved-finding result distinguishes `verified`, `no_change`, `blocked`, and+`failed`. Preserve a blocked or failed result with its missing prerequisite or+failed check; do not relabel it as fixed. Keep unrelated seeded scenarios out of+this patch. The original app in the source checkout remains the vulnerable sample.++## 5. Explicitly verify the original boundary++After reviewing the patch, request a separate read-only assessment from the+patched app directory. Use the preserved original finding file to assess the+same claim and evidence independently of its current status in scan history.+File input targets the current directory; keep it set to the patched app. The+verification result labels this single input `finding-1`; `finding.json` retains+its original occurrence ID.++```bash+codex-security verify-fix "$invoice_run/finding.json" --json \+ > "$invoice_run/verify-result.json"+```++For the illustrated finding, the evidence should establish all of the following+against the patched standalone app:++- A North Studio member's request for a Cedar Workshop invoice is rejected,+ following the application's existing not-found behavior.+- The same member can still read their own workspace's invoice.+- The route uses the intended ownership check, and a nearby alternative path+ does not preserve the reported bypass.+- Relevant regression checks ran, or the result states why they could not run.++`verify-fix` reports `fixed`, `still_vulnerable`, or `inconclusive`; it does not+apply another patch. Its exit codes are `0`, `1`, and `2`, respectively, with `2`+also covering failed checks. Preserve exact commands and evidence. Another scan+omitting the finding is not sufficient verification, and unavailable relevant+checks must remain visible in an inconclusive result.++Keep the source revision, CLI version, saved scan and settings, patch result,+final diff, and verification result together. Review them before sharing. To+measure discovery separately, compare the original scan with the+[QA scoring guide](qa/README.md#scanner-evaluation), counting distinct root causes,+misses, uncertain cases, additional findings, and duplicates.