Skip to content

A coding-agent skill for multi-phase security audits with independently verified, machine-readable findings

JavaScript

Bring the whole skill into one coherent house style

Dan Jonesc1239b903704
diff --git a/skills/security-audit/AI-AND-LLM.md b/skills/security-audit/AI-AND-LLM.md
index 1981e47..fb47ca8 100644
--- a/skills/security-audit/AI-AND-LLM.md
+++ b/skills/security-audit/AI-AND-LLM.md
@@ -58,7 +58,7 @@ Conversation history, embeddings, or the KV/prompt cache keyed too broadly, so o
- **Draw the boundary before hunting.** Enumerate: what identity do tools run as, what's in the context window, who can write to each context source, where does output go. Most AI findings fall out of a correct map of these four; most AI false positives come from not drawing it.
- **Find the capability, then find who can reach it.** Start from the most dangerous tool (delete, spend, exec, fetch-internal) and work backwards to whether untrusted text can reach its arguments. Power × reachability, same as any privileged interface.
-## Verification discipline (apply before reporting ANY finding here)
+## Validation rules (apply before reporting ANY finding here)
1. **Name the boundary crossed.** State exactly who the attacker is, whose session/identity the payload executes in, and what they get that they couldn't get directly. If attacker and victim are the same principal and the capability is one they already have, it is not a finding.
2. **For confused-deputy / excessive-agency claims, prove both halves.** Show (a) the tool performs no per-resource check scoped to the requesting user, AND (b) the action is one the user could not perform through a normal authenticated request. A shared service credential with per-user query scoping fails both tests and is not a finding.
diff --git a/skills/security-audit/CLIENT-SIDE.md b/skills/security-audit/CLIENT-SIDE.md
index 1183060..d5901bd 100644
--- a/skills/security-audit/CLIENT-SIDE.md
+++ b/skills/security-audit/CLIENT-SIDE.md
@@ -58,7 +58,7 @@ An attacker-controlled key (`__proto__`, `constructor.prototype`) reaching a *ne
- **Server escaping ends where the fragment begins.** Data after `#`, plus `window.name` and cross-window messages, never reaches the server — so server-side filters can't see it. That blind spot is the DOM-XSS goldmine.
- **Enumerate the escape hatches.** In an auto-escaping framework, the candidate list *is* every `dangerouslySetInnerHTML`/`v-html`/`bypassSecurityTrust*`/`$sce.trustAs*` call. Start there.
-## Verification discipline (apply before reporting ANY finding here)
+## Validation rules (apply before reporting ANY finding here)
1. **Confirm a controllable source AND an executing sink on the client path.** Cite the source (`location.hash`, `event.data`, `window.name`) and the sink (`innerHTML`, `eval`, navigation), and show untrusted data reaching the sink without sanitization. A source with no sink, or a sink fed only trusted data, is not a finding.
2. **For prototype pollution, prove the recursive write AND a gadget.** Show the nested/recursive assignment that reaches `Object.prototype`, then the code that later reads the polluted property to a security-relevant effect. Pollution with no reachable gadget is not exploitable.
diff --git a/skills/security-audit/HUNTING.md b/skills/security-audit/HUNTING.md
index a588d4b..5ed47b9 100644
--- a/skills/security-audit/HUNTING.md
+++ b/skills/security-audit/HUNTING.md
@@ -79,10 +79,6 @@ Think about these angles:
that reveal whether a record exists. Response size differences. HTTP headers that
disclose versions. Debug endpoints that survived into production.
-YOU CAN SPAWN SUB-AGENTS. If you need to understand a subsystem in depth to evaluate
-a potential finding — use the Task tool to launch a research agent. Don't try to hold
-everything in your own context. Go deep where it matters.
-
11. WHAT PARAMETERS OVERRIDE SECURITY-RELEVANT DEFAULTS?
Where a default is safe but a user-supplied parameter can change it. Look for
every input that overrides a security-relevant default and check if the override
@@ -92,6 +88,12 @@ everything in your own context. Go deep where it matters.
Anywhere self-declared identity, capability, or metadata influences an access
or trust decision without independent verification.
+GO DEEP, AND PROVE IT. You can spawn sub-agents: if evaluating a candidate finding needs
+deep understanding of a subsystem, use the Task tool to launch a research agent instead of
+holding everything in one context. And where the code is locally runnable, don't just reason
+about it — extract the suspect function into a minimal harness (or build and run the target)
+and test the hypothesis directly. A reproduced result beats an argued one.
+
YOUR SCOPE IS YOUR PRIMARY FOCUS, NOT A BOUNDARY.
If while investigating your assigned area you notice something wrong in a different
category — a permission issue while tracing injection, a race condition while reviewing
diff --git a/skills/security-audit/MEMORY-SAFETY-AND-BINARY.md b/skills/security-audit/MEMORY-SAFETY-AND-BINARY.md
index 7b0d4b6..d138ddd 100644
--- a/skills/security-audit/MEMORY-SAFETY-AND-BINARY.md
+++ b/skills/security-audit/MEMORY-SAFETY-AND-BINARY.md
@@ -49,7 +49,7 @@ Pick the relevant classes based on Phase 1. Split per subsystem for large target
- **Chain a weak primitive.** A blocked path means you haven't found the right pivot, not that it's unexploitable. Always ask "what does this actually let me do, and what runs automatically once I can put bytes on disk?" (plugin dirs, autoload, `.git/hooks`, `conftest.py`).
- **Hunt where the crowd isn't.** The tools researchers themselves trust — debuggers, disassemblers, scanners, dev tooling — are under-audited and high-impact. Old code and obscure formats are gold.
-## Verification discipline (apply before reporting ANY finding here)
+## Validation rules (apply before reporting ANY finding here)
1. **Build a debuggable target first.** Wire in crash dumps + a debugger before you claim exploitability. You can't iterate on what you can't observe.
2. **Read the offset from the crash, not the disassembly.** Send a cyclic (De Bruijn) pattern; the faulting register values give the exact offset. A variable-length prefix (handle, optional field, padding) shifts the geometry off the static prediction.
diff --git a/skills/security-audit/RECONNAISSANCE.md b/skills/security-audit/RECONNAISSANCE.md
index 4b7bf04..a903acc 100644
--- a/skills/security-audit/RECONNAISSANCE.md
+++ b/skills/security-audit/RECONNAISSANCE.md
@@ -1,4 +1,4 @@
-# Phase 1: Understand the Application
+# Reconnaissance
### Phase 1: Understand the application
diff --git a/skills/security-audit/SKILL.md b/skills/security-audit/SKILL.md
index ca27307..3484ac8 100644
--- a/skills/security-audit/SKILL.md
+++ b/skills/security-audit/SKILL.md
@@ -30,7 +30,7 @@ All files written during the audit go in the output directory:
- `FINDINGS-DETAIL.md` — detailed data flows for MEDIUM+ findings (Phase 4)
- `findings.json` — machine-readable structured output (Phase 5)
-Subagents (Phases 2, 3, 6) do NOT write files — they return results to you via the Task tool. You are responsible for writing all files to the output directory.
+Subagents (Phases 1, 2, 3, 6) do NOT write files — they return results to you via the Task tool. You are responsible for writing all files to the output directory.
### Coverage and prior runs
@@ -51,6 +51,10 @@ Include a brief summary of prior runs in the architecture summary so Phase 2 age
Every finding must have a concrete attack scenario: who is the attacker, what do they do, and what do they get? "An attacker could theoretically..." is not a finding. "Send this request, get this result" is.
+### Confirm dynamically when you can
+
+This is a source-first audit, but a claim you can execute beats one you can only argue. Where the target is locally buildable — a parser, a library, a CLI, a native component — build and run it: reproduce the crash, run the payload, diff the two parsers on the same bytes. Better still, **extract the suspect code into a minimal standalone harness** and test the hypothesis in isolation — fuzz the one function, feed it the crafted input, watch what it does. Where confirmation needs infrastructure you don't have — a proxy chain, a live cache, production auth — you cannot confirm from source alone: mark it "requires deployment testing" and do not report it as confirmed. Dynamic evidence is what resolves the memory-safety and request-framing classes that static reading leaves ambiguous.
+
### Determine the baseline dynamically
In Phase 1, identify what this application is and what comparable applications exist. Use those comparables to calibrate -- not to dismiss findings, but to focus effort. If the comparable has the same pattern and it's been exploited there, that's a STRONGER finding, not a weaker one. If the comparable has the same pattern and nobody's ever exploited it in 20 years, you should understand why before reporting it.
@@ -59,7 +63,7 @@ Do NOT hardcode a specific comparable. A CMS gets compared to other CMSes. An AP
### Defense-in-depth gaps are not vulnerabilities
-If Layer A prevents the attack, the absence of Layer B is a hardening suggestion, not a finding. Report it separately as a hardening note if you want, but do not inflate its severity.
+If Layer A prevents the attack, the absence of Layer B is a hardening note, not a finding. Report it separately if you want, but do not inflate its severity.
### Severity requires impact
@@ -68,12 +72,15 @@ Severity is the combination of **likelihood** (how easy to exploit, what access
- **CRITICAL**: Unauthenticated RCE, full database dump, admin account takeover without credentials
- **HIGH**: Authenticated RCE, SQL injection with data exfiltration, stored XSS that fires for all users, auth bypass. Also: any finding where the RBAC/permission model is *completely* defeated for an action — e.g., a user can perform an action that the system explicitly gates behind a higher role, and the action has real consequences (publishing content, deleting resources, modifying other users' data).
- **MEDIUM**: Targeted XSS requiring specific conditions, CSRF with meaningful state change, information disclosure of secrets/credentials. Also: business logic bypasses with real but limited consequences — e.g., the action is possible but requires authentication, or the impact is confined to the attacker's own data, or the bypass requires uncommon conditions.
-- **LOW**: Information disclosure of non-secret data, DoS requiring sustained effort, hardening gaps
+- **LOW**: Information disclosure of non-secret data, DoS requiring sustained effort
+- **INFORMATIONAL**: A confirmed but minimal-impact observation with no standalone exploit — useful mainly as a building block for another finding. Pure defense-in-depth gaps belong in hardening notes, not here.
-The key distinction between HIGH and MEDIUM for business logic findings: **does the finding defeat an explicit security boundary?** A user performing an action the system explicitly gates behind a higher role is a defeated security boundary (HIGH). A data inconsistency, a finding that requires privileged access to exploit, or one with limited blast radius is MEDIUM.
+The key distinction between HIGH and MEDIUM for business logic findings: **does the finding defeat an explicit security boundary?** Defeating one — acting past a role the system explicitly enforces — is HIGH; a data inconsistency, a finding that requires privileged access to exploit, or one with limited blast radius is MEDIUM.
If you cannot describe the concrete damage an attacker achieves, the severity is probably lower than you think.
+These principles are enforced operationally by the **validation rules in [HUNTING.md](HUNTING.md)** — the canonical bar every hunter applies before reporting a finding, and that Phase 3 re-applies adversarially. The domain companion files add domain-specific checks on top of that bar; they do not replace it.
+
## Workflow overview
Follow all six phases in order:
diff --git a/skills/security-audit/VALIDATION-AND-REPORTING.md b/skills/security-audit/VALIDATION-AND-REPORTING.md
index 6758709..4e3dc0a 100644
--- a/skills/security-audit/VALIDATION-AND-REPORTING.md
+++ b/skills/security-audit/VALIDATION-AND-REPORTING.md
@@ -9,11 +9,11 @@ For findings from the same attack surface, batch them into one validation agent.
Each validation agent prompt should:
1. State the specific finding being validated (title, claimed attack, claimed impact)
2. Ask the agent to read the exact code paths and verify each step of the trace
-3. Ask it to apply these tests:
+3. Ask it to apply these tests (the adversarial, Phase 3 form of the canonical validation rules in [HUNTING.md](HUNTING.md) — here a separate agent tries to make each one fail):
**Validation tests:**
1. **Exploitation test**: Read the actual code at each step of the trace. Does the data flow work as claimed? Can you construct the exact input (HTTP request, CLI invocation, API call, crafted file, etc.) that triggers this?
-2. **Impact test**: What does the attacker actually get? If the answer is "they learn field names" or "they cause an error", that's LOW at best.
+2. **Impact test**: What does the attacker actually get? If the answer is "they learn field names" or "they cause an error", that's not meaningful impact — not a finding on its own (at most a building block for a chain).
3. **Baseline test**: Does the identified comparable have the same pattern? If yes, has it been exploited? If never exploited in years of production use, understand why before reporting.
4. **Mitigation test**: Is there another layer that prevents exploitation? Check middleware, database constraints, framework defaults.
5. **Parser/runtime behavior test**: If the exploit depends on how a parser or runtime handles specific input, verify against the actual spec or implementation — do not reason from intuition.
@@ -61,7 +61,7 @@ The schema supports two verdict types via `oneOf`:
**Before writing `findings.json`:**
1. Read `report-schema.json` from this skill's directory. Follow it exactly — `additionalProperties: false` is enforced, so extra fields will make the output invalid.
-2. For each finding, populate every required field. If you cannot fill `trace` with real file paths and line numbers verified against the source, the finding is not sufficiently verified — go back and verify it or reject it.
+2. For each finding, populate every required field. If you cannot fill `trace` with real file paths and line numbers verified against the source, the finding is not sufficiently verified — go back and verify it or reject it. Mind the required fields that aren't self-evident: `intended_behavior` (what the code is *supposed* to do, so the defect is legible), `confidence` (`low`/`medium`/`high`, with a reason), and the `severity` object (`likelihood`/`impact`/`overall_severity`). All `severity` scores use the schema's **lowercase** enum — `informational`/`low`/`medium`/`high`/`critical`; the UPPERCASE tiers in SKILL.md and REPORT.md are prose labels, not valid JSON values.
3. Run `node <skill-dir>/validate-findings.cjs <output-dir>/findings.json` to validate. It checks required fields, enum values, structural constraints, and `additionalProperties`. This is a structural check only — it confirms the JSON conforms to the schema, not that the findings are correct. Factual verification is Phase 6's job. Fix any failures before proceeding.
### Phase 6: Independent verification
@@ -81,16 +81,18 @@ You are an independent verifier. You did NOT write this finding. Your job is to
2. Verify the root_cause statement by reading the cited file and confirming the described defect exists.
-3. Verify the execution payloads would actually work:
- - Does the endpoint exist at the claimed URL?
- - Does the HTTP method match?
- - Would the input pass validation as described?
- - Would auth/access checks pass as described?
+3. Verify the execution payloads would actually work, in terms that fit the target:
+ - Does the entry point exist as claimed — the endpoint/URL, CLI command, exported function, syscall/ioctl, message handler, or tool the attacker invokes?
+ - Does the invocation match — HTTP method, argument shape, call signature, or message format?
+ - Would the input survive validation and parsing on the real code path?
+ - Would the relevant authentication, authorization, or ownership check pass as described?
4. Verify conditions are complete — are there prerequisites the finding missed?
5. Check the remediation code_changes — would the fix actually prevent the attack without breaking normal functionality?
+6. Verify `intended_behavior` accurately states what the code should do, and that `confidence` matches the strength of the evidence — don't leave `high` on a claim you couldn't fully trace.
+
Return one of:
- "VERIFIED" — all claims checked out against the source
- "CORRECTED: [field]: [what was wrong] → [what it should be]" — factual error in a specific field
diff --git a/skills/security-audit/WEB-PROTOCOL-AND-AUTH.md b/skills/security-audit/WEB-PROTOCOL-AND-AUTH.md
index 0637567..0294c3e 100644
--- a/skills/security-audit/WEB-PROTOCOL-AND-AUTH.md
+++ b/skills/security-audit/WEB-PROTOCOL-AND-AUTH.md
@@ -75,7 +75,7 @@ The densest source of auth bypasses. Check, in the verification code:
- **Enumerate every door to the same identity.** SSO, password login, API key, password reset, impersonation — each is a parallel path that mints a session. The weakest one sets the account's real security; a hardened login means nothing if reset is trivial.
- **Audit the compat/fallback path.** A legacy endpoint version, a deprecated header, or a "for old clients" branch that skips a guard the main path added. Old auth code is where the reverted or forgotten check hides.
-## Verification discipline (apply before reporting ANY finding here)
+## Validation rules (apply before reporting ANY finding here)
1. **Source-visibility gate — this domain lives partly outside the repo.** Framing bugs (proxy chain), cache poisoning/deception (cache-key config), secret strength, and token entropy frequently depend on components, config, or values NOT in the audited tree. If confirming the bug requires a component/config/secret you cannot read, it is **unverifiable from source: flag it "requires deployment testing" and do NOT report it as a confirmed finding.** "Downgrade" is not enough — an unconfirmable HIGH reported as a MEDIUM is still a false positive.
2. **For framing/cache findings, name both components and the divergent parse.** "The Go net/http back-end accepts a bare-LF `Transfer-Encoding` that the front proxy treats as CL" — not "smuggling may be possible." A single server with no proxy in front has no smuggling surface. If you've confirmed only the in-repo half (the back-end genuinely mishandles a specific ambiguous input — bare-LF `Transfer-Encoding`, duplicate CL), record it as a lead with the exact bytes — "requires paired front-end testing" — a real observation, not a severity-rated finding.