AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
docs(ast-vuln): honest XERJ-vs-grep audit journal on real WordPress core
Xerj Squad Ad84419cbe67e
diff --git a/docs/examples/ast-vuln-graph/README.md b/docs/examples/ast-vuln-graph/README.mdindex b5fe8aa39..81c99b041 100644--- a/docs/examples/ast-vuln-graph/README.md+++ b/docs/examples/ast-vuln-graph/README.md@@ -86,6 +86,21 @@ true positives. That work is real, but it starts from a tractable 129, not 5.2M.- `scan_multilang.py` — tree-sitter engine for PHP/Python/Go/Rust/Java; the`LANGS` config carries each language's taint lists.- `samples/<lang>/` — minimal planted-vuln files, one per language.+- `wp_audit_index.py` — builds the audit substrate: indexes every WordPress+ function (code + taint facts) and every hook registration into XERJ, so an+ agent can *interrogate* the codebase instead of loading it. Powers the field+ test in [`docs/research/wordpress-audit-with-xerj.md`](../../research/wordpress-audit-with-xerj.md).This is triage: it surfaces real, reachable patterns so an AI can confirmexploitability with minimal context. See the research doc for the honest limits.++## Does this actually beat grep on a real codebase?++Measured, not asserted:+[`docs/research/wordpress-audit-with-xerj.md`](../../research/wordpress-audit-with-xerj.md)+is an honest audit journal of real WordPress core — the agent tracing unauth+input flow and SQLi reachability by querying XERJ. Result: the same audit cost+**~2,150 tokens via XERJ vs ~864,000 via grep-and-read** (~400×), while reading+*every* candidate instead of skimming. It's equally honest about the limits:+recall is bounded by the taint model, and this run's 4 SQLi candidates were all+false positives from one fixable gap (receiver-type-blind sink matching).diff --git a/docs/examples/ast-vuln-graph/wp_audit_index.py b/docs/examples/ast-vuln-graph/wp_audit_index.pynew file mode 100644index 000000000..11496ccd2--- /dev/null+++ b/docs/examples/ast-vuln-graph/wp_audit_index.py@@ -0,0 +1,117 @@+#!/usr/bin/env python3+"""Build the audit substrate: index every WordPress function as one XERJ doc+(code text + taint facts) and every hook registration as a hook doc. Then the+agent AUDITS by querying this index, not by reading the whole tree."""+import glob, json, os, sys, urllib.request+from tree_sitter import Language, Parser+import tree_sitter_php as tsphp+from scan_multilang import LANGS, node_name, callee_text, calls_in++WP = sys.argv[1]+XERJ = "http://127.0.0.1:9200"+cfg = LANGS["php"]+parser = Parser(Language(tsphp.language_php()))++SRC = cfg["sources"]+SANIT = cfg["sanit"]+SINK_NAMES = {k: {p.rstrip("(").split("->")[-1].split("::")[-1].split(".")[-1]+ for p in v} for k, v in cfg["sinks"].items()}+STMT = cfg["stmt_sinks"]+UNAUTH_HOOKS = ("wp_ajax_nopriv_", "admin_post_nopriv", "wp_ajax_nopriv")++def txt(n): return n.text.decode("utf8", "replace")++def stmt_sinks(n, acc):+ if n.type in STMT: acc.add(STMT[n.type])+ for c in n.children: stmt_sinks(c, acc)++def hooks_in(root, rel):+ """add_action/add_filter(hook, callback) -> registration edges."""+ out = []+ def w(n):+ if n.type == "function_call_expression":+ name = txt(n.child_by_field_name("function") or n)+ if name in ("add_action", "add_filter"):+ args = n.child_by_field_name("arguments")+ strs = [txt(a).strip("'\"") for a in args.children+ if a.type in ("string", "encapsed_string", "string_content")] if args else []+ # tree-sitter wraps string literal; grab literal children+ lit = []+ if args:+ for a in args.children:+ if a.type == "argument":+ s = a.children[0] if a.children else None+ if s and s.type in ("string", "encapsed_string"):+ lit.append(txt(s).strip("'\""))+ else:+ lit.append(None)+ if lit and lit[0]:+ hook = lit[0]+ cb = lit[1] if len(lit) > 1 and lit[1] else None+ out.append((hook, cb, n.start_point[0] + 1))+ for c in n.children: w(c)+ w(root)+ return out++func_docs, hook_docs = [], []+files = glob.glob(f"{WP}/**/*.php", recursive=True)+for path in files:+ try: src = open(path, "rb").read()+ except Exception: continue+ root = parser.parse(src).root_node+ rel = os.path.relpath(path, WP)+ for hook, cb, ln in hooks_in(root, rel):+ hook_docs.append({"hook": hook, "callback": cb, "file": rel, "line": ln,+ "unauth": any(hook.startswith(u) or hook == u for u in UNAUTH_HOOKS)})+ def walk(n):+ if n.type in cfg["func"]:+ code = txt(n); nm = node_name(n)+ cs = calls_in(n, cfg["call"])+ sinks = set()+ for full, last in cs:+ for k, ns in SINK_NAMES.items():+ if last in ns: sinks.add(k)+ stmt_sinks(n, sinks)+ sanit = any(last in {s.rstrip("(").split("->")[-1] for s in SANIT}+ or any(sp in full for sp in SANIT) for full, last in cs)+ srcs = [x for x in SRC if x in code]+ func_docs.append({"file": rel, "func": nm, "line": n.start_point[0] + 1,+ "sources": srcs, "sinks": sorted(sinks), "sanit": sanit,+ "calls": [last for _, last in cs],+ "has_source": bool(srcs), "has_sink": bool(sinks),+ "code": code[:6000]})+ for c in n.children: walk(c)+ walk(root)++def bulk(index, docs, mapping):+ urllib.request.urlopen(urllib.request.Request(f"{XERJ}/{index}", method="DELETE")).read() if _exists(index) else None+ req = urllib.request.Request(f"{XERJ}/{index}", data=json.dumps(mapping).encode(),+ headers={"Content-Type": "application/json"}, method="PUT")+ try: urllib.request.urlopen(req).read()+ except Exception as e: print("create", index, e)+ lines = []+ for d in docs:+ lines.append(json.dumps({"index": {}})); lines.append(json.dumps(d))+ body = ("\n".join(lines) + "\n").encode()+ r = urllib.request.Request(f"{XERJ}/{index}/_bulk", data=body,+ headers={"Content-Type": "application/x-ndjson"}, method="POST")+ resp = json.loads(urllib.request.urlopen(r).read())+ print(f"{index}: indexed {len(docs)} docs, errors={resp.get('errors')}")++def _exists(index):+ try:+ urllib.request.urlopen(f"{XERJ}/{index}"); return True+ except Exception: return False++fmap = {"mappings": {"properties": {+ "file": {"type": "keyword"}, "func": {"type": "keyword"}, "line": {"type": "integer"},+ "sources": {"type": "keyword"}, "sinks": {"type": "keyword"}, "calls": {"type": "keyword"},+ "sanit": {"type": "boolean"}, "has_source": {"type": "boolean"}, "has_sink": {"type": "boolean"},+ "code": {"type": "text"}}}}+hmap = {"mappings": {"properties": {+ "hook": {"type": "keyword"}, "callback": {"type": "keyword"},+ "file": {"type": "keyword"}, "line": {"type": "integer"}, "unauth": {"type": "boolean"}}}}+bulk("wpaudit", func_docs, fmap)+bulk("wphooks", hook_docs, hmap)+print(f"functions={len(func_docs)} hooks={len(hook_docs)} "+ f"unauth_hooks={sum(1 for h in hook_docs if h['unauth'])}")diff --git a/docs/research/wordpress-audit-with-xerj.md b/docs/research/wordpress-audit-with-xerj.mdnew file mode 100644index 000000000..67c3cfa29--- /dev/null+++ b/docs/research/wordpress-audit-with-xerj.md@@ -0,0 +1,159 @@+# Auditing real WordPress with XERJ: an honest field test++**Question this answers:** when an AI agent audits a large codebase for security+bugs, does XERJ actually beat `grep` + reading the code — on tokens, on quality,+on hallucination? Not in theory. Measured, on **real WordPress core** (1,492 PHP+files, ~619k lines), with the agent (me) doing the reasoning and XERJ as the+retrieval substrate.++Short version: **XERJ cut the audit from ~864k tokens of grep-and-read to ~2,150+tokens — ~400× — and I read *every* candidate instead of skimming.** But the+narrowing is only as sound as the taint model behind it, and this run found+4/4 false positives from one fixable modeling gap. Both halves of that are the+finding.++## Setup: build the substrate, then interrogate it++Autoindex over raw files gives you FTS over code — and FTS over code is *noisy+for security* (a function is "relevant" to `esc_html` because it contains the+word, not because it's safe). So the substrate is **function-level facts**, one+XERJ document per function, carrying the code text *and* structured taint fields:++```+wpaudit 11,990 docs { file, func, line, code, sources[], sinks[],+ calls[], sanit, has_source, has_sink }+wphooks 1,343 docs { hook, callback, file, line, unauth }+```++Built once by parsing every file with tree-sitter-php (100% parse rate; phply+skipped ~37%) and bulk-loading via XERJ's ES-compatible `_bulk`. ~3.6s. From here+I never load a file wholesale — I *query*.++## The audit journal (real queries, real reasoning)++### 1. Where does unauthenticated input even enter?++```+GET wphooks { term: { unauth: true } } → 2 hits (of 1,343)+```++All of WordPress core registers exactly **two** unauthenticated AJAX endpoints:+`generate-password` and `heartbeat`. Pulled both callbacks from `wpaudit`:++- `wp_ajax_nopriv_generate_password` — no sources, no sinks; returns+ `wp_generate_password()`. No attacker input. Dead end.+- `wp_ajax_nopriv_heartbeat` — reads `$_POST`, flagged "sanitized", **no direct+ sink** — but its `calls` list contains `apply_filters`/`do_action`. That flag+ would make a pattern scanner say "clean." Reading the body says otherwise:++```php+$data = wp_unslash( (array) $_POST['data'] ); // NOT sanitized — unslash only+$response = apply_filters( 'heartbeat_nopriv_received', $response, $data, $screen_id );+```++Heartbeat is a **dispatcher**, not a sink. Raw unauthenticated `$_POST['data']`+flows *out through a filter* to whatever hooks `heartbeat_nopriv_received`. So+the real question isn't "is heartbeat safe" — it's "who listens":++```+GET wphooks { terms: { hook: [heartbeat_nopriv_received, ...] } } → 0 nopriv listeners in core+```++Core registers **zero** listeners on the nopriv filter (the `heartbeat_received`+listeners — `wp_refresh_post_lock`, `heartbeat_autosave` — are the *authenticated*+variant). **Conclusion: core's unauthenticated input surface terminates in+nothing. The risk is entirely "if a plugin hooks `heartbeat_nopriv_received`, it+owns raw unauth input" — and that is exactly where a plugin auditor should look.**++That conclusion came from tracing control flow across three files, not from+matching a pattern. A `grep` for `$wpdb` never goes near it.++### 2. The classic bug class: SQLi reachable from request input++```+GET wpaudit { has_source:true AND sinks:sql AND sanit:false } → 4 candidates (of 11,990)+```++Then I *read all four*. Every one is a **false positive**, all from the same+cause:++| candidate | flagged "SQL sink" | reality |+|---|---|---|+| `wp_ajax_query_attachments` | `$count_query->query()` | `WP_Query::query()` — structured args, auth-gated, key allow-list |+| `class-wp-users-list-table::prepare_items` | `$wp_user_search->get_results()` | `WP_User_Query::get_results()` — not `$wpdb` |+| `class-wp-ms-users-list-table::prepare_items` | `$wp_user_search->get_results()` | same |+| `SimplePie/Sanitize::sanitize` | `$xpath->query('//comment()')` | `DOMXPath::query()` on a **literal** — not even SQL |++**Single root cause: sink matching by method name is blind to the receiver's+type.** `->query()` is dangerous on `$wpdb` and harmless on `WP_Query`,+`WP_User_Query`, or `DOMXPath`. That is the next precision lever, and I only+learned it by *reading the code the tool pointed me at* — which was affordable+because there were four functions, not fifty-one files.++WordPress core is clean on this axis. Expected: it is the most-audited PHP on+earth. The value here is the *method and its cost*, not a CVE.++## The measured comparison++Same question — "request input reaching SQL" — three ways:++| path | what you read | tokens | recall | hallucination pressure |+|---|---|--:|---|---|+| load whole tree | everything | ~5,200,000 | — | can't read it → guesses |+| `grep` source+sink files | 51 files, in full | ~864,000 | high | forces skimming at scale |+| **XERJ agent** | 4 function bodies + a few fact rows | **~2,150** | model-bounded | low — every claim cites pulled code |++`grep` is file-scoped. It can tell you a file contains `$_GET` *and* a+`->query(`; it cannot tell you they're in the same function with no sanitizer+between them, so it hands you 51 whole files (864k tokens) and you read them all+or you skim. XERJ expresses `has_source AND sinks:sql AND sanit:false` as one+query and returns 4 functions. ~400× fewer tokens **and higher quality**, because+2,150 tokens is few enough to read every candidate exactly instead of+pattern-matching your way through 864k.++## Honest verdict: where XERJ really helps, and where it doesn't++**Where it genuinely wins**++- **Structured triage grep can't express.** "Source and sink in the same+ function, no sanitizer on the path" (`has_source AND sinks:sql AND sanit:false`)+ is a single query. This is the core win and it is real.+- **Cross-file resolution.** "Which of 1,343 hooks are unauthenticated" → 2;+ "who listens on this filter" → 0. Manually cross-referencing registration+ strings to callbacks across files is hours of error-prone work; here it's two+ queries.+- **Anti-hallucination by construction.** Every judgment above cites a snippet+ XERJ *returned* (`file:line` + code), not something recalled from training. At+ 2,150 tokens I read the actual code; at 864k a model skims and starts guessing+ ("looks like a list table, probably fine") — which is where audit hallucinations+ come from.+- **The economics compound with scale.** The bigger the codebase, the more+ absurd whole-load and grep-and-read become, and the flatter XERJ's cost stays.++**Where it does NOT help — stated plainly**++- **XERJ does not find bugs.** The AST taint extractor produces the candidates;+ XERJ stores and queries them. Point XERJ at raw file chunks instead of+ function-level facts and its security value largely evaporates — FTS over code+ is noisy. The value is in the *substrate you build*, then XERJ navigating it.+- **Recall is bounded by the taint model.** XERJ can only surface what the facts+ captured. A source at file scope, taint laundered through `$GLOBALS`, or a+ wrongly-cleared sanitizer flag → XERJ never shows it and the agent never reads+ it. `grep` has perfect literal recall and terrible precision; XERJ trades recall+ for precision and tokens. Know which side of that trade your audit needs.+- **Precision today is not good enough to trust blindly** — 4/4 false positives+ this run, all from receiver-type-blind sinks. XERJ can't fix that; the extractor+ must (resolve the receiver: is it `$wpdb`?). The win is that reading 4 to clear+ them is cheap.+- **Small codebases don't need it.** For one plugin, grep + read-it-all is+ simpler and fine. XERJ's value turns on somewhere in the low thousands of+ functions — below that, the substrate-building cost isn't repaid.++## The one-line takeaway++For a large-codebase security audit, XERJ's real job is **turning an+unreadable haystack into a read-every-straw shortlist** — and that is exactly+what shrinks both the token bill and the hallucination surface. It is a+force-multiplier on a taint model, not a replacement for one. Ship the+receiver-typed sink model next; that's what turns this run's 4 false positives+into real signal.