AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
ci(landing): guard the site's hand-duplicated constants against drift
xerj-orgb56ade4daaad
diff --git a/.github/scripts/landing-constants-guard.sh b/.github/scripts/landing-constants-guard.shnew file mode 100755index 000000000..f13dbabf9--- /dev/null+++ b/.github/scripts/landing-constants-guard.sh@@ -0,0 +1,178 @@+#!/usr/bin/env bash+# landing-constants-guard.sh — fail when the website's hand-duplicated constants drift.+#+# WHY THIS EXISTS+# landing/ is plain static HTML with no build step, so every published constant+# (ES-YAML conformance count, canonical-operation count, release version, the+# kNN exactness claim) is copy-pasted across dozens of files. A 2026-08-12+# first-contact agent study found FOUR different conformance figures live on the+# site at once, a page headlining "SEVEN canonical operations" whose own card said+# "six", version footers frozen three releases back, and three machine-readable+# schema files that nothing linked to. Individually cosmetic; collectively they+# cost trust on a site whose entire pitch is verified numbers — and agents read+# whichever copy they land on first.+#+# Findings: demo/playbooks/FTX_PROBE_2026-08-12.md+#+# HISTORICAL CAPTURES ARE ALLOWED, BUT MUST SAY SO+# Pages that publish a dated captured run (demo/index.html) legitimately carry an+# old number — rewriting it would falsify a real measurement. Mark those with an+# inline HTML comment immediately before the figure:+#+# <!-- snapshot:YYYY-MM-DD -->1305 / 1329+#+# and state the current value nearby in prose. The marker is what distinguishes+# "deliberately historical" from "nobody updated this".+#+# USAGE+# .github/scripts/landing-constants-guard.sh # check+# CONFORMANCE=1366/1369 .github/scripts/landing-constants-guard.sh+#+# ENV OVERRIDES+# LANDING root of the site tree (default: landing)+# CONFORMANCE canonical "passed/total" (default: read from below)+# VERSION expected release stamp, e.g. rc.15 (default: newest git tag)++set -uo pipefail++LANDING="${LANDING:-landing}"++# Canonical ES-YAML conformance. Source of truth is main's own CI job+# ("ES-compat YAML conformance"), NOT any doc — per CLAUDE.md the pass count grows+# as cases are added and only "0 failed" is invariant. Update this line when CI does.+CONFORMANCE="${CONFORMANCE:-1366/1369}"+CONF_PASS="${CONFORMANCE%%/*}"+CONF_TOTAL="${CONFORMANCE##*/}"++# Newest released tag, unless pinned. Note rc.N existing in CHANGELOG does NOT mean+# it is released — the 2026-08-12 study found rc.16 in the changelog while rc.15 was+# still the latest GitHub release, and footers claiming an unreleased version are+# exactly the kind of drift this guard is for.+if [ -z "${VERSION:-}" ]; then+ VERSION="$(git tag --list 'v1.0.0-rc.*' 2>/dev/null | sort -V | tail -1 | sed 's/^v//')"+fi++fails=0+note() { printf ' %s\n' "$1"; }+fail() { printf 'FAIL %s\n' "$1"; fails=$((fails + 1)); }+pass() { printf 'ok %s\n' "$1"; }++if [ ! -d "$LANDING" ]; then+ echo "FAIL no such directory: $LANDING (run from the repo root)"+ exit 1+fi++echo "landing-constants-guard"+echo " tree=$LANDING conformance=$CONF_PASS/$CONF_TOTAL version=${VERSION:-<unknown>}"+echo++# ---------------------------------------------------------------- conformance ---+# Any "NNNN / NNNN" pair in the 1300-1399 range is treated as a conformance claim.+# Accepts thousands separators and "of" as the separator. Snapshot-marked lines are+# exempt: the marker is a deliberate assertion that the figure is historical.+conf_bad=0+while IFS= read -r hit; do+ [ -z "$hit" ] && continue+ case "$hit" in *"<!-- snapshot:"*) continue ;; esac+ file="${hit%%:*}"+ rest="${hit#*:}"+ line="${rest%%:*}"+ text="${rest#*:}"+ # Pull each conformance-shaped pair out of the line and compare against canonical.+ echo "$text" | grep -oE '1,?3[0-9]{2}[[:space:]]*(/|of)[[:space:]]*1,?3[0-9]{2}' | while read -r fig; do+ got_pass="$(echo "$fig" | sed -E 's/^(1,?3[0-9]{2}).*/\1/' | tr -d ,)"+ got_total="$(echo "$fig" | sed -E 's/.*(1,?3[0-9]{2})$/\1/' | tr -d ,)"+ # "N / N executed cases of the M-case suite" is the honest passed/executed form,+ # not a passed/total claim — accept it when N is the canonical pass count.+ if [ "$got_pass" = "$CONF_PASS" ] && [ "$got_total" = "$CONF_PASS" ]; then+ continue+ fi+ if [ "$got_pass" != "$CONF_PASS" ] || [ "$got_total" != "$CONF_TOTAL" ]; then+ printf '%s:%s: %s\n' "$file" "$line" "$fig"+ fi+ done+done < <(grep -rnE '1,?3[0-9]{2}[[:space:]]*(/|of)[[:space:]]*1,?3[0-9]{2}' \+ "$LANDING" --include='*.html' --include='*.txt' --include='*.json' 2>/dev/null) > /tmp/.lcg_conf.$$ || true++if [ -s /tmp/.lcg_conf.$$ ]; then+ fail "stale ES-YAML conformance figures (canonical: $CONF_PASS / $CONF_TOTAL)"+ while IFS= read -r l; do note "$l"; done < /tmp/.lcg_conf.$$+ note "fix: update to the canonical figure, or mark a dated capture with <!-- snapshot:YYYY-MM-DD -->"+ conf_bad=1+else+ pass "ES-YAML conformance figures agree ($CONF_PASS / $CONF_TOTAL)"+fi+rm -f /tmp/.lcg_conf.$$++# -------------------------------------------------------------------- version ---+# Footer stamps must not advertise a version other than the newest released tag.+#+# Scoped to the site footer (the "XERJ.AI ·" brand line) ON PURPOSE. Version strings+# elsewhere are usually measurement provenance — "XERJ v1.0.0-rc.6 vs Elasticsearch+# 8.13.4" on the benchmarks page names the build that was actually benchmarked, and+# the demo page's captured transcript really did run on rc.1. Rewriting those would+# falsify a real run. Only the footer claims "this is the current release".+if [ -n "${VERSION:-}" ]; then+ vre="$(echo "$VERSION" | sed 's/\./\\./g')"+ bad_ver="$(grep -rniE 'XERJ\.AI[^<]*V1\.0\.0-RC\.?[0-9]+' "$LANDING" --include='*.html' 2>/dev/null \+ | grep -viE "V${vre}([^0-9]|$)" \+ | grep -v '<!-- snapshot:' || true)"+ if [ -n "$bad_ver" ]; then+ fail "version stamps disagree with the newest released tag (v$VERSION)"+ echo "$bad_ver" | head -20 | while IFS= read -r l; do note "${l:0:160}"; done+ n="$(echo "$bad_ver" | wc -l)"; [ "$n" -gt 20 ] && note "... and $((n - 20)) more"+ else+ pass "version stamps match v$VERSION"+ fi+else+ note "skipped version check: no v1.0.0-rc.* tag found"+fi++# ------------------------------------------------------------- operation count ---+# The canonical-operation count must not contradict itself across the agent pages.+ops_six="$(grep -rniE '\b(all )?six (canonical )?(agent )?operations?\b' "$LANDING" --include='*.html' --include='*.txt' 2>/dev/null || true)"+ops_seven="$(grep -rniE '\bseven (canonical )?(agent )?operations?\b' "$LANDING" --include='*.html' --include='*.txt' 2>/dev/null || true)"+if [ -n "$ops_six" ] && [ -n "$ops_seven" ]; then+ fail "the site claims both six and seven canonical agent operations"+ echo "$ops_six" | head -5 | while IFS= read -r l; do note "six: ${l:0:140}"; done+ echo "$ops_seven" | head -5 | while IFS= read -r l; do note "seven: ${l:0:140}"; done+ note "note: xerj_autoindex is the seventh and is CLI-only — it has no HTTP route"+else+ pass "canonical-operation count is self-consistent"+fi++# ----------------------------------------------------------------- kNN honesty ---+# Unfiltered kNN is HNSW-served (approximate, exact-rescored). Claiming it is exact+# is an honest-claims violation. Filtered / aggs-bearing / small-corpus kNN really+# IS exact brute-force, so only flag the unqualified "exact nearest-neighbor" form.+knn_bad="$(grep -rniE 'exact nearest[- ]neighbou?r' "$LANDING" \+ --include='*.html' --include='*.txt' --include='*.json' 2>/dev/null \+ | grep -viE 'rescor|filtered|brute|when|otherwise|below|small' || true)"+if [ -n "$knn_bad" ]; then+ fail "unqualified 'exact nearest-neighbor' claim (unfiltered kNN is HNSW-served)"+ echo "$knn_bad" | while IFS= read -r l; do note "${l:0:160}"; done+else+ pass "kNN exactness claims are qualified"+fi++# ------------------------------------------------------- orphaned agent artifacts ---+# Machine-readable artifacts an agent cannot discover are equivalent to absent ones.+# The 2026-08-12 study found all three tool-schema files reachable only by guessing+# the URL — an agent found them solely by enumerating a local mirror directory.+for artifact in "$LANDING"/docs/agents/schemas/*.json; do+ [ -e "$artifact" ] || continue+ base="$(basename "$artifact")"+ refs="$(grep -rl "$base" "$LANDING" --include='*.html' --include='*.xml' --include='*.txt' 2>/dev/null | grep -v "^$artifact$" || true)"+ if [ -z "$refs" ]; then+ fail "orphaned machine-readable artifact: $artifact (no inbound link; a crawler cannot find it)"+ else+ pass "$base linked from: $(echo "$refs" | tr '\n' ' ')"+ fi+done++echo+if [ "$fails" -gt 0 ]; then+ echo "landing-constants-guard: $fails check(s) failed"+ exit 1+fi+echo "landing-constants-guard: all checks passed"diff --git a/landing/demo/index.html b/landing/demo/index.htmlindex ef9d71b05..8bdb8b5f2 100644--- a/landing/demo/index.html+++ b/landing/demo/index.html@@ -265,7 +265,7 @@ footer .links { display: flex; gap: 28px; flex-wrap: wrap; }<div class="stat"><div class="v">47 ms</div><div class="l">cold start (median, 5 runs)</div></div><div class="stat"><div class="v">519 MB</div><div class="l">RSS · 655 K-doc index</div></div><div class="stat"><div class="v">3.3×</div><div class="l">disk smaller than raw</div></div>- <div class="stat"><div class="v">1305 / 1329</div><div class="l">ES YAML conformance</div></div>+ <div class="stat"><div class="v"><!-- snapshot:2026-04-27 -->1305 / 1329</div><div class="l">ES YAML conformance · 2026-04-27 snapshot (current: 1,366 / 1,369)</div></div></div><p style="font-size: 12px; color: var(--dim); letter-spacing: 0.1em; text-transform: uppercase; margin-top: 18px;">Every number above + every CLI block on this page was captured live on this@@ -1193,8 +1193,10 @@ node demo/scripts/full-demo-flow.js<h2><span class="num">08</span>Drop-in for ES tooling</h2><p class="why">The migration story. Xerj implements the Elasticsearch wire protocol —- 1305 of 1329 official ES YAML conformance tests pass on- <a href="https://github.com/xerj-org/xerj/tree/main/engine/tests/es-compat-yaml">the ES 8.13 test suite</a>.+ <!-- snapshot:2026-04-27 -->1305 of 1329 official ES YAML conformance tests passed on+ <a href="https://github.com/xerj-org/xerj/tree/main/engine/tests/es-compat-yaml">the ES 8.13 test suite</a>+ when this page was captured on 2026-04-27; the suite now stands at+ <strong>1,366 / 1,369 (0 failed, 3 skipped)</strong>.That means existing curl, official ES SDKs (Python, Go, Java, JS), andeven Kibana itself can point at port 9200 unchanged.</p>@@ -1210,9 +1212,10 @@ node demo/scripts/full-demo-flow.js<div class="compare"><div class="xerj-side"><div class="label">Xerj · ES wire-protocol</div>- <div class="num">1305 / 1329</div>- <div class="small">98.2 % of ES 8.13 official YAML conformance tests pass.- The remaining 24 are documented in+ <div class="num"><!-- snapshot:2026-04-27 -->1305 / 1329</div>+ <div class="small">98.2 % of ES 8.13 official YAML conformance tests passed at this+ page's 2026-04-27 capture; the current figure is <strong>1,366 / 1,369</strong>+ (0 failed, 3 skipped). The 24 outstanding at capture time were documented in<a href="https://github.com/xerj-org/xerj/blob/main/engine/reports/PATH_TO_100_PCT_v0.6.0_to_v1.0.md"><code>PATH_TO_100_PCT</code></a>.This includes search DSL, all <!-- generated:query-type-count -->50<!-- /generated:query-type-count --> query types, all aggs, bulk, scroll,index templates, aliases, knn, hybrid, semantic.</div>