Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

docs(rc4-w5): Kibana-feedback delivery scorecard + rc5 Wave 7 (Console/UX) roadmap

xerj-org92738d6ac2f5
diff --git a/demo/playbooks/KIBANA_FEEDBACK_SCORECARD_RC4.md b/demo/playbooks/KIBANA_FEEDBACK_SCORECARD_RC4.md
new file mode 100644
index 000000000..ed47c6f80
--- /dev/null
+++ b/demo/playbooks/KIBANA_FEEDBACK_SCORECARD_RC4.md
@@ -0,0 +1,303 @@
+# XERJ rc4 — Kibana-Feedback Delivery Scorecard & rc5 Console Roadmap
+
+**Date:** 2026-07-22
+**Scope:** shipped rc4 XERJ (engine + bundled Console) scored against the real Kibana user-feedback corpus (`user-feedback/kibana/`, 15,282 artifacts across 15 categories, 107 scored pain points). Companion to `USER_FEEDBACK_SCORECARD_RC4.md` (the ES-engine scorecard) — same method, same rules.
+**Status:** INTERNAL strategy document. This is a capability self-assessment, not marketing. The gaps are the point — do not soften them, and do not paste verbatim competitor/GH/SO/HN quotes into external material.
+**Method:** each pain point was scored against current rc4 code (file:line) and the live binary, NOT the marketing copy.
+DELIVERED = positive proof it works today. PARTIAL = works with named limitations. GAP = the pain is not answered today (rc5 candidate). NA = does not apply to a single-binary/embedded console.
+**Credit where due:** the just-merged console dashboards rework (`CONSOLE_UX_REWORK.md`, merges `9a5e06a` + `040646d`, gate 1360/0/3) flips a large block of this corpus — durable backend CRUD with ETag concurrency, a real panel builder with live preview, post-creation rebinding of viz/index/query, free-form `{x,y,w,h}` grid, and idempotent never-clobber seeding. Several DELIVERED verdicts below exist only because that work landed.
+
+---
+
+## 1. Aggregate delivery
+
+**30 DELIVERED / 60 PARTIAL / 17 GAP / 0 NA** across 107 pain points.
+
+Of the 107 applicable pain points: **28% fully delivered, 56% partial, 16% open gap.** The shape of this scorecard is very different from the ES one. The engine is rarely the problem — cardinality, pipeline aggs, nested queries/aggs/kNN, multi-field sort, composite paging, geo grid aggs, highlighting, IANA time_zone, script_fields, and a real z-score ML pipeline all execute server-side today. The dominant PARTIAL pattern (roughly half of the 60) is **the engine has it, the Console can't express it or wires it wrong** — the exact engine-vs-UI gap Kibana users spent a decade complaining about, now reproduced in miniature. The two structural stories are strong: everything Kibana paywalls (auth, roles, audit, ML, "reporting") ships free in the Apache-2.0 binary, and everything that breaks in Kibana because it is a *separate process* (version lockstep, can't-reach-ES, plugin breakage, saved-object migrations, optimize step) is eliminated by construction. The one category-sized hole is **alerting**: no rule engine, no scheduler, no notification path, and a mock Alerts view that advertises an engine that does not exist.
+
+### Category table
+
+| # | Category | Pain pts | Delivered | Partial | Gap | NA |
+|---|----------|:--------:|:---------:|:-------:|:---:|:--:|
+| 01 | dashboard-authoring | 7 | 2 | 5 | 0 | 0 |
+| 02 | discover-and-kql | 8 | 1 | 5 | 2 | 0 |
+| 03 | visualization-quality | 8 | 2 | 5 | 1 | 0 |
+| 04 | performance-and-load | 7 | 3 | 4 | 0 | 0 |
+| 05 | index-patterns-data-views | 8 | 3 | 5 | 0 | 0 |
+| 06 | sharing-and-export | 6 | 2 | 3 | 1 | 0 |
+| 07 | alerting-and-watcher | 7 | 0 | 3 | 4 | 0 |
+| 08 | spaces-and-rbac | 7 | 2 | 4 | 1 | 0 |
+| 09 | upgrades-and-migration | 7 | 2 | 5 | 0 | 0 |
+| 10 | docs-and-ux | 8 | 3 | 5 | 0 | 0 |
+| 11 | ml-and-anomaly | 7 | 1 | 3 | 3 | 0 |
+| 12 | maps-and-geo | 6 | 1 | 2 | 3 | 0 |
+| 13 | observability-apm | 6 | 3 | 2 | 1 | 0 |
+| 14 | siem-and-security | 7 | 1 | 6 | 0 | 0 |
+| 15 | plugin-and-ecosystem | 8 | 4 | 3 | 1 | 0 |
+| | **TOTAL** | **107** | **30** | **60** | **17** | **0** |
+
+**Ranking by strength:** plugin-and-ecosystem, performance-and-load, observability-apm (logs half), and upgrades-and-migration are strongest — all structural wins. **Alerting-and-watcher is the weakest category on the board (0 DELIVERED)**, followed by maps-and-geo and ml-and-anomaly. The mass in the middle is Console wiring debt: 02/03/05/10/14 are PARTIAL-heavy for the same "engine has it, Console doesn't" reason, and most of those fixes are S/M effort.
+
+---
+
+## 2. Per-category delivery
+
+### 2.1 01-dashboard-authoring (2 DELIVERED / 5 PARTIAL / 0 GAP)
+
+The just-merged console rework genuinely covers the core Kibana dashboard-authoring loop: durable backend CRUD with ETag concurrency, a real panel builder with live preview, post-creation rebinding of chart type/index/query (top-5 corpus asks), free-form grid, and a role-based locking model Kibana took years to ship. The embarrassment is that the category's single highest-scored ask — export chart to image — has a helper and a click handler in the code but no button anywhere in the DOM. The builder also exposes a fraction of what the engine can do.
+
+- **PARTIAL — export chart image / data CSV (the #1 ask):** CSV from search hits works end-to-end (button `xerj-ux/src/ux/charts-ops.js:162` → handler `xerj-ux/src/app.js:1606-1613` → RFC-4180 builder `xerj-ux/src/data/export.js:18-27,46-56`). PNG export is **dead code**: `svgToPng` exists (`export.js:68-123`) and a handler listens for `[data-export-png]` (`app.js:1616-1626`), but no element in all of `xerj-ux/` ever renders `data-export-png`. No CSV from dashboard table/agg panels.
+- **DELIVERED — rebind viz type/index/query after creation (rework credit):** `openBuilder(editingId)` reloads an existing panel's viz type, index, query kind, field, metric, q, interval into the editable builder (`app.js:1102-1136`); `commitBuilder` replaces the panel in place preserving geometry (`app.js:1173-1190`). No separate index-pattern/saved-search object exists to orphan (`xerj-ux/src/data/panel-query.js:73-128`). Persists via PATCH with 409 retry (`xerj-ux/src/data/dashboard-store.js:98-116,340-345`); live-verified in `CONSOLE_UX_REWORK.md`.
+- **PARTIAL — per-panel local time range:** `query.time` is a boolean only — follow global range or ignore time (`panel-query.js:22-23,79`; toggle `app.js:768,1121`); a panel cannot pin its own fixed span. Backend carries `time_default` (`dashboard-store.js:44`) but app.js never reads it.
+- **PARTIAL — query/agg expressiveness (unique counts, NOT-exists, pipeline aggs, second Y-axis):** the ENGINE delivers all of it — cardinality (`engine/crates/xerj-engine/src/aggs.rs:2091`, `fast_aggs.rs:1086`), bucket_selector/bucket_script (`aggs.rs:260-302,433-457`), exists inside bool must_not (`engine/crates/xerj-query/src/parser.rs:290`). The builder exposes 5 query kinds + 4 metrics with one bucket + one inner metric (`panel-query.js:19-23,53,73-128`); no `_exists_:` syntax in query_string (zero grep hits); charts are single-series (`xerj-ux/src/ux/panel-render.js:89-93`).
+- **DELIVERED — dashboard locking / read-only views (rework credit):** four-role model with viewer (`engine/crates/xerj-console-api/src/auth/magic.rs:41,98-100`); `enforce_write` gates managed/default to admin/owner, shared to editor+ (`dashboards.rs:498-520`); seeded defaults undeletable (`dashboards.rs:377-381`); visibility private/shared/default is a first-class enforced doc field (`dashboards.rs:78-87,190,225,299,335-337`); edit is an explicit mode, not the default.
+- **PARTIAL — log/doc table with chosen columns:** `table`/`events` panels render live hits (`panel-render.js:119-146`) but the hits table hardcodes INDEX/ID/SCORE and drops `_source` (`panel-render.js:120-124`); events hardcodes @timestamp/level/message; no column picker in the builder.
+- **PARTIAL — editor churn/fragmentation (Kibana's 3+ competing editors):** exactly one builder over one declarative panel schema and one render path, persisted as plain versioned JSON (`app.js:751-770`, `panel-render.js:53-160`, `dashboards.rs:78-136`). But only 11 of ~23 catalog viz types are data-driven (`panel-render.js:24-27`); the exotic types fall through to **canned demo renders** (`panel-render.js:147-155`) — the catalog offers types a user cannot feed with real data.
+
+### 2.2 02-discover-and-kql (1 DELIVERED / 5 PARTIAL / 2 GAP)
+
+The engine genuinely answers the category's query-semantics pains (real query_string/simple_query_string, ES-style `.keyword` exact match, multi-field server-side sort, exact terms aggs, UTF-8-safe highlighting) — fronted by a Console whose **live wiring is half-finished**. Against a real backend: column sort does nothing, the facet sidebar renders empty, the TIME column is blank, the global time picker is ignored, and two panels are fabricated client-side while their captions claim live data. The rc5 win here is cheap: roughly one week of Console-to-engine wiring, because the engine capability already exists.
+
+- **PARTIAL — doc-table ergonomics (sort/resize/time column):** click-to-sort headers exist (`charts-ops.js:148-153`, `app.js:1588-1597`) but the sort is **mock-only** — the live query body has no sort clause (`xerj-ux/src/data/backends/xerj.js:182-191`) even though the engine fully supports multi-field sort (`engine/crates/xerj-api/src/es_compat.rs:4612,4966-5080`). Live hits map timestamp to `'@timestamp'` (`xerj.js:129`) while the table reads `h._ts` (`charts-ops.js:187`) → blank TIME column. No column resize.
+- **PARTIAL — exact-match vs analyzed / query-language confusion:** engine has distinct term/match/match_phrase/wildcard + Lucene query_string/simple_query_string (`parser.rs:293,303,1049-1135`) and synthetic `.keyword` multi-fields (`es_compat.rs:8803-8837`); the Console shows a live request-DSL panel (`search-discover.js:218-220`). But the Discover box has no free-text query language (term mode = `field=value` regex, match hardcodes `message` — `backends/xerj.js:148-173`), and the "QUERY PLAN · FROM EXPLAIN-PLAN ENDPOINT" panel is **fabricated client-side** (`buildPlan`, `search-discover.js:72-136`) under a caption claiming live data — an honesty defect.
+- **PARTIAL — "Discover shows no results" mystery:** the Console never blanks (graceful fallback `app.js:324-370`, actionable empty state `charts-ops.js:167-169`), but it sidesteps the time-range trap by not implementing it: the live Discover query **ignores the global time picker** (no range clause in `backends/xerj.js:182-191`), and the DATE_HISTOGRAM panel renders mock buckets even in live mode (`search-discover.js:187-194` vs `xerj.js:136-142`).
+- **PARTIAL — field top-values exploration:** right design (click-to-filter facet sidebar via terms aggs, `search-discover.js:178-185`; engine terms/cardinality are exact, `aggs.rs:6-15`) but **broken live**: backend returns facets keyed `by_level/by_service/by_host` (`xerj.js:131-135`) while the UI reads `facets.level/.service/…` (`search-discover.js:180-183`) — every facet renders empty in live mode.
+- **GAP — surrounding log context (a decade-old top-3 ask):** no context/surrounding-documents feature anywhere; primitives (server-side sort, search_after) exist but no surface uses them.
+- **GAP — query-bar autocomplete that doesn't melt the backend:** no suggest implementation in the engine, no typeahead in the Console. XERJ can't reproduce Kibana's cluster-killing suggestion queries — but only because the wanted capability doesn't exist.
+- **PARTIAL — highlighting:** engine highlight is real and multibyte-safe (`engine/crates/xerj-engine/src/index.rs:15548,15700-15880`) but term-by-term — phrases highlight per-token (`index.rs:15652-15666`) — and the Console hits table never renders highlight fragments at all.
+- **DELIVERED — getting to searchable logs at all (no ELK ordeal):** the Kibana-can't-reach-ES failure class is structurally removed — Console served by the engine process itself at `/_xerj-console` on the same :9200 (`engine/crates/xerj-console-api/src/spa.rs:12-14,41-48`); live-verified single-binary boot (`CONSOLE_UX_SMOKE.md:6`).
+
+### 2.3 03-visualization-quality (2 DELIVERED / 5 PARTIAL / 1 GAP)
+
+Strongest of the console categories. Percentages render by default everywhere the #1-scored ask wanted them; the entire legend-truncation and tooltip-occlusion failure classes are designed out via typography-first charts; 22 chart types with post-creation type switching Kibana never had. The honest debits: saved panels of the 11 non-data-driven types silently render canned demo data — a lying-chart bug of exactly the kind this corpus punishes — and global filter pills don't reach user-built panels.
+
+- **PARTIAL — percent-of-total (the highest-scored Kibana viz ask ever):** rendered by default — Dist legend pct (`charts.js:102-108`), TopN pct column (`layout.js:57,71`), table PCT column (`panel-render.js:129-134`). Missing: a computed percentage METRIC (ratio-of-two-queries); metric set is sum/avg/max/min only (`panel-query.js:53`).
+- **DELIVERED — unusable legends:** designed away — no floating legends at all (`charts.js:3-4`); series identity is full-width text (TopN label column `layout.js:55-73`; Dist segments become labeled wrapping micro-metrics `charts.js:101-109`); panels freely resizable (`app.js:667-669`, rework credit).
+- **PARTIAL — color control:** the color-confusion class is deliberately eliminated (monochrome currentColor, rank as opacity — `charts.js:39,67,93,145`), but zero user color assignment and no conditional/threshold coloring; `viz.options` saved but always `{}` (`app.js:759`).
+- **PARTIAL — click-to-filter interactivity:** exists end-to-end for code dashboards (TopN rows emit `data-filter-add` `layout.js:59-63` → global filter pills `app.js:1408-1439` → term clauses `backends/xerj.js:177`). But user-built declarative panels **ignore the global filter pills** — `buildBody` consumes only `ctx.range`, never `ctx.filters` (`panel-query.js:73-127`) — and only TopN rows are clickable despite a stale comment claiming more (`app.js:1405-1407`).
+- **PARTIAL — missing/other terms buckets:** the ENGINE delivers both — terms `missing` param (`aggs.rs:2554-2559`), real `sum_other_doc_count` (`aggs.rs:2780-2785,2837-2838`, semantics verified vs live ES). The CONSOLE throws it away: `normalise()` drops `sum_other_doc_count` (`panel-query.js:144-148`); no missing toggle in the builder.
+- **GAP — field formatters (lookup tables, bytes/duration):** only compact numeric `fmt()` exists; bucket keys and hit fields render raw (`panel-render.js:123,132,140-143`); no formatter concept anywhere in console or engine.
+- **DELIVERED — chart-type poverty and lock-in:** 22-type registry with ADD picker (`chart-types.js:55-356`); horizontal-bar ask answered by TopN ("Replaces horizontal bar", `chart-types.js:142`); post-creation type switching via the QUERY edit button reopening the builder preloaded (`app.js:664,1106-1112,782,757-759`) — rework credit.
+- **PARTIAL — charts break/lie at the edges:** tooltip failure class designed away (values are permanent text, `charts.js:64-74,127-151`; renderer never throws, `panel-render.js:11-13,157-159`). Two real debits: user panels cannot drive multi-series/split-series with live data — stacked/heatmap/multiples/scatter fall through to **canned demo data in a saved panel** (`panel-render.js:24-27,148-155`); no per-point readout on a dense Series.
+
+### 2.4 04-performance-and-load (3 DELIVERED / 4 PARTIAL / 0 GAP)
+
+XERJ structurally wins resource weight, Kibana-server heft, and license distrust — one ~40MB no-JVM binary with an RSS-keyed circuit breaker, a compile-time-embedded console with zero optimize step, and Apache-2.0. The two pains that define "performance under load" are only half-answered, and both are already Wave-1 items on the ES scorecard: the RSS-runaway ticket and mixed read-under-write p99.
+
+- **DELIVERED — ELK too heavy for modest hardware:** single static binary + jemalloc tuned decay (`engine/crates/xerj-server/src/main.rs:25-50`); admission governed against ACTUAL RSS vs cgroup limit (`engine/crates/xerj-engine/src/governor.rs:55-62,92-119`); Console byte-embedded, no Node process (`spa.rs:1-33`); native OTLP + syslog, no Logstash/Beats (`engine/crates/xerj-api/src/router.rs:87-90`). Live-verified idle RSS ~54MB.
+- **PARTIAL — OOM-crash under sustained ingest:** parent circuit breaker on real RSS + memtable budget → 429 not death (`governor.rs:47-119`); disk flood-stage block (`governor.rs:120-134`); per-query 512MB guard (`governor.rs:142-157`). LIMITATION: **RSS-runaway-under-tiny-batch-churn still open** (564MB anon RSS for 200k tiny docs); XERJ itself OOM'd a scale-prover session at 112GB on 2026-07-09. → ES-scorecard Wave 1.
+- **PARTIAL — dashboards slow as data grows:** panel queries hit the O(matching) doc-values columnar fast path (`panel-query.js:85` → `engine/crates/xerj-engine/src/fast_aggs.rs:1-19,62-69,499-602`; 112× date_histogram, 7-30× vs ES). LIMITATION: steady-state only — reads under a high-rate writer (the live-log-dashboard case) remain the one genuine loss, 62-152ms p99 vs ES 2-19ms (`MIXED_READ_UNDER_WRITE_FINDING_2026-07-08.md`). → ES-scorecard Wave 1.
+- **DELIVERED — Kibana's own server heft (optimize step, 30-min bootstrap):** no runtime asset build at all — build.rs embeds `xerj-ux/` at COMPILE time as `include_bytes!` tuples, O(log n) static lookup (`spa.rs:1-45`); vanilla JS, no bundler; deploy = copy one binary.
+- **PARTIAL — enabling security degrades/breaks the stack:** auth is a constant-time byte compare, no realm subsystem, no license gate (`engine/crates/xerj-api/src/auth.rs:99-166`; sessions HMAC ct_eq `auth/sessions.rs:30,93`); health stays auth-exempt. LIMITATION: TLS off by default; `role_descriptors` accepted-but-ignored (every key superuser).
+- **DELIVERED — the 2021 license rug-pull:** Apache-2.0 (`LICENSE:1-3`), no paywalled tier, no license metering code anywhere in engine/crates.
+- **PARTIAL — incremental access (Changes API, agg paging):** agg paging answered — composite agg + after_key semantics (`aggs.rs:2101,5983-6027`; `es_compat.rs:3848-3894,6524-6559`), search_after ships. The Changes-API half is a gap: no `_changes` tail endpoint anywhere; consumers must poll — the exact workaround the 330-score ask complains about.
+
+### 2.5 05-index-patterns-data-views (3 DELIVERED / 5 PARTIAL / 0 GAP)
+
+XERJ's strongest structural move here is deleting the index-pattern abstraction entirely: the three pure-ceremony pains (rebinding, stale mapping caches, .kibana-style bootstrap breakage) are genuinely DELIVERED. But the category's #1 pain by a 6× score margin — nested fields — is only half-answered, and two Kibana mistakes are quietly repeated (hardcoded `@timestamp`, system indices leaking into `*` wildcards).
+
+- **PARTIAL — nested/object fields unusable in the UI:** engine delivers — nested query per-element (`parser.rs:308,2875`; `index.rs:18863-18874`), nested + reverse_nested aggs (`aggs.rs:2139-2146`), nested kNN (`index.rs:6503,16856-16925`). Console does NOT: schema evolution records only top-level keys (`index.rs:10215-10218`, `engine/crates/xerj-common/src/schema.rs:218`), so no dotted sub-path ever appears in the builder datalist (`engine/crates/xerj-console-api/src/data_sources.rs:177-188`; `app.js:789,821-822`) — users hand-type paths blind.
+- **DELIVERED — visualizations welded to their index pattern:** no index-pattern object to orphan — panels carry a plain index string resolved at query time (`panel-query.js:77-78`); builder reopens any panel in EDIT mode with a free-text index input + live datalist (`app.js:800,848,1113-1115,811-812,784`). Rework credit.
+- **DELIVERED — index-pattern ceremony + .kibana bootstrap fragility:** the entity doesn't exist; console state bootstraps idempotently inside the same binary at boot (`engine/crates/xerj-server/src/main.rs:1187-1190`); ETag If-Match makes stale writes 409 instead of corrupting (`dashboards.rs:1-18`); dynamic-mapped data searchable immediately (`index.rs:1352-1382`).
+- **DELIVERED — stale field-list cache:** no persisted field cache anywhere — fields endpoint reads live schema per request (`data_sources.rs:159-189`), schema auto-evolves on ingest (`index.rs:10204-10243`), field_caps computed live (`es_compat.rs:15047-15106`), SPA re-fetches on every builder open (`app.js:1092-1098`).
+- **PARTIAL — time-field friction:** ISO-8601 auto-infers as Date (`schema.rs:227-241`); `date_nanos` accepted but collapsed to plain Date (`engine.rs:1672`). Console hardcodes `@timestamp` for the range filter (`panel-query.js:30,64-66`); `q.timeField` only feeds date_histogram and the builder never exposes it (`app.js:760-769`) — the exact "timestamp not appearing" failure mode.
+- **PARTIAL — system indices pollute user wildcards:** console listings and snapshots hide `.xerj_*` correctly (`data_sources.rs:125-131,169-171`; `engine.rs:1519,1524-1531,1609-1614`), but the ES-compat port repeats Kibana's mistake: `list_indices()` doesn't filter (`engine.rs:1036-1048`), so `*`/`_all` in `_msearch`/`_field_caps` enumerates `.xerj_users/.xerj_sessions/.xerj_dashboards` (`es_compat.rs:20960-20967,15053-15061`), and `*` templates match system indices (`engine.rs:868-876`).
+- **PARTIAL — mapping rigidity / type conflicts:** lenient first-doc-wins inference (`schema.rs:199-241`), `ignore_malformed` on dates (`bulk.rs:1851-1918`), `_reindex` exists (`es_compat.rs:14826,14859`). But the same heuristics can mistype exactly like the corpus complaints, `put_mapping` still forbids type changes (`es_compat.rs:1262-1267`), and the console surfaces no mapping-conflict diagnostics.
+- **PARTIAL — computed/scripted fields dead end:** engine: real per-hit Painless `script_fields` (STUB_AUDIT fix, commit 65ce844), `runtime_mappings` validated (`es_compat.rs:591,744,2189`), scripted_metric + top_metrics (`aggs.rs:2171,2135`). Console: zero surface — count/sum/avg/max/min on a raw field only (`app.js:776`; `panel-query.js:53`).
+
+### 2.6 06-sharing-and-export (2 DELIVERED / 3 PARTIAL / 1 GAP)
+
+XERJ answers the single loudest ask — one-click CSV — but only for the loaded 25-hit page, which is exactly the "export is not my full data" complaint users filed against Kibana Reporting. Chart-to-PNG is three-quarters built but unreachable. Where XERJ genuinely beats Kibana is structural: dashboards are a plain JSON CRUD+bulk API, share URLs are short typed params instead of rison blobs, and the "enabling reporting broke my login / needs a license" failure class cannot occur.
+
+- **PARTIAL — CSV export of results (the #1 Kibana ask of all time, score 372):** real ↓CSV button + RFC-4180 + blob download (`charts-ops.js:162`; `export.js:18-27,33-43`; `app.js:1606-1613`). Limitations: exports only the loaded `size:25` page (`backends/xerj.js:184`), no UTF-8 BOM (the exact Excel/Korean complaint reproduces), TIME column blank on live data (`@timestamp` vs `_ts`). Full-corpus export only via raw scroll/PIT API.
+- **PARTIAL — chart/dashboard image + PDF:** complete client-side SVG→PNG rasterizer (`export.js:68-123`) + wired handler (`app.js:1616-1626`) but **no element ever emits `data-export-png`** — unreachable dead code. No PDF, no print stylesheet, no scheduled reports.
+- **GAP — external embedding (iframe auth):** session cookie hard-coded `SameSite=Lax` with no knob (`auth/sessions.rs:105,114`, rationale `:12`) — cross-site iframes never send it, the exact corpus pain; no embed/chromeless mode, no anonymous role, no signed share token anywhere.
+- **PARTIAL — share-by-URL (rison-blob pain):** dashboard state encodes as short typed query params via URLSearchParams (`app.js:209-254`), `history.replaceState` on change, reapplied on hashchange (`app.js:1905-1916`). Limitations: the Discover query itself lives in localStorage, not the URL (`app.js:372-375`) — a shared link doesn't carry the sender's query; no COPY LINK affordance.
+- **DELIVERED — saved-object import/export as an API (45-vote ask, rework credit):** dashboards are first-class REST resources incl. `POST /api/v1/dashboards/_bulk` whole-set upsert+delete (`router.rs:102-121`; `dashboards.rs:393-423`); panel payload round-trips byte-for-byte (`dashboards.rs:40-46`); If-Match ETags; saved views have their own CRUD. curl is enough.
+- **DELIVERED — reporting as a license-gated add-on that breaks the install:** `/_license` gates nothing (`es_compat.rs:21725-21737`), no license check on any console/export path, export runs purely client-side (`export.js:8-10`) — the failure class is structurally absent.
+
+### 2.7 07-alerting-and-watcher (0 DELIVERED / 3 PARTIAL / 4 GAP)
+
+**XERJ's emptiest category.** The `_watcher` API is a confirmed non-executing CRUD stub (stored JSON, no scheduler, no conditions, no actions, lost on restart), there is zero outbound-notification code anywhere in the binary, and the Console Alerts view is hardcoded mock data advertising a "rules as code" engine that does not exist — an active honesty liability. The strategic sting: this corpus's users left ELK looking for exactly what XERJ already is (a lightweight single binary), and the one feature they left FOR (built-in free alerting) is the one XERJ hasn't built.
+
+- **GAP — free built-in threshold alerting:** `put_watch` stores JSON and returns a canned "condition met: always" (`es_compat.rs:20608-20629`); watches = in-memory DashMap, no persistence, no scheduler (`engine.rs:249,376`); confirmed stub (`STUB_AUDIT.md:134`). No rule-evaluation loop exists anywhere in engine/crates.
+- **PARTIAL — lightweight monitoring/alerting stack:** the lightweight half is delivered and live-verified (~54.5MB idle single binary); the alerting half is absent. XERJ is exactly the box these users want, minus the alerting they want it for.
+- **GAP — alert authoring/debugging UX:** the Console Alerts view is a static mockup — hardcoded "7 open"/"48 rules"/"4 healthy connectors" (`xerj-ux/src/dashboards/alerts.js:36-54`), `Math.random()` fire series (`:59`), invented rules/events (`:79-104`), and a promised directory-watching "rules as code" engine (`:106-134`) that exists in no crate. Shipping this mock is itself an honesty liability.
+- **GAP — notification actions/connectors:** grep for webhook/notify/smtp/slack/pagerduty across engine/crates matches only console-auth user-email fields — zero outbound-notification code. The "4 healthy connectors" line is fiction.
+- **GAP — timezone-aware scheduling:** there is no watch scheduler at all to be timezone-aware; the only recurring loop is the ML datafeed fixed-interval timer (`es_compat.rs:25582`).
+- **PARTIAL — anomaly-based (not just threshold) alerting:** genuinely implemented detect-half — z-score detectors over real buckets with moving baseline (`es_compat.rs:25366-25456`), continuous datafeeds on background tokio tasks (`es_compat.rs:25582,25810-25815`), queryable results, persisted registries (`state.rs:456-457`). Poll-only; the console anomaly view runs a client-side detector self-described as "aspirational".
+- **PARTIAL — reliability across restarts:** ML registries persist and reload (`state.rs:267-374,456-457`) — better than Kibana's encrypted-saved-object fragility. But watches vanish on restart, and a "started" datafeed's scoring task is NOT respawned at boot (`spawn_datafeed_task` only called from `_start`, `es_compat.rs:25814`) — continuous scoring **silently stops after any restart**, precisely the "silent alerts" failure mode.
+
+### 2.8 08-spaces-and-rbac (2 DELIVERED / 4 PARTIAL / 1 GAP)
+
+XERJ's strongest answer is economic and structural: the whole security layer Kibana paywalls ships free and Apache-2.0, and the first-run/lockout ergonomics are genuinely engineered against the corpus's login-breakage horror stories. The honest hole: authorization stops at dashboards — the engine data plane is single-superuser, there is no SSO, and TLS is off by default.
+
+- **DELIVERED — basic security paywalled behind X-Pack:** Apache-2.0 with zero license gates; engine API-key auth with constant-time compare (`auth.rs:39-107,103,166-175`), on by default with auto-generated admin key (`config.rs:291-299`); Console 4-role model (`magic.rs:41`), passkey/WebAuthn (`auth/passkey.rs`), per-IP rate limiting (`auth/rate_limit.rs:19-20`), audit trail to `.xerj_audit` (`auth/audit.rs:17-49`). The exact features Kibana gates behind Gold/Platinum, free.
+- **GAP — enterprise SSO (SAML/OIDC):** no OIDC/SAML implementation anywhere (grep of both Cargo.tomls); `Session.idp` merely reserves the strings (`auth/store.rs:229`); comment says "SSO ships in v1.1" (`auth/me.rs:66-68`). Passkeys are real but are not IdP federation.
+- **PARTIAL — granular RBAC:** console side genuinely enforced (role-gated writes `dashboards.rs:498-522,403-410`; visibility escalation owner-only `:540-552`; invite privilege ceiling `magic.rs:79,98-101`). Data plane all-or-nothing: every engine key is superuser, `role_descriptors` accepted-not-enforced (`auth.rs:130-134`; `es_compat.rs:21597`), roles API returns `"enforced": false` (`native.rs:824-833`), token scopes stored but never checked, any console role can browse every index.
+- **PARTIAL — Spaces multi-tenancy brittleness:** no Spaces; flat per-dashboard visibility model instead (`dashboards.rs:525-532,488-496`), single idempotent seed set that never clobbers (`bootstrap.rs:56-59`, `seed.rs`) — the cross-space copy/desync failure class structurally cannot occur. But zero team/tenant isolation; `org_id` is a dormant placeholder (`dashboards.rs:424-426`).
+- **DELIVERED — auth setup/upgrades break login itself:** idempotent bootstrap mints a single-use 30-min owner magic link, no-op once a user exists (`bootstrap.rs:10-17,41-59`); recovery documented in the banner (`bootstrap.rs:242`); health probes auth-exempt with a regression test (`auth.rs:33,42-44,462-495`); users can't delete their last passkey (`auth/me.rs:66-73`); audit writes best-effort so they can never block login (`auth/audit.rs:3-5`).
+- **PARTIAL — insecure-by-default bundled webapps:** engine auth on by default; Console has no default credentials at all (bootstrap link + passkeys); HMAC'd sessions with 0600 master key; rate-limited auth endpoints. LIMITATION: TLS off by default — cookies and magic links traverse plaintext HTTP out of the box.
+- **PARTIAL — API-key lifecycle:** Console tokens managed well (named, scoped, revocable, plaintext never stored — `auth/tokens.rs:3-10,47-137`). Engine ES-compat keys are create-only over the wire: no GET listing, no DELETE/invalidate endpoint, no metadata (`router.rs:538-545`) — even though the store already carries `invalidated` + expiry the auth path honors (`auth.rs:148-161`).
+
+### 2.9 09-upgrades-and-migration (2 DELIVERED / 5 PARTIAL / 0 GAP)
+
+A category XERJ mostly wins by architecture: one static binary with the console compiled in deletes the two highest-scored pains (deploy sprawl, version lockstep), and there is no startup saved-object migration to fail. The honest caveat: the upgrade-safety story is **machinery without mileage** — only format v1 and rc releases exist, so no real version bump has ever been survived in the field.
+
+- **DELIVERED — stack too complex to deploy:** engine + console SPA on one listener (`router.rs:1-6,42-47`; `spa.rs:21-24`); one-line installer with SHA-256 verify (`landing/get:1-14`); 8-target release matrix (`.github/workflows/release.yml:41-86`); Helm chart.
+- **PARTIAL — upgrades break everything (saved-object migrations):** no startup migration step to fail — dashboards are plain versioned docs; seeding is idempotent and never clobbers edits (`seed.rs:672-706`); concurrent writes 409 loudly (`dashboards.rs:5-15`); data-dir format marker refuses newer/corrupt dirs BEFORE any destructive step (`index_store.rs:500-504,2426`; `segment.rs:243-245`). AGAINST: only format v1 has ever existed — the promise is machinery, not track record; no in-place migrator, no downgrade path, no UPGRADING.md.
+- **DELIVERED — version lockstep + "what version am I on":** console/engine skew impossible by construction (same binary, same listener); real version exposed via `/cluster/info` (`cluster.rs:31-36`) and consumed by the SPA (`data-sources.js:319`). Cosmetic nit: the Footer shows a hardcoded 'v0.1' default (`chrome.js:304`, `app.js:1033`).
+- **PARTIAL — saved-object portability:** full CRUD API round-trips byte-for-byte + admin `_bulk` upsert/delete (`router.rs:109-112`); index pattern embedded inline per panel — nothing to orphan; real snapshot/restore reaches the dashboards index (`es_compat.rs:19631-19673`). AGAINST: no NDJSON file export/import endpoint or Console UI; views have no bulk path; no load-from-filesystem provisioning.
+- **PARTIAL — upgrades wipe customizations:** the seed contract explicitly protects edits — edited/soft-deleted defaults are off-limits to re-seed (`seed.rs:658-665,697-704`); managed docs editable-but-never-deletable (`dashboards.rs:86-92`). AGAINST: only covers dashboards; no branding surface; never exercised across a shipped release boundary.
+- **PARTIAL — migrating data in:** ES wire compat (1,360/1,363 ES-YAML) + local `_reindex` (`es_compat.rs:14826`) + a live-verified scroll+bulk migration recipe (`docs/recipes/production-deployment.md:323-403`). AGAINST: reindex-from-remote explicitly rejected server-side (`es_compat.rs:14866-14885`); no ES-snapshot restore — migration is client-side scripting only.
+- **PARTIAL — saved-object authorization:** per-dashboard authz enforced in code (visibility + role-gated writes, `dashboards.rs:81-92`); every doc carries `org_id`. AGAINST: `org_id` is always "default" — no multi-org enforcement; engine-level RBAC below the console is superuser-only.
+
+### 2.10 10-docs-and-ux (3 DELIVERED / 5 PARTIAL / 0 GAP)
+
+XERJ structurally dodges the biggest gripes rather than patching them: Apache-2.0 kills the license-rage cluster, stdout structured logs kill "where is the error log", the Console never white-screens, and theme is per-user server-side. The honest weakness is a repeating shape: **the engine has the capability but the bundled Console can't express it** — precisely the engine-vs-UI gap Kibana users spent a decade complaining about.
+
+- **DELIVERED — fails dark (blank page, no findable error log):** structured stdout logs via XERJ_LOG/RUST_LOG with JSON mode (`main.rs:463-481`); documented location + journalctl examples (`landing/docs/operations.html:184,164,179`) + dedicated troubleshooting page; Console renders an explicit ERROR scene with stack + RETRY (`app.js:924-928`) and live nav status (`app.js:312-322`).
+- **DELIVERED — dark mode not per-user:** `[data-theme]` night/day (`tokens.css:50-63`), `applyTheme` + localStorage (`app.js:495-497`), `xerj.theme` mirrored to a per-user server-side prefs doc (`xerj-console-sync.js:34-43`; `prefs.rs:19-39`; `router.rs:99-100`).
+- **PARTIAL — text vs .keyword footgun:** aggs and sort strip `.keyword` and fall back (`aggs.rs:2331-2367`; `index.rs:21641-21655`); mappings/field_caps synthesize `.keyword` multi-fields (`es_compat.rs:8803-8841`). LIMITATION: the term-QUERY path lacks the fallback — XERJ's own autoindex catalog documents "term on .keyword subfields returns 0 hits" (`engine/crates/xerj-autoindex/src/catalog.rs:58`). The exact silent-zero-hits failure is still live.
+- **PARTIAL — no machine-readable API spec:** llms.txt + llms-full.txt, agent tool schemas in 3 formats (`landing/docs/agents/schemas/`), MCP server (`xerj-mcp/src/main.rs:222-337`). LIMITATION: no OpenAPI 3.x spec of either REST surface — the literal ask. → ES-scorecard Wave 6.
+- **PARTIAL — install/first-run friction (Windows service):** one-line installers for sh + PowerShell (`landing/get`, `get.ps1:1-16`), Windows MSVC targets, hardened systemd unit, first-launch seeded dashboards so the Console is never empty (`seed.rs:1-69`, rework credit), verified recipes. LIMITATION: no Windows service story at all.
+- **DELIVERED — SSPL relicensing as business risk:** Apache-2.0 at repo root; no dual-license, no gated x-pack analog. Honest caveat: single-vendor, no foundation governance.
+- **PARTIAL — timezone surprises:** engine date_histogram honors ES `time_zone` with full IANA + DST (`aggs.rs:4196-4245`), offset only when explicitly supplied (`aggs.rs:4290-4293`); Console API stores canonical UTC (`time.rs:10-14`). LIMITATION: the panel builder never sends `time_zone` and offers no display-timezone control (`panel-query.js:101-104`) — all Console buckets/axes are UTC-only.
+- **PARTIAL — engine features locked out of the UI:** pipeline aggs (bucket_script/selector/moving_fn/cumulative_sum/derivative/serial_diff, `aggs.rs:232-302`) and multi-field sort (`index.rs:21625-21668`) are implemented; the Console reproduces Kibana's original sin — single-column click-to-sort (`search-discover.js:164`), builder can only express metric/terms/date_histogram (`panel-query.js:93-118`).
+
+### 2.11 11-ml-and-anomaly (1 DELIVERED / 3 PARTIAL / 3 GAP)
+
+XERJ genuinely answers the category's center of gravity — the corpus's loudest complaint is that Elastic anomaly detection is paywalled, needs ML nodes, and is a black box people route around with DIY z-score pipelines — and XERJ ships exactly that DIY answer built-in and free. But it is a basic detector, and the worst self-inflicted wound is that **the Console's own seeded Anomaly dashboard never calls the engine's `/_ml` endpoints** while `_xpack` reports ml `available:false` — both XERJ's own UI and capability-probing clients think the feature doesn't exist.
+
+- **DELIVERED — ML paywalled + dedicated ML nodes:** real free built-in detector in the single binary — full `/_ml` route surface (`router.rs:585-628`); deterministic z-score over date_histogram+metric aggs (`es_compat.rs:24926-24987`); configs persisted (`state.rs:267-292`); documented + recipes. Scope is count/mean/min/max/sum — a fraction of ES ML, but the corpus pain is access, not breadth.
+- **PARTIAL — opaque black-box jobs that fail silently:** scoring is fully explainable (actual/typical/std_dev/z_score per record, `es_compat.rs:25328-25355,25491-25512`). BUT XERJ reproduces the silent-failure pattern: `datafeed_score_pass` swallows all scoring errors (`Err(_) => return (since_ms, 0)`, `es_compat.rs:25543-25544`) — a datafeed against a broken index produces nothing forever with no surfaced error.
+- **GAP — seasonality:** flat trailing mean/std baseline (`es_compat.rs:24968-24976,25328-25339`) — a legitimate daily peak flags as an anomaly every day.
+- **GAP — log categorization / rare-category detection:** functions strictly numeric (`es_compat.rs:24956-24964`, rejects others at `:25083-25095`); no `rare`, no partition_field, no categorize_text anywhere.
+- **GAP — anomaly → notification:** records are queryable (`es_compat.rs:25848+`) but nothing pushes; Watcher is a config-store stub (see 2.7); no webhook/email anywhere.
+- **PARTIAL — anomalies drawn on charts:** seeded Anomaly dashboard with a dedicated AnomalyBand panel (`anomaly-detect.js:19-66`; renderer `charts-ops.js:226-238`; `seed.rs:307,318`). BUT not wired to the engine's ML: the live backend queries a user-populated 'anomalies' index and blends into mock data (`backends/xerj.js:545-580`), never calling `/_ml/*`; the file header still calls the native pipeline "aspirational" (`anomaly-detect.js:4-8`) — stale, since it shipped.
+- **PARTIAL — production-grade continuity/discoverability:** continuous scoring is real (`es_compat.rs:25582-25619,25800-25812`), but started datafeeds reload as 'stopped' after restart (`state.rs:355-357`), results are in-memory only capped at 10k/job (`es_compat.rs:25472`; `state.rs:428-431`), and `_xpack` reports `"ml": {"available": false}` (`es_compat.rs:21523`) while `/_ml` routes exist — probes conclude ML is absent.
+
+### 2.12 12-maps-and-geo (1 DELIVERED / 2 PARTIAL / 3 GAP)
+
+Split verdict, and the split is stark: the ENGINE's geo surface is real and better than expected — geo_point mapping, four geo query types, all five ES geo aggregations with ES-convention geotile keys, test-exercised — which fully answers the highest-scored genuine ask (grid clustering + zoom-to-extent). But the CONSOLE, the thing this Kibana category is actually about, has **zero map capability**.
+
+- **GAP — "I have lat/lon, put it on a map":** the complete chartTypeList (`chart-types.js:350-356`) contains no map/geo panel type; zero hits for leaflet/maplibre/geohash/geotile across xerj-ux; `themes/design-inputs.md:127` records the response as literally "TODO · ASCII region grid".
+- **DELIVERED — server-side geo clustering + zoom-to-extent:** geohash_grid/geotile_grid/geo_distance/geo_bounds/geo_centroid all dispatch and execute (`aggs.rs:2107-2130`; impls `:8717,8855,9832,9867`); geotile keys use ES's z/x/y convention (`aggs.rs:8546-8560`); composite-source support (`aggs.rs:6229,6364-6415`); test-exercised (`integration.rs:669,1410`; `battle_test.rs:571`). Caveat: geo predicates run on the doc-scan path with per-doc haversine (`index.rs:17863,18729`) — correct but O(N), no spatial index.
+- **PARTIAL — geo_shape support:** the geo_shape QUERY exists (`parser.rs:323`; `ast.rs:694-706`; `index.rs:19176-19204`) but only against geo_point docs; no geo_shape FIELD type (`types.rs:180,201`; `engine.rs:1675` maps only geo_point) — you cannot index a polygon.
+- **GAP — distance-ordered results:** no `_geo_distance` sort key anywhere — only the query (`parser.rs:304,1934`) and agg (`aggs.rs:2109,8630`); XERJ can filter within a radius but cannot order by proximity.
+- **PARTIAL — silent geo-data failure:** point parsing deliberately liberal with explicit errors ({lat,lon} / [lon,lat] / "lat,lon" — `parser.rs:2041-2082`; `aggs.rs:8499-8506`), but geohash-string points are silently dropped (`aggs.rs:8505-8506`) and malformed stored points vanish with no diagnostic — the "map shows nothing, nothing says why" class.
+- **GAP — proprietary hosted tile service:** no map exists, so nothing phones home — but nothing is delivered either; design intent ("no offline-tile pain") still marked TODO.
+
+### 2.13 13-observability-apm (3 DELIVERED / 2 PARTIAL / 1 GAP)
+
+Two very different halves. The dominant pain — the ELK observability stack is too heavy, so users defect to lighter single-binary alternatives — is exactly what XERJ is, and the log-side plumbing is genuinely shipped. But the APM half is zero (no trace ingest, model, or UI), and the data-lifecycle story is a facade (store-only ILM, dead `xerj-logs` crate) — both already flagged on the ES scorecard.
+
+- **DELIVERED — stack too heavy for log search:** single binary + embedded console (`main.rs:187-189`; `spa.rs:24,113-114`); direct OTLP + syslog ingest, no Logstash/Beats (`router.rs:87-90`; `native.rs:1141-1230`); built-in Prometheus (`router.rs:107`; `metrics.rs:1-50`); Discover-style search view.
+- **GAP — APM/distributed tracing:** the only OTel endpoint is logs-only (`native.rs:1143,1257` — ExportLogsServiceRequest, LogRecords flattened); no span/trace model in xerj-engine, no trace view in xerj-ux. Deliberate scope-out, but since XERJ courts the logs-and-traces defector audience, it is a GAP, not NA.
+- **DELIVERED — fragile agent/collector chain:** answered structurally by deleting the chain — shippers POST directly (OTLP `router.rs:88`, syslog `:89-90`, ES-compatible `_bulk` so existing Beats/Vector/Fluent Bit configs work unchanged). No fleet/agent-policy subsystem to break. Named limitation: no XERJ-managed host-agent story.
+- **PARTIAL — retention/rollover/lifecycle:** data streams + manual rollover are real (`engine.rs:1165-1218`; `router.rs:398-403`) but nothing is automatic: ILM CRUD only stores JSON (`es_compat.rs:19047-19056`, no executor task); `retention_days` has zero runtime enforcement (`config.rs:737,747`); `xerj-logs` (containing retention.rs) is a dead dependency referenced by no code. → ES-scorecard Wave 2 (ILM executor + xerj-logs wiring).
+- **PARTIAL — log-parsing ergonomics:** real transform-pipeline engine with Grok/JsonParse/TimestampParse processors (`pipeline.rs:15-16,197,215,268`; `engine.rs:1361-1374`) on native + ES routes. Named limitations: per-item pipeline on `_bulk` is a confirmed gap (`STUB_AUDIT.md:94` — shippers silently bypass processors), and no multiline-join or stacktrace-fingerprint processor.
+- **DELIVERED — engine self-observability:** structured JSON self-logs (`main.rs:463-481`); full ES-shaped per-query `profile` response — query + aggregations + fetch phases (`es_compat.rs:2171,2315,4666-4667,10596-10900`); Prometheus + liveness/readiness probes.
+
+### 2.14 14-siem-and-security (1 DELIVERED / 6 PARTIAL / 0 GAP)
+
+Where the pain is "the SIEM stack is heavy, license-poisoned, slow, and a CVE treadmill," XERJ genuinely lands. Where the pain is the actual SIEM job — detection rules that fire actions, least-privilege analyst access, a complete audit trail — **XERJ is scaffolding wearing real clothes**: store-only Watcher, mock Alerts page, explicitly unenforced RBAC, and an audit log with exactly one call site.
+
+- **DELIVERED — all-in-one self-hosted SIEM box:** single binary + bundled Console (`spa.rs:12-14,41-46`); Apache-2.0; in-process rustls TLS with auto self-signed cert + 0600 key tightening (`main.rs:343-385,204`); auth-on posture with exempt health probes (`auth.rs:23-44`).
+- **PARTIAL — CVE treadmill:** structurally smaller surface (no JVM, no plugins, vanilla-JS console); passkey auth with anti-enumeration fake challenges (`auth/login.rs:1-11`); rate limiting; constant-time compares. LIMITATION: no cargo-audit/deny gate, no fuzz harnesses (→ ES-scorecard Wave 4), and no CSP header on the served SPA.
+- **PARTIAL — detection rules that actually fire:** real half — ML anomaly detection with continuous datafeeds (`es_compat.rs:24956-24987,25767-25818`). Missing half — Watcher is a store-only stub (`es_compat.rs:20608-20629`); `.xerj_alert_rules`/`.xerj_alert_fires` are seeded but nothing ever writes fires (`indices.rs:32-33`); the Console Alerts page renders hardcoded mock numbers (`alerts.js:38-41`).
+- **PARTIAL — slow at SIEM volumes:** doc-values prefilters + prefilter cache give single-digit-ms uncached segment reads (`index.rs:601-609,7893,8351`); slow-query capture built in (`es_compat.rs:8042-8053`). LIMITATION: read-latency tails under heavy writes (the ingest-plus-triage pattern) are the open mixed-p99 defect. → ES-scorecard Wave 1.
+- **PARTIAL — least-privilege analyst access:** auth is real (admin key + minted API keys, HTTP+gRPC identical, `auth.rs:87-107`) but authorization is binary — rbac.rs's own header says "NOT ENFORCED… every authenticated caller is superuser" (`rbac.rs:1-12`). A SOC cannot hand an analyst a read-only credential today. → ES-scorecard Wave 4.
+- **PARTIAL — audit trail:** a tamper-evident SHA-256 hash-chained WORM audit log exists, queryable + verifiable (`audit.rs:1-18,108-119`; `router.rs:121-122`). LIMITATIONS proven in code: exactly ONE append call site (search only, subject hardcoded "anonymous", `es_compat.rs:8056-8062`); in-memory 4096-entry ring; chain breaks after rotation; resets on restart.
+- **PARTIAL — fragile shipper onboarding:** the data path is delivered via wire compat (`_bulk`, data streams + rollover, ingest pipelines that execute — `es_compat.rs:1667-1685,1792,1913`); deliberately no Fleet equivalent — removes the Fleet-breakage class but leaves onboarding manual and undocumented.
+
+### 2.15 15-plugin-and-ecosystem (4 DELIVERED / 3 PARTIAL / 1 GAP)
+
+This category's loudest signal is license rage (the AWS-fork threads are 3 of the top 6 items), and XERJ answers it cleanly. The plugin-breakage pain is **eliminated by construction rather than solved** — there is no plugin system, the whole Console is include_bytes'd into the server — a genuine win for install/upgrade complaints, but XERJ also has no answer for the crowd who needed a third-party extension point.
+
+- **DELIVERED — license rage / paywalled essentials:** verbatim Apache-2.0 (`LICENSE:1-2`; `engine/Cargo.toml:26`); the paywalled features ship in the one open binary (full console auth stack + `/_security/api_key`); no license-check code anywhere (grepped).
+- **DELIVERED — plugin fragility (Sense install hell, break-on-every-upgrade):** no plugin loader exists at all; the entire Console is compile-time embedded and served from memory (`spa.rs:1-33`, index.html no-cache so a new binary surfaces the new UI); all views are in-tree modules version-locked in one artifact (`registry.js:15-68`).
+- **PARTIAL — no sanctioned extension/embed point:** in-product customization is now real and durable (dashboards CRUD `dashboards.rs:167-398` + declarative panel builder `panel-query.js:1-30` + `panel-render.js`, live-verified end-to-end — rework credit). LIMITATION: dashboard *modules* are compiled in — a third party cannot add a new view/chart type without rebuilding XERJ; no embed/iframe story.
+- **PARTIAL — built-in query console (Dev Tools):** Discover ships in the binary as an interactive query console — 8 query types, live search, facets, live request-DSL preview (`search-discover.js:2-24,218-219`). LIMITATIONS: only 8 of 24 DSL query types exposed; no freeform raw-REST editor; the query-plan panel is fake (`search-discover.js:72`).
+- **DELIVERED — install/packaging friction:** POSIX-sh installer with SHA-256 verify, "No JVM, no dependencies" (`landing/get:1-13`); `get.ps1`; Helm; no package repos or signing-key dance.
+- **DELIVERED — HTTP/2:** native gRPC listener (h2c by definition, `grpc.rs:9,278`) + REST-over-TLS negotiates h2 via ALPN (`main.rs:440-447,565,581-585`). Nuance: plaintext `--insecure` REST is http/1.1 only.
+- **PARTIAL — admin tasks locked out of the UI (index templates):** engine side real (`_index_template` CRUD `router.rs:323-326`; `_simulate_index` `:679`); Console side not — the Data section's own caption admits mock values (`data.js:51`), no template form anywhere in xerj-ux.
+- **GAP — i18n:** no i18n layer; all UI strings hardcoded English literals (`registry.js:78-84`; `search-discover.js:148`).
+
+---
+
+## 3. Confirmed strengths (code-proven / live-verified wins)
+
+- **The console dashboards rework flips the authoring core.** Durable backend CRUD with ETag optimistic concurrency (POST/PATCH proven on the wire, survives localStorage.clear() AND server restart), real panel builder with live preview, post-creation rebinding of viz/index/query, free-form `{x,y,w,h}` grid, idempotent never-clobber seeding — the top-5 dashboard-authoring corpus asks and the "orphaned visualization" class, closed at Kibana quality under the existing skin (`CONSOLE_UX_REWORK.md`, gate 1360/0/3).
+- **One process, by construction.** Console served by the engine at `/_xerj-console` on :9200: the Kibana-can't-reach-ES, version-lockstep, "server not ready", optimize-step, and plugin-breakage failure classes *cannot occur*. No second service, no bundler, no plugin loader.
+- **No saved-object migration to fail.** Dashboards are plain versioned JSON docs; seeding is idempotent and protects operator edits; concurrent writes 409 instead of corrupting; the data-dir format marker refuses newer/corrupt dirs before any destructive step.
+- **No index-pattern ceremony.** The entity doesn't exist — panels carry a plain editable index string, field lists compute live from an auto-evolving schema (no "refresh field list" button because there is no cache), data is searchable the moment it's ingested.
+- **Everything Kibana paywalls ships free.** API-key auth on by default, 4-role Console with passkey/WebAuthn, invites with a privilege ceiling, rate limiting, audit index, ML anomaly detection, CSV export — all Apache-2.0, zero license-gate code. The "enabling reporting/security broke my install" class is structurally absent.
+- **Login-breakage engineering.** Idempotent bootstrap magic link, banner-documented recovery, auth-exempt health probes (regression-tested), last-passkey guard, best-effort audit writes — designed against the corpus's lockout horror stories.
+- **Design-away wins in visualization.** No floating legends, no hover tooltips, monochrome+opacity encoding: the legend-truncation, tooltip-occlusion/TypeError, and palette-unreadability failure classes are removed, not patched. Percent-of-total renders by default (the #1 viz ask).
+- **The engine outclasses its own UI.** Cardinality, pipeline aggs, nested query/aggs/kNN, multi-field sort, composite paging, exact terms aggs, IANA time_zone date_histogram, UTF-8-safe highlighting, script_fields/runtime_mappings, all five ES geo aggs, ES-shaped `profile` — implemented and mostly test-exercised. Most Kibana-corpus PARTIALs are Console wiring, not engine work.
+- **Real free ML anomaly detection.** Deterministic, explainable z-score detectors with continuous background datafeeds and persisted configs — the built-in answer to the paywalled-ML complaint, in the same ~54MB binary.
+- **Share URLs stay human-short** (typed query params, not rison blobs) and saved objects are a curl-able JSON CRUD+bulk API — the 45-vote import/export-as-API ask, delivered.
+
+---
+
+## 4. Honest gaps — where XERJ does NOT yet replace Kibana
+
+Stated without spin. Three of these are honesty liabilities in the shipped binary, not just missing features.
+
+1. **Alerting does not exist (0 DELIVERED — the worst category).** `_watcher` is a non-executing CRUD stub; there is zero outbound-notification code in the binary; watches vanish on restart. The corpus's users left ELK *for a lightweight box with free built-in alerting* — XERJ is the box without the alerting.
+2. **The Console ships mock surfaces that claim to be live — an active honesty liability.** The Alerts view hardcodes "7 open / 48 rules / 4 healthy connectors" and advertises a "rules as code" engine that exists in no crate; the Discover query-plan panel is fabricated client-side under a "FROM EXPLAIN-PLAN ENDPOINT" caption; the date-histogram panel renders mock buckets in live mode; the Data admin section admits its values are mock; the Anomaly view computes client-side and never calls `/_ml`; 11 of 22 chart types render canned demo data in saved panels. This is exactly the lying-chart / silent-failure class the corpus punishes hardest.
+3. **Discover's live wiring is half-finished.** Against a real backend: column sort silently does nothing, the facet sidebar renders empty (key mismatch), the TIME column is blank (`@timestamp` vs `_ts`), and the global time picker is ignored. Every one of these is an S-effort fix; together they make the flagship view untrustworthy on live data.
+4. **PNG export is unreachable dead code** (helper + handler, no button) and CSV exports only the loaded 25-hit page with no UTF-8 BOM — reproducing the "export is not my full data" and Excel-encoding complaints verbatim. No PDF/scheduled reporting.
+5. **The engine-vs-UI gap, reproduced.** Cardinality, pipeline aggs, nested sub-fields, multi-field sort, missing/other buckets, time_zone, scripted/runtime fields, per-panel time ranges, table columns — all engine-real, all inexpressible from the builder. Kibana's decade-defining complaint, now XERJ's.
+6. **No maps at all.** The engine ships all five ES geo aggregations; the Console has zero geo panel types. The category's dominant pain ("I have lat/lon, put it on a map") is completely unanswered; also missing: geo_shape field indexing, `_geo_distance` sort.
+7. **Data-plane authorization is binary.** Every engine API key is superuser (`role_descriptors` stored-not-enforced, roles API stamps `enforced:false`), Console token scopes are decorative, any viewer can browse every index. No OIDC/SAML SSO. TLS off by default. (RBAC/TLS → ES-scorecard Wave 4; OIDC is new here.)
+8. **ML detection is real but operationally fragile.** Started datafeeds silently stop on restart, scoring errors are swallowed, results are a 10k in-memory ring lost on restart, and `_xpack` reports ml `available:false` — XERJ's own UI and capability probes think the feature doesn't exist.
+9. **No embedding story.** SameSite=Lax hardcoded, no embed mode, no share/anonymous token — the iframe-auth failure class reproduced.
+10. **No APM.** OTLP ingest is logs-only; no trace model or UI — a gap against the exact single-binary alternatives this corpus praises.
+11. **Repeated Kibana hygiene mistakes:** `.xerj_*` system indices leak into `*` wildcard search/field_caps and `*` template matching (the .kibana pollution class); `@timestamp` hardcoded as the panel time field with no UI escape hatch; term-on-`.keyword` still silently returns 0 hits (documented as a gotcha in XERJ's own autoindex catalog).
+12. **Structural non-answers (deliberate scope, keep positioned honestly):** no third-party extension point (no plugin system is also why upgrades can't break), no Spaces/multi-team isolation (`org_id` dormant), no i18n, no Windows service, no Changes-API tail, no reindex-from-remote.
+
+---
+
+## 5. rc5 candidates from the Kibana corpus — ranked
+
+Ranking = severity × reach (pain points / categories closed) × inverse effort. Items already on the ES-scorecard rc5 roadmap (mixed-p99, RSS-runaway, ILM executor, xerj-logs wiring, role_descriptors, TLS default, audit hardening, OpenAPI, cargo-audit/fuzz, multi-version upgrade proof) are **not repeated** — this list is the net-new Console/UX work. The headline: because the engine already outclasses the UI, most of this list is S/M wiring, and the top three items together would flip ~25 PARTIAL/GAP verdicts.
+
+### Tier 1 — category flips and honesty-critical fixes
+
+| # | Candidate | Effort | Closes |
+|---|-----------|:------:|--------|
+| 1 | **Native alerting rule engine**: persisted JSON rules (query + window + threshold + interval, tz-aware schedule from day one), tokio scheduler over the existing search path, fires written to `.xerj_alert_fires`, one generic templated **webhook action**; bridge ML anomaly records into the same action path; wire (or until then hide/label) the mock Alerts view | L | The entire 07 category (4 GAP + 3 PARTIAL), 14's detection pain, 11's anomaly→notification GAP — the single highest-leverage item on either scorecard |
+| 2 | **Discover live-wiring batch** (6 × S): push sort into the live body; map `@timestamp`→`_ts`; fix the facet key mismatch; add global time range + real date_histogram agg to the live body (drop the mock overlay); request+render highlight fragments; wire the real explain endpoint or delete the fake plan panel | S–M total | 5 of 8 pains in 02 + the mock-histogram/fake-plan honesty defects — ~1 week makes the flagship view trustworthy on live data |
+| 3 | **Mock/demo honesty sweep**: render the ↓PNG button (the helper + handler already exist); saved panels of the 11 non-data-driven viz types show UNCONFIGURED instead of demo data (demo only inside the ADD picker); wire the Data section to live `_cat/indices`/`_mapping`/`_index_template`; point the Anomaly view at `/_ml` and delete the stale "aspirational" comment; flip `_xpack` ml to available:true | S–M | Honesty debt across 01/03/06/11/15 — the lying-chart class this corpus punishes hardest |
+
+### Tier 2 — builder expressiveness (engine-real, UI-inexpressible)
+
+| # | Candidate | Effort | Closes |
+|---|-----------|:------:|--------|
+| 4 | **Panel-builder expressiveness batch**: cardinality ("unique") + percent metrics; columns picker for table/events panels; per-panel time-range override + wire `time_default`; MISSING toggle + OTHER row from `sum_other_doc_count` (already on the wire); threshold colors in `viz.options`; time-field selector; browser `time_zone` on date_histograms; thread `ctx.filters` into panel queries; raw-DSL escape-hatch panel kind | M (batched S items) | ~10 PARTIALs across 01/03/05/10 — closes the reproduced engine-vs-UI gap for the common cases |
+| 5 | **ML operational hardening**: respawn started datafeeds at boot; surface last_error/last_pass on datafeeds; persist anomaly records to an internal index; `_exists_:` in query_string; `.keyword`→base fallback on term/terms queries; exclude `.xerj_*` from wildcard expansion + `*` template matching | S–M | 11's fragility PARTIALs + the two repeated-Kibana-hygiene bugs in 05/10 |
+| 6 | **Sub-field discovery**: flatten object/nested sub-paths into schema evolution, field_caps, and the console fields endpoint so dotted paths appear in the builder datalist (nested-marked) | M | The 6×-margin #1 pain of 05 (engine already executes nested query/aggs/kNN) |
+
+### Tier 3 — new surfaces (larger, deliberate bets)
+
+| # | Candidate | Effort | Closes |
+|---|-----------|:------:|--------|
+| 7 | **MAP panel type** over the already-shipped geotile/geohash grid + geo_bounds aggs; inline-bundled public-domain world geometry (CSP-safe, air-gapped for free) | L | 12's dominant GAP — the engine half is DELIVERED, only the render is missing |
+| 8 | **Embed + share**: read-only signed embed tokens, `?embed=1` chromeless mode, configurable SameSite; plus q/type/index in the share URL + COPY LINK button | L (URL part S) | 06's embedding GAP + share-link PARTIAL |
+| 9 | **OIDC SSO** (single IdP, Authorization Code + PKCE, JIT provisioning onto the existing invite roles — `Session.idp` already reserves it) | L | 08's only GAP; SAML stays out of scope |
+| 10 | **Full-result CSV export** (search_after/PIT streaming loop, progress + row cap, UTF-8 BOM) and log-context viewer (±N docs via two sorted searches) + field/value typeahead from the FST term dictionary with hard caps | M each | 06's #1-ask limitation + 02's two decade-old GAPs |
+| 11 | **NDJSON dashboards+views export/import** endpoint + Console button; optional `--seed-dir` provisioning | M | 09's portability PARTIAL |
+| 12 | **OTLP trace ingest** (ExportTraceServiceRequest → span docs) + trace_id drill-down in Discover; full waterfall UI deferred | M (+L UI) | 13's APM GAP, incremental |
+
+**Explicit non-goals to state in docs rather than half-ship:** third-party plugin system, Spaces-style index-scoped multi-tenancy, i18n, PDF/scheduled reporting via headless browser (print stylesheet instead), SAML.
+
+---
+
+## 6. Strategic read
+
+- **The rework moved the needle — say so, then finish the wiring.** Before the console dashboards rework this scorecard's 01/05/06/15 categories would have been GAP-heavy; durable CRUD + the builder flipped them. But the same audit shows the pattern that remains: **56% of the corpus is PARTIAL, and roughly half of those are Console-to-engine wiring where the engine already works.** Tier 1+2 above is mostly S/M effort and would move the aggregate from 28% to well over 50% delivered — the cheapest big number on either scorecard.
+- **Alerting is the strategic hole, not a feature request.** It is the one category at 0 DELIVERED, it is the stated reason users in this corpus left ELK for Graylog/Loki, and XERJ already owns the thing they wanted it attached to (the lightweight single binary). The rule engine + webhook is rc5's anchor item.
+- **The honesty posture is at risk from the Console, not the docs.** The ES scorecard's honesty debt was marketing numbers; this one's is *shipped mock UI claiming to be live* (Alerts, query plan, demo-data panels, mock Data section). That class must be fixed or labeled before any external "Kibana replacement" claim — it is the exact failure mode this corpus punishes.
+- **Structural wins are already bankable.** One-process console, no index patterns, no saved-object migrations, no plugin breakage, everything-paywalled-ships-free: these are code-proven, live-verified, and unmatched by Kibana. Market those today; the wiring debt is real but shallow.
diff --git a/demo/playbooks/USER_FEEDBACK_SCORECARD_RC4.md b/demo/playbooks/USER_FEEDBACK_SCORECARD_RC4.md
index f5d714be3..1d9f9c444 100644
--- a/demo/playbooks/USER_FEEDBACK_SCORECARD_RC4.md
+++ b/demo/playbooks/USER_FEEDBACK_SCORECARD_RC4.md
@@ -1,7 +1,7 @@
# XERJ rc4 — User-Feedback Delivery Scorecard & rc5 Roadmap
**Date:** 2026-07-12
-**Scope:** shipped rc4 XERJ scored against the real Elasticsearch user-feedback corpus (`user-feedback/`, 41 pain-point files across 15 categories, 98 scored pain points).
+**Scope:** shipped rc4 XERJ scored against the real Elasticsearch user-feedback corpus (`user-feedback/`, 41 pain-point files across 15 categories, 98 scored pain points). Companion: `KIBANA_FEEDBACK_SCORECARD_RC4.md` scores the Console against the Kibana user-feedback corpus (107 pain points) with the same method; its top-ranked candidates feed Wave 7 below.
**Status:** INTERNAL strategy document. This is a capability self-assessment, not marketing. The gaps are the point — do not soften them, and do not paste verbatim competitor/G2/HN/Jepsen quotes into external material.
**Method:** each pain point was scored against current rc4 code (file:line) and the live binary (read-only probes), NOT the marketing copy.
DELIVERED = positive proof it works today. PARTIAL = works with named limitations. GAP = the pain is not answered today (rc5 candidate). NA = does not apply to a single-binary/embedded engine.
@@ -171,6 +171,22 @@ Ranking = severity × reach (how many pain points / clusters it closes) × inver
| **Prove the no-post-refresh global-ordinals spike** (add first-query-after-refresh probe) | LOW | M | query-performance PARTIAL → DELIVERED with evidence |
| **Distroless/scratch container image** (or drop the ~30MB number) | LOW | M | ops-simplicity + resource-cost image-size overclaim |
+### Wave 7 — Console/UX (Kibana-replacement) work
+*Sourced from the Kibana-corpus scorecard (`KIBANA_FEEDBACK_SCORECARD_RC4.md`, 30 D / 60 P / 17 G over 107 pain points). Dominant finding: the engine already has the capability for roughly half the PARTIALs — the Console can't express it or wires it wrong — so most of this wave is S/M wiring. Deduped against Waves 1–6: mixed-p99 + RSS-runaway (W1), ILM/retention executor + `xerj-logs` wiring (W2), `role_descriptors` enforcement + TLS-by-default + tamper-evident audit (W4), and OpenAPI + multi-version-upgrade proof (W6) all recur verbatim in the Kibana corpus — those rows are referenced, not repeated.*
+
+| Candidate | Sev | Effort | Closes |
+|---|---|---|---|
+| **Native alerting rule engine + webhook action** (persisted tz-aware rules → tokio scheduler → fires index → templated webhook; bridge ML anomaly records; wire or hide the mock Alerts view) | HIGH | L | Kibana 07 — the only 0-DELIVERED category (4 GAP), plus SIEM detection and the anomaly→notification gap; the corpus's stated reason for choosing Graylog/Loki over ELK |
+| **Discover live-wiring batch** (sort clause into the live body, `@timestamp`→`_ts`, facet key mismatch, global time range + real date_histogram, render highlights, real-or-deleted query-plan panel) | HIGH | S–M | 5 of 8 pains in Kibana 02 — the flagship view currently no-ops/blanks/fakes against a live backend |
+| **Mock/demo honesty sweep** (render the existing ↓PNG button; UNCONFIGURED instead of canned demo data on saved panels of the 11 non-data-driven viz types; wire the mock Data section to live `_cat`/`_mapping`/`_index_template`; point the Anomaly view at `/_ml`; flip `_xpack` ml available:true) | HIGH | S–M | The shipped-mock-UI honesty debt across Kibana 01/03/06/11/15 — the Console analog of the W6 claims-reconciliation |
+| **Panel-builder expressiveness batch** (cardinality/percent metrics, table columns picker, per-panel time range, missing/other buckets, threshold colors, time-field + browser time_zone, global filters into panels, raw-DSL panel kind) | MED | M | ~10 PARTIALs across Kibana 01/03/05/10 — the reproduced engine-vs-UI gap |
+| **Hygiene + ML hardening batch** (`.keyword`→base fallback on term queries; exclude `.xerj_*` from `*` wildcards/templates; respawn started datafeeds at boot; surface datafeed errors; persist anomaly records; nested sub-paths into the field picker) | MED | S–M | The two repeated-Kibana mistakes (silent 0-hits, system-index leakage) + Kibana 11's silent-stop fragility + 05's #1 nested pain |
+| **MAP panel type** over the already-shipped geotile/geohash/geo_bounds aggs (inline-bundled world geometry, air-gapped by construction) | MED | L | Kibana 12's dominant gap — engine half already DELIVERED |
+| **Embed + share** (signed read-only embed tokens, chromeless mode, configurable SameSite; query in the share URL + COPY LINK) | MED | L | Kibana 06's embedding gap |
+| **OIDC SSO for the Console** (single IdP, PKCE, JIT provisioning onto existing invite roles) | MED | L | Kibana 08's only gap; complements the W4 RBAC enforcement row |
+| **Export completion** (full-result CSV via search_after/PIT stream + UTF-8 BOM; NDJSON dashboards/views export-import + `--seed-dir`) | LOW | M | Kibana 06's #1-ask limitation + 09's portability partial |
+| **Log-context viewer + query-bar typeahead** (±N-docs modal via sorted search_after; FST-backed field/value suggestions with hard caps) | LOW | M | Kibana 02's two decade-old outright gaps |
+
---
## 6. Strategic read
@@ -179,3 +195,4 @@ Ranking = severity × reach (how many pain points / clusters it closes) × inver
- **Wave 3 is the fork in the road.** Multi-node HA is XL and changes XERJ's identity from "the simple single-node ES escape hatch" to "an HA search platform." Until it lands, the single-node SPOF, sub-ES power-loss durability, and no-scale-out ceiling are *positioning constraints, not bugs* — market to the ICP where they don't bite (dev/CI, single-node edge, cost-driven right-sizing) and stop implying HA that doesn't exist.
- **Waves 4–6 are cheap credibility.** The security process gates and the numbers-reconciliation are small and directly protect the honesty posture that is itself a differentiator vs ES's "Documentation ≠ API ≠ Reality."
- **The moat that already exists:** no-JVM CVE/GC elimination, Apache-2.0 trust, ES-wire reversibility, filter-pushdown HNSW, `/_memory`, and GDSR-forcemerge purge are real today and unmatched by ES. Defend and market those; fix the rest honestly.
+- **Console vs Kibana:** the dashboards rework made the Console a credible Kibana replacement for the authoring/ops core (durable CRUD, builder, one-process, no index patterns, everything-paywalled-free), but the Kibana scorecard shows the remaining debt is mostly cheap Console-to-engine wiring plus one category-sized hole (alerting, 0 DELIVERED) and a shipped-mock-UI honesty class that must be fixed or labeled before any external replacement claim — Wave 7 is that work.