Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

security(graph): make a brain a real per-tenant boundary, incl. detached ML-datafeed tasks (#131)

xerj-org3370ef1840d3
diff --git a/docs/design/SECOND_BRAIN_SPEC.md b/docs/design/SECOND_BRAIN_SPEC.md
index e4916724c..54aa43c0d 100644
--- a/docs/design/SECOND_BRAIN_SPEC.md
+++ b/docs/design/SECOND_BRAIN_SPEC.md
@@ -496,10 +496,29 @@ Shared conventions:
are always numbers.
- Error body shape (identical to memory_api but typed `graph_error`):
`{"error": {"type": "graph_error", "reason": "<msg>"}, "status": <n>}`.
-- Auth: process-wide API-key middleware only, same caveat block as
- memory_api's module docs (no per-brain authz; copy the warning).
-- Unknown brain on read endpoints → 404. `link` creates lazily (like
- `ensure_backing_index`).
+- Auth: process-wide API-key middleware **plus** per-brain authorization
+ (issue #79). A brain authorizes against its edges index,
+ `.xerj-memory-{brain}-edges`, which sits in the reserved `.xerj-memory-*`
+ namespace that `xerj-api::authz` gates on every surface — the graph API,
+ `/_memory/*`, the raw ES-compat index routes and the native
+ `/v1/indices/{name}/…` routes alike. The decision is made against the index
+ the handler **actually touches**, not the one in the URL: a route that takes
+ its index from the body (`_bulk` action lines, `_msearch` headers, `_mget`
+ `docs[]._index`, `_aliases`, `_reindex`, `terms`/`lookup` joins) is
+ authorized against what the body names, aliases are resolved to their backing
+ index before the decision, and `xerj_engine::index_guard` re-checks at the
+ point a name becomes an index so a route nobody thought of still fails
+ closed. Reach: open mode (`--insecure`,
+ point-at-a-folder) and the admin key get every brain; a key minted with
+ `role_descriptors` naming the index gets that brain at the granted
+ privilege; a key minted without them gets **no** brain. `ego`/`overview`
+ need `read`, `link`/`unlink` need `write`, and reading node summaries
+ additionally needs `read` on the resolved nodes index.
+- Unknown brain on read endpoints → 404 **for a caller authorized for it**.
+ An unauthorized caller gets 403 whether or not the brain exists — the
+ authorization check runs before the existence probe, so the status code
+ cannot be used to enumerate brains. `link` creates lazily (like
+ `ensure_backing_index`), which is part of `write`, not a separate `manage`.
### 4.1 POST /_graph/{brain}/link — assert an edge
diff --git a/engine/crates/xerj-api/src/auth.rs b/engine/crates/xerj-api/src/auth.rs
index 5a5cc08e5..45a36326f 100644
--- a/engine/crates/xerj-api/src/auth.rs
+++ b/engine/crates/xerj-api/src/auth.rs
@@ -1,4 +1,4 @@
-//! API key authentication middleware.
+//! API key authentication middleware — *who* the caller is.
//!
//! Checks the `Authorization` header for any of:
//! - `Authorization: ApiKey <key>`
@@ -7,14 +7,21 @@
//!
//! When `config.auth.enabled` is `false` (or `--insecure` mode was set by
//! clearing the key), the check is skipped entirely.
+//!
+//! Authentication answers *who*; [`crate::authz`] answers *what they may
+//! touch*. The bridge between them is [`Principal`], produced here by
+//! [`authenticate`] and consulted there. `is_authorized` — the boolean the
+//! gRPC interceptor and the HTTP middleware share — is now a thin wrapper over
+//! `authenticate`, so the two surfaces cannot drift.
use axum::{
- extract::{Request, State},
- http::StatusCode,
+ extract::{FromRequestParts, Request, State},
+ http::{request::Parts, StatusCode},
middleware::Next,
response::{IntoResponse, Response},
Json,
};
+use xerj_engine::rbac::{Privilege, Role};
use crate::error::{EsErrorBody, EsErrorResponse, EsRootCause};
use crate::state::AppState;
@@ -33,6 +40,111 @@ use crate::state::AppState;
/// `"live"`/`"ready"` string), so leaving them open is safe.
pub const AUTH_EXEMPT_PATHS: [&str; 2] = ["/health/live", "/health/ready"];
+// ─────────────────────────────────────────────────────────────────────────────
+// Principal
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// The authenticated identity behind a request.
+///
+/// Authentication used to collapse to a bool, which is exactly why a brain was
+/// not a security boundary: with no principal on the request there was nothing
+/// for an authorization check to consult (issue #79). Every variant below is a
+/// *capability statement*, and the ordering is deliberate — the further down
+/// the list, the less a caller may reach.
+#[derive(Clone, Debug, PartialEq, Eq)]
+pub enum Principal {
+ /// Full access. Two ways to be one, and only two:
+ /// - **open mode** — `auth.enabled = false` or no admin key configured.
+ /// This is `xerj --insecure` and the point-at-a-folder first run, where
+ /// there is exactly one user and no configuration; it must keep working
+ /// with zero setup, so it is deliberately unrestricted.
+ /// - the configured **admin/superuser key** (`auth.admin_api_key`).
+ Superuser,
+ /// A key minted by `POST /_security/api_key` **with** usable
+ /// `role_descriptors`. Confined: it may touch only the indices its roles
+ /// name, and nothing that is not index-scoped.
+ Scoped {
+ /// The minted key's id (for audit/log lines, never compared).
+ key_id: String,
+ /// Index grants parsed from `role_descriptors`. Never empty — an
+ /// empty grant list is [`Principal::Unscoped`].
+ roles: Vec<Role>,
+ },
+ /// A key minted **without** `role_descriptors` — the shape every key had
+ /// before per-brain authorization existed.
+ ///
+ /// It keeps its historical reach over the general ES-compat surface (so
+ /// hardened Kibana/Beats deployments do not break), but it holds **no**
+ /// privilege on the reserved `.xerj-memory-*` namespace. That is the
+ /// fail-closed half: an unconfigured principal is denied the brain, not
+ /// granted it.
+ Unscoped {
+ /// The minted key's id.
+ key_id: String,
+ },
+ /// No valid credential. Normally unreachable inside a handler — the auth
+ /// middleware answers 401 first — but it is what the [`Principal`]
+ /// extractor yields if a handler is ever mounted without that middleware,
+ /// so the default is "deny", never "allow".
+ Denied,
+}
+
+impl Principal {
+ /// Does this principal hold `privilege` on `index`?
+ ///
+ /// This is the single authorization predicate; [`crate::authz`] routes
+ /// every enforcement point through it. `Unscoped` answers `true` for
+ /// ordinary indices (its historical reach) and `false` for the reserved
+ /// namespace, which [`crate::authz::is_reserved_index`] defines.
+ pub fn allows_index(&self, index: &str, privilege: Privilege) -> bool {
+ match self {
+ Principal::Superuser => true,
+ Principal::Scoped { roles, .. } => roles.iter().any(|r| r.allows(index, privilege)),
+ Principal::Unscoped { .. } => !crate::authz::is_reserved_index(index),
+ Principal::Denied => false,
+ }
+ }
+
+ /// True for the one principal that is allowed to see and do everything —
+ /// used to skip the authorization work entirely on the local-dev path.
+ pub fn is_superuser(&self) -> bool {
+ matches!(self, Principal::Superuser)
+ }
+
+ /// Short identity string for error/log lines. Never contains key material.
+ pub fn label(&self) -> &str {
+ match self {
+ Principal::Superuser => "superuser",
+ Principal::Scoped { key_id, .. } | Principal::Unscoped { key_id } => key_id.as_str(),
+ Principal::Denied => "unauthenticated",
+ }
+ }
+}
+
+/// Extract the principal straight from the request's `Authorization` header.
+///
+/// Deliberately *re-derives* rather than reading a request extension planted
+/// by the middleware: a handler that authorizes must be safe by construction
+/// even if it is mounted on a router where the middleware was forgotten. The
+/// cost is one map lookup plus a constant-time compare. When no credential
+/// resolves, the result is [`Principal::Denied`] — never an error that a
+/// caller could turn into a bypass.
+#[axum::async_trait]
+impl FromRequestParts<AppState> for Principal {
+ type Rejection = std::convert::Infallible;
+
+ async fn from_request_parts(
+ parts: &mut Parts,
+ state: &AppState,
+ ) -> Result<Self, Self::Rejection> {
+ let header = parts
+ .headers
+ .get(axum::http::header::AUTHORIZATION)
+ .and_then(|v| v.to_str().ok());
+ Ok(authenticate(state, header))
+ }
+}
+
/// Axum middleware that enforces API key authentication.
///
/// Call via `middleware::from_fn_with_state(state, auth_middleware)` in the
@@ -90,15 +202,27 @@ pub async fn auth_middleware(State(state): State<AppState>, req: Request, next:
/// value (e.g. `"ApiKey abc"`, `"Bearer abc"`, or `"Basic base64(...)"`), or
/// `None` when absent.
pub fn is_authorized(state: &AppState, auth_header: Option<&str>) -> bool {
+ !matches!(authenticate(state, auth_header), Principal::Denied)
+}
+
+/// Resolve the request's credential to a [`Principal`].
+///
+/// Same acceptance rules as [`is_authorized`] (which is now defined in terms of
+/// this), but it reports *which* identity authenticated so authorization has
+/// something to decide against. Returns [`Principal::Denied`] when nothing
+/// authenticates.
+pub fn authenticate(state: &AppState, auth_header: Option<&str>) -> Principal {
let cfg = &state.config.auth;
- // Skip auth when disabled or no admin key is configured.
+ // Open mode: auth disabled or no admin key configured. This is the
+ // `--insecure` / point-at-a-folder posture — one user, no configuration —
+ // and it stays fully unrestricted on purpose.
if !cfg.enabled || cfg.admin_api_key.is_empty() {
- return true;
+ return Principal::Superuser;
}
let Some(header) = auth_header else {
- return false;
+ return Principal::Denied;
};
if let Some(key) = extract_key(header) {
@@ -109,12 +233,14 @@ pub fn is_authorized(state: &AppState, auth_header: Option<&str>) -> bool {
// the single most valuable credential in the system — must not be
// weaker. `constant_time_eq` still returns early on a length mismatch,
// which only reveals the key *length*, matching the created-key path.
- return constant_time_eq(key.as_bytes(), cfg.admin_api_key.as_bytes())
- // A key minted by `POST /_security/api_key`.
- || authenticate_api_key(state, key);
+ if constant_time_eq(key.as_bytes(), cfg.admin_api_key.as_bytes()) {
+ return Principal::Superuser;
+ }
+ // A key minted by `POST /_security/api_key`.
+ return authenticate_api_key(state, key);
}
- basic_authorized(state, header)
+ basic_principal(state, header)
}
/// Validate `Authorization: Basic <base64(user:pass)>`.
@@ -128,29 +254,29 @@ pub fn is_authorized(state: &AppState, auth_header: Option<&str>) -> bool {
/// - or, treating the whole decoded `user:pass` as `id:secret`, a key minted
/// by `POST /_security/api_key` (its `id:secret` shape already matches
/// `user:pass` exactly, so no separate encoding is needed here).
-fn basic_authorized(state: &AppState, header: &str) -> bool {
+fn basic_principal(state: &AppState, header: &str) -> Principal {
let Some(prefix) = header.get(..6) else {
- return false;
+ return Principal::Denied;
};
if !prefix.eq_ignore_ascii_case("basic ") {
- return false;
+ return Principal::Denied;
}
let encoded = header[6..].trim();
let decoded = match crate::es_compat::base64_decode(encoded) {
Some(d) => d,
- None => return false,
+ None => return Principal::Denied,
};
let decoded = match std::str::from_utf8(&decoded) {
Ok(s) => s,
- Err(_) => return false,
+ Err(_) => return Principal::Denied,
};
let Some((user, pass)) = decoded.split_once(':') else {
- return false;
+ return Principal::Denied;
};
let cfg = &state.config.auth;
if constant_time_eq(pass.as_bytes(), cfg.admin_api_key.as_bytes()) {
- return true;
+ return Principal::Superuser;
}
check_minted_key(state, user, pass)
}
@@ -178,20 +304,21 @@ fn metrics_token_authorized(state: &AppState, req: &Request) -> bool {
///
/// The presented value is `base64("id:api_key")`. Decode it, split on the
/// first `':'`, look the id up in the engine's key store, and constant-time
-/// compare the secret. All keys authenticate as the single superuser
-/// (role_descriptors are accepted at creation but not enforced).
-fn authenticate_api_key(state: &AppState, presented: &str) -> bool {
+/// compare the secret. The resulting principal carries the key's stored
+/// grants: [`Principal::Scoped`] when it was minted with usable
+/// `role_descriptors`, [`Principal::Unscoped`] otherwise.
+fn authenticate_api_key(state: &AppState, presented: &str) -> Principal {
let decoded = match crate::es_compat::base64_decode(presented) {
Some(d) => d,
- None => return false,
+ None => return Principal::Denied,
};
let decoded = match std::str::from_utf8(&decoded) {
Ok(s) => s,
- Err(_) => return false,
+ Err(_) => return Principal::Denied,
};
let (id, secret) = match decoded.split_once(':') {
Some(parts) => parts,
- None => return false,
+ None => return Principal::Denied,
};
check_minted_key(state, id, secret)
}
@@ -199,21 +326,37 @@ fn authenticate_api_key(state: &AppState, presented: &str) -> bool {
/// Look up a minted `POST /_security/api_key` credential by `(id, secret)`.
/// Shared by the `ApiKey <base64(id:secret)>` and `Basic <base64(id:secret)>`
/// paths — both decode to the same `id:secret` shape.
-fn check_minted_key(state: &AppState, id: &str, secret: &str) -> bool {
+///
+/// A minted key is **never** a superuser: the strongest thing it can be is
+/// `Unscoped`, which has no reach into the reserved `.xerj-memory-*`
+/// namespace. That is what stops "any authenticated caller reads any brain".
+fn check_minted_key(state: &AppState, id: &str, secret: &str) -> Principal {
let record = match state.engine.api_keys.get(id) {
Some(r) => r,
- None => return false,
+ None => return Principal::Denied,
};
if record.invalidated {
- return false;
+ return Principal::Denied;
}
if let Some(exp) = record.expiration_ms {
let now_ms = chrono::Utc::now().timestamp_millis().max(0) as u64;
if now_ms >= exp {
- return false;
+ return Principal::Denied;
+ }
+ }
+ if !constant_time_eq(record.secret.as_bytes(), secret.as_bytes()) {
+ return Principal::Denied;
+ }
+ if record.roles.is_empty() {
+ Principal::Unscoped {
+ key_id: id.to_string(),
+ }
+ } else {
+ Principal::Scoped {
+ key_id: id.to_string(),
+ roles: record.roles.clone(),
}
}
- constant_time_eq(record.secret.as_bytes(), secret.as_bytes())
}
/// Length-independent-only constant-time byte comparison. Avoids leaking the
diff --git a/engine/crates/xerj-api/src/authz.rs b/engine/crates/xerj-api/src/authz.rs
new file mode 100644
index 000000000..f5a905ccc
--- /dev/null
+++ b/engine/crates/xerj-api/src/authz.rs
@@ -0,0 +1,2639 @@
+//! Per-resource authorization — *what* an authenticated caller may touch.
+//!
+//! [`crate::auth`] answers *who* the caller is ([`Principal`]); this module is
+//! the decision function and the enforcement points. It exists because of
+//! issue #79: a brain used to be isolated **by name only**, so any credential
+//! that could reach the node could read, write and destroy every brain.
+//!
+//! ## The rule, in one sentence
+//!
+//! The `.xerj-memory-*` index namespace — every agent-memory namespace and
+//! every second brain — is **reserved**: reaching it requires a principal that
+//! holds the matching privilege on the specific index, and the only principals
+//! that hold anything there are the superuser and a key explicitly granted the
+//! index by name.
+//!
+//! ## The mistake this file is built to make impossible
+//!
+//! The first cut of #79 decided against the index named in the **URL path**.
+//! That is not the index several handlers operate on. `_msearch` takes it from
+//! an NDJSON header line, `_bulk` from an action line, `_mget` from
+//! `docs[]._index`, `_aliases` from its action list, `_reindex` from
+//! `source`/`dest`, a `terms` lookup and a `lookup` runtime field from deep
+//! inside a query, an index template from a *pattern* that will match brains
+//! created later, `_sql` from the table name in the statement — in every one of
+//! those the path index is only a *default* that the body overrides. Four of
+//! them were proven live against a running node: read another tenant's brain
+//! through `_msearch`, forge and delete its edges through `_bulk`, read a
+//! document through `_mget`, and launder a whole index into reach by pointing
+//! an alias at it.
+//!
+//! A per-handler patch list would have closed those four and left the fifth.
+//! So authorization is decided at two places that no handler can route around:
+//!
+//! 1. **Here, before the handler runs.** [`authz_middleware`] resolves the
+//! complete target set of a request — from the path *and* from the body —
+//! resolves aliases to concrete indices, and authorizes each one with the
+//! privilege the request actually needs. This produces the precise
+//! ES-shaped 403 and gets read-vs-write-vs-manage right.
+//! 2. **In the engine, at the point the name becomes an index.**
+//! [`xerj_engine::index_guard`] installs the request's principal as a
+//! task-local visibility rule that `Engine::get_index`,
+//! `get_or_create_index`, `create_index`, `delete_index`,
+//! `list_indices` and `index_name_list` all consult. A handler cannot
+//! forget this check, because a handler does not call it — it is inside the
+//! only funnel to index data. Anything the first layer did not know to look
+//! for (a body shape added next year, an `_sql` table name, a scroll
+//! context) still fails closed there.
+//!
+//! Layer 2 answers "not found" rather than "forbidden", deliberately: it makes
+//! a denied brain indistinguishable from an absent one, and it means fan-out
+//! (`POST /_search`, `_cat/indices`, `logs-*`) *filters* instead of failing —
+//! which is what lets the global verbs keep working for ordinary credentials
+//! instead of being refused wholesale.
+//!
+//! ## Enforcement points (all of them)
+//!
+//! | Door | Enforced by |
+//! |---|---|
+//! | `POST /_graph/{brain}/link` | `graph_api::link` + middleware |
+//! | `DELETE /_graph/{brain}/link/{edge_id}` | `graph_api::unlink` + middleware |
+//! | `GET /_graph/{brain}/ego` | `graph_api::ego` + middleware |
+//! | `GET /_graph/{brain}/overview` | `graph_api::overview` + middleware |
+//! | `POST/GET/DELETE /_memory/{ns}[/…]` (the brain's *nodes* index) | `memory_api` + middleware |
+//! | any ES-compat route naming a reserved index in the path | middleware |
+//! | any route naming an index in its **body** (`_bulk`, `_msearch`, `_mget`, `_aliases`, `_reindex`, snapshot `indices`, `terms` lookups, `lookup` runtime fields, `POST /v1/indices`) | middleware ([`body_targets`]) |
+//! | an index template whose `index_patterns` would own a brain's mapping | middleware ([`authorize_template_patterns`]) |
+//! | an ML job/datafeed config naming the index the server will go and read | middleware ([`BodyShape::MlConfig`]) |
+//! | the detached datafeed scorer `_start` spawns | `es_compat::spawn_datafeed_task`, carrying the starter's rule across the `tokio::spawn` |
+//! | an **alias** pointing into the reserved namespace | middleware (resolved before the decision) + `Engine::get_index` (resolved before the guard) |
+//! | index patterns (`logs-*`, `_all`, `*`) | expanded only over the principal's visible set ([`xerj_engine::index_guard`]) |
+//! | unnamed fan-out (`POST /_search`, `/_bulk`, `/_all/*`) | authorized per named index; anything unnamed is filtered by the guard |
+//! | enumeration (`GET /_mapping`, `/_cat/indices`, `/_resolve/index/*`) | filtered at `Engine::list_indices`, then pruned in the response |
+//! | the native router's `/v1/indices/{name}/…` spelling of the same index | middleware ([`Target::Indices`] classifies both routers) |
+//! | the native router's body-named create (`POST /v1/indices`) | middleware ([`BodyShape::NativeCreate`]) |
+//! | the gRPC listener (`:8081`) | `xerj-server::grpc`, using [`Principal::allows_index`] |
+//! | privilege escalation via `POST /_security/api_key` | `es_compat::security_create_api_key` |
+//! | anything else that resolves an index name from anywhere | `xerj_engine::index_guard` |
+//!
+//! ## Fail-closed
+//!
+//! Matching the precedent set elsewhere in the tree (a corrupt sidecar, an
+//! unknown marker version and an unsatisfiable doc all refuse rather than
+//! guess), every unresolved case here denies:
+//! - an unknown/expired/invalidated credential → [`Principal::Denied`] → deny;
+//! - a key minted with no usable `role_descriptors` → [`Principal::Unscoped`]
+//! → **no** privilege on the reserved namespace;
+//! - an unrecognized ES privilege name → grants nothing;
+//! - a body this module must parse to find its target but cannot → deny;
+//! - a snapshot or restore that names no indices (i.e. *all* of them,
+//! including brains) → superuser only;
+//! - an index expression that resolves to nothing this principal can see →
+//! resolves to nothing, not to everything.
+//!
+//! ## What this does **not** claim
+//!
+//! Broad RBAC over the general ES-compat surface is still deferred: an
+//! `Unscoped` key keeps its historical superuser-equivalent reach over
+//! *ordinary* indices. This module makes the reserved namespace a real
+//! boundary and confines a `Scoped` key to its grants; it does not turn xerj
+//! into a general multi-tenant authorization system. `xerj_engine::rbac`'s
+//! named `RoleStore` remains unenforced data.
+//!
+//! Nor does it second-guess an explicit grant. A scoped key minted with
+//! `names: ["*"]` **can** read the reserved namespace, because `*` matches
+//! every index and that is what the operator asked for; only a superuser can
+//! mint one. See [`xerj_engine::rbac::Role::applies_to`] for the reasoning and
+//! `a_star_grant_reaches_the_reserved_namespace` below for the pinned
+//! behaviour. Operators who want brain isolation must grant concrete names or
+//! a prefix that excludes `.xerj-memory-`.
+
+// The decision functions return `Result<(), Response>` where `Err` IS the
+// ready-to-send 403. That trips `clippy::result_large_err` (an `axum::Response`
+// is a fat value), but boxing it would add an allocation on the deny path to
+// satisfy a lint aimed at hot `Ok` paths, and would obscure that the error
+// *is* the response.
+#![allow(clippy::result_large_err)]
+
+use std::collections::HashSet;
+use std::sync::Arc;
+
+use axum::{
+ body::{Body, Bytes},
+ extract::{Request, State},
+ http::{header, Method, StatusCode},
+ middleware::Next,
+ response::{IntoResponse, Response},
+ Json,
+};
+use serde_json::Value;
+
+use crate::auth::{authenticate, Principal, AUTH_EXEMPT_PATHS};
+use crate::error::{EsErrorBody, EsErrorResponse, EsRootCause};
+use crate::state::AppState;
+use xerj_engine::index_guard::IndexVisibility;
+use xerj_engine::rbac::Privilege;
+
+// ─────────────────────────────────────────────────────────────────────────────
+// The reserved namespace
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// Index-name prefix reserved for agent-memory namespaces (`memory_api`) and
+/// second brains (`graph_api`). Kept in one place so the three writers of the
+/// name — `memory_api::MEMORY_PREFIX`, `graph_api::edges_index` and this
+/// module — cannot drift.
+pub const RESERVED_INDEX_PREFIX: &str = ".xerj-memory-";
+
+/// Is `index` inside the reserved namespace?
+pub fn is_reserved_index(index: &str) -> bool {
+ index.starts_with(RESERVED_INDEX_PREFIX)
+}
+
+/// The edges index backing brain `brain` (SECOND_BRAIN_SPEC §1). This is the
+/// resource name a `role_descriptors` grant must name to unlock
+/// `/_graph/{brain}/*`.
+pub fn brain_edges_index(brain: &str) -> String {
+ format!("{RESERVED_INDEX_PREFIX}{brain}-edges")
+}
+
+/// The index backing agent-memory namespace `ns` — which is also the default
+/// *nodes* index of the brain of the same name.
+pub fn memory_namespace_index(ns: &str) -> String {
+ format!("{RESERVED_INDEX_PREFIX}{ns}")
+}
+
+/// Could this index *expression* (a literal name or a pattern) reach the
+/// reserved namespace?
+///
+/// Deliberately conservative in the deny direction: a pattern is judged by the
+/// literal text before its first `*`, and if that prefix is a prefix of — or is
+/// prefixed by — the reserved prefix, the expression is treated as reaching it.
+/// `*`, `_all`, `.*`, `.xerj-*` and `.xerj-memory-alice*` all qualify;
+/// `logs-*` does not.
+///
+/// Read patterns are no longer *refused* on the strength of this — they are
+/// expanded over the principal's visible set instead, which is both safe and
+/// the only way a granted `logs-*` can work. It is still what decides whether
+/// a name a caller is about to *create* (an alias, a fresh index) is squatting
+/// the reserved namespace.
+pub fn may_reach_reserved(expression: &str) -> bool {
+ let expr = expression.trim();
+ if expr.is_empty() {
+ return false;
+ }
+ if expr == "_all" {
+ return true;
+ }
+ let head = expr.split('*').next().unwrap_or("");
+ if expr.contains('*') {
+ head.starts_with(RESERVED_INDEX_PREFIX) || RESERVED_INDEX_PREFIX.starts_with(head)
+ } else {
+ is_reserved_index(expr)
+ }
+}
+
+/// Is this expression a pattern rather than one concrete name?
+fn is_pattern(expr: &str) -> bool {
+ expr.contains('*') || expr == "_all"
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+// Decisions
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// Authorize `principal` for `privilege` on the single index `index`.
+///
+/// Ordered before any existence check on purpose: an unauthorized caller gets
+/// the same 403 whether or not the resource exists, so 403-vs-404 cannot be
+/// used to enumerate brains.
+pub fn authorize_index(
+ principal: &Principal,
+ index: &str,
+ privilege: Privilege,
+) -> Result<(), Response> {
+ if principal.allows_index(index, privilege) {
+ Ok(())
+ } else {
+ Err(forbidden(principal, index, privilege))
+ }
+}
+
+/// Authorize `principal` for `privilege` on brain `brain` — i.e. on its edges
+/// index. Every `/_graph/{brain}/*` handler calls this before it does anything
+/// else that could reveal whether the brain exists.
+pub fn authorize_brain(
+ principal: &Principal,
+ brain: &str,
+ privilege: Privilege,
+) -> Result<(), Response> {
+ authorize_index(principal, &brain_edges_index(brain), privilege)
+}
+
+/// Authorize `principal` for `privilege` on agent-memory namespace `ns`.
+pub fn authorize_memory_namespace(
+ principal: &Principal,
+ ns: &str,
+ privilege: Privilege,
+) -> Result<(), Response> {
+ authorize_index(principal, &memory_namespace_index(ns), privilege)
+}
+
+/// ES-shaped 403. Names the resource and the privilege the caller lacks — the
+/// caller supplied the resource name, so echoing it back confirms nothing —
+/// and points at the grant that would fix it.
+pub fn forbidden(principal: &Principal, resource: &str, privilege: Privilege) -> Response {
+ let action = match privilege {
+ Privilege::ReadIndex => "read",
+ Privilege::WriteIndex => "write",
+ Privilege::AdminIndex => "manage",
+ Privilege::SnapshotCreate => "create_snapshot",
+ Privilege::SnapshotRestore => "restore_snapshot",
+ Privilege::SecurityAdmin => "manage_security",
+ Privilege::AuditRead => "read_audit",
+ };
+ let reason = format!(
+ "action [{action}] is unauthorized for this credential on [{resource}]: mint a key whose \
+ role_descriptors grant [{action}] on that index (POST /_security/api_key), or use the \
+ configured admin key"
+ );
+ tracing::debug!(
+ principal = principal.label(),
+ resource,
+ action,
+ "authorization denied"
+ );
+ es_error(StatusCode::FORBIDDEN, reason)
+}
+
+fn es_error(status: StatusCode, reason: String) -> Response {
+ let error_type = "security_exception".to_string();
+ let body = EsErrorResponse {
+ error: EsErrorBody {
+ root_cause: vec![EsRootCause {
+ error_type: error_type.clone(),
+ reason: reason.clone(),
+ resource_type: None,
+ resource_id: None,
+ index_uuid: None,
+ index: None,
+ }],
+ error_type,
+ reason,
+ resource_type: None,
+ resource_id: None,
+ index_uuid: None,
+ index: None,
+ request_id: None,
+ },
+ status: status.as_u16(),
+ };
+ (status, Json(body)).into_response()
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+// The request's visible set (layer 2)
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// Adapts a [`Principal`] to the engine's [`IndexVisibility`] hook.
+///
+/// "Visible" is *any* index privilege, not a specific one: this layer decides
+/// whether the principal may reach the index at all, which is the property that
+/// closes a body-named target the middleware did not know to parse. Which
+/// privilege the request needs on it stays the middleware's decision, so a
+/// read-only grant still cannot write through the front door.
+struct PrincipalVisibility(Principal);
+
+impl IndexVisibility for PrincipalVisibility {
+ fn visible(&self, index: &str) -> bool {
+ self.0.allows_index(index, Privilege::ReadIndex)
+ || self.0.allows_index(index, Privilege::WriteIndex)
+ || self.0.allows_index(index, Privilege::AdminIndex)
+ }
+}
+
+/// The visibility rule for `principal`, for installing around a request.
+pub fn visibility_for(principal: &Principal) -> Arc<dyn IndexVisibility> {
+ Arc::new(PrincipalVisibility(principal.clone()))
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+// Request classification
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// What a request path targets.
+///
+/// `op_start` on [`Target::Indices`] is the segment index at which the
+/// operation begins, because the two routers spell resources differently:
+/// ES-compat puts the index first (`/{index}/_search`, op at 1) while the
+/// native router nests it (`/v1/indices/{index}/search`, op at 3). Both are
+/// doors into the reserved namespace, so both are classified here.
+#[derive(Debug, PartialEq, Eq)]
+enum Target {
+ /// `/_graph/{brain}/…`
+ Brain(String),
+ /// `/_memory/{namespace}[/…]`
+ Memory(String),
+ /// One or more index expressions, plus where the op segment starts.
+ Indices(Vec<String>, usize),
+ /// A cluster/global endpoint whose path names no index: `/_cat/*`,
+ /// `/_mapping`, `/_nodes`, but also `/_search`, `/_bulk`, `/_msearch`.
+ /// Reads are answered over the principal's visible set; mutations must
+ /// name their targets in the body (which [`body_targets`] extracts and
+ /// authorizes) or belong to a principal that holds the general surface.
+ Cluster,
+ /// Not authorization-relevant (health probes, the version banner).
+ Exempt,
+}
+
+/// Read-only ops that are POSTed rather than GETed. Without this list a
+/// `POST /{index}/_search` would be classified as a write. Covers both
+/// spellings: ES-compat (`_search`) and native (`search`).
+const POST_READ_OPS: &[&str] = &[
+ "_search",
+ "_count",
+ "_msearch",
+ "_mget",
+ "_explain",
+ "_validate",
+ "_field_caps",
+ "_termvectors",
+ "_mtermvectors",
+ "_search_shards",
+ "_rank_eval",
+ "_knn_search",
+ "_pit",
+ "_async_search",
+ "_sql",
+ "_esql",
+ "_eql",
+ "_analyze",
+ "_graph",
+ "_resolve",
+ "_terms_enum",
+ "_disk_usage",
+ // Cluster endpoints that carry a body but change nothing: privilege
+ // probes, template rendering, pipeline/painless simulation. Only ever
+ // consulted for POST-shaped verbs, so `PUT /_ingest/pipeline/{id}` and
+ // `PUT /_scripts/{id}` remain the mutations they are.
+ "_security",
+ "_ingest",
+ "_render",
+ "_scripts",
+ // native router
+ "search",
+ "explain-plan",
+ "encodings",
+];
+
+/// Index-lifecycle ops. These reshape or destroy the index itself, so they
+/// need `manage`, not `write`.
+const ADMIN_OPS: &[&str] = &[
+ "_settings",
+ "_mapping",
+ "_mappings",
+ "_close",
+ "_open",
+ "_shrink",
+ "_split",
+ "_clone",
+ "_rollover",
+ "_alias",
+ "_aliases",
+ "_cache",
+ "_forcemerge",
+ "_freeze",
+ "_unfreeze",
+ "_block",
+ "_upgrade",
+ "_migration",
+];
+
+/// Split a URI path into percent-decoded, non-empty segments.
+fn segments(path: &str) -> Vec<String> {
+ path.split('/')
+ .filter(|s| !s.is_empty())
+ .map(percent_decode)
+ .collect()
+}
+
+/// Minimal percent-decoder. The middleware compares raw path text against
+/// index names, so `%2Exerj-memory-alice-edges` must not slip past a check
+/// that `.xerj-memory-alice-edges` would fail. Invalid escapes are left
+/// verbatim (they cannot decode into the reserved prefix).
+fn percent_decode(s: &str) -> String {
+ if !s.contains('%') {
+ return s.to_string();
+ }
+ let bytes = s.as_bytes();
+ let mut out: Vec<u8> = Vec::with_capacity(bytes.len());
+ let mut i = 0;
+ while i < bytes.len() {
+ if bytes[i] == b'%' && i + 2 < bytes.len() {
+ let hex = std::str::from_utf8(&bytes[i + 1..i + 3]).ok();
+ if let Some(b) = hex.and_then(|h| u8::from_str_radix(h, 16).ok()) {
+ out.push(b);
+ i += 3;
+ continue;
+ }
+ }
+ out.push(bytes[i]);
+ i += 1;
+ }
+ String::from_utf8_lossy(&out).into_owned()
+}
+
+/// Classify a request path.
+fn classify(path: &str) -> Target {
+ // Probes, the scrape endpoint, and "who am I" — none of which name or
+ // reveal an index. `_security/_authenticate` in particular must stay open
+ // to a scoped key, or a caller cannot verify its own credential.
+ if AUTH_EXEMPT_PATHS.contains(&path)
+ || path == "/v1/metrics"
+ || path == "/_security/_authenticate"
+ {
+ return Target::Exempt;
+ }
+ let segs = segments(path);
+ let Some(first) = segs.first() else {
+ // `GET /` — the version banner. No data.
+ return Target::Exempt;
+ };
+ match first.as_str() {
+ "_graph" => match segs.get(1) {
+ Some(brain) => Target::Brain(brain.clone()),
+ // No brain in the path: no route matches, let it 404.
+ None => Target::Exempt,
+ },
+ "_memory" => match segs.get(1) {
+ Some(ns) => Target::Memory(ns.clone()),
+ None => Target::Exempt,
+ },
+ // Native router (`build_native_router`, :8080). Same engine, same
+ // indices, different spelling — and therefore the same boundary:
+ // `GET /v1/indices/.xerj-memory-alice-edges` must be no more reachable
+ // than `GET /.xerj-memory-alice-edges`.
+ "v1" => match (segs.get(1).map(String::as_str), segs.get(2)) {
+ (Some("indices"), Some(expr)) | (Some("schema"), Some(expr)) => {
+ Target::Indices(split_expressions(expr), 3)
+ }
+ // `POST /v1/indices` names its index in the body — authorized as
+ // `BodyShape::NativeCreate`, not refused for lack of a path target.
+ _ => Target::Cluster,
+ },
+ // `/_all/_search` and `/*/_search` are index-scoped routes whose index
+ // happens to be a pattern — not cluster endpoints. Classifying them
+ // here is what lets them expand over the caller's visible set like any
+ // other pattern instead of being refused for naming nothing.
+ s if s == "*" || s == "_all" => Target::Indices(split_expressions(s), 1),
+ s if s.starts_with('_') => Target::Cluster,
+ s => Target::Indices(split_expressions(s), 1),
+ }
+}
+
+/// Split a comma-separated multi-index expression (`logs-a,logs-b`).
+fn split_expressions(s: &str) -> Vec<String> {
+ s.split(',')
+ .map(str::trim)
+ .filter(|e| !e.is_empty())
+ .map(String::from)
+ .collect()
+}
+
+/// The privilege a request needs on whatever it targets.
+///
+/// Conservative by construction: anything not recognized as a read or a
+/// lifecycle op is treated as a write, so an unclassified mutation can never
+/// be waved through as a read.
+fn required_privilege(method: &Method, segs: &[String], op_start: usize) -> Privilege {
+ if method == Method::GET || method == Method::HEAD {
+ return Privilege::ReadIndex;
+ }
+ match segs.get(op_start).map(String::as_str) {
+ Some(o) if POST_READ_OPS.contains(&o) => Privilege::ReadIndex,
+ Some(o) if ADMIN_OPS.contains(&o) => Privilege::AdminIndex,
+ // No op segment at all: the request targets the index itself, so it is
+ // creating or destroying it.
+ None => Privilege::AdminIndex,
+ _ => Privilege::WriteIndex,
+ }
+}
+
+/// Does this cluster-level request only *read*?
+///
+/// `GET`/`HEAD` always do. So does any verb whose op is one of the read ops:
+/// `POST /_search`, `/_count`, `/_msearch`, `/_mget` are the reason the global
+/// verbs are usable at all, and `DELETE /_search/scroll` / `DELETE /_pit` /
+/// `DELETE /_async_search/{id}` release a session the caller already opened.
+fn cluster_is_read(method: &Method, segs: &[String]) -> bool {
+ if method == Method::GET || method == Method::HEAD {
+ return true;
+ }
+ match segs.first().map(String::as_str) {
+ Some(op) => POST_READ_OPS.contains(&op),
+ None => true,
+ }
+}
+
+/// The privilege a `/_graph/{brain}/…` or `/_memory/{ns}[/…]` request needs.
+///
+/// Kept separate from [`required_privilege`] because these paths spell their
+/// ops without a leading underscore (`link`) and because deleting a whole
+/// namespace is a lifecycle op while deleting one document is not.
+fn reserved_api_privilege(method: &Method, segs: &[String]) -> Privilege {
+ if method == Method::GET || method == Method::HEAD {
+ return Privilege::ReadIndex;
+ }
+ if method == Method::POST {
+ // `POST /_memory/{ns}/_recall` reads; `POST /_memory/{ns}` and
+ // `POST /_graph/{b}/link` write. Creating the backing index lazily is
+ // part of writing a brain, not a separate `manage` step.
+ return if segs.iter().any(|s| s == "_recall") {
+ Privilege::ReadIndex
+ } else {
+ Privilege::WriteIndex
+ };
+ }
+ if method == Method::DELETE {
+ // `DELETE /_memory/{ns}` drops the whole namespace → manage.
+ // `DELETE /_memory/{ns}/{id}` and `DELETE /_graph/{b}/link/{id}` are
+ // document-level → write.
+ return if segs.len() <= 2 {
+ Privilege::AdminIndex
+ } else {
+ Privilege::WriteIndex
+ };
+ }
+ Privilege::WriteIndex
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+// Body-named targets
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// The shape in which a route's body can name the indices it operates on.
+///
+/// This table is the exhaustive answer to "which handlers take an index from
+/// the request body". It is not the *only* protection — [`PrincipalVisibility`]
+/// in the engine catches whatever is missing from it — but it is what produces
+/// a precise 403 with the right privilege instead of a downstream "not found".
+#[derive(Clone, Copy, Debug, PartialEq, Eq)]
+enum BodyShape {
+ /// Nothing in the body names an index.
+ None,
+ /// `_bulk` NDJSON: `{"index"|"create"|"update"|"delete": {"_index": …}}`.
+ Bulk,
+ /// `_msearch` / `_msearch/template` NDJSON header lines: `{"index": …}`,
+ /// where the value may be a string or an array of strings.
+ MsearchHeaders,
+ /// `_mget`: `{"docs": [{"_index": …}]}`.
+ MgetDocs,
+ /// `POST /_aliases`: `{"actions": [{"add"|"remove"|"remove_index": …}]}`.
+ AliasActions,
+ /// `_reindex`: `{"source": {"index": …}, "dest": {"index": …}}`.
+ Reindex,
+ /// Snapshot create / restore: `{"indices": "a,b" | ["a","b"]}`.
+ SnapshotIndices,
+ /// Native `POST /v1/indices`: `{"name": …}`.
+ NativeCreate,
+ /// `PUT /{index}`: the `aliases` block creates alias names.
+ CreateIndex,
+ /// Any search-shaped body — a `terms` lookup names another index to read
+ /// its terms out of, at arbitrary depth.
+ Query,
+ /// `POST /_monitoring/bulk` writes one fixed internal index.
+ MonitoringBulk,
+ /// An index template's `index_patterns`. It names no index that exists
+ /// yet, but it decides the mapping of every index created under a matching
+ /// name later — including a brain — so the patterns are checked in their
+ /// own right by [`authorize_template_patterns`].
+ IndexTemplate,
+ /// An ML job or datafeed config: `PUT /_ml/anomaly_detectors/{id}`
+ /// (`source_index`, `index`, and the combined-form `datafeed_config`) and
+ /// `PUT /_ml/datafeeds/{id}` (`indices` / `indexes`).
+ ///
+ /// These name an index the *server* will go and read, on a schedule, on
+ /// the caller's behalf. Registering one against an index the caller cannot
+ /// read is wrong on its own terms — it was accepted with a 200 before this
+ /// arm existed — and it was also the front half of a live cross-brain
+ /// read: the detached scorer `POST /_ml/datafeeds/{id}/_start` spawns had
+ /// no visibility rule, so it fetched what the caller could not. The
+ /// detached half is fixed at the spawn site (`es_compat`'s
+ /// `spawn_datafeed_task` carries the rule across); this is the half that
+ /// refuses the configuration in the first place.
+ MlConfig,
+}
+
+/// Search-shaped ops whose body can carry a `terms` lookup.
+const QUERY_BODY_OPS: &[&str] = &[
+ "_search",
+ "_count",
+ "_explain",
+ "_validate",
+ "_delete_by_query",
+ "_update_by_query",
+ "_async_search",
+ "_eql",
+ "_rank_eval",
+ "_terms_enum",
+ "_field_caps",
+ "_knn_search",
+ "_pit",
+ "_graph",
+ "_sql",
+ "_esql",
+];
+
+/// The index `POST /_monitoring/bulk` ingests into
+/// (`es_compat::ingest_monitoring_ndjson`).
+const MONITORING_INDEX: &str = "xerj-monitoring";
+
+/// Which body shape does this route carry?
+fn body_shape(method: &Method, segs: &[String]) -> BodyShape {
+ let first = segs.first().map(String::as_str).unwrap_or("");
+ // Native router. Its ingest bodies are plain document arrays under an
+ // index named in the path; the only body-named index it has is the create.
+ if first == "v1" {
+ return match (segs.get(1).map(String::as_str), segs.len()) {
+ (Some("indices"), 2) if method == Method::POST => BodyShape::NativeCreate,
+ _ => BodyShape::None,
+ };
+ }
+ let has = |op: &str| segs.iter().any(|s| s == op);
+ if has("_bulk") {
+ return BodyShape::Bulk;
+ }
+ if has("_msearch") {
+ return BodyShape::MsearchHeaders;
+ }
+ if has("_mget") {
+ return BodyShape::MgetDocs;
+ }
+ if first == "_aliases" && method != Method::GET {
+ return BodyShape::AliasActions;
+ }
+ if first == "_reindex" {
+ return BodyShape::Reindex;
+ }
+ if first == "_snapshot" {
+ return BodyShape::SnapshotIndices;
+ }
+ if first == "_monitoring" {
+ return BodyShape::MonitoringBulk;
+ }
+ if (first == "_index_template" || first == "_template") && method != Method::GET {
+ return BodyShape::IndexTemplate;
+ }
+ // `PUT|POST /_ml/anomaly_detectors/{id}` and `PUT|POST /_ml/datafeeds/{id}`
+ // — exactly three segments, so the sub-verbs (`…/{id}/_start`, `_stop`,
+ // `_score`, `results/records`) are not swept in: they carry no index name
+ // of their own, and their reach is decided by the config that was already
+ // authorized here plus the guard the spawned scorer carries.
+ if first == "_ml" && segs.len() == 3 && (method == Method::PUT || method == Method::POST) {
+ return match segs[1].as_str() {
+ "anomaly_detectors" | "datafeeds" => BodyShape::MlConfig,
+ _ => BodyShape::None,
+ };
+ }
+ if segs.len() == 1 && method == Method::PUT && !first.starts_with('_') {
+ return BodyShape::CreateIndex;
+ }
+ if QUERY_BODY_OPS.iter().any(|op| has(op)) {
+ return BodyShape::Query;
+ }
+ BodyShape::None
+}
+
+/// One index expression a request will touch, and what it needs on it.
+type Demand = (String, Privilege);
+
+/// Refuse a request whose body should have named its targets but could not be
+/// parsed. Fail-closed: an unreadable target is not an absent one.
+fn unresolvable(principal: &Principal, what: &str) -> Response {
+ tracing::debug!(
+ principal = principal.label(),
+ what,
+ "authorization refused: request body names indices but could not be resolved"
+ );
+ es_error(
+ StatusCode::FORBIDDEN,
+ format!(
+ "this credential is authorized per index, and the {what} could not be resolved from \
+ the request body; send a well-formed body that names its indices"
+ ),
+ )
+}
+
+/// Extract every index this request's **body** will touch, with the privilege
+/// it needs on each.
+///
+/// `default_index` is the path index for the index-scoped spellings
+/// (`/{index}/_bulk`, `/{index}/_msearch`, `/{index}/_mget`), which the body
+/// may override — the override being the whole bug this exists for.
+fn body_targets(
+ principal: &Principal,
+ shape: BodyShape,
+ body: &[u8],
+ default_index: Option<&str>,
+) -> Result<Vec<Demand>, Response> {
+ let mut out: Vec<Demand> = Vec::new();
+ match shape {
+ BodyShape::None => {}
+ BodyShape::MonitoringBulk => {
+ out.push((MONITORING_INDEX.to_string(), Privilege::WriteIndex))
+ }
+ BodyShape::Bulk => {
+ // NDJSON, alternating action and (except for `delete`) source
+ // lines. Every action line must parse — one that does not is a
+ // target we cannot see, so the request is refused rather than
+ // handed to a bulk pipeline that would happily route it.
+ let Ok(text) = std::str::from_utf8(body) else {
+ return Err(unresolvable(principal, "bulk action lines"));
+ };
+ // Fast path for the index-scoped spelling every client uses by
+ // default: with no `_index` anywhere in the payload, every action
+ // targets the path index, and there is nothing to find by parsing
+ // a million action lines.
+ if !text.contains("\"_index\"") {
+ if let Some(d) = default_index {
+ out.push((d.to_string(), Privilege::WriteIndex));
+ }
+ return Ok(out);
+ }
+ let mut expect_action = true;
+ for line in text.lines() {
+ let line = line.trim();
+ if line.is_empty() {
+ continue;
+ }
+ if !expect_action {
+ expect_action = true;
+ continue;
+ }
+ let Ok(action) = serde_json::from_str::<Value>(line) else {
+ return Err(unresolvable(principal, "bulk action lines"));
+ };
+ let Some(obj) = action.as_object() else {
+ return Err(unresolvable(principal, "bulk action lines"));
+ };
+ let Some((verb, meta)) = obj.iter().next() else {
+ return Err(unresolvable(principal, "bulk action lines"));
+ };
+ // `delete` carries no source line; everything else does.
+ expect_action = verb == "delete";
+ let named = meta.get("_index").and_then(Value::as_str);
+ if let Some(index) = named.or(default_index) {
+ out.push((index.to_string(), Privilege::WriteIndex));
+ }
+ }
+ }
+ BodyShape::MsearchHeaders => {
+ let Ok(text) = std::str::from_utf8(body) else {
+ return Err(unresolvable(principal, "multi-search header lines"));
+ };
+ let mut expect_header = true;
+ for line in text.lines() {
+ let line = line.trim();
+ if line.is_empty() {
+ continue;
+ }
+ let Ok(parsed) = serde_json::from_str::<Value>(line) else {
+ return Err(unresolvable(principal, "multi-search header lines"));
+ };
+ if expect_header {
+ match parsed.get("index") {
+ Some(Value::String(s)) => out.push((s.clone(), Privilege::ReadIndex)),
+ Some(Value::Array(items)) => {
+ for i in items {
+ if let Some(s) = i.as_str() {
+ out.push((s.to_string(), Privilege::ReadIndex));
+ }
+ }
+ }
+ // A header that names no index means "the path index,
+ // else every index" — the latter is answered over the
+ // visible set, so nothing to demand here.
+ _ => {
+ if let Some(d) = default_index {
+ out.push((d.to_string(), Privilege::ReadIndex));
+ }
+ }
+ }
+ } else {
+ // The search body half — it can still carry a terms lookup.
+ collect_query_body_indices(&parsed, &mut out);
+ }
+ expect_header = !expect_header;
+ }
+ }
+ BodyShape::MgetDocs => {
+ let Ok(parsed) = serde_json::from_slice::<Value>(body) else {
+ return Err(unresolvable(principal, "multi-get documents"));
+ };
+ match parsed.get("docs").and_then(Value::as_array) {
+ Some(docs) => {
+ for doc in docs {
+ let named = doc.get("_index").and_then(Value::as_str);
+ match named.or(default_index) {
+ Some(index) => out.push((index.to_string(), Privilege::ReadIndex)),
+ None => return Err(unresolvable(principal, "multi-get documents")),
+ }
+ }
+ }
+ // `{"ids": [...]}` short form uses the path index.
+ None => {
+ if let Some(d) = default_index {
+ out.push((d.to_string(), Privilege::ReadIndex));
+ }
+ }
+ }
+ }
+ BodyShape::AliasActions => {
+ let Ok(parsed) = serde_json::from_slice::<Value>(body) else {
+ return Err(unresolvable(principal, "alias actions"));
+ };
+ let Some(actions) = parsed.get("actions").and_then(Value::as_array) else {
+ return Err(unresolvable(principal, "alias actions"));
+ };
+ for action in actions {
+ let Some(obj) = action.as_object() else {
+ return Err(unresolvable(principal, "alias actions"));
+ };
+ for (_verb, params) in obj {
+ // Pointing an alias at an index is a change to that index's
+ // addressing, so it needs `manage` on it — the same
+ // privilege `PUT /{index}/_alias/{alias}` already demands.
+ push_names(params.get("index"), Privilege::AdminIndex, &mut out);
+ push_names(params.get("indices"), Privilege::AdminIndex, &mut out);
+ // …and the alias NAME is authorized in its own right, so a
+ // caller cannot squat an alias inside the reserved
+ // namespace and have a brain resolve through it later.
+ push_names(params.get("alias"), Privilege::AdminIndex, &mut out);
+ push_names(params.get("aliases"), Privilege::AdminIndex, &mut out);
+ }
+ }
+ if out.is_empty() {
+ return Err(unresolvable(principal, "alias actions"));
+ }
+ }
+ BodyShape::Reindex => {
+ let Ok(parsed) = serde_json::from_slice::<Value>(body) else {
+ return Err(unresolvable(principal, "reindex source and destination"));
+ };
+ let before = out.len();
+ if let Some(source) = parsed.get("source") {
+ push_names(source.get("index"), Privilege::ReadIndex, &mut out);
+ push_names(source.get("indices"), Privilege::ReadIndex, &mut out);
+ if let Some(q) = source.get("query") {
+ collect_query_body_indices(q, &mut out);
+ }
+ }
+ if let Some(dest) = parsed.get("dest") {
+ push_names(dest.get("index"), Privilege::WriteIndex, &mut out);
+ }
+ if out.len() == before {
+ return Err(unresolvable(principal, "reindex source and destination"));
+ }
+ }
+ BodyShape::SnapshotIndices => {
+ // Absent `indices` means "every index on the node", which for a
+ // non-superuser would be a bulk read (or overwrite) of every
+ // brain. `decide` turns an empty demand list on this route into a
+ // refusal; here we only report what was named. A restore *writes*
+ // its targets — `decide` upgrades the privilege, since only the
+ // path says which of the two this is.
+ if let Ok(parsed) = serde_json::from_slice::<Value>(body) {
+ push_names(parsed.get("indices"), Privilege::ReadIndex, &mut out);
+ }
+ }
+ BodyShape::NativeCreate => {
+ let Ok(parsed) = serde_json::from_slice::<Value>(body) else {
+ return Err(unresolvable(principal, "index name"));
+ };
+ match parsed.get("name").and_then(Value::as_str) {
+ Some(name) => out.push((name.to_string(), Privilege::AdminIndex)),
+ None => return Err(unresolvable(principal, "index name")),
+ }
+ }
+ BodyShape::CreateIndex => {
+ // `PUT /{index}` names its index in the path (already demanded);
+ // its body can additionally mint alias names.
+ if let Ok(parsed) = serde_json::from_slice::<Value>(body) {
+ if let Some(aliases) = parsed.get("aliases").and_then(Value::as_object) {
+ for alias in aliases.keys() {
+ out.push((alias.clone(), Privilege::AdminIndex));
+ }
+ }
+ }
+ }
+ BodyShape::Query => {
+ if let Ok(parsed) = serde_json::from_slice::<Value>(body) {
+ collect_query_body_indices(&parsed, &mut out);
+ }
+ }
+ BodyShape::MlConfig => {
+ let Ok(parsed) = serde_json::from_slice::<Value>(body) else {
+ // Not JSON: it names no index, and the handler will reject it
+ // for the missing `source_index` / `job_id` anyway.
+ return Ok(out);
+ };
+ // Every spelling the two ML handlers actually read. `put_ml_datafeed`
+ // takes `indices` OR `indexes`; `put_ml_anomaly_detector` takes
+ // `source_index` (string or array) OR `index`; the ES combined form
+ // nests a whole datafeed under `datafeed_config`.
+ let push_ml = |v: Option<&Value>, out: &mut Vec<Demand>| {
+ push_names(v, Privilege::ReadIndex, out);
+ };
+ push_ml(parsed.get("source_index"), &mut out);
+ push_ml(parsed.get("index"), &mut out);
+ push_ml(parsed.get("indices"), &mut out);
+ push_ml(parsed.get("indexes"), &mut out);
+ if let Some(dfc) = parsed.get("datafeed_config") {
+ push_ml(dfc.get("indices"), &mut out);
+ push_ml(dfc.get("indexes"), &mut out);
+ push_ml(dfc.get("index"), &mut out);
+ }
+ // A body that names nothing is NOT refused: `PUT /_ml/datafeeds/{id}`
+ // may carry only a `job_id`, in which case the handler inherits the
+ // detector's `source_index` — a name this middleware cannot see, but
+ // one that was authorized when the detector itself was created, and
+ // that the scorer re-checks under the starter's own visibility rule.
+ // `decide` reads the empty list on this shape as "nothing named",
+ // not as "unrestricted".
+ }
+ // Patterns, not names — `authorize_template_patterns` handles them.
+ BodyShape::IndexTemplate => {}
+ }
+ // A bulk usually names two or three indices across a million action lines.
+ out.sort_by(|a, b| a.0.cmp(&b.0).then(rank(a.1).cmp(&rank(b.1))));
+ out.dedup();
+ Ok(out)
+}
+
+/// Stable ordering key for a privilege, so the demand list can be deduped
+/// without asking `xerj_engine::rbac` for an `Ord` it has no other use for.
+fn rank(p: Privilege) -> u8 {
+ match p {
+ Privilege::ReadIndex => 0,
+ Privilege::WriteIndex => 1,
+ Privilege::AdminIndex => 2,
+ Privilege::SnapshotCreate => 3,
+ Privilege::SnapshotRestore => 4,
+ Privilege::SecurityAdmin => 5,
+ Privilege::AuditRead => 6,
+ }
+}
+
+/// An index template applies to indices that do not exist yet, so there is no
+/// name to authorize — only a pattern. A template whose `index_patterns` can
+/// match the reserved namespace would pick the mapping of a brain created
+/// later, which is a write to that brain by another route, so a non-superuser
+/// may not register one.
+fn authorize_template_patterns(principal: &Principal, body: &[u8]) -> Result<(), Response> {
+ let Ok(parsed) = serde_json::from_slice::<Value>(body) else {
+ // Not JSON: it names no pattern, and the handler will reject it.
+ return Ok(());
+ };
+ let mut patterns: Vec<Demand> = Vec::new();
+ push_names(
+ parsed.get("index_patterns"),
+ Privilege::AdminIndex,
+ &mut patterns,
+ );
+ // ES 6-era `_template` spelled it `template`.
+ push_names(parsed.get("template"), Privilege::AdminIndex, &mut patterns);
+ for (pattern, _) in patterns {
+ if !may_reach_reserved(&pattern) {
+ continue;
+ }
+ // Only a principal that was *granted* the namespace may template over
+ // it. `Unscoped::allows_index` answers "not reserved" for a pattern
+ // like `*` — true of the literal text, useless as an answer here — so
+ // it is excluded explicitly rather than consulted.
+ let held = match principal {
+ Principal::Superuser => true,
+ Principal::Scoped { .. } => principal.allows_index(&pattern, Privilege::AdminIndex),
+ _ => false,
+ };
+ if !held {
+ return Err(forbidden(principal, &pattern, Privilege::AdminIndex));
+ }
+ }
+ Ok(())
+}
+
+/// Push a `"a"` / `"a,b"` / `["a","b"]` index field onto the demand list.
+fn push_names(v: Option<&Value>, privilege: Privilege, out: &mut Vec<Demand>) {
+ match v {
+ Some(Value::String(s)) => {
+ for part in split_expressions(s) {
+ out.push((part, privilege));
+ }
+ }
+ Some(Value::Array(items)) => {
+ for item in items {
+ if let Some(s) = item.as_str() {
+ for part in split_expressions(s) {
+ out.push((part, privilege));
+ }
+ }
+ }
+ }
+ _ => {}
+ }
+}
+
+/// Collect every index a **search body** reads that the URL never names.
+///
+/// Two shapes, both found by auditing the tree rather than by being attacked —
+/// which is the point of auditing:
+///
+/// - a `terms` lookup, `{"terms": {"field": {"index": "other", "id": "1",
+/// "path": "vals"}}}`, fetches a document out of `other` and substitutes its
+/// values into the query;
+/// - a `lookup` runtime field, `{"runtime_mappings": {"f": {"type": "lookup",
+/// "target_index": "other", …}}}`, joins rows out of `other` into the hits.
+///
+/// Both are matched structurally (the full lookup triple, the `type: lookup`
+/// marker) rather than on the key name alone, so a document field that happens
+/// to be called `index` is never mistaken for one.
+fn collect_query_body_indices(q: &Value, out: &mut Vec<Demand>) {
+ match q {
+ Value::Object(obj) => {
+ if let Some(Value::Object(terms)) = obj.get("terms") {
+ for (field, spec) in terms {
+ if field == "boost" {
+ continue;
+ }
+ if let Some(s) = spec.as_object() {
+ // Only a full lookup triple actually fetches.
+ if s.contains_key("id") && s.contains_key("path") {
+ if let Some(ix) = s.get("index").and_then(Value::as_str) {
+ out.push((ix.to_string(), Privilege::ReadIndex));
+ }
+ }
+ }
+ }
+ }
+ if obj.get("type").and_then(Value::as_str) == Some("lookup") {
+ if let Some(ix) = obj.get("target_index").and_then(Value::as_str) {
+ out.push((ix.to_string(), Privilege::ReadIndex));
+ }
+ }
+ for child in obj.values() {
+ collect_query_body_indices(child, out);
+ }
+ }
+ Value::Array(items) => {
+ for item in items {
+ collect_query_body_indices(item, out);
+ }
+ }
+ _ => {}
+ }
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+// Alias resolution
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// Resolves an alias to the indices behind it.
+///
+/// Authorization consults this **before** deciding, so `POST /pwned/_search`
+/// on an alias pointed at someone else's brain is authorized against that
+/// brain, not against the alias name. `Engine::get_index` resolves aliases too
+/// and re-checks after doing so, which is what covers an alias created between
+/// the decision and the read.
+pub trait AliasResolver {
+ fn targets(&self, name: &str) -> Option<Vec<String>>;
+}
+
+impl AliasResolver for AppState {
+ fn targets(&self, name: &str) -> Option<Vec<String>> {
+ self.engine.aliases.get(name).map(|e| e.value().clone())
+ }
+}
+
+/// No aliases at all — used by the unit tests and as the fallback when a
+/// decision is made without engine state.
+pub struct NoAliases;
+
+impl AliasResolver for NoAliases {
+ fn targets(&self, _name: &str) -> Option<Vec<String>> {
+ None
+ }
+}
+
+/// Authorize one index expression, resolving aliases first.
+///
+/// A pattern is *not* refused: it is expanded over the principal's visible set
+/// by the engine (see [`xerj_engine::index_guard`]), so a granted `logs-*`
+/// works and an ungranted `*` silently resolves to nothing instead of leaking
+/// what it would have matched.
+fn authorize_expression(
+ principal: &Principal,
+ aliases: &dyn AliasResolver,
+ expr: &str,
+ privilege: Privilege,
+) -> Result<(), Response> {
+ if is_pattern(expr) {
+ return Ok(());
+ }
+ match aliases.targets(expr) {
+ Some(backing) if !backing.is_empty() => {
+ for index in backing {
+ authorize_index(principal, &index, privilege)?;
+ }
+ Ok(())
+ }
+ _ => authorize_index(principal, expr, privilege),
+ }
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+// Middleware
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// Largest response this middleware will buffer in order to prune reserved
+/// index names out of it. Metadata listings are kilobytes; the cap only exists
+/// so a pathological response cannot be turned into an OOM.
+const MAX_PRUNABLE_RESPONSE_BYTES: usize = 64 * 1024 * 1024;
+
+/// Authorization middleware for both routers.
+///
+/// Layered *inside* [`crate::auth::auth_middleware`] (added to the router
+/// before it, so it runs after it): authentication has already rejected
+/// anonymous callers, and this decides what the authenticated one may reach.
+///
+/// A superuser — open mode (`--insecure`, point-at-a-folder) or the configured
+/// admin key — short-circuits on the first line, so the zero-configuration
+/// local path pays nothing: no body buffering, no visibility guard, no
+/// response pruning.
+pub async fn authz_middleware(State(state): State<AppState>, req: Request, next: Next) -> Response {
+ let path = req.uri().path().to_string();
+ let target = classify(&path);
+ if matches!(target, Target::Exempt) {
+ return next.run(req).await;
+ }
+
+ let principal = authenticate(
+ &state,
+ req.headers()
+ .get(header::AUTHORIZATION)
+ .and_then(|v| v.to_str().ok()),
+ );
+ if principal.is_superuser() {
+ return next.run(req).await;
+ }
+
+ let segs = segments(&path);
+ let method = req.method().clone();
+ let shape = body_shape(&method, &segs);
+
+ // Buffer the body only for the routes that can name an index inside it.
+ // `Bytes` is refcounted, so handing the same buffer to the handler costs a
+ // clone of a pointer, not of the payload.
+ let (req, body) = if shape == BodyShape::None {
+ (req, Bytes::new())
+ } else {
+ let limit = state.config.limits.max_body_bytes;
+ let (parts, raw) = req.into_parts();
+ match axum::body::to_bytes(raw, limit).await {
+ Ok(bytes) => (Request::from_parts(parts, Body::from(bytes.clone())), bytes),
+ Err(_) => {
+ return es_error(
+ StatusCode::PAYLOAD_TOO_LARGE,
+ "request body exceeds limits.max_body_bytes and cannot be authorized"
+ .to_string(),
+ )
+ }
+ }
+ };
+
+ if let Err(denied) = decide(&principal, &method, &segs, &target, shape, &body, &state) {
+ return denied;
+ }
+
+ // Layer 2: whatever the handler resolves from here on — including names
+ // this module never parsed — is checked against the same principal at the
+ // engine's index funnel.
+ let response =
+ xerj_engine::index_guard::scoped(visibility_for(&principal), next.run(req)).await;
+ prune_response(response, &segs, &principal, &state).await
+}
+
+/// The whole decision, split out so it can be unit-tested without a router.
+#[allow(clippy::too_many_arguments)]
+fn decide(
+ principal: &Principal,
+ method: &Method,
+ segs: &[String],
+ target: &Target,
+ shape: BodyShape,
+ body: &[u8],
+ aliases: &dyn AliasResolver,
+) -> Result<(), Response> {
+ // Restated here and not only in the middleware, so this function is a
+ // complete decision on its own and cannot deny the local-dev superuser if
+ // it is ever called from somewhere else.
+ if principal.is_superuser() {
+ return Ok(());
+ }
+ if matches!(principal, Principal::Denied) {
+ return Err(forbidden(principal, "<cluster>", Privilege::ReadIndex));
+ }
+ match target {
+ Target::Exempt => Ok(()),
+ Target::Brain(brain) => {
+ authorize_brain(principal, brain, reserved_api_privilege(method, segs))
+ }
+ Target::Memory(ns) => {
+ authorize_memory_namespace(principal, ns, reserved_api_privilege(method, segs))
+ }
+ Target::Indices(expressions, op_start) => {
+ let privilege = required_privilege(method, segs, *op_start);
+ for expr in expressions {
+ authorize_expression(principal, aliases, expr, privilege)?;
+ }
+ // `PUT|DELETE /{index}/_alias/{alias}` mints or drops an alias
+ // name; the name itself is a resource, or the reserved namespace
+ // could be squatted by an alias nobody authorized.
+ if segs.get(*op_start).map(String::as_str) == Some("_alias") {
+ if let Some(alias) = segs.get(op_start + 1) {
+ authorize_index(principal, alias, Privilege::AdminIndex)?;
+ }
+ }
+ // The path index is the default for a body that may override it —
+ // `/{index}/_bulk`, `/{index}/_msearch`, `/{index}/_mget`.
+ let default = expressions.first().map(String::as_str);
+ authorize_body(principal, aliases, shape, body, default)
+ }
+ Target::Cluster => {
+ // Snapshot and restore that name no indices cover *every* index on
+ // the node, brains included. There is no per-index decision to
+ // make, so they are superuser-only.
+ let snapshotting = segs.first().map(String::as_str) == Some("_snapshot")
+ && segs.len() >= 3
+ && method != Method::GET
+ && method != Method::HEAD;
+ let restoring = segs.last().map(String::as_str) == Some("_restore");
+ if shape == BodyShape::IndexTemplate {
+ authorize_template_patterns(principal, body)?;
+ }
+ let mut demands = body_targets(principal, shape, body, None)?;
+ if restoring {
+ // A restore overwrites the indices it names.
+ for demand in demands.iter_mut() {
+ demand.1 = Privilege::WriteIndex;
+ }
+ }
+ if snapshotting && demands.is_empty() {
+ return Err(forbidden(
+ principal,
+ "<all indices>",
+ if restoring {
+ Privilege::SnapshotRestore
+ } else {
+ Privilege::SnapshotCreate
+ },
+ ));
+ }
+ for (expr, privilege) in &demands {
+ authorize_expression(principal, aliases, expr, *privilege)?;
+ }
+ if cluster_is_read(method, segs) {
+ // Cluster reads (`_cat`, `_mapping`, `_nodes`, `_cluster/*`,
+ // and the global `_search`/`_count`/`_msearch`/`_mget`) are
+ // answered over the principal's visible set, so a scoped key
+ // gets a filtered cluster rather than a blanket 403 — which is
+ // what Kibana needs to function at all.
+ return Ok(());
+ }
+ if !demands.is_empty() {
+ // A mutating cluster verb that named its targets: they are all
+ // authorized above. `POST /_bulk`, `/_reindex` and `/_aliases`
+ // live here — the global bulk is the default write path for
+ // essentially every ES client.
+ return Ok(());
+ }
+ if shape == BodyShape::MlConfig {
+ // An ML config that named no index inherits one that was
+ // authorized when the detector was created (see the `MlConfig`
+ // arm of `body_targets`). Falling through to the
+ // names-nothing branch below would 403 a scoped key for the
+ // ordinary `PUT /_ml/datafeeds/{id} {"job_id": …}`, which
+ // reaches nothing this principal was not already granted.
+ return Ok(());
+ }
+ match principal {
+ // A cluster-level mutation that names nothing: an unscoped key
+ // keeps its historical reach over the general surface, a scoped
+ // one has to name what it is changing.
+ Principal::Unscoped { .. } => Ok(()),
+ _ => Err(forbidden(
+ principal,
+ "<cluster>",
+ required_privilege(method, segs, 1),
+ )),
+ }
+ }
+ }
+}
+
+/// Authorize the indices a request's body names, if any.
+fn authorize_body(
+ principal: &Principal,
+ aliases: &dyn AliasResolver,
+ shape: BodyShape,
+ body: &[u8],
+ default_index: Option<&str>,
+) -> Result<(), Response> {
+ for (expr, privilege) in body_targets(principal, shape, body, default_index)? {
+ authorize_expression(principal, aliases, &expr, privilege)?;
+ }
+ Ok(())
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+// Enumeration pruning
+//
+// Which endpoints get pruned is decided by `prune_sites` / `cat_index_columns`
+// below rather than by a list of path roots: an endpoint is prunable exactly
+// when a position is known for it, so "we prune here" and "here is where the
+// names are" cannot drift apart. Everything else — every search response
+// included — is passed through without being buffered.
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// Remove index names the principal cannot read from a metadata response.
+///
+/// `Engine::list_indices` already filters, so most listings arrive clean; this
+/// is the second pass for the handful of handlers that read the
+/// `index_settings` / `index_mappings` side maps directly, and it is what keeps
+/// `GET /_mapping` from handing over the list of brains on the node.
+async fn prune_response(
+ response: Response,
+ segs: &[String],
+ principal: &Principal,
+ state: &AppState,
+) -> Response {
+ let sites = prune_sites(segs);
+ let columns = cat_index_columns(segs).unwrap_or(&[]);
+ if sites.is_empty() && columns.is_empty() {
+ // Nothing in this response names an index. Skip the buffer entirely —
+ // it is also the only correct thing to do: pruning a body with no
+ // index-name position in it can subtract, never protect.
+ return response;
+ }
+ let (parts, body) = response.into_parts();
+ let content_type = parts
+ .headers
+ .get(header::CONTENT_TYPE)
+ .and_then(|v| v.to_str().ok())
+ .unwrap_or("")
+ .to_string();
+ let is_json = content_type.contains("json");
+ let is_text = content_type.starts_with("text/");
+ if !is_json && !is_text {
+ return Response::from_parts(parts, body);
+ }
+ let bytes = match axum::body::to_bytes(body, MAX_PRUNABLE_RESPONSE_BYTES).await {
+ Ok(b) => b,
+ // Could not buffer it, so could not prune it. Refuse rather than ship
+ // an unfiltered listing.
+ Err(_) => {
+ return es_error(
+ StatusCode::INTERNAL_SERVER_ERROR,
+ "response too large to authorize; name an explicit index".to_string(),
+ )
+ }
+ };
+
+ // The real index names on the node. A response key is only a candidate for
+ // pruning when it IS one — otherwise `"mappings"`, `"settings"` and every
+ // other structural key would be dropped for a scoped principal, which
+ // would corrupt the very responses this is protecting. Read outside the
+ // request's visibility scope on purpose: this needs the unfiltered truth.
+ let known: HashSet<String> = state.engine.index_name_list().into_iter().collect();
+
+ let pruned: Vec<u8> = if is_json {
+ match serde_json::from_slice::<Value>(&bytes) {
+ Ok(mut v) => {
+ prune_json(&mut v, sites, principal, &known);
+ serde_json::to_vec(&v).unwrap_or_else(|_| bytes.to_vec())
+ }
+ // Not JSON after all (some `_cat` handlers answer text under a
+ // JSON content type); fall through to the line filter.
+ Err(_) => prune_text(&bytes, columns, principal, &known),
+ }
+ } else {
+ prune_text(&bytes, columns, principal, &known)
+ };
+
+ let mut parts = parts;
+ parts.headers.remove(header::CONTENT_LENGTH);
+ Response::from_parts(parts, Body::from(pruned))
+}
+
+/// Is this index name one the principal must not even see?
+fn hidden(name: &str, principal: &Principal, known: &HashSet<String>) -> bool {
+ (is_reserved_index(name) || known.contains(name))
+ && !principal.allows_index(name, Privilege::ReadIndex)
+}
+
+/// One place in an endpoint's response where index **names** appear.
+///
+/// This exists because the first cut pruned by *key name at every depth*: any
+/// key equal to a real index name was deleted, wherever it appeared. An index
+/// name is an arbitrary string, so ordinary names collide with the structural
+/// keys of the very responses being pruned, and the collision only bites a
+/// scoped key — i.e. exactly the multi-tenant and Kibana case this whole
+/// branch exists to enable. Measured, on a node with one ordinary index of
+/// each name: `status` cost `GET /_cluster/health` its `status` field, the
+/// single most-polled field in the API; `indices` cost `GET /_cluster/stats`
+/// its entire `indices` section; `type` cost global `GET /_mapping` the `type`
+/// of every field definition, which is the endpoint Kibana reads to build
+/// index patterns.
+///
+/// So the positions are enumerated per endpoint instead, from the handlers'
+/// actual response shapes. `path` walks object keys from the document root,
+/// `/`-separated, where `*` matches any one key or array element and `""` is
+/// the root itself.
+#[derive(Clone, Copy, Debug, PartialEq, Eq)]
+enum Site {
+ /// The object at `path` is **keyed** by index name.
+ KeyedByIndex(&'static str),
+ /// The array at `path` **holds** index names: bare strings, or objects
+ /// whose `field` names one (`""` = bare strings only).
+ NamedIn(&'static str, &'static str),
+}
+
+/// Where index names appear in this endpoint's JSON response.
+///
+/// Anything not listed prunes nothing, which is the right default: a response
+/// that never names an index cannot leak one, and pruning it can only damage
+/// it. `_cluster/stats` is the worked example — its `indices` object holds
+/// aggregate counts, not a map keyed by index — and so are `_cluster/settings`,
+/// `_cluster/pending_tasks`, `_nodes` and `/v1/cluster/health`.
+fn prune_sites(segs: &[String]) -> &'static [Site] {
+ // `_mapping`, `_settings`, `_alias`, `_aliases`, `_recovery` and
+ // `_shard_stores` all answer one object keyed by index name at the root.
+ const ROOT_KEYED: &[Site] = &[Site::KeyedByIndex("")];
+ // `_stats` and `_segments` put the same map one level down.
+ const INDICES_KEYED: &[Site] = &[Site::KeyedByIndex("indices")];
+ match segs.first().map(String::as_str) {
+ Some("_mapping")
+ | Some("_mappings")
+ | Some("_settings")
+ | Some("_alias")
+ | Some("_aliases")
+ | Some("_recovery")
+ | Some("_shard_stores") => ROOT_KEYED,
+ Some("_stats") | Some("_segments") => INDICES_KEYED,
+ Some("_cluster") => match segs.get(1).map(String::as_str) {
+ // `level=indices` / `level=shards` add the per-index breakdown;
+ // the root is cluster-wide scalars (`status`, `timed_out`, …).
+ Some("health") => INDICES_KEYED,
+ Some("state") => &[
+ Site::KeyedByIndex("metadata/indices"),
+ Site::KeyedByIndex("routing_table/indices"),
+ ],
+ _ => &[],
+ },
+ // `{"indices": [name], "fields": {f: {type: {…, "indices": [name]}}}}`
+ Some("_field_caps") => &[
+ Site::NamedIn("indices", ""),
+ Site::NamedIn("fields/*/*/indices", ""),
+ ],
+ // `{"indices": [{name}], "aliases": [{name, indices}], "data_streams": []}`
+ Some("_resolve") => &[
+ Site::NamedIn("indices", "name"),
+ Site::NamedIn("aliases", "name"),
+ Site::NamedIn("aliases/*/indices", ""),
+ Site::NamedIn("data_streams", "name"),
+ Site::NamedIn("data_streams/*/backing_indices", ""),
+ ],
+ // `{"data_streams": [{name, indices: [{index_name}]}]}`
+ Some("_data_stream") => &[
+ Site::NamedIn("data_streams", "name"),
+ Site::NamedIn("data_streams/*/indices", "index_name"),
+ ],
+ // A `_cat` table in its `format=json` form is an array of row objects.
+ // Only the per-index tables have an `index` column — `_cat/templates`
+ // keys its rows by TEMPLATE name, which is not an index name and must
+ // not be matched against one.
+ Some("_cat") if cat_index_columns(segs).is_some() => &[Site::NamedIn("", "index")],
+ Some("v1") => match segs.get(1).map(String::as_str) {
+ // `{"data": {"indices": [{name, …}], …}}`
+ Some("dashboard") => &[Site::NamedIn("data/indices", "name")],
+ _ => &[],
+ },
+ _ => &[],
+ }
+}
+
+/// The whitespace-separated column(s) holding an index name in a `_cat`
+/// table's plain-text form, by sub-verb; `None` means this table has no index
+/// column, so no row of it may be dropped for containing an index name.
+///
+/// Column numbers are read off the handlers in `es_compat`, not guessed. The
+/// alternative — "drop a row if ANY token is a hidden index name" — has the
+/// same collision as key-matching did: an ordinary index named `open`, `green`
+/// or `p` that a scoped key cannot read would empty `_cat/indices` and
+/// `_cat/shards` completely.
+fn cat_index_columns(segs: &[String]) -> Option<&'static [usize]> {
+ match segs.get(1).map(String::as_str)? {
+ // health status INDEX uuid pri rep docs.count docs.deleted store …
+ "indices" => Some(&[2]),
+ // ALIAS INDEX filter routing.index routing.search is_write_index.
+ // Column 0 is an alias, not an index — but an alias name IS a name in
+ // this namespace elsewhere in this module (it is authorized as a
+ // resource, and it resolves to indices), so a reserved-namespace alias
+ // is hidden here too.
+ "aliases" => Some(&[0, 1]),
+ // INDEX shard prirep state docs store ip node
+ "shards" => Some(&[0]),
+ // INDEX shard time type stage …
+ "recovery" => Some(&[0]),
+ // INDEX shard prirep ip segment …
+ "segments" => Some(&[0]),
+ // INDEX present field nodes tombstones …
+ "ann" => Some(&[0]),
+ // id host ip node INDEX size
+ "fielddata" => Some(&[4]),
+ // Everything else under `_cat` — nodes, health, master, plugins,
+ // templates, thread_pool, nodeattrs, pending_tasks, allocation, count,
+ // ml/* — reports no index name at all.
+ _ => None,
+ }
+}
+
+/// Apply `f` to every node the site `path` selects. `*` matches any one object
+/// key or array element; an empty path selects the node itself.
+fn for_each_at(v: &mut Value, path: &str, f: &mut dyn FnMut(&mut Value)) {
+ if path.is_empty() {
+ f(v);
+ return;
+ }
+ let (head, rest) = path.split_once('/').unwrap_or((path, ""));
+ match v {
+ Value::Object(map) => {
+ if head == "*" {
+ for (_, child) in map.iter_mut() {
+ for_each_at(child, rest, f);
+ }
+ } else if let Some(child) = map.get_mut(head) {
+ for_each_at(child, rest, f);
+ }
+ }
+ Value::Array(items) if head == "*" => {
+ for item in items.iter_mut() {
+ for_each_at(item, rest, f);
+ }
+ }
+ _ => {}
+ }
+}
+
+/// Drop the index names this principal may not read, at the positions this
+/// endpoint actually carries them — and nowhere else.
+fn prune_json(v: &mut Value, sites: &[Site], principal: &Principal, known: &HashSet<String>) {
+ for site in sites {
+ match *site {
+ Site::KeyedByIndex(path) => for_each_at(v, path, &mut |node| {
+ if let Value::Object(map) = node {
+ map.retain(|k, _| !hidden(k, principal, known));
+ }
+ }),
+ Site::NamedIn(path, field) => for_each_at(v, path, &mut |node| {
+ if let Value::Array(items) = node {
+ items.retain(|item| match item {
+ Value::String(s) => !hidden(s, principal, known),
+ Value::Object(o) if !field.is_empty() => !o
+ .get(field)
+ .and_then(Value::as_str)
+ .map(|n| hidden(n, principal, known))
+ .unwrap_or(false),
+ _ => true,
+ });
+ }
+ }),
+ }
+ }
+}
+
+/// Line filter for `_cat` in its default text form: a row whose index column
+/// names a hidden index is dropped whole. A table with no index column
+/// (`columns` is empty) is returned untouched.
+fn prune_text(
+ bytes: &[u8],
+ columns: &[usize],
+ principal: &Principal,
+ known: &HashSet<String>,
+) -> Vec<u8> {
+ if columns.is_empty() {
+ return bytes.to_vec();
+ }
+ let Ok(text) = std::str::from_utf8(bytes) else {
+ return bytes.to_vec();
+ };
+ let mut out = String::with_capacity(text.len());
+ let mut dropped_any = false;
+ for line in text.split_inclusive('\n') {
+ let fields: Vec<&str> = line.split_whitespace().collect();
+ let hides = columns.iter().any(|&c| {
+ fields
+ .get(c)
+ .map(|token| {
+ hidden(
+ token.trim_matches(|ch: char| ch == '"' || ch == ','),
+ principal,
+ known,
+ )
+ })
+ .unwrap_or(false)
+ });
+ if hides {
+ dropped_any = true;
+ } else {
+ out.push_str(line);
+ }
+ }
+ if dropped_any {
+ out.into_bytes()
+ } else {
+ bytes.to_vec()
+ }
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+// Tests
+// ─────────────────────────────────────────────────────────────────────────────
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use std::collections::HashMap;
+ use xerj_engine::rbac::Role;
+
+ fn scoped(indices: &[&str], privileges: &[Privilege]) -> Principal {
+ Principal::Scoped {
+ key_id: "k1".into(),
+ roles: vec![Role::new(
+ "r",
+ privileges.iter().copied().collect::<HashSet<_>>(),
+ indices.iter().map(|s| s.to_string()).collect(),
+ )],
+ }
+ }
+
+ fn unscoped() -> Principal {
+ Principal::Unscoped {
+ key_id: "k2".into(),
+ }
+ }
+
+ struct Map(HashMap<String, Vec<String>>);
+ impl AliasResolver for Map {
+ fn targets(&self, name: &str) -> Option<Vec<String>> {
+ self.0.get(name).cloned()
+ }
+ }
+
+ /// Run the whole decision the way the middleware does.
+ fn check(p: &Principal, method: Method, path: &str, body: &str) -> bool {
+ check_with(p, method, path, body, &NoAliases)
+ }
+
+ fn check_with(
+ p: &Principal,
+ method: Method,
+ path: &str,
+ body: &str,
+ aliases: &dyn AliasResolver,
+ ) -> bool {
+ let segs = segments(path);
+ let target = classify(path);
+ let shape = body_shape(&method, &segs);
+ decide(p, &method, &segs, &target, shape, body.as_bytes(), aliases).is_ok()
+ }
+
+ #[test]
+ fn reserved_prefix_recognition() {
+ assert!(is_reserved_index(".xerj-memory-alice-edges"));
+ assert!(is_reserved_index(".xerj-memory-alice"));
+ assert!(!is_reserved_index("logs-2026"));
+ assert!(!is_reserved_index(".kibana"));
+ }
+
+ #[test]
+ fn wildcards_that_could_reach_the_namespace_are_recognized() {
+ for pattern in ["*", "_all", ".*", ".xerj-*", ".xerj-memory-alice*", ".x*"] {
+ assert!(may_reach_reserved(pattern), "{pattern} must be caught");
+ }
+ for pattern in ["logs-*", "notes", ".kibana*", ".xerj-other-*"] {
+ assert!(!may_reach_reserved(pattern), "{pattern} must be allowed");
+ }
+ }
+
+ #[test]
+ fn percent_encoded_reserved_names_are_decoded() {
+ // `%2E` is `.` — without decoding, the leading-dot check would miss it.
+ assert_eq!(
+ percent_decode("%2Exerj-memory-alice-edges"),
+ ".xerj-memory-alice-edges"
+ );
+ assert_eq!(
+ classify("/%2Exerj-memory-alice-edges/_search"),
+ Target::Indices(vec![".xerj-memory-alice-edges".into()], 1)
+ );
+ }
+
+ #[test]
+ fn classification() {
+ assert_eq!(classify("/_graph/alice/ego"), Target::Brain("alice".into()));
+ assert_eq!(
+ classify("/_memory/alice/_recall"),
+ Target::Memory("alice".into())
+ );
+ assert_eq!(classify("/_search"), Target::Cluster);
+ assert_eq!(classify("/_bulk"), Target::Cluster);
+ assert_eq!(classify("/_mapping"), Target::Cluster);
+ assert_eq!(classify("/_cat/indices"), Target::Cluster);
+ // A pattern is an index expression, not a cluster endpoint.
+ assert_eq!(
+ classify("/_all/_search"),
+ Target::Indices(vec!["_all".into()], 1)
+ );
+ assert_eq!(classify("/*/_search"), Target::Indices(vec!["*".into()], 1));
+ assert_eq!(classify("/health/live"), Target::Exempt);
+ assert_eq!(classify("/"), Target::Exempt);
+ assert_eq!(
+ classify("/logs-a,logs-b/_search"),
+ Target::Indices(vec!["logs-a".into(), "logs-b".into()], 1)
+ );
+ // Native router: the index is the third segment, not the first.
+ assert_eq!(
+ classify("/v1/indices/.xerj-memory-bob-edges/search"),
+ Target::Indices(vec![".xerj-memory-bob-edges".into()], 3)
+ );
+ assert_eq!(classify("/v1/dashboard/summary"), Target::Cluster);
+ assert_eq!(classify("/v1/indices"), Target::Cluster);
+ }
+
+ /// Every route that can take an index name from its body must be mapped to
+ /// the shape that finds it. This is the audit, executable.
+ #[test]
+ fn body_shape_table() {
+ let cases: &[(&str, &str, BodyShape)] = &[
+ ("POST", "/_bulk", BodyShape::Bulk),
+ ("POST", "/logs/_bulk", BodyShape::Bulk),
+ ("POST", "/_msearch", BodyShape::MsearchHeaders),
+ ("POST", "/logs/_msearch", BodyShape::MsearchHeaders),
+ ("POST", "/_msearch/template", BodyShape::MsearchHeaders),
+ ("POST", "/logs/_msearch/template", BodyShape::MsearchHeaders),
+ ("POST", "/_mget", BodyShape::MgetDocs),
+ ("POST", "/logs/_mget", BodyShape::MgetDocs),
+ ("POST", "/_aliases", BodyShape::AliasActions),
+ ("POST", "/_reindex", BodyShape::Reindex),
+ ("PUT", "/_snapshot/repo/snap", BodyShape::SnapshotIndices),
+ (
+ "POST",
+ "/_snapshot/repo/snap/_restore",
+ BodyShape::SnapshotIndices,
+ ),
+ ("POST", "/v1/indices", BodyShape::NativeCreate),
+ ("PUT", "/logs-2026", BodyShape::CreateIndex),
+ ("POST", "/_monitoring/bulk", BodyShape::MonitoringBulk),
+ ("PUT", "/_index_template/t", BodyShape::IndexTemplate),
+ ("PUT", "/_template/t", BodyShape::IndexTemplate),
+ (
+ "POST",
+ "/_index_template/_simulate",
+ BodyShape::IndexTemplate,
+ ),
+ ("PUT", "/_ml/anomaly_detectors/j", BodyShape::MlConfig),
+ ("PUT", "/_ml/datafeeds/f", BodyShape::MlConfig),
+ ("POST", "/_ml/datafeeds/f", BodyShape::MlConfig),
+ // Sub-verbs and reads carry no index name of their own.
+ ("POST", "/_ml/datafeeds/f/_start", BodyShape::None),
+ ("POST", "/_ml/anomaly_detectors/j/_score", BodyShape::None),
+ ("GET", "/_ml/anomaly_detectors/j", BodyShape::None),
+ ("POST", "/_search", BodyShape::Query),
+ ("POST", "/logs/_search", BodyShape::Query),
+ ("POST", "/_count", BodyShape::Query),
+ ("POST", "/logs/_delete_by_query", BodyShape::Query),
+ ("POST", "/logs/_update_by_query", BodyShape::Query),
+ ("POST", "/_sql", BodyShape::Query),
+ ("POST", "/logs/_eql/search", BodyShape::Query),
+ // Opaque document bodies: nothing in them is an index name.
+ ("PUT", "/logs/_doc/1", BodyShape::None),
+ ("POST", "/logs/_update/1", BodyShape::None),
+ ("POST", "/v1/indices/logs/docs", BodyShape::None),
+ ("POST", "/v1/indices/logs/docs/_bulk", BodyShape::None),
+ ];
+ for (method, path, want) in cases {
+ let m: Method = method.parse().unwrap();
+ let got = body_shape(&m, &segments(path));
+ assert_eq!(got, *want, "{method} {path}");
+ }
+ }
+
+ #[test]
+ fn privilege_mapping() {
+ let s = |p: &str| segments(p);
+ assert_eq!(
+ required_privilege(&Method::POST, &s("/idx/_search"), 1),
+ Privilege::ReadIndex
+ );
+ assert_eq!(
+ required_privilege(&Method::POST, &s("/idx/_doc/1"), 1),
+ Privilege::WriteIndex
+ );
+ assert_eq!(
+ required_privilege(&Method::DELETE, &s("/idx"), 1),
+ Privilege::AdminIndex
+ );
+ assert_eq!(
+ required_privilege(&Method::PUT, &s("/idx/_settings"), 1),
+ Privilege::AdminIndex
+ );
+ // Native spellings resolve to the same privileges.
+ assert_eq!(
+ required_privilege(&Method::POST, &s("/v1/indices/idx/search"), 3),
+ Privilege::ReadIndex
+ );
+ assert_eq!(
+ required_privilege(&Method::POST, &s("/v1/indices/idx/docs"), 3),
+ Privilege::WriteIndex
+ );
+ assert_eq!(
+ required_privilege(&Method::DELETE, &s("/v1/indices/idx"), 3),
+ Privilege::AdminIndex
+ );
+ assert_eq!(
+ reserved_api_privilege(&Method::POST, &s("/_graph/a/link")),
+ Privilege::WriteIndex
+ );
+ assert_eq!(
+ reserved_api_privilege(&Method::GET, &s("/_graph/a/ego")),
+ Privilege::ReadIndex
+ );
+ assert_eq!(
+ reserved_api_privilege(&Method::DELETE, &s("/_memory/a")),
+ Privilege::AdminIndex
+ );
+ assert_eq!(
+ reserved_api_privilege(&Method::DELETE, &s("/_memory/a/doc1")),
+ Privilege::WriteIndex
+ );
+ }
+
+ /// The decision table, stated once. `decide` is the only thing between an
+ /// authenticated caller and someone else's brain.
+ #[test]
+ fn decision_table() {
+ let alice = scoped(
+ &[".xerj-memory-alice-edges", ".xerj-memory-alice"],
+ &[Privilege::ReadIndex, Privilege::WriteIndex],
+ );
+
+ // Alice reaches her own brain …
+ assert!(check(&alice, Method::GET, "/_graph/alice/ego", ""));
+ assert!(check(&alice, Method::GET, "/_graph/alice/overview", ""));
+ assert!(check(&alice, Method::POST, "/_graph/alice/link", ""));
+ assert!(check(&alice, Method::POST, "/_memory/alice/_recall", ""));
+ // … and nothing of bob's, by any door.
+ assert!(!check(&alice, Method::GET, "/_graph/bob/ego", ""));
+ assert!(!check(&alice, Method::POST, "/_graph/bob/link", ""));
+ assert!(!check(&alice, Method::DELETE, "/_graph/bob/link/e1", ""));
+ assert!(!check(
+ &alice,
+ Method::POST,
+ "/.xerj-memory-bob-edges/_search",
+ ""
+ ));
+ assert!(!check(
+ &alice,
+ Method::POST,
+ "/.xerj-memory-bob-edges/_doc/x",
+ ""
+ ));
+ assert!(!check(
+ &alice,
+ Method::DELETE,
+ "/.xerj-memory-bob-edges",
+ ""
+ ));
+ assert!(!check(&alice, Method::POST, "/_memory/bob/_recall", ""));
+ // Including the native router's spelling of the same index.
+ assert!(!check(
+ &alice,
+ Method::POST,
+ "/v1/indices/.xerj-memory-bob-edges/search",
+ ""
+ ));
+ // A grant is a grant: alice may use her own backing index directly.
+ assert!(check(
+ &alice,
+ Method::POST,
+ "/.xerj-memory-alice-edges/_search",
+ ""
+ ));
+
+ // A read-only grant does not write.
+ let ro = scoped(&[".xerj-memory-alice-edges"], &[Privilege::ReadIndex]);
+ assert!(check(&ro, Method::GET, "/_graph/alice/ego", ""));
+ assert!(!check(&ro, Method::POST, "/_graph/alice/link", ""));
+ assert!(!check(&ro, Method::DELETE, "/.xerj-memory-alice-edges", ""));
+
+ // A legacy key with no grants: keeps ordinary indices, loses every
+ // brain door. This is the fail-closed half.
+ let legacy = unscoped();
+ assert!(check(&legacy, Method::POST, "/logs-2026/_search", ""));
+ assert!(check(&legacy, Method::GET, "/logs-*/_search", ""));
+ assert!(check(&legacy, Method::GET, "/_mapping", ""));
+ assert!(!check(&legacy, Method::GET, "/_graph/alice/ego", ""));
+ assert!(!check(&legacy, Method::POST, "/_memory/alice/_recall", ""));
+ assert!(!check(
+ &legacy,
+ Method::POST,
+ "/.xerj-memory-alice-edges/_search",
+ ""
+ ));
+ assert!(check(&legacy, Method::PUT, "/logs-2026", ""));
+
+ // No credential resolves to nothing at all.
+ let denied = Principal::Denied;
+ assert!(!check(&denied, Method::GET, "/_graph/alice/ego", ""));
+ assert!(!check(&denied, Method::GET, "/logs/_search", ""));
+ assert!(!check(&denied, Method::GET, "/_mapping", ""));
+
+ // The superuser is untouched — this is the `--insecure` path.
+ let root = Principal::Superuser;
+ for path in [
+ "/_graph/bob/ego",
+ "/_mapping",
+ "/_search",
+ "/.xerj-memory-bob-edges/_search",
+ "/*/_search",
+ ] {
+ assert!(
+ check(&root, Method::GET, path, ""),
+ "superuser blocked on {path}"
+ );
+ }
+ }
+
+ /// BYPASS 1-3: the index in the body is the one that gets authorized.
+ #[test]
+ fn body_named_indices_are_authorized_not_the_path_one() {
+ let alice = scoped(
+ &[".xerj-memory-alice-edges", ".xerj-memory-alice"],
+ &[Privilege::ReadIndex, Privilege::WriteIndex],
+ );
+
+ // _msearch: the header line overrides the path index.
+ let msearch = "{\"index\":\".xerj-memory-bob-edges\"}\n{\"query\":{\"match_all\":{}}}\n";
+ assert!(!check(
+ &alice,
+ Method::POST,
+ "/.xerj-memory-alice-edges/_msearch",
+ msearch
+ ));
+ assert!(!check(&alice, Method::POST, "/_msearch", msearch));
+ // Her own brain through the same door still works.
+ let mine = "{\"index\":\".xerj-memory-alice-edges\"}\n{\"query\":{\"match_all\":{}}}\n";
+ assert!(check(&alice, Method::POST, "/_msearch", mine));
+
+ // _bulk: the action line's `_index` overrides the path index.
+ let forge =
+ "{\"index\":{\"_index\":\".xerj-memory-bob-edges\",\"_id\":\"f\"}}\n{\"src\":\"x\"}\n";
+ assert!(!check(
+ &alice,
+ Method::POST,
+ "/.xerj-memory-alice-edges/_bulk",
+ forge
+ ));
+ assert!(!check(&alice, Method::POST, "/_bulk", forge));
+ let destroy = "{\"delete\":{\"_index\":\".xerj-memory-bob-edges\",\"_id\":\"e1\"}}\n";
+ assert!(!check(&alice, Method::POST, "/_bulk", destroy));
+ // A bulk that stays inside her grant is allowed — the global `_bulk`
+ // is the default write path for every ES client and must work.
+ let ok = "{\"index\":{\"_index\":\".xerj-memory-alice-edges\",\"_id\":\"a\"}}\n{\"src\":\"x\"}\n";
+ assert!(check(&alice, Method::POST, "/_bulk", ok));
+
+ // _mget: docs[]._index overrides the path index.
+ let mget = "{\"docs\":[{\"_index\":\".xerj-memory-bob-edges\",\"_id\":\"1\"}]}";
+ assert!(!check(
+ &alice,
+ Method::POST,
+ "/.xerj-memory-alice-edges/_mget",
+ mget
+ ));
+ assert!(!check(&alice, Method::POST, "/_mget", mget));
+ assert!(check(
+ &alice,
+ Method::POST,
+ "/.xerj-memory-alice-edges/_mget",
+ "{\"ids\":[\"1\"]}"
+ ));
+
+ // An unscoped legacy key holds nothing in the reserved namespace by
+ // any of the three doors either.
+ let legacy = unscoped();
+ assert!(!check(&legacy, Method::POST, "/_msearch", msearch));
+ assert!(!check(&legacy, Method::POST, "/_bulk", forge));
+ assert!(!check(&legacy, Method::POST, "/_mget", mget));
+ // …but its ordinary-index traffic is untouched.
+ let plain = "{\"index\":{\"_index\":\"logs-2026\"}}\n{\"m\":1}\n";
+ assert!(check(&legacy, Method::POST, "/_bulk", plain));
+ }
+
+ /// BYPASS 4: `POST /_aliases` authorizes the indices it names, and an
+ /// alias already pointing into the namespace does not launder reads.
+ #[test]
+ fn aliases_cannot_launder_access() {
+ let legacy = unscoped();
+ let add =
+ "{\"actions\":[{\"add\":{\"index\":\".xerj-memory-bob-edges\",\"alias\":\"pwned\"}}]}";
+ assert!(!check(&legacy, Method::POST, "/_aliases", add));
+ // Also refused for a scoped key that holds a different brain.
+ let alice = scoped(
+ &[".xerj-memory-alice-edges"],
+ &[
+ Privilege::ReadIndex,
+ Privilege::WriteIndex,
+ Privilege::AdminIndex,
+ ],
+ );
+ assert!(!check(&alice, Method::POST, "/_aliases", add));
+ // An ordinary alias over an ordinary index still works for the legacy
+ // key that has the general surface.
+ let ordinary = "{\"actions\":[{\"add\":{\"index\":\"logs-2026\",\"alias\":\"logs\"}}]}";
+ assert!(check(&legacy, Method::POST, "/_aliases", ordinary));
+ // Squatting an alias NAME inside the reserved namespace is refused.
+ let squat =
+ "{\"actions\":[{\"add\":{\"index\":\"logs-2026\",\"alias\":\".xerj-memory-victim\"}}]}";
+ assert!(!check(&legacy, Method::POST, "/_aliases", squat));
+ // …by the path spelling too.
+ assert!(!check(
+ &legacy,
+ Method::PUT,
+ "/logs-2026/_alias/.xerj-memory-victim",
+ ""
+ ));
+
+ // Reading THROUGH an alias that already points at a brain resolves to
+ // the brain before the decision is made.
+ let mut map = HashMap::new();
+ map.insert(
+ "pwned".to_string(),
+ vec![".xerj-memory-bob-edges".to_string()],
+ );
+ let aliases = Map(map);
+ assert!(!check_with(
+ &legacy,
+ Method::POST,
+ "/pwned/_search",
+ "",
+ &aliases
+ ));
+ assert!(!check_with(
+ &legacy,
+ Method::POST,
+ "/_mget",
+ "{\"docs\":[{\"_index\":\"pwned\",\"_id\":\"1\"}]}",
+ &aliases
+ ));
+ }
+
+ /// The fifth shape, found by audit rather than by attack: a `terms` lookup
+ /// reads a document out of an index the URL never names.
+ #[test]
+ fn terms_lookup_index_is_authorized() {
+ let alice = scoped(&["logs-2026"], &[Privilege::ReadIndex]);
+ let lookup = r#"{"query":{"bool":{"filter":[{"terms":{"user":{"index":".xerj-memory-bob-edges","id":"1","path":"dst"}}}]}}}"#;
+ assert!(!check(&alice, Method::POST, "/logs-2026/_search", lookup));
+ // A lookup into an index she holds is fine.
+ let own = r#"{"query":{"terms":{"user":{"index":"logs-2026","id":"1","path":"u"}}}}"#;
+ assert!(check(&alice, Method::POST, "/logs-2026/_search", own));
+ // A plain `terms` filter (no lookup) names no index and is untouched —
+ // a document field called `index` must not be read as one.
+ let plain = r#"{"query":{"terms":{"index":["a","b"]}}}"#;
+ assert!(check(&alice, Method::POST, "/logs-2026/_search", plain));
+
+ // The same shape one layer over: a `lookup` runtime field joins rows
+ // out of another index into the hits.
+ let runtime = r#"{"runtime_mappings":{"u":{"type":"lookup","target_index":".xerj-memory-bob-edges","input_field":"src","target_field":"_id","fetch_fields":["dst"]}}}"#;
+ assert!(!check(&alice, Method::POST, "/logs-2026/_search", runtime));
+ let own = r#"{"runtime_mappings":{"u":{"type":"lookup","target_index":"logs-2026","input_field":"src","target_field":"_id"}}}"#;
+ assert!(check(&alice, Method::POST, "/logs-2026/_search", own));
+ }
+
+ /// `_reindex` reads its source and writes its destination.
+ #[test]
+ fn reindex_authorizes_source_and_destination() {
+ let p = scoped(
+ &["logs-2026", "logs-copy"],
+ &[Privilege::ReadIndex, Privilege::WriteIndex],
+ );
+ let ok = r#"{"source":{"index":"logs-2026"},"dest":{"index":"logs-copy"}}"#;
+ assert!(check(&p, Method::POST, "/_reindex", ok));
+ let steal = r#"{"source":{"index":".xerj-memory-bob-edges"},"dest":{"index":"logs-copy"}}"#;
+ assert!(!check(&p, Method::POST, "/_reindex", steal));
+ let overwrite =
+ r#"{"source":{"index":"logs-2026"},"dest":{"index":".xerj-memory-bob-edges"}}"#;
+ assert!(!check(&p, Method::POST, "/_reindex", overwrite));
+ // A body that names neither is refused rather than guessed at.
+ assert!(!check(&p, Method::POST, "/_reindex", "{}"));
+ assert!(!check(&p, Method::POST, "/_reindex", "not json"));
+ }
+
+ /// The native router's body-named create is authorized, not blanket-refused.
+ #[test]
+ fn native_body_named_create_is_authorized() {
+ let legacy = unscoped();
+ assert!(check(
+ &legacy,
+ Method::POST,
+ "/v1/indices",
+ r#"{"name":"logs-2026","fields":[]}"#
+ ));
+ assert!(!check(
+ &legacy,
+ Method::POST,
+ "/v1/indices",
+ r#"{"name":".xerj-memory-victim-edges","fields":[]}"#
+ ));
+ assert!(!check(&legacy, Method::POST, "/v1/indices", "{}"));
+ }
+
+ /// A snapshot or restore that names no indices covers every brain on the
+ /// node, so it is superuser-only.
+ #[test]
+ fn unbounded_snapshot_is_superuser_only() {
+ let legacy = unscoped();
+ assert!(!check(&legacy, Method::PUT, "/_snapshot/repo/s1", "{}"));
+ assert!(!check(
+ &legacy,
+ Method::POST,
+ "/_snapshot/repo/s1/_restore",
+ "{}"
+ ));
+ assert!(check(
+ &legacy,
+ Method::PUT,
+ "/_snapshot/repo/s1",
+ r#"{"indices":["logs-2026"]}"#
+ ));
+ assert!(!check(
+ &legacy,
+ Method::PUT,
+ "/_snapshot/repo/s1",
+ r#"{"indices":[".xerj-memory-bob-edges"]}"#
+ ));
+ assert!(check(
+ &Principal::Superuser,
+ Method::PUT,
+ "/_snapshot/repo/s1",
+ "{}"
+ ));
+ }
+
+ /// COMPATIBILITY: the regression that made the branch unusable. Every
+ /// global verb works for an ordinary credential, and a scoped key can read
+ /// cluster metadata and use a wildcard it was granted.
+ #[test]
+ fn ordinary_credentials_keep_the_global_surface() {
+ let legacy = unscoped();
+ let scoped_all = scoped(
+ &["*"],
+ &[
+ Privilege::ReadIndex,
+ Privilege::WriteIndex,
+ Privilege::AdminIndex,
+ ],
+ );
+ let logs = scoped(&["logs-*"], &[Privilege::ReadIndex, Privilege::WriteIndex]);
+
+ for p in [&legacy, &scoped_all, &logs] {
+ assert!(check(p, Method::POST, "/_search", "{}"), "global _search");
+ assert!(check(p, Method::POST, "/_count", "{}"), "global _count");
+ assert!(
+ check(p, Method::POST, "/_msearch", "{}\n{\"query\":{}}\n"),
+ "global _msearch"
+ );
+ assert!(
+ check(
+ p,
+ Method::POST,
+ "/_bulk",
+ "{\"index\":{\"_index\":\"logs-2026\"}}\n{\"m\":1}\n"
+ ),
+ "global _bulk"
+ );
+ assert!(
+ check(
+ p,
+ Method::POST,
+ "/_mget",
+ "{\"docs\":[{\"_index\":\"logs-2026\",\"_id\":\"1\"}]}"
+ ),
+ "global _mget"
+ );
+ assert!(check(p, Method::GET, "/_cluster/health", ""), "health");
+ assert!(check(p, Method::GET, "/_nodes", ""), "nodes");
+ assert!(check(p, Method::GET, "/_mapping", ""), "mapping");
+ assert!(check(p, Method::GET, "/_cat/indices", ""), "cat");
+ assert!(check(p, Method::GET, "/v1/health", ""), "native health");
+ assert!(
+ check(p, Method::GET, "/_resolve/index/logs-*", ""),
+ "resolve"
+ );
+ }
+ // A granted wildcard is usable, and an ungranted one resolves to
+ // nothing rather than 403 (the engine expands it over the visible set).
+ assert!(check(&logs, Method::GET, "/logs-*/_search", ""));
+ assert!(check(&scoped_all, Method::GET, "/logs-*/_search", ""));
+ assert!(check(&logs, Method::GET, "/.xerj-memory-*/_search", ""));
+ // A scoped key still cannot NAME another tenant's index.
+ assert!(!check(
+ &logs,
+ Method::GET,
+ "/.xerj-memory-bob-edges/_search",
+ ""
+ ));
+ // …nor mutate the cluster itself.
+ assert!(!check(&logs, Method::PUT, "/_cluster/settings", "{}"));
+ assert!(check(&legacy, Method::PUT, "/_cluster/settings", "{}"));
+ }
+
+ /// Prune the way the middleware does: pick the endpoint's sites from its
+ /// path, then apply them.
+ fn prune_at(path: &str, v: &mut Value, p: &Principal, known: &HashSet<String>) {
+ prune_json(v, prune_sites(&segments(path)), p, known);
+ }
+
+ fn prune_rows(path: &str, table: &str, p: &Principal, known: &HashSet<String>) -> String {
+ let cols = cat_index_columns(&segments(path)).unwrap_or(&[]);
+ String::from_utf8(prune_text(table.as_bytes(), cols, p, known)).expect("utf8")
+ }
+
+ #[test]
+ fn json_pruning_hides_unreadable_indices() {
+ let p = scoped(&[".xerj-memory-alice-edges"], &[Privilege::ReadIndex]);
+ let known: HashSet<String> = [
+ ".xerj-memory-alice-edges",
+ ".xerj-memory-bob-edges",
+ "logs-2026",
+ ]
+ .iter()
+ .map(|s| s.to_string())
+ .collect();
+ let mut mapping = serde_json::json!({
+ ".xerj-memory-alice-edges": {"mappings": {}},
+ ".xerj-memory-bob-edges": {"mappings": {}},
+ "logs-2026": {"mappings": {}}
+ });
+ prune_at("/_mapping", &mut mapping, &p, &known);
+ assert!(mapping.get(".xerj-memory-alice-edges").is_some());
+ assert!(mapping.get(".xerj-memory-bob-edges").is_none());
+ // A scoped key does not see another tenant's ordinary index either …
+ assert!(mapping.get("logs-2026").is_none());
+ // … but structural keys are never mistaken for index names.
+ assert!(mapping[".xerj-memory-alice-edges"]
+ .get("mappings")
+ .is_some());
+
+ // Array shapes: `_cat/indices?format=json` and `_resolve/index`.
+ let mut cat = serde_json::json!([
+ {"index": ".xerj-memory-bob-edges", "docs.count": "1"},
+ {"index": ".xerj-memory-alice-edges", "docs.count": "9"}
+ ]);
+ prune_at("/_cat/indices", &mut cat, &p, &known);
+ assert_eq!(cat.as_array().unwrap().len(), 1);
+
+ let mut caps =
+ serde_json::json!({"indices": [".xerj-memory-bob-edges", ".xerj-memory-alice-edges"]});
+ prune_at("/_field_caps", &mut caps, &p, &known);
+ assert_eq!(
+ caps["indices"],
+ serde_json::json!([".xerj-memory-alice-edges"])
+ );
+ // …including the per-field `indices` lists buried under `fields`.
+ let mut caps = serde_json::json!({
+ "indices": [".xerj-memory-alice-edges", ".xerj-memory-bob-edges"],
+ "fields": {"host": {"keyword": {
+ "type": "keyword",
+ "indices": [".xerj-memory-alice-edges", ".xerj-memory-bob-edges"]
+ }}}
+ });
+ prune_at("/_field_caps", &mut caps, &p, &known);
+ assert_eq!(
+ caps["fields"]["host"]["keyword"]["indices"],
+ serde_json::json!([".xerj-memory-alice-edges"])
+ );
+ assert_eq!(caps["fields"]["host"]["keyword"]["type"], "keyword");
+ }
+
+ /// FINDING B. An index name is an arbitrary string, so the previous
+ /// "delete any key equal to a known index name, at every depth" collided
+ /// with the structural keys of the responses it was protecting — and only
+ /// ever for a scoped key, i.e. exactly the multi-tenant case this branch
+ /// exists to enable. Each case below is one measured breakage.
+ #[test]
+ fn structural_keys_survive_ordinary_indices_that_share_their_name() {
+ // Ordinary indices with unfortunate names, none of them readable by
+ // this principal.
+ let p = scoped(&["logs-2026"], &[Privilege::ReadIndex]);
+ let known: HashSet<String> = ["status", "indices", "type", "nodes", "count", "logs-2026"]
+ .iter()
+ .map(|s| s.to_string())
+ .collect();
+
+ // `GET /_cluster/health` kept its `status` — the most-polled field in
+ // the API — while still pruning the per-index breakdown.
+ let mut health = serde_json::json!({
+ "cluster_name": "xerj", "status": "green", "timed_out": false,
+ "indices": {"logs-2026": {"status": "green"}, "status": {"status": "green"}}
+ });
+ prune_at("/_cluster/health", &mut health, &p, &known);
+ assert_eq!(health["status"], "green");
+ assert!(health["indices"].get("logs-2026").is_some());
+ assert!(health["indices"].get("status").is_none());
+
+ // `GET /_cluster/stats` kept its whole `indices` section: those are
+ // aggregate counters, not a map keyed by index name.
+ let mut stats = serde_json::json!({
+ "status": "green",
+ "indices": {"count": 5, "docs": {"count": 42}},
+ "nodes": {"count": {"total": 1}}
+ });
+ let before = stats.clone();
+ prune_at("/_cluster/stats", &mut stats, &p, &known);
+ assert_eq!(stats, before, "_cluster/stats names no index anywhere");
+
+ // Global `GET /_mapping` kept every field's `type`, which is what
+ // Kibana reads to build an index pattern.
+ let mut mapping = serde_json::json!({
+ "logs-2026": {"mappings": {"properties": {
+ "host": {"type": "keyword"},
+ "count": {"type": "long"}
+ }}},
+ "type": {"mappings": {"properties": {}}}
+ });
+ prune_at("/_mapping", &mut mapping, &p, &known);
+ assert_eq!(
+ mapping["logs-2026"]["mappings"]["properties"]["host"]["type"],
+ "keyword"
+ );
+ assert_eq!(
+ mapping["logs-2026"]["mappings"]["properties"]["count"]["type"],
+ "long"
+ );
+ assert!(
+ mapping.get("type").is_none(),
+ "the index actually named `type` is still hidden"
+ );
+
+ // `GET /_cluster/state` prunes its two index maps and leaves the node
+ // map alone, even when an index shares a node's name.
+ let mut state = serde_json::json!({
+ "nodes": {"nodes": {"name": "nodes"}},
+ "metadata": {"templates": {}, "indices": {"logs-2026": {}, "count": {}}},
+ "routing_table": {"indices": {"logs-2026": {}, "count": {}}}
+ });
+ prune_at("/_cluster/state", &mut state, &p, &known);
+ assert!(state["nodes"].get("nodes").is_some(), "node map untouched");
+ assert!(state["metadata"].get("templates").is_some());
+ assert!(state["metadata"]["indices"].get("logs-2026").is_some());
+ assert!(state["metadata"]["indices"].get("count").is_none());
+ assert!(state["routing_table"]["indices"].get("count").is_none());
+ }
+
+ /// The same collision in the `_cat` text tables: matching *any* token
+ /// meant one unreadable index called `open` or `green` emptied the table.
+ #[test]
+ fn cat_rows_are_matched_on_the_index_column_only() {
+ let p = scoped(&["logs-2026"], &[Privilege::ReadIndex]);
+ let known: HashSet<String> = ["logs-2026", "open", "green", ".xerj-memory-bob-edges"]
+ .iter()
+ .map(|s| s.to_string())
+ .collect();
+
+ let table = "green open logs-2026 uuid 1 0 9 0 1kb 1kb\n\
+ green open .xerj-memory-bob-edges uuid 1 0 1 0 1kb 1kb\n\
+ green open open uuid 1 0 3 0 1kb 1kb\n";
+ let out = prune_rows("/_cat/indices", table, &p, &known);
+ assert!(out.contains("logs-2026"), "own index survives: {out}");
+ assert!(
+ !out.contains(".xerj-memory-bob-edges"),
+ "brain hidden: {out}"
+ );
+ assert_eq!(
+ out.lines().count(),
+ 1,
+ "only the row whose INDEX column is hidden goes: {out}"
+ );
+
+ // `_cat/health` has no index column at all, so nothing may be dropped
+ // from it however its values happen to read.
+ let health = "1785000000 12:00:00 xerj green 1 1 4 4 0 0 0 0 - 100.0%\n";
+ assert_eq!(prune_rows("/_cat/health", health, &p, &known), health);
+
+ // `_cat/fielddata` carries the index in column 4.
+ let fd = "n1 127.0.0.1 127.0.0.1 n1 logs-2026 0b\n\
+ n1 127.0.0.1 127.0.0.1 n1 .xerj-memory-bob-edges 0b\n";
+ let out = prune_rows("/_cat/fielddata", fd, &p, &known);
+ assert!(out.contains("logs-2026"));
+ assert!(!out.contains(".xerj-memory-bob-edges"));
+ }
+
+ /// A `_cat/templates` row is keyed by TEMPLATE name. It must not be
+ /// matched against index names — a template and an index may share one.
+ #[test]
+ fn cat_templates_rows_are_never_index_matched() {
+ let p = scoped(&["logs-2026"], &[Privilege::ReadIndex]);
+ let known: HashSet<String> = ["count"].iter().map(|s| s.to_string()).collect();
+ assert!(cat_index_columns(&segments("/_cat/templates")).is_none());
+ assert!(prune_sites(&segments("/_cat/templates")).is_empty());
+ let rows = "count logs-* 100 -\n";
+ assert_eq!(prune_rows("/_cat/templates", rows, &p, &known), rows);
+ }
+
+ /// FINDING C. Pinned, not fixed: `names: ["*"]` matches everything,
+ /// including `.xerj-memory-*`. Only a superuser can mint a scoped key, so
+ /// this is an operator's explicit "the whole node" rather than an
+ /// escalation — but it must not change without someone meaning it.
+ /// See `xerj_engine::rbac::Role::applies_to`.
+ #[test]
+ fn a_star_grant_reaches_the_reserved_namespace() {
+ let star = scoped(&["*"], &[Privilege::ReadIndex, Privilege::WriteIndex]);
+ assert!(star.allows_index(".xerj-memory-bob-edges", Privilege::ReadIndex));
+ assert!(check(
+ &star,
+ Method::GET,
+ "/.xerj-memory-bob-edges/_search",
+ ""
+ ));
+ assert!(check(&star, Method::GET, "/_memory/bob", ""));
+ // A prefix grant that stops short of the reserved namespace does not.
+ let logs = scoped(&["logs-*"], &[Privilege::ReadIndex]);
+ assert!(!logs.allows_index(".xerj-memory-bob-edges", Privilege::ReadIndex));
+ assert!(!check(
+ &logs,
+ Method::GET,
+ "/.xerj-memory-bob-edges/_search",
+ ""
+ ));
+ // …and `*` is still not a licence to squat the namespace with a
+ // template, which decides the mapping of brains created later.
+ assert!(!check(
+ &star,
+ Method::PUT,
+ "/_index_template/t",
+ r#"{"index_patterns":[".xerj-memory-*"]}"#
+ ));
+ }
+
+ /// FINDING A, config half. The ML config endpoints name an index the
+ /// server will then go and read on a schedule; before this they were
+ /// accepted with a 200 for a principal holding nothing on it.
+ #[test]
+ fn ml_config_authorizes_the_index_it_will_read() {
+ let legacy = unscoped();
+ let alice = scoped(&[".xerj-memory-alice-edges"], &[Privilege::ReadIndex]);
+
+ for (path, body) in [
+ (
+ "/_ml/anomaly_detectors/leak2",
+ r#"{"source_index":".xerj-memory-bob-edges","time_field":"created_at"}"#,
+ ),
+ (
+ "/_ml/anomaly_detectors/leak2",
+ r#"{"index":".xerj-memory-bob-edges"}"#,
+ ),
+ (
+ "/_ml/anomaly_detectors/leak2",
+ r#"{"source_index":"logs-2026","datafeed_config":{"indices":[".xerj-memory-bob-edges"]}}"#,
+ ),
+ (
+ "/_ml/datafeeds/leak2-feed",
+ r#"{"job_id":"leak2","indices":[".xerj-memory-bob-edges"]}"#,
+ ),
+ (
+ "/_ml/datafeeds/leak2-feed",
+ r#"{"job_id":"leak2","indexes":".xerj-memory-bob-edges"}"#,
+ ),
+ ] {
+ assert!(
+ !check(&legacy, Method::PUT, path, body),
+ "unscoped key configured ML over a brain: PUT {path} {body}"
+ );
+ assert!(
+ !check(&alice, Method::PUT, path, body),
+ "alice configured ML over bob's brain: PUT {path} {body}"
+ );
+ }
+
+ // Its own brain is fine, and so is an ordinary index for a key that
+ // holds the general surface.
+ assert!(check(
+ &alice,
+ Method::PUT,
+ "/_ml/anomaly_detectors/mine",
+ r#"{"source_index":".xerj-memory-alice-edges","time_field":"t"}"#
+ ));
+ assert!(check(
+ &legacy,
+ Method::PUT,
+ "/_ml/anomaly_detectors/ok",
+ r#"{"source_index":"logs-2026","time_field":"t"}"#
+ ));
+ // A datafeed that names no index inherits the (already authorized)
+ // detector's source, so a scoped key is not refused for it.
+ assert!(check(
+ &alice,
+ Method::PUT,
+ "/_ml/datafeeds/mine-feed",
+ r#"{"job_id":"mine"}"#
+ ));
+ // The sub-verbs carry no index of their own, so they are decided by
+ // the pre-existing cluster rule and this change does not move them: a
+ // key that holds the general surface may start a datafeed, a scoped
+ // one may not (a cluster-level mutation naming nothing). Pinned so the
+ // MlConfig arm cannot be read as having widened either one.
+ assert_eq!(
+ body_shape(&Method::POST, &segments("/_ml/datafeeds/f/_start")),
+ BodyShape::None
+ );
+ assert!(check(
+ &legacy,
+ Method::POST,
+ "/_ml/datafeeds/f/_start",
+ "{}"
+ ));
+ assert!(!check(
+ &alice,
+ Method::POST,
+ "/_ml/datafeeds/f/_start",
+ "{}"
+ ));
+ }
+
+ /// A template's patterns decide the mapping of indices that do not exist
+ /// yet, so a pattern reaching the reserved namespace is a write to a brain
+ /// by another route.
+ #[test]
+ fn index_template_patterns_cannot_reach_the_namespace() {
+ let legacy = unscoped();
+ assert!(check(
+ &legacy,
+ Method::PUT,
+ "/_index_template/logs",
+ r#"{"index_patterns":["logs-*"]}"#
+ ));
+ for patterns in [
+ r#"{"index_patterns":[".xerj-memory-*"]}"#,
+ r#"{"index_patterns":["*"]}"#,
+ r#"{"index_patterns":[".xerj-memory-bob-edges"]}"#,
+ r#"{"index_patterns":"logs-*,.xerj-*"}"#,
+ ] {
+ assert!(
+ !check(&legacy, Method::PUT, "/_index_template/grab", patterns),
+ "{patterns} must be refused"
+ );
+ assert!(
+ !check(&legacy, Method::PUT, "/_template/grab", patterns),
+ "legacy template {patterns} must be refused"
+ );
+ }
+ // The superuser still registers whatever it likes.
+ assert!(check(
+ &Principal::Superuser,
+ Method::PUT,
+ "/_index_template/grab",
+ r#"{"index_patterns":["*"]}"#
+ ));
+ }
+
+ /// The engine-side backstop is derived from the same principal, so a name
+ /// the middleware never parsed is still refused where it is resolved.
+ #[test]
+ fn visibility_matches_the_principal() {
+ let alice = scoped(&[".xerj-memory-alice-edges"], &[Privilege::ReadIndex]);
+ let v = visibility_for(&alice);
+ assert!(v.visible(".xerj-memory-alice-edges"));
+ assert!(!v.visible(".xerj-memory-bob-edges"));
+ assert!(!v.visible("logs-2026"));
+
+ let legacy = visibility_for(&unscoped());
+ assert!(legacy.visible("logs-2026"));
+ assert!(!legacy.visible(".xerj-memory-bob-edges"));
+
+ let root = visibility_for(&Principal::Superuser);
+ assert!(root.visible(".xerj-memory-bob-edges"));
+
+ let denied = visibility_for(&Principal::Denied);
+ assert!(!denied.visible("logs-2026"));
+ }
+}
diff --git a/engine/crates/xerj-api/src/es_compat.rs b/engine/crates/xerj-api/src/es_compat.rs
index 3908b413a..ac4df7b55 100644
--- a/engine/crates/xerj-api/src/es_compat.rs
+++ b/engine/crates/xerj-api/src/es_compat.rs
@@ -8567,6 +8567,14 @@ async fn search_impl(
// (for example when the client disconnects) aborts the child task;
// awaiting its JoinHandle also converts executor panics into HTTP 500
// instead of unwinding through axum.
+ //
+ // No `index_guard` rule is carried across this spawn, and does not need
+ // to be: the task closes over `idx`, an `Index` handle the guarded
+ // request already resolved and was authorized for, and an `Index` holds
+ // no `Engine`, so nothing inside can name a second index. The two ways
+ // a search body *does* name another index — a `terms` lookup and a
+ // `lookup` runtime field — are both resolved earlier in this function,
+ // on the request's own task (`resolve_terms_lookups`), under the guard.
#[cfg(test)]
let inject_task_panic = *idx_name == "test-panic-search-task";
#[cfg(test)]
@@ -16403,16 +16411,76 @@ pub struct AliasActionsBody {
pub actions: Vec<AliasAction>,
}
+/// Is this alias-action `index` field a pattern rather than one concrete name?
+fn alias_target_is_pattern(spec: &str) -> bool {
+ spec.split(',')
+ .map(str::trim)
+ .any(|p| p == "_all" || p.contains('*'))
+}
+
pub async fn post_aliases(
State(state): State<AppState>,
Json(body): Json<AliasActionsBody>,
) -> impl IntoResponse {
+ // An `add` whose `index` is a wildcard has to be expanded before the alias
+ // can point anywhere. Adding the pattern TEXT as the alias target — which
+ // is what the raw `add_alias` below used to do for every action — invents
+ // an alias resolving to an index name that cannot exist, and answers
+ // `acknowledged: true` for a write that did not happen. It read as a
+ // successful alias creation to a caller that had just been told, by the
+ // very same response, that it had reached the reserved namespace:
+ // `POST /_aliases {"actions":[{"add":{"index":".xerj-memory-*", …}}]}`
+ // returned 200 and left a dangling alias behind.
+ //
+ // ES resolves the wildcard and 404s `index_not_found_exception` when it
+ // matches nothing; so do we. A concrete name is left exactly as it was —
+ // ES lets an alias be attached to a name before the index exists, and
+ // clients rely on that ordering.
+ // Resolve every action first and refuse the whole request if any `add`
+ // resolved to nothing, so a rejected batch leaves no half-applied aliases
+ // behind.
+ enum Resolved<'a> {
+ Add(&'a AliasActionParams, Vec<String>),
+ Remove(&'a AliasActionParams, Vec<String>),
+ }
+ let mut plan: Vec<Resolved> = Vec::with_capacity(body.actions.len());
for action in &body.actions {
if let Some(add) = &action.add {
- state.engine.add_alias(&add.alias, &add.index);
+ let targets = if alias_target_is_pattern(&add.index) {
+ let expanded = resolve_index_selector(&state, &add.index).await;
+ if expanded.is_empty() {
+ let e = xerj_common::XerjError::index_not_found(&add.index);
+ return ApiError::new(e).into_response();
+ }
+ expanded
+ } else {
+ vec![add.index.clone()]
+ };
+ plan.push(Resolved::Add(add, targets));
}
if let Some(remove) = &action.remove {
- state.engine.remove_alias(&remove.alias, &remove.index);
+ // A `remove` that matches nothing removes nothing — idempotent,
+ // and it leaves no junk behind, so it is not refused.
+ let targets = if alias_target_is_pattern(&remove.index) {
+ resolve_index_selector(&state, &remove.index).await
+ } else {
+ vec![remove.index.clone()]
+ };
+ plan.push(Resolved::Remove(remove, targets));
+ }
+ }
+ for step in &plan {
+ match step {
+ Resolved::Add(add, targets) => {
+ for target in targets {
+ state.engine.add_alias(&add.alias, target);
+ }
+ }
+ Resolved::Remove(remove, targets) => {
+ for target in targets {
+ state.engine.remove_alias(&remove.alias, target);
+ }
+ }
}
}
Json(json!({ "acknowledged": true })).into_response()
@@ -16600,11 +16668,19 @@ pub async fn put_alias(
};
if targets.is_empty() {
- attach(&index);
- } else {
- for idx in &targets {
- attach(idx);
- }
+ // `resolve_index_selector` only ever comes back empty for a wildcard
+ // that matched nothing — a concrete name always resolves to itself.
+ // Attaching the alias to the pattern text in that case (what this used
+ // to do) fabricates an alias pointing at an index name that cannot
+ // exist and reports `acknowledged: true` for a write that did not
+ // happen; `PUT /.xerj-memory-*/_alias/X` was the case that made it
+ // look like the reserved namespace had just been aliased. Refuse, the
+ // way ES does.
+ let e = xerj_common::XerjError::index_not_found(&index);
+ return ApiError::new(e).into_response();
+ }
+ for idx in &targets {
+ attach(idx);
}
Json(json!({ "acknowledged": true })).into_response()
}
@@ -20111,6 +20187,11 @@ pub async fn delete_by_query(
let task_key = handle.key().to_string();
let spawned_key = task_key.clone();
let tasks = state.tasks.clone();
+ // Detached, so no `index_guard` rule — and none is needed: the task
+ // owns `idx`, the single already-authorized `Index` handle this request
+ // resolved, and an `Index` cannot name another index (it holds no
+ // `Engine`). See `xerj_engine::index_guard` for the contract a spawn
+ // that *can* reach `Engine::get_index` has to follow instead.
tokio::spawn(async move {
let response = run_delete_by_query(&idx, &search_req).await;
tasks.complete(&spawned_key, response);
@@ -20246,6 +20327,8 @@ pub async fn update_by_query(
let task_key = handle.key().to_string();
let spawned_key = task_key.clone();
let tasks = state.tasks.clone();
+ // Same reasoning as the `_delete_by_query` spawn above: the task owns
+ // one already-authorized `Index` handle and can reach no other index.
tokio::spawn(async move {
let response = run_update_by_query(&idx, &search_req, script).await;
tasks.complete(&spawned_key, response);
@@ -25328,8 +25411,22 @@ pub async fn security_get_user_profile(
pub async fn security_create_api_key(
State(state): State<AppState>,
+ principal: crate::auth::Principal,
body: Option<Json<Value>>,
) -> impl IntoResponse {
+ // Minting is how privilege is handed out, so it is also the obvious way to
+ // escalate: without this gate a caller with no brain access could simply
+ // mint itself a key that grants one (issue #79). Only the superuser may
+ // create a *scoped* key; a scoped caller may not mint at all (an unscoped
+ // key it minted would out-reach it on the general surface); an unscoped
+ // caller may mint, but only more unscoped keys.
+ if matches!(principal, crate::auth::Principal::Scoped { .. }) {
+ return crate::authz::forbidden(
+ &principal,
+ "<api keys>",
+ xerj_engine::rbac::Privilege::SecurityAdmin,
+ );
+ }
let payload = body.map(|Json(v)| v);
let name = payload
.as_ref()
@@ -25349,21 +25446,43 @@ pub async fn security_create_api_key(
// ES returns base64("id:api_key") as the `encoded` credential.
let encoded = base64_encode(&format!("{key_id}:{api_key}"));
- // Honest surface (item 6): `role_descriptors` are accepted for ES/Kibana
- // wire-compatibility but NOT enforced — every key authenticates as the
- // single superuser. Warn loudly so an operator relying on scoped keys is
- // not silently over-privileged. (Full per-key RBAC is deferred.)
- if payload
+ // `role_descriptors` are now parsed into the key's index grants and ARE
+ // enforced for the reserved `.xerj-memory-*` namespace (issue #79) — see
+ // `authz`. Two honest caveats remain, both warned about below:
+ // * cluster privileges and FLS/DLS inside a descriptor are still
+ // ignored, and
+ // * a key minted with NO usable descriptor keeps its historical reach
+ // over ordinary indices (it just holds nothing in the reserved
+ // namespace), because broad RBAC over the general ES surface is still
+ // deferred.
+ let requested_descriptors = payload
.as_ref()
.and_then(|b| b.get("role_descriptors"))
- .is_some_and(|v| !v.is_null() && v != &json!({}))
- {
- tracing::warn!(
- key = %name,
- "POST /_security/api_key: role_descriptors are NOT enforced; the \
- minted key has full superuser access. See rbac docs."
- );
- }
+ .filter(|v| !v.is_null() && *v != &json!({}));
+ let roles = match (&principal, requested_descriptors) {
+ (crate::auth::Principal::Superuser, Some(descriptors)) => {
+ let parsed = xerj_engine::rbac::roles_from_role_descriptors(descriptors);
+ if parsed.is_empty() {
+ tracing::warn!(
+ key = %name,
+ "POST /_security/api_key: role_descriptors granted nothing usable \
+ (only index names + index privileges are enforced); the key is \
+ unscoped and holds NO access to the reserved .xerj-memory-* namespace"
+ );
+ }
+ parsed
+ }
+ (_, Some(_)) => {
+ // A non-superuser cannot hand out grants it does not itself hold.
+ tracing::warn!(
+ key = %name,
+ "POST /_security/api_key: role_descriptors ignored — only the admin key \
+ may mint a scoped key; the minted key is unscoped"
+ );
+ Vec::new()
+ }
+ (_, None) => Vec::new(),
+ };
// Persist the key so the auth middleware can re-authenticate an inbound
// `Authorization: ApiKey <encoded>` header. `encoded` decodes to
@@ -25379,6 +25498,7 @@ pub async fn security_create_api_key(
creation_ms: now_ms,
expiration_ms,
invalidated: false,
+ roles,
},
);
@@ -29731,43 +29851,70 @@ async fn datafeed_score_pass(
/// new anomaly records. Breaks cleanly when the datafeed is stopped/deleted or
/// its job disappears. The first `interval` tick fires immediately and is
/// consumed here (the synchronous start pass already covered "now").
+///
+/// ## Why this carries a visibility rule
+///
+/// The per-index boundary (issue #79) is enforced in part by a `tokio`
+/// task-local ([`xerj_engine::index_guard`]), and a task-local does not
+/// survive `tokio::spawn`. This was the one request path in the tree that
+/// detached, and it was a live cross-brain read: `_start` under a principal
+/// with no access to `.xerj-memory-bob-edges` returned an empty result set
+/// from the synchronous pass — correctly denied, because that pass runs inside
+/// the request's scope — and then this task ticked a couple of seconds later
+/// with no guard at all and appended the brain's field values to the job's
+/// results, where the same principal read them straight back out of
+/// `GET /_ml/anomaly_detectors/{job}/results/records`.
+///
+/// So the rule the *starting* request is running under is captured on that
+/// request's task and re-installed here. The scorer therefore sees exactly
+/// what its starter could see: a superuser's datafeed keeps working unchanged
+/// (a superuser has no guard installed, so `None` is carried and the task
+/// stays unrestricted), and a scoped one silently scores nothing on an index
+/// it was not granted, because `Engine::get_index` answers "not found" and
+/// `datafeed_score_pass` treats that as "no new records".
fn spawn_datafeed_task(state: AppState, feed_id: String) -> tokio::task::JoinHandle<()> {
- tokio::spawn(async move {
- let freq = state
- .ml_datafeeds
- .get(&feed_id)
- .map(|f| f.frequency_secs)
- .unwrap_or(60)
- .max(1);
- let mut ticker = tokio::time::interval(std::time::Duration::from_secs(freq));
- ticker.tick().await; // consume the immediate first tick
- loop {
- ticker.tick().await;
- // Snapshot the datafeed; stop when gone or no longer started.
- let feed = match state.ml_datafeeds.get(&feed_id) {
- Some(f) if f.state == "started" => f.value().clone(),
- _ => break,
- };
- let detector = match state.ml_detectors.get(&feed.job_id) {
- Some(d) => d.value().clone(),
- None => break,
- };
- let (cursor, _appended) = datafeed_score_pass(
- &state.engine,
- &state.ml_results,
- &detector,
- &feed,
- feed.last_scored_ms,
- )
- .await;
- // Persist the cursor unless a concurrent _stop already flipped state.
- if let Some(mut f) = state.ml_datafeeds.get_mut(&feed_id) {
- if f.state == "started" {
- f.last_scored_ms = cursor;
+ // Captured HERE — on the request's own task, before the spawn. Reading it
+ // inside the spawned future would always find `None`.
+ let visibility = xerj_engine::index_guard::current();
+ tokio::spawn(xerj_engine::index_guard::scoped_opt(
+ visibility,
+ async move {
+ let freq = state
+ .ml_datafeeds
+ .get(&feed_id)
+ .map(|f| f.frequency_secs)
+ .unwrap_or(60)
+ .max(1);
+ let mut ticker = tokio::time::interval(std::time::Duration::from_secs(freq));
+ ticker.tick().await; // consume the immediate first tick
+ loop {
+ ticker.tick().await;
+ // Snapshot the datafeed; stop when gone or no longer started.
+ let feed = match state.ml_datafeeds.get(&feed_id) {
+ Some(f) if f.state == "started" => f.value().clone(),
+ _ => break,
+ };
+ let detector = match state.ml_detectors.get(&feed.job_id) {
+ Some(d) => d.value().clone(),
+ None => break,
+ };
+ let (cursor, _appended) = datafeed_score_pass(
+ &state.engine,
+ &state.ml_results,
+ &detector,
+ &feed,
+ feed.last_scored_ms,
+ )
+ .await;
+ // Persist the cursor unless a concurrent _stop already flipped state.
+ if let Some(mut f) = state.ml_datafeeds.get_mut(&feed_id) {
+ if f.state == "started" {
+ f.last_scored_ms = cursor;
+ }
}
}
- }
- })
+ },
+ ))
}
/// PUT /_ml/datafeeds/{feed_id} — create/replace a datafeed for a job.
diff --git a/engine/crates/xerj-api/src/graph_api.rs b/engine/crates/xerj-api/src/graph_api.rs
index 19b53f8bd..6fb8a1342 100644
--- a/engine/crates/xerj-api/src/graph_api.rs
+++ b/engine/crates/xerj-api/src/graph_api.rs
@@ -32,16 +32,52 @@
//! what was withheld (clipped edges/frontiers, bi-temporally excluded edges,
//! segments skipped, dangling node refs, agg tails).
//!
-//! ## Authorization model
+//! ## Authorization model — a brain IS a boundary (issue #79)
//!
-//! ⚠️ The `/_graph/*` endpoints are guarded ONLY by the process-wide API-key
-//! auth middleware (the admin key or any key minted via
-//! `POST /_security/api_key`). There is **no per-brain authorization**: any
-//! credential that can reach the node can read, write, and invalidate edges
-//! in **every** brain. Brains isolate data *by name*, NOT by credential — do
-//! not treat a brain as a security boundary between mutually-distrusting
-//! tenants. Per-brain access control depends on the deferred RBAC enforcement
-//! (see `xerj_engine::rbac`).
+//! Every endpoint below authorizes the caller for **this brain** before it
+//! does anything else, including before it checks whether the brain exists —
+//! so a 403 is returned identically for a brain that is not yours and a brain
+//! that does not exist, and the response code cannot be used to enumerate.
+//!
+//! The resource a brain authorizes against is its edges index,
+//! `.xerj-memory-{brain}-edges`. Who holds what:
+//!
+//! | Principal | Reach |
+//! |---|---|
+//! | open mode (`--insecure`, point-at-a-folder) | every brain — one user, no config, unchanged |
+//! | the configured admin key | every brain |
+//! | a key minted **with** `role_descriptors` naming the edges index | that brain, at the granted privilege |
+//! | a key minted **without** `role_descriptors` | **no** brain |
+//! | no/invalid credential | nothing |
+//!
+//! To grant brain `alice` to a key, name its indices at mint time:
+//!
+//! ```json
+//! POST /_security/api_key
+//! { "name": "alice-agent", "role_descriptors": { "alice": { "indices": [
+//! { "names": [".xerj-memory-alice-edges", ".xerj-memory-alice"],
+//! "privileges": ["read", "write"] } ] } } }
+//! ```
+//!
+//! `ego`/`overview` need `read`; `link`/`unlink` need `write` (creating the
+//! edges index lazily is part of writing a brain, not a separate `manage`
+//! step). The second name is the brain's *nodes* index — needed for `ego`'s
+//! node hydration and `overview`'s note count, and it is also the
+//! agent-memory namespace of the same name.
+//!
+//! ⚠️ An access check *here alone* would not have created a boundary. A
+//! brain's edges live in an ordinary index and `IndexName::validate` admits a
+//! leading `.`, so `.xerj-memory-{brain}-edges` is reachable through the
+//! generic ES-compat and native index routes: `_search` reads it around this
+//! module's `as_of`/`not_shown` semantics, `_doc/{id}` **forges** edges around
+//! the derived-`edge_id` invariant (§2.3) that only [`link`] enforces, `DELETE`
+//! destroys the brain, and `GET /_mapping` lists every brain name. All of
+//! those doors are closed in [`crate::authz`], which owns the rule and
+//! enumerates the enforcement points; the checks in this module are the
+//! in-handler half of the same decision, so these handlers stay safe even if
+//! mounted without that middleware.
+//!
+//! `tests/brain_is_a_security_boundary.rs` executes every door.
use axum::{
extract::{Path, Query, State},
@@ -53,6 +89,8 @@ use serde::Deserialize;
use serde_json::{json, Map, Value};
use std::collections::{HashMap, HashSet};
+use crate::auth::Principal;
+use crate::authz;
use crate::es_compat::{
self, EsSearchBody, EsSearchJson, EsSearchQueryParams, GetDocParams, IndexDocParams,
};
@@ -61,6 +99,7 @@ use crate::state::AppState;
use xerj_engine::graph::{
GraphDirection, GraphEdgeLite, GraphExpandRequest, GRAPH_HOPS_CAP_REASON,
};
+use xerj_engine::rbac::Privilege;
/// Contract version string, returned by every read endpoint.
pub const GRAPH_CONTRACT: &str = "xerj-second-brain/1";
@@ -83,16 +122,24 @@ const EGO_SEEDS_CAP: usize = 64;
/// Max dangling node ids listed verbatim in `not_shown.dangling_ids`.
const MAX_DANGLING_LISTED: usize = 50;
-/// Edges index name for a brain (SECOND_BRAIN_SPEC §1). The leading dot passes
-/// `IndexName::validate` and is unreachable through user index APIs.
+/// Edges index name for a brain (SECOND_BRAIN_SPEC §1).
+///
+/// The leading dot passes `IndexName::validate` and keeps the index out of the
+/// *conventional* listing surfaces, the same way `.kibana` is conventionally
+/// hidden. It does **not** by itself make the index unreachable — an earlier
+/// version of this comment claimed that, and it was wrong; `.`-prefixed names
+/// are accepted by the generic index routes. What makes it unreachable is that
+/// it sits in the reserved [`authz::RESERVED_INDEX_PREFIX`] namespace, which
+/// [`authz`] gates on every surface. This name is also the resource a
+/// `role_descriptors` grant must name to unlock the brain.
fn edges_index(brain: &str) -> String {
- format!(".xerj-memory-{brain}-edges")
+ authz::brain_edges_index(brain)
}
/// Default nodes index for a brain: the agent-memory namespace of the same
/// name. Autoindex brains override this via the §2.5 meta doc.
fn default_nodes_index(brain: &str) -> String {
- format!(".xerj-memory-{brain}")
+ authz::memory_namespace_index(brain)
}
/// Brain-name validation (SECOND_BRAIN_SPEC §1): identical rules to
@@ -359,11 +406,16 @@ pub struct LinkBody {
pub async fn link(
State(state): State<AppState>,
Path(brain): Path<String>,
+ principal: Principal,
body: OptionalJson<LinkBody>,
) -> Response {
if let Err(reason) = validate_brain(&brain) {
return error_response(StatusCode::BAD_REQUEST, reason);
}
+ // Before anything that could touch or create the brain (issue #79).
+ if let Err(denied) = authz::authorize_brain(&principal, &brain, Privilege::WriteIndex) {
+ return denied;
+ }
let Some(body) = body.0 else {
return error_response(
StatusCode::BAD_REQUEST,
@@ -515,11 +567,17 @@ pub struct UnlinkParams {
pub async fn unlink(
State(state): State<AppState>,
Path((brain, edge_id)): Path<(String, String)>,
+ principal: Principal,
Query(params): Query<UnlinkParams>,
) -> Response {
if let Err(reason) = validate_brain(&brain) {
return error_response(StatusCode::BAD_REQUEST, reason);
}
+ // Ordered before the existence probe below so an unauthorized caller
+ // cannot tell "not yours" from "not there" (issue #79).
+ if let Err(denied) = authz::authorize_brain(&principal, &brain, Privilege::WriteIndex) {
+ return denied;
+ }
let not_found = || {
error_response(
StatusCode::NOT_FOUND,
@@ -655,11 +713,23 @@ pub struct EgoParams {
pub async fn ego(
State(state): State<AppState>,
Path(brain): Path<String>,
+ principal: Principal,
Query(params): Query<EgoParams>,
) -> Response {
if let Err(reason) = validate_brain(&brain) {
return error_response(StatusCode::BAD_REQUEST, reason);
}
+ if let Err(denied) = authz::authorize_brain(&principal, &brain, Privilege::ReadIndex) {
+ return denied;
+ }
+ // `nodes_index` lets the caller redirect hydration at an arbitrary index,
+ // so it is authorized separately — otherwise a grant on one brain would be
+ // a read primitive for any index on the node.
+ if let Some(nodes_index) = params.nodes_index.as_deref() {
+ if let Err(denied) = authz::authorize_index(&principal, nodes_index, Privilege::ReadIndex) {
+ return denied;
+ }
+ }
if params.node.is_some() && params.nodes.is_some() {
return error_response(
StatusCode::BAD_REQUEST,
@@ -828,6 +898,15 @@ pub async fn ego(
Some(ni) if !ni.is_empty() => ni.to_string(),
_ => resolve_nodes_index(&state, &brain, &index).await,
};
+ // The meta doc's `nodes_index` is caller-controlled data (anyone with
+ // write on the brain can point it anywhere), so hydration is
+ // authorized against the RESOLVED index, not just the brain. Without
+ // this, write on one brain would be a read primitive for every index
+ // on the node.
+ if let Err(denied) = authz::authorize_index(&principal, &nodes_index, Privilege::ReadIndex)
+ {
+ return denied;
+ }
let mut found: HashSet<String> = HashSet::new();
let search_body = EsSearchBody {
query: Some(json!({ "ids": { "values": result.reachable } })),
@@ -1088,11 +1167,17 @@ fn embedder_id(state: &AppState) -> String {
pub async fn overview(
State(state): State<AppState>,
Path(brain): Path<String>,
+ principal: Principal,
Query(params): Query<OverviewParams>,
) -> Response {
if let Err(reason) = validate_brain(&brain) {
return error_response(StatusCode::BAD_REQUEST, reason);
}
+ // Before the `exists: false` probe below, so a caller cannot map the
+ // node's brains by watching 404 vs 403 (issue #79).
+ if let Err(denied) = authz::authorize_brain(&principal, &brain, Privilege::ReadIndex) {
+ return denied;
+ }
let as_of = match params.as_of.as_deref() {
None => now_ms(),
Some(s) => match parse_ms_str(s) {
@@ -1216,6 +1301,13 @@ pub async fn overview(
// nodes index was never created (edges asserted through the API alone)
// truthfully has 0 stored notes — reported as such, not a 404 and never
// fabricated from edge endpoints.
+ //
+ // The nodes index comes from the brain's meta doc, which is writable by
+ // anyone with write on the brain — so it is authorized in its own right
+ // (see the matching check in `ego`).
+ if let Err(denied) = authz::authorize_index(&principal, &nodes_index, Privilege::ReadIndex) {
+ return denied;
+ }
let nodes_total = if index_exists(&state, &nodes_index) {
let count_body = EsSearchBody {
query: Some(json!({ "match_all": {} })),
@@ -1293,6 +1385,7 @@ mod tests {
let resp = link(
State(state.clone()),
Path(brain.to_string()),
+ Principal::Superuser,
OptionalJson(Some(b)),
)
.await;
@@ -1308,6 +1401,7 @@ mod tests {
let resp = unlink(
State(state.clone()),
Path((brain.to_string(), edge_id.to_string())),
+ Principal::Superuser,
Query(UnlinkParams {
invalid_at: invalid_at.map(String::from),
}),
@@ -1317,7 +1411,13 @@ mod tests {
}
async fn do_ego(state: &AppState, brain: &str, params: EgoParams) -> (StatusCode, Value) {
- let resp = ego(State(state.clone()), Path(brain.to_string()), Query(params)).await;
+ let resp = ego(
+ State(state.clone()),
+ Path(brain.to_string()),
+ Principal::Superuser,
+ Query(params),
+ )
+ .await;
drain_json(resp).await
}
@@ -1690,6 +1790,7 @@ mod tests {
let resp = overview(
State(state.clone()),
Path("notes".to_string()),
+ Principal::Superuser,
Query(OverviewParams {
as_of: Some(as_of.to_string()),
..Default::default()
@@ -1778,6 +1879,7 @@ mod tests {
let resp = overview(
State(state.clone()),
Path("nope".to_string()),
+ Principal::Superuser,
Query(OverviewParams::default()),
)
.await;
@@ -1954,6 +2056,7 @@ mod tests {
let resp = overview(
State(state.clone()),
Path("notes".to_string()),
+ Principal::Superuser,
Query(OverviewParams {
as_of: Some(AS_OF.to_string()),
..Default::default()
diff --git a/engine/crates/xerj-api/src/lib.rs b/engine/crates/xerj-api/src/lib.rs
index fd50a0830..876ba358e 100644
--- a/engine/crates/xerj-api/src/lib.rs
+++ b/engine/crates/xerj-api/src/lib.rs
@@ -57,6 +57,7 @@
//! xerj-api itself does not depend on xerj-console-api.
pub mod auth;
+pub mod authz;
pub mod binary_protocol;
pub mod error;
pub mod es_compat;
diff --git a/engine/crates/xerj-api/src/memory_api.rs b/engine/crates/xerj-api/src/memory_api.rs
index 828519271..2c757b14c 100644
--- a/engine/crates/xerj-api/src/memory_api.rs
+++ b/engine/crates/xerj-api/src/memory_api.rs
@@ -58,18 +58,21 @@ use serde::Deserialize;
use serde_json::{json, Value};
use uuid::Uuid;
+use crate::auth::Principal;
+use crate::authz;
use crate::es_compat::{
self, DeleteDocParams, DeleteIndexParams, EsSearchBody, EsSearchJson, EsSearchQueryParams,
IndexDocParams,
};
use crate::extract::OptionalJson;
use crate::state::AppState;
+use xerj_engine::rbac::Privilege;
/// Reserved backing-index prefix. Real (user) indices can never collide: the
/// ES-compat `create_index` handler rejects any index name that starts with
/// `_`, `-`, or `+`, and a caller cannot address a `.`-leading system index
/// through `/_memory/*`.
-const MEMORY_PREFIX: &str = ".xerj-memory-";
+const MEMORY_PREFIX: &str = crate::authz::RESERVED_INDEX_PREFIX;
/// Default page size for the introspection (`GET`) endpoint when `size` is
/// omitted. Callers page further with `from`/`size` (see [`ListParams`]).
@@ -306,11 +309,17 @@ async fn dedup_probe(
pub async fn store(
State(state): State<AppState>,
Path(namespace): Path<String>,
+ principal: Principal,
body: OptionalJson<StoreBody>,
) -> Response {
if let Err(reason) = validate_namespace(&namespace) {
return error_response(StatusCode::BAD_REQUEST, reason);
}
+ if let Err(denied) =
+ authz::authorize_memory_namespace(&principal, &namespace, Privilege::WriteIndex)
+ {
+ return denied;
+ }
let body = match body.0 {
Some(b) => b,
None => return error_response(StatusCode::BAD_REQUEST, "missing request body"),
@@ -472,11 +481,17 @@ const MAX_RESTRICT_IDS: usize = 10_000;
pub async fn recall(
State(state): State<AppState>,
Path(namespace): Path<String>,
+ principal: Principal,
body: OptionalJson<RecallBody>,
) -> Response {
if let Err(reason) = validate_namespace(&namespace) {
return error_response(StatusCode::BAD_REQUEST, reason);
}
+ if let Err(denied) =
+ authz::authorize_memory_namespace(&principal, &namespace, Privilege::ReadIndex)
+ {
+ return denied;
+ }
// Recall must say what to recall BY: exactly one of `vector` (a query
// embedding) or `query` (text). A missing body, both-at-once, or an
// empty `query` are hard 400s — the old lenient fallback degraded every
@@ -544,6 +559,17 @@ pub async fn recall(
};
let index = backing_index(&namespace);
let edges_index_name = format!("{MEMORY_PREFIX}{namespace}-edges");
+ // Graph coupling reads the brain's EDGES, a different resource from the
+ // namespace itself — a credential granted only the notes must not read the
+ // link graph through `graph:` (issue #79). Authorized only when coupling is
+ // actually requested, so ungrouped recall is unaffected.
+ if graph_ctx.is_some() {
+ if let Err(denied) =
+ authz::authorize_index(&principal, &edges_index_name, Privilege::ReadIndex)
+ {
+ return denied;
+ }
+ }
let caller_filter = body.filter.take();
// Graph restrict: expand the seeds FIRST and fold the reachable set into
@@ -1112,11 +1138,17 @@ pub struct ListParams {
pub async fn list(
State(state): State<AppState>,
Path(namespace): Path<String>,
+ principal: Principal,
axum::extract::Query(params): axum::extract::Query<ListParams>,
) -> Response {
if let Err(reason) = validate_namespace(&namespace) {
return error_response(StatusCode::BAD_REQUEST, reason);
}
+ if let Err(denied) =
+ authz::authorize_memory_namespace(&principal, &namespace, Privilege::ReadIndex)
+ {
+ return denied;
+ }
// `from`/`after` are aliases (offset cursor); `size` is clamped to
// [1, MAX_LIST_SIZE] so a caller can neither page from a negative offset
// nor pull an unbounded set in one call.
@@ -1193,10 +1225,16 @@ pub async fn list(
pub async fn forget_one(
State(state): State<AppState>,
Path((namespace, id)): Path<(String, String)>,
+ principal: Principal,
) -> Response {
if let Err(reason) = validate_namespace(&namespace) {
return error_response(StatusCode::BAD_REQUEST, reason);
}
+ if let Err(denied) =
+ authz::authorize_memory_namespace(&principal, &namespace, Privilege::WriteIndex)
+ {
+ return denied;
+ }
let index = backing_index(&namespace);
if !index_exists(&state, &index) {
return error_response(
@@ -1231,10 +1269,16 @@ pub async fn forget_one(
pub async fn forget_namespace(
State(state): State<AppState>,
Path(namespace): Path<String>,
+ principal: Principal,
) -> Response {
if let Err(reason) = validate_namespace(&namespace) {
return error_response(StatusCode::BAD_REQUEST, reason);
}
+ if let Err(denied) =
+ authz::authorize_memory_namespace(&principal, &namespace, Privilege::AdminIndex)
+ {
+ return denied;
+ }
let index = backing_index(&namespace);
if !index_exists(&state, &index) {
return (
@@ -1286,6 +1330,7 @@ mod tests {
let resp = store(
State(state.clone()),
Path(ns.to_string()),
+ Principal::Superuser,
OptionalJson(Some(b)),
)
.await;
@@ -1297,6 +1342,7 @@ mod tests {
let resp = recall(
State(state.clone()),
Path(ns.to_string()),
+ Principal::Superuser,
OptionalJson(Some(b)),
)
.await;
@@ -1380,6 +1426,7 @@ mod tests {
let resp = forget_one(
State(state.clone()),
Path(("agent1".to_string(), "m1".to_string())),
+ Principal::Superuser,
)
.await;
let (s, _) = drain_json(resp).await;
@@ -1473,6 +1520,7 @@ mod tests {
let resp = list(
State(state.clone()),
Path(ns.to_string()),
+ Principal::Superuser,
axum::extract::Query(ListParams::default()),
)
.await;
@@ -1484,6 +1532,7 @@ mod tests {
let resp = list(
State(state.clone()),
Path(ns.to_string()),
+ Principal::Superuser,
axum::extract::Query(params),
)
.await;
@@ -1657,6 +1706,7 @@ mod tests {
let resp = crate::graph_api::link(
State(state.clone()),
Path("notes".to_string()),
+ Principal::Superuser,
OptionalJson(Some(b)),
)
.await;
@@ -1870,6 +1920,7 @@ mod tests {
let resp = crate::graph_api::link(
State(state.clone()),
Path("ghost".to_string()),
+ Principal::Superuser,
OptionalJson(Some(b)),
)
.await;
diff --git a/engine/crates/xerj-api/src/router.rs b/engine/crates/xerj-api/src/router.rs
index d33cb3cd3..68f523f81 100644
--- a/engine/crates/xerj-api/src/router.rs
+++ b/engine/crates/xerj-api/src/router.rs
@@ -27,7 +27,9 @@ use tracing::Level;
use uuid::Uuid;
use xerj_common::config::CorsConfig;
-use crate::{auth::auth_middleware, es_compat, graph_api, memory_api, native, state::AppState};
+use crate::{
+ auth::auth_middleware, authz, es_compat, graph_api, memory_api, native, state::AppState,
+};
// ─────────────────────────────────────────────────────────────────────────────
// Native xerj router (:8080)
@@ -150,6 +152,15 @@ pub fn build_native_router(state: AppState) -> Router {
// Shared state
.with_state(state.clone())
// Middleware stack (applied outermost-last)
+ //
+ // The native router reaches the same indices under a different
+ // spelling (`/v1/indices/{name}/…`), so it carries the same
+ // authorization layer — otherwise the reserved `.xerj-memory-*`
+ // namespace would be closed on :9200 and open on :8080.
+ .layer(middleware::from_fn_with_state(
+ state.clone(),
+ authz::authz_middleware,
+ ))
.layer(middleware::from_fn_with_state(state, auth_middleware))
.layer(middleware::from_fn(request_id_middleware))
.layer(trace_layer(access_log))
@@ -773,6 +784,18 @@ pub fn build_es_compat_router(state: AppState) -> Router {
// Shared state
.with_state(state.clone())
// Middleware stack (applied outermost-last)
+ //
+ // Authorization sits INSIDE authentication — added first here, so it
+ // runs second: `auth_middleware` decides whether the caller is anyone
+ // at all, then `authz_middleware` decides what that caller may reach.
+ // It is the enforcement point that makes a brain a real boundary
+ // rather than a naming convention (issue #79); see `authz.rs` for the
+ // full table of doors it closes. A superuser — `--insecure`, or the
+ // configured admin key — short-circuits it on the first line.
+ .layer(middleware::from_fn_with_state(
+ state.clone(),
+ authz::authz_middleware,
+ ))
.layer(middleware::from_fn_with_state(
state.clone(),
auth_middleware,
diff --git a/engine/crates/xerj-api/tests/body_named_index_bypasses.rs b/engine/crates/xerj-api/tests/body_named_index_bypasses.rs
new file mode 100644
index 000000000..c4107b826
--- /dev/null
+++ b/engine/crates/xerj-api/tests/body_named_index_bypasses.rs
@@ -0,0 +1,784 @@
+//! The four proven bypasses of per-brain authorization, fired at a **real
+//! running node** over a real socket — the way they were found.
+//!
+//! Every case below was measured against a live server on the first cut of
+//! issue #79's fix and came back `200 OK` with another tenant's data. They are
+//! not hypotheses, so they are pinned here as regressions rather than argued
+//! about. The common cause was one sentence long: authorization was decided
+//! against the index named in the URL **path**, while these handlers take the
+//! index they actually operate on from the request **body**, where the path
+//! index is only a default the body overrides.
+//!
+//! | # | Verb | How the body overrode the path |
+//! |---|---|---|
+//! | 1 | `_msearch` | an NDJSON header line's `index` |
+//! | 2 | `_bulk` | an action line's `_index` (forge **and** delete) |
+//! | 3 | `_mget` | `docs[]._index` |
+//! | 4 | `_aliases` | an alias pointed at the victim, then read through |
+//!
+//! This runs over a TCP listener rather than `tower::ServiceExt::oneshot`
+//! deliberately. `oneshot` calls the router directly; the bypasses live in the
+//! interaction between middleware, body buffering and handlers, and a body
+//! that has to survive being read once for authorization and again by the
+//! handler only behaves the same over a real connection. The client below is
+//! deliberately dumb (write bytes, read bytes) so nothing in it can smooth over
+//! a difference.
+
+use std::net::SocketAddr;
+
+use tokio::io::{AsyncReadExt, AsyncWriteExt};
+use tokio::net::TcpStream;
+use xerj_api::{
+ router::{build_es_compat_router, build_native_router},
+ state::AppState,
+};
+use xerj_common::{config::Config, metrics::Metrics};
+use xerj_engine::Engine;
+
+const ADMIN_KEY: &str = "admin-secret-key-for-bypass-test";
+const T0: i64 = 1_753_600_000_000;
+
+/// A live node — both listeners, one engine — plus the data dir it must
+/// outlive. The native router is served too because it reaches the same
+/// indices under a different spelling, so a boundary that holds on `:9200` and
+/// falls open on `:8080` is not a boundary.
+struct Node {
+ addr: SocketAddr,
+ native_addr: SocketAddr,
+ _data_dir: tempfile::TempDir,
+}
+
+/// Bind an ephemeral port and serve `app` on it.
+async fn serve(app: axum::Router) -> SocketAddr {
+ let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
+ .await
+ .expect("bind");
+ let addr = listener.local_addr().expect("local addr");
+ tokio::spawn(async move {
+ let _ = axum::serve(listener, app).await;
+ });
+ addr
+}
+
+async fn start_node() -> Node {
+ let dir = tempfile::tempdir().expect("tempdir");
+ let mut config = Config::default();
+ config.server.data_dir = dir.path().to_str().unwrap().to_string();
+ config.auth.enabled = true;
+ config.auth.admin_api_key = ADMIN_KEY.to_string();
+ let metrics = Metrics::new().expect("metrics");
+ let engine = Engine::new(config.clone()).expect("engine");
+ let state = AppState::new(config, engine, metrics);
+ Node {
+ addr: serve(build_es_compat_router(state.clone())).await,
+ native_addr: serve(build_native_router(state)).await,
+ _data_dir: dir,
+ }
+}
+
+/// One HTTP/1.1 request over a fresh connection. Returns (status, body).
+async fn http(node: &Node, method: &str, path: &str, auth: &str, body: &str) -> (u16, String) {
+ request(node.addr, method, path, auth, body).await
+}
+
+/// The same, against the native `:8080`-shaped listener.
+async fn http_native(
+ node: &Node,
+ method: &str,
+ path: &str,
+ auth: &str,
+ body: &str,
+) -> (u16, String) {
+ request(node.native_addr, method, path, auth, body).await
+}
+
+async fn request(
+ addr: SocketAddr,
+ method: &str,
+ path: &str,
+ auth: &str,
+ body: &str,
+) -> (u16, String) {
+ let mut stream = TcpStream::connect(addr).await.expect("connect");
+ let mut request = format!(
+ "{method} {path} HTTP/1.1\r\nHost: 127.0.0.1\r\nConnection: close\r\n\
+ Content-Type: application/json\r\nContent-Length: {}\r\n",
+ body.len()
+ );
+ if !auth.is_empty() {
+ request.push_str(&format!("Authorization: {auth}\r\n"));
+ }
+ request.push_str("\r\n");
+ request.push_str(body);
+ stream
+ .write_all(request.as_bytes())
+ .await
+ .expect("write request");
+ stream.flush().await.expect("flush");
+ let mut raw = Vec::new();
+ stream.read_to_end(&mut raw).await.expect("read response");
+ let text = String::from_utf8_lossy(&raw).into_owned();
+ let status: u16 = text
+ .split_whitespace()
+ .nth(1)
+ .and_then(|s| s.parse().ok())
+ .unwrap_or_else(|| panic!("no status line in response: {text}"));
+ // `Connection: close` means the body runs to EOF; chunked framing markers
+ // are stripped so assertions can look at the payload.
+ let body = match text.split_once("\r\n\r\n") {
+ Some((_, rest)) => rest
+ .lines()
+ .filter(|l| !l.trim().is_empty() && u32::from_str_radix(l.trim(), 16).is_err())
+ .collect::<Vec<_>>()
+ .join(""),
+ None => String::new(),
+ };
+ (status, body)
+}
+
+fn admin() -> String {
+ format!("ApiKey {ADMIN_KEY}")
+}
+
+/// Mint a key and return its `Authorization` header value.
+async fn mint(node: &Node, body: &str) -> String {
+ let (status, resp) = http(node, "POST", "/_security/api_key", &admin(), body).await;
+ assert_eq!(status, 200, "minting a key: {resp}");
+ let v: serde_json::Value = serde_json::from_str(&resp).expect("key json");
+ format!("ApiKey {}", v["encoded"].as_str().expect("encoded"))
+}
+
+/// A key scoped to exactly one brain — the documented grant shape.
+fn brain_grant(brain: &str) -> String {
+ format!(
+ r#"{{"name":"{brain}-agent","role_descriptors":{{"{brain}":{{"indices":[
+ {{"names":[".xerj-memory-{brain}-edges",".xerj-memory-{brain}"],
+ "privileges":["read","write"]}}]}}}}}}"#
+ )
+}
+
+/// Two brains, seeded as the admin. Bob's edge carries the marker the
+/// verifier used, so a leak is unmistakable in an assertion message.
+async fn seed(node: &Node) {
+ for (brain, dst) in [("alice", "doc:2"), ("bob", "TOPSECRET:2")] {
+ let (status, resp) = http(
+ node,
+ "POST",
+ &format!("/_graph/{brain}/link"),
+ &admin(),
+ &format!(
+ r#"{{"src":"doc:1","type":"mentions","dst":"{dst}","valid_at":{T0},"created_at":{T0}}}"#
+ ),
+ )
+ .await;
+ assert_eq!(status, 201, "seeding {brain}: {resp}");
+ }
+ for index in [".xerj-memory-alice-edges", ".xerj-memory-bob-edges"] {
+ let (status, _) = http(node, "POST", &format!("/{index}/_refresh"), &admin(), "").await;
+ assert_eq!(status, 200);
+ }
+}
+
+/// The `_id` of bob's real edge, read as the admin — bypass 2's delete and
+/// bypass 3's read both need a genuine document id, not a guess.
+async fn bobs_edge_id(node: &Node) -> String {
+ let (status, resp) = http(
+ node,
+ "POST",
+ "/.xerj-memory-bob-edges/_search",
+ &admin(),
+ r#"{"query":{"match_all":{}},"size":50}"#,
+ )
+ .await;
+ assert_eq!(status, 200, "reading bob as admin: {resp}");
+ let v: serde_json::Value = serde_json::from_str(&resp).expect("search json");
+ v["hits"]["hits"]
+ .as_array()
+ .expect("hits")
+ .iter()
+ .find(|h| h["_source"].get("edge_id").is_some())
+ .map(|h| h["_id"].as_str().expect("_id").to_string())
+ .expect("bob must have a real edge")
+}
+
+/// Bob's brain, verbatim, as the admin sees it. The point of comparison for
+/// "the attack changed nothing".
+/// Bob's brain contents, as the admin sees them, for before/after comparison.
+///
+/// Only `hits` is returned, deliberately: the surrounding envelope carries
+/// `took`, a wall-clock millisecond count, so comparing whole response bodies
+/// makes the "nothing touched bob" assertion fail whenever the two reads
+/// happen to straddle a millisecond boundary. That is a flake, not a breach —
+/// it fired once here with byte-identical `hits` and `took` 0 vs 1.
+async fn bobs_edges(node: &Node) -> String {
+ let (status, resp) = http(
+ node,
+ "POST",
+ "/.xerj-memory-bob-edges/_search",
+ &admin(),
+ r#"{"query":{"match_all":{}},"size":50}"#,
+ )
+ .await;
+ assert_eq!(status, 200, "reading bob as admin: {resp}");
+ let parsed: serde_json::Value = serde_json::from_str(&resp).unwrap_or(serde_json::Value::Null);
+ assert!(
+ parsed.get("hits").is_some(),
+ "reading bob as admin returned no hits envelope: {resp}"
+ );
+ parsed["hits"].to_string()
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// BYPASS 1 — read, scoped key, via `_msearch`.
+///
+/// PROVED: a key granting only alice's two indices sent
+/// `POST /.xerj-memory-alice-edges/_msearch` with a header line naming
+/// `.xerj-memory-bob-edges`, and got 200 with bob's documents including
+/// `{"src":"doc:1","dst":"TOPSECRET:2","type":"mentions"}`.
+#[tokio::test]
+async fn msearch_header_cannot_override_the_authorized_index() {
+ let node = start_node().await;
+ seed(&node).await;
+ let alice = mint(&node, &brain_grant("alice")).await;
+
+ let attack = "{\"index\":\".xerj-memory-bob-edges\"}\n{\"query\":{\"match_all\":{}}}\n";
+ for path in [
+ "/.xerj-memory-alice-edges/_msearch",
+ "/_msearch",
+ "/.xerj-memory-alice-edges/_msearch/template",
+ ] {
+ let (status, resp) = http(&node, "POST", path, &alice, attack).await;
+ assert_eq!(status, 403, "POST {path} must be refused, got {status}");
+ assert!(
+ !resp.contains("TOPSECRET"),
+ "POST {path} leaked bob: {resp}"
+ );
+ }
+
+ // The same verb against her own brain still works — a fix that breaks
+ // _msearch is not a fix.
+ let own = "{\"index\":\".xerj-memory-alice-edges\"}\n{\"query\":{\"match_all\":{}}}\n";
+ let (status, resp) = http(&node, "POST", "/_msearch", &alice, own).await;
+ assert_eq!(status, 200, "alice's own _msearch: {resp}");
+ assert!(resp.contains("doc:2"), "alice's own edge missing: {resp}");
+}
+
+/// BYPASS 2 — write and destroy, scoped key, via `_bulk`.
+///
+/// PROVED: alice sent `POST /.xerj-memory-alice-edges/_bulk` with an action
+/// line naming `.xerj-memory-bob-edges` and got `"result":"created"`; an
+/// admin `GET` of that id returned `found:true`. A `delete` action against a
+/// real edge id then DELETED bob's genuine edge — the two doors
+/// (`authz.rs`'s own module header calls them "forge an edge around the
+/// derived-edge_id invariant" and "destroy the brain") it claimed to have shut.
+#[tokio::test]
+async fn bulk_action_line_cannot_override_the_authorized_index() {
+ let node = start_node().await;
+ seed(&node).await;
+ let alice = mint(&node, &brain_grant("alice")).await;
+ let victim_id = bobs_edge_id(&node).await;
+ let before = bobs_edges(&node).await;
+
+ // Forge.
+ let forge = "{\"index\":{\"_index\":\".xerj-memory-bob-edges\",\"_id\":\"forged-s\"}}\n\
+ {\"src\":\"doc:1\",\"dst\":\"attacker:payload\"}\n";
+ for path in ["/.xerj-memory-alice-edges/_bulk", "/_bulk"] {
+ let (status, resp) = http(&node, "POST", path, &alice, forge).await;
+ assert_eq!(status, 403, "POST {path} forge must be refused: {resp}");
+ }
+ let (status, resp) = http(
+ &node,
+ "GET",
+ "/.xerj-memory-bob-edges/_doc/forged-s",
+ &admin(),
+ "",
+ )
+ .await;
+ assert!(
+ status == 404 || resp.contains("\"found\":false"),
+ "the forged document reached bob's brain: {status} {resp}"
+ );
+
+ // Destroy.
+ let destroy = format!(
+ "{{\"delete\":{{\"_index\":\".xerj-memory-bob-edges\",\"_id\":\"{victim_id}\"}}}}\n"
+ );
+ for path in ["/.xerj-memory-alice-edges/_bulk", "/_bulk"] {
+ let (status, resp) = http(&node, "POST", path, &alice, &destroy).await;
+ assert_eq!(status, 403, "POST {path} delete must be refused: {resp}");
+ }
+ let after = bobs_edges(&node).await;
+ assert_eq!(before, after, "bob's brain was modified by the attack");
+ assert!(after.contains("TOPSECRET"), "bob's edge is gone: {after}");
+
+ // Alice's own bulk, by both spellings, still works.
+ let own = "{\"index\":{\"_index\":\".xerj-memory-alice-edges\",\"_id\":\"own-1\"}}\n\
+ {\"src\":\"a\",\"dst\":\"b\"}\n";
+ let (status, resp) = http(&node, "POST", "/_bulk", &alice, own).await;
+ assert_eq!(status, 200, "alice's own global _bulk: {resp}");
+ assert!(
+ !resp.contains("\"errors\":true"),
+ "own bulk errored: {resp}"
+ );
+ let scoped = "{\"index\":{\"_id\":\"own-2\"}}\n{\"src\":\"a\",\"dst\":\"c\"}\n";
+ let (status, resp) = http(
+ &node,
+ "POST",
+ "/.xerj-memory-alice-edges/_bulk",
+ &alice,
+ scoped,
+ )
+ .await;
+ assert_eq!(status, 200, "alice's own index-scoped _bulk: {resp}");
+ assert!(
+ !resp.contains("\"errors\":true"),
+ "own bulk errored: {resp}"
+ );
+}
+
+/// BYPASS 3 — read, scoped key, via `_mget`.
+///
+/// PROVED: `POST /.xerj-memory-alice-edges/_mget` with
+/// `{"docs":[{"_index":".xerj-memory-bob-edges","_id":"<real id>"}]}` returned
+/// 200 with bob's edge source containing `"dst":"TOPSECRET:2"`.
+#[tokio::test]
+async fn mget_docs_cannot_override_the_authorized_index() {
+ let node = start_node().await;
+ seed(&node).await;
+ let alice = mint(&node, &brain_grant("alice")).await;
+ let victim_id = bobs_edge_id(&node).await;
+
+ let attack =
+ format!(r#"{{"docs":[{{"_index":".xerj-memory-bob-edges","_id":"{victim_id}"}}]}}"#);
+ for path in ["/.xerj-memory-alice-edges/_mget", "/_mget"] {
+ let (status, resp) = http(&node, "POST", path, &alice, &attack).await;
+ assert_eq!(status, 403, "POST {path} must be refused, got {status}");
+ assert!(
+ !resp.contains("TOPSECRET"),
+ "POST {path} leaked bob: {resp}"
+ );
+ }
+
+ // The short form against her own index is untouched.
+ let (status, resp) = http(
+ &node,
+ "POST",
+ "/.xerj-memory-alice-edges/_mget",
+ &alice,
+ r#"{"ids":["__xerj-brain-meta"]}"#,
+ )
+ .await;
+ assert_eq!(status, 200, "alice's own _mget: {resp}");
+}
+
+/// BYPASS 4 — read, unscoped key, via aliases.
+///
+/// PROVED: a key with no `role_descriptors` sent `POST /_aliases` adding an
+/// alias onto `.xerj-memory-bob-edges` (200 acknowledged), then searched the
+/// alias and got bob's edges. `PUT /.xerj-memory-bob-edges/_alias/pwned` was
+/// already refused, so this was precisely the "one missed shape" the module
+/// documentation warned about. The second half matters as much: an alias an
+/// operator had already pointed into the namespace made it readable by every
+/// unscoped key.
+#[tokio::test]
+async fn aliases_cannot_launder_a_reserved_index() {
+ let node = start_node().await;
+ seed(&node).await;
+ let unscoped = mint(&node, r#"{"name":"plain"}"#).await;
+
+ // Creating the alias is refused, by both spellings.
+ let add = r#"{"actions":[{"add":{"index":".xerj-memory-bob-edges","alias":"pwned-u"}}]}"#;
+ let (status, resp) = http(&node, "POST", "/_aliases", &unscoped, add).await;
+ assert_eq!(status, 403, "POST /_aliases must be refused: {resp}");
+ let (status, resp) = http(
+ &node,
+ "PUT",
+ "/.xerj-memory-bob-edges/_alias/pwned-p",
+ &unscoped,
+ "",
+ )
+ .await;
+ assert_eq!(status, 403, "PUT .../_alias must be refused: {resp}");
+ let (status, resp) = http(&node, "POST", "/pwned-u/_search", &unscoped, "{}").await;
+ assert!(
+ !resp.contains("TOPSECRET"),
+ "the alias was created after all: {status} {resp}"
+ );
+
+ // Second half: an alias the OPERATOR already pointed into the namespace
+ // must not become a back door either. Only the admin can make one.
+ let operator_alias =
+ r#"{"actions":[{"add":{"index":".xerj-memory-bob-edges","alias":"ops-view"}}]}"#;
+ let (status, resp) = http(&node, "POST", "/_aliases", &admin(), operator_alias).await;
+ assert_eq!(status, 200, "admin must still manage aliases: {resp}");
+
+ for path in ["/ops-view/_search", "/ops-view/_count"] {
+ let (status, resp) = http(&node, "POST", path, &unscoped, "{}").await;
+ assert_eq!(
+ status, 403,
+ "reading through an operator alias must be refused: {resp}"
+ );
+ assert!(!resp.contains("TOPSECRET"), "alias leaked bob: {resp}");
+ }
+ // …including through a body-named door.
+ let (status, resp) = http(
+ &node,
+ "POST",
+ "/_msearch",
+ &unscoped,
+ "{\"index\":\"ops-view\"}\n{\"query\":{\"match_all\":{}}}\n",
+ )
+ .await;
+ assert_eq!(status, 403, "aliased _msearch must be refused: {resp}");
+ assert!(
+ !resp.contains("TOPSECRET"),
+ "aliased _msearch leaked: {resp}"
+ );
+
+ // An alias over an ordinary index is ordinary work and still allowed.
+ let (status, resp) = http(
+ &node,
+ "PUT",
+ "/logs-2026",
+ &unscoped,
+ r#"{"mappings":{"properties":{"m":{"type":"integer"}}}}"#,
+ )
+ .await;
+ assert_eq!(status, 200, "creating an ordinary index: {resp}");
+ let ordinary = r#"{"actions":[{"add":{"index":"logs-2026","alias":"logs"}}]}"#;
+ let (status, resp) = http(&node, "POST", "/_aliases", &unscoped, ordinary).await;
+ assert_eq!(status, 200, "an ordinary alias must still work: {resp}");
+ // But an alias NAME inside the reserved namespace is squatting.
+ let squat = r#"{"actions":[{"add":{"index":"logs-2026","alias":".xerj-memory-victim"}}]}"#;
+ let (status, resp) = http(&node, "POST", "/_aliases", &unscoped, squat).await;
+ assert_eq!(status, 403, "squatting a reserved alias name: {resp}");
+}
+
+/// The fifth shape, found by auditing the tree rather than by being attacked:
+/// a `terms` lookup names an index to read terms out of, at arbitrary depth
+/// inside a query body that the URL says is about something else.
+#[tokio::test]
+async fn terms_lookup_cannot_read_an_unauthorized_index() {
+ let node = start_node().await;
+ seed(&node).await;
+ let alice = mint(&node, &brain_grant("alice")).await;
+ let victim_id = bobs_edge_id(&node).await;
+
+ let attack = format!(
+ r#"{{"query":{{"bool":{{"filter":[{{"terms":{{"src":{{"index":".xerj-memory-bob-edges","id":"{victim_id}","path":"dst"}}}}}}]}}}}}}"#
+ );
+ let (status, resp) = http(
+ &node,
+ "POST",
+ "/.xerj-memory-alice-edges/_search",
+ &alice,
+ &attack,
+ )
+ .await;
+ assert_eq!(status, 403, "a terms lookup into bob must be refused");
+ assert!(!resp.contains("TOPSECRET"), "terms lookup leaked: {resp}");
+
+ // The same shape one layer over: a `lookup` runtime field joins rows out
+ // of another index into the hits.
+ let runtime = r#"{"runtime_mappings":{"u":{"type":"lookup","target_index":".xerj-memory-bob-edges","input_field":"src","target_field":"_id","fetch_fields":["dst"]}},"fields":["u"]}"#;
+ let (status, resp) = http(
+ &node,
+ "POST",
+ "/.xerj-memory-alice-edges/_search",
+ &alice,
+ runtime,
+ )
+ .await;
+ assert_eq!(status, 403, "a lookup runtime field into bob: {resp}");
+ assert!(!resp.contains("TOPSECRET"), "runtime lookup leaked: {resp}");
+}
+
+/// The reason this fix is two layers and not a patch list.
+///
+/// `_sql` names its index in the FROM clause of a SQL string, an ingest
+/// pipeline names one in a `route` processor it stored earlier, an index
+/// template names a *pattern* that decides the mapping of brains created
+/// later. None of those is a JSON field the middleware parses — and none of
+/// them has to be, because the name still has to become an index through
+/// `Engine::get_index` / `get_or_create_index` / `create_index`, where the
+/// request's principal is installed as a visibility rule. This is the property
+/// that a per-handler patch list cannot have.
+#[tokio::test]
+async fn shapes_the_middleware_never_parses_still_fail_closed() {
+ let node = start_node().await;
+ seed(&node).await;
+ let alice = mint(&node, &brain_grant("alice")).await;
+ let unscoped = mint(&node, r#"{"name":"plain"}"#).await;
+ let before = bobs_edges(&node).await;
+
+ // `_sql` — the table name is inside the statement.
+ let (status, resp) = http(
+ &node,
+ "POST",
+ "/_sql",
+ &alice,
+ r#"{"query":"SELECT * FROM \".xerj-memory-bob-edges\""}"#,
+ )
+ .await;
+ assert_ne!(status, 200, "_sql reached bob's brain: {resp}");
+ assert!(!resp.contains("TOPSECRET"), "_sql leaked bob: {resp}");
+
+ // An index template that would own the mapping of every brain created
+ // from here on.
+ let (status, resp) = http(
+ &node,
+ "PUT",
+ "/_index_template/grab",
+ &unscoped,
+ r#"{"index_patterns":[".xerj-memory-*"],"template":{"mappings":{"properties":{"dst":{"type":"keyword"}}}}}"#,
+ )
+ .await;
+ assert_eq!(status, 403, "a namespace-wide template: {resp}");
+
+ // `POST /v1/pipelines/{name}` can register a `route` processor whose
+ // target index is chosen at ingest time, not at request time.
+ let (status, resp) = http_native(
+ &node,
+ "PUT",
+ "/v1/pipelines/exfil",
+ &unscoped,
+ r#"{"steps":[{"route":{"index":".xerj-memory-bob-edges"}}]}"#,
+ )
+ .await;
+ // Whether the pipeline definition is accepted is the pipeline API's
+ // business; what matters is that ingesting through it cannot reach bob.
+ let _ = (status, resp);
+ let (status, resp) = http_native(
+ &node,
+ "POST",
+ "/v1/indices/logs-pipe/ingest?pipeline=exfil",
+ &unscoped,
+ r#"{"documents":[{"src":"a","dst":"b"}]}"#,
+ )
+ .await;
+ assert!(
+ !resp.contains("TOPSECRET"),
+ "pipeline ingest leaked bob: {status} {resp}"
+ );
+
+ // Nothing above touched bob's brain.
+ let after = bobs_edges(&node).await;
+ assert_eq!(before, after, "bob's brain changed: {before} vs {after}");
+}
+
+/// COMPATIBILITY — the regression the same branch introduced, also measured.
+///
+/// Every non-superuser credential, including a plain minted key, got 403 on
+/// `POST /_bulk`, `/_msearch`, `/_mget`, `/_search` and `/_count`. The global
+/// `_bulk` is the default write path for essentially every ES client, so that
+/// broke ordinary usage rather than securing it. A scoped key additionally got
+/// 403 on all cluster metadata and on every wildcard, including one it had
+/// been granted, which Kibana cannot survive.
+#[tokio::test]
+async fn ordinary_credentials_keep_the_global_surface() {
+ let node = start_node().await;
+ seed(&node).await;
+ let unscoped = mint(&node, r#"{"name":"plain"}"#).await;
+ let alice = mint(&node, &brain_grant("alice")).await;
+ let wide = mint(
+ &node,
+ r#"{"name":"wide","role_descriptors":{"w":{"indices":[
+ {"names":["logs-*"],"privileges":["all"]}]}}}"#,
+ )
+ .await;
+
+ let (status, resp) = http(
+ &node,
+ "PUT",
+ "/logs-2026",
+ &unscoped,
+ r#"{"mappings":{"properties":{"m":{"type":"integer"}}}}"#,
+ )
+ .await;
+ assert_eq!(status, 200, "creating an ordinary index: {resp}");
+
+ // (a) The five global verbs answer for an ordinary credential.
+ for (method, path, body) in [
+ ("POST", "/_search", "{}"),
+ ("POST", "/_count", "{}"),
+ ("POST", "/_msearch", "{}\n{\"query\":{\"match_all\":{}}}\n"),
+ (
+ "POST",
+ "/_mget",
+ r#"{"docs":[{"_index":"logs-2026","_id":"1"}]}"#,
+ ),
+ (
+ "POST",
+ "/_bulk",
+ "{\"index\":{\"_index\":\"logs-2026\",\"_id\":\"1\"}}\n{\"m\":1}\n",
+ ),
+ ] {
+ for (who, cred) in [("unscoped", &unscoped), ("wide", &wide)] {
+ let (status, resp) = http(&node, method, path, cred, body).await;
+ assert_eq!(
+ status, 200,
+ "{who} {method} {path} must work, got {status}: {resp}"
+ );
+ }
+ }
+
+ // (b) Cluster metadata answers for a SCOPED key, filtered rather than
+ // refused. Kibana polls all of these on every page load.
+ for path in [
+ "/_cluster/health",
+ "/_nodes",
+ "/_mapping",
+ "/_settings",
+ "/_alias",
+ "/_cat/indices",
+ "/_cat/health",
+ "/_cat/nodes",
+ "/_cat/aliases",
+ "/_cluster/state",
+ "/_cluster/stats",
+ "/_resolve/index/*",
+ "/_xpack",
+ ] {
+ let (status, resp) = http(&node, "GET", path, &alice, "").await;
+ assert_eq!(
+ status, 200,
+ "a scoped key must read {path}, got {status}: {resp}"
+ );
+ assert!(
+ !resp.contains("TOPSECRET") && !resp.contains("bob"),
+ "{path} leaked another tenant: {resp}"
+ );
+ }
+ // …and the native router's own metadata, which lives on the other listener.
+ for path in ["/v1/health", "/v1/cluster/health", "/v1/dashboard/summary"] {
+ let (status, resp) = http_native(&node, "GET", path, &alice, "").await;
+ assert_eq!(
+ status, 200,
+ "a scoped key must read native {path}, got {status}: {resp}"
+ );
+ assert!(!resp.contains("bob"), "native {path} leaked: {resp}");
+ }
+
+ // (c) A granted wildcard actually works, and the data it returns is the
+ // grant's, not the cluster's.
+ let (status, resp) = http(
+ &node,
+ "POST",
+ "/_bulk",
+ &wide,
+ "{\"index\":{\"_index\":\"logs-2026\",\"_id\":\"w1\"}}\n{\"m\":7}\n",
+ )
+ .await;
+ assert_eq!(status, 200, "granted wildcard write: {resp}");
+ let (status, _) = http(&node, "POST", "/logs-2026/_refresh", &admin(), "").await;
+ assert_eq!(status, 200);
+ let (status, resp) = http(&node, "POST", "/logs-*/_search", &wide, "{}").await;
+ assert_eq!(status, 200, "granted wildcard read: {resp}");
+ assert!(resp.contains("logs-2026"), "wildcard found nothing: {resp}");
+ assert!(!resp.contains("TOPSECRET"), "wildcard leaked bob: {resp}");
+
+ // (d) An UNgranted wildcard resolves to nothing instead of leaking what it
+ // would have matched — and instead of a 403 that would tell the caller the
+ // pattern was interesting.
+ for path in ["/*/_search", "/_all/_search", "/.xerj-memory-*/_search"] {
+ let (status, resp) = http(&node, "POST", path, &wide, "{}").await;
+ assert_eq!(status, 200, "POST {path}: {resp}");
+ assert!(!resp.contains("TOPSECRET"), "POST {path} leaked: {resp}");
+ }
+}
+
+/// The zero-configuration local path — `xerj --insecure`, point-at-a-folder —
+/// is one user with no credentials and must be completely unaffected.
+#[tokio::test]
+async fn insecure_single_user_mode_is_unchanged() {
+ let dir = tempfile::tempdir().expect("tempdir");
+ let mut config = Config::default();
+ config.server.data_dir = dir.path().to_str().unwrap().to_string();
+ config.auth.enabled = false;
+ config.auth.admin_api_key = String::new();
+ let metrics = Metrics::new().expect("metrics");
+ let engine = Engine::new(config.clone()).expect("engine");
+ let state = AppState::new(config, engine, metrics);
+ let node = Node {
+ addr: serve(build_es_compat_router(state.clone())).await,
+ native_addr: serve(build_native_router(state)).await,
+ _data_dir: dir,
+ };
+
+ let (status, resp) = http(
+ &node,
+ "POST",
+ "/_graph/solo/link",
+ "",
+ &format!(r#"{{"src":"a","type":"mentions","dst":"b","valid_at":{T0},"created_at":{T0}}}"#),
+ )
+ .await;
+ assert_eq!(status, 201, "no-credential link must work: {resp}");
+ for (method, path, body) in [
+ ("GET", "/_graph/solo/overview", ""),
+ ("POST", "/_search", "{}"),
+ (
+ "POST",
+ "/_bulk",
+ "{\"index\":{\"_index\":\"anything\"}}\n{\"a\":1}\n",
+ ),
+ ("GET", "/_cat/indices", ""),
+ ("GET", "/_mapping", ""),
+ ("POST", "/.xerj-memory-solo-edges/_search", "{}"),
+ (
+ "POST",
+ "/_aliases",
+ r#"{"actions":[{"add":{"index":".xerj-memory-solo-edges","alias":"solo"}}]}"#,
+ ),
+ ("POST", "/solo/_search", "{}"),
+ ] {
+ let (status, resp) = http(&node, method, path, "", body).await;
+ assert_eq!(
+ status, 200,
+ "{method} {path} must work with no credential: {resp}"
+ );
+ }
+ // The native router's body-named create, unrestricted for the single user.
+ let (status, resp) = http_native(
+ &node,
+ "POST",
+ "/v1/indices",
+ "",
+ r#"{"name":"solo-notes","fields":[]}"#,
+ )
+ .await;
+ assert_eq!(status, 201, "native create with no credential: {resp}");
+}
+
+/// The native router's `POST /v1/indices` names its index in the body. The
+/// first cut refused it outright for every non-superuser, which broke the
+/// native create path; it is authorized against the name it carries instead.
+#[tokio::test]
+async fn native_body_named_create_is_authorized_not_refused() {
+ let node = start_node().await;
+ let unscoped = mint(&node, r#"{"name":"plain"}"#).await;
+
+ let (status, resp) = http_native(
+ &node,
+ "POST",
+ "/v1/indices",
+ &unscoped,
+ r#"{"name":"logs-native","fields":[]}"#,
+ )
+ .await;
+ assert_eq!(status, 201, "an ordinary native create must work: {resp}");
+
+ // Squatting a brain name the caller could then not read is refused.
+ let (status, resp) = http_native(
+ &node,
+ "POST",
+ "/v1/indices",
+ &unscoped,
+ r#"{"name":".xerj-memory-victim-edges","fields":[]}"#,
+ )
+ .await;
+ assert_eq!(status, 403, "squatting the namespace: {resp}");
+ let (status, resp) = http(&node, "GET", "/.xerj-memory-victim-edges", &admin(), "").await;
+ assert_eq!(status, 404, "the squatted index was created: {resp}");
+}
diff --git a/engine/crates/xerj-api/tests/brain_is_a_security_boundary.rs b/engine/crates/xerj-api/tests/brain_is_a_security_boundary.rs
new file mode 100644
index 000000000..e186eb1f1
--- /dev/null
+++ b/engine/crates/xerj-api/tests/brain_is_a_security_boundary.rs
@@ -0,0 +1,692 @@
+//! Executable statement of the second-brain authorization posture (issue #79).
+//!
+//! A brain **is** a security boundary. This file is the inverted descendant of
+//! `brain_is_not_a_security_boundary.rs`, which pinned the measured fact that
+//! one did *not* exist: any authenticated caller could read, write, forge and
+//! destroy every brain, and `GET /_mapping` handed over the list of brain
+//! names so the attacker did not even have to guess one. Every assertion that
+//! file made now runs in reverse.
+//!
+//! The point of enumerating so many doors is that the exposure was never about
+//! the four `/_graph/*` handlers. A brain's edges live in an ordinary index and
+//! `IndexName::validate` admits a leading `.`, so `.xerj-memory-{brain}-edges`
+//! was reachable through the generic ES-compat surface *and* through the
+//! native router's `/v1/indices/{name}/…` spelling. An access check on
+//! `/_graph/*` alone would have left all of that open — a boundary that only
+//! looks like one. So this test walks every door, and a fix that closes only
+//! some of them fails here.
+//!
+//! Every request below is made with a **minted, non-admin** API key, because
+//! the issue is about what an ordinary authenticated caller can reach — not
+//! about the superuser.
+
+use axum::body::Body;
+use axum::http::{Request, StatusCode};
+use http_body_util::BodyExt;
+use serde_json::Value;
+use tower::ServiceExt;
+use xerj_api::{
+ router::{build_es_compat_router, build_native_router},
+ state::AppState,
+};
+use xerj_common::{config::Config, metrics::Metrics};
+use xerj_engine::Engine;
+
+const ADMIN_KEY: &str = "admin-secret-key-for-boundary-test";
+
+/// Fixture instants. Pinned rather than "now" so a link and the `as_of` that
+/// reads it back can never land in the same millisecond.
+const T0: i64 = 1_753_600_000_000;
+const AS_OF: i64 = 1_753_700_000_000;
+
+/// An auth-enabled node over a fresh data directory. The `TempDir` is returned
+/// and must be held for the whole test: the Engine keeps the directory open,
+/// and these are engine data dirs, so leaking them fills `/tmp`.
+fn auth_enabled_state() -> (AppState, tempfile::TempDir) {
+ let dir = tempfile::tempdir().expect("tempdir");
+ let state = state_over(dir.path().to_str().unwrap(), true);
+ (state, dir)
+}
+
+fn state_over(data_dir: &str, auth: bool) -> AppState {
+ let mut config = Config::default();
+ config.server.data_dir = data_dir.to_string();
+ config.auth.enabled = auth;
+ config.auth.admin_api_key = if auth {
+ ADMIN_KEY.to_string()
+ } else {
+ String::new()
+ };
+ let metrics = Metrics::new().expect("metrics");
+ let engine = Engine::new(config.clone()).expect("engine");
+ AppState::new(config, engine, metrics)
+}
+
+/// Like [`send`] but returns the body verbatim — `_cat` answers a plain-text
+/// table, which is exactly the shape the line-level pruning has to handle.
+async fn send_text(
+ app: &axum::Router,
+ method: &str,
+ uri: &str,
+ auth: &str,
+) -> (StatusCode, String) {
+ let mut builder = Request::builder().method(method).uri(uri);
+ if !auth.is_empty() {
+ builder = builder.header("authorization", auth);
+ }
+ let req = builder.body(Body::empty()).expect("request");
+ let resp = app.clone().oneshot(req).await.expect("response");
+ let status = resp.status();
+ let bytes = resp.into_body().collect().await.expect("body").to_bytes();
+ (status, String::from_utf8_lossy(&bytes).into_owned())
+}
+
+async fn send(
+ app: &axum::Router,
+ method: &str,
+ uri: &str,
+ auth: &str,
+ body: &str,
+) -> (StatusCode, Value) {
+ let mut builder = Request::builder()
+ .method(method)
+ .uri(uri)
+ .header("content-type", "application/json");
+ if !auth.is_empty() {
+ builder = builder.header("authorization", auth);
+ }
+ let req = builder.body(Body::from(body.to_string())).expect("request");
+ let resp = app.clone().oneshot(req).await.expect("response");
+ let status = resp.status();
+ let bytes = resp.into_body().collect().await.expect("body").to_bytes();
+ let json: Value = serde_json::from_slice(&bytes).unwrap_or(Value::Null);
+ (status, json)
+}
+
+/// Mint an API key with the given JSON body and return its `Authorization`
+/// header value.
+async fn mint(app: &axum::Router, minter: &str, body: &str) -> String {
+ let (status, resp) = send(app, "POST", "/_security/api_key", minter, body).await;
+ assert_eq!(
+ status,
+ StatusCode::OK,
+ "minting a key should succeed: {resp}"
+ );
+ let encoded = resp["encoded"].as_str().expect("encoded key").to_string();
+ format!("ApiKey {encoded}")
+}
+
+/// A key scoped to exactly one brain: its edges index and its nodes index.
+/// This is the documented grant shape from `graph_api`'s module docs.
+fn brain_grant(brain: &str, privileges: &str) -> String {
+ format!(
+ r#"{{"name":"{brain}-agent","role_descriptors":{{"{brain}":{{"indices":[
+ {{"names":[".xerj-memory-{brain}-edges",".xerj-memory-{brain}"],
+ "privileges":[{privileges}]}}]}}}}}}"#
+ )
+}
+
+/// Seed two brains, as the admin, so there is something to keep apart.
+async fn seed_two_brains(app: &axum::Router) {
+ let admin = format!("ApiKey {ADMIN_KEY}");
+ for (brain, dst) in [("alice", "doc:2"), ("bob", "secret:2")] {
+ let (status, body) = send(
+ app,
+ "POST",
+ &format!("/_graph/{brain}/link"),
+ &admin,
+ &format!(
+ r#"{{"src":"doc:1","type":"mentions","dst":"{dst}",
+ "valid_at":{T0},"created_at":{T0}}}"#
+ ),
+ )
+ .await;
+ assert_eq!(status, StatusCode::CREATED, "seeding {brain}: {body}");
+ }
+ let (status, _) = send(app, "POST", "/.xerj-memory-bob-edges/_refresh", &admin, "").await;
+ assert_eq!(status, StatusCode::OK);
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// Every door into someone else's brain, walked with a credential scoped to a
+/// different brain. This is the assertion the issue asked for, and the exact
+/// inverse of what the previous test measured.
+#[tokio::test]
+async fn a_scoped_caller_cannot_reach_another_brain_through_any_door() {
+ let (state, _data_dir) = auth_enabled_state();
+ let app = build_es_compat_router(state.clone());
+ let native = build_native_router(state);
+ seed_two_brains(&app).await;
+
+ let admin = format!("ApiKey {ADMIN_KEY}");
+ let alice = mint(&app, &admin, &brain_grant("alice", r#""read","write""#)).await;
+
+ // ── 1. The graph API: all four entry points, denied for bob. ───────────
+ for (method, uri) in [
+ ("GET", "/_graph/bob/ego?node=doc:1"),
+ ("GET", "/_graph/bob/overview"),
+ ("POST", "/_graph/bob/link"),
+ ("DELETE", "/_graph/bob/link/whatever"),
+ ] {
+ let (status, body) = send(
+ &app,
+ method,
+ uri,
+ &alice,
+ r#"{"src":"a","type":"mentions","dst":"b"}"#,
+ )
+ .await;
+ assert_eq!(
+ status,
+ StatusCode::FORBIDDEN,
+ "{method} {uri} must be forbidden, got {status}: {body}"
+ );
+ }
+
+ // ── 2. The raw index surface, which is where the real exposure lived. ──
+ // Read around `/_graph`, forge an edge past the derived-`edge_id`
+ // invariant, refresh, and destroy the brain outright.
+ for (method, uri, body) in [
+ (
+ "POST",
+ "/.xerj-memory-bob-edges/_search",
+ r#"{"query":{"match_all":{}}}"#,
+ ),
+ (
+ "POST",
+ "/.xerj-memory-bob-edges/_doc/forged",
+ r#"{"edge_id":"forged","src":"doc:1","dst":"attacker:payload","type":"mentions"}"#,
+ ),
+ ("POST", "/.xerj-memory-bob-edges/_refresh", ""),
+ ("GET", "/.xerj-memory-bob-edges/_doc/anything", ""),
+ ("DELETE", "/.xerj-memory-bob-edges", ""),
+ // Percent-encoded, in case the check compared raw path text.
+ ("POST", "/%2Exerj-memory-bob-edges/_search", "{}"),
+ ] {
+ let (status, resp) = send(&app, method, uri, &alice, body).await;
+ assert_eq!(
+ status,
+ StatusCode::FORBIDDEN,
+ "{method} {uri} must be forbidden, got {status}: {resp}"
+ );
+ }
+
+ // ── 2b. Patterns are answered, not refused — over the caller's visible
+ // set only. Refusing them outright was the first cut of this fix and it
+ // cost every legitimate `logs-*` grant; expanding them over what the
+ // principal may see is both usable and closed. What must never appear is
+ // one byte of bob's.
+ for uri in [
+ "/.xerj-memory-*/_search",
+ "/*/_search",
+ "/_all/_search",
+ "/.xerj-memory-bob*/_search",
+ ] {
+ let (status, resp) = send(&app, "POST", uri, &alice, "{}").await;
+ assert_eq!(
+ status,
+ StatusCode::OK,
+ "POST {uri} should be answered over the visible set, got {status}: {resp}"
+ );
+ assert!(
+ !resp.to_string().contains("bob"),
+ "POST {uri} leaked bob: {resp}"
+ );
+ }
+
+ // ── 3. The agent-memory surface: bob's brain keeps its NODES in the
+ // `.xerj-memory-bob` namespace, so `/_memory/*` is a door to the same data.
+ for (method, uri) in [
+ ("POST", "/_memory/bob/_recall"),
+ ("GET", "/_memory/bob"),
+ ("POST", "/_memory/bob"),
+ ("DELETE", "/_memory/bob"),
+ ("DELETE", "/_memory/bob/some-id"),
+ ] {
+ let (status, resp) = send(&app, method, uri, &alice, r#"{"text":"x","query":"x"}"#).await;
+ assert_eq!(
+ status,
+ StatusCode::FORBIDDEN,
+ "{method} {uri} must be forbidden, got {status}: {resp}"
+ );
+ }
+
+ // ── 4. Enumeration: a scoped caller gets metadata for the indices it
+ // holds and no trace of anything else, so brain names stay unguessable
+ // WITHOUT the blanket 403 that stopped Kibana from working at all.
+ for uri in [
+ "/_mapping",
+ "/_cat/indices",
+ "/_alias",
+ "/_settings",
+ "/_cluster/stats",
+ "/_resolve/index/*",
+ "/_cluster/health",
+ "/_nodes",
+ ] {
+ let (status, resp) = send(&app, "GET", uri, &alice, "").await;
+ assert_eq!(
+ status,
+ StatusCode::OK,
+ "GET {uri} must be answered for a scoped key, got {status}: {resp}"
+ );
+ assert!(
+ !resp.to_string().contains("bob"),
+ "GET {uri} enumerated bob's brain: {resp}"
+ );
+ }
+ let (status, table) = send_text(&app, "GET", "/_cat/indices", &alice).await;
+ assert_eq!(status, StatusCode::OK);
+ assert!(
+ !table.contains("bob"),
+ "_cat/indices enumerated bob's brain: {table}"
+ );
+
+ // ── 5. Unnamed fan-out. `POST /_search` with no index reads every index
+ // the caller can see — which is alice's, and only alice's. The global
+ // verbs are the default paths of every ES client, so they answer; what
+ // they must not do is answer with someone else's data.
+ for (method, uri, body) in [
+ ("POST", "/_search", "{}"),
+ ("POST", "/_msearch", "{}\n{\"query\":{\"match_all\":{}}}\n"),
+ ("POST", "/_count", "{}"),
+ ] {
+ let (status, resp) = send(&app, method, uri, &alice, body).await;
+ assert_eq!(
+ status,
+ StatusCode::OK,
+ "{method} {uri} must be answered over the visible set, got {status}: {resp}"
+ );
+ assert!(
+ !resp.to_string().contains("bob"),
+ "{method} {uri} leaked bob: {resp}"
+ );
+ }
+ // The global write path works for the caller's own index …
+ let (status, resp) = send(
+ &app,
+ "POST",
+ "/_bulk",
+ &alice,
+ "{\"index\":{\"_index\":\".xerj-memory-alice-edges\",\"_id\":\"own\"}}\n{\"src\":\"a\"}\n",
+ )
+ .await;
+ assert_eq!(
+ status,
+ StatusCode::OK,
+ "global _bulk into its own index must work: {resp}"
+ );
+ // … and refuses the moment the body names someone else's.
+ for body in [
+ "{\"index\":{\"_index\":\".xerj-memory-bob-edges\",\"_id\":\"forged\"}}\n{\"src\":\"a\"}\n",
+ "{\"delete\":{\"_index\":\".xerj-memory-bob-edges\",\"_id\":\"e1\"}}\n",
+ ] {
+ let (status, resp) = send(&app, "POST", "/_bulk", &alice, body).await;
+ assert_eq!(
+ status,
+ StatusCode::FORBIDDEN,
+ "a body-named _bulk into bob must be forbidden, got {status}: {resp}"
+ );
+ }
+ // Same for the body-named forms of _msearch and _mget.
+ let (status, resp) = send(
+ &app,
+ "POST",
+ "/_msearch",
+ &alice,
+ "{\"index\":\".xerj-memory-bob-edges\"}\n{\"query\":{\"match_all\":{}}}\n",
+ )
+ .await;
+ assert_eq!(status, StatusCode::FORBIDDEN, "body-named _msearch: {resp}");
+ let (status, resp) = send(
+ &app,
+ "POST",
+ "/_mget",
+ &alice,
+ "{\"docs\":[{\"_index\":\".xerj-memory-bob-edges\",\"_id\":\"x\"}]}",
+ )
+ .await;
+ assert_eq!(status, StatusCode::FORBIDDEN, "body-named _mget: {resp}");
+
+ // ── 6. The native router is the same engine under a different spelling.
+ for (method, uri) in [
+ ("POST", "/v1/indices/.xerj-memory-bob-edges/search"),
+ ("GET", "/v1/indices/.xerj-memory-bob-edges"),
+ ("DELETE", "/v1/indices/.xerj-memory-bob-edges"),
+ ("POST", "/v1/indices/.xerj-memory-bob-edges/docs"),
+ ] {
+ let (status, resp) = send(&native, method, uri, &alice, r#"{"query":{}}"#).await;
+ assert_eq!(
+ status,
+ StatusCode::FORBIDDEN,
+ "native {method} {uri} must be forbidden, got {status}: {resp}"
+ );
+ }
+
+ // ── 7. Escalation: a scoped caller must not be able to mint itself a key
+ // that grants bob.
+ let (status, resp) = send(
+ &app,
+ "POST",
+ "/_security/api_key",
+ &alice,
+ &brain_grant("bob", r#""all""#),
+ )
+ .await;
+ assert_eq!(
+ status,
+ StatusCode::FORBIDDEN,
+ "a scoped key must not mint keys, got {status}: {resp}"
+ );
+
+ // ── 8. Existence must not leak through the status code: a brain that does
+ // not exist is refused exactly like one that does.
+ let (status, _) = send(&app, "GET", "/_graph/nosuchbrain/overview", &alice, "").await;
+ assert_eq!(
+ status,
+ StatusCode::FORBIDDEN,
+ "403 (not 404) for an unauthorized brain, or the code maps the node"
+ );
+}
+
+/// The boundary is only worth anything if the authorized side still works.
+/// A key scoped to `alice` must be able to do everything a brain owner does.
+#[tokio::test]
+async fn a_scoped_caller_retains_full_use_of_its_own_brain() {
+ let (state, _data_dir) = auth_enabled_state();
+ let app = build_es_compat_router(state);
+ seed_two_brains(&app).await;
+
+ let admin = format!("ApiKey {ADMIN_KEY}");
+ let alice = mint(&app, &admin, &brain_grant("alice", r#""read","write""#)).await;
+
+ // Write.
+ let (status, body) = send(
+ &app,
+ "POST",
+ "/_graph/alice/link",
+ &alice,
+ &format!(
+ r#"{{"src":"doc:1","type":"mentions","dst":"doc:3",
+ "valid_at":{T0},"created_at":{T0}}}"#
+ ),
+ )
+ .await;
+ assert_eq!(status, StatusCode::CREATED, "own-brain link failed: {body}");
+ let edge_id = body["edge_id"].as_str().expect("edge_id").to_string();
+
+ let (status, _) = send(
+ &app,
+ "POST",
+ "/.xerj-memory-alice-edges/_refresh",
+ &alice,
+ "",
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "own backing index is usable");
+
+ // Read, both entry points.
+ let (status, ego) = send(
+ &app,
+ "GET",
+ &format!("/_graph/alice/ego?node=doc:1&as_of={AS_OF}"),
+ &alice,
+ "",
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "own-brain ego failed: {ego}");
+ assert!(
+ ego.to_string().contains("doc:3"),
+ "own edge should be visible: {ego}"
+ );
+ let (status, overview) = send(
+ &app,
+ "GET",
+ &format!("/_graph/alice/overview?as_of={AS_OF}"),
+ &alice,
+ "",
+ )
+ .await;
+ assert_eq!(
+ status,
+ StatusCode::OK,
+ "own-brain overview failed: {overview}"
+ );
+
+ // Invalidate.
+ let (status, body) = send(
+ &app,
+ "DELETE",
+ &format!("/_graph/alice/link/{edge_id}"),
+ &alice,
+ "",
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "own-brain unlink failed: {body}");
+
+ // A read-only grant reads but does not write — the privilege half of the
+ // model, not just the resource half.
+ let reader = mint(&app, &admin, &brain_grant("alice", r#""read""#)).await;
+ let (status, _) = send(&app, "GET", "/_graph/alice/overview", &reader, "").await;
+ assert_eq!(status, StatusCode::OK, "read grant must read");
+ let (status, _) = send(
+ &app,
+ "POST",
+ "/_graph/alice/link",
+ &reader,
+ r#"{"src":"a","type":"mentions","dst":"b"}"#,
+ )
+ .await;
+ assert_eq!(status, StatusCode::FORBIDDEN, "read grant must not write");
+ let (status, _) = send(&app, "DELETE", "/.xerj-memory-alice-edges", &reader, "").await;
+ assert_eq!(
+ status,
+ StatusCode::FORBIDDEN,
+ "read grant must not destroy the brain"
+ );
+
+ // Resource granularity within one brain: the notes and the link graph are
+ // two indices, and `POST /_memory/{ns}/_recall` with `graph:` reads the
+ // second. A credential given only the notes must not get the graph through
+ // the side door — while plain recall over the notes it *was* granted keeps
+ // working.
+ let notes_only = mint(
+ &app,
+ &admin,
+ r#"{"name":"notes-only","role_descriptors":{"n":{"indices":[
+ {"names":[".xerj-memory-alice"],"privileges":["read"]}]}}}"#,
+ )
+ .await;
+ let (status, _) = send(
+ &app,
+ "POST",
+ "/_memory/alice/_recall",
+ &notes_only,
+ r#"{"query":"anything"}"#,
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "plain recall on the granted notes");
+ let (status, resp) = send(
+ &app,
+ "POST",
+ "/_memory/alice/_recall",
+ &notes_only,
+ r#"{"query":"anything","graph":{"mode":"restrict","seeds":["doc:1"]}}"#,
+ )
+ .await;
+ assert_eq!(
+ status,
+ StatusCode::FORBIDDEN,
+ "graph coupling reads the edges index and needs its own grant: {resp}"
+ );
+}
+
+/// The fail-closed half. A key minted with no `role_descriptors` — the shape
+/// every key had before this existed — holds **nothing** in the reserved
+/// namespace, and cannot mint itself out of that.
+#[tokio::test]
+async fn an_unconfigured_key_is_denied_every_brain_and_cannot_escalate() {
+ let (state, _data_dir) = auth_enabled_state();
+ let app = build_es_compat_router(state);
+ seed_two_brains(&app).await;
+
+ let admin = format!("ApiKey {ADMIN_KEY}");
+ let legacy = mint(&app, &admin, r#"{"name":"legacy"}"#).await;
+
+ for (method, uri) in [
+ ("GET", "/_graph/alice/ego?node=doc:1"),
+ ("GET", "/_graph/bob/overview"),
+ ("POST", "/_graph/alice/link"),
+ ("POST", "/.xerj-memory-alice-edges/_search"),
+ ("DELETE", "/.xerj-memory-bob-edges"),
+ ("POST", "/_memory/alice/_recall"),
+ ] {
+ let (status, resp) = send(
+ &app,
+ method,
+ uri,
+ &legacy,
+ r#"{"src":"a","type":"t","dst":"b","query":"x"}"#,
+ )
+ .await;
+ assert_eq!(
+ status,
+ StatusCode::FORBIDDEN,
+ "unconfigured key on {method} {uri} must be denied, got {status}: {resp}"
+ );
+ }
+
+ // It keeps its historical reach over ordinary indices — broad RBAC over the
+ // general surface is still deferred, and this fix does not pretend
+ // otherwise.
+ let (status, resp) = send(&app, "PUT", "/logs-2026", &legacy, "{}").await;
+ assert!(
+ status.is_success(),
+ "a legacy key must still create an ordinary index, got {status}: {resp}"
+ );
+ let (status, resp) = send(
+ &app,
+ "POST",
+ "/logs-2026/_doc/1",
+ &legacy,
+ r#"{"msg":"hello"}"#,
+ )
+ .await;
+ assert!(
+ status.is_success(),
+ "a legacy key must still write an ordinary index, got {status}: {resp}"
+ );
+
+ // But enumeration hides the brains: `_mapping` answers, without them.
+ let (status, mapping) = send(&app, "GET", "/_mapping", &legacy, "").await;
+ assert_eq!(status, StatusCode::OK, "metadata still answers");
+ let text = mapping.to_string();
+ assert!(
+ !text.contains(".xerj-memory-"),
+ "brain names must not be enumerable: {text}"
+ );
+ assert!(
+ text.contains("logs-2026"),
+ "ordinary indices are still listed: {text}"
+ );
+ let (status, cat) = send_text(&app, "GET", "/_cat/indices", &legacy).await;
+ assert_eq!(status, StatusCode::OK);
+ assert!(
+ !cat.contains(".xerj-memory-"),
+ "_cat/indices must not enumerate brains: {cat}"
+ );
+ assert!(
+ cat.contains("logs-2026"),
+ "_cat/indices still lists ordinary indices: {cat}"
+ );
+
+ // And it cannot mint its way in: `role_descriptors` from a non-superuser
+ // are dropped, so the child key is unscoped too.
+ let child = mint(&app, &legacy, &brain_grant("bob", r#""all""#)).await;
+ let (status, resp) = send(&app, "GET", "/_graph/bob/overview", &child, "").await;
+ assert_eq!(
+ status,
+ StatusCode::FORBIDDEN,
+ "a key minted by a non-superuser must not carry grants: {resp}"
+ );
+}
+
+/// The local-dev path must not have moved. `xerj --insecure` /
+/// point-at-a-folder has one user and no configuration; every brain stays
+/// reachable with no credential at all.
+#[tokio::test]
+async fn insecure_single_user_mode_is_unchanged() {
+ let dir = tempfile::tempdir().expect("tempdir");
+ let state = state_over(dir.path().to_str().unwrap(), false);
+ let app = build_es_compat_router(state);
+
+ for brain in ["alice", "bob"] {
+ let (status, body) = send(
+ &app,
+ "POST",
+ &format!("/_graph/{brain}/link"),
+ "",
+ r#"{"src":"doc:1","type":"mentions","dst":"doc:2"}"#,
+ )
+ .await;
+ assert_eq!(status, StatusCode::CREATED, "insecure link failed: {body}");
+ let (status, _) = send(&app, "GET", &format!("/_graph/{brain}/overview"), "", "").await;
+ assert_eq!(status, StatusCode::OK, "insecure overview must work");
+ }
+ // Including the doors a scoped key is refused: there is nothing to scope.
+ let (status, _) = send(&app, "GET", "/_mapping", "", "").await;
+ assert_eq!(status, StatusCode::OK, "insecure enumeration must work");
+ let (status, cat) = send_text(&app, "GET", "/_cat/indices", "").await;
+ assert_eq!(status, StatusCode::OK);
+ assert!(
+ cat.contains(".xerj-memory-"),
+ "the single local user still sees their own brains: {cat}"
+ );
+ let (status, _) = send(
+ &app,
+ "POST",
+ "/_search",
+ "",
+ r#"{"query":{"match_all":{}}}"#,
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "insecure fan-out must work");
+}
+
+/// Grants are persisted with the key. A node restart must not silently turn a
+/// scoped key back into an unscoped one — nor, worse, into a superuser.
+#[tokio::test]
+async fn grants_survive_a_restart() {
+ let dir = tempfile::tempdir().expect("tempdir");
+ let data_dir = dir.path().to_str().unwrap().to_string();
+
+ // Scope the first node so its data-dir lock is released before the second
+ // one boots over the same directory.
+ let alice = {
+ let state = state_over(&data_dir, true);
+ let app = build_es_compat_router(state.clone());
+ seed_two_brains(&app).await;
+ let admin = format!("ApiKey {ADMIN_KEY}");
+ let alice = mint(&app, &admin, &brain_grant("alice", r#""read","write""#)).await;
+ drop(app);
+ drop(state);
+ alice
+ };
+
+ // Boot a second engine over the same data directory — the restart.
+ let restarted = build_es_compat_router(state_over(&data_dir, true));
+
+ let (status, resp) = send(&restarted, "GET", "/_graph/alice/overview", &alice, "").await;
+ assert_eq!(
+ status,
+ StatusCode::OK,
+ "the grant must survive restart, got {status}: {resp}"
+ );
+ let (status, resp) = send(&restarted, "GET", "/_graph/bob/overview", &alice, "").await;
+ assert_eq!(
+ status,
+ StatusCode::FORBIDDEN,
+ "the boundary must survive restart, got {status}: {resp}"
+ );
+}
diff --git a/engine/crates/xerj-api/tests/ml_datafeed_cannot_read_a_brain.rs b/engine/crates/xerj-api/tests/ml_datafeed_cannot_read_a_brain.rs
new file mode 100644
index 000000000..1f2ae0e21
--- /dev/null
+++ b/engine/crates/xerj-api/tests/ml_datafeed_cannot_read_a_brain.rs
@@ -0,0 +1,541 @@
+//! The ML datafeed is not a way around the per-index boundary (issue #79).
+//!
+//! This is the exploit that got past the first two-layer cut, run end to end
+//! against a real auth-enforced router, and it is deliberately written as the
+//! attack rather than as a unit test of either half of the fix.
+//!
+//! ## What it was
+//!
+//! The engine-side guard is a `tokio` task-local, so it lives on the task that
+//! is handling the request. `POST /_ml/datafeeds/{id}/_start` did two things:
+//! one synchronous scoring pass, on the request's own task, and then a
+//! detached `tokio::spawn` that re-scored every `frequency` seconds. The
+//! synchronous pass was inside the guard and was correctly denied — which is
+//! what made this so quiet. The caller saw an empty result set and could
+//! reasonably conclude it had been stopped. A couple of seconds later the
+//! background tick ran with no guard installed at all, read the brain, and
+//! appended the field values it found to the job's results, where the same
+//! caller read them straight back out of
+//! `GET /_ml/anomaly_detectors/{job}/results/records`.
+//!
+//! The credential used is an **unscoped** key: minted with no
+//! `role_descriptors`, the shape every key had before per-brain authorization
+//! existed. The design guarantees it holds *nothing* on the reserved
+//! namespace, and `a_direct_read_of_the_brain_is_still_denied` re-proves that
+//! for this exact key inside every test below, so a passing run cannot be an
+//! artifact of a credential that was allowed all along.
+//!
+//! ## What is asserted
+//!
+//! Both halves, separately, because neither alone closes it:
+//!
+//! - **config time** — an ML job or datafeed naming an index the caller cannot
+//! read is refused, instead of being accepted with a 200;
+//! - **run time** — even when the configuration was planted by the superuser,
+//! so there was nothing to refuse, the detached scorer runs under the
+//! *starting* principal's visibility rule and finds nothing.
+
+use axum::body::Body;
+use axum::http::{Request, StatusCode};
+use http_body_util::BodyExt;
+use serde_json::Value;
+use tower::ServiceExt;
+use xerj_api::{router::build_es_compat_router, state::AppState};
+use xerj_common::{config::Config, metrics::Metrics};
+use xerj_engine::Engine;
+
+const ADMIN_KEY: &str = "admin-secret-key-for-ml-datafeed-test";
+const BRAIN: &str = ".xerj-memory-bob-edges";
+
+/// The value planted on bob's TOPSECRET edge. Distinctive enough that a
+/// substring search over the whole response body is a sound leak test.
+const SECRET: i64 = 987_654_321;
+
+/// An auth-enabled node over a fresh data directory. The `TempDir` must be
+/// held for the whole test — the Engine keeps the directory open.
+fn auth_enabled_state() -> (AppState, tempfile::TempDir) {
+ let dir = tempfile::tempdir().expect("tempdir");
+ let mut config = Config::default();
+ config.server.data_dir = dir.path().to_str().unwrap().to_string();
+ config.auth.enabled = true;
+ config.auth.admin_api_key = ADMIN_KEY.to_string();
+ let metrics = Metrics::new().expect("metrics");
+ let engine = Engine::new(config.clone()).expect("engine");
+ (AppState::new(config, engine, metrics), dir)
+}
+
+async fn send(
+ app: &axum::Router,
+ method: &str,
+ uri: &str,
+ auth: &str,
+ body: &str,
+) -> (StatusCode, Value) {
+ let req = Request::builder()
+ .method(method)
+ .uri(uri)
+ .header("content-type", "application/json")
+ .header("authorization", auth)
+ .body(Body::from(body.to_string()))
+ .expect("request");
+ let resp = app.clone().oneshot(req).await.expect("response");
+ let status = resp.status();
+ let bytes = resp.into_body().collect().await.expect("body").to_bytes();
+ (
+ status,
+ serde_json::from_slice(&bytes).unwrap_or(Value::Null),
+ )
+}
+
+fn admin() -> String {
+ format!("ApiKey {ADMIN_KEY}")
+}
+
+/// Mint a key with **no** `role_descriptors` — `Principal::Unscoped`.
+async fn mint_unscoped(app: &axum::Router) -> String {
+ let (status, resp) = send(
+ app,
+ "POST",
+ "/_security/api_key",
+ &admin(),
+ r#"{"name":"leak-key"}"#,
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "minting: {resp}");
+ format!("ApiKey {}", resp["encoded"].as_str().expect("encoded key"))
+}
+
+/// Plant bob's brain as the superuser: four ordinary edges, then the TOPSECRET
+/// one. The detector needs four normal buckets of baseline before it will
+/// score anything, so the fifth bucket is the anomaly and the record's
+/// `actual` is the exact planted `valid_at`.
+async fn plant_the_brain(app: &axum::Router) {
+ for i in 1..=4 {
+ let (status, body) = send(
+ app,
+ "PUT",
+ &format!("/{BRAIN}/_doc/e{i}"),
+ &admin(),
+ &format!(
+ r#"{{"from":"bob","to":"n{i}","type":"knows",
+ "created_at":"2026-01-01T00:00:0{i}Z","valid_at":100{i}}}"#
+ ),
+ )
+ .await;
+ assert!(status.is_success(), "seeding e{i}: {body}");
+ }
+ let (status, body) = send(
+ app,
+ "PUT",
+ &format!("/{BRAIN}/_doc/secret?refresh=true"),
+ &admin(),
+ &format!(
+ r#"{{"from":"bob","to":"TOPSECRET","type":"knows",
+ "created_at":"2026-01-01T00:00:09Z","valid_at":{SECRET}}}"#
+ ),
+ )
+ .await;
+ assert!(status.is_success(), "seeding the secret edge: {body}");
+}
+
+/// The detector and datafeed configs exactly as the reported exploit spells
+/// them.
+const DETECTOR_BODY: &str = r#"{"source_index":".xerj-memory-bob-edges","time_field":"created_at",
+ "function":"max","field":"valid_at","bucket_span":"1s"}"#;
+const FEED_BODY: &str = r#"{"job_id":"leak2","indices":[".xerj-memory-bob-edges"],
+ "frequency":"1s"}"#;
+
+/// The control every assertion in this file rests on: the key really is
+/// denied the brain by the front door, so anything it learns by another route
+/// is a bypass and not a grant.
+async fn a_direct_read_of_the_brain_is_still_denied(app: &axum::Router, key: &str) {
+ let (status, body) = send(
+ app,
+ "POST",
+ &format!("/{BRAIN}/_search"),
+ key,
+ r#"{"query":{"match_all":{}}}"#,
+ )
+ .await;
+ assert_eq!(
+ status,
+ StatusCode::FORBIDDEN,
+ "the credential under test must be denied the brain directly: {body}"
+ );
+}
+
+/// Read the job's anomaly records and assert the planted value is not in them,
+/// whatever shape the response takes. Returns the body so a failure can print
+/// what actually came back.
+async fn records_must_stay_clean(app: &axum::Router, key: &str, when: &str) -> Value {
+ let (status, body) = send(
+ app,
+ "GET",
+ "/_ml/anomaly_detectors/leak2/results/records",
+ key,
+ "",
+ )
+ .await;
+ let text = body.to_string();
+ assert!(
+ !text.contains(&SECRET.to_string()),
+ "{when}: the reserved field value {SECRET} reached a credential that \
+ cannot read {BRAIN} — HTTP {status}, body {text}"
+ );
+ assert_eq!(
+ body["count"].as_i64().unwrap_or(0),
+ 0,
+ "{when}: the datafeed produced records off a brain it may not read: {text}"
+ );
+ body
+}
+
+/// Wait out several datafeed ticks. The feed's `frequency` is 1s and the tick
+/// that leaked fired on the second one, so four seconds is three chances to
+/// fail — real elapsed time, because the scorer runs on a real interval.
+async fn let_the_background_scorer_tick() {
+ tokio::time::sleep(std::time::Duration::from_millis(4_200)).await;
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// FINDING A, both halves, as the reported exploit runs it.
+///
+/// Steps 1-3 are the config; step 4 is the `_start`; step 5 is the tell (empty
+/// immediately, because the synchronous pass is inside the guard); step 6 is
+/// where it used to leak.
+#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
+async fn the_reported_ml_datafeed_exploit_end_to_end() {
+ let (state, _dir) = auth_enabled_state();
+ let app = build_es_compat_router(state);
+ plant_the_brain(&app).await;
+ let attacker = mint_unscoped(&app).await;
+ a_direct_read_of_the_brain_is_still_denied(&app, &attacker).await;
+
+ // 1. The superuser can score it — so the secret really is reachable by
+ // this configuration, and an empty result later is the boundary
+ // working rather than the fixture being wrong.
+ let (status, body) = send(
+ &app,
+ "PUT",
+ "/_ml/anomaly_detectors/control",
+ &admin(),
+ DETECTOR_BODY,
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "admin detector: {body}");
+ let (status, scored) = send(
+ &app,
+ "POST",
+ "/_ml/anomaly_detectors/control/_score",
+ &admin(),
+ "{}",
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "admin _score: {scored}");
+ assert!(
+ scored.to_string().contains(&SECRET.to_string()),
+ "fixture check: the superuser must be able to score the planted value, \
+ otherwise this test proves nothing. Got: {scored}"
+ );
+
+ // 2. PUT /_ml/anomaly_detectors/leak2 — refused at config time now.
+ let (status, body) = send(
+ &app,
+ "PUT",
+ "/_ml/anomaly_detectors/leak2",
+ &attacker,
+ DETECTOR_BODY,
+ )
+ .await;
+ assert_eq!(
+ status,
+ StatusCode::FORBIDDEN,
+ "an ML job may not name an index its creator cannot read: {body}"
+ );
+
+ // 3. PUT /_ml/datafeeds/leak2-feed — likewise.
+ let (status, body) = send(
+ &app,
+ "PUT",
+ "/_ml/datafeeds/leak2-feed",
+ &attacker,
+ FEED_BODY,
+ )
+ .await;
+ assert_eq!(
+ status,
+ StatusCode::FORBIDDEN,
+ "a datafeed may not name an index its creator cannot read: {body}"
+ );
+
+ // 4. …so there is nothing to start, and nothing to read.
+ let (status, _) = send(
+ &app,
+ "POST",
+ "/_ml/datafeeds/leak2-feed/_start",
+ &attacker,
+ "{}",
+ )
+ .await;
+ assert_ne!(
+ status,
+ StatusCode::OK,
+ "a datafeed that was never created must not start"
+ );
+
+ // 5 & 6. Empty immediately, and still empty after the ticks that used to
+ // carry the value out.
+ records_must_stay_clean(&app, &attacker, "immediately after _start").await;
+ let_the_background_scorer_tick().await;
+ records_must_stay_clean(&app, &attacker, "after the background ticks").await;
+}
+
+/// FINDING A, run-time half **in isolation**.
+///
+/// The config-time check cannot help here: the superuser plants the job and
+/// the datafeed, so both already exist and name the brain legitimately. All
+/// the attacker does is press `_start`. That is enough, because `_start` is
+/// what spawns the detached scorer — and the scorer must therefore run under
+/// the rule of whoever started it.
+///
+/// This is the assertion that fails if only the `BodyShape` half is applied.
+#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
+async fn a_superuser_planted_datafeed_started_by_an_unscoped_key_reads_nothing() {
+ let (state, _dir) = auth_enabled_state();
+ let app = build_es_compat_router(state);
+ plant_the_brain(&app).await;
+ let attacker = mint_unscoped(&app).await;
+ a_direct_read_of_the_brain_is_still_denied(&app, &attacker).await;
+
+ // The superuser configures everything. Nothing here is refusable.
+ let (status, body) = send(
+ &app,
+ "PUT",
+ "/_ml/anomaly_detectors/leak2",
+ &admin(),
+ DETECTOR_BODY,
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "admin detector: {body}");
+ let (status, body) = send(
+ &app,
+ "PUT",
+ "/_ml/datafeeds/leak2-feed",
+ &admin(),
+ FEED_BODY,
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "admin datafeed: {body}");
+
+ // The attacker only presses start.
+ let (status, body) = send(
+ &app,
+ "POST",
+ "/_ml/datafeeds/leak2-feed/_start",
+ &attacker,
+ "{}",
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "_start under the attacker: {body}");
+
+ // The tell: empty right away, because the synchronous pass ran inside the
+ // request's guarded scope. This was true before the fix too.
+ records_must_stay_clean(&app, &attacker, "immediately after _start").await;
+
+ // The bug: the detached tick. Before the fix this came back with
+ // `actual: 987654321.0`.
+ let_the_background_scorer_tick().await;
+ records_must_stay_clean(&app, &attacker, "after the background ticks").await;
+
+ // And the brain is still not readable by the front door afterwards, so
+ // the empty results are the boundary and not a broken datafeed.
+ a_direct_read_of_the_brain_is_still_denied(&app, &attacker).await;
+}
+
+/// The other side of the same change: a datafeed the **superuser** starts must
+/// keep working exactly as before. A superuser has no visibility rule
+/// installed, so `None` is what gets carried across the spawn, and the scorer
+/// stays unrestricted.
+#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
+async fn a_superuser_started_datafeed_still_scores() {
+ let (state, _dir) = auth_enabled_state();
+ let app = build_es_compat_router(state);
+ plant_the_brain(&app).await;
+
+ let (status, body) = send(
+ &app,
+ "PUT",
+ "/_ml/anomaly_detectors/leak2",
+ &admin(),
+ DETECTOR_BODY,
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "detector: {body}");
+ let (status, body) = send(
+ &app,
+ "PUT",
+ "/_ml/datafeeds/leak2-feed",
+ &admin(),
+ FEED_BODY,
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "datafeed: {body}");
+ let (status, body) = send(
+ &app,
+ "POST",
+ "/_ml/datafeeds/leak2-feed/_start",
+ &admin(),
+ "{}",
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "_start: {body}");
+
+ // The synchronous start pass alone should already have produced it.
+ let (status, records) = send(
+ &app,
+ "GET",
+ "/_ml/anomaly_detectors/leak2/results/records",
+ &admin(),
+ "",
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK);
+ assert!(
+ records.to_string().contains(&SECRET.to_string()),
+ "the superuser's own datafeed must still score its index: {records}"
+ );
+
+ // …and the background ticks must not break it either.
+ let_the_background_scorer_tick().await;
+ let (_, records) = send(
+ &app,
+ "GET",
+ "/_ml/anomaly_detectors/leak2/results/records",
+ &admin(),
+ "",
+ )
+ .await;
+ assert!(
+ records.to_string().contains(&SECRET.to_string()),
+ "the superuser's datafeed stopped scoring after a background tick: {records}"
+ );
+}
+
+/// A scoped key configuring ML over an index it **holds** must not be caught
+/// by the new config-time check — the point is to bound the datafeed to its
+/// creator, not to refuse everyone.
+///
+/// It also pins what this change deliberately did **not** touch. The ML
+/// *verbs* (`_score`, `_start`, `_stop`) name no index of their own, so they
+/// fall to the pre-existing cluster rule that a scoped key must name what it
+/// mutates, and are refused — as they were before this change. That is a
+/// usability gap in the scoped-key surface, not a hole: it fails closed, and
+/// widening it is a change to the cluster arm rather than to anything issue
+/// #79 is about. Pinned here so the difference is visible and deliberate.
+#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
+async fn a_scoped_key_still_configures_ml_over_its_own_index() {
+ let (state, _dir) = auth_enabled_state();
+ let app = build_es_compat_router(state);
+ plant_the_brain(&app).await;
+
+ // Same fixture shape, under a name the scoped key will be granted.
+ for (i, (ts, val)) in [
+ ("2026-01-01T00:00:01Z", 1001),
+ ("2026-01-01T00:00:02Z", 1002),
+ ("2026-01-01T00:00:03Z", 1003),
+ ("2026-01-01T00:00:04Z", 1004),
+ ("2026-01-01T00:00:09Z", SECRET),
+ ]
+ .iter()
+ .enumerate()
+ {
+ let refresh = if i == 4 { "?refresh=true" } else { "" };
+ let (status, body) = send(
+ &app,
+ "PUT",
+ &format!("/logs-2026/_doc/d{i}{refresh}"),
+ &admin(),
+ &format!(r#"{{"created_at":"{ts}","valid_at":{val}}}"#),
+ )
+ .await;
+ assert!(status.is_success(), "seeding logs-2026 d{i}: {body}");
+ }
+
+ let (status, minted) = send(
+ &app,
+ "POST",
+ "/_security/api_key",
+ &admin(),
+ r#"{"name":"logs-agent","role_descriptors":{"logs":{"indices":[
+ {"names":["logs-2026"],"privileges":["read","write"]}]}}}"#,
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "minting a scoped key: {minted}");
+ let agent = format!("ApiKey {}", minted["encoded"].as_str().expect("encoded"));
+
+ let (status, body) = send(
+ &app,
+ "PUT",
+ "/_ml/anomaly_detectors/mine",
+ &agent,
+ r#"{"source_index":"logs-2026","time_field":"created_at",
+ "function":"max","field":"valid_at","bucket_span":"1s"}"#,
+ )
+ .await;
+ assert_eq!(
+ status,
+ StatusCode::OK,
+ "a scoped key must still configure ML over an index it holds: {body}"
+ );
+ // …but it still cannot point one at someone else's brain.
+ let (status, body) = send(
+ &app,
+ "PUT",
+ "/_ml/anomaly_detectors/theirs",
+ &agent,
+ DETECTOR_BODY,
+ )
+ .await;
+ assert_eq!(
+ status,
+ StatusCode::FORBIDDEN,
+ "cross-brain ML config: {body}"
+ );
+
+ // The verbs stay where the pre-existing cluster rule leaves them: refused
+ // for a scoped key, because they name nothing. Unchanged by this commit.
+ for (method, uri) in [
+ ("POST", "/_ml/anomaly_detectors/mine/_score"),
+ ("POST", "/_ml/datafeeds/mine-feed/_start"),
+ ] {
+ let (status, body) = send(&app, method, uri, &agent, "{}").await;
+ assert_eq!(
+ status,
+ StatusCode::FORBIDDEN,
+ "{method} {uri} is expected to stay refused for a scoped key \
+ (pre-existing cluster-mutation rule, fails closed): {body}"
+ );
+ }
+
+ // The superuser can still drive the very same job, so the detector the
+ // scoped key configured is real and usable — the gap is the verb, not the
+ // config.
+ let (status, scored) = send(
+ &app,
+ "POST",
+ "/_ml/anomaly_detectors/mine/_score",
+ &admin(),
+ "{}",
+ )
+ .await;
+ assert_eq!(
+ status,
+ StatusCode::OK,
+ "admin _score of the scoped job: {scored}"
+ );
+ assert!(
+ scored.to_string().contains(&SECRET.to_string()),
+ "the job the scoped key configured must score its own index: {scored}"
+ );
+}
diff --git a/engine/crates/xerj-api/tests/scoped_keys_get_intact_responses.rs b/engine/crates/xerj-api/tests/scoped_keys_get_intact_responses.rs
new file mode 100644
index 000000000..01e74338d
--- /dev/null
+++ b/engine/crates/xerj-api/tests/scoped_keys_get_intact_responses.rs
@@ -0,0 +1,407 @@
+//! A scoped key gets **structurally intact** metadata responses (issue #79).
+//!
+//! Enumeration pruning removes index names a caller may not read. The first
+//! cut did it by deleting any object key equal to a known index name, at every
+//! depth of the response. An index name is an arbitrary string, so ordinary
+//! names collide with the structural keys of the very responses being pruned —
+//! and only for a scoped key, i.e. exactly the multi-tenant and Kibana case the
+//! branch exists to serve. A superuser skips pruning entirely and never saw it.
+//!
+//! Every index created here has a perfectly ordinary name that happens to be a
+//! structural key somewhere: `status`, `indices`, `type`, `nodes`, `count`.
+//! None of them is granted to the scoped key, so all of them are legitimately
+//! prunable *as index names* — which is what makes them the sharp test. The
+//! responses must keep their shape anyway.
+//!
+//! The opposite failure is equally a failure, so every test also re-asserts
+//! that bob's brain is still absent from the same response. "Prune nothing"
+//! would pass the structural half and lose the boundary.
+
+use axum::body::Body;
+use axum::http::{Request, StatusCode};
+use http_body_util::BodyExt;
+use serde_json::Value;
+use tower::ServiceExt;
+use xerj_api::{router::build_es_compat_router, state::AppState};
+use xerj_common::{config::Config, metrics::Metrics};
+use xerj_engine::Engine;
+
+const ADMIN_KEY: &str = "admin-secret-key-for-pruning-test";
+
+/// Ordinary index names that are also structural keys in the metadata
+/// responses a client polls. Each one broke a different endpoint.
+const COLLIDING: &[&str] = &["status", "indices", "type", "nodes", "count"];
+
+fn auth_enabled_state() -> (AppState, tempfile::TempDir) {
+ let dir = tempfile::tempdir().expect("tempdir");
+ let mut config = Config::default();
+ config.server.data_dir = dir.path().to_str().unwrap().to_string();
+ config.auth.enabled = true;
+ config.auth.admin_api_key = ADMIN_KEY.to_string();
+ let metrics = Metrics::new().expect("metrics");
+ let engine = Engine::new(config.clone()).expect("engine");
+ (AppState::new(config, engine, metrics), dir)
+}
+
+fn admin() -> String {
+ format!("ApiKey {ADMIN_KEY}")
+}
+
+async fn send(
+ app: &axum::Router,
+ method: &str,
+ uri: &str,
+ auth: &str,
+ body: &str,
+) -> (StatusCode, Value) {
+ let req = Request::builder()
+ .method(method)
+ .uri(uri)
+ .header("content-type", "application/json")
+ .header("authorization", auth)
+ .body(Body::from(body.to_string()))
+ .expect("request");
+ let resp = app.clone().oneshot(req).await.expect("response");
+ let status = resp.status();
+ let bytes = resp.into_body().collect().await.expect("body").to_bytes();
+ (
+ status,
+ serde_json::from_slice(&bytes).unwrap_or(Value::Null),
+ )
+}
+
+async fn send_text(app: &axum::Router, uri: &str, auth: &str) -> (StatusCode, String) {
+ let req = Request::builder()
+ .method("GET")
+ .uri(uri)
+ .header("authorization", auth)
+ .body(Body::empty())
+ .expect("request");
+ let resp = app.clone().oneshot(req).await.expect("response");
+ let status = resp.status();
+ let bytes = resp.into_body().collect().await.expect("body").to_bytes();
+ (status, String::from_utf8_lossy(&bytes).into_owned())
+}
+
+/// A node holding: the scoped key's own index, one brain it must never learn
+/// of, and one ordinary index per colliding structural key.
+async fn seed(app: &axum::Router) -> String {
+ let mut names: Vec<String> = vec!["logs-2026".into(), ".xerj-memory-bob-edges".into()];
+ names.extend(COLLIDING.iter().map(|s| s.to_string()));
+ for name in &names {
+ let (status, body) = send(
+ app,
+ "PUT",
+ &format!("/{name}/_doc/1?refresh=true"),
+ &admin(),
+ r#"{"host":"h1","amount":3}"#,
+ )
+ .await;
+ assert!(status.is_success(), "seeding {name}: {body}");
+ }
+ let (status, minted) = send(
+ app,
+ "POST",
+ "/_security/api_key",
+ &admin(),
+ r#"{"name":"logs-agent","role_descriptors":{"logs":{"indices":[
+ {"names":["logs-2026"],"privileges":["read","write"]}]}}}"#,
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "minting: {minted}");
+ format!("ApiKey {}", minted["encoded"].as_str().expect("encoded"))
+}
+
+/// The boundary half, asserted alongside every structural assertion so
+/// "prune nothing" cannot pass.
+fn must_not_leak_the_brain(what: &str, body: &str) {
+ assert!(
+ !body.contains("bob"),
+ "{what} enumerated a brain the caller may not read: {body}"
+ );
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// `GET /_cluster/health` keeps `status` — the single most-polled field in the
+/// API — with an ordinary index called `status` on the node.
+#[tokio::test]
+async fn cluster_health_keeps_its_status_field() {
+ let (state, _dir) = auth_enabled_state();
+ let app = build_es_compat_router(state);
+ let agent = seed(&app).await;
+
+ let (status, body) = send(&app, "GET", "/_cluster/health", &agent, "").await;
+ assert_eq!(status, StatusCode::OK, "{body}");
+ assert!(
+ body.get("status").is_some(),
+ "_cluster/health lost its `status` field: {body}"
+ );
+ assert!(body.get("number_of_nodes").is_some(), "{body}");
+ assert!(body.get("active_shards").is_some(), "{body}");
+ must_not_leak_the_brain("_cluster/health", &body.to_string());
+
+ // With the per-index breakdown, the breakdown itself is still filtered:
+ // the caller's own index is there and nothing else is.
+ let (status, body) = send(&app, "GET", "/_cluster/health?level=indices", &agent, "").await;
+ assert_eq!(status, StatusCode::OK, "{body}");
+ assert!(body.get("status").is_some(), "level=indices lost `status`");
+ let indices = body["indices"].as_object().expect("indices map");
+ assert!(
+ indices.contains_key("logs-2026"),
+ "own index missing: {body}"
+ );
+ for name in COLLIDING {
+ assert!(
+ !indices.contains_key(*name),
+ "unreadable index `{name}` is still enumerated: {body}"
+ );
+ }
+ must_not_leak_the_brain("_cluster/health?level=indices", &body.to_string());
+}
+
+/// `GET /_cluster/stats` keeps its whole `indices` section with an ordinary
+/// index called `indices` on the node. That section is aggregate counters, not
+/// a map keyed by index name, so there is nothing there to prune.
+#[tokio::test]
+async fn cluster_stats_keeps_its_indices_section() {
+ let (state, _dir) = auth_enabled_state();
+ let app = build_es_compat_router(state);
+ let agent = seed(&app).await;
+
+ let (status, body) = send(&app, "GET", "/_cluster/stats", &agent, "").await;
+ assert_eq!(status, StatusCode::OK, "{body}");
+ assert!(
+ body.get("indices").is_some(),
+ "_cluster/stats lost its entire `indices` section: {body}"
+ );
+ assert!(
+ body["indices"].get("count").is_some(),
+ "_cluster/stats lost `indices.count`: {body}"
+ );
+ assert!(
+ body["indices"].get("docs").is_some(),
+ "_cluster/stats lost `indices.docs`: {body}"
+ );
+ assert!(body.get("status").is_some(), "lost `status`: {body}");
+ assert!(
+ body["nodes"].get("count").is_some(),
+ "_cluster/stats lost `nodes.count`: {body}"
+ );
+ must_not_leak_the_brain("_cluster/stats", &body.to_string());
+}
+
+/// Global `GET /_mapping` keeps every field's `type` with an ordinary index
+/// called `type` on the node. This is the endpoint Kibana reads to build an
+/// index pattern; without `type` every field is undefined.
+#[tokio::test]
+async fn global_mapping_keeps_every_field_type() {
+ let (state, _dir) = auth_enabled_state();
+ let app = build_es_compat_router(state);
+ let agent = seed(&app).await;
+
+ let (status, body) = send(&app, "GET", "/_mapping", &agent, "").await;
+ assert_eq!(status, StatusCode::OK, "{body}");
+
+ let own = body
+ .get("logs-2026")
+ .unwrap_or_else(|| panic!("the caller's own index is missing from _mapping: {body}"));
+ let props = own["mappings"]["properties"]
+ .as_object()
+ .unwrap_or_else(|| panic!("_mapping lost `mappings.properties`: {body}"));
+ assert!(!props.is_empty(), "_mapping returned no fields: {body}");
+ for (field, spec) in props {
+ assert!(
+ spec.get("type").is_some(),
+ "field `{field}` lost its `type` — Kibana cannot build a pattern \
+ from this: {body}"
+ );
+ }
+
+ // …and the enumeration boundary still holds.
+ for name in COLLIDING {
+ assert!(
+ body.get(*name).is_none(),
+ "unreadable index `{name}` is still enumerated by _mapping: {body}"
+ );
+ }
+ must_not_leak_the_brain("_mapping", &body.to_string());
+}
+
+/// `_cat/indices` keeps every row whose index column the caller can read, and
+/// drops exactly the rows it cannot — not every row that happens to contain a
+/// colliding token.
+#[tokio::test]
+async fn cat_indices_keeps_the_rows_it_should() {
+ let (state, _dir) = auth_enabled_state();
+ let app = build_es_compat_router(state);
+ let agent = seed(&app).await;
+
+ let (status, table) = send_text(&app, "/_cat/indices", &agent).await;
+ assert_eq!(status, StatusCode::OK, "{table}");
+ assert!(
+ table.contains("logs-2026"),
+ "_cat/indices lost the caller's own index: {table}"
+ );
+ assert_eq!(
+ table.lines().filter(|l| !l.trim().is_empty()).count(),
+ 1,
+ "_cat/indices should list exactly the one readable index: {table}"
+ );
+ must_not_leak_the_brain("_cat/indices", &table);
+
+ // `_cat/health` has no index column at all, so it must survive whole even
+ // though `status` and `count` are unreadable index names on this node.
+ let (status, health) = send_text(&app, "/_cat/health", &agent).await;
+ assert_eq!(status, StatusCode::OK);
+ assert!(
+ !health.trim().is_empty(),
+ "_cat/health was emptied by index-name matching: {health:?}"
+ );
+}
+
+/// `GET /_settings` and `GET /_alias` are the same top-level index-keyed shape
+/// and must behave the same way.
+#[tokio::test]
+async fn settings_and_alias_listings_stay_intact() {
+ let (state, _dir) = auth_enabled_state();
+ let app = build_es_compat_router(state);
+ let agent = seed(&app).await;
+
+ for uri in ["/_settings", "/_alias"] {
+ let (status, body) = send(&app, "GET", uri, &agent, "").await;
+ assert_eq!(status, StatusCode::OK, "GET {uri}: {body}");
+ assert!(
+ body.get("logs-2026").is_some(),
+ "GET {uri} lost the caller's own index: {body}"
+ );
+ for name in COLLIDING {
+ assert!(
+ body.get(*name).is_none(),
+ "GET {uri} still enumerates `{name}`: {body}"
+ );
+ }
+ must_not_leak_the_brain(uri, &body.to_string());
+ }
+
+ // `_settings` keeps the settings object under the index it did return.
+ let (_, body) = send(&app, "GET", "/_settings", &agent, "").await;
+ assert!(
+ body["logs-2026"].get("settings").is_some(),
+ "_settings lost its `settings` block: {body}"
+ );
+}
+
+/// `_field_caps` names indices in two places — the top-level list and one per
+/// field/type entry — and neither may take the `fields` map down with it.
+#[tokio::test]
+async fn field_caps_keeps_its_fields_map() {
+ let (state, _dir) = auth_enabled_state();
+ let app = build_es_compat_router(state);
+ let agent = seed(&app).await;
+
+ let (status, body) = send(&app, "GET", "/_field_caps?fields=*", &agent, "").await;
+ assert_eq!(status, StatusCode::OK, "{body}");
+ assert!(
+ body.get("fields").is_some(),
+ "_field_caps lost its `fields` map: {body}"
+ );
+ let listed = body["indices"].as_array().expect("indices array");
+ assert!(
+ listed.iter().any(|v| v == "logs-2026"),
+ "_field_caps lost the caller's own index: {body}"
+ );
+ for name in COLLIDING {
+ assert!(
+ !listed.iter().any(|v| v == name),
+ "_field_caps still enumerates `{name}`: {body}"
+ );
+ }
+ must_not_leak_the_brain("_field_caps", &body.to_string());
+}
+
+/// FINDING D. An alias `add` whose index is a wildcard that matches nothing
+/// used to answer `acknowledged: true` and leave an alias pointing at the
+/// pattern TEXT — a write that never happened, reported as one that did, on
+/// the reserved namespace of all places.
+#[tokio::test]
+async fn a_wildcard_alias_that_matches_nothing_is_refused() {
+ let (state, _dir) = auth_enabled_state();
+ let app = build_es_compat_router(state);
+ seed(&app).await;
+
+ // `POST /_aliases` with an `add` naming a reserved wildcard.
+ let (status, body) = send(
+ &app,
+ "POST",
+ "/_aliases",
+ &admin(),
+ r#"{"actions":[{"add":{"index":".xerj-memory-nope-*","alias":"X"}}]}"#,
+ )
+ .await;
+ assert_ne!(
+ status,
+ StatusCode::OK,
+ "a wildcard alias add that matched nothing was acknowledged: {body}"
+ );
+
+ // `PUT /{wildcard}/_alias/{name}`, the other spelling.
+ let (status, body) = send(&app, "PUT", "/.xerj-memory-nope-*/_alias/Y", &admin(), "{}").await;
+ assert_ne!(
+ status,
+ StatusCode::OK,
+ "PUT /{{wildcard}}/_alias was acknowledged: {body}"
+ );
+
+ // Neither junk alias exists.
+ let (_, aliases) = send(&app, "GET", "/_aliases", &admin(), "").await;
+ let text = aliases.to_string();
+ assert!(!text.contains("\"X\""), "junk alias X was created: {text}");
+ assert!(!text.contains("\"Y\""), "junk alias Y was created: {text}");
+
+ // A wildcard that DOES match still works, on both spellings — the fix is
+ // "refuse a write that did not happen", not "refuse wildcards".
+ let (status, body) = send(
+ &app,
+ "POST",
+ "/_aliases",
+ &admin(),
+ r#"{"actions":[{"add":{"index":"logs-*","alias":"live"}}]}"#,
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "a matching wildcard add: {body}");
+ let (status, body) = send(&app, "PUT", "/logs-*/_alias/live2", &admin(), "{}").await;
+ assert_eq!(status, StatusCode::OK, "a matching wildcard PUT: {body}");
+
+ // Both resolve to the real index, not to the pattern text.
+ let (_, resolved) = send(&app, "GET", "/live/_search", &admin(), "{}").await;
+ assert!(
+ resolved.get("hits").is_some(),
+ "the alias does not resolve to a real index: {resolved}"
+ );
+ let (_, aliases) = send(&app, "GET", "/_aliases", &admin(), "").await;
+ assert!(
+ aliases["logs-2026"]["aliases"].get("live").is_some(),
+ "the alias was not attached to the resolved index: {aliases}"
+ );
+ assert!(
+ aliases["logs-2026"]["aliases"].get("live2").is_some(),
+ "PUT /{{pattern}}/_alias did not attach to the resolved index: {aliases}"
+ );
+
+ // A concrete name is unchanged: ES lets an alias be attached before the
+ // index exists, and clients rely on that ordering.
+ let (status, body) = send(
+ &app,
+ "POST",
+ "/_aliases",
+ &admin(),
+ r#"{"actions":[{"add":{"index":"not-created-yet","alias":"early"}}]}"#,
+ )
+ .await;
+ assert_eq!(
+ status,
+ StatusCode::OK,
+ "a concrete-name alias add must still be accepted: {body}"
+ );
+}
diff --git a/engine/crates/xerj-api/tests/script_limits_http.rs b/engine/crates/xerj-api/tests/script_limits_http.rs
index 2b93f6319..a94ca60a7 100644
--- a/engine/crates/xerj-api/tests/script_limits_http.rs
+++ b/engine/crates/xerj-api/tests/script_limits_http.rs
@@ -17,6 +17,69 @@ use axum::http::{Request, StatusCode};
use serde_json::{json, Value};
use tower::ServiceExt;
+/// Stack size for every thread these tests evaluate a script on, replacing the
+/// 2 MiB `std`/`libtest` default that `#[tokio::test]` would have given them.
+///
+/// `xerj_engine::painless::MAX_CALL_DEPTH` is a **stack** budget: 32 is
+/// documented as measured in a *release* build, where the adversarial worst
+/// case costs 1.20 MiB of a 2 MiB worker stack and the rest is left for the
+/// axum/search frames underneath. A debug build's frames are several times
+/// larger, so that same recursion lands much closer to the edge here than the
+/// constant's own arithmetic implies — and `_delete_by_query` is the deepest
+/// of these paths, since it runs a search inside the by-query wrapper.
+///
+/// Adding the issue-#79 authorization layer to the router pushed the debug
+/// profile of that one path over 2 MiB. The cost is the *layer*, not the
+/// authorization: these tests configure no admin key, so the caller is a
+/// superuser and `authz_middleware` returns on its second statement. Measured
+/// on this branch by bisecting `RUST_MIN_STACK`,
+/// `delete_by_query_refuses_a_selection_a_script_limit_truncated` aborts at
+/// 2,097,152 bytes and passes at 2,359,296 — roughly 200 KiB of debug frames
+/// for one more `tower` layer. The same test passes on `main` at the default,
+/// so the margin it had there was under 200 KiB.
+///
+/// The release binary was checked directly rather than inferred: the same
+/// recursion with 10, 50 and 90 nested blocks in the closure body (90 being
+/// the documented worst case) answers `400 script_exception` on a running
+/// release node, which stays up. So the production margin still holds and the
+/// budget in `painless.rs` is unchanged.
+///
+/// Sizing the stack here keeps these tests measuring what they are for — that
+/// the depth guard trips and the refusal reaches the caller — instead of
+/// measuring how large `rustc -C opt-level=0` makes an async frame.
+const WORKER_STACK_BYTES: usize = 4 * 1024 * 1024;
+
+/// Drive `fut` to completion with every stack involved sized to
+/// [`WORKER_STACK_BYTES`].
+///
+/// Both threads have to be sized, because the two shapes here evaluate the
+/// script in different places: `_search` hands the work to a task
+/// (`search_impl` spawns it) so it lands on a **runtime worker**, while
+/// `_delete_by_query` with the default `wait_for_completion=true` awaits
+/// `run_delete_by_query` inline, so it lands on the thread that called
+/// `block_on` — which under `libtest` is the per-test thread, and that one
+/// takes its size from `RUST_MIN_STACK` (2 MiB by default), not from the
+/// runtime builder.
+fn block_on_sized<F>(fut: F)
+where
+ F: std::future::Future<Output = ()> + Send + 'static,
+{
+ std::thread::Builder::new()
+ .stack_size(WORKER_STACK_BYTES)
+ .spawn(move || {
+ tokio::runtime::Builder::new_multi_thread()
+ .worker_threads(4)
+ .thread_stack_size(WORKER_STACK_BYTES)
+ .enable_all()
+ .build()
+ .expect("test runtime")
+ .block_on(fut)
+ })
+ .expect("spawn test thread")
+ .join()
+ .expect("test thread panicked");
+}
+
/// Self-application recursion that runs past `MAX_CALL_DEPTH` (32), with the
/// recursive call wrapped in nested blocks — the shape from the PR #88
/// process-abort repro. Short enough to clear the static source-size and
@@ -75,8 +138,12 @@ async fn post(app: &axum::Router, path: &str, body: Value) -> (StatusCode, Value
/// The headline of #97. `apply_function_score` returns a bare `f32`, so it
/// mapped the call-depth error to `0.0` and the request succeeded with a score
/// that no script produced.
-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
-async fn script_score_past_the_call_depth_limit_is_an_error_not_a_zero_score() {
+#[test]
+fn script_score_past_the_call_depth_limit_is_an_error_not_a_zero_score() {
+ block_on_sized(script_score_past_the_call_depth_limit_is_an_error_not_a_zero_score_inner());
+}
+
+async fn script_score_past_the_call_depth_limit_is_an_error_not_a_zero_score_inner() {
let (app, _dir) = seeded_app().await;
let (status, body) = post(
&app,
@@ -115,8 +182,12 @@ async fn script_score_past_the_call_depth_limit_is_an_error_not_a_zero_score() {
/// `if let Ok(pv)` dropped a limit trip on the floor and the field just wasn't
/// in the response — indistinguishable from a script that legitimately
/// returned nothing.
-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
-async fn script_field_past_the_call_depth_limit_is_an_error_not_a_missing_field() {
+#[test]
+fn script_field_past_the_call_depth_limit_is_an_error_not_a_missing_field() {
+ block_on_sized(script_field_past_the_call_depth_limit_is_an_error_not_a_missing_field_inner());
+}
+
+async fn script_field_past_the_call_depth_limit_is_an_error_not_a_missing_field_inner() {
let (app, _dir) = seeded_app().await;
let (status, body) = post(
&app,
@@ -141,8 +212,14 @@ async fn script_field_past_the_call_depth_limit_is_an_error_not_a_missing_field(
/// A script-bucketed terms agg mapped a limit trip to "no buckets" — an empty
/// aggregation that reads as a legitimate answer.
-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
-async fn script_bucketed_agg_past_the_call_depth_limit_is_an_error_not_empty_buckets() {
+#[test]
+fn script_bucketed_agg_past_the_call_depth_limit_is_an_error_not_empty_buckets() {
+ block_on_sized(
+ script_bucketed_agg_past_the_call_depth_limit_is_an_error_not_empty_buckets_inner(),
+ );
+}
+
+async fn script_bucketed_agg_past_the_call_depth_limit_is_an_error_not_empty_buckets_inner() {
let (app, _dir) = seeded_app().await;
let (status, body) = post(
&app,
@@ -167,8 +244,12 @@ async fn script_bucketed_agg_past_the_call_depth_limit_is_an_error_not_empty_buc
/// `_delete_by_query` selects with the same matching machinery. A fail-closed
/// script trip there means deleting an arbitrary subset and reporting a
/// completed run — destructive as well as wrong.
-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
-async fn delete_by_query_refuses_a_selection_a_script_limit_truncated() {
+#[test]
+fn delete_by_query_refuses_a_selection_a_script_limit_truncated() {
+ block_on_sized(delete_by_query_refuses_a_selection_a_script_limit_truncated_inner());
+}
+
+async fn delete_by_query_refuses_a_selection_a_script_limit_truncated_inner() {
let (app, _dir) = seeded_app().await;
let (_, body) = post(
&app,
@@ -201,8 +282,12 @@ async fn delete_by_query_refuses_a_selection_a_script_limit_truncated() {
/// support is NOT a resource limit. It must keep degrading to a neutral score
/// on a 200, exactly as before, or every out-of-subset script in the ES-compat
/// surface becomes a spurious 400.
-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
-async fn unsupported_script_syntax_still_degrades_quietly() {
+#[test]
+fn unsupported_script_syntax_still_degrades_quietly() {
+ block_on_sized(unsupported_script_syntax_still_degrades_quietly_inner());
+}
+
+async fn unsupported_script_syntax_still_degrades_quietly_inner() {
let (app, _dir) = seeded_app().await;
for source in [
"someUnsupportedThing(1,2,3)",
@@ -234,8 +319,12 @@ async fn unsupported_script_syntax_still_degrades_quietly() {
/// A search that trips a limit must not leave the response cache poisoned with
/// its degraded scores, and the fault must not be re-reported against the next
/// caller of a different query.
-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
-async fn a_faulted_search_is_neither_cached_nor_replayed() {
+#[test]
+fn a_faulted_search_is_neither_cached_nor_replayed() {
+ block_on_sized(a_faulted_search_is_neither_cached_nor_replayed_inner());
+}
+
+async fn a_faulted_search_is_neither_cached_nor_replayed_inner() {
let (app, _dir) = seeded_app().await;
let bad = json!({
"query": {
diff --git a/engine/crates/xerj-api/tests/script_limits_remaining_surfaces_http.rs b/engine/crates/xerj-api/tests/script_limits_remaining_surfaces_http.rs
index 796dbad13..5b9e2ab9a 100644
--- a/engine/crates/xerj-api/tests/script_limits_remaining_surfaces_http.rs
+++ b/engine/crates/xerj-api/tests/script_limits_remaining_surfaces_http.rs
@@ -25,6 +25,40 @@ use axum::http::{Request, StatusCode};
use serde_json::{json, Value};
use tower::ServiceExt;
+/// Stack size for every thread these tests evaluate a script on, replacing the
+/// 2 MiB `std`/`libtest` default. Same reasoning, and the same measurements, as
+/// `tests/script_limits_http.rs` — see the comment on its `WORKER_STACK_BYTES`.
+/// In short: `MAX_CALL_DEPTH` is a stack budget measured in a *release* build,
+/// a debug build's frames are several times larger, and the issue-#79
+/// authorization layer spends what little margin a debug build had left.
+/// `delete_by_query_script_refusal_carries_its_own_http_status` is the one that
+/// aborted here; the release binary answers `400 script_exception` and stays
+/// up, including at the adversarial worst case, so the budget itself is sound.
+const WORKER_STACK_BYTES: usize = 4 * 1024 * 1024;
+
+/// Drive `fut` with both the runtime workers and the calling thread sized to
+/// [`WORKER_STACK_BYTES`]: `_search`-shaped surfaces evaluate the script on a
+/// spawned task, `_delete_by_query` awaits it inline on the caller.
+fn block_on_sized<F>(fut: F)
+where
+ F: std::future::Future<Output = ()> + Send + 'static,
+{
+ std::thread::Builder::new()
+ .stack_size(WORKER_STACK_BYTES)
+ .spawn(move || {
+ tokio::runtime::Builder::new_multi_thread()
+ .worker_threads(4)
+ .thread_stack_size(WORKER_STACK_BYTES)
+ .enable_all()
+ .build()
+ .expect("test runtime")
+ .block_on(fut)
+ })
+ .expect("spawn test thread")
+ .join()
+ .expect("test thread panicked");
+}
+
/// Self-application recursion that runs past `MAX_CALL_DEPTH` (32), with the
/// recursive call wrapped in nested blocks. Short enough to clear the static
/// source-size and parse-depth guards, so the limit can only trip
@@ -112,8 +146,12 @@ async fn send(
/// call-depth limit degrades every hit to 0.0 — and then `_rank_eval`
/// publishes a `metric_score` computed over that degraded ranking as a
/// successful measurement.
-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
-async fn rank_eval_past_the_call_depth_limit_is_an_error_not_a_metric_score() {
+#[test]
+fn rank_eval_past_the_call_depth_limit_is_an_error_not_a_metric_score() {
+ block_on_sized(rank_eval_past_the_call_depth_limit_is_an_error_not_a_metric_score_inner());
+}
+
+async fn rank_eval_past_the_call_depth_limit_is_an_error_not_a_metric_score_inner() {
let (app, _dir) = seeded_app().await;
let (status, body) = post(
&app,
@@ -169,8 +207,12 @@ async fn rank_eval_past_the_call_depth_limit_is_an_error_not_a_metric_score() {
/// A body where only ONE of several requests carries a limit-tripping script
/// must still measure the others. Failing all of them was the first attempt.
-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
-async fn rank_eval_failure_is_scoped_to_the_offending_request() {
+#[test]
+fn rank_eval_failure_is_scoped_to_the_offending_request() {
+ block_on_sized(rank_eval_failure_is_scoped_to_the_offending_request_inner());
+}
+
+async fn rank_eval_failure_is_scoped_to_the_offending_request_inner() {
let (app, _dir) = seeded_app().await;
let (status, body) = post(
&app,
@@ -211,8 +253,12 @@ async fn rank_eval_failure_is_scoped_to_the_offending_request() {
/// A limit trip in the *matching* path truncates the ranking instead of
/// skewing it: fewer retrieved ids, so precision/recall are computed over a
/// selection the script silently cut short.
-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
-async fn rank_eval_refuses_a_ranking_a_script_limit_truncated() {
+#[test]
+fn rank_eval_refuses_a_ranking_a_script_limit_truncated() {
+ block_on_sized(rank_eval_refuses_a_ranking_a_script_limit_truncated_inner());
+}
+
+async fn rank_eval_refuses_a_ranking_a_script_limit_truncated_inner() {
let (app, _dir) = seeded_app().await;
let (status, body) = post(
&app,
@@ -243,8 +289,12 @@ async fn rank_eval_refuses_a_ranking_a_script_limit_truncated() {
/// running the query with an `ids` filter. A fail-closed script makes the
/// answer `matched: false`, which is indistinguishable from a document that
/// genuinely does not match.
-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
-async fn explain_past_the_call_depth_limit_is_an_error_not_an_unmatched_doc() {
+#[test]
+fn explain_past_the_call_depth_limit_is_an_error_not_an_unmatched_doc() {
+ block_on_sized(explain_past_the_call_depth_limit_is_an_error_not_an_unmatched_doc_inner());
+}
+
+async fn explain_past_the_call_depth_limit_is_an_error_not_an_unmatched_doc_inner() {
let (app, _dir) = seeded_app().await;
let (status, body) = post(
&app,
@@ -287,8 +337,12 @@ async fn explain_past_the_call_depth_limit_is_an_error_not_an_unmatched_doc() {
/// The second defect in #123. `run_delete_by_query` refuses correctly, but the
/// handler wrapped the refusal in `Json(..)` alone — HTTP 200 carrying
/// `{"error": …, "status": 400}`.
-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
-async fn delete_by_query_script_refusal_carries_its_own_http_status() {
+#[test]
+fn delete_by_query_script_refusal_carries_its_own_http_status() {
+ block_on_sized(delete_by_query_script_refusal_carries_its_own_http_status_inner());
+}
+
+async fn delete_by_query_script_refusal_carries_its_own_http_status_inner() {
let (app, _dir) = seeded_app().await;
let (status, body) = post(
&app,
@@ -316,8 +370,12 @@ async fn delete_by_query_script_refusal_carries_its_own_http_status() {
}
/// Same defect on `_update_by_query`.
-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
-async fn update_by_query_script_refusal_carries_its_own_http_status() {
+#[test]
+fn update_by_query_script_refusal_carries_its_own_http_status() {
+ block_on_sized(update_by_query_script_refusal_carries_its_own_http_status_inner());
+}
+
+async fn update_by_query_script_refusal_carries_its_own_http_status_inner() {
let (app, _dir) = seeded_app().await;
let (status, body) = post(
&app,
@@ -340,8 +398,12 @@ async fn update_by_query_script_refusal_carries_its_own_http_status() {
/// A successful `_delete_by_query` must stay a 200 — the status now comes from
/// the body, so this pins the non-error half of that mapping.
-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
-async fn delete_by_query_without_a_script_is_still_a_200() {
+#[test]
+fn delete_by_query_without_a_script_is_still_a_200() {
+ block_on_sized(delete_by_query_without_a_script_is_still_a_200_inner());
+}
+
+async fn delete_by_query_without_a_script_is_still_a_200_inner() {
let (app, _dir) = seeded_app().await;
let (status, body) = post(
&app,
@@ -360,8 +422,12 @@ async fn delete_by_query_without_a_script_is_still_a_200() {
/// wrong summary and reports `acknowledged: true` over it — the same defect as
/// an under-delete, except the wrong numbers are now persisted in an index
/// that later queries will read as fact.
-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
-async fn pivot_transform_refuses_a_source_a_script_limit_truncated() {
+#[test]
+fn pivot_transform_refuses_a_source_a_script_limit_truncated() {
+ block_on_sized(pivot_transform_refuses_a_source_a_script_limit_truncated_inner());
+}
+
+async fn pivot_transform_refuses_a_source_a_script_limit_truncated_inner() {
let (app, _dir) = seeded_app().await;
let (put_status, put_body) = send(
@@ -399,8 +465,12 @@ async fn pivot_transform_refuses_a_source_a_script_limit_truncated() {
/// The counterpart, restated for the new surfaces: a script our interpreter
/// does not support is NOT a resource limit and must keep degrading quietly.
-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
-async fn unsupported_script_syntax_still_degrades_quietly_on_the_new_surfaces() {
+#[test]
+fn unsupported_script_syntax_still_degrades_quietly_on_the_new_surfaces() {
+ block_on_sized(unsupported_script_syntax_still_degrades_quietly_on_the_new_surfaces_inner());
+}
+
+async fn unsupported_script_syntax_still_degrades_quietly_on_the_new_surfaces_inner() {
let (app, _dir) = seeded_app().await;
let unsupported = json!({
"function_score": {
diff --git a/engine/crates/xerj-engine/src/engine.rs b/engine/crates/xerj-engine/src/engine.rs
index 23f43d27e..39eccddc9 100644
--- a/engine/crates/xerj-engine/src/engine.rs
+++ b/engine/crates/xerj-engine/src/engine.rs
@@ -173,6 +173,22 @@ pub struct ApiKeyRecord {
pub expiration_ms: Option<u64>,
/// Set once the key has been invalidated (revoked).
pub invalidated: bool,
+ /// Index-scoped grants parsed from the `role_descriptors` supplied at
+ /// creation (`crate::rbac::roles_from_role_descriptors`).
+ ///
+ /// **Empty means "no grant", not "all grants."** A key with no roles is an
+ /// *unscoped* legacy credential: it keeps the historical superuser reach
+ /// over the general ES-compat surface, but it holds no privilege at all on
+ /// the reserved `.xerj-memory-*` namespace (agent-memory namespaces and
+ /// second brains), which is the fail-closed half of the per-brain
+ /// authorization model. A key with a non-empty list is *scoped* and may
+ /// touch only what these roles allow.
+ ///
+ /// `#[serde(default)]` so an `api_keys.json` written before this field
+ /// existed still loads — those keys deserialize as unscoped, which is the
+ /// safe reading of "was minted when nothing was enforced".
+ #[serde(default)]
+ pub roles: Vec<crate::rbac::Role>,
}
/// Atomically write a **secret** file (API-key store) with owner-only (0600)
@@ -762,6 +778,17 @@ impl Engine {
) -> Result<()> {
let index_name = IndexName::new(name).map_err(EngineError::Common)?;
+ // Per-index authorization backstop (issue #79): a caller that may not
+ // see this name must not be able to bring it into existence either —
+ // otherwise it could squat a brain name it cannot read. Reported as a
+ // create failure rather than a distinct refusal so the two are
+ // indistinguishable. See `crate::index_guard`.
+ if !crate::index_guard::visible(name) {
+ return Err(EngineError::Common(
+ xerj_common::XerjError::index_not_found(name),
+ ));
+ }
+
if self.indices.contains_key(name) {
return Err(EngineError::Common(
xerj_common::XerjError::index_already_exists(name),
@@ -1077,6 +1104,14 @@ impl Engine {
/// semantics) and clears the `closed_indices` flag so the name is
/// truly gone when another test recreates it.
pub async fn delete_index(&self, name: &str) -> Result<()> {
+ // Per-index authorization backstop (issue #79) — "destroy the brain"
+ // is the loudest door, so it is checked before the index is removed
+ // from the map. See `crate::index_guard`.
+ if !crate::index_guard::visible(name) {
+ return Err(EngineError::Common(
+ xerj_common::XerjError::index_not_found(name),
+ ));
+ }
let idx =
self.indices.remove(name).map(|(_, v)| v).ok_or_else(|| {
EngineError::Common(xerj_common::XerjError::index_not_found(name))
@@ -1114,10 +1149,24 @@ impl Engine {
/// Get a reference to an index by name, resolving aliases first.
/// If the name is an alias pointing to multiple indices, returns the first one.
+ ///
+ /// This is the single funnel every handler goes through to reach index
+ /// data, whether it read the name from the URL path, from a `_bulk` action
+ /// line, from an `_mget` `docs[]._index`, from a `terms` lookup or from the
+ /// table name inside an `_sql` statement — which is exactly why the
+ /// per-index authorization backstop is here (issue #79, see
+ /// [`crate::index_guard`]). The check runs **after** alias resolution, on
+ /// the concrete name, so pointing an alias at someone else's index cannot
+ /// launder access to it.
pub fn get_index(&self, name: &str) -> Result<Arc<Index>> {
// Check if name is an alias — if so, resolve to the first backing index.
if let Some(aliased) = self.aliases.get(name) {
if let Some(real_name) = aliased.first() {
+ if !crate::index_guard::visible(real_name) {
+ return Err(EngineError::Common(
+ xerj_common::XerjError::index_not_found(name),
+ ));
+ }
return self
.indices
.get(real_name.as_str())
@@ -1127,6 +1176,11 @@ impl Engine {
});
}
}
+ if !crate::index_guard::visible(name) {
+ return Err(EngineError::Common(
+ xerj_common::XerjError::index_not_found(name),
+ ));
+ }
self.indices
.get(name)
.map(|r| Arc::clone(r.value()))
@@ -1134,7 +1188,19 @@ impl Engine {
}
/// Return an index by name, creating it if it doesn't exist (ES behaviour).
+ ///
+ /// The visibility check is repeated here rather than left to
+ /// [`Engine::get_index`]: that call reports "not found" for a denied name,
+ /// and "not found" is precisely the branch this function answers by
+ /// **creating** the index. Without its own check, auto-create would be the
+ /// bypass (`create_index` catches it too — belt and braces on the write
+ /// door).
pub fn get_or_create_index(&self, name: &str) -> Result<Arc<Index>> {
+ if !crate::index_guard::visible(name) {
+ return Err(EngineError::Common(
+ xerj_common::XerjError::index_not_found(name),
+ ));
+ }
if let Ok(idx) = self.get_index(name) {
return Ok(idx);
}
@@ -1176,8 +1242,17 @@ impl Engine {
/// Used by PIT expansion and other handlers that need to iterate
/// the live index list without paying for the `list_indices`
/// snapshot or `get_settings()` call.
+ /// Filtered to the current request's visible set (issue #79) — this and
+ /// [`Engine::list_indices`] are where `*`, `_all` and `logs-*` are turned
+ /// into concrete names, so filtering here is what makes a wildcard resolve
+ /// to "the indices you may see" instead of having to be refused outright.
+ /// Unfiltered outside a request; see [`crate::index_guard`].
pub fn index_name_list(&self) -> Vec<String> {
- self.indices.iter().map(|e| e.key().clone()).collect()
+ self.indices
+ .iter()
+ .map(|e| e.key().clone())
+ .filter(|n| crate::index_guard::visible(n))
+ .collect()
}
/// Sum the internal query-result cache hit/miss counters across every open
@@ -1194,9 +1269,18 @@ impl Engine {
(hits, misses)
}
+ /// Filtered to the current request's visible set — see
+ /// [`Engine::index_name_list`] and [`crate::index_guard`]. Every metadata
+ /// listing (`_cat/indices`, `_mapping`, `_alias`, `_resolve/index`,
+ /// `/v1/dashboard/summary`, cluster health) enumerates through here, so a
+ /// scoped credential gets a filtered view of the cluster rather than a
+ /// blanket refusal.
pub async fn list_indices(&self) -> Vec<IndexInfo> {
let mut list = Vec::new();
for entry in self.indices.iter() {
+ if !crate::index_guard::visible(entry.key()) {
+ continue;
+ }
let stats = entry.value().stats().await;
list.push(IndexInfo {
name: stats.name,
diff --git a/engine/crates/xerj-engine/src/index_guard.rs b/engine/crates/xerj-engine/src/index_guard.rs
new file mode 100644
index 000000000..53c3b7888
--- /dev/null
+++ b/engine/crates/xerj-engine/src/index_guard.rs
@@ -0,0 +1,281 @@
+//! Request-scoped index **visibility** — the engine-side half of the per-index
+//! authorization boundary (issue #79).
+//!
+//! ## Why this lives in the engine and not only in the API middleware
+//!
+//! The first cut of per-brain authorization decided against the index named in
+//! the **URL path**. Several handlers take the index they actually operate on
+//! from the **request body** — `_msearch` header lines, `_bulk` action lines,
+//! `_mget` `docs[]._index`, `_aliases` actions, `_reindex` `source`/`dest`, a
+//! `terms` lookup buried in a query, the table name inside `_sql` — so a
+//! caller could name one index in the path, be authorized against it, and have
+//! the handler touch a different one. Four of those were proven live.
+//!
+//! Patching each handler is how that class of bug comes back: the next handler
+//! that resolves a name from a body is written by someone who does not know the
+//! list. So the check is placed where **every** path converges instead — the
+//! two engine functions that turn a name into an [`crate::index::Index`]
+//! ([`crate::Engine::get_index`] and [`crate::Engine::get_or_create_index`]),
+//! the two that create and destroy one, and the two that enumerate them.
+//! Nothing in the process reaches index data without going through one of
+//! those, so a handler cannot forget this check — it does not call it.
+//!
+//! ## Shape
+//!
+//! The API layer installs an [`IndexVisibility`] for the duration of a request
+//! (a `tokio` task-local, set by `xerj_api::authz::authz_middleware` around the
+//! inner service call, so every future awaited while handling that request sees
+//! it). Code running **outside** a request — startup index discovery, the
+//! background flush/merge timers, WAL replay, snapshot restore — has no
+//! task-local set and is unrestricted, which is what
+//! [`visible`]'s `unwrap_or(true)` means: absent guard = engine-internal work,
+//! not "permission granted to a caller".
+//!
+//! ## Detached work loses the guard, and that was a real hole
+//!
+//! A `tokio` task-local lives on the task, so anything a request *detaches*
+//! from itself — `tokio::spawn`, `spawn_blocking`, a `rayon` closure that
+//! lands on a pool worker — runs with no guard installed and is therefore
+//! unrestricted.
+//!
+//! The first cut of this module recorded that shape as a residual risk and
+//! claimed "no request path in the tree does that today". That was wrong. The
+//! ML datafeed does: `POST /_ml/datafeeds/{id}/_start` spawned a detached
+//! scorer that re-read its source index every `frequency` seconds, so a
+//! principal with no access to `.xerj-memory-bob-edges` got an empty result
+//! set from the synchronous start pass (correctly denied, inside the request's
+//! scope) and the brain's field values a few seconds later, from the tick.
+//!
+//! So detaching now has an explicit contract instead of a footnote:
+//!
+//! - [`current`] captures the guard the request is running under.
+//! - [`scoped`] re-installs it inside the spawned future.
+//!
+//! Any future `tokio::spawn` that can reach [`crate::Engine::get_index`] must
+//! carry the rule across with those two, and any that cannot must say why in a
+//! comment at the spawn site. `spawn_datafeed_task` in `xerj-api` is the
+//! worked example.
+//!
+//! ### The audit
+//!
+//! Every non-test `tokio::spawn` / `spawn_blocking` in the workspace, and why
+//! it is or is not a door. Only the first row was a hole.
+//!
+//! | Site | Verdict |
+//! |---|---|
+//! | `xerj-api::es_compat::spawn_datafeed_task` | **carries the rule** — the one detached path that resolves an index name |
+//! | `es_compat::search_impl`'s search task | owns one already-authorized `Index`; `terms`/`lookup` targets are resolved earlier, on the request's task |
+//! | `es_compat::delete_by_query` / `update_by_query` (`wait_for_completion=false`) | same: one already-authorized `Index` handle, no `Engine` |
+//! | `Engine::spawn_pit_sweeper`, `spawn_search_context_sweeper` | started by `Engine::new`, capture only the PIT/scroll maps, touch no index |
+//! | `Index`'s flush/merge/warm tasks (`index.rs`, `write_publication.rs`) | methods **on** an already-resolved `Index`; the name has already been through the funnel |
+//! | `xerj-server::main`'s listeners, metrics loop, autoindex/brain CLIs | startup, outside any request; unrestricted by design (see `visible`) |
+//! | `xerj-cluster` transport/replication/coordinator | peer-to-peer, authorized by `xerj_cluster::auth`, not by a caller's principal |
+//! | `xerj-ai::embedder`, `xerj-storage` cache/backend/merge | no `Engine`, no index names |
+//! | `xerj-api::binary_protocol::serve` | not wired to any listener in `xerj-server`; module is unreachable at runtime |
+//! | `xerj-server::grpc` | its spawns are `#[cfg(test)]`; the live listener authorizes per call with `Principal::allows_index` |
+//!
+//! `rayon` does **not** inherit the task-local — the first cut claimed it did.
+//! `visible` reads a thread-local that `tokio` maintains only while it polls
+//! the owning task, and a rayon worker is an ordinary OS thread that `tokio`
+//! never touches, so a closure that lands on one sees an absent guard.
+//! `absent_guard_on_a_rayon_worker` below pins that. It is not a live hole
+//! because no `rayon` closure in the tree resolves an index name: the bulk
+//! ingest fan-out (`bulk::process_bulk_with_opts`) parallelises NDJSON
+//! *parsing* only, and every `get_or_create_index` in that function runs
+//! sequentially on the request's own task, after the parse collects.
+//!
+//! ## Denial shape: "not found", not "forbidden"
+//!
+//! A denied name reports the ordinary
+//! [`xerj_common::XerjError::index_not_found`], for two reasons:
+//!
+//! 1. **No enumeration.** A caller cannot tell a brain it may not read from a
+//! brain that does not exist, so it cannot map the node by probing.
+//! 2. **Fan-out keeps working.** `POST /_search`, `_cat/indices`, `_mapping`,
+//! a `logs-*` wildcard — all of them enumerate and skip what is missing.
+//! Filtering the enumeration ([`crate::Engine::list_indices`],
+//! [`crate::Engine::index_name_list`]) plus a not-found on the rest gives
+//! "you see exactly your own indices" for free, instead of the blanket 403
+//! that made global verbs unusable.
+//!
+//! The precise privilege (read vs write vs manage) and the ES-shaped 403 are
+//! still the API middleware's job. This is the backstop that makes
+//! *forgetting* impossible, not a replacement for the front-line decision.
+
+use std::sync::Arc;
+
+/// Decides whether the principal behind the current request may see an index
+/// at all. Implemented by `xerj_api::authz` over the request's `Principal`.
+pub trait IndexVisibility: Send + Sync {
+ /// May the current principal touch the concrete index `index`?
+ ///
+ /// `index` is always a real index name — aliases are resolved before this
+ /// is consulted, so an alias cannot launder access to its target.
+ fn visible(&self, index: &str) -> bool;
+}
+
+tokio::task_local! {
+ static VISIBILITY: Arc<dyn IndexVisibility>;
+}
+
+/// Is `index` visible to the principal behind the current request?
+///
+/// `true` when no guard is installed — see the module docs: that means the
+/// caller is the engine itself, not an unauthenticated request.
+pub fn visible(index: &str) -> bool {
+ VISIBILITY.try_with(|v| v.visible(index)).unwrap_or(true)
+}
+
+/// Run `fut` with `guard` installed as the current request's visibility rule.
+pub async fn scoped<F>(guard: Arc<dyn IndexVisibility>, fut: F) -> F::Output
+where
+ F: std::future::Future,
+{
+ VISIBILITY.scope(guard, fut).await
+}
+
+/// The guard the caller is currently running under, if any.
+///
+/// The half of the detached-work contract that runs *before* the spawn: call
+/// it on the request's own task, move the result into the spawned future, and
+/// hand it back to [`scoped`] there. `None` means the caller is already
+/// unrestricted (engine-internal work, or a superuser request, for which the
+/// API layer installs no guard at all), and the spawned task inherits exactly
+/// that — an unrestricted caller does not become restricted by detaching, and
+/// a restricted one does not become unrestricted.
+pub fn current() -> Option<Arc<dyn IndexVisibility>> {
+ VISIBILITY.try_with(Arc::clone).ok()
+}
+
+/// Run `fut` under `guard` when there is one, unrestricted when there is not.
+///
+/// The spawn-site half of the contract, so a caller does not have to spell the
+/// `match` out (and cannot get it backwards):
+///
+/// ```ignore
+/// let rule = index_guard::current(); // on the request's task
+/// tokio::spawn(async move {
+/// index_guard::scoped_opt(rule, async move { /* … */ }).await
+/// });
+/// ```
+pub async fn scoped_opt<F>(guard: Option<Arc<dyn IndexVisibility>>, fut: F) -> F::Output
+where
+ F: std::future::Future,
+{
+ match guard {
+ Some(g) => scoped(g, fut).await,
+ None => fut.await,
+ }
+}
+
+/// Is a guard installed at all — i.e. is this a request rather than
+/// engine-internal work? Distinguishes the two cases that [`visible`]
+/// deliberately collapses to `true`; nothing in the decision path needs it,
+/// but a caller reasoning about which side of the boundary it is on does.
+pub fn guarded() -> bool {
+ VISIBILITY.try_with(|_| true).unwrap_or(false)
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ struct Only(&'static str);
+ impl IndexVisibility for Only {
+ fn visible(&self, index: &str) -> bool {
+ index == self.0
+ }
+ }
+
+ #[tokio::test]
+ async fn absent_guard_is_unrestricted() {
+ assert!(visible("anything"));
+ assert!(!guarded());
+ }
+
+ /// The contract detached work runs under: capture on the request's task,
+ /// re-install inside the spawned one. Without the capture the spawned task
+ /// is unrestricted, which is the ML-datafeed hole.
+ #[tokio::test]
+ async fn a_captured_guard_survives_tokio_spawn() {
+ let (uncarried, carried) = scoped(Arc::new(Only("mine")), async {
+ // Detaching without carrying the rule loses it — this is the shape
+ // the bug had.
+ let uncarried = tokio::spawn(async { (guarded(), visible("yours")) })
+ .await
+ .expect("join");
+ // Carrying it across keeps the denial.
+ let rule = current();
+ let carried = tokio::spawn(async move {
+ scoped_opt(rule, async {
+ (guarded(), visible("yours"), visible("mine"))
+ })
+ .await
+ })
+ .await
+ .expect("join");
+ (uncarried, carried)
+ })
+ .await;
+ assert_eq!(uncarried, (false, true), "a bare spawn loses the guard");
+ assert_eq!(
+ carried,
+ (true, false, true),
+ "a carried guard denies in the spawned task exactly as it did in the request"
+ );
+ }
+
+ /// Outside a request there is nothing to carry, and carrying `None` must
+ /// not invent a restriction.
+ #[tokio::test]
+ async fn carrying_an_absent_guard_stays_unrestricted() {
+ assert!(current().is_none());
+ let seen = tokio::spawn(scoped_opt(None, async { (guarded(), visible("anything")) }))
+ .await
+ .expect("join");
+ assert_eq!(seen, (false, true));
+ }
+
+ /// The first cut of this module claimed `rayon` inherits the task-local.
+ /// It does not: a rayon worker is an OS thread `tokio` never polls a task
+ /// on, so the thread-local backing `VISIBILITY` is simply unset there.
+ ///
+ /// `ThreadPool::spawn` (rather than `install`/`join`/`par_iter`) is used
+ /// deliberately — it *always* runs the closure on a pool worker, never on
+ /// the calling thread, so this cannot pass by accidentally staying home.
+ #[tokio::test]
+ async fn absent_guard_on_a_rayon_worker() {
+ let pool = rayon::ThreadPoolBuilder::new()
+ .num_threads(1)
+ .build()
+ .expect("rayon pool");
+ let (tx, rx) = std::sync::mpsc::channel();
+ scoped(Arc::new(Only("mine")), async {
+ assert!(!visible("yours"), "denied on the request's own thread");
+ pool.spawn(move || {
+ let _ = tx.send((guarded(), visible("yours")));
+ });
+ })
+ .await;
+ assert_eq!(
+ rx.recv().expect("rayon result"),
+ (false, true),
+ "a rayon worker sees no guard — so no rayon closure may resolve an index name"
+ );
+ }
+
+ #[tokio::test]
+ async fn guard_applies_inside_the_scope_only() {
+ scoped(Arc::new(Only("mine")), async {
+ assert!(guarded());
+ assert!(visible("mine"));
+ assert!(!visible("yours"));
+ // A nested await keeps the guard.
+ tokio::task::yield_now().await;
+ assert!(!visible("yours"));
+ })
+ .await;
+ // Outside the scope it is gone again.
+ assert!(visible("yours"));
+ }
+}
diff --git a/engine/crates/xerj-engine/src/lib.rs b/engine/crates/xerj-engine/src/lib.rs
index be0b21f85..83b1a44e6 100644
--- a/engine/crates/xerj-engine/src/lib.rs
+++ b/engine/crates/xerj-engine/src/lib.rs
@@ -22,6 +22,7 @@ pub mod bulk;
pub mod engine;
pub mod governor;
pub mod index;
+pub mod index_guard;
pub mod ingest_memory;
pub mod memtable;
pub mod painless;
diff --git a/engine/crates/xerj-engine/src/rbac.rs b/engine/crates/xerj-engine/src/rbac.rs
index 76eaa2f20..2954904c0 100644
--- a/engine/crates/xerj-engine/src/rbac.rs
+++ b/engine/crates/xerj-engine/src/rbac.rs
@@ -1,15 +1,19 @@
-//! Role-based access control — data model only. **NOT ENFORCED.**
+//! Role-based access control — data model, plus the per-key grants that the
+//! reserved-namespace authorization in `xerj-api::authz` enforces.
//!
-//! ⚠️ HONEST-SURFACE WARNING (RC4 item 6): the types below store roles, but
-//! **nothing in the auth path consults them.** `is_authorized`
-//! (`xerj-api::auth`) is binary — a request either carries the admin key / a
-//! valid minted API key (→ full superuser access) or it is rejected. A role
-//! `PUT` via `/_security/role/{name}` is recorded and can be read back, but it
-//! grants and restricts **nothing**; every authenticated caller is superuser.
-//! The `/_security/role*` handlers therefore stamp every response with
-//! `"enforced": false` so an operator cannot mistake this for a working
-//! authorization system. Full RBAC enforcement (per-request privilege checks,
-//! per-key `role_descriptors`, FLS/DLS) is DEFERRED.
+//! ⚠️ HONEST-SURFACE WARNING, NARROWED (was RC4 item 6): the named
+//! [`RoleStore`] below — the thing `PUT /_security/role/{name}` writes — is
+//! still **data only**. Nothing in the auth path consults it, which is why the
+//! `/_security/role*` handlers stamp every response with `"enforced": false`.
+//! Broad RBAC over the general ES-compat surface (FLS/DLS, cluster privileges,
+//! named-role binding) remains DEFERRED.
+//!
+//! What *is* enforced: the [`Role`] values attached to a **minted API key**
+//! (`ApiKeyRecord::roles`, parsed from `role_descriptors` by
+//! [`roles_from_role_descriptors`]). Those gate the reserved `.xerj-memory-*`
+//! namespace — agent-memory namespaces and second brains — and confine a
+//! scoped key to the indices it names. See `xerj-api::authz` for the decision
+//! function and the enumeration of enforcement points.
//!
//! What ships today:
//! - `Privilege` enum covering the seven core ops (read / write / admin
@@ -17,9 +21,11 @@
//! - `Role` — name + privileges + index-pattern allow list.
//! - `RoleStore` — in-memory map of roles, default seeded with
//! `admin`, `write`, `read`, `read_only_index`, `snapshot_admin`, `auditor`.
+//! - `roles_from_role_descriptors` — ES `role_descriptors` → `Vec<Role>`.
use parking_lot::RwLock;
use serde::{Deserialize, Serialize};
+use serde_json::Value;
use std::collections::{HashMap, HashSet};
use std::sync::Arc;
@@ -42,7 +48,7 @@ pub enum Privilege {
AuditRead,
}
-#[derive(Clone, Debug, Serialize, Deserialize)]
+#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct Role {
pub name: String,
pub privileges: HashSet<Privilege>,
@@ -66,6 +72,30 @@ impl Role {
/// Does this role apply to the named index? Glob: "*" matches
/// everything; literal names must match exactly; suffix-`*` (e.g.
/// `logs-*`) matches by prefix.
+ ///
+ /// # `names: ["*"]` includes the reserved namespace, deliberately
+ ///
+ /// "Everything" means everything: a role granted `*` matches
+ /// `.xerj-memory-alice-edges` like any other index, so a key minted with
+ /// it can read and write every second brain and every agent-memory
+ /// namespace on the node. `xerj_api::authz::may_reach_reserved` is *not*
+ /// consulted on this path — it decides whether a name a caller is about to
+ /// **create** (an alias, a fresh index, a template pattern) is squatting
+ /// the reserved prefix, not whether an existing grant covers a read.
+ ///
+ /// This is an administrator's explicit choice rather than an escalation:
+ /// the only principal that can mint a scoped key at all is the superuser
+ /// (`POST /_security/api_key`), and `*` is the plainest possible way to
+ /// write "this key gets the whole node". Narrowing it silently would break
+ /// the operator who wrote `*` and meant it — a Kibana service account, a
+ /// backup runner — with no error to explain the missing data.
+ ///
+ /// **Operators wanting brain isolation must not hand out `names: ["*"]`.**
+ /// Grant the concrete indices, or a prefix that excludes the reserved
+ /// namespace (`logs-*`, `app-*`); `.xerj-memory-*` is reachable only by a
+ /// grant that reaches it. `a_star_grant_reaches_the_reserved_namespace` in
+ /// `xerj_api::authz`'s tests pins this behaviour so it cannot be changed
+ /// by accident.
pub fn applies_to(&self, idx: &str) -> bool {
for pat in &self.indices {
if pat == "*" || pat == idx {
@@ -85,6 +115,93 @@ impl Role {
}
}
+/// Map one Elasticsearch index-privilege name onto this crate's [`Privilege`]s.
+///
+/// Unknown names map to **nothing** — an unrecognized privilege grants no
+/// access rather than being waved through, so a typo (or an ES privilege xerj
+/// has not implemented) fails closed.
+fn es_index_privilege(name: &str) -> &'static [Privilege] {
+ use Privilege::*;
+ match name {
+ "all" => &[ReadIndex, WriteIndex, AdminIndex],
+ "read" | "read_cross_cluster" | "view_index_metadata" | "monitor" => &[ReadIndex],
+ "write" | "index" | "create" | "create_doc" | "delete" | "maintenance" => &[WriteIndex],
+ "manage" | "create_index" | "delete_index" | "manage_ilm" | "manage_follow_index" => {
+ &[AdminIndex]
+ }
+ _ => &[],
+ }
+}
+
+/// Parse an Elasticsearch-shaped `role_descriptors` object — exactly what
+/// `POST /_security/api_key` already accepts on the wire — into the roles this
+/// crate enforces:
+///
+/// ```json
+/// { "alice-brain": { "indices": [
+/// { "names": [".xerj-memory-alice-edges", ".xerj-memory-alice"],
+/// "privileges": ["read", "write"] } ] } }
+/// ```
+///
+/// One [`Role`] is emitted per `indices[]` entry (named `"{descriptor}[{i}]"`),
+/// because a `Role` carries a single privilege set while one descriptor may
+/// list several entries with different privileges.
+///
+/// Deliberate omissions, all of which fail closed (they grant nothing):
+/// - `cluster` privileges are ignored — xerj has no cluster-privilege
+/// enforcement, so honouring them would be theatre.
+/// - `field_security` / `query` (FLS/DLS) are ignored; a descriptor that only
+/// makes sense with FLS/DLS therefore over-grants at the field level within
+/// an index it was already granted. Callers that need FLS must not rely on
+/// this.
+/// - Only a trailing `*` globs (see [`Role::applies_to`]); a mid-pattern `*`
+/// matches nothing.
+pub fn roles_from_role_descriptors(descriptors: &Value) -> Vec<Role> {
+ let Some(map) = descriptors.as_object() else {
+ return Vec::new();
+ };
+ let mut roles = Vec::new();
+ for (descriptor_name, descriptor) in map {
+ let Some(entries) = descriptor.get("indices").and_then(|v| v.as_array()) else {
+ continue;
+ };
+ for (i, entry) in entries.iter().enumerate() {
+ let names: Vec<String> = entry
+ .get("names")
+ .and_then(|v| v.as_array())
+ .map(|a| {
+ a.iter()
+ .filter_map(|v| v.as_str())
+ .filter(|s| !s.is_empty())
+ .map(String::from)
+ .collect()
+ })
+ .unwrap_or_default();
+ let privileges: HashSet<Privilege> = entry
+ .get("privileges")
+ .and_then(|v| v.as_array())
+ .map(|a| {
+ a.iter()
+ .filter_map(|v| v.as_str())
+ .flat_map(|p| es_index_privilege(p).iter().copied())
+ .collect()
+ })
+ .unwrap_or_default();
+ if names.is_empty() || privileges.is_empty() {
+ // Nothing to grant. Skipping keeps `roles.is_empty()` an
+ // accurate "this key was given no usable grant" signal.
+ continue;
+ }
+ roles.push(Role::new(
+ format!("{descriptor_name}[{i}]"),
+ privileges,
+ names,
+ ));
+ }
+ }
+ roles
+}
+
pub struct RoleStore {
roles: RwLock<HashMap<String, Role>>,
}
@@ -201,4 +318,67 @@ mod tests {
assert!(!r.allows("metrics-prod", Privilege::ReadIndex));
assert!(!r.allows("logs-prod", Privilege::WriteIndex));
}
+
+ #[test]
+ fn role_descriptors_parse_into_index_grants() {
+ let roles = roles_from_role_descriptors(&serde_json::json!({
+ "alice-brain": {
+ "cluster": ["all"],
+ "indices": [
+ { "names": [".xerj-memory-alice-edges"], "privileges": ["read", "write"] },
+ { "names": [".xerj-memory-alice"], "privileges": ["read"] }
+ ]
+ }
+ }));
+ assert_eq!(roles.len(), 2, "one Role per indices[] entry");
+ assert!(roles
+ .iter()
+ .any(|r| r.allows(".xerj-memory-alice-edges", Privilege::WriteIndex)));
+ assert!(roles
+ .iter()
+ .any(|r| r.allows(".xerj-memory-alice", Privilege::ReadIndex)));
+ // The read-only entry does not silently grant writes, and nothing
+ // grants anything on a brain that was never named.
+ assert!(!roles
+ .iter()
+ .any(|r| r.allows(".xerj-memory-alice", Privilege::WriteIndex)));
+ assert!(!roles
+ .iter()
+ .any(|r| r.allows(".xerj-memory-bob-edges", Privilege::ReadIndex)));
+ // `cluster` privileges are ignored, so "all" there granted no index.
+ assert!(!roles
+ .iter()
+ .any(|r| r.allows("anything", Privilege::ReadIndex)));
+ }
+
+ #[test]
+ fn unusable_role_descriptors_grant_nothing() {
+ // Not an object, no `indices`, empty names, empty/unknown privileges —
+ // every one of these must produce zero roles, so `roles.is_empty()`
+ // stays a truthful "no grant" signal for the fail-closed path.
+ for d in [
+ serde_json::json!("all"),
+ serde_json::json!({ "r": { "cluster": ["all"] } }),
+ serde_json::json!({ "r": { "indices": [{ "names": [], "privileges": ["all"] }] } }),
+ serde_json::json!({ "r": { "indices": [{ "names": ["x"], "privileges": [] }] } }),
+ serde_json::json!({ "r": { "indices": [{ "names": ["x"], "privileges": ["bogus"] }] } }),
+ ] {
+ assert!(
+ roles_from_role_descriptors(&d).is_empty(),
+ "must grant nothing: {d}"
+ );
+ }
+ }
+
+ #[test]
+ fn all_privilege_covers_read_write_admin() {
+ let roles = roles_from_role_descriptors(&serde_json::json!({
+ "r": { "indices": [{ "names": ["logs-*"], "privileges": ["all"] }] }
+ }));
+ let r = &roles[0];
+ assert!(r.allows("logs-prod", Privilege::ReadIndex));
+ assert!(r.allows("logs-prod", Privilege::WriteIndex));
+ assert!(r.allows("logs-prod", Privilege::AdminIndex));
+ assert!(!r.allows("metrics", Privilege::ReadIndex));
+ }
}
diff --git a/engine/crates/xerj-server/src/grpc.rs b/engine/crates/xerj-server/src/grpc.rs
index fe36e247d..cd68c6629 100644
--- a/engine/crates/xerj-server/src/grpc.rs
+++ b/engine/crates/xerj-server/src/grpc.rs
@@ -16,7 +16,9 @@ use anyhow::Context;
use tonic::{Request, Response, Status, Streaming};
use tracing::info;
+use xerj_api::auth::Principal;
use xerj_api::AppState;
+use xerj_engine::rbac::Privilege;
use xerj_query::parse_request;
/// Generated prost messages + tonic client/server stubs for `xerj.v1`.
@@ -43,13 +45,59 @@ impl GrpcService {
}
}
+/// Authorize an RPC that names an index.
+///
+/// The listener speaks the same engine as the HTTP surfaces, so the reserved
+/// `.xerj-memory-*` namespace has to be gated here too (issue #79) — otherwise
+/// per-brain authorization would hold on :9200 and :8080 and fall open on
+/// :8081, which is worse than not having it. [`GrpcAuth`] plants the
+/// [`Principal`] in the request extensions; a missing one means the
+/// interceptor did not run, and is treated as no credential at all.
+///
+/// `clippy::result_large_err` fires because `tonic::Status` is a fat value, but
+/// `Err` here IS the RPC's denial status — the shape every other handler in
+/// this file already returns — so boxing it would add an allocation on the deny
+/// path and force an unwrap at every `?`.
+#[allow(clippy::result_large_err)]
+fn authorize(
+ extensions: &tonic::Extensions,
+ index: &str,
+ privilege: Privilege,
+) -> Result<(), Status> {
+ let principal = extensions.get::<Principal>().unwrap_or(&Principal::Denied);
+ if principal.allows_index(index, privilege) {
+ Ok(())
+ } else {
+ Err(Status::permission_denied(format!(
+ "this credential is not authorized for [{index}]"
+ )))
+ }
+}
+
+/// Same rule as the HTTP middleware: a pattern that could reach the reserved
+/// namespace is refused rather than expanded, and a scoped credential must
+/// name the index it was granted.
+#[allow(clippy::result_large_err)] // same rationale as `authorize`
+fn reject_wildcards(principal: &Principal, index: &str) -> Result<(), Status> {
+ if (index.contains('*') || index == "_all") && !principal.is_superuser() {
+ return Err(Status::permission_denied(format!(
+ "index patterns are not available to this credential: [{index}]"
+ )));
+ }
+ Ok(())
+}
+
#[tonic::async_trait]
impl XerjSearch for GrpcService {
async fn search(
&self,
request: Request<pb::SearchRequest>,
) -> Result<Response<pb::SearchResponse>, Status> {
- let req = request.into_inner();
+ let (_meta, extensions, req) = request.into_parts();
+ let principal = extensions.get::<Principal>().unwrap_or(&Principal::Denied);
+ reject_wildcards(principal, &req.index)?;
+ // Before `get_index`, so 403-vs-not-found cannot map the node's brains.
+ authorize(&extensions, &req.index, Privilege::ReadIndex)?;
let idx = self
.state
@@ -111,7 +159,8 @@ impl XerjSearch for GrpcService {
&self,
request: Request<pb::IndexRequest>,
) -> Result<Response<pb::IndexResponse>, Status> {
- let req = request.into_inner();
+ let (_meta, extensions, req) = request.into_parts();
+ authorize(&extensions, &req.index, Privilege::WriteIndex)?;
// Index-on-write: ES auto-creates the index on first document.
let idx = self
@@ -146,12 +195,20 @@ impl XerjSearch for GrpcService {
&self,
request: Request<Streaming<pb::IndexRequest>>,
) -> Result<Response<pb::BulkResponse>, Status> {
- let mut stream = request.into_inner();
+ let (_meta, extensions, mut stream) = request.into_parts();
let started = std::time::Instant::now();
let mut indexed = 0i32;
let mut errors = false;
while let Some(item) = stream.message().await? {
+ // Per message: one stream may name many indices, and an
+ // unauthorized one must not ride in on an authorized one. A refused
+ // item is counted as an error, matching how this loop already
+ // reports a bad index or an unparsable source.
+ if authorize(&extensions, &item.index, Privilege::WriteIndex).is_err() {
+ errors = true;
+ continue;
+ }
let idx = match self.state.engine.get_or_create_index(&item.index) {
Ok(i) => i,
Err(_) => {
@@ -190,7 +247,11 @@ impl XerjSearch for GrpcService {
&self,
request: Request<pb::GetRequest>,
) -> Result<Response<pb::GetResponse>, Status> {
- let req = request.into_inner();
+ let (_meta, extensions, req) = request.into_parts();
+ // Deliberately a hard denial rather than this RPC's soft "not found":
+ // answering `found: false` for an unauthorized index would still let a
+ // caller probe which indices exist by timing or by error shape.
+ authorize(&extensions, &req.index, Privilege::ReadIndex)?;
// Unknown index → "not found" (ES GET semantics), not an error.
let idx = match self.state.engine.get_index(&req.index) {
@@ -229,7 +290,8 @@ impl XerjSearch for GrpcService {
&self,
request: Request<pb::DeleteRequest>,
) -> Result<Response<pb::DeleteResponse>, Status> {
- let req = request.into_inner();
+ let (_meta, extensions, req) = request.into_parts();
+ authorize(&extensions, &req.index, Privilege::WriteIndex)?;
let idx = self
.state
@@ -285,19 +347,23 @@ struct GrpcAuth {
}
impl tonic::service::Interceptor for GrpcAuth {
- fn call(&mut self, request: Request<()>) -> Result<Request<()>, Status> {
+ fn call(&mut self, mut request: Request<()>) -> Result<Request<()>, Status> {
let auth_header = request
.metadata()
.get("authorization")
.and_then(|v| v.to_str().ok());
- if xerj_api::auth::is_authorized(&self.state, auth_header) {
- Ok(request)
- } else {
- Err(Status::unauthenticated(
+ // Resolve the identity, not just a yes/no, and carry it into the RPC:
+ // the index an RPC touches lives in its message body, so the per-index
+ // decision cannot be made here — only the principal can be.
+ let principal = xerj_api::auth::authenticate(&self.state, auth_header);
+ if matches!(principal, Principal::Denied) {
+ return Err(Status::unauthenticated(
"missing or invalid API key in authorization metadata",
- ))
+ ));
}
+ request.extensions_mut().insert(principal);
+ Ok(request)
}
}
@@ -584,4 +650,236 @@ mod tests {
let _ = tx.send(());
let _ = server.await;
}
+
+ /// Standard-alphabet base64, so the test can build the `id:secret`
+ /// credential a minted key presents. `xerj_api`'s encoder is crate-private.
+ fn b64(input: &str) -> String {
+ const A: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
+ let bytes = input.as_bytes();
+ let mut out = String::new();
+ for chunk in bytes.chunks(3) {
+ let b = [
+ chunk[0],
+ *chunk.get(1).unwrap_or(&0),
+ *chunk.get(2).unwrap_or(&0),
+ ];
+ let n = ((b[0] as u32) << 16) | ((b[1] as u32) << 8) | b[2] as u32;
+ out.push(A[(n >> 18) as usize & 63] as char);
+ out.push(A[(n >> 12) as usize & 63] as char);
+ out.push(if chunk.len() > 1 {
+ A[(n >> 6) as usize & 63] as char
+ } else {
+ '='
+ });
+ out.push(if chunk.len() > 2 {
+ A[n as usize & 63] as char
+ } else {
+ '='
+ });
+ }
+ out
+ }
+
+ /// Mint a key straight into the engine's store and return the
+ /// `authorization` metadata value a client would present.
+ fn mint_key(state: &AppState, id: &str, roles: Vec<xerj_engine::rbac::Role>) -> String {
+ state.engine.persist_api_key(
+ id.to_string(),
+ xerj_engine::engine::ApiKeyRecord {
+ name: id.to_string(),
+ secret: "s3cret".into(),
+ creation_ms: 0,
+ expiration_ms: None,
+ invalidated: false,
+ roles,
+ },
+ );
+ format!("ApiKey {}", b64(&format!("{id}:s3cret")))
+ }
+
+ /// Issue #79 on the gRPC listener. Per-brain authorization has to hold on
+ /// :8081 exactly as it does on :9200 and :8080 — these RPCs take an index
+ /// name straight out of the message body, so before this the boolean auth
+ /// check let any authenticated client read, forge and delete any brain's
+ /// edges here even while the HTTP surfaces refused.
+ #[tokio::test]
+ async fn grpc_enforces_per_index_authorization() {
+ use std::collections::HashSet;
+ use tonic::Code;
+ use xerj_engine::rbac::{Privilege, Role};
+
+ let dir = TempDir::new().unwrap();
+ let admin = "grpc-admin-secret";
+ let state = app_state_with_auth(&dir, admin);
+
+ // A key scoped to alice's brain, and a legacy key with no grants.
+ let alice = mint_key(
+ &state,
+ "alice-key",
+ vec![Role::new(
+ "alice",
+ [Privilege::ReadIndex, Privilege::WriteIndex]
+ .into_iter()
+ .collect::<HashSet<_>>(),
+ vec![".xerj-memory-alice-edges".into()],
+ )],
+ );
+ let legacy = mint_key(&state, "legacy-key", Vec::new());
+
+ let port = free_port();
+ let addr: SocketAddr = format!("127.0.0.1:{port}").parse().unwrap();
+ let (tx, rx) = tokio::sync::oneshot::channel::<()>();
+ let server = tokio::spawn(serve_grpc(addr, state, async move {
+ let _ = rx.await;
+ }));
+ let url = format!("http://127.0.0.1:{port}");
+ let mut client = connect(&url).await;
+
+ let with = |cred: &str| {
+ let v: tonic::metadata::MetadataValue<tonic::metadata::Ascii> = cred.parse().unwrap();
+ v
+ };
+
+ // Seed bob's brain as the admin, so there is something to keep apart.
+ let mut req = tonic::Request::new(pb::IndexRequest {
+ index: ".xerj-memory-bob-edges".into(),
+ id: "e1".into(),
+ source_json: r#"{"src":"doc:1","dst":"secret:2","type":"mentions"}"#.into(),
+ });
+ req.metadata_mut()
+ .insert("authorization", with(&format!("ApiKey {admin}")));
+ assert_eq!(
+ client
+ .index(req)
+ .await
+ .expect("admin seed")
+ .into_inner()
+ .result,
+ "created"
+ );
+
+ // ── alice cannot touch bob, by any RPC ──────────────────────────────
+ let mut req = tonic::Request::new(pb::SearchRequest {
+ index: ".xerj-memory-bob-edges".into(),
+ query_json: String::new(),
+ size: 10,
+ from: 0,
+ });
+ req.metadata_mut().insert("authorization", with(&alice));
+ let e = client
+ .search(req)
+ .await
+ .expect_err("search bob must be denied");
+ assert_eq!(e.code(), Code::PermissionDenied, "search: {e:?}");
+
+ let mut req = tonic::Request::new(pb::GetRequest {
+ index: ".xerj-memory-bob-edges".into(),
+ id: "e1".into(),
+ });
+ req.metadata_mut().insert("authorization", with(&alice));
+ let e = client
+ .get_document(req)
+ .await
+ .expect_err("get bob must be denied");
+ assert_eq!(e.code(), Code::PermissionDenied, "get: {e:?}");
+
+ let mut req = tonic::Request::new(pb::IndexRequest {
+ index: ".xerj-memory-bob-edges".into(),
+ id: "forged".into(),
+ source_json: r#"{"src":"doc:1","dst":"attacker:payload"}"#.into(),
+ });
+ req.metadata_mut().insert("authorization", with(&alice));
+ let e = client
+ .index(req)
+ .await
+ .expect_err("forge into bob must be denied");
+ assert_eq!(e.code(), Code::PermissionDenied, "index: {e:?}");
+
+ let mut req = tonic::Request::new(pb::DeleteRequest {
+ index: ".xerj-memory-bob-edges".into(),
+ id: "e1".into(),
+ });
+ req.metadata_mut().insert("authorization", with(&alice));
+ let e = client
+ .delete_document(req)
+ .await
+ .expect_err("delete from bob must be denied");
+ assert_eq!(e.code(), Code::PermissionDenied, "delete: {e:?}");
+
+ // A wildcard is refused rather than expanded.
+ let mut req = tonic::Request::new(pb::SearchRequest {
+ index: "*".into(),
+ query_json: String::new(),
+ size: 10,
+ from: 0,
+ });
+ req.metadata_mut().insert("authorization", with(&alice));
+ let e = client
+ .search(req)
+ .await
+ .expect_err("wildcard must be denied");
+ assert_eq!(e.code(), Code::PermissionDenied, "wildcard: {e:?}");
+
+ // ── alice keeps full use of her own brain ───────────────────────────
+ let mut req = tonic::Request::new(pb::IndexRequest {
+ index: ".xerj-memory-alice-edges".into(),
+ id: "e1".into(),
+ source_json: r#"{"src":"doc:1","dst":"doc:2","type":"mentions"}"#.into(),
+ });
+ req.metadata_mut().insert("authorization", with(&alice));
+ assert_eq!(
+ client
+ .index(req)
+ .await
+ .expect("own write")
+ .into_inner()
+ .result,
+ "created"
+ );
+ let mut req = tonic::Request::new(pb::GetRequest {
+ index: ".xerj-memory-alice-edges".into(),
+ id: "e1".into(),
+ });
+ req.metadata_mut().insert("authorization", with(&alice));
+ assert!(
+ client
+ .get_document(req)
+ .await
+ .expect("own read")
+ .into_inner()
+ .found
+ );
+
+ // ── the legacy key holds nothing in the reserved namespace, but keeps
+ // ── ordinary indices.
+ let mut req = tonic::Request::new(pb::GetRequest {
+ index: ".xerj-memory-alice-edges".into(),
+ id: "e1".into(),
+ });
+ req.metadata_mut().insert("authorization", with(&legacy));
+ let e = client
+ .get_document(req)
+ .await
+ .expect_err("legacy key must not read a brain");
+ assert_eq!(e.code(), Code::PermissionDenied, "legacy reserved: {e:?}");
+
+ let mut req = tonic::Request::new(pb::IndexRequest {
+ index: "logs-2026".into(),
+ id: "d1".into(),
+ source_json: r#"{"msg":"hello"}"#.into(),
+ });
+ req.metadata_mut().insert("authorization", with(&legacy));
+ assert_eq!(
+ client
+ .index(req)
+ .await
+ .expect("legacy key keeps ordinary indices")
+ .into_inner()
+ .result,
+ "created"
+ );
+
+ let _ = tx.send(());
+ let _ = server.await;
+ }
}