AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
security(graph): make a brain a real per-tenant boundary, incl. detached ML-datafeed tasks (#131)
xerj-org3370ef1840d3
diff --git a/docs/design/SECOND_BRAIN_SPEC.md b/docs/design/SECOND_BRAIN_SPEC.mdindex e4916724c..54aa43c0d 100644--- a/docs/design/SECOND_BRAIN_SPEC.md+++ b/docs/design/SECOND_BRAIN_SPEC.md@@ -496,10 +496,29 @@ Shared conventions:are always numbers.- Error body shape (identical to memory_api but typed `graph_error`):`{"error": {"type": "graph_error", "reason": "<msg>"}, "status": <n>}`.-- Auth: process-wide API-key middleware only, same caveat block as- memory_api's module docs (no per-brain authz; copy the warning).-- Unknown brain on read endpoints → 404. `link` creates lazily (like- `ensure_backing_index`).+- Auth: process-wide API-key middleware **plus** per-brain authorization+ (issue #79). A brain authorizes against its edges index,+ `.xerj-memory-{brain}-edges`, which sits in the reserved `.xerj-memory-*`+ namespace that `xerj-api::authz` gates on every surface — the graph API,+ `/_memory/*`, the raw ES-compat index routes and the native+ `/v1/indices/{name}/…` routes alike. The decision is made against the index+ the handler **actually touches**, not the one in the URL: a route that takes+ its index from the body (`_bulk` action lines, `_msearch` headers, `_mget`+ `docs[]._index`, `_aliases`, `_reindex`, `terms`/`lookup` joins) is+ authorized against what the body names, aliases are resolved to their backing+ index before the decision, and `xerj_engine::index_guard` re-checks at the+ point a name becomes an index so a route nobody thought of still fails+ closed. Reach: open mode (`--insecure`,+ point-at-a-folder) and the admin key get every brain; a key minted with+ `role_descriptors` naming the index gets that brain at the granted+ privilege; a key minted without them gets **no** brain. `ego`/`overview`+ need `read`, `link`/`unlink` need `write`, and reading node summaries+ additionally needs `read` on the resolved nodes index.+- Unknown brain on read endpoints → 404 **for a caller authorized for it**.+ An unauthorized caller gets 403 whether or not the brain exists — the+ authorization check runs before the existence probe, so the status code+ cannot be used to enumerate brains. `link` creates lazily (like+ `ensure_backing_index`), which is part of `write`, not a separate `manage`.### 4.1 POST /_graph/{brain}/link — assert an edgediff --git a/engine/crates/xerj-api/src/auth.rs b/engine/crates/xerj-api/src/auth.rsindex 5a5cc08e5..45a36326f 100644--- a/engine/crates/xerj-api/src/auth.rs+++ b/engine/crates/xerj-api/src/auth.rs@@ -1,4 +1,4 @@-//! API key authentication middleware.+//! API key authentication middleware — *who* the caller is.//!//! Checks the `Authorization` header for any of://! - `Authorization: ApiKey <key>`@@ -7,14 +7,21 @@//!//! When `config.auth.enabled` is `false` (or `--insecure` mode was set by//! clearing the key), the check is skipped entirely.+//!+//! Authentication answers *who*; [`crate::authz`] answers *what they may+//! touch*. The bridge between them is [`Principal`], produced here by+//! [`authenticate`] and consulted there. `is_authorized` — the boolean the+//! gRPC interceptor and the HTTP middleware share — is now a thin wrapper over+//! `authenticate`, so the two surfaces cannot drift.use axum::{- extract::{Request, State},- http::StatusCode,+ extract::{FromRequestParts, Request, State},+ http::{request::Parts, StatusCode},middleware::Next,response::{IntoResponse, Response},Json,};+use xerj_engine::rbac::{Privilege, Role};use crate::error::{EsErrorBody, EsErrorResponse, EsRootCause};use crate::state::AppState;@@ -33,6 +40,111 @@ use crate::state::AppState;/// `"live"`/`"ready"` string), so leaving them open is safe.pub const AUTH_EXEMPT_PATHS: [&str; 2] = ["/health/live", "/health/ready"];+// ─────────────────────────────────────────────────────────────────────────────+// Principal+// ─────────────────────────────────────────────────────────────────────────────++/// The authenticated identity behind a request.+///+/// Authentication used to collapse to a bool, which is exactly why a brain was+/// not a security boundary: with no principal on the request there was nothing+/// for an authorization check to consult (issue #79). Every variant below is a+/// *capability statement*, and the ordering is deliberate — the further down+/// the list, the less a caller may reach.+#[derive(Clone, Debug, PartialEq, Eq)]+pub enum Principal {+ /// Full access. Two ways to be one, and only two:+ /// - **open mode** — `auth.enabled = false` or no admin key configured.+ /// This is `xerj --insecure` and the point-at-a-folder first run, where+ /// there is exactly one user and no configuration; it must keep working+ /// with zero setup, so it is deliberately unrestricted.+ /// - the configured **admin/superuser key** (`auth.admin_api_key`).+ Superuser,+ /// A key minted by `POST /_security/api_key` **with** usable+ /// `role_descriptors`. Confined: it may touch only the indices its roles+ /// name, and nothing that is not index-scoped.+ Scoped {+ /// The minted key's id (for audit/log lines, never compared).+ key_id: String,+ /// Index grants parsed from `role_descriptors`. Never empty — an+ /// empty grant list is [`Principal::Unscoped`].+ roles: Vec<Role>,+ },+ /// A key minted **without** `role_descriptors` — the shape every key had+ /// before per-brain authorization existed.+ ///+ /// It keeps its historical reach over the general ES-compat surface (so+ /// hardened Kibana/Beats deployments do not break), but it holds **no**+ /// privilege on the reserved `.xerj-memory-*` namespace. That is the+ /// fail-closed half: an unconfigured principal is denied the brain, not+ /// granted it.+ Unscoped {+ /// The minted key's id.+ key_id: String,+ },+ /// No valid credential. Normally unreachable inside a handler — the auth+ /// middleware answers 401 first — but it is what the [`Principal`]+ /// extractor yields if a handler is ever mounted without that middleware,+ /// so the default is "deny", never "allow".+ Denied,+}++impl Principal {+ /// Does this principal hold `privilege` on `index`?+ ///+ /// This is the single authorization predicate; [`crate::authz`] routes+ /// every enforcement point through it. `Unscoped` answers `true` for+ /// ordinary indices (its historical reach) and `false` for the reserved+ /// namespace, which [`crate::authz::is_reserved_index`] defines.+ pub fn allows_index(&self, index: &str, privilege: Privilege) -> bool {+ match self {+ Principal::Superuser => true,+ Principal::Scoped { roles, .. } => roles.iter().any(|r| r.allows(index, privilege)),+ Principal::Unscoped { .. } => !crate::authz::is_reserved_index(index),+ Principal::Denied => false,+ }+ }++ /// True for the one principal that is allowed to see and do everything —+ /// used to skip the authorization work entirely on the local-dev path.+ pub fn is_superuser(&self) -> bool {+ matches!(self, Principal::Superuser)+ }++ /// Short identity string for error/log lines. Never contains key material.+ pub fn label(&self) -> &str {+ match self {+ Principal::Superuser => "superuser",+ Principal::Scoped { key_id, .. } | Principal::Unscoped { key_id } => key_id.as_str(),+ Principal::Denied => "unauthenticated",+ }+ }+}++/// Extract the principal straight from the request's `Authorization` header.+///+/// Deliberately *re-derives* rather than reading a request extension planted+/// by the middleware: a handler that authorizes must be safe by construction+/// even if it is mounted on a router where the middleware was forgotten. The+/// cost is one map lookup plus a constant-time compare. When no credential+/// resolves, the result is [`Principal::Denied`] — never an error that a+/// caller could turn into a bypass.+#[axum::async_trait]+impl FromRequestParts<AppState> for Principal {+ type Rejection = std::convert::Infallible;++ async fn from_request_parts(+ parts: &mut Parts,+ state: &AppState,+ ) -> Result<Self, Self::Rejection> {+ let header = parts+ .headers+ .get(axum::http::header::AUTHORIZATION)+ .and_then(|v| v.to_str().ok());+ Ok(authenticate(state, header))+ }+}+/// Axum middleware that enforces API key authentication.////// Call via `middleware::from_fn_with_state(state, auth_middleware)` in the@@ -90,15 +202,27 @@ pub async fn auth_middleware(State(state): State<AppState>, req: Request, next:/// value (e.g. `"ApiKey abc"`, `"Bearer abc"`, or `"Basic base64(...)"`), or/// `None` when absent.pub fn is_authorized(state: &AppState, auth_header: Option<&str>) -> bool {+ !matches!(authenticate(state, auth_header), Principal::Denied)+}++/// Resolve the request's credential to a [`Principal`].+///+/// Same acceptance rules as [`is_authorized`] (which is now defined in terms of+/// this), but it reports *which* identity authenticated so authorization has+/// something to decide against. Returns [`Principal::Denied`] when nothing+/// authenticates.+pub fn authenticate(state: &AppState, auth_header: Option<&str>) -> Principal {let cfg = &state.config.auth;- // Skip auth when disabled or no admin key is configured.+ // Open mode: auth disabled or no admin key configured. This is the+ // `--insecure` / point-at-a-folder posture — one user, no configuration —+ // and it stays fully unrestricted on purpose.if !cfg.enabled || cfg.admin_api_key.is_empty() {- return true;+ return Principal::Superuser;}let Some(header) = auth_header else {- return false;+ return Principal::Denied;};if let Some(key) = extract_key(header) {@@ -109,12 +233,14 @@ pub fn is_authorized(state: &AppState, auth_header: Option<&str>) -> bool {// the single most valuable credential in the system — must not be// weaker. `constant_time_eq` still returns early on a length mismatch,// which only reveals the key *length*, matching the created-key path.- return constant_time_eq(key.as_bytes(), cfg.admin_api_key.as_bytes())- // A key minted by `POST /_security/api_key`.- || authenticate_api_key(state, key);+ if constant_time_eq(key.as_bytes(), cfg.admin_api_key.as_bytes()) {+ return Principal::Superuser;+ }+ // A key minted by `POST /_security/api_key`.+ return authenticate_api_key(state, key);}- basic_authorized(state, header)+ basic_principal(state, header)}/// Validate `Authorization: Basic <base64(user:pass)>`.@@ -128,29 +254,29 @@ pub fn is_authorized(state: &AppState, auth_header: Option<&str>) -> bool {/// - or, treating the whole decoded `user:pass` as `id:secret`, a key minted/// by `POST /_security/api_key` (its `id:secret` shape already matches/// `user:pass` exactly, so no separate encoding is needed here).-fn basic_authorized(state: &AppState, header: &str) -> bool {+fn basic_principal(state: &AppState, header: &str) -> Principal {let Some(prefix) = header.get(..6) else {- return false;+ return Principal::Denied;};if !prefix.eq_ignore_ascii_case("basic ") {- return false;+ return Principal::Denied;}let encoded = header[6..].trim();let decoded = match crate::es_compat::base64_decode(encoded) {Some(d) => d,- None => return false,+ None => return Principal::Denied,};let decoded = match std::str::from_utf8(&decoded) {Ok(s) => s,- Err(_) => return false,+ Err(_) => return Principal::Denied,};let Some((user, pass)) = decoded.split_once(':') else {- return false;+ return Principal::Denied;};let cfg = &state.config.auth;if constant_time_eq(pass.as_bytes(), cfg.admin_api_key.as_bytes()) {- return true;+ return Principal::Superuser;}check_minted_key(state, user, pass)}@@ -178,20 +304,21 @@ fn metrics_token_authorized(state: &AppState, req: &Request) -> bool {////// The presented value is `base64("id:api_key")`. Decode it, split on the/// first `':'`, look the id up in the engine's key store, and constant-time-/// compare the secret. All keys authenticate as the single superuser-/// (role_descriptors are accepted at creation but not enforced).-fn authenticate_api_key(state: &AppState, presented: &str) -> bool {+/// compare the secret. The resulting principal carries the key's stored+/// grants: [`Principal::Scoped`] when it was minted with usable+/// `role_descriptors`, [`Principal::Unscoped`] otherwise.+fn authenticate_api_key(state: &AppState, presented: &str) -> Principal {let decoded = match crate::es_compat::base64_decode(presented) {Some(d) => d,- None => return false,+ None => return Principal::Denied,};let decoded = match std::str::from_utf8(&decoded) {Ok(s) => s,- Err(_) => return false,+ Err(_) => return Principal::Denied,};let (id, secret) = match decoded.split_once(':') {Some(parts) => parts,- None => return false,+ None => return Principal::Denied,};check_minted_key(state, id, secret)}@@ -199,21 +326,37 @@ fn authenticate_api_key(state: &AppState, presented: &str) -> bool {/// Look up a minted `POST /_security/api_key` credential by `(id, secret)`./// Shared by the `ApiKey <base64(id:secret)>` and `Basic <base64(id:secret)>`/// paths — both decode to the same `id:secret` shape.-fn check_minted_key(state: &AppState, id: &str, secret: &str) -> bool {+///+/// A minted key is **never** a superuser: the strongest thing it can be is+/// `Unscoped`, which has no reach into the reserved `.xerj-memory-*`+/// namespace. That is what stops "any authenticated caller reads any brain".+fn check_minted_key(state: &AppState, id: &str, secret: &str) -> Principal {let record = match state.engine.api_keys.get(id) {Some(r) => r,- None => return false,+ None => return Principal::Denied,};if record.invalidated {- return false;+ return Principal::Denied;}if let Some(exp) = record.expiration_ms {let now_ms = chrono::Utc::now().timestamp_millis().max(0) as u64;if now_ms >= exp {- return false;+ return Principal::Denied;+ }+ }+ if !constant_time_eq(record.secret.as_bytes(), secret.as_bytes()) {+ return Principal::Denied;+ }+ if record.roles.is_empty() {+ Principal::Unscoped {+ key_id: id.to_string(),+ }+ } else {+ Principal::Scoped {+ key_id: id.to_string(),+ roles: record.roles.clone(),}}- constant_time_eq(record.secret.as_bytes(), secret.as_bytes())}/// Length-independent-only constant-time byte comparison. Avoids leaking thediff --git a/engine/crates/xerj-api/src/authz.rs b/engine/crates/xerj-api/src/authz.rsnew file mode 100644index 000000000..f5a905ccc--- /dev/null+++ b/engine/crates/xerj-api/src/authz.rs@@ -0,0 +1,2639 @@+//! Per-resource authorization — *what* an authenticated caller may touch.+//!+//! [`crate::auth`] answers *who* the caller is ([`Principal`]); this module is+//! the decision function and the enforcement points. It exists because of+//! issue #79: a brain used to be isolated **by name only**, so any credential+//! that could reach the node could read, write and destroy every brain.+//!+//! ## The rule, in one sentence+//!+//! The `.xerj-memory-*` index namespace — every agent-memory namespace and+//! every second brain — is **reserved**: reaching it requires a principal that+//! holds the matching privilege on the specific index, and the only principals+//! that hold anything there are the superuser and a key explicitly granted the+//! index by name.+//!+//! ## The mistake this file is built to make impossible+//!+//! The first cut of #79 decided against the index named in the **URL path**.+//! That is not the index several handlers operate on. `_msearch` takes it from+//! an NDJSON header line, `_bulk` from an action line, `_mget` from+//! `docs[]._index`, `_aliases` from its action list, `_reindex` from+//! `source`/`dest`, a `terms` lookup and a `lookup` runtime field from deep+//! inside a query, an index template from a *pattern* that will match brains+//! created later, `_sql` from the table name in the statement — in every one of+//! those the path index is only a *default* that the body overrides. Four of+//! them were proven live against a running node: read another tenant's brain+//! through `_msearch`, forge and delete its edges through `_bulk`, read a+//! document through `_mget`, and launder a whole index into reach by pointing+//! an alias at it.+//!+//! A per-handler patch list would have closed those four and left the fifth.+//! So authorization is decided at two places that no handler can route around:+//!+//! 1. **Here, before the handler runs.** [`authz_middleware`] resolves the+//! complete target set of a request — from the path *and* from the body —+//! resolves aliases to concrete indices, and authorizes each one with the+//! privilege the request actually needs. This produces the precise+//! ES-shaped 403 and gets read-vs-write-vs-manage right.+//! 2. **In the engine, at the point the name becomes an index.**+//! [`xerj_engine::index_guard`] installs the request's principal as a+//! task-local visibility rule that `Engine::get_index`,+//! `get_or_create_index`, `create_index`, `delete_index`,+//! `list_indices` and `index_name_list` all consult. A handler cannot+//! forget this check, because a handler does not call it — it is inside the+//! only funnel to index data. Anything the first layer did not know to look+//! for (a body shape added next year, an `_sql` table name, a scroll+//! context) still fails closed there.+//!+//! Layer 2 answers "not found" rather than "forbidden", deliberately: it makes+//! a denied brain indistinguishable from an absent one, and it means fan-out+//! (`POST /_search`, `_cat/indices`, `logs-*`) *filters* instead of failing —+//! which is what lets the global verbs keep working for ordinary credentials+//! instead of being refused wholesale.+//!+//! ## Enforcement points (all of them)+//!+//! | Door | Enforced by |+//! |---|---|+//! | `POST /_graph/{brain}/link` | `graph_api::link` + middleware |+//! | `DELETE /_graph/{brain}/link/{edge_id}` | `graph_api::unlink` + middleware |+//! | `GET /_graph/{brain}/ego` | `graph_api::ego` + middleware |+//! | `GET /_graph/{brain}/overview` | `graph_api::overview` + middleware |+//! | `POST/GET/DELETE /_memory/{ns}[/…]` (the brain's *nodes* index) | `memory_api` + middleware |+//! | any ES-compat route naming a reserved index in the path | middleware |+//! | any route naming an index in its **body** (`_bulk`, `_msearch`, `_mget`, `_aliases`, `_reindex`, snapshot `indices`, `terms` lookups, `lookup` runtime fields, `POST /v1/indices`) | middleware ([`body_targets`]) |+//! | an index template whose `index_patterns` would own a brain's mapping | middleware ([`authorize_template_patterns`]) |+//! | an ML job/datafeed config naming the index the server will go and read | middleware ([`BodyShape::MlConfig`]) |+//! | the detached datafeed scorer `_start` spawns | `es_compat::spawn_datafeed_task`, carrying the starter's rule across the `tokio::spawn` |+//! | an **alias** pointing into the reserved namespace | middleware (resolved before the decision) + `Engine::get_index` (resolved before the guard) |+//! | index patterns (`logs-*`, `_all`, `*`) | expanded only over the principal's visible set ([`xerj_engine::index_guard`]) |+//! | unnamed fan-out (`POST /_search`, `/_bulk`, `/_all/*`) | authorized per named index; anything unnamed is filtered by the guard |+//! | enumeration (`GET /_mapping`, `/_cat/indices`, `/_resolve/index/*`) | filtered at `Engine::list_indices`, then pruned in the response |+//! | the native router's `/v1/indices/{name}/…` spelling of the same index | middleware ([`Target::Indices`] classifies both routers) |+//! | the native router's body-named create (`POST /v1/indices`) | middleware ([`BodyShape::NativeCreate`]) |+//! | the gRPC listener (`:8081`) | `xerj-server::grpc`, using [`Principal::allows_index`] |+//! | privilege escalation via `POST /_security/api_key` | `es_compat::security_create_api_key` |+//! | anything else that resolves an index name from anywhere | `xerj_engine::index_guard` |+//!+//! ## Fail-closed+//!+//! Matching the precedent set elsewhere in the tree (a corrupt sidecar, an+//! unknown marker version and an unsatisfiable doc all refuse rather than+//! guess), every unresolved case here denies:+//! - an unknown/expired/invalidated credential → [`Principal::Denied`] → deny;+//! - a key minted with no usable `role_descriptors` → [`Principal::Unscoped`]+//! → **no** privilege on the reserved namespace;+//! - an unrecognized ES privilege name → grants nothing;+//! - a body this module must parse to find its target but cannot → deny;+//! - a snapshot or restore that names no indices (i.e. *all* of them,+//! including brains) → superuser only;+//! - an index expression that resolves to nothing this principal can see →+//! resolves to nothing, not to everything.+//!+//! ## What this does **not** claim+//!+//! Broad RBAC over the general ES-compat surface is still deferred: an+//! `Unscoped` key keeps its historical superuser-equivalent reach over+//! *ordinary* indices. This module makes the reserved namespace a real+//! boundary and confines a `Scoped` key to its grants; it does not turn xerj+//! into a general multi-tenant authorization system. `xerj_engine::rbac`'s+//! named `RoleStore` remains unenforced data.+//!+//! Nor does it second-guess an explicit grant. A scoped key minted with+//! `names: ["*"]` **can** read the reserved namespace, because `*` matches+//! every index and that is what the operator asked for; only a superuser can+//! mint one. See [`xerj_engine::rbac::Role::applies_to`] for the reasoning and+//! `a_star_grant_reaches_the_reserved_namespace` below for the pinned+//! behaviour. Operators who want brain isolation must grant concrete names or+//! a prefix that excludes `.xerj-memory-`.++// The decision functions return `Result<(), Response>` where `Err` IS the+// ready-to-send 403. That trips `clippy::result_large_err` (an `axum::Response`+// is a fat value), but boxing it would add an allocation on the deny path to+// satisfy a lint aimed at hot `Ok` paths, and would obscure that the error+// *is* the response.+#![allow(clippy::result_large_err)]++use std::collections::HashSet;+use std::sync::Arc;++use axum::{+ body::{Body, Bytes},+ extract::{Request, State},+ http::{header, Method, StatusCode},+ middleware::Next,+ response::{IntoResponse, Response},+ Json,+};+use serde_json::Value;++use crate::auth::{authenticate, Principal, AUTH_EXEMPT_PATHS};+use crate::error::{EsErrorBody, EsErrorResponse, EsRootCause};+use crate::state::AppState;+use xerj_engine::index_guard::IndexVisibility;+use xerj_engine::rbac::Privilege;++// ─────────────────────────────────────────────────────────────────────────────+// The reserved namespace+// ─────────────────────────────────────────────────────────────────────────────++/// Index-name prefix reserved for agent-memory namespaces (`memory_api`) and+/// second brains (`graph_api`). Kept in one place so the three writers of the+/// name — `memory_api::MEMORY_PREFIX`, `graph_api::edges_index` and this+/// module — cannot drift.+pub const RESERVED_INDEX_PREFIX: &str = ".xerj-memory-";++/// Is `index` inside the reserved namespace?+pub fn is_reserved_index(index: &str) -> bool {+ index.starts_with(RESERVED_INDEX_PREFIX)+}++/// The edges index backing brain `brain` (SECOND_BRAIN_SPEC §1). This is the+/// resource name a `role_descriptors` grant must name to unlock+/// `/_graph/{brain}/*`.+pub fn brain_edges_index(brain: &str) -> String {+ format!("{RESERVED_INDEX_PREFIX}{brain}-edges")+}++/// The index backing agent-memory namespace `ns` — which is also the default+/// *nodes* index of the brain of the same name.+pub fn memory_namespace_index(ns: &str) -> String {+ format!("{RESERVED_INDEX_PREFIX}{ns}")+}++/// Could this index *expression* (a literal name or a pattern) reach the+/// reserved namespace?+///+/// Deliberately conservative in the deny direction: a pattern is judged by the+/// literal text before its first `*`, and if that prefix is a prefix of — or is+/// prefixed by — the reserved prefix, the expression is treated as reaching it.+/// `*`, `_all`, `.*`, `.xerj-*` and `.xerj-memory-alice*` all qualify;+/// `logs-*` does not.+///+/// Read patterns are no longer *refused* on the strength of this — they are+/// expanded over the principal's visible set instead, which is both safe and+/// the only way a granted `logs-*` can work. It is still what decides whether+/// a name a caller is about to *create* (an alias, a fresh index) is squatting+/// the reserved namespace.+pub fn may_reach_reserved(expression: &str) -> bool {+ let expr = expression.trim();+ if expr.is_empty() {+ return false;+ }+ if expr == "_all" {+ return true;+ }+ let head = expr.split('*').next().unwrap_or("");+ if expr.contains('*') {+ head.starts_with(RESERVED_INDEX_PREFIX) || RESERVED_INDEX_PREFIX.starts_with(head)+ } else {+ is_reserved_index(expr)+ }+}++/// Is this expression a pattern rather than one concrete name?+fn is_pattern(expr: &str) -> bool {+ expr.contains('*') || expr == "_all"+}++// ─────────────────────────────────────────────────────────────────────────────+// Decisions+// ─────────────────────────────────────────────────────────────────────────────++/// Authorize `principal` for `privilege` on the single index `index`.+///+/// Ordered before any existence check on purpose: an unauthorized caller gets+/// the same 403 whether or not the resource exists, so 403-vs-404 cannot be+/// used to enumerate brains.+pub fn authorize_index(+ principal: &Principal,+ index: &str,+ privilege: Privilege,+) -> Result<(), Response> {+ if principal.allows_index(index, privilege) {+ Ok(())+ } else {+ Err(forbidden(principal, index, privilege))+ }+}++/// Authorize `principal` for `privilege` on brain `brain` — i.e. on its edges+/// index. Every `/_graph/{brain}/*` handler calls this before it does anything+/// else that could reveal whether the brain exists.+pub fn authorize_brain(+ principal: &Principal,+ brain: &str,+ privilege: Privilege,+) -> Result<(), Response> {+ authorize_index(principal, &brain_edges_index(brain), privilege)+}++/// Authorize `principal` for `privilege` on agent-memory namespace `ns`.+pub fn authorize_memory_namespace(+ principal: &Principal,+ ns: &str,+ privilege: Privilege,+) -> Result<(), Response> {+ authorize_index(principal, &memory_namespace_index(ns), privilege)+}++/// ES-shaped 403. Names the resource and the privilege the caller lacks — the+/// caller supplied the resource name, so echoing it back confirms nothing —+/// and points at the grant that would fix it.+pub fn forbidden(principal: &Principal, resource: &str, privilege: Privilege) -> Response {+ let action = match privilege {+ Privilege::ReadIndex => "read",+ Privilege::WriteIndex => "write",+ Privilege::AdminIndex => "manage",+ Privilege::SnapshotCreate => "create_snapshot",+ Privilege::SnapshotRestore => "restore_snapshot",+ Privilege::SecurityAdmin => "manage_security",+ Privilege::AuditRead => "read_audit",+ };+ let reason = format!(+ "action [{action}] is unauthorized for this credential on [{resource}]: mint a key whose \+ role_descriptors grant [{action}] on that index (POST /_security/api_key), or use the \+ configured admin key"+ );+ tracing::debug!(+ principal = principal.label(),+ resource,+ action,+ "authorization denied"+ );+ es_error(StatusCode::FORBIDDEN, reason)+}++fn es_error(status: StatusCode, reason: String) -> Response {+ let error_type = "security_exception".to_string();+ let body = EsErrorResponse {+ error: EsErrorBody {+ root_cause: vec![EsRootCause {+ error_type: error_type.clone(),+ reason: reason.clone(),+ resource_type: None,+ resource_id: None,+ index_uuid: None,+ index: None,+ }],+ error_type,+ reason,+ resource_type: None,+ resource_id: None,+ index_uuid: None,+ index: None,+ request_id: None,+ },+ status: status.as_u16(),+ };+ (status, Json(body)).into_response()+}++// ─────────────────────────────────────────────────────────────────────────────+// The request's visible set (layer 2)+// ─────────────────────────────────────────────────────────────────────────────++/// Adapts a [`Principal`] to the engine's [`IndexVisibility`] hook.+///+/// "Visible" is *any* index privilege, not a specific one: this layer decides+/// whether the principal may reach the index at all, which is the property that+/// closes a body-named target the middleware did not know to parse. Which+/// privilege the request needs on it stays the middleware's decision, so a+/// read-only grant still cannot write through the front door.+struct PrincipalVisibility(Principal);++impl IndexVisibility for PrincipalVisibility {+ fn visible(&self, index: &str) -> bool {+ self.0.allows_index(index, Privilege::ReadIndex)+ || self.0.allows_index(index, Privilege::WriteIndex)+ || self.0.allows_index(index, Privilege::AdminIndex)+ }+}++/// The visibility rule for `principal`, for installing around a request.+pub fn visibility_for(principal: &Principal) -> Arc<dyn IndexVisibility> {+ Arc::new(PrincipalVisibility(principal.clone()))+}++// ─────────────────────────────────────────────────────────────────────────────+// Request classification+// ─────────────────────────────────────────────────────────────────────────────++/// What a request path targets.+///+/// `op_start` on [`Target::Indices`] is the segment index at which the+/// operation begins, because the two routers spell resources differently:+/// ES-compat puts the index first (`/{index}/_search`, op at 1) while the+/// native router nests it (`/v1/indices/{index}/search`, op at 3). Both are+/// doors into the reserved namespace, so both are classified here.+#[derive(Debug, PartialEq, Eq)]+enum Target {+ /// `/_graph/{brain}/…`+ Brain(String),+ /// `/_memory/{namespace}[/…]`+ Memory(String),+ /// One or more index expressions, plus where the op segment starts.+ Indices(Vec<String>, usize),+ /// A cluster/global endpoint whose path names no index: `/_cat/*`,+ /// `/_mapping`, `/_nodes`, but also `/_search`, `/_bulk`, `/_msearch`.+ /// Reads are answered over the principal's visible set; mutations must+ /// name their targets in the body (which [`body_targets`] extracts and+ /// authorizes) or belong to a principal that holds the general surface.+ Cluster,+ /// Not authorization-relevant (health probes, the version banner).+ Exempt,+}++/// Read-only ops that are POSTed rather than GETed. Without this list a+/// `POST /{index}/_search` would be classified as a write. Covers both+/// spellings: ES-compat (`_search`) and native (`search`).+const POST_READ_OPS: &[&str] = &[+ "_search",+ "_count",+ "_msearch",+ "_mget",+ "_explain",+ "_validate",+ "_field_caps",+ "_termvectors",+ "_mtermvectors",+ "_search_shards",+ "_rank_eval",+ "_knn_search",+ "_pit",+ "_async_search",+ "_sql",+ "_esql",+ "_eql",+ "_analyze",+ "_graph",+ "_resolve",+ "_terms_enum",+ "_disk_usage",+ // Cluster endpoints that carry a body but change nothing: privilege+ // probes, template rendering, pipeline/painless simulation. Only ever+ // consulted for POST-shaped verbs, so `PUT /_ingest/pipeline/{id}` and+ // `PUT /_scripts/{id}` remain the mutations they are.+ "_security",+ "_ingest",+ "_render",+ "_scripts",+ // native router+ "search",+ "explain-plan",+ "encodings",+];++/// Index-lifecycle ops. These reshape or destroy the index itself, so they+/// need `manage`, not `write`.+const ADMIN_OPS: &[&str] = &[+ "_settings",+ "_mapping",+ "_mappings",+ "_close",+ "_open",+ "_shrink",+ "_split",+ "_clone",+ "_rollover",+ "_alias",+ "_aliases",+ "_cache",+ "_forcemerge",+ "_freeze",+ "_unfreeze",+ "_block",+ "_upgrade",+ "_migration",+];++/// Split a URI path into percent-decoded, non-empty segments.+fn segments(path: &str) -> Vec<String> {+ path.split('/')+ .filter(|s| !s.is_empty())+ .map(percent_decode)+ .collect()+}++/// Minimal percent-decoder. The middleware compares raw path text against+/// index names, so `%2Exerj-memory-alice-edges` must not slip past a check+/// that `.xerj-memory-alice-edges` would fail. Invalid escapes are left+/// verbatim (they cannot decode into the reserved prefix).+fn percent_decode(s: &str) -> String {+ if !s.contains('%') {+ return s.to_string();+ }+ let bytes = s.as_bytes();+ let mut out: Vec<u8> = Vec::with_capacity(bytes.len());+ let mut i = 0;+ while i < bytes.len() {+ if bytes[i] == b'%' && i + 2 < bytes.len() {+ let hex = std::str::from_utf8(&bytes[i + 1..i + 3]).ok();+ if let Some(b) = hex.and_then(|h| u8::from_str_radix(h, 16).ok()) {+ out.push(b);+ i += 3;+ continue;+ }+ }+ out.push(bytes[i]);+ i += 1;+ }+ String::from_utf8_lossy(&out).into_owned()+}++/// Classify a request path.+fn classify(path: &str) -> Target {+ // Probes, the scrape endpoint, and "who am I" — none of which name or+ // reveal an index. `_security/_authenticate` in particular must stay open+ // to a scoped key, or a caller cannot verify its own credential.+ if AUTH_EXEMPT_PATHS.contains(&path)+ || path == "/v1/metrics"+ || path == "/_security/_authenticate"+ {+ return Target::Exempt;+ }+ let segs = segments(path);+ let Some(first) = segs.first() else {+ // `GET /` — the version banner. No data.+ return Target::Exempt;+ };+ match first.as_str() {+ "_graph" => match segs.get(1) {+ Some(brain) => Target::Brain(brain.clone()),+ // No brain in the path: no route matches, let it 404.+ None => Target::Exempt,+ },+ "_memory" => match segs.get(1) {+ Some(ns) => Target::Memory(ns.clone()),+ None => Target::Exempt,+ },+ // Native router (`build_native_router`, :8080). Same engine, same+ // indices, different spelling — and therefore the same boundary:+ // `GET /v1/indices/.xerj-memory-alice-edges` must be no more reachable+ // than `GET /.xerj-memory-alice-edges`.+ "v1" => match (segs.get(1).map(String::as_str), segs.get(2)) {+ (Some("indices"), Some(expr)) | (Some("schema"), Some(expr)) => {+ Target::Indices(split_expressions(expr), 3)+ }+ // `POST /v1/indices` names its index in the body — authorized as+ // `BodyShape::NativeCreate`, not refused for lack of a path target.+ _ => Target::Cluster,+ },+ // `/_all/_search` and `/*/_search` are index-scoped routes whose index+ // happens to be a pattern — not cluster endpoints. Classifying them+ // here is what lets them expand over the caller's visible set like any+ // other pattern instead of being refused for naming nothing.+ s if s == "*" || s == "_all" => Target::Indices(split_expressions(s), 1),+ s if s.starts_with('_') => Target::Cluster,+ s => Target::Indices(split_expressions(s), 1),+ }+}++/// Split a comma-separated multi-index expression (`logs-a,logs-b`).+fn split_expressions(s: &str) -> Vec<String> {+ s.split(',')+ .map(str::trim)+ .filter(|e| !e.is_empty())+ .map(String::from)+ .collect()+}++/// The privilege a request needs on whatever it targets.+///+/// Conservative by construction: anything not recognized as a read or a+/// lifecycle op is treated as a write, so an unclassified mutation can never+/// be waved through as a read.+fn required_privilege(method: &Method, segs: &[String], op_start: usize) -> Privilege {+ if method == Method::GET || method == Method::HEAD {+ return Privilege::ReadIndex;+ }+ match segs.get(op_start).map(String::as_str) {+ Some(o) if POST_READ_OPS.contains(&o) => Privilege::ReadIndex,+ Some(o) if ADMIN_OPS.contains(&o) => Privilege::AdminIndex,+ // No op segment at all: the request targets the index itself, so it is+ // creating or destroying it.+ None => Privilege::AdminIndex,+ _ => Privilege::WriteIndex,+ }+}++/// Does this cluster-level request only *read*?+///+/// `GET`/`HEAD` always do. So does any verb whose op is one of the read ops:+/// `POST /_search`, `/_count`, `/_msearch`, `/_mget` are the reason the global+/// verbs are usable at all, and `DELETE /_search/scroll` / `DELETE /_pit` /+/// `DELETE /_async_search/{id}` release a session the caller already opened.+fn cluster_is_read(method: &Method, segs: &[String]) -> bool {+ if method == Method::GET || method == Method::HEAD {+ return true;+ }+ match segs.first().map(String::as_str) {+ Some(op) => POST_READ_OPS.contains(&op),+ None => true,+ }+}++/// The privilege a `/_graph/{brain}/…` or `/_memory/{ns}[/…]` request needs.+///+/// Kept separate from [`required_privilege`] because these paths spell their+/// ops without a leading underscore (`link`) and because deleting a whole+/// namespace is a lifecycle op while deleting one document is not.+fn reserved_api_privilege(method: &Method, segs: &[String]) -> Privilege {+ if method == Method::GET || method == Method::HEAD {+ return Privilege::ReadIndex;+ }+ if method == Method::POST {+ // `POST /_memory/{ns}/_recall` reads; `POST /_memory/{ns}` and+ // `POST /_graph/{b}/link` write. Creating the backing index lazily is+ // part of writing a brain, not a separate `manage` step.+ return if segs.iter().any(|s| s == "_recall") {+ Privilege::ReadIndex+ } else {+ Privilege::WriteIndex+ };+ }+ if method == Method::DELETE {+ // `DELETE /_memory/{ns}` drops the whole namespace → manage.+ // `DELETE /_memory/{ns}/{id}` and `DELETE /_graph/{b}/link/{id}` are+ // document-level → write.+ return if segs.len() <= 2 {+ Privilege::AdminIndex+ } else {+ Privilege::WriteIndex+ };+ }+ Privilege::WriteIndex+}++// ─────────────────────────────────────────────────────────────────────────────+// Body-named targets+// ─────────────────────────────────────────────────────────────────────────────++/// The shape in which a route's body can name the indices it operates on.+///+/// This table is the exhaustive answer to "which handlers take an index from+/// the request body". It is not the *only* protection — [`PrincipalVisibility`]+/// in the engine catches whatever is missing from it — but it is what produces+/// a precise 403 with the right privilege instead of a downstream "not found".+#[derive(Clone, Copy, Debug, PartialEq, Eq)]+enum BodyShape {+ /// Nothing in the body names an index.+ None,+ /// `_bulk` NDJSON: `{"index"|"create"|"update"|"delete": {"_index": …}}`.+ Bulk,+ /// `_msearch` / `_msearch/template` NDJSON header lines: `{"index": …}`,+ /// where the value may be a string or an array of strings.+ MsearchHeaders,+ /// `_mget`: `{"docs": [{"_index": …}]}`.+ MgetDocs,+ /// `POST /_aliases`: `{"actions": [{"add"|"remove"|"remove_index": …}]}`.+ AliasActions,+ /// `_reindex`: `{"source": {"index": …}, "dest": {"index": …}}`.+ Reindex,+ /// Snapshot create / restore: `{"indices": "a,b" | ["a","b"]}`.+ SnapshotIndices,+ /// Native `POST /v1/indices`: `{"name": …}`.+ NativeCreate,+ /// `PUT /{index}`: the `aliases` block creates alias names.+ CreateIndex,+ /// Any search-shaped body — a `terms` lookup names another index to read+ /// its terms out of, at arbitrary depth.+ Query,+ /// `POST /_monitoring/bulk` writes one fixed internal index.+ MonitoringBulk,+ /// An index template's `index_patterns`. It names no index that exists+ /// yet, but it decides the mapping of every index created under a matching+ /// name later — including a brain — so the patterns are checked in their+ /// own right by [`authorize_template_patterns`].+ IndexTemplate,+ /// An ML job or datafeed config: `PUT /_ml/anomaly_detectors/{id}`+ /// (`source_index`, `index`, and the combined-form `datafeed_config`) and+ /// `PUT /_ml/datafeeds/{id}` (`indices` / `indexes`).+ ///+ /// These name an index the *server* will go and read, on a schedule, on+ /// the caller's behalf. Registering one against an index the caller cannot+ /// read is wrong on its own terms — it was accepted with a 200 before this+ /// arm existed — and it was also the front half of a live cross-brain+ /// read: the detached scorer `POST /_ml/datafeeds/{id}/_start` spawns had+ /// no visibility rule, so it fetched what the caller could not. The+ /// detached half is fixed at the spawn site (`es_compat`'s+ /// `spawn_datafeed_task` carries the rule across); this is the half that+ /// refuses the configuration in the first place.+ MlConfig,+}++/// Search-shaped ops whose body can carry a `terms` lookup.+const QUERY_BODY_OPS: &[&str] = &[+ "_search",+ "_count",+ "_explain",+ "_validate",+ "_delete_by_query",+ "_update_by_query",+ "_async_search",+ "_eql",+ "_rank_eval",+ "_terms_enum",+ "_field_caps",+ "_knn_search",+ "_pit",+ "_graph",+ "_sql",+ "_esql",+];++/// The index `POST /_monitoring/bulk` ingests into+/// (`es_compat::ingest_monitoring_ndjson`).+const MONITORING_INDEX: &str = "xerj-monitoring";++/// Which body shape does this route carry?+fn body_shape(method: &Method, segs: &[String]) -> BodyShape {+ let first = segs.first().map(String::as_str).unwrap_or("");+ // Native router. Its ingest bodies are plain document arrays under an+ // index named in the path; the only body-named index it has is the create.+ if first == "v1" {+ return match (segs.get(1).map(String::as_str), segs.len()) {+ (Some("indices"), 2) if method == Method::POST => BodyShape::NativeCreate,+ _ => BodyShape::None,+ };+ }+ let has = |op: &str| segs.iter().any(|s| s == op);+ if has("_bulk") {+ return BodyShape::Bulk;+ }+ if has("_msearch") {+ return BodyShape::MsearchHeaders;+ }+ if has("_mget") {+ return BodyShape::MgetDocs;+ }+ if first == "_aliases" && method != Method::GET {+ return BodyShape::AliasActions;+ }+ if first == "_reindex" {+ return BodyShape::Reindex;+ }+ if first == "_snapshot" {+ return BodyShape::SnapshotIndices;+ }+ if first == "_monitoring" {+ return BodyShape::MonitoringBulk;+ }+ if (first == "_index_template" || first == "_template") && method != Method::GET {+ return BodyShape::IndexTemplate;+ }+ // `PUT|POST /_ml/anomaly_detectors/{id}` and `PUT|POST /_ml/datafeeds/{id}`+ // — exactly three segments, so the sub-verbs (`…/{id}/_start`, `_stop`,+ // `_score`, `results/records`) are not swept in: they carry no index name+ // of their own, and their reach is decided by the config that was already+ // authorized here plus the guard the spawned scorer carries.+ if first == "_ml" && segs.len() == 3 && (method == Method::PUT || method == Method::POST) {+ return match segs[1].as_str() {+ "anomaly_detectors" | "datafeeds" => BodyShape::MlConfig,+ _ => BodyShape::None,+ };+ }+ if segs.len() == 1 && method == Method::PUT && !first.starts_with('_') {+ return BodyShape::CreateIndex;+ }+ if QUERY_BODY_OPS.iter().any(|op| has(op)) {+ return BodyShape::Query;+ }+ BodyShape::None+}++/// One index expression a request will touch, and what it needs on it.+type Demand = (String, Privilege);++/// Refuse a request whose body should have named its targets but could not be+/// parsed. Fail-closed: an unreadable target is not an absent one.+fn unresolvable(principal: &Principal, what: &str) -> Response {+ tracing::debug!(+ principal = principal.label(),+ what,+ "authorization refused: request body names indices but could not be resolved"+ );+ es_error(+ StatusCode::FORBIDDEN,+ format!(+ "this credential is authorized per index, and the {what} could not be resolved from \+ the request body; send a well-formed body that names its indices"+ ),+ )+}++/// Extract every index this request's **body** will touch, with the privilege+/// it needs on each.+///+/// `default_index` is the path index for the index-scoped spellings+/// (`/{index}/_bulk`, `/{index}/_msearch`, `/{index}/_mget`), which the body+/// may override — the override being the whole bug this exists for.+fn body_targets(+ principal: &Principal,+ shape: BodyShape,+ body: &[u8],+ default_index: Option<&str>,+) -> Result<Vec<Demand>, Response> {+ let mut out: Vec<Demand> = Vec::new();+ match shape {+ BodyShape::None => {}+ BodyShape::MonitoringBulk => {+ out.push((MONITORING_INDEX.to_string(), Privilege::WriteIndex))+ }+ BodyShape::Bulk => {+ // NDJSON, alternating action and (except for `delete`) source+ // lines. Every action line must parse — one that does not is a+ // target we cannot see, so the request is refused rather than+ // handed to a bulk pipeline that would happily route it.+ let Ok(text) = std::str::from_utf8(body) else {+ return Err(unresolvable(principal, "bulk action lines"));+ };+ // Fast path for the index-scoped spelling every client uses by+ // default: with no `_index` anywhere in the payload, every action+ // targets the path index, and there is nothing to find by parsing+ // a million action lines.+ if !text.contains("\"_index\"") {+ if let Some(d) = default_index {+ out.push((d.to_string(), Privilege::WriteIndex));+ }+ return Ok(out);+ }+ let mut expect_action = true;+ for line in text.lines() {+ let line = line.trim();+ if line.is_empty() {+ continue;+ }+ if !expect_action {+ expect_action = true;+ continue;+ }+ let Ok(action) = serde_json::from_str::<Value>(line) else {+ return Err(unresolvable(principal, "bulk action lines"));+ };+ let Some(obj) = action.as_object() else {+ return Err(unresolvable(principal, "bulk action lines"));+ };+ let Some((verb, meta)) = obj.iter().next() else {+ return Err(unresolvable(principal, "bulk action lines"));+ };+ // `delete` carries no source line; everything else does.+ expect_action = verb == "delete";+ let named = meta.get("_index").and_then(Value::as_str);+ if let Some(index) = named.or(default_index) {+ out.push((index.to_string(), Privilege::WriteIndex));+ }+ }+ }+ BodyShape::MsearchHeaders => {+ let Ok(text) = std::str::from_utf8(body) else {+ return Err(unresolvable(principal, "multi-search header lines"));+ };+ let mut expect_header = true;+ for line in text.lines() {+ let line = line.trim();+ if line.is_empty() {+ continue;+ }+ let Ok(parsed) = serde_json::from_str::<Value>(line) else {+ return Err(unresolvable(principal, "multi-search header lines"));+ };+ if expect_header {+ match parsed.get("index") {+ Some(Value::String(s)) => out.push((s.clone(), Privilege::ReadIndex)),+ Some(Value::Array(items)) => {+ for i in items {+ if let Some(s) = i.as_str() {+ out.push((s.to_string(), Privilege::ReadIndex));+ }+ }+ }+ // A header that names no index means "the path index,+ // else every index" — the latter is answered over the+ // visible set, so nothing to demand here.+ _ => {+ if let Some(d) = default_index {+ out.push((d.to_string(), Privilege::ReadIndex));+ }+ }+ }+ } else {+ // The search body half — it can still carry a terms lookup.+ collect_query_body_indices(&parsed, &mut out);+ }+ expect_header = !expect_header;+ }+ }+ BodyShape::MgetDocs => {+ let Ok(parsed) = serde_json::from_slice::<Value>(body) else {+ return Err(unresolvable(principal, "multi-get documents"));+ };+ match parsed.get("docs").and_then(Value::as_array) {+ Some(docs) => {+ for doc in docs {+ let named = doc.get("_index").and_then(Value::as_str);+ match named.or(default_index) {+ Some(index) => out.push((index.to_string(), Privilege::ReadIndex)),+ None => return Err(unresolvable(principal, "multi-get documents")),+ }+ }+ }+ // `{"ids": [...]}` short form uses the path index.+ None => {+ if let Some(d) = default_index {+ out.push((d.to_string(), Privilege::ReadIndex));+ }+ }+ }+ }+ BodyShape::AliasActions => {+ let Ok(parsed) = serde_json::from_slice::<Value>(body) else {+ return Err(unresolvable(principal, "alias actions"));+ };+ let Some(actions) = parsed.get("actions").and_then(Value::as_array) else {+ return Err(unresolvable(principal, "alias actions"));+ };+ for action in actions {+ let Some(obj) = action.as_object() else {+ return Err(unresolvable(principal, "alias actions"));+ };+ for (_verb, params) in obj {+ // Pointing an alias at an index is a change to that index's+ // addressing, so it needs `manage` on it — the same+ // privilege `PUT /{index}/_alias/{alias}` already demands.+ push_names(params.get("index"), Privilege::AdminIndex, &mut out);+ push_names(params.get("indices"), Privilege::AdminIndex, &mut out);+ // …and the alias NAME is authorized in its own right, so a+ // caller cannot squat an alias inside the reserved+ // namespace and have a brain resolve through it later.+ push_names(params.get("alias"), Privilege::AdminIndex, &mut out);+ push_names(params.get("aliases"), Privilege::AdminIndex, &mut out);+ }+ }+ if out.is_empty() {+ return Err(unresolvable(principal, "alias actions"));+ }+ }+ BodyShape::Reindex => {+ let Ok(parsed) = serde_json::from_slice::<Value>(body) else {+ return Err(unresolvable(principal, "reindex source and destination"));+ };+ let before = out.len();+ if let Some(source) = parsed.get("source") {+ push_names(source.get("index"), Privilege::ReadIndex, &mut out);+ push_names(source.get("indices"), Privilege::ReadIndex, &mut out);+ if let Some(q) = source.get("query") {+ collect_query_body_indices(q, &mut out);+ }+ }+ if let Some(dest) = parsed.get("dest") {+ push_names(dest.get("index"), Privilege::WriteIndex, &mut out);+ }+ if out.len() == before {+ return Err(unresolvable(principal, "reindex source and destination"));+ }+ }+ BodyShape::SnapshotIndices => {+ // Absent `indices` means "every index on the node", which for a+ // non-superuser would be a bulk read (or overwrite) of every+ // brain. `decide` turns an empty demand list on this route into a+ // refusal; here we only report what was named. A restore *writes*+ // its targets — `decide` upgrades the privilege, since only the+ // path says which of the two this is.+ if let Ok(parsed) = serde_json::from_slice::<Value>(body) {+ push_names(parsed.get("indices"), Privilege::ReadIndex, &mut out);+ }+ }+ BodyShape::NativeCreate => {+ let Ok(parsed) = serde_json::from_slice::<Value>(body) else {+ return Err(unresolvable(principal, "index name"));+ };+ match parsed.get("name").and_then(Value::as_str) {+ Some(name) => out.push((name.to_string(), Privilege::AdminIndex)),+ None => return Err(unresolvable(principal, "index name")),+ }+ }+ BodyShape::CreateIndex => {+ // `PUT /{index}` names its index in the path (already demanded);+ // its body can additionally mint alias names.+ if let Ok(parsed) = serde_json::from_slice::<Value>(body) {+ if let Some(aliases) = parsed.get("aliases").and_then(Value::as_object) {+ for alias in aliases.keys() {+ out.push((alias.clone(), Privilege::AdminIndex));+ }+ }+ }+ }+ BodyShape::Query => {+ if let Ok(parsed) = serde_json::from_slice::<Value>(body) {+ collect_query_body_indices(&parsed, &mut out);+ }+ }+ BodyShape::MlConfig => {+ let Ok(parsed) = serde_json::from_slice::<Value>(body) else {+ // Not JSON: it names no index, and the handler will reject it+ // for the missing `source_index` / `job_id` anyway.+ return Ok(out);+ };+ // Every spelling the two ML handlers actually read. `put_ml_datafeed`+ // takes `indices` OR `indexes`; `put_ml_anomaly_detector` takes+ // `source_index` (string or array) OR `index`; the ES combined form+ // nests a whole datafeed under `datafeed_config`.+ let push_ml = |v: Option<&Value>, out: &mut Vec<Demand>| {+ push_names(v, Privilege::ReadIndex, out);+ };+ push_ml(parsed.get("source_index"), &mut out);+ push_ml(parsed.get("index"), &mut out);+ push_ml(parsed.get("indices"), &mut out);+ push_ml(parsed.get("indexes"), &mut out);+ if let Some(dfc) = parsed.get("datafeed_config") {+ push_ml(dfc.get("indices"), &mut out);+ push_ml(dfc.get("indexes"), &mut out);+ push_ml(dfc.get("index"), &mut out);+ }+ // A body that names nothing is NOT refused: `PUT /_ml/datafeeds/{id}`+ // may carry only a `job_id`, in which case the handler inherits the+ // detector's `source_index` — a name this middleware cannot see, but+ // one that was authorized when the detector itself was created, and+ // that the scorer re-checks under the starter's own visibility rule.+ // `decide` reads the empty list on this shape as "nothing named",+ // not as "unrestricted".+ }+ // Patterns, not names — `authorize_template_patterns` handles them.+ BodyShape::IndexTemplate => {}+ }+ // A bulk usually names two or three indices across a million action lines.+ out.sort_by(|a, b| a.0.cmp(&b.0).then(rank(a.1).cmp(&rank(b.1))));+ out.dedup();+ Ok(out)+}++/// Stable ordering key for a privilege, so the demand list can be deduped+/// without asking `xerj_engine::rbac` for an `Ord` it has no other use for.+fn rank(p: Privilege) -> u8 {+ match p {+ Privilege::ReadIndex => 0,+ Privilege::WriteIndex => 1,+ Privilege::AdminIndex => 2,+ Privilege::SnapshotCreate => 3,+ Privilege::SnapshotRestore => 4,+ Privilege::SecurityAdmin => 5,+ Privilege::AuditRead => 6,+ }+}++/// An index template applies to indices that do not exist yet, so there is no+/// name to authorize — only a pattern. A template whose `index_patterns` can+/// match the reserved namespace would pick the mapping of a brain created+/// later, which is a write to that brain by another route, so a non-superuser+/// may not register one.+fn authorize_template_patterns(principal: &Principal, body: &[u8]) -> Result<(), Response> {+ let Ok(parsed) = serde_json::from_slice::<Value>(body) else {+ // Not JSON: it names no pattern, and the handler will reject it.+ return Ok(());+ };+ let mut patterns: Vec<Demand> = Vec::new();+ push_names(+ parsed.get("index_patterns"),+ Privilege::AdminIndex,+ &mut patterns,+ );+ // ES 6-era `_template` spelled it `template`.+ push_names(parsed.get("template"), Privilege::AdminIndex, &mut patterns);+ for (pattern, _) in patterns {+ if !may_reach_reserved(&pattern) {+ continue;+ }+ // Only a principal that was *granted* the namespace may template over+ // it. `Unscoped::allows_index` answers "not reserved" for a pattern+ // like `*` — true of the literal text, useless as an answer here — so+ // it is excluded explicitly rather than consulted.+ let held = match principal {+ Principal::Superuser => true,+ Principal::Scoped { .. } => principal.allows_index(&pattern, Privilege::AdminIndex),+ _ => false,+ };+ if !held {+ return Err(forbidden(principal, &pattern, Privilege::AdminIndex));+ }+ }+ Ok(())+}++/// Push a `"a"` / `"a,b"` / `["a","b"]` index field onto the demand list.+fn push_names(v: Option<&Value>, privilege: Privilege, out: &mut Vec<Demand>) {+ match v {+ Some(Value::String(s)) => {+ for part in split_expressions(s) {+ out.push((part, privilege));+ }+ }+ Some(Value::Array(items)) => {+ for item in items {+ if let Some(s) = item.as_str() {+ for part in split_expressions(s) {+ out.push((part, privilege));+ }+ }+ }+ }+ _ => {}+ }+}++/// Collect every index a **search body** reads that the URL never names.+///+/// Two shapes, both found by auditing the tree rather than by being attacked —+/// which is the point of auditing:+///+/// - a `terms` lookup, `{"terms": {"field": {"index": "other", "id": "1",+/// "path": "vals"}}}`, fetches a document out of `other` and substitutes its+/// values into the query;+/// - a `lookup` runtime field, `{"runtime_mappings": {"f": {"type": "lookup",+/// "target_index": "other", …}}}`, joins rows out of `other` into the hits.+///+/// Both are matched structurally (the full lookup triple, the `type: lookup`+/// marker) rather than on the key name alone, so a document field that happens+/// to be called `index` is never mistaken for one.+fn collect_query_body_indices(q: &Value, out: &mut Vec<Demand>) {+ match q {+ Value::Object(obj) => {+ if let Some(Value::Object(terms)) = obj.get("terms") {+ for (field, spec) in terms {+ if field == "boost" {+ continue;+ }+ if let Some(s) = spec.as_object() {+ // Only a full lookup triple actually fetches.+ if s.contains_key("id") && s.contains_key("path") {+ if let Some(ix) = s.get("index").and_then(Value::as_str) {+ out.push((ix.to_string(), Privilege::ReadIndex));+ }+ }+ }+ }+ }+ if obj.get("type").and_then(Value::as_str) == Some("lookup") {+ if let Some(ix) = obj.get("target_index").and_then(Value::as_str) {+ out.push((ix.to_string(), Privilege::ReadIndex));+ }+ }+ for child in obj.values() {+ collect_query_body_indices(child, out);+ }+ }+ Value::Array(items) => {+ for item in items {+ collect_query_body_indices(item, out);+ }+ }+ _ => {}+ }+}++// ─────────────────────────────────────────────────────────────────────────────+// Alias resolution+// ─────────────────────────────────────────────────────────────────────────────++/// Resolves an alias to the indices behind it.+///+/// Authorization consults this **before** deciding, so `POST /pwned/_search`+/// on an alias pointed at someone else's brain is authorized against that+/// brain, not against the alias name. `Engine::get_index` resolves aliases too+/// and re-checks after doing so, which is what covers an alias created between+/// the decision and the read.+pub trait AliasResolver {+ fn targets(&self, name: &str) -> Option<Vec<String>>;+}++impl AliasResolver for AppState {+ fn targets(&self, name: &str) -> Option<Vec<String>> {+ self.engine.aliases.get(name).map(|e| e.value().clone())+ }+}++/// No aliases at all — used by the unit tests and as the fallback when a+/// decision is made without engine state.+pub struct NoAliases;++impl AliasResolver for NoAliases {+ fn targets(&self, _name: &str) -> Option<Vec<String>> {+ None+ }+}++/// Authorize one index expression, resolving aliases first.+///+/// A pattern is *not* refused: it is expanded over the principal's visible set+/// by the engine (see [`xerj_engine::index_guard`]), so a granted `logs-*`+/// works and an ungranted `*` silently resolves to nothing instead of leaking+/// what it would have matched.+fn authorize_expression(+ principal: &Principal,+ aliases: &dyn AliasResolver,+ expr: &str,+ privilege: Privilege,+) -> Result<(), Response> {+ if is_pattern(expr) {+ return Ok(());+ }+ match aliases.targets(expr) {+ Some(backing) if !backing.is_empty() => {+ for index in backing {+ authorize_index(principal, &index, privilege)?;+ }+ Ok(())+ }+ _ => authorize_index(principal, expr, privilege),+ }+}++// ─────────────────────────────────────────────────────────────────────────────+// Middleware+// ─────────────────────────────────────────────────────────────────────────────++/// Largest response this middleware will buffer in order to prune reserved+/// index names out of it. Metadata listings are kilobytes; the cap only exists+/// so a pathological response cannot be turned into an OOM.+const MAX_PRUNABLE_RESPONSE_BYTES: usize = 64 * 1024 * 1024;++/// Authorization middleware for both routers.+///+/// Layered *inside* [`crate::auth::auth_middleware`] (added to the router+/// before it, so it runs after it): authentication has already rejected+/// anonymous callers, and this decides what the authenticated one may reach.+///+/// A superuser — open mode (`--insecure`, point-at-a-folder) or the configured+/// admin key — short-circuits on the first line, so the zero-configuration+/// local path pays nothing: no body buffering, no visibility guard, no+/// response pruning.+pub async fn authz_middleware(State(state): State<AppState>, req: Request, next: Next) -> Response {+ let path = req.uri().path().to_string();+ let target = classify(&path);+ if matches!(target, Target::Exempt) {+ return next.run(req).await;+ }++ let principal = authenticate(+ &state,+ req.headers()+ .get(header::AUTHORIZATION)+ .and_then(|v| v.to_str().ok()),+ );+ if principal.is_superuser() {+ return next.run(req).await;+ }++ let segs = segments(&path);+ let method = req.method().clone();+ let shape = body_shape(&method, &segs);++ // Buffer the body only for the routes that can name an index inside it.+ // `Bytes` is refcounted, so handing the same buffer to the handler costs a+ // clone of a pointer, not of the payload.+ let (req, body) = if shape == BodyShape::None {+ (req, Bytes::new())+ } else {+ let limit = state.config.limits.max_body_bytes;+ let (parts, raw) = req.into_parts();+ match axum::body::to_bytes(raw, limit).await {+ Ok(bytes) => (Request::from_parts(parts, Body::from(bytes.clone())), bytes),+ Err(_) => {+ return es_error(+ StatusCode::PAYLOAD_TOO_LARGE,+ "request body exceeds limits.max_body_bytes and cannot be authorized"+ .to_string(),+ )+ }+ }+ };++ if let Err(denied) = decide(&principal, &method, &segs, &target, shape, &body, &state) {+ return denied;+ }++ // Layer 2: whatever the handler resolves from here on — including names+ // this module never parsed — is checked against the same principal at the+ // engine's index funnel.+ let response =+ xerj_engine::index_guard::scoped(visibility_for(&principal), next.run(req)).await;+ prune_response(response, &segs, &principal, &state).await+}++/// The whole decision, split out so it can be unit-tested without a router.+#[allow(clippy::too_many_arguments)]+fn decide(+ principal: &Principal,+ method: &Method,+ segs: &[String],+ target: &Target,+ shape: BodyShape,+ body: &[u8],+ aliases: &dyn AliasResolver,+) -> Result<(), Response> {+ // Restated here and not only in the middleware, so this function is a+ // complete decision on its own and cannot deny the local-dev superuser if+ // it is ever called from somewhere else.+ if principal.is_superuser() {+ return Ok(());+ }+ if matches!(principal, Principal::Denied) {+ return Err(forbidden(principal, "<cluster>", Privilege::ReadIndex));+ }+ match target {+ Target::Exempt => Ok(()),+ Target::Brain(brain) => {+ authorize_brain(principal, brain, reserved_api_privilege(method, segs))+ }+ Target::Memory(ns) => {+ authorize_memory_namespace(principal, ns, reserved_api_privilege(method, segs))+ }+ Target::Indices(expressions, op_start) => {+ let privilege = required_privilege(method, segs, *op_start);+ for expr in expressions {+ authorize_expression(principal, aliases, expr, privilege)?;+ }+ // `PUT|DELETE /{index}/_alias/{alias}` mints or drops an alias+ // name; the name itself is a resource, or the reserved namespace+ // could be squatted by an alias nobody authorized.+ if segs.get(*op_start).map(String::as_str) == Some("_alias") {+ if let Some(alias) = segs.get(op_start + 1) {+ authorize_index(principal, alias, Privilege::AdminIndex)?;+ }+ }+ // The path index is the default for a body that may override it —+ // `/{index}/_bulk`, `/{index}/_msearch`, `/{index}/_mget`.+ let default = expressions.first().map(String::as_str);+ authorize_body(principal, aliases, shape, body, default)+ }+ Target::Cluster => {+ // Snapshot and restore that name no indices cover *every* index on+ // the node, brains included. There is no per-index decision to+ // make, so they are superuser-only.+ let snapshotting = segs.first().map(String::as_str) == Some("_snapshot")+ && segs.len() >= 3+ && method != Method::GET+ && method != Method::HEAD;+ let restoring = segs.last().map(String::as_str) == Some("_restore");+ if shape == BodyShape::IndexTemplate {+ authorize_template_patterns(principal, body)?;+ }+ let mut demands = body_targets(principal, shape, body, None)?;+ if restoring {+ // A restore overwrites the indices it names.+ for demand in demands.iter_mut() {+ demand.1 = Privilege::WriteIndex;+ }+ }+ if snapshotting && demands.is_empty() {+ return Err(forbidden(+ principal,+ "<all indices>",+ if restoring {+ Privilege::SnapshotRestore+ } else {+ Privilege::SnapshotCreate+ },+ ));+ }+ for (expr, privilege) in &demands {+ authorize_expression(principal, aliases, expr, *privilege)?;+ }+ if cluster_is_read(method, segs) {+ // Cluster reads (`_cat`, `_mapping`, `_nodes`, `_cluster/*`,+ // and the global `_search`/`_count`/`_msearch`/`_mget`) are+ // answered over the principal's visible set, so a scoped key+ // gets a filtered cluster rather than a blanket 403 — which is+ // what Kibana needs to function at all.+ return Ok(());+ }+ if !demands.is_empty() {+ // A mutating cluster verb that named its targets: they are all+ // authorized above. `POST /_bulk`, `/_reindex` and `/_aliases`+ // live here — the global bulk is the default write path for+ // essentially every ES client.+ return Ok(());+ }+ if shape == BodyShape::MlConfig {+ // An ML config that named no index inherits one that was+ // authorized when the detector was created (see the `MlConfig`+ // arm of `body_targets`). Falling through to the+ // names-nothing branch below would 403 a scoped key for the+ // ordinary `PUT /_ml/datafeeds/{id} {"job_id": …}`, which+ // reaches nothing this principal was not already granted.+ return Ok(());+ }+ match principal {+ // A cluster-level mutation that names nothing: an unscoped key+ // keeps its historical reach over the general surface, a scoped+ // one has to name what it is changing.+ Principal::Unscoped { .. } => Ok(()),+ _ => Err(forbidden(+ principal,+ "<cluster>",+ required_privilege(method, segs, 1),+ )),+ }+ }+ }+}++/// Authorize the indices a request's body names, if any.+fn authorize_body(+ principal: &Principal,+ aliases: &dyn AliasResolver,+ shape: BodyShape,+ body: &[u8],+ default_index: Option<&str>,+) -> Result<(), Response> {+ for (expr, privilege) in body_targets(principal, shape, body, default_index)? {+ authorize_expression(principal, aliases, &expr, privilege)?;+ }+ Ok(())+}++// ─────────────────────────────────────────────────────────────────────────────+// Enumeration pruning+//+// Which endpoints get pruned is decided by `prune_sites` / `cat_index_columns`+// below rather than by a list of path roots: an endpoint is prunable exactly+// when a position is known for it, so "we prune here" and "here is where the+// names are" cannot drift apart. Everything else — every search response+// included — is passed through without being buffered.+// ─────────────────────────────────────────────────────────────────────────────++/// Remove index names the principal cannot read from a metadata response.+///+/// `Engine::list_indices` already filters, so most listings arrive clean; this+/// is the second pass for the handful of handlers that read the+/// `index_settings` / `index_mappings` side maps directly, and it is what keeps+/// `GET /_mapping` from handing over the list of brains on the node.+async fn prune_response(+ response: Response,+ segs: &[String],+ principal: &Principal,+ state: &AppState,+) -> Response {+ let sites = prune_sites(segs);+ let columns = cat_index_columns(segs).unwrap_or(&[]);+ if sites.is_empty() && columns.is_empty() {+ // Nothing in this response names an index. Skip the buffer entirely —+ // it is also the only correct thing to do: pruning a body with no+ // index-name position in it can subtract, never protect.+ return response;+ }+ let (parts, body) = response.into_parts();+ let content_type = parts+ .headers+ .get(header::CONTENT_TYPE)+ .and_then(|v| v.to_str().ok())+ .unwrap_or("")+ .to_string();+ let is_json = content_type.contains("json");+ let is_text = content_type.starts_with("text/");+ if !is_json && !is_text {+ return Response::from_parts(parts, body);+ }+ let bytes = match axum::body::to_bytes(body, MAX_PRUNABLE_RESPONSE_BYTES).await {+ Ok(b) => b,+ // Could not buffer it, so could not prune it. Refuse rather than ship+ // an unfiltered listing.+ Err(_) => {+ return es_error(+ StatusCode::INTERNAL_SERVER_ERROR,+ "response too large to authorize; name an explicit index".to_string(),+ )+ }+ };++ // The real index names on the node. A response key is only a candidate for+ // pruning when it IS one — otherwise `"mappings"`, `"settings"` and every+ // other structural key would be dropped for a scoped principal, which+ // would corrupt the very responses this is protecting. Read outside the+ // request's visibility scope on purpose: this needs the unfiltered truth.+ let known: HashSet<String> = state.engine.index_name_list().into_iter().collect();++ let pruned: Vec<u8> = if is_json {+ match serde_json::from_slice::<Value>(&bytes) {+ Ok(mut v) => {+ prune_json(&mut v, sites, principal, &known);+ serde_json::to_vec(&v).unwrap_or_else(|_| bytes.to_vec())+ }+ // Not JSON after all (some `_cat` handlers answer text under a+ // JSON content type); fall through to the line filter.+ Err(_) => prune_text(&bytes, columns, principal, &known),+ }+ } else {+ prune_text(&bytes, columns, principal, &known)+ };++ let mut parts = parts;+ parts.headers.remove(header::CONTENT_LENGTH);+ Response::from_parts(parts, Body::from(pruned))+}++/// Is this index name one the principal must not even see?+fn hidden(name: &str, principal: &Principal, known: &HashSet<String>) -> bool {+ (is_reserved_index(name) || known.contains(name))+ && !principal.allows_index(name, Privilege::ReadIndex)+}++/// One place in an endpoint's response where index **names** appear.+///+/// This exists because the first cut pruned by *key name at every depth*: any+/// key equal to a real index name was deleted, wherever it appeared. An index+/// name is an arbitrary string, so ordinary names collide with the structural+/// keys of the very responses being pruned, and the collision only bites a+/// scoped key — i.e. exactly the multi-tenant and Kibana case this whole+/// branch exists to enable. Measured, on a node with one ordinary index of+/// each name: `status` cost `GET /_cluster/health` its `status` field, the+/// single most-polled field in the API; `indices` cost `GET /_cluster/stats`+/// its entire `indices` section; `type` cost global `GET /_mapping` the `type`+/// of every field definition, which is the endpoint Kibana reads to build+/// index patterns.+///+/// So the positions are enumerated per endpoint instead, from the handlers'+/// actual response shapes. `path` walks object keys from the document root,+/// `/`-separated, where `*` matches any one key or array element and `""` is+/// the root itself.+#[derive(Clone, Copy, Debug, PartialEq, Eq)]+enum Site {+ /// The object at `path` is **keyed** by index name.+ KeyedByIndex(&'static str),+ /// The array at `path` **holds** index names: bare strings, or objects+ /// whose `field` names one (`""` = bare strings only).+ NamedIn(&'static str, &'static str),+}++/// Where index names appear in this endpoint's JSON response.+///+/// Anything not listed prunes nothing, which is the right default: a response+/// that never names an index cannot leak one, and pruning it can only damage+/// it. `_cluster/stats` is the worked example — its `indices` object holds+/// aggregate counts, not a map keyed by index — and so are `_cluster/settings`,+/// `_cluster/pending_tasks`, `_nodes` and `/v1/cluster/health`.+fn prune_sites(segs: &[String]) -> &'static [Site] {+ // `_mapping`, `_settings`, `_alias`, `_aliases`, `_recovery` and+ // `_shard_stores` all answer one object keyed by index name at the root.+ const ROOT_KEYED: &[Site] = &[Site::KeyedByIndex("")];+ // `_stats` and `_segments` put the same map one level down.+ const INDICES_KEYED: &[Site] = &[Site::KeyedByIndex("indices")];+ match segs.first().map(String::as_str) {+ Some("_mapping")+ | Some("_mappings")+ | Some("_settings")+ | Some("_alias")+ | Some("_aliases")+ | Some("_recovery")+ | Some("_shard_stores") => ROOT_KEYED,+ Some("_stats") | Some("_segments") => INDICES_KEYED,+ Some("_cluster") => match segs.get(1).map(String::as_str) {+ // `level=indices` / `level=shards` add the per-index breakdown;+ // the root is cluster-wide scalars (`status`, `timed_out`, …).+ Some("health") => INDICES_KEYED,+ Some("state") => &[+ Site::KeyedByIndex("metadata/indices"),+ Site::KeyedByIndex("routing_table/indices"),+ ],+ _ => &[],+ },+ // `{"indices": [name], "fields": {f: {type: {…, "indices": [name]}}}}`+ Some("_field_caps") => &[+ Site::NamedIn("indices", ""),+ Site::NamedIn("fields/*/*/indices", ""),+ ],+ // `{"indices": [{name}], "aliases": [{name, indices}], "data_streams": []}`+ Some("_resolve") => &[+ Site::NamedIn("indices", "name"),+ Site::NamedIn("aliases", "name"),+ Site::NamedIn("aliases/*/indices", ""),+ Site::NamedIn("data_streams", "name"),+ Site::NamedIn("data_streams/*/backing_indices", ""),+ ],+ // `{"data_streams": [{name, indices: [{index_name}]}]}`+ Some("_data_stream") => &[+ Site::NamedIn("data_streams", "name"),+ Site::NamedIn("data_streams/*/indices", "index_name"),+ ],+ // A `_cat` table in its `format=json` form is an array of row objects.+ // Only the per-index tables have an `index` column — `_cat/templates`+ // keys its rows by TEMPLATE name, which is not an index name and must+ // not be matched against one.+ Some("_cat") if cat_index_columns(segs).is_some() => &[Site::NamedIn("", "index")],+ Some("v1") => match segs.get(1).map(String::as_str) {+ // `{"data": {"indices": [{name, …}], …}}`+ Some("dashboard") => &[Site::NamedIn("data/indices", "name")],+ _ => &[],+ },+ _ => &[],+ }+}++/// The whitespace-separated column(s) holding an index name in a `_cat`+/// table's plain-text form, by sub-verb; `None` means this table has no index+/// column, so no row of it may be dropped for containing an index name.+///+/// Column numbers are read off the handlers in `es_compat`, not guessed. The+/// alternative — "drop a row if ANY token is a hidden index name" — has the+/// same collision as key-matching did: an ordinary index named `open`, `green`+/// or `p` that a scoped key cannot read would empty `_cat/indices` and+/// `_cat/shards` completely.+fn cat_index_columns(segs: &[String]) -> Option<&'static [usize]> {+ match segs.get(1).map(String::as_str)? {+ // health status INDEX uuid pri rep docs.count docs.deleted store …+ "indices" => Some(&[2]),+ // ALIAS INDEX filter routing.index routing.search is_write_index.+ // Column 0 is an alias, not an index — but an alias name IS a name in+ // this namespace elsewhere in this module (it is authorized as a+ // resource, and it resolves to indices), so a reserved-namespace alias+ // is hidden here too.+ "aliases" => Some(&[0, 1]),+ // INDEX shard prirep state docs store ip node+ "shards" => Some(&[0]),+ // INDEX shard time type stage …+ "recovery" => Some(&[0]),+ // INDEX shard prirep ip segment …+ "segments" => Some(&[0]),+ // INDEX present field nodes tombstones …+ "ann" => Some(&[0]),+ // id host ip node INDEX size+ "fielddata" => Some(&[4]),+ // Everything else under `_cat` — nodes, health, master, plugins,+ // templates, thread_pool, nodeattrs, pending_tasks, allocation, count,+ // ml/* — reports no index name at all.+ _ => None,+ }+}++/// Apply `f` to every node the site `path` selects. `*` matches any one object+/// key or array element; an empty path selects the node itself.+fn for_each_at(v: &mut Value, path: &str, f: &mut dyn FnMut(&mut Value)) {+ if path.is_empty() {+ f(v);+ return;+ }+ let (head, rest) = path.split_once('/').unwrap_or((path, ""));+ match v {+ Value::Object(map) => {+ if head == "*" {+ for (_, child) in map.iter_mut() {+ for_each_at(child, rest, f);+ }+ } else if let Some(child) = map.get_mut(head) {+ for_each_at(child, rest, f);+ }+ }+ Value::Array(items) if head == "*" => {+ for item in items.iter_mut() {+ for_each_at(item, rest, f);+ }+ }+ _ => {}+ }+}++/// Drop the index names this principal may not read, at the positions this+/// endpoint actually carries them — and nowhere else.+fn prune_json(v: &mut Value, sites: &[Site], principal: &Principal, known: &HashSet<String>) {+ for site in sites {+ match *site {+ Site::KeyedByIndex(path) => for_each_at(v, path, &mut |node| {+ if let Value::Object(map) = node {+ map.retain(|k, _| !hidden(k, principal, known));+ }+ }),+ Site::NamedIn(path, field) => for_each_at(v, path, &mut |node| {+ if let Value::Array(items) = node {+ items.retain(|item| match item {+ Value::String(s) => !hidden(s, principal, known),+ Value::Object(o) if !field.is_empty() => !o+ .get(field)+ .and_then(Value::as_str)+ .map(|n| hidden(n, principal, known))+ .unwrap_or(false),+ _ => true,+ });+ }+ }),+ }+ }+}++/// Line filter for `_cat` in its default text form: a row whose index column+/// names a hidden index is dropped whole. A table with no index column+/// (`columns` is empty) is returned untouched.+fn prune_text(+ bytes: &[u8],+ columns: &[usize],+ principal: &Principal,+ known: &HashSet<String>,+) -> Vec<u8> {+ if columns.is_empty() {+ return bytes.to_vec();+ }+ let Ok(text) = std::str::from_utf8(bytes) else {+ return bytes.to_vec();+ };+ let mut out = String::with_capacity(text.len());+ let mut dropped_any = false;+ for line in text.split_inclusive('\n') {+ let fields: Vec<&str> = line.split_whitespace().collect();+ let hides = columns.iter().any(|&c| {+ fields+ .get(c)+ .map(|token| {+ hidden(+ token.trim_matches(|ch: char| ch == '"' || ch == ','),+ principal,+ known,+ )+ })+ .unwrap_or(false)+ });+ if hides {+ dropped_any = true;+ } else {+ out.push_str(line);+ }+ }+ if dropped_any {+ out.into_bytes()+ } else {+ bytes.to_vec()+ }+}++// ─────────────────────────────────────────────────────────────────────────────+// Tests+// ─────────────────────────────────────────────────────────────────────────────++#[cfg(test)]+mod tests {+ use super::*;+ use std::collections::HashMap;+ use xerj_engine::rbac::Role;++ fn scoped(indices: &[&str], privileges: &[Privilege]) -> Principal {+ Principal::Scoped {+ key_id: "k1".into(),+ roles: vec![Role::new(+ "r",+ privileges.iter().copied().collect::<HashSet<_>>(),+ indices.iter().map(|s| s.to_string()).collect(),+ )],+ }+ }++ fn unscoped() -> Principal {+ Principal::Unscoped {+ key_id: "k2".into(),+ }+ }++ struct Map(HashMap<String, Vec<String>>);+ impl AliasResolver for Map {+ fn targets(&self, name: &str) -> Option<Vec<String>> {+ self.0.get(name).cloned()+ }+ }++ /// Run the whole decision the way the middleware does.+ fn check(p: &Principal, method: Method, path: &str, body: &str) -> bool {+ check_with(p, method, path, body, &NoAliases)+ }++ fn check_with(+ p: &Principal,+ method: Method,+ path: &str,+ body: &str,+ aliases: &dyn AliasResolver,+ ) -> bool {+ let segs = segments(path);+ let target = classify(path);+ let shape = body_shape(&method, &segs);+ decide(p, &method, &segs, &target, shape, body.as_bytes(), aliases).is_ok()+ }++ #[test]+ fn reserved_prefix_recognition() {+ assert!(is_reserved_index(".xerj-memory-alice-edges"));+ assert!(is_reserved_index(".xerj-memory-alice"));+ assert!(!is_reserved_index("logs-2026"));+ assert!(!is_reserved_index(".kibana"));+ }++ #[test]+ fn wildcards_that_could_reach_the_namespace_are_recognized() {+ for pattern in ["*", "_all", ".*", ".xerj-*", ".xerj-memory-alice*", ".x*"] {+ assert!(may_reach_reserved(pattern), "{pattern} must be caught");+ }+ for pattern in ["logs-*", "notes", ".kibana*", ".xerj-other-*"] {+ assert!(!may_reach_reserved(pattern), "{pattern} must be allowed");+ }+ }++ #[test]+ fn percent_encoded_reserved_names_are_decoded() {+ // `%2E` is `.` — without decoding, the leading-dot check would miss it.+ assert_eq!(+ percent_decode("%2Exerj-memory-alice-edges"),+ ".xerj-memory-alice-edges"+ );+ assert_eq!(+ classify("/%2Exerj-memory-alice-edges/_search"),+ Target::Indices(vec![".xerj-memory-alice-edges".into()], 1)+ );+ }++ #[test]+ fn classification() {+ assert_eq!(classify("/_graph/alice/ego"), Target::Brain("alice".into()));+ assert_eq!(+ classify("/_memory/alice/_recall"),+ Target::Memory("alice".into())+ );+ assert_eq!(classify("/_search"), Target::Cluster);+ assert_eq!(classify("/_bulk"), Target::Cluster);+ assert_eq!(classify("/_mapping"), Target::Cluster);+ assert_eq!(classify("/_cat/indices"), Target::Cluster);+ // A pattern is an index expression, not a cluster endpoint.+ assert_eq!(+ classify("/_all/_search"),+ Target::Indices(vec!["_all".into()], 1)+ );+ assert_eq!(classify("/*/_search"), Target::Indices(vec!["*".into()], 1));+ assert_eq!(classify("/health/live"), Target::Exempt);+ assert_eq!(classify("/"), Target::Exempt);+ assert_eq!(+ classify("/logs-a,logs-b/_search"),+ Target::Indices(vec!["logs-a".into(), "logs-b".into()], 1)+ );+ // Native router: the index is the third segment, not the first.+ assert_eq!(+ classify("/v1/indices/.xerj-memory-bob-edges/search"),+ Target::Indices(vec![".xerj-memory-bob-edges".into()], 3)+ );+ assert_eq!(classify("/v1/dashboard/summary"), Target::Cluster);+ assert_eq!(classify("/v1/indices"), Target::Cluster);+ }++ /// Every route that can take an index name from its body must be mapped to+ /// the shape that finds it. This is the audit, executable.+ #[test]+ fn body_shape_table() {+ let cases: &[(&str, &str, BodyShape)] = &[+ ("POST", "/_bulk", BodyShape::Bulk),+ ("POST", "/logs/_bulk", BodyShape::Bulk),+ ("POST", "/_msearch", BodyShape::MsearchHeaders),+ ("POST", "/logs/_msearch", BodyShape::MsearchHeaders),+ ("POST", "/_msearch/template", BodyShape::MsearchHeaders),+ ("POST", "/logs/_msearch/template", BodyShape::MsearchHeaders),+ ("POST", "/_mget", BodyShape::MgetDocs),+ ("POST", "/logs/_mget", BodyShape::MgetDocs),+ ("POST", "/_aliases", BodyShape::AliasActions),+ ("POST", "/_reindex", BodyShape::Reindex),+ ("PUT", "/_snapshot/repo/snap", BodyShape::SnapshotIndices),+ (+ "POST",+ "/_snapshot/repo/snap/_restore",+ BodyShape::SnapshotIndices,+ ),+ ("POST", "/v1/indices", BodyShape::NativeCreate),+ ("PUT", "/logs-2026", BodyShape::CreateIndex),+ ("POST", "/_monitoring/bulk", BodyShape::MonitoringBulk),+ ("PUT", "/_index_template/t", BodyShape::IndexTemplate),+ ("PUT", "/_template/t", BodyShape::IndexTemplate),+ (+ "POST",+ "/_index_template/_simulate",+ BodyShape::IndexTemplate,+ ),+ ("PUT", "/_ml/anomaly_detectors/j", BodyShape::MlConfig),+ ("PUT", "/_ml/datafeeds/f", BodyShape::MlConfig),+ ("POST", "/_ml/datafeeds/f", BodyShape::MlConfig),+ // Sub-verbs and reads carry no index name of their own.+ ("POST", "/_ml/datafeeds/f/_start", BodyShape::None),+ ("POST", "/_ml/anomaly_detectors/j/_score", BodyShape::None),+ ("GET", "/_ml/anomaly_detectors/j", BodyShape::None),+ ("POST", "/_search", BodyShape::Query),+ ("POST", "/logs/_search", BodyShape::Query),+ ("POST", "/_count", BodyShape::Query),+ ("POST", "/logs/_delete_by_query", BodyShape::Query),+ ("POST", "/logs/_update_by_query", BodyShape::Query),+ ("POST", "/_sql", BodyShape::Query),+ ("POST", "/logs/_eql/search", BodyShape::Query),+ // Opaque document bodies: nothing in them is an index name.+ ("PUT", "/logs/_doc/1", BodyShape::None),+ ("POST", "/logs/_update/1", BodyShape::None),+ ("POST", "/v1/indices/logs/docs", BodyShape::None),+ ("POST", "/v1/indices/logs/docs/_bulk", BodyShape::None),+ ];+ for (method, path, want) in cases {+ let m: Method = method.parse().unwrap();+ let got = body_shape(&m, &segments(path));+ assert_eq!(got, *want, "{method} {path}");+ }+ }++ #[test]+ fn privilege_mapping() {+ let s = |p: &str| segments(p);+ assert_eq!(+ required_privilege(&Method::POST, &s("/idx/_search"), 1),+ Privilege::ReadIndex+ );+ assert_eq!(+ required_privilege(&Method::POST, &s("/idx/_doc/1"), 1),+ Privilege::WriteIndex+ );+ assert_eq!(+ required_privilege(&Method::DELETE, &s("/idx"), 1),+ Privilege::AdminIndex+ );+ assert_eq!(+ required_privilege(&Method::PUT, &s("/idx/_settings"), 1),+ Privilege::AdminIndex+ );+ // Native spellings resolve to the same privileges.+ assert_eq!(+ required_privilege(&Method::POST, &s("/v1/indices/idx/search"), 3),+ Privilege::ReadIndex+ );+ assert_eq!(+ required_privilege(&Method::POST, &s("/v1/indices/idx/docs"), 3),+ Privilege::WriteIndex+ );+ assert_eq!(+ required_privilege(&Method::DELETE, &s("/v1/indices/idx"), 3),+ Privilege::AdminIndex+ );+ assert_eq!(+ reserved_api_privilege(&Method::POST, &s("/_graph/a/link")),+ Privilege::WriteIndex+ );+ assert_eq!(+ reserved_api_privilege(&Method::GET, &s("/_graph/a/ego")),+ Privilege::ReadIndex+ );+ assert_eq!(+ reserved_api_privilege(&Method::DELETE, &s("/_memory/a")),+ Privilege::AdminIndex+ );+ assert_eq!(+ reserved_api_privilege(&Method::DELETE, &s("/_memory/a/doc1")),+ Privilege::WriteIndex+ );+ }++ /// The decision table, stated once. `decide` is the only thing between an+ /// authenticated caller and someone else's brain.+ #[test]+ fn decision_table() {+ let alice = scoped(+ &[".xerj-memory-alice-edges", ".xerj-memory-alice"],+ &[Privilege::ReadIndex, Privilege::WriteIndex],+ );++ // Alice reaches her own brain …+ assert!(check(&alice, Method::GET, "/_graph/alice/ego", ""));+ assert!(check(&alice, Method::GET, "/_graph/alice/overview", ""));+ assert!(check(&alice, Method::POST, "/_graph/alice/link", ""));+ assert!(check(&alice, Method::POST, "/_memory/alice/_recall", ""));+ // … and nothing of bob's, by any door.+ assert!(!check(&alice, Method::GET, "/_graph/bob/ego", ""));+ assert!(!check(&alice, Method::POST, "/_graph/bob/link", ""));+ assert!(!check(&alice, Method::DELETE, "/_graph/bob/link/e1", ""));+ assert!(!check(+ &alice,+ Method::POST,+ "/.xerj-memory-bob-edges/_search",+ ""+ ));+ assert!(!check(+ &alice,+ Method::POST,+ "/.xerj-memory-bob-edges/_doc/x",+ ""+ ));+ assert!(!check(+ &alice,+ Method::DELETE,+ "/.xerj-memory-bob-edges",+ ""+ ));+ assert!(!check(&alice, Method::POST, "/_memory/bob/_recall", ""));+ // Including the native router's spelling of the same index.+ assert!(!check(+ &alice,+ Method::POST,+ "/v1/indices/.xerj-memory-bob-edges/search",+ ""+ ));+ // A grant is a grant: alice may use her own backing index directly.+ assert!(check(+ &alice,+ Method::POST,+ "/.xerj-memory-alice-edges/_search",+ ""+ ));++ // A read-only grant does not write.+ let ro = scoped(&[".xerj-memory-alice-edges"], &[Privilege::ReadIndex]);+ assert!(check(&ro, Method::GET, "/_graph/alice/ego", ""));+ assert!(!check(&ro, Method::POST, "/_graph/alice/link", ""));+ assert!(!check(&ro, Method::DELETE, "/.xerj-memory-alice-edges", ""));++ // A legacy key with no grants: keeps ordinary indices, loses every+ // brain door. This is the fail-closed half.+ let legacy = unscoped();+ assert!(check(&legacy, Method::POST, "/logs-2026/_search", ""));+ assert!(check(&legacy, Method::GET, "/logs-*/_search", ""));+ assert!(check(&legacy, Method::GET, "/_mapping", ""));+ assert!(!check(&legacy, Method::GET, "/_graph/alice/ego", ""));+ assert!(!check(&legacy, Method::POST, "/_memory/alice/_recall", ""));+ assert!(!check(+ &legacy,+ Method::POST,+ "/.xerj-memory-alice-edges/_search",+ ""+ ));+ assert!(check(&legacy, Method::PUT, "/logs-2026", ""));++ // No credential resolves to nothing at all.+ let denied = Principal::Denied;+ assert!(!check(&denied, Method::GET, "/_graph/alice/ego", ""));+ assert!(!check(&denied, Method::GET, "/logs/_search", ""));+ assert!(!check(&denied, Method::GET, "/_mapping", ""));++ // The superuser is untouched — this is the `--insecure` path.+ let root = Principal::Superuser;+ for path in [+ "/_graph/bob/ego",+ "/_mapping",+ "/_search",+ "/.xerj-memory-bob-edges/_search",+ "/*/_search",+ ] {+ assert!(+ check(&root, Method::GET, path, ""),+ "superuser blocked on {path}"+ );+ }+ }++ /// BYPASS 1-3: the index in the body is the one that gets authorized.+ #[test]+ fn body_named_indices_are_authorized_not_the_path_one() {+ let alice = scoped(+ &[".xerj-memory-alice-edges", ".xerj-memory-alice"],+ &[Privilege::ReadIndex, Privilege::WriteIndex],+ );++ // _msearch: the header line overrides the path index.+ let msearch = "{\"index\":\".xerj-memory-bob-edges\"}\n{\"query\":{\"match_all\":{}}}\n";+ assert!(!check(+ &alice,+ Method::POST,+ "/.xerj-memory-alice-edges/_msearch",+ msearch+ ));+ assert!(!check(&alice, Method::POST, "/_msearch", msearch));+ // Her own brain through the same door still works.+ let mine = "{\"index\":\".xerj-memory-alice-edges\"}\n{\"query\":{\"match_all\":{}}}\n";+ assert!(check(&alice, Method::POST, "/_msearch", mine));++ // _bulk: the action line's `_index` overrides the path index.+ let forge =+ "{\"index\":{\"_index\":\".xerj-memory-bob-edges\",\"_id\":\"f\"}}\n{\"src\":\"x\"}\n";+ assert!(!check(+ &alice,+ Method::POST,+ "/.xerj-memory-alice-edges/_bulk",+ forge+ ));+ assert!(!check(&alice, Method::POST, "/_bulk", forge));+ let destroy = "{\"delete\":{\"_index\":\".xerj-memory-bob-edges\",\"_id\":\"e1\"}}\n";+ assert!(!check(&alice, Method::POST, "/_bulk", destroy));+ // A bulk that stays inside her grant is allowed — the global `_bulk`+ // is the default write path for every ES client and must work.+ let ok = "{\"index\":{\"_index\":\".xerj-memory-alice-edges\",\"_id\":\"a\"}}\n{\"src\":\"x\"}\n";+ assert!(check(&alice, Method::POST, "/_bulk", ok));++ // _mget: docs[]._index overrides the path index.+ let mget = "{\"docs\":[{\"_index\":\".xerj-memory-bob-edges\",\"_id\":\"1\"}]}";+ assert!(!check(+ &alice,+ Method::POST,+ "/.xerj-memory-alice-edges/_mget",+ mget+ ));+ assert!(!check(&alice, Method::POST, "/_mget", mget));+ assert!(check(+ &alice,+ Method::POST,+ "/.xerj-memory-alice-edges/_mget",+ "{\"ids\":[\"1\"]}"+ ));++ // An unscoped legacy key holds nothing in the reserved namespace by+ // any of the three doors either.+ let legacy = unscoped();+ assert!(!check(&legacy, Method::POST, "/_msearch", msearch));+ assert!(!check(&legacy, Method::POST, "/_bulk", forge));+ assert!(!check(&legacy, Method::POST, "/_mget", mget));+ // …but its ordinary-index traffic is untouched.+ let plain = "{\"index\":{\"_index\":\"logs-2026\"}}\n{\"m\":1}\n";+ assert!(check(&legacy, Method::POST, "/_bulk", plain));+ }++ /// BYPASS 4: `POST /_aliases` authorizes the indices it names, and an+ /// alias already pointing into the namespace does not launder reads.+ #[test]+ fn aliases_cannot_launder_access() {+ let legacy = unscoped();+ let add =+ "{\"actions\":[{\"add\":{\"index\":\".xerj-memory-bob-edges\",\"alias\":\"pwned\"}}]}";+ assert!(!check(&legacy, Method::POST, "/_aliases", add));+ // Also refused for a scoped key that holds a different brain.+ let alice = scoped(+ &[".xerj-memory-alice-edges"],+ &[+ Privilege::ReadIndex,+ Privilege::WriteIndex,+ Privilege::AdminIndex,+ ],+ );+ assert!(!check(&alice, Method::POST, "/_aliases", add));+ // An ordinary alias over an ordinary index still works for the legacy+ // key that has the general surface.+ let ordinary = "{\"actions\":[{\"add\":{\"index\":\"logs-2026\",\"alias\":\"logs\"}}]}";+ assert!(check(&legacy, Method::POST, "/_aliases", ordinary));+ // Squatting an alias NAME inside the reserved namespace is refused.+ let squat =+ "{\"actions\":[{\"add\":{\"index\":\"logs-2026\",\"alias\":\".xerj-memory-victim\"}}]}";+ assert!(!check(&legacy, Method::POST, "/_aliases", squat));+ // …by the path spelling too.+ assert!(!check(+ &legacy,+ Method::PUT,+ "/logs-2026/_alias/.xerj-memory-victim",+ ""+ ));++ // Reading THROUGH an alias that already points at a brain resolves to+ // the brain before the decision is made.+ let mut map = HashMap::new();+ map.insert(+ "pwned".to_string(),+ vec![".xerj-memory-bob-edges".to_string()],+ );+ let aliases = Map(map);+ assert!(!check_with(+ &legacy,+ Method::POST,+ "/pwned/_search",+ "",+ &aliases+ ));+ assert!(!check_with(+ &legacy,+ Method::POST,+ "/_mget",+ "{\"docs\":[{\"_index\":\"pwned\",\"_id\":\"1\"}]}",+ &aliases+ ));+ }++ /// The fifth shape, found by audit rather than by attack: a `terms` lookup+ /// reads a document out of an index the URL never names.+ #[test]+ fn terms_lookup_index_is_authorized() {+ let alice = scoped(&["logs-2026"], &[Privilege::ReadIndex]);+ let lookup = r#"{"query":{"bool":{"filter":[{"terms":{"user":{"index":".xerj-memory-bob-edges","id":"1","path":"dst"}}}]}}}"#;+ assert!(!check(&alice, Method::POST, "/logs-2026/_search", lookup));+ // A lookup into an index she holds is fine.+ let own = r#"{"query":{"terms":{"user":{"index":"logs-2026","id":"1","path":"u"}}}}"#;+ assert!(check(&alice, Method::POST, "/logs-2026/_search", own));+ // A plain `terms` filter (no lookup) names no index and is untouched —+ // a document field called `index` must not be read as one.+ let plain = r#"{"query":{"terms":{"index":["a","b"]}}}"#;+ assert!(check(&alice, Method::POST, "/logs-2026/_search", plain));++ // The same shape one layer over: a `lookup` runtime field joins rows+ // out of another index into the hits.+ let runtime = r#"{"runtime_mappings":{"u":{"type":"lookup","target_index":".xerj-memory-bob-edges","input_field":"src","target_field":"_id","fetch_fields":["dst"]}}}"#;+ assert!(!check(&alice, Method::POST, "/logs-2026/_search", runtime));+ let own = r#"{"runtime_mappings":{"u":{"type":"lookup","target_index":"logs-2026","input_field":"src","target_field":"_id"}}}"#;+ assert!(check(&alice, Method::POST, "/logs-2026/_search", own));+ }++ /// `_reindex` reads its source and writes its destination.+ #[test]+ fn reindex_authorizes_source_and_destination() {+ let p = scoped(+ &["logs-2026", "logs-copy"],+ &[Privilege::ReadIndex, Privilege::WriteIndex],+ );+ let ok = r#"{"source":{"index":"logs-2026"},"dest":{"index":"logs-copy"}}"#;+ assert!(check(&p, Method::POST, "/_reindex", ok));+ let steal = r#"{"source":{"index":".xerj-memory-bob-edges"},"dest":{"index":"logs-copy"}}"#;+ assert!(!check(&p, Method::POST, "/_reindex", steal));+ let overwrite =+ r#"{"source":{"index":"logs-2026"},"dest":{"index":".xerj-memory-bob-edges"}}"#;+ assert!(!check(&p, Method::POST, "/_reindex", overwrite));+ // A body that names neither is refused rather than guessed at.+ assert!(!check(&p, Method::POST, "/_reindex", "{}"));+ assert!(!check(&p, Method::POST, "/_reindex", "not json"));+ }++ /// The native router's body-named create is authorized, not blanket-refused.+ #[test]+ fn native_body_named_create_is_authorized() {+ let legacy = unscoped();+ assert!(check(+ &legacy,+ Method::POST,+ "/v1/indices",+ r#"{"name":"logs-2026","fields":[]}"#+ ));+ assert!(!check(+ &legacy,+ Method::POST,+ "/v1/indices",+ r#"{"name":".xerj-memory-victim-edges","fields":[]}"#+ ));+ assert!(!check(&legacy, Method::POST, "/v1/indices", "{}"));+ }++ /// A snapshot or restore that names no indices covers every brain on the+ /// node, so it is superuser-only.+ #[test]+ fn unbounded_snapshot_is_superuser_only() {+ let legacy = unscoped();+ assert!(!check(&legacy, Method::PUT, "/_snapshot/repo/s1", "{}"));+ assert!(!check(+ &legacy,+ Method::POST,+ "/_snapshot/repo/s1/_restore",+ "{}"+ ));+ assert!(check(+ &legacy,+ Method::PUT,+ "/_snapshot/repo/s1",+ r#"{"indices":["logs-2026"]}"#+ ));+ assert!(!check(+ &legacy,+ Method::PUT,+ "/_snapshot/repo/s1",+ r#"{"indices":[".xerj-memory-bob-edges"]}"#+ ));+ assert!(check(+ &Principal::Superuser,+ Method::PUT,+ "/_snapshot/repo/s1",+ "{}"+ ));+ }++ /// COMPATIBILITY: the regression that made the branch unusable. Every+ /// global verb works for an ordinary credential, and a scoped key can read+ /// cluster metadata and use a wildcard it was granted.+ #[test]+ fn ordinary_credentials_keep_the_global_surface() {+ let legacy = unscoped();+ let scoped_all = scoped(+ &["*"],+ &[+ Privilege::ReadIndex,+ Privilege::WriteIndex,+ Privilege::AdminIndex,+ ],+ );+ let logs = scoped(&["logs-*"], &[Privilege::ReadIndex, Privilege::WriteIndex]);++ for p in [&legacy, &scoped_all, &logs] {+ assert!(check(p, Method::POST, "/_search", "{}"), "global _search");+ assert!(check(p, Method::POST, "/_count", "{}"), "global _count");+ assert!(+ check(p, Method::POST, "/_msearch", "{}\n{\"query\":{}}\n"),+ "global _msearch"+ );+ assert!(+ check(+ p,+ Method::POST,+ "/_bulk",+ "{\"index\":{\"_index\":\"logs-2026\"}}\n{\"m\":1}\n"+ ),+ "global _bulk"+ );+ assert!(+ check(+ p,+ Method::POST,+ "/_mget",+ "{\"docs\":[{\"_index\":\"logs-2026\",\"_id\":\"1\"}]}"+ ),+ "global _mget"+ );+ assert!(check(p, Method::GET, "/_cluster/health", ""), "health");+ assert!(check(p, Method::GET, "/_nodes", ""), "nodes");+ assert!(check(p, Method::GET, "/_mapping", ""), "mapping");+ assert!(check(p, Method::GET, "/_cat/indices", ""), "cat");+ assert!(check(p, Method::GET, "/v1/health", ""), "native health");+ assert!(+ check(p, Method::GET, "/_resolve/index/logs-*", ""),+ "resolve"+ );+ }+ // A granted wildcard is usable, and an ungranted one resolves to+ // nothing rather than 403 (the engine expands it over the visible set).+ assert!(check(&logs, Method::GET, "/logs-*/_search", ""));+ assert!(check(&scoped_all, Method::GET, "/logs-*/_search", ""));+ assert!(check(&logs, Method::GET, "/.xerj-memory-*/_search", ""));+ // A scoped key still cannot NAME another tenant's index.+ assert!(!check(+ &logs,+ Method::GET,+ "/.xerj-memory-bob-edges/_search",+ ""+ ));+ // …nor mutate the cluster itself.+ assert!(!check(&logs, Method::PUT, "/_cluster/settings", "{}"));+ assert!(check(&legacy, Method::PUT, "/_cluster/settings", "{}"));+ }++ /// Prune the way the middleware does: pick the endpoint's sites from its+ /// path, then apply them.+ fn prune_at(path: &str, v: &mut Value, p: &Principal, known: &HashSet<String>) {+ prune_json(v, prune_sites(&segments(path)), p, known);+ }++ fn prune_rows(path: &str, table: &str, p: &Principal, known: &HashSet<String>) -> String {+ let cols = cat_index_columns(&segments(path)).unwrap_or(&[]);+ String::from_utf8(prune_text(table.as_bytes(), cols, p, known)).expect("utf8")+ }++ #[test]+ fn json_pruning_hides_unreadable_indices() {+ let p = scoped(&[".xerj-memory-alice-edges"], &[Privilege::ReadIndex]);+ let known: HashSet<String> = [+ ".xerj-memory-alice-edges",+ ".xerj-memory-bob-edges",+ "logs-2026",+ ]+ .iter()+ .map(|s| s.to_string())+ .collect();+ let mut mapping = serde_json::json!({+ ".xerj-memory-alice-edges": {"mappings": {}},+ ".xerj-memory-bob-edges": {"mappings": {}},+ "logs-2026": {"mappings": {}}+ });+ prune_at("/_mapping", &mut mapping, &p, &known);+ assert!(mapping.get(".xerj-memory-alice-edges").is_some());+ assert!(mapping.get(".xerj-memory-bob-edges").is_none());+ // A scoped key does not see another tenant's ordinary index either …+ assert!(mapping.get("logs-2026").is_none());+ // … but structural keys are never mistaken for index names.+ assert!(mapping[".xerj-memory-alice-edges"]+ .get("mappings")+ .is_some());++ // Array shapes: `_cat/indices?format=json` and `_resolve/index`.+ let mut cat = serde_json::json!([+ {"index": ".xerj-memory-bob-edges", "docs.count": "1"},+ {"index": ".xerj-memory-alice-edges", "docs.count": "9"}+ ]);+ prune_at("/_cat/indices", &mut cat, &p, &known);+ assert_eq!(cat.as_array().unwrap().len(), 1);++ let mut caps =+ serde_json::json!({"indices": [".xerj-memory-bob-edges", ".xerj-memory-alice-edges"]});+ prune_at("/_field_caps", &mut caps, &p, &known);+ assert_eq!(+ caps["indices"],+ serde_json::json!([".xerj-memory-alice-edges"])+ );+ // …including the per-field `indices` lists buried under `fields`.+ let mut caps = serde_json::json!({+ "indices": [".xerj-memory-alice-edges", ".xerj-memory-bob-edges"],+ "fields": {"host": {"keyword": {+ "type": "keyword",+ "indices": [".xerj-memory-alice-edges", ".xerj-memory-bob-edges"]+ }}}+ });+ prune_at("/_field_caps", &mut caps, &p, &known);+ assert_eq!(+ caps["fields"]["host"]["keyword"]["indices"],+ serde_json::json!([".xerj-memory-alice-edges"])+ );+ assert_eq!(caps["fields"]["host"]["keyword"]["type"], "keyword");+ }++ /// FINDING B. An index name is an arbitrary string, so the previous+ /// "delete any key equal to a known index name, at every depth" collided+ /// with the structural keys of the responses it was protecting — and only+ /// ever for a scoped key, i.e. exactly the multi-tenant case this branch+ /// exists to enable. Each case below is one measured breakage.+ #[test]+ fn structural_keys_survive_ordinary_indices_that_share_their_name() {+ // Ordinary indices with unfortunate names, none of them readable by+ // this principal.+ let p = scoped(&["logs-2026"], &[Privilege::ReadIndex]);+ let known: HashSet<String> = ["status", "indices", "type", "nodes", "count", "logs-2026"]+ .iter()+ .map(|s| s.to_string())+ .collect();++ // `GET /_cluster/health` kept its `status` — the most-polled field in+ // the API — while still pruning the per-index breakdown.+ let mut health = serde_json::json!({+ "cluster_name": "xerj", "status": "green", "timed_out": false,+ "indices": {"logs-2026": {"status": "green"}, "status": {"status": "green"}}+ });+ prune_at("/_cluster/health", &mut health, &p, &known);+ assert_eq!(health["status"], "green");+ assert!(health["indices"].get("logs-2026").is_some());+ assert!(health["indices"].get("status").is_none());++ // `GET /_cluster/stats` kept its whole `indices` section: those are+ // aggregate counters, not a map keyed by index name.+ let mut stats = serde_json::json!({+ "status": "green",+ "indices": {"count": 5, "docs": {"count": 42}},+ "nodes": {"count": {"total": 1}}+ });+ let before = stats.clone();+ prune_at("/_cluster/stats", &mut stats, &p, &known);+ assert_eq!(stats, before, "_cluster/stats names no index anywhere");++ // Global `GET /_mapping` kept every field's `type`, which is what+ // Kibana reads to build an index pattern.+ let mut mapping = serde_json::json!({+ "logs-2026": {"mappings": {"properties": {+ "host": {"type": "keyword"},+ "count": {"type": "long"}+ }}},+ "type": {"mappings": {"properties": {}}}+ });+ prune_at("/_mapping", &mut mapping, &p, &known);+ assert_eq!(+ mapping["logs-2026"]["mappings"]["properties"]["host"]["type"],+ "keyword"+ );+ assert_eq!(+ mapping["logs-2026"]["mappings"]["properties"]["count"]["type"],+ "long"+ );+ assert!(+ mapping.get("type").is_none(),+ "the index actually named `type` is still hidden"+ );++ // `GET /_cluster/state` prunes its two index maps and leaves the node+ // map alone, even when an index shares a node's name.+ let mut state = serde_json::json!({+ "nodes": {"nodes": {"name": "nodes"}},+ "metadata": {"templates": {}, "indices": {"logs-2026": {}, "count": {}}},+ "routing_table": {"indices": {"logs-2026": {}, "count": {}}}+ });+ prune_at("/_cluster/state", &mut state, &p, &known);+ assert!(state["nodes"].get("nodes").is_some(), "node map untouched");+ assert!(state["metadata"].get("templates").is_some());+ assert!(state["metadata"]["indices"].get("logs-2026").is_some());+ assert!(state["metadata"]["indices"].get("count").is_none());+ assert!(state["routing_table"]["indices"].get("count").is_none());+ }++ /// The same collision in the `_cat` text tables: matching *any* token+ /// meant one unreadable index called `open` or `green` emptied the table.+ #[test]+ fn cat_rows_are_matched_on_the_index_column_only() {+ let p = scoped(&["logs-2026"], &[Privilege::ReadIndex]);+ let known: HashSet<String> = ["logs-2026", "open", "green", ".xerj-memory-bob-edges"]+ .iter()+ .map(|s| s.to_string())+ .collect();++ let table = "green open logs-2026 uuid 1 0 9 0 1kb 1kb\n\+ green open .xerj-memory-bob-edges uuid 1 0 1 0 1kb 1kb\n\+ green open open uuid 1 0 3 0 1kb 1kb\n";+ let out = prune_rows("/_cat/indices", table, &p, &known);+ assert!(out.contains("logs-2026"), "own index survives: {out}");+ assert!(+ !out.contains(".xerj-memory-bob-edges"),+ "brain hidden: {out}"+ );+ assert_eq!(+ out.lines().count(),+ 1,+ "only the row whose INDEX column is hidden goes: {out}"+ );++ // `_cat/health` has no index column at all, so nothing may be dropped+ // from it however its values happen to read.+ let health = "1785000000 12:00:00 xerj green 1 1 4 4 0 0 0 0 - 100.0%\n";+ assert_eq!(prune_rows("/_cat/health", health, &p, &known), health);++ // `_cat/fielddata` carries the index in column 4.+ let fd = "n1 127.0.0.1 127.0.0.1 n1 logs-2026 0b\n\+ n1 127.0.0.1 127.0.0.1 n1 .xerj-memory-bob-edges 0b\n";+ let out = prune_rows("/_cat/fielddata", fd, &p, &known);+ assert!(out.contains("logs-2026"));+ assert!(!out.contains(".xerj-memory-bob-edges"));+ }++ /// A `_cat/templates` row is keyed by TEMPLATE name. It must not be+ /// matched against index names — a template and an index may share one.+ #[test]+ fn cat_templates_rows_are_never_index_matched() {+ let p = scoped(&["logs-2026"], &[Privilege::ReadIndex]);+ let known: HashSet<String> = ["count"].iter().map(|s| s.to_string()).collect();+ assert!(cat_index_columns(&segments("/_cat/templates")).is_none());+ assert!(prune_sites(&segments("/_cat/templates")).is_empty());+ let rows = "count logs-* 100 -\n";+ assert_eq!(prune_rows("/_cat/templates", rows, &p, &known), rows);+ }++ /// FINDING C. Pinned, not fixed: `names: ["*"]` matches everything,+ /// including `.xerj-memory-*`. Only a superuser can mint a scoped key, so+ /// this is an operator's explicit "the whole node" rather than an+ /// escalation — but it must not change without someone meaning it.+ /// See `xerj_engine::rbac::Role::applies_to`.+ #[test]+ fn a_star_grant_reaches_the_reserved_namespace() {+ let star = scoped(&["*"], &[Privilege::ReadIndex, Privilege::WriteIndex]);+ assert!(star.allows_index(".xerj-memory-bob-edges", Privilege::ReadIndex));+ assert!(check(+ &star,+ Method::GET,+ "/.xerj-memory-bob-edges/_search",+ ""+ ));+ assert!(check(&star, Method::GET, "/_memory/bob", ""));+ // A prefix grant that stops short of the reserved namespace does not.+ let logs = scoped(&["logs-*"], &[Privilege::ReadIndex]);+ assert!(!logs.allows_index(".xerj-memory-bob-edges", Privilege::ReadIndex));+ assert!(!check(+ &logs,+ Method::GET,+ "/.xerj-memory-bob-edges/_search",+ ""+ ));+ // …and `*` is still not a licence to squat the namespace with a+ // template, which decides the mapping of brains created later.+ assert!(!check(+ &star,+ Method::PUT,+ "/_index_template/t",+ r#"{"index_patterns":[".xerj-memory-*"]}"#+ ));+ }++ /// FINDING A, config half. The ML config endpoints name an index the+ /// server will then go and read on a schedule; before this they were+ /// accepted with a 200 for a principal holding nothing on it.+ #[test]+ fn ml_config_authorizes_the_index_it_will_read() {+ let legacy = unscoped();+ let alice = scoped(&[".xerj-memory-alice-edges"], &[Privilege::ReadIndex]);++ for (path, body) in [+ (+ "/_ml/anomaly_detectors/leak2",+ r#"{"source_index":".xerj-memory-bob-edges","time_field":"created_at"}"#,+ ),+ (+ "/_ml/anomaly_detectors/leak2",+ r#"{"index":".xerj-memory-bob-edges"}"#,+ ),+ (+ "/_ml/anomaly_detectors/leak2",+ r#"{"source_index":"logs-2026","datafeed_config":{"indices":[".xerj-memory-bob-edges"]}}"#,+ ),+ (+ "/_ml/datafeeds/leak2-feed",+ r#"{"job_id":"leak2","indices":[".xerj-memory-bob-edges"]}"#,+ ),+ (+ "/_ml/datafeeds/leak2-feed",+ r#"{"job_id":"leak2","indexes":".xerj-memory-bob-edges"}"#,+ ),+ ] {+ assert!(+ !check(&legacy, Method::PUT, path, body),+ "unscoped key configured ML over a brain: PUT {path} {body}"+ );+ assert!(+ !check(&alice, Method::PUT, path, body),+ "alice configured ML over bob's brain: PUT {path} {body}"+ );+ }++ // Its own brain is fine, and so is an ordinary index for a key that+ // holds the general surface.+ assert!(check(+ &alice,+ Method::PUT,+ "/_ml/anomaly_detectors/mine",+ r#"{"source_index":".xerj-memory-alice-edges","time_field":"t"}"#+ ));+ assert!(check(+ &legacy,+ Method::PUT,+ "/_ml/anomaly_detectors/ok",+ r#"{"source_index":"logs-2026","time_field":"t"}"#+ ));+ // A datafeed that names no index inherits the (already authorized)+ // detector's source, so a scoped key is not refused for it.+ assert!(check(+ &alice,+ Method::PUT,+ "/_ml/datafeeds/mine-feed",+ r#"{"job_id":"mine"}"#+ ));+ // The sub-verbs carry no index of their own, so they are decided by+ // the pre-existing cluster rule and this change does not move them: a+ // key that holds the general surface may start a datafeed, a scoped+ // one may not (a cluster-level mutation naming nothing). Pinned so the+ // MlConfig arm cannot be read as having widened either one.+ assert_eq!(+ body_shape(&Method::POST, &segments("/_ml/datafeeds/f/_start")),+ BodyShape::None+ );+ assert!(check(+ &legacy,+ Method::POST,+ "/_ml/datafeeds/f/_start",+ "{}"+ ));+ assert!(!check(+ &alice,+ Method::POST,+ "/_ml/datafeeds/f/_start",+ "{}"+ ));+ }++ /// A template's patterns decide the mapping of indices that do not exist+ /// yet, so a pattern reaching the reserved namespace is a write to a brain+ /// by another route.+ #[test]+ fn index_template_patterns_cannot_reach_the_namespace() {+ let legacy = unscoped();+ assert!(check(+ &legacy,+ Method::PUT,+ "/_index_template/logs",+ r#"{"index_patterns":["logs-*"]}"#+ ));+ for patterns in [+ r#"{"index_patterns":[".xerj-memory-*"]}"#,+ r#"{"index_patterns":["*"]}"#,+ r#"{"index_patterns":[".xerj-memory-bob-edges"]}"#,+ r#"{"index_patterns":"logs-*,.xerj-*"}"#,+ ] {+ assert!(+ !check(&legacy, Method::PUT, "/_index_template/grab", patterns),+ "{patterns} must be refused"+ );+ assert!(+ !check(&legacy, Method::PUT, "/_template/grab", patterns),+ "legacy template {patterns} must be refused"+ );+ }+ // The superuser still registers whatever it likes.+ assert!(check(+ &Principal::Superuser,+ Method::PUT,+ "/_index_template/grab",+ r#"{"index_patterns":["*"]}"#+ ));+ }++ /// The engine-side backstop is derived from the same principal, so a name+ /// the middleware never parsed is still refused where it is resolved.+ #[test]+ fn visibility_matches_the_principal() {+ let alice = scoped(&[".xerj-memory-alice-edges"], &[Privilege::ReadIndex]);+ let v = visibility_for(&alice);+ assert!(v.visible(".xerj-memory-alice-edges"));+ assert!(!v.visible(".xerj-memory-bob-edges"));+ assert!(!v.visible("logs-2026"));++ let legacy = visibility_for(&unscoped());+ assert!(legacy.visible("logs-2026"));+ assert!(!legacy.visible(".xerj-memory-bob-edges"));++ let root = visibility_for(&Principal::Superuser);+ assert!(root.visible(".xerj-memory-bob-edges"));++ let denied = visibility_for(&Principal::Denied);+ assert!(!denied.visible("logs-2026"));+ }+}diff --git a/engine/crates/xerj-api/src/es_compat.rs b/engine/crates/xerj-api/src/es_compat.rsindex 3908b413a..ac4df7b55 100644--- a/engine/crates/xerj-api/src/es_compat.rs+++ b/engine/crates/xerj-api/src/es_compat.rs@@ -8567,6 +8567,14 @@ async fn search_impl(// (for example when the client disconnects) aborts the child task;// awaiting its JoinHandle also converts executor panics into HTTP 500// instead of unwinding through axum.+ //+ // No `index_guard` rule is carried across this spawn, and does not need+ // to be: the task closes over `idx`, an `Index` handle the guarded+ // request already resolved and was authorized for, and an `Index` holds+ // no `Engine`, so nothing inside can name a second index. The two ways+ // a search body *does* name another index — a `terms` lookup and a+ // `lookup` runtime field — are both resolved earlier in this function,+ // on the request's own task (`resolve_terms_lookups`), under the guard.#[cfg(test)]let inject_task_panic = *idx_name == "test-panic-search-task";#[cfg(test)]@@ -16403,16 +16411,76 @@ pub struct AliasActionsBody {pub actions: Vec<AliasAction>,}+/// Is this alias-action `index` field a pattern rather than one concrete name?+fn alias_target_is_pattern(spec: &str) -> bool {+ spec.split(',')+ .map(str::trim)+ .any(|p| p == "_all" || p.contains('*'))+}+pub async fn post_aliases(State(state): State<AppState>,Json(body): Json<AliasActionsBody>,) -> impl IntoResponse {+ // An `add` whose `index` is a wildcard has to be expanded before the alias+ // can point anywhere. Adding the pattern TEXT as the alias target — which+ // is what the raw `add_alias` below used to do for every action — invents+ // an alias resolving to an index name that cannot exist, and answers+ // `acknowledged: true` for a write that did not happen. It read as a+ // successful alias creation to a caller that had just been told, by the+ // very same response, that it had reached the reserved namespace:+ // `POST /_aliases {"actions":[{"add":{"index":".xerj-memory-*", …}}]}`+ // returned 200 and left a dangling alias behind.+ //+ // ES resolves the wildcard and 404s `index_not_found_exception` when it+ // matches nothing; so do we. A concrete name is left exactly as it was —+ // ES lets an alias be attached to a name before the index exists, and+ // clients rely on that ordering.+ // Resolve every action first and refuse the whole request if any `add`+ // resolved to nothing, so a rejected batch leaves no half-applied aliases+ // behind.+ enum Resolved<'a> {+ Add(&'a AliasActionParams, Vec<String>),+ Remove(&'a AliasActionParams, Vec<String>),+ }+ let mut plan: Vec<Resolved> = Vec::with_capacity(body.actions.len());for action in &body.actions {if let Some(add) = &action.add {- state.engine.add_alias(&add.alias, &add.index);+ let targets = if alias_target_is_pattern(&add.index) {+ let expanded = resolve_index_selector(&state, &add.index).await;+ if expanded.is_empty() {+ let e = xerj_common::XerjError::index_not_found(&add.index);+ return ApiError::new(e).into_response();+ }+ expanded+ } else {+ vec![add.index.clone()]+ };+ plan.push(Resolved::Add(add, targets));}if let Some(remove) = &action.remove {- state.engine.remove_alias(&remove.alias, &remove.index);+ // A `remove` that matches nothing removes nothing — idempotent,+ // and it leaves no junk behind, so it is not refused.+ let targets = if alias_target_is_pattern(&remove.index) {+ resolve_index_selector(&state, &remove.index).await+ } else {+ vec![remove.index.clone()]+ };+ plan.push(Resolved::Remove(remove, targets));+ }+ }+ for step in &plan {+ match step {+ Resolved::Add(add, targets) => {+ for target in targets {+ state.engine.add_alias(&add.alias, target);+ }+ }+ Resolved::Remove(remove, targets) => {+ for target in targets {+ state.engine.remove_alias(&remove.alias, target);+ }+ }}}Json(json!({ "acknowledged": true })).into_response()@@ -16600,11 +16668,19 @@ pub async fn put_alias(};if targets.is_empty() {- attach(&index);- } else {- for idx in &targets {- attach(idx);- }+ // `resolve_index_selector` only ever comes back empty for a wildcard+ // that matched nothing — a concrete name always resolves to itself.+ // Attaching the alias to the pattern text in that case (what this used+ // to do) fabricates an alias pointing at an index name that cannot+ // exist and reports `acknowledged: true` for a write that did not+ // happen; `PUT /.xerj-memory-*/_alias/X` was the case that made it+ // look like the reserved namespace had just been aliased. Refuse, the+ // way ES does.+ let e = xerj_common::XerjError::index_not_found(&index);+ return ApiError::new(e).into_response();+ }+ for idx in &targets {+ attach(idx);}Json(json!({ "acknowledged": true })).into_response()}@@ -20111,6 +20187,11 @@ pub async fn delete_by_query(let task_key = handle.key().to_string();let spawned_key = task_key.clone();let tasks = state.tasks.clone();+ // Detached, so no `index_guard` rule — and none is needed: the task+ // owns `idx`, the single already-authorized `Index` handle this request+ // resolved, and an `Index` cannot name another index (it holds no+ // `Engine`). See `xerj_engine::index_guard` for the contract a spawn+ // that *can* reach `Engine::get_index` has to follow instead.tokio::spawn(async move {let response = run_delete_by_query(&idx, &search_req).await;tasks.complete(&spawned_key, response);@@ -20246,6 +20327,8 @@ pub async fn update_by_query(let task_key = handle.key().to_string();let spawned_key = task_key.clone();let tasks = state.tasks.clone();+ // Same reasoning as the `_delete_by_query` spawn above: the task owns+ // one already-authorized `Index` handle and can reach no other index.tokio::spawn(async move {let response = run_update_by_query(&idx, &search_req, script).await;tasks.complete(&spawned_key, response);@@ -25328,8 +25411,22 @@ pub async fn security_get_user_profile(pub async fn security_create_api_key(State(state): State<AppState>,+ principal: crate::auth::Principal,body: Option<Json<Value>>,) -> impl IntoResponse {+ // Minting is how privilege is handed out, so it is also the obvious way to+ // escalate: without this gate a caller with no brain access could simply+ // mint itself a key that grants one (issue #79). Only the superuser may+ // create a *scoped* key; a scoped caller may not mint at all (an unscoped+ // key it minted would out-reach it on the general surface); an unscoped+ // caller may mint, but only more unscoped keys.+ if matches!(principal, crate::auth::Principal::Scoped { .. }) {+ return crate::authz::forbidden(+ &principal,+ "<api keys>",+ xerj_engine::rbac::Privilege::SecurityAdmin,+ );+ }let payload = body.map(|Json(v)| v);let name = payload.as_ref()@@ -25349,21 +25446,43 @@ pub async fn security_create_api_key(// ES returns base64("id:api_key") as the `encoded` credential.let encoded = base64_encode(&format!("{key_id}:{api_key}"));- // Honest surface (item 6): `role_descriptors` are accepted for ES/Kibana- // wire-compatibility but NOT enforced — every key authenticates as the- // single superuser. Warn loudly so an operator relying on scoped keys is- // not silently over-privileged. (Full per-key RBAC is deferred.)- if payload+ // `role_descriptors` are now parsed into the key's index grants and ARE+ // enforced for the reserved `.xerj-memory-*` namespace (issue #79) — see+ // `authz`. Two honest caveats remain, both warned about below:+ // * cluster privileges and FLS/DLS inside a descriptor are still+ // ignored, and+ // * a key minted with NO usable descriptor keeps its historical reach+ // over ordinary indices (it just holds nothing in the reserved+ // namespace), because broad RBAC over the general ES surface is still+ // deferred.+ let requested_descriptors = payload.as_ref().and_then(|b| b.get("role_descriptors"))- .is_some_and(|v| !v.is_null() && v != &json!({}))- {- tracing::warn!(- key = %name,- "POST /_security/api_key: role_descriptors are NOT enforced; the \- minted key has full superuser access. See rbac docs."- );- }+ .filter(|v| !v.is_null() && *v != &json!({}));+ let roles = match (&principal, requested_descriptors) {+ (crate::auth::Principal::Superuser, Some(descriptors)) => {+ let parsed = xerj_engine::rbac::roles_from_role_descriptors(descriptors);+ if parsed.is_empty() {+ tracing::warn!(+ key = %name,+ "POST /_security/api_key: role_descriptors granted nothing usable \+ (only index names + index privileges are enforced); the key is \+ unscoped and holds NO access to the reserved .xerj-memory-* namespace"+ );+ }+ parsed+ }+ (_, Some(_)) => {+ // A non-superuser cannot hand out grants it does not itself hold.+ tracing::warn!(+ key = %name,+ "POST /_security/api_key: role_descriptors ignored — only the admin key \+ may mint a scoped key; the minted key is unscoped"+ );+ Vec::new()+ }+ (_, None) => Vec::new(),+ };// Persist the key so the auth middleware can re-authenticate an inbound// `Authorization: ApiKey <encoded>` header. `encoded` decodes to@@ -25379,6 +25498,7 @@ pub async fn security_create_api_key(creation_ms: now_ms,expiration_ms,invalidated: false,+ roles,},);@@ -29731,43 +29851,70 @@ async fn datafeed_score_pass(/// new anomaly records. Breaks cleanly when the datafeed is stopped/deleted or/// its job disappears. The first `interval` tick fires immediately and is/// consumed here (the synchronous start pass already covered "now").+///+/// ## Why this carries a visibility rule+///+/// The per-index boundary (issue #79) is enforced in part by a `tokio`+/// task-local ([`xerj_engine::index_guard`]), and a task-local does not+/// survive `tokio::spawn`. This was the one request path in the tree that+/// detached, and it was a live cross-brain read: `_start` under a principal+/// with no access to `.xerj-memory-bob-edges` returned an empty result set+/// from the synchronous pass — correctly denied, because that pass runs inside+/// the request's scope — and then this task ticked a couple of seconds later+/// with no guard at all and appended the brain's field values to the job's+/// results, where the same principal read them straight back out of+/// `GET /_ml/anomaly_detectors/{job}/results/records`.+///+/// So the rule the *starting* request is running under is captured on that+/// request's task and re-installed here. The scorer therefore sees exactly+/// what its starter could see: a superuser's datafeed keeps working unchanged+/// (a superuser has no guard installed, so `None` is carried and the task+/// stays unrestricted), and a scoped one silently scores nothing on an index+/// it was not granted, because `Engine::get_index` answers "not found" and+/// `datafeed_score_pass` treats that as "no new records".fn spawn_datafeed_task(state: AppState, feed_id: String) -> tokio::task::JoinHandle<()> {- tokio::spawn(async move {- let freq = state- .ml_datafeeds- .get(&feed_id)- .map(|f| f.frequency_secs)- .unwrap_or(60)- .max(1);- let mut ticker = tokio::time::interval(std::time::Duration::from_secs(freq));- ticker.tick().await; // consume the immediate first tick- loop {- ticker.tick().await;- // Snapshot the datafeed; stop when gone or no longer started.- let feed = match state.ml_datafeeds.get(&feed_id) {- Some(f) if f.state == "started" => f.value().clone(),- _ => break,- };- let detector = match state.ml_detectors.get(&feed.job_id) {- Some(d) => d.value().clone(),- None => break,- };- let (cursor, _appended) = datafeed_score_pass(- &state.engine,- &state.ml_results,- &detector,- &feed,- feed.last_scored_ms,- )- .await;- // Persist the cursor unless a concurrent _stop already flipped state.- if let Some(mut f) = state.ml_datafeeds.get_mut(&feed_id) {- if f.state == "started" {- f.last_scored_ms = cursor;+ // Captured HERE — on the request's own task, before the spawn. Reading it+ // inside the spawned future would always find `None`.+ let visibility = xerj_engine::index_guard::current();+ tokio::spawn(xerj_engine::index_guard::scoped_opt(+ visibility,+ async move {+ let freq = state+ .ml_datafeeds+ .get(&feed_id)+ .map(|f| f.frequency_secs)+ .unwrap_or(60)+ .max(1);+ let mut ticker = tokio::time::interval(std::time::Duration::from_secs(freq));+ ticker.tick().await; // consume the immediate first tick+ loop {+ ticker.tick().await;+ // Snapshot the datafeed; stop when gone or no longer started.+ let feed = match state.ml_datafeeds.get(&feed_id) {+ Some(f) if f.state == "started" => f.value().clone(),+ _ => break,+ };+ let detector = match state.ml_detectors.get(&feed.job_id) {+ Some(d) => d.value().clone(),+ None => break,+ };+ let (cursor, _appended) = datafeed_score_pass(+ &state.engine,+ &state.ml_results,+ &detector,+ &feed,+ feed.last_scored_ms,+ )+ .await;+ // Persist the cursor unless a concurrent _stop already flipped state.+ if let Some(mut f) = state.ml_datafeeds.get_mut(&feed_id) {+ if f.state == "started" {+ f.last_scored_ms = cursor;+ }}}- }- })+ },+ ))}/// PUT /_ml/datafeeds/{feed_id} — create/replace a datafeed for a job.diff --git a/engine/crates/xerj-api/src/graph_api.rs b/engine/crates/xerj-api/src/graph_api.rsindex 19b53f8bd..6fb8a1342 100644--- a/engine/crates/xerj-api/src/graph_api.rs+++ b/engine/crates/xerj-api/src/graph_api.rs@@ -32,16 +32,52 @@//! what was withheld (clipped edges/frontiers, bi-temporally excluded edges,//! segments skipped, dangling node refs, agg tails).//!-//! ## Authorization model+//! ## Authorization model — a brain IS a boundary (issue #79)//!-//! ⚠️ The `/_graph/*` endpoints are guarded ONLY by the process-wide API-key-//! auth middleware (the admin key or any key minted via-//! `POST /_security/api_key`). There is **no per-brain authorization**: any-//! credential that can reach the node can read, write, and invalidate edges-//! in **every** brain. Brains isolate data *by name*, NOT by credential — do-//! not treat a brain as a security boundary between mutually-distrusting-//! tenants. Per-brain access control depends on the deferred RBAC enforcement-//! (see `xerj_engine::rbac`).+//! Every endpoint below authorizes the caller for **this brain** before it+//! does anything else, including before it checks whether the brain exists —+//! so a 403 is returned identically for a brain that is not yours and a brain+//! that does not exist, and the response code cannot be used to enumerate.+//!+//! The resource a brain authorizes against is its edges index,+//! `.xerj-memory-{brain}-edges`. Who holds what:+//!+//! | Principal | Reach |+//! |---|---|+//! | open mode (`--insecure`, point-at-a-folder) | every brain — one user, no config, unchanged |+//! | the configured admin key | every brain |+//! | a key minted **with** `role_descriptors` naming the edges index | that brain, at the granted privilege |+//! | a key minted **without** `role_descriptors` | **no** brain |+//! | no/invalid credential | nothing |+//!+//! To grant brain `alice` to a key, name its indices at mint time:+//!+//! ```json+//! POST /_security/api_key+//! { "name": "alice-agent", "role_descriptors": { "alice": { "indices": [+//! { "names": [".xerj-memory-alice-edges", ".xerj-memory-alice"],+//! "privileges": ["read", "write"] } ] } } }+//! ```+//!+//! `ego`/`overview` need `read`; `link`/`unlink` need `write` (creating the+//! edges index lazily is part of writing a brain, not a separate `manage`+//! step). The second name is the brain's *nodes* index — needed for `ego`'s+//! node hydration and `overview`'s note count, and it is also the+//! agent-memory namespace of the same name.+//!+//! ⚠️ An access check *here alone* would not have created a boundary. A+//! brain's edges live in an ordinary index and `IndexName::validate` admits a+//! leading `.`, so `.xerj-memory-{brain}-edges` is reachable through the+//! generic ES-compat and native index routes: `_search` reads it around this+//! module's `as_of`/`not_shown` semantics, `_doc/{id}` **forges** edges around+//! the derived-`edge_id` invariant (§2.3) that only [`link`] enforces, `DELETE`+//! destroys the brain, and `GET /_mapping` lists every brain name. All of+//! those doors are closed in [`crate::authz`], which owns the rule and+//! enumerates the enforcement points; the checks in this module are the+//! in-handler half of the same decision, so these handlers stay safe even if+//! mounted without that middleware.+//!+//! `tests/brain_is_a_security_boundary.rs` executes every door.use axum::{extract::{Path, Query, State},@@ -53,6 +89,8 @@ use serde::Deserialize;use serde_json::{json, Map, Value};use std::collections::{HashMap, HashSet};+use crate::auth::Principal;+use crate::authz;use crate::es_compat::{self, EsSearchBody, EsSearchJson, EsSearchQueryParams, GetDocParams, IndexDocParams,};@@ -61,6 +99,7 @@ use crate::state::AppState;use xerj_engine::graph::{GraphDirection, GraphEdgeLite, GraphExpandRequest, GRAPH_HOPS_CAP_REASON,};+use xerj_engine::rbac::Privilege;/// Contract version string, returned by every read endpoint.pub const GRAPH_CONTRACT: &str = "xerj-second-brain/1";@@ -83,16 +122,24 @@ const EGO_SEEDS_CAP: usize = 64;/// Max dangling node ids listed verbatim in `not_shown.dangling_ids`.const MAX_DANGLING_LISTED: usize = 50;-/// Edges index name for a brain (SECOND_BRAIN_SPEC §1). The leading dot passes-/// `IndexName::validate` and is unreachable through user index APIs.+/// Edges index name for a brain (SECOND_BRAIN_SPEC §1).+///+/// The leading dot passes `IndexName::validate` and keeps the index out of the+/// *conventional* listing surfaces, the same way `.kibana` is conventionally+/// hidden. It does **not** by itself make the index unreachable — an earlier+/// version of this comment claimed that, and it was wrong; `.`-prefixed names+/// are accepted by the generic index routes. What makes it unreachable is that+/// it sits in the reserved [`authz::RESERVED_INDEX_PREFIX`] namespace, which+/// [`authz`] gates on every surface. This name is also the resource a+/// `role_descriptors` grant must name to unlock the brain.fn edges_index(brain: &str) -> String {- format!(".xerj-memory-{brain}-edges")+ authz::brain_edges_index(brain)}/// Default nodes index for a brain: the agent-memory namespace of the same/// name. Autoindex brains override this via the §2.5 meta doc.fn default_nodes_index(brain: &str) -> String {- format!(".xerj-memory-{brain}")+ authz::memory_namespace_index(brain)}/// Brain-name validation (SECOND_BRAIN_SPEC §1): identical rules to@@ -359,11 +406,16 @@ pub struct LinkBody {pub async fn link(State(state): State<AppState>,Path(brain): Path<String>,+ principal: Principal,body: OptionalJson<LinkBody>,) -> Response {if let Err(reason) = validate_brain(&brain) {return error_response(StatusCode::BAD_REQUEST, reason);}+ // Before anything that could touch or create the brain (issue #79).+ if let Err(denied) = authz::authorize_brain(&principal, &brain, Privilege::WriteIndex) {+ return denied;+ }let Some(body) = body.0 else {return error_response(StatusCode::BAD_REQUEST,@@ -515,11 +567,17 @@ pub struct UnlinkParams {pub async fn unlink(State(state): State<AppState>,Path((brain, edge_id)): Path<(String, String)>,+ principal: Principal,Query(params): Query<UnlinkParams>,) -> Response {if let Err(reason) = validate_brain(&brain) {return error_response(StatusCode::BAD_REQUEST, reason);}+ // Ordered before the existence probe below so an unauthorized caller+ // cannot tell "not yours" from "not there" (issue #79).+ if let Err(denied) = authz::authorize_brain(&principal, &brain, Privilege::WriteIndex) {+ return denied;+ }let not_found = || {error_response(StatusCode::NOT_FOUND,@@ -655,11 +713,23 @@ pub struct EgoParams {pub async fn ego(State(state): State<AppState>,Path(brain): Path<String>,+ principal: Principal,Query(params): Query<EgoParams>,) -> Response {if let Err(reason) = validate_brain(&brain) {return error_response(StatusCode::BAD_REQUEST, reason);}+ if let Err(denied) = authz::authorize_brain(&principal, &brain, Privilege::ReadIndex) {+ return denied;+ }+ // `nodes_index` lets the caller redirect hydration at an arbitrary index,+ // so it is authorized separately — otherwise a grant on one brain would be+ // a read primitive for any index on the node.+ if let Some(nodes_index) = params.nodes_index.as_deref() {+ if let Err(denied) = authz::authorize_index(&principal, nodes_index, Privilege::ReadIndex) {+ return denied;+ }+ }if params.node.is_some() && params.nodes.is_some() {return error_response(StatusCode::BAD_REQUEST,@@ -828,6 +898,15 @@ pub async fn ego(Some(ni) if !ni.is_empty() => ni.to_string(),_ => resolve_nodes_index(&state, &brain, &index).await,};+ // The meta doc's `nodes_index` is caller-controlled data (anyone with+ // write on the brain can point it anywhere), so hydration is+ // authorized against the RESOLVED index, not just the brain. Without+ // this, write on one brain would be a read primitive for every index+ // on the node.+ if let Err(denied) = authz::authorize_index(&principal, &nodes_index, Privilege::ReadIndex)+ {+ return denied;+ }let mut found: HashSet<String> = HashSet::new();let search_body = EsSearchBody {query: Some(json!({ "ids": { "values": result.reachable } })),@@ -1088,11 +1167,17 @@ fn embedder_id(state: &AppState) -> String {pub async fn overview(State(state): State<AppState>,Path(brain): Path<String>,+ principal: Principal,Query(params): Query<OverviewParams>,) -> Response {if let Err(reason) = validate_brain(&brain) {return error_response(StatusCode::BAD_REQUEST, reason);}+ // Before the `exists: false` probe below, so a caller cannot map the+ // node's brains by watching 404 vs 403 (issue #79).+ if let Err(denied) = authz::authorize_brain(&principal, &brain, Privilege::ReadIndex) {+ return denied;+ }let as_of = match params.as_of.as_deref() {None => now_ms(),Some(s) => match parse_ms_str(s) {@@ -1216,6 +1301,13 @@ pub async fn overview(// nodes index was never created (edges asserted through the API alone)// truthfully has 0 stored notes — reported as such, not a 404 and never// fabricated from edge endpoints.+ //+ // The nodes index comes from the brain's meta doc, which is writable by+ // anyone with write on the brain — so it is authorized in its own right+ // (see the matching check in `ego`).+ if let Err(denied) = authz::authorize_index(&principal, &nodes_index, Privilege::ReadIndex) {+ return denied;+ }let nodes_total = if index_exists(&state, &nodes_index) {let count_body = EsSearchBody {query: Some(json!({ "match_all": {} })),@@ -1293,6 +1385,7 @@ mod tests {let resp = link(State(state.clone()),Path(brain.to_string()),+ Principal::Superuser,OptionalJson(Some(b)),).await;@@ -1308,6 +1401,7 @@ mod tests {let resp = unlink(State(state.clone()),Path((brain.to_string(), edge_id.to_string())),+ Principal::Superuser,Query(UnlinkParams {invalid_at: invalid_at.map(String::from),}),@@ -1317,7 +1411,13 @@ mod tests {}async fn do_ego(state: &AppState, brain: &str, params: EgoParams) -> (StatusCode, Value) {- let resp = ego(State(state.clone()), Path(brain.to_string()), Query(params)).await;+ let resp = ego(+ State(state.clone()),+ Path(brain.to_string()),+ Principal::Superuser,+ Query(params),+ )+ .await;drain_json(resp).await}@@ -1690,6 +1790,7 @@ mod tests {let resp = overview(State(state.clone()),Path("notes".to_string()),+ Principal::Superuser,Query(OverviewParams {as_of: Some(as_of.to_string()),..Default::default()@@ -1778,6 +1879,7 @@ mod tests {let resp = overview(State(state.clone()),Path("nope".to_string()),+ Principal::Superuser,Query(OverviewParams::default()),).await;@@ -1954,6 +2056,7 @@ mod tests {let resp = overview(State(state.clone()),Path("notes".to_string()),+ Principal::Superuser,Query(OverviewParams {as_of: Some(AS_OF.to_string()),..Default::default()diff --git a/engine/crates/xerj-api/src/lib.rs b/engine/crates/xerj-api/src/lib.rsindex fd50a0830..876ba358e 100644--- a/engine/crates/xerj-api/src/lib.rs+++ b/engine/crates/xerj-api/src/lib.rs@@ -57,6 +57,7 @@//! xerj-api itself does not depend on xerj-console-api.pub mod auth;+pub mod authz;pub mod binary_protocol;pub mod error;pub mod es_compat;diff --git a/engine/crates/xerj-api/src/memory_api.rs b/engine/crates/xerj-api/src/memory_api.rsindex 828519271..2c757b14c 100644--- a/engine/crates/xerj-api/src/memory_api.rs+++ b/engine/crates/xerj-api/src/memory_api.rs@@ -58,18 +58,21 @@ use serde::Deserialize;use serde_json::{json, Value};use uuid::Uuid;+use crate::auth::Principal;+use crate::authz;use crate::es_compat::{self, DeleteDocParams, DeleteIndexParams, EsSearchBody, EsSearchJson, EsSearchQueryParams,IndexDocParams,};use crate::extract::OptionalJson;use crate::state::AppState;+use xerj_engine::rbac::Privilege;/// Reserved backing-index prefix. Real (user) indices can never collide: the/// ES-compat `create_index` handler rejects any index name that starts with/// `_`, `-`, or `+`, and a caller cannot address a `.`-leading system index/// through `/_memory/*`.-const MEMORY_PREFIX: &str = ".xerj-memory-";+const MEMORY_PREFIX: &str = crate::authz::RESERVED_INDEX_PREFIX;/// Default page size for the introspection (`GET`) endpoint when `size` is/// omitted. Callers page further with `from`/`size` (see [`ListParams`]).@@ -306,11 +309,17 @@ async fn dedup_probe(pub async fn store(State(state): State<AppState>,Path(namespace): Path<String>,+ principal: Principal,body: OptionalJson<StoreBody>,) -> Response {if let Err(reason) = validate_namespace(&namespace) {return error_response(StatusCode::BAD_REQUEST, reason);}+ if let Err(denied) =+ authz::authorize_memory_namespace(&principal, &namespace, Privilege::WriteIndex)+ {+ return denied;+ }let body = match body.0 {Some(b) => b,None => return error_response(StatusCode::BAD_REQUEST, "missing request body"),@@ -472,11 +481,17 @@ const MAX_RESTRICT_IDS: usize = 10_000;pub async fn recall(State(state): State<AppState>,Path(namespace): Path<String>,+ principal: Principal,body: OptionalJson<RecallBody>,) -> Response {if let Err(reason) = validate_namespace(&namespace) {return error_response(StatusCode::BAD_REQUEST, reason);}+ if let Err(denied) =+ authz::authorize_memory_namespace(&principal, &namespace, Privilege::ReadIndex)+ {+ return denied;+ }// Recall must say what to recall BY: exactly one of `vector` (a query// embedding) or `query` (text). A missing body, both-at-once, or an// empty `query` are hard 400s — the old lenient fallback degraded every@@ -544,6 +559,17 @@ pub async fn recall(};let index = backing_index(&namespace);let edges_index_name = format!("{MEMORY_PREFIX}{namespace}-edges");+ // Graph coupling reads the brain's EDGES, a different resource from the+ // namespace itself — a credential granted only the notes must not read the+ // link graph through `graph:` (issue #79). Authorized only when coupling is+ // actually requested, so ungrouped recall is unaffected.+ if graph_ctx.is_some() {+ if let Err(denied) =+ authz::authorize_index(&principal, &edges_index_name, Privilege::ReadIndex)+ {+ return denied;+ }+ }let caller_filter = body.filter.take();// Graph restrict: expand the seeds FIRST and fold the reachable set into@@ -1112,11 +1138,17 @@ pub struct ListParams {pub async fn list(State(state): State<AppState>,Path(namespace): Path<String>,+ principal: Principal,axum::extract::Query(params): axum::extract::Query<ListParams>,) -> Response {if let Err(reason) = validate_namespace(&namespace) {return error_response(StatusCode::BAD_REQUEST, reason);}+ if let Err(denied) =+ authz::authorize_memory_namespace(&principal, &namespace, Privilege::ReadIndex)+ {+ return denied;+ }// `from`/`after` are aliases (offset cursor); `size` is clamped to// [1, MAX_LIST_SIZE] so a caller can neither page from a negative offset// nor pull an unbounded set in one call.@@ -1193,10 +1225,16 @@ pub async fn list(pub async fn forget_one(State(state): State<AppState>,Path((namespace, id)): Path<(String, String)>,+ principal: Principal,) -> Response {if let Err(reason) = validate_namespace(&namespace) {return error_response(StatusCode::BAD_REQUEST, reason);}+ if let Err(denied) =+ authz::authorize_memory_namespace(&principal, &namespace, Privilege::WriteIndex)+ {+ return denied;+ }let index = backing_index(&namespace);if !index_exists(&state, &index) {return error_response(@@ -1231,10 +1269,16 @@ pub async fn forget_one(pub async fn forget_namespace(State(state): State<AppState>,Path(namespace): Path<String>,+ principal: Principal,) -> Response {if let Err(reason) = validate_namespace(&namespace) {return error_response(StatusCode::BAD_REQUEST, reason);}+ if let Err(denied) =+ authz::authorize_memory_namespace(&principal, &namespace, Privilege::AdminIndex)+ {+ return denied;+ }let index = backing_index(&namespace);if !index_exists(&state, &index) {return (@@ -1286,6 +1330,7 @@ mod tests {let resp = store(State(state.clone()),Path(ns.to_string()),+ Principal::Superuser,OptionalJson(Some(b)),).await;@@ -1297,6 +1342,7 @@ mod tests {let resp = recall(State(state.clone()),Path(ns.to_string()),+ Principal::Superuser,OptionalJson(Some(b)),).await;@@ -1380,6 +1426,7 @@ mod tests {let resp = forget_one(State(state.clone()),Path(("agent1".to_string(), "m1".to_string())),+ Principal::Superuser,).await;let (s, _) = drain_json(resp).await;@@ -1473,6 +1520,7 @@ mod tests {let resp = list(State(state.clone()),Path(ns.to_string()),+ Principal::Superuser,axum::extract::Query(ListParams::default()),).await;@@ -1484,6 +1532,7 @@ mod tests {let resp = list(State(state.clone()),Path(ns.to_string()),+ Principal::Superuser,axum::extract::Query(params),).await;@@ -1657,6 +1706,7 @@ mod tests {let resp = crate::graph_api::link(State(state.clone()),Path("notes".to_string()),+ Principal::Superuser,OptionalJson(Some(b)),).await;@@ -1870,6 +1920,7 @@ mod tests {let resp = crate::graph_api::link(State(state.clone()),Path("ghost".to_string()),+ Principal::Superuser,OptionalJson(Some(b)),).await;diff --git a/engine/crates/xerj-api/src/router.rs b/engine/crates/xerj-api/src/router.rsindex d33cb3cd3..68f523f81 100644--- a/engine/crates/xerj-api/src/router.rs+++ b/engine/crates/xerj-api/src/router.rs@@ -27,7 +27,9 @@ use tracing::Level;use uuid::Uuid;use xerj_common::config::CorsConfig;-use crate::{auth::auth_middleware, es_compat, graph_api, memory_api, native, state::AppState};+use crate::{+ auth::auth_middleware, authz, es_compat, graph_api, memory_api, native, state::AppState,+};// ─────────────────────────────────────────────────────────────────────────────// Native xerj router (:8080)@@ -150,6 +152,15 @@ pub fn build_native_router(state: AppState) -> Router {// Shared state.with_state(state.clone())// Middleware stack (applied outermost-last)+ //+ // The native router reaches the same indices under a different+ // spelling (`/v1/indices/{name}/…`), so it carries the same+ // authorization layer — otherwise the reserved `.xerj-memory-*`+ // namespace would be closed on :9200 and open on :8080.+ .layer(middleware::from_fn_with_state(+ state.clone(),+ authz::authz_middleware,+ )).layer(middleware::from_fn_with_state(state, auth_middleware)).layer(middleware::from_fn(request_id_middleware)).layer(trace_layer(access_log))@@ -773,6 +784,18 @@ pub fn build_es_compat_router(state: AppState) -> Router {// Shared state.with_state(state.clone())// Middleware stack (applied outermost-last)+ //+ // Authorization sits INSIDE authentication — added first here, so it+ // runs second: `auth_middleware` decides whether the caller is anyone+ // at all, then `authz_middleware` decides what that caller may reach.+ // It is the enforcement point that makes a brain a real boundary+ // rather than a naming convention (issue #79); see `authz.rs` for the+ // full table of doors it closes. A superuser — `--insecure`, or the+ // configured admin key — short-circuits it on the first line.+ .layer(middleware::from_fn_with_state(+ state.clone(),+ authz::authz_middleware,+ )).layer(middleware::from_fn_with_state(state.clone(),auth_middleware,diff --git a/engine/crates/xerj-api/tests/body_named_index_bypasses.rs b/engine/crates/xerj-api/tests/body_named_index_bypasses.rsnew file mode 100644index 000000000..c4107b826--- /dev/null+++ b/engine/crates/xerj-api/tests/body_named_index_bypasses.rs@@ -0,0 +1,784 @@+//! The four proven bypasses of per-brain authorization, fired at a **real+//! running node** over a real socket — the way they were found.+//!+//! Every case below was measured against a live server on the first cut of+//! issue #79's fix and came back `200 OK` with another tenant's data. They are+//! not hypotheses, so they are pinned here as regressions rather than argued+//! about. The common cause was one sentence long: authorization was decided+//! against the index named in the URL **path**, while these handlers take the+//! index they actually operate on from the request **body**, where the path+//! index is only a default the body overrides.+//!+//! | # | Verb | How the body overrode the path |+//! |---|---|---|+//! | 1 | `_msearch` | an NDJSON header line's `index` |+//! | 2 | `_bulk` | an action line's `_index` (forge **and** delete) |+//! | 3 | `_mget` | `docs[]._index` |+//! | 4 | `_aliases` | an alias pointed at the victim, then read through |+//!+//! This runs over a TCP listener rather than `tower::ServiceExt::oneshot`+//! deliberately. `oneshot` calls the router directly; the bypasses live in the+//! interaction between middleware, body buffering and handlers, and a body+//! that has to survive being read once for authorization and again by the+//! handler only behaves the same over a real connection. The client below is+//! deliberately dumb (write bytes, read bytes) so nothing in it can smooth over+//! a difference.++use std::net::SocketAddr;++use tokio::io::{AsyncReadExt, AsyncWriteExt};+use tokio::net::TcpStream;+use xerj_api::{+ router::{build_es_compat_router, build_native_router},+ state::AppState,+};+use xerj_common::{config::Config, metrics::Metrics};+use xerj_engine::Engine;++const ADMIN_KEY: &str = "admin-secret-key-for-bypass-test";+const T0: i64 = 1_753_600_000_000;++/// A live node — both listeners, one engine — plus the data dir it must+/// outlive. The native router is served too because it reaches the same+/// indices under a different spelling, so a boundary that holds on `:9200` and+/// falls open on `:8080` is not a boundary.+struct Node {+ addr: SocketAddr,+ native_addr: SocketAddr,+ _data_dir: tempfile::TempDir,+}++/// Bind an ephemeral port and serve `app` on it.+async fn serve(app: axum::Router) -> SocketAddr {+ let listener = tokio::net::TcpListener::bind("127.0.0.1:0")+ .await+ .expect("bind");+ let addr = listener.local_addr().expect("local addr");+ tokio::spawn(async move {+ let _ = axum::serve(listener, app).await;+ });+ addr+}++async fn start_node() -> Node {+ let dir = tempfile::tempdir().expect("tempdir");+ let mut config = Config::default();+ config.server.data_dir = dir.path().to_str().unwrap().to_string();+ config.auth.enabled = true;+ config.auth.admin_api_key = ADMIN_KEY.to_string();+ let metrics = Metrics::new().expect("metrics");+ let engine = Engine::new(config.clone()).expect("engine");+ let state = AppState::new(config, engine, metrics);+ Node {+ addr: serve(build_es_compat_router(state.clone())).await,+ native_addr: serve(build_native_router(state)).await,+ _data_dir: dir,+ }+}++/// One HTTP/1.1 request over a fresh connection. Returns (status, body).+async fn http(node: &Node, method: &str, path: &str, auth: &str, body: &str) -> (u16, String) {+ request(node.addr, method, path, auth, body).await+}++/// The same, against the native `:8080`-shaped listener.+async fn http_native(+ node: &Node,+ method: &str,+ path: &str,+ auth: &str,+ body: &str,+) -> (u16, String) {+ request(node.native_addr, method, path, auth, body).await+}++async fn request(+ addr: SocketAddr,+ method: &str,+ path: &str,+ auth: &str,+ body: &str,+) -> (u16, String) {+ let mut stream = TcpStream::connect(addr).await.expect("connect");+ let mut request = format!(+ "{method} {path} HTTP/1.1\r\nHost: 127.0.0.1\r\nConnection: close\r\n\+ Content-Type: application/json\r\nContent-Length: {}\r\n",+ body.len()+ );+ if !auth.is_empty() {+ request.push_str(&format!("Authorization: {auth}\r\n"));+ }+ request.push_str("\r\n");+ request.push_str(body);+ stream+ .write_all(request.as_bytes())+ .await+ .expect("write request");+ stream.flush().await.expect("flush");+ let mut raw = Vec::new();+ stream.read_to_end(&mut raw).await.expect("read response");+ let text = String::from_utf8_lossy(&raw).into_owned();+ let status: u16 = text+ .split_whitespace()+ .nth(1)+ .and_then(|s| s.parse().ok())+ .unwrap_or_else(|| panic!("no status line in response: {text}"));+ // `Connection: close` means the body runs to EOF; chunked framing markers+ // are stripped so assertions can look at the payload.+ let body = match text.split_once("\r\n\r\n") {+ Some((_, rest)) => rest+ .lines()+ .filter(|l| !l.trim().is_empty() && u32::from_str_radix(l.trim(), 16).is_err())+ .collect::<Vec<_>>()+ .join(""),+ None => String::new(),+ };+ (status, body)+}++fn admin() -> String {+ format!("ApiKey {ADMIN_KEY}")+}++/// Mint a key and return its `Authorization` header value.+async fn mint(node: &Node, body: &str) -> String {+ let (status, resp) = http(node, "POST", "/_security/api_key", &admin(), body).await;+ assert_eq!(status, 200, "minting a key: {resp}");+ let v: serde_json::Value = serde_json::from_str(&resp).expect("key json");+ format!("ApiKey {}", v["encoded"].as_str().expect("encoded"))+}++/// A key scoped to exactly one brain — the documented grant shape.+fn brain_grant(brain: &str) -> String {+ format!(+ r#"{{"name":"{brain}-agent","role_descriptors":{{"{brain}":{{"indices":[+ {{"names":[".xerj-memory-{brain}-edges",".xerj-memory-{brain}"],+ "privileges":["read","write"]}}]}}}}}}"#+ )+}++/// Two brains, seeded as the admin. Bob's edge carries the marker the+/// verifier used, so a leak is unmistakable in an assertion message.+async fn seed(node: &Node) {+ for (brain, dst) in [("alice", "doc:2"), ("bob", "TOPSECRET:2")] {+ let (status, resp) = http(+ node,+ "POST",+ &format!("/_graph/{brain}/link"),+ &admin(),+ &format!(+ r#"{{"src":"doc:1","type":"mentions","dst":"{dst}","valid_at":{T0},"created_at":{T0}}}"#+ ),+ )+ .await;+ assert_eq!(status, 201, "seeding {brain}: {resp}");+ }+ for index in [".xerj-memory-alice-edges", ".xerj-memory-bob-edges"] {+ let (status, _) = http(node, "POST", &format!("/{index}/_refresh"), &admin(), "").await;+ assert_eq!(status, 200);+ }+}++/// The `_id` of bob's real edge, read as the admin — bypass 2's delete and+/// bypass 3's read both need a genuine document id, not a guess.+async fn bobs_edge_id(node: &Node) -> String {+ let (status, resp) = http(+ node,+ "POST",+ "/.xerj-memory-bob-edges/_search",+ &admin(),+ r#"{"query":{"match_all":{}},"size":50}"#,+ )+ .await;+ assert_eq!(status, 200, "reading bob as admin: {resp}");+ let v: serde_json::Value = serde_json::from_str(&resp).expect("search json");+ v["hits"]["hits"]+ .as_array()+ .expect("hits")+ .iter()+ .find(|h| h["_source"].get("edge_id").is_some())+ .map(|h| h["_id"].as_str().expect("_id").to_string())+ .expect("bob must have a real edge")+}++/// Bob's brain, verbatim, as the admin sees it. The point of comparison for+/// "the attack changed nothing".+/// Bob's brain contents, as the admin sees them, for before/after comparison.+///+/// Only `hits` is returned, deliberately: the surrounding envelope carries+/// `took`, a wall-clock millisecond count, so comparing whole response bodies+/// makes the "nothing touched bob" assertion fail whenever the two reads+/// happen to straddle a millisecond boundary. That is a flake, not a breach —+/// it fired once here with byte-identical `hits` and `took` 0 vs 1.+async fn bobs_edges(node: &Node) -> String {+ let (status, resp) = http(+ node,+ "POST",+ "/.xerj-memory-bob-edges/_search",+ &admin(),+ r#"{"query":{"match_all":{}},"size":50}"#,+ )+ .await;+ assert_eq!(status, 200, "reading bob as admin: {resp}");+ let parsed: serde_json::Value = serde_json::from_str(&resp).unwrap_or(serde_json::Value::Null);+ assert!(+ parsed.get("hits").is_some(),+ "reading bob as admin returned no hits envelope: {resp}"+ );+ parsed["hits"].to_string()+}++// ─────────────────────────────────────────────────────────────────────────────++/// BYPASS 1 — read, scoped key, via `_msearch`.+///+/// PROVED: a key granting only alice's two indices sent+/// `POST /.xerj-memory-alice-edges/_msearch` with a header line naming+/// `.xerj-memory-bob-edges`, and got 200 with bob's documents including+/// `{"src":"doc:1","dst":"TOPSECRET:2","type":"mentions"}`.+#[tokio::test]+async fn msearch_header_cannot_override_the_authorized_index() {+ let node = start_node().await;+ seed(&node).await;+ let alice = mint(&node, &brain_grant("alice")).await;++ let attack = "{\"index\":\".xerj-memory-bob-edges\"}\n{\"query\":{\"match_all\":{}}}\n";+ for path in [+ "/.xerj-memory-alice-edges/_msearch",+ "/_msearch",+ "/.xerj-memory-alice-edges/_msearch/template",+ ] {+ let (status, resp) = http(&node, "POST", path, &alice, attack).await;+ assert_eq!(status, 403, "POST {path} must be refused, got {status}");+ assert!(+ !resp.contains("TOPSECRET"),+ "POST {path} leaked bob: {resp}"+ );+ }++ // The same verb against her own brain still works — a fix that breaks+ // _msearch is not a fix.+ let own = "{\"index\":\".xerj-memory-alice-edges\"}\n{\"query\":{\"match_all\":{}}}\n";+ let (status, resp) = http(&node, "POST", "/_msearch", &alice, own).await;+ assert_eq!(status, 200, "alice's own _msearch: {resp}");+ assert!(resp.contains("doc:2"), "alice's own edge missing: {resp}");+}++/// BYPASS 2 — write and destroy, scoped key, via `_bulk`.+///+/// PROVED: alice sent `POST /.xerj-memory-alice-edges/_bulk` with an action+/// line naming `.xerj-memory-bob-edges` and got `"result":"created"`; an+/// admin `GET` of that id returned `found:true`. A `delete` action against a+/// real edge id then DELETED bob's genuine edge — the two doors+/// (`authz.rs`'s own module header calls them "forge an edge around the+/// derived-edge_id invariant" and "destroy the brain") it claimed to have shut.+#[tokio::test]+async fn bulk_action_line_cannot_override_the_authorized_index() {+ let node = start_node().await;+ seed(&node).await;+ let alice = mint(&node, &brain_grant("alice")).await;+ let victim_id = bobs_edge_id(&node).await;+ let before = bobs_edges(&node).await;++ // Forge.+ let forge = "{\"index\":{\"_index\":\".xerj-memory-bob-edges\",\"_id\":\"forged-s\"}}\n\+ {\"src\":\"doc:1\",\"dst\":\"attacker:payload\"}\n";+ for path in ["/.xerj-memory-alice-edges/_bulk", "/_bulk"] {+ let (status, resp) = http(&node, "POST", path, &alice, forge).await;+ assert_eq!(status, 403, "POST {path} forge must be refused: {resp}");+ }+ let (status, resp) = http(+ &node,+ "GET",+ "/.xerj-memory-bob-edges/_doc/forged-s",+ &admin(),+ "",+ )+ .await;+ assert!(+ status == 404 || resp.contains("\"found\":false"),+ "the forged document reached bob's brain: {status} {resp}"+ );++ // Destroy.+ let destroy = format!(+ "{{\"delete\":{{\"_index\":\".xerj-memory-bob-edges\",\"_id\":\"{victim_id}\"}}}}\n"+ );+ for path in ["/.xerj-memory-alice-edges/_bulk", "/_bulk"] {+ let (status, resp) = http(&node, "POST", path, &alice, &destroy).await;+ assert_eq!(status, 403, "POST {path} delete must be refused: {resp}");+ }+ let after = bobs_edges(&node).await;+ assert_eq!(before, after, "bob's brain was modified by the attack");+ assert!(after.contains("TOPSECRET"), "bob's edge is gone: {after}");++ // Alice's own bulk, by both spellings, still works.+ let own = "{\"index\":{\"_index\":\".xerj-memory-alice-edges\",\"_id\":\"own-1\"}}\n\+ {\"src\":\"a\",\"dst\":\"b\"}\n";+ let (status, resp) = http(&node, "POST", "/_bulk", &alice, own).await;+ assert_eq!(status, 200, "alice's own global _bulk: {resp}");+ assert!(+ !resp.contains("\"errors\":true"),+ "own bulk errored: {resp}"+ );+ let scoped = "{\"index\":{\"_id\":\"own-2\"}}\n{\"src\":\"a\",\"dst\":\"c\"}\n";+ let (status, resp) = http(+ &node,+ "POST",+ "/.xerj-memory-alice-edges/_bulk",+ &alice,+ scoped,+ )+ .await;+ assert_eq!(status, 200, "alice's own index-scoped _bulk: {resp}");+ assert!(+ !resp.contains("\"errors\":true"),+ "own bulk errored: {resp}"+ );+}++/// BYPASS 3 — read, scoped key, via `_mget`.+///+/// PROVED: `POST /.xerj-memory-alice-edges/_mget` with+/// `{"docs":[{"_index":".xerj-memory-bob-edges","_id":"<real id>"}]}` returned+/// 200 with bob's edge source containing `"dst":"TOPSECRET:2"`.+#[tokio::test]+async fn mget_docs_cannot_override_the_authorized_index() {+ let node = start_node().await;+ seed(&node).await;+ let alice = mint(&node, &brain_grant("alice")).await;+ let victim_id = bobs_edge_id(&node).await;++ let attack =+ format!(r#"{{"docs":[{{"_index":".xerj-memory-bob-edges","_id":"{victim_id}"}}]}}"#);+ for path in ["/.xerj-memory-alice-edges/_mget", "/_mget"] {+ let (status, resp) = http(&node, "POST", path, &alice, &attack).await;+ assert_eq!(status, 403, "POST {path} must be refused, got {status}");+ assert!(+ !resp.contains("TOPSECRET"),+ "POST {path} leaked bob: {resp}"+ );+ }++ // The short form against her own index is untouched.+ let (status, resp) = http(+ &node,+ "POST",+ "/.xerj-memory-alice-edges/_mget",+ &alice,+ r#"{"ids":["__xerj-brain-meta"]}"#,+ )+ .await;+ assert_eq!(status, 200, "alice's own _mget: {resp}");+}++/// BYPASS 4 — read, unscoped key, via aliases.+///+/// PROVED: a key with no `role_descriptors` sent `POST /_aliases` adding an+/// alias onto `.xerj-memory-bob-edges` (200 acknowledged), then searched the+/// alias and got bob's edges. `PUT /.xerj-memory-bob-edges/_alias/pwned` was+/// already refused, so this was precisely the "one missed shape" the module+/// documentation warned about. The second half matters as much: an alias an+/// operator had already pointed into the namespace made it readable by every+/// unscoped key.+#[tokio::test]+async fn aliases_cannot_launder_a_reserved_index() {+ let node = start_node().await;+ seed(&node).await;+ let unscoped = mint(&node, r#"{"name":"plain"}"#).await;++ // Creating the alias is refused, by both spellings.+ let add = r#"{"actions":[{"add":{"index":".xerj-memory-bob-edges","alias":"pwned-u"}}]}"#;+ let (status, resp) = http(&node, "POST", "/_aliases", &unscoped, add).await;+ assert_eq!(status, 403, "POST /_aliases must be refused: {resp}");+ let (status, resp) = http(+ &node,+ "PUT",+ "/.xerj-memory-bob-edges/_alias/pwned-p",+ &unscoped,+ "",+ )+ .await;+ assert_eq!(status, 403, "PUT .../_alias must be refused: {resp}");+ let (status, resp) = http(&node, "POST", "/pwned-u/_search", &unscoped, "{}").await;+ assert!(+ !resp.contains("TOPSECRET"),+ "the alias was created after all: {status} {resp}"+ );++ // Second half: an alias the OPERATOR already pointed into the namespace+ // must not become a back door either. Only the admin can make one.+ let operator_alias =+ r#"{"actions":[{"add":{"index":".xerj-memory-bob-edges","alias":"ops-view"}}]}"#;+ let (status, resp) = http(&node, "POST", "/_aliases", &admin(), operator_alias).await;+ assert_eq!(status, 200, "admin must still manage aliases: {resp}");++ for path in ["/ops-view/_search", "/ops-view/_count"] {+ let (status, resp) = http(&node, "POST", path, &unscoped, "{}").await;+ assert_eq!(+ status, 403,+ "reading through an operator alias must be refused: {resp}"+ );+ assert!(!resp.contains("TOPSECRET"), "alias leaked bob: {resp}");+ }+ // …including through a body-named door.+ let (status, resp) = http(+ &node,+ "POST",+ "/_msearch",+ &unscoped,+ "{\"index\":\"ops-view\"}\n{\"query\":{\"match_all\":{}}}\n",+ )+ .await;+ assert_eq!(status, 403, "aliased _msearch must be refused: {resp}");+ assert!(+ !resp.contains("TOPSECRET"),+ "aliased _msearch leaked: {resp}"+ );++ // An alias over an ordinary index is ordinary work and still allowed.+ let (status, resp) = http(+ &node,+ "PUT",+ "/logs-2026",+ &unscoped,+ r#"{"mappings":{"properties":{"m":{"type":"integer"}}}}"#,+ )+ .await;+ assert_eq!(status, 200, "creating an ordinary index: {resp}");+ let ordinary = r#"{"actions":[{"add":{"index":"logs-2026","alias":"logs"}}]}"#;+ let (status, resp) = http(&node, "POST", "/_aliases", &unscoped, ordinary).await;+ assert_eq!(status, 200, "an ordinary alias must still work: {resp}");+ // But an alias NAME inside the reserved namespace is squatting.+ let squat = r#"{"actions":[{"add":{"index":"logs-2026","alias":".xerj-memory-victim"}}]}"#;+ let (status, resp) = http(&node, "POST", "/_aliases", &unscoped, squat).await;+ assert_eq!(status, 403, "squatting a reserved alias name: {resp}");+}++/// The fifth shape, found by auditing the tree rather than by being attacked:+/// a `terms` lookup names an index to read terms out of, at arbitrary depth+/// inside a query body that the URL says is about something else.+#[tokio::test]+async fn terms_lookup_cannot_read_an_unauthorized_index() {+ let node = start_node().await;+ seed(&node).await;+ let alice = mint(&node, &brain_grant("alice")).await;+ let victim_id = bobs_edge_id(&node).await;++ let attack = format!(+ r#"{{"query":{{"bool":{{"filter":[{{"terms":{{"src":{{"index":".xerj-memory-bob-edges","id":"{victim_id}","path":"dst"}}}}}}]}}}}}}"#+ );+ let (status, resp) = http(+ &node,+ "POST",+ "/.xerj-memory-alice-edges/_search",+ &alice,+ &attack,+ )+ .await;+ assert_eq!(status, 403, "a terms lookup into bob must be refused");+ assert!(!resp.contains("TOPSECRET"), "terms lookup leaked: {resp}");++ // The same shape one layer over: a `lookup` runtime field joins rows out+ // of another index into the hits.+ let runtime = r#"{"runtime_mappings":{"u":{"type":"lookup","target_index":".xerj-memory-bob-edges","input_field":"src","target_field":"_id","fetch_fields":["dst"]}},"fields":["u"]}"#;+ let (status, resp) = http(+ &node,+ "POST",+ "/.xerj-memory-alice-edges/_search",+ &alice,+ runtime,+ )+ .await;+ assert_eq!(status, 403, "a lookup runtime field into bob: {resp}");+ assert!(!resp.contains("TOPSECRET"), "runtime lookup leaked: {resp}");+}++/// The reason this fix is two layers and not a patch list.+///+/// `_sql` names its index in the FROM clause of a SQL string, an ingest+/// pipeline names one in a `route` processor it stored earlier, an index+/// template names a *pattern* that decides the mapping of brains created+/// later. None of those is a JSON field the middleware parses — and none of+/// them has to be, because the name still has to become an index through+/// `Engine::get_index` / `get_or_create_index` / `create_index`, where the+/// request's principal is installed as a visibility rule. This is the property+/// that a per-handler patch list cannot have.+#[tokio::test]+async fn shapes_the_middleware_never_parses_still_fail_closed() {+ let node = start_node().await;+ seed(&node).await;+ let alice = mint(&node, &brain_grant("alice")).await;+ let unscoped = mint(&node, r#"{"name":"plain"}"#).await;+ let before = bobs_edges(&node).await;++ // `_sql` — the table name is inside the statement.+ let (status, resp) = http(+ &node,+ "POST",+ "/_sql",+ &alice,+ r#"{"query":"SELECT * FROM \".xerj-memory-bob-edges\""}"#,+ )+ .await;+ assert_ne!(status, 200, "_sql reached bob's brain: {resp}");+ assert!(!resp.contains("TOPSECRET"), "_sql leaked bob: {resp}");++ // An index template that would own the mapping of every brain created+ // from here on.+ let (status, resp) = http(+ &node,+ "PUT",+ "/_index_template/grab",+ &unscoped,+ r#"{"index_patterns":[".xerj-memory-*"],"template":{"mappings":{"properties":{"dst":{"type":"keyword"}}}}}"#,+ )+ .await;+ assert_eq!(status, 403, "a namespace-wide template: {resp}");++ // `POST /v1/pipelines/{name}` can register a `route` processor whose+ // target index is chosen at ingest time, not at request time.+ let (status, resp) = http_native(+ &node,+ "PUT",+ "/v1/pipelines/exfil",+ &unscoped,+ r#"{"steps":[{"route":{"index":".xerj-memory-bob-edges"}}]}"#,+ )+ .await;+ // Whether the pipeline definition is accepted is the pipeline API's+ // business; what matters is that ingesting through it cannot reach bob.+ let _ = (status, resp);+ let (status, resp) = http_native(+ &node,+ "POST",+ "/v1/indices/logs-pipe/ingest?pipeline=exfil",+ &unscoped,+ r#"{"documents":[{"src":"a","dst":"b"}]}"#,+ )+ .await;+ assert!(+ !resp.contains("TOPSECRET"),+ "pipeline ingest leaked bob: {status} {resp}"+ );++ // Nothing above touched bob's brain.+ let after = bobs_edges(&node).await;+ assert_eq!(before, after, "bob's brain changed: {before} vs {after}");+}++/// COMPATIBILITY — the regression the same branch introduced, also measured.+///+/// Every non-superuser credential, including a plain minted key, got 403 on+/// `POST /_bulk`, `/_msearch`, `/_mget`, `/_search` and `/_count`. The global+/// `_bulk` is the default write path for essentially every ES client, so that+/// broke ordinary usage rather than securing it. A scoped key additionally got+/// 403 on all cluster metadata and on every wildcard, including one it had+/// been granted, which Kibana cannot survive.+#[tokio::test]+async fn ordinary_credentials_keep_the_global_surface() {+ let node = start_node().await;+ seed(&node).await;+ let unscoped = mint(&node, r#"{"name":"plain"}"#).await;+ let alice = mint(&node, &brain_grant("alice")).await;+ let wide = mint(+ &node,+ r#"{"name":"wide","role_descriptors":{"w":{"indices":[+ {"names":["logs-*"],"privileges":["all"]}]}}}"#,+ )+ .await;++ let (status, resp) = http(+ &node,+ "PUT",+ "/logs-2026",+ &unscoped,+ r#"{"mappings":{"properties":{"m":{"type":"integer"}}}}"#,+ )+ .await;+ assert_eq!(status, 200, "creating an ordinary index: {resp}");++ // (a) The five global verbs answer for an ordinary credential.+ for (method, path, body) in [+ ("POST", "/_search", "{}"),+ ("POST", "/_count", "{}"),+ ("POST", "/_msearch", "{}\n{\"query\":{\"match_all\":{}}}\n"),+ (+ "POST",+ "/_mget",+ r#"{"docs":[{"_index":"logs-2026","_id":"1"}]}"#,+ ),+ (+ "POST",+ "/_bulk",+ "{\"index\":{\"_index\":\"logs-2026\",\"_id\":\"1\"}}\n{\"m\":1}\n",+ ),+ ] {+ for (who, cred) in [("unscoped", &unscoped), ("wide", &wide)] {+ let (status, resp) = http(&node, method, path, cred, body).await;+ assert_eq!(+ status, 200,+ "{who} {method} {path} must work, got {status}: {resp}"+ );+ }+ }++ // (b) Cluster metadata answers for a SCOPED key, filtered rather than+ // refused. Kibana polls all of these on every page load.+ for path in [+ "/_cluster/health",+ "/_nodes",+ "/_mapping",+ "/_settings",+ "/_alias",+ "/_cat/indices",+ "/_cat/health",+ "/_cat/nodes",+ "/_cat/aliases",+ "/_cluster/state",+ "/_cluster/stats",+ "/_resolve/index/*",+ "/_xpack",+ ] {+ let (status, resp) = http(&node, "GET", path, &alice, "").await;+ assert_eq!(+ status, 200,+ "a scoped key must read {path}, got {status}: {resp}"+ );+ assert!(+ !resp.contains("TOPSECRET") && !resp.contains("bob"),+ "{path} leaked another tenant: {resp}"+ );+ }+ // …and the native router's own metadata, which lives on the other listener.+ for path in ["/v1/health", "/v1/cluster/health", "/v1/dashboard/summary"] {+ let (status, resp) = http_native(&node, "GET", path, &alice, "").await;+ assert_eq!(+ status, 200,+ "a scoped key must read native {path}, got {status}: {resp}"+ );+ assert!(!resp.contains("bob"), "native {path} leaked: {resp}");+ }++ // (c) A granted wildcard actually works, and the data it returns is the+ // grant's, not the cluster's.+ let (status, resp) = http(+ &node,+ "POST",+ "/_bulk",+ &wide,+ "{\"index\":{\"_index\":\"logs-2026\",\"_id\":\"w1\"}}\n{\"m\":7}\n",+ )+ .await;+ assert_eq!(status, 200, "granted wildcard write: {resp}");+ let (status, _) = http(&node, "POST", "/logs-2026/_refresh", &admin(), "").await;+ assert_eq!(status, 200);+ let (status, resp) = http(&node, "POST", "/logs-*/_search", &wide, "{}").await;+ assert_eq!(status, 200, "granted wildcard read: {resp}");+ assert!(resp.contains("logs-2026"), "wildcard found nothing: {resp}");+ assert!(!resp.contains("TOPSECRET"), "wildcard leaked bob: {resp}");++ // (d) An UNgranted wildcard resolves to nothing instead of leaking what it+ // would have matched — and instead of a 403 that would tell the caller the+ // pattern was interesting.+ for path in ["/*/_search", "/_all/_search", "/.xerj-memory-*/_search"] {+ let (status, resp) = http(&node, "POST", path, &wide, "{}").await;+ assert_eq!(status, 200, "POST {path}: {resp}");+ assert!(!resp.contains("TOPSECRET"), "POST {path} leaked: {resp}");+ }+}++/// The zero-configuration local path — `xerj --insecure`, point-at-a-folder —+/// is one user with no credentials and must be completely unaffected.+#[tokio::test]+async fn insecure_single_user_mode_is_unchanged() {+ let dir = tempfile::tempdir().expect("tempdir");+ let mut config = Config::default();+ config.server.data_dir = dir.path().to_str().unwrap().to_string();+ config.auth.enabled = false;+ config.auth.admin_api_key = String::new();+ let metrics = Metrics::new().expect("metrics");+ let engine = Engine::new(config.clone()).expect("engine");+ let state = AppState::new(config, engine, metrics);+ let node = Node {+ addr: serve(build_es_compat_router(state.clone())).await,+ native_addr: serve(build_native_router(state)).await,+ _data_dir: dir,+ };++ let (status, resp) = http(+ &node,+ "POST",+ "/_graph/solo/link",+ "",+ &format!(r#"{{"src":"a","type":"mentions","dst":"b","valid_at":{T0},"created_at":{T0}}}"#),+ )+ .await;+ assert_eq!(status, 201, "no-credential link must work: {resp}");+ for (method, path, body) in [+ ("GET", "/_graph/solo/overview", ""),+ ("POST", "/_search", "{}"),+ (+ "POST",+ "/_bulk",+ "{\"index\":{\"_index\":\"anything\"}}\n{\"a\":1}\n",+ ),+ ("GET", "/_cat/indices", ""),+ ("GET", "/_mapping", ""),+ ("POST", "/.xerj-memory-solo-edges/_search", "{}"),+ (+ "POST",+ "/_aliases",+ r#"{"actions":[{"add":{"index":".xerj-memory-solo-edges","alias":"solo"}}]}"#,+ ),+ ("POST", "/solo/_search", "{}"),+ ] {+ let (status, resp) = http(&node, method, path, "", body).await;+ assert_eq!(+ status, 200,+ "{method} {path} must work with no credential: {resp}"+ );+ }+ // The native router's body-named create, unrestricted for the single user.+ let (status, resp) = http_native(+ &node,+ "POST",+ "/v1/indices",+ "",+ r#"{"name":"solo-notes","fields":[]}"#,+ )+ .await;+ assert_eq!(status, 201, "native create with no credential: {resp}");+}++/// The native router's `POST /v1/indices` names its index in the body. The+/// first cut refused it outright for every non-superuser, which broke the+/// native create path; it is authorized against the name it carries instead.+#[tokio::test]+async fn native_body_named_create_is_authorized_not_refused() {+ let node = start_node().await;+ let unscoped = mint(&node, r#"{"name":"plain"}"#).await;++ let (status, resp) = http_native(+ &node,+ "POST",+ "/v1/indices",+ &unscoped,+ r#"{"name":"logs-native","fields":[]}"#,+ )+ .await;+ assert_eq!(status, 201, "an ordinary native create must work: {resp}");++ // Squatting a brain name the caller could then not read is refused.+ let (status, resp) = http_native(+ &node,+ "POST",+ "/v1/indices",+ &unscoped,+ r#"{"name":".xerj-memory-victim-edges","fields":[]}"#,+ )+ .await;+ assert_eq!(status, 403, "squatting the namespace: {resp}");+ let (status, resp) = http(&node, "GET", "/.xerj-memory-victim-edges", &admin(), "").await;+ assert_eq!(status, 404, "the squatted index was created: {resp}");+}diff --git a/engine/crates/xerj-api/tests/brain_is_a_security_boundary.rs b/engine/crates/xerj-api/tests/brain_is_a_security_boundary.rsnew file mode 100644index 000000000..e186eb1f1--- /dev/null+++ b/engine/crates/xerj-api/tests/brain_is_a_security_boundary.rs@@ -0,0 +1,692 @@+//! Executable statement of the second-brain authorization posture (issue #79).+//!+//! A brain **is** a security boundary. This file is the inverted descendant of+//! `brain_is_not_a_security_boundary.rs`, which pinned the measured fact that+//! one did *not* exist: any authenticated caller could read, write, forge and+//! destroy every brain, and `GET /_mapping` handed over the list of brain+//! names so the attacker did not even have to guess one. Every assertion that+//! file made now runs in reverse.+//!+//! The point of enumerating so many doors is that the exposure was never about+//! the four `/_graph/*` handlers. A brain's edges live in an ordinary index and+//! `IndexName::validate` admits a leading `.`, so `.xerj-memory-{brain}-edges`+//! was reachable through the generic ES-compat surface *and* through the+//! native router's `/v1/indices/{name}/…` spelling. An access check on+//! `/_graph/*` alone would have left all of that open — a boundary that only+//! looks like one. So this test walks every door, and a fix that closes only+//! some of them fails here.+//!+//! Every request below is made with a **minted, non-admin** API key, because+//! the issue is about what an ordinary authenticated caller can reach — not+//! about the superuser.++use axum::body::Body;+use axum::http::{Request, StatusCode};+use http_body_util::BodyExt;+use serde_json::Value;+use tower::ServiceExt;+use xerj_api::{+ router::{build_es_compat_router, build_native_router},+ state::AppState,+};+use xerj_common::{config::Config, metrics::Metrics};+use xerj_engine::Engine;++const ADMIN_KEY: &str = "admin-secret-key-for-boundary-test";++/// Fixture instants. Pinned rather than "now" so a link and the `as_of` that+/// reads it back can never land in the same millisecond.+const T0: i64 = 1_753_600_000_000;+const AS_OF: i64 = 1_753_700_000_000;++/// An auth-enabled node over a fresh data directory. The `TempDir` is returned+/// and must be held for the whole test: the Engine keeps the directory open,+/// and these are engine data dirs, so leaking them fills `/tmp`.+fn auth_enabled_state() -> (AppState, tempfile::TempDir) {+ let dir = tempfile::tempdir().expect("tempdir");+ let state = state_over(dir.path().to_str().unwrap(), true);+ (state, dir)+}++fn state_over(data_dir: &str, auth: bool) -> AppState {+ let mut config = Config::default();+ config.server.data_dir = data_dir.to_string();+ config.auth.enabled = auth;+ config.auth.admin_api_key = if auth {+ ADMIN_KEY.to_string()+ } else {+ String::new()+ };+ let metrics = Metrics::new().expect("metrics");+ let engine = Engine::new(config.clone()).expect("engine");+ AppState::new(config, engine, metrics)+}++/// Like [`send`] but returns the body verbatim — `_cat` answers a plain-text+/// table, which is exactly the shape the line-level pruning has to handle.+async fn send_text(+ app: &axum::Router,+ method: &str,+ uri: &str,+ auth: &str,+) -> (StatusCode, String) {+ let mut builder = Request::builder().method(method).uri(uri);+ if !auth.is_empty() {+ builder = builder.header("authorization", auth);+ }+ let req = builder.body(Body::empty()).expect("request");+ let resp = app.clone().oneshot(req).await.expect("response");+ let status = resp.status();+ let bytes = resp.into_body().collect().await.expect("body").to_bytes();+ (status, String::from_utf8_lossy(&bytes).into_owned())+}++async fn send(+ app: &axum::Router,+ method: &str,+ uri: &str,+ auth: &str,+ body: &str,+) -> (StatusCode, Value) {+ let mut builder = Request::builder()+ .method(method)+ .uri(uri)+ .header("content-type", "application/json");+ if !auth.is_empty() {+ builder = builder.header("authorization", auth);+ }+ let req = builder.body(Body::from(body.to_string())).expect("request");+ let resp = app.clone().oneshot(req).await.expect("response");+ let status = resp.status();+ let bytes = resp.into_body().collect().await.expect("body").to_bytes();+ let json: Value = serde_json::from_slice(&bytes).unwrap_or(Value::Null);+ (status, json)+}++/// Mint an API key with the given JSON body and return its `Authorization`+/// header value.+async fn mint(app: &axum::Router, minter: &str, body: &str) -> String {+ let (status, resp) = send(app, "POST", "/_security/api_key", minter, body).await;+ assert_eq!(+ status,+ StatusCode::OK,+ "minting a key should succeed: {resp}"+ );+ let encoded = resp["encoded"].as_str().expect("encoded key").to_string();+ format!("ApiKey {encoded}")+}++/// A key scoped to exactly one brain: its edges index and its nodes index.+/// This is the documented grant shape from `graph_api`'s module docs.+fn brain_grant(brain: &str, privileges: &str) -> String {+ format!(+ r#"{{"name":"{brain}-agent","role_descriptors":{{"{brain}":{{"indices":[+ {{"names":[".xerj-memory-{brain}-edges",".xerj-memory-{brain}"],+ "privileges":[{privileges}]}}]}}}}}}"#+ )+}++/// Seed two brains, as the admin, so there is something to keep apart.+async fn seed_two_brains(app: &axum::Router) {+ let admin = format!("ApiKey {ADMIN_KEY}");+ for (brain, dst) in [("alice", "doc:2"), ("bob", "secret:2")] {+ let (status, body) = send(+ app,+ "POST",+ &format!("/_graph/{brain}/link"),+ &admin,+ &format!(+ r#"{{"src":"doc:1","type":"mentions","dst":"{dst}",+ "valid_at":{T0},"created_at":{T0}}}"#+ ),+ )+ .await;+ assert_eq!(status, StatusCode::CREATED, "seeding {brain}: {body}");+ }+ let (status, _) = send(app, "POST", "/.xerj-memory-bob-edges/_refresh", &admin, "").await;+ assert_eq!(status, StatusCode::OK);+}++// ─────────────────────────────────────────────────────────────────────────────++/// Every door into someone else's brain, walked with a credential scoped to a+/// different brain. This is the assertion the issue asked for, and the exact+/// inverse of what the previous test measured.+#[tokio::test]+async fn a_scoped_caller_cannot_reach_another_brain_through_any_door() {+ let (state, _data_dir) = auth_enabled_state();+ let app = build_es_compat_router(state.clone());+ let native = build_native_router(state);+ seed_two_brains(&app).await;++ let admin = format!("ApiKey {ADMIN_KEY}");+ let alice = mint(&app, &admin, &brain_grant("alice", r#""read","write""#)).await;++ // ── 1. The graph API: all four entry points, denied for bob. ───────────+ for (method, uri) in [+ ("GET", "/_graph/bob/ego?node=doc:1"),+ ("GET", "/_graph/bob/overview"),+ ("POST", "/_graph/bob/link"),+ ("DELETE", "/_graph/bob/link/whatever"),+ ] {+ let (status, body) = send(+ &app,+ method,+ uri,+ &alice,+ r#"{"src":"a","type":"mentions","dst":"b"}"#,+ )+ .await;+ assert_eq!(+ status,+ StatusCode::FORBIDDEN,+ "{method} {uri} must be forbidden, got {status}: {body}"+ );+ }++ // ── 2. The raw index surface, which is where the real exposure lived. ──+ // Read around `/_graph`, forge an edge past the derived-`edge_id`+ // invariant, refresh, and destroy the brain outright.+ for (method, uri, body) in [+ (+ "POST",+ "/.xerj-memory-bob-edges/_search",+ r#"{"query":{"match_all":{}}}"#,+ ),+ (+ "POST",+ "/.xerj-memory-bob-edges/_doc/forged",+ r#"{"edge_id":"forged","src":"doc:1","dst":"attacker:payload","type":"mentions"}"#,+ ),+ ("POST", "/.xerj-memory-bob-edges/_refresh", ""),+ ("GET", "/.xerj-memory-bob-edges/_doc/anything", ""),+ ("DELETE", "/.xerj-memory-bob-edges", ""),+ // Percent-encoded, in case the check compared raw path text.+ ("POST", "/%2Exerj-memory-bob-edges/_search", "{}"),+ ] {+ let (status, resp) = send(&app, method, uri, &alice, body).await;+ assert_eq!(+ status,+ StatusCode::FORBIDDEN,+ "{method} {uri} must be forbidden, got {status}: {resp}"+ );+ }++ // ── 2b. Patterns are answered, not refused — over the caller's visible+ // set only. Refusing them outright was the first cut of this fix and it+ // cost every legitimate `logs-*` grant; expanding them over what the+ // principal may see is both usable and closed. What must never appear is+ // one byte of bob's.+ for uri in [+ "/.xerj-memory-*/_search",+ "/*/_search",+ "/_all/_search",+ "/.xerj-memory-bob*/_search",+ ] {+ let (status, resp) = send(&app, "POST", uri, &alice, "{}").await;+ assert_eq!(+ status,+ StatusCode::OK,+ "POST {uri} should be answered over the visible set, got {status}: {resp}"+ );+ assert!(+ !resp.to_string().contains("bob"),+ "POST {uri} leaked bob: {resp}"+ );+ }++ // ── 3. The agent-memory surface: bob's brain keeps its NODES in the+ // `.xerj-memory-bob` namespace, so `/_memory/*` is a door to the same data.+ for (method, uri) in [+ ("POST", "/_memory/bob/_recall"),+ ("GET", "/_memory/bob"),+ ("POST", "/_memory/bob"),+ ("DELETE", "/_memory/bob"),+ ("DELETE", "/_memory/bob/some-id"),+ ] {+ let (status, resp) = send(&app, method, uri, &alice, r#"{"text":"x","query":"x"}"#).await;+ assert_eq!(+ status,+ StatusCode::FORBIDDEN,+ "{method} {uri} must be forbidden, got {status}: {resp}"+ );+ }++ // ── 4. Enumeration: a scoped caller gets metadata for the indices it+ // holds and no trace of anything else, so brain names stay unguessable+ // WITHOUT the blanket 403 that stopped Kibana from working at all.+ for uri in [+ "/_mapping",+ "/_cat/indices",+ "/_alias",+ "/_settings",+ "/_cluster/stats",+ "/_resolve/index/*",+ "/_cluster/health",+ "/_nodes",+ ] {+ let (status, resp) = send(&app, "GET", uri, &alice, "").await;+ assert_eq!(+ status,+ StatusCode::OK,+ "GET {uri} must be answered for a scoped key, got {status}: {resp}"+ );+ assert!(+ !resp.to_string().contains("bob"),+ "GET {uri} enumerated bob's brain: {resp}"+ );+ }+ let (status, table) = send_text(&app, "GET", "/_cat/indices", &alice).await;+ assert_eq!(status, StatusCode::OK);+ assert!(+ !table.contains("bob"),+ "_cat/indices enumerated bob's brain: {table}"+ );++ // ── 5. Unnamed fan-out. `POST /_search` with no index reads every index+ // the caller can see — which is alice's, and only alice's. The global+ // verbs are the default paths of every ES client, so they answer; what+ // they must not do is answer with someone else's data.+ for (method, uri, body) in [+ ("POST", "/_search", "{}"),+ ("POST", "/_msearch", "{}\n{\"query\":{\"match_all\":{}}}\n"),+ ("POST", "/_count", "{}"),+ ] {+ let (status, resp) = send(&app, method, uri, &alice, body).await;+ assert_eq!(+ status,+ StatusCode::OK,+ "{method} {uri} must be answered over the visible set, got {status}: {resp}"+ );+ assert!(+ !resp.to_string().contains("bob"),+ "{method} {uri} leaked bob: {resp}"+ );+ }+ // The global write path works for the caller's own index …+ let (status, resp) = send(+ &app,+ "POST",+ "/_bulk",+ &alice,+ "{\"index\":{\"_index\":\".xerj-memory-alice-edges\",\"_id\":\"own\"}}\n{\"src\":\"a\"}\n",+ )+ .await;+ assert_eq!(+ status,+ StatusCode::OK,+ "global _bulk into its own index must work: {resp}"+ );+ // … and refuses the moment the body names someone else's.+ for body in [+ "{\"index\":{\"_index\":\".xerj-memory-bob-edges\",\"_id\":\"forged\"}}\n{\"src\":\"a\"}\n",+ "{\"delete\":{\"_index\":\".xerj-memory-bob-edges\",\"_id\":\"e1\"}}\n",+ ] {+ let (status, resp) = send(&app, "POST", "/_bulk", &alice, body).await;+ assert_eq!(+ status,+ StatusCode::FORBIDDEN,+ "a body-named _bulk into bob must be forbidden, got {status}: {resp}"+ );+ }+ // Same for the body-named forms of _msearch and _mget.+ let (status, resp) = send(+ &app,+ "POST",+ "/_msearch",+ &alice,+ "{\"index\":\".xerj-memory-bob-edges\"}\n{\"query\":{\"match_all\":{}}}\n",+ )+ .await;+ assert_eq!(status, StatusCode::FORBIDDEN, "body-named _msearch: {resp}");+ let (status, resp) = send(+ &app,+ "POST",+ "/_mget",+ &alice,+ "{\"docs\":[{\"_index\":\".xerj-memory-bob-edges\",\"_id\":\"x\"}]}",+ )+ .await;+ assert_eq!(status, StatusCode::FORBIDDEN, "body-named _mget: {resp}");++ // ── 6. The native router is the same engine under a different spelling.+ for (method, uri) in [+ ("POST", "/v1/indices/.xerj-memory-bob-edges/search"),+ ("GET", "/v1/indices/.xerj-memory-bob-edges"),+ ("DELETE", "/v1/indices/.xerj-memory-bob-edges"),+ ("POST", "/v1/indices/.xerj-memory-bob-edges/docs"),+ ] {+ let (status, resp) = send(&native, method, uri, &alice, r#"{"query":{}}"#).await;+ assert_eq!(+ status,+ StatusCode::FORBIDDEN,+ "native {method} {uri} must be forbidden, got {status}: {resp}"+ );+ }++ // ── 7. Escalation: a scoped caller must not be able to mint itself a key+ // that grants bob.+ let (status, resp) = send(+ &app,+ "POST",+ "/_security/api_key",+ &alice,+ &brain_grant("bob", r#""all""#),+ )+ .await;+ assert_eq!(+ status,+ StatusCode::FORBIDDEN,+ "a scoped key must not mint keys, got {status}: {resp}"+ );++ // ── 8. Existence must not leak through the status code: a brain that does+ // not exist is refused exactly like one that does.+ let (status, _) = send(&app, "GET", "/_graph/nosuchbrain/overview", &alice, "").await;+ assert_eq!(+ status,+ StatusCode::FORBIDDEN,+ "403 (not 404) for an unauthorized brain, or the code maps the node"+ );+}++/// The boundary is only worth anything if the authorized side still works.+/// A key scoped to `alice` must be able to do everything a brain owner does.+#[tokio::test]+async fn a_scoped_caller_retains_full_use_of_its_own_brain() {+ let (state, _data_dir) = auth_enabled_state();+ let app = build_es_compat_router(state);+ seed_two_brains(&app).await;++ let admin = format!("ApiKey {ADMIN_KEY}");+ let alice = mint(&app, &admin, &brain_grant("alice", r#""read","write""#)).await;++ // Write.+ let (status, body) = send(+ &app,+ "POST",+ "/_graph/alice/link",+ &alice,+ &format!(+ r#"{{"src":"doc:1","type":"mentions","dst":"doc:3",+ "valid_at":{T0},"created_at":{T0}}}"#+ ),+ )+ .await;+ assert_eq!(status, StatusCode::CREATED, "own-brain link failed: {body}");+ let edge_id = body["edge_id"].as_str().expect("edge_id").to_string();++ let (status, _) = send(+ &app,+ "POST",+ "/.xerj-memory-alice-edges/_refresh",+ &alice,+ "",+ )+ .await;+ assert_eq!(status, StatusCode::OK, "own backing index is usable");++ // Read, both entry points.+ let (status, ego) = send(+ &app,+ "GET",+ &format!("/_graph/alice/ego?node=doc:1&as_of={AS_OF}"),+ &alice,+ "",+ )+ .await;+ assert_eq!(status, StatusCode::OK, "own-brain ego failed: {ego}");+ assert!(+ ego.to_string().contains("doc:3"),+ "own edge should be visible: {ego}"+ );+ let (status, overview) = send(+ &app,+ "GET",+ &format!("/_graph/alice/overview?as_of={AS_OF}"),+ &alice,+ "",+ )+ .await;+ assert_eq!(+ status,+ StatusCode::OK,+ "own-brain overview failed: {overview}"+ );++ // Invalidate.+ let (status, body) = send(+ &app,+ "DELETE",+ &format!("/_graph/alice/link/{edge_id}"),+ &alice,+ "",+ )+ .await;+ assert_eq!(status, StatusCode::OK, "own-brain unlink failed: {body}");++ // A read-only grant reads but does not write — the privilege half of the+ // model, not just the resource half.+ let reader = mint(&app, &admin, &brain_grant("alice", r#""read""#)).await;+ let (status, _) = send(&app, "GET", "/_graph/alice/overview", &reader, "").await;+ assert_eq!(status, StatusCode::OK, "read grant must read");+ let (status, _) = send(+ &app,+ "POST",+ "/_graph/alice/link",+ &reader,+ r#"{"src":"a","type":"mentions","dst":"b"}"#,+ )+ .await;+ assert_eq!(status, StatusCode::FORBIDDEN, "read grant must not write");+ let (status, _) = send(&app, "DELETE", "/.xerj-memory-alice-edges", &reader, "").await;+ assert_eq!(+ status,+ StatusCode::FORBIDDEN,+ "read grant must not destroy the brain"+ );++ // Resource granularity within one brain: the notes and the link graph are+ // two indices, and `POST /_memory/{ns}/_recall` with `graph:` reads the+ // second. A credential given only the notes must not get the graph through+ // the side door — while plain recall over the notes it *was* granted keeps+ // working.+ let notes_only = mint(+ &app,+ &admin,+ r#"{"name":"notes-only","role_descriptors":{"n":{"indices":[+ {"names":[".xerj-memory-alice"],"privileges":["read"]}]}}}"#,+ )+ .await;+ let (status, _) = send(+ &app,+ "POST",+ "/_memory/alice/_recall",+ ¬es_only,+ r#"{"query":"anything"}"#,+ )+ .await;+ assert_eq!(status, StatusCode::OK, "plain recall on the granted notes");+ let (status, resp) = send(+ &app,+ "POST",+ "/_memory/alice/_recall",+ ¬es_only,+ r#"{"query":"anything","graph":{"mode":"restrict","seeds":["doc:1"]}}"#,+ )+ .await;+ assert_eq!(+ status,+ StatusCode::FORBIDDEN,+ "graph coupling reads the edges index and needs its own grant: {resp}"+ );+}++/// The fail-closed half. A key minted with no `role_descriptors` — the shape+/// every key had before this existed — holds **nothing** in the reserved+/// namespace, and cannot mint itself out of that.+#[tokio::test]+async fn an_unconfigured_key_is_denied_every_brain_and_cannot_escalate() {+ let (state, _data_dir) = auth_enabled_state();+ let app = build_es_compat_router(state);+ seed_two_brains(&app).await;++ let admin = format!("ApiKey {ADMIN_KEY}");+ let legacy = mint(&app, &admin, r#"{"name":"legacy"}"#).await;++ for (method, uri) in [+ ("GET", "/_graph/alice/ego?node=doc:1"),+ ("GET", "/_graph/bob/overview"),+ ("POST", "/_graph/alice/link"),+ ("POST", "/.xerj-memory-alice-edges/_search"),+ ("DELETE", "/.xerj-memory-bob-edges"),+ ("POST", "/_memory/alice/_recall"),+ ] {+ let (status, resp) = send(+ &app,+ method,+ uri,+ &legacy,+ r#"{"src":"a","type":"t","dst":"b","query":"x"}"#,+ )+ .await;+ assert_eq!(+ status,+ StatusCode::FORBIDDEN,+ "unconfigured key on {method} {uri} must be denied, got {status}: {resp}"+ );+ }++ // It keeps its historical reach over ordinary indices — broad RBAC over the+ // general surface is still deferred, and this fix does not pretend+ // otherwise.+ let (status, resp) = send(&app, "PUT", "/logs-2026", &legacy, "{}").await;+ assert!(+ status.is_success(),+ "a legacy key must still create an ordinary index, got {status}: {resp}"+ );+ let (status, resp) = send(+ &app,+ "POST",+ "/logs-2026/_doc/1",+ &legacy,+ r#"{"msg":"hello"}"#,+ )+ .await;+ assert!(+ status.is_success(),+ "a legacy key must still write an ordinary index, got {status}: {resp}"+ );++ // But enumeration hides the brains: `_mapping` answers, without them.+ let (status, mapping) = send(&app, "GET", "/_mapping", &legacy, "").await;+ assert_eq!(status, StatusCode::OK, "metadata still answers");+ let text = mapping.to_string();+ assert!(+ !text.contains(".xerj-memory-"),+ "brain names must not be enumerable: {text}"+ );+ assert!(+ text.contains("logs-2026"),+ "ordinary indices are still listed: {text}"+ );+ let (status, cat) = send_text(&app, "GET", "/_cat/indices", &legacy).await;+ assert_eq!(status, StatusCode::OK);+ assert!(+ !cat.contains(".xerj-memory-"),+ "_cat/indices must not enumerate brains: {cat}"+ );+ assert!(+ cat.contains("logs-2026"),+ "_cat/indices still lists ordinary indices: {cat}"+ );++ // And it cannot mint its way in: `role_descriptors` from a non-superuser+ // are dropped, so the child key is unscoped too.+ let child = mint(&app, &legacy, &brain_grant("bob", r#""all""#)).await;+ let (status, resp) = send(&app, "GET", "/_graph/bob/overview", &child, "").await;+ assert_eq!(+ status,+ StatusCode::FORBIDDEN,+ "a key minted by a non-superuser must not carry grants: {resp}"+ );+}++/// The local-dev path must not have moved. `xerj --insecure` /+/// point-at-a-folder has one user and no configuration; every brain stays+/// reachable with no credential at all.+#[tokio::test]+async fn insecure_single_user_mode_is_unchanged() {+ let dir = tempfile::tempdir().expect("tempdir");+ let state = state_over(dir.path().to_str().unwrap(), false);+ let app = build_es_compat_router(state);++ for brain in ["alice", "bob"] {+ let (status, body) = send(+ &app,+ "POST",+ &format!("/_graph/{brain}/link"),+ "",+ r#"{"src":"doc:1","type":"mentions","dst":"doc:2"}"#,+ )+ .await;+ assert_eq!(status, StatusCode::CREATED, "insecure link failed: {body}");+ let (status, _) = send(&app, "GET", &format!("/_graph/{brain}/overview"), "", "").await;+ assert_eq!(status, StatusCode::OK, "insecure overview must work");+ }+ // Including the doors a scoped key is refused: there is nothing to scope.+ let (status, _) = send(&app, "GET", "/_mapping", "", "").await;+ assert_eq!(status, StatusCode::OK, "insecure enumeration must work");+ let (status, cat) = send_text(&app, "GET", "/_cat/indices", "").await;+ assert_eq!(status, StatusCode::OK);+ assert!(+ cat.contains(".xerj-memory-"),+ "the single local user still sees their own brains: {cat}"+ );+ let (status, _) = send(+ &app,+ "POST",+ "/_search",+ "",+ r#"{"query":{"match_all":{}}}"#,+ )+ .await;+ assert_eq!(status, StatusCode::OK, "insecure fan-out must work");+}++/// Grants are persisted with the key. A node restart must not silently turn a+/// scoped key back into an unscoped one — nor, worse, into a superuser.+#[tokio::test]+async fn grants_survive_a_restart() {+ let dir = tempfile::tempdir().expect("tempdir");+ let data_dir = dir.path().to_str().unwrap().to_string();++ // Scope the first node so its data-dir lock is released before the second+ // one boots over the same directory.+ let alice = {+ let state = state_over(&data_dir, true);+ let app = build_es_compat_router(state.clone());+ seed_two_brains(&app).await;+ let admin = format!("ApiKey {ADMIN_KEY}");+ let alice = mint(&app, &admin, &brain_grant("alice", r#""read","write""#)).await;+ drop(app);+ drop(state);+ alice+ };++ // Boot a second engine over the same data directory — the restart.+ let restarted = build_es_compat_router(state_over(&data_dir, true));++ let (status, resp) = send(&restarted, "GET", "/_graph/alice/overview", &alice, "").await;+ assert_eq!(+ status,+ StatusCode::OK,+ "the grant must survive restart, got {status}: {resp}"+ );+ let (status, resp) = send(&restarted, "GET", "/_graph/bob/overview", &alice, "").await;+ assert_eq!(+ status,+ StatusCode::FORBIDDEN,+ "the boundary must survive restart, got {status}: {resp}"+ );+}diff --git a/engine/crates/xerj-api/tests/ml_datafeed_cannot_read_a_brain.rs b/engine/crates/xerj-api/tests/ml_datafeed_cannot_read_a_brain.rsnew file mode 100644index 000000000..1f2ae0e21--- /dev/null+++ b/engine/crates/xerj-api/tests/ml_datafeed_cannot_read_a_brain.rs@@ -0,0 +1,541 @@+//! The ML datafeed is not a way around the per-index boundary (issue #79).+//!+//! This is the exploit that got past the first two-layer cut, run end to end+//! against a real auth-enforced router, and it is deliberately written as the+//! attack rather than as a unit test of either half of the fix.+//!+//! ## What it was+//!+//! The engine-side guard is a `tokio` task-local, so it lives on the task that+//! is handling the request. `POST /_ml/datafeeds/{id}/_start` did two things:+//! one synchronous scoring pass, on the request's own task, and then a+//! detached `tokio::spawn` that re-scored every `frequency` seconds. The+//! synchronous pass was inside the guard and was correctly denied — which is+//! what made this so quiet. The caller saw an empty result set and could+//! reasonably conclude it had been stopped. A couple of seconds later the+//! background tick ran with no guard installed at all, read the brain, and+//! appended the field values it found to the job's results, where the same+//! caller read them straight back out of+//! `GET /_ml/anomaly_detectors/{job}/results/records`.+//!+//! The credential used is an **unscoped** key: minted with no+//! `role_descriptors`, the shape every key had before per-brain authorization+//! existed. The design guarantees it holds *nothing* on the reserved+//! namespace, and `a_direct_read_of_the_brain_is_still_denied` re-proves that+//! for this exact key inside every test below, so a passing run cannot be an+//! artifact of a credential that was allowed all along.+//!+//! ## What is asserted+//!+//! Both halves, separately, because neither alone closes it:+//!+//! - **config time** — an ML job or datafeed naming an index the caller cannot+//! read is refused, instead of being accepted with a 200;+//! - **run time** — even when the configuration was planted by the superuser,+//! so there was nothing to refuse, the detached scorer runs under the+//! *starting* principal's visibility rule and finds nothing.++use axum::body::Body;+use axum::http::{Request, StatusCode};+use http_body_util::BodyExt;+use serde_json::Value;+use tower::ServiceExt;+use xerj_api::{router::build_es_compat_router, state::AppState};+use xerj_common::{config::Config, metrics::Metrics};+use xerj_engine::Engine;++const ADMIN_KEY: &str = "admin-secret-key-for-ml-datafeed-test";+const BRAIN: &str = ".xerj-memory-bob-edges";++/// The value planted on bob's TOPSECRET edge. Distinctive enough that a+/// substring search over the whole response body is a sound leak test.+const SECRET: i64 = 987_654_321;++/// An auth-enabled node over a fresh data directory. The `TempDir` must be+/// held for the whole test — the Engine keeps the directory open.+fn auth_enabled_state() -> (AppState, tempfile::TempDir) {+ let dir = tempfile::tempdir().expect("tempdir");+ let mut config = Config::default();+ config.server.data_dir = dir.path().to_str().unwrap().to_string();+ config.auth.enabled = true;+ config.auth.admin_api_key = ADMIN_KEY.to_string();+ let metrics = Metrics::new().expect("metrics");+ let engine = Engine::new(config.clone()).expect("engine");+ (AppState::new(config, engine, metrics), dir)+}++async fn send(+ app: &axum::Router,+ method: &str,+ uri: &str,+ auth: &str,+ body: &str,+) -> (StatusCode, Value) {+ let req = Request::builder()+ .method(method)+ .uri(uri)+ .header("content-type", "application/json")+ .header("authorization", auth)+ .body(Body::from(body.to_string()))+ .expect("request");+ let resp = app.clone().oneshot(req).await.expect("response");+ let status = resp.status();+ let bytes = resp.into_body().collect().await.expect("body").to_bytes();+ (+ status,+ serde_json::from_slice(&bytes).unwrap_or(Value::Null),+ )+}++fn admin() -> String {+ format!("ApiKey {ADMIN_KEY}")+}++/// Mint a key with **no** `role_descriptors` — `Principal::Unscoped`.+async fn mint_unscoped(app: &axum::Router) -> String {+ let (status, resp) = send(+ app,+ "POST",+ "/_security/api_key",+ &admin(),+ r#"{"name":"leak-key"}"#,+ )+ .await;+ assert_eq!(status, StatusCode::OK, "minting: {resp}");+ format!("ApiKey {}", resp["encoded"].as_str().expect("encoded key"))+}++/// Plant bob's brain as the superuser: four ordinary edges, then the TOPSECRET+/// one. The detector needs four normal buckets of baseline before it will+/// score anything, so the fifth bucket is the anomaly and the record's+/// `actual` is the exact planted `valid_at`.+async fn plant_the_brain(app: &axum::Router) {+ for i in 1..=4 {+ let (status, body) = send(+ app,+ "PUT",+ &format!("/{BRAIN}/_doc/e{i}"),+ &admin(),+ &format!(+ r#"{{"from":"bob","to":"n{i}","type":"knows",+ "created_at":"2026-01-01T00:00:0{i}Z","valid_at":100{i}}}"#+ ),+ )+ .await;+ assert!(status.is_success(), "seeding e{i}: {body}");+ }+ let (status, body) = send(+ app,+ "PUT",+ &format!("/{BRAIN}/_doc/secret?refresh=true"),+ &admin(),+ &format!(+ r#"{{"from":"bob","to":"TOPSECRET","type":"knows",+ "created_at":"2026-01-01T00:00:09Z","valid_at":{SECRET}}}"#+ ),+ )+ .await;+ assert!(status.is_success(), "seeding the secret edge: {body}");+}++/// The detector and datafeed configs exactly as the reported exploit spells+/// them.+const DETECTOR_BODY: &str = r#"{"source_index":".xerj-memory-bob-edges","time_field":"created_at",+ "function":"max","field":"valid_at","bucket_span":"1s"}"#;+const FEED_BODY: &str = r#"{"job_id":"leak2","indices":[".xerj-memory-bob-edges"],+ "frequency":"1s"}"#;++/// The control every assertion in this file rests on: the key really is+/// denied the brain by the front door, so anything it learns by another route+/// is a bypass and not a grant.+async fn a_direct_read_of_the_brain_is_still_denied(app: &axum::Router, key: &str) {+ let (status, body) = send(+ app,+ "POST",+ &format!("/{BRAIN}/_search"),+ key,+ r#"{"query":{"match_all":{}}}"#,+ )+ .await;+ assert_eq!(+ status,+ StatusCode::FORBIDDEN,+ "the credential under test must be denied the brain directly: {body}"+ );+}++/// Read the job's anomaly records and assert the planted value is not in them,+/// whatever shape the response takes. Returns the body so a failure can print+/// what actually came back.+async fn records_must_stay_clean(app: &axum::Router, key: &str, when: &str) -> Value {+ let (status, body) = send(+ app,+ "GET",+ "/_ml/anomaly_detectors/leak2/results/records",+ key,+ "",+ )+ .await;+ let text = body.to_string();+ assert!(+ !text.contains(&SECRET.to_string()),+ "{when}: the reserved field value {SECRET} reached a credential that \+ cannot read {BRAIN} — HTTP {status}, body {text}"+ );+ assert_eq!(+ body["count"].as_i64().unwrap_or(0),+ 0,+ "{when}: the datafeed produced records off a brain it may not read: {text}"+ );+ body+}++/// Wait out several datafeed ticks. The feed's `frequency` is 1s and the tick+/// that leaked fired on the second one, so four seconds is three chances to+/// fail — real elapsed time, because the scorer runs on a real interval.+async fn let_the_background_scorer_tick() {+ tokio::time::sleep(std::time::Duration::from_millis(4_200)).await;+}++// ─────────────────────────────────────────────────────────────────────────────++/// FINDING A, both halves, as the reported exploit runs it.+///+/// Steps 1-3 are the config; step 4 is the `_start`; step 5 is the tell (empty+/// immediately, because the synchronous pass is inside the guard); step 6 is+/// where it used to leak.+#[tokio::test(flavor = "multi_thread", worker_threads = 4)]+async fn the_reported_ml_datafeed_exploit_end_to_end() {+ let (state, _dir) = auth_enabled_state();+ let app = build_es_compat_router(state);+ plant_the_brain(&app).await;+ let attacker = mint_unscoped(&app).await;+ a_direct_read_of_the_brain_is_still_denied(&app, &attacker).await;++ // 1. The superuser can score it — so the secret really is reachable by+ // this configuration, and an empty result later is the boundary+ // working rather than the fixture being wrong.+ let (status, body) = send(+ &app,+ "PUT",+ "/_ml/anomaly_detectors/control",+ &admin(),+ DETECTOR_BODY,+ )+ .await;+ assert_eq!(status, StatusCode::OK, "admin detector: {body}");+ let (status, scored) = send(+ &app,+ "POST",+ "/_ml/anomaly_detectors/control/_score",+ &admin(),+ "{}",+ )+ .await;+ assert_eq!(status, StatusCode::OK, "admin _score: {scored}");+ assert!(+ scored.to_string().contains(&SECRET.to_string()),+ "fixture check: the superuser must be able to score the planted value, \+ otherwise this test proves nothing. Got: {scored}"+ );++ // 2. PUT /_ml/anomaly_detectors/leak2 — refused at config time now.+ let (status, body) = send(+ &app,+ "PUT",+ "/_ml/anomaly_detectors/leak2",+ &attacker,+ DETECTOR_BODY,+ )+ .await;+ assert_eq!(+ status,+ StatusCode::FORBIDDEN,+ "an ML job may not name an index its creator cannot read: {body}"+ );++ // 3. PUT /_ml/datafeeds/leak2-feed — likewise.+ let (status, body) = send(+ &app,+ "PUT",+ "/_ml/datafeeds/leak2-feed",+ &attacker,+ FEED_BODY,+ )+ .await;+ assert_eq!(+ status,+ StatusCode::FORBIDDEN,+ "a datafeed may not name an index its creator cannot read: {body}"+ );++ // 4. …so there is nothing to start, and nothing to read.+ let (status, _) = send(+ &app,+ "POST",+ "/_ml/datafeeds/leak2-feed/_start",+ &attacker,+ "{}",+ )+ .await;+ assert_ne!(+ status,+ StatusCode::OK,+ "a datafeed that was never created must not start"+ );++ // 5 & 6. Empty immediately, and still empty after the ticks that used to+ // carry the value out.+ records_must_stay_clean(&app, &attacker, "immediately after _start").await;+ let_the_background_scorer_tick().await;+ records_must_stay_clean(&app, &attacker, "after the background ticks").await;+}++/// FINDING A, run-time half **in isolation**.+///+/// The config-time check cannot help here: the superuser plants the job and+/// the datafeed, so both already exist and name the brain legitimately. All+/// the attacker does is press `_start`. That is enough, because `_start` is+/// what spawns the detached scorer — and the scorer must therefore run under+/// the rule of whoever started it.+///+/// This is the assertion that fails if only the `BodyShape` half is applied.+#[tokio::test(flavor = "multi_thread", worker_threads = 4)]+async fn a_superuser_planted_datafeed_started_by_an_unscoped_key_reads_nothing() {+ let (state, _dir) = auth_enabled_state();+ let app = build_es_compat_router(state);+ plant_the_brain(&app).await;+ let attacker = mint_unscoped(&app).await;+ a_direct_read_of_the_brain_is_still_denied(&app, &attacker).await;++ // The superuser configures everything. Nothing here is refusable.+ let (status, body) = send(+ &app,+ "PUT",+ "/_ml/anomaly_detectors/leak2",+ &admin(),+ DETECTOR_BODY,+ )+ .await;+ assert_eq!(status, StatusCode::OK, "admin detector: {body}");+ let (status, body) = send(+ &app,+ "PUT",+ "/_ml/datafeeds/leak2-feed",+ &admin(),+ FEED_BODY,+ )+ .await;+ assert_eq!(status, StatusCode::OK, "admin datafeed: {body}");++ // The attacker only presses start.+ let (status, body) = send(+ &app,+ "POST",+ "/_ml/datafeeds/leak2-feed/_start",+ &attacker,+ "{}",+ )+ .await;+ assert_eq!(status, StatusCode::OK, "_start under the attacker: {body}");++ // The tell: empty right away, because the synchronous pass ran inside the+ // request's guarded scope. This was true before the fix too.+ records_must_stay_clean(&app, &attacker, "immediately after _start").await;++ // The bug: the detached tick. Before the fix this came back with+ // `actual: 987654321.0`.+ let_the_background_scorer_tick().await;+ records_must_stay_clean(&app, &attacker, "after the background ticks").await;++ // And the brain is still not readable by the front door afterwards, so+ // the empty results are the boundary and not a broken datafeed.+ a_direct_read_of_the_brain_is_still_denied(&app, &attacker).await;+}++/// The other side of the same change: a datafeed the **superuser** starts must+/// keep working exactly as before. A superuser has no visibility rule+/// installed, so `None` is what gets carried across the spawn, and the scorer+/// stays unrestricted.+#[tokio::test(flavor = "multi_thread", worker_threads = 4)]+async fn a_superuser_started_datafeed_still_scores() {+ let (state, _dir) = auth_enabled_state();+ let app = build_es_compat_router(state);+ plant_the_brain(&app).await;++ let (status, body) = send(+ &app,+ "PUT",+ "/_ml/anomaly_detectors/leak2",+ &admin(),+ DETECTOR_BODY,+ )+ .await;+ assert_eq!(status, StatusCode::OK, "detector: {body}");+ let (status, body) = send(+ &app,+ "PUT",+ "/_ml/datafeeds/leak2-feed",+ &admin(),+ FEED_BODY,+ )+ .await;+ assert_eq!(status, StatusCode::OK, "datafeed: {body}");+ let (status, body) = send(+ &app,+ "POST",+ "/_ml/datafeeds/leak2-feed/_start",+ &admin(),+ "{}",+ )+ .await;+ assert_eq!(status, StatusCode::OK, "_start: {body}");++ // The synchronous start pass alone should already have produced it.+ let (status, records) = send(+ &app,+ "GET",+ "/_ml/anomaly_detectors/leak2/results/records",+ &admin(),+ "",+ )+ .await;+ assert_eq!(status, StatusCode::OK);+ assert!(+ records.to_string().contains(&SECRET.to_string()),+ "the superuser's own datafeed must still score its index: {records}"+ );++ // …and the background ticks must not break it either.+ let_the_background_scorer_tick().await;+ let (_, records) = send(+ &app,+ "GET",+ "/_ml/anomaly_detectors/leak2/results/records",+ &admin(),+ "",+ )+ .await;+ assert!(+ records.to_string().contains(&SECRET.to_string()),+ "the superuser's datafeed stopped scoring after a background tick: {records}"+ );+}++/// A scoped key configuring ML over an index it **holds** must not be caught+/// by the new config-time check — the point is to bound the datafeed to its+/// creator, not to refuse everyone.+///+/// It also pins what this change deliberately did **not** touch. The ML+/// *verbs* (`_score`, `_start`, `_stop`) name no index of their own, so they+/// fall to the pre-existing cluster rule that a scoped key must name what it+/// mutates, and are refused — as they were before this change. That is a+/// usability gap in the scoped-key surface, not a hole: it fails closed, and+/// widening it is a change to the cluster arm rather than to anything issue+/// #79 is about. Pinned here so the difference is visible and deliberate.+#[tokio::test(flavor = "multi_thread", worker_threads = 4)]+async fn a_scoped_key_still_configures_ml_over_its_own_index() {+ let (state, _dir) = auth_enabled_state();+ let app = build_es_compat_router(state);+ plant_the_brain(&app).await;++ // Same fixture shape, under a name the scoped key will be granted.+ for (i, (ts, val)) in [+ ("2026-01-01T00:00:01Z", 1001),+ ("2026-01-01T00:00:02Z", 1002),+ ("2026-01-01T00:00:03Z", 1003),+ ("2026-01-01T00:00:04Z", 1004),+ ("2026-01-01T00:00:09Z", SECRET),+ ]+ .iter()+ .enumerate()+ {+ let refresh = if i == 4 { "?refresh=true" } else { "" };+ let (status, body) = send(+ &app,+ "PUT",+ &format!("/logs-2026/_doc/d{i}{refresh}"),+ &admin(),+ &format!(r#"{{"created_at":"{ts}","valid_at":{val}}}"#),+ )+ .await;+ assert!(status.is_success(), "seeding logs-2026 d{i}: {body}");+ }++ let (status, minted) = send(+ &app,+ "POST",+ "/_security/api_key",+ &admin(),+ r#"{"name":"logs-agent","role_descriptors":{"logs":{"indices":[+ {"names":["logs-2026"],"privileges":["read","write"]}]}}}"#,+ )+ .await;+ assert_eq!(status, StatusCode::OK, "minting a scoped key: {minted}");+ let agent = format!("ApiKey {}", minted["encoded"].as_str().expect("encoded"));++ let (status, body) = send(+ &app,+ "PUT",+ "/_ml/anomaly_detectors/mine",+ &agent,+ r#"{"source_index":"logs-2026","time_field":"created_at",+ "function":"max","field":"valid_at","bucket_span":"1s"}"#,+ )+ .await;+ assert_eq!(+ status,+ StatusCode::OK,+ "a scoped key must still configure ML over an index it holds: {body}"+ );+ // …but it still cannot point one at someone else's brain.+ let (status, body) = send(+ &app,+ "PUT",+ "/_ml/anomaly_detectors/theirs",+ &agent,+ DETECTOR_BODY,+ )+ .await;+ assert_eq!(+ status,+ StatusCode::FORBIDDEN,+ "cross-brain ML config: {body}"+ );++ // The verbs stay where the pre-existing cluster rule leaves them: refused+ // for a scoped key, because they name nothing. Unchanged by this commit.+ for (method, uri) in [+ ("POST", "/_ml/anomaly_detectors/mine/_score"),+ ("POST", "/_ml/datafeeds/mine-feed/_start"),+ ] {+ let (status, body) = send(&app, method, uri, &agent, "{}").await;+ assert_eq!(+ status,+ StatusCode::FORBIDDEN,+ "{method} {uri} is expected to stay refused for a scoped key \+ (pre-existing cluster-mutation rule, fails closed): {body}"+ );+ }++ // The superuser can still drive the very same job, so the detector the+ // scoped key configured is real and usable — the gap is the verb, not the+ // config.+ let (status, scored) = send(+ &app,+ "POST",+ "/_ml/anomaly_detectors/mine/_score",+ &admin(),+ "{}",+ )+ .await;+ assert_eq!(+ status,+ StatusCode::OK,+ "admin _score of the scoped job: {scored}"+ );+ assert!(+ scored.to_string().contains(&SECRET.to_string()),+ "the job the scoped key configured must score its own index: {scored}"+ );+}diff --git a/engine/crates/xerj-api/tests/scoped_keys_get_intact_responses.rs b/engine/crates/xerj-api/tests/scoped_keys_get_intact_responses.rsnew file mode 100644index 000000000..01e74338d--- /dev/null+++ b/engine/crates/xerj-api/tests/scoped_keys_get_intact_responses.rs@@ -0,0 +1,407 @@+//! A scoped key gets **structurally intact** metadata responses (issue #79).+//!+//! Enumeration pruning removes index names a caller may not read. The first+//! cut did it by deleting any object key equal to a known index name, at every+//! depth of the response. An index name is an arbitrary string, so ordinary+//! names collide with the structural keys of the very responses being pruned —+//! and only for a scoped key, i.e. exactly the multi-tenant and Kibana case the+//! branch exists to serve. A superuser skips pruning entirely and never saw it.+//!+//! Every index created here has a perfectly ordinary name that happens to be a+//! structural key somewhere: `status`, `indices`, `type`, `nodes`, `count`.+//! None of them is granted to the scoped key, so all of them are legitimately+//! prunable *as index names* — which is what makes them the sharp test. The+//! responses must keep their shape anyway.+//!+//! The opposite failure is equally a failure, so every test also re-asserts+//! that bob's brain is still absent from the same response. "Prune nothing"+//! would pass the structural half and lose the boundary.++use axum::body::Body;+use axum::http::{Request, StatusCode};+use http_body_util::BodyExt;+use serde_json::Value;+use tower::ServiceExt;+use xerj_api::{router::build_es_compat_router, state::AppState};+use xerj_common::{config::Config, metrics::Metrics};+use xerj_engine::Engine;++const ADMIN_KEY: &str = "admin-secret-key-for-pruning-test";++/// Ordinary index names that are also structural keys in the metadata+/// responses a client polls. Each one broke a different endpoint.+const COLLIDING: &[&str] = &["status", "indices", "type", "nodes", "count"];++fn auth_enabled_state() -> (AppState, tempfile::TempDir) {+ let dir = tempfile::tempdir().expect("tempdir");+ let mut config = Config::default();+ config.server.data_dir = dir.path().to_str().unwrap().to_string();+ config.auth.enabled = true;+ config.auth.admin_api_key = ADMIN_KEY.to_string();+ let metrics = Metrics::new().expect("metrics");+ let engine = Engine::new(config.clone()).expect("engine");+ (AppState::new(config, engine, metrics), dir)+}++fn admin() -> String {+ format!("ApiKey {ADMIN_KEY}")+}++async fn send(+ app: &axum::Router,+ method: &str,+ uri: &str,+ auth: &str,+ body: &str,+) -> (StatusCode, Value) {+ let req = Request::builder()+ .method(method)+ .uri(uri)+ .header("content-type", "application/json")+ .header("authorization", auth)+ .body(Body::from(body.to_string()))+ .expect("request");+ let resp = app.clone().oneshot(req).await.expect("response");+ let status = resp.status();+ let bytes = resp.into_body().collect().await.expect("body").to_bytes();+ (+ status,+ serde_json::from_slice(&bytes).unwrap_or(Value::Null),+ )+}++async fn send_text(app: &axum::Router, uri: &str, auth: &str) -> (StatusCode, String) {+ let req = Request::builder()+ .method("GET")+ .uri(uri)+ .header("authorization", auth)+ .body(Body::empty())+ .expect("request");+ let resp = app.clone().oneshot(req).await.expect("response");+ let status = resp.status();+ let bytes = resp.into_body().collect().await.expect("body").to_bytes();+ (status, String::from_utf8_lossy(&bytes).into_owned())+}++/// A node holding: the scoped key's own index, one brain it must never learn+/// of, and one ordinary index per colliding structural key.+async fn seed(app: &axum::Router) -> String {+ let mut names: Vec<String> = vec!["logs-2026".into(), ".xerj-memory-bob-edges".into()];+ names.extend(COLLIDING.iter().map(|s| s.to_string()));+ for name in &names {+ let (status, body) = send(+ app,+ "PUT",+ &format!("/{name}/_doc/1?refresh=true"),+ &admin(),+ r#"{"host":"h1","amount":3}"#,+ )+ .await;+ assert!(status.is_success(), "seeding {name}: {body}");+ }+ let (status, minted) = send(+ app,+ "POST",+ "/_security/api_key",+ &admin(),+ r#"{"name":"logs-agent","role_descriptors":{"logs":{"indices":[+ {"names":["logs-2026"],"privileges":["read","write"]}]}}}"#,+ )+ .await;+ assert_eq!(status, StatusCode::OK, "minting: {minted}");+ format!("ApiKey {}", minted["encoded"].as_str().expect("encoded"))+}++/// The boundary half, asserted alongside every structural assertion so+/// "prune nothing" cannot pass.+fn must_not_leak_the_brain(what: &str, body: &str) {+ assert!(+ !body.contains("bob"),+ "{what} enumerated a brain the caller may not read: {body}"+ );+}++// ─────────────────────────────────────────────────────────────────────────────++/// `GET /_cluster/health` keeps `status` — the single most-polled field in the+/// API — with an ordinary index called `status` on the node.+#[tokio::test]+async fn cluster_health_keeps_its_status_field() {+ let (state, _dir) = auth_enabled_state();+ let app = build_es_compat_router(state);+ let agent = seed(&app).await;++ let (status, body) = send(&app, "GET", "/_cluster/health", &agent, "").await;+ assert_eq!(status, StatusCode::OK, "{body}");+ assert!(+ body.get("status").is_some(),+ "_cluster/health lost its `status` field: {body}"+ );+ assert!(body.get("number_of_nodes").is_some(), "{body}");+ assert!(body.get("active_shards").is_some(), "{body}");+ must_not_leak_the_brain("_cluster/health", &body.to_string());++ // With the per-index breakdown, the breakdown itself is still filtered:+ // the caller's own index is there and nothing else is.+ let (status, body) = send(&app, "GET", "/_cluster/health?level=indices", &agent, "").await;+ assert_eq!(status, StatusCode::OK, "{body}");+ assert!(body.get("status").is_some(), "level=indices lost `status`");+ let indices = body["indices"].as_object().expect("indices map");+ assert!(+ indices.contains_key("logs-2026"),+ "own index missing: {body}"+ );+ for name in COLLIDING {+ assert!(+ !indices.contains_key(*name),+ "unreadable index `{name}` is still enumerated: {body}"+ );+ }+ must_not_leak_the_brain("_cluster/health?level=indices", &body.to_string());+}++/// `GET /_cluster/stats` keeps its whole `indices` section with an ordinary+/// index called `indices` on the node. That section is aggregate counters, not+/// a map keyed by index name, so there is nothing there to prune.+#[tokio::test]+async fn cluster_stats_keeps_its_indices_section() {+ let (state, _dir) = auth_enabled_state();+ let app = build_es_compat_router(state);+ let agent = seed(&app).await;++ let (status, body) = send(&app, "GET", "/_cluster/stats", &agent, "").await;+ assert_eq!(status, StatusCode::OK, "{body}");+ assert!(+ body.get("indices").is_some(),+ "_cluster/stats lost its entire `indices` section: {body}"+ );+ assert!(+ body["indices"].get("count").is_some(),+ "_cluster/stats lost `indices.count`: {body}"+ );+ assert!(+ body["indices"].get("docs").is_some(),+ "_cluster/stats lost `indices.docs`: {body}"+ );+ assert!(body.get("status").is_some(), "lost `status`: {body}");+ assert!(+ body["nodes"].get("count").is_some(),+ "_cluster/stats lost `nodes.count`: {body}"+ );+ must_not_leak_the_brain("_cluster/stats", &body.to_string());+}++/// Global `GET /_mapping` keeps every field's `type` with an ordinary index+/// called `type` on the node. This is the endpoint Kibana reads to build an+/// index pattern; without `type` every field is undefined.+#[tokio::test]+async fn global_mapping_keeps_every_field_type() {+ let (state, _dir) = auth_enabled_state();+ let app = build_es_compat_router(state);+ let agent = seed(&app).await;++ let (status, body) = send(&app, "GET", "/_mapping", &agent, "").await;+ assert_eq!(status, StatusCode::OK, "{body}");++ let own = body+ .get("logs-2026")+ .unwrap_or_else(|| panic!("the caller's own index is missing from _mapping: {body}"));+ let props = own["mappings"]["properties"]+ .as_object()+ .unwrap_or_else(|| panic!("_mapping lost `mappings.properties`: {body}"));+ assert!(!props.is_empty(), "_mapping returned no fields: {body}");+ for (field, spec) in props {+ assert!(+ spec.get("type").is_some(),+ "field `{field}` lost its `type` — Kibana cannot build a pattern \+ from this: {body}"+ );+ }++ // …and the enumeration boundary still holds.+ for name in COLLIDING {+ assert!(+ body.get(*name).is_none(),+ "unreadable index `{name}` is still enumerated by _mapping: {body}"+ );+ }+ must_not_leak_the_brain("_mapping", &body.to_string());+}++/// `_cat/indices` keeps every row whose index column the caller can read, and+/// drops exactly the rows it cannot — not every row that happens to contain a+/// colliding token.+#[tokio::test]+async fn cat_indices_keeps_the_rows_it_should() {+ let (state, _dir) = auth_enabled_state();+ let app = build_es_compat_router(state);+ let agent = seed(&app).await;++ let (status, table) = send_text(&app, "/_cat/indices", &agent).await;+ assert_eq!(status, StatusCode::OK, "{table}");+ assert!(+ table.contains("logs-2026"),+ "_cat/indices lost the caller's own index: {table}"+ );+ assert_eq!(+ table.lines().filter(|l| !l.trim().is_empty()).count(),+ 1,+ "_cat/indices should list exactly the one readable index: {table}"+ );+ must_not_leak_the_brain("_cat/indices", &table);++ // `_cat/health` has no index column at all, so it must survive whole even+ // though `status` and `count` are unreadable index names on this node.+ let (status, health) = send_text(&app, "/_cat/health", &agent).await;+ assert_eq!(status, StatusCode::OK);+ assert!(+ !health.trim().is_empty(),+ "_cat/health was emptied by index-name matching: {health:?}"+ );+}++/// `GET /_settings` and `GET /_alias` are the same top-level index-keyed shape+/// and must behave the same way.+#[tokio::test]+async fn settings_and_alias_listings_stay_intact() {+ let (state, _dir) = auth_enabled_state();+ let app = build_es_compat_router(state);+ let agent = seed(&app).await;++ for uri in ["/_settings", "/_alias"] {+ let (status, body) = send(&app, "GET", uri, &agent, "").await;+ assert_eq!(status, StatusCode::OK, "GET {uri}: {body}");+ assert!(+ body.get("logs-2026").is_some(),+ "GET {uri} lost the caller's own index: {body}"+ );+ for name in COLLIDING {+ assert!(+ body.get(*name).is_none(),+ "GET {uri} still enumerates `{name}`: {body}"+ );+ }+ must_not_leak_the_brain(uri, &body.to_string());+ }++ // `_settings` keeps the settings object under the index it did return.+ let (_, body) = send(&app, "GET", "/_settings", &agent, "").await;+ assert!(+ body["logs-2026"].get("settings").is_some(),+ "_settings lost its `settings` block: {body}"+ );+}++/// `_field_caps` names indices in two places — the top-level list and one per+/// field/type entry — and neither may take the `fields` map down with it.+#[tokio::test]+async fn field_caps_keeps_its_fields_map() {+ let (state, _dir) = auth_enabled_state();+ let app = build_es_compat_router(state);+ let agent = seed(&app).await;++ let (status, body) = send(&app, "GET", "/_field_caps?fields=*", &agent, "").await;+ assert_eq!(status, StatusCode::OK, "{body}");+ assert!(+ body.get("fields").is_some(),+ "_field_caps lost its `fields` map: {body}"+ );+ let listed = body["indices"].as_array().expect("indices array");+ assert!(+ listed.iter().any(|v| v == "logs-2026"),+ "_field_caps lost the caller's own index: {body}"+ );+ for name in COLLIDING {+ assert!(+ !listed.iter().any(|v| v == name),+ "_field_caps still enumerates `{name}`: {body}"+ );+ }+ must_not_leak_the_brain("_field_caps", &body.to_string());+}++/// FINDING D. An alias `add` whose index is a wildcard that matches nothing+/// used to answer `acknowledged: true` and leave an alias pointing at the+/// pattern TEXT — a write that never happened, reported as one that did, on+/// the reserved namespace of all places.+#[tokio::test]+async fn a_wildcard_alias_that_matches_nothing_is_refused() {+ let (state, _dir) = auth_enabled_state();+ let app = build_es_compat_router(state);+ seed(&app).await;++ // `POST /_aliases` with an `add` naming a reserved wildcard.+ let (status, body) = send(+ &app,+ "POST",+ "/_aliases",+ &admin(),+ r#"{"actions":[{"add":{"index":".xerj-memory-nope-*","alias":"X"}}]}"#,+ )+ .await;+ assert_ne!(+ status,+ StatusCode::OK,+ "a wildcard alias add that matched nothing was acknowledged: {body}"+ );++ // `PUT /{wildcard}/_alias/{name}`, the other spelling.+ let (status, body) = send(&app, "PUT", "/.xerj-memory-nope-*/_alias/Y", &admin(), "{}").await;+ assert_ne!(+ status,+ StatusCode::OK,+ "PUT /{{wildcard}}/_alias was acknowledged: {body}"+ );++ // Neither junk alias exists.+ let (_, aliases) = send(&app, "GET", "/_aliases", &admin(), "").await;+ let text = aliases.to_string();+ assert!(!text.contains("\"X\""), "junk alias X was created: {text}");+ assert!(!text.contains("\"Y\""), "junk alias Y was created: {text}");++ // A wildcard that DOES match still works, on both spellings — the fix is+ // "refuse a write that did not happen", not "refuse wildcards".+ let (status, body) = send(+ &app,+ "POST",+ "/_aliases",+ &admin(),+ r#"{"actions":[{"add":{"index":"logs-*","alias":"live"}}]}"#,+ )+ .await;+ assert_eq!(status, StatusCode::OK, "a matching wildcard add: {body}");+ let (status, body) = send(&app, "PUT", "/logs-*/_alias/live2", &admin(), "{}").await;+ assert_eq!(status, StatusCode::OK, "a matching wildcard PUT: {body}");++ // Both resolve to the real index, not to the pattern text.+ let (_, resolved) = send(&app, "GET", "/live/_search", &admin(), "{}").await;+ assert!(+ resolved.get("hits").is_some(),+ "the alias does not resolve to a real index: {resolved}"+ );+ let (_, aliases) = send(&app, "GET", "/_aliases", &admin(), "").await;+ assert!(+ aliases["logs-2026"]["aliases"].get("live").is_some(),+ "the alias was not attached to the resolved index: {aliases}"+ );+ assert!(+ aliases["logs-2026"]["aliases"].get("live2").is_some(),+ "PUT /{{pattern}}/_alias did not attach to the resolved index: {aliases}"+ );++ // A concrete name is unchanged: ES lets an alias be attached before the+ // index exists, and clients rely on that ordering.+ let (status, body) = send(+ &app,+ "POST",+ "/_aliases",+ &admin(),+ r#"{"actions":[{"add":{"index":"not-created-yet","alias":"early"}}]}"#,+ )+ .await;+ assert_eq!(+ status,+ StatusCode::OK,+ "a concrete-name alias add must still be accepted: {body}"+ );+}diff --git a/engine/crates/xerj-api/tests/script_limits_http.rs b/engine/crates/xerj-api/tests/script_limits_http.rsindex 2b93f6319..a94ca60a7 100644--- a/engine/crates/xerj-api/tests/script_limits_http.rs+++ b/engine/crates/xerj-api/tests/script_limits_http.rs@@ -17,6 +17,69 @@ use axum::http::{Request, StatusCode};use serde_json::{json, Value};use tower::ServiceExt;+/// Stack size for every thread these tests evaluate a script on, replacing the+/// 2 MiB `std`/`libtest` default that `#[tokio::test]` would have given them.+///+/// `xerj_engine::painless::MAX_CALL_DEPTH` is a **stack** budget: 32 is+/// documented as measured in a *release* build, where the adversarial worst+/// case costs 1.20 MiB of a 2 MiB worker stack and the rest is left for the+/// axum/search frames underneath. A debug build's frames are several times+/// larger, so that same recursion lands much closer to the edge here than the+/// constant's own arithmetic implies — and `_delete_by_query` is the deepest+/// of these paths, since it runs a search inside the by-query wrapper.+///+/// Adding the issue-#79 authorization layer to the router pushed the debug+/// profile of that one path over 2 MiB. The cost is the *layer*, not the+/// authorization: these tests configure no admin key, so the caller is a+/// superuser and `authz_middleware` returns on its second statement. Measured+/// on this branch by bisecting `RUST_MIN_STACK`,+/// `delete_by_query_refuses_a_selection_a_script_limit_truncated` aborts at+/// 2,097,152 bytes and passes at 2,359,296 — roughly 200 KiB of debug frames+/// for one more `tower` layer. The same test passes on `main` at the default,+/// so the margin it had there was under 200 KiB.+///+/// The release binary was checked directly rather than inferred: the same+/// recursion with 10, 50 and 90 nested blocks in the closure body (90 being+/// the documented worst case) answers `400 script_exception` on a running+/// release node, which stays up. So the production margin still holds and the+/// budget in `painless.rs` is unchanged.+///+/// Sizing the stack here keeps these tests measuring what they are for — that+/// the depth guard trips and the refusal reaches the caller — instead of+/// measuring how large `rustc -C opt-level=0` makes an async frame.+const WORKER_STACK_BYTES: usize = 4 * 1024 * 1024;++/// Drive `fut` to completion with every stack involved sized to+/// [`WORKER_STACK_BYTES`].+///+/// Both threads have to be sized, because the two shapes here evaluate the+/// script in different places: `_search` hands the work to a task+/// (`search_impl` spawns it) so it lands on a **runtime worker**, while+/// `_delete_by_query` with the default `wait_for_completion=true` awaits+/// `run_delete_by_query` inline, so it lands on the thread that called+/// `block_on` — which under `libtest` is the per-test thread, and that one+/// takes its size from `RUST_MIN_STACK` (2 MiB by default), not from the+/// runtime builder.+fn block_on_sized<F>(fut: F)+where+ F: std::future::Future<Output = ()> + Send + 'static,+{+ std::thread::Builder::new()+ .stack_size(WORKER_STACK_BYTES)+ .spawn(move || {+ tokio::runtime::Builder::new_multi_thread()+ .worker_threads(4)+ .thread_stack_size(WORKER_STACK_BYTES)+ .enable_all()+ .build()+ .expect("test runtime")+ .block_on(fut)+ })+ .expect("spawn test thread")+ .join()+ .expect("test thread panicked");+}+/// Self-application recursion that runs past `MAX_CALL_DEPTH` (32), with the/// recursive call wrapped in nested blocks — the shape from the PR #88/// process-abort repro. Short enough to clear the static source-size and@@ -75,8 +138,12 @@ async fn post(app: &axum::Router, path: &str, body: Value) -> (StatusCode, Value/// The headline of #97. `apply_function_score` returns a bare `f32`, so it/// mapped the call-depth error to `0.0` and the request succeeded with a score/// that no script produced.-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]-async fn script_score_past_the_call_depth_limit_is_an_error_not_a_zero_score() {+#[test]+fn script_score_past_the_call_depth_limit_is_an_error_not_a_zero_score() {+ block_on_sized(script_score_past_the_call_depth_limit_is_an_error_not_a_zero_score_inner());+}++async fn script_score_past_the_call_depth_limit_is_an_error_not_a_zero_score_inner() {let (app, _dir) = seeded_app().await;let (status, body) = post(&app,@@ -115,8 +182,12 @@ async fn script_score_past_the_call_depth_limit_is_an_error_not_a_zero_score() {/// `if let Ok(pv)` dropped a limit trip on the floor and the field just wasn't/// in the response — indistinguishable from a script that legitimately/// returned nothing.-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]-async fn script_field_past_the_call_depth_limit_is_an_error_not_a_missing_field() {+#[test]+fn script_field_past_the_call_depth_limit_is_an_error_not_a_missing_field() {+ block_on_sized(script_field_past_the_call_depth_limit_is_an_error_not_a_missing_field_inner());+}++async fn script_field_past_the_call_depth_limit_is_an_error_not_a_missing_field_inner() {let (app, _dir) = seeded_app().await;let (status, body) = post(&app,@@ -141,8 +212,14 @@ async fn script_field_past_the_call_depth_limit_is_an_error_not_a_missing_field(/// A script-bucketed terms agg mapped a limit trip to "no buckets" — an empty/// aggregation that reads as a legitimate answer.-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]-async fn script_bucketed_agg_past_the_call_depth_limit_is_an_error_not_empty_buckets() {+#[test]+fn script_bucketed_agg_past_the_call_depth_limit_is_an_error_not_empty_buckets() {+ block_on_sized(+ script_bucketed_agg_past_the_call_depth_limit_is_an_error_not_empty_buckets_inner(),+ );+}++async fn script_bucketed_agg_past_the_call_depth_limit_is_an_error_not_empty_buckets_inner() {let (app, _dir) = seeded_app().await;let (status, body) = post(&app,@@ -167,8 +244,12 @@ async fn script_bucketed_agg_past_the_call_depth_limit_is_an_error_not_empty_buc/// `_delete_by_query` selects with the same matching machinery. A fail-closed/// script trip there means deleting an arbitrary subset and reporting a/// completed run — destructive as well as wrong.-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]-async fn delete_by_query_refuses_a_selection_a_script_limit_truncated() {+#[test]+fn delete_by_query_refuses_a_selection_a_script_limit_truncated() {+ block_on_sized(delete_by_query_refuses_a_selection_a_script_limit_truncated_inner());+}++async fn delete_by_query_refuses_a_selection_a_script_limit_truncated_inner() {let (app, _dir) = seeded_app().await;let (_, body) = post(&app,@@ -201,8 +282,12 @@ async fn delete_by_query_refuses_a_selection_a_script_limit_truncated() {/// support is NOT a resource limit. It must keep degrading to a neutral score/// on a 200, exactly as before, or every out-of-subset script in the ES-compat/// surface becomes a spurious 400.-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]-async fn unsupported_script_syntax_still_degrades_quietly() {+#[test]+fn unsupported_script_syntax_still_degrades_quietly() {+ block_on_sized(unsupported_script_syntax_still_degrades_quietly_inner());+}++async fn unsupported_script_syntax_still_degrades_quietly_inner() {let (app, _dir) = seeded_app().await;for source in ["someUnsupportedThing(1,2,3)",@@ -234,8 +319,12 @@ async fn unsupported_script_syntax_still_degrades_quietly() {/// A search that trips a limit must not leave the response cache poisoned with/// its degraded scores, and the fault must not be re-reported against the next/// caller of a different query.-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]-async fn a_faulted_search_is_neither_cached_nor_replayed() {+#[test]+fn a_faulted_search_is_neither_cached_nor_replayed() {+ block_on_sized(a_faulted_search_is_neither_cached_nor_replayed_inner());+}++async fn a_faulted_search_is_neither_cached_nor_replayed_inner() {let (app, _dir) = seeded_app().await;let bad = json!({"query": {diff --git a/engine/crates/xerj-api/tests/script_limits_remaining_surfaces_http.rs b/engine/crates/xerj-api/tests/script_limits_remaining_surfaces_http.rsindex 796dbad13..5b9e2ab9a 100644--- a/engine/crates/xerj-api/tests/script_limits_remaining_surfaces_http.rs+++ b/engine/crates/xerj-api/tests/script_limits_remaining_surfaces_http.rs@@ -25,6 +25,40 @@ use axum::http::{Request, StatusCode};use serde_json::{json, Value};use tower::ServiceExt;+/// Stack size for every thread these tests evaluate a script on, replacing the+/// 2 MiB `std`/`libtest` default. Same reasoning, and the same measurements, as+/// `tests/script_limits_http.rs` — see the comment on its `WORKER_STACK_BYTES`.+/// In short: `MAX_CALL_DEPTH` is a stack budget measured in a *release* build,+/// a debug build's frames are several times larger, and the issue-#79+/// authorization layer spends what little margin a debug build had left.+/// `delete_by_query_script_refusal_carries_its_own_http_status` is the one that+/// aborted here; the release binary answers `400 script_exception` and stays+/// up, including at the adversarial worst case, so the budget itself is sound.+const WORKER_STACK_BYTES: usize = 4 * 1024 * 1024;++/// Drive `fut` with both the runtime workers and the calling thread sized to+/// [`WORKER_STACK_BYTES`]: `_search`-shaped surfaces evaluate the script on a+/// spawned task, `_delete_by_query` awaits it inline on the caller.+fn block_on_sized<F>(fut: F)+where+ F: std::future::Future<Output = ()> + Send + 'static,+{+ std::thread::Builder::new()+ .stack_size(WORKER_STACK_BYTES)+ .spawn(move || {+ tokio::runtime::Builder::new_multi_thread()+ .worker_threads(4)+ .thread_stack_size(WORKER_STACK_BYTES)+ .enable_all()+ .build()+ .expect("test runtime")+ .block_on(fut)+ })+ .expect("spawn test thread")+ .join()+ .expect("test thread panicked");+}+/// Self-application recursion that runs past `MAX_CALL_DEPTH` (32), with the/// recursive call wrapped in nested blocks. Short enough to clear the static/// source-size and parse-depth guards, so the limit can only trip@@ -112,8 +146,12 @@ async fn send(/// call-depth limit degrades every hit to 0.0 — and then `_rank_eval`/// publishes a `metric_score` computed over that degraded ranking as a/// successful measurement.-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]-async fn rank_eval_past_the_call_depth_limit_is_an_error_not_a_metric_score() {+#[test]+fn rank_eval_past_the_call_depth_limit_is_an_error_not_a_metric_score() {+ block_on_sized(rank_eval_past_the_call_depth_limit_is_an_error_not_a_metric_score_inner());+}++async fn rank_eval_past_the_call_depth_limit_is_an_error_not_a_metric_score_inner() {let (app, _dir) = seeded_app().await;let (status, body) = post(&app,@@ -169,8 +207,12 @@ async fn rank_eval_past_the_call_depth_limit_is_an_error_not_a_metric_score() {/// A body where only ONE of several requests carries a limit-tripping script/// must still measure the others. Failing all of them was the first attempt.-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]-async fn rank_eval_failure_is_scoped_to_the_offending_request() {+#[test]+fn rank_eval_failure_is_scoped_to_the_offending_request() {+ block_on_sized(rank_eval_failure_is_scoped_to_the_offending_request_inner());+}++async fn rank_eval_failure_is_scoped_to_the_offending_request_inner() {let (app, _dir) = seeded_app().await;let (status, body) = post(&app,@@ -211,8 +253,12 @@ async fn rank_eval_failure_is_scoped_to_the_offending_request() {/// A limit trip in the *matching* path truncates the ranking instead of/// skewing it: fewer retrieved ids, so precision/recall are computed over a/// selection the script silently cut short.-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]-async fn rank_eval_refuses_a_ranking_a_script_limit_truncated() {+#[test]+fn rank_eval_refuses_a_ranking_a_script_limit_truncated() {+ block_on_sized(rank_eval_refuses_a_ranking_a_script_limit_truncated_inner());+}++async fn rank_eval_refuses_a_ranking_a_script_limit_truncated_inner() {let (app, _dir) = seeded_app().await;let (status, body) = post(&app,@@ -243,8 +289,12 @@ async fn rank_eval_refuses_a_ranking_a_script_limit_truncated() {/// running the query with an `ids` filter. A fail-closed script makes the/// answer `matched: false`, which is indistinguishable from a document that/// genuinely does not match.-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]-async fn explain_past_the_call_depth_limit_is_an_error_not_an_unmatched_doc() {+#[test]+fn explain_past_the_call_depth_limit_is_an_error_not_an_unmatched_doc() {+ block_on_sized(explain_past_the_call_depth_limit_is_an_error_not_an_unmatched_doc_inner());+}++async fn explain_past_the_call_depth_limit_is_an_error_not_an_unmatched_doc_inner() {let (app, _dir) = seeded_app().await;let (status, body) = post(&app,@@ -287,8 +337,12 @@ async fn explain_past_the_call_depth_limit_is_an_error_not_an_unmatched_doc() {/// The second defect in #123. `run_delete_by_query` refuses correctly, but the/// handler wrapped the refusal in `Json(..)` alone — HTTP 200 carrying/// `{"error": …, "status": 400}`.-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]-async fn delete_by_query_script_refusal_carries_its_own_http_status() {+#[test]+fn delete_by_query_script_refusal_carries_its_own_http_status() {+ block_on_sized(delete_by_query_script_refusal_carries_its_own_http_status_inner());+}++async fn delete_by_query_script_refusal_carries_its_own_http_status_inner() {let (app, _dir) = seeded_app().await;let (status, body) = post(&app,@@ -316,8 +370,12 @@ async fn delete_by_query_script_refusal_carries_its_own_http_status() {}/// Same defect on `_update_by_query`.-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]-async fn update_by_query_script_refusal_carries_its_own_http_status() {+#[test]+fn update_by_query_script_refusal_carries_its_own_http_status() {+ block_on_sized(update_by_query_script_refusal_carries_its_own_http_status_inner());+}++async fn update_by_query_script_refusal_carries_its_own_http_status_inner() {let (app, _dir) = seeded_app().await;let (status, body) = post(&app,@@ -340,8 +398,12 @@ async fn update_by_query_script_refusal_carries_its_own_http_status() {/// A successful `_delete_by_query` must stay a 200 — the status now comes from/// the body, so this pins the non-error half of that mapping.-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]-async fn delete_by_query_without_a_script_is_still_a_200() {+#[test]+fn delete_by_query_without_a_script_is_still_a_200() {+ block_on_sized(delete_by_query_without_a_script_is_still_a_200_inner());+}++async fn delete_by_query_without_a_script_is_still_a_200_inner() {let (app, _dir) = seeded_app().await;let (status, body) = post(&app,@@ -360,8 +422,12 @@ async fn delete_by_query_without_a_script_is_still_a_200() {/// wrong summary and reports `acknowledged: true` over it — the same defect as/// an under-delete, except the wrong numbers are now persisted in an index/// that later queries will read as fact.-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]-async fn pivot_transform_refuses_a_source_a_script_limit_truncated() {+#[test]+fn pivot_transform_refuses_a_source_a_script_limit_truncated() {+ block_on_sized(pivot_transform_refuses_a_source_a_script_limit_truncated_inner());+}++async fn pivot_transform_refuses_a_source_a_script_limit_truncated_inner() {let (app, _dir) = seeded_app().await;let (put_status, put_body) = send(@@ -399,8 +465,12 @@ async fn pivot_transform_refuses_a_source_a_script_limit_truncated() {/// The counterpart, restated for the new surfaces: a script our interpreter/// does not support is NOT a resource limit and must keep degrading quietly.-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]-async fn unsupported_script_syntax_still_degrades_quietly_on_the_new_surfaces() {+#[test]+fn unsupported_script_syntax_still_degrades_quietly_on_the_new_surfaces() {+ block_on_sized(unsupported_script_syntax_still_degrades_quietly_on_the_new_surfaces_inner());+}++async fn unsupported_script_syntax_still_degrades_quietly_on_the_new_surfaces_inner() {let (app, _dir) = seeded_app().await;let unsupported = json!({"function_score": {diff --git a/engine/crates/xerj-engine/src/engine.rs b/engine/crates/xerj-engine/src/engine.rsindex 23f43d27e..39eccddc9 100644--- a/engine/crates/xerj-engine/src/engine.rs+++ b/engine/crates/xerj-engine/src/engine.rs@@ -173,6 +173,22 @@ pub struct ApiKeyRecord {pub expiration_ms: Option<u64>,/// Set once the key has been invalidated (revoked).pub invalidated: bool,+ /// Index-scoped grants parsed from the `role_descriptors` supplied at+ /// creation (`crate::rbac::roles_from_role_descriptors`).+ ///+ /// **Empty means "no grant", not "all grants."** A key with no roles is an+ /// *unscoped* legacy credential: it keeps the historical superuser reach+ /// over the general ES-compat surface, but it holds no privilege at all on+ /// the reserved `.xerj-memory-*` namespace (agent-memory namespaces and+ /// second brains), which is the fail-closed half of the per-brain+ /// authorization model. A key with a non-empty list is *scoped* and may+ /// touch only what these roles allow.+ ///+ /// `#[serde(default)]` so an `api_keys.json` written before this field+ /// existed still loads — those keys deserialize as unscoped, which is the+ /// safe reading of "was minted when nothing was enforced".+ #[serde(default)]+ pub roles: Vec<crate::rbac::Role>,}/// Atomically write a **secret** file (API-key store) with owner-only (0600)@@ -762,6 +778,17 @@ impl Engine {) -> Result<()> {let index_name = IndexName::new(name).map_err(EngineError::Common)?;+ // Per-index authorization backstop (issue #79): a caller that may not+ // see this name must not be able to bring it into existence either —+ // otherwise it could squat a brain name it cannot read. Reported as a+ // create failure rather than a distinct refusal so the two are+ // indistinguishable. See `crate::index_guard`.+ if !crate::index_guard::visible(name) {+ return Err(EngineError::Common(+ xerj_common::XerjError::index_not_found(name),+ ));+ }+if self.indices.contains_key(name) {return Err(EngineError::Common(xerj_common::XerjError::index_already_exists(name),@@ -1077,6 +1104,14 @@ impl Engine {/// semantics) and clears the `closed_indices` flag so the name is/// truly gone when another test recreates it.pub async fn delete_index(&self, name: &str) -> Result<()> {+ // Per-index authorization backstop (issue #79) — "destroy the brain"+ // is the loudest door, so it is checked before the index is removed+ // from the map. See `crate::index_guard`.+ if !crate::index_guard::visible(name) {+ return Err(EngineError::Common(+ xerj_common::XerjError::index_not_found(name),+ ));+ }let idx =self.indices.remove(name).map(|(_, v)| v).ok_or_else(|| {EngineError::Common(xerj_common::XerjError::index_not_found(name))@@ -1114,10 +1149,24 @@ impl Engine {/// Get a reference to an index by name, resolving aliases first./// If the name is an alias pointing to multiple indices, returns the first one.+ ///+ /// This is the single funnel every handler goes through to reach index+ /// data, whether it read the name from the URL path, from a `_bulk` action+ /// line, from an `_mget` `docs[]._index`, from a `terms` lookup or from the+ /// table name inside an `_sql` statement — which is exactly why the+ /// per-index authorization backstop is here (issue #79, see+ /// [`crate::index_guard`]). The check runs **after** alias resolution, on+ /// the concrete name, so pointing an alias at someone else's index cannot+ /// launder access to it.pub fn get_index(&self, name: &str) -> Result<Arc<Index>> {// Check if name is an alias — if so, resolve to the first backing index.if let Some(aliased) = self.aliases.get(name) {if let Some(real_name) = aliased.first() {+ if !crate::index_guard::visible(real_name) {+ return Err(EngineError::Common(+ xerj_common::XerjError::index_not_found(name),+ ));+ }return self.indices.get(real_name.as_str())@@ -1127,6 +1176,11 @@ impl Engine {});}}+ if !crate::index_guard::visible(name) {+ return Err(EngineError::Common(+ xerj_common::XerjError::index_not_found(name),+ ));+ }self.indices.get(name).map(|r| Arc::clone(r.value()))@@ -1134,7 +1188,19 @@ impl Engine {}/// Return an index by name, creating it if it doesn't exist (ES behaviour).+ ///+ /// The visibility check is repeated here rather than left to+ /// [`Engine::get_index`]: that call reports "not found" for a denied name,+ /// and "not found" is precisely the branch this function answers by+ /// **creating** the index. Without its own check, auto-create would be the+ /// bypass (`create_index` catches it too — belt and braces on the write+ /// door).pub fn get_or_create_index(&self, name: &str) -> Result<Arc<Index>> {+ if !crate::index_guard::visible(name) {+ return Err(EngineError::Common(+ xerj_common::XerjError::index_not_found(name),+ ));+ }if let Ok(idx) = self.get_index(name) {return Ok(idx);}@@ -1176,8 +1242,17 @@ impl Engine {/// Used by PIT expansion and other handlers that need to iterate/// the live index list without paying for the `list_indices`/// snapshot or `get_settings()` call.+ /// Filtered to the current request's visible set (issue #79) — this and+ /// [`Engine::list_indices`] are where `*`, `_all` and `logs-*` are turned+ /// into concrete names, so filtering here is what makes a wildcard resolve+ /// to "the indices you may see" instead of having to be refused outright.+ /// Unfiltered outside a request; see [`crate::index_guard`].pub fn index_name_list(&self) -> Vec<String> {- self.indices.iter().map(|e| e.key().clone()).collect()+ self.indices+ .iter()+ .map(|e| e.key().clone())+ .filter(|n| crate::index_guard::visible(n))+ .collect()}/// Sum the internal query-result cache hit/miss counters across every open@@ -1194,9 +1269,18 @@ impl Engine {(hits, misses)}+ /// Filtered to the current request's visible set — see+ /// [`Engine::index_name_list`] and [`crate::index_guard`]. Every metadata+ /// listing (`_cat/indices`, `_mapping`, `_alias`, `_resolve/index`,+ /// `/v1/dashboard/summary`, cluster health) enumerates through here, so a+ /// scoped credential gets a filtered view of the cluster rather than a+ /// blanket refusal.pub async fn list_indices(&self) -> Vec<IndexInfo> {let mut list = Vec::new();for entry in self.indices.iter() {+ if !crate::index_guard::visible(entry.key()) {+ continue;+ }let stats = entry.value().stats().await;list.push(IndexInfo {name: stats.name,diff --git a/engine/crates/xerj-engine/src/index_guard.rs b/engine/crates/xerj-engine/src/index_guard.rsnew file mode 100644index 000000000..53c3b7888--- /dev/null+++ b/engine/crates/xerj-engine/src/index_guard.rs@@ -0,0 +1,281 @@+//! Request-scoped index **visibility** — the engine-side half of the per-index+//! authorization boundary (issue #79).+//!+//! ## Why this lives in the engine and not only in the API middleware+//!+//! The first cut of per-brain authorization decided against the index named in+//! the **URL path**. Several handlers take the index they actually operate on+//! from the **request body** — `_msearch` header lines, `_bulk` action lines,+//! `_mget` `docs[]._index`, `_aliases` actions, `_reindex` `source`/`dest`, a+//! `terms` lookup buried in a query, the table name inside `_sql` — so a+//! caller could name one index in the path, be authorized against it, and have+//! the handler touch a different one. Four of those were proven live.+//!+//! Patching each handler is how that class of bug comes back: the next handler+//! that resolves a name from a body is written by someone who does not know the+//! list. So the check is placed where **every** path converges instead — the+//! two engine functions that turn a name into an [`crate::index::Index`]+//! ([`crate::Engine::get_index`] and [`crate::Engine::get_or_create_index`]),+//! the two that create and destroy one, and the two that enumerate them.+//! Nothing in the process reaches index data without going through one of+//! those, so a handler cannot forget this check — it does not call it.+//!+//! ## Shape+//!+//! The API layer installs an [`IndexVisibility`] for the duration of a request+//! (a `tokio` task-local, set by `xerj_api::authz::authz_middleware` around the+//! inner service call, so every future awaited while handling that request sees+//! it). Code running **outside** a request — startup index discovery, the+//! background flush/merge timers, WAL replay, snapshot restore — has no+//! task-local set and is unrestricted, which is what+//! [`visible`]'s `unwrap_or(true)` means: absent guard = engine-internal work,+//! not "permission granted to a caller".+//!+//! ## Detached work loses the guard, and that was a real hole+//!+//! A `tokio` task-local lives on the task, so anything a request *detaches*+//! from itself — `tokio::spawn`, `spawn_blocking`, a `rayon` closure that+//! lands on a pool worker — runs with no guard installed and is therefore+//! unrestricted.+//!+//! The first cut of this module recorded that shape as a residual risk and+//! claimed "no request path in the tree does that today". That was wrong. The+//! ML datafeed does: `POST /_ml/datafeeds/{id}/_start` spawned a detached+//! scorer that re-read its source index every `frequency` seconds, so a+//! principal with no access to `.xerj-memory-bob-edges` got an empty result+//! set from the synchronous start pass (correctly denied, inside the request's+//! scope) and the brain's field values a few seconds later, from the tick.+//!+//! So detaching now has an explicit contract instead of a footnote:+//!+//! - [`current`] captures the guard the request is running under.+//! - [`scoped`] re-installs it inside the spawned future.+//!+//! Any future `tokio::spawn` that can reach [`crate::Engine::get_index`] must+//! carry the rule across with those two, and any that cannot must say why in a+//! comment at the spawn site. `spawn_datafeed_task` in `xerj-api` is the+//! worked example.+//!+//! ### The audit+//!+//! Every non-test `tokio::spawn` / `spawn_blocking` in the workspace, and why+//! it is or is not a door. Only the first row was a hole.+//!+//! | Site | Verdict |+//! |---|---|+//! | `xerj-api::es_compat::spawn_datafeed_task` | **carries the rule** — the one detached path that resolves an index name |+//! | `es_compat::search_impl`'s search task | owns one already-authorized `Index`; `terms`/`lookup` targets are resolved earlier, on the request's task |+//! | `es_compat::delete_by_query` / `update_by_query` (`wait_for_completion=false`) | same: one already-authorized `Index` handle, no `Engine` |+//! | `Engine::spawn_pit_sweeper`, `spawn_search_context_sweeper` | started by `Engine::new`, capture only the PIT/scroll maps, touch no index |+//! | `Index`'s flush/merge/warm tasks (`index.rs`, `write_publication.rs`) | methods **on** an already-resolved `Index`; the name has already been through the funnel |+//! | `xerj-server::main`'s listeners, metrics loop, autoindex/brain CLIs | startup, outside any request; unrestricted by design (see `visible`) |+//! | `xerj-cluster` transport/replication/coordinator | peer-to-peer, authorized by `xerj_cluster::auth`, not by a caller's principal |+//! | `xerj-ai::embedder`, `xerj-storage` cache/backend/merge | no `Engine`, no index names |+//! | `xerj-api::binary_protocol::serve` | not wired to any listener in `xerj-server`; module is unreachable at runtime |+//! | `xerj-server::grpc` | its spawns are `#[cfg(test)]`; the live listener authorizes per call with `Principal::allows_index` |+//!+//! `rayon` does **not** inherit the task-local — the first cut claimed it did.+//! `visible` reads a thread-local that `tokio` maintains only while it polls+//! the owning task, and a rayon worker is an ordinary OS thread that `tokio`+//! never touches, so a closure that lands on one sees an absent guard.+//! `absent_guard_on_a_rayon_worker` below pins that. It is not a live hole+//! because no `rayon` closure in the tree resolves an index name: the bulk+//! ingest fan-out (`bulk::process_bulk_with_opts`) parallelises NDJSON+//! *parsing* only, and every `get_or_create_index` in that function runs+//! sequentially on the request's own task, after the parse collects.+//!+//! ## Denial shape: "not found", not "forbidden"+//!+//! A denied name reports the ordinary+//! [`xerj_common::XerjError::index_not_found`], for two reasons:+//!+//! 1. **No enumeration.** A caller cannot tell a brain it may not read from a+//! brain that does not exist, so it cannot map the node by probing.+//! 2. **Fan-out keeps working.** `POST /_search`, `_cat/indices`, `_mapping`,+//! a `logs-*` wildcard — all of them enumerate and skip what is missing.+//! Filtering the enumeration ([`crate::Engine::list_indices`],+//! [`crate::Engine::index_name_list`]) plus a not-found on the rest gives+//! "you see exactly your own indices" for free, instead of the blanket 403+//! that made global verbs unusable.+//!+//! The precise privilege (read vs write vs manage) and the ES-shaped 403 are+//! still the API middleware's job. This is the backstop that makes+//! *forgetting* impossible, not a replacement for the front-line decision.++use std::sync::Arc;++/// Decides whether the principal behind the current request may see an index+/// at all. Implemented by `xerj_api::authz` over the request's `Principal`.+pub trait IndexVisibility: Send + Sync {+ /// May the current principal touch the concrete index `index`?+ ///+ /// `index` is always a real index name — aliases are resolved before this+ /// is consulted, so an alias cannot launder access to its target.+ fn visible(&self, index: &str) -> bool;+}++tokio::task_local! {+ static VISIBILITY: Arc<dyn IndexVisibility>;+}++/// Is `index` visible to the principal behind the current request?+///+/// `true` when no guard is installed — see the module docs: that means the+/// caller is the engine itself, not an unauthenticated request.+pub fn visible(index: &str) -> bool {+ VISIBILITY.try_with(|v| v.visible(index)).unwrap_or(true)+}++/// Run `fut` with `guard` installed as the current request's visibility rule.+pub async fn scoped<F>(guard: Arc<dyn IndexVisibility>, fut: F) -> F::Output+where+ F: std::future::Future,+{+ VISIBILITY.scope(guard, fut).await+}++/// The guard the caller is currently running under, if any.+///+/// The half of the detached-work contract that runs *before* the spawn: call+/// it on the request's own task, move the result into the spawned future, and+/// hand it back to [`scoped`] there. `None` means the caller is already+/// unrestricted (engine-internal work, or a superuser request, for which the+/// API layer installs no guard at all), and the spawned task inherits exactly+/// that — an unrestricted caller does not become restricted by detaching, and+/// a restricted one does not become unrestricted.+pub fn current() -> Option<Arc<dyn IndexVisibility>> {+ VISIBILITY.try_with(Arc::clone).ok()+}++/// Run `fut` under `guard` when there is one, unrestricted when there is not.+///+/// The spawn-site half of the contract, so a caller does not have to spell the+/// `match` out (and cannot get it backwards):+///+/// ```ignore+/// let rule = index_guard::current(); // on the request's task+/// tokio::spawn(async move {+/// index_guard::scoped_opt(rule, async move { /* … */ }).await+/// });+/// ```+pub async fn scoped_opt<F>(guard: Option<Arc<dyn IndexVisibility>>, fut: F) -> F::Output+where+ F: std::future::Future,+{+ match guard {+ Some(g) => scoped(g, fut).await,+ None => fut.await,+ }+}++/// Is a guard installed at all — i.e. is this a request rather than+/// engine-internal work? Distinguishes the two cases that [`visible`]+/// deliberately collapses to `true`; nothing in the decision path needs it,+/// but a caller reasoning about which side of the boundary it is on does.+pub fn guarded() -> bool {+ VISIBILITY.try_with(|_| true).unwrap_or(false)+}++#[cfg(test)]+mod tests {+ use super::*;++ struct Only(&'static str);+ impl IndexVisibility for Only {+ fn visible(&self, index: &str) -> bool {+ index == self.0+ }+ }++ #[tokio::test]+ async fn absent_guard_is_unrestricted() {+ assert!(visible("anything"));+ assert!(!guarded());+ }++ /// The contract detached work runs under: capture on the request's task,+ /// re-install inside the spawned one. Without the capture the spawned task+ /// is unrestricted, which is the ML-datafeed hole.+ #[tokio::test]+ async fn a_captured_guard_survives_tokio_spawn() {+ let (uncarried, carried) = scoped(Arc::new(Only("mine")), async {+ // Detaching without carrying the rule loses it — this is the shape+ // the bug had.+ let uncarried = tokio::spawn(async { (guarded(), visible("yours")) })+ .await+ .expect("join");+ // Carrying it across keeps the denial.+ let rule = current();+ let carried = tokio::spawn(async move {+ scoped_opt(rule, async {+ (guarded(), visible("yours"), visible("mine"))+ })+ .await+ })+ .await+ .expect("join");+ (uncarried, carried)+ })+ .await;+ assert_eq!(uncarried, (false, true), "a bare spawn loses the guard");+ assert_eq!(+ carried,+ (true, false, true),+ "a carried guard denies in the spawned task exactly as it did in the request"+ );+ }++ /// Outside a request there is nothing to carry, and carrying `None` must+ /// not invent a restriction.+ #[tokio::test]+ async fn carrying_an_absent_guard_stays_unrestricted() {+ assert!(current().is_none());+ let seen = tokio::spawn(scoped_opt(None, async { (guarded(), visible("anything")) }))+ .await+ .expect("join");+ assert_eq!(seen, (false, true));+ }++ /// The first cut of this module claimed `rayon` inherits the task-local.+ /// It does not: a rayon worker is an OS thread `tokio` never polls a task+ /// on, so the thread-local backing `VISIBILITY` is simply unset there.+ ///+ /// `ThreadPool::spawn` (rather than `install`/`join`/`par_iter`) is used+ /// deliberately — it *always* runs the closure on a pool worker, never on+ /// the calling thread, so this cannot pass by accidentally staying home.+ #[tokio::test]+ async fn absent_guard_on_a_rayon_worker() {+ let pool = rayon::ThreadPoolBuilder::new()+ .num_threads(1)+ .build()+ .expect("rayon pool");+ let (tx, rx) = std::sync::mpsc::channel();+ scoped(Arc::new(Only("mine")), async {+ assert!(!visible("yours"), "denied on the request's own thread");+ pool.spawn(move || {+ let _ = tx.send((guarded(), visible("yours")));+ });+ })+ .await;+ assert_eq!(+ rx.recv().expect("rayon result"),+ (false, true),+ "a rayon worker sees no guard — so no rayon closure may resolve an index name"+ );+ }++ #[tokio::test]+ async fn guard_applies_inside_the_scope_only() {+ scoped(Arc::new(Only("mine")), async {+ assert!(guarded());+ assert!(visible("mine"));+ assert!(!visible("yours"));+ // A nested await keeps the guard.+ tokio::task::yield_now().await;+ assert!(!visible("yours"));+ })+ .await;+ // Outside the scope it is gone again.+ assert!(visible("yours"));+ }+}diff --git a/engine/crates/xerj-engine/src/lib.rs b/engine/crates/xerj-engine/src/lib.rsindex be0b21f85..83b1a44e6 100644--- a/engine/crates/xerj-engine/src/lib.rs+++ b/engine/crates/xerj-engine/src/lib.rs@@ -22,6 +22,7 @@ pub mod bulk;pub mod engine;pub mod governor;pub mod index;+pub mod index_guard;pub mod ingest_memory;pub mod memtable;pub mod painless;diff --git a/engine/crates/xerj-engine/src/rbac.rs b/engine/crates/xerj-engine/src/rbac.rsindex 76eaa2f20..2954904c0 100644--- a/engine/crates/xerj-engine/src/rbac.rs+++ b/engine/crates/xerj-engine/src/rbac.rs@@ -1,15 +1,19 @@-//! Role-based access control — data model only. **NOT ENFORCED.**+//! Role-based access control — data model, plus the per-key grants that the+//! reserved-namespace authorization in `xerj-api::authz` enforces.//!-//! ⚠️ HONEST-SURFACE WARNING (RC4 item 6): the types below store roles, but-//! **nothing in the auth path consults them.** `is_authorized`-//! (`xerj-api::auth`) is binary — a request either carries the admin key / a-//! valid minted API key (→ full superuser access) or it is rejected. A role-//! `PUT` via `/_security/role/{name}` is recorded and can be read back, but it-//! grants and restricts **nothing**; every authenticated caller is superuser.-//! The `/_security/role*` handlers therefore stamp every response with-//! `"enforced": false` so an operator cannot mistake this for a working-//! authorization system. Full RBAC enforcement (per-request privilege checks,-//! per-key `role_descriptors`, FLS/DLS) is DEFERRED.+//! ⚠️ HONEST-SURFACE WARNING, NARROWED (was RC4 item 6): the named+//! [`RoleStore`] below — the thing `PUT /_security/role/{name}` writes — is+//! still **data only**. Nothing in the auth path consults it, which is why the+//! `/_security/role*` handlers stamp every response with `"enforced": false`.+//! Broad RBAC over the general ES-compat surface (FLS/DLS, cluster privileges,+//! named-role binding) remains DEFERRED.+//!+//! What *is* enforced: the [`Role`] values attached to a **minted API key**+//! (`ApiKeyRecord::roles`, parsed from `role_descriptors` by+//! [`roles_from_role_descriptors`]). Those gate the reserved `.xerj-memory-*`+//! namespace — agent-memory namespaces and second brains — and confine a+//! scoped key to the indices it names. See `xerj-api::authz` for the decision+//! function and the enumeration of enforcement points.//!//! What ships today://! - `Privilege` enum covering the seven core ops (read / write / admin@@ -17,9 +21,11 @@//! - `Role` — name + privileges + index-pattern allow list.//! - `RoleStore` — in-memory map of roles, default seeded with//! `admin`, `write`, `read`, `read_only_index`, `snapshot_admin`, `auditor`.+//! - `roles_from_role_descriptors` — ES `role_descriptors` → `Vec<Role>`.use parking_lot::RwLock;use serde::{Deserialize, Serialize};+use serde_json::Value;use std::collections::{HashMap, HashSet};use std::sync::Arc;@@ -42,7 +48,7 @@ pub enum Privilege {AuditRead,}-#[derive(Clone, Debug, Serialize, Deserialize)]+#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]pub struct Role {pub name: String,pub privileges: HashSet<Privilege>,@@ -66,6 +72,30 @@ impl Role {/// Does this role apply to the named index? Glob: "*" matches/// everything; literal names must match exactly; suffix-`*` (e.g./// `logs-*`) matches by prefix.+ ///+ /// # `names: ["*"]` includes the reserved namespace, deliberately+ ///+ /// "Everything" means everything: a role granted `*` matches+ /// `.xerj-memory-alice-edges` like any other index, so a key minted with+ /// it can read and write every second brain and every agent-memory+ /// namespace on the node. `xerj_api::authz::may_reach_reserved` is *not*+ /// consulted on this path — it decides whether a name a caller is about to+ /// **create** (an alias, a fresh index, a template pattern) is squatting+ /// the reserved prefix, not whether an existing grant covers a read.+ ///+ /// This is an administrator's explicit choice rather than an escalation:+ /// the only principal that can mint a scoped key at all is the superuser+ /// (`POST /_security/api_key`), and `*` is the plainest possible way to+ /// write "this key gets the whole node". Narrowing it silently would break+ /// the operator who wrote `*` and meant it — a Kibana service account, a+ /// backup runner — with no error to explain the missing data.+ ///+ /// **Operators wanting brain isolation must not hand out `names: ["*"]`.**+ /// Grant the concrete indices, or a prefix that excludes the reserved+ /// namespace (`logs-*`, `app-*`); `.xerj-memory-*` is reachable only by a+ /// grant that reaches it. `a_star_grant_reaches_the_reserved_namespace` in+ /// `xerj_api::authz`'s tests pins this behaviour so it cannot be changed+ /// by accident.pub fn applies_to(&self, idx: &str) -> bool {for pat in &self.indices {if pat == "*" || pat == idx {@@ -85,6 +115,93 @@ impl Role {}}+/// Map one Elasticsearch index-privilege name onto this crate's [`Privilege`]s.+///+/// Unknown names map to **nothing** — an unrecognized privilege grants no+/// access rather than being waved through, so a typo (or an ES privilege xerj+/// has not implemented) fails closed.+fn es_index_privilege(name: &str) -> &'static [Privilege] {+ use Privilege::*;+ match name {+ "all" => &[ReadIndex, WriteIndex, AdminIndex],+ "read" | "read_cross_cluster" | "view_index_metadata" | "monitor" => &[ReadIndex],+ "write" | "index" | "create" | "create_doc" | "delete" | "maintenance" => &[WriteIndex],+ "manage" | "create_index" | "delete_index" | "manage_ilm" | "manage_follow_index" => {+ &[AdminIndex]+ }+ _ => &[],+ }+}++/// Parse an Elasticsearch-shaped `role_descriptors` object — exactly what+/// `POST /_security/api_key` already accepts on the wire — into the roles this+/// crate enforces:+///+/// ```json+/// { "alice-brain": { "indices": [+/// { "names": [".xerj-memory-alice-edges", ".xerj-memory-alice"],+/// "privileges": ["read", "write"] } ] } }+/// ```+///+/// One [`Role`] is emitted per `indices[]` entry (named `"{descriptor}[{i}]"`),+/// because a `Role` carries a single privilege set while one descriptor may+/// list several entries with different privileges.+///+/// Deliberate omissions, all of which fail closed (they grant nothing):+/// - `cluster` privileges are ignored — xerj has no cluster-privilege+/// enforcement, so honouring them would be theatre.+/// - `field_security` / `query` (FLS/DLS) are ignored; a descriptor that only+/// makes sense with FLS/DLS therefore over-grants at the field level within+/// an index it was already granted. Callers that need FLS must not rely on+/// this.+/// - Only a trailing `*` globs (see [`Role::applies_to`]); a mid-pattern `*`+/// matches nothing.+pub fn roles_from_role_descriptors(descriptors: &Value) -> Vec<Role> {+ let Some(map) = descriptors.as_object() else {+ return Vec::new();+ };+ let mut roles = Vec::new();+ for (descriptor_name, descriptor) in map {+ let Some(entries) = descriptor.get("indices").and_then(|v| v.as_array()) else {+ continue;+ };+ for (i, entry) in entries.iter().enumerate() {+ let names: Vec<String> = entry+ .get("names")+ .and_then(|v| v.as_array())+ .map(|a| {+ a.iter()+ .filter_map(|v| v.as_str())+ .filter(|s| !s.is_empty())+ .map(String::from)+ .collect()+ })+ .unwrap_or_default();+ let privileges: HashSet<Privilege> = entry+ .get("privileges")+ .and_then(|v| v.as_array())+ .map(|a| {+ a.iter()+ .filter_map(|v| v.as_str())+ .flat_map(|p| es_index_privilege(p).iter().copied())+ .collect()+ })+ .unwrap_or_default();+ if names.is_empty() || privileges.is_empty() {+ // Nothing to grant. Skipping keeps `roles.is_empty()` an+ // accurate "this key was given no usable grant" signal.+ continue;+ }+ roles.push(Role::new(+ format!("{descriptor_name}[{i}]"),+ privileges,+ names,+ ));+ }+ }+ roles+}+pub struct RoleStore {roles: RwLock<HashMap<String, Role>>,}@@ -201,4 +318,67 @@ mod tests {assert!(!r.allows("metrics-prod", Privilege::ReadIndex));assert!(!r.allows("logs-prod", Privilege::WriteIndex));}++ #[test]+ fn role_descriptors_parse_into_index_grants() {+ let roles = roles_from_role_descriptors(&serde_json::json!({+ "alice-brain": {+ "cluster": ["all"],+ "indices": [+ { "names": [".xerj-memory-alice-edges"], "privileges": ["read", "write"] },+ { "names": [".xerj-memory-alice"], "privileges": ["read"] }+ ]+ }+ }));+ assert_eq!(roles.len(), 2, "one Role per indices[] entry");+ assert!(roles+ .iter()+ .any(|r| r.allows(".xerj-memory-alice-edges", Privilege::WriteIndex)));+ assert!(roles+ .iter()+ .any(|r| r.allows(".xerj-memory-alice", Privilege::ReadIndex)));+ // The read-only entry does not silently grant writes, and nothing+ // grants anything on a brain that was never named.+ assert!(!roles+ .iter()+ .any(|r| r.allows(".xerj-memory-alice", Privilege::WriteIndex)));+ assert!(!roles+ .iter()+ .any(|r| r.allows(".xerj-memory-bob-edges", Privilege::ReadIndex)));+ // `cluster` privileges are ignored, so "all" there granted no index.+ assert!(!roles+ .iter()+ .any(|r| r.allows("anything", Privilege::ReadIndex)));+ }++ #[test]+ fn unusable_role_descriptors_grant_nothing() {+ // Not an object, no `indices`, empty names, empty/unknown privileges —+ // every one of these must produce zero roles, so `roles.is_empty()`+ // stays a truthful "no grant" signal for the fail-closed path.+ for d in [+ serde_json::json!("all"),+ serde_json::json!({ "r": { "cluster": ["all"] } }),+ serde_json::json!({ "r": { "indices": [{ "names": [], "privileges": ["all"] }] } }),+ serde_json::json!({ "r": { "indices": [{ "names": ["x"], "privileges": [] }] } }),+ serde_json::json!({ "r": { "indices": [{ "names": ["x"], "privileges": ["bogus"] }] } }),+ ] {+ assert!(+ roles_from_role_descriptors(&d).is_empty(),+ "must grant nothing: {d}"+ );+ }+ }++ #[test]+ fn all_privilege_covers_read_write_admin() {+ let roles = roles_from_role_descriptors(&serde_json::json!({+ "r": { "indices": [{ "names": ["logs-*"], "privileges": ["all"] }] }+ }));+ let r = &roles[0];+ assert!(r.allows("logs-prod", Privilege::ReadIndex));+ assert!(r.allows("logs-prod", Privilege::WriteIndex));+ assert!(r.allows("logs-prod", Privilege::AdminIndex));+ assert!(!r.allows("metrics", Privilege::ReadIndex));+ }}diff --git a/engine/crates/xerj-server/src/grpc.rs b/engine/crates/xerj-server/src/grpc.rsindex fe36e247d..cd68c6629 100644--- a/engine/crates/xerj-server/src/grpc.rs+++ b/engine/crates/xerj-server/src/grpc.rs@@ -16,7 +16,9 @@ use anyhow::Context;use tonic::{Request, Response, Status, Streaming};use tracing::info;+use xerj_api::auth::Principal;use xerj_api::AppState;+use xerj_engine::rbac::Privilege;use xerj_query::parse_request;/// Generated prost messages + tonic client/server stubs for `xerj.v1`.@@ -43,13 +45,59 @@ impl GrpcService {}}+/// Authorize an RPC that names an index.+///+/// The listener speaks the same engine as the HTTP surfaces, so the reserved+/// `.xerj-memory-*` namespace has to be gated here too (issue #79) — otherwise+/// per-brain authorization would hold on :9200 and :8080 and fall open on+/// :8081, which is worse than not having it. [`GrpcAuth`] plants the+/// [`Principal`] in the request extensions; a missing one means the+/// interceptor did not run, and is treated as no credential at all.+///+/// `clippy::result_large_err` fires because `tonic::Status` is a fat value, but+/// `Err` here IS the RPC's denial status — the shape every other handler in+/// this file already returns — so boxing it would add an allocation on the deny+/// path and force an unwrap at every `?`.+#[allow(clippy::result_large_err)]+fn authorize(+ extensions: &tonic::Extensions,+ index: &str,+ privilege: Privilege,+) -> Result<(), Status> {+ let principal = extensions.get::<Principal>().unwrap_or(&Principal::Denied);+ if principal.allows_index(index, privilege) {+ Ok(())+ } else {+ Err(Status::permission_denied(format!(+ "this credential is not authorized for [{index}]"+ )))+ }+}++/// Same rule as the HTTP middleware: a pattern that could reach the reserved+/// namespace is refused rather than expanded, and a scoped credential must+/// name the index it was granted.+#[allow(clippy::result_large_err)] // same rationale as `authorize`+fn reject_wildcards(principal: &Principal, index: &str) -> Result<(), Status> {+ if (index.contains('*') || index == "_all") && !principal.is_superuser() {+ return Err(Status::permission_denied(format!(+ "index patterns are not available to this credential: [{index}]"+ )));+ }+ Ok(())+}+#[tonic::async_trait]impl XerjSearch for GrpcService {async fn search(&self,request: Request<pb::SearchRequest>,) -> Result<Response<pb::SearchResponse>, Status> {- let req = request.into_inner();+ let (_meta, extensions, req) = request.into_parts();+ let principal = extensions.get::<Principal>().unwrap_or(&Principal::Denied);+ reject_wildcards(principal, &req.index)?;+ // Before `get_index`, so 403-vs-not-found cannot map the node's brains.+ authorize(&extensions, &req.index, Privilege::ReadIndex)?;let idx = self.state@@ -111,7 +159,8 @@ impl XerjSearch for GrpcService {&self,request: Request<pb::IndexRequest>,) -> Result<Response<pb::IndexResponse>, Status> {- let req = request.into_inner();+ let (_meta, extensions, req) = request.into_parts();+ authorize(&extensions, &req.index, Privilege::WriteIndex)?;// Index-on-write: ES auto-creates the index on first document.let idx = self@@ -146,12 +195,20 @@ impl XerjSearch for GrpcService {&self,request: Request<Streaming<pb::IndexRequest>>,) -> Result<Response<pb::BulkResponse>, Status> {- let mut stream = request.into_inner();+ let (_meta, extensions, mut stream) = request.into_parts();let started = std::time::Instant::now();let mut indexed = 0i32;let mut errors = false;while let Some(item) = stream.message().await? {+ // Per message: one stream may name many indices, and an+ // unauthorized one must not ride in on an authorized one. A refused+ // item is counted as an error, matching how this loop already+ // reports a bad index or an unparsable source.+ if authorize(&extensions, &item.index, Privilege::WriteIndex).is_err() {+ errors = true;+ continue;+ }let idx = match self.state.engine.get_or_create_index(&item.index) {Ok(i) => i,Err(_) => {@@ -190,7 +247,11 @@ impl XerjSearch for GrpcService {&self,request: Request<pb::GetRequest>,) -> Result<Response<pb::GetResponse>, Status> {- let req = request.into_inner();+ let (_meta, extensions, req) = request.into_parts();+ // Deliberately a hard denial rather than this RPC's soft "not found":+ // answering `found: false` for an unauthorized index would still let a+ // caller probe which indices exist by timing or by error shape.+ authorize(&extensions, &req.index, Privilege::ReadIndex)?;// Unknown index → "not found" (ES GET semantics), not an error.let idx = match self.state.engine.get_index(&req.index) {@@ -229,7 +290,8 @@ impl XerjSearch for GrpcService {&self,request: Request<pb::DeleteRequest>,) -> Result<Response<pb::DeleteResponse>, Status> {- let req = request.into_inner();+ let (_meta, extensions, req) = request.into_parts();+ authorize(&extensions, &req.index, Privilege::WriteIndex)?;let idx = self.state@@ -285,19 +347,23 @@ struct GrpcAuth {}impl tonic::service::Interceptor for GrpcAuth {- fn call(&mut self, request: Request<()>) -> Result<Request<()>, Status> {+ fn call(&mut self, mut request: Request<()>) -> Result<Request<()>, Status> {let auth_header = request.metadata().get("authorization").and_then(|v| v.to_str().ok());- if xerj_api::auth::is_authorized(&self.state, auth_header) {- Ok(request)- } else {- Err(Status::unauthenticated(+ // Resolve the identity, not just a yes/no, and carry it into the RPC:+ // the index an RPC touches lives in its message body, so the per-index+ // decision cannot be made here — only the principal can be.+ let principal = xerj_api::auth::authenticate(&self.state, auth_header);+ if matches!(principal, Principal::Denied) {+ return Err(Status::unauthenticated("missing or invalid API key in authorization metadata",- ))+ ));}+ request.extensions_mut().insert(principal);+ Ok(request)}}@@ -584,4 +650,236 @@ mod tests {let _ = tx.send(());let _ = server.await;}++ /// Standard-alphabet base64, so the test can build the `id:secret`+ /// credential a minted key presents. `xerj_api`'s encoder is crate-private.+ fn b64(input: &str) -> String {+ const A: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";+ let bytes = input.as_bytes();+ let mut out = String::new();+ for chunk in bytes.chunks(3) {+ let b = [+ chunk[0],+ *chunk.get(1).unwrap_or(&0),+ *chunk.get(2).unwrap_or(&0),+ ];+ let n = ((b[0] as u32) << 16) | ((b[1] as u32) << 8) | b[2] as u32;+ out.push(A[(n >> 18) as usize & 63] as char);+ out.push(A[(n >> 12) as usize & 63] as char);+ out.push(if chunk.len() > 1 {+ A[(n >> 6) as usize & 63] as char+ } else {+ '='+ });+ out.push(if chunk.len() > 2 {+ A[n as usize & 63] as char+ } else {+ '='+ });+ }+ out+ }++ /// Mint a key straight into the engine's store and return the+ /// `authorization` metadata value a client would present.+ fn mint_key(state: &AppState, id: &str, roles: Vec<xerj_engine::rbac::Role>) -> String {+ state.engine.persist_api_key(+ id.to_string(),+ xerj_engine::engine::ApiKeyRecord {+ name: id.to_string(),+ secret: "s3cret".into(),+ creation_ms: 0,+ expiration_ms: None,+ invalidated: false,+ roles,+ },+ );+ format!("ApiKey {}", b64(&format!("{id}:s3cret")))+ }++ /// Issue #79 on the gRPC listener. Per-brain authorization has to hold on+ /// :8081 exactly as it does on :9200 and :8080 — these RPCs take an index+ /// name straight out of the message body, so before this the boolean auth+ /// check let any authenticated client read, forge and delete any brain's+ /// edges here even while the HTTP surfaces refused.+ #[tokio::test]+ async fn grpc_enforces_per_index_authorization() {+ use std::collections::HashSet;+ use tonic::Code;+ use xerj_engine::rbac::{Privilege, Role};++ let dir = TempDir::new().unwrap();+ let admin = "grpc-admin-secret";+ let state = app_state_with_auth(&dir, admin);++ // A key scoped to alice's brain, and a legacy key with no grants.+ let alice = mint_key(+ &state,+ "alice-key",+ vec![Role::new(+ "alice",+ [Privilege::ReadIndex, Privilege::WriteIndex]+ .into_iter()+ .collect::<HashSet<_>>(),+ vec![".xerj-memory-alice-edges".into()],+ )],+ );+ let legacy = mint_key(&state, "legacy-key", Vec::new());++ let port = free_port();+ let addr: SocketAddr = format!("127.0.0.1:{port}").parse().unwrap();+ let (tx, rx) = tokio::sync::oneshot::channel::<()>();+ let server = tokio::spawn(serve_grpc(addr, state, async move {+ let _ = rx.await;+ }));+ let url = format!("http://127.0.0.1:{port}");+ let mut client = connect(&url).await;++ let with = |cred: &str| {+ let v: tonic::metadata::MetadataValue<tonic::metadata::Ascii> = cred.parse().unwrap();+ v+ };++ // Seed bob's brain as the admin, so there is something to keep apart.+ let mut req = tonic::Request::new(pb::IndexRequest {+ index: ".xerj-memory-bob-edges".into(),+ id: "e1".into(),+ source_json: r#"{"src":"doc:1","dst":"secret:2","type":"mentions"}"#.into(),+ });+ req.metadata_mut()+ .insert("authorization", with(&format!("ApiKey {admin}")));+ assert_eq!(+ client+ .index(req)+ .await+ .expect("admin seed")+ .into_inner()+ .result,+ "created"+ );++ // ── alice cannot touch bob, by any RPC ──────────────────────────────+ let mut req = tonic::Request::new(pb::SearchRequest {+ index: ".xerj-memory-bob-edges".into(),+ query_json: String::new(),+ size: 10,+ from: 0,+ });+ req.metadata_mut().insert("authorization", with(&alice));+ let e = client+ .search(req)+ .await+ .expect_err("search bob must be denied");+ assert_eq!(e.code(), Code::PermissionDenied, "search: {e:?}");++ let mut req = tonic::Request::new(pb::GetRequest {+ index: ".xerj-memory-bob-edges".into(),+ id: "e1".into(),+ });+ req.metadata_mut().insert("authorization", with(&alice));+ let e = client+ .get_document(req)+ .await+ .expect_err("get bob must be denied");+ assert_eq!(e.code(), Code::PermissionDenied, "get: {e:?}");++ let mut req = tonic::Request::new(pb::IndexRequest {+ index: ".xerj-memory-bob-edges".into(),+ id: "forged".into(),+ source_json: r#"{"src":"doc:1","dst":"attacker:payload"}"#.into(),+ });+ req.metadata_mut().insert("authorization", with(&alice));+ let e = client+ .index(req)+ .await+ .expect_err("forge into bob must be denied");+ assert_eq!(e.code(), Code::PermissionDenied, "index: {e:?}");++ let mut req = tonic::Request::new(pb::DeleteRequest {+ index: ".xerj-memory-bob-edges".into(),+ id: "e1".into(),+ });+ req.metadata_mut().insert("authorization", with(&alice));+ let e = client+ .delete_document(req)+ .await+ .expect_err("delete from bob must be denied");+ assert_eq!(e.code(), Code::PermissionDenied, "delete: {e:?}");++ // A wildcard is refused rather than expanded.+ let mut req = tonic::Request::new(pb::SearchRequest {+ index: "*".into(),+ query_json: String::new(),+ size: 10,+ from: 0,+ });+ req.metadata_mut().insert("authorization", with(&alice));+ let e = client+ .search(req)+ .await+ .expect_err("wildcard must be denied");+ assert_eq!(e.code(), Code::PermissionDenied, "wildcard: {e:?}");++ // ── alice keeps full use of her own brain ───────────────────────────+ let mut req = tonic::Request::new(pb::IndexRequest {+ index: ".xerj-memory-alice-edges".into(),+ id: "e1".into(),+ source_json: r#"{"src":"doc:1","dst":"doc:2","type":"mentions"}"#.into(),+ });+ req.metadata_mut().insert("authorization", with(&alice));+ assert_eq!(+ client+ .index(req)+ .await+ .expect("own write")+ .into_inner()+ .result,+ "created"+ );+ let mut req = tonic::Request::new(pb::GetRequest {+ index: ".xerj-memory-alice-edges".into(),+ id: "e1".into(),+ });+ req.metadata_mut().insert("authorization", with(&alice));+ assert!(+ client+ .get_document(req)+ .await+ .expect("own read")+ .into_inner()+ .found+ );++ // ── the legacy key holds nothing in the reserved namespace, but keeps+ // ── ordinary indices.+ let mut req = tonic::Request::new(pb::GetRequest {+ index: ".xerj-memory-alice-edges".into(),+ id: "e1".into(),+ });+ req.metadata_mut().insert("authorization", with(&legacy));+ let e = client+ .get_document(req)+ .await+ .expect_err("legacy key must not read a brain");+ assert_eq!(e.code(), Code::PermissionDenied, "legacy reserved: {e:?}");++ let mut req = tonic::Request::new(pb::IndexRequest {+ index: "logs-2026".into(),+ id: "d1".into(),+ source_json: r#"{"msg":"hello"}"#.into(),+ });+ req.metadata_mut().insert("authorization", with(&legacy));+ assert_eq!(+ client+ .index(req)+ .await+ .expect("legacy key keeps ordinary indices")+ .into_inner()+ .result,+ "created"+ );++ let _ = tx.send(());+ let _ = server.await;+ }}