Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

console: session-authorized graph read path so a signed-in operator can see brains (#936)

xerj-org5089d5b99da6
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c5adc4349..57af256b9 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
+### Fixed
+
+- **The Second Brain dashboard and the reader's graph panel are no longer 401
+ for a signed-in operator on an auth-enabled engine (the default).** The
+ console holds a passkey session, not an engine API key, so the SPA's direct
+ `/_graph/{brain}/*`, `_cat/indices/.xerj-memory-*` and reserved-namespace
+ `_search` reads were refused — while a share-link guest, who holds a key,
+ could read the graph. The console now serves its own brain-scoped,
+ session-authorized graph read path
+ (`GET /_xerj-console/api/v1/graph/brains`, `/{brain}/ego`, `/{brain}/overview`,
+ `POST /{brain}/edges/_search`, `POST /{brain}/nodes/_search`) in the same
+ `xerj-second-brain/1` contract as the data plane: console roles `owner` and
+ `admin` may read brains, a role-refused read and a nonexistent brain are
+ byte-identical 404s (no existence oracle), and the nodes/edges searches are
+ pinned server-side to the brain's own meta-doc indices, so the endpoints
+ cannot become a general search proxy and the generic data-sources proxy
+ still refuses the reserved namespace. `/v1/metrics` (the dashboard's
+ searches-per-index tile) stays key-only by decision; the tile shows the
+ refusal honestly. [#936](https://github.com/xerj-org/xerj/issues/936)
+
## [1.0.0-rc.75] - 2026-09-20
### Added
diff --git a/docs/CONSOLE_READER.md b/docs/CONSOLE_READER.md
index e304a3bf6..66594375c 100644
--- a/docs/CONSOLE_READER.md
+++ b/docs/CONSOLE_READER.md
@@ -138,8 +138,11 @@ extracted text, not attachment bytes.
### The graph panel
-The panel is a `GET /_graph/{brain}/ego?node=<id>&hops=1&direction=both` call
-([SECOND_BRAIN.md](./SECOND_BRAIN.md#get-_graphbrainego)). Neighbours are
+The panel is a `GET /_xerj-console/api/v1/graph/{brain}/ego?node=<id>&hops=1&direction=both`
+call through the console's own session-authorized graph read path
+(`xerj-console-api/src/graph.rs`, same `xerj-second-brain/1` contract as the
+data plane's `/_graph/{brain}/ego` in
+[SECOND_BRAIN.md](./SECOND_BRAIN.md#get-_graphbrainego)). Neighbours are
grouped by edge type, each one a link that opens in the same Reader.
autoindex's file-level detectors (`same_dir`, `mdlink`, `pathcite`, `href`)
@@ -154,16 +157,28 @@ The panel states, in words, which of these is true: no brain for this index, a
brain with no links for this record, links clipped by the limit, links to ids
with no document behind them, or a refusal.
-**On an auth-enabled engine (the default) the graph panel is refused for a
-signed-in operator.** The console holds a passkey session, not an engine API
-key, and the console's search proxy deliberately cannot read the reserved
-`.xerj-memory-*` namespace
-([SECURITY_MODEL.md](./SECURITY_MODEL.md)). Search, records and attachments go
-through the session-authenticated proxy and work; the graph panel says "the
-graph API refused this console session (HTTP 401)" rather than claiming there
-are no links. On `--insecure` it works. A guest's graph panel works on an
-auth-enabled engine, because a guest *has* a key (below). Tracked as
-[#936](https://github.com/xerj-org/xerj/issues/936).
+**On an auth-enabled engine (the default) the graph panel works for the
+operator tier.** The console holds a passkey session, not an engine API key,
+so the panel cannot read `/_graph/*` (Authorization-header credential) and the
+generic search proxy still deliberately refuses the reserved `.xerj-memory-*`
+namespace ([SECURITY_MODEL.md](./SECURITY_MODEL.md)) — both decisions stand.
+Instead the console exposes brain-scoped, role-gated read endpoints of its own
+(`/_xerj-console/api/v1/graph/brains`, `/{brain}/ego`, `/{brain}/overview`,
+`/{brain}/edges/_search`, `/{brain}/nodes/_search`; issue
+[#936](https://github.com/xerj-org/xerj/issues/936)):
+
+- Console roles `owner` and `admin` may read brains; `editor` and `viewer`
+ may not (the operator tier — the same roles that may issue invites; brains
+ are node-local until a tenant model exists).
+- A role-refused read and a nonexistent brain are **indistinguishable**
+ (byte-identical 404s), so a session that may not read brains cannot
+ enumerate them; the reader then says "no brain for this index".
+- The nodes/edges searches are pinned server-side to the brain's own meta-doc
+ indices — the endpoints cannot be turned into a general search proxy, and
+ the generic proxy's refusal of the reserved namespace is unchanged.
+
+A guest's graph panel works on an auth-enabled engine, because a guest *has* a
+key (below).
### Discover
@@ -304,8 +319,11 @@ skip (CI does).
- The share link itself — minting, passcodes, claim limits, revocation — is a
separate surface; this page covers only what the console does with the
record it is handed.
-- The operator's graph panel needs `--insecure` today (above,
- [#936](https://github.com/xerj-org/xerj/issues/936)).
+- The Second Brain dashboard's "searches per index" tile reads
+ `/v1/metrics`, which stays a data-plane, engine-key surface by decision;
+ for a signed-in operator the tile shows the refusal honestly rather than an
+ invented number. Everything else on that dashboard reads through the
+ console's graph endpoints (above).
- The console's passkey sign-in only works when the console is reached at
`http://localhost:9200`: the WebAuthn relying-party origin is fixed
(`xerj-console-api/src/state.rs`, `RpConfig::default`), so a node on another
diff --git a/docs/SECURITY_MODEL.md b/docs/SECURITY_MODEL.md
index 02cb9e76b..19ba86790 100644
--- a/docs/SECURITY_MODEL.md
+++ b/docs/SECURITY_MODEL.md
@@ -133,7 +133,14 @@ Three consequences follow, and all three are load-bearing:
`authz_middleware` never runs for it and its in-process engine calls carry no
guard. The Console data-sources proxy therefore refuses the reserved namespace
itself, with the same `404` a Console system index gets
- (`xerj-console-api/src/data_sources.rs:37-48`). The gRPC listener does the
+ (`xerj-console-api/src/data_sources.rs:37-48`). The Console's own graph read
+ path reaches that namespace deliberately — but only through brain-scoped
+ routes that gate on the console session's role (`owner`/`admin` may read
+ brains), resolve the indices from each brain's own meta document, and answer
+ a role-refused read and a nonexistent brain identically, so it cannot be
+ used as an existence oracle or a general search proxy
+ (`xerj-console-api/src/graph.rs`, issue
+ [#936](https://github.com/xerj-org/xerj/issues/936)). The gRPC listener does the
same with `Principal::allows_index` (`xerj-server/src/grpc.rs:62-75`) and
refuses index patterns outright for non-superusers (`grpc.rs:81-88`).
diff --git a/engine/crates/xerj-console-api/src/graph.rs b/engine/crates/xerj-console-api/src/graph.rs
new file mode 100644
index 000000000..7cee0261d
--- /dev/null
+++ b/engine/crates/xerj-console-api/src/graph.rs
@@ -0,0 +1,1074 @@
+//! Session-authorized Second-Brain graph reads (issue #936).
+//!
+//! The data plane's `/_graph/{brain}/ego|overview` routes authenticate an
+//! engine *credential* (`Authorization` header — API key), and the
+//! session-authenticated data-sources proxy deliberately refuses the reserved
+//! `.xerj-memory-*` namespace (RC10 B1, pinned by
+//! `tests/console_cannot_read_a_brain.rs`). Both decisions are right; together
+//! they meant a signed-in console operator could not see any brain at all on
+//! an auth-enabled engine — the default posture — while a share-link *guest*,
+//! who holds a key, could.
+//!
+//! This module is the console's own read path for the graph. It authorizes
+//! the **console session's role** and then reads the engine in-process (the
+//! same `Engine` handle `data_sources` already uses), so no data-plane
+//! credential is involved and the data plane's Authorization-header boundary
+//! is untouched.
+//!
+//! ## The authorization rule
+//!
+//! Console roles `owner` and `admin` may read brains; `editor` and `viewer`
+//! may not. Brains are node-local today: there is no per-tenant ownership
+//! model anywhere yet, so the operator tier is the node's operator — the same
+//! tier that may issue invites (`auth::magic::issue` gates `owner|admin`).
+//! When a tenant model lands, this gate is where a per-user allow-list goes.
+//!
+//! No existence oracle, mirroring the discipline the search proxy set: a
+//! role-refused read and a nonexistent brain return **byte-identical**
+//! responses (same status, same body modulo the brain name the caller
+//! supplied), so a session that may not read brains cannot enumerate them.
+//! The `brains` listing is a flat role gate — 403 for a disallowed role —
+//! which reveals no per-brain fact.
+//!
+//! ## Endpoints
+//!
+//! ```text
+//! GET /_xerj-console/api/v1/graph/brains list brains (+ nodes_index)
+//! GET /_xerj-console/api/v1/graph/{brain}/ego §4.3 neighborhood
+//! GET /_xerj-console/api/v1/graph/{brain}/overview §4.4 brain stats
+//! POST /_xerj-console/api/v1/graph/{brain}/edges/_search ES _search on the
+//! brain's edges index only
+//! POST /_xerj-console/api/v1/graph/{brain}/nodes/_search ES _search on ONE
+//! of the brain's nodes
+//! indices (meta-doc set)
+//! ```
+//!
+//! The read endpoints re-derive their answers with the same engine machinery
+//! the data plane uses — `xerj_engine::graph::Index::graph_expand` for the
+//! traversal, `Index::search` for the bounded `ids` hydrations and the
+//! overview aggregations — and answer in the same JSON contract
+//! (`xerj-second-brain/1`) so the SPA consumes one shape whether it reads
+//! through the console session (here) or a scoped key (data plane). The
+//! implementation mirrors `xerj-api/src/graph_api.rs` handler-for-handler;
+//! each mirrored block names its source. Deliberate differences:
+//!
+//! - the `ego?nodes_index=` override is refused (400): on the data plane it
+//! is authorized against the caller's grants, but a console session holds
+//! no grants, and silently ignoring it would change semantics between the
+//! two surfaces. Hydration always resolves through the brain's meta doc.
+//! - the reserved namespace is reached only through these brain-scoped
+//! routes. The generic data-sources proxy still refuses it (B1 unchanged),
+//! and `link`/`unlink` have no console path — brains are written by the
+//! engine's own writers (`xerj brain`, agents with keys), read here.
+//!
+//! `/v1/metrics` for the Second Brain dashboard's "searches per index" tile
+//! deliberately stays key-only: the metrics surface is an ops plane, and the
+//! tile already renders the refusal honestly.
+
+use std::collections::{HashMap, HashSet};
+
+use axum::{
+ extract::{Path, Query, State},
+ http::StatusCode,
+ response::{IntoResponse, Response},
+ Json,
+};
+use serde::Deserialize;
+use serde_json::{json, Map, Value};
+
+use crate::auth::sessions::AuthSession;
+use crate::auth::store;
+use crate::error::{ConsoleApiError, ConsoleResult};
+use crate::response::ok;
+use crate::state::ConsoleState;
+use xerj_common::types::RESERVED_INDEX_PREFIX;
+use xerj_engine::graph::{
+ GraphDirection, GraphEdgeLite, GraphExpandRequest, GRAPH_HOPS_CAP_REASON,
+};
+use xerj_engine::Engine;
+
+/// Contract version string, returned by every read endpoint. Byte-identical
+/// to `xerj_api::graph_api::GRAPH_CONTRACT` — the console answers the same
+/// contract the data plane does.
+const GRAPH_CONTRACT: &str = "xerj-second-brain/1";
+
+/// Reserved `_id` of the per-brain meta document (SECOND_BRAIN_SPEC §2.5).
+const BRAIN_META_ID: &str = "__xerj-brain-meta";
+
+/// Max returned edges for `ego` (same clamp as the data plane).
+const MAX_EGO_LIMIT: usize = 1000;
+
+/// Max seed ids accepted by `ego`'s `nodes=` param (same cap as the data
+/// plane; excess seeds are dropped and counted in `not_shown.frontier_clipped`).
+const EGO_SEEDS_CAP: usize = 64;
+
+/// Max dangling node ids listed verbatim in `not_shown.dangling_ids`.
+const MAX_DANGLING_LISTED: usize = 50;
+
+/// Max brains listed by `GET /brains`; the excess is counted, never silently
+/// dropped. Real nodes hold a handful of brains; the cap is a bound on
+/// per-brain meta-doc reads, not a product limit.
+const BRAINS_LISTED_CAP: usize = 256;
+
+/// Edges index name for a brain (SECOND_BRAIN_SPEC §1) — same construction as
+/// `xerj_api::authz::brain_edges_index`, which names the resource a
+/// data-plane grant must hold.
+fn edges_index(brain: &str) -> String {
+ format!("{RESERVED_INDEX_PREFIX}{brain}-edges")
+}
+
+/// Default nodes index for a brain: the agent-memory namespace of the same
+/// name. Autoindex brains override this via the §2.5 meta doc.
+fn default_nodes_index(brain: &str) -> String {
+ format!("{RESERVED_INDEX_PREFIX}{brain}")
+}
+
+/// Brain-name validation (SECOND_BRAIN_SPEC §1): byte-identical rules to
+/// `xerj_api::graph_api::validate_brain` (lowercase/digit start, `[a-z0-9._-]`,
+/// ≤200 chars, no `..`, `-edges` suffix reserved). A copy, not a shared fn,
+/// matching the existing deliberate duplication (`memory_api`,
+/// `xerj_autoindex::detect`) — the console crate does not depend on
+/// `xerj-api`, on purpose (see `lib.rs`).
+fn validate_brain(brain: &str) -> Result<(), String> {
+ if brain.is_empty() {
+ return Err("brain name must not be empty".into());
+ }
+ if brain.len() > 200 {
+ return Err("brain name too long (max 200 chars)".into());
+ }
+ let first = brain.chars().next().unwrap();
+ if !(first.is_ascii_lowercase() || first.is_ascii_digit()) {
+ return Err("brain name must start with a lowercase letter or digit".into());
+ }
+ for c in brain.chars() {
+ let ok = c.is_ascii_lowercase() || c.is_ascii_digit() || matches!(c, '_' | '-' | '.');
+ if !ok {
+ return Err(format!(
+ "brain name contains illegal character '{c}' (allowed: a-z 0-9 _ - .)"
+ ));
+ }
+ }
+ if brain.contains("..") {
+ return Err("brain name must not contain '..'".into());
+ }
+ if brain.ends_with("-edges") {
+ return Err("namespace suffix '-edges' is reserved for graph edge indices".into());
+ }
+ Ok(())
+}
+
+/// May this console session read brains? The node's operator tier — the same
+/// roles that may issue invites (`auth::magic::issue`). See module docs for
+/// the rule and its no-tenant-model caveat. Public because
+/// `tests/console_graph_read.rs` pins the tier boundary; not part of the
+/// HTTP surface.
+#[doc(hidden)]
+pub fn may_read_brains(user: &store::User) -> bool {
+ matches!(user.role.as_str(), "owner" | "admin")
+}
+
+/// The 404 every "you may not know about this brain" path returns — identical
+/// for a role-refused read and a missing brain, modulo the caller-supplied
+/// name. Mirrors the data plane's unknown-brain 404 shape (`graph_error`)
+/// so the SPA sees one contract.
+fn brain_not_found(brain: &str) -> Response {
+ (
+ StatusCode::NOT_FOUND,
+ Json(json!({
+ "error": {
+ "type": "graph_error",
+ "reason": format!("brain '{brain}' does not exist (no edges index)"),
+ },
+ "status": 404,
+ })),
+ )
+ .into_response()
+}
+
+/// A uniform graph error response (same shape as the data plane's).
+fn graph_error(status: StatusCode, reason: impl Into<String>) -> Response {
+ (
+ status,
+ Json(json!({
+ "error": { "type": "graph_error", "reason": reason.into() },
+ "status": status.as_u16(),
+ })),
+ )
+ .into_response()
+}
+
+/// Validate the brain name, then apply the role gate in the oracle-safe
+/// order: a session that may not read brains gets exactly the response a
+/// nonexistent brain produces. Malformed names 400 for every role — a
+/// validation error leaks no existence fact. `Some(response)` = return it.
+fn gate_brain(user: &store::User, brain: &str) -> Option<Response> {
+ if let Err(reason) = validate_brain(brain) {
+ return Some(graph_error(StatusCode::BAD_REQUEST, reason));
+ }
+ if !may_read_brains(user) {
+ return Some(brain_not_found(brain));
+ }
+ None
+}
+
+/// Run one ES-shaped `_search` body against one index, in-process. The same
+/// pattern `data_sources::search` uses (`parse_request` → `Index::search`);
+/// errors surface as a Console 400/500 like every other console read.
+async fn run_search(
+ engine: &Engine,
+ index: &str,
+ body: &Value,
+) -> ConsoleResult<xerj_query::executor::SearchResult> {
+ let idx = engine.get_index(index)?;
+ let req = xerj_query::parser::parse_request(body)
+ .map_err(|e| ConsoleApiError::BadRequest(e.to_string()))?;
+ idx.search(&req)
+ .await
+ .map_err(|e| ConsoleApiError::Internal(e.to_string()))
+}
+
+/// One document by id, via the ids-prefilter fast path (the same read the
+/// data plane's `get_doc` resolves to). `None` when absent.
+async fn get_doc_source(engine: &Engine, index: &str, id: &str) -> ConsoleResult<Option<Value>> {
+ let body = json!({ "query": { "ids": { "values": [id] } }, "size": 1 });
+ let r = run_search(engine, index, &body).await?;
+ Ok(r.hits.into_iter().next().map(|h| h.source))
+}
+
+/// The concrete names in a `nodes_index` value — `xerj brain` records a
+/// multi-dataset folder as one comma-joined string (`"ax-mail,ax-pdfs"`).
+/// Mirrors `graph_api::nodes_index_names`.
+fn nodes_index_names(nodes_index: &str) -> impl Iterator<Item = &str> {
+ nodes_index
+ .split(',')
+ .map(str::trim)
+ .filter(|n| !n.is_empty())
+}
+
+/// Resolve the nodes index for a brain: the meta doc's `nodes_index`, else
+/// the `.xerj-memory-{brain}` default. Mirrors `graph_api::resolve_nodes_index`
+/// (explicit-param override excluded — see module docs).
+async fn resolve_nodes_index(engine: &Engine, brain: &str, edges: &str) -> String {
+ if let Some(src) = get_doc_source(engine, edges, BRAIN_META_ID)
+ .await
+ .ok()
+ .flatten()
+ {
+ if let Some(ni) = src.get("nodes_index").and_then(Value::as_str) {
+ if !ni.is_empty() {
+ return ni.to_string();
+ }
+ }
+ }
+ default_nodes_index(brain)
+}
+
+/// Parse a query-string timestamp: decimal epoch-ms first, RFC3339 second
+/// (mirrors `graph_api::parse_ms_str`).
+fn parse_ms_str(s: &str) -> Option<i64> {
+ if let Ok(n) = s.parse::<i64>() {
+ return Some(n);
+ }
+ chrono::DateTime::parse_from_rfc3339(s)
+ .ok()
+ .map(|dt| dt.timestamp_millis())
+}
+
+/// Server "now" in epoch milliseconds.
+fn now_ms() -> i64 {
+ chrono::Utc::now().timestamp_millis()
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+// GET /brains — discovery listing
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// `GET /_xerj-console/api/v1/graph/brains` — every brain on this node, with
+/// the `nodes_index` its meta doc records. Replaces the SPA's direct
+/// `_cat/indices/.xerj-memory-*` walk + per-brain meta-doc reads, both of
+/// which 401 for a session. Flat role gate: 403 for a role that may not read
+/// brains (reveals no per-brain fact), listing for the operator tier.
+pub async fn brains(
+ State(state): State<ConsoleState>,
+ sess: AuthSession,
+) -> ConsoleResult<Response> {
+ if !may_read_brains(&sess.user) {
+ return Err(ConsoleApiError::Forbidden(format!(
+ "the console role '{}' may not read brains",
+ sess.user.role
+ )));
+ }
+
+ // Brain = a reserved `.xerj-memory-{brain}-edges` index that exists.
+ let mut names: Vec<String> = state
+ .engine
+ .index_name_list()
+ .into_iter()
+ .filter(|n| {
+ n.starts_with(RESERVED_INDEX_PREFIX)
+ && n.ends_with("-edges")
+ && n.len() > RESERVED_INDEX_PREFIX.len() + "-edges".len()
+ })
+ .map(|n| n[RESERVED_INDEX_PREFIX.len()..n.len() - "-edges".len()].to_string())
+ .filter(|b| validate_brain(b).is_ok())
+ .collect();
+ names.sort();
+
+ let total = names.len();
+ let clipped = total.saturating_sub(BRAINS_LISTED_CAP);
+ names.truncate(BRAINS_LISTED_CAP);
+
+ let mut brains: Vec<Value> = Vec::with_capacity(names.len());
+ for brain in names {
+ let edges = edges_index(&brain);
+ let nodes_index = resolve_nodes_index(&state.engine, &brain, &edges).await;
+ brains.push(json!({
+ "name": brain,
+ "nodes_index": nodes_index,
+ "contract": GRAPH_CONTRACT,
+ }));
+ }
+
+ Ok(ok(
+ json!({
+ "brains": brains,
+ "total": total,
+ "not_shown": { "brains_clipped": clipped as u64 },
+ }),
+ None,
+ ))
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+// GET /{brain}/ego — bounded neighborhood
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// Query params for `GET /graph/{brain}/ego` — the data plane's §4.3 set,
+/// minus the `nodes_index` override (refused here; see module docs).
+#[derive(Debug, Default, Deserialize)]
+pub struct EgoParams {
+ #[serde(default)]
+ pub node: Option<String>,
+ #[serde(default)]
+ pub nodes: Option<String>,
+ #[serde(default)]
+ pub hops: Option<u64>,
+ #[serde(default)]
+ pub direction: Option<String>,
+ #[serde(default)]
+ pub types: Option<String>,
+ #[serde(default)]
+ pub limit: Option<u64>,
+ #[serde(default)]
+ pub as_of: Option<String>,
+ #[serde(default)]
+ pub include_expired: Option<bool>,
+ #[serde(default)]
+ pub include_nodes: Option<bool>,
+ #[serde(default)]
+ pub nodes_index: Option<String>,
+ #[serde(default)]
+ pub include_evidence: Option<bool>,
+}
+
+/// `GET /_xerj-console/api/v1/graph/{brain}/ego` — the §4.3 neighborhood of
+/// one node (or ≤64 via `nodes=`). Mirrors `graph_api::ego` step for step;
+/// the authorization halves differ (console role gate instead of Principal
+/// grants, in-process hydrations instead of composed ES-compat handlers).
+pub async fn ego(
+ State(state): State<ConsoleState>,
+ sess: AuthSession,
+ Path(brain): Path<String>,
+ Query(params): Query<EgoParams>,
+) -> Response {
+ if let Some(resp) = gate_brain(&sess.user, &brain) {
+ return resp;
+ }
+ if let Some(ni) = params.nodes_index.as_deref() {
+ let _ = ni;
+ return graph_error(
+ StatusCode::BAD_REQUEST,
+ "`nodes_index` is not supported on the console path: hydration resolves \
+ through the brain's meta document",
+ );
+ }
+ if params.node.is_some() && params.nodes.is_some() {
+ return graph_error(
+ StatusCode::BAD_REQUEST,
+ "`node` and `nodes` are mutually exclusive — `nodes` is the multi-seed form",
+ );
+ }
+ // Seed list: `node` is the 1-element case of `nodes`. Deduped preserving
+ // order (the order is part of the `reachable` contract), then clamped to
+ // EGO_SEEDS_CAP with the clip counted — never silent.
+ let raw_seeds: Vec<String> = match (params.node.as_deref(), params.nodes.as_deref()) {
+ (Some(n), None) if !n.is_empty() => vec![n.to_string()],
+ (None, Some(ns)) => ns
+ .split(',')
+ .map(str::trim)
+ .filter(|s| !s.is_empty())
+ .map(String::from)
+ .collect(),
+ _ => Vec::new(),
+ };
+ let mut seeds: Vec<String> = Vec::with_capacity(raw_seeds.len());
+ {
+ let mut seen: HashSet<&str> = HashSet::with_capacity(raw_seeds.len());
+ for id in &raw_seeds {
+ if seen.insert(id.as_str()) {
+ seeds.push(id.clone());
+ }
+ }
+ }
+ if seeds.is_empty() {
+ return graph_error(
+ StatusCode::BAD_REQUEST,
+ "`node` (or comma-separated `nodes`) is required: the node id(s) to expand from",
+ );
+ }
+ let mut seeds_clipped = 0u64;
+ if seeds.len() > EGO_SEEDS_CAP {
+ seeds_clipped = (seeds.len() - EGO_SEEDS_CAP) as u64;
+ seeds.truncate(EGO_SEEDS_CAP);
+ }
+ let hops = params.hops.unwrap_or(1);
+ if hops == 0 || hops > 2 {
+ return graph_error(StatusCode::BAD_REQUEST, GRAPH_HOPS_CAP_REASON);
+ }
+ let direction_str = params.direction.as_deref().unwrap_or("both");
+ let direction = match direction_str {
+ "out" => GraphDirection::Out,
+ "in" => GraphDirection::In,
+ "both" => GraphDirection::Both,
+ other => {
+ return graph_error(
+ StatusCode::BAD_REQUEST,
+ format!("`direction` must be 'out', 'in', or 'both' (got '{other}')"),
+ );
+ }
+ };
+ let types: Option<Vec<String>> = params.types.as_deref().map(|t| {
+ t.split(',')
+ .map(str::trim)
+ .filter(|s| !s.is_empty())
+ .map(String::from)
+ .collect()
+ });
+ let types = types.filter(|t| !t.is_empty());
+ let limit = (params.limit.unwrap_or(100) as usize).clamp(1, MAX_EGO_LIMIT);
+ let as_of = match params.as_of.as_deref() {
+ None => now_ms(),
+ Some(s) => match parse_ms_str(s) {
+ Some(ms) => ms,
+ None => {
+ return graph_error(
+ StatusCode::BAD_REQUEST,
+ "`as_of` must be an epoch-ms number or an RFC3339 string",
+ );
+ }
+ },
+ };
+ let include_expired = params.include_expired.unwrap_or(false);
+ let include_nodes = params.include_nodes.unwrap_or(false);
+ let include_evidence = params.include_evidence.unwrap_or(true);
+
+ let edges = edges_index(&brain);
+ let Ok(idx) = state.engine.get_index(&edges) else {
+ return brain_not_found(&brain);
+ };
+
+ let req = GraphExpandRequest {
+ frontier: seeds.clone(),
+ hops: hops as u8,
+ direction,
+ types,
+ as_of_ms: as_of,
+ include_expired,
+ max_result_edges: limit,
+ };
+ let result = match idx.graph_expand(&req) {
+ Ok(r) => r,
+ Err(e) => return graph_error(StatusCode::BAD_REQUEST, e.to_string()),
+ };
+
+ // Per-edge direction relative to the expansion (mirrors graph_api::ego):
+ // an edge discovered at hop h is "out" iff its src was in hop h's
+ // frontier; the hop-1 frontier is the seed set, the hop-2 frontier is
+ // every endpoint hop 1 discovered.
+ let frontier1: HashSet<&str> = seeds.iter().map(String::as_str).collect();
+ let mut frontier2: HashSet<&str> = HashSet::new();
+ for e in result.edges.iter().filter(|e| e.hop == 1) {
+ for id in [e.src.as_str(), e.dst.as_str()] {
+ if !frontier1.contains(id) {
+ frontier2.insert(id);
+ }
+ }
+ }
+ let edge_direction = |e: &GraphEdgeLite| -> &'static str {
+ let frontier = if e.hop == 1 { &frontier1 } else { &frontier2 };
+ if frontier.contains(e.src.as_str()) {
+ "out"
+ } else {
+ "in"
+ }
+ };
+
+ // Post-traversal hydration (bounded ≤ limit ≤ 1000): ONE `ids` search on
+ // the edges index for evidence/envelope fields — rides the ids-prefilter
+ // fast path; the hop itself never touched `_source`.
+ let mut hydrated: HashMap<String, Value> = HashMap::new();
+ if include_evidence && !result.edges.is_empty() {
+ let ids: Vec<&str> = result.edges.iter().map(|e| e.edge_id.as_str()).collect();
+ let body = json!({
+ "query": { "ids": { "values": ids } },
+ "size": ids.len(),
+ "_source": ["edge_id", "created_at", "detector", "confidence", "evidence", "expired_at"],
+ });
+ if let Ok(r) = run_search(&state.engine, &edges, &body).await {
+ for h in r.hits {
+ if let Some(id) = h.source.get("edge_id").and_then(Value::as_str) {
+ hydrated.insert(id.to_string(), h.source);
+ }
+ }
+ }
+ }
+
+ // Node summaries (`include_nodes=true`): ONE `ids` search per concrete
+ // nodes-index name for every reachable id. Ids that resolve nowhere are
+ // DANGLING — the edge is kept, the honesty is counted, and up to 50 ids
+ // are listed.
+ let mut nodes_obj = Map::new();
+ let mut dangling_nodes = 0u64;
+ let mut dangling_ids: Vec<String> = Vec::new();
+ if include_nodes {
+ let nodes_index = resolve_nodes_index(&state.engine, &brain, &edges).await;
+ let mut found: HashSet<String> = HashSet::new();
+ for name in nodes_index_names(&nodes_index) {
+ // A nodes index that does not exist (edges asserted through the
+ // API alone) contributes nothing; every reachable id is then
+ // reported dangling, never fabricated.
+ if state.engine.get_index(name).is_err() {
+ continue;
+ }
+ let ids: Vec<&str> = result.reachable.iter().map(String::as_str).collect();
+ let body = json!({
+ "query": { "ids": { "values": ids } },
+ "size": ids.len(),
+ "_source": ["title", "text", "body", "ax_path"],
+ });
+ let Ok(r) = run_search(&state.engine, name, &body).await else {
+ continue;
+ };
+ for h in r.hits {
+ let id = h.id.clone();
+ let title = h
+ .source
+ .get("title")
+ .and_then(Value::as_str)
+ .map(Value::from)
+ .unwrap_or(Value::Null);
+ let preview = h
+ .source
+ .get("text")
+ .and_then(Value::as_str)
+ .or_else(|| h.source.get("body").and_then(Value::as_str))
+ .map(|t| Value::from(t.chars().take(160).collect::<String>()))
+ .unwrap_or(Value::Null);
+ // Truthful file label: the autoindex writer stamps `ax_path`
+ // on every note; null when absent — never fabricated.
+ let path = h
+ .source
+ .get("ax_path")
+ .and_then(Value::as_str)
+ .map(Value::from)
+ .unwrap_or(Value::Null);
+ found.insert(id.clone());
+ nodes_obj.entry(id).or_insert(json!({
+ "title": title,
+ "preview": preview,
+ "path": path,
+ "index": name,
+ }));
+ }
+ }
+ let mut dangling: Vec<&String> = result
+ .reachable
+ .iter()
+ .filter(|id| !found.contains(*id))
+ .collect();
+ dangling.sort();
+ dangling_nodes = dangling.len() as u64;
+ dangling_ids = dangling
+ .into_iter()
+ .take(MAX_DANGLING_LISTED)
+ .cloned()
+ .collect();
+ }
+
+ // Edges in the §3.2 stable order (the engine already sorted), each
+ // annotated with its expansion direction; `invalid_at` is JSON null when
+ // unset (response-side null is fine — the omit-rule binds stored docs).
+ let edges_json: Vec<Value> = result
+ .edges
+ .iter()
+ .map(|e| {
+ let mut obj = json!({
+ "edge_id": e.edge_id,
+ "src": e.src,
+ "dst": e.dst,
+ "type": e.edge_type,
+ "weight": e.weight,
+ "hop": e.hop,
+ "direction": edge_direction(e),
+ "valid_at": e.valid_at_ms,
+ "invalid_at": e.invalid_at_ms.map(Value::from).unwrap_or(Value::Null),
+ });
+ if let Some(src) = hydrated.get(&e.edge_id) {
+ for field in [
+ "created_at",
+ "detector",
+ "confidence",
+ "evidence",
+ "expired_at",
+ ] {
+ if let Some(v) = src.get(field) {
+ obj[field] = v.clone();
+ }
+ }
+ }
+ obj
+ })
+ .collect();
+
+ // Neighbors: first-discovery order following the sorted edge list,
+ // excluding the seed nodes; `via_edge` is the first sorted edge that
+ // reached each one.
+ let mut neighbors: Vec<Value> = Vec::new();
+ let mut seen: HashSet<&str> = seeds.iter().map(String::as_str).collect();
+ for e in &result.edges {
+ for id in [e.src.as_str(), e.dst.as_str()] {
+ if seen.insert(id) {
+ neighbors.push(json!({ "id": id, "hop": e.hop, "via_edge": e.edge_id }));
+ }
+ }
+ }
+
+ // `seeds` echoes the ADMITTED seed list (post-dedupe, post-clamp) so the
+ // caller can bookkeep exactly what was expanded; `node` stays on the
+ // response whenever there is exactly one seed (the 1-element case keeps
+ // its historical shape). Handler-clipped seeds fold into the same
+ // `frontier_clipped` counter the engine uses — one honesty channel.
+ let mut resp = json!({
+ "brain": brain,
+ "contract": GRAPH_CONTRACT,
+ "seeds": seeds,
+ "as_of": as_of,
+ "hops": hops,
+ "direction": direction_str,
+ "edges": edges_json,
+ "neighbors": neighbors,
+ "not_shown": {
+ "edges_clipped": result.stats.edges_clipped,
+ "frontier_clipped": result.stats.frontier_clipped + seeds_clipped,
+ "expired_excluded": result.stats.expired_excluded,
+ "type_filtered": result.stats.type_filtered,
+ "segments_without_columns": result.stats.segments_without_columns,
+ "memtable_docs_scanned": result.stats.memtable_docs_scanned,
+ "dangling_nodes": dangling_nodes,
+ "dangling_ids": dangling_ids,
+ }
+ });
+ if let [only] = resp["seeds"].as_array().map(Vec::as_slice).unwrap_or(&[]) {
+ // §8.5 is a normative instance including key order: `node` sits
+ // between `contract` and `seeds`, so rebuild in place rather than
+ // appending (which would serialize `node` last).
+ let only = only.clone();
+ let old = std::mem::take(resp.as_object_mut().expect("ego response is an object"));
+ let mut ordered = serde_json::Map::with_capacity(old.len() + 1);
+ for (k, v) in old {
+ if k == "seeds" {
+ ordered.insert("node".to_string(), only.clone());
+ }
+ ordered.insert(k, v);
+ }
+ resp = Value::Object(ordered);
+ }
+ if include_nodes {
+ resp["nodes"] = Value::Object(nodes_obj);
+ }
+ Json(resp).into_response()
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+// GET /{brain}/overview — brain-level stats
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// Query params for `GET /graph/{brain}/overview` (§4.4; same as the data
+/// plane's `OverviewParams`).
+#[derive(Debug, Default, Deserialize)]
+pub struct OverviewParams {
+ #[serde(default)]
+ pub as_of: Option<String>,
+ #[serde(default)]
+ pub top: Option<u64>,
+ #[serde(default)]
+ pub histogram_interval: Option<String>,
+}
+
+/// Terms-agg buckets → `[{"<key_name>": key, "<count_name>": doc_count}]`,
+/// plus the agg's `sum_other_doc_count` (the not-listed tail, reported
+/// in-band). Mirrors `graph_api::terms_list`.
+fn terms_list(aggs: &Value, agg: &str, key_name: &str, count_name: &str) -> (Vec<Value>, u64) {
+ let buckets = aggs
+ .pointer(&format!("/{agg}/buckets"))
+ .and_then(Value::as_array);
+ let list = buckets
+ .map(|bs| {
+ bs.iter()
+ .map(|b| {
+ json!({
+ key_name: b.get("key").cloned().unwrap_or(Value::Null),
+ count_name: b.get("doc_count").cloned().unwrap_or(json!(0)),
+ })
+ })
+ .collect()
+ })
+ .unwrap_or_default();
+ let other = aggs
+ .pointer(&format!("/{agg}/sum_other_doc_count"))
+ .and_then(Value::as_u64)
+ .unwrap_or(0);
+ (list, other)
+}
+
+/// The honesty marker for the default embedder (§0 invariant 8): the built-in
+/// embedder is LEXICAL feature-hashing — no surface may imply neural
+/// semantics. Mirrors `graph_api::embedder_id`, read off the same engine
+/// config the data plane's AppState holds.
+fn embedder_id(engine: &Engine) -> String {
+ let emb = &engine.config().embedding;
+ match emb.mode.as_str() {
+ "neural" => emb.neural_model.clone(),
+ "proxy" if !emb.default_model.is_empty() => emb.default_model.clone(),
+ "proxy" => "proxy".into(),
+ "auto" if !emb.default_endpoint.is_empty() && !emb.default_model.is_empty() => {
+ emb.default_model.clone()
+ }
+ "auto" if !emb.default_endpoint.is_empty() => "proxy".into(),
+ _ => "lexical-feature-hash".into(),
+ }
+}
+
+/// `GET /_xerj-console/api/v1/graph/{brain}/overview` — totals, live slice
+/// (types/detectors/hubs), the created-over-time histogram, and the notes
+/// total. Mirrors `graph_api::overview`; exactly three searches on the edges
+/// index plus one size-0 count per concrete nodes-index name.
+pub async fn overview(
+ State(state): State<ConsoleState>,
+ sess: AuthSession,
+ Path(brain): Path<String>,
+ Query(params): Query<OverviewParams>,
+) -> Response {
+ if let Some(resp) = gate_brain(&sess.user, &brain) {
+ return resp;
+ }
+ let as_of = match params.as_of.as_deref() {
+ None => now_ms(),
+ Some(s) => match parse_ms_str(s) {
+ Some(ms) => ms,
+ None => {
+ return graph_error(
+ StatusCode::BAD_REQUEST,
+ "`as_of` must be an epoch-ms number or an RFC3339 string",
+ );
+ }
+ },
+ };
+ let top = (params.top.unwrap_or(10) as usize).clamp(1, 50);
+ let interval = params.histogram_interval.as_deref().unwrap_or("day");
+ if !matches!(interval, "day" | "hour") {
+ return graph_error(
+ StatusCode::BAD_REQUEST,
+ "`histogram_interval` must be 'day' or 'hour'",
+ );
+ }
+
+ let edges = edges_index(&brain);
+ if state.engine.get_index(&edges).is_err() {
+ // Same body the data plane returns for an unknown brain (the SPA
+ // treats `exists: false` as a valid answer, not an error) — and the
+ // same body a role-refused read got above.
+ return (
+ StatusCode::NOT_FOUND,
+ Json(json!({ "brain": brain, "contract": GRAPH_CONTRACT, "exists": false })),
+ )
+ .into_response();
+ }
+
+ // 1. Totals: every edge ever asserted (`exists src` excludes the meta doc).
+ let totals_body = json!({
+ "query": { "exists": { "field": "src" } },
+ "size": 0,
+ "track_total_hits": true,
+ });
+ let totals = match run_search(&state.engine, &edges, &totals_body).await {
+ Ok(v) => v,
+ Err(e) => return graph_error(StatusCode::INTERNAL_SERVER_ERROR, e.to_string()),
+ };
+ let total = totals.total.value;
+
+ // 2. Live slice at as_of, with the type/detector/hub breakdowns.
+ let live_body = json!({
+ "query": {
+ "bool": {
+ "filter": [
+ { "exists": { "field": "src" } },
+ { "range": { "valid_at": { "lte": as_of } } }
+ ],
+ "must_not": [
+ { "range": { "invalid_at": { "lte": as_of } } }
+ ]
+ }
+ },
+ "size": 0,
+ "track_total_hits": true,
+ "aggs": {
+ "by_type": { "terms": { "field": "type", "size": top } },
+ "by_detector": { "terms": { "field": "detector", "size": top } },
+ "top_src": { "terms": { "field": "src", "size": top } },
+ "top_dst": { "terms": { "field": "dst", "size": top } },
+ },
+ });
+ let live_resp = match run_search(&state.engine, &edges, &live_body).await {
+ Ok(v) => v,
+ Err(e) => return graph_error(StatusCode::INTERNAL_SERVER_ERROR, e.to_string()),
+ };
+ let live = live_resp.total.value;
+ let aggs = live_resp.aggs.clone().unwrap_or(Value::Null);
+ let (types, types_other) = terms_list(&aggs, "by_type", "type", "live");
+ let (detectors, detectors_other) = terms_list(&aggs, "by_detector", "detector", "live");
+ let (hubs_out, hubs_out_other) = terms_list(&aggs, "top_src", "id", "live_edges");
+ let (hubs_in, hubs_in_other) = terms_list(&aggs, "top_dst", "id", "live_edges");
+
+ // 3. Created-over-time histogram (all asserted edges, not just live —
+ // the timeline shows assertion activity, invalidation does not erase it).
+ let timeline_body = json!({
+ "query": { "exists": { "field": "src" } },
+ "size": 0,
+ "aggs": {
+ "created": {
+ "date_histogram": { "field": "created_at", "calendar_interval": interval }
+ }
+ },
+ });
+ let timeline = match run_search(&state.engine, &edges, &timeline_body).await {
+ Ok(v) => v,
+ Err(e) => return graph_error(StatusCode::INTERNAL_SERVER_ERROR, e.to_string()),
+ };
+ let created_over_time: Vec<Value> = timeline
+ .aggs
+ .as_ref()
+ .and_then(|a| a.pointer("/created/buckets"))
+ .and_then(Value::as_array)
+ .map(|bs| {
+ bs.iter()
+ .map(|b| {
+ json!({
+ "t": b.get("key").cloned().unwrap_or(Value::Null),
+ "count": b.get("doc_count").cloned().unwrap_or(json!(0)),
+ })
+ })
+ .collect()
+ })
+ .unwrap_or_default();
+
+ let nodes_index = resolve_nodes_index(&state.engine, &brain, &edges).await;
+
+ // 4. Notes total: one size-0 count per concrete nodes-index name. A brain
+ // whose nodes index was never created (edges asserted through the API
+ // alone) truthfully has 0 stored notes — reported as such, never
+ // fabricated from edge endpoints.
+ let mut nodes_total: u64 = 0;
+ for name in nodes_index_names(&nodes_index) {
+ if state.engine.get_index(name).is_err() {
+ continue;
+ }
+ let count_body = json!({
+ "query": { "match_all": {} },
+ "size": 0,
+ "track_total_hits": true,
+ });
+ if let Ok(r) = run_search(&state.engine, name, &count_body).await {
+ nodes_total = nodes_total.saturating_add(r.total.value);
+ }
+ }
+
+ Json(json!({
+ "brain": brain,
+ "contract": GRAPH_CONTRACT,
+ "exists": true,
+ "as_of": as_of,
+ "nodes_index": nodes_index,
+ "nodes": { "total": nodes_total },
+ "embedder": embedder_id(&state.engine),
+ "edges": {
+ "total": total,
+ "live": live,
+ "invalidated": total.saturating_sub(live),
+ },
+ "types": types,
+ "detectors": detectors,
+ "hubs": { "out": hubs_out, "in": hubs_in },
+ "created_over_time": created_over_time,
+ "not_shown": {
+ "types_not_listed": types_other,
+ "detectors_not_listed": detectors_other,
+ "hubs_out_not_listed": hubs_out_other,
+ "hubs_in_not_listed": hubs_in_other,
+ }
+ }))
+ .into_response()
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+// POST /{brain}/edges/_search — the brain's edges index, ES wire shape
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// Run an ES-shaped `_search` against one index and answer in the ES wire
+/// shape (the mapping `data_sources::search` already established for
+/// session-authenticated searches).
+async fn es_wire_search(engine: &Engine, index: &str, body: &Value) -> Response {
+ let result = match run_search(engine, index, body).await {
+ Ok(r) => r,
+ Err(e) => return e.into_response(),
+ };
+ let hits: Vec<Value> = result
+ .hits
+ .iter()
+ .map(|h| {
+ json!({
+ "_index": index,
+ "_id": h.id,
+ "_score": h.score,
+ "_source": h.source,
+ })
+ })
+ .collect();
+ Json(json!({
+ "took": result.took_ms,
+ "timed_out": result.timed_out,
+ "hits": {
+ "total": { "value": result.total.value, "relation": result.total.relation },
+ "max_score": result.max_score,
+ "hits": hits,
+ },
+ "aggregations": result.aggs,
+ }))
+ .into_response()
+}
+
+/// `POST /_xerj-console/api/v1/graph/{brain}/edges/_search` — an ES `_search`
+/// against exactly this brain's edges index, for the dashboard's
+/// recent-retirements and file-type-crossing reads (both used to go direct to
+/// `/.xerj-memory-{brain}-edges/_search`, which 401s for a session). The
+/// index is pinned server-side — the caller cannot name another index — so
+/// this opens the reserved namespace only through the same role gate as every
+/// other console graph read, and the generic data-sources proxy keeps
+/// refusing it (RC10 B1 unchanged).
+pub async fn edges_search(
+ State(state): State<ConsoleState>,
+ sess: AuthSession,
+ Path(brain): Path<String>,
+ Json(body): Json<Value>,
+) -> Response {
+ if let Some(resp) = gate_brain(&sess.user, &brain) {
+ return resp;
+ }
+ if state.engine.get_index(&edges_index(&brain)).is_err() {
+ return brain_not_found(&brain);
+ }
+ es_wire_search(&state.engine, &edges_index(&brain), &body).await
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+// POST /{brain}/nodes/_search — one of the brain's nodes indices
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// Query params for `POST /graph/{brain}/nodes/_search`.
+#[derive(Debug, Default, Deserialize)]
+pub struct NodesSearchParams {
+ /// WHICH of the brain's nodes indices to search — mandatory when the
+ /// meta doc records more than one (`xerj brain` over a multi-dataset
+ /// folder writes `"ax-mail,ax-pdfs"`), ignored-correct when it records
+ /// exactly one. Must be one of the brain's own names; anything else is
+ /// the same 404 an unknown brain gets.
+ #[serde(default)]
+ pub index: Option<String>,
+}
+
+/// `POST /_xerj-console/api/v1/graph/{brain}/nodes/_search` — an ES `_search`
+/// against ONE of the brain's nodes indices, resolved from the brain's own
+/// meta doc (never caller-chosen): the dashboard's name hydration, notes
+/// tally and FIND reads. Multi-index brains fan out client-side (the SPA
+/// merges the per-index answers), so no multi-index merge semantics are
+/// invented here.
+pub async fn nodes_search(
+ State(state): State<ConsoleState>,
+ sess: AuthSession,
+ Path(brain): Path<String>,
+ Query(params): Query<NodesSearchParams>,
+ Json(body): Json<Value>,
+) -> Response {
+ if let Some(resp) = gate_brain(&sess.user, &brain) {
+ return resp;
+ }
+ let edges = edges_index(&brain);
+ if state.engine.get_index(&edges).is_err() {
+ return brain_not_found(&brain);
+ }
+ let nodes_index = resolve_nodes_index(&state.engine, &brain, &edges).await;
+ let names: Vec<&str> = nodes_index_names(&nodes_index).collect();
+ let target = match params.index.as_deref() {
+ Some(want) => {
+ if !names.contains(&want) {
+ return brain_not_found(&brain);
+ }
+ want
+ }
+ None => match names.as_slice() {
+ [only] => only,
+ _ => {
+ return graph_error(
+ StatusCode::BAD_REQUEST,
+ format!(
+ "this brain's nodes_index spans {} indices; pass ?index= with one of: {}",
+ names.len(),
+ names.join(", ")
+ ),
+ );
+ }
+ },
+ };
+ if state.engine.get_index(target).is_err() {
+ // A brain can hold edges before its nodes index is created — that is
+ // an honest empty answer, not a missing brain.
+ return es_wire_search_empty_ok();
+ }
+ es_wire_search(&state.engine, target, &body).await
+}
+
+/// An empty ES `_search` response for a nodes index that does not exist
+/// (edges asserted through the API alone): zero hits, honestly.
+fn es_wire_search_empty_ok() -> Response {
+ Json(json!({
+ "took": 0,
+ "timed_out": false,
+ "hits": {
+ "total": { "value": 0, "relation": "eq" },
+ "max_score": null,
+ "hits": [],
+ },
+ "aggregations": {},
+ }))
+ .into_response()
+}
diff --git a/engine/crates/xerj-console-api/src/lib.rs b/engine/crates/xerj-console-api/src/lib.rs
index aeb0cc09d..3874004ca 100644
--- a/engine/crates/xerj-console-api/src/lib.rs
+++ b/engine/crates/xerj-console-api/src/lib.rs
@@ -28,6 +28,9 @@
//! - [`views`] — saved views CRUD.
//! - [`data_sources`] — `/data-sources/connections` + descendants
//! (read-only MVP backed by an auto-provisioned `built-in` adapter).
+//! - [`graph`] — session-authorized Second-Brain graph reads (`/graph/*`,
+//! issue #936): brains discovery, `ego`, `overview`, brain-scoped edges and
+//! nodes searches.
//!
//! Coming after RC: `/auth/magic/issue`, `/auth/api-tokens`, `/users`
//! admin surface, `/cluster/raft*`, dashboards/views SSE streams, write
@@ -49,6 +52,7 @@ pub mod cluster;
pub mod dashboards;
pub mod data_sources;
pub mod error;
+pub mod graph;
pub mod indices;
pub mod prefs;
pub mod response;
diff --git a/engine/crates/xerj-console-api/src/router.rs b/engine/crates/xerj-console-api/src/router.rs
index a9d1a01d9..ccb2e5c68 100644
--- a/engine/crates/xerj-console-api/src/router.rs
+++ b/engine/crates/xerj-console-api/src/router.rs
@@ -30,7 +30,9 @@ use axum::{
Router,
};
-use crate::{auth, cluster, dashboards, data_sources, prefs, spa, state::ConsoleState, views};
+use crate::{
+ auth, cluster, dashboards, data_sources, graph, prefs, spa, state::ConsoleState, views,
+};
/// Build the full Xerj Console router. Mount at the root of an axum Router
/// (it owns the `/_xerj-console/*` prefix internally):
@@ -144,6 +146,24 @@ pub fn xerj_console_router(state: ConsoleState) -> Router {
"/_xerj-console/api/v1/data-sources/connections/:id/indices/:name/search",
post(data_sources::search),
)
+ // ── API: Second-Brain graph reads (session-authorized, issue #936) ─
+ // The data plane's /_graph/* routes stay credential-only; these are
+ // the console's own role-gated read paths over the same engine
+ // traversals. See src/graph.rs for the authorization rule.
+ .route("/_xerj-console/api/v1/graph/brains", get(graph::brains))
+ .route("/_xerj-console/api/v1/graph/:brain/ego", get(graph::ego))
+ .route(
+ "/_xerj-console/api/v1/graph/:brain/overview",
+ get(graph::overview),
+ )
+ .route(
+ "/_xerj-console/api/v1/graph/:brain/edges/_search",
+ post(graph::edges_search),
+ )
+ .route(
+ "/_xerj-console/api/v1/graph/:brain/nodes/_search",
+ post(graph::nodes_search),
+ )
.with_state(state)
}
@@ -178,5 +198,10 @@ pub fn known_routes() -> &'static [&'static str] {
"/_xerj-console/api/v1/data-sources/connections/:id/indices",
"/_xerj-console/api/v1/data-sources/connections/:id/indices/:name/fields",
"/_xerj-console/api/v1/data-sources/connections/:id/indices/:name/search",
+ "/_xerj-console/api/v1/graph/brains",
+ "/_xerj-console/api/v1/graph/:brain/ego",
+ "/_xerj-console/api/v1/graph/:brain/overview",
+ "/_xerj-console/api/v1/graph/:brain/edges/_search",
+ "/_xerj-console/api/v1/graph/:brain/nodes/_search",
]
}
diff --git a/engine/crates/xerj-console-api/tests/console_graph_read.rs b/engine/crates/xerj-console-api/tests/console_graph_read.rs
new file mode 100644
index 000000000..6ada4e62c
--- /dev/null
+++ b/engine/crates/xerj-console-api/tests/console_graph_read.rs
@@ -0,0 +1,567 @@
+//! Session-authorized console graph reads (issue #936).
+//!
+//! The console holds a passkey session, not an engine API key, and the
+//! data plane's `/_graph/{brain}/*` routes authenticate only
+//! Authorization-header credentials — so on an auth-enabled engine (the
+//! default) a signed-in operator could not read any brain while a
+//! share-link guest could. `xerj-console-api::graph` is the console's own
+//! role-gated read path. These tests pin its boundary:
+//!
+//! - the operator tier (`owner`/`admin`) reads brains through the console
+//! endpoints — listing, `ego` with node hydration, `overview`, the
+//! brain-pinned edges and nodes searches;
+//! - a disallowed role (`viewer`) and a nonexistent brain are
+//! **indistinguishable** on every per-brain endpoint (no existence
+//! oracle), and no brain content reaches a refused session;
+//! - the `brains` listing is a flat role gate (403, no per-brain fact);
+//! - no session at all is a plain 401 on every route.
+//!
+//! The data plane's own boundary (`/_graph/*` still requires an API key;
+//! the generic data-sources proxy still refuses the reserved namespace,
+//! RC10 B1) is pinned where it lives: `xerj-api`'s
+//! `brain_is_a_security_boundary.rs` and `console_cannot_read_a_brain.rs`.
+
+use axum::{body::Body, http::Request, Router};
+use http_body_util::BodyExt;
+use serde_json::{json, Value};
+use tempfile::TempDir;
+use tower::ServiceExt;
+use xerj_common::config::Config;
+use xerj_common::types::{FieldConfig, FieldType, Schema};
+use xerj_console_api::{
+ auth::{sessions, store},
+ state::ClusterMode,
+ xerj_console_router, ConsoleState,
+};
+use xerj_engine::Engine;
+
+/// Content that must never reach a session that may not read brains.
+const EDGE_SECRET: &str = "alice-private-fact-9f83c1";
+
+const T0: i64 = 1_753_600_000_000; // valid_at / created_at of the fixture edges
+const T_INVALID: i64 = 1_753_650_000_000; // invalid_at of the retired edge
+
+struct TestApp {
+ router: Router,
+ owner_cookie: String,
+ viewer_cookie: String,
+ _dir: TempDir,
+}
+
+/// The §2.1 edges-index schema (the subset the traversal and the hydrations
+/// read): keyword columns for the hop, numeric timestamps for the
+/// bi-temporal cut, keyword aggregates for `overview`.
+fn edges_schema() -> Schema {
+ let mut s = Schema::empty();
+ fn add(s: &mut Schema, name: &str, t: FieldType) {
+ s.add_field(FieldConfig::new(name, t)).expect("add field");
+ }
+ for f in ["edge_id", "src", "dst", "type", "detector"] {
+ add(&mut s, f, FieldType::Keyword);
+ }
+ for f in ["valid_at", "invalid_at", "created_at", "expired_at"] {
+ add(&mut s, f, FieldType::Date);
+ }
+ for f in ["weight", "confidence"] {
+ add(&mut s, f, FieldType::Double);
+ }
+ add(&mut s, "schema_version", FieldType::Long);
+ s
+}
+
+async fn add_session_user(state: &ConsoleState, engine: &Engine, id: &str, role: &str) -> String {
+ let user = store::User {
+ id: id.to_string(),
+ email: format!("{id}@example.com"),
+ display_name: id.to_string(),
+ role: role.to_string(),
+ status: store::UserStatus::Active,
+ created_at: xerj_console_api::time::now_iso(),
+ last_seen_at: Some(xerj_console_api::time::now_iso()),
+ };
+ store::upsert_user(engine, &user).await.unwrap();
+ let (_s, signed) = sessions::mint_session(state, &user.id, "passkey", None, None)
+ .await
+ .unwrap();
+ format!("xerj_session={signed}")
+}
+
+/// Boot the console over one brain, exactly as `xerj brain` would leave the
+/// node: an edges index with a §2.5 meta doc and three edges (one retired),
+/// plus the `ax-docs` nodes index the meta doc names.
+async fn boot_with_brain() -> TestApp {
+ let dir = TempDir::new().unwrap();
+ let mut cfg = Config::default();
+ cfg.server.data_dir = dir.path().to_str().unwrap().to_string();
+ let engine = Engine::new(cfg).expect("engine");
+ let outcome = xerj_console_api::bootstrap::run(&engine, dir.path(), "http://localhost:9200")
+ .await
+ .unwrap();
+ let state = ConsoleState::new(
+ engine.clone(),
+ "local".into(),
+ outcome.master_key,
+ ClusterMode::Standalone,
+ );
+
+ let owner_cookie = add_session_user(&state, &engine, "owner-test", "owner").await;
+ let viewer_cookie = add_session_user(&state, &engine, "viewer-test", "viewer").await;
+
+ engine
+ .create_index(".xerj-memory-casefile-edges", edges_schema())
+ .expect("create edges index");
+ let edges = engine.get_index(".xerj-memory-casefile-edges").unwrap();
+
+ // §2.5 meta doc: this brain's nodes index is the ordinary dataset index.
+ edges
+ .index_document(
+ Some("__xerj-brain-meta".into()),
+ json!({
+ "meta_version": 1,
+ "brain": "casefile",
+ "nodes_index": "ax-docs",
+ "created_at": T0,
+ }),
+ )
+ .await
+ .unwrap();
+
+ let edge = |id: &str, src: &str, dst: &str, t: &str, extra: Value| {
+ let mut doc = json!({
+ "edge_id": id,
+ "src": src,
+ "dst": dst,
+ "type": t,
+ "weight": 1.0,
+ "valid_at": T0,
+ "created_at": T0,
+ "detector": "same_dir@1",
+ "confidence": 1.0,
+ "schema_version": 1,
+ "evidence": { "quote": EDGE_SECRET, "source": "inbox/01.eml" },
+ });
+ if let Some(obj) = extra.as_object() {
+ for (k, v) in obj {
+ doc[k] = v.clone();
+ }
+ }
+ (Some(id.to_string()), doc)
+ };
+ edges
+ .index_documents_batched(vec![
+ edge("e1", "file-01", "file-02", "same_dir", json!({})),
+ edge("e2", "file-01", "file-03", "mdlink", json!({})),
+ edge(
+ "e3",
+ "file-02",
+ "file-03",
+ "pathcite",
+ json!({ "invalid_at": T_INVALID, "expired_at": T_INVALID }),
+ ),
+ ])
+ .await;
+
+ engine
+ .create_index("ax-docs", Schema::empty())
+ .expect("create nodes index");
+ let nodes = engine.get_index("ax-docs").unwrap();
+ nodes
+ .index_documents_batched(vec![
+ (
+ Some("file-01".into()),
+ json!({ "title": "01.eml", "text": "the first message", "ax_path": "inbox/01.eml" }),
+ ),
+ (
+ Some("file-02".into()),
+ json!({ "title": "02.eml", "text": "the second message", "ax_path": "inbox/02.eml" }),
+ ),
+ (
+ Some("file-03".into()),
+ json!({ "title": "03.eml", "text": "the third message", "ax_path": "inbox/03.eml" }),
+ ),
+ ])
+ .await;
+
+ let router = xerj_console_router(state);
+ TestApp {
+ router,
+ owner_cookie,
+ viewer_cookie,
+ _dir: dir,
+ }
+}
+
+async fn body_json(resp: axum::response::Response) -> (axum::http::StatusCode, Value, String) {
+ let status = resp.status();
+ let bytes = resp.into_body().collect().await.unwrap().to_bytes();
+ let text = String::from_utf8_lossy(&bytes).to_string();
+ let v = if bytes.is_empty() {
+ Value::Null
+ } else {
+ serde_json::from_slice(&bytes).unwrap_or(Value::Null)
+ };
+ (status, v, text)
+}
+
+fn req(method: &str, path: &str, cookie: Option<&str>, body: Option<Value>) -> Request<Body> {
+ let mut b = Request::builder().method(method).uri(path);
+ if let Some(c) = cookie {
+ b = b.header("cookie", c);
+ }
+ let body = match body {
+ Some(v) => {
+ b = b.header("content-type", "application/json");
+ Body::from(v.to_string())
+ }
+ None => Body::empty(),
+ };
+ b.body(body).unwrap()
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+
+/// The full product path the issue reports broken: the operator lists the
+/// node's brains, walks one node's neighborhood with hydrated titles, reads
+/// the brain-level stats, and the dashboard's edges/nodes searches answer.
+#[tokio::test]
+async fn owner_reads_the_graph() {
+ let app = boot_with_brain().await;
+
+ // Discovery: the listing replaces `_cat/indices/.xerj-memory-*` + the
+ // per-brain meta-doc reads, both of which 401 for a session.
+ let (status, body, _) = body_json(
+ app.router
+ .clone()
+ .oneshot(req(
+ "GET",
+ "/_xerj-console/api/v1/graph/brains",
+ Some(&app.owner_cookie),
+ None,
+ ))
+ .await
+ .unwrap(),
+ )
+ .await;
+ assert_eq!(status, 200, "brains listing: {body}");
+ let brains = body["data"]["brains"].as_array().expect("brains array");
+ assert_eq!(brains.len(), 1, "one brain: {body}");
+ assert_eq!(brains[0]["name"], "casefile");
+ assert_eq!(brains[0]["nodes_index"], "ax-docs");
+
+ // ego: 2 live edges at now (e3 is retired), neighbors, hydrated titles.
+ let (status, body, _) = body_json(
+ app.router
+ .clone()
+ .oneshot(req(
+ "GET",
+ "/_xerj-console/api/v1/graph/casefile/ego?node=file-01&hops=1&direction=both&include_nodes=true",
+ Some(&app.owner_cookie),
+ None,
+ ))
+ .await
+ .unwrap(),
+ )
+ .await;
+ assert_eq!(status, 200, "ego: {body}");
+ assert_eq!(body["brain"], "casefile");
+ assert_eq!(body["node"], "file-01", "single-seed keeps the `node` key");
+ let edges = body["edges"].as_array().expect("edges");
+ assert_eq!(edges.len(), 2, "the retired edge is excluded: {body}");
+ assert!(edges.iter().all(|e| e["invalid_at"].is_null()));
+ assert!(
+ body["nodes"]["file-02"]["title"] == "02.eml",
+ "node summaries hydrate: {body}"
+ );
+ assert_eq!(body["not_shown"]["dangling_nodes"], 0);
+
+ // include_expired brings the retirement back (the ledger's struck rows).
+ let (status, body, _) = body_json(
+ app.router
+ .clone()
+ .oneshot(req(
+ "GET",
+ "/_xerj-console/api/v1/graph/casefile/ego?node=file-02&include_expired=true",
+ Some(&app.owner_cookie),
+ None,
+ ))
+ .await
+ .unwrap(),
+ )
+ .await;
+ assert_eq!(status, 200);
+ assert_eq!(body["edges"].as_array().unwrap().len(), 2);
+
+ // overview: totals, live/invalidated split, hubs, notes total.
+ let (status, body, _) = body_json(
+ app.router
+ .clone()
+ .oneshot(req(
+ "GET",
+ "/_xerj-console/api/v1/graph/casefile/overview?top=10&histogram_interval=day",
+ Some(&app.owner_cookie),
+ None,
+ ))
+ .await
+ .unwrap(),
+ )
+ .await;
+ assert_eq!(status, 200, "overview: {body}");
+ assert_eq!(body["exists"], true);
+ assert_eq!(body["edges"]["total"], 3);
+ assert_eq!(body["edges"]["live"], 2);
+ assert_eq!(body["edges"]["invalidated"], 1);
+ assert_eq!(body["nodes"]["total"], 3, "notes counted on ax-docs");
+ assert_eq!(body["nodes_index"], "ax-docs");
+ assert_eq!(body["embedder"], "lexical-feature-hash");
+ assert!(
+ !body["hubs"]["out"].as_array().expect("hubs out").is_empty(),
+ "hub ids from the live slice"
+ );
+
+ // The dashboard's edges search (recent retirements) — brain-pinned.
+ let (status, body, _) = body_json(
+ app.router
+ .clone()
+ .oneshot(req(
+ "POST",
+ "/_xerj-console/api/v1/graph/casefile/edges/_search",
+ Some(&app.owner_cookie),
+ Some(json!({
+ "query": { "exists": { "field": "invalid_at" } },
+ "sort": [{ "invalid_at": { "order": "desc" } }],
+ "size": 3,
+ })),
+ ))
+ .await
+ .unwrap(),
+ )
+ .await;
+ assert_eq!(status, 200, "edges search: {body}");
+ let hits = body["hits"]["hits"].as_array().expect("hits");
+ assert_eq!(hits.len(), 1, "exactly the retired edge: {body}");
+ assert_eq!(hits[0]["_id"], "e3");
+
+ // The dashboard's nodes search (notes tally / FIND).
+ let (status, body, _) = body_json(
+ app.router
+ .clone()
+ .oneshot(req(
+ "POST",
+ "/_xerj-console/api/v1/graph/casefile/nodes/_search",
+ Some(&app.owner_cookie),
+ Some(json!({
+ "size": 0,
+ "track_total_hits": true,
+ "aggs": { "formats": { "terms": { "field": "ax_format", "size": 12 } } },
+ })),
+ ))
+ .await
+ .unwrap(),
+ )
+ .await;
+ assert_eq!(status, 200, "nodes search: {body}");
+ assert_eq!(body["hits"]["total"]["value"], 3);
+}
+
+/// A viewer's refusal and a nonexistent brain are byte-identical on every
+/// per-brain endpoint — the same no-existence-oracle discipline the search
+/// proxy pinned — and no brain content reaches the refused session.
+#[tokio::test]
+async fn refusal_is_no_existence_oracle() {
+ let app = boot_with_brain().await;
+ const GHOST: &str = "no-such-brain";
+
+ async fn viewer_call(
+ app: &TestApp,
+ method: &str,
+ brain: &str,
+ tail: &str,
+ body: Option<Value>,
+ ) -> (axum::http::StatusCode, Value, String) {
+ let path = format!("/_xerj-console/api/v1/graph/{brain}{tail}");
+ body_json(
+ app.router
+ .clone()
+ .oneshot(req(method, &path, Some(&app.viewer_cookie), body))
+ .await
+ .unwrap(),
+ )
+ .await
+ }
+
+ let cases: &[(&str, &str, &str)] = &[
+ ("GET", "/ego?node=file-01", ""),
+ ("GET", "/overview", ""),
+ ("POST", "/edges/_search", r#"{"query":{"match_all":{}}}"#),
+ ("POST", "/nodes/_search", r#"{"query":{"match_all":{}}}"#),
+ ];
+ for (method, tail, raw_body) in cases {
+ let body = if raw_body.is_empty() {
+ None
+ } else {
+ serde_json::from_str(raw_body).ok()
+ };
+ let (real_status, real_body, real_text) =
+ viewer_call(&app, method, "casefile", tail, body.clone()).await;
+ let (ghost_status, _ghost_body, ghost_text) =
+ viewer_call(&app, method, GHOST, tail, body).await;
+ assert_eq!(real_status, ghost_status, "{method} {tail}: status differs");
+ assert_eq!(real_status, 404, "{method} {tail}: refusal must 404");
+ assert_eq!(
+ real_text.replace("casefile", GHOST),
+ ghost_text,
+ "{method} {tail}: existing vs missing brain must be indistinguishable"
+ );
+ assert!(
+ !real_text.contains(EDGE_SECRET),
+ "{method} {tail}: brain content leaked to a viewer"
+ );
+ assert!(
+ real_body["error"]["type"] == "graph_error" || real_body["exists"] == false,
+ "{method} {tail}: unexpected refusal body {real_body}"
+ );
+ }
+
+ // The listing is a flat role gate: 403, and it names no brain.
+ let (status, body, text) = body_json(
+ app.router
+ .clone()
+ .oneshot(req(
+ "GET",
+ "/_xerj-console/api/v1/graph/brains",
+ Some(&app.viewer_cookie),
+ None,
+ ))
+ .await
+ .unwrap(),
+ )
+ .await;
+ assert_eq!(status, 403, "viewer listing: {body}");
+ assert!(
+ !text.contains("casefile"),
+ "listing leaked a brain name: {text}"
+ );
+}
+
+/// The admin role may read; the editor role may not (the rule is the
+/// operator tier, not one role name).
+#[tokio::test]
+async fn the_operator_tier_is_owner_and_admin() {
+ let user = |role: &str| store::User {
+ id: "u".into(),
+ email: "[email protected]".into(),
+ display_name: "u".into(),
+ role: role.into(),
+ status: store::UserStatus::Active,
+ created_at: xerj_console_api::time::now_iso(),
+ last_seen_at: None,
+ };
+ assert!(xerj_console_api::graph::may_read_brains(&user("owner")));
+ assert!(xerj_console_api::graph::may_read_brains(&user("admin")));
+ assert!(!xerj_console_api::graph::may_read_brains(&user("editor")));
+ assert!(!xerj_console_api::graph::may_read_brains(&user("viewer")));
+}
+
+/// No session at all is a plain 401 on every graph route (the AuthSession
+/// gate runs before anything brain-related).
+#[tokio::test]
+async fn no_session_is_unauthorized() {
+ let app = boot_with_brain().await;
+ let routes: &[(&str, &str, Option<Value>)] = &[
+ ("GET", "/_xerj-console/api/v1/graph/brains", None),
+ (
+ "GET",
+ "/_xerj-console/api/v1/graph/casefile/ego?node=file-01",
+ None,
+ ),
+ ("GET", "/_xerj-console/api/v1/graph/casefile/overview", None),
+ (
+ "POST",
+ "/_xerj-console/api/v1/graph/casefile/edges/_search",
+ Some(json!({"query":{"match_all":{}}})),
+ ),
+ (
+ "POST",
+ "/_xerj-console/api/v1/graph/casefile/nodes/_search",
+ Some(json!({"query":{"match_all":{}}})),
+ ),
+ ];
+ for (method, path, body) in routes {
+ let (status, _, _) = body_json(
+ app.router
+ .clone()
+ .oneshot(req(method, path, None, body.clone()))
+ .await
+ .unwrap(),
+ )
+ .await;
+ assert_eq!(status, 401, "{method} {path} must 401 without a session");
+ }
+}
+
+/// A malformed brain name is a 400 for every role — validation leaks no
+/// existence fact and precedes the role gate (same order as the data plane).
+#[tokio::test]
+async fn malformed_brain_name_is_400_for_every_role() {
+ let app = boot_with_brain().await;
+ for cookie in [&app.owner_cookie, &app.viewer_cookie] {
+ let (status, body, _) = body_json(
+ app.router
+ .clone()
+ .oneshot(req(
+ "GET",
+ "/_xerj-console/api/v1/graph/BAD_BRAIN/ego?node=x",
+ Some(cookie),
+ None,
+ ))
+ .await
+ .unwrap(),
+ )
+ .await;
+ assert_eq!(status, 400, "malformed brain: {body}");
+ }
+}
+
+/// `ego?nodes_index=` is refused: on the data plane the override is
+/// authorized against the caller's grants; a console session holds none, and
+/// silently ignoring it would change semantics between the two surfaces.
+#[tokio::test]
+async fn ego_refuses_the_nodes_index_override() {
+ let app = boot_with_brain().await;
+ let (status, body, _) = body_json(
+ app.router
+ .clone()
+ .oneshot(req(
+ "GET",
+ "/_xerj-console/api/v1/graph/casefile/ego?node=file-01&nodes_index=ax-docs",
+ Some(&app.owner_cookie),
+ None,
+ ))
+ .await
+ .unwrap(),
+ )
+ .await;
+ assert_eq!(status, 400, "nodes_index override: {body}");
+}
+
+/// The nodes search is pinned to the brain's own meta-doc set: another index
+/// on the node — even one that exists — is the same 404 an unknown brain
+/// gets, so the endpoint cannot be turned into a general search proxy.
+#[tokio::test]
+async fn nodes_search_is_pinned_to_the_brains_indices() {
+ let app = boot_with_brain().await;
+ let (status, _, _) = body_json(
+ app.router
+ .clone()
+ .oneshot(req(
+ "POST",
+ "/_xerj-console/api/v1/graph/casefile/nodes/_search?index=other-index",
+ Some(&app.owner_cookie),
+ Some(json!({ "query": { "match_all": {} } })),
+ ))
+ .await
+ .unwrap(),
+ )
+ .await;
+ assert_eq!(status, 404, "a foreign index name must 404");
+}
diff --git a/xerj-ux/src/dashboards/second-brain.js b/xerj-ux/src/dashboards/second-brain.js
index 9230546a9..d74d04f3d 100644
--- a/xerj-ux/src/dashboards/second-brain.js
+++ b/xerj-ux/src/dashboards/second-brain.js
@@ -23,8 +23,9 @@
// the ledger.
//
// LIVE DATA ONLY. data/backends/xerj.js routes this dashboard to
-// data/second-brain-api.js, which reads /_graph/{brain}/overview and
-// /_graph/{brain}/ego and NEVER falls back to mock — an empty or
+// data/second-brain-api.js, which reads the console's own session-authorized
+// graph endpoints (`/_xerj-console/api/v1/graph/{brain}/overview` and
+// `…/{brain}/ego`, issue #936) and NEVER falls back to mock — an empty or
// unreachable brain renders an honest state that says how to fill it
// (`xerj brain ~/your-folder`), not invented beliefs.
// ============================================================
diff --git a/xerj-ux/src/data/brains-probe.js b/xerj-ux/src/data/brains-probe.js
index be37c63bc..eabf6b10d 100644
--- a/xerj-ux/src/data/brains-probe.js
+++ b/xerj-ux/src/data/brains-probe.js
@@ -2,23 +2,25 @@
// Xerj Console — "does at least one brain exist?" probe
//
// The Second Brain dashboard only earns a nav entry once the engine
-// actually holds a brain — a console pointed at an engine nobody ever
-// ran `xerj brain <folder>` against should not advertise an empty
-// dashboard. This module answers exactly that yes/no question.
+// actually holds a brain — a console pointed at an engine nobody ever ran
+// `xerj brain <folder>` against should not advertise an empty dashboard.
+// This module answers exactly that yes/no question.
//
-// It wraps the same discovery pattern second-brain-api.js uses
-// (`_cat/indices/.xerj-memory-*` — PREFIX wildcard only: this
-// engine's index-pattern matcher silently returns empty for an infix
-// wildcard like `.xerj-memory-*-edges`, live-verified 2026-07-30, so
-// the `-edges` cut happens client-side).
+// It asks the console's own brains listing
+// (`/_xerj-console/api/v1/graph/brains`), authenticated by the console
+// session. The old direct `_cat/indices/.xerj-memory-*` call 401s a session
+// on an auth-enabled engine (the default), which kept the nav entry hidden
+// for a signed-in operator exactly when the dashboard finally had a
+// session-authorized way to read the graph (issue #936).
//
-// Honesty stance: `false` on ANY transport failure. An unreachable
-// engine means we cannot claim a brain exists, so the nav entry stays
-// hidden — the deep-link route still resolves (app.js never filters
-// the route table, only the nav list), so nothing is lost.
+// Honesty stance: `false` on ANY transport failure. An unreachable engine
+// — or a role that may not read brains (403) — means we cannot claim a
+// brain exists, so the nav entry stays hidden; the deep-link route still
+// resolves (app.js never filters the route table, only the nav list), so
+// nothing is lost.
//
// Results are cached per baseUrl with a short TTL, so:
-// - the boot probe + periodic re-probes don't hammer `_cat`,
+// - the boot probe + periodic re-probes don't hammer the listing,
// - switching backend/base-URL naturally misses the cache and
// re-probes the new target without an explicit invalidation hook.
// ============================================================
@@ -35,11 +37,12 @@ export function invalidateBrainsProbe() {
}
/**
- * True iff the engine at `baseUrl` holds at least one brain
- * (a reserved `.xerj-memory-{brain}-edges` index).
+ * True iff this engine holds at least one brain (a reserved
+ * `.xerj-memory-{brain}-edges` index) that the console session may read.
*
- * Never throws; returns false on transport failure, non-OK status,
- * or an empty/absent baseUrl.
+ * Never throws; returns false on transport failure, non-OK status, or an
+ * empty/absent baseUrl. The graph listing is same-origin (the console
+ * serves the SPA), so `baseUrl` only keys the cache.
*/
export async function sbBrainsPresent(baseUrl, signal) {
const base = (baseUrl || '').replace(/\/+$/, '');
@@ -50,18 +53,16 @@ export async function sbBrainsPresent(baseUrl, signal) {
let present = false;
try {
- const r = await fetch(`${base}/_cat/indices/.xerj-memory-*`, {
+ const r = await fetch('/_xerj-console/api/v1/graph/brains', {
signal,
- headers: { accept: 'text/plain, application/json' },
+ credentials: 'same-origin',
+ headers: { accept: 'application/json' },
});
- // A wildcard matching nothing is 200 with an EMPTY body on this
- // engine (live-verified; only a concrete name 404s), which parses
- // to zero brains below. Any non-OK status (404 from an older build
- // without the pattern route, 5xx, auth) is treated as "cannot
- // prove a brain exists", so we do not claim one.
+ // 403 = this console role may not read brains; 401 = no session. Both
+ // are "cannot prove a brain exists", so we do not claim one.
if (r.ok) {
- const text = await r.text();
- present = parseBrainIndices(text).length > 0;
+ const j = await r.json();
+ present = ((j && j.data && j.data.brains) || []).length > 0;
}
} catch {
present = false; // engine down / CORS / abort — no claim
@@ -70,25 +71,3 @@ export async function sbBrainsPresent(baseUrl, signal) {
cache.set(base, { at: Date.now(), value: present });
return present;
}
-
-/**
- * Parse a `_cat/indices` body into the list of brain names. `_cat` on
- * this engine emits plain text lines (`health status NAME uuid …`);
- * tolerate a JSON array in case a later build adds format=json.
- * Exported for the node self-test.
- */
-export function parseBrainIndices(text) {
- const trimmed = (text || '').trim();
- let names = [];
- if (trimmed.startsWith('[')) {
- try { names = JSON.parse(trimmed).map((i) => i.index).filter(Boolean); } catch { names = []; }
- } else if (trimmed) {
- names = trimmed.split('\n')
- .map((l) => l.trim().split(/\s+/)[2])
- .filter(Boolean);
- }
- return names
- .filter((n) => n.startsWith('.xerj-memory-') && n.endsWith('-edges'))
- .map((n) => n.slice('.xerj-memory-'.length, -'-edges'.length))
- .filter((b) => b.length > 0);
-}
diff --git a/xerj-ux/src/data/second-brain-api.js b/xerj-ux/src/data/second-brain-api.js
index 819af641e..5cf562e57 100644
--- a/xerj-ux/src/data/second-brain-api.js
+++ b/xerj-ux/src/data/second-brain-api.js
@@ -1,11 +1,25 @@
// ============================================================
// Xerj Console — SECOND BRAIN data layer + interaction controller
//
-// This dashboard reads the LIVE graph endpoints ONLY:
+// This dashboard reads the LIVE graph through the console's own
+// session-authorized endpoints (issue #936) — same-origin, session
+// cookie, no engine key. The old direct data-plane calls
+// (`/_cat/indices/.xerj-memory-*`, `/_graph/{brain}/ego|overview`,
+// `/{edges}/_search`, `/{nodes}/_search`) 401 a console session on an
+// auth-enabled engine, which is the default posture:
//
-// GET /_cat/indices/.xerj-memory-*-edges → discover brains
-// GET /_graph/{brain}/overview → §4.4 of the contract
-// GET /_graph/{brain}/ego → §4.3 of the contract
+// GET /_xerj-console/api/v1/graph/brains → discover brains
+// GET /_xerj-console/api/v1/graph/{brain}/overview → §4.4 of the contract
+// GET /_xerj-console/api/v1/graph/{brain}/ego → §4.3 of the contract
+// POST /_xerj-console/api/v1/graph/{brain}/edges/_search → recent
+// retirements, file-type crossings (the brain's links index)
+// POST /_xerj-console/api/v1/graph/{brain}/nodes/_search → note
+// tallies, name hydration, FIND (ONE of the brain's own indices)
+//
+// The one remaining direct call is `/v1/metrics` (the searches-per-index
+// tile): the metrics surface stays a data-plane, engine-key read by
+// decision, and the tile shows the refusal honestly rather than an
+// invented number.
//
// There is deliberately NO mock fallback: a fake brain would defeat
// the product ("what does my agent believe?" answered with invented
@@ -209,12 +223,12 @@ function saveBrainState() {
// ----- HTTP ---------------------------------------------------------
async function getJson(url, signal) {
- const r = await fetch(url, { signal, headers: { accept: 'application/json' } });
+ const r = await fetch(url, { signal, credentials: 'same-origin', headers: { accept: 'application/json' } });
const text = await r.text();
let body = null;
try { body = text ? JSON.parse(text) : null; } catch { /* non-JSON error page */ }
if (!r.ok) {
- // The graph API 404s an unknown brain WITH a shaped body
+ // The graph read path 404s an unknown brain WITH a shaped body
// (`exists: false`) — that is a valid answer, not an error.
if (body && body.exists === false) return body;
const reason = body && body.error && body.error.reason
@@ -225,43 +239,72 @@ async function getJson(url, signal) {
return body;
}
+/** The console's graph read path — same-origin, session-authenticated
+ * (see xerj-console-api/src/graph.rs). The `baseUrl` the backend passes
+ * is deliberately unused as a request target: these endpoints are served
+ * by the same Console instance that served the SPA, exactly like the
+ * panel-search proxy (data/backends/xerj.js#rawSearch). */
+const GRAPH = '/_xerj-console/api/v1/graph';
+
+const enc = encodeURIComponent;
+
/**
- * Brains = the reserved `.xerj-memory-{brain}-edges` indices. `_cat`
- * on this engine emits plain text (no format=json), so parse lines;
- * tolerate a JSON array in case a later build adds it. A wildcard
- * matching nothing is an empty 200 — an engine with no brains is a
- * normal state, not an error.
- *
- * The fetch pattern is `.xerj-memory-*` (prefix only): live-verified
- * 2026-07-30 that this engine's index-pattern matcher supports a
- * single leading or trailing `*` but NOT an infix wildcard —
- * `.xerj-memory-*-edges` returns an empty 200 even when the index
- * exists. The `-edges` cut happens in the client filter below either
- * way, so the result set is identical.
+ * Brains = what the console's brains listing returns (the reserved
+ * `.xerj-memory-{brain}-edges` indices this session's role may read,
+ * each with its meta-doc `nodes_index`). An empty listing is a normal
+ * state (no brains / a role that may not read them), not an error; a
+ * transport failure is.
*/
-async function discoverBrains(baseUrl, signal) {
- const r = await fetch(`${baseUrl}/_cat/indices/.xerj-memory-*`, { signal });
- if (r.status === 404) return [];
- if (!r.ok) throw new Error(`_cat/indices HTTP ${r.status}`);
- const text = await r.text();
- let names = [];
- const trimmed = text.trim();
- if (trimmed.startsWith('[')) {
- try { names = JSON.parse(trimmed).map((i) => i.index).filter(Boolean); } catch { names = []; }
- } else if (trimmed) {
- // cat line: `health status NAME uuid pri rep docs deleted size size`
- names = trimmed.split('\n')
- .map((l) => l.trim().split(/\s+/)[2])
- .filter(Boolean);
- }
- return names
- .filter((n) => n.startsWith('.xerj-memory-') && n.endsWith('-edges'))
- .map((n) => n.slice('.xerj-memory-'.length, -'-edges'.length))
- .filter((b) => b.length > 0)
+async function discoverBrains(_baseUrl, signal) {
+ const r = await fetch(`${GRAPH}/brains`, { signal, credentials: 'same-origin', headers: { accept: 'application/json' } });
+ if (r.status === 401 || r.status === 403 || r.status === 404) return [];
+ if (!r.ok) throw new Error(`graph/brains HTTP ${r.status}`);
+ const j = await r.json();
+ return (((j && j.data && j.data.brains) || [])
+ .map((b) => (b && typeof b.name === 'string' ? b.name : null)))
+ .filter(Boolean)
.sort();
}
-const enc = encodeURIComponent;
+/** The concrete nodes indices this brain reads — the meta doc's
+ * `nodes_index`, comma-joined when `xerj brain` indexed a multi-dataset
+ * folder ("ax-mail,ax-pdfs"). */
+function nodesNames() {
+ const o = S.overview;
+ if (!o || o.exists === false || !o.nodes_index) return [];
+ return String(o.nodes_index).split(',').map((x) => x.trim()).filter(Boolean);
+}
+
+/** One ES `_search` against ONE of the brain's own nodes indices, through
+ * the console's brain-scoped endpoint (the index set is pinned server-side
+ * to the meta doc's names — this can never name a foreign index). */
+async function nodesSearch(index, body, signal) {
+ const names = nodesNames();
+ const qs = names.length > 1 ? `?index=${enc(index)}` : '';
+ const r = await fetch(`${GRAPH}/${enc(S.brain)}/nodes/_search${qs}`, {
+ method: 'POST',
+ credentials: 'same-origin',
+ headers: { 'content-type': 'application/json' },
+ body: JSON.stringify(body),
+ signal,
+ });
+ if (!r.ok) throw new Error(`nodes search HTTP ${r.status}`);
+ return r.json();
+}
+
+/** One ES `_search` against this brain's links index (edges), through the
+ * console's brain-scoped endpoint. */
+async function edgesSearch(body, signal) {
+ const r = await fetch(`${GRAPH}/${enc(S.brain)}/edges/_search`, {
+ method: 'POST',
+ credentials: 'same-origin',
+ headers: { 'content-type': 'application/json' },
+ body: JSON.stringify(body),
+ signal,
+ });
+ if (!r.ok) throw new Error(`links search HTTP ${r.status}`);
+ return r.json();
+}
/**
* Hub lists and focus crumbs come back from the overview as raw node
@@ -271,9 +314,10 @@ const enc = encodeURIComponent;
* resolve keeps rendering as its (shortened) id, never as a guess.
*/
async function hydrateNames(signal) {
- const o = S.overview;
- if (!o || o.exists === false || !o.nodes_index) return;
+ const names = nodesNames();
+ if (!names.length) return;
const ids = new Set();
+ const o = S.overview;
for (const h of (o.hubs && o.hubs.in) || []) ids.add(h.id);
for (const h of (o.hubs && o.hubs.out) || []) ids.add(h.id);
for (const t of S.trail) ids.add(t);
@@ -283,31 +327,29 @@ async function hydrateNames(signal) {
if (!want.length) return;
try {
logRead('notes index', `${want.length} note name${want.length === 1 ? '' : 's'}`, 'turning ids into titles');
- const r = await fetch(`${S.baseUrl}/${enc(o.nodes_index)}/_search`, {
- method: 'POST',
- headers: { 'content-type': 'application/json' },
- body: JSON.stringify({
- query: { ids: { values: want } },
- size: want.length,
- // ax_path/path: the real file behind a note — a truthful label
- // for the focus card and the map, never derived from the id.
- _source: ['title', 'text', 'body', 'ax_path', 'path'],
- }),
- signal,
- });
- if (!r.ok) return; // cosmetic enrichment only — ids still render
- const j = await r.json();
- for (const h of (j.hits && j.hits.hits) || []) {
- const src = h._source || {};
- const title = typeof src.title === 'string' && src.title.trim()
- ? src.title.trim()
- : (typeof src.text === 'string' && src.text.trim()
- ? src.text.trim().slice(0, 60)
- : (typeof src.body === 'string' && src.body.trim() ? src.body.trim().slice(0, 60) : null));
- const path = (typeof src.ax_path === 'string' && src.ax_path.trim())
- ? src.ax_path.trim()
- : (typeof src.path === 'string' && src.path.trim() ? src.path.trim() : null);
- if (title || path) S.names[h._id] = { title, path };
+ const body = {
+ query: { ids: { values: want } },
+ size: want.length,
+ // ax_path/path: the real file behind a note — a truthful label
+ // for the focus card and the map, never derived from the id.
+ _source: ['title', 'text', 'body', 'ax_path', 'path'],
+ };
+ for (const name of names) {
+ // A failed hydration is cosmetic — ids still render — so per-index
+ // failures fall through to the shared catch below.
+ const j = await nodesSearch(name, body, signal);
+ for (const h of (j.hits && j.hits.hits) || []) {
+ const src = h._source || {};
+ const title = typeof src.title === 'string' && src.title.trim()
+ ? src.title.trim()
+ : (typeof src.text === 'string' && src.text.trim()
+ ? src.text.trim().slice(0, 60)
+ : (typeof src.body === 'string' && src.body.trim() ? src.body.trim().slice(0, 60) : null));
+ const path = (typeof src.ax_path === 'string' && src.ax_path.trim())
+ ? src.ax_path.trim()
+ : (typeof src.path === 'string' && src.path.trim() ? src.path.trim() : null);
+ if (title || path) S.names[h._id] = { title, path };
+ }
}
} catch { /* names are sugar; the ids remain the truth */ }
}
@@ -316,7 +358,7 @@ function fetchOverview(signal) {
const qs = new URLSearchParams({ top: '10', histogram_interval: 'day' });
if (S.asOf != null) qs.set('as_of', String(S.asOf));
logRead('link counts', 'one overview read', 'the tiles on this page');
- return getJson(`${S.baseUrl}/_graph/${enc(S.brain)}/overview?${qs}`, signal);
+ return getJson(`${GRAPH}/${enc(S.brain)}/overview?${qs}`, signal);
}
/** The focus note's display name for the read log — hydrated title if
@@ -341,19 +383,12 @@ async function fetchRecentRetired(signal) {
if (!inval) { S.recentRetired = []; return; }
try {
logRead('links index', 'three newest retirements', 'the retired tile');
- const r = await fetch(`${S.baseUrl}/${enc(`.xerj-memory-${S.brain}-edges`)}/_search`, {
- method: 'POST',
- headers: { 'content-type': 'application/json' },
- body: JSON.stringify({
- query: { exists: { field: 'invalid_at' } },
- sort: [{ invalid_at: { order: 'desc' } }],
- size: 3,
- _source: ['src', 'dst', 'type', 'detector', 'evidence', 'valid_at', 'invalid_at'],
- }),
- signal,
- });
- if (!r.ok) { S.recentRetired = null; return; }
- const j = await r.json();
+ const j = await edgesSearch({
+ query: { exists: { field: 'invalid_at' } },
+ sort: [{ invalid_at: { order: 'desc' } }],
+ size: 3,
+ _source: ['src', 'dst', 'type', 'detector', 'evidence', 'valid_at', 'invalid_at'],
+ }, signal);
S.recentRetired = (((j.hits && j.hits.hits) || []))
.map((h) => ({ edge_id: h._id, ...(h._source || {}) }));
} catch { S.recentRetired = null; }
@@ -373,7 +408,7 @@ function fetchBelief(signal) {
});
if (S.asOf != null) qs.set('as_of', String(S.asOf));
logRead('the ledger', `one 2-hop walk from “${focusLabel()}”`, 'you focused this note');
- return getJson(`${S.baseUrl}/_graph/${enc(S.brain)}/ego?${qs}`, signal);
+ return getJson(`${GRAPH}/${enc(S.brain)}/ego?${qs}`, signal);
}
/**
@@ -388,7 +423,7 @@ function fetchTimeline(signal) {
include_expired: 'true', include_evidence: 'false',
});
logRead('belief strip', `one 1-hop walk from “${focusLabel()}”`, 'drawing every link’s lifetime');
- return getJson(`${S.baseUrl}/_graph/${enc(S.brain)}/ego?${qs}`, signal);
+ return getJson(`${GRAPH}/${enc(S.brain)}/ego?${qs}`, signal);
}
// ----- statistics-row fetches (all best-effort, never throw) --------
@@ -400,35 +435,44 @@ function fetchTimeline(signal) {
* only when this tally failed (see NotesPanel's rationale).
*/
async function fetchNodeStats(signal) {
- const o = S.overview;
- if (!o || o.exists === false || !o.nodes_index) { S.nodeStats = null; return; }
+ const names = nodesNames();
+ if (!names.length) { S.nodeStats = null; return; }
try {
logRead('notes index', 'one tally by file type', 'the notes tile');
- const r = await fetch(`${S.baseUrl}/${enc(o.nodes_index)}/_search`, {
- method: 'POST',
- headers: { 'content-type': 'application/json' },
- body: JSON.stringify({
- size: 0,
- // Exact, not the engine's 10,000 'gte' floor: the tile's
- // headline is the note count, and this brain's own per-type
- // rows summed to 2.1M under a "10,000 NOTES" headline before
- // this flag (live-caught on `repo`, 2026-07-30).
- track_total_hits: true,
- aggs: { formats: { terms: { field: 'ax_format', size: 12 } } },
- }),
- signal,
- });
- if (!r.ok) { S.nodeStats = { error: `HTTP ${r.status}` }; return; }
- const j = await r.json();
- const t = j.hits && j.hits.total;
- let total = t == null ? null : (typeof t === 'object' ? t.value : t);
- // A 'gte' floor is not a count — refuse it rather than print it.
- if (t && typeof t === 'object' && t.relation === 'gte') total = null;
- const agg = (j.aggregations && j.aggregations.formats) || {};
+ const body = {
+ size: 0,
+ // Exact, not the engine's 10,000 'gte' floor: the tile's
+ // headline is the note count, and this brain's own per-type
+ // rows summed to 2.1M under a "10,000 NOTES" headline before
+ // this flag (live-caught on `repo`, 2026-07-30).
+ track_total_hits: true,
+ aggs: { formats: { terms: { field: 'ax_format', size: 12 } } },
+ };
+ // A multi-dataset brain reads each of its nodes indices and merges
+ // client-side (the console endpoint is pinned to ONE index per call).
+ let total = 0;
+ let exact = true;
+ const byFormat = new Map();
+ let otherFormats = 0;
+ for (const name of names) {
+ const j = await nodesSearch(name, body, signal);
+ const t = j.hits && j.hits.total;
+ let v = t == null ? null : (typeof t === 'object' ? t.value : t);
+ // A 'gte' floor is not a count — refuse it rather than print it.
+ if (t && typeof t === 'object' && t.relation === 'gte') v = null;
+ if (Number.isFinite(v)) total += v; else exact = false;
+ const agg = (j.aggregations && j.aggregations.formats) || {};
+ for (const b of agg.buckets || []) {
+ byFormat.set(b.key, (byFormat.get(b.key) || 0) + b.doc_count);
+ }
+ otherFormats += agg.sum_other_doc_count || 0;
+ }
S.nodeStats = {
- total: Number.isFinite(total) ? total : null,
- formats: (agg.buckets || []).map((b) => ({ format: b.key, count: b.doc_count })),
- otherFormats: agg.sum_other_doc_count || 0,
+ total: exact ? total : null,
+ formats: [...byFormat.entries()]
+ .map(([format, count]) => ({ format, count }))
+ .sort((a, b) => b.count - a.count || (a.format < b.format ? -1 : a.format > b.format ? 1 : 0)),
+ otherFormats,
};
} catch (e) {
if (e && e.name === 'AbortError') return;
@@ -446,27 +490,20 @@ async function fetchCrossings(signal) {
if (!S.brain) { S.crossings = null; return; }
try {
logRead('links index', 'one file-type crossing tally', 'the crossings tile');
- const r = await fetch(`${S.baseUrl}/${enc(`.xerj-memory-${S.brain}-edges`)}/_search`, {
- method: 'POST',
- headers: { 'content-type': 'application/json' },
- body: JSON.stringify({
- size: 0,
- // Exact, not the engine's 10,000 'gte' floor — the tile prints
- // this as "N stamped links tallied" and a floor there is a
- // false number (live-caught at 20,930 links, 2026-07-30).
- track_total_hits: true,
- query: { exists: { field: 'src_format' } },
- aggs: {
- src: {
- terms: { field: 'src_format', size: 20 },
- aggs: { dst: { terms: { field: 'dst_format', size: 20 } } },
- },
+ const j = await edgesSearch({
+ size: 0,
+ // Exact, not the engine's 10,000 'gte' floor — the tile prints
+ // this as "N stamped links tallied" and a floor there is a
+ // false number (live-caught at 20,930 links, 2026-07-30).
+ track_total_hits: true,
+ query: { exists: { field: 'src_format' } },
+ aggs: {
+ src: {
+ terms: { field: 'src_format', size: 20 },
+ aggs: { dst: { terms: { field: 'dst_format', size: 20 } } },
},
- }),
- signal,
- });
- if (!r.ok) { S.crossings = { error: `HTTP ${r.status}` }; return; }
- const j = await r.json();
+ },
+ }, signal);
const t = j.hits && j.hits.total;
const total = (t == null ? 0 : (typeof t === 'object' ? t.value : t)) || 0;
const srcAgg = (j.aggregations && j.aggregations.src) || {};
@@ -593,7 +630,7 @@ export async function liveSecondBrain(baseUrl, _ctx, signal) {
S.baseUrl = (baseUrl || '').replace(/\/+$/, '');
S.uiError = null;
try {
- logRead('brain list', 'one index listing', 'finding your brains');
+ logRead('brain list', 'one brains listing', 'finding your brains');
S.brains = await discoverBrains(S.baseUrl, signal);
S.connected = true;
S.error = null;
@@ -1300,40 +1337,41 @@ async function runFind(q) {
const slot = document.querySelector('[data-sb-findresults]');
if (!slot) return;
if (!q) { slot.hidden = true; slot.innerHTML = ''; return; }
- const o = S.overview;
- if (!o || o.exists === false || !o.nodes_index) return;
+ const names = nodesNames();
+ if (!names.length) return;
const seq = ++findSeq;
try {
logRead('notes index', `one text search “${q.slice(0, 24)}${q.length > 24 ? '…' : ''}”`, 'you typed in FIND');
- const r = await fetch(`${S.baseUrl}/${enc(o.nodes_index)}/_search`, {
- method: 'POST',
- headers: { 'content-type': 'application/json' },
- body: JSON.stringify({
- query: {
- bool: {
- should: [
- { match: { title: q } },
- { match: { text: q } },
- { match: { body: q } },
- ],
- },
+ const body = {
+ query: {
+ bool: {
+ should: [
+ { match: { title: q } },
+ { match: { text: q } },
+ { match: { body: q } },
+ ],
},
- size: 7,
- _source: ['title'],
- }),
- });
- if (seq !== findSeq) return; // a newer keystroke owns the slot
- if (!r.ok) {
- slot.innerHTML = `<span class="sb-sidenote mono faint">FIND FAILED · HTTP ${r.status}</span>`;
- slot.hidden = false;
- return;
+ },
+ size: 7,
+ _source: ['title'],
+ };
+ // A multi-dataset brain queries each of its nodes indices and merges
+ // by score (the same order a multi-index ES search would return).
+ const found = [];
+ for (const name of names) {
+ const j = await nodesSearch(name, body, undefined);
+ if (seq !== findSeq) return; // a newer keystroke owns the slot
+ for (const h of (j.hits && j.hits.hits) || []) {
+ found.push({
+ id: h._id,
+ score: typeof h._score === 'number' ? h._score : 0,
+ title: (h._source && typeof h._source.title === 'string' && h._source.title.trim())
+ ? h._source.title.trim() : null,
+ });
+ }
}
- const j = await r.json();
- const hits = ((j.hits && j.hits.hits) || []).map((h) => ({
- id: h._id,
- title: (h._source && typeof h._source.title === 'string' && h._source.title.trim())
- ? h._source.title.trim() : null,
- }));
+ found.sort((a, b) => b.score - a.score);
+ const hits = found.slice(0, 7).map(({ id, title }) => ({ id, title }));
slot.innerHTML = FindResults({ hits, names: S.names });
slot.hidden = false;
} catch (e) {
diff --git a/xerj-ux/src/data/transport-console.js b/xerj-ux/src/data/transport-console.js
index d65a97ef7..d1f6d62bd 100644
--- a/xerj-ux/src/data/transport-console.js
+++ b/xerj-ux/src/data/transport-console.js
@@ -2,35 +2,37 @@
// XERJ Console — reader transport for the signed-in operator
//
// A console session is NOT an engine credential. Everything the reader can
-// get through the session-authenticated console API, it gets there, so it
-// works the same on an auth-enabled engine (the default) as on `--insecure`:
+// get, it gets through the session-authenticated console API, so it works
+// the same on an auth-enabled engine (the default) as on `--insecure`:
//
// search the panel proxy (one exact index; the proxy refuses patterns,
// system indices and the reserved brain namespace — see
// xerj-console-api/src/data_sources.rs).
// mapping the console's field list for the index, reshaped to the
// `_mapping` wire form data/reader-api.js reads.
+// ego the console's graph read path
+// (`/_xerj-console/api/v1/graph/{brain}/ego`) — session-authorized
+// for the operator roles (owner/admin), same §4.3 contract as the
+// data plane (xerj-console-api/src/graph.rs, issue #936).
+// discoverBrains
+// one call to the console's brains listing
+// (`/_xerj-console/api/v1/graph/brains`), which returns each
+// brain's `nodes_index` — this replaces the old direct
+// `_cat/indices/.xerj-memory-*` walk plus per-brain meta-doc
+// reads, both of which 401 for a session.
//
-// The graph has no such path, on purpose (the proxy cannot read a brain —
-// RC10 B1), so these two are direct same-origin calls:
-//
-// ego `GET /_graph/{brain}/ego`
-// discoverBrains `_cat/indices` + each brain's meta document
-//
-// On an auth-enabled engine they answer 401. The reader then says exactly
-// that — "the graph API refused this console session" — instead of claiming
-// the record has no links. Records, search and attachments still work.
+// A role that may not read brains (editor/viewer) gets the same 404 an
+// unknown brain gets, so the reader says "no brain for this index" — the
+// honest user-facing statement for that role — and never a leak.
// ============================================================
-import { parseBrainIndices } from './brains-probe.js';
import { fieldTypes } from './console-index-api.js';
const PROXY = '/_xerj-console/api/v1/data-sources/connections/built-in/indices';
-const BRAIN_META_ID = '__xerj-brain-meta';
-/** Brains scanned for a meta document, and how many matching brains a record's
- * graph panel walks. Two `xerj brain` runs over different folders land in
- * the same `ax-docs` index with two brains that both list it. */
-const BRAINS_SCANNED = 32;
+const GRAPH = '/_xerj-console/api/v1/graph';
+/** How many of the discovered brains a record's graph panel walks. Two
+ * `xerj brain` runs over different folders land in the same `ax-docs`
+ * index with two brains that both list it. */
export const BRAINS_WALKED = 4;
const enc = encodeURIComponent;
@@ -80,33 +82,31 @@ export function makeConsoleTransport() {
},
async ego(brain, params, signal) {
const qs = new URLSearchParams(params);
- const r = await request(`/_graph/${enc(brain)}/ego?${qs}`, { signal, credentials: 'same-origin', headers: { accept: 'application/json' } });
+ const r = await request(`${GRAPH}/${enc(brain)}/ego?${qs}`, { signal, credentials: 'same-origin', headers: { accept: 'application/json' } });
let body = null;
try { body = await r.json(); } catch { body = null; }
return { status: r.status, body };
},
/**
* EVERY brain whose meta doc lists `index` in `nodes_index` (what
- * `xerj brain` writes), in `_cat` order, at most BRAINS_WALKED of them.
+ * `xerj brain` writes), in listing order, at most BRAINS_WALKED of them.
* `[]` when none does or nothing can be read. Best-effort; never throws.
*
* A list, not the first match: after `xerj brain casefile` and
* `xerj brain vendors` on one node both brains list `ax-docs`, and each
* holds links only for its own files. Returning the first brain made the
* reader say "records no links for this record" for half the corpus,
- * with which half depending on `_cat` order (PR #945 review).
+ * with which half depending on listing order (PR #945 review).
*/
async discoverBrains(index, signal) {
const out = [];
try {
- const r = await fetch('/_cat/indices/.xerj-memory-*', { signal, credentials: 'same-origin', headers: { accept: 'text/plain, application/json' } });
+ const r = await fetch(`${GRAPH}/brains`, { signal, credentials: 'same-origin', headers: { accept: 'application/json' } });
if (!r.ok) return out;
- for (const b of parseBrainIndices(await r.text()).slice(0, BRAINS_SCANNED)) {
- const m = await fetch(`/${enc(`.xerj-memory-${b}-edges`)}/_doc/${enc(BRAIN_META_ID)}`, { signal, credentials: 'same-origin', headers: { accept: 'application/json' } });
- if (!m.ok) continue;
- const doc = await m.json();
- const ni = String((doc && doc._source && doc._source.nodes_index) || '');
- if (ni.split(',').map((x) => x.trim()).includes(index)) out.push(b);
+ const j = await r.json();
+ for (const b of (j && j.data && j.data.brains) || []) {
+ const ni = String((b && b.nodes_index) || '');
+ if (ni.split(',').map((x) => x.trim()).includes(index)) out.push(b.name);
if (out.length >= BRAINS_WALKED) break;
}
} catch (e) {
diff --git a/xerj-ux/test/console-graph.test.mjs b/xerj-ux/test/console-graph.test.mjs
new file mode 100644
index 000000000..b7f396ab7
--- /dev/null
+++ b/xerj-ux/test/console-graph.test.mjs
@@ -0,0 +1,201 @@
+// The console's graph read path (issue #936): a signed-in operator reads
+// brains through the console's own session-authorized endpoints, never
+// through the data plane (whose `/_graph/*` and reserved-namespace
+// `/_search` routes 401 a session on an auth-enabled engine — the default).
+//
+// Run: node --test xerj-ux/test/
+import test from 'node:test';
+import assert from 'node:assert/strict';
+
+// second-brain-api.js reads location.hash while assembling a moment; the
+// rest of its browser surface is guarded (document, storage, rAF).
+globalThis.location = { hash: '' };
+
+const calls = [];
+let respond = () => json(404, {});
+globalThis.fetch = async (url, init) => { calls.push({ url: String(url), init }); return respond(String(url), init); };
+const json = (status, body) => ({ status, ok: status >= 200 && status < 300, json: async () => body, text: async () => (typeof body === 'string' ? body : JSON.stringify(body)) });
+
+const { makeConsoleTransport, BRAINS_WALKED } = await import('../src/data/transport-console.js');
+const { sbBrainsPresent, invalidateBrainsProbe } = await import('../src/data/brains-probe.js');
+const { liveSecondBrain } = await import('../src/data/second-brain-api.js');
+
+test('the reader transport reads the graph through the console session, same-origin', async () => {
+ calls.length = 0;
+ respond = (url) => {
+ assert.match(url, /^\/_xerj-console\/api\/v1\/graph\/casefile\/ego\?node=file-01&hops=1&direction=both/, url);
+ return json(200, { brain: 'casefile', edges: [] });
+ };
+ const r = await makeConsoleTransport().ego('casefile', { node: 'file-01', hops: '1', direction: 'both' }, undefined);
+ assert.equal(r.status, 200);
+ assert.equal(r.body.brain, 'casefile');
+ assert.equal(calls[0].init.credentials, 'same-origin', 'the session cookie must ride the console read');
+});
+
+test('brain discovery is ONE listing call that says which nodes index each brain feeds', async () => {
+ calls.length = 0;
+ respond = () => json(200, { data: { brains: [
+ { name: 'casefile', nodes_index: 'ax-docs' },
+ { name: 'vendors', nodes_index: 'ax-mail,ax-pdfs' },
+ { name: 'other', nodes_index: 'ax-mail' },
+ { name: 'no-dataset', nodes_index: '.xerj-memory-no-dataset' },
+ ], total: 4 } });
+ const t = makeConsoleTransport();
+ assert.deepEqual(await t.discoverBrains('ax-docs', undefined), ['casefile']);
+ assert.deepEqual(await t.discoverBrains('ax-pdfs', undefined), ['vendors']);
+ assert.deepEqual(await t.discoverBrains('ax-mail', undefined), ['vendors', 'other']);
+ assert.deepEqual(await t.discoverBrains('nothing-lists-this', undefined), []);
+ assert.equal(calls.length, 4, 'one listing per question — no _cat walk, no meta-doc reads');
+ for (const c of calls) {
+ assert.equal(c.url, '/_xerj-console/api/v1/graph/brains');
+ assert.equal(c.init.credentials, 'same-origin');
+ }
+ // A role that may not read brains gets [] — the reader then says "no
+ // brain for this index", which is the honest statement for that role.
+ calls.length = 0;
+ respond = () => json(403, { error: { reason: 'the console role "viewer" may not read brains' } });
+ assert.deepEqual(await t.discoverBrains('ax-docs', undefined), []);
+});
+
+test('the listing caps the walk at BRAINS_WALKED brains, in listing order', async () => {
+ const many = Array.from({ length: BRAINS_WALKED + 3 }, (_, i) => ({ name: `b-${i}`, nodes_index: 'ax-docs' }));
+ respond = () => json(200, { data: { brains: many, total: many.length } });
+ const out = await makeConsoleTransport().discoverBrains('ax-docs', undefined);
+ assert.equal(out.length, BRAINS_WALKED);
+ assert.deepEqual(out, many.slice(0, BRAINS_WALKED).map((b) => b.name));
+});
+
+test('the nav probe claims a brain only when the listing proves one', async () => {
+ invalidateBrainsProbe();
+ respond = () => json(200, { data: { brains: [{ name: 'casefile', nodes_index: 'ax-docs' }], total: 1 } });
+ assert.equal(await sbBrainsPresent('http://eng-a:9200'), true);
+ respond = () => json(403, {});
+ assert.equal(await sbBrainsPresent('http://eng-b:9200'), false, '403 is not "no brains on this node", but it is no proof of one');
+ respond = () => json(200, { data: { brains: [], total: 0 } });
+ assert.equal(await sbBrainsPresent('http://eng-c:9200'), false);
+ respond = () => { throw new TypeError('Failed to fetch'); };
+ assert.equal(await sbBrainsPresent('http://eng-d:9200'), false, 'engine unreachable → no claim');
+ assert.equal(await sbBrainsPresent(''), false);
+});
+
+// ── the Second Brain dashboard's own reads ─────────────────────────────────
+//
+// A full liveSecondBrain() pass over a multi-dataset brain ("ax-mail,ax-pdfs"):
+// every fetch must go through the console graph endpoints — except the
+// searches-per-index tile's /v1/metrics, which stays a data-plane read by
+// decision and shows its refusal honestly when it is refused.
+
+const OVERVIEW = {
+ brain: 'casefile', contract: 'xerj-second-brain/1', exists: true,
+ nodes_index: 'ax-mail,ax-pdfs', nodes: { total: 14 }, embedder: 'lexical-feature-hash',
+ edges: { total: 3, live: 2, invalidated: 1 },
+ types: [], detectors: [],
+ hubs: { out: [{ id: 'file-01', live_edges: 2 }], in: [] },
+ created_over_time: [], not_shown: {},
+};
+
+const egoBody = (qs) => ({
+ brain: 'casefile', node: qs.get('node'), edges: [
+ { edge_id: 'e1', src: 'file-01', dst: 'file-02', type: 'same_dir', valid_at: 1753600000000, invalid_at: null },
+ { edge_id: 'e3', src: 'file-02', dst: 'file-03', type: 'pathcite', valid_at: 1753600000000, invalid_at: 1753650000000 },
+ ],
+ nodes: { 'file-01': { title: '01.eml' } }, neighbors: [], not_shown: {},
+});
+
+function graphRouter(url) {
+ if (url === '/_xerj-console/api/v1/graph/brains') {
+ return json(200, { data: { brains: [{ name: 'casefile', nodes_index: 'ax-mail,ax-pdfs', contract: 'xerj-second-brain/1' }], total: 1, not_shown: { brains_clipped: 0 } } });
+ }
+ if (url.startsWith('/_xerj-console/api/v1/graph/casefile/overview')) return json(200, OVERVIEW);
+ if (url.startsWith('/_xerj-console/api/v1/graph/casefile/ego?')) {
+ return json(200, egoBody(new URLSearchParams(url.slice(url.indexOf('?') + 1))));
+ }
+ return json(404, {});
+}
+
+test('the dashboard reads everything through the console graph endpoints', async () => {
+ calls.length = 0;
+ respond = (url, init) => {
+ const body = init && init.body ? JSON.parse(init.body) : null;
+ if (url.startsWith('/_xerj-console/api/v1/graph/casefile/edges/_search')) {
+ if (body && body.query && body.query.exists && body.query.exists.field === 'invalid_at') {
+ return json(200, { hits: { total: { value: 1, relation: 'eq' }, hits: [{ _index: '.xerj-memory-casefile-edges', _id: 'e3', _source: { src: 'file-02', dst: 'file-03', type: 'pathcite' } }] } });
+ }
+ if (body && body.query && body.query.exists && body.query.exists.field === 'src_format') {
+ return json(200, { hits: { total: { value: 3, relation: 'eq' } }, aggregations: { src: { buckets: [{ key: 'eml', doc_count: 2, dst: { buckets: [{ key: 'eml', doc_count: 2 }] } }] } } });
+ }
+ return json(400, {});
+ }
+ if (url.startsWith('/_xerj-console/api/v1/graph/casefile/nodes/_search')) {
+ const which = new URLSearchParams(url.slice(url.indexOf('?') + 1)).get('index');
+ if (body && body.aggs && body.aggs.formats) {
+ return json(200, which === 'ax-mail'
+ ? { hits: { total: { value: 10, relation: 'eq' } }, aggregations: { formats: { buckets: [{ key: 'eml', doc_count: 6 }, { key: 'pdf', doc_count: 2 }] } } }
+ : { hits: { total: { value: 4, relation: 'eq' } }, aggregations: { formats: { buckets: [{ key: 'pdf', doc_count: 3 }] } } });
+ }
+ // name hydration (ids query)
+ return json(200, { hits: { total: { value: 3, relation: 'eq' }, hits: [
+ { _id: 'file-01', _source: { title: '01.eml', ax_path: 'inbox/01.eml' } },
+ { _id: 'file-02', _source: { title: '02.eml', ax_path: 'inbox/02.eml' } },
+ { _id: 'file-03', _source: { title: '03.eml', ax_path: 'inbox/03.eml' } },
+ ] } });
+ }
+ if (url === 'http://eng:9200/v1/metrics') {
+ return json(200, 'xerj_queries_by_index_total{index="ax-mail"} 7\nxerj_queries_by_index_total{index="ax-pdfs"} 2\n');
+ }
+ return graphRouter(url);
+ };
+
+ const data = await liveSecondBrain('http://eng:9200', {}, undefined);
+
+ // WHAT was read: only console graph endpoints + the one metrics read.
+ const urls = calls.map((c) => c.url);
+ for (const u of urls) {
+ assert.ok(
+ u.startsWith('/_xerj-console/api/v1/graph/') || u === 'http://eng:9200/v1/metrics',
+ `the dashboard must not read the data plane directly: ${u}`,
+ );
+ assert.ok(!u.includes('.xerj-memory-') && !u.includes('/_graph/') && !u.includes('/_cat'), u);
+ }
+ for (const c of calls) {
+ if (c.url.startsWith('/_xerj-console')) {
+ assert.equal(c.init.credentials, 'same-origin', `${c.url} must carry the session cookie`);
+ }
+ }
+ assert.ok(urls.includes('/_xerj-console/api/v1/graph/brains'));
+ assert.ok(urls.some((u) => u.includes('/casefile/overview?')));
+ assert.equal(urls.filter((u) => u.includes('/casefile/ego?')).length, 2, 'belief (2-hop) + belief strip (1-hop)');
+ assert.equal(urls.filter((u) => u.includes('/edges/_search')).length, 2, 'recent retirements + file-type crossings');
+
+ // The multi-dataset fan-out names WHICH of the brain's indices it reads.
+ const nodeSearchUrls = urls.filter((u) => u.includes('/nodes/_search'));
+ assert.equal(nodeSearchUrls.length, 4, 'notes tally + name hydration, one per dataset index');
+ assert.deepEqual(nodeSearchUrls.map((u) => new URLSearchParams(u.slice(u.indexOf('?') + 1)).get('index')).sort(), ['ax-mail', 'ax-mail', 'ax-pdfs', 'ax-pdfs'],
+ 'a multi-name nodes_index passes ?index= — the endpoint is pinned to ONE of the brain\'s own indices');
+
+ // And the answers are merged client-side, ran numbers only.
+ assert.deepEqual(data.sb.brains, ['casefile']);
+ assert.equal(data.overview.edges.total, 3);
+ assert.equal(data.sb.nodeStats.total, 14, '10 on ax-mail + 4 on ax-pdfs');
+ assert.deepEqual(data.sb.nodeStats.formats, [
+ { format: 'eml', count: 6 }, { format: 'pdf', count: 5 },
+ ], 'per-format buckets merged across the two indices, count desc');
+ assert.equal(data.sb.crossings.total, 3);
+ assert.deepEqual(data.sb.serverReads.rows, [
+ { index: 'ax-mail', count: 7 }, { index: 'ax-pdfs', count: 2 },
+ ]);
+ assert.equal(data.sb.recentRetired.length, 1);
+ assert.equal(data.sb.recentRetired[0].edge_id, 'e3');
+ assert.equal(data.sb.names['file-02'].title, '02.eml', 'hub ids hydrated to titles');
+ assert.equal(data.sb.focus, 'file-01', 'default focus = strongest hub');
+});
+
+test('a role that may not read brains gets an honest empty state, not a leak or a loop', async () => {
+ calls.length = 0;
+ respond = () => json(403, { error: { type: 'graph_error', reason: 'the console role "viewer" may not read brains' } });
+ const data = await liveSecondBrain('http://eng:9200', {}, undefined);
+ assert.deepEqual(data.sb.brains, []);
+ assert.equal(data.brain, null);
+ assert.equal(data.overview, null, 'no brain, no overview read — nothing further was fetched');
+ assert.equal(calls.filter((c) => c.url !== '/_xerj-console/api/v1/graph/brains').length, 0);
+});