AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
feat(cli): add opt-in Claude Code session handoff hooks (#2849)
Jarvis89eebbee0cc6
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex bcdc49c58..5e90c5aed 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -9,6 +9,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0### Added+- Opt-in Claude Code session handoff with `xerj init --memory` on macOS/Linux+ (Refs #2645). SessionEnd queues a background redacting import and lexical+ autoindex; SessionStart returns bounded, cited historical excerpts from a+ project memory namespace, with an offline cache. `--memory --remove` removes+ only the owned hooks and retains data. Ordinary init and manual session+ import are unchanged; Codex lifecycle hooks are not included in this first version.+ Each queued delivery is isolated: terminal input refusals are retained in a+ private rejected queue with recovery reasons, while transient failures retry.+ Memory excerpts and displayed briefs filter Unicode controls and format marks.+- A second Claude plugin in the xerj marketplace, `xerj-knowledge`, fordocument search rather than code: a skill and `/xerj-knowledge:index` and`/xerj-knowledge:find` commands that search local folders, notes, PDFs,diff --git a/docs/RERANK.md b/docs/RERANK.mdindex 11e554a26..be9a7277f 100644--- a/docs/RERANK.md+++ b/docs/RERANK.md@@ -406,6 +406,11 @@ the picture:`xerj batch-search` sends the batch's queries or identifiers to `/_msearch`and optionally reads index mappings at that node, with the configured APIkey when present (`--url` / `--auth` override `XERJ_URL` / `XERJ_AUTH`).+ `xerj sessions hook` (installed only by `xerj init --memory`) sends redacted+ transcript exports and short excerpts to its configured loopback node for+ indexing/memory storage, and reads memories back for SessionStart. It refuses+ remote URLs and disables redirects and proxies; it adds no node-side egress.+ Runtime authentication is sent only to that loopback node.`xerj feedback --open-pr` runs your `gh` to open a GitHub pull request withthe report it drafted; without that flag it does nothing over the network.diff --git a/docs/recipes/air-gapped-deployment.md b/docs/recipes/air-gapped-deployment.mdindex ef49b6354..b0a49f995 100644--- a/docs/recipes/air-gapped-deployment.md+++ b/docs/recipes/air-gapped-deployment.md@@ -47,6 +47,7 @@ those two files before entering the enclave.| `xerj hub feedback` and automatic corpus reviews | On for `xerj hub pull` and the MCP hub tools | A client, not the node. It POSTs a structured review (scores, corpus slug, agent family, a random install id; no document or query text) to `XERJ_CORPUS_FEEDBACK_URL`, default `https://hub.xerj.org/api/corpus-feedback`. Set `XERJ_DISABLE_FEEDBACK=true` (or pass `--disable-feedback`) to send nothing || Cluster | Disabled | Single-node startup does not initialize the Raft transport || REST / ES-compatible listeners | `127.0.0.1` | The default is loopback-only |+| `xerj sessions hook` (opt-in `xerj init --memory`) | Off until installed | Sends redacted session exports/excerpts and runtime auth to the configured loopback node, reads memories back; remote URLs, redirects and proxies are refused. No external service or model is required || `autoindex` and MCP clients | Localhost defaults | They connect to `localhost`; they do not create a public listener || Runtime telemetry/update/license activation | None | No outbound call is made by the running binary for these purposes |diff --git a/engine/crates/xerj-autoindex/src/init.rs b/engine/crates/xerj-autoindex/src/init.rsindex f3bc2c6f0..740bb96e2 100644--- a/engine/crates/xerj-autoindex/src/init.rs+++ b/engine/crates/xerj-autoindex/src/init.rs@@ -64,12 +64,21 @@ OPTIONS:--url <U> node endpoint baked into the MCP config(default $XERJ_URL or http://localhost:9200)--dry-run print what would be written, write nothing+ --memory opt in to project-local Claude Code session handoff (macOS/Linux)+ --remove with --memory, remove only these hooks; keep private data--disable-feedbackaccepted; turns off the feedback invitation on helpscreens. Honoured in any position (env XERJ_DISABLE_FEEDBACK)-h, --help this help-After init: restart the agent session so it picks up the new tools, then+With --memory, init manages only SessionEnd/SessionStart hooks in+.claude/settings.local.json. User/assistant text is PII-redacted, saved locally+and indexed on the chosen loopback node in the background. The next session+gets bounded, cited excerpts, not an LLM summary. No key is saved. Private+archives and pending work stay in .claude/xerj-memory/; --remove retains them.+Read tools/session-memory/README.md for limits, inspection and deletion.++After ordinary init: restart the agent session so it picks up the new tools, then`xerj autoindex <folder> --prefix <name>` anything you want searchable.Pass that same `--prefix <name>` to `xerj search` and `xerj def`; without itthey search the default `ax` prefix. {hub_next}init also prints the MCP registration line for each client.@@ -573,6 +582,8 @@ struct InitFlags {url: String,dry: bool,help: bool,+ memory: bool,+ remove: bool,}/// `Err` is the unexpected-argument line, without the help page.@@ -582,6 +593,8 @@ fn read_init_flags(args: impl IntoIterator<Item = String>) -> Result<InitFlags,.filter(|s| !s.is_empty()).unwrap_or_else(|| DEFAULT_URL.to_string());let mut dry = false;+ let mut memory = false;+ let mut remove = false;let mut it = args.into_iter();while let Some(a) = it.next() {match a.as_str() {@@ -590,23 +603,47 @@ fn read_init_flags(args: impl IntoIterator<Item = String>) -> Result<InitFlags,url,dry,help: true,+ memory,+ remove,});}"--url" => url = it.next().unwrap_or(url),"--dry-run" => dry = true,+ "--memory" => memory = true,+ "--remove" => remove = true,// Also stripped by args_after_subcommand. Accepted here so a// caller that passes the raw list still does not exit 2."--disable-feedback" => {}other => return Err(format!("xerj init: unexpected argument '{other}'")),}}+ if remove && !memory {+ return Err("xerj init: --remove requires --memory".into());+ }Ok(InitFlags {url,dry,help: false,+ memory,+ remove,})}+#[cfg(test)]+mod memory_flag_regression {+ #[test]+ fn memory_hooks_are_an_explicit_init_opt_in() {+ assert!(super::read_init_flags(["--remove".into()])+ .err()+ .unwrap()+ .contains("--remove requires --memory"));+ let plain = super::read_init_flags(Vec::new()).unwrap();+ assert!(!plain.memory && !plain.remove);+ let result = super::read_init_flags(["--memory", "--dry-run"].map(String::from));+ assert!(result.is_ok(), "memory opt-in refused: {:?}", result.err());+ }+}+/// Entry point for the `init` subcommand. Returns a process exit code.pub fn run_init_cli() -> i32 {let flags = match read_init_flags(xerj_common::argv::args_after_subcommand()) {@@ -623,6 +660,25 @@ pub fn run_init_cli() -> i32 {let url = flags.url;let dry = flags.dry;let cwd = std::env::current_dir().unwrap_or_else(|_| PathBuf::from("."));+ if flags.memory {+ let result = std::env::current_exe()+ .map_err(anyhow::Error::from)+ .and_then(|binary| {+ crate::session_memory::configure(&cwd, &url, dry, flags.remove, &binary)+ });+ return match result {+ Ok(lines) => {+ for line in lines {+ println!("{line}");+ }+ 0+ }+ Err(error) => {+ eprintln!("xerj init --memory: {error}");+ 1+ }+ };+ }let hub = xerj_common::hub::hub_key_configured();match run_init_with(&cwd, &url, dry, hub) {Ok(lines) => {diff --git a/engine/crates/xerj-autoindex/src/lib.rs b/engine/crates/xerj-autoindex/src/lib.rsindex cefa89979..c2b251253 100644--- a/engine/crates/xerj-autoindex/src/lib.rs+++ b/engine/crates/xerj-autoindex/src/lib.rs@@ -50,6 +50,7 @@ mod reconcile_plan;pub mod remove;pub mod resources;pub mod search;+mod session_memory;pub mod sessions;pub mod skills;pub mod sniff;diff --git a/engine/crates/xerj-autoindex/src/session_memory.rs b/engine/crates/xerj-autoindex/src/session_memory.rsnew file mode 100644index 000000000..9f49dd134--- /dev/null+++ b/engine/crates/xerj-autoindex/src/session_memory.rs@@ -0,0 +1,871 @@+//! Opt-in Claude Code session handoff. Host events are data, never commands.+//! Only a loopback node is supported; the existing offline importer owns redaction.+use anyhow::{bail, ensure, Context, Result};+use fs2::FileExt;+use serde::{Deserialize, Serialize};+use serde_json::{json, Value};+use sha2::{Digest, Sha256};+use std::fs::{self, File, OpenOptions};+use std::io::{Read, Write};+use std::path::{Path, PathBuf};+use std::process::{Command, Stdio};+use std::time::{Duration, Instant};++const STATE: &str = ".claude/xerj-memory";+const SETTINGS: &str = ".claude/settings.local.json";+const LIMIT: u64 = 64 * 1024;++#[derive(Clone, Serialize, Deserialize)]+struct Config {+ version: u32,+ enabled: bool,+ project: PathBuf,+ url: String,+ namespace: String,+ prefix: String,+ hooks: Value,+}++fn digest(bytes: &[u8]) -> String {+ format!("{:x}", Sha256::digest(bytes))+}+fn state(project: &Path) -> PathBuf {+ project.join(STATE)+}+fn check_path(path: &Path) -> Result<()> {+ for ancestor in path.ancestors() {+ match fs::symlink_metadata(ancestor) {+ Ok(meta) => ensure!(+ !meta.file_type().is_symlink(),+ "session memory refuses symlink paths"+ ),+ Err(error) if error.kind() == std::io::ErrorKind::NotFound => (),+ Err(error) => return Err(error.into()),+ }+ }+ Ok(())+}+fn read(path: &Path, limit: u64) -> Result<Option<Vec<u8>>> {+ check_path(path)?;+ let mut options = OpenOptions::new();+ options.read(true);+ #[cfg(unix)]+ {+ use std::os::unix::fs::OpenOptionsExt;+ options.custom_flags(libc::O_NOFOLLOW | libc::O_NONBLOCK);+ }+ let file = match options.open(path) {+ Ok(file) => file,+ Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None),+ Err(error) => return Err(error.into()),+ };+ ensure!(+ file.metadata()?.is_file(),+ "session memory needs regular files"+ );+ let mut bytes = Vec::new();+ file.take(limit + 1).read_to_end(&mut bytes)?;+ ensure!(+ bytes.len() as u64 <= limit,+ "session memory file exceeds its read budget"+ );+ Ok(Some(bytes))+}+fn private_dir(path: &Path) -> Result<()> {+ check_path(path)?;+ if !path.exists() {+ let mut builder = fs::DirBuilder::new();+ // Use the same mutable API on every target, including non-Unix builds.+ builder.recursive(false);+ #[cfg(unix)]+ {+ use std::os::unix::fs::DirBuilderExt;+ builder.mode(0o700);+ }+ builder.create(path)?;+ }+ #[cfg(unix)]+ {+ use std::os::unix::fs::MetadataExt;+ let meta = fs::metadata(path)?;+ ensure!(+ meta.is_dir() && meta.uid() == unsafe { libc::geteuid() } && meta.mode() & 0o077 == 0,+ "session memory directory must be owned by you with mode 0700"+ );+ }+ Ok(())+}+fn atomic(path: &Path, bytes: &[u8]) -> Result<()> {+ check_path(path)?;+ let mut temp = tempfile::NamedTempFile::new_in(path.parent().context("missing parent")?)?;+ temp.write_all(bytes)?;+ temp.as_file().sync_all()?;+ temp.persist(path).map_err(|error| error.error)?;+ Ok(())+}+fn encode(value: &impl Serialize) -> Result<Vec<u8>> {+ let mut bytes = serde_json::to_vec_pretty(value)?;+ bytes.push(b'\n');+ Ok(bytes)+}+fn lock(path: &Path) -> Result<File> {+ check_path(path)?;+ let mut options = OpenOptions::new();+ options.read(true).write(true).create(true).truncate(false);+ #[cfg(unix)]+ {+ use std::os::unix::fs::OpenOptionsExt;+ options+ .mode(0o600)+ .custom_flags(libc::O_NOFOLLOW | libc::O_NONBLOCK);+ }+ let file = options.open(path)?;+ ensure!(+ file.metadata()?.is_file(),+ "session memory lock must be a regular file"+ );+ file.try_lock_exclusive()+ .context("session memory worker is busy; retry after it finishes")?;+ Ok(file)+}+fn loopback(url: &str) -> Result<()> {+ let url = reqwest::Url::parse(url)?;+ ensure!(+ matches!(url.scheme(), "http" | "https")+ && url.username().is_empty()+ && url.password().is_none()+ && url.query().is_none()+ && url.fragment().is_none()+ && url.path() == "/"+ && xerj_common::localauth::url_is_loopback(url.as_str()),+ "session memory only supports a loopback node URL without credentials or a path"+ );+ Ok(())+}+fn config(project: &Path) -> Result<Config> {+ let bytes = read(&state(project).join("config.json"), LIMIT)?+ .context("session memory is not enabled")?;+ let config: Config = serde_json::from_slice(&bytes)?;+ ensure!(+ config.version == 1 && config.enabled && config.project == project,+ "session memory is disabled or has an unsupported configuration"+ );+ loopback(&config.url)?;+ let key = digest(+ project+ .to_str()+ .context("project path must be UTF-8")?+ .as_bytes(),+ );+ let expected = format!("sessions-{}", &key[..24]);+ ensure!(+ config.namespace == expected && config.prefix == expected,+ "session memory scope does not match this project"+ );+ Ok(config)+}+fn quote(text: &str) -> String {+ format!("'{}'", text.replace('\'', "'\\''"))+}+fn hooks(project: &Path, binary: &Path) -> Result<Value> {+ let project = project.to_str().context("project path must be UTF-8")?;+ let binary = binary.to_str().context("binary path must be UTF-8")?;+ ensure!(+ !project.chars().any(char::is_control) && !binary.chars().any(char::is_control),+ "control characters in hook paths"+ );+ let command = format!(+ "{} sessions hook --project {} --event",+ quote(binary),+ quote(project)+ );+ Ok(json!({+ "SessionStart": {"hooks":[{"type":"command","command":format!("{command} start"),"timeout":2}]},+ "SessionEnd": {"hooks":[{"type":"command","command":format!("{command} end"),"timeout":2}]}+ }))+}++/// Memory setup is separate from ordinary init: no other agent configuration is touched.+/// Both files are preflighted; disable precedes hook removal. Data is never deleted here.+pub(crate) fn configure(+ project: &Path,+ url: &str,+ dry: bool,+ remove: bool,+ binary: &Path,+) -> Result<Vec<String>> {+ ensure!(+ cfg!(unix),+ "session memory hooks currently support macOS and Linux only"+ );+ let project = project.canonicalize()?;+ loopback(url)?;+ let settings_path = project.join(SETTINGS);+ let directory = state(&project);+ let config_path = directory.join("config.json");+ let old_settings = read(&settings_path, 1024 * 1024)?;+ let old_config = read(&config_path, LIMIT)?;+ let previous: Option<Config> = old_config+ .as_deref()+ .map(serde_json::from_slice)+ .transpose()?;+ if let Some(previous) = &previous {+ ensure!(+ previous.version == 1 && previous.project == project,+ "foreign or unsupported session memory configuration"+ );+ }+ let mut settings: Value = old_settings+ .as_deref()+ .map(serde_json::from_slice)+ .transpose()?+ .unwrap_or(json!({}));+ ensure!(+ settings.is_object(),+ "Claude settings must be a JSON object; not touching it"+ );+ if remove && previous.as_ref().is_none_or(|p| !p.enabled) {+ return Ok(vec!["session memory is already disabled; data kept".into()]);+ }+ let wanted = hooks(&project, binary)?;+ let root = settings.as_object_mut().expect("checked object");+ let hook_map = root.entry("hooks").or_insert(json!({}));+ let map = hook_map+ .as_object_mut()+ .context("Claude hooks must be an object; not touching it")?;+ // Validate both event arrays before mutating either file. Remove only exact owned groups.+ for event in ["SessionStart", "SessionEnd"] {+ if let Some(groups) = map.get(event) {+ ensure!(+ groups.is_array(),+ "Claude hook event must be an array; not touching it"+ );+ }+ if let Some(previous) = previous.as_ref().filter(|p| p.enabled) {+ let owned = &previous.hooks[event];+ ensure!(+ map.get(event)+ .and_then(Value::as_array)+ .is_some_and(|groups| groups.iter().filter(|g| *g == owned).count() == 1),+ "managed hook was edited or removed; restore it before changing session memory"+ );+ }+ let owned_matches_wanted = previous+ .as_ref()+ .is_some_and(|p| p.enabled && p.hooks[event] == wanted[event]);+ let matches = map+ .get(event)+ .and_then(Value::as_array)+ .map_or(0, |groups| {+ groups.iter().filter(|g| *g == &wanted[event]).count()+ });+ ensure!(+ matches == usize::from(owned_matches_wanted),+ "matching session hook exists without unique ownership; not touching it"+ );+ }+ for event in ["SessionStart", "SessionEnd"] {+ let groups = map+ .entry(event)+ .or_insert(json!([]))+ .as_array_mut()+ .expect("validated array");+ if let Some(previous) = previous.as_ref().filter(|p| p.enabled) {+ groups.retain(|g| g != &previous.hooks[event]);+ }+ if !remove {+ groups.push(wanted[event].clone());+ }+ if groups.is_empty() {+ map.remove(event);+ }+ }+ if map.is_empty() {+ root.remove("hooks");+ }+ let key = digest(+ project+ .to_str()+ .context("project path must be UTF-8")?+ .as_bytes(),+ );+ let next = Config {+ version: 1,+ enabled: !remove,+ project: project.clone(),+ url: url.trim_end_matches('/').into(),+ namespace: format!("sessions-{}", &key[..24]),+ prefix: format!("sessions-{}", &key[..24]),+ hooks: wanted,+ };+ let next_settings = encode(&settings)?;+ let next_config = encode(&next)?;+ let mut lines = vec![+ format!(+ "{} Claude Code session memory (project-local, loopback only)",+ if dry && remove {+ "would disable"+ } else if dry {+ "would configure"+ } else if remove {+ "disabled"+ } else {+ "enabled"+ }+ ),+ format!("sessions prefix / memory namespace: {}", next.prefix),+ format!(+ "private data: {} (retained on --remove)",+ directory.display()+ ),+ ];+ if dry {+ return Ok(lines);+ }+ fs::create_dir_all(project.join(".claude"))?;+ private_dir(&directory)?;+ let _guard = lock(&directory.join("worker.lock"))?;+ for name in ["pending", "archives", "exports", "receipts"] {+ private_dir(&directory.join(name))?;+ }+ atomic(&directory.join(".gitignore"), b"*\n!.gitignore\n")?;+ if old_settings.as_deref() == Some(next_settings.as_slice())+ && old_config.as_deref() == Some(next_config.as_slice())+ {+ lines[0] = "kept Claude Code session memory (already configured)".into();+ return Ok(lines);+ }+ if let Some(bytes) = &old_settings {+ let backup = directory.join(format!("settings-{}.bak", &digest(bytes)[..16]));+ match read(&backup, 1024 * 1024)? {+ Some(found) => ensure!(+ found == *bytes,+ "session memory backup collision; not touching settings"+ ),+ None => {+ let mut temp = tempfile::NamedTempFile::new_in(project.join(".claude"))?;+ temp.write_all(bytes)?;+ temp.as_file().sync_all()?;+ temp.persist_noclobber(backup).map_err(|e| e.error)?;+ }+ }+ }+ atomic(&config_path, &next_config)?;+ if let Err(error) = atomic(&settings_path, &next_settings) {+ match old_config {+ Some(bytes) => atomic(&config_path, &bytes)?,+ None => fs::remove_file(&config_path)?,+ }+ return Err(error);+ }+ Ok(lines)+}++#[derive(Serialize, Deserialize)]+struct Event {+ session_id: String,+ transcript_path: PathBuf,+ delivery_id: String,+}+fn queue_name(event: &Event) -> String {+ format!(+ "{}-{}.json",+ digest(event.session_id.as_bytes()),+ event.delivery_id+ )+}+fn event(project: &Path, value: &Value, name: &str) -> Result<Event> {+ ensure!(value["hook_event_name"] == name, "wrong hook event");+ let cwd = Path::new(value["cwd"].as_str().context("missing cwd")?).canonicalize()?;+ ensure!(cwd.starts_with(project), "hook belongs to another project");+ let sid = value["session_id"].as_str().context("missing session id")?;+ ensure!(+ sid.len() <= 128 && uuid::Uuid::parse_str(sid).is_ok(),+ "invalid session id"+ );+ Ok(Event {+ session_id: sid.into(),+ transcript_path: PathBuf::from(+ value["transcript_path"]+ .as_str()+ .context("missing transcript")?,+ ),+ delivery_id: uuid::Uuid::new_v4().to_string(),+ })+}+fn spawn(project: &Path) -> Result<()> {+ let mut command = Command::new(std::env::current_exe()?);+ command+ .args(["sessions", "hook", "--project"])+ .arg(project)+ .args(["--event", "drain"])+ .current_dir(project)+ .stdin(Stdio::null())+ .stdout(Stdio::null())+ .stderr(Stdio::null());+ #[cfg(unix)]+ {+ use std::os::unix::process::CommandExt;+ // The host's SessionEnd budget must not include import/indexing work.+ unsafe {+ command.pre_exec(|| {+ if libc::setsid() == -1 {+ return Err(std::io::Error::last_os_error());+ }+ Ok(())+ });+ }+ }+ let mut child = command.spawn()?;+ std::thread::spawn(move || {+ let _ = child.wait();+ });+ Ok(())+}+fn client(config: &Config, timeout: Duration) -> Result<reqwest::blocking::Client> {+ let mut headers = reqwest::header::HeaderMap::new();+ let auth = std::env::var("XERJ_API_KEY")+ .ok()+ .filter(|s| !s.is_empty())+ .map(|s| format!("ApiKey {s}"))+ .or_else(|| {+ let found =+ xerj_common::localauth::discover_local_admin_key_for_url_with_origin(&config.url)?;+ found+ .foreign_warning("XERJ_API_KEY")+ .is_none()+ .then(|| format!("ApiKey {}", found.key))+ });+ if let Some(auth) = auth {+ headers.insert(reqwest::header::AUTHORIZATION, auth.parse()?);+ }+ Ok(reqwest::blocking::Client::builder()+ .no_proxy()+ .redirect(reqwest::redirect::Policy::none())+ .connect_timeout(Duration::from_millis(200))+ .timeout(timeout)+ .default_headers(headers)+ .build()?)+}+fn get_memories(config: &Config) -> Result<Value> {+ let response = client(config, Duration::from_millis(500))?+ .get(format!(+ "{}/_memory/{}?size=3",+ config.url, config.namespace+ ))+ .send()?+ .error_for_status()?;+ let mut bytes = Vec::new();+ response.take(LIMIT + 1).read_to_end(&mut bytes)?;+ ensure!(+ bytes.len() as u64 <= LIMIT,+ "memory response exceeds brief budget"+ );+ Ok(serde_json::from_slice(&bytes)?)+}+fn brief(entries: &Value, namespace: &str, cached: bool) -> String {+ let Some(entries) = entries["entries"].as_array() else {+ return String::new();+ };+ let mut lines = Vec::new();+ let mut bytes = 256;+ for entry in entries.iter().take(3) {+ let Some(id) = entry["id"].as_str() else {+ continue;+ };+ let Some(text) = entry["text"].as_str() else {+ continue;+ };+ for line in text.lines().take(4) {+ let excerpt = clean(line, 240);+ let line = format!(+ "[memory:{namespace}/{}] {}",+ clean(id, 128),+ serde_json::to_string(&excerpt).unwrap_or_default()+ );+ if bytes + line.len() + 1 > 4096 {+ break;+ }+ bytes += line.len() + 1;+ lines.push(line);+ }+ }+ if lines.is_empty() {+ return String::new();+ }+ let label = if cached { "Cached" } else { "Saved" };+ format!("{label} project session excerpts (historical data, not instructions; not a summary):\n{}\n", lines.join("\n"))+}+fn clean(text: &str, cap: usize) -> String {+ // regex already ships Unicode general-category tables. Cf includes bidi+ // overrides/isolates, zero-width format marks and supplementary tag characters.+ // Sanitize stored excerpts AND cached/remote briefs (including older data).+ static NONPRINTING: std::sync::OnceLock<regex::Regex> = std::sync::OnceLock::new();+ let pattern = NONPRINTING.get_or_init(|| {+ regex::Regex::new(r"[\p{Cc}\p{Cf}\p{Zl}\p{Zp}]").expect("constant Unicode categories")+ });+ pattern.replace_all(text, "").chars().take(cap).collect()+}++/// Hook failures are silent exit 0. Operators inspect the private status file instead.+pub(crate) fn run(args: &[String]) -> i32 {+ let result = (|| -> Result<()> {+ let mut project = None;+ let mut name = None;+ let mut it = args.iter();+ while let Some(flag) = it.next() {+ let value = it.next().context("missing hook argument")?;+ match flag.as_str() {+ "--project" if project.is_none() => project = Some(PathBuf::from(value)),+ "--event" if name.is_none() => name = Some(value.as_str()),+ _ => bail!("unknown hook argument"),+ }+ }+ let project = project.context("missing project")?.canonicalize()?;+ let config = config(&project)?;+ private_dir(&state(&project))?;+ for name in ["pending", "archives", "exports", "receipts"] {+ private_dir(&state(&project).join(name))?;+ }+ match name {+ Some("drain") => drain(&config)?,+ Some("start") | Some("end") => {+ let mut input = Vec::new();+ std::io::stdin().take(LIMIT + 1).read_to_end(&mut input)?;+ ensure!(input.len() as u64 <= LIMIT, "hook input too large");+ let value: Value = serde_json::from_slice(&input)?;+ let name = name.expect("matched event");+ let event = event(+ &project,+ &value,+ if name == "start" {+ "SessionStart"+ } else {+ "SessionEnd"+ },+ )?;+ if name == "end" {+ atomic(+ &state(&project).join("pending").join(queue_name(&event)),+ &encode(&event)?,+ )?;+ } else {+ let memories = get_memories(&config).ok();+ let cached = memories.is_none();+ let memories = memories.or_else(|| {+ read(&state(&project).join("brief.json"), LIMIT)+ .ok()+ .flatten()+ .and_then(|b| serde_json::from_slice(&b).ok())+ });+ if let Some(memories) = memories {+ print!("{}", brief(&memories, &config.namespace, cached));+ }+ }+ if fs::read_dir(state(&project).join("pending"))?+ .next()+ .is_some()+ {+ let _ = spawn(&project);+ }+ }+ _ => bail!("unknown hook event"),+ }+ Ok(())+ })();+ let _ = result;+ 0+}++fn autoindex(config: &Config, directory: &Path, lease: &File) -> Result<()> {+ let deadline = Instant::now() + Duration::from_secs(60);+ for dry in [true, false] {+ let mut estimate = tempfile::NamedTempFile::new_in(directory)?;+ let mut command = Command::new(std::env::current_exe()?);+ command+ .arg("autoindex")+ .arg(directory.join("exports"))+ .args([+ "--url",+ &config.url,+ "--prefix",+ &config.prefix,+ "--state-dir",+ ])+ .arg(directory.join("index-state"))+ .args([+ "--no-graph",+ "--no-semantic",+ "--workers",+ "1",+ "--max-minutes",+ "0",+ "--progress",+ "none",+ ])+ .env("XERJ_DISABLE_FEEDBACK", "true")+ .env("XERJ_NUM_CPUS", "2")+ .stdin(Stdio::null());+ if dry {+ command+ .arg("--dry-run")+ .stdout(estimate.as_file().try_clone()?)+ .stderr(estimate.as_file().try_clone()?);+ } else {+ command.stdout(Stdio::null()).stderr(Stdio::null());+ }+ #[cfg(unix)]+ {+ use std::os::{fd::AsRawFd, unix::process::CommandExt};+ let fd = lease.as_raw_fd();+ // Retain this lease if the supervisor dies while autoindex is alive.+ // Unlike Journal's StateLock, this File has no explicit unlock on drop.+ unsafe {+ command.pre_exec(move || {+ if libc::fcntl(fd, libc::F_SETFD, 0) == -1 {+ return Err(std::io::Error::last_os_error());+ }+ Ok(())+ });+ }+ }+ #[cfg(not(unix))]+ let _ = lease;+ let mut child = command.spawn()?;+ loop {+ if let Some(status) = child.try_wait()? {+ ensure!(+ matches!(status.code(), Some(0 | 3)),+ "session autoindex did not complete"+ );+ break;+ }+ if Instant::now() >= deadline {+ child.kill()?;+ child.wait()?;+ bail!("session autoindex deadline; pending import kept for retry");+ }+ std::thread::sleep(Duration::from_millis(50));+ }+ if dry {+ estimate.flush()?;+ estimate.as_file().sync_all()?;+ estimate+ .persist(directory.join("last-estimate.txt"))+ .map_err(|e| e.error)?;+ }+ }+ Ok(())+}++/// Only explicit, pre-publication input refusals are terminal. Transport,+/// filesystem and incomplete/growing-transcript failures remain retryable.+#[derive(Debug)]+struct Rejected(&'static str);+impl std::fmt::Display for Rejected {+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {+ f.write_str(self.0)+ }+}+impl std::error::Error for Rejected {}++fn import(config: &Config, event: &Event, directory: &Path, lease: &File) -> Result<()> {+ let metadata = fs::metadata(&event.transcript_path).map_err(|error| {+ if error.kind() == std::io::ErrorKind::NotFound {+ anyhow::Error::new(Rejected(+ "transcript missing; restore it and send a new SessionEnd",+ ))+ } else {+ error.into()+ }+ })?;+ if metadata.len() > 16 * 1024 * 1024 {+ bail!(Rejected(+ "transcript exceeds 16 MiB; use manual sessions import"+ ));+ }+ let temporary = tempfile::tempdir_in(directory)?;+ let export = temporary.path().join("session.jsonl");+ let summary = crate::sessions::import_claude_code(&event.transcript_path, &export)?;+ let file = File::open(&export)?;+ let mut last = std::collections::VecDeque::new();+ use std::io::BufRead;+ for (line, row) in std::io::BufReader::new(file).lines().enumerate() {+ let row: Value = serde_json::from_str(&row?)?;+ if row["session_id"] != event.session_id {+ bail!(Rejected(+ "transcript belongs to a different session; send the matching session event"+ ));+ }+ if row["is_sidechain"] == true {+ continue;+ }+ last.push_back((line + 1, row));+ if last.len() > 4 {+ last.pop_front();+ }+ }+ let sid = digest(event.session_id.as_bytes());+ let archive = format!("{}.jsonl", summary.output_sha256);+ let archive_path = directory.join("archives").join(&archive);+ if !archive_path.exists() {+ let mut temp = tempfile::NamedTempFile::new_in(directory.join("archives"))?;+ std::io::copy(&mut File::open(&export)?, &mut temp)?;+ temp.as_file().sync_all()?;+ temp.persist_noclobber(&archive_path).map_err(|e| e.error)?;+ } else {+ check_path(&archive_path)?;+ ensure!(+ fs::metadata(&archive_path)?.is_file(),+ "archive must be a regular file"+ );+ let mut file = File::open(&archive_path)?;+ let mut hash = Sha256::new();+ std::io::copy(&mut file, &mut hash)?;+ ensure!(+ format!("{:x}", hash.finalize()) == summary.output_sha256,+ "archive bytes disagree with their identity"+ );+ }+ let current = directory.join("exports").join(format!("{sid}.jsonl"));+ let mut temp = tempfile::NamedTempFile::new_in(directory.join("exports"))?;+ std::io::copy(&mut File::open(&export)?, &mut temp)?;+ temp.as_file().sync_all()?;+ temp.persist(¤t).map_err(|e| e.error)?;+ let receipt = directory.join("receipts").join(format!("{sid}.json"));+ let identity =+ json!({"source_sha256":summary.source_sha256,"output_sha256":summary.output_sha256});+ client(config, Duration::from_secs(1))?+ .get(&config.url)+ .send()?+ .error_for_status()?;+ autoindex(config, directory, lease)?;+ if !last.is_empty() {+ let text = last+ .iter()+ .map(|(line, row)| {+ format!(+ "[archives/{archive}:{line}; {}] {}: {}",+ clean(row["message_id"].as_str().unwrap_or_default(), 128),+ clean(row["role"].as_str().unwrap_or_default(), 16),+ clean(row["body"].as_str().unwrap_or_default(), 200)+ )+ })+ .collect::<Vec<_>>()+ .join("\n");+ let payload = json!({"id":sid,"text":text,"dedup":false,"metadata":{"source":"xerj-session-memory-v1","export_sha256":summary.output_sha256}});+ client(config, Duration::from_secs(2))?+ .post(format!("{}/_memory/{}", config.url, config.namespace))+ .json(&payload)+ .send()?+ .error_for_status()?;+ let cache = json!({"entries":[{"id":sid,"text":text}]});+ atomic(&directory.join("brief.json"), &encode(&cache)?)?;+ }+ atomic(&receipt, &encode(&identity)?)?;+ Ok(())+}+fn drain(config: &Config) -> Result<()> {+ let directory = state(&config.project);+ let guard = lock(&directory.join("worker.lock"))?;+ // Removal or reconfiguration may have won the lease after argv was read.+ let config = &self::config(&config.project)?;+ let deadline = Instant::now() + Duration::from_secs(90);+ let pending = directory.join("pending");+ let mut failed = false;+ // Snapshot only this bounded batch; no item failure may terminate its siblings.+ // An immutable rejected copy is published before its pending entry is removed.+ let batch: Vec<_> = fs::read_dir(&pending)?.take(8).collect();+ for item in batch {+ if Instant::now() >= deadline {+ break;+ }+ let path = match item {+ Ok(item) => item.path(),+ Err(_) => {+ failed = true;+ continue;+ }+ };+ if path.extension().and_then(|x| x.to_str()) != Some("json") {+ continue;+ }+ let mut original = None;+ let mut session = None;+ let result = (|| -> Result<()> {+ let Some(bytes) = read(&path, LIMIT)? else {+ return Ok(());+ };+ original = Some(bytes);+ let event: Event = serde_json::from_slice(original.as_ref().expect("read entry"))+ .map_err(|_| {+ Rejected("invalid queue JSON; inspect rejected entry and send a new SessionEnd")+ })?;+ session = Some(digest(event.session_id.as_bytes()));+ if path.file_name().and_then(|s| s.to_str()) != Some(queue_name(&event).as_str()) {+ bail!(Rejected("queue identity mismatch; send a new SessionEnd"));+ }+ import(config, &event, &directory, &guard)+ })();+ let rejection = result+ .as_ref()+ .err()+ .and_then(|error| error.downcast_ref::<Rejected>());+ let outcome = if result.is_ok() {+ "complete"+ } else if rejection.is_some() {+ "rejected"+ } else {+ "pending"+ };+ let reason = rejection.map_or_else(+ || {+ if result.is_ok() {+ "imported"+ } else {+ "import or node unavailable; retry on next session; check node and transcript"+ }+ },+ |error| error.0,+ );+ let status = json!({"version":1,"state":outcome,"session":session,"reason":reason});+ // Never retire work unless status and the rejected copy are persisted.+ let finish = (|| -> Result<()> {+ atomic(&directory.join("status.json"), &encode(&status)?)?;+ if let Some(bytes) = original.as_deref() {+ if rejection.is_some() {+ let rejected = directory.join("rejected");+ private_dir(&rejected)?;+ let target = rejected.join(path.file_name().context("missing queue name")?);+ check_path(&target)?;+ atomic(&target.with_extension("status"), &encode(&status)?)?;+ match read(&target, LIMIT)? {+ Some(found) => ensure!(found == bytes, "rejected entry collision"),+ None => {+ let mut temp = tempfile::NamedTempFile::new_in(&rejected)?;+ temp.write_all(bytes)?;+ temp.as_file().sync_all()?;+ temp.persist_noclobber(&target)+ .map_err(|error| error.error)?;+ }+ }+ }+ if (result.is_ok() || rejection.is_some())+ && read(&path, LIMIT)?.as_deref() == Some(bytes)+ {+ fs::remove_file(&path)?;+ }+ }+ Ok(())+ })();+ failed |= result.is_err() || finish.is_err();+ }+ ensure!(+ !failed,+ "session drain had failed items; inspect status.json, pending/ and rejected/"+ );+ Ok(())+}++#[cfg(all(test, unix))]+mod tests;diff --git a/engine/crates/xerj-autoindex/src/session_memory/tests.rs b/engine/crates/xerj-autoindex/src/session_memory/tests.rsnew file mode 100644index 000000000..4a4b58071--- /dev/null+++ b/engine/crates/xerj-autoindex/src/session_memory/tests.rs@@ -0,0 +1,512 @@+use super::*;++fn project() -> tempfile::TempDir {+ tempfile::tempdir().unwrap()+}+fn install(path: &Path, dry: bool) -> Result<Vec<String>> {+ configure(+ path,+ "http://127.0.0.1:9200",+ dry,+ false,+ Path::new("/opt/xerj's bin/xerj"),+ )+}++#[test]+fn dry_run_has_no_side_effects_and_plain_init_is_not_changed() {+ let project = project();+ install(project.path(), true).unwrap();+ assert_eq!(fs::read_dir(project.path()).unwrap().count(), 0);+}++#[test]+fn merge_is_idempotent_and_remove_preserves_foreign_hooks_and_all_data() {+ let project = project();+ fs::create_dir(project.path().join(".claude")).unwrap();+ let original = json!({"permissions":{"allow":["Read"]},"hooks":{"SessionStart":[{"hooks":[{"type":"command","command":"echo original"}]}],"Stop":[{"hooks":[]}]}});+ let settings = project.path().join(SETTINGS);+ fs::write(&settings, encode(&original).unwrap()).unwrap();+ install(project.path(), false).unwrap();+ let first = fs::read(&settings).unwrap();+ let parsed: Value = serde_json::from_slice(&first).unwrap();+ assert_eq!(parsed["hooks"]["SessionStart"].as_array().unwrap().len(), 2);+ assert_eq!(parsed["permissions"], original["permissions"]);+ assert!(parsed["hooks"]["SessionEnd"][0]["hooks"][0]["command"]+ .as_str()+ .unwrap()+ .contains("'\\''"));+ let config_path = state(project.path()).join("config.json");+ let first_config = fs::read(&config_path).unwrap();+ install(project.path(), false).unwrap();+ assert_eq!(fs::read(&settings).unwrap(), first);+ assert_eq!(fs::read(&config_path).unwrap(), first_config);+ let archive = state(project.path()).join("archives/keep.jsonl");+ fs::write(&archive, b"kept").unwrap();+ configure(+ project.path(),+ "http://127.0.0.1:9200",+ false,+ true,+ Path::new("/new/xerj"),+ )+ .unwrap();+ let removed: Value = serde_json::from_slice(&fs::read(&settings).unwrap()).unwrap();+ assert_eq!(removed, original);+ assert_eq!(fs::read(archive).unwrap(), b"kept");+ assert!(config(&project.path().canonicalize().unwrap()).is_err());+ configure(+ project.path(),+ "http://127.0.0.1:9200",+ false,+ true,+ Path::new("/new/xerj"),+ )+ .unwrap();+}++#[test]+fn malformed_settings_or_edited_hooks_are_refused_without_rewriting() {+ let project = project();+ fs::create_dir(project.path().join(".claude")).unwrap();+ let settings = project.path().join(SETTINGS);+ for input in [+ b"{bad".as_slice(),+ b"[]",+ b"{\"hooks\":{\"SessionEnd\":false}}",+ ] {+ fs::write(&settings, input).unwrap();+ assert!(install(project.path(), false).is_err());+ assert_eq!(fs::read(&settings).unwrap(), input);+ assert!(!state(project.path()).exists());+ }+ fs::remove_file(&settings).unwrap();+ install(project.path(), false).unwrap();+ let config_path = state(project.path()).join("config.json");+ let before = fs::read(&config_path).unwrap();+ let mut modified: Value = serde_json::from_slice(&fs::read(&settings).unwrap()).unwrap();+ modified["hooks"]["SessionEnd"][0]["hooks"][0]["timeout"] = json!(10);+ fs::write(&settings, encode(&modified).unwrap()).unwrap();+ assert!(install(project.path(), false).is_err());+ assert_eq!(fs::read(&config_path).unwrap(), before);+ assert_eq!(+ serde_json::from_slice::<Value>(&fs::read(settings).unwrap()).unwrap(),+ modified+ );+}++#[test]+fn uninstall_refuses_an_active_worker_and_keeps_configuration_intact() {+ let project = project();+ install(project.path(), false).unwrap();+ let settings = fs::read(project.path().join(SETTINGS)).unwrap();+ let _guard = lock(&state(&project.path().canonicalize().unwrap()).join("worker.lock")).unwrap();+ assert!(configure(+ project.path(),+ "http://127.0.0.1:9200",+ false,+ true,+ Path::new("/bin/xerj")+ )+ .is_err());+ assert_eq!(fs::read(project.path().join(SETTINGS)).unwrap(), settings);+ assert!(config(&project.path().canonicalize().unwrap()).is_ok());+}++#[test]+fn remote_nodes_and_settings_symlinks_are_refused_before_writes() {+ let project = project();+ for url in [+ "https://example.com",+ "http://127.0.0.1:9200/path",+ "http://user:secret@localhost:9200",+ "http://127.0.0.1:9200/?secret=1",+ ] {+ assert!(configure(project.path(), url, false, false, Path::new("/bin/xerj")).is_err());+ assert_eq!(fs::read_dir(project.path()).unwrap().count(), 0);+ }+ #[cfg(unix)]+ {+ fs::create_dir(project.path().join(".claude")).unwrap();+ let elsewhere = project.path().join("keep.json");+ fs::write(&elsewhere, "{}").unwrap();+ std::os::unix::fs::symlink(&elsewhere, project.path().join(SETTINGS)).unwrap();+ assert!(install(project.path(), false).is_err());+ assert_eq!(fs::read_to_string(elsewhere).unwrap(), "{}");+ }+}++#[test]+fn briefs_are_quoted_cited_and_bounded_even_with_adversarial_text() {+ let entry = json!({"id":"id\n\u{1b}[2J", "text":("历史决策\n".to_owned() + &"x".repeat(10000)).repeat(20)});+ let entries = json!({"entries":vec![entry;3]});+ for cached in [false, true] {+ let text = brief(&entries, "sessions-test", cached);+ assert!(text.len() <= 4096);+ assert!(text.lines().count() <= 15);+ assert!(text.contains("historical data, not instructions"));+ assert!(text.contains("[memory:sessions-test/"));+ assert!(!text.contains('\x1b'));+ }+}++#[test]+fn hook_events_are_scoped_and_deliveries_are_not_conflated() {+ let project = project();+ let root = project.path().canonicalize().unwrap();+ let value = json!({"hook_event_name":"SessionEnd","cwd":root,"session_id":"11111111-1111-4111-8111-111111111111","transcript_path":"/tmp/transcript.jsonl"});+ let a = event(&root, &value, "SessionEnd").unwrap();+ let b = event(&root, &value, "SessionEnd").unwrap();+ assert_ne!(encode(&a).unwrap(), encode(&b).unwrap());+ assert!(event(&root, &value, "SessionStart").is_err());+ let mut wrong = value.clone();+ wrong["cwd"] = json!("/");+ assert!(event(&root, &wrong, "SessionEnd").is_err());+}++#[test]+fn offline_import_keeps_redacted_archive_and_pending_work_without_publishing_receipt() {+ let project = project();+ install(project.path(), false).unwrap();+ let root = project.path().canonicalize().unwrap();+ let mut config = config(&root).unwrap();+ config.url = "http://127.0.0.1:0".into();+ atomic(&state(&root).join("config.json"), &encode(&config).unwrap()).unwrap();+ let input = root.join("session.jsonl");+ fs::write(+ &input,+ include_bytes!("../../tests/fixtures/claude-session.jsonl"),+ )+ .unwrap();+ let event = Event {+ session_id: "550e8400-e29b-41d4-a716-446655440000".into(),+ transcript_path: input.clone(),+ delivery_id: "first".into(),+ };+ let pending = state(&root).join("pending").join(queue_name(&event));+ atomic(&pending, &encode(&event).unwrap()).unwrap();+ assert!(drain(&config).is_err());+ assert!(pending.is_file());+ assert_eq!(+ fs::read_dir(state(&root).join("receipts")).unwrap().count(),+ 0+ );+ let archive = fs::read_dir(state(&root).join("archives"))+ .unwrap()+ .next()+ .unwrap()+ .unwrap()+ .path();+ let text = fs::read_to_string(&archive).unwrap();+ assert!(text.contains("[REDACTED_EMAIL]"));+ assert!(!text.contains("[email protected]"));+ assert!(!text.contains("DO_NOT_EXPORT"));+ let current = fs::read_dir(state(&root).join("exports"))+ .unwrap()+ .next()+ .unwrap()+ .unwrap()+ .path();+ assert_eq!(fs::read(¤t).unwrap(), text.as_bytes());+ let mut malformed = fs::read(&input).unwrap();+ malformed.extend_from_slice(b"{bad-private-tail");+ fs::write(&input, malformed).unwrap();+ assert!(drain(&config).is_err());+ assert_eq!(fs::read(¤t).unwrap(), text.as_bytes());+ assert_eq!(fs::read(&archive).unwrap(), text.as_bytes());+}++#[test]+fn event_session_must_match_the_actual_transcript_before_archive_publication() {+ let project = project();+ install(project.path(), false).unwrap();+ let root = project.path().canonicalize().unwrap();+ let config = config(&root).unwrap();+ let input = root.join("session.jsonl");+ fs::write(+ &input,+ include_bytes!("../../tests/fixtures/claude-session.jsonl"),+ )+ .unwrap();+ let event = Event {+ session_id: "11111111-1111-4111-8111-111111111111".into(),+ transcript_path: input,+ delivery_id: "first".into(),+ };+ let guard = lock(&state(&root).join("worker.lock")).unwrap();+ assert!(import(&config, &event, &state(&root), &guard)+ .unwrap_err()+ .to_string()+ .contains("different session"));+ assert_eq!(+ fs::read_dir(state(&root).join("archives")).unwrap().count(),+ 0+ );+ assert_eq!(+ fs::read_dir(state(&root).join("exports")).unwrap().count(),+ 0+ );+}++#[test]+fn matching_unowned_or_duplicated_groups_are_never_claimed_or_removed() {+ let project = project();+ fs::create_dir(project.path().join(".claude")).unwrap();+ let root = project.path().canonicalize().unwrap();+ let wanted = hooks(&root, Path::new("/opt/xerj's bin/xerj")).unwrap();+ let settings = project.path().join(SETTINGS);+ let foreign = json!({"hooks":{"SessionStart":[wanted["SessionStart"].clone()]}});+ fs::write(&settings, encode(&foreign).unwrap()).unwrap();+ assert!(install(project.path(), false).is_err());+ assert!(!state(&root).exists());+ fs::write(&settings, "{}").unwrap();+ install(project.path(), false).unwrap();+ let mut duplicate: Value = serde_json::from_slice(&fs::read(&settings).unwrap()).unwrap();+ duplicate["hooks"]["SessionEnd"]+ .as_array_mut()+ .unwrap()+ .push(wanted["SessionEnd"].clone());+ fs::write(&settings, encode(&duplicate).unwrap()).unwrap();+ assert!(configure(+ project.path(),+ "http://127.0.0.1:9200",+ false,+ true,+ Path::new("/other/xerj")+ )+ .is_err());+ assert_eq!(+ serde_json::from_slice::<Value>(&fs::read(settings).unwrap()).unwrap(),+ duplicate+ );+}++#[test]+fn review_unicode_format_controls_are_removed_at_import_and_brief_boundaries() {+ let hostile = "safe\u{202e}\u{200b}\u{e0049}\u{2066}\u{feff}\u{ad}中文";+ assert_eq!(clean(hostile, 200), "safe中文");+ for cached in [false, true] {+ let text = brief(+ &json!({"entries":[{"id":hostile,"text":hostile}]}),+ "sessions-test",+ cached,+ );+ assert!(text.contains("safe中文"));+ for c in [+ '\u{202e}',+ '\u{200b}',+ '\u{e0049}',+ '\u{2066}',+ '\u{feff}',+ '\u{ad}',+ ] {+ assert!(!text.contains(c), "unsafe format character {c:?}");+ }+ }+}++#[test]+fn review_bad_queue_entries_do_not_prevent_later_imports() {+ let project = project();+ install(project.path(), false).unwrap();+ let root = project.path().canonicalize().unwrap();+ let mut config = config(&root).unwrap();+ config.url = "http://127.0.0.1:0".into();+ let directory = state(&root);+ atomic(&directory.join("config.json"), &encode(&config).unwrap()).unwrap();+ let pending = directory.join("pending");+ let missing = Event {+ session_id: "11111111-1111-4111-8111-111111111111".into(),+ transcript_path: root.join("missing"),+ delivery_id: "missing".into(),+ };+ atomic(+ &pending.join(queue_name(&missing)),+ &encode(&missing).unwrap(),+ )+ .unwrap();+ fs::write(pending.join("corrupt.json"), b"{bad").unwrap();+ let input = root.join("valid.jsonl");+ fs::write(+ &input,+ include_bytes!("../../tests/fixtures/claude-session.jsonl"),+ )+ .unwrap();+ let valid = Event {+ session_id: "550e8400-e29b-41d4-a716-446655440000".into(),+ transcript_path: input,+ delivery_id: "valid".into(),+ };+ let oversized_path = root.join("oversized.jsonl");+ File::create(&oversized_path)+ .unwrap()+ .set_len(16 * 1024 * 1024 + 1)+ .unwrap();+ let oversized = Event {+ session_id: missing.session_id.clone(),+ transcript_path: oversized_path,+ delivery_id: "oversized".into(),+ };+ atomic(+ &pending.join(queue_name(&oversized)),+ &encode(&oversized).unwrap(),+ )+ .unwrap();+ let mismatch = Event {+ session_id: missing.session_id.clone(),+ transcript_path: valid.transcript_path.clone(),+ delivery_id: "mismatch".into(),+ };+ atomic(+ &pending.join(queue_name(&mismatch)),+ &encode(&mismatch).unwrap(),+ )+ .unwrap();+ let valid_path = pending.join(queue_name(&valid));+ atomic(&valid_path, &encode(&valid).unwrap()).unwrap();+ assert!(drain(&config).is_err());+ assert_eq!(+ fs::read_dir(directory.join("archives")).unwrap().count(),+ 1,+ "valid sibling was not attempted"+ );+ assert!(valid_path.exists());+ assert!(!pending.join(queue_name(&missing)).exists());+ assert!(!pending.join("corrupt.json").exists());+ let rejected = directory.join("rejected");+ assert_eq!(+ fs::read_dir(&rejected)+ .unwrap()+ .filter(|entry| entry.as_ref().unwrap().path().extension().unwrap() == "json")+ .count(),+ 4+ );+ assert_eq!(+ fs::read(rejected.join(queue_name(&missing))).unwrap(),+ encode(&missing).unwrap()+ );+ assert_eq!(fs::read(rejected.join("corrupt.json")).unwrap(), b"{bad");+ for (name, reason) in [+ (queue_name(&missing), "transcript missing"),+ (queue_name(&oversized), "exceeds 16 MiB"),+ (queue_name(&mismatch), "different session"),+ ("corrupt.json".into(), "invalid queue JSON"),+ ] {+ let status: Value = serde_json::from_slice(+ &fs::read(rejected.join(name).with_extension("status")).unwrap(),+ )+ .unwrap();+ assert_eq!(status["state"], "rejected");+ assert!(status["reason"].as_str().unwrap().contains(reason));+ }+ assert_eq!(fs::read_dir(&pending).unwrap().count(), 1);+ // Fixing a missing source does not silently revive a rejected delivery.+ fs::write(&missing.transcript_path, b"{}").unwrap();+ assert!(drain(&config).is_err());+ assert_eq!(fs::read_dir(&pending).unwrap().count(), 1);+}++#[test]+fn review_remove_dry_run_names_the_action_and_preserves_settings() {+ let project = project();+ install(project.path(), false).unwrap();+ let before = fs::read(project.path().join(SETTINGS)).unwrap();+ let output = configure(+ project.path(),+ "http://127.0.0.1:9200",+ true,+ true,+ Path::new("/bin/xerj"),+ )+ .unwrap();+ assert!(output[0].starts_with("would disable"));+ assert_eq!(fs::read(project.path().join(SETTINGS)).unwrap(), before);+}++#[test]+fn review_transient_failure_does_not_skip_other_retryable_sessions() {+ let project = project();+ install(project.path(), false).unwrap();+ let root = project.path().canonicalize().unwrap();+ let directory = state(&root);+ let mut config = config(&root).unwrap();+ config.url = "http://127.0.0.1:0".into();+ atomic(&directory.join("config.json"), &encode(&config).unwrap()).unwrap();+ for (name, sid) in [+ ("first", "550e8400-e29b-41d4-a716-446655440000"),+ ("second", "11111111-1111-4111-8111-111111111111"),+ ] {+ let input = root.join(format!("{name}.jsonl"));+ let source =+ String::from_utf8(include_bytes!("../../tests/fixtures/claude-session.jsonl").to_vec())+ .unwrap();+ fs::write(+ &input,+ source.replace("550e8400-e29b-41d4-a716-446655440000", sid),+ )+ .unwrap();+ let event = Event {+ session_id: sid.into(),+ transcript_path: input,+ delivery_id: name.into(),+ };+ atomic(+ &directory.join("pending").join(queue_name(&event)),+ &encode(&event).unwrap(),+ )+ .unwrap();+ }+ assert!(drain(&config).is_err());+ assert_eq!(fs::read_dir(directory.join("archives")).unwrap().count(), 2);+ assert_eq!(fs::read_dir(directory.join("pending")).unwrap().count(), 2);+ assert_eq!(fs::read_dir(directory.join("receipts")).unwrap().count(), 0);+ assert!(!directory.join("rejected").exists());+}++#[test]+fn review_rejection_publication_failure_retains_pending_bytes_for_recovery() {+ let project = project();+ install(project.path(), false).unwrap();+ let root = project.path().canonicalize().unwrap();+ let directory = state(&root);+ let config = config(&root).unwrap();+ let path = directory.join("pending/corrupt.json");+ fs::write(&path, b"{original-invalid").unwrap();+ fs::write(directory.join("rejected"), b"foreign-blocker").unwrap();+ assert!(drain(&config).is_err());+ assert_eq!(fs::read(&path).unwrap(), b"{original-invalid");+ assert_eq!(+ fs::read(directory.join("rejected")).unwrap(),+ b"foreign-blocker"+ );+ fs::remove_file(directory.join("rejected")).unwrap();+ assert!(drain(&config).is_err());+ assert!(!path.exists());+ assert_eq!(+ fs::read(directory.join("rejected/corrupt.json")).unwrap(),+ b"{original-invalid"+ );+}++#[test]+fn review_moved_executable_updates_only_owned_hook_groups() {+ let project = project();+ install(project.path(), false).unwrap();+ configure(+ project.path(),+ "http://127.0.0.1:9200",+ false,+ false,+ Path::new("/new/version/xerj"),+ )+ .unwrap();+ let settings: Value =+ serde_json::from_slice(&fs::read(project.path().join(SETTINGS)).unwrap()).unwrap();+ for event in ["SessionStart", "SessionEnd"] {+ let groups = settings["hooks"][event].as_array().unwrap();+ assert_eq!(groups.len(), 1);+ let command = groups[0]["hooks"][0]["command"].as_str().unwrap();+ assert!(command.contains("/new/version/xerj"));+ assert!(!command.contains("/opt/xerj"));+ }+}diff --git a/engine/crates/xerj-autoindex/src/sessions.rs b/engine/crates/xerj-autoindex/src/sessions.rsindex 8b101a1ea..8a3263257 100644--- a/engine/crates/xerj-autoindex/src/sessions.rs+++ b/engine/crates/xerj-autoindex/src/sessions.rs@@ -36,6 +36,9 @@ text and tools/reasoning; uses line-and-byte-digest event IDs, no inferred parenor files. exec --json and external history_base are unsupported.Input <=512 MiB; line <=1 MiB; turn text <=256 KiB; <=100000 messages.Exit 0: exported, JSON summary on stdout; 2: usage; 1: import/publication error.++Optional automation (Claude Code, macOS/Linux): xerj init --memory.+Disable with xerj init --memory --remove; existing exports and node data are kept.";#[derive(Debug, Serialize)]@@ -418,6 +421,9 @@ fn import_session(input: &Path, output: &Path, format: SessionFormat) -> Result<}pub fn run_cli(args: &[String]) -> i32 {+ if args.first().is_some_and(|verb| verb == "hook") {+ return crate::session_memory::run(&args[1..]);+ }if args.is_empty() || args.iter().any(|a| matches!(a.as_str(), "--help" | "-h")) {print!("{HELP}");return 0;diff --git a/engine/crates/xerj-rerank/tests/egress_inventory.rs b/engine/crates/xerj-rerank/tests/egress_inventory.rsindex 6b4111b97..5226847aa 100644--- a/engine/crates/xerj-rerank/tests/egress_inventory.rs+++ b/engine/crates/xerj-rerank/tests/egress_inventory.rs@@ -86,6 +86,12 @@ const KNOWN: &[(&str, Role)] = &[Role::Node("**Cluster transport**"),),("xerj-mcp/src/lib.rs", Role::Client("`xerj mcp`")),+ // Opt-in host hooks send redacted exports/excerpts only to the configured+ // loopback node, with redirects and proxies disabled. No node egress added.+ (+ "xerj-autoindex/src/session_memory.rs",+ Role::Client("`xerj sessions hook`"),+ ),// Batch retrieval uses the same operator-selected node URL and auth as// `xerj mcp` (localhost:9200 by default). It sends queries to /_msearch// and optionally reads mappings; it adds no node-side egress path.diff --git a/tools/session-memory/README.md b/tools/session-memory/README.mdnew file mode 100644index 000000000..7bab6022a--- /dev/null+++ b/tools/session-memory/README.md@@ -0,0 +1,148 @@+# Opt-in Claude Code session handoff++`xerj init --memory` connects the existing Claude Code transcript importer+(#2406) to SessionEnd and SessionStart. This is the Claude Code part of+[#2645](https://github.com/xerj-org/xerj/issues/2645); Codex's existing offline+importer (#2413) is unchanged. Codex lifecycle integration is deferred.++On macOS/Linux, from the trusted project folder, with your local node already+running:++```sh+xerj init --memory --url http://localhost:9200 --dry-run+xerj init --memory --url http://localhost:9200+```++Restart the Claude Code session to load the hooks. This option manages only+`.claude/settings.local.json` and the private `.claude/xerj-memory/` directory;+ordinary `xerj init` keeps its existing MCP/teaching-file behavior. It does not+start a node, alter other agents or install hooks unless explicitly requested.+Existing settings and unrelated hook groups are retained. Settings backups+are kept inside the private data directory. An edited, duplicated or unowned+matching hook is refused rather than claimed or silently overwritten.++The node URL must be loopback, with no credentials, path, query or fragment.+Keys are never saved in configuration: `XERJ_API_KEY`, or the existing local+admin-key discovery, supplies auth at runtime. Redirects and HTTP proxies are+disabled for hook memory requests. A moved project needs a fresh configuration;+existing configuration bound to another canonical project path is refused.+Hooks also retain the absolute executable path. If an upgrade moves the binary,+rerun `xerj init --memory --url <same-loopback-url>` from the project with the+new binary, then restart Claude Code; owned hook groups are updated in place.++## What happens++- SessionEnd writes a private queue entry and starts a detached worker. It+ prints nothing and does not wait for import or indexing. Independent delivery+ filenames prevent an arriving event from being erased by an older job.+- The worker uses the existing native PII-redacting importer. It saves an+ immutable content-addressed JSONL archive and replaces this session's current+ export only after the complete transcript validates. Tool results, thinking+ and tool payloads stay excluded; explicitly referenced file paths are kept+ under the existing importer contract. Different-session input is refused.+- Current exports are dry-run first, then reconciled with ordinary autoindex+ using one worker, `--no-graph --no-semantic`, the printed project prefix and a+ private journal. Opting in authorizes these automatic runs (`--max-minutes 0`);+ the subprocess has a shared 60-second dry-run/index deadline. The private+ `last-estimate.txt` records the dry-run output. No model is downloaded by the+ indexer. Server-side memory embedding follows the node's configured mode;+ the default is lexical hashing, not neural understanding.+- After indexing, the last four main-conversation text turns are excerpted+ into one `/_memory/<namespace>` entry per session. Archive filename/line and+ message ID accompany each excerpt. Repeated events reconcile and upsert the+ same session identity, without adding duplicate corpus rows or memory IDs.+- SessionStart reads the namespace's three most recent entries, including+ explicit notes you stored there. It prints at most 15 lines / 4096 UTF-8 bytes+ of quoted historical text. This is not an LLM summary, a decision classifier+ or an instruction source. On a node/read failure it can use the last locally+ saved session excerpt, explicitly labeled Cached. Nothing is printed when+ neither source is available. The memory request has a 500 ms total timeout.+- The full redacted conversation stays in the current export/search index;+ short excerpts are just a handoff. Original transcript files are never changed.++Hook stdin follows the [official Claude Code hook protocol](https://code.claude.com/docs/en/hooks):+`hook_event_name`, `cwd`, `session_id`, `transcript_path`. `cwd` must belong to+the configured project; supported transcript/session UUID rules are those of+`xerj sessions import --format claude-code`. Hook commands use quoted absolute+binary/project paths and never interpret event data as shell commands.++Imported memory excerpts and displayed briefs strip Unicode control and format+characters (including bidi overrides, zero-width format marks and tag characters).+This also sanitizes cached briefs written by an earlier version. Archives and+search exports preserve the existing importer's sanitized source text.++## Failures, retention and removal++Hooks always exit 0 and keep errors out of the agent session. Inspect+`.claude/xerj-memory/status.json`, `pending/`, and `last-estimate.txt` to see+the most recent item attempt. `pending/` is the authority for remaining work;+a successful latest item does not imply the whole queue is empty. Each item+is attempted independently.+Missing or over-16-MiB transcripts, session mismatches and corrupt queue JSON+are moved to private `rejected/` (original queue bytes plus a `.status` reason).+These deliveries are not retried automatically. Restore the missing source or+correct the session input, then end that Claude Code session again to enqueue+a new delivery; oversized transcripts require explicit manual import instead.+Do not edit queue identities or copy rejected files back into `pending/`.+Other failures remain queued; a later SessionStart or SessionEnd retries them. Each drain considers at most+eight directory entries and checks its 90-second window between jobs. It is not a daemon.+A retryable failure does not stop other entries in that batch. Each indexing+attempt retains its dry-run estimate before the real autoindex run, under one+60-second child deadline; large-history cost has not been measured.+Malformed/growing transcripts, offline nodes and indexing failures keep the+last valid export/receipt. Node publication, memory upsert and local receipt+are separate steps: a failure may leave indexed data before a receipt is+committed; replay reconciles the same identities rather than duplicating them.++Automatic capture is limited to 16 MiB per transcript; larger transcripts need+explicit manual import. The importer still enforces its existing per-line,+per-turn and record limits. There is no automatic retention purge: archives,+queue entries and node data can accumulate. The PII redactor handles its+existing email/IP/card/SSN/phone patterns; **it does not remove every secret or+personal name**. Only enable this for sessions you intend to retain locally.+Private folders use mode 0700 and new data files mode 0600. Hook/settings/state+symlinks are refused. The project and its configuration are trusted, owned by+the operator; this is not a sandbox against another process running as you.++```sh+xerj init --memory --remove --dry-run+xerj init --memory --remove+```++Removal refuses a busy worker and tells you to retry, preserving the current+settings. Its lease remains with an autoindex child if the supervisor dies.+Once removal succeeds, stale hook commands are silent no-ops. Removal keeps+archives, exports, pending/rejected jobs, backups, catalog/index data and memories.+Restore an edited managed hook from your private settings backup before+removing; unrelated settings are never restored wholesale.++To delete data intentionally, first disable the hooks. The setup output and+private config name the exact prefix/namespace; substitute those values below:++- Run `xerj autoindex remove --prefix <printed-prefix> --root <absolute-project>/.claude/xerj-memory/exports --url <node-url>` and inspect its ownership plan before confirming. This removes only the managed export corpus, not other project indices.+- Delete the dedicated memory namespace with the documented `DELETE /_memory/<printed-namespace>` API, using the same node/auth. This also deletes any manual notes you put in that namespace.+- After node data has been handled, delete this project's `.claude/xerj-memory/` directory if you also want to erase archives, cache, pending work and backups. Re-enabling afterward starts fresh; leaving data in place permits retries and reuse.++## Reproduce without personal sessions++```sh+python3 tools/session-memory/smoke.py --binary engine/target/release/xerj+python3 tools/session-memory/smoke.py --binary engine/target/release/xerj --authenticated+```++The script uses synthetic transcripts, a copied executable, quoted paths and a+private temporary node/data/project. It invokes the commands actually installed+in Claude settings, checks the searchable records and memory, repeats an event,+changes input, tests an offline cached start and malformed-tail refusal, restarts+the node, retries pending work and removes hooks while checking data retention.+It stops its own node and removes its own temporary files. It does not launch+Claude Code, use your transcripts or prove a specific host version's shutdown+behavior. Windows, remote nodes and Codex lifecycle hooks are unsupported here.++Reference implementation read before writing: `init.rs` configuration merge+and backup behavior, `sessions.rs` offline redaction/publication, `memory_api.rs`+namespaced store/list, and `state.rs` plus `journal_lock_inheritance.rs` for the+open-file-description lock lifetime. Exact pinned locations and measured+results are recorded in [VERIFICATION.md](VERIFICATION.md). No peer code was+copied. Claude Code's hook documentation supplies the wire/event contract;+its SessionEnd budget is why import/index work lives in a separate process.diff --git a/tools/session-memory/VERIFICATION.md b/tools/session-memory/VERIFICATION.mdnew file mode 100644index 000000000..042ab96e8--- /dev/null+++ b/tools/session-memory/VERIFICATION.md@@ -0,0 +1,266 @@+# Session handoff verification (2026-11-01)++Written and executed by OpenAI Codex, operated by Jarvis636431, on macOS arm64+with Rust 1.97.1. Implementation `b099cde6` is based directly on upstream+`720b9990`; no unmerged prerequisite or duplicate prerequisite commits.+The source-built candidate reports `xerj v1.0.0-rc.100`. These receipts do not+claim this feature exists in the published release.++## Regression and final checks++Before implementation, added and ran from `engine/`:++```sh+cargo +1.97.1 test --locked --offline -p xerj-autoindex memory_hooks_are_an_explicit_init_opt_in+```++Observed **0 passed, 1 failed**:++```text+memory opt-in refused: Some("xerj init: unexpected argument '--memory'")+```++The final crate run passes this regression and the ten new session-memory+unit tests. They cover configuration merge/no-op/remove, dry-run, malformed+settings, edited/duplicated ownership, worker exclusion, remote/symlink refusal,+bounded quoted citations, independent deliveries, offline retained work,+redaction and mismatched-session refusal.++Commands from `engine/`:++```sh+cargo +1.97.1 fmt --all+cargo +1.97.1 fmt --all -- --check+cargo +1.97.1 build --locked --offline --release -j 32 -p xerj-autoindex -p xerj-server -p es-yaml-runner+TMPDIR=/private/tmp cargo +1.97.1 test --locked --offline -p xerj-autoindex -- --test-threads=1 --skip content::tests::lossy_display_collisions_keep_distinct_alias_identities --skip corpus_data_skip_tests::corpus_index_summary_splits_indexed_extensions_from_ignored_data --skip inventory_delta_tests::a_non_utf8_excluded_name_is_matched_on_disk_via_path_id+cargo +1.97.1 clippy --locked --offline --no-deps -p xerj-autoindex --all-targets -- -D warnings -A dead_code+```++Observed: scoped release build and formatting passed; **2017 passed, 0 failed,+5 existing ignored, 3 filtered** across the crate run. The filters are existing+macOS filesystem exclusions recorded in [session-import verification](../session-import/VERIFICATION.md).+No existing assertion or skip policy was weakened. Local mock HTTP ports were+allowed. Clippy passed with the existing `content.rs::resolve_reporting`+dead-code allowance. No dependency/lockfile change. Python compilation,+diff whitespace and workflow timeout guard passed (41 jobs / 13 workflows).++Executed binary SHA-256:+`ff806fdede92049e24de9e98f243a5c4eed1fdf3ebd0b668eff7295deb4234ed`.+The copied executable hashed is the same private copy actually run.++## CLI/HTTP workflow++From the repository root:++```sh+python3 tools/session-memory/smoke.py --binary engine/target/release/xerj+python3 tools/session-memory/smoke.py --binary engine/target/release/xerj --authenticated+```++Both passed on the same candidate, with an insecure private node and with+default node authentication plus a temporary runtime `XERJ_API_KEY`:++```text+PASS dry-run, quoted paths, install and unchanged re-install+Synthetic input: 645 bytes; dry-run output captured+PASS redaction, 2 searchable records, one memory and cited SessionStart brief+PASS duplicate event idempotence and changed-transcript replacement (3 records, one memory)+PASS offline cached brief, malformed-tail atomicity and retained-job recovery after restart+PASS uninstall keeps foreign hooks and data; stale hook is a silent no-op+PASS documented ownership-plan cleanup and dedicated memory deletion+```++The harness invokes commands read from installed settings, including binary+and project paths containing apostrophes/spaces. Every hook exited 0 with empty+stderr; SessionEnd stdout was empty. Each run's two nodes exited 0. The archive+excluded fixture email, thinking and tool payload sentinels. Repeated events+kept one archive/memory and stable corpus counts; changed input kept two+immutable archives and replaced the current session export. These are small+correctness fixtures, not benchmarks.++The scoped ES-YAML runner used another fresh temporary node, with+`--url <private-node-url> --dir engine/tests/es-compat-yaml/yaml` from the root:++```text+ES-COMPAT YAML RUNNER · 233 files+1453 passed · 0 failed · 10 skipped · 1463 total+74 cases ran catch-lenient (an expected error XERJ does not raise yet; see catch-lenient.txt)+private conformance node exit 0+```++Skip/catch-lenient policies are unchanged. This is the repository suite,+not complete ES-protocol coverage. Hosted PR CI remains separate.++## Contribution-review audit and limits++- Effective diff preserves ordinary init, explicit import, upstream workflows,+ parser contracts and unrelated hooks. Production changes stay in the+ autoindex client crate. There is no new server API, engine storage format,+ dependency or Cargo feature. Setup is explicit and loopback-only.+- Queue delivery IDs, immutable archives, current exports, stable memory IDs+ and post-publication receipts serve different purposes. Failed publication+ retains pending work; replay reconciles identities. Memory/index/receipt+ publication is not one cross-system transaction. Removal rechecks its lease+ and keeps data; an autoindex child inherits the lease if its supervisor dies.+- Real tests cover first run, unchanged/changed input, malformed partial input,+ pending retry, node restart, cached reads, removal and documented cleanup.+ Config tests refuse busy workers and edited/foreign/duplicated/symlinked+ state. No model or hub corpus is used.+- Unsupported: Windows, remote nodes, Codex lifecycle hooks. Not exercised:+ actual Claude Code host delivery/process cleanup, Linux execution, forced+ worker SIGKILL, hostile same-user filesystem races, large histories, every+ transcript variant, authenticated automatic key discovery, or neural nodes.+ Authenticated smoke uses an explicit runtime key. There is no comprehensive+ secret-redaction guarantee, automatic retention purge or old-state migration.+ Unknown versions/foreign project configurations refuse.+- Engine migrations, query accelerators, feature-gated algorithms, ambiguous+ schema matching and performance/resource optimization are not applicable:+ those paths are unchanged. Existing feedback asks are unchanged; the+ agent-facing-file study was read before editing init help.++Reference code retrieved before implementing (baseline `720b9990`):+`init.rs:415` (backup), `init.rs:448` (merge), `init.rs:657` (ordinary setup),+`sessions.rs:280` (bounded redacting import), `memory_api.rs:389` / `1318`+(store/list), `state.rs:817` / `822` and+`tests/journal_lock_inheritance.rs:108` (open-file-description lock lifetime).+No peer implementation was copied. The [official Claude Code hook protocol](https://code.claude.com/docs/en/hooks)+was retrieved on this date for event fields, context output and SessionEnd's+short budget; import/index work is detached for that reason.+++## PR review follow-up (2026-11-02)++Review target: `256fa5cbb72b65656268a89da1282c972a2414d7`, still directly+based on upstream `720b9990`. This follow-up addresses the queue/Unicode+blockers and the egress-inventory CI failure, without changing the node API,+index schema, dependencies, hooks protocol or existing test exclusions.++Observed before production changes: the three new `review_` regressions failed+for the intended reasons: the healthy queue sibling had no archive, removal+preview did not say `would disable`, and `clean()` retained U+202E, U+200B,+U+E0049, U+2066, U+FEFF and U+00AD. The pre-fix run selected two unrelated+existing tests as well; both passed. The hosted failing egress job independently+identified the unclassified `session_memory.rs` client.++The queue now snapshots a bounded batch before retiring entries and attempts+siblings even after import or finalization failure. Only explicit terminal+pre-publication refusals (missing/oversized transcript, session/queue identity+mismatch, malformed queue JSON) are isolated in private `rejected/`; original+bytes and a payload-free reason file are retained before removing pending work.+Other failures, including an incomplete transcript and an offline node, remain+retryable. Existing queues/configs need no migration. Status version stays 1;+`state: rejected` is additive and an unknown session is JSON null.++The new state-transition coverage includes multiple offline sessions, terminal+refusals mixed with a valid import, retention on quarantine-publication failure,+and retry after that obstruction is removed. Unicode category filtering uses+the existing regex dependency's Cc/Cf/Zl/Zp tables at both excerpt creation and+brief display, including old cached/remote data. No ASCII-only fallback or+hand-maintained Unicode ranges were introduced. Archives and full search+exports retain the existing importer's source-preservation contract.++Other review follow-ups: `--remove` without `--memory` is asserted, dry removal+names its action, the directory builder uses its mutable API on non-Unix too,+and the guide explains reinstalling hooks after a binary-path change. The+double dry-run/index traversal is retained under AGENTS.md's estimate-before-+index contract, with the shared deadline and unmeasured history cost documented.+The new client is classified as CLI-only, with loopback data flow documented+in RERANK.md and the air-gapped inventory; node DATA_EGRESS remains unchanged+because no node-side connection was added.++Local verification commands (from `engine/` unless a path says otherwise):++```sh+cargo fmt --all+cargo fmt --all -- --check+TMPDIR=/private/tmp cargo test --locked --offline -p xerj-autoindex --lib session_memory -- --test-threads=1+cargo test --locked --offline -p xerj-rerank --test egress_inventory+cargo clippy --locked --offline --no-deps -p xerj-autoindex --all-targets -- -D warnings -A dead_code+cargo build --locked --offline --release -j 32 -p xerj-autoindex -p xerj-server -p es-yaml-runner+```++The first complete-crate attempt in the restricted sandbox hit existing+loopback/process restrictions; its failures are not a passing receipt. The+host-permitted run uses the three pre-existing macOS filesystem filters listed+above. The existing `content::resolve_reporting` warning remains unchanged.+No existing assertion or skip policy is weakened.++Limits remain: actual Claude Code host delivery and shutdown, Linux/Windows+execution, worker SIGKILL, hostile same-user races and large histories are not+locally verified. No performance or broad retrieval-quality claims are made.++The added `--remove` assertion was also mutation-checked in isolation: extracted+the exact production `InitFlags`/`read_init_flags` and its regression module,+compiled with `rustc --test --edition=2021` plus the existing DEFAULT_URL+constant, and ran both controls. Unmodified source passed its one test (exit 0);+removing only the `if remove && !memory` guard failed it (exit 101). This was an+isolated parser/test-source probe, not a mutated full-crate build; production+source was not changed during concurrent builds.++Final-source test receipts: complete autoindex crate **2023 passed, 0 failed,+5 existing ignored, 3 existing macOS filesystem filters**; focused session+suite **16 passed, 0 failed**; complete default-feature rerank crate **59 passed,+0 failed**, including all four egress-inventory checks. The final autoindex+Clippy command above passed with only the explicit existing dead-code allowance.+Formatting/check, Python compilation (temporary external bytecode cache),+whitespace and workflow-timeout guard passed.++Contribution-review audit: effective ancestry stays independent of other open+PRs. Existing init/manual import, auth, hooks ownership, lock inheritance,+source redaction, retryable malformed tails, offline caches, restart and+explicit cleanup remain covered. Terminal rejection happens before archive,+export, index or memory publication; failure to retain a rejected entry keeps+its pending bytes. Original queue bytes and reason files are separate writes,+so this is not a cross-file crash-transaction guarantee. Memory text filtering+has no stored-index migration: new memory writes are clean and old cached or+remote briefs are sanitized on display. There is no retention purge or implicit+revival of a rejected delivery. The new recovery route is a corrected fresh+SessionEnd, or explicit manual import for oversized transcripts.++N/A: node API/ES semantics, index-schema migrations, clustering, model/provider+execution identity, telemetry, optional accelerators and feature algorithms:+none changes here. No performance claims or public release validation. Existing+operational setup/removal commands retain their contracts; the smoke exercises+the installed commands on owned temporary data, including negative opt-in/+dry-run checks, rejected input, offline/restart recovery and explicit cleanup.++Final shipping-profile build passed with the command above. Both committed+smoke invocations passed on executable SHA-256+`f860ba006cd1cedfb41589c609c912de213ba6416d400ad9184ba4ad5d5f6d11`:++- Insecure and API-key-authenticated temporary lexical nodes used the same+ source-built rc.100 executable bytes; no personal transcripts or hub packs.+- Missing/oversized/mismatched/corrupt deliveries were retained as four rejected+ queue files plus four reason files; the healthy sibling completed normally.+- The synthetic transcript was 669 bytes. Searchable records changed from two+ to three while memory remained one entry; duplicate events did not grow either.+- New memory writes and displayed briefs excluded bidi, zero-width and tag+ characters. An intentionally contaminated legacy offline cache was filtered+ again at display. Existing redaction/citation and byte/line bounds passed.+- The executable was moved to a new quoted path, hooks reconfigured, and the+ installed commands then executed from that path. Plain `--remove` refused+ before settings/state writes, and removal dry-run said `would disable`.+- Offline cache, malformed-tail publication refusal, retained-job recovery+ after node restart, ownership-plan cleanup and memory deletion passed.+ Uninstall retained all rejected bytes/reasons and foreign hooks. Stale hooks+ stayed silent. Every observed hook invocation exited 0 without stderr;+ every private node stopped with exit 0.++Final same-snapshot ES-YAML gate, against a fresh temporary insecure lexical+node, using the unchanged release runner and `--dir tests/es-compat-yaml/yaml`+from `engine/`:++```text+ES-COMPAT YAML RUNNER · 233 files+1453 passed · 0 failed · 10 skipped · 1463 total+74 cases ran catch-lenient+Runner exit: 0+Private node exit: 0+```++The outer Python wrapper's first 240-second cap interrupted an incomplete run;+its node also stopped normally. Repeating with a 900-second cap and retained+raw runner output completed in 280.98 seconds, with the same executable hash+above and no runner/YAML/policy edits. Those caps are local harness bounds,+not product timeouts or performance claims. Hosted CI for the pushed commit+remains a separate verification.diff --git a/tools/session-memory/smoke.py b/tools/session-memory/smoke.pynew file mode 100644index 000000000..dbe837482--- /dev/null+++ b/tools/session-memory/smoke.py@@ -0,0 +1,286 @@+#!/usr/bin/env python3+"""Exercise installed session hooks against a private node and synthetic transcript only."""+import argparse+import hashlib+import json+import os+from pathlib import Path+import shutil+import socket+import subprocess+import tempfile+import time+from urllib.request import Request, build_opener, ProxyHandler+++def main():+ parser = argparse.ArgumentParser(description=__doc__)+ parser.add_argument('--binary', type=Path, required=True)+ parser.add_argument('--authenticated', action='store_true', help='use the node default API-key authentication')+ args = parser.parse_args()+ original = args.binary.resolve(strict=True)+ env = {k: v for k, v in os.environ.items() if not k.startswith('XERJ_')+ and k.lower() not in ('http_proxy', 'https_proxy', 'all_proxy', 'no_proxy')}+ env.update(XERJ_DISABLE_FEEDBACK='true', XERJ_NUM_CPUS='2')+ opener = build_opener(ProxyHandler({}))+ with tempfile.TemporaryDirectory(prefix='xerj-session-memory-') as temporary:+ work = Path(temporary).resolve()+ binary = work / "xerj's binary"+ shutil.copyfile(original, binary)+ binary.chmod(0o700)+ print('Executable SHA-256:', hashlib.sha256(binary.read_bytes()).hexdigest(), flush=True)+ project = work / "project's work"+ project.mkdir()+ claude = project / '.claude'+ claude.mkdir()+ settings = claude / 'settings.local.json'+ foreign = {'permissions': {'allow': ['Read']}, 'hooks': {'Stop': [+ {'hooks': [{'type': 'command', 'command': 'echo foreign-hook'}]}]}}+ settings.write_text(json.dumps(foreign))+ state = claude / 'xerj-memory'+ sockets = [socket.socket(), socket.socket()]+ for sock in sockets:+ sock.bind(('127.0.0.1', 0))+ port, rest = [sock.getsockname()[1] for sock in sockets]+ for sock in sockets:+ sock.close()+ url = f'http://127.0.0.1:{port}'+ node_config = work / 'node.toml'+ node_config.write_text(f'[server]\nes_compat_port={port}\nrest_port={rest}\ngrpc_port=0\n'+ f'data_dir={json.dumps(str(work / "data"))}\n'+ '[limits]\ndisk_flood_stage_percent=99\n')+ sid = '550e8400-e29b-41d4-a716-446655440000'+ transcript = work / 'session.jsonl'+ rows = [+ {'type': 'user', 'sessionId': sid, 'uuid': '550e8400-e29b-41d4-a716-446655440001',+ 'message': {'role': 'user', 'content': 'How do we recover? Contact [email protected].'}},+ {'type': 'assistant', 'sessionId': sid, 'uuid': '550e8400-e29b-41d4-a716-446655440002',+ 'message': {'role': 'assistant', 'content': [+ {'type': 'thinking', 'thinking': 'DO_NOT_EXPORT_THINKING'},+ {'type': 'text', 'text': 'We chose snapshots because the journal preserves recovery.\u202e\u200b\U000e0049'},+ {'type': 'tool_use', 'name': 'Edit', 'input': {'file_path': 'src/state.rs', 'new_string': 'DO_NOT_EXPORT_TOOL_INPUT'}}]}}+ ]++ def write_rows():+ transcript.write_text(''.join(json.dumps(row) + '\n' for row in rows))++ def cli(*args, expected=0):+ result = subprocess.run([str(binary), *args], cwd=project, env=env,+ text=True, capture_output=True, timeout=10)+ assert result.returncode == expected, (args, result.stdout, result.stderr)+ return result++ def request(path, body=None):+ headers = {'Content-Type': 'application/json'}+ if 'XERJ_API_KEY' in env:+ headers['Authorization'] = 'ApiKey ' + env['XERJ_API_KEY']+ req = Request(url + path, None if body is None else json.dumps(body).encode(), headers=headers)+ with opener.open(req, timeout=5) as response:+ return json.load(response)++ def wait_for(predicate, label, seconds=75):+ deadline = time.monotonic() + seconds+ while time.monotonic() < deadline:+ if predicate():+ return+ time.sleep(.1)+ status = (state / 'status.json').read_text() if (state / 'status.json').exists() else 'no status'+ raise AssertionError(f'{label}: {status}')++ def hook(event, command=None):+ payload = {'hook_event_name': event, 'cwd': str(project), 'session_id': sid,+ 'transcript_path': str(transcript), 'source': 'startup', 'reason': 'other'}+ if command is None:+ command = json.loads(settings.read_text())['hooks'][event][-1]['hooks'][0]['command']+ started = time.monotonic()+ result = subprocess.run(command, shell=True, cwd=project, env=env,+ input=json.dumps(payload), text=True, capture_output=True, timeout=4)+ elapsed = time.monotonic() - started+ assert result.returncode == 0 and not result.stderr, (result.stdout, result.stderr)+ if event == 'SessionEnd':+ assert not result.stdout+ print(f'{event}: {elapsed:.3f}s, exit 0, stderr empty', flush=True)+ return result.stdout++ def complete():+ return (state / 'receipts').exists() and any((state / 'receipts').iterdir()) and not any((state / 'pending').iterdir())++ node = None+ logs = []++ def start():+ nonlocal node+ log = (work / f'node-{len(logs)}.log').open('w')+ logs.append(log)+ flags = [] if args.authenticated else ['--insecure']+ node = subprocess.Popen([str(binary), *flags, '-c', str(node_config)],+ cwd=work, env=env, stdout=log, stderr=log)+ def ready():+ assert node.poll() is None, 'node exited early'+ if args.authenticated:+ key = work / 'data' / 'admin.key'+ if not key.exists():+ return False+ env['XERJ_API_KEY'] = key.read_text().strip()+ try:+ request('/')+ return True+ except OSError:+ return False+ wait_for(ready, 'readiness', 60)++ def stop():+ nonlocal node+ if node is not None:+ node.terminate()+ node.wait(timeout=30)+ assert node.returncode == 0, node.returncode+ print('private node exit 0', flush=True)+ node = None++ try:+ write_rows()+ original_settings = settings.read_bytes()+ cli('init', '--remove', expected=2)+ assert settings.read_bytes() == original_settings and not state.exists()+ cli('init', '--memory', '--url', url, '--dry-run')+ assert settings.read_bytes() == original_settings and not state.exists()+ cli('init', '--memory', '--url', url)+ installed = settings.read_bytes()+ cli('init', '--memory', '--url', url)+ assert settings.read_bytes() == installed+ # Versioned upgrades can move the executable. Reconfigure with the+ # new binary, then exercise the updated installed hook commands.+ old_binary = binary+ binary = work / "upgraded xerj's binary"+ old_binary.rename(binary)+ assert not old_binary.exists()+ cli('init', '--memory', '--url', url)+ installed = settings.read_bytes()+ assert 'upgraded' in installed.decode()+ config = json.loads((state / 'config.json').read_text())+ prefix, namespace = config['prefix'], config['namespace']+ print('PASS dry-run, quoted paths, install and unchanged re-install', flush=True)+ start()+ # Poison entries must not starve a healthy host delivery. Sources+ # and queue entries are synthetic and private to this temporary project.+ oversized = work / 'oversized.jsonl'+ with oversized.open('wb') as stream:+ stream.truncate(16 * 1024 * 1024 + 1)+ rejected_sid = '11111111-1111-4111-8111-111111111111'+ for delivery, source in [('missing', work / 'missing.jsonl'),+ ('oversized', oversized), ('mismatch', transcript)]:+ entry = {'session_id': rejected_sid, 'delivery_id': delivery,+ 'transcript_path': str(source)}+ name = hashlib.sha256(rejected_sid.encode()).hexdigest() + '-' + delivery + '.json'+ (state / 'pending' / name).write_text(json.dumps(entry))+ (state / 'pending' / 'corrupt.json').write_text('{bad')+ hook('SessionEnd')+ wait_for(complete, 'first import with poison siblings')+ rejected = state / 'rejected'+ assert len(list(rejected.glob('*.json'))) == 4+ assert len(list(rejected.glob('*.status'))) == 4+ assert (rejected / 'corrupt.json').read_text() == '{bad'+ assert all(json.loads(path.read_text())['state'] == 'rejected'+ for path in rejected.glob('*.status'))+ print('PASS missing/oversized/mismatched/corrupt deliveries quarantined; healthy sibling completed', flush=True)+ for line in (state / 'last-estimate.txt').read_text().splitlines():+ if 'autoindex:' in line and ('files (' in line or 'estimated index' in line):+ print(line, flush=True)+ assert (state / 'last-estimate.txt').stat().st_size > 0+ print(f'Synthetic input: {transcript.stat().st_size} bytes; dry-run output captured', flush=True)+ archives = list((state / 'archives').glob('*.jsonl'))+ assert len(archives) == 1+ exported = archives[0].read_text()+ assert '[REDACTED_EMAIL]' in exported and '[email protected]' not in exported+ assert 'DO_NOT_EXPORT' not in exported+ assert len(exported.splitlines()) == 2+ count = request('/' + prefix + '-*/_count')['count']+ assert count == 2, count+ memories = request('/_memory/' + namespace + '?size=10')+ assert memories['count'] == 1, memories+ text = hook('SessionStart')+ assert 'We chose snapshots' in text and 'archives/' in text and '[memory:' in text, text+ assert len(text.encode()) <= 4096 and len(text.splitlines()) <= 15+ for c in ['\u202e', '\u200b', '\U000e0049']:+ assert c not in text+ assert c not in memories['entries'][0]['text']++ print('PASS redaction, 2 searchable records, one memory and cited SessionStart brief', flush=True)+ hook('SessionEnd')+ wait_for(complete, 'duplicate import')+ assert request('/' + prefix + '-*/_count')['count'] == count+ assert request('/_memory/' + namespace + '?size=10')['count'] == 1+ assert len(list((state / 'archives').glob('*.jsonl'))) == 1+ rows.append({'type': 'assistant', 'sessionId': sid, 'uuid': '550e8400-e29b-41d4-a716-446655440003',+ 'message': {'role': 'assistant', 'content': 'Open item: validate the watcher after restart.'}})+ write_rows()+ hook('SessionEnd')+ wait_for(complete, 'changed transcript')+ assert request('/' + prefix + '-*/_count')['count'] == 3+ assert request('/_memory/' + namespace + '?size=10')['count'] == 1+ assert len(list((state / 'archives').glob('*.jsonl'))) == 2+ assert 'Open item: validate' in hook('SessionStart')+ print('PASS duplicate event idempotence and changed-transcript replacement (3 records, one memory)', flush=True)+ stop()+ cache = state / 'brief.json'+ legacy = json.loads(cache.read_text())+ legacy['entries'][0]['text'] += '\u202e\u200b\U000e0049'+ cache.write_text(json.dumps(legacy))+ cached = hook('SessionStart')+ assert not any(c in cached for c in ['\u202e', '\u200b', '\U000e0049'])+ assert cached.startswith('Cached') and 'Open item: validate' in cached+ before = next((state / 'exports').glob('*.jsonl')).read_bytes()+ transcript.write_text(transcript.read_text() + '{bad-private-tail')+ hook('SessionEnd')+ wait_for(lambda: (state / 'status.json').exists() and json.loads((state / 'status.json').read_text())['state'] == 'pending', 'malformed pending', 10)+ assert next((state / 'exports').glob('*.jsonl')).read_bytes() == before+ assert len(list((state / 'archives').glob('*.jsonl'))) == 2+ write_rows()+ start()+ hook('SessionStart')+ wait_for(complete, 'retry after node restart')+ assert request('/' + prefix + '-*/_count')['count'] == 3+ assert request('/_memory/' + namespace + '?size=10')['count'] == 1+ print('PASS offline cached brief, malformed-tail atomicity and retained-job recovery after restart', flush=True)+ rejected_before = {path.name: path.read_bytes() for path in rejected.iterdir()}+ old_command = json.loads(settings.read_text())['hooks']['SessionEnd'][-1]['hooks'][0]['command']+ removal = cli('init', '--memory', '--remove', '--dry-run')+ assert 'would disable' in removal.stdout+ assert settings.read_bytes() == installed+ # A completed receipt can precede lease release by a few instructions.+ def remove():+ result = subprocess.run([str(binary), 'init', '--memory', '--remove'], cwd=project,+ env=env, text=True, capture_output=True, timeout=4)+ if result.returncode == 0:+ return True+ assert 'busy' in result.stderr, result.stderr+ return False+ wait_for(remove, 'remove', 10)+ assert json.loads(settings.read_text()) == foreign+ assert {path.name: path.read_bytes() for path in rejected.iterdir()} == rejected_before+ assert not json.loads((state / 'config.json').read_text())['enabled']+ assert len(list((state / 'archives').glob('*.jsonl'))) == 2+ assert request('/' + prefix + '-*/_count')['count'] == 3+ assert request('/_memory/' + namespace + '?size=10')['count'] == 1+ hook('SessionEnd', old_command)+ assert not any((state / 'pending').iterdir())+ print('PASS uninstall keeps foreign hooks and data; stale hook is a silent no-op', flush=True)+ # Explicit cleanup only after the retention assertions, on this script's own node.+ plan = cli('autoindex', 'remove', '--prefix', prefix, '--root', str(state / 'exports'), '--url', url, expected=4)+ assert prefix in plan.stdout + plan.stderr+ cli('autoindex', 'remove', '--prefix', prefix, '--root', str(state / 'exports'), '--url', url, '--yes')+ headers = {'Authorization': 'ApiKey ' + env['XERJ_API_KEY']} if 'XERJ_API_KEY' in env else {}+ req = Request(url + '/_memory/' + namespace, method='DELETE', headers=headers)+ with opener.open(req, timeout=5) as response:+ assert response.status == 200+ assert request('/_memory/' + namespace + '?size=10')['count'] == 0+ print('PASS documented ownership-plan cleanup and dedicated memory deletion', flush=True)+ finally:+ stop()+ for log in logs:+ log.close()+++if __name__ == '__main__':+ main()