Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

Merge pull request #327 from probelabs/feat/definition-aware-code-search

xerj-orga9013f7ac3ae
diff --cc engine/crates/xerj-api/src/auth.rs
index b00abbd58,046c42b1a..1a8b1fd1e
--- a/engine/crates/xerj-api/src/auth.rs
+++ b/engine/crates/xerj-api/src/auth.rs
@@@ -460,17 -420,10 +460,18 @@@ fn unauthorized_response(data_dir: &str
request_id: None,
},
status: 401,
+ xerj_hint: None,
};
- (StatusCode::UNAUTHORIZED, Json(body)).into_response()
+ (
+ StatusCode::UNAUTHORIZED,
+ [(
+ axum::http::header::WWW_AUTHENTICATE,
+ WWW_AUTHENTICATE_CHALLENGE,
+ )],
+ Json(body),
+ )
+ .into_response()
}
// ─────────────────────────────────────────────────────────────────────────────
diff --cc engine/crates/xerj-autoindex/src/lib.rs
index b3800c64a,3e3dec046..a0ead54d7
--- a/engine/crates/xerj-autoindex/src/lib.rs
+++ b/engine/crates/xerj-autoindex/src/lib.rs
@@@ -4857,15 -4694,56 +4857,62 @@@ pub fn run_index_report(cfg: IndexCfg)
);
}
println!(
- "\nnext: `xerj autoindex map --url {}` for the data map; search via GET /{}-*/_search",
- cfg.url, cfg.prefix
+ "{}",
+ next_hint(
+ &cfg.url,
+ &cfg.prefix,
+ cfg.api_key.as_deref(),
+ std::env::var("XERJ_API_KEY").ok().as_deref(),
+ cfg.api_key_file.as_deref(),
+ )
);
+ // This line is where an agent learns how to search, so it is where the
+ // code-aware form belongs. A blind usability run showed the cost of
+ // omitting it: the agent followed exactly this guidance, got whole
+ // files back, and reinvented client-side slicing rather than
+ // discovering `_passage` — which it never saw mentioned anywhere.
+ // Only printed when source code was actually indexed; for a PDF or log
+ // corpus it would be noise.
+ let indexed_code = plan.files.values().any(|fa| fa.family == "code");
+ if indexed_code {
+ println!(
+ " code was indexed — for a function/class instead of a whole file:\n\
+ \x20 curl -s $URL/{}-*/_search -H 'Content-Type: application/json' \\\n\
+ \x20 -d '{{\"query\":{{\"bool\":{{\"should\":[{{\"multi_match\":{{\"query\":\"<symbol or phrase>\",\
+ \"fields\":[\"body\",\"defs\"],\"type\":\"most_fields\"}}}},{{\"match_phrase\":{{\"defs\":{{\"query\":\"<symbol or phrase>\",\"boost\":4}}}}}}]}}}},\
+ \"_source\":[\"ax_path\",\"title\"],\"fields\":[\"_passage\"]}}'\n\
+ \x20 (the `match_phrase` clause ranks the file that DEFINES a symbol above files that merely call it; `_passage` returns \
+ the enclosing block, not the file)",
+ cfg.prefix
+ );
+ }
+ // A mixed corpus needs one more thing said out loud. Code volume
+ // swamps prose on shared vocabulary: in a blind run, a question about
+ // documentation ("exit code 3") matched WordPress PHP on `exit` and
+ // `code` and returned nothing but source, costing a wasted round trip.
+ // The agent worked out the fix itself afterwards — scope the query —
+ // which is exactly the sort of thing that should not have to be
+ // rediscovered.
+ let indexed_prose = plan
+ .files
+ .values()
+ .any(|fa| fa.family.starts_with("txt") || fa.family == "html");
+ if indexed_code && indexed_prose {
+ println!(
+ " this corpus mixes code and prose — code volume can swamp prose on\n\
+ \x20 shared words. Scope the side you want:\n\
+ \x20 …,\"query\":{{\"bool\":{{\"must\":[{{\"match\":{{\"body\":\"<text>\"}}}}],\
+ \"filter\":[{{\"term\":{{\"ax_format\":\"code\"}}}}]}}}}\n\
+ \x20 (values for `ax_format` in this run: {})",
+ {
+ let mut fams: Vec<&str> =
+ plan.files.values().map(|fa| fa.family.as_str()).collect();
+ fams.sort_unstable();
+ fams.dedup();
+ fams.join(", ")
+ }
+ );
+ }
}
// Exit 3 means "completed, some input was unusable". Backend rejections
// are not consulted and must not be: a rejected item aborts the run with
diff --cc engine/crates/xerj-server/src/main.rs
index cf026cfd3,558bfb9a1..12ac6c373
--- a/engine/crates/xerj-server/src/main.rs
+++ b/engine/crates/xerj-server/src/main.rs
@@@ -247,11 -254,19 +265,23 @@@ fn help_text() -> String
xerj autoindex map print the discovered data map\n\
xerj brain <folder> one command: index a folder into a running, browsable\n\
second brain in your browser (see xerj brain --help)\n\
+ xerj mcp [opts] Model Context Protocol stdio server: exposes 10 tools\n\
+ (search, semantic, vector, hybrid, memory, second-brain)\n\
+ to any MCP client. Proxies to a node you already started\n\
+ — set XERJ_URL or --url (see xerj mcp --help)\n\
\n\
+ SEARCHING A CODEBASE:\n\
+ A search hit returns the whole indexed document, which for source code is the\n\
+ whole file. To get only the matching function or class, ask for the passage:\n\
+ \n\
+ curl -s $URL/<index>/_search -H 'Content-Type: application/json' \\\n\
+ -d '{{\"query\":{{\"bool\":{{\"should\":[{{\"multi_match\":{{\"query\":\"<symbol or phrase>\",\n\
+ \"fields\":[\"body\",\"defs\"],\"type\":\"most_fields\"}}}},{{\"match_phrase\":{{\"defs\":{{\"query\":\"<symbol or phrase>\",\"boost\":4}}}}}}]}}}},\n\
+ \"_source\":[\"ax_path\",\"title\"],\"fields\":[\"_passage\"]}}'\n\
+ \n\
+ the `match_phrase` clause ranks the file that DEFINES a symbol above files that merely call it;\n\
+ `_passage` returns the enclosing block. Without them you get whole files.\n\
+ \n\
ENVIRONMENT:\n\
XERJ_LOG Log level filter (default: info)\n\
XERJ_CONFIG Config file path\n\