Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

fix(hub): treat an uncovered topic as no coverage, not an ambiguous match (#2267)

xerj-org558dfebfb7fd
diff --git a/docs/CORPUS_HUB_V2_IMPLEMENTATION.md b/docs/CORPUS_HUB_V2_IMPLEMENTATION.md
index 71a8936a0..179c2fee6 100644
--- a/docs/CORPUS_HUB_V2_IMPLEMENTATION.md
+++ b/docs/CORPUS_HUB_V2_IMPLEMENTATION.md
@@ -389,8 +389,14 @@ Order and scoring, exactly:
lowercase words; for each corpus, score = (tokens present in its sketch) /
(tokens total), ×0.8; tokens present in more than 25% of all sketches count half.
4. Merge by slug taking the max score. `Confident` if top ≥ 0.6 and top ≥ 1.5 × second
- (or no second). `Ambiguous` if two or more are ≥ 0.3. Else `NoCorpus`.
- `--corpus` skips routing. Corpora already cached (`cached`) get +0.05.
+ (or no second). `Ambiguous` if two or more are ≥ 0.3 and the best reaches 0.35
+ (a full overlap of common tokens scores 0.4 and stays a real tie). Two or more
+ between 0.3 and 0.35 is `NoCoverage`: a weak overlap, not a choice. Else `NoCorpus`.
+ A question that names a project no entry contains (`Phaser`, `setInteractive`) is
+ `NoCoverage` even when another token ties several corpora. `xerj ask` exits 1 for
+ both `NoCorpus` and `NoCoverage` (the latter names `xerj autoindex` and `xerj hub find`);
+ exit 4 is only a real ambiguity. `--corpus` skips routing. Corpora already cached
+ (`cached`) get +0.05.
A question that names a language or a technology is decided before those four
steps, from the catalogue entry itself (slug, topics, a language word in
@@ -406,6 +412,9 @@ Order and scoring, exactly:
2. Route. `Ambiguous` → print the candidates (`slug — ask_for`, one per line) and
exit 4 (the same "needs a decision" code autoindex uses); the agent re-asks with
`--corpus`. `NoCorpus` → exit 1 with `no hub corpus matches; see xerj.org/llms.txt`.
+ `NoCoverage` → exit 1 with `no hub corpus covers this topic` and the local-index
+ hint (`xerj autoindex <path>`, then `xerj code` or `xerj search`; `xerj hub find`
+ to see what the hub does have). Not exit 4.
3. If the pack is cached locally → answer locally (`AnsweredFrom::LocalPack`).
4. Else if hosted is allowed → answer through the hosted node now; then, unless
`--no-fetch`/`--offline`, start the pull in the background: spawn
diff --git a/engine/crates/xerj-autoindex/src/init.rs b/engine/crates/xerj-autoindex/src/init.rs
index 0e2e725c9..c65b0a2d7 100644
--- a/engine/crates/xerj-autoindex/src/init.rs
+++ b/engine/crates/xerj-autoindex/src/init.rs
@@ -172,9 +172,12 @@ more than one corpus defines the name.
## When it does not answer
-- Exit 4: more than one corpus fits. The candidates print, one per line. Run again with
- `--corpus <slug>`.
+- Exit 4: more than one corpus actually fits. The candidates print, one per line. Run again with
+ `--corpus <slug>`. A topic no corpus covers is not this exit.
- Exit 1, `no hub corpus matches`: the hub does not hold it. Say so. Do not answer from memory.
+- Exit 1, `no hub corpus covers this topic`: the question names something the hub does not
+ have, or every overlap is weak. Index it locally (`xerj autoindex <path>`, then `xerj code`
+ or `xerj search`). `xerj hub find` lists what the hub does have. Do not pick a nearest corpus.
- Exit 3: a downloaded pack failed its checksum or signature check and was discarded. Do not use it.
Tell the user.
- Exit 5: this build has no hub public key, so no pack can be verified and nothing was downloaded.
@@ -296,7 +299,7 @@ fn self_bin() -> String {
/// `skill_md()` or `HUB_SKILL_MD` changes, so a file an older init wrote (and
/// the user never edited) is upgraded on the next run. A unit test pins the
/// texts' hashes to this number, so a text change without a bump fails CI.
-const INIT_FILES_VERSION: u32 = 5;
+const INIT_FILES_VERSION: u32 = 6;
/// The marker's prefix. The full line is
/// `<!-- xerj-init-version: <N> sha256:<16 hex> -->`: the generation, and the
@@ -1534,7 +1537,7 @@ mod tests {
content_hash(&skill_md()).as_str(),
content_hash(HUB_SKILL_MD).as_str()
),
- (5, "78d8c53415efea51", "131aa9224f56d182"),
+ (6, "78d8c53415efea51", "2cdaa10f4b99c69f"),
"a teaching text changed: bump INIT_FILES_VERSION and re-pin"
);
}
diff --git a/engine/crates/xerj-hub/src/ask.rs b/engine/crates/xerj-hub/src/ask.rs
index 5ef88df8f..113b35daa 100644
--- a/engine/crates/xerj-hub/src/ask.rs
+++ b/engine/crates/xerj-hub/src/ask.rs
@@ -13,7 +13,9 @@
//! corpus, or from a tight group of at most three which are searched
//! together; the answer's first line names them (`using: slug, slug`).
//! Genuinely ambiguous → a few candidates, best first, and exit code **4**;
-//! nothing credible → exit **1** with the llms.txt pointer.
+//! nothing credible → exit **1** with the llms.txt pointer. A weak overlap,
+//! or a tie on some other word while the question names a project no corpus
+//! contains, is the same exit **1** with [`NO_COVERAGE_MESSAGE`] — not exit 4.
//! 3. **Cached pack** → answer locally ([`AnsweredFrom::LocalPack`]); the
//! network is not touched.
//! 4. **No pack, hosted allowed** → answer through the hosted hub now, then (unless
@@ -50,10 +52,10 @@
//! | code | meaning |
//! |------|---------|
//! | 0 | answered, at least one hit |
-//! | 1 | no match; no corpus; offline/no-fetch with no pack; pull or node failure |
+//! | 1 | no match; no corpus; no coverage (the topic is absent — index it locally); offline/no-fetch with no pack; pull or node failure |
//! | 2 | usage error; unknown `--corpus`; a pull rejected the slug |
//! | 3 | a pulled pack failed verification (sha256 / signature / format) |
-//! | 4 | ambiguous — re-run with `--corpus` (the code `xerj autoindex` uses for "needs a decision") |
+//! | 4 | ambiguous — several corpora actually match; re-run with `--corpus` (the code `xerj autoindex` uses for "needs a decision") |
//! | 5 | this build has no hub public key, so no pack can be verified (nothing was downloaded) |
use std::cell::RefCell;
@@ -97,6 +99,12 @@ pub const EXIT_HUB_UNAVAILABLE: i32 = 6;
/// Printed (stderr) when the router finds nothing credible.
pub const NO_CORPUS_MESSAGE: &str = "no hub corpus matches; see xerj.org/llms.txt";
+/// Printed (stderr) when candidates exist but none covers the question: every
+/// score is a weak overlap, or the question names a project no corpus contains.
+/// Exit [`EXIT_NO_ANSWER`], not the ambiguity exit — there is nothing to pick.
+pub const NO_COVERAGE_MESSAGE: &str = "no hub corpus covers this topic; index it locally with \
+ `xerj autoindex <path>`, then `xerj code` or `xerj search`. Discover what the hub does have \
+ with `xerj hub find`";
/// Printed once (stderr) the first time `--hosted auto` resolves to off inside a
/// git work tree.
pub const HOSTED_OFF_NOTICE: &str =
@@ -808,7 +816,9 @@ fn with_symbol_evidence(routing: Routing, hits: &[String]) -> Routing {
let both: Vec<String> = slugs.iter().filter(|s| hits.contains(s)).cloned().collect();
pick(both).unwrap_or(Routing::Ambiguous(slugs))
}
- Routing::NoCorpus => pick(hits.to_vec()).unwrap_or(Routing::NoCorpus),
+ // A coverage gap names no corpus. A symbol defined in one or several
+ // corpora is still evidence: it decides, the same way it does for a miss.
+ Routing::NoCorpus | Routing::NoCoverage => pick(hits.to_vec()).unwrap_or(routing),
}
}
@@ -1013,6 +1023,13 @@ fn select_corpus(
out.exit = EXIT_NEEDS_DECISION;
None
}
+ Routing::NoCoverage => {
+ // Not a choice among the nearest packs. Discovery is `xerj hub find`;
+ // the topic itself has to be indexed locally.
+ out.stderr.push(NO_COVERAGE_MESSAGE.to_string());
+ out.exit = EXIT_NO_ANSWER;
+ None
+ }
Routing::NoCorpus => {
out.stderr.push(NO_CORPUS_MESSAGE.to_string());
// Nothing was credible, but something may have been close: name the
@@ -1877,13 +1894,17 @@ pub struct FoundCorpus {
/// What [`find`] decided: the router's verdict and the candidates, best first.
#[derive(Debug, Clone, serde::Serialize)]
pub struct FindOutcome {
- /// `confident` | `ambiguous` | `no_corpus` | `different_project`.
+ /// `confident` | `ambiguous` | `no_corpus` | `no_coverage` | `different_project`.
///
/// `different_project` means the router would have answered from a slug
/// whose name the question only shares: the question names another project
/// (`fabric.js`) and this pack is a different one (`danielmiessler/fabric`).
/// `xerj ask` does not answer that as a silent match; it prints the pack's
/// identity and exits 4.
+ ///
+ /// `no_coverage` means candidates may be listed but none covers the topic
+ /// (weak scores, or the question names a project the catalogue does not
+ /// contain). `xerj ask` exits 1 with the local-index hint, not 4.
pub routing: &'static str,
pub corpora: Vec<FoundCorpus>,
}
@@ -1917,7 +1938,7 @@ pub fn find(
let mut order: Vec<String> = match &routing {
Routing::Confident(slug) => vec![slug.clone()],
Routing::Group(slugs) | Routing::Ambiguous(slugs) => slugs.clone(),
- Routing::NoCorpus => Vec::new(),
+ Routing::NoCorpus | Routing::NoCoverage => Vec::new(),
};
for slug in ranked.iter().map(|(s, _)| s).chain(by_symbol.iter()) {
if !order.contains(slug) {
@@ -1961,7 +1982,7 @@ pub fn find(
Routing::Group(slugs) | Routing::Ambiguous(slugs) => {
slugs.iter().map(String::as_str).collect()
}
- Routing::NoCorpus => Vec::new(),
+ Routing::NoCorpus | Routing::NoCoverage => Vec::new(),
};
// A confident (or grouped) name match that is actually a different project
// is not reported as confident: the caller must see the mismatch.
@@ -1974,6 +1995,7 @@ pub fn find(
_ if collided => "different_project",
Routing::Confident(_) | Routing::Group(_) => "confident",
Routing::Ambiguous(_) => "ambiguous",
+ Routing::NoCoverage => "no_coverage",
Routing::NoCorpus => "no_corpus",
},
corpora,
@@ -2042,7 +2064,11 @@ pub fn collision_sentence(token: &str, slug: &str, upstream: &str) -> String {
/// instead of reading as a plain match.
pub fn render_find(found: &FindOutcome) -> String {
if found.corpora.is_empty() {
- return NO_CORPUS_MESSAGE.to_string();
+ return if found.routing == "no_coverage" {
+ NO_COVERAGE_MESSAGE.to_string()
+ } else {
+ NO_CORPUS_MESSAGE.to_string()
+ };
}
let mut lines: Vec<String> = Vec::new();
for c in &found.corpora {
@@ -2068,6 +2094,8 @@ pub fn render_find(found: &FindOutcome) -> String {
ask it with xerj_hub_ask {question, corpus} only if that pack is the one you meant"
.to_string(),
);
+ } else if found.routing == "no_coverage" {
+ lines.push(format!("routing: no_coverage; {NO_COVERAGE_MESSAGE}"));
} else {
lines.push(format!(
"routing: {}; ask one with xerj_hub_ask {{question, corpus}}",
@@ -2704,6 +2732,109 @@ mod tests {
);
}
+ /// Issue #2179. A question that names a project no corpus contains used to
+ /// exit 4 ("more than one corpus matches") when several packs shared some
+ /// other word. That is a coverage gap: exit 1, and say how to index locally.
+ /// A real tie still exits 4. An exact slug is still answered.
+ #[test]
+ fn no_coverage_exits_1_and_points_at_local_indexing() {
+ let rig = Rig::new().in_git_repo();
+ let mut entries = Vec::new();
+ for slug in ["whatwg-html", "lol-html", "parse5", "jsdom"] {
+ let mut e = entry(slug, 1, LicenceClass::AdaptWithAttribution, &[]);
+ e.ask_for = "An agent writing HTML queries this for the document format.".into();
+ entries.push(e);
+ }
+ for i in 0..(151 - entries.len()) {
+ let mut e = entry(
+ &format!("filler-{i}"),
+ 1,
+ LicenceClass::AdaptWithAttribution,
+ &[],
+ );
+ e.ask_for = "an agent working on storage queries this for internals".into();
+ entries.push(e);
+ }
+ let mut lz4_entry = lz4(1024, LicenceClass::AdaptWithAttribution);
+ lz4_entry.ask_for =
+ "an agent needing fast block compression queries this for the decoder".into();
+ entries.push(lz4_entry);
+ let cat = catalogue(entries);
+
+ let backend = FakeBackend::new(true);
+ let out = rig.run(
+ &args("How does Phaser draw HTML?", None),
+ &cat,
+ None,
+ &backend,
+ );
+ assert_eq!(out.exit, EXIT_NO_ANSWER, "{out:?}");
+ assert_ne!(out.exit, EXIT_NEEDS_DECISION);
+ let err = out.stderr.join("\n");
+ assert!(err.contains(NO_COVERAGE_MESSAGE), "{err}");
+ assert!(err.contains("xerj autoindex <path>"), "{err}");
+ assert!(err.contains("xerj code"), "{err}");
+ assert!(err.contains("xerj search"), "{err}");
+ assert!(err.contains("xerj hub find"), "{err}");
+ assert!(!err.contains("more than one corpus matches"), "{err}");
+ assert!(
+ out.stdout.is_empty(),
+ "not a candidate list: {}",
+ out.stdout
+ );
+ assert!(backend.calls().is_empty(), "{:?}", backend.calls());
+
+ let found = find(
+ "How does Phaser draw HTML?",
+ 5,
+ rig.cwd.path(),
+ rig.hub.path(),
+ &cat,
+ );
+ assert_eq!(found.routing, "no_coverage", "{found:?}");
+ let text = render_find(&found);
+ assert!(text.contains("routing: no_coverage"), "{text}");
+ assert!(text.contains("no hub corpus covers this topic"), "{text}");
+ assert!(text.contains("xerj autoindex"), "{text}");
+
+ // Lowercase project name, same gap.
+ let out = rig.run(
+ &args("how does phaser draw HTML?", None),
+ &cat,
+ None,
+ &FakeBackend::new(true),
+ );
+ assert_eq!(out.exit, EXIT_NO_ANSWER, "{out:?}");
+ assert!(out.stderr.iter().any(|l| l.contains(NO_COVERAGE_MESSAGE)));
+
+ // The HTML packs really do tie when the question is about HTML.
+ let out = rig.run(
+ &args("HTML document format", None),
+ &cat,
+ None,
+ &FakeBackend::new(true),
+ );
+ assert_eq!(out.exit, EXIT_NEEDS_DECISION, "{out:?}");
+ assert!(
+ out.stderr
+ .iter()
+ .any(|l| l.contains("more than one corpus matches")),
+ "{:?}",
+ out.stderr
+ );
+
+ // Exact slug.
+ let backend = FakeBackend::new(true);
+ let out = rig.run(
+ &args("how does the lz4 decoder work", None),
+ &cat,
+ None,
+ &backend,
+ );
+ assert_eq!(out.exit, EXIT_OK, "{:?}", out.stderr);
+ assert_eq!(backend.calls(), ["ready:lz4", "answer_local:lz4"]);
+ }
+
#[test]
fn no_corpus_exits_1_with_the_llms_txt_pointer() {
let rig = Rig::new();
@@ -3238,10 +3369,12 @@ mod tests {
// The sentence alone: the symbol is one of six words and its lower-case
// twin never matches, so the score cannot reach "confident".
let cached: Vec<String> = vec![];
+ // The identifier is a name no description contains, so the words are a
+ // coverage gap rather than a corpus choice. The symbol, below, still routes.
assert!(
matches!(
crate::router::route(q, rig.cwd.path(), &cat, None, &cached),
- Routing::NoCorpus
+ Routing::NoCorpus | Routing::NoCoverage
),
"the words alone do not route"
);
@@ -3360,6 +3493,15 @@ mod tests {
with_symbol_evidence(Routing::NoCorpus, &[]),
Routing::NoCorpus
);
+ assert_eq!(
+ with_symbol_evidence(Routing::NoCoverage, &["b".to_string()]),
+ Routing::Confident("b".to_string()),
+ "a symbol still routes when the words were a coverage gap"
+ );
+ assert_eq!(
+ with_symbol_evidence(Routing::NoCoverage, &[]),
+ Routing::NoCoverage
+ );
}
/// FTX shell-less trial (2026-10-17): `xerj_hub_ask` with no `corpus` refused
diff --git a/engine/crates/xerj-hub/src/cli.rs b/engine/crates/xerj-hub/src/cli.rs
index 6c4f5ddee..9bb338973 100644
--- a/engine/crates/xerj-hub/src/cli.rs
+++ b/engine/crates/xerj-hub/src/cli.rs
@@ -69,7 +69,8 @@ const HUB_USAGE: &str = "usage: xerj hub <command>
Which corpora could answer, best first. Each line names the upstream
repo (owner/name, when the catalogue has it) and the one-line description
of what that pack is. A query that names a different project of the same
- slug (fabric.js) exits 4 instead of matching silently.
+ slug (fabric.js) exits 4 instead of matching silently. A question no
+ corpus covers exits 1 (not 4) and names `xerj autoindex`.
feedback <slug> --usefulness <1-5> --found <yes|partial|no> [...]
Rate a corpus after using it: one structured review (no query text) for
its Agent score on hub.xerj.org (see `xerj hub feedback -h`).
@@ -1387,7 +1388,7 @@ fn run_find_cmd(args: &[String]) -> i32 {
}
match found.routing {
"different_project" | "ambiguous" => ask::EXIT_NEEDS_DECISION,
- "no_corpus" => ask::EXIT_NO_ANSWER,
+ "no_corpus" | "no_coverage" => ask::EXIT_NO_ANSWER,
_ => 0,
}
}
@@ -1459,10 +1460,13 @@ options:
--url <node> the node that holds the packs (default: XERJ_URL, then
localhost:9200, else a local node is started)
-exit codes: 0 answered; 1 no match or nothing to answer from; 2 usage error or
-unknown corpus; 3 a pack failed verification; 4 more than one corpus matches
-and none clearly wins (a few candidates are printed, best first; re-run with
---corpus <slug>); 5 this build has no hub
+exit codes: 0 answered; 1 no match, nothing to answer from, or no hub corpus
+covers the topic (index it locally: xerj autoindex <path>, then xerj code or
+xerj search; xerj hub find lists what the hub does have); 2 usage error or
+unknown corpus; 3 a pack failed verification; 4 more than one corpus actually
+matches and none clearly wins (a few candidates are printed, best first;
+re-run with --corpus <slug>). A weak overlap, or a tie while the question
+names a project no corpus contains, is exit 1, not 4; 5 this build has no hub
public key, so no pack can be verified (nothing was downloaded); 6 the hub
catalogue could not be loaded (hub unreachable, no signed catalogue served, or
--offline with none cached): a hub problem, not an answer about any corpus";
diff --git a/engine/crates/xerj-hub/src/router.rs b/engine/crates/xerj-hub/src/router.rs
index efbf6641a..682b15b8f 100644
--- a/engine/crates/xerj-hub/src/router.rs
+++ b/engine/crates/xerj-hub/src/router.rs
@@ -33,10 +33,12 @@
//! distinctive words drive the score.
//!
//! The merged scores decide: a dominant, high-scoring corpus is
-//! [`Routing::Confident`]; a cluster of plausible ones is
+//! [`Routing::Confident`]; a cluster of good matches is
//! [`Routing::Ambiguous`] (best-first; the ask path shows at most
//! [`AMBIGUOUS_SHOWN`] after symbol evidence, and the caller disambiguates
-//! with `--corpus`); nothing credible is
+//! with `--corpus`); a cluster that only weakly overlaps the question, or a
+//! tie on some other word while the question names a project no entry
+//! contains, is [`Routing::NoCoverage`]; nothing credible is
//! [`Routing::NoCorpus`]. A corpus already cached locally gets a small nudge so
//! a tie breaks toward the pack the agent already has. A corpus the question
//! names by slug still decides outright, ahead of the identity score.
@@ -86,6 +88,11 @@ pub enum Routing {
/// [`AMBIGUOUS_SHOWN`] after symbol evidence has had a chance to narrow
/// the list; the caller must pick one.
Ambiguous(Vec<String>),
+ /// Several corpora overlapped the question, but none covers it: every
+ /// candidate is a weak score, or the question names a project no catalogue
+ /// entry contains and the tie is on some other word. Not a choice among
+ /// packs — the topic is absent.
+ NoCoverage,
/// Nothing credible matched.
NoCorpus,
}
@@ -115,6 +122,17 @@ const CONFIDENT_MIN: f32 = 0.6;
const DOMINANCE: f32 = 1.5;
/// A slug must reach this to be an `Ambiguous` candidate.
const CANDIDATE_MIN: f32 = 0.3;
+/// A multi-candidate tie whose best score is below this is a weak overlap, not
+/// a choice. It sits under a full overlap of common tokens
+/// ([`COMMON_CREDIT`] × [`SKETCH_WEIGHT`] = 0.4), which stays
+/// [`Routing::Ambiguous`], and above [`CANDIDATE_MIN`]. A minority overlap
+/// (two tokens out of five scores 0.32) is [`Routing::NoCoverage`].
+const AMBIGUOUS_GOOD_MIN: f32 = 0.35;
+/// Identity-channel scores (name weight × idf) sit above this. Sketch, hosted
+/// and dependency scores do not (those top out at 1.0, plus the cached bonus).
+/// An unknown lowercase word may break an identity tie without breaking a
+/// sketch tie.
+const IDENTITY_SCORE_FLOOR: f32 = 2.0;
/// Score a corpus named in the question earns (it outranks everything: the
/// asker said which corpus they mean).
const NAMED_SCORE: f32 = 1.0;
@@ -279,7 +297,8 @@ pub fn route_ranked(
let (routing, ranked) = decide(evidence, cached);
// A slug the question spells out decides before identity scoring. "lz4 or
- // zstd?" is the asker choosing, not a group to search together.
+ // zstd?" is the asker choosing, not a group to search together. That
+ // choice is not rewritten into a coverage gap: the asker named the packs.
match named.len() {
1 => return (Routing::Confident(named[0].clone()), ranked),
n if n > 1 => {
@@ -292,10 +311,169 @@ pub fn route_ranked(
}
_ => {}
}
- if let Some(identity) = identity_route(question, cat) {
- return identity;
+ if let Some((identity, identity_ranked)) = identity_route(question, cat) {
+ return (
+ apply_coverage(question, cat, identity, &identity_ranked),
+ identity_ranked,
+ );
}
- (routing, ranked)
+ (apply_coverage(question, cat, routing, &ranked), ranked)
+}
+
+/// Turn a tie or a miss into [`Routing::NoCoverage`] when the question names a
+/// project the catalogue does not contain.
+///
+/// A capitalized, underscored, or digit-bearing token (`Phaser`,
+/// `setInteractive`, `HTML5`) that no slug, topic, or description contains is
+/// that project. A lowercase word does the same to an identity tie only (those
+/// scores sit above [`IDENTITY_SCORE_FLOOR`]), so "how does phaser draw HTML?"
+/// is not a choice among the HTML packs, while a sketch tie on words the
+/// corpora actually contain stays [`Routing::Ambiguous`]. A slug the question
+/// spells out is handled before this runs.
+fn apply_coverage(
+ question: &str,
+ cat: &Catalogue,
+ routing: Routing,
+ ranked: &[(String, f32)],
+) -> Routing {
+ let proper = !uncovered_names(question, cat).is_empty();
+ let identity_tie = ranked
+ .first()
+ .is_some_and(|(_, score)| *score > IDENTITY_SCORE_FLOOR);
+ match routing {
+ Routing::NoCorpus if proper => Routing::NoCoverage,
+ Routing::Ambiguous(_) if proper || (identity_tie && has_unknown_word(question, cat)) => {
+ Routing::NoCoverage
+ }
+ other => other,
+ }
+}
+
+/// Tokens the question writes as a name — a capital, a digit, or an underscore
+/// — that no catalogue entry's slug, topics, or description contains.
+/// Lowercase how-to words are not names: "handle" must not turn a real HTML
+/// tie into a coverage gap, while "Phaser" must.
+fn uncovered_names(question: &str, cat: &Catalogue) -> Vec<String> {
+ let known = catalogue_words(cat);
+ let mut out = Vec::new();
+ let mut seen = HashSet::new();
+ for raw in question_raw_tokens(question) {
+ let lower = raw.to_ascii_lowercase();
+ if lower.len() < IDENTITY_MIN_LEN || is_identity_stopword(&lower) {
+ continue;
+ }
+ let name_shaped = raw.contains('_')
+ || raw
+ .chars()
+ .any(|c| c.is_ascii_uppercase() || c.is_ascii_digit());
+ if name_shaped && !known.contains(&lower) && seen.insert(lower.clone()) {
+ out.push(lower);
+ }
+ }
+ out
+}
+
+/// A lowercase word of the question that no catalogue entry contains and that
+/// is not ordinary how-to English (`draw`, `parser`). "phaser" qualifies;
+/// "handle" does not.
+fn has_unknown_word(question: &str, cat: &Catalogue) -> bool {
+ let known = catalogue_words(cat);
+ question_raw_tokens(question).any(|raw| {
+ let lower = raw.to_ascii_lowercase();
+ lower.len() >= IDENTITY_MIN_LEN
+ && !is_identity_stopword(&lower)
+ && !is_common_question_word(&lower)
+ && raw.chars().all(|c| c.is_ascii_lowercase())
+ && !known.contains(&lower)
+ })
+}
+
+fn catalogue_words(cat: &Catalogue) -> HashSet<String> {
+ let mut known = HashSet::new();
+ for entry in &cat.corpora {
+ let profile = IdentityProfile::from_entry(entry);
+ known.extend(profile.slug);
+ known.extend(profile.tags);
+ known.extend(profile.desc);
+ }
+ known
+}
+
+fn question_raw_tokens(question: &str) -> impl Iterator<Item = &str> {
+ question
+ .split(|c: char| !(c.is_ascii_alphanumeric() || matches!(c, '_' | '-' | '.')))
+ .map(|word| word.trim_matches(|c| c == '.' || c == '-'))
+ .filter(|word| !word.is_empty())
+}
+
+/// Ordinary words of a how-to question. Not stopwords (those are dropped
+/// earlier): these are content words that are not a project name, so an
+/// unknown one does not mean the hub lacks the topic. Sorted for binary search.
+fn is_common_question_word(word: &str) -> bool {
+ const WORDS: &[&str] = &[
+ "area",
+ "block",
+ "body",
+ "call",
+ "calls",
+ "canvas",
+ "code",
+ "compress",
+ "content",
+ "data",
+ "decoder",
+ "default",
+ "document",
+ "draw",
+ "element",
+ "engine",
+ "error",
+ "file",
+ "files",
+ "format",
+ "function",
+ "game",
+ "games",
+ "handle",
+ "handles",
+ "input",
+ "internals",
+ "level",
+ "line",
+ "lines",
+ "loop",
+ "method",
+ "mode",
+ "name",
+ "names",
+ "object",
+ "objects",
+ "parser",
+ "path",
+ "point",
+ "queries",
+ "query",
+ "read",
+ "reference",
+ "render",
+ "return",
+ "size",
+ "state",
+ "text",
+ "time",
+ "type",
+ "types",
+ "value",
+ "values",
+ "version",
+ "window",
+ "word",
+ "words",
+ "work",
+ "write",
+ "writing",
+ ];
+ WORDS.binary_search(&word).is_ok()
}
/// Shorten an [`Routing::Ambiguous`] list to [`AMBIGUOUS_SHOWN`], best first.
@@ -621,7 +799,12 @@ fn decide(evidence: Vec<Evidence>, cached: &[String]) -> (Routing, Vec<(String,
.map(|(slug, _)| slug.clone())
.collect();
if candidates.len() >= 2 {
- return (Routing::Ambiguous(candidates), ranked);
+ // A full overlap of common tokens still scores 0.4 and is a real tie.
+ // A minority overlap clears the candidate floor and is not a choice.
+ if top_score >= AMBIGUOUS_GOOD_MIN {
+ return (Routing::Ambiguous(candidates), ranked);
+ }
+ return (Routing::NoCoverage, ranked);
}
(Routing::NoCorpus, ranked)
@@ -1072,6 +1255,84 @@ mod tests {
}
}
+ #[test]
+ fn a_minority_overlap_is_no_coverage_not_ambiguous() {
+ // Eight corpora so a token shared by two is not common vocabulary
+ // (the quarter cut). Five query tokens, two hits each: 2/5 × 0.8 = 0.32,
+ // over the 0.3 candidate floor and under the 0.35 good-tie bar.
+ let mut entries = vec![
+ entry("alpha", &[], &["aaa", "bbb"]),
+ entry("beta", &[], &["aaa", "ccc"]),
+ ];
+ for i in 0..6 {
+ entries.push(entry(
+ &format!("other{i}"),
+ &[],
+ &[&format!("uniq{i}a"), &format!("uniq{i}b")],
+ ));
+ }
+ let cat = catalogue(entries);
+ let routing = route(
+ "aaa bbb ccc noisea noiseb",
+ empty_project().path(),
+ &cat,
+ None,
+ &[],
+ );
+ assert_eq!(routing, Routing::NoCoverage);
+ }
+
+ #[test]
+ fn a_project_the_catalogue_does_not_contain_is_no_coverage() {
+ // "html" is a language word in four descriptions. On a catalogue large
+ // enough for that to stay distinctive, the identity router used to
+ // call the tie ambiguous. The question names Phaser, which no entry
+ // contains, so the HTML packs do not cover it. A lowercase spelling
+ // is the same gap. "HTML" alone is a real tie. An exact slug still wins.
+ let mut entries = Vec::new();
+ for slug in ["whatwg-html", "lol-html", "parse5", "jsdom"] {
+ let mut e = entry(slug, &[], &[]);
+ e.ask_for = "An agent writing HTML queries this for the document format.".into();
+ entries.push(e);
+ }
+ for i in 0..(151 - entries.len()) {
+ let mut e = entry(&format!("filler-{i}"), &[], &[]);
+ e.ask_for = "an agent working on storage queries this for internals".into();
+ entries.push(e);
+ }
+ let mut lz4 = entry("lz4", &[], &["lz4", "block", "decoder"]);
+ lz4.ask_for = "an agent needing fast block compression queries this for the decoder".into();
+ entries.push(lz4);
+ let cat = catalogue(entries);
+ let proj = empty_project();
+
+ assert_eq!(
+ route("How does Phaser draw HTML?", proj.path(), &cat, None, &[]),
+ Routing::NoCoverage
+ );
+ assert_eq!(
+ route("how does phaser draw HTML?", proj.path(), &cat, None, &[]),
+ Routing::NoCoverage
+ );
+ match route("HTML document format", proj.path(), &cat, None, &[]) {
+ Routing::Ambiguous(slugs) => {
+ assert!(slugs.len() >= 2, "{slugs:?}");
+ assert!(slugs.iter().any(|s| s == "whatwg-html"), "{slugs:?}");
+ }
+ other => panic!("expected Ambiguous, got {other:?}"),
+ }
+ assert_eq!(
+ route(
+ "how does the lz4 decoder work",
+ proj.path(),
+ &cat,
+ None,
+ &[]
+ ),
+ Routing::Confident("lz4".to_string())
+ );
+ }
+
#[test]
fn nonsense_question_is_no_corpus() {
let cat = catalogue(vec![
diff --git a/engine/crates/xerj-hub/tests/identity_routing.rs b/engine/crates/xerj-hub/tests/identity_routing.rs
index 88fe7be7c..c33f085b6 100644
--- a/engine/crates/xerj-hub/tests/identity_routing.rs
+++ b/engine/crates/xerj-hub/tests/identity_routing.rs
@@ -82,7 +82,7 @@ fn picked(routing: &Routing) -> Vec<String> {
match routing {
Routing::Confident(slug) => vec![slug.clone()],
Routing::Group(slugs) | Routing::Ambiguous(slugs) => slugs.clone(),
- Routing::NoCorpus => Vec::new(),
+ Routing::NoCorpus | Routing::NoCoverage => Vec::new(),
}
}
diff --git a/engine/crates/xerj-hub/tests/routing.rs b/engine/crates/xerj-hub/tests/routing.rs
index 5f23ec978..757d7e5de 100644
--- a/engine/crates/xerj-hub/tests/routing.rs
+++ b/engine/crates/xerj-hub/tests/routing.rs
@@ -258,7 +258,7 @@ fn top1_ok(routing: &Routing, expected: &str) -> bool {
Routing::Group(slugs) | Routing::Ambiguous(slugs) => {
slugs.first().map(|s| s == expected).unwrap_or(false)
}
- Routing::NoCorpus => false,
+ Routing::NoCorpus | Routing::NoCoverage => false,
}
}
@@ -313,6 +313,144 @@ fn routing_fixture_meets_accuracy_targets() {
);
}
+/// Issue #2179: a Phaser question must not exit as an ambiguous tie when no
+/// corpus covers that project. A real tie (every query word is in two corpora,
+/// and the score clears the good-match bar) stays ambiguous. An exact slug
+/// still routes.
+#[test]
+fn no_fit_is_no_coverage_a_real_tie_stays_ambiguous_and_a_slug_routes() {
+ let proj = project_dir(&[]);
+
+ // Weak overlap: five tokens, two hits each, score 0.32. Not a choice.
+ let mut weak = vec![
+ corpus_entry("alpha", "", &["aaa", "bbb"]),
+ corpus_entry("beta", "", &["aaa", "ccc"]),
+ ];
+ for i in 0..6 {
+ weak.push(corpus_entry(
+ &format!("other{i}"),
+ "",
+ &[&format!("uniq{i}a"), &format!("uniq{i}b")],
+ ));
+ }
+ assert_eq!(
+ route(
+ "aaa bbb ccc noisea noiseb",
+ proj.path(),
+ &catalogue_of(weak),
+ None,
+ &[]
+ ),
+ Routing::NoCoverage
+ );
+
+ // The question names Phaser. Four corpora mention HTML and nothing names
+ // Phaser; that used to be "multiple corpus match".
+ let mut html = Vec::new();
+ for slug in ["whatwg-html", "lol-html", "parse5", "jsdom"] {
+ html.push(corpus_entry(
+ slug,
+ "An agent writing HTML queries this for the document format.",
+ &[],
+ ));
+ }
+ for i in 0..(151 - html.len()) {
+ html.push(corpus_entry(
+ &format!("filler-{i}"),
+ "an agent working on storage queries this for internals",
+ &[],
+ ));
+ }
+ let html_cat = catalogue_of(html);
+ assert_eq!(
+ route(
+ "How does Phaser draw HTML?",
+ proj.path(),
+ &html_cat,
+ None,
+ &[]
+ ),
+ Routing::NoCoverage,
+ "a project no corpus contains is not an ambiguous tie"
+ );
+ match route("HTML document format", proj.path(), &html_cat, None, &[]) {
+ Routing::Ambiguous(slugs) => assert!(
+ slugs.len() >= 2 && slugs.iter().any(|s| s == "whatwg-html"),
+ "{slugs:?}"
+ ),
+ other => panic!("expected a real HTML tie, got {other:?}"),
+ }
+
+ // Exact slug.
+ let named = catalogue_of(vec![
+ corpus_entry(
+ "lz4",
+ "an agent needing fast block compression queries this for the decoder",
+ &["lz4", "block", "decoder"],
+ ),
+ corpus_entry(
+ "zstd",
+ "an agent tuning compression queries this for frames",
+ &["zstd", "frame"],
+ ),
+ ]);
+ assert_eq!(
+ route(
+ "how does the lz4 decoder work",
+ proj.path(),
+ &named,
+ None,
+ &[]
+ ),
+ Routing::Confident("lz4".into())
+ );
+
+ // Both query words sit in two corpora. Common-token weighting scores the
+ // tie at 0.4, which is still a real ambiguity.
+ let tie = catalogue_of(vec![
+ corpus_entry("alpha", "", &["aaa", "bbb", "ccc", "ddd"]),
+ corpus_entry("beta", "", &["aaa", "bbb", "eee", "fff"]),
+ corpus_entry("gamma", "", &["server", "socket", "packet", "router"]),
+ corpus_entry("delta", "", &["ledger", "balance", "invoice", "levy"]),
+ ]);
+ match route("aaa bbb", proj.path(), &tie, None, &[]) {
+ Routing::Ambiguous(slugs) => {
+ assert!(slugs.contains(&"alpha".to_string()) && slugs.contains(&"beta".to_string()));
+ }
+ other => panic!("expected Ambiguous, got {other:?}"),
+ }
+}
+
+fn corpus_entry(slug: &str, ask_for: &str, terms: &[&str]) -> CatalogueEntry {
+ CatalogueEntry {
+ slug: slug.to_string(),
+ domain: "TEST".to_string(),
+ topics: Vec::new(),
+ ask_for: ask_for.to_string(),
+ upstream: String::new(),
+ pin_sha: "0".repeat(40),
+ pin_date: "2026-01-01".to_string(),
+ refresh_cadence_days: 7,
+ licence_spdx: "Apache-2.0".to_string(),
+ licence_class: LicenceClass::AdaptWithAttribution,
+ pack_bytes: 0,
+ pack_url: String::new(),
+ pack_sha256: String::new(),
+ sig_url: String::new(),
+ dependency_names: Vec::new(),
+ quality_grade: None,
+ term_sketch: xerj_hub::sketch::build(terms.iter().map(|t| (*t).to_string())),
+ }
+}
+
+fn catalogue_of(corpora: Vec<CatalogueEntry>) -> Catalogue {
+ Catalogue {
+ format_version: 1,
+ generated_at: "2026-01-01T00:00:00Z".to_string(),
+ corpora,
+ }
+}
+
#[test]
fn every_expected_slug_is_in_the_catalogue() {
let slugs: Vec<&str> = CORPORA.iter().map(|(s, _, _)| *s).collect();
diff --git a/engine/crates/xerj-mcp/src/hub.rs b/engine/crates/xerj-mcp/src/hub.rs
index 1dde1f928..f920fd4a7 100644
--- a/engine/crates/xerj-mcp/src/hub.rs
+++ b/engine/crates/xerj-mcp/src/hub.rs
@@ -56,7 +56,9 @@ pub fn specs() -> Vec<Value> {
"its pin, licence and whether the pack is already on this machine — not passages; ",
"ask one with xerj_hub_ask. A query that names a different project of the same slug ",
"(fabric.js vs the pack's repo) is reported as a different project instead of a ",
- "silent match. Routed exactly as `xerj ask` routes: your project's dependencies, ",
+ "silent match. A question that names a project no corpus covers is no_coverage \
+ (index it locally with xerj autoindex, then xerj code or xerj search); that is not \
+ an ambiguous tie. Routed exactly as `xerj ask` routes: your project's dependencies, ",
"the signed catalogue's term sketches, and the exact symbol a definition question ",
"names (offline: reads the catalogue `xerj hub sync` cached and never fetches it; ",
"this build has no hosted search). Excluded-licence corpora are never listed."),
@@ -109,7 +111,9 @@ pub fn specs() -> Vec<Value> {
"Ask a hub corpus a question in one turn — no clone, no local indexing. Local packs ",
"first, hosted second, exactly like `xerj ask`. The corpus is optional (routed by ",
"the offline sketch when omitted) and is always named in the answer; when more than ",
- "one corpus fits, the candidates come back and you re-ask with `corpus`. Never ",
+ "one corpus fits, the candidates come back and you re-ask with `corpus`. When no \
+ corpus covers the topic, the result says so and points at local indexing instead of \
+ asking you to pick a corpus. Never ",
"downloads a pack: a corpus not on this machine is answered with the `xerj hub ",
"pull` line to run, and never fetches the catalogue: with none cached it says to run ",
"`xerj hub sync`. Result text is the hub rendering (corpus, pin date, licence, ",
diff --git a/landing/docs/agents/schemas/mcp-tools.json b/landing/docs/agents/schemas/mcp-tools.json
index 424ae46a4..4bf3971d9 100644
--- a/landing/docs/agents/schemas/mcp-tools.json
+++ b/landing/docs/agents/schemas/mcp-tools.json
@@ -865,7 +865,7 @@
},
{
"name": "xerj_hub_find",
- "description": "Discover which hub corpora can answer a question. Each candidate names the upstream repo (owner/name) and a one-line description of what that pack is, plus its pin, licence and whether the pack is already on this machine — not passages; ask one with xerj_hub_ask. A query that names a different project of the same slug (fabric.js vs the pack's repo) is reported as a different project instead of a silent match. Routed exactly as `xerj ask` routes: your project's dependencies, the signed catalogue's term sketches, and the exact symbol a definition question names (offline: reads the catalogue `xerj hub sync` cached and never fetches it; this build has no hosted search). Excluded-licence corpora are never listed.",
+ "description": "Discover which hub corpora can answer a question. Each candidate names the upstream repo (owner/name) and a one-line description of what that pack is, plus its pin, licence and whether the pack is already on this machine — not passages; ask one with xerj_hub_ask. A query that names a different project of the same slug (fabric.js vs the pack's repo) is reported as a different project instead of a silent match. A question that names a project no corpus covers is no_coverage (index it locally with xerj autoindex, then xerj code or xerj search); that is not an ambiguous tie. Routed exactly as `xerj ask` routes: your project's dependencies, the signed catalogue's term sketches, and the exact symbol a definition question names (offline: reads the catalogue `xerj hub sync` cached and never fetches it; this build has no hosted search). Excluded-licence corpora are never listed.",
"inputSchema": {
"type": "object",
"properties": {
@@ -914,7 +914,7 @@
},
{
"name": "xerj_hub_ask",
- "description": "Ask a hub corpus a question in one turn — no clone, no local indexing. Local packs first, hosted second, exactly like `xerj ask`. The corpus is optional (routed by the offline sketch when omitted) and is always named in the answer; when more than one corpus fits, the candidates come back and you re-ask with `corpus`. Never downloads a pack: a corpus not on this machine is answered with the `xerj hub pull` line to run, and never fetches the catalogue: with none cached it says to run `xerj hub sync`. Result text is the hub rendering (corpus, pin date, licence, file:line); structured content is the Answer JSON.",
+ "description": "Ask a hub corpus a question in one turn — no clone, no local indexing. Local packs first, hosted second, exactly like `xerj ask`. The corpus is optional (routed by the offline sketch when omitted) and is always named in the answer; when more than one corpus fits, the candidates come back and you re-ask with `corpus`. When no corpus covers the topic, the result says so and points at local indexing instead of asking you to pick a corpus. Never downloads a pack: a corpus not on this machine is answered with the `xerj hub pull` line to run, and never fetches the catalogue: with none cached it says to run `xerj hub sync`. Result text is the hub rendering (corpus, pin date, licence, file:line); structured content is the Answer JSON.",
"inputSchema": {
"type": "object",
"properties": {
diff --git a/landing/llms-full.txt b/landing/llms-full.txt
index c7dc0d566..953be5a49 100644
--- a/landing/llms-full.txt
+++ b/landing/llms-full.txt
@@ -1693,14 +1693,18 @@ looks like.
(a pack already on this machine answers `<slug>: already present`).
EXIT CODES of `xerj ask`
- 0 answered · 1 no match, or nothing to answer from · 2 usage error or unknown corpus ·
+ 0 answered · 1 no match, nothing to answer from, or no hub corpus covers the topic ·
+ 2 usage error or unknown corpus ·
3 a pack failed its checksum or signature check and was discarded · 4 more than one
- corpus matches: the candidates print one per line, run again with `--corpus <slug>` ·
+ corpus actually matches: the candidates print one per line, run again with `--corpus <slug>` ·
5 this build has no hub public key, so no pack can be verified and nothing was
downloaded (tell the user; a retry will not help).
`no hub corpus matches` followed by `closest corpora: <slug>, …` means routing
was not sure: ask again with `--corpus <slug>` for the one that fits. Without
that line the hub does not hold the answer. Say so; do not answer from memory.
+ `no hub corpus covers this topic` is also exit 1: weak overlaps are not a choice.
+ Index it locally with `xerj autoindex <path>`, then `xerj code` or `xerj search`.
+ `xerj hub find` lists what the hub does have. Do not pick a nearest corpus.
OTHER FLAGS: `--how` ranked passages · `--files` the relevant files with their
licences · `--offline` cached catalogue and packs only · `--no-fetch` do not download
diff --git a/landing/llms.txt b/landing/llms.txt
index 198c2ef91..131de6fef 100644
--- a/landing/llms.txt
+++ b/landing/llms.txt
@@ -65,9 +65,9 @@ Tell the user the corpus, the pin date and the licence your answer rests on, wit
Exit codes of `xerj ask`:
- `0` answered · `2` usage error or unknown corpus
-- `1` no match. If a `closest corpora: …` line follows `no hub corpus matches`, ask again with `--corpus <slug>` for the one that fits; without it the hub does not hold the answer: say so, do not answer from memory
+- `1` no match. If a `closest corpora: …` line follows `no hub corpus matches`, ask again with `--corpus <slug>` for the one that fits; without it the hub does not hold the answer: say so, do not answer from memory. `no hub corpus covers this topic` is the same exit: a weak overlap, or a tie on some other word while the question names a project no corpus contains, is not a choice. Index it locally with `xerj autoindex <path>`, then `xerj code` or `xerj search`. `xerj hub find` lists what the hub does have
- `3` a pack failed its checksum or signature check and was discarded
-- `4` more than one corpus fits: the candidates print, one per line; run again with `--corpus <slug>`
+- `4` more than one corpus actually fits: the candidates print, one per line; run again with `--corpus <slug>`. A topic no corpus covers is exit 1, not this
- `5` this build has no hub public key, so nothing was downloaded: tell the user, a retry will not help
XERJ uses the node at `--url`, `$XERJ_URL` or `localhost:9200` if one answers; otherwise it starts its own on a free loopback port and records its PID.