AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
docs(corpus): say corpora do not follow git remotes, and how to refresh one (#2416) (#2704)
Cristian Groza466740057794
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex ac831cd35..8091c2ca2 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -41,6 +41,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0### Fixed+- `xerj corpus --help`, `xerj autoindex --help` (`--watch`) and the XERJ.code README now say that corpora and folders do not follow their git remotes, and give the refresh recipe: `git pull` in each checkout, re-run `xerj corpus add` so `corpus.json` records the new commit, then `xerj corpus index`. (#2416)- `xerj_batch_search` (and `xerj batch-search`): a batch whose combined `_msearch` response passes the 32 MiB cap no longer fails every item. The request is retried in halves, and only an item whose own response is over the cap fails, with `response_too_large`. (#2694)- `xerj autoindex` and `xerj autoindex map`: when the node is unreachable at a `localhost` URL, the error now also suggests the same URL with `127.0.0.1`, since the node binds the IPv4 loopback by default and `localhost` can resolve to `::1` first (seen on Windows). (#2688)- The node binds its ES-compat and REST listeners before it opens itsdiff --git a/engine/crates/xerj-autoindex/src/cli.rs b/engine/crates/xerj-autoindex/src/cli.rsindex 6cf47945a..ee5ee17ea 100644--- a/engine/crates/xerj-autoindex/src/cli.rs+++ b/engine/crates/xerj-autoindex/src/cli.rs@@ -445,6 +445,9 @@ pub fn help_text_with(feedback: bool) -> String {\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20--fresh. One OS watch per indexed directory, no polling;\n\\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20respects the same ignore rules as a re-run.\n\\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20docs/LIVE_REINDEXING.md has the measurements.\n\+ \x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20It watches the working tree and never fetches a git\n\+ \x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20remote: it reindexes what a `git pull` changes, and\n\+ \x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20without --watch you re-run the same command.\n\\x20\x20\x20\x20--debounce <MS> --watch quiet period before a pass (default 400, max\n\\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x2060000). One editor save is several filesystem events.\n\\x20\x20\x20\x20--max-minutes <N> stop and ask before indexing if phase A's MEASURED estimate\n\diff --git a/engine/crates/xerj-autoindex/src/xc.rs b/engine/crates/xerj-autoindex/src/xc.rsindex 4aac2a118..872f4973c 100644--- a/engine/crates/xerj-autoindex/src/xc.rs+++ b/engine/crates/xerj-autoindex/src/xc.rs@@ -480,6 +480,12 @@ not marked own unless you pass the flag, and a --from manifest cannot setit — that file is untrusted. Re-adding the corpus keeps a marker alreadyrecorded. `xerj corpus own <name>` sets it on an existing corpus; `--off`clears it. Hub pack licence classes are unchanged;+a corpus does not follow its git remotes. To refresh one added from+git URLs, run `git -C <checkout> pull --ff-only` in each checkout under+corpora/<name>/, re-run the same `xerj corpus add <name> <git-url>...` so+corpus.json records the new commit, then `xerj corpus index <name>`, which+updates the build in place. A --from corpus stays at its manifest's pins;+change the manifest to move it;list shows what is loaded on this node.";pub(crate) fn git(dir: Option<&Path>, args: &[&str]) -> Result<(i32, String)> {diff --git a/tools/xerj-code/README.md b/tools/xerj-code/README.mdindex 0a00469dd..b95705525 100644--- a/tools/xerj-code/README.md+++ b/tools/xerj-code/README.md@@ -98,6 +98,27 @@ the clone.Pre-built *indexes* are a different question (on-disk format version, embedderidentity, redistribution of the indexed source) and are not offered here.+## Following upstream++A corpus does not track its git remotes: nothing fetches or polls them. A+`xerj corpus index` run reads the checkouts as they are, so an upstream commit+is not searchable until you pull it. To refresh a corpus added from git URLs:++```sh+git -C ~/.xerj-code/corpora/<name>/<repo> pull --ff-only # each checkout+xerj corpus add <name> <git-url>... # same URLs; records the new commit SHAs+xerj corpus index <name> # updates the build in place+```++`corpus add` keeps a checkout it finds (`[skip] … already cloned`) and records+its current commit. `corpus index` alone would index the pulled files too, but+`corpus.json` would keep naming the old commit, and a `--from` rebuild of it+would come back without them. A corpus built `--from` a manifest stays at the+manifest's pins; change the SHAs there to move it.++For a folder indexed with `xerj autoindex <dir>`, pull and re-run the same+command; `--watch` reindexes the files a pull changes, but never fetches.+## Licences are recorded, and reviewed`xerj corpus add` classifies each repo from the licence file it actually ships,