AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
docs: say plainly that git remotes are not tracked, and give the pull + incremental re-run recipe (#2416) (#2818)
xerj-orgdaede2ac055d
diff --git a/docs/LIVE_REINDEXING.md b/docs/LIVE_REINDEXING.mdindex d5dc1f4d4..f01ff7248 100644--- a/docs/LIVE_REINDEXING.md+++ b/docs/LIVE_REINDEXING.md@@ -326,10 +326,47 @@ operations a month, over four times the entire free allowance. A local `--watch`session makes **zero** API calls while idle. (Watching an object-storage sourceis not implemented; `--watch` watches a local folder.)+## Corpora built from a git repository: following upstream++xerj does **not** track git remotes. It indexes the files in a working tree on+disk and never runs `git fetch` or `git pull` itself, so a corpus built from a+clone stays at whatever commit the clone was at when it was last indexed. There+is no `corpus update` / `pull` verb (#2416).++The supported way to follow upstream is to update the working tree yourself and+re-run the **same** index command (same folder, same `--prefix`, same+`--state-dir` if you passed one). Without `--fresh` that re-run is incremental:+it re-hashes the tree, republishes only files whose bytes changed, and drops+the records of files the pull deleted (the correctness contract below).++```sh+git -C ~/src/project pull --ff-only+xerj autoindex ~/src/project --prefix project+```++To do it on a schedule, put the same two lines in cron, a systemd timer or a CI+job; stop on a failed pull so a merge conflict is never indexed:++```sh+# every hour, at minute 7+7 * * * * git -C "$HOME/src/project" pull --ff-only && xerj autoindex "$HOME/src/project" --prefix project+```++Or keep a watcher on the clone (`xerj autoindex ~/src/project --watch --no-graph+--prefix project`): a `git pull` rewrites files in the working tree, and those+writes are ordinary file events, so the next pass picks them up. Events under+`.git/` are dropped (hidden names), so the watcher reacts to the checked-out+files, not to git's own bookkeeping. You still run `git pull` yourself.++Use `--fresh` only when you want dataset names re-elected from the current tree+(see "The correctness contract"); it rebuilds instead of reconciling.+## What is NOT implemented* No watching on the default graph path — `--watch` requires `--no-graph`.* No object-storage or remote source watching. `--watch` takes a local folder.+* No git-remote awareness: xerj never fetches or pulls. Update the clone, then+ re-run the index (see "Corpora built from a git repository" above).* No incremental generation snapshot: each pass still seals a snapshot over thewhole corpus, which is what dominates per-change latency on a large tree(measured above).