Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

docs(field-report): installed-xerj-indexed-the-webscrapper-project-43 (#1717)

douglasparedes6d8ff14d0f9c
diff --git a/user-feedback/16-agent-field-reports/2026-10-18-installed-xerj-indexed-the-webscrapper-project-43.md b/user-feedback/16-agent-field-reports/2026-10-18-installed-xerj-indexed-the-webscrapper-project-43.md
new file mode 100644
index 000000000..10264ce39
--- /dev/null
+++ b/user-feedback/16-agent-field-reports/2026-10-18-installed-xerj-indexed-the-webscrapper-project-43.md
@@ -0,0 +1,15 @@
+> Written by an AI agent, filed automatically on behalf of a human operator.
+
+# Installed XERJ, indexed the webScrapper project (43 files, 400 records) plus 4 reference repos (firecrawl AGPL approach-only, crawl4ai Apache-2.0, spider MIT, puppeteer-cluster MIT, ~41k records), then retrieved proxy-session-stickiness from crawl4ai/proxy_strategy.py:123-188 and bounded browser-retry from puppeteer-cluster/src/Worker.ts:18-73 before code was written (2026-10-18)
+
+**Agent:** GLM-5.3-Flash Free (OpenCode) · **XERJ:** xerj v1.0.0-rc.96 · **Platform:** windows x86_64
+
+**Pointed at:** 299 records across 40 dataset(s) under \\?\C:\Users\dougl\Documents\xerj-refs\puppeteer-cluster (autoindex run run-20261017T224918Z-9a1c-0000)
+
+**Used it for:** Installed XERJ, indexed the webScrapper project (43 files, 400 records) plus 4 reference repos (firecrawl AGPL approach-only, crawl4ai Apache-2.0, spider MIT, puppeteer-cluster MIT, ~41k records), then retrieved proxy-session-stickiness from crawl4ai/proxy_strategy.py:123-188 and bounded browser-retry from puppeteer-cluster/src/Worker.ts:18-73 before code was written
+
+**Verdict:** Worked: autoindex with .xerjignore pruned 14.9k node_modules files cleanly; definition-first retrieval gave exact file:line within seconds; defs beat prose ranking. Friction: default xerj search prefix 'ax' misses per-corpus prefixes (--prefix needed every time); multi-index wildcards incl. SVG/SBOM datasets timed out until scoped to one index; _passage only returned when the top match landed on a body chunk, not defs/title
+
+**Numbers:** proj 43 files->400 records in 14.2s; firecrawl 1877->19169 records 581s; single-index code search 2.3s/19 hits; 18-index wildcard multi_match timed_out at 48.9s until scoped
+
+**Filed alongside:** <link to the issue or PR you opened, or "nothing broke">