AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
bench(harbor-3arm): flag partial arm coverage + neutralize the tuned prompt's pre-asserted claim (#516)
xerj-orgd99b7de25000
diff --git a/tools/benchmarks/harbor-3arm/collect_results.py b/tools/benchmarks/harbor-3arm/collect_results.pyindex 58e01ce03..d5d8aa031 100755--- a/tools/benchmarks/harbor-3arm/collect_results.py+++ b/tools/benchmarks/harbor-3arm/collect_results.py@@ -103,12 +103,22 @@ def main():for (bench, model, arm), tasks in cells.items():by_bm[(bench, model)][arm] = {t for t, v in tasks.items() if v["reward"] is not None}+ canonical_arms = {"native", "xerj", "xerj-tuned"}for (bench, model), arms in sorted(by_bm.items()):if len(arms) < 2:continue+ present = set(arms)common = set.intersection(*arms.values())+ # A comparison that isn't the full 3-arm set must say so LOUDLY — an+ # absent arm otherwise reads as a footnote and a reader can mistake, e.g.,+ # native vs xerj-tuned for the headline result (#516).+ coverage = "" if present == canonical_arms else (+ f" ⚠ PARTIAL ARM COVERAGE — missing "+ f"{sorted(canonical_arms - present) or 'nothing'}, "+ f"unexpected {sorted(present - canonical_arms) or 'nothing'}; "+ f"this is NOT the canonical 3-arm comparison")print(f"\n{bench} / {model}: {len(common)} tasks finished in all "- f"{len(arms)} arms")+ f"{len(arms)} arms ({', '.join(sorted(present))}){coverage}")for arm in sorted(arms):tv = cells[(bench, model, arm)]k = sum(1 for t in common if tv[t]["resolved"])diff --git a/tools/benchmarks/harbor-3arm/xerj_prompt_tuned.j2 b/tools/benchmarks/harbor-3arm/xerj_prompt_tuned.j2index d68b3e8a4..4a3702851 100644--- a/tools/benchmarks/harbor-3arm/xerj_prompt_tuned.j2+++ b/tools/benchmarks/harbor-3arm/xerj_prompt_tuned.j2@@ -2,8 +2,9 @@---TOOL NOTE: XERJ code search is installed (server at http://localhost:9210) and-tuned for this work. It is usually MUCH faster than grep for finding the right-definition in an unfamiliar repo. Playbook:+tuned for this work. It can return the enclosing definition of a symbol directly+(the whole function/class body), so you rarely need to open files to read a+definition. Playbook:1. Index once (seconds; --no-semantic keeps it pure BM25 — the embedder islexical, so semantic fields add cost without adding meaning):