Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

docs(feedback): field report for multi-array JSON autoindex (#2132)

xerj-orgaaac8012c2e6
diff --git a/user-feedback/16-agent-field-reports/2026-10-22-json-array-autoindex.md b/user-feedback/16-agent-field-reports/2026-10-22-json-array-autoindex.md
new file mode 100644
index 000000000..958c748cb
--- /dev/null
+++ b/user-feedback/16-agent-field-reports/2026-10-22-json-array-autoindex.md
@@ -0,0 +1,13 @@
+# Indexing a multi-array JSON file with autoindex (2026-10-22)
+
+**Agent:** Cursor Grok 4.7 · **XERJ:** xerj v1.0.0-rc.97 · **Platform:** Linux 6.12.94+ x86_64
+
+**Pointed at:** the public #1856 file `subsystem_map.json` (1,081,818 bytes, two top-level arrays of objects) and a synthetic five-array JSON stand-in (533,186 bytes). Lexical feature-hash; `--embed-mode neural` was not set. The original #1978 file (42.7 MB) was not attached, so it was not indexed.
+
+**Used it for:** `xerj autoindex` of each file (`--insecure --no-graph --disable-feedback --max-minutes 0`), then a mapping check and one match query on a third file that tripped the semantic cap.
+
+**Verdict:** One object with several arrays kept only the largest as rows and pasted every other array onto each row. The pasted `runs` blob was 25,493 bytes of English keys (word_ratio 0.6016), so it became `semantic_text`, and file-level progress sat still for the whole file. After the split the public file finished as two indices in 1.7s, and a prose column over the byte cap stayed searchable as `text` with a warning that names `--no-semantic`. I would not quote the new memory ceiling as RSS; it is a structural estimate, and the five-array stand-in is not the missing 42.7 MB file.
+
+**Numbers:** `subsystem_map.json`: `xerj-done ok=true exit=0 wall=1.7s records=8953` (8589 and 364 docs); client VmHWM 164228 kB, server VmHWM 288064 kB, python wall 1.760s. Five-array file: 9430 records, `wall=1.8s exit=0`, client VmHWM 154316 kB; server VmHWM 545280 kB, already 288064 kB before that run. Cap file 10,498,891 bytes, 4000 rows of 2600 characters: search total 4000, `note` mapped `text`, `wall=1.7s`, client VmHWM 183956 kB.
+
+**Filed alongside:** https://github.com/xerj-org/xerj/pull/2131