Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

feat(autoindex): AST extraction for Clojure and Scheme

Yogthos3fef95e65d6e
diff --git a/ROADMAP.md b/ROADMAP.md
index 73296fb1e..b96d31091 100644
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -124,7 +124,7 @@ These are implemented and exercised by real API requests / the test suite / benc
**Exactness, precisely.** No probabilistic sketch sits in the metric path: `cardinality` is a true distinct count rather than an HLL estimate, and `terms` `doc_count` is precise. Two deliberate exceptions, stated the same way in `engine/README.md` and `llms-full.txt`: (1) the **sampling family** is a sample by definition — `run_sampler` sorts the matched documents by `_score` and keeps the first `shard_size` (default **200**), so every sub-aggregation under `sampler`, `random_sampler` or `diversified_sampler` is computed over that slice rather than the whole match set, `diversified_sampler` additionally caps documents per `field` value, and `random_sampler` shares the `sampler` implementation and **ignores ES's `probability`** (an accepted-and-ignored input, #204); (2) `percentiles` with the `hdr` option returns HdrHistogram-quantized values, deliberately, so ES's own outputs reproduce — the default `tdigest` path sorts every value and interpolates instead.
- **Dense-vector kNN** (`knn` query and ES 8.x top-level `knn`): unfiltered kNN on a full-precision cosine field (≥1,024 docs) is served by a **persisted HNSW graph with exact rescoring** — measured recall@10 1.00 on the official bench query, 100-probe mean 0.976 (ES 8.13.4 same protocol: 0.937); filtered/nested kNN, non-cosine similarity, SQ8 fields, and small indexes run the exact brute-force scan (cosine mapped to `(1+cos)/2`).
- **Hybrid search** — BM25 + kNN combined in a single request via the `hybrid` **query type** with `rrf|linear` fusion, verified live. (`fusion: "learned"` is parsed and **rejected** with a 400 naming the supported values — it is not implemented.) (The ES-native top-level `{query, knn}` body also unions both halves since rc.72 — see *Known partials* for its performance caveat.)
-- **Zero-config folder onboarding** — `xerj autoindex <folder>` sniffs files, infers datasets, and creates one index per dataset: tree-sitter AST extraction for 34 languages (symbols, defs, line numbers — the [#295](https://github.com/xerj-org/xerj/issues/295) expansion; still open: Clojure and source-SQL wait on usable grammar crates, Nim/Crystal have none, fixed-form Fortran is deliberately unclaimed), CSV/JSON/JSONL/XML/YAML/SQLite/PDF/DOCX/HTML/log/`.eml`/mbox (Google Takeout) formats, `.gitignore`/`.xerjignore` support, incremental re-runs, and a machine-parseable progress stream.
+- **Zero-config folder onboarding** — `xerj autoindex <folder>` sniffs files, infers datasets, and creates one index per dataset: tree-sitter AST extraction for 36 languages (symbols, defs, line numbers — the [#295](https://github.com/xerj-org/xerj/issues/295) expansion, plus Clojure and Scheme; still open: source-SQL waits on a usable grammar crate, Nim/Crystal have none, fixed-form Fortran is deliberately unclaimed), CSV/JSON/JSONL/XML/YAML/SQLite/PDF/DOCX/HTML/log/`.eml`/mbox (Google Takeout) formats, `.gitignore`/`.xerjignore` support, incremental re-runs, and a machine-parseable progress stream.
- **Agent-memory REST API** (`/_memory/*`), **second-brain knowledge graph** (`/_graph`), **anomaly detection** (`_ml` with continuous datafeeds), **auto-embed on ingest** (default embedder is deterministic **lexical** feature-hashing — never described as neural; `--embed-mode neural` runs the in-binary BERT encoder, `--embed-mode proxy` an external endpoint).
- **The System One wire, answered locally** (since rc.76) — `POST /v1/systemone` (native REST) and `POST /_decide` (ES-compat): typed judgement questions (`noul` / `choice`) answered by a rank-weighted kNN vote over a labelled-history index — no model, no provider key, no egress ([benchmarks/decisions-as-retrieval](./benchmarks/decisions-as-retrieval): Banking77 0.933 accuracy / ECE 0.012, SMS spam 0.983). The acceptance gate is the unmodified pip `jev-reranker` ranked off a XERJ node ([benchmarks/systemone-gate](./benchmarks/systemone-gate)). It is **not** a judge model and claims no zero-shot judgement: where there is no history, use the `rerank` stage and its provider.
- **Columnar storage** — the ZBS2 columnar block with 9 domain-aware encodings, ZSTD/LZ4 codecs, and SQ8 vector quantization, wired into the segment write path.
diff --git a/engine/Cargo.lock b/engine/Cargo.lock
index 3c6e674a0..8a07f2687 100644
--- a/engine/Cargo.lock
+++ b/engine/Cargo.lock
@@ -6037,6 +6037,16 @@ dependencies = [
"tree-sitter-language",
]
+[[package]]
+name = "tree-sitter-clojure-orchard"
+version = "0.2.8"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "67b854ba5935d62a713280b867c09cae08c6ad2286895734b5550ed02122d8fd"
+dependencies = [
+ "cc",
+ "tree-sitter-language",
+]
+
[[package]]
name = "tree-sitter-cpp"
version = "0.23.4"
@@ -6304,6 +6314,16 @@ dependencies = [
"tree-sitter-language",
]
+[[package]]
+name = "tree-sitter-scheme"
+version = "0.24.7"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "8a7e7f156bdf38145f26705d1733185698845307d3e9d9c071ecce4375575131"
+dependencies = [
+ "cc",
+ "tree-sitter-language",
+]
+
[[package]]
name = "tree-sitter-solidity"
version = "1.2.13"
@@ -7331,6 +7351,7 @@ dependencies = [
"tree-sitter-bash",
"tree-sitter-c",
"tree-sitter-c-sharp",
+ "tree-sitter-clojure-orchard",
"tree-sitter-cpp",
"tree-sitter-dart",
"tree-sitter-elixir",
@@ -7357,6 +7378,7 @@ dependencies = [
"tree-sitter-ruby",
"tree-sitter-rust",
"tree-sitter-scala",
+ "tree-sitter-scheme",
"tree-sitter-solidity",
"tree-sitter-swift",
"tree-sitter-typescript",
diff --git a/engine/crates/xerj-autoindex/Cargo.toml b/engine/crates/xerj-autoindex/Cargo.toml
index 7feee60ca..964a564da 100644
--- a/engine/crates/xerj-autoindex/Cargo.toml
+++ b/engine/crates/xerj-autoindex/Cargo.toml
@@ -101,8 +101,11 @@ tree-sitter-haskell = "0.23"
tree-sitter-elixir = "0.3"
# --- #295 Tier 2: common in ecosystems we already serve.
# NOT tree-sitter-clojure: its only release (0.1.0) hard-links core ^0.25.6,
-# mutually exclusive with the core 0.26 that Tier 1's perl grammar forces —
-# revisit when a clojure release against 0.26 exists.
+# mutually exclusive with the core 0.26 that Tier 1's perl grammar forces.
+# The -orchard crate packages the same sogaiu grammar through
+# tree-sitter-language (core is a dev-dependency only), so it loads fine.
+tree-sitter-clojure-orchard = "0.2"
+tree-sitter-scheme = "0.24"
# NOT tree-sitter-sql: 0.0.2 is immature (the issue itself says evaluate
# first or defer) and `.sql` routing must not steal sqldump files — deferred.
tree-sitter-erlang = "0.20"
diff --git a/engine/crates/xerj-autoindex/src/extract/code.rs b/engine/crates/xerj-autoindex/src/extract/code.rs
index c0c49daec..cd40f07a0 100644
--- a/engine/crates/xerj-autoindex/src/extract/code.rs
+++ b/engine/crates/xerj-autoindex/src/extract/code.rs
@@ -296,6 +296,21 @@ fn registry() -> &'static [LangDef] {
tree_sitter_solidity::LANGUAGE.into(),
SOLIDITY_Q,
),
+ def(
+ "clojure",
+ &["clj", "cljs", "cljc"],
+ tree_sitter_clojure_orchard::LANGUAGE.into(),
+ CLOJURE_Q,
+ ),
+ // `.sls` (R6RS library) is deliberately NOT claimed: it is also a
+ // SaltStack state file, which is YAML, and a single-claimant
+ // extension is never content-probed.
+ def(
+ "scheme",
+ &["scm", "ss", "sld", "sps"],
+ tree_sitter_scheme::LANGUAGE.into(),
+ SCHEME_Q,
+ ),
]
})
}
@@ -402,7 +417,7 @@ const SIBLING_SCAN_BUDGET: usize = 256;
const SLICE_MAX_BYTES: usize = DECL_CAP * 32;
/// Does this node kind hold a declaration's IMPLEMENTATION rather than its
-/// signature? 34 grammars spell that node ~20 ways (`block`, `statement_block`,
+/// signature? 36 grammars spell that node ~20 ways (`block`, `statement_block`,
/// `compound_statement`, `class_body`, `body_statement`, `function_body`,
/// `declaration_list`, `field_declaration_list`, `do_block`, `suite`, …), so
/// match the shape rather than enumerate them. A spelling this misses only
@@ -575,6 +590,129 @@ fn is_attribute_kind(kind: &str) -> bool {
&& (kind.contains("attribute") || kind.contains("annotation") || kind.contains("decorator"))
}
+/// Clojure and Scheme spell every form as the same list node, so none of the
+/// shape rules above apply: there is no body kind, no container kind and no
+/// `body:` field, and the climb in `declaration_node` would run from a define
+/// inside an R6RS `(library …)` or a Clojure `(comment …)` up to the whole
+/// enclosing form. Both get the form-based rules below instead.
+fn is_lisp(lang: &str) -> bool {
+ matches!(lang, "clojure" | "scheme")
+}
+
+fn is_lisp_list(kind: &str) -> bool {
+ matches!(kind, "list_lit" | "list")
+}
+
+/// The first element of a list form. Clojure marks elements with the `value:`
+/// field (metadata and comments are children too); Scheme has no fields, so
+/// take the first named child that is not a comment.
+fn lisp_head(list: Node<'_>) -> Option<Node<'_>> {
+ list.child_by_field_name("value").or_else(|| {
+ let mut cur = list.walk();
+ let head = list
+ .named_children(&mut cur)
+ .find(|c| !c.kind().contains("comment"));
+ head
+ })
+}
+
+/// The head symbol's text without namespace or metadata: `defn` for both
+/// `(defn …)` and `(clojure.core/defn …)`.
+fn lisp_head_name<'a>(list: Node<'_>, text: &'a str) -> Option<&'a str> {
+ let head = lisp_head(list)?;
+ let name = head.child_by_field_name("name").unwrap_or(head);
+ text.get(name.byte_range())
+}
+
+/// The definition form a captured name belongs to: the nearest enclosing list
+/// whose head is not, and does not contain, the name. That is `(defn f …)` for
+/// `f`, and for Scheme's `(define (f a) …)` the outer form rather than the
+/// `(f a)` signature list.
+fn lisp_form<'t>(name: Node<'t>) -> Option<Node<'t>> {
+ let mut cur = name.parent();
+ while let Some(n) = cur {
+ if is_lisp_list(n.kind()) {
+ let head = lisp_head(n)?;
+ if head.end_byte() <= name.start_byte() || head.start_byte() >= name.end_byte() {
+ return Some(n);
+ }
+ }
+ cur = n.parent();
+ }
+ None
+}
+
+/// Where a Lisp definition's signature ends: after the element holding the
+/// name and, for Clojure, through a docstring, attribute map or `defmethod`
+/// dispatch value up to the parameter vector. A form on one line is its own
+/// declaration (`None`: the caller takes the whole form).
+fn lisp_head_end(decl: Node<'_>, after: usize) -> Option<usize> {
+ if decl.end_position().row == decl.start_position().row {
+ return None;
+ }
+ let mut end = None;
+ let mut cur = decl.walk();
+ for child in decl.named_children(&mut cur) {
+ let kind = child.kind();
+ if kind.contains("comment") || kind.ends_with("meta_lit") {
+ continue;
+ }
+ if end.is_none() {
+ if child.start_byte() < after && child.end_byte() >= after {
+ end = Some(child.end_byte());
+ }
+ continue;
+ }
+ match kind {
+ "str_lit" | "map_lit" | "kwd_lit" => {}
+ "vec_lit" => return Some(child.end_byte()),
+ _ => break,
+ }
+ }
+ end
+}
+
+/// Forms whose definitions are local: an internal define in a procedure body,
+/// a `defn` built inside a `let`, and anything quoted (a macro template is not
+/// a definition of this file).
+fn lisp_is_local(decl: Node<'_>, text: &str) -> bool {
+ let mut cur = decl.parent();
+ while let Some(n) = cur {
+ if matches!(
+ n.kind(),
+ "quoting_lit" | "syn_quoting_lit" | "quote" | "quasiquote" | "syntax" | "quasisyntax"
+ ) {
+ return true;
+ }
+ if is_lisp_list(n.kind())
+ && lisp_head_name(n, text).is_some_and(|h| {
+ matches!(
+ h,
+ "define"
+ | "define-syntax"
+ | "lambda"
+ | "case-lambda"
+ | "let"
+ | "let*"
+ | "letrec"
+ | "letrec*"
+ | "let-values"
+ | "defn"
+ | "defn-"
+ | "defmacro"
+ | "fn"
+ | "letfn"
+ | "loop"
+ )
+ })
+ {
+ return true;
+ }
+ cur = n.parent();
+ }
+ false
+}
+
/// The declaration node a captured NAME belongs to: climb until the parent is a
/// scope container (a body/block/list) or the tree root. Shape-based because
/// the capture depth of the name differs per grammar — Rust captures the
@@ -582,6 +720,9 @@ fn is_attribute_kind(kind: &str) -> bool {
/// `field_declaration > variable_declarator > identifier`, TypeScript
/// `export_statement > lexical_declaration > variable_declarator > identifier`.
fn declaration_node<'t>(lang: &str, name: Node<'t>) -> Node<'t> {
+ if is_lisp(lang) {
+ return lisp_form(name).unwrap_or(name);
+ }
let mut decl = name;
let mut budget = SIBLING_SCAN_BUDGET;
for _ in 0..DECL_MAX_DEPTH {
@@ -647,6 +788,9 @@ fn gap_is_blank(text: &str, from: usize, to: usize) -> bool {
/// order, so a lambda nested deeper in the body can never win over the body
/// itself.
fn body_start(lang: &str, decl: Node<'_>, after: usize) -> Option<usize> {
+ if is_lisp(lang) {
+ return lisp_head_end(decl, after);
+ }
body_by_kind(lang, decl, after)
.or_else(|| body_by_field(decl, after).map(|b| b.start_byte()))
.or_else(|| end_delimited_head(decl))
@@ -1003,6 +1147,9 @@ fn parse_symbols(def: &LangDef, text: &str) -> Option<Vec<Symbol>> {
// wrong line entirely. Slice [declaration head .. body start), so
// the unit is the complete signature rather than a fragment of it.
let decl = declaration_node(def.name, node);
+ if is_lisp(def.name) && lisp_is_local(decl, text) {
+ continue;
+ }
let head = declaration_head(decl, text);
let from = snap_to_indent(text, head.start_byte());
let to = body_start(def.name, decl, node.end_byte()).unwrap_or_else(|| decl.end_byte());
@@ -1874,6 +2021,40 @@ const SOLIDITY_Q: &str = r#"
(modifier_definition name: (identifier) @function)
"#;
+// No tags.scm in tree-sitter-clojure-orchard 0.2.8. A definition is a list
+// whose head symbol is a def form and whose next element names it; matching
+// the head's `name:` field accepts `clojure.core/defn` too, and the name's
+// sym_name leaves `^:private` metadata out. `lisp_is_local` drops quoted
+// templates and defs built inside a fn body.
+const CLOJURE_Q: &str = r#"
+(list_lit . value: (sym_lit name: (sym_name) @_kw) . value: (sym_lit name: (sym_name) @function)
+ (#any-of? @_kw "defn" "defn-" "defmulti"))
+(list_lit . value: (sym_lit name: (sym_name) @_kw) . value: (sym_lit name: (sym_name) @method)
+ (#eq? @_kw "defmethod"))
+(list_lit . value: (sym_lit name: (sym_name) @_kw) . value: (sym_lit name: (sym_name) @macro)
+ (#eq? @_kw "defmacro"))
+(list_lit . value: (sym_lit name: (sym_name) @_kw) . value: (sym_lit name: (sym_name) @const)
+ (#any-of? @_kw "def" "defonce"))
+(list_lit . value: (sym_lit name: (sym_name) @_kw) . value: (sym_lit name: (sym_name) @interface)
+ (#any-of? @_kw "defprotocol" "definterface"))
+(list_lit . value: (sym_lit name: (sym_name) @_kw) . value: (sym_lit name: (sym_name) @class)
+ (#any-of? @_kw "defrecord" "deftype" "defstruct"))
+(list_lit . value: (sym_lit name: (sym_name) @_kw) . value: (sym_lit name: (sym_name) @module)
+ (#eq? @_kw "ns"))
+"#;
+
+// tree-sitter-scheme 0.24.7 ships no tags.scm and every form is a `list`.
+// `(define (f a) …)` is a procedure, `(define x …)` a value. A library's name
+// is a list, `(my lib)`, captured whole.
+const SCHEME_Q: &str = r#"
+(list . (symbol) @_kw . (list . (symbol) @function) (#eq? @_kw "define"))
+(list . (symbol) @_kw . (symbol) @const (#eq? @_kw "define"))
+(list . (symbol) @_kw . (symbol) @macro (#any-of? @_kw "define-syntax" "define-macro"))
+(list . (symbol) @_kw . (symbol) @struct (#eq? @_kw "define-record-type"))
+(list . (symbol) @_kw . (list . (symbol) @struct) (#eq? @_kw "define-record-type"))
+(list . (symbol) @_kw . (list) @module (#any-of? @_kw "library" "define-library"))
+"#;
+
#[cfg(test)]
mod tests {
use super::*;
@@ -2545,15 +2726,15 @@ mod tests {
/// been counted as its own language by these docs.
#[test]
fn documented_language_count() {
- assert_eq!(registry().len(), 35, "registry row count changed");
+ assert_eq!(registry().len(), 37, "registry row count changed");
let langs: std::collections::HashSet<&str> = registry()
.iter()
.map(|d| d.name.trim_end_matches("_interface"))
.collect();
assert_eq!(
langs.len(),
- 34,
- "docs say 34 languages; update ROADMAP.md and landing/index.html"
+ 36,
+ "docs say 36 languages; update ROADMAP.md and landing/index.html"
);
}
@@ -2600,6 +2781,13 @@ mod tests {
("f95", "fortran"),
("f03", "fortran"),
("sol", "solidity"),
+ ("clj", "clojure"),
+ ("cljs", "clojure"),
+ ("cljc", "clojure"),
+ ("scm", "scheme"),
+ ("ss", "scheme"),
+ ("sld", "scheme"),
+ ("sps", "scheme"),
] {
assert!(is_code_ext(ext), "{ext} must be recognised as code");
let d = registry().iter().find(|d| d.exts.contains(&ext));
@@ -2611,9 +2799,10 @@ mod tests {
}
// Deliberately NOT claimed, each for a stated reason: fixed-form
// Fortran would mis-parse under the free-form grammar; nim/crystal
- // have no usable published grammar; sql routing is deferred; clojure
- // waits on a crate release against core 0.26.
- for ext in ["f", "nim", "cr", "sql", "clj"] {
+ // have no usable published grammar; sql routing is deferred; `.sls`
+ // is an R6RS library but also a SaltStack state file (YAML), and a
+ // single-claimant extension is never content-probed; `.edn` is data.
+ for ext in ["f", "nim", "cr", "sql", "sls", "edn"] {
assert!(!is_code_ext(ext), "{ext} must stay unclaimed (see #295)");
}
}
@@ -3778,6 +3967,105 @@ mod tests {
assert!(!has(&s, "local", "const"), "captured a local: {s:?}");
}
+ fn sym<'a>(s: &'a [Symbol], name: &str) -> &'a Symbol {
+ s.iter()
+ .find(|x| x.name == name)
+ .unwrap_or_else(|| panic!("no {name} in {s:?}"))
+ }
+
+ #[test]
+ fn clojure() {
+ let s = syms(
+ "clojure",
+ "(ns my.app\n (:require [clojure.string :as str]))\n\
+ \n\
+ (def ^:private limit 10)\n\
+ \n\
+ (defn greet\n \"Says hello.\"\n [name]\n (str \"hi \" name))\n\
+ \n\
+ (defn- helper [x] (inc x))\n\
+ (defmacro unless [c & body]\n `(if ~c nil (do ~@body)))\n\
+ (defprotocol Shape\n (area [s]))\n\
+ (defrecord Circle [r]\n Shape\n (area [_] (* r r)))\n\
+ (defmulti render :type)\n\
+ (defmethod render :circle [s]\n s)\n\
+ (comment\n (defn scratch [] 1))\n\
+ (clojure.core/defn qualified [] 2)\n",
+ );
+ assert!(has(&s, "my.app", "module"), "got {s:?}");
+ assert!(has(&s, "limit", "const"), "got {s:?}");
+ assert!(has(&s, "greet", "function"), "got {s:?}");
+ assert!(has(&s, "helper", "function"), "got {s:?}");
+ assert!(has(&s, "unless", "macro"), "got {s:?}");
+ assert!(has(&s, "Shape", "interface"), "got {s:?}");
+ assert!(has(&s, "Circle", "class"), "got {s:?}");
+ assert!(has(&s, "render", "function"), "got {s:?}");
+ assert!(has(&s, "render", "method"), "got {s:?}");
+ assert!(has(&s, "qualified", "function"), "got {s:?}");
+ // Call sites and parameters are not definitions.
+ assert!(
+ !s.iter().any(|x| x.name == "str" || x.name == "x"),
+ "got {s:?}"
+ );
+
+ // The declaration runs through the parameter vector, docstring included,
+ // and stops before the body.
+ let g = sym(&s, "greet");
+ assert_eq!(g.line, 6, "{g:?}");
+ assert_eq!(g.end_line, 9, "{g:?}");
+ assert_eq!(g.code, "(defn greet\n \"Says hello.\"\n [name]", "{g:?}");
+ // A one-line form is its own declaration.
+ assert_eq!(sym(&s, "helper").code, "(defn- helper [x] (inc x))");
+ assert_eq!(sym(&s, "unless").code, "(defmacro unless [c & body]");
+ assert_eq!(sym(&s, "Circle").code, "(defrecord Circle [r]");
+ // A form nested in another (here `comment`) is sliced as itself, not
+ // as the enclosing form.
+ let n = sym(&s, "scratch");
+ assert_eq!(n.code, "(defn scratch [] 1)", "{n:?}");
+ assert_eq!((n.line, n.end_line), (23, 23), "{n:?}");
+ }
+
+ #[test]
+ fn scheme() {
+ let s = syms(
+ "scheme",
+ "(library (my lib)\n (export add)\n (import (rnrs))\n\
+ \x20 (define (add a b)\n (+ a b))\n\
+ \x20 (define limit 10)\n\
+ \x20 (define-syntax swap!\n (syntax-rules ()\n ((_ a b) (let ((t a)) (set! a b) (set! b t)))))\n\
+ \x20 (define-record-type point (fields x y)))\n\
+ (define (top x)\n (define (inner y) y)\n (inner x))\n",
+ );
+ assert!(has(&s, "(my lib)", "module"), "got {s:?}");
+ assert!(has(&s, "add", "function"), "got {s:?}");
+ assert!(has(&s, "limit", "const"), "got {s:?}");
+ assert!(has(&s, "swap!", "macro"), "got {s:?}");
+ assert!(has(&s, "point", "struct"), "got {s:?}");
+ assert!(has(&s, "top", "function"), "got {s:?}");
+ assert!(
+ !s.iter().any(|x| x.name == "a" || x.name == "t"),
+ "got {s:?}"
+ );
+ // An internal define is a local of its procedure, like a
+ // function-local const elsewhere.
+ assert!(
+ !s.iter().any(|x| x.name == "inner"),
+ "captured a local: {s:?}"
+ );
+
+ // Inside an R6RS `library` form a define is sliced as itself, not as
+ // the whole library.
+ let a = sym(&s, "add");
+ assert_eq!(a.code, "(define (add a b)", "{a:?}");
+ assert_eq!((a.line, a.end_line), (4, 5), "{a:?}");
+ assert_eq!(sym(&s, "limit").code, "(define limit 10)");
+ assert_eq!(sym(&s, "swap!").code, "(define-syntax swap!");
+ assert_eq!(sym(&s, "(my lib)").code, "(library (my lib)");
+ let t = sym(&s, "top");
+ assert_eq!(t.code, "(define (top x)", "{t:?}");
+ assert_eq!((t.line, t.end_line), (11, 13), "{t:?}");
+ }
+
#[test]
fn objc() {
let s = syms(
diff --git a/landing/index.html b/landing/index.html
index f2434bcfb..ef68b57c9 100644
--- a/landing/index.html
+++ b/landing/index.html
@@ -429,7 +429,7 @@ $ curl -sXPOST localhost:9200/_memory/agent-demo/_recall \
<div class="row">
<div class="k">AST code search</div>
- <div class="v">Source is parsed with tree-sitter (34 languages) into symbols with kind and line number plus a searchable definitions field — the agent retrieves a <em>function with its contract</em>, not a grep line that needs the whole file opened to judge</div>
+ <div class="v">Source is parsed with tree-sitter (36 languages) into symbols with kind and line number plus a searchable definitions field — the agent retrieves a <em>function with its contract</em>, not a grep line that needs the whole file opened to judge</div>
</div>
<div class="row">
<div class="k">Token economics</div>