Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

fix(autoindex): refuse records_field on non-.json files; state cisa-kev licence and snapshot precisely

xerj-org519eab7cc19e
diff --git a/engine/crates/xerj-autoindex/src/harvest/mod.rs b/engine/crates/xerj-autoindex/src/harvest/mod.rs
index 21a5288ca..567fbccde 100644
--- a/engine/crates/xerj-autoindex/src/harvest/mod.rs
+++ b/engine/crates/xerj-autoindex/src/harvest/mod.rs
@@ -376,10 +376,19 @@ fn parse_file(path: &Path, records_field: Option<&str>) -> Result<Vec<Value>> {
.and_then(|n| n.to_str())
.unwrap_or_default()
.to_lowercase();
- if name.ends_with(".md") || name.ends_with(".markdown") {
+ let markdown = name.ends_with(".md") || name.ends_with(".markdown");
+ let jsonl = name.ends_with(".jsonl") || name.ends_with(".ndjson");
+ // `records_field` only means something for a wrapped `.json` catalog; on
+ // any other file type it would be silently ignored, so refuse instead.
+ if let (Some(field), true) = (records_field, markdown || jsonl) {
+ bail!(
+ "{}: records_field '{field}' applies only to .json files, not .md/.jsonl",
+ path.display()
+ );
+ }
+ if markdown {
return Ok(vec![Value::String(raw)]);
}
- let jsonl = name.ends_with(".jsonl") || name.ends_with(".ndjson");
if jsonl {
let mut out = Vec::new();
for (n, line) in raw.lines().enumerate() {
@@ -511,6 +520,14 @@ mod tests {
.unwrap_err()
.to_string();
assert!(err.contains("missing or not a JSON array"), "{err}");
+ let err = parse_file(&b, Some("vulnerabilities"))
+ .unwrap_err()
+ .to_string();
+ assert!(err.contains("applies only to .json files"), "{err}");
+ let err = parse_file(&d, Some("vulnerabilities"))
+ .unwrap_err()
+ .to_string();
+ assert!(err.contains("applies only to .json files"), "{err}");
}
// ── end to end: recipe → harvest → identity → pack, twice ─────────────
diff --git a/tools/packs/cisa-kev/README.md b/tools/packs/cisa-kev/README.md
index e372ba101..5bde17963 100644
--- a/tools/packs/cisa-kev/README.md
+++ b/tools/packs/cisa-kev/README.md
@@ -33,8 +33,26 @@ SHA-256 so the three cannot drift apart.
`shortDescription`, `requiredAction`, `knownRansomwareCampaignUse`, `cwes`).
- **Derived fields** via `regex_extract` (`cwe_ids`) and `map`
(`ransomware_status`: `"known"` vs `"unknown"`).
-- **Public domain licence (`CC0-1.0`)**: US federal government work under
- 17 U.S.C. § 105.
+- **Licence (`CC0-1.0`)**: CISA distributes the KEV catalog under Creative
+ Commons CC0 1.0 (stated in the `LICENSE` of CISA's official mirror,
+ [cisagov/kev-data](https://github.com/cisagov/kev-data/blob/develop/LICENSE)),
+ consistent with its status as a US federal government work
+ (17 U.S.C. § 105). Two caveats from that licence travel with the pack:
+ content at third-party links inside the `notes` field is governed by those
+ sites' own terms, and using the data does not authorize use of the CISA logo
+ or DHS seal, nor imply endorsement by CISA or DHS.
+
+## Snapshot and provenance
+
+The source is CISA's live feed URL, which is **not version-pinned** and grows
+as CISA adds entries; a rebuild on a later day produces more records than the
+table above. The numbers above were measured against catalog version
+`2026.10.15` (`dateReleased` 2026-10-15T17:00:11Z, `count` 1746) and
+independently re-counted from that same feed during review. Refresh them with
+`cargo test -p xerj-autoindex --test pack_claims measure_cisa_kev_pack_live -- --ignored --nocapture`.
+
+This pack is **not** published or signed by `.github/workflows/pack-publish.yml`
+and ships no signing key; build it locally from the recipe.
## Build, consume, query