Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
fix(wiki): drop a leading BOM on a page with no frontmatter
Max Freedom Pollard54a933f0afea
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex d88c7516..a0fdfbe6 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -36,6 +36,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0per-event disposition runs. The install-time "enforced"/"NOT in force"messaging is corrected to match: a script-fallback install is now theonly path flagged as unenforced. (#661)+- A UTF-8 BOM on a hand-edited wiki page that has no frontmatter no longer+ rides into the page body. `markdown::parse` stripped the mark before looking+ for the frontmatter fence, but the no-frontmatter path returned the untouched+ input as the body, so the mark stayed in front of the first line. Two things+ followed from that: `derive_title` no longer read the leading `# ` as a+ heading, so the page was indexed under its filename instead of its title, and+ the OKF v0.2 file pass (which conforms frontmatter and leaves the body alone)+ re-emitted the mark after the closing `---` fence, where it is a stray+ zero-width no-break space rather than a byte-order mark. Both paths now drop+ the leading BOM, which is what the frontmatter path already did. (#663)## [2.1.0] - 2026-09-06diff --git a/crates/ai-memory-wiki/src/markdown.rs b/crates/ai-memory-wiki/src/markdown.rsindex 5fa24735..837fe2c0 100644--- a/crates/ai-memory-wiki/src/markdown.rs+++ b/crates/ai-memory-wiki/src/markdown.rs@@ -28,6 +28,12 @@ pub struct Markdown {/// Recognises only the canonical `---\n<yaml>\n---\n` block at the very/// start of the document. Anything else is treated as body.///+/// A leading UTF-8 BOM is dropped either way. It only means "this file is+/// UTF-8" while it sits at offset zero; carried into `body` it is a+/// zero-width no-break space in front of the first line, which hides an H1+/// from [`derive_title`] and rides into the body a later re-emit writes+/// back after the frontmatter fence.+////// # Errors/// Returns [`WikiError::Yaml`] if the frontmatter block exists but does/// not parse as YAML.@@ -47,7 +53,7 @@ pub fn parse(input: &str) -> WikiResult<Markdown> {}Ok(Markdown {frontmatter: serde_json::Value::Null,- body: input.to_string(),+ body: trimmed.to_string(),})}@@ -551,6 +557,26 @@ mod tests {assert_eq!(md.body, "Body\n");}+ /// A page a Windows editor saved with a UTF-8 BOM and no frontmatter:+ /// the mark belongs to the file, not to the first line. Left in `body`+ /// it sits in front of the `#`, so the H1 stops being a heading and the+ /// page is indexed under its filename instead of its title.+ #[test]+ fn parses_bom_prefixed_body_without_frontmatter() {+ let src = "\u{FEFF}# Hand written\n\nBody.\n";+ let md = parse(src).unwrap();+ assert!(md.frontmatter.is_null());+ assert_eq!(md.body, "# Hand written\n\nBody.\n");+ assert_eq!(+ derive_title(+ &md.frontmatter,+ &md.body,+ &PagePath::new("notes/hand-written.md").unwrap(),+ ),+ "Hand written"+ );+ }+#[test]fn malformed_frontmatter_returns_error() {let src = "---\ntitle: [unterminated\n---\nBody\n";diff --git a/crates/ai-memory-wiki/src/migrations/m2026_09_okf_conformance.rs b/crates/ai-memory-wiki/src/migrations/m2026_09_okf_conformance.rsindex eb252039..1429aae7 100644--- a/crates/ai-memory-wiki/src/migrations/m2026_09_okf_conformance.rs+++ b/crates/ai-memory-wiki/src/migrations/m2026_09_okf_conformance.rs@@ -542,6 +542,34 @@ mod tests {assert!(crate::backup::BackupReceipt::load(tmp.path()).is_none());}+ /// The file pass leaves the body untouched, and a UTF-8 BOM is not part+ /// of the body: it means "this file is UTF-8" only at offset zero. A+ /// hand-edited page a Windows editor saved with one and no frontmatter+ /// used to come back through `parse` with the mark still on the front of+ /// the body, so this pass wrote it out AFTER the frontmatter fence, where+ /// it is a stray zero-width no-break space ahead of the H1.+ #[test]+ fn conforming_a_bom_prefixed_page_does_not_move_the_mark_into_the_body() {+ let tmp = TempDir::new().unwrap();+ let rel = Path::new("w/p/notes/hand-written.md");+ let abs = tmp.path().join(rel);+ std::fs::create_dir_all(abs.parent().unwrap()).unwrap();+ std::fs::write(&abs, "\u{FEFF}# Hand written\n\nBody.\n").unwrap();++ conform_file(tmp.path(), rel, &std::collections::HashMap::new()).unwrap();++ let conformed = std::fs::read_to_string(&abs).unwrap();+ assert!(+ !conformed.contains('\u{FEFF}'),+ "the mark survived the file pass: {conformed:?}"+ );+ assert!(+ conformed.ends_with("---\n# Hand written\n\nBody.\n"),+ "the body must reach disk exactly as it was authored: {conformed:?}"+ );+ assert_eq!(parse(&conformed).unwrap().frontmatter["type"], "Note");+ }+// ---- #633: the safety archive must be taken BEFORE the DB migration ----/// The core assertion for #633: the pre-open snapshot captures the DB as it