Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
feat(sanitize): widen builtin patterns to six more credential shapes
dk96-creatorbc01c8235b08
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 9d0e27e4..80fe3863 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]+### Added+- The built-in sanitizer now redacts six further credential shapes, completing+ three vendor families it already covered only in part. **GitHub:** every+ token prefix — `gho_` (OAuth, the form `gh auth login` writes to disk),+ `ghu_`, `ghs_`, `ghr_` — where previously only `ghp_` and `github_pat_`+ were caught. **AWS:** `ASIA…` STS temporary key ids alongside `AKIA…`.+ **Stripe:** `rk_live_` restricted keys alongside `sk_live_`. Plus three+ providers with no prior coverage: **Google OAuth refresh tokens** (`1//…`,+ longer-lived than the `AIza…` keys already handled — they mint access+ tokens until revoked), **Meta / Facebook Graph access tokens** (`EAA…`,+ ad-account, page and business-management scope), and **Telegram bot+ tokens** (`<bot-id>:AA…`). These run in `BUILTIN_PATTERN_STRS`, so unlike+ operator `[sanitize].extra_patterns` they also scrub client-side, before an+ excerpt reaches the local spool or the wire.++ Behaviour change: text matching these shapes is now replaced with+ `[REDACTED]` where it previously persisted verbatim. An operator who+ *wants* one of them kept visible (for example a public bot id) can add it+ to `[sanitize].allowlist`.+## [1.28.0] - 2026-08-17### Fixeddiff --git a/crates/ai-memory-core/src/sanitize.rs b/crates/ai-memory-core/src/sanitize.rsindex d2ffa1e0..d54229b5 100644--- a/crates/ai-memory-core/src/sanitize.rs+++ b/crates/ai-memory-core/src/sanitize.rs@@ -14,8 +14,11 @@//! ## What we redact//!//! Built-in patterns cover bearer tokens, vendor-prefixed API keys-//! (Anthropic / OpenAI / OpenRouter sk-…, Stripe sk_live_…, GitHub PATs,-//! Google AIza…, Slack xoxb/xoxp…, AWS AKIA…), PEM-bracketed private+//! (Anthropic / OpenAI / OpenRouter sk-…, Stripe sk_live_/rk_live_…,+//! all GitHub token prefixes ghp_/gho_/ghu_/ghs_/ghr_ and fine-grained+//! github_pat_…, Google AIza… plus OAuth refresh tokens 1//…, Meta /+//! Facebook Graph EAA…, Telegram bot tokens, Slack xoxb/xoxp…, AWS+//! AKIA/ASIA…), PEM-bracketed private//! keys, URL-embedded credentials (`postgres://user:pass@host`), and//! anything matching the generic `*_(KEY|TOKEN|SECRET|PASSWORD|//! CREDENTIAL)=value` shape. Operators can extend the list via@@ -50,12 +53,29 @@ const BUILTIN_PATTERN_STRS: &[&str] = &[r#"(?i)bearer\s+[A-Za-z0-9._\-+/=]{16,}"#,// Vendor-prefixed API keys.r"sk-[A-Za-z0-9_\-]{16,}",- r"sk_live_[A-Za-z0-9_\-]{16,}",- r"ghp_[A-Za-z0-9]{20,}",+ // Stripe secret *and* restricted keys. `rk_live_` is scoped rather than+ // full-access, but the scope is operator-chosen and routinely includes+ // charges/refunds — not meaningfully safer than `sk_live_`.+ r"(?:sk|rk)_live_[A-Za-z0-9_\-]{16,}",+ // Every GitHub token prefix, not only personal-access: `gho_` (OAuth —+ // what `gh auth login` stores on disk), `ghu_` (user-to-server),+ // `ghs_` (server-to-server / Actions), `ghr_` (refresh).+ r"gh[pousr]_[A-Za-z0-9]{20,}",r"github_pat_[A-Za-z0-9_]{20,}",- r"AKIA[0-9A-Z]{12,}",+ // AWS access-key IDs: long-lived (AKIA) and STS temporary (ASIA).+ r"(?:AKIA|ASIA)[0-9A-Z]{12,}",// Naked Google / Gemini API keys.r"AIza[A-Za-z0-9_\-]{30,}",+ // Google OAuth refresh tokens. Longer-lived than the AIza keys above:+ // they mint fresh access tokens until explicitly revoked, so a leaked+ // one outlives the session it came from.+ r"1//[0-9A-Za-z_\-]{20,}",+ // Meta / Facebook Graph API access tokens (ad accounts, pages,+ // business management).+ r"EAA[A-Za-z0-9]{20,}",+ // Telegram bot tokens: <bot-id>:AA<secret>. Grants full control of the+ // bot, including reading every message it can see.+ r"\b\d{8,10}:AA[A-Za-z0-9_\-]{32,}",// Slack tokens (bot/user/admin/app-level/refresh).r"xox[abprs]-[A-Za-z0-9\-]{10,}",r"xapp-[A-Za-z0-9\-]{10,}",@@ -264,6 +284,75 @@ mod tests {assert!(!out.contains("AIzaSy"));}+ // Every fixture below is a SHAPE, never a live value — same rule as the+ // AIzaSy… test above, and the same `FAKE` convention used by the+ // `ghp_FAKE…` fixture in ai-memory-wiki. Keep them obviously synthetic+ // so credential scanners do not flag this repository.++ #[test]+ fn scrubs_all_github_token_prefixes() {+ // ghp_ was already covered; gho_/ghu_/ghs_/ghr_ were not, and gho_+ // is the prefix `gh auth login` writes to disk.+ for tok in [+ "ghp_FAKEfakeFAKEfakeFAKEfake012345678",+ "gho_FAKEfakeFAKEfakeFAKEfake012345678",+ "ghu_FAKEfakeFAKEfakeFAKEfake012345678",+ "ghs_FAKEfakeFAKEfakeFAKEfake012345678",+ "ghr_FAKEfakeFAKEfakeFAKEfake012345678",+ ] {+ let out = s().scrub(&format!("token={tok}"));+ assert!(out.contains("[REDACTED]"), "not redacted: {tok}");+ assert!(!out.contains("FAKEfake"), "leaked: {tok}");+ }+ }++ #[test]+ fn scrubs_aws_temporary_session_key_id() {+ // ASIA… is an STS short-lived key id; AKIA… was already covered.+ let out = s().scrub("aws_access_key_id ASIAFAKEFAKEFAKEFAKE");+ assert!(out.contains("[REDACTED]"));+ assert!(!out.contains("ASIAFAKE"));+ }++ #[test]+ fn scrubs_stripe_restricted_key() {+ let out = s().scrub("stripe=rk_live_FAKEfakeFAKE1234");+ assert!(out.contains("[REDACTED]"));+ assert!(!out.contains("FAKEfakeFAKE"));+ }++ #[test]+ fn scrubs_meta_graph_access_token() {+ let out = s().scrub("fb=EAAFAKEfakeFAKEfake0123456789");+ assert!(out.contains("[REDACTED]"));+ assert!(!out.contains("FAKEfake"));+ }++ #[test]+ fn meta_pattern_does_not_eat_short_base64_runs() {+ // Negative control. The OpenSSH fixture in+ // `scrubs_pem_private_key_block` contains the substring "EAAAAA";+ // the {20,} tail is what stops `EAA…` from matching every base64+ // blob that happens to contain it.+ let out = s().scrub("harmless b3BlbnNzaC1rZXktdjEAAAAA value");+ assert!(!out.contains("[REDACTED]"));+ }++ #[test]+ fn scrubs_google_oauth_refresh_token() {+ let out = s().scrub("refresh_token: 1//0gFAKEfakeFAKEfake0123456789");+ assert!(out.contains("[REDACTED]"));+ assert!(!out.contains("FAKEfake"));+ }++ #[test]+ fn scrubs_telegram_bot_token() {+ let out = s().scrub("TG 123456789:AAFAKEfakeFAKEfakeFAKEfake0123456789 done");+ assert!(out.contains("[REDACTED]"));+ assert!(!out.contains("FAKEfake"));+ assert!(out.contains("done"), "should not swallow trailing context");+ }+#[test]fn scrubs_slack_bot_token() {let out = s().scrub("slack=xoxb-1234567890-abcdefghij");