Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

feat(sanitize): widen builtin patterns to six more credential shapes

dk96-creatorbc01c8235b08
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 9d0e27e4..80fe3863 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
+### Added
+- The built-in sanitizer now redacts six further credential shapes, completing
+ three vendor families it already covered only in part. **GitHub:** every
+ token prefix — `gho_` (OAuth, the form `gh auth login` writes to disk),
+ `ghu_`, `ghs_`, `ghr_` — where previously only `ghp_` and `github_pat_`
+ were caught. **AWS:** `ASIA…` STS temporary key ids alongside `AKIA…`.
+ **Stripe:** `rk_live_` restricted keys alongside `sk_live_`. Plus three
+ providers with no prior coverage: **Google OAuth refresh tokens** (`1//…`,
+ longer-lived than the `AIza…` keys already handled — they mint access
+ tokens until revoked), **Meta / Facebook Graph access tokens** (`EAA…`,
+ ad-account, page and business-management scope), and **Telegram bot
+ tokens** (`<bot-id>:AA…`). These run in `BUILTIN_PATTERN_STRS`, so unlike
+ operator `[sanitize].extra_patterns` they also scrub client-side, before an
+ excerpt reaches the local spool or the wire.
+
+ Behaviour change: text matching these shapes is now replaced with
+ `[REDACTED]` where it previously persisted verbatim. An operator who
+ *wants* one of them kept visible (for example a public bot id) can add it
+ to `[sanitize].allowlist`.
+
## [1.28.0] - 2026-08-17
### Fixed
diff --git a/crates/ai-memory-core/src/sanitize.rs b/crates/ai-memory-core/src/sanitize.rs
index d2ffa1e0..d54229b5 100644
--- a/crates/ai-memory-core/src/sanitize.rs
+++ b/crates/ai-memory-core/src/sanitize.rs
@@ -14,8 +14,11 @@
//! ## What we redact
//!
//! Built-in patterns cover bearer tokens, vendor-prefixed API keys
-//! (Anthropic / OpenAI / OpenRouter sk-…, Stripe sk_live_…, GitHub PATs,
-//! Google AIza…, Slack xoxb/xoxp…, AWS AKIA…), PEM-bracketed private
+//! (Anthropic / OpenAI / OpenRouter sk-…, Stripe sk_live_/rk_live_…,
+//! all GitHub token prefixes ghp_/gho_/ghu_/ghs_/ghr_ and fine-grained
+//! github_pat_…, Google AIza… plus OAuth refresh tokens 1//…, Meta /
+//! Facebook Graph EAA…, Telegram bot tokens, Slack xoxb/xoxp…, AWS
+//! AKIA/ASIA…), PEM-bracketed private
//! keys, URL-embedded credentials (`postgres://user:pass@host`), and
//! anything matching the generic `*_(KEY|TOKEN|SECRET|PASSWORD|
//! CREDENTIAL)=value` shape. Operators can extend the list via
@@ -50,12 +53,29 @@ const BUILTIN_PATTERN_STRS: &[&str] = &[
r#"(?i)bearer\s+[A-Za-z0-9._\-+/=]{16,}"#,
// Vendor-prefixed API keys.
r"sk-[A-Za-z0-9_\-]{16,}",
- r"sk_live_[A-Za-z0-9_\-]{16,}",
- r"ghp_[A-Za-z0-9]{20,}",
+ // Stripe secret *and* restricted keys. `rk_live_` is scoped rather than
+ // full-access, but the scope is operator-chosen and routinely includes
+ // charges/refunds — not meaningfully safer than `sk_live_`.
+ r"(?:sk|rk)_live_[A-Za-z0-9_\-]{16,}",
+ // Every GitHub token prefix, not only personal-access: `gho_` (OAuth —
+ // what `gh auth login` stores on disk), `ghu_` (user-to-server),
+ // `ghs_` (server-to-server / Actions), `ghr_` (refresh).
+ r"gh[pousr]_[A-Za-z0-9]{20,}",
r"github_pat_[A-Za-z0-9_]{20,}",
- r"AKIA[0-9A-Z]{12,}",
+ // AWS access-key IDs: long-lived (AKIA) and STS temporary (ASIA).
+ r"(?:AKIA|ASIA)[0-9A-Z]{12,}",
// Naked Google / Gemini API keys.
r"AIza[A-Za-z0-9_\-]{30,}",
+ // Google OAuth refresh tokens. Longer-lived than the AIza keys above:
+ // they mint fresh access tokens until explicitly revoked, so a leaked
+ // one outlives the session it came from.
+ r"1//[0-9A-Za-z_\-]{20,}",
+ // Meta / Facebook Graph API access tokens (ad accounts, pages,
+ // business management).
+ r"EAA[A-Za-z0-9]{20,}",
+ // Telegram bot tokens: <bot-id>:AA<secret>. Grants full control of the
+ // bot, including reading every message it can see.
+ r"\b\d{8,10}:AA[A-Za-z0-9_\-]{32,}",
// Slack tokens (bot/user/admin/app-level/refresh).
r"xox[abprs]-[A-Za-z0-9\-]{10,}",
r"xapp-[A-Za-z0-9\-]{10,}",
@@ -264,6 +284,75 @@ mod tests {
assert!(!out.contains("AIzaSy"));
}
+ // Every fixture below is a SHAPE, never a live value — same rule as the
+ // AIzaSy… test above, and the same `FAKE` convention used by the
+ // `ghp_FAKE…` fixture in ai-memory-wiki. Keep them obviously synthetic
+ // so credential scanners do not flag this repository.
+
+ #[test]
+ fn scrubs_all_github_token_prefixes() {
+ // ghp_ was already covered; gho_/ghu_/ghs_/ghr_ were not, and gho_
+ // is the prefix `gh auth login` writes to disk.
+ for tok in [
+ "ghp_FAKEfakeFAKEfakeFAKEfake012345678",
+ "gho_FAKEfakeFAKEfakeFAKEfake012345678",
+ "ghu_FAKEfakeFAKEfakeFAKEfake012345678",
+ "ghs_FAKEfakeFAKEfakeFAKEfake012345678",
+ "ghr_FAKEfakeFAKEfakeFAKEfake012345678",
+ ] {
+ let out = s().scrub(&format!("token={tok}"));
+ assert!(out.contains("[REDACTED]"), "not redacted: {tok}");
+ assert!(!out.contains("FAKEfake"), "leaked: {tok}");
+ }
+ }
+
+ #[test]
+ fn scrubs_aws_temporary_session_key_id() {
+ // ASIA… is an STS short-lived key id; AKIA… was already covered.
+ let out = s().scrub("aws_access_key_id ASIAFAKEFAKEFAKEFAKE");
+ assert!(out.contains("[REDACTED]"));
+ assert!(!out.contains("ASIAFAKE"));
+ }
+
+ #[test]
+ fn scrubs_stripe_restricted_key() {
+ let out = s().scrub("stripe=rk_live_FAKEfakeFAKE1234");
+ assert!(out.contains("[REDACTED]"));
+ assert!(!out.contains("FAKEfakeFAKE"));
+ }
+
+ #[test]
+ fn scrubs_meta_graph_access_token() {
+ let out = s().scrub("fb=EAAFAKEfakeFAKEfake0123456789");
+ assert!(out.contains("[REDACTED]"));
+ assert!(!out.contains("FAKEfake"));
+ }
+
+ #[test]
+ fn meta_pattern_does_not_eat_short_base64_runs() {
+ // Negative control. The OpenSSH fixture in
+ // `scrubs_pem_private_key_block` contains the substring "EAAAAA";
+ // the {20,} tail is what stops `EAA…` from matching every base64
+ // blob that happens to contain it.
+ let out = s().scrub("harmless b3BlbnNzaC1rZXktdjEAAAAA value");
+ assert!(!out.contains("[REDACTED]"));
+ }
+
+ #[test]
+ fn scrubs_google_oauth_refresh_token() {
+ let out = s().scrub("refresh_token: 1//0gFAKEfakeFAKEfake0123456789");
+ assert!(out.contains("[REDACTED]"));
+ assert!(!out.contains("FAKEfake"));
+ }
+
+ #[test]
+ fn scrubs_telegram_bot_token() {
+ let out = s().scrub("TG 123456789:AAFAKEfakeFAKEfakeFAKEfake0123456789 done");
+ assert!(out.contains("[REDACTED]"));
+ assert!(!out.contains("FAKEfake"));
+ assert!(out.contains("done"), "should not swallow trailing context");
+ }
+
#[test]
fn scrubs_slack_bot_token() {
let out = s().scrub("slack=xoxb-1234567890-abcdefghij");