Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

fix(sanitize): match Telegram's documented bot-token shape

dk96-creator49f6a2bfc812
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c243be8b..9cbda278 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -18,7 +18,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
longer-lived than the `AIza…` keys already handled — they mint access
tokens until revoked), **Meta / Facebook Graph access tokens** (`EAA…`,
ad-account, page and business-management scope), **Telegram bot
- tokens** (`<bot-id>:AA…`), and **GoHighLevel Private Integration Tokens**
+ tokens** (`<bot-id>:…`, covering both the `AA…` form every issued token uses
+ and the shape Telegram's own docs publish), and **GoHighLevel Private
+ Integration Tokens**
(`pit-…`, which do not expire until manually revoked). These run in
`BUILTIN_PATTERN_STRS`, so unlike
operator `[sanitize].extra_patterns` they also scrub client-side, before an
diff --git a/crates/ai-memory-core/src/sanitize.rs b/crates/ai-memory-core/src/sanitize.rs
index 916257f0..52e368cd 100644
--- a/crates/ai-memory-core/src/sanitize.rs
+++ b/crates/ai-memory-core/src/sanitize.rs
@@ -73,9 +73,18 @@ const BUILTIN_PATTERN_STRS: &[&str] = &[
// Meta / Facebook Graph API access tokens (ad accounts, pages,
// business management).
r"EAA[A-Za-z0-9]{20,}",
- // Telegram bot tokens: <bot-id>:AA<secret>. Grants full control of the
- // bot, including reading every message it can see.
- r"\b\d{8,10}:AA[A-Za-z0-9_\-]{32,}",
+ // Telegram bot tokens: <bot-id>:<secret>. Grants full control of the bot,
+ // including reading every message it can see. Two branches on purpose:
+ // - `AA…` is the shape every issued token has taken, left open-ended so a
+ // future length change cannot silently retire the rule.
+ // - the second branch matches the shape Telegram's own docs publish
+ // (`123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11` — 6-digit id, no `AA`),
+ // length-anchored because without the `AA` anchor a bare `\d+:[\w-]+`
+ // also matches timestamps, ratios, `host:port` maps, SRT cues and
+ // `<short-sha>:<hex>` pairs.
+ // Thanks to @tahazarif10 for spotting that the documented example fell
+ // outside the original `\d{8,10}:AA…` form.
+ r"\b\d{6,10}:(?:AA[A-Za-z0-9_\-]{30,}|[A-Za-z0-9_\-]{34,35})\b",
// GoHighLevel Private Integration Tokens. The `pit-` prefix is what the
// vendor documents (their MCP guide shows `Bearer pit-your-token`); the
// tail is NOT documented anywhere, and every token observed in the wild
@@ -361,6 +370,35 @@ mod tests {
assert!(out.contains("done"), "should not swallow trailing context");
}
+ #[test]
+ fn scrubs_telegram_documented_example_shape() {
+ // Regression for #408: Telegram's own Bot API docs publish
+ // `123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11` — a 6-digit id and no
+ // `AA` prefix — which the original `\d{8,10}:AA…` form could not match.
+ // This is the vendor's placeholder, not a live token.
+ let out = s().scrub("token 123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11 ok");
+ assert!(out.contains("[REDACTED]"));
+ assert!(!out.contains("ABC-DEF1234"));
+ assert!(out.contains("ok"), "should not swallow trailing context");
+ }
+
+ #[test]
+ fn telegram_pattern_does_not_eat_colon_separated_prose() {
+ // Negative control, and the reason the non-`AA` branch is
+ // length-anchored rather than open-ended: dropping the anchor entirely
+ // makes `\d+:[\w-]+` match all of these.
+ for benign in [
+ "built 2026:08 release notes",
+ "aspect ratio 16:9 widescreen",
+ "ports 8080:my-service-name-here",
+ "00:00:00,000 --> 00:00:04,120 caption",
+ "commit 12345678:deadbeefcafebabe0123456789abcdef",
+ ] {
+ let out = s().scrub(benign);
+ assert!(!out.contains("[REDACTED]"), "false positive on: {benign}");
+ }
+ }
+
#[test]
fn scrubs_gohighlevel_private_integration_token() {
// Uppercase hex on purpose: the vendor documents no case, so the