The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu
Go agent-ai-cliagent-harnessesagent-securitybrowser-extensionciclaude-codecloudcodexcursordetection-engineeringjevknowledge-basememorymobile-device-managementobservabilitysecuritysecurity-information-and-event-managementsecurity-toolstelemetrytraces
fix(privacy): redact the credential after any Authorization scheme (#1232)
Saurav Singh8868be6edcf9
diff --git a/docs/changelog/cli.mdx b/docs/changelog/cli.mdxindex 71c34624..9e5af7e5 100644--- a/docs/changelog/cli.mdx+++ b/docs/changelog/cli.mdx@@ -19,6 +19,8 @@ Changes on `main` since v1.3.60. They ship in the next release.- **A skipped backfill also covers runtimes added later** · An install or connect with `--no-backfill` or `BEACON_BACKFILL=0` floors each session-store runtime that existed then, so a session sync does not read the 30 days the operator opted out of. A runtime a later upgrade added had no floor and no cursor, so the first session sync read and, on a connected endpoint, forwarded its whole 30-day window. Such a runtime now reads from the earliest recorded floor.- **`devin sync --watch` no longer records hooked sessions twice** · The watch computed which sessions the Devin hooks already cover once, at start, so a session the hooks began recording afterwards had its prompts and tool events written again by every sweep. Coverage is now read again before each sweep; `antigravity sync --watch` does the same.- **Inventory names each MCP server's package and each skill's source** · An inventory snapshot knew an MCP server or skill only by its configured name, which several Beacon Intel artifacts can share. Each MCP server record now carries `package` (`npm:<name>` or `pypi:<name>`, no version) and `package_registry` `public` when its launch command runs a package from the public registry. When the command, the server's env, or the npm, pip or uv configuration in the home, the system's locations or the project a server runs in points that ecosystem elsewhere, `package_registry` is `private` and the name stays off the record. On Windows that configuration includes the per-user `AppData` and machine `ProgramData` locations even when the scanning job's environment does not name them. A skill the skills CLI (`npx skills add`) installed from GitHub carries `intel_key`, `skill:<owner>/<repo>#<name>`, from the CLI's `~/.agents/.skill-lock.json`; a lock beside a repository's own skills is ignored, since the repository could claim any source. The system inventory job runs as root, so it reads that configuration in a short-lived child process running as the console user with their groups: what the user's own launchers cannot read, Beacon does not either, and a snapshot never reflects a file only root can see. Configuration that exists but cannot be read (a file or folder the user may not open, a folder macOS privacy protection guards, a FIFO) holds its ecosystem back as `private`. `intel_check_installed` now checks such a skill by default, by its exact key; other skills still go by name only with `include_private_names`. Every endpoint sends one fresh snapshot after upgrading.+- **Redaction covers an Authorization header with any scheme** · An `Authorization` header whose scheme was not Bearer, Basic, Token or Digest kept its credential. `Authorization: ApiKey <key>` (Elasticsearch), and the same with `SSWS` (Okta), `Bot` (Discord), `Splunk`, `Negotiate` or a vendor's own scheme, was written as `Authorization: [REDACTED] <key>`: the scheme word was replaced and the key stayed, in command lines, tool output and everything else Beacon retains or forwards. The scheme and the credential are now replaced together, as `Authorization: [REDACTED]`, which is what a header quoted whole as a JSON value already became. Bearer, Basic, Token and Digest keep their scheme name as before. A scheme that sends a list of parameters (Digest, Hawk, AWS Signature V4) has every one replaced, where the scheme rules used to take the first parameter's name and leave the rest; a copy of the credential elsewhere in the same text is removed, as a labeled secret's copies are; and a base64url credential that starts with `-` is no longer mistaken for a flag and kept.+- **Redacted values inside JSON-encoded commands stay valid JSON** · A command held as a JSON-encoded string, such as a tool's arguments, quotes its arguments with `\"`. An unquoted secret after a label, a URL parameter or an Authorization scheme took the backslash of the closing `\"` with it, so `"curl -d \"token=x\""` became `"curl -d \"token=[REDACTED]"`, a bare quote that ended the string, and whatever read the arguments back could not parse them. The value now ends before the backslash.## November 2026diff --git a/docs/security/retention-redaction.mdx b/docs/security/retention-redaction.mdxindex 3cb192a9..8e7bb7b5 100644--- a/docs/security/retention-redaction.mdx+++ b/docs/security/retention-redaction.mdx@@ -24,7 +24,7 @@ Redaction runs on retained text (prompts, command output, diffs, tool arguments|------|----------|| A value labeled as a secret | `token=…`, `GITHUB_TOKEN=…`, `AWS_SECRET_ACCESS_KEY=…`, `{"api_key": "…"}`, `client_secret: …`, and a URL query parameter `?api_key=…`, whose value ends at the next `&` or `#` so the other parameters stay || A structured field named for a credential | the value of any field whose name ends in `password`, `secret`, `token`, `api_key`, `access_key`, `private_key`, `authorization`, `credential(s)` or `cookie` |-| Authorization headers and flags | `Authorization: Bearer …`, `Authorization: Basic …`, `--password …`, `--token=…`, and snake_case or key flags such as `--client_secret …`, `--api_key …`, `--private-key …`, `--secret-key …` |+| Authorization headers and flags | `Authorization: Bearer …`, `Authorization: Basic …`, an Authorization header with any other scheme, where the scheme goes with the credential (`Authorization: ApiKey …` becomes `Authorization: [REDACTED]`), and every parameter of a scheme that sends a list of them (Digest, Hawk, AWS Signature V4), along with any copy of the credential elsewhere in the same text, `--password …`, `--token=…`, and snake_case or key flags such as `--client_secret …`, `--api_key …`, `--private-key …`, `--secret-key …` || URL credentials | the password in `postgres://user:…@host`, `https://x-access-token:…@github.com` || Private keys | PEM and OpenSSH `PRIVATE KEY` blocks, including a block cut off before its end line || Recognizable tokens (the public type prefix stays, as in `ghp_[REDACTED]`) | Anthropic and OpenAI `sk-…`, AWS `AKIA…`/`ASIA…`, GitHub `ghp_…`/`github_pat_…`, GitLab `glpat-…`, Slack `xox…-` and `xapp-…`, Stripe `sk_live_…` and webhook signing secrets `whsec_…`, Google `AIza…`, npm, Hugging Face, HashiCorp Vault `hvs.…`, Groq `gsk_…`, Perplexity `pplx-…`, Replicate `r8_…`, xAI `xai-…`, LangSmith `lsv2_…`, Pinecone `pcsk_…`, DigitalOcean `dop_v1_…`, Databricks `dapi…`, PyPI `pypi-…`, Shopify `shpat_…`, Linear `lin_api_…`, Doppler `dp.pt.…`, SendGrid `SG.…`, Postman `PMAK-…`, Docker Hub `dckr_pat_…`/`dckr_oat_…`, Supabase `sbp_…` and secret keys `sb_secret_…`, Netlify `nfp_…`, Beacon keys and tokens (`bcn_api_…`, `bcn_device_…`, `bcn_enfdev_…`, `bcn_mcp_…`, `bcn_mcprt_…`, and the `beacon login` token `bcn_cli_…` with its `bcn_code_…`, `bcn_client_…` and `bcn_exchange_…` values), and JWTs |diff --git a/pkg/asymptoteobserve/dedupe.go b/pkg/asymptoteobserve/dedupe.goindex 75750e3c..7efa9f7e 100644--- a/pkg/asymptoteobserve/dedupe.go+++ b/pkg/asymptoteobserve/dedupe.go@@ -217,8 +217,9 @@ func mayDuplicate(line, session []byte, action string) bool {return bytes.Contains(line, session) && containsFold(line, action)}-// containsFold reports whether s holds needle, which is lower case ASCII, in any case.-func containsFold(s []byte, needle string) bool {+// containsFold reports whether s holds needle, which is lower case ASCII, in any case. A window+// of len(needle) bytes folds to an ASCII needle only byte for byte, so ASCII folding is all of it.+func containsFold[S ~string | ~[]byte](s S, needle string) bool {if needle == "" {return true}@@ -227,7 +228,14 @@ func containsFold(s []byte, needle string) bool {upper = first - 'a' + 'A'}for i := 0; i+len(needle) <= len(s); i++ {- if (s[i] == first || s[i] == upper) && bytes.EqualFold(s[i:i+len(needle)], []byte(needle)) {+ if s[i] != first && s[i] != upper {+ continue+ }+ match := true+ for j := 1; j < len(needle) && match; j++ {+ match = asciiLower(s[i+j]) == asciiLower(needle[j])+ }+ if match {return true}}diff --git a/pkg/asymptoteobserve/privacy.go b/pkg/asymptoteobserve/privacy.goindex d097684b..b58d69a7 100644--- a/pkg/asymptoteobserve/privacy.go+++ b/pkg/asymptoteobserve/privacy.go@@ -101,11 +101,19 @@ func quotedRules(kind quotedKind, keep func(string) bool) []redactionRule {// with them, so the two read the same value.const redactValueStops = redactQuotes + "," + regexpSpace-// redactUnquoted is an unquoted value, up to one of redactValueStops. Brackets, braces,-// backslashes, `#`, `&` and `?` are part of it: a generated password is full of them. It never-// starts with a quote, escaped or not, so it leaves a value the quoted rule already replaced+// redactEscapedNonQuote is a backslash and the byte it escapes, unless that byte is a quote. An+// unquoted value takes it, since a generated password is full of backslashes, but stops before a+// backslash that escapes a quote: that is the closing quote of a value inside a JSON-encoded+// string, such as a command held in a tool's arguments (`"curl -d \"token=x\""`), and a value that+// took the backslash left a bare quote that ended the string early. redactUnquotedEnd reads values+// the same way.+const redactEscapedNonQuote = `\\[^` + redactQuotes + `]`++// redactUnquoted is an unquoted value, up to one of redactValueStops or an escaped quote. Brackets,+// braces, backslashes, `#`, `&` and `?` are part of it: a generated password is full of them. It+// never starts with a quote, escaped or not, so it leaves a value the quoted rule already replaced// alone.-const redactUnquoted = `(?:[^` + redactValueStops + `\\]|\\[^` + redactQuotes + `])[^` + redactValueStops + `]*`+const redactUnquoted = `(?:[^` + redactValueStops + `\\]|` + redactEscapedNonQuote + `)+`// redactParamStops are the bytes a URL query parameter's value stops at: redactValueStops, and// the `&` that opens the next parameter and the `#` that opens the fragment.@@ -114,7 +122,27 @@ const redactParamStops = redactValueStops + "&#"// redactParamValue is redactUnquoted for the value of a URL query parameter, a label opened by// `?` or `&`: `?api_key=x&user=alice` keeps `&user=alice`. Only there do `&` and `#` end a value;// after any other label they are a password's own characters (`DB_PASSWORD=#Xy9&z`).-const redactParamValue = `(?:[^` + redactParamStops + `\\]|\\[^` + redactQuotes + `])[^` + redactParamStops + `]*`+const redactParamValue = `(?:[^` + redactParamStops + `\\]|` + redactEscapedNonQuote + `)+`++// redactUnquotedEnd is where an unquoted value starting at t[i] ends, read as redactUnquoted reads+// it with stops: at a stop, or at a backslash that escapes a quote or ends the text. A backslash+// takes the byte after it, whatever it is.+func redactUnquotedEnd(t string, i int, stops string) int {+ for i < len(t) {+ if t[i] == '\\' {+ if i+1 == len(t) || strings.IndexByte(redactQuotes, t[i+1]) >= 0 {+ break+ }+ i += 2+ continue+ }+ if strings.IndexByte(stops, t[i]) >= 0 {+ break+ }+ i+++ }+ return i+}// redactBare is a bare value after a quoted label: a JSON number or literal, which ends at the// structure around it.@@ -369,6 +397,25 @@ func skipFlagSeparator(rest string) (string, bool) {// regexpSpace is \s in Go's regexp syntax: ASCII whitespace, never Unicode.const regexpSpace = " \t\n\f\r"+// redactAuthorizationLabel is an Authorization header's name and separator, as the header rules+// read it: `Authorization: `, `AUTHORIZATION=`, `"authorization": "`, `Proxy-Authorization: `.+const redactAuthorizationLabel = `authorization["']?\s*[:=]\s*["']?`++// redactListedAuthSchemes are the Authorization schemes the header rule names, and whose name it+// keeps: detection rules match on what kind of credential a line held. isAuthSchemeName reads the+// same list.+const redactListedAuthSchemes = `bearer|basic|token|digest`++// redactAuthParam is one auth-param of an Authorization header (RFC 9110 §11.2): a name, `=`, and+// a token or a quoted string.+const redactAuthParam = `[a-z0-9_.-]+[ \t]*=[ \t]*(?:"(?:[^"\\\n]|\\.)*"|` + redactUnquoted + `)`++// redactAuthCredentials is what follows an Authorization header's scheme: a comma-separated list+// of auth-params, the way Digest, Hawk and AWS Signature V4 send theirs, or a single token (RFC+// 9110's token68, or whatever a vendor sends, up to a stop). The list is taken whole, so no+// parameter is left behind a marker that took only its name.+const redactAuthCredentials = `(?:` + redactAuthParam + `(?:[ \t]*,[ \t]*` + redactAuthParam + `)*|` + redactUnquoted + `)`+// isAuthScheme reports whether a labeled value is an Authorization scheme, alone or with its// credential already redacted, rather than a credential. `Authorization: Bearer x` is redacted to// `Authorization: Bearer [REDACTED]` by the header rule, and the assignment rules must not then@@ -378,13 +425,109 @@ func isAuthScheme(value string) bool {if len(fields) == 0 || len(fields) > 2 || (len(fields) == 2 && fields[1] != redactedValue) {return false}- switch strings.ToLower(fields[0]) {- case "bearer", "basic", "token", "digest":+ return isAuthSchemeName(fields[0])+}++// isAuthSchemeName reports whether word is one of redactListedAuthSchemes.+func isAuthSchemeName(word string) bool {+ for rest := redactListedAuthSchemes; rest != ""; {+ var name string+ name, rest, _ = strings.Cut(rest, "|")+ if strings.EqualFold(word, name) {+ return true+ }+ }+ return false+}++// keepAuthorizationValue reports whether the rule for an Authorization value with any scheme should+// leave a match alone, for the rules after it to read as they did before it existed:+// - The value opens with a scheme the header rule names. That rule has already replaced the+// credential after it and kept the scheme.+// - The second word is not a credential but the next item on the line: a flag named for a secret+// (`--token`, `--password=`) or a secret label and its separator (`password:`, `PASSWORD=`,+// `"api_key":`), whose own rule then redacts what it introduces; a short option (`-u`), whose+// value a command's own rule reads, as curl's; or a `#` comment. Taken as the credential,+// these would be replaced and the secret after them would stay. Any other word goes: a+// credential can start with `-` (base64url), and a long flag that names no secret is safe to+// replace.+//+// A list of auth-params, which holds a comma, is never kept: the rule takes it whole, values+// included, so no label in it has a value left to lose.+func keepAuthorizationValue(value string) bool {+ fields := strings.Fields(value)+ if len(fields) == 0 {+ return false+ }+ if isAuthSchemeName(fields[0]) {+ return true+ }+ if len(fields) < 2 || strings.Contains(value, ",") {+ return false+ }+ second := fields[1]+ if second[0] == '#' || len(second) == 2 && second[0] == '-' && second[1] != '-' {+ return true+ }+ if flag, _, _ := strings.Cut(second, "="); IsSecretFlag(flag) {return true}+ if i := strings.IndexAny(second, ":="); i > 0 {+ return labelEndsAt(second[:i+1])+ }return false}+// authorizationCredentialCopy is the credential after scheme, the first word of an Authorization+// header's value, which ends at t[end], for its bare copies elsewhere in the text to be removed as+// an assigned value's are; or "". The capture of assigned values reads a label's value only up to+// the blank after the scheme, so without this a copy of the credential (`… ; echo <key>`) stayed.+// A match the header rules would not make is not taken: a first word that is no scheme-shaped+// word, such as a marker an earlier pass left, or a match the any-scheme rule keeps. Nor is a list+// of auth-params, whose labeled values the capture already takes, nor a word of letters alone,+// which is more likely the next word of prose than a credential.+func authorizationCredentialCopy(t, scheme string, end int) string {+ if !isAuthSchemeWord(scheme) {+ return ""+ }+ from := end+ for from < len(t) && (t[from] == ' ' || t[from] == '\t') {+ from+++ }+ if from == end {+ return ""+ }+ credential := t[from:redactUnquotedEnd(t, from, redactValueStops)]+ if strings.Contains(strings.TrimRight(credential, "="), "=") || onlyASCIILetters(credential) {+ return ""+ }+ if !isAuthSchemeName(scheme) && keepAuthorizationValue(scheme+" "+credential) {+ return ""+ }+ return credential+}++// isAuthSchemeWord reports whether word has the shape the any-scheme rule reads as a scheme,+// `[a-z][a-z0-9_.-]*` in any case.+func isAuthSchemeWord(word string) bool {+ for i := range len(word) {+ c := asciiLower(word[i])+ if (c < 'a' || c > 'z') && (i == 0 || (c < '0' || c > '9') && c != '_' && c != '.' && c != '-') {+ return false+ }+ }+ return word != ""+}++func onlyASCIILetters(s string) bool {+ for i := range len(s) {+ if c := asciiLower(s[i]); c < 'a' || c > 'z' {+ return false+ }+ }+ return true+}+// isRedactedJWTPrefix keeps the `eyJ` the header rule left in `Bearer eyJ[REDACTED]`, which the// bearer rule would otherwise take for a token of its own.func isRedactedJWTPrefix(value string) bool {@@ -414,9 +557,19 @@ var redactionRules = concatRules([]redactionRule{// An Authorization header's credential after its scheme: Bearer, Basic, Token, Digest. A JWT// keeps its public `eyJ` prefix, as the shape rules below keep theirs, so `Bearer eyJ[REDACTED]`// still says a token was there where `Bearer [REDACTED]` may only have been the word "token" or- // a `$VARIABLE`.- foldedRule(`(?i)(authorization["']?\s*[:=]\s*["']?(?:bearer|basic|token|digest)[ \t]+(?-i:eyJ)?)([^"'\s,]+)()`, "${1}"+redactedValue, nil, "authorization"),+ // a `$VARIABLE`. Digest's parameters go together (redactAuthCredentials).+ foldedRule(`(?i)(`+redactAuthorizationLabel+`(?:`+redactListedAuthSchemes+`)[ \t]+(?-i:eyJ)?)(`+redactAuthCredentials+`)()`, "${1}"+redactedValue, nil, "authorization"),foldedRule(`(?i)(\bbearer[ \t]+(?-i:eyJ)?)(?P<value>[a-z0-9._~+/=-]+)`, "${1}"+redactedValue, isRedactedJWTPrefix, "bearer"),+ // An Authorization header with any other scheme: `ApiKey` (Elasticsearch), `SSWS` (Okta), `Bot`+ // (Discord), `Splunk`, `Negotiate`, Hawk and AWS Signature V4's parameter lists, and whatever a+ // vendor names its own. The label rules below read the scheme word as the value and stop at the+ // blank after it, which replaced the scheme and left the credential in the text. The scheme and+ // the credential go together here, as they already do when the value is quoted whole+ // (`"Authorization": "ApiKey x"`). The first word is not kept the way a listed scheme is: a+ // credential sent with no scheme is also one word, and kept as if it were a scheme it would stay+ // whenever something follows it on the line. A second word that is a secret flag or a secret+ // label is left to the rules for those (keepAuthorizationValue).+ foldedRule(`(?i)(`+redactAuthorizationLabel+`)(?P<value>[a-z][a-z0-9_.-]*[ \t]+`+redactAuthCredentials+`)`, "${1}"+redactedValue, keepAuthorizationValue, "authorization"),// A labeled assignment in prose, shell, env files, YAML, TOML or JSON: `token=x`,// `export GITHUB_TOKEN="x"`, `{"api_key": "x"}`. The closing quote of a quoted label is allowed// between the label and the separator. A quoted value runs to its closing quote; the unquoted@@ -692,9 +845,7 @@ func urlRunsOn(t string, m []int) int {// The label's value: the marker the URL rule wrote and what follows it. A key id it ends// in, before any separators it takes (`:`, `|`), takes its secret with it.for run := end; ; {- for end < len(t) && strings.IndexByte(redactValueStops, t[end]) < 0 {- end++- }+ end = redactUnquotedEnd(t, end, redactValueStops)tailEnd := awsPairTailEnd(t, run, end)if tailEnd < 0 {break@@ -1084,6 +1235,14 @@ func keptLabel(t string, end int) bool {// that a later rule can change it, or make it long enough to remove.func captureAssigned(text *redactGate, engine *redactEngine, assigned map[string]struct{}) (_ map[string]struct{}, kept bool) {value := text.value+ add := func(s string) {+ if len(s) >= redactCopyMin && s != redactedValue && !isAuthScheme(s) {+ if assigned == nil {+ assigned = map[string]struct{}{}+ }+ assigned[s] = struct{}{}+ }+ }for match := range redactAssignedRule().matches(text, &engine.assigned) {// The value is in one group: each way of writing it is an alternative.g := 2@@ -1093,15 +1252,15 @@ func captureAssigned(text *redactGate, engine *redactEngine, assigned map[stringif g+1 >= len(match) {continue}- kept = kept || g+2 == len(match) && strings.ContainsAny(value[match[0]:match[g]], redactQuotes)+ unquoted := g+2 == len(match)+ kept = kept || unquoted && strings.ContainsAny(value[match[0]:match[g]], redactQuotes)// A value that is itself a label hides the assignment it begins from the capture, as a kept// one does from the label rules.text.hidden = text.hidden || keptLabel(value, match[g+1])- if s := value[match[g]:match[g+1]]; len(s) >= redactCopyMin && s != redactedValue && !isAuthScheme(s) {- if assigned == nil {- assigned = map[string]struct{}{}- }- assigned[s] = struct{}{}+ s := value[match[g]:match[g+1]]+ add(s)+ if unquoted && containsFold(value[match[0]:match[g]], "authorization") {+ add(authorizationCredentialCopy(value, s, match[g+1]))}}return assigned, keptdiff --git a/pkg/asymptoteobserve/privacy_test.go b/pkg/asymptoteobserve/privacy_test.goindex fb8620b1..ae40b4df 100644--- a/pkg/asymptoteobserve/privacy_test.go+++ b/pkg/asymptoteobserve/privacy_test.go@@ -376,6 +376,184 @@ func TestRedactStringIsIdempotent(t *testing.T) {}}+// An Authorization header names a scheme and then the credential. For a scheme the header rule+// does not list, the label rules took the scheme word for the value and stopped at the blank, so+// the scheme was replaced and the credential after it stayed: `Authorization: [REDACTED] <key>`.+// Elasticsearch (ApiKey, the scheme docs/log-forwarding/elastic.mdx has people type), Okta (SSWS),+// Discord (Bot), Splunk HEC and Kerberos (Negotiate) all send theirs this way.+func TestRedactStringAuthorizationCredentialAfterAnyScheme(t *testing.T) {+ const credential = "VnVhQ2ZHY0JDZGJrUW0tZTVhT3g6dWkybHAyYXhUTm1zeWFrdzl0dk5udw=="+ for input, want := range map[string]string{+ "Authorization: ApiKey " + credential: "Authorization: [REDACTED]",+ "Authorization: SSWS " + credential: "Authorization: [REDACTED]",+ "Authorization: Bot " + credential: "Authorization: [REDACTED]",+ "Authorization: Splunk " + credential: "Authorization: [REDACTED]",+ "Authorization: sso-key " + credential: "Authorization: [REDACTED]",+ "authorization=ApiKey " + credential: "authorization=[REDACTED]",+ "Proxy-Authorization: Negotiate " + credential: "Proxy-Authorization: [REDACTED]",+ `curl -H "Authorization: ApiKey ` + credential + `" https://es.example:9200/`: `curl -H "Authorization: [REDACTED]" https://es.example:9200/`,+ "curl -H 'Authorization: SSWS " + credential + "' https://example.okta.com": "curl -H 'Authorization: [REDACTED]' https://example.okta.com",+ "headers:\n Authorization: ApiKey " + credential + "\n Accept: json": "headers:\n Authorization: [REDACTED]\n Accept: json",+ // Quoted whole, the value was already one marker; the two forms now agree.+ `{"Authorization": "ApiKey ` + credential + `"}`: `{"Authorization": "[REDACTED]"}`,+ // A variable reference after the scheme goes with it, as it does after Bearer.+ `-H "Authorization: ApiKey $ES_API_KEY"`: `-H "Authorization: [REDACTED]"`,+ } {+ got := RedactString(input)+ if got != want {+ t.Errorf("RedactString(%q) = %q, want %q", input, got, want)+ }+ if strings.Contains(got, credential) {+ t.Errorf("RedactString(%q) kept the credential: %q", input, got)+ }+ if twice := RedactString(got); twice != got {+ t.Errorf("RedactString is not idempotent:\nonce %q\ntwice %q", got, twice)+ }+ }++ // The schemes the header rule lists keep their name, as before: detection rules match on what+ // kind of credential a line held.+ for input, want := range map[string]string{+ "Authorization: Bearer " + credential: "Authorization: Bearer [REDACTED]",+ "Authorization: Basic " + credential: "Authorization: Basic [REDACTED]",+ "Authorization: Token " + credential: "Authorization: Token [REDACTED]",+ "Authorization: Digest " + credential: "Authorization: Digest [REDACTED]",+ "Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.e30.sig": "Authorization: Bearer eyJ[REDACTED]",+ `{"authorization": "Basic dXNlcjpwYXNz"}`: `{"authorization": "Basic [REDACTED]"}`,+ } {+ if got := RedactString(input); got != want {+ t.Errorf("RedactString(%q) = %q, want %q", input, got, want)+ }+ }++ // A credential sent with no scheme is the first word, and is still the one replaced. Reading+ // it as a scheme to keep would leave it in the text whenever something follows it on the line,+ // so the word after it goes too, unless it is a comment. A long flag that names no secret goes+ // with it; replacing it hides nothing.+ for input, want := range map[string]string{+ "Authorization: " + credential: "Authorization: [REDACTED]",+ "Authorization: sk-live-abcdef0123456789 # prod": "Authorization: [REDACTED] # prod",+ "AUTHORIZATION=abcdef0123456789 --verbose": "AUTHORIZATION=[REDACTED]",+ "Authorization: $TOKEN extra": "Authorization: [REDACTED] extra",+ "authorization: " + credential + "\nnext: line": "authorization: [REDACTED]\nnext: line",+ "-H 'Authorization: " + credential + "' -H 'A: b'": "-H 'Authorization: [REDACTED]' -H 'A: b'",+ } {+ got := RedactString(input)+ if got != want {+ t.Errorf("RedactString(%q) = %q, want %q", input, got, want)+ }+ if twice := RedactString(got); twice != got {+ t.Errorf("RedactString is not idempotent:\nonce %q\ntwice %q", got, twice)+ }+ }++ // A flag or a secret label after the first word is the next item on the line, not a credential.+ // Taken as one, the secret it introduces would stay: `--token x` lost `--token`, and the flag+ // rule never saw `x`. Each of these is redacted as it was before the rule existed.+ for input, want := range map[string]string{+ "AUTHORIZATION=abc --token hunter2hunter2": "AUTHORIZATION=[REDACTED] --token [REDACTED]",+ "AUTHORIZATION=abc --password hunter2hunter2": "AUTHORIZATION=[REDACTED] --password [REDACTED]",+ "AUTHORIZATION=abc -u admin:hunter2hunter2": "AUTHORIZATION=[REDACTED] -u admin:hunter2hunter2",+ "authorization: abc password: hunter2hunter2": "authorization: [REDACTED] password: [REDACTED]",+ `authorization: abc password="hunter2 hunter2"`: `authorization: [REDACTED] password="[REDACTED]"`,+ "AUTHORIZATION=abc PASSWORD=hunter2hunter2": "AUTHORIZATION=[REDACTED] PASSWORD=[REDACTED]",+ `authorization: abc "api_key": "hunter2hunter2"`: `authorization: [REDACTED] "api_key": "[REDACTED]"`,+ "Authorization: ApiKey --api-key hunter2hunter2": "Authorization: [REDACTED] --api-key [REDACTED]",+ "Authorization: ApiKey abc --api-key hunter2hunter2": "Authorization: [REDACTED] --api-key [REDACTED]",+ } {+ got := RedactString(input)+ if got != want {+ t.Errorf("RedactString(%q) = %q, want %q", input, got, want)+ }+ if strings.Contains(got, "hunter2hunter2") && !strings.Contains(input, "-u admin:") {+ t.Errorf("RedactString(%q) kept the secret after the authorization value: %q", input, got)+ }+ if twice := RedactString(got); twice != got {+ t.Errorf("RedactString is not idempotent:\nonce %q\ntwice %q", got, twice)+ }+ }++ // A base64url credential can start with `-`. It is not a flag, and goes with its scheme.+ for _, input := range []string{"Authorization: SSWS -Xq9abcdefghijklmnopqrstuv", "Authorization: ApiKey -_abcdefghijklmnop"} {+ if got := RedactString(input); got != "Authorization: [REDACTED]" {+ t.Errorf("RedactString(%q) = %q, want the scheme and the credential replaced", input, got)+ }+ }++ // A scheme that sends a list of auth-params has every parameter replaced with it; Digest, a+ // listed scheme, keeps its name. Nothing is left of the list behind a marker.+ for input, want := range map[string]string{+ `Authorization: Hawk id="dh37fgj492je", ts="1353832234", nonce="j4h3g2", mac="6R4rV5iE+NPoym+WwjeHzjAGXUtLNIxmo1vpMofpLAE="`: "Authorization: [REDACTED]",+ "Authorization: AWS4-HMAC-SHA256 Credential=AKIAIOSFODNN7EXAMPLE/20240101/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=fe5f80f77d5fa3beca038a248ff027d0445342fe2855ddc963176630326f1024": "Authorization: [REDACTED]",+ `Authorization: Digest username="alice", realm="api", nonce="dcd98b7102dd2f0e", response="6629fae49393a05397450978507c4ef1"`: "Authorization: Digest [REDACTED]",+ "Authorization: Custom user=alice,secret=s3cr3tvalue": "Authorization: [REDACTED]",+ } {+ got := RedactString(input)+ if got != want {+ t.Errorf("RedactString(%q) = %q, want %q", input, got, want)+ }+ if twice := RedactString(got); twice != got {+ t.Errorf("RedactString is not idempotent:\nonce %q\ntwice %q", got, twice)+ }+ }++ // A copy of the credential elsewhere in the text goes too, as an assigned value's copies do.+ for _, scheme := range []string{"ApiKey", "Bearer"} {+ input := "Authorization: " + scheme + " " + credential + " ; echo " + credential+ if got := RedactString(input); strings.Contains(got, credential) {+ t.Errorf("RedactString(%q) kept a copy of the credential: %q", input, got)+ }+ }+ // The word after a bare key in prose is no credential, and its copies stay.+ if got := RedactString("Authorization: required everywhere. Everywhere means everywhere"); !strings.Contains(got, "means everywhere") {+ t.Errorf("a prose word after Authorization: was removed as a copy: %q", got)+ }++ // The same header reaches the log through an event's command line.+ event := NewEvent(NewEventOptions{Action: "command.executed", Harness: HarnessInfo{Name: "claude_code"}})+ event.Command = &CommandInfo{Command: `curl -H "Authorization: ApiKey ` + credential + `" https://es.example:9200/`}+ if got := SanitizeEvent(event, 64*1024).Command.Command; strings.Contains(got, credential) {+ t.Errorf("SanitizeEvent kept the credential in command.command: %s", got)+ }+}++// A command held as a JSON-encoded string (a tool's arguments, a raw payload) quotes its arguments+// with `\"`. An unquoted value ends at the backslash that escapes the closing quote, so redaction+// leaves the escape in place and the text still decodes. Every unquoted value took the backslash,+// an Authorization credential, a labeled value or a URL parameter's alike, and left a bare quote+// that ended the string early.+func TestRedactStringKeepsJSONEncodedCommandsValid(t *testing.T) {+ const credential = "VnVhQ2ZHY0JDZGJrUW0tZTVhT3g6dWkybHAyYXhUTm1zeWFrdzl0dk5udw=="+ for command, want := range map[string]string{+ `curl -H "Authorization: ApiKey ` + credential + `" https://es.example:9200/`: `curl -H "Authorization: [REDACTED]" https://es.example:9200/`,+ `curl -H "Authorization: SSWS ` + credential + `" https://es.example:9200/`: `curl -H "Authorization: [REDACTED]" https://es.example:9200/`,+ `curl -H "Authorization: Bearer ` + credential + `" https://es.example:9200/`: `curl -H "Authorization: Bearer [REDACTED]" https://es.example:9200/`,+ `curl -H "Authorization: Basic ` + credential + `" https://es.example:9200/`: `curl -H "Authorization: Basic [REDACTED]" https://es.example:9200/`,+ `curl -H "Authorization: Token ` + credential + `" https://es.example:9200/`: `curl -H "Authorization: Token [REDACTED]" https://es.example:9200/`,+ `curl -H "Authorization: Digest ` + credential + `" https://es.example:9200/`: `curl -H "Authorization: Digest [REDACTED]" https://es.example:9200/`,+ `curl -H "X-Api-Key: ` + credential + `" https://es.example:9200/`: `curl -H "X-Api-Key: [REDACTED]" https://es.example:9200/`,+ `curl -d "token=` + credential + `" https://es.example:9200/`: `curl -d "token=[REDACTED]" https://es.example:9200/`,+ `curl "https://es.example:9200/?api_key=` + credential + `"`: `curl "https://es.example:9200/?api_key=[REDACTED]"`,+ } {+ encoded, err := json.Marshal(map[string]string{"command": command})+ if err != nil {+ t.Fatal(err)+ }+ got := RedactString(string(encoded))+ var decoded map[string]string+ if err := json.Unmarshal([]byte(got), &decoded); err != nil {+ t.Errorf("redaction broke the JSON: %v\n in: %s\n out: %s", err, encoded, got)+ continue+ }+ if decoded["command"] != want {+ t.Errorf("decoded command = %q, want %q", decoded["command"], want)+ }+ if twice := RedactString(got); twice != got {+ t.Errorf("RedactString is not idempotent:\nonce %q\ntwice %q", got, twice)+ }+ }+}+// A URL whose user name is a secret-named label, with `&` in its password and a query string after// it. `x-access-token:` is a label, not a query parameter, so `&` is part of its value, which runs// to the whitespace over the host and the query string, as any label's unquoted value does; the@@ -474,6 +652,20 @@ var redactIdempotenceInputs = []string{"config dump\napi_key=sk-abcdefghijklmnopqrstuvwxyz0123\nOPENAI sk-abcdefghijklmnopqrstuvwxyz0123",`{"token": "opaque-session-value-9", "password": "hunter2hunter2"}`,"Authorization: Bearer abc.def.ghi and --password hunter2hunter2",+ // A scheme the header rule does not list, with more after the credential on the line.+ "Authorization: ApiKey VnVhQ2ZHY0JDZGJrUW0tZTVhT3g6dWkybHAyYXhUTm1zeWFrdzl0dk5udw== and more after it",+ "Authorization: Hawk id=dh37fgj492je, ts=1353832234, mac=6R4rV5iENPoymWwjeHzjAGXUtLNIxmo1vpMofpLAE=",+ "Proxy-Authorization: Negotiate YIIabcdef Authorization: SSWS 00abcdefghij Authorization: a b c d",+ // A flag or a secret label after the first word, and a header in a JSON-encoded command.+ "AUTHORIZATION=abc --token hunter2hunter2 authorization: abc password: hunter2hunter2",+ `{"command":"curl -H \"Authorization: ApiKey VnVhQ2ZHY0JDZGJr\" -H \"Authorization: Bearer abcdef0123456789\" https://x/"}`,+ // A short scheme and credential leave a marker the capture of copies must not read as a scheme+ // on the next pass (found by FuzzRedactIsIdempotent).+ "AUTHORIZATION=A 0 00000000",+ // A credential copied elsewhere, a dash-led credential, and parameter lists with quoted values.+ "Authorization: ApiKey Zm9vYmFyYmF6cXV4MTIz ; echo Zm9vYmFyYmF6cXV4MTIz",+ "Authorization: SSWS -Xq9abcdefghijklmnop --token hunter2hunter2",+ `Authorization: Digest username="alice", response="6629fae4", Authorization: Hawk id="x", mac="y"`,"postgres://app:pa55w0rd@db/app","curl -u admin:hunter2hunter2 -H 'Cookie: sid=abcdef123456789' https://x","curl -b 'sessionid=abcdef123456789' --cookie sid=abcdef123456789 https://x",