The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu
Go agent-ai-cliagent-harnessesagent-securitybrowser-extensionciclaude-codecloudcodexcursordetection-engineeringjevknowledge-basememorymobile-device-managementobservabilitysecuritysecurity-information-and-event-managementsecurity-toolstelemetrytraces
fix(privacy): redact Postman, Docker Hub, Supabase and Netlify tokens (#1242)
Samira Siddiqua18385308b6b3
diff --git a/docs/changelog/cli.mdx b/docs/changelog/cli.mdxindex 24381aee..a3ef3a8b 100644--- a/docs/changelog/cli.mdx+++ b/docs/changelog/cli.mdx@@ -13,6 +13,8 @@ Stay up to date with the latest changes to the Agent Beacon CLI.Changes on `main` since v1.3.58. They ship in the next release.+- **Postman, Docker Hub, Supabase and Netlify tokens are redacted** · Written without a label beside them (a positional argument, a pasted line, command output), Postman API keys (`PMAK-…`), Docker Hub personal and organization access tokens (`dckr_pat_…`, `dckr_oat_…`), Supabase personal access tokens (`sbp_…`, `sbp_v0_…`, `sbp_oauth_…`) and secret API keys (`sb_secret_…`, the replacement for the `service_role` key), and Netlify personal access tokens (`nfp_…`) were kept as they were. Each is now redacted after its prefix, as in `dckr_pat_[REDACTED]`. Supabase publishable keys (`sb_publishable_…`) are meant to be public and stay readable.+- **`install --dry-run` no longer plans scheduled jobs the host cannot run** · On a host with no scheduler for Beacon's jobs (Windows, or a supervised collector in a container or under WSL), `beacon endpoint install --dry-run` and `repair --dry-run` listed a launchd plist under `~/Library/LaunchAgents` for the inventory heartbeat job, and for the session sync job with `--session-sync`, plus a step that started each. The install does none of that there: it skips both jobs and says why. The plan now shows a `skip_unit` line with that same reason instead. On macOS and on Linux with systemd the plan is unchanged.- **An AWS secret key after a labeled key id is redacted** · Where a label's or a flag's value ended in an AWS access key id and its secret access key followed after a comma, a blank or a quote (`token=AKIA…,<secret>`, `--access-key AKIA… <secret>`, `x-access-token:abc@AKIA…,<secret>`), the value rule replaced the key id and the rule for a key id and its secret then found no key id, so the secret was kept. The secret is now replaced with the value, keeping the separators: `token=[REDACTED],[REDACTED]`. A URL password that runs on over a key id written right after a host name (`…@hostAKIA…,<secret>`) takes the secret too.diff --git a/docs/security/retention-redaction.mdx b/docs/security/retention-redaction.mdxindex da2e66f7..3cb192a9 100644--- a/docs/security/retention-redaction.mdx+++ b/docs/security/retention-redaction.mdx@@ -27,7 +27,7 @@ Redaction runs on retained text (prompts, command output, diffs, tool arguments| Authorization headers and flags | `Authorization: Bearer …`, `Authorization: Basic …`, `--password …`, `--token=…`, and snake_case or key flags such as `--client_secret …`, `--api_key …`, `--private-key …`, `--secret-key …` || URL credentials | the password in `postgres://user:…@host`, `https://x-access-token:…@github.com` || Private keys | PEM and OpenSSH `PRIVATE KEY` blocks, including a block cut off before its end line |-| Recognizable tokens (the public type prefix stays, as in `ghp_[REDACTED]`) | Anthropic and OpenAI `sk-…`, AWS `AKIA…`/`ASIA…`, GitHub `ghp_…`/`github_pat_…`, GitLab `glpat-…`, Slack `xox…-` and `xapp-…`, Stripe `sk_live_…` and webhook signing secrets `whsec_…`, Google `AIza…`, npm, Hugging Face, HashiCorp Vault `hvs.…`, Groq `gsk_…`, Perplexity `pplx-…`, Replicate `r8_…`, xAI `xai-…`, LangSmith `lsv2_…`, Pinecone `pcsk_…`, DigitalOcean `dop_v1_…`, Databricks `dapi…`, PyPI `pypi-…`, Shopify `shpat_…`, Linear `lin_api_…`, Doppler `dp.pt.…`, SendGrid `SG.…`, Beacon keys and tokens (`bcn_api_…`, `bcn_device_…`, `bcn_enfdev_…`, `bcn_mcp_…`, `bcn_mcprt_…`, and the `beacon login` token `bcn_cli_…` with its `bcn_code_…`, `bcn_client_…` and `bcn_exchange_…` values), and JWTs |+| Recognizable tokens (the public type prefix stays, as in `ghp_[REDACTED]`) | Anthropic and OpenAI `sk-…`, AWS `AKIA…`/`ASIA…`, GitHub `ghp_…`/`github_pat_…`, GitLab `glpat-…`, Slack `xox…-` and `xapp-…`, Stripe `sk_live_…` and webhook signing secrets `whsec_…`, Google `AIza…`, npm, Hugging Face, HashiCorp Vault `hvs.…`, Groq `gsk_…`, Perplexity `pplx-…`, Replicate `r8_…`, xAI `xai-…`, LangSmith `lsv2_…`, Pinecone `pcsk_…`, DigitalOcean `dop_v1_…`, Databricks `dapi…`, PyPI `pypi-…`, Shopify `shpat_…`, Linear `lin_api_…`, Doppler `dp.pt.…`, SendGrid `SG.…`, Postman `PMAK-…`, Docker Hub `dckr_pat_…`/`dckr_oat_…`, Supabase `sbp_…` and secret keys `sb_secret_…`, Netlify `nfp_…`, Beacon keys and tokens (`bcn_api_…`, `bcn_device_…`, `bcn_enfdev_…`, `bcn_mcp_…`, `bcn_mcprt_…`, and the `beacon login` token `bcn_cli_…` with its `bcn_code_…`, `bcn_client_…` and `bcn_exchange_…` values), and JWTs || An AWS secret key beside its key id | the 40-character secret that follows an `AKIA…` or `ASIA…` key id within a few spaces, commas, colons, pipes or quotes, as in a credentials CSV (`AKIA…,wJalr…`) or a table |Redaction is pattern-based. A credential with no label and no recognizable shape, such as a bare random password, is not detected, so treat retained content as sensitive. Metadata-only forwarding is the control for keeping content off Beacon Cloud entirely, and [`beacon share`](/cli/share#metadata-only-endpoints) follows it: a Metadata-only endpoint shares no content either.diff --git a/pkg/asymptoteobserve/privacy.go b/pkg/asymptoteobserve/privacy.goindex 6f1e1a46..c4f9eeda 100644--- a/pkg/asymptoteobserve/privacy.go+++ b/pkg/asymptoteobserve/privacy.go@@ -494,20 +494,27 @@ var redactionRules = concatRules([]redactionRule{shapeRule(`\b(hf_)[A-Za-z0-9]{30,}`, "${1}"+redactedValue, "hf_"), // Hugging Face// Model providers and developer platforms whose keys agents handle all day. Each length and// alphabet is gitleaks' rule of the same name, or trufflehog's detector where gitleaks has none.- shapeRule(`\b(hv[sbr]\.)[A-Za-z0-9_-]{24,}`, "${1}"+redactedValue, "hvs.", "hvb.", "hvr."), // HashiCorp Vault- shapeRule(`\b(gsk_)[A-Za-z0-9]{52}\b`, "${1}"+redactedValue, "gsk_"), // Groq- shapeRule(`\b(pplx-)[A-Za-z0-9]{48}\b`, "${1}"+redactedValue, "pplx-"), // Perplexity- shapeRule(`\b(r8_)[A-Za-z0-9_-]{37}`, "${1}"+redactedValue, "r8_"), // Replicate- shapeRule(`\b(xai-)[A-Za-z0-9_]{80}\b`, "${1}"+redactedValue, "xai-"), // xAI- shapeRule(`\b(lsv2_(?:pt|sk)_)[0-9a-f]{32}_[0-9a-f]{10}\b`, "${1}"+redactedValue, "lsv2_pt_", "lsv2_sk_"), // LangSmith- shapeRule(`\b(pcsk_)[A-Za-z0-9]{5,6}_[A-Za-z0-9]{63}\b`, "${1}"+redactedValue, "pcsk_"), // Pinecone- shapeRule(`\b(do[por]_v1_)[0-9a-f]{64}\b`, "${1}"+redactedValue, "dop_v1_", "doo_v1_", "dor_v1_"), // DigitalOcean- shapeRule(`\b(dapi)[0-9a-f]{32}(?:-\d)?\b`, "${1}"+redactedValue, "dapi"), // Databricks- shapeRule(`\b(pypi-)AgEIcHlwaS5vcmc[A-Za-z0-9_-]{50,}`, "${1}"+redactedValue, "pypi-AgEIcHlwaS5vcmc"), // PyPI upload token- shapeRule(`\b(shp(?:at|ca|pa|ss)_)[a-fA-F0-9]{32}\b`, "${1}"+redactedValue, "shpat_", "shpca_", "shppa_", "shpss_"), // Shopify- shapeRule(`\b(lin_api_)[A-Za-z0-9]{40}\b`, "${1}"+redactedValue, "lin_api_"), // Linear- shapeRule(`\b(dp\.pt\.)[A-Za-z0-9]{43}\b`, "${1}"+redactedValue, "dp.pt."), // Doppler- shapeRule(`\b(SG\.)[A-Za-z0-9_-]{22}\.[A-Za-z0-9_-]{43}`, "${1}"+redactedValue, "SG."), // SendGrid+ // Where prefixes differ in a letter or two, the rules share one needle (shp, lsv2_, _v1_): the+ // engine tracks at most 64 distinct needles, and each new one spends a slot.+ shapeRule(`\b(hv[sbr]\.)[A-Za-z0-9_-]{24,}`, "${1}"+redactedValue, "hvs.", "hvb.", "hvr."), // HashiCorp Vault+ shapeRule(`\b(gsk_)[A-Za-z0-9]{52}\b`, "${1}"+redactedValue, "gsk_"), // Groq+ shapeRule(`\b(pplx-)[A-Za-z0-9]{48}\b`, "${1}"+redactedValue, "pplx-"), // Perplexity+ shapeRule(`\b(r8_)[A-Za-z0-9_-]{37}`, "${1}"+redactedValue, "r8_"), // Replicate+ shapeRule(`\b(xai-)[A-Za-z0-9_]{80}\b`, "${1}"+redactedValue, "xai-"), // xAI+ shapeRule(`\b(lsv2_(?:pt|sk)_)[0-9a-f]{32}_[0-9a-f]{10}\b`, "${1}"+redactedValue, "lsv2_"), // LangSmith+ shapeRule(`\b(pcsk_)[A-Za-z0-9]{5,6}_[A-Za-z0-9]{63}\b`, "${1}"+redactedValue, "pcsk_"), // Pinecone+ shapeRule(`\b(do[por]_v1_)[0-9a-f]{64}\b`, "${1}"+redactedValue, "_v1_").before(3), // DigitalOcean+ shapeRule(`\b(dapi)[0-9a-f]{32}(?:-\d)?\b`, "${1}"+redactedValue, "dapi"), // Databricks+ shapeRule(`\b(pypi-)AgEIcHlwaS5vcmc[A-Za-z0-9_-]{50,}`, "${1}"+redactedValue, "pypi-AgEIcHlwaS5vcmc"), // PyPI upload token+ shapeRule(`\b(shp(?:at|ca|pa|ss)_)[a-fA-F0-9]{32}\b`, "${1}"+redactedValue, "shp"), // Shopify+ shapeRule(`\b(lin_api_)[A-Za-z0-9]{40}\b`, "${1}"+redactedValue, "lin_api_"), // Linear+ shapeRule(`\b(dp\.pt\.)[A-Za-z0-9]{43}\b`, "${1}"+redactedValue, "dp.pt."), // Doppler+ shapeRule(`\b(SG\.)[A-Za-z0-9_-]{22}\.[A-Za-z0-9_-]{43}`, "${1}"+redactedValue, "SG."), // SendGrid+ shapeRule(`\b(PMAK-)[a-fA-F0-9]{24}-[a-fA-F0-9]{34}\b`, "${1}"+redactedValue, "PMAK-"), // Postman+ shapeRule(`\b(dckr_(?:pat|oat)_)[A-Za-z0-9_-]{27,}`, "${1}"+redactedValue, "dckr_"), // Docker Hub personal and organization access tokens+ shapeRule(`\b(sbp_(?:oauth_|v0_)?)[a-f0-9]{40}\b`, "${1}"+redactedValue, "sbp_"), // Supabase personal access token+ shapeRule(`\b(sb_secret_)[A-Za-z0-9_-]{22}_[A-Za-z0-9_-]{8,}`, "${1}"+redactedValue, "sb_secret_"), // Supabase secret API key, as Supabase documents it; not the publishable key, which is public+ shapeRule(`\b(nfp_)[A-Za-z0-9_]{36}\b`, "${1}"+redactedValue, "nfp_"), // Netlify personal access token// Beacon's own keys and tokens: API keys, device and enforcement keys, MCP tokens and their refresh// tokens, and the CLI sign-in's access token, authorization code, client and exchange values.shapeRule(`\b(bcn_(?:api|device|mcprt|mcp|enfdev|cli|code|client|exchange)_)[A-Za-z0-9_-]{8,}`, "${1}"+redactedValue, "bcn_"),diff --git a/pkg/asymptoteobserve/privacy_test.go b/pkg/asymptoteobserve/privacy_test.goindex f41ddfe3..831111bb 100644--- a/pkg/asymptoteobserve/privacy_test.go+++ b/pkg/asymptoteobserve/privacy_test.go@@ -225,6 +225,14 @@ func TestRedactStringRedactsProviderTokenShapes(t *testing.T) {{"linear", "lin_api_" + alnum(40), "lin_api_[REDACTED]"},{"doppler", "created service token " + "dp.pt." + alnum(43) + " for ci", "created service token dp.pt.[REDACTED] for ci"},{"sendgrid", "SG." + alnum(22) + "." + alnum(43), "SG.[REDACTED]"},+ {"postman api key", "found PMAK-" + hex(24) + "-" + hex(34), "found PMAK-[REDACTED]"},+ {"docker hub personal access token", "found dckr_pat_" + alnum(27), "found dckr_pat_[REDACTED]"},+ {"docker hub organization access token", "docker login -p dckr_oat_" + alnum(32), "docker login -p dckr_oat_[REDACTED]"},+ {"supabase personal access token", "found sbp_" + hex(40), "found sbp_[REDACTED]"},+ {"supabase versioned personal access token", "sbp_v0_" + hex(40), "sbp_v0_[REDACTED]"},+ {"supabase oauth token", "sbp_oauth_" + hex(40), "sbp_oauth_[REDACTED]"},+ {"supabase secret key", `createClient(url, "sb_secret_` + alnum(10) + "-" + alnum(11) + "_" + alnum(8) + `")`, `createClient(url, "sb_secret_[REDACTED]")`},+ {"netlify personal access token", "netlify deploy --auth nfp_" + alnum(36), "netlify deploy --auth nfp_[REDACTED]"},}for _, tc := range tests {t.Run(tc.name, func(t *testing.T) {@@ -246,12 +254,17 @@ func TestRedactStringLeavesProviderTokenLookalikes(t *testing.T) {"lsv2_pt_notahexvalue_atall","pcsk_abc","dop_v1_0123abcd",+ "api_v1_" + strings.Repeat("0", 64),"dapi_reference and dapiVersion","pip install pypi-simple","shpat_ZZZZ","lin_api_key","dp.pt.short","SG.aaaa.bbbb",+ "dckr_pat_short",+ "sbp_" + strings.Repeat("z", 40),+ "sb_publishable_" + strings.Repeat("A", 22) + "_" + strings.Repeat("B", 8),+ "nfp_short","hvs.short","rapidapi" + strings.Repeat("0", 32),} {