Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.
redact: a text masked for a vendor before is masked as it was then, so a cached prompt stays cached
yetone50f52b540b91
diff --git a/docs/subsystems/gateway-routing.md b/docs/subsystems/gateway-routing.mdindex a137c3bb7..e8c5ce433 100644--- a/docs/subsystems/gateway-routing.md+++ b/docs/subsystems/gateway-routing.md@@ -75,7 +75,7 @@ recorded as a prediction failure (`noPrediction`).## Runtime path1. `handle(proto)` reads the body and the model (`requestBody`, `requestModel`), then calls `serveAgent`. Every route that reads an agent's body (each API, count_tokens, Gemini's, embeddings, images, videos, System One, the Codex backend path) reads it through `readBoundedRequestBody` in [`request_bounds.go`](../../internal/gateway/request_bounds.go). It decodes a `Content-Encoding` of `gzip` or `zstd` (`decodedBody`; Codex sends zstd to a custom provider's `/v1/responses` too, #1223) and drops the header, so the provider is sent plain JSON, with one exception: a ChatGPT account's turn (POST `CodexBase`/responses, from the accounts loop in `forwardOnce` or relayed by `codexUpstreamOn`) is compressed again with zstd and a window of at most 2 MiB, as Codex's own client sends it (`provider.CodexCompressed`; openai/codex `responses_request_compression` and `http-client/src/request.rs`). It is compressed once the body is final, and again on each try. `/responses/compact`, an API key's requests and every other provider stay plain. Sent plain, a 1.1 MB text-only turn was answered 503 (ylorn on X). The plain body is what magpie keeps and shows, and `codexDropped` judges a reset by the bytes that went on the wire. `codexDropped` also reads Envoy's 507 "exceeded request buffer limit while retrying upstream" (the backend failed and the front couldn't retry a body that large) as that reset, at any size, told as a 503 so magpie, Codex and pi ask again and nobody rests (#2034). The size limit (`defaultBodyLimit`, 512 MB) holds for the decoded bytes (413), and any other encoding is refused 415. It was 128 MB until a long Codex session that had read many images went past it decoded, its inline screenshots resent each turn, and got the 413 on all five of Codex's retries (#1422, `TestImageHeavyCodexTurnIsRead`). magpie's own 413 says it is magpie's gateway limit, with the size, and that the provider was not asked (`overLimit`); `writeError` adds to any 413 it isn't holding that this is a byte limit, not the model's context, and to `/compact` or start a new conversation (`TestOverLimitSaysWhoseAndWhatToDo`). Every shape but Gemini's types a 413 `request_too_large`, Anthropic's own type, which pi's overflow table knows: pi compacts and tries once more rather than stopping on an OpenAI-shaped `api_error` (@wilsonxuuuu on X, `TestPayloadTooLargeSaysRequestTooLargeToPi`). `keyLimited` reads a limited key's compressed body the same way, so its reservation is the decoded size. `serveAgent` runs the [middleware](gateway-middleware.md) chain, then `serve`.-2. `serve` redacts secrets in the request (`redacted`; `redact.MaskJSON` leaves ids alone, and in any text a backend's id (a UUID, or 32 hex digits or more) goes as written: no rule, model label or value known from before masks a piece of one (a phone number at the end of `…-a13800138000`, a card number across a UUID's groups), and a whole one is masked only by the user's own words and rules or as a credential, by the password, API key and token rules or a secret rule whose field is named for one (`api_key=`, `refresh_token`), never one named for an id, a cursor, a request or a digest (`secret_id`, `next_page_token`, `Idempotency-Key`, `secret_hash`), a bare `key`, a serial number, or what the model takes a UUID for (`noIDs`, [`internal/redact/ids.go`](../../internal/redact/ids.go)); and a tool's name and its MCP server's (`namespace`, `server_label`, `server_name`, #3143); the request's tool definitions and output schemas, found by their place in each protocol (top-level `tools`, `tool_choice`, `response_format`, `text`, `generationConfig`, `stream_options`, the client's ids in `client_metadata` and the like, and the tools of a Responses `additional_tools` or `tool_search_output` item), go as written save their descriptions, so a word the user masks can't break a grammar or an enum (#1911); and what the vendor sealed with a signature, a thinking block, Gemini's thoughts or Responses' encrypted reasoning, goes back as the vendor wrote it: only the values masked elsewhere in the request, and secrets masked before, are put back as their placeholders; a value masked once, at least 8 characters long, is masked again wherever a request has it, as an exact match whatever words are around it, so what the vendor wrote about a placeholder doesn't go back with the value in it, `redact/known.go`), resolves the model to a provider or a group, and refuses what can't be served. The reply has the values put back on its way to the agent (`redact.Writer`, a stream event by event, a whole body at its end): a placeholder the model wrote as it was given, or as models rewrite one (spaces inside the braces, a brace or both dropped, another case), is the value again when what is left names one magpie made, and one split across events is held until it is whole (`redact.Restore`, `partialTail`, [`internal/redact/redact.go`](../../internal/redact/redact.go)). A request whose provider and fallbacks, or whose group's models, all skip redaction (`unredactedRoute`, `provider.SkipsRedaction`: set *Send requests unmasked*, `Unredacted`, with every address on this machine or the local network) goes to them as the agent wrote it; the log and recent calls keep the masked body. count_tokens and embeddings follow the same rule. What a routing group's Jev is told of the turn (the message, the user's message before it and the reply to it) is masked all the same unless Jev's own provider skips redaction too (`maskedState` in `askJev`, [`decide.go`](../../internal/gateway/decide.go)): a remote Jev is none of the group's members. The masking is the built-in plugin `redact`'s (`gwmw.Masker`, [Plugin runtime](plugin-runtime.md)); when the settings ask for masking and no masker serves, or it fails, the request is a 503 and nothing is sent (`redacted`, `gwmw.Masking`).+2. `serve` redacts secrets in the request (`redacted`; `redact.MaskJSON` leaves ids alone, and in any text a backend's id (a UUID, or 32 hex digits or more) goes as written: no rule, model label or value known from before masks a piece of one (a phone number at the end of `…-a13800138000`, a card number across a UUID's groups), and a whole one is masked only by the user's own words and rules or as a credential, by the password, API key and token rules or a secret rule whose field is named for one (`api_key=`, `refresh_token`), never one named for an id, a cursor, a request or a digest (`secret_id`, `next_page_token`, `Idempotency-Key`, `secret_hash`), a bare `key`, a serial number, or what the model takes a UUID for (`noIDs`, [`internal/redact/ids.go`](../../internal/redact/ids.go)); and a tool's name and its MCP server's (`namespace`, `server_label`, `server_name`, #3143); the request's tool definitions and output schemas, found by their place in each protocol (top-level `tools`, `tool_choice`, `response_format`, `text`, `generationConfig`, `stream_options`, the client's ids in `client_metadata` and the like, and the tools of a Responses `additional_tools` or `tool_search_output` item), go as written save their descriptions, so a word the user masks can't break a grammar or an enum (#1911); and what the vendor sealed with a signature, a thinking block, Gemini's thoughts or Responses' encrypted reasoning, goes back as the vendor wrote it: only the values masked elsewhere in the request, and secrets masked before, are put back as their placeholders; a value masked once, at least 8 characters long, is masked again wherever a request has it, as an exact match whatever words are around it, so what the vendor wrote about a placeholder doesn't go back with the value in it, `redact/known.go`; a text masked for a vendor before, such as a system prompt or a turn already sent, is masked as it was then, so a value a rule first finds later in a conversation doesn't change the prompt the vendor has cached and rewrite all of it to the cache, `sentKnownSpans`), resolves the model to a provider or a group, and refuses what can't be served. The reply has the values put back on its way to the agent (`redact.Writer`, a stream event by event, a whole body at its end): a placeholder the model wrote as it was given, or as models rewrite one (spaces inside the braces, a brace or both dropped, another case), is the value again when what is left names one magpie made, and one split across events is held until it is whole (`redact.Restore`, `partialTail`, [`internal/redact/redact.go`](../../internal/redact/redact.go)). A request whose provider and fallbacks, or whose group's models, all skip redaction (`unredactedRoute`, `provider.SkipsRedaction`: set *Send requests unmasked*, `Unredacted`, with every address on this machine or the local network) goes to them as the agent wrote it; the log and recent calls keep the masked body. count_tokens and embeddings follow the same rule. What a routing group's Jev is told of the turn (the message, the user's message before it and the reply to it) is masked all the same unless Jev's own provider skips redaction too (`maskedState` in `askJev`, [`decide.go`](../../internal/gateway/decide.go)): a remote Jev is none of the group's members. The masking is the built-in plugin `redact`'s (`gwmw.Masker`, [Plugin runtime](plugin-runtime.md)); when the settings ask for masking and no masker serves, or it fails, the request is a 503 and nothing is sent (`redacted`, `gwmw.Masking`).- With *Mask personal data* on, each kind in `redact.Categories` ([`internal/redact/more.go`](../../internal/redact/more.go)) is masked unless the user turned it off (`redactKinds` in settings, `Options.Kinds`). On by default: US SSNs, passport numbers, IBANs that pass mod 97, birthdays next to a word that says so, MAC addresses, and serial numbers next to S/N or the like. Off by default: the user name in a home folder, user@host in a prompt or ssh command, public IPv4 and IPv6 addresses (never private, loopback, link-local, CGNAT or documentation ranges) and cloud bucket names, since agents send them in nearly every request and the model must copy the placeholder exactly inside a path or URL. Context rules run only near their words (`rule.matches`). What reads as an address but ends in a file's extension that no domain has (`…@2x.jpg`, `[email protected]`; `fileExt` in `realEmail`), or a scale before an extension that is a domain too (`[email protected]`, `[email protected]`; `scaleExt`), is a file's name, for the rule and the local model alike (#3029). `[email protected]` is still an address. The Settings rows come from `REDACT_KINDS` in app.js, which `TestRedactKindsAreRedacts` holds to the same ids and defaults.- With *Find with a local model* on (`settings.RedactModel`, this computer's own, kept by `KeepOwn`), a local model finds what no rule can, a name or an address in free text, under the same switches: a name, address, phone, email or account number is personal data, a secret a secret (`modelKinds`, `modelSpans` in [`internal/redact/model.go`](../../internal/redact/model.go)). `MaskJSON` asks it once per request, for every text it would mask, each once (`withModel`); its spans join the rules' with a rule's kind kept where both find a value, and ids a vendor or tool made (`req_…`, `toolu_…`) are never taken, nor a snake_case word as a name (Codex's `sequential_cutoff`, #3144), nor as a secret what names a thing: a UUID, a digest with its algorithm's name, a file's path (`namesThing`; the model takes all three for secrets). The model misses many Chinese names (张伟 after 给, 陈建国 after 客户, however low its threshold) and runs a name it finds on into the words after it (「王芳,她下周一」), so with it on a Chinese name is also found by its surname and a word around it that says a person is meant (客户, 我叫, 给…发, …先生, …说, joined by 和 to another name), and the model's are cut at punctuation and titles ([`zhname.go`](../../internal/redact/zhname.go)); a surname with nothing around it (王国的, 张开的) is left. The model is OpenAI's Privacy Filter as a q8 GGUF (1.6 GB), run by `pf-serve` ([`native/privacy-filter`](../../native/privacy-filter), built from privacy-filter.cpp on ggml and released under `pf-v*` tags): Settings → Privacy's On downloads both (`pii.Download`, each checked against its pinned sha256, from hf-mirror first in China, `source.chinaFirst`) and saves the setting only once they are on disk (`/api/privacy-model`, [`privacymodel.go`](../../internal/gui/privacymodel.go)). `pii.Find` starts `pf-serve` on the first request (Metal, Vulkan, else the CPU; one that dies on the GPU is started on the CPU for the rest of the run), joins short texts into 12 KB chunks and maps the spans back, keeps the last 20000 answers by hash, and ends it after 10 minutes with no request ([`internal/pii`](../../internal/pii)). A model that is on and can't answer (not on disk, didn't start, hung) panics, so the request is held back with the 503 above rather than sent with the rules' masking alone (`masking.model`).- With *Mask secrets* on, the user's own masking rules (`redactRules`, `redact.Rule`: a placeholder name and a prefix or an RE2 pattern, [`internal/redact/rules.go`](../../internal/redact/rules.go)) run before magpie's. Settings → Privacy adds them, removes them, and edits one in place (`POST /api/settings/redact-rule` with the rule's index and its current value, `editRedactRule` in [`internal/gui/api.go`](../../internal/gui/api.go)). Every rule is checked by `redact.CheckRules` when settings are saved; an edit that doesn't compile, matches the empty string, repeats another rule or names a rule no longer at that index is refused with the rules left as they were. The masker reads the rules from settings on each call (`options` in [`internal/masking`](../../internal/masking/masking.go)), so a change applies to the next request.diff --git a/internal/gateway/masked_cache_prefix_test.go b/internal/gateway/masked_cache_prefix_test.gonew file mode 100644index 000000000..53dfba3c3--- /dev/null+++ b/internal/gateway/masked_cache_prefix_test.go@@ -0,0 +1,134 @@+package gateway++import (+ "encoding/json"+ "fmt"+ "io"+ "net/http"+ "net/http/httptest"+ "strings"+ "sync"+ "testing"++ "github.com/yetone/magpie/internal/provider"+ "github.com/yetone/magpie/internal/settings"+)++// With Privacy masking on, a conversation's earlier turns reach the+// vendor as the same bytes on every turn, so its prompt cache is read: a+// value a rule finds in one turn's tool output is masked from then on in+// new text, and the system prompt and the turns already sent with it as+// written go as they went. Masking it there too rewrote the whole prompt+// to the cache on the next turn (audie on Discord: DeepSeek Harness on a+// group with a custom Anthropic provider, 412K to 427K tokens written and+// none read on adjacent turns, ten seconds apart). Here as dsh sends it,+// Chat Completions to a group, and as Claude Code sends it, Messages.+func TestMaskedConversationKeepsItsCachedPrefix(t *testing.T) {+ fresh(t)+ var mu sync.Mutex+ var bodies []map[string]json.RawMessage+ up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {+ var q map[string]json.RawMessage+ b, _ := io.ReadAll(r.Body)+ json.Unmarshal(b, &q)+ mu.Lock()+ bodies = append(bodies, q)+ mu.Unlock()+ if string(q["stream"]) == "true" {+ w.Header().Set("Content-Type", "text/event-stream")+ io.WriteString(w, "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"id\":\"m\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[],\"usage\":{\"input_tokens\":1,\"output_tokens\":0}}}\n\n"++ "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n"++ "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"ok\"}}\n\n"++ "event: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0}\n\n"++ "event: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"end_turn\"},\"usage\":{\"output_tokens\":1}}\n\n"++ "event: message_stop\ndata: {\"type\":\"message_stop\"}\n\n")+ return+ }+ w.Header().Set("Content-Type", "application/json")+ io.WriteString(w, `{"id":"m","type":"message","role":"assistant","content":[{"type":"text","text":"ok"}],"stop_reason":"end_turn","usage":{"input_tokens":2,"output_tokens":1}}`)+ }))+ t.Cleanup(up.Close)+ if err := provider.Save(provider.Provider{ID: "relay", Name: "Relay", Key: "k", Models: []string{"claude-opus-5-5"}, Anthropic: up.URL + "/v1"}); err != nil {+ t.Fatal(err)+ }+ if err := provider.SaveGroup(provider.Group{ID: "flash", Name: "flash", Members: []string{"relay/claude-opus-5-5"}, Routing: provider.Ordered}); err != nil {+ t.Fatal(err)+ }+ if err := settings.Save(settings.Settings{Redact: true, RedactPersonal: true}); err != nil {+ t.Fatal(err)+ }+ s := New()+ for _, c := range []struct {+ name, path, secret string+ body func(turns []string) string+ turn func(i int, out string) []string+ }{+ {"chat", "/v1/chat/completions", "Hq4cacheDsh81z",+ func(turns []string) string {+ return `{"model":"group/flash","messages":[{"role":"system","content":"You are dsh. Staging deploys with Hq4cacheDsh81z for now."},` ++ `{"role":"user","content":"read the env file"}` + strings.Join(turns, "") + `],` ++ `"tools":[{"type":"function","function":{"name":"bash","description":"run","parameters":{"type":"object","properties":{"cmd":{"type":"string"}}}}}]}`+ },+ func(i int, out string) []string {+ id := fmt.Sprintf("call_%d", i)+ return []string{`,{"role":"assistant","content":"","tool_calls":[{"id":"` + id + `","type":"function","function":{"name":"bash","arguments":"{\"cmd\":\"cat .env\"}"}}]}`,+ `,{"role":"tool","tool_call_id":"` + id + `","content":"` + out + `"}`}+ }},+ {"messages", "/v1/messages", "Wm6cacheCc29qy",+ func(turns []string) string {+ return `{"model":"relay/claude-opus-5-5","max_tokens":1024,"system":"You are Claude. Staging deploys with Wm6cacheCc29qy for now.",` ++ `"tools":[{"name":"bash","description":"run","input_schema":{"type":"object","properties":{"cmd":{"type":"string"}}}}],` ++ `"messages":[{"role":"user","content":"read the env file"}` + strings.Join(turns, "") + `]}`+ },+ func(i int, out string) []string {+ id := fmt.Sprintf("toolu_%d", i)+ return []string{`,{"role":"assistant","content":[{"type":"tool_use","id":"` + id + `","name":"bash","input":{"cmd":"cat .env"}}]}`,+ `,{"role":"user","content":[{"type":"tool_result","tool_use_id":"` + id + `","content":"` + out + `"}]}`}+ }},+ } {+ mu.Lock()+ bodies = nil+ mu.Unlock()+ var turns []string+ for i := range 3 {+ out := "PORT=8080"+ if i == 1 {+ out = "DB_PASSWORD=" + c.secret // a rule finds it here first+ }+ turns = append(turns, c.turn(i, out)...)+ if code, raw := postJSON(t, s, c.path, c.body(turns)); code != 200 {+ t.Fatalf("%s turn %d: %d %s", c.name, i, code, raw)+ }+ }+ mu.Lock()+ got := bodies+ mu.Unlock()+ if len(got) != 3 {+ t.Fatalf("%s: %d requests upstream", c.name, len(got))+ }+ for i := 1; i < len(got); i++ {+ before, now := got[i-1], got[i]+ for _, k := range []string{"system", "tools"} {+ if string(before[k]) != string(now[k]) {+ t.Errorf("%s turn %d: %s changed:\n%s\nwas\n%s", c.name, i, k, now[k], before[k])+ }+ }+ var a, b []json.RawMessage+ json.Unmarshal(before["messages"], &a)+ json.Unmarshal(now["messages"], &b)+ // the turns before, up to the one the cache mark was on+ for j := 0; j < len(a)-1; j++ {+ if string(a[j]) != string(b[j]) {+ t.Errorf("%s turn %d: message %d changed:\n%s\nwas\n%s", c.name, i, j, b[j], a[j])+ }+ }+ }+ last := got[len(got)-1]+ if !strings.Contains(string(last["system"]), c.secret) {+ t.Errorf("%s: the system prompt, sent with the value as written, has it masked now: %s", c.name, last["system"])+ }+ if strings.Contains(string(last["messages"]), c.secret) {+ t.Errorf("%s: the value a rule found went out: %s", c.name, last["messages"])+ }+ }+}diff --git a/internal/redact/json.go b/internal/redact/json.goindex b7ca635d5..dc044230f 100644--- a/internal/redact/json.go+++ b/internal/redact/json.go@@ -369,6 +369,7 @@ func jsonEscape(s string) string {// placeholder: either way the text no longer matched its signature, and// Anthropic answers 400 "Invalid `signature` in `thinking` block".func MaskJSON(body []byte, o Options) ([]byte, int) {+ o.sent = truesigned, args, defs := scan(body)masked := func(path, key, s string) bool {return !keepIn(path, key, s, args, defs) && (signed == nil || !under(path, signed))@@ -427,7 +428,7 @@ func MaskJSON(body []byte, o Options) ([]byte, int) {// number or an address the vendor wrote of its own can be one// masked in another conversation (13800138000 is in many), and// masked, its text would no longer match its signature- t, n := apply(t, knownSpans(t, Options{Secrets: o.Secrets}), placeholder, o)+ t, n := apply(t, sentKnownSpans(t, Options{Secrets: o.Secrets}), placeholder, o)total += nreturn t})diff --git a/internal/redact/known.go b/internal/redact/known.goindex 8e84d88e3..1d6b3ddbc 100644--- a/internal/redact/known.go+++ b/internal/redact/known.go@@ -1,8 +1,11 @@package redactimport (+ "crypto/sha256"+ "fmt""slices""strings"+ "sync""sync/atomic")@@ -198,3 +201,80 @@ func standsAlone(kind, s string, a, b int) bool {}return !(is(before, alnum) && is(s[a], alnum)) && !(is(after, alnum) && is(s[b-1], alnum))}++// A text sent before goes again as it went: the known values found in it+// the first time it was masked are the ones masked in it from then on,+// and a value that became known since is masked in new text only.+//+// An agent sends its whole conversation on every turn, and a vendor+// caches it by its bytes up to a mark (Anthropic's cache_control, the+// prefix OpenAI and DeepSeek cache on their own). A value found by a rule+// in a turn's tool output became known, and the next turn masked it again+// in the system prompt and every earlier message that had it as written:+// the prefix changed at its first block, and a 400K-token conversation was+// written to the cache whole instead of read (audie on Discord: cache_read+// 0 and cache_write 412,081 to 426,782 on four turns, ten seconds apart).+// Masking it there keeps nothing from the vendor, which was sent that text+// as written already.+//+// What is remembered is a hash of the text and of what o covers, and the+// places of the known values in it; nothing of a value. A text no longer+// remembered (sentEntries, or magpie started again) is masked with what is+// known then.+const sentEntries = 1 << 16++var sentKnown struct {+ sync.Mutex+ cur, old map[[32]byte][]span+}++// sentKnownSpans is knownSpans for text going to a vendor: the same as the+// first time s was masked under what o covers.+func sentKnownSpans(s string, o Options) []span {+ if !o.Secrets && !o.Personal || len(s) < minKnown {+ return nil+ }+ h := sha256.New()+ fmt.Fprintf(h, "%t\x00%t\x00", o.Secrets, o.Personal)+ kinds := make([]string, 0, len(o.Kinds))+ for k, on := range o.Kinds {+ kinds = append(kinds, fmt.Sprintf("%s=%t", k, on))+ }+ slices.Sort(kinds)+ fmt.Fprintf(h, "%s\x00", strings.Join(kinds, ","))+ h.Write([]byte(s))+ var k [32]byte+ h.Sum(k[:0])+ sentKnown.Lock()+ sp, ok := sentKnown.cur[k]+ if !ok {+ if sp, ok = sentKnown.old[k]; ok {+ sentKnownAdd(k, sp) // still in use: kept past the next turnover+ }+ }+ sentKnown.Unlock()+ if ok {+ return slices.Clone(sp)+ }+ sp = knownSpans(s, o)+ sentKnown.Lock()+ sentKnownAdd(k, slices.Clone(sp))+ sentKnown.Unlock()+ return sp+}++// sentKnownAdd is called with sentKnown held. Two generations, as+// scrubMemo keeps: when the newer is full it becomes the older.+func sentKnownAdd(k [32]byte, sp []span) {+ if sentKnown.cur == nil || len(sentKnown.cur) >= sentEntries {+ sentKnown.old, sentKnown.cur = sentKnown.cur, make(map[[32]byte][]span)+ }+ sentKnown.cur[k] = sp+}++// resetSent forgets the texts sent (a test starts from nothing).+func resetSent() {+ sentKnown.Lock()+ defer sentKnown.Unlock()+ sentKnown.cur, sentKnown.old = nil, nil+}diff --git a/internal/redact/redact.go b/internal/redact/redact.goindex 9675a96f2..46fbb8414 100644--- a/internal/redact/redact.go+++ b/internal/redact/redact.go@@ -62,6 +62,9 @@ type Options struct {// found are the spans Model found, by text, for one request's textsfound map[string][]Found+ // sent is masking for a vendor (Mask, MaskJSON): a text masked+ // before has the known values it had then (sentKnownSpans)+ sent bool}// covers says o masks what a rule of category cat finds: "" are the@@ -478,6 +481,7 @@ func anyCase(w string) *regexp.Regexp {// Mask swaps what o covers in s for placeholders, and says how many.func Mask(s string, o Options) (string, int) {+ o.sent = truereturn mask(s, withModel(o, []string{s}), placeholder, false)}@@ -603,7 +607,11 @@ func mask(raw string, o Options, put func(kind, v string) string, escaped bool)// after the rules', so where a rule finds the same value it keeps the// kind it has always hadfound = append(found, modelSpans(raw, o)...) // the model was asked about raw- found = append(found, knownSpans(s, o)...)+ if o.sent {+ found = append(found, sentKnownSpans(s, o)...)+ } else {+ found = append(found, knownSpans(s, o)...)+ }if escaped {found = outsideEscapes(raw, found)}diff --git a/internal/redact/sent_known_test.go b/internal/redact/sent_known_test.gonew file mode 100644index 000000000..2dc65708c--- /dev/null+++ b/internal/redact/sent_known_test.go@@ -0,0 +1,57 @@+package redact++import (+ "strings"+ "testing"+)++// A text sent before goes as it went, though a value in it became known+// since: a vendor caches a conversation by its bytes, and masking the+// value now in a system prompt already sent with it rewrote the whole+// prompt to the cache on the next turn (audie on Discord: 400K tokens+// written, none read, ten seconds after the turn before). New text has+// the value masked as before.+func TestSentTextKeepsItsMasking(t *testing.T) {+ resetSent()+ t.Cleanup(resetSent)+ o := Options{Secrets: true, Personal: true}+ const secret = "Rv9sentPass27x"+ system := "You are dsh. Deploy notes: the staging pass is " + secret + " for now."+ first, n := Mask(system, o)+ if first != system || n != 0 {+ t.Fatalf("no rule finds it here, and nothing is known: %q", first)+ }+ body := `{"messages":[{"role":"system","content":"` + system + `"},{"role":"user","content":"go"}]}`+ firstBody, _ := MaskJSON([]byte(body), o)++ // a tool's output in the next turn has it where a rule finds it+ if out, _ := Mask("DB_PASSWORD="+secret, o); strings.Contains(out, secret) {+ t.Fatalf("the rule should mask it: %q", out)+ }++ // the system prompt and the body sent before go as they went+ if again, _ := Mask(system, o); again != first {+ t.Fatalf("a text sent before was masked again:\n%q\nwas\n%q", again, first)+ }+ if again, _ := MaskJSON([]byte(body), o); string(again) != string(firstBody) {+ t.Fatalf("a body sent before was masked again:\n%s\nwas\n%s", again, firstBody)+ }+ // with more after it, the earlier text is the same bytes+ longer := `{"messages":[{"role":"system","content":"` + system + `"},{"role":"user","content":"go"},{"role":"assistant","content":"the pass is ` + secret + `"}]}`+ got, _ := MaskJSON([]byte(longer), o)+ if !strings.HasPrefix(string(got), string(firstBody[:strings.Index(string(firstBody), `,{"role":"user"`)])) {+ t.Fatalf("the prefix changed: %s", got)+ }+ // the new text has the value as its placeholder+ if strings.Count(string(got), secret) != 1 {+ t.Fatalf("the new turn has the value as written: %s", got)+ }+ if out, _ := Mask("now the pass is "+secret+" again", o); strings.Contains(out, secret) {+ t.Fatalf("a known value went out in new text: %q", out)+ }++ // what is covered changing is another text: masked with what is known+ if out, _ := Mask(system, Options{Secrets: true}); strings.Contains(out, secret) {+ t.Fatalf("other settings, masked as known: %q", out)+ }+}