Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.
guard: on a Mac AstrLink Guard runs through CoreML, 512 tokens in 12ms where the CPU takes 400
yetonef0ffee34ce82
diff --git a/docs/subsystems/gateway-routing.md b/docs/subsystems/gateway-routing.mdindex a00f276b0..73476208a 100644--- a/docs/subsystems/gateway-routing.md+++ b/docs/subsystems/gateway-routing.md@@ -133,7 +133,7 @@ recorded as a prediction failure (`noPrediction`).1. `handle(proto)` reads the body and the model (`requestBody`, `requestModel`), then calls `serveAgent`. Every route that reads an agent's body (each API, count_tokens, Gemini's, embeddings, images, videos, System One, the Codex backend path) reads it through `readBoundedRequestBody` in [`request_bounds.go`](../../internal/gateway/request_bounds.go). It decodes a `Content-Encoding` of `gzip` or `zstd` (`decodedBody`; Codex sends zstd to a custom provider's `/v1/responses` too, #1223) and drops the header, so the provider is sent plain JSON, with one exception: a ChatGPT account's turn (POST `CodexBase`/responses, from the accounts loop in `forwardOnce` or relayed by `codexUpstreamOn`) is compressed again with zstd and a window of at most 2 MiB, as Codex's own client sends it (`provider.CodexCompressed`; openai/codex `responses_request_compression` and `http-client/src/request.rs`). It is compressed once the body is final, and again on each try. `/responses/compact`, an API key's requests and every other provider stay plain. Sent plain, a 1.1 MB text-only turn was answered 503 (ylorn on X). The plain body is what magpie keeps and shows, and `codexDropped` judges a reset by the bytes that went on the wire. `codexDropped` also reads Envoy's 507 "exceeded request buffer limit while retrying upstream" (the backend failed and the front couldn't retry a body that large) as that reset, at any size, told as a 503 so magpie, Codex and pi ask again and nobody rests (#2034). The size limit (`defaultBodyLimit`, 512 MB) holds for the decoded bytes (413), and any other encoding is refused 415. It was 128 MB until a long Codex session that had read many images went past it decoded, its inline screenshots resent each turn, and got the 413 on all five of Codex's retries (#1422, `TestImageHeavyCodexTurnIsRead`). magpie's own 413 says it is magpie's gateway limit, with the size, and that the provider was not asked (`overLimit`); `writeError` adds to any 413 it isn't holding that this is a byte limit, not the model's context, and to `/compact` or start a new conversation (`TestOverLimitSaysWhoseAndWhatToDo`). Every shape but Gemini's types a 413 `request_too_large`, Anthropic's own type, which pi's overflow table knows: pi compacts and tries once more rather than stopping on an OpenAI-shaped `api_error` (@wilsonxuuuu on X, `TestPayloadTooLargeSaysRequestTooLargeToPi`). `keyLimited` reads a limited key's compressed body the same way, so its reservation is the decoded size. `serveAgent` runs the [middleware](gateway-middleware.md) chain, then `serve`.2. `serve` redacts secrets in the request (`redacted`; `redact.MaskJSON` leaves ids alone, and in any text a backend's id (a UUID, or 32 hex digits or more) goes as written: no rule, model label or value known from before masks a piece of one (a phone number at the end of `…-a13800138000`, a card number across a UUID's groups), and a whole one is masked only by the user's own words and rules or as a credential, by the password, API key and token rules or a secret rule whose field is named for one (`api_key=`, `refresh_token`), never one named for an id, a cursor, a request or a digest (`secret_id`, `next_page_token`, `Idempotency-Key`, `secret_hash`), a bare `key`, a serial number, or what the model takes a UUID for (`noIDs`, [`internal/redact/ids.go`](../../internal/redact/ids.go)); and a tool's name and its MCP server's (`namespace`, `server_label`, `server_name`, #3143); the request's tool definitions and output schemas, found by their place in each protocol (top-level `tools`, `tool_choice`, `response_format`, `text`, `generationConfig`, `stream_options`, the client's ids in `client_metadata` and the like, and the tools of a Responses `additional_tools` or `tool_search_output` item), go as written save their descriptions, so a word the user masks can't break a grammar or an enum (#1911); and what the vendor sealed with a signature, a thinking block, Gemini's thoughts or Responses' encrypted reasoning, goes back as the vendor wrote it: only the values masked elsewhere in the request, and secrets masked before, are put back as their placeholders; a value masked once, at least 8 characters long, is masked again wherever a request has it, as an exact match whatever words are around it, so what the vendor wrote about a placeholder doesn't go back with the value in it, `redact/known.go`; a text masked for a vendor before, such as a system prompt or a turn already sent, is masked as it was then, so a value a rule first finds later in a conversation doesn't change the prompt the vendor has cached and rewrite all of it to the cache, `sentKnownSpans`), resolves the model to a provider or a group, and refuses what can't be served. The reply has the values put back on its way to the agent (`redact.Writer`, a stream event by event, a whole body at its end): a placeholder the model wrote as it was given, or as models rewrite one (spaces inside the braces, a brace or both dropped, another case), is the value again when what is left names one magpie made, and one split across events is held until it is whole (`redact.Restore`, `partialTail`, [`internal/redact/redact.go`](../../internal/redact/redact.go)). A request whose provider and fallbacks, or whose group's models, all skip redaction (`unredactedRoute`, `provider.SkipsRedaction`: set *Send requests unmasked*, `Unredacted`, with every address on this machine or the local network) goes to them as the agent wrote it; the log and recent calls keep the masked body. count_tokens and embeddings follow the same rule. What a routing group's Jev is told of the turn (the message, the user's message before it and the reply to it) is masked all the same unless Jev's own provider skips redaction too (`maskedState` in `askJev`, [`decide.go`](../../internal/gateway/decide.go)): a remote Jev is none of the group's members. The masking is the built-in plugin `redact`'s (`gwmw.Masker`, [Plugin runtime](plugin-runtime.md)); when the settings ask for masking and no masker serves, or it fails, the request is a 503 and nothing is sent (`redacted`, `gwmw.Masking`).- With *Mask personal data* on, each kind in `redact.Categories` ([`internal/redact/more.go`](../../internal/redact/more.go)) is masked unless the user turned it off (`redactKinds` in settings, `Options.Kinds`). On by default: US SSNs, passport numbers, IBANs that pass mod 97, birthdays next to a word that says so (a date over a year ago: one in the last year or ahead is the day an agent runs on, `dateOK`), MAC addresses, and serial numbers next to S/N or the like. Off by default: the user name in a home folder, user@host in a prompt or ssh command, public IPv4 and IPv6 addresses (never private, loopback, link-local, CGNAT or documentation ranges) and cloud bucket names, since agents send them in nearly every request and the model must copy the placeholder exactly inside a path or URL. Context rules run only near their words (`rule.matches`). What reads as an address but ends in a file's extension that no domain has (`…@2x.jpg`, `[email protected]`; `fileExt` in `realEmail`), or a scale before an extension that is a domain too (`[email protected]`, `[email protected]`; `scaleExt`), is a file's name, for the rule and the local model alike (#3029). `[email protected]` is still an address. The Settings rows come from `REDACT_KINDS` in app.js, which `TestRedactKindsAreRedacts` holds to the same ids and defaults.- - With *Find with a local model* on (`settings.RedactModel`, this computer's own, kept by `KeepOwn`), a local model finds what no rule can, a name or an address in free text, under the same switches: a name, address, phone, email or account number is personal data, a secret a secret (`modelKinds`, `modelSpans` in [`internal/redact/model.go`](../../internal/redact/model.go)). `MaskJSON` asks it once per request, for every text it would mask, each once (`withModel`); its spans join the rules' with a rule's kind kept where both find a value, and ids a vendor or tool made (`req_…`, `toolu_…`) are never taken, nor a snake_case word as a name (Codex's `sequential_cutoff`, #3144), nor as a secret what names a thing: a UUID, a digest with its algorithm's name, a file's path (`namesThing`; the model takes all three for secrets). The model misses many Chinese names (张伟 after 给, 陈建国 after 客户, however low its threshold) and runs a name it finds on into the words after it (「王芳,她下周一」), so with it on a Chinese name is also found by its surname and a word around it that says a person is meant (客户, 我叫, 给…发, …先生, …说, joined by 和 to another name), and the model's are cut at punctuation and titles ([`zhname.go`](../../internal/redact/zhname.go)); a surname with nothing around it (王国的, 张开的) is left. The model is the user's pick (`settings.RedactModelName`, `pii.Choose`; the app's menu beside the switch, `pmPick`): AstrLink Guard (QuantumNous/astrlink-guard at a pinned revision, Apache-2.0, a 39 MB int8 Multilingual-MiniLM token classifier for Chinese and English), run in magpie's own process on the CPU by [`internal/pii/guard`](../../internal/pii/guard) (an ONNX reader, the XLM-R tokenizer and int8 kernels in Go, NEON/SDOT on arm64 and SSE2 on amd64; its labels named as Privacy Filter's, `guardLabels`), loaded on the first request and let go after 10 minutes with none; or OpenAI's Privacy Filter. With no pick, a user who had Privacy Filter on or downloaded keeps it and everyone else gets AstrLink Guard (#3578): on 234 segments of real code and config it found 2 spans where Privacy Filter found 100 (38 of them go.sum hashes after redact's own checks), and with the rules it found 42 of 42 test values to Privacy Filter's 41. Each model has its own download, cache file and turn (`backend`). Both find at a floor of 0.5 (`pii.Floor`) and give each span its score (AstrLink Guard: the least of its tokens' probabilities, rounded to a float32 so a cached span reads back the same; Privacy Filter: pf-serve's own), and masking keeps those at the model's threshold or over (`pii.Threshold` in `masking.model`): the user's for that model (`settings.RedactModelScore`, this computer's own; Settings → Privacy's Confidence threshold, `pmScore`, 0.5 to 0.95), else the model's own (`MinScore`: Privacy Filter 0.5, AstrLink Guard 0.8, its publisher's). The defaults stay because on the eval a higher one mostly loses values the checks after the model would have kept (Privacy Filter at 0.85 lost a Chinese name; AstrLink Guard at 0.9 a phone in a tool result), while the false alarms it would drop are mostly dropped by those checks already (Yayoi_no_yume on X). Privacy Filter is a q8 GGUF (1.6 GB), run by `pf-serve` ([`native/privacy-filter`](../../native/privacy-filter), built from privacy-filter.cpp on ggml and released under `pf-v*` tags; on a GPU it holds the model once, in the device's memory, #3590; `pf-serve-vulkan` runs its MoE in 2048-token chunks, `PF_MOE_CHUNK`, 1.77× faster than its default 4096 on an RX 470/580-class GPU and a little slower on Metal, so Vulkan's alone, and a value the user set is kept, `moeChunk`): Settings → Privacy's On downloads both (`pii.Download`, each checked against its pinned sha256, through magpie's proxy (`netproxy.Func`, as its other requests; hh0592821 on Discord, whose download went direct before), from hf-mirror first in China (`source.chinaFirst`) and from hf-mirror after Hugging Face everywhere else, the switch off too (`source.modelMirror`), so a poisoned or blocked huggingface.co doesn't stop it; hf-mirror serves AstrLink Guard's files itself and sends Privacy Filter's back to Hugging Face) and saves the setting only once they are on disk (`/api/privacy-model`, [`privacymodel.go`](../../internal/gui/privacymodel.go)). `pii.Find` starts `pf-serve` on the first request (Metal, Vulkan, else the CPU; one that dies on the GPU, doesn't load there in 40s (`gpuLoadTimeout`, well inside the gateway's 2 minutes), or doesn't answer in time there twice in a row, is started on the CPU, and Settings says why; a request it didn't answer in time once is tried on the GPU again, since a slow request on a weak GPU is no lost GPU (#3682). The GPU is tried again 10 minutes later, doubling to 2 hours while it keeps failing, with no restart (`gpuRetryAfter`), and one that answers is the model's again), joins short texts into 12 KB chunks and maps the spans back, cuts a text over 12 KB into pieces of 12 KB at most that overlap by 1 KB (`pieces`: after a line's end, else a space, else at a rune's start), each kept by its own hash, their spans put back at the text's offsets and merged where pieces overlap (`merge`: one found twice is one, one cut by a piece's end is whole), and asks pf-serve a chunk at a time (#3682: a 480 KB text whole took an RX 470/580 90s and kept nothing when cut off), keeps the last 20000 answers by hash, on disk too (`found-<model>.bin` beside the model: hashes, offsets, labels and each span's score, never text; a `pfound01` file from before scores is read with every span sure, so it masks as it did; written through temp-and-rename, a bad read taken as empty), so a restart doesn't ask the model about every history again (#3682), and ends it after 10 minutes with no request. Its requests take turns on its one pipe (`turn`), and a request reads the cache again when its turn comes; a request waits for its turn only while its context lives (`pii.FindContext`, through `gwmw.ContextTeller`), so one whose client went away leaves the queue (the gateway's 2 minutes leave masking's time out, #3682); the chunk pf-serve is on is finished and kept, so a request sent again asks only about what is left, and between chunks the turn is given up and taken again, so a short request goes before the rest of a long one. The status (`pii.Now`), `pii.Stop` and the idle timer never wait for a load or a request: `srv.mu` is held only to read or set the state, and `Stop` ends a load or a request under way at once. A request held before routing while it waits for the model is answered 504 saying so, and how to send without it (`unroutedMsg`); one of an older tag (`oldTags`, magpie updated since the download) runs until the pinned tag's is fetched (`upgrade`, on its start), which is then started on the next request and the old deleted ([`internal/pii`](../../internal/pii)). A failure names what ended `pf-serve`: magpie's own end of a request it didn't answer in time, else its exit status or signal and its last stderr lines past ggml's startup chatter (`died`, `lastSaid`). After one, requests are held back at once with that failure, not after a load each, until the model is tried again 30s later, doubling to 10 minutes while it keeps failing (`heldBack`, `retryAfter`; lazyeo on Discord, an M2 whose every request waited out the 504). A model that is on and can't answer (not on disk, didn't start, hung, held back) panics, so the request is held back with the 503 above rather than sent with the rules' masking alone (`masking.model`).+ - With *Find with a local model* on (`settings.RedactModel`, this computer's own, kept by `KeepOwn`), a local model finds what no rule can, a name or an address in free text, under the same switches: a name, address, phone, email or account number is personal data, a secret a secret (`modelKinds`, `modelSpans` in [`internal/redact/model.go`](../../internal/redact/model.go)). `MaskJSON` asks it once per request, for every text it would mask, each once (`withModel`); its spans join the rules' with a rule's kind kept where both find a value, and ids a vendor or tool made (`req_…`, `toolu_…`) are never taken, nor a snake_case word as a name (Codex's `sequential_cutoff`, #3144), nor as a secret what names a thing: a UUID, a digest with its algorithm's name, a file's path (`namesThing`; the model takes all three for secrets). The model misses many Chinese names (张伟 after 给, 陈建国 after 客户, however low its threshold) and runs a name it finds on into the words after it (「王芳,她下周一」), so with it on a Chinese name is also found by its surname and a word around it that says a person is meant (客户, 我叫, 给…发, …先生, …说, joined by 和 to another name), and the model's are cut at punctuation and titles ([`zhname.go`](../../internal/redact/zhname.go)); a surname with nothing around it (王国的, 张开的) is left. The model is the user's pick (`settings.RedactModelName`, `pii.Choose`; the app's menu beside the switch, `pmPick`): AstrLink Guard (QuantumNous/astrlink-guard at a pinned revision, Apache-2.0, a 39 MB int8 Multilingual-MiniLM token classifier for Chinese and English), run in magpie's own process on the CPU by [`internal/pii/guard`](../../internal/pii/guard) (an ONNX reader, the XLM-R tokenizer and int8 kernels in Go, NEON/SDOT on arm64 and SSE2 on amd64; its labels named as Privacy Filter's, `guardLabels`), and on a Mac through CoreML on the Neural Engine or the GPU ([`coreml.go`](../../internal/pii/guard/coreml.go), [`internal/pii/guard/coreml`](../../internal/pii/guard/coreml); yyhhyyyyyy on X): the same network from the same int8 weights in float16, written from the downloaded model into a program made by `coreml_gen.py` and compiled once into the model's folder, 512 tokens in about 12 ms where the CPU takes 400, its probabilities within .07 of the publisher's fp32 model where the CPU's are within .13 (the CPU quantizes activations as the int8 model does, CoreML doesn't), so a span near the threshold can fall the other side (a go.mod module path is a secret at .81 on CoreML, .79 on the CPU, .785 in fp32); a window under 16 tokens (`coremlMin`), a Mac before macOS 13 or any failure goes to the CPU, loaded on the first request and let go after 10 minutes with none; or OpenAI's Privacy Filter. With no pick, a user who had Privacy Filter on or downloaded keeps it and everyone else gets AstrLink Guard (#3578): on 234 segments of real code and config it found 2 spans where Privacy Filter found 100 (38 of them go.sum hashes after redact's own checks), and with the rules it found 42 of 42 test values to Privacy Filter's 41. Each model has its own download, cache file and turn (`backend`). Both find at a floor of 0.5 (`pii.Floor`) and give each span its score (AstrLink Guard: the least of its tokens' probabilities, rounded to a float32 so a cached span reads back the same; Privacy Filter: pf-serve's own), and masking keeps those at the model's threshold or over (`pii.Threshold` in `masking.model`): the user's for that model (`settings.RedactModelScore`, this computer's own; Settings → Privacy's Confidence threshold, `pmScore`, 0.5 to 0.95), else the model's own (`MinScore`: Privacy Filter 0.5, AstrLink Guard 0.8, its publisher's). The defaults stay because on the eval a higher one mostly loses values the checks after the model would have kept (Privacy Filter at 0.85 lost a Chinese name; AstrLink Guard at 0.9 a phone in a tool result), while the false alarms it would drop are mostly dropped by those checks already (Yayoi_no_yume on X). Privacy Filter is a q8 GGUF (1.6 GB), run by `pf-serve` ([`native/privacy-filter`](../../native/privacy-filter), built from privacy-filter.cpp on ggml and released under `pf-v*` tags; on a GPU it holds the model once, in the device's memory, #3590; `pf-serve-vulkan` runs its MoE in 2048-token chunks, `PF_MOE_CHUNK`, 1.77× faster than its default 4096 on an RX 470/580-class GPU and a little slower on Metal, so Vulkan's alone, and a value the user set is kept, `moeChunk`): Settings → Privacy's On downloads both (`pii.Download`, each checked against its pinned sha256, through magpie's proxy (`netproxy.Func`, as its other requests; hh0592821 on Discord, whose download went direct before), from hf-mirror first in China (`source.chinaFirst`) and from hf-mirror after Hugging Face everywhere else, the switch off too (`source.modelMirror`), so a poisoned or blocked huggingface.co doesn't stop it; hf-mirror serves AstrLink Guard's files itself and sends Privacy Filter's back to Hugging Face) and saves the setting only once they are on disk (`/api/privacy-model`, [`privacymodel.go`](../../internal/gui/privacymodel.go)). `pii.Find` starts `pf-serve` on the first request (Metal, Vulkan, else the CPU; one that dies on the GPU, doesn't load there in 40s (`gpuLoadTimeout`, well inside the gateway's 2 minutes), or doesn't answer in time there twice in a row, is started on the CPU, and Settings says why; a request it didn't answer in time once is tried on the GPU again, since a slow request on a weak GPU is no lost GPU (#3682). The GPU is tried again 10 minutes later, doubling to 2 hours while it keeps failing, with no restart (`gpuRetryAfter`), and one that answers is the model's again), joins short texts into 12 KB chunks and maps the spans back, cuts a text over 12 KB into pieces of 12 KB at most that overlap by 1 KB (`pieces`: after a line's end, else a space, else at a rune's start), each kept by its own hash, their spans put back at the text's offsets and merged where pieces overlap (`merge`: one found twice is one, one cut by a piece's end is whole), and asks pf-serve a chunk at a time (#3682: a 480 KB text whole took an RX 470/580 90s and kept nothing when cut off), keeps the last 20000 answers by hash, on disk too (`found-<model>.bin` beside the model: hashes, offsets, labels and each span's score, never text; a `pfound01` file from before scores is read with every span sure, so it masks as it did; written through temp-and-rename, a bad read taken as empty), so a restart doesn't ask the model about every history again (#3682), and ends it after 10 minutes with no request. Its requests take turns on its one pipe (`turn`), and a request reads the cache again when its turn comes; a request waits for its turn only while its context lives (`pii.FindContext`, through `gwmw.ContextTeller`), so one whose client went away leaves the queue (the gateway's 2 minutes leave masking's time out, #3682); the chunk pf-serve is on is finished and kept, so a request sent again asks only about what is left, and between chunks the turn is given up and taken again, so a short request goes before the rest of a long one. The status (`pii.Now`), `pii.Stop` and the idle timer never wait for a load or a request: `srv.mu` is held only to read or set the state, and `Stop` ends a load or a request under way at once. A request held before routing while it waits for the model is answered 504 saying so, and how to send without it (`unroutedMsg`); one of an older tag (`oldTags`, magpie updated since the download) runs until the pinned tag's is fetched (`upgrade`, on its start), which is then started on the next request and the old deleted ([`internal/pii`](../../internal/pii)). A failure names what ended `pf-serve`: magpie's own end of a request it didn't answer in time, else its exit status or signal and its last stderr lines past ggml's startup chatter (`died`, `lastSaid`). After one, requests are held back at once with that failure, not after a load each, until the model is tried again 30s later, doubling to 10 minutes while it keeps failing (`heldBack`, `retryAfter`; lazyeo on Discord, an M2 whose every request waited out the 504). A model that is on and can't answer (not on disk, didn't start, hung, held back) panics, so the request is held back with the 503 above rather than sent with the rules' masking alone (`masking.model`).- With *Mask secrets* on, the user's own masking rules (`redactRules`, `redact.Rule`: a placeholder name and a prefix or an RE2 pattern, [`internal/redact/rules.go`](../../internal/redact/rules.go)) run before magpie's. Settings → Privacy adds them, removes them, and edits one in place (`POST /api/settings/redact-rule` with the rule's index and its current value, `editRedactRule` in [`internal/gui/api.go`](../../internal/gui/api.go)). Every rule is checked by `redact.CheckRules` when settings are saved; an edit that doesn't compile, matches the empty string, repeats another rule or names a rule no longer at that index is refused with the rules left as they were. The masker reads the rules from settings on each call (`options` in [`internal/masking`](../../internal/masking/masking.go)), so a change applies to the next request.- With *Keep a key's form* on (`settings.RedactShape`, `Options.Shape`, under *Mask secrets*; Charles Cao on X), a secret of 16 characters or more, all letters, digits, `-`, `_` or `.`, goes as a made-up key of its form instead of a placeholder: its prefix kept (`sk-proj-`, `ghp_`, `AKIA`, up to four letters and their separator; `keyHead`), the same length and separators, and each other character a random one of its kind (a hex key's any hex digit of its case), drawn from an HMAC of the value under the placeholder key, so one key goes as the same made-up one every turn and a vendor's cache still matches (`shapeOf`, `shaped` in [`internal/redact/shape.go`](../../internal/redact/shape.go)). Personal data, the user's words and anything shorter or with other characters go as placeholders still. A made-up key is kept in the masker's memory with the placeholders, so it is recorded, re-masked in what a vendor sealed, and put back wherever the reply has it, before the placeholders (`restoreShapes`); a stream holds the end of an event that may be the start of one (`shapeTail`, after the last whole one), and a made-up key the vendor sends back unrestored isn't masked again (`outsideShapes`). `TestShapeKeepsForm`, `TestShapeJSON`, `TestShapeStream`. Insights › Masking and Settings' *Try it* mark a made-up key where they mark a placeholder, matching the text against the request's own values only, so a key of the same form the user wrote isn't marked (`maskMatches` in `plugins/redact/page.js`; #3599, masking-records and mask-actions GUI tests).- What is done with a value found is the user's per kind (#1977, `redactActions` in settings, `Options.Actions`, set by `POST /api/settings/redact-actions`; a kind left out is masked). `redact.Warn` sends it as written: the masker tells it, with a placeholder no one remembers, and it isn't counted as masked. `redact.Block` masks it as any other, and the gateway holds the request back with a 400 that names the kind and the value's hint, never the value (`blocked`, `maskBlocked` in [`redact.go`](../../internal/gateway/redact.go)); nothing is sent and the record is kept as `Blocked` (`noteBlocked`). A value on the *Never mask* list (`redactAllow`, `Options.Allow`, `POST /api/settings/redact-allow`) goes as written when it is the whole value found, in any case, and isn't told; a longer value it is part of is masked. Settings → Privacy's *Try it* posts a text to `POST /api/masking/preview` ([`masking.go`](../../internal/gui/masking.go)), which masks it as a request would be (`redact.Preview`) and remembers nothing: its placeholders stand for nothing afterwards, its values aren't masked later as seen before, and it returns only the masked text and each hit's kind, finder, action, placeholder and hint. The page keeps the text in no storage.diff --git a/internal/pii/guard/bert.go b/internal/pii/guard/bert.goindex 98bcd8d28..f815bde94 100644--- a/internal/pii/guard/bert.go+++ b/internal/pii/guard/bert.go@@ -297,8 +297,14 @@ func layerNorm(x []float32, rows int, w, b []float32) {// logits are the classifier's scores for each of ids, a sequence of up to// maxPos tokens with its <s> and </s>: [len(ids), labels].func (nt *net) logits(ids []int) []float32 {+ return nt.encode(nt.embed(ids, 0), len(ids))+}++// embed is the layer-normed embeddings of ids, [len(ids), hidden], in a+// slice of at least rows rows (the rest zero).+func (nt *net) embed(ids []int, rows int) []float32 {n := len(ids)- x := make([]float32, n*hidden)+ x := make([]float32, max(n, rows)*hidden)for t, id := range ids {row := x[t*hidden : t*hidden+hidden]w := nt.word[id*hidden : id*hidden+hidden]@@ -308,6 +314,11 @@ func (nt *net) logits(ids []int) []float32 {}}layerNorm(x, n, nt.embLNw, nt.embLNb)+ return x+}++// encode is the classifier's scores for embeddings x of n tokens.+func (nt *net) encode(x []float32, n int) []float32 {for i := range nt.layers {l := &nt.layers[i]qkv := l.qkv.matmul(x, n)diff --git a/internal/pii/guard/coreml.go b/internal/pii/guard/coreml.gonew file mode 100644index 000000000..3a4047c69--- /dev/null+++ b/internal/pii/guard/coreml.go@@ -0,0 +1,236 @@+package guard++import (+ "crypto/sha256"+ _ "embed"+ "encoding/binary"+ "encoding/hex"+ "encoding/json"+ "errors"+ "fmt"+ "math"+ "os"+ "path/filepath"++ "github.com/yetone/magpie/internal/pii/guard/coreml"+)++// On a Mac the encoder runs on CoreML, which puts it on the Neural Engine+// or the GPU: a window of 512 tokens takes about 12 ms there, where+// bert.go takes 400 (an M4 Pro). The program (coreml.mlmodel, made by+// coreml_gen.py) is the same network from the same int8 weights,+// dequantized to float16 and written here from the model downloaded, so+// nothing else is fetched; it is compiled once (about 5 s) and kept+// beside the model. Held to the publisher's float32 model, its+// probabilities are within .07 where bert.go's are within .13, with the+// same labels. When it can't be made or fails, bert.go reads; a weight+// that isn't the one the program was made from (another revision) is+// such a failure, and coreml_gen.py is run again for it.++//go:embed coreml.mlmodel+var coremlProgram []byte++//go:embed coreml.json+var coremlLayout []byte++// coremlMin is the fewest tokens a window goes to CoreML with: under it+// bert.go is as quick as a padded window (8 tokens 9 ms, 16 tokens 17).+const coremlMin = 16++// openCoreML opens the program for nt, compiling it into dir/coreml+// first if it isn't there.+func openCoreML(dir string, nt *net) (*coreml.Model, error) {+ if !coreml.Available {+ return nil, errors.New("no CoreML here")+ }+ sum := sha256.New()+ sum.Write(coremlProgram)+ sum.Write(coremlLayout)+ path := filepath.Join(dir, "coreml", hex.EncodeToString(sum.Sum(nil))[:12]+".mlmodelc")+ if _, err := os.Stat(filepath.Join(path, "model.mil")); err != nil {+ if err := compileCoreML(path, nt); err != nil {+ return nil, err+ }+ }+ return coreml.Open(path)+}++// compileCoreML writes the package, its weights from nt, and compiles it+// to path; the folders of other programs are removed once it is there.+func compileCoreML(path string, nt *net) error {+ w, err := coremlWeights(nt)+ if err != nil {+ return err+ }+ root := filepath.Dir(path)+ if err := os.MkdirAll(root, 0o755); err != nil {+ return err+ }+ tmp, err := os.MkdirTemp(root, "pkg-")+ if err != nil {+ return err+ }+ defer os.RemoveAll(tmp)+ pkg := filepath.Join(tmp, "m.mlpackage")+ data := filepath.Join(pkg, "Data", "com.apple.CoreML")+ if err := os.MkdirAll(filepath.Join(data, "weights"), 0o755); err != nil {+ return err+ }+ if err := os.WriteFile(filepath.Join(pkg, "Manifest.json"), []byte(coremlManifest), 0o644); err != nil {+ return err+ }+ if err := os.WriteFile(filepath.Join(data, "model.mlmodel"), coremlProgram, 0o644); err != nil {+ return err+ }+ if err := os.WriteFile(filepath.Join(data, "weights", "weight.bin"), w, 0o644); err != nil {+ return err+ }+ if err := coreml.Compile(pkg, path); err != nil {+ return err+ }+ es, _ := os.ReadDir(root)+ for _, e := range es {+ if n := e.Name(); n != filepath.Base(path) && n != filepath.Base(tmp) {+ os.RemoveAll(filepath.Join(root, n))+ }+ }+ return nil+}++const coremlManifest = `{+ "fileFormatVersion": "1.0.0",+ "itemInfoEntries": {+ "3855EAB2-EF6A-4533-A292-4BC2F7AA31FD": {+ "author": "com.apple.CoreML",+ "description": "CoreML Model Weights",+ "name": "weights",+ "path": "com.apple.CoreML/weights"+ },+ "F8FABAE7-63BE-4D77-ABB4-5EFDA56D0923": {+ "author": "com.apple.CoreML",+ "description": "CoreML Model Specification",+ "name": "model.mlmodel",+ "path": "com.apple.CoreML/model.mlmodel"+ }+ },+ "rootModelIdentifier": "F8FABAE7-63BE-4D77-ABB4-5EFDA56D0923"+}+`++// coremlWeights is the program's weight.bin from nt's weights, as+// coremltools writes it: a 64-byte header (the count, version 2), then+// each weight at the offset coreml.json gives, a 64-byte header (0xDEADBEEF,+// type 1 for float16, its size, where its data starts) and its float16s.+func coremlWeights(nt *net) ([]byte, error) {+ var lay struct {+ Size int+ Sha256 string+ Blobs [][2]any+ }+ if err := json.Unmarshal(coremlLayout, &lay); err != nil {+ return nil, err+ }+ src := map[string][]float32{}+ deq := func(m *qmat) []float32 {+ out := make([]float32, len(m.w))+ for c := 0; c < m.n; c++ {+ for r := 0; r < m.k; r++ {+ out[c*m.k+r] = float32(int32(m.w[c*m.k+r])-m.zp[c]) * m.scale[c]+ }+ }+ return out+ }+ for i := range nt.layers {+ l := &nt.layers[i]+ for k, v := range map[string][]float32{+ "qkv_weight": deq(&l.qkv), "qkv_bias": l.qkvBias,+ "ao_weight": deq(&l.attnOut), "ao_bias": l.attnBias,+ "ln1_gamma": l.ln1w, "ln1_beta": l.ln1b,+ "up_weight": deq(&l.up), "up_bias": l.upBias, "smooth_y": l.smooth,+ "dn_weight": deq(&l.down), "dn_bias": l.downBias,+ "ln2_gamma": l.ln2w, "ln2_beta": l.ln2b,+ } {+ src[fmt.Sprintf("l%d_%s", i, k)] = v+ }+ }+ src["logits_weight"], src["logits_bias"] = deq(&nt.cls), nt.clsBias+ out := make([]byte, lay.Size)+ le := binary.LittleEndian+ le.PutUint32(out, uint32(len(lay.Blobs)))+ le.PutUint32(out[4:], 2)+ for _, b := range lay.Blobs {+ key, _ := b[0].(string)+ off, _ := b[1].(float64)+ v, ok := src[key]+ at := int(off)+ if !ok || at+64+2*len(v) > len(out) {+ return nil, fmt.Errorf("the CoreML program's %s isn't the model's", key)+ }+ le.PutUint32(out[at:], 0xDEADBEEF)+ le.PutUint32(out[at+4:], 1)+ le.PutUint64(out[at+8:], uint64(2*len(v)))+ le.PutUint64(out[at+16:], uint64(at+64))+ for i, f := range v {+ le.PutUint16(out[at+64+2*i:], half(f))+ }+ delete(src, key)+ }+ if len(src) != 0 {+ return nil, fmt.Errorf("the CoreML program has %d weights fewer than the model", len(src))+ }+ if s := sha256.Sum256(out); hex.EncodeToString(s[:]) != lay.Sha256 {+ return nil, errors.New("the CoreML weights aren't the ones the program was made with")+ }+ return out, nil+}++// half is f in float16, rounded to the nearest even as numpy does.+func half(f float32) uint16 {+ b := math.Float32bits(f)+ sign := uint16(b>>16) & 0x8000+ man := b & 0x7fffff+ if b&0x7fffffff >= 0x7f800000 {+ if man != 0 {+ return sign | 0x7e00+ }+ return sign | 0x7c00+ }+ exp := int32(b>>23&0xff) - 127 + 15+ if exp >= 31 {+ return sign | 0x7c00+ }+ if exp <= 0 { // under float16's normals+ if exp < -10 {+ return sign+ }+ man |= 0x800000+ shift := uint32(14 - exp)+ h, rem, mid := man>>shift, man&(1<<shift-1), uint32(1)<<(shift-1)+ if rem > mid || rem == mid && h&1 == 1 {+ h+++ }+ return sign | uint16(h)+ }+ h, rem := uint32(exp)<<10|man>>13, man&0x1fff+ if rem > 0x1000 || rem == 0x1000 && h&1 == 1 {+ h++ // a carry into the exponent is still right+ }+ return sign | uint16(h)+}++// coremlLogits are nt.logits(ids), from m.+func coremlLogits(m *coreml.Model, nt *net, ids []int) ([]float32, error) {+ n := len(ids)+ if n > maxPos {+ return nil, errors.New("more tokens than a window")+ }+ mask := make([]float32, maxPos)+ for t := n; t < maxPos; t++ {+ mask[t] = -1e4+ }+ out, err := m.Run(nt.embed(ids, maxPos), mask, maxPos, hidden, nt.labels)+ if err != nil {+ return nil, err+ }+ return out[:n*nt.labels], nil+}diff --git a/internal/pii/guard/coreml.json b/internal/pii/guard/coreml.jsonnew file mode 100644index 000000000..c17cafa95--- /dev/null+++ b/internal/pii/guard/coreml.json@@ -0,0 +1,160 @@+{"size": 42650346, "sha256": "267f9b34318dc952731157200451833e60fa66a7dbe1f5b5dfa4ae9b3aba3314", "blobs": [+["l0_qkv_weight", 64],+["l0_qkv_bias", 884864],+["l0_ao_weight", 887232],+["l0_ao_bias", 1182208],+["l0_ln1_gamma", 1183040],+["l0_ln1_beta", 1183872],+["l0_up_weight", 1184704],+["l0_up_bias", 2364416],+["l0_smooth_y", 2367552],+["l0_dn_weight", 2370688],+["l0_dn_bias", 3550400],+["l0_ln2_gamma", 3551232],+["l0_ln2_beta", 3552064],+["l1_qkv_weight", 3552896],+["l1_qkv_bias", 4437696],+["l1_ao_weight", 4440064],+["l1_ao_bias", 4735040],+["l1_ln1_gamma", 4735872],+["l1_ln1_beta", 4736704],+["l1_up_weight", 4737536],+["l1_up_bias", 5917248],+["l1_smooth_y", 5920384],+["l1_dn_weight", 5923520],+["l1_dn_bias", 7103232],+["l1_ln2_gamma", 7104064],+["l1_ln2_beta", 7104896],+["l2_qkv_weight", 7105728],+["l2_qkv_bias", 7990528],+["l2_ao_weight", 7992896],+["l2_ao_bias", 8287872],+["l2_ln1_gamma", 8288704],+["l2_ln1_beta", 8289536],+["l2_up_weight", 8290368],+["l2_up_bias", 9470080],+["l2_smooth_y", 9473216],+["l2_dn_weight", 9476352],+["l2_dn_bias", 10656064],+["l2_ln2_gamma", 10656896],+["l2_ln2_beta", 10657728],+["l3_qkv_weight", 10658560],+["l3_qkv_bias", 11543360],+["l3_ao_weight", 11545728],+["l3_ao_bias", 11840704],+["l3_ln1_gamma", 11841536],+["l3_ln1_beta", 11842368],+["l3_up_weight", 11843200],+["l3_up_bias", 13022912],+["l3_smooth_y", 13026048],+["l3_dn_weight", 13029184],+["l3_dn_bias", 14208896],+["l3_ln2_gamma", 14209728],+["l3_ln2_beta", 14210560],+["l4_qkv_weight", 14211392],+["l4_qkv_bias", 15096192],+["l4_ao_weight", 15098560],+["l4_ao_bias", 15393536],+["l4_ln1_gamma", 15394368],+["l4_ln1_beta", 15395200],+["l4_up_weight", 15396032],+["l4_up_bias", 16575744],+["l4_smooth_y", 16578880],+["l4_dn_weight", 16582016],+["l4_dn_bias", 17761728],+["l4_ln2_gamma", 17762560],+["l4_ln2_beta", 17763392],+["l5_qkv_weight", 17764224],+["l5_qkv_bias", 18649024],+["l5_ao_weight", 18651392],+["l5_ao_bias", 18946368],+["l5_ln1_gamma", 18947200],+["l5_ln1_beta", 18948032],+["l5_up_weight", 18948864],+["l5_up_bias", 20128576],+["l5_smooth_y", 20131712],+["l5_dn_weight", 20134848],+["l5_dn_bias", 21314560],+["l5_ln2_gamma", 21315392],+["l5_ln2_beta", 21316224],+["l6_qkv_weight", 21317056],+["l6_qkv_bias", 22201856],+["l6_ao_weight", 22204224],+["l6_ao_bias", 22499200],+["l6_ln1_gamma", 22500032],+["l6_ln1_beta", 22500864],+["l6_up_weight", 22501696],+["l6_up_bias", 23681408],+["l6_smooth_y", 23684544],+["l6_dn_weight", 23687680],+["l6_dn_bias", 24867392],+["l6_ln2_gamma", 24868224],+["l6_ln2_beta", 24869056],+["l7_qkv_weight", 24869888],+["l7_qkv_bias", 25754688],+["l7_ao_weight", 25757056],+["l7_ao_bias", 26052032],+["l7_ln1_gamma", 26052864],+["l7_ln1_beta", 26053696],+["l7_up_weight", 26054528],+["l7_up_bias", 27234240],+["l7_smooth_y", 27237376],+["l7_dn_weight", 27240512],+["l7_dn_bias", 28420224],+["l7_ln2_gamma", 28421056],+["l7_ln2_beta", 28421888],+["l8_qkv_weight", 28422720],+["l8_qkv_bias", 29307520],+["l8_ao_weight", 29309888],+["l8_ao_bias", 29604864],+["l8_ln1_gamma", 29605696],+["l8_ln1_beta", 29606528],+["l8_up_weight", 29607360],+["l8_up_bias", 30787072],+["l8_smooth_y", 30790208],+["l8_dn_weight", 30793344],+["l8_dn_bias", 31973056],+["l8_ln2_gamma", 31973888],+["l8_ln2_beta", 31974720],+["l9_qkv_weight", 31975552],+["l9_qkv_bias", 32860352],+["l9_ao_weight", 32862720],+["l9_ao_bias", 33157696],+["l9_ln1_gamma", 33158528],+["l9_ln1_beta", 33159360],+["l9_up_weight", 33160192],+["l9_up_bias", 34339904],+["l9_smooth_y", 34343040],+["l9_dn_weight", 34346176],+["l9_dn_bias", 35525888],+["l9_ln2_gamma", 35526720],+["l9_ln2_beta", 35527552],+["l10_qkv_weight", 35528384],+["l10_qkv_bias", 36413184],+["l10_ao_weight", 36415552],+["l10_ao_bias", 36710528],+["l10_ln1_gamma", 36711360],+["l10_ln1_beta", 36712192],+["l10_up_weight", 36713024],+["l10_up_bias", 37892736],+["l10_smooth_y", 37895872],+["l10_dn_weight", 37899008],+["l10_dn_bias", 39078720],+["l10_ln2_gamma", 39079552],+["l10_ln2_beta", 39080384],+["l11_qkv_weight", 39081216],+["l11_qkv_bias", 39966016],+["l11_ao_weight", 39968384],+["l11_ao_bias", 40263360],+["l11_ln1_gamma", 40264192],+["l11_ln1_beta", 40265024],+["l11_up_weight", 40265856],+["l11_up_bias", 41445568],+["l11_smooth_y", 41448704],+["l11_dn_weight", 41451840],+["l11_dn_bias", 42631552],+["l11_ln2_gamma", 42632384],+["l11_ln2_beta", 42633216],+["logits_weight", 42634048],+["logits_bias", 42650240]+]}diff --git a/internal/pii/guard/coreml.mlmodel b/internal/pii/guard/coreml.mlmodelnew file mode 100644index 000000000..1cbd77171Binary files /dev/null and b/internal/pii/guard/coreml.mlmodel differdiff --git a/internal/pii/guard/coreml/coreml_darwin.go b/internal/pii/guard/coreml/coreml_darwin.gonew file mode 100644index 000000000..9ebf02dc9--- /dev/null+++ b/internal/pii/guard/coreml/coreml_darwin.go@@ -0,0 +1,159 @@+//go:build darwin && cgo++// Package coreml runs a compiled CoreML program with an input x+// [1, s, h] and mask [1, 1, 1, s] and an output logits [1, s, k]: the+// part of internal/pii/guard that links CoreML, apart as a package with+// cgo can't have guard's Go assembly.+package coreml++/*+#cgo CFLAGS: -x objective-c -fobjc-arc+#cgo LDFLAGS: -framework CoreML -framework Foundation+#include <stdlib.h>+#include <string.h>+#import <CoreML/CoreML.h>++static char *gmlErr(NSError *e, const char *what) {+ if (e == nil) return strdup(what);+ return strdup(e.localizedDescription.UTF8String ?: what);+}++// gmlCompile compiles the package at pkg into out (a .mlmodelc).+static char *gmlCompile(const char *pkg, const char *out) {+ @autoreleasepool {+ if (@available(macOS 13, *)) {+ } else {+ return strdup("CoreML programs need macOS 13");+ }+ NSError *err = nil;+ NSURL *c = [MLModel compileModelAtURL:[NSURL fileURLWithPath:@(pkg)] error:&err];+ if (c == nil) return gmlErr(err, "compiling failed");+ NSFileManager *fm = [NSFileManager defaultManager];+ NSURL *o = [NSURL fileURLWithPath:@(out)];+ [fm removeItemAtURL:o error:nil];+ if (![fm moveItemAtURL:c toURL:o error:&err]) {+ [fm removeItemAtURL:c error:nil];+ return gmlErr(err, "moving the compiled model failed");+ }+ return NULL;+ }+}++static void *gmlOpen(const char *path, char **errp) {+ @autoreleasepool {+ MLModelConfiguration *cfg = [MLModelConfiguration new];+ cfg.computeUnits = MLComputeUnitsAll;+ NSError *err = nil;+ MLModel *m = [MLModel modelWithContentsOfURL:[NSURL fileURLWithPath:@(path)] configuration:cfg error:&err];+ if (m == nil) {+ *errp = gmlErr(err, "loading failed");+ return NULL;+ }+ return (__bridge_retained void *)m;+ }+}++static void gmlClose(void *m) { CFRelease(m); }++static MLMultiArray *gmlArray(const float *src, NSArray<NSNumber *> *shape, NSArray<NSNumber *> *strides, size_t n, NSError **err) {+ float *b = malloc(n * sizeof(float));+ memcpy(b, src, n * sizeof(float));+ return [[MLMultiArray alloc] initWithDataPointer:b shape:shape dataType:MLMultiArrayDataTypeFloat32+ strides:strides deallocator:^(void *p) { free(p); } error:err];+}++// gmlRun reads x [1, s, h] and mask [1, 1, 1, s] into out [s, k].+static char *gmlRun(void *mp, const float *x, const float *mask, long s, long h, float *out, long k) {+ @autoreleasepool {+ MLModel *m = (__bridge MLModel *)mp;+ NSError *err = nil;+ MLMultiArray *xa = gmlArray(x, @[@1, @(s), @(h)], @[@(s * h), @(h), @1], s * h, &err);+ if (xa == nil) return gmlErr(err, "x");+ MLMultiArray *ma = gmlArray(mask, @[@1, @1, @1, @(s)], @[@(s), @(s), @(s), @1], s, &err);+ if (ma == nil) return gmlErr(err, "mask");+ MLDictionaryFeatureProvider *in = [[MLDictionaryFeatureProvider alloc] initWithDictionary:@{+ @"x": [MLFeatureValue featureValueWithMultiArray:xa],+ @"mask": [MLFeatureValue featureValueWithMultiArray:ma],+ } error:&err];+ if (in == nil) return gmlErr(err, "inputs");+ id<MLFeatureProvider> r = [m predictionFromFeatures:in error:&err];+ if (r == nil) return gmlErr(err, "prediction failed");+ MLMultiArray *lg = [r featureValueForName:@"logits"].multiArrayValue;+ if (lg == nil || lg.shape.count != 3 || lg.shape[1].longValue != s || lg.shape[2].longValue != k) return strdup("logits have another shape");+ long s1 = lg.strides[1].longValue, s2 = lg.strides[2].longValue;+ MLMultiArrayDataType dt = lg.dataType;+ if (dt != MLMultiArrayDataTypeFloat32 && dt != MLMultiArrayDataTypeFloat16) return strdup("logits are of another type");+ [lg getBytesWithHandler:^(const void *b, NSInteger size) {+ for (long t = 0; t < s; t++) {+ for (long c = 0; c < k; c++) {+ long i = t * s1 + c * s2;+ out[t * k + c] = dt == MLMultiArrayDataTypeFloat32 ? ((const float *)b)[i] : (float)((const __fp16 *)b)[i];+ }+ }+ }];+ return NULL;+ }+}+*/+import "C"++import (+ "errors"+ "runtime"+ "sync"+ "unsafe"+)++// Available says CoreML is here.+const Available = true++// Compile compiles the package at pkg into out (a .mlmodelc).+func Compile(pkg, out string) error {+ cp, co := C.CString(pkg), C.CString(out)+ defer C.free(unsafe.Pointer(cp))+ defer C.free(unsafe.Pointer(co))+ if ce := C.gmlCompile(cp, co); ce != nil {+ defer C.free(unsafe.Pointer(ce))+ return errors.New(C.GoString(ce))+ }+ return nil+}++// Model is a compiled program, opened for the Neural Engine, the GPU or+// the CPU, whichever CoreML picks; it is let go with the last reference.+type Model struct {+ mu sync.Mutex+ m unsafe.Pointer+}++// Open opens the compiled program at path.+func Open(path string) (*Model, error) {+ cp := C.CString(path)+ defer C.free(unsafe.Pointer(cp))+ var ce *C.char+ p := C.gmlOpen(cp, &ce)+ if p == nil {+ defer C.free(unsafe.Pointer(ce))+ return nil, errors.New(C.GoString(ce))+ }+ m := &Model{m: p}+ runtime.AddCleanup(m, func(p unsafe.Pointer) { C.gmlClose(p) }, p)+ return m, nil+}++// Run is the logits [s, k] for x [s, h] and mask [s].+func (m *Model) Run(x, mask []float32, s, h, k int) ([]float32, error) {+ if len(x) != s*h || len(mask) != s {+ return nil, errors.New("inputs of another size")+ }+ out := make([]float32, s*k)+ m.mu.Lock()+ ce := C.gmlRun(m.m, (*C.float)(&x[0]), (*C.float)(&mask[0]), C.long(s), C.long(h), (*C.float)(&out[0]), C.long(k))+ m.mu.Unlock()+ runtime.KeepAlive(m)+ if ce != nil {+ defer C.free(unsafe.Pointer(ce))+ return nil, errors.New(C.GoString(ce))+ }+ return out, nil+}diff --git a/internal/pii/guard/coreml/coreml_other.go b/internal/pii/guard/coreml/coreml_other.gonew file mode 100644index 000000000..f045df316--- /dev/null+++ b/internal/pii/guard/coreml/coreml_other.go@@ -0,0 +1,18 @@+//go:build !darwin || !cgo++package coreml++import "errors"++// Available says CoreML is here: only on a Mac (coreml_darwin.go).+const Available = false++var errNone = errors.New("no CoreML here")++func Compile(pkg, out string) error { return errNone }++type Model struct{}++func Open(path string) (*Model, error) { return nil, errNone }++func (*Model) Run(x, mask []float32, s, h, k int) ([]float32, error) { return nil, errNone }diff --git a/internal/pii/guard/coreml_gen.py b/internal/pii/guard/coreml_gen.pynew file mode 100644index 000000000..6c58a8809--- /dev/null+++ b/internal/pii/guard/coreml_gen.py@@ -0,0 +1,117 @@+# Writes coreml.mlmodel and coreml.json, the CoreML program coreml.go+# runs AstrLink Guard's encoder with, from the int8 model's own weights:+#+# python3 -I coreml_gen.py <the model's folder> (coremltools 9, numpy, onnx)+#+# The program is bert.go's network after the embeddings, for one window of+# 512 tokens: x [1, 512, 384] (the embeddings, layer-normed) and mask+# [1, 1, 1, 512] (0 for a token, -1e4 for padding) in, logits [1, 512, 21]+# out, in float16. Its weights are the int8 ones dequantized; they are not+# kept here but written on each Mac from the model downloaded+# (coremlWeights), into weight.bin at the offsets coreml.json gives, and+# coreml.json's sha256 is that file's as coremltools writes it.+import hashlib+import json+import os+import shutil+import sys+import tempfile++import coremltools as ct+import numpy as np+import onnx+from coremltools.converters.mil import Builder as mb+from coremltools.proto import Model_pb2+from onnx import numpy_helper++S, H, NH, D = 512, 384, 12, 32++m = onnx.load(os.path.join(sys.argv[1], "model_int8.onnx"))+T = {i.name: numpy_helper.to_array(i) for i in m.graph.initializer}+++def mat(p):+ q = T[p + "_quantized"].astype(np.float32)+ z = T[p + "_zero_point"].astype(np.float32)+ return ((q - z) * T[p + "_scale"]).T.copy() # [n, k], as linear takes it+++# Each weight by the key coremlWeights knows it by.+W = {}+for i in range(12):+ p, mm = f"inner.bert.encoder.layer.{i}.", 1299 + 16 * i+ W.update({+ f"l{i}_qkv_weight": mat(f"Attention_{i}_qkv_weight"),+ f"l{i}_qkv_bias": T[f"Attention_{i}_qkv_bias"],+ f"l{i}_ao_weight": mat(f"onnx::MatMul_{mm}"),+ f"l{i}_ao_bias": T[p + "attention.output.dense.bias"],+ f"l{i}_ln1_gamma": T[p + "attention.output.LayerNorm.weight"],+ f"l{i}_ln1_beta": T[p + "attention.output.LayerNorm.bias"],+ f"l{i}_up_weight": mat(f"onnx::MatMul_{mm + 1}"),+ f"l{i}_up_bias": T[p + "intermediate.dense.bias"],+ f"l{i}_smooth_y": T[f"r15_smooth_inv_s_{i}"],+ f"l{i}_dn_weight": mat(f"onnx::MatMul_{mm + 2}"),+ f"l{i}_dn_bias": T[p + "output.dense.bias"],+ f"l{i}_ln2_gamma": T[p + "output.LayerNorm.weight"],+ f"l{i}_ln2_beta": T[p + "output.LayerNorm.bias"],+ })+W["logits_weight"] = mat("onnx::MatMul_1478")+W["logits_bias"] = T["inner.classifier.bias"]++[email protected](input_specs=[mb.TensorSpec(shape=(1, S, H)), mb.TensorSpec(shape=(1, 1, 1, S))])+def prog(x, mask):+ def heads(t):+ return mb.transpose(x=mb.reshape(x=t, shape=[1, S, NH, D]), perm=[0, 2, 1, 3])++ for i in range(12):+ def lin(t, k):+ return mb.linear(x=t, weight=W[f"l{i}_{k}_weight"], bias=W[f"l{i}_{k}_bias"], name=f"l{i}_{k}")++ def norm(t, k):+ return mb.layer_norm(x=t, axes=[-1], gamma=W[f"l{i}_{k}_gamma"], beta=W[f"l{i}_{k}_beta"],+ epsilon=1e-5, name=f"l{i}_{k}")++ q, k, v = mb.split(x=lin(x, "qkv"), num_splits=3, axis=-1)+ q = mb.mul(x=heads(q), y=np.float32(1 / np.sqrt(D)))+ s = mb.add(x=mb.matmul(x=q, y=heads(k), transpose_y=True), y=mask)+ c = mb.matmul(x=mb.softmax(x=s, axis=-1), y=heads(v))+ c = mb.reshape(x=mb.transpose(x=c, perm=[0, 2, 1, 3]), shape=[1, S, H])+ x = norm(mb.add(x=lin(c, "ao"), y=x), "ln1")+ h = mb.mul(x=mb.gelu(x=lin(x, "up"), mode="EXACT"), y=W[f"l{i}_smooth_y"], name=f"l{i}_smooth")+ x = norm(mb.add(x=lin(h, "dn"), y=x), "ln2")+ return mb.linear(x=x, weight=W["logits_weight"], bias=W["logits_bias"], name="logits")+++ml = ct.convert(prog, convert_to="mlprogram", compute_precision=ct.precision.FLOAT16,+ minimum_deployment_target=ct.target.macOS13)+tmp = tempfile.mkdtemp()+pkg = os.path.join(tmp, "m.mlpackage")+ml.save(pkg)+data = os.path.join(pkg, "Data", "com.apple.CoreML")+spec = Model_pb2.Model()+spec.ParseFromString(open(os.path.join(data, "model.mlmodel"), "rb").read())+spec.description.metadata.Clear() # the date it was made, coremltools' version++blobs = []+fn = spec.mlProgram.functions["main"]+for op in fn.block_specializations[fn.opset].operations:+ v = op.attributes["val"] if op.type == "const" else None+ if v is None or not v.HasField("blobFileValue"):+ continue+ name = op.outputs[0].name+ assert name.endswith("_0_to_fp16"), name+ key = name[:-len("_0_to_fp16")]+ assert key in W and W[key].size == np.prod([d.constant.size for d in v.type.tensorType.dimensions]), key+ blobs.append([key, v.blobFileValue.offset])+assert sorted(k for k, _ in blobs) == sorted(W), "a weight is not in the program"++here = os.path.dirname(os.path.abspath(__file__))+with open(os.path.join(here, "coreml.mlmodel"), "wb") as f:+ f.write(spec.SerializeToString(deterministic=True)) # map fields in order+wb = open(os.path.join(data, "weights", "weight.bin"), "rb").read()+shutil.rmtree(tmp)+with open(os.path.join(here, "coreml.json"), "w") as f:+ f.write('{"size": %d, "sha256": "%s", "blobs": [\n' % (len(wb), hashlib.sha256(wb).hexdigest()))+ f.write(",\n".join(json.dumps(b) for b in blobs))+ f.write("\n]}\n")diff --git a/internal/pii/guard/coreml_test.go b/internal/pii/guard/coreml_test.gonew file mode 100644index 000000000..747281e68--- /dev/null+++ b/internal/pii/guard/coreml_test.go@@ -0,0 +1,137 @@+package guard++import (+ "bytes"+ "math"+ "os"+ "path/filepath"+ "strings"+ "testing"+ "time"++ "github.com/yetone/magpie/internal/pii/guard/coreml"+)++// half rounds as numpy's astype(float16) does, which made the sha256 in+// coreml.json: to the nearest, a tie to the even one.+func TestHalfRounds(t *testing.T) {+ for _, c := range []struct {+ f float32+ h uint16+ }{+ {1, 0x3c00}, {-2, 0xc000}, {65504, 0x7bff}, {65520, 0x7c00},+ {float32(math.Ldexp(1, -24)), 0x0001}, {float32(math.Ldexp(1, -25)), 0},+ {float32(math.Ldexp(3, -25)), 0x0002}, {float32(math.Ldexp(1, -14)), 0x0400},+ {1 + float32(math.Ldexp(1, -11)), 0x3c00}, {1 + float32(math.Ldexp(3, -11)), 0x3c02},+ {float32(math.Copysign(0, -1)), 0x8000}, {float32(math.NaN()), 0x7e00},+ {float32(math.Inf(-1)), 0xfc00},+ } {+ if got := half(c.f); got != c.h {+ t.Errorf("half(%g) = %#04x, want %#04x", c.f, got, c.h)+ }+ }+}++// The weights written from the model are byte for byte the ones+// coremltools wrote from it (coremlWeights checks the sha256).+func TestCoreMLWeights(t *testing.T) {+ nt, err := loadNet(dir(t) + "/model_int8.onnx")+ if err != nil {+ t.Fatal(err)+ }+ if _, err := coremlWeights(nt); err != nil {+ t.Fatal(err)+ }+}++// On a Mac, CoreML labels a long window as bert.go does, but for a token+// in a hundred, and finds the same spans, their scores within .2 (each is+// up to .13 from the publisher's float32 model, on either side). Load+// compiles the program into MAGPIE_GUARD_DIR/coreml the first time.+func TestCoreMLMatchesGo(t *testing.T) {+ d := dir(t)+ if !coreml.Available {+ t.Skip("no CoreML here")+ }+ t0 := time.Now()+ m, err := Load(d)+ if err != nil {+ t.Fatal(err)+ }+ if m.ml == nil {+ t.Fatalf("CoreML didn't open: %v", m.MLErr)+ }+ t.Logf("loaded in %v", time.Since(t0))+ text := strings.Repeat("联系人:张三,电话 13800138000,邮箱 [email protected]。The host 10.20.30.40 logs in as deploy with key sk-proj-AbCdEf0123456789xyz.\n", 12)+ toks := m.tok.encode(text)+ ids := []int{0}+ for _, k := range toks[:min(window, len(toks))] {+ ids = append(ids, k.id)+ }+ ids = append(ids, 2)+ want := m.net.logits(ids)+ t1 := time.Now()+ got, err := coremlLogits(m.ml, m.net, ids)+ if err != nil {+ t.Fatal(err)+ }+ t.Logf("%d tokens: CoreML %v", len(ids), time.Since(t1))+ k, same := len(labels), 0+ for i := range ids {+ if argmax(got[i*k:(i+1)*k]) == argmax(want[i*k:(i+1)*k]) {+ same+++ }+ }+ if same*100 < len(ids)*99 {+ t.Fatalf("CoreML labels %d of %d tokens as bert.go does", same, len(ids))+ }+ goModel := &Model{tok: m.tok, net: m.net}+ a, b := m.Find(text), goModel.Find(text)+ if len(a) != len(b) {+ t.Fatalf("CoreML finds %d spans, bert.go %d", len(a), len(b))+ }+ for i := range a {+ if a[i].Start != b[i].Start || a[i].End != b[i].End || a[i].Label != b[i].Label || math.Abs(a[i].Score-b[i].Score) > .2 {+ t.Errorf("CoreML %v, bert.go %v", a[i], b[i])+ }+ }+}++// When the program can't be made (here, weights that aren't the ones it+// was made from), the model loads and bert.go reads, and the program+// compiled before is left.+func TestCoreMLFailureFallsBack(t *testing.T) {+ d := dir(t)+ if !coreml.Available {+ t.Skip("no CoreML here")+ }+ before, _ := os.ReadDir(filepath.Join(d, "coreml"))+ saved := coremlLayout+ t.Cleanup(func() { coremlLayout = saved })+ coremlLayout = bytes.Replace(saved, []byte(`"sha256": "`), []byte(`"sha256": "0`), 1)+ m, err := Load(d)+ if err != nil {+ t.Fatal(err)+ }+ if m.ml != nil || m.MLErr == nil {+ t.Fatalf("CoreML opened: %v", m.MLErr)+ }+ text := strings.Repeat("联系人:张三,电话 13800138000。", 8)+ if got := m.Find(text); len(got) == 0 {+ t.Fatal("bert.go found nothing")+ }+ after, _ := os.ReadDir(filepath.Join(d, "coreml"))+ if len(after) != len(before) {+ t.Fatalf("the coreml folder went from %d to %d", len(before), len(after))+ }+}++func argmax(r []float32) int {+ best := 0+ for i, v := range r {+ if v > r[best] {+ best = i+ }+ }+ return best+}diff --git a/internal/pii/guard/guard.go b/internal/pii/guard/guard.goindex 09fe5e0b4..914e03d78 100644--- a/internal/pii/guard/guard.go+++ b/internal/pii/guard/guard.go@@ -1,14 +1,16 @@// Package guard runs AstrLink Guard (QuantumNous/astrlink-guard,// Apache-2.0), a token classifier for secrets and personal data in Chinese-// and English text, on the CPU in Go: its int8 ONNX model (37 MB) read by-// onnx.go and run by bert.go, its tokenizer by tokenizer.go. Nothing-// native is linked or started.+// and English text: its int8 ONNX model (37 MB) read by onnx.go and run+// by bert.go on the CPU in Go, or on a Mac by CoreML (coreml.go),+// its tokenizer by tokenizer.go. Nothing is started.package guardimport ("fmt""math""path/filepath"++ "github.com/yetone/magpie/internal/pii/guard/coreml")// labels are the model's BIO tags, by output index (its labels.json).@@ -26,6 +28,9 @@ var Files = []string{"model_int8.onnx", "model_int8.onnx_data", "tokenizer.json"type Model struct {tok *tokenizernet *net+ ml *coreml.Model // nil where CoreML isn't, or couldn't be made+ // MLErr is why CoreML isn't used, on a Mac.+ MLErr error}// Span is what the model found: text[Start:End], a Label of its (email,@@ -67,7 +72,20 @@ func Load(dir string) (*Model, error) {if nt.labels != len(labels) {return nil, fmt.Errorf("model: %d labels, not %d", nt.labels, len(labels))}- return &Model{tok, nt}, nil+ m := &Model{tok: tok, net: nt}+ m.ml, m.MLErr = openCoreML(dir, nt)+ return m, nil+}++// logits are the network's scores for ids, from CoreML when it is there+// and the window is long enough to gain by it.+func (m *Model) logits(ids []int) []float32 {+ if m.ml != nil && len(ids) >= coremlMin {+ if lg, err := coremlLogits(m.ml, m.net, ids); err == nil {+ return lg+ }+ }+ return m.net.logits(ids)}// Find is what the model finds in text, at Floor, each span with its score.@@ -91,7 +109,7 @@ func (m *Model) Find(text string) []Span {ids = append(ids, t.id)}ids = append(ids, 2)- lg := m.net.logits(ids)+ lg := m.logits(ids)k := len(labels)for i := 0; i < end-start; i++ {j := start + idiff --git a/internal/pii/guard/guard_test.go b/internal/pii/guard/guard_test.goindex 39eed0a81..bf1d541fd 100644--- a/internal/pii/guard/guard_test.go+++ b/internal/pii/guard/guard_test.go@@ -92,6 +92,9 @@ func TestModelFinds(t *testing.T) {if err != nil {t.Fatal(err)}+ // bert.go's own; TestCoreMLMatchesGo holds CoreML to it. CoreML gives+ // the module path .81, the publisher's fp32 model .785.+ m.ml = niltext := "联系人:张三,电话 13800138000,邮箱 [email protected]。\n" +"go 1.26\nrequire github.com/clipperhouse/uax29/v2 v2.2.0\n"// what masking keeps at the publisher's threshold: Find gives downdiff --git a/site/public/docs/ja/masking.html b/site/public/docs/ja/masking.htmlindex 772f1fef6..721be47a7 100644--- a/site/public/docs/ja/masking.html+++ b/site/public/docs/ja/masking.html@@ -113,7 +113,7 @@<tr><td><span class="ui">シークレットをマスク</span></td><td>秘密鍵のブロック(<code>-----BEGIN … PRIVATE KEY-----</code>)。形で見分ける API キー:<code>sk-…</code>(OpenAI、Anthropic、OpenRouter)、<code>ghp_</code>/<code>github_pat_</code>、<code>glpat-</code>、<code>AIza…</code>、<code>xox…-</code>、<code>sk_live_</code>、<code>AKIA</code>/<code>ASIA</code>、<code>hf_</code>、<code>gsk_</code>、<code>xai-</code>、<code>npm_</code>、<code>pypi-</code>、<code>dop_v1_</code>、<code>SG.</code>、それに <code>oc_sk_…</code> のような中継サービスのキー。JWT。<code>user:password@host</code> のパスワード。<code>.env</code> や設定ファイルでシークレットの名前が付いたフィールドの値:<code>DB_PASSWORD=…</code>、<code>api_key: …</code>、<code>client_secret</code>、<code>token</code>、<code>credential</code>。それと自分の <span class="ui">マスクルール</span>。</td></tr><tr><td><span class="ui">個人情報をマスク</span></td><td>メールアドレス、中国本土の携帯番号、チェックサムの合う中国の身分証番号と銀行カード番号。既定でオンのもの:米国の社会保障番号、パスポート番号(<em>passport</em>、护照、パスポートの後)、IBAN、誕生日(<em>birthday</em>、<em>DOB</em>、生日の近くにある、一年以上前の日付。今日の日付はそのまま)、MAC アドレス、シリアル番号(<em>S/N</em> の後)。エージェントがほぼ毎回送るので、自分でオンにするまでオフのもの:ホームフォルダのユーザー名、ユーザー名とコンピュータ名(<code>user@host</code>)、パブリック IP アドレス、クラウドストレージのバケット。</td></tr><tr><td><span class="ui">マスクする語</span></td><td>入れた語そのままを <code>{{TERM_…}}</code> に。</td></tr>- <tr><td><span class="ui">ローカルモデルで検出</span></td><td><span class="ui">検出方法</span> の中にあります。このコンピュータで動くモデルが、ルールでは見つからない自由文中の名前や住所を見つけます。<span class="ui">シークレットをマスク</span> か <span class="ui">個人情報をマスク</span> がオンのときに使え、見つけたものは同じスイッチに従います。横のメニューでモデルを選べます:AstrLink Guard(中国語と英語、39 MB のダウンロード、コードでの誤検出が少ない。既定)か OpenAI の Privacy Filter(1.6 GB、GPU を使える、コード中の普通の値も多く隠す)。</td></tr>+ <tr><td><span class="ui">ローカルモデルで検出</span></td><td><span class="ui">検出方法</span> の中にあります。このコンピュータで動くモデルが、ルールでは見つからない自由文中の名前や住所を見つけます。<span class="ui">シークレットをマスク</span> か <span class="ui">個人情報をマスク</span> がオンのときに使え、見つけたものは同じスイッチに従います。横のメニューでモデルを選べます:AstrLink Guard(中国語と英語、39 MB のダウンロード、コードでの誤検出が少ない、Mac では Neural Engine で動く。既定)か OpenAI の Privacy Filter(1.6 GB、GPU を使える、コード中の普通の値も多く隠す)。</td></tr></table></div><p>プレースホルダーは「種類 + 鍵付きハッシュの 8 文字」です。同じ値はいつも同じプレースホルダーになるので、会話は毎ターン同じに読め、ベンダーのプロンプトキャッシュも有効なままです。値はメモリにだけ置かれます。</p>diff --git a/site/public/docs/masking.html b/site/public/docs/masking.htmlindex aa0ccdb9d..120fce544 100644--- a/site/public/docs/masking.html+++ b/site/public/docs/masking.html@@ -113,7 +113,7 @@<tr><td><span class="ui">Mask secrets</span></td><td>Private key blocks (<code>-----BEGIN … PRIVATE KEY-----</code>). API keys by their shape: <code>sk-…</code> (OpenAI, Anthropic, OpenRouter), <code>ghp_</code>/<code>github_pat_</code>, <code>glpat-</code>, <code>AIza…</code>, <code>xox…-</code>, <code>sk_live_</code>, <code>AKIA</code>/<code>ASIA</code>, <code>hf_</code>, <code>gsk_</code>, <code>xai-</code>, <code>npm_</code>, <code>pypi-</code>, <code>dop_v1_</code>, <code>SG.</code>, and relay keys like <code>oc_sk_…</code>. JWTs. The password in <code>user:password@host</code>. The value of a field named for a secret, as <code>.env</code> files and configs have them: <code>DB_PASSWORD=…</code>, <code>api_key: …</code>, <code>client_secret</code>, <code>token</code>, <code>credential</code>. Your own <span class="ui">Masking rules</span>.</td></tr><tr><td><span class="ui">Mask personal data</span></td><td>Emails, mainland China mobile numbers, Chinese ID numbers and bank card numbers whose checksum adds up. Also on by default: US Social Security numbers, passport numbers (after <em>passport</em>, 护照 or パスポート), IBANs, birthdays (next to <em>birthday</em>, <em>DOB</em> or 生日, and over a year ago, so today’s date stays), MAC addresses and serial numbers (after <em>S/N</em>). Off until you turn them on, since agents send them in nearly every request: the user name in home folders, user and computer names (<code>user@host</code>), public IP addresses and cloud storage buckets.</td></tr><tr><td><span class="ui">Masked words</span></td><td>Each word you listed, exactly, as <code>{{TERM_…}}</code>.</td></tr>- <tr><td><span class="ui">Find with a local model</span></td><td>Under <span class="ui">How it’s found</span>. A model on this computer finds what no rule can: a name or an address in free text. It needs <span class="ui">Mask secrets</span> or <span class="ui">Mask personal data</span> on, and finds under the same switches. The menu beside it picks the model: AstrLink Guard (Chinese and English, a 39 MB download, few false alarms in code; the default) or OpenAI’s Privacy Filter (1.6 GB, uses the GPU, masks more ordinary values in code).</td></tr>+ <tr><td><span class="ui">Find with a local model</span></td><td>Under <span class="ui">How it’s found</span>. A model on this computer finds what no rule can: a name or an address in free text. It needs <span class="ui">Mask secrets</span> or <span class="ui">Mask personal data</span> on, and finds under the same switches. The menu beside it picks the model: AstrLink Guard (Chinese and English, a 39 MB download, few false alarms in code, on a Mac it runs on the Neural Engine; the default) or OpenAI’s Privacy Filter (1.6 GB, uses the GPU, masks more ordinary values in code).</td></tr></table></div><p>A placeholder is the kind plus eight characters of a keyed hash. The same value gets the same placeholder every time, so the conversation reads the same turn after turn and the vendor’s prompt cache stays valid. The values are held in memory only.</p>diff --git a/site/public/docs/zh/masking.html b/site/public/docs/zh/masking.htmlindex 547ba2e9b..8febf269e 100644--- a/site/public/docs/zh/masking.html+++ b/site/public/docs/zh/masking.html@@ -113,7 +113,7 @@<tr><td><span class="ui">脱敏密钥</span></td><td>私钥块(<code>-----BEGIN … PRIVATE KEY-----</code>)。按格式识别的 API Key:<code>sk-…</code>(OpenAI、Anthropic、OpenRouter)、<code>ghp_</code>/<code>github_pat_</code>、<code>glpat-</code>、<code>AIza…</code>、<code>xox…-</code>、<code>sk_live_</code>、<code>AKIA</code>/<code>ASIA</code>、<code>hf_</code>、<code>gsk_</code>、<code>xai-</code>、<code>npm_</code>、<code>pypi-</code>、<code>dop_v1_</code>、<code>SG.</code>,以及 <code>oc_sk_…</code> 这类中转站的 key。JWT。<code>user:password@host</code> 里的密码。<code>.env</code> 和配置文件里以密钥命名的字段的值:<code>DB_PASSWORD=…</code>、<code>api_key: …</code>、<code>client_secret</code>、<code>token</code>、<code>credential</code>。以及你的 <span class="ui">自定义脱敏规则</span>。</td></tr><tr><td><span class="ui">脱敏个人信息</span></td><td>邮箱、中国大陆手机号、校验位对得上的身份证号和银行卡号。默认也开着的:美国社会安全号、护照号(跟在 <em>passport</em>、护照或パスポート后面)、IBAN、生日(旁边有 <em>birthday</em>、<em>DOB</em> 或「生日」,且是一年以前的日期,所以今天的日期不动)、MAC 地址、序列号(跟在 <em>S/N</em> 后面)。默认关闭、需要自己打开的(Agent 几乎每个请求都会带):主目录里的用户名、用户名和主机名(<code>user@host</code>)、公网 IP 地址、云存储桶。</td></tr><tr><td><span class="ui">脱敏词</span></td><td>你填的每个词,精确匹配,换成 <code>{{TERM_…}}</code>。</td></tr>- <tr><td><span class="ui">用本地模型查找</span></td><td>在 <span class="ui">怎么找</span> 下面。一个在这台电脑上运行的模型能找出规则找不到的东西:自由文本里的人名、地址。需要先打开 <span class="ui">脱敏密钥</span> 或 <span class="ui">脱敏个人信息</span>,找到的内容按同样的开关处理。旁边的菜单选择模型:AstrLink Guard(中英文,下载 39 MB,代码里很少误判;默认)或 OpenAI 的 Privacy Filter(1.6 GB,可用 GPU,代码里会遮盖更多普通值)。</td></tr>+ <tr><td><span class="ui">用本地模型查找</span></td><td>在 <span class="ui">怎么找</span> 下面。一个在这台电脑上运行的模型能找出规则找不到的东西:自由文本里的人名、地址。需要先打开 <span class="ui">脱敏密钥</span> 或 <span class="ui">脱敏个人信息</span>,找到的内容按同样的开关处理。旁边的菜单选择模型:AstrLink Guard(中英文,下载 39 MB,代码里很少误判,在 Mac 上用神经网络引擎运行;默认)或 OpenAI 的 Privacy Filter(1.6 GB,可用 GPU,代码里会遮盖更多普通值)。</td></tr></table></div><p>占位符是「类型 + 8 位带密钥的哈希」。同一个值每次得到同一个占位符,所以对话每一轮读起来都一样,厂商的提示词缓存也不会失效。原值只保存在内存里。</p>