Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.
agents: a line magpie writes into a TOML, YAML or env config has a note above it saying so
yetone0c4e3455ee59
diff --git a/docs/subsystems/agent-wiring.md b/docs/subsystems/agent-wiring.mdindex 3da5d1e7b..7d914b025 100644--- a/docs/subsystems/agent-wiring.md+++ b/docs/subsystems/agent-wiring.md@@ -83,6 +83,7 @@ something else shows up as drift.- JSON/JSONC config edits and their previews reject malformed input rather than editing a partially parsed document. Object edits and VS Code's model-group array edits leave the original file untouched on a parse error; valid comments and trailing commas remain supported (`internal/edit`). An edit's result is checked as the apps read JSONC (`edit.ValidJSONC`: jsonc-parser with `allowTrailingComma`, as OpenCode, VS Code and Zed parse; a trailing comma after a value is fine, one with nothing before it, `{,}` or `},,`, is not), and an edit that would turn a file the app reads into one it can't is refused, nothing written (`checkJSONC`). A delete takes the last member's own trailing comma with it, an insert into an object of comments only adds no comma, and a lone comma an older magpie left (`"mcp": {,}` after the hub was toggled for OpenCode, chenxiang0010 on Discord) is dropped by the next edit of that file (`dropLoneCommas`).- A config that starts with a UTF-8 byte order mark (a Windows `settings.json` or `config.toml` saved by PowerShell, Notepad or a channel switcher, #1534) is read without it (`edit.Read`), and a write keeps it (`keepBOM` in `WriteAtomic`): the user's file keeps the encoding it was saved in. That covers every format `internal/edit` writes (JSON/JSONC and its arrays, TOML, YAML, env); Codex reads a TOML file through a BOM too, so magpie no longer refuses one. A file read some other way trims it with `edit.TrimBOM` (the Claude Code and CC Switch imports). A line an edit takes out of a CRLF file goes with its `\r\n`.- Codex's model picker tags each of its own models by where a pick on the Agents page sends it: via magpie while routed, joined, or failing over, and direct while it is only beside its sign-in on one of magpie's models (`beside`, not joined, failover off), where such a pick takes magpie out (#940) and the GUI asks first (`TestCodexBesideOwnModelsSayDirect`).+- A line magpie writes into an agent's TOML, YAML or dotenv config gets a comment line of its own just above it, `# magpie: <what it is>; <how to undo it>` (`edit.Mark`, set to `agent.lineNote` by the CLI's `run` in [`mark.go`](../../internal/agent/mark.go); the edit side in [`internal/edit/mark.go`](../../internal/edit/mark.go)). Jevik_AI on X deleted magpie's folders by hand, and Codex's `config.toml` kept `openai_base_url` on the gateway with nothing in it naming magpie: every turn was a 502. Only a one-line scalar that points at the gateway or names magpie's model list gets one, and only when the write adds or changes that line, or it had a note already; a line of the user's own that magpie leaves as it is gets none. The note is never on the value's line: Grok App's `providers.rs` reads the rest of a line as the value. Every write takes the notes out first and puts them back over the lines that still need one, so a note goes with its line and a Disconnect leaves none. A YAML note that would change what the file reads as isn't added (`sameYAMLValue`). JSON has no comments and gets none. Tests set no `Mark`, so their exact bytes stay; `TestMarkedLinesEveryAgent` connects every agent with notes on and checks each gateway line has one, a second sync writes nothing, and Disconnect leaves none; `TestCodexMarkedLines`, `internal/edit`'s `TestMark*`.- Codex signed in to ChatGPT stays beside its sign-in (`openai_base_url`) unless the user picks magpie API. magpie becomes its provider only while the Codex app holds the active account, which it sends nothing for: `/wham/usage` says it isn't allowed, and it has no credits and isn't a workspace still within its overage (`codexHeld`, read by `provider.CodexUsedUp`). Where the app goes by what usage can't show (an experiment's gate, a reserve, a Team plan's overage under the reserve experiment), it counts as held: a wrong move costs ChatGPT extras, a missed one every turn. A saved reading kept through a failed read stops saying held once its used-up window has started again; a hold with no window used up stays till a fresh read. The app then loses its ChatGPT state (durable threads, remote control), so a window at 100% alone, which credits get past, doesn't move it (`codex.out` marks the move, and Sync undoes it once the account has room). The same holds on one of Codex's own models joined beside the sign-in (MasterYi on Discord: the Codex app greyed Send for every model on a used-up account): Sync makes magpie the provider on that model as magpie serves it (`codexOwnViaMagpie`, `codex/<model>`), keeps the model picked in `codex.out_own`, and once the account has room puts Codex back joined on it; a model the user picked meanwhile stays theirs. With no ChatGPT account in magpie to serve it, Codex is left as it is (`TestCodexOwnModelGoesThroughMagpieWhileUsedUp`, `TestCodexOwnModelKeptOnChatGPT`). The account is read again at once when a reply says it may be out (a `usage_limit_reached` rest, or an `x-codex-*` window heard at 100%; `provider.CodexMayBeUsedUp`, at most every 20s, the active account only), not only on the five-minute look. A turn relayed to Codex's own sign-in, the one account on (`codexUpstreamOn` with no account, the gateway's `codexRelayedAccount`), is heard the same way: its `x-codex-*` windows and `codex.rate_limits` event kept as the sign-in's usage (`heardCodexLimits`), and a `usage_limit_reached` refusal reading it again. Before, only turns served through magpie's accounts were, so with one account magpie learned the account was out only at the next look, minutes on, and the Codex app kept Send greyed until it was restarted (unsplashz on Discord; `TestCodexUsedUpRelayedAsksAtOnce`, `TestCodexRelayedWindowsHeard`). A Codex started before magpie became its provider keeps sending its own model's turns to its built-in provider; while the sign-in is known out (`provider.CodexUsedUpKnown`) and the request signs as it, those go through routing as `codex/<model>` (`codexAccounts`), as a restarted Codex's do (`TestCodexHeldSignInServedWithoutRestart`). The Codex app greys Send for every model, magpie's too, while its `authMethod` is `chatgpt` and its rate limit says not allowed (its composer's gate in ChatGPT.app 26.1013), and reads `authMethod` from its app-server's `account/read` only as the app-server starts; with magpie the provider (`requires_openai_auth` false) that answers no account, so the gate lifts. So the gateway process restarts the app's own app-server each time magpie becomes Codex's provider for the used-up account (`Agent.AppOut`) and again when Codex is put back on the account after its reset, and likewise whenever what the app-server reads as it starts and magpie writes changes (`model_provider`, `openai_base_url`, `model_catalog_json`, the magpie table's `base_url`, and a hash of what the catalog file holds, `codexCatalogTag` in [`codex_app_route.go`](../../internal/agent/codex_app_route.go), read again only when its size or time changes; `codexAppRoute`, through the `provider.CodexAppRoute` hook): a Disconnect, a Connect, failover written or taken out, a provider or group renamed or a model added. Without the catalog in it the app kept the names and models it started with until it was restarted (lucis0116 on X). Before this an app-server started while Codex was connected kept sending to the gateway after Disconnect until the app was restarted (JiangLiuer222 on X). At the watcher's first look only a used-up account's app-servers are ended, since what the others read before magpie started isn't known. It runs on every system (`KeepCodexAppOnItsProvider`, every 3s; Ethereal499 and Isdavexxx on X, MasterYi on Discord, #540: the earlier once-only reopen ran on macOS from the GUI process only, so Windows, Linux, a GUI-less magpie and the switch back stayed greyed). It ends only app-servers on magpie's Codex home (`codexAppHomeIs`: the process's `CODEX_HOME`, else `HOME/.codex`, read by `proc.EnvOf`; Windows can't tell and counts every one), only once no Codex request is in flight through the gateway nor ended in the last 10s (`Server.CodexBusy`) and no session under the Codex home has a turn started and not ended (`codexTurnOpen`), and with a clean exit (`proc.End`: SIGTERM, `TerminateProcess` with code 0), since the app shows any other exit as a crash. The app starts another app-server at once and keeps its window, threads and drafts (tried in the real app in a sandbox `CODEX_HOME`, both ways). An app-server that outlives three ends, or may not be ended, leaves the row saying to quit and reopen the app (`CodexAppStuck`). Until the reset the app's ChatGPT side is off (cloud tasks, its usage meter, Log out); the Agents row says so with the reset's time (`appOut` in `/api/state`, `CodexOutUntil`), and it all comes back at the reset. While the active account is held or a window is at 90% or more, its usage is read every minute rather than every five (`codexLookEvery`). `TestCodexAppServerRestartedEachWay`, `TestCodexAppServerRestartedOnDisconnect`, `TestCodexAppRouteFollowsConnectAndDisconnect`, `TestCodexAppRouteFollowsTheCatalog`, `TestCodexAppServerUnknown`, `TestCodexAppServerStuck`, `TestCodexAppServersOnHome`, `TestCodexTurnOpen`, `TestCodexBusy`, `codex-app-out.test.cjs`.- Moving a field from one of magpie's models to another (the agent's own picker) is not drift. Moving it off magpie is.- Claude Desktop applying a configLibrary profile other than magpie's is not drift when that profile is a gateway profile for this magpie's gateway (`desktopAppliedGateway` in [`claudedesktop.go`](../../internal/agent/claudedesktop.go)): Desktop's "Import configuration" adds a file exported on another computer, magpie's profile there with the user's connectors (`managedMcpServers`), as a new entry under a random id (aa86222 on Discord). magpie keeps it applied on Reconnect and through Sync, and writes in it only `inferenceGatewayApiKey` (with a missing auth scheme) when the gateway would refuse its key (`desktopKeyApplied`; magpie's own keys count as taken, `desktopKeyTaken`). A profile for another gateway, CC Switch's among them, is still drift, and Reconnect applies magpie's. The first switch-on still starts magpie's profile from an applied one (#2035). `TestClaudeDesktopImportedProfileIsOnMagpie`.diff --git a/internal/agent/dsh.go b/internal/agent/dsh.goindex e77551d29..115abb62a 100644--- a/internal/agent/dsh.go+++ b/internal/agent/dsh.go@@ -451,6 +451,11 @@ func dshParse(path, raw string) (head []string, items []dshItem, err error) {}opened := false // a "---" or "[]" came before the first entryfor _, l := range splitLinesKeep(raw) {+ // magpie's notes are left out, and dshWrite puts them back: a+ // route is compared with the lines magpie would write+ if edit.IsNote(l) {+ continue+ }t := strings.TrimSpace(l)switch {case strings.HasPrefix(l, "- ") || l == "-":@@ -515,6 +520,9 @@ func dshWrite(path string, head []string, items []dshItem) error {}out = append(out, "[]") // dsh wants a list, even an empty one}+ // a route on the gateway says so (edit.Mark)+ orig, _ := edit.Read(path)+ out = edit.MarkYAMLLines(out, string(orig))return edit.WriteAtomic(path, []byte(strings.Join(out, "\n")+"\n"))}diff --git a/internal/agent/mark.go b/internal/agent/mark.gonew file mode 100644index 000000000..78b2beccc--- /dev/null+++ b/internal/agent/mark.go@@ -0,0 +1,56 @@+package agent++import (+ "net/url"+ "strings"++ "github.com/yetone/magpie/internal/edit"+ "github.com/yetone/magpie/internal/gateway"+)++// MarkWritten has every TOML, YAML and dotenv line magpie writes that+// points an agent at it carry a comment saying so and how to undo it+// (lineNote). A Codex left with openai_base_url on a gateway that no+// longer ran, after magpie's folders were deleted by hand, failed every+// turn with nothing in its config naming magpie (Jevik_AI on X). Set by+// main; tests of an agent's exact bytes run without it.+func MarkWritten() { edit.Mark = lineNote }++const undo = "Disconnect in magpie or `magpie uninstall` undoes it"++// lineNote is the note a config line gets, "" for one that isn't+// magpie's: a URL on the gateway's port, and Codex's own keys that name+// magpie.+func lineNote(table, key, value string) string {+ switch {+ case table == "" && key == "model_provider" && value == magpieID:+ return "Codex on magpie's local gateway; " + undo + ", or delete this line and the model line"+ case table == "" && key == "model_catalog_json" && strings.Contains(value, "magpie"):+ return "magpie's model list for Codex; " + undo + ", or delete this line"+ case !onGateway(value):+ return ""+ case table == "model_providers."+magpieID:+ return "for Codex threads started on magpie; unused while model_provider doesn't name magpie"+ case table == "":+ return "points at magpie's local gateway; " + undo + ", or delete this line"+ }+ return "points at magpie's local gateway; " + undo + ", or remove this entry and pick another model"+}++// onGateway says whether v is a URL on the gateway's port: on loopback, or+// on the address a WSL distro reaches the host by.+func onGateway(v string) bool {+ if !strings.HasPrefix(v, "http://") && !strings.HasPrefix(v, "https://") {+ return false+ }+ u, err := url.Parse(v)+ if err != nil || u.Port() == "" {+ return false+ }+ for _, base := range []string{gateway.URL(), gateway.SavedURL()} {+ if b, err := url.Parse(base); err == nil && b.Port() == u.Port() {+ return true+ }+ }+ return false+}diff --git a/internal/agent/mark_test.go b/internal/agent/mark_test.gonew file mode 100644index 000000000..08424843c--- /dev/null+++ b/internal/agent/mark_test.go@@ -0,0 +1,153 @@+package agent++import (+ "os"+ "path/filepath"+ "slices"+ "strings"+ "testing"+ "time"++ "github.com/yetone/magpie/internal/edit"+ "github.com/yetone/magpie/internal/gateway"+)++// commentable says whether an agent reads path as TOML, YAML or dotenv,+// where a line can carry a comment. JSON can't, and gets none.+func commentable(path string) bool {+ switch strings.ToLower(filepath.Ext(path)) {+ case ".toml", ".yaml", ".yml", ".env":+ return true+ }+ return filepath.Base(path) == ".env"+}++// rewrittenEachSync are the agents whose config every sync writes again+// unchanged, notes or none (#3870)+var rewrittenEachSync = map[string]bool{"kimi": true, "grok": true, "atomcode": true}++func modTimes(home string) map[string]time.Time {+ m := map[string]time.Time{}+ filepath.WalkDir(home, func(p string, e os.DirEntry, err error) error {+ if err == nil && !e.IsDir() {+ if fi, err := e.Info(); err == nil {+ rel, _ := filepath.Rel(home, p)+ m[rel] = fi.ModTime()+ }+ }+ return nil+ })+ return m+}++// Every agent magpie writes into a TOML, YAML or dotenv config gets a+// comment line over each line that points at the gateway, saying magpie+// wrote it and how to undo it: Jevik_AI on X deleted magpie's folders by hand and+// was left with a Codex config naming a gateway that no longer ran, and+// nothing in it naming magpie. Disconnect takes each note with its line.+func TestMarkedLinesEveryAgent(t *testing.T) {+ home, _ := codexHome(t, "", "")+ edit.Mark = lineNote+ t.Cleanup(func() { edit.Mark = nil })+ managed := claudeManaged+ claudeManaged = func() string { return filepath.Join(home, "managed-settings.json") }+ t.Cleanup(func() { claudeManaged = managed })+ startAlma(t)+ host := strings.TrimPrefix(gateway.URL(), "http://")+ var marked []string+ for _, a := range All() {+ if a.Launch != nil || a.Native != nil || a.WSL != "" || len(a.Fields) == 0 {+ continue+ }+ t.Run(a.ID, func(t *testing.T) {+ if _, err := a.ConnectHow(); err != nil {+ t.Skip("can't be connected here:", err)+ }+ if !a.Wired() {+ t.Skip("not wired by Connect here")+ }+ // a note is read back as written: a sync doesn't write the+ // file again and again, as one that compared a line with the+ // one it would write did (dsh's routes)+ SyncCatalog()+ dshWiredOnce()+ synced, when := agentFiles(t, home), modTimes(home)+ SyncCatalog()+ dshWiredOnce()+ later := modTimes(home)+ for p, b := range agentFiles(t, home) {+ if synced[p] != b {+ t.Errorf("a second sync changed %s:\n--- first\n%s\n--- second\n%s", p, synced[p], b)+ } else if commentable(p) && !when[p].Equal(later[p]) && !rewrittenEachSync[a.ID] {+ t.Errorf("a second sync wrote %s again", p)+ }+ }+ for p, b := range agentFiles(t, home) {+ if !commentable(p) {+ continue+ }+ lines := strings.Split(b, "\n")+ for i, l := range lines {+ if strings.Contains(l, "# "+edit.MarkTag) {+ marked = append(marked, a.ID)+ // a line of its own: Grok App reads the rest of+ // a line as its value+ if !edit.IsNote(l) {+ t.Errorf("%s: a note beside a value: %q", p, l)+ }+ }+ above := i > 0 && edit.IsNote(lines[i-1])+ // a scalar on its own line; an inline table or a list+ // can't carry a note of its own+ if !strings.Contains(l, host) || edit.IsNote(l) || above || strings.HasPrefix(strings.TrimSpace(l), "- ") || strings.Contains(l, "= {") || strings.Contains(l, "= [") {+ continue+ }+ t.Errorf("%s: a line on the gateway without a note: %q", p, l)+ }+ }+ if err := a.Disconnect(); err != nil {+ t.Fatal(err)+ }+ for p, b := range agentFiles(t, home) {+ for _, l := range strings.Split(b, "\n") {+ if strings.Contains(l, "# "+edit.MarkTag) && !strings.Contains(l, "Codex threads started on magpie") {+ t.Errorf("%s: a note outlived Disconnect: %q", p, l)+ }+ }+ }+ })+ }+ if !slices.Contains(marked, "codex") {+ t.Fatal("Codex's config.toml got no note")+ }+}++// Codex's own keys that name magpie get a note on how to undo them by+// hand, not only the base URL.+func TestCodexMarkedLines(t *testing.T) {+ codexHome(t, "", "model = \"gpt-5.5\"\n")+ edit.Mark = lineNote+ t.Cleanup(func() { edit.Mark = nil })+ a, _ := Find("codex")+ if _, err := a.ConnectHow(); err != nil {+ t.Fatal(err)+ }+ b, _ := os.ReadFile(a.Path)+ got := string(b)+ for _, want := range []string{+ "# magpie: Codex on magpie's local gateway; Disconnect in magpie or `magpie uninstall` undoes it, or delete this line and the model line\nmodel_provider = \"magpie\"\n",+ "# magpie: magpie's model list for Codex; " + undo + ", or delete this line\nmodel_catalog_json = ",+ "# magpie: for Codex threads started on magpie; unused while model_provider doesn't name magpie\nbase_url = \"" + gateway.URL() + "/v1\"\n",+ } {+ if !strings.Contains(got, want) {+ t.Errorf("missing %q in:\n%s", want, got)+ }+ }+ if err := a.Disconnect(); err != nil {+ t.Fatal(err)+ }+ b, _ = os.ReadFile(a.Path)+ if !strings.HasPrefix(string(b), "model = \"gpt-5.5\"\n") {+ t.Fatalf("after Disconnect:\n%s", b)+ }+}diff --git a/internal/cli/main.go b/internal/cli/main.goindex 07943348d..3dfa7585c 100644--- a/internal/cli/main.go+++ b/internal/cli/main.go@@ -342,6 +342,9 @@ func run(args []string) error {// a Codex that reaches magpie for account failover alone is handed// only its own models (#1385)gateway.CodexOwnOnly = agent.CodexOwnOnly+ // what magpie writes into an agent's TOML, YAML or dotenv config says+ // so in a comment, for whoever reads it once magpie is gone+ agent.MarkWritten()// the Codex app's titles, descriptions and summaries, which it asks// for on a model of its own, on the model of magpie's picked beside// its ChatGPT sign-in rather than on that account (atom_Ray on X)diff --git a/internal/edit/env.go b/internal/edit/env.goindex 0f8c47836..1a74e89a7 100644--- a/internal/edit/env.go+++ b/internal/edit/env.go@@ -58,6 +58,7 @@ func SetEnvFile(path string, kvs ...KV) error {return "", false})}+ lines = markEnv(lines, string(raw))return WriteAtomic(path, []byte(joinLinesLike(lines, string(raw))))}@@ -84,7 +85,9 @@ func DelEnvFile(path string, keys ...string) error {if !changed {return nil}- if strings.TrimSpace(strings.Join(out, "")) == "" {+ // a note magpie put above a key goes with it+ out = markEnv(out, string(raw))+ if strings.TrimSpace(strings.Join(withoutNotes(out), "")) == "" {return Remove(path)}return WriteAtomic(path, []byte(joinLines(out)))diff --git a/internal/edit/mark.go b/internal/edit/mark.gonew file mode 100644index 000000000..1cc5f1f32--- /dev/null+++ b/internal/edit/mark.go@@ -0,0 +1,213 @@+package edit++import (+ "strings"++ "gopkg.in/yaml.v3"+)++// Mark, when set, is the note a line magpie writes into an agent's TOML,+// YAML or dotenv config gets, as a comment line of its own just above it:+// what wrote it and how to undo it, for someone reading the file after+// magpie is gone (Jevik_AI on X: a Codex config left pointing at a gateway+// that no longer ran). table is the TOML table or the YAML mapping's+// dotted path the key is in, "" at the top level; value is the value as+// read. "" leaves the line without a note. JSON has no comments, so+// nothing is marked there. Set by main, so a test of an agent's exact+// bytes sees none.+//+// A note goes on a line this write adds or changes, and stays on one that+// had it; a line of the user's own that magpie leaves as it is gets none.+// It is a line of its own, never a comment after the value: an agent that+// reads its config line by line (Grok App's providers.rs) takes the rest+// of a line for the value, but skips a line that starts with #.+var Mark func(table, key, value string) string++// MarkTag starts every note: a comment line that starts with it is+// magpie's, and is written again, or taken out, with the line under it.+const MarkTag = "magpie: "++const markComment = "# " + MarkTag++func noteOf(table, key, value string) string {+ if Mark == nil {+ return ""+ }+ return Mark(table, key, value)+}++// IsNote says whether line is one of magpie's notes.+func IsNote(line string) bool {+ return strings.HasPrefix(strings.TrimSpace(line), markComment)+}++// withoutNotes is lines without magpie's notes.+func withoutNotes(lines []string) []string {+ var out []string+ for _, l := range lines {+ if !IsNote(l) {+ out = append(out, l)+ }+ }+ if len(out) == len(lines) {+ return lines+ }+ return out+}++// noted is, for each line of orig, whether a note of magpie's was over it+func noted(orig string) map[string]bool {+ m := map[string]bool{}+ prev := false+ for _, l := range splitLines(orig) {+ if IsNote(l) {+ prev = true+ continue+ }+ k := strings.TrimSuffix(l, "\r")+ m[k] = m[k] || prev+ prev = false+ }+ return m+}++// withNotes is lines (notes taken out) with note[i] over line i where the+// line is new or changed from orig, or had a note there.+func withNotes(lines []string, orig string, note map[int]string) []string {+ if len(note) == 0 {+ return lines+ }+ was := noted(orig)+ out := make([]string, 0, len(lines)+len(note))+ for i, l := range lines {+ k := strings.TrimSuffix(l, "\r")+ if n := note[i]; n != "" {+ if had, there := was[k]; had || !there {+ indent := k[:len(k)-len(strings.TrimLeft(k, " \t"))]+ nl := indent + markComment + n+ if strings.HasSuffix(l, "\r") {+ nl += "\r"+ }+ out = append(out, nl)+ }+ }+ out = append(out, l)+ }+ return out+}++// markTOML puts Mark's notes over lines' one-line scalar keys. A file that+// doesn't parse is left as it is, for writeTOML to refuse.+func markTOML(lines []string, orig string) []string {+ if Mark == nil {+ return lines+ }+ bare := withoutNotes(lines)+ doc, err := parseTOMLFile(bare)+ if err != nil {+ return lines+ }+ note := map[int]string{}+ mark := func(table string, keys []tomlKeySpan) {+ for _, k := range keys {+ if k.scalar && k.to == k.from+1 {+ if n := noteOf(table, k.name, k.value); n != "" {+ note[k.from] = n+ }+ }+ }+ }+ mark("", doc.root.keys)+ for _, t := range doc.tables {+ mark(t.name, t.keys)+ }+ return withNotes(bare, orig, note)+}++// MarkYAMLLines puts Mark's notes in a YAML file's lines, orig being the+// file as it was: over a key whose scalar value is on the key's own line,+// in a block mapping, at the line's indent. Should a note change what the+// file reads as, none is added.+func MarkYAMLLines(lines []string, orig string) []string {+ if Mark == nil {+ return lines+ }+ bare := withoutNotes(lines)+ root, err := parseYAMLDocument([]byte(strings.Join(bare, "\n")))+ if err != nil || root == nil {+ return lines+ }+ note := map[int]string{}+ var walk func(n *yaml.Node, table string)+ walk = func(n *yaml.Node, table string) {+ switch n.Kind {+ case yaml.DocumentNode, yaml.SequenceNode:+ for _, c := range n.Content {+ walk(c, table)+ }+ case yaml.MappingNode:+ if n.Style&yaml.FlowStyle != 0 {+ return+ }+ for i := 0; i+1 < len(n.Content); i += 2 {+ k, v := n.Content[i], n.Content[i+1]+ if v.Kind != yaml.ScalarNode {+ walk(v, joinPath(table, k.Value))+ continue+ }+ at := k.Line - 1+ if v.Line != k.Line || v.Style&(yaml.LiteralStyle|yaml.FoldedStyle) != 0 || at < 0 || at >= len(bare) {+ continue+ }+ if s := noteOf(table, k.Value, v.Value); s != "" {+ note[at] = s+ }+ }+ }+ }+ walk(root, "")+ out := withNotes(bare, orig, note)+ if len(out) == len(lines) && strings.Join(out, "\n") == strings.Join(lines, "\n") {+ return lines+ }+ again, err := parseYAMLDocument([]byte(strings.Join(out, "\n")))+ if err != nil || again == nil || !sameYAMLValue(root, again) {+ return bare+ }+ return out+}++// markEnv puts Mark's notes over a dotenv file's keys. Dotenv readers end+// a value at an unquoted #, but some don't, so a note never shares the+// value's line.+func markEnv(lines []string, orig string) []string {+ if Mark == nil {+ return lines+ }+ bare := withoutNotes(lines)+ note := map[int]string{}+ for i, l := range bare {+ m := envLine.FindStringSubmatch(l)+ if m == nil {+ continue+ }+ v, ok := unquote(m[2])+ if !ok {+ v = m[2]+ if j := strings.IndexByte(v, '#'); j >= 0 {+ v = strings.TrimSpace(v[:j])+ }+ }+ if n := noteOf("", m[1], v); n != "" {+ note[i] = n+ }+ }+ return withNotes(bare, orig, note)+}++func joinPath(table, key string) string {+ if table == "" {+ return key+ }+ return table + "." + key+}diff --git a/internal/edit/mark_test.go b/internal/edit/mark_test.gonew file mode 100644index 000000000..17dcc3771--- /dev/null+++ b/internal/edit/mark_test.go@@ -0,0 +1,227 @@+package edit++import (+ "os"+ "strings"+ "testing"+)++// marking says "N" for any value on port 3425, as agent.lineNote does for+// the gateway's+func marking(t *testing.T) {+ t.Helper()+ Mark = func(table, key, value string) string {+ if strings.Contains(value, ":3425") {+ if table != "" {+ return "T " + table+ }+ return "N"+ }+ return ""+ }+ t.Cleanup(func() { Mark = nil })+}++func TestMarkTOML(t *testing.T) {+ marking(t)+ p := tmpFile(t, "config.toml", "model = \"gpt-5\"\nmine = \"http://127.0.0.1:3425/x\" # my own\n")+ if err := SetTOMLTop(p, KV{Path: "openai_base_url", Value: "http://127.0.0.1:3425/backend-api/codex"}); err != nil {+ t.Fatal(err)+ }+ if err := SetTOMLTable(p, "model_providers.magpie", KV{Path: "base_url", Value: "http://127.0.0.1:3425/v1"}, KV{Path: "name", Value: "magpie"}); err != nil {+ t.Fatal(err)+ }+ got := read(t, p)+ for _, want := range []string{+ "# magpie: N\nopenai_base_url = \"http://127.0.0.1:3425/backend-api/codex\"\n",+ "# magpie: T model_providers.magpie\nbase_url = \"http://127.0.0.1:3425/v1\"\n",+ "\nname = \"magpie\"\n",+ "model = \"gpt-5\"\n",+ // a line of the user's own, which magpie didn't write, gets none,+ // and their comment stays theirs+ "\"gpt-5\"\nmine = \"http://127.0.0.1:3425/x\" # my own\n",+ } {+ if !strings.Contains(got, want) {+ t.Fatalf("missing %q in:\n%s", want, got)+ }+ }+ // written again: one note, not two+ if err := SetTOMLTopPreserving(p, KV{Path: "openai_base_url", Value: "http://127.0.0.1:3425/backend-api/codex"}); err != nil {+ t.Fatal(err)+ }+ if n := strings.Count(read(t, p), "# magpie: N"); n != 1 {+ t.Fatalf("%d notes on openai_base_url:\n%s", n, read(t, p))+ }+ // a value of the user's put in its place, its spacing kept: the note goes+ if err := SetTOMLTopPreserving(p, KV{Path: "openai_base_url", Value: "https://proxy.example/v1"}); err != nil {+ t.Fatal(err)+ }+ if got := read(t, p); !strings.Contains(got, "openai_base_url = \"https://proxy.example/v1\"\n") {+ t.Fatalf("note kept on the user's value:\n%s", got)+ }+ // taken out, with its note+ if err := DelTOMLKey(p, "model_providers.magpie", "base_url"); err != nil {+ t.Fatal(err)+ }+ if got := read(t, p); strings.Contains(got, "T model_providers") {+ t.Fatalf("note outlived its key:\n%s", got)+ }+ if err := validateTOML([]byte(read(t, p))); err != nil {+ t.Fatal(err)+ }+}++func TestMarkTOMLKeepsCRLF(t *testing.T) {+ marking(t)+ p := tmpFile(t, "config.toml", "model = \"gpt-5\"\r\n")+ if err := SetTOMLTop(p, KV{Path: "openai_base_url", Value: "http://127.0.0.1:3425/x"}); err != nil {+ t.Fatal(err)+ }+ if got := read(t, p); got != "model = \"gpt-5\"\r\n# magpie: N\r\nopenai_base_url = \"http://127.0.0.1:3425/x\"\r\n" {+ t.Fatalf("got %q", got)+ }+}++func TestMarkTOMLOff(t *testing.T) {+ p := tmpFile(t, "config.toml", "")+ if err := SetTOMLTop(p, KV{Path: "openai_base_url", Value: "http://127.0.0.1:3425/x"}); err != nil {+ t.Fatal(err)+ }+ if got := read(t, p); strings.Contains(got, "#") {+ t.Fatalf("a note with Mark unset: %q", got)+ }+}++func TestMarkYAML(t *testing.T) {+ marking(t)+ p := tmpFile(t, "config.yaml", "model:\n default: gpt-5\n")+ if err := SetYAML(p, KV{Path: "model.base_url", Value: "http://127.0.0.1:3425/v1"}); err != nil {+ t.Fatal(err)+ }+ got := read(t, p)+ if !strings.Contains(got, " # magpie: T model\n base_url: http://127.0.0.1:3425/v1\n") {+ t.Fatalf("got:\n%s", got)+ }+ if err := SetYAML(p, KV{Path: "model.base_url", Value: "https://api.example/v1"}); err != nil {+ t.Fatal(err)+ }+ if got := read(t, p); strings.Contains(got, "magpie:") {+ t.Fatalf("note kept on the user's value:\n%s", got)+ }+}++func TestMarkYAMLTop(t *testing.T) {+ marking(t)+ p := tmpFile(t, "config.yaml", "GOOSE_MODEL: gpt-5 # mine\n")+ if err := SetYAMLTop(p, KV{Path: "OPENAI_HOST", Value: "http://127.0.0.1:3425"}); err != nil {+ t.Fatal(err)+ }+ got := read(t, p)+ if got != "GOOSE_MODEL: gpt-5 # mine\n# magpie: N\nOPENAI_HOST: \"http://127.0.0.1:3425\"\n" {+ t.Fatalf("got %q", got)+ }+ if v, _ := GetYAMLTop(p, "OPENAI_HOST"); v != "http://127.0.0.1:3425" {+ t.Fatalf("read back %q", v)+ }+ if err := SetYAMLTop(p, KV{Path: "OPENAI_HOST", Value: "http://127.0.0.1:3425"}); err != nil {+ t.Fatal(err)+ }+ if n := strings.Count(read(t, p), "magpie: N"); n != 1 {+ t.Fatalf("%d notes:\n%s", n, read(t, p))+ }+ if err := DelYAMLTop(p, "OPENAI_HOST"); err != nil {+ t.Fatal(err)+ }+ if got := read(t, p); got != "GOOSE_MODEL: gpt-5 # mine\n" {+ t.Fatalf("got %q", got)+ }+}++func TestMarkEnv(t *testing.T) {+ marking(t)+ p := tmpFile(t, ".env", "# my keys\nGEMINI_API_KEY=abc\n")+ if err := SetEnvFile(p, KV{Path: "GOOGLE_GEMINI_BASE_URL", Value: "http://127.0.0.1:3425"}); err != nil {+ t.Fatal(err)+ }+ if got := read(t, p); got != "# my keys\nGEMINI_API_KEY=abc\n# magpie: N\nGOOGLE_GEMINI_BASE_URL=http://127.0.0.1:3425\n" {+ t.Fatalf("got %q", got)+ }+ if v, _ := GetEnvFile(p, "GOOGLE_GEMINI_BASE_URL"); v != "http://127.0.0.1:3425" {+ t.Fatalf("read back %q", v)+ }+ if err := SetEnvFile(p, KV{Path: "GOOGLE_GEMINI_BASE_URL", Value: "http://127.0.0.1:3425"}); err != nil {+ t.Fatal(err)+ }+ if n := strings.Count(read(t, p), "magpie: N"); n != 1 {+ t.Fatalf("%d notes:\n%s", n, read(t, p))+ }+ // the user's comment above their own key is never taken+ if err := SetEnvFile(p, KV{Path: "GEMINI_API_KEY", Value: "def"}); err != nil {+ t.Fatal(err)+ }+ if !strings.HasPrefix(read(t, p), "# my keys\nGEMINI_API_KEY=def\n") {+ t.Fatalf("got %q", read(t, p))+ }+ if err := DelEnvFile(p, "GOOGLE_GEMINI_BASE_URL"); err != nil {+ t.Fatal(err)+ }+ if got := read(t, p); got != "# my keys\nGEMINI_API_KEY=def\n" {+ t.Fatalf("got %q", got)+ }+}++// a .env magpie made for its key alone still goes when the key does,+// its note with it+func TestMarkEnvFileGoesWhole(t *testing.T) {+ marking(t)+ p := tmpFile(t, ".env", "")+ if err := SetEnvFile(p, KV{Path: "GOOGLE_GEMINI_BASE_URL", Value: "http://127.0.0.1:3425"}); err != nil {+ t.Fatal(err)+ }+ if err := DelEnvFile(p, "GOOGLE_GEMINI_BASE_URL"); err != nil {+ t.Fatal(err)+ }+ if _, err := os.Stat(p); !os.IsNotExist(err) {+ t.Fatalf("file left: %q", read(t, p))+ }+}++// A line magpie leaves as the user had it gets no note, even on the+// gateway, while one magpie writes again keeps its own.+func TestMarkOnlyWhatMagpieWrites(t *testing.T) {+ marking(t)+ p := tmpFile(t, "config.toml", "model_provider = \"x\"\n[model_providers.x]\nbase_url = \"http://127.0.0.1:3425/v1\"\n")+ if err := SetTOMLTop(p, KV{Path: "model", Value: "gpt-5"}); err != nil {+ t.Fatal(err)+ }+ if got := read(t, p); strings.Contains(got, "magpie:") {+ t.Fatalf("a note on the user's own line:\n%s", got)+ }+ if err := SetTOMLTop(p, KV{Path: "openai_base_url", Value: "http://127.0.0.1:3425/x"}); err != nil {+ t.Fatal(err)+ }+ if err := SetTOMLTop(p, KV{Path: "model", Value: "gpt-5.5"}); err != nil {+ t.Fatal(err)+ }+ if got := read(t, p); strings.Count(got, "# magpie: ") != 1 || !strings.Contains(got, "# magpie: N\nopenai_base_url") {+ t.Fatalf("got:\n%s", got)+ }+}++// A note is a line of its own: a reader that takes the rest of a line for+// its value, as Grok App's does, reads the value as written.+func TestMarkNeverSharesTheValuesLine(t *testing.T) {+ marking(t)+ p := tmpFile(t, "config.toml", "")+ if err := SetTOMLTable(p, "providers.magpie", KV{Path: "base_url", Value: "http://127.0.0.1:3425/v1"}); err != nil {+ t.Fatal(err)+ }+ for _, l := range strings.Split(read(t, p), "\n") {+ if strings.Contains(l, "3425") && strings.Contains(l, "#") {+ t.Fatalf("a note beside the value: %q", l)+ }+ }+ if !strings.Contains(read(t, p), "# magpie: T providers.magpie\nbase_url") {+ t.Fatalf("got:\n%s", read(t, p))+ }+}diff --git a/internal/edit/toml.go b/internal/edit/toml.goindex 370b44fbf..075abdd0a 100644--- a/internal/edit/toml.go+++ b/internal/edit/toml.go@@ -648,6 +648,7 @@ func tomlParseError(p *unstable.Parser, err error) error {// the document, so comments and formatting survive validation unchanged.func writeTOML(path string, lines []string) error {orig, _ := Read(path)+ lines = markTOML(lines, string(orig))data := []byte(joinLinesLike(lines, string(orig)))if err := validateTOML(data); err != nil {return fmt.Errorf("%s: edited TOML is invalid: %w", path, err)diff --git a/internal/edit/yaml.go b/internal/edit/yaml.goindex a0a0ff33d..a794a8e84 100644--- a/internal/edit/yaml.go+++ b/internal/edit/yaml.go@@ -392,7 +392,12 @@ func writeYAML(path string, root *yaml.Node) error {return fmt.Errorf("%s: %w", path, err)}orig, _ := Read(path)- return WriteAtomic(path, keepCRLF(buf.Bytes(), orig))+ out := buf.Bytes()+ if Mark != nil {+ lines := splitLines(string(out))+ out = []byte(strings.Join(MarkYAMLLines(lines, string(orig)), "\n"))+ }+ return WriteAtomic(path, keepCRLF(out, orig))}func lookupYAML(n *yaml.Node, parts []string) *yaml.Node {diff --git a/internal/edit/yaml_top.go b/internal/edit/yaml_top.goindex 64c521e82..3d4715f86 100644--- a/internal/edit/yaml_top.go+++ b/internal/edit/yaml_top.go@@ -218,6 +218,7 @@ func sameYAMLValue(a, b *yaml.Node) bool {func writeYAMLTop(path string, lines []string) error {orig, _ := Read(path)+ lines = MarkYAMLLines(lines, string(orig))raw := []byte(joinLinesLike(lines, string(orig)))if _, _, err := parseYAMLTop(raw); err != nil {return fmt.Errorf("%s: %w", path, err)