Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.
usage: a deleted provider's requests merged into another, or named, from the Requests filter
yetone396e0e463805
diff --git a/docs/subsystems/providers-accounts.md b/docs/subsystems/providers-accounts.mdindex 1184951c2..5ce0c8a98 100644--- a/docs/subsystems/providers-accounts.md+++ b/docs/subsystems/providers-accounts.md@@ -24,7 +24,7 @@ keys and accounts per request is in [Gateway routing and fallback](gateway-routi| Extra body | `ExtraBody` (`extraBody` in providers.json, the editor's *Extra body*, #1604) is a JSON object whose fields `WithExtraBody` adds at the top level of every JSON body sent to the provider: Messages, Chat, Responses and Gemini, relayed as they came or translated, in the gateway's `forwardOnce` and in Test and Detect (`tested`). A field the body has already is kept; a name written `"!name"` replaces it. Values are put in as written, objects whole, not merged. `${sessionId}` in any string, at any depth, becomes the request's session: the agent's session header (`sessionOf`), else the id `conversationID` derives from the first user message (`magpie-` and its hash), so it stays the same through one conversation; Test and Detect send `magpie-test-` and 12 random letters, a new one each time. `model`, `stream` and the conversation (`messages`, `input`, `contents`, `system`, `systemInstruction`, `system_instruction`, `instructions`, `tools`, `ExtraBodyReserved`) are magpie's: Save refuses them with or without `!` (`ExtraBodyOK`), the editor says so as they are typed, and one written into providers.json by hand is dropped on read (`cleanExtraBody`). Not sent with it: `count_tokens`, embeddings, images, video and System One's decision model (other APIs), and a signed-in account's requests, except a ChatGPT (Codex) account's (magpie-releases#18): its requests are built as Codex CLI's (`codexBody`) and take the fields Codex CLI adds by choice, such as `access_programs` (OpenAI's Daybreak), while `store`, `tool_choice` and `include`, which the backend needs as `codexBody` sets them, can't be set (`codexOwned`). Its extraBody is written into providers.json by hand under the account's entry and kept onto the account by `withPicks`; the editor has no box for it. A client's own top-level `access_programs` on Responses or Chat is carried to a Responses build as it came (`Request.AccessPrograms`), so a request translated (a Codex account is streamed, so a whole Responses request is made again) keeps it as a relayed one does. It reaches only the upstreams Codex itself sends it to (`keepsAccessPrograms`, #3172, see [gateway routing](gateway-routing.md)); it is taken out before the extraBody merge, so an `access_programs` the user puts in a provider's extraBody still goes. One object goes to every API; fields for one API alone are not a thing yet. A `thinking` budget it puts in on an Anthropic endpoint is fitted to a Claude that thinks only adaptively after the merge, as an agent's is (`adaptiveThinking`, `TestExtraBodyBudgetFittedToAdaptiveModel`). | [`extrabody.go`](../../internal/provider/extrabody.go), [`gateway.go`](../../internal/gateway/gateway.go) || Native compaction | `NativeCompaction` (`nativeCompaction` in providers.json, the editor's *Compaction* row, `magpie provider set <id> compaction=native\|magpie`; #1099) has Codex's compaction asked of the provider as Codex sent it and its sealed item returned unchanged. Off by default. Only a provider whose wire carries it can be set so (`CarriesCompaction`: a Responses API, a ChatGPT account, GitHub Copilot; not another agent's sign-in or a plugin's), and `CompactsOn(model)` also needs the model asked on Responses. `SetNativeCompaction`, the CLI and the GUI's save refuse it elsewhere. An account keeps it through `Save`'s picks (`withPicks`), and `AddCopy` carries it. The gateway's half is in [gateway-routing](gateway-routing.md). | [`native_compaction.go`](../../internal/provider/native_compaction.go) || providers.json | `Path` is `providers.json` in magpie's config folder. A read that fails returns `ErrUnreadable` and is never treated as empty. `store` writes atomically through a rename and drops the cached catalog (`catalog.Touched`). | [`provider.go`](../../internal/provider/provider.go) |-| A provider renamed | `Rename` (the editor's id field, `magpie provider set <id> id=<new>`) keeps the old id in `Was`, so a model still picked by it reaches the provider. Usage is recorded under the id a call went out by and never rewritten; whatever reads it names the provider by its id now through `UsageRenames().Of(id, time)`: the summary, the Requests page (its blocks, the probe rows, other computers' days), the Sessions tab's vias, window holds, `NewPricer`/`NewCoster` and the gateway's `loggedAnswers` (cybercaius on Discord: a renamed provider was listed under both ids). `Visit` hands records out as recorded, so `Of` is taken once. An id another provider takes later (a rename onto it, or `Save` of a new provider under it) moves from the first one's `Was` to its `Gave` with the time (`giveID`): calls recorded under it before then stay the first one's, later ones are the new one's. An id held by a provider or a subscription now is never renamed (`Renamed`), and `freeID` doesn't hand out an id in any `Was`, so adding a vendor again gets `-2`. Ids reused before `Gave` existed have no time and go to whoever holds the id now. A provider deleted (`Delete`) is kept in providers.json's `deleted` with its name, `Was`, `Gave` and when (`provider.Deleted`): its calls stay under its id while no provider has that id, and once one is added under it (or a later deleted one had it) they go to `Deleted.UsageID`, `<id>~<when, UTC>`, so the new provider has only its own (cybercaius on Discord). The Usage page names either by the deleted provider's name and marks it with when it was deleted (`Renames.Gone`, `Named.Deleted`). Deletes before this was kept have no record and go to whoever holds the id now. A providers.json that can't be read keeps the last renames read, not none. The summary and Requests caches key on `Renames.Key`, the vias cache too. | [`rename.go`](../../internal/provider/rename.go), [`block_price.go`](../../internal/usage/block_price.go) |+| A provider renamed | `Rename` (the editor's id field, `magpie provider set <id> id=<new>`) keeps the old id in `Was`, so a model still picked by it reaches the provider. Usage is recorded under the id a call went out by and never rewritten; whatever reads it names the provider by its id now through `UsageRenames().Of(id, time)`: the summary, the Requests page (its blocks, the probe rows, other computers' days), the Sessions tab's vias, window holds, `NewPricer`/`NewCoster` and the gateway's `loggedAnswers` (cybercaius on Discord: a renamed provider was listed under both ids). `Visit` hands records out as recorded, so `Of` is taken once. An id another provider takes later (a rename onto it, or `Save` of a new provider under it) moves from the first one's `Was` to its `Gave` with the time (`giveID`): calls recorded under it before then stay the first one's, later ones are the new one's. An id held by a provider or a subscription now is never renamed (`Renamed`), and `freeID` doesn't hand out an id in any `Was`, so adding a vendor again gets `-2`. Ids reused before `Gave` existed have no time and go to whoever holds the id now. A provider deleted (`Delete`) is kept in providers.json's `deleted` with its name, `Was`, `Gave` and when (`provider.Deleted`): its calls stay under its id while no provider has that id, and once one is added under it (or a later deleted one had it) they go to `Deleted.UsageID`, `<id>~<when, UTC>`, so the new provider has only its own (cybercaius on Discord). The Usage page names either by the deleted provider's name and marks it with when it was deleted (`Renames.Gone`, `Named.Deleted`). Deletes before this was kept have no record and go to whoever holds the id now. A deleted provider's calls can be merged into a provider there is now, or named by the user, from a "…" beside the Requests page's Provider filter while it is picked (`ledProviderMore` in `plugins/usage/page.js`, `POST /api/usage/deleted`, `GET /api/usage/deleted/into`; CyberCaius on Discord, who had renamed a provider onto the old id to do it). Both are kept on its `Deleted` record, not in settings and never in the log: `MergeDeleted` sets `Into` and `MergedAt`, `NameDeleted` sets `Label` (`Deleted.Shown`), so a provider added later under the old id inherits neither. `Renames.Both(id, time)` gives the id a call is shown under (`Of`, merges followed) and the one it was made to (`Own`): a merged call is listed, filtered and summed under the target, but priced, held and answered by its own (`Record.priceID`, `block_price.go`'s `as`, window holds, `loggedAnswers`), so a merge moves no cost or token. The target's face lists what is merged into it (`Named.Merged`), and the same "…" parts each again (`into` ""). Renaming the target carries the merge; deleting it ends it (a merge older than the target's deletion is not followed). A providers.json that can't be read keeps the last renames read, not none. The summary and Requests caches key on `Renames.Key`, the vias cache too. | [`rename.go`](../../internal/provider/rename.go), [`block_price.go`](../../internal/usage/block_price.go) || The list | `All` merges stored providers with `Accounts()`. An entry with no URL stores the user's picks for an account. The order is `file.Order`. Reads within one request share one build (`Hold`, #746). | [`provider.go`](../../internal/provider/provider.go) || Model picks | `Models` are the models the user picked for agents; none means the vendor's list, up to 24 (`Exposed`), unless the user chose Select none (`NoModels`, `magpie provider models <id> none`): then none of its models is served, not to agents, not to any routing group (auto groups included), not by a `provider/model` id, not as a classifier, while its balance and usage are still read (`ServesNone`, #1818). Codex's own list at `/backend-api/codex/models` drops the ChatGPT account's visible models too, as spawn_agent's list of them does, and the list's ETag changes so a running Codex asks again (`CodexNativePicked` in [`codex_models.go`](../../internal/provider/codex_models.go), #2530); the hidden ones Codex runs jobs of its own on (codex-auto-review, gpt-reserve) and other providers' models stay. The editor's serves control names it Usage only, beside Vendor's list and Picked only, and the card says "usage only" (#2068); Picked only from there picks the models the vendor's list serves, to unpick from. A file without `noModels` reads as before, and a pick clears it. `Save` keeps beside new picks the models listed then (`PickedFrom`, `listedWith`), and the same picks saved again keep it. When the picks held every model listed then, one the vendor lists since is served first beside them (`withNewlyListed`, `Picks`); one listed then and left unpicked stays out. Picks saved before `PickedFrom` was kept are taken to hold every model when they hold every one released up to the newest of them (models.dev's `release_date`), so a Claude account whose user picked them all is served Haiku 5.5 (wakaka). The editor and the TUI start from `Picks`, so a later Save keeps the new model. While some picks are set aside (the list is known to be short), a save with new picks keeps the models the earlier `PickedFrom` had beside those listed now, so a model the user saw and left unpicked isn't new when that list has it again; a list short of no pick is taken as it is. A pick a list fetched again stops listing (`Refetch`, `KeepListsFresh`, and a plugin's hourly `KeepPluginsListed`) leaves `Models` for `Delisted` (`dropGonePicks`), not served, and goes back among the picks when a later list has it (`relistPicks`): Our Free Model's plugin lists its account channels' models only once its service has read them after a restart, and its free pools only while they answer, so each magpie update had its user pick every model again (多喝热水 on Discord). Every save keeps `Delisted` but for what it picks again (`delistedWith`); Select none clears it. A provider whose every pick is in `Delisted` serves none of its models (`waitsForPicks`), not the vendor's list. `KeepPluginsListed` asks again after `relistSoon` (5 minutes) when a list left a pick out. `TestPluginPicksComeBackWithTheirList`, `TestEveryPickDelistedServesNone` and `TestRelayPicksComeBackWithTheirList` cover it. | [`models.go`](../../internal/provider/models.go), [`provider.go`](../../internal/provider/provider.go) || List schedule and rules | A provider of the user's own (not an account, a plugin's or a decision-only one) can set `RefreshEvery` (`magpie provider refresh-every <id> 6h`, or 1d, or off, at least 10m) and `AutoInclude` (`magpie provider auto-include <id> 'gpt-*' 're:claude-.*'`, or none; a glob with `*`, or `re:` matching the whole id, both in any case (`compilePattern`; `(?-i)` inside a `re:` makes case matter); a plain id is refused) (#2135). The gateway (`magpie serve` or the app) looks each minute and fetches a list once `Fetched` is that old, through the same `Refetch` as Fetch models (`KeepListsFresh`); a failed fetch waits `min(every, 15m)`. `RefetchTaking` adds to the picks a model the fetch newly lists that a rule matches (`takeMatching`), starting from what is served now (`Exposed`), so nothing served is taken away and a model the user unticks stays out (it is not new to the next fetch). Setting the rules takes the current list's matches. Every save that doesn't set them keeps them, the window's editor among them; the editor has no fields for them yet. A Remote magpie with no `RefreshEvery` is fetched every 10m (`RemoteMagpieEvery`, `RefreshSchedule`), so a model the other magpie lists later (ZCode's GLM-5.3-Trial once its plugin read the gift, 6igsteven on Discord) is served without Fetch models; `off` set on it is kept as `"off"` and asks it only when told (`TestRemoteMagpieListFollowsTheOtherMagpie`). | [`autolist.go`](../../internal/provider/autolist.go), [`models.go`](../../internal/provider/models.go), [`providers_cli.go`](../../internal/cli/providers_cli.go) |diff --git a/internal/gateway/codex_backend.go b/internal/gateway/codex_backend.goindex adcdf2218..24d58bc71 100644--- a/internal/gateway/codex_backend.go+++ b/internal/gateway/codex_backend.go@@ -442,7 +442,9 @@ func loggedAnswers(convs ...string) []loggedAnswer {// from itself)renamed := provider.UsageRenames()usage.Visit(time.Now().Add(-sealerKeep), func(r usage.Record) {- r.Provider = renamed.Of(r.Provider, r.Time)+ // the provider that answered, not one a deleted provider's+ // calls were merged into on the Usage page (Renames.Own)+ r.Provider = renamed.Own(r.Provider, r.Time)if r.Provider == "" || r.Provider == "magpie" || r.IsRejected() || r.IsReported() || r.Status <= 0 || r.Status >= 400 || r.Computer != "" || isTitleKind(r.Kind) {return}diff --git a/internal/gui/assets/app.css b/internal/gui/assets/app.cssindex a1c9c8f3e..932fe892b 100644--- a/internal/gui/assets/app.css+++ b/internal/gui/assets/app.css@@ -4370,3 +4370,6 @@ body .row.agent .ag-prefs .row.pref .who { display: block; }:is(.ag-prefs, .agent-prefs) .compact-at { flex: 0 1 auto; min-width: 0; flex-wrap: wrap; row-gap: 6px; }/* Settings' search finds them; not searching, the links stay out */#view-settings:not(.set-searching) #agentPrefLinks { display: none; }+.sess-pick.led-more { padding: 0 6px; }+.sess-pick.led-more svg { transform: none; }+.sess-tools .led-rename { flex: none; width: 180px; height: 26px; padding: 0 8px; border-radius: 7px; font-size: 12px; }diff --git a/internal/gui/assets/plugins/usage/client.js b/internal/gui/assets/plugins/usage/client.jsindex 28d1e32d9..56a9ad637 100644--- a/internal/gui/assets/plugins/usage/client.js+++ b/internal/gui/assets/plugins/usage/client.js@@ -58,6 +58,7 @@ magpie.plugin("usage", (ctx) => {<button class="sess-pick" id="ledAgent" type="button" hidden></button><button class="sess-pick" id="ledPurpose" type="button" hidden></button><button class="sess-pick" id="ledProvider" type="button" hidden></button>+ <button class="sess-pick led-more" id="ledProviderMore" type="button" aria-haspopup="menu" hidden></button><button class="sess-pick" id="ledAccount" type="button" hidden></button><button class="sess-pick" id="ledKey" type="button" hidden></button><div class="segs" id="ledStatus"></div>diff --git a/internal/gui/assets/plugins/usage/i18n.js b/internal/gui/assets/plugins/usage/i18n.jsindex 08cc54e40..a266020d6 100644--- a/internal/gui/assets/plugins/usage/i18n.js+++ b/internal/gui/assets/plugins/usage/i18n.js@@ -6,6 +6,21 @@ magpie.strings({"+{n} keys": "+{n} 个密钥","One balance for the account these keys share: {keys}": "这些密钥同属一个账号,余额只显示一次:{keys}","The balances added up, each account's once": "余额之和,每个账号只计一次",+ "Merge or rename this deleted provider": "合并或重命名这个已删除的服务商",+ "Providers merged into this one": "合并到这里的服务商",+ "Separate {name}": "分开 {name}",+ "List its requests on their own again": "把它的请求重新单独列出",+ "Merged providers": "已合并的服务商",+ "{name}'s requests are listed on their own again": "{name} 的请求已重新单独列出",+ "Merge into…": "合并到…",+ "List its requests under a provider you have": "把它的请求列在你现有的一个服务商下",+ "Rename…": "重命名…",+ "The name this page shows it by": "本页显示它用的名字",+ "Deleted provider": "已删除的服务商",+ "No provider to merge into": "没有可合并到的服务商",+ "{from}'s requests are listed under {to} now": "{from} 的请求现在列在 {to} 下",+ "Merge into": "合并到",+ "Search providers…": "搜索服务商…","Deleted {date}": "{date} 已删除","Bar chart": "柱状图","Line chart": "折线图",@@ -213,6 +228,21 @@ magpie.strings({"+{n} keys": "+{n} 個金鑰","One balance for the account these keys share: {keys}": "這些金鑰同屬一個帳號,餘額只顯示一次:{keys}","The balances added up, each account's once": "餘額之和,每個帳號只計一次",+ "Merge or rename this deleted provider": "合併或重新命名這個已刪除的供應商",+ "Providers merged into this one": "合併到這裡的供應商",+ "Separate {name}": "分開 {name}",+ "List its requests on their own again": "把它的請求重新單獨列出",+ "Merged providers": "已合併的供應商",+ "{name}'s requests are listed on their own again": "{name} 的請求已重新單獨列出",+ "Merge into…": "合併到…",+ "List its requests under a provider you have": "把它的請求列在你現有的一個供應商下",+ "Rename…": "重新命名…",+ "The name this page shows it by": "本頁顯示它用的名稱",+ "Deleted provider": "已刪除的供應商",+ "No provider to merge into": "沒有可合併到的供應商",+ "{from}'s requests are listed under {to} now": "{from} 的請求現在列在 {to} 下",+ "Merge into": "合併到",+ "Search providers…": "搜尋供應商…","Deleted {date}": "{date} 已刪除","Bar chart": "長條圖","Line chart": "折線圖",@@ -420,6 +450,21 @@ magpie.strings({"+{n} keys": "+{n} 個のキー","One balance for the account these keys share: {keys}": "これらのキーは同じアカウントのもので、残高は一度だけ表示されます:{keys}","The balances added up, each account's once": "残高の合計(各アカウントは一度だけ)",+ "Merge or rename this deleted provider": "この削除済みプロバイダを統合または名前変更",+ "Providers merged into this one": "ここに統合されたプロバイダ",+ "Separate {name}": "{name} を分ける",+ "List its requests on their own again": "リクエストを再び単独で表示",+ "Merged providers": "統合されたプロバイダ",+ "{name}'s requests are listed on their own again": "{name} のリクエストを再び単独で表示しています",+ "Merge into…": "統合先…",+ "List its requests under a provider you have": "リクエストを既存のプロバイダの下に表示",+ "Rename…": "名前を変更…",+ "The name this page shows it by": "このページでの表示名",+ "Deleted provider": "削除済みプロバイダ",+ "No provider to merge into": "統合先のプロバイダがありません",+ "{from}'s requests are listed under {to} now": "{from} のリクエストは {to} の下に表示されます",+ "Merge into": "統合先",+ "Search providers…": "プロバイダを検索…","Deleted {date}": "{date} に削除","Bar chart": "棒グラフ","Line chart": "折れ線グラフ",@@ -627,6 +672,21 @@ magpie.strings({"+{n} keys": "+{n} Schlüssel","One balance for the account these keys share: {keys}": "Ein Guthaben für das Konto, zu dem diese Schlüssel gehören: {keys}","The balances added up, each account's once": "Die Guthaben zusammengezählt, jedes Konto einmal",+ "Merge or rename this deleted provider": "Diesen gelöschten Anbieter zusammenführen oder umbenennen",+ "Providers merged into this one": "Hierher zusammengeführte Anbieter",+ "Separate {name}": "{name} trennen",+ "List its requests on their own again": "Seine Anfragen wieder einzeln auflisten",+ "Merged providers": "Zusammengeführte Anbieter",+ "{name}'s requests are listed on their own again": "Die Anfragen von {name} werden wieder einzeln aufgeführt",+ "Merge into…": "Zusammenführen mit…",+ "List its requests under a provider you have": "Seine Anfragen unter einem vorhandenen Anbieter auflisten",+ "Rename…": "Umbenennen…",+ "The name this page shows it by": "Der Name, unter dem diese Seite ihn zeigt",+ "Deleted provider": "Gelöschter Anbieter",+ "No provider to merge into": "Kein Anbieter zum Zusammenführen",+ "{from}'s requests are listed under {to} now": "Die Anfragen von {from} stehen jetzt unter {to}",+ "Merge into": "Zusammenführen mit",+ "Search providers…": "Anbieter suchen…","Deleted {date}": "Gelöscht am {date}","Bar chart": "Balkendiagramm","Line chart": "Liniendiagramm",diff --git a/internal/gui/assets/plugins/usage/page.js b/internal/gui/assets/plugins/usage/page.jsindex 39f0dac25..b1908b92f 100644--- a/internal/gui/assets/plugins/usage/page.js+++ b/internal/gui/assets/plugins/usage/page.js@@ -1782,6 +1782,90 @@ function ledFit() {hint.hidden = !tight;}+// ledProviderMore is the menu beside the Provider filter while a deleted+// provider is picked in it (CyberCaius on Discord): its requests merged+// into a provider there is now, or shown by a name of the reader's. With+// a provider picked that has deleted ones merged in, it parts each again.+// Nothing in the log is rewritten (POST /api/usage/deleted), and a merged+// request costs what it did.+function ledProviderMore(p) {+ const b = $("#ledProviderMore");+ const merged = p?.merged || [];+ b.hidden = !p || (!p.deleted && !merged.length);+ if (b.hidden) return;+ b.replaceChildren(svg(DOTS, 14, 2));+ const tip = t(p.deleted ? "Merge or rename this deleted provider" : "Providers merged into this one");+ b.title = tip;+ b.setAttribute("aria-label", tip);+ const reload = () => { ledOffset = 0; return loadLedger(); };+ const send = (body) => api("usage/deleted", Object.assign({ id: p.id }, body));+ b.onclick = (e) => {+ e.stopPropagation();+ if (b.classList.contains("open")) return closeProtoMenu();+ if (!p.deleted) {+ openProtoMenu(b, merged.map((m) => ({ v: m.id, name: t("Separate {name}", { name: m.name }), literalName: true, note: "List its requests on their own again" })), "", async (id) => {+ const m = merged.find((x) => x.id === id);+ try {+ await api("usage/deleted", { id, into: "" });+ ledProvider = id;+ ledOwn("provider");+ await reload();+ status(t("{name}'s requests are listed on their own again", { name: m.name }));+ } catch (err) { status(err.message, "err"); }+ }, "Merged providers", "sess-menu", "right");+ return;+ }+ openProtoMenu(b, [+ { v: "merge", name: "Merge into…", note: "List its requests under a provider you have" },+ { v: "rename", name: "Rename…", note: "The name this page shows it by" },+ ], "", (v) => (v === "merge" ? mergeInto() : renameDeleted()), "Deleted provider", "sess-menu", "right");+ };+ // a provider there is now, picked from a menu of them+ const mergeInto = async () => {+ let into;+ try { into = await api("usage/deleted/into"); } catch (err) { return status(err.message, "err"); }+ const opts = into.map((x) => ({ v: x.id, name: x.name, literalName: true, note: x.name === x.id ? "" : x.id }));+ if (!opts.length) return status(t("No provider to merge into"), "err");+ openProtoMenu(b, opts, "", async (id) => {+ const to = opts.find((o) => o.v === id);+ try {+ await send({ into: id });+ ledProvider = id;+ ledOwn("provider");+ await reload();+ status(t("{from}'s requests are listed under {to} now", { from: p.name, to: to.name }));+ } catch (err) { status(err.message, "err"); }+ }, "Merge into", "sess-menu", "right", false, opts.length > 8 ? "Search providers…" : "");+ };+ // the filter's own button takes the name, as a key's row does+ const renameDeleted = () => {+ const pick = $("#ledProvider");+ const i = input(p.name, t("Name"));+ i.className = "rename-in led-rename";+ i.setAttribute("aria-label", t("Rename"));+ let over = false;+ const done = async (save) => {+ if (over) return;+ over = true;+ i.remove();+ pick.hidden = false;+ b.hidden = false;+ if (!save || i.value.trim() === p.name) return;+ try {+ await send({ name: i.value });+ await reload();+ } catch (err) { status(err.message, "err"); }+ };+ i.onkeydown = (e) => { e.stopPropagation(); if (e.key === "Enter") done(true); else if (e.key === "Escape") done(false); };+ i.onblur = () => done(true);+ pick.hidden = true;+ b.hidden = true;+ pick.after(i);+ i.focus();+ i.select();+ };+}+function renderLedger() {const l = ledger;const callers = l.callerKeys || [];@@ -1821,6 +1905,7 @@ function renderLedger() {sessPick($("#ledComputer"), "All computers", ledComputer, computers, "Computer", (v) => { ledComputer = v; ledOwn("computer"); ledOffset = 0; loadLedger().catch((e) => status(e.message, "err")); });sessPick($("#ledAgent"), "All agents", ledAgent, l.agents.map((a) => ({ v: a.id, name: a.name, note: "" })), "Agent", (v) => { ledAgent = v; ledOwn("agent"); ledOffset = 0; loadLedger().catch((e) => status(e.message, "err")); });sessPick($("#ledProvider"), "All providers", ledProvider, providers.map((p) => ({ v: p.id, name: t(p.name), note: p.deleted ? t("Deleted {date}", { date: new Date(p.deleted).toLocaleDateString(intlLang(), { year: "numeric", month: "short", day: "numeric" }) }) : "" })), "Provider", (v) => { ledProvider = v; ledOwn("provider"); ledOffset = 0; loadLedger().catch((e) => status(e.message, "err")); });+ ledProviderMore(providers.find((p) => p.id === ledProvider));sessPick($("#ledAccount"), "All accounts", ledAccount, accounts.map((a) => ({ v: a.id, name: a.name || a.id, note: [a.name && a.id, ...(a.providers || []).map((p) => t(p))].filter(Boolean).join(" · ") })), "Account", (v) => { ledAccount = v; ledOffset = 0; loadLedger().catch((e) => status(e.message, "err")); });sessPick($("#ledKey"), "All gateway keys", ledCallerKey, callers.map((k) => ({v: k.id, name: k.name, note: "",diff --git a/internal/gui/tests/usage-ledger-deleted-merge.test.cjs b/internal/gui/tests/usage-ledger-deleted-merge.test.cjsnew file mode 100644index 000000000..baeb6ba8b--- /dev/null+++ b/internal/gui/tests/usage-ledger-deleted-merge.test.cjs@@ -0,0 +1,169 @@+// Run with Node's test runner and Playwright on the module path; see README.md.+// Usage › Requests with a deleted provider picked in the Provider filter+// (CyberCaius on Discord): a "…" beside it merges its requests into a+// provider there is now, or names it, each in one step; with the merged+// provider picked, the same "…" parts them again. The page asks the+// gateway (POST /api/usage/deleted) and never rewrites a row itself. A+// provider that is not deleted and has none merged has no "…". Every+// language, both engines, a wide and a narrow window; the API is faked.+const assert = require("node:assert/strict");+const fs = require("node:fs/promises");+const path = require("node:path");+const { test } = require("node:test");+const { chromium, webkit } = require("playwright");+const reader = require("./reader.cjs");++const assets = path.resolve(__dirname, "../assets");+const now = Date.now();+const H = 3600e3;+const hour = new Date(Math.floor(now / H) * H).toISOString();+const GONE = new Date(now - 5 * 86400e3).toISOString();+const base = { agent: "claude", agentName: "Claude Code", icon: "claudecode-color", req: "m", model: "m", in: 1000, out: 100, cache_read: 0, cache_write: 0, ms: 2000, status: 200, ep: "/v1/messages", cost: 0.01, priced: true };++function ledger(q, s) {+ const named = s.name || "relay";+ const rows = [+ { ...base, t: new Date(now - 60e3).toISOString(), rid: "req_main", provider: "main", providerName: "Main" },+ { ...base, t: new Date(now - 120e3).toISOString(), rid: "req_relay", provider: s.into ? "main" : "relay", providerName: s.into ? "Main" : named },+ ].filter((r) => !q.get("provider") || r.provider === q.get("provider"));+ const providers = s.into+ ? [{ id: "main", name: "Main", icon: "generic", merged: [{ id: "relay", name: named }] }]+ : [{ id: "main", name: "Main", icon: "generic" }, { id: "relay", name: named, icon: "generic", deleted: GONE }];+ const part = { calls: rows.length, tokens: rows.length * 1100, cost: rows.length * 0.01 };+ return {+ period: q.get("period"), rows, offset: 0, total: rows.length, calls: rows.length, errors: 0,+ input: rows.length * 1000, output: rows.length * 100, cache_read: 0, cache_write: 0, cost: rows.length * 0.01, unpriced: 0,+ by: { model: [{ id: "m", name: "m", icon: "generic", calls: rows.length, input: 0, output: 0, cache_read: 0, cache_write: 0, cost: part.cost }] },+ bucket: "hour",+ series: [{ label: "", time: hour, calls: rows.length, input: rows.length * 1000, output: rows.length * 100, by: { model: { m: part } } }],+ agents: [{ id: "claude", name: "Claude Code", icon: "claudecode-color" }],+ providers, callerKeys: [],+ };+}++function server(lang, s) {+ return async (route) => {+ const req = route.request();+ const url = new URL(req.url());+ const json = (data) => route.fulfill({ json: data });+ if (url.pathname === "/boot.js") return route.fulfill({ contentType: "text/javascript", body: `window.bootPrefs = {lang:"${lang}",theme:"light",web:false};` });+ if (url.pathname === "/wails/runtime.js") return route.fulfill({ contentType: "text/javascript", body: "export const Window = {};" });+ if (url.pathname === "/api/state") return json({ agents: [], profiles: [], settings: { lang, theme: "light" } });+ if (url.pathname === "/api/usage/requests") { s.asked.push(url.searchParams); return json(ledger(url.searchParams, s)); }+ if (url.pathname === "/api/usage/deleted/into") return json([{ id: "main", name: "Main", icon: "generic" }]);+ if (url.pathname === "/api/usage/deleted") {+ assert.equal(req.method(), "POST");+ const b = JSON.parse(req.postData());+ s.posted.push(b);+ if ("into" in b) s.into = b.into;+ if ("name" in b) s.name = b.name.trim();+ return json({ ok: true });+ }+ if (url.pathname === "/api/usage/quotas") return json([]);+ if (url.pathname === "/api/usage") return json({ calls: 0, errors: 0, input: 0, output: 0, reasoning: 0, unpriced: 0, cost: 0, bucket: "day", series: [], agents: [], models: [], path: "" });+ if (url.pathname === "/api/providers") return json({ providers: [], presets: [], excluded: [], models: [], gateway: { running: false } });+ if (url.pathname === "/api/sessions") return json({ sessions: [], dirs: [] });+ if (url.pathname === "/api/sessions/stats") return json({ from: "", to: "", days: [], agents: {} });+ if (url.pathname === "/api/groups") return json({ groups: [], models: [] });+ if (url.pathname.startsWith("/api/")) return json({});+ const file = path.join(assets, url.pathname === "/" ? "index.html" : url.pathname);+ const contentType = { ".html": "text/html", ".js": "text/javascript", ".css": "text/css", ".svg": "image/svg+xml", ".png": "image/png" }[path.extname(file)];+ await route.fulfill({ body: await fs.readFile(file), contentType });+ };+}++const L = {+ en: { merge: "Merge into…", rename: "Rename…", separate: "Separate Old relay", merged: "Old relay's requests are listed under Main now" },+ zh: { merge: "合并到…", rename: "重命名…", separate: "分开 Old relay", merged: "Old relay 的请求现在列在 Main 下" },+ "zh-TW": { merge: "合併到…", rename: "重新命名…", separate: "分開 Old relay", merged: "Old relay 的請求現在列在 Main 下" },+ ja: { merge: "統合先…", rename: "名前を変更…", separate: "Old relay を分ける", merged: "Old relay のリクエストは Main の下に表示されます" },+ de: { merge: "Zusammenführen mit…", rename: "Umbenennen…", separate: "Old relay trennen", merged: "Die Anfragen von Old relay stehen jetzt unter Main" },+};++async function open(browser, lang, width) {+ const errors = [];+ const s = { asked: [], posted: [], into: "", name: "" };+ const ctx = await browser.newContext({ viewport: { width, height: 900 }, reducedMotion: "reduce" });+ const p = await ctx.newPage();+ p.setDefaultTimeout(5000);+ p.on("pageerror", (e) => errors.push(e.message));+ await p.route("**/*", server(lang, s));+ await p.goto("http://magpie.test/");+ await p.locator('[data-view="usage"]').first().click();+ await p.locator(":is(#usageTab .opt, .nav-s):visible").nth(1).click();+ await p.locator("#ledWrap .led tbody tr").first().waitFor();+ await p.waitForTimeout(300);+ return { ctx, p, errors, s };+}++const pickProvider = async (p, name) => {+ await reader.click(p, p.locator("#ledProvider"));+ await reader.click(p, p.locator(".sess-menu .pm-item").filter({ hasText: name }).first());+};+const rows = (p, n) => p.waitForFunction((n) => document.querySelectorAll("#ledWrap .led tbody tr.led-row").length === n, n);++for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium", "webkit"])) {+ test(engine + ": a deleted provider merged, named and parted", async (t) => {+ assert(["chromium", "webkit"].includes(engine), "BROWSER must be chromium or webkit");+ const browser = await (engine === "webkit" ? webkit.launch() : chromium.launch({ channel: "chromium" }));+ t.after(() => browser.close());+ for (const lang of Object.keys(L)) {+ const w = L[lang];+ for (const width of [1400, 760]) {+ await t.test(`${lang} at ${width}px`, async () => {+ const { ctx, p, errors, s } = await open(browser, lang, width);+ t.after(() => ctx.close());+ const more = p.locator("#ledProviderMore");+ assert.equal(await more.isVisible(), false, "no … with every provider listed");+ await pickProvider(p, "Main");+ await rows(p, 1);+ assert.equal(await more.isVisible(), false, "no … beside a provider with nothing merged");++ // rename: the filter's button turns into the name, Escape keeps it+ await pickProvider(p, "relay");+ await p.waitForFunction(() => !document.querySelector("#ledProviderMore").hidden);+ await reader.click(p, more);+ const items = await p.locator(".sess-menu .pm-item").allTextContents();+ assert(items.some((x) => x.includes(w.merge)) && items.some((x) => x.includes(w.rename)), items.join(" | "));+ await reader.click(p, p.locator(".sess-menu .pm-item").filter({ hasText: w.rename }));+ const field = p.locator("#ledgerPane input.led-rename");+ await field.waitFor();+ assert.equal(await field.inputValue(), "relay");+ assert.equal(await p.locator("#ledProvider").isVisible(), false);+ await p.keyboard.press("Escape");+ await p.waitForFunction(() => !document.querySelector("input.led-rename"));+ assert.equal(s.posted.length, 0, "Escape sends nothing");+ await reader.click(p, more);+ await reader.click(p, p.locator(".sess-menu .pm-item").filter({ hasText: w.rename }));+ await field.fill(" Old relay ");+ await p.keyboard.press("Enter");+ await p.waitForFunction(() => document.querySelector("#ledProvider").textContent.includes("Old relay"));+ assert.deepEqual(s.posted.at(-1), { id: "relay", name: " Old relay " });+ assert.equal(await p.locator("#ledProvider").isVisible(), true);++ // merge: a provider there is now picked from the same …+ await p.waitForFunction(() => !document.querySelector("#ledProviderMore").hidden);+ await reader.click(p, more);+ await reader.click(p, p.locator(".sess-menu .pm-item").filter({ hasText: w.merge }));+ await reader.click(p, p.locator(".sess-menu .pm-item").filter({ hasText: "Main" }));+ await p.waitForFunction(() => document.querySelector("#ledProvider").textContent.includes("Main"));+ assert.deepEqual(s.posted.at(-1), { id: "relay", into: "main" });+ assert.equal(s.asked.at(-1).get("provider"), "main", "the page moves to the provider merged into");+ await rows(p, 2);+ assert.equal((await p.locator("#status").textContent()).trim(), w.merged);++ // the merged provider's … parts them again+ await p.waitForFunction(() => !document.querySelector("#ledProviderMore").hidden);+ await reader.click(p, more);+ await reader.click(p, p.locator(".sess-menu .pm-item").filter({ hasText: w.separate }));+ await p.waitForFunction(() => document.querySelector("#ledProvider").textContent.includes("Old relay"));+ assert.deepEqual(s.posted.at(-1), { id: "relay", into: "" });+ assert.equal(s.asked.at(-1).get("provider"), "relay");+ await rows(p, 1);+ assert.equal(await p.locator("#view-usage").evaluate((v) => v.scrollWidth <= v.clientWidth + 1), true, "the page doesn't scroll sideways");+ assert.deepEqual(errors, []);+ });+ }+ }+ });+}diff --git a/internal/gui/usage.go b/internal/gui/usage.goindex 836e52bdd..517fb3ae6 100644--- a/internal/gui/usage.go+++ b/internal/gui/usage.go@@ -101,6 +101,62 @@ func usageRoutes(mux *http.ServeMux, w Windows) {// the days and hours of the reader's zone, which the page nameswriteJSON(rw, usageState(periodOf(r.URL.Query().Get("period")), usageapi.ZoneOf(r)))})+ // the providers a deleted one's calls can be merged into: those there+ // are now and shown, by name+ mux.HandleFunc("GET /api/usage/deleted/into", func(rw http.ResponseWriter, r *http.Request) {+ type target struct {+ ID string `json:"id"`+ Name string `json:"name"`+ Icon string `json:"icon"`+ }+ out := []target{}+ for _, p := range provider.All() {+ if p.Hidden {+ continue+ }+ name, icon := p.Name, p.Icon+ if name == "" {+ n, i := provider.KnownFace(p.ID)+ name = n+ if icon == "" {+ icon = i+ }+ }+ if name == "" {+ name = p.ID+ }+ out = append(out, target{p.ID, name, icon})+ }+ writeJSON(rw, out)+ })+ // a deleted provider's calls, as the Usage page lists them under id:+ // merged into a provider there is now (Into, "" to part them again)+ // or shown by a name of the user's (Name, "" for its own); one not+ // sent stays as it is (CyberCaius on Discord)+ mux.HandleFunc("POST /api/usage/deleted", func(rw http.ResponseWriter, r *http.Request) {+ var in struct {+ ID string+ Into *string+ Name *string+ }+ if err := json.NewDecoder(r.Body).Decode(&in); err != nil {+ fail(rw, err)+ return+ }+ if in.Into != nil {+ if err := provider.MergeDeleted(in.ID, *in.Into); err != nil {+ fail(rw, err)+ return+ }+ }+ if in.Name != nil {+ if err := provider.NameDeleted(in.ID, *in.Name); err != nil {+ fail(rw, err)+ return+ }+ }+ writeJSON(rw, map[string]bool{"ok": true})+ })// the ledger: the period's calls, newest first, a page at a timemux.HandleFunc("GET /api/usage/requests", func(rw http.ResponseWriter, r *http.Request) {q := r.URL.Query()diff --git a/internal/gui/usage_deleted_test.go b/internal/gui/usage_deleted_test.gonew file mode 100644index 000000000..1bffcaabc--- /dev/null+++ b/internal/gui/usage_deleted_test.go@@ -0,0 +1,62 @@+package gui++import (+ "encoding/json"+ "net/http"+ "net/http/httptest"+ "strings"+ "testing"+ "time"++ "github.com/yetone/magpie/internal/provider"+ "github.com/yetone/magpie/internal/usage"+)++// The Usage page's "…" beside a deleted provider (CyberCaius on Discord):+// the providers it can be merged into, and a POST that merges, names and+// parts it, the request log left as it was.+func TestUsageDeletedProviderRoutes(t *testing.T) {+ sandboxHome(t)+ for _, p := range []provider.Provider{+ {ID: "relay", Key: "k", Chat: "http://127.0.0.1:1/v1", Models: []string{"gpt-9"}},+ {ID: "main", Name: "Main", Key: "k2", Chat: "http://127.0.0.1:2/v1", Models: []string{"gpt-9"}},+ } {+ if err := provider.Save(p); err != nil {+ t.Fatal(err)+ }+ }+ usage.Append(usage.Record{Time: time.Now().Add(-time.Hour), Agent: "codex", Provider: "relay", Model: "gpt-9", Input: 10, Status: 200})+ if err := provider.Delete("relay"); err != nil {+ t.Fatal(err)+ }+ mux := http.NewServeMux()+ usageRoutes(mux, folderOnly{})+ do := func(method, path, body string, code int) string {+ t.Helper()+ w := httptest.NewRecorder()+ mux.ServeHTTP(w, httptest.NewRequest(method, path, strings.NewReader(body)))+ if w.Code != code {+ t.Fatalf("%s %s: %d %s", method, path, w.Code, w.Body)+ }+ return w.Body.String()+ }+ var into []struct{ ID, Name string }+ if err := json.Unmarshal([]byte(do("GET", "/api/usage/deleted/into", "", 200)), &into); err != nil {+ t.Fatal(err)+ }+ if len(into) != 1 || into[0].ID != "main" || into[0].Name != "Main" {+ t.Fatalf("into: %+v", into)+ }+ do("POST", "/api/usage/deleted", `{"id":"relay","name":"Old relay","into":"main"}`, 200)+ gone, _ := provider.UsageRenames().Gone("relay")+ if gone.Label != "Old relay" || gone.Into != "main" {+ t.Fatalf("after the POST: %+v", gone)+ }+ // a field not sent stays as it is+ do("POST", "/api/usage/deleted", `{"id":"relay","into":""}`, 200)+ if gone, _ := provider.UsageRenames().Gone("relay"); gone.Label != "Old relay" || gone.Into != "" {+ t.Fatalf("parted: %+v", gone)+ }+ do("POST", "/api/usage/deleted", `{"id":"relay","into":"nobody"}`, http.StatusBadRequest)+ do("POST", "/api/usage/deleted", `{"id":"main","name":"x"}`, http.StatusBadRequest)+}diff --git a/internal/provider/rename.go b/internal/provider/rename.goindex e1121c5a5..e61c874c5 100644--- a/internal/provider/rename.go+++ b/internal/provider/rename.go@@ -85,6 +85,12 @@ func Rename(from, to string) error {f.Order[j] = to}}+ // a deleted provider merged into it stays merged (MergeDeleted)+ for j := range f.Deleted {+ if f.Deleted[j].Into == from {+ f.Deleted[j].Into = to+ }+ }for j := range f.Groups {g := &f.Groups[j]for k, m := range g.Members {@@ -176,6 +182,37 @@ type Deleted struct {At time.Time `json:"at"`Was []string `json:"was,omitempty"`Gave []Gave `json:"gave,omitempty"`+ // Into is the provider the user merged its calls into on the Usage+ // page (MergeDeleted): they are listed and counted as that one's,+ // priced as they were (Renames.Own). Only this record is merged, so+ // a provider added under its id later keeps its own calls. A provider+ // deleted under Into after MergedAt ends the merge.+ Into string `json:"into,omitempty"`+ MergedAt time.Time `json:"mergedAt,omitzero"`+ // Label is the name the user gave it on the Usage page (NameDeleted),+ // shown instead of Name.+ Label string `json:"label,omitempty"`+}++// Shown is the name the Usage page lists it by: the user's Label, its+// own name, or its id.+func (d Deleted) Shown() string {+ for _, n := range []string{d.Label, d.Name} {+ if n = strings.TrimSpace(n); n != "" {+ return n+ }+ }+ return d.ID+}++// merged is the provider its calls go to now, "" when it is not merged:+// a provider deleted under Into since the merge ends it, as there is no+// one to show them under (its own calls are shown as deleted again).+func (d Deleted) merged(all []Deleted) string {+ if d.Into == "" || slices.ContainsFunc(all, func(l Deleted) bool { return l.ID == d.Into && l.At.After(d.MergedAt) }) {+ return ""+ }+ return d.Into}// UsageID is the id its calls are counted under: its id and when it was@@ -199,6 +236,11 @@ type Renames struct {sites map[string]string// gone are the deleted providers by UsageIDgone map[string]Deleted+ // into are the deleted providers merged into another (Deleted.Into),+ // the id their calls go to otherwise to the provider's; merged is+ // the other way, each provider's merged ones+ into map[string]string+ merged map[string][]string}type gaveTo struct {@@ -208,8 +250,30 @@ type gaveTo struct {// Of is the id the provider a call recorded under id at the time at has// now; id itself when it was not renamed. A zero at is before every-// hand-over.+// hand-over. A deleted provider the user merged into another is that+// one (Deleted.Into); Own is the id without the merge, which the call is+// priced by.func (r Renames) Of(id string, at time.Time) string {+ shown, _ := r.Both(id, at)+ return shown+}++// Both is Of, and Own beside it: the provider a call is shown under and+// the one it is priced as. They differ only for a deleted provider merged+// into another, whose calls cost what they did before the merge.+func (r Renames) Both(id string, at time.Time) (shown, own string) {+ own = r.Own(id, at)+ if to, ok := r.into[own]; ok {+ return to, own+ }+ return own, own+}++// Own is Of without merges: the provider the call was made to, by the id+// it has now, a deleted one by its own (UsageID). What it cost is worked+// out under it, and what decides by who answered (the gateway's+// loggedAnswers, window holds) goes by it.+func (r Renames) Own(id string, at time.Time) string {for _, g := range r.gave[id] {if at.Before(g.Until) {return g.To@@ -283,6 +347,10 @@ func (r Renames) Gone(id string) (Deleted, bool) {return d, ok}+// Merged are the deleted providers merged into id, by the ids Gone+// knows them by, oldest deleted first.+func (r Renames) Merged(id string) []string { return r.merged[id] }+// Timed is whether the id's calls go to more than one provider by when// they were made, so Of needs each call's time.func (r Renames) Timed(id string) bool { return len(r.gave[id]) > 0 }@@ -304,11 +372,11 @@ func (r Renames) Each(id string) []string {func RenamesOf(now map[string]string) Renames { return Renames{now: now} }// Empty is whether no id is renamed.-func (r Renames) Empty() bool { return len(r.now) == 0 && len(r.gave) == 0 }+func (r Renames) Empty() bool { return len(r.now) == 0 && len(r.gave) == 0 && len(r.into) == 0 }// Key is r as text, for a cache of what was worked out with it.func (r Renames) Key() string {- b, _ := json.Marshal([]any{r.now, r.gave})+ b, _ := json.Marshal([]any{r.now, r.gave, r.into})return string(b)}@@ -354,6 +422,13 @@ func renamesOf(ps []Provider, deleted []Deleted) Renames {r.gone = map[string]Deleted{}}r.gone[to] = d+ if into := d.merged(deleted); into != "" {+ if r.into == nil {+ r.into, r.merged = map[string]string{}, map[string][]string{}+ }+ r.into[to] = into+ r.merged[into] = append(r.merged[into], to)+ }if to != d.ID {give(d.ID, d.At, to)}@@ -454,3 +529,64 @@ func FreeID(id string) string { return freeID(id) }func hasEndpoint(p Provider) bool {return p.Chat != "" || p.Responses != "" || p.Anthropic != "" || p.Gemini != "" || p.Decide != "" || p.IsVertex()}++// deletedAt is the index in f.Deleted of the deleted provider whose calls+// the Usage page lists under id (Renames.Gone).+func deletedAt(f *file, id string) (int, error) {+ d, ok := renamesOf(f.Providers, f.Deleted).gone[id]+ if !ok {+ return -1, fmt.Errorf("no deleted provider %q", id)+ }+ i := slices.IndexFunc(f.Deleted, func(l Deleted) bool { return l.ID == d.ID && l.At.Equal(d.At) })+ if i < 0 {+ return -1, fmt.Errorf("no deleted provider %q", id)+ }+ return i, nil+}++// MergeDeleted has the calls of the deleted provider the Usage page lists+// under id listed and counted as into's, a provider there is now, or, with+// into "", as its own again (CyberCaius on Discord, who had renamed a+// provider onto a deleted one's id to see their calls together). Nothing+// in the usage log is rewritten: the merge is kept on its Deleted record+// and read through Renames, its calls priced as they were. A provider+// added later under the deleted one's id is not merged.+func MergeDeleted(id, into string) error {+ into = strings.ToLower(strings.TrimSpace(into))+ f, err := read()+ if err != nil {+ return err+ }+ i, err := deletedAt(&f, id)+ if err != nil {+ return err+ }+ d := &f.Deleted[i]+ if into == "" {+ d.Into, d.MergedAt = "", time.Time{}+ return store(f)+ }+ if !slices.ContainsFunc(All(), func(p Provider) bool { return p.ID == into }) {+ return fmt.Errorf("no provider %q to merge into", into)+ }+ d.Into, d.MergedAt = into, time.Now()+ return store(f)+}++// NameDeleted names the deleted provider the Usage page lists under id+// as the user wants it shown there; "" shows it by its own name again.+func NameDeleted(id, name string) error {+ f, err := read()+ if err != nil {+ return err+ }+ i, err := deletedAt(&f, id)+ if err != nil {+ return err+ }+ f.Deleted[i].Label = strings.TrimSpace(name)+ if f.Deleted[i].Label == f.Deleted[i].Name {+ f.Deleted[i].Label = ""+ }+ return store(f)+}diff --git a/internal/usage/block_price.go b/internal/usage/block_price.goindex 19d38aa17..0de55460a 100644--- a/internal/usage/block_price.go+++ b/internal/usage/block_price.go@@ -52,7 +52,8 @@ func newBlockPricer(renames provider.Renames, wires map[string]string, price fun}// as is priceRow's answer for a gateway call of these names, prov the-// id the provider has now (Renames).+// id the provider has now without a merge (Renames.Own): what it is+// shown under does not change what it cost.func (b *blockPricer) as(prov, model, served, effort string) pricedAs {k := [4]string{prov, model, served, effort}if a, ok := b.by[k]; ok {@@ -121,15 +122,15 @@ func (b *blockPricer) block(packed *rowChunk) *rowChunk {flags uint16price *catalog.Price}- names := map[[5]uint32]ids{}+ names := map[[6]uint32]ids{}agents := map[uint32][2]uint32{}- shapes := map[[2]uint32]ids{}+ shapes := map[[3]uint32]ids{}// the provider each shape names, as it is now: one an id was given// away from since has its calls told by when each was made, so its// rows take the shape per provider they go to (Renames.Timed)type renamedTo struct {- to uint32- timed bool+ to, own uint32 // shown under, and priced as (Renames.Both)+ timed bool}provs := map[uint32]renamedTo{}for i := range c.Rows {@@ -140,21 +141,23 @@ func (b *blockPricer) block(packed *rowChunk) *rowChunk {id := c.Strings[raw]to = renamedTo{timed: b.renames.Timed(id)}if !to.timed {- to.to = intern(b.renames.Of(id, p.Time), raw)+ shown, own := b.renames.Both(id, p.Time)+ to.to, to.own = intern(shown, raw), intern(own, raw)}provs[raw] = to}if to.timed {- to.to = intern(b.renames.Of(c.Strings[raw], p.Time), raw)+ shown, own := b.renames.Both(c.Strings[raw], p.Time)+ to.to, to.own = intern(shown, raw), intern(own, raw)}- sk := [2]uint32{p.Shape, to.to}+ sk := [3]uint32{p.Shape, to.to, to.own}got, ok := shapes[sk]if !ok {t := c.Shapes[p.Shape]- k := [5]uint32{t[1], t[5], t[7], t[8], to.to}+ k := [6]uint32{t[1], t[5], t[7], t[8], to.to, to.own}named, ok := names[k]if !ok {- a := b.as(c.Strings[k[4]], c.Strings[k[1]], c.Strings[k[2]], c.Strings[k[3]])+ a := b.as(c.Strings[k[5]], c.Strings[k[1]], c.Strings[k[2]], c.Strings[k[3]])named = ids{k[4], a.flags, a.price}names[k] = named}diff --git a/internal/usage/ledger.go b/internal/usage/ledger.goindex 9524a724a..ec4bd8576 100644--- a/internal/usage/ledger.go+++ b/internal/usage/ledger.go@@ -459,7 +459,7 @@ func ledgerMatched(since, until time.Time, f Filter, recs []Record, logs []sessiif !since.IsZero() && r.Time.Before(since) || after(until, r.Time) {continue}- r.Provider = renamed.Of(r.Provider, r.Time)+ r.renameBy(renamed)actor := ""if call, ok := pairs[i]; ok && call >= 0 && call < len(logs) {actor = logs[call].ThreadID@@ -494,7 +494,7 @@ func ledgerMatched(since, until time.Time, f Filter, recs []Record, logs []sessiif !since.IsZero() && r.Time.Before(since) || after(until, r.Time) {continue}- r.Provider = renamed.Of(r.Provider, r.Time)+ r.renameBy(renamed)add(r, priceOf(r), c.Source, "")}// the two logs, by when each call began@@ -530,14 +530,15 @@ func Pricer() func(Record) *catalog.Price {if served == r.Model {served = ""}- k := [3]string{r.Provider, r.Model, served}+ id := r.priceID()+ k := [3]string{id, r.Model, served}if pr, ok := prices[k]; ok {return pr}var pr *catalog.Price- v, ok := provider.EffectivePriceIn(s, r.Provider, r.Model)+ v, ok := provider.EffectivePriceIn(s, id, r.Model)if !ok && served != "" {- v, ok = provider.EffectivePriceIn(s, r.Provider, served)+ v, ok = provider.EffectivePriceIn(s, id, served)}if ok {pr = &v@@ -557,7 +558,7 @@ func NewPricer() func([]Record) Totals {return func(recs []Record) Totals {var sum Totalsfor _, r := range recs {- r.Provider = renamed.Of(r.Provider, r.Time)+ r.renameBy(renamed)sum.add(r, priceOf(r))}return sum@@ -1120,7 +1121,7 @@ func Visit(since time.Time, fn func(Record)) { readLogSnapshot().visit(since, fnfunc NewCoster() func(Record) (float64, bool) {priceOf, renamed := Pricer(), provider.UsageRenames()return func(r Record) (float64, bool) {- r.Provider = renamed.Of(r.Provider, r.Time)+ r.renameBy(renamed)var t Totalst.add(r, priceOf(r))return t.Cost, t.Unpriced == 0diff --git a/internal/usage/rename_usage_test.go b/internal/usage/rename_usage_test.goindex 68252c755..33eae7785 100644--- a/internal/usage/rename_usage_test.go+++ b/internal/usage/rename_usage_test.go@@ -10,6 +10,7 @@ import ("github.com/yetone/magpie/internal/probelog""github.com/yetone/magpie/internal/provider"+ "github.com/yetone/magpie/internal/settings")// A provider renamed is shown by the id it has now everywhere usage names@@ -272,3 +273,143 @@ func TestDeletedProviderKeepsItsCallsFromOneAddedUnderItsID(t *testing.T) {t.Fatalf("summary %v", calls)}}++// CyberCaius on Discord: a deleted provider's calls can be merged into a+// provider there is now, from the Usage page, without renaming either:+// they are listed, filtered and summed as that one's, each still costing+// what it did (priced as the deleted provider's), and parted again on+// undo. A provider added later under the deleted one's id keeps its own+// calls; renaming the provider merged into keeps the merge, and deleting+// it ends it.+func TestDeletedProviderMergedIntoAnother(t *testing.T) {+ reportedHome(t)+ s := settings.Load()+ s.ModelPrices = map[string]settings.ModelPrice{+ "relay/gpt-9": {Input: new(1.0), Output: new(2.0), CacheRead: new(0.0), CacheWrite: new(0.0)},+ "main/gpt-9": {Input: new(100.0), Output: new(200.0), CacheRead: new(0.0), CacheWrite: new(0.0)},+ }+ if err := settings.Save(s); err != nil {+ t.Fatal(err)+ }+ for _, p := range []provider.Provider{+ {ID: "relay", Name: "Old relay", Key: "k", Chat: "http://127.0.0.1:1/v1", Models: []string{"gpt-9"}},+ {ID: "main", Name: "Main", Key: "k2", Chat: "http://127.0.0.1:2/v1", Models: []string{"gpt-9"}},+ } {+ if err := provider.Save(p); err != nil {+ t.Fatal(err)+ }+ }+ then := time.Now().Add(-time.Hour)+ Append(Record{Time: then, Agent: "codex", Session: "s1", Provider: "relay", Model: "gpt-9", Input: 1e6, Output: 1e6, Status: 200})+ Append(Record{Time: then.Add(time.Second), Agent: "codex", Session: "s1", Provider: "relay", Model: "gpt-9", Input: 1e6, Output: 1e6, Status: 200})+ Append(Record{Time: then.Add(2 * time.Second), Agent: "codex", Session: "s2", Provider: "main", Model: "gpt-9", Input: 1e6, Output: 1e6, Status: 200})+ if err := provider.Delete("relay"); err != nil {+ t.Fatal(err)+ }+ before := QueryPage(All, Filter{}, 0, 50)+ if !slices.Equal(before.Providers, []string{"main", "relay"}) {+ t.Fatalf("before: %v", before.Providers)+ }+ cost := func(s Summary) (float64, map[string]int) {+ calls, sum := map[string]int{}, 0.0+ for _, g := range s.Models {+ calls[g.ID] += g.Calls+ sum += g.Cost+ }+ return sum, calls+ }+ sumBefore, _ := cost(Summarize(All))++ if err := provider.MergeDeleted("relay", "main"); err != nil {+ t.Fatal(err)+ }+ page := QueryPage(All, Filter{}, 0, 50)+ if !slices.Equal(page.Providers, []string{"main"}) {+ t.Fatalf("merged: %v", page.Providers)+ }+ for _, r := range page.Rows {+ if r.Provider != "main" {+ t.Fatalf("a row under %q", r.Provider)+ }+ }+ // 2 calls at relay's $3 and one at main's $300, as before the merge+ if page.Sum.Cost != before.Sum.Cost || page.Sum.Cost != 306 {+ t.Fatalf("cost %v, before %v", page.Sum.Cost, before.Sum.Cost)+ }+ if p := QueryPage(All, Filter{Provider: "main"}, 0, 50); p.Sum.Calls != 3 || p.Sum.Cost != 306 {+ t.Fatalf("main's: %+v", p.Sum)+ }+ sumAfter, calls := cost(Summarize(All))+ if sumAfter != sumBefore || calls["main/gpt-9"] != 3 || calls["relay/gpt-9"] != 0 {+ t.Fatalf("summary %v (cost %v, before %v)", calls, sumAfter, sumBefore)+ }+ if vs := Vias(time.Time{}).Of("codex|s1"); len(vs) != 1 || vs[0].Provider != "main" {+ t.Fatalf("vias %+v", vs)+ }+ coster := NewCoster()+ if c, ok := coster(Record{Time: then, Provider: "relay", Model: "gpt-9", Input: 1e6, Output: 1e6}); !ok || c != 3 {+ t.Fatalf("coster %v %v", c, ok)+ }+ if got := NewPricer()([]Record{{Time: then, Provider: "relay", Model: "gpt-9", Input: 1e6, Output: 1e6}}); got.Cost != 3 {+ t.Fatalf("pricer %+v", got)+ }++ // a provider added under relay's id later is not merged+ if err := provider.Save(provider.Provider{ID: "relay", Name: "New relay", Key: "k3", Chat: "http://127.0.0.1:3/v1", Models: []string{"gpt-9"}}); err != nil {+ t.Fatal(err)+ }+ Append(Record{Time: time.Now().Add(time.Second), Agent: "codex", Session: "s3", Provider: "relay", Model: "gpt-9", Input: 1e6, Output: 1e6, Status: 200})+ if p := QueryPage(All, Filter{Provider: "relay"}, 0, 50); p.Sum.Calls != 1 {+ t.Fatalf("the new relay's: %+v", p.Sum)+ }+ mergedSum := QueryPage(All, Filter{Provider: "main"}, 0, 50).Sum+ if mergedSum.Calls != 3 {+ t.Fatalf("main's, a relay added since: %+v", mergedSum)+ }++ // the provider merged into renamed: still merged+ if err := provider.Rename("main", "primary"); err != nil {+ t.Fatal(err)+ }+ if p := QueryPage(All, Filter{Provider: "primary"}, 0, 50); p.Sum.Calls != 3 {+ t.Fatalf("renamed: %+v (providers %v)", p.Sum, p.Providers)+ }++ // undone: the deleted relay's calls are its own again, by the id the+ // page lists it under now that another has relay+ old := ""+ for _, id := range QueryPage(All, Filter{}, 0, 50).Providers {+ if strings.HasPrefix(id, "relay~") {+ old = id+ }+ }+ if old != "" {+ t.Fatalf("listed apart while merged: %q", old)+ }+ gone := provider.UsageRenames().Merged("primary")+ if len(gone) != 1 || !strings.HasPrefix(gone[0], "relay~") {+ t.Fatalf("merged into primary: %v", gone)+ }+ if err := provider.MergeDeleted(gone[0], ""); err != nil {+ t.Fatal(err)+ }+ // the parts sum to what the merge showed, each call as it cost+ parted, own := QueryPage(All, Filter{Provider: gone[0]}, 0, 50).Sum, QueryPage(All, Filter{Provider: "primary"}, 0, 50).Sum+ if parted.Calls != 2 || own.Calls != 1 || own.Cost != 300 || parted.Cost+own.Cost != mergedSum.Cost || parted.Unpriced+own.Unpriced != mergedSum.Unpriced {+ t.Fatalf("parted %+v, primary's own %+v, merged %+v", parted, own, mergedSum)+ }++ // merged again, then the provider merged into deleted: the merge ends+ if err := provider.MergeDeleted(gone[0], "primary"); err != nil {+ t.Fatal(err)+ }+ if err := provider.Delete("primary"); err != nil {+ t.Fatal(err)+ }+ if p := QueryPage(All, Filter{Provider: gone[0]}, 0, 50); p.Sum.Calls != 2 {+ t.Fatalf("its target deleted: %+v (providers %v)", p.Sum, QueryPage(All, Filter{}, 0, 50).Providers)+ }+ if err := provider.MergeDeleted(gone[0], "nobody"); err == nil {+ t.Fatal("merged into a provider there is not")+ }+}diff --git a/internal/usage/request_page.go b/internal/usage/request_page.goindex 3342f1e83..eda9c3262 100644--- a/internal/usage/request_page.go+++ b/internal/usage/request_page.go@@ -866,14 +866,14 @@ func readRequestBlocks(idx *requestIndex, st requestState, p Period, now time.Tireturn func(r Record, source string) Row {sent := r.Modelif source == "" {- sent = provider.SentNameIn(cfg.ModelWires, r.Provider, r.Model, r.Effort)+ sent = provider.SentNameIn(cfg.ModelWires, r.priceID(), r.Model, r.Effort)}row := Row{Record: r, Source: source, Routed: GroupRouted(sent, r.Served)}if r.Served != "" {- k := [4]string{r.Provider, r.Model, sent, r.Served}+ k := [4]string{r.priceID(), r.Model, sent, r.Served}s, ok := swapped[k]if !ok {- s = Swapped(sent, r.Served) && !ListedAs(r.Provider, r.Model, r.Served)+ s = Swapped(sent, r.Served) && !ListedAs(r.priceID(), r.Model, r.Served)swapped[k] = s}row.Swapped = s@@ -1000,7 +1000,7 @@ func readRequestBlocks(idx *requestIndex, st requestState, p Period, now time.Tifor i, call := range d.Calls {r := call.Recordr.Computer = ""- r.Provider = renames.Of(r.Provider, r.Time)+ r.renameBy(renames)c.add(priceRow(r, call.Source), "", int64(i), false)}c.Bytes -= int64(32 * len(c.Strings))diff --git a/internal/usage/usage.go b/internal/usage/usage.goindex 31539244f..1b98f6e7e 100644--- a/internal/usage/usage.go+++ b/internal/usage/usage.go@@ -225,6 +225,30 @@ type Record struct {// export with bodies on, #538) and never written to usage.jsonl.BodyIn string `json:"-"`BodyOut string `json:"-"`+ // pricedAs is the provider the call is priced as when it is shown+ // under another: a deleted provider's merged into one on the Usage+ // page, which costs what it did (provider.Renames.Both). Never kept.+ pricedAs string+}++// renameBy has r's provider the one its calls are shown under now+// (Renames.Of), and keeps the one it is priced as beside it when they+// differ.+func (r *Record) renameBy(rn provider.Renames) {+ shown, own := rn.Both(r.Provider, r.Time)+ r.Provider, r.pricedAs = shown, ""+ if own != shown {+ r.pricedAs = own+ }+}++// priceID is the provider r is priced as: its own, not one it is shown+// under after a merge (renameBy).+func (r Record) priceID() string {+ if r.pricedAs != "" {+ return r.pricedAs+ }+ return r.Provider}// Path is the log file: ~/.config/magpie/usage.jsonl (XDG-aware).@@ -935,7 +959,7 @@ func summarizeFrom(p Period, now time.Time, first time.Time, historicalKeys map[if r.Time.Before(from) || after(until, r.Time) {return}- r.Provider = renamed.Of(r.Provider, r.Time)+ r.renameBy(renamed)fn(r)})}diff --git a/internal/usage/window_holds.go b/internal/usage/window_holds.goindex ea5b68d47..84152cae8 100644--- a/internal/usage/window_holds.go+++ b/internal/usage/window_holds.go@@ -221,7 +221,8 @@ func windowHolds(qs []provider.SubscriptionQuota, now, first time.Time, priceOfif r.IsRejected() || r.IsReported() || r.Input+r.Output == 0 {return}- r.Provider = renamed.Of(r.Provider, r.Time)+ // who answered, not who a merge shows it under (Renames.Own)+ r.Provider = renamed.Own(r.Provider, r.Time)acct := r.Provider + "\x00" + strings.ToLower(r.Account())var runs []runKeyfor i, d := range dry[acct] {@@ -326,13 +327,14 @@ func apiPricer() func(Record) *catalog.Price {if served == r.Model {served = ""}- k := [3]string{r.Provider, r.Model, served}+ id := r.priceID()+ k := [3]string{id, r.Model, served}if pr, ok := prices[k]; ok {return pr}- pr := one(r.Provider, r.Model)+ pr := one(id, r.Model)if pr == nil && served != "" {- pr = one(r.Provider, served)+ pr = one(id, served)}prices[k] = prreturn prdiff --git a/internal/usageapi/usageapi.go b/internal/usageapi/usageapi.goindex cb3ac2987..3bd404f1d 100644--- a/internal/usageapi/usageapi.go+++ b/internal/usageapi/usageapi.go@@ -149,10 +149,7 @@ func providerFaces(all []provider.Provider) func(id string) Named {a.Name, a.Icon = p.Name, p.Icon} else if d, ok := renames.Gone(id); ok {at := d.At- a.Name, a.Icon, a.Deleted = d.Name, d.Icon, &at- if a.Name == "" {- a.Name = d.ID- }+ a.Name, a.Icon, a.Deleted = d.Shown(), d.Icon, &at} else if id == usage.UnknownProvider {a.Name = "Local session"} else {@@ -164,6 +161,13 @@ func providerFaces(all []provider.Provider) func(id string) Named {if a.Icon == "" {a.Icon = "generic"}+ // the deleted providers the user merged into it, so the page can+ // part them again+ for _, m := range renames.Merged(id) {+ if d, ok := renames.Gone(m); ok {+ a.Merged = append(a.Merged, Merged{ID: m, Name: d.Shown()})+ }+ }return a}}@@ -285,6 +289,15 @@ type Named struct {// Deleted is when the provider was deleted, for one whose calls are// kept apart from a provider's added under its id sinceDeleted *time.Time `json:"deleted,omitempty"`+ // Merged are the deleted providers whose calls the user merged into+ // this one (provider.MergeDeleted), by the ids they are kept apart by+ Merged []Merged `json:"merged,omitempty"`+}++// Merged is a deleted provider merged into another.+type Merged struct {+ ID string `json:"id"`+ Name string `json:"name"`}// LedgerPage is one page of the ledger: limit rows (100 when none isdiff --git a/internal/usageapi/usageapi_test.go b/internal/usageapi/usageapi_test.goindex e624f6d44..e1f177adb 100644--- a/internal/usageapi/usageapi_test.go+++ b/internal/usageapi/usageapi_test.go@@ -114,3 +114,67 @@ func TestDeletedProviderListedApartAndMarked(t *testing.T) {t.Errorf("the deleted one: %+v (all %+v)", n, got)}}++// CyberCaius on Discord: a deleted provider can be shown by a name of the+// user's, and once merged into another it is listed as that one, which+// names it among its merged ones so the page can part them again.+func TestDeletedProviderNamedAndMerged(t *testing.T) {+ home := t.TempDir()+ testenv.SetHome(t, home)+ t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config"))+ t.Setenv("XDG_CACHE_HOME", filepath.Join(home, ".cache"))+ for _, p := range []provider.Provider{+ {ID: "relay", Key: "k", Chat: "http://127.0.0.1:1/v1", Models: []string{"gpt-9"}},+ {ID: "main", Name: "Main", Key: "k2", Chat: "http://127.0.0.1:2/v1", Models: []string{"gpt-9"}},+ } {+ if err := provider.Save(p); err != nil {+ t.Fatal(err)+ }+ }+ usage.Append(usage.Record{Time: time.Now().Add(-time.Hour), Agent: "codex", Provider: "relay", Model: "gpt-9", Input: 10, Output: 2})+ usage.Append(usage.Record{Time: time.Now().Add(-time.Minute), Agent: "codex", Provider: "main", Model: "gpt-9", Input: 10, Output: 2})+ if err := provider.Delete("relay"); err != nil {+ t.Fatal(err)+ }+ faces := func() map[string]Named {+ got := map[string]Named{}+ for _, p := range LedgerPage(usage.All, usage.Filter{}, 0, 100).Providers {+ got[p.ID] = p+ }+ return got+ }+ if err := provider.NameDeleted("relay", " Leee (old key) "); err != nil {+ t.Fatal(err)+ }+ if n := faces()["relay"]; n.Name != "Leee (old key)" || n.Deleted == nil {+ t.Fatalf("named: %+v", n)+ }+ if err := provider.NameDeleted("relay", ""); err != nil {+ t.Fatal(err)+ }+ if n := faces()["relay"]; n.Name != "relay" {+ t.Fatalf("its own name again: %+v", n)+ }+ if err := provider.NameDeleted("relay", "Leee"); err != nil {+ t.Fatal(err)+ }+ if err := provider.MergeDeleted("relay", "main"); err != nil {+ t.Fatal(err)+ }+ got := faces()+ if _, ok := got["relay"]; ok || len(got) != 1 {+ t.Fatalf("merged, still listed: %+v", got)+ }+ if m := got["main"].Merged; len(m) != 1 || m[0].ID != "relay" || m[0].Name != "Leee" {+ t.Fatalf("main's merged: %+v", got["main"])+ }+ if err := provider.MergeDeleted("relay", ""); err != nil {+ t.Fatal(err)+ }+ if got := faces(); got["relay"].Name != "Leee" || got["main"].Merged != nil {+ t.Fatalf("parted: %+v", got)+ }+ if err := provider.NameDeleted("main", "x"); err == nil {+ t.Fatal("named a provider that is not deleted")+ }+}