Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.
usage: magpie's own Test, Detect APIs and model list requests are rows of the request log, counted nowhere
yetoneffe493773c86
diff --git a/docs/subsystems/gateway-sessions.md b/docs/subsystems/gateway-sessions.mdindex e510bc74f..66b78f882 100644--- a/docs/subsystems/gateway-sessions.md+++ b/docs/subsystems/gateway-sessions.md@@ -408,6 +408,44 @@ however old, so a client's history can be backfilled.`TestUsageIngest*` and `TestIngestReported*` cover these, includingconcurrent posts beside the gateway's own writes, and a backfill of a full5000-record batch landing on the right 7d/30d/all days.++### Requests magpie sends on its own++The provider editor's Test, Detect APIs and per-model detection, and a+model list's refresh, reach the vendor outside any agent's turn (neiko and+gys619 on Discord: a test left no trace). Each such request is one row of+`usage.jsonl` with `source: "probe"`, agent `magpie`, the provider's id+(the host for one not yet saved), its account, model, endpoint, status,+time and the error clipped to 500 bytes, and a `kind` saying what it was+for: `provider_test`, `api_detect` or `model_list`+([`internal/probelog`](../../internal/probelog/probelog.go)).++- **Marking:** `Provider.ownRequests` marks the context in `Test`,+ `TestModels`, `Detect`, `DetectModels`, `DetectDecide`, `refetch`,+ `fetch`, `List`, `fetchOne` and `fetchDue`; a kind set inside another+ replaces it, so the list a Test fetches is a model list. A request is+ written once: by `Provider.Do`, which then marks the context taken, or+ else by `probelog.Transport`, which `netproxy.Install` puts under+ `http.DefaultClient` for the catalog's list fetches. A Claude account's+ test (the Claude Code CLI) and the translated test (through the+ gateway) are one row each from their result (`NoteResult`). A plugin's+ model list runs in the plugin host and is not written. Only the host and+ path of a URL are kept, never its query (Gemini's `?key=`) or userinfo.+- **Counted nowhere:** the log's index keeps these rows out of its blocks+ (`logSnapshot.probes`), so `Visit`, `Load`, `Summarize`, a gateway key's+ budget, a window's holds, `Vias`, the heatmap, shared days, `LastSeen`+ and the observers (`Append` skips `observeLedger`) never meet them. The+ request page lists them as a chunk of their own (`probeChunk`) and adds+ them to no total, cell, group, chart or filter count; `?via=` other than+ all leaves them out.+- **Showing:** the row's badge is the kind's name ("Test", "Detect APIs",+ "Model list", from `purposeKinds`), and its tooltip and the "magpie's+ own" badge's say it counts in no total (`probeWhy` in+ `plugins/usage/page.js`).++`TestProbeRowsListedNotCounted`, `TestTestWritesItsRequestsDown`,+`TestDetectWritesItsRequestsDown`, `internal/probelog`'s tests and+`usage-ledger-probe.test.cjs` cover these.`TestReportedTouchesNothingLive` and `TestUsageIngestTouchesNoLimit` go redwhen any of the live guards is removed. Browser:`usage-ledger-reported.test.cjs`.diff --git a/internal/gui/assets/app.css b/internal/gui/assets/app.cssindex 903fdad79..9b8a868ca 100644--- a/internal/gui/assets/app.css+++ b/internal/gui/assets/app.css@@ -1440,6 +1440,7 @@ body.window .field.solo { grid-column: auto; }.led .src.access-subscription { --badge-ink: var(--tk-k); }.led .src.proto { --badge-ink: var(--muted); font-weight: 500; }.led .src.reported { --badge-ink: var(--tk-o); }+.led .src.probe { --badge-ink: var(--muted); }.led .duration { font-weight: 550; }.led .duration-fast { color: var(--tk-s); }.led .duration-slow { color: var(--amber); }diff --git a/internal/gui/assets/plugins/usage/i18n.js b/internal/gui/assets/plugins/usage/i18n.jsindex 2d3197cd6..76f114a19 100644--- a/internal/gui/assets/plugins/usage/i18n.js+++ b/internal/gui/assets/plugins/usage/i18n.js@@ -45,6 +45,7 @@ magpie.strings({"{n} cache read": "缓存读取 {n}","reported by apps": "由应用上报","Reported by the app that made it, through POST /v1/magpie/usage/requests: the call went straight to its vendor, not through magpie.": "由发起请求的应用通过 POST /v1/magpie/usage/requests 上报:这次请求直接发给了服务商,没有经过 magpie。",+ "magpie sent this itself to test the provider (Test, Detect APIs or a model list), not for an agent. It is not counted in any total.": "这是 magpie 自己为测试供应商发出的请求(测试、检测协议或模型列表),不是 agent 发出的,不计入任何汇总。","all requests": "全部请求","Show only these requests": "只看这些请求","Show every request again": "重新显示全部请求",@@ -246,6 +247,7 @@ magpie.strings({"{n} cache read": "快取讀取 {n}","reported by apps": "由應用程式回報","Reported by the app that made it, through POST /v1/magpie/usage/requests: the call went straight to its vendor, not through magpie.": "由發出請求的應用程式透過 POST /v1/magpie/usage/requests 回報:這次請求直接送往服務商,沒有經過 magpie。",+ "magpie sent this itself to test the provider (Test, Detect APIs or a model list), not for an agent. It is not counted in any total.": "這是 magpie 自己為測試供應商發出的請求(測試、檢測協議或模型列表),不是 agent 發出的,不計入任何彙總。","all requests": "全部請求","Show only these requests": "只看這些請求","Show every request again": "重新顯示全部請求",@@ -447,6 +449,7 @@ magpie.strings({"{n} cache read": "キャッシュ読み取り {n}","reported by apps": "アプリからの報告","Reported by the app that made it, through POST /v1/magpie/usage/requests: the call went straight to its vendor, not through magpie.": "リクエストを送ったアプリが POST /v1/magpie/usage/requests で報告したものです。このリクエストは magpie を経由せず、直接ベンダーに送られました。",+ "magpie sent this itself to test the provider (Test, Detect APIs or a model list), not for an agent. It is not counted in any total.": "プロバイダをテストするために magpie 自身が送ったリクエストです(テスト、API を検出、モデル一覧)。エージェントのものではなく、どの合計にも数えません。","all requests": "すべてのリクエスト","Show only these requests": "これらのリクエストだけを表示","Show every request again": "すべてのリクエストを再表示",@@ -648,6 +651,7 @@ magpie.strings({"{n} cache read": "{n} Cache-Lesen","reported by apps": "von Apps gemeldet","Reported by the app that made it, through POST /v1/magpie/usage/requests: the call went straight to its vendor, not through magpie.": "Von der App gemeldet, die die Anfrage gestellt hat (POST /v1/magpie/usage/requests): Die Anfrage ging direkt an den Anbieter, nicht über magpie.",+ "magpie sent this itself to test the provider (Test, Detect APIs or a model list), not for an agent. It is not counted in any total.": "magpie hat dies selbst gesendet, um den Anbieter zu testen (Testen, APIs erkennen oder eine Modellliste), nicht für einen Agent. Es zählt in keiner Summe.","all requests": "alle Anfragen","Show only these requests": "Nur diese Anfragen zeigen","Show every request again": "Wieder alle Anfragen zeigen",diff --git a/internal/gui/assets/plugins/usage/page.js b/internal/gui/assets/plugins/usage/page.jsindex b4934c1b7..a4705bf16 100644--- a/internal/gui/assets/plugins/usage/page.js+++ b/internal/gui/assets/plugins/usage/page.js@@ -337,6 +337,8 @@ function costHead(c, note, title) {cost.append(el("span", note ? "unpriced" : "", note || t("effective prices")));cost.title = title;}+// a request magpie sent a provider by itself to test it (SourceProbe)+const probeWhy = () => t("magpie sent this itself to test the provider (Test, Detect APIs or a model list), not for an agent. It is not counted in any total.");const unpricedCalls = (n) => t(n === 1 ? "{n} call unpriced" : "{n} calls unpriced", { n: ledNum(n) });const unpricedCallsWhy = (n) => n ? t(n === 1 ? "{n} call had no known price and is not counted" : "{n} calls had no known price and are not counted", { n }) : t("Estimated using effective model prices, including custom prices");// renderCosts redraws whatever on the Usage page shows a cost, once the@@ -1936,10 +1938,17 @@ function renderLedger() {// one made on another computer, whose usage sync brought (#542)const at = r.computerName ? t("{agent} · on {computer}", { agent: name, computer: r.computerName }) : name;who.append(icon(r.icon || "generic"), el("span", "", [r.via ? t("{agent} · via {host}", { agent: at, host: r.via }) : at, r.callerKeyLabel || r.callerKeyName].filter(Boolean).join(" · ")));- // a request magpie made by itself, apart from the agents' own (#1839)- if (r.agent === "magpie") who.append(ownBadge());+ // a request magpie made by itself, apart from the agents' own (#1839);+ // one that tested the provider (the editor's Test, Detect APIs, a model+ // list) is counted nowhere, and says so (neiko on Discord)+ const probe = r.source === "probe";+ if (r.agent === "magpie") {+ const own = ownBadge();+ if (probe) own.title = probeWhy();+ who.append(own);+ }td(who, "", [r.kind, r.session && t("session {id}", { id: r.session })].filter(Boolean).join(" · "));- td(r.req || "—", "model" + (r.req ? "" : " faint"), r.req || t("Not kept for requests before this version"));+ td(r.req || "—", "model" + (r.req ? "" : " faint"), r.req || (probe ? "" : t("Not kept for requests before this version")));const local = r.source === "log";// A session file names the provider it was set up to call, which is// where a call that went straight to a vendor went: nothing stood@@ -1981,6 +1990,11 @@ function renderLedger() {k.title = t("Reported by the app that made it, through POST /v1/magpie/usage/requests: the call went straight to its vendor, not through magpie.");badges.append(k);}+ if (probe) {+ const k = el("span", "src probe", window.kindName ? window.kindName(r.kind) : r.kind);+ k.title = probeWhy();+ badges.append(k);+ }if (badges.childElementCount) wc.append(badges);// the model sent is what the gateway sent the vendor: a session file has none// The log names a model, but does not capture the outbound HTTP request.diff --git a/internal/gui/assets/purposes.js b/internal/gui/assets/purposes.jsindex 0da8b0ce7..661504cc4 100644--- a/internal/gui/assets/purposes.js+++ b/internal/gui/assets/purposes.js@@ -12,6 +12,10 @@ const REQUEST_KINDS = {"purpose": "kind:ambient_suggestions","name": "Suggestions"},+ "api_detect": {+ "purpose": "kind:api_detect",+ "name": "Detect APIs"+ },"auto_review": {"purpose": "kind:guardian","name": "Approval check"@@ -52,6 +56,14 @@ const REQUEST_KINDS = {"purpose": "kind:memory_consolidation","name": "Memory"},+ "model_list": {+ "purpose": "kind:model_list",+ "name": "Model list"+ },+ "provider_test": {+ "purpose": "kind:provider_test",+ "name": "Test"+ },"review": {"purpose": "kind:review","name": "Review"diff --git a/internal/gui/providers_detect_draft_test.go b/internal/gui/providers_detect_draft_test.goindex e9a2d0d5d..ec9b6c4c2 100644--- a/internal/gui/providers_detect_draft_test.go+++ b/internal/gui/providers_detect_draft_test.go@@ -148,9 +148,19 @@ func TestProviderDetectDraft(t *testing.T) {if got := keys(); len(got) != 1 || got[0] != "sk-typed" {t.Fatalf("copy with a key typed asked with %v", got)}- if after := draftFiles(t, appdir.Config()); !draftSame(before, after) {+ // the request log is the one file a detection writes: each request it+ // sent, as magpie's own (usage.SourceProbe), and nothing else changes+ after := draftFiles(t, appdir.Config())+ logBefore, logAfter := before["usage.jsonl"], after["usage.jsonl"]+ delete(before, "usage.jsonl")+ delete(after, "usage.jsonl")+ if !draftSame(before, after) {t.Fatalf("a detection wrote to the config folder:\nbefore %v\nafter %v", draftNames(before), draftNames(after))}+ if added := strings.TrimPrefix(logAfter, logBefore); added == logAfter || !strings.Contains(added, `"source":"probe"`) ||+ !strings.Contains(added, `"kind":"api_detect"`) || strings.Contains(added, "sk-") {+ t.Fatalf("the request log has no detection of its own:\n%s", added)+ }if ps := provider.All(); len(ps) != 1 || ps[0].ID != "relay" || len(ps[0].Models) != 1 {t.Fatalf("providers now %+v", ps)}diff --git a/internal/gui/tests/usage-ledger-probe.test.cjs b/internal/gui/tests/usage-ledger-probe.test.cjsnew file mode 100644index 000000000..c5a3cc543--- /dev/null+++ b/internal/gui/tests/usage-ledger-probe.test.cjs@@ -0,0 +1,112 @@+// Run with Node's test runner and Playwright on the module path; see README.md.+// A request magpie sent a provider on its own (the editor's Test, Detect+// APIs, a model list: neiko and gys619 on Discord) is a row of the Requests+// tab, with a badge naming what it was for and saying it is counted in no+// total. In every language, Chromium and WebKit, wide and narrow, light and+// dark; no backend, the API is faked.+const assert = require("node:assert/strict");+const fs = require("node:fs/promises");+const path = require("node:path");+const { test } = require("node:test");+const { chromium, webkit } = require("playwright");++const assets = path.resolve(__dirname, "../assets");+const tot = (calls, tokens, cost) => ({ calls, errors: 0, input: tokens * 0.2, output: tokens * 0.1, cache_write: 0, cache_read: tokens * 0.7, cost });+const ROW = { t: new Date().toISOString(), agent: "claude", agentName: "Claude Code", icon: "claudecode-color", provider: "anthropic", providerName: "Claude", model: "claude-sonnet-5", in: 2, out: 600, cache_write: 0, cache_read: 3900, ms: 2380, status: 200, cost: 0.08, priced: true };+const PROBE = { ...ROW, agent: "magpie", agentName: "magpie", icon: "", provider: "relay", providerName: "Relay", model: "gpt-9", in: 0, out: 0, cache_read: 0, ms: 120, status: 401, cost: 0, priced: false, source: "probe", kind: "provider_test", endpoint: "/v1/chat/completions" };++function page(q, reported) {+ const rows = [ROW, ...(reported ? [PROBE] : [])];+ return {+ period: "today", rows, offset: 0, total: rows.length, ...tot(1, 4500, 0.08), reasoning: 0, unpriced: 0,+ bucket: "hour", series: [], by: { provider: [], agent: [], model: [] },+ agents: [{ id: "claude", name: "Claude Code", icon: "claudecode-color" }], providers: [{ id: "anthropic", name: "Claude", icon: "claudecode-color" }],+ through: tot(1, 4500, 0.08), direct: tot(0, 0, 0),+ };+}++function server(lang, theme, reported, asked) {+ return async (route) => {+ const req = route.request(), url = new URL(req.url());+ const json = (data) => route.fulfill({ json: data });+ if (url.pathname === "/boot.js") return route.fulfill({ contentType: "text/javascript", body: `window.bootPrefs = {lang:"${lang}",theme:"${theme}",web:false};` });+ if (url.pathname === "/wails/runtime.js") return route.fulfill({ contentType: "text/javascript", body: "export const Window = {};" });+ if (url.pathname === "/api/state") return json({ agents: [], profiles: [], settings: { lang, theme }, fx: { rate: 7.2, at: new Date().toISOString() } });+ if (url.pathname === "/api/usage/requests") { asked.push(url.searchParams); return json(page(url.searchParams, reported)); }+ if (url.pathname === "/api/usage/quotas") return json([]);+ if (url.pathname === "/api/usage") return json({ calls: 1, errors: 0, input: 1, output: 1, cache_read: 0, cache_write: 0, reasoning: 0, unpriced: 0, cost: 1, bucket: "day", series: [], agents: [], models: [], path: "~/.config/magpie/usage.jsonl" });+ if (url.pathname === "/api/sessions") return json({ sessions: [], dirs: [] });+ if (url.pathname === "/api/sessions/stats") return json({ from: "", to: "", days: [], agents: {} });+ if (url.pathname === "/api/groups") return json({ groups: [], models: [] });+ if (url.pathname.startsWith("/api/")) return json({});+ const file = path.join(assets, url.pathname === "/" ? "index.html" : url.pathname);+ const contentType = { ".html": "text/html", ".js": "text/javascript", ".css": "text/css", ".svg": "image/svg+xml", ".png": "image/png" }[path.extname(file)];+ await route.fulfill({ body: await fs.readFile(file), contentType });+ };+}++// the badge, as each language names the editor's Test, and a word of the+// tooltip that says it is counted in nothing+const L = {+ en: { badge: "Test", why: "not counted in any total" },+ zh: { badge: "测试", why: "不计入任何汇总" },+ "zh-TW": { badge: "測試", why: "不計入任何彙總" },+ ja: { badge: "テスト", why: "どの合計にも数えません" },+ de: { badge: "Testen", why: "zählt in keiner Summe" },+};++for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium", "webkit"])) {+ test(engine + ": magpie's own test is a row of its own", async (t) => {+ assert(["chromium", "webkit"].includes(engine), "BROWSER must be chromium or webkit");+ const browser = await (engine === "webkit" ? webkit.launch() : chromium.launch({ channel: "chromium" }));+ t.after(() => browser.close());+ const shots = process.env.ARTIFACT_DIR;+ if (shots) await fs.mkdir(shots, { recursive: true });++ const open = async (lang, { theme = "light", width = 1180, reported = true } = {}) => {+ const errors = [], asked = [];+ const context = await browser.newContext({ viewport: { width, height: 760 }, reducedMotion: "reduce" });+ t.after(() => context.close());+ const p = await context.newPage();+ p.setDefaultTimeout(5000);+ p.on("pageerror", (e) => errors.push(e.message));+ await p.route("**/*", server(lang, theme, reported, asked));+ await p.goto("http://magpie.test/");+ await p.locator('[data-view="usage"]').first().click();+ await p.locator(":is(#usageTab .opt, .nav-s):visible").nth(1).click();+ await p.locator("#ledVia .blk").first().waitFor();+ return { p, errors };+ };++ for (const lang of Object.keys(L)) {+ const w = L[lang];+ for (const width of [1180, 420]) {+ await t.test(`${lang} at ${width}`, async () => {+ const { p, errors } = await open(lang, { width, theme: width < 600 ? "dark" : "light" });+ const badge = p.locator(".led .src.probe");+ await badge.first().waitFor();+ assert.equal(await badge.count(), 1);+ assert.equal(await badge.textContent(), w.badge);+ assert((await badge.getAttribute("title")).includes(w.why), await badge.getAttribute("title"));+ // the gateway's row has none, and the probe's own badge says it isn't counted+ assert.equal(await p.locator(".led .src.reported").count(), 0);+ const own = p.locator(".led tr:has(.src.probe) .own-badge");+ assert.equal(await own.count(), 1, "the row is magpie's own");+ assert((await own.getAttribute("title")).includes(w.why), "and its badge says it counts nowhere");+ // no coloured stripe down its side, and the page doesn't scroll sideways+ assert(parseFloat(await badge.evaluate((b) => getComputedStyle(b).borderLeftWidth)) <= 1);+ assert(await p.evaluate(() => document.documentElement.scrollWidth <= innerWidth + 1), "no sideways scroll");+ if (shots) await p.screenshot({ path: path.join(shots, `probe-${engine}-${lang}-${width}.png`) });+ assert.deepEqual(errors, []);+ });+ }+ }++ await t.test("no probe: no badge", async () => {+ const { p, errors } = await open("en", { reported: false });+ await p.locator(".led tr", { hasText: "claude-sonnet-5" }).first().waitFor();+ assert.equal(await p.locator(".led .src.probe").count(), 0);+ assert.deepEqual(errors, []);+ });+ });+}diff --git a/internal/netproxy/netproxy.go b/internal/netproxy/netproxy.goindex 07da998e9..6fae26f3a 100644--- a/internal/netproxy/netproxy.go+++ b/internal/netproxy/netproxy.go@@ -20,20 +20,24 @@ import ("sync""time"+ "github.com/yetone/magpie/internal/probelog""github.com/yetone/magpie/internal/settings")// Install routes http.DefaultTransport (and so http.DefaultClient) through// Func, and http.DefaultClient's requests made on a provider's behalf// (With) through a transport of that provider's proxy (Dispatch). Its-// HTTP/2 connections are checked as the gateway's are (HTTP2Health).+// HTTP/2 connections are checked as the gateway's are (HTTP2Health), and+// the requests magpie sends on its own are written down (probelog).func Install() {if t, ok := http.DefaultTransport.(*http.Transport); ok {t.Proxy = Funcif t.HTTP2 == nil {t.HTTP2 = HTTP2Health()}- http.DefaultClient.Transport = Dispatch(t)+ // a request magpie sends a provider on its own (a Test, a model+ // list) is written into the request log as it goes (probelog)+ http.DefaultClient.Transport = probelog.Transport(Dispatch(t))}}diff --git a/internal/probelog/probelog.go b/internal/probelog/probelog.gonew file mode 100644index 000000000..c44985604--- /dev/null+++ b/internal/probelog/probelog.go@@ -0,0 +1,192 @@+// Package probelog writes down the requests magpie sends a provider on its+// own behalf, outside any agent's turn: the provider editor's Test, its+// Detect APIs and per-model detection, and a model list's refresh (neiko+// and gys619 on Discord: a test left no trace in the request log). Each+// such request is one row of the request log, marked as magpie's own test+// (usage.SourceProbe) and never counted in a usage or cost total.+//+// The provider package marks a context with the kind of test (With); every+// request sent under it is written down once, by the provider's Do or by+// the transport the HTTP client goes through (Transport), whichever sends+// it first. Package usage sets Sink.+package probelog++import (+ "context"+ "net/http"+ "strings"+ "time"+)++// The kinds a row is written under, as the request log's "Called for"+// names them (usage.purposeKinds).+const (+ KindTest = "provider_test" // the editor's Test, a model's test+ KindDetect = "api_detect" // Detect APIs, a model's API detection+ KindList = "model_list" // a model list asked for, refreshed+)++// Row is one request written down.+type Row struct {+ Time time.Time+ Kind string+ Provider string // the provider's id, or the host for one not yet saved+ Account string+ Model string+ Host string+ Endpoint string+ Status int+ Millis int64+ Error string+}++// Sink takes each row; nil writes nothing. usage sets it to its log.+var Sink func(Row)++type meta struct {+ kind, provider, account, model string+ done bool+}++type key struct{}++func from(ctx context.Context) (meta, bool) {+ if ctx == nil {+ return meta{}, false+ }+ m, ok := ctx.Value(key{}).(meta)+ return m, ok+}++// With marks ctx: the requests sent under it are of kind, for provider (its+// id) and account. A kind set inside another replaces it, so a model list+// fetched for a Test is written as a model list. The model set before stays.+func With(ctx context.Context, kind, provider, account string) context.Context {+ m, _ := from(ctx)+ m.kind, m.provider, m.account, m.done = kind, provider, account, false+ return context.WithValue(ctx, key{}, m)+}++// WithModel is ctx with the model the requests under it ask, for a ctx+// With marked; any other ctx is returned as it is.+func WithModel(ctx context.Context, model string) context.Context {+ m, ok := from(ctx)+ if !ok {+ return ctx+ }+ m.model = model+ return context.WithValue(ctx, key{}, m)+}++// Kind is the kind ctx is marked with, "" for none.+func Kind(ctx context.Context) string {+ m, _ := from(ctx)+ return m.kind+}++// Taken is ctx with its requests already written down by its caller, so+// the transport under it doesn't write them again.+func Taken(ctx context.Context) context.Context {+ m, ok := from(ctx)+ if !ok || m.done {+ return ctx+ }+ m.done = true+ return context.WithValue(ctx, key{}, m)+}++// Pending is whether a request sent under ctx is still to be written down:+// ctx is marked and no caller has taken it.+func Pending(ctx context.Context) bool {+ m, ok := from(ctx)+ return ok && !m.done && m.kind != ""+}++// Note writes down one request under ctx's mark, sent to u (a URL; only+// its host and path are kept) at start, answered status or failed with+// err. Nothing is written for a ctx with no mark.+func Note(ctx context.Context, u, method string, start time.Time, status int, err error) {+ m, ok := from(ctx)+ if !ok || m.kind == "" || Sink == nil {+ return+ }+ host, path := split(u)+ r := Row{Time: start, Kind: m.kind, Provider: m.provider, Account: m.account, Model: m.model,+ Host: host, Endpoint: path, Status: status, Millis: time.Since(start).Milliseconds()}+ if method != "" && method != http.MethodPost {+ r.Endpoint = method + " " + path+ }+ if err != nil {+ r.Error = err.Error()+ }+ if r.Provider == "" {+ r.Provider = host // a provider being added has no id yet+ }+ if r.Provider == "" {+ r.Provider = "magpie" // never "": the log reads that as turned away+ }+ Sink(r)+}++// NoteResult writes down one request that isn't sent through an HTTP+// client of magpie's (Claude Code run for a test, the gateway's translated+// test), from what came of it.+func NoteResult(ctx context.Context, endpoint string, start time.Time, status int, errText string) {+ m, ok := from(ctx)+ if !ok || m.kind == "" || Sink == nil {+ return+ }+ r := Row{Time: start, Kind: m.kind, Provider: m.provider, Account: m.account, Model: m.model,+ Endpoint: endpoint, Status: status, Millis: time.Since(start).Milliseconds(), Error: errText}+ if r.Provider == "" {+ r.Provider = "magpie"+ }+ Sink(r)+}++func split(u string) (host, path string) {+ s := u+ if i := strings.Index(s, "://"); i >= 0 {+ s = s[i+3:]+ }+ host, path = s, "/"+ if i := strings.IndexAny(s, "/?#"); i >= 0 {+ host, path = s[:i], s[i:]+ }+ if i := strings.IndexAny(path, "?#"); i >= 0 {+ path = path[:i] // a query can carry a key (Gemini's ?key=)+ }+ if path == "" {+ path = "/"+ }+ if i := strings.LastIndex(host, "@"); i >= 0 {+ host = host[i+1:] // never a URL's user:password+ }+ return host, path+}++// Transport writes down each request sent through next under a marked+// context no caller took. Every request the HTTP client sends goes through+// it (netproxy.Install), so a model list asked through the client of any+// package is written as the provider's own requests are.+func Transport(next http.RoundTripper) http.RoundTripper { return transport{next} }++type transport struct{ next http.RoundTripper }++func (t transport) RoundTrip(req *http.Request) (*http.Response, error) {+ ctx := req.Context()+ if !Pending(ctx) {+ return t.next.RoundTrip(req)+ }+ start := time.Now()+ res, err := t.next.RoundTrip(req)+ status := 0+ if err == nil {+ status = res.StatusCode+ }+ Note(ctx, req.URL.String(), req.Method, start, status, err)+ return res, err+}++// Unwrap is the transport under it, for code that looks for netproxy's.+func (t transport) Unwrap() http.RoundTripper { return t.next }diff --git a/internal/probelog/probelog_test.go b/internal/probelog/probelog_test.gonew file mode 100644index 000000000..19d8c0bd6--- /dev/null+++ b/internal/probelog/probelog_test.go@@ -0,0 +1,66 @@+package probelog++import (+ "context"+ "errors"+ "net/http"+ "net/http/httptest"+ "testing"+ "time"+)++func catch(t *testing.T) *[]Row {+ t.Helper()+ var rows []Row+ old := Sink+ Sink = func(r Row) { rows = append(rows, r) }+ t.Cleanup(func() { Sink = old })+ return &rows+}++// A row keeps a URL's host and path, never its query (Gemini's ?key=) or+// its user:password; a GET says so; a provider not yet saved goes by its+// host; a context with no mark writes nothing.+func TestNoteKeepsNoSecret(t *testing.T) {+ rows := catch(t)+ ctx := WithModel(With(context.Background(), KindList, "", ""), "m")+ Note(ctx, "https://user:[email protected]/v1beta/models?key=sk-1#x", http.MethodGet, time.Now(), 200, nil)+ Note(context.Background(), "https://api.example/v1", http.MethodPost, time.Now(), 200, nil)+ Note(With(context.Background(), KindTest, "relay", "me"), "https://api.example", http.MethodPost, time.Now(), 0, errors.New("dial: refused"))+ if len(*rows) != 2 {+ t.Fatalf("%+v", *rows)+ }+ a, b := (*rows)[0], (*rows)[1]+ if a.Host != "api.example" || a.Endpoint != "GET /v1beta/models" || a.Provider != "api.example" || a.Kind != KindList || a.Model != "m" {+ t.Errorf("%+v", a)+ }+ if b.Endpoint != "/" || b.Provider != "relay" || b.Account != "me" || b.Error != "dial: refused" || b.Model != "" {+ t.Errorf("%+v", b)+ }+}++// The transport writes a marked request down once, and none that a caller+// took or that has no mark; a kind set inside another replaces it and+// keeps the model.+func TestTransportOnce(t *testing.T) {+ rows := catch(t)+ srv := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) { rw.WriteHeader(http.StatusTeapot) }))+ defer srv.Close()+ c := &http.Client{Transport: Transport(http.DefaultTransport)}+ marked := WithModel(With(context.Background(), KindTest, "p", ""), "m")+ inner := With(marked, KindList, "p", "")+ for _, ctx := range []context.Context{marked, Taken(marked), context.Background(), inner} {+ req, _ := http.NewRequestWithContext(ctx, http.MethodPost, srv.URL+"/x", nil)+ res, err := c.Do(req)+ if err != nil {+ t.Fatal(err)+ }+ res.Body.Close()+ }+ if len(*rows) != 2 || (*rows)[0].Status != http.StatusTeapot || (*rows)[0].Kind != KindTest || (*rows)[1].Kind != KindList || (*rows)[1].Model != "m" {+ t.Fatalf("%+v", *rows)+ }+ if Pending(Taken(marked)) || !Pending(inner) || Kind(inner) != KindList {+ t.Fatal("marks")+ }+}diff --git a/internal/provider/claude_probe.go b/internal/provider/claude_probe.goindex ccd1bc471..b77c422d6 100644--- a/internal/provider/claude_probe.go+++ b/internal/provider/claude_probe.go@@ -6,7 +6,10 @@ package providerimport ("context""errors"+ "net/http""time"++ "github.com/yetone/magpie/internal/probelog")// errClaudeViaCLI is what a request straight to Anthropic's API on a@@ -46,8 +49,15 @@ func (p Provider) testClaude(ctx context.Context, model string) Result {}start := time.Now()version := ClaudeLoginVersion(p.Account.User)- err = claudeCLIProbe(ctx, dir, model)+ err = claudeCLIProbe(probelog.Taken(ctx), dir, model)r.Millis = time.Since(start).Milliseconds()+ // Claude Code sent it, not a client of magpie's: written down from+ // what it came to+ status, msg := http.StatusOK, ""+ if err != nil {+ status, msg = 0, err.Error()+ }+ probelog.NoteResult(probelog.WithModel(ctx, model), "claude -p", start, status, msg)if err != nil {NoteClaudeSignInFailure(p.Account.User, version, err.Error())r.Error = err.Error()diff --git a/internal/provider/decide.go b/internal/provider/decide.goindex 04334b0c8..d41769aec 100644--- a/internal/provider/decide.go+++ b/internal/provider/decide.go@@ -25,6 +25,7 @@ import ("time""github.com/yetone/magpie/internal/catalog"+ "github.com/yetone/magpie/internal/probelog""github.com/yetone/magpie/internal/settings")@@ -929,7 +930,7 @@ func (p Provider) AskSystemOne(ctx context.Context, model string) error {"questions": map[string]any{"ok": map[string]any{"type": "noul", "instructions": "Is this a test?"}}})via := p.DecideVia()body = DecideAsk(via, model, body)- req, err := http.NewRequestWithContext(ctx, http.MethodPost, u, strings.NewReader(string(body)))+ req, err := http.NewRequestWithContext(probelog.WithModel(ctx, model), http.MethodPost, u, strings.NewReader(string(body)))if err != nil {return err}diff --git a/internal/provider/detect.go b/internal/provider/detect.goindex 66e34c0fa..d9be4a030 100644--- a/internal/provider/detect.go+++ b/internal/provider/detect.go@@ -10,6 +10,7 @@ import ("time""github.com/yetone/magpie/internal/catalog"+ "github.com/yetone/magpie/internal/probelog")// Which APIs a vendor or relay serves behind the one URL the user typed is@@ -83,7 +84,7 @@ func (p Provider) Detect(ctx context.Context, base, model string, picked ...striif err != nil {return nil, err}- ctx = q.Via(ctx)+ ctx = q.ownRequests(q.Via(ctx), probelog.KindDetect)model = strings.TrimSpace(model)var ids []stringif model == "" {@@ -196,7 +197,7 @@ func (p Provider) DetectDecide(ctx context.Context, model string) Detection {return d}t0 := time.Now()- err := p.AskSystemOne(p.Via(ctx), model)+ err := p.AskSystemOne(p.ownRequests(p.Via(ctx), probelog.KindDetect), model)d.Millis = time.Since(t0).Milliseconds()if err != nil {d.Error = err.Error()@@ -245,7 +246,7 @@ func (p Provider) DetectModels(ctx context.Context, base string, models []stringif len(ids) > DetectMax {ids = ids[:DetectMax]}- ctx = q.Via(ctx)+ ctx = q.ownRequests(q.Via(ctx), probelog.KindDetect)out := make([]ModelDetection, len(ids))sem := make(chan struct{}, DetectWide)var wg sync.WaitGroupdiff --git a/internal/provider/models.go b/internal/provider/models.goindex e51182ec3..3b262ff90 100644--- a/internal/provider/models.go+++ b/internal/provider/models.go@@ -17,6 +17,7 @@ import ("github.com/yetone/magpie/internal/catalog""github.com/yetone/magpie/internal/plugin"+ "github.com/yetone/magpie/internal/probelog""github.com/yetone/magpie/internal/settings")@@ -235,6 +236,7 @@ func (p Provider) RefetchTaking(ctx context.Context) (ms []catalog.Model, droppe// refetch is Refetch, with the ids the provider's list had before it// (none for an account's, a plugin's or a decision-only provider's).func (p Provider) refetch(ctx context.Context) ([]catalog.Model, []string, map[string]bool, error) {+ ctx = p.ownRequests(ctx, probelog.KindList)if p.IsPlugin() {// its plugin's list, read again: a pick it had and has no more// goes as a built-in's does (plugin_relist.go)@@ -357,7 +359,7 @@ func relistPicks(q *Provider, now map[string]bool) bool {// fetch is Fetch without the picks looked after.func (p Provider) fetch(ctx context.Context) ([]catalog.Model, error) {- ctx = p.Via(ctx)+ ctx = p.ownRequests(p.Via(ctx), probelog.KindList)if p.DecideOnly() {return p.fetchDecide(ctx)}@@ -407,7 +409,7 @@ func (p Provider) fetch(ctx context.Context) ([]catalog.Model, error) {// nothing: a provider still being added (#578, the add form's Fetch models)// is shown its vendor's list to pick from before it is saved.func (p Provider) List(ctx context.Context) ([]catalog.Model, error) {- ctx = p.Via(ctx)+ ctx = p.ownRequests(p.Via(ctx), probelog.KindList)if pr := Preset(p.Preset); pr != nil && pr.NoList && strings.TrimSpace(p.ModelsURL) == "" && !p.listRegion(pr) {return catalog.Chat(p.planModels(nil)), nil}@@ -611,7 +613,7 @@ func fetchDue(due []newFetch, timeout time.Duration) (failed bool) {ctx, cancel := context.WithTimeout(context.Background(), timeout)var err errorif f.decide && p.Account == nil {- _, err = p.fetchDecide(p.Via(ctx))+ _, err = p.fetchDecide(p.ownRequests(p.Via(ctx), probelog.KindList))} else {_, err = p.Fetch(ctx)}@@ -642,6 +644,7 @@ type fetched struct {// answered left every other endpoint's models out of the catalog, and so out// of the agents' model lists and of a model's own protocol setting.func (p Provider) fetchOne(ctx context.Context) (fetched, error) {+ ctx = p.ownRequests(ctx, probelog.KindList) // a detection's list tooif u := strings.TrimSpace(p.ModelsURL); u != "" {// asked where the user said, and nowhere else: the base URLs// list nothing, or the wrong thingdiff --git a/internal/provider/plugins.go b/internal/provider/plugins.goindex 4f0fb9c2b..9f82eb2d7 100644--- a/internal/provider/plugins.go+++ b/internal/provider/plugins.go@@ -15,6 +15,7 @@ import ("github.com/yetone/magpie/internal/catalog""github.com/yetone/magpie/internal/plugin"+ "github.com/yetone/magpie/internal/probelog")// A plugin's provider (see internal/plugin) is a subscription like the@@ -682,7 +683,26 @@ func firstOf(ss ...string) string {// Do sends req, through the account's own transport when it has one (a// plugin's), the client the account asks for this request when it asks// for one (ZCode's Start Plan: zcodeStartClient), and client otherwise.+//+// A request sent for magpie's own test (probelog.With) is written into the+// request log here, whichever way it goes: a plugin's fetch and a client+// of the account's own pass no transport that would.func (p Provider) Do(client *http.Client, req *http.Request) (*http.Response, error) {+ if ctx := req.Context(); probelog.Pending(ctx) {+ start := time.Now()+ req = req.WithContext(probelog.Taken(ctx))+ res, err := p.do(client, req)+ status := 0+ if err == nil {+ status = res.StatusCode+ }+ probelog.Note(ctx, req.URL.String(), req.Method, start, status, err)+ return res, err+ }+ return p.do(client, req)+}++func (p Provider) do(client *http.Client, req *http.Request) (*http.Response, error) {if p.Account != nil && p.Account.transport != nil {return p.Account.transport(req)}diff --git a/internal/provider/probelog_test.go b/internal/provider/probelog_test.gonew file mode 100644index 000000000..e5674c61d--- /dev/null+++ b/internal/provider/probelog_test.go@@ -0,0 +1,135 @@+package provider++import (+ "context"+ "net/http"+ "net/http/httptest"+ "path/filepath"+ "sort"+ "strings"+ "sync"+ "sync/atomic"+ "testing"++ "github.com/yetone/magpie/internal/probelog"+)++// probeRows catches what probelog writes down while the test runs, through+// the HTTP client's transport as netproxy.Install sets it.+func probeRows(t *testing.T) func() []probelog.Row {+ t.Helper()+ var mu sync.Mutex+ var rows []probelog.Row+ sink, tr := probelog.Sink, http.DefaultClient.Transport+ probelog.Sink = func(r probelog.Row) { mu.Lock(); rows = append(rows, r); mu.Unlock() }+ next := tr+ if next == nil {+ next = http.DefaultTransport+ }+ http.DefaultClient.Transport = probelog.Transport(next)+ t.Cleanup(func() { probelog.Sink, http.DefaultClient.Transport = sink, tr })+ return func() []probelog.Row {+ mu.Lock()+ defer mu.Unlock()+ return append([]probelog.Row(nil), rows...)+ }+}++func probeHome(t *testing.T) {+ home := t.TempDir()+ t.Setenv("HOME", home)+ t.Setenv("USERPROFILE", home)+ t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config"))+ t.Setenv("XDG_CACHE_HOME", filepath.Join(home, ".cache"))+}++// The editor's Test writes each request it sends down once: the model list+// as a model list, each endpoint's test as a test, with its model and the+// status the vendor answered (neiko, gys619 on Discord: a test left no+// trace in the request log).+func TestTestWritesItsRequestsDown(t *testing.T) {+ probeHome(t)+ rows := probeRows(t)+ var hits atomic.Int32+ srv := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {+ hits.Add(1)+ if r.Method == http.MethodGet {+ _, _ = rw.Write([]byte(`{"data":[{"id":"gpt-6-sol"}]}`))+ return+ }+ if r.URL.Path == "/v1/messages" {+ rw.WriteHeader(http.StatusUnauthorized)+ _, _ = rw.Write([]byte(`{"error":{"message":"bad key"}}`))+ return+ }+ _, _ = rw.Write([]byte(`{}`))+ }))+ defer srv.Close()+ p := Provider{ID: "relay", Name: "Relay", Chat: srv.URL + "/v1", Anthropic: srv.URL,+ Key: "sk-chat", KeyProtocol: Chat, Models: []string{"gpt-6-sol"},+ Keys: []KeyAccount{{Key: "sk-anthropic", Protocol: Anthropic}}}+ p.Test(context.Background())+ got := rows()+ if int(hits.Load()) != len(got) {+ t.Fatalf("%d requests, %d rows: %+v", hits.Load(), len(got), got)+ }+ var lines []string+ for _, r := range got {+ if r.Provider != "relay" || r.Host != strings.TrimPrefix(srv.URL, "http://") {+ t.Errorf("row %+v", r)+ }+ lines = append(lines, r.Kind+" "+r.Endpoint+" "+r.Model+" "+http.StatusText(r.Status))+ }+ sort.Strings(lines)+ s := strings.Join(lines, "\n")+ for _, want := range []string{"model_list GET /v1/models OK", "provider_test /v1/chat/completions gpt-6-sol OK", "provider_test /v1/messages "} {+ if !strings.Contains(s, want) {+ t.Errorf("no %q in rows:\n%s", want, s)+ }+ }+ if !strings.Contains(s, "Unauthorized") {+ t.Errorf("the refused test's status is lost:\n%s", s)+ }+}++// Detect APIs writes its requests down as a detection; a request sent+// with no mark (an agent's turn) is written nowhere.+func TestDetectWritesItsRequestsDown(t *testing.T) {+ probeHome(t)+ rows := probeRows(t)+ var hits atomic.Int32+ srv := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {+ hits.Add(1)+ if r.Method == http.MethodGet {+ _, _ = rw.Write([]byte(`{"data":[{"id":"m1"}]}`))+ return+ }+ _, _ = rw.Write([]byte(`{}`))+ }))+ defer srv.Close()+ if _, err := (Provider{ID: "relay", Key: "k"}).Detect(context.Background(), srv.URL, "m1"); err != nil {+ t.Fatal(err)+ }+ got := rows()+ if len(got) == 0 || int(hits.Load()) != len(got) {+ t.Fatalf("%d requests, %d rows: %+v", hits.Load(), len(got), got)+ }+ for _, r := range got {+ if r.Kind != probelog.KindDetect && r.Kind != probelog.KindList || r.Provider != "relay" {+ t.Errorf("row %+v", r)+ }+ if r.Kind == probelog.KindDetect && !strings.HasPrefix(r.Endpoint, "GET ") && r.Model != "m1" {+ t.Errorf("a detection's row has no model: %+v", r)+ }+ }+ before := len(got)+ req, _ := http.NewRequest(http.MethodPost, srv.URL+"/v1/chat/completions", strings.NewReader(`{}`))+ res, err := (Provider{ID: "relay"}).Do(http.DefaultClient, req)+ if err != nil {+ t.Fatal(err)+ }+ res.Body.Close()+ if len(rows()) != before {+ t.Fatal("an agent's request was written down as magpie's own")+ }+}diff --git a/internal/provider/test.go b/internal/provider/test.goindex 96e761149..e4e704b84 100644--- a/internal/provider/test.go+++ b/internal/provider/test.go@@ -17,6 +17,7 @@ import ("time""github.com/yetone/magpie/internal/catalog"+ "github.com/yetone/magpie/internal/probelog")// Result is what a probe of one endpoint came back with.@@ -34,7 +35,7 @@ type Result struct {// serves, signed with a key made for it and asking for a model that key// sees, and reports what came back.func (p Provider) Test(ctx context.Context) []Result {- ctx = p.Via(ctx)+ ctx = p.ownRequests(p.Via(ctx), probelog.KindTest)if p.DecideOnly() {return p.testDecide(ctx)}@@ -239,7 +240,7 @@ func tinyDrawing(q Provider, model string) (url, body string) {// on the endpoint it's served on and with a key that sees it: whether each// answers, not only whether the vendor does. Results are in models' order.func (p Provider) TestModels(ctx context.Context, models []string) []Result {- ctx = p.Via(ctx)+ ctx = p.ownRequests(p.Via(ctx), probelog.KindTest)out := make([]Result, len(models))sem := make(chan struct{}, 4)var wg sync.WaitGroup@@ -486,13 +487,23 @@ func AuthHeaders(p Provider, proto Protocol) map[string]string {return map[string]string{"Authorization": "Bearer " + p.Key}}+// ownRequests is ctx with the requests sent under it written into the+// request log as magpie's own, of kind (probelog), for p.+func (p Provider) ownRequests(ctx context.Context, kind string) context.Context {+ account := ""+ if p.Account != nil {+ account = p.Account.User+ }+ return probelog.With(ctx, kind, p.ID, account)+}+func probe(ctx context.Context, p Provider, proto Protocol, url string, body []byte, model string, wait time.Duration) Result {r := Result{Protocol: proto, Model: model}if model == "" {r.Error = "no model to try: expose one, or refresh the model list"return r}- ctx, cancel := context.WithTimeout(ctx, wait)+ ctx, cancel := context.WithTimeout(probelog.WithModel(ctx, model), wait)defer cancel()req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(body))if err != nil {diff --git a/internal/provider/translated_probe.go b/internal/provider/translated_probe.goindex d763b3925..5c5bd8e40 100644--- a/internal/provider/translated_probe.go+++ b/internal/provider/translated_probe.go@@ -3,6 +3,8 @@ package providerimport ("context""time"++ "github.com/yetone/magpie/internal/probelog")// A Google sign-in (Antigravity, Gemini CLI) speaks Code Assist alone, an@@ -40,8 +42,17 @@ func (p Provider) testTranslated(ctx context.Context, model string) Result {ctx, cancel := context.WithTimeout(ctx, testWait)defer cancel()start := time.Now()- status, err := translatedProbe(ctx, p, CodeAssist, model)+ status, err := translatedProbe(probelog.Taken(ctx), p, CodeAssist, model)r.Millis = time.Since(start).Milliseconds()+ // the gateway's translator sent it: written down once, from what it+ // came to+ noted, msg := status, ""+ if err != nil {+ msg = err.Error()+ } else {+ noted = 200+ }+ probelog.NoteResult(probelog.WithModel(ctx, model), string(CodeAssist), start, noted, msg)if err != nil {r.Status, r.Error = status, err.Error()return rdiff --git a/internal/usage/heatmap.go b/internal/usage/heatmap.goindex b9dfafbb4..d666f28c9 100644--- a/internal/usage/heatmap.go+++ b/internal/usage/heatmap.go@@ -65,7 +65,7 @@ func heatmapOf(since, now time.Time, visit func(add func(Row))) *Heatmap {type ymd struct{ y, yd int }names := map[ymd]string{}visit(func(r Row) {- if r.IsRejected() || r.Time.Before(since) {+ if r.IsRejected() || r.IsProbe() || r.Time.Before(since) {return}t := r.Time.In(now.Location())diff --git a/internal/usage/ledger.go b/internal/usage/ledger.goindex b1cd8b2cb..cca0d6bdf 100644--- a/internal/usage/ledger.go+++ b/internal/usage/ledger.go@@ -105,7 +105,7 @@ const (// under its totals that local rejections are not counted — so it is "" and// picking either cell leaves it out, as the cells' own counts already do.func SourceOf(r Row) string {- if r.IsRejected() {+ if r.IsRejected() || r.IsProbe() { // magpie's own test is neither tooreturn ""}switch r.Source {diff --git a/internal/usage/log_index.go b/internal/usage/log_index.goindex e6b5022e0..ff305799d 100644--- a/internal/usage/log_index.go+++ b/internal/usage/log_index.go@@ -48,6 +48,10 @@ type logSnapshot struct {// reach back to a subscription window's start.firstOwn time.TimekeyProviders map[string]bool+ // probes are the requests magpie sent on its own (SourceProbe), kept+ // out of blocks: every reader of those counts what it reads, and the+ // request log alone lists these (queryPage)+ probes []probeRow}var logIndex struct {@@ -116,6 +120,7 @@ func readLogSnapshot() *logSnapshot {next.off, next.first, next.firstOwn = old.off, old.first, old.firstOwnnext.blocks = slices.Clone(old.blocks)next.keyProviders = maps.Clone(old.keyProviders)+ next.probes = slices.Clip(old.probes) // appended to, never written in}logIndex.reads++f, err := os.Open(path)@@ -147,6 +152,10 @@ func readLogSnapshot() *logSnapshot {if json.Unmarshal(b, &r) != nil {continue}+ if r.IsProbe() {+ next.probes = append(next.probes, probeRow{r, next.off})+ continue+ }if tail == nil {tail = &rowChunk{}}diff --git a/internal/usage/probes.go b/internal/usage/probes.gonew file mode 100644index 000000000..ed9ae9c2a--- /dev/null+++ b/internal/usage/probes.go@@ -0,0 +1,73 @@+package usage++import (+ "time"++ "github.com/yetone/magpie/internal/probelog"+)++// SourceProbe is Record.Source for a request magpie sent a provider on its+// own, outside any agent's turn (probelog): the editor's Test, Detect+// APIs, a model list's refresh. Its Kind says which (probelog.KindTest,+// KindDetect, KindList). Such a row is listed on the request log, told+// apart there, and counted nowhere: no total, cost, chart, group, budget,+// window hold or observer reads it. The log's index keeps it apart from+// the calls (logSnapshot.probes) so a reader of those never meets it.+const SourceProbe = "probe"++// IsProbe is a request magpie sent on its own (SourceProbe).+func (r Record) IsProbe() bool { return r.Source == SourceProbe }++// IsProbe is a row of such a request. A packed row keeps the row's Source,+// not its record's, so the request log asks the row.+func (r Row) IsProbe() bool { return r.Source == SourceProbe || r.Record.IsProbe() }++// probeRow is a probe's record and where its line ends in the log, as a+// gateway row's Order is.+type probeRow struct {+ Record+ off int64+}++func init() {+ probelog.Sink = func(p probelog.Row) {+ Append(Record{Time: p.Time, Agent: "magpie", Provider: p.Provider, ProviderAccount: p.Account,+ Host: p.Host, Model: p.Model, Requested: p.Model, Status: p.Status, Millis: p.Millis,+ Error: clipProbeError(p.Error), Endpoint: p.Endpoint, Kind: p.Kind, Source: SourceProbe})+ }+}++// clipProbeError keeps an error's first 500 bytes, as a vendor's answer can+// be a whole page.+func clipProbeError(s string) string {+ const most = 500+ if len(s) <= most {+ return s+ }+ cut := most+ for cut > 0 && s[cut]&0xC0 == 0x80 { // not inside a character+ cut--+ }+ return s[:cut] + "…"+}++// probeChunk is the probes made since since as one chunk of the request+// log's rows, each in the order its line has in the log as a gateway+// row's is; nil for none.+func probeChunk(probes []probeRow, since time.Time) *rowChunk {+ var c *rowChunk+ for _, p := range probes {+ if !since.IsZero() && p.Time.Before(since) {+ continue+ }+ if c == nil {+ c = &rowChunk{}+ }+ c.add(Row{Record: p.Record, Source: SourceProbe}, "", p.off, false)+ }+ if c == nil {+ return nil+ }+ c.dict, c.shapeIDs = nil, nil+ return c+}diff --git a/internal/usage/probes_test.go b/internal/usage/probes_test.gonew file mode 100644index 000000000..5861d6f6c--- /dev/null+++ b/internal/usage/probes_test.go@@ -0,0 +1,87 @@+package usage++import (+ "strings"+ "testing"+ "time"++ "github.com/yetone/magpie/internal/probelog"+)++// ledgerSpy is an Observer that only counts the rows Append hands it.+type ledgerSpy struct{ rows []Record }++func (*ledgerSpy) Serve() (stop func()) { return func() {} }+func (*ledgerSpy) Bodies() Bodies { var b Bodies; return b }+func (*ledgerSpy) Begin(Request) Watch { return nil }+func (s *ledgerSpy) Ledger(r Record) { s.rows = append(s.rows, r) }++// A request magpie sends a provider on its own (the editor's Test, Detect+// APIs, a model list) is a row of the request log, marked as magpie's own+// with what it was for, and counts in nothing: no total, source, group,+// summary, Load, LastSeen, shared day or observer (neiko, gys619).+func TestProbeRowsListedNotCounted(t *testing.T) {+ reportedHome(t)+ spy := &ledgerSpy{}+ t.Cleanup(Observe(spy))+ now := time.Now().Add(-time.Hour)+ Append(Record{Time: now, Agent: "claude", Provider: "anthropic", Model: "claude-opus-5-5", Input: 1000, Output: 100, Status: 200})+ probelog.Sink(probelog.Row{Time: now.Add(time.Second), Kind: probelog.KindTest, Provider: "relay", Account: "me", Model: "gpt-9",+ Host: "relay.example", Endpoint: "/v1/chat/completions", Status: 401, Millis: 120, Error: strings.Repeat("页", 400)})+ probelog.Sink(probelog.Row{Time: now.Add(2 * time.Second), Kind: probelog.KindList, Provider: "relay", Host: "relay.example", Endpoint: "GET /v1/models", Status: 200, Millis: 30})++ page := QueryPage(All, Filter{}, 0, 50)+ if page.Total != 3 || len(page.Rows) != 3 {+ t.Fatalf("total %d, %d rows: %+v", page.Total, len(page.Rows), page.Rows)+ }+ var probes []Row+ for _, r := range page.Rows {+ if r.Source == SourceProbe {+ probes = append(probes, r)+ }+ }+ if len(probes) != 2 {+ t.Fatalf("probe rows %+v", page.Rows)+ }+ test := probes[1] // newest first+ if probes[0].Kind != probelog.KindList || test.Kind != probelog.KindTest || test.Agent != "magpie" || test.Provider != "relay" ||+ test.ProviderAccount != "me" || test.Model != "gpt-9" || test.Status != 401 || test.Endpoint != "/v1/chat/completions" {+ t.Fatalf("probe rows %+v", probes)+ }+ if len(test.Error) > 504 || !strings.HasSuffix(test.Error, "…") || !strings.HasPrefix(test.Error, "页") {+ t.Fatalf("error not clipped at a character: %d bytes", len(test.Error))+ }+ if page.Sum.Calls != 1 || page.Sum.Errors != 0 || page.Through.Calls != 1 || page.Direct.Calls != 0 || page.Reported.Calls != 0 {+ t.Fatalf("sum %+v through %+v direct %+v", page.Sum, page.Through, page.Direct)+ }+ for dim, shares := range page.By {+ for _, s := range shares {+ if s.ID == "relay" || s.ID == "magpie" || s.ID == "gpt-9" {+ t.Fatalf("%s counts a probe: %+v", dim, shares)+ }+ }+ }+ if s := Summarize(All); s.Calls != 1 || s.Errors != 0 {+ t.Fatalf("summary %+v", s.Totals)+ }+ if rs := Load(time.Time{}); len(rs) != 1 {+ t.Fatalf("Load %+v", rs)+ }+ if !LastSeen("magpie").IsZero() {+ t.Fatal("a probe is a request from agent magpie")+ }+ n := 0+ for _, calls := range OwnDays(now.Add(-time.Hour)) {+ n += len(calls)+ }+ if n != 1 {+ t.Fatalf("%d shared calls", n)+ }+ if len(spy.rows) != 1 || spy.rows[0].IsProbe() {+ t.Fatalf("observer saw %+v", spy.rows)+ }+ // a filter on the probe's provider lists it and still counts nothing+ if p := QueryPage(All, Filter{Provider: "relay"}, 0, 50); p.Total != 2 || p.Sum.Calls != 0 {+ t.Fatalf("provider filter: total %d sum %+v", p.Total, p.Sum)+ }+}diff --git a/internal/usage/purpose.go b/internal/usage/purpose.goindex 87f4cb2d6..f1ce325da 100644--- a/internal/usage/purpose.go+++ b/internal/usage/purpose.go@@ -30,6 +30,9 @@ var purposeKinds = func() map[string]PurposeKind {add("Web search", "web_search")add("Image description", "vision")add("Routing classifier", "routing")+ add("Test", "provider_test") // magpie's own (probelog.KindTest)+ add("Detect APIs", "api_detect") // probelog.KindDetect+ add("Model list", "model_list") // probelog.KindListreturn out}()diff --git a/internal/usage/request_page.go b/internal/usage/request_page.goindex cafa1e597..34731dc6d 100644--- a/internal/usage/request_page.go+++ b/internal/usage/request_page.go@@ -720,6 +720,9 @@ func queryPage(p Period, f Filter, offset, limit int, readSource func(sessions.Cgateways = append(gateways, c)}}+ if c := probeChunk(snapshot.probes, gatewaySince); c != nil {+ gateways = append(gateways, c) // listed, never counted (SourceProbe)+ }on := map[string]bool{}var chunks []*rowChunkvar resolver *sessionResolver@@ -1314,6 +1317,9 @@ func buildRequestBlocksOf(p Period, f Filter, offset, limit int, now time.Time,if r.Provider != "" && !providers[r.Provider] {providers[r.Provider] = true}+ // a call turned away, and a request magpie sent on its own+ // (SourceProbe), are listed and counted in no total+ uncounted := r.IsRejected() || r.IsProbe()keep := f.keepsRow(r)if keep {out.Total++@@ -1325,20 +1331,20 @@ func buildRequestBlocksOf(p Period, f Filter, offset, limit int, now time.Time,}}var d Totals- if !r.IsRejected() {+ if !uncounted {d.addRow(*r)if keep {out.Sum.plus(&d)}+ addCallerRow(callers, r, &d)+ addAccountRow(accounts, r, &d)}- addCallerRow(callers, r, &d)- addAccountRow(accounts, r, &d)// The two sources are counted with the Via pick cleared, as the// dimension groups above are: the cells are what the reader// switches between, so each must go on saying its own number// while another is picked, or one of them would read 0 and// there would be no way back.- if !r.IsRejected() && viaFilter.keepsRow(r) {+ if !uncounted && viaFilter.keepsRow(r) {switch r.Source {case "log":out.Direct.plus(&d)@@ -1348,7 +1354,7 @@ func buildRequestBlocksOf(p Period, f Filter, offset, limit int, now time.Time,out.Through.plus(&d)}}- if r.IsRejected() {+ if uncounted {return}if keep || f.Day != "" && chartFilter.keepsRow(r) {diff --git a/internal/usage/usage.go b/internal/usage/usage.goindex 164a52e70..196e6d6c0 100644--- a/internal/usage/usage.go+++ b/internal/usage/usage.go@@ -234,7 +234,9 @@ func Append(r Record) {if r.Time.IsZero() {r.Time = Clock()}- observeLedger(r)+ if !r.IsProbe() { // magpie's own test is no call to export+ observeLedger(r)+ }b, err := json.Marshal(r)if err != nil {return@@ -286,7 +288,7 @@ func Load(since time.Time) []Record {break} // an unfinished line is left for the next readvar r Record- if json.Unmarshal(b, &r) == nil && (since.IsZero() || !r.Time.Before(since)) {+ if json.Unmarshal(b, &r) == nil && !r.IsProbe() && (since.IsZero() || !r.Time.Before(since)) {out = append(out, r)}}@@ -1127,7 +1129,7 @@ func LastSeen(agent string) time.Time {sc.Buffer(make([]byte, 64<<10), 1<<20)for sc.Scan() {var r Record- if json.Unmarshal(sc.Bytes(), &r) == nil && r.Agent == agent && r.Via == "" && r.Time.After(last) {+ if json.Unmarshal(sc.Bytes(), &r) == nil && r.Agent == agent && r.Via == "" && !r.IsProbe() && r.Time.After(last) {last = r.Time}}