Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.
gateway: with no S3 bucket the request archive keeps calls on this computer, oldest removed past 1 GiB (CyberCaius)
yetonee219f699ca6d
diff --git a/docs/subsystems/gui-shell.md b/docs/subsystems/gui-shell.mdindex 45d559f12..decea538d 100644--- a/docs/subsystems/gui-shell.md+++ b/docs/subsystems/gui-shell.md@@ -312,7 +312,11 @@ record it); a route id is not a call id. When the ring no longer has it,the story shows the request archive's copy through `archivePanel` if theroute names one (`Route.Archive`, `call.archiveName()`, set only when thearchive was on for that request), and otherwise says how to turn the-archive on. Nothing more is captured than before. See+archive on. Nothing more is captured than before. The archive's switch+(`POST /api/settings/archive`) sets `RequestArchive` while sync has an S3+bucket and `RequestArchiveLocal` while it has none; `GET+/api/archive?name=…` reads this computer's copy first and then the bucket's+(`TestArchiveSwitchKeepsOnThisComputer`, `request-archive.test.cjs`). See`TestRoutingRequestOpensItsBodies`, `TestArchiveWholeBodies` and`routing-bodies.test.cjs`.diff --git a/docs/subsystems/sync-backup.md b/docs/subsystems/sync-backup.mdindex df8708d82..f1f9d0a7f 100644--- a/docs/subsystems/sync-backup.md+++ b/docs/subsystems/sync-backup.md@@ -153,8 +153,17 @@ repository can keep the token. The branch is part of the sync state key.creation returns `errChanged`. Other 422responses stay actionable errors. Authentication, permission and branchfailures remain distinct from missing files. Rate limits use sync backoff.-- Request archival remains an S3-only feature; its objects are separate from- the encrypted setup and usage files.+- The request archive goes to S3 only when sync has a bucket. Its objects are+ separate from the encrypted setup and usage files. With no bucket the same+ files are kept on this computer instead (`gateway.DiskArchive`, in+ `<settings.Dir()>/request-archive/<date>/<id>.json`, written through a+ temporary file and renamed, 0600), under its own switch,+ `RequestArchiveLocal`. That switch and its size, `RequestArchiveLocalMB`+ (1 GiB unless set; the oldest calls are removed first past it), are in+ `KeepOwn`, so sync never turns on writing bodies to another computer's+ disk, and `RequestArchive` synced from a computer with a bucket keeps+ nothing on one without (`TestArchiveNeverOnDiskUnasked`,+ `TestRequestArchiveLocalIsThisComputers`).## Verificationdiff --git a/internal/gateway/archive.go b/internal/gateway/archive.goindex b292bb3af..0c0f1a5a4 100644--- a/internal/gateway/archive.go+++ b/internal/gateway/archive.go@@ -3,7 +3,9 @@ package gateway// The request archive: with it on (settings' RequestArchive), each call the// gateway serves is kept — the headers and bodies both ways, every secret// taken out first — in the user's own S3 bucket, the one sync keeps its-// backup in, at <prefix>/magpie/archive/<date>/<id>.json: the call's date,+// backup in, at <prefix>/magpie/archive/<date>/<id>.json, or, while sync+// has no bucket and RequestArchiveLocal is on, in the same file on this+// computer (archive_local.go): the call's date,// in UTC, and an id of its own, sent back on the response as// X-Magpie-Archive-Id. The call, and its rows in the usage log, carry// "<date>/<id>", for the Gateway and Usage pages to read it back by.@@ -41,6 +43,7 @@ type Putter interface {// ArchiveBucket is the bucket the archive goes to, and false when there is// none (sync is off, or to a WebDAV folder): set by main to the S3 sync's.+// With none, the archive is this computer's (archiveDest).var ArchiveBucket func() (Putter, bool)// ArchiveHeader carries a call's archive id on its response.@@ -76,15 +79,33 @@ type wire struct {to Putter}-// archiving is r's wire when the archive is on and has a bucket, nil-// otherwise: nothing is kept, and nothing sent. body is the request's, as-// Recent calls has it before the first 256 KB is cut from it.-func archiving(r *http.Request, res *captureResponseWriter, start time.Time, body []byte) *wire {- if ArchiveBucket == nil || !settings.Load().RequestArchive {- return nil+// archiveDest is where a call is archived, and false when it isn't: the S3+// bucket sync is to while RequestArchive is on, or, with no bucket, this+// computer's folder while RequestArchiveLocal is on (archive_local.go). A+// bucket that goes away never sends calls to the disk in its place: that+// takes the local switch, turned on at this computer.+func archiveDest() (Putter, bool) {+ s := settings.Load()+ if !s.RequestArchive && !s.RequestArchiveLocal {+ return nil, false // off, as it is unless turned on: sync's file isn't read+ }+ if ArchiveBucket != nil {+ if b, ok := ArchiveBucket(); ok && b != nil {+ return b, s.RequestArchive+ }}- to, ok := ArchiveBucket()- if !ok || to == nil {+ if s.RequestArchiveLocal {+ return LocalArchive(), true+ }+ return nil, false+}++// archiving is r's wire when the archive is on, nil otherwise: nothing is+// kept, and nothing sent. body is the request's, as Recent calls has it+// before the first 256 KB is cut from it.+func archiving(r *http.Request, res *captureResponseWriter, start time.Time, body []byte) *wire {+ to, ok := archiveDest()+ if !ok {return nil}b := make([]byte, 8)diff --git a/internal/gateway/archive_local.go b/internal/gateway/archive_local.gonew file mode 100644index 000000000..8cc9f91bd--- /dev/null+++ b/internal/gateway/archive_local.go@@ -0,0 +1,218 @@+package gateway++// The request archive on this computer (CyberCaius on Discord): while sync+// has no S3 bucket for it, a call is kept in magpie's own folder, at+// request-archive/<date>/<id>.json, the same file the bucket would get. It+// has a switch of its own, settings' RequestArchiveLocal, which sync never+// carries to another computer, so request bodies are written to a disk only+// where the user turned it on. The folder grows to RequestArchiveLocalMB+// (1 GiB unless set), and past that the oldest calls are removed first.++import (+ "context"+ "errors"+ "fmt"+ "io"+ "io/fs"+ "os"+ "path/filepath"+ "sort"+ "strings"+ "sync"+ "time"++ "github.com/yetone/magpie/internal/settings"+)++// ErrNotArchived is a name the archive on this computer has nothing at.+var ErrNotArchived = errors.New("not in the archive on this computer")++// LocalArchiveCap is how large the archive on this computer grows, in+// bytes: settings' RequestArchiveLocalMB, a GiB unless set.+func LocalArchiveCap() int64 {+ mb := settings.Load().RequestArchiveLocalMB+ if mb <= 0 {+ mb = 1024+ }+ return int64(min(mb, 1<<20)) << 20+}++// DiskArchive is the archive in a folder on this computer: Put as a bucket+// is, and Open to read a call back.+type DiskArchive struct {+ Dir string+ // cap is the most it holds, in bytes; LocalArchiveCap when nil+ cap func() int64++ mu sync.Mutex+ total int64 // the folder's size as last counted, and added to since+ known bool // total has been counted+}++var (+ localMu sync.Mutex+ localArchives = map[string]*DiskArchive{}+)++// LocalArchive is the archive in magpie's folder: request-archive under+// ~/.config/magpie, or a portable magpie's data folder.+func LocalArchive() *DiskArchive {+ dir := filepath.Join(settings.Dir(), "request-archive")+ localMu.Lock()+ defer localMu.Unlock()+ d := localArchives[dir]+ if d == nil {+ d = &DiskArchive{Dir: dir}+ localArchives[dir] = d+ }+ return d+}++// file is where name, "archive/<date>/<id>.json" as ArchiveName makes it,+// is kept; false for any other name.+func (d *DiskArchive) file(name string) (string, bool) {+ rest, ok := strings.CutPrefix(name, "archive/")+ date, base, ok2 := strings.Cut(rest, "/")+ id, ok3 := strings.CutSuffix(base, ".json")+ if !ok || !ok2 || !ok3 {+ return "", false+ }+ if _, ok := ArchiveName(date, id); !ok {+ return "", false+ }+ return filepath.Join(d.Dir, date, id+".json"), true+}++// Put writes data at name: to a temporary file beside it first, renamed+// over it once whole, so a reader never sees half a call. Then the oldest+// calls are removed while the folder is over its size.+func (d *DiskArchive) Put(_ context.Context, name string, data []byte) error {+ path, ok := d.file(name)+ if !ok {+ return fmt.Errorf("%s is not a request archive's name", name)+ }+ dir := filepath.Dir(path)+ if err := os.MkdirAll(dir, 0o700); err != nil {+ return err+ }+ tmp, err := os.CreateTemp(dir, "."+filepath.Base(path)+"-*.part")+ if err != nil {+ return err+ }+ _, err = tmp.Write(data)+ if err == nil {+ err = tmp.Sync()+ }+ if cerr := tmp.Close(); err == nil {+ err = cerr+ }+ if err == nil {+ err = os.Rename(tmp.Name(), path)+ }+ if err != nil {+ os.Remove(tmp.Name())+ return err+ }+ d.added(int64(len(data)), path)+ return nil+}++// Open is the call kept at name, as it is read, and its size;+// ErrNotArchived when there is none. The caller closes it.+func (d *DiskArchive) Open(_ context.Context, name string) (io.ReadCloser, int64, error) {+ path, ok := d.file(name)+ if !ok {+ return nil, 0, ErrNotArchived+ }+ f, err := os.Open(path)+ if errors.Is(err, fs.ErrNotExist) {+ return nil, 0, ErrNotArchived+ }+ if err != nil {+ return nil, 0, err+ }+ fi, err := f.Stat()+ if err != nil {+ f.Close()+ return nil, 0, err+ }+ return f, fi.Size(), nil+}++type keptCall struct {+ path, rel string+ size int64+}++// scan is every call in the folder, oldest first: "<date>/<HHMMSS>-…"+// sorts by when it was made. A temporary file left by a write that never+// finished, an hour old, is removed on the way.+func (d *DiskArchive) scan() []keptCall {+ var out []keptCall+ filepath.WalkDir(d.Dir, func(p string, e fs.DirEntry, err error) error {+ if err != nil || e.IsDir() {+ return nil+ }+ fi, err := e.Info()+ if err != nil {+ return nil+ }+ if strings.HasPrefix(e.Name(), ".") {+ if strings.HasSuffix(e.Name(), ".part") && time.Since(fi.ModTime()) > time.Hour {+ os.Remove(p)+ }+ return nil+ }+ if !strings.HasSuffix(e.Name(), ".json") {+ return nil+ }+ rel, _ := filepath.Rel(d.Dir, p)+ out = append(out, keptCall{p, filepath.ToSlash(rel), fi.Size()})+ return nil+ })+ sort.Slice(out, func(i, j int) bool { return out[i].rel < out[j].rel })+ return out+}++// added counts n more bytes, written at kept, and past the folder's size+// removes the oldest calls until it is a tenth under it, so the folder is+// walked again only once that much more is written. The call just kept is+// never removed, though it alone be over the size.+func (d *DiskArchive) added(n int64, kept string) {+ d.mu.Lock()+ defer d.mu.Unlock()+ if !d.known {+ d.total, d.known = 0, true+ for _, c := range d.scan() {+ d.total += c.size+ }+ } else {+ d.total += n+ }+ most := LocalArchiveCap()+ if d.cap != nil {+ most = d.cap()+ }+ if d.total <= most {+ return+ }+ calls := d.scan()+ var total int64+ for _, c := range calls {+ total += c.size+ }+ goal := most - most/10+ for _, c := range calls {+ if total <= goal {+ break+ }+ if c.path == kept {+ continue+ }+ if os.Remove(c.path) == nil {+ total -= c.size+ os.Remove(filepath.Dir(c.path)) // the day's folder, once it is empty+ }+ }+ d.total = total+}diff --git a/internal/gateway/archive_local_test.go b/internal/gateway/archive_local_test.gonew file mode 100644index 000000000..1eb35a699--- /dev/null+++ b/internal/gateway/archive_local_test.go@@ -0,0 +1,212 @@+package gateway++import (+ "context"+ "encoding/json"+ "fmt"+ "io"+ "os"+ "path/filepath"+ "runtime"+ "strings"+ "testing"+ "time"++ "github.com/yetone/magpie/internal/settings"+)++// With sync not to S3 and the archive's switch on at this computer+// (CyberCaius on Discord), a call is kept in magpie's folder, the same file+// the bucket gets: its secrets taken out, its id on the response and the+// call, written whole through a temporary file, readable by this user only.+func TestArchiveOnThisComputer(t *testing.T) {+ fresh(t)+ serveOn(t, "fake", "k", []string{"m1"}, archiveVendor{})+ was := ArchiveBucket+ ArchiveBucket = func() (Putter, bool) { return nil, false } // sync to WebDAV, or none+ t.Cleanup(func() { ArchiveBucket = was })+ settings.Save(settings.Settings{RequestArchiveLocal: true})+ s := New()+ rec := archivePost(t, s)+ id := rec.Header().Get(ArchiveHeader)+ c := s.Recent()[0]+ name := c.Time.UTC().Format("2006-01-02") + "/" + id+ if id == "" || c.Archive != name {+ t.Fatalf("not archived: id %q, call %q", id, c.Archive)+ }+ path := filepath.Join(settings.Dir(), "request-archive", filepath.FromSlash(name)+".json")+ data, err := os.ReadFile(path)+ if err != nil {+ t.Fatalf("not on this computer: %v", err)+ }+ for _, secret := range []string{archiveKey, "named-secret", "magpie-gateway-key", "another-secret", "cookie-secret", "abc123", "query-secret"} {+ if strings.Contains(string(data), secret) {+ t.Errorf("%s in the archive:\n%s", secret, data)+ }+ }+ var a Archived+ if err := json.Unmarshal(data, &a); err != nil {+ t.Fatal(err)+ }+ if a.ID != name || a.Call.Model != "fake/m1" || a.Request.Headers["Authorization"][0] != "[REDACTED]" ||+ !strings.Contains(a.Request.Body, "deploy with [REDACTED:API_KEY]") || !strings.Contains(a.Response.Body, `"content":"done"`) {+ t.Errorf("archived %+v", a)+ }+ if fi, _ := os.Stat(path); runtime.GOOS != "windows" && fi.Mode().Perm() != 0o600 {+ t.Errorf("mode %v, want 0600", fi.Mode().Perm())+ }+ if left, _ := filepath.Glob(filepath.Join(filepath.Dir(path), ".*")); len(left) != 0 {+ t.Errorf("temporary files left: %v", left)+ }+ // read back by its name, as the Usage page asks for it+ date, aid, _ := strings.Cut(name, "/")+ file, _ := ArchiveName(date, aid)+ r, size, err := LocalArchive().Open(context.Background(), file)+ if err != nil || size != int64(len(data)) {+ t.Fatalf("Open: %d bytes, %v", size, err)+ }+ r.Close()+ if _, _, err := LocalArchive().Open(context.Background(), "archive/2026-10-02/101010-00000000000000bb.json"); err != ErrNotArchived {+ t.Errorf("a call never kept: %v", err)+ }+}++// Request bodies are written to a disk only where the user turned the+// archive on at that computer: RequestArchive, which sync carries from a+// computer with an S3 bucket, keeps nothing on one with none, and the+// local switch sends nothing to the disk while a bucket is there.+func TestArchiveNeverOnDiskUnasked(t *testing.T) {+ fresh(t)+ serveOn(t, "fake", "k", []string{"m1"}, archiveVendor{})+ was := ArchiveBucket+ t.Cleanup(func() { ArchiveBucket = was })+ dir := filepath.Join(settings.Dir(), "request-archive")+ onDisk := func() []string {+ var got []string+ filepath.Walk(dir, func(p string, fi os.FileInfo, err error) error {+ if err == nil && !fi.IsDir() {+ got = append(got, p)+ }+ return nil+ })+ return got+ }++ // off, as it is unless turned on+ ArchiveBucket = func() (Putter, bool) { return nil, false }+ s := New()+ if rec := archivePost(t, s); rec.Header().Get(ArchiveHeader) != "" || len(onDisk()) != 0 {+ t.Fatalf("archived while off: %v", onDisk())+ }+ // the S3 archive's switch, synced here, with no bucket here+ settings.Save(settings.Settings{RequestArchive: true})+ if rec := archivePost(t, s); rec.Header().Get(ArchiveHeader) != "" || len(onDisk()) != 0 {+ t.Fatalf("the synced switch wrote to this computer: %v", onDisk())+ }+ // the local switch with a bucket there: the bucket's switch decides+ b := &memBucket{objs: map[string][]byte{}}+ archiveTo(t, b)+ settings.Save(settings.Settings{RequestArchiveLocal: true})+ if rec := archivePost(t, s); rec.Header().Get(ArchiveHeader) != "" || len(onDisk()) != 0 || len(b.objs) != 0 {+ t.Fatalf("archived with the bucket's switch off: disk %v, bucket %v", onDisk(), keys(b.objs))+ }+ settings.Save(settings.Settings{RequestArchive: true, RequestArchiveLocal: true})+ if archivePost(t, s); len(b.objs) != 1 || len(onDisk()) != 0 {+ t.Fatalf("with a bucket: bucket %v, disk %v", keys(b.objs), onDisk())+ }+}++// Past its size the folder loses its oldest calls first, down to a tenth+// under it; the call just kept stays, and a temporary file a write left an+// hour ago goes.+func TestLocalArchiveRemovesOldestPastItsSize(t *testing.T) {+ d := &DiskArchive{Dir: t.TempDir(), cap: func() int64 { return 1000 }}+ ctx := context.Background()+ body := []byte(strings.Repeat("x", 200))+ name := func(day, i int) string {+ return fmt.Sprintf("archive/2026-10-%02d/%06d-00000000000000aa.json", day, 100000+i)+ }+ stale := filepath.Join(d.Dir, "2026-10-01", ".old.json-1.part")+ os.MkdirAll(filepath.Dir(stale), 0o700)+ os.WriteFile(stale, body, 0o600)+ old := time.Now().Add(-2 * time.Hour)+ os.Chtimes(stale, old, old)+ // four on the 1st, one on the 2nd: 1000 bytes, at the size+ for i := range 4 {+ if err := d.Put(ctx, name(1, i), body); err != nil {+ t.Fatal(err)+ }+ }+ if err := d.Put(ctx, name(2, 0), body); err != nil {+ t.Fatal(err)+ }+ if _, err := os.Stat(stale); err == nil {+ t.Error("an hour-old temporary file is still there")+ }+ has := func(n string) bool {+ r, _, err := d.Open(ctx, n)+ if err == nil {+ r.Close()+ }+ return err == nil+ }+ for i := range 4 {+ if !has(name(1, i)) {+ t.Fatalf("removed at the size, not past it: %s", name(1, i))+ }+ }+ // one more is past it: the two oldest go, 800 bytes left+ if err := d.Put(ctx, name(2, 1), body); err != nil {+ t.Fatal(err)+ }+ for i, want := range []bool{false, false, true, true} {+ if has(name(1, i)) != want {+ t.Errorf("%s kept %v, want %v", name(1, i), !want, want)+ }+ }+ if !has(name(2, 0)) || !has(name(2, 1)) {+ t.Error("a newer call was removed")+ }+ // one alone larger than the size is kept, and every other goes+ big := []byte(strings.Repeat("y", 1500))+ if err := d.Put(ctx, name(3, 0), big); err != nil {+ t.Fatal(err)+ }+ if !has(name(3, 0)) {+ t.Error("the call just kept was removed")+ }+ for _, n := range []string{name(1, 2), name(1, 3), name(2, 0), name(2, 1)} {+ if has(n) {+ t.Errorf("%s kept past the size", n)+ }+ }+ if days, _ := os.ReadDir(d.Dir); len(days) != 1 {+ t.Errorf("empty days left: %v", days)+ }+ r, _, _ := d.Open(ctx, name(3, 0))+ got, _ := io.ReadAll(r)+ r.Close()+ if string(got) != string(big) {+ t.Errorf("read back %d bytes", len(got))+ }+ // a name not of the archive's shape is never a path+ for _, n := range []string{"archive/../../settings.json", "archive/2026-10-02/../x.json", "settings.json", "archive/2026-10-02/ABC.json"} {+ if err := d.Put(ctx, n, body); err == nil {+ t.Errorf("Put %s", n)+ }+ if _, _, err := d.Open(ctx, n); err != ErrNotArchived {+ t.Errorf("Open %s: %v", n, err)+ }+ }+}++func TestLocalArchiveCap(t *testing.T) {+ t.Setenv("XDG_CONFIG_HOME", t.TempDir())+ if got := LocalArchiveCap(); got != 1<<30 {+ t.Errorf("default %d, want a GiB", got)+ }+ settings.Save(settings.Settings{RequestArchiveLocalMB: 5})+ if got := LocalArchiveCap(); got != 5<<20 {+ t.Errorf("5 MiB: %d", got)+ }+}diff --git a/internal/gui/api.go b/internal/gui/api.goindex 7c0f85682..b85d25250 100644--- a/internal/gui/api.go+++ b/internal/gui/api.go@@ -1239,6 +1239,9 @@ func Handler(w Windows, gw *gateway.Server) http.Handler {in.RedactActions = cur.RedactActions // set on their own (redact-actions below)in.RedactAllow = cur.RedactAllow // set on its own (redact-allow below)in.DisabledModels = cur.DisabledModels // set on their own (disabled-models below, #1867)+ // the request archive on this computer: the archive's switch while no+ // S3 bucket, and its size in settings.json only+ in.RequestArchiveLocal, in.RequestArchiveLocalMB = cur.RequestArchiveLocal, cur.RequestArchiveLocalMB// used or left is the Usage page's toggle as much as Settings', set on its ownin.QuotaLeft = cur.QuotaLeftin.QuotaReads = cur.QuotaReads // set on its own (quota-reads below)diff --git a/internal/gui/archive.go b/internal/gui/archive.goindex e4d392471..ff9602336 100644--- a/internal/gui/archive.go+++ b/internal/gui/archive.go@@ -27,6 +27,10 @@ var archiveBucket = func() (archiveStore, bool) {return b, ok}+// localArchive is the archive on this computer, kept while sync has no S3+// bucket (gateway/archive_local.go).+var localArchive = func() archiveStore { return gateway.LocalArchive() }+// archivePreviewBody is the most of a body the page is sent to show; a// longer one is left out, for the file to be downloaded (#447). Its size// is told here, so a large body never reaches the page.@@ -66,16 +70,24 @@ func preview(p gateway.ArchivePart) *previewPart {}// openArchive is the call named by r's ?date= and ?id=, as it is read from-// the bucket, its size, and "<date>-<id>"; the error is the page's to show.+// this computer's archive, or else the bucket, its size, and "<date>-<id>";+// the error is the page's to show. A call kept here while sync had no+// bucket is read here after one is set up, and the other way round.func openArchive(r *http.Request) (io.ReadCloser, int64, string, error) {date, id := r.URL.Query().Get("date"), r.URL.Query().Get("id")name, ok := gateway.ArchiveName(date, id)if !ok {return nil, 0, "", fmt.Errorf("%s/%s is not a request archive's date and id", date, id)}+ if body, size, err := localArchive().Open(r.Context(), name); !errors.Is(err, gateway.ErrNotArchived) {+ if err != nil {+ return nil, 0, "", err+ }+ return body, size, date + "-" + id, nil+ }b, ok := archiveBucket()if !ok {- return nil, 0, "", errors.New("The request archive goes to the S3 bucket sync keeps its backup in, and sync isn't set up with one")+ return nil, 0, "", errors.New("Not in the archive on this computer: it is saved just after the call, the oldest are removed past the archive's size, or the save failed (magpie's log says why)")}body, size, err := b.Open(r.Context(), name)if errors.Is(err, davsync.ErrNoObject) {@@ -87,9 +99,12 @@ func openArchive(r *http.Request) (io.ReadCloser, int64, string, error) {return body, size, date + "-" + id, nil}-// archiveRoutes are the request archive: its switch on the Gateway page, a-// call read back from the bucket by its date and id, for the Gateway and-// Usage pages, and its file, downloaded whole (gateway/archive.go).+// archiveRoutes are the request archive: its switch on the Usage page, a+// call read back from this computer or the bucket by its date and id, for+// the Usage and Routing pages, and its file, downloaded whole+// (gateway/archive.go). With sync to an S3 bucket the switch is+// RequestArchive, which sync carries to every computer; with none it is+// RequestArchiveLocal, this computer's own, and calls are kept on its disk.func archiveRoutes(mux *http.ServeMux) {mux.HandleFunc("POST /api/settings/archive", func(rw http.ResponseWriter, r *http.Request) {var in struct{ On bool }@@ -97,12 +112,12 @@ func archiveRoutes(mux *http.ServeMux) {fail(rw, err)return}- if _, ok := davsync.S3Bucket(); in.On && !ok {- fail(rw, errors.New("The request archive goes to the S3 bucket sync keeps its backup in: set up Sync and backup in Settings with an s3:// address first"))- return- }s := settings.Load()- s.RequestArchive = in.On+ if _, ok := davsync.S3Bucket(); ok {+ s.RequestArchive = in.On+ } else {+ s.RequestArchiveLocal = in.On+ }if err := settings.Save(s); err != nil {fail(rw, err)return@@ -200,19 +215,25 @@ func saveDownload(dir, stem, ext string, r io.Reader) (string, error) {return name, nil}-// archiveJSON is the request archive as the Gateway page shows it: on or-// off, the bucket it goes to (none when sync isn't to S3), and why the last-// upload failed, if it did.+// archiveJSON is the request archive as the Usage page shows it: on or+// off, the bucket it goes to, or, when sync isn't to S3, the folder on this+// computer and the size it is kept under, and why the last upload or save+// failed, if it did.type archiveJSON struct {On bool `json:"on"`Bucket string `json:"bucket,omitempty"`+ Folder string `json:"folder,omitempty"`+ Cap int64 `json:"cap,omitempty"` // bytesError string `json:"error,omitempty"`}func archiveState() archiveJSON {- a := archiveJSON{On: settings.Load().RequestArchive}+ s := settings.Load()+ var a archiveJSONif b, ok := davsync.S3Bucket(); ok {- a.Bucket = b.Where()+ a.On, a.Bucket = s.RequestArchive, b.Where()+ } else {+ a.On, a.Folder, a.Cap = s.RequestArchiveLocal, tilde(gateway.LocalArchive().Dir), gateway.LocalArchiveCap()}if a.On {a.Error, _ = gateway.ArchiveError()diff --git a/internal/gui/archive_test.go b/internal/gui/archive_test.goindex e66414942..6d7ab1389 100644--- a/internal/gui/archive_test.go+++ b/internal/gui/archive_test.go@@ -14,6 +14,7 @@ import ("github.com/yetone/magpie/internal/davsync""github.com/yetone/magpie/internal/gateway"+ "github.com/yetone/magpie/internal/settings")// fakeArchive is a bucket of archived calls in memory; size, when set,@@ -145,3 +146,50 @@ func TestArchiveOldPreview(t *testing.T) {t.Fatalf("%d %s", rec.Code, rec.Body)}}++// With sync not to S3 the switch turns on the archive on this computer,+// its own setting, not the S3 one sync carries; the page is told the folder+// and its size, and a call kept there is read back and downloaded from it+// (CyberCaius on Discord).+func TestArchiveSwitchKeepsOnThisComputer(t *testing.T) {+ h := archiveHome(t, fakeArchive{objs: map[string][]byte{}})+ archiveBucket = func() (archiveStore, bool) { return nil, false }+ rec := httptest.NewRecorder()+ h.ServeHTTP(rec, httptest.NewRequest("POST", "/api/settings/archive", strings.NewReader(`{"on":true}`)))+ var a archiveJSON+ json.Unmarshal(rec.Body.Bytes(), &a)+ if rec.Code != 200 || !a.On || a.Bucket != "" || !strings.HasSuffix(a.Folder, "request-archive") || a.Cap != 1<<30 {+ t.Fatalf("%d %s", rec.Code, rec.Body)+ }+ if s := settings.Load(); !s.RequestArchiveLocal || s.RequestArchive {+ t.Fatalf("switched local %v, synced %v", s.RequestArchiveLocal, s.RequestArchive)+ }+ obj := archivedCall(t, `{"q":"here"}`, `{"a":"kept"}`)+ if err := gateway.LocalArchive().Put(context.Background(), archiveObject, obj); err != nil {+ t.Fatal(err)+ }+ rec = httptest.NewRecorder()+ h.ServeHTTP(rec, httptest.NewRequest("GET", "/api/archive"+archiveQuery, nil))+ var p archivePreview+ json.Unmarshal(rec.Body.Bytes(), &p)+ if rec.Code != 200 || p.Request == nil || p.Request.Body != `{"q":"here"}` || p.Bytes != int64(len(obj)) {+ t.Fatalf("read back %d %s", rec.Code, rec.Body)+ }+ rec = httptest.NewRecorder()+ h.ServeHTTP(rec, httptest.NewRequest("GET", "/api/archive/file"+archiveQuery, nil))+ if rec.Code != 200 || !bytes.Equal(rec.Body.Bytes(), obj) {+ t.Fatalf("download %d, %d bytes", rec.Code, rec.Body.Len())+ }+ // one it never kept says so in this computer's words+ rec = httptest.NewRecorder()+ h.ServeHTTP(rec, httptest.NewRequest("GET", "/api/archive?date=2026-10-02&id=101010-00000000000000bb", nil))+ if rec.Code != http.StatusBadRequest || !strings.Contains(rec.Body.String(), "Not in the archive on this computer") {+ t.Fatalf("missing: %d %s", rec.Code, rec.Body)+ }+ // and off again+ rec = httptest.NewRecorder()+ h.ServeHTTP(rec, httptest.NewRequest("POST", "/api/settings/archive", strings.NewReader(`{"on":false}`)))+ if s := settings.Load(); rec.Code != 200 || s.RequestArchiveLocal {+ t.Fatalf("off: %d %s", rec.Code, rec.Body)+ }+}diff --git a/internal/gui/assets/app.js b/internal/gui/assets/app.jsindex 4efff79e5..9d3fab0f1 100644--- a/internal/gui/assets/app.js+++ b/internal/gui/assets/app.js@@ -7074,13 +7074,10 @@ function callBodyPanel(label, raw, truncated, id, key = id) {return panel;}-// The request archive: with it on, each call's headers and bodies, secrets-// taken out, go to the S3 bucket sync keeps its backup in, and a request's-// row on the Usage page reads its own back from there (gateway/archive.go).-// Its switch sits over that list, at its right (Jorben on Discord); what it-// keeps, and where, or what it needs first, is its tooltip, and a failed-// upload says so beside it. The switch is the same element across redraws,-// changed in place, so one being pressed or focused is never swapped out.+// The request archive: each call, secrets out, kept in sync's S3 bucket or+// else a folder here (gateway/archive*.go), read back by its Usage row. Its+// switch is over that list (Jorben); where it keeps is its tooltip, a failed+// save is beside it. It is changed in place, never swapped out mid-press.let archiveSaving = false;function renderArchive() {const box = $("#ledArchive");@@ -7114,12 +7111,12 @@ function renderArchive() {b.disabled = archiveSaving;b.querySelector(".led-arch-name").textContent = t("Request archive");b.title = a.bucket ? t("Keeps each call’s headers and bodies, secrets taken out, in {where}", { where: a.bucket })- : t("Keeps each call’s headers and bodies, secrets taken out, in your S3 bucket. Set up Sync and backup in Settings with an s3:// address first");+ : t("Keeps each call’s headers and bodies, secrets taken out, on this computer in {where}, the oldest removed past {size}", { where: a.folder || "", size: fmtBytes(a.cap || 0) });let err = box.querySelector(".led-arch-err");if (a.on && a.error) {if (!err) { err = el("span", "led-arch-err"); box.prepend(err); }- err.textContent = t("Upload failed");- err.title = t("Last upload failed: {e}", { e: a.error });+ err.textContent = t(a.bucket ? "Upload failed" : "Not saved");+ err.title = t(a.bucket ? "Last upload failed: {e}" : "Last save failed: {e}", { e: a.error });} else err?.remove();}diff --git a/internal/gui/assets/i18n.js b/internal/gui/assets/i18n.jsindex 14d0b66cc..2f1da3b86 100644--- a/internal/gui/assets/i18n.js+++ b/internal/gui/assets/i18n.js@@ -2545,11 +2545,12 @@ const I18N = {"first 256 KB": "前 256 KB","Request archive": "请求存档","Keeps each call’s headers and bodies, secrets taken out, in {where}": "每次调用的请求/响应头和正文去除密钥后存到 {where}",- "Keeps each call’s headers and bodies, secrets taken out, in your S3 bucket. Set up Sync and backup in Settings with an s3:// address first": "每次调用的请求/响应头和正文去除密钥后存到你的 S3 存储桶。请先在设置的“同步与备份”中填写 s3:// 地址",+ "Keeps each call’s headers and bodies, secrets taken out, on this computer in {where}, the oldest removed past {size}": "每次调用的请求/响应头和正文去除密钥后存到本机的 {where},超过 {size} 时先删除最早的",+ "Not saved": "未保存",+ "Last save failed: {e}": "上次保存失败:{e}",+ "Not in the archive on this computer: it is saved just after the call, the oldest are removed past the archive's size, or the save failed (magpie's log says why)": "本机存档中没有此调用:调用结束后才保存,存档超过大小上限时会删除最早的,或保存失败(原因见 magpie 日志)","Last upload failed: {e}": "上次上传失败:{e}","Upload failed": "上传失败",- "The request archive goes to the S3 bucket sync keeps its backup in: set up Sync and backup in Settings with an s3:// address first": "请求存档会上传到同步备份所用的 S3 存储桶:请先在设置的“同步与备份”中填写 s3:// 地址",- "The request archive goes to the S3 bucket sync keeps its backup in, and sync isn't set up with one": "请求存档会上传到同步备份所用的 S3 存储桶,但同步尚未设置 S3","Not in the archive yet: it is uploaded just after the call, or the upload failed (magpie's log says why)": "存档中尚无此调用:调用结束后才上传,或上传失败(原因见 magpie 日志)","Fetch from archive": "从存档取回","Fetching…": "取回中…",@@ -6460,11 +6461,12 @@ const I18N = {"first 256 KB": "前 256 KB","Request archive": "請求存檔","Keeps each call’s headers and bodies, secrets taken out, in {where}": "每次呼叫的請求/回應頭和正文去除金鑰後存到 {where}",- "Keeps each call’s headers and bodies, secrets taken out, in your S3 bucket. Set up Sync and backup in Settings with an s3:// address first": "每次呼叫的請求/回應頭和正文去除金鑰後存到你的 S3 儲存桶。請先在設定的「同步與備份」中填寫 s3:// 位址",+ "Keeps each call’s headers and bodies, secrets taken out, on this computer in {where}, the oldest removed past {size}": "每次呼叫的請求/回應頭和正文去除金鑰後存到本機的 {where},超過 {size} 時先刪除最早的",+ "Not saved": "未儲存",+ "Last save failed: {e}": "上次儲存失敗:{e}",+ "Not in the archive on this computer: it is saved just after the call, the oldest are removed past the archive's size, or the save failed (magpie's log says why)": "本機存檔中沒有此呼叫:呼叫結束後才儲存,存檔超過大小上限時會刪除最早的,或儲存失敗(原因見 magpie 日誌)","Last upload failed: {e}": "上次上傳失敗:{e}","Upload failed": "上傳失敗",- "The request archive goes to the S3 bucket sync keeps its backup in: set up Sync and backup in Settings with an s3:// address first": "請求存檔會上傳到同步備份所用的 S3 儲存桶:請先在設定的「同步與備份」中填寫 s3:// 位址",- "The request archive goes to the S3 bucket sync keeps its backup in, and sync isn't set up with one": "請求存檔會上傳到同步備份所用的 S3 儲存桶,但同步尚未設定 S3","Not in the archive yet: it is uploaded just after the call, or the upload failed (magpie's log says why)": "存檔中尚無此呼叫:呼叫結束後才上傳,或上傳失敗(原因見 magpie 日誌)","Fetch from archive": "從存檔取回","Fetching…": "取回中…",@@ -10373,11 +10375,12 @@ const I18N = {"first 256 KB": "最初の 256 KB","Request archive": "リクエストアーカイブ","Keeps each call’s headers and bodies, secrets taken out, in {where}": "各呼び出しのヘッダーと本文を、秘密情報を除いて {where} に保存します",- "Keeps each call’s headers and bodies, secrets taken out, in your S3 bucket. Set up Sync and backup in Settings with an s3:// address first": "各呼び出しのヘッダーと本文を、秘密情報を除いてあなたの S3 バケットに保存します。先に「設定」の「同期とバックアップ」で s3:// アドレスを設定してください",+ "Keeps each call’s headers and bodies, secrets taken out, on this computer in {where}, the oldest removed past {size}": "各呼び出しのヘッダーと本文を、秘密情報を除いてこのコンピュータの {where} に保存します。{size} を超えると古いものから削除します",+ "Not saved": "未保存",+ "Last save failed: {e}": "前回の保存に失敗しました:{e}",+ "Not in the archive on this computer: it is saved just after the call, the oldest are removed past the archive's size, or the save failed (magpie's log says why)": "このコンピュータのアーカイブにありません。呼び出しの直後に保存され、アーカイブが上限を超えると古いものから削除されます。または保存に失敗しました(理由は magpie のログにあります)","Last upload failed: {e}": "前回のアップロードに失敗しました:{e}","Upload failed": "アップロード失敗",- "The request archive goes to the S3 bucket sync keeps its backup in: set up Sync and backup in Settings with an s3:// address first": "リクエストアーカイブは同期がバックアップを保存する S3 バケットに送られます。先に「設定」の「同期とバックアップ」で s3:// アドレスを設定してください",- "The request archive goes to the S3 bucket sync keeps its backup in, and sync isn't set up with one": "リクエストアーカイブは同期がバックアップを保存する S3 バケットに送られますが、同期に S3 が設定されていません","Not in the archive yet: it is uploaded just after the call, or the upload failed (magpie's log says why)": "まだアーカイブにありません。呼び出しの直後にアップロードされるか、アップロードに失敗しました(理由は magpie のログにあります)","Fetch from archive": "アーカイブから取得","Fetching…": "取得中…",@@ -14296,11 +14299,12 @@ const I18N = {"first 256 KB": "erste 256 KB","Request archive": "Anfragearchiv","Keeps each call’s headers and bodies, secrets taken out, in {where}": "Speichert Header und Bodys jedes Aufrufs ohne Geheimnisse in {where}",- "Keeps each call’s headers and bodies, secrets taken out, in your S3 bucket. Set up Sync and backup in Settings with an s3:// address first": "Speichert Header und Bodys jedes Aufrufs ohne Geheimnisse in Ihrem S3-Bucket. Richten Sie zuerst unter „Einstellungen“ die „Synchronisierung und Backup“ mit einer s3://-Adresse ein",+ "Keeps each call’s headers and bodies, secrets taken out, on this computer in {where}, the oldest removed past {size}": "Speichert Header und Bodys jedes Aufrufs ohne Geheimnisse auf diesem Computer in {where}; über {size} werden die ältesten entfernt",+ "Not saved": "Nicht gespeichert",+ "Last save failed: {e}": "Letztes Speichern fehlgeschlagen: {e}",+ "Not in the archive on this computer: it is saved just after the call, the oldest are removed past the archive's size, or the save failed (magpie's log says why)": "Nicht im Archiv auf diesem Computer: Es wird kurz nach dem Aufruf gespeichert, über der Archivgröße werden die ältesten entfernt, oder das Speichern ist fehlgeschlagen (Grund im magpie-Protokoll)","Last upload failed: {e}": "Letzter Upload fehlgeschlagen: {e}","Upload failed": "Upload fehlgeschlagen",- "The request archive goes to the S3 bucket sync keeps its backup in: set up Sync and backup in Settings with an s3:// address first": "Das Anfragearchiv landet im S3-Bucket, in dem die Synchronisierung ihr Backup ablegt: Richten Sie zuerst unter „Einstellungen“ die „Synchronisierung und Backup“ mit einer s3://-Adresse ein",- "The request archive goes to the S3 bucket sync keeps its backup in, and sync isn't set up with one": "Das Anfragearchiv landet im S3-Bucket, in dem die Synchronisierung ihr Backup ablegt, aber die Synchronisierung ist ohne S3-Bucket eingerichtet","Not in the archive yet: it is uploaded just after the call, or the upload failed (magpie's log says why)": "Noch nicht im Archiv: Der Upload erfolgt kurz nach dem Aufruf, oder er ist fehlgeschlagen (Grund im magpie-Log)","Fetch from archive": "Aus Archiv abrufen","Fetching…": "Wird abgerufen…",diff --git a/internal/gui/settings_keep_test.go b/internal/gui/settings_keep_test.goindex 85d912e06..5261870de 100644--- a/internal/gui/settings_keep_test.go+++ b/internal/gui/settings_keep_test.go@@ -185,6 +185,7 @@ func TestSettingsSaveKeepsWhatItDoesNotSend(t *testing.T) {was.DesktopSubagentEffort = "high"was.CodexAsyncCompact, was.CodexAsyncCompactPercent = true, 70was.ClaudeRunsMost = 3+ was.RequestArchiveLocal, was.RequestArchiveLocalMB = true, 256was.ProxyCustom = "http://me:[email protected]:8080"if err := settings.Save(was); err != nil {t.Fatal(err)diff --git a/internal/gui/tests/request-archive.test.cjs b/internal/gui/tests/request-archive.test.cjsindex 933e0416f..006ae285f 100644--- a/internal/gui/tests/request-archive.test.cjs+++ b/internal/gui/tests/request-archive.test.cjs@@ -1,9 +1,11 @@// Run with Node's test runner and Playwright on the module path; see README.md.// The request archive (Jorben on Discord): its switch sits at the top right// of the Usage page's request list, beside the requests' sum, off unless-// turned on; with sync not set up with an s3:// bucket it can't be turned on-// and says why, with one it posts settings/archive and its tooltip says the-// bucket; a failed upload is said beside it. The Gateway page has no switch+// turned on; with sync not set up with an s3:// bucket it turns on all the+// same and keeps calls on this computer, its tooltip the folder and the size+// they are kept under (CyberCaius on Discord), with one its tooltip says the+// bucket; a failed upload, or a failed save on this computer, is said+// beside it. The Gateway page has no switch// any more, and its recent calls show their own bodies with nothing of the// archive under them, even one it kept: reading a call back from the archive// is the Usage page's (request-archive-usage.test.cjs). No click moves the@@ -32,7 +34,8 @@ const ROWS = [row(0, "kept", KEPT), row(1, "plain")];function serve(lang, seen) {const where = "bkt/team on https://s3.example.com";- const gateway = { running: true, window: true, mine: true, url: "http://127.0.0.1:3999", groups: [], archive: { on: false },+ const here = { folder: "~/.config/magpie/request-archive", cap: 1 << 30 };+ const gateway = { running: true, window: true, mine: true, url: "http://127.0.0.1:3999", groups: [], archive: { on: false, ...here },calls: [call(0, "fixture/kept", KEPT), call(1, "fixture/plain")] };const providers = { providers: [{ id: "fixture", name: "Fixture", icon: "generic", models: [{ id: "kept", name: "Kept", on: true }], agents: [] }], gateway };return async (route) => {@@ -54,8 +57,7 @@ function serve(lang, seen) {const body = route.request().postDataJSON();seen.push(["switch", body]);await new Promise((r) => setTimeout(r, 80));- if (body.on && !seen.bucket) return json({ error: "The request archive goes to the S3 bucket sync keeps its backup in: set up Sync and backup in Settings with an s3:// address first" }, 400);- gateway.archive = { on: body.on, ...(seen.bucket ? { bucket: where } : {}), ...(body.on && seen.failing ? { error: "HTTP 403" } : {}) };+ gateway.archive = { on: body.on, ...(seen.bucket ? { bucket: where } : here), ...(body.on && seen.failing ? { error: seen.bucket ? "HTTP 403" : "disk full" } : {}) };return json(gateway.archive);}if (url.pathname === "/api/archive") {@@ -78,14 +80,14 @@ function serve(lang, seen) {const words = {en: {name: "Request archive", fetch: "Fetch from archive", failed: "Upload failed", failedWhy: "Last upload failed: HTTP 403",- setup: "Set up Sync and backup in Settings with an s3:// address first",- refused: "The request archive goes to the S3 bucket sync keeps its backup in: set up Sync and backup in Settings with an s3:// address first",+ notSaved: "Not saved", notSavedWhy: "Last save failed: disk full",+ local: "Keeps each call’s headers and bodies, secrets taken out, on this computer in ~/.config/magpie/request-archive, the oldest removed past 1.1 GB",bucket: "Keeps each call’s headers and bodies, secrets taken out, in bkt/team on https://s3.example.com",},zh: {name: "请求存档", fetch: "从存档取回", failed: "上传失败", failedWhy: "上次上传失败:HTTP 403",- setup: "请先在设置的“同步与备份”中填写 s3:// 地址",- refused: "请求存档会上传到同步备份所用的 S3 存储桶:请先在设置的“同步与备份”中填写 s3:// 地址",+ notSaved: "未保存", notSavedWhy: "上次保存失败:disk full",+ local: "每次调用的请求/响应头和正文去除密钥后存到本机的 ~/.config/magpie/request-archive,超过 1.1 GB 时先删除最早的",bucket: "每次调用的请求/响应头和正文去除密钥后存到 bkt/team on https://s3.example.com",},};@@ -137,24 +139,29 @@ for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium",assert.equal(await sw.getAttribute("role"), "switch");assert.equal(await sw.innerText(), w.name);assert.equal(await sw.getAttribute("aria-checked"), "false", "off unless turned on");- assert((await sw.getAttribute("title")).includes(w.setup));+ assert.equal(await sw.getAttribute("title"), w.local, "no bucket: kept on this computer");const [swBox, sumBox, listBox] = await Promise.all([sw, page.locator("#ledSum"), page.locator("#ledWrap")].map((l) => l.boundingBox()));assert(Math.abs(swBox.x + swBox.width - (listBox.x + listBox.width)) <= 4, `the switch ends at the list's right edge: ${JSON.stringify([swBox, listBox])}`);assert(swBox.y + swBox.height <= listBox.y, "the switch is over the list");assert(Math.abs(swBox.y + swBox.height / 2 - (sumBox.y + sumBox.height / 2)) <= 3, "on the line of the requests' sum");- // no bucket: turning it on is refused, and it stays off+ // no bucket: on all the same, kept on this computer, and off againawait press(sw, "#view-usage");await wait(1);assert.deepEqual(seen[0], ["switch", { on: true }]);- await page.waitForFunction((s) => document.querySelector("#status").textContent === s, w.refused);+ const onHere = page.locator('#ledArchive .led-arch-sw[aria-checked="true"]:not([disabled])');+ await onHere.waitFor();+ assert.equal(await onHere.getAttribute("title"), w.local);+ assert.equal(await page.locator("#ledArchive .led-arch-err").count(), 0);+ await press(onHere, "#view-usage");+ await wait(2);+ assert.deepEqual(seen[1], ["switch", { on: false }]);await page.locator('#ledArchive .led-arch-sw[aria-checked="false"]:not([disabled])').waitFor();-// with one: on, the bucket in its tooltipseen.bucket = true;await press(page.locator("#ledArchive .led-arch-sw"), "#view-usage");- await wait(2);- assert.deepEqual(seen[1], ["switch", { on: true }]);+ await wait(3);+ assert.deepEqual(seen[2], ["switch", { on: true }]);const on = page.locator('#ledArchive .led-arch-sw[aria-checked="true"]');await on.waitFor();assert.equal(await on.getAttribute("title"), w.bucket);@@ -167,29 +174,41 @@ for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium",// turned off againawait press(on, "#view-usage");- await wait(3);- assert.deepEqual(seen[2], ["switch", { on: false }]);+ await wait(4);+ assert.deepEqual(seen[3], ["switch", { on: false }]);await page.locator('#ledArchive .led-arch-sw[aria-checked="false"]:not([disabled])').waitFor();// an upload that failed is said beside it, why in its tooltipseen.failing = true;await press(page.locator("#ledArchive .led-arch-sw"), "#view-usage");- await wait(4);+ await wait(5);const err = page.locator("#ledArchive .led-arch-err");await err.waitFor();assert.equal(await err.innerText(), w.failed);assert.equal(await err.getAttribute("title"), w.failedWhy);++ // and a save on this computer that failed says so in its own words+ seen.bucket = false;+ await press(page.locator("#ledArchive .led-arch-sw"), "#view-usage");+ await wait(6);+ assert.deepEqual(seen[5], ["switch", { on: false }]);+ await page.locator('#ledArchive .led-arch-sw[aria-checked="false"]:not([disabled])').waitFor();+ await press(page.locator("#ledArchive .led-arch-sw"), "#view-usage");+ await wait(7);+ assert.deepEqual(seen[6], ["switch", { on: true }]);+ await page.waitForFunction((s) => document.querySelector("#ledArchive .led-arch-err")?.textContent === s, w.notSaved);+ assert.equal(await page.locator("#ledArchive .led-arch-err").getAttribute("title"), w.notSavedWhy);assert.equal(seen.filter(([k]) => k === "fetch").length, 0, "nothing was read from the archive");const border = await page.evaluate(() => [...document.querySelectorAll(".led-head, .led-head *")].map((e) => getComputedStyle(e).borderLeftWidth).filter((b) => parseFloat(b) > 1));assert.deepEqual(border, [], "no left-border accent");const missing = await page.evaluate(() => [- "Request archive", "Upload failed", "Last upload failed: {e}",+ "Request archive", "Upload failed", "Last upload failed: {e}", "Not saved", "Last save failed: {e}","Keeps each call’s headers and bodies, secrets taken out, in {where}",- "Keeps each call’s headers and bodies, secrets taken out, in your S3 bucket. Set up Sync and backup in Settings with an s3:// address first",- "The request archive goes to the S3 bucket sync keeps its backup in: set up Sync and backup in Settings with an s3:// address first",- ].filter((k) => !I18N.zh[k]));- assert.deepEqual(missing, [], "every string has its Chinese");+ "Keeps each call’s headers and bodies, secrets taken out, on this computer in {where}, the oldest removed past {size}",+ "Not in the archive on this computer: it is saved just after the call, the oldest are removed past the archive's size, or the save failed (magpie's log says why)",+ ].flatMap((k) => ["zh", "zh-TW", "ja", "de"].filter((l) => !I18N[l][k]).map((l) => l + " " + k)));+ assert.deepEqual(missing, [], "every string has its Chinese, Japanese and German");assert.deepEqual(errors, []);});}diff --git a/internal/settings/archive_local_test.go b/internal/settings/archive_local_test.gonew file mode 100644index 000000000..7910b806f--- /dev/null+++ b/internal/settings/archive_local_test.go@@ -0,0 +1,21 @@+package settings++import "testing"++// The archive on this computer's disk is this computer's own: sync, a+// Restore or a backup file never turns it on, or off, at another+// (CyberCaius on Discord; RequestArchive, the S3 archive's, is synced).+func TestRequestArchiveLocalIsThisComputers(t *testing.T) {+ from := Settings{RequestArchive: true, RequestArchiveLocal: true, RequestArchiveLocalMB: 64}+ from.KeepOwn(Settings{})+ if from.RequestArchiveLocal || from.RequestArchiveLocalMB != 0 {+ t.Fatalf("another computer's local archive came in: %+v", from)+ }+ if !from.RequestArchive {+ t.Fatal("the S3 archive's switch is synced")+ }+ from.KeepOwn(Settings{RequestArchiveLocal: true, RequestArchiveLocalMB: 8})+ if !from.RequestArchiveLocal || from.RequestArchiveLocalMB != 8 {+ t.Fatalf("this computer's own was lost: %+v", from)+ }+}diff --git a/internal/settings/settings.go b/internal/settings/settings.goindex 6ff2bdcba..1ee7c692a 100644--- a/internal/settings/settings.go+++ b/internal/settings/settings.go@@ -158,6 +158,17 @@ type Settings struct {// RequestArchiveMaxMB is how much of each body the archive keeps, in// MiB: 0 for 32, at most 1024 (#447)RequestArchiveMaxMB int `json:"requestArchiveMaxMB,omitempty"`+ // RequestArchiveLocal keeps the request archive on this computer's disk,+ // in magpie's folder (gateway/archive_local.go), while sync has no S3+ // bucket for it (CyberCaius on Discord). Its own switch, separate from+ // RequestArchive, which sync carries to every computer: this computer's+ // own (KeepOwn), so a request body is written to a disk only where the+ // user turned it on.+ RequestArchiveLocal bool `json:"requestArchiveLocal,omitempty"`+ // RequestArchiveLocalMB is how large the archive on this computer grows+ // before its oldest calls are removed, in MiB: 0 for 1024. In+ // settings.json only; this computer's own (KeepOwn).+ RequestArchiveLocalMB int `json:"requestArchiveLocalMB,omitempty"`// GatewayConversations is local, opt-in recording of session traffic.// Only SetGatewayConversations changes it; other saves keep local consent.GatewayConversations bool `json:"gatewayConversations,omitempty"`@@ -1080,6 +1091,7 @@ func (s *Settings) KeepOwn(cur Settings) {s.SessionTerminal = cur.SessionTerminals.RedactModel = cur.RedactModels.ClaudeRunsMost = cur.ClaudeRunsMost+ s.RequestArchiveLocal, s.RequestArchiveLocalMB = cur.RequestArchiveLocal, cur.RequestArchiveLocalMB}// RenamePerModel moves what the user said of a provider's models to the iddiff --git a/site/public/docs/ja/privacy.html b/site/public/docs/ja/privacy.htmlindex f9b6d4d04..01ced4c2d 100644--- a/site/public/docs/ja/privacy.html+++ b/site/public/docs/ja/privacy.html@@ -164,6 +164,7 @@<tr><td><code>partner-counts.json</code>(キャッシュフォルダー)</td><td>まだ送っていないパートナーの回数。最大 14 日分</td></tr></table></div><p><code>install-id</code> と <code>stats-sent</code> は magpie のフォルダーにあります。macOS と Linux では <code>~/.config/magpie</code>、Windows では <code>%USERPROFILE%\.config\magpie</code>、<code>$XDG_CONFIG_HOME</code> が設定されていれば <code>$XDG_CONFIG_HOME/magpie</code>、ポータブル版ではそのデータフォルダーです。<code>partner-counts.json</code> は <code>~/.cache/magpie</code>(<code>$XDG_CACHE_HOME/magpie</code>、ポータブル版ではデータフォルダーの <code>cache</code>)にあります。どれも削除できます。<code>install-id</code> を消すと次の送信で新しいランダムな ID が作られ、新しいユーザーとして数えられます。カウントをオフにすれば、どれも再び書かれません。</p>+ <p>リクエストアーカイブ(<span class="ui">使用量</span>ページのリクエスト一覧の上にある<span class="ui">リクエストアーカイブ</span>スイッチ)は<b>既定でオフ</b>です。同期に S3 バケットがないときにオンにすると、ゲートウェイを通る各呼び出しのヘッダーと本文を、S3 アーカイブと同じようにシークレットを伏せて、magpie のフォルダーの <code>request-archive</code> に 1 呼び出し 1 ファイルで保存します。読めるのはあなたのユーザーだけです。上限は 1 GiB(settings.json の <code>requestArchiveLocalMB</code>)で、超えると古い呼び出しから削除します。このスイッチはこのコンピューターだけのもので、同期で別のコンピューターでオンになることはありません。S3 バケットがあるときはバケットに送られ、ここには何も書かれません。フォルダーはいつでも削除できます。</p><h2 id="site"><a class="anchor" href="#site">usemagpie.ai に見えるもの</a></h2><p>magpie はプライバシー設定にかかわらず、どのアプリも自分の更新サーバーに尋ねるのと同じように、自分のサイトにいくつか尋ねます。どのリクエストにもインストール ID やアカウントは付きません。</p>diff --git a/site/public/docs/privacy.html b/site/public/docs/privacy.htmlindex bbdd4b4d2..715c90a84 100644--- a/site/public/docs/privacy.html+++ b/site/public/docs/privacy.html@@ -164,6 +164,7 @@<tr><td><code>partner-counts.json</code> (in the cache folder)</td><td>Partner counts not yet sent, at most 14 days</td></tr></table></div><p><code>install-id</code> and <code>stats-sent</code> are in magpie’s folder: <code>~/.config/magpie</code> on macOS and Linux, <code>%USERPROFILE%\.config\magpie</code> on Windows, <code>$XDG_CONFIG_HOME/magpie</code> when that is set, and the data folder of a portable magpie. <code>partner-counts.json</code> is in <code>~/.cache/magpie</code> (<code>$XDG_CACHE_HOME/magpie</code>, or <code>cache</code> in a portable magpie’s data folder). You can delete any of them: a deleted <code>install-id</code> is replaced by a new random one at the next send, so you are counted as a new user; with the count off, none of them is written again.</p>+ <p>The request archive (the <span class="ui">Request archive</span> switch on the <span class="ui">Usage</span> page, above the list of requests) is <b>off by default</b>. Turned on while sync has no S3 bucket, it keeps each gateway call’s headers and bodies, secrets masked as for the S3 archive, in <code>request-archive</code> in magpie’s folder, one file per call, readable by your user only. It grows to 1 GiB (<code>requestArchiveLocalMB</code> in settings.json) and the oldest calls are removed past that. The switch is this computer’s own: sync never turns it on at another computer. With an S3 bucket the calls go to the bucket instead, and nothing is written here. Delete the folder whenever you like.</p><h2 id="site"><a class="anchor" href="#site">What usemagpie.ai sees</a></h2><p>magpie asks its own site for a few things, whatever the privacy settings say, as any app asks its update server. None of these requests carries the install id or an account.</p>diff --git a/site/public/docs/zh/privacy.html b/site/public/docs/zh/privacy.htmlindex ec8343391..0972b4b85 100644--- a/site/public/docs/zh/privacy.html+++ b/site/public/docs/zh/privacy.html@@ -164,6 +164,7 @@<tr><td><code>partner-counts.json</code>(缓存文件夹)</td><td>还没发送的合作伙伴次数,最多保留 14 天</td></tr></table></div><p><code>install-id</code> 和 <code>stats-sent</code> 在 magpie 的文件夹里:macOS 和 Linux 上是 <code>~/.config/magpie</code>,Windows 上是 <code>%USERPROFILE%\.config\magpie</code>,设置了 <code>$XDG_CONFIG_HOME</code> 时是 <code>$XDG_CONFIG_HOME/magpie</code>,便携版则在它的数据文件夹里。<code>partner-counts.json</code> 在 <code>~/.cache/magpie</code>(或 <code>$XDG_CACHE_HOME/magpie</code>,便携版数据文件夹里的 <code>cache</code>)。这些文件都可以删:删掉 <code>install-id</code> 后,下次发送会生成新的随机 ID,你会被算作新用户;关掉计数后,它们都不会再写入。</p>+ <p>请求存档(<span class="ui">用量</span>页请求列表上方的<span class="ui">请求存档</span>开关)<b>默认关闭</b>。同步没有 S3 存储桶时打开它,每次经过网关的调用的请求头和正文会存到 magpie 文件夹里的 <code>request-archive</code>,一次调用一个文件,只有你的用户能读,密钥按 S3 存档同样的方式遮掉。它最多占 1 GiB(settings.json 里的 <code>requestArchiveLocalMB</code>),超过后先删最早的调用。这个开关只属于这台电脑,同步不会在别的电脑上把它打开。有 S3 存储桶时调用存到桶里,这里不写任何东西。这个文件夹随时可以删。</p><h2 id="site"><a class="anchor" href="#site">usemagpie.ai 能看到什么</a></h2><p>magpie 会向自己的网站请求几样东西,不受隐私设置影响,就像任何应用访问自己的更新服务器。这些请求都不带安装 ID 或账号。</p>