Skip to content

Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.

gateway: with no S3 bucket the request archive keeps calls on this computer, oldest removed past 1 GiB (CyberCaius)

yetonee219f699ca6d
diff --git a/docs/subsystems/gui-shell.md b/docs/subsystems/gui-shell.md
index 45d559f12..decea538d 100644
--- a/docs/subsystems/gui-shell.md
+++ b/docs/subsystems/gui-shell.md
@@ -312,7 +312,11 @@ record it); a route id is not a call id. When the ring no longer has it,
the story shows the request archive's copy through `archivePanel` if the
route names one (`Route.Archive`, `call.archiveName()`, set only when the
archive was on for that request), and otherwise says how to turn the
-archive on. Nothing more is captured than before. See
+archive on. Nothing more is captured than before. The archive's switch
+(`POST /api/settings/archive`) sets `RequestArchive` while sync has an S3
+bucket and `RequestArchiveLocal` while it has none; `GET
+/api/archive?name=…` reads this computer's copy first and then the bucket's
+(`TestArchiveSwitchKeepsOnThisComputer`, `request-archive.test.cjs`). See
`TestRoutingRequestOpensItsBodies`, `TestArchiveWholeBodies` and
`routing-bodies.test.cjs`.
diff --git a/docs/subsystems/sync-backup.md b/docs/subsystems/sync-backup.md
index df8708d82..f1f9d0a7f 100644
--- a/docs/subsystems/sync-backup.md
+++ b/docs/subsystems/sync-backup.md
@@ -153,8 +153,17 @@ repository can keep the token. The branch is part of the sync state key.
creation returns `errChanged`. Other 422
responses stay actionable errors. Authentication, permission and branch
failures remain distinct from missing files. Rate limits use sync backoff.
-- Request archival remains an S3-only feature; its objects are separate from
- the encrypted setup and usage files.
+- The request archive goes to S3 only when sync has a bucket. Its objects are
+ separate from the encrypted setup and usage files. With no bucket the same
+ files are kept on this computer instead (`gateway.DiskArchive`, in
+ `<settings.Dir()>/request-archive/<date>/<id>.json`, written through a
+ temporary file and renamed, 0600), under its own switch,
+ `RequestArchiveLocal`. That switch and its size, `RequestArchiveLocalMB`
+ (1 GiB unless set; the oldest calls are removed first past it), are in
+ `KeepOwn`, so sync never turns on writing bodies to another computer's
+ disk, and `RequestArchive` synced from a computer with a bucket keeps
+ nothing on one without (`TestArchiveNeverOnDiskUnasked`,
+ `TestRequestArchiveLocalIsThisComputers`).
## Verification
diff --git a/internal/gateway/archive.go b/internal/gateway/archive.go
index b292bb3af..0c0f1a5a4 100644
--- a/internal/gateway/archive.go
+++ b/internal/gateway/archive.go
@@ -3,7 +3,9 @@ package gateway
// The request archive: with it on (settings' RequestArchive), each call the
// gateway serves is kept — the headers and bodies both ways, every secret
// taken out first — in the user's own S3 bucket, the one sync keeps its
-// backup in, at <prefix>/magpie/archive/<date>/<id>.json: the call's date,
+// backup in, at <prefix>/magpie/archive/<date>/<id>.json, or, while sync
+// has no bucket and RequestArchiveLocal is on, in the same file on this
+// computer (archive_local.go): the call's date,
// in UTC, and an id of its own, sent back on the response as
// X-Magpie-Archive-Id. The call, and its rows in the usage log, carry
// "<date>/<id>", for the Gateway and Usage pages to read it back by.
@@ -41,6 +43,7 @@ type Putter interface {
// ArchiveBucket is the bucket the archive goes to, and false when there is
// none (sync is off, or to a WebDAV folder): set by main to the S3 sync's.
+// With none, the archive is this computer's (archiveDest).
var ArchiveBucket func() (Putter, bool)
// ArchiveHeader carries a call's archive id on its response.
@@ -76,15 +79,33 @@ type wire struct {
to Putter
}
-// archiving is r's wire when the archive is on and has a bucket, nil
-// otherwise: nothing is kept, and nothing sent. body is the request's, as
-// Recent calls has it before the first 256 KB is cut from it.
-func archiving(r *http.Request, res *captureResponseWriter, start time.Time, body []byte) *wire {
- if ArchiveBucket == nil || !settings.Load().RequestArchive {
- return nil
+// archiveDest is where a call is archived, and false when it isn't: the S3
+// bucket sync is to while RequestArchive is on, or, with no bucket, this
+// computer's folder while RequestArchiveLocal is on (archive_local.go). A
+// bucket that goes away never sends calls to the disk in its place: that
+// takes the local switch, turned on at this computer.
+func archiveDest() (Putter, bool) {
+ s := settings.Load()
+ if !s.RequestArchive && !s.RequestArchiveLocal {
+ return nil, false // off, as it is unless turned on: sync's file isn't read
+ }
+ if ArchiveBucket != nil {
+ if b, ok := ArchiveBucket(); ok && b != nil {
+ return b, s.RequestArchive
+ }
}
- to, ok := ArchiveBucket()
- if !ok || to == nil {
+ if s.RequestArchiveLocal {
+ return LocalArchive(), true
+ }
+ return nil, false
+}
+
+// archiving is r's wire when the archive is on, nil otherwise: nothing is
+// kept, and nothing sent. body is the request's, as Recent calls has it
+// before the first 256 KB is cut from it.
+func archiving(r *http.Request, res *captureResponseWriter, start time.Time, body []byte) *wire {
+ to, ok := archiveDest()
+ if !ok {
return nil
}
b := make([]byte, 8)
diff --git a/internal/gateway/archive_local.go b/internal/gateway/archive_local.go
new file mode 100644
index 000000000..8cc9f91bd
--- /dev/null
+++ b/internal/gateway/archive_local.go
@@ -0,0 +1,218 @@
+package gateway
+
+// The request archive on this computer (CyberCaius on Discord): while sync
+// has no S3 bucket for it, a call is kept in magpie's own folder, at
+// request-archive/<date>/<id>.json, the same file the bucket would get. It
+// has a switch of its own, settings' RequestArchiveLocal, which sync never
+// carries to another computer, so request bodies are written to a disk only
+// where the user turned it on. The folder grows to RequestArchiveLocalMB
+// (1 GiB unless set), and past that the oldest calls are removed first.
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "io"
+ "io/fs"
+ "os"
+ "path/filepath"
+ "sort"
+ "strings"
+ "sync"
+ "time"
+
+ "github.com/yetone/magpie/internal/settings"
+)
+
+// ErrNotArchived is a name the archive on this computer has nothing at.
+var ErrNotArchived = errors.New("not in the archive on this computer")
+
+// LocalArchiveCap is how large the archive on this computer grows, in
+// bytes: settings' RequestArchiveLocalMB, a GiB unless set.
+func LocalArchiveCap() int64 {
+ mb := settings.Load().RequestArchiveLocalMB
+ if mb <= 0 {
+ mb = 1024
+ }
+ return int64(min(mb, 1<<20)) << 20
+}
+
+// DiskArchive is the archive in a folder on this computer: Put as a bucket
+// is, and Open to read a call back.
+type DiskArchive struct {
+ Dir string
+ // cap is the most it holds, in bytes; LocalArchiveCap when nil
+ cap func() int64
+
+ mu sync.Mutex
+ total int64 // the folder's size as last counted, and added to since
+ known bool // total has been counted
+}
+
+var (
+ localMu sync.Mutex
+ localArchives = map[string]*DiskArchive{}
+)
+
+// LocalArchive is the archive in magpie's folder: request-archive under
+// ~/.config/magpie, or a portable magpie's data folder.
+func LocalArchive() *DiskArchive {
+ dir := filepath.Join(settings.Dir(), "request-archive")
+ localMu.Lock()
+ defer localMu.Unlock()
+ d := localArchives[dir]
+ if d == nil {
+ d = &DiskArchive{Dir: dir}
+ localArchives[dir] = d
+ }
+ return d
+}
+
+// file is where name, "archive/<date>/<id>.json" as ArchiveName makes it,
+// is kept; false for any other name.
+func (d *DiskArchive) file(name string) (string, bool) {
+ rest, ok := strings.CutPrefix(name, "archive/")
+ date, base, ok2 := strings.Cut(rest, "/")
+ id, ok3 := strings.CutSuffix(base, ".json")
+ if !ok || !ok2 || !ok3 {
+ return "", false
+ }
+ if _, ok := ArchiveName(date, id); !ok {
+ return "", false
+ }
+ return filepath.Join(d.Dir, date, id+".json"), true
+}
+
+// Put writes data at name: to a temporary file beside it first, renamed
+// over it once whole, so a reader never sees half a call. Then the oldest
+// calls are removed while the folder is over its size.
+func (d *DiskArchive) Put(_ context.Context, name string, data []byte) error {
+ path, ok := d.file(name)
+ if !ok {
+ return fmt.Errorf("%s is not a request archive's name", name)
+ }
+ dir := filepath.Dir(path)
+ if err := os.MkdirAll(dir, 0o700); err != nil {
+ return err
+ }
+ tmp, err := os.CreateTemp(dir, "."+filepath.Base(path)+"-*.part")
+ if err != nil {
+ return err
+ }
+ _, err = tmp.Write(data)
+ if err == nil {
+ err = tmp.Sync()
+ }
+ if cerr := tmp.Close(); err == nil {
+ err = cerr
+ }
+ if err == nil {
+ err = os.Rename(tmp.Name(), path)
+ }
+ if err != nil {
+ os.Remove(tmp.Name())
+ return err
+ }
+ d.added(int64(len(data)), path)
+ return nil
+}
+
+// Open is the call kept at name, as it is read, and its size;
+// ErrNotArchived when there is none. The caller closes it.
+func (d *DiskArchive) Open(_ context.Context, name string) (io.ReadCloser, int64, error) {
+ path, ok := d.file(name)
+ if !ok {
+ return nil, 0, ErrNotArchived
+ }
+ f, err := os.Open(path)
+ if errors.Is(err, fs.ErrNotExist) {
+ return nil, 0, ErrNotArchived
+ }
+ if err != nil {
+ return nil, 0, err
+ }
+ fi, err := f.Stat()
+ if err != nil {
+ f.Close()
+ return nil, 0, err
+ }
+ return f, fi.Size(), nil
+}
+
+type keptCall struct {
+ path, rel string
+ size int64
+}
+
+// scan is every call in the folder, oldest first: "<date>/<HHMMSS>-…"
+// sorts by when it was made. A temporary file left by a write that never
+// finished, an hour old, is removed on the way.
+func (d *DiskArchive) scan() []keptCall {
+ var out []keptCall
+ filepath.WalkDir(d.Dir, func(p string, e fs.DirEntry, err error) error {
+ if err != nil || e.IsDir() {
+ return nil
+ }
+ fi, err := e.Info()
+ if err != nil {
+ return nil
+ }
+ if strings.HasPrefix(e.Name(), ".") {
+ if strings.HasSuffix(e.Name(), ".part") && time.Since(fi.ModTime()) > time.Hour {
+ os.Remove(p)
+ }
+ return nil
+ }
+ if !strings.HasSuffix(e.Name(), ".json") {
+ return nil
+ }
+ rel, _ := filepath.Rel(d.Dir, p)
+ out = append(out, keptCall{p, filepath.ToSlash(rel), fi.Size()})
+ return nil
+ })
+ sort.Slice(out, func(i, j int) bool { return out[i].rel < out[j].rel })
+ return out
+}
+
+// added counts n more bytes, written at kept, and past the folder's size
+// removes the oldest calls until it is a tenth under it, so the folder is
+// walked again only once that much more is written. The call just kept is
+// never removed, though it alone be over the size.
+func (d *DiskArchive) added(n int64, kept string) {
+ d.mu.Lock()
+ defer d.mu.Unlock()
+ if !d.known {
+ d.total, d.known = 0, true
+ for _, c := range d.scan() {
+ d.total += c.size
+ }
+ } else {
+ d.total += n
+ }
+ most := LocalArchiveCap()
+ if d.cap != nil {
+ most = d.cap()
+ }
+ if d.total <= most {
+ return
+ }
+ calls := d.scan()
+ var total int64
+ for _, c := range calls {
+ total += c.size
+ }
+ goal := most - most/10
+ for _, c := range calls {
+ if total <= goal {
+ break
+ }
+ if c.path == kept {
+ continue
+ }
+ if os.Remove(c.path) == nil {
+ total -= c.size
+ os.Remove(filepath.Dir(c.path)) // the day's folder, once it is empty
+ }
+ }
+ d.total = total
+}
diff --git a/internal/gateway/archive_local_test.go b/internal/gateway/archive_local_test.go
new file mode 100644
index 000000000..1eb35a699
--- /dev/null
+++ b/internal/gateway/archive_local_test.go
@@ -0,0 +1,212 @@
+package gateway
+
+import (
+ "context"
+ "encoding/json"
+ "fmt"
+ "io"
+ "os"
+ "path/filepath"
+ "runtime"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/yetone/magpie/internal/settings"
+)
+
+// With sync not to S3 and the archive's switch on at this computer
+// (CyberCaius on Discord), a call is kept in magpie's folder, the same file
+// the bucket gets: its secrets taken out, its id on the response and the
+// call, written whole through a temporary file, readable by this user only.
+func TestArchiveOnThisComputer(t *testing.T) {
+ fresh(t)
+ serveOn(t, "fake", "k", []string{"m1"}, archiveVendor{})
+ was := ArchiveBucket
+ ArchiveBucket = func() (Putter, bool) { return nil, false } // sync to WebDAV, or none
+ t.Cleanup(func() { ArchiveBucket = was })
+ settings.Save(settings.Settings{RequestArchiveLocal: true})
+ s := New()
+ rec := archivePost(t, s)
+ id := rec.Header().Get(ArchiveHeader)
+ c := s.Recent()[0]
+ name := c.Time.UTC().Format("2006-01-02") + "/" + id
+ if id == "" || c.Archive != name {
+ t.Fatalf("not archived: id %q, call %q", id, c.Archive)
+ }
+ path := filepath.Join(settings.Dir(), "request-archive", filepath.FromSlash(name)+".json")
+ data, err := os.ReadFile(path)
+ if err != nil {
+ t.Fatalf("not on this computer: %v", err)
+ }
+ for _, secret := range []string{archiveKey, "named-secret", "magpie-gateway-key", "another-secret", "cookie-secret", "abc123", "query-secret"} {
+ if strings.Contains(string(data), secret) {
+ t.Errorf("%s in the archive:\n%s", secret, data)
+ }
+ }
+ var a Archived
+ if err := json.Unmarshal(data, &a); err != nil {
+ t.Fatal(err)
+ }
+ if a.ID != name || a.Call.Model != "fake/m1" || a.Request.Headers["Authorization"][0] != "[REDACTED]" ||
+ !strings.Contains(a.Request.Body, "deploy with [REDACTED:API_KEY]") || !strings.Contains(a.Response.Body, `"content":"done"`) {
+ t.Errorf("archived %+v", a)
+ }
+ if fi, _ := os.Stat(path); runtime.GOOS != "windows" && fi.Mode().Perm() != 0o600 {
+ t.Errorf("mode %v, want 0600", fi.Mode().Perm())
+ }
+ if left, _ := filepath.Glob(filepath.Join(filepath.Dir(path), ".*")); len(left) != 0 {
+ t.Errorf("temporary files left: %v", left)
+ }
+ // read back by its name, as the Usage page asks for it
+ date, aid, _ := strings.Cut(name, "/")
+ file, _ := ArchiveName(date, aid)
+ r, size, err := LocalArchive().Open(context.Background(), file)
+ if err != nil || size != int64(len(data)) {
+ t.Fatalf("Open: %d bytes, %v", size, err)
+ }
+ r.Close()
+ if _, _, err := LocalArchive().Open(context.Background(), "archive/2026-10-02/101010-00000000000000bb.json"); err != ErrNotArchived {
+ t.Errorf("a call never kept: %v", err)
+ }
+}
+
+// Request bodies are written to a disk only where the user turned the
+// archive on at that computer: RequestArchive, which sync carries from a
+// computer with an S3 bucket, keeps nothing on one with none, and the
+// local switch sends nothing to the disk while a bucket is there.
+func TestArchiveNeverOnDiskUnasked(t *testing.T) {
+ fresh(t)
+ serveOn(t, "fake", "k", []string{"m1"}, archiveVendor{})
+ was := ArchiveBucket
+ t.Cleanup(func() { ArchiveBucket = was })
+ dir := filepath.Join(settings.Dir(), "request-archive")
+ onDisk := func() []string {
+ var got []string
+ filepath.Walk(dir, func(p string, fi os.FileInfo, err error) error {
+ if err == nil && !fi.IsDir() {
+ got = append(got, p)
+ }
+ return nil
+ })
+ return got
+ }
+
+ // off, as it is unless turned on
+ ArchiveBucket = func() (Putter, bool) { return nil, false }
+ s := New()
+ if rec := archivePost(t, s); rec.Header().Get(ArchiveHeader) != "" || len(onDisk()) != 0 {
+ t.Fatalf("archived while off: %v", onDisk())
+ }
+ // the S3 archive's switch, synced here, with no bucket here
+ settings.Save(settings.Settings{RequestArchive: true})
+ if rec := archivePost(t, s); rec.Header().Get(ArchiveHeader) != "" || len(onDisk()) != 0 {
+ t.Fatalf("the synced switch wrote to this computer: %v", onDisk())
+ }
+ // the local switch with a bucket there: the bucket's switch decides
+ b := &memBucket{objs: map[string][]byte{}}
+ archiveTo(t, b)
+ settings.Save(settings.Settings{RequestArchiveLocal: true})
+ if rec := archivePost(t, s); rec.Header().Get(ArchiveHeader) != "" || len(onDisk()) != 0 || len(b.objs) != 0 {
+ t.Fatalf("archived with the bucket's switch off: disk %v, bucket %v", onDisk(), keys(b.objs))
+ }
+ settings.Save(settings.Settings{RequestArchive: true, RequestArchiveLocal: true})
+ if archivePost(t, s); len(b.objs) != 1 || len(onDisk()) != 0 {
+ t.Fatalf("with a bucket: bucket %v, disk %v", keys(b.objs), onDisk())
+ }
+}
+
+// Past its size the folder loses its oldest calls first, down to a tenth
+// under it; the call just kept stays, and a temporary file a write left an
+// hour ago goes.
+func TestLocalArchiveRemovesOldestPastItsSize(t *testing.T) {
+ d := &DiskArchive{Dir: t.TempDir(), cap: func() int64 { return 1000 }}
+ ctx := context.Background()
+ body := []byte(strings.Repeat("x", 200))
+ name := func(day, i int) string {
+ return fmt.Sprintf("archive/2026-10-%02d/%06d-00000000000000aa.json", day, 100000+i)
+ }
+ stale := filepath.Join(d.Dir, "2026-10-01", ".old.json-1.part")
+ os.MkdirAll(filepath.Dir(stale), 0o700)
+ os.WriteFile(stale, body, 0o600)
+ old := time.Now().Add(-2 * time.Hour)
+ os.Chtimes(stale, old, old)
+ // four on the 1st, one on the 2nd: 1000 bytes, at the size
+ for i := range 4 {
+ if err := d.Put(ctx, name(1, i), body); err != nil {
+ t.Fatal(err)
+ }
+ }
+ if err := d.Put(ctx, name(2, 0), body); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := os.Stat(stale); err == nil {
+ t.Error("an hour-old temporary file is still there")
+ }
+ has := func(n string) bool {
+ r, _, err := d.Open(ctx, n)
+ if err == nil {
+ r.Close()
+ }
+ return err == nil
+ }
+ for i := range 4 {
+ if !has(name(1, i)) {
+ t.Fatalf("removed at the size, not past it: %s", name(1, i))
+ }
+ }
+ // one more is past it: the two oldest go, 800 bytes left
+ if err := d.Put(ctx, name(2, 1), body); err != nil {
+ t.Fatal(err)
+ }
+ for i, want := range []bool{false, false, true, true} {
+ if has(name(1, i)) != want {
+ t.Errorf("%s kept %v, want %v", name(1, i), !want, want)
+ }
+ }
+ if !has(name(2, 0)) || !has(name(2, 1)) {
+ t.Error("a newer call was removed")
+ }
+ // one alone larger than the size is kept, and every other goes
+ big := []byte(strings.Repeat("y", 1500))
+ if err := d.Put(ctx, name(3, 0), big); err != nil {
+ t.Fatal(err)
+ }
+ if !has(name(3, 0)) {
+ t.Error("the call just kept was removed")
+ }
+ for _, n := range []string{name(1, 2), name(1, 3), name(2, 0), name(2, 1)} {
+ if has(n) {
+ t.Errorf("%s kept past the size", n)
+ }
+ }
+ if days, _ := os.ReadDir(d.Dir); len(days) != 1 {
+ t.Errorf("empty days left: %v", days)
+ }
+ r, _, _ := d.Open(ctx, name(3, 0))
+ got, _ := io.ReadAll(r)
+ r.Close()
+ if string(got) != string(big) {
+ t.Errorf("read back %d bytes", len(got))
+ }
+ // a name not of the archive's shape is never a path
+ for _, n := range []string{"archive/../../settings.json", "archive/2026-10-02/../x.json", "settings.json", "archive/2026-10-02/ABC.json"} {
+ if err := d.Put(ctx, n, body); err == nil {
+ t.Errorf("Put %s", n)
+ }
+ if _, _, err := d.Open(ctx, n); err != ErrNotArchived {
+ t.Errorf("Open %s: %v", n, err)
+ }
+ }
+}
+
+func TestLocalArchiveCap(t *testing.T) {
+ t.Setenv("XDG_CONFIG_HOME", t.TempDir())
+ if got := LocalArchiveCap(); got != 1<<30 {
+ t.Errorf("default %d, want a GiB", got)
+ }
+ settings.Save(settings.Settings{RequestArchiveLocalMB: 5})
+ if got := LocalArchiveCap(); got != 5<<20 {
+ t.Errorf("5 MiB: %d", got)
+ }
+}
diff --git a/internal/gui/api.go b/internal/gui/api.go
index 7c0f85682..b85d25250 100644
--- a/internal/gui/api.go
+++ b/internal/gui/api.go
@@ -1239,6 +1239,9 @@ func Handler(w Windows, gw *gateway.Server) http.Handler {
in.RedactActions = cur.RedactActions // set on their own (redact-actions below)
in.RedactAllow = cur.RedactAllow // set on its own (redact-allow below)
in.DisabledModels = cur.DisabledModels // set on their own (disabled-models below, #1867)
+ // the request archive on this computer: the archive's switch while no
+ // S3 bucket, and its size in settings.json only
+ in.RequestArchiveLocal, in.RequestArchiveLocalMB = cur.RequestArchiveLocal, cur.RequestArchiveLocalMB
// used or left is the Usage page's toggle as much as Settings', set on its own
in.QuotaLeft = cur.QuotaLeft
in.QuotaReads = cur.QuotaReads // set on its own (quota-reads below)
diff --git a/internal/gui/archive.go b/internal/gui/archive.go
index e4d392471..ff9602336 100644
--- a/internal/gui/archive.go
+++ b/internal/gui/archive.go
@@ -27,6 +27,10 @@ var archiveBucket = func() (archiveStore, bool) {
return b, ok
}
+// localArchive is the archive on this computer, kept while sync has no S3
+// bucket (gateway/archive_local.go).
+var localArchive = func() archiveStore { return gateway.LocalArchive() }
+
// archivePreviewBody is the most of a body the page is sent to show; a
// longer one is left out, for the file to be downloaded (#447). Its size
// is told here, so a large body never reaches the page.
@@ -66,16 +70,24 @@ func preview(p gateway.ArchivePart) *previewPart {
}
// openArchive is the call named by r's ?date= and ?id=, as it is read from
-// the bucket, its size, and "<date>-<id>"; the error is the page's to show.
+// this computer's archive, or else the bucket, its size, and "<date>-<id>";
+// the error is the page's to show. A call kept here while sync had no
+// bucket is read here after one is set up, and the other way round.
func openArchive(r *http.Request) (io.ReadCloser, int64, string, error) {
date, id := r.URL.Query().Get("date"), r.URL.Query().Get("id")
name, ok := gateway.ArchiveName(date, id)
if !ok {
return nil, 0, "", fmt.Errorf("%s/%s is not a request archive's date and id", date, id)
}
+ if body, size, err := localArchive().Open(r.Context(), name); !errors.Is(err, gateway.ErrNotArchived) {
+ if err != nil {
+ return nil, 0, "", err
+ }
+ return body, size, date + "-" + id, nil
+ }
b, ok := archiveBucket()
if !ok {
- return nil, 0, "", errors.New("The request archive goes to the S3 bucket sync keeps its backup in, and sync isn't set up with one")
+ return nil, 0, "", errors.New("Not in the archive on this computer: it is saved just after the call, the oldest are removed past the archive's size, or the save failed (magpie's log says why)")
}
body, size, err := b.Open(r.Context(), name)
if errors.Is(err, davsync.ErrNoObject) {
@@ -87,9 +99,12 @@ func openArchive(r *http.Request) (io.ReadCloser, int64, string, error) {
return body, size, date + "-" + id, nil
}
-// archiveRoutes are the request archive: its switch on the Gateway page, a
-// call read back from the bucket by its date and id, for the Gateway and
-// Usage pages, and its file, downloaded whole (gateway/archive.go).
+// archiveRoutes are the request archive: its switch on the Usage page, a
+// call read back from this computer or the bucket by its date and id, for
+// the Usage and Routing pages, and its file, downloaded whole
+// (gateway/archive.go). With sync to an S3 bucket the switch is
+// RequestArchive, which sync carries to every computer; with none it is
+// RequestArchiveLocal, this computer's own, and calls are kept on its disk.
func archiveRoutes(mux *http.ServeMux) {
mux.HandleFunc("POST /api/settings/archive", func(rw http.ResponseWriter, r *http.Request) {
var in struct{ On bool }
@@ -97,12 +112,12 @@ func archiveRoutes(mux *http.ServeMux) {
fail(rw, err)
return
}
- if _, ok := davsync.S3Bucket(); in.On && !ok {
- fail(rw, errors.New("The request archive goes to the S3 bucket sync keeps its backup in: set up Sync and backup in Settings with an s3:// address first"))
- return
- }
s := settings.Load()
- s.RequestArchive = in.On
+ if _, ok := davsync.S3Bucket(); ok {
+ s.RequestArchive = in.On
+ } else {
+ s.RequestArchiveLocal = in.On
+ }
if err := settings.Save(s); err != nil {
fail(rw, err)
return
@@ -200,19 +215,25 @@ func saveDownload(dir, stem, ext string, r io.Reader) (string, error) {
return name, nil
}
-// archiveJSON is the request archive as the Gateway page shows it: on or
-// off, the bucket it goes to (none when sync isn't to S3), and why the last
-// upload failed, if it did.
+// archiveJSON is the request archive as the Usage page shows it: on or
+// off, the bucket it goes to, or, when sync isn't to S3, the folder on this
+// computer and the size it is kept under, and why the last upload or save
+// failed, if it did.
type archiveJSON struct {
On bool `json:"on"`
Bucket string `json:"bucket,omitempty"`
+ Folder string `json:"folder,omitempty"`
+ Cap int64 `json:"cap,omitempty"` // bytes
Error string `json:"error,omitempty"`
}
func archiveState() archiveJSON {
- a := archiveJSON{On: settings.Load().RequestArchive}
+ s := settings.Load()
+ var a archiveJSON
if b, ok := davsync.S3Bucket(); ok {
- a.Bucket = b.Where()
+ a.On, a.Bucket = s.RequestArchive, b.Where()
+ } else {
+ a.On, a.Folder, a.Cap = s.RequestArchiveLocal, tilde(gateway.LocalArchive().Dir), gateway.LocalArchiveCap()
}
if a.On {
a.Error, _ = gateway.ArchiveError()
diff --git a/internal/gui/archive_test.go b/internal/gui/archive_test.go
index e66414942..6d7ab1389 100644
--- a/internal/gui/archive_test.go
+++ b/internal/gui/archive_test.go
@@ -14,6 +14,7 @@ import (
"github.com/yetone/magpie/internal/davsync"
"github.com/yetone/magpie/internal/gateway"
+ "github.com/yetone/magpie/internal/settings"
)
// fakeArchive is a bucket of archived calls in memory; size, when set,
@@ -145,3 +146,50 @@ func TestArchiveOldPreview(t *testing.T) {
t.Fatalf("%d %s", rec.Code, rec.Body)
}
}
+
+// With sync not to S3 the switch turns on the archive on this computer,
+// its own setting, not the S3 one sync carries; the page is told the folder
+// and its size, and a call kept there is read back and downloaded from it
+// (CyberCaius on Discord).
+func TestArchiveSwitchKeepsOnThisComputer(t *testing.T) {
+ h := archiveHome(t, fakeArchive{objs: map[string][]byte{}})
+ archiveBucket = func() (archiveStore, bool) { return nil, false }
+ rec := httptest.NewRecorder()
+ h.ServeHTTP(rec, httptest.NewRequest("POST", "/api/settings/archive", strings.NewReader(`{"on":true}`)))
+ var a archiveJSON
+ json.Unmarshal(rec.Body.Bytes(), &a)
+ if rec.Code != 200 || !a.On || a.Bucket != "" || !strings.HasSuffix(a.Folder, "request-archive") || a.Cap != 1<<30 {
+ t.Fatalf("%d %s", rec.Code, rec.Body)
+ }
+ if s := settings.Load(); !s.RequestArchiveLocal || s.RequestArchive {
+ t.Fatalf("switched local %v, synced %v", s.RequestArchiveLocal, s.RequestArchive)
+ }
+ obj := archivedCall(t, `{"q":"here"}`, `{"a":"kept"}`)
+ if err := gateway.LocalArchive().Put(context.Background(), archiveObject, obj); err != nil {
+ t.Fatal(err)
+ }
+ rec = httptest.NewRecorder()
+ h.ServeHTTP(rec, httptest.NewRequest("GET", "/api/archive"+archiveQuery, nil))
+ var p archivePreview
+ json.Unmarshal(rec.Body.Bytes(), &p)
+ if rec.Code != 200 || p.Request == nil || p.Request.Body != `{"q":"here"}` || p.Bytes != int64(len(obj)) {
+ t.Fatalf("read back %d %s", rec.Code, rec.Body)
+ }
+ rec = httptest.NewRecorder()
+ h.ServeHTTP(rec, httptest.NewRequest("GET", "/api/archive/file"+archiveQuery, nil))
+ if rec.Code != 200 || !bytes.Equal(rec.Body.Bytes(), obj) {
+ t.Fatalf("download %d, %d bytes", rec.Code, rec.Body.Len())
+ }
+ // one it never kept says so in this computer's words
+ rec = httptest.NewRecorder()
+ h.ServeHTTP(rec, httptest.NewRequest("GET", "/api/archive?date=2026-10-02&id=101010-00000000000000bb", nil))
+ if rec.Code != http.StatusBadRequest || !strings.Contains(rec.Body.String(), "Not in the archive on this computer") {
+ t.Fatalf("missing: %d %s", rec.Code, rec.Body)
+ }
+ // and off again
+ rec = httptest.NewRecorder()
+ h.ServeHTTP(rec, httptest.NewRequest("POST", "/api/settings/archive", strings.NewReader(`{"on":false}`)))
+ if s := settings.Load(); rec.Code != 200 || s.RequestArchiveLocal {
+ t.Fatalf("off: %d %s", rec.Code, rec.Body)
+ }
+}
diff --git a/internal/gui/assets/app.js b/internal/gui/assets/app.js
index 4efff79e5..9d3fab0f1 100644
--- a/internal/gui/assets/app.js
+++ b/internal/gui/assets/app.js
@@ -7074,13 +7074,10 @@ function callBodyPanel(label, raw, truncated, id, key = id) {
return panel;
}
-// The request archive: with it on, each call's headers and bodies, secrets
-// taken out, go to the S3 bucket sync keeps its backup in, and a request's
-// row on the Usage page reads its own back from there (gateway/archive.go).
-// Its switch sits over that list, at its right (Jorben on Discord); what it
-// keeps, and where, or what it needs first, is its tooltip, and a failed
-// upload says so beside it. The switch is the same element across redraws,
-// changed in place, so one being pressed or focused is never swapped out.
+// The request archive: each call, secrets out, kept in sync's S3 bucket or
+// else a folder here (gateway/archive*.go), read back by its Usage row. Its
+// switch is over that list (Jorben); where it keeps is its tooltip, a failed
+// save is beside it. It is changed in place, never swapped out mid-press.
let archiveSaving = false;
function renderArchive() {
const box = $("#ledArchive");
@@ -7114,12 +7111,12 @@ function renderArchive() {
b.disabled = archiveSaving;
b.querySelector(".led-arch-name").textContent = t("Request archive");
b.title = a.bucket ? t("Keeps each call’s headers and bodies, secrets taken out, in {where}", { where: a.bucket })
- : t("Keeps each call’s headers and bodies, secrets taken out, in your S3 bucket. Set up Sync and backup in Settings with an s3:// address first");
+ : t("Keeps each call’s headers and bodies, secrets taken out, on this computer in {where}, the oldest removed past {size}", { where: a.folder || "", size: fmtBytes(a.cap || 0) });
let err = box.querySelector(".led-arch-err");
if (a.on && a.error) {
if (!err) { err = el("span", "led-arch-err"); box.prepend(err); }
- err.textContent = t("Upload failed");
- err.title = t("Last upload failed: {e}", { e: a.error });
+ err.textContent = t(a.bucket ? "Upload failed" : "Not saved");
+ err.title = t(a.bucket ? "Last upload failed: {e}" : "Last save failed: {e}", { e: a.error });
} else err?.remove();
}
diff --git a/internal/gui/assets/i18n.js b/internal/gui/assets/i18n.js
index 14d0b66cc..2f1da3b86 100644
--- a/internal/gui/assets/i18n.js
+++ b/internal/gui/assets/i18n.js
@@ -2545,11 +2545,12 @@ const I18N = {
"first 256 KB": "前 256 KB",
"Request archive": "请求存档",
"Keeps each call’s headers and bodies, secrets taken out, in {where}": "每次调用的请求/响应头和正文去除密钥后存到 {where}",
- "Keeps each call’s headers and bodies, secrets taken out, in your S3 bucket. Set up Sync and backup in Settings with an s3:// address first": "每次调用的请求/响应头和正文去除密钥后存到你的 S3 存储桶。请先在设置的“同步与备份”中填写 s3:// 地址",
+ "Keeps each call’s headers and bodies, secrets taken out, on this computer in {where}, the oldest removed past {size}": "每次调用的请求/响应头和正文去除密钥后存到本机的 {where},超过 {size} 时先删除最早的",
+ "Not saved": "未保存",
+ "Last save failed: {e}": "上次保存失败:{e}",
+ "Not in the archive on this computer: it is saved just after the call, the oldest are removed past the archive's size, or the save failed (magpie's log says why)": "本机存档中没有此调用:调用结束后才保存,存档超过大小上限时会删除最早的,或保存失败(原因见 magpie 日志)",
"Last upload failed: {e}": "上次上传失败:{e}",
"Upload failed": "上传失败",
- "The request archive goes to the S3 bucket sync keeps its backup in: set up Sync and backup in Settings with an s3:// address first": "请求存档会上传到同步备份所用的 S3 存储桶:请先在设置的“同步与备份”中填写 s3:// 地址",
- "The request archive goes to the S3 bucket sync keeps its backup in, and sync isn't set up with one": "请求存档会上传到同步备份所用的 S3 存储桶,但同步尚未设置 S3",
"Not in the archive yet: it is uploaded just after the call, or the upload failed (magpie's log says why)": "存档中尚无此调用:调用结束后才上传,或上传失败(原因见 magpie 日志)",
"Fetch from archive": "从存档取回",
"Fetching…": "取回中…",
@@ -6460,11 +6461,12 @@ const I18N = {
"first 256 KB": "前 256 KB",
"Request archive": "請求存檔",
"Keeps each call’s headers and bodies, secrets taken out, in {where}": "每次呼叫的請求/回應頭和正文去除金鑰後存到 {where}",
- "Keeps each call’s headers and bodies, secrets taken out, in your S3 bucket. Set up Sync and backup in Settings with an s3:// address first": "每次呼叫的請求/回應頭和正文去除金鑰後存到你的 S3 儲存桶。請先在設定的「同步與備份」中填寫 s3:// 位址",
+ "Keeps each call’s headers and bodies, secrets taken out, on this computer in {where}, the oldest removed past {size}": "每次呼叫的請求/回應頭和正文去除金鑰後存到本機的 {where},超過 {size} 時先刪除最早的",
+ "Not saved": "未儲存",
+ "Last save failed: {e}": "上次儲存失敗:{e}",
+ "Not in the archive on this computer: it is saved just after the call, the oldest are removed past the archive's size, or the save failed (magpie's log says why)": "本機存檔中沒有此呼叫:呼叫結束後才儲存,存檔超過大小上限時會刪除最早的,或儲存失敗(原因見 magpie 日誌)",
"Last upload failed: {e}": "上次上傳失敗:{e}",
"Upload failed": "上傳失敗",
- "The request archive goes to the S3 bucket sync keeps its backup in: set up Sync and backup in Settings with an s3:// address first": "請求存檔會上傳到同步備份所用的 S3 儲存桶:請先在設定的「同步與備份」中填寫 s3:// 位址",
- "The request archive goes to the S3 bucket sync keeps its backup in, and sync isn't set up with one": "請求存檔會上傳到同步備份所用的 S3 儲存桶,但同步尚未設定 S3",
"Not in the archive yet: it is uploaded just after the call, or the upload failed (magpie's log says why)": "存檔中尚無此呼叫:呼叫結束後才上傳,或上傳失敗(原因見 magpie 日誌)",
"Fetch from archive": "從存檔取回",
"Fetching…": "取回中…",
@@ -10373,11 +10375,12 @@ const I18N = {
"first 256 KB": "最初の 256 KB",
"Request archive": "リクエストアーカイブ",
"Keeps each call’s headers and bodies, secrets taken out, in {where}": "各呼び出しのヘッダーと本文を、秘密情報を除いて {where} に保存します",
- "Keeps each call’s headers and bodies, secrets taken out, in your S3 bucket. Set up Sync and backup in Settings with an s3:// address first": "各呼び出しのヘッダーと本文を、秘密情報を除いてあなたの S3 バケットに保存します。先に「設定」の「同期とバックアップ」で s3:// アドレスを設定してください",
+ "Keeps each call’s headers and bodies, secrets taken out, on this computer in {where}, the oldest removed past {size}": "各呼び出しのヘッダーと本文を、秘密情報を除いてこのコンピュータの {where} に保存します。{size} を超えると古いものから削除します",
+ "Not saved": "未保存",
+ "Last save failed: {e}": "前回の保存に失敗しました:{e}",
+ "Not in the archive on this computer: it is saved just after the call, the oldest are removed past the archive's size, or the save failed (magpie's log says why)": "このコンピュータのアーカイブにありません。呼び出しの直後に保存され、アーカイブが上限を超えると古いものから削除されます。または保存に失敗しました(理由は magpie のログにあります)",
"Last upload failed: {e}": "前回のアップロードに失敗しました:{e}",
"Upload failed": "アップロード失敗",
- "The request archive goes to the S3 bucket sync keeps its backup in: set up Sync and backup in Settings with an s3:// address first": "リクエストアーカイブは同期がバックアップを保存する S3 バケットに送られます。先に「設定」の「同期とバックアップ」で s3:// アドレスを設定してください",
- "The request archive goes to the S3 bucket sync keeps its backup in, and sync isn't set up with one": "リクエストアーカイブは同期がバックアップを保存する S3 バケットに送られますが、同期に S3 が設定されていません",
"Not in the archive yet: it is uploaded just after the call, or the upload failed (magpie's log says why)": "まだアーカイブにありません。呼び出しの直後にアップロードされるか、アップロードに失敗しました(理由は magpie のログにあります)",
"Fetch from archive": "アーカイブから取得",
"Fetching…": "取得中…",
@@ -14296,11 +14299,12 @@ const I18N = {
"first 256 KB": "erste 256 KB",
"Request archive": "Anfragearchiv",
"Keeps each call’s headers and bodies, secrets taken out, in {where}": "Speichert Header und Bodys jedes Aufrufs ohne Geheimnisse in {where}",
- "Keeps each call’s headers and bodies, secrets taken out, in your S3 bucket. Set up Sync and backup in Settings with an s3:// address first": "Speichert Header und Bodys jedes Aufrufs ohne Geheimnisse in Ihrem S3-Bucket. Richten Sie zuerst unter „Einstellungen“ die „Synchronisierung und Backup“ mit einer s3://-Adresse ein",
+ "Keeps each call’s headers and bodies, secrets taken out, on this computer in {where}, the oldest removed past {size}": "Speichert Header und Bodys jedes Aufrufs ohne Geheimnisse auf diesem Computer in {where}; über {size} werden die ältesten entfernt",
+ "Not saved": "Nicht gespeichert",
+ "Last save failed: {e}": "Letztes Speichern fehlgeschlagen: {e}",
+ "Not in the archive on this computer: it is saved just after the call, the oldest are removed past the archive's size, or the save failed (magpie's log says why)": "Nicht im Archiv auf diesem Computer: Es wird kurz nach dem Aufruf gespeichert, über der Archivgröße werden die ältesten entfernt, oder das Speichern ist fehlgeschlagen (Grund im magpie-Protokoll)",
"Last upload failed: {e}": "Letzter Upload fehlgeschlagen: {e}",
"Upload failed": "Upload fehlgeschlagen",
- "The request archive goes to the S3 bucket sync keeps its backup in: set up Sync and backup in Settings with an s3:// address first": "Das Anfragearchiv landet im S3-Bucket, in dem die Synchronisierung ihr Backup ablegt: Richten Sie zuerst unter „Einstellungen“ die „Synchronisierung und Backup“ mit einer s3://-Adresse ein",
- "The request archive goes to the S3 bucket sync keeps its backup in, and sync isn't set up with one": "Das Anfragearchiv landet im S3-Bucket, in dem die Synchronisierung ihr Backup ablegt, aber die Synchronisierung ist ohne S3-Bucket eingerichtet",
"Not in the archive yet: it is uploaded just after the call, or the upload failed (magpie's log says why)": "Noch nicht im Archiv: Der Upload erfolgt kurz nach dem Aufruf, oder er ist fehlgeschlagen (Grund im magpie-Log)",
"Fetch from archive": "Aus Archiv abrufen",
"Fetching…": "Wird abgerufen…",
diff --git a/internal/gui/settings_keep_test.go b/internal/gui/settings_keep_test.go
index 85d912e06..5261870de 100644
--- a/internal/gui/settings_keep_test.go
+++ b/internal/gui/settings_keep_test.go
@@ -185,6 +185,7 @@ func TestSettingsSaveKeepsWhatItDoesNotSend(t *testing.T) {
was.DesktopSubagentEffort = "high"
was.CodexAsyncCompact, was.CodexAsyncCompactPercent = true, 70
was.ClaudeRunsMost = 3
+ was.RequestArchiveLocal, was.RequestArchiveLocalMB = true, 256
was.ProxyCustom = "http://me:[email protected]:8080"
if err := settings.Save(was); err != nil {
t.Fatal(err)
diff --git a/internal/gui/tests/request-archive.test.cjs b/internal/gui/tests/request-archive.test.cjs
index 933e0416f..006ae285f 100644
--- a/internal/gui/tests/request-archive.test.cjs
+++ b/internal/gui/tests/request-archive.test.cjs
@@ -1,9 +1,11 @@
// Run with Node's test runner and Playwright on the module path; see README.md.
// The request archive (Jorben on Discord): its switch sits at the top right
// of the Usage page's request list, beside the requests' sum, off unless
-// turned on; with sync not set up with an s3:// bucket it can't be turned on
-// and says why, with one it posts settings/archive and its tooltip says the
-// bucket; a failed upload is said beside it. The Gateway page has no switch
+// turned on; with sync not set up with an s3:// bucket it turns on all the
+// same and keeps calls on this computer, its tooltip the folder and the size
+// they are kept under (CyberCaius on Discord), with one its tooltip says the
+// bucket; a failed upload, or a failed save on this computer, is said
+// beside it. The Gateway page has no switch
// any more, and its recent calls show their own bodies with nothing of the
// archive under them, even one it kept: reading a call back from the archive
// is the Usage page's (request-archive-usage.test.cjs). No click moves the
@@ -32,7 +34,8 @@ const ROWS = [row(0, "kept", KEPT), row(1, "plain")];
function serve(lang, seen) {
const where = "bkt/team on https://s3.example.com";
- const gateway = { running: true, window: true, mine: true, url: "http://127.0.0.1:3999", groups: [], archive: { on: false },
+ const here = { folder: "~/.config/magpie/request-archive", cap: 1 << 30 };
+ const gateway = { running: true, window: true, mine: true, url: "http://127.0.0.1:3999", groups: [], archive: { on: false, ...here },
calls: [call(0, "fixture/kept", KEPT), call(1, "fixture/plain")] };
const providers = { providers: [{ id: "fixture", name: "Fixture", icon: "generic", models: [{ id: "kept", name: "Kept", on: true }], agents: [] }], gateway };
return async (route) => {
@@ -54,8 +57,7 @@ function serve(lang, seen) {
const body = route.request().postDataJSON();
seen.push(["switch", body]);
await new Promise((r) => setTimeout(r, 80));
- if (body.on && !seen.bucket) return json({ error: "The request archive goes to the S3 bucket sync keeps its backup in: set up Sync and backup in Settings with an s3:// address first" }, 400);
- gateway.archive = { on: body.on, ...(seen.bucket ? { bucket: where } : {}), ...(body.on && seen.failing ? { error: "HTTP 403" } : {}) };
+ gateway.archive = { on: body.on, ...(seen.bucket ? { bucket: where } : here), ...(body.on && seen.failing ? { error: seen.bucket ? "HTTP 403" : "disk full" } : {}) };
return json(gateway.archive);
}
if (url.pathname === "/api/archive") {
@@ -78,14 +80,14 @@ function serve(lang, seen) {
const words = {
en: {
name: "Request archive", fetch: "Fetch from archive", failed: "Upload failed", failedWhy: "Last upload failed: HTTP 403",
- setup: "Set up Sync and backup in Settings with an s3:// address first",
- refused: "The request archive goes to the S3 bucket sync keeps its backup in: set up Sync and backup in Settings with an s3:// address first",
+ notSaved: "Not saved", notSavedWhy: "Last save failed: disk full",
+ local: "Keeps each call’s headers and bodies, secrets taken out, on this computer in ~/.config/magpie/request-archive, the oldest removed past 1.1 GB",
bucket: "Keeps each call’s headers and bodies, secrets taken out, in bkt/team on https://s3.example.com",
},
zh: {
name: "请求存档", fetch: "从存档取回", failed: "上传失败", failedWhy: "上次上传失败:HTTP 403",
- setup: "请先在设置的“同步与备份”中填写 s3:// 地址",
- refused: "请求存档会上传到同步备份所用的 S3 存储桶:请先在设置的“同步与备份”中填写 s3:// 地址",
+ notSaved: "未保存", notSavedWhy: "上次保存失败:disk full",
+ local: "每次调用的请求/响应头和正文去除密钥后存到本机的 ~/.config/magpie/request-archive,超过 1.1 GB 时先删除最早的",
bucket: "每次调用的请求/响应头和正文去除密钥后存到 bkt/team on https://s3.example.com",
},
};
@@ -137,24 +139,29 @@ for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium",
assert.equal(await sw.getAttribute("role"), "switch");
assert.equal(await sw.innerText(), w.name);
assert.equal(await sw.getAttribute("aria-checked"), "false", "off unless turned on");
- assert((await sw.getAttribute("title")).includes(w.setup));
+ assert.equal(await sw.getAttribute("title"), w.local, "no bucket: kept on this computer");
const [swBox, sumBox, listBox] = await Promise.all([sw, page.locator("#ledSum"), page.locator("#ledWrap")].map((l) => l.boundingBox()));
assert(Math.abs(swBox.x + swBox.width - (listBox.x + listBox.width)) <= 4, `the switch ends at the list's right edge: ${JSON.stringify([swBox, listBox])}`);
assert(swBox.y + swBox.height <= listBox.y, "the switch is over the list");
assert(Math.abs(swBox.y + swBox.height / 2 - (sumBox.y + sumBox.height / 2)) <= 3, "on the line of the requests' sum");
- // no bucket: turning it on is refused, and it stays off
+ // no bucket: on all the same, kept on this computer, and off again
await press(sw, "#view-usage");
await wait(1);
assert.deepEqual(seen[0], ["switch", { on: true }]);
- await page.waitForFunction((s) => document.querySelector("#status").textContent === s, w.refused);
+ const onHere = page.locator('#ledArchive .led-arch-sw[aria-checked="true"]:not([disabled])');
+ await onHere.waitFor();
+ assert.equal(await onHere.getAttribute("title"), w.local);
+ assert.equal(await page.locator("#ledArchive .led-arch-err").count(), 0);
+ await press(onHere, "#view-usage");
+ await wait(2);
+ assert.deepEqual(seen[1], ["switch", { on: false }]);
await page.locator('#ledArchive .led-arch-sw[aria-checked="false"]:not([disabled])').waitFor();
-
// with one: on, the bucket in its tooltip
seen.bucket = true;
await press(page.locator("#ledArchive .led-arch-sw"), "#view-usage");
- await wait(2);
- assert.deepEqual(seen[1], ["switch", { on: true }]);
+ await wait(3);
+ assert.deepEqual(seen[2], ["switch", { on: true }]);
const on = page.locator('#ledArchive .led-arch-sw[aria-checked="true"]');
await on.waitFor();
assert.equal(await on.getAttribute("title"), w.bucket);
@@ -167,29 +174,41 @@ for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium",
// turned off again
await press(on, "#view-usage");
- await wait(3);
- assert.deepEqual(seen[2], ["switch", { on: false }]);
+ await wait(4);
+ assert.deepEqual(seen[3], ["switch", { on: false }]);
await page.locator('#ledArchive .led-arch-sw[aria-checked="false"]:not([disabled])').waitFor();
// an upload that failed is said beside it, why in its tooltip
seen.failing = true;
await press(page.locator("#ledArchive .led-arch-sw"), "#view-usage");
- await wait(4);
+ await wait(5);
const err = page.locator("#ledArchive .led-arch-err");
await err.waitFor();
assert.equal(await err.innerText(), w.failed);
assert.equal(await err.getAttribute("title"), w.failedWhy);
+
+ // and a save on this computer that failed says so in its own words
+ seen.bucket = false;
+ await press(page.locator("#ledArchive .led-arch-sw"), "#view-usage");
+ await wait(6);
+ assert.deepEqual(seen[5], ["switch", { on: false }]);
+ await page.locator('#ledArchive .led-arch-sw[aria-checked="false"]:not([disabled])').waitFor();
+ await press(page.locator("#ledArchive .led-arch-sw"), "#view-usage");
+ await wait(7);
+ assert.deepEqual(seen[6], ["switch", { on: true }]);
+ await page.waitForFunction((s) => document.querySelector("#ledArchive .led-arch-err")?.textContent === s, w.notSaved);
+ assert.equal(await page.locator("#ledArchive .led-arch-err").getAttribute("title"), w.notSavedWhy);
assert.equal(seen.filter(([k]) => k === "fetch").length, 0, "nothing was read from the archive");
const border = await page.evaluate(() => [...document.querySelectorAll(".led-head, .led-head *")].map((e) => getComputedStyle(e).borderLeftWidth).filter((b) => parseFloat(b) > 1));
assert.deepEqual(border, [], "no left-border accent");
const missing = await page.evaluate(() => [
- "Request archive", "Upload failed", "Last upload failed: {e}",
+ "Request archive", "Upload failed", "Last upload failed: {e}", "Not saved", "Last save failed: {e}",
"Keeps each call’s headers and bodies, secrets taken out, in {where}",
- "Keeps each call’s headers and bodies, secrets taken out, in your S3 bucket. Set up Sync and backup in Settings with an s3:// address first",
- "The request archive goes to the S3 bucket sync keeps its backup in: set up Sync and backup in Settings with an s3:// address first",
- ].filter((k) => !I18N.zh[k]));
- assert.deepEqual(missing, [], "every string has its Chinese");
+ "Keeps each call’s headers and bodies, secrets taken out, on this computer in {where}, the oldest removed past {size}",
+ "Not in the archive on this computer: it is saved just after the call, the oldest are removed past the archive's size, or the save failed (magpie's log says why)",
+ ].flatMap((k) => ["zh", "zh-TW", "ja", "de"].filter((l) => !I18N[l][k]).map((l) => l + " " + k)));
+ assert.deepEqual(missing, [], "every string has its Chinese, Japanese and German");
assert.deepEqual(errors, []);
});
}
diff --git a/internal/settings/archive_local_test.go b/internal/settings/archive_local_test.go
new file mode 100644
index 000000000..7910b806f
--- /dev/null
+++ b/internal/settings/archive_local_test.go
@@ -0,0 +1,21 @@
+package settings
+
+import "testing"
+
+// The archive on this computer's disk is this computer's own: sync, a
+// Restore or a backup file never turns it on, or off, at another
+// (CyberCaius on Discord; RequestArchive, the S3 archive's, is synced).
+func TestRequestArchiveLocalIsThisComputers(t *testing.T) {
+ from := Settings{RequestArchive: true, RequestArchiveLocal: true, RequestArchiveLocalMB: 64}
+ from.KeepOwn(Settings{})
+ if from.RequestArchiveLocal || from.RequestArchiveLocalMB != 0 {
+ t.Fatalf("another computer's local archive came in: %+v", from)
+ }
+ if !from.RequestArchive {
+ t.Fatal("the S3 archive's switch is synced")
+ }
+ from.KeepOwn(Settings{RequestArchiveLocal: true, RequestArchiveLocalMB: 8})
+ if !from.RequestArchiveLocal || from.RequestArchiveLocalMB != 8 {
+ t.Fatalf("this computer's own was lost: %+v", from)
+ }
+}
diff --git a/internal/settings/settings.go b/internal/settings/settings.go
index 6ff2bdcba..1ee7c692a 100644
--- a/internal/settings/settings.go
+++ b/internal/settings/settings.go
@@ -158,6 +158,17 @@ type Settings struct {
// RequestArchiveMaxMB is how much of each body the archive keeps, in
// MiB: 0 for 32, at most 1024 (#447)
RequestArchiveMaxMB int `json:"requestArchiveMaxMB,omitempty"`
+ // RequestArchiveLocal keeps the request archive on this computer's disk,
+ // in magpie's folder (gateway/archive_local.go), while sync has no S3
+ // bucket for it (CyberCaius on Discord). Its own switch, separate from
+ // RequestArchive, which sync carries to every computer: this computer's
+ // own (KeepOwn), so a request body is written to a disk only where the
+ // user turned it on.
+ RequestArchiveLocal bool `json:"requestArchiveLocal,omitempty"`
+ // RequestArchiveLocalMB is how large the archive on this computer grows
+ // before its oldest calls are removed, in MiB: 0 for 1024. In
+ // settings.json only; this computer's own (KeepOwn).
+ RequestArchiveLocalMB int `json:"requestArchiveLocalMB,omitempty"`
// GatewayConversations is local, opt-in recording of session traffic.
// Only SetGatewayConversations changes it; other saves keep local consent.
GatewayConversations bool `json:"gatewayConversations,omitempty"`
@@ -1080,6 +1091,7 @@ func (s *Settings) KeepOwn(cur Settings) {
s.SessionTerminal = cur.SessionTerminal
s.RedactModel = cur.RedactModel
s.ClaudeRunsMost = cur.ClaudeRunsMost
+ s.RequestArchiveLocal, s.RequestArchiveLocalMB = cur.RequestArchiveLocal, cur.RequestArchiveLocalMB
}
// RenamePerModel moves what the user said of a provider's models to the id
diff --git a/site/public/docs/ja/privacy.html b/site/public/docs/ja/privacy.html
index f9b6d4d04..01ced4c2d 100644
--- a/site/public/docs/ja/privacy.html
+++ b/site/public/docs/ja/privacy.html
@@ -164,6 +164,7 @@
<tr><td><code>partner-counts.json</code>(キャッシュフォルダー)</td><td>まだ送っていないパートナーの回数。最大 14 日分</td></tr>
</table></div>
<p><code>install-id</code> と <code>stats-sent</code> は magpie のフォルダーにあります。macOS と Linux では <code>~/.config/magpie</code>、Windows では <code>%USERPROFILE%\.config\magpie</code>、<code>$XDG_CONFIG_HOME</code> が設定されていれば <code>$XDG_CONFIG_HOME/magpie</code>、ポータブル版ではそのデータフォルダーです。<code>partner-counts.json</code> は <code>~/.cache/magpie</code>(<code>$XDG_CACHE_HOME/magpie</code>、ポータブル版ではデータフォルダーの <code>cache</code>)にあります。どれも削除できます。<code>install-id</code> を消すと次の送信で新しいランダムな ID が作られ、新しいユーザーとして数えられます。カウントをオフにすれば、どれも再び書かれません。</p>
+ <p>リクエストアーカイブ(<span class="ui">使用量</span>ページのリクエスト一覧の上にある<span class="ui">リクエストアーカイブ</span>スイッチ)は<b>既定でオフ</b>です。同期に S3 バケットがないときにオンにすると、ゲートウェイを通る各呼び出しのヘッダーと本文を、S3 アーカイブと同じようにシークレットを伏せて、magpie のフォルダーの <code>request-archive</code> に 1 呼び出し 1 ファイルで保存します。読めるのはあなたのユーザーだけです。上限は 1 GiB(settings.json の <code>requestArchiveLocalMB</code>)で、超えると古い呼び出しから削除します。このスイッチはこのコンピューターだけのもので、同期で別のコンピューターでオンになることはありません。S3 バケットがあるときはバケットに送られ、ここには何も書かれません。フォルダーはいつでも削除できます。</p>
<h2 id="site"><a class="anchor" href="#site">usemagpie.ai に見えるもの</a></h2>
<p>magpie はプライバシー設定にかかわらず、どのアプリも自分の更新サーバーに尋ねるのと同じように、自分のサイトにいくつか尋ねます。どのリクエストにもインストール ID やアカウントは付きません。</p>
diff --git a/site/public/docs/privacy.html b/site/public/docs/privacy.html
index bbdd4b4d2..715c90a84 100644
--- a/site/public/docs/privacy.html
+++ b/site/public/docs/privacy.html
@@ -164,6 +164,7 @@
<tr><td><code>partner-counts.json</code> (in the cache folder)</td><td>Partner counts not yet sent, at most 14 days</td></tr>
</table></div>
<p><code>install-id</code> and <code>stats-sent</code> are in magpie’s folder: <code>~/.config/magpie</code> on macOS and Linux, <code>%USERPROFILE%\.config\magpie</code> on Windows, <code>$XDG_CONFIG_HOME/magpie</code> when that is set, and the data folder of a portable magpie. <code>partner-counts.json</code> is in <code>~/.cache/magpie</code> (<code>$XDG_CACHE_HOME/magpie</code>, or <code>cache</code> in a portable magpie’s data folder). You can delete any of them: a deleted <code>install-id</code> is replaced by a new random one at the next send, so you are counted as a new user; with the count off, none of them is written again.</p>
+ <p>The request archive (the <span class="ui">Request archive</span> switch on the <span class="ui">Usage</span> page, above the list of requests) is <b>off by default</b>. Turned on while sync has no S3 bucket, it keeps each gateway call’s headers and bodies, secrets masked as for the S3 archive, in <code>request-archive</code> in magpie’s folder, one file per call, readable by your user only. It grows to 1 GiB (<code>requestArchiveLocalMB</code> in settings.json) and the oldest calls are removed past that. The switch is this computer’s own: sync never turns it on at another computer. With an S3 bucket the calls go to the bucket instead, and nothing is written here. Delete the folder whenever you like.</p>
<h2 id="site"><a class="anchor" href="#site">What usemagpie.ai sees</a></h2>
<p>magpie asks its own site for a few things, whatever the privacy settings say, as any app asks its update server. None of these requests carries the install id or an account.</p>
diff --git a/site/public/docs/zh/privacy.html b/site/public/docs/zh/privacy.html
index ec8343391..0972b4b85 100644
--- a/site/public/docs/zh/privacy.html
+++ b/site/public/docs/zh/privacy.html
@@ -164,6 +164,7 @@
<tr><td><code>partner-counts.json</code>(缓存文件夹)</td><td>还没发送的合作伙伴次数,最多保留 14 天</td></tr>
</table></div>
<p><code>install-id</code> 和 <code>stats-sent</code> 在 magpie 的文件夹里:macOS 和 Linux 上是 <code>~/.config/magpie</code>,Windows 上是 <code>%USERPROFILE%\.config\magpie</code>,设置了 <code>$XDG_CONFIG_HOME</code> 时是 <code>$XDG_CONFIG_HOME/magpie</code>,便携版则在它的数据文件夹里。<code>partner-counts.json</code> 在 <code>~/.cache/magpie</code>(或 <code>$XDG_CACHE_HOME/magpie</code>,便携版数据文件夹里的 <code>cache</code>)。这些文件都可以删:删掉 <code>install-id</code> 后,下次发送会生成新的随机 ID,你会被算作新用户;关掉计数后,它们都不会再写入。</p>
+ <p>请求存档(<span class="ui">用量</span>页请求列表上方的<span class="ui">请求存档</span>开关)<b>默认关闭</b>。同步没有 S3 存储桶时打开它,每次经过网关的调用的请求头和正文会存到 magpie 文件夹里的 <code>request-archive</code>,一次调用一个文件,只有你的用户能读,密钥按 S3 存档同样的方式遮掉。它最多占 1 GiB(settings.json 里的 <code>requestArchiveLocalMB</code>),超过后先删最早的调用。这个开关只属于这台电脑,同步不会在别的电脑上把它打开。有 S3 存储桶时调用存到桶里,这里不写任何东西。这个文件夹随时可以删。</p>
<h2 id="site"><a class="anchor" href="#site">usemagpie.ai 能看到什么</a></h2>
<p>magpie 会向自己的网站请求几样东西,不受隐私设置影响,就像任何应用访问自己的更新服务器。这些请求都不带安装 ID 或账号。</p>