Skip to content

Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.

gateway: a Responses request may name the response it follows (previous_response_id, kevinzhow on X)

yetoneba3a7384fb2a
diff --git a/docs/subsystems/gateway-routing.md b/docs/subsystems/gateway-routing.md
index 3b4f01339..c5d19e682 100644
--- a/docs/subsystems/gateway-routing.md
+++ b/docs/subsystems/gateway-routing.md
@@ -1157,6 +1157,46 @@ the row of the entry that id reaches, with its label, under the bare id.
`internal/gui/tests/bare-ids.test.cjs`.
- A URL magpie downloads but didn't choose itself is downloaded from public addresses only, wherever the request came from ([`image_fetch_guard.go`](../../internal/gateway/image_fetch_guard.go)). That is an image a caller names (an `/v1/images/edits` or `/v1/images/generations` reference image, for an images API that takes files, Gemini's own API or Code Assist: `fetchPicture` in [`draw.go`](../../internal/gateway/draw.go), `publicClient`), a page `web_fetch` reads (`webFetchClient`), and an image or video a vendor answers by URL, which what the model was asked to draw or do can steer (the inline download for a `b64_json` caller in `draw.go` and `/v1/videos/{id}/content` in [`video.go`](../../internal/gateway/video.go), `vendorClient`; generate_image's own download in [`internal/imagemcp`](../../internal/imagemcp/imagemcp.go), `PublicClient`). Every address the host resolves to must be public (`publicAddr`, which is [`internal/netguard`](../../internal/netguard/netguard.go)'s `Public`, the rule a provider's icon uses too: not loopback, RFC 1918, fc00::/7, link-local with 169.254.169.254, CGNAT 100.64/10, unspecified, multicast, documentation, reserved, benchmarking, Teredo, nor an IPv4-mapped, NAT64 or 6to4 form of one; an AAAA record Loon synthesizes for an A answer in the same reply, `fd27:712::/96` with that IPv4 address inside, is left out and the A answer judged in its place, `dropLoon`), and the request goes to the address checked, so a name that resolves elsewhere the second time changes nothing; a direct connection is checked again on the address dialled (`guardDialer`). Through a proxy the proxy is sent that address, never the name (an `https://` CONNECT or SOCKS keeps the name for TLS and Host; a plain `http://` URL through an HTTP proxy is asked for by the address in Host too). A fake-IP resolver's answer is the one shape let past that (#2593; ggbdpq, with h4rk8s's and TomXPRIME's reports): Clash's TUN with fake-ip, Clash Verge, Surge's enhanced mode and Loon answer every name in 198.18.0.0/15 (`netguard.FakeIP`), so with one on every download was refused as a private address. A name (never an address literal) whose every answer is in that range is fetched: with no proxy (pure TUN) its fake address is pinned and dialled, the dial guard letting that one address in, and the TUN carries it; through a proxy on this computer (a loopback address or `localhost`, `onThisComputer`) the proxy is given the name, never the fake address, which a real proxy can't route (h4rk8s: a CONNECT by the name went through where one by its 198.18 address failed), and a plain `http://` request's Host is the checked name, never the caller's own; through a proxy elsewhere it is refused. An answer with any other address beside the fake ones (private or public) is judged as any, so a fake one beside 192.168.x is refused. The trust this asks for is the fake-IP resolver's: which host the name stands for, and so where the TUN or the proxy connects, is decided where magpie can't see, and a name that really points at the user's network is reached if that resolver's rules send it there. A provider's icon was already fetched this way (`publicIP` in [`internal/provider/icon.go`](../../internal/provider/icon.go), #252), and shares the rule. `TestFakeIPNameGoesToALocalProxyByName`, `TestFakeIPNameDialledThroughTheTUN`, `TestFakeIPRefusesTheRest` in [`fakeip_test.go`](../../internal/gateway/fakeip_test.go). Each redirect, up to 5, is checked the same, and a caller's image must sniff as an image. A request from this computer gets nothing more: a reverse proxy or relay in front of magpie on the same host makes every request it carries arrive from loopback, so where a request came from is never a reason to reach a private address. Two things are: settings.json's `fetchPrivate` (`settings.FetchPrivate`, none by default, no Settings page, this computer's own and never brought by a sync or restore), a list of names, `name:port`s, IPs and CIDR ranges the user lets magpie download from (`allowFor`; a settings.json that can't be read lets nothing extra in), and for a vendor's URL the `host:port` of that provider's own endpoints, which the user named themselves (a provider on the LAN that serves its images from itself). A vendor's image that can't be downloaded is answered by its URL, never dropped; its video is a 502 naming why. A URL left for the vendor to fetch (a chat model's message) is the vendor's to fetch.
+## A Responses request that names the response it follows
+
+A Responses client may send only its new input with `previous_response_id`
+(kevinzhow on X: every turn sent the whole conversation again, images
+included). `previousResponse` in
+[`prev_response.go`](../../internal/gateway/prev_response.go) runs in
+`serveAgent` before anything else reads the body.
+
+- Every Responses reply magpie passes with status 200, not failed or
+ cancelled, is kept by its id with the conversation it answered (input,
+ then output) in `prevResponses`: in memory only, least recently used
+ first out past 256 MB, each kept 6 hours (`prevTTL`, `prevCap`). A
+ request with `store: false` keeps nothing. Codex over HTTP sends
+ `store: false` with the whole input and never names a previous response
+ (only its websocket request has the field), so nothing changes for it.
+- A turn is kept for the caller that made it (`prevScope`): its gateway
+ key, the remote magpie it came through, and the address of a keyless
+ remote caller. Another caller naming the id is told it isn't there.
+- A request naming a kept id is given the conversation whole
+ (`previous_response_id` taken out, the kept items before its own input),
+ so every route, translation and masking below sees what it would have
+ seen had the client sent it all. The stored items are what the client
+ sent, and masking runs on each request as it goes, so a stored secret
+ is masked the same way on every later turn.
+- An id magpie doesn't keep (never seen, another caller's, expired, or
+ lost to a restart) gets OpenAI's 400 `previous_response_not_found`
+ (`writePrevNotFound`) on a route that translates, before anything is
+ sent. A Responses relay is sent the request as it came, and its own
+ answer reaches the client.
+- Where the provider keeps responses itself (`prevKeeps`: an API key, no
+ plugin, on api.openai.com, Azure, or Ark's `/api/v3` pay-as-you-go
+ Responses) and it answered the previous turn on the same key
+ (`prevRoute`), with `store` not off, the request goes as the client's
+ new input with the provider's own id (`prevDelta`), so images aren't
+ uploaded again. A provider that says it doesn't know that id
+ (`prevRefused`) is sent the whole conversation at once.
+- Not covered: the kept turns don't outlive the gateway, and a kept-id
+ request is still read and masked whole on this computer before the
+ delta is cut.
+
## Verification
```sh
@@ -1164,6 +1204,7 @@ go test -tags nogui ./internal/gateway -run 'TestGroupAsksOneRestingMemberNotEac
go test -tags nogui ./internal/gateway -run 'TestMemberTooSmallIsNotAsked|TestMemberTooSmallIsTheLastResort|TestClaudeAccountsSpentFallBackToCodex|TestConversationOutgrowingItsMemberMoves|TestRoomyLeavesWhatItCantJudge|TestPayloadTooLarge|TestOverflow|TestContextOverflow|TestRelayInputCap|TestEveryVendorsOverflow'
go test -tags nogui ./internal/gateway -run 'TestPlain403|TestOrderedKeepsAnAccountAt98|TestPromptRefusalRestsNobody'
go test -tags nogui ./internal/gateway -run 'TestVertex'
+go test -tags nogui ./internal/gateway -run 'TestPreviousResponse|TestUnknownPreviousResponse|TestPrevStore'
go test -tags nogui ./internal/gateway -run 'TestBedrock'
go test -tags nogui ./internal/gateway -run 'TestPublicAddr|TestEditImageURL|TestPrivateDialer|TestFetchPrivateEntries|TestVendorImageURLAtPrivateAddress|TestVideoContentAtPrivateAddress'
go test -tags nogui ./internal/gateway ./internal/provider ./internal/cli ./internal/gui -run 'TestDisabled|TestModelDisableCmd'
diff --git a/internal/gateway/gateway.go b/internal/gateway/gateway.go
index 5fea4c7ac..410ad2292 100644
--- a/internal/gateway/gateway.go
+++ b/internal/gateway/gateway.go
@@ -4050,6 +4050,13 @@ func (s *Server) attempt(w http.ResponseWriter, r *http.Request, from provider.P
// translation and a plugin's fetch alike
body = chatCallIDs(body)
}
+ // a response the request named that magpie doesn't hold goes only to
+ // a provider it is relayed to as written (prev_response.go): Claude's
+ // and the ChatGPT backend's accounts drop it, as a translation does
+ noteAnswered(r, "")
+ if id := prevLost(r); id != "" && p.Account != nil && (p.Account.Agent == "claude" || p.Account.Agent == "codex") {
+ return writePrevNotFound(w, id)
+ }
// every request to the provider goes through its own proxy, if it has
// one (#237)
r = r.WithContext(p.Via(r.Context()))
@@ -4190,6 +4197,9 @@ func (s *Server) attempt(w http.ResponseWriter, r *http.Request, from provider.P
if ownAPI && slices.Contains(to, from) && !firstElsewhere(p, model, from) {
to = []provider.Protocol{from}
}
+ if id := prevLost(r); id != "" {
+ return writePrevNotFound(w, id)
+ }
call.To = to[0]
return s.translate(w, r, p, from, to[0], model, body, &call.Usage)
}
@@ -4865,10 +4875,31 @@ func (s *Server) passthrough(w http.ResponseWriter, r *http.Request, p provider.
if proto == provider.Responses && p.Account != nil && (p.Account.Agent == "grok" || p.PluginProvider() == "grok" || p.PluginProvider() == "zed") {
named = namespacedIn(body)
}
+ // the new input alone, after the id the provider keeps the previous
+ // response under, when it answered that one (prev_response.go)
+ sentWhole, delta := body, false
+ if proto == provider.Responses {
+ body, delta = prevDelta(r, p, body)
+ }
res, err := s.forward(r.Context(), p, proto, path, p.Prepare(body), r.Header)
if err != nil {
return writeError(w, proto, 502, p.Name+": "+err.Error()), err.Error(), true
}
+ if delta && badRequest(res.StatusCode) || delta && res.StatusCode == http.StatusNotFound {
+ // one it no longer holds: the conversation whole
+ b, _ := io.ReadAll(io.LimitReader(res.Body, 1<<20))
+ res.Body.Close()
+ res.Body = io.NopCloser(bytes.NewReader(b))
+ if prevRefused(res.StatusCode, b) {
+ body = sentWhole
+ if res, err = s.forward(r.Context(), p, proto, path, p.Prepare(body), r.Header); err != nil {
+ return writeError(w, proto, 502, p.Name+": "+err.Error()), err.Error(), true
+ }
+ }
+ }
+ if proto == provider.Responses && prevKeptBy(p) && !storeOff(body) {
+ noteAnswered(r, prevRoute(p))
+ }
// structured output turned away (Xiaomi MiMo's Token Plan takes no
// json_schema, only text and json_object, #2126): a schema is asked
// for as any JSON object with the schema in the system prompt, and if
diff --git a/internal/gateway/image_body_bench_test.go b/internal/gateway/image_body_bench_test.go
new file mode 100644
index 000000000..b4699dea4
--- /dev/null
+++ b/internal/gateway/image_body_bench_test.go
@@ -0,0 +1,194 @@
+package gateway
+
+import (
+ "bytes"
+ "context"
+ "encoding/base64"
+ "encoding/json"
+ "fmt"
+ "image"
+ "image/color"
+ "image/png"
+ "io"
+ "math/rand/v2"
+ "net/http"
+ "net/http/httptest"
+ "sync/atomic"
+ "testing"
+
+ "github.com/yetone/magpie/internal/provider"
+)
+
+// noisePNG is a real PNG of w×h random pixels: it doesn't compress, so its
+// base64 is as large as a dense screenshot's of the same size.
+func noisePNG(w, h int, seed uint64) []byte {
+ r := rand.New(rand.NewPCG(seed, seed))
+ img := image.NewNRGBA(image.Rect(0, 0, w, h))
+ for i := 0; i < len(img.Pix); i += 4 {
+ v := r.Uint32()
+ img.Pix[i], img.Pix[i+1], img.Pix[i+2], img.Pix[i+3] = uint8(v), uint8(v>>8), uint8(v>>16), 255
+ }
+ _ = color.White
+ var b bytes.Buffer
+ png.Encode(&b, img)
+ return b.Bytes()
+}
+
+// imageConversation is a conversation of turns turns, each with one image
+// of about 2.6MB base64, as its client sends it.
+func imageConversation(proto provider.Protocol, model string, turns int) []byte {
+ type m = map[string]any
+ var msgs []any
+ for i := range turns {
+ url := "data:image/png;base64," + base64.StdEncoding.EncodeToString(noisePNG(900, 700, uint64(i+1)))
+ text := "what does this screenshot show?"
+ switch proto {
+ case provider.Anthropic:
+ data := url[len("data:image/png;base64,"):]
+ msgs = append(msgs, m{"role": "user", "content": []any{m{"type": "image", "source": m{"type": "base64", "media_type": "image/png", "data": data}}, m{"type": "text", "text": text}}},
+ m{"role": "assistant", "content": []any{m{"type": "text", "text": "It shows noise."}}})
+ case provider.Chat:
+ msgs = append(msgs, m{"role": "user", "content": []any{m{"type": "image_url", "image_url": m{"url": url}}, m{"type": "text", "text": text}}},
+ m{"role": "assistant", "content": "It shows noise."})
+ default:
+ msgs = append(msgs, m{"type": "message", "role": "user", "content": []any{m{"type": "input_image", "image_url": url}, m{"type": "input_text", "text": text}}},
+ m{"type": "message", "role": "assistant", "content": []any{m{"type": "output_text", "text": "It shows noise."}}})
+ }
+ }
+ body := m{"model": model, "stream": false}
+ switch proto {
+ case provider.Anthropic:
+ msgs = append(msgs, m{"role": "user", "content": "go on"})
+ body["max_tokens"] = 1000
+ body["messages"] = msgs
+ case provider.Chat:
+ msgs = append(msgs, m{"role": "user", "content": "go on"})
+ body["messages"] = msgs
+ default:
+ msgs = append(msgs, m{"type": "message", "role": "user", "content": []any{m{"type": "input_text", "text": "go on"}}})
+ body["input"] = msgs
+ }
+ b, _ := json.Marshal(body)
+ return b
+}
+
+func BenchmarkImageRequest(b *testing.B) {
+ chatReply := `{"id":"c1","object":"chat.completion","choices":[{"index":0,"message":{"role":"assistant","content":"ok"},"finish_reason":"stop"}],"usage":{"prompt_tokens":7,"completion_tokens":1}}`
+ chatSSE := "data: {\"id\":\"c1\",\"object\":\"chat.completion.chunk\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"ok\"},\"finish_reason\":\"stop\"}],\"usage\":{\"prompt_tokens\":7,\"completion_tokens\":1}}\n\ndata: [DONE]\n\n"
+ for _, tc := range []struct {
+ name, path string
+ in, up provider.Protocol
+ reply string
+ sse bool
+ }{
+ {"responses-to-chat", "/v1/responses", provider.Responses, provider.Chat, chatSSE, true},
+ {"chat-to-chat", "/v1/chat/completions", provider.Chat, provider.Chat, chatReply, false},
+ {"messages-to-chat", "/v1/messages", provider.Anthropic, provider.Chat, chatSSE, true},
+ {"responses-to-responses", "/v1/responses", provider.Responses, provider.Responses, `{"id":"r1","object":"response","status":"completed","output":[{"type":"message","role":"assistant","content":[{"type":"output_text","text":"ok"}]}],"usage":{"input_tokens":7,"output_tokens":1}}`, false},
+ } {
+ b.Run(tc.name, func(b *testing.B) {
+ b.Setenv("XDG_CONFIG_HOME", b.TempDir())
+ b.Setenv("XDG_CACHE_HOME", b.TempDir())
+ f := &fake{ctype: "application/json", reply: tc.reply}
+ if tc.sse {
+ f.ctype = ""
+ }
+ up := httptest.NewServer(f)
+ defer up.Close()
+ p := provider.Provider{ID: "fake", Name: "Fake", Key: "k", Models: []string{"m1"}}
+ switch tc.up {
+ case provider.Chat:
+ p.Chat = up.URL + "/v1"
+ case provider.Responses:
+ p.Responses = up.URL + "/v1"
+ }
+ if err := provider.Save(p); err != nil {
+ b.Fatal(err)
+ }
+ body := imageConversation(tc.in, "fake/m1", 8)
+ s := New()
+ b.SetBytes(int64(len(body)))
+ b.ResetTimer()
+ for range b.N {
+ ctx, done := context.WithCancel(context.Background())
+ rec := httptest.NewRecorder()
+ s.Handler().ServeHTTP(rec, httptest.NewRequest("POST", tc.path, bytes.NewReader(body)).WithContext(ctx))
+ done()
+ if rec.Code != 200 || f.calls == 0 {
+ b.Fatalf("%d %s", rec.Code, rec.Body.String())
+ }
+ }
+ b.ReportMetric(float64(len(body))/1e6, "MB/req")
+ })
+ }
+}
+
+// BenchmarkPreviousResponseTurn is a turn after an 8-image conversation
+// that a client sends as previous_response_id and its new text alone
+// (kevinzhow on X), against the whole conversation sent again (whole):
+// to a provider that keeps responses, which is sent the new input after
+// its own id (kept), and to one on Chat, which magpie gives the whole
+// conversation (chat). up-MB/req is what went upstream.
+func BenchmarkPreviousResponseTurn(b *testing.B) {
+ for _, tc := range []struct {
+ name string
+ chat, whole bool
+ }{{"whole-to-responses", false, true}, {"prev-to-kept-responses", false, false}, {"prev-to-chat", true, false}} {
+ b.Run(tc.name, func(b *testing.B) {
+ b.Setenv("XDG_CONFIG_HOME", b.TempDir())
+ b.Setenv("XDG_CACHE_HOME", b.TempDir())
+ defer func(old func(provider.Provider) bool) { prevKeptBy = old }(prevKeptBy)
+ prevKeptBy = func(p provider.Provider) bool { return p.ID == "fake" }
+ var n, upBytes atomic.Int64
+ up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ got, _ := io.ReadAll(r.Body)
+ upBytes.Add(int64(len(got)))
+ i := n.Add(1)
+ if tc.chat {
+ w.Header().Set("Content-Type", "text/event-stream")
+ io.WriteString(w, "data: {\"id\":\"c1\",\"object\":\"chat.completion.chunk\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"ok\"},\"finish_reason\":\"stop\"}]}\n\ndata: [DONE]\n\n")
+ return
+ }
+ w.Header().Set("Content-Type", "application/json")
+ fmt.Fprintf(w, `{"id":"resp_%d","object":"response","status":"completed","output":[{"type":"message","role":"assistant","content":[{"type":"output_text","text":"ok"}]}],"usage":{"input_tokens":7,"output_tokens":1}}`, i)
+ }))
+ defer up.Close()
+ p := provider.Provider{ID: "fake", Name: "Fake", Key: "k", Models: []string{"m1"}}
+ if tc.chat {
+ p.Chat = up.URL + "/v1"
+ } else {
+ p.Responses = up.URL + "/v1"
+ }
+ if err := provider.Save(p); err != nil {
+ b.Fatal(err)
+ }
+ s := New()
+ ask := func(body []byte) *httptest.ResponseRecorder {
+ rec := httptest.NewRecorder()
+ req := httptest.NewRequest("POST", "/v1/responses", bytes.NewReader(body))
+ req.RemoteAddr = "127.0.0.1:5555"
+ s.Handler().ServeHTTP(rec, req)
+ if rec.Code != 200 {
+ b.Fatalf("%d %.300s", rec.Code, rec.Body.String())
+ }
+ return rec
+ }
+ whole := imageConversation(provider.Responses, "fake/m1", 8)
+ var seed struct{ ID string }
+ json.Unmarshal(ask(whole).Body.Bytes(), &seed)
+ turn := []byte(`{"model":"fake/m1","stream":false,"previous_response_id":"` + seed.ID + `","input":"go on"}`)
+ if tc.whole {
+ turn = whole
+ }
+ upBytes.Store(0)
+ b.SetBytes(int64(len(turn)))
+ b.ResetTimer()
+ for range b.N {
+ ask(turn)
+ }
+ b.StopTimer()
+ b.ReportMetric(float64(len(turn))/1e6, "client-MB/req")
+ b.ReportMetric(float64(upBytes.Load())/float64(b.N)/1e6, "up-MB/req")
+ })
+ }
+}
diff --git a/internal/gateway/middleware.go b/internal/gateway/middleware.go
index c46ac21f6..dc99ad997 100644
--- a/internal/gateway/middleware.go
+++ b/internal/gateway/middleware.go
@@ -21,6 +21,10 @@ import (
// magpie's to read.
func (s *Server) serveAgent(w http.ResponseWriter, r *http.Request, from provider.Protocol, body []byte) {
withGatewaySession(w, r)
+ // a Responses request naming the response it follows is given the
+ // conversation whole before anything reads it (prev_response.go)
+ w, r, body, kept := previousResponse(w, r, from, body)
+ defer kept()
w, recorded := recordConversation(w, r, from, body)
defer recorded()
// a conversation going round one tool call is answered here, before
diff --git a/internal/gateway/prev_response.go b/internal/gateway/prev_response.go
new file mode 100644
index 000000000..4a6cc655a
--- /dev/null
+++ b/internal/gateway/prev_response.go
@@ -0,0 +1,431 @@
+package gateway
+
+import (
+ "bytes"
+ "container/list"
+ "context"
+ "crypto/sha256"
+ "encoding/hex"
+ "encoding/json"
+ "net"
+ "net/http"
+ "strings"
+ "sync"
+ "time"
+
+ "github.com/tidwall/gjson"
+ "github.com/yetone/magpie/internal/access"
+ "github.com/yetone/magpie/internal/provider"
+)
+
+// A Responses client may send only a turn's new input and name the
+// response it follows (previous_response_id), as OpenAI's own API lets it
+// (kevinzhow on X: every request carried the whole conversation, images
+// and all). magpie keeps each response it gave such a client, its input
+// and its output, under the response's id, and a request that names one
+// is given the conversation whole before anything else reads it: a
+// translation, a hook, the masking. A provider that keeps responses itself
+// (prevKeeps), and answered the one named, is sent the new input alone
+// after its own id, so what it holds isn't sent again.
+//
+// What is kept is the client's own form, before masking: the conversation
+// it is given back into goes through the masking as every request does.
+// It is kept in memory alone, bounded (prevCap) and for prevTTL, under the
+// caller's gateway key (prevScope): one key never reads another's.
+// A client that says store false (Codex over HTTP, whose requests never
+// name a previous response: codex-rs/codex-api/src/common.rs keeps
+// previous_response_id to its websocket's response.create) has nothing
+// kept.
+
+const (
+ prevTTL = 6 * time.Hour
+ // prevCap bounds the bytes kept, counted as each turn adds them: a
+ // turn's history is the earlier turn's items, shared, not copied
+ prevCap = 256 << 20
+ // prevReplyMax is the most of one reply read for its output
+ prevReplyMax = 64 << 20
+)
+
+type prevKey struct{ scope, id string }
+
+// prevTurn is one response kept: the conversation up to and with it.
+type prevTurn struct {
+ key prevKey
+ items []json.RawMessage // input, then the response's output
+ own int // bytes this turn added to what it follows
+ at time.Time
+ // the provider that keeps it too, if one does: the id it gave it
+ // and which provider and key (prevRoute)
+ upstreamID, route string
+}
+
+type prevStore struct {
+ mu sync.Mutex
+ m map[prevKey]*list.Element
+ order list.List // front the newest
+ size int
+ cap int
+ ttl time.Duration
+ now func() time.Time
+}
+
+var prevResponses = newPrevStore(prevCap, prevTTL)
+
+func newPrevStore(cap int, ttl time.Duration) *prevStore {
+ return &prevStore{m: map[prevKey]*list.Element{}, cap: cap, ttl: ttl, now: time.Now}
+}
+
+func (s *prevStore) get(k prevKey) (*prevTurn, bool) {
+ s.mu.Lock()
+ defer s.mu.Unlock()
+ e, ok := s.m[k]
+ if !ok {
+ return nil, false
+ }
+ t := e.Value.(*prevTurn)
+ if s.now().Sub(t.at) > s.ttl {
+ s.drop(e)
+ return nil, false
+ }
+ s.order.MoveToFront(e)
+ return t, true
+}
+
+func (s *prevStore) put(t *prevTurn) {
+ if t.own > s.cap/2 {
+ return // one turn would push out nearly everything
+ }
+ s.mu.Lock()
+ defer s.mu.Unlock()
+ if e, ok := s.m[t.key]; ok {
+ s.drop(e)
+ }
+ t.at = s.now()
+ s.m[t.key] = s.order.PushFront(t)
+ s.size += t.own
+ for s.size > s.cap {
+ s.drop(s.order.Back())
+ }
+ // and what has outlived its time, from the oldest
+ for e := s.order.Back(); e != nil && s.now().Sub(e.Value.(*prevTurn).at) > s.ttl; e = s.order.Back() {
+ s.drop(e)
+ }
+}
+
+func (s *prevStore) drop(e *list.Element) {
+ t := e.Value.(*prevTurn)
+ s.order.Remove(e)
+ delete(s.m, t.key)
+ s.size -= t.own
+}
+
+// prevScope is whose responses a request may name: its gateway key, the
+// computer a remote magpie passed it on from, and for a request with no
+// key from another computer, that computer's address.
+func prevScope(r *http.Request) string {
+ keyID := access.Caller(r.Context()).KeyID
+ s := "key:" + keyID
+ if via := callerOf(r).via; via != "" {
+ s += "\x00via:" + via
+ }
+ if keyID == "" && !local(r) {
+ host, _, err := net.SplitHostPort(r.RemoteAddr)
+ if err != nil {
+ host = r.RemoteAddr
+ }
+ s += "\x00ip:" + host
+ }
+ return s
+}
+
+// prevRoute is the provider and API key a response came from: one the
+// provider keeps is named to it again only with the same key.
+func prevRoute(p provider.Provider) string {
+ sum := sha256.Sum256([]byte(p.Key))
+ return p.ID + "\x00" + hex.EncodeToString(sum[:8])
+}
+
+// prevKeeps is whether p keeps the responses it gives and takes
+// previous_response_id: OpenAI's API and Azure OpenAI, and Ark's own
+// /api/v3 (tried with a real key on 10-29: a stored response's word came
+// back after its id, and an unknown id is a 400 PreviousResponseNotFound).
+// A subscription's or a plugin's account is never asked so.
+var prevKeptBy = prevKeeps
+
+func prevKeeps(p provider.Provider) bool {
+ if p.Account != nil || p.PluginProvider() != "" {
+ return false
+ }
+ host := p.Host()
+ if host == "api.openai.com" || p.IsAzure() {
+ return true
+ }
+ return host == "ark.cn-beijing.volces.com" && strings.Contains(p.Base(provider.Responses), "/api/v3")
+}
+
+// prevAsk is what serveAgent made of a request's previous_response_id,
+// for the tries of it (attempt, passthrough).
+type prevAsk struct {
+ id string // as the client named it
+ known bool // the conversation was given back whole
+ // the items of the whole conversation, and how many at its end are
+ // the client's own new input
+ total, fresh int
+ upstreamID string // the id the provider that answered it keeps it under
+ route string
+ // which try answered: its route if it was relayed to a provider
+ // that keeps what it is sent, empty otherwise
+ mu sync.Mutex
+ answered string
+}
+
+type prevAskKey struct{}
+
+func prevAskOf(ctx context.Context) *prevAsk {
+ a, _ := ctx.Value(prevAskKey{}).(*prevAsk)
+ return a
+}
+
+// prevLost is the id a request named that magpie doesn't hold, which a
+// try that doesn't relay it to a provider as written can't honour.
+func prevLost(r *http.Request) string {
+ if a := prevAskOf(r.Context()); a != nil && a.id != "" && !a.known {
+ return a.id
+ }
+ return ""
+}
+
+// writePrevNotFound is OpenAI's own answer to an id it doesn't hold.
+func writePrevNotFound(w http.ResponseWriter, id string) (int, string) {
+ msg := "Previous response with id '" + id + "' not found."
+ return writeErrorSaid(w, provider.Responses, 400, msg, "previous_response_not_found", "previous_response_id"), msg
+}
+
+// noteAnswered is called by each try: the last is the one that answered.
+func noteAnswered(r *http.Request, route string) {
+ if a := prevAskOf(r.Context()); a != nil {
+ a.mu.Lock()
+ a.answered = route
+ a.mu.Unlock()
+ }
+}
+
+// previousResponse gives a Responses request that names a response magpie
+// holds the conversation whole, and keeps the reply for the next turn.
+// Its done is deferred by the caller.
+func previousResponse(w http.ResponseWriter, r *http.Request, from provider.Protocol, body []byte) (http.ResponseWriter, *http.Request, []byte, func()) {
+ nothing := func() {}
+ if from != provider.Responses {
+ return w, r, body, nothing
+ }
+ var q map[string]json.RawMessage
+ if decodeJSONView(body, &q) != nil {
+ return w, r, body, nothing
+ }
+ scope := prevScope(r)
+ ask := &prevAsk{}
+ var prior []json.RawMessage
+ if raw := q["previous_response_id"]; len(raw) > 0 && !bytes.Equal(raw, []byte("null")) {
+ var id string
+ if json.Unmarshal(raw, &id) == nil && id != "" {
+ ask.id = id
+ if t, ok := prevResponses.get(prevKey{scope, id}); ok {
+ prior = t.items
+ ask.known, ask.upstreamID, ask.route = true, t.upstreamID, t.route
+ }
+ }
+ }
+ input, fresh := prevInput(q["input"])
+ if fresh == nil {
+ // an input magpie can't read as items: left as it came
+ return w, r, body, nothing
+ }
+ if ask.known {
+ input = append(prior[:len(prior):len(prior)], fresh...)
+ if b, err := json.Marshal(input); err == nil {
+ q["input"] = b
+ delete(q, "previous_response_id")
+ if nb, err := json.Marshal(q); err == nil {
+ body = nb
+ }
+ }
+ ask.total, ask.fresh = len(input), len(fresh)
+ }
+ if ask.id != "" {
+ r = r.WithContext(context.WithValue(r.Context(), prevAskKey{}, ask))
+ }
+ var store bool
+ if json.Unmarshal(q["store"], &store) == nil && !store {
+ return w, r, body, nothing // the client asked that nothing be kept
+ }
+ // what this turn adds: its whole body, which its items hold
+ own := len(body)
+ if ask.known {
+ own = 0
+ for _, it := range fresh {
+ own += len(it)
+ }
+ }
+ if ask.id == "" {
+ // the answer note is wanted for a request naming none too
+ r = r.WithContext(context.WithValue(r.Context(), prevAskKey{}, ask))
+ }
+ sink := &conversationWriter{captureResponseWriter: &captureResponseWriter{
+ ResponseWriter: w, full: &spool{limit: prevReplyMax, pattern: "magpie-prevresp-*"},
+ }}
+ return sink, r, body, func() {
+ defer sink.full.discard()
+ if sink.status != 0 && sink.status != http.StatusOK || sink.full.cut() {
+ return
+ }
+ reply, err := sink.full.read()
+ if err != nil || len(reply) == 0 {
+ return
+ }
+ id, output, ok := prevReply(sink.Header().Get("Content-Type"), reply)
+ if !ok {
+ return
+ }
+ for _, it := range output {
+ own += len(it)
+ }
+ t := &prevTurn{key: prevKey{scope, id}, items: append(input[:len(input):len(input)], output...), own: own}
+ ask.mu.Lock()
+ if ask.answered != "" {
+ t.upstreamID, t.route = id, ask.answered
+ }
+ ask.mu.Unlock()
+ prevResponses.put(t)
+ }
+}
+
+// prevInput is a Responses input as items: a string is one user message.
+// fresh is nil for an input that is neither. The items share the body's
+// bytes, which nothing writes once read: a turn kept holds its body, not
+// a copy of it.
+func prevInput(raw json.RawMessage) (all, fresh []json.RawMessage) {
+ raw = bytes.TrimSpace(raw)
+ if len(raw) == 0 {
+ return nil, nil
+ }
+ if raw[0] == '"' {
+ var s string
+ if json.Unmarshal(raw, &s) != nil {
+ return nil, nil
+ }
+ m, _ := json.Marshal(map[string]any{"type": "message", "role": "user", "content": s})
+ return []json.RawMessage{m}, []json.RawMessage{m}
+ }
+ items, ok := inputList(raw)
+ if !ok || raw[0] != '[' {
+ return nil, nil
+ }
+ if items == nil {
+ items = []json.RawMessage{}
+ }
+ return items, items
+}
+
+// prevReply is a Responses reply's id and output items, a stream's or a
+// whole body's; ok false for one that didn't complete.
+func prevReply(ctype string, reply []byte) (id string, output []json.RawMessage, ok bool) {
+ var resp json.RawMessage
+ if strings.Contains(ctype, "event-stream") || bytes.HasPrefix(reply, []byte("event:")) || bytes.HasPrefix(reply, []byte("data:")) {
+ items := map[int]json.RawMessage{}
+ readSSE(bytes.NewReader(reply), func(event, data string) error {
+ var e struct {
+ Type string `json:"type"`
+ Response json.RawMessage `json:"response"`
+ Item json.RawMessage `json:"item"`
+ OutputIndex int `json:"output_index"`
+ }
+ if json.Unmarshal([]byte(data), &e) != nil {
+ return nil
+ }
+ if e.Type == "" {
+ e.Type = event
+ }
+ switch e.Type {
+ case "response.output_item.done":
+ if len(e.Item) > 0 {
+ items[e.OutputIndex] = e.Item
+ }
+ case "response.completed", "response.incomplete":
+ resp = e.Response
+ }
+ return nil
+ })
+ if len(resp) == 0 {
+ return "", nil, false
+ }
+ resp = withStreamedOutput(resp, items)
+ } else {
+ resp = reply
+ }
+ var r struct {
+ ID string `json:"id"`
+ Object string `json:"object"`
+ Status string `json:"status"`
+ Output []json.RawMessage `json:"output"`
+ }
+ if json.Unmarshal(resp, &r) != nil || r.ID == "" || r.Status == "failed" || r.Status == "cancelled" {
+ return "", nil, false
+ }
+ return r.ID, r.Output, true
+}
+
+// prevDelta is body, a Responses request magpie gave its conversation
+// whole, with the new input alone after the id p keeps the previous
+// response under, when p is the provider and key that answered it and
+// keeps what it is sent. The tries' changes to the input are kept: the
+// new items are the body's own last ones, and only when the body still
+// has as many items as magpie gave it.
+func prevDelta(r *http.Request, p provider.Provider, body []byte) ([]byte, bool) {
+ a := prevAskOf(r.Context())
+ if a == nil || !a.known || a.upstreamID == "" || a.route != prevRoute(p) || !prevKeptBy(p) {
+ return body, false
+ }
+ var q map[string]json.RawMessage
+ if decodeJSONView(body, &q) != nil {
+ return body, false
+ }
+ var store bool
+ if json.Unmarshal(q["store"], &store) == nil && !store {
+ return body, false
+ }
+ items, ok := inputList(q["input"])
+ if !ok || len(items) != a.total || a.fresh > len(items) {
+ return body, false
+ }
+ in, err := json.Marshal(items[len(items)-a.fresh:])
+ if err != nil {
+ return body, false
+ }
+ id, _ := json.Marshal(a.upstreamID)
+ out := make(map[string]json.RawMessage, len(q)+1)
+ for k, v := range q {
+ out[k] = v
+ }
+ out["input"], out["previous_response_id"] = in, id
+ nb, err := json.Marshal(out)
+ if err != nil {
+ return body, false
+ }
+ return nb, true
+}
+
+// prevRefused is an upstream's answer that it doesn't hold the previous
+// response, or takes no such field: the conversation is sent whole.
+func prevRefused(status int, b []byte) bool {
+ if !badRequest(status) && status != http.StatusNotFound {
+ return false
+ }
+ s := strings.ToLower(string(b))
+ return strings.Contains(s, "previous_response") || strings.Contains(s, "previousresponse")
+}
+
+// storeOff is whether a Responses body says store false.
+func storeOff(body []byte) bool {
+ v := gjson.GetBytes(body, "store")
+ return v.Exists() && v.Type == gjson.False
+}
diff --git a/internal/gateway/prev_response_test.go b/internal/gateway/prev_response_test.go
new file mode 100644
index 000000000..755ad0253
--- /dev/null
+++ b/internal/gateway/prev_response_test.go
@@ -0,0 +1,263 @@
+package gateway
+
+import (
+ "context"
+ "encoding/json"
+ "fmt"
+ "io"
+ "net/http"
+ "net/http/httptest"
+ "strings"
+ "sync"
+ "sync/atomic"
+ "testing"
+ "time"
+
+ "github.com/yetone/magpie/internal/access"
+ "github.com/yetone/magpie/internal/provider"
+ "github.com/yetone/magpie/internal/settings"
+)
+
+// prevEnv is a sandbox with one provider, fresh responses kept, and an
+// upstream that records every request and answers with answer.
+type prevUp struct {
+ mu sync.Mutex
+ got [][]byte
+ srv *httptest.Server
+ next func(n int, body []byte) (int, string, string) // status, content type, body
+}
+
+func (u *prevUp) last() map[string]any {
+ u.mu.Lock()
+ defer u.mu.Unlock()
+ var q map[string]any
+ json.Unmarshal(u.got[len(u.got)-1], &q)
+ return q
+}
+
+func (u *prevUp) calls() int {
+ u.mu.Lock()
+ defer u.mu.Unlock()
+ return len(u.got)
+}
+
+func newPrevUp(t *testing.T, next func(n int, body []byte) (int, string, string)) *prevUp {
+ u := &prevUp{next: next}
+ u.srv = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ b, _ := io.ReadAll(r.Body)
+ u.mu.Lock()
+ u.got = append(u.got, b)
+ n := len(u.got)
+ u.mu.Unlock()
+ code, ct, reply := u.next(n, b)
+ w.Header().Set("Content-Type", ct)
+ w.WriteHeader(code)
+ io.WriteString(w, reply)
+ }))
+ t.Cleanup(u.srv.Close)
+ return u
+}
+
+func prevSandbox(t *testing.T, p provider.Provider) *Server {
+ t.Helper()
+ t.Setenv("XDG_CONFIG_HOME", t.TempDir())
+ t.Setenv("XDG_CACHE_HOME", t.TempDir())
+ old := prevResponses
+ prevResponses = newPrevStore(prevCap, prevTTL)
+ t.Cleanup(func() { prevResponses = old })
+ if err := provider.Save(p); err != nil {
+ t.Fatal(err)
+ }
+ return New()
+}
+
+// chatSSEReply is a Chat stream that says text.
+func chatSSEReply(text string) string {
+ c, _ := json.Marshal(text)
+ return "data: {\"id\":\"c1\",\"object\":\"chat.completion.chunk\",\"choices\":[{\"index\":0,\"delta\":{\"content\":" + string(c) + "},\"finish_reason\":\"stop\"}],\"usage\":{\"prompt_tokens\":7,\"completion_tokens\":1}}\n\ndata: [DONE]\n\n"
+}
+
+func askResponses(t *testing.T, s *Server, keyID string, body string) *httptest.ResponseRecorder {
+ t.Helper()
+ ctx := context.Background()
+ if keyID != "" {
+ ctx = access.WithIdentity(ctx, access.Identity{KeyID: keyID, KeyName: keyID})
+ }
+ rec := httptest.NewRecorder()
+ req := httptest.NewRequest("POST", "/v1/responses", strings.NewReader(body)).WithContext(ctx)
+ req.RemoteAddr = "127.0.0.1:5555"
+ s.Handler().ServeHTTP(rec, req)
+ return rec
+}
+
+func responseID(t *testing.T, rec *httptest.ResponseRecorder) string {
+ t.Helper()
+ if rec.Code != 200 {
+ t.Fatalf("%d %s", rec.Code, rec.Body.String())
+ }
+ var r struct {
+ ID string `json:"id"`
+ }
+ if json.Unmarshal(rec.Body.Bytes(), &r) != nil || r.ID == "" {
+ t.Fatalf("no id in %s", rec.Body.String())
+ }
+ return r.ID
+}
+
+// A client that sends only the new input after previous_response_id
+// (kevinzhow on X) has the conversation given back whole, here to a
+// provider on Chat: the model sees the earlier turn and its own answer.
+func TestPreviousResponseGivenWholeToChat(t *testing.T) {
+ up := newPrevUp(t, func(n int, _ []byte) (int, string, string) {
+ return 200, "text/event-stream", chatSSEReply(fmt.Sprintf("answer %d", n))
+ })
+ s := prevSandbox(t, provider.Provider{ID: "fake", Name: "Fake", Key: "k", Models: []string{"m1"}, Chat: up.srv.URL + "/v1"})
+ id := responseID(t, askResponses(t, s, "", `{"model":"fake/m1","instructions":"be brief","input":"my name is Ada"}`))
+ id2 := responseID(t, askResponses(t, s, "", `{"model":"fake/m1","previous_response_id":"`+id+`","input":[{"role":"user","content":"what is my name?"}]}`))
+ sent, _ := json.Marshal(up.last()["messages"])
+ for _, want := range []string{"my name is Ada", "answer 1", "what is my name?"} {
+ if !strings.Contains(string(sent), want) {
+ t.Fatalf("turn 2 went without %q: %s", want, sent)
+ }
+ }
+ // instructions go with the request that gives them, as OpenAI does
+ if strings.Contains(string(sent), "be brief") {
+ t.Fatalf("turn 1's instructions carried: %s", sent)
+ }
+ // and a third turn after the second has all of it
+ responseID(t, askResponses(t, s, "", `{"model":"fake/m1","previous_response_id":"`+id2+`","input":"and again?"}`))
+ sent, _ = json.Marshal(up.last()["messages"])
+ for _, want := range []string{"my name is Ada", "answer 1", "what is my name?", "answer 2", "and again?"} {
+ if !strings.Contains(string(sent), want) {
+ t.Fatalf("turn 3 went without %q: %s", want, sent)
+ }
+ }
+}
+
+// An id magpie doesn't hold, to a provider it would be translated for,
+// is OpenAI's own 400; so is one another gateway key's response, and one
+// the client asked not be kept (store false).
+func TestPreviousResponseNotFound(t *testing.T) {
+ up := newPrevUp(t, func(int, []byte) (int, string, string) {
+ return 200, "text/event-stream", chatSSEReply("ok")
+ })
+ s := prevSandbox(t, provider.Provider{ID: "fake", Name: "Fake", Key: "k", Models: []string{"m1"}, Chat: up.srv.URL + "/v1"})
+ notFound := func(rec *httptest.ResponseRecorder, id string) {
+ t.Helper()
+ var e struct {
+ Error struct{ Message, Type, Param, Code string }
+ }
+ json.Unmarshal(rec.Body.Bytes(), &e)
+ if rec.Code != 400 || e.Error.Code != "previous_response_not_found" || e.Error.Param != "previous_response_id" || e.Error.Type != "invalid_request_error" || !strings.Contains(e.Error.Message, id) {
+ t.Fatalf("%d %s", rec.Code, rec.Body.String())
+ }
+ }
+ notFound(askResponses(t, s, "", `{"model":"fake/m1","previous_response_id":"resp_nowhere","input":"hi"}`), "resp_nowhere")
+ if up.calls() != 0 {
+ t.Fatalf("sent upstream without its history: %d", up.calls())
+ }
+ a := responseID(t, askResponses(t, s, "alice", `{"model":"fake/m1","input":"secret plan"}`))
+ notFound(askResponses(t, s, "bob", `{"model":"fake/m1","previous_response_id":"`+a+`","input":"what was it?"}`), a)
+ notFound(askResponses(t, s, "", `{"model":"fake/m1","previous_response_id":"`+a+`","input":"what was it?"}`), a)
+ responseID(t, askResponses(t, s, "alice", `{"model":"fake/m1","previous_response_id":"`+a+`","input":"what was it?"}`))
+ off := responseID(t, askResponses(t, s, "", `{"model":"fake/m1","store":false,"input":"forget me"}`))
+ notFound(askResponses(t, s, "", `{"model":"fake/m1","previous_response_id":"`+off+`","input":"again"}`), off)
+}
+
+// An id magpie doesn't hold, to a provider on Responses the request is
+// relayed to, goes as it came: a response the provider made before
+// magpie restarted is still its own to find.
+func TestUnknownPreviousResponseRelayedAsSent(t *testing.T) {
+ up := newPrevUp(t, func(int, []byte) (int, string, string) {
+ return 200, "application/json", `{"id":"resp_up","object":"response","status":"completed","output":[{"type":"message","role":"assistant","content":[{"type":"output_text","text":"ok"}]}],"usage":{"input_tokens":3,"output_tokens":1}}`
+ })
+ s := prevSandbox(t, provider.Provider{ID: "fake", Name: "Fake", Key: "k", Models: []string{"m1"}, Responses: up.srv.URL + "/v1"})
+ responseID(t, askResponses(t, s, "", `{"model":"fake/m1","previous_response_id":"resp_before","input":"hi"}`))
+ if got := up.last()["previous_response_id"]; got != "resp_before" {
+ t.Fatalf("sent %v", got)
+ }
+}
+
+// A provider that keeps responses (OpenAI's, Ark's) and answered the one
+// named is sent the new input alone after its own id: the images it holds
+// aren't sent again. One that no longer holds it is sent the whole.
+func TestPreviousResponseKeptUpstreamSentNewInputOnly(t *testing.T) {
+ defer func(old func(provider.Provider) bool) { prevKeptBy = old }(prevKeptBy)
+ prevKeptBy = func(p provider.Provider) bool { return p.ID == "fake" }
+ var forget atomic.Bool
+ up := newPrevUp(t, func(n int, body []byte) (int, string, string) {
+ if forget.Load() && strings.Contains(string(body), "previous_response_id") {
+ return 400, "application/json", `{"error":{"message":"Previous response with id 'resp_up1' not found.","type":"invalid_request_error","param":"previous_response_id","code":"previous_response_not_found"}}`
+ }
+ return 200, "application/json", fmt.Sprintf(`{"id":"resp_up%d","object":"response","status":"completed","output":[{"type":"message","id":"msg_%d","role":"assistant","content":[{"type":"output_text","text":"answer %d"}]}],"usage":{"input_tokens":3,"output_tokens":1}}`, n, n, n)
+ })
+ s := prevSandbox(t, provider.Provider{ID: "fake", Name: "Fake", Key: "k", Models: []string{"m1"}, Responses: up.srv.URL + "/v1"})
+ img := `{"type":"message","role":"user","content":[{"type":"input_image","image_url":"data:image/png;base64,` + strings.Repeat("QUFB", 50000) + `"},{"type":"input_text","text":"what is this?"}]}`
+ id := responseID(t, askResponses(t, s, "", `{"model":"fake/m1","input":[`+img+`]}`))
+ if id != "resp_up1" {
+ t.Fatalf("client given %s", id)
+ }
+ rec := askResponses(t, s, "", `{"model":"fake/m1","previous_response_id":"`+id+`","input":"and now?"}`)
+ if responseID(t, rec) != "resp_up2" {
+ t.Fatal(rec.Body.String())
+ }
+ q := up.last()
+ in, _ := json.Marshal(q["input"])
+ up.mu.Lock()
+ sent := len(up.got[1])
+ up.mu.Unlock()
+ if q["previous_response_id"] != "resp_up1" || strings.Contains(string(in), "QUFB") || !strings.Contains(string(in), "and now?") || sent > 2000 {
+ t.Fatalf("sent %d bytes: prev %v input %s", sent, q["previous_response_id"], in)
+ }
+ // the upstream lost it: asked again with the conversation whole
+ forget.Store(true)
+ responseID(t, askResponses(t, s, "", `{"model":"fake/m1","previous_response_id":"resp_up2","input":"once more"}`))
+ q = up.last()
+ in, _ = json.Marshal(q["input"])
+ if _, has := q["previous_response_id"]; has || !strings.Contains(string(in), "QUFB") || !strings.Contains(string(in), "answer 1") || !strings.Contains(string(in), "and now?") || !strings.Contains(string(in), "once more") {
+ t.Fatalf("whole: prev %v input %.300s", q["previous_response_id"], in)
+ }
+}
+
+// What a response named brings back is masked as the client's own words
+// are: a key in an earlier turn never reaches the vendor.
+func TestPreviousResponseMasked(t *testing.T) {
+ const key = "sk-proj-abcdEFGH1234ijklMNOP5678qrst"
+ up := newPrevUp(t, func(int, []byte) (int, string, string) {
+ return 200, "text/event-stream", chatSSEReply("noted")
+ })
+ s := prevSandbox(t, provider.Provider{ID: "fake", Name: "Fake", Key: "k", Models: []string{"m1"}, Chat: up.srv.URL + "/v1"})
+ if err := settings.Save(settings.Settings{Redact: true}); err != nil {
+ t.Fatal(err)
+ }
+ id := responseID(t, askResponses(t, s, "", `{"model":"fake/m1","input":"my key is `+key+`"}`))
+ responseID(t, askResponses(t, s, "", `{"model":"fake/m1","previous_response_id":"`+id+`","input":"what was it?"}`))
+ sent, _ := json.Marshal(up.last())
+ if strings.Contains(string(sent), key) || !strings.Contains(string(sent), "my key is {{") {
+ t.Fatalf("sent %s", sent)
+ }
+}
+
+// The kept responses are bounded by bytes and time, the oldest out first.
+func TestPrevStoreBounded(t *testing.T) {
+ st := newPrevStore(100, prevTTL)
+ item := json.RawMessage(strings.Repeat("x", 40))
+ for i := range 4 {
+ st.put(&prevTurn{key: prevKey{"s", fmt.Sprint(i)}, items: []json.RawMessage{item}, own: 40})
+ }
+ if _, ok := st.get(prevKey{"s", "0"}); ok {
+ t.Fatal("oldest kept past the cap")
+ }
+ if _, ok := st.get(prevKey{"s", "3"}); !ok || st.size > 100 {
+ t.Fatalf("newest gone, size %d", st.size)
+ }
+ st.put(&prevTurn{key: prevKey{"s", "big"}, own: 60})
+ if _, ok := st.get(prevKey{"s", "big"}); ok {
+ t.Fatal("a turn past half the cap kept")
+ }
+ now := st.now()
+ st.now = func() time.Time { return now.Add(prevTTL + 1) }
+ if _, ok := st.get(prevKey{"s", "3"}); ok {
+ t.Fatal("kept past its time")
+ }
+}