Skip to content

Secure, Fast, and Extensible Sandbox runtime for AI agents.

fix(sdk-go): retry gateway wake timeouts when starting an SSE stream

JasonBuildAI992e06d02cc3
diff --git a/docs/sdks/go.md b/docs/sdks/go.md
index d9547601f..c854c203c 100644
--- a/docs/sdks/go.md
+++ b/docs/sdks/go.md
@@ -503,11 +503,11 @@ config.Retry = &opensandbox.RetryConfig{
}
```
-SSE bootstraps do not use the generic retry loop, even when a write status code is opted in.
+SSE bootstraps do not use the generic retry loop, even when a write status code is opted in. The OSEP-0024 wake retry below still covers the request that starts the stream.
### Wake retry
-A request to a paused sandbox may arrive while its restore is still in progress. The gateway answers `503` with the `OpenSandbox-Wake: expired` marker; the client recognizes it and resends the original request after the `Retry-After` delay (safe for any method — the marker proves the request was never forwarded). The wake budget defaults to 5 attempts / 60 seconds across all wake cycles; when it is exhausted the client returns a `*SandboxWakeTimeoutError` (carrying `Attempts` and `Waited`) while the server-side resume continues in the background. Bare `503` responses follow the existing retry rules untouched. Wake retry is active whenever a `WithRetry` config is set (`WakeRetryEnabled` defaults to enabled even for hand-built configs; pass `WakeRetryDisabled()` to opt out); streaming connections bypass wake retry — a waking sandbox fails fast with the marked 503 instead of parking the connect — matching the other official SDKs, and established streams are never re-executed.
+A request to a paused sandbox may arrive while its restore is still in progress. The gateway answers `503` with the `OpenSandbox-Wake: expired` marker; the client recognizes it and resends the original request after the `Retry-After` delay (safe for any method — the marker proves the request was never forwarded). The wake budget defaults to 5 attempts / 60 seconds across all wake cycles; when it is exhausted the client returns a `*SandboxWakeTimeoutError` (carrying `Attempts` and `Waited`) while the server-side resume continues in the background. Bare `503` responses follow the existing retry rules untouched. Wake retry is active whenever a `WithRetry` config is set (`WakeRetryEnabled` defaults to enabled even for hand-built configs; pass `WakeRetryDisabled()` to opt out). It also covers the request that starts an SSE stream, so a command or code execution against an auto-paused sandbox wakes it like any other request; once a stream is established, its events are never re-executed.
## Fsb Template Management
diff --git a/sdks/sandbox/go/README.md b/sdks/sandbox/go/README.md
index aea6b241d..148ba5fca 100644
--- a/sdks/sandbox/go/README.md
+++ b/sdks/sandbox/go/README.md
@@ -16,7 +16,7 @@ go get github.com/alibaba/OpenSandbox/sdks/sandbox/go
## Retry Behavior
-Idempotent methods (`GET`, `HEAD`, `PUT`, `DELETE`, `OPTIONS`) retry `429`, `502`, `503`, and `504` by default. `POST` and `PATCH` retry only dial, proxy-connect, and TLS-handshake failures that are provably pre-send; status-code retries for writes are opt-in because a replay can duplicate a sandbox, session, command, or file mutation. Set `RetryableStatusCodesNonIdempotent` only when the operation is idempotent at the application layer or duplicate execution is acceptable. SSE bootstraps are never retried.
+Idempotent methods (`GET`, `HEAD`, `PUT`, `DELETE`, `OPTIONS`) retry `429`, `502`, `503`, and `504` by default. `POST` and `PATCH` retry only dial, proxy-connect, and TLS-handshake failures that are provably pre-send; status-code retries for writes are opt-in because a replay can duplicate a sandbox, session, command, or file mutation. Set `RetryableStatusCodesNonIdempotent` only when the operation is idempotent at the application layer or duplicate execution is acceptable. SSE bootstraps never use the generic retry loop; only the OSEP-0024 wake retry covers the request that starts a stream, because its marker proves the gateway never forwarded it.
## Quick Start
diff --git a/sdks/sandbox/go/http.go b/sdks/sandbox/go/http.go
index c716e62a0..5a4a48985 100644
--- a/sdks/sandbox/go/http.go
+++ b/sdks/sandbox/go/http.go
@@ -442,14 +442,18 @@ func (c *Client) doRequestOnce(ctx context.Context, method, path string, body an
}
// doStreamRequest builds an HTTP request, executes it, and streams SSE events
-// through handler. SSE bootstraps deliberately bypass the generic retry loop:
-// the stream may already have delivered partial data or submitted a command,
-// and OSEP-0017 requires the bootstrap to stay unwrapped. Once streaming begins,
-// errors are not retried.
+// through handler. SSE bootstraps bypass the generic retry loop: the stream may
+// already have delivered partial data or submitted a command, and OSEP-0017
+// requires the bootstrap to stay unwrapped by it. The OSEP-0024 wake contract
+// still applies to the bootstrap: a marked wake-timeout 503 proves the gateway
+// never forwarded the request, so replaying it cannot run anything twice.
+// Without that retry a command against an auto-paused sandbox fails instead of
+// waking it, unlike every non-streaming request. Once streaming begins, errors
+// are not retried.
func (c *Client) doStreamRequest(ctx context.Context, method, path string, body any, handler EventHandler) error {
var resp *http.Response
- connectErr := func() error {
+ connectErr := c.withWakeRetry(ctx, func() error {
var bodyReader io.Reader
if body != nil {
buf, err := json.Marshal(body)
@@ -491,7 +495,7 @@ func (c *Client) doStreamRequest(ctx context.Context, method, path string, body
resp = r
return nil
- }()
+ })
if connectErr != nil {
return connectErr
}
diff --git a/sdks/sandbox/go/retry.go b/sdks/sandbox/go/retry.go
index 62a55c68f..cd7729f1c 100644
--- a/sdks/sandbox/go/retry.go
+++ b/sdks/sandbox/go/retry.go
@@ -365,8 +365,23 @@ func (c *Client) withRetryMethod(ctx context.Context, method string, fn func() e
}
// withRetryWake applies the generic policy together with the OSEP-0024 wake
-// contract. Streaming transports intentionally call neither layer.
+// contract.
func (c *Client) withRetryWake(ctx context.Context, method string, fn func() error) error {
+ return c.retryLoop(ctx, method, true, fn)
+}
+
+// withWakeRetry applies only the OSEP-0024 wake contract, never the generic
+// policy. The marked wake-timeout 503 proves the gateway parked the request and
+// never forwarded it, so replaying it cannot execute anything twice and is safe
+// even for a streaming POST bootstrap. Every other failure, including a bare
+// 503, is returned to the caller unwrapped (OSEP-0017).
+func (c *Client) withWakeRetry(ctx context.Context, fn func() error) error {
+ return c.retryLoop(ctx, http.MethodGet, false, fn)
+}
+
+// retryLoop is the shared retry engine. generic controls whether transient
+// failures other than a marked wake timeout are retried according to method.
+func (c *Client) retryLoop(ctx context.Context, method string, generic bool, fn func() error) error {
if c.retry == nil || (c.retry.MaxRetries == 0 && !c.retry.wakeEnabled()) {
return fn()
}
@@ -427,7 +442,7 @@ func (c *Client) withRetryWake(ctx context.Context, method string, fn func() err
continue
}
- if !isTransientErrorForMethod(lastErr, c.retry, method) {
+ if !generic || !isTransientErrorForMethod(lastErr, c.retry, method) {
return lastErr
}
if retriesUsed >= c.retry.MaxRetries {
diff --git a/sdks/sandbox/go/stream_wake_retry_test.go b/sdks/sandbox/go/stream_wake_retry_test.go
new file mode 100644
index 000000000..c9c3f6cbd
--- /dev/null
+++ b/sdks/sandbox/go/stream_wake_retry_test.go
@@ -0,0 +1,82 @@
+// Copyright 2026 The OpenSandbox Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package opensandbox
+
+import (
+ "context"
+ "net/http"
+ "net/http/httptest"
+ "sync/atomic"
+ "testing"
+ "time"
+)
+
+func writeStreamWakeTimeout(w http.ResponseWriter) {
+ w.Header().Set(wakeMarkerHeader, wakeMarkerExpired)
+ w.Header().Set("Retry-After", "0")
+ w.WriteHeader(http.StatusServiceUnavailable)
+ _, _ = w.Write([]byte(`{"code":"sandbox_wake_timeout","message":"restore in progress"}`))
+}
+
+// A bare 503 carries no proof that the request was not forwarded, so the stream
+// bootstrap must stay unwrapped by any retry (OSEP-0017). This pins the boundary
+// the OSEP-0024 wake retry must not cross.
+func TestStreamWakeRetry_Bare503OnCommandStreamIsNotRetried(t *testing.T) {
+ var attempts atomic.Int32
+ srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ attempts.Add(1)
+ w.WriteHeader(http.StatusServiceUnavailable)
+ _, _ = w.Write([]byte(`{"code":"UNAVAILABLE","message":"try again"}`))
+ }))
+ defer srv.Close()
+
+ client := NewExecdClient(srv.URL, "tok", WithRetry(RetryConfig{
+ MaxRetries: 2,
+ WakeMaxAttempts: 5,
+ WakeOverallDeadline: time.Minute,
+ }))
+
+ err := client.RunCommand(context.Background(), RunCommandRequest{Command: "echo hello"}, func(StreamEvent) error {
+ return nil
+ })
+ require.Error(t, err, "a bare 503 on a POST stream must not be replayed")
+ require.Equal(t, 1, int(attempts.Load()), "stream bootstrap attempts")
+ var apiErr *APIError
+ require.ErrorAs(t, err, &apiErr)
+ require.Equal(t, http.StatusServiceUnavailable, apiErr.StatusCode, "status code")
+}
+
+// Exhausting the wake budget on a stream must surface the same typed error as
+// the non-streaming paths so callers can tell "never forwarded" from a command
+// failure.
+func TestStreamWakeRetry_BudgetExhaustionReturnsTypedError(t *testing.T) {
+ var attempts atomic.Int32
+ srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ attempts.Add(1)
+ writeStreamWakeTimeout(w)
+ }))
+ defer srv.Close()
+
+ client := NewExecdClient(srv.URL, "tok", WithRetry(wakeConfig(3, time.Minute)))
+
+ err := client.RunCommand(context.Background(), RunCommandRequest{Command: "echo hello"}, func(StreamEvent) error {
+ return nil
+ })
+ require.Error(t, err, "exhausted wake retry on a stream")
+ var wakeErr *SandboxWakeTimeoutError
+ require.ErrorAs(t, err, &wakeErr)
+ require.Equal(t, 3, wakeErr.Attempts, "wake attempts")
+ require.Equal(t, 3, int(attempts.Load()), "stream bootstrap attempts")
+}
diff --git a/sdks/sandbox/go/wake_retry_test.go b/sdks/sandbox/go/wake_retry_test.go
index 77a6284fd..7028e44ab 100644
--- a/sdks/sandbox/go/wake_retry_test.go
+++ b/sdks/sandbox/go/wake_retry_test.go
@@ -526,11 +526,11 @@ func intPtr(v int) *int { return &v }
func boolPtr(v bool) *bool { return &v }
-func TestWakeRetry_StreamConnectDoesNotParkOnWake(t *testing.T) {
- // The other SDKs leave streaming transports unwrapped: a marked 503 while
- // establishing a stream must fail fast instead of parking the connect for
- // up to the wake budget. The POST stream is not replayed by the generic
- // policy either, so the caller receives the marked error.
+func TestWakeRetry_StreamConnectRetriesMarkedWake(t *testing.T) {
+ // A marked wake 503 proves the gateway parked the request and never
+ // forwarded it, so the stream connect retries it under the OSEP-0024 wake
+ // contract exactly like a non-streaming request. Failing fast here would
+ // make a command against an auto-paused sandbox fail instead of waking it.
var attempts atomic.Int32
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
n := attempts.Add(1)
@@ -559,10 +559,7 @@ func TestWakeRetry_StreamConnectDoesNotParkOnWake(t *testing.T) {
events = append(events, event)
return nil
})
- require.Error(t, err, "a marked wake 503 on a POST stream must fail fast")
- var apiErr *APIError
- require.ErrorAs(t, err, &apiErr)
- require.True(t, apiErr.WakeExpired)
- require.Equal(t, 1, int(attempts.Load()))
- require.Len(t, events, 0)
+ require.NoError(t, err, "a marked wake 503 on a POST stream must be retried")
+ require.Equal(t, 2, int(attempts.Load()))
+ require.Len(t, events, 1)
}