Skip to content

The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu

feat(endpoint): backfill recent agent sessions on install and ship them on first connect

Claudeb1d2b9dea84f
diff --git a/CLAUDE.md b/CLAUDE.md
index 20cbcfeb..a638f63a 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -27,7 +27,8 @@ Do not recreate or depend on removed `asymptote` mirror trees. Keep new work foc
- Beacon is visibility-first endpoint telemetry for local AI agent runtimes, not a hosted policy service or general endpoint protection product.
- Preserve the local-only default. The public Beacon build must not require a hosted account, remote policy fetch, hosted dashboard, or external network dependency during normal hook execution, and hooks themselves never touch the network.
- `beacon login` is the account-identity channel for interactive users and is required by the interactive `beacon endpoint install` wizard. It uses browser PKCE against beacon.sh and stores the scoped CLI token only in `~/.beacon/auth/session.json` (0600 under a 0700 directory). Signing in alone never starts forwarding. User-mode `beacon endpoint connect` may send that token once to `POST /api/cli/enroll/account` to authorize minting a separate device ingest key, and `beacon mcp token create` may send it to `POST /api/cli/mcp/tokens` (scope `mcp:token:create`, listed on the sign-in page) to mint a personal MCP token that is printed once to stdout and never written to a file; never copy the account token into endpoint config, Vector, hook environments, logs, or command output. System/root/package/MDM/CI/piped-stdin/dry-run installs stay noninteractive and account-free.
-- Beacon Cloud forwarding (`cli/beacon/internal/endpoint/asymptote`, `beacon endpoint connect`, `beacon endpoint asymptote`) is Vector-shaped: Beacon writes local JSONL, Vector ships runtime lines only from the connection point forward and reads inventory from the beginning to establish the endpoint's current baseline, and the `bcn_device_*` key stays only in `<BaseDir>/asymptote/vector-secrets.json` via Vector's secret backend. The onboarding wizard preselects Beacon Cloud, and confirming it connects this endpoint in the same command once the local install has succeeded; an endpoint that is already enrolled is never re-enrolled, because enrollment rotates the device key. Standard mode ships locally sanitized retained content; metadata-only uses local Vector transforms to remove prompts, responses, reasoning, tool arguments/results, command output, raw fields, diffs, inventory content, and MCP definitions before buffering/upload. Keep the ingest URL `https://`, the wire contract aligned with `beacon-ingest` (`/v1/ingest/{runtime,inventory}` and `/v1/ingest/health`), and do not add another endpoint network channel. The cloud-sandbox shuttle is the bounded exception: only with `BEACON_ORIGIN=cloud`, it can upload new runtime lines to GCS, S3, or the same Beacon ingest contract because no Vector/service manager/browser/persistent disk exists there.
+- Beacon Cloud forwarding (`cli/beacon/internal/endpoint/asymptote`, `beacon endpoint connect`, `beacon endpoint asymptote`) is Vector-shaped: Beacon writes local JSONL, Vector ships runtime lines only from the connection point forward and reads inventory from the beginning to establish the endpoint's current baseline, plus the one bounded exception of the session backfill (below), and the `bcn_device_*` key stays only in `<BaseDir>/asymptote/vector-secrets.json` via Vector's secret backend. The onboarding wizard preselects Beacon Cloud, and confirming it connects this endpoint in the same command once the local install has succeeded; an endpoint that is already enrolled is never re-enrolled, because enrollment rotates the device key. Standard mode ships locally sanitized retained content; metadata-only uses local Vector transforms to remove prompts, responses, reasoning, tool arguments/results, command output, raw fields, diffs, inventory content, and MCP definitions before buffering/upload. Keep the ingest URL `https://`, the wire contract aligned with `beacon-ingest` (`/v1/ingest/{runtime,inventory}` and `/v1/ingest/health`), and do not add another endpoint network channel. The cloud-sandbox shuttle is the bounded exception: only with `BEACON_ORIGIN=cloud`, it can upload new runtime lines to GCS, S3, or the same Beacon ingest contract because no Vector/service manager/browser/persistent disk exists there.
+- Session backfill (`cli/beacon/cmd/endpoint_backfill.go`; docs in `docs/cli/endpoint-install.mdx#session-backfill` and `docs/cli/endpoint-connect.mdx#session-backfill`): after a successful user-mode install, every session-store collector runs once with `CollectOptions.Since` (30 days; `sessionwindow.Recent` keeps those sessions, newest first, and never touches the cursors of the rest) and a shared `writer.Budget` (24 MiB, at most 12 MiB per runtime through `Budget.Sub`; a refused append returns `writer.ErrBudgetSpent`, the collector stops with cursors past only what was written, and a manual sync reads the rest). A new session-store collector takes both fields and joins `endpointBackfillRunners`. A first user-mode connect runs the same sweep before enrollment (skipped when install just ran it) and, before the forwarder starts, `asymptote.StageBackfill` copies the window's `harness.collection_method=poll` lines into `cloud-backfill.jsonl` beside the runtime log, which the `beacon_backfill` Vector source reads from the beginning into the runtime sink through the same privacy transform. Live hook/OTLP/plugin history before the connection point stays local. Re-connects never stage, a privacy-mode change deletes the staged file with the checkpoints, and `disconnect` removes it. The wizard's confirm screen names the upload (`WizardOptions.BackfillDays`). System-mode, dry-run, `--no-backfill` and `BEACON_BACKFILL=0` skip all of it, so unattended paths are unchanged; the cmd tests run with `BEACON_BACKFILL=0` set in `TestMain`.
- `beacon endpoint install` has a full-screen onboarding wizard on interactive user TTYs. It signs in, preselects Beacon Cloud, requires an explicit choice to opt out to Local, and asks Managed users to choose Standard or Metadata-only privacy. Signing in alone still moves no telemetry. Confirming Beacon Cloud does: the confirm screen states that it installs Beacon and starts forwarding, and names what the selected privacy mode sends, so that screen is the consent point. The managed destination is recorded only after enrollment succeeds, so a failed enrollment leaves a working local install, prints `beacon endpoint connect` as the retry, and re-offers the question on the next install. The onboarding record is written only after `lifecycle.Install` returns, because a profile written first left a failed install looking onboarded and silently skipped the wizard on the retry. Explicit `install --connect` is unchanged. The wizard is skipped for system/root/CI/piped-stdin/dry-run or `BEACON_ONBOARDING=0`; those unattended paths must never gain an account or browser dependency. Legacy `BEACON_ONBOARDING_EMAIL`/`USAGE` submissions remain only for fleet compatibility. Implementation lives in `cli/beacon/internal/onboarding` and `cli/beacon/cmd/endpoint_onboarding.go`; keep `docs/cli/endpoint-onboarding.mdx` in sync.
- Do not add dependency vulnerability scanning, OSV/GHSA lookups, package remediation, or other vulnerability-enforcement flows to the public hook path.
- Do not add broad runtime enforcement unless explicitly requested. Current control behavior is limited to hook-native approvals/denials exposed by supported agent runtimes.
diff --git a/cli/beacon/cmd/endpoint.go b/cli/beacon/cmd/endpoint.go
index 98e27142..d2bb7616 100644
--- a/cli/beacon/cmd/endpoint.go
+++ b/cli/beacon/cmd/endpoint.go
@@ -39,6 +39,7 @@ var endpointOpts struct {
onboardingReset bool
onboardingResend bool
connect bool
+ noBackfill bool
noBrowser bool
fix bool
allTargets bool
@@ -428,6 +429,7 @@ func init() {
endpointInstallCmd.Flags().BoolVar(&endpointOpts.noBrowser, "no-browser", false, "During interactive setup, print the sign-in URL instead of opening a browser")
endpointInstallCmd.Flags().StringVar(&endpointOpts.serviceKind, "service", "", "Service manager to use: auto (default), launchd, systemd, or none for a supervised collector process")
endpointInstallCmd.Flags().BoolVar(&endpointOpts.dryRun, "dry-run", false, "Print planned actions without changing endpoint files or services")
+ endpointInstallCmd.Flags().BoolVar(&endpointOpts.noBackfill, "no-backfill", false, "Skip reading the last 30 days of agent sessions from their local session stores into the runtime log (also "+endpointBackfillEnv+"=0)")
endpointInstallCmd.Flags().BoolVar(&endpointOpts.connect, "connect", false, "After installing, attempt to connect this endpoint to Beacon Cloud (user mode uses the signed-in account; system mode opens browser approval)")
endpointOnboardingCmd.Flags().BoolVar(&endpointOpts.onboardingReset, "reset", false, "Clear the onboarding record so the question is asked again")
endpointOnboardingCmd.Flags().BoolVar(&endpointOpts.onboardingResend, "resend", false, "Retry a signup that could not be delivered")
diff --git a/cli/beacon/cmd/endpoint_backfill.go b/cli/beacon/cmd/endpoint_backfill.go
new file mode 100644
index 00000000..67da413c
--- /dev/null
+++ b/cli/beacon/cmd/endpoint_backfill.go
@@ -0,0 +1,257 @@
+package cmd
+
+import (
+ "errors"
+ "fmt"
+ "io"
+ "os"
+ "strings"
+ "time"
+
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/claudesession"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/clinesession"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/codexsession"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/copilotsession"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/cursorsession"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/dshsession"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/asymptote"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/writer"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/factorysession"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/fxsession"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/groksession"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/hermessession"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/openclawsession"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/opencodesession"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/pisession"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/primesession"
+)
+
+// The install-time session backfill reads recent history from every agent's own session store,
+// the same sweep `beacon endpoint <runtime> sync` runs, so the local dashboard and a first Beacon
+// Cloud connect have something to show before any agent runs again.
+//
+// It is bounded twice. The window keeps it to recent sessions, and the byte budget keeps it to a
+// fraction of the runtime log's rotation window (DefaultRotateBytes times the live file and
+// DefaultRotateArchives archives, 60 MiB), so a machine with years of history cannot rotate out
+// the live telemetry the log already holds. No one runtime may take more than its share, so a
+// heavy Claude Code history still leaves room for Codex. Whatever does not fit is not lost: the
+// cursors move only past what was written, and a manual sync reads the rest.
+const (
+ endpointBackfillWindow = 30 * 24 * time.Hour
+ // endpointBackfillEnv set to 0 turns the backfill off for install and connect.
+ endpointBackfillEnv = "BEACON_BACKFILL"
+)
+
+// The byte limits are variables only so tests can make them small.
+var (
+ endpointBackfillMaxBytes int64 = 24 << 20
+ endpointBackfillShareBytes int64 = 12 << 20
+)
+
+// endpointBackfillRun is what one runtime's sweep needs from the orchestrator.
+type endpointBackfillRun struct {
+ LogPath string
+ Since time.Time
+ Budget *writer.Budget
+}
+
+// endpointBackfillRunner is one session-store collector.
+type endpointBackfillRunner struct {
+ // Name is how the summary names the runtime.
+ Name string
+ // Sync is the command that reads the rest when the budget runs out.
+ Sync string
+ // Run sweeps the store once and reports how many events it wrote.
+ Run func(endpointBackfillRun) (int, error)
+}
+
+// endpointBackfillRunners lists every session-store collector, a seam for tests. Each one runs with
+// the same user-mode defaults its own sync command uses.
+var endpointBackfillRunners = func() []endpointBackfillRunner {
+ return []endpointBackfillRunner{
+ {Name: "Claude Code", Sync: "beacon endpoint claude sync", Run: func(r endpointBackfillRun) (int, error) {
+ s, err := claudesession.CollectOnce(claudesession.CollectOptions{Write: true, UserMode: true, LogPath: r.LogPath, StatePath: resolveClaudeStatePath("", true), Since: r.Since, Budget: r.Budget})
+ return s.EventsEmitted, err
+ }},
+ {Name: "Codex CLI", Sync: "beacon endpoint codex sync", Run: func(r endpointBackfillRun) (int, error) {
+ s, err := codexsession.CollectOnce(codexsession.CollectOptions{Write: true, UserMode: true, LogPath: r.LogPath, StatePath: resolveCodexStatePath("", true), Since: r.Since, Budget: r.Budget})
+ return s.EventsEmitted, err
+ }},
+ {Name: "Cursor", Sync: "beacon endpoint cursor sync", Run: func(r endpointBackfillRun) (int, error) {
+ s, err := cursorsession.CollectOnce(cursorsession.CollectOptions{Write: true, UserMode: true, LogPath: r.LogPath, StatePath: resolveCursorStatePath("", true), Since: r.Since, Budget: r.Budget})
+ return s.EventsEmitted, err
+ }},
+ {Name: "OpenCode", Sync: "beacon endpoint opencode sync", Run: func(r endpointBackfillRun) (int, error) {
+ s, err := opencodesession.CollectOnce(opencodesession.CollectOptions{Write: true, UserMode: true, LogPath: r.LogPath, StatePath: resolveOpenCodeStatePath("", true), Since: r.Since, Budget: r.Budget})
+ return s.EventsEmitted, err
+ }},
+ {Name: "GitHub Copilot CLI", Sync: "beacon endpoint copilot sync", Run: func(r endpointBackfillRun) (int, error) {
+ s, err := copilotsession.CollectOnce(copilotsession.CollectOptions{Write: true, UserMode: true, LogPath: r.LogPath, StatePath: resolveCopilotStatePath("", true), Since: r.Since, Budget: r.Budget})
+ return s.EventsEmitted, err
+ }},
+ {Name: "Cline", Sync: "beacon endpoint cline sync", Run: func(r endpointBackfillRun) (int, error) {
+ s, err := clinesession.CollectOnce(clinesession.CollectOptions{Write: true, UserMode: true, LogPath: r.LogPath, StatePath: resolveClineStatePath("", true), Since: r.Since, Budget: r.Budget})
+ return s.EventsEmitted, err
+ }},
+ {Name: "Factory Droid", Sync: "beacon endpoint factory sync", Run: func(r endpointBackfillRun) (int, error) {
+ s, err := factorysession.CollectOnce(factorysession.CollectOptions{Write: true, UserMode: true, LogPath: r.LogPath, StatePath: resolveFactoryStatePath("", true), Since: r.Since, Budget: r.Budget})
+ return s.EventsEmitted, err
+ }},
+ {Name: "Pi", Sync: "beacon endpoint pi sync", Run: func(r endpointBackfillRun) (int, error) {
+ s, err := pisession.CollectOnce(pisession.CollectOptions{Write: true, UserMode: true, LogPath: r.LogPath, StatePath: resolvePiStatePath("", true), Since: r.Since, Budget: r.Budget})
+ return s.EventsEmitted, err
+ }},
+ {Name: "Prime Agent", Sync: "beacon endpoint prime sync", Run: func(r endpointBackfillRun) (int, error) {
+ s, err := primesession.CollectOnce(primesession.CollectOptions{Write: true, UserMode: true, LogPath: r.LogPath, StatePath: resolvePrimeStatePath("", true), Since: r.Since, Budget: r.Budget})
+ return s.EventsEmitted, err
+ }},
+ {Name: "Hermes Agent", Sync: "beacon endpoint hermes sync", Run: func(r endpointBackfillRun) (int, error) {
+ s, err := hermessession.CollectOnce(hermessession.CollectOptions{Write: true, UserMode: true, LogPath: r.LogPath, StatePath: resolveHermesStatePath("", true), Since: r.Since, Budget: r.Budget})
+ return s.EventsEmitted, err
+ }},
+ {Name: "DeepSeek Harness", Sync: "beacon endpoint dsh sync", Run: func(r endpointBackfillRun) (int, error) {
+ s, err := dshsession.CollectOnce(dshsession.CollectOptions{Write: true, UserMode: true, LogPath: r.LogPath, StatePath: resolveDshStatePath("", true), Since: r.Since, Budget: r.Budget})
+ return s.EventsEmitted, err
+ }},
+ {Name: "fx", Sync: "beacon endpoint fx sync", Run: func(r endpointBackfillRun) (int, error) {
+ s, err := fxsession.CollectOnce(fxsession.CollectOptions{Write: true, UserMode: true, LogPath: r.LogPath, StatePath: resolveFxStatePath("", true), Since: r.Since, Budget: r.Budget})
+ return s.EventsEmitted, err
+ }},
+ {Name: "Grok Build", Sync: "beacon endpoint grok sync", Run: func(r endpointBackfillRun) (int, error) {
+ s, err := groksession.CollectOnce(groksession.CollectOptions{Write: true, UserMode: true, LogPath: r.LogPath, StatePath: resolveGrokStatePath("", true), Since: r.Since, Budget: r.Budget})
+ return s.EventsEmitted, err
+ }},
+ {Name: "OpenClaw Gateway", Sync: "beacon endpoint integrations openclaw sessions sync", Run: func(r endpointBackfillRun) (int, error) {
+ s, err := openclawsession.CollectOnce(openclawsession.CollectOptions{Write: true, UserMode: true, LogPath: r.LogPath, StatePath: resolveOpenClawSessionsStatePath("", true), Since: r.Since, Budget: r.Budget})
+ return s.EventsEmitted, err
+ }},
+ }
+}
+
+// endpointBackfillNow is time.Now, a seam for tests.
+var endpointBackfillNow = time.Now
+
+// endpointBackfillRuntimeResult is one runtime's part of a backfill.
+type endpointBackfillRuntimeResult struct {
+ Name string
+ Sync string
+ Events int
+ Limited bool
+ Err error
+}
+
+// endpointBackfillResult is what one backfill wrote.
+type endpointBackfillResult struct {
+ Since time.Time
+ Events int
+ Bytes int64
+ Limited bool
+ Runtimes []endpointBackfillRuntimeResult
+}
+
+// endpointBackfillEnabled reports whether this install or connect should backfill. Only a user-mode
+// endpoint does: the collectors read the invoking user's home, which for a system install is root's,
+// and the unattended system paths stay exactly as they were.
+func endpointBackfillEnabled(userMode bool) bool {
+ if !userMode || endpointOpts.dryRun || endpointOpts.noBackfill {
+ return false
+ }
+ return strings.TrimSpace(os.Getenv(endpointBackfillEnv)) != "0"
+}
+
+// endpointBackfillDays is the backfill window in days for an interactive user install, or zero when
+// the backfill is off; the onboarding wizard names it on its confirm screen.
+func endpointBackfillDays() int {
+ if !endpointBackfillEnabled(true) {
+ return 0
+ }
+ return int(endpointBackfillWindow / (24 * time.Hour))
+}
+
+// runEndpointBackfill sweeps every session store once into logPath, newest sessions first, within
+// the window and the budget. A runtime that fails is reported and the rest still run: the backfill
+// is a convenience on top of a working install and never fails it.
+func runEndpointBackfill(logPath string) endpointBackfillResult {
+ result := endpointBackfillResult{Since: endpointBackfillNow().Add(-endpointBackfillWindow)}
+ total := writer.NewBudget(endpointBackfillMaxBytes)
+ for _, runner := range endpointBackfillRunners() {
+ share := total.Sub(endpointBackfillShareBytes)
+ events, err := runner.Run(endpointBackfillRun{LogPath: logPath, Since: result.Since, Budget: share})
+ runtime := endpointBackfillRuntimeResult{Name: runner.Name, Sync: runner.Sync, Events: events}
+ if errors.Is(err, writer.ErrBudgetSpent) || share.Refused() {
+ runtime.Limited = true
+ result.Limited = true
+ }
+ if err != nil && !onlyBudgetSpent(err) {
+ runtime.Err = err
+ }
+ result.Events += events
+ if events > 0 || runtime.Limited || runtime.Err != nil {
+ result.Runtimes = append(result.Runtimes, runtime)
+ }
+ }
+ result.Bytes = total.Used()
+ return result
+}
+
+// onlyBudgetSpent reports whether err says nothing but that the budget ran out.
+func onlyBudgetSpent(err error) bool {
+ if joined, ok := err.(interface{ Unwrap() []error }); ok {
+ for _, e := range joined.Unwrap() {
+ if !onlyBudgetSpent(e) {
+ return false
+ }
+ }
+ return true
+ }
+ return errors.Is(err, writer.ErrBudgetSpent)
+}
+
+// reportEndpointBackfill prints what a backfill wrote: one line when it went cleanly, plus a line
+// per runtime that hit the limit or failed, each naming the command that picks up from there.
+func reportEndpointBackfill(out io.Writer, result endpointBackfillResult) {
+ days := int(endpointBackfillWindow / (24 * time.Hour))
+ if result.Events == 0 && !result.Limited && len(result.Runtimes) == 0 {
+ fmt.Fprintf(out, "Session backfill: no new agent session activity from the last %d days.\n", days)
+ return
+ }
+ var parts []string
+ for _, r := range result.Runtimes {
+ if r.Events > 0 {
+ parts = append(parts, fmt.Sprintf("%s %d", r.Name, r.Events))
+ }
+ }
+ line := fmt.Sprintf("Session backfill: %d events from the last %d days", result.Events, days)
+ if len(parts) > 0 {
+ line += " (" + strings.Join(parts, ", ") + ")"
+ }
+ fmt.Fprintln(out, line+".")
+ for _, r := range result.Runtimes {
+ switch {
+ case r.Err != nil:
+ fmt.Fprintf(out, " %s: %v; retry with `%s`.\n", r.Name, r.Err, r.Sync)
+ case r.Limited:
+ fmt.Fprintf(out, " %s: stopped at the backfill size limit; `%s` reads the rest.\n", r.Name, r.Sync)
+ }
+ }
+}
+
+// endpointConnectBackfill is the backfill a user-mode connect ships, or nil when it ships none.
+// synced says this process has just run the install-time backfill, so connect does not sweep again
+// and spend a second budget; it still stages what the first sweep wrote.
+func endpointConnectBackfill(out io.Writer, userMode bool, logPath string, synced bool) *asymptote.ConnectBackfill {
+ if !endpointBackfillEnabled(userMode) {
+ return nil
+ }
+ backfill := &asymptote.ConnectBackfill{Options: asymptote.BackfillOptions{
+ Since: endpointBackfillNow().Add(-endpointBackfillWindow),
+ MaxBytes: endpointBackfillMaxBytes,
+ }}
+ if !synced {
+ backfill.Sync = func() error {
+ reportEndpointBackfill(out, runEndpointBackfill(logPath))
+ return nil
+ }
+ }
+ return backfill
+}
diff --git a/cli/beacon/cmd/endpoint_backfill_test.go b/cli/beacon/cmd/endpoint_backfill_test.go
new file mode 100644
index 00000000..5aea3c5a
--- /dev/null
+++ b/cli/beacon/cmd/endpoint_backfill_test.go
@@ -0,0 +1,231 @@
+package cmd
+
+import (
+ "context"
+ "errors"
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/account"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/asymptote"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/lifecycle"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/service"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/writer"
+ "github.com/spf13/cobra"
+)
+
+// stubBackfill replaces the collectors with runners that each try to write events events, so a
+// test sees how the orchestrator shares the budget without any real session store.
+func stubBackfill(t *testing.T, maxBytes, shareBytes int64, runners ...endpointBackfillRunner) {
+ t.Helper()
+ t.Setenv(endpointBackfillEnv, "")
+ oldRunners, oldMax, oldShare, oldNow := endpointBackfillRunners, endpointBackfillMaxBytes, endpointBackfillShareBytes, endpointBackfillNow
+ t.Cleanup(func() {
+ endpointBackfillRunners, endpointBackfillMaxBytes, endpointBackfillShareBytes, endpointBackfillNow = oldRunners, oldMax, oldShare, oldNow
+ })
+ endpointBackfillRunners = func() []endpointBackfillRunner { return runners }
+ endpointBackfillMaxBytes, endpointBackfillShareBytes = maxBytes, shareBytes
+ endpointBackfillNow = func() time.Time { return time.Date(2026, 10, 5, 12, 0, 0, 0, time.UTC) }
+}
+
+func writingRunner(name string, events int, since *time.Time) endpointBackfillRunner {
+ return endpointBackfillRunner{Name: name, Sync: "beacon endpoint " + strings.ToLower(name) + " sync", Run: func(r endpointBackfillRun) (int, error) {
+ if since != nil {
+ *since = r.Since
+ }
+ written := 0
+ for i := 0; i < events; i++ {
+ event := schema.NewEvent(schema.NewEventOptions{
+ Action: "prompt.submitted",
+ Category: "agent",
+ Severity: schema.SeverityInfo,
+ Harness: schema.HarnessInfo{Name: strings.ToLower(name), CollectionMethod: schema.CollectionMethodPoll},
+ Message: strings.Repeat("x", 200) + name + string(rune('a'+i%26)) + string(rune('a'+i/26)),
+ })
+ if _, err := writer.AppendEvent(event, writer.Options{Path: r.LogPath, UserMode: true, Budget: r.Budget}); err != nil {
+ return written, errors.Join(err)
+ }
+ written++
+ }
+ return written, nil
+ }}
+}
+
+func TestEndpointBackfillGivesEachRuntimeAShareAndNamesWhereToContinue(t *testing.T) {
+ var since time.Time
+ stubBackfill(t, 6000, 2500,
+ writingRunner("Heavy", 100, &since),
+ writingRunner("Light", 2, nil),
+ writingRunner("Late", 100, nil),
+ endpointBackfillRunner{Name: "Broken", Sync: "beacon endpoint broken sync", Run: func(endpointBackfillRun) (int, error) {
+ return 0, errors.New("store unreadable")
+ }},
+ writingRunner("Absent", 0, nil),
+ )
+ logPath := filepath.Join(t.TempDir(), "runtime.jsonl")
+ result := runEndpointBackfill(logPath)
+
+ if want := time.Date(2026, 9, 5, 12, 0, 0, 0, time.UTC); !since.Equal(want) {
+ t.Fatalf("collectors were given since=%s, want %s", since, want)
+ }
+ byName := map[string]endpointBackfillRuntimeResult{}
+ for _, r := range result.Runtimes {
+ byName[r.Name] = r
+ }
+ if _, ok := byName["Absent"]; ok {
+ t.Fatal("a runtime with nothing to read is reported")
+ }
+ if !byName["Heavy"].Limited || byName["Light"].Limited || byName["Light"].Events != 2 || !byName["Late"].Limited {
+ t.Fatalf("runtimes = %+v", result.Runtimes)
+ }
+ if byName["Late"].Events == 0 {
+ t.Fatal("the first runtime's share left nothing for the ones after it")
+ }
+ if byName["Broken"].Err == nil || byName["Heavy"].Err != nil {
+ t.Fatalf("errors: heavy=%v broken=%v", byName["Heavy"].Err, byName["Broken"].Err)
+ }
+ info, err := os.Stat(logPath)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if info.Size() > 6000 || result.Bytes != info.Size() || !result.Limited {
+ t.Fatalf("log holds %d bytes, result=%+v", info.Size(), result)
+ }
+
+ var out strings.Builder
+ reportEndpointBackfill(&out, result)
+ text := out.String()
+ for _, want := range []string{
+ "Session backfill: ",
+ "from the last 30 days",
+ "Heavy: stopped at the backfill size limit; `beacon endpoint heavy sync` reads the rest.",
+ "Broken: store unreadable; retry with `beacon endpoint broken sync`.",
+ } {
+ if !strings.Contains(text, want) {
+ t.Fatalf("report missing %q:\n%s", want, text)
+ }
+ }
+}
+
+func TestEndpointBackfillReportsAnEmptyMachinePlainly(t *testing.T) {
+ stubBackfill(t, 4000, 2500, writingRunner("Absent", 0, nil))
+ var out strings.Builder
+ reportEndpointBackfill(&out, runEndpointBackfill(filepath.Join(t.TempDir(), "runtime.jsonl")))
+ if got := strings.TrimSpace(out.String()); got != "Session backfill: no new agent session activity from the last 30 days." {
+ t.Fatalf("report = %q", got)
+ }
+}
+
+func TestEndpointInstallBackfillsAfterTheInstallSucceeds(t *testing.T) {
+ recordEndpointLifecycle(t)
+ logPath := filepath.Join(t.TempDir(), "runtime.jsonl")
+ endpointLifecycleInstall = func(lifecycle.InstallOptions) (lifecycle.InstallResult, error) {
+ return lifecycle.InstallResult{LogPath: logPath}, nil
+ }
+ stubBackfill(t, 1<<20, 1<<20, writingRunner("Codex", 3, nil))
+
+ run := func(mutate func()) string {
+ t.Helper()
+ _ = os.Remove(logPath)
+ endpointOpts.harnesses = ""
+ endpointOpts.noBackfill = false
+ endpointOpts.userMode, endpointOpts.systemMode = true, false
+ mutate()
+ var out strings.Builder
+ cmd := &cobra.Command{Use: "install"}
+ cmd.SetOut(&out)
+ cmd.SetErr(&out)
+ if err := runEndpointInstall(cmd, nil); err != nil {
+ t.Fatal(err)
+ }
+ return out.String()
+ }
+
+ if text := run(func() {}); !strings.Contains(text, "Session backfill: 3 events from the last 30 days (Codex 3).") {
+ t.Fatalf("install did not report the backfill:\n%s", text)
+ }
+ if _, err := os.Stat(logPath); err != nil {
+ t.Fatalf("install did not write the backfill to the runtime log: %v", err)
+ }
+ for name, mutate := range map[string]func(){
+ "--no-backfill": func() { endpointOpts.noBackfill = true },
+ "BEACON_BACKFILL=0": func() { t.Setenv(endpointBackfillEnv, "0") },
+ } {
+ if text := run(mutate); strings.Contains(text, "Session backfill") {
+ t.Fatalf("%s still backfilled:\n%s", name, text)
+ }
+ if _, err := os.Stat(logPath); !os.IsNotExist(err) {
+ t.Fatalf("%s still wrote the runtime log", name)
+ }
+ t.Setenv(endpointBackfillEnv, "")
+ }
+ if endpointBackfillEnabled(false) {
+ t.Fatal("a system install backfills root's home")
+ }
+}
+
+func TestConnectShipsTheBackfillAndSaysSo(t *testing.T) {
+ originalLoad, originalConnect := connectAccountLoad, connectManagedEndpoint
+ originalConnectOpts, originalEndpointOpts := connectOpts, endpointOpts
+ t.Cleanup(func() {
+ connectAccountLoad, connectManagedEndpoint = originalLoad, originalConnect
+ connectOpts, endpointOpts = originalConnectOpts, originalEndpointOpts
+ })
+ stubBackfill(t, 1<<20, 1<<20, writingRunner("Codex", 2, nil))
+ connectOpts.privacyMode = "standard"
+ connectAccountLoad = func() (*account.Session, error) {
+ return &account.Session{
+ BaseURL: "https://beacon.sh", AccessToken: "bcn_cli_secret", ExpiresAt: time.Now().Add(time.Hour),
+ Scopes: []string{account.ScopeProfileRead, account.ScopeDeviceEnroll},
+ }, nil
+ }
+ var captured asymptote.ConnectOptions
+ staged := 0
+ connectManagedEndpoint = func(_ context.Context, options asymptote.ConnectOptions) (*asymptote.ConnectResult, error) {
+ captured = options
+ result := &asymptote.ConnectResult{
+ Enrollment: asymptote.Enrollment{DeviceID: "dev-1", DashboardURL: "https://beacon.sh"},
+ ForwarderState: service.Status{Loaded: true, Running: true},
+ }
+ if options.Backfill != nil {
+ result.Backfill = &asymptote.BackfillResult{Events: staged}
+ }
+ return result, nil
+ }
+ connect := func(backfilled bool) string {
+ t.Helper()
+ var out strings.Builder
+ command := &cobra.Command{}
+ command.SetOut(&out)
+ command.SetErr(&out)
+ if err := connectEndpoint(command, true, filepath.Join(t.TempDir(), "runtime.jsonl"), backfilled); err != nil {
+ t.Fatal(err)
+ }
+ return out.String()
+ }
+
+ staged = 2
+ text := connect(false)
+ if captured.Backfill == nil || captured.Backfill.Sync == nil || captured.Backfill.Options.MaxBytes != endpointBackfillMaxBytes ||
+ !captured.Backfill.Options.Since.Equal(time.Date(2026, 9, 5, 12, 0, 0, 0, time.UTC)) {
+ t.Fatalf("connect backfill = %+v", captured.Backfill)
+ }
+ if !strings.Contains(text, "Uploading 2 events of recent session history; they appear at https://beacon.sh/dashboard") {
+ t.Fatalf("connect does not say the history is on its way:\n%s", text)
+ }
+
+ connect(true)
+ if captured.Backfill == nil || captured.Backfill.Sync != nil {
+ t.Fatalf("connect after an install backfill swept again: %+v", captured.Backfill)
+ }
+
+ t.Setenv(endpointBackfillEnv, "0")
+ staged = 0
+ if text := connect(false); captured.Backfill != nil || strings.Contains(text, "Uploading") {
+ t.Fatalf("BEACON_BACKFILL=0 still shipped a backfill: %+v\n%s", captured.Backfill, text)
+ }
+}
diff --git a/cli/beacon/cmd/endpoint_connect.go b/cli/beacon/cmd/endpoint_connect.go
index 46dc7320..14cb230b 100644
--- a/cli/beacon/cmd/endpoint_connect.go
+++ b/cli/beacon/cmd/endpoint_connect.go
@@ -73,6 +73,7 @@ func init() {
endpointConnectCmd.Flags().StringVar(&connectOpts.dashboardURL, "dashboard-url", "", "Beacon service URL (defaults to "+auth.DefaultDashboardURL+", or "+auth.DashboardURLEnv+")")
endpointConnectCmd.Flags().BoolVar(&connectOpts.noBrowser, "no-browser", false, "System mode: print the device approval URL instead of opening a browser")
endpointConnectCmd.Flags().StringVar(&connectOpts.vectorBin, "vector-bin", "", "Vector binary to run (defaults to "+asymptote.VectorBinEnv+", "+asymptote.PackagedVectorPath+", Homebrew, then PATH)")
+ endpointConnectCmd.Flags().BoolVar(&endpointOpts.noBackfill, "no-backfill", false, "On a first connect, do not upload the last 30 days of agent sessions read from their local session stores (also "+endpointBackfillEnv+"=0)")
endpointConnectCmd.Flags().StringVar(&connectOpts.privacyMode, "privacy-mode", "", "Beacon Cloud forwarding privacy: standard or metadata-only (defaults to onboarding choice)")
endpointDisconnectCmd.Flags().BoolVar(&connectOpts.keepCredentials, "keep-credentials", false, "Keep the enrollment record and device key so a later connect can reuse this device")
endpointCmd.AddCommand(endpointConnectCmd)
@@ -84,13 +85,14 @@ func runEndpointConnect(cmd *cobra.Command, args []string) error {
if !userMode && !lifecycle.HasSystemPrivileges() {
return fmt.Errorf("connecting a system endpoint needs root: rerun with sudo, or pass --user for a per-user install")
}
- return connectEndpoint(cmd, userMode, loadOrDefaultConfig().LogPath)
+ return connectEndpoint(cmd, userMode, loadOrDefaultConfig().LogPath, false)
}
// connectEndpoint runs the enrollment and forwarder setup for the given mode and log
// path, printing the outcome. `endpoint install --connect` and the onboarding offer
-// call it after a successful install; `endpoint connect` calls it directly.
-func connectEndpoint(cmd *cobra.Command, userMode bool, logPath string) error {
+// call it after a successful install; `endpoint connect` calls it directly. backfilled
+// says the caller has just run the session backfill, so connect only ships it.
+func connectEndpoint(cmd *cobra.Command, userMode bool, logPath string, backfilled bool) error {
hostname, _ := os.Hostname()
out := cmd.OutOrStdout()
if endpointOpts.jsonOutput {
@@ -105,6 +107,7 @@ func connectEndpoint(cmd *cobra.Command, userMode bool, logPath string) error {
LogPath: logPath,
VectorBin: connectOpts.vectorBin,
PrivacyMode: privacyMode,
+ Backfill: endpointConnectBackfill(out, userMode, logPath, backfilled),
Out: out,
Enroll: asymptote.EnrollOptions{
DashboardURL: connectOpts.dashboardURL,
@@ -160,9 +163,14 @@ func connectEndpoint(cmd *cobra.Command, userMode bool, logPath string) error {
fmt.Fprintf(out, "Device key: %s (never printed; the Vector forwarder reads it)\n", result.SecretsFile)
// Close on what to do next rather than on more state. Setup is finished here,
// and this is the moment the user has the most intent and the least idea what
- // happens now. Nothing is forwarded yet: the runtime source starts at the
- // connection point, so the dashboard stays empty until an agent actually runs.
+ // happens now. The runtime source starts at the connection point, so apart from
+ // the staged session backfill the dashboard fills as agents run.
fmt.Fprintln(out)
+ if result.Backfill != nil && result.Backfill.Events > 0 {
+ fmt.Fprintf(out, "Uploading %d events of recent session history; they appear at %s within a few minutes.\n", result.Backfill.Events, dashboardHomeURL(result.Enrollment.DashboardURL))
+ fmt.Fprintln(out, "Next: keep using your agents as you normally would; new sessions appear within a minute of the activity.")
+ return nil
+ }
fmt.Fprintln(out, "Next: run any supported agent as you normally would.")
fmt.Fprintf(out, "Its sessions appear at %s within a minute of the activity.\n", dashboardHomeURL(result.Enrollment.DashboardURL))
return nil
diff --git a/cli/beacon/cmd/endpoint_connect_test.go b/cli/beacon/cmd/endpoint_connect_test.go
index c3ea7fec..e7c2538c 100644
--- a/cli/beacon/cmd/endpoint_connect_test.go
+++ b/cli/beacon/cmd/endpoint_connect_test.go
@@ -113,7 +113,7 @@ func TestConnectUsesSignedInAccountAndSelectedPrivacy(t *testing.T) {
command := &cobra.Command{}
command.SetOut(&out)
command.SetErr(&out)
- if err := connectEndpoint(command, true, "/tmp/runtime.jsonl"); err != nil {
+ if err := connectEndpoint(command, true, "/tmp/runtime.jsonl", false); err != nil {
t.Fatalf("connectEndpoint returned error: %v", err)
}
if captured.AccountEnroll == nil || captured.AccountEnroll.AccessToken != "bcn_cli_secret" {
@@ -177,7 +177,7 @@ func TestConnectRequiresSignedInAccountInUserMode(t *testing.T) {
return nil, errors.New("must not run")
}
command := &cobra.Command{}
- err := connectEndpoint(command, true, "/tmp/runtime.jsonl")
+ err := connectEndpoint(command, true, "/tmp/runtime.jsonl", false)
if err == nil || !strings.Contains(err.Error(), "beacon login") {
t.Fatalf("error = %v", err)
}
diff --git a/cli/beacon/cmd/endpoint_install.go b/cli/beacon/cmd/endpoint_install.go
index 16629edb..8513581b 100644
--- a/cli/beacon/cmd/endpoint_install.go
+++ b/cli/beacon/cmd/endpoint_install.go
@@ -179,11 +179,19 @@ func runEndpointInstall(cmd *cobra.Command, args []string) error {
}
printLingerGap(cmd.ErrOrStderr(), result)
installHookTargetsFromEndpointInstall(cmd.ErrOrStderr(), selection.Hooks)
+ // Read recent history from the agents' own session stores so the dashboard has sessions
+ // in it before any agent runs again. Local only, bounded, and never a reason to fail the
+ // install; a connect below ships what it wrote rather than sweeping again.
+ backfilled := false
+ if endpointBackfillEnabled(endpointUserMode()) {
+ reportEndpointBackfill(cmd.OutOrStdout(), runEndpointBackfill(result.LogPath))
+ backfilled = true
+ }
if connectAfterInstall {
// The install is complete and stands on its own; a failed connect is reported
// with the retry command rather than turning a working install into an error.
fmt.Fprintln(cmd.OutOrStdout())
- if err := connectEndpoint(cmd, endpointUserMode(), result.LogPath); err != nil {
+ if err := connectEndpoint(cmd, endpointUserMode(), result.LogPath, backfilled); err != nil {
fmt.Fprintf(cmd.ErrOrStderr(), "beacon: managed ingest was not connected (%v). Run `beacon endpoint connect` to try again.\n", err)
} else {
recordDestinationAsymptote(cmd)
diff --git a/cli/beacon/cmd/endpoint_onboarding.go b/cli/beacon/cmd/endpoint_onboarding.go
index 8ad9f403..d3ef41e1 100644
--- a/cli/beacon/cmd/endpoint_onboarding.go
+++ b/cli/beacon/cmd/endpoint_onboarding.go
@@ -163,6 +163,7 @@ func runAccountOnboarding(cmd *cobra.Command, profile *onboarding.Profile, desti
DestinationOnly: destinationOnly,
PresetDestination: preset,
PresetPrivacyMode: profile.Onboarding.PrivacyMode,
+ BackfillDays: endpointBackfillDays(),
NoBrowser: endpointOpts.noBrowser,
NoDisplay: onboardingDisplayCheck(),
Now: onboardingClock,
diff --git a/cli/beacon/cmd/history_env_test.go b/cli/beacon/cmd/history_env_test.go
index a7605d5b..899cdbcf 100644
--- a/cli/beacon/cmd/history_env_test.go
+++ b/cli/beacon/cmd/history_env_test.go
@@ -10,7 +10,9 @@ import (
)
// Every test in this package runs with the local history pointed at a path that does not exist, so
-// a command under test never reads or writes a developer's real ~/.beacon/endpoint/history.db.
+// a command under test never reads or writes a developer's real ~/.beacon/endpoint/history.db, and
+// with the session backfill off, so an install under test never sweeps a real agent session store
+// that a runtime-specific environment variable points outside the test's home.
func TestMain(m *testing.M) {
dir, err := os.MkdirTemp("", "beacon-cmd-test")
if err != nil {
@@ -18,6 +20,7 @@ func TestMain(m *testing.M) {
os.Exit(1)
}
_ = os.Setenv(endpointconfig.HistoryStoreEnv, filepath.Join(dir, "absent", "history.db"))
+ _ = os.Setenv(endpointBackfillEnv, "0")
code := m.Run()
_ = os.RemoveAll(dir)
os.Exit(code)
diff --git a/cli/beacon/internal/claudesession/collect.go b/cli/beacon/internal/claudesession/collect.go
index ce069c1a..7c2dd4ba 100644
--- a/cli/beacon/internal/claudesession/collect.go
+++ b/cli/beacon/internal/claudesession/collect.go
@@ -8,9 +8,11 @@ import (
"os"
"path/filepath"
"strings"
+ "time"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/writer"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/sessionwindow"
)
const StateVersion = 1
@@ -40,6 +42,13 @@ type CollectOptions struct {
// collector prune archives past the shared count on their next rotation, and readers only
// look that far, so a sweep that kept more would lose them anyway.
RotateBytes int64
+ // Since, when set, limits the sweep to sessions modified at or after it, newest first. Older
+ // sessions are not read and their cursors do not move, so a later sync still collects them.
+ // The install-time backfill sets it; an ordinary sync leaves it zero.
+ Since time.Time
+ // Budget, when set, caps the bytes this sweep may append. The sweep stops at the first event
+ // that does not fit and returns writer.ErrBudgetSpent, with cursors past only what was written.
+ Budget *writer.Budget
}
type Summary struct {
@@ -134,6 +143,7 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
if err != nil {
return summary, err
}
+ refs = sessionwindow.Recent(refs, opts.Since, func(r SessionRef) int64 { return r.ModTimeUnixMS })
summary.Sessions = len(refs)
if len(refs) == 0 {
return summary, nil
@@ -167,6 +177,15 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
summary.EventsEmitted, summary.SessionsPending, collectErr))
break
}
+ if errors.Is(collectErr, writer.ErrBudgetSpent) {
+ // Not a failed session: this one and the rest are left for a later sync, with cursors
+ // that have not moved past anything that was not written.
+ if changed {
+ summary.SessionsChanged++
+ }
+ errs = append(errs, collectErr)
+ break
+ }
if collectErr != nil {
summary.Errors++
errs = append(errs, fmt.Errorf("Claude session %s: %w", ref.ID, collectErr))
@@ -278,6 +297,6 @@ func emit(event schema.Event, opts CollectOptions, guard *writer.RetentionGuard)
if !opts.Write {
return nil
}
- _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode, RotateSize: opts.RotateBytes, Guard: guard})
+ _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode, RotateSize: opts.RotateBytes, Guard: guard, Budget: opts.Budget})
return err
}
diff --git a/cli/beacon/internal/clinesession/collect.go b/cli/beacon/internal/clinesession/collect.go
index a7f581c6..7b68f513 100644
--- a/cli/beacon/internal/clinesession/collect.go
+++ b/cli/beacon/internal/clinesession/collect.go
@@ -10,9 +10,11 @@ import (
"os"
"path/filepath"
"strings"
+ "time"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/writer"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/sessionwindow"
)
const StateVersion = 1
@@ -99,6 +101,13 @@ type CollectOptions struct {
UserMode bool
Print bool
Out io.Writer
+ // Since, when set, limits the sweep to sessions modified at or after it, newest first. Older
+ // sessions are not read and their cursors do not move, so a later sync still collects them.
+ // The install-time backfill sets it; an ordinary sync leaves it zero.
+ Since time.Time
+ // Budget, when set, caps the bytes this sweep may append. The sweep stops at the first event
+ // that does not fit and returns writer.ErrBudgetSpent, with cursors past only what was written.
+ Budget *writer.Budget
}
type Summary struct {
@@ -117,6 +126,7 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
if err != nil {
return summary, err
}
+ refs = sessionwindow.Recent(refs, opts.Since, func(r TraceRef) int64 { return r.UpdatedAtUnixMS })
summary.Traces = len(refs)
state, err := LoadState(opts.StatePath)
if err != nil {
@@ -131,6 +141,15 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
var errs []error
for _, ref := range refs {
changed, collectErr := collectTrace(store, ref, state, opts, &summary)
+ if errors.Is(collectErr, writer.ErrBudgetSpent) {
+ // Not a failed session: this one and the rest are left for a later sync, with cursors
+ // that have not moved past anything that was not written.
+ if changed {
+ summary.TracesChanged++
+ }
+ errs = append(errs, collectErr)
+ break
+ }
if collectErr != nil {
summary.Errors++
errs = append(errs, fmt.Errorf("Cline trace %s: %w", ref.ID, collectErr))
@@ -235,7 +254,7 @@ func emit(event schema.Event, opts CollectOptions) error {
}
}
if opts.Write {
- if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode}); err != nil {
+ if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode, Budget: opts.Budget}); err != nil {
return err
}
}
diff --git a/cli/beacon/internal/codexsession/backfill_test.go b/cli/beacon/internal/codexsession/backfill_test.go
new file mode 100644
index 00000000..f82f467e
--- /dev/null
+++ b/cli/beacon/internal/codexsession/backfill_test.go
@@ -0,0 +1,107 @@
+package codexsession
+
+import (
+ "errors"
+ "os"
+ "path/filepath"
+ "testing"
+ "time"
+
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/writer"
+)
+
+// writeDatedSession writes a one-prompt rollout for id and sets its modification time to at.
+func writeDatedSession(t *testing.T, codexDir, id string, at time.Time) {
+ t.Helper()
+ path := filepath.Join(codexDir, "sessions", "2026", "09", "19", "rollout-2026-09-19T20-00-00-"+id+".jsonl")
+ writeFile(t, path, sessionMetaLine(id, "/tmp/repo")+"\n"+messageLine("user", id+"-u1", "prompt for "+id)+"\n")
+ if err := os.Chtimes(path, at, at); err != nil {
+ t.Fatal(err)
+ }
+}
+
+func sessionIDs(t *testing.T, logPath string) []string {
+ t.Helper()
+ var ids []string
+ seen := map[string]bool{}
+ for _, ev := range readLog(t, logPath) {
+ if ev.Session == nil || seen[ev.Session.ID] {
+ continue
+ }
+ seen[ev.Session.ID] = true
+ ids = append(ids, ev.Session.ID)
+ }
+ return ids
+}
+
+func TestCollectOnceSinceReadsRecentSessionsNewestFirstAndLeavesOlderOnes(t *testing.T) {
+ dir := t.TempDir()
+ codexDir := filepath.Join(dir, ".codex")
+ now := time.Now()
+ writeDatedSession(t, codexDir, "old", now.Add(-60*24*time.Hour))
+ writeDatedSession(t, codexDir, "recent", now.Add(-2*time.Hour))
+ writeDatedSession(t, codexDir, "newest", now.Add(-time.Hour))
+ statePath := filepath.Join(dir, "state", "codex.json")
+ logPath := filepath.Join(dir, "runtime.jsonl")
+
+ opts := CollectOptions{CodexDir: codexDir, StatePath: statePath, LogPath: logPath, Write: true, UserMode: true, Since: now.Add(-30 * 24 * time.Hour)}
+ if _, err := CollectOnce(opts); err != nil {
+ t.Fatalf("windowed CollectOnce: %v", err)
+ }
+ if got := sessionIDs(t, logPath); len(got) != 2 || got[0] != "newest" || got[1] != "recent" {
+ t.Fatalf("windowed sweep read sessions %v, want [newest recent]", got)
+ }
+
+ // The session outside the window was never consumed: an ordinary sync still reads it.
+ if _, err := CollectOnce(CollectOptions{CodexDir: codexDir, StatePath: statePath, LogPath: logPath, Write: true, UserMode: true}); err != nil {
+ t.Fatalf("ordinary CollectOnce: %v", err)
+ }
+ if got := sessionIDs(t, logPath); len(got) != 3 || got[2] != "old" {
+ t.Fatalf("ordinary sweep after the window read %v, want old last", got)
+ }
+}
+
+func TestCollectOnceStopsAtTheBudgetAndLeavesTheRestForTheNextSweep(t *testing.T) {
+ dir := t.TempDir()
+ codexDir := filepath.Join(dir, ".codex")
+ now := time.Now()
+ writeDatedSession(t, codexDir, "first", now.Add(-time.Hour))
+ writeDatedSession(t, codexDir, "second", now.Add(-2*time.Hour))
+ statePath := filepath.Join(dir, "state", "codex.json")
+ logPath := filepath.Join(dir, "runtime.jsonl")
+
+ // Learn how many bytes the newest session takes, then give a fresh sweep exactly that.
+ probeDir := t.TempDir()
+ probe := writer.NewBudget(1 << 30)
+ writeDatedSession(t, filepath.Join(probeDir, ".codex"), "first", now.Add(-time.Hour))
+ if _, err := CollectOnce(CollectOptions{CodexDir: filepath.Join(probeDir, ".codex"), StatePath: filepath.Join(probeDir, "state.json"), LogPath: filepath.Join(probeDir, "runtime.jsonl"), Write: true, UserMode: true, Since: now.Add(-24 * time.Hour), Budget: probe}); err != nil {
+ t.Fatal(err)
+ }
+
+ budget := writer.NewBudget(probe.Used())
+ summary, err := CollectOnce(CollectOptions{CodexDir: codexDir, StatePath: statePath, LogPath: logPath, Write: true, UserMode: true, Since: now.Add(-24 * time.Hour), Budget: budget})
+ if !errors.Is(err, writer.ErrBudgetSpent) {
+ t.Fatalf("budgeted CollectOnce = %v, want ErrBudgetSpent", err)
+ }
+ if summary.Errors != 0 {
+ t.Fatalf("a spent budget counted as %d session errors", summary.Errors)
+ }
+ if got := sessionIDs(t, logPath); len(got) != 1 || got[0] != "first" {
+ t.Fatalf("budgeted sweep read %v, want only the newest session", got)
+ }
+
+ if _, err := CollectOnce(CollectOptions{CodexDir: codexDir, StatePath: statePath, LogPath: logPath, Write: true, UserMode: true}); err != nil {
+ t.Fatalf("follow-up CollectOnce: %v", err)
+ }
+ events := readLog(t, logPath)
+ if got := sessionIDs(t, logPath); len(got) != 2 || got[1] != "second" {
+ t.Fatalf("follow-up sweep read %v, want second added", got)
+ }
+ perSession := map[string]int{}
+ for _, ev := range events {
+ perSession[ev.Session.ID]++
+ }
+ if perSession["first"] != perSession["second"] {
+ t.Fatalf("a session was written twice or partly: %v", perSession)
+ }
+}
diff --git a/cli/beacon/internal/codexsession/collect.go b/cli/beacon/internal/codexsession/collect.go
index 1472b313..e1aa8687 100644
--- a/cli/beacon/internal/codexsession/collect.go
+++ b/cli/beacon/internal/codexsession/collect.go
@@ -8,9 +8,11 @@ import (
"os"
"path/filepath"
"strings"
+ "time"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/writer"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/sessionwindow"
)
const StateVersion = 1
@@ -35,6 +37,13 @@ type CollectOptions struct {
UserMode bool
Print bool
Out io.Writer
+ // Since, when set, limits the sweep to sessions modified at or after it, newest first. Older
+ // sessions are not read and their cursors do not move, so a later sync still collects them.
+ // The install-time backfill sets it; an ordinary sync leaves it zero.
+ Since time.Time
+ // Budget, when set, caps the bytes this sweep may append. The sweep stops at the first event
+ // that does not fit and returns writer.ErrBudgetSpent, with cursors past only what was written.
+ Budget *writer.Budget
}
type Summary struct {
@@ -117,6 +126,7 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
if err != nil {
return summary, err
}
+ refs = sessionwindow.Recent(refs, opts.Since, func(r SessionRef) int64 { return r.ModTimeUnixMS })
summary.Sessions = len(refs)
if len(refs) == 0 {
return summary, nil
@@ -134,6 +144,15 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
var errs []error
for _, ref := range refs {
changed, collectErr := collectSession(store, ref, state, opts, &summary)
+ if errors.Is(collectErr, writer.ErrBudgetSpent) {
+ // Not a failed session: this one and the rest are left for a later sync, with cursors
+ // that have not moved past anything that was not written.
+ if changed {
+ summary.SessionsChanged++
+ }
+ errs = append(errs, collectErr)
+ break
+ }
if collectErr != nil {
summary.Errors++
errs = append(errs, fmt.Errorf("Codex session %s: %w", ref.ID, collectErr))
@@ -205,7 +224,7 @@ func emit(event schema.Event, opts CollectOptions) error {
if !opts.Write {
return nil
}
- _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode})
+ _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode, Budget: opts.Budget})
return err
}
diff --git a/cli/beacon/internal/copilotsession/collect.go b/cli/beacon/internal/copilotsession/collect.go
index 917afd58..6da931b2 100644
--- a/cli/beacon/internal/copilotsession/collect.go
+++ b/cli/beacon/internal/copilotsession/collect.go
@@ -8,9 +8,11 @@ import (
"os"
"path/filepath"
"strings"
+ "time"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/writer"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/sessionwindow"
)
type Cursor struct {
@@ -36,6 +38,13 @@ type CollectOptions struct {
Out io.Writer
Workspace string
SessionID string
+ // Since, when set, limits the sweep to sessions modified at or after it, newest first. Older
+ // sessions are not read and their cursors do not move, so a later sync still collects them.
+ // The install-time backfill sets it; an ordinary sync leaves it zero.
+ Since time.Time
+ // Budget, when set, caps the bytes this sweep may append. The sweep stops at the first event
+ // that does not fit and returns writer.ErrBudgetSpent, with cursors past only what was written.
+ Budget *writer.Budget
}
type Summary struct {
@@ -123,6 +132,7 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
return summary, err
}
refs = FilterRefs(refs, opts)
+ refs = sessionwindow.Recent(refs, opts.Since, func(r SessionRef) int64 { return r.ModTimeUnixMS })
summary.Sessions = len(refs)
if len(refs) == 0 {
return summary, nil
@@ -140,6 +150,15 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
var errs []error
for _, ref := range refs {
changed, collectErr := collectSession(store, ref, state, opts, &summary)
+ if errors.Is(collectErr, writer.ErrBudgetSpent) {
+ // Not a failed session: this one and the rest are left for a later sync, with cursors
+ // that have not moved past anything that was not written.
+ if changed {
+ summary.SessionsChanged++
+ }
+ errs = append(errs, collectErr)
+ break
+ }
if collectErr != nil {
summary.Errors++
errs = append(errs, fmt.Errorf("GitHub Copilot CLI session %s: %w", ref.ID, collectErr))
@@ -233,7 +252,7 @@ func emit(event schema.Event, opts CollectOptions) error {
}
}
if opts.Write {
- if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode}); err != nil {
+ if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode, Budget: opts.Budget}); err != nil {
return err
}
}
diff --git a/cli/beacon/internal/cursorsession/collect.go b/cli/beacon/internal/cursorsession/collect.go
index 31bf73ad..ae73bac8 100644
--- a/cli/beacon/internal/cursorsession/collect.go
+++ b/cli/beacon/internal/cursorsession/collect.go
@@ -8,9 +8,11 @@ import (
"os"
"path/filepath"
"strings"
+ "time"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/writer"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/sessionwindow"
)
const StateVersion = 1
@@ -100,6 +102,13 @@ type CollectOptions struct {
UserMode bool
Print bool
Out io.Writer
+ // Since, when set, limits the sweep to sessions modified at or after it, newest first. Older
+ // sessions are not read and their cursors do not move, so a later sync still collects them.
+ // The install-time backfill sets it; an ordinary sync leaves it zero.
+ Since time.Time
+ // Budget, when set, caps the bytes this sweep may append. The sweep stops at the first event
+ // that does not fit and returns writer.ErrBudgetSpent, with cursors past only what was written.
+ Budget *writer.Budget
}
type Summary struct {
@@ -119,6 +128,7 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
if listErr != nil && len(refs) == 0 {
return summary, listErr
}
+ refs = sessionwindow.Recent(refs, opts.Since, func(r TraceRef) int64 { return r.UpdatedAtUnixMS })
summary.Traces = len(refs)
state, err := LoadState(opts.StatePath)
if err != nil {
@@ -137,6 +147,15 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
}
for _, ref := range refs {
changed, collectErr := collectTrace(store, ref, state, opts, &summary)
+ if errors.Is(collectErr, writer.ErrBudgetSpent) {
+ // Not a failed session: this one and the rest are left for a later sync, with cursors
+ // that have not moved past anything that was not written.
+ if changed {
+ summary.TracesChanged++
+ }
+ errs = append(errs, collectErr)
+ break
+ }
if collectErr != nil {
summary.Errors++
errs = append(errs, fmt.Errorf("Cursor trace %s: %w", ref.ID, collectErr))
@@ -213,7 +232,7 @@ func emit(event schema.Event, opts CollectOptions) error {
}
}
if opts.Write {
- if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode}); err != nil {
+ if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode, Budget: opts.Budget}); err != nil {
return err
}
}
diff --git a/cli/beacon/internal/dshsession/collect.go b/cli/beacon/internal/dshsession/collect.go
index 23dcc48d..f69b1d77 100644
--- a/cli/beacon/internal/dshsession/collect.go
+++ b/cli/beacon/internal/dshsession/collect.go
@@ -8,9 +8,11 @@ import (
"os"
"path/filepath"
"strings"
+ "time"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/writer"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/sessionwindow"
)
type Cursor struct {
@@ -38,6 +40,13 @@ type CollectOptions struct {
Out io.Writer
Workspace string
SessionID string
+ // Since, when set, limits the sweep to sessions modified at or after it, newest first. Older
+ // sessions are not read and their cursors do not move, so a later sync still collects them.
+ // The install-time backfill sets it; an ordinary sync leaves it zero.
+ Since time.Time
+ // Budget, when set, caps the bytes this sweep may append. The sweep stops at the first event
+ // that does not fit and returns writer.ErrBudgetSpent, with cursors past only what was written.
+ Budget *writer.Budget
}
type Summary struct {
@@ -121,6 +130,7 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
return summary, err
}
refs = filterRefs(refs, opts)
+ refs = sessionwindow.Recent(refs, opts.Since, func(r SessionRef) int64 { return r.ModTimeUnixMS })
summary.Sessions = len(refs)
if len(refs) == 0 {
return summary, nil
@@ -137,6 +147,15 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
var errs []error
for _, ref := range refs {
changed, collectErr := collectSession(store, ref, state, opts, &summary)
+ if errors.Is(collectErr, writer.ErrBudgetSpent) {
+ // Not a failed session: this one and the rest are left for a later sync, with cursors
+ // that have not moved past anything that was not written.
+ if changed {
+ summary.SessionsChanged++
+ }
+ errs = append(errs, collectErr)
+ break
+ }
if collectErr != nil {
summary.Errors++
errs = append(errs, fmt.Errorf("DeepSeek session %s: %w", ref.ID, collectErr))
@@ -241,7 +260,7 @@ func emit(event schema.Event, opts CollectOptions) error {
}
}
if opts.Write {
- if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode}); err != nil {
+ if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode, Budget: opts.Budget}); err != nil {
return err
}
}
diff --git a/cli/beacon/internal/endpoint/asymptote/asymptote.go b/cli/beacon/internal/endpoint/asymptote/asymptote.go
index 8d4209b1..12983321 100644
--- a/cli/beacon/internal/endpoint/asymptote/asymptote.go
+++ b/cli/beacon/internal/endpoint/asymptote/asymptote.go
@@ -142,7 +142,7 @@ func RenderVectorConfig(opts RenderOptions) (string, error) {
}
content = strings.Replace(content, privacyTransformMarker, strings.TrimSpace(string(transforms)), 1)
content = strings.Replace(content,
- "[sinks.asymptote_runtime]\ntype = \"http\"\ninputs = [\"beacon_runtime\"]",
+ "[sinks.asymptote_runtime]\ntype = \"http\"\ninputs = [\"beacon_runtime\", \"beacon_backfill\"]",
"[sinks.asymptote_runtime]\ntype = \"http\"\ninputs = [\"beacon_runtime_metadata\"]", 1)
content = strings.Replace(content,
"[sinks.asymptote_inventory]\ntype = \"http\"\ninputs = [\"beacon_inventory\"]",
diff --git a/cli/beacon/internal/endpoint/asymptote/backfill.go b/cli/beacon/internal/endpoint/asymptote/backfill.go
new file mode 100644
index 00000000..1f3b7ba7
--- /dev/null
+++ b/cli/beacon/internal/endpoint/asymptote/backfill.go
@@ -0,0 +1,147 @@
+package asymptote
+
+import (
+ "bufio"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "os"
+ "path/filepath"
+ "time"
+
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/siempack"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/writer"
+)
+
+// BackfillLogName is the file a first connect stages the session backfill in, beside the runtime
+// log.
+const BackfillLogName = siempack.BackfillLogName
+
+// BackfillLogPath is where the forwarder's backfill source reads, derived from the runtime log so
+// a hand-run pack and the managed forwarder agree on it.
+func BackfillLogPath(logPath string) string { return siempack.BackfillLogPath(logPath) }
+
+// BackfillOptions bounds what a first connect ships from before the connection point.
+type BackfillOptions struct {
+ // Since drops events older than it.
+ Since time.Time
+ // MaxBytes keeps the newest events that fit; older ones are dropped.
+ MaxBytes int64
+}
+
+// BackfillResult reports what StageBackfill wrote.
+type BackfillResult struct {
+ Path string `json:"path"`
+ Events int `json:"events"`
+ Bytes int64 `json:"bytes"`
+ // Trimmed is set when older events in the window were left out to stay under MaxBytes.
+ Trimmed bool `json:"trimmed,omitempty"`
+}
+
+// StageBackfill copies the session-store backfill already in the runtime log into the file the
+// forwarder's backfill source reads from the beginning.
+//
+// The runtime source starts at the connection point, so history recorded before it is never
+// shipped. The one exception is the bounded session backfill, which is what makes the Beacon Cloud
+// dashboard useful the moment an endpoint connects: events a session-store sync wrote
+// (harness.collection_method=poll) at or after Since, newest kept first up to MaxBytes. Live hook,
+// OTLP and plugin events from before the connection stay local. The lines are copied byte for
+// byte, so the forwarder's privacy transforms apply to them exactly as to live lines.
+//
+// It runs before the forwarder starts and only on a first connect: a re-connect resumes from
+// Vector's checkpoints and would otherwise ship the same history twice. The file is replaced
+// atomically, 0600, so a reader never sees half of it.
+func StageBackfill(logPath string, opts BackfillOptions) (BackfillResult, error) {
+ result := BackfillResult{Path: BackfillLogPath(logPath)}
+ if logPath == "" {
+ return result, errors.New("runtime log path is required")
+ }
+ paths := writer.RetainedLogPaths(logPath)
+ var lines [][]byte
+ var total int64
+ // Oldest file first so the staged lines keep the log's order.
+ for i := len(paths) - 1; i >= 0; i-- {
+ err := scanBackfillLines(paths[i], opts.Since, func(line []byte) {
+ lines = append(lines, line)
+ total += int64(len(line)) + 1
+ })
+ if err != nil {
+ return result, err
+ }
+ }
+ if opts.MaxBytes > 0 {
+ drop := 0
+ for total > opts.MaxBytes && drop < len(lines) {
+ total -= int64(len(lines[drop])) + 1
+ drop++
+ }
+ if drop > 0 {
+ result.Trimmed = true
+ lines = lines[drop:]
+ }
+ }
+ if len(lines) == 0 {
+ return result, nil
+ }
+ if err := os.MkdirAll(filepath.Dir(result.Path), 0o755); err != nil {
+ return result, err
+ }
+ data := make([]byte, 0, total)
+ for _, line := range lines {
+ data = append(append(data, line...), '\n')
+ }
+ if err := writeFileAtomic(result.Path, data, 0o600); err != nil {
+ return result, fmt.Errorf("stage the session backfill: %w", err)
+ }
+ result.Events = len(lines)
+ result.Bytes = int64(len(data))
+ return result, nil
+}
+
+// RemoveBackfill deletes the staged backfill file. Disconnect calls it so a later first connect
+// stages a fresh one rather than shipping a stale copy.
+func RemoveBackfill(logPath string) error {
+ if logPath == "" {
+ return nil
+ }
+ if err := os.Remove(BackfillLogPath(logPath)); err != nil && !os.IsNotExist(err) {
+ return err
+ }
+ return nil
+}
+
+type backfillLine struct {
+ Timestamp string `json:"timestamp"`
+ Harness struct {
+ CollectionMethod string `json:"collection_method"`
+ } `json:"harness"`
+}
+
+func scanBackfillLines(path string, since time.Time, keep func([]byte)) error {
+ f, err := os.Open(path)
+ if err != nil {
+ if os.IsNotExist(err) {
+ return nil
+ }
+ return err
+ }
+ defer f.Close()
+ scanner := bufio.NewScanner(f)
+ scanner.Buffer(make([]byte, 0, 64*1024), writer.MaxEventBytes+1024)
+ for scanner.Scan() {
+ raw := scanner.Bytes()
+ var line backfillLine
+ if json.Unmarshal(raw, &line) != nil || line.Harness.CollectionMethod != schema.CollectionMethodPoll {
+ continue
+ }
+ if !since.IsZero() {
+ ts, err := schema.ParseTimestamp(line.Timestamp)
+ if err != nil || ts.Before(since) {
+ continue
+ }
+ }
+ keep(append([]byte(nil), raw...))
+ }
+ return scanner.Err()
+}
diff --git a/cli/beacon/internal/endpoint/asymptote/backfill_test.go b/cli/beacon/internal/endpoint/asymptote/backfill_test.go
new file mode 100644
index 00000000..e1887a2e
--- /dev/null
+++ b/cli/beacon/internal/endpoint/asymptote/backfill_test.go
@@ -0,0 +1,237 @@
+package asymptote
+
+import (
+ "context"
+ "errors"
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/testenv"
+)
+
+func backfillTestLine(ts, method, id string) string {
+ return `{"timestamp":"` + ts + `","vendor":"beacon","event":{"action":"prompt.submitted","id":"` + id + `"},"harness":{"name":"codex","collection_method":"` + method + `"}}`
+}
+
+func writeLines(t *testing.T, path string, lines ...string) {
+ t.Helper()
+ if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
+ t.Fatal(err)
+ }
+ if err := os.WriteFile(path, []byte(strings.Join(lines, "\n")+"\n"), 0o600); err != nil {
+ t.Fatal(err)
+ }
+}
+
+func stagedIDs(t *testing.T, path string) []string {
+ t.Helper()
+ data, err := os.ReadFile(path)
+ if err != nil {
+ t.Fatal(err)
+ }
+ var ids []string
+ for _, line := range strings.Split(strings.TrimSpace(string(data)), "\n") {
+ start := strings.Index(line, `"id":"`) + len(`"id":"`)
+ ids = append(ids, line[start:start+strings.Index(line[start:], `"`)])
+ }
+ return ids
+}
+
+func TestStageBackfillCopiesRecentPollLinesOldestFileFirst(t *testing.T) {
+ logPath := filepath.Join(t.TempDir(), "runtime.jsonl")
+ writeLines(t, logPath+".1",
+ backfillTestLine("2026-09-01T00:00:00Z", "poll", "too-old"),
+ backfillTestLine("2026-09-20T00:00:00Z", "poll", "archived"),
+ )
+ writeLines(t, logPath,
+ backfillTestLine("2026-09-21T00:00:00Z", "hook", "live-hook"),
+ "not json",
+ backfillTestLine("2026-09-22T00:00:00Z", "poll", "recent"),
+ )
+
+ result, err := StageBackfill(logPath, BackfillOptions{Since: time.Date(2026, 9, 10, 0, 0, 0, 0, time.UTC)})
+ if err != nil {
+ t.Fatal(err)
+ }
+ if result.Path != filepath.Join(filepath.Dir(logPath), BackfillLogName) || result.Events != 2 || result.Trimmed {
+ t.Fatalf("result = %+v", result)
+ }
+ if got := stagedIDs(t, result.Path); strings.Join(got, ",") != "archived,recent" {
+ t.Fatalf("staged %v, want [archived recent]", got)
+ }
+ if testenv.HasPOSIXFileModes() {
+ if info, err := os.Stat(result.Path); err != nil || info.Mode().Perm() != 0o600 {
+ t.Fatalf("staged file mode: %v %v", info, err)
+ }
+ }
+}
+
+func TestStageBackfillKeepsTheNewestLinesThatFit(t *testing.T) {
+ logPath := filepath.Join(t.TempDir(), "runtime.jsonl")
+ older := backfillTestLine("2026-09-20T00:00:00Z", "poll", "older")
+ newer := backfillTestLine("2026-09-21T00:00:00Z", "poll", "newer")
+ writeLines(t, logPath, older, newer)
+
+ result, err := StageBackfill(logPath, BackfillOptions{MaxBytes: int64(len(newer) + 1)})
+ if err != nil {
+ t.Fatal(err)
+ }
+ if !result.Trimmed || result.Events != 1 || result.Bytes != int64(len(newer)+1) {
+ t.Fatalf("result = %+v", result)
+ }
+ if got := stagedIDs(t, result.Path); strings.Join(got, ",") != "newer" {
+ t.Fatalf("staged %v, want [newer]", got)
+ }
+}
+
+func TestStageBackfillWritesNothingWithoutBackfill(t *testing.T) {
+ logPath := filepath.Join(t.TempDir(), "runtime.jsonl")
+ writeLines(t, logPath, backfillTestLine("2026-09-21T00:00:00Z", "otlp", "live"))
+ result, err := StageBackfill(logPath, BackfillOptions{})
+ if err != nil {
+ t.Fatal(err)
+ }
+ if result.Events != 0 {
+ t.Fatalf("result = %+v", result)
+ }
+ if _, err := os.Stat(result.Path); !os.IsNotExist(err) {
+ t.Fatalf("an empty backfill left a file behind: %v", err)
+ }
+}
+
+func TestVectorConfigReadsTheStagedBackfillFromTheBeginningInEveryPrivacyMode(t *testing.T) {
+ logPath := "/tmp/beacon/runtime.jsonl"
+ for _, mode := range []string{"standard", "metadata-only"} {
+ got, err := RenderVectorConfig(RenderOptions{
+ LogPath: logPath,
+ IngestURL: "https://ingest.example.test",
+ SecretsFile: "/tmp/secrets.json",
+ DataDir: "/tmp/vector-data",
+ PrivacyMode: mode,
+ })
+ if err != nil {
+ t.Fatal(err)
+ }
+ source := "[sources.beacon_backfill]\ntype = \"file\"\ninclude = [\"/tmp/beacon/" + BackfillLogName + "\"]\nread_from = \"beginning\""
+ if !strings.Contains(got, source) {
+ t.Fatalf("%s config has no backfill source:\n%s", mode, got)
+ }
+ want := `inputs = ["beacon_runtime", "beacon_backfill"]`
+ if mode == "metadata-only" {
+ // The backfill goes through the same transform as live lines, never around it.
+ if !strings.Contains(got, "[transforms.beacon_runtime_metadata]\ntype = \"remap\"\n"+want) ||
+ strings.Count(got, `"beacon_backfill"`) != 1 {
+ t.Fatalf("metadata-only config lets the backfill bypass the transform:\n%s", got)
+ }
+ continue
+ }
+ if !strings.Contains(got, "[sinks.asymptote_runtime]\ntype = \"http\"\n"+want) {
+ t.Fatalf("standard config does not ship the backfill:\n%s", got)
+ }
+ }
+ hand, err := VectorConfig(logPath)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if strings.Contains(hand, "{{") || !strings.Contains(hand, "/tmp/beacon/"+BackfillLogName) {
+ t.Fatalf("hand-run config does not resolve the backfill path:\n%s", hand)
+ }
+}
+
+func TestFirstConnectStagesTheBackfillBeforeTheForwarderStartsAndReconnectDoesNot(t *testing.T) {
+ testenv.SetHome(t, t.TempDir())
+ isolateVectorDiscovery(t)
+ fd := newFakeDashboard(t)
+ vector := fakeVector(t, "0.56.0", 0)
+ fwd := &fakeForwarder{supported: true}
+ opts := connectOptions(t, fd, fwd, vector)
+ staged := BackfillLogPath(opts.LogPath)
+
+ synced := 0
+ opts.Backfill = &ConnectBackfill{
+ Sync: func() error {
+ synced++
+ // The sweep runs before any key exists.
+ if _, err := ReadDeviceKey(true); err == nil {
+ t.Error("backfill sync ran after the device key was stored")
+ }
+ writeLines(t, opts.LogPath, backfillTestLine(time.Now().UTC().Format(time.RFC3339), "poll", "fresh"))
+ return errors.New("one runtime could not be read")
+ },
+ }
+ var loadsAtStage int
+ fwd.onLoad = func() {
+ if _, err := os.Stat(staged); err == nil {
+ loadsAtStage++
+ }
+ }
+ result, err := Connect(context.Background(), opts)
+ if err != nil {
+ t.Fatalf("a failing backfill sync failed the connect: %v", err)
+ }
+ if synced != 1 || result.Backfill == nil || result.Backfill.Events != 1 || loadsAtStage != 1 {
+ t.Fatalf("synced=%d backfill=%+v staged-before-load=%d", synced, result.Backfill, loadsAtStage)
+ }
+
+ // A re-connect resumes from the forwarder's checkpoints: no sweep, no new stage.
+ if err := os.Remove(staged); err != nil {
+ t.Fatal(err)
+ }
+ again, err := Connect(context.Background(), opts)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if synced != 1 || again.Backfill != nil {
+ t.Fatalf("re-connect synced=%d backfill=%+v", synced, again.Backfill)
+ }
+ if _, err := os.Stat(staged); !os.IsNotExist(err) {
+ t.Fatal("re-connect staged the backfill again")
+ }
+}
+
+func TestPrivacyModeChangeDropsTheStagedBackfillWithTheCheckpoints(t *testing.T) {
+ testenv.SetHome(t, t.TempDir())
+ isolateVectorDiscovery(t)
+ fd := newFakeDashboard(t)
+ vector := fakeVector(t, "0.56.0", 0)
+ fwd := &fakeForwarder{supported: true}
+ opts := connectOptions(t, fd, fwd, vector)
+ writeLines(t, opts.LogPath, backfillTestLine(time.Now().UTC().Format(time.RFC3339), "poll", "fresh"))
+ opts.Backfill = &ConnectBackfill{}
+ if _, err := Connect(context.Background(), opts); err != nil {
+ t.Fatal(err)
+ }
+ staged := BackfillLogPath(opts.LogPath)
+ if _, err := os.Stat(staged); err != nil {
+ t.Fatalf("first connect did not stage: %v", err)
+ }
+ opts.PrivacyMode = "metadata-only"
+ if _, err := Connect(context.Background(), opts); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := os.Stat(staged); !os.IsNotExist(err) {
+ t.Fatal("a privacy mode change kept a staged backfill whose checkpoint it cleared")
+ }
+}
+
+func TestDisconnectRemovesTheStagedBackfill(t *testing.T) {
+ testenv.SetHome(t, t.TempDir())
+ isolateVectorDiscovery(t)
+ fd := newFakeDashboard(t)
+ fwd := &fakeForwarder{supported: true}
+ opts := connectOptions(t, fd, fwd, fakeVector(t, "0.56.0", 0))
+ writeLines(t, opts.LogPath, backfillTestLine(time.Now().UTC().Format(time.RFC3339), "poll", "fresh"))
+ opts.Backfill = &ConnectBackfill{}
+ if _, err := Connect(context.Background(), opts); err != nil {
+ t.Fatal(err)
+ }
+ if err := Disconnect(DisconnectOptions{UserMode: true, LogPath: opts.LogPath, Forwarder: fwd}); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := os.Stat(BackfillLogPath(opts.LogPath)); !os.IsNotExist(err) {
+ t.Fatal("disconnect left the staged backfill behind")
+ }
+}
diff --git a/cli/beacon/internal/endpoint/asymptote/connect.go b/cli/beacon/internal/endpoint/asymptote/connect.go
index 9aec47ac..cfbc0a96 100644
--- a/cli/beacon/internal/endpoint/asymptote/connect.go
+++ b/cli/beacon/internal/endpoint/asymptote/connect.go
@@ -49,11 +49,25 @@ type ConnectOptions struct {
InstallID string
// PrivacyMode controls the Vector-side transform before managed upload.
PrivacyMode string
- Out io.Writer
+ // Backfill, when set, ships recent session history on a first connect. Nil ships nothing
+ // recorded before the connection point.
+ Backfill *ConnectBackfill
+ Out io.Writer
// Now is injectable for tests.
Now func() time.Time
}
+// ConnectBackfill is the bounded session backfill a first connect ships. Neither step can fail
+// the connect: a backfill that could not be staged leaves a working forwarder, and says so.
+type ConnectBackfill struct {
+ // Sync reads the agents' session stores into the runtime log, so the backfill covers what
+ // they wrote since install. It runs before enrollment, so it never delays a step that holds a
+ // freshly issued key. Nil skips it.
+ Sync func() error
+ // Options bounds what StageBackfill copies.
+ Options BackfillOptions
+}
+
// ConnectResult reports what Connect wrote and where telemetry now goes.
type ConnectResult struct {
Enrollment Enrollment `json:"enrollment"`
@@ -63,6 +77,8 @@ type ConnectResult struct {
Forwarder string `json:"forwarder"`
ReEnrolled bool `json:"re_enrolled"`
ForwarderState service.Status `json:"forwarder_state"`
+ // Backfill reports the session history staged for upload; nil when none was.
+ Backfill *BackfillResult `json:"backfill,omitempty"`
}
// Connect enrolls this machine and starts the forwarder.
@@ -167,6 +183,14 @@ func Connect(ctx context.Context, opts ConnectOptions) (_ *ConnectResult, err er
return nil, err
}
+ // The session-store sweep is local and can take a while on a machine with a lot of history,
+ // so it runs here, after every check that could refuse the connect and before a key exists.
+ if opts.Backfill != nil && opts.Backfill.Sync != nil && !reconnect {
+ if err := opts.Backfill.Sync(); err != nil {
+ fmt.Fprintf(out, "Session backfill: %v\n", err)
+ }
+ }
+
// Pin the id before the dashboard learns it, so a failure after approval retries as
// the same device.
if err := WriteInstallID(opts.UserMode, installID); err != nil {
@@ -252,6 +276,23 @@ func Connect(ctx context.Context, opts ConnectOptions) (_ *ConnectResult, err er
if err := os.MkdirAll(dataDir, 0o700); err != nil {
return nil, err
}
+ // Clearing the checkpoints would have the backfill source read the file staged by
+ // the first connect from the beginning again, shipping that history twice.
+ if err := RemoveBackfill(logPath); err != nil {
+ return nil, fmt.Errorf("could not remove the staged session backfill: %w", err)
+ }
+ }
+ }
+ // Staged after the config is written and before the forwarder starts, so the runtime source's
+ // connection point falls after every line staged here and nothing is shipped twice. Only a
+ // first connect stages: a re-connect resumes from the forwarder's checkpoints.
+ var backfill *BackfillResult
+ if opts.Backfill != nil && !reconnect {
+ staged, err := StageBackfill(logPath, opts.Backfill.Options)
+ if err != nil {
+ fmt.Fprintf(out, "Session backfill was not staged: %v\n", err)
+ } else {
+ backfill = &staged
}
}
if err := writeFileAtomic(configPath, []byte(rendered), 0o644); err != nil {
@@ -310,6 +351,7 @@ func Connect(ctx context.Context, opts ConnectOptions) (_ *ConnectResult, err er
Forwarder: manager.Label(),
ReEnrolled: previous != nil,
ForwarderState: manager.Status(),
+ Backfill: backfill,
}, nil
}
@@ -352,6 +394,9 @@ func Disconnect(opts DisconnectOptions) error {
}
manager.RemoveUnits()
}
+ if err := RemoveBackfill(opts.LogPath); err != nil {
+ problems = append(problems, fmt.Errorf("remove staged session backfill: %w", err))
+ }
if err := RemoveState(opts.UserMode, opts.KeepCredentials); err != nil {
problems = append(problems, fmt.Errorf("remove %s: %w", Dir(opts.UserMode), err))
}
diff --git a/cli/beacon/internal/endpoint/asymptote/connect_test.go b/cli/beacon/internal/endpoint/asymptote/connect_test.go
index fcea39f2..85c2f4ee 100644
--- a/cli/beacon/internal/endpoint/asymptote/connect_test.go
+++ b/cli/beacon/internal/endpoint/asymptote/connect_test.go
@@ -28,6 +28,7 @@ type fakeForwarder struct {
removed int
loadErr error
writeErr error
+ onLoad func()
}
func (f *fakeForwarder) Supported() bool { return f.supported }
@@ -46,7 +47,13 @@ func (f *fakeForwarder) WriteUnit(vectorBin, configPath string) (string, error)
}
return f.unitPath, nil
}
-func (f *fakeForwarder) Load() error { f.loads++; return f.loadErr }
+func (f *fakeForwarder) Load() error {
+ f.loads++
+ if f.onLoad != nil {
+ f.onLoad()
+ }
+ return f.loadErr
+}
func (f *fakeForwarder) Unload() error { f.unloads++; return nil }
func (f *fakeForwarder) RemoveUnits() {
f.removed++
diff --git a/cli/beacon/internal/endpoint/asymptote/pack/README.md b/cli/beacon/internal/endpoint/asymptote/pack/README.md
index 65264f01..37a8b342 100644
--- a/cli/beacon/internal/endpoint/asymptote/pack/README.md
+++ b/cli/beacon/internal/endpoint/asymptote/pack/README.md
@@ -11,8 +11,10 @@ and Vector has been started.
## What leaves the machine
-Every line of `runtime.jsonl` written after the forwarder starts, and every
-line of `inventory_state.jsonl`, exactly as Beacon wrote them locally. Local
+Every line of `runtime.jsonl` written after the forwarder starts, every line
+of `inventory_state.jsonl`, and every line of `cloud-backfill.jsonl` (the
+bounded session backfill `beacon endpoint connect` stages on a first connect;
+absent otherwise), exactly as Beacon wrote them locally. Local
redaction, retention and size limits apply before a line is written, so they
apply to what is forwarded. The ingest service adds a `tenant` block
(organization, device, approving user, receive time) server-side and ignores
@@ -56,7 +58,8 @@ The generated files point at the Beacon log path selected by the CLI:
- System mode: `/var/log/beacon-agent/runtime.jsonl`
- Custom mode: the value passed with `--log-path`
-The inventory log is the sibling `inventory_state.jsonl` in the same directory.
+The inventory log is the sibling `inventory_state.jsonl` in the same directory,
+and the staged session backfill is the sibling `cloud-backfill.jsonl`.
## Run Vector by hand
diff --git a/cli/beacon/internal/endpoint/asymptote/pack/metadata-transforms.toml b/cli/beacon/internal/endpoint/asymptote/pack/metadata-transforms.toml
index c45283f3..3ed19d3f 100644
--- a/cli/beacon/internal/endpoint/asymptote/pack/metadata-transforms.toml
+++ b/cli/beacon/internal/endpoint/asymptote/pack/metadata-transforms.toml
@@ -1,6 +1,6 @@
[transforms.beacon_runtime_metadata]
type = "remap"
-inputs = ["beacon_runtime"]
+inputs = ["beacon_runtime", "beacon_backfill"]
drop_on_abort = true
source = '''
event = parse_json!(string!(.message))
diff --git a/cli/beacon/internal/endpoint/asymptote/pack/vector.toml.tmpl b/cli/beacon/internal/endpoint/asymptote/pack/vector.toml.tmpl
index e8af0389..1f4a6805 100644
--- a/cli/beacon/internal/endpoint/asymptote/pack/vector.toml.tmpl
+++ b/cli/beacon/internal/endpoint/asymptote/pack/vector.toml.tmpl
@@ -26,6 +26,13 @@ include = ["{{LOG_PATH}}"]
# Nothing recorded before the device was approved is shipped.
read_from = "end"
+# The bounded session backfill a first connect stages from the agents' own
+# session stores, so the dashboard shows recent history at once. Read once.
+[sources.beacon_backfill]
+type = "file"
+include = ["{{BACKFILL_LOG_PATH}}"]
+read_from = "beginning"
+
[sources.beacon_inventory]
type = "file"
include = ["{{INVENTORY_LOG_PATH}}"]
@@ -35,7 +42,7 @@ read_from = "beginning"
[sinks.asymptote_runtime]
type = "http"
-inputs = ["beacon_runtime"]
+inputs = ["beacon_runtime", "beacon_backfill"]
uri = "${BEACON_ASYMPTOTE_INGEST_URL}/v1/ingest/runtime"
method = "post"
compression = "gzip"
diff --git a/cli/beacon/internal/endpoint/siempack/siempack.go b/cli/beacon/internal/endpoint/siempack/siempack.go
index be9948c9..95ec6ab5 100644
--- a/cli/beacon/internal/endpoint/siempack/siempack.go
+++ b/cli/beacon/internal/endpoint/siempack/siempack.go
@@ -11,6 +11,12 @@ import (
const logPathToken = "{{LOG_PATH}}"
const inventoryLogPathToken = "{{INVENTORY_LOG_PATH}}"
+const backfillLogPathToken = "{{BACKFILL_LOG_PATH}}"
+
+// BackfillLogName is the file Beacon Cloud's first connect stages the bounded session backfill in,
+// beside the runtime log. Its name does not start with the runtime log's, so readers of the runtime
+// log and its archives never see it.
+const BackfillLogName = "cloud-backfill.jsonl"
type File struct {
Name string
@@ -29,7 +35,16 @@ func ReadFile(fsys fs.FS, path string) (string, error) {
func RenderLogPath(content, logPath string) string {
content = strings.ReplaceAll(content, logPathToken, logPath)
- return strings.ReplaceAll(content, inventoryLogPathToken, InventoryLogPath(logPath))
+ content = strings.ReplaceAll(content, inventoryLogPathToken, InventoryLogPath(logPath))
+ return strings.ReplaceAll(content, backfillLogPathToken, BackfillLogPath(logPath))
+}
+
+// BackfillLogPath is BackfillLogName beside logPath.
+func BackfillLogPath(logPath string) string {
+ if logPath == "" {
+ return BackfillLogName
+ }
+ return filepath.Join(filepath.Dir(logPath), BackfillLogName)
}
func InventoryLogPath(logPath string) string {
diff --git a/cli/beacon/internal/endpoint/writer/budget.go b/cli/beacon/internal/endpoint/writer/budget.go
new file mode 100644
index 00000000..37b1edc2
--- /dev/null
+++ b/cli/beacon/internal/endpoint/writer/budget.go
@@ -0,0 +1,99 @@
+package writer
+
+import (
+ "errors"
+ "sync"
+)
+
+// ErrBudgetSpent is returned by an append whose line would take its Budget past the limit. Nothing
+// is written when it is returned.
+var ErrBudgetSpent = errors.New("the byte budget for this backfill is spent")
+
+// Budget caps the bytes one bounded run of appends may add to the runtime log. The install-time
+// session backfill uses it so a machine with years of agent history fills a bounded part of the
+// log's rotation window instead of rotating out everything else it holds, live telemetry included.
+//
+// A collector that hits the limit stops where it is. Its cursor has advanced only past what was
+// written, so a later sync carries on from there. The zero value has no room; use NewBudget. A
+// Budget is safe for concurrent use.
+type Budget struct {
+ mu sync.Mutex
+ limit int64
+ used int64
+ refused bool
+ parent *Budget
+}
+
+// NewBudget returns a budget of limit bytes.
+func NewBudget(limit int64) *Budget {
+ return &Budget{limit: limit}
+}
+
+// Sub returns a budget of at most limit bytes whose appends also count against b, so one share of a
+// shared budget cannot take all of it.
+func (b *Budget) Sub(limit int64) *Budget {
+ return &Budget{limit: limit, parent: b}
+}
+
+// Used is the number of bytes written against the budget.
+func (b *Budget) Used() int64 {
+ if b == nil {
+ return 0
+ }
+ b.mu.Lock()
+ defer b.mu.Unlock()
+ return b.used
+}
+
+// Remaining is the number of bytes still available.
+func (b *Budget) Remaining() int64 {
+ if b == nil {
+ return 0
+ }
+ b.mu.Lock()
+ defer b.mu.Unlock()
+ if b.used >= b.limit {
+ return 0
+ }
+ return b.limit - b.used
+}
+
+// Refused reports whether an append was turned away because the budget had no room for it.
+func (b *Budget) Refused() bool {
+ if b == nil {
+ return false
+ }
+ b.mu.Lock()
+ defer b.mu.Unlock()
+ return b.refused
+}
+
+// reserve takes n bytes if they fit in b and in every budget above it.
+func (b *Budget) reserve(n int64) bool {
+ b.mu.Lock()
+ if b.used+n > b.limit {
+ b.refused = true
+ b.mu.Unlock()
+ return false
+ }
+ b.used += n
+ b.mu.Unlock()
+ if b.parent != nil && !b.parent.reserve(n) {
+ b.mu.Lock()
+ b.used -= n
+ b.refused = true
+ b.mu.Unlock()
+ return false
+ }
+ return true
+}
+
+// release returns n bytes reserved for an append that did not happen.
+func (b *Budget) release(n int64) {
+ b.mu.Lock()
+ b.used -= n
+ b.mu.Unlock()
+ if b.parent != nil {
+ b.parent.release(n)
+ }
+}
diff --git a/cli/beacon/internal/endpoint/writer/budget_test.go b/cli/beacon/internal/endpoint/writer/budget_test.go
new file mode 100644
index 00000000..5e815d97
--- /dev/null
+++ b/cli/beacon/internal/endpoint/writer/budget_test.go
@@ -0,0 +1,77 @@
+package writer
+
+import (
+ "errors"
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
+)
+
+func budgetTestEvent(i int) schema.Event {
+ return schema.NewEvent(schema.NewEventOptions{
+ Action: "prompt.submitted",
+ Category: "agent",
+ Severity: schema.SeverityInfo,
+ Harness: schema.HarnessInfo{Name: "codex"},
+ Message: strings.Repeat("x", 100) + string(rune('a'+i)),
+ })
+}
+
+func TestBudgetRefusesTheAppendThatWouldNotFit(t *testing.T) {
+ path := filepath.Join(t.TempDir(), "runtime.jsonl")
+ budget := NewBudget(1)
+ _, err := AppendEvent(budgetTestEvent(0), Options{Path: path, Budget: budget})
+ if !errors.Is(err, ErrBudgetSpent) {
+ t.Fatalf("AppendEvent over budget = %v, want ErrBudgetSpent", err)
+ }
+ if _, statErr := os.Stat(path); !os.IsNotExist(statErr) {
+ t.Fatalf("a refused append wrote the log: %v", statErr)
+ }
+ if !budget.Refused() || budget.Used() != 0 {
+ t.Fatalf("budget refused=%t used=%d, want refused and nothing used", budget.Refused(), budget.Used())
+ }
+}
+
+func TestBudgetCountsWrittenBytes(t *testing.T) {
+ path := filepath.Join(t.TempDir(), "runtime.jsonl")
+ budget := NewBudget(1 << 20)
+ for i := 0; i < 3; i++ {
+ if _, err := AppendEvent(budgetTestEvent(i), Options{Path: path, Budget: budget}); err != nil {
+ t.Fatal(err)
+ }
+ }
+ info, err := os.Stat(path)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if budget.Used() != info.Size() {
+ t.Fatalf("budget used %d bytes, log holds %d", budget.Used(), info.Size())
+ }
+ if budget.Refused() {
+ t.Fatal("budget reports a refusal it never made")
+ }
+}
+
+func TestSubBudgetIsBoundByItsParent(t *testing.T) {
+ parent := NewBudget(150)
+ first := parent.Sub(100)
+ if !first.reserve(100) {
+ t.Fatal("first share refused bytes within its limit")
+ }
+ second := parent.Sub(100)
+ if second.reserve(60) {
+ t.Fatal("second share took bytes its parent no longer had")
+ }
+ if !second.reserve(50) {
+ t.Fatal("second share refused bytes its parent still had")
+ }
+ if parent.Used() != 150 || parent.Remaining() != 0 {
+ t.Fatalf("parent used=%d remaining=%d, want 150 and 0", parent.Used(), parent.Remaining())
+ }
+ if !second.Refused() || first.Refused() {
+ t.Fatalf("refused flags first=%t second=%t", first.Refused(), second.Refused())
+ }
+}
diff --git a/cli/beacon/internal/endpoint/writer/writer.go b/cli/beacon/internal/endpoint/writer/writer.go
index 767ac2cd..48980656 100644
--- a/cli/beacon/internal/endpoint/writer/writer.go
+++ b/cli/beacon/internal/endpoint/writer/writer.go
@@ -33,6 +33,9 @@ type Options struct {
// Guard, when set, keeps this append from rotating out anything the guard's earlier appends
// wrote, and records where this one landed. See RetentionGuard.
Guard *RetentionGuard
+ // Budget, when set, caps what this append may add to the log; an event that does not fit
+ // is refused with ErrBudgetSpent. See Budget.
+ Budget *Budget
}
// SystemLogPath is the system-mode runtime log, resolved per platform.
@@ -101,7 +104,14 @@ func AppendEvent(event schema.Event, opts Options) (string, error) {
data = stamped
}
}
- if err := appendJSONL(opts.Path, append(data, '\n'), opts.RotateSize, opts.RotateArchives, opts.Guard); err != nil {
+ line := append(data, '\n')
+ if opts.Budget != nil && !opts.Budget.reserve(int64(len(line))) {
+ return "", ErrBudgetSpent
+ }
+ if err := appendJSONL(opts.Path, line, opts.RotateSize, opts.RotateArchives, opts.Guard); err != nil {
+ if opts.Budget != nil {
+ opts.Budget.release(int64(len(line)))
+ }
return "", err
}
return opts.Path, nil
diff --git a/cli/beacon/internal/factorysession/collect.go b/cli/beacon/internal/factorysession/collect.go
index 8ec6935b..773a7714 100644
--- a/cli/beacon/internal/factorysession/collect.go
+++ b/cli/beacon/internal/factorysession/collect.go
@@ -8,9 +8,11 @@ import (
"os"
"path/filepath"
"strings"
+ "time"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/writer"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/sessionwindow"
)
const StateVersion = 1
@@ -99,6 +101,13 @@ type CollectOptions struct {
UserMode bool
Print bool
Out io.Writer
+ // Since, when set, limits the sweep to sessions modified at or after it, newest first. Older
+ // sessions are not read and their cursors do not move, so a later sync still collects them.
+ // The install-time backfill sets it; an ordinary sync leaves it zero.
+ Since time.Time
+ // Budget, when set, caps the bytes this sweep may append. The sweep stops at the first event
+ // that does not fit and returns writer.ErrBudgetSpent, with cursors past only what was written.
+ Budget *writer.Budget
}
type Summary struct {
@@ -119,6 +128,7 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
if err != nil {
return summary, err
}
+ refs = sessionwindow.Recent(refs, opts.Since, func(r SessionRef) int64 { return r.ModTimeUnixMS })
summary.Sessions = len(refs)
if len(refs) == 0 {
return summary, nil
@@ -136,6 +146,15 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
var errs []error
for _, ref := range refs {
changed, collectErr := collectSession(store, ref, state, opts, &summary)
+ if errors.Is(collectErr, writer.ErrBudgetSpent) {
+ // Not a failed session: this one and the rest are left for a later sync, with cursors
+ // that have not moved past anything that was not written.
+ if changed {
+ summary.SessionsChanged++
+ }
+ errs = append(errs, collectErr)
+ break
+ }
if collectErr != nil {
summary.Errors++
errs = append(errs, fmt.Errorf("Factory session %s: %w", ref.ID, collectErr))
@@ -236,7 +255,7 @@ func emit(event schema.Event, opts CollectOptions) error {
}
}
if opts.Write {
- if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode}); err != nil {
+ if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode, Budget: opts.Budget}); err != nil {
return err
}
}
diff --git a/cli/beacon/internal/fxsession/collect.go b/cli/beacon/internal/fxsession/collect.go
index 8d441c68..88c9a223 100644
--- a/cli/beacon/internal/fxsession/collect.go
+++ b/cli/beacon/internal/fxsession/collect.go
@@ -8,9 +8,11 @@ import (
"os"
"path/filepath"
"strings"
+ "time"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/writer"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/sessionwindow"
)
// StateVersion is the on-disk version of the collector's cursor file. It exists so a future change
@@ -135,6 +137,13 @@ type CollectOptions struct {
// StatePath so a dry run shows the same events every time.
Print bool
Out io.Writer
+ // Since, when set, limits the sweep to sessions modified at or after it, newest first. Older
+ // sessions are not read and their cursors do not move, so a later sync still collects them.
+ // The install-time backfill sets it; an ordinary sync leaves it zero.
+ Since time.Time
+ // Budget, when set, caps the bytes this sweep may append. The sweep stops at the first event
+ // that does not fit and returns writer.ErrBudgetSpent, with cursors past only what was written.
+ Budget *writer.Budget
}
// Summary reports what a sweep did. Malformed and PartialSessions are reported rather than logged
@@ -167,6 +176,7 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
if err != nil {
return summary, err
}
+ refs = sessionwindow.Recent(refs, opts.Since, func(r SessionRef) int64 { return r.ModTimeUnixMS })
summary.Sessions = len(refs)
if len(refs) == 0 {
return summary, nil
@@ -190,6 +200,15 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
var errs []error
for _, ref := range refs {
changed, collectErr := collectSession(store, ref, state, opts, &summary)
+ if errors.Is(collectErr, writer.ErrBudgetSpent) {
+ // Not a failed session: this one and the rest are left for a later sync, with cursors
+ // that have not moved past anything that was not written.
+ if changed {
+ summary.SessionsChanged++
+ }
+ errs = append(errs, collectErr)
+ break
+ }
if collectErr != nil {
summary.Errors++
errs = append(errs, fmt.Errorf("fx session %s: %w", ref.ID, collectErr))
@@ -360,7 +379,7 @@ func emit(event schema.Event, opts CollectOptions) error {
}
}
if opts.Write {
- if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode}); err != nil {
+ if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode, Budget: opts.Budget}); err != nil {
return err
}
}
diff --git a/cli/beacon/internal/groksession/collect.go b/cli/beacon/internal/groksession/collect.go
index f2bce8b7..47a7ced4 100644
--- a/cli/beacon/internal/groksession/collect.go
+++ b/cli/beacon/internal/groksession/collect.go
@@ -8,9 +8,11 @@ import (
"os"
"path/filepath"
"strings"
+ "time"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/writer"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/sessionwindow"
)
const StateVersion = 1
@@ -115,6 +117,13 @@ type CollectOptions struct {
UserMode bool
Print bool
Out io.Writer
+ // Since, when set, limits the sweep to sessions modified at or after it, newest first. Older
+ // sessions are not read and their cursors do not move, so a later sync still collects them.
+ // The install-time backfill sets it; an ordinary sync leaves it zero.
+ Since time.Time
+ // Budget, when set, caps the bytes this sweep may append. The sweep stops at the first event
+ // that does not fit and returns writer.ErrBudgetSpent, with cursors past only what was written.
+ Budget *writer.Budget
}
type SummaryResult struct {
@@ -137,6 +146,7 @@ func CollectOnce(opts CollectOptions) (summary SummaryResult, err error) {
if err != nil {
return summary, err
}
+ refs = sessionwindow.Recent(refs, opts.Since, func(r SessionRef) int64 { return r.ModTimeUnixMS })
summary.Sessions = len(refs)
if len(refs) == 0 {
return summary, nil
@@ -154,6 +164,15 @@ func CollectOnce(opts CollectOptions) (summary SummaryResult, err error) {
var errs []error
for _, ref := range refs {
changed, collectErr := collectSession(store, ref, state, opts, &summary)
+ if errors.Is(collectErr, writer.ErrBudgetSpent) {
+ // Not a failed session: this one and the rest are left for a later sync, with cursors
+ // that have not moved past anything that was not written.
+ if changed {
+ summary.SessionsChanged++
+ }
+ errs = append(errs, collectErr)
+ break
+ }
if collectErr != nil {
summary.Errors++
errs = append(errs, fmt.Errorf("Grok session %s: %w", ref.ID, collectErr))
@@ -274,7 +293,7 @@ func emit(event schema.Event, opts CollectOptions) error {
}
}
if opts.Write {
- if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode}); err != nil {
+ if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode, Budget: opts.Budget}); err != nil {
return err
}
}
diff --git a/cli/beacon/internal/hermessession/collect.go b/cli/beacon/internal/hermessession/collect.go
index be876ca0..3a439ee8 100644
--- a/cli/beacon/internal/hermessession/collect.go
+++ b/cli/beacon/internal/hermessession/collect.go
@@ -18,6 +18,7 @@ import (
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/writer"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/sessionwindow"
"github.com/asymptote-labs/agent-beacon/pkg/asymptoteobserve"
_ "modernc.org/sqlite"
)
@@ -53,6 +54,13 @@ type CollectOptions struct {
UserMode bool
Print bool
Out io.Writer
+ // Since, when set, limits the sweep to sessions modified at or after it, newest first. Older
+ // sessions are not read and their cursors do not move, so a later sync still collects them.
+ // The install-time backfill sets it; an ordinary sync leaves it zero.
+ Since time.Time
+ // Budget, when set, caps the bytes this sweep may append. The sweep stops at the first event
+ // that does not fit and returns writer.ErrBudgetSpent, with cursors past only what was written.
+ Budget *writer.Budget
}
type Summary struct {
@@ -198,6 +206,7 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
if err != nil {
return summary, err
}
+ sessions = sessionwindow.Recent(sessions, opts.Since, func(r Session) int64 { return max(r.StartedAtMS, r.EndedAtMS) })
summary.Sessions = len(sessions)
state, err := LoadState(opts.StatePath)
if err != nil {
@@ -211,6 +220,15 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
var errs []error
for _, session := range sessions {
changed, collectErr := collectSession(store, session, state, opts, &summary)
+ if errors.Is(collectErr, writer.ErrBudgetSpent) {
+ // Not a failed session: this one and the rest are left for a later sync, with cursors
+ // that have not moved past anything that was not written.
+ if changed {
+ summary.SessionsChanged++
+ }
+ errs = append(errs, collectErr)
+ break
+ }
if collectErr != nil {
summary.Errors++
errs = append(errs, fmt.Errorf("Hermes session %s: %w", session.ID, collectErr))
@@ -307,7 +325,7 @@ func emit(event schema.Event, opts CollectOptions) error {
}
}
if opts.Write {
- if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode}); err != nil {
+ if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode, Budget: opts.Budget}); err != nil {
return err
}
}
diff --git a/cli/beacon/internal/onboarding/wizard.go b/cli/beacon/internal/onboarding/wizard.go
index 79e20b88..bb177b76 100644
--- a/cli/beacon/internal/onboarding/wizard.go
+++ b/cli/beacon/internal/onboarding/wizard.go
@@ -60,6 +60,10 @@ type WizardOptions struct {
DestinationOnly bool
PresetDestination string
PresetPrivacyMode string
+ // BackfillDays is how far back the install reads agent sessions from their local session
+ // stores, and a first connect uploads them; zero when the backfill is off. The confirm screen
+ // names it, because it is part of what confirming Beacon Cloud sends.
+ BackfillDays int
// SignIn signs in in place. When nil the wizard keeps its original contract:
// it exits with NeedLogin and the caller signs in and runs it again.
@@ -634,8 +638,14 @@ func (m wizardModel) View() string {
body += "\nPrivacy: " + managedprivacy.Label(m.result.PrivacyMode) +
"\n\n" + wizardWarn.Render("This installs Beacon and starts forwarding to beacon.sh.") +
"\n" + wizardDim.Render(privacySends(m.result.PrivacyMode))
+ if m.options.BackfillDays > 0 {
+ body += "\n" + wizardDim.Render(fmt.Sprintf("It also uploads your agent sessions from the last %d days, read from their session history on this machine.", m.options.BackfillDays))
+ }
} else {
body += "\n\nYour telemetry stays on this machine."
+ if m.options.BackfillDays > 0 {
+ body += fmt.Sprintf(" Beacon reads your agent sessions from the last %d days into its local log.", m.options.BackfillDays)
+ }
if m.result.WithoutAccount {
body += " You are not signed in; connect later with `beacon endpoint connect`."
}
diff --git a/cli/beacon/internal/onboarding/wizard_test.go b/cli/beacon/internal/onboarding/wizard_test.go
index 163e5867..26ce008c 100644
--- a/cli/beacon/internal/onboarding/wizard_test.go
+++ b/cli/beacon/internal/onboarding/wizard_test.go
@@ -661,3 +661,35 @@ func TestSignInScreenHonorsNoBrowser(t *testing.T) {
}
}
}
+
+// Confirming Beacon Cloud also uploads the session backfill, so the confirm screen, which is the
+// consent point, has to say so; and a local choice says what the install reads.
+func TestConfirmScreenNamesTheSessionBackfill(t *testing.T) {
+ render := func(destination string, days int) string {
+ model := newWizardModel(WizardOptions{
+ SignedIn: true,
+ Email: "[email protected]",
+ OfferManaged: true,
+ DestinationOnly: true,
+ PresetDestination: destination,
+ PresetPrivacyMode: "standard",
+ BackfillDays: days,
+ })
+ model.width, model.height = 100, 30
+ model.screen = confirmScreen
+ model.result.Destination = destination
+ model.result.PrivacyMode = "standard"
+ return strings.Join(strings.Fields(model.View()), " ")
+ }
+ if view := render(DestinationAsymptote, 30); !strings.Contains(view, "It also uploads your agent sessions from the last 30 days") {
+ t.Fatalf("managed confirm screen does not name the backfill:\n%s", view)
+ }
+ if view := render(DestinationLocal, 30); !strings.Contains(view, "Beacon reads your agent sessions from the last 30 days into its local log.") {
+ t.Fatalf("local confirm screen does not name the backfill:\n%s", view)
+ }
+ for _, destination := range []string{DestinationAsymptote, DestinationLocal} {
+ if view := render(destination, 0); strings.Contains(view, "last 30 days") || strings.Contains(view, "agent sessions from") {
+ t.Fatalf("confirm screen names a backfill that is off:\n%s", view)
+ }
+ }
+}
diff --git a/cli/beacon/internal/openclawsession/collect.go b/cli/beacon/internal/openclawsession/collect.go
index eab79df4..79ecfc21 100644
--- a/cli/beacon/internal/openclawsession/collect.go
+++ b/cli/beacon/internal/openclawsession/collect.go
@@ -8,9 +8,11 @@ import (
"os"
"path/filepath"
"strings"
+ "time"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/writer"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/sessionwindow"
)
const StateVersion = 1
@@ -99,6 +101,13 @@ type CollectOptions struct {
UserMode bool
Print bool
Out io.Writer
+ // Since, when set, limits the sweep to sessions modified at or after it, newest first. Older
+ // sessions are not read and their cursors do not move, so a later sync still collects them.
+ // The install-time backfill sets it; an ordinary sync leaves it zero.
+ Since time.Time
+ // Budget, when set, caps the bytes this sweep may append. The sweep stops at the first event
+ // that does not fit and returns writer.ErrBudgetSpent, with cursors past only what was written.
+ Budget *writer.Budget
}
type Summary struct {
@@ -117,6 +126,7 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
if err != nil {
return summary, err
}
+ refs = sessionwindow.Recent(refs, opts.Since, func(r TraceRef) int64 { return r.UpdatedAtUnixMS })
summary.Traces = len(refs)
state, err := LoadState(opts.StatePath)
if err != nil {
@@ -131,6 +141,15 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
var errs []error
for _, ref := range refs {
changed, collectErr := collectTrace(store, ref, state, opts, &summary)
+ if errors.Is(collectErr, writer.ErrBudgetSpent) {
+ // Not a failed session: this one and the rest are left for a later sync, with cursors
+ // that have not moved past anything that was not written.
+ if changed {
+ summary.TracesChanged++
+ }
+ errs = append(errs, collectErr)
+ break
+ }
if collectErr != nil {
summary.Errors++
errs = append(errs, fmt.Errorf("OpenClaw trace %s: %w", ref.ID, collectErr))
@@ -207,7 +226,7 @@ func emit(event schema.Event, opts CollectOptions) error {
}
}
if opts.Write {
- if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode}); err != nil {
+ if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode, Budget: opts.Budget}); err != nil {
return err
}
}
diff --git a/cli/beacon/internal/opencodesession/collect.go b/cli/beacon/internal/opencodesession/collect.go
index f5250095..66302619 100644
--- a/cli/beacon/internal/opencodesession/collect.go
+++ b/cli/beacon/internal/opencodesession/collect.go
@@ -8,9 +8,11 @@ import (
"os"
"path/filepath"
"strings"
+ "time"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/writer"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/sessionwindow"
)
const StateVersion = 1
@@ -100,6 +102,13 @@ type CollectOptions struct {
UserMode bool
Print bool
Out io.Writer
+ // Since, when set, limits the sweep to sessions modified at or after it, newest first. Older
+ // sessions are not read and their cursors do not move, so a later sync still collects them.
+ // The install-time backfill sets it; an ordinary sync leaves it zero.
+ Since time.Time
+ // Budget, when set, caps the bytes this sweep may append. The sweep stops at the first event
+ // that does not fit and returns writer.ErrBudgetSpent, with cursors past only what was written.
+ Budget *writer.Budget
}
type Summary struct {
@@ -118,6 +127,7 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
if listErr != nil && len(refs) == 0 {
return summary, listErr
}
+ refs = sessionwindow.Recent(refs, opts.Since, func(r TraceRef) int64 { return r.UpdatedAtUnixMS })
summary.Traces = len(refs)
state, err := LoadState(opts.StatePath)
if err != nil {
@@ -136,6 +146,15 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
}
for _, ref := range refs {
changed, collectErr := collectTrace(store, ref, state, opts, &summary)
+ if errors.Is(collectErr, writer.ErrBudgetSpent) {
+ // Not a failed session: this one and the rest are left for a later sync, with cursors
+ // that have not moved past anything that was not written.
+ if changed {
+ summary.TracesChanged++
+ }
+ errs = append(errs, collectErr)
+ break
+ }
if collectErr != nil {
summary.Errors++
errs = append(errs, fmt.Errorf("OpenCode trace %s: %w", ref.ID, collectErr))
@@ -222,7 +241,7 @@ func emit(event schema.Event, opts CollectOptions) error {
}
}
if opts.Write {
- if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode}); err != nil {
+ if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode, Budget: opts.Budget}); err != nil {
return err
}
}
diff --git a/cli/beacon/internal/pisession/collect.go b/cli/beacon/internal/pisession/collect.go
index 8b99ef7a..790676e0 100644
--- a/cli/beacon/internal/pisession/collect.go
+++ b/cli/beacon/internal/pisession/collect.go
@@ -5,9 +5,11 @@ import (
"errors"
"fmt"
"io"
+ "time"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/writer"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/sessionwindow"
)
type CollectOptions struct {
@@ -18,6 +20,13 @@ type CollectOptions struct {
UserMode bool
Print bool
Out io.Writer
+ // Since, when set, limits the sweep to sessions modified at or after it, newest first. Older
+ // sessions are not read and their cursors do not move, so a later sync still collects them.
+ // The install-time backfill sets it; an ordinary sync leaves it zero.
+ Since time.Time
+ // Budget, when set, caps the bytes this sweep may append. The sweep stops at the first event
+ // that does not fit and returns writer.ErrBudgetSpent, with cursors past only what was written.
+ Budget *writer.Budget
}
type Summary struct {
@@ -37,6 +46,7 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
if err != nil {
return summary, err
}
+ refs = sessionwindow.Recent(refs, opts.Since, func(r SessionRef) int64 { return r.ModTimeMS })
summary.Sessions = len(refs)
if len(refs) == 0 {
return summary, nil
@@ -54,6 +64,15 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
var errs []error
for _, ref := range refs {
changed, collectErr := collectFile(store, ref, state, opts, &summary)
+ if errors.Is(collectErr, writer.ErrBudgetSpent) {
+ // Not a failed session: this one and the rest are left for a later sync, with cursors
+ // that have not moved past anything that was not written.
+ if changed {
+ summary.SessionsChanged++
+ }
+ errs = append(errs, collectErr)
+ break
+ }
if collectErr != nil {
summary.Errors++
errs = append(errs, fmt.Errorf("Pi session %s: %w", ref.Path, collectErr))
@@ -143,7 +162,7 @@ func emit(event schema.Event, opts CollectOptions) error {
}
}
if opts.Write {
- if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode}); err != nil {
+ if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode, Budget: opts.Budget}); err != nil {
return err
}
}
diff --git a/cli/beacon/internal/primesession/collect.go b/cli/beacon/internal/primesession/collect.go
index 87d154f7..0843725f 100644
--- a/cli/beacon/internal/primesession/collect.go
+++ b/cli/beacon/internal/primesession/collect.go
@@ -5,9 +5,11 @@ import (
"errors"
"fmt"
"io"
+ "time"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/writer"
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/sessionwindow"
)
type CollectOptions struct {
@@ -19,6 +21,13 @@ type CollectOptions struct {
UserMode bool
Print bool
Out io.Writer
+ // Since, when set, limits the sweep to sessions modified at or after it, newest first. Older
+ // sessions are not read and their cursors do not move, so a later sync still collects them.
+ // The install-time backfill sets it; an ordinary sync leaves it zero.
+ Since time.Time
+ // Budget, when set, caps the bytes this sweep may append. The sweep stops at the first event
+ // that does not fit and returns writer.ErrBudgetSpent, with cursors past only what was written.
+ Budget *writer.Budget
}
type Summary struct {
@@ -38,6 +47,7 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
if err != nil {
return summary, err
}
+ refs = sessionwindow.Recent(refs, opts.Since, func(r SessionRef) int64 { return r.ModTimeMS })
summary.Sessions = len(refs)
if len(refs) == 0 {
return summary, nil
@@ -55,6 +65,15 @@ func CollectOnce(opts CollectOptions) (summary Summary, err error) {
var errs []error
for _, ref := range refs {
changed, collectErr := collectFile(store, ref, state, opts, &summary)
+ if errors.Is(collectErr, writer.ErrBudgetSpent) {
+ // Not a failed session: this one and the rest are left for a later sync, with cursors
+ // that have not moved past anything that was not written.
+ if changed {
+ summary.SessionsChanged++
+ }
+ errs = append(errs, collectErr)
+ break
+ }
if collectErr != nil {
summary.Errors++
errs = append(errs, fmt.Errorf("Prime Agent session %s: %w", ref.Path, collectErr))
@@ -145,7 +164,7 @@ func emit(event schema.Event, opts CollectOptions) error {
}
}
if opts.Write {
- if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode}); err != nil {
+ if _, err := writer.AppendEvent(event, writer.Options{Path: opts.LogPath, UserMode: opts.UserMode, Budget: opts.Budget}); err != nil {
return err
}
}
diff --git a/cli/beacon/internal/sessionwindow/sessionwindow.go b/cli/beacon/internal/sessionwindow/sessionwindow.go
new file mode 100644
index 00000000..adaa211b
--- /dev/null
+++ b/cli/beacon/internal/sessionwindow/sessionwindow.go
@@ -0,0 +1,40 @@
+// Package sessionwindow selects the sessions a bounded backfill sweep reads.
+//
+// Every session-store collector lists its sessions in its own order and reads all of them. A
+// backfill that runs on its own, at install or connect, should read only recent history and should
+// spend whatever budget it has on the newest sessions first, so the collectors share this one rule
+// rather than each growing its own.
+package sessionwindow
+
+import (
+ "sort"
+ "time"
+)
+
+// Recent returns the refs modified at or after since, newest first. A ref whose modification time
+// is unknown (zero or negative) is kept, after every dated one, because the store gave no reason to
+// skip it. A zero since returns refs unchanged, in their original order: that is every ordinary
+// sync.
+//
+// Refs left out are not consumed: their collector cursors are never touched, so a later
+// `beacon endpoint <runtime> sync` still reads them.
+func Recent[T any](refs []T, since time.Time, modifiedMS func(T) int64) []T {
+ if since.IsZero() {
+ return refs
+ }
+ cutoff := since.UnixMilli()
+ out := make([]T, 0, len(refs))
+ for _, ref := range refs {
+ if ms := modifiedMS(ref); ms <= 0 || ms >= cutoff {
+ out = append(out, ref)
+ }
+ }
+ sort.SliceStable(out, func(i, j int) bool {
+ mi, mj := modifiedMS(out[i]), modifiedMS(out[j])
+ if (mi <= 0) != (mj <= 0) {
+ return mj <= 0
+ }
+ return mi > mj
+ })
+ return out
+}
diff --git a/cli/beacon/internal/sessionwindow/sessionwindow_test.go b/cli/beacon/internal/sessionwindow/sessionwindow_test.go
new file mode 100644
index 00000000..dab739f1
--- /dev/null
+++ b/cli/beacon/internal/sessionwindow/sessionwindow_test.go
@@ -0,0 +1,45 @@
+package sessionwindow
+
+import (
+ "reflect"
+ "testing"
+ "time"
+)
+
+type ref struct {
+ id string
+ ms int64
+}
+
+func ids(refs []ref) []string {
+ out := []string{}
+ for _, r := range refs {
+ out = append(out, r.id)
+ }
+ return out
+}
+
+func TestRecentKeepsOrderWithoutAWindow(t *testing.T) {
+ refs := []ref{{"a", 3}, {"b", 1}, {"c", 2}}
+ got := Recent(refs, time.Time{}, func(r ref) int64 { return r.ms })
+ if !reflect.DeepEqual(ids(got), []string{"a", "b", "c"}) {
+ t.Fatalf("order changed without a window: %v", ids(got))
+ }
+}
+
+func TestRecentDropsOldSessionsAndSortsNewestFirst(t *testing.T) {
+ now := time.Date(2026, 10, 5, 12, 0, 0, 0, time.UTC)
+ since := now.Add(-30 * 24 * time.Hour)
+ refs := []ref{
+ {"old", since.Add(-time.Hour).UnixMilli()},
+ {"edge", since.UnixMilli()},
+ {"unknown", 0},
+ {"newest", now.UnixMilli()},
+ {"middle", now.Add(-time.Hour).UnixMilli()},
+ }
+ got := Recent(refs, since, func(r ref) int64 { return r.ms })
+ want := []string{"newest", "middle", "edge", "unknown"}
+ if !reflect.DeepEqual(ids(got), want) {
+ t.Fatalf("Recent = %v, want %v", ids(got), want)
+ }
+}
diff --git a/docs/cli/asymptote.mdx b/docs/cli/asymptote.mdx
index 7686a7f7..b78b7e99 100644
--- a/docs/cli/asymptote.mdx
+++ b/docs/cli/asymptote.mdx
@@ -46,7 +46,7 @@ beacon endpoint asymptote [command]
| User mode | `~/.beacon/endpoint/logs/runtime.jsonl` |
| System mode | `/var/log/beacon-agent/runtime.jsonl` |
-The inventory log is the sibling `inventory_state.jsonl` in the same directory.
+The inventory log is the sibling `inventory_state.jsonl` in the same directory. The forwarder also reads the sibling `cloud-backfill.jsonl` once from the beginning: the bounded [session backfill](/cli/endpoint-connect#session-backfill) a first `beacon endpoint connect` stages there. A hand-run forwarder ships it only if that file exists.
## beacon endpoint asymptote print-config
diff --git a/docs/cli/endpoint-connect.mdx b/docs/cli/endpoint-connect.mdx
index 54948f25..2c92ac57 100644
--- a/docs/cli/endpoint-connect.mdx
+++ b/docs/cli/endpoint-connect.mdx
@@ -8,7 +8,10 @@ description: "Connect a Beacon endpoint to Beacon Cloud ingest, or disconnect it
`beacon endpoint connect` forwards this endpoint's telemetry to Beacon so it appears on the Beacon dashboard. For a user endpoint, the signed-in CLI account authorizes a device-specific ingest key without another browser approval. The account token is never copied into endpoint state or given to Vector. System mode retains browser approval because unattended/system installs are account-free.
Forwarding is opt-in and revocable. Runtime telemetry starts at the end of the
-existing runtime log, so pre-connect runtime events are not sent. Inventory starts
+existing runtime log, so pre-connect runtime events are not sent, with one bounded
+exception: a first user-mode connect uploads the last 30 days of agent sessions read
+from the runtimes' own session stores, so the dashboard shows your recent work at once
+(see [Session backfill](#session-backfill)). Inventory starts
at the beginning of `inventory_state.jsonl` so the dashboard receives the endpoint's
current inventory baseline, including inventory records written before connection.
Revoking the device from the dashboard stops ingestion within about a minute.
@@ -30,10 +33,11 @@ beacon endpoint disconnect [flags]
1. Locates Vector: `--vector-bin`, then `BEACON_VECTOR_BIN`, then `/opt/beacon/bin/vector`, the `beacon-vector` keg in each Homebrew prefix, a linked `vector` in each Homebrew prefix, then `PATH`. The `beacon-vector` keg is tried ahead of any other Homebrew Vector because it is the copy the tap pins at a supported version.
2. Renders the forwarder config for the selected privacy mode into a scratch directory, with a placeholder key, and runs `vector validate` on it. A config Vector rejects stops `connect` here, before anything is sent to the server.
-3. Uses the signed-in account's `device:enroll` scope to register this machine's hostname, operating system, architecture, Beacon version, install id, and install mode. The server returns a separate `bcn_device_*` ingest key. System mode uses the existing browser PKCE device approval instead.
-4. Writes the device key once to `asymptote/vector-secrets.json` with mode `0600`. The account token is not persisted by the endpoint flow.
-5. Renders `asymptote/vector.toml` with the selected privacy mode, runs `vector validate` on the file the service will run, installs a service unit, and starts it.
-6. Records the non-secret enrollment, including privacy mode, in `asymptote/enrollment.json` and `config.json`'s `managed_ingest` block.
+3. On a first user-mode connect, runs the [session backfill](#session-backfill) sweep into the runtime log. It is local and runs before any key exists.
+4. Uses the signed-in account's `device:enroll` scope to register this machine's hostname, operating system, architecture, Beacon version, install id, and install mode. The server returns a separate `bcn_device_*` ingest key. System mode uses the existing browser PKCE device approval instead.
+5. Writes the device key once to `asymptote/vector-secrets.json` with mode `0600`. The account token is not persisted by the endpoint flow.
+6. Renders `asymptote/vector.toml` with the selected privacy mode and runs `vector validate` on the file the service will run. On a first connect, stages the session backfill in `cloud-backfill.jsonl` beside the runtime log. Then installs a service unit and starts it.
+7. Records the non-secret enrollment, including privacy mode, in `asymptote/enrollment.json` and `config.json`'s `managed_ingest` block.
A first connect that fails at `vector validate` or at loading the service saves no enrollment record, so the endpoint is not connected and `beacon endpoint disconnect` cleans up what was written. After `disconnect --keep-credentials`, status reports not connected and names the kept device id.
@@ -41,6 +45,26 @@ Re-running `connect` on a connected machine reuses its install id, so the server
Some steps can only fail after the server has issued the key: storing it, validating the config that carries the server's ingest URL, writing the service unit, starting the service, and recording the enrollment. `connect` writes `asymptote/connect-pending` as soon as the key arrives and removes it only when every step has succeeded. If a step in between fails, `connect` says the connect is incomplete and names the retry, and `beacon endpoint status` reports Beacon Cloud as not connected with `connect_incomplete: true` in `--json`. On a re-connect, the forwarder still running holds the key the server just rotated out, so the previous enrollment record and config are not treated as a working connection. On a first connect, Beacon stops and removes any partial forwarder before returning the error, so a later onboarding retry can safely select local-only telemetry. Run `beacon endpoint connect` again once the cause is fixed. An approval that is cancelled or expires before a key is issued rotates nothing and leaves an existing connection as it was.
+## Session backfill
+
+A first user-mode connect uploads recent session history so the Beacon Cloud dashboard is not
+empty until an agent runs again. Beacon first runs the same bounded sweep as
+[`beacon endpoint install`](/cli/endpoint-install#session-backfill): the last 30 days of sessions
+from each runtime's own session store, newest first, at most 24 MiB. `endpoint install --connect`
+and the onboarding wizard skip that sweep, because the install has just run it. Then, before the
+forwarder starts, Beacon copies the events those sweeps wrote (`harness.collection_method=poll`,
+timestamped in the last 30 days, newest kept up to 24 MiB) from the runtime log and its archives
+into `cloud-backfill.jsonl` beside the runtime log (`0600`). The forwarder reads that file once from
+the beginning into the same runtime upload, through the same privacy transform as live events, so
+Metadata-only strips the backfill exactly as it strips live telemetry. Hook, OTLP and plugin events
+recorded before the connection stay local.
+
+Only a first connect stages the backfill. A re-connect resumes from the forwarder's checkpoints;
+one that changes the privacy mode clears them and deletes the staged file, so nothing is uploaded
+twice. `disconnect` removes the file. The dashboard collapses any re-sent event on its
+deterministic event id. `--no-backfill` or `BEACON_BACKFILL=0` connects without uploading any
+history. System-mode connects never backfill.
+
## Privacy modes
The onboarding wizard records one of two modes. `connect` uses that choice unless
@@ -66,6 +90,7 @@ are not uploaded under the new one.
| Pending-connect marker (only while a connect is unfinished) | `~/.beacon/endpoint/asymptote/connect-pending` (0600, timestamp and reconnect flag only) | `…/Endpoint/asymptote/connect-pending` | `/etc/beacon/endpoint/asymptote/connect-pending` |
| Device key | `~/.beacon/endpoint/asymptote/vector-secrets.json` (0600) | `…/Endpoint/asymptote/vector-secrets.json` | `/etc/beacon/endpoint/asymptote/vector-secrets.json` |
| Vector config | `~/.beacon/endpoint/asymptote/vector.toml` (0644, no secret) | `…/Endpoint/asymptote/vector.toml` | `/etc/beacon/endpoint/asymptote/vector.toml` |
+| Staged session backfill (first connect only) | `~/.beacon/endpoint/logs/cloud-backfill.jsonl` (0600) | not written | not written |
| Buffer and checkpoints | `~/.beacon/endpoint/asymptote/vector-data/` | `…/Endpoint/asymptote/vector-data/` | `/etc/beacon/endpoint/asymptote/vector-data/` |
| Service | `~/Library/LaunchAgents/com.beacon.endpoint.asymptote-forwarder.plist` | `/Library/LaunchDaemons/com.beacon.endpoint.asymptote-forwarder.plist` | `/etc/systemd/system/beacon-asymptote-forwarder.service` (user mode: `~/.config/systemd/user/`) |
@@ -79,6 +104,7 @@ The directory is `0700`. The device key appears only in the secrets file, which
| `--no-browser` | System mode only: print the approval URL instead of opening a browser |
| `--vector-bin <path>` | Vector binary to run |
| `--privacy-mode <mode>` | Override the onboarding choice with `standard` or `metadata-only` |
+| `--no-backfill` | On a first connect, do not upload the [session backfill](#session-backfill). `BEACON_BACKFILL=0` does the same |
| `--user` | Use per-user endpoint paths. Enabled by default |
| `--system` | Use system endpoint paths and a LaunchDaemon or system unit. Requires root; the browser opens as the active console user |
| `--log-path <path>` | Runtime JSONL log path |
diff --git a/docs/cli/endpoint-install.mdx b/docs/cli/endpoint-install.mdx
index 6fd99cfb..5e3bd417 100644
--- a/docs/cli/endpoint-install.mdx
+++ b/docs/cli/endpoint-install.mdx
@@ -51,8 +51,34 @@ The `claude` harness writes both Claude Code's OpenTelemetry settings and its us
| `--falcon-sourcetype <type>` | Optional Falcon LogScale parser or sourcetype |
| `--falcon-insecure-skip-verify` | Skip Falcon LogScale HEC TLS certificate verification |
| `--falcon-ca-file <path>` | Optional CA certificate path for Falcon LogScale HEC TLS verification |
+| `--no-backfill` | Skip the [session backfill](#session-backfill). `BEACON_BACKFILL=0` does the same |
| `--connect` | After a successful install, attempt to connect this endpoint to Beacon Cloud ingest. User mode uses the signed-in CLI account; system mode opens browser approval. A connection failure does not roll back or fail the completed local install; verify with `beacon endpoint status --json` and retry with [`beacon endpoint connect`](/cli/endpoint-connect) |
+## Session backfill
+
+After a successful user-mode install, Beacon reads the last 30 days of agent sessions from each
+runtime's own session store into the runtime log, so the [local dashboard](/cli/dashboard) has
+your recent sessions before any agent runs again. It is the same sweep each runtime's sync command
+runs (`beacon endpoint claude sync`, `codex sync`, `cursor sync`, `opencode sync`, `copilot sync`,
+`cline sync`, `factory sync`, `pi sync`, `prime sync`, `hermes sync`, `dsh sync`, `fx sync`,
+`grok sync`, and `integrations openclaw sessions sync`), and its events carry
+`harness.collection_method=poll` like theirs. It is local only and never touches the network.
+
+The backfill is bounded so a machine with a lot of history cannot crowd live telemetry out of the
+runtime log's rotation window:
+
+- Only sessions modified in the last 30 days are read, newest first.
+- It writes at most 24 MiB in total, and no single runtime takes more than 12 MiB of that.
+
+Nothing past a limit is lost. The collector cursors move only past what was written, and sessions
+left out are not touched, so the runtime's sync command reads the rest; install names the command
+for each runtime that hit the limit. A runtime whose store cannot be read is reported as a warning
+and never fails the install. Re-running install reads only what changed since the last sweep.
+
+The backfill runs only for user-mode installs: system, package, MDM and `--dry-run` installs skip
+it. Turn it off with `--no-backfill` or `BEACON_BACKFILL=0`. If this install also connects to
+Beacon Cloud, the same history is [uploaded on the first connect](/cli/endpoint-connect#session-backfill).
+
## Interactive onboarding wizard
On a first interactive install the [onboarding wizard](/cli/endpoint-onboarding)
@@ -60,8 +86,8 @@ signs in through beacon.sh, then asks where this machine's telemetry should go:
| Answer | What happens |
|--------|--------------|
-| Beacon Cloud (default) | Shows what will be sent, asks for Standard or Metadata-only privacy, then installs Beacon and connects this endpoint. The confirm screen states that forwarding starts before you accept. An endpoint that is already enrolled keeps its device key and is not re-enrolled. |
-| Local only | Explicitly opts out of managed forwarding. Nothing is sent anywhere. |
+| Beacon Cloud (default) | Shows what will be sent, asks for Standard or Metadata-only privacy, then installs Beacon and connects this endpoint. The confirm screen states that forwarding starts, and that the last 30 days of agent sessions are uploaded, before you accept. An endpoint that is already enrolled keeps its device key and is not re-enrolled. |
+| Local only | Explicitly opts out of managed forwarding. Nothing is sent anywhere; the session backfill stays in the local log. |
`--connect` presets Beacon Cloud and skips the destination question. Whether
connect was chosen in the wizard or by the flag, a failed enrollment leaves the
diff --git a/docs/cli/endpoint-onboarding.mdx b/docs/cli/endpoint-onboarding.mdx
index 2af11cd6..daf8f80a 100644
--- a/docs/cli/endpoint-onboarding.mdx
+++ b/docs/cli/endpoint-onboarding.mdx
@@ -61,11 +61,12 @@ to opt out of managed forwarding.
Choosing Beacon Cloud shows a separate disclosure: your agents' activity on this
machine will be sent to your Beacon account as it happens, and `beacon endpoint
-disconnect` stops it. The screen makes no claim about data already on disk, because
-the honest version of that claim needs a caveat: runtime events recorded before you
-connect stay local, but the inventory snapshot of which agent runtimes, skills, and
-MCP servers are installed does upload once as the endpoint's baseline. That detail is
-documented here rather than shown on the screen where someone is choosing.
+disconnect` stops it. The confirm screen also says that your agent sessions from the
+last 30 days, read from their session history on this machine, are uploaded. Live
+runtime events recorded before you connect stay local, but the inventory snapshot of
+which agent runtimes, skills, and MCP servers are installed does upload once as the
+endpoint's baseline. That detail is documented here rather than shown on the screen
+where someone is choosing.
Standard forwards the complete locally sanitized event. Metadata only strips retained
prompts, responses, reasoning, tool arguments/results, command output, raw fields,
@@ -79,9 +80,14 @@ authorized. An endpoint that is already enrolled is not re-enrolled: enrollment
rotates the device key, so a connected machine keeps the key it has and simply
records the destination.
-Pre-connect runtime history is not uploaded. On the first connection, inventory is
-read from the beginning of `inventory_state.jsonl` so the hosted dashboard receives
-the endpoint's current inventory baseline.
+The install reads the last 30 days of agent sessions from each runtime's own session
+store into the local log (the [session backfill](/cli/endpoint-install#session-backfill)).
+Choosing Beacon Cloud uploads that history on the first connection, so the hosted
+dashboard shows your recent sessions immediately, and the confirm screen says so.
+Choosing Local keeps it on this machine. Other pre-connect runtime history (hook, OTLP
+and plugin events) is not uploaded. On the first connection, inventory is read from
+the beginning of `inventory_state.jsonl` so the hosted dashboard receives the
+endpoint's current inventory baseline.
The managed destination is recorded only once enrollment succeeds. If it fails, the
local install still succeeds and prints `beacon endpoint connect` as the retry
diff --git a/docs/cli/endpoint-paths.mdx b/docs/cli/endpoint-paths.mdx
index b4ac0cf7..1c5f9b27 100644
--- a/docs/cli/endpoint-paths.mdx
+++ b/docs/cli/endpoint-paths.mdx
@@ -12,6 +12,7 @@ Beacon manages different paths depending on whether you install in user mode or
| Config | `~/.beacon/endpoint/config.json` | `/Library/Application Support/Beacon/Endpoint/config.json` | `/etc/beacon/endpoint/config.json` | `%ProgramData%\Beacon\Endpoint\config.json` |
| Base directory | `~/.beacon/endpoint` | `/Library/Application Support/Beacon/Endpoint` | `/etc/beacon/endpoint` | `%ProgramData%\Beacon\Endpoint` |
| Runtime log | `~/.beacon/endpoint/logs/runtime.jsonl` | `/var/log/beacon-agent/runtime.jsonl` | `/var/log/beacon-agent/runtime.jsonl` | `%ProgramData%\Beacon\Endpoint\logs\runtime.jsonl` |
+| Staged Beacon Cloud session backfill (written by a first connect) | `~/.beacon/endpoint/logs/cloud-backfill.jsonl` | not written | not written | not written |
| Rotated runtime archives | `~/.beacon/endpoint/logs/runtime.jsonl.1` through `.5` | `/var/log/beacon-agent/runtime.jsonl.1` through `.5` | `/var/log/beacon-agent/runtime.jsonl.1` through `.5` | `%ProgramData%\Beacon\Endpoint\logs\runtime.jsonl.1` through `.5` |
| Threat rule store | `~/.beacon/endpoint/rules` | `/Library/Application Support/Beacon/Endpoint/rules` | `/etc/beacon/endpoint/rules` | `%ProgramData%\Beacon\Endpoint\rules` |
| Collector config | `~/.beacon/endpoint/otelcol.yaml` | `/Library/Application Support/Beacon/Endpoint/otelcol.yaml` | `/etc/beacon/endpoint/otelcol.yaml` | `%ProgramData%\Beacon\Endpoint\otelcol.yaml` |
diff --git a/docs/log-forwarding/asymptote.mdx b/docs/log-forwarding/asymptote.mdx
index 79a29c3c..fd216d91 100644
--- a/docs/log-forwarding/asymptote.mdx
+++ b/docs/log-forwarding/asymptote.mdx
@@ -14,8 +14,11 @@ Generate the pack with [`beacon endpoint asymptote`](/cli/asymptote). `beacon en
## What leaves the machine
For runtime telemetry, Standard mode forwards every line written after the
-forwarder starts exactly as Beacon wrote it locally; pre-connect runtime history is
-not sent. Inventory is read from the beginning of `inventory_state.jsonl` so a
+forwarder starts exactly as Beacon wrote it locally. Pre-connect runtime history is
+not sent, except the bounded [session backfill](/cli/endpoint-connect#session-backfill)
+a first user-mode connect stages: the last 30 days of events read from the agents'
+own session stores (at most 24 MiB), which go through the same privacy transform.
+Inventory is read from the beginning of `inventory_state.jsonl` so a
newly connected endpoint sends its current inventory baseline, including inventory
records written before connection. Metadata-only mode applies a local Vector
transform first, removing retained prompts, responses, reasoning, tool