Skip to content

The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu

add ci destination exports

Justin D'Souzacd3318cd4964
diff --git a/action.yml b/action.yml
index 24a80bf7..23978f49 100644
--- a/action.yml
+++ b/action.yml
@@ -53,8 +53,15 @@ inputs:
description: "SIEM HEC endpoint URL for the selected --forward provider."
required: false
default: ""
+ upload:
+ description: >-
+ Optional post-run artifact upload destinations: 's3', 'gcs', or a
+ comma-separated list. Configure provider credentials at the job level
+ using the normal AWS or Google Cloud CI mechanisms.
+ required: false
+ default: ""
upload-artifact:
- description: "Upload the runtime JSONL log as a workflow artifact."
+ description: "Upload the runtime JSONL log as the universal workflow artifact."
required: false
default: "true"
artifact-name:
@@ -145,6 +152,7 @@ runs:
INPUT_REQUIRE_TELEMETRY: ${{ inputs.require-telemetry }}
INPUT_FORWARD: ${{ inputs.forward }}
INPUT_FORWARD_ENDPOINT: ${{ inputs.forward-endpoint }}
+ INPUT_UPLOAD: ${{ inputs.upload }}
run: |
set -euo pipefail
@@ -165,6 +173,15 @@ runs:
args+=(--forward-endpoint "${INPUT_FORWARD_ENDPOINT}")
fi
fi
+ if [ -n "${INPUT_UPLOAD}" ]; then
+ IFS=',' read -ra upload_values <<< "${INPUT_UPLOAD}"
+ for upload in "${upload_values[@]}"; do
+ upload="${upload//[[:space:]]/}"
+ if [ -n "${upload}" ]; then
+ args+=(--upload "${upload}")
+ fi
+ done
+ fi
# INPUT_COMMAND is passed through the environment (not interpolated into
# this script) so workflow inputs cannot inject shell commands.
diff --git a/cli/beacon/README.md b/cli/beacon/README.md
index fa446f05..c0355c67 100644
--- a/cli/beacon/README.md
+++ b/cli/beacon/README.md
@@ -119,6 +119,24 @@ keeps the SIEM token in the runner environment and writes only collector
environment references into `otelcol.yaml`; excluding the collector config from
artifacts still avoids exposing forwarding endpoints or local collector details.
+The JSONL artifact is the universal CI export contract. Existing destination
+packs can consume the same file after it is downloaded or copied into the
+customer-managed forwarder path:
+
+| Destination | CI export path |
+| --- | --- |
+| Wazuh | Download the artifact and tail it with the Wazuh localfile pack. |
+| Elastic | Download the artifact and ingest it with Filebeat or Elastic Agent pack assets. |
+| Datadog | Download the artifact and tail it with the Datadog Agent log collection pack. |
+| Sumo Logic | Download the artifact for Vector-based forwarding, or use `--upload` when object storage is the handoff point. |
+| Rapid7 | Download the artifact for Vector/webhook forwarding. |
+| S3 | Use direct `--upload s3` in CI, or upload/download the artifact manually. |
+| CloudWatch Logs | Download the artifact and forward with the Vector CloudWatch pack. |
+| GCS | Use direct `--upload gcs` in CI, or upload/download the artifact manually. |
+| Sentinel | Download the artifact and ingest with the Azure Monitor Agent/DCR pack. |
+| Splunk | Use direct `--forward splunk`, or ingest the JSONL artifact with an existing Splunk pipeline. |
+| Falcon LogScale | Use direct `--forward falcon`, or ingest the JSONL artifact with an existing pipeline. |
+
### Forwarding to a customer-managed SIEM
Because CI runners are ephemeral, the local JSONL is destroyed when the runner
@@ -144,6 +162,31 @@ beacon ci exec \
- Beacon remains a local JSONL producer; egress goes only to infrastructure the
customer already operates.
+### Uploading CI JSONL to object storage
+
+For CI runners that should hand off telemetry through object storage, `ci exec`
+can upload the completed `runtime.jsonl` after telemetry validation:
+
+```bash
+export BEACON_CI_S3_BUCKET="my-beacon-telemetry"
+export BEACON_CI_S3_PREFIX="github-actions"
+beacon ci exec --upload s3 -- claude --print "Summarize this repository"
+```
+
+```bash
+export BEACON_CI_GCS_BUCKET="my-beacon-telemetry"
+export BEACON_CI_GCS_PREFIX="github-actions"
+beacon ci exec --upload gcs -- claude --print "Summarize this repository"
+```
+
+- S3 upload uses `aws s3 cp` and the standard AWS credential provider chain.
+- GCS upload uses `gcloud storage cp` when available, falling back to `gsutil cp`.
+- Beacon strips common AWS and Google credential environment variables from the
+ child agent process, while keeping them available to the post-run uploader.
+- Object keys default to
+ `<prefix>/<repository>/<run_id>/<run_attempt>/runtime.jsonl` in GitHub Actions,
+ with a timestamp fallback outside GitHub Actions.
+
By default `ci exec` fails the step when no Beacon events reach the runtime log,
which surfaces a broken telemetry pipeline. Pass `--require-telemetry=false` to
downgrade that to a warning when you do not want telemetry health to gate the
@@ -182,6 +225,25 @@ jobs:
forward-endpoint: https://splunk.example:8088/services/collector
```
+To upload the completed JSONL artifact to object storage, configure cloud
+credentials at the job level and set `upload`:
+
+```yaml
+- uses: aws-actions/configure-aws-credentials@v4
+ with:
+ role-to-assume: ${{ secrets.BEACON_TELEMETRY_ROLE_ARN }}
+ aws-region: us-east-1
+
+- name: Run Claude with Beacon telemetry
+ uses: asymptote-labs/[email protected]
+ env:
+ BEACON_CI_S3_BUCKET: my-beacon-telemetry
+ BEACON_CI_S3_PREFIX: github-actions
+ with:
+ command: claude --print "Summarize this repository"
+ upload: s3
+```
+
See [`examples/github-actions/claude-code-telemetry.yml`](../../examples/github-actions/claude-code-telemetry.yml)
for a complete reference workflow.
diff --git a/cli/beacon/cmd/ci.go b/cli/beacon/cmd/ci.go
index 60e0e00c..7ecc210a 100644
--- a/cli/beacon/cmd/ci.go
+++ b/cli/beacon/cmd/ci.go
@@ -6,6 +6,7 @@ import (
"fmt"
"os"
"os/signal"
+ "strings"
"syscall"
"time"
@@ -29,6 +30,7 @@ var ciOpts struct {
minEvents int
forward string
forwardEndpoint string
+ uploads []string
requireTelemetry bool
}
@@ -73,6 +75,7 @@ func init() {
ciExecCmd.Flags().BoolVar(&ciOpts.keepArtifacts, "keep-artifacts", true, "Keep CI runtime log and collector config after exit")
ciExecCmd.Flags().StringVar(&ciOpts.forward, "forward", "", "Optionally forward events to a customer-managed SIEM: splunk or falcon (token read from the environment)")
ciExecCmd.Flags().StringVar(&ciOpts.forwardEndpoint, "forward-endpoint", "", "SIEM HEC endpoint URL for --forward (token comes from BEACON_CI_*_HEC_TOKEN)")
+ ciExecCmd.Flags().StringArrayVar(&ciOpts.uploads, "upload", nil, "Upload final CI runtime JSONL after validation: s3 or gcs (repeatable)")
ciExecCmd.Flags().BoolVar(&ciOpts.requireTelemetry, "require-telemetry", true, "Fail the command when telemetry validation fails; set false to warn only")
for _, name := range []string{"base-dir", "work-dir", "collector", "otlp-grpc-port", "otlp-http-port"} {
_ = ciExecCmd.Flags().MarkHidden(name)
@@ -94,6 +97,7 @@ func runCIExec(cmd *cobra.Command, args []string) error {
KeepArtifacts: ciOpts.keepArtifacts,
Forward: ciOpts.forward,
ForwardEndpoint: ciOpts.forwardEndpoint,
+ Uploads: ciOpts.uploads,
})
if err != nil {
return err
@@ -123,6 +127,21 @@ func runCIExec(cmd *cobra.Command, args []string) error {
Validation: result,
ArtifactMessage: fmt.Sprintf("Beacon CI artifacts: log=%s config=%s", session.LogPath, session.ConfigPath),
}
+ if result.Status != "fail" && len(session.Uploads) > 0 {
+ uploadCtx, uploadCancel := context.WithTimeout(context.Background(), 2*time.Minute)
+ uploadResults, uploadErr := session.UploadArtifacts(uploadCtx)
+ uploadCancel()
+ execResult.Uploads = uploadResults
+ if uploadErr != nil {
+ if ciOpts.jsonOutput {
+ _ = json.NewEncoder(os.Stdout).Encode(execResult)
+ }
+ return fmt.Errorf("Beacon CI upload failed: %w", uploadErr)
+ }
+ if len(uploadResults) > 0 {
+ execResult.ArtifactMessage += fmt.Sprintf(" uploads=%s", uploadTargets(uploadResults))
+ }
+ }
if !ciOpts.keepArtifacts && result.Status != "fail" && childExit == 0 && ciOpts.baseDir == "" && ciOpts.logPath == "" {
if err := os.RemoveAll(session.BaseDir); err != nil {
fmt.Fprintf(os.Stderr, "Warning: artifact cleanup: %v\n", err)
@@ -152,6 +171,16 @@ func runCIExec(cmd *cobra.Command, args []string) error {
return nil
}
+func uploadTargets(results []beaconci.UploadResult) string {
+ targets := make([]string, 0, len(results))
+ for _, result := range results {
+ if result.Target != "" {
+ targets = append(targets, result.Target)
+ }
+ }
+ return strings.Join(targets, ",")
+}
+
func runCIValidate(cmd *cobra.Command, args []string) error {
logPath := ciOpts.logPath
if logPath == "" {
diff --git a/cli/beacon/cmd/ci_test.go b/cli/beacon/cmd/ci_test.go
index 26117e52..bf49b45b 100644
--- a/cli/beacon/cmd/ci_test.go
+++ b/cli/beacon/cmd/ci_test.go
@@ -1,6 +1,10 @@
package cmd
-import "testing"
+import (
+ "os"
+ "strings"
+ "testing"
+)
func TestCICommandsRegistered(t *testing.T) {
for _, path := range [][]string{
@@ -24,7 +28,7 @@ func TestCICommandsRegistered(t *testing.T) {
t.Fatalf("ci validate missing --%s", name)
}
}
- for _, name := range []string{"content-retention", "keep-artifacts"} {
+ for _, name := range []string{"content-retention", "keep-artifacts", "upload"} {
if ciExecCmd.Flags().Lookup(name) == nil {
t.Fatalf("ci exec missing --%s", name)
}
@@ -43,6 +47,19 @@ func TestCICommandsRegistered(t *testing.T) {
}
}
+func TestCIActionExposesUploadInput(t *testing.T) {
+ data, err := os.ReadFile("../../../action.yml")
+ if err != nil {
+ t.Fatalf("read action.yml: %v", err)
+ }
+ text := string(data)
+ for _, want := range []string{" upload:", "INPUT_UPLOAD", "--upload"} {
+ if !strings.Contains(text, want) {
+ t.Fatalf("action.yml missing %q", want)
+ }
+ }
+}
+
func TestCIHarnessDefaultIsClaudeOnly(t *testing.T) {
flag := ciExecCmd.Flags().Lookup("harness")
if flag == nil {
diff --git a/cli/beacon/internal/ci/session.go b/cli/beacon/internal/ci/session.go
index 69a67132..d29e8e64 100644
--- a/cli/beacon/internal/ci/session.go
+++ b/cli/beacon/internal/ci/session.go
@@ -47,6 +47,7 @@ type Options struct {
// token is read from the environment only.
Forward string
ForwardEndpoint string
+ Uploads []string
}
type Session struct {
@@ -62,8 +63,9 @@ type Session struct {
Run *schema.RunInfo `json:"run,omitempty"`
// Forward and ForwardEndpoint describe the optional SIEM egress. The token
// is never recorded here so it cannot leak into --json output.
- Forward string `json:"forward,omitempty"`
- ForwardEndpoint string `json:"forward_endpoint,omitempty"`
+ Forward string `json:"forward,omitempty"`
+ ForwardEndpoint string `json:"forward_endpoint,omitempty"`
+ Uploads []UploadDestination `json:"uploads,omitempty"`
cfg endpointconfig.Config
startedAt time.Time
@@ -75,6 +77,7 @@ type ExecResult struct {
Session Session `json:"session"`
ChildExitCode int `json:"child_exit_code"`
Validation ValidationResult `json:"validation"`
+ Uploads []UploadResult `json:"uploads,omitempty"`
ArtifactMessage string `json:"artifact_message,omitempty"`
}
@@ -134,6 +137,12 @@ func Provision(opts Options) (*Session, error) {
// environment and can resolve them; the child has these vars stripped.
replaceTokensWithEnvRefs(&cfg)
}
+ startedAt := time.Now().UTC()
+ run := detectRunInfo()
+ uploads, err := resolveUploadDestinations(opts.Uploads, run, logPath, startedAt)
+ if err != nil {
+ return nil, err
+ }
if err := os.MkdirAll(filepath.Dir(logPath), 0755); err != nil {
return nil, err
}
@@ -151,7 +160,6 @@ func Provision(opts Options) (*Session, error) {
if err != nil {
return nil, err
}
- startedAt := time.Now().UTC()
return &Session{
BaseDir: baseDir,
LogPath: logPath,
@@ -162,9 +170,10 @@ func Provision(opts Options) (*Session, error) {
Harness: DefaultHarness,
WorkDir: opts.WorkDir,
StartedAt: startedAt.Format(time.RFC3339),
- Run: detectRunInfo(),
+ Run: run,
Forward: forward,
ForwardEndpoint: forwardEndpoint,
+ Uploads: uploads,
cfg: cfg,
startedAt: startedAt,
}, nil
@@ -248,8 +257,12 @@ func (s *Session) RunChild(ctx context.Context, args []string, stdout, stderr io
"BEACON_CI_CONFIG_PATH="+s.ConfigPath,
"BEACON_CI_LOG_PATH="+s.LogPath,
)
- // The agent never needs the SIEM token; keep it out of the child process.
- cmd.Env = stripEnv(env, forwardTokenEnv...)
+ // The agent never needs destination credentials; keep them out of the child process.
+ stripKeys := append([]string{}, forwardTokenEnv...)
+ if len(s.Uploads) > 0 {
+ stripKeys = append(stripKeys, uploadCredentialEnv...)
+ }
+ cmd.Env = stripEnv(env, stripKeys...)
cmd.Stdout = stdout
cmd.Stderr = stderr
err := cmd.Run()
diff --git a/cli/beacon/internal/ci/session_test.go b/cli/beacon/internal/ci/session_test.go
index e77fe8c1..c32402a6 100644
--- a/cli/beacon/internal/ci/session_test.go
+++ b/cli/beacon/internal/ci/session_test.go
@@ -392,6 +392,42 @@ func TestRunChildStripsForwardToken(t *testing.T) {
}
}
+func TestRunChildStripsUploadCredentialsWhenUploadConfigured(t *testing.T) {
+ dir := t.TempDir()
+ output := filepath.Join(dir, "env.txt")
+ t.Setenv("AWS_ACCESS_KEY_ID", "aws-key")
+ t.Setenv("AWS_SECRET_ACCESS_KEY", "aws-secret")
+ t.Setenv("AWS_SESSION_TOKEN", "aws-session")
+ t.Setenv("GOOGLE_APPLICATION_CREDENTIALS", "/tmp/google-creds.json")
+ child := fakeExecutable(t, "child", "#!/bin/sh\nenv > \"$1\"\n")
+ session := &Session{
+ BaseDir: dir,
+ ConfigPath: filepath.Join(dir, "otelcol.yaml"),
+ LogPath: filepath.Join(dir, "runtime.jsonl"),
+ GRPCEndpoint: "http://127.0.0.1:4317",
+ Uploads: []UploadDestination{
+ {Provider: UploadS3, URI: "s3://bucket/key"},
+ },
+ cfg: endpointconfig.Default(true, filepath.Join(dir, "runtime.jsonl")),
+ }
+ if _, err := session.RunChild(context.Background(), []string{child, output}, nil, nil); err != nil {
+ t.Fatalf("RunChild returned error: %v", err)
+ }
+ data, err := os.ReadFile(output)
+ if err != nil {
+ t.Fatal(err)
+ }
+ env := string(data)
+ for _, leaked := range []string{"AWS_ACCESS_KEY_ID=", "AWS_SECRET_ACCESS_KEY=", "AWS_SESSION_TOKEN=", "GOOGLE_APPLICATION_CREDENTIALS=", "aws-secret", "/tmp/google-creds.json"} {
+ if strings.Contains(env, leaked) {
+ t.Fatalf("child env exposed upload credential %q:\n%s", leaked, env)
+ }
+ }
+ if !strings.Contains(env, "CLAUDE_CODE_ENABLE_TELEMETRY=1") {
+ t.Fatalf("child env missing Claude telemetry vars:\n%s", env)
+ }
+}
+
func fakeExecutable(t *testing.T, name, script string) string {
t.Helper()
path := filepath.Join(t.TempDir(), name)
diff --git a/cli/beacon/internal/ci/upload.go b/cli/beacon/internal/ci/upload.go
new file mode 100644
index 00000000..54d0b593
--- /dev/null
+++ b/cli/beacon/internal/ci/upload.go
@@ -0,0 +1,234 @@
+package ci
+
+import (
+ "context"
+ "fmt"
+ "os"
+ "os/exec"
+ "path"
+ "path/filepath"
+ "strings"
+ "time"
+
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
+)
+
+const (
+ UploadS3 = "s3"
+ UploadGCS = "gcs"
+)
+
+const (
+ EnvS3Bucket = "BEACON_CI_S3_BUCKET"
+ EnvS3Prefix = "BEACON_CI_S3_PREFIX"
+ EnvGCSBucket = "BEACON_CI_GCS_BUCKET"
+ EnvGCSPrefix = "BEACON_CI_GCS_PREFIX"
+)
+
+var uploadCredentialEnv = []string{
+ "AWS_ACCESS_KEY_ID",
+ "AWS_SECRET_ACCESS_KEY",
+ "AWS_SESSION_TOKEN",
+ "AWS_PROFILE",
+ "AWS_WEB_IDENTITY_TOKEN_FILE",
+ "GOOGLE_APPLICATION_CREDENTIALS",
+ "CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE",
+}
+
+var (
+ uploadLookPath = exec.LookPath
+ runUploadCommand = func(ctx context.Context, name string, args ...string) ([]byte, error) {
+ return exec.CommandContext(ctx, name, args...).CombinedOutput()
+ }
+)
+
+type UploadDestination struct {
+ Provider string `json:"provider"`
+ Bucket string `json:"bucket"`
+ Key string `json:"key"`
+ URI string `json:"uri"`
+}
+
+type UploadResult struct {
+ Provider string `json:"provider"`
+ Target string `json:"target"`
+ Status string `json:"status"`
+ Message string `json:"message,omitempty"`
+}
+
+func resolveUploadDestinations(providers []string, run *schema.RunInfo, logPath string, startedAt time.Time) ([]UploadDestination, error) {
+ normalized, err := normalizeUploads(providers)
+ if err != nil {
+ return nil, err
+ }
+ if len(normalized) == 0 {
+ return nil, nil
+ }
+ destinations := make([]UploadDestination, 0, len(normalized))
+ for _, provider := range normalized {
+ switch provider {
+ case UploadS3:
+ dest, err := uploadDestination(provider, os.Getenv(EnvS3Bucket), os.Getenv(EnvS3Prefix), run, logPath, startedAt)
+ if err != nil {
+ return nil, err
+ }
+ destinations = append(destinations, dest)
+ case UploadGCS:
+ dest, err := uploadDestination(provider, os.Getenv(EnvGCSBucket), os.Getenv(EnvGCSPrefix), run, logPath, startedAt)
+ if err != nil {
+ return nil, err
+ }
+ destinations = append(destinations, dest)
+ }
+ }
+ return destinations, nil
+}
+
+func normalizeUploads(values []string) ([]string, error) {
+ seen := map[string]struct{}{}
+ var providers []string
+ for _, value := range values {
+ for _, part := range strings.Split(value, ",") {
+ provider := strings.ToLower(strings.TrimSpace(part))
+ if provider == "" {
+ continue
+ }
+ switch provider {
+ case UploadS3, UploadGCS:
+ if _, ok := seen[provider]; ok {
+ continue
+ }
+ seen[provider] = struct{}{}
+ providers = append(providers, provider)
+ default:
+ return nil, fmt.Errorf("unsupported --upload %q; supported values are %s and %s", part, UploadS3, UploadGCS)
+ }
+ }
+ }
+ return providers, nil
+}
+
+func uploadDestination(provider, bucket, prefix string, run *schema.RunInfo, logPath string, startedAt time.Time) (UploadDestination, error) {
+ bucket = strings.TrimSpace(bucket)
+ if bucket == "" {
+ return UploadDestination{}, fmt.Errorf("--upload %s requires a bucket; set %s", provider, uploadBucketEnv(provider))
+ }
+ key := uploadObjectKey(prefix, run, logPath, startedAt)
+ uri := fmt.Sprintf("%s://%s/%s", uploadScheme(provider), bucket, key)
+ return UploadDestination{Provider: provider, Bucket: bucket, Key: key, URI: uri}, nil
+}
+
+func uploadObjectKey(prefix string, run *schema.RunInfo, logPath string, startedAt time.Time) string {
+ name := filepath.Base(logPath)
+ if strings.TrimSpace(name) == "" || name == "." || name == string(filepath.Separator) {
+ name = "runtime.jsonl"
+ }
+ parts := cleanKeyParts(prefix)
+ if run != nil && strings.TrimSpace(run.Repository) != "" && strings.TrimSpace(run.RunID) != "" {
+ parts = append(parts, cleanKeyParts(run.Repository)...)
+ parts = append(parts, cleanKeyPart(run.RunID))
+ if strings.TrimSpace(run.RunAttempt) != "" {
+ parts = append(parts, cleanKeyPart(run.RunAttempt))
+ }
+ } else {
+ if startedAt.IsZero() {
+ startedAt = time.Now().UTC()
+ }
+ parts = append(parts, "ci", fmt.Sprintf("%d", startedAt.UTC().UnixNano()))
+ }
+ parts = append(parts, cleanKeyPart(name))
+ return path.Join(parts...)
+}
+
+func cleanKeyParts(value string) []string {
+ var parts []string
+ for _, part := range strings.Split(strings.Trim(value, "/"), "/") {
+ if cleaned := cleanKeyPart(part); cleaned != "" {
+ parts = append(parts, cleaned)
+ }
+ }
+ return parts
+}
+
+func cleanKeyPart(value string) string {
+ cleaned := strings.TrimSpace(value)
+ cleaned = strings.ReplaceAll(cleaned, "\\", "-")
+ cleaned = strings.Trim(cleaned, "/")
+ if cleaned == "." || cleaned == ".." {
+ return ""
+ }
+ return cleaned
+}
+
+func uploadBucketEnv(provider string) string {
+ if provider == UploadGCS {
+ return EnvGCSBucket
+ }
+ return EnvS3Bucket
+}
+
+func uploadScheme(provider string) string {
+ if provider == UploadGCS {
+ return "gs"
+ }
+ return "s3"
+}
+
+func (s *Session) UploadArtifacts(ctx context.Context) ([]UploadResult, error) {
+ if s == nil || len(s.Uploads) == 0 {
+ return nil, nil
+ }
+ results := make([]UploadResult, 0, len(s.Uploads))
+ for _, dest := range s.Uploads {
+ result, err := uploadArtifact(ctx, s.LogPath, dest)
+ results = append(results, result)
+ if err != nil {
+ return results, err
+ }
+ }
+ return results, nil
+}
+
+func uploadArtifact(ctx context.Context, logPath string, dest UploadDestination) (UploadResult, error) {
+ var name string
+ var args []string
+ switch dest.Provider {
+ case UploadS3:
+ if _, err := uploadLookPath("aws"); err != nil {
+ return uploadFailure(dest, "aws CLI not found in PATH"), fmt.Errorf("--upload s3 requires the aws CLI in PATH")
+ }
+ name = "aws"
+ args = []string{"s3", "cp", logPath, dest.URI}
+ case UploadGCS:
+ switch {
+ case commandAvailable("gcloud"):
+ name = "gcloud"
+ args = []string{"storage", "cp", logPath, dest.URI}
+ case commandAvailable("gsutil"):
+ name = "gsutil"
+ args = []string{"cp", logPath, dest.URI}
+ default:
+ return uploadFailure(dest, "gcloud or gsutil not found in PATH"), fmt.Errorf("--upload gcs requires gcloud or gsutil in PATH")
+ }
+ default:
+ return uploadFailure(dest, "unsupported provider"), fmt.Errorf("unsupported upload provider %q", dest.Provider)
+ }
+ output, err := runUploadCommand(ctx, name, args...)
+ if err != nil {
+ message := strings.TrimSpace(string(output))
+ if message == "" {
+ message = err.Error()
+ }
+ return uploadFailure(dest, message), fmt.Errorf("upload %s to %s failed: %w", dest.Provider, dest.URI, err)
+ }
+ return UploadResult{Provider: dest.Provider, Target: dest.URI, Status: "ok"}, nil
+}
+
+func uploadFailure(dest UploadDestination, message string) UploadResult {
+ return UploadResult{Provider: dest.Provider, Target: dest.URI, Status: "fail", Message: message}
+}
+
+func commandAvailable(name string) bool {
+ _, err := uploadLookPath(name)
+ return err == nil
+}
diff --git a/cli/beacon/internal/ci/upload_test.go b/cli/beacon/internal/ci/upload_test.go
new file mode 100644
index 00000000..60d9e910
--- /dev/null
+++ b/cli/beacon/internal/ci/upload_test.go
@@ -0,0 +1,174 @@
+package ci
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "os/exec"
+ "path/filepath"
+ "reflect"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"
+)
+
+func TestNormalizeUploadsSplitsAndDedupes(t *testing.T) {
+ got, err := normalizeUploads([]string{"s3,gcs", "s3"})
+ if err != nil {
+ t.Fatalf("normalizeUploads returned error: %v", err)
+ }
+ if want := []string{UploadS3, UploadGCS}; !reflect.DeepEqual(got, want) {
+ t.Fatalf("normalizeUploads = %#v, want %#v", got, want)
+ }
+}
+
+func TestNormalizeUploadsRejectsUnsupportedProvider(t *testing.T) {
+ if _, err := normalizeUploads([]string{"elastic"}); err == nil || !strings.Contains(err.Error(), "unsupported --upload") {
+ t.Fatalf("normalizeUploads error = %v, want unsupported provider", err)
+ }
+}
+
+func TestResolveUploadDestinationsRequiresBucket(t *testing.T) {
+ t.Setenv(EnvS3Bucket, "")
+ _, err := resolveUploadDestinations([]string{"s3"}, nil, "/tmp/runtime.jsonl", time.Unix(1700000000, 0).UTC())
+ if err == nil || !strings.Contains(err.Error(), EnvS3Bucket) {
+ t.Fatalf("resolveUploadDestinations error = %v, want missing bucket", err)
+ }
+}
+
+func TestResolveUploadDestinationsBuildsGitHubObjectKeys(t *testing.T) {
+ t.Setenv(EnvS3Bucket, "beacon-ci")
+ t.Setenv(EnvS3Prefix, "logs/")
+ run := &schema.RunInfo{
+ Repository: "asymptote-labs/agent-beacon",
+ RunID: "123",
+ RunAttempt: "2",
+ }
+ destinations, err := resolveUploadDestinations([]string{"s3"}, run, "/tmp/runtime.jsonl", time.Unix(1700000000, 0).UTC())
+ if err != nil {
+ t.Fatalf("resolveUploadDestinations returned error: %v", err)
+ }
+ if len(destinations) != 1 {
+ t.Fatalf("destinations = %d, want 1", len(destinations))
+ }
+ dest := destinations[0]
+ if dest.Key != "logs/asymptote-labs/agent-beacon/123/2/runtime.jsonl" {
+ t.Fatalf("Key = %q", dest.Key)
+ }
+ if dest.URI != "s3://beacon-ci/logs/asymptote-labs/agent-beacon/123/2/runtime.jsonl" {
+ t.Fatalf("URI = %q", dest.URI)
+ }
+}
+
+func TestResolveUploadDestinationsBuildsFallbackObjectKeys(t *testing.T) {
+ t.Setenv(EnvGCSBucket, "beacon-ci")
+ startedAt := time.Unix(0, 42).UTC()
+ destinations, err := resolveUploadDestinations([]string{"gcs"}, nil, "/tmp/runtime.jsonl", startedAt)
+ if err != nil {
+ t.Fatalf("resolveUploadDestinations returned error: %v", err)
+ }
+ if got, want := destinations[0].Key, "ci/42/runtime.jsonl"; got != want {
+ t.Fatalf("Key = %q, want %q", got, want)
+ }
+ if got, want := destinations[0].URI, "gs://beacon-ci/ci/42/runtime.jsonl"; got != want {
+ t.Fatalf("URI = %q, want %q", got, want)
+ }
+}
+
+func TestUploadArtifactRunsS3Command(t *testing.T) {
+ calls, restore := stubUploadCommands(t, map[string]bool{"aws": true}, nil)
+ defer restore()
+
+ dest := UploadDestination{Provider: UploadS3, URI: "s3://bucket/key"}
+ result, err := uploadArtifact(context.Background(), "/tmp/runtime.jsonl", dest)
+ if err != nil {
+ t.Fatalf("uploadArtifact returned error: %v", err)
+ }
+ if result.Status != "ok" || result.Target != dest.URI {
+ t.Fatalf("result = %#v", result)
+ }
+ if got, want := strings.Join(*calls, "\n"), "aws s3 cp /tmp/runtime.jsonl s3://bucket/key"; got != want {
+ t.Fatalf("upload command = %q, want %q", got, want)
+ }
+}
+
+func TestUploadArtifactRunsGCloudCommand(t *testing.T) {
+ calls, restore := stubUploadCommands(t, map[string]bool{"gcloud": true}, nil)
+ defer restore()
+
+ dest := UploadDestination{Provider: UploadGCS, URI: "gs://bucket/key"}
+ if _, err := uploadArtifact(context.Background(), "/tmp/runtime.jsonl", dest); err != nil {
+ t.Fatalf("uploadArtifact returned error: %v", err)
+ }
+ if got, want := strings.Join(*calls, "\n"), "gcloud storage cp /tmp/runtime.jsonl gs://bucket/key"; got != want {
+ t.Fatalf("upload command = %q, want %q", got, want)
+ }
+}
+
+func TestUploadArtifactFallsBackToGsutil(t *testing.T) {
+ calls, restore := stubUploadCommands(t, map[string]bool{"gsutil": true}, nil)
+ defer restore()
+
+ dest := UploadDestination{Provider: UploadGCS, URI: "gs://bucket/key"}
+ if _, err := uploadArtifact(context.Background(), "/tmp/runtime.jsonl", dest); err != nil {
+ t.Fatalf("uploadArtifact returned error: %v", err)
+ }
+ if got, want := strings.Join(*calls, "\n"), "gsutil cp /tmp/runtime.jsonl gs://bucket/key"; got != want {
+ t.Fatalf("upload command = %q, want %q", got, want)
+ }
+}
+
+func TestUploadArtifactMissingCLI(t *testing.T) {
+ _, restore := stubUploadCommands(t, nil, nil)
+ defer restore()
+
+ result, err := uploadArtifact(context.Background(), "/tmp/runtime.jsonl", UploadDestination{Provider: UploadS3, URI: "s3://bucket/key"})
+ if err == nil || !strings.Contains(err.Error(), "aws CLI") {
+ t.Fatalf("uploadArtifact error = %v, want missing aws CLI", err)
+ }
+ if result.Status != "fail" {
+ t.Fatalf("result status = %q, want fail", result.Status)
+ }
+}
+
+func TestUploadArtifactCommandFailure(t *testing.T) {
+ _, restore := stubUploadCommands(t, map[string]bool{"aws": true}, errors.New("boom"))
+ defer restore()
+
+ result, err := uploadArtifact(context.Background(), "/tmp/runtime.jsonl", UploadDestination{Provider: UploadS3, URI: "s3://bucket/key"})
+ if err == nil || !strings.Contains(err.Error(), "upload s3") {
+ t.Fatalf("uploadArtifact error = %v, want upload failure", err)
+ }
+ if result.Status != "fail" || result.Message != "boom" {
+ t.Fatalf("result = %#v", result)
+ }
+}
+
+func stubUploadCommands(t *testing.T, available map[string]bool, commandErr error) (*[]string, func()) {
+ t.Helper()
+ oldLookPath := uploadLookPath
+ oldRun := runUploadCommand
+ var calls []string
+ uploadLookPath = func(name string) (string, error) {
+ if available[name] {
+ return filepath.Join("/bin", name), nil
+ }
+ return "", exec.ErrNotFound
+ }
+ runUploadCommand = func(ctx context.Context, name string, args ...string) ([]byte, error) {
+ calls = append(calls, strings.Join(append([]string{name}, args...), " "))
+ if commandErr != nil {
+ return []byte(commandErr.Error()), commandErr
+ }
+ return nil, nil
+ }
+ return &calls, func() {
+ if t.Failed() {
+ t.Logf("upload command calls: %s", fmt.Sprint(calls))
+ }
+ uploadLookPath = oldLookPath
+ runUploadCommand = oldRun
+ }
+}
diff --git a/examples/github-actions/claude-code-telemetry.yml b/examples/github-actions/claude-code-telemetry.yml
index 9411a74c..59c4f0df 100644
--- a/examples/github-actions/claude-code-telemetry.yml
+++ b/examples/github-actions/claude-code-telemetry.yml
@@ -1,6 +1,6 @@
# Reference workflow for capturing Claude Code telemetry in CI with the
# Beacon composite action. Copy this into .github/workflows/ in your own
-# repository and adjust the command and forwarding settings.
+# repository and adjust the command, artifact, forwarding, or upload settings.
#
# This file lives under examples/ (not .github/workflows/) so it is a
# reference only and does not run in this repository.
@@ -14,10 +14,13 @@ on:
jobs:
claude-telemetry:
runs-on: ubuntu-latest
- # Set the SIEM token at the job level (from a secret) so the ephemeral
- # collector can resolve it. Omit this block if you only upload an artifact.
+ # Set destination config at the job level so Beacon can use it without
+ # exposing secrets to the Claude child process. Omit this block if you only
+ # upload the default GitHub artifact.
env:
BEACON_CI_SPLUNK_HEC_TOKEN: ${{ secrets.SPLUNK_HEC_TOKEN }}
+ BEACON_CI_S3_BUCKET: ${{ vars.BEACON_CI_S3_BUCKET }}
+ BEACON_CI_S3_PREFIX: github-actions
steps:
- uses: actions/checkout@v4
@@ -26,6 +29,12 @@ jobs:
- name: Install Claude Code
run: npm install -g @anthropic-ai/claude-code
+ # Optional: configure cloud credentials before using upload: s3.
+ # - uses: aws-actions/configure-aws-credentials@v4
+ # with:
+ # role-to-assume: ${{ secrets.BEACON_TELEMETRY_ROLE_ARN }}
+ # aws-region: us-east-1
+
- name: Run Claude with Beacon telemetry
uses: asymptote-labs/[email protected]
env:
@@ -36,5 +45,8 @@ jobs:
# Requires a Beacon release that includes ci exec --forward.
forward: splunk
forward-endpoint: https://splunk.example:8088/services/collector
+ # Optional: upload the completed runtime JSONL to object storage.
+ # Supported values: s3, gcs, or s3,gcs.
+ # upload: s3
# The runtime log is uploaded as an artifact by default.
upload-artifact: "true"