The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu
Go agent-ai-cliagent-harnessesagent-securitybrowser-extensionciclaude-codecloudcodexcursordetection-engineeringjevknowledge-basememorymobile-device-managementobservabilitysecuritysecurity-information-and-event-managementsecurity-toolstelemetrytraces
add ci destination exports
Justin D'Souzacd3318cd4964
diff --git a/action.yml b/action.ymlindex 24a80bf7..23978f49 100644--- a/action.yml+++ b/action.yml@@ -53,8 +53,15 @@ inputs:description: "SIEM HEC endpoint URL for the selected --forward provider."required: falsedefault: ""+ upload:+ description: >-+ Optional post-run artifact upload destinations: 's3', 'gcs', or a+ comma-separated list. Configure provider credentials at the job level+ using the normal AWS or Google Cloud CI mechanisms.+ required: false+ default: ""upload-artifact:- description: "Upload the runtime JSONL log as a workflow artifact."+ description: "Upload the runtime JSONL log as the universal workflow artifact."required: falsedefault: "true"artifact-name:@@ -145,6 +152,7 @@ runs:INPUT_REQUIRE_TELEMETRY: ${{ inputs.require-telemetry }}INPUT_FORWARD: ${{ inputs.forward }}INPUT_FORWARD_ENDPOINT: ${{ inputs.forward-endpoint }}+ INPUT_UPLOAD: ${{ inputs.upload }}run: |set -euo pipefail@@ -165,6 +173,15 @@ runs:args+=(--forward-endpoint "${INPUT_FORWARD_ENDPOINT}")fifi+ if [ -n "${INPUT_UPLOAD}" ]; then+ IFS=',' read -ra upload_values <<< "${INPUT_UPLOAD}"+ for upload in "${upload_values[@]}"; do+ upload="${upload//[[:space:]]/}"+ if [ -n "${upload}" ]; then+ args+=(--upload "${upload}")+ fi+ done+ fi# INPUT_COMMAND is passed through the environment (not interpolated into# this script) so workflow inputs cannot inject shell commands.diff --git a/cli/beacon/README.md b/cli/beacon/README.mdindex fa446f05..c0355c67 100644--- a/cli/beacon/README.md+++ b/cli/beacon/README.md@@ -119,6 +119,24 @@ keeps the SIEM token in the runner environment and writes only collectorenvironment references into `otelcol.yaml`; excluding the collector config fromartifacts still avoids exposing forwarding endpoints or local collector details.+The JSONL artifact is the universal CI export contract. Existing destination+packs can consume the same file after it is downloaded or copied into the+customer-managed forwarder path:++| Destination | CI export path |+| --- | --- |+| Wazuh | Download the artifact and tail it with the Wazuh localfile pack. |+| Elastic | Download the artifact and ingest it with Filebeat or Elastic Agent pack assets. |+| Datadog | Download the artifact and tail it with the Datadog Agent log collection pack. |+| Sumo Logic | Download the artifact for Vector-based forwarding, or use `--upload` when object storage is the handoff point. |+| Rapid7 | Download the artifact for Vector/webhook forwarding. |+| S3 | Use direct `--upload s3` in CI, or upload/download the artifact manually. |+| CloudWatch Logs | Download the artifact and forward with the Vector CloudWatch pack. |+| GCS | Use direct `--upload gcs` in CI, or upload/download the artifact manually. |+| Sentinel | Download the artifact and ingest with the Azure Monitor Agent/DCR pack. |+| Splunk | Use direct `--forward splunk`, or ingest the JSONL artifact with an existing Splunk pipeline. |+| Falcon LogScale | Use direct `--forward falcon`, or ingest the JSONL artifact with an existing pipeline. |+### Forwarding to a customer-managed SIEMBecause CI runners are ephemeral, the local JSONL is destroyed when the runner@@ -144,6 +162,31 @@ beacon ci exec \- Beacon remains a local JSONL producer; egress goes only to infrastructure thecustomer already operates.+### Uploading CI JSONL to object storage++For CI runners that should hand off telemetry through object storage, `ci exec`+can upload the completed `runtime.jsonl` after telemetry validation:++```bash+export BEACON_CI_S3_BUCKET="my-beacon-telemetry"+export BEACON_CI_S3_PREFIX="github-actions"+beacon ci exec --upload s3 -- claude --print "Summarize this repository"+```++```bash+export BEACON_CI_GCS_BUCKET="my-beacon-telemetry"+export BEACON_CI_GCS_PREFIX="github-actions"+beacon ci exec --upload gcs -- claude --print "Summarize this repository"+```++- S3 upload uses `aws s3 cp` and the standard AWS credential provider chain.+- GCS upload uses `gcloud storage cp` when available, falling back to `gsutil cp`.+- Beacon strips common AWS and Google credential environment variables from the+ child agent process, while keeping them available to the post-run uploader.+- Object keys default to+ `<prefix>/<repository>/<run_id>/<run_attempt>/runtime.jsonl` in GitHub Actions,+ with a timestamp fallback outside GitHub Actions.+By default `ci exec` fails the step when no Beacon events reach the runtime log,which surfaces a broken telemetry pipeline. Pass `--require-telemetry=false` todowngrade that to a warning when you do not want telemetry health to gate the@@ -182,6 +225,25 @@ jobs:forward-endpoint: https://splunk.example:8088/services/collector```+To upload the completed JSONL artifact to object storage, configure cloud+credentials at the job level and set `upload`:++```yaml+- uses: aws-actions/configure-aws-credentials@v4+ with:+ role-to-assume: ${{ secrets.BEACON_TELEMETRY_ROLE_ARN }}+ aws-region: us-east-1++- name: Run Claude with Beacon telemetry+ uses: asymptote-labs/[email protected]+ env:+ BEACON_CI_S3_BUCKET: my-beacon-telemetry+ BEACON_CI_S3_PREFIX: github-actions+ with:+ command: claude --print "Summarize this repository"+ upload: s3+```+See [`examples/github-actions/claude-code-telemetry.yml`](../../examples/github-actions/claude-code-telemetry.yml)for a complete reference workflow.diff --git a/cli/beacon/cmd/ci.go b/cli/beacon/cmd/ci.goindex 60e0e00c..7ecc210a 100644--- a/cli/beacon/cmd/ci.go+++ b/cli/beacon/cmd/ci.go@@ -6,6 +6,7 @@ import ("fmt""os""os/signal"+ "strings""syscall""time"@@ -29,6 +30,7 @@ var ciOpts struct {minEvents intforward stringforwardEndpoint string+ uploads []stringrequireTelemetry bool}@@ -73,6 +75,7 @@ func init() {ciExecCmd.Flags().BoolVar(&ciOpts.keepArtifacts, "keep-artifacts", true, "Keep CI runtime log and collector config after exit")ciExecCmd.Flags().StringVar(&ciOpts.forward, "forward", "", "Optionally forward events to a customer-managed SIEM: splunk or falcon (token read from the environment)")ciExecCmd.Flags().StringVar(&ciOpts.forwardEndpoint, "forward-endpoint", "", "SIEM HEC endpoint URL for --forward (token comes from BEACON_CI_*_HEC_TOKEN)")+ ciExecCmd.Flags().StringArrayVar(&ciOpts.uploads, "upload", nil, "Upload final CI runtime JSONL after validation: s3 or gcs (repeatable)")ciExecCmd.Flags().BoolVar(&ciOpts.requireTelemetry, "require-telemetry", true, "Fail the command when telemetry validation fails; set false to warn only")for _, name := range []string{"base-dir", "work-dir", "collector", "otlp-grpc-port", "otlp-http-port"} {_ = ciExecCmd.Flags().MarkHidden(name)@@ -94,6 +97,7 @@ func runCIExec(cmd *cobra.Command, args []string) error {KeepArtifacts: ciOpts.keepArtifacts,Forward: ciOpts.forward,ForwardEndpoint: ciOpts.forwardEndpoint,+ Uploads: ciOpts.uploads,})if err != nil {return err@@ -123,6 +127,21 @@ func runCIExec(cmd *cobra.Command, args []string) error {Validation: result,ArtifactMessage: fmt.Sprintf("Beacon CI artifacts: log=%s config=%s", session.LogPath, session.ConfigPath),}+ if result.Status != "fail" && len(session.Uploads) > 0 {+ uploadCtx, uploadCancel := context.WithTimeout(context.Background(), 2*time.Minute)+ uploadResults, uploadErr := session.UploadArtifacts(uploadCtx)+ uploadCancel()+ execResult.Uploads = uploadResults+ if uploadErr != nil {+ if ciOpts.jsonOutput {+ _ = json.NewEncoder(os.Stdout).Encode(execResult)+ }+ return fmt.Errorf("Beacon CI upload failed: %w", uploadErr)+ }+ if len(uploadResults) > 0 {+ execResult.ArtifactMessage += fmt.Sprintf(" uploads=%s", uploadTargets(uploadResults))+ }+ }if !ciOpts.keepArtifacts && result.Status != "fail" && childExit == 0 && ciOpts.baseDir == "" && ciOpts.logPath == "" {if err := os.RemoveAll(session.BaseDir); err != nil {fmt.Fprintf(os.Stderr, "Warning: artifact cleanup: %v\n", err)@@ -152,6 +171,16 @@ func runCIExec(cmd *cobra.Command, args []string) error {return nil}+func uploadTargets(results []beaconci.UploadResult) string {+ targets := make([]string, 0, len(results))+ for _, result := range results {+ if result.Target != "" {+ targets = append(targets, result.Target)+ }+ }+ return strings.Join(targets, ",")+}+func runCIValidate(cmd *cobra.Command, args []string) error {logPath := ciOpts.logPathif logPath == "" {diff --git a/cli/beacon/cmd/ci_test.go b/cli/beacon/cmd/ci_test.goindex 26117e52..bf49b45b 100644--- a/cli/beacon/cmd/ci_test.go+++ b/cli/beacon/cmd/ci_test.go@@ -1,6 +1,10 @@package cmd-import "testing"+import (+ "os"+ "strings"+ "testing"+)func TestCICommandsRegistered(t *testing.T) {for _, path := range [][]string{@@ -24,7 +28,7 @@ func TestCICommandsRegistered(t *testing.T) {t.Fatalf("ci validate missing --%s", name)}}- for _, name := range []string{"content-retention", "keep-artifacts"} {+ for _, name := range []string{"content-retention", "keep-artifacts", "upload"} {if ciExecCmd.Flags().Lookup(name) == nil {t.Fatalf("ci exec missing --%s", name)}@@ -43,6 +47,19 @@ func TestCICommandsRegistered(t *testing.T) {}}+func TestCIActionExposesUploadInput(t *testing.T) {+ data, err := os.ReadFile("../../../action.yml")+ if err != nil {+ t.Fatalf("read action.yml: %v", err)+ }+ text := string(data)+ for _, want := range []string{" upload:", "INPUT_UPLOAD", "--upload"} {+ if !strings.Contains(text, want) {+ t.Fatalf("action.yml missing %q", want)+ }+ }+}+func TestCIHarnessDefaultIsClaudeOnly(t *testing.T) {flag := ciExecCmd.Flags().Lookup("harness")if flag == nil {diff --git a/cli/beacon/internal/ci/session.go b/cli/beacon/internal/ci/session.goindex 69a67132..d29e8e64 100644--- a/cli/beacon/internal/ci/session.go+++ b/cli/beacon/internal/ci/session.go@@ -47,6 +47,7 @@ type Options struct {// token is read from the environment only.Forward stringForwardEndpoint string+ Uploads []string}type Session struct {@@ -62,8 +63,9 @@ type Session struct {Run *schema.RunInfo `json:"run,omitempty"`// Forward and ForwardEndpoint describe the optional SIEM egress. The token// is never recorded here so it cannot leak into --json output.- Forward string `json:"forward,omitempty"`- ForwardEndpoint string `json:"forward_endpoint,omitempty"`+ Forward string `json:"forward,omitempty"`+ ForwardEndpoint string `json:"forward_endpoint,omitempty"`+ Uploads []UploadDestination `json:"uploads,omitempty"`cfg endpointconfig.ConfigstartedAt time.Time@@ -75,6 +77,7 @@ type ExecResult struct {Session Session `json:"session"`ChildExitCode int `json:"child_exit_code"`Validation ValidationResult `json:"validation"`+ Uploads []UploadResult `json:"uploads,omitempty"`ArtifactMessage string `json:"artifact_message,omitempty"`}@@ -134,6 +137,12 @@ func Provision(opts Options) (*Session, error) {// environment and can resolve them; the child has these vars stripped.replaceTokensWithEnvRefs(&cfg)}+ startedAt := time.Now().UTC()+ run := detectRunInfo()+ uploads, err := resolveUploadDestinations(opts.Uploads, run, logPath, startedAt)+ if err != nil {+ return nil, err+ }if err := os.MkdirAll(filepath.Dir(logPath), 0755); err != nil {return nil, err}@@ -151,7 +160,6 @@ func Provision(opts Options) (*Session, error) {if err != nil {return nil, err}- startedAt := time.Now().UTC()return &Session{BaseDir: baseDir,LogPath: logPath,@@ -162,9 +170,10 @@ func Provision(opts Options) (*Session, error) {Harness: DefaultHarness,WorkDir: opts.WorkDir,StartedAt: startedAt.Format(time.RFC3339),- Run: detectRunInfo(),+ Run: run,Forward: forward,ForwardEndpoint: forwardEndpoint,+ Uploads: uploads,cfg: cfg,startedAt: startedAt,}, nil@@ -248,8 +257,12 @@ func (s *Session) RunChild(ctx context.Context, args []string, stdout, stderr io"BEACON_CI_CONFIG_PATH="+s.ConfigPath,"BEACON_CI_LOG_PATH="+s.LogPath,)- // The agent never needs the SIEM token; keep it out of the child process.- cmd.Env = stripEnv(env, forwardTokenEnv...)+ // The agent never needs destination credentials; keep them out of the child process.+ stripKeys := append([]string{}, forwardTokenEnv...)+ if len(s.Uploads) > 0 {+ stripKeys = append(stripKeys, uploadCredentialEnv...)+ }+ cmd.Env = stripEnv(env, stripKeys...)cmd.Stdout = stdoutcmd.Stderr = stderrerr := cmd.Run()diff --git a/cli/beacon/internal/ci/session_test.go b/cli/beacon/internal/ci/session_test.goindex e77fe8c1..c32402a6 100644--- a/cli/beacon/internal/ci/session_test.go+++ b/cli/beacon/internal/ci/session_test.go@@ -392,6 +392,42 @@ func TestRunChildStripsForwardToken(t *testing.T) {}}+func TestRunChildStripsUploadCredentialsWhenUploadConfigured(t *testing.T) {+ dir := t.TempDir()+ output := filepath.Join(dir, "env.txt")+ t.Setenv("AWS_ACCESS_KEY_ID", "aws-key")+ t.Setenv("AWS_SECRET_ACCESS_KEY", "aws-secret")+ t.Setenv("AWS_SESSION_TOKEN", "aws-session")+ t.Setenv("GOOGLE_APPLICATION_CREDENTIALS", "/tmp/google-creds.json")+ child := fakeExecutable(t, "child", "#!/bin/sh\nenv > \"$1\"\n")+ session := &Session{+ BaseDir: dir,+ ConfigPath: filepath.Join(dir, "otelcol.yaml"),+ LogPath: filepath.Join(dir, "runtime.jsonl"),+ GRPCEndpoint: "http://127.0.0.1:4317",+ Uploads: []UploadDestination{+ {Provider: UploadS3, URI: "s3://bucket/key"},+ },+ cfg: endpointconfig.Default(true, filepath.Join(dir, "runtime.jsonl")),+ }+ if _, err := session.RunChild(context.Background(), []string{child, output}, nil, nil); err != nil {+ t.Fatalf("RunChild returned error: %v", err)+ }+ data, err := os.ReadFile(output)+ if err != nil {+ t.Fatal(err)+ }+ env := string(data)+ for _, leaked := range []string{"AWS_ACCESS_KEY_ID=", "AWS_SECRET_ACCESS_KEY=", "AWS_SESSION_TOKEN=", "GOOGLE_APPLICATION_CREDENTIALS=", "aws-secret", "/tmp/google-creds.json"} {+ if strings.Contains(env, leaked) {+ t.Fatalf("child env exposed upload credential %q:\n%s", leaked, env)+ }+ }+ if !strings.Contains(env, "CLAUDE_CODE_ENABLE_TELEMETRY=1") {+ t.Fatalf("child env missing Claude telemetry vars:\n%s", env)+ }+}+func fakeExecutable(t *testing.T, name, script string) string {t.Helper()path := filepath.Join(t.TempDir(), name)diff --git a/cli/beacon/internal/ci/upload.go b/cli/beacon/internal/ci/upload.gonew file mode 100644index 00000000..54d0b593--- /dev/null+++ b/cli/beacon/internal/ci/upload.go@@ -0,0 +1,234 @@+package ci++import (+ "context"+ "fmt"+ "os"+ "os/exec"+ "path"+ "path/filepath"+ "strings"+ "time"++ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"+)++const (+ UploadS3 = "s3"+ UploadGCS = "gcs"+)++const (+ EnvS3Bucket = "BEACON_CI_S3_BUCKET"+ EnvS3Prefix = "BEACON_CI_S3_PREFIX"+ EnvGCSBucket = "BEACON_CI_GCS_BUCKET"+ EnvGCSPrefix = "BEACON_CI_GCS_PREFIX"+)++var uploadCredentialEnv = []string{+ "AWS_ACCESS_KEY_ID",+ "AWS_SECRET_ACCESS_KEY",+ "AWS_SESSION_TOKEN",+ "AWS_PROFILE",+ "AWS_WEB_IDENTITY_TOKEN_FILE",+ "GOOGLE_APPLICATION_CREDENTIALS",+ "CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE",+}++var (+ uploadLookPath = exec.LookPath+ runUploadCommand = func(ctx context.Context, name string, args ...string) ([]byte, error) {+ return exec.CommandContext(ctx, name, args...).CombinedOutput()+ }+)++type UploadDestination struct {+ Provider string `json:"provider"`+ Bucket string `json:"bucket"`+ Key string `json:"key"`+ URI string `json:"uri"`+}++type UploadResult struct {+ Provider string `json:"provider"`+ Target string `json:"target"`+ Status string `json:"status"`+ Message string `json:"message,omitempty"`+}++func resolveUploadDestinations(providers []string, run *schema.RunInfo, logPath string, startedAt time.Time) ([]UploadDestination, error) {+ normalized, err := normalizeUploads(providers)+ if err != nil {+ return nil, err+ }+ if len(normalized) == 0 {+ return nil, nil+ }+ destinations := make([]UploadDestination, 0, len(normalized))+ for _, provider := range normalized {+ switch provider {+ case UploadS3:+ dest, err := uploadDestination(provider, os.Getenv(EnvS3Bucket), os.Getenv(EnvS3Prefix), run, logPath, startedAt)+ if err != nil {+ return nil, err+ }+ destinations = append(destinations, dest)+ case UploadGCS:+ dest, err := uploadDestination(provider, os.Getenv(EnvGCSBucket), os.Getenv(EnvGCSPrefix), run, logPath, startedAt)+ if err != nil {+ return nil, err+ }+ destinations = append(destinations, dest)+ }+ }+ return destinations, nil+}++func normalizeUploads(values []string) ([]string, error) {+ seen := map[string]struct{}{}+ var providers []string+ for _, value := range values {+ for _, part := range strings.Split(value, ",") {+ provider := strings.ToLower(strings.TrimSpace(part))+ if provider == "" {+ continue+ }+ switch provider {+ case UploadS3, UploadGCS:+ if _, ok := seen[provider]; ok {+ continue+ }+ seen[provider] = struct{}{}+ providers = append(providers, provider)+ default:+ return nil, fmt.Errorf("unsupported --upload %q; supported values are %s and %s", part, UploadS3, UploadGCS)+ }+ }+ }+ return providers, nil+}++func uploadDestination(provider, bucket, prefix string, run *schema.RunInfo, logPath string, startedAt time.Time) (UploadDestination, error) {+ bucket = strings.TrimSpace(bucket)+ if bucket == "" {+ return UploadDestination{}, fmt.Errorf("--upload %s requires a bucket; set %s", provider, uploadBucketEnv(provider))+ }+ key := uploadObjectKey(prefix, run, logPath, startedAt)+ uri := fmt.Sprintf("%s://%s/%s", uploadScheme(provider), bucket, key)+ return UploadDestination{Provider: provider, Bucket: bucket, Key: key, URI: uri}, nil+}++func uploadObjectKey(prefix string, run *schema.RunInfo, logPath string, startedAt time.Time) string {+ name := filepath.Base(logPath)+ if strings.TrimSpace(name) == "" || name == "." || name == string(filepath.Separator) {+ name = "runtime.jsonl"+ }+ parts := cleanKeyParts(prefix)+ if run != nil && strings.TrimSpace(run.Repository) != "" && strings.TrimSpace(run.RunID) != "" {+ parts = append(parts, cleanKeyParts(run.Repository)...)+ parts = append(parts, cleanKeyPart(run.RunID))+ if strings.TrimSpace(run.RunAttempt) != "" {+ parts = append(parts, cleanKeyPart(run.RunAttempt))+ }+ } else {+ if startedAt.IsZero() {+ startedAt = time.Now().UTC()+ }+ parts = append(parts, "ci", fmt.Sprintf("%d", startedAt.UTC().UnixNano()))+ }+ parts = append(parts, cleanKeyPart(name))+ return path.Join(parts...)+}++func cleanKeyParts(value string) []string {+ var parts []string+ for _, part := range strings.Split(strings.Trim(value, "/"), "/") {+ if cleaned := cleanKeyPart(part); cleaned != "" {+ parts = append(parts, cleaned)+ }+ }+ return parts+}++func cleanKeyPart(value string) string {+ cleaned := strings.TrimSpace(value)+ cleaned = strings.ReplaceAll(cleaned, "\\", "-")+ cleaned = strings.Trim(cleaned, "/")+ if cleaned == "." || cleaned == ".." {+ return ""+ }+ return cleaned+}++func uploadBucketEnv(provider string) string {+ if provider == UploadGCS {+ return EnvGCSBucket+ }+ return EnvS3Bucket+}++func uploadScheme(provider string) string {+ if provider == UploadGCS {+ return "gs"+ }+ return "s3"+}++func (s *Session) UploadArtifacts(ctx context.Context) ([]UploadResult, error) {+ if s == nil || len(s.Uploads) == 0 {+ return nil, nil+ }+ results := make([]UploadResult, 0, len(s.Uploads))+ for _, dest := range s.Uploads {+ result, err := uploadArtifact(ctx, s.LogPath, dest)+ results = append(results, result)+ if err != nil {+ return results, err+ }+ }+ return results, nil+}++func uploadArtifact(ctx context.Context, logPath string, dest UploadDestination) (UploadResult, error) {+ var name string+ var args []string+ switch dest.Provider {+ case UploadS3:+ if _, err := uploadLookPath("aws"); err != nil {+ return uploadFailure(dest, "aws CLI not found in PATH"), fmt.Errorf("--upload s3 requires the aws CLI in PATH")+ }+ name = "aws"+ args = []string{"s3", "cp", logPath, dest.URI}+ case UploadGCS:+ switch {+ case commandAvailable("gcloud"):+ name = "gcloud"+ args = []string{"storage", "cp", logPath, dest.URI}+ case commandAvailable("gsutil"):+ name = "gsutil"+ args = []string{"cp", logPath, dest.URI}+ default:+ return uploadFailure(dest, "gcloud or gsutil not found in PATH"), fmt.Errorf("--upload gcs requires gcloud or gsutil in PATH")+ }+ default:+ return uploadFailure(dest, "unsupported provider"), fmt.Errorf("unsupported upload provider %q", dest.Provider)+ }+ output, err := runUploadCommand(ctx, name, args...)+ if err != nil {+ message := strings.TrimSpace(string(output))+ if message == "" {+ message = err.Error()+ }+ return uploadFailure(dest, message), fmt.Errorf("upload %s to %s failed: %w", dest.Provider, dest.URI, err)+ }+ return UploadResult{Provider: dest.Provider, Target: dest.URI, Status: "ok"}, nil+}++func uploadFailure(dest UploadDestination, message string) UploadResult {+ return UploadResult{Provider: dest.Provider, Target: dest.URI, Status: "fail", Message: message}+}++func commandAvailable(name string) bool {+ _, err := uploadLookPath(name)+ return err == nil+}diff --git a/cli/beacon/internal/ci/upload_test.go b/cli/beacon/internal/ci/upload_test.gonew file mode 100644index 00000000..60d9e910--- /dev/null+++ b/cli/beacon/internal/ci/upload_test.go@@ -0,0 +1,174 @@+package ci++import (+ "context"+ "errors"+ "fmt"+ "os/exec"+ "path/filepath"+ "reflect"+ "strings"+ "testing"+ "time"++ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"+)++func TestNormalizeUploadsSplitsAndDedupes(t *testing.T) {+ got, err := normalizeUploads([]string{"s3,gcs", "s3"})+ if err != nil {+ t.Fatalf("normalizeUploads returned error: %v", err)+ }+ if want := []string{UploadS3, UploadGCS}; !reflect.DeepEqual(got, want) {+ t.Fatalf("normalizeUploads = %#v, want %#v", got, want)+ }+}++func TestNormalizeUploadsRejectsUnsupportedProvider(t *testing.T) {+ if _, err := normalizeUploads([]string{"elastic"}); err == nil || !strings.Contains(err.Error(), "unsupported --upload") {+ t.Fatalf("normalizeUploads error = %v, want unsupported provider", err)+ }+}++func TestResolveUploadDestinationsRequiresBucket(t *testing.T) {+ t.Setenv(EnvS3Bucket, "")+ _, err := resolveUploadDestinations([]string{"s3"}, nil, "/tmp/runtime.jsonl", time.Unix(1700000000, 0).UTC())+ if err == nil || !strings.Contains(err.Error(), EnvS3Bucket) {+ t.Fatalf("resolveUploadDestinations error = %v, want missing bucket", err)+ }+}++func TestResolveUploadDestinationsBuildsGitHubObjectKeys(t *testing.T) {+ t.Setenv(EnvS3Bucket, "beacon-ci")+ t.Setenv(EnvS3Prefix, "logs/")+ run := &schema.RunInfo{+ Repository: "asymptote-labs/agent-beacon",+ RunID: "123",+ RunAttempt: "2",+ }+ destinations, err := resolveUploadDestinations([]string{"s3"}, run, "/tmp/runtime.jsonl", time.Unix(1700000000, 0).UTC())+ if err != nil {+ t.Fatalf("resolveUploadDestinations returned error: %v", err)+ }+ if len(destinations) != 1 {+ t.Fatalf("destinations = %d, want 1", len(destinations))+ }+ dest := destinations[0]+ if dest.Key != "logs/asymptote-labs/agent-beacon/123/2/runtime.jsonl" {+ t.Fatalf("Key = %q", dest.Key)+ }+ if dest.URI != "s3://beacon-ci/logs/asymptote-labs/agent-beacon/123/2/runtime.jsonl" {+ t.Fatalf("URI = %q", dest.URI)+ }+}++func TestResolveUploadDestinationsBuildsFallbackObjectKeys(t *testing.T) {+ t.Setenv(EnvGCSBucket, "beacon-ci")+ startedAt := time.Unix(0, 42).UTC()+ destinations, err := resolveUploadDestinations([]string{"gcs"}, nil, "/tmp/runtime.jsonl", startedAt)+ if err != nil {+ t.Fatalf("resolveUploadDestinations returned error: %v", err)+ }+ if got, want := destinations[0].Key, "ci/42/runtime.jsonl"; got != want {+ t.Fatalf("Key = %q, want %q", got, want)+ }+ if got, want := destinations[0].URI, "gs://beacon-ci/ci/42/runtime.jsonl"; got != want {+ t.Fatalf("URI = %q, want %q", got, want)+ }+}++func TestUploadArtifactRunsS3Command(t *testing.T) {+ calls, restore := stubUploadCommands(t, map[string]bool{"aws": true}, nil)+ defer restore()++ dest := UploadDestination{Provider: UploadS3, URI: "s3://bucket/key"}+ result, err := uploadArtifact(context.Background(), "/tmp/runtime.jsonl", dest)+ if err != nil {+ t.Fatalf("uploadArtifact returned error: %v", err)+ }+ if result.Status != "ok" || result.Target != dest.URI {+ t.Fatalf("result = %#v", result)+ }+ if got, want := strings.Join(*calls, "\n"), "aws s3 cp /tmp/runtime.jsonl s3://bucket/key"; got != want {+ t.Fatalf("upload command = %q, want %q", got, want)+ }+}++func TestUploadArtifactRunsGCloudCommand(t *testing.T) {+ calls, restore := stubUploadCommands(t, map[string]bool{"gcloud": true}, nil)+ defer restore()++ dest := UploadDestination{Provider: UploadGCS, URI: "gs://bucket/key"}+ if _, err := uploadArtifact(context.Background(), "/tmp/runtime.jsonl", dest); err != nil {+ t.Fatalf("uploadArtifact returned error: %v", err)+ }+ if got, want := strings.Join(*calls, "\n"), "gcloud storage cp /tmp/runtime.jsonl gs://bucket/key"; got != want {+ t.Fatalf("upload command = %q, want %q", got, want)+ }+}++func TestUploadArtifactFallsBackToGsutil(t *testing.T) {+ calls, restore := stubUploadCommands(t, map[string]bool{"gsutil": true}, nil)+ defer restore()++ dest := UploadDestination{Provider: UploadGCS, URI: "gs://bucket/key"}+ if _, err := uploadArtifact(context.Background(), "/tmp/runtime.jsonl", dest); err != nil {+ t.Fatalf("uploadArtifact returned error: %v", err)+ }+ if got, want := strings.Join(*calls, "\n"), "gsutil cp /tmp/runtime.jsonl gs://bucket/key"; got != want {+ t.Fatalf("upload command = %q, want %q", got, want)+ }+}++func TestUploadArtifactMissingCLI(t *testing.T) {+ _, restore := stubUploadCommands(t, nil, nil)+ defer restore()++ result, err := uploadArtifact(context.Background(), "/tmp/runtime.jsonl", UploadDestination{Provider: UploadS3, URI: "s3://bucket/key"})+ if err == nil || !strings.Contains(err.Error(), "aws CLI") {+ t.Fatalf("uploadArtifact error = %v, want missing aws CLI", err)+ }+ if result.Status != "fail" {+ t.Fatalf("result status = %q, want fail", result.Status)+ }+}++func TestUploadArtifactCommandFailure(t *testing.T) {+ _, restore := stubUploadCommands(t, map[string]bool{"aws": true}, errors.New("boom"))+ defer restore()++ result, err := uploadArtifact(context.Background(), "/tmp/runtime.jsonl", UploadDestination{Provider: UploadS3, URI: "s3://bucket/key"})+ if err == nil || !strings.Contains(err.Error(), "upload s3") {+ t.Fatalf("uploadArtifact error = %v, want upload failure", err)+ }+ if result.Status != "fail" || result.Message != "boom" {+ t.Fatalf("result = %#v", result)+ }+}++func stubUploadCommands(t *testing.T, available map[string]bool, commandErr error) (*[]string, func()) {+ t.Helper()+ oldLookPath := uploadLookPath+ oldRun := runUploadCommand+ var calls []string+ uploadLookPath = func(name string) (string, error) {+ if available[name] {+ return filepath.Join("/bin", name), nil+ }+ return "", exec.ErrNotFound+ }+ runUploadCommand = func(ctx context.Context, name string, args ...string) ([]byte, error) {+ calls = append(calls, strings.Join(append([]string{name}, args...), " "))+ if commandErr != nil {+ return []byte(commandErr.Error()), commandErr+ }+ return nil, nil+ }+ return &calls, func() {+ if t.Failed() {+ t.Logf("upload command calls: %s", fmt.Sprint(calls))+ }+ uploadLookPath = oldLookPath+ runUploadCommand = oldRun+ }+}diff --git a/examples/github-actions/claude-code-telemetry.yml b/examples/github-actions/claude-code-telemetry.ymlindex 9411a74c..59c4f0df 100644--- a/examples/github-actions/claude-code-telemetry.yml+++ b/examples/github-actions/claude-code-telemetry.yml@@ -1,6 +1,6 @@# Reference workflow for capturing Claude Code telemetry in CI with the# Beacon composite action. Copy this into .github/workflows/ in your own-# repository and adjust the command and forwarding settings.+# repository and adjust the command, artifact, forwarding, or upload settings.## This file lives under examples/ (not .github/workflows/) so it is a# reference only and does not run in this repository.@@ -14,10 +14,13 @@ on:jobs:claude-telemetry:runs-on: ubuntu-latest- # Set the SIEM token at the job level (from a secret) so the ephemeral- # collector can resolve it. Omit this block if you only upload an artifact.+ # Set destination config at the job level so Beacon can use it without+ # exposing secrets to the Claude child process. Omit this block if you only+ # upload the default GitHub artifact.env:BEACON_CI_SPLUNK_HEC_TOKEN: ${{ secrets.SPLUNK_HEC_TOKEN }}+ BEACON_CI_S3_BUCKET: ${{ vars.BEACON_CI_S3_BUCKET }}+ BEACON_CI_S3_PREFIX: github-actionssteps:- uses: actions/checkout@v4@@ -26,6 +29,12 @@ jobs:- name: Install Claude Coderun: npm install -g @anthropic-ai/claude-code+ # Optional: configure cloud credentials before using upload: s3.+ # - uses: aws-actions/configure-aws-credentials@v4+ # with:+ # role-to-assume: ${{ secrets.BEACON_TELEMETRY_ROLE_ARN }}+ # aws-region: us-east-1+- name: Run Claude with Beacon telemetryuses: asymptote-labs/[email protected]env:@@ -36,5 +45,8 @@ jobs:# Requires a Beacon release that includes ci exec --forward.forward: splunkforward-endpoint: https://splunk.example:8088/services/collector+ # Optional: upload the completed runtime JSONL to object storage.+ # Supported values: s3, gcs, or s3,gcs.+ # upload: s3# The runtime log is uploaded as an artifact by default.upload-artifact: "true"