The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu
Go agent-ai-cliagent-harnessesagent-securitybrowser-extensionciclaude-codecloudcodexcursordetection-engineeringjevknowledge-basememorymobile-device-managementobservabilitysecuritysecurity-information-and-event-managementsecurity-toolstelemetrytraces
add elastic endpoint integration pack
shukieshahe9dbdde7cf7a
diff --git a/CLAUDE.md b/CLAUDE.mdindex 0e678c98..ccdde966 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -19,7 +19,7 @@ Do not recreate or depend on removed `asymptote` mirror trees. Keep new work foc- Preserve the local-only product posture. The public Beacon build should not require a hosted account, remote policy fetch, hosted dashboard, or external network dependency during normal hook execution.- Do not add dependency vulnerability scanning, OSV/GHSA lookups, package remediation, or other vulnerability-enforcement flows to the public hook path.- Do not add broad runtime enforcement unless explicitly requested. Current control behavior is limited to hook-native approvals/denials exposed by supported agent runtimes.-- Keep direct destination support scoped to local JSONL/Wazuh unless explicitly requested. Other SIEM/observability systems should consume the local output through customer-managed forwarding.+- Keep direct destination support scoped to local JSONL/Wazuh unless explicitly requested. Elastic support is a file-tailing pack over local JSONL; Beacon itself must not store Elastic credentials or require a hosted Elastic dependency.- Default content retention is `full`: configured prompt text, command output, raw tool inputs, raw OTLP attributes, and raw diffs may be written to local or customer-controlled logs, still subject to local redaction and size limits where supported. Keep `metadata` and `redacted` modes available for stricter deployments.## Telemetry Scope@@ -30,12 +30,13 @@ Supported runtime surfaces today:- Cursor hook telemetry for sessions, prompt submission, tool use, command execution, MCP-like tool activity, approval decisions, and file edits where hook payloads expose those fields.- Claude Cowork admin-configured OpenTelemetry setup guidance and local validation.- `beaconjson` OpenTelemetry Collector exporter that converts OTLP logs, traces, metrics, and resource attributes into Beacon endpoint JSONL.+- Elasticsearch/Filebeat content pack generation for forwarding local Beacon JSONL into customer-managed Elastic deployments or the bundled loopback-only development stack.- A local-only dashboard served by `beacon endpoint dashboard`, bound to loopback by default and backed by the runtime JSONL log.Current non-goals unless explicitly requested:- Kernel/process monitoring, EDR replacement, shell history scraping, cloud audit ingestion, browser/SaaS telemetry, credential-use attribution, and MCP configuration inventory.-- Direct hosted integrations for Datadog, Splunk, Elastic, Snowflake, Chronicle, Panther, or other SIEM destinations.+- Direct hosted integrations for Datadog, Snowflake, Chronicle, Panther, or other SIEM destinations beyond explicitly supported local/customer-managed forwarding patterns.- Dependency vulnerability scanning or package security remediation.## Common Commandsdiff --git a/README.md b/README.mdindex 481cfc04..14eae9bc 100644--- a/README.md+++ b/README.md@@ -40,7 +40,7 @@ locally.Beacon is built to be easy to deploy for Security and IT teams through[MDM deployment](https://docs.asymptotelabs.ai/cli/security-it-teams) and to-connect to Wazuh, Splunk HEC, or customer-managed SIEM pipelines, while+connect to Wazuh, Elastic, Splunk HEC, or customer-managed SIEM pipelines, whileremaining visibility-first and local-first during normal endpoint collection.## High-Level Architecture@@ -57,7 +57,7 @@ leaving forwarding under customer control.- **Beacon endpoint layer:** Local processing normalizes events, appliesretention and redaction settings, and writes durable endpoint telemetry.- **Output layer:** Teams inspect events in the local dashboard, retain JSONL,- or forward records into Wazuh, Splunk HEC, and customer-managed SIEM+ or forward records into Wazuh, Elastic, Splunk HEC, and customer-managed SIEMpipelines.Beacon filters generic process and runtime metrics, such as Node.js event loop,@@ -82,6 +82,23 @@ records during rollout, testing, and investigations.<img src="images/dashboard-log-search.png" alt="Beacon dashboard log search" width="860"></p>+## Elastic++Beacon ships an Elastic content pack for teams that want to search endpoint+events in Elasticsearch and Kibana without giving Beacon cluster credentials.+The pack tails the same local `runtime.jsonl` file with Filebeat or standalone+Elastic Agent, installs ECS-oriented templates and an ingest pipeline, and+includes starter Kibana assets.++```bash+beacon endpoint elastic install-pack --output ./beacon-elastic-pack+beacon endpoint elastic up --pack-dir ./beacon-elastic-pack+```++The local stack binds Elasticsearch and Kibana to loopback. Existing self-managed+or Elastic Cloud deployments can use the same pack by pointing Filebeat at their+cluster with `ES_HOSTS` and `ES_API_KEY`.+## Start Here- [Beacon CLI docs](https://docs.asymptotelabs.ai/cli) — full documentation index.diff --git a/cli/beacon/cmd/endpoint.go b/cli/beacon/cmd/endpoint.goindex dbbbe468..c1852a0b 100644--- a/cli/beacon/cmd/endpoint.go+++ b/cli/beacon/cmd/endpoint.go@@ -1,15 +1,20 @@package cmdimport (+ "context""encoding/json""fmt""os"+ "os/exec"+ "path/filepath"+ "runtime""strings""github.com/spf13/cobra"endpointconfig "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/config""github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/dashboard"+ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/elastic""github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/harness"endpointhooks "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/hooks""github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/integrations/cowork"@@ -41,6 +46,7 @@ var endpointOpts struct {coworkNgrok boolcoworkOpen boolcoworkSince string+ elasticPackDir stringhookLevel stringcontentRetention stringsplunkHECEndpoint string@@ -108,6 +114,11 @@ var endpointWazuhCmd = &cobra.Command{Short: "Manage Wazuh integration content",}+var endpointElasticCmd = &cobra.Command{+ Use: "elastic",+ Short: "Manage Elasticsearch integration content",+}+var endpointIntegrationsCmd = &cobra.Command{Use: "integrations",Short: "Manage admin-configured endpoint integrations",@@ -230,6 +241,40 @@ var endpointWazuhValidateCmd = &cobra.Command{RunE: runEndpointWazuhValidate,}+var endpointElasticPrintConfigCmd = &cobra.Command{+ Use: "print-config",+ Short: "Print a Filebeat input for Beacon endpoint events",+ Run: func(cmd *cobra.Command, args []string) {+ cfg := loadOrDefaultConfig()+ fmt.Print(elastic.InputSnippet(cfg.LogPath))+ },+}++var endpointElasticInstallPackCmd = &cobra.Command{+ Use: "install-pack",+ Short: "Write Elasticsearch templates, pipeline, and Filebeat content to a directory",+ SilenceUsage: true,+ RunE: runEndpointElasticInstallPack,+}++var endpointElasticUpCmd = &cobra.Command{+ Use: "up",+ Short: "Start a local Elasticsearch, Kibana, and Filebeat stack",+ SilenceUsage: true,+ RunE: func(cmd *cobra.Command, args []string) error {+ return runEndpointElasticUp(cmd.Context())+ },+}++var endpointElasticDownCmd = &cobra.Command{+ Use: "down",+ Short: "Stop the local Elasticsearch stack",+ SilenceUsage: true,+ RunE: func(cmd *cobra.Command, args []string) error {+ return runEndpointElasticDown(cmd.Context())+ },+}+func init() {rootCmd.AddCommand(endpointCmd)@@ -240,11 +285,16 @@ func init() {endpointCmd.AddCommand(endpointRepairCmd)endpointCmd.AddCommand(endpointDashboardCmd)endpointCmd.AddCommand(endpointWazuhCmd)+ endpointCmd.AddCommand(endpointElasticCmd)endpointCmd.AddCommand(endpointIntegrationsCmd)endpointCmd.AddCommand(endpointHooksCmd)endpointWazuhCmd.AddCommand(endpointWazuhPrintConfigCmd)endpointWazuhCmd.AddCommand(endpointWazuhInstallPackCmd)endpointWazuhCmd.AddCommand(endpointWazuhValidateCmd)+ endpointElasticCmd.AddCommand(endpointElasticPrintConfigCmd)+ endpointElasticCmd.AddCommand(endpointElasticInstallPackCmd)+ endpointElasticCmd.AddCommand(endpointElasticUpCmd)+ endpointElasticCmd.AddCommand(endpointElasticDownCmd)endpointIntegrationsCmd.AddCommand(endpointCoworkCmd)endpointHooksCmd.AddCommand(endpointHooksInstallCmd)endpointHooksCmd.AddCommand(endpointHooksUninstallCmd)@@ -294,6 +344,14 @@ func init() {endpointWazuhValidateCmd.Flags().BoolVar(&endpointOpts.userMode, "user", true, "Use per-user endpoint paths")endpointWazuhValidateCmd.Flags().BoolVar(&endpointOpts.systemMode, "system", false, "Use system endpoint paths and launch daemon")endpointWazuhValidateCmd.Flags().StringVar(&endpointOpts.logPath, "log-path", "", "Runtime JSONL log path")+ for _, c := range []*cobra.Command{endpointElasticPrintConfigCmd, endpointElasticInstallPackCmd, endpointElasticUpCmd} {+ c.Flags().BoolVar(&endpointOpts.userMode, "user", true, "Use per-user endpoint paths")+ c.Flags().BoolVar(&endpointOpts.systemMode, "system", false, "Use system endpoint paths and launch daemon")+ c.Flags().StringVar(&endpointOpts.logPath, "log-path", "", "Runtime JSONL log path")+ }+ endpointElasticInstallPackCmd.Flags().StringVar(&endpointOpts.outputDir, "output", "", "Output directory for Elasticsearch content pack")+ endpointElasticUpCmd.Flags().StringVar(&endpointOpts.elasticPackDir, "pack-dir", elastic.DefaultOutputDir, "Elasticsearch pack directory")+ endpointElasticDownCmd.Flags().StringVar(&endpointOpts.elasticPackDir, "pack-dir", elastic.DefaultOutputDir, "Elasticsearch pack directory")for _, c := range []*cobra.Command{endpointCoworkPrintConfigCmd, endpointCoworkSetupCmd, endpointCoworkStatusCmd, endpointCoworkValidateCmd} {c.Flags().BoolVar(&endpointOpts.userMode, "user", true, "Use per-user endpoint paths")c.Flags().BoolVar(&endpointOpts.systemMode, "system", false, "Use system endpoint paths and launch daemon")@@ -442,6 +500,123 @@ func runEndpointWazuhValidate(cmd *cobra.Command, args []string) error {return nil}+func runEndpointElasticInstallPack(cmd *cobra.Command, args []string) error {+ cfg := loadOrDefaultConfig()+ outputDir := endpointOpts.outputDir+ if outputDir == "" {+ outputDir = elastic.DefaultOutputDir+ }+ if err := elastic.InstallPack(outputDir, cfg.LogPath); err != nil {+ return err+ }+ fmt.Printf("Elasticsearch content pack written to %s\n", outputDir)+ return nil+}++func runEndpointElasticUp(ctx context.Context) error {+ if runtime.GOOS != "darwin" {+ return fmt.Errorf("beacon endpoint elastic up is currently macOS-only")+ }+ cfg := loadOrDefaultConfig()+ logPath, err := filepath.Abs(cfg.LogPath)+ if err != nil {+ return err+ }+ packDir := endpointOpts.elasticPackDir+ if packDir == "" {+ packDir = elastic.DefaultOutputDir+ }+ if err := ensureElasticPack(packDir, logPath); err != nil {+ return err+ }+ if err := ensureLogFile(logPath); err != nil {+ return err+ }+ env := os.Environ()+ env = append(env, "BEACON_LOG_DIR="+filepath.Dir(logPath))+ if err := runDockerCompose(ctx, packDir, env, "up", "-d"); err != nil {+ return err+ }+ fmt.Printf("Elasticsearch ready at http://localhost:%s\n", envDefault("BEACON_ELASTIC_ES_PORT", "9200"))+ fmt.Printf("Kibana ready at http://localhost:%s\n", envDefault("BEACON_ELASTIC_KIBANA_PORT", "5601"))+ fmt.Printf("Filebeat tailing %s\n", logPath)+ return nil+}++func runEndpointElasticDown(ctx context.Context) error {+ if runtime.GOOS != "darwin" {+ return fmt.Errorf("beacon endpoint elastic down is currently macOS-only")+ }+ packDir := endpointOpts.elasticPackDir+ if packDir == "" {+ packDir = elastic.DefaultOutputDir+ }+ if _, err := os.Stat(filepath.Join(packDir, "docker-compose.yml")); os.IsNotExist(err) {+ fmt.Printf("No Elasticsearch stack found for %s\n", packDir)+ return nil+ } else if err != nil {+ return err+ }+ logPath, err := filepath.Abs(loadOrDefaultConfig().LogPath)+ if err != nil {+ return err+ }+ env := append(os.Environ(), "BEACON_LOG_DIR="+filepath.Dir(logPath))+ if err := runDockerCompose(ctx, packDir, env, "down", "--remove-orphans"); err != nil {+ return err+ }+ fmt.Printf("Elasticsearch stack stopped for %s\n", packDir)+ return nil+}++func ensureElasticPack(packDir, logPath string) error {+ if _, err := os.Stat(filepath.Join(packDir, "docker-compose.yml")); err == nil {+ return nil+ } else if !os.IsNotExist(err) {+ return err+ }+ if err := elastic.InstallPack(packDir, logPath); err != nil {+ return err+ }+ fmt.Printf("Elasticsearch content pack written to %s\n", packDir)+ return nil+}++func ensureLogFile(path string) error {+ if path == "" {+ return nil+ }+ if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil {+ return err+ }+ file, err := os.OpenFile(path, os.O_CREATE|os.O_APPEND, 0644)+ if err != nil {+ return err+ }+ return file.Close()+}++func runDockerCompose(ctx context.Context, dir string, env []string, args ...string) error {+ if _, err := os.Stat(filepath.Join(dir, "docker-compose.yml")); err != nil {+ return fmt.Errorf("docker-compose.yml not found in %s: %w", dir, err)+ }+ fullArgs := append([]string{"compose"}, args...)+ cmd := exec.CommandContext(ctx, "docker", fullArgs...)+ cmd.Dir = dir+ cmd.Env = env+ cmd.Stdout = os.Stdout+ cmd.Stderr = os.Stderr+ return cmd.Run()+}++func envDefault(name, fallback string) string {+ value := strings.TrimSpace(os.Getenv(name))+ if value == "" {+ return fallback+ }+ return value+}+func runEndpointDashboard(cmd *cobra.Command, args []string) error {cfg := loadOrDefaultConfig()userMode := endpointUserMode()diff --git a/cli/beacon/cmd/endpoint_test.go b/cli/beacon/cmd/endpoint_test.goindex 98ac217a..37e9ac40 100644--- a/cli/beacon/cmd/endpoint_test.go+++ b/cli/beacon/cmd/endpoint_test.go@@ -1,6 +1,8 @@package cmdimport (+ "os"+ "path/filepath""testing""github.com/spf13/cobra"@@ -50,6 +52,72 @@ func TestEndpointCoworkSetupCommandRegistered(t *testing.T) {}}+func TestEndpointElasticCommandsRegistered(t *testing.T) {+ for _, path := range [][]string{+ {"elastic", "print-config"},+ {"elastic", "install-pack"},+ {"elastic", "up"},+ {"elastic", "down"},+ } {+ cmd, _, err := endpointCmd.Find(path)+ if err != nil {+ t.Fatalf("Find %v returned error: %v", path, err)+ }+ if cmd == nil || cmd.Use != path[len(path)-1] {+ t.Fatalf("elastic command %v not registered: %#v", path, cmd)+ }+ }+ if endpointElasticInstallPackCmd.Flags().Lookup("output") == nil {+ t.Fatal("elastic install-pack command missing --output flag")+ }+ if endpointElasticUpCmd.Flags().Lookup("pack-dir") == nil {+ t.Fatal("elastic up command missing --pack-dir flag")+ }+ if endpointElasticDownCmd.Flags().Lookup("pack-dir") == nil {+ t.Fatal("elastic down command missing --pack-dir flag")+ }+}++func TestEnsureElasticPackDoesNotOverwriteExistingPack(t *testing.T) {+ dir := t.TempDir()+ composePath := filepath.Join(dir, "docker-compose.yml")+ if err := os.WriteFile(composePath, []byte("custom compose"), 0644); err != nil {+ t.Fatal(err)+ }+ if err := ensureElasticPack(dir, "/tmp/beacon/runtime.jsonl"); err != nil {+ t.Fatalf("ensureElasticPack returned error: %v", err)+ }+ got, err := os.ReadFile(composePath)+ if err != nil {+ t.Fatal(err)+ }+ if string(got) != "custom compose" {+ t.Fatalf("ensureElasticPack overwrote existing pack: %s", got)+ }+}++func TestRunEndpointElasticDownIgnoresMissingPack(t *testing.T) {+ oldPackDir := endpointOpts.elasticPackDir+ endpointOpts.elasticPackDir = filepath.Join(t.TempDir(), "missing-pack")+ t.Cleanup(func() {+ endpointOpts.elasticPackDir = oldPackDir+ })+ if err := runEndpointElasticDown(t.Context()); err != nil {+ t.Fatalf("runEndpointElasticDown returned error for missing pack: %v", err)+ }+}++func TestEnvDefault(t *testing.T) {+ t.Setenv("BEACON_TEST_VALUE", "")+ if got := envDefault("BEACON_TEST_VALUE", "fallback"); got != "fallback" {+ t.Fatalf("envDefault empty = %q", got)+ }+ t.Setenv("BEACON_TEST_VALUE", " 12345 ")+ if got := envDefault("BEACON_TEST_VALUE", "fallback"); got != "12345" {+ t.Fatalf("envDefault value = %q", got)+ }+}+func TestEndpointCoworkValidateSinceFlagRegistered(t *testing.T) {cmd, _, err := endpointCmd.Find([]string{"integrations", "claude-cowork", "validate"})if err != nil {diff --git a/cli/beacon/internal/endpoint/elastic/elastic.go b/cli/beacon/internal/endpoint/elastic/elastic.gonew file mode 100644index 00000000..246ad948--- /dev/null+++ b/cli/beacon/internal/endpoint/elastic/elastic.go@@ -0,0 +1,82 @@+package elastic++import (+ "embed"+ "fmt"+ "os"+ "path/filepath"+ "strings"+)++//go:embed pack/*+var packFS embed.FS++const (+ DefaultLogPath = "/var/log/beacon-agent/runtime.jsonl"+ DefaultOutputDir = "beacon-elastic-pack"+)++type File struct {+ Name string+ Content string+}++func Files() []File {+ return []File{+ {Name: "README.md", Content: mustRead("pack/README.md")},+ {Name: "filebeat.yml", Content: InputSnippet(DefaultLogPath)},+ {Name: "elastic-agent-standalone.yml", Content: standaloneAgentConfig(DefaultLogPath)},+ {Name: "ilm-policy.json", Content: mustRead("pack/ilm-policy.json")},+ {Name: "component-template-mappings.json", Content: mustRead("pack/component-template-mappings.json")},+ {Name: "component-template-settings.json", Content: mustRead("pack/component-template-settings.json")},+ {Name: "index-template.json", Content: mustRead("pack/index-template.json")},+ {Name: "ingest-pipeline.json", Content: mustRead("pack/ingest-pipeline.json")},+ {Name: "kibana-assets.ndjson", Content: mustRead("pack/kibana-assets.ndjson")},+ {Name: "docker-compose.yml", Content: mustRead("pack/docker-compose.yml")},+ {Name: "sample-event.jsonl", Content: mustRead("pack/sample-event.jsonl")},+ }+}++func InputSnippet(logPath string) string {+ if logPath == "" {+ logPath = DefaultLogPath+ }+ return strings.ReplaceAll(mustRead("pack/filebeat.yml.tmpl"), "{{LOG_PATH}}", logPath)+}++func InstallPack(outputDir, logPath string) error {+ if outputDir == "" {+ outputDir = DefaultOutputDir+ }+ if err := os.MkdirAll(outputDir, 0755); err != nil {+ return err+ }+ for _, file := range Files() {+ content := file.Content+ switch file.Name {+ case "filebeat.yml":+ content = InputSnippet(logPath)+ case "elastic-agent-standalone.yml":+ content = standaloneAgentConfig(logPath)+ }+ if err := os.WriteFile(filepath.Join(outputDir, file.Name), []byte(content), 0644); err != nil {+ return err+ }+ }+ return nil+}++func standaloneAgentConfig(logPath string) string {+ if logPath == "" {+ logPath = DefaultLogPath+ }+ return strings.ReplaceAll(mustRead("pack/elastic-agent-standalone.yml.tmpl"), "{{LOG_PATH}}", logPath)+}++func mustRead(path string) string {+ data, err := packFS.ReadFile(path)+ if err != nil {+ panic(fmt.Sprintf("elastic pack asset %s: %v", path, err))+ }+ return string(data)+}diff --git a/cli/beacon/internal/endpoint/elastic/elastic_test.go b/cli/beacon/internal/endpoint/elastic/elastic_test.gonew file mode 100644index 00000000..2cffcf23--- /dev/null+++ b/cli/beacon/internal/endpoint/elastic/elastic_test.go@@ -0,0 +1,137 @@+package elastic++import (+ "bufio"+ "encoding/json"+ "os"+ "path/filepath"+ "strings"+ "testing"+)++func TestInputSnippetUsesConfiguredPath(t *testing.T) {+ got := InputSnippet("/tmp/beacon/runtime.jsonl")+ if !strings.Contains(got, "/tmp/beacon/runtime.jsonl") {+ t.Fatalf("snippet did not include configured path: %s", got)+ }+ if strings.Contains(got, "{{LOG_PATH}}") {+ t.Fatalf("snippet still contains template token: %s", got)+ }+ if !strings.Contains(got, "filestream") {+ t.Fatalf("snippet should use Filebeat filestream input: %s", got)+ }+}++func TestInstallPackWritesExpectedFiles(t *testing.T) {+ dir := t.TempDir()+ if err := InstallPack(dir, "/tmp/beacon/runtime.jsonl"); err != nil {+ t.Fatalf("InstallPack returned error: %v", err)+ }+ for _, name := range []string{+ "README.md",+ "filebeat.yml",+ "elastic-agent-standalone.yml",+ "ilm-policy.json",+ "component-template-mappings.json",+ "component-template-settings.json",+ "index-template.json",+ "ingest-pipeline.json",+ "kibana-assets.ndjson",+ "docker-compose.yml",+ "sample-event.jsonl",+ } {+ if _, err := os.Stat(filepath.Join(dir, name)); err != nil {+ t.Fatalf("expected %s: %v", name, err)+ }+ }+ filebeat, err := os.ReadFile(filepath.Join(dir, "filebeat.yml"))+ if err != nil {+ t.Fatal(err)+ }+ if !strings.Contains(string(filebeat), "/tmp/beacon/runtime.jsonl") {+ t.Fatalf("generated filebeat.yml missing configured log path: %s", filebeat)+ }+ for _, emptyCredential := range []string{`api_key: "${ES_API_KEY:}"`, `username: "${ES_USERNAME:}"`, `password: "${ES_PASSWORD:}"`} {+ if strings.Contains(string(filebeat), emptyCredential) {+ t.Fatalf("generated filebeat.yml should not emit empty credential defaults: %s", emptyCredential)+ }+ }+ compose, err := os.ReadFile(filepath.Join(dir, "docker-compose.yml"))+ if err != nil {+ t.Fatal(err)+ }+ if !strings.Contains(string(compose), "${BEACON_LOG_DIR}:${BEACON_LOG_DIR}:ro") {+ t.Fatalf("docker-compose.yml should mount the log directory at the same path: %s", compose)+ }+}++func TestPackJSONFilesAreValid(t *testing.T) {+ for _, path := range []string{+ "pack/ilm-policy.json",+ "pack/component-template-mappings.json",+ "pack/component-template-settings.json",+ "pack/index-template.json",+ "pack/ingest-pipeline.json",+ } {+ var doc map[string]interface{}+ if err := json.Unmarshal([]byte(mustRead(path)), &doc); err != nil {+ t.Fatalf("%s is not valid JSON: %v", path, err)+ }+ }+}++func TestKibanaAssetsAreParseableNDJSON(t *testing.T) {+ scanner := bufio.NewScanner(strings.NewReader(mustRead("pack/kibana-assets.ndjson")))+ line := 0+ for scanner.Scan() {+ line+++ var doc map[string]interface{}+ if err := json.Unmarshal(scanner.Bytes(), &doc); err != nil {+ t.Fatalf("kibana-assets.ndjson line %d is not valid JSON: %v", line, err)+ }+ if doc["type"] == "" || doc["id"] == "" {+ t.Fatalf("kibana-assets.ndjson line %d missing type/id: %#v", line, doc)+ }+ }+ if err := scanner.Err(); err != nil {+ t.Fatal(err)+ }+ if line == 0 {+ t.Fatal("kibana-assets.ndjson was empty")+ }+}++func TestSampleEventsCoverHookAndOTelShapes(t *testing.T) {+ scanner := bufio.NewScanner(strings.NewReader(mustRead("pack/sample-event.jsonl")))+ var sawHook, sawOTel bool+ for scanner.Scan() {+ var doc map[string]interface{}+ if err := json.Unmarshal(scanner.Bytes(), &doc); err != nil {+ t.Fatalf("sample-event.jsonl is not valid JSONL: %v", err)+ }+ if raw, ok := doc["raw"].(map[string]interface{}); ok && raw["otel_signal"] != nil {+ sawOTel = true+ }+ if _, ok := doc["tool"].(map[string]interface{}); ok {+ sawHook = true+ }+ }+ if err := scanner.Err(); err != nil {+ t.Fatal(err)+ }+ if !sawHook || !sawOTel {+ t.Fatalf("sample events should include hook-rich and OTel shapes, sawHook=%t sawOTel=%t", sawHook, sawOTel)+ }+}++func TestIngestPipelineMentionsKeyMappings(t *testing.T) {+ pipeline := mustRead("pack/ingest-pipeline.json")+ for _, want := range []string{"event.duration", "'otel'", "host.hostname", "process.command_line", "gen_ai"} {+ if !strings.Contains(pipeline, want) {+ t.Fatalf("ingest pipeline missing %s", want)+ }+ }+ if strings.Contains(pipeline, "event.action = 'file.' + ctx.file.operation") {+ t.Fatal("ingest pipeline should preserve Beacon file actions instead of deriving lossy file operation actions")+ }+}diff --git a/cli/beacon/internal/endpoint/elastic/pack/README.md b/cli/beacon/internal/endpoint/elastic/pack/README.mdnew file mode 100644index 00000000..31d1cbd0--- /dev/null+++ b/cli/beacon/internal/endpoint/elastic/pack/README.md@@ -0,0 +1,79 @@+# Beacon Endpoint Agent Elastic Pack++This pack forwards Beacon endpoint JSONL events into Elasticsearch with Filebeat+or standalone Elastic Agent. Beacon still writes one local source of truth:+`runtime.jsonl`. Elastic credentials and cluster URLs live in the forwarding+tool, not in Beacon endpoint configuration.++## Local Stack++Generate the pack and start the bundled local development stack:++```bash+beacon endpoint elastic install-pack --output ./beacon-elastic-pack+beacon endpoint elastic up --pack-dir ./beacon-elastic-pack+```++The pack includes a Beacon data view and a starter Discover saved search. The+stack binds Elasticsearch and Kibana to loopback:++- Elasticsearch: `http://localhost:9200`+- Kibana: `http://localhost:5601`++If those ports are already in use, set `BEACON_ELASTIC_ES_PORT` or+`BEACON_ELASTIC_KIBANA_PORT` before running `elastic up`.++The Docker stack loads the ILM policy, component templates, index template,+ingest pipeline, and starter Kibana saved objects before Filebeat ships events.++Stop it with:++```bash+beacon endpoint elastic down --pack-dir ./beacon-elastic-pack+```++## Existing Elastic Deployments++Install the JSON assets in this order:++```bash+curl -X PUT "$ES_HOSTS/_ilm/policy/beacon-endpoint" -H 'Content-Type: application/json' --data-binary @ilm-policy.json+curl -X PUT "$ES_HOSTS/_component_template/beacon-endpoint-mappings" -H 'Content-Type: application/json' --data-binary @component-template-mappings.json+curl -X PUT "$ES_HOSTS/_component_template/beacon-endpoint-settings" -H 'Content-Type: application/json' --data-binary @component-template-settings.json+curl -X PUT "$ES_HOSTS/_index_template/beacon-endpoint" -H 'Content-Type: application/json' --data-binary @index-template.json+curl -X PUT "$ES_HOSTS/_ingest/pipeline/beacon-endpoint" -H 'Content-Type: application/json' --data-binary @ingest-pipeline.json+```++For Kibana, import `kibana-assets.ndjson` through Stack Management or the saved+objects import API.++Configure Filebeat with `filebeat.yml`, setting:++- `ES_HOSTS`, for example `http://localhost:9200` or an Elastic Cloud URL.+- One authentication method for secured clusters: uncomment `api_key`, or+ uncomment `username` and `password`.+- `ES_SSL_VERIFY` when your cluster needs a non-default TLS verification mode.+- `ssl.certificate_authorities` in the generated YAML when your cluster needs a+ custom CA bundle.++Minimum versions: Elasticsearch, Kibana, and Filebeat 8.x.++## Minimal Elasticsearch Role++Filebeat needs cluster `monitor` plus `auto_configure`, `create_doc`, and+`view_index_metadata` on `logs-beacon.endpoint-*`.++## Pipeline Simulation++Convert the sample JSONL into a simulate request and post it to Elasticsearch:++```bash+awk '{print "{\"docs\":[{\"_source\":" $0 "}]}"}' sample-event.jsonl | \+ curl -X POST "$ES_HOSTS/_ingest/pipeline/beacon-endpoint/_simulate" \+ -H 'Content-Type: application/json' \+ --data-binary @-+```++If you run `beacon endpoint elastic up`, the host log directory is mounted into+the Filebeat container at the same absolute path so the generated `filebeat.yml`+can tail the configured Beacon log path unchanged.diff --git a/cli/beacon/internal/endpoint/elastic/pack/component-template-mappings.json b/cli/beacon/internal/endpoint/elastic/pack/component-template-mappings.jsonnew file mode 100644index 00000000..13edc775--- /dev/null+++ b/cli/beacon/internal/endpoint/elastic/pack/component-template-mappings.json@@ -0,0 +1,248 @@+{+ "template": {+ "mappings": {+ "dynamic": true,+ "dynamic_templates": [+ {+ "beacon_keywords": {+ "path_match": "beacon.*",+ "match_mapping_type": "string",+ "mapping": {+ "type": "keyword",+ "ignore_above": 1024,+ "fields": {+ "text": {+ "type": "match_only_text"+ }+ }+ }+ }+ },+ {+ "otel_keywords": {+ "path_match": "beacon.otel.attributes.*",+ "match_mapping_type": "string",+ "mapping": {+ "type": "keyword",+ "ignore_above": 1024,+ "fields": {+ "text": {+ "type": "match_only_text"+ }+ }+ }+ }+ }+ ],+ "properties": {+ "@timestamp": { "type": "date" },+ "message": { "type": "match_only_text" },+ "agent": {+ "properties": {+ "name": { "type": "keyword" },+ "type": { "type": "keyword" },+ "version": { "type": "keyword" }+ }+ },+ "beacon": {+ "properties": {+ "vendor": { "type": "keyword" },+ "product": { "type": "keyword" },+ "schema_version": { "type": "keyword" },+ "severity": { "type": "keyword" },+ "truncated": { "type": "boolean" },+ "harness": {+ "properties": {+ "name": { "type": "keyword" },+ "version": { "type": "keyword" },+ "executable_path": { "type": "keyword" },+ "config_path": { "type": "keyword" }+ }+ },+ "session": {+ "properties": {+ "id": { "type": "keyword" }+ }+ },+ "tool": {+ "properties": {+ "name": { "type": "keyword" }+ }+ },+ "file": {+ "properties": {+ "operation": { "type": "keyword" },+ "language": { "type": "keyword" }+ }+ },+ "mcp": {+ "properties": {+ "server": { "type": "keyword" },+ "tool": { "type": "keyword" }+ }+ },+ "approval": {+ "properties": {+ "required": { "type": "boolean" },+ "decision": { "type": "keyword" },+ "reason": { "type": "match_only_text" }+ }+ },+ "policy": {+ "properties": {+ "id": { "type": "keyword" },+ "name": { "type": "keyword" },+ "decision": { "type": "keyword" },+ "enforcement": { "type": "keyword" },+ "reason": { "type": "match_only_text" }+ }+ },+ "prompt": {+ "properties": {+ "text": { "type": "match_only_text" }+ }+ },+ "content": {+ "properties": {+ "retention": { "type": "keyword" },+ "included": { "type": "boolean" },+ "redacted": { "type": "boolean" },+ "truncated": { "type": "boolean" }+ }+ },+ "destination": {+ "properties": {+ "type": { "type": "keyword" },+ "mode": { "type": "keyword" },+ "status": { "type": "keyword" }+ }+ },+ "health": {+ "properties": {+ "component": { "type": "keyword" },+ "status": { "type": "keyword" },+ "reason": { "type": "match_only_text" }+ }+ },+ "metric": {+ "properties": {+ "name": { "type": "keyword" }+ }+ },+ "otel": {+ "properties": {+ "signal": { "type": "keyword" },+ "metric_description": { "type": "match_only_text" },+ "metric_unit": { "type": "keyword" },+ "attribute_count": { "type": "long" },+ "attributes": {+ "type": "object",+ "dynamic": true+ }+ }+ }+ }+ },+ "data_stream": {+ "properties": {+ "type": { "type": "constant_keyword", "value": "logs" },+ "dataset": { "type": "constant_keyword", "value": "beacon.endpoint" },+ "namespace": { "type": "keyword" }+ }+ },+ "event": {+ "properties": {+ "kind": { "type": "keyword" },+ "action": { "type": "keyword" },+ "category": { "type": "keyword" },+ "dataset": { "type": "constant_keyword", "value": "beacon.endpoint" },+ "duration": { "type": "long" },+ "outcome": { "type": "keyword" },+ "severity": { "type": "short" }+ }+ },+ "host": {+ "properties": {+ "hostname": { "type": "keyword" },+ "name": { "type": "keyword" },+ "os": {+ "properties": {+ "family": { "type": "keyword" },+ "type": { "type": "keyword" }+ }+ }+ }+ },+ "user": {+ "properties": {+ "name": { "type": "keyword" },+ "id": { "type": "keyword" }+ }+ },+ "service": {+ "properties": {+ "name": { "type": "keyword" }+ }+ },+ "trace": {+ "properties": {+ "id": { "type": "keyword" }+ }+ },+ "process": {+ "properties": {+ "working_directory": { "type": "keyword" },+ "command_line": { "type": "wildcard" },+ "executable": { "type": "keyword" },+ "exit_code": { "type": "long" }+ }+ },+ "file": {+ "properties": {+ "path": { "type": "keyword" },+ "directory": { "type": "keyword" },+ "name": { "type": "keyword" },+ "extension": { "type": "keyword" },+ "size": { "type": "long" },+ "hash": {+ "properties": {+ "sha256": { "type": "keyword" }+ }+ }+ }+ },+ "gen_ai": {+ "properties": {+ "request": {+ "properties": {+ "model": { "type": "keyword" }+ }+ }+ }+ },+ "vcs": {+ "properties": {+ "repository": {+ "properties": {+ "url": { "type": "keyword" }+ }+ },+ "ref": {+ "properties": {+ "head": {+ "properties": {+ "name": { "type": "keyword" }+ }+ }+ }+ }+ }+ }+ }+ }+ },+ "_meta": {+ "description": "Beacon endpoint ECS and beacon namespace mappings",+ "ecs_version": "8.11.0"+ }+}diff --git a/cli/beacon/internal/endpoint/elastic/pack/component-template-settings.json b/cli/beacon/internal/endpoint/elastic/pack/component-template-settings.jsonnew file mode 100644index 00000000..7dd4b643--- /dev/null+++ b/cli/beacon/internal/endpoint/elastic/pack/component-template-settings.json@@ -0,0 +1,13 @@+{+ "template": {+ "settings": {+ "index.refresh_interval": "5s",+ "index.number_of_shards": 1,+ "index.number_of_replicas": 0,+ "index.lifecycle.name": "beacon-endpoint"+ }+ },+ "_meta": {+ "description": "Beacon endpoint data stream settings"+ }+}diff --git a/cli/beacon/internal/endpoint/elastic/pack/docker-compose.yml b/cli/beacon/internal/endpoint/elastic/pack/docker-compose.ymlnew file mode 100644index 00000000..91f28737--- /dev/null+++ b/cli/beacon/internal/endpoint/elastic/pack/docker-compose.yml@@ -0,0 +1,71 @@+services:+ elasticsearch:+ image: docker.elastic.co/elasticsearch/elasticsearch:8.15.0+ environment:+ - discovery.type=single-node+ - xpack.security.enabled=false+ - ES_JAVA_OPTS=-Xms1g -Xmx1g+ ports:+ - "127.0.0.1:${BEACON_ELASTIC_ES_PORT:-9200}:9200"+ healthcheck:+ test: ["CMD-SHELL", "curl -fsS http://localhost:9200/_cluster/health?wait_for_status=yellow&timeout=1s >/dev/null"]+ interval: 10s+ timeout: 5s+ retries: 30+ volumes:+ - beacon-es-data:/usr/share/elasticsearch/data++ kibana:+ image: docker.elastic.co/kibana/kibana:8.15.0+ environment:+ - ELASTICSEARCH_HOSTS=http://elasticsearch:9200+ ports:+ - "127.0.0.1:${BEACON_ELASTIC_KIBANA_PORT:-5601}:5601"+ depends_on:+ elasticsearch:+ condition: service_healthy+ healthcheck:+ test: ["CMD-SHELL", "curl -fsS http://localhost:5601/api/status >/dev/null"]+ interval: 10s+ timeout: 5s+ retries: 30++ setup:+ image: curlimages/curl:8.10.1+ working_dir: /pack+ volumes:+ - ".:/pack:ro"+ depends_on:+ elasticsearch:+ condition: service_healthy+ kibana:+ condition: service_healthy+ command:+ - sh+ - -eu+ - -c+ - |+ curl -fsS -X PUT http://elasticsearch:9200/_ilm/policy/beacon-endpoint -H 'Content-Type: application/json' --data-binary @ilm-policy.json+ curl -fsS -X PUT http://elasticsearch:9200/_component_template/beacon-endpoint-mappings -H 'Content-Type: application/json' --data-binary @component-template-mappings.json+ curl -fsS -X PUT http://elasticsearch:9200/_component_template/beacon-endpoint-settings -H 'Content-Type: application/json' --data-binary @component-template-settings.json+ curl -fsS -X PUT http://elasticsearch:9200/_index_template/beacon-endpoint -H 'Content-Type: application/json' --data-binary @index-template.json+ curl -fsS -X PUT http://elasticsearch:9200/_ingest/pipeline/beacon-endpoint -H 'Content-Type: application/json' --data-binary @ingest-pipeline.json+ curl -fsS -X POST http://kibana:5601/api/saved_objects/_import?overwrite=true -H 'kbn-xsrf: beacon' --form [email protected]++ filebeat:+ image: docker.elastic.co/beats/filebeat:8.15.0+ user: root+ command: ["filebeat", "-e", "--strict.perms=false"]+ environment:+ - ES_HOSTS=http://elasticsearch:9200+ volumes:+ - "${BEACON_LOG_DIR}:${BEACON_LOG_DIR}:ro"+ - "./filebeat.yml:/usr/share/filebeat/filebeat.yml:ro"+ depends_on:+ elasticsearch:+ condition: service_healthy+ setup:+ condition: service_completed_successfully++volumes:+ beacon-es-data:diff --git a/cli/beacon/internal/endpoint/elastic/pack/elastic-agent-standalone.yml.tmpl b/cli/beacon/internal/endpoint/elastic/pack/elastic-agent-standalone.yml.tmplnew file mode 100644index 00000000..291340b9--- /dev/null+++ b/cli/beacon/internal/endpoint/elastic/pack/elastic-agent-standalone.yml.tmpl@@ -0,0 +1,27 @@+outputs:+ default:+ type: elasticsearch+ hosts: ["${ES_HOSTS:http://localhost:9200}"]+ ssl.verification_mode: "${ES_SSL_VERIFY:full}"+ # For hosted or secured clusters, configure one auth method:+ # api_key: "${ES_API_KEY}"+ # username: "${ES_USERNAME}"+ # password: "${ES_PASSWORD}"++inputs:+ - id: beacon-endpoint+ type: filestream+ use_output: default+ streams:+ - id: beacon-endpoint-jsonl+ data_stream:+ type: logs+ dataset: beacon.endpoint+ namespace: default+ paths:+ - "{{LOG_PATH}}"+ parsers:+ - ndjson:+ keys_under_root: true+ add_error_key: true+ overwrite_keys: truediff --git a/cli/beacon/internal/endpoint/elastic/pack/filebeat.yml.tmpl b/cli/beacon/internal/endpoint/elastic/pack/filebeat.yml.tmplnew file mode 100644index 00000000..b98754a8--- /dev/null+++ b/cli/beacon/internal/endpoint/elastic/pack/filebeat.yml.tmpl@@ -0,0 +1,28 @@+filebeat.inputs:+ - type: filestream+ id: beacon-endpoint+ enabled: true+ paths:+ - "{{LOG_PATH}}"+ parsers:+ - ndjson:+ keys_under_root: true+ add_error_key: true+ overwrite_keys: true+ fields_under_root: true+ fields:+ data_stream.type: logs+ data_stream.dataset: beacon.endpoint+ data_stream.namespace: default++output.elasticsearch:+ hosts: ["${ES_HOSTS:http://localhost:9200}"]+ ssl.verification_mode: "${ES_SSL_VERIFY:full}"+ index: "logs-beacon.endpoint-default"+ # For hosted or secured clusters, configure one auth method:+ # api_key: "${ES_API_KEY}"+ # username: "${ES_USERNAME}"+ # password: "${ES_PASSWORD}"++setup.template.enabled: false+setup.ilm.enabled: falsediff --git a/cli/beacon/internal/endpoint/elastic/pack/ilm-policy.json b/cli/beacon/internal/endpoint/elastic/pack/ilm-policy.jsonnew file mode 100644index 00000000..cdae1b2f--- /dev/null+++ b/cli/beacon/internal/endpoint/elastic/pack/ilm-policy.json@@ -0,0 +1,20 @@+{+ "policy": {+ "phases": {+ "hot": {+ "actions": {+ "rollover": {+ "max_age": "30d",+ "max_primary_shard_size": "50gb"+ }+ }+ },+ "delete": {+ "min_age": "90d",+ "actions": {+ "delete": {}+ }+ }+ }+ }+}diff --git a/cli/beacon/internal/endpoint/elastic/pack/index-template.json b/cli/beacon/internal/endpoint/elastic/pack/index-template.jsonnew file mode 100644index 00000000..b4465635--- /dev/null+++ b/cli/beacon/internal/endpoint/elastic/pack/index-template.json@@ -0,0 +1,16 @@+{+ "index_patterns": ["logs-beacon.endpoint-*"],+ "data_stream": {},+ "priority": 200,+ "composed_of": ["beacon-endpoint-mappings", "beacon-endpoint-settings"],+ "template": {+ "settings": {+ "index.default_pipeline": "beacon-endpoint",+ "index.lifecycle.name": "beacon-endpoint"+ }+ },+ "_meta": {+ "beacon_schema_version": "1.0",+ "ecs_version": "8.11.0"+ }+}diff --git a/cli/beacon/internal/endpoint/elastic/pack/ingest-pipeline.json b/cli/beacon/internal/endpoint/elastic/pack/ingest-pipeline.jsonnew file mode 100644index 00000000..25fa8ada--- /dev/null+++ b/cli/beacon/internal/endpoint/elastic/pack/ingest-pipeline.json@@ -0,0 +1,92 @@+{+ "description": "Normalize Beacon endpoint JSONL events into ECS plus beacon.* fields",+ "processors": [+ {+ "set": {+ "field": "event.dataset",+ "value": "beacon.endpoint"+ }+ },+ {+ "set": {+ "field": "data_stream.type",+ "value": "logs"+ }+ },+ {+ "set": {+ "field": "data_stream.dataset",+ "value": "beacon.endpoint"+ }+ },+ {+ "set": {+ "field": "data_stream.namespace",+ "value": "default",+ "override": false+ }+ },+ {+ "date": {+ "field": "timestamp",+ "target_field": "@timestamp",+ "formats": ["ISO8601"],+ "ignore_failure": true+ }+ },+ {+ "script": {+ "description": "Copy Beacon fields to ECS and preserve native fields under beacon.*",+ "lang": "painless",+ "source": "Map ensure(Map parent, String key) { if (parent[key] == null || !(parent[key] instanceof Map)) { parent[key] = new HashMap(); } return parent[key]; } String str(def v) { if (v == null) { return null; } String s = v.toString(); return s.length() == 0 ? null : s; } Map beacon = ensure(ctx, 'beacon'); if (ctx.vendor != null) { beacon.vendor = ctx.vendor; } if (ctx.product != null) { beacon.product = ctx.product; } if (ctx.schema_version != null) { beacon.schema_version = ctx.schema_version; } if (ctx.severity != null) { beacon.severity = ctx.severity; } if (ctx.field_truncated != null) { beacon.truncated = ctx.field_truncated; } Map agent = ensure(ctx, 'agent'); agent.name = 'beacon'; agent.type = 'endpoint-agent'; if (ctx.endpoint instanceof Map && ctx.endpoint.agent_version != null) { agent.version = ctx.endpoint.agent_version; } else if (ctx.schema_version != null) { agent.version = ctx.schema_version; } Map event = ensure(ctx, 'event'); event.kind = 'event'; if (ctx.event instanceof Map) { if (ctx.event.action != null) { event.action = ctx.event.action; } if (ctx.event.category != null) { event.category = [ctx.event.category.toString()]; } } if (ctx.severity != null) { Map sev = new HashMap(); sev.put('info', 2); sev.put('low', 3); sev.put('medium', 4); sev.put('high', 5); sev.put('critical', 6); String key = ctx.severity.toString().toLowerCase(); if (sev.containsKey(key)) { event.severity = sev[key]; } } if (ctx.endpoint instanceof Map) { Map host = ensure(ctx, 'host'); if (ctx.endpoint.hostname != null) { host.hostname = ctx.endpoint.hostname; host.name = ctx.endpoint.hostname; } if (ctx.endpoint.os != null) { Map os = ensure(host, 'os'); String osName = ctx.endpoint.os.toString(); os.family = osName; os.type = osName == 'darwin' ? 'macos' : osName; } } if (ctx.user instanceof Map) { Map user = ensure(ctx, 'user'); if (ctx.user.name != null) { user.name = ctx.user.name; } if (ctx.user.uid != null) { user.id = ctx.user.uid; } } if (ctx.harness instanceof Map) { Map harness = ensure(beacon, 'harness'); if (ctx.harness.name != null) { harness.name = ctx.harness.name; ensure(ctx, 'service').name = ctx.harness.name; } if (ctx.harness.version != null) { harness.version = ctx.harness.version; } if (ctx.harness.executable_path != null) { harness.executable_path = ctx.harness.executable_path; } if (ctx.harness.config_path != null) { harness.config_path = ctx.harness.config_path; } } if (ctx.session instanceof Map) { Map session = ensure(beacon, 'session'); if (ctx.session.id != null) { session.id = ctx.session.id; if (ctx.trace == null) { ctx.trace = new HashMap(); } if (ctx.trace.id == null && ctx.session.id.toString().length() == 32) { ctx.trace.id = ctx.session.id; } } if (ctx.session.working_directory != null) { ensure(ctx, 'process').working_directory = ctx.session.working_directory; } } if (ctx.tool instanceof Map) { Map tool = ensure(beacon, 'tool'); if (ctx.tool.name != null) { tool.name = ctx.tool.name; } Map process = ensure(ctx, 'process'); if (process.command_line == null && ctx.tool.command != null) { process.command_line = ctx.tool.command; } if (ctx.tool.path != null) { process.executable = ctx.tool.path; } } if (ctx.command instanceof Map) { Map process = ensure(ctx, 'process'); if (ctx.command.command != null) { process.command_line = ctx.command.command; } if (ctx.command.exit_code != null) { process.exit_code = ctx.command.exit_code; } if (ctx.command.duration_ms != null) { event.duration = ((long)ctx.command.duration_ms) * 1000000L; } } if (ctx.file instanceof Map) { Map file = ensure(ctx, 'file'); Map beaconFile = ensure(beacon, 'file'); if (ctx.file.path != null) { String p = ctx.file.path.toString(); file.path = p; int slash = p.lastIndexOf('/'); String name = slash >= 0 ? p.substring(slash + 1) : p; if (slash > 0) { file.directory = p.substring(0, slash); } file.name = name; int dot = name.lastIndexOf('.'); if (dot > 0 && dot < name.length() - 1) { file.extension = name.substring(dot + 1); } } if (ctx.file.diff_hash != null) { ensure(file, 'hash').sha256 = ctx.file.diff_hash; } if (ctx.file.diff_bytes != null) { file.size = ctx.file.diff_bytes; } if (ctx.file.operation != null) { beaconFile.operation = ctx.file.operation; } if (ctx.file.language != null) { beaconFile.language = ctx.file.language; } } if (ctx.mcp instanceof Map) { Map mcp = ensure(beacon, 'mcp'); if (ctx.mcp.server != null) { mcp.server = ctx.mcp.server; } if (ctx.mcp.tool != null) { mcp.tool = ctx.mcp.tool; } } if (ctx.approval instanceof Map) { Map approval = ensure(beacon, 'approval'); if (ctx.approval.required != null) { approval.required = ctx.approval.required; } if (ctx.approval.decision != null) { String decision = ctx.approval.decision.toString(); approval.decision = decision; if (decision == 'allow' || decision == 'approved') { event.outcome = 'success'; } else if (decision == 'deny' || decision == 'denied' || decision == 'block' || decision == 'blocked') { event.outcome = 'failure'; } } if (ctx.approval.reason != null) { approval.reason = ctx.approval.reason; } } if (ctx.policy instanceof Map) { Map policy = ensure(beacon, 'policy'); if (ctx.policy.id != null) { policy.id = ctx.policy.id; } if (ctx.policy.name != null) { policy.name = ctx.policy.name; } if (ctx.policy.decision != null) { policy.decision = ctx.policy.decision; } if (ctx.policy.enforcement != null) { policy.enforcement = ctx.policy.enforcement; } if (ctx.policy.reason != null) { policy.reason = ctx.policy.reason; } } if (ctx.prompt instanceof Map && ctx.prompt.text != null) { ensure(beacon, 'prompt').text = ctx.prompt.text; } if (ctx.content instanceof Map) { Map content = ensure(beacon, 'content'); if (ctx.content.retention != null) { content.retention = ctx.content.retention; } if (ctx.content.included != null) { content.included = ctx.content.included; } if (ctx.content.redacted != null) { content.redacted = ctx.content.redacted; } if (ctx.content.truncated != null) { content.truncated = ctx.content.truncated; } } if (ctx.destination instanceof Map) { Map destination = ensure(beacon, 'destination'); if (ctx.destination.type != null) { destination.type = ctx.destination.type; } if (ctx.destination.mode != null) { destination.mode = ctx.destination.mode; } if (ctx.destination.status != null) { destination.status = ctx.destination.status; } } if (ctx.health instanceof Map) { Map health = ensure(beacon, 'health'); if (ctx.health.component != null) { health.component = ctx.health.component; } if (ctx.health.status != null) { health.status = ctx.health.status; } if (ctx.health.reason != null) { health.reason = ctx.health.reason; } } if (ctx.model != null) { ensure(ensure(ctx, 'gen_ai'), 'request').model = ctx.model; } if (ctx.repository != null) { ensure(ensure(ctx, 'vcs'), 'repository').url = ctx.repository; } if (ctx.branch != null) { ensure(ensure(ensure(ctx, 'vcs'), 'ref'), 'head').name = ctx.branch; } if (ctx.raw instanceof Map) { Map otel = ensure(beacon, 'otel'); if (ctx.raw.otel_signal != null) { otel.signal = ctx.raw.otel_signal; } if (ctx.raw.metric_name != null) { ensure(beacon, 'metric').name = ctx.raw.metric_name; if (event.action == null || event.action == 'metric.observed') { event.action = ctx.raw.metric_name; } } if (ctx.raw.metric_description != null) { otel.metric_description = ctx.raw.metric_description; } if (ctx.raw.metric_unit != null) { otel.metric_unit = ctx.raw.metric_unit; } if (ctx.raw.attribute_count != null) { otel.attribute_count = ctx.raw.attribute_count; } if (ctx.raw.attributes != null) { otel.attributes = ctx.raw.attributes; } }"+ }+ },+ {+ "remove": {+ "field": [+ "timestamp",+ "vendor",+ "product",+ "schema_version",+ "severity",+ "endpoint",+ "harness",+ "session",+ "tool",+ "command",+ "mcp",+ "approval",+ "policy",+ "prompt",+ "content",+ "destination",+ "health",+ "model",+ "repository",+ "branch",+ "raw",+ "field_truncated"+ ],+ "ignore_missing": true+ }+ }+ ],+ "on_failure": [+ {+ "set": {+ "field": "error.message",+ "value": "{{ _ingest.on_failure_message }}"+ }+ },+ {+ "set": {+ "field": "_index",+ "value": "beacon-endpoint-failures"+ }+ }+ ],+ "_meta": {+ "beacon_schema_version": "1.0",+ "ecs_version": "8.11.0"+ }+}diff --git a/cli/beacon/internal/endpoint/elastic/pack/kibana-assets.ndjson b/cli/beacon/internal/endpoint/elastic/pack/kibana-assets.ndjsonnew file mode 100644index 00000000..d8b123b9--- /dev/null+++ b/cli/beacon/internal/endpoint/elastic/pack/kibana-assets.ndjson@@ -0,0 +1,2 @@+{"type":"index-pattern","id":"beacon-endpoint-data-view","attributes":{"title":"logs-beacon.endpoint-*","timeFieldName":"@timestamp","name":"Beacon Endpoint"},"references":[],"coreMigrationVersion":"8.8.0","typeMigrationVersion":"8.0.0"}+{"type":"search","id":"beacon-all-agent-activity","attributes":{"title":"Beacon: all agent activity","description":"All Beacon endpoint events","columns":["@timestamp","event.action","beacon.harness.name","user.name","message"],"sort":[["@timestamp","desc"]],"kibanaSavedObjectMeta":{"searchSourceJSON":"{\"query\":{\"language\":\"kuery\",\"query\":\"event.dataset: beacon.endpoint\"},\"filter\":[],\"indexRefName\":\"kibanaSavedObjectMeta.searchSourceJSON.index\"}"}},"references":[{"name":"kibanaSavedObjectMeta.searchSourceJSON.index","type":"index-pattern","id":"beacon-endpoint-data-view"}],"coreMigrationVersion":"8.8.0","typeMigrationVersion":"8.0.0"}diff --git a/packaging/macos/smoke-endpoint.sh b/packaging/macos/smoke-endpoint.shindex ec27fcf2..f065c568 100755--- a/packaging/macos/smoke-endpoint.sh+++ b/packaging/macos/smoke-endpoint.sh@@ -74,6 +74,9 @@ run_beacon endpoint status --user --log-path "$LOG_PATH" >/dev/nullecho "Writing Wazuh validation event..."run_beacon endpoint wazuh validate --user --log-path "$LOG_PATH" >/dev/null+echo "Checking Elastic Filebeat config generation..."+run_beacon endpoint elastic print-config --user --log-path "$LOG_PATH" >/dev/null+if ! grep -q '"action":"telemetry.enabled"' "$LOG_PATH"; thenecho "expected telemetry.enabled event in runtime log" >&2exit 1