Skip to content

The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu

fix(codex): record the final assistant reply from a Stop hook

Claudef3cd940e3522
diff --git a/CLAUDE.md b/CLAUDE.md
index 93fcc126..d39518f1 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -42,7 +42,7 @@ Do not recreate or depend on removed `asymptote` mirror trees. Keep new work foc
Supported runtime surfaces today:
-- Claude Code and Codex CLI use local OpenTelemetry settings, and `beacon endpoint install` writes their user-level hooks alongside those settings (`otlpTargetCarriesHook` in `cli/beacon/cmd/endpoint_targets.go`), because Claude's OTLP export has no session lifecycle, file activity, or subagent events and Codex's SessionStart hook supplies the OS-user context its spans lack. `beacon endpoint codex sync` is the offline poll backfill from `~/.codex/sessions`; it never reads `~/.codex/auth.json` and cannot approve, deny, or delay tool calls.
+- Claude Code and Codex CLI use local OpenTelemetry settings, and `beacon endpoint install` writes their user-level hooks alongside those settings (`otlpTargetCarriesHook` in `cli/beacon/cmd/endpoint_targets.go`), because Claude's OTLP export has no session lifecycle, file activity, or subagent events and Codex's SessionStart hook supplies the OS-user context its spans lack. Both runtimes' Stop hooks record the turn's final reply from `last_assistant_message` as `agent.message` (Codex through the dedicated `codex-assistant-response` command, since its OTLP export carries no response text and it gets no generic hooks). `beacon endpoint codex sync` is the offline poll backfill from `~/.codex/sessions`; it never reads `~/.codex/auth.json` and cannot approve, deny, or delay tool calls.
- Cursor hook telemetry for sessions, prompt submission, tool use, command execution, MCP-like tool activity, approval decisions, file edits, and agent reasoning (`afterAgentThought` thinking text recorded as `agent.reasoning` events in the OTel GenAI `gen_ai.output.messages` reasoning-part shape) where hook payloads expose those fields.
- Cline uses a Beacon-managed plugin (`~/.cline/plugins/beacon.ts`) for live capture plus `beacon endpoint cline sync` for local history backfill. Cline's own OTel export/hosted prompt storage are not used, and approvals are not exposed or synthesized.
- Pi-family runtimes use Beacon-managed extensions and the shared `cli/beacon-hooks/cmd/pi_family.go` mapper while keeping separate harness names and install roots: Pi (`pi_cli`), Oh My Pi (`omp`), Prime Agent (`prime_agent`), and Senpi/OMO (`omo_senpi`). Pi, Prime, and Senpi do not expose operator approval decisions, so Beacon does not synthesize them. Oh My Pi exposes real approval events and the extension carries the remembered tool input onto those approvals.
diff --git a/cli/beacon-hooks/cmd/codex_assistant_response.go b/cli/beacon-hooks/cmd/codex_assistant_response.go
new file mode 100644
index 00000000..f2819216
--- /dev/null
+++ b/cli/beacon-hooks/cmd/codex_assistant_response.go
@@ -0,0 +1,38 @@
+package cmd
+
+import (
+ "github.com/spf13/cobra"
+)
+
+var codexAssistantResponseCmd = &cobra.Command{
+ Use: "codex-assistant-response",
+ Short: "Record the final assistant reply of a Codex turn",
+ Hidden: true,
+ Run: runCodexAssistantResponse,
+}
+
+func init() {
+ rootCmd.AddCommand(codexAssistantResponseCmd)
+}
+
+// runCodexAssistantResponse is Codex's Stop hook, and it records one thing: the turn's final reply,
+// which Codex's Stop payload documents as `last_assistant_message`.
+//
+// Codex's OTLP export carries prompts, tool decisions, tool results and token usage but never what
+// the agent said, so before this the reply reached the log only when someone ran
+// `beacon endpoint codex sync`. It is a dedicated command rather than the shared `stop` for the
+// reason Codex has no other generic hooks: OTLP already reports the turn, and a second completion
+// event from a hook would duplicate it. Codex reads a Stop hook's stdout as JSON, so every path
+// answers `{}`, which continues the turn's normal end.
+func runCodexAssistantResponse(cmd *cobra.Command, args []string) {
+ input, err := readStdinJSON()
+ if err != nil || platformFlag != "codex" {
+ outputJSON(emptyResponse)
+ return
+ }
+ sessionID := resolveSessionID(input, "codex")
+ logger := newHookLogger("codex-assistant-response", "codex", sessionID)
+ fields := sessionFieldsForPlatform(sessionID, input, "codex")
+ emitAssistantMessage(logger, input, fields, getFirstStr(input, "last_assistant_message"))
+ outputJSON(emptyResponse)
+}
diff --git a/cli/beacon-hooks/cmd/codex_assistant_response_test.go b/cli/beacon-hooks/cmd/codex_assistant_response_test.go
new file mode 100644
index 00000000..12f5026d
--- /dev/null
+++ b/cli/beacon-hooks/cmd/codex_assistant_response_test.go
@@ -0,0 +1,89 @@
+package cmd
+
+import (
+ "strings"
+ "testing"
+)
+
+// codexStopPayload is Codex's documented Stop input: the common fields plus turn_id,
+// stop_hook_active and last_assistant_message (string or null).
+func codexStopPayload(lastMessage interface{}) map[string]interface{} {
+ return map[string]interface{}{
+ "session_id": "019a-codex-thread",
+ "transcript_path": "/nonexistent/rollout.jsonl",
+ "cwd": "/repo",
+ "hook_event_name": "Stop",
+ "model": "gpt-5.5-codex",
+ "permission_mode": "default",
+ "turn_id": "turn-3",
+ "stop_hook_active": false,
+ "last_assistant_message": lastMessage,
+ }
+}
+
+// Codex's OTLP export never carries what the agent said; the Stop payload does. The hook records it
+// as the same agent.message the codex sync backfill writes, under the live turn's identity.
+func TestCodexAssistantResponseRecordsTheFinalReply(t *testing.T) {
+ reply := "Renamed the handler and updated both call sites."
+ logPath, stdout := runHookProcess(t, codexStopPayload(reply), "--platform", "codex", "codex-assistant-response")
+
+ if strings.TrimSpace(string(stdout)) != "{}" {
+ t.Fatalf("stdout = %q, want {}: Codex reads a Stop hook's stdout as JSON", stdout)
+ }
+ events := endpointEvents(t, logPath)
+ if len(events) != 1 {
+ t.Fatalf("wrote %d events, want exactly the reply (OTLP already reports the turn): %#v", len(events), events)
+ }
+ message := events[0]
+ if got := leaf(message, "event", "action"); got != "agent.message" {
+ t.Fatalf("event.action = %q, want agent.message", got)
+ }
+ if got := outputText(t, message); got != reply {
+ t.Fatalf("reply = %q, want %q", got, reply)
+ }
+ for _, check := range []struct {
+ path []string
+ want string
+ }{
+ {[]string{"harness", "name"}, "codex_cli"},
+ {[]string{"harness", "collection_method"}, "hook"},
+ {[]string{"session", "id"}, "019a-codex-thread"},
+ {[]string{"prompt", "id"}, "turn-3"},
+ {[]string{"model"}, "gpt-5.5-codex"},
+ } {
+ if got := leaf(message, check.path...); got != check.want {
+ t.Errorf("%s = %q, want %q", strings.Join(check.path, "."), got, check.want)
+ }
+ }
+ if _, ok := message["content"].(map[string]interface{}); !ok {
+ t.Fatalf("no content marker: %#v", message)
+ }
+}
+
+// A turn that ended without a reply -- Codex sends null -- writes nothing at all, and still answers
+// with valid JSON so Codex ends the turn normally.
+func TestCodexAssistantResponseWithoutAReplyWritesNothing(t *testing.T) {
+ for name, value := range map[string]interface{}{"null": nil, "empty": "", "blank": " \n"} {
+ t.Run(name, func(t *testing.T) {
+ logPath, stdout := runHookProcess(t, codexStopPayload(value), "--platform", "codex", "codex-assistant-response")
+ if strings.TrimSpace(string(stdout)) != "{}" {
+ t.Fatalf("stdout = %q, want {}", stdout)
+ }
+ if events := endpointEventsIfAny(t, logPath); len(events) != 0 {
+ t.Fatalf("wrote %d events for a turn with no reply: %#v", len(events), events)
+ }
+ })
+ }
+}
+
+// The command is Codex's alone: under another platform flag it records nothing, like
+// codex-session-context.
+func TestCodexAssistantResponseIgnoresOtherPlatforms(t *testing.T) {
+ logPath, stdout := runHookProcess(t, codexStopPayload("not codex"), "--platform", "claude", "codex-assistant-response")
+ if strings.TrimSpace(string(stdout)) != "{}" {
+ t.Fatalf("stdout = %q, want {}", stdout)
+ }
+ if events := endpointEventsIfAny(t, logPath); len(events) != 0 {
+ t.Fatalf("recorded %d events under --platform claude: %#v", len(events), events)
+ }
+}
diff --git a/cli/beacon/internal/endpoint/hooks/codex.go b/cli/beacon/internal/endpoint/hooks/codex.go
index 2881b31f..1ea1181b 100644
--- a/cli/beacon/internal/endpoint/hooks/codex.go
+++ b/cli/beacon/internal/endpoint/hooks/codex.go
@@ -64,12 +64,18 @@ func codexStatusFromRuntime(status runtimeStatus) CodexStatus {
func installCodexHooks(path, binaryPath, logPath, configPath string) error {
prefix := endpointCommandPrefix("codex", binaryPath, logPath, configPath)
- // Only the OS-user session context is hook-driven. Inventory used to ride SessionStart and
+ // Hooks supply only what Codex's OTLP export lacks: the OS-user session context, and the turn's
+ // final reply, which Codex's Stop payload carries as last_assistant_message and no span or log
+ // does. Both are dedicated commands rather than the generic ones, because OTLP already reports
+ // the session, prompts, tools and turn. Inventory used to ride SessionStart and
// UserPromptSubmit here; it is written by the scheduled inventory job now.
endpointHooks := map[string]settingsHookGroup{
"SessionStart": {Hooks: []settingsHookRef{
{Type: "command", Command: prefix + " codex-session-context", Timeout: 10},
}},
+ "Stop": {Hooks: []settingsHookRef{
+ {Type: "command", Command: prefix + " codex-assistant-response", Timeout: 10},
+ }},
}
return installSettingsEndpointHooks(path, "codex", endpointHooks)
}
diff --git a/cli/beacon/internal/endpoint/hooks/codex_test.go b/cli/beacon/internal/endpoint/hooks/codex_test.go
index 7cb59430..c52e70fc 100644
--- a/cli/beacon/internal/endpoint/hooks/codex_test.go
+++ b/cli/beacon/internal/endpoint/hooks/codex_test.go
@@ -1,6 +1,7 @@
package hooks
import (
+ "encoding/json"
"os"
"path/filepath"
"strings"
@@ -9,7 +10,7 @@ import (
"github.com/asymptote-labs/agent-beacon/cli/beacon/internal/testenv"
)
-func TestInstallCodexHooksWritesSessionContextOnly(t *testing.T) {
+func TestInstallCodexHooksWritesSessionContextAndAssistantResponse(t *testing.T) {
path := filepath.Join(t.TempDir(), "hooks.json")
existing := `{"hooks":{"SessionStart":[{"hooks":[{"type":"command","command":"echo keep"},{"type":"command","command":"BEACON_ENDPOINT_MODE=1 beacon-hooks --platform codex inventory-heartbeat"}]}],"Stop":[{"hooks":[{"type":"command","command":"BEACON_ENDPOINT_MODE=1 beacon-hooks --platform codex codex-usage-sync"}]}]}}`
if err := os.WriteFile(path, []byte(existing), 0600); err != nil {
@@ -30,6 +31,7 @@ func TestInstallCodexHooksWritesSessionContextOnly(t *testing.T) {
"SessionStart",
"--platform codex",
"codex-session-context",
+ "codex-assistant-response",
"--log '/tmp/runtime.jsonl'",
"--config '/tmp/config.json'",
} {
@@ -43,8 +45,49 @@ func TestInstallCodexHooksWritesSessionContextOnly(t *testing.T) {
t.Fatalf("Codex hooks should not install duplicate generic runtime hooks %q:\n%s", forbidden, text)
}
}
- if !strings.Contains(text, "SessionStart") || strings.Contains(text, `"Stop"`) {
- t.Fatalf("existing non-Beacon SessionStart should remain and old Beacon Stop should be removed:\n%s", text)
+ if !strings.Contains(text, "SessionStart") {
+ t.Fatalf("existing non-Beacon SessionStart should remain:\n%s", text)
+ }
+ // The old Beacon Stop entry is replaced, not kept beside the new one: Stop holds exactly the
+ // assistant-response command.
+ var parsed struct {
+ Hooks map[string][]settingsHookGroup `json:"hooks"`
+ }
+ if err := json.Unmarshal(data, &parsed); err != nil {
+ t.Fatalf("decode hooks: %v", err)
+ }
+ var stopCommands []string
+ for _, group := range parsed.Hooks["Stop"] {
+ for _, hook := range group.Hooks {
+ stopCommands = append(stopCommands, hook.Command)
+ }
+ }
+ if len(stopCommands) != 1 || !strings.HasSuffix(stopCommands[0], " codex-assistant-response") {
+ t.Fatalf("Stop commands = %q, want only codex-assistant-response", stopCommands)
+ }
+}
+
+func TestInstallCodexHooksIsIdempotent(t *testing.T) {
+ path := filepath.Join(t.TempDir(), "hooks.json")
+ for i := 0; i < 2; i++ {
+ if err := installCodexHooks(path, "/tmp/beacon-hooks", "/tmp/runtime.jsonl", ""); err != nil {
+ t.Fatalf("install %d: %v", i+1, err)
+ }
+ }
+ data, err := os.ReadFile(path)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if n := strings.Count(string(data), "codex-assistant-response"); n != 1 {
+ t.Fatalf("codex-assistant-response appears %d times after two installs, want 1:\n%s", n, data)
+ }
+ removed, err := removeCodexEndpointHooks(path)
+ if err != nil || !removed {
+ t.Fatalf("remove = %v, %v", removed, err)
+ }
+ data, _ = os.ReadFile(path)
+ if strings.Contains(string(data), "codex-assistant-response") || strings.Contains(string(data), "codex-session-context") {
+ t.Fatalf("uninstall left Beacon hooks behind:\n%s", data)
}
}
diff --git a/docs/runtimes/codex-cli.mdx b/docs/runtimes/codex-cli.mdx
index 91b7830f..4d55194f 100644
--- a/docs/runtimes/codex-cli.mdx
+++ b/docs/runtimes/codex-cli.mdx
@@ -5,7 +5,7 @@ description: "Beacon support details for Codex CLI and Codex Desktop endpoint te
## Runtime overview
-Beacon supports Codex CLI runtime telemetry through local OTLP logs and one selective usage-bearing turn trace. Codex Desktop uses the same Codex OTLP configuration path through its app-server process, and Beacon records it as `codex_desktop` instead of merging it with `codex_cli`. A lightweight SessionStart hook records the local OS user for CLI token attribution.
+Beacon supports Codex CLI runtime telemetry through local OTLP logs and one selective usage-bearing turn trace. Codex Desktop uses the same Codex OTLP configuration path through its app-server process, and Beacon records it as `codex_desktop` instead of merging it with `codex_cli`. A lightweight SessionStart hook records the local OS user for CLI token attribution, and a Stop hook records each turn's final assistant reply, which Codex's OTLP export does not carry.
## Prerequisites
@@ -20,7 +20,7 @@ Before enabling Codex telemetry, make sure:
## Collection path
-Codex CLI exports local OpenTelemetry logs and traces to Beacon's localhost collector. Codex Desktop exports the same Codex semantic telemetry from `codex-app-server` when user-level `~/.codex/config.toml` enables OTLP. Beacon uses Codex semantic logs for session, prompt, approval, and tool-result events. For tokens it keeps only the completed `session_task.turn` span carrying `thread.id`, `turn.id`, model, and usage; hundreds of transport and internal spans remain suppressed. The CLI SessionStart hook writes a metadata-only `session.context` event; it never reads transcript content. Inventory is written by the scheduled inventory job, not by this hook.
+Codex CLI exports local OpenTelemetry logs and traces to Beacon's localhost collector. Codex Desktop exports the same Codex semantic telemetry from `codex-app-server` when user-level `~/.codex/config.toml` enables OTLP. Beacon uses Codex semantic logs for session, prompt, approval, and tool-result events. For tokens it keeps only the completed `session_task.turn` span carrying `thread.id`, `turn.id`, model, and usage; hundreds of transport and internal spans remain suppressed. The CLI SessionStart hook writes a metadata-only `session.context` event. The CLI Stop hook writes the turn's final reply from the payload's `last_assistant_message` as an `agent.message` event, and nothing for a turn that ended without one. Neither hook reads the transcript. Inventory is written by the scheduled inventory job, not by this hook.
Use `--include-codex-spans` during install or repair only when you need every raw Codex span for troubleshooting. It is not required for token attribution.
@@ -92,24 +92,27 @@ Beacon detects Codex CLI through the `codex` executable and `$CODEX_HOME/config.
`beacon endpoint install` writes Codex OTLP exporter tables to `$CODEX_HOME/config.toml` (default `~/.codex/config.toml`).
`beacon endpoint install --harness codex` installs the required user-level
-session context and inventory hooks with the OTLP configuration. To refresh
+session context and assistant response hooks with the OTLP configuration. To refresh
them independently, run:
```bash
beacon endpoint hooks install --harness codex
```
-The hook installer writes the metadata-only session context hook to `$CODEX_HOME/hooks.json` (default `~/.codex/hooks.json`) or project `.codex/hooks.json`, depending on `--level`. A successful user-level install prints:
+The hook installer writes the session context (`SessionStart`) and assistant response (`Stop`) hooks to `$CODEX_HOME/hooks.json` (default `~/.codex/hooks.json`) or project `.codex/hooks.json`, depending on `--level`. A successful user-level install prints:
```text
Codex CLI endpoint hooks installed: /Users/you/.codex/hooks.json
```
+Codex runs a non-managed hook only after you review and trust its exact definition. Run `/hooks` in Codex to trust the Beacon hooks after installing or upgrading; until then Codex skips them, and the assistant reply reaches the log only through `beacon endpoint codex sync`.
+
## Telemetry coverage
| Area | Support |
|------|---------|
| Prompt telemetry | Supported when emitted through Codex semantic logs |
+| Agent responses | Supported live through the Stop hook (`last_assistant_message`, recorded as `agent.message`); Codex's OTLP export carries no response text. `beacon endpoint codex sync` also recovers every committed assistant message |
| Turn identity | Supported. Codex's `turn_id` from hooks, `turn.id` from OTLP, and the turn bracket in `beacon endpoint codex sync` are promoted to `prompt.id` |
| Command, tool, and file telemetry | Supported when emitted through Codex semantic logs |
| Token usage | Supported per turn through a selective OTLP trace (input, output, cache read, cache creation, reasoning) |