The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu
Go agent-ai-cliagent-harnessesagent-securitybrowser-extensionciclaude-codecloudcodexcursordetection-engineeringjevknowledge-basememorymobile-device-managementobservabilitysecuritysecurity-information-and-event-managementsecurity-toolstelemetrytraces
fix(cline): read the run outcome at task end instead of assuming success
Claude7db19e999c88
diff --git a/cli/beacon-hooks/cmd/cline_event.go b/cli/beacon-hooks/cmd/cline_event.goindex 296e8154..840d104d 100644--- a/cli/beacon-hooks/cmd/cline_event.go+++ b/cli/beacon-hooks/cmd/cline_event.go@@ -127,19 +127,23 @@ func clineEndpointEvents(input map[string]interface{}, sessionID string) []normacase clineStageToolAfter:return clineToolAfterEvents(input, fields)case clineStageTaskEnd:+ // A task end does not imply a task success. The plugin surface has one run-completion+ // handler for all three outcomes, so a cancel or a failure arrives here rather than at+ // the cancel and error stages, which only the file-based surface names separately.+ switch clineTaskOutcome(input) {+ case clineOutcomeCancelled:+ return clineCancelEvents(input, fields, one)+ case clineOutcomeFailed:+ return clineErrorEvents(input, fields, one)+ }if usage := clineUsage(input); len(usage) > 0 {fields["gen_ai"] = mergeNested(fields["gen_ai"], map[string]interface{}{"usage": usage})}return one("session.ended", "session", "info", "Cline task completed", fields)case clineStageTaskCancel:- if usage := clineUsage(input); len(usage) > 0 {- fields["gen_ai"] = mergeNested(fields["gen_ai"], map[string]interface{}{"usage": usage})- }- fields["session"] = mergeNested(fields["session"], map[string]interface{}{"cancel_reason": clineCancelReason(input)})- return one("session.ended", "session", "info", "Cline task cancelled", fields)+ return clineCancelEvents(input, fields, one)case clineStageTaskError:- fields["error"] = map[string]interface{}{"type": clineErrorType(input)}- return one("session.error", "session", "high", "Cline task ended with an error", fields)+ return clineErrorEvents(input, fields, one)default:return nil}@@ -686,6 +690,71 @@ func clineToolAfterEvents(input map[string]interface{}, fields map[string]interf}}}+// Outcomes a run-completion payload can report, beyond plain success.+const (+ clineOutcomeCancelled = "cancelled"+ clineOutcomeFailed = "failed"+)++// clineTaskOutcome reads how a Cline task actually finished.+//+// This exists because the two Cline surfaces disagree about where the outcome lives. The+// file-based hooks name it in the stage itself -- TaskCancel, error -- so the stage alone is+// enough. The plugin SDK, which is the surface Beacon installs, has a single run-completion+// handler for every outcome and reports the difference as a field on the context. Without this+// read, an aborted or failed task reaches the log as a clean session.ended, and the cancel and+// error paths below would be unreachable through Beacon's own plugin.+//+// Returns "" when nothing says otherwise, so a payload that reports no outcome stays a success --+// the same behavior as before this read existed. Cline is not documented down to the field level+// here, so several spellings are accepted and an unrecognized value is treated as success rather+// than guessed at: mislabelling a completed task as failed is worse than missing a label.+func clineTaskOutcome(input map[string]interface{}) string {+ status := getFirstStr(input, "status", "outcome", "runStatus", "run_status", "completionStatus", "completion_status")+ if status == "" {+ for _, key := range []string{"result", "run", "task", "taskMetadata", "task_metadata", "metadata"} {+ if nested := firstMap(input, key); nested != nil {+ status = getFirstStr(nested, "status", "outcome", "runStatus", "run_status", "completionStatus", "completion_status")+ if status != "" {+ break+ }+ }+ }+ }+ switch strings.ToLower(strings.TrimSpace(status)) {+ case "aborted", "abort", "cancelled", "canceled", "cancel", "user_cancelled", "user_canceled", "interrupted":+ return clineOutcomeCancelled+ case "failed", "failure", "error", "errored":+ return clineOutcomeFailed+ }+ // Deliberately no fallback to "there is an error object on the payload". clineToolFailure+ // answers that question for a tool call, where the error belongs to the call itself; a run+ // context is not the same thing, and a run that carried an error field for any other reason+ // would be reported as a failed task. A false session.error is worse than a missed one in a+ // log people alert on, so the status field is the only signal until a capture shows another.+ return ""+}++// clineCancelEvents and clineErrorEvents keep the cancel and failure shapes in one place, since+// both the stage-named surface and the outcome-carrying one have to produce them identically.+func clineCancelEvents(input, fields map[string]interface{}, one func(action, category, severity, message string, values map[string]interface{}) []normalizedEvent) []normalizedEvent {+ if usage := clineUsage(input); len(usage) > 0 {+ fields["gen_ai"] = mergeNested(fields["gen_ai"], map[string]interface{}{"usage": usage})+ }+ fields["session"] = mergeNested(fields["session"], map[string]interface{}{"cancel_reason": clineCancelReason(input)})+ return one("session.ended", "session", "info", "Cline task cancelled", fields)+}++func clineErrorEvents(input, fields map[string]interface{}, one func(action, category, severity, message string, values map[string]interface{}) []normalizedEvent) []normalizedEvent {+ // Usage is recorded on a failed task too: the tokens were spent whether or not the task+ // finished, and dropping them would understate the task's cost.+ if usage := clineUsage(input); len(usage) > 0 {+ fields["gen_ai"] = mergeNested(fields["gen_ai"], map[string]interface{}{"usage": usage})+ }+ fields["error"] = map[string]interface{}{"type": clineErrorType(input)}+ return one("session.error", "session", "high", "Cline task ended with an error", fields)+}+// clineCancelReason reports why a task ended without completing: cancelled, abandoned, or whatever// else the payload says.//@@ -716,6 +785,13 @@ func clineCancelReason(input map[string]interface{}) string {return reason}}+ // A cancel recognized from the run status rather than the stage name has that status as its+ // only stated reason, and "aborted" says more than the generic default does.+ for _, outer := range outers {+ if status := getFirstStr(outer, "status", "outcome", "runStatus", "run_status"); status != "" {+ return status+ }+ }return "cancelled"}diff --git a/cli/beacon-hooks/cmd/cline_event_test.go b/cli/beacon-hooks/cmd/cline_event_test.goindex ed1b1c09..24b856ca 100644--- a/cli/beacon-hooks/cmd/cline_event_test.go+++ b/cli/beacon-hooks/cmd/cline_event_test.go@@ -748,3 +748,105 @@ func TestClineCancelReasonReadsBothShapes(t *testing.T) {})}}++// The plugin surface -- the one Beacon installs -- has a single run-completion handler for every+// outcome, so an aborted or failed task arrives as afterRun and says so in a field. Before this was+// read, both reached the log as a clean session.ended, which made the cancel and error paths+// unreachable through Beacon's own plugin no matter what the file-hook tests proved.+func TestClineEventAfterRunHonorsTheRunOutcome(t *testing.T) {+ for _, tc := range []struct {+ name string+ payload map[string]interface{}+ wantAction string+ wantSeverity string+ wantCancel string+ wantErrType string+ }{+ {+ name: "no status is still a success",+ payload: map[string]interface{}{},+ wantAction: "session.ended",+ wantSeverity: "info",+ },+ {+ name: "explicit success",+ payload: map[string]interface{}{"status": "completed"},+ wantAction: "session.ended",+ wantSeverity: "info",+ },+ {+ name: "aborted is a cancel",+ payload: map[string]interface{}{"status": "aborted"},+ wantAction: "session.ended",+ wantSeverity: "info",+ wantCancel: "aborted",+ },+ {+ name: "failed is an error",+ payload: map[string]interface{}{"status": "failed"},+ wantAction: "session.error",+ wantSeverity: "high",+ wantErrType: "task_error",+ },+ {+ name: "status on the nested result",+ payload: map[string]interface{}{"result": map[string]interface{}{"status": "cancelled"}},+ wantAction: "session.ended",+ wantSeverity: "info",+ wantCancel: "cancelled",+ },+ {+ name: "a failure names its error type when it has one",+ payload: map[string]interface{}{"status": "failed", "error": map[string]interface{}{"name": "ProviderTimeout"}},+ wantAction: "session.error",+ wantSeverity: "high",+ wantErrType: "ProviderTimeout",+ },+ {+ // An unrecognized value must not be guessed at: a false session.error is worse than a+ // missing one in a log people alert on.+ name: "an unknown status stays a success",+ payload: map[string]interface{}{"status": "whatever-cline-adds-next"},+ wantAction: "session.ended",+ wantSeverity: "info",+ },+ } {+ t.Run(tc.name, func(t *testing.T) {+ logPath := clineTestLog(t)+ payload := map[string]interface{}{+ "type": "afterRun",+ "taskId": "task-outcome",+ "usage": map[string]interface{}{"inputTokens": 70, "outputTokens": 9},+ }+ for k, v := range tc.payload {+ payload[k] = v+ }+ runHookWithInput(t, runClineEvent, payload)++ event := clineEventWithAction(t, logPath, tc.wantAction)+ if got := event["severity"]; got != tc.wantSeverity {+ t.Errorf("severity = %q, want %q", got, tc.wantSeverity)+ }+ if tc.wantCancel == "" {+ if session, ok := event["session"].(map[string]interface{}); ok {+ if reason, ok := session["cancel_reason"]; ok {+ t.Errorf("cancel_reason = %v, want none", reason)+ }+ }+ } else if got := nested(t, event, "session")["cancel_reason"]; got != tc.wantCancel {+ t.Errorf("session.cancel_reason = %v, want %q", got, tc.wantCancel)+ }+ if tc.wantErrType == "" {+ if _, ok := event["error"]; ok {+ t.Errorf("carried an error field: %v", event["error"])+ }+ } else if got := nested(t, event, "error")["type"]; got != tc.wantErrType {+ t.Errorf("error.type = %v, want %q", got, tc.wantErrType)+ }+ // Whatever the outcome, the tokens were spent and must be reported.+ if got := nested(t, event, "gen_ai", "usage")["input_tokens"]; got != float64(70) {+ t.Errorf("gen_ai.usage.input_tokens = %v, want 70", got)+ }+ })+ }+}diff --git a/docs/runtimes/cline.mdx b/docs/runtimes/cline.mdxindex 345af4bc..de667b02 100644--- a/docs/runtimes/cline.mdx+++ b/docs/runtimes/cline.mdx@@ -57,7 +57,7 @@ beacon endpoint hooks install --harness clineCline addresses files relative to the workspace root. Beacon resolves those paths against the workspace before writing them, so a Cline file event is comparable with the same file seen through another runtime and matches threat rules written against absolute paths. A path that is already absolute is left alone, and a relative path with no resolvable workspace root stays relative rather than being joined to a guessed root.-A cancelled task ends rather than fails. Cline fires its cancel hook for the cleanup that follows someone stopping or abandoning a task, so Beacon records `session.ended` at info severity with `session.cancel_reason` set, not `session.error` — otherwise every deliberate stop would count as an incident in error rollups and alerting. A cancelled task still reports the tokens it spent.+A cancelled task ends rather than fails. Cline fires its cancel hook for the cleanup that follows someone stopping or abandoning a task, so Beacon records `session.ended` at info severity with `session.cancel_reason` set, not `session.error` — otherwise every deliberate stop would count as an incident in error rollups and alerting. A cancelled task still reports the tokens it spent. Cancels and failures reach Beacon two ways: the file-based hooks name them in the hook itself, and the plugin reports them as a status on its single run-completion handler. Both are read, so the distinction holds on the surface Beacon actually installs; a run that reports no recognized outcome is recorded as a completion.Token usage is recorded at task end only. Beacon's token rollups sum `gen_ai.usage` across events, so reporting the same tokens at a per-model-call boundary as well would double every Cline task's totals. Cost is recorded only as Cline reports it; Beacon never derives cost from a local price table.diff --git a/docs/telemetry-schema/normalization.mdx b/docs/telemetry-schema/normalization.mdxindex cbecd054..e08b1246 100644--- a/docs/telemetry-schema/normalization.mdx+++ b/docs/telemetry-schema/normalization.mdx@@ -65,6 +65,14 @@ semantics, and to `tool.failed` when the payload reports an error; run end to`session.cancel_reason` set, since a cancel is an end rather than a failure; andtask errors to `session.error`.+Run end is read for its outcome rather than assumed to be a success. The plugin+surface has a single run-completion handler for every outcome and reports the+difference as a status on the context, so an aborted run normalizes to the cancel+shape and a failed one to `session.error`, exactly as the separately named+file-based hooks do. A run that states no outcome, or states one Beacon does not+recognize, stays a completion: mislabelling a finished task as failed is worse+than missing a label.+The same mapping accepts Cline's file-based hook names (`TaskStart`,`UserPromptSubmit`, `PreToolUse`, `PostToolUse`, `TaskComplete`, `TaskCancel`)alongside its plugin handler names, so both of Cline's hook surfaces normalize