Open-source AI coworkers that each get a computer of their own: a browser, files and tools, with every action decided before it happens and recorded after. Bring any AG-UI agent.
Say Did not happen about a call the audit page had been calling Allowed (#351)
kevin89bc040b5525
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 6149eb5a..63f6c6e6 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,6 +8,17 @@ Newest first. `Unreleased` is what is on `main` and not yet tagged.## Unreleased+### A tool call that failed no longer reads as one that worked++The audit page draws a row it does not recognise as `Allowed`, which is right for the many rows that+are neither a refusal nor a failure. Two rows that are failures were falling through to it: a+connector tool call this deployment permitted and the vendor did not complete, and a component's+data read that was granted and then broke. Both were drawn in the same muted colour as a call that+went through, and neither appeared under `Did not happen`, so the view an administrator opens to ask+what did not work here was short by exactly the rows they came for. A per-person connector fails on+this path every time somebody's token expires, so this was the most common failure the product has+and the one the trail was quietest about. Both now read as `Did not happen`, and both are in that+saved view. Neither is filed as a refusal: nothing was forbidden on either row.### A stray space in NODE_ENV no longer lets the public example key throughA deployment that never changed `KEY_ENCRYPTION_KEY` encrypts its credential vault with the keydiff --git a/app/src/lib/audit/outcome.ts b/app/src/lib/audit/outcome.tsindex be2af9e1..a114211c 100644--- a/app/src/lib/audit/outcome.ts+++ b/app/src/lib/audit/outcome.ts@@ -61,6 +61,27 @@ export const DID_NOT_HAPPEN_EVENT_TYPES = ["agent.handoff_failed",/** A question that reached nobody: the Bot stopped, and the person was never asked. */"agent.escalation_failed",+ /*+ * A tool call this deployment permitted and the vendor did not complete.+ *+ * The direct sibling of `computer.action_failed`, written by the same decide-record-then-act+ * order on the other acting surface: `callTool` writes the decision row first, attempts the+ * call, and then writes this instead of `mcp.call_succeeded` when the vendor answers `isError`+ * or the attempt throws. Every failure of a per-person connector lands here — no connection for+ * the asker, a refresh token the vendor stopped accepting, an API not enabled for the project —+ * which makes it the row somebody most often comes to this page to find, and it was the one row+ * of the family drawn in the same muted colour as a call that worked.+ */+ "mcp.call_failed",+ /*+ * A component's read that was granted and then broke.+ *+ * `server/src/audit.ts` says this one is "deliberately not a refusal", and that is right: nothing+ * was forbidden. But it is not an allowance either. The row exists precisely because the read did+ * not happen, and the page already labels it "Could not be read" while colouring it as though it+ * had been.+ */+ "component.function_failed",] as const;export type AuditOutcome = "refused" | "did-not-happen" | "allowed";diff --git a/app/tests/audit-outcome.test.ts b/app/tests/audit-outcome.test.tsindex 8aa1e591..9487655c 100644--- a/app/tests/audit-outcome.test.ts+++ b/app/tests/audit-outcome.test.ts@@ -53,6 +53,30 @@ describe("what the trail says a row was", () => {expect(outcomeOf("agent.stream_stalled")).toBe("did-not-happen");});+ /*+ * The two acting surfaces have to answer this the same way.+ *+ * A browser action that was permitted and then failed was already drawn as "Did not happen". The+ * same shape on the other two surfaces — a tool call the vendor did not complete, a component read+ * that broke — fell through to "Allowed", which is what the page says about a call that worked.+ */+ test("tells a permitted call that failed from one that went through", () => {+ // Written by `callTool` after the policy allowed the call and the vendor answered `isError` or+ // the attempt threw. A per-person connector fails here on every expired refresh token.+ expect(outcomeOf("mcp.call_failed")).toBe("did-not-happen");+ // Granted, attempted, and the read broke. Not a refusal, and not an allowance either.+ expect(outcomeOf("component.function_failed")).toBe("did-not-happen");+ // The surface that already got this right, kept here so the three cannot drift apart again.+ expect(outcomeOf("computer.action_failed")).toBe("did-not-happen");+ });++ test("a failed call is not filed as something this deployment refused", () => {+ // The other wrong answer. Nothing was forbidden on either row, and filing a broken call as a+ // policy event teaches a reader to distrust the policy events that are real.+ expect(outcomeOf("mcp.call_failed")).not.toBe("refused");+ expect(outcomeOf("component.function_failed")).not.toBe("refused");+ });+test("still calls something that went through allowed", () => {for (const eventType of ["computer.action_allowed",@@ -101,6 +125,20 @@ describe("the saved views ask the same question the rows do", () => {}});+ /*+ * The half of the failure that is harder to see. A row drawn in the wrong colour is at least on+ * the page; a row missing from this view is absent from the answer to "what did not happen here",+ * and the view is not empty, it is just short.+ */+ test("Did not happen names the calls that were permitted and failed", () => {+ const filtered = eventTypeFilter(DID_NOT_HAPPEN_EVENT_TYPES)+ .replace("?eventType=", "")+ .split(",");++ expect(filtered).toContain("mcp.call_failed");+ expect(filtered).toContain("component.function_failed");+ });+test("no event type is in both families", () => {const refused = new Set<string>(REFUSED_EVENT_TYPES);for (const eventType of DID_NOT_HAPPEN_EVENT_TYPES) {