Skip to content

armgit / eve

Public Imported from GitHub
0

The Open Framework for Building Agents

[2/2] test(eve): add conformance rules for typed approval authorization and the caller a tool sees (#4352)

Ben Pankow7ce9beb03f18
diff --git a/packages/eve/src/internal/testing/channel-conformance/MATRIX.md b/packages/eve/src/internal/testing/channel-conformance/MATRIX.md
index 06d3cff3b..f450babeb 100644
--- a/packages/eve/src/internal/testing/channel-conformance/MATRIX.md
+++ b/packages/eve/src/internal/testing/channel-conformance/MATRIX.md
@@ -44,18 +44,28 @@ pnpm --filter eve exec vitest run --config vitest.unit.config.ts channel-conform
| a message while an approval is pending cancels it, so typing approve afterwards runs nothing and the next message gets a reply | —<sup>[6](#note-6)</sup> | ✅ | ✅ | ✅ | ✅ | —<sup>[2](#note-2)</sup> | —<sup>[2](#note-2)</sup> | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| pressing Approve on an approval a message cancelled runs nothing | —<sup>[6](#note-6)</sup> | ✅ | ✅ | ✅ | —<sup>[1](#note-1)</sup> | —<sup>[2](#note-2)</sup> | —<sup>[2](#note-2)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> |
+## Approval permissions
+
+| Rule | `tui` | `web chat` | `chat-sdk` | `chat-sdk-dm` | `chat-sdk-text` | `discord` | `discord-dm` | `linq` | `linq-dm` | `slack` | `slack-dm` | `teams` | `teams-dm` | `telegram` | `telegram-dm` | `github` | `linear` | `photon` | `twilio` |
+| --- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+| the requester typing approve on a requester-only approval runs the tool | —<sup>[6](#note-6)</sup> | ✅ | ✅ | | ✅ | —<sup>[2](#note-2)</sup> | | ✅ | | ✅ | | ✅ | | ✅ | | ✅ | ✅ | ✅ | ✅ |
+| another person pressing Cancel or Approve on a requester-only approval leaves it pending | —<sup>[9](#note-9)</sup> | —<sup>[9](#note-9)</sup> | ✅ | | —<sup>[10](#note-10)</sup> | ✅ | | —<sup>[10](#note-10)</sup> | | ✅ | | ✅ | | ✅ | | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[10](#note-10)</sup> | —<sup>[10](#note-10)</sup> |
+| another person typing cancel or approve doesn't settle a requester-only approval | —<sup>[9](#note-9)</sup> | —<sup>[9](#note-9)</sup> | ✅ | | —<sup>[9](#note-9)</sup> | —<sup>[2](#note-2)</sup> | | —<sup>[9](#note-9)</sup> | | ✅ | | ✅ | | ✅ | | ✅ | ✅ | —<sup>[9](#note-9)</sup> | —<sup>[9](#note-9)</sup> |
+| another person's rejected press leaves the approval's buttons in place | —<sup>[9](#note-9)</sup> | —<sup>[9](#note-9)</sup> | ✅ | | —<sup>[10](#note-10)</sup> | ✅ | | —<sup>[10](#note-10)</sup> | | ✅ | | ✅ | | ✅ | | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[10](#note-10)</sup> | —<sup>[10](#note-10)</sup> |
+| another person pressing Approve on an open approval runs the tool | —<sup>[9](#note-9)</sup> | —<sup>[9](#note-9)</sup> | ✅ | | —<sup>[10](#note-10)</sup> | ✅ | | —<sup>[10](#note-10)</sup> | | ✅ | | ✅ | | ✅ | | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[10](#note-10)</sup> | —<sup>[10](#note-10)</sup> |
+
## Answered prompts
| Rule | `tui` | `web chat` | `chat-sdk` | `chat-sdk-dm` | `chat-sdk-text` | `discord` | `discord-dm` | `linq` | `linq-dm` | `slack` | `slack-dm` | `teams` | `teams-dm` | `telegram` | `telegram-dm` | `github` | `linear` | `photon` | `twilio` |
| --- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| pressing an option clears the question's buttons | ✅ | ✅ | ✅ | | —<sup>[1](#note-1)</sup> | ✅ | | —<sup>[1](#note-1)</sup> | | ✅ | | ✅ | | ✅ | | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> |
-| answering a question by text clears its buttons | ✅ | ✅ | ✅ | ✅ | —<sup>[1](#note-1)</sup> | —<sup>[2](#note-2)</sup> | —<sup>[2](#note-2)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | ❌<sup>[9](#note-9)</sup> | ❌<sup>[9](#note-9)</sup> | ✅ | ✅ | ✅ | ✅ | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> |
-| pressing an option names who answered on the question | —<sup>[10](#note-10)</sup> | —<sup>[11](#note-11)</sup> | ❌<sup>[12](#note-12)</sup> | | —<sup>[1](#note-1)</sup> | ❌<sup>[12](#note-12)</sup> | | —<sup>[1](#note-1)</sup> | | ✅ | | ❌<sup>[12](#note-12)</sup> | | ❌<sup>[12](#note-12)</sup> | | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> |
-| answering a question by text names who answered on the question | —<sup>[10](#note-10)</sup> | —<sup>[11](#note-11)</sup> | ❌<sup>[12](#note-12)</sup> | ❌<sup>[12](#note-12)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[2](#note-2)</sup> | —<sup>[2](#note-2)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | ❌<sup>[9](#note-9)</sup> | ❌<sup>[9](#note-9)</sup> | ❌<sup>[12](#note-12)</sup> | ❌<sup>[12](#note-12)</sup> | ❌<sup>[12](#note-12)</sup> | ❌<sup>[12](#note-12)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> |
+| answering a question by text clears its buttons | ✅ | ✅ | ✅ | ✅ | —<sup>[1](#note-1)</sup> | —<sup>[2](#note-2)</sup> | —<sup>[2](#note-2)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | ❌<sup>[11](#note-11)</sup> | ❌<sup>[11](#note-11)</sup> | ✅ | ✅ | ✅ | ✅ | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> |
+| pressing an option names who answered on the question | —<sup>[12](#note-12)</sup> | —<sup>[13](#note-13)</sup> | ❌<sup>[14](#note-14)</sup> | | —<sup>[1](#note-1)</sup> | ❌<sup>[14](#note-14)</sup> | | —<sup>[1](#note-1)</sup> | | ✅ | | ❌<sup>[14](#note-14)</sup> | | ❌<sup>[14](#note-14)</sup> | | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> |
+| answering a question by text names who answered on the question | —<sup>[12](#note-12)</sup> | —<sup>[13](#note-13)</sup> | ❌<sup>[14](#note-14)</sup> | ❌<sup>[14](#note-14)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[2](#note-2)</sup> | —<sup>[2](#note-2)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | ❌<sup>[11](#note-11)</sup> | ❌<sup>[11](#note-11)</sup> | ❌<sup>[14](#note-14)</sup> | ❌<sup>[14](#note-14)</sup> | ❌<sup>[14](#note-14)</sup> | ❌<sup>[14](#note-14)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> |
| pressing Approve clears the approval's buttons | ✅ | ✅ | ✅ | | —<sup>[1](#note-1)</sup> | ✅ | | —<sup>[1](#note-1)</sup> | | ✅ | | ✅ | | ✅ | | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> |
| approving by text clears the approval's buttons | —<sup>[6](#note-6)</sup> | ✅ | ✅ | ✅ | —<sup>[1](#note-1)</sup> | —<sup>[2](#note-2)</sup> | —<sup>[2](#note-2)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> |
-| pressing Approve names who approved on the approval | —<sup>[10](#note-10)</sup> | —<sup>[11](#note-11)</sup> | ❌<sup>[12](#note-12)</sup> | | —<sup>[1](#note-1)</sup> | ❌<sup>[12](#note-12)</sup> | | —<sup>[1](#note-1)</sup> | | ✅ | | ✅ | | ❌<sup>[12](#note-12)</sup> | | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> |
-| approving by text names who approved on the approval | —<sup>[6](#note-6)</sup> | —<sup>[11](#note-11)</sup> | ❌<sup>[12](#note-12)</sup> | ❌<sup>[12](#note-12)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[2](#note-2)</sup> | —<sup>[2](#note-2)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | ✅ | ✅ | ❌<sup>[12](#note-12)</sup> | ❌<sup>[12](#note-12)</sup> | ❌<sup>[12](#note-12)</sup> | ❌<sup>[12](#note-12)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> |
+| pressing Approve names who approved on the approval | —<sup>[12](#note-12)</sup> | —<sup>[13](#note-13)</sup> | ❌<sup>[14](#note-14)</sup> | | —<sup>[1](#note-1)</sup> | ❌<sup>[14](#note-14)</sup> | | —<sup>[1](#note-1)</sup> | | ✅ | | ✅ | | ❌<sup>[14](#note-14)</sup> | | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> |
+| approving by text names who approved on the approval | —<sup>[6](#note-6)</sup> | —<sup>[13](#note-13)</sup> | ❌<sup>[14](#note-14)</sup> | ❌<sup>[14](#note-14)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[2](#note-2)</sup> | —<sup>[2](#note-2)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | ✅ | ✅ | ❌<sup>[14](#note-14)</sup> | ❌<sup>[14](#note-14)</sup> | ❌<sup>[14](#note-14)</sup> | ❌<sup>[14](#note-14)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> |
## Budget prompts
@@ -72,19 +82,21 @@ pnpm --filter eve exec vitest run --config vitest.unit.config.ts channel-conform
| Rule | `tui` | `web chat` | `chat-sdk` | `chat-sdk-dm` | `chat-sdk-text` | `discord` | `discord-dm` | `linq` | `linq-dm` | `slack` | `slack-dm` | `teams` | `teams-dm` | `telegram` | `telegram-dm` | `github` | `linear` | `photon` | `twilio` |
| --- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
-| a sign-in names the service and shows its sign-in link | ✅ | ✅ | ✅ | ✅ | ✅ | ❌<sup>[13](#note-13)</sup> | ❌<sup>[13](#note-13)</sup> | ❌<sup>[14](#note-14)</sup> | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | —<sup>[15](#note-15)</sup> | ✅ | ✅ | ✅ |
-| a sign-in shows its confirmation code | ✅ | ✅ | ✅ | ✅ | ✅ | ❌<sup>[13](#note-13)</sup> | ❌<sup>[13](#note-13)</sup> | ❌<sup>[14](#note-14)</sup> | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | —<sup>[15](#note-15)</sup> | ✅ | ✅ | ✅ |
-| only the person signing in sees the sign-in link and code | —<sup>[16](#note-16)</sup> | —<sup>[16](#note-16)</sup> | ✅ | —<sup>[16](#note-16)</sup> | —<sup>[16](#note-16)</sup> | ✅ | —<sup>[16](#note-16)</sup> | ✅ | —<sup>[16](#note-16)</sup> | ✅ | —<sup>[16](#note-16)</sup> | ❌<sup>[17](#note-17)</sup> | —<sup>[16](#note-16)</sup> | ✅ | —<sup>[16](#note-16)</sup> | ✅ | ❌<sup>[18](#note-18)</sup> | —<sup>[16](#note-16)</sup> | —<sup>[16](#note-16)</sup> |
-| a sign-in without a link shows its instructions | ✅ | ✅ | ✅ | ✅ | ✅ | ❌<sup>[13](#note-13)</sup> | ❌<sup>[13](#note-13)</sup> | ❌<sup>[14](#note-14)</sup> | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌<sup>[13](#note-13)</sup> | ✅ | ✅ | ✅ |
+| a sign-in names the service and shows its sign-in link | ✅ | ✅ | ✅ | ✅ | ✅ | ❌<sup>[15](#note-15)</sup> | ❌<sup>[15](#note-15)</sup> | ❌<sup>[16](#note-16)</sup> | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | —<sup>[17](#note-17)</sup> | ✅ | ✅ | ✅ |
+| a sign-in shows its confirmation code | ✅ | ✅ | ✅ | ✅ | ✅ | ❌<sup>[15](#note-15)</sup> | ❌<sup>[15](#note-15)</sup> | ❌<sup>[16](#note-16)</sup> | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | —<sup>[17](#note-17)</sup> | ✅ | ✅ | ✅ |
+| only the person signing in sees the sign-in link and code | —<sup>[18](#note-18)</sup> | —<sup>[18](#note-18)</sup> | ✅ | —<sup>[18](#note-18)</sup> | —<sup>[18](#note-18)</sup> | ✅ | —<sup>[18](#note-18)</sup> | ✅ | —<sup>[18](#note-18)</sup> | ✅ | —<sup>[18](#note-18)</sup> | ❌<sup>[19](#note-19)</sup> | —<sup>[18](#note-18)</sup> | ✅ | —<sup>[18](#note-18)</sup> | ✅ | ❌<sup>[20](#note-20)</sup> | —<sup>[18](#note-18)</sup> | —<sup>[18](#note-18)</sup> |
+| a sign-in without a link shows its instructions | ✅ | ✅ | ✅ | ✅ | ✅ | ❌<sup>[15](#note-15)</sup> | ❌<sup>[15](#note-15)</sup> | ❌<sup>[16](#note-16)</sup> | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌<sup>[15](#note-15)</sup> | ✅ | ✅ | ✅ |
| after signing in, the agent carries on with the request | ✅ | ✅ | ✅ | | ✅ | ✅ | | ✅ | | ✅ | | ✅ | | ✅ | | ✅ | ✅ | ✅ | ✅ |
-| completing a sign-in tells the person it succeeded | ✅ | ✅ | ✅ | ✅ | ✅ | ❌<sup>[13](#note-13)</sup> | ❌<sup>[13](#note-13)</sup> | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌<sup>[13](#note-13)</sup> | ✅ | ✅ | ✅ |
+| completing a sign-in tells the person it succeeded | ✅ | ✅ | ✅ | ✅ | ✅ | ❌<sup>[15](#note-15)</sup> | ❌<sup>[15](#note-15)</sup> | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌<sup>[15](#note-15)</sup> | ✅ | ✅ | ✅ |
| message after ignored sign-in gets an answer, signing in late doesn't run | ✅ | ✅ | ✅ | ✅ | ✅ | —<sup>[2](#note-2)</sup> | —<sup>[2](#note-2)</sup> | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
-| message after ignored sign-in tells user it was cancelled | ✅ | ✅ | ✅ | ✅ | ✅ | —<sup>[2](#note-2)</sup> | —<sup>[2](#note-2)</sup> | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌<sup>[13](#note-13)</sup> | ✅ | ✅ | ✅ |
-| the requester pressing Approve on a requester-only approval runs the tool | ✅ | ✅ | ✅ | | —<sup>[1](#note-1)</sup> | ✅ | | —<sup>[1](#note-1)</sup> | | ✅ | | ✅ | | ✅ | | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> | —<sup>[1](#note-1)</sup> |
-| another person pressing Approve on a requester-only approval leaves it pending | —<sup>[19](#note-19)</sup> | —<sup>[19](#note-19)</sup> | ✅ | | —<sup>[20](#note-20)</sup> | ✅ | | —<sup>[20](#note-20)</sup> | | ✅ | | ✅ | | ✅ | | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> |
-| another person pressing Cancel on a requester-only approval leaves it pending | —<sup>[19](#note-19)</sup> | —<sup>[19](#note-19)</sup> | ✅ | | —<sup>[20](#note-20)</sup> | ✅ | | —<sup>[20](#note-20)</sup> | | ✅ | | ✅ | | ✅ | | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> |
-| another person's rejected press leaves the approval's buttons in place | —<sup>[19](#note-19)</sup> | —<sup>[19](#note-19)</sup> | ✅ | | —<sup>[20](#note-20)</sup> | ✅ | | —<sup>[20](#note-20)</sup> | | ✅ | | ✅ | | ✅ | | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> |
-| another person pressing Approve on an open approval runs the tool | —<sup>[19](#note-19)</sup> | —<sup>[19](#note-19)</sup> | ✅ | | —<sup>[20](#note-20)</sup> | ✅ | | —<sup>[20](#note-20)</sup> | | ✅ | | ✅ | | ✅ | | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> |
+| message after ignored sign-in tells user it was cancelled | ✅ | ✅ | ✅ | ✅ | ✅ | —<sup>[2](#note-2)</sup> | —<sup>[2](#note-2)</sup> | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌<sup>[15](#note-15)</sup> | ✅ | ✅ | ✅ |
+
+## Tool callers
+
+| Rule | `tui` | `web chat` | `chat-sdk` | `chat-sdk-dm` | `chat-sdk-text` | `discord` | `discord-dm` | `linq` | `linq-dm` | `slack` | `slack-dm` | `teams` | `teams-dm` | `telegram` | `telegram-dm` | `github` | `linear` | `photon` | `twilio` |
+| --- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+| a tool sees the person who sent the message as its caller | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| another person's message reaches tools as a different caller, and each person stays the same caller | —<sup>[9](#note-9)</sup> | —<sup>[9](#note-9)</sup> | ✅ | | —<sup>[9](#note-9)</sup> | ✅ | | —<sup>[9](#note-9)</sup> | | ✅ | | ✅ | | ✅ | | ✅ | ✅ | —<sup>[9](#note-9)</sup> | —<sup>[9](#note-9)</sup> |
## Attachments
@@ -105,17 +117,17 @@ pnpm --filter eve exec vitest run --config vitest.unit.config.ts channel-conform
6. <a id="note-6"></a>the approval drawer holds the keyboard; a person answers it with y or n
7. <a id="note-7"></a>a reply sent while approvals wait goes out as a steering message, which sometimes restarts the turn instead of answering
8. <a id="note-8"></a>one card shows every approval a step raises, so a typed reply can't say which it answers
-9. <a id="note-9"></a>only the button interaction handler edits a question; a typed answer leaves it
-10. <a id="note-10"></a>one person answers at their own terminal; there's nobody else to tell
-11. <a id="note-11"></a>one person answers in their own browser tab; there's nobody else to tell
-12. <a id="note-12"></a>a resolved prompt doesn't say who answered; input.resolved carries no responder
-13. <a id="note-13"></a>the channel has no default sign-in renderer
-14. <a id="note-14"></a>Linq's openDM needs the person's phone handle, but a message names its sender by an opaque handle id, so the bot can only say to continue in a direct message
-15. <a id="note-15"></a>the rule applies only where the conversation is shared or private, and this one is public
-16. <a id="note-16"></a>the rule applies only where the conversation is public or shared, and this one is private
-17. <a id="note-17"></a>the sign-in prompt, link included, is posted to the whole thread
-18. <a id="note-18"></a>the code is in the elicitation body the whole issue sees; who sees the auth signal's link is unverified
-19. <a id="note-19"></a>the platform has no second person who can act
-20. <a id="note-20"></a>the platform has no second person who can act or buttons a person can press
+9. <a id="note-9"></a>the platform has no second person who can act
+10. <a id="note-10"></a>the platform has no second person who can act or buttons a person can press
+11. <a id="note-11"></a>only the button interaction handler edits a question; a typed answer leaves it
+12. <a id="note-12"></a>one person answers at their own terminal; there's nobody else to tell
+13. <a id="note-13"></a>one person answers in their own browser tab; there's nobody else to tell
+14. <a id="note-14"></a>a resolved prompt doesn't say who answered; input.resolved carries no responder
+15. <a id="note-15"></a>the channel has no default sign-in renderer
+16. <a id="note-16"></a>Linq's openDM needs the person's phone handle, but a message names its sender by an opaque handle id, so the bot can only say to continue in a direct message
+17. <a id="note-17"></a>the rule applies only where the conversation is shared or private, and this one is public
+18. <a id="note-18"></a>the rule applies only where the conversation is public or shared, and this one is private
+19. <a id="note-19"></a>the sign-in prompt, link included, is posted to the whole thread
+20. <a id="note-20"></a>the code is in the elicitation body the whole issue sees; who sees the auth signal's link is unverified
21. <a id="note-21"></a>the platform has no files a person can send
22. <a id="note-22"></a>each slash command starts its own session, so no later message shares one with the file
diff --git a/packages/eve/src/internal/testing/channel-conformance/chat-sdk-driver.ts b/packages/eve/src/internal/testing/channel-conformance/chat-sdk-driver.ts
index 59e787371..bcfc2dfc9 100644
--- a/packages/eve/src/internal/testing/channel-conformance/chat-sdk-driver.ts
+++ b/packages/eve/src/internal/testing/channel-conformance/chat-sdk-driver.ts
@@ -52,6 +52,7 @@ interface InboundFile extends Pick<SentFile, "mediaType" | "name"> {
type Inbound =
| {
readonly kind: "message";
+ readonly person: Person;
readonly text: string;
/** Files on the message, as the platform lists them: a URL to each, not its bytes. */
readonly files?: readonly InboundFile[];
@@ -195,6 +196,7 @@ function chatSdkDriverWith(input: {
return {
name: input.name,
+ personId: PERSON.userId,
inbound,
createChannel(record) {
const dm = input.surface === "private";
@@ -244,7 +246,7 @@ function chatSdkDriverWith(input: {
}
return bridge.channel;
},
- message: (text, files = []) =>
+ message: (text, person, files = []) =>
inbound({
files: files.map((file) => {
const url = `https://files.conformance.example/${uploads.size + 1}/${encodeURIComponent(file.name)}`;
@@ -252,6 +254,7 @@ function chatSdkDriverWith(input: {
return { mediaType: file.mediaType, name: file.name, url };
}),
kind: "message",
+ person,
text,
}),
postedText(call: PlatformCall) {
@@ -336,7 +339,7 @@ function fakeAdapter({
adapter,
threadId,
// A person mentions the bot to start a channel thread; Chat routes the rest by subscription.
- inboundMessage(threadId, id, body.text, !dm, body.files, fetchData),
+ inboundMessage(threadId, id, body.text, !dm, PEOPLE[body.person], body.files, fetchData),
options,
);
}
@@ -395,6 +398,7 @@ function inboundMessage(
id: string,
text: string,
isMention = false,
+ author: (typeof PEOPLE)[Person] = PERSON,
files: readonly InboundFile[] = [],
fetchData?: (url: string) => Promise<Buffer>,
): Message {
@@ -407,7 +411,7 @@ function inboundMessage(
// Private to the platform, as Slack's or Teams' are: only `fetchData` can download it.
url: file.url,
})),
- author: PERSON,
+ author,
formatted: parseMarkdown(text),
id,
isMention,
diff --git a/packages/eve/src/internal/testing/channel-conformance/conformance.ts b/packages/eve/src/internal/testing/channel-conformance/conformance.ts
index 19532ec88..7ebec54ed 100644
--- a/packages/eve/src/internal/testing/channel-conformance/conformance.ts
+++ b/packages/eve/src/internal/testing/channel-conformance/conformance.ts
@@ -273,6 +273,8 @@ const channelConformance = {
"a message while an approval is pending cancels it, so typing approve afterwards runs nothing and the next message gets a reply":
TUI_TYPED_APPROVAL,
"pressing Approve on an approval a message cancelled runs nothing": TUI_TYPED_APPROVAL,
+ "the requester typing approve on a requester-only approval runs the tool":
+ TUI_TYPED_APPROVAL,
"pressing Approve names who approved on the approval": TUI_SINGLE_PERSON,
"pressing an option names who answered on the question": TUI_SINGLE_PERSON,
"answering a question by text names who answered on the question": TUI_SINGLE_PERSON,
diff --git a/packages/eve/src/internal/testing/channel-conformance/contract.ts b/packages/eve/src/internal/testing/channel-conformance/contract.ts
index 1c003dab9..faebd3482 100644
--- a/packages/eve/src/internal/testing/channel-conformance/contract.ts
+++ b/packages/eve/src/internal/testing/channel-conformance/contract.ts
@@ -9,6 +9,7 @@ import {
GATED_TOOL,
type GatedTool,
OPEN_GATED_TOOL,
+ type Person,
PLAIN_TOOL,
type RenderedOption,
REQUESTER_GATED_TOOL,
@@ -105,6 +106,21 @@ function option(
const ASIDE = "Alice also wants the changelog summarized.";
const FOLLOW_UP = "Alice asks what is still left to do.";
+/** Each person in turn asks for {@link PLAIN_TOOL}; returns the caller each run saw. */
+async function lookUpNotesAs(
+ conversation: ChannelConversation,
+ people: readonly Person[],
+): Promise<readonly (string | null)[]> {
+ for (const [index, person] of people.entries()) {
+ // Numbered, so each run's reply, which quotes its message, can be told apart.
+ const request = `request ${index + 1}`;
+ await conversation.say(`Use ${PLAIN_TOOL} to find the review notes, ${request}.`, person);
+ await conversation.waitForShown(new RegExp(`Used ${PLAIN_TOOL} for .*${request}`, "su"));
+ }
+ expect(conversation.runsOf(PLAIN_TOOL), `${PLAIN_TOOL} runs`).toBe(people.length);
+ return conversation.callersOf(PLAIN_TOOL);
+}
+
const HOTFIX = `Use ${REQUESTER_GATED_TOOL} to ship the fix.`;
const HOTFIX_PROMPT = "Approve Release hotfix?";
@@ -138,12 +154,12 @@ async function expectReleased(conversation: ChannelConversation) {
await expectRan(conversation, REQUESTER_GATED_TOOL, { released: true });
}
-/** Once the session settles after Bob's press, the requester-only approval is still pending. */
+/** Once the session settles after Bob answers, the requester-only approval is still pending. */
async function expectStillPending(conversation: ChannelConversation) {
await conversation.waitForRest();
expect(
conversation.runsOf(REQUESTER_GATED_TOOL),
- `${REQUESTER_GATED_TOOL} ran on Bob's press`,
+ `${REQUESTER_GATED_TOOL} ran on Bob's answer`,
).toBe(0);
}
@@ -268,7 +284,7 @@ async function attachmentsSeen(
conversation: ChannelConversation,
files?: readonly SentFile[],
): Promise<readonly SeenFile[]> {
- await conversation.say(LIST_ATTACHMENTS, files);
+ await conversation.say(LIST_ATTACHMENTS, "alice", files);
const shown = await conversation.waitForShown(ATTACHMENTS_REPLY);
return JSON.parse(ATTACHMENTS_REPLY.exec(shown)![1]!) as SeenFile[];
}
@@ -927,39 +943,47 @@ const signInRules = [
await conversation.waitForShown(/\b(?:cancel|declin)/iu);
},
},
+] as const satisfies readonly ContractRule[];
+
+const approvalPermissionRules = [
{
- rule: "the requester pressing Approve on a requester-only approval runs the tool",
+ rule: "the requester typing approve on a requester-only approval runs the tool",
source: "docs/tools/human-in-the-loop.md#authorizing-approval-responses",
- requires: ["buttons"],
+ requires: ["text-replies"],
async run(conversation) {
- const options = await askToReleaseHotfix(conversation);
- await conversation.press(option(options, APPROVE_LABELS));
+ await askToReleaseHotfix(conversation);
+ await conversation.say("approve");
await expectReleased(conversation);
},
},
{
- rule: "another person pressing Approve on a requester-only approval leaves it pending",
+ rule: "another person pressing Cancel or Approve on a requester-only approval leaves it pending",
source: "docs/tools/human-in-the-loop.md#authorizing-approval-responses",
requires: ["another-person", "buttons"],
async run(conversation) {
const options = await askToReleaseHotfix(conversation);
const approve = option(options, APPROVE_LABELS);
+ await conversation.press(option(options, CANCEL_LABELS), "bob");
await conversation.press(approve, "bob");
await expectStillPending(conversation);
+ // A cancel that got through would settle the call as denied before this approval.
await conversation.press(approve);
await expectReleased(conversation);
},
},
{
- rule: "another person pressing Cancel on a requester-only approval leaves it pending",
- source: "docs/tools/human-in-the-loop.md#authorizing-approval-responses",
- requires: ["another-person", "buttons"],
+ rule: "another person typing cancel or approve doesn't settle a requester-only approval",
+ source: "docs/tools/human-in-the-loop.md#how-pause-and-resume-works",
+ requires: ["another-person", "text-replies"],
async run(conversation) {
- const options = await askToReleaseHotfix(conversation);
- await conversation.press(option(options, CANCEL_LABELS), "bob");
+ await askToReleaseHotfix(conversation);
+ // Today another person's message waits for the turn to end, so it never reaches the
+ // response policy; the approval stays pending either way.
+ await conversation.say("cancel", "bob");
+ await conversation.say("approve", "bob");
await expectStillPending(conversation);
- // A cancel that got through would settle the call as denied before this approval.
- await conversation.press(option(options, APPROVE_LABELS));
+ // A cancel taken as Alice's would settle the call as denied before this approval.
+ await conversation.say("approve");
await expectReleased(conversation);
},
},
@@ -986,6 +1010,37 @@ const signInRules = [
},
] as const satisfies readonly ContractRule[];
+const callerRules = [
+ {
+ rule: "a tool sees the person who sent the message as its caller",
+ source: "docs/tools/overview.mdx",
+ requires: [],
+ // Platforms name the sender differently in a DM, e.g. Discord's user rather than member.
+ variesByConversation: true,
+ async run(conversation) {
+ const [caller] = await lookUpNotesAs(conversation, ["alice"]);
+ expect(caller, `${PLAIN_TOOL} ran with no caller`).not.toBeNull();
+ const [principalId] = JSON.parse(caller!) as [string];
+ // Platform principals are namespaced, e.g. `slack:T01:U_ALICE`; a client's is the bare id.
+ expect(
+ principalId === conversation.personId || principalId.endsWith(`:${conversation.personId}`),
+ `${PLAIN_TOOL} ran as ${principalId}, which doesn't name ${conversation.personId}`,
+ ).toBe(true);
+ },
+ },
+ {
+ rule: "another person's message reaches tools as a different caller, and each person stays the same caller",
+ source: "docs/tools/overview.mdx",
+ requires: ["another-person"],
+ async run(conversation) {
+ const [alice, bob, aliceAgain] = await lookUpNotesAs(conversation, ["alice", "bob", "alice"]);
+ expect(bob, `${PLAIN_TOOL} ran with no caller for Bob`).not.toBeNull();
+ expect(bob, "Bob's message ran as Alice").not.toBe(alice);
+ expect(aliceAgain, "Alice ran as a different caller the second time").toBe(alice);
+ },
+ },
+] as const satisfies readonly ContractRule[];
+
const attachmentRules = [
{
rule: "an image a person sends reaches the agent with its bytes and type",
@@ -1025,7 +1080,7 @@ const attachmentRules = [
requires: ["attachments"],
async run(conversation) {
const text = `Alice attached ${DIAGRAM.name}.`;
- await conversation.say(text, [DIAGRAM]);
+ await conversation.say(text, "alice", [DIAGRAM]);
await conversation.waitForReplyTo(text);
await conversation.say(FOLLOW_UP);
await conversation.waitForReplyTo(FOLLOW_UP);
@@ -1042,9 +1097,11 @@ const attachmentRules = [
export const channelContractSections = [
{ title: "Questions", rules: questionRules },
{ title: "Tool approvals", rules: approvalRules },
+ { title: "Approval permissions", rules: approvalPermissionRules },
{ title: "Answered prompts", rules: answeredPromptRules },
{ title: "Budget prompts", rules: budgetRules },
{ title: "Sign-ins", rules: signInRules },
+ { title: "Tool callers", rules: callerRules },
{ title: "Attachments", rules: attachmentRules },
] as const;
diff --git a/packages/eve/src/internal/testing/channel-conformance/discord-driver.ts b/packages/eve/src/internal/testing/channel-conformance/discord-driver.ts
index 9d600a403..838c1ea92 100644
--- a/packages/eve/src/internal/testing/channel-conformance/discord-driver.ts
+++ b/packages/eve/src/internal/testing/channel-conformance/discord-driver.ts
@@ -126,6 +126,7 @@ export function discordDriver(surface: Exclude<Surface, "public"> = "shared"): C
return {
name: dm ? "discord-dm" : "discord",
+ personId: PERSON.id,
capabilities: dm ? ["attachments", "buttons"] : ["attachments", "another-person", "buttons"],
surface,
createChannel: (record) =>
@@ -133,7 +134,7 @@ export function discordDriver(surface: Exclude<Surface, "public"> = "shared"): C
api: { fetch: recordingFetch(record, decode) },
credentials: { applicationId: "APP1", botToken: "bot-token", publicKey: publicKeyHex },
}),
- message: (text, files = []) => {
+ message: (text, person, files = []) => {
const id = nextInteraction();
// Each file rides an attachment option (type 11) whose value names it in `resolved`.
const attached = files.map((file, index) => {
@@ -158,7 +159,7 @@ export function discordDriver(surface: Exclude<Surface, "public"> = "shared"): C
});
return signed(
JSON.stringify({
- ...where(),
+ ...where(PEOPLE[person]),
application_id: "APP1",
channel_id: channelId,
data: {
diff --git a/packages/eve/src/internal/testing/channel-conformance/github-driver.ts b/packages/eve/src/internal/testing/channel-conformance/github-driver.ts
index a1da551be..3b5236417 100644
--- a/packages/eve/src/internal/testing/channel-conformance/github-driver.ts
+++ b/packages/eve/src/internal/testing/channel-conformance/github-driver.ts
@@ -3,6 +3,7 @@ import { signGitHubWebhookBody } from "#public/channels/github/verify.js";
import {
type ChannelDriver,
type PlatformCall,
+ type Person,
type RenderedOption,
recordingFetch,
} from "#internal/testing/channel-conformance/harness.js";
@@ -10,6 +11,11 @@ import {
const SECRET = "github-conformance-secret";
let nextConversation = 0;
+const PEOPLE = {
+ alice: { id: 583231, login: "octocat", type: "User" },
+ bob: { id: 2, login: "hubot-bob", type: "User" },
+} as const;
+
/** Drives GitHub issue comments through the signed App webhook route. */
export function githubDriver(): ChannelDriver {
// Repository and issue identities are the continuation key, so isolate each driver instance.
@@ -19,7 +25,8 @@ export function githubDriver(): ChannelDriver {
let delivery = 0;
let commentId = 0;
- function message(text: string): Request {
+ function message(text: string, person: Person): Request {
+ const user = PEOPLE[person];
delivery += 1;
commentId += 1;
const payload = {
@@ -27,7 +34,7 @@ export function githubDriver(): ChannelDriver {
comment: {
body: `@testbot ${text}`,
id: commentId,
- user: { id: 1, login: "octocat", type: "User" },
+ user,
},
installation: { id: 55 },
issue: { number: issueNumber },
@@ -37,7 +44,7 @@ export function githubDriver(): ChannelDriver {
name: "eve",
owner: { login: "vercel" },
},
- sender: { id: 1, login: "octocat", type: "User" },
+ sender: user,
};
const body = JSON.stringify(payload);
return new Request("https://agent.example.com/eve/v1/github", {
@@ -63,7 +70,8 @@ export function githubDriver(): ChannelDriver {
return {
name: "github",
- capabilities: ["text-replies"],
+ personId: String(PEOPLE.alice.id),
+ capabilities: ["another-person", "text-replies"],
surface: "public",
createChannel: (record) =>
githubChannel({
diff --git a/packages/eve/src/internal/testing/channel-conformance/harness.ts b/packages/eve/src/internal/testing/channel-conformance/harness.ts
index 8dd79af6c..e7980339d 100644
--- a/packages/eve/src/internal/testing/channel-conformance/harness.ts
+++ b/packages/eve/src/internal/testing/channel-conformance/harness.ts
@@ -125,6 +125,8 @@ export type Surface =
*/
export interface ChannelDriver {
readonly name: string;
+ /** The platform's id for Alice, which the principal eve derives for her must name. */
+ readonly personId: string;
readonly capabilities: readonly ChannelCapability[];
readonly surface: Surface;
/**
@@ -134,8 +136,12 @@ export interface ChannelDriver {
createChannel(record: (call: PlatformCall) => void): unknown;
/** Undoes anything `createChannel` installed outside the channel, such as a global `fetch`. */
dispose?(): void;
- /** A webhook request carrying a person's message. */
- message(text: string, files?: readonly SentFile[]): Request;
+ /**
+ * A webhook request carrying `person`'s message, with `files` attached if
+ * given. Drivers with the `another-person` capability must send as `"bob"`
+ * when asked.
+ */
+ message(text: string, person: Person, files?: readonly SentFile[]): Request;
/**
* The options a person can see in one outbound call that posts the question:
* `undefined` when the call isn't the question, `[]` when it shows no options.
@@ -188,8 +194,8 @@ export interface SentFile {
/** What a person can do and see in one channel conversation. Contract rules use only this. */
export interface ChannelConversation {
- /** The person sends a plain-text message, with `files` attached if given. */
- say(text: string, files?: readonly SentFile[]): Promise<void>;
+ /** `person`, Alice unless given, sends a plain-text message, with `files` attached if given. */
+ say(text: string, person?: Person, files?: readonly SentFile[]): Promise<void>;
/** Waits for the bot to post `prompt` with choices, returning them. */
waitForQuestion(prompt: string): Promise<readonly RenderedOption[]>;
/**
@@ -243,7 +249,15 @@ export interface ChannelConversation {
shownPrompt(prompt: string): Promise<ShownMessage>;
/** How the person appears in the platform's text, in any form. */
readonly personShownAs: readonly string[];
+ /** Alice's id on the platform, or as the client authenticates her. */
+ readonly personId: string;
runsOf(tool: CountedTool): number;
+ /**
+ * The caller each run of `tool` saw, in order, as one string per principal,
+ * or `null` for no caller. Equal strings are the same principal. Each is JSON
+ * that begins with the principal id.
+ */
+ callersOf(tool: typeof PLAIN_TOOL): readonly (string | null)[];
}
/**
@@ -264,8 +278,8 @@ export interface ClientDriver {
/** A running client, as a person sees and uses it. */
export interface ClientView {
- /** The person sends a plain-text message, with `files` attached if given. */
- say(text: string, files?: readonly SentFile[]): Promise<void>;
+ /** `person` sends a plain-text message, with `files` attached if given. */
+ say(text: string, person: Person, files?: readonly SentFile[]): Promise<void>;
/**
* Waits for the client to show one of `prompts`, returning which one and its
* choices. A client may show several pending requests one at a time.
@@ -279,6 +293,8 @@ export interface ClientView {
shownPrompt?(prompt: string): ShownMessage | Promise<ShownMessage>;
/** How the person appears in the client's text, in any form. */
readonly personShownAs?: readonly string[];
+ /** Alice's id on the platform; a client authenticates her as {@link CLIENT_PERSON}. */
+ readonly personId?: string;
/** See {@link ChannelDriver.nextAddress}. */
nextAddress?(): string | undefined;
/** Everything the client shows now that a person can read or open, one entry per message. */
@@ -493,7 +509,7 @@ function webhookView(
const promptsFrom = new Map<string, number>();
return {
- say: (text, files) => post(driver.message(text, files)),
+ say: (text, person, files) => post(driver.message(text, person, files)),
press: (option, person) => post(driver.press(option, person)),
waitForQuestion: (prompts) =>
wait(
@@ -532,6 +548,7 @@ function webhookView(
.at(-1)!;
},
personShownAs: driver.personShownAs ?? [],
+ personId: driver.personId,
nextAddress: driver.nextAddress?.bind(driver),
shown: () =>
calls.flatMap((call) => {
@@ -590,6 +607,7 @@ async function converse(
): Promise<void> {
if (!isCompiledChannel(created)) throw new Error(`${label} is not a compiled channel.`);
const channel: CompiledChannel = created;
+ const callers: (string | null)[] = [];
const runs: Record<CountedTool, number> = {
[GATED_TOOL]: 0,
[REQUESTER_GATED_TOOL]: 0,
@@ -643,8 +661,9 @@ async function converse(
loadNamespace: async () => ({
default: defineTool({
description: `Looks up meeting notes. Only call when asked to use ${PLAIN_TOOL}.`,
- execute: async () => {
+ execute: async (_input, ctx) => {
runs[PLAIN_TOOL] += 1;
+ callers.push(callerKey(ctx.session.auth.current));
return { notes: "Bob's review notes" };
},
inputSchema: z.object({}),
@@ -808,12 +827,12 @@ async function converse(
let signInsCompleted = 0;
const conversation: ChannelConversation = {
- async say(text, files) {
+ async say(text, person = "alice", files) {
await waitForStepsToFinish([...sessions.values()], wait);
// A step can be done before its aliases are claimed; see waitForAddress.
const address = view.nextAddress?.();
if (address !== undefined) await waitForAddress(address);
- await view.say(text, files);
+ await view.say(text, person, files);
},
press: (option, person = "alice") => view.press(option, person),
async waitForQuestion(prompt) {
@@ -904,12 +923,14 @@ async function converse(
await deliverSignInCallback(callbackUrl);
},
runsOf: (tool) => runs[tool],
+ callersOf: () => callers,
waitForRest: () => waitForRest([...sessions.values()], wait),
async shownPrompt(prompt) {
if (view.shownPrompt === undefined) throw new Error(`${label} cannot read shown messages.`);
return await view.shownPrompt(prompt);
},
personShownAs: view.personShownAs ?? [],
+ personId: view.personId ?? CLIENT_PERSON.principalId,
};
/**
@@ -1170,6 +1191,12 @@ function gatedTool(
};
}
+function callerKey(auth: SessionAuthContext | null): string | null {
+ if (auth === null) return null;
+ // The id first, so a truncated assertion message still shows who it was.
+ return JSON.stringify([auth.principalId, auth.principalType, auth.authenticator, auth.issuer]);
+}
+
function samePrincipal(a: SessionAuthContext, b: SessionAuthContext): boolean {
return (
a.authenticator === b.authenticator &&
diff --git a/packages/eve/src/internal/testing/channel-conformance/linear-driver.ts b/packages/eve/src/internal/testing/channel-conformance/linear-driver.ts
index 84ae2963a..d5477d475 100644
--- a/packages/eve/src/internal/testing/channel-conformance/linear-driver.ts
+++ b/packages/eve/src/internal/testing/channel-conformance/linear-driver.ts
@@ -13,6 +13,9 @@ import { signLinearWebhookBody } from "#public/channels/linear/verify.js";
const SECRET = "linear-conformance-secret";
let nextConversation = 0;
+// Anyone in the workspace can reply in an issue's agent session.
+const PEOPLE = { alice: "user_1", bob: "user_2" } as const;
+
/** Drives Linear Agent Session webhooks and Agent Activity GraphQL mutations. */
export function linearDriver(): ChannelDriver {
nextConversation += 1;
@@ -84,7 +87,8 @@ export function linearDriver(): ChannelDriver {
return {
name: "linear",
- capabilities: ["attachments", "text-replies"],
+ personId: PEOPLE.alice,
+ capabilities: ["another-person", "attachments", "text-replies"],
// An agent session lives on an issue the whole workspace can see.
surface: "shared",
createChannel: (record) =>
@@ -92,13 +96,14 @@ export function linearDriver(): ChannelDriver {
api: { fetch: recordingFetch(record, decode) },
credentials: { accessToken: "linear-token", webhookSecret: SECRET },
}),
- message: (words, files = []) => {
+ message: (words, person, files = []) => {
const text = withUploads(words, files);
+ const user = PEOPLE[person];
if (delivery === 0) {
return webhook({
action: "created",
agentSession: {
- creator: { id: "user_1" },
+ creator: { id: user },
id: sessionId,
issue: { id: issueId, identifier: issueIdentifier, title: "Conformance test" },
issueId,
@@ -112,8 +117,8 @@ export function linearDriver(): ChannelDriver {
agentActivity: {
content: { body: text, type: "prompt" },
id: `prompt-${delivery}`,
- user: { id: "user_1" },
- userId: "user_1",
+ user: { id: user },
+ userId: user,
},
});
},
diff --git a/packages/eve/src/internal/testing/channel-conformance/linq-driver.ts b/packages/eve/src/internal/testing/channel-conformance/linq-driver.ts
index 8c25cc782..3e616a05c 100644
--- a/packages/eve/src/internal/testing/channel-conformance/linq-driver.ts
+++ b/packages/eve/src/internal/testing/channel-conformance/linq-driver.ts
@@ -88,6 +88,7 @@ export function linqDriver(surface: Exclude<Surface, "public"> = "shared"): Chan
return {
name: group ? "linq" : "linq-dm",
+ personId: PERSON.id,
capabilities: ["attachments", "text-replies"],
surface,
createChannel(record) {
@@ -141,7 +142,7 @@ export function linqDriver(surface: Exclude<Surface, "public"> = "shared"): Chan
dispose() {
restoreFetch?.();
},
- message: signedMessage,
+ message: (text, _person, files) => signedMessage(text, files),
findOptions(call, prompt) {
if (!call.method.endsWith("/messages") || !postedText(call)?.includes(prompt))
return undefined;
diff --git a/packages/eve/src/internal/testing/channel-conformance/photon-driver.ts b/packages/eve/src/internal/testing/channel-conformance/photon-driver.ts
index a0072c0a3..719d40d2c 100644
--- a/packages/eve/src/internal/testing/channel-conformance/photon-driver.ts
+++ b/packages/eve/src/internal/testing/channel-conformance/photon-driver.ts
@@ -104,6 +104,7 @@ export function photonDriver(): ChannelDriver {
return {
name: "photon",
+ personId: PERSON,
capabilities: ["attachments", "text-replies"],
surface: "private",
createChannel(record) {
@@ -172,7 +173,7 @@ export function photonDriver(): ChannelDriver {
dispose() {
restore?.();
},
- message: webhook,
+ message: (text, _person, files) => webhook(text, files),
findOptions(call, prompt) {
const text = postedText(call);
if (text === undefined || !text.includes(prompt)) return undefined;
diff --git a/packages/eve/src/internal/testing/channel-conformance/slack-driver.ts b/packages/eve/src/internal/testing/channel-conformance/slack-driver.ts
index 79568f80f..f6345dc2d 100644
--- a/packages/eve/src/internal/testing/channel-conformance/slack-driver.ts
+++ b/packages/eve/src/internal/testing/channel-conformance/slack-driver.ts
@@ -137,6 +137,7 @@ export function slackDriver(surface: Exclude<Surface, "public"> = "shared"): Cha
return {
name: dm ? "slack-dm" : "slack",
+ personId: PERSON,
capabilities: dm
? ["attachments", "buttons", "text-replies"]
: ["attachments", "another-person", "buttons", "text-replies"],
@@ -146,7 +147,7 @@ export function slackDriver(surface: Exclude<Surface, "public"> = "shared"): Cha
api: { fetch: recordingFetch(record, decode) },
credentials: { botToken: "xoxb-conformance", signingSecret: SIGNING_SECRET },
}),
- message: (text, files = []) => {
+ message: (text, person, files = []) => {
const ts = threadStarted ? nextTs() : threadTs;
const thread = threadStarted ? { thread_ts: threadTs } : {};
threadStarted = true;
@@ -170,7 +171,7 @@ export function slackDriver(surface: Exclude<Surface, "public"> = "shared"): Cha
channel: CHANNEL,
event_ts: ts,
ts,
- user: PERSON,
+ user: PEOPLE[person].id,
},
// Slack names the installation that received the event, which is how eve knows its
// own bot user, e.g. to strip that mention from a typed answer.
diff --git a/packages/eve/src/internal/testing/channel-conformance/teams-driver.ts b/packages/eve/src/internal/testing/channel-conformance/teams-driver.ts
index 1a976ac56..69e04beff 100644
--- a/packages/eve/src/internal/testing/channel-conformance/teams-driver.ts
+++ b/packages/eve/src/internal/testing/channel-conformance/teams-driver.ts
@@ -145,6 +145,7 @@ export function teamsDriver(surface: Exclude<Surface, "public"> = "shared"): Cha
return {
name: personal ? "teams-dm" : "teams",
+ personId: PERSON.id,
capabilities: personal
? ["attachments", "buttons", "text-replies"]
: ["attachments", "another-person", "buttons", "text-replies"],
@@ -154,14 +155,15 @@ export function teamsDriver(surface: Exclude<Surface, "public"> = "shared"): Cha
api: { fetch: recordingFetch(record, decode) },
credentials: { tokenProvider: () => "test-token", webhookVerifier: () => true },
}),
- message: (text, files = []) =>
+ message: (text, person, files = []) =>
personal
- ? activity({ attachments: attached(files), text, type: "message" })
+ ? activity({ attachments: attached(files), from: PEOPLE[person], text, type: "message" })
: // In a channel the default policy hears only mentions, so a person mentions the bot
// each time, on its own line so the test model's line-based directives still read it.
activity({
attachments: attached(files),
entities: [{ mentioned: BOT, text: MENTION, type: "mention" }],
+ from: PEOPLE[person],
text: `${text}\n${MENTION}`,
textFormat: "xml",
type: "message",
diff --git a/packages/eve/src/internal/testing/channel-conformance/telegram-driver.ts b/packages/eve/src/internal/testing/channel-conformance/telegram-driver.ts
index 460ce0d60..4acb27a3f 100644
--- a/packages/eve/src/internal/testing/channel-conformance/telegram-driver.ts
+++ b/packages/eve/src/internal/testing/channel-conformance/telegram-driver.ts
@@ -160,6 +160,7 @@ export function telegramDriver(surface: Exclude<Surface, "public"> = "shared"):
return {
name: group ? "telegram" : "telegram-dm",
+ personId: String(PERSON.id),
capabilities: !group
? ["attachments", "buttons", "text-replies"]
: ["attachments", "another-person", "buttons", "text-replies"],
@@ -170,11 +171,11 @@ export function telegramDriver(surface: Exclude<Surface, "public"> = "shared"):
botUsername: BOT.username,
credentials: { botToken: "bot-token", webhookSecretToken: SECRET },
}),
- message: (text, files = []) => {
+ message: (text, person, files = []) => {
const message: Record<string, unknown> = {
chat: CHAT,
date: 0,
- from: PERSON,
+ from: PEOPLE[person],
message_id: 1000 + updateId,
...addressed(text),
};
diff --git a/packages/eve/src/internal/testing/channel-conformance/twilio-driver.ts b/packages/eve/src/internal/testing/channel-conformance/twilio-driver.ts
index 0f009c048..a2093856f 100644
--- a/packages/eve/src/internal/testing/channel-conformance/twilio-driver.ts
+++ b/packages/eve/src/internal/testing/channel-conformance/twilio-driver.ts
@@ -75,6 +75,7 @@ export function twilioDriver(): ChannelDriver {
return {
name: "twilio",
+ personId: PERSON,
capabilities: ["attachments", "text-replies"],
surface: "private",
createChannel: (record) =>
@@ -84,7 +85,7 @@ export function twilioDriver(): ChannelDriver {
api: { fetch: recordingFetch(record, decode) },
messaging: { from: TO },
}),
- message,
+ message: (text, _person, files) => message(text, files),
findOptions(call, prompt) {
if (call.method !== "Messages.json") return undefined;
const body = call.body as { readonly Body?: unknown };
diff --git a/packages/eve/src/public/channels/github/githubChannel.test.ts b/packages/eve/src/public/channels/github/githubChannel.test.ts
index 0169bf3dc..7083a4b25 100644
--- a/packages/eve/src/public/channels/github/githubChannel.test.ts
+++ b/packages/eve/src/public/channels/github/githubChannel.test.ts
@@ -897,47 +897,6 @@ describe("githubChannel", () => {
for (const ciEvent of ciEvents) expect(ciEvent.send).not.toHaveBeenCalled();
});
- it("posts final messages through the issue comments API", async () => {
- const fetchMock = vi.fn().mockResolvedValue(new Response(JSON.stringify({ id: 77 })));
- const adapter = withState(
- getAdapter(
- githubChannel({
- api: { apiBaseUrl: "https://github.test", fetch: fetchMock },
- credentials: {
- installationToken: "ghs_test",
- webhookSecret: SECRET,
- },
- }),
- ),
- {
- conversationKind: "issue",
- installationId: 55,
- issueNumber: 5,
- owner: "vercel",
- repo: "eve",
- repositoryId: 123,
- },
- );
- const ctx = buildAdapterContext(adapter, stubAccessor());
-
- await callEvent(
- adapter,
- makeEvent("message.completed", {
- finishReason: "stop",
- message: "Final answer",
- sequence: 0,
- stepIndex: 0,
- turnId: "t1",
- }),
- ctx,
- );
-
- expect(fetchMock).toHaveBeenCalledTimes(1);
- expect(fetchMock.mock.calls[0]?.[0]).toBe(
- "https://github.test/repos/vercel/eve/issues/5/comments",
- );
- });
-
it("renders the mention instruction from a lazy botName resolver", async () => {
const fetchMock = vi.fn().mockResolvedValue(new Response(JSON.stringify({ id: 79 })));
const adapter = withState(
diff --git a/packages/eve/src/public/channels/slack/defaults.test.ts b/packages/eve/src/public/channels/slack/defaults.test.ts
index 3fefb7662..296fb9224 100644
--- a/packages/eve/src/public/channels/slack/defaults.test.ts
+++ b/packages/eve/src/public/channels/slack/defaults.test.ts
@@ -441,86 +441,6 @@ describe("defaultEvents approval lifecycle", () => {
expect(postEphemeral).toHaveBeenCalledWith("U777", "GitHub write access is required.");
});
- it("updates the shared card only after settlement", async () => {
- const { channel, request } = buildChannelStub({
- slackUsersByPrincipal: { "slack:T1:U777": "U777" },
- pendingApprovalCards: {
- "approval-1": {
- messageBlocks: [
- {
- actions: [{ action_id: "eve_input:tool-approval:approval-1:button:1" }],
- body: { text: "Approve?", type: "mrkdwn" },
- type: "card",
- },
- ],
- messageTs: "123.456",
- },
- },
- });
-
- await defaultEvents["approval.settled"]!(
- {
- outcome: "approved",
- requestId: "approval-1",
- responderPrincipalId: "slack:T1:U777",
- sequence: 1,
- stepIndex: 0,
- turnId: "turn-1",
- },
- channel,
- sessionCtx,
- );
-
- expect(request).toHaveBeenCalledWith(
- "chat.update",
- expect.objectContaining({ channel: "C123", text: "Answered: Approve", ts: "123.456" }),
- );
- const update = request.mock.calls.find(([method]) => method === "chat.update")?.[1] as {
- blocks?: unknown[];
- };
- expect(JSON.stringify(update.blocks)).toContain("Answered by <@U777>");
- expect(channel.state.pendingApprovalCards).toEqual({});
- });
-
- it("settles the request when its buttons carry tool-approval metadata", async () => {
- const { channel, request } = buildChannelStub({
- slackUsersByPrincipal: { "slack:T1:U777": "U777" },
- pendingApprovalCards: {
- "approval-1": {
- messageBlocks: [
- {
- actions: [
- { action_id: "eve_input:tool-approval:approval-1:button:0" },
- { action_id: "eve_input:tool-approval:approval-1:button:1" },
- ],
- body: { text: "Approve?", type: "mrkdwn" },
- type: "card",
- },
- ],
- messageTs: "123.456",
- },
- },
- });
-
- await defaultEvents["approval.settled"]!(
- {
- outcome: "approved",
- requestId: "approval-1",
- responderPrincipalId: "slack:T1:U777",
- sequence: 1,
- stepIndex: 0,
- turnId: "turn-1",
- },
- channel,
- sessionCtx,
- );
-
- const update = request.mock.calls.find(([method]) => method === "chat.update")?.[1] as {
- blocks?: unknown[];
- };
- expect(JSON.stringify(update.blocks)).not.toContain("eve_input:tool-approval:approval-1");
- });
-
it("keeps earlier grouped cards settled when approvals are answered out of order", async () => {
const requestIds = Array.from({ length: 5 }, (_, index) => `approval-${index + 1}`);
const messageBlocks = requestIds.map((requestId) => ({
diff --git a/packages/eve/src/public/channels/teams/defaults.test.ts b/packages/eve/src/public/channels/teams/defaults.test.ts
index 45469cab5..e8e1cde07 100644
--- a/packages/eve/src/public/channels/teams/defaults.test.ts
+++ b/packages/eve/src/public/channels/teams/defaults.test.ts
@@ -42,37 +42,6 @@ function authRequiredEvent(overrides: { displayName?: string } = {}) {
}
describe("defaultEvents approval lifecycle", () => {
- it("records a posted tool approval card for settlement", async () => {
- const { channel, post } = buildChannelStub();
-
- await defaultEvents["input.requested"]!(
- {
- requests: [
- {
- action: { callId: "call-1", input: {}, kind: "tool-call", toolName: "deploy" },
- kind: "tool-approval",
- options: [
- { id: "approve", label: "Approve" },
- { id: "cancel", label: "Cancel" },
- ],
- prompt: "Approve deployment?",
- requestId: "approval_1",
- },
- ],
- sequence: 1,
- stepIndex: 0,
- turnId: "turn-1",
- },
- channel,
- sessionCtx,
- );
-
- expect(post).toHaveBeenCalledTimes(1);
- expect(channel.state.pendingPromptCards).toEqual({
- approval_1: { activityId: "act1", prompt: "Approve deployment?" },
- });
- });
-
it("replaces the settled approval card with the responder's identity", async () => {
const { channel, update } = buildChannelStub({
approvalResponderAccounts: { "teams:TENANT:USER": { id: "USER", name: "Ada" } },