Skip to content

armgit / eve

Public Imported from GitHub
0

The Open Framework for Building Agents

[2/4] feat(eve): deliver Photon and Linq attachments to the agent (#4388)

Ben Pankowbc45ddfb0d3a
diff --git a/.changeset/chat-sdk-attachment-downloads.md b/.changeset/chat-sdk-attachment-downloads.md
new file mode 100644
index 000000000..c3d195b8f
--- /dev/null
+++ b/.changeset/chat-sdk-attachment-downloads.md
@@ -0,0 +1,5 @@
+---
+"eve": patch
+---
+
+Files people send over Chat SDK adapters that download attachments, such as Slack, Google Chat, WhatsApp, and Twilio, now reach the agent instead of a URL the model provider can't open. eve downloads each one in the step with the adapter's credentials, rebuilt through the adapter's `rehydrateAttachment`; a failed download, or a file over 25 MB, reaches the agent as a short note.
diff --git a/.changeset/photon-attachments.md b/.changeset/photon-attachments.md
new file mode 100644
index 000000000..8e51a2cae
--- /dev/null
+++ b/.changeset/photon-attachments.md
@@ -0,0 +1,5 @@
+---
+"eve": patch
+---
+
+Photon iMessage now passes files people send to the agent. eve downloads each attachment from Photon by its id, since the webhook carries no bytes; a failed download reaches the agent as a short note instead of being dropped.
diff --git a/docs/channels/chat-sdk.mdx b/docs/channels/chat-sdk.mdx
index dd0db412e..a7f714fa8 100644
--- a/docs/channels/chat-sdk.mdx
+++ b/docs/channels/chat-sdk.mdx
@@ -208,7 +208,7 @@ await to(channel, { adapterName: "resend", threadId: "resend:[email protected]" }
### Attachments
-`send` takes plain text or an AI SDK `UserContent` array. To forward a Chat SDK message's attachments, convert it with `messageToUserContent`, which returns `message.text` when there are no attachments and a `UserContent` array (text plus one file part per attachment URL) when there are:
+`send` takes plain text or an AI SDK `UserContent` array. To forward a Chat SDK message's attachments, convert it with `messageToUserContent`. It returns `message.text` when there are no attachments, and a `UserContent` array (the text plus one part per attachment) when there are:
```ts
import { messageToUserContent } from "eve/channels/chat-sdk";
@@ -218,6 +218,8 @@ bot.onNewMention(async (thread, message) => {
});
```
+When the adapter can download an attachment (it sets `fetchData`, as the Slack, Google Chat, WhatsApp, and Twilio adapters do), eve downloads it with the adapter's credentials when the turn runs, not in the webhook, and passes the bytes. Platform file URLs usually need those credentials, so the model provider couldn't open them. eve rebuilds the download with the adapter's `rehydrateAttachment`; an adapter without one can't download after the webhook returns, so its files reach the agent as a short note. A failed download, or a file over 25 MB, also becomes a note. An attachment with only a URL is passed as that URL.
+
See [File uploads](./custom#file-uploads) for how eve stages remote file URLs before the model call.
## Configuration reference
diff --git a/docs/channels/linq.mdx b/docs/channels/linq.mdx
index 36b4a03cf..112e9a596 100644
--- a/docs/channels/linq.mdx
+++ b/docs/channels/linq.mdx
@@ -20,6 +20,8 @@ The default webhook route is `/eve/v1/linq`. The channel verifies forwarded webh
Set `turnPolicy: "queue"` when every response should finish before Linq starts the next message.
+Images and other files people send reach the agent as file parts. eve downloads each one from Linq's CDN and passes the bytes; a failed download, or a file over 25 MB, reaches the agent as a short note.
+
Customize inbound dispatch with `onMessage`. Return `null` to ignore a message, or return `title` alongside `auth` to set the title when the dispatch starts a run:
```ts
diff --git a/docs/channels/photon.mdx b/docs/channels/photon.mdx
index 2a219ad0e..6122fb161 100644
--- a/docs/channels/photon.mdx
+++ b/docs/channels/photon.mdx
@@ -32,6 +32,11 @@ boundary. See [Overlapping messages](./overview#overlapping-messages).
Set `turnPolicy: "queue"` when every response should finish before Photon starts the next message.
+Images and other files people send reach the agent as file parts. Photon's
+webhook lists each file without its bytes, so eve downloads it from Photon by
+its attachment id with the project's credentials. A failed download, or a file
+over 25 MB, reaches the agent as a short note.
+
Customize inbound dispatch with `onMessage`. Return `null` to ignore a message, or return `title` alongside `auth` to set the title when the dispatch starts a run:
```ts
diff --git a/packages/eve/src/internal/attachments/limited-read.test.ts b/packages/eve/src/internal/attachments/limited-read.test.ts
new file mode 100644
index 000000000..ec532ad17
--- /dev/null
+++ b/packages/eve/src/internal/attachments/limited-read.test.ts
@@ -0,0 +1,67 @@
+import { describe, expect, it } from "vitest";
+
+import { EveAttachmentError } from "#internal/attachments/errors.js";
+import { assertWithinLimit, readLimitedBytes } from "#internal/attachments/limited-read.js";
+
+const MB = 1024 * 1024;
+
+/** A body that streams `total` bytes in `chunk`-sized pieces and declares no length. */
+function streamed(total: number, chunk: number): Response {
+ let sent = 0;
+ const body = new ReadableStream<Uint8Array>({
+ pull(controller) {
+ if (sent >= total) return controller.close();
+ const size = Math.min(chunk, total - sent);
+ sent += size;
+ controller.enqueue(new Uint8Array(size));
+ },
+ });
+ return new Response(body);
+}
+
+describe("readLimitedBytes", () => {
+ it("reads a body within the limit", async () => {
+ const bytes = await readLimitedBytes(new Response("hello"), 5, "test");
+ expect(bytes.toString()).toBe("hello");
+ });
+
+ it("refuses a body whose declared length is over the limit before reading it", async () => {
+ const response = new Response(new Uint8Array(10), {
+ headers: { "content-length": "26214401" },
+ });
+ await expect(readLimitedBytes(response, 25 * MB, "test")).rejects.toThrow(
+ new EveAttachmentError({
+ adapterKind: "test",
+ kind: "resolver-threw",
+ message: "it is over the 25 MB upload limit.",
+ }),
+ );
+ });
+
+ it("stops a large body without a declared length once it passes the limit", async () => {
+ let pulled = 0;
+ const response = streamed(30 * MB, MB);
+ const counting = new Response(
+ response.body!.pipeThrough(
+ new TransformStream({
+ transform(chunk, controller) {
+ pulled += chunk.byteLength;
+ controller.enqueue(chunk);
+ },
+ }),
+ ),
+ );
+
+ await expect(readLimitedBytes(counting, 25 * MB, "test")).rejects.toThrow(
+ "it is over the 25 MB upload limit.",
+ );
+ expect(pulled).toBeLessThanOrEqual(27 * MB);
+ });
+});
+
+describe("assertWithinLimit", () => {
+ it("fails bytes over the limit", () => {
+ expect(() => assertWithinLimit(new Uint8Array(6), 5, "test")).toThrow(EveAttachmentError);
+ expect(() => assertWithinLimit(new Uint8Array(5), 5, "test")).not.toThrow();
+ });
+});
diff --git a/packages/eve/src/internal/attachments/limited-read.ts b/packages/eve/src/internal/attachments/limited-read.ts
new file mode 100644
index 000000000..a114a729e
--- /dev/null
+++ b/packages/eve/src/internal/attachments/limited-read.ts
@@ -0,0 +1,52 @@
+import { EveAttachmentError } from "#internal/attachments/errors.js";
+
+/**
+ * Reads a download's body, failing as soon as it passes `maxBytes`. Checks a
+ * declared `content-length` first, then counts what streams in, so a body that
+ * misstates or omits its length still stops at the limit.
+ */
+export async function readLimitedBytes(
+ response: Response,
+ maxBytes: number,
+ adapterKind: string,
+): Promise<Buffer> {
+ const declared = Number(response.headers.get("content-length") ?? Number.NaN);
+ if (Number.isFinite(declared) && declared > maxBytes) {
+ await response.body?.cancel();
+ throw overLimit(maxBytes, adapterKind);
+ }
+ if (response.body === null) return Buffer.alloc(0);
+
+ const reader = response.body.getReader();
+ const chunks: Uint8Array[] = [];
+ let received = 0;
+ for (;;) {
+ const { done, value } = await reader.read();
+ if (done) break;
+ received += value.byteLength;
+ if (received > maxBytes) {
+ await reader.cancel();
+ throw overLimit(maxBytes, adapterKind);
+ }
+ chunks.push(value);
+ }
+ return Buffer.concat(chunks);
+}
+
+/** Fails a download whose bytes arrived some other way than a `Response`, once they're in hand. */
+export function assertWithinLimit(bytes: Uint8Array, maxBytes: number, adapterKind: string): void {
+ if (bytes.byteLength > maxBytes) throw overLimit(maxBytes, adapterKind);
+}
+
+function overLimit(maxBytes: number, adapterKind: string): EveAttachmentError {
+ return new EveAttachmentError({
+ adapterKind,
+ kind: "resolver-threw",
+ message: `it is over the ${formatMegabytes(maxBytes)} upload limit.`,
+ });
+}
+
+function formatMegabytes(bytes: number): string {
+ const megabytes = bytes / (1024 * 1024);
+ return `${Number.isInteger(megabytes) ? megabytes : megabytes.toFixed(1)} MB`;
+}
diff --git a/packages/eve/src/internal/testing/channel-conformance/MATRIX.md b/packages/eve/src/internal/testing/channel-conformance/MATRIX.md
index 257386f61..640dc4587 100644
--- a/packages/eve/src/internal/testing/channel-conformance/MATRIX.md
+++ b/packages/eve/src/internal/testing/channel-conformance/MATRIX.md
@@ -85,10 +85,10 @@ pnpm --filter eve exec vitest run --config vitest.unit.config.ts channel-conform
| Rule | `tui` | `web chat` | `chat-sdk` | `chat-sdk-dm` | `chat-sdk-text` | `discord` | `discord-dm` | `linq` | `linq-dm` | `slack` | `slack-dm` | `teams` | `teams-dm` | `telegram` | `telegram-dm` | `github` | `linear` | `photon` | `twilio` |
| --- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
-| an image a person sends reaches the agent with its bytes and type | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> | ❌<sup>[21](#note-21)</sup> | ❌<sup>[21](#note-21)</sup> | ❌<sup>[21](#note-21)</sup> | ❌<sup>[22](#note-22)</sup> | ❌<sup>[22](#note-22)</sup> | ❌<sup>[21](#note-21)</sup> | ❌<sup>[21](#note-21)</sup> | ✅ | ✅ | ❌<sup>[23](#note-23)</sup> | ❌<sup>[23](#note-23)</sup> | ❌<sup>[24](#note-24)</sup> | ❌<sup>[24](#note-24)</sup> | —<sup>[20](#note-20)</sup> | ✅ | ❌<sup>[25](#note-25)</sup> | ❌<sup>[26](#note-26)</sup> |
-| a PDF a person sends reaches the agent with its bytes and type | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> | ❌<sup>[21](#note-21)</sup> | | ❌<sup>[21](#note-21)</sup> | ❌<sup>[22](#note-22)</sup> | | ❌<sup>[21](#note-21)</sup> | | ✅ | | ❌<sup>[23](#note-23)</sup> | | ❌<sup>[24](#note-24)</sup> | | —<sup>[20](#note-20)</sup> | ❌<sup>[27](#note-27)</sup> | ❌<sup>[25](#note-25)</sup> | ❌<sup>[26](#note-26)</sup> |
-| a file that can't be downloaded reaches the agent as a note, not a link, and the next message still works | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> | ❌<sup>[21](#note-21)</sup> | | ❌<sup>[21](#note-21)</sup> | ❌<sup>[22](#note-22)</sup> | | ❌<sup>[21](#note-21)</sup> | | ✅ | | ❌<sup>[23](#note-23)</sup> | | ✅ | | —<sup>[20](#note-20)</sup> | ❌<sup>[28](#note-28)</sup> | ❌<sup>[25](#note-25)</sup> | ❌<sup>[26](#note-26)</sup> |
-| a file sent earlier in the conversation is still there on a later message | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> | ❌<sup>[21](#note-21)</sup> | | ❌<sup>[21](#note-21)</sup> | —<sup>[29](#note-29)</sup> | | ❌<sup>[21](#note-21)</sup> | | ✅ | | ❌<sup>[23](#note-23)</sup> | | ❌<sup>[24](#note-24)</sup> | | —<sup>[20](#note-20)</sup> | ✅ | ❌<sup>[25](#note-25)</sup> | ❌<sup>[26](#note-26)</sup> |
+| an image a person sends reaches the agent with its bytes and type | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> | ❌<sup>[21](#note-21)</sup> | ❌<sup>[21](#note-21)</sup> | ❌<sup>[21](#note-21)</sup> | ❌<sup>[22](#note-22)</sup> | ❌<sup>[22](#note-22)</sup> | ✅ | ✅ | ✅ | ✅ | ❌<sup>[23](#note-23)</sup> | ❌<sup>[23](#note-23)</sup> | ❌<sup>[24](#note-24)</sup> | ❌<sup>[24](#note-24)</sup> | —<sup>[20](#note-20)</sup> | ✅ | ✅ | ❌<sup>[25](#note-25)</sup> |
+| a PDF a person sends reaches the agent with its bytes and type | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> | ❌<sup>[21](#note-21)</sup> | | ❌<sup>[21](#note-21)</sup> | ❌<sup>[22](#note-22)</sup> | | ✅ | | ✅ | | ❌<sup>[23](#note-23)</sup> | | ❌<sup>[24](#note-24)</sup> | | —<sup>[20](#note-20)</sup> | ❌<sup>[26](#note-26)</sup> | ✅ | ❌<sup>[25](#note-25)</sup> |
+| a file that can't be downloaded reaches the agent as a note, not a link, and the next message still works | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> | ❌<sup>[21](#note-21)</sup> | | ❌<sup>[21](#note-21)</sup> | ❌<sup>[22](#note-22)</sup> | | ✅ | | ✅ | | ❌<sup>[23](#note-23)</sup> | | ✅ | | —<sup>[20](#note-20)</sup> | ❌<sup>[27](#note-27)</sup> | ✅ | ❌<sup>[25](#note-25)</sup> |
+| a file sent earlier in the conversation is still there on a later message | —<sup>[20](#note-20)</sup> | —<sup>[20](#note-20)</sup> | ❌<sup>[21](#note-21)</sup> | | ❌<sup>[21](#note-21)</sup> | —<sup>[28](#note-28)</sup> | | ✅ | | ✅ | | ❌<sup>[23](#note-23)</sup> | | ❌<sup>[24](#note-24)</sup> | | —<sup>[20](#note-20)</sup> | ✅ | ✅ | ❌<sup>[25](#note-25)</sup> |
## Notes
@@ -116,8 +116,7 @@ pnpm --filter eve exec vitest run --config vitest.unit.config.ts channel-conform
22. <a id="note-22"></a>eve reads only the slash command's message option, not its attachment options
23. <a id="note-23"></a>Teams drops files unless files.enabled is set and their host is allowlisted
24. <a id="note-24"></a>#1217: Telegram serves files as application/octet-stream, which wins over their known type
-25. <a id="note-25"></a>the iMessage adapter lists each attachment's name and type but no way to download it, so eve drops it
-26. <a id="note-26"></a>the channel parses MMS media but never passes it to the agent
-27. <a id="note-27"></a>eve reads only uploaded images from Linear; other files stay links in the text
-28. <a id="note-28"></a>a Linear upload eve can't download stays a link in the text, with no note
-29. <a id="note-29"></a>each slash command starts its own session, so no later message shares one with the file
+25. <a id="note-25"></a>the channel parses MMS media but never passes it to the agent
+26. <a id="note-26"></a>eve reads only uploaded images from Linear; other files stay links in the text
+27. <a id="note-27"></a>a Linear upload eve can't download stays a link in the text, with no note
+28. <a id="note-28"></a>each slash command starts its own session, so no later message shares one with the file
diff --git a/packages/eve/src/internal/testing/channel-conformance/conformance.ts b/packages/eve/src/internal/testing/channel-conformance/conformance.ts
index cbdd8c2d7..ef1f64eb8 100644
--- a/packages/eve/src/internal/testing/channel-conformance/conformance.ts
+++ b/packages/eve/src/internal/testing/channel-conformance/conformance.ts
@@ -151,10 +151,6 @@ function attachmentUrlPassedThrough(
}
const ATTACHMENT_URL_PASSED_THROUGH = attachmentUrlPassedThrough(/AI_DownloadError/u);
-// Linq posts nothing for a failed turn, so the reply never comes.
-const LINQ_ATTACHMENT_URL_PASSED_THROUGH = attachmentUrlPassedThrough(
- /Timed out waiting for (?:the bot to show|a reply to "Alice attached)/u,
-);
const SIGN_IN_NOT_SHOWN = /Timed out waiting for the bot to show/u;
@@ -341,34 +337,9 @@ const channelConformance = {
},
},
],
- linq: [
- {
- driver: linqDriver,
- broken: {
- ...SIGN_IN_ONLY_IN_DMS,
- ...LINQ_ATTACHMENT_URL_PASSED_THROUGH,
- },
- },
- ],
- "linq-dm": [
- {
- dm: true,
- driver: () => linqDriver("private"),
- broken: { ...LINQ_ATTACHMENT_URL_PASSED_THROUGH },
- },
- ],
- photon: [
- {
- driver: photonDriver,
- broken: filesNotSeen(
- "the iMessage adapter lists each attachment's name and type but no way to download it, so eve drops it",
- IMAGE_REACHES_AGENT,
- PDF_REACHES_AGENT,
- UNDOWNLOADABLE_FILE,
- EARLIER_FILE,
- ),
- },
- ],
+ linq: [{ driver: linqDriver, broken: SIGN_IN_ONLY_IN_DMS }],
+ "linq-dm": [{ dm: true, driver: () => linqDriver("private") }],
+ photon: [{ driver: photonDriver }],
slack: [{ driver: slackDriver, broken: SLACK_BROKEN }],
"slack-dm": [{ dm: true, driver: () => slackDriver("private"), broken: SLACK_BROKEN }],
teams: [{ driver: teamsDriver, broken: { ...TEAMS_BROKEN, ...SIGN_IN_LINK_POSTED_TO_THREAD } }],
diff --git a/packages/eve/src/internal/testing/channel-conformance/photon-driver.ts b/packages/eve/src/internal/testing/channel-conformance/photon-driver.ts
index 81f36257c..a0072c0a3 100644
--- a/packages/eve/src/internal/testing/channel-conformance/photon-driver.ts
+++ b/packages/eve/src/internal/testing/channel-conformance/photon-driver.ts
@@ -35,6 +35,8 @@ export function photonDriver(): ChannelDriver {
const chatGuid = `iMessage;-;${PERSON}${nextChat}`;
let sequence = 0;
let restore: (() => void) | undefined;
+ /** Files a person sent, by the attachment id the webhook lists each under. */
+ const uploads = new Map<string, SentFile>();
function webhook(text: string, files: readonly SentFile[] = []): Request {
sequence += 1;
@@ -45,15 +47,19 @@ export function photonDriver(): ChannelDriver {
// A message with files arrives as a group: its text, then one attachment per file.
items: [
{ content: { text, type: "text" } },
- ...files.map((file, index) => ({
- content: {
- id: `photon-attachment-${sequence}-${index}`,
- mimeType: file.mediaType,
- name: file.name,
- size: file.bytes.length,
- type: "attachment",
- },
- })),
+ ...files.map((file, index) => {
+ const id = `photon-attachment-${sequence}-${index}`;
+ uploads.set(id, file);
+ return {
+ content: {
+ id,
+ mimeType: file.mediaType,
+ name: file.name,
+ size: file.bytes.length,
+ type: "attachment",
+ },
+ };
+ }),
],
type: "group",
};
@@ -84,6 +90,18 @@ export function photonDriver(): ChannelDriver {
});
}
+ async function getUpload(_ctx: unknown, id: string) {
+ const file = uploads.get(id);
+ if (file === undefined) return undefined;
+ return {
+ async read() {
+ if (file.downloadable === false)
+ throw new Error(`Attachment ${id} could not be downloaded`);
+ return Buffer.from(file.bytes);
+ },
+ };
+ }
+
return {
name: "photon",
capabilities: ["attachments", "text-replies"],
@@ -109,9 +127,31 @@ export function photonDriver(): ChannelDriver {
startTyping: async () => {},
stopTyping: async () => {},
};
+ // The webhook never carries a file's bytes; eve fetches each by id through
+ // iMessage's `getAttachment`, which this app's iMessage runtime serves.
+ const app = {
+ __internal: {
+ platforms: new Map([
+ [
+ "iMessage",
+ {
+ client: null,
+ config: null,
+ store: null,
+ definition: { actions: { getAttachment: getUpload } },
+ },
+ ],
+ ]),
+ },
+ };
// The channel builds its adapter internally, so patch the prototype's
// private seams to spectrum-ts: building the app and looking up a Space.
- const fakes = { ensureApp: async () => {}, resolveSpace: async () => space };
+ const fakes = {
+ async ensureApp(this: { app: unknown }) {
+ this.app = app;
+ },
+ resolveSpace: async () => space,
+ };
const originals = Object.keys(fakes).map(
(key) => [key, Reflect.get(iMessageAdapter.prototype, key)] as const,
);
diff --git a/packages/eve/src/public/channels/chat-sdk/attachment-refs.ts b/packages/eve/src/public/channels/chat-sdk/attachment-refs.ts
new file mode 100644
index 000000000..046d6d0a7
--- /dev/null
+++ b/packages/eve/src/public/channels/chat-sdk/attachment-refs.ts
@@ -0,0 +1,41 @@
+import type { Attachment } from "#compiled/chat/index.js";
+
+/**
+ * URL protocol for a Chat SDK attachment the adapter downloads. The URL names
+ * the adapter and carries the attachment's serializable fields, so the
+ * channel's `fetchFile` can rebuild the download with the adapter's
+ * `rehydrateAttachment` in the step, after the message crosses the queue.
+ */
+export const CHAT_SDK_FILE_PROTOCOL = "chat-sdk-file:";
+
+/** The serializable part of an attachment, as Chat SDK's `Message.toJSON` keeps it. */
+export interface ChatSdkFileRef {
+ readonly adapter: string;
+ readonly attachment: Pick<
+ Attachment,
+ "fetchMetadata" | "mimeType" | "name" | "size" | "type" | "url"
+ >;
+}
+
+export function encodeChatSdkFileRef(ref: ChatSdkFileRef): URL {
+ const { fetchMetadata, mimeType, name, size, type, url } = ref.attachment;
+ const payload = {
+ adapter: ref.adapter,
+ attachment: { fetchMetadata, mimeType, name, size, type, url },
+ };
+ const encoded = new URL(CHAT_SDK_FILE_PROTOCOL);
+ encoded.searchParams.set("p", Buffer.from(JSON.stringify(payload)).toString("base64url"));
+ return encoded;
+}
+
+export function parseChatSdkFileRef(url: URL): ChatSdkFileRef | null {
+ if (url.protocol !== CHAT_SDK_FILE_PROTOCOL) return null;
+ const encoded = url.searchParams.get("p");
+ if (encoded === null) return null;
+ try {
+ const ref = JSON.parse(Buffer.from(encoded, "base64url").toString("utf8")) as ChatSdkFileRef;
+ return typeof ref.adapter === "string" && typeof ref.attachment === "object" ? ref : null;
+ } catch {
+ return null;
+ }
+}
diff --git a/packages/eve/src/public/channels/chat-sdk/chatSdkChannel.test.ts b/packages/eve/src/public/channels/chat-sdk/chatSdkChannel.test.ts
index 294f872c2..855ea8609 100644
--- a/packages/eve/src/public/channels/chat-sdk/chatSdkChannel.test.ts
+++ b/packages/eve/src/public/channels/chat-sdk/chatSdkChannel.test.ts
@@ -19,6 +19,7 @@ import type { RouteHandlerArgs } from "#public/definitions/channel.js";
import type {
Adapter,
AdapterPostableMessage,
+ Attachment,
ChatInstance,
FetchResult,
FormattedContent,
@@ -916,6 +917,97 @@ describe("messageToUserContent", () => {
});
});
+describe("attachments the adapter downloads", () => {
+ function withDownload(fetchData: () => Promise<Buffer>, size?: number): Message {
+ return new Message({
+ attachments: [
+ {
+ fetchData,
+ fetchMetadata: { fileId: "F1" },
+ mimeType: "image/png",
+ name: "diagram.png",
+ size,
+ type: "image",
+ url: "https://files.test/F1",
+ },
+ ],
+ author: author(),
+ formatted: parseMarkdown(""),
+ id: "message-5",
+ isMention: true,
+ metadata: metadata(),
+ raw: {},
+ text: "",
+ threadId: THREAD_ID,
+ });
+ }
+
+ /** The channel's `fetchFile` for the one file part `messageToUserContent` made. */
+ async function fetchDeferred(adapter: TestAdapter & Adapter, inbound: Message) {
+ const bridge = chatSdkChannel({
+ adapters: { test: adapter },
+ state: memoryState(),
+ userName: "bot",
+ });
+ const [part] = messageToUserContent(inbound) as Exclude<
+ ReturnType<typeof messageToUserContent>,
+ string
+ >;
+ // Only the URL crosses the queue; the message's own fetchData doesn't.
+ const href = ((part as { data: URL }).data as URL).href;
+ return await getAdapter(bridge.channel).fetchFile!(href);
+ }
+
+ it("defers the download to the step and rebuilds it with the adapter's rehydrateAttachment", async () => {
+ const adapter = testAdapter();
+ adapter.rehydrateAttachment = (attachment) => ({
+ ...attachment,
+ fetchData: async () => Buffer.from(`bytes of ${String(attachment.fetchMetadata?.fileId)}`),
+ });
+ const fetchData = vi.fn(async () => Buffer.from("eager"));
+
+ const resolved = await fetchDeferred(adapter, withDownload(fetchData));
+
+ expect(fetchData).not.toHaveBeenCalled();
+ expect(resolved).toEqual({ bytes: Buffer.from("bytes of F1"), mediaType: "image/png" });
+ });
+
+ it("fails a download over the upload limit", async () => {
+ const adapter = testAdapter();
+ adapter.rehydrateAttachment = (attachment) => ({
+ ...attachment,
+ fetchData: async () => Buffer.alloc(25 * 1024 * 1024 + 1),
+ });
+
+ await expect(
+ fetchDeferred(
+ adapter,
+ withDownload(async () => Buffer.alloc(0)),
+ ),
+ ).rejects.toThrow("it is over the 25 MB upload limit.");
+ });
+
+ it("notes a file the adapter reports as over the limit without deferring it", () => {
+ expect(
+ messageToUserContent(withDownload(async () => Buffer.alloc(0), 25 * 1024 * 1024 + 1)),
+ ).toEqual([
+ {
+ text: "Attachment diagram.png was not retrieved: it is over the upload limit.",
+ type: "text",
+ },
+ ]);
+ });
+
+ it("fails the download when the adapter can't rebuild it after the webhook", async () => {
+ await expect(
+ fetchDeferred(
+ testAdapter(),
+ withDownload(async () => Buffer.alloc(0)),
+ ),
+ ).rejects.toThrow("the test adapter can't download it after the webhook returns.");
+ });
+});
+
describe("isNotImplemented", () => {
it("matches errors by name and by code", () => {
expect(isNotImplemented(new NotImplementedError("startTyping"))).toBe(true);
@@ -953,6 +1045,7 @@ class TestAdapter {
typingStatuses: Array<string | undefined> = [];
startTypingError: Error | null = null;
editError: Error | null = null;
+ rehydrateAttachment?: (attachment: Attachment) => Attachment;
async initialize(chat: ChatInstance): Promise<void> {
this.chat = chat;
diff --git a/packages/eve/src/public/channels/chat-sdk/chatSdkChannel.ts b/packages/eve/src/public/channels/chat-sdk/chatSdkChannel.ts
index 22f7c957d..81c07aadd 100644
--- a/packages/eve/src/public/channels/chat-sdk/chatSdkChannel.ts
+++ b/packages/eve/src/public/channels/chat-sdk/chatSdkChannel.ts
@@ -8,8 +8,11 @@ import type {
import type { SessionAuthContext, TurnPolicy } from "#channel/types.js";
import { ContextContainer, contextStorage } from "#context/container.js";
import { ContextKey } from "#context/key.js";
+import { EveAttachmentError } from "#internal/attachments/errors.js";
+import { assertWithinLimit } from "#internal/attachments/limited-read.js";
import { createLogger, extractErrorId, formatErrorHint } from "#internal/logging.js";
import type { InputResolution, UnstampedMessageStreamEvent } from "#protocol/message.js";
+import type { FetchFileResult } from "#shared/channel-definition.js";
import type { InputRequest } from "#shared/input.js";
import {
type InputResponse,
@@ -26,7 +29,9 @@ import type {
} from "#compiled/chat/index.js";
import { Chat, Message, ThreadImpl } from "#compiled/chat/index.js";
import { defaultAuthorizationEvents } from "#public/channels/chat-sdk/authorization.js";
+import { parseChatSdkFileRef } from "#public/channels/chat-sdk/attachment-refs.js";
import { decodeInputAction, renderInputRequests } from "#public/channels/chat-sdk/input-actions.js";
+import { DEFAULT_UPLOAD_POLICY } from "#public/channels/upload-policy.js";
import { isNotImplemented } from "#public/channels/chat-sdk/notImplemented.js";
import {
defineChannel,
@@ -298,6 +303,7 @@ export function chatSdkChannel<TAdapters extends ChatSdkAdapters>(
>({
kindHint: "chat-sdk",
turnPolicy: config.turnPolicy,
+ fetchFile: (url) => fetchAttachment(bot, url),
state: initialState(),
...chatSdkInstrumentation,
context(state) {
@@ -628,6 +634,44 @@ function initialState(): ChatSdkChannelState {
return { pendingAuthMessageIds: {}, thread: null };
}
+/**
+ * Downloads an attachment `messageToUserContent` deferred to the step: rebuilds
+ * the download with the adapter's `rehydrateAttachment`, since the message's
+ * own `fetchData` didn't survive the queue.
+ */
+async function fetchAttachment<TAdapters extends ChatSdkAdapters>(
+ bot: Chat<TAdapters>,
+ url: string,
+): Promise<FetchFileResult | null> {
+ const ref = parseChatSdkFileRef(new URL(url));
+ if (ref === null) return null;
+ // The step may run where the webhook didn't, so the adapters may not be initialized yet.
+ await bot.initialize();
+ const rehydrated = bot.getAdapter(ref.adapter).rehydrateAttachment?.(ref.attachment);
+ if (rehydrated?.fetchData === undefined) {
+ throw new EveAttachmentError({
+ adapterKind: "chat-sdk",
+ kind: "resolver-threw",
+ message: `the ${ref.adapter} adapter can't download it after the webhook returns.`,
+ });
+ }
+ let data: Buffer | ArrayBuffer;
+ try {
+ data = await rehydrated.fetchData();
+ } catch (cause) {
+ if (cause instanceof EveAttachmentError) throw cause;
+ throw new EveAttachmentError({
+ adapterKind: "chat-sdk",
+ cause,
+ kind: "resolver-threw",
+ message: `the ${ref.adapter} download failed.`,
+ });
+ }
+ const bytes = Buffer.isBuffer(data) ? data : Buffer.from(data);
+ assertWithinLimit(bytes, DEFAULT_UPLOAD_POLICY.maxBytes, "chat-sdk");
+ return { bytes, mediaType: ref.attachment.mimeType };
+}
+
function threadFromState<TAdapters extends ChatSdkAdapters>(
bot: Chat<TAdapters>,
state: ChatSdkChannelState,
diff --git a/packages/eve/src/public/channels/chat-sdk/messageToUserContent.ts b/packages/eve/src/public/channels/chat-sdk/messageToUserContent.ts
index 3763d2a53..f394d9ceb 100644
--- a/packages/eve/src/public/channels/chat-sdk/messageToUserContent.ts
+++ b/packages/eve/src/public/channels/chat-sdk/messageToUserContent.ts
@@ -1,6 +1,8 @@
import type { UserContent } from "ai";
-import type { Message } from "#compiled/chat/index.js";
+import type { Attachment, Message } from "#compiled/chat/index.js";
+import { encodeChatSdkFileRef } from "#public/channels/chat-sdk/attachment-refs.js";
+import { DEFAULT_UPLOAD_POLICY } from "#public/channels/upload-policy.js";
type UserContentParts = Exclude<UserContent, string>;
@@ -8,11 +10,17 @@ type UserContentParts = Exclude<UserContent, string>;
* Converts a Chat SDK `Message` into the input shape `chatSdkChannel().send`
* accepts.
*
- * Returns `message.text` when the message has no attachments. When attachments
- * are present, returns an AI SDK `UserContent` array: the text (when non-empty)
- * followed by one `file` part per attachment that exposes a URL. Attachments
- * without a URL are skipped, and a message whose only attachments lack URLs
- * falls back to `message.text`.
+ * Returns `message.text` when the message has no attachments. Otherwise
+ * returns an AI SDK `UserContent` array: the text (when non-empty) followed by
+ * one part per attachment.
+ *
+ * An attachment the adapter can download (`fetchData`) becomes a file part the
+ * channel downloads later, in the step, with the adapter's credentials: a
+ * platform's file URL usually needs them, so the model provider couldn't open
+ * it. The channel rebuilds the download with the adapter's
+ * `rehydrateAttachment`. A file over 25 MB, or one that fails to download,
+ * reaches the agent as a short note. An attachment with only a URL is passed as
+ * that URL. Attachments with neither are skipped.
*/
export function messageToUserContent(message: Message): string | UserContent {
const attachments = message.attachments ?? [];
@@ -24,14 +32,34 @@ export function messageToUserContent(message: Message): string | UserContent {
if (message.text) {
parts.push({ text: message.text, type: "text" });
}
+ // Chat SDK thread ids start with the name of the adapter that owns them.
+ const adapter = message.threadId.split(":")[0]!;
for (const attachment of attachments) {
- if (!attachment.url) continue;
- parts.push({
- data: new URL(attachment.url),
- filename: attachment.name,
- mediaType: attachment.mimeType ?? "application/octet-stream",
- type: "file",
- });
+ const part = attachmentToPart(adapter, attachment);
+ if (part !== null) parts.push(part);
}
return parts.length > 0 ? parts : message.text;
}
+
+function attachmentToPart(
+ adapter: string,
+ attachment: Attachment,
+): UserContentParts[number] | null {
+ const mediaType = attachment.mimeType ?? "application/octet-stream";
+ if (attachment.fetchData === undefined) {
+ if (!attachment.url) return null;
+ return { data: new URL(attachment.url), filename: attachment.name, mediaType, type: "file" };
+ }
+ if (attachment.size !== undefined && attachment.size > DEFAULT_UPLOAD_POLICY.maxBytes) {
+ return {
+ text: `Attachment ${attachment.name ?? "file"} was not retrieved: it is over the upload limit.`,
+ type: "text",
+ };
+ }
+ return {
+ data: encodeChatSdkFileRef({ adapter, attachment }),
+ filename: attachment.name,
+ mediaType,
+ type: "file",
+ };
+}
diff --git a/packages/eve/src/public/channels/linq/inboundContent.ts b/packages/eve/src/public/channels/linq/inboundContent.ts
index 800ab2060..aace3dd8a 100644
--- a/packages/eve/src/public/channels/linq/inboundContent.ts
+++ b/packages/eve/src/public/channels/linq/inboundContent.ts
@@ -1,7 +1,9 @@
import type { UserContent } from "ai";
-import type { Message } from "#compiled/chat/index.js";
+import type { Attachment, Message } from "#compiled/chat/index.js";
+import { readLimitedBytes } from "#internal/attachments/limited-read.js";
import { messageToUserContent } from "#public/channels/chat-sdk/index.js";
+import { DEFAULT_UPLOAD_POLICY } from "#public/channels/upload-policy.js";
/** Returns model-visible Linq content, or `undefined` for blank messages. */
export function linqInboundContent(message: Message): string | UserContent | undefined {
@@ -9,3 +11,21 @@ export function linqInboundContent(message: Message): string | UserContent | und
if (typeof content === "string") return content.trim().length > 0 ? content : undefined;
return content.length > 0 ? content : undefined;
}
+
+/**
+ * Rebuilds a Linq attachment's download after the message crosses the queue.
+ * The Linq adapter downloads from its CDN URL without credentials but has no
+ * `rehydrateAttachment`, so this does the same, stopping at the upload limit.
+ */
+export function rehydrateLinqAttachment(attachment: Attachment): Attachment {
+ const { url } = attachment;
+ if (!url) return attachment;
+ return {
+ ...attachment,
+ async fetchData() {
+ const response = await fetch(url);
+ if (!response.ok) throw new Error(`Linq CDN returned HTTP ${response.status}.`);
+ return await readLimitedBytes(response, DEFAULT_UPLOAD_POLICY.maxBytes, "linq");
+ },
+ };
+}
diff --git a/packages/eve/src/public/channels/linq/linqChannel.ts b/packages/eve/src/public/channels/linq/linqChannel.ts
index f5e0efd8a..e6c9c4f1d 100644
--- a/packages/eve/src/public/channels/linq/linqChannel.ts
+++ b/packages/eve/src/public/channels/linq/linqChannel.ts
@@ -18,7 +18,10 @@ import {
type LinqWebhookVerifier,
} from "#compiled/@linqapp/chat-sdk-adapter/index.js";
import type { Message, Thread } from "#compiled/chat/index.js";
-import { linqInboundContent } from "#public/channels/linq/inboundContent.js";
+import {
+ linqInboundContent,
+ rehydrateLinqAttachment,
+} from "#public/channels/linq/inboundContent.js";
/** Context passed to {@link LinqChannelConfig.onMessage}. */
export interface LinqInboundMessageContext {
@@ -95,6 +98,8 @@ export function linqChannel(config: LinqChannelConfig): LinqChannel {
const adapterConfig: LinqAdapterConfig = normalizeCredentials(config.credentials);
if (config.baseURL !== undefined) adapterConfig.baseURL = config.baseURL;
const linq = createLinqAdapter(adapterConfig);
+ // The channel downloads attachments in the step, after the message's own fetchData is gone.
+ linq.rehydrateAttachment = rehydrateLinqAttachment;
const bridge = chatSdkChannel({
adapters: { linq },
concurrency: "concurrent",
diff --git a/packages/eve/src/public/channels/photon/attachments.test.ts b/packages/eve/src/public/channels/photon/attachments.test.ts
new file mode 100644
index 000000000..2062c9c80
--- /dev/null
+++ b/packages/eve/src/public/channels/photon/attachments.test.ts
@@ -0,0 +1,102 @@
+import { describe, expect, it, vi } from "vitest";
+
+import { type Attachment, Message } from "#compiled/chat/index.js";
+import {
+ addPhotonAttachmentDownloads,
+ addPhotonAttachmentRehydration,
+} from "#public/channels/photon/attachments.js";
+
+function message(raw: unknown, names: readonly string[]): Message {
+ return new Message({
+ attachments: names.map((name) => ({ mimeType: "image/jpeg", name, type: "image" as const })),
+ formatted: { type: "root", children: [] },
+ metadata: { dateSent: new Date(0), edited: false },
+ author: { fullName: "User", isBot: false, isMe: false, userId: "user", userName: "user" },
+ id: "message-id",
+ raw,
+ text: "",
+ threadId: "thread-id",
+ });
+}
+
+function appServing(files: Record<string, string>) {
+ const getAttachment = vi.fn(async (_ctx: unknown, id: string, _phone?: string) =>
+ files[id] === undefined ? undefined : { read: async () => Buffer.from(files[id]!) },
+ );
+ const runtime = {
+ client: "client",
+ config: "config",
+ definition: { actions: { getAttachment } },
+ };
+ const app = {
+ __internal: { platforms: new Map([["iMessage", { ...runtime, store: "store" }]]) },
+ };
+ const adapter: { app: unknown; rehydrateAttachment?(attachment: Attachment): Attachment } = {
+ app,
+ };
+ return { adapter, getAttachment };
+}
+
+describe("addPhotonAttachmentDownloads", () => {
+ it("downloads each webhook attachment by its id, in the order the adapter lists them", async () => {
+ const { adapter, getAttachment } = appServing({ "a-1": "first", "a-2": "second" });
+ const inbound = message(
+ {
+ content: {
+ items: [
+ { content: { text: "Here you go", type: "text" } },
+ { content: { id: "a-1", name: "one.jpg", type: "attachment" } },
+ {
+ content: {
+ content: { id: "a-2", name: "two.jpg", type: "attachment" },
+ type: "reply",
+ },
+ },
+ ],
+ type: "group",
+ },
+ space: { id: "chat", phone: "+15550100" },
+ },
+ ["one.jpg", "two.jpg"],
+ );
+
+ addPhotonAttachmentDownloads(inbound, adapter);
+
+ const [one, two] = inbound.attachments;
+ expect(await one?.fetchData?.()).toEqual(Buffer.from("first"));
+ expect(await two?.fetchData?.()).toEqual(Buffer.from("second"));
+ expect(getAttachment).toHaveBeenCalledWith(
+ { client: "client", config: "config", store: "store" },
+ "a-1",
+ "+15550100",
+ );
+ });
+
+ it("fails the download when iMessage no longer has the attachment", async () => {
+ const { adapter } = appServing({});
+ const inbound = message({ content: { id: "gone", type: "attachment" } }, ["gone.jpg"]);
+
+ addPhotonAttachmentDownloads(inbound, adapter);
+
+ await expect(inbound.attachments[0]?.fetchData?.()).rejects.toThrow(
+ "iMessage attachment gone was not found.",
+ );
+ });
+
+ it("rebuilds the download from fetchMetadata alone, as after the queue", async () => {
+ const { adapter, getAttachment } = appServing({ "a-1": "first" });
+ const inbound = message(
+ { content: { id: "a-1", type: "attachment" }, space: { id: "chat", phone: "+15550100" } },
+ ["one.jpg"],
+ );
+ addPhotonAttachmentDownloads(inbound, adapter);
+ addPhotonAttachmentRehydration(adapter);
+
+ const { fetchData: _dropped, ...serialized } = inbound.attachments[0]!;
+ const rehydrated = adapter.rehydrateAttachment!(serialized);
+
+ expect(serialized.fetchMetadata).toEqual({ id: "a-1", phone: "+15550100" });
+ expect(await rehydrated.fetchData?.()).toEqual(Buffer.from("first"));
+ expect(getAttachment).toHaveBeenCalledWith(expect.anything(), "a-1", "+15550100");
+ });
+});
diff --git a/packages/eve/src/public/channels/photon/attachments.ts b/packages/eve/src/public/channels/photon/attachments.ts
new file mode 100644
index 000000000..cff062183
--- /dev/null
+++ b/packages/eve/src/public/channels/photon/attachments.ts
@@ -0,0 +1,118 @@
+import type { Attachment, Message } from "#compiled/chat/index.js";
+
+const IMESSAGE_PLATFORM = "iMessage";
+
+/** The part of a Spectrum message eve reads to find each attachment's id. */
+interface SpectrumContent {
+ readonly content?: SpectrumContent;
+ readonly id?: unknown;
+ readonly items?: readonly { readonly content?: SpectrumContent }[];
+ readonly type?: unknown;
+}
+
+/** An attachment iMessage's `getAttachment` resolves, as spectrum-ts wraps it. */
+interface SpectrumAttachment {
+ read(): Promise<Buffer>;
+}
+
+/**
+ * The slice of a spectrum-ts app that holds each platform's runtime. It is
+ * what `imessage(app)` reads to bind iMessage's instance actions, such as
+ * `getAttachment`.
+ */
+interface SpectrumApp {
+ readonly __internal: {
+ readonly platforms: ReadonlyMap<
+ string,
+ {
+ readonly client: unknown;
+ readonly config: unknown;
+ readonly store: unknown;
+ readonly definition: {
+ readonly actions?: {
+ readonly getAttachment?: (
+ ctx: { readonly client: unknown; readonly config: unknown; readonly store: unknown },
+ id: string,
+ phone?: string,
+ ) => Promise<SpectrumAttachment | undefined>;
+ };
+ };
+ }
+ >;
+ };
+}
+
+/** The adapter that owns the spectrum-ts app eve downloads attachments through. */
+interface PhotonAdapter {
+ readonly app: unknown;
+ rehydrateAttachment?(attachment: Attachment): Attachment;
+}
+
+/**
+ * Lets eve download each file a person sent over iMessage. The Spectrum
+ * webhook lists an attachment's name and type but never its bytes, so this
+ * records each attachment's id in its `fetchMetadata` and gives it a
+ * `fetchData`; the channel downloads it later, in the step, through
+ * {@link addPhotonAttachmentRehydration}.
+ */
+export function addPhotonAttachmentDownloads(message: Message, adapter: PhotonAdapter): void {
+ const attachments = message.attachments ?? [];
+ if (attachments.length === 0) return;
+ const raw = message.raw as { readonly content?: SpectrumContent; readonly space?: unknown };
+ // The adapter lists attachments in the order it walks the content, so their ids line up.
+ const ids = contentAttachmentIds(raw.content);
+ const phone = spacePhone(raw.space);
+ attachments.forEach((attachment: Attachment, index) => {
+ const id = ids[index];
+ if (attachment.fetchData !== undefined || id === undefined) return;
+ attachment.fetchMetadata = phone === undefined ? { id } : { id, phone };
+ attachment.fetchData = () => downloadAttachment(adapter.app, id, phone);
+ });
+}
+
+/**
+ * Gives the adapter a `rehydrateAttachment`, which the iMessage adapter lacks,
+ * so the channel can rebuild an attachment's download from its `fetchMetadata`
+ * after the message crosses the queue.
+ */
+export function addPhotonAttachmentRehydration(adapter: PhotonAdapter): void {
+ adapter.rehydrateAttachment = (attachment) => {
+ const { id, phone } = attachment.fetchMetadata ?? {};
+ if (id === undefined) return attachment;
+ return { ...attachment, fetchData: () => downloadAttachment(adapter.app, id, phone) };
+ };
+}
+
+async function downloadAttachment(app: unknown, id: string, phone?: string): Promise<Buffer> {
+ const runtime = (app as SpectrumApp | null)?.__internal?.platforms.get(IMESSAGE_PLATFORM);
+ const getAttachment = runtime?.definition.actions?.getAttachment;
+ if (runtime === undefined || getAttachment === undefined) {
+ throw new Error("The iMessage adapter has no Spectrum app to download attachments with.");
+ }
+ const found = await getAttachment(
+ { client: runtime.client, config: runtime.config, store: runtime.store },
+ id,
+ phone,
+ );
+ if (found === undefined) throw new Error(`iMessage attachment ${id} was not found.`);
+ return await found.read();
+}
+
+/** Walks content the way the adapter does, so the result lines up with `message.attachments`. */
+function contentAttachmentIds(content: SpectrumContent | undefined): (string | undefined)[] {
+ if (content === undefined) return [];
+ if (content.type === "attachment" || content.type === "voice") {
+ return [typeof content.id === "string" && content.id.length > 0 ? content.id : undefined];
+ }
+ if (content.type === "reply") return contentAttachmentIds(content.content);
+ if (content.type === "group") {
+ return (content.items ?? []).flatMap((item) => contentAttachmentIds(item.content));
+ }
+ return [];
+}
+
+function spacePhone(space: unknown): string | undefined {
+ if (typeof space !== "object" || space === null) return undefined;
+ const phone = (space as { readonly phone?: unknown }).phone;
+ return typeof phone === "string" ? phone : undefined;
+}
diff --git a/packages/eve/src/public/channels/photon/photonIMessageChannel.test.ts b/packages/eve/src/public/channels/photon/photonIMessageChannel.test.ts
index 6bb76ce94..7c0ef7df8 100644
--- a/packages/eve/src/public/channels/photon/photonIMessageChannel.test.ts
+++ b/packages/eve/src/public/channels/photon/photonIMessageChannel.test.ts
@@ -22,7 +22,7 @@ vi.mock("#compiled/@chat-adapter/state-memory/index.js", () => ({
createMemoryState: vi.fn(),
}));
vi.mock("#compiled/@photon-ai/chat-adapter-imessage/index.js", () => ({
- createiMessageAdapter: vi.fn(),
+ createiMessageAdapter: vi.fn(() => ({})),
}));
vi.mock("#public/channels/auth.js", () => ({ vercelOidc: vi.fn() }));
diff --git a/packages/eve/src/public/channels/photon/photonIMessageChannel.ts b/packages/eve/src/public/channels/photon/photonIMessageChannel.ts
index 3955433e1..0bda13a52 100644
--- a/packages/eve/src/public/channels/photon/photonIMessageChannel.ts
+++ b/packages/eve/src/public/channels/photon/photonIMessageChannel.ts
@@ -14,6 +14,10 @@ import {
type iMessageCredentialProvider,
type iMessageWebhookVerifier,
} from "#compiled/@photon-ai/chat-adapter-imessage/index.js";
+import {
+ addPhotonAttachmentDownloads,
+ addPhotonAttachmentRehydration,
+} from "#public/channels/photon/attachments.js";
import { photonInboundContent } from "#public/channels/photon/inboundContent.js";
/** Photon project credentials used by {@link photonIMessageChannel}. */
@@ -85,6 +89,7 @@ export function photonIMessageChannel(config: PhotonIMessageChannelConfig): Phot
? { webhookSecret }
: { webhookVerifier: vercelOidc() }),
});
+ addPhotonAttachmentRehydration(imessage);
const bridge = chatSdkChannel({
adapters: { imessage },
concurrency: "concurrent",
@@ -136,7 +141,9 @@ async function dispatchMessage(
): Promise<void> {
const result = await onMessage({ thread }, message);
if (result === null) return;
- await markReadBestEffort(bridge.bot.getAdapter("imessage"), thread, message);
+ const adapter = bridge.bot.getAdapter("imessage");
+ await markReadBestEffort(adapter, thread, message);
+ addPhotonAttachmentDownloads(message, adapter);
const content = photonInboundContent(message);
if (content === undefined) return;
await bridge.send(content, {
diff --git a/packages/eve/src/runtime/agent/mock-model-adapter.ts b/packages/eve/src/runtime/agent/mock-model-adapter.ts
index 00533b9c9..52a5eef61 100644
--- a/packages/eve/src/runtime/agent/mock-model-adapter.ts
+++ b/packages/eve/src/runtime/agent/mock-model-adapter.ts
@@ -716,7 +716,10 @@ function listPromptAttachments(prompt: BootstrapPrompt): readonly Record<string,
if (message.role !== "user" || typeof message.content === "string") return [];
return message.content.flatMap((part): Record<string, unknown>[] => {
if (typeof part === "string") return [];
- if (part.type === "text") return ATTACHMENT_NOTE.test(part.text) ? [{ note: part.text }] : [];
+ // Single quotes, so a channel that strips Markdown escapes can't break the listing's JSON.
+ if (part.type === "text") {
+ return ATTACHMENT_NOTE.test(part.text) ? [{ note: part.text.replaceAll('"', "'") }] : [];
+ }
if (part.type !== "file") return [];
const name = part.filename?.split("/").at(-1) ?? null;
const data = fileData(part.data);