The Open Framework for Building Agents
title: "Voice Notes" description: "Transcribe voice notes, voice messages, and other audio attachments with an AI Gateway transcription model so an eve agent can answer them."
Voice notes and other audio files reach your agent, but by default the model does not transcribe them. To give your agent the ability to answer a voice note, you will need to add a tool that transcribes the staged file.
This works for any channel that delivers audio as a file, such as iMessage voice notes through Photon, Slack voice clips, and files sent through the eve and Chat SDK channels. See Inbound attachments for how eve stages files.
Add a transcription tool
import { posix } from "node:path";
import { transcribe } from "ai";
import { defineTool } from "eve/tools";
import { z } from "zod";
const TRANSCRIPTION_MODEL = "openai/whisper-1";
const ATTACHMENTS_DIR = "/workspace/.eve/attachments/";
export default defineTool({
description:
"Transcribe an audio attachment, such as a voice note, to text. Pass the path from its `Attached file` label.",
inputSchema: z.object({
path: z.string().describe(`Attachment path under ${ATTACHMENTS_DIR}`),
}),
outputSchema: z.object({ transcript: z.string() }),
async execute({ path }, ctx) {
const file = posix.normalize(path);
if (!file.startsWith(ATTACHMENTS_DIR)) {
throw new Error(`Only attachments under ${ATTACHMENTS_DIR} can be transcribed.`);
}
const sandbox = await ctx.getSandbox();
const audio = await sandbox.readBinaryFile({ path: file, abortSignal: ctx.abortSignal });
if (!audio) {
throw new Error(`Attachment not found: ${file}. Ask the user to send the voice note again.`);
}
try {
const { text } = await transcribe({
model: TRANSCRIPTION_MODEL,
audio,
abortSignal: ctx.abortSignal,
});
return { transcript: text.trim() };
} catch (error) {
const reason = error instanceof Error ? error.message : String(error);
throw new Error(
`Transcription failed (${reason}). Tell the user you couldn't understand the voice note and ask them to type it.`,
);
}
},
});
transcribe() uses the same credentials as your eve agent.
Tell the agent when to transcribe
When a message has an audio attachment (`Attached file ... (audio/...)`), such as a voice note, call `transcribe_audio` with its path first and treat the transcript as what the user said. Do not mention the transcription step unless it fails.
When transcription fails
When execute throws, eve gives the error to the model as a tool error, which it can relay to the user. See When a tool throws. A file over the channel's upload limit, 25 MB by default, or a file eve can't download reaches the agent as a short note instead of a file.