Skip to content

Free, Open Source, Self-Hosted WhatsApp API Gateway

fix(baileys): decode a voice note's waveform through ffmpeg

Yudhi Armyndharis1d3a3f5f476c
diff --git a/.env.example b/.env.example
index 8e49e458f..5bf6d7ad6 100644
--- a/.env.example
+++ b/.env.example
@@ -642,7 +642,7 @@ WEBHOOK_SSRF_PROTECT=true
# Media conversion. Each number below must be a positive integer, or the boot fails, and the timeout
# no greater than 2147483647 ms (Node's timer ceiling), even with conversion disabled.
# MEDIA_CONVERSION_ENABLED=false # default false
-# FFMPEG_PATH=ffmpeg # absolute path, for a binary outside PATH
+# FFMPEG_PATH=ffmpeg # absolute path, for a binary outside PATH; also reads a Baileys voice note's waveform
# MEDIA_CONVERSION_TIMEOUT_MS=60000 # kill one conversion after this long (default 60s)
# MEDIA_CONVERSION_MAX_OUTPUT_BYTES=52428800 # cap the CONVERTED bytes (default 50 MiB)
# MEDIA_CONVERSION_CONCURRENCY=2 # max concurrent ffmpeg processes; a short queue
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 89cfdbf8d..f793544b0 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -111,7 +111,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Preserve webhook outbox payloads until delivery or durable failure storage succeeds.
- Keep one terminal webhook failure per receiver idempotency key.
- Reject poll votes with no matching options while preserving explicit vote clearing.
-- Read the duration and waveform of an outgoing Baileys audio file in a separate process, so a crafted file can no longer abort or exhaust the gateway.
+- Read the duration of an outgoing Baileys audio file in a separate process and a voice note's waveform through ffmpeg, status voice notes included, so a crafted file can no longer abort or exhaust the gateway.
- Bound `GET /api/search` to 1000 characters and 32 distinct words and run a repeated word once, so one request can no longer hold the event loop on SQLite for seconds.
### Documentation
@@ -148,6 +148,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- A storage import stops at `STORAGE_IMPORT_MAX_TOTAL_BYTES` (default 10 GiB); raise it before importing a larger export.
- Creating or updating an API key with more than 2000 `allowedChats` entries answers `400`; a stored longer list keeps working until it is edited.
- `GET /api/search` answers `400` for a `q` over 1000 characters or with more than 32 distinct words; a repeated word counts once.
+- Without an ffmpeg binary (`FFMPEG_PATH`), a Baileys voice note goes out with flat waveform bars instead of its real ones; the official image ships ffmpeg.
- With `DATABASE_SSL=true`, the PostgreSQL certificate is now verified against `DATABASE_HOST` even when it is an IP address (`pg` 8.23.1). A certificate that names only `localhost` stops matching a connection by IP: use the certificate's DNS name as the host or add the IP to its subject alternative names. `DATABASE_SSL_REJECT_UNAUTHORIZED=false` remains the escape hatch.
- The first boot builds three indexes on `messages` (`IDX_messages_session_timestamp_id`, `IDX_messages_session_chat_timestamp_id` and `IDX_messages_session_direction_timestamp_id`) not concurrently; on a large table boot takes longer, on PostgreSQL writes to that table wait until each build finishes, and the table with its indexes grows by about 240 MB per million messages.
- Baileys answers `400` for an animated sticker with more than 500 frames.
diff --git a/docs/06-api-specification.md b/docs/06-api-specification.md
index 805610466..8f4f84f97 100644
--- a/docs/06-api-specification.md
+++ b/docs/06-api-specification.md
@@ -1963,7 +1963,7 @@ Send a video (by URL or base64) with an optional caption. Uses the same `SendMed
#### POST /api/sessions/:sessionId/messages/send-audio
-Send an audio message (by URL or base64). Uses `SendAudioMessageDto`. A `caption` is accepted by the DTO but not persisted for audio. Set `ptt: true` to send a real WhatsApp **voice note** (microphone bubble + waveform) instead of a plain audio file. `ptt` is a JSON boolean, exclusive to this endpoint and to `audio` items of `send-bulk`, and — because voice notes require `audio/ogg; codecs=opus` — the server defaults the mimetype to that when you set `ptt` without one; for reliable playback (especially on the Baileys engine, which does not transcode) supply OGG/Opus bytes. A `ptt` voice note is stored as message `type: "voice"`.
+Send an audio message (by URL or base64). Uses `SendAudioMessageDto`. A `caption` is accepted by the DTO but not persisted for audio. Set `ptt: true` to send a real WhatsApp **voice note** (microphone bubble + waveform) instead of a plain audio file. `ptt` is a JSON boolean, exclusive to this endpoint and to `audio` items of `send-bulk`, and — because voice notes require `audio/ogg; codecs=opus` — the server defaults the mimetype to that when you set `ptt` without one; for reliable playback (especially on the Baileys engine, which does not transcode) supply OGG/Opus bytes. A `ptt` voice note is stored as message `type: "voice"`. On the Baileys engine its waveform is read through ffmpeg (`FFMPEG_PATH`, also with media conversion disabled), at most the first 35 minutes of it, and a note whose waveform cannot be read goes out with flat bars; the duration is read in a separate process and is `0` when it cannot be read.
**Auth:** API key (OPERATOR)
diff --git a/src/engine/adapters/baileys-audio-probe.spec.ts b/src/engine/adapters/baileys-audio-probe.spec.ts
index b115d6b02..5f8d92339 100644
--- a/src/engine/adapters/baileys-audio-probe.spec.ts
+++ b/src/engine/adapters/baileys-audio-probe.spec.ts
@@ -1,4 +1,9 @@
-import { probeAudio, WAVEFORM_BARS } from './baileys-audio-probe';
+import { describeAudio, WAVEFORM_BARS } from './baileys-audio-probe';
+import { runFfmpeg } from '../../modules/media/ffmpeg';
+
+// ffmpeg is the one dependency of the waveform; the duration read runs the real child process.
+jest.mock('../../modules/media/ffmpeg', () => ({ runFfmpeg: jest.fn() }));
+const ffmpeg = runFfmpeg as jest.Mock;
const u32be = (value: number): Buffer => {
const out = Buffer.alloc(4);
@@ -17,7 +22,7 @@ function oversizedAiff(): Buffer {
return Buffer.concat([Buffer.from('FORM'), u32be(body.length), body]);
}
-/** Unsigned 8-bit mono PCM at 8 kHz: a square wave, so every stretch of it is as loud as any other. */
+/** Unsigned 8-bit mono PCM at 8 kHz, a square wave: music-metadata reads its duration from the header. */
function wav(seconds: number): Buffer {
const rate = 8000;
const samples = Buffer.alloc(rate * seconds);
@@ -38,34 +43,93 @@ function wav(seconds: number): Buffer {
return Buffer.concat([header, samples]);
}
-describe('probeAudio', () => {
- // Each probe is a Node process that loads a parser, and a decoder for a voice note.
+/** What ffmpeg writes with `-ac 1 -ar 8000 -f s16le`: `count` samples, each `level(index)` high. */
+function pcm(count: number, level: (index: number) => number): Buffer {
+ const out = Buffer.alloc(count * 2);
+ for (let i = 0; i < count; i++) out.writeInt16LE(level(i), i * 2);
+ return out;
+}
+
+describe('describeAudio', () => {
+ // The duration read is a Node process that loads a parser.
jest.setTimeout(60_000);
- it('reads the duration of an audio file', async () => {
- const probe = await probeAudio(wav(3), false);
+ beforeEach(() => ffmpeg.mockReset());
+
+ it('reads the duration of an audio file and decodes nothing for a plain audio file', async () => {
+ const described = await describeAudio(wav(3), false);
+
+ expect(described).toEqual({ seconds: 3 });
+ expect(ffmpeg).not.toHaveBeenCalled();
+ });
+
+ it('builds a voice note waveform of 64 bars, the loudest scaled to 100, from ffmpeg samples', async () => {
+ // Quiet first half, loud second half: the bars must follow the signal, not be flat.
+ const samples = 8000 * 3;
+ ffmpeg.mockResolvedValue(pcm(samples, i => (i < samples / 2 ? 1000 : 8000)));
+
+ const described = await describeAudio(wav(3), true);
+
+ expect(described.seconds).toBe(3);
+ const bars = described.waveform as Uint8Array;
+ expect(bars).toHaveLength(WAVEFORM_BARS);
+ expect(Math.max(...bars)).toBe(100);
+ expect(bars[0]).toBe(12); // 1000 / 8000
+ expect(bars[WAVEFORM_BARS - 1]).toBe(100);
+ // Decoded as 8 kHz mono signed 16-bit and stopped at a fixed size, whatever the file claims to hold.
+ expect(ffmpeg).toHaveBeenCalledWith(
+ expect.any(Buffer),
+ 'bin',
+ 'raw',
+ expect.arrayContaining(['-ac', '1', '-ar', '8000', '-f', 's16le']),
+ expect.objectContaining({ maxOutputBytes: 32 * 1024 * 1024 }),
+ );
+ });
+
+ it.each([
+ ['ffmpeg cannot run or refuses the file', () => ffmpeg.mockRejectedValue(new Error('no ffmpeg'))],
+ ['the decoded audio is silent', () => ffmpeg.mockResolvedValue(pcm(8000, () => 0))],
+ ['the decoded audio is shorter than one sample per bar', () => ffmpeg.mockResolvedValue(pcm(10, () => 500))],
+ ])('sends flat bars when %s', async (_name, arrange) => {
+ arrange();
+
+ const described = await describeAudio(wav(3), true);
+
+ expect(described.seconds).toBe(3);
+ expect(described.waveform).toEqual(new Uint8Array(WAVEFORM_BARS).fill(40));
+ });
- expect(probe.seconds).toBe(3);
- expect(probe.waveform).toBeUndefined();
+ it('takes the duration from the decoded samples when the parser could not read it', async () => {
+ ffmpeg.mockResolvedValue(pcm(8000 * 5, i => (i % 2 === 0 ? 3000 : -3000)));
+
+ const described = await describeAudio(Buffer.from('not audio at all'), true);
+
+ expect(described.seconds).toBe(5);
});
- it('reads the waveform of a voice note as 64 bars scaled so the loudest is 100', async () => {
- const probe = await probeAudio(wav(3), true);
+ it('does not trust a duration from samples that ffmpeg was cut off at', async () => {
+ ffmpeg.mockResolvedValue(pcm((32 * 1024 * 1024) / 2, i => (i % 2 === 0 ? 3000 : -3000)));
+
+ const described = await describeAudio(Buffer.from('not audio at all'), true);
- expect(probe.seconds).toBe(3);
- expect(probe.waveform).toBeInstanceOf(Uint8Array);
- expect(probe.waveform).toHaveLength(WAVEFORM_BARS);
- expect(Math.max(...(probe.waveform as Uint8Array))).toBe(100);
+ expect(described.seconds).toBe(0);
});
- it('answers nothing for bytes no parser reads, rather than rejecting', async () => {
- await expect(probeAudio(Buffer.from('not audio at all'), true)).resolves.toEqual({});
+ it('answers a zero duration and flat bars for bytes nothing reads, rather than rejecting', async () => {
+ ffmpeg.mockRejectedValue(new Error('Invalid data found when processing input'));
+
+ await expect(describeAudio(Buffer.from('not audio at all'), true)).resolves.toEqual({
+ seconds: 0,
+ waveform: new Uint8Array(WAVEFORM_BARS).fill(40),
+ });
});
it('survives a file that aborts the process of a reader that opens it', async () => {
// Opened as a file, music-metadata dies on this one with SIGABRT in node::fs::Read. Whatever the
// child does with it, the caller must get an answer and keep running.
- await expect(probeAudio(oversizedAiff(), false)).resolves.toEqual({});
- await expect(probeAudio(oversizedAiff(), true)).resolves.toEqual({});
+ ffmpeg.mockRejectedValue(new Error('refused'));
+
+ await expect(describeAudio(oversizedAiff(), false)).resolves.toEqual({ seconds: 0 });
+ await expect(describeAudio(oversizedAiff(), true)).resolves.toMatchObject({ seconds: 0 });
});
});
diff --git a/src/engine/adapters/baileys-audio-probe.ts b/src/engine/adapters/baileys-audio-probe.ts
index de84c4e47..7e918c6b0 100644
--- a/src/engine/adapters/baileys-audio-probe.ts
+++ b/src/engine/adapters/baileys-audio-probe.ts
@@ -1,136 +1,169 @@
import { spawn } from 'node:child_process';
import { join } from 'node:path';
import { ConcurrencyLimiter } from '../../common/utils/concurrency-limiter';
+import { runFfmpeg } from '../../modules/media/ffmpeg';
-/** What could be read from an outgoing audio file; a field is absent when it could not be read. */
-export interface AudioProbe {
- /** Playing time in whole seconds. */
- seconds?: number;
- /** The amplitude bars WhatsApp draws on a voice note, 0 to 100 each. */
+/** What an outgoing audio message must carry so that Baileys reads nothing from the file itself. */
+export interface AudioDescription {
+ /** Playing time in whole seconds; 0 when it could not be read. */
+ seconds: number;
+ /** The amplitude bars WhatsApp draws on a voice note, 0 to 100 each; only on a voice note. */
waveform?: Uint8Array;
}
/** Bars on a voice note, the count Baileys' own waveform uses. */
export const WAVEFORM_BARS = 64;
+/** Height of every bar of a voice note whose waveform could not be read: visible, and not a loud one. */
+const FLAT_BAR = 40;
-/** Wall-clock ceiling for one probe; the child is killed past it. */
-const PROBE_TIMEOUT_MS = 30_000;
-/** What the child may print: a duration and 64 small integers, a line each. */
-const MAX_OUTPUT_CHARS = 4096;
-/** Probes running at once. Each is a whole Node process with a decoder loaded in it. */
-const probes = new ConcurrencyLimiter(2);
+/** Wall-clock ceiling for the duration read; the child is killed past it. */
+const DURATION_TIMEOUT_MS = 30_000;
+/** What the child may print: one small JSON line. */
+const MAX_OUTPUT_CHARS = 1024;
+/**
+ * The waveform is read from 8 kHz mono 16-bit samples, 16 KB for each second of audio, and ffmpeg is
+ * stopped at this many bytes, about 35 minutes. That is the most memory the bars are ever computed
+ * from, whatever the file claims to hold. A longer note goes out with flat bars.
+ */
+const WAVEFORM_MAX_PCM_BYTES = 32 * 1024 * 1024;
+const WAVEFORM_TIMEOUT_MS = 30_000;
+const PCM_RATE = 8000;
+/** Reads running at once. Each is a whole child process. */
+const reads = new ConcurrencyLimiter(2);
/** The directory holding node_modules, whether this file runs from src/ or from dist/. */
const APP_ROOT = join(__dirname, '..', '..', '..');
/**
- * Run in the child, with the file on stdin. The same two reads Baileys makes for an outgoing audio
- * message, music-metadata for the duration and audio-decode for the voice-note bars, with the same
- * bar arithmetic, so a file both can read gets the same answer here.
+ * Run in a child, with the file on stdin: the duration the way Baileys reads it, with music-metadata.
+ * A parser that aborts the process on a crafted file costs the child, not the gateway.
*/
-const PROBE_SCRIPT = `
-const bars = ${WAVEFORM_BARS};
-const wantWaveform = process.argv[1] === 'ptt';
+const DURATION_SCRIPT = `
const chunks = [];
process.stdin.on('data', chunk => chunks.push(chunk));
process.stdin.on('end', async () => {
- const input = Buffer.concat(chunks);
try {
const mm = await import('music-metadata');
- const metadata = await mm.parseBuffer(input, undefined, { duration: true });
+ const metadata = await mm.parseBuffer(Buffer.concat(chunks), undefined, { duration: true });
const duration = metadata.format.duration;
if (typeof duration === 'number' && Number.isFinite(duration) && duration >= 0) {
process.stdout.write(JSON.stringify({ seconds: Math.floor(duration) }) + '\\n');
}
} catch {}
- if (wantWaveform) {
- try {
- const { default: decode } = await import('audio-decode');
- const raw = (await decode(input)).getChannelData(0);
- const block = Math.floor(raw.length / bars);
- const averages = [];
- for (let i = 0; i < bars; i++) {
- let sum = 0;
- for (let j = 0; j < block; j++) sum += Math.abs(raw[block * i + j]);
- averages.push(sum / block);
- }
- const peak = Math.max(...averages);
- if (peak > 0 && Number.isFinite(peak)) {
- process.stdout.write(JSON.stringify({ waveform: averages.map(value => Math.floor((100 * value) / peak)) }) + '\\n');
- }
- } catch {}
- }
});
`;
-function parseProbe(output: string): AudioProbe {
- const probe: AudioProbe = {};
- for (const line of output.split('\n')) {
- let value: { seconds?: unknown; waveform?: unknown };
- try {
- value = JSON.parse(line) as { seconds?: unknown; waveform?: unknown };
- } catch {
- continue;
- }
- if (typeof value.seconds === 'number' && Number.isInteger(value.seconds) && value.seconds >= 0) {
- probe.seconds = value.seconds;
- }
- const bars = value.waveform;
- if (
- Array.isArray(bars) &&
- bars.length === WAVEFORM_BARS &&
- bars.every(bar => typeof bar === 'number' && Number.isInteger(bar) && bar >= 0 && bar <= 100)
- ) {
- probe.waveform = Uint8Array.from(bars as number[]);
- }
- }
- return probe;
-}
-
-function runProbe(data: Buffer, ptt: boolean): Promise<AudioProbe> {
+function readDuration(data: Buffer): Promise<number | undefined> {
return new Promise(resolve => {
let settled = false;
let output = '';
- const child = spawn(process.execPath, ['-e', PROBE_SCRIPT, ptt ? 'ptt' : 'audio'], {
+ const child = spawn(process.execPath, ['-e', DURATION_SCRIPT], {
cwd: APP_ROOT,
env: { PATH: process.env.PATH ?? '' },
stdio: ['pipe', 'pipe', 'ignore'],
});
- const finish = (probe: AudioProbe): void => {
+ const finish = (seconds: number | undefined): void => {
if (settled) return;
settled = true;
clearTimeout(timer);
- resolve(probe);
+ resolve(seconds);
+ };
+ const parse = (): number | undefined => {
+ try {
+ const { seconds } = JSON.parse(output.trim()) as { seconds?: unknown };
+ return typeof seconds === 'number' && Number.isInteger(seconds) && seconds >= 0 ? seconds : undefined;
+ } catch {
+ return undefined;
+ }
};
const timer = setTimeout(() => {
child.kill('SIGKILL');
- finish(parseProbe(output));
- }, PROBE_TIMEOUT_MS);
+ finish(undefined);
+ }, DURATION_TIMEOUT_MS);
child.stdout.setEncoding('utf8');
child.stdout.on('data', (chunk: string) => {
output += chunk;
if (output.length > MAX_OUTPUT_CHARS) {
child.kill('SIGKILL');
- finish({});
+ finish(undefined);
}
});
- child.on('error', () => finish({}));
- // Whatever the child printed before it died still counts: the duration is printed before the decode.
- child.on('close', () => finish(parseProbe(output)));
+ child.on('error', () => finish(undefined));
+ child.on('close', () => finish(parse()));
// The child may end before it has read everything it was sent.
child.stdin.on('error', () => undefined);
child.stdin.end(data);
});
}
+/** Bars of a mono 16-bit little-endian sample stream, scaled so the loudest is 100, and the sample count. */
+function barsOf(pcm: Buffer): { waveform?: Uint8Array; samples: number } {
+ const samples = Math.floor(pcm.length / 2);
+ const view = new Int16Array(pcm.buffer.slice(pcm.byteOffset, pcm.byteOffset + samples * 2));
+ const block = Math.floor(samples / WAVEFORM_BARS);
+ if (block === 0) return { samples };
+ const averages: number[] = [];
+ for (let bar = 0; bar < WAVEFORM_BARS; bar++) {
+ let sum = 0;
+ for (let at = block * bar, end = at + block; at < end; at++) sum += Math.abs(view[at]);
+ averages.push(sum / block);
+ }
+ const peak = Math.max(...averages);
+ if (!(peak > 0)) return { samples };
+ return { waveform: Uint8Array.from(averages, average => Math.floor((100 * average) / peak)), samples };
+}
+
+/**
+ * Decode through ffmpeg, which streams and is stopped at a fixed output size, instead of through a
+ * decoder that holds every sample in memory: a few hundred kilobytes of Opus packets decode into
+ * gigabytes there, and a container that kills its whole cgroup on out-of-memory takes every session
+ * with it. Without an ffmpeg binary there are no bars, and the note goes out with flat ones.
+ */
+async function readWaveform(data: Buffer): Promise<{ waveform?: Uint8Array; samples?: number }> {
+ try {
+ const pcm = await runFfmpeg(
+ data,
+ 'bin',
+ 'raw',
+ ['-vn', '-map', '0:a:0', '-ac', '1', '-ar', String(PCM_RATE), '-f', 's16le'],
+ {
+ ffmpegPath: process.env.FFMPEG_PATH || 'ffmpeg',
+ timeoutMs: WAVEFORM_TIMEOUT_MS,
+ maxOutputBytes: WAVEFORM_MAX_PCM_BYTES,
+ },
+ );
+ return barsOf(pcm);
+ } catch {
+ return {};
+ }
+}
+
/**
- * Read the duration, and for a voice note the waveform, of an outgoing audio file in a separate process.
+ * The duration of an outgoing audio file and, for a voice note, its waveform, read outside this process.
*
- * Baileys makes both reads in the gateway's own process when the message does not carry them. They run
- * parsers and decoders over bytes a caller chose: a 30-byte AIFF file aborts the process in
- * music-metadata's file read, and a few hundred kilobytes of Opus packets decode into gigabytes of
- * samples. Neither is an error the caller of the gateway can catch. Here a file that does either
- * costs the child, and the answer is simply missing. Never rejects.
+ * Baileys makes both reads in the gateway's own process when the message does not carry them, with a
+ * parser and a decoder run over bytes a caller chose: a 30-byte AIFF file aborts the process in
+ * music-metadata's file read, and a small Opus file decodes into gigabytes of samples. Neither is an
+ * error a request can catch. Every outgoing audio content is built from this answer instead, so
+ * Baileys is never left to read the file: what could not be read goes out as a zero duration and flat
+ * bars. Never rejects.
*/
-export function probeAudio(data: Buffer, ptt: boolean): Promise<AudioProbe> {
- return probes.run(() => runProbe(data, ptt)).catch(() => ({}));
+export async function describeAudio(data: Buffer, ptt: boolean): Promise<AudioDescription> {
+ const found = await reads
+ .run(async () => {
+ const [seconds, wave] = await Promise.all([
+ readDuration(data),
+ ptt ? readWaveform(data) : Promise.resolve<{ waveform?: Uint8Array; samples?: number }>({}),
+ ]);
+ // The decoded samples give the duration when the parser could not, as long as they were not cut off.
+ const decoded =
+ seconds === undefined && wave.samples !== undefined && wave.samples * 2 < WAVEFORM_MAX_PCM_BYTES
+ ? Math.floor(wave.samples / PCM_RATE)
+ : undefined;
+ return { seconds: seconds ?? decoded, waveform: wave.waveform };
+ })
+ .catch(() => ({ seconds: undefined as number | undefined, waveform: undefined as Uint8Array | undefined }));
+ return {
+ seconds: found.seconds ?? 0,
+ ...(ptt ? { waveform: found.waveform ?? new Uint8Array(WAVEFORM_BARS).fill(FLAT_BAR) } : {}),
+ };
}
diff --git a/src/engine/adapters/baileys-messaging.ts b/src/engine/adapters/baileys-messaging.ts
index 7c1a8e5ba..334eee28c 100644
--- a/src/engine/adapters/baileys-messaging.ts
+++ b/src/engine/adapters/baileys-messaging.ts
@@ -44,10 +44,7 @@ import { EngineNotSentError } from '../../common/errors/engine-not-sent.error';
import { parseWaId, userPart } from '../identity/wa-id';
import { BAILEYS_QUERY_BUDGET_MS, withQueryDeadline } from './baileys-query-deadline';
import { ConcurrencyLimiter } from '../../common/utils/concurrency-limiter';
-import { probeAudio, WAVEFORM_BARS } from './baileys-audio-probe';
-
-/** Height of every bar of a voice note whose waveform could not be read: visible, and not a loud one. */
-const FLAT_WAVEFORM_BAR = 40;
+import { describeAudio } from './baileys-audio-probe';
/**
* Messaging-domain operations extracted from BaileysAdapter. The adapter keeps the public
@@ -464,25 +461,15 @@ export class BaileysMessaging {
this.host.ensureReady();
const { data, mimetype } = await resolveMediaBuffer(media, this.host.sessionProxyUrl(), 'audio/mpeg');
const ptt = media.ptt ?? false;
- // Baileys reads the duration, and a voice note's waveform, in this process whenever the content does
- // not carry them, which a crafted file turns into an abort or an out-of-memory kill of every session.
- // Both are always given, from a child process, so Baileys never reads the file itself: a file the
- // child cannot read sends with a zero duration and flat bars, as one Baileys could not read did.
- const probe = await probeAudio(data, ptt);
return this.sendContent(
chatId,
// Audio carries no caption, so a mention here tags the recipient through contextInfo without
// visible @text. It is still forwarded: the route accepts `mentions` (SendAudioMessageDto
// extends SendMediaMessageDto) and whatsapp-web.js sends it, so dropping it here made the same
// request notify participants on one engine and silently not on the other.
- {
- audio: data,
- mimetype,
- ptt,
- seconds: probe.seconds ?? 0,
- ...(ptt ? { waveform: probe.waveform ?? new Uint8Array(WAVEFORM_BARS).fill(FLAT_WAVEFORM_BAR) } : {}),
- ...this.withMentions(media.mentions),
- },
+ // The duration and a voice note's waveform come from describeAudio, never from Baileys, which
+ // would read the file in this process.
+ { audio: data, mimetype, ptt, ...(await describeAudio(data, ptt)), ...this.withMentions(media.mentions) },
await this.quoteOption(media.quotedMessageId),
);
}
diff --git a/src/engine/adapters/baileys-status.ts b/src/engine/adapters/baileys-status.ts
index 7a81046dc..adc55f736 100644
--- a/src/engine/adapters/baileys-status.ts
+++ b/src/engine/adapters/baileys-status.ts
@@ -2,6 +2,7 @@ import type { AnyMessageContent, WAMessage, WASocket } from '@whiskeysockets/bai
import { MediaInput, StatusPostOptions, StatusResult } from '../interfaces/whatsapp-engine.interface';
import { BadRequestException } from '@nestjs/common';
import { resolveMediaBuffer } from './baileys-messaging';
+import { describeAudio } from './baileys-audio-probe';
import { EngineRefusedError } from '../../common/errors/engine-refused.error';
import { EngineNotReadyError } from '../../common/errors/engine-not-ready.error';
@@ -93,7 +94,7 @@ export class BaileysStatus {
? { image: data, caption: options.caption, mimetype }
: kind === 'video'
? { video: data, caption: options.caption, mimetype }
- : { audio: data, mimetype, ptt: true };
+ : { audio: data, mimetype, ptt: true, ...(await describeAudio(data, true)) };
return this.postStatus(content, options);
}
diff --git a/src/engine/adapters/baileys.adapter.spec.ts b/src/engine/adapters/baileys.adapter.spec.ts
index 4364ea52f..ea936ba9c 100644
--- a/src/engine/adapters/baileys.adapter.spec.ts
+++ b/src/engine/adapters/baileys.adapter.spec.ts
@@ -10,8 +10,7 @@ jest.mock('../../common/media/load-remote-media', () => ({
// The probe runs a child process; its own spec drives the real one.
jest.mock('./baileys-audio-probe', () => ({
- WAVEFORM_BARS: 64,
- probeAudio: jest.fn(() => Promise.resolve({})),
+ describeAudio: jest.fn(() => Promise.resolve({ seconds: 0 })),
}));
// A fake Baileys socket: an event emitter wearing the methods the adapter calls.
@@ -185,7 +184,7 @@ import { EngineTransportError } from '../../common/errors/engine-transport.error
import { LidNotMappedError } from '../../common/errors/lid-not-mapped.error';
import { countsTowardSendBreaker, sentNothing } from '../../modules/message/send-pacing.service';
import { loadRemoteMediaBuffer } from '../../common/media/load-remote-media';
-import { probeAudio } from './baileys-audio-probe';
+import { describeAudio } from './baileys-audio-probe';
import * as safeLinkPreview from './safe-link-preview';
const fakeStore = {
@@ -4215,14 +4214,14 @@ describe('BaileysAdapter media sends', () => {
it('sendAudioMessage hands Baileys the duration and waveform read in the child, so it reads nothing itself', async () => {
const waveform = Uint8Array.from({ length: 64 }, (_, bar) => bar);
- (probeAudio as jest.Mock).mockResolvedValueOnce({ seconds: 7, waveform });
+ (describeAudio as jest.Mock).mockResolvedValueOnce({ seconds: 7, waveform });
const adapter = await ready();
await adapter.sendAudioMessage('[email protected]', {
mimetype: 'audio/ogg; codecs=opus',
data: Buffer.from([1]),
ptt: true,
});
- expect(probeAudio).toHaveBeenLastCalledWith(Buffer.from([1]), true);
+ expect(describeAudio).toHaveBeenLastCalledWith(Buffer.from([1]), true);
expect(fakeSock.sendMessage).toHaveBeenCalledWith('[email protected]', {
audio: Buffer.from([1]),
mimetype: 'audio/ogg; codecs=opus',
@@ -4258,13 +4257,11 @@ describe('BaileysAdapter media sends', () => {
data: Buffer.from([1]),
ptt: true,
});
- // A file the child could not read still goes out with both fields set, flat bars for a voice note.
expect(fakeSock.sendMessage).toHaveBeenCalledWith('[email protected]', {
audio: Buffer.from([1]),
mimetype: 'audio/ogg; codecs=opus',
ptt: true,
seconds: 0,
- waveform: new Uint8Array(64).fill(40),
});
});
@@ -8472,6 +8469,23 @@ describe('BaileysAdapter voice status', () => {
expect(options.statusJidList).toEqual(['[email protected]']);
});
+ // Baileys reads the file itself, in this process, for whichever of the two the content leaves out.
+ it('hands Baileys the duration and waveform read in the child, as a sent voice note does', async () => {
+ const waveform = Uint8Array.from({ length: 64 }, (_, bar) => bar);
+ (describeAudio as jest.Mock).mockResolvedValueOnce({ seconds: 4, waveform });
+ const adapter = await ready();
+
+ await adapter.postVoiceStatus(
+ { mimetype: 'audio/ogg; codecs=opus', data: Buffer.from('audio').toString('base64') },
+ { recipients: ['[email protected]'] },
+ );
+
+ expect(describeAudio).toHaveBeenLastCalledWith(Buffer.from('audio'), true);
+ const [, content] = fakeSock.sendMessage.mock.calls[0] as [string, { seconds?: number; waveform?: Uint8Array }];
+ expect(content.seconds).toBe(4);
+ expect(content.waveform).toBe(waveform);
+ });
+
// WhatsApp has nowhere to render a caption on a status voice note.
it('carries no caption', async () => {
const adapter = await ready();