Free, Open Source, Self-Hosted WhatsApp API Gateway
TypeScript apibotgatewayself-hostedwhatsappwhatsapp-apiwhatsapp-automationwhatsapp-botwhatsapp-gatewaywhatsapp-gateway-apiwhatsapp-webwhatsapp-web-api
fix(baileys): decode a voice note's waveform through ffmpeg
Yudhi Armyndharis1d3a3f5f476c
diff --git a/.env.example b/.env.exampleindex 8e49e458f..5bf6d7ad6 100644--- a/.env.example+++ b/.env.example@@ -642,7 +642,7 @@ WEBHOOK_SSRF_PROTECT=true# Media conversion. Each number below must be a positive integer, or the boot fails, and the timeout# no greater than 2147483647 ms (Node's timer ceiling), even with conversion disabled.# MEDIA_CONVERSION_ENABLED=false # default false-# FFMPEG_PATH=ffmpeg # absolute path, for a binary outside PATH+# FFMPEG_PATH=ffmpeg # absolute path, for a binary outside PATH; also reads a Baileys voice note's waveform# MEDIA_CONVERSION_TIMEOUT_MS=60000 # kill one conversion after this long (default 60s)# MEDIA_CONVERSION_MAX_OUTPUT_BYTES=52428800 # cap the CONVERTED bytes (default 50 MiB)# MEDIA_CONVERSION_CONCURRENCY=2 # max concurrent ffmpeg processes; a short queuediff --git a/CHANGELOG.md b/CHANGELOG.mdindex 89cfdbf8d..f793544b0 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -111,7 +111,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0- Preserve webhook outbox payloads until delivery or durable failure storage succeeds.- Keep one terminal webhook failure per receiver idempotency key.- Reject poll votes with no matching options while preserving explicit vote clearing.-- Read the duration and waveform of an outgoing Baileys audio file in a separate process, so a crafted file can no longer abort or exhaust the gateway.+- Read the duration of an outgoing Baileys audio file in a separate process and a voice note's waveform through ffmpeg, status voice notes included, so a crafted file can no longer abort or exhaust the gateway.- Bound `GET /api/search` to 1000 characters and 32 distinct words and run a repeated word once, so one request can no longer hold the event loop on SQLite for seconds.### Documentation@@ -148,6 +148,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0- A storage import stops at `STORAGE_IMPORT_MAX_TOTAL_BYTES` (default 10 GiB); raise it before importing a larger export.- Creating or updating an API key with more than 2000 `allowedChats` entries answers `400`; a stored longer list keeps working until it is edited.- `GET /api/search` answers `400` for a `q` over 1000 characters or with more than 32 distinct words; a repeated word counts once.+- Without an ffmpeg binary (`FFMPEG_PATH`), a Baileys voice note goes out with flat waveform bars instead of its real ones; the official image ships ffmpeg.- With `DATABASE_SSL=true`, the PostgreSQL certificate is now verified against `DATABASE_HOST` even when it is an IP address (`pg` 8.23.1). A certificate that names only `localhost` stops matching a connection by IP: use the certificate's DNS name as the host or add the IP to its subject alternative names. `DATABASE_SSL_REJECT_UNAUTHORIZED=false` remains the escape hatch.- The first boot builds three indexes on `messages` (`IDX_messages_session_timestamp_id`, `IDX_messages_session_chat_timestamp_id` and `IDX_messages_session_direction_timestamp_id`) not concurrently; on a large table boot takes longer, on PostgreSQL writes to that table wait until each build finishes, and the table with its indexes grows by about 240 MB per million messages.- Baileys answers `400` for an animated sticker with more than 500 frames.diff --git a/docs/06-api-specification.md b/docs/06-api-specification.mdindex 805610466..8f4f84f97 100644--- a/docs/06-api-specification.md+++ b/docs/06-api-specification.md@@ -1963,7 +1963,7 @@ Send a video (by URL or base64) with an optional caption. Uses the same `SendMed#### POST /api/sessions/:sessionId/messages/send-audio-Send an audio message (by URL or base64). Uses `SendAudioMessageDto`. A `caption` is accepted by the DTO but not persisted for audio. Set `ptt: true` to send a real WhatsApp **voice note** (microphone bubble + waveform) instead of a plain audio file. `ptt` is a JSON boolean, exclusive to this endpoint and to `audio` items of `send-bulk`, and — because voice notes require `audio/ogg; codecs=opus` — the server defaults the mimetype to that when you set `ptt` without one; for reliable playback (especially on the Baileys engine, which does not transcode) supply OGG/Opus bytes. A `ptt` voice note is stored as message `type: "voice"`.+Send an audio message (by URL or base64). Uses `SendAudioMessageDto`. A `caption` is accepted by the DTO but not persisted for audio. Set `ptt: true` to send a real WhatsApp **voice note** (microphone bubble + waveform) instead of a plain audio file. `ptt` is a JSON boolean, exclusive to this endpoint and to `audio` items of `send-bulk`, and — because voice notes require `audio/ogg; codecs=opus` — the server defaults the mimetype to that when you set `ptt` without one; for reliable playback (especially on the Baileys engine, which does not transcode) supply OGG/Opus bytes. A `ptt` voice note is stored as message `type: "voice"`. On the Baileys engine its waveform is read through ffmpeg (`FFMPEG_PATH`, also with media conversion disabled), at most the first 35 minutes of it, and a note whose waveform cannot be read goes out with flat bars; the duration is read in a separate process and is `0` when it cannot be read.**Auth:** API key (OPERATOR)diff --git a/src/engine/adapters/baileys-audio-probe.spec.ts b/src/engine/adapters/baileys-audio-probe.spec.tsindex b115d6b02..5f8d92339 100644--- a/src/engine/adapters/baileys-audio-probe.spec.ts+++ b/src/engine/adapters/baileys-audio-probe.spec.ts@@ -1,4 +1,9 @@-import { probeAudio, WAVEFORM_BARS } from './baileys-audio-probe';+import { describeAudio, WAVEFORM_BARS } from './baileys-audio-probe';+import { runFfmpeg } from '../../modules/media/ffmpeg';++// ffmpeg is the one dependency of the waveform; the duration read runs the real child process.+jest.mock('../../modules/media/ffmpeg', () => ({ runFfmpeg: jest.fn() }));+const ffmpeg = runFfmpeg as jest.Mock;const u32be = (value: number): Buffer => {const out = Buffer.alloc(4);@@ -17,7 +22,7 @@ function oversizedAiff(): Buffer {return Buffer.concat([Buffer.from('FORM'), u32be(body.length), body]);}-/** Unsigned 8-bit mono PCM at 8 kHz: a square wave, so every stretch of it is as loud as any other. */+/** Unsigned 8-bit mono PCM at 8 kHz, a square wave: music-metadata reads its duration from the header. */function wav(seconds: number): Buffer {const rate = 8000;const samples = Buffer.alloc(rate * seconds);@@ -38,34 +43,93 @@ function wav(seconds: number): Buffer {return Buffer.concat([header, samples]);}-describe('probeAudio', () => {- // Each probe is a Node process that loads a parser, and a decoder for a voice note.+/** What ffmpeg writes with `-ac 1 -ar 8000 -f s16le`: `count` samples, each `level(index)` high. */+function pcm(count: number, level: (index: number) => number): Buffer {+ const out = Buffer.alloc(count * 2);+ for (let i = 0; i < count; i++) out.writeInt16LE(level(i), i * 2);+ return out;+}++describe('describeAudio', () => {+ // The duration read is a Node process that loads a parser.jest.setTimeout(60_000);- it('reads the duration of an audio file', async () => {- const probe = await probeAudio(wav(3), false);+ beforeEach(() => ffmpeg.mockReset());++ it('reads the duration of an audio file and decodes nothing for a plain audio file', async () => {+ const described = await describeAudio(wav(3), false);++ expect(described).toEqual({ seconds: 3 });+ expect(ffmpeg).not.toHaveBeenCalled();+ });++ it('builds a voice note waveform of 64 bars, the loudest scaled to 100, from ffmpeg samples', async () => {+ // Quiet first half, loud second half: the bars must follow the signal, not be flat.+ const samples = 8000 * 3;+ ffmpeg.mockResolvedValue(pcm(samples, i => (i < samples / 2 ? 1000 : 8000)));++ const described = await describeAudio(wav(3), true);++ expect(described.seconds).toBe(3);+ const bars = described.waveform as Uint8Array;+ expect(bars).toHaveLength(WAVEFORM_BARS);+ expect(Math.max(...bars)).toBe(100);+ expect(bars[0]).toBe(12); // 1000 / 8000+ expect(bars[WAVEFORM_BARS - 1]).toBe(100);+ // Decoded as 8 kHz mono signed 16-bit and stopped at a fixed size, whatever the file claims to hold.+ expect(ffmpeg).toHaveBeenCalledWith(+ expect.any(Buffer),+ 'bin',+ 'raw',+ expect.arrayContaining(['-ac', '1', '-ar', '8000', '-f', 's16le']),+ expect.objectContaining({ maxOutputBytes: 32 * 1024 * 1024 }),+ );+ });++ it.each([+ ['ffmpeg cannot run or refuses the file', () => ffmpeg.mockRejectedValue(new Error('no ffmpeg'))],+ ['the decoded audio is silent', () => ffmpeg.mockResolvedValue(pcm(8000, () => 0))],+ ['the decoded audio is shorter than one sample per bar', () => ffmpeg.mockResolvedValue(pcm(10, () => 500))],+ ])('sends flat bars when %s', async (_name, arrange) => {+ arrange();++ const described = await describeAudio(wav(3), true);++ expect(described.seconds).toBe(3);+ expect(described.waveform).toEqual(new Uint8Array(WAVEFORM_BARS).fill(40));+ });- expect(probe.seconds).toBe(3);- expect(probe.waveform).toBeUndefined();+ it('takes the duration from the decoded samples when the parser could not read it', async () => {+ ffmpeg.mockResolvedValue(pcm(8000 * 5, i => (i % 2 === 0 ? 3000 : -3000)));++ const described = await describeAudio(Buffer.from('not audio at all'), true);++ expect(described.seconds).toBe(5);});- it('reads the waveform of a voice note as 64 bars scaled so the loudest is 100', async () => {- const probe = await probeAudio(wav(3), true);+ it('does not trust a duration from samples that ffmpeg was cut off at', async () => {+ ffmpeg.mockResolvedValue(pcm((32 * 1024 * 1024) / 2, i => (i % 2 === 0 ? 3000 : -3000)));++ const described = await describeAudio(Buffer.from('not audio at all'), true);- expect(probe.seconds).toBe(3);- expect(probe.waveform).toBeInstanceOf(Uint8Array);- expect(probe.waveform).toHaveLength(WAVEFORM_BARS);- expect(Math.max(...(probe.waveform as Uint8Array))).toBe(100);+ expect(described.seconds).toBe(0);});- it('answers nothing for bytes no parser reads, rather than rejecting', async () => {- await expect(probeAudio(Buffer.from('not audio at all'), true)).resolves.toEqual({});+ it('answers a zero duration and flat bars for bytes nothing reads, rather than rejecting', async () => {+ ffmpeg.mockRejectedValue(new Error('Invalid data found when processing input'));++ await expect(describeAudio(Buffer.from('not audio at all'), true)).resolves.toEqual({+ seconds: 0,+ waveform: new Uint8Array(WAVEFORM_BARS).fill(40),+ });});it('survives a file that aborts the process of a reader that opens it', async () => {// Opened as a file, music-metadata dies on this one with SIGABRT in node::fs::Read. Whatever the// child does with it, the caller must get an answer and keep running.- await expect(probeAudio(oversizedAiff(), false)).resolves.toEqual({});- await expect(probeAudio(oversizedAiff(), true)).resolves.toEqual({});+ ffmpeg.mockRejectedValue(new Error('refused'));++ await expect(describeAudio(oversizedAiff(), false)).resolves.toEqual({ seconds: 0 });+ await expect(describeAudio(oversizedAiff(), true)).resolves.toMatchObject({ seconds: 0 });});});diff --git a/src/engine/adapters/baileys-audio-probe.ts b/src/engine/adapters/baileys-audio-probe.tsindex de84c4e47..7e918c6b0 100644--- a/src/engine/adapters/baileys-audio-probe.ts+++ b/src/engine/adapters/baileys-audio-probe.ts@@ -1,136 +1,169 @@import { spawn } from 'node:child_process';import { join } from 'node:path';import { ConcurrencyLimiter } from '../../common/utils/concurrency-limiter';+import { runFfmpeg } from '../../modules/media/ffmpeg';-/** What could be read from an outgoing audio file; a field is absent when it could not be read. */-export interface AudioProbe {- /** Playing time in whole seconds. */- seconds?: number;- /** The amplitude bars WhatsApp draws on a voice note, 0 to 100 each. */+/** What an outgoing audio message must carry so that Baileys reads nothing from the file itself. */+export interface AudioDescription {+ /** Playing time in whole seconds; 0 when it could not be read. */+ seconds: number;+ /** The amplitude bars WhatsApp draws on a voice note, 0 to 100 each; only on a voice note. */waveform?: Uint8Array;}/** Bars on a voice note, the count Baileys' own waveform uses. */export const WAVEFORM_BARS = 64;+/** Height of every bar of a voice note whose waveform could not be read: visible, and not a loud one. */+const FLAT_BAR = 40;-/** Wall-clock ceiling for one probe; the child is killed past it. */-const PROBE_TIMEOUT_MS = 30_000;-/** What the child may print: a duration and 64 small integers, a line each. */-const MAX_OUTPUT_CHARS = 4096;-/** Probes running at once. Each is a whole Node process with a decoder loaded in it. */-const probes = new ConcurrencyLimiter(2);+/** Wall-clock ceiling for the duration read; the child is killed past it. */+const DURATION_TIMEOUT_MS = 30_000;+/** What the child may print: one small JSON line. */+const MAX_OUTPUT_CHARS = 1024;+/**+ * The waveform is read from 8 kHz mono 16-bit samples, 16 KB for each second of audio, and ffmpeg is+ * stopped at this many bytes, about 35 minutes. That is the most memory the bars are ever computed+ * from, whatever the file claims to hold. A longer note goes out with flat bars.+ */+const WAVEFORM_MAX_PCM_BYTES = 32 * 1024 * 1024;+const WAVEFORM_TIMEOUT_MS = 30_000;+const PCM_RATE = 8000;+/** Reads running at once. Each is a whole child process. */+const reads = new ConcurrencyLimiter(2);/** The directory holding node_modules, whether this file runs from src/ or from dist/. */const APP_ROOT = join(__dirname, '..', '..', '..');/**- * Run in the child, with the file on stdin. The same two reads Baileys makes for an outgoing audio- * message, music-metadata for the duration and audio-decode for the voice-note bars, with the same- * bar arithmetic, so a file both can read gets the same answer here.+ * Run in a child, with the file on stdin: the duration the way Baileys reads it, with music-metadata.+ * A parser that aborts the process on a crafted file costs the child, not the gateway.*/-const PROBE_SCRIPT = `-const bars = ${WAVEFORM_BARS};-const wantWaveform = process.argv[1] === 'ptt';+const DURATION_SCRIPT = `const chunks = [];process.stdin.on('data', chunk => chunks.push(chunk));process.stdin.on('end', async () => {- const input = Buffer.concat(chunks);try {const mm = await import('music-metadata');- const metadata = await mm.parseBuffer(input, undefined, { duration: true });+ const metadata = await mm.parseBuffer(Buffer.concat(chunks), undefined, { duration: true });const duration = metadata.format.duration;if (typeof duration === 'number' && Number.isFinite(duration) && duration >= 0) {process.stdout.write(JSON.stringify({ seconds: Math.floor(duration) }) + '\\n');}} catch {}- if (wantWaveform) {- try {- const { default: decode } = await import('audio-decode');- const raw = (await decode(input)).getChannelData(0);- const block = Math.floor(raw.length / bars);- const averages = [];- for (let i = 0; i < bars; i++) {- let sum = 0;- for (let j = 0; j < block; j++) sum += Math.abs(raw[block * i + j]);- averages.push(sum / block);- }- const peak = Math.max(...averages);- if (peak > 0 && Number.isFinite(peak)) {- process.stdout.write(JSON.stringify({ waveform: averages.map(value => Math.floor((100 * value) / peak)) }) + '\\n');- }- } catch {}- }});`;-function parseProbe(output: string): AudioProbe {- const probe: AudioProbe = {};- for (const line of output.split('\n')) {- let value: { seconds?: unknown; waveform?: unknown };- try {- value = JSON.parse(line) as { seconds?: unknown; waveform?: unknown };- } catch {- continue;- }- if (typeof value.seconds === 'number' && Number.isInteger(value.seconds) && value.seconds >= 0) {- probe.seconds = value.seconds;- }- const bars = value.waveform;- if (- Array.isArray(bars) &&- bars.length === WAVEFORM_BARS &&- bars.every(bar => typeof bar === 'number' && Number.isInteger(bar) && bar >= 0 && bar <= 100)- ) {- probe.waveform = Uint8Array.from(bars as number[]);- }- }- return probe;-}--function runProbe(data: Buffer, ptt: boolean): Promise<AudioProbe> {+function readDuration(data: Buffer): Promise<number | undefined> {return new Promise(resolve => {let settled = false;let output = '';- const child = spawn(process.execPath, ['-e', PROBE_SCRIPT, ptt ? 'ptt' : 'audio'], {+ const child = spawn(process.execPath, ['-e', DURATION_SCRIPT], {cwd: APP_ROOT,env: { PATH: process.env.PATH ?? '' },stdio: ['pipe', 'pipe', 'ignore'],});- const finish = (probe: AudioProbe): void => {+ const finish = (seconds: number | undefined): void => {if (settled) return;settled = true;clearTimeout(timer);- resolve(probe);+ resolve(seconds);+ };+ const parse = (): number | undefined => {+ try {+ const { seconds } = JSON.parse(output.trim()) as { seconds?: unknown };+ return typeof seconds === 'number' && Number.isInteger(seconds) && seconds >= 0 ? seconds : undefined;+ } catch {+ return undefined;+ }};const timer = setTimeout(() => {child.kill('SIGKILL');- finish(parseProbe(output));- }, PROBE_TIMEOUT_MS);+ finish(undefined);+ }, DURATION_TIMEOUT_MS);child.stdout.setEncoding('utf8');child.stdout.on('data', (chunk: string) => {output += chunk;if (output.length > MAX_OUTPUT_CHARS) {child.kill('SIGKILL');- finish({});+ finish(undefined);}});- child.on('error', () => finish({}));- // Whatever the child printed before it died still counts: the duration is printed before the decode.- child.on('close', () => finish(parseProbe(output)));+ child.on('error', () => finish(undefined));+ child.on('close', () => finish(parse()));// The child may end before it has read everything it was sent.child.stdin.on('error', () => undefined);child.stdin.end(data);});}+/** Bars of a mono 16-bit little-endian sample stream, scaled so the loudest is 100, and the sample count. */+function barsOf(pcm: Buffer): { waveform?: Uint8Array; samples: number } {+ const samples = Math.floor(pcm.length / 2);+ const view = new Int16Array(pcm.buffer.slice(pcm.byteOffset, pcm.byteOffset + samples * 2));+ const block = Math.floor(samples / WAVEFORM_BARS);+ if (block === 0) return { samples };+ const averages: number[] = [];+ for (let bar = 0; bar < WAVEFORM_BARS; bar++) {+ let sum = 0;+ for (let at = block * bar, end = at + block; at < end; at++) sum += Math.abs(view[at]);+ averages.push(sum / block);+ }+ const peak = Math.max(...averages);+ if (!(peak > 0)) return { samples };+ return { waveform: Uint8Array.from(averages, average => Math.floor((100 * average) / peak)), samples };+}++/**+ * Decode through ffmpeg, which streams and is stopped at a fixed output size, instead of through a+ * decoder that holds every sample in memory: a few hundred kilobytes of Opus packets decode into+ * gigabytes there, and a container that kills its whole cgroup on out-of-memory takes every session+ * with it. Without an ffmpeg binary there are no bars, and the note goes out with flat ones.+ */+async function readWaveform(data: Buffer): Promise<{ waveform?: Uint8Array; samples?: number }> {+ try {+ const pcm = await runFfmpeg(+ data,+ 'bin',+ 'raw',+ ['-vn', '-map', '0:a:0', '-ac', '1', '-ar', String(PCM_RATE), '-f', 's16le'],+ {+ ffmpegPath: process.env.FFMPEG_PATH || 'ffmpeg',+ timeoutMs: WAVEFORM_TIMEOUT_MS,+ maxOutputBytes: WAVEFORM_MAX_PCM_BYTES,+ },+ );+ return barsOf(pcm);+ } catch {+ return {};+ }+}+/**- * Read the duration, and for a voice note the waveform, of an outgoing audio file in a separate process.+ * The duration of an outgoing audio file and, for a voice note, its waveform, read outside this process.*- * Baileys makes both reads in the gateway's own process when the message does not carry them. They run- * parsers and decoders over bytes a caller chose: a 30-byte AIFF file aborts the process in- * music-metadata's file read, and a few hundred kilobytes of Opus packets decode into gigabytes of- * samples. Neither is an error the caller of the gateway can catch. Here a file that does either- * costs the child, and the answer is simply missing. Never rejects.+ * Baileys makes both reads in the gateway's own process when the message does not carry them, with a+ * parser and a decoder run over bytes a caller chose: a 30-byte AIFF file aborts the process in+ * music-metadata's file read, and a small Opus file decodes into gigabytes of samples. Neither is an+ * error a request can catch. Every outgoing audio content is built from this answer instead, so+ * Baileys is never left to read the file: what could not be read goes out as a zero duration and flat+ * bars. Never rejects.*/-export function probeAudio(data: Buffer, ptt: boolean): Promise<AudioProbe> {- return probes.run(() => runProbe(data, ptt)).catch(() => ({}));+export async function describeAudio(data: Buffer, ptt: boolean): Promise<AudioDescription> {+ const found = await reads+ .run(async () => {+ const [seconds, wave] = await Promise.all([+ readDuration(data),+ ptt ? readWaveform(data) : Promise.resolve<{ waveform?: Uint8Array; samples?: number }>({}),+ ]);+ // The decoded samples give the duration when the parser could not, as long as they were not cut off.+ const decoded =+ seconds === undefined && wave.samples !== undefined && wave.samples * 2 < WAVEFORM_MAX_PCM_BYTES+ ? Math.floor(wave.samples / PCM_RATE)+ : undefined;+ return { seconds: seconds ?? decoded, waveform: wave.waveform };+ })+ .catch(() => ({ seconds: undefined as number | undefined, waveform: undefined as Uint8Array | undefined }));+ return {+ seconds: found.seconds ?? 0,+ ...(ptt ? { waveform: found.waveform ?? new Uint8Array(WAVEFORM_BARS).fill(FLAT_BAR) } : {}),+ };}diff --git a/src/engine/adapters/baileys-messaging.ts b/src/engine/adapters/baileys-messaging.tsindex 7c1a8e5ba..334eee28c 100644--- a/src/engine/adapters/baileys-messaging.ts+++ b/src/engine/adapters/baileys-messaging.ts@@ -44,10 +44,7 @@ import { EngineNotSentError } from '../../common/errors/engine-not-sent.error';import { parseWaId, userPart } from '../identity/wa-id';import { BAILEYS_QUERY_BUDGET_MS, withQueryDeadline } from './baileys-query-deadline';import { ConcurrencyLimiter } from '../../common/utils/concurrency-limiter';-import { probeAudio, WAVEFORM_BARS } from './baileys-audio-probe';--/** Height of every bar of a voice note whose waveform could not be read: visible, and not a loud one. */-const FLAT_WAVEFORM_BAR = 40;+import { describeAudio } from './baileys-audio-probe';/*** Messaging-domain operations extracted from BaileysAdapter. The adapter keeps the public@@ -464,25 +461,15 @@ export class BaileysMessaging {this.host.ensureReady();const { data, mimetype } = await resolveMediaBuffer(media, this.host.sessionProxyUrl(), 'audio/mpeg');const ptt = media.ptt ?? false;- // Baileys reads the duration, and a voice note's waveform, in this process whenever the content does- // not carry them, which a crafted file turns into an abort or an out-of-memory kill of every session.- // Both are always given, from a child process, so Baileys never reads the file itself: a file the- // child cannot read sends with a zero duration and flat bars, as one Baileys could not read did.- const probe = await probeAudio(data, ptt);return this.sendContent(chatId,// Audio carries no caption, so a mention here tags the recipient through contextInfo without// visible @text. It is still forwarded: the route accepts `mentions` (SendAudioMessageDto// extends SendMediaMessageDto) and whatsapp-web.js sends it, so dropping it here made the same// request notify participants on one engine and silently not on the other.- {- audio: data,- mimetype,- ptt,- seconds: probe.seconds ?? 0,- ...(ptt ? { waveform: probe.waveform ?? new Uint8Array(WAVEFORM_BARS).fill(FLAT_WAVEFORM_BAR) } : {}),- ...this.withMentions(media.mentions),- },+ // The duration and a voice note's waveform come from describeAudio, never from Baileys, which+ // would read the file in this process.+ { audio: data, mimetype, ptt, ...(await describeAudio(data, ptt)), ...this.withMentions(media.mentions) },await this.quoteOption(media.quotedMessageId),);}diff --git a/src/engine/adapters/baileys-status.ts b/src/engine/adapters/baileys-status.tsindex 7a81046dc..adc55f736 100644--- a/src/engine/adapters/baileys-status.ts+++ b/src/engine/adapters/baileys-status.ts@@ -2,6 +2,7 @@ import type { AnyMessageContent, WAMessage, WASocket } from '@whiskeysockets/baiimport { MediaInput, StatusPostOptions, StatusResult } from '../interfaces/whatsapp-engine.interface';import { BadRequestException } from '@nestjs/common';import { resolveMediaBuffer } from './baileys-messaging';+import { describeAudio } from './baileys-audio-probe';import { EngineRefusedError } from '../../common/errors/engine-refused.error';import { EngineNotReadyError } from '../../common/errors/engine-not-ready.error';@@ -93,7 +94,7 @@ export class BaileysStatus {? { image: data, caption: options.caption, mimetype }: kind === 'video'? { video: data, caption: options.caption, mimetype }- : { audio: data, mimetype, ptt: true };+ : { audio: data, mimetype, ptt: true, ...(await describeAudio(data, true)) };return this.postStatus(content, options);}diff --git a/src/engine/adapters/baileys.adapter.spec.ts b/src/engine/adapters/baileys.adapter.spec.tsindex 4364ea52f..ea936ba9c 100644--- a/src/engine/adapters/baileys.adapter.spec.ts+++ b/src/engine/adapters/baileys.adapter.spec.ts@@ -10,8 +10,7 @@ jest.mock('../../common/media/load-remote-media', () => ({// The probe runs a child process; its own spec drives the real one.jest.mock('./baileys-audio-probe', () => ({- WAVEFORM_BARS: 64,- probeAudio: jest.fn(() => Promise.resolve({})),+ describeAudio: jest.fn(() => Promise.resolve({ seconds: 0 })),}));// A fake Baileys socket: an event emitter wearing the methods the adapter calls.@@ -185,7 +184,7 @@ import { EngineTransportError } from '../../common/errors/engine-transport.errorimport { LidNotMappedError } from '../../common/errors/lid-not-mapped.error';import { countsTowardSendBreaker, sentNothing } from '../../modules/message/send-pacing.service';import { loadRemoteMediaBuffer } from '../../common/media/load-remote-media';-import { probeAudio } from './baileys-audio-probe';+import { describeAudio } from './baileys-audio-probe';import * as safeLinkPreview from './safe-link-preview';const fakeStore = {@@ -4215,14 +4214,14 @@ describe('BaileysAdapter media sends', () => {it('sendAudioMessage hands Baileys the duration and waveform read in the child, so it reads nothing itself', async () => {const waveform = Uint8Array.from({ length: 64 }, (_, bar) => bar);- (probeAudio as jest.Mock).mockResolvedValueOnce({ seconds: 7, waveform });+ (describeAudio as jest.Mock).mockResolvedValueOnce({ seconds: 7, waveform });const adapter = await ready();await adapter.sendAudioMessage('[email protected]', {mimetype: 'audio/ogg; codecs=opus',data: Buffer.from([1]),ptt: true,});- expect(probeAudio).toHaveBeenLastCalledWith(Buffer.from([1]), true);+ expect(describeAudio).toHaveBeenLastCalledWith(Buffer.from([1]), true);expect(fakeSock.sendMessage).toHaveBeenCalledWith('[email protected]', {audio: Buffer.from([1]),mimetype: 'audio/ogg; codecs=opus',@@ -4258,13 +4257,11 @@ describe('BaileysAdapter media sends', () => {data: Buffer.from([1]),ptt: true,});- // A file the child could not read still goes out with both fields set, flat bars for a voice note.expect(fakeSock.sendMessage).toHaveBeenCalledWith('[email protected]', {audio: Buffer.from([1]),mimetype: 'audio/ogg; codecs=opus',ptt: true,seconds: 0,- waveform: new Uint8Array(64).fill(40),});});@@ -8472,6 +8469,23 @@ describe('BaileysAdapter voice status', () => {expect(options.statusJidList).toEqual(['[email protected]']);});+ // Baileys reads the file itself, in this process, for whichever of the two the content leaves out.+ it('hands Baileys the duration and waveform read in the child, as a sent voice note does', async () => {+ const waveform = Uint8Array.from({ length: 64 }, (_, bar) => bar);+ (describeAudio as jest.Mock).mockResolvedValueOnce({ seconds: 4, waveform });+ const adapter = await ready();++ await adapter.postVoiceStatus(+ { mimetype: 'audio/ogg; codecs=opus', data: Buffer.from('audio').toString('base64') },+ { recipients: ['[email protected]'] },+ );++ expect(describeAudio).toHaveBeenLastCalledWith(Buffer.from('audio'), true);+ const [, content] = fakeSock.sendMessage.mock.calls[0] as [string, { seconds?: number; waveform?: Uint8Array }];+ expect(content.seconds).toBe(4);+ expect(content.waveform).toBe(waveform);+ });+// WhatsApp has nowhere to render a caption on a status voice note.it('carries no caption', async () => {const adapter = await ready();