Skip to content

armgit / macro

Public Imported from GitHub
0

Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.

test(whatsapp): run the linked-device fleet end to end and document it (#8541)

Wolf Mermelsteinb310044eb227
diff --git a/.github/workspace-dep-closures.json b/.github/workspace-dep-closures.json
index e59b130178..2f79d186e0 100644
--- a/.github/workspace-dep-closures.json
+++ b/.github/workspace-dep-closures.json
@@ -13951,6 +13951,59 @@
"crates/webhook_signature",
"crates/workspace-hack"
],
+ "whatsapp_e2e": [
+ "crates/agent",
+ "crates/ai_toolset",
+ "crates/ai_usage",
+ "crates/attachment",
+ "crates/bot_id",
+ "crates/bot_token",
+ "crates/cowlike",
+ "crates/database_env_vars",
+ "crates/document_sub_type",
+ "crates/entity_access",
+ "crates/entity_access_db_utils",
+ "crates/entity_registry/entity_registry",
+ "crates/entity_registry/entity_registry_db_utils",
+ "crates/entity_registry/shared_entity_registry",
+ "crates/genai_telemetry",
+ "crates/harness_id",
+ "crates/harness_token",
+ "crates/macro_auth",
+ "crates/macro_authorization",
+ "crates/macro_aws_config",
+ "crates/macro_config",
+ "crates/macro_config_derive",
+ "crates/macro_entrypoint",
+ "crates/macro_env",
+ "crates/macro_env_var",
+ "crates/macro_service_urls",
+ "crates/macro_user_id",
+ "crates/macro_uuid",
+ "crates/model",
+ "crates/model-entity",
+ "crates/model-error-response",
+ "crates/model_file_type",
+ "crates/model_owner",
+ "crates/model_user",
+ "crates/models_bulk_upload",
+ "crates/models_entity_access_management",
+ "crates/models_pagination",
+ "crates/models_permissions",
+ "crates/non_empty",
+ "crates/prompt",
+ "crates/remote_env_var",
+ "crates/share_permission_db_utils",
+ "crates/whatsapp_e2e",
+ "crates/whatsapp_fake_server",
+ "crates/whatsapp_linked",
+ "crates/whatsapp_store_pg",
+ "crates/whatsmeow",
+ "crates/whatsmeow_proto",
+ "crates/whatsmeow_signal",
+ "crates/workspace-hack",
+ "services/whatsapp_session_service"
+ ],
"whatsapp_fake_server": [
"crates/whatsapp_fake_server",
"crates/whatsmeow",
diff --git a/Cargo.lock b/Cargo.lock
index 678d0959df..0bac64528d 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -21569,6 +21569,30 @@ version = "0.1.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a28ac98ddc8b9274cb41bb4d9d4d5c425b6020c50c46f25559911905610b4a88"
+[[package]]
+name = "whatsapp_e2e"
+version = "0.1.0"
+dependencies = [
+ "async-trait",
+ "chrono",
+ "entity_access",
+ "entity_access_db_utils",
+ "macro_db_migrator",
+ "sqlx",
+ "tokio",
+ "tokio-util",
+ "tracing",
+ "uuid",
+ "whatsapp_fake_server",
+ "whatsapp_linked",
+ "whatsapp_session_service",
+ "whatsapp_store_pg",
+ "whatsmeow",
+ "whatsmeow_signal",
+ "workspace-hack",
+ "zeroize",
+]
+
[[package]]
name = "whatsapp_fake_server"
version = "0.1.0"
diff --git a/Cargo.toml b/Cargo.toml
index f51432ea13..f8879db36f 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -130,6 +130,7 @@ members = [
"crates/unfurl",
"crates/user_api_key",
"crates/webhook_signature",
+ "crates/whatsapp_e2e",
"crates/whatsapp_fake_server",
"crates/whatsapp_linked",
"crates/whatsapp_store_pg",
diff --git a/crates/whatsapp_e2e/AGENTS.md b/crates/whatsapp_e2e/AGENTS.md
new file mode 100644
index 0000000000..819ce4d885
--- /dev/null
+++ b/crates/whatsapp_e2e/AGENTS.md
@@ -0,0 +1,54 @@
+# whatsapp_e2e
+
+## What it is
+
+End-to-end tests for WhatsApp linked devices. Real session-service replicas
+(supervisor, advisory locks, account runners, whatsmeow clients, the encrypted
+Postgres device store) run against `whatsapp_fake_server`. The DSS side runs
+through `whatsapp_linked::LinkedService`, and both share one `#[sqlx::test]`
+database. Nothing talks to WhatsApp or AWS (the KMS is
+`whatsapp_store_pg`'s `FakeWhatsAppKms`).
+
+## How it works
+
+- `Replica::start` runs one replica on its own Tokio runtime, built by the
+ service's `replica::start` with `test_signal()`, the fake server's dialer, a
+ shared fake KMS and short timings. `kill()` drops the runtime: sockets, lock
+ connection and client die at once, like a crashed process. `drain()` is
+ SIGTERM.
+- `watch_tables` adds statement triggers that `NOTIFY e2e_changed` on the
+ standalone group and linked-account tables; `wait_for_rows` re-checks after each notification
+ instead of polling.
+- Scenarios (`src/test.rs`): pair by QR through DSS, adopt, inbound once with
+ two observing accounts, send from your own number, kill takeover without
+ loss or duplicates, drain handover, crash between enqueue and ingest, inbound
+ image download for import, outbound attachment, a send cut off after
+ submission is `unknown` and never resent, a send without an ack times out as
+ `unknown`, and a real client's group events through the service's mapping.
+
+## Checks
+
+- `cargo test -p whatsapp_e2e` with `SQLX_OFFLINE` unset and `DATABASE_URL`
+ pointing at a migrated MacroDB (the query macros check against it; each test
+ gets its own database on that server). If the shared `macrodb` is stale,
+ create and migrate your own `macrodb_<name>` with
+ `sqlx migrate run --source crates/macro_db_client/migrations`.
+
+## Gotchas
+
+- sqlx derives test database names from test paths: two concurrent runs of
+ this crate against one Postgres server collide (`database ... already
+ exists`). Run one at a time per server.
+- Fleet transport tests inject a permissive `SendRateLimiter`; DSS request throttling is tested at the domain and Redis-adapter boundaries separately.
+- The change triggers live in `src/watch_tables.sql`; add a table there when a
+ test waits on it.
+- A failed ingest wait prints the `whatsapp_inbound_event` and
+ `whatsapp_group_message` rows; start debugging there.
+- Fake phones send each group message as a real phone does: a sender key
+ distribution message in a per-device `pkmsg`, plus the `skmsg`, under one
+ message ID. The service must ignore the key-only part.
+
+## Related
+
+- `crates/whatsapp_fake_server/AGENTS.md`, `services/whatsapp_session_service/AGENTS.md`,
+ `crates/whatsapp_linked/AGENTS.md`
diff --git a/crates/whatsapp_e2e/CLAUDE.md b/crates/whatsapp_e2e/CLAUDE.md
new file mode 120000
index 0000000000..47dc3e3d86
--- /dev/null
+++ b/crates/whatsapp_e2e/CLAUDE.md
@@ -0,0 +1 @@
+AGENTS.md
\ No newline at end of file
diff --git a/crates/whatsapp_e2e/Cargo.toml b/crates/whatsapp_e2e/Cargo.toml
new file mode 100644
index 0000000000..ae36881766
--- /dev/null
+++ b/crates/whatsapp_e2e/Cargo.toml
@@ -0,0 +1,28 @@
+[package]
+edition = "2024"
+name = "whatsapp_e2e"
+publish = false
+version = "0.1.0"
+description = "End-to-end tests of WhatsApp linked devices: session-service replicas, the linked-devices domain and MacroDB against a fake WhatsApp server."
+
+[dependencies]
+entity_access_db_utils = { path = "../entity_access_db_utils" }
+async-trait = { workspace = true }
+entity_access = { path = "../entity_access", default-features = false, features = ["ports"] }
+sqlx = { workspace = true }
+tokio = { workspace = true, features = ["macros", "rt-multi-thread", "sync", "time"] }
+tokio-util = { workspace = true, features = ["rt"] }
+tracing = { workspace = true }
+uuid = { workspace = true }
+whatsapp_fake_server = { path = "../whatsapp_fake_server" }
+whatsapp_linked = { path = "../whatsapp_linked", features = ["outbound"] }
+whatsapp_session_service = { path = "../../services/whatsapp_session_service" }
+whatsapp_store_pg = { path = "../whatsapp_store_pg", features = ["fake-kms"] }
+whatsmeow = { path = "../whatsmeow" }
+workspace-hack = { version = "0.1", path = "../workspace-hack" }
+zeroize = { workspace = true }
+
+[dev-dependencies]
+chrono = { workspace = true }
+whatsmeow_signal = { path = "../whatsmeow_signal" }
+macro_db_migrator = { path = "../macro_db_migrator" }
diff --git a/crates/whatsapp_e2e/src/lib.rs b/crates/whatsapp_e2e/src/lib.rs
new file mode 100644
index 0000000000..b5848017c6
--- /dev/null
+++ b/crates/whatsapp_e2e/src/lib.rs
@@ -0,0 +1,252 @@
+//! End-to-end tests of WhatsApp linked devices.
+//!
+//! The tests run the real session service (supervisor, advisory locks,
+//! account runners, whatsmeow clients, the Postgres device store) against a
+//! [`whatsapp_fake_server::FakeWhatsApp`], and drive the DSS side through the
+//! linked-devices domain service, all on one test database.
+//!
+//! Each [`Replica`] runs on its own Tokio runtime, like a separate process:
+//! [`Replica::kill`] drops every task and socket at once (the lock connection
+//! closes, so Postgres frees its advisory locks), and [`Replica::drain`] is
+//! what SIGTERM does.
+
+#![deny(missing_docs)]
+// The account runner future nests the domain, store and client futures.
+#![recursion_limit = "256"]
+
+#[cfg(test)]
+mod test;
+
+use std::collections::HashMap;
+use std::sync::Arc;
+use std::time::Duration;
+
+use sqlx::postgres::{PgConnectOptions, PgListener, PgPool, PgPoolOptions};
+use tokio::sync::oneshot;
+use tokio_util::sync::CancellationToken;
+use whatsapp_fake_server::{FakeWhatsApp, test_signal};
+use whatsapp_session_service::replica::{self, ReplicaDeps, Timings};
+use whatsapp_store_pg::kms::{FakeWhatsAppKms, GeneratedDataKey, WhatsAppKms, WhatsAppKmsError};
+use whatsmeow::{Dialer, WebSocketDialer};
+use zeroize::Zeroizing;
+
+/// How long a submitted send waits for the server's ack before it is
+/// recorded as unknown. Short, so the unknown path runs within a test.
+pub const SEND_TIMEOUT: Duration = Duration::from_secs(5);
+
+/// Replica timings shortened so takeovers and timeouts happen within a test.
+pub const TIMINGS: Timings = Timings {
+ lock_keepalive: Duration::from_secs(1),
+ send_timeout: SEND_TIMEOUT,
+ rescan_every: Duration::from_secs(1),
+ tick_every: Duration::from_secs(1),
+ release_timeout: Duration::from_secs(7),
+ stop_timeout: Duration::from_secs(5),
+ drain_deadline: Duration::from_secs(30),
+ health_every: Duration::from_secs(60),
+};
+
+/// One fake KMS shared by every replica, as they share the real KMS key.
+#[derive(Clone)]
+pub struct SharedKms(pub Arc<FakeWhatsAppKms>);
+
+impl SharedKms {
+ /// A fresh fake KMS.
+ pub fn new() -> SharedKms {
+ SharedKms(Arc::new(FakeWhatsAppKms::new("e2e-kms-key")))
+ }
+}
+
+impl Default for SharedKms {
+ fn default() -> Self {
+ SharedKms::new()
+ }
+}
+
+impl WhatsAppKms for SharedKms {
+ async fn generate_data_key(
+ &self,
+ encryption_context: HashMap<String, String>,
+ ) -> Result<GeneratedDataKey, WhatsAppKmsError> {
+ self.0.generate_data_key(encryption_context).await
+ }
+
+ async fn decrypt(
+ &self,
+ kms_key_id: &str,
+ ciphertext: &[u8],
+ encryption_context: HashMap<String, String>,
+ ) -> Result<Zeroizing<Vec<u8>>, WhatsAppKmsError> {
+ self.0
+ .decrypt(kms_key_id, ciphertext, encryption_context)
+ .await
+ }
+}
+
+/// A running session-service replica.
+pub struct Replica {
+ runtime: Option<tokio::runtime::Runtime>,
+ shutdown: CancellationToken,
+ finished: Option<oneshot::Receiver<Result<(), String>>>,
+ /// The Postgres backend of the replica's lock connection.
+ pub lock_backend_pid: i32,
+}
+
+impl Drop for Replica {
+ fn drop(&mut self) {
+ if let Some(runtime) = self.runtime.take() {
+ runtime.shutdown_background();
+ }
+ }
+}
+
+impl Replica {
+ /// Starts a replica against the test database and the fake server, the
+ /// with the service's own `replica::start`.
+ pub async fn start(
+ database: &PgConnectOptions,
+ whatsapp: &FakeWhatsApp,
+ kms: &SharedKms,
+ ) -> Replica {
+ let runtime = tokio::runtime::Builder::new_multi_thread()
+ .worker_threads(2)
+ .enable_all()
+ .build()
+ .expect("a replica runtime starts");
+ let shutdown = CancellationToken::new();
+ let (ready, started) = oneshot::channel();
+ let (done, finished) = oneshot::channel();
+ let database = database.clone();
+ let dialer: Arc<dyn Dialer> = Arc::new(WebSocketDialer {
+ url: whatsapp.websocket_url(),
+ });
+ let media_base_url = whatsapp.media_base_url();
+ let kms = kms.clone();
+ let stop = shutdown.clone();
+ runtime.spawn(async move {
+ let result = run_replica(database, dialer, media_base_url, kms, stop, ready).await;
+ let _ = done.send(result);
+ });
+ let lock_backend_pid = started
+ .await
+ .expect("the replica reports startup")
+ .expect("the replica starts");
+ Replica {
+ runtime: Some(runtime),
+ shutdown,
+ finished: Some(finished),
+ lock_backend_pid,
+ }
+ }
+
+ /// SIGTERM: drain every account and wait for the supervisor to return.
+ pub async fn drain(mut self, timeout: Duration) -> Result<(), String> {
+ self.shutdown.cancel();
+ let finished = self.finished.take().expect("a replica drains once");
+ let result = tokio::time::timeout(timeout, finished)
+ .await
+ .map_err(|_| "the replica did not finish draining".to_owned())?
+ .map_err(|_| "the replica task died".to_owned())?;
+ if let Some(runtime) = self.runtime.take() {
+ runtime.shutdown_background();
+ }
+ result
+ }
+
+ /// Process death: every task, websocket and database connection of the
+ /// replica goes away at once.
+ pub fn kill(mut self) {
+ if let Some(runtime) = self.runtime.take() {
+ runtime.shutdown_background();
+ }
+ }
+}
+
+async fn run_replica(
+ database: PgConnectOptions,
+ dialer: Arc<dyn Dialer>,
+ media_base_url: String,
+ kms: SharedKms,
+ shutdown: CancellationToken,
+ ready: oneshot::Sender<Result<i32, String>>,
+) -> Result<(), String> {
+ let started = async {
+ // CI runs these beside the whole workspace's database tests on one
+ // Postgres (max_connections = 100); two accounts need only a few.
+ let pool = PgPoolOptions::new()
+ .max_connections(4)
+ .connect_with(database.clone())
+ .await
+ .map_err(|error| error.to_string())?;
+ replica::start(
+ ReplicaDeps {
+ pool,
+ lock_options: database,
+ kms,
+ signal: test_signal(),
+ dialer,
+ media_base_url: Some(media_base_url),
+ max_accounts: 50,
+ drain_batch: 20,
+ timings: TIMINGS,
+ },
+ shutdown,
+ )
+ .await
+ .map_err(|error| error.to_string())
+ }
+ .await;
+ let running = match started {
+ Ok(running) => {
+ let _ = ready.send(Ok(running.lock_backend_pid()));
+ running
+ }
+ Err(error) => {
+ let _ = ready.send(Err(error.clone()));
+ return Err(error);
+ }
+ };
+ running.wait().await.map_err(|error| error.to_string())
+}
+
+/// The channel the test triggers notify on.
+const CHANGED_CHANNEL: &str = "e2e_changed";
+
+/// Makes the linked-device tables `NOTIFY e2e_changed` on every write, so
+/// tests can wait for rows the service writes without polling. Test
+/// databases only.
+pub async fn watch_tables(pool: &PgPool) {
+ sqlx::raw_sql(include_str!("watch_tables.sql"))
+ .execute(pool)
+ .await
+ .expect("the change triggers are created");
+}
+
+/// Waits until `check` finds something, re-running it after every write to
+/// a watched table (see [`watch_tables`]).
+pub async fn wait_for_rows<Found, Check, Checking>(
+ pool: &PgPool,
+ timeout: Duration,
+ check: Check,
+) -> Option<Found>
+where
+ Check: Fn() -> Checking,
+ Checking: Future<Output = Option<Found>>,
+{
+ let mut listener = PgListener::connect_with(pool)
+ .await
+ .expect("a change listener connects");
+ listener
+ .listen(CHANGED_CHANNEL)
+ .await
+ .expect("the change listener subscribes");
+ let wait = async {
+ loop {
+ if let Some(found) = check().await {
+ return found;
+ }
+ listener.recv().await.expect("the change listener stays up");
+ }
+ };
+ tokio::time::timeout(timeout, wait).await.ok()
+}
diff --git a/crates/whatsapp_e2e/src/test.rs b/crates/whatsapp_e2e/src/test.rs
new file mode 100644
index 0000000000..e0baedbe52
--- /dev/null
+++ b/crates/whatsapp_e2e/src/test.rs
@@ -0,0 +1,859 @@
+//! Linked WhatsApp accounts end to end: pairing, adoption, inbound messages,
+//! sends, failover and drain, against the fake WhatsApp server.
+
+use std::sync::Arc;
+use std::time::Duration;
+
+use async_trait::async_trait;
+use entity_access::domain::models::{
+ AccessLevel, Entity, EntityAccessAuth, EntityAccessReceipt, EntityPermission, EntityType,
+ ViewAccessLevel,
+};
+use macro_db_migrator::MACRO_DB_MIGRATIONS;
+use sqlx::{Connection, PgConnection, PgPool};
+use uuid::Uuid;
+use whatsapp_fake_server::{FakePhone, FakeWhatsApp};
+use whatsapp_linked::domain::models::{
+ AccountState, AdoptGroup, Binding, Features, ReplyContent, SendAuthority, SendReply,
+};
+use whatsapp_linked::domain::ports::{Authorizer, LinkedResult};
+use whatsapp_linked::domain::service::LinkedService;
+use whatsapp_linked::outbound::pg::PgLinkedRepository;
+use whatsmeow::types::Jid;
+
+use super::*;
+
+const TIMEOUT: Duration = Duration::from_secs(60);
+const ALICE: &str = "macro|[email protected]";
+const BOB: &str = "macro|[email protected]";
+
+/// Runs one scenario at a time across nextest's test processes. Each one
+/// holds a dozen or more connections (replica pools, lock connection,
+/// listeners); in parallel with the rest of the workspace they exhaust CI's
+/// Postgres (`max_connections` = 100). Advisory locks are per database, so
+/// the lock lives in the shared `postgres` database, not the test's own.
+async fn take_fleet_slot(pool: &PgPool) -> PgConnection {
+ let options = (*pool.connect_options()).clone().database("postgres");
+ let mut slot = PgConnection::connect_with(&options)
+ .await
+ .expect("the fleet lock connection opens");
+ sqlx::raw_sql("SELECT pg_advisory_lock(7346525012)")
+ .execute(&mut slot)
+ .await
+ .expect("the fleet lock is taken");
+ slot
+}
+
+struct Allow;
+
+#[async_trait]
+impl Authorizer for Allow {
+ async fn require_group_owner(&self, _: Uuid, _: &str) -> LinkedResult<()> {
+ Ok(())
+ }
+}
+
+/// The database, the DSS-side domain service and the fake WhatsApp world:
+/// Alice and Bob (Macro users with their own WhatsApp) and a customer who is
+/// only on WhatsApp, all in one group.
+struct World {
+ /// Holds the fleet lock until the scenario ends (see [`take_fleet_slot`]).
+ _fleet_slot: PgConnection,
+ pool: PgPool,
+ linked: LinkedService,
+ whatsapp: FakeWhatsApp,
+ kms: SharedKms,
+ alice: FakePhone,
+ bob: FakePhone,
+ customer: FakePhone,
+ group: Jid,
+}
+
+impl World {
+ async fn new(pool: PgPool) -> World {
+ let fleet_slot = take_fleet_slot(&pool).await;
+ watch_tables(&pool).await;
+ for id in [ALICE, BOB] {
+ user(&pool, id).await;
+ }
+ let team = Uuid::now_v7();
+ sqlx::query!(
+ "INSERT INTO team (id, name, owner_id) VALUES ($1, 'Linked', $2)",
+ team,
+ ALICE
+ )
+ .execute(&pool)
+ .await
+ .unwrap();
+ for (id, role) in [(ALICE, "owner"), (BOB, "member")] {
+ sqlx::query!(
+ "INSERT INTO team_user (team_id, user_id, team_role) VALUES ($1, $2, $3::text::team_role)",
+ team,
+ id,
+ role
+ )
+ .execute(&pool)
+ .await
+ .unwrap();
+ }
+ let linked = LinkedService::new(
+ Features {
+ pairing: true,
+ adoption: true,
+ sending: true,
+ },
+ Arc::new(PgLinkedRepository::new(pool.clone())),
+ Arc::new(Allow),
+ Arc::new(Allow),
+ );
+ let whatsapp = FakeWhatsApp::start().await.unwrap();
+ let alice = whatsapp.add_user("15550000001", "Alice").await;
+ let bob = whatsapp.add_user("15550000002", "Bob").await;
+ let customer = whatsapp.add_user("15558880001", "Customer").await;
+ let group = whatsapp.create_group("Customer escalations", &[&customer, &alice, &bob]);
+ World {
+ _fleet_slot: fleet_slot,
+ pool,
+ linked,
+ whatsapp,
+ kms: SharedKms::new(),
+ alice,
+ bob,
+ customer,
+ group,
+ }
+ }
+
+ async fn replica(&self) -> Replica {
+ Replica::start(&self.pool.connect_options(), &self.whatsapp, &self.kms).await
+ }
+
+ /// Pairs `user` through DSS: start a pairing, wait for the replica's QR,
+ /// scan it with `phone`, wait for the account to connect. Returns the
+ /// linked device.
+ async fn pair(&self, user: &str, phone: &FakePhone) -> Jid {
+ let pairing = self
+ .linked
+ .connect(user, false, SendAuthority::Human)
+ .await
+ .unwrap();
+ let qr = wait_for_rows(&self.pool, TIMEOUT, || async {
+ self.linked
+ .pair_state(user, pairing.id, SendAuthority::Human)
+ .await
+ .ok()?
+ .qr
+ })
+ .await
+ .expect("the replica shows a QR code");
+ let device = phone
+ .scan_qr(&qr)
+ .await
+ .expect("the phone approves the QR code");
+ let identity = phone.jid().to_string();
+ wait_for_rows(&self.pool, TIMEOUT, || async {
+ let account = self.linked.account(user).await.ok()??;
+ (account.state == AccountState::Connected
+ && account.identity_jid.as_deref() == Some(identity.as_str()))
+ .then_some(())
+ })
+ .await
+ .expect("the account connects as the phone's number");
+ device
+ }
+
+ /// Adopts the group for `user` once the replica has listed it.
+ async fn adopt(&self, user: &str, users: &[&str]) -> Binding {
+ let group = self.group.to_string();
+ wait_for_rows(&self.pool, TIMEOUT, || async {
+ let groups = self.linked.groups(user).await.ok()?;
+ groups.iter().any(|info| info.jid == group).then_some(())
+ })
+ .await
+ .expect("the replica lists the group");
+ let binding = self
+ .linked
+ .adopt(
+ user,
+ &AdoptGroup {
+ group_jid: group,
+ name: Some("Escalations".to_owned()),
+ },
+ SendAuthority::Human,
+ )
+ .await
+ .unwrap();
+ let mut tx = self.pool.begin().await.unwrap();
+ for viewer in users {
+ entity_access_db_utils::direct_share::set(
+ &mut tx,
+ &binding.id,
+ EntityType::WhatsAppGroup,
+ &(*viewer).to_owned().try_into().unwrap(),
+ Some(AccessLevel::View),
+ false,
+ )
+ .await
+ .unwrap();
+ }
+ tx.commit().await.unwrap();
+ binding
+ }
+
+ /// Waits until `user`'s own account observes the group shared with them,
+ /// which happens on the account's next roster read; ask for one.
+ async fn observe(&self, user: &str, binding: &Binding) {
+ sqlx::query!(
+ "SELECT pg_notify('whatsapp_groups_requested', id::text) FROM whatsapp_linked_account WHERE user_id = $1",
+ user
+ )
+ .execute(&self.pool)
+ .await
+ .unwrap();
+ let observing = wait_for_rows(&self.pool, TIMEOUT, || async {
+ sqlx::query_scalar!(
+ r#"SELECT m.account_id FROM whatsapp_group_membership m
+ JOIN whatsapp_linked_account a ON a.id = m.account_id
+ WHERE m.binding_id = $1 AND a.user_id = $2 AND m.consent_ended_at IS NULL AND m.remote_member"#,
+ binding.id,
+ user
+ )
+ .fetch_optional(&self.pool)
+ .await
+ .unwrap()
+ })
+ .await;
+ self.expect(observing, "the viewer's account observes the shared group")
+ .await;
+ }
+
+ /// Alice paired and observing the adopted group.
+ async fn alice_adopted(&self) -> (Jid, Binding) {
+ let device = self.pair(ALICE, &self.alice).await;
+ let binding = self.adopt(ALICE, &[]).await;
+ (device, binding)
+ }
+
+ /// Waits until the WhatsApp message `source_id` is a standalone group message and
+ /// returns every stored copy of it (there must be one).
+ async fn timeline(&self, binding: &Binding, source_id: &str) -> Vec<String> {
+ let copies = wait_for_rows(&self.pool, TIMEOUT, || async {
+ let copies = group_messages(&self.pool, binding.id, source_id).await;
+ (!copies.is_empty()).then_some(copies)
+ })
+ .await;
+ match copies {
+ Some(copies) => copies,
+ None => {
+ let mut whatsapp = String::new();
+ for phone in [&self.alice, &self.bob] {
+ for device in self.whatsapp.linked_devices(phone) {
+ whatsapp.push_str(&format!(
+ " {device}: online {}, deliveries {}, pending {}\n",
+ self.whatsapp.is_online(&device),
+ self.whatsapp.delivery_count(&device, source_id),
+ self.whatsapp.pending_count(&device),
+ ));
+ }
+ }
+ panic!(
+ "message {source_id} was not ingested\nfake WhatsApp:\n{whatsapp}{}",
+ self.dump().await
+ )
+ }
+ }
+ }
+
+ /// `found`, or a panic that shows the linked-device rows.
+ async fn expect<Found>(&self, found: Option<Found>, what: &str) -> Found {
+ match found {
+ Some(found) => found,
+ None => panic!("{what}\n{}", self.dump().await),
+ }
+ }
+
+ /// The linked-device rows a failed wait should explain itself with.
+ async fn dump(&self) -> String {
+ let pool = &self.pool;
+ let sections = [
+ (
+ "inbound events (account, state, kind)",
+ sqlx::query_scalar!(
+ r#"SELECT account_id::text || ' ' || state || ' ' || coalesce(payload->>'kind', '') AS "row!" FROM whatsapp_inbound_event"#
+ )
+ .fetch_all(pool)
+ .await,
+ ),
+ (
+ "external messages (source, origin, message)",
+ sqlx::query_scalar!(
+ r#"SELECT source_id || ' ' || origin || ' ' || coalesce(message_id::text, '-') AS "row!" FROM whatsapp_group_message"#
+ )
+ .fetch_all(pool)
+ .await,
+ ),
+ (
+ "observations (source, account)",
+ sqlx::query_scalar!(
+ r#"SELECT source_id || ' ' || account_id::text AS "row!" FROM whatsapp_message_observation"#
+ )
+ .fetch_all(pool)
+ .await,
+ ),
+ (
+ "accounts (id, state, lease epoch)",
+ sqlx::query_scalar!(
+ r#"SELECT id::text || ' ' || state || ' ' || lease_epoch AS "row!" FROM whatsapp_linked_account"#
+ )
+ .fetch_all(pool)
+ .await,
+ ),
+ (
+ "sends (state, source)",
+ sqlx::query_scalar!(
+ r#"SELECT state || ' ' || coalesce(allocated_source_id, '-') AS "row!" FROM whatsapp_send"#
+ )
+ .fetch_all(pool)
+ .await,
+ ),
+ ];
+ let mut dump = String::new();
+ for (title, rows) in sections {
+ dump.push_str(&format!("{title}:\n"));
+ let rows = rows.unwrap_or_else(|error| vec![format!("query failed: {error}")]);
+ for row in rows {
+ dump.push_str(&format!(" {row}\n"));
+ }
+ }
+ dump
+ }
+
+ async fn send(&self, user: &str, binding: &Binding, text: &str) -> Uuid {
+ self.linked
+ .send_reply(
+ &receipt::<ViewAccessLevel>(user, binding.id),
+ &SendReply {
+ idempotency_key: Uuid::now_v7(),
+ message: ReplyContent::Text {
+ text: text.to_owned(),
+ },
+ reply_to: None,
+ },
+ SendAuthority::Human,
+ )
+ .await
+ .unwrap()
+ .id
+ }
+
+ async fn send_state(&self, send: Uuid) -> (String, Option<String>) {
+ let row = sqlx::query!(
+ "SELECT state, allocated_source_id FROM whatsapp_send WHERE id = $1",
+ send
+ )
+ .fetch_one(&self.pool)
+ .await
+ .unwrap();
+ (row.state, row.allocated_source_id)
+ }
+
+ /// Waits until the send's state is one of `states`.
+ async fn wait_send(&self, send: Uuid, states: &[&str]) -> (String, Option<String>) {
+ let state = wait_for_rows(&self.pool, TIMEOUT, || async {
+ let state = self.send_state(send).await;
+ states.contains(&state.0.as_str()).then_some(state)
+ })
+ .await;
+ self.expect(state, &format!("the send reaches one of {states:?}"))
+ .await
+ }
+}
+
+async fn user(pool: &PgPool, id: &str) {
+ let email = id.trim_start_matches("macro|");
+ let macro_user = Uuid::now_v7();
+ sqlx::query!(
+ "INSERT INTO macro_user (id, username, email, stripe_customer_id) VALUES ($1, $2, $3, $2)",
+ macro_user,
+ id,
+ email
+ )
+ .execute(pool)
+ .await
+ .unwrap();
+ sqlx::query!(
+ r#"INSERT INTO "User" (id, email, macro_user_id) VALUES ($1, $2, $3)"#,
+ id,
+ email,
+ macro_user
+ )
+ .execute(pool)
+ .await
+ .unwrap();
+}
+
+fn receipt<Permission: entity_access::domain::models::RequiredPermission>(
+ user: &str,
+ channel: Uuid,
+) -> EntityAccessReceipt<Permission> {
+ EntityAccessReceipt::try_new(
+ EntityAccessAuth::Authenticated(user.to_owned().try_into().unwrap()),
+ Entity {
+ entity_id: channel.to_string(),
+ entity_type: EntityType::WhatsAppGroup,
+ },
+ EntityPermission::AccessLevel {
+ access_level: AccessLevel::View,
+ },
+ )
+ .unwrap()
+}
+
+/// The content of every standalone message for one WhatsApp message.
+async fn group_messages(pool: &PgPool, binding: Uuid, source_id: &str) -> Vec<String> {
+ sqlx::query_scalar!(
+ "SELECT body FROM whatsapp_group_message WHERE binding_id=$1 AND source_id=$2 AND message_id IS NOT NULL",
+ binding,
+ source_id
+ )
+ .fetch_all(pool)
+ .await
+ .unwrap()
+}
+
+async fn observations(pool: &PgPool, binding: Uuid, source_id: &str) -> i64 {
+ sqlx::query_scalar!(
+ r#"SELECT count(*) AS "count!" FROM whatsapp_message_observation WHERE binding_id = $1 AND source_id = $2"#,
+ binding,
+ source_id
+ )
+ .fetch_one(pool)
+ .await
+ .unwrap()
+}
+
+#[sqlx::test(migrator = "MACRO_DB_MIGRATIONS")]
+async fn pair_adopt_receive_once_and_send_from_your_own_number(pool: PgPool) {
+ let world = World::new(pool).await;
+ let replica = world.replica().await;
+ world.pair(ALICE, &world.alice).await;
+ world.pair(BOB, &world.bob).await;
+ let binding = world.adopt(ALICE, &[BOB]).await;
+ world.observe(BOB, &binding).await;
+
+ // Two linked accounts see the customer's message; it lands once.
+ let source = world
+ .customer
+ .send_text(&world.group, "my order is late")
+ .await
+ .unwrap();
+ let copies = world.timeline(&binding, &source).await;
+ assert_eq!(copies.len(), 1);
+ assert!(copies[0].contains("my order is late"), "{copies:?}");
+ let observed = wait_for_rows(&world.pool, TIMEOUT, || async {
+ (observations(&world.pool, binding.id, &source).await == 2).then_some(())
+ })
+ .await;
+ world
+ .expect(observed, "both accounts observe the message")
+ .await;
+ assert_eq!(
+ group_messages(&world.pool, binding.id, &source).await.len(),
+ 1
+ );
+
+ // Alice replies from Macro; the customer gets it from Alice's number.
+ let send = world.send(ALICE, &binding, "sorry, on it").await;
+ let received = world
+ .customer
+ .wait_for_message(TIMEOUT, |message| message.text() == Some("sorry, on it"))
+ .await
+ .expect("the customer's phone receives the reply");
+ assert_eq!(received.sender_phone, world.alice.jid());
+ assert_eq!(received.chat, world.group);
+ let (_, source_id) = world
+ .wait_send(send, &["accepted", "sent", "delivered", "read"])
+ .await;
+ assert_eq!(source_id.as_deref(), Some(received.id.as_str()));
+
+ // A shared second sender uses their own connection and phone number.
+ let second_send = world.send(BOB, &binding, "Bob here too").await;
+ let second_reply = world
+ .customer
+ .wait_for_message(TIMEOUT, |message| message.text() == Some("Bob here too"))
+ .await
+ .expect("the shared viewer sends from their own WhatsApp");
+ assert_eq!(second_reply.sender_phone, world.bob.jid());
+ world
+ .wait_send(second_send, &["accepted", "sent", "delivered", "read"])
+ .await;
+
+ replica.drain(TIMEOUT).await.unwrap();
+}
+
+#[sqlx::test(migrator = "MACRO_DB_MIGRATIONS")]
+async fn a_killed_replica_is_taken_over_without_losing_or_duplicating(pool: PgPool) {
+ let world = World::new(pool).await;
+ let first = world.replica().await;
+ let (device, binding) = world.alice_adopted().await;
+ let second = world.replica().await;
+
+ let before = world
+ .customer
+ .send_text(&world.group, "before")
+ .await
+ .unwrap();
+ assert_eq!(world.timeline(&binding, &before).await.len(), 1);
+
+ first.kill();
+ assert!(world.whatsapp.wait_until_offline(&device, TIMEOUT).await);
+ let during = world
+ .customer
+ .send_text(&world.group, "during")
+ .await
+ .unwrap();
+ let after_takeover = world.whatsapp.wait_until_online(&device, TIMEOUT).await;
+ assert!(after_takeover, "the second replica takes the account over");
+ let after = world
+ .customer
+ .send_text(&world.group, "after")
+ .await
+ .unwrap();
+
+ for (source, text) in [(&before, "before"), (&during, "during"), (&after, "after")] {
+ let copies = world.timeline(&binding, source).await;
+ assert_eq!(copies.len(), 1, "{text} lands exactly once");
+ assert!(copies[0].contains(text));
+ }
+ let send = world.send(ALICE, &binding, "still here").await;
+ world
+ .wait_send(send, &["accepted", "sent", "delivered", "read"])
+ .await;
+ world
+ .customer
+ .wait_for_message(TIMEOUT, |message| message.text() == Some("still here"))
+ .await
+ .expect("sends work after the takeover");
+
+ second.drain(TIMEOUT).await.unwrap();
+}
+
+#[sqlx::test(migrator = "MACRO_DB_MIGRATIONS")]
+async fn a_drained_replica_hands_over_without_loss(pool: PgPool) {
+ let world = World::new(pool).await;
+ let first = world.replica().await;
+ let (device, binding) = world.alice_adopted().await;
+ let second = world.replica().await;
+
+ let before = world
+ .customer
+ .send_text(&world.group, "before drain")
+ .await
+ .unwrap();
+ assert_eq!(world.timeline(&binding, &before).await.len(), 1);
+ let send = world.send(ALICE, &binding, "sent while draining").await;
+
+ first
+ .drain(TIMEOUT)
+ .await
+ .expect("the replica drains cleanly");
+ let during = world
+ .customer
+ .send_text(&world.group, "during handover")
+ .await
+ .unwrap();
+ assert!(world.whatsapp.wait_until_online(&device, TIMEOUT).await);
+
+ assert_eq!(world.timeline(&binding, &during).await.len(), 1);
+ assert_eq!(
+ group_messages(&world.pool, binding.id, &before).await.len(),
+ 1
+ );
+ let (_, source_id) = world
+ .wait_send(send, &["accepted", "sent", "delivered", "read"])
+ .await;
+ let source_id = source_id.expect("a dispatched send has a source id");
+ assert_eq!(world.whatsapp.submission_count(&device, &source_id), 1);
+
+ second.drain(TIMEOUT).await.unwrap();
+}
+
+#[sqlx::test(migrator = "MACRO_DB_MIGRATIONS")]
+async fn a_send_cut_off_after_submission_is_unknown_and_never_resent(pool: PgPool) {
+ let world = World::new(pool).await;
+ let first = world.replica().await;
+ let (device, binding) = world.alice_adopted().await;
+
+ // WhatsApp gets the message, but the replica dies before the ack.
+ world.whatsapp.withhold_acks(&device, true);
+ let submitted_before = world.whatsapp.submissions(&device).len();
+ let send = world.send(ALICE, &binding, "did this go out?").await;
+ assert!(
+ world
+ .whatsapp
+ .wait_for_submissions(&device, submitted_before + 1, TIMEOUT)
+ .await
+ );
+ first.kill();
+ world.whatsapp.withhold_acks(&device, false);
+
+ let second = world.replica().await;
+ let (state, source_id) = world.wait_send(send, &["unknown"]).await;
+ assert_eq!(state, "unknown");
+ let source_id = source_id.expect("the dispatch fence recorded the source id");
+
+ // A later send goes out through the new replica; the unknown one doesn't.
+ let later = world.send(ALICE, &binding, "a later message").await;
+ world
+ .wait_send(later, &["accepted", "sent", "delivered", "read"])
+ .await;
+ assert_eq!(world.whatsapp.submission_count(&device, &source_id), 1);
+ assert_eq!(world.send_state(send).await.0, "unknown");
+ let copies = world
+ .customer
+ .received()
+ .into_iter()
+ .filter(|message| message.text() == Some("did this go out?"))
+ .count();
+ assert_eq!(copies, 1, "the customer got the cut-off message once");
+
+ second.drain(TIMEOUT).await.unwrap();
+}
+
+#[sqlx::test(migrator = "MACRO_DB_MIGRATIONS")]
+async fn a_send_without_an_ack_times_out_as_unknown(pool: PgPool) {
+ let world = World::new(pool).await;
+ let replica = world.replica().await;
+ let (device, binding) = world.alice_adopted().await;
+
+ world.whatsapp.withhold_acks(&device, true);
+ let send = world.send(ALICE, &binding, "no ack for this").await;
+ let (_, source_id) = world.wait_send(send, &["unknown"]).await;
+ let source_id = source_id.expect("the dispatch fence recorded the source id");
+ world.whatsapp.withhold_acks(&device, false);
+
+ let later = world.send(ALICE, &binding, "acked").await;
+ world
+ .wait_send(later, &["accepted", "sent", "delivered", "read"])
+ .await;
+ assert_eq!(world.whatsapp.submission_count(&device, &source_id), 1);
+
+ replica.drain(TIMEOUT).await.unwrap();
+}
+
+#[tokio::test]
+async fn a_phone_group_message_maps_to_one_inbound_text() {
+ use whatsapp_linked::domain::events::{InboundKind, MessageKind};
+ use whatsapp_session_service::outbound::whatsapp::events::{Mapped, map_event};
+
+ let whatsapp = FakeWhatsApp::start().await.unwrap();
+ let alice = whatsapp.add_user("15550000001", "Alice").await;
+ let bob = whatsapp.add_user("15550000002", "Bob").await;
+ let store = Arc::new(whatsmeow::store::MemoryStore::default());
+ let device = store.new_device(&whatsmeow_signal::LibsignalCurveSignatures::new());
+ let client = whatsmeow::Client::new(
+ device,
+ whatsapp_fake_server::test_signal(),
+ whatsmeow::ClientConfig {
+ dialer: whatsapp.dialer(),
+ synchronous_ack: true,
+ ..Default::default()
+ },
+ );
+ let mut events = client.subscribe();
+ let mut qr = client
+ .get_qr_channel(tokio_util::sync::CancellationToken::new())
+ .await
+ .unwrap();
+ client.connect().await.unwrap();
+ let Some(whatsmeow::QrChannelItem::Code { code, .. }) = qr.recv().await else {
+ panic!("expected a QR code");
+ };
+ let linked = alice.scan_qr(&code).await.unwrap();
+ assert!(whatsapp.wait_until_keys_uploaded(&linked, TIMEOUT).await);
+ let group = whatsapp.create_group("Escalations", &[&bob, &alice]);
+ let id = bob.send_text(&group, "hello").await.unwrap();
+
+ // A phone's first group message is two events with one ID: the sender
+ // key alone, then the text. Only the text is group content.
+ let mut mapped = Vec::new();
+ while mapped.len() < 2 {
+ let event = tokio::time::timeout(TIMEOUT, events.recv())
+ .await
+ .expect("the client emits the message events")
+ .expect("the event channel is open");
+ if let whatsmeow::Event::Message(message) = &event {
+ assert_eq!(message.info.id, id);
+ mapped.push(map_event(
+ Uuid::now_v7(),
+ &event,
+ chrono::Utc::now(),
+ &|_| None,
+ ));
+ }
+ }
+ assert!(mapped[0].is_none(), "the key-only part is not content");
+ let Some(Mapped::Inbound(event)) = &mapped[1] else {
+ panic!("the text maps to an inbound event");
+ };
+ assert_eq!(event.group_jid, group.to_string());
+ let InboundKind::Message(message) = &event.kind else {
+ panic!("a message event");
+ };
+ assert_eq!(message.kind, MessageKind::Text);
+ assert_eq!(message.text, "hello");
+ assert_eq!(message.source_id, id);
+ assert_eq!(message.sender.jid, bob.lid().to_string());
+ assert_eq!(message.sender.alternate_jid, Some(bob.jid().to_string()));
+}
+
+#[sqlx::test(migrator = "MACRO_DB_MIGRATIONS")]
+async fn a_phone_image_is_downloaded_for_import(pool: PgPool) {
+ let world = World::new(pool).await;
+ let replica = world.replica().await;
+ let (_, binding) = world.alice_adopted().await;
+
+ let photo = b"jpeg bytes from the customer's camera".repeat(40);
+ let source = world
+ .customer
+ .send_image(&world.group, &photo, "the damaged box")
+ .await
+ .unwrap();
+ let copies = world.timeline(&binding, &source).await;
+ assert_eq!(copies.len(), 1);
+ assert!(copies[0].contains("the damaged box"), "{copies:?}");
+ let downloaded = wait_for_rows(&world.pool, TIMEOUT, || async {
+ sqlx::query_scalar!(
+ "SELECT b.bytes FROM whatsapp_group_attachment a
+ JOIN whatsapp_media_blob b ON b.document_id = a.document_id
+ WHERE a.binding_id = $1 AND a.source_id = $2 AND a.state = 'downloaded'",
+ binding.id,
+ &source
+ )
+ .fetch_optional(&world.pool)
+ .await
+ .ok()
+ .flatten()
+ })
+ .await;
+ let downloaded = world
+ .expect(downloaded, "the replica downloads the image for import")
+ .await;
+ assert_eq!(downloaded, photo);
+
+ replica.drain(TIMEOUT).await.unwrap();
+}
+
+#[sqlx::test(migrator = "MACRO_DB_MIGRATIONS")]
+async fn a_macro_attachment_reaches_the_group_intact(pool: PgPool) {
+ let world = World::new(pool).await;
+ let replica = world.replica().await;
+ let (_, binding) = world.alice_adopted().await;
+
+ let photo = b"a photo attached in Macro".repeat(40);
+ let uploaded = world
+ .linked
+ .upload_attachment(
+ &receipt::<ViewAccessLevel>(ALICE, binding.id),
+ "label.jpg",
+ "image/jpeg",
+ &photo,
+ SendAuthority::Human,
+ )
+ .await
+ .unwrap();
+ let send = world
+ .linked
+ .send_reply(
+ &receipt::<ViewAccessLevel>(ALICE, binding.id),
+ &SendReply {
+ idempotency_key: Uuid::now_v7(),
+ message: ReplyContent::Media {
+ upload_id: uploaded.upload_id,
+ caption: Some("return label".to_owned()),
+ },
+ reply_to: None,
+ },
+ SendAuthority::Human,
+ )
+ .await
+ .unwrap()
+ .id;
+ let (_, source_id) = world
+ .wait_send(send, &["accepted", "sent", "delivered", "read"])
+ .await;
+ let source_id = source_id.expect("a dispatched send has a source id");
+ let received = world
+ .customer
+ .wait_for_message(TIMEOUT, |message| message.id == source_id)
+ .await
+ .expect("the customer's phone receives the image");
+ let image = received
+ .message
+ .image_message
+ .as_deref()
+ .expect("an image message");
+ assert_eq!(image.caption.as_deref(), Some("return label"));
+ assert_eq!(
+ world.customer.download_image(&received).await.unwrap(),
+ photo
+ );
+
+ replica.drain(TIMEOUT).await.unwrap();
+}
+
+#[sqlx::test(migrator = "MACRO_DB_MIGRATIONS")]
+async fn a_message_queued_before_a_crash_is_ingested_once(pool: PgPool) {
+ let world = World::new(pool).await;
+ let first = world.replica().await;
+ let (device, binding) = world.alice_adopted().await;
+
+ // Hold ingestion back so the replica can only queue the event, then
+ // crash it in that window.
+ let mut blocker = world.pool.begin().await.unwrap();
+ sqlx::raw_sql("LOCK TABLE whatsapp_group_message IN SHARE ROW EXCLUSIVE MODE")
+ .execute(&mut *blocker)
+ .await
+ .unwrap();
+ let source = world
+ .customer
+ .send_text(&world.group, "queued, then crashed")
+ .await
+ .unwrap();
+ let pattern = format!("%{source}%");
+ let queued = wait_for_rows(&world.pool, TIMEOUT, || async {
+ let count = sqlx::query_scalar!(
+ r#"SELECT count(*) AS "count!" FROM whatsapp_inbound_event WHERE payload::text LIKE $1"#,
+ &pattern
+ )
+ .fetch_one(&world.pool)
+ .await
+ .ok()?;
+ (count > 0).then_some(())
+ })
+ .await;
+ world
+ .expect(queued, "the replica queues the event before acking it")
+ .await;
+ assert!(
+ world.whatsapp.wait_until_acked(&device, TIMEOUT).await,
+ "the node is acked once queued"
+ );
+ first.kill();
+ blocker.rollback().await.unwrap();
+
+ let second = world.replica().await;
+ let copies = world.timeline(&binding, &source).await;
+ assert_eq!(copies.len(), 1);
+ assert!(copies[0].contains("queued, then crashed"));
+ assert_eq!(world.whatsapp.delivery_count(&device, &source), 1);
+
+ second.drain(TIMEOUT).await.unwrap();
+}
+
+#[async_trait]
+impl whatsapp_linked::domain::ports::SendRateLimiter for Allow {
+ async fn check(&self, _: &str) -> LinkedResult<()> {
+ Ok(())
+ }
+}
diff --git a/crates/whatsapp_e2e/src/watch_tables.sql b/crates/whatsapp_e2e/src/watch_tables.sql
new file mode 100644
index 0000000000..70b76da275
--- /dev/null
+++ b/crates/whatsapp_e2e/src/watch_tables.sql
@@ -0,0 +1,26 @@
+-- Statement triggers that wake whatsapp_e2e::wait_for_rows. Test databases only.
+CREATE OR REPLACE FUNCTION e2e_notify_changed() RETURNS trigger AS $$
+BEGIN
+ PERFORM pg_notify('e2e_changed', TG_TABLE_NAME);
+ RETURN NULL;
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE TRIGGER e2e_changed AFTER INSERT OR UPDATE OR DELETE ON whatsapp_linked_account
+ FOR EACH STATEMENT EXECUTE FUNCTION e2e_notify_changed();
+CREATE TRIGGER e2e_changed AFTER INSERT OR UPDATE OR DELETE ON whatsapp_pairing
+ FOR EACH STATEMENT EXECUTE FUNCTION e2e_notify_changed();
+CREATE TRIGGER e2e_changed AFTER INSERT OR UPDATE OR DELETE ON whatsapp_group_roster
+ FOR EACH STATEMENT EXECUTE FUNCTION e2e_notify_changed();
+CREATE TRIGGER e2e_changed AFTER INSERT OR UPDATE OR DELETE ON whatsapp_group
+ FOR EACH STATEMENT EXECUTE FUNCTION e2e_notify_changed();
+CREATE TRIGGER e2e_changed AFTER INSERT OR UPDATE OR DELETE ON whatsapp_group_message
+ FOR EACH STATEMENT EXECUTE FUNCTION e2e_notify_changed();
+CREATE TRIGGER e2e_changed AFTER INSERT OR UPDATE OR DELETE ON whatsapp_message_observation
+ FOR EACH STATEMENT EXECUTE FUNCTION e2e_notify_changed();
+CREATE TRIGGER e2e_changed AFTER INSERT OR UPDATE OR DELETE ON whatsapp_inbound_event
+ FOR EACH STATEMENT EXECUTE FUNCTION e2e_notify_changed();
+CREATE TRIGGER e2e_changed AFTER INSERT OR UPDATE OR DELETE ON whatsapp_send
+ FOR EACH STATEMENT EXECUTE FUNCTION e2e_notify_changed();
+CREATE TRIGGER e2e_changed AFTER INSERT OR UPDATE OR DELETE ON whatsapp_group_attachment
+ FOR EACH STATEMENT EXECUTE FUNCTION e2e_notify_changed();
diff --git a/docs/README.md b/docs/README.md
index f5037ef77b..0824da274f 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -10,3 +10,6 @@
| [`agent-guide/`](agent-guide/) | How browser agents operate the app, not how to develop it. |
Docs that belong to one app, service, or crate live next to its code.
+
+WhatsApp linked devices: [architecture](architecture/whatsapp-linked-devices.md)
+and [External view walkthrough](agent-guide/external.md).
diff --git a/docs/architecture/whatsapp-linked-devices.md b/docs/architecture/whatsapp-linked-devices.md
new file mode 100644
index 0000000000..8ab0fd25f8
--- /dev/null
+++ b/docs/architecture/whatsapp-linked-devices.md
@@ -0,0 +1,162 @@
+# WhatsApp linked devices
+
+Personal WhatsApp accounts linked to Macro as devices. Each person connects
+their own account (QR), adopts groups they already belong to, and each adopted
+group becomes an independently shared `whatsapp_group` entity. Each person sends from their own
+account. Ported from the domain logic of #8304; the Go session service and its
+HTTP API were replaced.
+
+## Processes
+
+```
+Web app ──REST──▶ DSS ──── MacroDB ◀──── whatsapp_session_service (N replicas)
+ rows + NOTIFY one task + one client per leased account
+```
+
+- **DSS** serves `/meta/linked/*` (`crates/whatsapp_linked/src/inbound`),
+ authorizes people, writes group entities and intents, and publishes generic notifications.
+- **Session service** holds each account's advisory lock, runs the WhatsApp
+ socket, and calls `whatsapp_linked::domain::session::SessionService`
+ in-process. There is no service-to-service HTTP.
+
+| Signal | Payload | Sent by → read by | Meaning |
+| --- | --- | --- | --- |
+| `whatsapp_pairing_requested` | pairing id | DSS → session | start, update or stop (cancelled) pairing |
+| `whatsapp_send_requested` | account id | DSS → session | claim pending sends |
+| `whatsapp_groups_requested` | account id | DSS → session | refresh the account's group metadata |
+| `whatsapp_account_changed` | account id | DSS → session | consent or adopted groups changed; re-read |
+| `whatsapp_account_released` | account id | session ↔ session | lease handed over |
+| `whatsapp_group_effects` | message id | session → DSS | a group notification revision is ready |
+
+## Ownership and fencing
+
+`whatsapp_linked_account.lease_epoch` is bumped by `begin_lease` when a replica
+takes an account. Every session-service write locks the account row at that
+epoch; a stale epoch fails with `Fenced` and the replica must stop the account.
+`generation` is separate: it counts explicit consent (pairing, disconnect,
+logout) and fences memberships and send intents.
+
+## Pairing
+
+DSS `connect` creates or reuses the person's account (`pairing`, generation +1)
+and a five-minute `whatsapp_pairing` row, then signals. The replica reports QR
+or pairing codes with `update_pairing`; the browser polls `GET pairing/{id}`.
+`Success` connects the account only if the identity matches a reconnect's
+expected identity and belongs to no other person. Cancel and expiry end the
+attempt and disconnect a pairing account. Codes are cleared when the
+transaction ends.
+
+## Groups and adoption
+
+The replica keeps `whatsapp_group_roster` (the account's view of its groups)
+fresh with `update_groups`. Adoption requires metadata observed within two
+minutes; stale facts request a refresh and return `refreshing`. Adoption
+creates `whatsapp_group`, registers it as `EntityType::WhatsAppGroup`, grants the
+adopter Owner through `entity_access`, and records their remote membership facts.
+The caller supplies a name and optionally their team; the server selects their
+team when omitted. A team/group pair has one active adoption. Existing adoption
+requires view access and never grants a new viewer access implicitly.
+
+Viewing uses ordinary entity sharing: direct people grants and access requests
+at `/entities/whatsapp_group/{id}/...`, canonical explicit team grants, and link
+sharing. It requires no connected WhatsApp account or remote group membership.
+The group detail includes `share_permission` and `access_level`; its `/permissions`
+GET/PATCH pair drives the standard Share dialog. Only the actual owner changes
+team/link policy or detaches the group. Leaving WhatsApp or disconnecting keeps
+established access and history. Roster updates never create sharing grants.
+
+## Inbound, timeline and notifications
+
+The replica keeps only events for adopted groups, persists them before ACK and
+calls `ingest_event`. Unadopted events never enter storage. Every observer converges
+on `(group, verified person, source id)` in `whatsapp_group_message`; messages do
+not reference native comms storage. Sender associations use verified phone/LID
+aliases; ambiguous identities are quarantined. Missing reaction/revision targets
+are deferred. Edits follow source revision order; deletes are permanent tombstones.
+Reactions and media live in `whatsapp_group_reaction` and `whatsapp_group_attachment`.
+
+`GET /meta/linked/whatsapp-groups` lists shared groups with viewer-specific unread
+counts and sending eligibility. Detail and messages live under
+`/meta/linked/whatsapp-groups/{group_id}`. Messages page newest-first using stable
+`(sent_at,id)` keysets with an opaque message-id `before` cursor. The `/read`
+endpoint advances `whatsapp_group_read_state` monotonically; history and the
+viewer's own messages do not increase unread counts.
+
+DSS drains a durable notification outbox through the generic notification ingress.
+`whatsapp_group_message` notifications target directly shared users, team members,
+and known link viewers who have read the group, except the verified sender. Link
+viewers must still have access when recipients are resolved; unknown link visitors cannot be
+enumerated. Notifications use the message id for idempotency and are suppressed
+for history, replays, edits and reactions. They link to the standalone group entity.
+GraphQL notification targets use the canonical entity enum while groups remain outside Soup.
+
+## Sending
+
+DSS records a `whatsapp_send` intent (idempotency key, request digest, group
+sequence) only while the sender's account is connected, observing and allowed
+to post. The replica claims pending sends in group order, calls
+`begin_dispatch` with the WhatsApp message id it will use (all facts are
+re-checked; this is the point of no return), submits, then calls
+`record_send_outcome`. An acceptance writes the group message with the
+sender's identity; the later echo converges on it. A replica that dies after
+`begin_dispatch` leaves the send `unknown`; it is never sent again, and a later
+echo settles it. Unattempted intents expire after ten minutes. Edits and
+deletes are limited to the sender's own messages for ten minutes.
+
+## Rollout switches
+
+DSS config `WHATSAPP_LINKED_PAIRING_ENABLED`, `WHATSAPP_LINKED_ADOPTION_ENABLED`
+and `WHATSAPP_LINKED_SENDING_ENABLED` default to false. Disable sending first
+during an incident. Turning switches off keeps history and access.
+
+## Media
+
+Inbound attachments are reserved per ordinal at ingest, with the download
+locator sealed by the session service under the observing account's key. That
+account's replica claims the download (`claim_media_imports`), checks the
+SHA-256 and hands over the bytes (`complete_media_import`), which wait in
+`whatsapp_media_blob`. DSS (signal `whatsapp_media_ready`) creates a private
+document owned by the group owner. Attachment documents inherit current group
+view access; sharing and revocation apply to previously imported files too. Ready
+bytes are never rewritten. Failures retry with backoff and then expose an
+`unavailable` attachment state; message bodies remain plain WhatsApp text.
+
+Outbound: the attachments route stages one file per send in the blob table;
+`ReplyContent::Media` names it, the replica fetches it with `outbound_media`,
+and on acceptance it becomes the message's document, owned by the sender.
+
+## Passkey pairing
+
+When WhatsApp asks for a passkey, the replica reports `PairingUpdate::Passkey`
+with the WebAuthn request options, shown as `PairState.challenge`. The owner's
+native authenticator posts the assertion to `pairing/{id}/passkey`; DSS stores
+it on the pairing row and signals; the replica takes it once
+(`take_passkey_response`) and forwards it.
+
+## Durable inbound
+
+The replica persists each event with `enqueue_inbound` before acknowledging it
+to WhatsApp, then ingests asynchronously with `replay_deferred`.
+
+## Send request budget
+
+DSS accepts at most one deliberate WhatsApp send request per second per
+user, shared across groups and DSS replicas through its existing Redis rate
+limiter. The domain consumes the budget before evaluating a send; requests that
+later fail still count. Exhaustion returns `429` with code `rate_limited` and
+`Retry-After: 1`; an unavailable counter fails closed with `503`. This does not
+retry a send or change its idempotency key or unknown-outcome reconciliation.
+
+Only the person who owns a connection sends from it, using their session or their
+own API credential, including a verified Macro API JWT with signed agent attribution.
+The domain resolves the sending account from the authenticated person, never from
+the group owner or request input. Media uploads follow the same owner policy.
+Bot, harness and internal principals never send, including bots acting for a user.
+Pairing reads (including QR codes and passkey challenges) and account/access
+controls require the person's bearer session; API credentials cannot perform them.
+
+## Follow-ups
+
+- Soup/search registration is deferred; External, shared links and notifications
+ are the group entry points.
+- Participant receipts are stored but not shown.