Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.
TypeScript agentaiai-agentsall-in-onecrmcrm-systememaillinearmcpmessagingnotesnotionofficerustslackslack-alternativestartupstartup-toolstypescriptworkspace
test(whatsapp): run the linked-device fleet end to end and document it (#8541)
Wolf Mermelsteinb310044eb227
diff --git a/.github/workspace-dep-closures.json b/.github/workspace-dep-closures.jsonindex e59b130178..2f79d186e0 100644--- a/.github/workspace-dep-closures.json+++ b/.github/workspace-dep-closures.json@@ -13951,6 +13951,59 @@"crates/webhook_signature","crates/workspace-hack"],+ "whatsapp_e2e": [+ "crates/agent",+ "crates/ai_toolset",+ "crates/ai_usage",+ "crates/attachment",+ "crates/bot_id",+ "crates/bot_token",+ "crates/cowlike",+ "crates/database_env_vars",+ "crates/document_sub_type",+ "crates/entity_access",+ "crates/entity_access_db_utils",+ "crates/entity_registry/entity_registry",+ "crates/entity_registry/entity_registry_db_utils",+ "crates/entity_registry/shared_entity_registry",+ "crates/genai_telemetry",+ "crates/harness_id",+ "crates/harness_token",+ "crates/macro_auth",+ "crates/macro_authorization",+ "crates/macro_aws_config",+ "crates/macro_config",+ "crates/macro_config_derive",+ "crates/macro_entrypoint",+ "crates/macro_env",+ "crates/macro_env_var",+ "crates/macro_service_urls",+ "crates/macro_user_id",+ "crates/macro_uuid",+ "crates/model",+ "crates/model-entity",+ "crates/model-error-response",+ "crates/model_file_type",+ "crates/model_owner",+ "crates/model_user",+ "crates/models_bulk_upload",+ "crates/models_entity_access_management",+ "crates/models_pagination",+ "crates/models_permissions",+ "crates/non_empty",+ "crates/prompt",+ "crates/remote_env_var",+ "crates/share_permission_db_utils",+ "crates/whatsapp_e2e",+ "crates/whatsapp_fake_server",+ "crates/whatsapp_linked",+ "crates/whatsapp_store_pg",+ "crates/whatsmeow",+ "crates/whatsmeow_proto",+ "crates/whatsmeow_signal",+ "crates/workspace-hack",+ "services/whatsapp_session_service"+ ],"whatsapp_fake_server": ["crates/whatsapp_fake_server","crates/whatsmeow",diff --git a/Cargo.lock b/Cargo.lockindex 678d0959df..0bac64528d 100644--- a/Cargo.lock+++ b/Cargo.lock@@ -21569,6 +21569,30 @@ version = "0.1.12"source = "registry+https://github.com/rust-lang/crates.io-index"checksum = "a28ac98ddc8b9274cb41bb4d9d4d5c425b6020c50c46f25559911905610b4a88"+[[package]]+name = "whatsapp_e2e"+version = "0.1.0"+dependencies = [+ "async-trait",+ "chrono",+ "entity_access",+ "entity_access_db_utils",+ "macro_db_migrator",+ "sqlx",+ "tokio",+ "tokio-util",+ "tracing",+ "uuid",+ "whatsapp_fake_server",+ "whatsapp_linked",+ "whatsapp_session_service",+ "whatsapp_store_pg",+ "whatsmeow",+ "whatsmeow_signal",+ "workspace-hack",+ "zeroize",+]+[[package]]name = "whatsapp_fake_server"version = "0.1.0"diff --git a/Cargo.toml b/Cargo.tomlindex f51432ea13..f8879db36f 100644--- a/Cargo.toml+++ b/Cargo.toml@@ -130,6 +130,7 @@ members = ["crates/unfurl","crates/user_api_key","crates/webhook_signature",+ "crates/whatsapp_e2e","crates/whatsapp_fake_server","crates/whatsapp_linked","crates/whatsapp_store_pg",diff --git a/crates/whatsapp_e2e/AGENTS.md b/crates/whatsapp_e2e/AGENTS.mdnew file mode 100644index 0000000000..819ce4d885--- /dev/null+++ b/crates/whatsapp_e2e/AGENTS.md@@ -0,0 +1,54 @@+# whatsapp_e2e++## What it is++End-to-end tests for WhatsApp linked devices. Real session-service replicas+(supervisor, advisory locks, account runners, whatsmeow clients, the encrypted+Postgres device store) run against `whatsapp_fake_server`. The DSS side runs+through `whatsapp_linked::LinkedService`, and both share one `#[sqlx::test]`+database. Nothing talks to WhatsApp or AWS (the KMS is+`whatsapp_store_pg`'s `FakeWhatsAppKms`).++## How it works++- `Replica::start` runs one replica on its own Tokio runtime, built by the+ service's `replica::start` with `test_signal()`, the fake server's dialer, a+ shared fake KMS and short timings. `kill()` drops the runtime: sockets, lock+ connection and client die at once, like a crashed process. `drain()` is+ SIGTERM.+- `watch_tables` adds statement triggers that `NOTIFY e2e_changed` on the+ standalone group and linked-account tables; `wait_for_rows` re-checks after each notification+ instead of polling.+- Scenarios (`src/test.rs`): pair by QR through DSS, adopt, inbound once with+ two observing accounts, send from your own number, kill takeover without+ loss or duplicates, drain handover, crash between enqueue and ingest, inbound+ image download for import, outbound attachment, a send cut off after+ submission is `unknown` and never resent, a send without an ack times out as+ `unknown`, and a real client's group events through the service's mapping.++## Checks++- `cargo test -p whatsapp_e2e` with `SQLX_OFFLINE` unset and `DATABASE_URL`+ pointing at a migrated MacroDB (the query macros check against it; each test+ gets its own database on that server). If the shared `macrodb` is stale,+ create and migrate your own `macrodb_<name>` with+ `sqlx migrate run --source crates/macro_db_client/migrations`.++## Gotchas++- sqlx derives test database names from test paths: two concurrent runs of+ this crate against one Postgres server collide (`database ... already+ exists`). Run one at a time per server.+- Fleet transport tests inject a permissive `SendRateLimiter`; DSS request throttling is tested at the domain and Redis-adapter boundaries separately.+- The change triggers live in `src/watch_tables.sql`; add a table there when a+ test waits on it.+- A failed ingest wait prints the `whatsapp_inbound_event` and+ `whatsapp_group_message` rows; start debugging there.+- Fake phones send each group message as a real phone does: a sender key+ distribution message in a per-device `pkmsg`, plus the `skmsg`, under one+ message ID. The service must ignore the key-only part.++## Related++- `crates/whatsapp_fake_server/AGENTS.md`, `services/whatsapp_session_service/AGENTS.md`,+ `crates/whatsapp_linked/AGENTS.md`diff --git a/crates/whatsapp_e2e/CLAUDE.md b/crates/whatsapp_e2e/CLAUDE.mdnew file mode 120000index 0000000000..47dc3e3d86--- /dev/null+++ b/crates/whatsapp_e2e/CLAUDE.md@@ -0,0 +1 @@+AGENTS.md\ No newline at end of filediff --git a/crates/whatsapp_e2e/Cargo.toml b/crates/whatsapp_e2e/Cargo.tomlnew file mode 100644index 0000000000..ae36881766--- /dev/null+++ b/crates/whatsapp_e2e/Cargo.toml@@ -0,0 +1,28 @@+[package]+edition = "2024"+name = "whatsapp_e2e"+publish = false+version = "0.1.0"+description = "End-to-end tests of WhatsApp linked devices: session-service replicas, the linked-devices domain and MacroDB against a fake WhatsApp server."++[dependencies]+entity_access_db_utils = { path = "../entity_access_db_utils" }+async-trait = { workspace = true }+entity_access = { path = "../entity_access", default-features = false, features = ["ports"] }+sqlx = { workspace = true }+tokio = { workspace = true, features = ["macros", "rt-multi-thread", "sync", "time"] }+tokio-util = { workspace = true, features = ["rt"] }+tracing = { workspace = true }+uuid = { workspace = true }+whatsapp_fake_server = { path = "../whatsapp_fake_server" }+whatsapp_linked = { path = "../whatsapp_linked", features = ["outbound"] }+whatsapp_session_service = { path = "../../services/whatsapp_session_service" }+whatsapp_store_pg = { path = "../whatsapp_store_pg", features = ["fake-kms"] }+whatsmeow = { path = "../whatsmeow" }+workspace-hack = { version = "0.1", path = "../workspace-hack" }+zeroize = { workspace = true }++[dev-dependencies]+chrono = { workspace = true }+whatsmeow_signal = { path = "../whatsmeow_signal" }+macro_db_migrator = { path = "../macro_db_migrator" }diff --git a/crates/whatsapp_e2e/src/lib.rs b/crates/whatsapp_e2e/src/lib.rsnew file mode 100644index 0000000000..b5848017c6--- /dev/null+++ b/crates/whatsapp_e2e/src/lib.rs@@ -0,0 +1,252 @@+//! End-to-end tests of WhatsApp linked devices.+//!+//! The tests run the real session service (supervisor, advisory locks,+//! account runners, whatsmeow clients, the Postgres device store) against a+//! [`whatsapp_fake_server::FakeWhatsApp`], and drive the DSS side through the+//! linked-devices domain service, all on one test database.+//!+//! Each [`Replica`] runs on its own Tokio runtime, like a separate process:+//! [`Replica::kill`] drops every task and socket at once (the lock connection+//! closes, so Postgres frees its advisory locks), and [`Replica::drain`] is+//! what SIGTERM does.++#![deny(missing_docs)]+// The account runner future nests the domain, store and client futures.+#![recursion_limit = "256"]++#[cfg(test)]+mod test;++use std::collections::HashMap;+use std::sync::Arc;+use std::time::Duration;++use sqlx::postgres::{PgConnectOptions, PgListener, PgPool, PgPoolOptions};+use tokio::sync::oneshot;+use tokio_util::sync::CancellationToken;+use whatsapp_fake_server::{FakeWhatsApp, test_signal};+use whatsapp_session_service::replica::{self, ReplicaDeps, Timings};+use whatsapp_store_pg::kms::{FakeWhatsAppKms, GeneratedDataKey, WhatsAppKms, WhatsAppKmsError};+use whatsmeow::{Dialer, WebSocketDialer};+use zeroize::Zeroizing;++/// How long a submitted send waits for the server's ack before it is+/// recorded as unknown. Short, so the unknown path runs within a test.+pub const SEND_TIMEOUT: Duration = Duration::from_secs(5);++/// Replica timings shortened so takeovers and timeouts happen within a test.+pub const TIMINGS: Timings = Timings {+ lock_keepalive: Duration::from_secs(1),+ send_timeout: SEND_TIMEOUT,+ rescan_every: Duration::from_secs(1),+ tick_every: Duration::from_secs(1),+ release_timeout: Duration::from_secs(7),+ stop_timeout: Duration::from_secs(5),+ drain_deadline: Duration::from_secs(30),+ health_every: Duration::from_secs(60),+};++/// One fake KMS shared by every replica, as they share the real KMS key.+#[derive(Clone)]+pub struct SharedKms(pub Arc<FakeWhatsAppKms>);++impl SharedKms {+ /// A fresh fake KMS.+ pub fn new() -> SharedKms {+ SharedKms(Arc::new(FakeWhatsAppKms::new("e2e-kms-key")))+ }+}++impl Default for SharedKms {+ fn default() -> Self {+ SharedKms::new()+ }+}++impl WhatsAppKms for SharedKms {+ async fn generate_data_key(+ &self,+ encryption_context: HashMap<String, String>,+ ) -> Result<GeneratedDataKey, WhatsAppKmsError> {+ self.0.generate_data_key(encryption_context).await+ }++ async fn decrypt(+ &self,+ kms_key_id: &str,+ ciphertext: &[u8],+ encryption_context: HashMap<String, String>,+ ) -> Result<Zeroizing<Vec<u8>>, WhatsAppKmsError> {+ self.0+ .decrypt(kms_key_id, ciphertext, encryption_context)+ .await+ }+}++/// A running session-service replica.+pub struct Replica {+ runtime: Option<tokio::runtime::Runtime>,+ shutdown: CancellationToken,+ finished: Option<oneshot::Receiver<Result<(), String>>>,+ /// The Postgres backend of the replica's lock connection.+ pub lock_backend_pid: i32,+}++impl Drop for Replica {+ fn drop(&mut self) {+ if let Some(runtime) = self.runtime.take() {+ runtime.shutdown_background();+ }+ }+}++impl Replica {+ /// Starts a replica against the test database and the fake server, the+ /// with the service's own `replica::start`.+ pub async fn start(+ database: &PgConnectOptions,+ whatsapp: &FakeWhatsApp,+ kms: &SharedKms,+ ) -> Replica {+ let runtime = tokio::runtime::Builder::new_multi_thread()+ .worker_threads(2)+ .enable_all()+ .build()+ .expect("a replica runtime starts");+ let shutdown = CancellationToken::new();+ let (ready, started) = oneshot::channel();+ let (done, finished) = oneshot::channel();+ let database = database.clone();+ let dialer: Arc<dyn Dialer> = Arc::new(WebSocketDialer {+ url: whatsapp.websocket_url(),+ });+ let media_base_url = whatsapp.media_base_url();+ let kms = kms.clone();+ let stop = shutdown.clone();+ runtime.spawn(async move {+ let result = run_replica(database, dialer, media_base_url, kms, stop, ready).await;+ let _ = done.send(result);+ });+ let lock_backend_pid = started+ .await+ .expect("the replica reports startup")+ .expect("the replica starts");+ Replica {+ runtime: Some(runtime),+ shutdown,+ finished: Some(finished),+ lock_backend_pid,+ }+ }++ /// SIGTERM: drain every account and wait for the supervisor to return.+ pub async fn drain(mut self, timeout: Duration) -> Result<(), String> {+ self.shutdown.cancel();+ let finished = self.finished.take().expect("a replica drains once");+ let result = tokio::time::timeout(timeout, finished)+ .await+ .map_err(|_| "the replica did not finish draining".to_owned())?+ .map_err(|_| "the replica task died".to_owned())?;+ if let Some(runtime) = self.runtime.take() {+ runtime.shutdown_background();+ }+ result+ }++ /// Process death: every task, websocket and database connection of the+ /// replica goes away at once.+ pub fn kill(mut self) {+ if let Some(runtime) = self.runtime.take() {+ runtime.shutdown_background();+ }+ }+}++async fn run_replica(+ database: PgConnectOptions,+ dialer: Arc<dyn Dialer>,+ media_base_url: String,+ kms: SharedKms,+ shutdown: CancellationToken,+ ready: oneshot::Sender<Result<i32, String>>,+) -> Result<(), String> {+ let started = async {+ // CI runs these beside the whole workspace's database tests on one+ // Postgres (max_connections = 100); two accounts need only a few.+ let pool = PgPoolOptions::new()+ .max_connections(4)+ .connect_with(database.clone())+ .await+ .map_err(|error| error.to_string())?;+ replica::start(+ ReplicaDeps {+ pool,+ lock_options: database,+ kms,+ signal: test_signal(),+ dialer,+ media_base_url: Some(media_base_url),+ max_accounts: 50,+ drain_batch: 20,+ timings: TIMINGS,+ },+ shutdown,+ )+ .await+ .map_err(|error| error.to_string())+ }+ .await;+ let running = match started {+ Ok(running) => {+ let _ = ready.send(Ok(running.lock_backend_pid()));+ running+ }+ Err(error) => {+ let _ = ready.send(Err(error.clone()));+ return Err(error);+ }+ };+ running.wait().await.map_err(|error| error.to_string())+}++/// The channel the test triggers notify on.+const CHANGED_CHANNEL: &str = "e2e_changed";++/// Makes the linked-device tables `NOTIFY e2e_changed` on every write, so+/// tests can wait for rows the service writes without polling. Test+/// databases only.+pub async fn watch_tables(pool: &PgPool) {+ sqlx::raw_sql(include_str!("watch_tables.sql"))+ .execute(pool)+ .await+ .expect("the change triggers are created");+}++/// Waits until `check` finds something, re-running it after every write to+/// a watched table (see [`watch_tables`]).+pub async fn wait_for_rows<Found, Check, Checking>(+ pool: &PgPool,+ timeout: Duration,+ check: Check,+) -> Option<Found>+where+ Check: Fn() -> Checking,+ Checking: Future<Output = Option<Found>>,+{+ let mut listener = PgListener::connect_with(pool)+ .await+ .expect("a change listener connects");+ listener+ .listen(CHANGED_CHANNEL)+ .await+ .expect("the change listener subscribes");+ let wait = async {+ loop {+ if let Some(found) = check().await {+ return found;+ }+ listener.recv().await.expect("the change listener stays up");+ }+ };+ tokio::time::timeout(timeout, wait).await.ok()+}diff --git a/crates/whatsapp_e2e/src/test.rs b/crates/whatsapp_e2e/src/test.rsnew file mode 100644index 0000000000..e0baedbe52--- /dev/null+++ b/crates/whatsapp_e2e/src/test.rs@@ -0,0 +1,859 @@+//! Linked WhatsApp accounts end to end: pairing, adoption, inbound messages,+//! sends, failover and drain, against the fake WhatsApp server.++use std::sync::Arc;+use std::time::Duration;++use async_trait::async_trait;+use entity_access::domain::models::{+ AccessLevel, Entity, EntityAccessAuth, EntityAccessReceipt, EntityPermission, EntityType,+ ViewAccessLevel,+};+use macro_db_migrator::MACRO_DB_MIGRATIONS;+use sqlx::{Connection, PgConnection, PgPool};+use uuid::Uuid;+use whatsapp_fake_server::{FakePhone, FakeWhatsApp};+use whatsapp_linked::domain::models::{+ AccountState, AdoptGroup, Binding, Features, ReplyContent, SendAuthority, SendReply,+};+use whatsapp_linked::domain::ports::{Authorizer, LinkedResult};+use whatsapp_linked::domain::service::LinkedService;+use whatsapp_linked::outbound::pg::PgLinkedRepository;+use whatsmeow::types::Jid;++use super::*;++const TIMEOUT: Duration = Duration::from_secs(60);+const ALICE: &str = "macro|[email protected]";+const BOB: &str = "macro|[email protected]";++/// Runs one scenario at a time across nextest's test processes. Each one+/// holds a dozen or more connections (replica pools, lock connection,+/// listeners); in parallel with the rest of the workspace they exhaust CI's+/// Postgres (`max_connections` = 100). Advisory locks are per database, so+/// the lock lives in the shared `postgres` database, not the test's own.+async fn take_fleet_slot(pool: &PgPool) -> PgConnection {+ let options = (*pool.connect_options()).clone().database("postgres");+ let mut slot = PgConnection::connect_with(&options)+ .await+ .expect("the fleet lock connection opens");+ sqlx::raw_sql("SELECT pg_advisory_lock(7346525012)")+ .execute(&mut slot)+ .await+ .expect("the fleet lock is taken");+ slot+}++struct Allow;++#[async_trait]+impl Authorizer for Allow {+ async fn require_group_owner(&self, _: Uuid, _: &str) -> LinkedResult<()> {+ Ok(())+ }+}++/// The database, the DSS-side domain service and the fake WhatsApp world:+/// Alice and Bob (Macro users with their own WhatsApp) and a customer who is+/// only on WhatsApp, all in one group.+struct World {+ /// Holds the fleet lock until the scenario ends (see [`take_fleet_slot`]).+ _fleet_slot: PgConnection,+ pool: PgPool,+ linked: LinkedService,+ whatsapp: FakeWhatsApp,+ kms: SharedKms,+ alice: FakePhone,+ bob: FakePhone,+ customer: FakePhone,+ group: Jid,+}++impl World {+ async fn new(pool: PgPool) -> World {+ let fleet_slot = take_fleet_slot(&pool).await;+ watch_tables(&pool).await;+ for id in [ALICE, BOB] {+ user(&pool, id).await;+ }+ let team = Uuid::now_v7();+ sqlx::query!(+ "INSERT INTO team (id, name, owner_id) VALUES ($1, 'Linked', $2)",+ team,+ ALICE+ )+ .execute(&pool)+ .await+ .unwrap();+ for (id, role) in [(ALICE, "owner"), (BOB, "member")] {+ sqlx::query!(+ "INSERT INTO team_user (team_id, user_id, team_role) VALUES ($1, $2, $3::text::team_role)",+ team,+ id,+ role+ )+ .execute(&pool)+ .await+ .unwrap();+ }+ let linked = LinkedService::new(+ Features {+ pairing: true,+ adoption: true,+ sending: true,+ },+ Arc::new(PgLinkedRepository::new(pool.clone())),+ Arc::new(Allow),+ Arc::new(Allow),+ );+ let whatsapp = FakeWhatsApp::start().await.unwrap();+ let alice = whatsapp.add_user("15550000001", "Alice").await;+ let bob = whatsapp.add_user("15550000002", "Bob").await;+ let customer = whatsapp.add_user("15558880001", "Customer").await;+ let group = whatsapp.create_group("Customer escalations", &[&customer, &alice, &bob]);+ World {+ _fleet_slot: fleet_slot,+ pool,+ linked,+ whatsapp,+ kms: SharedKms::new(),+ alice,+ bob,+ customer,+ group,+ }+ }++ async fn replica(&self) -> Replica {+ Replica::start(&self.pool.connect_options(), &self.whatsapp, &self.kms).await+ }++ /// Pairs `user` through DSS: start a pairing, wait for the replica's QR,+ /// scan it with `phone`, wait for the account to connect. Returns the+ /// linked device.+ async fn pair(&self, user: &str, phone: &FakePhone) -> Jid {+ let pairing = self+ .linked+ .connect(user, false, SendAuthority::Human)+ .await+ .unwrap();+ let qr = wait_for_rows(&self.pool, TIMEOUT, || async {+ self.linked+ .pair_state(user, pairing.id, SendAuthority::Human)+ .await+ .ok()?+ .qr+ })+ .await+ .expect("the replica shows a QR code");+ let device = phone+ .scan_qr(&qr)+ .await+ .expect("the phone approves the QR code");+ let identity = phone.jid().to_string();+ wait_for_rows(&self.pool, TIMEOUT, || async {+ let account = self.linked.account(user).await.ok()??;+ (account.state == AccountState::Connected+ && account.identity_jid.as_deref() == Some(identity.as_str()))+ .then_some(())+ })+ .await+ .expect("the account connects as the phone's number");+ device+ }++ /// Adopts the group for `user` once the replica has listed it.+ async fn adopt(&self, user: &str, users: &[&str]) -> Binding {+ let group = self.group.to_string();+ wait_for_rows(&self.pool, TIMEOUT, || async {+ let groups = self.linked.groups(user).await.ok()?;+ groups.iter().any(|info| info.jid == group).then_some(())+ })+ .await+ .expect("the replica lists the group");+ let binding = self+ .linked+ .adopt(+ user,+ &AdoptGroup {+ group_jid: group,+ name: Some("Escalations".to_owned()),+ },+ SendAuthority::Human,+ )+ .await+ .unwrap();+ let mut tx = self.pool.begin().await.unwrap();+ for viewer in users {+ entity_access_db_utils::direct_share::set(+ &mut tx,+ &binding.id,+ EntityType::WhatsAppGroup,+ &(*viewer).to_owned().try_into().unwrap(),+ Some(AccessLevel::View),+ false,+ )+ .await+ .unwrap();+ }+ tx.commit().await.unwrap();+ binding+ }++ /// Waits until `user`'s own account observes the group shared with them,+ /// which happens on the account's next roster read; ask for one.+ async fn observe(&self, user: &str, binding: &Binding) {+ sqlx::query!(+ "SELECT pg_notify('whatsapp_groups_requested', id::text) FROM whatsapp_linked_account WHERE user_id = $1",+ user+ )+ .execute(&self.pool)+ .await+ .unwrap();+ let observing = wait_for_rows(&self.pool, TIMEOUT, || async {+ sqlx::query_scalar!(+ r#"SELECT m.account_id FROM whatsapp_group_membership m+ JOIN whatsapp_linked_account a ON a.id = m.account_id+ WHERE m.binding_id = $1 AND a.user_id = $2 AND m.consent_ended_at IS NULL AND m.remote_member"#,+ binding.id,+ user+ )+ .fetch_optional(&self.pool)+ .await+ .unwrap()+ })+ .await;+ self.expect(observing, "the viewer's account observes the shared group")+ .await;+ }++ /// Alice paired and observing the adopted group.+ async fn alice_adopted(&self) -> (Jid, Binding) {+ let device = self.pair(ALICE, &self.alice).await;+ let binding = self.adopt(ALICE, &[]).await;+ (device, binding)+ }++ /// Waits until the WhatsApp message `source_id` is a standalone group message and+ /// returns every stored copy of it (there must be one).+ async fn timeline(&self, binding: &Binding, source_id: &str) -> Vec<String> {+ let copies = wait_for_rows(&self.pool, TIMEOUT, || async {+ let copies = group_messages(&self.pool, binding.id, source_id).await;+ (!copies.is_empty()).then_some(copies)+ })+ .await;+ match copies {+ Some(copies) => copies,+ None => {+ let mut whatsapp = String::new();+ for phone in [&self.alice, &self.bob] {+ for device in self.whatsapp.linked_devices(phone) {+ whatsapp.push_str(&format!(+ " {device}: online {}, deliveries {}, pending {}\n",+ self.whatsapp.is_online(&device),+ self.whatsapp.delivery_count(&device, source_id),+ self.whatsapp.pending_count(&device),+ ));+ }+ }+ panic!(+ "message {source_id} was not ingested\nfake WhatsApp:\n{whatsapp}{}",+ self.dump().await+ )+ }+ }+ }++ /// `found`, or a panic that shows the linked-device rows.+ async fn expect<Found>(&self, found: Option<Found>, what: &str) -> Found {+ match found {+ Some(found) => found,+ None => panic!("{what}\n{}", self.dump().await),+ }+ }++ /// The linked-device rows a failed wait should explain itself with.+ async fn dump(&self) -> String {+ let pool = &self.pool;+ let sections = [+ (+ "inbound events (account, state, kind)",+ sqlx::query_scalar!(+ r#"SELECT account_id::text || ' ' || state || ' ' || coalesce(payload->>'kind', '') AS "row!" FROM whatsapp_inbound_event"#+ )+ .fetch_all(pool)+ .await,+ ),+ (+ "external messages (source, origin, message)",+ sqlx::query_scalar!(+ r#"SELECT source_id || ' ' || origin || ' ' || coalesce(message_id::text, '-') AS "row!" FROM whatsapp_group_message"#+ )+ .fetch_all(pool)+ .await,+ ),+ (+ "observations (source, account)",+ sqlx::query_scalar!(+ r#"SELECT source_id || ' ' || account_id::text AS "row!" FROM whatsapp_message_observation"#+ )+ .fetch_all(pool)+ .await,+ ),+ (+ "accounts (id, state, lease epoch)",+ sqlx::query_scalar!(+ r#"SELECT id::text || ' ' || state || ' ' || lease_epoch AS "row!" FROM whatsapp_linked_account"#+ )+ .fetch_all(pool)+ .await,+ ),+ (+ "sends (state, source)",+ sqlx::query_scalar!(+ r#"SELECT state || ' ' || coalesce(allocated_source_id, '-') AS "row!" FROM whatsapp_send"#+ )+ .fetch_all(pool)+ .await,+ ),+ ];+ let mut dump = String::new();+ for (title, rows) in sections {+ dump.push_str(&format!("{title}:\n"));+ let rows = rows.unwrap_or_else(|error| vec![format!("query failed: {error}")]);+ for row in rows {+ dump.push_str(&format!(" {row}\n"));+ }+ }+ dump+ }++ async fn send(&self, user: &str, binding: &Binding, text: &str) -> Uuid {+ self.linked+ .send_reply(+ &receipt::<ViewAccessLevel>(user, binding.id),+ &SendReply {+ idempotency_key: Uuid::now_v7(),+ message: ReplyContent::Text {+ text: text.to_owned(),+ },+ reply_to: None,+ },+ SendAuthority::Human,+ )+ .await+ .unwrap()+ .id+ }++ async fn send_state(&self, send: Uuid) -> (String, Option<String>) {+ let row = sqlx::query!(+ "SELECT state, allocated_source_id FROM whatsapp_send WHERE id = $1",+ send+ )+ .fetch_one(&self.pool)+ .await+ .unwrap();+ (row.state, row.allocated_source_id)+ }++ /// Waits until the send's state is one of `states`.+ async fn wait_send(&self, send: Uuid, states: &[&str]) -> (String, Option<String>) {+ let state = wait_for_rows(&self.pool, TIMEOUT, || async {+ let state = self.send_state(send).await;+ states.contains(&state.0.as_str()).then_some(state)+ })+ .await;+ self.expect(state, &format!("the send reaches one of {states:?}"))+ .await+ }+}++async fn user(pool: &PgPool, id: &str) {+ let email = id.trim_start_matches("macro|");+ let macro_user = Uuid::now_v7();+ sqlx::query!(+ "INSERT INTO macro_user (id, username, email, stripe_customer_id) VALUES ($1, $2, $3, $2)",+ macro_user,+ id,+ )+ .execute(pool)+ .await+ .unwrap();+ sqlx::query!(+ r#"INSERT INTO "User" (id, email, macro_user_id) VALUES ($1, $2, $3)"#,+ id,+ email,+ macro_user+ )+ .execute(pool)+ .await+ .unwrap();+}++fn receipt<Permission: entity_access::domain::models::RequiredPermission>(+ user: &str,+ channel: Uuid,+) -> EntityAccessReceipt<Permission> {+ EntityAccessReceipt::try_new(+ EntityAccessAuth::Authenticated(user.to_owned().try_into().unwrap()),+ Entity {+ entity_id: channel.to_string(),+ entity_type: EntityType::WhatsAppGroup,+ },+ EntityPermission::AccessLevel {+ access_level: AccessLevel::View,+ },+ )+ .unwrap()+}++/// The content of every standalone message for one WhatsApp message.+async fn group_messages(pool: &PgPool, binding: Uuid, source_id: &str) -> Vec<String> {+ sqlx::query_scalar!(+ "SELECT body FROM whatsapp_group_message WHERE binding_id=$1 AND source_id=$2 AND message_id IS NOT NULL",+ binding,+ source_id+ )+ .fetch_all(pool)+ .await+ .unwrap()+}++async fn observations(pool: &PgPool, binding: Uuid, source_id: &str) -> i64 {+ sqlx::query_scalar!(+ r#"SELECT count(*) AS "count!" FROM whatsapp_message_observation WHERE binding_id = $1 AND source_id = $2"#,+ binding,+ source_id+ )+ .fetch_one(pool)+ .await+ .unwrap()+}++#[sqlx::test(migrator = "MACRO_DB_MIGRATIONS")]+async fn pair_adopt_receive_once_and_send_from_your_own_number(pool: PgPool) {+ let world = World::new(pool).await;+ let replica = world.replica().await;+ world.pair(ALICE, &world.alice).await;+ world.pair(BOB, &world.bob).await;+ let binding = world.adopt(ALICE, &[BOB]).await;+ world.observe(BOB, &binding).await;++ // Two linked accounts see the customer's message; it lands once.+ let source = world+ .customer+ .send_text(&world.group, "my order is late")+ .await+ .unwrap();+ let copies = world.timeline(&binding, &source).await;+ assert_eq!(copies.len(), 1);+ assert!(copies[0].contains("my order is late"), "{copies:?}");+ let observed = wait_for_rows(&world.pool, TIMEOUT, || async {+ (observations(&world.pool, binding.id, &source).await == 2).then_some(())+ })+ .await;+ world+ .expect(observed, "both accounts observe the message")+ .await;+ assert_eq!(+ group_messages(&world.pool, binding.id, &source).await.len(),+ 1+ );++ // Alice replies from Macro; the customer gets it from Alice's number.+ let send = world.send(ALICE, &binding, "sorry, on it").await;+ let received = world+ .customer+ .wait_for_message(TIMEOUT, |message| message.text() == Some("sorry, on it"))+ .await+ .expect("the customer's phone receives the reply");+ assert_eq!(received.sender_phone, world.alice.jid());+ assert_eq!(received.chat, world.group);+ let (_, source_id) = world+ .wait_send(send, &["accepted", "sent", "delivered", "read"])+ .await;+ assert_eq!(source_id.as_deref(), Some(received.id.as_str()));++ // A shared second sender uses their own connection and phone number.+ let second_send = world.send(BOB, &binding, "Bob here too").await;+ let second_reply = world+ .customer+ .wait_for_message(TIMEOUT, |message| message.text() == Some("Bob here too"))+ .await+ .expect("the shared viewer sends from their own WhatsApp");+ assert_eq!(second_reply.sender_phone, world.bob.jid());+ world+ .wait_send(second_send, &["accepted", "sent", "delivered", "read"])+ .await;++ replica.drain(TIMEOUT).await.unwrap();+}++#[sqlx::test(migrator = "MACRO_DB_MIGRATIONS")]+async fn a_killed_replica_is_taken_over_without_losing_or_duplicating(pool: PgPool) {+ let world = World::new(pool).await;+ let first = world.replica().await;+ let (device, binding) = world.alice_adopted().await;+ let second = world.replica().await;++ let before = world+ .customer+ .send_text(&world.group, "before")+ .await+ .unwrap();+ assert_eq!(world.timeline(&binding, &before).await.len(), 1);++ first.kill();+ assert!(world.whatsapp.wait_until_offline(&device, TIMEOUT).await);+ let during = world+ .customer+ .send_text(&world.group, "during")+ .await+ .unwrap();+ let after_takeover = world.whatsapp.wait_until_online(&device, TIMEOUT).await;+ assert!(after_takeover, "the second replica takes the account over");+ let after = world+ .customer+ .send_text(&world.group, "after")+ .await+ .unwrap();++ for (source, text) in [(&before, "before"), (&during, "during"), (&after, "after")] {+ let copies = world.timeline(&binding, source).await;+ assert_eq!(copies.len(), 1, "{text} lands exactly once");+ assert!(copies[0].contains(text));+ }+ let send = world.send(ALICE, &binding, "still here").await;+ world+ .wait_send(send, &["accepted", "sent", "delivered", "read"])+ .await;+ world+ .customer+ .wait_for_message(TIMEOUT, |message| message.text() == Some("still here"))+ .await+ .expect("sends work after the takeover");++ second.drain(TIMEOUT).await.unwrap();+}++#[sqlx::test(migrator = "MACRO_DB_MIGRATIONS")]+async fn a_drained_replica_hands_over_without_loss(pool: PgPool) {+ let world = World::new(pool).await;+ let first = world.replica().await;+ let (device, binding) = world.alice_adopted().await;+ let second = world.replica().await;++ let before = world+ .customer+ .send_text(&world.group, "before drain")+ .await+ .unwrap();+ assert_eq!(world.timeline(&binding, &before).await.len(), 1);+ let send = world.send(ALICE, &binding, "sent while draining").await;++ first+ .drain(TIMEOUT)+ .await+ .expect("the replica drains cleanly");+ let during = world+ .customer+ .send_text(&world.group, "during handover")+ .await+ .unwrap();+ assert!(world.whatsapp.wait_until_online(&device, TIMEOUT).await);++ assert_eq!(world.timeline(&binding, &during).await.len(), 1);+ assert_eq!(+ group_messages(&world.pool, binding.id, &before).await.len(),+ 1+ );+ let (_, source_id) = world+ .wait_send(send, &["accepted", "sent", "delivered", "read"])+ .await;+ let source_id = source_id.expect("a dispatched send has a source id");+ assert_eq!(world.whatsapp.submission_count(&device, &source_id), 1);++ second.drain(TIMEOUT).await.unwrap();+}++#[sqlx::test(migrator = "MACRO_DB_MIGRATIONS")]+async fn a_send_cut_off_after_submission_is_unknown_and_never_resent(pool: PgPool) {+ let world = World::new(pool).await;+ let first = world.replica().await;+ let (device, binding) = world.alice_adopted().await;++ // WhatsApp gets the message, but the replica dies before the ack.+ world.whatsapp.withhold_acks(&device, true);+ let submitted_before = world.whatsapp.submissions(&device).len();+ let send = world.send(ALICE, &binding, "did this go out?").await;+ assert!(+ world+ .wait_for_submissions(&device, submitted_before + 1, TIMEOUT)+ .await+ );+ first.kill();+ world.whatsapp.withhold_acks(&device, false);++ let second = world.replica().await;+ let (state, source_id) = world.wait_send(send, &["unknown"]).await;+ assert_eq!(state, "unknown");+ let source_id = source_id.expect("the dispatch fence recorded the source id");++ // A later send goes out through the new replica; the unknown one doesn't.+ let later = world.send(ALICE, &binding, "a later message").await;+ world+ .wait_send(later, &["accepted", "sent", "delivered", "read"])+ .await;+ assert_eq!(world.whatsapp.submission_count(&device, &source_id), 1);+ assert_eq!(world.send_state(send).await.0, "unknown");+ let copies = world+ .customer+ .received()+ .into_iter()+ .filter(|message| message.text() == Some("did this go out?"))+ .count();+ assert_eq!(copies, 1, "the customer got the cut-off message once");++ second.drain(TIMEOUT).await.unwrap();+}++#[sqlx::test(migrator = "MACRO_DB_MIGRATIONS")]+async fn a_send_without_an_ack_times_out_as_unknown(pool: PgPool) {+ let world = World::new(pool).await;+ let replica = world.replica().await;+ let (device, binding) = world.alice_adopted().await;++ world.whatsapp.withhold_acks(&device, true);+ let send = world.send(ALICE, &binding, "no ack for this").await;+ let (_, source_id) = world.wait_send(send, &["unknown"]).await;+ let source_id = source_id.expect("the dispatch fence recorded the source id");+ world.whatsapp.withhold_acks(&device, false);++ let later = world.send(ALICE, &binding, "acked").await;+ world+ .wait_send(later, &["accepted", "sent", "delivered", "read"])+ .await;+ assert_eq!(world.whatsapp.submission_count(&device, &source_id), 1);++ replica.drain(TIMEOUT).await.unwrap();+}++#[tokio::test]+async fn a_phone_group_message_maps_to_one_inbound_text() {+ use whatsapp_linked::domain::events::{InboundKind, MessageKind};+ use whatsapp_session_service::outbound::whatsapp::events::{Mapped, map_event};++ let whatsapp = FakeWhatsApp::start().await.unwrap();+ let alice = whatsapp.add_user("15550000001", "Alice").await;+ let bob = whatsapp.add_user("15550000002", "Bob").await;+ let store = Arc::new(whatsmeow::store::MemoryStore::default());+ let device = store.new_device(&whatsmeow_signal::LibsignalCurveSignatures::new());+ let client = whatsmeow::Client::new(+ device,+ whatsapp_fake_server::test_signal(),+ whatsmeow::ClientConfig {+ dialer: whatsapp.dialer(),+ synchronous_ack: true,+ ..Default::default()+ },+ );+ let mut events = client.subscribe();+ let mut qr = client+ .get_qr_channel(tokio_util::sync::CancellationToken::new())+ .await+ .unwrap();+ client.connect().await.unwrap();+ let Some(whatsmeow::QrChannelItem::Code { code, .. }) = qr.recv().await else {+ panic!("expected a QR code");+ };+ let linked = alice.scan_qr(&code).await.unwrap();+ assert!(whatsapp.wait_until_keys_uploaded(&linked, TIMEOUT).await);+ let group = whatsapp.create_group("Escalations", &[&bob, &alice]);+ let id = bob.send_text(&group, "hello").await.unwrap();++ // A phone's first group message is two events with one ID: the sender+ // key alone, then the text. Only the text is group content.+ let mut mapped = Vec::new();+ while mapped.len() < 2 {+ let event = tokio::time::timeout(TIMEOUT, events.recv())+ .await+ .expect("the client emits the message events")+ .expect("the event channel is open");+ if let whatsmeow::Event::Message(message) = &event {+ assert_eq!(message.info.id, id);+ mapped.push(map_event(+ Uuid::now_v7(),+ &event,+ chrono::Utc::now(),+ &|_| None,+ ));+ }+ }+ assert!(mapped[0].is_none(), "the key-only part is not content");+ let Some(Mapped::Inbound(event)) = &mapped[1] else {+ panic!("the text maps to an inbound event");+ };+ assert_eq!(event.group_jid, group.to_string());+ let InboundKind::Message(message) = &event.kind else {+ panic!("a message event");+ };+ assert_eq!(message.kind, MessageKind::Text);+ assert_eq!(message.text, "hello");+ assert_eq!(message.source_id, id);+ assert_eq!(message.sender.jid, bob.lid().to_string());+ assert_eq!(message.sender.alternate_jid, Some(bob.jid().to_string()));+}++#[sqlx::test(migrator = "MACRO_DB_MIGRATIONS")]+async fn a_phone_image_is_downloaded_for_import(pool: PgPool) {+ let world = World::new(pool).await;+ let replica = world.replica().await;+ let (_, binding) = world.alice_adopted().await;++ let photo = b"jpeg bytes from the customer's camera".repeat(40);+ let source = world+ .customer+ .send_image(&world.group, &photo, "the damaged box")+ .await+ .unwrap();+ let copies = world.timeline(&binding, &source).await;+ assert_eq!(copies.len(), 1);+ assert!(copies[0].contains("the damaged box"), "{copies:?}");+ let downloaded = wait_for_rows(&world.pool, TIMEOUT, || async {+ sqlx::query_scalar!(+ "SELECT b.bytes FROM whatsapp_group_attachment a+ JOIN whatsapp_media_blob b ON b.document_id = a.document_id+ WHERE a.binding_id = $1 AND a.source_id = $2 AND a.state = 'downloaded'",+ binding.id,+ &source+ )+ .fetch_optional(&world.pool)+ .await+ .ok()+ .flatten()+ })+ .await;+ let downloaded = world+ .expect(downloaded, "the replica downloads the image for import")+ .await;+ assert_eq!(downloaded, photo);++ replica.drain(TIMEOUT).await.unwrap();+}++#[sqlx::test(migrator = "MACRO_DB_MIGRATIONS")]+async fn a_macro_attachment_reaches_the_group_intact(pool: PgPool) {+ let world = World::new(pool).await;+ let replica = world.replica().await;+ let (_, binding) = world.alice_adopted().await;++ let photo = b"a photo attached in Macro".repeat(40);+ let uploaded = world+ .linked+ .upload_attachment(+ &receipt::<ViewAccessLevel>(ALICE, binding.id),+ "label.jpg",+ "image/jpeg",+ &photo,+ SendAuthority::Human,+ )+ .await+ .unwrap();+ let send = world+ .linked+ .send_reply(+ &receipt::<ViewAccessLevel>(ALICE, binding.id),+ &SendReply {+ idempotency_key: Uuid::now_v7(),+ message: ReplyContent::Media {+ upload_id: uploaded.upload_id,+ caption: Some("return label".to_owned()),+ },+ reply_to: None,+ },+ SendAuthority::Human,+ )+ .await+ .unwrap()+ .id;+ let (_, source_id) = world+ .wait_send(send, &["accepted", "sent", "delivered", "read"])+ .await;+ let source_id = source_id.expect("a dispatched send has a source id");+ let received = world+ .customer+ .wait_for_message(TIMEOUT, |message| message.id == source_id)+ .await+ .expect("the customer's phone receives the image");+ let image = received+ .message+ .image_message+ .as_deref()+ .expect("an image message");+ assert_eq!(image.caption.as_deref(), Some("return label"));+ assert_eq!(+ world.customer.download_image(&received).await.unwrap(),+ photo+ );++ replica.drain(TIMEOUT).await.unwrap();+}++#[sqlx::test(migrator = "MACRO_DB_MIGRATIONS")]+async fn a_message_queued_before_a_crash_is_ingested_once(pool: PgPool) {+ let world = World::new(pool).await;+ let first = world.replica().await;+ let (device, binding) = world.alice_adopted().await;++ // Hold ingestion back so the replica can only queue the event, then+ // crash it in that window.+ let mut blocker = world.pool.begin().await.unwrap();+ sqlx::raw_sql("LOCK TABLE whatsapp_group_message IN SHARE ROW EXCLUSIVE MODE")+ .execute(&mut *blocker)+ .await+ .unwrap();+ let source = world+ .customer+ .send_text(&world.group, "queued, then crashed")+ .await+ .unwrap();+ let pattern = format!("%{source}%");+ let queued = wait_for_rows(&world.pool, TIMEOUT, || async {+ let count = sqlx::query_scalar!(+ r#"SELECT count(*) AS "count!" FROM whatsapp_inbound_event WHERE payload::text LIKE $1"#,+ &pattern+ )+ .fetch_one(&world.pool)+ .await+ .ok()?;+ (count > 0).then_some(())+ })+ .await;+ world+ .expect(queued, "the replica queues the event before acking it")+ .await;+ assert!(+ world.whatsapp.wait_until_acked(&device, TIMEOUT).await,+ "the node is acked once queued"+ );+ first.kill();+ blocker.rollback().await.unwrap();++ let second = world.replica().await;+ let copies = world.timeline(&binding, &source).await;+ assert_eq!(copies.len(), 1);+ assert!(copies[0].contains("queued, then crashed"));+ assert_eq!(world.whatsapp.delivery_count(&device, &source), 1);++ second.drain(TIMEOUT).await.unwrap();+}++#[async_trait]+impl whatsapp_linked::domain::ports::SendRateLimiter for Allow {+ async fn check(&self, _: &str) -> LinkedResult<()> {+ Ok(())+ }+}diff --git a/crates/whatsapp_e2e/src/watch_tables.sql b/crates/whatsapp_e2e/src/watch_tables.sqlnew file mode 100644index 0000000000..70b76da275--- /dev/null+++ b/crates/whatsapp_e2e/src/watch_tables.sql@@ -0,0 +1,26 @@+-- Statement triggers that wake whatsapp_e2e::wait_for_rows. Test databases only.+CREATE OR REPLACE FUNCTION e2e_notify_changed() RETURNS trigger AS $$+BEGIN+ PERFORM pg_notify('e2e_changed', TG_TABLE_NAME);+ RETURN NULL;+END;+$$ LANGUAGE plpgsql;++CREATE TRIGGER e2e_changed AFTER INSERT OR UPDATE OR DELETE ON whatsapp_linked_account+ FOR EACH STATEMENT EXECUTE FUNCTION e2e_notify_changed();+CREATE TRIGGER e2e_changed AFTER INSERT OR UPDATE OR DELETE ON whatsapp_pairing+ FOR EACH STATEMENT EXECUTE FUNCTION e2e_notify_changed();+CREATE TRIGGER e2e_changed AFTER INSERT OR UPDATE OR DELETE ON whatsapp_group_roster+ FOR EACH STATEMENT EXECUTE FUNCTION e2e_notify_changed();+CREATE TRIGGER e2e_changed AFTER INSERT OR UPDATE OR DELETE ON whatsapp_group+ FOR EACH STATEMENT EXECUTE FUNCTION e2e_notify_changed();+CREATE TRIGGER e2e_changed AFTER INSERT OR UPDATE OR DELETE ON whatsapp_group_message+ FOR EACH STATEMENT EXECUTE FUNCTION e2e_notify_changed();+CREATE TRIGGER e2e_changed AFTER INSERT OR UPDATE OR DELETE ON whatsapp_message_observation+ FOR EACH STATEMENT EXECUTE FUNCTION e2e_notify_changed();+CREATE TRIGGER e2e_changed AFTER INSERT OR UPDATE OR DELETE ON whatsapp_inbound_event+ FOR EACH STATEMENT EXECUTE FUNCTION e2e_notify_changed();+CREATE TRIGGER e2e_changed AFTER INSERT OR UPDATE OR DELETE ON whatsapp_send+ FOR EACH STATEMENT EXECUTE FUNCTION e2e_notify_changed();+CREATE TRIGGER e2e_changed AFTER INSERT OR UPDATE OR DELETE ON whatsapp_group_attachment+ FOR EACH STATEMENT EXECUTE FUNCTION e2e_notify_changed();diff --git a/docs/README.md b/docs/README.mdindex f5037ef77b..0824da274f 100644--- a/docs/README.md+++ b/docs/README.md@@ -10,3 +10,6 @@| [`agent-guide/`](agent-guide/) | How browser agents operate the app, not how to develop it. |Docs that belong to one app, service, or crate live next to its code.++WhatsApp linked devices: [architecture](architecture/whatsapp-linked-devices.md)+and [External view walkthrough](agent-guide/external.md).diff --git a/docs/architecture/whatsapp-linked-devices.md b/docs/architecture/whatsapp-linked-devices.mdnew file mode 100644index 0000000000..8ab0fd25f8--- /dev/null+++ b/docs/architecture/whatsapp-linked-devices.md@@ -0,0 +1,162 @@+# WhatsApp linked devices++Personal WhatsApp accounts linked to Macro as devices. Each person connects+their own account (QR), adopts groups they already belong to, and each adopted+group becomes an independently shared `whatsapp_group` entity. Each person sends from their own+account. Ported from the domain logic of #8304; the Go session service and its+HTTP API were replaced.++## Processes++```+Web app ──REST──▶ DSS ──── MacroDB ◀──── whatsapp_session_service (N replicas)+ rows + NOTIFY one task + one client per leased account+```++- **DSS** serves `/meta/linked/*` (`crates/whatsapp_linked/src/inbound`),+ authorizes people, writes group entities and intents, and publishes generic notifications.+- **Session service** holds each account's advisory lock, runs the WhatsApp+ socket, and calls `whatsapp_linked::domain::session::SessionService`+ in-process. There is no service-to-service HTTP.++| Signal | Payload | Sent by → read by | Meaning |+| --- | --- | --- | --- |+| `whatsapp_pairing_requested` | pairing id | DSS → session | start, update or stop (cancelled) pairing |+| `whatsapp_send_requested` | account id | DSS → session | claim pending sends |+| `whatsapp_groups_requested` | account id | DSS → session | refresh the account's group metadata |+| `whatsapp_account_changed` | account id | DSS → session | consent or adopted groups changed; re-read |+| `whatsapp_account_released` | account id | session ↔ session | lease handed over |+| `whatsapp_group_effects` | message id | session → DSS | a group notification revision is ready |++## Ownership and fencing++`whatsapp_linked_account.lease_epoch` is bumped by `begin_lease` when a replica+takes an account. Every session-service write locks the account row at that+epoch; a stale epoch fails with `Fenced` and the replica must stop the account.+`generation` is separate: it counts explicit consent (pairing, disconnect,+logout) and fences memberships and send intents.++## Pairing++DSS `connect` creates or reuses the person's account (`pairing`, generation +1)+and a five-minute `whatsapp_pairing` row, then signals. The replica reports QR+or pairing codes with `update_pairing`; the browser polls `GET pairing/{id}`.+`Success` connects the account only if the identity matches a reconnect's+expected identity and belongs to no other person. Cancel and expiry end the+attempt and disconnect a pairing account. Codes are cleared when the+transaction ends.++## Groups and adoption++The replica keeps `whatsapp_group_roster` (the account's view of its groups)+fresh with `update_groups`. Adoption requires metadata observed within two+minutes; stale facts request a refresh and return `refreshing`. Adoption+creates `whatsapp_group`, registers it as `EntityType::WhatsAppGroup`, grants the+adopter Owner through `entity_access`, and records their remote membership facts.+The caller supplies a name and optionally their team; the server selects their+team when omitted. A team/group pair has one active adoption. Existing adoption+requires view access and never grants a new viewer access implicitly.++Viewing uses ordinary entity sharing: direct people grants and access requests+at `/entities/whatsapp_group/{id}/...`, canonical explicit team grants, and link+sharing. It requires no connected WhatsApp account or remote group membership.+The group detail includes `share_permission` and `access_level`; its `/permissions`+GET/PATCH pair drives the standard Share dialog. Only the actual owner changes+team/link policy or detaches the group. Leaving WhatsApp or disconnecting keeps+established access and history. Roster updates never create sharing grants.++## Inbound, timeline and notifications++The replica keeps only events for adopted groups, persists them before ACK and+calls `ingest_event`. Unadopted events never enter storage. Every observer converges+on `(group, verified person, source id)` in `whatsapp_group_message`; messages do+not reference native comms storage. Sender associations use verified phone/LID+aliases; ambiguous identities are quarantined. Missing reaction/revision targets+are deferred. Edits follow source revision order; deletes are permanent tombstones.+Reactions and media live in `whatsapp_group_reaction` and `whatsapp_group_attachment`.++`GET /meta/linked/whatsapp-groups` lists shared groups with viewer-specific unread+counts and sending eligibility. Detail and messages live under+`/meta/linked/whatsapp-groups/{group_id}`. Messages page newest-first using stable+`(sent_at,id)` keysets with an opaque message-id `before` cursor. The `/read`+endpoint advances `whatsapp_group_read_state` monotonically; history and the+viewer's own messages do not increase unread counts.++DSS drains a durable notification outbox through the generic notification ingress.+`whatsapp_group_message` notifications target directly shared users, team members,+and known link viewers who have read the group, except the verified sender. Link+viewers must still have access when recipients are resolved; unknown link visitors cannot be+enumerated. Notifications use the message id for idempotency and are suppressed+for history, replays, edits and reactions. They link to the standalone group entity.+GraphQL notification targets use the canonical entity enum while groups remain outside Soup.++## Sending++DSS records a `whatsapp_send` intent (idempotency key, request digest, group+sequence) only while the sender's account is connected, observing and allowed+to post. The replica claims pending sends in group order, calls+`begin_dispatch` with the WhatsApp message id it will use (all facts are+re-checked; this is the point of no return), submits, then calls+`record_send_outcome`. An acceptance writes the group message with the+sender's identity; the later echo converges on it. A replica that dies after+`begin_dispatch` leaves the send `unknown`; it is never sent again, and a later+echo settles it. Unattempted intents expire after ten minutes. Edits and+deletes are limited to the sender's own messages for ten minutes.++## Rollout switches++DSS config `WHATSAPP_LINKED_PAIRING_ENABLED`, `WHATSAPP_LINKED_ADOPTION_ENABLED`+and `WHATSAPP_LINKED_SENDING_ENABLED` default to false. Disable sending first+during an incident. Turning switches off keeps history and access.++## Media++Inbound attachments are reserved per ordinal at ingest, with the download+locator sealed by the session service under the observing account's key. That+account's replica claims the download (`claim_media_imports`), checks the+SHA-256 and hands over the bytes (`complete_media_import`), which wait in+`whatsapp_media_blob`. DSS (signal `whatsapp_media_ready`) creates a private+document owned by the group owner. Attachment documents inherit current group+view access; sharing and revocation apply to previously imported files too. Ready+bytes are never rewritten. Failures retry with backoff and then expose an+`unavailable` attachment state; message bodies remain plain WhatsApp text.++Outbound: the attachments route stages one file per send in the blob table;+`ReplyContent::Media` names it, the replica fetches it with `outbound_media`,+and on acceptance it becomes the message's document, owned by the sender.++## Passkey pairing++When WhatsApp asks for a passkey, the replica reports `PairingUpdate::Passkey`+with the WebAuthn request options, shown as `PairState.challenge`. The owner's+native authenticator posts the assertion to `pairing/{id}/passkey`; DSS stores+it on the pairing row and signals; the replica takes it once+(`take_passkey_response`) and forwards it.++## Durable inbound++The replica persists each event with `enqueue_inbound` before acknowledging it+to WhatsApp, then ingests asynchronously with `replay_deferred`.++## Send request budget++DSS accepts at most one deliberate WhatsApp send request per second per+user, shared across groups and DSS replicas through its existing Redis rate+limiter. The domain consumes the budget before evaluating a send; requests that+later fail still count. Exhaustion returns `429` with code `rate_limited` and+`Retry-After: 1`; an unavailable counter fails closed with `503`. This does not+retry a send or change its idempotency key or unknown-outcome reconciliation.++Only the person who owns a connection sends from it, using their session or their+own API credential, including a verified Macro API JWT with signed agent attribution.+The domain resolves the sending account from the authenticated person, never from+the group owner or request input. Media uploads follow the same owner policy.+Bot, harness and internal principals never send, including bots acting for a user.+Pairing reads (including QR codes and passkey challenges) and account/access+controls require the person's bearer session; API credentials cannot perform them.++## Follow-ups++- Soup/search registration is deferred; External, shared links and notifications+ are the group entry points.+- Participant receipts are stored but not shown.