Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.
document_storage_service
What it is
Documents feature: deployable service (DSS) hosting document/item CRUD, the "Soup" GraphQL API, and most cross-entity concerns — history, annotations, threads, pins, recents, notifications, folders, activity.
Start here
src/api/sync.rs: dedicated authenticated/sync/wsendpoint;crates/sync_routerowns multiplex routing and cancellation. Sync-service still owns document state and validates scoped tokens. DSS shutdown drains these upgraded sockets. This binary WebSocket protocol is intentionally outside the REST OpenAPI/SDK surface.src/api.rs: route module list —documents,items,entity,folders_query,graphql_soup,threads,annotations,history,pins,recents,notification,instructions,internal./databases/querymounts thedatabases_sqlHTTP adapter, sharing the database/access services and the primary Soup reader; SQL authorization stays in its domain service.src/service/saved_views.rs: composes saved-view storage with the primary-backed Teams repository, reusingget_user_teamsandget_all_team_members. REST and GraphQL share its owner-only domain policy.src/api/graphql_soup.rs: the/soup/graphql(+/soup/graphql/wssubscriptions) GraphQL API — the frontend's main read path for Home/Recent/ search ("Soup" items); seecrates/graphql_soupfor resolvers/loaders.src/server.rs: composition root, including access-request routes and the tracked delivery dispatcher;src/outbound/access_requests.rscomposes the notification/withdrawal/realtime ports. Policy lives incrates/access_requests. It also builds the attention feeds (Home's work feed and the notifications inbox) from one set ofattention_feedadapters, triggers and viewer lookup.src/external_chat.rs: composesExternalChatService<WhatsAppProvider>, mounts/external-chats(crates/external_chat) and/whatsapp(crates/whatsapp_linked) and runs the notification/import worker (LISTENexternal_chat_effects,whatsapp_media_ready; attachments become static files through the static file service, which also deletes released ones) and theexternal_chat_changedpusher (bare connection-gateway events to the account owner or a chat's viewers); enforces a Redis-shared one-send-request-per-second budget per user. No backend switch: the web flagenable-whatsapp-linkedgates rollout.- Guest call chat is mounted outside account auth at
/call/guest-chat/{call_id};callverifies the RTC session andmessagesowns the shared discussion. - Authenticated call routes are composed from
crates/callunder/call;src/api/swagger.rslists their OpenAPI paths and schemas, whilesrc/server.rssupplies the per-caller start limiter. src/service/entity_mutation.rs: generic entity-mutation plumbing used across item types. Folder duplication runs on its own task and copies documents and chats throughentity_mutation/project_contents.rs, the projects domain'sProjectContentCopierport.src/outbound/docx_export.rs: primary-database parts and immutable DOCX export objects keyed by part paths/hashes. The export route maps the documents domain's pending result to 409 withRetry-After; never reuse document-id-only archives. Location and export URLs share signing helpers insrc/service/cloudfront.rs.src/outbound/calendar_refresh.rs: therefresh_calendarnudge for a new request to join a calendar event, which this service hosts while answers belong tocalendar_service. It finds delegates bylink_idalone, as the request list admits them;get_primaries_for_linkalso filters on the child, which a promoted shared mailbox's older events do not carry as their owner. Nudges follow a committed write, so they shareevent_broker_tracker's bounded shutdown drain with event broker publishes.src/service/delete_document_worker.rs: background worker handling document deletion side effects (feature-gated:delete_document_worker).src/service/conn_gateway.rs: pushes realtime updates throughconnection_gateway(feature-gated:connection_gateway).README.md: local setup (just run_dbs,.envfrom.env.sample).
Scope
- Owns: document/item HTTP CRUD, the Soup GraphQL schema, and cross-entity surfaces (history, annotations, threads, pins, recents, notifications, folders, saved views, activity).
- Not here: document content/collaboration itself →
crates/collab_surfaceand the sync service; AI chat →document_cognition_service.
Recipes
- Add a Soup GraphQL field: resolver/loader in
crates/graphql_soup→src/api/graphql_soup.rsif wiring changes. - Add a REST route:
src/api/<area>.rs→src/api.rs(router) →src/api/swagger.rs; register query-only schemas explicitly in its components list. Thenbun run gen-api cloud-storageinapps/webregeneratesservice-storage/openapi.jsonand its clients. - Add a config value or feature-gated behavior:
src/config.rs→ the[features]table inCargo.tomlif it should be independently toggleable.
Checks
cargo test -p document_storage_service(SQLX_OFFLINEunset; needs MacroDB + Redis —just run_dbsfrom repo root; DynamoDB local viajust ensure::ensure_dynamodbfromservices/document_storage_service, whosejustfiledeclares theensuremodule).- After query changes:
nix develop --command just prepare_dbfrom the repo root.
Gotchas
Provision required
POSTHOG_API_KEYbefore startup;enable-call-channel-threadevaluation has a 300 ms timeout and keeps the call's own chat on failure.The same flag client evaluates
enable-programmatic-callsper caller for the explicit call start, join, and leave routes; failures disable those routes.Compose Soup without a separate property-definition reader; REST and GraphQL validate DATE range inputs before calling the service.
Several behaviors are Cargo-feature-gated (
connection_gateway,delete_document_worker,location_check, all on bydefault) — a feature disabled locally can hide a bug that only shows in deployed builds.Soup GraphQL reads a replica with multi-second lag; a read immediately after a write through the same request needs the primary-reading loader, not the default one — see
crates/graphql_soup.Dev's readonly pool keeps 2–20 connections per task, independently of the primary pool's 25–100. Once all tasks use this cap, three dev tasks allow 60 sessions, or 120 while a rolling deployment overlaps all three. Old tasks retain their previous cap until they drain during the initial rollout. Production and local pools retain their existing primary/readonly limits.
The GraphQL calendar feed (occurrence ranges and
calendarChanges) reads the primary, like the REST calendar routes: arefresh_calendarpoke goes out as a write commits, so a replica-backed delta misses the change it announces.
Related
Rules:
.agents/rules/rust/db.md,.agents/rules/observability.mdNegotiate Mail cache profiles through
x-macro-mail-cache-profile(CORS-enabled HTTP) andmailCacheProfile(WebSocket connection init); preserve authenticated connection data and default unknown capabilities to v2.The HTTP boundary validates agent correlation; activity consumption restores Kafka envelope attribution before materialization.