Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.
sync-service
What it is
Powers real-time collaboration on Documents and other collab surfaces (spreadsheets, forms): a Cloudflare Worker (Rust compiled to wasm) hosting one Durable Object per collaborative document ("room"), syncing Loro CRDT updates over WebSocket.
Start here
README.md: project structure, Durable Object lifecycle (in-memory vs persistent state, the keep-alive alarm), JWT auth, and Bebop codegen. Read it before changing this service.src/inbound/durable_object.rs+src/inbound/durable_object/document_api.rs: the Durable Object — WebSocket handling, loro document state, alarm-driven snapshotting.src/inbound/durable_object/surface_api.rs,surface_migration.rs: non-document surfaces (spreadsheets, forms, etc.) hosted the same way.src/outbound/storage/+src/outbound/storage/backends/: snapshot/op-log persistence, pluggable across D1/KV/R2/combined backends (see the Cargo feature flags).src/inbound/auth.rs+src/inbound/credentials.rs: route-specific access extractors and JWT verification against tokens minted bydocument_storage_service.build.rs+bebop-schema/,bebop/: Bebop wire schema and codegen for Rust and TypeScript bindings.src/inbound/router.rs+src/inbound/worker.rs: Axum document/surface routing and worker proxy/copy orchestration.src/domain/permissions.rs+src/domain/document.rs: session permission policy and content-agnostic atomic update use cases.
Scope
- Owns: real-time sync of Loro documents over WebSocket, snapshot persistence, and the sync-service's own auth/permission checks on established connections.
- Not here: minting the document-access JWT →
document_storage_service; document domain logic → the owning domain crate (e.g.collab_surface).
Recipes
- Change the wire protocol with the repository-pinned Bebop generator; update
both service and collaboration package bindings (see README), then
worker-buildregenerates Rust bindings viabuild.rs. - Add a storage backend:
src/outbound/storage/backends/(new file +mod.rs) → gate it behind a new feature inCargo.toml→ wire intosrc/outbound/storage.rs. - Add a non-document surface:
src/inbound/durable_object/surface_api.rs→src/domain/(surface-specific logic) → migration insrc/inbound/durable_object/surface_migration.rsif it changes stored shape.
Active-peer queries with include_ai=false exclude signed agent actors and the
reserved AI peer blocks (ai_peer.rs); Canvas workers use random full-width
peer IDs. peer_user_map records each peer's user and agent actor; history and
blame attribute edits from it, so never reuse an AI peer id.
Document read/write/debug handlers declare their minimum session grant through Axum extractors; atomic reads/updates require a signed document grant even when an internal key is present. Surface grants remain kind-discriminated and exact to the requested surface. These are scoped sync JWTs, not app-session tokens.
Checks
just test(runsworker-build, thennpx vitest runandcargo test) from this directory.just test-no-alarmto verify the legacy feature selection; alarms now run regardless of the retainedalarm-keep-alivecompatibility flag.- Use Node.js 22.12+/24.x/26+ for Vitest 5; install JS deps with
npm ci(package-lock.jsonis authoritative, notbun.lock).
Gotchas
- Bumping
loro-crdttouches three lockfiles:bun.lock, the workspaceCargo.lock, and this directory'spackage-lock.json— update all three. - Durable Object in-memory state (including the loro document) evicts after
10s of inactivity even with open connections; the alarm-based heartbeat in
src/keepalive.rsis what keeps it warm, not WebSocket traffic. - The Durable Object handles one request or WebSocket message at a time
(
mutation_barrierinsrc/inbound/durable_object.rs). Keep per-update work proportional to the update;LoroDoc::diff,fork, and checkouts cost time that grows with document history. A backlog pastDEFAULT_TIMEOUT_MS(4.5 s) makes the front worker fail connects, which clients see as close 1006. - Document history names each Loro peer from D1
peer_user_map, so every path that writes operations maps the peers it writes with: WebSocket and HTTP/updatewrites to their user and signed agent actor,/initializesnapshots to the system author (SYSTEM_AUTHOR_ID), and/copycopies the source's mappings first. A peer's first owner keeps it; registration only claims peers the system author holds. A new write path does the same (tests/peer_authors.test.ts). migration-test-hooksis test-only fault injection — never enable it in a deployed worker (see the feature's doc comment inCargo.toml).- Provision
LEXICAL_SERVICE_AUTH_KEYto match lexical-service before deploying;just check-secrets <environment>checks binding presence without reading values. - Deploy
PeerCommentUpdatesupport before clients send it; validate these frames as comment-only for editors as well as commenters, and reject viewers. - Never run
just deploy-dev/just deploy-prodwithout explicit approval. - Wait at least five minutes after a manual worker deployment before publishing clients that require its new protocol; the CI worker job waits out Durable Objects' default 300-second code-update deferral.
- A new input to the worker's bundle that lives outside its workspace closure must be added to
.github/scripts/sync-worker-key.sh, or CI can skip deploying a changed worker.
Related
Docs: this directory's
README.mdRoom
/existschecks the durable snapshot, not the document ID set by a connection. To regress connect-before-initialize, build withoutcreate-default-stateand runSYNC_TEST_UNINITIALIZED_ROOMS=true npx vitest run tests/canvas_initialization.test.ts; the usual test build seeds automatically.Preserve signed document-grant agent correlation in editor metadata and pending batches; different invocations remain distinct editors.