Skip to content

armgit / macro

Public Imported from GitHub
0

Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.

27.7 KB
Raw Blame

Running locally

This guide covers two ways to run Macro on your machine.

If you only change the frontend, run the frontend against hosted services. You do not need Docker or the local stack.

If you change a backend service, the database, or behavior that must stay on your machine, run the local stack.

Choose a path

Shared prerequisites

Install Nix before you start:

  1. Nix package manager

Clone the repository:

git clone https://github.com/macro-inc/macro.git
cd macro

The Nix shell provides just, Cargo, the Rust toolchain, Bun, wasm-pack, sqlx, zig, cargo-zigbuild, prek, and gitleaks. You do not need to install these tools separately.

nix develop

Entering the shell installs git hooks (prek install). A pre-commit hook scans staged added lines for customer emails, phones, addresses, and secrets. See tooling/pii/exceptions and the git skill.

If nix develop fails, enable the experimental features:

nix develop --extra-experimental-features nix-command --extra-experimental-features flakes

Nix requires these experimental features to work. The command above enables them for one run. To enable them permanently, set this in ~/.config/nix/nix.conf:

experimental-features = nix-command flakes

The default shell does not include the Tauri platform dependencies. They are large, so they live in their own shells. For Linux desktop development, use nix develop .#tauri-linux. For Android development on x86_64 Linux, use nix develop .#tauri-android.

For automated Linux desktop offline tests, use nix develop .#tauri-e2e and the native E2E guide. It runs the real Tauri webview/native cache with deterministic API fixtures, without the local stack.

Run native app probes in their own memory-capped user scope, including inspector and WebDriver reproductions. Renderer regressions can allocate enough memory to terminate unrelated agents before a stalled probe times out:

systemd-run --user --scope -p MemoryMax=6G -p MemorySwapMax=0 \
  nix develop .#tauri-e2e --command <native-probe-command>

Run the frontend against hosted services

The web app talks to hosted *-dev services when you run bun run dev from the web app.

Limits:

  • You still need Nix. The first bun run dev may compile wasm. Later runs skip that compile when versions match.
  • The UI calls hosted *-dev services and shared data.
  • Sign-in is not the local Mailpit flow. If you need a private database or to change a backend service, use the local stack.

From the repository root, inside the Nix shell:

bun install
cd apps/web
bun run dev

The first run, or a run after a wasm version change, may build wasm packages. Vite detects your hostname and prints https://<hostname>:3000/app/ when ready (PORT overrides the port). It generates a certificate with the same stable development CA as just local; trust infra/local/certs/ca.pem once on the machine running the browser. See the certificate README. The visiting machine must resolve the hostname and reach that port. No Docker proxy or Tailscale is needed. Vite forwards hosted development API and WebSocket requests through /__macro_dev/ on this same origin, so arbitrary machine hostnames do not need to be added to the deployed CORS allowlist. Authentication cookies are scoped to the visiting hostname; sign in with an email code on that hostname rather than relying on an existing .macro.com cookie. Google/SSO redirects still use the hosted authentication service’s redirect allowlist, which does not currently include arbitrary development hostnames.

Use MACRO_DEV_HTTPS=false bun run dev for an HTTP-only workflow. Tauri and the local stack's internal Vite server keep using HTTP automatically; the stack provides HTTPS through its proxy. Builds and preview servers are unchanged. MACRO_DEV_PROXY=false opts out of the hosted API proxy.

The dev server speaks HTTP/2, gzips module responses, and links each module's source map (/@macro-sourcemap/<hash>.map) instead of inlining it; browsers only fetch maps while DevTools is open. On a slow link this is the difference between ~100 MB and ~13 MB per load. Trust the CA rather than clicking through the certificate warning: Chrome caches nothing from a site with a certificate error. MACRO_DEV_COMPRESSION=false and MACRO_DEV_INLINE_SOURCEMAPS=true restore Vite's defaults.

Run the local stack

The local stack runs without Doppler. It runs Postgres, Redis, LocalStack, OpenSearch, Kafka, and FusionAuth in Docker, with dummy AWS credentials and fixed test secrets.

On Linux, the Nix dev shell supplies the Docker CLI, daemon, Compose, and fuse-overlayfs. Nix is the only host dependency.

On macOS, install a Docker runtime such as Docker Desktop, OrbStack, or Colima. The Nix dev shell supplies the Docker CLI, but macOS still needs the runtime to provide the daemon.

Install workspace dependencies from the repository root inside the Nix shell for each checkout or worktree before starting the stack:

bun install --frozen-lockfile

Run the preflight check before the first start:

just doctor-local

The check tests the Docker daemon, the toolchain, and the required ports. It reports any problem and suggests a fix. If a start fails, run the check again.

Run this command from the repository root if you do not have Doppler access:

just run_local --no-doppler

The local stack does not need Doppler. It uses the code-defined local configuration with dummy AWS credentials and fixed test secrets. Most contributors are not on the team, so this is the common path.

Local authentication includes a working signing key pair for Macro API tokens, which settings actions such as enabling CRM require. New teams start with CRM disabled; enable it in Settings > CRM before creating pipelines.

The stack supplies stubbed values for required configuration, including third-party integrations (Google, GitHub, Stripe, CloudFront). Those flows do not work against real services with the stubs. Local auth, documents, and search work independently. The agent harness currently also requires a configured Google Gmail identity provider for its public MCP routes; without real Google OAuth configuration, it exits during startup.

To use a real integration locally, supply its keys via --env-file — see Integration Secrets below.

Run this command if you have Doppler access. It pulls the lcl_personal config. Then it overlays the code-defined local defaults. Every integration value is real:

just run_local

If you prefer to test against real cloud infrastructure, you need Doppler for secrets management.

This command:

  • Builds the Rust backend services
  • Starts the local infrastructure (Postgres, Redis, LocalStack, OpenSearch, Kafka, FusionAuth)
  • Starts the backend services
  • Starts the local proxy and the frontend

When startup finishes, the command prints the frontend URL and the important service URLs.

The reverse proxy is HTTPS at https://localhost:8090 (or the instance's proxy port) using a generated machine certificate signed by the checked-in development CA. Trust infra/local/certs/ca.pem once so the browser accepts it; see that directory's README. Local Caddy also reflects any Origin (wildcard CORS) so https:// and *.localhost frontends can call the proxy. For document sync, local Caddy normalizes the upstream Origin to the worker's accepted localhost origin so HTTPS machine hostnames can open WebSockets. Sync's local Durable Object, KV, R2 and D1 state lives in the Compose sync_state volume so replacing its container preserves document content. When upgrading an existing stack without that mount, stop sync and copy /app/.wrangler/state into the volume before replacing the container. A restart alone does not rebuild its compiled worker; rebuild the sync image after backend protocol changes.

Open the frontend URL in your browser.

The local environment supplies both LOCAL_AWS_URL (the container endpoint) and LOCAL_AWS_PUBLIC_URL (this machine's hostname on the proxy port, with /local-storage, in both the hot and the static-frontend flavor). Document downloads and presigned uploads use this storage proxy, so remote browsers do not need a separate localhost storage connection. Container clients translate that public endpoint back to LocalStack internally. If a download or upload still attempts HTTP localhost, rebuild the services, reload the generated environment and proxy, and hard-refresh the browser to discard cached URLs.

If LocalStack loses its temporary resources after a restart, restore missing buckets, queues, tables, and keys without resetting Postgres or volumes: cargo x localstack-provision --instance <name>. Include --port-base if the instance uses an explicit port base. This restores resources, not lost objects.

The stack does not create accounts in advance. Passwordless login creates a user on demand. Register with any email address. FusionAuth sends you a one-time code by email. That email lands in Mailpit at http://localhost:8025, not in a real inbox.

Recover an existing instance without resetting data

run_local and stack up initialize clean state, including snapshot restoration. Do not use them to recover an instance whose application or auth data must survive. Keep its generated env/Compose files and existing named volumes; start stopped containers with docker start, or recreate only the affected services with the same Compose project/files and up -d --no-deps --no-build <service>. Never use down -v, reset_local, or destroy_local for this recovery.

If a local debug service aborts with tokio-rt-worker stack overflow, give only that service RUST_MIN_STACK=16777216 in an additional local Compose override, then recreate it with the same project, generated environment, and volumes. Keep this runtime override outside feature commits. Verify the request that triggered the crash, not just the health endpoint: document storage can start successfully and then exhaust the default worker stack on a Soup request.

On SELinux hosts, FusionAuth can become healthy while silently skipping an unreadable kickstart. The generated per-instance kickstart directory uses :ro,Z: read-only with a private container label. Regenerate the override with cargo x gen-compose --instance <name>, then recreate only FusionAuth against its existing volumes. Verify the kickstart is readable in the container and the configured application API returns 200 with the local API key; a healthy process alone does not prove kickstart ran. Do not disable SELinux, change host permissions, or delete the FusionAuth database. Diagnose other bind-mount denials separately; private labels must not be applied to directories shared by several containers.

For the synthetic Slack recovery harness and separate browser/native coverage, see the Slack import runbook.

Seeding sample data (recommended)

A bare stack has no content to click through. The seed CLI creates a realistic world: users, teams, channels, projects, documents, tasks, chats, calls, emails, and messages. The world uses realistic permissions.

From the repository root, after the stack is up:

just seed-scenario apply --file seed/scenarios/team-perms.json

apply creates a FusionAuth account for each persona. It prints a login link per persona, for example http://alice.localhost:3000/app/[email protected]. Open each link in a plain browser tab. Each persona hostname has its own cookie jar. You can drive several personas side by side against one stack.

Useful commands:

  • just seed-scenario status --file seed/scenarios/team-perms.json — show what is seeded and re-print the login links.
  • just seed-scenario reset --file seed/scenarios/team-perms.json — remove the scenario's rows and its user accounts by email.
  • just seed-scenario matrix --file seed/scenarios/team-perms.json — check the expected access level for every user and entity pair against the live database.

apply touches only rows that carry the scenario 5eed id marker, plus the persona accounts it created. It is safe to run against a stack that you tested in.

Integration Secrets

A --no-doppler stack supplies deterministic stubs for required configuration. External integrations still need real values, and the agent harness additionally needs the Google Gmail identity provider created from real Google OAuth configuration:

Integration Keys Stub behavior
Google login / Gmail GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET_KEY Google SSO and Gmail inbox linking are unavailable. Local signup still works. The email service skips inbox syncing without a Gmail grant. The agent harness cannot start its public MCP routes without the Google Gmail identity provider.
Pipedream PIPEDREAM_CLIENT_ID, PIPEDREAM_CLIENT_SECRET, PIPEDREAM_PROJECT_ID Placeholders satisfy the harness configuration loader; Pipedream integrations require real credentials.
GitHub login GITHUB_CLIENT_ID, GITHUB_CLIENT_SECRET, GITHUB_IDP_ID Login with GitHub is unavailable
Stripe billing STRIPE_SECRET_KEY, STRIPE_PRICE_ID Checkout and subscription endpoints fail. Signup skips Stripe and leaves profile billing unset, so free-team creation works. Only the required legacy account field stores a local placeholder.
CloudFront signed URLs DOCUMENT_STORAGE_SERVICE_CLOUDFRONT_DISTRIBUTION_URL, DOCUMENT_STORAGE_SERVICE_CLOUDFRONT_SIGNER_PUBLIC_KEY_ID, DOCUMENT_STORAGE_SERVICE_CLOUDFRONT_SIGNER_PRIVATE_KEY Document download URLs are unsigned (fine against local S3)

The other stubbed keys (REDIS_HOST, MACRO_DB_URL, INTERNAL_API_KEY, AUTHENTICATION_SERVICE_SECRET_KEY, ACCOUNT_LINK_STATE_SECRET, OPENSEARCH_USERNAME, OPENSEARCH_PASSWORD) are internal plumbing with correct local values — you never need to override them.

To turn on an integration, create a local.env with the real values. Then pass it to run_local:

just run_local --no-doppler --env-file ./local.env

Keys in the file override the code-defined defaults, so you only need to list the integrations you care about. With Doppler access, just run_local (without --no-doppler) supplies everything automatically.

Tracing, Logs, and the Debug Browser

Local service console logs rotate at 20 MB per file and retain three files per container. The limits apply when a container is created; recreate older containers with the existing Compose project/files to adopt them. Repeated worker errors still need their underlying dependency or configuration fixed.

just run_local and just stack up support two global (per-machine, shared across instances) debugging containers:

  • LGTM collector (--traces lgtm, the default): Grafana at http://localhost:3001 with Tempo (traces), Loki (service logs), and Prometheus behind it. Rust services export spans and tracing events over OTLP; the frontend exports browser spans through the proxy and propagates traceparent, so one trace covers browser → proxy → services. Swap with --traces jaeger|datadog, or disable with --traces off.
  • Agent browser (opt-in via --with-chrome): Chromium with the DevTools protocol on http://localhost:9222, for agents driving the app (the chrome-devtools MCP server in .mcp.json / opencode.json / .cursor/mcp.json points at it). Watch what an agent is doing live at http://localhost:6080/vnc.html.

See .agents/skills/live-debug/SKILL.md for query recipes (Tempo/Loki HTTP APIs) and the browser-debugging workflow.

For agent turns, use Tempo's TraceQL query {span.gen_ai.operation.name="invoke_agent"}. The session actor records the ACP prompt, output and tool activity on that trace and sends its context in params._meta["macro.dev/trace-context"]. Macro's in-process runtime restores that context so its model calls and backend work appear in the same trace. Other runtimes must explicitly consume this metadata to correlate their internal spans; their ACP activity is still traced by the session actor.

GenAI content is bounded by genai_telemetry; check macro.genai.content_truncated before using a span for evaluations.

Control the Running Stack

While run_local is attached:

  • Press r to rebuild the changed Rust services and reload them.
  • Press f to restart Vite with the same frontend port and configuration. This also recovers a stuck frontend reload and leaves backend services and data intact.
  • Press q to stop the stack and exit.

Use q, not the terminal close button. q stops and removes the containers at once. The next start does not have to clean up a stale stack.

Run More than One Stack

Use named instances for several local stacks at once. This helps across worktrees:

just run_local --instance agent-a
just run_local --instance agent-b

Each instance has its own resources:

  • a Compose project
  • volumes and networks
  • env files
  • a proxy port, a frontend port, and backend ports

The ports are deterministic for the instance name. The same name gets the same port window on every run.

If the port window conflicts with another program, change the base port:

just run_local --instance agent-a --port-base 23000

The generated files for an instance live here:

infra/local/generated/<instance>

Access a remote dev server through one URL

just local (the run_local alias) and run_dev call hostname, add that name to Vite's allowed hosts, and issue a certificate for it using the stable CA in infra/local/certs. Startup prints https://<hostname>:<proxy-port>/app/. Caddy serves backend routes directly and forwards frontend assets and HMR to Vite, so the browser only needs the HTTPS proxy port. Login fallback redirects, Pipedream origins, MCP callbacks, and file permalinks use that same HTTPS origin for attached local stacks. If your host firewall blocks Docker-to-host traffic, allow the instance's Docker network to reach the Vite port through host.docker.internal. Startup verifies /app/ through the HTTPS proxy before printing “ready”; a listening Vite port alone is insufficient. Persistent 502s with proxy logs such as dial tcp <host-gateway>:<vite-port>: i/o timeout can indicate this firewall path is blocked. Firewall rules must cover the current instance's Docker network and frontend port, which can differ between instances.

If /app/ succeeds but the page stays blank, inspect script requests as well as API requests. Intermittent 502s for Vite modules can come from the burst of Docker-to-host connections during startup. The generated proxy limits Vite to 16 upstream connections so module requests queue instead of exhausting the host listener. After applying an updated proxy configuration, restart only the instance's proxy and reload the page; existing databases and volumes can stay.

Trust infra/local/certs/ca.pem in the visiting browser once (see the certificate README), then open that URL. The visiting machine must resolve the hostname and reach the proxy port over your network. Tailscale is not required. The generated certificate also covers localhost, *.localhost, 127.0.0.1, and ::1. Other aliases require a matching certificate and a Vite allowlist entry; arbitrary names are not accepted automatically.

An SSH forward to the HTTP Vite port still works for localhost-only access: ssh -N -L 3000:127.0.0.1:20110 your-dev-host, then open http://localhost:3000/app/. HMR follows the page's host, port, and protocol.

The launcher sets VITE_LOCAL_BACKEND_ORIGIN=same-origin and supplies Vite's server-only MACRO_LOCAL_BACKEND_PROXY and MACRO_LOCAL_BACKEND_ROUTES from the selected instance and backend inventory. The proxy preserves paths, query strings, cookies, streaming responses and WebSocket upgrades. AI-editing and enabled browser telemetry also use same-origin paths. Bare bun run dev without these variables still uses hosted services; TAURI_DEV_HOST remains an explicit native HMR override. Keep local dev stacks private: same-origin routing does not add authentication or make passwordless local login safe to publish.

The launcher sets NODE_EXTRA_CA_CERTS for Vite so its HTTP and WebSocket forwarding verifies the backend certificate. Plain HTTP on a remote hostname cannot retain the app's secure login cookies; use the printed HTTPS URL.

The stack launches Vite directly with Node (available in the Nix shell), because Bun's Node HTTP compatibility currently hangs on Vite's proxied WebSocket upgrades. Bun is still used for dependency installation and builds.

Port Conflicts (macOS)

The default instance binds a fixed set of host ports. macOS reserves some of them for its own services. If the app loads but API calls return unexpected HTML, a port is probably hijacked by an unrelated process. The two most common conflicts on a fresh Mac:

  • Port 8080 — macOS WebDriver service (com.apple.WebDriver.HTTPService). It listens on this port when remote automation is on. The auth service cannot bind it.
  • Port 8090 — another project's dev server, for example an Expo server with --port 8090. The proxy cannot bind it.

The frontend loads, but login and API calls hit the other process. You see HTML or console errors instead of JSON. just doctor-local reports the busy ports before you start.

Run the stack on a port window that is free on your machine. You do not need to kill the other process:

just doctor-local                         # see which default ports are busy
just doctor-local --instance test --port-base 31000   # check the new window is free
just run_local --no-doppler --instance test --port-base 31000

A named instance binds every service at port-base + offset. A free base like 31000 moves the whole stack to one contiguous window. Use any base that is free on your machine. See just doctor-local for the busy ports. Keep the same --instance name and --port-base on later runs so the ports stay deterministic.

Use the same two flags for every command. Run the stack, seed it, and check it with the same --instance and --port-base values:

just run_local --no-doppler --instance test --port-base 31000
just seed-scenario --instance test --port-base 31000 apply --file seed/scenarios/team-perms.json
just seed-scenario --instance test --port-base 31000 status --file seed/scenarios/team-perms.json
just status_local --instance test --port-base 31000

If you omit --port-base, a named instance gets a deterministic port window derived from its name. That window is different from the one you chose. A stack started with an explicit --port-base must be seeded with the same explicit --port-base, or the seed CLI looks at the wrong database. The default instance (no --instance) always uses the fixed ports and needs no extra flags.

The seeded persona login links embed the frontend port. If you switch ports, run just seed-scenario apply again to get links that match the new window. just status_local, with the same two flags, prints the live endpoints.

What the Stack Rebuilds

The Rust services are built on the host with cargo zigbuild. The binaries are mounted into a shared runtime image. Docker does not compile these services during a normal run_local.

Press r to rebuild the binaries. Only the services whose binaries changed restart.

Three services have Docker-built images. They are not rebuilt by default:

  • sync_service
  • lexical_service
  • websocket_service

If you change these services, the running stack can use a stale image. Force a rebuild with this flag:

just run_local --build-aux-services

When you start the stack with --build-aux-services, press r to rebuild those images and recreate their containers. This is slower, so leave the flag off unless you work on those services.

If you started without the flag and suspect a stale image, press q. Then start again with the flag.

Headless Mode

just stack runs the same stack without an attached terminal. There is no hotkey loop and no dev server. The frontend is built once and served statically by the proxy. The bundle counts as local, as run_local's dev server does: flags, routes and tools gated on LOCAL_ONLY are on. The whole product lives behind one origin. A finished up leaves only Docker containers running.

Both headless and attached startup prepare the browser WebAssembly packages through bun run dev:prepare. A fresh worktree builds them before serving the app; later starts reuse the generated packages when they are current. See Integration Secrets for credential-free startup limitations.

just stack up                  # bring everything up, print URLs, return
just stack status --json      # machine-readable state (containers, health, URLs)
just stack update             # rebuild and reload only the changed services (the `r` hotkey)
just stack update --frontend  # also rebuild the frontend bundle
just stack update --binaries-dir <dir>  # remount a prebuilt set; volumes stay
just stack down               # remove containers, volumes, and state

All the run_local flags apply to stack too. This includes --instance, --no-doppler, --no-build, and --binaries-dir.

The app is served at <proxy>/app/. The bundle resolves its backend from the origin it is served on. The same stack works on localhost or behind any hostname without a rebuild.

For Pipedream Connect on a machine hostname, include that HTTPS origin in PIPEDREAM_ALLOWED_ORIGINS through --env-file; the headless default names only localhost. Reuse that override file on subsequent stack invocations.

Init Snapshots

just run_local and stack up both cache the expensive infrastructure initialization. The first cold run:

  • Migrates the database
  • Creates the Kafka topics
  • Waits for the FusionAuth kickstart
  • Creates the search indices

It saves these volumes as an init snapshot. The snapshot is content-addressed and stored under infra/local/generated/.snapshots. Later runs restore the snapshot and skip the initialization. An input change causes a cache miss and a normal full init — the key is the definition of clean state, so the full-delete/full-create guarantee is unchanged.

Useful commands:

just run_local --no-snapshot  # skip the snapshot cache
just stack up --no-snapshot   # same flag, headless

Cursor Cloud bakes the snapshot during environment install. Later stack up restores it.

Common Commands

Run local binaries against shared dev resources instead of a full local stack:

just run_dev

run_dev uses shared dev resources. It needs Doppler and real cloud access. It is for contributors with team access.

See what a running or stopped instance looks like. The output shows endpoints with live reachability probes, plus the state and host ports of every container. It does not start or rebuild anything:

just status_local

Stop an instance but keep its volumes:

just stop_local --instance agent-a

Remove the containers, volumes, and named-instance networks of an instance:

just destroy_local --instance agent-a

Drop, recreate, and migrate an instance database:

just reset_local --instance agent-a

Finding out where a bring-up spent its time

Every run prints its slowest stages before the summary. To compare runs, point MACRO_LOCAL_TIMINGS at a file — each run appends one JSON line of every stage and its duration:

MACRO_LOCAL_TIMINGS=/tmp/run-local-timings.jsonl just run_local

For the default instance, omit --instance.