Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.
soup
What it is
Backs the Home, Recent, Signal/work-feed, and GraphQL Soup query surfaces. Soup is the amalgamated query service: callers filter across many entity types (documents, channels, chats, calendar events, agent sessions, database rows, initiatives, ...) and get back a single unified, paginated result.
Start here
src/domain/service.rs: the main query service;src/domain/ports.rsfor its repository/collaborator traits.src/domain/models.rs(+grouping.rs,exclusions.rs,contact_listing.rs): the unifiedSoupItem/grouping/exclusion shapes.src/domain/models/sort.rs: property/timestamp sort descriptors and ordinary position cursors; timestamp encodings remain backward-compatible.src/domain/service/favorites.rs,agent_metadata.rs: entity-kind-specific query logic layered onto the core service.src/outbound/pg_soup_repo.rs: the Postgres repo, split by entity kind (database_row.rs,calendar_event.rs,agent_session.rs,initiative.rs,attention.rs,notified.rs,touched.rs) and by shape (expanded/hydrated,unexpanded/id-only).src/inbound/axum_router.rs+src/inbound/toolset/list_entities.rs: the HTTP route and the AI-tool surface over the same service.
Scope
- Owns: cross-entity filtering, sorting, grouping, exclusion, and pagination logic, and the Postgres queries that back it.
- Not here: how a specific entity type is created/edited (channels, documents,
initiatives, ...); per-entity-type cache projection →
soup_filter_projection; realtime fan-out of Soup patches →soup_realtime.
Boundaries
- Depends on:
entity_access,properties,crm,item_filters,filter_ast,favorites,frecency,channels,email,call,github_pull_requests,foreign_entity,activity,agent_changes. - Used by:
ai_tools,attention_feed,complete_graph,databases_sql,graphql_initiative,graphql_soup,graphql_work_feed,memory,skills,soup_filter_projection.
Recipes
- Add a new queryable entity kind:
src/domain/models.rs(variant) →src/outbound/pg_soup_repo/<kind>.rs(expanded + unexpanded if both apply) →src/domain/service.rswiring →src/domain/ports.rsif a new collaborator is needed. - Add a filter/sort option:
item_filters/filter_ast(the filter language) →src/domain/models.rs→ the matchingsrc/outbound/pg_soup_repo/*query. - Change which candidates attention feeds (Home's work feed, the
notifications inbox) page:
src/domain/service/tests/attention.rs→src/outbound/pg_soup_repo/attention.rs(attention/query.sql). Channel-thread keying applies only toAttentionKeying::ThreadScoped; keep it in step withattention_feed'skeying.rs.
Tool contracts
- The AI
ListEntitiesadapter includes agent sessions and excludes legacy chats; public feed contracts retain historical chat support.
Checks
cargo test -p soup --features outbound(SQLX_OFFLINEunset; needs local MacroDB). Without the feature the Postgres repo and its tests don't compile.- After schema/query changes:
nix develop --command just prepare_db(seejustfile). cargo clippy -p soup --no-default-features, and again with--features ports(both run injust clippyand CI): most dependents build soup withoutoutbound. Gate a domain helper on the feature its callers compile under:portsfordomain/service,outboundfor the Postgres repository.
Gotchas
Project grouping uses the system Project property (initiative membership), including canonical TASK assignments, rather than legacy folder
project_id.SoupService::group_labelsnames the groups a grouping field leaves unlabeled (today, project groups, from the initiative names the caller can read); REST group meta and GraphQL bins carry the result asGroupLabels, and a key it leaves out keeps itself as its label.crm_association.rsresolves CRM scope before filtering, sorting, and paging. Activity uses property edges rather than references and matches email within the caller's mailbox. Keep browser requests bounded; never expand every CRM record into client-side filters or raise ingress limits to fit them.A
crm_domains(ecd) scope that email rejects as not found or not permitted (archived company, email sync off) leaves email threads out of the page instead of failing it;handle_email_requestmatches those two errors only. Email's precheck is the only gate on the team-wide widening, so never drop just the failing domains and keep the scope.Agent sessions named by id (an
Idliteral incursor_soup, which GraphQLagentSessionand its log use, andby_ids) also accept a grant on the document or call whose live thread the session runs in, matchingentity_access. Broad lists use session grants only, and leave outhiddensessions (thread-started, DM contexts) except for their followers (agent_session_follower, people who prompted the session from the session itself), who may also read them through the thread's parent. The notified and work feeds share that rule throughnotified::agent_session_gate.expanded/vsunexpanded/queries exist because hydrating full entities is expensive — don't widenunexpandedto return hydrated data; add a newexpandedpath instead.Extend the existing dynamic query for property sorts and date-range groups; do not rebuild its source UNION in a candidate pipeline.
Keep property definition IDs unchecked like existing filters/grouping; enforce item visibility through the existing source access gates.
Order property pages by typed key with explicit ASC/DESC and NULLS LAST, then update timestamp and descending ID; keep timestamp SQL unchanged.
Validate caller DATE ranges in REST/GraphQL input conversion; render typed UTC bounds with
to_rfc3339()and map numeric range indices to caller keys in Rust.Preserve projection-backed hydration, the UUID-keyed item/fact pools, and per-group total/has-more metadata through the shared grouped response.
Flat property sorts omit calendar events, email, channels, calls, companies, and agent sessions; grouped reads retain calendar events and timestamp queries retain their existing sources.
Preserve deployed timestamp cursor encodings; client query keys restart pages when ranges or selections change.
Keep flat timestamp top clauses lightweight; bind the caller's source ids, then document-ID arrays, after the sort's fixed parameters and skip subtype namespace joins when the filter proves TASK-only unless other predicates need them.
The dynamic query's document, chat, and project arms match
entity_access.source_id = ANY(<bound source ids>), not a join to theuser_source_idsCTE: Postgres costs a CTE join at the average source's grant count and probed the item table once per grant for callers holding hundreds of thousands. The query runs in a transaction that raisesparallel_setup_cost, because parallel plans made callers with a few thousand grants twice as slow.Read grouping/sort definition types through uncorrelated scalar lookups; join grouping options once outside the value expansion and preserve missing-option groups.
Build option-sort order maps with an uncorrelated
jsonb_object_agglookup using the sort definition bind; ignore unknown option IDs and keep unknown-only values missing.Materialize grouped
TopItemsonly for property sorts so the sort key is computed before multi-value group expansion; leave timestamp CTEs inlineable.candidate_gates.rsencodes access-gate SQL shared across repo queries; changing it changes what every entity kind can return.Keep the notified feed's notification lookup behind a lateral
OFFSET 0fence so the caller'suser_notificationrows drive the join; aggregate the latest notification before applying the cursor or notification-state gates.A notification-state conjunct drives a listing from the caller's notifications (
strip_notification_conjunction, which folds(unseen OR seen) AND unseentounseen): the dynamic query joinsNotificationItems, the calendar leg semi-joinsnotified_event_ids_sqloveridx_user_notification_viewer_entity. The per-rowEXISTSfold stays only for trees the stripper rejects; on the calendar leg it seq-scanned every event in the table.Calendar access is
owner_id = $1 OR source_link_id = ANY(ARRAY(...)), not anEXISTSovermacro_user_links, so the planner can index both sides of the OR.Queries over a known id list (
*/by_ids.rs, hydration) and the initiative listing (initiative_access_clause) check access per row withEXISTSonentity_access, matching sources with= ANY(ARRAY(SELECT ...))so they are collected once rather than semi-joined per row. Building the caller's whole accessible set first scansentity_accessonce per source, so cost grows with the table and the caller's channels, not the page. A listingEXISTSalso needsPER_CANDIDATE, or Postgres hashes it back into the whole set.
Related
- Skill:
database-core,cloud-storage-hexagonal-architecture· Rules:.agents/rules/rust/db.md,.agents/rules/rust/architecture.md