Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.
Database backend
Databases contain tables of typed rows. Cells use the properties system's
DATABASE_ROW discriminator, and column placements bind to existing property
definitions and value types. Rows of an app database participate as entities;
rows of core storage alone do not acquire app access. There is one cell store.
Storage and app entities
Each databases row identifies reusable storage. Tables, column definitions, views,
saved queries, rows and their cells belong to that resource. A resource has no
app name, owner, or trash state and can exist without any Macro entity.
An optional database_entities row holds the app metadata: name, user_id (the owner),
created_at, updated_at, and trashed_at. Its database_id is both its primary
key and a foreign key to databases.id. The app uses that shared identity for
routes, grants, references and API responses. Database remains the app-facing
Rust model. Soup discovery and database/row access require this entity; merely
creating core storage does not make it a shareable Macro database.
This is the first use of the plural <kind>_entities convention for databases.
New app entities should follow it, keeping reusable technology separate from
app ownership and lifecycle. Existing unrelated entities are not renamed.
Collaboration surfaces are a similar reusable resource; this change does not
turn them into entities or change their parent-based access model.
DatabaseStorage exposes allocation, validated storage batches, and deletion
without app metadata. The CRM pipeline domain holds its own reference to core
storage and authorizes access through its own service. Its composition root
supplies the storage adapter. The port is trusted persistence, not a new public
API: consumers validate their own commands and authorize their own resources.
The Macro database service creates its entity and owner grant atomically through
its repository and uses CellStore to hold the entity's lifecycle lock during writes.
Both paths use the same transaction engine, table versions, cells and journal.
An app entity's foreign key prevents deleting its storage through the core port.
DatabaseStorageService reuses the typed operation planner for hosts that accept
DatabaseOp batches. It reads schema and paginated cells without app metadata,
then writes through DatabaseStorage::apply_storage_writes; it does not mint
app access receipts or publish app-entity events. The host holds its lifecycle
guard while it calls the service. External property bindings and relations are
refused unless a future host-specific capability authorizes them.
DatabaseStorageProvisioner creates an initial schema in its host's transaction.
CRM commits pipeline metadata, grants, and storage together through this port,
without creating a database_entities row. The one-time legacy Sales copy runs
through the normal SQLx migration runner.
The migration is a destructive cutover: it discards existing database content,
grants, saved queries, journal entries, and starter markers, then removes app
metadata from databases. Deploy it with the updated service; old binaries are
not supported. Rollback also discards database content.
Deleting an app entity (including through owner deletion) cleans up its owned storage, journal, and grants. Storage deletion cascades to tables, definitions, and cells. The core storage adapter purges its journal in the same transaction.
Reads and writes
Rows are read as Soup items, scoped to the databases the caller can reach as
entity_accessanswers it.Every write to a database's schema or data is one batch of typed ops (
models_databases::DatabaseOp,POST /databases/{id}/ops,DatabasesService::apply_ops), grouped by the resource they change. Each op is{kind, <ids>, change: {kind, ...}}:Op kindNames change.kindtabletablecreate,rename,delete,reorder_columns,reorder_viewscolumntable,columncreate,rename,change_type,delete,add_options,update_option,delete_optionrowstableinsert,update,deleteviewtable,viewcreate,update,delete,move_cardreorder_tables(every table, as order)none Ops are checked against the receipt's database and apply in order in one transaction, all or nothing; any refusal writes nothing. Lists are ordered by fractional keys (
models_databases::position), storedCOLLATE "C".Each op answers one
models_databases::OpResult, in the order sent, grouped the same way: the result's outerkindis its op's, it echoes the op's ids with the table's version after the commit (tableVersion, left out when the table was deleted), and itschangesays what happened, in the past tense:Result kindchange.kindtablecreated,renamed,deleted,columns_reordered,views_reordered(with the views'positions)columncreated,renamed,type_changed,deleted,options_added(with the idsadded),option_updated,option_deletedrowsinserted(with the newrows),updatedanddeleted(with how many wereaffected)viewcreatedandupdated(with the storedview),deleted,card_moved(with the writtenpositions)reorder_tablesnone; tableslists every table with its versionResults are grouped like the ops because a caller, and the coming change journal with undo, pairs each result with its op by index and kind, with no second lookup table from op kinds to result kinds. The flat result list had already started grouping ad hoc: one
rows_writtenanswered all three row ops, oneoption_changedanswered both option edits, and oneview_writtenanswered both view writes.Later ops of a batch see what earlier ops did. New tables, columns, options and views carry ids the client mints (UUIDv7), so a later op can name a table, column, option or view the same batch created: create a table, give it columns, then insert rows into them, in one request. An id that already names something, or is minted twice in one batch, refuses the batch with a 400 whose
takenis{kind: "table" | "column" | "option" | "view", id}; a retry of a committed batch lands there instead of writing twice. Rows keep server-minted ids, answered in theinsertedresult.The optional request-level
baseVersionsnames tables and the version the caller read; a table at another version answers 409 and writes nothing. A column type change follows one cast rule for the type menu, the agent tool andALTER COLUMN: it converts every value or refuses with the count and a few quoted misfits. Data is only destroyed by an explicit delete of a cell, row, column, table or option; an unknown option label is refused, never created. To keep the original, the conversion read (POST /databases/{id}/tables/{table_id}/columns/{column_id}/conversion,DatabasesService::column_conversion) answers the values that convert, the option labels they need and how many do not; the client then sends one batch: a columncreateafter the original, then a rowsupdatewith those values, at the conversion's table version.Successful commits publish their versions and change notifications.
A duplicate (
POST /databases/{id}/duplicate,DatabaseDuplicateService::duplicate_database, and theDuplicateEntitycapability) needs view access. It reads the source whole, gives every table, column, database-owned definition and option, row and view a new id, and writes the copy as one batch throughDatabaseDuplicateRepo, in the same transaction that creates the database and its owner's grant. Relations into another database stay when the duplicator can see it and are dropped, with their view references, when they cannot. A column bound to a property shared beyond the database stays bound to it when the duplicator may bind it, and otherwise gets a property of the copy's own with the same options and values. Relation cells naming rows that are gone are left out, and formula values are computed again in the copy from its own columns. Grants, the team share, forms, webhooks, saved queries, column protections and the journal stay behind; the copy's history starts empty.
SQL lives outside this crate. The browser compiles statements with the
database_sql engine and posts the ops it emits; agents run the same engine
through the databases_sql adapter, which reads through Soup and writes
through apply_ops.
Review map
| Concern | Implementation |
|---|---|
| Domain contracts and orchestration | src/domain/models.rs, ports.rs, service.rs |
| Catalog entries | src/domain/catalog.rs |
| Typed ops | src/domain/service/ops.rs, models_databases |
| Rows, cells, imports, and column definitions | src/outbound/pg_databases_repo.rs, pg_cell_store.rs, pg_definition_store.rs |
| Table, column and cell ops | src/domain/service/ops/tables.rs, columns.rs, cells.rs, src/outbound/pg_databases_repo/schema.rs |
| Column casts and inference | src/domain/service/casts.rs, column_types.rs, infer_column_type.rs |
| Typed views and a board's card places | models_databases::views, src/domain/service/ops/views.rs, src/outbound/pg_databases_repo/views.rs |
| Saved queries, sharing, imports, starter data | Corresponding modules under src/domain/ |
| Duplicates | src/domain/duplicate.rs, src/domain/service/duplicate.rs, src/outbound/pg_cell_store/duplicate.rs |
| HTTP transport | src/inbound/axum_router.rs, starter_router.rs |
| Service construction and notifications | src/outbound/build.rs, gateway_event_publisher.rs |
Authorization and business policy live in domain services. HTTP adapters obtain typed access receipts and pass requests inward. Persistence adapters implement domain ports, including transaction and locking requirements.
The document storage service mounts the routes. Writes inside a database go
through POST /databases/{id}/ops; the other routes are the database list and
detail, create, duplicate, starter, CSV import, a board's card positions, a column's casts
and type inference, awareness, permissions and saved queries.
Validation
Run from the repository root inside Nix, with a migrated local PostgreSQL database
and SQLX_OFFLINE unset:
cargo test -p databases --features postgres,inbound,ai_tools,gateway,entity_mutation
The suite covers permission scoping, typed round trips, relations, safe casts, rollback, stale/concurrent writes, sharing, typed views and card moves, and retry-safe import/starter provisioning. SQLx tests create isolated databases using the repository migrator.
Required columns
A trusted host can create a column with Column.nullable = false through
DatabaseStorage::apply_storage_writes. Existing columns and ordinary user-created
columns default to nullable. This does not add a UI toggle for requiredness.
The planner rejects omitted required cells on insert and explicit clears on edit. Both write ports also check affected required cells in the final transaction state under table locks, including schema changes and undo. A batch can add a required column and populate existing rows atomically. Empty selections, entity-reference lists and link lists are absent; empty text, zero and false are present. Undo restores the column setting alongside its cells.
Enforcement belongs to the Rust database engine. Direct SQL and writes through unrelated property APIs bypass it; the migration only adds the nullable flag.
Host-owned storage reads accept the common ViewQuery and retained row IDs.
The storage service reads bounded batches, evaluates the shared query compiler
and fold over the complete table, then returns pages with table versions. This
preserves filtering, option ordering, row positions and sorting across pages.
The current implementation scans the table for each page; indexed query pushdown
is a future optimization, not a different query language for each host.